Why unit tests are not always good

17

Unit tests are good to detect most bugs in your code but not all bugs. When you are writing standard unit tests for a class you are doing the following

  • Create a fresh class instance (ex using Setup method in DUnit framework);
  • Run a code under test (usually a single call of a single method) on the instance;
  • Free the instance (ex using TearDown method in DUnit framework).

And this is how unit tests should be written; if your test detects a bug you immediately know the bug’s origin.

The problem with the above scenario is that it is ideal to hide some badly reproducible bugs such as access violation (AV) bugs. To detect such a bad bug with good probability you need something different, probably to do multiple calls of a method on the same instance, or to call different methods in the same test, and this approach is quite opposite to the idea of unit testing.

Introduction to unit testing with Lazarus

5

1. Lazarus 1.0 comes with built-in unit testing framework called FPCUnit. FPCUnit is another Pascal clone of Java JUnit framework, like DUnit framework supplied with Delphi, but different from DUnit in some details.
If you are absolutely new to unit testing (or to Lazarus, like me) create your first unit test project by running the wizard. Select File->New… from IDE menu, choose FPCUnit Test Application and click ‘OK’:

Check two checkboxes and click ‘OK’ in the next dialog:

The wizard is very primitive, but you need not anything more. You can save the generated unit test project as a template and never run the wizard again.

2. Now time add a unit under test to the project. Most usual unit under test is a class, but it also can be a record with methods, or a pascal unit with flat procedures in interface section. For demonstration purposes I have written TCalc class implementing a simple calculator to be a unit under test:

unit Calcs;

{$mode objfpc}{$H+}

interface

uses
  Classes, SysUtils;

type
  TCalc = class
  private
    FAccumulator: Integer;
  public
    procedure Clear;
    procedure Add(Value: Integer);
    procedure Sub(Value: Integer);
    property Accumulator: Integer read FAccumulator;
  end;

implementation

{ TCalc }

procedure TCalc.Clear;
begin
  FAccumulator:= 0;
end;

procedure TCalc.Add(Value: Integer);
begin
  FAccumulator:= FAccumulator + Value;
end;

procedure TCalc.Sub(Value: Integer);
begin
  FAccumulator:= FAccumulator - Value;
end;

end.

3. To implement a testing of a unit we create a test case. Our generated unit test project already contains one test case – that is TTestCase1 class. Our unit under test has 3 functions to be tested: the methods TCalc.Clear, TCalc.Add and TCalc.Sub. Edit testcase1.pas unit as follows:

unit TestCase1;

{$mode objfpc}{$H+}

interface

uses
  Classes, SysUtils, fpcunit, testutils, testregistry, Calcs;

type
  TTestCase1= class(TTestCase)
  private
    FCalc: TCalc;
  protected
    procedure SetUp; override;
    procedure TearDown; override;
  published
    procedure Clear;
    procedure Add;
    procedure Sub;
  end;

implementation

procedure TTestCase1.SetUp;
begin
  FCalc:= TCalc.Create;
end;

procedure TTestCase1.TearDown;
begin
  FCalc.Free;
end;

procedure TTestCase1.Clear;
begin
  FCalc.Clear;
  AssertEquals(FCalc.Accumulator, 0);
end;

procedure TTestCase1.Add;
begin
  FCalc.Add(42);
  AssertEquals(FCalc.Accumulator, 42);
end;

procedure TTestCase1.Sub;
begin
  FCalc.Sub(42);
  AssertEquals(FCalc.Accumulator, -42);
end;

initialization
  RegisterTest(TTestCase1);
end.

4. Our first unit test project is ready now. Run it and see the result:

5. We created a published method of a test case for every tested function of a unit under test. Unit testing framework includes published methods of test case class into a test runner application. Notice that Setup and TearDown methods of a test case are called every time a published method of a test case is called. When I started to use unit testing I thought that Setup (TearDown) is called when test case class is created (destroyed) – that is not true, they are called ‘per function’.

6. You can add more units under test to a unit test project. If you have several units under test (and correspondent test cases) in a project you can have a separate test register unit. Remove the initialization section from the test case unit, add a second unit testcase2.pas with TTestCase2 test case class (you can make a replica of the first) to the project, and create a new RegTests.pas unit in to register test cases:

unit RegTests;

{$mode objfpc}{$H+}

interface

uses
  Classes, SysUtils, testregistry, testcase1, testcase2;

implementation

initialization
  RegisterTest(TTestCase1);
  RegisterTest(TTestCase2);
end.

Now we have a unit test project with 2 test cases:

7. If your unit test project grows one day you will want to have a hierarchical structure of test cases instead of a flat one. You can create a test hierarchy using a different RegisterTest overload. Create an additional hierarchy levels by updating RegTests.pas unit as follows:

unit RegTests;

{$mode objfpc}{$H+}

interface

uses
  Classes, SysUtils, testregistry, testcase1, testcase2;

implementation

initialization
  RegisterTest('Level1.Level2', TTestCase1);
  RegisterTest('Level1.Level2', TTestCase2);
end.

8. Our oversimplified test case class contains all testing code inside the published methods. More realistic published methods are wrappers for other functions which perform actual testing. You may be tempted to add private methods and fields to test case classes; it is OK if your test case class is simple, otherwise you will soon find your test case class cluttered with a mess of methods corresponding to different tests (published methods). My own experience lead me to creating (if needed) a separate helper class for every test. These helper classes inherit from a common base class which provides access to TTestCase functions:

type
  TTestHelper = class
  private
    FTestCase: TTestCase;
  public
    constructor Create(ATestCase: TTestCase);
    property TestCase: TTestCase read FTestCase;
  end;

implementation

constructor TTestHelper.Create(ATestCase: TTestCase);
begin
  FTestCase:= ATestCase;
end;

Happy unit testing with FPCUnit and Lazarus!

Anonymous methods in unit testing

0

Anonymous methods (aka Closures) are rare visitors in Delphi code. I have upgraded to Delphi 2009 a year before and never used the anonimous methods. But last week it finally happened.
I was writing the unit tests for fast fourier transform code. The idea of some tests was to compare the results of fast fourier trasform and discrete fourier transform of arbitrary function, and it appeared that the anonymous method is the best way to pass the function to the test.

type
  TRealFunc = reference to function(N: Integer): Extended;

type
  TTestFFT = class(TTestCase)
  private
    procedure TestRealFunc(Func: TRealFunc; Count: Integer);
  published
    procedure TestRealFFT00;
    ...
  end;

All checks are done within TestRealFunc procedure. It receives a function to transform as an argument, makes fourier transform of the function using two different methods and compares the results:

procedure TTestFFT.TestRealFunc(Func: TRealFunc; Count: Integer);
var
  FFTData: array of Extended;
  DFTData: array of TksComplex;
  M, N: Integer;
  Arg: TksComplex;

begin
  SetLength(FFTData, Count);
  SetLength(DFTData, Count);
  for M:= 0 to Count - 1 do begin
    FFTData[M]:= Func(M);

// discrete Fourier transform;
    DFTData[M]:= 0;
    for N:= 0 to Count - 1 do begin
      Arg.Re:= 0;
      Arg.Im:= (2 * Pi * N * M) / Count;
      DFTData[M]:= DFTData[M] + Func(N) * TksComplex.Exp(Arg);
    end;
  end;

// fast Fourier transform
  RealFFT(@FFTData[0], Count, False);

  CheckEquals(FFTData[0], DFTData[0].Re, 1E-15);
  CheckEquals(FFTData[1], DFTData[Count shr 1 - 1].Re, 1E-15);
  CheckEquals(0, DFTData[0].Im, 1E-15);
  CheckEquals(0, DFTData[Count shr 1 - 1].Im, 1E-15);
  for N:= 1 to Count shr 1 - 1 do begin
    CheckEquals(FFTData[2 * N], DFTData[N].Re, 1E-15);
    CheckEquals(FFTData[2 * N + 1], DFTData[N].Im, 1E-15);
    CheckEquals(FFTData[2 * N], DFTData[Count - N].Re, 1E-15);
    CheckEquals(FFTData[2 * N + 1], - DFTData[Count - N].Im, 1E-15);
  end;
end;

The test procedures themselves just calls the TestRealFunc with different arguments. For example,

procedure TTestFFT.TestRealFFT00;
begin
  TestRealFunc(
    function(N: Integer): Extended
    begin
      Result:= Sin(pi / 4 * N) + 0.5 * Sin(pi / 2 * N + 3 * pi / 4);
    end,
    8
  );
end;

Though the syntax looks cumbersome, the solution based on anonimous methods is very logical.