Dabbling in BASM


TForge 0.67 adds SHA-256 algorithm BASM optimization (applies also to SHA-224 which is based on SHA-256). Now the library contains BASM-optimized implementations of MD5, SHA1, SHA224 and SHA256 algorithms for Win32 and Win64 platforms.

Here are some benchmarks; I measured the time required to hash 500 MB of data on my laptop (Win7 64-bits, Intel Core i3 CPU) in milliseconds. Both Pascal and BASM implementations use fully unrolled compression routines. I used default compiler settings.

32-bit compiler (Delphi XE):



64-bit compiler (FPC 2.6.2):



High Performance Hash Library


TForge 0.66 released.

What’s new:

1. More hash algorithms added:

  • SHA224
  • SHA384
  • SHA512

2. MD5 and SHA1 compression routines were rewritten in BASM for CPUX86-WIN32 and CPUX64-WIN64 platforms and times faster now compared to pure pascal code (about 4 times for 32-bit compiler (Delphi XE) and about 3 times for 64-bit compiler (FPC 2.6.2) on my system).

3. Memory allocation bug in HMAC implementation was fixed.

User-friendly Hash Library


TForge 0.65 released.

The release features a hash library with fluent coding support. While I was writing the library I was inspired by usability of the Python’s hashlib.

Current release supports:

  • Cryptographic hash algorithms:
    1. MD5
    2. SHA1
    3. SHA256
  • Non-cryptographic hash algorithms:
    1. CRC32
    2. Jenkins-One-At-Time
  • Hash-based MAC algorithm (HMAC)
  • Key derivation algorithms:
    1. PBKDF1
    2. PBKDF2

Let us consider a common problem: calculate MD5 and SHA1 digests of a file. The simplest way to do it is:

program HashFile;


  SysUtils, Classes, tfTypes, tfHashes;

procedure CalcHash(const FileName: string);
  Writeln('MD5:  ', THash.MD5.UpdateFile(FileName).Digest.ToHex);
  Writeln('SHA1: ', THash.SHA1.UpdateFile(FileName).Digest.ToHex);

    if ParamCount = 1 then begin
      Writeln('Usage: > HashFile filename');
    on E: Exception do
      Writeln(E.ClassName, ': ', E.Message);

The above application demonstrates the beauty of fluent coding. The code is compact and clear – you create an instance of a hash algorithm, feed a file data to it, generate resulting digest and convert it to hexadecimal; the instance is freed automatically, no need for explicit call of the Free method.

But the above code is not optimal – it reads a file twice. A better solutions involves some coding:

procedure CalcHash(const FileName: string);
  BufSize = 16 * 1024;

  MD5, SHA1: THash;
  Stream: TStream;
  Buffer: array[0 .. BufSize - 1] of Byte;
  N: Integer;

  MD5:= THash.MD5;
  SHA1:= THash.SHA1;
    Stream:= TFileStream.Create(FileName,
                                fmOpenRead or fmShareDenyWrite);
        N:= Stream.Read(Buffer, BufSize);
        if N <= 0 then Break
        else begin
          MD5.Update(Buffer, N);
          SHA1.Update(Buffer, N);
      until False;
    Writeln('MD5:  ', MD5.Digest.ToHex);
    Writeln('SHA1: ', SHA1.Digest.ToHex);

The code also demonstrate the use of Burn method; it is not needed here and could be safely removed with corresponding try/finally block but can be useful in other cases – it destroys all sensitive data in an instance. The use of Burn method is optional – it is called anyway when an instance is freed, but explicit call of the Burn method gives you full control over erasing the sensitive data.

The Free method does not free an instance; it only decrements the instance’s reference count, and since the compiler can create hidden references to the instance the moment when the reference count turns zero and the instance is freed is generally controlled by the compiler.

The use of non-cryptographic hash algorithms has one caveat – since they actually return an integer value the bytes of a digest are reversed. The idiomatic way to get the correct result is to cast the digest to integer type:

  Writeln('CRC32: ',
    IntToHex(LongWord(THash.CRC32.UpdateFile(FileName).Digest), 8));

or else you can reverse the digest’s bytes:

  Writeln('CRC32: ',

HMAC algorithm generates digest using a cryptographic hash algorithm and a secret key. Here is an example of calculating SHA1-HMAC digest of a file:

procedure SHA1_HMAC_File(const FileName: string;
                         const Key: ByteArray);
  Writeln('SHA1-HMAC: ',

    ByteArray.FromText('My Secret Key'));

Key derivation algorithms generate keys from user passwords by applying hash algorithms. PBKDF1 applies a cryptographic hash algorithm directly, PBKDF2 uses HMAC. Here are usage examples:

procedure DeriveKeys(const Password, Salt: ByteArray);
  Writeln('PBKDF1 Key: ',
    THash.SHA1.DeriveKey(Password, Salt,
                         10000,   // number of rounds
                         16       // key length in bytes
  Writeln('PBKDF2 Key: ',
    THMAC.SHA1.DeriveKey(Password, Salt,
                         10000,   // number of rounds
                         32       // key length in bytes

  DeriveKeys(ByteArray.FromText('User Password'),

Configuration and Installation

The release contains 2 runtime packages TForge and THashes. You should build them, first TForge, next THashes.

For Delphi users:

The packages are in Packages\DXE subfolder (Delphi XE only).

You should make the folder Source\Include available via project’s search path before you can build the packages. To do it open “Project Options” dialog for each package, set “Build Configuration” to “Base” and replace the path to TFL.inc by your path (depends on where you unpacked the downloaded archive):


Now use the project manager and build the packages in “Debug” and “Release” configurations:


Optionally, to enable Ctrl-click navigation in the editor, add paths to the source code units to the Browsing path:

To make the packages available to an application open the application’s “Project Options” dialog, set “Build Configuration” to “Base” and add the next path to search path:


For FPC/Lazarus users:

The packages are in Packages\FPC subfolder.

I don’t know much about Lazarus packages. Here are the package configuration options I used:



Byte Array redux


I am continuing to experiment with the coding patterns used in my numerics dll project and releasing TForge 0.60.

First of all TForge is a modern cryptographic library for Delphi and Lazarus/FPC (which I am currently working on) implemented as a set of runtime packages. The current release contains only one package (also named TForge); it is core package of the whole TForge project and is required by other packages (to be released later).

The purpose of the release is to introduce a new type – ByteArray; the type is an enhanced version of the standard RTL TBytes type. If you want to test the ByteArray you need to build tforge package; the release containes tforge packages for Delphi XE and Lazarus.

The release includes ByteArrayDemo console application which demonstrates functionality of the ByteArray type. For example, you can concatenate byte arrays:

  A1, A2: ByteArray;

  A1:= ByteArray(1);
  A2:= TBytes.Create(2, 3, 4);
  Writeln('A1 + A2 = ', (A1 + A2).ToString); // 1, 2, 3, 4

perform bitwise boolean operations (xor is most useful):

  A1, A2: ByteArray;

  A1:= ByteArray(1);
  A2:= TBytes.Create(2, 3, 4);
  Writeln('A1 xor A2 = ', (A1 xor A2).ToString); // 3 (= 1 xor 2, min array length used)

use fluent coding style (which appears to be very handy once one gets accustomed to it):

    ByteArray.FromText(' 123 ')).Reverse.ToText); // JIHGFED 321 CBA

and so on. It was fun to code ByteArray; I am using it more and more now as a TBytes replacement because of better usability.

Why PDWORD is not a pointer to DWORD


Once upon a time the Delphi team lead decided that Delphi should contain declarations of C++ types like unsigned long for the sake of better C++ compatibility. So he has written

  CppULongInt = LongWord;

No, that is not good thought the team lead. Delphi’s LongWord is a 32-bit type while the C++’s unsigned long is a platform-dependent type. So let us declare CppULongInt as a distinct type instead of an alias type:

  CppULongInt = type LongWord;

That is better. Now how about DWORD type? It is declared in Windows headers as

  typedef unsigned long       DWORD;

So for compatibility sake we will declare DWORD as

  DWORD = CppULongInt;

Great, thought the Delphi team lead. He call two juniors (let us call them DevA and DevB) and said:

- DevA, create a branch (BranchA) in the Delphi project, declare type DWORD = CppULongInt; and fix all bugs that may appear after;
DevB, create a branch (BranchB) in the Delphi project, declare type PDWORD = ^CppULongInt; and fix all bugs that may appear after;

After you both are ready we will merge the branches and declare PDWORD type as it should be:


In a due time the happy DevB came to the Delphi team lead and said: “I did everything as you said Sir!. Now type DWORD = CppULongInt;, and everything is fine!”.

But the second guy DevA was not happy. He said: “Sir, we have tons of code like that:

procedure Foo(var Value: DWord);
  Value:= 0;

  D: Longword;


If I declare

  DWORD = CppULongInt;

Then it does not compile. What should I do?”

-“Nothing”, said the Delphi team lead. “The BranchB will be the main trunk now”.

Disclaimer: the tale was written after reading this SO question

Why unit tests are not always good


Unit tests are good to detect most bugs in your code but not all bugs. When you are writing standard unit tests for a class you are doing the following

  • Create a fresh class instance (ex using Setup method in DUnit framework);
  • Run a code under test (usually a single call of a single method) on the instance;
  • Free the instance (ex using TearDown method in DUnit framework).

And this is how unit tests should be written; if your test detects a bug you immediately know the bug’s origin.

The problem with the above scenario is that it is ideal to hide some badly reproducible bugs such as access violation (AV) bugs. To detect such a bad bug with good probability you need something different, probably to do multiple calls of a method on the same instance, or to call different methods in the same test, and this approach is quite opposite to the idea of unit testing.

Numerics 0.57 released (Hashtables, Bugfix)


1. The main purpose of the release is to implement hash tables (aka associative arrays) with keys of BigCardinal or BigInteger type; such hash tables are important for cryptographic applications. The hash tables in Delphi are implemented by a generic TDictionary<TKey,TValue> class. The release includes a new unit GNumerics.pas with two helper generic classes TBigIntegerDictionary<TValue> and TBigCardinalDictionary<TValue> specializing the base generic dictionary class. For example to work with a hash table having BigInteger keys and string values you need something like this:

uses GNumerics;

  HashTable: TBigIntegerDictionary<string>;
  A: BigInteger;

// create hash table instance
  HashTable:= TBigIntegerDictionary<string>.Create;
  A:= BigInteger('1234567890987654321');
// fill hash table with data
    HashTable.Add(A * A, 'some string value');
// destroy hash table

2. Some bugs fixed; in particular a nasty access violation bug in BigInteger.ModPow function was fixed.

3. Minor changes in BigCardinal/BigInteger methods.

Link to the download page


Version 0.58 fixes the conversion bug from (U)Int64 to BigInteger/BigCardinal.