Chapter 5: Writing a Benchmark#

How to write a benchmark.

This final demonstrates how to write benchmarks using Qt Test.

Writing a Benchmark#

To create a benchmark we extend a test function with a QBENCHMARK macro. A benchmark test function will then typically consist of setup code and a QBENCHMARK macro that contains the code to be measured. This test function benchmarks localeAwareCompare() .

def simple(self):

    str1 = "This is a test string"
    str2 = "This is a test string"
    QCOMPARE(str1.localeAwareCompare(str2), 0)

Setup can be done at the beginning of the function. At this point, the clock is not running. The code inside the QBENCHMARK macro will be measured, and possibly repeated several times in order to get an accurate measurement.

Several back-ends are available and can be selected on the command line.

Data Functions#

Data functions are useful for creating benchmarks that compare multiple data inputs, for example locale aware compare against standard compare.

def multiple_data(self):

    QTest.newRow("locale aware compare") << True
    QTest.newRow("standard compare") << False

The test function then uses the data to determine what to benchmark.

def multiple(self):

    QFETCH(bool, useLocaleCompare)
    str1 = "This is a test string"
    str2 = "This is a test string"
    result = int()
    if useLocaleCompare:
        QBENCHMARK {
            result = str1.localeAwareCompare(str2)

        QBENCHMARK {
            result = (str1 == str2)


The “if (useLocaleCompare)” switch is placed outside the QBENCHMARK macro to avoid measuring its overhead. Each benchmark test function can have one active QBENCHMARK macro.

External Tools#

Tools for handling and visualizing test data are available as part of the qtestlib-tools project. These include a tool for comparing performance data obtained from test runs and a utility to generate Web-based graphs of performance data.

See the qtestlib-tools announcement for more information on these tools and a simple graphing example.

Building the Executable#

You can build the test case executable using CMake or qmake.

Building with CMake#

Configure your build settings in your CMakeLists.txt file:

<Code snippet "/data/snapshot-qt5full-6.3/qt5/qtbase/tutorial5/CMakeLists.txt" not found>

Next, from the command line, run either cmake or use the qt-cmake convenience script located in Qt-prefix/<version>/<platform>/bin/qt-cmake:

<Qt-prefix>/<version>/<platform>/bin/qt-cmake <source-dir> <build-dir> -G Ninja

Then, run your preferred generator tool to build the executable. Here, we’re using Ninja:


Building with qmake#

Configure your build settings in your .pro file:

<Code snippet "/data/snapshot-qt5full-6.3/qt5/qtbase/tutorial5/" not found>

Next, run qmake, and, finally, run make to build your executable:


Running the Executable#

Running the resulting executable should give you the following output:

********* Start testing of TestBenchmark *********
Config: Using QtTest library %VERSION%, Qt %VERSION%
PASS   : TestBenchmark::initTestCase()
PASS   : TestBenchmark::simple()
RESULT : TestBenchmark::simple():
     0.00030 msecs per iteration (total: 79, iterations: 262144)
PASS   : TestBenchmark::multiple(locale aware compare)
RESULT : TestBenchmark::multiple():"locale aware compare":
     0.00029 msecs per iteration (total: 78, iterations: 262144)
PASS   : TestBenchmark::series(locale aware compare--8001)
RESULT : TestBenchmark::series():"locale aware compare--8001":
     0.039 msecs per iteration (total: 81, iterations: 2048)
Totals: 15 passed, 0 failed, 0 skipped, 0 blacklisted, 3971ms
********* Finished testing of TestBenchmark *********

Example project @