Adding Tests

There are two main ways of testing your plugin code:

• Plugin Tests
• Auto Tests

Both have their specific use cases and setup, which is described in the following sections.

Setting up CMake

Before adding tests, prepare your build files. They need to look for the QtTest dependency and have a CMake option for building your plugin with tests:

# Add a CMake option that enables building your plugin with tests.
# You don't want your released plugin binaries to contain tests,
# so make that default to 'NO'.
# Enable tests by passing -DWITH_TESTS=ON to CMake.
option(WITH_TESTS "Builds with tests" NO)

if(WITH_TESTS)
  # Look for QtTest
  find_package(${QtX} REQUIRED COMPONENTS Test)
  # Tell CMake functions like add_qtc_plugin about the QtTest component.
  set(IMPLICIT_DEPENDS Qt::Test)

  # Enable ctest for auto tests.
  enable_testing()
endif()

Plugin Tests

Plugin tests are deeply integrated into your plugin and its interaction with Qt Creator. To add a test for something that requires the infrastructure of Qt Creator or your plugin to be set up, write a plugin test.

Plugin tests are executed by starting Qt Creator with the -test <pluginname> command-line argument. Qt Creator then fully loads your plugin and all the plugins that it depends on, going through the normal Plugin Life Cycle. After your plugin and all dependencies are fully initialized, your tests are executed. Afterwards, Qt Creator automatically closes. Therefore, your plugin tests have access to all exported functionality of all Qt Creator plugins that your plugin depends on, like Core::ICore. Use QtTest's normal test macros, like QVERIFY or QCOMPARE to report your test's success or failure.

To add plugin tests, add a QObject based class with private slots for your tests, and register it with ExtensionSystem::IPlugin::addTest() in your plugin's ExtensionSystem::IPlugin::initialize() method. Guard all test related code with a check for WITH_TESTS, to avoid shipping a binary release of your plugin with test functions.

Include QtTest:

#ifdef WITH_TESTS
#include <QtTest>
#endif

Then implement the test functions:

#ifdef WITH_TESTS
class MyPluginTests : public QObject
{
    Q_OBJECT

private slots:
    void testMyTest()
    {
        // a failing test
        QVERIFY(false);
    }
};
#endif

Register your test in ExtensionSystem::IPlugin::initialize():

#ifdef WITH_TESTS
    addTest<MyPluginTests>();
#endif

If you declared the test object in the source file, like in this example, also include the .moc file that is required for Qt's meta object compiler:

#ifdef WITH_TESTS
#include "example.moc"
#endif

Auto Tests

To add a test that does not depend on a running Qt Creator infrastructure, use an auto test that lives independent of your plugin interface. Parsers are a common example, but you can test many things in this way if they have been written in a modular way.

Even though your test does not live in your plugin interface, like with plugin tests, you can still link the test to libraries and also your plugin library itself, to avoid code duplication or duplicate compilation of code.

In principle you can use any auto test framework, but QtTest is a simple one that integrates well with Qt, and is also used for the Plugin Tests.

To add your test, add the test's C++ file, and use add_qtc_test in your CMake file to add the test target. If your test uses your plugin library, add it as a dependency with DEPENDS.

In the following example, the plugin exports a function addOne:

#pragma once

#include "example_global.h"

namespace Example {

int EXAMPLE_EXPORT addOne(int i);

} // namespace Example

And implements it in a source file:

int addOne(int i)
{
    return i; // that is wrong!
}

The test is linked against the plugin library target with DEPENDS:

# conditionally add auto tests
if(WITH_TESTS)
  add_qtc_test(tst_mytest
    SOURCES tst_mytest.cpp
    DEPENDS Example
  )
endif()

The QtTest based test then includes the header from the plugin and tests the function:

#include "examplefunctions.h"

#include <QtTest>

class tst_MyTest : public QObject
{
    Q_OBJECT

private slots:
    void mytest();
};

void tst_MyTest::mytest()
{
    // a failing test
    QCOMPARE(Example::addOne(1), 2);
}

QTEST_GUILESS_MAIN(tst_MyTest)

#include "tst_mytest.moc"