Test suites and Squish Coco

Execution comment, name and status

Test suites are able to provide to CoverageBrowser the name of the test, a comment in HTML format and also the status of their execution (passed or failed). Two methods are possible:

  • The functions __coveragescanner_testname(), __coveragescanner_add_html_comment() and __coveragescanner_teststate() can be used to specify the test name, its comment and its status. This is particularly useful for unit tests in which the test framework is compiled into the application to test (see __coveragescanner_testname()).
  • One can write these data directly into the .csexe file. (See below.)

The .csexe file is a simple text file on which is it possible to append additional lines to extend the information provided to CoverageBrowser.

To set the name of a test, simply add before executing it the following line: "*<name of the test>\n" The character "*" must be the first character of the line. The name of the test is placed directly after it and the line is terminated with a carriage return.

Note: It is also possible to begin the line with a paragraph sign ("§") instead of a "*". The "§" must be the in Latin-1 encoding and has the numerical value of 167. Since this may lead to internationalization problems, we strongly dicourage its use. It is only supported for compatibility with older versions of the program.

To set the status of a test, simply add after executing it the following line: "!<status>\n" The character "!" must be the first character of the line. The status of the test is placed directly after it and the line is terminated with a carriage return. The status can be one of the following strings:

  • "PASSED": to indicate that the test was successfully executed,
  • "FAILED": to indicate that the test was not successfully passed, and
  • "CHECK_MANUALLY": to indicate that it was not possible to determinate whether the test was successfully executed.

To append an execution comment, insert the contents of an HTML document just after the execution of the application.

Example: The following batch file execute the test First Test and set the status of the execution to CHECK_MANUALLY.

    echo *First Test >> myapp.csexe
    myapp
    echo "<HTML><BODY>Execution of myapp</BODY></HTML>" >> myapp.csexe
    echo !CHECK_MANUALLY >> myapp.csexe

There may be more than one HTML comment, and the comments and the status declaration may occur in any order, but they all must be appended to the .csexe file after the test has been executed.

Unit testing

CoverageBrowser imports execution results for all object files that are part of an application. This means that, if an unit test uses the same object files as the application, CoverageBrowser can import the execution results of the unit tests and merge them into the code coverage of the compiled application.

Example:

An application consists of three files:

  • app.cpp: This file contains the main() function of the application.
  • library.cpp: This file contains functions that are called by app.cpp.
  • testlibrary.cpp: This file contains the test code for the functions in library.cpp. It has its own main() function.

Under Microsoft® Windows, the application is compiled with the following commands:

    cscl app.cpp /Foapp.obj
    cscl library.cpp /Folibrary.obj
    cscl library.obj app.obj /Feapp.exe

The following commands compile the unit test program:

    cscl testlibrary.cpp /Fotestlibrary.obj
    cscl library.obj testlibrary.obj /Fetestlibrary.exe

To import the execution report of the unit tests into the instrumentation database of the main application, two methods can be used:

  • The execution of testlibrary.exe generates the execution report testlibrary.exe.csexe. This report is then loaded into the measurement database app.exe.csmes.
  • The execution of testlibrary.exe generates the execution report testlibrary.exe.csexe. This report is then loaded into the measurement databaseof the test code, testlibrary.exe.csmes. This can be done with the function "FileImport Unit Tests…" of the CoverageBrowser (see Importing Unit Tests).

In both cases, only the code coverage analysis of the file library.cpp is loaded and the execution report of the test code ignored.

© 2022 The Qt Company Ltd. Documentation contributions included herein are the copyrights of their respective owners. The documentation provided herein is licensed under the terms of the GNU Free Documentation License version 1.3 as published by the Free Software Foundation. Qt and respective logos are trademarks of The Qt Company Ltd. in Finland and/or other countries worldwide. All other trademarks are property of their respective owners.