Support for Build Automation Systems

Make and its variants

Coco can also generate code coverage information for a project without the need to modify it. The principle is to prepend to the PATH variable the path of the CoverageScanner compiler wrappers and to set the instrumentation parameters with the COVERAGESCANNER_ARGS environment variable. To activate the instrumentation, --cs-on must be present in COVERAGESCANNER_ARGS. If this is not the case, CoverageScanner is completely deactivated.

GNU Make

On a UNIX^® system, proceed as follows to instrument a project which can be generated using GNU Make:

export PATH=/opt/SquishCoco/wrapper/bin:$PATH
export COVERAGESCANNER_ARGS=--cs-on
make clean
make

For macOS, replace the first line with:

export PATH=/opt/SquishCoco/wrapper:$PATH

Microsoft NMake

To instrument a project that is generated with NMake:

set PATH=%SQUISHCOCO%\visualstudio;%PATH%
set COVERAGESCANNER_ARGS=--cs-on
nmake clean
nmake

Microsoft MSBuild

To instrument a project that is generated with MSBuild:

set PATH=%SQUISHCOCO%\visualstudio;%PATH%
set COVERAGESCANNER_ARGS=--cs-on
msbuild /p:UseEnv=true myproject.sln /t:ReBuild

CMake

CMake is a platform independent build tool from Kitware.

When Coco is used with CMake, the changes are partially dependent on the toolchain that is used for compilation. We will now first describe the addition of a new build type, which is independent from the toolchain, and then the additional changes for Microsoft^® Visual Studio^® and GNU GCC.

Adding new build type for instrumented compilation

The first step is independent of the toolchain that is used. Its purpose is to declare the CMake variables that are used to specify the instrumented compilation. In CMake this is done by declaring a build type, which we will here call COVERAGE.

To do this, add to the to CMakeLists.txt file the following lines. The variable COVERAGE_FLAGS in the first line specifies the CoverageScanner command line options. Change its value to fit your needs. Only --cs-on must always be present.

SET(COVERAGE_FLAGS "--cs-on --cs-count")
SET(CMAKE_CXX_FLAGS_COVERAGE
    "${CMAKE_CXX_FLAGS_RELEASE} ${COVERAGE_FLAGS}" CACHE STRING
    "Flags used by the C++ compiler during coverage builds."
    FORCE )
SET(CMAKE_C_FLAGS_COVERAGE
    "${CMAKE_C_FLAGS_RELEASE} ${COVERAGE_FLAGS}" CACHE STRING
    "Flags used by the C compiler during coverage builds."
    FORCE )
SET(CMAKE_EXE_LINKER_FLAGS_COVERAGE
    "${CMAKE_EXE_LINKER_FLAGS_RELEASE} ${COVERAGE_FLAGS}" CACHE STRING
    "Flags used for linking binaries during coverage builds."
    FORCE )
SET(CMAKE_SHARED_LINKER_FLAGS_COVERAGE
    "${CMAKE_SHARED_LINKER_FLAGS_RELEASE} ${COVERAGE_FLAGS}" CACHE STRING
    "Flags used by the shared libraries linker during coverage builds."
    FORCE )
SET( CMAKE_STATIC_LINKER_FLAGS_COVERAGE
    "${CMAKE_STATIC_LINKER_FLAGS_RELEASE} ${COVERAGE_FLAGS}" CACHE STRING
    "Flags used by the static libraries linker during coverage builds."
    FORCE )
MARK_AS_ADVANCED(
    CMAKE_CXX_FLAGS_COVERAGE
    CMAKE_C_FLAGS_COVERAGE
    CMAKE_EXE_LINKER_FLAGS_COVERAGE
    CMAKE_SHARED_LINKER_FLAGS_COVERAGE
    CMAKE_STATIC_LINKER_FLAGS_COVERAGE
    COMPILE_DEFINITIONS_COVERAGE
)

These commands take the compiler and linker flags of the Release build type and add to them the coverage flags. If you want to use instead the flags of another build type, replace the suffix _RELEASE in this code with the name of another build type, such as _DEBUG.

Microsoft Visual Studio

In Visual Studio, we need to make the new Coverage build type visible to the IDE. To do this, we add an item to the configurations list in in CMakeSettings.json that is based on your Release configuration.

{
"configurations": [
     "_comment" : "other configurations here... ",
      {
        "name": "CodeCoverage-x64-Release",
        "generator": "Ninja",
        "configurationType": "Release",
        "buildRoot": "${projectDir}\\out\\build\\${name}",
        "installRoot": "${projectDir}\\out\\install\\${name}",
        "cmakeCommandArgs": "-DCMAKE_TOOLCHAIN_FILE=cl.cmake -DCMAKE_BUILD_TYPE=COVERAGE",
        "buildCommandArgs": "",
        "ctestCommandArgs": "",
        "inheritEnvironments": [ "msvc_x64_x64" ],
        "variables": []
      }
   ]
}

The important detail here is the cmakeCommandArgs which tells cmake to use the cl.cmake toolchain file, and to use the COVERAGE build type. The other arguments/variables can be copied or customized from another working configuration.

Compilation with Microsoft NMake

In a project that is compiled with NMake:

1. Create a toolchain definition file cl.cmake which replaces the compiler and linker with their CoverageScanner wrappers.

   For example:

   # this one is important
   SET(CMAKE_SYSTEM_NAME Windows)
   
   # specify the cross compiler
   FILE(TO_CMAKE_PATH "$ENV{SQUISHCOCO}/visualstudio" SQUISHCOCO)
   SET(CMAKE_C_COMPILER ${SQUISHCOCO}/cl.exe
       CACHE FILEPATH "CoverageScanner wrapper" FORCE)
   SET(CMAKE_CXX_COMPILER ${SQUISHCOCO}/cl.exe
       CACHE FILEPATH "CoverageScanner wrapper" FORCE)
   SET(CMAKE_LINKER ${SQUISHCOCO}/link.exe
       CACHE FILEPATH "CoverageScanner wrapper" FORCE)

2. Create a Makefile project. Set the toolchain to the CoverageScanner wrapper and the build mode to COVERAGE.

   For example:

   cmake.exe -DCMAKE_TOOLCHAIN_FILE=cl.cmake -DCMAKE_BUILD_TYPE=COVERAGE \
             -G "NMake Makefiles" <i>path of cmake project</i>

3. Build the project with nmake.

Compilation with GNU GCC

The following must be done in a project that is compiled with gcc:

1. Create a toolchain definition file gcc.cmake which replaces the compiler and linker with their CoverageScanner wrappers.

   For example:

   find_program(CODE_COVERAGE_GCC gcc
       PATHS /opt/SquishCoco/wrapper/bin "$ENV{HOME}/SquishCoco/wrapper/bin"
       NO_DEFAULT_PATH )
   find_program(CODE_COVERAGE_GXX g++
       PATHS /opt/SquishCoco/wrapper/bin "$ENV{HOME}/SquishCoco/wrapper/bin"
       NO_DEFAULT_PATH )
   find_program(CODE_COVERAGE_AR ar
       PATHS /opt/SquishCoco/wrapper/bin "$ENV{HOME}/SquishCoco/wrapper/bin"
       NO_DEFAULT_PATH )
   
   # specify the cross compiler
   SET(CMAKE_C_COMPILER "${CODE_COVERAGE_GCC}"
       CACHE FILEPATH "CoverageScanner wrapper" FORCE)
   SET(CMAKE_CXX_COMPILER "${CODE_COVERAGE_GXX}"
       CACHE FILEPATH "CoverageScanner wrapper" FORCE)
   SET(CMAKE_LINKER "${CODE_COVERAGE_GXX}"
       CACHE FILEPATH "CoverageScanner wrapper" FORCE)
   SET(CMAKE_AR "${CODE_COVERAGE_AR}"
       CACHE FILEPATH "CoverageScanner wrapper" FORCE)

2. Create a Makefile project. Set the toolchain to the CoverageScanner wrapper and the build mode to COVERAGE.

   For example:

   cmake -DCMAKE_TOOLCHAIN_FILE=gcc.cmake -DCMAKE_BUILD_TYPE=COVERAGE \
         -G "Unix Makefiles" <i>path of cmake project</i>

3. Build the project with make.

Qt framework

qmake

qmake is a tool that is used by Qt to generate a makefile in a platform-independent way, using as specification a so-called project file. It is also used by Qt Creator.

By default, qmake chooses the programs that are used for compilation. We can use the Coco wrappers by setting some of qmake's variables to new values. This can be done by putting some declarations into the qmake project files. Since one needs to build the project with and without coverage, the new definitions must be put into a scope. A scope is a region in a qmake project file that can be activated on demand.

The following listing shows a template for such a scope, named CoverageScanner, which should be sufficient for most projects.

CodeCoverage {
  COVERAGE_OPTIONS =

  QMAKE_CFLAGS   += $$COVERAGE_OPTIONS
  QMAKE_CXXFLAGS += $$COVERAGE_OPTIONS
  QMAKE_LFLAGS   += $$COVERAGE_OPTIONS

  QMAKE_CC=cs$$QMAKE_CC
  QMAKE_CXX=cs$$QMAKE_CXX
  QMAKE_LINK=cs$$QMAKE_LINK
  QMAKE_LINK_SHLIB=cs$$QMAKE_LINK_SHLIB
  QMAKE_AR=cs$$QMAKE_AR
  QMAKE_LIB=cs$$QMAKE_LIB
}

Here we also set the variables QMAKE_LINK_SHLIB and QMAKE_AR, which contain the names of the command to link shared libraries and that to generate archives. Furthermore, you can use COVERAGE_OPTIONS to set coveragescanner command line options (see List of options) to customize the project. An empty value for COVERAGE_OPTIONS will also work and results in a default instrumentation.

In a small project, the CodeCoverage scope may then be copied to all profile files of the project, i.e. those that end in .pro. In fact, it is enough to insert them only into those files that actually compile code and not those that only include other files. If the project is larger, it has very often a file with common settings that is included by all profiles: This is then the most convenient place to insert the CodeCoverage scope only once.

The new coverage scope is by default inactive. To enable code coverage in a project you want to built, just add the name of the scope to the CONFIG variable when configuring it with qmake:

$ qmake CONFIG+=CodeCoverage

moc

The Meta-Object Compiler (moc) generates some methods for each class derived from QObject. For example, the translation function tr, the source code for all signals, the cast operator qt_cast, …In order to instrument the code using the Qt Framework and not moc-generated code, CoverageScanner provides the command line options --cs-qt3 for Qt3 and --cs-qt4 for Qt4 to Qt6. They are enabled by default.

In this case:

• Q_OBJECT macro is no longer instrumented.
• All signals are instrumented in order to track their emission.
• The glue code necessary for the signal/slot mechanism is not instrumented.
• The code of Q_FOREACH macro is not instrumented in Qt4.

qbs

To use CoverageScanner with the Qt Build Suite (qbs), set it up as a toolchain. You can then use the toolchain for all qbs projects.

To set up CoverageScanner as a toolchain, issue the following command:

qbs setup-toolchains --type gcc /opt/SquishCoco/bin/csgcc csgcc

For Unix-based operating systems, some additional configuration steps are necessary:

qbs config profiles.csgcc.cpp.archiverPath /opt/SquishCoco/bin/csar
qbs config profiles.csgcc.cpp.linkerName csg++
qbs config profiles.csgcc.cpp.nmPath /opt/SquishCoco/bin/csgcc-nm

The csgcc toolchain can then also be used as base profile for Qt projects:

qbs setup-qt /opt/Qt/bin/qmake qt-csgcc
qbs config profiles.qt-csgcc.baseProfile csgcc

SCons

To use Coco with SCons:

1. Prepend the path of CoverageScanner's wrapper (csgcc, cscl, and son on) to the PATH environment variable. The PATH environment variable should be set to be able to execute CoverageScanner wrappers.
2. Set CC, AR and LINK variables to the corresponding CoverageScanner wrapper. For example: when using Microsoft Visual Studio compiler, set CC to cscl.exe, LINK to cslink.exe, and AR to cslib.exe.

   Note: Do no use absolute file paths to the compiler wrapper since some versions of SCons do not properly handle spaces in file names.

3. Add code coverage settings to exclude from the instrumentation for example files to the variables CCFLAGS, ARFLAGS and LINKFLAGS.

Here is a code snippet which can be used for Visual Studio command line tools:

import os
from os.path import pathsep

env = Environment()

# Add the path of Squish Coco compiler wrapper
env[ 'ENV' ][ 'PATH' ] = os.environ[ 'SQUISHCOCO' ] + pathsep + env[ 'ENV' ][ 'PATH' ]
# TEMP variable need to be defined
env[ 'ENV' ][ 'TEMP' ] = os.environ[ 'TEMP' ]
env[ 'ENV' ][ 'INCLUDE' ] = os.environ[ 'INCLUDE' ]

# Set the compiler to Squish Coco wrappers
env[ 'CC' ]   = 'cs' + env[ 'CC' ] ;
env[ 'AR' ]   = 'cs' + env[ 'AR' ] ;
env[ 'LINK' ] = 'cs' + env[ 'LINK' ] ;

# Code coverage settings
coverageflags = [ '--cs-count' ]
env[ 'CCFLAGS' ]   = env[ 'CCFLAGS' ] + coverageflags ;
env[ 'ARFLAGS' ]   = env[ 'ARFLAGS' ] + coverageflags ;
env[ 'LINKFLAGS' ] = env[ 'LINKFLAGS' ] + coverageflags ;

To correctly set the build environment, you must start the build from the Visual Studio's developer prompt (available through the start menu) or the Coco's console provided by Build Environment Selection.