Code Excluded from Instrumentation cocoqmlscanner

How to handle Coveragescanner error messages and warnings

What does the warning "Instrumentation is different" mean?

Sometimes during compilation of a C/C++ program, Coco issues a warning of the form

Warning (Squish Coco): Compilation of 'insertargs/header.h': Instrumentation of source file 'insertargs/header.h' is different

followed by several more lines. What does this mean?

Explanation

In C and C++, the content of a source file does not completely determine to what code it compiles. This is because preprocessor symbols can be set at compile time and determine – usually via #ifdef statements – which source code gets actually translated.

This means that it is possible that a source file is compiled several times with different preprocessor settings. This may lead to different compiled code and, if Coco is involved, to different instrumentations of the same file. If Coco notices this, it issues the "Instrumentation is different" message.

The message is however only a warning. Coco treats the different versions of the file as different files and when they are displayed in CoverageBrowser or in a report, the versions are distinguished by numbers at the end. A file header.h could be displayed as header.h#1 and header.h#2 if there are two versions of it.

Example with multiple code paths for a single function

What exactly is going on, and how to understand the error message, are best seen from an example. First, we have a source file, header.h that can be compiled to different code depending on an external setting. It defines a function, letter(), the compilation of which depends on whether the preprocessor symbol A_H is defined or not:

#ifndef HEADER_H
#define HEADER_H

static inline char letter()
{
#ifdef A_H
    return 'a';
#else
    return 'b';
#endif
}

#endif

Then there are two files that use this header. One is a.cpp,

#include "a.h"
#include "header.h"
#include <iostream>

void a()
{
    std::cout << "Letter A: " << letter() << "\n";
}

the other one, b.cpp. (The header files, a.h and b.h, are here omitted.)

#include "b.h"
#include "header.h"
#include <iostream>

void b()
{
    std::cout << "Letter B: " << letter() << "\n";
}

Note that the .cpp files are almost the same, and by themselves, do not contain code that is influenced by the symbol A_H. However, both files include header.h which does. Therefore, the preprocessed versions of a.cpp and b.cpp – the versions that CoverageScanner sees and instruments – contain differently-preprocessed versions of header.h.

Finally we have a main.cpp, that uses them both.

#include "a.h"
#include "b.h"

int main()
{
    a();
    b();
    return 0;
}

a.cpp and b.cpp are effectively compiled with different flags, where the symbol A_H is defined for a.cpp but not for b.cpp, and we are also instrumenting with Coco, so we expect to see the above warning.

If this project was compiled with a Makefile, it might look like this:

main: main.cpp a.o b.o
        g++ -o main main.cpp a.o b.o

a.o: a.cpp a.h header.h
        g++ -c -o a.o a.cpp

b.o: b.cpp b.h header.h
        g++ -c -o b.o b.cpp

clean:
        rm *.o *.csexe *.csmes

If we build and run this program, the output looks like this:

Letter A: a
Letter B: b

This program demonstrates a situation where you end up with 2 different code paths when calling the same function, letter(), from different code modules.

We build this program with code coverage, using an instrumented script that resembles this:

#!/bin/bash

# this script assumes the COCO wrappers are already in your PATH.

export COVERAGESCANNER_ARGS='--cs-on'
export COVERAGESCANNER_ARGS+=' --cs-mcdc'
export COVERAGESCANNER_ARGS+=' --cs-mcc'
"$@"

When such a project is compiled with instrumentation enabled, the warning from above appears. Its full text is:

g++ -c -o a.o a.cpp
g++ -c -o b.o b.cpp
g++ -o main main.cpp a.o b.o
Warning (Squish Coco): Compilation of '/samples/insertargs/header.h': Instrumentation of source file '/samples/insertargs/header.h' is different
Warning (Squish Coco):   Source line 7 of file '/samples/insertargs/header.h' is differently instrumented in the database 'main.csmes' and 'b.o.csmes'
Warning (Squish Coco): Original:
Warning (Squish Coco):     4: static inline char letter()
Warning (Squish Coco):     5: {
Warning (Squish Coco):     6: #ifdef A_H
Warning (Squish Coco): >>> 7:     return 'a';
Warning (Squish Coco):     8: #else
Warning (Squish Coco):     9:     return 'b';
Warning (Squish Coco):     10: #endif

One sees here:

The first line shows that the error occurred during the compilation of the file header.h.
The second line shows that the problem was found when the information from the file b.o.csmes was merged into main.csmes, and that it occurred on line 7.
What actually happened was that at first, main.csmes did not contain any information about header.h. Then, during the link step, a.o.csmes got merged. It contained information about header.h in the version with the symbol A_H defined. And after that, b.o.csmes got merged too – with a different version of header.h. This then caused the inconsistency.
The next lines show the line that is instrumented differently: it is the line with return 'a'; and the first line that is only part of the code when A_H is defined.
The three lines before and three lines after this line are also shown. The number of context lines can be set with the flag –cs-verbose-source-lines.
These differences could be caused by compiler options. The following lines, beginning with the first line that starts with [ALL], summarizes the options in a form that is easy to read.
The summary consists of several sections, each prefixed by an expression in square braces. The prefix is either [ALL] or something like [1/2]. Lines that start with [ALL] refer to all files while others refer to a group of files that were compiled with the same command line options. The first number is the number of the file group and the second is the number of all file groups. This means that [1/2] refers to the first of two groups.

All file group sections have the same form: First there is a line that lists the contents of the group. In our example, the first group (and also the second) consists of just one file:
```
Warning (Squish Coco): [1/2] Source files:                     insertargs/a.cpp
```
The following line then lists the distinct define flags, that is those that do not appear in all groups. In the first group, this is -DA_H, as expected, since we have defined the symbol A_H only for the file a.cpp. The second group has no distinct flags.

There are actually four kinds of command line options that can be listed for each group:
- Distinct define flags: The flags that define compiler symbols.
- Distinct include directories: This is for the flags the define the directories in which #include finds header files. (The directories are sorted alphabetically, so that the rare errors in which the compilation depends on the order of the include paths cannot be caught this way.)
- Distinct instrumentation options: This is for coveragescanner options.
- Distinct compiler flags (others): For the rest of the flags.
After the file groups and their options, the command line options common to all the source files are listed with the prefix [ALL].

In our example, only instrumentation options are listed.

Resolving the warning with `--cs-inject-arg`

While this is only a warning, it is possible to use Coco to remove it. This is done by injecting (or removing) command line arguments to/from the build commands, using –cs-inject-arg or –cs-remove-arg

Try adding this line to the instrumented script above:

export COVERAGESCANNER_ARGS+=' --cs-inject-arg=/b.cpp/-DA_H'

This causes an additional argument -DA_H to be passed to the compiler when it is processing b.cpp.

This changes the behavior of the AUT without modifying the source code and might be dangerous, but it shows one possible use case for --cs-inject-arg.

When this instrumented program is built, we see no warning anymore, and when we run it, the output will be:

Letter A: a
Letter B: a