Tutorial: Coverage of GUI elements

GUI Coverage is a form of program verification that tests whether all GUI elements of a program were activated, in a test suite. Squish now has support for GUI coverage of Qt Widgets and QtQuick2 elements. There is no support for QtQuick1 (also known as QtDeclarative). This section describes how Squish GUI coverage is activated and used.

Activation

By default, GUI coverage is switched off. To activate it, set the environment variable SQUISH_GUI_COVERAGE to 1. With any other value, GUI coverage will be switched off.

From the Squish IDE, GUI coverage can be activated in a test suite for AUTs started with startApplication(). From Test Suite Settings, select the AUT tab. Then enter the variable SQUISH_GUI_COVERAGE into the Environment table and set its value to 1.

Activation from Command Line

To run Squish from the command line with GUI coverage enabled, set the SQUISH_GUI_COVERAGE environment variable in the shell. Then one can run squishserver directly in the form

$ export SQUISH_GUI_COVERAGE=1
$ squish/bin/squishserver

Browsing Coverage Results

The test cases can be run as usual, but now, there is an extra entry at the end of the result list in the Test Results window, with the title GUI coverage results. In the Message column, the location of the XML file with the test results is shown. This can be useful for debugging.

Double-click the entry, and the uibrowser is started and displays the coverage results.

The UI browser display has five windows:

• Element Tree (top left) – A hierarchy of elements is shown, together with their coverage.

  GUI coverage is expressed by terms like "3/4 75%". Such an entry means that 3 of 4 required activities were done with the GUI element, 75% of all. In a complex GUI element, like a window or a menu, the numbers refer to the element together with the elements that it contains. In the element tree, the subelements of an elements are displayed hierarchically below it.

• Screen Shot (bottom left) – Display of the selected element.
• Element Properties (top right) – This windows shows static properties of the selected GUI element.
  • class is the Qt class that is used to implement the element.
  • type is an abstract type that describes the element independent of the GUI toolkit that was used.
• Usage (middle right) – List of the events registered for the selected element and its subelements.

  Each entry in this list refers to one usage counter. Each counter refers to a certain activity of the element. E.g. the counter "shown" is incremented every time an element is displayed, and "clicked" is incremented when an element is clicked. Each type of GUI element has its own list of required events – "clicked" belongs e.g. to menu entries, but not to menus.

• Style check (bottom right) – If the design of the element does not follow certain style rules, warnings are displayed in this window.

Browsing Results from Command Line

When squishrunner has ended, it prints the location of the results file in a form like

2015-07-07T15:30:55  ATTACHMENT GUI Coverage Result \
    addressbook/suite_py/tst_development/coverage_200790021.xml

You can then call the uibrowser directly to see the content of this file:

$ squish/bin/uibrowser addressbook/suite_py/tst_development/coverage_200790021.xml

For the full usage of this command, see uibrowser.

Configuration

GUI coverage can be configured by changing the settings in the XML file <SQUISHDIR>/lib/qt/extensions/squishqtuicoverage.ext. The file has three entries which can be set to 0 or 1 to disable or enable the corresponding feature. By default all are enabled.

• enable – enables or disables GUI coverage as a whole.
• qtwidgets – enables or disables coverage of Qt widgets.
• qtquick – enables or disables coverage of Qt Quick.
• takeScreenshotsOfElements – enables or disables taking screen shots of elements. Disable this if you experience memory problems or if you are not interested in them.

You can use qtwidgets and qtquick to disable GUI coverage partially if there are problems. The use of enable is however not recommended; use SQUISH_GUI_COVERAGE instead.