Squish for GDC Tutorials
Learn how to test GDC applications.
Tutorial: Starting to Test GDC Applications
In this tutorial we create a first test for a program written with Four J's GDC. We use a small example program that can convert currencies between Euros, US dollars, and Japanese Yens.
In the screenshot below you can see the GUI of the program: in the right pane you can choose a currency you want to convert. Once you clicked on one of the buttons, the line edit for that currency becomes editable. After you entered a value, the other line edits are updated to show the converted values.
You can find this example program in
example/gdc/currencyconverter of your Squish installation. The test suite we develop is located in
In the following sections we will create a test suite and then create some tests, but first we will very briefly review some key Squish concepts.
To perform testing, you need:
- An application to test, known as the Application Under Test (AUT).
- A test script that exercises the AUT.
One fundamental aspect of Squish's approach is that the AUT and the test script that exercises it are always executed in two separate processes. This ensures that even if the AUT crashes, it should not crash Squish. In such cases, the test script will fail gracefully and log an error message. In addition to insulating Squish and test scripts from AUT crashes, running the AUT and the test script in separate processes brings other benefits. For example, it makes it easier to store the test scripts in a central location and to perform remote testing on different machines and platforms. The ability to do remote testing is particularly useful for testing AUTs that run on multiple platforms and for testing AUTs that run on embedded devices.
Squish runs a small server, squishserver, that handles the communication between the AUT and the test script. The test script is executed by the squishrunner tool, which in turn connects to squishserver. squishserver starts the instrumented AUT on the device, which starts the Squish hook. The hook is a small library that makes the AUT's live running objects accessible, and allows communication with squishserver. With the hook in place, squishserver can query AUT objects regarding their state and can execute commands on behalf of squishrunner. squishrunner directs the AUT to perform whatever actions the test script specifies.
All the communication takes place using network sockets which means that everything can be done on a single machine, or the test script can be executed on one machine and the AUT can be tested over the network on another machine.
The following diagram illustrates how the individual Squish tools work together.
From the test engineer's perspective, this separation is not noticeable because all the communication is handled transparently behind the scenes.
Tests can be written and executed using the Squish IDE, in which case squishserver is started and stopped automatically, and the test results are displayed in the Squish IDE's Test Results view. The following diagram illustrates what happens behind the scenes when the Squish IDE is used.
The Squish tools can also be used from the command line without the Squish IDE. This is useful if you prefer to use you own tools, such as your favorite editor or want to perform automatic batch testing. For example, when running regression tests overnight. In these cases, squishserver must be started manually and stopped when all the testing is complete or started and stopped for each test.
For Squish to make it possible for test scripts to be able to query and control an AUT, Squish must be able to access the AUT's internals, and this is made possible by the use of bindings. Bindings are in effect libraries that provide access to the objects—and in turn to the objects' properties and methods—that are available from a GUI toolkit, or from the AUT itself.
There are two sets of bindings that are of interest when developing tests using Squish.
- GUI toolkit bindings—Squish provides bindings for all the GUI toolkits it supports, including Qt, Java AWT/Swing, Java SWT, Web, etc. This means that all the standard objects (including the GUI widgets) provided by these toolkits can be queried and controlled by Squish test scripts.
- AUT-specific bindings—it is possible to create bindings that provide access to the AUT's own API for those cases where the toolkit's bindings don't provide sufficient functionality for proper testing. For Java- and Qt-based AUTs Squish automatically creates bindings to the AUTs objects—including custom classes. See How to Create and Access Application Bindings.
The need to make AUT-specific bindings is rarely needed in practice, but if it really is necessary, Squish provides a tool to make the process as simple as possible. The tool, squishidl, is used to instrument the AUT (and any additional components) to generate AUT-specific bindings. The generated bindings library is seamlessly integrated with the standard GUI toolkit bindings and in the same way will automatically be loaded on demand by the Squish test tools.
When Squish automatically creates bindings to AUT classes, for Qt applications this means that the properties and slots of the AUT's custom widgets can be accessed without having to take any special action, and for Java AUTs this means that objects of custom classes are automatically available in test scripts without needing to be registered.
Note: The Squish documentation mostly uses the term widget when referring to GUI objects, such as buttons, menus, menu items, labels, and table controls. Windows users might be more familiar with the terms control and container, but here we use the term widget for both. Similarly, macOS users may be used to the term view.
Making an Application Testable
In most cases, nothing special needs to be done to make an application testable, since the toolkit's API (e.g., Qt) provides enough functionality to implement and record test scripts. The connection to the squishserver is also established automatically, when the Squish IDE starts the AUT.
Note: Throughout the manual, we often refer to the
SQUISHDIR directory. This means the directory where Squish is installed, which might be
/opt/local/squish, or something else, depending on where you installed it. The exact location doesn't matter, so long as you mentally translate the
SQUISHDIR directory to whatever the directory really is when you see paths and filenames in this manual.
Creating a Test Suite
Squish groups tests into test suites and test cases: test suites are a container for the test cases, whereas test cases contain the test scripts. One test suite defines certain settings that are used for all test cases it contains, like the application that is started for executing the test.
So first we have to create a new test suite: start the Squish IDE and open the new test suite wizard with File > New Test Suite (Ctrl+N):
In the first wizard page, you have to enter a name for the test suite and a directory where the test suite is generated. Squish requires that a test suite's name starts with the prefix
suite_. But it ensures this automatically, so it is sufficient to enter just the name -
currencyconverter in our case.
In the treeview you have to choose a directory where the test suite is stored.
Note: Squish organizes test suites, test cases, and all other data belonging to a test suite in text files in a directory named
suite_name. This enables you to check tests into a source control system or edit tests outside the Squish IDE.
Press Next to get to the next wizard page:
In this page you must select the toolkit your application uses. For testing a Four J's GDC application, choose GDC.
On the next page, you have to choose the scripting language you want to use for writing your tests. In this example we use Python - the other scripting languages work very similar.
On the last wizard page, you have to select the application under test (AUT). For testing Four J's GDC based programs, this is always the gdc application. The first time you create a new test suite, you have to add the application with its full path: click the browse button ( ) and choose the path to the gdc binary.
Once gdc is registered as an application, you can simply select it in the Application combobox.
Click the Finish button now and the Squish IDE displays the newly created test suite. The suite contains some subitems: an item for the Object Map (see Object Map and the Object Map view), an item for shared test data and shared scripts (see How to Create and Use Shared Data and Shared Scripts) and an item for listing the results of previous test runs.
Recording a Test
Now that we have a test suite, we can record our first test: first you have to create a new test case in the suite; choose File > New Test Case (Ctrl+N). The Squish IDE inserts a new test case into the tree with the cursor ready to enter a name for the test case. Type
eur_conversion and press Return. This adds the test case with the name
tst_eur_conversion (Squish's convention for test case naming is similar to the naming of test suites: it requires them to start with the prefix
Creating a new test case not only adds the entry to the tree, but also creates an empty test script. You can open this test script in the editor by clicking on the test.py entry in the tree.
The Squish IDE should now look like:
Next, we start recording the test case: select the entry tst_eur_conversion and right-click it to open the context menu. Choose Record from the menu. Now the Squish IDE opens a dialog where you can choose certain recording options. The defaults are just fine for us, so press Ok. The Squish IDE then minimizes itself to a window just containing recording controls and it starts the application under test, i.e. Four J's GDC. All actions you do in Four J's GDC are now recorded until you exit it.
For this test we just want to add
10 in the field for the Euros and test that the calculated values are correct. So after Four J's GDC we start the shortcut for the currency converter (we assume that the shortcut is already added).
In the currency converter window we choose the EUR button in the menu and type
10 in the line edit for Euros. Pressing Ok calculates and displays the converted values into US dollars and Yens.
Now quit the application: first we close the currency converter window and exit Four J's GDC with the Quit button. The Squish IDE recognizes that you finished Four J's GDC and it shows its normal window again with the newly recorded script. The script looks like:
def main(): snooze(1.5) waitForObject(":MonitorView.qt_central_widget." + "m_ShortcutsStartToolButton") clickButton(":MonitorView.qt_central_widget." + "m_ShortcutsStartToolButton") waitForObject(":Edit.screen..MenuAction_eur") clickButton(":Edit.screen..MenuAction_eur") waitForObject(":Edit.screen.Edit.FormField_eur") type(":Edit.screen.Edit.FormField_eur", "10") waitForObject(":Edit.screen..Action_accept") clickButton(":Edit.screen..Action_accept") waitForObject(":Edit.screen..") sendEvent("QCloseEvent", ":Edit.screen..") waitForObject(":MonitorView.qt_central_widget.m_ExitToolButton") clickButton(":MonitorView.qt_central_widget.m_ExitToolButton")
Squish generates scripts that are human readable and that you can later edit and change to meet your needs. So let's take a look at some statements:
snoozefunction waits for the specified number of seconds.
waitForObjectwaits until an object is available. This is important for synchronizing the script; for example, starting the application takes a certain amount of time. So when we want to press the Start in Four J's GDC, we have to be sure that the button object is already there. Using
waitForObjectis a very stable way to do it since it doesn't depend on any timing (unlike if you would do the synchronization with the
- With the function
clickButtonwe send a mouse click to the object specified. The
typefunction sends keyboard input to the object and
sendEventsends other events to the object (like the
QCloseEventin the above example).
Note: Objects are addressed through names. Squish automatically generates names for objects. If the GUI changes or if you want to use more intuitive names, Squish supports the concept of Object Maps that allow a mapping from symbolic names to the real names. See Object Map and the Object Map view for more information on Object Maps.
The recorded script is ready to be played back again. Click the play button ( ) and you can watch how Squish replays the above script and repeats your input. But it actually doesn't test anything yet. The next step is to add verification points to the script so that it actually tests something.
Inserting a Verification Point
Now we want to test that the currency converter application converted our input (
10) correctly to US dollars and Yens. Squish allows you to insert verification points interactively. For this, you have to run the test until the state you want to test. Then you can choose the widgets and their properties you want to test.
In our case we want to run the test until line 12, that is until we accepted the input, but before we close the windows. So we place a breakpoint on the line
You do this by clicking in the editor on the area showing the line number. The result should look like:
If you play back the script, it is executed until it hits the line with the breakpoint and the Squish IDE shows up with all controls again. The current script line is highlighted in the editor and the lower half of the editor shows the variable watcher. The variable watcher allows you to inspect the variables of your script - this is useful if you have to debug a test script. But for our case of inserting a verfication point, we have to start the Spy instead. You can use the Spy to inspect the objects of your application (address the objects by point and click rather than using the object name and inspect all properties of the objects that are accessible to Squish).
Start the Spy by clicking on the corresponding button ( ) in the toolbar. After a short moment Squish replaces the variable watcher with the spy window. Here you can either select an object by choosing it from the tree view or by using the object picker. If you want to access a visible object it is usually much easier to use the object picker.
Start the object picking by clicking the picker icon ( ) from the toolbar. The Squish IDE goes into mini-mode again and you can select the object you want to access with simple point and click: Squish draws a red frame around the object it finds under the cursor. If you click, it selects the object, the Squish IDE leaves the mini-mode and shows the selectd object in the Spy:
In the above example we picked the line edit that displays the value in US dollars. Since we want to verify that the value displayed is 12.00 check the checkbox of the line edit object. The Squish IDE automatically checks the
text property of the line edit as well. This is done for convenience: Squish checks the common properties by default (i.e. that one that a user is most likely intrested in testing for a certain object type).
You can check additonal properties and uncheck those you are not intrested in. In our small example however, we only want to check the text property. So we don't have to do anything else.
So we selected the object we want to verify and the properties of the object we want to verify. Now we just have to choose a name for the verification point and then we can add the verification point to the script: change the text
VP1 in the Verification Point line edit to
VP_usd and press the green check mark icon to add the verifcation point to the script.
For the line edit dislaying the value in Yens, we do the same procedure to add a second verification point:
- Choose the pick tool( ) from the toolbar and pick the line edit that displays the value for Yens.
- Check the object in the tree and make sure that its text property is checked as well.
- Change the name of the verification point to
VP_yenand press the green check mark to add the verification point.
Now we have all verification points we need and we can stop the test execution. So select Test Suite > Stop from the menu. If you now take a closer look at the script you see that Squish added two lines:
These are the two verification point tests we just added. So now we want to run our test with the verification points. First remove the breakpoint by clicking on it and press the play button ( ). Squish replays the test again, but this time when the test is finished you can see two additional messages in the Test Log view about the verification points that passed.
If you want to take a closer look at the verification point itself, you can open it in the tree: first you have to open the tst_eur_conversion entry in the tree and then open the verificationPoints entry below it. If you click on one of the two verification point, the editor shows what the verification point actually tests and you can change the values.
If you change the value of the
VP_usd verification point to
13.00 and run the test again, you get one fail and one pass as the test result.
See How to Create and Use Verification Points for more details on this topic.
In this tutorial we just saw the basic usage of Squish. There is much more to discover for efficient usage of Squish.
In order to keep tests maintainable, it might be useful in to edit a test script after you recorded it and factor out certain parts to functions. More complex tasks (like iterating over the properties of an object or use branching depending on certain conditions) can only be done by editing the script. See How to Create and Use Property Verification Points in Test Scripts for more details on scripting and see API Reference for a complete reference of the Squish specific scripting API. If you want to share certain script functions between different test cases or test suites, you can used shared scripts - see How to Store and Locate Shared Scripts and Shared Data Files.
Apart from the Squish specific functions, you can also use the features that come with the scripting language of your choice - for example you can use all Python modules to achieve your goal, like the modules for accessing a SQL database. For more details on this, please refer to the documentation of the scripting language you use.
Data-driven testing is a way to separate the test data from the test script code. By doing so you can run the same test script with different input and test it against different expected results. See How to Do Data-Driven Testing for details.
Squish is actually a collection of commandline tools tied together by the Squish IDE with an easy to use interface. But you can use the commandline tools without the Squish IDE to achieve the tasks as well. This is especially useful for automating the test execution. See How to Do Automated Batch Testing for some example Python scripts to execute your tests and to post-process the test results. Automated Batch Testing has a more detailed description on this topic. You can find a complete reference on the commandline tools in Command Line Reference