Squish for Qt Tutorials

Learn how to test applications which are based on the Qt framework.

• Tutorial: Starting to Test Qt Applications
• Tutorial: Designing Behavior Driven Development (BDD) Tests
• Tutorial: Migration of existing tests to BDD
• Tutorial: Model-based Testing

Tutorial: Starting to Test Qt Applications

In this tutorial, we use a simple Address Book application as our AUT. It can be found in <SQUISHDIR>/examples/qt/addressbook. It uses basic Qt Widgets: a menu bar with pull down menus, a toolbar, and a central area—in this case showing a table. It supports in-place editing and also has a pop-up modal dialog for adding items.

For other examples testing Qt-specific features including views, as well as all the standard editing widgets, see How to Create Test Scripts and How to Test Qt Applications.

The screenshot shows the application in action with a user adding a new name and address.

Squish Multi-Process Architecture and IPC

Squish runs a small server, squishserver, that handles the communication between the AUT and the test script. The test script is executed by the squishrunner tool, which in turn connects to squishserver. squishserver starts the instrumented AUT on the device, which starts the Squish Hook. With the hook in place, squishserver can query AUT objects regarding their state and can execute commands on behalf of squishrunner. squishrunner directs the AUT to perform whatever actions the test script specifies.

All the communication takes place using network sockets which means that everything can be done on a single machine, or the test script can be executed on one machine and the AUT can be tested over the network on another machine.

The following diagram illustrates how the individual Squish tools work together.

Tests can be written and executed using the Squish IDE, in which case squishserver is started and stopped automatically, and the test results are displayed in the Squish IDE's Test Results view. The following diagram illustrates what happens behind the scenes when the Squish IDE is used.

Under the covers, squishrunner is used to execute test cases. If we need to automate the execution of test cases from a script, we would use this command directly.

Making your Application Testable

Standard QWidgets and QtQuick Controls provided by Qt can be queried and controlled by Squish test scripts. Custom QObject-derived classes are introspected by Squish through the QMetaObject interface. This is why you need binary-compatibility between Squish and the AUT.

Squish should work with unmodified binary executables of Qt AUTs, provided the Qt version used by the AUT and Squish are binary-compatible with each other. This means that the major and minor versions must match. For example, Squish 8.1 for Qt 6.7 works with AUTs that are built with Qt 6.7.2 and not Qt 6.6.3 or Qt 6.8.1.

Creating Test Suites from Squish IDE

Start up the Squish IDE, by clicking or double-clicking the Squish IDE icon, by launching Squish IDE from the taskbar menu or by executing squishide on the command line, whichever you prefer and find suitable for the platform you are using.

Once Squish starts up, you might be greeted with a Welcome Page. Click the Workbench button in the upper right to dismiss it. Then, the Squish IDE will look similar to the screenshot.

Before we can create Test Cases, we must first create a Test Suite to group them together. .

Once Squish has started, click File > New Test Suite to pop-up the New Squish Test Suite wizard shown below.

Enter a name for your Test Suite and choose the folder where you want the it to be stored. In the screenshot, we have called the test suite suite_py. Once the details are complete, click Next to go on to the Toolkit (or Scripting Language) page.

For the 'Toolkit' page, select Qt since we are testing a Qt application. Then click Next to go to the Scripting Language page.

Choose your desired scripting language on this page. Our example tests are in 5 different languages and you can pick the one that suits you. The functionality offered by Squish is the same for all languages.

If you are creating a new test suite for an AUT that Squish already knows about, simply click the combobox to drop-down the list of AUTs and choose the one you want. If the combobox is empty or your AUT isn't listed, click the Browse button to the right of the combobox—this will pop-up a file open dialog from which you can navigate to <SQUISHDIR>/examples/qt/addressbook/ to select your AUT.

In the case of Qt programs, the AUT is the application's executable (e.g., addressbook.exe on Windows). Once you have chosen the AUT, click Finish and Squish will create a Test Suite. The wizard will then close and Squish IDE will look similar to the screenshot below.

We are now ready to start creating tests.

Recording Tests and Verification Points

Recordings are made into existing test cases. You can create a New Script Test Case in the following ways:

• Select File > New Test Case to open the New Squish Test Case wizard, enter the name for the test case, and select Finish.
• Click the New Script Test Case () toolbar button to the right of the Test Cases label in the Test Suites view. This creates a new test case with a default name, which you can easily change.

Give the new test case the name tst_general.

Squish automatically creates a sub-folder inside the test suite's folder with this name and also a test file, for example test.py. If you choose JavaScript as the scripting language, the file is called test.js, and correspondingly for Perl, Ruby, or Tcl.

If you get a sample .feature file instead of a "Hello World" script, click the arrow left of the Run Test Suite () and select New Script Test Case ().

To make the test script file (such as, test.js or test.py) appear in an Editor view, click or double-click the test case, depending on the Preferences > General > Open mode setting. This selects the Script as the active one and makes visible its corresponding Record () and Run Test Case () buttons.

The checkboxes are used to control which test cases are run when the Run Test Suite () toolbar button is clicked. We can also run a single test case by clicking its Run Test Case () button. If the test case is not currently active, the button may be invisible until the mouse is hovered over it.

Initially, the script's main() function logs Hello World to the test results.

Once the new test case has been created, we are free to write test code manually or to record a test. Clicking on the test case's Record () button replaces the test's code with a new recording. Alternatively, you can record snippets and insert them into existing test cases, as instructed in How to Edit and Debug Test Scripts.

Recording Our First Test

Here is a test scenario we would like to record:

1. Open the MyAddresses.adr address file.
2. Navigate to the second address and then add a new name and address.
3. Navigate to the fourth address (that was the third address) and change the surname field.
4. Navigate to the first address and remove it.
5. Verify that the first address is now the new one that was added.

Click the Record () button to the right of the tst_general test case shown in the Test Suites view's Test Cases list. This will cause Squish to launch and hook up with the AUT. Once the AUT is running, perform the following actions—and don't worry about how long it takes since Squish doesn't record idle time:

1. Click File > Open, and once the file dialog appears, click the MyAddresses.adr filename, then click the Open button.
2. Click the second row, then click Edit > Add, then in the Add dialog's first line edit type in "Jane". Now click (or tab to) the second line edit and type in "Doe". Continue similarly, to set an email address of "jane.doe@nowhere.com" and a phone number of "555 123 4567". Don't worry about typing mistakes—just backspace delete as normal and fix them. Finally, click the OK button. There should now be a new second address with the details you typed in.
3. Click the fourth row's second (surname) column, delete its text and replace it with "Doe". (You can do this simply by overtyping.) Then press Enter to confirm your edit.
4. Click in the first row, then click Edit > Remove, and then click the Yes button in the message box. At the end of this, your "Jane Doe" entry should be the first row.
5. Click the Verify () toolbar button in the Squish Control Bar Window (the second button from the left) and select Properties.

   

   This will make the Squish IDE appear. In the Application Objects view, use the Object Picker () or navigate through the object tree, to select "item_0/0" QModelIndex object, which will make its properties appear in the Properties view. Check the text property's checkbox. Now select the next column's table cell and check its text property. Finally, click the Save and Insert Verifications button (at the bottom of the Verification Point Creator view) to have the forename and surname verifications for the first row inserted into the recorded test script. (See the screenshot below.) Once the verification points are inserted, the Squish IDE's window will be hidden again and the Control Bar window and the AUT will be back in view.

6. We've now completed the test, so in the AUT click File > Quit, then click No in the message box, since we don't want to save any changes.

Once we quit the AUT, the recorded test will look something like what we see in the in this screenshot.

If the recorded test doesn't appear, click (or double-click depending on your platform and settings) the tst_general test case; this will make Squish show the test's test.py file in an editor window as shown in the screenshot.

Now that we've recorded the test, we want to Run Test Case (). The two verifications we inserted should be checked on playback as the next screenshot shows.

After a recording is finished, we can Insert Additional Verification Points, through the use of breakpoints and recording snippets.

Running Tests from IDE

To run a test case in the Squish IDE, click the Run Test Case () that appears when the test case is hovered or selected in the Test Suites view.

To run two or more test cases one after another or to run only the selected test cases, click Run Test Suite ().

Running Tests from Command Line

To playback a recorded test from the command line, we execute the squishrunner program. We provide squishrunner the path to a test suite, and optionally also the name of a test case.

A squishserver must be running when running a test, and we can provide squishrunner an IP/Port of an already running one, or use the --local option which creates one for the duration of the process. For more information, see squishserver.

For example, assuming we are in the directory that contains the test suite's directory:

squishrunner --testsuite suite_py --testcase tst_general --local

Examining the Recorded Script

If you look at the code in the screenshot (or the code snippet shown below) you will see that it consists of lots of Object waitForObject(objectOrName) calls as parameters to various other calls such as activateItem(itemObject), clickButton(objectOrName), clickItem(objectOrName, itemOrIndex, x, y, modifierState, button), and type(objectOrName, text). The Object waitForObject(objectOrName) function waits until a GUI object is ready to be interacted with (i.e., becomes visible and enabled), and is then followed by some function that interacts with the object. The typical interactions are activate (pop-up) a menu, click a menu option or a button, or type in some text.

For a complete overview of Squish's script commands see How to Create Test Scripts, How to Test Applications - Specifics, API Reference, and Tools Reference.

Objects are identified by names that Squish generates. See How to Identify and Access Objects for full details.

The script is about 35 lines of code. Here's an extract that just shows how Squish records clicking the Edit menu's Add action, typing in Jane Doe's details into the Add dialog, and clicking OK at the end to close the dialog and update the table.

    clickButton(waitForObject(names.address_Book_MyAddresses_adr_Add_QToolButton))
    snooze(.5)  # qt4/linux workaround
    type(waitForObject(names.forename_LineEdit), "Jane")
    type(waitForObject(names.forename_LineEdit), "<Tab>")
    type(waitForObject(names.surname_LineEdit), "Doe")
    type(waitForObject(names.surname_LineEdit), "<Tab>")
    type(waitForObject(names.email_LineEdit), "jane.doe@nowhere.com")
    type(waitForObject(names.email_LineEdit), "<Tab>")
    type(waitForObject(names.phone_LineEdit), "123 555 1212")
    clickButton(waitForObject(names.address_Book_Add_OK_QPushButton))

    clickButton(waitForObject(names.addressBookMyAddressesAdrAddQToolButton));
    type(waitForObject(names.forenameLineEdit), "Jane");
    type(waitForObject(names.forenameLineEdit), "<Tab>");
    type(waitForObject(names.surnameLineEdit), "Doe");
    type(waitForObject(names.surnameLineEdit), "<Tab>");
    type(waitForObject(names.emailLineEdit), "Jane.doe@nowhere.com");
    type(waitForObject(names.emailLineEdit), "<Tab>");
    type(waitForObject(names.phoneLineEdit), "123 555 1212");
    clickButton(waitForObject(names.addressBookAddOKQPushButton));

    clickButton(waitForObject($Names::address_book_myaddresses_adr_add_qtoolbutton));
    type(waitForObject($Names::forename_lineedit), "Jane");
    type(waitForObject($Names::forename_lineedit), "<Tab>");
    type(waitForObject($Names::surname_lineedit), "Doe");
    type(waitForObject($Names::surname_lineedit), "<Tab>");
    type(waitForObject($Names::email_lineedit), "Jane.Doe\@nowhere.com");
    type(waitForObject($Names::email_lineedit), "<Tab>");
    type(waitForObject($Names::phone_lineedit), "123 555 1212");
    clickButton(waitForObject($Names::address_book_add_ok_qpushbutton));

    clickButton(waitForObject(Names::Address_Book_MyAddresses_adr_Add_QToolButton))
    type(waitForObject(Names::Forename_LineEdit), "Jane")
    type(waitForObject(Names::Forename_LineEdit), "<Tab>")
    type(waitForObject(Names::Surname_LineEdit), "Doe")
    type(waitForObject(Names::Surname_LineEdit), "<Tab>")
    type(waitForObject(Names::Email_LineEdit), "jane.doe@nowhere.com")
    type(waitForObject(Names::Email_LineEdit), "<Tab>")
    type(waitForObject(Names::Phone_LineEdit), "123 555 1212")
    clickButton(waitForObject(Names::Address_Book_Add_OK_QPushButton))

    invoke clickButton [waitForObject $names::Address_Book_MyAddresses_adr_Add_QToolButton]
    invoke type [waitForObject $names::Forename_LineEdit] "Jane"
    invoke type [waitForObject $names::Forename_LineEdit] "<Tab>"
    invoke type [waitForObject $names::Surname_LineEdit] "Doe"
    invoke type [waitForObject $names::Surname_LineEdit] "<Tab>"
    invoke type [waitForObject $names::Email_LineEdit] "jane.doe@nowhere.com"
    invoke type [waitForObject $names::Email_LineEdit] "<Tab>"
    invoke type [waitForObject $names::Phone_LineEdit] "123 555 1212"
    invoke clickButton [waitForObject $names::Address_Book_Add_OK_QPushButton]

Here, the tester used the keyboard to tab from one text field to another and clicked the OK button using the mouse rather than with a key press. If the tester had clicked the button any other way (for example, by pressing Alt+O, or by tabbing to the OK button and pressing the spacebar), the outcome should be the same. With most Qt GUI applications, TMTOWTDI, and each of the ways can be recorded and tested separately by Squish.

Squish recordings refer to objects using variables that begin with a names. prefix. These are known as Symbolic Names. Each variable contains, as a value, the corresponding Real Name.

The advantage of using symbolic names (instead of real names) in your scripts, is that if the application changes in a way that results in different names being needed, it is possible to update Squish's Object Map and thereby avoid the need to change our test scripts.

When a Symbolic Name is under the cursor, the editor's context menu allows you to Open Symbolic Name, showing its entry in the Object Map, or Convert to Real Name, which places an inline mapping in your script language at the cursor, allowing you to hand-edit the properties in the script itself.

Now that we have seen how to record and play back a test and have seen the code that Squish generates, let's go a step further and make sure that at particular points in the test's execution certain conditions hold.

Inserting Additional Verification Points

In the previous section we saw how easy it is to insert verification points during the recording of test scripts. Verification points can also be inserted into existing test scripts, by recording snippets, or simply by editing a test script and putting in calls to Squish's test. functions such as test.compare and test.verify.

Squish supports these kinds of Verification Points:

• Properties VPs, which can be Scriptified or XML-based. They verify that 1 or more properties of 1 or more objects have certain values.
• Table VPs, which verify the contents of an entire table.
• Screenshot VPs, which contain an XML-encoded image for comparison. Image Search, a newer feature of Squish, is preferred over creating VPs of this type.
• Visual VPs, which contain properties and screenshots for an entire tree of objects.

Regular Verification Points are stored as XML files in the test case or test suite resources, and contain the value(s) that need to be compared. These verification points can be reused across test cases, and can verify many values in a single line of script code.

Scriptified Property Verification points are direct calls to the test.compare function, with two arguments—the value of a particular property for a particular object, and an expected value. We can manually write calls to the test.compare function in a script, or we can get Squish to insert them for us using Scriptified Property Verification Points.

We will verify that the "Jane Doe" entry's email address and phone number match the ones entered, and put the verifications immediately after the ones we inserted during recording.

To insert a verification point using the Squish IDE we start by putting a break point in the script (whether recorded or manually written—it does not matter to Squish), at the point where we want to verify.

As the above screenshot shows, we have set a breakpoint just before the end of the test case. This is done simply by double-clicking, or right-clicking in the gutter (next to the line number in the editor) and selecting the Add Breakpoint context menu item. We chose this line because it follows the script lines where the first address is removed, so at this point (just before invoking the File menu to close the application), the first address should be that of "Jane Doe". The screenshot shows the verifications that were entered using the Squish IDE during recording. Our additional verifications will follow them.

Having set the breakpoint, we now run the test as usual by clicking the Run Test Case () button, or by clicking the Run > Run Test Case menu option. The test run and then stop when the breakpoint is reached. Squish's main window will reappear (which may obscure the AUT). At this point, the Squish IDE will switch to the Test Debugging Perspective.

Perspectives and Views

The Squish IDE works just like the Eclipse IDE. If you aren't used to Eclipse, it is crucial to understand the following key concepts: Views and Perspectives. In Eclipse, and therefore in the Squish IDE, a View is essentially a child window, such as a dock window or a tab in an existing window. A Perspective is a collection of views arranged together. Both are accessible through the Window menu.

The Squish IDE is supplied with the following perspectives:

• Test Management Perspective that the Squish IDE starts with, and that is shown in all previous screenshots
• Test Debugging Perspective
• Spy Perspective

You can modify these perspectives to show additional views, to hide views that you don't want, or to create your own perspectives with exactly the views you want.

If you notice all of your Views change dramatically, it just means that the perspective changed. Use the Window menu to change back to the perspective you want. Keep in mind, Squish automatically changes perspectives to reflect the current situation, so you should not need to change perspective manually very often.

Inserting Verification Points

As the screenshot below shows, when Squish stops at a breakpoint, the Squish IDE automatically changes to the Test Debugging Perspective. The perspective shows the Variables view, the Editor view, the Debug view, the Application Objects view, and the Properties view, Methods view, and Test Results view.

The normal Test Management Perspective can be returned to at any time by choosing it from the Window menu (or by clicking its toolbar button), although the Squish IDE will automatically return to it if you stop the script or run it to completion.

To insert a verification point, we can navigate through Application Objects or use the Object Picker (). Once the correct object is selected in Application Objects, we simply check the desired properties in the Properties view and we can see a Verification Point Creator like in the screenshot.

We want to verify the text property of the last 2 columns of the first row, containing Email Address and Phone.

After the first one is checked, the Verification Point Creator view appears as shown in the screenshot.

At this point the verification point has not been added to the test script. Navigate or use the Object Picker () to select the other table cell and check its text property. Now both verifications will appear in the Verification Point Creator view as the screenshot shows.

We have now said that we expect these properties to have the values shown, that is, an email address of "jane.doe@nowhere.com" and phone number of "555 123 4567". We must click the Save and Insert Verifications button to actually insert the verification point, so do that now.

We are still stopped at a breakpoint. Now, we can Terminate() or Resume(), depending on whether we want to continue running the test case to the end. The test case should look like the screenshot below. (We have also highlighted the lines of code that Squish inserted).

After the AUT terminates and we are back in Test Management Perspective, we should disable the break point. Right-click the break point and click the Disable Breakpoint menu option in the context menu.

We are now ready to run the test without any breakpoints but with the verification points in place. Click Run Test Case (). This time we will get some additional test results — as the screenshot shows — we have expanded them to show details.

Now there are four passing tests comparing the forename, surname, email, and phone number of the newly inserted entry.

Manually Written Property Verifications

Another way to insert verification points is to write the code manually. We can add our own calls to Squish's test. functions, such as test.compare and test.verify in an existing script.

First we set a breakpoint where we intend on adding our verifications. Then we Run Test Case () until it stops there. Next, we use the Object Picker () or navigate in the Application Objects view for the the object we want to verify.

Right click the Application Object entry select the Copy Symbolic Name context menu option—this adds the object to the Object Map if necessary, and returns the symbolic name for that entry.

Now we can edit the test script, paste name into the script where we need to find the object.

(Don't forget to disable the break point once it isn't needed any more.)

For this manual verification, we want to check the number of addresses present in the QTableWidget after reading in the MyAddresses.adr file, then after the new address is added, and finally after the first address is removed. The screenshot shows two of the lines of code we entered to get one of these three verifications, plus the results of running the test script.

When writing scripts by hand, we use Squish's test module's functions to verify conditions at certain points during our test script's execution. As the screenshot (and the code snippets below) show, we begin by retrieving a reference to the object we are interested in. Using the Object waitForObject(objectOrName) function is standard practice for manually written test scripts. This function waits for the object to be available (i.e., visible and enabled), and then returns a reference to it. (Otherwise it times out and raises a catchable exception.) We then use this reference to access the item's properties—in this case the QTableWidget's rowCount property—and verify that the value is what we expect it to be using the test.compare function.

Here is the code we entered manually for the first verification. We added calls to the test.compare function, reusing the table object reference we obtained in the code shown below.

    table = waitForObject(names.address_Book_MyAddresses_adr_File_QTableWidget)
    test.compare(table.rowCount, 125)

    var table = waitForObject(names.addressBookMyAddressesAdrFileQTableWidget)
    test.compare(table.rowCount, 125);

    my $table = waitForObject($Names::address_book_myaddresses_adr_file_qtablewidget);
    test::compare($table->rowCount, 125);

    table = waitForObject(Names::Address_Book_MyAddresses_adr_File_QTableWidget)
    Test.compare(table.rowCount, 125)

    set table [waitForObject $names::Address_Book_MyAddresses_adr_File_QTableWidget]
    test compare [property get $table rowCount] 125

For each property of interest, we retrieve a reference to the object we are interested in, and then verify its properties using one of Squish's verification functions. We can also call methods on the object to interact with it, if we wish.

For more examples of manually written tests, see Creating Tests by Hand, How to Create Test Scripts, and How to Test Applications - Specifics.

For more information about verification points, see How to Create and Use Verification Points.

Test Results

After each test run finishes, the test results—including those for the verification points—are shown in the Test Results view at the bottom of the Squish IDE.

This is a detailed report of the test run and would also contain details of any failures or errors, etc. If you click on a Test Results item, the Squish IDE highlights the script line which generated it. If you expand the item, you can see additional details of it.

Squish's interface for reporting test results is very flexible. The default report generator simply prints the results to stdout when Squish is run from the command line, or to the Test Results view when Squish IDE is being used. You can save the test results from the Squish IDE as XML by right clicking on the Test Results and choosing the Export Results menu option. For a list of report generators, see squishrunner –reportgen: Generating Reports.

It is possible to Upload the Results to Test Center, where they are stored in a database for analysis later.

Creating Tests by Hand

Now that we have seen how to record a test and modify it by inserting verification points, we are ready to see how to create tests manually. The easiest way to do this is to modify and refactor recorded tests, although it is also perfectly possible to create manual tests from scratch.

For each object of interest, we need a Symbolic or Real Name to access it.

If we have not interacted yet with the object of interest, we can get a name for it this way.

1. click the Launch AUT () toolbar button. This starts the AUT and switches to the Spy Perspective. We can then interact with the AUT until the object we are interested in is visible.
2. From the Application Objects, use the Object Picker () or tree navigation to choose the desired object.
3. use the context menu to Add to Object Map or Copy (Symbolic | Real) Name to Clipboard (so that we can paste it into our test script).

If Squish has previously interacted with the object before, we can get its name from the Object Map.

We can view the Object Map by clicking the Object Map() toolbar button, or from the Script Editor context menu, Open Symbolic Name when right-clicking on an object name in the script editor. Every application object that Squish interacts with is listed here, either as a top-level object, or as a child object (the view is a tree view).

We can retrieve the symbolic name used by Squish in recorded scripts by right-clicking the object we are interested in and then clicking the context menu's Copy Object Name (to get the Symbolic Name variable) or Copy Real Name (to get the actual key-value pairs stored in the variable). This is useful for when we want to modify existing test scripts or when we want to create test scripts from scratch, as we will see later on in the tutorial.

Modifying and Refactoring Recorded Tests

Suppose we want to test the AUT's Add functionality by adding three new names and addresses. We could of course record such a test but it is just as easy to do everything in code. The steps we need the test script to do are: first click File > New to create a new address book, then for each new name and address, click Edit > Add, then fill in the details, and click OK. And finally, click File > Quit without saving. We also want to verify at the start that there are no rows of data and at the end that there are three rows. We will also refactor as we go, to make our code as neat and modular as possible.

First, we must create a new test case. Click New Script Test Case () and set the test case's name to be tst_adding.

The first thing we need is a way to start the AUT and then invoke a menu option. Here are the first few lines from the recorded tst_general script:

import names
import os

def main():
    startApplication('"' + os.environ["SQUISH_PREFIX"] + '/examples/qt/addressbook/addressbook"')
    activateItem(waitForObjectItem(names.address_Book_QMenuBar, "File"))
    activateItem(waitForObjectItem(names.address_Book_File_QMenu, "Open..."))

import * as names from 'names.js';

function main() {
    startApplication('"' + OS.getenv("SQUISH_PREFIX") + '/examples/qt/addressbook/addressbook"');
    activateItem(waitForObjectItem(names.addressBookQMenuBar, "File"));
    activateItem(waitForObjectItem(names.addressBookFileQMenu, "Open..."));

require 'names.pl';

sub main
{
    startApplication("\"$ENV{'SQUISH_PREFIX'}/examples/qt/addressbook/addressbook\"");
    activateItem(waitForObjectItem($Names::address_book_qmenubar, "File"));
    activateItem(waitForObjectItem($Names::address_book_file_qmenu, "Open..."));

require 'names'
include Squish

def main
    startApplication("\"#{ENV['SQUISH_PREFIX']}/examples/qt/addressbook/addressbook\"")
    activateItem(waitForObjectItem(Names::Address_Book_QMenuBar, "File"))
    activateItem(waitForObjectItem(Names::Address_Book_File_QMenu, "Open..."))

source [findFile "scripts" "names.tcl"]

proc main {} {
    startApplication "\"$::env(SQUISH_PREFIX)/examples/qt/addressbook/addressbook\""
    invoke activateItem [waitForObjectItem $names::Address_Book_QMenuBar "File"]
    invoke activateItem [waitForObjectItem $names::Address_Book_File_QMenu "Open..."]

The pattern in the code is simple: start the AUT, then wait for the menu bar, then activate the menu bar; wait for the menu item, then activate the menu item. In both cases we have used the Object waitForObjectItem(objectOrName, itemOrIndex) function. This function is used for a multi-valued objects (such as lists, tables, trees—or in this case, a menubar and a menu), and allows us to access the object's items (which are themselves objects of course), by passing the name of the object containing the item and the item's text as arguments.

Reducing the number of redundant object names

If you look at the recorded test (tst_general) or in the Object Map you will see that Squish sometimes uses different names for the same things. For example, the menubar is identified in three different ways, initially as AddressBook_QMenuBar then if the user clicks File > New, it is identified as AddressBook_Unnamed_QMenuBar, and if the user clicks File > Open and opens the MyAddresses.adr file, then the menubar is identified as AddressBook_MyAddressesadr_QMenuBar. The reason for this is that Squish tries to uniquely identify every object in a given context, and it uses whatever information it has to hand. So in the case of identifying menubars (and many other objects), Squish uses the window title text to give it some context. (For example, an application's File or Edit menus may have different options depending on whether a file is loaded and what state the application is in.)

In our AUT, there is always only one Main Window, so when we write supporting functions for our test cases, we don't want those functions to have to know or care about the window title. Therefore, we need to use less specific object names in functions.

We added a custom object map entry that looks like this:

mainWindow = {"type": "MainWindow", "unnamed": 1, "visible": 1}

export var addressBookMainWindow = {"type": "MainWindow", "unnamed": 1, "visible": 1};

our $mainwindow = {"type" => "MainWindow", "unnamed" => 1, "visible" => 1};

MainWindow = {:type => "MainWindow", :unnamed => 1, :visible => 1}

set MainWindow [::Squish::ObjectName type MainWindow unnamed 1 visible 1]

Subsequent recordings will make use of this entry (as long as more specific entries do not exist) instead of creating new entries which are context sensitive to the window title. You might want to start with a new Test Suite with only this object map entry and observe how the newly added object names are fewer and simpler after that.

Object Not Found Dialog

If the AUT appears to freeze during test execution, wait for Squish to time out the AUT (about 20 seconds), and show the Object Not Found dialog, indicating an error like this:

This usually means that Squish doesn't have an object with the given name, or property values, in the Object Map. From here, we can Pick a new object, Debug, Throw Error or, after picking a new object, Retry.

Picking a new object will update the object map entry for the symbolic name. In addition to the Object Picker (), we can use the Spy's Application Objects view to locate the objects we are interested in and use the Add to the Object Map context menu action to to access their real or symbolic names.

Naming is important because it is probably the part of writing scripts that leads to the most error messages, usually of the object ... not found kind shown above. Once we have identified the objects to access in our tests, writing test scripts using Squish is very straightforward. Especially, as Squish most likely supports the scripting language you are most familiar with.

Pre-Populating the Object Map

Before writing our own test script, we want to add some entries to the object map. One way to do this is by recording a dummy test case. So click File > New Test Case and set the test case's name to be tst_dummy. Then click the dummy test case's Record () button.

Once the AUT starts, click File > New, then click the (empty) table, then click Edit > Add and add an item, then press Return or click OK. Finally, click File > Quit to finish, and say No to saving changes. Then replay this test just to confirm that everything works okay. The sole purpose of this is to make sure that Squish adds the necessary names to the Object Map since it is probably quicker to do it this way than to use the Spy for every object of interest. After replaying the dummy test you can delete it if you want to.

Now we can hand-write test scripts. We will start with the main function, and then we will look at the supporting functions that it calls.

import names
import os

def main():
    startApplication('"' + os.environ["SQUISH_PREFIX"] + '/examples/qt/addressbook/addressbook"')
    invokeMenuItem("File", "New")
    table = waitForObject({"type": "QTableWidget"})
    test.verify(table.rowCount == 0)
    data = [("Andy", "Beach", "andy.beach@nowhere.com", "555 123 6786"),
            ("Candy", "Deane", "candy.deane@nowhere.com", "555 234 8765"),
            ("Ed", "Fernleaf", "ed.fernleaf@nowhere.com", "555 876 4654")]
    for oneNameAndAddress in data:
        addNameAndAddress(oneNameAndAddress)
    waitForObject(table)
    test.compare(table.rowCount, len(data), "table contains as many rows as added data")
    closeWithoutSaving()

import * as names from 'names.js';

function invokeMenuItem(menu, item)
{
    activateItem(waitForObjectItem({"type": "QMenuBar"}, menu));
    activateItem(waitForObjectItem({"title": menu, "type": "QMenu"}, item));
}

require 'names.pl';

sub main
{
    startApplication("\"$ENV{'SQUISH_PREFIX'}/examples/qt/addressbook/addressbook\"");
    invokeMenuItem("File", "New");
    my $table = waitForObject({"type" => "QTableWidget"});
    test::verify($table->rowCount == 0);
    my @data = (["Andy", "Beach", "andy.beach\@nowhere.com", "555 123 6786"],
                ["Candy", "Deane", "candy.deane\@nowhere.com", "555 234 8765"],
                ["Ed", "Fernleaf", "ed.fernleaf\@nowhere.com", "555 876 4654"]);
    foreach my $oneNameAndAddress (@data) {
        addNameAndAddress(@{$oneNameAndAddress});
    }
    test::compare($table->rowCount, scalar(@data), "table contains as many rows as added data");
    closeWithoutSaving();
}

require 'names'
include Squish

def main
    startApplication("\"#{ENV['SQUISH_PREFIX']}/examples/qt/addressbook/addressbook\"")
    invokeMenuItem("File", "New")
    table = waitForObject({:type => "QTableWidget"})
    Test.verify(table.rowCount == 0)
    data = [["Andy", "Beach", "andy.beach@nowhere.com", "555 123 6786"],
        ["Candy", "Deane", "candy.deane@nowhere.com", "555 234 8765"],
        ["Ed", "Fernleaf", "ed.fernleaf@nowhere.com", "555 876 4654"]]
    data.each do |oneNameAndAddress|
        addNameAndAddress(oneNameAndAddress)
    end
    Test.compare(table.rowCount, data.length, "table contains as many rows as added data")
    closeWithoutSaving
end

source [findFile "scripts" "names.tcl"]

proc main {} {
    startApplication "\"$::env(SQUISH_PREFIX)/examples/qt/addressbook/addressbook\""
    set table [waitForObject [::Squish::ObjectName type QTableWidget]]
    test compare [property get $table rowCount] 0
    invokeMenuItem "File" "New"
    set data [list \
        [list "Andy" "Beach" "andy.beach@nowhere.com" "555 123 6786"] \
        [list "Candy" "Deane" "candy.deane@nowhere.com" "555 234 8765"] \
        [list "Ed" "Fernleaf" "ed.fernleaf@nowhere.com" "555 876 4654"] ]
    for {set i 0} {$i < [llength $data]} {incr i} {
        addNameAndAddress [lindex $data $i]
    }
    waitForObject $table
    test compare [property get $table rowCount] [llength $data] "table contains as many rows as added data"
    closeWithoutSaving
}

We begin by starting the application with a call to the ApplicationContext startApplication(autName) function. The name we pass as a string is the name registered with Squish (normally the name of the executable). Then we obtain a reference to the QTableWidget. The object name we used is a Real Name, containing key-value pairs.

The Object waitForObject(objectOrName) function waits until an object is ready (visible and enabled) and returns a reference to it—or it times out and raises a catchable exception. Once we have the table reference we can use it to access any of the QTableWidget's public methods and properties.

The invokeMenuItem function is one we have created specially for this test. It takes a menu name and a menu option name and invokes the menu option. It also uses real names to describe objects, and demonstrates how to parametrize values from variables in each script language. After using the invokeMenuItem function to do File > New, we verify that the table's row count is 0.

Next, we create some sample data and call a custom addNameAndAddress function to populate the table with the data using the AUT's Add dialog. Then we again compare the table's row count, this time to the number of rows in our sample data. And finally we call a custom closeWithoutSaving function to terminate the application.

We will now review each of the three supporting functions, so as to cover all the code in the tst_adding test case, starting with the invokeMenuItem function.

def invokeMenuItem(menu, item):
    activateItem(waitForObjectItem({"type": "QMenuBar"}, menu))
    activateItem(waitForObjectItem({'type': 'QMenu', 'title': menu}, item))

function invokeMenuItem(menu, item)
{
    activateItem(waitForObjectItem({"type": "QMenuBar"}, menu));
    activateItem(waitForObjectItem({"title": menu, "type": "QMenu"}, item));
}

sub invokeMenuItem
{
    my ($menu, $item) = @_;
    activateItem(waitForObjectItem({"type" => "QMenuBar"}, $menu));
    activateItem(waitForObjectItem({"title" => $menu, "type" => "QMenu"}, $item));
}

def invokeMenuItem(menu, item)
    activateItem(waitForObjectItem({:type => "QMenuBar"}, menu))
    activateItem(waitForObjectItem({:title => menu, :type => "QMenu"}, item))
end

proc invokeMenuItem {menu item} {
    invoke activateItem [waitForObjectItem [::Squish::ObjectName type QMenuBar] $menu]
    invoke activateItem [waitForObjectItem [::Squish::ObjectName title $menu type QMenu] $item]
}

As we mentioned earlier, the object names Squish uses for menus and menu items (and other objects) can vary depending on the context, and the context is partially derived from the window's title. For our reusable functions, we would like object names to match the desired objects regardless of the context or window title. Here, instead of reusing an existing symbolic name, we copied its real name and removed the properties we don't want to check for.

Every real name must specify the type property, and usually at least one other property. Here we've used the type to uniquely identify the menubar, and type and title properties to uniquely identify the menu. By using real names, we can create an object name that is general to enough match our desired objects regardless of the window title.

Once we have identified the object we want to interact with, we use the Object waitForObjectItem(objectOrName, itemOrIndex) function to retrieve a reference to it. Then we apply the activateItem(itemObject) function on it. The Object waitForObjectItem(objectOrName, itemOrIndex) function pauses Squish until the specified object and its item are visible and enabled. So, here, we waited for the menu bar and one of its menu bar items, and then we waited for a menu bar item and one of its menu items. And as soon as the waiting is over each time we activate the object and its item using the activateItem(itemObject) function.

def addNameAndAddress(oneNameAndAddress):
    invokeMenuItem("Edit", "Add...")
    type(waitForObject(names.forename_LineEdit), oneNameAndAddress[0])
    type(waitForObject(names.surname_LineEdit), oneNameAndAddress[1])
    type(waitForObject(names.email_LineEdit), oneNameAndAddress[2])
    type(waitForObject(names.phone_LineEdit), oneNameAndAddress[3])
    clickButton(waitForObject(names.address_Book_Add_OK_QPushButton))

function addNameAndAddress(oneNameAndAddress)
{
    invokeMenuItem("Edit", "Add...");
    type(waitForObject(names.forenameLineEdit), oneNameAndAddress[0]);
    type(waitForObject(names.surnameLineEdit), oneNameAndAddress[1]);
    type(waitForObject(names.emailLineEdit), oneNameAndAddress[2]);
    type(waitForObject(names.phoneLineEdit), oneNameAndAddress[3]);

    clickButton(waitForObject(names.addressBookAddOKQPushButton));
}

sub addNameAndAddress
{
    my(@oneNameAndAddress) = @_;
    invokeMenuItem("Edit", "Add...");
    type(waitForObject($Names::forename_lineedit), $_[0]);
    type(waitForObject($Names::surname_lineedit), $_[1]);
    type(waitForObject($Names::email_lineedit), $_[2]);
    type(waitForObject($Names::phone_lineedit), $_[3]);
    clickButton(waitForObject($Names::address_book_add_ok_qpushbutton));
}

def addNameAndAddress(oneNameAndAddress)
    invokeMenuItem("Edit", "Add...")
    type(waitForObject(Names::Forename_LineEdit), oneNameAndAddress[0])
    type(waitForObject(Names::Surname_LineEdit), oneNameAndAddress[1])
    type(waitForObject(Names::Email_LineEdit), oneNameAndAddress[2])
    type(waitForObject(Names::Phone_LineEdit), oneNameAndAddress[3])
    clickButton(waitForObject(Names::Address_Book_Add_OK_QPushButton))
end

proc addNameAndAddress {oneNameAndAddress} {
    invokeMenuItem "Edit" "Add..."
    invoke type [waitForObject $names::Forename_LineEdit] [lindex $oneNameAndAddress 0]
    invoke type [waitForObject $names::Surname_LineEdit] [lindex $oneNameAndAddress 1]
    invoke type [waitForObject $names::Email_LineEdit] [lindex $oneNameAndAddress 2]
    invoke type [waitForObject $names::Phone_LineEdit] [lindex $oneNameAndAddress 3]
    invoke clickButton [waitForObject $names::Address_Book_Add_OK_QPushButton]
}

For each set of name and address data we invoke the Edit > Add menu option to pop up the Add dialog. Then for each value received we populate the appropriate field by waiting for the relevant QLineEdit to be ready and then typing in the text using the type(objectOrName, text) function. At the end we click the dialog's OK button. The code from this function very similar to what was in the tst_general test case.

def closeWithoutSaving():
    sendEvent("QCloseEvent", waitForObject(names.mainWindow))
    clickButton(waitForObject(names.address_Book_No_QPushButton))

function closeWithoutSaving()
{
    sendEvent("QCloseEvent", waitForObject(names.mainWindow));
    clickButton(waitForObject(names.addressBookNoQPushButton));
}

sub closeWithoutSaving
{
    sendEvent( "QCloseEvent", waitForObject($Names::mainwindow) );
    clickButton(waitForObject($Names::address_book_no_qpushbutton));
}

def closeWithoutSaving
    sendEvent("QCloseEvent", waitForObject(Names::MainWindow))
    clickButton(waitForObject(Names::Address_Book_No_QPushButton))
end

proc closeWithoutSaving {} {
    sendEvent QCloseEvent [waitForObject $names::MainWindow]
    invoke clickButton [waitForObject $names::Address_Book_No_QPushButton]
}

Here we use the sendEvent function to simulate closing the main window.

Next, we click the Save unsaved changes? dialog's No button. The last line was copied from the recorded test.

The entire test is under 30 lines of code—and would be even less if we put some of the common functions (such as invokeMenuItem and closeWithoutSaving) in a shared script. And much of the code was copied directly from the recorded test, and in some cases parametrized.

This should be sufficient to give a flavor of writing test scripts for an AUT. Keep in mind that Squish provides far more functionality than we used here, (all of which is covered in the API Reference and the Tools Reference). Squish also provides access to the entire public APIs of the AUT's objects.

However, one aspect of the test case is not very satisfactory. Although embedding test data as we did here is sensible for small amounts, it is rather limiting, especially when we want to use a lot of test data. Also, we didn't test any of the data that was added to see if it correctly ended up in the QTableWidget. In the next section we will create a new version of this test, only this time we will pull in the data from an external data source, and check that the data we add to the QTableWidget is correct.

Creating Data Driven Tests

In the previous section we put three hard-coded names and addresses in our test. But what if we want to test lots of data? Or what if we want to change the data without having to change our test script's source code. One approach is to import a dataset into Squish and use the dataset as the source of the values we insert into our tests. Squish can import data in .tsv (tab-separated values format), .csv (comma-separated values format), .xls, or .xlsx (Microsoft Excel spreadsheet formats).

For the addressbook application we want to import the MyAddresses.tsv data file. To do this we must start by clicking File > Import Test Resource to pop-up the Import Squish Resource dialog. Inside the dialog, click the Browse button to choose the file to import—in this case MyAddresses.tsv. This file was already imported into the example suites and so copies of it can be found under shared/testdata of <SQUISHDIR>/examples/qt/addressbook/suite_xy. Make sure that the Import As combobox is set to "TestData". By default the Squish IDE will import the test data just for the current test case, but we want the test data to be available to all the test suite's test cases: to do this check the Copy to Test Suite for Sharing radio button. Now click the Finish button. You can now see the file listed in the Test Suite Resources view (in the Test Data tab), and if you click the file's name it will be shown in an Editor view. The screenshot shows Squish after the test data has been added.

Although in real life we would modify our tst_adding test case to use the test data, for the purpose of the tutorial we will make a new test case called tst_adding_data that is a copy of tst_adding and which we will modify to make use of the test data.

The only function we have to change is main, where instead of iterating over hard-coded items of data, we iterate over all the records in the dataset. We also need to update the expected row count at the end since we are adding a lot more records now, and we will also add a function to verify each record that's added.

import names
import os

def main():
    startApplication('"' + os.environ["SQUISH_PREFIX"] + '/examples/qt/addressbook/addressbook"')
    invokeMenuItem("File", "New")
    table = waitForObject({"type": "QTableWidget"})
    test.verify(table.rowCount == 0)
    limit = 10  # To avoid testing 100s of rows since that would be boring
    for row, record in enumerate(testData.dataset("MyAddresses.tsv")):
        forename = testData.field(record, "Forename")
        surname = testData.field(record, "Surname")
        email = testData.field(record, "Email")
        phone = testData.field(record, "Phone")
        table.setCurrentCell(0, 0)  # always insert at the start
        addNameAndAddress((forename, surname, email, phone))  # pass as a single tuple
        checkNameAndAddress(table, record)
        if row > limit:
            break
    test.compare(table.rowCount, row + 1, "table contains as many rows as added data")
    closeWithoutSaving()

import * as names from 'names.js';

function invokeMenuItem(menu, item)
{
    activateItem(waitForObjectItem({"type": "QMenuBar"}, menu));
    activateItem(waitForObjectItem({"title": menu, "type": "QMenu"}, item));
}

require 'names.pl';

sub main
{
    startApplication("\"$ENV{'SQUISH_PREFIX'}/examples/qt/addressbook/addressbook\"");
    invokeMenuItem("File", "New");
    my $table = waitForObject({"type" => "QTableWidget"});
    test::verify($table->rowCount == 0);
    my $limit = 10; # To avoid testing 100s of rows since that would be boring
    my @records = testData::dataset("MyAddresses.tsv");
    my $row = 0;
    for (; $row < scalar(@records); ++$row) {
        my $record = $records[$row];
        my $forename = testData::field($record, "Forename");
        my $surname = testData::field($record, "Surname");
        my $email = testData::field($record, "Email");
        my $phone = testData::field($record, "Phone");
        $table->setCurrentCell(0, 0); # always insert at the start
        addNameAndAddress($forename, $surname, $email, $phone);
        checkNameAndAddress($table, $record);
        if ($row > $limit) {
            last;
        }
    }
    test::verify($table->rowCount == $row + 1);
    closeWithoutSaving();
}

require 'names'
include Squish

def main
    startApplication("\"#{ENV['SQUISH_PREFIX']}/examples/qt/addressbook/addressbook\"")
    invokeMenuItem("File", "New")
    table = waitForObject({:type => "QTableWidget"})
    Test.verify(table.rowCount == 0)
    limit = 10 # To avoid testing 100s of rows since that would be boring
    rows = 0
    TestData.dataset("MyAddresses.tsv").each_with_index do
        |record, row|
        forename = TestData.field(record, "Forename")
        surname = TestData.field(record, "Surname")
        email = TestData.field(record, "Email")
        phone = TestData.field(record, "Phone")
        table.setCurrentCell(0, 0) # always insert at the start
        addNameAndAddress([forename, surname, email, phone]) # pass as a single Array
        checkNameAndAddress(table, record)
        break if row > limit
        rows += 1
    end
    Test.compare(table.rowCount, rows + 1, "table contains as many rows as added data")
    closeWithoutSaving
end

source [findFile "scripts" "names.tcl"]

proc main {} {
    startApplication "\"$::env(SQUISH_PREFIX)/examples/qt/addressbook/addressbook\""
    invokeMenuItem "File" "New"
    set table [waitForObject [::Squish::ObjectName type QTableWidget]]
    test compare [property get $table rowCount] 0
    # To avoid testing 100s of rows since that would be boring
    set limit 10
    set data [testData dataset "MyAddresses.tsv"]
    set columns [llength [testData fieldNames [lindex $data 0]]]
    set row 0
    for {} {$row < [llength $data]} {incr row} {
        set record [lindex $data $row]
        set forename [testData field $record "Forename"]
        set surname [testData field $record "Surname"]
        set email [testData field $record "Email"]
        set phone [testData field $record "Phone"]
        set details [list $forename $surname $email $phone]
        invoke $table setCurrentCell 0 0
        addNameAndAddress $details
        checkNameAndAddress $table $record
        if {$row > $limit} {
            break
        }
    }
    test compare [property get $table rowCount] [expr $row + 1]
    closeWithoutSaving
}

Squish provides access to test data through its testData module's functions—here we used the Dataset testData.dataset(filename) function to access the data file and make its records available, and the String testData.field(record, fieldName) function to retrieve each record's individual fields.

Having used the test data to populate the QTableView we want to be confident that the data in the table is the same as what we have added, so that's why we added the checkNameAndAddress function. We also added a limit to how many records we would compare, just to make the test run faster.

def checkNameAndAddress(table, record):
    for column in range(len(testData.fieldNames(record))):
        test.compare(table.item(0, column).text(),  # New addresses are inserted at the start
                     testData.field(record, column))

function checkNameAndAddress(table, record)
{
    for (var column = 0; column < testData.fieldNames(record).length; ++column)
        test.compare(table.item(0, column).text(), // New addresses are inserted at the start
                     testData.field(record, column));
}

sub checkNameAndAddress
{
    my($table, $record) = @_;
    my @columnNames = testData::fieldNames($record);
    for (my $column = 0; $column < scalar(@columnNames); $column++) {
        test::compare($table->item(0, $column)->text(), # New addresses are inserted at the start
                      testData::field($record, $column));
    }
}

def checkNameAndAddress(table, record)
    for column in 0...TestData.fieldNames(record).length
        Test.compare(table.item(0, column).text(),
                     TestData.field(record, column))
        # New addresses are inserted at the start
    end
end

proc checkNameAndAddress {table record} {
    set columns [llength [testData fieldNames $record]]
    for {set column 0} {$column < $columns} {incr column} {
        set value [invoke [invoke $table item 0 $column] text]
        test compare $value [testData field $record $column]
    }
}

This function accesses the QTableWidget's first row and extracts each of its columns' values. We use Squish's SequenceOfStrings testData.fieldNames(record) function to get a column count and then use the test.compare function to check that each value in the table is the same as the value in the test data we used. Note that for this particular test we always insert new rows at the start of the table. The effect of this is that every new name and address is always added as the first row, so this is why we hard-coded the row to be 0.

The screenshot shows Squish's Test Summary log after the data-driven tests have been run.

Learning More

We have now completed the tutorial. Squish can do much more than we have shown here, but the aim has been to get you started with basic testing as quickly and easily as possible. The How to Create Test Scripts, and How to Test Applications - Specifics sections provide many more examples, including those that show how tests can interact with particular input elements, such as selects, select-ones, texts, and text-areas.

The API Reference and Tools Reference give full details of Squish's testing API and the numerous functions it offers to make testing as easy and efficient as possible. It is well worth reading the How to Create Test Scripts and How to Test Applications - Specifics, as well as browsing the API Reference and Tools Reference. The time you invested will be repaid because you'll know what functionality Squish provides out of the box and can avoid reinventing things that are already available.

The key Qt examples with links to the places they are used are given below.

• The Payment Form example (How to Test Stateful and Single-Valued Widgets) shows how to test single-valued Qt widgets such as: QCheckBox, QComboBox, QDateEdit, QLineEdit, QPushButton, QRadioButton, and QSpinBox.
• The Item Views example (How to Test Items in Item Views, Item Widgets, and Models) shows how to test Qt's multi-item widgets and views, including: QListWidget and QListView, QTableWidget and QTableView, QTreeWidget and QTreeView, as well as the underlying Qt models that hold the data for the views.
• The CsvTable example (How to Test Table Widgets and Use External Data Files) shows how to do data-driven testing using external data files.

In addition to the documented examples listed above, further Qt example applications and their corresponding tests are provided in <SQUISHDIR>/examples/qt.

Tutorial: Designing Behavior Driven Development (BDD) Tests

This tutorial will show you how to create, run, and modify Behavior Driven Development (BDD) tests for an example application. You will learn about Squish's most frequently used features. By the end of the tutorial you will be able to write your own tests for your own applications.

For this chapter we will use a simple Address Book application as our AUT. This is a very basic application that allows users to load an existing address book or create a new one, add, edit, and remove entries. The screenshot shows the application in action with a user adding a new name and address.

Introduction to Behavior Driven Development

Behavior-Driven Development (BDD) is an extension of the Test-Driven Development approach which puts the definition of acceptance criteria at the beginning of the development process as opposed to writing tests after the software has been developed. With possible cycles of code changes done after testing.

Behavior Driven Tests are built out of a set of Feature files, which describe product features through the expected application behavior in one or many Scenarios. Each Scenario is built out of a sequence of Steps which represent actions or verifications that need to be tested for that Scenario.

BDD focuses on expected application behavior, not on implementation details. Therefore BDD tests are described in a human-readable Domain Specific Language (DSL). As this language is not technical, such tests can be created not only by programmers, but also by product owners, testers or business analysts. Additionally, during the product development, such tests serve as living product documentation. For Squish usage, BDD tests shall be created using Gherkin syntax. The previously written product specification (BDD tests) can be turned into executable tests. This step by step tutorial presents automating BDD tests with Squish IDE support.

Gherkin syntax

Gherkin files describe product features through the expected application behavior in one or many Scenarios. Here is an example showing the "Filling of addressbook" feature of the addressbook example application.

Feature: Filling of addressbook
    As a user I want to fill the addressbook with entries

    Scenario: Initial state of created address book
        Given addressbook application is running
        When I create a new addressbook
        Then addressbook should have zero entries

    Scenario: State after adding one entry
        Given addressbook application is running
        When I create a new addressbook
        And I add a new person 'John','Doe','john@m.com','500600700' to address book
        Then '1' entries should be present

    Scenario: State after adding two entries
        Given addressbook application is running
        When I create a new addressbook
        And I add new persons to address book
            | forname   | surname  | email        | phone  |
            | John      | Smith    | john@m.com   | 123123 |
            | Alice     | Thomson  | alice@m.com  | 234234 |
        Then '2' entries should be present

    Scenario: Forename and surname is added to table
        Given addressbook application is running
        When I create a new addressbook
        When I add a new person 'Bob','Doe','Bob@m.com','123321231' to address book
        Then previously entered forename and surname shall be at the top

Most of the above is free form text (does not have to be English). It's just the Feature/Scenario structure and the leading keywords like "Given", "And", "When" and "Then" that are fixed. Each of those keywords marks a Step defining preconditions, user actions and expected results. This application behavior specification can be passed to software developers to implement features, and at the same time it can be passed to software testers to implement automated tests.

Test implementation

Creating Test Suite

First, we need to create a Test Suite, which is a container for all Test Cases. Start the Squish IDE and select File > New Test Suite. Follow the New Test Suite wizard, provide a Test Suite name, choose the Qt Toolkit and scripting language of your choice and finally register Address Book application as AUT (if necessary). See Creating a Test Suite for more details about creating new Test Suites.

Creating Test Case

Squish supports different types of Test Cases: "Script", "BDD" and "MBT". "Script" is the default. In order to create new "BDD Test Case" we need to use the context menu by clicking on the expander next to New Script Test Case () button and choosing the option New BDD Test Case. The Squish IDE will remember your choice and the "BDD Test Case" will become the default when clicking on the button in the future.

The new BDD Test Case consists of a test.feature file (filled with a Gherkin template while creating a new BDD test case), a file named test.(py|js|pl|rb|tcl) which will drive the execution (there is no need to edit this file), and a Test Suite Resources file named shared/steps/steps.(py|js|pl|rb|tcl) where step implementation code will be placed.

We need to replace the Gherkin template with a Feature for the addressbook example application. To do this, copy the Feature description below and paste it into the Feature file.

Feature: Filling of addressbook
    As a user I want to fill the addressbook with entries

    Scenario: Initial state of created address book
        Given addressbook application is running
        When I create a new addressbook
        Then addressbook should have zero entries

When editing the test.feature file, a Feature file warning No implementation found is displayed for each undefined step. The implementations are in the steps subdirectory, in Test Case Resources, or in Test Suite Resources. Running our Feature test now will currently fail at the first step with a No Matching Step Definition and the following steps will be skipped.

Recording Step implementation

In order to record the Scenario, press the Record () button next to the respective Scenario that is listed in the Scenarios tab in Test Case Resources view.

This will cause Squish to run the AUT so that we can interact with it. Additionally, the Control Bar is displayed with a list of all steps that need to be recorded. Now all interaction with the AUT or any verification points added to the script will be recorded under the first step Given addressbook application is running (which is bolded in the Step list on the Control Bar). In order to verify that this precondition is met, we will add a Verification Point. To do this, click on Verify () in the Control Bar and select Properties.

As a result, the Squish IDE is put into Spy mode which displays all application objects and their properties. In the Application Objects view, select (don't check) the addressbook MainWindow item. Selecting it will update the Properties view to its right. Next, click on the checkbox in front of the enabled property in the Properties View. Finally, click on the button Save and Insert Verifications. The Squish IDE disappears and the Control Bar is shown again.

When we are done with each step, we can move to the next undefined step (playing back the ones that were previously defined) by clicking on the Finish Recording Step() arrow button in the Control Bar that is located to the left of the current step.

Next, for the step When I create a new addressbook, click on the New button () in the toolbar of the addressbook application and click on Finish Recording Step() to move on to the next step.

Finally, for the step Then addressbook should have zero entries verify that the table containing the address entries is empty. To record this verification, click on Verify () while recording, and select Properties. In the Application Objects, navigate or use the Object Picker () to select (not check) the table object containing the address book entries (in our case this table is empty). Then, check the rowCount property from the Properties View, and click Save and Insert Verifications. Finally, click on the last Finish Recording Step() button.

As a result, Squish will generate the following step definitions in the steps.* file (at Test Suites > Test Suite Resources):

@Given("addressbook application is running")
def step(context):
    startApplication('"' + os.environ["SQUISH_PREFIX"] + '/examples/qt/addressbook/addressbook"')
    test.compare(waitForObjectExists(names.address_Book_MainWindow).enabled, True)

@Step("I create a new addressbook")
def step(context):
    clickButton(waitForObject(names.address_Book_New_QToolButton))

@Then("addressbook should have zero entries")
def step(context):
    test.compare(waitForObjectExists(names.address_Book_Unnamed_File_QTableWidget).rowCount, 0)

Given("addressbook application is running") do |context|
    startApplication("\"#{ENV['SQUISH_PREFIX']}/examples/qt/addressbook/addressbook\"")
    Test.compare(waitForObjectExists(Names::Address_Book_MainWindow).enabled, true)
end

When("I create a new addressbook") do |context|
    clickButton(waitForObject(Names::Address_Book_New_QToolButton))
end

Then("addressbook should have zero entries") do |context|
    Test.compare(waitForObjectExists(Names::Address_Book_Unnamed_File_QTableWidget).rowCount, 0)
end

Given("addressbook application is running", function(context) {
    startApplication('"' + OS.getenv("SQUISH_PREFIX") + '/examples/qt/addressbook/addressbook"');
    test.compare(waitForObjectExists(names.addressBookMainWindow).enabled, true);
});

When("I create a new addressbook", function(context) {
    clickButton(waitForObject(names.addressBookNewQToolButton));
});

Then("addressbook should have zero entries", function(context) {
    test.compare(waitForObjectExists(names.addressBookUnnamedFileQTableWidget).rowCount, 0);
});

Given("addressbook application is running", sub {
    my $context = shift;
    startApplication("\"$ENV{'SQUISH_PREFIX'}/examples/qt/addressbook/addressbook\"");
    test::compare(waitForObjectExists($Names::address_book_mainwindow)->enabled, 1 );
});

When("I create a new addressbook", sub {
    my $context = shift;
    clickButton(waitForObject($Names::address_book_new_qtoolbutton) );
});

Then("addressbook should have zero entries", sub {
    my $context = shift;
    test::compare(waitForObjectExists($Names::address_book_unnamed_file_qtablewidget)->rowCount,0);
});

Given "addressbook application is running" {context} {
    startApplication "\"$::env(SQUISH_PREFIX)/examples/qt/addressbook/addressbook\""
    test compare [property get [waitForObjectExists $names::Address_Book_MainWindow] enabled] true
}

When "I create a new addressbook" {context} {
    invoke clickButton [waitForObject $names::Address_Book_New_QToolButton]
}

Then "addressbook should have zero entries" {context} {
    test compare [property get [waitForObjectExists $names::Address_Book_Unnamed_File_QTableWidget] rowCount] 0
}

The application is started at the beginning of the first step due to the startApplication() call. At the end of each Scenario, the OnScenarioEnd hook function is called, causing detach() to be called on the application context. Because the AUT was started with startApplication(), this causes it to terminate.

This hook function is found in the file bdd_hooks.(py|js|pl|rb|tcl), which is located in the Scripts tab of the Test Suite Resources view. You can define additional hook functions here. For a list of all available hooks, please refer to Performing Actions During Test Execution Via Hooks.

@OnScenarioEnd
def OnScenarioEnd(context):
    for ctx in applicationContextList():
        ctx.detach()

OnScenarioEnd(function(context) {
    applicationContextList().forEach(function(ctx) { ctx.detach(); });
});

OnScenarioEnd(sub {
    foreach (applicationContextList()) {
        $_->detach();
    }
});

OnScenarioEnd do |context|
    applicationContextList().each { |ctx| ctx.detach() }
end

OnScenarioEnd {context} {
    foreach ctx [applicationContextList] {
        applicationContext $ctx detach
    }
}

Step parametrization

So far, our steps did not use any parameters and all values were hardcoded. Gherkin supports different types of parameters like any, integer or word, allowing our step definitions to be more reusable. Let us add a new Scenario to our Feature file which will provide step parameters for both the Test Data and the expected results. Copy the below section into your Feature file.

Scenario: State after adding one entry
    Given addressbook application is running
    When I create a new addressbook
    And I add a new person 'John','Doe','john@m.com','500600700' to address book
    Then '1' entries should be present

After auto-saving the Feature file, the Squish IDE provides a hint that only 2 steps need to be implemented: When I add a new person 'John', 'Doe','john@m.com','500600700' to address book and Then '1' entries should be present. The remaining steps already have a matching step implementation.

To record the missing steps, hit the Record () button next to the test case name in the Test Suites view. The script will play until it gets to the missing step and then prompt you to implement it. At this point, Squish is recording that step.

Select the Add button, then type in the information for a new entry. Click on the Finish Recording Step() to move to the next step.

For the second missing step, we could record an object property verification like we did with the step Then addressbook should have zero entries. Or we could copy that step's implementation in the steps.(py|js|pl|rb|tcl) file and increment the number at the end of the test.compare line. Instead of testing for zero items, we are testing for one item.

Next, we parametrize the generated When step implementation by replacing the values with parameter types. Since we want to be able to add different names, replace 'John' with '|word|'. Note that each parameter will be passed to the step implementation function in the order of appearance in the descriptive name of the step. Finish parametrizing by editing the typed values into keywords, looking like this example step: When I add a new person 'John', 'Doe','john@m.com','500600700'

@When("I add a new person '|word|','|word|','|any|','|integer|' to address book")
def step(context, forename, surname, email, phone):
    clickButton(waitForObject(names.address_Book_Add_QToolButton))
    type(waitForObject(names.forename_LineEdit), forename)
    type(waitForObject(names.surname_LineEdit), surname)
    type(waitForObject(names.email_LineEdit), email)
    type(waitForObject(names.phone_LineEdit), phone)
    clickButton(waitForObject(names.address_Book_Add_OK_QPushButton))

When("I add a new person '|word|','|word|','|any|','|integer|' to address book",
    function (context, forename, surname, email, phone){
        clickButton(waitForObject(names.addressBookUnnamedAddQToolButton));
        type(waitForObject(names.forenameLineEdit), forename);
        type(waitForObject(names.surnameLineEdit), surname);
        type(waitForObject(names.emailLineEdit), email);
        type(waitForObject(names.phoneLineEdit), phone);
        clickButton(waitForObject(names.addressBookAddOKQPushButton));
        context.userData["forename"] = forename;
        context.userData["surname"] = surname;
});

When("I add a new person '|word|','|word|','|any|','|integer|' to address book", sub {
    my $context = shift;
    my ($forename, $surname, $email, $phone) = @_;
    clickButton(waitForObject($Names::address_book_unnamed_add_qtoolbutton));
    type(waitForObject($Names::forename_lineedit), $forename);
    type(waitForObject($Names::surname_lineedit),  $surname);
    type(waitForObject($Names::email_lineedit),    $email);
    type(waitForObject($Names::phone_lineedit),    $phone);
    clickButton(waitForObject($Names::address_book_add_ok_qpushbutton));
    $context->{userData}{'forename'} = $forename;
    $context->{userData}{'surname'} = $surname;
});

When("I add a new person '|word|','|word|','|any|','|integer|' to address book") do |context, forename, surname, email, phone|
    clickButton(waitForObject(Names::Address_Book_Unnamed_Add_QToolButton))
    type(waitForObject(Names::Forename_LineEdit), forename)
    type(waitForObject(Names::Surname_LineEdit), surname)
    type(waitForObject(Names::Email_LineEdit), email)
    type(waitForObject(Names::Phone_LineEdit), phone)
    clickButton(waitForObject(Names::Address_Book_Add_OK_QPushButton))
    context.userData = Hash.new
    context.userData[:forename] = forename
    context.userData[:surname] = surname
end

When "I add a new person '|word|','|word|','|any|','|integer|' to address book" {context forename surname email phone} {
    invoke clickButton [waitForObject $names::Address_Book_Unnamed_Add_QToolButton]
    invoke type [waitForObject $names::Forename_LineEdit] $forename
    invoke type [waitForObject $names::Surname_LineEdit] $surname
    invoke type [waitForObject $names::Email_LineEdit] $email
    invoke type [waitForObject $names::Phone_LineEdit] $phone
    invoke clickButton [waitForObject $names::Address_Book_Add_OK_QPushButton]
    $context userData [dict create forename $forename surname $surname]
}

If we recorded the final Then as a missing step, and verified rowCount is 1 in the table, we can modify the step so that it takes a parameter, so it can verify other integer values later.

@Then("'|integer|' entries should be present")
def step(context, num):
    snooze(.25)
    test.compare(waitForObjectExists(names.address_Book_Unnamed_File_QTableWidget).rowCount, num)

Then("'|integer|' entries should be present", function(context, rowCount) {
    snooze(0.25)
    test.compare(waitForObjectExists(names.addressBookUnnamedFileQTableWidget).rowCount, rowCount);
});

Then("'|integer|' entries should be present", sub {
    my $context = shift;
    my $num = shift;
    test::compare(waitForObjectExists($Names::address_book_unnamed_file_qtablewidget)->rowCount, $num);
});

Then("'|integer|' entries should be present") do |context, num|
    Test.compare(waitForObjectExists(Names::Address_Book_Unnamed_File_QTableWidget).rowCount, num)
end

Then "'|integer|' entries should be present" {context num} {
   test compare [property get [waitForObjectExists $names::Address_Book_Unnamed_File_QTableWidget] rowCount] $num
}

Provide parameters for Step in table

The next Scenario will test adding multiple entries to the address book. We could use step When I add a new person John','Doe','john@m.com','500600700' to address book multiple times just with different data. But lets instead define a new step called When I add a new person to address book which will handle data from a table.

Scenario: State after adding two entries
    Given addressbook application is running
    When I create a new addressbook
    And I add new persons to address book
        | forename  | surname  | email        | phone  |
        | John      | Smith    | john@m.com   | 123123 |
        | Alice     | Thomson  | alice@m.com  | 234234 |
    Then '2' entries should be present

The step implementation to handle such tables looks like this:

@Step("I add new persons to address book")
def step(context):
    table = context.table
    # Drop initial row with column headers
    table.pop(0)
    for (forname, surname, email, phone) in table:
        clickButton(waitForObject(names.address_Book_Add_QToolButton))
        snooze(.5) # workaround for qt4
        type(waitForObject(names.forename_LineEdit), forname)
        type(waitForObject(names.surname_LineEdit), surname)
        type(waitForObject(names.email_LineEdit), email)
        type(waitForObject(names.phone_LineEdit), phone)
        clickButton(waitForObject(names.address_Book_Add_OK_QPushButton))
        test.log("Added entry: "+forname+","+surname+","+email+","+phone)

When("I add new persons to address book", function(context) {
    var table = context.table;
    // Drop initial row with column headers
    for (var i = 1; i < table.length; ++i) {
        var forename = table[i][0];
        var surname = table[i][1];
        var email = table[i][2];
        var phone = table[i][3];
        clickButton(waitForObject(names.addressBookUnnamedAddQToolButton))
        snooze(0.5) // workaround for qt4
        type(waitForObject(names.forenameLineEdit), forename)
        type(waitForObject(names.surnameLineEdit), surname)
        type(waitForObject(names.emailLineEdit), email)
        type(waitForObject(names.phoneLineEdit), phone)
        clickButton(waitForObject(names.addressBookAddOKQPushButton))
    }
});

When("I add new persons to address book", sub {
    my %context = %{shift()};
    my @table = @{$context{'table'}};
    # Drop initial row with column headers
    shift(@table);
    for my $row (@table) {
        my ($forename, $surname, $email, $phone) = @{$row};
        clickButton( waitForObject($Names::address_book_unnamed_add_qtoolbutton) );
        snooze(0.5); # workaround for qt4
        type( waitForObject($Names::forename_lineedit), $forename );
        type( waitForObject($Names::surname_lineedit),  $surname );
        type( waitForObject($Names::email_lineedit),    $email );
        type( waitForObject($Names::phone_lineedit),    $phone );
        clickButton( waitForObject($Names::address_book_add_ok_qpushbutton) );
    }
});

When("I add new persons to address book") do |context|
    table = context.table
    # Drop initial row with column headers
    table.shift
    for forename, surname, email, phone in table do
        clickButton(waitForObject(Names::Address_Book_Unnamed_Add_QToolButton))
        snooze(0.5) # qt4 workaround
        type(waitForObject(Names::Forename_LineEdit), forename)
        type(waitForObject(Names::Surname_LineEdit), surname)
        type(waitForObject(Names::Email_LineEdit), email)
        type(waitForObject(Names::Phone_LineEdit), phone)
        clickButton(waitForObject(Names::Address_Book_Add_OK_QPushButton))
    end
end

When "I add new persons to address book" {context} {
    set table [$context table]
    # Drop initial row with column headers
    foreach row [lreplace $table 0 0] {
        foreach {forename surname email phone} $row break
        invoke clickButton [waitForObject $names::Address_Book_Unnamed_Add_QToolButton]
        # qt4 workaround:
        snooze 0.5
        invoke type [waitForObject $names::Forename_LineEdit] $forename
        invoke type [waitForObject $names::Surname_LineEdit] $surname
        invoke type [waitForObject $names::Email_LineEdit] $email
        invoke type [waitForObject $names::Phone_LineEdit] $phone
        invoke clickButton [waitForObject $names::Address_Book_Add_OK_QPushButton]
    }
}

Sharing data between Steps and Scenarios

Lets add a new Scenario to the Feature file. This time we would like to check not the number of entries in address book list, but if this list contains proper data. Because we enter data into the address book in one step and verify them in another, we must share information about entered data among those steps in order to perform a verification.

Scenario: Forename and surname is added to table
    Given addressbook application is running
    When I create a new addressbook
    When I add a new person 'Bob','Doe','Bob@m.com','123321231' to address book
    Then previously entered forename and surname shall be at the top

To share this data, the context.userData property can be used.

@When("I add a new person '|word|','|word|','|any|','|integer|' to address book")
def step(context, forename, surname, email, phone):
    clickButton(waitForObject(names.address_Book_Add_QToolButton))
    type(waitForObject(names.forename_LineEdit), forename)
    type(waitForObject(names.surname_LineEdit), surname)
    type(waitForObject(names.email_LineEdit), email)
    type(waitForObject(names.phone_LineEdit), phone)
    clickButton(waitForObject(names.address_Book_Add_OK_QPushButton))
    context.userData = {}
    context.userData['forename'] = forename
    context.userData['surname'] = surname

When("I add a new person '|word|','|word|','|any|','|integer|' to address book",
    function (context, forename, surname, email, phone){
        clickButton(waitForObject(names.addressBookUnnamedAddQToolButton));
        type(waitForObject(names.forenameLineEdit), forename);
        type(waitForObject(names.surnameLineEdit), surname);
        type(waitForObject(names.emailLineEdit), email);
        type(waitForObject(names.phoneLineEdit), phone);
        clickButton(waitForObject(names.addressBookAddOKQPushButton));
        context.userData["forename"] = forename;
        context.userData["surname"] = surname;
});

When("I add a new person '|word|','|word|','|any|','|integer|' to address book", sub {
    my $context = shift;
    my ($forename, $surname, $email, $phone) = @_;
    clickButton(waitForObject($Names::address_book_unnamed_add_qtoolbutton));
    type(waitForObject($Names::forename_lineedit), $forename);
    type(waitForObject($Names::surname_lineedit),  $surname);
    type(waitForObject($Names::email_lineedit),    $email);
    type(waitForObject($Names::phone_lineedit),    $phone);
    clickButton(waitForObject($Names::address_book_add_ok_qpushbutton));
    $context->{userData}{'forename'} = $forename;
    $context->{userData}{'surname'} = $surname;
});

When("I add a new person '|word|','|word|','|any|','|integer|' to address book") do |context, forename, surname, email, phone|
    clickButton(waitForObject(Names::Address_Book_Unnamed_Add_QToolButton))
    type(waitForObject(Names::Forename_LineEdit), forename)
    type(waitForObject(Names::Surname_LineEdit), surname)
    type(waitForObject(Names::Email_LineEdit), email)
    type(waitForObject(Names::Phone_LineEdit), phone)
    clickButton(waitForObject(Names::Address_Book_Add_OK_QPushButton))
    context.userData = Hash.new
    context.userData[:forename] = forename
    context.userData[:surname] = surname
end

When "I add a new person '|word|','|word|','|any|','|integer|' to address book" {context forename surname email phone} {
    invoke clickButton [waitForObject $names::Address_Book_Unnamed_Add_QToolButton]
    invoke type [waitForObject $names::Forename_LineEdit] $forename
    invoke type [waitForObject $names::Surname_LineEdit] $surname
    invoke type [waitForObject $names::Email_LineEdit] $email
    invoke type [waitForObject $names::Phone_LineEdit] $phone
    invoke clickButton [waitForObject $names::Address_Book_Add_OK_QPushButton]
    $context userData [dict create forename $forename surname $surname]
}

All data stored in context.userData can be accessed in all steps and Hooks of all Scenarios of the given Feature.

Finally, we need to implement the step Then previously entered forename and lastname shall be at the top. Before this, record a snippet that opens the addressbook and clicks on the first 2 cells of the first row of the table so that we have entries in our object map for them.

@Then("previously entered forename and surname shall be at the top")
def step(context):
    test.compare(waitForObjectExists(names.file_0_0_QModelIndex).text,context.userData['forename'])
    test.compare(waitForObjectExists(names.file_0_1_QModelIndex).text,context.userData['surname'])

Then("previously entered forename and surname shall be at the top",function(context){
    test.compare(waitForObjectExists(names.file00QModelIndex).text,context.userData["forename"], "forname?");
    test.compare(waitForObjectExists(names.file01QModelIndex).text,context.userData["surname"], "surname?");
});

Then("previously entered forename and surname shall be at the top", sub {
    my $context = shift;
    test::compare(waitForObjectExists($Names::file_0_0_qmodelindex)->text,  $context->{userData}{'forename'}, "forename?" );
    test::compare(waitForObjectExists($Names::file_0_1_qmodelindex)->text,  $context->{userData}{'surname'}, "surname?" );
});

Then("previously entered forename and surname shall be at the top") do |context|
    Test.compare(waitForObjectExists(Names::File_0_0_QModelIndex).text, context.userData[:forename], "forename?")
    Test.compare(waitForObjectExists(Names::File_0_1_QModelIndex).text, context.userData[:surname], "surname?")
end

Then "previously entered forename and surname shall be at the top" {context} {
    test compare [property get [waitForObjectExists $names::File_0_0_QModelIndex] text] [dict get [$context userData] forename]
    test compare [property get [waitForObjectExists $names::File_0_1_QModelIndex] text] [dict get [$context userData] surname]
}

By recording a snippet that verifies the text property of the table cells in row 0, columns 0 and 1, we get scriptified verification points on QModelIndex objects. We replaced the actual values in the snippet with the stored context.userData values from the previous step.

Scenario Outline

Assume our Feature contains the following two Scenarios:

Scenario: State after adding one entry
    Given addressbook application is running
    When I create a new addressbook
    And I add a new person 'John','Doe','john@m.com','500600700' to address book
    Then '1' entries should be present

Scenario: State after adding one entry
    Given addressbook application is running
    When I create a new addressbook
    And I add a new person 'Bob','Koo','bob@m.com','500600800' to address book
    Then '1' entries should be present

As we can see, these Scenarios perform the same actions using different test data. Something similar can be achieved by using a Scenario Outline (a Scenario template with placeholders) and Examples (a table with parameters).

Scenario Outline: Adding single entries multiple time
   Given addressbook application is running
   When I create a new addressbook
   And I add a new person '<forename>','<lastname>','<email>','<phone>' to address book
   Then '1' entries should be present
   Examples:
      | forename | lastname | email       | phone     |
      | John     | Doe      | john@m.com  | 500600700 |
      | Bob      | Koo      | bob@m.com   | 500600800 |

Test execution

In the Squish IDE, users can execute all Scenarios in a Feature, or execute only one selected Scenario. In order to execute all Scenarios, the proper Test Case has to be executed by clicking on the Run Test Case () button in the Test Suites view.

In order to execute only one Scenario, you need to open the Feature file, right-click on the given Scenario and choose Run Scenario. An alternative approach is to click on the Run Test Case () button while hovering over the respective Scenario in the Scenarios tab in Test Case Resources.

After a Scenario is executed, the Feature file is colored according to the execution results. More detailed information (like logs) can be found in the Test Results view.

Test debugging

Squish offers the possibility to pause an execution of a Test Case at any point in order to check script variables, spy application objects or run custom code in the Squish Script Console. To do this, a breakpoint has to be placed before starting the execution, either in the Feature file at any line containing a step or at any line of executed code (i.e., in the middle of step definition code).

After the breakpoint is reached, you can inspect all application objects and their properties. If a breakpoint is placed at a step definition or a hook is reached, then you can additionally add Verification Points or record code snippets.

Re-using Step definitions

BDD test maintainability can be increased by reusing step definitions in test cases located in another directory. For more information, see collectStepDefinitions().

Tutorial: Migration of existing tests to BDD

This chapter is for users that have existing Squish tests and who would like to introduce Behavior Driven Testing. The first section describes how to keep the existing tests and just create new tests with the BDD approach. The second section describes how to convert Script Test Cases into BDD tests.

Extend existing tests to BDD

The first option is to keep any existing Squish tests and extend them by adding new BDD tests. It's possible to have a Test Suite containing both Script Test Cases and BDD Test Cases. Simply open existing Test Suite with test cases and choose New BDD Test Case option from drop down menu.

Assuming your existing Test Cases make use of a library and you are calling shared functions to interact with the AUT, those functions can be used in newly created BDD Test Cases also. In the example below, a function is used from multiple Script Test Cases:

def createNewAddressBook():
    clickButton(waitForObject(names.address_Book_New_QToolButton))

function createNewAddressBook(){
     clickButton(waitForObject(names.addressBookNewQToolButton));
}

sub createNewAddressBook{
        clickButton(waitForObject($Names::address_book_new_qtoolbutton));
}

def createNewAddressBook
    clickButton(waitForObject(Names::Address_Book_New_QToolButton))
end

proc createNewAddressBook {} {
    invoke clickButton [waitForObject $names::Address_Book_New_QToolButton]
}

Different BDD Test Cases can use that function:

@When("I create a new addressbook")
def step(context):
    createNewAddressBook()

When("I create a new addressbook",function(context){
    createNewAddressBook()
});

When("I create a new addressbook", sub {
    createNewAddressBook();
});

When("I create a new addressbook") do |context|
  createNewAddressBook
end

When "I create a new addressbook" {context} {
    createNewAddressBook
}

Convert existing tests to BDD

The second option is to convert an existing Test Suite that contains Script Test Cases into behavior driven tests. Since a Test Suite can contain Script Test Cases and also BDD Test Cases, migration can be done gradually. A Test Suite containing a mix of both Test Case types can be executed and results analyzed without any extra effort required.

The first step is to review all Test Cases of the existing Test Suite and group them by the Feature they test. Each Script Test Case will be transformed into a Scenario, which is a part of a Feature. For example, assume we have 5 Script Test Cases. After review, we realize that those Script Test Cases examine two Features. Therefore, when migration is completed, our Test Suite will contain two BDD Test Cases, each of them containing one Feature. Each Feature will contain multiple Scenarios. In our example, the first Feature contains three Scenarios and the second Feature contains two Scenarios.

First, open a Test Suite in the Squish IDE that contains Script Squish tests that are planned to be migrated to BDD tests. Next, create a New Test Case by choosing New BDD Test Case option from its drop-down menu. Each BDD Test Case contains a test.feature file that can be filled with maximum one Feature. Next, open the test.feature file to describe the Features using the Gherkin language. Following the syntax from the template, edit the Feature name and optionally provide a short description. Next, analyze which actions and verifications are performed in the Script Test Case that need to be migrated. This is how an example Test Case for the addressbook application might look:

import names
import os

def main():
    startApplication('"' + os.environ["SQUISH_PREFIX"] + '/examples/qt/addressbook/addressbook"')
    invokeMenuItem("File", "New")
    table = waitForObject({"type": "QTableWidget"})
    test.verify(table.rowCount == 0)
    data = [("Andy", "Beach", "andy.beach@nowhere.com", "555 123 6786"),
            ("Candy", "Deane", "candy.deane@nowhere.com", "555 234 8765"),
            ("Ed", "Fernleaf", "ed.fernleaf@nowhere.com", "555 876 4654")]
    for oneNameAndAddress in data:
        addNameAndAddress(oneNameAndAddress)
    waitForObject(table)
    test.compare(table.rowCount, len(data), "table contains as many rows as added data")
    closeWithoutSaving()

import * as names from 'names.js';

function invokeMenuItem(menu, item)
{
    activateItem(waitForObjectItem({"type": "QMenuBar"}, menu));
    activateItem(waitForObjectItem({"title": menu, "type": "QMenu"}, item));
}

require 'names.pl';

sub main
{
    startApplication("\"$ENV{'SQUISH_PREFIX'}/examples/qt/addressbook/addressbook\"");
    invokeMenuItem("File", "New");
    my $table = waitForObject({"type" => "QTableWidget"});
    test::verify($table->rowCount == 0);
    my @data = (["Andy", "Beach", "andy.beach\@nowhere.com", "555 123 6786"],
                ["Candy", "Deane", "candy.deane\@nowhere.com", "555 234 8765"],
                ["Ed", "Fernleaf", "ed.fernleaf\@nowhere.com", "555 876 4654"]);
    foreach my $oneNameAndAddress (@data) {
        addNameAndAddress(@{$oneNameAndAddress});
    }
    test::compare($table->rowCount, scalar(@data), "table contains as many rows as added data");
    closeWithoutSaving();
}

require 'names'
include Squish

def main
    startApplication("\"#{ENV['SQUISH_PREFIX']}/examples/qt/addressbook/addressbook\"")
    invokeMenuItem("File", "New")
    table = waitForObject({:type => "QTableWidget"})
    Test.verify(table.rowCount == 0)
    data = [["Andy", "Beach", "andy.beach@nowhere.com", "555 123 6786"],
        ["Candy", "Deane", "candy.deane@nowhere.com", "555 234 8765"],
        ["Ed", "Fernleaf", "ed.fernleaf@nowhere.com", "555 876 4654"]]
    data.each do |oneNameAndAddress|
        addNameAndAddress(oneNameAndAddress)
    end
    Test.compare(table.rowCount, data.length, "table contains as many rows as added data")
    closeWithoutSaving
end

source [findFile "scripts" "names.tcl"]

proc main {} {
    startApplication "\"$::env(SQUISH_PREFIX)/examples/qt/addressbook/addressbook\""
    set table [waitForObject [::Squish::ObjectName type QTableWidget]]
    test compare [property get $table rowCount] 0
    invokeMenuItem "File" "New"
    set data [list \
        [list "Andy" "Beach" "andy.beach@nowhere.com" "555 123 6786"] \
        [list "Candy" "Deane" "candy.deane@nowhere.com" "555 234 8765"] \
        [list "Ed" "Fernleaf" "ed.fernleaf@nowhere.com" "555 876 4654"] ]
    for {set i 0} {$i < [llength $data]} {incr i} {
        addNameAndAddress [lindex $data $i]
    }
    waitForObject $table
    test compare [property get $table rowCount] [llength $data] "table contains as many rows as added data"
    closeWithoutSaving
}

After analyzing the above Script Test Case we can create the following Scenario, with the first three steps:

Scenario: Initial state of created address book
   Given addressbook application is running
   When I create a new addressbook
   Then addressbook should have zero entries

Next, right-click on the Scenario and choose the option Create Missing Step Implementations from the context menu. This will create a skeleton of steps definitions:

@Given("addressbook application is running")
def step(context):
    test.warning("TODO implement addressbook application is running")

@When("I create a new addressbook")
def step(context):
    test.warning("TODO implement I create a new addressbook")

@Then("addressbook should have zero entries")
def step(context):
    test.warning("TODO implement addressbook should have zero entries")

Given("addressbook application is running", function(context) {
    test.warning("TODO implement addressbook application is running");
});

When("I create a new addressbook", function(context) {
    test.warning("TODO implement I create a new addressbook");
});

Then("addressbook should have zero entries", function(context) {
    test.warning("TODO implement addressbook should have zero entries");
});

Given("addressbook application is running", sub {
    my $context = shift;
    test::warning("TODO implement addressbook application is running");
});

When("I create a new addressbook", sub {
    my $context = shift;
    test::warning("TODO implement I create a new addressbook");
});

Then("addressbook should have zero entries", sub {
    my $context = shift;
    test::warning("TODO implement addressbook should have zero entries");
});

Given("addressbook application is running") do |context|
    Test.warning "TODO implement addressbook application is running"
end

When("I create a new addressbook") do |context|
    Test.warning "TODO implement I create a new addressbook"
end

Then("addressbook should have zero entries") do |context|
    Test.warning "TODO implement addressbook should have zero entries"
end

Given "addressbook application is running" {context} {
    test warning "TODO implement addressbook application is running"
}

When "I create a new addressbook" {context} {
    test warning "TODO implement I create a new addressbook"
}

Then "addressbook should have zero entries" {context} {
    test warning "TODO implement addressbook should have zero entries"
}

Now we put code snippets from the Script Test Case into respective step definitions and remove the lines containing test.warning. If your Script Test Cases make use of shared scripts, you can call those functions from the step definition as well. For example, the final result might look like this:

@Given("addressbook application is running")
def step(context):
    startApplication("addressbook")

@When("I create a new addressbook")
def step(context):
    invokeMenuItem("File", "New")

@Then("addressbook should have zero entries")
def step(context):
    table = waitForObject({"type": "QTableWidget"})
    test.compare(table.rowCount, 0)

Given("addressbook application is running", function(context) {
    startApplication("addressbook");
});

When("I create a new addressbook", function(context) {
    invokeMenuItem("File", "New");
});

Then("addressbook should have zero entries", function(context) {
    var table = waitForObject({"type": "QTableWidget"});
    test.compare(table.rowCount, 0);
});

Given("addressbook application is running", sub {
    my $context = shift;
    startApplication("addressbook");
});

When("I create a new addressbook", sub {
    my $context = shift;
    invokeMenuItem("File", "New");
});

Then("addressbook should have zero entries", sub {
    my $table = waitForObject({"type" => "QTableWidget"});
    test::compare($table->rowCount, 0);
});

Given("addressbook application is running") do |context|
    startApplication("addressbook")
end

When("I create a new addressbook") do |context|
    invokeMenuItem("File", "New")
end

Then("addressbook should have zero entries") do |context|
    table = waitForObject({:type => "QTableWidget"})
    Test.compare(table.rowCount, 0)
end

Given "addressbook application is running" {context} {
    startApplication "addressbook"
}

When "I create a new addressbook" {context} {
    invokeMenuItem "File" "New"
}

Then "addressbook should have zero entries" {context} {
    set table [waitForObject [::Squish::ObjectName type QTableWidget]]
    test compare [property get $table rowCount] 0
}

Note that the test.log("Create new addressbook") got removed while migrating this Script Test Case to BDD. When the step I create a new addressbook is executed, the step name will be logged into Test Results, so the test.log call would have been redundant.

After creating the first BDD Test Case in your suite, a shared/scripts/hooks file is created in the correct script language. From the Squish IDE, it is located in the Scripts tab of Test Suite Resources. Initially, these files contain a hook for OnScenarioEnd, called when each Scenario is finished. It attempts to ensure that the AUT is terminated.

@OnScenarioEnd
def hook(context):
    for ctx in applicationContextList():
        ctx.detach()

OnScenarioEnd(function(context) {
    applicationContextList().forEach(function(ctx) { ctx.detach(); });
});

OnScenarioEnd(sub {
    foreach (applicationContextList()) {
        $_->detach();
    }
});

OnScenarioEnd do |context|
    applicationContextList().each { |ctx| ctx.detach() }
end

OnScenarioEnd {context} {
    foreach ctx [applicationContextList] {
        applicationContext $ctx detach
    }
}

The above example was simplified for this tutorial. In order to take full advantage of Behavior Driven Testing in Squish, please familiarize yourself with the section Behavior Driven Testing in API Reference.