Squish for Web Tutorials

Learn how to test web applications.

• Tutorial: Starting to Test Web Applications
• Tutorial: Designing Behavior Driven Development (BDD) Tests
• Tutorial: Migration of existing tests to BDD

Tutorial: Starting to Test Web Applications

Squish comes with an IDE and command line tools. Whenever we show how to achieve something using the Squish IDE, we will follow with an explanation of how to do the same thing using the command line tools. Using the Squish IDE is the easiest and best way to start, but once you build up lots of tests, you will want to automate them. For example, to do nightly runs of your regression test suite. Therefore, it is worth knowing how to use the command line tools that can be run from batch files or shell scripts.

We use a simple Address Book application as our AUT. The application is shipped with Squish in SQUISHDIR/examples/web/addressbook. This is a very basic application that allows users to interact with a fake existing address book or create a new one, and add, edit, and remove entries. Despite the application's simplicity, it has all the key features that most standard web applications have: buttons, radio buttons, line edits, pop-up dialogs, and a central area—in this case showing a table. All the ideas and practices that you learn to test this application can easily be adapted to your own applications. For more examples of testing web-specific features and standard editing widgets, see How to Create Test Scripts and How to Test Web Applications.

The screenshots show the application in action.

Using the Examples

This tutorial's example is an HTML and JavaScript web application contained in the file SQUISHDIR/examples/web/addressbook/AddressBook.html. Squish for Web is designed to test real web applications served over http: by a web server, and the AddressBook example requires being hosted on one.

The server is written in Python; simply run it from the command line using the Python interpreter on your system, or one under SQUISHDIR (python2 or python3).

$ python SQUISHDIR/examples/web/addressbook/server.py

If you are on windows:

C:\> cd SQUISHDIR\examples\web\addressbook
C:\SQUISHDIR\examples\web\addressbook> ..\..\..\python2\python server.py

Once the server is running, you can access the web addressbook example application using the URL http://localhost:9090/AddressBook.html. (If port 9090 conflicts with anything else on your machine simply pass an unused port number as a command line argument to server.py and it will use that instead. Naturally, if you use a different port you must use that port throughout the tutorial.)

Windows Security Dialog

When starting any TCP/IP server for the first time (including squishserver, or server.py) on Windows, depending on your security settings, Windows may pop up a dialog asking if you want to allow or block the server from running. If you get this dialog, you must choose Unblock so that Squish can function correctly.

Browsers on Mobile Platforms

To test web applications running inside a browser (like Safari or Chrome) on iOS (e.g., iPhones and iPads) or Android devices, once Squish for Web is installed, some extra device-specific setup is required. See Browsers on mobile devices.

Firefox Users

A technical limitation prevents the recording or or playing back of tests on Firefox if the browser is already running. If you try, a new tab will appear in Firefox but the test won't run. The solution is to close Firefox. Then, when you record or play back a test, Squish will start and close Firefox automatically as needed.

Squish Concepts

In the following sections we will create a test suite and then create some tests, but first we will very briefly review some key Squish concepts.

To perform testing, you need:

1. An application to test, known as the Application Under Test (AUT).
2. A test script that exercises the AUT.

One fundamental aspect of Squish's approach is that the AUT and the test script that exercises it are always executed in two separate processes. This ensures that even if the AUT crashes, it should not crash Squish. In such cases, the test script will fail gracefully and log an error message. In addition to insulating Squish and test scripts from AUT crashes, running the AUT and the test script in separate processes brings other benefits. For example, it makes it easier to store the test scripts in a central location and to perform remote testing on different machines and platforms. The ability to do remote testing is particularly useful for testing AUTs that run on multiple platforms and for testing AUTs that run on embedded devices.

Squish runs a small server, squishserver, that handles the communication between the AUT and the test script. The test script is executed by the squishrunner tool, which in turn connects to squishserver. squishserver starts the instrumented AUT on the device, which starts the Squish hook. The hook is a small library that makes the AUT's live running objects accessible, and allows communication with squishserver. With the hook in place, squishserver can query AUT objects regarding their state and can execute commands on behalf of squishrunner. squishrunner directs the AUT to perform whatever actions the test script specifies.

All the communication takes place using network sockets which means that everything can be done on a single machine, or the test script can be executed on one machine and the AUT can be tested over the network on another machine.

The following diagram illustrates how the individual Squish tools work together.

From the test engineer's perspective, this separation is not noticeable because all the communication is handled transparently behind the scenes.

Tests can be written and executed using the Squish IDE, in which case squishserver is started and stopped automatically, and the test results are displayed in the Squish IDE's Test Results view. The following diagram illustrates what happens behind the scenes when the Squish IDE is used.

The Squish tools can also be used from the command line without the Squish IDE. This is useful if you prefer to use you own tools, such as your favorite editor or want to perform automatic batch testing. For example, when running regression tests overnight. In these cases, squishserver must be started manually and stopped when all the testing is complete or started and stopped for each test.

For Squish to make it possible for test scripts to be able to query and control an AUT, Squish must be able to access the AUT's internals, and this is made possible by the use of bindings. Bindings are in effect libraries that provide access to the objects—and in turn to the objects' properties and methods—that are available. The bindings for Web Object API is a fixed set of properties and methods mostly for manipulating HTML objects in the loaded web page.

Creating a Test Suite

A test suite is a collection of one or more test cases (tests). Using a test suite is convenient since it makes it easy to share scripts and test data between a group of related tests.

Here, and throughout the tutorial, we will start by describing how to do things using the Squish IDE, with the information for command line users following.

Creating Test Suites from Squish IDE

Start up the Squish IDE, by clicking or double-clicking the squishide icon, by launching squishide from the taskbar menu or by executing squishide on the command line, whichever you prefer and find suitable for the platform you are using. Once Squish starts up, you might be greeted with a Welcome Page. Click the Workbench button in the upper right to dismiss it. Then, the squishide will look similar to the screenshot, but probably slightly different depending on the windowing system, colors, fonts, and theme that you use.

Once Squish has started, click File > New Test Suite to pop-up the New Squish Test Case wizard shown below.

Enter a name for your test suite and choose the folder where you want the test suite to be stored. In the screenshot we have called the test suite suite_js and will put it inside the addressbook folder. (For your own tests you might use a more meaningful name such as suite_addressbook; we chose suite_js because for the sake of the tutorial we will create several suites, one for each scripting language that Squish supports.) Naturally, you can choose whatever name and folder you prefer. Once the details are complete, click Next to go on to the Toolkit (or Scripting Language) page.

If you get this wizard page, click the toolkit your AUT uses. For this example, we must click Web since we are testing a Web application. Then click Next to go to the Scripting Language page.

Choose whichever scripting language you want—the only constraint is that you can only use one scripting language per test suite. (So if you want to use multiple scripting languages, just create multiple test suites, one for each scripting language you want to use.) The functionality offered by Squish is the same for all languages.

Having chosen a scripting language, click Finish to complete the creation of a new test suite. At this point Squish will create a sub-folder with the same name as the test suite, and will create a file inside that folder called suite.conf that contains the test suite's configuration details. The wizard will then close and Squish IDE will look similar to the screenshot below.

We are now ready to start creating tests. Read on to learn how to create test suites without using the Squish IDE, or skip ahead to Recording Tests and Verification Points if you prefer.

We are now ready to record our first test.

Recording Tests and Verification Points

Squish records tests using the scripting language that was specified for the test suite. Once a test has been recorded, we can run the test and Squish will faithfully repeat all the actions that we performed when recording the test, but without the pauses that humans are prone to but which computers don't need. It is also possible and very common to edit recorded tests, or to copy parts of recorded tests into manually created tests, as we will see later on in the tutorial.

Recordings are made into existing test cases. You can create a New Script Test Case in the following ways:

• Select File > New Test Case to open the New Squish Test Case wizard, enter the name for the test case, and select Finish.
• Click the New Script Test Case ( ) toolbar button to the right of the Test Cases label in the Test Suites view. This creates a new test case with a default name, which you can easily change.

Give the new test case the name "tst_general".

Squish automatically creates a sub-folder inside the test suite's folder with this name and also a test file, for example test.py. If you choose JavaScript as the scripting language, the file is called test.js, and correspondingly for Perl, Ruby, or Tcl.

If you get a sample .feature file instead of a "Hello World" script, click the arrow left of the Run Test Suite ( ) and select New Script Test Case ( ).

To make the test script file (such as, test.js or test.py) appear in an Editor view, click or double-click the test case, depending on the Preferences > General > Open mode setting. This selects the Script as the active one and makes visible its corresponding Record ( ) and Run Test ( ) buttons.

The checkboxes are used to control which test cases are run when the Run Test Suite ( ) toolbar button is clicked. We can also run a single test case by clicking its Run Test ( ) button. If the test case is not currently active, the button may be invisible until the mouse is hovered over it.

Initially, the script's main() logs Hello World to the test results. To create a test manually, as we will do later on in the tutorial, we must create a main function, and we should import the same imports at the top. The name main is special to Squish. Tests may contain as many functions and other code as you like, as supported by the scripting language, but when the test is executed (that is, run), Squish always executes the main function. You can share commonly used code between test scripts, as described in How to Create and Use Shared Data and Shared Scripts.

Two other function names are also special to Squish: cleanup and init. For more information, see Tester-Created Special Functions.

Once the new test case has been created, we are free to write test code manually or to record a test. Clicking on the test case's Record ( ) button replaces the test's code with a new recording. Alternatively, you can record snippets and insert them into existing test cases, as instructed in How to Edit and Debug Test Scripts.

Recording Our First Test

Before we dive into recording let's briefly review our very simple test scenario:

1. Add a new name and address.
2. Change the fourth name and address's surname field.
3. Remove the second name and address
4. Verify that the last address is now the new one that was added.

We are now ready to record our first test. Click Record ( ) to the right of the tst_general test case shown in the Test Suites view's Test Cases list. For Web testing, Squish will prompt for the browser's starting URL. For this tutorial that should be http://localhost:9090/AddressBook.html

Once the browser is running and the page is loaded, perform the following actions—and don't worry about how long it takes since Squish doesn't record idle time:

1. Click the Add button and fill in the form with forename, "Jane", surname "Doe", email address "jane.doe@nowhere.com", and a phone number of "555 123 4567". Click or press the Tab key to navigate between fields. Finally, click the Save button. There should now be a new last address with the details you typed in.
2. Click the fourth row's checkbox to select that record and then click the Edit button. In the form tab to or click the surname field and change the surname to "Doe". Finally, click the Save button. The change should be reflected in the list of addresses.
3. Click the second row's checkbox to select that record and then click the Remove button. Click the pop-up confirmation dialog's OK button. The change should be reflected in the list of addresses.
4. Click the Verify toolbar button in the Squish Control Bar Window (the second button from the left) and select Properties.

   

   This will make the Squish IDE appear. In the Application Objects view click the Object Picker ( ) button, then in the AUT move the mouse to the last forename (i.e., the "Jane" that you entered earlier)—each web element you hover over should be highlighted with a red outline. Once the correct table cell is highlighted (i.e., "Jane"), click it. Now go back to the Squish IDE and check the Properties view's innerText property. Now click the Object Picker ( ) button again, and this time click the last surname showing in the AUT (i.e., "Doe"). Now go back to the Squish IDE and again check the innerText property. Finally, click the Save and Insert Verifications button (at the bottom of the Verification Point Creator view) to have the forename and surname verifications for the selected row inserted into the recorded test script. (See the screenshot below.) Once the verification points are inserted the Squish IDE's window will be hidden again and the Control Bar Window and the AUT will be back in view.

5. We've now completed the test, so click the Control Bar Window's Stop Recording button (the left-most button).

Once the recording is finished, the recorded test will appear in Squish IDE as the screenshot illustrates. The exact code that is recorded will vary depending on how you interact. For example, you might invoke menu options by clicking them or by using key sequences—it doesn't matter which you use, but since they are different, Squish will record them differently.)

If the recorded test doesn't appear, click (or double-click depending on your platform and settings) the tst_general test case; this will make Squish show the test's test.js file in an editor window as shown in the screenshot.

Running Tests

Now that we've recorded the test we can play it back, i.e., run it. This in itself is useful in that if the play back failed it might mean that the application has been broken. Furthermore, the two verifications we put in will be checked on play back as the screenshot shows.

Inserting verification points during test recording is very convenient. Here we inserted two in one go, but we can insert as many as we like as often as we like during the test recording process. However, sometimes we might forget to insert a verification, or later on we might want to insert a new verification. We can easily insert additional verifications into a recorded test script as we will see in the next section, Inserting Additional Verification Points.

Playback Failure

Even if you followed the instructions correctly, the test case will still fail to play back. This is because the selection state of the HTML Input field was not recorded/played back properly during the edit step, so when typeText played back, the typed string was appended (or prepended, depending on the browser) to the previous value. The recommended workaround is to insert on the line above, a call to HTML_TextBase.selectAll() on the HTML element itself, from the Web Object API. Examples in all script languages are shown below.

    clickButton(waitForObject(names.editButton_button))
    # manually inserted selectAll():
    waitForObject(names.oneitem_surnameEdit_text).selectAll()
    typeText(waitForObject(names.oneitem_surnameEdit_text), "Doe")
    clickButton(waitForObject(names.save_button))
    clickButton(waitForObject(names.froglogicAddressbookEditButtonButton));
    // manually inserted selectAll():
    waitForObject(names.froglogicAddressbookSurnameEditText).selectAll();
    typeText(waitForObject(names.froglogicAddressbookSurnameEditText), "Doe");
    clickButton(waitForObject(names.froglogicAddressbookSaveButton));
    clickButton(waitForObject($Names::editbutton_button));
    # manually inserted selectAll():
    waitForObject($Names::oneitem_surnameedit_text)->selectAll();
    typeText(waitForObject($Names::oneitem_surnameedit_text), "Doe");
    clickButton(waitForObject($Names::save_button));
    clickButton(waitForObject(Names::EditButton_button))
    # manually inserted selectAll()
    waitForObject(Names::Oneitem_surnameEdit_text).selectAll()
    typeText(waitForObject(Names::Oneitem_surnameEdit_text), "Doe")
    clickButton(waitForObject(Names::Save_button))
    invoke clickButton [waitForObject $names::editButton_button]
    # manually inserted selectAll:
    invoke [waitForObject $names::oneitem_surnameEdit_text] selectAll
    invoke typeText [waitForObject $names::oneitem_surnameEdit_text] "Doe"
    invoke clickButton [waitForObject $names::Save_button]

Running Tests from IDE

To run a test case in the Squish IDE, click the Run Test ( ) that appears when the test case is hovered or selected in the Test Suites view.

To run two or more test cases one after another or to run only the selected test cases, clic Run Test Suite ( ).

Running Tests from Command Line

The squishserver must always be running when running a test, or the --local option must be provided to squishrunner. For more information, see squishserver.

To play back a recorded test from the command line, we execute the squishrunner program and specify the test suite our recorded script is in and the test case we want to play. For example, assuming we are in the directory that contains the test suite's directory:

squishrunner --testsuite suite_js --testcase tst_general --local

Examining the Generated Code

If you look at the code in the screenshot (or the code snippet shown below) you will see that it consists of lots of Object waitForObject(objectOrName) calls as parameters to various other calls such as typeText(objectOrName, text) and clickButton(objectOrName). The Object waitForObject(objectOrName) function waits until a GUI object is ready to be interacted with (i.e., becomes visible and enabled), and is then followed by some function that interacts with the object. The typical interactions are activate (pop-up) a menu, click a menu option or a button, or type in some text.

For a complete overview of Squish's script commands see How to Create Test Scripts, How to Test Applications - Specifics, API Reference, and Tools Reference.

Objects are identified by names that Squish generates. See How to Identify and Access Objects for full details.

The generated code is about 20 lines of code. Here's an extract that just shows how Squish records clicking the Edit menu's Add option, typing in Jane Doe's details into the Add dialog, and clicking OK at the end to close the dialog and update the table.

    clickButton(waitForObject(names.addButton_button))
    typeText(waitForObject(names.oneitem_forenameEdit_text), "Jane")
    typeText(waitForObject(names.oneitem_surnameEdit_text), "Doe")
    typeText(waitForObject(names.oneitem_emailEdit_text), "jane.doe@nowhere.com")
    typeText(waitForObject(names.oneitem_phoneEdit_text), "555 123 4567")
    clickButton(waitForObject(names.save_button))

    clickButton(waitForObject(names.addButtonButton));
    typeText(waitForObject(names.froglogicAddressbookForenameEditText), "Jane");
    typeText(waitForObject(names.froglogicAddressbookSurnameEditText), "Doe");
    typeText(waitForObject(names.froglogicAddressbookEmailEditText), "jane.doe@nowhere.com");
    typeText(waitForObject(names.froglogicAddressbookPhoneEditText), "123 555 4567");
    clickButton(waitForObject(names.froglogicAddressbookSaveButton));

    clickButton(waitForObject($Names::addbutton_button));
    typeText(waitForObject($Names::oneitem_forenameedit_text), "Jane");
    typeText(waitForObject($Names::oneitem_surnameedit_text), "Doe");
    typeText(waitForObject($Names::oneitem_emailedit_text), "jane.doe\@nowhere.com");
    typeText(waitForObject($Names::oneitem_phoneedit_text), "555 123 4567");
    clickButton(waitForObject($Names::save_button));

    clickButton(waitForObject(Names::AddButton_button))
    typeText(waitForObject(Names::Oneitem_forenameEdit_text), "Jane")
    typeText(waitForObject(Names::Oneitem_surnameEdit_text), "Doe")
    typeText(waitForObject(Names::Oneitem_emailEdit_text), "jane.doe@nowhere.com")
    typeText(waitForObject(Names::Oneitem_phoneEdit_text), "555 123 4567")
    clickButton(waitForObject(Names::Save_button))

    invoke clickButton [waitForObject $names::addButton_button]
    invoke typeText [waitForObject $names::oneitem_forenameEdit_text] "Jane"
    invoke typeText [waitForObject $names::oneitem_surnameEdit_text] "Doe"
    invoke typeText [waitForObject $names::oneitem_emailEdit_text] "jane.doe@nowhere.com"
    invoke typeText [waitForObject $names::oneitem_phoneEdit_text] "555 123 4567"
    invoke clickButton [waitForObject $names::Save_button]

The Add and Edit buttons are visible when when the AUT is showing the list of addresses. When the Add or Edit button is clicked, the list is hidden and instead a form is shown where a new address can be added, or where the selected address can be edited. The form has Save and Cancel buttons—when either of these are clicked, the form is hidden and the list of addresses is shown once more.

Squish recordings refer to objects using variables that begin with a names. prefix, which identifies them as Symbolic Names. Each variable contains, as a value, the corresponding Real Name, which can be string-based, or implemented as a key-value mapping of properties to values. Squish supports several naming schemes, all of which can be used—and mixed—in scripts. The advantage of using symbolic names is that if the application changes in a way that results in different names being needed, we can simply update Squish's Object Map (which relates symbolic names to real names), and thereby avoid the need to change our test scripts. See Object Map and Object Map view for more about the Object Map.

When a Symbolic Name is under the cursor, the editor's context menu allows you to Open Symbolic Name, showing its entry in the Object Map, or Convert to Real Name, which places an inline mapping in your script language at the cursor, allowing you to hand-edit the properties in the script itself.

If you look at the recorded test (tst_general) or in the Object Map you will see that Squish for Web sometimes uses Hierarchical Names that describe a path in the the DOM (Document Object Model) of the HTML page under test. For more information about naming and finding of web objects in Squish, see Name Generation Algorithm used by Squish for Web and How to Find and Query Web Objects.

Now that we have seen how to record and play back a test and have seen the code that Squish generates, let's go a step further and make sure that at particular points in the test's execution certain conditions hold.

Inserting Additional Verification Points

In the previous section we saw how easy it is to insert verification points during the recording of test scripts. Verification points can also be inserted into existing test scripts, either by setting a breakpoint and using the Squish IDE, or simply by editing a test script and putting in calls to Squish's test functions such as Boolean test.compare(value1, value2) and Boolean test.verify(condition).

Squish supports many kinds of verification points: those that verify that object properties have particular values—known as "Object Property Verifications"; those that verify that an entire table has the contents we expect—known as "Table Verifications"; those that verify that two images match—known as "Screenshot Verifications"; and a hybrid verification type that includes properties and screenshots from multiple objects, known as "Visual Verifications". In addition, it is possible to verify that a search image exists somewhere on the screen, or that certain text is found by OCR. The most commonly used kind is object property verifications, and it is these that we will cover in the tutorial. For further reading, see How to Create and Use Verification Points).

Regular (non-scriptified) property verification points are stored as XML files in the test case or test suite resources, and contain the value(s) that need to be passed to test.compare(). These verification points can be reused across test cases, and can verify many values in a single line of script code.

Scriptified property verification points are direct calls to the Boolean test.compare(value1, value2) function, with two arguments—the value of a particular property for a particular object, and an expected value. We can manually insert calls to the Boolean test.compare(value1, value2) function in a recorded or hand written script, or we can get Squish to insert them for us using scriptified verification points. In the previous section we showed how to use the Squish IDE to insert verifications during recording. Here we will first show how to use the Squish IDE to insert verifications into an existing test script, and then we will show how to insert a verification by hand.

Before asking Squish to insert verification points, it is best to make sure that we have a list of what we want to verify and when. There are many potential verifications we could add to the test case, but since our concern here is simply to show how to do it, we will only do two—we will verify that the "Jane Doe" entry's email address and phone number match the ones entered, and put the verifications immediately before the ones we inserted during recording.

To insert a verification point using the Squish IDE we start by putting a break point in the script (whether recorded or manually written—it does not matter to Squish), at the point where we want to verify.

As the above screenshot shows, we have set a breakpoint at line 22. This is done simply by double-clicking, or right-clicking in the gutter (next to the line number in the editor) and selecting the Add Breakpoint context menu item. We chose this line because it is just before the previously recorded verification points. The screenshot shows the verifications that were entered using the Squish IDE during recording. Our additional verifications will precede them. (Note that your line number may be different if you recorded the test in a different way, for example, using keyboard shortcuts rather than clicking menu items.)

Having set the breakpoint, we now run the test as usual by clicking the Run Test ( ) or by clicking the Run > Run Test Case menu option. Unlike a normal test run the test will stop when the breakpoint is reached (i.e., at line 12, or at whatever line you set), and Squish's main window will reappear (which will probably obscure the AUT). At this point the Squish IDE will automatically switch to the Test Debugging Perspective.

Perspectives and Views

The Squish IDE works just like the Eclipse IDE. If you aren't used to Eclipse, it is crucial to understand the following key concepts: Views and Perspectives. In Eclipse, and therefore in the Squish IDE, a View is essentially a child window, such as a dock window or a tab in an existing window. A Perspective is a collection of views arranged together. Both are accessible through the Window menu.

The Squish IDE is supplied with the following perspectives:

• Test Management Perspective that the Squish IDE starts with, and that is shown in all previous screenshots
• Test Debugging Perspective
• Spy Perspective

You can change these perspectives to show additional views or to hide views that you don't want, or create your own perspectives with exactly the views you want. So if your windows change dramatically, it just means that the perspective changed. Use the Window menu to change back to the perspective you want. However, Squish automatically changes perspective to reflect the current situation, so you should not need to change perspective manually.

Inserting Verification Points

As the screenshot below shows, when Squish stops at a breakpoint the Squish IDE automatically changes to the Test Debugging Perspective. The perspective shows the Variables view, the Editor view, the Debug view, the Application Objects view, and the Properties view, Methods view, and Test Results view.

To insert a verification point, we can expand items in the Application Objects view until we find the object we want to verify, or we can use the Object Picker ( ) toolbar button to visually pick the relevant object in the AUT.

The normal Test Management Perspective can be returned to at any time by choosing it from the Window menu (or by clicking its toolbar button), although the Squish IDE will automatically return to it if you stop the script or run it to completion.

In this example we want to verify the new row's email address and phone number (since we already have verifications for the forename and surname). It is easiest to find them in the AUT rather than navigate the Application Objects view. First click the Object Picker ( ) toolbar button then click new row's email entry (i.e., "jane.doe@nowhere.com") in the AUT. Now back in the Squish IDE in the Properties view check the innerText property.

At this point the verification point has not been added to the test script. We could easily add it by clicking the Save and Insert Verifications button. But before doing that we'll add one more thing to be verified.

Click the Object Picker ( ) and then click the new row's phone number. Now back in the Squish IDE in the Properties view check the innerText property. Now both verifications will appear in the Verification Point Creator view as the screenshot shows.

We have now said that we expect these properties to have the values shown, that is, an email address of "jane.doe@nowhere.com" and phone number of "555 123 4567". We must click the Insert button to actually insert the verification point, so do that now.

We don't need to continue running the test now, so we can either stop running the test at this point (by clicking the Stop toolbar button), or we can continue (by clicking the Resume button).

Once we have finished inserting verifications and stopped or finished running the test we should now disable the break point. Just right click the break point and click the Disable Breakpoint menu option in the context menu. We are now ready to run the test without any breakpoints but with the verification points in place. Click the Run Test ( ). This time we will get some additional test results—as the screenshot shows—one of which we have expanded to show its details. (We have also selected the lines of code that Squish inserted to perform the verifications—notice that the code is structurally identical to the code inserted during recording.)

These particular verification points generate four tests comparing the forename, surname, email, and phone number of the newly inserted entry.

Another way to insert verification points is to write them in code. In theory, we can just add our own calls to Squish's test functions such as Boolean test.compare(value1, value2) and Boolean test.verify(condition) anywhere we like in an existing script. In practice, it is best to make sure that Squish knows about the objects we want to verify first so that it can find them when the test is run. This involves a very similar procedure to inserting them using the Squish IDE.

1. Set a breakpoint where we intend to add verifications.
2. Run the test script until it stops.
3. Navigate in the Application Objects view until we find the object we want to verify—or use the Object Picker ( ) to visually choose it in the AUT.
4. Right-click the object we are interested in and click the Add to Object Map context menu option to ensure that Squish can access the object.
5. Right-click again and click the Copy Symbolic Name context menu option—this gives us the name of the object that Squish will use to identify it.

Now we can edit the test script to add our own verification and finish or stop the execution. Don't forget to disable the break point once it isn't needed any more.

Although we can write our test script code to be exactly the same style as the automatically generated code, it is usually clearer and easier to do things in a slightly different style, as we will explain in a moment.

For our manual verifications we want to check the number of addresses present in the <table> after loading the initial addresses, then after the new address is added, and finally after the second address is removed.

The original test recording does not store details of the <table> in Squish's Object Map because they aren't needed for the test we recorded. But for us to count the number of rows we must have access to the table. This is easily done. First we add a breakpoint to one of the lines (it doesn't matter which—so long as the list of addresses is showing at the time the breakpoint happens—we used line 5). Then, we run the test and when it is stopped by the breakpoint we click the Application Objects view's Object Picker ( ) toolbar button. Then we click the entire table in the AUT. (This can be slightly tricky—just hover until the red outline includes the whole table.) Back in the Squish IDE we right-click the table in the Squish IDE's Application Objects view and click the Add to Object Map context menu option. Then we right-click again and click the Copy Symbolic Name context menu option—then paste this name into the test script where it will become part of a numberOfRows function. (Don't forget to remove the breakpoint afterwards, since it isn't needed anymore.)

The screenshot shows the line of code we entered to get one of these three verifications; it also shows the results of running the test script. (We will see the tiny custom numberOfRows function in a moment.)

Here is the code we entered manually for the first verification for all the scripting languages that Squish supports. Naturally, you only need to look at the code for the language that you will be using for your own tests. For all the row count verifications we just did calls to the Boolean test.verify(condition) function—or to the Boolean test.compare(value1, value2) function for Tcl since it's more convenient.

    test.compare(numberOfRows(), 125)

    test.compare(numberOfRows(), 125)

    test::compare(numberOfRows(), 125);

    Test.compare(numberOfRows, 125)

    test compare [numberOfRows] 125

And here is the custom numberOfRows function that our verification depends upon.

def numberOfRows():
    table = waitForObject(names.dOCUMENT_HTML1_BODY1_DIV1_DIV2_DIV3_DIV3_DIV1_TABLE1)
    results = table.evaluateXPath(".//TR[contains(@class, 'jqgrow')]")
    return results.snapshotLength

function numberOfRows() {
    var table = waitForObject(names.dOCUMENTHTML1BODY1DIV1DIV2DIV3DIV3DIV1TABLE1);
    var results = table.evaluateXPath(".//TR[contains(@class, 'jqgrow')]");
    return results.snapshotLength;
}

sub numberOfRows
{
    my $table = waitForObject($Names::document_html1_body1_div1_div2_div3_div3_div1_table1);
    my $results = $table->evaluateXPath(".//TR[contains(\@class, 'jqgrow')]");
    return $results->snapshotLength;
}

def numberOfRows
    table = waitForObject(Names::DOCUMENT_HTML1_BODY1_DIV1_DIV2_DIV3_DIV3_DIV1_TABLE1)
    results = table.evaluateXPath(".//TR[contains(@class,'jqgrow')]")
    results.snapshotLength
end

proc numberOfRows {} {
    set table [waitForObject \
        $names::DOCUMENT_HTML1_BODY1_DIV1_DIV2_DIV3_DIV3_DIV1_TABLE1]
    set results [invoke $table evaluateXPath {.//TR[contains(@class,'jqgrow')]}]
    return [property get $results snapshotLength]
}

When writing scripts by hand, we use Squish's test module's functions to verify conditions at certain points during our test script's execution. As the screenshot (and the code snippets) show, we begin by retrieving a reference to the object we are interested in. Using the Object waitForObject(objectOrName) function is standard practice for manually written test scripts. This function waits for the object to be available (i.e., visible and enabled), and then returns a reference to it. (Otherwise it times out and raises a catchable exception.) We then use this reference to access the item's properties. In this case we get a reference to the <table> and evaluate an XPath query (see How to Use XPath). The query used here says find all <tr> tags under the given element (i.e., in the table) which have a class property that contains the value "jqgrow". This produces an HTML_XPathResult Class object containing all the matching <tr>s. Since each address has a single row we know that the number of <tr> tags with class "jqgrow" is equal to the number of rows, so we simply return the number of items in the result using its HTML_XPathResult.snapshotLength property.

For more examples of manually written tests, see Creating Tests by Hand, How to Create Test Scripts, and How to Test Applications - Specifics.

For complete coverage of verification points, see How to Create and Use Verification Points.

Test Results

After each test run finishes, the test results—including those for the verification points—are shown in the Test Results view at the bottom of the Squish IDE.

This is a detailed report of the test run and would also contain details of any failures or errors, etc. If you click on a Test Results item, the Squish IDE highlights the script line which generated the test result. And if you expand a Test Results item, you can see additional details of the test.

Squish's interface for test results is very flexible. By implementing custom report generators it is possible to process test results in many different ways, for example to store them in a database, or to output them as HTML files. The default report generator simply prints the results to stdout when Squish is run from the command line, or to the Test Results view when Squish IDE is being used. You can save the test results from the Squish IDE as XML by right clicking on the Test Results and choosing the Export Results menu option. For a list of report generators, see squishrunner --reportgen: Generating Reports. It is also possible to log test results directly to a database. See How to Access Databases from Squish Test Scripts.

If you run tests on the command line using squishrunner, you can also export the results in different formats and save them to files. See the sections Processing Test Results and How to Use Test Statements for more information.

Creating Tests by Hand

Now that we have seen how to record a test and modify it by inserting verification points, we are ready to see how to create tests manually. The easiest way to do this is to modify and refactor recorded tests, although it is also perfectly possible to create manual tests from scratch.

Potentially the most challenging part of writing manual tests is to use the right object names, but in practice, this is rarely a problem. We can either copy the symbolic names that Squish has already added to the Object Map when recording previous tests, or we can copy object names directly from recorded tests. And if we haven't recorded any tests and are starting from scratch we can use the Spy. We do this by clicking the Launch AUT toolbar button. This starts the AUT and switches to the Spy Perspective. We can then interact with the AUT until the object we are interested in is visible. Then, inside the Squish IDE we can navigate to the object in the Application Objects view—or use the Object Picker ( ) toolbar button—and use the context menu to both add the object to the Object Map (so that Squish will remember it) and to the clipboard (so that we can paste it into our test script). And at the end we can click the Quit AUT toolbar button to terminate the AUT and return Squish to the Test Management Perspective. See How to Use the Spy for more details on using the Spy.

We can view the Object Map by clicking the Object Map toolbar button (see also, the Object Map view). Every application object that Squish interacts with is listed here, either as a top-level object, or as a child object (the view is a tree view). We can retrieve the symbolic name used by Squish in recorded scripts by right-clicking the object we are interested in and then clicking the context menu's Copy item. This is useful for when we want to modify existing test scripts or when we want to create test scripts from scratch, as we will see later on in the tutorial.

Modifying and Refactoring Recorded Tests

Suppose we want to test the AUT's Add functionality by adding three new names and addresses. We could record such a test but it is just as easy to do everything in code. The steps we need the test script to do are: start the application, then for each new name and address, click the Add button, then fill in the details, and click Save. We also want to verify after clicking New that there are no rows of data and at the end that there are three rows. We will also refactor as we go, to make our code as neat and modular as possible.

First we must create a new test case. Click File > New Test Case and set the test case's name to be tst_adding. Squish will automatically create a test.js (or test.py, and so on) file.

Command line users can simply create a tst_adding directory inside the test suite's directory and create and edit the test.js file (or test.py and so on) within that directory.

The first thing we need is a way to start the AUT. Here are the first few lines from the recorded tst_general script:

import names

def main():
    startBrowser("http://localhost:9090/AddressBook.html")

import * as names from 'names.js';

function main() {
    startBrowser("http://localhost:9090/AddressBook.html");

require 'names.pl';

sub main
{
    startBrowser("http://localhost:9090/AddressBook.html");

require 'squish'
require 'names'
include Squish

def main
    startBrowser("http://localhost:9090/AddressBook.html")

source [findFile "scripts" "names.tcl"]

proc main {} {
    invoke startBrowser "http://localhost:9090/AddressBook.html"

Notice that the pattern in the code is simple: import the object map first, and then in main, start the AUT, then wait for the page to be ready.

If the AUT appears to freeze during test execution, wait for Squish to time out the AUT (about 20 seconds), and show the Object Not Found dialog, indicating an error like this:

This usually means that Squish doesn't have an object with the given name, or property values, in the Object Map. From here, we can Pick a new object, Debug, Throw Error or, after picking a new object, Retry.

Picking a new object will update the object map entry for the symbolic name. In addition to the Object Picker ( ) we can use the Spy's Application Objects view to locate the objects we are interested in and use the Add to the Object Map context menu action to to access their real or symbolic names.

Naming is important because it is probably the part of writing scripts that leads to the most error messages, usually of the object ... not found kind shown above. Once we have identified the objects to access in our tests, writing test scripts using Squish is very straightforward. Especially, as Squish most likely supports the scripting language you are most familiar with.

We are now almost ready to write our own test script. It is probably easiest to begin by recording a dummy test. So click File > New Test Case and set the test case's name to be tst_dummy. Then click the dummy test case's Record ( ). Once the AUT starts, click the New button then click OK. This will clear out the example data and leave the table empty and ready for new data. Click the Control Bar Window's Stop Recording button. Replay this test just to confirm that everything works okay. The sole purpose of this is to make sure that Squish adds the necessary names to the Object Map since it is probably quicker to do it this way than to use the Spy for every object of interest. After replaying the dummy test you can delete it if you want to.

With all the object names we need in the Object Map, we can now write our own test script completely from scratch. We will start with the main function, and then we will look at the supporting functions that the main function uses.

import names

def main():
    startBrowser("http://localhost:9090/AddressBook.html")
    confirmPopup(names.newButton_button)
    test.verify(numberOfRows() == 0, "%d" % numberOfRows())
    data = [("Andy", "Beach", "andy.beach@nowhere.com", "555 123 6786"),
            ("Candy", "Deane", "candy.deane@nowhere.com", "555 234 8765"),
            ("Ed", "Fernleaf", "ed.fernleaf@nowhere.com", "555 876 4654")]
    for oneNameAndAddress in data:
        addNameAndAddress(oneNameAndAddress)
    test.compare(numberOfRows(), 3)

import * as names from 'names.js';

function main() {
    startBrowser("http://localhost:9090/AddressBook.html");
    confirmPopup(names.newButtonButton);
    test.verify(numberOfRows() == 0);
    var data = new Array(
        new Array("Andy", "Beach", "andy.beach@nowhere.com", "555 123 6786"),
        new Array("Candy", "Deane", "candy.deane@nowhere.com", "555 234 8765"),
        new Array("Ed", "Fernleaf", "ed.fernleaf@nowhere.com", "555 876 4654"));
    for (var row = 0; row < data.length; ++row)
        addNameAndAddress(data[row]);
    test.compare(numberOfRows(), 3);
}

require 'names.pl';

sub main
{
    startBrowser("http://localhost:9090/AddressBook.html");
    confirmPopup($Names::newbutton_button);
    test::verify(numberOfRows() == 0);
    my @data = (["Andy", "Beach", "andy.beach\@nowhere.com", "555 123 6786"],
                ["Candy", "Deane", "candy.deane\@nowhere.com", "555 234 8765"],
                ["Ed", "Fernleaf", "ed.fernleaf\@nowhere.com", "555 876 4654"]);
    foreach $oneNameAndAddress (@data) {
        addNameAndAddress(@{$oneNameAndAddress});
    }
    test::compare(numberOfRows(), 3);
}

require 'names';

# encoding: UTF-8
require 'squish'
include Squish

def main
    startBrowser("http://localhost:9090/AddressBook.html")
    confirmPopup(Names::NewButton_button)
    Test.verify(numberOfRows == 0)
    data = [["Andy", "Beach", "andy.beach@nowhere.com", "555 123 6786"],
          ["Candy", "Deane", "candy.deane@nowhere.com", "555 234 8765"],
          ["Ed", "Fernleaf", "ed.fernleaf@nowhere.com", "555 876 4654"]]
    data.each do |oneNameAndAddress|
        addNameAndAddress(oneNameAndAddress)
    end
    Test.compare(numberOfRows, 3)
end

source [findFile "scripts" "names.tcl"]

proc main {} {
    invoke startBrowser "http://localhost:9090/AddressBook.html"
    confirmPopup $names::newButton_button
    test compare [numberOfRows] 0
    set data [list \
        [list "Andy" "Beach" "andy.beach@nowhere.com" "555 123 6786"] \
        [list "Candy" "Deane" "candy.deane@nowhere.com" "555 234 8765"] \
        [list "Ed" "Fernleaf" "ed.fernleaf@nowhere.com" "555 876 4654"] ]
    for {set i 0} {$i < [llength $data]} {incr i} {
        addNameAndAddress [lindex $data $i]
    }
    test compare [numberOfRows] 3
}

We begin by starting the AUT with a call to the startBrowser(url) function parametrized by the name of the web page we want it to start with. This means that Squish won't ask us to confirm the page to load. Next we call a custom confirmPopup function that clicks New (which in turn causes an OK/Cancel dialog to appear), and confirms (i.e., clicks OK). This will empty the table. (We copied the name of the New button from the Object Map—it was put there by the dummy test we recorded.) Next we call the custom numberOfRows function that we created earlier to verify that the table is empty.

Next, we create some sample data and call a custom addNameAndAddress function to populate the table with the data using the AUT's Add dialog. And finally, we again compare the table's row count, this time to the number of rows in our sample data.

We will now review two of the three supporting functions, so as to cover all the code in the tst_adding test case, starting with the confirmPopup function. (The third function, numberOfRows was discussed earlier.)

def confirmPopup(button):
    clickButton(waitForObject(button))
    snooze(1.8)
    closeConfirm(names.confirmPopup, True)

function confirmPopup(button) {
    clickButton(waitForObject(button));
    snooze(1.8);
    closeConfirm(names.confirmPopup, true);
}

sub confirmPopup
{
    my ($button) = @_;
    clickButton(waitForObject($button));
    snooze(1.8);
    closeConfirm($Names::confirmpopup, 1);
}

def confirmPopup(button)
    clickButton(waitForObject(button))
    snooze(1.8)
    closeConfirm(Names::ConfirmPopup, true)
end

proc confirmPopup {button} {
    invoke clickButton [waitForObject $button]
    snooze 1.8
    invoke closeConfirm $names::ConfirmPopup true
}

This function is called with the symbolic name of the button we want to click and confirm. (The name was copied from the Object Map.) After clicking we force Squish to wait a short time (1.8 seconds using the snooze(seconds) function) and then close the dialog that popped up using the closeConfirm(":dummy", confirmed) function. The first argument can be any text but the second must either be true (which means click OK) or false (which means click Cancel). So here we have clicked OK.

def addNameAndAddress(oneNameAndAddress):
    clickButton(waitForObject(names.addButton_button))
    typeText(waitForObject(names.oneitem_forenameEdit_text), oneNameAndAddress[0])
    typeText(waitForObject(names.oneitem_surnameEdit_text), oneNameAndAddress[1])
    typeText(waitForObject(names.oneitem_emailEdit_text), oneNameAndAddress[2])
    typeText(waitForObject(names.oneitem_phoneEdit_text), oneNameAndAddress[3])
    clickButton(waitForObject(names.save_button))

function addNameAndAddress(oneNameAndAddress) {
    clickButton(waitForObject(names.addButtonButton));
    typeText(waitForObject(names.froglogicAddressbookForenameEditText), oneNameAndAddress[0]);
    typeText(waitForObject(names.froglogicAddressbookSurnameEditText), oneNameAndAddress[1]);
    typeText(waitForObject(names.froglogicAddressbookEmailEditText), oneNameAndAddress[2]);
    typeText(waitForObject(names.froglogicAddressbookPhoneEditText), oneNameAndAddress[3]);
    clickButton(waitForObject(names.saveButton));
}

sub addNameAndAddress
{
    my (@oneNameAndAddress) = @_;
    clickButton(waitForObject($Names::addbutton_button));
    typeText(waitForObject($Names::oneitem_forenameedit_text), $oneNameAndAddress[0]);
    typeText(waitForObject($Names::oneitem_surnameedit_text), $oneNameAndAddress[1]);
    typeText(waitForObject($Names::oneitem_emailedit_text), $oneNameAndAddress[2]);
    typeText(waitForObject($Names::oneitem_phoneedit_text), $oneNameAndAddress[3]);
    clickButton(waitForObject($Names::save_button));
}

def addNameAndAddress(oneNameAndAddress)
    clickButton(waitForObject(Names::AddButton_button))
    typeText(waitForObject(Names::Oneitem_forenameEdit_text), oneNameAndAddress[0])
    typeText(waitForObject(Names::Oneitem_surnameEdit_text), oneNameAndAddress[1])
    typeText(waitForObject(Names::Oneitem_emailEdit_text), oneNameAndAddress[2])
    typeText(waitForObject(Names::Oneitem_phoneEdit_text), oneNameAndAddress[3])
    clickButton(waitForObject(Names::Save_button))
end

proc addNameAndAddress {oneNameAndAddress} {
    invoke clickButton [waitForObject $names::addButton_button]
    invoke typeText [waitForObject $names::oneitem_forenameEdit_text] [lindex $oneNameAndAddress 0]
    invoke typeText [waitForObject $names::oneitem_surnameEdit_text] [lindex $oneNameAndAddress 1]
    invoke typeText [waitForObject $names::oneitem_emailEdit_text] [lindex $oneNameAndAddress 2]
    invoke typeText [waitForObject $names::oneitem_phoneEdit_text] [lindex $oneNameAndAddress 3]
    invoke clickButton [waitForObject $names::Save_button]
}

For each set of name and address data we click the Add button to make the Add form visible. Then for each value received we populate the appropriate field by waiting for the relevant text field to be ready and then typing in the text using the setText(objectOrName, text) function. And at the end we click the form's Save button. We got the line at the heart of the function by copying it from the recorded tst_general test and simply parametrizing it by the field name and text. Similarly, we copied the code for clicking the Save button from the tst_general test case's code.

The entire test is around 35 lines of code—and would be even less if we put some of the common functions (such as confirmPopup and numberOfRows) in a shared script. And much of the code was copied directly from the recorded test, and in some cases parametrized.

This should be sufficient to give a flavor of writing test scripts for an AUT. Keep in mind that Squish provides far more functionality than we used here, (all of which is covered in the API Reference and the Tools Reference). And Squish also provides access to the entire public APIs of the AUT's objects.

However, one aspect of the test case is not very satisfactory. Although embedding test data as we did here is sensible for small amounts, it is rather limiting, especially when we want to use a lot of test data. Also, we didn't test any of the data that was added to see if it correctly ended up in the table. In the next section we will create a new version of this test, only this time we will pull in the data from an external data source, and check that the data we add to the table is correct.

Creating Data Driven Tests

In the previous section we put three hard-coded names and addresses in our test. But what if we want to test lots of data? Or what if we want to change the data without having to change our test script's source code. One approach is to import a dataset into Squish and use the dataset as the source of the values we insert into our tests. Squish can import data in .tsv (tab-separated values format), .csv (comma-separated values format), .xls or .xlsx (Microsoft Excel spreadsheet formats).

Test data can either be imported using the Squish IDE, or manually using a file manager or console commands. We will describe both approaches, starting with using the Squish IDE.

For the addressbook application we want to import the MyAddresses.tsv data file. To do this we must start by clicking File > Import Test Resource to pop-up the Import Squish Resource dialog. Inside the dialog click the Browse button to choose the file to import—in this case MyAddresses.tsv. Make sure that the Import As combobox is set to "TestData". By default the Squish IDE will import the test data just for the current test case, but we want the test data to be available to all the test suite's test cases: to do this check the Copy to Test Suite for Sharing radio button. Now click the Finish button. You can now see the file listed in the Test Suite Resources view (in the Test Data tab), and if you click the file's name it will be shown in an Editor view. The screenshot shows Squish after the test data has been added.

To import test data from outside the Squish IDE, use a file manager, such as File Explorer or Finder, or console commands. Create a directory called shared inside the test suite's directory. Then, create a directory called testdata inside the shared directory. Copy the data file (in this example, MyAddresses.tsv) into the shared\testdata directory.

Restart the Squish IDE if it is running. If you click the Test Suite Resources view's Test Data tab, you should see the data file. Click the file name to see the file in an Editor view.

Although in real life we would modify our tst_adding test case to use the test data, for the purpose of the tutorial we will make a new test case called tst_adding_data that is a copy of tst_adding and which we will modify to make use of the test data.

The only function we have to change is main, where instead of iterating over hard-coded items of data, we iterate over all the records in the dataset. We also need to update the expected row count at the end since we are adding a lot more records now, and we will also add a function to verify each record that's added.

import names

def main():
    startBrowser("http://localhost:9090/AddressBook.html")
    confirmPopup(names.newButton_button)
    test.verify(numberOfRows() == 0)
    limit = 10
    for row, record in enumerate(testData.dataset("MyAddresses.tsv")):
        forename = testData.field(record, "Forename")
        surname = testData.field(record, "Surname")
        email = testData.field(record, "Email")
        phone = testData.field(record, "Phone")
        addNameAndAddress((forename, surname, email, phone)) # pass as a single tuple
        checkNameAndAddress(record)
        if row > limit:
            break
    test.compare(numberOfRows(), row + 1)

import * as names from 'names.js';

function main() {
    startBrowser("http://localhost:9090/AddressBook.html");
    confirmPopup(names.newButtonButton);
    test.verify(numberOfRows() == 0);
    var limit = 10;
    var records = testData.dataset("MyAddresses.tsv");
    for (var row = 0; row < records.length; ++row) {
        var record = records[row];
        var forename = testData.field(record, "Forename");
        var surname = testData.field(record, "Surname");
        var email = testData.field(record, "Email");
        var phone = testData.field(record, "Phone");
        addNameAndAddress(new Array(forename, surname, email, phone));
        checkNameAndAddress(record);
        if (row > limit)
            break;
    }
    test.compare(numberOfRows(), row + 1);
}

require 'names.pl';

sub main
{
    startBrowser("http://localhost:9090/AddressBook.html");
    confirmPopup($Names::newbutton_button);
    test::verify(numberOfRows() == 0);
    my @records = testData::dataset("MyAddresses.tsv");
    my $limit = 10;
    my $row = 0;
    for (; $row < scalar(@records); ++$row) {
        my $record = $records[$row];
        my $forename = testData::field($record, "Forename");
        my $surname = testData::field($record, "Surname");
        my $email = testData::field($record, "Email");
        my $phone = testData::field($record, "Phone");
        addNameAndAddress(($forename, $surname, $email, $phone));
        checkNameAndAddress($record);
        if ($row > $limit) {
            last;
        }
    }
    test::compare(numberOfRows(), $row + 1);
}

require 'squish'
require 'names';
include Squish

def main
    startBrowser("http://localhost:9090/AddressBook.html")
    confirmPopup(Names::NewButton_button)
    Test.verify(numberOfRows == 0)
    limit = 10
    rows = 0
    TestData.dataset("MyAddresses.tsv").each_with_index do
        |record, row|
        forename = TestData.field(record, "Forename")
        surname = TestData.field(record, "Surname")
        email = TestData.field(record, "Email")
        phone = TestData.field(record, "Phone")
        addNameAndAddress([forename, surname, email, phone]) # pass as an Array
        checkNameAndAddress(record)
        break if row > limit
        rows += 1
    end
    Test.compare(numberOfRows, rows + 1)
end

source [findFile "scripts" "names.tcl"]

proc main {} {
    invoke startBrowser "http://localhost:9090/AddressBook.html"
    confirmPopup $names::newButton_button
    test compare [numberOfRows] 0
    set limit 10
    set data [testData dataset "MyAddresses.tsv"]
    set columns [llength [testData fieldNames [lindex $data 0]]]
    set row 0
    for {} {$row < [llength $data]} {incr row} {
        set record [lindex $data $row]
        set forename [testData field $record "Forename"]
        set surname [testData field $record "Surname"]
        set email [testData field $record "Email"]
        set phone [testData field $record "Phone"]
        set details [list $forename $surname $email $phone]
        addNameAndAddress $details
        checkNameAndAddress $record
        if {$row > $limit} {
            break
        }
    }
    test compare [numberOfRows] [expr $row + 1]
}

Squish provides access to test data through its testData module's functions—here we used the Dataset testData.dataset(filename) function to access the data file and make its records available, and the String testData.field(record, fieldName) function to retrieve each record's individual fields.

Having used the test data to populate the HTML table we want to be confident that the data in the table is the same as what we have added, so that's why we added the checkNameAndAddress function. We also added a limit to how many records we would compare, just to make the test run faster.

def checkNameAndAddress(record):
    table = waitForObject(names.dOCUMENT_HTML1_BODY1_DIV1_DIV2_DIV3_DIV3_DIV1_TABLE1)
    cells = (table.evaluateXPath(".//TR/TD[2]"),
             table.evaluateXPath(".//TR/TD[3]"),
             table.evaluateXPath(".//TR/TD[4]"),
             table.evaluateXPath(".//TR/TD[5]"))
    for column in range(len(testData.fieldNames(record))):
        cell = cells[column].snapshotItem(cells[column].snapshotLength-1).innerText
        field = testData.field(record, column)
        test.compare(cell, field)

function checkNameAndAddress(record)
{
    var table = waitForObject(names.dOCUMENTHTML1BODY1DIV1DIV2DIV3DIV3DIV1TABLE1);
    var cells = [table.evaluateXPath(".//TR/TD[2]"),
                 table.evaluateXPath(".//TR/TD[3]"),
                 table.evaluateXPath(".//TR/TD[4]"),
                 table.evaluateXPath(".//TR/TD[5]")];
    for (var column = 0; column < testData.fieldNames(record).length;
            ++column) {
        var cell = cells[column].snapshotItem(cells[column].snapshotLength-1).innerText;
        var field = testData.field(record, column);
        test.compare(cell, field);
    }
}

sub checkNameAndAddress
{
    my ($record) = @_;
    my $table = waitForObject($Names::document_html1_body1_div1_div2_div3_div3_div1_table1);
    my @cells = ($table->evaluateXPath(".//TR/TD[2]"),
                 $table->evaluateXPath(".//TR/TD[3]"),
                 $table->evaluateXPath(".//TR/TD[4]"),
                 $table->evaluateXPath(".//TR/TD[5]"));
    my @columnNames = testData::fieldNames($record);
    for (my $column = 0; $column < scalar(@columnNames); ++$column) {
        my $cell = $cells[$column]->snapshotItem($cells[$column]->snapshotLength - 1)->innerText;
        my $field = testData::field($record, $column);
        test::compare($cell, $field);
    }
}

def checkNameAndAddress(record)
    table = waitForObject(Names::DOCUMENT_HTML1_BODY1_DIV1_DIV2_DIV3_DIV3_DIV1_TABLE1)
    cells = [table.evaluateXPath(".//TR/TD[2]"),
           table.evaluateXPath(".//TR/TD[3]"),
           table.evaluateXPath(".//TR/TD[4]"),
           table.evaluateXPath(".//TR/TD[5]")]
    for column in 0...TestData.fieldNames(record).length
        cell = cells[column].snapshotItem(cells[column].snapshotLength-1).innerText
        field = TestData.field(record, column)
        Test.compare(cell, field)
    end
end

proc checkNameAndAddress {record} {
    set table [waitForObject $names::DOCUMENT_HTML1_BODY1_DIV1_DIV2_DIV3_DIV3_DIV1_TABLE1]
    set cells [list \
        [invoke $table evaluateXPath {.//TR/TD[2]}] \
        [invoke $table evaluateXPath {.//TR/TD[3]}] \
        [invoke $table evaluateXPath {.//TR/TD[4]}] \
        [invoke $table evaluateXPath {.//TR/TD[5]}]]
    set columns [llength [testData fieldNames $record]]
    for {set column 0} {$column < $columns} {incr column} {
        set itemPos [expr [property get [lindex $cells $column] snapshotLength ] - 1 ]
        set cell [property get [invoke [lindex $cells $column] \
            snapshotItem $itemPos] innerText]
        set field [testData field $record $column]
        test compare $cell $field
    }
}

The calls to the HTML_XPathResult HTML_Object.evaluateXPath(statement) function use XPath queries to access the HTML table and find the first cells that match. Since the addressbook application always adds new addresses at the end the code needs to use the last item from the query. We use Squish's SequenceOfStrings testData.fieldNames(record) function to get a column count and then use the Boolean test.compare(value1, value2) function to check that each value in the table is the same as the value in the test data we used.

The screenshot show Squish's Test Summary log after the data-driven tests have been run.

Squish can also do keyword-driven testing. This is a bit more sophisticated than data-driven testing. See How to Do Keyword-Driven Testing.

Learning More

We have now completed the tutorial. Squish can do much more than we have shown here, but the aim has been to get you started with basic testing as quickly and easily as possible. The How to Create Test Scripts, and How to Test Applications - Specifics sections provide many more examples, including those that show how tests can interact with particular input elements, such as selects, select-ones, texts, and text-areas.

The API Reference and Tools Reference give full details of Squish's testing API and the numerous functions it offers to make testing as easy and efficient as possible. It is well worth reading the How to Create Test Scripts and How to Test Applications - Specifics, as well as browsing the API Reference and Tools Reference. The time you invested will be repaid because you'll know what functionality Squish provides out of the box and can avoid reinventing things that are already available.

Tutorial: Designing Behavior Driven Development (BDD) Tests

This tutorial will show you how to create, run, and modify Behavior Driven Development (BDD) tests for an example application. You will learn about Squish's most frequently used features. By the end of the tutorial you will be able to write your own tests for your own applications.

For this chapter we will use a simple Address Book application as our AUT. The application is shipped with Squish in SQUISHDIR/examples/web/addressbook. This is a very basic application that allows users to interact with a fake existing address book or create a new one, and add, edit, and remove entries. Despite the application's simplicity, it has all the key features that most standard web applications have: buttons, radio buttons, line edits, pop-up dialogs, and a central area—in this case showing a table. All the ideas and practices that you learn to test this application can easily be adapted to your own applications. For more examples of testing various web-specific features and standard editing widgets, see How to Create Test Scripts and How to Test Web Applications.

The screenshots show the application in action.

Adding a new Address.

Using the Examples

This tutorial's example is an HTML and JavaScript web application contained in the file SQUISHDIR/examples/web/addressbook/AddressBook.html. Squish for Web is designed to test real web applications served over http: by a web server, and the AddressBook example requires being hosted on one.

The server is written in Python; simply run it from the command line using the Python interpreter on your system, or one under SQUISHDIR (python2 or python3).

$ python SQUISHDIR/examples/web/addressbook/server.py

If you are on windows:

C:\> cd SQUISHDIR\examples\web\addressbook
C:\SQUISHDIR\examples\web\addressbook> ..\..\..\python2\python server.py

Once the server is running, you can access the web addressbook example application using the URL http://localhost:9090/AddressBook.html. (If port 9090 conflicts with anything else on your machine simply pass an unused port number as a command line argument to server.py and it will use that instead. Naturally, if you use a different port you must use that port throughout the tutorial.)

Introduction to Behavior Driven Development

Behavior-Driven Development (BDD) is an extension of the Test-Driven Development approach which puts the definition of acceptance criteria at the beginning of the development process as opposed to writing tests after the software has been developed. With possible cycles of code changes done after testing.

Behavior Driven Tests are built out of a set of Feature files, which describe product features through the expected application behavior in one or many Scenarios. Each Scenario is built out of a sequence of steps which represent actions or verifications that need to be tested for that Scenario.

BDD focuses on expected application behavior, not on implementation details. Therefore BDD tests are described in a human-readable Domain Specific Language (DSL). As this language is not technical, such tests can be created not only by programmers, but also by product owners, testers or business analysts. Additionally, during the product development, such tests serve as living product documentation. For Squish usage, BDD tests shall be created using Gherkin syntax. The previously written product specification (BDD tests) can be turned into executable tests. This step by step tutorial presents automating BDD tests with Squish IDE support.

Gherkin syntax

Gherkin files describe product features through the expected application behavior in one or many Scenarios. An example showing the "Filling of addressbook" feature of the addressbook example application.

Feature: Filling of addressbook
    As a user I want to fill the addressbook with entries

    Scenario: Initial state of created address book
        Given addressbook application is running
        When I create a new addressbook
        Then addressbook should have zero entries

    Scenario: State after adding one entry
        Given addressbook application is running
        When I create a new addressbook
        And I add a new person 'John','Doe','john@m.com','500600700' to address book
        Then '1' entries should be present

    Scenario: State after adding two entries
        Given addressbook application is running
        When I create a new addressbook
        And I add new persons to address book
            | forename  | surname  | email        | phone   |
            | John      | Smith    | john@m.com   | 1231231 |
            | Alice     | Thomson  | alice@m.com  | 2342342 |
        Then '2' entries should be present

    Scenario: Forename and surname is added to table
        Given addressbook application is running
        When I create a new addressbook
        When I add a new person 'Bob','Doe','Bob@m.com','123321231' to address book
        Then previously entered forename and surname shall be at the top

Most of the above is free form text (does not have to be English). It's just the Feature/Scenario structure and the leading keywords like Given, And, When and Then that are fixed. Each of those keywords marks a step defining preconditions, user actions or expected results. The above application behavior description can be passed to software developers to implement these features and at the same time the same description can be used by software testers to implement automated tests.

Test implementation

Creating Test Suite

First, we need to create a Test Suite, which is a container for all Test Cases. Start the squishide and select File > New Test Suite. Please follow the New Test Suite wizard, provide a Test Suite name, choose the Web Toolkit and scripting language of your choice. Please refer to Creating a Test Suite for more details about creating new Test Suites.

Creating Test Case

Squish offers two types of Test Cases: "Script Test Case" and "BDD Test Case". As "Script Test Case" is the default one, in order to create new "BDD Test Case" we need to use drop-down menu by clicking on the expander next to New Script Test Case ( ) and choosing the option New BDD Test Case. The Squish IDE will remember your choice and the "BDD Test Case" will become the default when clicking on the button in the future.

The newly created BDD Test Case consists of a test.feature file (filled with a Gherkin template while creating a new BDD test case), a file named test.(py|js|pl) which will drive the execution (there is no need to edit this file), and a Test Suite Resources file named steps/steps.(py|js|pl) where step implementation code will be placed.

We need to replace the Gherkin template with a Feature for the addressbook example application. To do this, copy the Feature description below and paste it into the Feature file.

Feature: Filling of addressbook
    As a user I want to fill the addressbook with entries

    Scenario: Initial state of created address book
        Given addressbook application is running
        When I create a new addressbook
        Then addressbook should have zero entries

When editing the test.feature file, a Feature file warning No implementation found is displayed for each undefined step. The implementations are in the steps subdirectory, in Test Case Resources, or in Test Suite Resources. Running our Feature test now will currently fail at the first step with a No Matching Step Definition and the following steps will be skipped.

Recording Step implementation

In order to record the Scenario, press the Record button next to the respective Scenario that is listed in the Scenarios tab in Test Case Resources view.

This will cause the Squish IDE to go away, and a dialog to pop up asking you for the starting URL of the web browser. Enter http://localhost:9090/AddressBook.html so the browser can start. Additionally, the Control Bar is displayed with a list of all steps that need to be recorded. Now all interaction with the AUT or any verification points added to the script will be recorded under the first step Given addressbook application is running (which is bolded in the Step list on the Control Bar). In order to verify that this precondition is met, we will add a Verification Point. To do this, click on Verify in the Control Bar and select Properties.

As a result the Squish IDE is put into Spy mode which displays all Application Objects and Properties in dockable Views near the bottom of the Squish IDE. In the Application Objects, select the DOCUMENT object. Selecting it will update the Properties view to its right. Next click on the checkbox in front of the property title in the Properties View. Finally, click on the button Save and Insert Verifications. The Squish IDE disappears and the Control Bar is shown again.

When we are done with each step, we can move to the next undefined step (playing back the ones that were previously defined) by clicking on the Finish Recording Step ( ) arrow button in the Control Bar that is located to the left of the current step.

Next, for the step When I create a new addressbook click on the New button and click on the Finish Recording Step ( ).

Finally, for the step Then addressbook should have zero entries verify that the table containing the address entries is empty. To record this verification, click on Verify while recording, select Properties and then select the body of the table in the Application Objects view. See below for the location of the tbody element, since it is deeply nested. Check the numChildren entry from the Properties view (It has a value of 1 since there is always an extra element in the table). Then, click on Save and Insert Verifications. Finally, click on the last Finish Recording Step ( ) arrow button in the Control Bar.

As a result, Squish will generate the following step definitions in the steps.* file at Test Suites > Test Suite Resources:

@Given("addressbook application is running")
def step(context):
    startBrowser("http://127.0.0.1:9090/AddressBook.html")
    test.compare(waitForObjectExists(names.dOCUMENT).title, "froglogic Addressbook")

@When("I create a new addressbook")
def step(context):
    clickButton(waitForObject(names.froglogic_Addressbook_newButton_button))
    snooze(2)
    closeConfirm(names.confirmPopup, True)

@Then("addressbook should have zero entries")
def step(context):
    test.compare(waitForObjectExists(names.dOCUMENT_HTML1_BODY1_DIV1_DIV2_DIV3_DIV3_DIV1_TABLE1_TBODY1).numChildren, 1)

Given("addressbook application is running", function(context) {
    startBrowser("http://127.0.0.1:9090/AddressBook.html");
    test.compare(waitForObjectExists(names.dOCUMENT).title, "froglogic Addressbook");
});

When("I create a new addressbook", function(context) {
    clickButton(waitForObject(names.froglogicAddressbookNewButtonButton));
    snooze(2);
    closeConfirm(names.confirmPopup, true);
});

Then("addressbook should have zero entries", function(context) {
    test.compare(waitForObjectExists(names.dOCUMENTHTML1BODY1DIV1DIV2DIV3DIV3DIV1TABLE1TBODY1).numChildren, 1);
});

Given("addressbook application is running", sub {
    my $context = shift;
    startBrowser("http://127.0.0.1:9090/AddressBook.html");
    test::compare(waitForObjectExists($Names::document)->title, "froglogic Addressbook");
});

When("I create a new addressbook", sub {
    my $context = shift;
    clickButton(waitForObject($Names::froglogic_addressbook_newbutton_button));
    snooze(2);
    closeConfirm($Names::confirmpopup, 1);
});

Then("addressbook should have zero entries", sub {
    my $context = shift;
    test::compare(waitForObjectExists($Names::document_html1_body1_div1_div2_div3_div3_div1_table1_tbody1)->numChildren, 1);
});

Given("addressbook application is running") do |context|
    startBrowser("http://127.0.0.1:9090/AddressBook.html")
    Test.compare(waitForObjectExists(Names::DOCUMENT).title, "froglogic Addressbook")
end

When("I create a new addressbook") do |context|
    clickButton(waitForObject(Names::Froglogic_Addressbook_newButton_button))
    snooze(2)
    closeConfirm(Names::ConfirmPopup, true)
end

Then("addressbook should have zero entries") do |context|
    Test.compare(waitForObjectExists(Names::DOCUMENT_HTML1_BODY1_DIV1_DIV2_DIV3_DIV3_DIV1_TABLE1_TBODY1).numChildren, 1)
end

Given "addressbook application is running" {context} {
    invoke startBrowser "http://127.0.0.1:9090/AddressBook.html"
    test compare [property get [waitForObject $names::DOCUMENT] title] "froglogic Addressbook"
}

When "I create a new addressbook" {context} {
    invoke clickButton [waitForObject $names::froglogic_Addressbook_newButton_button]
    snooze 2
    invoke closeConfirm $names::ConfirmPopup true
}

Then "addressbook should have zero entries" {context} {
    test compare [property get [waitForObjectExists $names::DOCUMENT_HTML1_BODY1_DIV1_DIV2_DIV3_DIV3_DIV1_TABLE1_TBODY1] numChildren] 1
}

The application is automatically started at the beginning of the first step due to the recorded startApplication() call. At the end of each Scenario, the OnScenarioEnd hook is called, causing detach() to be called on the application context. Because the AUT was started with startApplication(), this causes it to terminate. This hook function is found in the file bdd_hooks.(py|js|pl|rb|tcl), which is located in the Scripts tab of the Test Suite Resources view. You can define additional hook functions here. For a list of all available hooks, please refer to Performing Actions During Test Execution Via Hooks.

@OnScenarioEnd
def hook(context):
    closeWindow(":[Window]");

OnScenarioEnd(function(context) {
    closeWindow(":[Window]");
});

OnScenarioEnd(sub {
    closeWindow(":[Window]");
});

OnScenarioEnd do |context|
  closeWindow(":[Window]");
end

OnScenarioEnd { context } {
    invoke closeWindow ":\[Window\]"
    }
}

Step parametrization

So far, our steps did not use any parameters and all values were hardcoded. Squish has different types of parameters like any, integer or word, allowing our step definitions to be more reusable. Let us add a new Scenario to our Feature file which will provide step parameters for both the Test Data and the expected results. Copy the below section into your Feature file.

Scenario Outline: Adding single entries multiple time
    Given addressbook application is running
    When I create a new addressbook
    And I add a new person '<forename>','<surname>','<email>','<phone>' to address book
    Then '1' entries should be present
    Examples:
        | forename | surname  | email       | phone     |
        | John     | Doe      | john@m.com  | 500600700 |
        | Bob      | Koo      | bob@m.com   | 500600800 |

After auto-saving the Feature file, the Squish IDE provides a hint that only 2 steps need to be implemented: When I add a new person 'John', 'Doe','john@m.com','500600700' to address book and Then '1' entries should be present. The remaining steps already have a matching step implementation.

To record the missing steps, hit the record button next to the test case name in the Test Suites view. The script will play until it gets to the missing step and then prompt you to implement it. If you select the Add button, then you can type in the information for a new entry. Click on the Finish Recording Step ( ) button to move to the next step. For the second missing step, we could record an object property verification like we did with the step Then addressbook should have zero entries. Or we could copy that step's implementation in the steps.(py|js|pl|rb|tcl) file and increment the number at the end of the test.compare line. Instead of testing for zero items, we are testing for one item.

Now we parametrize the generated step implementation by replacing the values with parameter types. Since we want to be able to add different names, replace 'John' with '|word|'. Note that each parameter will be passed to the step implementation function in the order of appearance in the descriptive name of the step. Finish parametrizing by editing the typed values into keywords, to look like this example step When I add a new person 'John', 'Doe','john@m.com','500600700' to address book:

@When("I add a new person '|word|','|word|','|any|','|integer|' to address book")
def step(context, forename, surname, email, phone):
    clickButton(waitForObject(names.froglogic_Addressbook_addButton_button))
    setText(waitForObject(names.froglogic_Addressbook_forenameEdit_text), forename)
    setText(waitForObject(names.froglogic_Addressbook_surnameEdit_text), surname)
    setText(waitForObject(names.froglogic_Addressbook_emailEdit_text), email)
    setText(waitForObject(names.froglogic_Addressbook_phoneEdit_text), phone)
    clickButton(waitForObject(names.froglogic_Addressbook_Save_button))

When("I add a new person '|word|','|word|','|any|','|integer|' to address book",
    function(context, forename, surname, email, phone) {
    clickButton(waitForObject(names.froglogicAddressbookAddButtonButton));
    mouseClick(waitForObject(names.froglogicAddressbookAddButtonButton));
    setText(waitForObject(names.froglogicAddressbookForenameEditText), forename);
    setText(waitForObject(names.froglogicAddressbookSurnameEditText), surname);
    setText(waitForObject(names.froglogicAddressbookEmailEditText), email);
    setText(waitForObject(names.froglogicAddressbookPhoneEditText), phone);
    clickButton(waitForObject(names.froglogicAddressbookSaveButton));

When("I add a new person '|word|','|word|','|any|','|integer|' to address book", sub {
    my $context = shift;
    my ($forename, $surname, $email, $phone) = @_;
    clickButton(waitForObject($Names::froglogic_addressbook_addbutton_button));
    setText(waitForObject($Names::froglogic_addressbook_forenameedit_text), $forename);
    setText(waitForObject($Names::froglogic_addressbook_surnameedit_text), $surname);
    setText(waitForObject($Names::froglogic_addressbook_emailedit_text), $email);
    setText(waitForObject($Names::froglogic_addressbook_phoneedit_text), $phone);
    clickButton(waitForObject($Names::froglogic_addressbook_save_button));

When("I add a new person '|word|','|word|','|any|','|integer|' to address book") do |context, forename, surname, email, phone|
    clickButton(waitForObject(Names::Froglogic_Addressbook_addButton_button))
    setText(waitForObject(Names::Froglogic_Addressbook_forenameEdit_text), forename)
    setText(waitForObject(Names::Froglogic_Addressbook_surnameEdit_text), surname)
    setText(waitForObject(Names::Froglogic_Addressbook_emailEdit_text), email)
    setText(waitForObject(Names::Froglogic_Addressbook_phoneEdit_text), phone)
    clickButton(waitForObject(Names::Froglogic_Addressbook_Save_button))

When "I add a new person '|word|','|word|','|any|','|integer|' to address book" {context forename surname email phone} {
    invoke clickButton [waitForObject $names::froglogic_Addressbook_addButton_button]
    invoke setText [waitForObject $names::froglogic_Addressbook_forenameEdit_text] $forename
    invoke setText [waitForObject $names::froglogic_Addressbook_surnameEdit_text] $surname
    invoke setText [waitForObject $names::froglogic_Addressbook_emailEdit_text] $email
    invoke setText [waitForObject $names::froglogic_Addressbook_phoneEdit_text] $phone
    invoke clickButton [waitForObject $names::froglogic_Addressbook_Save_button]

If we recorded the final Then as a missing step, and verified the numChildren is 1 in the <TBODY>, we can modify the step so that it takes a parameter, so it can verify other integer values later.

@Then("'|integer|' entries should be present")
def step(context, entries):
    test.compare(waitForObjectExists(names.dOCUMENT_HTML1_BODY1_DIV1_DIV2_DIV3_DIV3_DIV1_TABLE1_TBODY1).numChildren, entries + 1)

Then("'|integer|' entries should be present", function(context, entries) {
    test.compare(waitForObjectExists(names.dOCUMENTHTML1BODY1DIV1DIV2DIV3DIV3DIV1TABLE1TBODY1).numChildren, entries + 1);
});

Then("'|integer|' entries should be present", sub {
    my $context = shift;
    my $entries = shift;
    test::compare(waitForObjectExists($Names::document_html1_body1_div1_div2_div3_div3_div1_table1_tbody1)->numChildren, $entries + 1);
});

Then("'|integer|' entries should be present") do |context, entries|
    Test.compare(waitForObjectExists(Names::DOCUMENT_HTML1_BODY1_DIV1_DIV2_DIV3_DIV3_DIV1_TABLE1_TBODY1).numChildren, entries + 1)
end

Then "'|integer|' entries should be present" {context entries} {
    test compare [property get [waitForObjectExists $names::DOCUMENT_HTML1_BODY1_DIV1_DIV2_DIV3_DIV3_DIV1_TABLE1_TBODY1] numChildren] [expr $entries + 1]
}

Provide parameters for Step in table

The next Scenario will test adding multiple entries to the address book. We could use step When I add a new person John','Doe','john@m.com','500600700' to address book multiple times just with different data. But lets instead define a new step called When I add a new person to address book which will handle data from a table.

Scenario: State after adding two entries
    Given addressbook application is running
    When I create a new addressbook
    And I add new persons to address book
        | forename  | surname  | email        | phone   |
        | John      | Smith    | john@m.com   | 1231231 |
        | Alice     | Thomson  | alice@m.com  | 2342342 |
    Then '2' entries should be present

The step implementation to handle such tables looks like this:

@When("I add new persons to address book")
def step(context):
    table = context.table
    # Drop initial row with column headers
    table.pop(0)
    for (forename, surname, email, phone) in table:
        clickButton(waitForObject(names.froglogic_Addressbook_addButton_button))
        setText(waitForObject(names.froglogic_Addressbook_forenameEdit_text), forename)
        setText(waitForObject(names.froglogic_Addressbook_surnameEdit_text), surname)
        setText(waitForObject(names.froglogic_Addressbook_emailEdit_text), email)
        setText(waitForObject(names.froglogic_Addressbook_phoneEdit_text), phone)
        clickButton(waitForObject(names.froglogic_Addressbook_Save_button))

When("I add new persons to address book", function(context) {
    var table = context.table;
    // Skip initial row with column headers by starting at index 1
    for (var i = 1; i < table.length; ++i) {
        var forename = table[i][0];
        var surname = table[i][1];
        var email = table[i][2];
        var phone = table[i][3];
        clickButton(waitForObject(names.froglogicAddressbookAddButtonButton));
        mouseClick(waitForObject(names.froglogicAddressbookAddButtonButton));
        setText(waitForObject(names.froglogicAddressbookForenameEditText), forename);
        setText(waitForObject(names.froglogicAddressbookSurnameEditText), surname);
        setText(waitForObject(names.froglogicAddressbookEmailEditText), email);
        setText(waitForObject(names.froglogicAddressbookPhoneEditText), phone);
        clickButton(waitForObject(names.froglogicAddressbookSaveButton));
    }
});

When("I add new persons to address book", sub {
    my %context = %{shift()};
    my @table = @{$context{'table'}};

    # Drop initial row with column headers
    shift(@table);

    for my $row (@table) {
        my ($forename, $surname, $email, $phone) = @{$row};
        clickButton(waitForObject($Names::froglogic_addressbook_addbutton_button));
        setText(waitForObject($Names::froglogic_addressbook_forenameedit_text), $forename);
        setText(waitForObject($Names::froglogic_addressbook_surnameedit_text), $surname);
        setText(waitForObject($Names::froglogic_addressbook_emailedit_text), $email);
        setText(waitForObject($Names::froglogic_addressbook_phoneedit_text), $phone);
        clickButton(waitForObject($Names::froglogic_addressbook_save_button));
    }
});

When("I add new persons to address book") do |context|
    table = context.table
    # Drop initial row with column headers
    table.shift
    for forename, surname, email, phone in table do
        clickButton(waitForObject(Names::Froglogic_Addressbook_addButton_button))
        setText(waitForObject(Names::Froglogic_Addressbook_forenameEdit_text), forename)
        setText(waitForObject(Names::Froglogic_Addressbook_surnameEdit_text), surname)
        setText(waitForObject(Names::Froglogic_Addressbook_emailEdit_text), email)
        setText(waitForObject(Names::Froglogic_Addressbook_phoneEdit_text), phone)
        clickButton(waitForObject(Names::Froglogic_Addressbook_Save_button))
    end
end

When "I add new persons to address book" {context} {
    set table [$context table]
    # Drop initial row with column headers
    foreach row [lreplace $table 0 0] {
        foreach {forename surname email phone} $row break
        invoke clickButton [waitForObject $names::froglogic_Addressbook_addButton_button]
        invoke setText [waitForObject $names::froglogic_Addressbook_forenameEdit_text] $forename
        invoke setText [waitForObject $names::froglogic_Addressbook_surnameEdit_text] $surname
        invoke setText [waitForObject $names::froglogic_Addressbook_emailEdit_text] $email
        invoke setText [waitForObject $names::froglogic_Addressbook_phoneEdit_text] $phone
        invoke clickButton [waitForObject $names::froglogic_Addressbook_Save_button]
    }
}

Sharing data between Steps and Scenarios

Lets add a new Scenario to the Feature file. This time we would like to check not the number of entries in address book list, but if this list contains proper data. Because we enter data into the address book in one step and verify them in another, we must share information about entered data among those steps in order to perform a verification.

Scenario: Forename and surname is added to table
    Given addressbook application is running
    When I create a new addressbook
    When I add a new person 'Bob','Doe','Bob@m.com','123321231' to address book
    Then previously entered forename and surname shall be at the top

To share this data, context.userData can be used.

@When("I add a new person '|word|','|word|','|any|','|integer|' to address book")
def step(context, forename, surname, email, phone):
    clickButton(waitForObject(names.froglogic_Addressbook_addButton_button))
    setText(waitForObject(names.froglogic_Addressbook_forenameEdit_text), forename)
    setText(waitForObject(names.froglogic_Addressbook_surnameEdit_text), surname)
    setText(waitForObject(names.froglogic_Addressbook_emailEdit_text), email)
    setText(waitForObject(names.froglogic_Addressbook_phoneEdit_text), phone)
    clickButton(waitForObject(names.froglogic_Addressbook_Save_button))
    # save userData for last step
    context.userData = {}
    context.userData['forename'] = forename
    context.userData['surname'] = surname

When("I add a new person '|word|','|word|','|any|','|integer|' to address book",
    function(context, forename, surname, email, phone) {
    clickButton(waitForObject(names.froglogicAddressbookAddButtonButton));
    mouseClick(waitForObject(names.froglogicAddressbookAddButtonButton));
    setText(waitForObject(names.froglogicAddressbookForenameEditText), forename);
    setText(waitForObject(names.froglogicAddressbookSurnameEditText), surname);
    setText(waitForObject(names.froglogicAddressbookEmailEditText), email);
    setText(waitForObject(names.froglogicAddressbookPhoneEditText), phone);
    clickButton(waitForObject(names.froglogicAddressbookSaveButton));
    // save userData for last step
    context.userData["forename"] = forename;
    context.userData["surname"] = surname;
});

When("I add a new person '|word|','|word|','|any|','|integer|' to address book", sub {
    my $context = shift;
    my ($forename, $surname, $email, $phone) = @_;
    clickButton(waitForObject($Names::froglogic_addressbook_addbutton_button));
    setText(waitForObject($Names::froglogic_addressbook_forenameedit_text), $forename);
    setText(waitForObject($Names::froglogic_addressbook_surnameedit_text), $surname);
    setText(waitForObject($Names::froglogic_addressbook_emailedit_text), $email);
    setText(waitForObject($Names::froglogic_addressbook_phoneedit_text), $phone);
    clickButton(waitForObject($Names::froglogic_addressbook_save_button));
    # save userData for last step
    $context->{userData}{'forename'} = $forename;
    $context->{userData}{'surname'} = $surname;
});

When("I add a new person '|word|','|word|','|any|','|integer|' to address book") do |context, forename, surname, email, phone|
    clickButton(waitForObject(Names::Froglogic_Addressbook_addButton_button))
    setText(waitForObject(Names::Froglogic_Addressbook_forenameEdit_text), forename)
    setText(waitForObject(Names::Froglogic_Addressbook_surnameEdit_text), surname)
    setText(waitForObject(Names::Froglogic_Addressbook_emailEdit_text), email)
    setText(waitForObject(Names::Froglogic_Addressbook_phoneEdit_text), phone)
    clickButton(waitForObject(Names::Froglogic_Addressbook_Save_button))
    # save userData for last step
    context.userData = Hash.new
    context.userData[:forename] = forename
    context.userData[:surname] = surname
end

When "I add a new person '|word|','|word|','|any|','|integer|' to address book" {context forename surname email phone} {
    invoke clickButton [waitForObject $names::froglogic_Addressbook_addButton_button]
    invoke setText [waitForObject $names::froglogic_Addressbook_forenameEdit_text] $forename
    invoke setText [waitForObject $names::froglogic_Addressbook_surnameEdit_text] $surname
    invoke setText [waitForObject $names::froglogic_Addressbook_emailEdit_text] $email
    invoke setText [waitForObject $names::froglogic_Addressbook_phoneEdit_text] $phone
    invoke clickButton [waitForObject $names::froglogic_Addressbook_Save_button]
    # save userData for last step
    $context userData [dict create forename $forename surname $surname]
}

All data stored in context.userData can be accessed in all steps and Hooks in all Scenarios of the given Feature. Finally, we need to implement the step Then previously entered forename and surname shall be at the top.

@Then("previously entered forename and surname shall be at the top")
def step(context):
    test.compare(waitForObjectExists("DOCUMENT.HTML1.BODY1.DIV1.DIV2.DIV3.DIV3.DIV1.TABLE1.TBODY1.TR2.TD2").innerText,
                 context.userData['forename'])
    test.compare(waitForObjectExists("DOCUMENT.HTML1.BODY1.DIV1.DIV2.DIV3.DIV3.DIV1.TABLE1.TBODY1.TR2.TD3").innerText,
                 context.userData['surname'])

Then("previously entered forename and surname shall be at the top", function(context) {
    test.compare(waitForObjectExists("DOCUMENT.HTML1.BODY1.DIV1.DIV2.DIV3.DIV3.DIV1.TABLE1.TBODY1.TR2.TD2").innerText,
        context.userData["forename"], "forename?");
    test.compare(waitForObjectExists("DOCUMENT.HTML1.BODY1.DIV1.DIV2.DIV3.DIV3.DIV1.TABLE1.TBODY1.TR2.TD3").innerText,
        context.userData["surname"], "surname?");
});

Then("previously entered forename and surname shall be at the top", sub {
    my $context = shift;
    test::compare( waitForObject("DOCUMENT.HTML1.BODY1.DIV1.DIV2.DIV3.DIV3.DIV1.TABLE1.TBODY1.TR2.TD2")->innerText,
        $context->{userData}{'forename'}, "forename?" );
    test::compare( waitForObject("DOCUMENT.HTML1.BODY1.DIV1.DIV2.DIV3.DIV3.DIV1.TABLE1.TBODY1.TR2.TD3")->innerText,
        $context->{userData}{'surname'}, "surname?" );
});

Then("previously entered forename and surname shall be at the top") do |context|
    Test.compare(waitForObjectExists("DOCUMENT.HTML1.BODY1.DIV1.DIV2.DIV3.DIV3.DIV1.TABLE1.TBODY1.TR2.TD2").innerText,
        context.userData[:forename], "forename?")
    Test.compare(waitForObjectExists("DOCUMENT.HTML1.BODY1.DIV1.DIV2.DIV3.DIV3.DIV1.TABLE1.TBODY1.TR2.TD3").innerText,
        context.userData[:surname], "surname?")
end

Then "previously entered forename and surname shall be at the top" {context} {
    test compare [property get [waitForObjectExists "DOCUMENT.HTML1.BODY1.DIV1.DIV2.DIV3.DIV3.DIV1.TABLE1.TBODY1.TR2.TD2"] innerText] [dict get [$context userData] forename]
    test compare [property get [waitForObjectExists "DOCUMENT.HTML1.BODY1.DIV1.DIV2.DIV3.DIV3.DIV1.TABLE1.TBODY1.TR2.TD3"] innerText] [dict get [$context userData] surname]
}

Scenario Outline

Assume our Feature contains the following two Scenarios:

Scenario: State after adding one entry
    Given addressbook application is running
    When I create a new addressbook
    And I add a new person 'John','Doe','john@m.com','500600700' to address book
    Then "1" entries should be present

Scenario: State after adding one entry
    Given addressbook application is running
    When I create a new addressbook
    And I add a new person 'Bob','Koo','bob@m.com','500600800' to address book
    Then "1" entries should be present

As we can see, those Scenarios perform the same actions using different test data. The same can be achieved by using a Scenario Outline (a Scenario template with placeholders) and Examples (a table with parameters).

Scenario Outline: Adding single entries multiple time
    Given addressbook application is running
    When I create a new addressbook
    And I add a new person '<forename>','<surname>','<email>','<phone>' to address book
    Then '1' entries should be present
    Examples:
        | forename | surname  | email       | phone     |
        | John     | Doe      | john@m.com  | 500600700 |
        | Bob      | Koo      | bob@m.com   | 500600800 |

Please note that the OnScenarioEnd hook will be executed at the end of each loop iteration in a Scenario Outline.

Test execution

In the Squish IDE, users can execute all Scenarios in a Feature, or execute only one selected Scenario. In order to execute all Scenarios, the proper Test Case has to be executed by clicking on the Play button in the Test Suites view.

Execute all Scenarios from Feature

In order to execute only one Scenario, you need to open the Feature file, right-click on the given Scenario and choose Run Scenario. An alternative approach is to click on the Play button next to the respective Scenario in the Scenarios tab in Test Case Resources.

Execute one Scenario from Feature

After a Scenario is executed, the Feature file is colored according to the execution results. More detailed information (like logs) can be found in the Test Results View.

Execution results in Feature file

Test debugging

Squish offers the possibility to pause an execution of a Test Case at any point in order to check script variables, spy application objects or run custom code in the Squish Script Console. To do this, a breakpoint has to be placed before starting the execution, either in the Feature file at any line containing a step or at any line of executed code (i.e., in the middle of step definition code).

Breakpoint in Feature file

After the breakpoint is reached, you can inspect all application objects and their properties. If a breakpoint is placed at a step definition or a hook is reached, then you can additionally add Verification Points or record code snippets.

Re-using Step definitions

BDD test maintainability can be increased by reusing step definitions in test cases located in another directory. For more information, see collectStepDefinitions().

Tutorial: Migration of existing tests to BDD

This chapter for users that have existing script-based tests and who would like to introduce Behavior Driven Testing. The first section describes how to keep the existing tests and simply add new tests with the BDD approach. The second section describes how to convert existing tests to BDD.

Extend existing tests to BDD

The first option is to keep any existing script-based Squish tests and extend them by adding new BDD tests. A Test Suite can contain both script-based and BDD Test Cases. Simply open an existing Test Suite with and choose New BDD Test Case from the drop down menu of the down-arrow button to the right of New Script Test Case ( ).

Creating new BDD Test Case

Assuming your existing Test Cases make use of a library and you are calling shared functions to interact with the AUT, those functions can still be used from BDD Test Cases. In the example below, these functions are used from multiple script-based Test Cases:

def numberOfRows():
    table = waitForObject(names.dOCUMENT_HTML1_BODY1_DIV1_DIV2_DIV3_DIV3_DIV1_TABLE1)
    results = table.evaluateXPath(".//TR[contains(@class, 'jqgrow')]")
    return results.snapshotLength

def createNewAddressBook():
    clickButton(waitForObject(names.froglogic_Addressbook_newButton_button))
    closeConfirm(names.confirmPopup, True)

function numberOfRows() {
    var table = waitForObject("DOCUMENT.HTML1.BODY1.DIV1.DIV2.DIV3.DIV3.DIV1.TABLE1");
    var results = table.evaluateXPath(".//TR[contains(@class, 'jqgrow')]");
    return results.snapshotLength;
}

function createNewAddressBook(){
    clickButton(waitForObject(names.froglogicAddressbookNewButtonButton));
    closeConfirm(names.confirmPopup, true);
}

sub numberOfRows
{
    my $table = waitForObject($Names::document_html1_body1_div1_div2_div3_div3_div1_table1);
    my $results = $table->evaluateXPath(".//TR[contains(\@class, 'jqgrow')]");
    return $results->snapshotLength;
}

sub createNewAddressBook{
    clickButton(waitForObject($Names::froglogic_addressbook_newbutton_button));
    closeConfirm($Names::confirmpopup, 1);
}

def numberOfRows
    table = waitForObject(Names::DOCUMENT_HTML1_BODY1_DIV1_DIV2_DIV3_DIV3_DIV1_TABLE1)
    results = table.evaluateXPath(".//TR[contains(@class,'jqgrow')]")
    results.snapshotLength
end

def createNewAddressBook
    clickButton(waitForObject(Names::Froglogic_Addressbook_newButton_button))
    closeConfirm(Names::ConfirmPopup, true)
end

proc numberOfRows {} {
    set table [waitForObject \
        $names::DOCUMENT_HTML1_BODY1_DIV1_DIV2_DIV3_DIV3_DIV1_TABLE1]
    set results [invoke $table evaluateXPath {.//TR[contains(@class,'jqgrow')]}]
    return [property get $results snapshotLength]
}

proc createNewAddressBook {} {
    invoke clickButton [waitForObject $names::froglogic_Addressbook_newButton_button]
    invoke closeConfirm $names::ConfirmPopup true
}

New BDD Test Cases can easily use the same function:

@When("I create a new addressbook")
def step(context):
    createNewAddressBook()
    test.verify(numberOfRows() == 0)

When("I create a new addressbook", function(context){
    createNewAddressBook()
    test.verify(numberOfRows() == 0);
});

When("I create a new addressbook", sub {
    createNewAddressBook();
    test::verify(numberOfRows() == 0);
});

When("I create a new addressbook") do |context|
    createNewAddressBook
    Test.verify(numberOfRows == 0)
end

When "I create a new addressbook" {context} {
        createNewAddressBook
        test compare [numberOfRows] 0
}

Convert existing tests to BDD

The second option is to convert script-based test cases from an existing Test Suite into behavior driven tests. Since a Test Suite can contain script-based and BDD Test Cases, migration can be done gradually. A Test Suite containing a mix of both Test Case types can be executed and results analyzed without any extra effort required.

The first step is to review all Test Cases of the existing Test Suite and group them by the Feature they test. Each script-based Test Case will be transformed into a Scenario, which is a part of a Feature. For example, assume we have 5 script-based Test Cases. After review, we realize that they examine two Features. Therefore, when migration is completed, our Test Suite will contain two BDD Test Cases, each of them containing one Feature. Each Feature will contain multiple Scenarios. In our example the first Feature contains three Scenarios and the second Feature contains two Scenarios.

Conversion Chart

At the beginning, open a Test Suite in the Squish IDE that contains Squish tests that are planned to be migrated to BDD. Next, create a New Script Test Case ( ) by choosing New BDD Test Case option from its context menu. Each BDD Test Case contains a test.feature file that can be filled with maximum one Feature. Next, open the test.feature file to describe the Features using the Gherkin language. Following the syntax from the template, edit the Feature name and optionally provide a short description. Next, analyze which actions and verifications are performed in the Test Case that are going to be migrated. This is how an example Test Case for the addressbook application might start:

def main():
    startBrowser("http://127.0.0.1:9090/AddressBook.html")
    confirmPopup(names.newButton_button)
    test.verify(numberOfRows() == 0, "%d" % numberOfRows())

function main(){
    startBrowser("http://localhost:9090/AddressBook.html");
    confirmPopup(names.newButtonButton);
    test.verify(numberOfRows() == 0);
}

sub main {
    startBrowser("http://127.0.0.1:9090/AddressBook.html");
    confirmPopup($Names::newbutton_button);
    test::verify(numberOfRows() == 0);
}

def main
    startBrowser("http://127.0.0.1:9090/AddressBook.html")
    confirmPopup(Names::NewButton_button)
    Test.verify(numberOfRows == 0)
end

proc main {} {
    invoke startBrowser "http://127.0.0.1:9090/AddressBook.html"
    confirmPopup $names::newButton_button
    test compare [numberOfRows] 0
}

After analyzing the above Test Case we can create the following Scenario and add it to test.feature file:

Scenario: Initial state of created address book
      Given addressbook application is running
      When I create a new addressbook
      Then addressbook should have zero entries

Next, right-click on the Scenario and choose the option Create Missing Step Implementations from the context menu. This will create a skeleton of steps definitions:

@Given("addressbook application is running")
def step(context):
    test.warning("TODO implement addressbook application is running")

@When("I create a new addressbook")
def step(context):
    test.warning("TODO implement I create a new addressbook")

@Then("addressbook should have zero entries")
def step(context):
    test.warning("TODO implement addressbook should have zero entries")

Given("addressbook application is running", function(context) {
    test.warning("TODO implement addressbook application is running");
});

When("I create a new addressbook", function(context) {
    test.warning("TODO implement I create a new addressbook");
});

Then("addressbook should have zero entries", function(context) {
    test.warning("TODO implement addressbook should have zero entries");
});

Given("addressbook application is running", sub {
    my $context = shift;
    test::warning("TODO implement addressbook application is running");
});

When("I create a new addressbook", sub {
    my $context = shift;
    test::warning("TODO implement I create a new addressbook");
});

Then("addressbook should have zero entries", sub {
    my $context = shift;
    test::warning("TODO implement addressbook should have zero entries");
});

Given("addressbook application is running") do |context|
    Test.warning "TODO implement addressbook application is running"
end

When("I create a new addressbook") do |context|
    Test.warning "TODO implement I create a new addressbook"
end

Then("addressbook should have zero entries") do |context|
    Test.warning "TODO implement addressbook should have zero entries"
end

Given "addressbook application is running" {context} {
    test warning "TODO implement addressbook application is running"
}

When "I create a new addressbook" {context} {
    test warning "TODO implement I create a new addressbook"
}

Then "addressbook should have zero entries" {context} {
    test warning "TODO implement addressbook should have zero entries"
}

Now we put code snippets from the script-based Test Case into respective step definitions and remove the lines containing test.warning. If your Test Cases make use of shared scripts, you can call those functions from the step definitions as well. For example, the final result could look like this:

@Given("addressbook application is running")
def step(context):
    startBrowser("http://127.0.0.1:9090/AddressBook.html")
    test.compare(waitForObjectExists("DOCUMENT").title, "froglogic Addressbook")

@When("I create a new addressbook")
def step(context):
    confirmPopup(names.newButton_button)

@Then("addressbook should have zero entries")
def step(context):
    test.verify(numberOfRows() == 0)

Given("addressbook application is running", function(context) {
    startBrowser("http://127.0.0.1:9090/AddressBook.html");
    test.compare(waitForObjectExists("DOCUMENT").title, "froglogic Addressbook");
});

When("I create a new addressbook", function(context) {
    confirmPopup(names.newButtonButton);
});

Then("addressbook should have zero entries", function(context) {
    test.verify(numberOfRows() == 0);
});

Given("addressbook application is running", sub {
    my $context = shift;
    startBrowser("http://127.0.0.1:9090/AddressBook.html");
    test::compare(waitForObjectExists("DOCUMENT")->title, "froglogic Addressbook");
});

When("I create a new addressbook", sub {
    my $context = shift;
    confirmPopup($Names::newbutton_button);
});

Then("addressbook should have zero entries", sub {
    my $context = shift;
    test::verify(numberOfRows() == 0);
});

Given("addressbook application is running") do |context|
    startBrowser("http://127.0.0.1:9090/AddressBook.html")
    Test.compare(waitForObjectExists("DOCUMENT").title, "froglogic Addressbook")
end

When("I create a new addressbook") do |context|
    confirmPopup(Names::NewButton_button)
end

Then("addressbook should have zero entries") do |context|
    Test.verify(numberOfRows == 0)
end

Given "addressbook application is running" {context} {
    invoke startBrowser "http://127.0.0.1:9090/AddressBook.html"
    test compare [property get [waitForObjectExists "DOCUMENT"] title] "froglogic Addressbook"
}

When "I create a new addressbook" {context} {
    confirmPopup $names::newButton_button
}

Then "addressbook should have zero entries" {context} {
    test compare [numberOfRows] 0
}

Note that the test.log("Create new addressbook”) got removed while migrating this Test Case to BDD. When the step I create a new addressbook is executed, the step name will be logged into Test Results, so the test.log call would have been redundant.

Additionally, when a script-based Test Case execution ends, Squish terminates the AUT. Squish ensures that the AUT is terminated at the end of each Scenario as well. This is done by the auto-generated OnScenarioEnd hook, shown below.

@OnScenarioEnd
def hook(context):
    closeWindow(":[Window]");

OnScenarioEnd(function(context) {
    closeWindow(":[Window]");
});

OnScenarioEnd(sub {
    closeWindow(":[Window]");
});

OnScenarioEnd do |context|
  closeWindow(":[Window]");
end

OnScenarioEnd { context } {
    invoke closeWindow ":\[Window\]"
}

The above example was simplified for this tutorial. In order to take full advantage of Behavior Driven Testing in Squish, please familiarize yourself with the section Behavior Driven Testing in API Reference.