Atlassian Bamboo Integration

Atlassian Bamboo is a continuous integration server used to build, test and release software. The Squish Runner plugin provides seamless Squish integration with Bamboo. Users can add one or more Squish Runner tasks to a Bamboo Job and execute Squish GUI tests on designated local or remote agents. After executions finish, Squish results are aggregated and added to the test results and made available in the entire Bamboo Plan (together with other tests, including unit tests). Additionally, artifacts for Squish tests are stored to allow efficient defect troubleshooting.

Atlassian Bamboo Plugin Installation

The Squish GUI Testing for Bamboo plugin is available from Atlassian Marketplace.

To install the Squish Runner plugin, go to Bamboo Administration and choose option Manage Add-ons. In this section click Upload Add-on to upload a jar file with a plugin. When the installation is completed, the Squish plugin is displayed in the user installed Add-ons list.

Bamboo user installed plugins

Squish Capability

An agent is a service that executes Bamboo builds, tests, and deployments. The Agent Capabilities setting allows users to specify which agents are capable of executing Squish GUI tests. If all local agents are able to run Squish tests and the Squish installation directory for all local agents is the same, the squish custom capability should be added to Bamboo administration > BUILD RESOURCES > Server capabilities with the key squish and the value set to the Squish installation path. The squish capability will be inherited by all local agents. If Squish tests are run on multiple local and remote agents, the squish capability must be set for a given agent using Bamboo administration > BUILD RESOURCES > Agents > AgentName > Agent-specific capabilities.

Adding Squish capability

Job Definition

The job, which executes the Squish tests, must be configured as well. On the Requirements tab, in the Add extra requirement section, the requirement squish with the condition exists must be specified. The Requirement setting indicates this particular job can only be built by agents whose capabilities meet this requirement. After the requirement is added, the list of agents capable of executing Squish tests is displayed.

Adding an extra job requirement

Next, define artifacts to be collected after the job finishes. Select the Artifact tab, and click Create definition. Set the artifact name to Squish and the Copy Pattern to squish/**. Select the Miscellaneous tab, and check (enable) the Clean working directory after each build option.

Adding a new artifact definition

Task Definition

A task is a work item which is executed as a part of a job. All tasks in a job are executed one at a time. Multiple Squish Runner tasks can be added to a single job. Other tasks (like unit tests execution) can also be added to the same job. Squish Runner tasks can be found in the Test category.

Adding a new Squish Runner task

Each task must be configured. The Task description name is used to distinguish each task. It is also used as the name in the Squish results for both the XML and HTML formats. The Test suite field defines the full path to the test suite to execute, e.g. /suites/suite_addressbook. To execute multiple test suites, provide each path in a separate line.

The Test cases field determinates which test cases are executed in the given test suite. If the field is empty, all test cases from test suite are executed. Define either a single test case to be executed or multiple test cases, separated by commas (e.g. tst_case1, tst_case2).

The host and port where squishserver is running must be defined. If the Start/stop squishserver option is checked, squishserver starts on the host where the agent is running, and listening on the port configured in the task. The squishserver is stopped when the test execution finishes.

The Environment variables field can be used to define environment variables (for both squishrunner and squishserver). Multiple environment variables can be added separated by a space.

The Extra Options field can be used to specify additional arguments to the squishrunner. Each option and possible arguments have to be given in separate lines.

Configuring the Squish Runner task

Test Results

When the job is finished, the Tests tab displays information about both failed and successful tests as well as their execution duration. Moreover, information is provided about which failed tests are new failures and which tests failed in previous builds.

Similarly, information is provided about which Squish test cases that failed in previous builds were successful in the current build. The Logs tab contains detailed logs from the Squish Runner task, including information pertaining to settings and how the squishserver and squishrunner processes were started.

Job Test Results

The Artifacts tab contains the XML and HTML reports collected after a job finishes. For each Squish Runner task defined in the job, separate XML and corresponding HTML files are created. To improve defect troubleshooting, HTML reports contain embedded screenshots taken by Squish for all instances of test case ERROR or FAILURE.

HTML report from job artifacts