Microsoft Azure DevOps Integration
Squish provides a rich command-line interface, allowing test executions and integrations with third-party tools, like Azure DevOps. Azure DevOps offers several services for augmenting development workflows, including Pipelines which can be used for Continuous Integration (CI) and Continuous Delivery (CD) of your project. Automated GUI tests created with Squish can be easily executed as part of these pipeline processes.
For executing a Squish test as part of a pipeline the Squish Integration extension can be used. It is available on Microsoft marketplace: Squish Integration.
The extension adds two tasks: one to run Squish tests, and another to create Azure DevOps results with the help of Squish Test Center.
The Pipeline Agent
Test execution requires a test environment. For Squish GUI tests, self-hosted agents (e.g., a local PC or laptop) or Virtual Machines (VMs) offered by the Azure cloud can be used. In the case of using the self-hosted solution, the Azure Pipeline agent needs to be installed.
Squish tests require access to a display, either a physical or virtual one. For Windows, the mentioned agent can be started in at least two different ways: as a service or as a process. As services do not have such access, the DevOps agent cannot be launched as a service to run Squish tests. Therefore, the agent has to be started as a process. For more details on running tests on Windows, see Automation on Windows.
The Squish Task can be added to a build or a release Pipeline stage, with the goal to run Squish GUI tests and gather results. To use this task, you must have Squish installed on the agent that will be running the task. Some settings of the task require special attention in order to work properly. For these, we provide additional information in the following section.
- Squish folder (required) - Path to Squish installation folder on the agent.
- Test suites (required) - Path to Squish test suites, one path per line.
- Test cases - Names of test cases to execute from the test suite. Values separated by semi-colon (;).
- Use stdout reportgen - Whether Squish should output the test result to
stdout, which can then be seen in the console of Azure DevOps.
- Use Squish Test Center reportgen - Whether Squish should use the Squish Test Center
reportgen. If selected, additional settings are required.
- Base URL of Squish Test Center instance (required) - URL to reach the Squish Test Center instance that should be the receiver of the results.
- Project (required) - Which Squish Test Center project should the results be pushed in.
- Upload token (required) - Which Squish Test Center upload token to use to push the results.
- Batch name - The name of the batch that will be created with the result. If left empty, then Squish Test Center will automatically generate a new one. Pipeline variables can be used here.
- Labels - List of ;-separated items that will be used to tag the results, in the form of "OS=Windows". Pipeline variables can be used here.
This task can be executed in parallel on different agents, enabling you to have the same set of Squish GUI tests run on multiple environments and, if wanted, have those results pushed to the same batch in Squish Test Center. The state of this task is directly linked to the results of Squish GUI tests: success if all tests passed without failures, warning if some tests contain failures, and failed if an error was encountered.
Squish Test Center Task
This task is targeted for Squish Test Center users of Azure DevOps traceability. Assuming that a mapping exists between Squish GUI tests and Azure DevOps tests in Squish Test Center, then this task can be used to trigger the creation of new runs and results in Azure DevOps based on the mapping. The settings required to run this task are the same that would be needed to do it manually and are listed below:
- Base URL of Squish Test Center instance (required) - URL of the Squish Test Center instance that will be used as the origin of the push action.
- Project (required) - Which Squish Test Center project will be used as source of the mapping.
- Batch name (required) - The name of the batch that contains the results to be pushed into Azure DevOps.
- In the Azure DevOps space, create a Build Pipeline.
We will not cover this step in detail as it depends highly on your project and the way you build your application. In this example, we used Hosted VS2017 Agent pool. The build artifacts (that is, the Application Under Test) are published in Azure Pipelines.
- In the Azure DevOps space, create a Release Pipeline.
The artifacts for the Release Pipeline include:
- The AUT, published in the previous step.
- Squish GUI tests stored in an external GitHub repository.
- Install Squish on the DevOps Agent
This step can be done manually or as part of the Release Pipeline. In the former approach, run the Squish installer on the DevOps Agent and follow the installation wizard.
In the latter approach, first deliver the Squish installer and perform unattended installation as a task in the Release Pipeline.
- Execute Squish tests
Add a Squish Task to the Build/Release Pipeline, and adapt the settings to your setup.
- Push results into Azure DevOps from Squish Test Center.
Push the results for the mentioned batch in Squish Test Center into Azure DevOps, assuming a mapping was previously made between Azure DevOps tests and Squish GUI tests in the Traceability page. Learn more about it in: Integration for Azure DevOps.