Automated Test Execution

You can now plan, execute and monitor automated tests in Test Center. In the Test Management View you can register tests, import automated test from your repository, or view already registered tests from uploaded results.

With the registered tests in Test Center you can create Test Plan Templates in the Test Plan Templates View, and add Test Runs to the template.

The Test Plan Template can now be scheduled to create a Test Plan Execution, where Test Center will go through the Test Runs and, at the scheduled time, will execute them on the appropriate servers.

Navigate to the Test Executions View to monitor the state and results of the executions.

This is a step-by-step guide on how to configure Test Center to execute automated tests.

Register Tests

Before registering tests, select or create a project where you want to manage the Tests.

There are three ways to make tests known to Test Center:

• Import tests from repository
• Register tests manually
• Uploading test results

Import Tests

You can import test suites and cases from the repository to a connected project. In Test Management View, click the Repository Information button to view the connected repositories to the project. A list of connected repositories will be shown. If the repository is a Git repository, then the respective default branch will be shown. This can be changed to select from which branch the import will take place. Clicking the Import Tests button will start the import process. This might take some seconds, depending on the amount of tests found.

Test Management View - Repositories panel

Register Tests Manually

Add a new Test Suite

Click button to to add a new test suite.

Tests view - Test Management

And in the New Suite dialog, specify a name for the suite, and select Automated test suite

New Suite dialog

Click Create Suite to create the test suite and make it appear in the Tests pane.

Create Test Cases or scenarios

To create tests within a test suite, click next to the test suite entry in the Tests pane, or select a test suite then click + Test Case in the details pane on the right.

Creation of test cases

Create Test Plan Template

After registering automated tests you can now create Test Plan Templates that can be later executed.

After switching to the plan templates tab, you need to click + Test Plan.

Test Plan Templates view

Then you can configure the test plan in the Test Plan Template Details view

Test Plan Template details view.

After creating the test plan you can change its default name by clicking the button, enter the new title, and click Ok.

Rename test plan dialog

You can set or update the description of your test plan by editing the contents of the Description field. Simply click on the description field and enter the new description.

And to assign tags to your test plan, you can edit the textbox

Add a test run to your Test Plan

Click + Test Run button in the Test Plan Details view. and you will go to the Add Test Run view.

Add Test Run view

Select the configuration for the automated test run from the Labels dropdown menu, where you can choose existing label value combinations or create new ones. Typically, you would specify the operating system, browser, or device that a automated test should be executed on. It is important to assign meaningful labels so that Test Center can select an appropriate server based on the configuration, as well to be able to later analyze the results in the Explore and History view.

You then need to select the test suites, or test cases for the test plan execution. The test details pane on the right-hand side displays more information about the selected test.

Clicking the Create Execution button will create the test execution, but stay in the current view. This is so that more Test Runs, or configurations through different labels can be created.

By navigate back to the Test Plan Details view.

Schedule Test Plan Execution

When you have a Test Template, you can schedule its execution and create Test Executions in two ways:

1. Clicking on the + Schedule button in the Test Plan Details view
2. Clicking on the button next to the test plan template name in the Test Plans view

In the Execute Test Plan dialog you need to fill in the mandatory fields: a unique test plan instance name and batch name. For Test Center to execute the automated tests, first you must enable the Schedule Automated Tests toggle, and then select at what time the tests should be executed, by setting the Scheduled Date, and finally you need to specify which repository branch or revision should the tests be executed from.

Execute Test Plan Dialog with enabled Schedule Automated Tests

Monitor Execution Progress

With the Test Plan Execution created, you can navigate to the Test Plan Executions View. In this page you will see the completion percentage of the Test Plan Executions, as well as the results of their respective completed tests.

Entering the Test Plan Details view, by clicking one of the Test Plan Executions, you can view the Completion, Result, and Status of each Test Run.

For even more details, clicking a Test Run will display its Tests, along with their execution Status and Result. If an error occurred during a Test Execution an error will be displayed on the Test Details, along with hints for possible solutions.

Edit Test Run Execution with error.

Execution errors

Test Execution Settings

To enable Test Center to execute Squish tests the following settings need to be configured.

Squish Settings

For Test Center to be able to execute Squish tests, Squish needs to be installed on the same machine Test Center is running on, and the installation directory needs to be setup within the Test Execution tab of the Global Settings

Squish Installation Settings

Squish Server Management

While the test scripts are executed locally on the Test Center instance, the test automation is expected to happen on separate machines, making use of the client server architecture of Squish. That also means the test scripts should be setup to work with a remote squishserver.

The squishserver instances that will be used for the test execution can be managed within the Squish Servers section of the Test Execution tab of the Global Settings.

Squish Server Configuration

Adding a Squish server

To add a new server, you can use the + Add Server button in the top right corner of the Squish Servers panel. This will add a new line at the bottom of the server list, where you can fill in the server details. To finalize adding the server, you need to press the confirm button on the right-hand side of the line that was added. If you want to abort adding the server, you can click on the trashcan icon.

Added server entry with confirm and removal button at the right-hand side.

Configuring Squish server

Any change you make to a server from the server list is only saved when you press the confirm button on the right-hand side of the row, that shows up whenever there are unsaved changes to a server entry.

In the Host and Port columns you can configure the server address. With the Enabled column you can temporarily disable a server, which means it will not be selected by Test Center when scheduling test cases for execution.

In the Capabilities column you configure the capabilities of the server, e.g. the operating system or GUI toolkit version that can be tested with the Squish installation of the server.

Capabilities & Server Assignment

When Test Center schedules tests for execution it selects servers based on their capabilities. The labels used for your planned test runs are the same labels used to denote server capabilities. All labels that share a label key, that is assigned to a server as a capability, are considered capabilities.

When Test Center tries to find a free server to execute a planned test run on, it checks all labels that are assigned to the test run and tries to find a server that matches all of the labels that are capabilities. While all capabilities of a test run need to be satisfied for it to be assigned to a specific server, the server could have additional capabilities that are not required by the test run.

As an example, a server might have the capabilities OS=Windows, qt-version=6.7.2 and OS-Version=11 while a matching test run might have the labels OS=Windows, qt-version=6.7.2 and branch=master. As long as branch is never used as a label key for any server capability it will be ignored when trying to find a suitable server. Furthermore, if the OS-Version is not relevant for the selected test run, then it will also be ignored.

Server Status

The Status column either displays Idle when the server is currently not executing any tests, or it displays Executing when there is a test assigned to the server for execution.