Group results

Test Center groups test results using projects, batches, reports and labels.

Project

In Test Center, projects help you to separate test results that are not related or that are part of separate products. Tests for the same product or feature should be uploaded to the same project. When deciding how to split up your tests by project, keep in mind that Test Center analysis features do not work across project borders.

Batch

Batches and reports help you organize and group your test results.

A batch represents a test execution across one or more test suites and across one or more configurations. If you end up with a single test result file after a test run, e.g. because you execute a single test suite on a single machine, the result file should be uploaded into its own separate batch. If you end up with multiple test result files after a test run, e.g. because you execute multiple test suites or run your tests on multiple machines and configurations, all result files that belong to the same test run should be uploaded to the same batch.

Report

When you upload one or more results to a batch, the results are grouped under a report. While a report can contain test results of one or more test suites, it should only contain test results that were executed on one specific configuration.

Label

A label indicates which configuration the tests of a report have been executed on. For example, if you run your tests on both Windows and Linux, you should upload your results to two separate reports and label one upload with OS=Windows and the other with OS=Linux. If there are other attributes that differentiate the configurations your tests run on, like the version of your AUT or the browser version, you can add a label for each of them.

You can upload all results of a specific configuration to the same report, but sometimes its more convenient to upload each result by itself and end up with many reports for one specific configuration. Both setups should work equally well.

Result grouping example

If your nightly test runs consisted of ten test suites executed across two operating systems and three browser versions, all results of the nightly test run should be uploaded to the same batch.

As a batch name, you should pick something that is easy to identify and unique. For nightly test runs, including the date makes sense, so you would end up with a batch name like this: Nightly 25.02.2020. In our example, the nightly test run is executed across six different configurations, so you should end up with at least six reports, each with a label for the operating system and another for the browser version.

If you distribute the tests across multiple machines, or decide to upload each suite result individually, you could end up with 60 reports in the Nightly 25.02.2020 batch, many of which will share the same sets of labels.