Test Results API

The Test Results API provides access to test execution results organized in a hierarchical structure: Batch → Report → Tests → Events. Batches group test reports from a single build or run. Reports represent individual test executions for a specific configuration (identified by labels). Tests are the individual test items within the hierarchy, and events are the granular log messages produced during execution.

All endpoints are served below the following base URL:

http://<host>:<port>/api/v1

Authorization

All endpoints use one of the following authorization methods described below, using the HTTP Authorization header.

Additionally the custom header AutomationToken = 1a7fcb5e50d630e4d50792acd52745bb4a11bc18 needs to be provided with every request.

Endpoint Overview

Shared Concepts

Common Parameters

Every endpoint requires the project0 query parameter specifying the project name. Most endpoints also require batch0 for the batch name. These are always provided as query parameters.

Result Codes

Test results are represented both as strings and integer codes:

XPASS (unexpected pass) is treated as a failure. XFAIL (expected failure) is treated as a pass.

Event Types

Events produced during test execution can be one of the following types:

ATTACHMENT, ENDVIDEO, ERROR, FAIL, FATAL, LOG, PASS, STARTVIDEO, WARNING, XFAIL, XPASS

Test Hierarchy Types

Tests are organized in a hierarchy. The following types are available and can be used as typename filters:

testsuite, testcase, feature, model-based-test, scenario, scenariooutline, row, step, section, execution

Types at the same hierarchy level are typically homogeneous but can be mixed in rare cases (e.g. a testcase as a root node without a parent testsuite).

Typename Filters

Several endpoints accept typename query parameters to scope results to a specific path in the test hierarchy. The format uses enumerated keys:

testsuite0=suite_thermostat&testcase0=tst_darkmode

The enumerated suffix (e.g. 0, 1) allows specifying cascaded or interleaved types of the same kind.

You can always either use the typename filters or a testid parameter.

Label Filters

Labels are key/value pairs assigned to reports that identify configurations (e.g. OS, architecture, branch). Labels can be used to filter results using enumerated query parameters:

label0=OS=Windows&label1=arch=x64

Multiple label filters are ANDed — reports must match all specified labels.

GET /api/v1/results/summary - Retrieve batch result summary

Retrieves a result summary for a batch of test reports. Provides a high-level overview of test outcomes with minimal test detail — useful for quickly determining whether errors exist and which configurations they affect.

Query Parameters

Response

{
    "batch": {
        "name": "Build c22de0e",
        "annotations": [],
        "failed_tests": {
             "children":[1],
             "descendants":[2,3]
         },
        "results": { "...Results Summary Object..." },
        "time": { "...Time Object..." }
    },
    "reports": [
        {
            "...Report Metadata...",
            "failed_tests": { "children": [], "descendants": [] },
            "results": { "...Results Summary Object..." }
        }
    ]
}

Batch object fields:

Report entries extend the standard Report Metadata with a results field containing a Results Summary Object scoped to that report, and a failed_tests object (same structure as the batch-level failed_tests) scoped to that report.

When typename filters are applied, the summary is scoped to only the specified path in the hierarchy.

Example

Use ^ instead of \ as endline on Windows.
curl --location "http://testcenter.example.com/api/v1/results/summary?project0=example&batch0=Build%20c22de0e" \
--header "Authorization: Token zX4AAEdKAAD3GwAAfnUAAFd8AADybwAAXnUAAFUaAAA" \
--header "AutomationToken: 1a7fcb5e50d630e4d50792acd52745bb4a11bc18"

{
    "batch": {
        "name": "Build c22de0e",
        "annotations": [],
        "failed_tests": {
             "children":[1],
             "descendants":[2,3]
         },
        "results": { "...Results Summary Object..." },
        "time": { "...Time Object..." }
    },
    "reports": [
        {
            "...Report Metadata...",
            "failed_tests": { "children": [], "descendants": [] },
            "results": { "...Results Summary Object..." }
        }
    ]
}

GET/POST /api/v1/results/batches - Retrieve batch-level summaries

Retrieves a high-level result summary at the batch level only, without per-report result breakdowns or test details. More efficient than /results/summary but less detailed. Supports querying multiple batches in a single request.

Query Parameters

Batches may be provided via query parameters, the POST body, or both. If no batch is provided at all, the response is an empty batches array ({"batches": []}).

Request Body (alternative)

Content-Type: application/json

{
    "batches": ["Build c22de0e", "Build a1b2c3d"]
}

Query-parameter batches and POST-body batches are merged.

Response

{
    "batches": [
        {
            "name": "Build c22de0e",
            "result": { "result": "FAIL", "resultcode": 0 },
            "reports": [
                {
                    "...Report Metadata..."
                }
            ],
            "time": { "...Time Object..." }
        }
    ]
}

Behavior notes:

• Label filtering scopes the batch-level result to only matching reports. Report-level results are not included for efficiency, but report metadata (labels, time) is still provided.
• A batch with no matching reports — whether because the batch name is unknown or because all of its reports are excluded by the label filters — returns an entry with result "NONE" (resultcode 31), an empty reports array, and a zeroed time (start and end are 0), rather than being omitted or causing an error.
• Batches are returned in the order the arguments are provided.
• Report order within a batch is unspecified.
• Typename filters are not supported on this endpoint.
• There is no limit on the number of batches per request.

Examples

curl --location "http://testcenter.example.com/api/v1/results/batches?project0=example" \
--header "Content-Type: application/json" \
--header "Authorization: Token zX4AAEdKAAD3GwAAfnUAAFd8AADybwAAXnUAAFUaAAA" \
--header "AutomationToken: 1a7fcb5e50d630e4d50792acd52745bb4a11bc18" \
--data '{
    "batches": ["Build c22de0e", "Build a1b2c3d"]
}'

set TMP_JSON={ ^
    "batches": ["Build c22de0e", "Build a1b2c3d"] ^
}
curl --location "http://testcenter.example.com/api/v1/results/batches?project0=example" ^
--header "Content-Type: application/json" ^
--header "Authorization: Token zX4AAEdKAAD3GwAAfnUAAFd8AADybwAAXnUAAFUaAAA" ^
--header "AutomationToken: 1a7fcb5e50d630e4d50792acd52745bb4a11bc18" ^
--data "%TMP_JSON:"=\"%"

{
    "batches": [
        {
            "name": "Build c22de0e",
            "result": { "result": "FAIL", "resultcode": 0 },
            "reports": [
                {
                    "...Report Metadata..."
                }
            ],
            "time": { "...Time Object..." }
        }
    ]
}

GET /api/v1/results/tests - Retrieve test result matrix

Retrieves detailed test results as a matrix of tests vs. reports. Rows correspond to tests and columns correspond to reports — answering which test was executed in which configuration and with what result.

The tests are always provided for a specific hierarchy level.

Query Parameters

Response

{
    "pagination": { "...Pagination..." },
    "reports": [ "...Report Metadata..." ],
    "tests": [
        { "id": 99, "name": "Switching Dark and Light modes", "type": "feature", "has_children": true }
    ],
    "results": [
        [
            { "result": "FAIL", "resultcode": 0, "failures": 1 },
            { "result": "PASS", "resultcode": 7, "failures": 0 }
        ]
    ],
    "resultid": "ctfC3TCA778",
    "annotations": [ "...Annotation..." ]
}

Result matrix:

• results[i] corresponds to tests[i].
• In groupBy=report mode (default): results[i][j] corresponds to reports[j].
• In groupBy=batch mode: each row is flattened to a single result object (not wrapped in an array), and the reports array is omitted.

Standard result cell:

Detailed result cell (when detail=detailed) adds:

resultid and stable pagination:

The response includes a resultid value (a hash of the generated test sequence). Pass this back on subsequent paginated requests to ensure the test hierarchy remains stable. The hierarchy can change when new results are uploaded. If the returned resultid differs from the one you sent, the hierarchy has changed and earlier pages may need to be reloaded.

Tests not executed in a particular report appear as a NONE result (resultcode 31).

option=failures includes any test that has at least one failure across any report; fully passing tests are hidden.

Example

Use ^ instead of \ as endline on Windows.
curl --location "http://testcenter.example.com/api/v1/results/tests?project0=example&batch0=Build%20c22de0e&detail=detailed" \
--header "Authorization: Token zX4AAEdKAAD3GwAAfnUAAFd8AADybwAAXnUAAFUaAAA" \
--header "AutomationToken: 1a7fcb5e50d630e4d50792acd52745bb4a11bc18"

{
    "pagination": { "...Pagination..." },
    "reports": [ "...Report Metadata..." ],
    "tests": [
        { "id": 99, "name": "Switching Dark and Light modes", "type": "feature", "has_children": true }
    ],
    "results": [
        [
            { "result": "FAIL", "resultcode": 0, "failures": 1 },
            { "result": "PASS", "resultcode": 7, "failures": 0 }
        ]
    ],
    "resultid": "ctfC3TCA778",
    "annotations": [ "...Annotation..." ]
}

POST /api/v1/results/tests - Retrieve results for specific tests

To retrieve results for specific test ids across all hierarchy levels. Uses the same result format as the GET variant, and the same query parameters except for those used to specify a hierarchy level or pagination parameters. Because it is not paginated, the response contains no pagination object and no resultid.

Each requested test id maps positionally to one row of the result matrix (results[i] corresponds to tests[i]), so the order of the returned rows follows the order of the ids in the request. To preserve this correspondence ids are never dropped: an unknown test id produces a placeholder entry in tests ({"id": 0, "name": "Virtual Hook ()", "type": ""}) whose row contains only NONE (resultcode 31) cells.

Request Body

Content-Type: application/json

The tests array is required and must not be empty (an empty array returns an error).

{
    "tests": [2109, 325008, 208955]
}

Examples

curl --location "http://testcenter.example.com/api/v1/results/tests?project0=example&batch0=Build%20c22de0e" \
--header "Content-Type: application/json" \
--header "Authorization: Token zX4AAEdKAAD3GwAAfnUAAFd8AADybwAAXnUAAFUaAAA" \
--header "AutomationToken: 1a7fcb5e50d630e4d50792acd52745bb4a11bc18" \
--data '{
    "tests": [2109, 325008, 208955]
}'

set TMP_JSON={ ^
    "tests": [2109, 325008, 208955] ^
}
curl --location "http://testcenter.example.com/api/v1/results/tests?project0=example&batch0=Build%20c22de0e" ^
--header "Content-Type: application/json" ^
--header "Authorization: Token zX4AAEdKAAD3GwAAfnUAAFd8AADybwAAXnUAAFUaAAA" ^
--header "AutomationToken: 1a7fcb5e50d630e4d50792acd52745bb4a11bc18" ^
--data "%TMP_JSON:"=\"%"

GET /api/v1/results/events - Retrieve test events

Retrieves individual log messages/events from test executions. This is the most detailed endpoint, providing records such as failures, passes, warnings, and attachments. Slightly slower than other endpoints, especially when not filtered by event type.

Query Parameters

* Required unless reportid is provided.

Query modes:

• Batch-scoped (default): Uses batch0, label0\unicode{0x2026}, and typename0\unicode{0x2026} to scope events across matching reports.
• Report-scoped: When reportid is provided, events are scoped to that single report. Labels are not used, but typename filters still apply.

When testid is provided, it takes precedence over typename filters in either mode.

Response

{
    "pagination": { "...Pagination..." },
    "reports": [ "...Report Metadata..." ],
    "tests": [
        { "id": 1, "name": "suite_thermostat", "type": "testsuite" },
        { "id": 8, "name": "tst_darkmode", "parent_index": 0, "type": "testcase" },
        { "id": 106, "name": "Then I see the app in Dark mode", "parent_index": 3, "type": "step" }
    ],
    "videos": [
        {
            "path": "2024/38/6131220_ThermostatApp_capture.mp4",
            "url": "http://host/Testcenter/import/video/2024/38/capture.mp4",
            "timestamp": 1773754452210
        }
    ],
    "events": [ "...Event objects..." ]
}

Test entries include id, name, type, and optionally parent_index (index into the tests array) to establish the parent-child hierarchy.

Video entries include path, url, and timestamp. STARTVIDEO and ENDVIDEO events are always paired. Multiple events can reference the same video, helping locate events relative to video playback.

Event structure:

Resource types:

Resources are attachments associated with an event. The type field determines the structure:

• file — A plain file attachment. Fields: type, path, url.
• screenshotVP, visualVP, tableVP, guicoverage — Verification points. These include actual (path, url) and expected (path, url) reference data in addition to type.

Example

Use ^ instead of \ as endline on Windows.
curl --location "http://testcenter.example.com/api/v1/results/events?project0=example&batch0=Build%20c22de0e&eventtype=failures" \
--header "Authorization: Token zX4AAEdKAAD3GwAAfnUAAFd8AADybwAAXnUAAFUaAAA" \
--header "AutomationToken: 1a7fcb5e50d630e4d50792acd52745bb4a11bc18"

{
    "pagination": { "...Pagination..." },
    "reports": [ "...Report Metadata..." ],
    "tests": [
        { "id": 1, "name": "suite_thermostat", "type": "testsuite" },
        { "id": 8, "name": "tst_darkmode", "parent_index": 0, "type": "testcase" },
        { "id": 106, "name": "Then I see the app in Dark mode", "parent_index": 3, "type": "step" }
    ],
    "videos": [
        {
            "path": "2024/38/6131220_ThermostatApp_capture.mp4",
            "url": "http://host/Testcenter/import/video/2024/38/capture.mp4",
            "timestamp": 1773754452210
        }
    ],
    "events": [ "...Event objects..." ]
}

GET /api/v1/results/timeline - Retrieve batch timeline

Retrieves batch names and their associated times without any test results. Useful for navigating batch history, building a timeline view, or determining which batches fall within a given time frame.

Query Parameters (common)

Mode 1 - Paginated (no batch0)

Returns batches ordered by time, optionally filtered to a time window.

starttime and endtime filter against the time field selected by orderby. They can be combined to define a window or used individually.

Mode 2 - Relative to a batch (batch0 provided)

Returns batches before and/or after the named reference batch. The reference batch itself is never included in the response. Pagination parameters are not used in this mode.

* At least one of next or previous must be non-zero. Both can be provided simultaneously to get a window around the reference batch.

Response

{
    "batches": [
        {
            "name": "Build c22de0e",
            "time": {
                "start": 1771335203000,
                "end": 1771335302424,
                "upload": 1773754622387
            }
        }
    ],
    "pagination": { "...Pagination (paginated mode only)..." }
}

The timeline time object differs from the standard Time Object: it includes start, end, and upload (all Unix milliseconds) but does not include duration.

Example

Use ^ instead of \ as endline on Windows.
curl --location "http://testcenter.example.com/api/v1/results/timeline?project0=example&orderby=execution&limit=50" \
--header "Authorization: Token zX4AAEdKAAD3GwAAfnUAAFd8AADybwAAXnUAAFUaAAA" \
--header "AutomationToken: 1a7fcb5e50d630e4d50792acd52745bb4a11bc18"

{
    "batches": [
        {
            "name": "Build c22de0e",
            "time": {
                "start": 1771335203000,
                "end": 1771335302424,
                "upload": 1773754622387
            }
        }
    ],
    "pagination": { "...Pagination (paginated mode only)..." }
}

Definitions

Error Response

All endpoints return errors as HTTP 500 with a JSON body:

{
  "error": {
    "code": 1004,
    "message": "Failed to resolve project name"
  }
}

The error code is a general-purpose numeric code and is not specific to individual failure cases.

An unknown or omitted batch name is not treated as an error — there is no notion of "finding" a batch, only of which reports match. When no reports match (unknown batch name, or all reports excluded by label filters), the batch result is NONE (resultcode 31); see the individual endpoint documentation.

Pagination

Endpoints that support pagination include a pagination object in the response:

{
  "pagination": {
    "page": 1,
    "limit": 50,
    "count": 12,
    "total": 12
  }
}

Pagination query parameters:

Report Metadata

Report metadata appears in most endpoint responses. It identifies a specific test execution configuration.

{
  "id": 171,
  "active": false,
  "labels": [
    { "key": "OS", "value": "Windows" },
    { "key": "arch", "value": "x64" },
    { "key": "branch", "value": "main" },
    { "key": "version", "value": "0.9.0" }
  ],
  "time": {
    "duration": 60547,
    "start": 1773754403000,
    "end": 1773754463547
  }
}

Some endpoints extend this object with additional fields such as results (see individual endpoint documentation).

The /results/timeline endpoint (when includes=reports is set) is an exception: it returns a reduced form of this object that omits active and omits duration from the time object (only start and end are provided).

Time Object

{
  "start": 1773754403000,
  "end": 1773754463547,
  "duration": 60547
}

All values are in milliseconds. start and end are Unix timestamps. duration is the active execution time and may be less than end - start if there are breaks between tests.

Annotation

Annotations are user-placed markers attached to specific tests or events. They appear in endpoints that return test-level detail.

{
  "id": 1,
  "name": "False Positive",
  "report_ids": [171],
  "test_id": 99,
  "type": "event"
}

A single test or event can have multiple annotations.

Results Summary Object

Used in /results/summary at both batch and report level:

{
  "result": "FAIL",
  "resultcode": 0,
  "tests": { "failure": 1, "pass": 0, "skip": 0, "total": 1 },
  "events": {}
}

The events field is an empty object in "standard" detail mode. In "detailed" mode it expands to:

{
  "events": {
    "failures": { "error": 0, "fail": 1, "fatal": 0, "xpass": 0 },
    "pass": 87,
    "skipped": 0,
    "warning": 23,
    "xfail": 0
  }
}