Integration for TestRail

TestRail is a test case management software that allows for the management, tracking and organization of one's software testing efforts. The TestRail plugin is used to automate the pushing of test results within Squish Test Center to this platform. This mapping also allows for automated tasks to be triggered by TestRail.

Configuration for Test Mapping

Once you have set up your TestRail instance (either cloud-based or server), log in to Squish Test Center. Hover over the user's icon, and choose Global Settings. Locate and click the Active button right below the TestRail Integration section. Fill out the required fields mentioned below and click Update.

{}

TestRail Integration Settings

  • Field Server URL: URL of your TestRail instance
  • Field Username: Username of TestRail account with TestRail API access
  • Field API Key: Password / API Key for the TestRail API

    Note: It is highly recommended that you use a TestRail API Key instead of an account password! You can find more information on how to set up your API Key here:

  • Button Test Connection: Test if a TestRail instance can be reached given the Server URL, Username and API Key provided. If the request is redirected, the actual URL will be set as Server URL, which you can save by clicking Update.
  • Section Project Associations: Allows the user to select the projects defined in the Squish Test Center for association with TestRail.
  • Button Sync Projects: Fetches a list of the TestRail projects from the TestRail instance.
  • Button Update Associations: Saves Squish Test Center-to-TestRail project associations.

After updating the TestRail address and credentials, you can fetch TestRail projects by clicking Sync Projects. If this operation succeeds, the list of TestRail projects should be visible when selecting a Squish Test Center project from the dropdown. Having selected a Squish Test Center project, associate any of the TestRail projects listed below and press Update Associations.

Pulling, Mapping and Pushing Tests

After you have mapped a Squish Test Center project to a TestRail project, you can use the Traceability Dropdown Button (on top of the Explore View of that project, in the navigation bar) to navigate to the TestRail section. On this page, press the Pull button and wait for the action to finish. The table below should now list all test cases of your associated TestRail projects.

With the buttons on the leftmost Mapping column, you can associate your TestRail tests with the Squish Test Center tests of the current project. In the shown dialog, select the tests you want to associate with your TestRail test case and press Apply Changes.

Pressing the Push will create a new TestRail run and push the associated (combined) results for each of the TestRail test cases.

Note: If the currently selected batch contains reports with a .testrail.run label, the results are pushed to the specified runs instead of creating new TestRail runs. For example, reports with the .testrail.run=R12345 label would have their results pushed to the TestRail run with the id 12345.

Note that you can also use the testcentercmd commandline tool to perform the push and pull actions (see Upload via the command line ).

Configuring automatic Squish test case execution

Warning: This is only available to the server (non-cloud) version of TestRail and requires an API-access-enabled Squish Test Center license!

You can enable the automatic Squish test case execution from TestRail by adding the provided UI script and trigger script to your TestRail instance as well as setting up the Squish Test Center Test Execution settings. This will add an Execute Associated Squish Test(s) button to the TestRail case, suite, run, and plan view pages. If a TestRail test case (or the cases contained in the suite/run) is mapped to one or more Squish test cases, it will be automatically executed and the results uploaded to Squish Test Center.

Configuration synopsis:

  • As a TestRail admin, create a new UI script in the TestRail web application and replace it with the contents of the provided UI script:
                name: Trigger Execution of associated Squish tests + PLANS AND RUNS
                description: Triggers execution of Squish tests for a run, test plan, suite or single case
                author: froglogic GmbH
                version: 1.0
                includes: ^(suites/view|cases/view|plans|runs)
                excludes: ^(runs/overview)
    
                js:
                $(document).ready( function() {
                    /* Create the button. */
                    var button = $( [
                        '<div class="toolbar content-header-toolbar">',
                        '<a class="toolbar-button toolbar-button-last toolbar-button-first content-header-button button-start" href="javascript:void(0)">',
                        'Execute Associated Squish Test(s)',
                        '</a>',
                        '</div>'
                    ].join('') );
                    /* Add it to the toolbar. */
                    $('#content-header .content-header-inner').prepend(button);
                    /* Bind the click event to trigger the associated Squish tests. */
                    $('a', button).click( function() {
                        // find out which page we are on (suite/case or plan or run)
                        // NOTE: a case always has suite context, but runs are not necessarily part of a plan, thus not always having a plan context
                        if ( uiscripts.context.run ) {
                            // run view page
                            fetchExecutionFromRun(
                                // on success
                                function( execution ) {
                                    triggerSquishTestExecutions( [ execution ] );
                                },
                                // on failure
                                function( errormsg ) {
                                    App.Dialogs.error( errormsg );
                                }
                            );
                        } else if ( uiscripts.context.plan ) {
                            // plan view page
                            fetchExecutionListFromPlan(
                                // on success
                                function( executionlist ) {
                                    triggerSquishTestExecutions( executionlist );
                                },
                                // on failure
                                function( errormsg ) {
                                    App.Dialogs.error( errormsg );
                                }
                            );
                        } else if ( uiscripts.context.case ) {
                            // case view page
                            var execution = {};
                            execution['user'] = uiscripts.context.user.email;
                            execution['tests'] = [ uiscripts.context.case.id ];
                            triggerSquishTestExecutions( [ execution ] );
                        } else if ( uiscripts.context.suite ) {
                            // suite view page
                            fetchExecutionFromSuite(
                                // on success
                                function( execution ) {
                                    triggerSquishTestExecutions( [ execution ] );
                                },
                                // on failure
                                function( errormsg ) {
                                    App.Dialogs.error( errormsg );
                                }
                            );
                        } else {
                            App.Dialogs.error( 'Could not find any context to execute a test in.' );
                        }
    
                    } );
                } );
    
                function fetchExecutionFromRun( onSuccess, onFailure )
                {
                    get( 'get_run/' + uiscripts.context.run.id,
                        function( run ) {
                            get( 'get_tests/' + uiscripts.context.run.id,
                                function( tests ) {
                                    get( 'get_configs/' + uiscripts.context.project.id,
                                        function( configs ) {
                                            var execution = {};
                                            execution['user'] = uiscripts.context.user.email;
                                            execution['tests'] = tests.map( function( item ) { return item.case_id; } );
                                            execution['labels'] = [ { 'key': '.testrail.run', 'value': 'R' + uiscripts.context.run.id.toString() } ];
                                            // if the run is part of a plan, attach a batch name and - if available - configurations
                                            if ( uiscripts.context.plan ) {
                                                execution['batch'] = createPlanBatchName();
                                                execution['labels'] = execution['labels'].concat( createLabelsFromConfigIds( configs, run.config_ids ) );
                                            }
                                            onSuccess( execution );
                                        },
                                        function() {
                                            onFailure( 'An error occurred while fetching run configurations for this project.' );
                                        }
                                    );
                                },
                                function() {
                                    onFailure( 'An error occurred while fetching the test cases for this run.' );
                                }
                            );
                        },
                        function() {
                            onFailure( 'An error occurred while fetching the run.' );
                        }
                    );
                }
    
                function fetchExecutionListFromPlan( onSuccess, onFailure )
                {
                    get( 'get_plan/' + uiscripts.context.plan.id,
                        function( plan ) {
                            get( 'get_configs/' + uiscripts.context.project.id,
                                function( configs ) {
                                    var run_ids = [];
                                    plan.entries.forEach( function( entry ) { run_ids = run_ids.concat( entry.runs.map( function( run ) { return run.id; } ) ) } );
                                    if ( run_ids.length == 0 ) {
                                        onFailure( 'Selected test plan does not contain any runs.' );
                                        return;
                                    }
                                    var executionlist = [];
                                    var batchName = createPlanBatchName();
                                    var collectTests = function( runIds ) {
                                        get( 'get_tests/' + runIds[0],
                                            function( tests ) {
                                                var execution = {};
                                                execution['user'] = uiscripts.context.user.email;
                                                execution['tests'] = tests.map( function( item ) { return item.case_id; } );
                                                execution['labels'] = [ { 'key': '.testrail.run', 'value': 'R' + runIds[0].toString() } ];
                                                execution['batch'] = batchName;
                                                // get run and add configuration labels
                                                plan.entries.forEach(
                                                    function( entry ) {
                                                        entry.runs.forEach(
                                                            function( run ) {
                                                                if ( run.id == runIds[0] ) {
                                                                    execution['labels'] = execution['labels'].concat( createLabelsFromConfigIds( configs, run.config_ids ) );
                                                                }
                                                            }
                                                        );
                                                    }
                                                );
                                                executionlist.push( execution );
                                                if ( runIds.length == 1 ) {
                                                    onSuccess( executionlist );
                                                } else {
                                                    collectTests( runIds.slice( 1, runIds.length ) );
                                                }
                                            },
                                            function() {
                                                onFailure( 'An error occurred while fetching tests.' );
                                            }
                                        );
                                    };
                                    collectTests( run_ids );
                                },
                                function() {
                                    onFailure( 'An error occurred while fetching run configurations for this project.' );
                                }
                            );
                        },
                        function() {
                            onFailure( 'An error occurred while fetching the test plan.' );
                        }
                    );
                }
    
                function fetchExecutionFromSuite( onSuccess, onFailure )
                {
                    get( 'get_cases/' + uiscripts.context.project.id + '&suite_id=' + uiscripts.context.suite.id,
                        function( cases ) {
                            var execution = {};
                            execution['user'] = uiscripts.context.user.email;
                            execution['tests'] = cases.map( function( item ) { return item.id; } );
                            onSuccess( execution );
                        },
                        function() {
                            onFailure( 'An error occurred while fetching the list cases for this suite.' );
                        }
                    );
                }
    
                function triggerSquishTestExecutions( executions )
                {
                    $.ajax( {
                        url: 'squishtestcenterintegration.php',
                        dataType: 'json',
                        contentType: 'application/json',
                        data: JSON.stringify( executions ),
                        type: 'POST',
                        success: function()
                        {
                            location.reload();
                        },
                        error: function( data )
                        {
                            App.Dialogs.error(
                                'An error occurred while trying to trigger the associated Squish tests:\n' +
                                JSON.stringify( data )
                            );
                        }
                    } );
                    App.Dialogs.message(
                        'Sending test execution(s) to Squish TestCenter...\n' +
                        'The page will reload once the Squish test executions have been scheduled.',
                        'Confirmation'
                    );
                }
    
                function get( apiCall, onSuccess, onFailure ) {
                    $.ajax( {
                        url: 'index.php?/api/v2/' + apiCall,
                        dataType: 'json',
                        contentType: 'application/json',
                        type: 'GET',
                        success: function( data ) {
                            onSuccess( data );
                        },
                        error: function( data ) {
                            onFailure( data );
                        }
                    } );
                }
    
                function createLabelsFromConfigIds( configData, configIds ) {
                    var groupTable = {};
                    var configTable = {};
                    configData.forEach(
                        function( configGroup ) {
                            groupTable[configGroup.id] = configGroup.name;
                            configGroup.configs.forEach(
                                function( config ) {
                                    configTable[config.id] = { name: config.name, groupId: config.group_id };
                                }
                            );
                        }
                    );
                    var labels = configIds.map(
                        function( id ) {
                            var config = configTable[id];
                            return { key: groupTable[config.groupId], value: config.name };
                        }
                    );
                    return labels;
                }
    
                function createPlanBatchName() {
                    return uiscripts.context.plan.name + '-' + Date.now();
                }
  • Place the squishtestcenterintegration.php trigger script file into the TestRail installation directory. See for more information on step 1 and 2.
  • In the trigger php script, edit the following lines to contain the URL address to your Squish Test Center instance, a user login and password.
                define('SQUISHTESTCENTER_ADDRESS', '[Squish TestCenter URL]');
                define('SQUISHTESTCENTER_API_USER', '[User login]');
                define('SQUISHTESTCENTER_API_PASSWORD', '[User password]');

    The provided user must have access rights to the Squish TestCenter API.

  • Make sure you have set up your test automation settings in the Squish Test Center Global Settings page. (See the Test Execution section and subsequent fields.) Setting the Squish Test Suite Directory is mandatory for Squish to find your test suites.
    • Field Squish Installation Directory: Location of the to-be-used Squish installation, which is to be located on the same machine as the Squish Test Center.
    • Field Custom Squishserver Host Address: Address/URL of (remote) host running squishserver.
    • Field Custom Squishserver Port: Used port of (remote) squishserver.
    • Field Squish Test Suite Directory: Filepath to folder containing the Squish test suites that are to be executed by the Squish Test Center.

If a run is specified (run/plan view), the Squish Test Center result uploads will contain a special .testrail.run label containing the TestRail run id with a 'R' prefix (e.g. .testrail.run=R12345).

Scheduled TestRail test executions can also be viewed in the Scheduling view.

© 2022 The Qt Company Ltd. Documentation contributions included herein are the copyrights of their respective owners. The documentation provided herein is licensed under the terms of the GNU Free Documentation License version 1.3 as published by the Free Software Foundation. Qt and respective logos are trademarks of The Qt Company Ltd. in Finland and/or other countries worldwide. All other trademarks are property of their respective owners.