Integration for TestRail

TestRail is a test case management software for managing, tracking, and organizing software testing. The TestRail plugin automates the pushing of test results from Test Center to TestRail, and enables automated tasks to be triggered by TestRail.

Configuration for test mapping

Once you have set up your TestRail instance (either cloud-based or server), log into Test Center and turn on TestRail Integration in Global Settings. Fill in the required fields and click Update to save the settings.

TestRail Integration settings

To establish the connection to TestRail, enter the following information:

• Server URL: URL of your TestRail instance.
• Username: Username of a TestRail account with TestRail API access.
• API Key: Password or API Key for the TestRail API.

  Note: It is highly recommended that you use a TestRail API key instead of an account password. For more information on how to set up your API key, see Username and API Key in the TestRail documentation.

Click Test Connection to test whether a TestRail instance can be reached at the server URL with the username and API key. If the request is redirected, the actual URL is set as the value of Server URL. You can save it by clicking Update.

Click the Sync Projects button to fetch TestRail projects and make them appear in the Associations list, where you can map each TestRail project to its Test Center counterpart.

Pulling, mapping, and pushing tests

To open the Traceability view, select TestRail in the History or Explore view of an associated project. Click the Pull button to fetch the test cases of the associated TestRail projects.

To map the pulled TestRail tests to tests managed by Test Center in the Traceability view, click the Edit button of a table entry in the Mapping column. Select the tests to associate with the TestRail test case, and click Apply Changes.

Click the Push button to create a new TestRail run and push the associated (combined) results for each of the TestRail test cases.

You can also use the testcentercmd command line tool to perform the push and pull actions, as instructed in Upload via the command line.

Configuring automatic Squish test case execution

To enable the automatic Squish test case execution from TestRail, add the provided UI script and trigger script to your TestRail instance and specify Test Center Test Execution settings. An Execute Associated Squish Test(s) button appears in the TestRail case, suite, run, and plan views. If a TestRail test case (or the cases contained in the suite or run) is mapped to one or more Squish test cases, it is automatically executed and the results are uploaded to Test Center.

To automize Squish test case execution:

1. As a TestRail admin, create a new UI script in the TestRail web application and replace it with the contents of the provided UI script:

   name: Trigger Execution of associated Squish tests + PLANS AND RUNS
   description: Triggers execution of Squish tests for a run, test plan, suite or single case
   author: froglogic GmbH
   version: 1.0
   includes: ^(suites/view|cases/view|plans|runs)
   excludes: ^(runs/overview)
   
   js:
   $(document).ready( function() {
       /* Create the button. */
       var button = $( [
           '<div class="toolbar content-header-toolbar">',
           '<a class="toolbar-button toolbar-button-last toolbar-button-first content-header-button button-start" href="javascript:void(0)">',
           'Execute Associated Squish Test(s)',
           '</a>',
           '</div>'
       ].join('') );
       /* Add it to the toolbar. */
       $('#content-header .content-header-inner').prepend(button);
       /* Bind the click event to trigger the associated Squish tests. */
       $('a', button).click( function() {
           // find out which page we are on (suite/case or plan or run)
           // NOTE: a case always has suite context, but runs are not necessarily part of a plan, thus not always having a plan context
           if ( uiscripts.context.run ) {
               // run view page
               fetchExecutionFromRun(
                   // on success
                   function( execution ) {
                       triggerSquishTestExecutions( [ execution ] );
                   },
                   // on failure
                   function( errormsg ) {
                       App.Dialogs.error( errormsg );
                   }
               );
           } else if ( uiscripts.context.plan ) {
               // plan view page
               fetchExecutionListFromPlan(
                   // on success
                   function( executionlist ) {
                       triggerSquishTestExecutions( executionlist );
                   },
                   // on failure
                   function( errormsg ) {
                       App.Dialogs.error( errormsg );
                   }
               );
           } else if ( uiscripts.context.case ) {
               // case view page
               var execution = {};
               execution['user'] = uiscripts.context.user.email;
               execution['tests'] = [ uiscripts.context.case.id ];
               triggerSquishTestExecutions( [ execution ] );
           } else if ( uiscripts.context.suite ) {
               // suite view page
               fetchExecutionFromSuite(
                   // on success
                   function( execution ) {
                       triggerSquishTestExecutions( [ execution ] );
                   },
                   // on failure
                   function( errormsg ) {
                       App.Dialogs.error( errormsg );
                   }
               );
           } else {
               App.Dialogs.error( 'Could not find any context to execute a test in.' );
           }
   
       } );
   } );
   
   function fetchExecutionFromRun( onSuccess, onFailure )
   {
       get( 'get_run/' + uiscripts.context.run.id,
           function( run ) {
               get( 'get_tests/' + uiscripts.context.run.id,
                   function( tests ) {
                       get( 'get_configs/' + uiscripts.context.project.id,
                           function( configs ) {
                               var execution = {};
                               execution['user'] = uiscripts.context.user.email;
                               execution['tests'] = tests.map( function( item ) { return item.case_id; } );
                               execution['labels'] = [ { 'key': '.testrail.run', 'value': 'R' + uiscripts.context.run.id.toString() } ];
                               // if the run is part of a plan, attach a batch name and - if available - configurations
                               if ( uiscripts.context.plan ) {
                                   execution['batch'] = createPlanBatchName();
                                   execution['labels'] = execution['labels'].concat( createLabelsFromConfigIds( configs, run.config_ids ) );
                               }
                               onSuccess( execution );
                           },
                           function() {
                               onFailure( 'An error occurred while fetching run configurations for this project.' );
                           }
                       );
                   },
                   function() {
                       onFailure( 'An error occurred while fetching the test cases for this run.' );
                   }
               );
           },
           function() {
               onFailure( 'An error occurred while fetching the run.' );
           }
       );
   }
   
   function fetchExecutionListFromPlan( onSuccess, onFailure )
   {
       get( 'get_plan/' + uiscripts.context.plan.id,
           function( plan ) {
               get( 'get_configs/' + uiscripts.context.project.id,
                   function( configs ) {
                       var run_ids = [];
                       plan.entries.forEach( function( entry ) { run_ids = run_ids.concat( entry.runs.map( function( run ) { return run.id; } ) ) } );
                       if ( run_ids.length == 0 ) {
                           onFailure( 'Selected test plan does not contain any runs.' );
                           return;
                       }
                       var executionlist = [];
                       var batchName = createPlanBatchName();
                       var collectTests = function( runIds ) {
                           get( 'get_tests/' + runIds[0],
                               function( tests ) {
                                   var execution = {};
                                   execution['user'] = uiscripts.context.user.email;
                                   execution['tests'] = tests.map( function( item ) { return item.case_id; } );
                                   execution['labels'] = [ { 'key': '.testrail.run', 'value': 'R' + runIds[0].toString() } ];
                                   execution['batch'] = batchName;
                                   // get run and add configuration labels
                                   plan.entries.forEach(
                                       function( entry ) {
                                           entry.runs.forEach(
                                               function( run ) {
                                                   if ( run.id == runIds[0] ) {
                                                       execution['labels'] = execution['labels'].concat( createLabelsFromConfigIds( configs, run.config_ids ) );
                                                   }
                                               }
                                           );
                                       }
                                   );
                                   executionlist.push( execution );
                                   if ( runIds.length == 1 ) {
                                       onSuccess( executionlist );
                                   } else {
                                       collectTests( runIds.slice( 1, runIds.length ) );
                                   }
                               },
                               function() {
                                   onFailure( 'An error occurred while fetching tests.' );
                               }
                           );
                       };
                       collectTests( run_ids );
                   },
                   function() {
                       onFailure( 'An error occurred while fetching run configurations for this project.' );
                   }
               );
           },
           function() {
               onFailure( 'An error occurred while fetching the test plan.' );
           }
       );
   }
   
   function fetchExecutionFromSuite( onSuccess, onFailure )
   {
       get( 'get_cases/' + uiscripts.context.project.id + '&suite_id=' + uiscripts.context.suite.id,
           function( cases ) {
               var execution = {};
               execution['user'] = uiscripts.context.user.email;
               execution['tests'] = cases.map( function( item ) { return item.id; } );
               onSuccess( execution );
           },
           function() {
               onFailure( 'An error occurred while fetching the list cases for this suite.' );
           }
       );
   }
   
   function triggerSquishTestExecutions( executions )
   {
       $.ajax( {
           url: 'squishtestcenterintegration.php',
           dataType: 'json',
           contentType: 'application/json',
           data: JSON.stringify( executions ),
           type: 'POST',
           success: function()
           {
               location.reload();
           },
           error: function( data )
           {
               App.Dialogs.error(
                   'An error occurred while trying to trigger the associated Squish tests:\n' +
                   JSON.stringify( data )
               );
           }
       } );
       App.Dialogs.message(
           'Sending test execution(s) to Squish TestCenter...\n' +
           'The page will reload once the Squish test executions have been scheduled.',
           'Confirmation'
       );
   }
   
   function get( apiCall, onSuccess, onFailure ) {
       $.ajax( {
           url: 'index.php?/api/v2/' + apiCall,
           dataType: 'json',
           contentType: 'application/json',
           type: 'GET',
           success: function( data ) {
               onSuccess( data );
           },
           error: function( data ) {
               onFailure( data );
           }
       } );
   }
   
   function createLabelsFromConfigIds( configData, configIds ) {
       var groupTable = {};
       var configTable = {};
       configData.forEach(
           function( configGroup ) {
               groupTable[configGroup.id] = configGroup.name;
               configGroup.configs.forEach(
                   function( config ) {
                       configTable[config.id] = { name: config.name, groupId: config.group_id };
                   }
               );
           }
       );
       var labels = configIds.map(
           function( id ) {
               var config = configTable[id];
               return { key: groupTable[config.groupId], value: config.name };
           }
       );
       return labels;
   }
   
   function createPlanBatchName() {
       return uiscripts.context.plan.name + '-' + Date.now();
   }

2. Place the squishtestcenterintegration.php trigger script file into the TestRail installation directory. For more information, see UI scripts for test automation in the TestRail documentation.
3. In the trigger php script, edit the following lines to contain the URL address to your Test Center instance and a user login and password.

   define('SQUISHTESTCENTER_ADDRESS', '[Squish TestCenter URL]');
   define('SQUISHTESTCENTER_API_USER', '[User login]');
   define('SQUISHTESTCENTER_API_PASSWORD', '[User password]');

   The provided user must have access rights to the Squish TestCenter API.

4. Check the test automation settings in the Test Center Global Settings > Test Execution view. Setting the Squish Test Suite Directory is mandatory for Squish to find your test suites.
   • Squish Installation Directory: Location of the Squish installation to use. It needs to be on the same machine as the Test Center.
   • Custom Squishserver Host Address: Address or URL of the (remote) host running squishserver.
   • Custom Squishserver Port: Used port of (remote) squishserver.
   • Squish Test Suite Directory: File path to the folder containing the Squish test suites that are to be executed by the Test Center.

If a run is specified (in the TestRail run or plan view), the Test Center result uploads contain the .testrail.run label that specifies the TestRail run id with an R prefix. For example, .testrail.run=R12345.

You can view scheduled TestRail test executions in the Scheduling view.