Working with Helix ALM REST API Part 6: Automated Testing

Today we'll conclude our series of posts focusing on various features from Perforce's Helix ALM REST API. Today's feature is about working with a CI/CD (continuous integration/continuous delivery) pipeline for automated testing and integrating it with the API. If you are unfamiliar with Helix ALM, it is a modular suite of ALM tools that you can use to trace requirements, tests, and issues. ALM stands for Application Lifecycle Management, REST for Representational State Transfer, and API for Application Programming Interface. The Helix ALM REST API has several endpoints that developers can use for interfacing with their organization's Helix ALM instance. The wide variety of endpoints provide many possibilities for developers to create tools or scripts to aid in their organization's workflow, with few limitations. The REST API is available at: https://{organization}:{port}/helix-alm/api/v0/ and the Swagger UI interactive documentation tool is available at: https://{organization}:{port} where {organization} is the address of the REST API server. The address can be configured in the JSON configuration file of the REST API, typically located at: ../Perforce/Helix ALM/helix-alm-rest-api/config/config.json on the server that is running the Helix ALM instance. Automated Testing Helix ALM provides integration with CI/CD pipelines, allowing organizations to have full traceability between their builds and tests, and associating the results back to test cases. The system offers two primary methods for submitting build and test results: the Helix ALM Test Management plugin for Jenkins and a REST API for other testing tools and environments. For this article, we will focus on the latter. At the core of Helix ALM's automated testing framework are automation suites, which serve as containers for grouping related automated tests. These suites are logical organizations of a set of tests based on a common factor. The system supports both automatic and manual association of test results with test cases. Achieving this association using the REST API is a multi-step process. First, we need to create an automation suite, specifying the script ID prefix and any test cases we want to associate. It's also possible to associate the test cases later in a separate request to the /{projectID}/automationSuites/{automationSuiteID}/testCases endpoint. A successful creation will return an HTTP 201 response along with the created automation suite in the response body. With our automation suite created, we can now run our automated tests for a build and submit the results to the REST API. We need to send the results to the /{projectID}/automationSuites/{automationSuiteID}/submitBuild endpoint, where the {automationSuiteID} is the ID we received in the response when we created the automation suite. The structure of the data that the endpoint is expecting is an Automation Build object with the following name-value pairs: Number (string) Description Branch Start date Duration External URL Results, array of Where a Result object has its own name-value pairs: Name Unique name Status Device Manufacturer Model OS OS version Browser Browser version Start date Duration (milliseconds) External URL There is one more name-value pair named "properties" which accepts a JSON object of any name-value pair. This is for unique properties from the build process that do not match the REST API's structure, and can be included as part of the Automation Build or Result objects. The first name-value pair "name" is where we will associate the test from the build with our Helix ALM test case by appending the automation suite's script ID prefix to the test case's number. The unique name is the name of the test from the build process. The "status" value is a specific JSON object that the REST API expects. It's comprised of ID and label values. The ID and label follow the following convention: 1 => Passed 2 => Failed 3 => Skipped 4 => Blocked 5 => Unknown Example Let's see how we can implement the above description of automation suites into a simple example. An automation suite item first needs a name, like "Alpha Tests" or "Final Tests"; then it needs a description; a true or false value if it is active; a list of test cases that are associated with the suite; a list of owners; and finally the script ID prefix, which can be like "Alpha-" or "Final-". Regardless of the programming language of choice, we want something, when serialized, to represent the following JSON object: { "automationSuitesData": [ { "name": "Example Test Suite", "description": "Automation suite for grouping our automated tests.", "active": true, "scriptIDPrefix": "EX-", "owners": [ { "lastName": "John", "firstName": "Doe", "id": 135, "username": "jdoe" }, … ], "testCases": { "testCasesData": [ { "id": 1, "number": 1 },

Mar 17, 2025 - 11:17

Working with Helix ALM REST API Part 6: Automated Testing

Today we'll conclude our series of posts focusing on various features from Perforce's Helix ALM REST API. Today's feature is about working with a CI/CD (continuous integration/continuous delivery) pipeline for automated testing and integrating it with the API. If you are unfamiliar with Helix ALM, it is a modular suite of ALM tools that you can use to trace requirements, tests, and issues. ALM stands for Application Lifecycle Management, REST for Representational State Transfer, and API for Application Programming Interface.

The Helix ALM REST API has several endpoints that developers can use for interfacing with their organization's Helix ALM instance. The wide variety of endpoints provide many possibilities for developers to create tools or scripts to aid in their organization's workflow, with few limitations.

The REST API is available at:

https://{organization}:{port}/helix-alm/api/v0/

and the Swagger UI interactive documentation tool is available at:

https://{organization}:{port}

where {organization} is the address of the REST API server. The address can be configured in the JSON configuration file of the REST API, typically located at:

../Perforce/Helix ALM/helix-alm-rest-api/config/config.json

on the server that is running the Helix ALM instance.

Automated Testing

Helix ALM provides integration with CI/CD pipelines, allowing organizations to have full traceability between their builds and tests, and associating the results back to test cases. The system offers two primary methods for submitting build and test results: the Helix ALM Test Management plugin for Jenkins and a REST API for other testing tools and environments. For this article, we will focus on the latter.

At the core of Helix ALM's automated testing framework are automation suites, which serve as containers for grouping related automated tests. These suites are logical organizations of a set of tests based on a common factor. The system supports both automatic and manual association of test results with test cases. Achieving this association using the REST API is a multi-step process.

First, we need to create an automation suite, specifying the script ID prefix and any test cases we want to associate. It's also possible to associate the test cases later in a separate request to the

/{projectID}/automationSuites/{automationSuiteID}/testCases

endpoint. A successful creation will return an HTTP 201 response along with the created automation suite in the response body.

With our automation suite created, we can now run our automated tests for a build and submit the results to the REST API. We need to send the results to the

/{projectID}/automationSuites/{automationSuiteID}/submitBuild

endpoint, where the {automationSuiteID} is the ID we received in the response when we created the automation suite. The structure of the data that the endpoint is expecting is an Automation Build object with the following name-value pairs:
Number (string)

Description
Branch
Start date
Duration
External URL
Results, array of

Where a Result object has its own name-value pairs:

Name
Unique name
Status
Device
Manufacturer
Model
OS
OS version
Browser
Browser version
Start date
Duration (milliseconds)
External URL

There is one more name-value pair named "properties" which accepts a JSON object of any name-value pair. This is for unique properties from the build process that do not match the REST API's structure, and can be included as part of the Automation Build or Result objects. The first name-value pair "name" is where we will associate the test from the build with our Helix ALM test case by appending the automation suite's script ID prefix to the test case's number. The unique name is the name of the test from the build process.

The "status" value is a specific JSON object that the REST API expects. It's comprised of ID and label values. The ID and label follow the following convention:
1 => Passed
2 => Failed
3 => Skipped
4 => Blocked
5 => Unknown

Example

Let's see how we can implement the above description of automation suites into a simple example.

An automation suite item first needs a name, like "Alpha Tests" or "Final Tests"; then it needs a description; a true or false value if it is active; a list of test cases that are associated with the suite; a list of owners; and finally the script ID prefix, which can be like "Alpha-" or "Final-".

Regardless of the programming language of choice, we want something, when serialized, to represent the following JSON object:

{
  "automationSuitesData": [
  {
    "name": "Example Test Suite",
    "description": "Automation suite for grouping our automated tests.",
    "active": true,
    "scriptIDPrefix": "EX-",
    "owners": [
      {
        "lastName": "John",
        "firstName": "Doe",
        "id": 135,
        "username": "jdoe"
      },
      …
    ],
    "testCases": {
      "testCasesData": [
        {
          "id": 1,
          "number": 1
        },
        …
      ]
      }
    }
  ]
}

With this we attach it to the body of our POST request and send it to the /{projectID}/automationSuites endpoint. Upon success, we will receive a response along with some JSON data in the body. What we need to record are the ID, associated test cases, and script ID prefix values.

{
"automationSuitesData": [
  {
    "id": 135,
    "name": "Example Test Suite",
    "description": "Automation suite for grouping our automated tests.",
    "active": true,
    "scriptIDPrefix": "EX-",
    "testCases": {
      "testCasesData": [
        {
          …
        }
      ]
     …
    }
  ]
}

Either the build process triggers the creation of an automation suite when it is completed or the automation suite is created before the build process begins, and this will depend on the individual organization. Nonetheless, the outcome is the same because we begin the process of associating the results.

The build process and the results of it will depend on what technology the organization is using. For this portion of the example, we will use a dummy test output for our automation suite process.

Test Report
Start Date: 2025–01–01T14:30:00Z

Operating System: Windows 10 Pro (Version 21H2)

Test Suite: TC-1
Test Case: testValidLogin
Duration: 1.5 seconds
Result: Passed

Test Suite: TC-2
Test Case: testSearchWithInvalidKeyword
Duration: 1.8 seconds
Result: Failed
Error Message: Expected product count: 0 but found: 3
Stack Trace:
at com.example.tests.ProductSearchTests.testSearchWithInvalidKeyword(ProductSearchTests.java:42)

This dummy test report includes test cases 1 & 2, labelled as "test suite" in the dummy report. Our recently created automation suite should already have these test cases associated with it, if not, then we need to add them by making a POST to the

/{projectID}/automationSuites/{automationSuiteID}/testCases

endpoint.

We now start the process of converting the test report into an automation build that can be accepted by the REST API. There will be a number of assumptions going forward and we will note them when clarity may be needed. First, we are assuming that the test report has already been parsed into a data structure that we can use. For this example, we will use C# and will map the dummy test report results to a list of Result records using a function that takes a "test suite" from the report and creates the appropriate Result. Recall that an Automation Build is an item with an array of Result items.

public List<Result> MapTestSuite(TestSuite testSuite) =>
  testSuite.TestCases.Select(testReportCase =>
  {
    return new Result
    {
      Name = ScriptIdPrefix + testReportCase.Number,
      UniqueName = testReportCase.Name,
      Duration = testReportCase.Duration,
      OS = testSuite.OS,
      OsVersion = testSuite.OsVersion,
      StartDate = testSuite.StartDate,
      Status = ConvertTestResult(testReportCase.Result),
      Properties = new List<ResultProperty>
      {
        new() {
          Name = "ErrorMessage",
          Value = testReportCase.Error.ErrorMessage
        },
        new() {
          Name = "StackTrace",
          Value = testReportCase.Error.StackTrace
        }
      }
    };
}).ToList();

For simplicity, we are assuming that the duration has been converted to milliseconds.

The "scriptIDPrefix" comes from when we created the automation suite, and it is needed to associate the test case with the result. Our test report calls the test case "test suite" in this example. We assume that the "TC-" prefix is removed when we are accessing the "Number" property, as shown on line 5.

If you are unfamiliar with C#, line 2 uses the "Select" extension method to project each element of a sequence into a new form by incorporating the element's index.

For line 12, we are assuming that the function "ConvertTestResult" converts the result of the test case to the "status" JSON object that the REST API is expecting. It follows the same pattern as mentioned above, where the ID: 1 => Passed, 2 => Failed, etc.

For the Properties object, the items are stored in a simple list, and we are ignoring any checks if the test case from the report contains an error message or not. There can be additional properties from the test report that require their own entry; for example, "failure type" or "failure value".

With the results created for the automation build, we can continue and add the remaining data: number, description, branch, start date, and the external URL where the test was run. All these values will depend on your individual organization. The only value required by the REST API is number, which it expects as a string and not a numeric value.

Serializing the data we should get something like:

{
  "description": "Example build",
  "branch": "Example Branch",
  "startDate": "2025–01–01T07:58:33.074Z",
  "duration": 533432,
  "number": "2",
  "externalURL": "https://cicd.example.com:8080/job/ExampleJob/2",
  "properties": [
    {
      "name": "Additional Property",
      "value": "Example value"
    }
  ],
  "results": [
    {
      "name": "EX-1",
      "uniqueName": "testValidLogin",
      "status": {
        "id": 1
      },
      "device": null,
      "manufacturer": null,
      "model": null,
      "os": "Windows 10",
      "osVersion": "21H2",
      "browser": null,
      "browserVersion": null,
      "startDate": "2025–01–01T07:58:33.074Z",
      "duration": 1500,
      "externalURL": null,
      "properties": []
    },
    {
      "name": "EX-2",
      "uniqueName": "testSearchWithInvalidKeyword",
      "status": {
        "id": 2
    },
    "os": "Windows 10",
    "osVersion": "21H2",
    "startDate": "2025–01–01T07:58:33.074Z",
    "duration": 1800,
    "properties": [
      {
        "name": "Error Message",
        "value": "Expected product count: 0 but found: 3"
      },
      {
        "name": "Stack Trace",
        "value": "at com.example.tests.ProductSearchTests.testSearchWithInvalidKeyword(ProductSearchTests.java:42)"
      },
     ]
    }
  ]
}

Now we attach it to the body of the POST request and send it to the

/{projectID}/automationSuites/{automationSuiteID}/submitBuild

endpoint. Upon success, we will receive a 201 response along with a
builds data object containing the ID of the build we just submitted.
Conclusion
And that's all there is to it. The difficulty comes with mapping the test report produced by your CI/CD process. With proper planning and understanding of both your testing framework and the Helix ALM REST API structure, this integration can provide invaluable traceability between your development process and your test management system.

This concludes our series about discussing various aspects about the Helix ALM REST API, we hope it has helped explain how things work and clarified any uncertainty. If you need help creating a solution for this process or need support with Helix ALM, contact us at Mecomis and we'll gladly help.

Mecomis has over 20 years of experience with using Helix ALM, so if you have further questions about Helix ALM or a general inquiry, please contact us.