How to send test results to Vansah from your CI/CD pipeline

vansah-connect is a npm command-line tool. After your automated suite runs, it uploads the report it produced (TestNG XML or Cucumber JSON) to Vansah, or records a single Test Case result directly - against a Jira issue, a Test Folder, or a Standard/Advanced Test Plan. It works with any pipeline - GitHub Actions, Jenkins, GitLab CI, Bitbucket Pipelines, Azure DevOps and connects to the Vansah REST API v2.

Step 1: Install vansah-connect

vansah-connect is a command-line tool, so install it globally with npm:

npm i -g @vansah/vansah-connect

Confirm it is available:

vansah-connect --help

Step 2: Configure your token, URL and Space Key

vansah-connect reads its settings from three sources, in order of priority:

For local use, save your settings once. They are stored in ~/.vansah-connect/.env and reused for every project:

vansah-connect -c "YOUR_CONNECT_TOKEN"

vansah-connect -v "https://prod.vansah.com"

vansah-connect -p "DEMO"

Optionally, save run properties applied to every run:

vansah-connect --environment "UAT" --sprint "Sprint 1" --release "v1.2"

For a pipeline, provide the same values as environment variables instead (see Step 5):

Note: The Space Key is required for Cucumber uploads and single-result logging. TestNG uploads read the project from the Case Key prefix in the report.

Tip: Run vansah-connect --show-config at any time to print the effective configuration (the Connect token is masked).

Step 3: Upload your results

Choose the option that matches how your tests produce results. The --format flag is required whenever you upload a file with -f.

Add Vansah custom attributes to your TestNG methods so each result knows its Test Case and Jira issue:

@Test(attributes = {

@CustomAttribute(name = "Case Key", values = "DEMO-C1"),

@CustomAttribute(name = "Tested Issue", values = "DEMO-1"),

@CustomAttribute(name = "Tested Sprint", values = "DEMO Sprint 1"),

@CustomAttribute(name = "Tested Environment", values = "SYS")})

public void Addition_Test() { /* ... */ }

The attribute names are constant and case-sensitive. Case Key and Tested Issue are mandatory; Tested Sprint and Tested Environment are optional. After the suite runs, upload the generated report:

vansah-connect -f ./target/testng-results.xml --format testng

Tag each scenario with its Vansah Test Case key in the form @{PROJECT}-

C{NUMBER}:

@DEMO-C54

Scenario: Valid login with correct credentials

Given I am on the login page

When I enter valid credentials

Then I should see the dashboard

Tip: You can export ready-tagged .feature files directly from Vansah - Export > Test Cases and Test Script > Cucumber (.feature). See Cucumber Feature Export.

Scenarios without a Vansah tag are skipped.

Generate the JSON report from your Cucumber run (the command differs per language, the format is the same):

npx cucumber-js --format json:cucumber.json

Then upload it, choosing where the runs are recorded with -a:

# Against a Jira issue

vansah-connect -f ./cucumber.json --format cucumber -a DEMO-9

# Against a Test Folder (any value containing "/")

vansah-connect -f ./cucumber.json --format cucumber -a "regression/login/"

Not sure of the folder path? See Getting the folder path of a test folder.

To record one result without a report file:

vansah-connect -t "DEMO-C50" -s "passed" -a "DEMO-9"

· -s accepts passed, failed, n/a, or untested.

· -a is a Test Folder path if it contains "/", otherwise a Jira issue key.

Step 4: Target a Test Plan (optional)

To record runs against a Standard Test Plan (STP) or Advanced Test Plan (ATP) instead of an issue or folder, use --mode with the matching plan key:

# Cucumber upload against a Standard Test Plan

vansah-connect -f ./cucumber.json --format cucumber --mode stp --stp DEMO-P2

# Cucumber upload against an Advanced Test Plan (needs -a as context)

vansah-connect -f ./cucumber.json --format cucumber --mode atp --atp DEMO-P1 -a DEMO-9

# Single result against a Standard Test Plan

vansah-connect -t DEMO-C50 -s passed --mode stp --stp DEMO-P2

Used on their own, --mode, --stp and --atp are saved as the default target for future runs. Inline flags always take precedence, and passing -a on a run forces a normal issue/folder run regardless of the saved mode.

Note: Advanced Test Plans require an issue/folder context (-a) alongside the plan key, and are supported for Cucumber uploads only - not for single results (-t).

Step 5: Run it in your pipeline

Provide the configuration as masked environment variables and call the tool after your tests run. Example - GitHub Actions:

- name: Upload results to Vansah

env:

VANSAH_TOKEN: ${{ secrets.VANSAH_TOKEN }}

VANSAH_URL: https://prod.vansah.com

VANSAH_PROJECT_KEY: DEMO

run: |

npm i -g @vansah/vansah-connect

vansah-connect -f ./cucumber.json --format cucumber -a DEMO-9

Step 6: View your results in Vansah

1. Open the linked Jira issue, Test Folder, or Test Plan in Vansah.

2. Confirm the new test run appears, with its step results (and for Cucumber, each Gherkin step).

"no Vansah Connect token found" - No token was found. Set VANSAH_TOKEN, add it to a .env in your working directory, or run vansah-connect -c "YOUR_TOKEN".

"--format is required with -f" - Always state the report format explicitly: --format testng or --format cucumber.

HTTP 401 / Unauthorized - The token is invalid or expired. Generate a new one in Apps > Vansah > Settings > Vansah API Tokens.

HTTP 404 / Not Found - Check that VANSAH_URL matches your workspace's API Connect URL and that VANSAH_PROJECT_KEY is your exact Jira project key.

Cucumber scenarios skipped - A scenario has no @{PROJECT}-C{NUMBER} tag, or the tagged Test Case does not exist in the Space. Confirm the tag and Space Key.

Test Plan errors - No plan key configured for the selected --mode (save one with --stp/--atp or pass it inline), or --mode atp was used without the required -a context / with a single result (-t), which is not supported.