Skip to main content

Sending Cucumber Test Results to Vansah

Connect Cucumber scenarios to Vansah Test Cases, configure the import scripts, upload JSON results, and check the Test Runs created.

Send the results of your automated Cucumber tests to existing Vansah Test Cases. Each tagged scenario creates a Test Run in the Jira work item, Test Folder, or Test Plan you choose.

Before you begin

  • Create the matching Test Cases in Vansah and make sure they are approved, linked to the destination you will use, and not blocked there.

  • Use a Vansah API token whose account can access the Jira space and has the Import Cucumber Report permission. Follow Creating a Vansah API Token.

  • Copy your workspace’s API Connect URL. Use this exact URL rather than guessing the regional host.

  • Generate a Cucumber JSON report in the format described below. The import records execution results; it does not create missing Test Cases.

1. Get the import scripts

Download or clone the Vansah Cucumber integration repository. It includes import_results.sh, import_results.ps1, and env.example.

git clone --branch demo --single-branch https://github.com/testpointcorp/vansah-cucumber.git
cd vansah-cucumber

Use Bash with curl on macOS or Linux, or PowerShell 5.1 or later on Windows. The scripts upload your existing report; your tests can remain in their own code repository.

2. Tag each scenario with its Test Case key

Add one Vansah Test Case tag immediately above each scenario. Replace TEST-C1 with the existing Test Case key.

Feature: User Authentication

@TEST-C1
Scenario: Valid login
Given I am on the login page
When I enter valid credentials
Then I should see the dashboard

The existing example below shows a separate Test Case tag for each scenario.

Cucumber scenarios tagged with TEST-C1 and TEST-C2

Supported tag

Example

Direct Test Case key

@TEST-C1

TC prefix

@TC-TEST-C1

TESTCASE prefix

@TESTCASE-TEST-C1

Use a scenario-level Test Case tag for each result you want to import. Avoid putting a shared Test Case tag on a feature that contains different Test Cases: the importer reads feature tags before scenario tags and uses the first matching key.

Scenarios without a recognized tag are left out before import. If none have a valid tag, the request returns NO_TEST_CASES.

If your scripts already exist in Vansah, you can export a Cucumber feature file with the Test Case tags. You still need the matching step definitions in your automation framework.

3. Generate the JSON report

Use the Cucumber JSON formatter. The importer expects a JSON array of features, each containing scenario elements, tags, and step results. Cucumber Messages (.ndjson), JUnit XML, and other JSON report structures are not accepted by this upload flow.

For an existing JavaScript or TypeScript Cucumber setup:

npx cucumber-js --format json:cucumber.json

See the Cucumber.js formatter documentation. For another framework, select a formatter or converter that produces the same Cucumber JSON structure; a .json extension alone does not make a report compatible.

Use a report from a completed test execution. The importer marks a scenario Failed if a reported step is failed or skipped; otherwise it marks it Passed. Check undefined or pending steps in your Cucumber output before uploading.

4. Configure the destination

In the integration directory, copy env.example to .env. Set one variable per line and quote values containing spaces.

VANSAH_TOKEN="YOUR_VANSAH_API_TOKEN"
VANSAH_PROJECT_KEY=TEST
VANSAH_URL="https://YOUR-API-CONNECT-HOST"
TEST_FOLDER_PATH="Automation/Cucumber"

Replace the token, URL, Jira space key, and folder path with your own values. VANSAH_PROJECT_KEY retains its script variable name and means the Jira space key. The folder must already exist; copy its path from Vansah.

Choose one destination below. Remove or comment out unused destination variables.

Destination

Configuration

Test Folder

TEST_FOLDER_PATH="Automation/Cucumber"

Jira work item

JIRA_ISSUE_KEY=TEST-123

Standard Test Plan

STANDARD_TEST_PLAN_KEY=TEST-P2

Advanced Test Plan

ADVANCED_TEST_PLAN_KEY=TEST-P1, together with TEST_FOLDER_PATH or JIRA_ISSUE_KEY to identify its testing scope.

For a Test Plan, these scripts use the existing Iteration 1. The API also accepts iterationNumber or iterationIdentifier to select another existing iteration, but the example scripts do not expose those fields as environment variables.

You can also set ENVIRONMENT_NAME, SPRINT_NAME, and RELEASE_NAME to add Test Run properties. Use CI secret variables for the token when running in a pipeline, and keep your populated .env file out of source control.

5. Import the report

Run the script from the directory containing your .env file. Supply the report path explicitly so the script uploads the correct file.

macOS or Linux

bash ./import_results.sh -f /path/to/cucumber.json

Windows PowerShell

.\import_results.ps1 -f "C:\path\to\cucumber.json"

The scripts send POST /api/v2/cucumber/import with the Vansah token in the Authorization header and upload the report as Testpath with Testformat=Cucumber_json. Each request supports up to 1,000 tagged scenarios.

6. Check the imported results

  1. Read the response fields imported, failed, skipped, attachments, and warnings. An HTTP success response can still include skipped or failed items.

  2. Open the selected destination in Vansah and inspect the created Test Runs, results, and execution details.

  3. Investigate warnings before uploading again. Missing, unapproved, unlinked, or blocked Test Cases can be skipped.

Important: Uploading the same report again creates additional Test Runs. After a timeout or partial import, check which runs were created before retrying.

Troubleshooting

Problem

What to check

Script or report file not found

Run from the integration directory and pass the correct report path with -f. Generate the report before importing.

Configuration is not picked up

Keep .env in the current working directory. Use NAME=value, quote values with spaces, and avoid inline comments after values. Existing process environment variables take precedence in the PowerShell script.

PowerShell says scripts are disabled

Follow your organization’s PowerShell policy or ask your administrator how to run the approved script.

NO_TEST_CASES

Check the report structure and scenario tags. Use a supported key such as @TEST-C1.

Destination or iteration not found

Check the existing folder path, work item or plan key, Jira space key, and the plan’s Iteration 1.

Authentication or permission error

Check token expiration, the API Connect URL, account access, and Import Cucumber Report permission.

Some items or attachments failed

Read the response warnings and inspect the Test Runs that were created. Share the error details with Vansah support if needed, with tokens and sensitive data removed.

Did this answer your question?