API regression testing

Run a test plan from any system that can send an HTTP request, including a CI QA.tech has no plugin for.

Start from a test plan. Use the API when you do not run GitHub Actions, GitLab CI, Bitrise, or Envoyer. Bitbucket, Azure DevOps, CircleCI, Jenkins, and a script on a server all work.

Set it up

Start Run API

POST /v1/run with testPlanShortId, plus URL or device overrides.

Wait for the result

Poll the run and fail the job when the result is not passed.

CLI

qatech run calls the same API. qatech status waits for the result.

1. Create an API key

   Go to Organization Settings → API Keys. Create a key and copy it when it is shown. It is not displayed again. Store it as a masked secret in your CI, for example QATECH_API_TOKEN.

   A project-scoped key is bound to one project, so requests can omit projectShortId. An organization-scoped key needs projectShortId on project-scoped endpoints. See Authentication.

2. Send an authenticated request

   Every request goes to https://api.qa.tech/v1 with the key as a Bearer token:

   curl https://api.qa.tech/v1/applications \
     -H "Authorization: Bearer $QATECH_API_TOKEN"
   

   A list of your applications means the key works. A 401 means the key is missing, mistyped, or revoked.

3. Get the project short ID

   Copy proj_… from the project URL or Settings (for example proj_abc123). Org-scoped keys need this on POST /v1/run. See Understanding Different IDs.

4. Get the test plan short ID

   Open the plan under Test Plans and copy pln_… from the URL (for example pln_abc123). See Find the test plan short ID.

Find the IDs you need

ID	Example	Used for	Where to find it
Project short ID	proj_abc123	Organization-scoped keys	Project URL or Settings
Test plan short ID	pln_abc123	Running a test plan	Find the test plan short ID
Application short ID	app_gXeBl2	Pointing a run or a review at a preview URL or build	Finding Short IDs
Environment short ID	env_aB3xY9	Targeting an existing environment instead of a URL	Finding Short IDs

The full list of ID types is in Understanding Different IDs.

What works over the API

Capability	API
Review on a preview URL, before merge	Not this endpoint. A test plan run does not post a review. See the pull request API.
Review after merge, with no preview URL	Not this endpoint.
Run a regression test plan from CI	Yes. POST /v1/run with testPlanShortId. You can override the URL or device for that run.
Run that plan against a mobile build	Yes. Upload the build, then pass applicationBuildShortId. See Application builds.
Block the pipeline until the plan finishes	Yes. Poll Get run and fail the job when the result is not passed.
Schedule	On the test plan, with Manage Schedules. See Manual and scheduled runs.

Patterns

Start a test plan

curl -X POST https://api.qa.tech/v1/run \
  -H "Authorization: Bearer $QATECH_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "testPlanShortId": "pln_abc123"
  }'

Mobile build

Upload the APK or simulator .app, then pass applicationBuildShortId instead of a URL. How application builds work is on Mobile regression testing. The shared upload script is on API pull request testing.

curl -X POST https://api.qa.tech/v1/run \
  -H "Authorization: Bearer $QATECH_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "testPlanShortId": "pln_abc123",
    "applications": [{
      "applicationShortId": "app_gXeBl2",
      "environment": {
        "applicationBuildShortId": "bld_abc123"
      }
    }]
  }'

Blocking

Poll the run and fail the job when the result is not passed:

RESPONSE=$(curl -sSf -X POST "https://api.qa.tech/v1/run" \
  -H "Authorization: Bearer $QATECH_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"testPlanShortId": "pln_abc123"}')
SHORT_ID=$(echo "$RESPONSE" | jq -r '.run.shortId')

while true; do
  STATUS_RESPONSE=$(curl -sSf "https://api.qa.tech/v1/run/$SHORT_ID" \
    -H "Authorization: Bearer $QATECH_API_TOKEN")
  STATUS=$(echo "$STATUS_RESPONSE" | jq -r '.status')
  if [[ "$STATUS" == "COMPLETED" || "$STATUS" == "ERROR" || "$STATUS" == "CANCELLED" ]]; then
    RESULT=$(echo "$STATUS_RESPONSE" | jq -r '.result')
    [[ "$RESULT" == "PASSED" ]] && exit 0 || exit 1
  fi
  sleep 30
done

See Get run for polling details.

What you can override on a run

• Preview or staging URL: set applications[].environment.url. See Preview Environments.
• Environment custom headers: attach auth or protection-bypass headers with customHeaders. See Environment custom headers.
• Device preset: pass devicePresetShortId to test mobile, tablet, or desktop without another test plan. See Start Run API.
• Slack channel: send results for one run to a different channel. See Per-run overrides.
• Post-run automation: poll Get run to update a status page, call a webhook, or send an alert when the run finishes.

What this does not do

• It does not post a pull request review. Use API pull request testing for that.
• It does not require GitHub, GitLab, Bitrise, or Envoyer. Those guides are convenience wrappers around this call.
• It does not schedule itself. Manage Schedules on the test plan runs it with no CI at all. See Manual and scheduled runs.