Bitrise regression testing

Run a test plan from Bitrise against an uploaded mobile build or a web URL.

Start from a test plan. Bitrise uploads the build or calls the API, then runs that plan. For a change review of a pull request build, use Bitrise pull request testing.

Set it up

Mobile build

Upload the APK or simulator app and run the plan against it.

Web app

Start the plan with one POST /v1/run request.

Blocking mode

Poll the run and fail the step when the result is not passed.

New to mobile app testing on QA.tech? Start with Mobile App Testing to create your mobile application first.

1. Store the API token in Bitrise

   Create a token in Organization Settings → API Keys. Open your app in Bitrise, go to Workflow Editor → Secrets, and add:

   • Key: QATECH_API_TOKEN
   • Value: Your API token
   • Keep Expose for Pull Requests disabled unless you need it
2. Get the project short ID

   Copy proj_… from the project URL or Settings (for example proj_abc123). Org-scoped keys need this on POST /v1/run. See Understanding Different IDs.

3. Get the application short ID

   Copy app_… from Settings → Applications & Envs (for example app_gXeBl2). You need this when uploading a mobile build. See Finding Short IDs.

4. Get the test plan short ID

   Open the plan under Test Plans and copy pln_… from the URL (for example pln_abc123). See Find the test plan short ID.

What works on Bitrise

Capability	Bitrise
Review on a preview URL, before merge	No. Bitrise does not post a pull request review.
Review after merge, with no preview URL	No. See Bitrise pull request testing if you call the change review API yourself.
Run a regression test plan from CI	Yes. Mobile: upload the build and start the plan against it. Web: one HTTP request with the test plan short ID.
Run that plan against a mobile build	Yes. This is the primary Bitrise path.
Block the workflow until the plan finishes	Yes. Poll the run and fail the step when the result is not passed.
Schedule	On the test plan, with Manage Schedules. See Manual and scheduled runs.

What this does not do

• The scripts on the Bitrise page do not post a GitHub or GitLab review.
• Bitrise does not replace a test plan schedule. Manage Schedules runs the plan without a Bitrise build.
• Envoyer is unrelated. It is a PHP deploy hook, not a mobile CI. See Envoyer.

Test your mobile builds

Add a script step after your build step (e.g. android-build or xcode-build-for-simulator). It uploads the build to QA.tech and starts a test plan run against it, in four parts:

1. Get a presigned upload URL
2. Upload the build file directly to storage
3. Create the build record
4. Start a test run pinned to that build

For a change review of the same upload (instead of a test plan), use the upload-and-review script on Bitrise pull request testing.

Android (APK)

Bitrise’s android-build step exposes the built APK as $BITRISE_APK_PATH:

- script@1:
    title: Run QA.tech tests on this build
    inputs:
      - content: |
          #!/usr/bin/env bash
          set -euo pipefail

          APP_ID="app_gXeBl2"               # Your QA.tech application short ID
          TEST_PLAN_ID="pln_abc123"         # Your test plan short ID
          BUILD_FILE="$BITRISE_APK_PATH"    # Set by the android-build step
          FILE_NAME=$(basename "$BUILD_FILE")

          # 1. Get a presigned upload URL
          UPLOAD_RESPONSE=$(curl -sSf -X POST "https://api.qa.tech/v1/applications/$APP_ID/builds/upload-url" \
            -H "Authorization: Bearer $QATECH_API_TOKEN" \
            -H "Content-Type: application/json" \
            -d "{\"fileName\": \"$FILE_NAME\"}")
          UPLOAD_URL=$(echo "$UPLOAD_RESPONSE" | jq -r '.uploadUrl')
          BUILD_TOKEN=$(echo "$UPLOAD_RESPONSE" | jq -r '.buildToken')

          # 2. Upload the file directly to storage
          curl -sSf -X PUT "$UPLOAD_URL" \
            --upload-file "$BUILD_FILE" \
            -H "Content-Type: application/octet-stream"

          # 3. Create the build record
          BUILD_RESPONSE=$(curl -sSf -X POST "https://api.qa.tech/v1/applications/$APP_ID/builds" \
            -H "Authorization: Bearer $QATECH_API_TOKEN" \
            -H "Content-Type: application/json" \
            -d "{\"platform\": \"android\", \"buildToken\": \"$BUILD_TOKEN\"}")
          BUILD_SHORT_ID=$(echo "$BUILD_RESPONSE" | jq -r '.applicationBuildShortId')
          echo "Build created: $BUILD_SHORT_ID"

          # 4. Start a test run against this build
          RUN_RESPONSE=$(curl -sSf -X POST "https://api.qa.tech/v1/run" \
            -H "Authorization: Bearer $QATECH_API_TOKEN" \
            -H "Content-Type: application/json" \
            -d "{
              \"testPlanShortId\": \"$TEST_PLAN_ID\",
              \"applications\": [{
                \"applicationShortId\": \"$APP_ID\",
                \"environment\": {
                  \"applicationBuildShortId\": \"$BUILD_SHORT_ID\"
                }
              }]
            }")
          echo "Test run started: $(echo "$RUN_RESPONSE" | jq -r '.run.url')"

Replace app_gXeBl2 and pln_abc123 with your values.

iOS (Simulator build)

QA.tech runs iOS tests on simulators, so the upload must be a simulator build (.app compressed as .zip or .tar.gz) - device and App Store .ipa builds cannot run on simulators. See Mobile App Testing for how to prepare a simulator build.

On Bitrise, use the xcode-build-for-simulator step instead of xcode-archive. It exposes the built .app directory as $BITRISE_APP_DIR_PATH. Zip it before the upload in the script above:

cd "$(dirname "$BITRISE_APP_DIR_PATH")"
zip -r app-simulator.zip "$(basename "$BITRISE_APP_DIR_PATH")"
BUILD_FILE="$PWD/app-simulator.zip"

and use "platform": "ios" when creating the build record.

Supported file types are .apk and .aab for Android, and .zip or .tar.gz containing your .app simulator build for iOS. Maximum file size is 4GB. See the Application Builds API for full request and response details.

Web applications

If you use Bitrise for a web app, trigger a test plan with a single request:

- script@1:
    title: Trigger QA.tech tests
    inputs:
      - content: |
          #!/usr/bin/env bash
          set -euo pipefail
          curl -sSf -X POST "https://api.qa.tech/v1/run" \
            -H "Authorization: Bearer $QATECH_API_TOKEN" \
            -H "Content-Type: application/json" \
            -d '{"testPlanShortId": "pln_abc123"}'

See the Start Run API for all available options, including environment URL overrides for staging or preview deployments.

Blocking mode

To fail the Bitrise build when tests fail (for example as a release gate), poll the run status after starting it:

# Start run and capture shortId
RUN_RESPONSE=$(curl -sSf -X POST "https://api.qa.tech/v1/run" \
  -H "Authorization: Bearer $QATECH_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d "{
    \"testPlanShortId\": \"$TEST_PLAN_ID\",
    \"applications\": [{
      \"applicationShortId\": \"$APP_ID\",
      \"environment\": { \"applicationBuildShortId\": \"$BUILD_SHORT_ID\" }
    }]
  }")
SHORT_ID=$(echo "$RUN_RESPONSE" | jq -r '.run.shortId')

# Poll until completion
while true; do
  STATUS_RESPONSE=$(curl -sSf "https://api.qa.tech/v1/run/$SHORT_ID" \
    -H "Authorization: Bearer $QATECH_API_TOKEN")
  STATUS=$(echo "$STATUS_RESPONSE" | jq -r '.status')
  if [[ "$STATUS" == "COMPLETED" || "$STATUS" == "ERROR" || "$STATUS" == "CANCELLED" ]]; then
    RESULT=$(echo "$STATUS_RESPONSE" | jq -r '.result')
    [[ "$RESULT" == "PASSED" ]] && exit 0 || exit 1
  fi
  sleep 30
done

See the Run Status API for polling details and error handling. If the polling step might exceed your step timeout, raise the step’s timeout in the Bitrise Workflow Editor.