> ## Documentation Index
> Fetch the complete documentation index at: https://docs.qa.tech/llms.txt
> Use this file to discover all available pages before exploring further.

# Azure DevOps pull request testing

> Review an Azure Repos pull request from Azure Pipelines, the API, or an @qa.tech comment.

Use this when the change is an Azure Repos pull request. To run a test plan you already have, use [Azure DevOps regression testing](https://docs.qa.tech/regression-testing/azure-devops.md).

## Set it up

> [card] [Pull request review](#connect-azure-devops)
>
> Azure Pipelines, the API, or an `@qa.tech` comment starts the review.

> [card] [After merge](#after-merge)
>
> No preview environment. Review the merged pull request after the deploy
> step.

## What works on Azure DevOps

| Capability | Azure DevOps |
| :- | :- |
| Review on a preview URL, before merge | Yes. Azure Pipelines, the API, or an `@qa.tech` comment sends the preview URL. Azure DevOps does not report a deployment URL. |
| Review after merge, with no preview URL | Yes. A pipeline step on the default branch can review the merged pull request against staging or production. |
| Review a mobile build for that pull request | Yes. Upload the binary and pass `applicationBuildShortId` on the change review request. The repository still has to be connected. See [Mobile pull request testing](https://docs.qa.tech/pr-testing/agents/mobile.md). |
| Run a regression test plan | Not from this guide. Azure Pipelines can call the Start Run API. See [Azure DevOps regression testing](https://docs.qa.tech/regression-testing/azure-devops.md). |
| Block the pipeline until the review finishes | No. The review posts a comment. The pipeline step fails only when the API rejects the request. |
| Require the review with a branch policy | No. QA.tech does not post a commit status on the pull request. |

## What this does not do

* Azure DevOps has no deployment event QA.tech can wait on. A review starts from Azure Pipelines, the API, or an `@qa.tech` comment.
* [Prepare change review](https://docs.qa.tech/api-reference/chat/prepare-change-review.md) accepts GitHub pull request URLs. An Azure DevOps review starts when you send the preview URL.
* The `triage` fields on the change review API apply to GitHub. On Azure DevOps, triage follows the repository toggle.
* The post-merge agent updates the suite. It does not run the GitHub retro that reads comment reactions.
* This connection is for Azure DevOps Services (`https://dev.azure.com/{organization}` or `https://{organization}.visualstudio.com`).

What a review does on every CI is on [How a review works](https://docs.qa.tech/pr-testing/overview.md#how-a-review-works).

## Connect Azure DevOps

Azure DevOps has no installable app. QA.tech uses an organization URL and a [personal access token](https://learn.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate?view=azure-devops). Comments are posted as the user who created the token.

> [!NOTE]
>
> Create the token on a dedicated Azure DevOps user, not a personal account. A
> personal token stops working when that user is removed, and every comment
> shows that user's name.

**1. Create a personal access token**

In Azure DevOps, create a token with **Code (Read & write)**. That scope lets QA.tech read the repository and post pull request comments, including screenshots.

**2. Connect the organization**

Open [Settings → Integrations → Azure DevOps](https://app.qa.tech/current-project/settings/integrations/azure-devops). Enter the organization URL, such as `https://dev.azure.com/your-organization`, and the token. The connection is shared by every project in the organization.

**3. Add a repository**

On the same page, add each repository QA.tech should review. Pick the applications those pull requests are tested against. With none selected, reviews use the project default application.

**4. Choose how reviews start**

**Review Trigger** defaults to **Azure Pipelines**. **API** is for a script you already run. **Manually** starts a review only from an `@qa.tech` comment. A comment that mentions `@qa.tech` starts or re-runs a review for every trigger.

**5. Subscribe to service hooks**

Saving the repository shows **Service hook setup**: a URL, the username `qatech`, and a basic auth secret. In the Azure DevOps project, open **Project settings → Service hooks**, add a **Web Hooks** subscription for each event below, and paste those three values. Azure DevOps does not sign the payload, so that secret is what authenticates the delivery. [Azure DevOps webhook documentation](https://learn.microsoft.com/en-us/azure/devops/service-hooks/services/webhooks?view=azure-devops) covers the form in their UI.

Subscribe to:

* Pull request created
* Pull request updated
* Pull request merge attempted
* Pull request commented on

**Pull request commented on** is what delivers an `@qa.tech` mention. **Pull request merge attempted** is what tells QA.tech a pull request merged, which the post-merge agent needs.

Each repository needs its own service hooks. The URL and secret appear after the repository is added.

Optional **Review Instructions** on the repository guide every review of that repository, up to 1024 characters. For rules that apply to every repository, use **PR Review Rules** in project knowledge.

## Start a review from Azure Pipelines

Put this step in the same job as the deploy step, after that step has set `DEPLOY_URL` to this pull request's preview URL. Azure DevOps service hooks never include that URL.

```yaml theme={null}
- script: |
    set -euo pipefail
    curl -sSf -X POST "https://api.qa.tech/v1/chat/change-review" \
      -H "Authorization: Bearer $QATECH_API_TOKEN" \
      -H 'Content-Type: application/json' \
      -d '{
        "projectShortId": "proj_abc123",
        "mode": "pr",
        "vcsProviderId": "azure-devops",
        "prUrl": "$(System.CollectionUri)$(System.TeamProject)/_git/$(Build.Repository.Name)/pullrequest/$(System.PullRequest.PullRequestId)",
        "applicationOverrides": [
          {
            "applicationShortId": "app_gXeBl2",
            "environment": { "url": "$(DEPLOY_URL)" }
          }
        ],
        "commitHash": "$(Build.SourceVersion)"
      }'
  displayName: Start QA.tech change review
  condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest'))
  env:
    QATECH_API_TOKEN: $(QATECH_API_TOKEN)
```

In the deploy step, publish the preview URL into `DEPLOY_URL`:

```bash theme={null}
echo "##vso[task.setvariable variable=DEPLOY_URL]$MY_PREVIEW_URL"
```

> [!NOTE]
>
> `$(System.CollectionUri)` already ends with a slash. The JSON body is
> single-quoted so Azure Pipelines expands `$(...)` before the shell runs.
> `$QATECH_API_TOKEN` is a shell variable and stays in double quotes. Secret
> pipeline variables are not exposed to scripts on their own, which is why the
> step maps the token through `env`.

* Create `QATECH_API_TOKEN` as a secret pipeline variable. The value is a QA.tech API key from **Organization Settings → API Keys**.
* Replace `proj_abc123` and `app_gXeBl2` with your project and application short IDs. An organization-scoped key needs `projectShortId`. A project-scoped key may omit it.
* Keep `curl -sSf` so a rejected request fails the step. The step does not wait for the review to finish.
* If the deploy step is in another job, pass `DEPLOY_URL` across jobs with an output variable.
* The repository settings include **Copy agent prompt** when the trigger is **Azure Pipelines**. That prompt contains this step with your project's short IDs filled in.

The same request is documented on [Start change review chat](https://docs.qa.tech/api-reference/chat/start-change-review-chat.md). Send `commitHash` as `$(Build.SourceVersion)` so the review is tied to the commit the pipeline built.

## Start a review from the API

Use this when something other than the pipeline step above already knows the preview URL. The repository still has to be connected, with service hooks, so QA.tech can read the diff and post the review.

```bash theme={null}
curl --request POST \
  --url 'https://api.qa.tech/v1/chat/change-review' \
  --header "Authorization: Bearer $QATECH_API_TOKEN" \
  --header 'Content-Type: application/json' \
  --data '{
    "projectShortId": "proj_abc123",
    "mode": "pr",
    "vcsProviderId": "azure-devops",
    "prUrl": "https://dev.azure.com/your-organization/your-project/_git/your-repository/pullrequest/123",
    "applicationOverrides": [
      {
        "applicationShortId": "app_gXeBl2",
        "environment": { "url": "https://preview.example.com" }
      }
    ],
    "commitHash": "abc123def456"
  }'
```

`prUrl` is the pull request URL: `https://dev.azure.com/{organization}/{project}/_git/{repository}/pullrequest/{id}`. The same path works on `{organization}.visualstudio.com`.

Optional `postMerge` (`enabled`, and an optional `prompt` of up to 4000 characters) overrides the repository's post-merge settings for that pull request. See [Post-merge agent](#post-merge-agent).

## Start a review from a comment

Comment on the pull request and mention `@qa.tech`. This needs the **Pull request commented on** service hook.

Add instructions (`@qa.tech test the checkout flow`) or a preview URL to test a specific environment. With no URL, the review uses the application's configured environment.

QA.tech replies with a link to the conversation. If the organization cannot start a run, that reply says so and the review does not start.

## What QA.tech posts

The finished review is one comment on the pull request. A later review of the same pull request updates that comment instead of adding another. Screenshots in the review are uploaded to the pull request, because Azure DevOps only renders images from its own host.

The comment is authored by the user who created the personal access token.

## Code access

**Code Access** is on for a new repository. Chat and reviews can clone the repository with the connection's token and search its code. Turn it off on the repository to stop cloning.

With **Code Access** on, **Logic Verification (beta)** can model rule-heavy changes (dates, money, permissions, limits) from source and check them with a solver. That can add a few minutes to those reviews.

## PR triage

**PR triage** is on for a new repository. QA.tech skips pull requests that cannot be tested with end-to-end or API tests, and does not post a review comment for those. It needs the pull request service hooks, because triage reads the pull request ingested from them.

The skip rules are the ones on [PR triage](https://docs.qa.tech/pr-testing/pr-triage.md). Azure DevOps does not post a status when a pull request is skipped. Comment `@qa.tech` to run the review anyway.

The API `triage` override is for GitHub pull requests. On Azure DevOps, use the repository's **PR Triage** toggle and **PR triage prompt**.

## Post-merge agent

**Post-merge agent** is off until you enable it on the repository. After a pull request merges, the agent keeps the test suite up to date. It only does what the prompt asks. It needs the **Pull request merge attempted** service hook, and a completed review of that pull request.

Example prompts are on [GitHub's post-merge agent](https://docs.qa.tech/pr-testing/github#post-merge-agent). Azure DevOps uses the same agent and does not run the GitHub retro afterward.

`postMerge` on a change review request overrides `enabled` and `prompt` for that pull request. Omit it to use the repository settings.

## After merge

When pull requests have no preview environment, run the review after the merge lands and the deploy finishes. The agent reads the merged diff and tests staging or production. The result is posted on the merged pull request.

Before you add the step:

* The repository is connected, with the pull request service hooks, so the agent can read the diff and post the result.
* Staging exists in **Settings → Applications & Envs**, and you have its environment short ID (for example `env_aB3xY9`).
* QA.tech can reach staging. If staging is private or behind a VPN, see [Environment access](https://docs.qa.tech/configuration/environment-access.md).

Place this step after the deploy step. It runs on builds that are not pull request validations. A default Azure Repos merge commit starts with `Merged PR 123:`. If your merge commits do not include that line, the step skips them.

```yaml theme={null}
- script: |
    set -euo pipefail
    PR_ID=$(printf '%s\n' "$BUILD_SOURCEVERSIONMESSAGE" | sed -n 's/^Merged PR \([0-9][0-9]*\):.*/\1/p')
    if [ -z "$PR_ID" ]; then
      echo "No merged pull request id in the commit message, skipping"
      exit 0
    fi
    PR_URL="${SYSTEM_COLLECTIONURI}${SYSTEM_TEAMPROJECT}/_git/${BUILD_REPOSITORY_NAME}/pullrequest/${PR_ID}"
    jq -n \
      --arg prUrl "$PR_URL" \
      --arg projectShortId "proj_abc123" \
      --arg applicationShortId "app_gXeBl2" \
      --arg environmentShortId "env_aB3xY9" \
      --arg commitHash "$BUILD_SOURCEVERSION" \
      '{
        projectShortId: $projectShortId,
        mode: "pr",
        vcsProviderId: "azure-devops",
        prUrl: $prUrl,
        commitHash: $commitHash,
        context: "This pull request has already been merged and deployed. Test the changes and report any issues found.",
        applicationOverrides: [
          {
            applicationShortId: $applicationShortId,
            environment: { shortId: $environmentShortId }
          }
        ]
      }' > qatech-request.json
    curl -sSf -X POST "https://api.qa.tech/v1/chat/change-review" \
      -H "Authorization: Bearer $QATECH_API_TOKEN" \
      -H "Content-Type: application/json" \
      --data @qatech-request.json
  displayName: Review the merged pull request
  condition: and(succeeded(), ne(variables['Build.Reason'], 'PullRequest'))
  env:
    QATECH_API_TOKEN: $(QATECH_API_TOKEN)
    BUILD_SOURCEVERSIONMESSAGE: $(Build.SourceVersionMessage)
    BUILD_SOURCEVERSION: $(Build.SourceVersion)
    SYSTEM_COLLECTIONURI: $(System.CollectionUri)
    SYSTEM_TEAMPROJECT: $(System.TeamProject)
    BUILD_REPOSITORY_NAME: $(Build.Repository.Name)
```

The agent tests the URL saved on that environment in QA.tech. When the staging URL changes, update the environment, not the pipeline. To test a URL that is not saved as an environment, send `environment.url` instead of `environment.shortId`.

`ubuntu-latest` includes `curl` and `jq`. The token is a secret pipeline variable. The other `env` entries pass predefined variables into the script without putting them inside the JSON by macro expansion.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.