# Application Builds API Source: https://docs.qa.tech/api-reference/application-builds Upload and manage mobile app builds for testing The Application Builds API enables you to upload mobile app builds (APK/IPA files) for testing with QA.tech. Use these endpoints to integrate mobile app build uploads into your CI/CD pipelines and automation workflows. ## When to Use This API * **CI/CD integration** – Automatically upload builds after successful compilation * **Mobile app testing** – Upload iOS and Android builds for automated testing * **Version management** – Track and test different build versions This API is for **mobile applications** (iOS and Android) only. For web applications, use environment URLs directly in the [Start Run API](/api-reference/runs/start-test-run) or [Chat API](/api-reference/chat). ## Supported File Types | Platform | File Types | | -------- | ---------- | | Android | `.apk` | | iOS | `.ipa` | Android App Bundles (`.aab`) are not supported directly. Convert them to `.apk` first — see [Mobile App Testing](/test-features/mobile-app-testing) for instructions. ## Authentication All endpoints require Bearer token authentication. Create your API key in the QA.tech dashboard: **Organization Settings → API Keys**. The key is shown only once, at creation. See the [API Introduction](/api-reference/introduction#authentication) for details. ## Upload Workflow Uploading a build is a two-step process, plus the file transfer itself: 1. **[Get Build Upload URL](/api-reference/application-builds/get-build-upload-url)** – Request a presigned URL and a `buildToken` for your file. 2. **Upload the file** – Send the raw file bytes with an HTTP `PUT` request to the presigned `uploadUrl`. No `Authorization` header is needed — the URL is presigned. 3. **[Create Application Build](/api-reference/application-builds/create-application-build)** – Register the uploaded file as a build, using the `buildToken` from step 1. The response includes the build's short ID (e.g. `build_abc123`). This flow allows efficient direct uploads to cloud storage without passing the file through the API server. See the linked reference pages for full request, response, and error details. The presigned URL is temporary and valid for about 2 hours. Upload the file promptly after requesting the URL. ## Using Builds in Test Runs After creating a build, reference its short ID as the environment's `applicationBuildShortId` in an application override when [starting a run](/api-reference/runs/start-test-run) or [creating a chat conversation](/api-reference/chat). The run or chat session will then test against that specific build. ## Related * [Mobile App Testing](/test-features/mobile-app-testing) – Mobile testing concepts * [Start Run API](/api-reference/runs/start-test-run) – Run tests against uploaded builds * [Applications API](/api-reference/applications) – List applications * [API Introduction](/api-reference/introduction) – Authentication and ID reference # Create application build Source: https://docs.qa.tech/api-reference/application-builds/create-application-build /api-reference/api.json post /v1/applications/{applicationShortId}/builds Step 2 of 2 for attaching a mobile build: registers a file already uploaded via `application_build_upload_url`. Pass the `buildToken` returned by that tool (after you PUT the file to its `uploadUrl`). Returns `applicationBuildShortId` — use it as `environment.applicationBuildShortId` inside an `applications[]` override in `start_run_with_test_cases` / `start_run_with_test_plan`, or an `applicationOverrides[]` override in `create_chat`, to run/chat against this build. # Get build upload URL Source: https://docs.qa.tech/api-reference/application-builds/get-build-upload-url /api-reference/api.json post /v1/applications/{applicationShortId}/builds/upload-url Step 1 of 2 for attaching a mobile build (.apk/.ipa) to a run. Returns a presigned `uploadUrl` and a `buildToken`. Next, upload the raw build file to `uploadUrl` with an HTTP PUT request (request body = the raw file bytes, `Content-Type` = the file's MIME type such as `application/octet-stream`; no auth header — the URL is presigned), then call `create_application_build` with the same `buildToken`. # Applications API Source: https://docs.qa.tech/api-reference/applications List applications and their environments programmatically Retrieve applications and their environments from your project. Use these endpoints to discover application and environment short IDs for use with other API endpoints like [Start Run](/api-reference/runs/start-test-run) and [Chat](/api-reference/chat). You can also find Application and Environment Short IDs in the QA.tech dashboard: **Settings → Applications & Envs**. The short IDs (e.g. `app_gXeBl2`, `env_aB3xY9`) are displayed in the UI and can be copied using the three-dot menu (⋮). ## Endpoints For full request, response, and error details, see the generated reference pages: * [List Applications](/api-reference/applications/list-applications) – all applications in your project, including their kind (web or mobile) * [List Application Environments](/api-reference/applications/list-application-environments) – the environments configured for a specific application For mobile applications, the iOS/Android platform is set per build when [uploading application builds](/api-reference/application-builds), not on the application itself. ## Authentication All endpoints require Bearer token authentication. Create your API key in the QA.tech dashboard: **Organization Settings → API Keys**. The key is shown only once, at creation. See the [API Introduction](/api-reference/introduction#authentication) for details, including how organization-scoped keys pass `projectShortId`. ## Typical Workflow A common pattern is to discover IDs dynamically instead of hardcoding them: 1. Call [List Applications](/api-reference/applications/list-applications) to find the application's `shortId`. 2. Call [List Application Environments](/api-reference/applications/list-application-environments) with that ID to find the environment you want to test against (for example, the first non-production environment, using the `isProduction` flag). 3. Pass the discovered IDs as application/environment overrides when [starting a run](/api-reference/runs/start-test-run) or [creating a chat conversation](/api-reference/chat). This is especially useful in CI/CD pipelines, where the target environment may vary per branch or deployment. ## Related * [Start Run API](/api-reference/runs/start-test-run) – Use application/environment IDs to run tests * [Chat API](/api-reference/chat) – Use application overrides in chat conversations * [Applications and Environments](/core-concepts/applications-and-environments) – Learn about the application model * [API Introduction](/api-reference/introduction) – Authentication and ID reference # List application environments Source: https://docs.qa.tech/api-reference/applications/list-application-environments /api-reference/api.json get /v1/applications/{applicationShortId}/environments List environments for a specific application # List applications Source: https://docs.qa.tech/api-reference/applications/list-applications /api-reference/api.json get /v1/applications Lists the applications under test in the project the API key is bound to. # Chat API Source: https://docs.qa.tech/api-reference/chat Interact with the QA.tech AI Chat Assistant programmatically The Chat API enables programmatic interaction with QA.tech's AI Chat Assistant. Create conversations, send messages, start change reviews, and poll for responses—all via REST API. This is ideal for integrating QA.tech's AI capabilities into your automation workflows, project management systems, or custom tooling. ## When to Use This API * **Automated test creation** – Send context from your system to create tests without opening the QA.tech UI * **PR/Change reviews** – Automatically trigger AI-powered change reviews from CI/CD pipelines * **Custom integrations** – Build buttons or workflows in your project management tools that interact with QA.tech Chat * **AI-assisted QA workflows** – Let your automation systems communicate with QA.tech's AI assistant For simpler chat interactions, consider using the [CLI chat command](/cli/commands/chat) which handles polling automatically: `qatech chat "Create a login test"` ## Authentication All endpoints require Bearer token authentication. Create your API key in the QA.tech dashboard: **Organization Settings → API Keys**. The key is shown only once, at creation. See the [API Introduction](/api-reference/introduction#authentication) for details. ## Endpoints For full request, response, and error details, see the generated reference pages: * [Start Chat Conversation](/api-reference/chat/start-chat-conversation) – Create a conversation and send the first message * [Start Change Review Chat](/api-reference/chat/start-change-review-chat) – Create a conversation that reviews a pull request or raw code changes * [Send Chat Message](/api-reference/chat/send-chat-message) – Send a follow-up message to an existing conversation * [Get Chat Conversation](/api-reference/chat/get-chat-conversation) – Retrieve conversation metadata and messages ## How Conversations Work Chat processing is **asynchronous**. Creating a conversation or sending a message returns `202 Accepted` immediately with an empty `messages` array — the assistant's reply is not included. To read the reply, poll [Get Chat Conversation](/api-reference/chat/get-chat-conversation) until the most recent `assistant` message reaches the `COMPLETED` status. We recommend polling every **2-5 seconds**. Assistant responses typically complete within 10-60 seconds depending on the complexity of the request. Test creation and execution may take several minutes. You can target a specific application environment for the conversation — for example a preview deployment or an uploaded mobile build — by passing application overrides when creating it. See the [Start Chat Conversation](/api-reference/chat/start-chat-conversation) reference for the override format. ## Change Reviews [Start Change Review Chat](/api-reference/chat/start-change-review-chat) creates a conversation that initiates an AI-powered review of code changes. It supports two modes: * **PR mode** – Provide a pull request URL (GitHub or GitLab) and QA.tech fetches the changes * **Raw changes mode** – Provide a change description and a git diff directly, useful when the changes are not in a hosted PR Both modes accept optional extra context to help the AI understand the changes, and require application overrides specifying which environment to review against. For GitHub pull requests (`mode: "pr"`, `vcsProviderId: "github"`), an explicit change-review request also creates or updates the **QA.tech / PR Review** check on the PR. This applies even when **Auto-run on PRs** is disabled in the GitHub App integration. See [PR Review check on GitHub](/configuration/github-app#pr-review-check-on-github). ## Related * [AI Chat Assistant](/core-concepts/ai-chat-assistant) – Learn about QA.tech's AI assistant * [API Introduction](/api-reference/introduction) – Authentication and ID reference * [Start Run API](/api-reference/runs/start-test-run) – Trigger test runs programmatically # Get chat conversation Source: https://docs.qa.tech/api-reference/chat/get-chat-conversation /api-reference/api.json get /v1/chat/{chatConversationShortId} Get conversation metadata and recent messages (newest first). Use `limit` to control how many messages are returned (default 20). To detect when an assistant reply is finished, poll until the most recent `assistant` message has `status: 'COMPLETED'` (or `CANCELLED`/`FAILED`). # Send chat message Source: https://docs.qa.tech/api-reference/chat/send-chat-message /api-reference/api.json post /v1/chat/{chatConversationShortId} Send a chat message to an existing conversation. Returns 202 immediately. Poll `get_chat_conversation` until the latest assistant message has `status: 'COMPLETED'` to read the reply. Wait at least 1s between polls. # Start change review chat Source: https://docs.qa.tech/api-reference/chat/start-change-review-chat /api-reference/api.json post /v1/chat/change-review Create a new chat conversation and start an autonomous change review from either a PR URL or raw change input. # Start chat conversation Source: https://docs.qa.tech/api-reference/chat/start-chat-conversation /api-reference/api.json post /v1/chat Create a new chat conversation and send its first message. This is the recommended way to author tests. The QA.tech agent explores the site, grounds tests in what actually exists, and sets up login and config dependencies, so the tests are reliable rather than flaky. Prefer it over `create_test_case` unless you have already verified the flow exists and gathered the details that tool requires. Returns 202 immediately with the created conversation. Processing is asynchronous: poll `get_chat_conversation` until the latest assistant message has `status: 'COMPLETED'` to read the reply. Wait at least 1s between polls. # Exporting Test Cases Source: https://docs.qa.tech/api-reference/exporting-test-cases Export your test case definitions, including steps, via the API or CLI Export all test case definitions from your project, including names, goals, expected results, and step-by-step instructions. This is useful for migrating to another test management tool (for example Jira with Xray or Zephyr), keeping external backups, or building custom reports. ## What Gets Exported The [List Test Cases](/api-reference/test-cases/list-test-cases) endpoint returns published test cases. Each test case includes its name, goal, expected result, labels, enabled state, classification, and the most recent run status. When you pass `includeSteps=true`, each test case also includes a `steps` array with step-by-step instructions. See the [List Test Cases reference](/api-reference/test-cases/list-test-cases) for the full response schema. Steps are opt-in via the `includeSteps` query parameter to keep default responses small. Test cases that are defined by a goal only (without a step breakdown) return an empty `steps` array. ## Export via the API Pass `includeSteps=true` to include the full step definitions: ```bash theme={null} curl "https://api.qa.tech/v1/test-cases?includeSteps=true&limit=1000" \ -H "Authorization: Bearer YOUR_API_TOKEN" ``` The endpoint returns up to 1000 test cases per request. If your project has more, page through the results with `offset`: ```bash theme={null} curl "https://api.qa.tech/v1/test-cases?includeSteps=true&limit=1000&offset=1000" \ -H "Authorization: Bearer YOUR_API_TOKEN" ``` See the [API Introduction](/api-reference/introduction#authentication) for how to create an API token. With an organization-scoped key, also pass `projectShortId` as a query parameter. ## Export via the CLI The [qatech CLI](/cli/overview) wraps the same endpoint: ```bash theme={null} qatech test-cases --include-steps --json > test-cases.json ``` See [qatech test-cases](/cli/commands/test-cases) for all options. ## Converting to CSV Many test management importers expect one row per step. This `jq` one-liner converts the JSON export into that shape: ```bash theme={null} qatech test-cases --include-steps --json | jq -r ' ["Test Case ID", "Test Case Name", "Step Number", "Step Instruction", "Step Expected Result"], (.testCases[] | . as $tc | .steps // [] | to_entries[] | [$tc.id, $tc.name, (.key + 1), .value.instruction, (.value.expectedResult // "")]) | @csv' > test-case-steps.csv ``` For a one-row-per-test-case shape with numbered steps in a single column: ```bash theme={null} qatech test-cases --include-steps --json | jq -r ' ["Test Case ID", "Name", "Goal", "Expected Result", "Steps"], (.testCases[] | [.id, .name, (.goal // ""), (.expectedResult // ""), ([.steps // [] | to_entries[] | "\(.key + 1). \(.value.instruction)"] | join("\n"))]) | @csv' > test-cases.csv ``` ## Related * [List Test Cases](/api-reference/test-cases/list-test-cases) - Full endpoint reference * [qatech test-cases](/cli/commands/test-cases) - CLI command reference * [API Introduction](/api-reference/introduction) - Authentication and ID reference # Get feature Source: https://docs.qa.tech/api-reference/features/get-feature /api-reference/api.json get /v1/features/{featureId} Fetch detail for a single feature, including pages it spans and actions that belong to it # List features Source: https://docs.qa.tech/api-reference/features/list-features /api-reference/api.json get /v1/features List features in the project's feature graph. Features are LLM-clustered groupings of related UI actions discovered during crawling, and describe what the product can do. # Get outbound IPs Source: https://docs.qa.tech/api-reference/infrastructure/get-outbound-ips /api-reference/api.json get /v1/outbound-ips Get the current outbound IP addresses QA.tech test runners use, for firewall rules and allowlists. No authentication required. # API Introduction Source: https://docs.qa.tech/api-reference/introduction Learn how to interact with QA.tech programmatically using our REST API The QA.tech REST API allows you to programmatically control test runs, create test cases, check results, and integrate QA.tech into your CI/CD pipelines and automation workflows. ## What Can You Do? * **Start test runs** programmatically from CI/CD pipelines or scripts * **Create and list test cases** via API for importing tests or generating them dynamically * **Check run status** and retrieve test results * **Rerun tests** (all or a subset) from a previous run * **Chat with AI assistant** to create tests and review changes via API * **List applications and environments** for dynamic test configuration * **Create remote tunnels** to test local environments * **Upload mobile app builds** for iOS and Android testing * **Get outbound IPs** for firewall allowlisting ## Quick Start Here's a complete example to start a test run: ```bash theme={null} curl -X POST https://api.qa.tech/v1/run \ -H "Authorization: Bearer YOUR_API_TOKEN" \ -H "Content-Type: application/json" \ -d '{"testPlanShortId": "pln_abc123"}' ``` See [Start Test Run](/api-reference/runs/start-test-run) for all request options, including application/environment overrides. **What you need:** 1. **API Token** - Created in **Organization Settings → API Keys** (the key is shown only once, at creation) 2. **Test Plan Short ID** - Prefixed ID from your test plan (e.g. `pln_abc123`). Alternatively pass `testCaseIds` to run specific test cases instead of a plan 3. **Other short IDs** as needed - Application, Environment, Config, Device Preset, Scenario (see table below) ## Authentication All API endpoints (except a few public ones such as [Get Outbound IPs](/api-reference/infrastructure/get-outbound-ips) and the status badge endpoints) require Bearer token authentication. Include your API token in the `Authorization` header: ```bash theme={null} Authorization: Bearer YOUR_API_TOKEN ``` **Where to find your API token:** 1. Navigate to **Organization Settings → API Keys** 2. Create a new API key 3. Copy the key when it is shown — it will not be displayed again API keys can be **project-scoped** or **organization-scoped**. With a project-scoped key the project is inferred from the token, so you can omit `projectShortId`. With an organization-scoped key you must pass `projectShortId` (e.g. `proj_abc123`) in the request body or query string of project-scoped endpoints. ## Understanding Different IDs QA.tech uses **prefixed short IDs** for most API resources. The prefix indicates the resource type. Some fields (e.g., dependency test case IDs in Create Test Case and Rerun endpoints) still use UUIDs and are documented on their respective pages: | ID Type | Format | Where to Find | Used In API | Example | | -------------------------- | -------------------------- | --------------------------------------------- | ----------------------------------------------- | ----------------- | | **Project Short ID** | `proj_` + alphanumeric | Project URLs, Settings | `projectShortId` (required for org-scoped keys) | `proj_abc123` | | **Test Plan Short ID** | `pln_` + alphanumeric | Test plan URLs, Settings | `testPlanShortId` in request body | `pln_abc123` | | **Application Short ID** | `app_` + alphanumeric | Settings → Applications & Envs | `applicationShortId` in applications array | `app_gXeBl2` | | **Environment Short ID** | `env_` + alphanumeric | Settings → Applications & Envs → Environments | `environment.shortId` in applications array | `env_aB3xY9` | | **Config Short ID** | `cfg_` + alphanumeric | Settings → Configs | `configShortIds` in test-cases body | `cfg_xyz789` | | **Device Preset Short ID** | `preset_` + alphanumeric | Settings → Device Presets | `devicePresetShortId` in applications array | `preset_abc123` | | **Scenario Short ID** | `scenario_` + alphanumeric | Scenarios | `scenarioShortId` in test-cases body | `scenario_abc123` | | **Chat Conversation ID** | `chat_` + alphanumeric | Chat API response | `chatConversationShortId` in chat endpoints | `chat_abc123` | | **Application Build ID** | `build_` + alphanumeric | Build API response | `applicationBuildShortId` in environment | `build_abc123` | ### Finding Application and Environment Short IDs Application and Environment Short IDs are shown in the UI with their `app_` and `env_` prefixes. Navigate to **Settings → Applications & Envs** in your project The Application Short ID (e.g. `app_gXeBl2`) appears in the applications table. Use the three-dot menu (⋮) to **Copy Short ID** Under **Settings → Applications & Envs → \[Select Application]**, the Environment Short ID (e.g. `env_aB3xY9`) appears in the **Environments** section. Use the copy action to copy it ## Next Steps * **[Start a Run](/api-reference/runs/start-test-run)** - Trigger test runs programmatically * **[Get Run Status](/api-reference/runs/get-run)** - Wait for test completion in CI/CD pipelines * **[Rerun Tests](/api-reference/runs/rerun-run)** - Rerun all or failed tests from a previous run * **[Create Test Cases](/api-reference/test-cases/create-test-case)** - Create tests via API * **[List Test Cases](/api-reference/test-cases/list-test-cases)** - Retrieve test cases from your project * **[Chat API](/api-reference/chat)** - Interact with the AI assistant programmatically * **[Applications API](/api-reference/applications)** - List applications and environments * **[Remote Tunnels API](/api-reference/remote-tunnels)** - Create tunnels to test local environments * **[Application Builds API](/api-reference/application-builds)** - Upload mobile app builds * **[Get Outbound IPs](/api-reference/infrastructure/get-outbound-ips)** - Retrieve IPs for firewall allowlisting * **[View OpenAPI Spec](/api-reference/api.json)** - Complete API reference with all endpoints ## Base URL All API requests are made to: ``` https://api.qa.tech/v1 ``` ## Migration from Legacy API If you used the legacy API format: * **Old:** `https://app.qa.tech/api/projects/{projectUuid}/runs` * **New:** `https://api.qa.tech/v1/run` (project is inferred from your API key) The old format may still work temporarily but is **deprecated**. Key changes in the current API: * No `projectUuid` in the path; the project is determined by your API token * All IDs use **prefixed short IDs** (e.g. `pln_abc123`, `app_gXeBl2`) instead of UUIDs where applicable * Base URL is `https://api.qa.tech/v1` Migrate to the new base URL and request format when possible. # Get issue Source: https://docs.qa.tech/api-reference/issues/get-issue /api-reference/api.json get /v1/issues/{shortId} Fetches a single issue by its short ID, including its tags and the recent run test cases where it was detected (newest first). Use the occurrences' `runTestCaseShortId` with `get_run_test_case` to inspect what the agent saw. # List issues Source: https://docs.qa.tech/api-reference/issues/list-issues /api-reference/api.json get /v1/issues Lists issues QA.tech has found in the project, deduplicated across the runs where they recur and sorted by most recent occurrence (newest first). Defaults to `ACTIVE` issues; filter by `severity`, `status`, or a `since`/`until` window. Each issue includes its severity, status, type, a help URL, timestamps, and a deep link. Pass a returned `shortId` to `get_issue` for the run test cases that triggered it. # Create knowledge item Source: https://docs.qa.tech/api-reference/knowledge/create-knowledge-item /api-reference/api.json post /v1/knowledge Add an item to the project's knowledge base and index it for the QA.tech agents. Use `text` for documentation notes, `link` to crawl and index a URL, or `memory` for agent-maintained observations. PDF and icon items are managed in the dashboard. # Delete knowledge item Source: https://docs.qa.tech/api-reference/knowledge/delete-knowledge-item /api-reference/api.json delete /v1/knowledge/{knowledgeItemId} Delete a knowledge item and remove it from the agents' knowledge index # Get knowledge item Source: https://docs.qa.tech/api-reference/knowledge/get-knowledge-item /api-reference/api.json get /v1/knowledge/{knowledgeItemId} Fetch a single knowledge item including its full text content # List knowledge items Source: https://docs.qa.tech/api-reference/knowledge/list-knowledge-items /api-reference/api.json get /v1/knowledge List the project's knowledge base items: text notes, crawled links, uploaded PDFs, agent memories and UI icons. Returns summaries without content; fetch a single item for its full content. # Update knowledge item Source: https://docs.qa.tech/api-reference/knowledge/update-knowledge-item /api-reference/api.json patch /v1/knowledge/{knowledgeItemId} Update a text, link or memory knowledge item. Changing content or url re-indexes the item; pdf and icon items are managed in the dashboard. # Get project metrics Source: https://docs.qa.tech/api-reference/metrics/get-project-metrics /api-reference/api.json get /v1/metrics Prometheus text exposition format for the project bound to the API key. # Create project Source: https://docs.qa.tech/api-reference/projects/create-project /api-reference/api.json post /v1/projects Create a new project in the organization the API key belongs to. Requires an organization-scoped API key with write scope. # List projects Source: https://docs.qa.tech/api-reference/projects/list-projects /api-reference/api.json get /v1/projects Lists the projects in the organization the API key belongs to. Organization-scoped keys see every project; project-scoped keys see only their own. Returned `shortId`s can be passed as `projectShortId` to any project-scoped tool to target that project. # Get release check Source: https://docs.qa.tech/api-reference/release-checks/get-release-check /api-reference/api.json get /v1/release-checks/{shortId} Fetch the compiled release check report by its short ID. Poll until `status` is `completed`. # Start release check Source: https://docs.qa.tech/api-reference/release-checks/start-release-check /api-reference/api.json post /v1/release-checks Start a release check: an orchestrated pre-release report that runs your regression suite and/or an autonomous exploratory change review, then compiles the results into a single report. Request at least one of a regression suite (`regression`) or an exploratory change review (`prUrl` or `changes`). All testing and analysis runs on QA.tech — it does not consume the caller's tokens or credits. Returns 202 immediately. Poll `get_release_check` until `status` is `completed` (or `failed`) to read the compiled report. # Remote Tunnels API Source: https://docs.qa.tech/api-reference/remote-tunnels Create and manage secure tunnels to expose local environments The Remote Tunnels API enables you to programmatically create secure tunnels that expose local ports via Cloudflare. This allows QA.tech to access locally-running applications for testing without deploying them to a public server. ## Prerequisites Before using the Remote Tunnels API, you need: 1. **cloudflared** – Cloudflare's tunnel client must be installed on your machine or CI runner ```bash theme={null} # macOS brew install cloudflared # Linux (Debian/Ubuntu) curl -L --output cloudflared.deb https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-linux-amd64.deb sudo dpkg -i cloudflared.deb # See https://developers.cloudflare.com/cloudflare-one/connections/connect-apps/install-and-setup/installation/ ``` 2. **Your application running locally** – The tunnel exposes local ports, so your app must be running ## Tunnel Expiration Tunnels expire **4 hours** after creation. The `expiresAt` field in the response indicates when the tunnel will be automatically torn down. For long-running development, you'll need to create a new tunnel when the current one expires. ## When to Use Remote Tunnels * **Local development testing** – Test your local environment without deploying * **CI/CD pipelines** – Expose ephemeral test servers during build processes * **Preview environments** – Create temporary public URLs for testing * **Firewall-protected environments** – Access applications behind corporate firewalls For CLI-based tunnel management, see the [tunnel command](/cli/commands/tunnel) documentation. The CLI provides a simpler interface for common tunnel operations. ## Authentication All endpoints require Bearer token authentication. Create your API key in the QA.tech dashboard: **Organization Settings → API Keys**. The key is shown only once, at creation. See the [API Introduction](/api-reference/introduction#authentication) for details. ## Endpoints For full request, response, and error details, see the generated reference pages: * [Create Remote Tunnel](/api-reference/remote-tunnels/create-remote-tunnel) – Expose one or more local ports; returns the tunnel's public hostnames and a `tunnelToken` * [List Remote Tunnels](/api-reference/remote-tunnels/list-remote-tunnels) – All tunnels for your project, including expired ones * [Get Remote Tunnel Status](/api-reference/remote-tunnels/get-remote-tunnel-status) – Live Cloudflare health status of a tunnel * [Delete Remote Tunnel](/api-reference/remote-tunnels/delete-remote-tunnel) – Tear down a tunnel and its DNS records ## Tunnel Lifecycle 1. **Create the tunnel** via [Create Remote Tunnel](/api-reference/remote-tunnels/create-remote-tunnel), specifying the local ports to expose. The response contains the public hostnames mapped to each port, a `tunnelToken`, and the `runnerId` used in subsequent calls. 2. **Start the cloudflared daemon** with the returned token: ```bash theme={null} cloudflared tunnel run --token YOUR_TUNNEL_TOKEN ``` The tunnel remains active as long as cloudflared is running and the tunnel has not expired (4-hour maximum lifetime). 3. **Check health** with [Get Remote Tunnel Status](/api-reference/remote-tunnels/get-remote-tunnel-status) before pointing tests at the tunnel. 4. **Use the public URLs** as environment overrides when [starting a run](/api-reference/runs/start-test-run) or [creating a chat conversation](/api-reference/chat). 5. **Clean up** with [Delete Remote Tunnel](/api-reference/remote-tunnels/delete-remote-tunnel) when you're done. For simpler tunnel management, consider using the [CLI tunnel command](/cli/commands/tunnel) which handles cloudflared automatically: `qatech tunnel start --port 3000` ## Hostname Format Each exposed port gets a public HTTPS hostname based on the tunnel's `runnerId`: * Single port without `subdomain`: `r-{runnerId}.quack.run` * Multiple ports without `subdomain`: `r-{runnerId}-p{localPort}.quack.run` * With `subdomain`: `r-{runnerId}-{subdomain}.quack.run` ## Related * [Tunnel CLI Command](/cli/commands/tunnel) – CLI interface for tunnel management * [SSH Tunnel](/configuration/ssh-tunnel) – Alternative tunneling via SSH * [Start Run API](/api-reference/runs/start-test-run) – Use tunnel URLs as environment overrides * [API Introduction](/api-reference/introduction) – Authentication and ID reference # Create remote tunnel Source: https://docs.qa.tech/api-reference/remote-tunnels/create-remote-tunnel /api-reference/api.json post /v1/remote-tunnels Create a new remote tunnel that exposes local ports via Cloudflare. # Delete remote tunnel Source: https://docs.qa.tech/api-reference/remote-tunnels/delete-remote-tunnel /api-reference/api.json delete /v1/remote-tunnels/{runnerId} Tear down a remote tunnel and its DNS records # Get remote tunnel status Source: https://docs.qa.tech/api-reference/remote-tunnels/get-remote-tunnel-status /api-reference/api.json get /v1/remote-tunnels/{runnerId}/status Get the live Cloudflare health status of a remote tunnel # List remote tunnels Source: https://docs.qa.tech/api-reference/remote-tunnels/list-remote-tunnels /api-reference/api.json get /v1/remote-tunnels List remote tunnels for the authenticated project # Create rule Source: https://docs.qa.tech/api-reference/rules/create-rule /api-reference/api.json post /v1/rules Create a rule: a standing instruction injected into a QA.tech agent's context. Agent rules support filters to target specific applications, labels, scenarios or URL paths. # Delete rule Source: https://docs.qa.tech/api-reference/rules/delete-rule /api-reference/api.json delete /v1/rules/{ruleId} Delete a rule so it is no longer injected into the agent's context # List rules Source: https://docs.qa.tech/api-reference/rules/list-rules /api-reference/api.json get /v1/rules List the project's rules: standing instructions injected into the QA.tech agents. Filter by consumer to see rules for the chat assistant, the test agent, or PR review. # Update rule Source: https://docs.qa.tech/api-reference/rules/update-rule /api-reference/api.json patch /v1/rules/{ruleId} Update a rule's title, content, filters or sort order. Pass null on a filter field to clear it. # Get run Source: https://docs.qa.tech/api-reference/runs/get-run /api-reference/api.json get /v1/run/{shortId} Fetches a run by its short ID. Pass `testCases: "all"` or `"failed"` to include nested test-case results, and `issues` to include detected issues (`all` or a category such as `accessibility`; omitted by default). Use `get_run_issues` to fetch only the issues. # Get run issues Source: https://docs.qa.tech/api-reference/runs/get-run-issues /api-reference/api.json get /v1/run/{shortId}/issues Lists issues detected during a run, deduplicated and aggregated across its test cases. Returns every issue type by default; pass `issueType` to filter to one category (e.g. `accessibility`). Each issue includes its type, severity, tags, a help URL, and the test cases where it was found. # Get run test case Source: https://docs.qa.tech/api-reference/runs/get-run-test-case /api-reference/api.json get /v1/run-test-case/{shortId} Fetches a single run test case (one test's execution within a run) by its short ID — the `` in a `/results/test/` URL. Returns the goal and success criteria, the evaluator's reasoning, the result and error classification, durations, and a deep link plus best-effort media URLs. Use `get_run_test_case_trace` for the step-by-step action trace and `get_test_case_history` for the pass/fail trend. # Get run test case trace Source: https://docs.qa.tech/api-reference/runs/get-run-test-case-trace /api-reference/api.json get /v1/run-test-case/{shortId}/trace Returns the agent's step-by-step action trace for a run test case: per step the reasoning, the tools it called with their arguments, the observed result, and a screenshot. Paginated via `limit`/`offset` so large traces don't overflow token limits. # Get test case history Source: https://docs.qa.tech/api-reference/runs/get-test-case-history /api-reference/api.json get /v1/run-test-case/{shortId}/history Returns the recent pass/fail history and durations for the test case behind a run test case short ID, across its most recent runs — for flaky-vs-consistent analysis. # List runs Source: https://docs.qa.tech/api-reference/runs/list-runs /api-reference/api.json get /v1/run Lists runs for the project, newest first. With no filters it returns the 20 most recent runs across all time; pass `since`/`until` (ISO 8601) to scope to a window such as the last 24 hours. Returned `shortId`s work directly with `get_run` and `rerun_run`. # Rerun run Source: https://docs.qa.tech/api-reference/runs/rerun-run /api-reference/api.json post /v1/run/{shortId} Reruns a previous run. Pass `failedOnly: true` to rerun only failed/skipped/errored cases, or `projectTestCaseIds` to rerun a specific subset. # Start test run Source: https://docs.qa.tech/api-reference/runs/start-test-run /api-reference/api.json post /v1/run Execute a run for a given project. Supports API or GITHUB trigger. When using GITHUB trigger, provide actor, branch, commitHash, commitMessage, and repository. Use applications to override environment or device preset per run. # Get example status badge Source: https://docs.qa.tech/api-reference/status-badge/get-example-status-badge /api-reference/api.json get /v1/badge-example.svg Get a sample status badge as SVG (no token). For live project badges use GET /v1/badge.svg. # Get status badge Source: https://docs.qa.tech/api-reference/status-badge/get-status-badge /api-reference/api.json get /v1/badge.svg Get status badge as SVG. Authenticate with the token query parameter (not the API Bearer key). Project is resolved from the token. # Create test case Source: https://docs.qa.tech/api-reference/test-cases/create-test-case /api-reference/api.json post /v1/test-cases Create a test case in the project. It is created in draft and a burn-in run starts automatically. IMPORTANT: gather context first, or the test will be flaky. This tool records the test exactly as you describe it. It does NOT explore the site to confirm the flow exists, so a vague or unverified goal produces low-value, flaky tests. Before calling, make sure you have: - Verified the functionality, pages, and UI labels actually exist in the target environment. Do not guess or assume. - A `goal` describing a single concrete user flow in plain UI terms, using generic data rather than specific entity names. Specific entities change over time and cause flakiness. - An `expectedResult` stating a UI-observable outcome a user can see on screen. - Prerequisites in place: a login/auth dependency via `resumeFromDependencyProjectTestCaseId` when the flow needs authentication, plus the right `configShortIds`. - Checked existing tests with `list_test_cases` to avoid duplicates. If you have not gathered this context, prefer `create_chat` instead. The QA.tech chat agent crawls the site, grounds the test in what actually exists, sets up login and config dependencies, and verifies the test runs. That is how reliable tests get created. Use this tool directly only when you already know the flow exists and have the details above. # Get test case Source: https://docs.qa.tech/api-reference/test-cases/get-test-case /api-reference/api.json get /v1/test-cases/{id} Fetches a test case's authored configuration by its ID (UUID, as returned by `list_test_cases`): goal, expected result, steps, start URL path, dependencies, labels, enabled state, and scenario. This is the authored config the agent executes — for a specific run's outcome use `get_run_test_case` instead. # List test cases Source: https://docs.qa.tech/api-reference/test-cases/list-test-cases /api-reference/api.json get /v1/test-cases Lists test cases for the project, optionally filtered by application, labels, or enabled state. # Update test case Source: https://docs.qa.tech/api-reference/test-cases/update-test-case /api-reference/api.json patch /v1/test-cases/{id} Updates a test case's authored configuration. Patch semantics: only the fields you provide change; pass `null` on nullable fields (`startUrlPath`, `scenarioShortId`, `resumeFromDependencyProjectTestCaseId`) to clear them. The edit takes effect immediately as a new published revision. To de-hardcode a test that 404s on a stale deep link, clear `startUrlPath` and set `resumeFromDependencyProjectTestCaseId` to a test that creates the entity it needs — it then starts from that test's end state with fresh data. # BankID Source: https://docs.qa.tech/applications/se-bank-id ## What is BankID BankID is a digital identification system widely used in countries like Sweden, Norway, and Finland. It allows individuals to securely prove their identity online and authorize electronic transactions. ## How to test BankID There are two approaches for handling BankID in your QA.tech tests: **stubbing** BankID in your testing environment, or **manual authentication** using the live screen stream. *** ## Option 1: Stubbing BankID (Recommended) Stubbing gives you fully automated tests without manual intervention. This requires changes to your application code by your development team. From BankID's official documentation: > **How do I test my BankID implementation? How about automation?** > > Testing can't be automated. Passwords/Security codes have to be manually entered in the BankID clients. We recommend building a so-called test stub that simulates the BankID service web service. It can also be used to perform load tests on your services. > [https://developers.bankid.com/support](https://developers.bankid.com/support) ### What is stubbing? Stubbing replaces the BankID integration with a placeholder that automatically grants access in your testing environment. For example, you accept a specific personnummer like `199001011234` and always grant that user access. This should only be permitted in a *testing* environment, never in production. On the client: ```javascript theme={null} if (environment === 'testing') { // Show stub UI for BankID } else { // Show real BankID } ``` On the server: ```javascript theme={null} if (environment === 'testing') { if (personalNumber === '199001011234') // Send an auth success response else // Send an auth failed response } else { // Auth using real BankID } ``` *** ## Option 2: Manual Authentication If you can't stub BankID, you can manually complete the authentication yourself. Since QA.tech live streams the browser screen during test execution, you can watch the test and scan the BankID QR code in real time. Create a dedicated login test that waits for you to authenticate, and then reuse that login state across all your other tests via [dependencies](/core-concepts/dependencies). You only need to authenticate **once per test run**. Create a test with instructions like: ``` Go to the login page and click "Log in with BankID". A QR code will appear. Wait up to 10 minutes for the login to complete. After login, verify that you are on the logged-in dashboard. ``` The long wait gives you enough time to find the running test, open the live stream, and scan the QR code with your phone. Start the test run. The login test will navigate to the BankID login page and wait. Find the running login test in the QA.tech dashboard and open its live screen stream. You'll see the BankID QR code on screen. Open the BankID app on your phone and scan the QR code shown in the stream. Confirm the login in your BankID app. The test will detect that the login succeeded and continue. ### Reusing Login State Across Tests Set up your other tests to reuse the authenticated session using **Resume From** dependencies. This way the BankID login only happens once. ```mermaid theme={null} graph TB subgraph "Browser Session" T1["Login with BankID
(manual QR scan)"] T2["Test: Check account settings
(Resume From Login)"] T3["Test: Submit a form
(Resume From Login)"] T1 -->|Resume From| T2 T1 -->|Resume From| T3 end ``` See [Test Dependencies](/core-concepts/dependencies) for details on how Resume From works. The login test's browser state is cached for up to 6 hours. If your BankID session lasts that long, you won't need to re-authenticate between consecutive test runs. # Creating Tests Source: https://docs.qa.tech/best-practices/creating-tests Learn how to create effective test cases in QA.tech and write tests that the AI agent can execute reliably. ## When to Create Tests QA.tech works best when tests target **features that already exist** in your application. Since the AI agent crawls your site and builds a knowledge graph to design test cases, it relies on the feature being built and accessible. ### Testing Features That Don't Exist Yet If you ask the chat to create a test for something that isn't built yet (e.g., "Go to the Admin page and add a user" when there's no Admin page), the agent will either: * Spend time searching for a feature it can't find * Report that it doesn't know how to complete the task This isn't a productive use of your time or the agent's capabilities. ### Recommended Workflow for New Features Instead of creating tests before the feature exists, **document what should be tested in your ticket or PR description**: When creating a Linear, Jira, or other ticket for a new feature, include acceptance criteria that describe what should be tested. For example: - "User can navigate to Admin page and add a new user" - "New user receives invitation email" - "Admin can see the new user in the user list" When you open the PR, include testing requirements in the description. The [GitHub App](/configuration/github-app) agent reads your PR and uses this context to create appropriate tests. When the PR is published and your preview environment is ready, our agent analyzes the PR changes and acceptance criteria, then creates and runs tests for the new functionality. See [GitHub App for PR Reviews](/configuration/github-app) for details. This workflow means you spend time describing *what* to test (which you'd do anyway for acceptance criteria), and the agent handles *how* to test it once the feature is ready. ## What Should Be a Test? A good example of a test is something a user would like to do, or achieve. A user story or user journey should probably map 1:1 to a test case. #### Test Examples * Log in with correct credentials * Log in with incorrect credentials * Create a new task * Edit a task's due date ## Creating a Test You can create test cases through the UI with AI-generated suggestions, or conversationally through the AI Chat Assistant. ### Create Tests via UI Click the **"Add Test Case"** button to open the test creation modal with two options: Add Test Case Button **Suggested tests** - QA.tech continuously crawls your application to discover testable features and interactions. These appear as AI-generated suggestions you can select and add to your project. Click "Analyze my site" to trigger a new crawl if you want fresh suggestions. Suggested tests modal **Create your own test** - Describe what you want to test in natural language. The AI agent will understand your goal and attempt to generate the test steps automatically. Create your own test modal Key fields: * **Name**: Clear, descriptive name (e.g., "Create admin user and verify access") * **Goal**: What the test should accomplish in natural language * **Expected Result** (optional): What success looks like * **Dependencies**: Configure test execution order (see [Test Dependencies](/core-concepts/dependencies) for WAIT\_FOR and RESUME\_FROM) * **Configurations**: Add required data like credentials or test accounts * **Advanced**: Agent selection and other settings ### Choosing an AI Agent When creating a test, you can select which AI agent executes it. Click the **Advanced** section in the test creation modal or test settings to see agent options. | Agent | Speed | Best For | | :----------------------------- | :------- | :---------------------------------------------- | | **Claude Haiku 4.5** (default) | Fastest | Most tests - recommended for day-to-day testing | | Claude Sonnet 4.5 | Moderate | Complex scenarios requiring deeper reasoning | **Claude Haiku 4.5** is the default for all new tests. It provides the fastest execution while handling most testing scenarios effectively - form filling, navigation, verification, and standard user flows. **When to consider Sonnet:** If a test consistently fails with Haiku on complex multi-step reasoning or edge cases, try switching to Sonnet for that specific test. Most users never need to change the default agent. QA.tech selects Haiku 4.5 because it offers the best balance of speed and capability for typical testing workflows. ### Create Tests via AI Chat Assistant The [AI Chat Assistant](/core-concepts/ai-chat-assistant) provides a conversational, exploratory approach to test creation: **Natural conversation** - Describe your testing needs in plain English. Ask for multiple tests, request specific coverage areas, or iterate on suggestions through back-and-forth conversation. **Upload context** - Drag and drop specification documents, design files, or requirements (PDFs, images, text files) directly into the chat. The AI uses this context to generate more accurate, relevant tests. **Safe experimentation** - Tests generated in chat aren't committed to your project until you explicitly click "Add Selected Tests". You can review, refine, or discard them without affecting your team. Uploaded files remain isolated to that chat conversation only. Example prompts: * "Generate 5 tests for the checkout flow" * "What areas of my product should I cover with test cases first?" * "Create a test that validates login and checks the user profile page" See [AI Chat Assistant](/core-concepts/ai-chat-assistant) for more examples. ### Review and Refine After creating a test, the AI agent automatically attempts to execute it and generate test steps. Click the **review button** to inspect the results: Review generated test Edit test page **Left sidebar** - View and edit the goal, expected result, and generated steps. The Settings tab lets you configure dependencies, add required configurations, or adjust agent settings. **Right panel** - Inspect the execution trace showing exactly what the agent did. This helps you verify the test behaves as intended. Update steps as needed and click **"Save & Run"** to test your changes. You can stop execution at any time with the **"Stop"** button. ### Refine Tests via Chat The fastest way to fix and improve tests: describe changes in the [AI Chat Assistant](/core-concepts/ai-chat-assistant), review the diff, and run immediately. **Fix after failures** - When a test fails, stay in chat and describe the fix. No context switching - see the failure, fix it, validate. **Build iteratively** - Create a rough test, watch it run, refine through conversation. The AI remembers what you both just saw. **Bulk refinement** - Describe what you want across your test suite and let the AI handle the details. It reads your tests, identifies which ones to change, and proposes edits for each. #### Example Prompts | You say... | What happens | | ---------------------------------------------- | --------------------------------------- | | "Change step 3 to wait for the spinner first" | AI proposes single step edit | | "Also verify the success message appears" | AI adds verification to current test | | "Make this resume from my Login test" | AI updates dependency | | "Add email verification to all checkout tests" | AI finds and edits multiple tests | | "Create tests for returns similar to checkout" | AI reads your tests, generates new ones | The AI already knows your test suite and remembers your conversation. You can say "that test" or "where it failed" - no need to be overly specific. Describe what you want in plain language and let the AI figure out the details. **Only for existing tests:** Editing works on tests that have been created. For suggestions you haven't added yet, keep describing changes and the AI will regenerate the suggestion. ## Activating and Organizing Tests After creating and reviewing your test, you'll want to activate it and organize it within your project. ### Activate Your Test Tests are created in draft mode so you can review and refine them before they run. When you're ready, click the **"Activate"** button to enable the test and make it part of your active test suite. If your test has dependencies that are also in draft mode, QA.tech will prompt you to activate them together to ensure proper execution order. Once activated, the button changes to **"Convert to draft"** - use it if you need to temporarily disable the test for maintenance or updates. ### Organize with Scenarios On the Test Cases page, you can drag and drop tests to organize them into Scenario groups. Scenarios help you: * Keep related tests together (e.g., "Checkout Flow", "User Management") * Create logical test groupings for better organization * Get a clear overview during test execution * Set up test dependencies within related workflows ## Writing Effective Tests ### Writing a Good Goal The goal is the main objective of the test. The agent uses this to build steps and adapt when your application changes. Focus on describing **what to do**, not what to validate (use expected result for that). **Good goal examples:** * Search for 'Chair', navigate to a product and add it to the cart * Invite a new member with Admin role to the project * Open the customer support chat and send a message **Keep your goals:** * **Action-oriented** - Start with verbs like "Create", "Search", "Navigate", "Add" * **Specific** - Include exact details (product name, user role, button labels) * **Focused** - Describe actions to take, not validation criteria ### Writing a Good Expected Result The expected result defines what the agent should verify at the end of the test. Describe what should be visible or observable when the test completes successfully. **Good expected result examples:** * The page should contain a user avatar * A success message appears and the user is redirected to the product list * The user receives an email with a password reset link **Keep your expected results:** * **Observable** - Focus on things that can be verified visually or through system responses * **Specific** - Include exact elements, messages, or states to check * **Outcome-focused** - Describe the end state, not how to get there Keep tests to 10 steps or less. If you need more steps, create a new test with a [dependency](/core-concepts/dependencies) instead. Shorter tests are faster to execute and easier to maintain. **Performance tip:** [Agent Cache](/core-concepts/agent-cache) is enabled by default, speeding up test execution by reusing AI reasoning from previous successful runs. Consider disabling cache when debugging flaky tests or testing new features where you want fresh AI analysis. ### Testing Email Flows When writing tests that involve emails (like password resets or notifications), follow these best practices to ensure the agent can find the correct email: * **Specify the Recipient**: Always mention the target email address in your test goal or step (e.g., "Wait for the welcome email sent to `user@example.com`"). * **Use Descriptive Queries**: Describe the email content clearly (e.g., "Find the email with subject 'Reset Password'"). * **Mind the Time Window**: Remember that tests can only see emails received *after* the test started. * If Test A triggers an email and finishes, Test B cannot see that email if it starts afterwards. * **Solution**: Combine the trigger and verification into one test, or use dependencies where the second test triggers the email resend. For more details on how the inbox works, see [Email Inbox](/test-features/email-inbox). ### Test Dependencies Tests can depend on other tests to control execution order and reuse browser state. This is essential for complex workflows where one test needs data or state from another. Learn more in [Test Dependencies](/core-concepts/dependencies). To reuse the same sequence of steps across many tests, see [Shared Steps](/core-concepts/shared-steps). **Testing with Multiple Users:** If your test requires multiple users logged in simultaneously (e.g., collaboration, sharing), create separate login tests for each user. Each login test becomes the root of an independent chain with its own isolated browser session, ensuring users don't interfere with each other. Learn more about [Multi-User Testing Scenarios](/core-concepts/dependencies#multi-user-testing-scenarios). ## Organizing Tests for Complex Projects For projects with multiple products, versions, or environments, see [Projects, Applications, Environments](/core-concepts/applications-and-environments) for organization patterns and best practices. # Authentication Source: https://docs.qa.tech/best-practices/handle-auth To enable authentication in your tests, you'll need to create **Configs** in your project settings. Configs store authentication credentials (like usernames, passwords, and 2FA secrets) that the AI agent uses automatically during test execution. Our AI needs to access your website and act as a registered user to interact with protected features and test user-specific functionality. If your preferred authentication method isn't listed below, let us know, and we'll consider adding it to our roadmap. For detailed information about all config types and their settings, see the [Configs documentation](/core-concepts/configs). **Security Notice: Use Test Credentials Only** Authentication credentials stored in [Configs](/core-concepts/configs) are **not encrypted** and are passed to AI language models during test execution. Always use dedicated test accounts - never real user credentials or production passwords. ## Authentication Support for QA.tech AI Bots | Authentication Method | Description | Setup Required | | ------------------------------- | ---------------------------------------------------------------------------- | ------------------------------------- | | Username + Password | Standard form-based login with username/email and password | Basic Config | | Email + Password | Login using email address and password | Basic Config | | OTP via Email | One-time passwords sent to email addresses | Email Config + Inbox | | Magic Link Login | Passwordless login via email links | Email Config + Inbox (auto-generated) | | Two-Factor Authentication (2FA) | Time-based codes from authenticator apps (Google Authenticator, Authy, etc.) | Advanced Config (see below) | ## What We Don't Support * **BankID** - Requires stubbing in your testing environment or manual authentication via live stream ([see guide](/applications/se-bank-id)) * **CAPTCHA challenges** - Most CAPTCHA types (reCAPTCHA, hCAPTCHA, etc.) are not supported. **Avoid CAPTCHA on staging/dev:** Whitelist QA.tech's IP addresses to prevent CAPTCHA challenges during test execution. This allows our AI agents to run tests without interruption. Find your IP addresses in [**Settings → Network**](https://app.qa.tech/dashboard/current-project/settings/network) and see the [IP Access Control guide](/configuration/ip-access-control) for setup instructions. *** ## Setting Up Authentication ### Basic Credentials (Username/Password or Email/Password) For standard login forms, create a config in your project settings: 1. Go to **Project Settings** → **Configs** 2. Click **Add Config** 3. Select **Username + Password Credentials** or **Valid Email + Password Login Credentials** 4. Fill in your test account credentials 5. Click **Save** The AI will automatically use these credentials when it encounters a login form during test execution. **Build a comprehensive knowledge graph:** If your application has different user types (admin, regular user, premium user, etc.), create separate authentication configs for each. Login tests can automatically trigger [crawling sessions](/core-concepts/crawling) after completion, allowing QA.tech to map out what each user type can access. This helps the AI generate more accurate tests tailored to different user permissions. ### Email-Based OTP For authentication flows that send one-time passwords to email: 1. QA.tech provides dedicated email inboxes for each project 2. The AI can automatically read OTP codes from emails sent to these addresses 3. Configure email settings in **Project Settings** → **Configs** 4. When writing tests, the AI will wait for and extract OTP codes automatically QA.tech's built-in email inbox system allows the AI to receive and process emails during test execution - no need to set up external email services or worry about email delivery. The AI can wait for emails (up to 3 minutes), extract verification codes, and click links automatically. Learn more about [Email Inbox](/test-features/email-inbox). ### Magic Link Login For passwordless authentication via email links: 1. QA.tech automatically provides a magic link email address for each project (no setup required) 2. Enter this email address in your application's login form 3. The AI automatically waits for the email, extracts the login link, and navigates to it Create a test user account in your application using the magic link email address shown in your project's [**Email for Magic Link Login** config](/core-concepts/configs) before running tests. ### Two-Factor Authentication (2FA) QA.tech allows testing login flows that are protected by Two-Factor Authentication (2FA). #### How 2FA Testing Works Most modern 2FA systems use **Time-based One-Time Passwords (TOTP)**. This is the constantly changing, 6-digit code you see in apps like **Google Authenticator**, **Authy**, or **Microsoft Authenticator**. To test this, our AI needs the secret key that your application uses to generate these codes. You provide this secret key to us once, and our AI handles the rest. * **Your Role (One-Time Setup):** You will create a special `Config` in your QA.tech project settings. Instead of giving us a static 6-digit code, you will provide a special secret key in a format called a **URI**. * **Our AI's Role (During Every Test):** When the AI agent encounters a 2FA screen during a test, it will use the secret URI you provided to generate a **fresh, valid 6-digit code** at that exact moment. It then automatically enters the code to complete the login. This way, you never have to worry about codes expiring or manually entering them during a test run. #### Setting Up a 2FA Config Follow these steps to create a config for a test user account that has 2FA enabled. Log in to the application you want to test and go to the security settings for your test user. Find the option to enable Two-Factor Authentication and proceed until the application shows you a **QR code**. Stop here. Do not scan it with your phone. We need to extract the secret key from this image. The QR code contains a secret key that our AI needs. You can use a free online tool to extract this key. 1. Take a screenshot of the QR code. 2. Go to a site like [**scanqr.org**](https://scanqr.org/) in your browser. 3. Upload the screenshot of the QR code. 4. The tool will decode it and reveal a string of text called a URI. Copy this entire string. It will look something like this: ```text theme={null} otpauth://totp/YourApp:test.user@example.com?secret=JBSWY3DPEHPK3PXP&issuer=YourApp ``` Now, let's add the credentials and the secret URI to QA.tech. 1. Go to **Project Settings** → **Configs** 2. Click the **Add Config** button. 3. From the list of types, select **Username + Password Credentials - with Two-Factor Authentication**. 4. Fill in the form: * **Config Name:** Give it a memorable name (e.g., "Admin User with 2FA"). * **Username:** The username for your test account. * **Password:** The password for your test account. * **One-Time Password URI:** Paste the full `otpauth://` URI you copied from the QR code scanner. 5. Click **Save**. #### Running Tests with 2FA Enabled You're all set! You do not need to add any special instructions to your tests like "enter the 2FA code." When you ask the AI to perform a test that involves logging in (e.g., "Log in and check the user dashboard"), the agent will automatically: 1. Use the username and password from your new config. 2. When it sees the 2FA input screen, it will automatically generate a valid code using the URI you provided. 3. It will enter the code and complete the login before proceeding with the rest of your test instructions. # Running Tests Source: https://docs.qa.tech/best-practices/running-tests Tests in QA.tech can be run in several ways. ## Trigger a Run from the UI The simplest way to get started is to use the "Run Tests" button found on the [Dashboard](https://app.qa.tech/current-project) or the [Test Cases](https://app.qa.tech/current-project/scenarios) page. You can also run individual tests by navigating to the test and clicking **Run Test**. ### Run with Dependencies When running an individual test that has dependencies, you have two options: * **Run Test** - Uses [dependency output state optimization](/core-concepts/output-state-optimization) to skip recently completed dependencies (completed within 6 hours) * **Run w. Dependencies** - Forces all dependency tests to run fresh, generating new output data Use **Run w. Dependencies** when you need fresh time-sensitive data like OTP codes or temporary tokens that may have expired. [Learn more](/core-concepts/output-state-optimization#my-test-is-using-stale-output-data-old-otp-codes-expired-tokens). When running multiple tests via a **Test Plan**, QA.tech automatically executes them in parallel where the dependency graph allows, with auto-scaling up to approximately 100 concurrent agents when no limit is configured. You can set **Maximum Concurrent Tests** on each environment to throttle parallel runs against sensitive URLs. Learn more in [Parallel Test Execution and Concurrency Limits](/core-concepts/parallel-test-execution) and [Test Plans](/core-concepts/test-plans#performance--parallel-execution). ## Trigger a Run as Part of a Pipeline You can also integrate test runs as part of your **CI pipeline**. You can use any system that supports sending an HTTPS POST request when you want to trigger the tests. For a great example, check out our [GitHub Actions documentation](/configuration/github-actions). ## Trigger a Run on a Schedule If you want your tests to run at a specific time, such as on a recurring weekly schedule (cron-style), we offer that functionality as well. Go to [Test Plans](https://app.qa.tech/current-project/test-plans) in your project dashboard. Click on an existing test plan that you want to schedule, or create a new one if needed. Click on **Manage Schedules** within the test plan. Add your cron schedule expression and a description, then click **Add Schedule**. If you're unsure how to write the cron format, you can use a tool like [crontab.guru](https://crontab.guru/) or reach out to us, and we'll be happy to help you get set up. # How to review test results Source: https://docs.qa.tech/best-practices/test-results Different stages of your development workflow need different testing approaches. This guide helps you choose the right testing strategy for your needs and efficiently manage results across your workflow. **Foundation for Testing** Before using any testing approach below, ensure you've [crawled your website](/core-concepts/crawling) to build the [knowledge graph](/core-concepts/knowledge-graph) and configured [authentication](/best-practices/handle-auth). These prerequisites enable the AI agent to test efficiently. ## Choosing Your Testing Strategy Different testing approaches fit different scenarios. Choose based on when you need results and what you're validating: | Testing Approach | Best For | When to Use | Setup Guide | | --------------------- | ------------------------------------------ | ------------------------------------------ | --------------------------------------------------- | | Manual test runs | Ad-hoc testing, debugging, development | Quick validation, iterating on test design | [Running Tests](/best-practices/running-tests) | | Scheduled test plans | Production monitoring, regression suites | Daily/nightly checks, SLA monitoring | [Test Plans](/core-concepts/test-plans) | | PR-triggered testing | Pre-merge validation, preview environments | Every PR, feature branches | [GitHub Actions](/configuration/github-actions) | | API-triggered testing | Custom workflows, deployment pipelines | Post-deploy validation, multi-environment | [API Reference](/api-reference/runs/start-test-run) | **Recommended progression:** * **Start with:** Manual runs while building tests, then add PR testing * **Add next:** Scheduled nightly runs for comprehensive coverage * **Advanced:** API integration for custom deployment workflows * **Pro tip:** Use test plans to organize tests for different scenarios (smoke vs. full regression) ## Organizing Tests with Test Plans Test plans help you run the right tests at the right time. Organize tests strategically to separate fast smoke tests from comprehensive regression suites. **Common test plan patterns:** * **Smoke tests plan** - 5-10 critical tests, runs on every PR (\~5 min) * **Regression plan** - Full suite, runs nightly (\~30-60 min) * **Production monitoring plan** - Key flows, runs every 4 hours * **Preview environment plan** - Dynamic URLs, runs on deploy Test plans enable parallel execution - 20 tests complete in \~3 minutes instead of 60 minutes sequential. See [Test Plans](/core-concepts/test-plans#performance--parallel-execution). Use test plans to: * Separate fast tests (PR-triggered) from slow tests (scheduled) * Target different environments (staging vs. production) * Control what runs when based on your workflow ### Quick Triage Workflow Use RESULT=FAILED + STATUS=COMPLETED to see only actionable failures. See [Tests and Results](/core-concepts/tests-and-results#filtering-and-searching-results) for filter options. Grouped failures often share root cause - review scenario by scenario Determine if it's a product bug, test needs update, flaky test, or environment issue Create issues for bugs, update tests for intentional changes, add waits for flaky tests ## Getting Notified Configure notifications based on your testing approach: * **PR tests** - GitHub comments post automatically (see [GitHub Actions](/configuration/github-actions)) * **Scheduled runs** - Slack/email for failures (see [Notifications](/core-concepts/notifications)) * **Production monitoring** - Immediate alerts for critical failures **Tip:** Set up different notification channels for different severity levels (Slack for critical, email for nightly) ## Exporting Issues When you find bugs during review: * Export to Linear, Jira, or Trello: [Configure integrations](/integrations/linear) * Include playback URL and context before exporting * Tag with environment and test run for traceability # Troubleshooting Source: https://docs.qa.tech/best-practices/troubleshooting What can you do to make QA.tech work as well as possible for you? ## The agent doesn't understand what to click We use a combination of the DOM and visual models to decide what to click or interact with. To make it easier for our agent to understand your site, we suggest taking the following measures. 1. Make your site more accessible Ensure all your elements are tagged with the correct `aria`-labels, as well as `title` and `alt` attributes. 2. Use semantic HTML tags For example, utilize `` or `