Skip to main content

​
When to Create Tests

QA.tech works best when tests target features that already exist in an accessible test environment. Web and mobile agents rely on the interface being available. API tests rely on a reachable HTTPS API with the behavior you want to validate. Before a Proof of Concept, use the POC Setup Checklist (or the Mobile POC Setup Checklist) to confirm network access, test accounts, and email delivery.
This page is the general way to create a test. Web-only setup is on Web. API setup is in Creating API Tests. Mobile setup is in Mobile App Testing.

​
Testing Features That Don’t Exist Yet

If you ask the chat to create a test for something that isn’t built yet (e.g., “Go to the Admin page and add a user” when there’s no Admin page), the agent will either:
  • Spend time searching for a feature it can’t find
  • Report that it doesn’t know how to complete the task
This isn’t a productive use of your time or the agent’s capabilities.

​
Keeping the suite current over time

Do not create tests for features that are still on the roadmap. Put acceptance criteria in the ticket or PR description, and let PR Testing create and maintain coverage when the feature ships.

​
What Should Be a Test?

A good example of a test is something a user would like to do, or achieve. A user story or user journey should probably map 1:1 to a test case.

​
Test Examples

  • Log in with correct credentials
  • Log in with incorrect credentials
  • Create a new task
  • Edit a task’s due date

​
Creating a Test

Create tests conversationally through the AI Chat Assistant. Describe what you want covered, review the draft tests, and add the ones you keep.

​
Create Tests via AI Chat Assistant

Natural conversation - Describe your testing needs in plain English. Ask for multiple tests, request specific coverage areas, or iterate through back-and-forth conversation. Reference tickets and pull requests - Point the assistant at a Jira or Linear ticket, or a GitHub pull request, so it can use the acceptance criteria and change description when drafting tests. Connect Jira or Linear when you want those tickets available in chat and during PR reviews. Upload context - Drag and drop specification documents, design files, or requirements (PDFs, images, text files) directly into the chat. The AI uses this context to generate more accurate, relevant tests. Safe experimentation - Tests generated in chat aren’t committed to your project until you explicitly click “Add Selected Tests”. You can review, refine, or discard them without affecting your team. Uploaded files remain isolated to that chat conversation only.
Example prompts:
  • “Generate 5 tests for the checkout flow”
  • “Create tests from PROJ-1234”
  • “Create tests for the changes in this PR”
  • “Create a test that validates login and checks the user profile page”
See AI Chat Assistant for more examples.

​
Review and Refine

After creating a test, the AI agent automatically attempts to execute it and generate test steps. Click the review button to inspect the results: Review generated test Edit test page Left sidebar - View and edit the goal, expected result, and generated steps. The Settings tab lets you configure dependencies, add required configurations, or adjust agent settings. Right panel - Inspect the execution trace showing exactly what the agent did. This helps you verify the test behaves as intended. Update steps as needed and click “Save & Run” to test your changes. You can stop execution at any time with the “Stop” button.

​
Refine Tests via Chat

The fastest way to fix and improve tests: describe changes in the AI Chat Assistant, review the diff, and run immediately. Fix after failures - When a test fails, stay in chat and describe the fix. No context switching - see the failure, fix it, validate. Build iteratively - Create a rough test, watch it run, refine through conversation. The AI remembers what you both just saw. Bulk refinement - Describe what you want across your test suite and let the AI handle the details. It reads your tests, identifies which ones to change, and proposes edits for each.

​
Example Prompts

The AI already knows your test suite and remembers your conversation. You can say “that test” or “where it failed” - no need to be overly specific. Describe what you want in plain language and let the AI figure out the details.
Only for existing tests: Editing works on tests that have already been added to the project. For drafts still in the chat review panel, keep describing changes and the AI will regenerate them before you add them.

​
Create Tests via UI

You can also click “Add Test Case” on the Test Cases page and describe the goal in the modal. Fill in name, goal, optional expected result, configs, dependencies, and agent settings, then let the agent generate steps on the first run. Prefer chat when you want to iterate or pull context from a ticket or PR.

​
Activating and Organizing Tests

After creating and reviewing your test, you’ll want to activate it and organize it within your project.

​
Activate Your Test

Tests are created in draft mode so you can review and refine them before they run. When you’re ready, click the “Activate” button to enable the test and make it part of your active test suite.
If your test has dependencies that are also in draft mode, QA.tech will prompt you to activate them together to ensure proper execution order.
Once activated, the button changes to “Convert to draft” - use it if you need to temporarily disable the test for maintenance or updates. See Test drafts and revisions for what Active, Draft, and Archived mean and when tests are created as drafts.

​
Organize with Scenarios

On the Test Cases page, you can drag and drop tests to organize them into Scenario groups. Scenarios help you:
  • Keep related tests together (e.g., “Checkout Flow”, “User Management”)
  • Create logical test groupings for better organization
  • Get a clear overview during test execution
  • Set up test dependencies within related workflows

​
Writing Effective Tests

​
Writing a Good Goal

The goal is the main objective of the test. The agent uses this to build steps and adapt when your application changes. Focus on describing what to do, not what to validate (use expected result for that). Good goal examples:
  • Search for ‘Chair’, navigate to a product and add it to the cart
  • Invite a new member with Admin role to the project
  • Open the customer support chat and send a message
Keep your goals:
  • Action-oriented - Start with verbs like “Create”, “Search”, “Navigate”, “Add”
  • Specific - Include exact details (product name, user role, button labels)
  • Focused - Describe actions to take, not validation criteria

​
Writing a Good Expected Result

The expected result defines what the agent should verify at the end of the test. Describe what should be visible or observable when the test completes successfully. Good expected result examples:
  • The page should contain a user avatar
  • A success message appears and the user is redirected to the product list
  • The user receives an email with a password reset link
Keep your expected results:
  • Observable - Focus on things that can be verified visually or through system responses
  • Specific - Include exact elements, messages, or states to check
  • Outcome-focused - Describe the end state, not how to get there
Keep tests to 10 steps or less. If you need more steps, create a new test with a dependency instead. Shorter tests are faster to execute and easier to maintain.

​
Test Dependencies

Tests can depend on other tests to control execution order and reuse browser state. This is essential for complex workflows where one test needs data or state from another. Learn more in Test Dependencies. To reuse the same sequence of steps across many tests, see Shared Steps.

​
Organizing Tests for Complex Projects

For projects with multiple products, versions, or environments, see Projects, Applications, Environments for organization patterns and best practices.