Using AI Agents for Testing: A Practical Starting Point for Developers

You are a developer. You asked Claude Code or Cursor or Codex to add a feature. The agent ran your existing tests, they passed, and now you are looking at the diff wondering whether to trust it. This piece is the operational answer to that question, written for someone who has never taken testing seriously as a discipline.

There is no ceremony below. No test pyramid diagrams. No week-one-hire onboarding. Just the shortest workflow that catches most of what agents get confidently wrong, plus the two habits that separate a developer using AI well from one shipping AI-generated bugs into production.

The mental model in one sentence

Every unit of AI-augmented work has three pieces: a contract, an action that checks the contract, and an artifact that proves the check ran. You have to write all three or the agent’s output is not trustworthy. If you already write tests, you already do this; it is just that the words are different.

• Contract: a precise statement of what the code should do, in plain-English bullets or better, in executable form (types, function signatures, example inputs/outputs).
• Action: the test that verifies the contract holds. Unit tests, integration tests, contract tests, type checks, whatever fails fast when reality drifts from the spec.
• Artifact: the evidence a human can look at in under 30 seconds. Screenshots, coverage reports, benchmark numbers, error traces, diff visualizations.

The pillar piece Testing in the Age of AI walks through why this trio is the whole game. For this piece, the operational question is: how do you actually build it?

The workflow, five steps

Here is the loop for a single change. Once you have run it three or four times it takes 90 seconds of overhead per feature and catches most category-error mistakes an agent makes.

Step 1: write the contract before the code

Before you type a single instruction to the agent, write out what the change should do. Not the implementation. The behavior. If you cannot state it in three sentences or a bulleted list, the agent will not know either. When it produces something confidently wrong, this is almost always why.

Concrete example. Say you are adding email validation to a signup form.

Bad contract (typical of what a rushed dev sends to Cursor):

Add email validation to the signup form.

Good contract:

Add email validation to the signup form’s email input. Rules:

• Must contain exactly one @ character.
• Must have at least one character before @ and at least one after.
• After @ must contain at least one . with a character on both sides.
• Trim leading and trailing whitespace before validation.
• Empty input shows “Email is required.” Invalid non-empty input shows “Enter a valid email address.” Valid input shows nothing.
• Validation runs on blur, not on every keystroke.

That is 15 seconds to write and it eliminates roughly two rounds of back-and-forth with the agent. It also becomes your test spec in step 3, and is exactly the kind of thing you want committed to the repo alongside the code.

Step 2: delegate implementation with the contract embedded

Paste the contract to the agent, ask for the implementation, and ask it to only implement, not test. Testing next.

Using the contract in the previous message, add email validation to
apps/web/src/components/SignupForm.tsx. Do NOT write tests yet. I'll
ask for those separately in the next message.

The reason to separate implementation and tests is that if you ask for both at once, agents often write the tests to match whatever implementation they produced, which is exactly backward. Tests should verify the contract, not the code the agent happened to write.

Step 3: have the agent generate tests against the contract

Once implementation looks right, ask for tests. Reference the contract again, not the code:

Now write tests for the email validation using Playwright (for the on-blur
UI behavior) and Vitest (for the pure validation function if you extracted
one). The tests must verify every rule in the contract above. Include one
test per rule and one edge case per rule.

The framework naming here is important. Playwright is the current default for browser-based UI testing (see the Playwright vs Cypress vs Selenium comparison for why). Vitest is the current default for JavaScript/TypeScript unit tests. Naming them explicitly gets you tests in the shape your codebase already uses.

Step 4: review the tests, not the implementation

Here is the counterintuitive part. Do not carefully read the implementation diff. Read the tests. If the tests correctly assert every rule in the contract, and they pass, then the implementation is correct enough for now.

If the tests do not correctly assert the rules, the implementation is one of two things:

• Wrong, but the tests are wrong in a compatible way. Both must be fixed.
• Correct, but the tests are not actually verifying what the contract said.

Either way, tests are where you catch the failure mode. Not the diff. This is the hardest habit to develop coming from a “read every line” review style, and it is the one that separates developers who scale with AI from developers who spend all day arguing with agents about implementation choices.

Step 5: verify the artifact

Run the tests. Look at the output. If you are using Playwright, run with --trace on so you get a trace file. Open it. Look at the screenshots. Confirm that the thing being validated is the thing the contract described. If you are using Vitest, generate a coverage report and check that the code the contract touches is actually covered.

30 seconds of eye time. It catches the case where the tests are “green” but the assertions accidentally check something trivial.

Common failure modes

Three failures show up over and over in AI-generated tests.

The test that passes but doesn’t check the thing

Signature move: expect(result).toBeTruthy() or expect(output).toBeDefined(). These are tests that will pass as long as the code returns anything at all. They do not verify the behavior.

Fix: your contract should have specific expected outputs for specific inputs. Tests should assert those specific outputs. Never accept toBeTruthy() or toBeDefined() in a test that is supposed to verify business logic.

The test that duplicates the implementation

Sometimes an agent writes tests that reimplement the code under test, then asserts the two implementations agree. The tests pass, but they are testing that the code equals itself.

Fix: tests should use example inputs and expected outputs from the contract, not from the code. If you find yourself reading a test and mentally translating it back into the implementation, the test is wrong.

The test that ignores boundary conditions

Every rule in your contract has a boundary. “Email must have at least one character before @” has a boundary at zero characters and one character. “Trim whitespace” has a boundary between “space only” and “space plus content.”

Fix: for every rule in the contract, ask “what is the smallest input that satisfies this rule?” and “what is the largest input that violates it?” Both should be test cases. Agents will write the middle cases; they miss the boundaries roughly half the time.

The Monday morning starter kit

Concrete list, in order. If you have never written serious tests before and want to be running the workflow above by the end of the week:

Day 1: install the frameworks.

# In your project root
pnpm add -D vitest @vitest/coverage-v8
pnpm add -D @playwright/test
pnpm exec playwright install chromium

Then add to package.json:

{
  "scripts": {
    "test": "vitest run",
    "test:e2e": "playwright test",
    "coverage": "vitest run --coverage"
  }
}

That is the whole setup. Two testing frameworks that will cover 95% of what you need. If you are on Python, substitute pytest and Playwright’s Python bindings; if you are on Java, JUnit and Selenium; the workflow does not change.

If you want an opinionated version already wired to good defaults, qarepo-playwright-starter is a clone-and-go template with a bundled demo app, per-worker DB isolation, and a sharded GitHub Actions workflow. The docs/reviewing-ai-tests.md page there is the concrete checklist for the three failure modes above.

Day 2: write your first contract-driven feature.

Pick something small. Email validation is a perfect first target because the rules are clear and the tests are quick. Follow the five steps above. Do not skip the contract even if the feature is small; the point is building the habit.

Day 3: try running playwright codegen against your own app.

pnpm exec playwright codegen http://localhost:3000

The tool records your clicks and generates a Playwright test. This is the fastest way to understand what a Playwright test looks like structurally without reading documentation. Take the generated test, clean it up, run it. You now have an e2e test.

Days 4-5: iterate on real work.

Every feature you deliver this week uses the five-step workflow. You will be slower than usual for the first two features and faster than usual by the fifth. The overhead compresses fast.

When to bring in a specialist

You do not need an SDET on the team yet. The workflow above scales to a team of three to five developers doing their own testing. Signals that you have outgrown it:

• CI takes more than 10 minutes and nobody knows how to make it faster.
• Tests are flaky and no one can predict which will fail today.
• Coverage numbers are misleading and no one can explain why.
• Contract testing between services keeps missing schema breakages.

Any two of those signals means you need someone who owns the test tooling as their day job. The piece What Is an SDET in 2026? explains what to actually look for when hiring that person and why the title is ambiguous.

Bottom line

Testing is not a specialist skill you can safely skip because you have AI. AI raises the ceiling on what a single developer can ship; it does not raise the floor of what can be trusted in production. The floor is still tests, and the tests still need to be good.

The good news is that the five-step workflow above turns “writing good tests” from a career skill into a 90-second-per-feature habit. Practice it for a week. Then look up in a month and notice that the features you ship are more likely to work on the first customer touch than the ones you shipped before you had the workflow. That is the entire point.

For the framework-specific skills, the Playwright vs Cypress vs Selenium comparison covers which tool to use when. The Test Coverage Explained piece covers what coverage numbers mean and when to trust them. The pillar covers why any of this matters in the age of AI-driven development. Together they map out the whole loop from the developer side.