CodeAdvanced

Triage a test suite that fails at random

My CI is red often enough that people rerun jobs instead of reading them. Failure data: {{paste test names, failure counts, and recent failure output}}. Stack and test runner: {{e.g. pytest + Postgres, Jest + jsdom}}. How tests run in CI: {{parallel workers, shared DB, retries on/off}}.

Do this:
1. Classify each failing test by likely cause: shared mutable state, test order dependence, real concurrency race, time or timezone, network or external service, resource limits, or a genuine intermittent product bug. Quote the evidence in the output that puts it in that bucket.
2. Separate the tests that are flaky from the tests that are correctly catching a real race in the product. Say which is which and why - this is the important distinction.
3. Rank fixes by (failure frequency x how cheap the fix is). Give the concrete change for the top three: the fixture, the isolation boundary, the wait condition.
4. Name what to stop doing - blanket retries, sleeps, mocking away the race - and what it hides.
5. A measurement plan: what to record for two weeks to prove the suite got better.

Do not suggest deleting or skipping a test without saying what coverage is lost.

How to use it

Paste failure counts per test, not just the latest log - frequency is what separates a real race from a shared-fixture problem. It cannot see your fixtures unless you include them, so verify each proposed isolation boundary against the actual setup code.

Compatible popular AI tools

These tools are mapped to this prompt based on their capabilities.