Flake Elimination

Field	Value
Type	Skill Resource
Source	~/.copilot/skills/quality/references/flake-elimination.md
Description	Not specified

Source Content

Flake Elimination

A flaky test — one that passes and fails on the same code — is a defect, not a nuisance. It erodes trust in the whole suite until people start ignoring red. The house rule: quarantine and fix; never paper over with a blanket retry-til-green.

Table of contents

• The rule
• The four common causes and their fixes
• The diagnose-fix-quarantine flow
• Retries: the narrow exception

The rule

When a test flakes:

1. Quarantine it (skip it with a tracking issue) so it stops poisoning the signal.
2. Diagnose the actual cause — flakiness always has one, it is not random.
3. Fix the cause, then return the test to the suite.

A global retry: 2 in the runner config hides flake instead of fixing it, and a hidden flake is a real bug the test was about to catch. Treat a flaky test exactly like a failing one.

The four common causes and their fixes

Most flake reduces to four causes. Check them in this order.

Timing and async

The test asserts before the async work finishes (or sleeps a fixed time hoping it does).

• Fix: assert with findBy* / waitFor, which retry until the condition holds or time out. Never setTimeout / fixed sleep to “wait for” the UI.
• Fix: await every userEvent call and every findBy*. A missing await is the single most common cause of intermittent UI failures.

Test-order dependency

Test B passes alone but fails after test A, because A left shared state behind.

• Fix: each test arranges and tears down its own state. No mutable module-level globals shared across tests.
• Fix: reset everything in afterEach — server.resetHandlers(), clear stores, restore mocks (vi.restoreAllMocks()).
• Diagnose: run with randomized order; if failures appear, you have an order dependency.

Real time and randomness

The test asserts against Date.now(), a timezone, or Math.random().

• Fix: fake the clock (vi.useFakeTimers() / vi.setSystemTime(...)) and restore it after.
• Fix: seed or inject randomness so the value is deterministic in tests.
• Fix: assert relative time (“3 days ago”) through the formatter against a fixed clock, never an absolute wall-clock string.

Real network

The test hit a live endpoint, a slow CDN, or an unmocked request that sometimes succeeds.

• Fix: MSW with onUnhandledRequest: "error" — every request is intercepted; an unmocked one fails loudly instead of flaking.
• Fix: model slow/error responses with MSW handlers, not by depending on real latency.

The diagnose-fix-quarantine flow

test flakes
   │
   ▼
quarantine it (skip + tracking issue)  ──► suite is green and trustworthy again
   │
   ▼
reproduce: run it 20× / randomized order / under load
   │
   ▼
classify cause: timing? order? time-randomness? network?
   │
   ▼
fix the cause (see section above)
   │
   ▼
prove it: run it 50×+ clean, then unquarantine

Do not unquarantine on a single green run — flake is intermittent by definition. Prove the fix with repeated runs before returning the test to the gate.

Retries: the narrow exception

Retries are acceptable only for genuinely external, non-deterministic boundaries you do not own — and even then, scoped, never global:

• A specific E2E step against a third-party system known to be eventually-consistent.
• Never on unit or component tests; those are fully deterministic and a retry there is hiding a real bug.

If you find yourself wanting a global retry, the suite has unfixed flake. Fix the cause.