the 5 kinds of flaky test i keep meeting, and what actually fixes them

Ive been the person who gets pinged when CI goes red for about six years now, first in a fintech and now in insurance. After a while you notice the flakes arent random. They come in the same handful of flavours. Heres my list, roughly in order of how often I see them.

1. Racing the UI

The test clicks before the button is ready, or reads text before the request came back. The fix people reach for is a sleep. Dont. In Playwright use the locator assertions, they retry until the condition holds or the timeout hits:

await page.getByRole('button', { name: 'Get quote' }).click();
await expect(page.getByTestId('premium')).toHaveText('€42.10');

If you write expect(await locator.textContent()).toBe(...) you read it exactly once and youre back to racing. Thats honestly half the flakes I fix.

2. Tests that depend on each other

Test B only passes because test A created a user first. Run them in a different order, or in parallel, and B falls over. Check with --repeat-each=10 and --fully-parallel locally, if a test cant survive that it isnt independent. Every test sets up its own data, every time. Yes its slower. Its less slow than re-running the pipeline three times.

3. Shared test data

Two workers both create test@example.com and one of them gets a unique constraint error. Put something unique into every bit of data you create, the worker index plus a timestamp is grand:

const email = 'quote-' + test.info().workerIndex + '-' + Date.now() + '@example.test';

4. Time

Anything with "today", "next month", time zones or daylight saving. Our renewal tests used to fail one night a year (long story, ill write it up some day). Freeze the clock. Playwright has page.clock now so you can fix the time in the browser, and on the backend inject a clock instead of calling new Date() all over the place. Also run CI in a different time zone than your laptop on purpose, once, and see what breaks. Something will.

5. The environment itself

A slow shared staging database, a third party sandbox thats down, a container with half the memory of your machine. These look like flakes but they arent your tests fault. Mock third parties at the network level with page.route in UI tests, and keep a small separate suite that hits the real sandbox and is allowed to go red without blocking merges.

what i do with a flake on day one

• Reproduce it with --repeat-each=50. Fails 1 in 50, you can fix it. Fails 0 in 50 locally, run it in CI with tracing on (trace: 'on-first-retry') and read the trace, its like a little flight recorder.
• Quarantine it if its blocking people, but give it an owner and a date. A skipped test with no owner is a deleted test that hasnt admitted it yet.
• Keep retries low. One retry in CI hides the odd blip and thats fine. Three retries hides real bugs. I watched a genuine race condition in payment code hide behind retries: 3 for months.

Thats it. None of this is clever. Its just boring consistency, which is most of QA if im honest.