The test that only failed on Tuesdays (yes, only Tuesdays)
Contents
ok this one took me 3 weeks and I need to share it because neta I still don't believe it.
Context: I do QA automation for a US insurance app, Playwright + TypeScript, around 900 e2e tests that run every night on CI at 2am Central. We had one test, renew policy shows correct premium, that started failing. Not every night. Some nights. Rerun, it pass. Classic flaky, we put it on the quarantine list and move on.
Then my team lead ask me to "please find the pattern, wey" (he didn't say wey, I add it). So I export 60 nights of results to a spreadsheet. And there it was. It failed on 8 nights. All 8 were Tuesdays. Every Tuesday, no exception, never any other day.
Suspect 1: timezone
Because it is always timezone. CI runs in UTC, the app thinks in America/Chicago, 2am Central is 7am UTC, same date. Not timezone. I was almost sad.
Suspect 2: something that runs on Tuesdays
I asked infra. Backups, cron jobs, staging reset. Nothing on Tuesday. Infra looked at me like I was crazy. Fair.
Suspect 3: the test itself
Finally I read the login helper, which I never read because "it works". It had this:
// spread load across test accounts so we don't get rate limited
const TEST_USERS = [
'qa_sun@example.test',
'qa_mon@example.test',
'qa_tue@example.test',
'qa_wed@example.test',
'qa_thu@example.test',
'qa_fri@example.test',
'qa_sat@example.test',
]
const user = TEST_USERS[new Date().getDay()]
getDay() returns 0 for Sunday, so 2 is Tuesday. And qa_tue was the account a manual tester used 4 months ago to test policy cancellation. Her policy was cancelled. The renew screen added a reinstatement fee, so the premium was different. Only on Tuesdays, because the test only logged in as her on Tuesdays 🤣
Somebody wrote this 2 years ago to avoid rate limits, and nobody remember.
The fix
- No shared accounts for e2e. Each test creates its own user and policy through the API in a fixture, and deletes it after. Maybe 2 seconds slower per test. I don't care.
- Time is controlled, not real. Playwright has a clock API (1.45+), so tests that care about dates set it explicitly:
test('renewal premium on the anniversary date', async ({ page }) => {
await page.clock.setFixedTime(new Date('2026-08-25T10:00:00-05:00'))
await page.goto('/policies/renew')
// ...
})
- Timezone and locale fixed in the config, so nobody's laptop is "special":
// playwright.config.ts
use: {
timezoneId: 'America/Chicago',
locale: 'en-US',
},
- And the most important: the quarantine list now has an owner and an expiry date for every test. If a test sits there more than 2 weeks, it goes to standup. Flaky is not a root cause, is a symptom.
Moral: when a test is flaky, export the history and look at the days. Look at the time. Look at the user. Not everything is a race condition, sometimes is just a Tuesday.
Ahorita I am checking every new Date() in our helpers. There are 41. Pray for me.
Get the best of #flaky_test_society weekly
One email each Monday with the most useful posts. No spam; unsubscribe in one click.
Comments
Sign in to join the conversation.
Sign in to comment