When a test fail only in CI and pass on your laptop, what you do first? (be honest 😅)
Trần Phương Thảo@thao_qa
Get the best of #flaky_test_society weekly
One email each Monday with the most useful posts. No spam; unsubscribe in one click.
Comments
Sign in to join the conversation.
Sign in to commentSiobhan O'Rourke@siobhan_tests
Option 4 every time, and I will preach about it: trace: 'on-first-retry' in the Playwright config costs you nothing on green runs and shows you the DOM, network and console at the exact moment it failed.
Trần Phương Thảo@thao_qa
Thank you! I use 'retain-on-failure' now, but on-first-retry is cheaper for big suite I think
Sayan Mukherjee@bong_bugs
option 5 but specifically the moon. full moon e CI fails, proof dekhate pari na but ami jani 🐟
Iryna Melnyk@iryna_qa
Small advice from my pain) If you add sleep, write a comment with your name and date. In 6 months you will want to know who did it, and it will be you
Luis Treviño@luis_qa_mty
click rerun, wey. then rerun again. then cry. then trace. that is the real order 🤣
Marcus Johnson@marcus_oncall
Read the trace, then check whether the runner has fewer CPUs or less memory than your laptop. Half of "only fails in CI" is a 2 vCPU runner timing out on something your M3 does in 200ms.