We let AI write our tests for a quarter. Coverage went up, confidence did not
Quick context: 12 people, 5 engineers including me, TypeScript monorepo, Postgres, Vitest. In Q2 we had a mandate (from me, honestly 😅) to "use AI to fix our test debt". Coverage was 41% and our investors' tech DD person had made a face. What we did Every engineer used an AI assistant to generate tests for untested modules, about 30 minutes a day. Rule: tests must pass and get reviewed in the PR like any other…
- 'The test enshrines the bug' is the sentence. I've been trying to explain this to my director for months. Coverage measures execution, not verification. Mutation score is the honest number, so glad you tracked it.