Skip to content
experience

I logged every AI-assisted coding task for a month. Here is the honest ledger

October 4, 2026 · 2 min read

I keep asking people to measure, so it is only fair that I did it myself. For September I logged every coding task at work where I used an AI assistant: what it was, how long it took, how long I estimate it would have taken without, and whether I had to fix the output. The estimates are mine and biased; I tried to be conservative in both directions.

The ledger

tasks logged:                       47
clearly faster:                     29   (est. 11.5 h saved)
about the same:                     11
slower (fixing confident errors):    7   (est. 4.0 h lost)
net:                                     about 7.5 h saved
time spent coding that month:            about 70 h

Roughly a 10% gain on coding time. Real, useful, and nowhere near 10x.

Where it saved time

  • Boilerplate: argument parsing, plotting code, test fixtures, SQL for a schema I already understood. The biggest wins and the least interesting.
  • Unfamiliar APIs: "how do I do X in library Y", where I could check the answer immediately by running it.
  • Translation: a pandas snippet to SQL, a bash loop to Python.

Where it cost time

  • Seven tasks where the answer looked right and was not: a deprecated parameter, an off-by-one in a date window, and a statistical test that assumed independence when my samples were paired. That last one cost the most, because the code ran fine and the p-value looked lovely.
  • The pattern: it costs time when I cannot verify the output faster than I could have written it.

What I would actually conclude

  • The gain is concentrated in tasks where checking is cheap. Choose tasks accordingly.
  • Seniority matters in a direction people do not expect: the paired-samples error would have gone unnoticed by someone who did not already know the answer. The tool makes me faster partly because I can tell when it is wrong.
  • I will keep using it. I will not claim it doubled my output, because it did not.

Jo, I believe you owe me a spreadsheet.

Get the best of #ai_hype_check weekly

One email each Monday with the most useful posts. No spam; unsubscribe in one click.

Comments

10

Sign in to join the conversation.

Sign in to comment
Jo Brennan@jo_ships8 rep in AI Hype Check
ok ok. my october log is running. preliminary numbers after 2 weeks: i think my '10x' is more like 2x on the boring stuff and 0.5x on anything involving stripe webhooks. you were right and i'm annoyed about it 🐶
2 net votes
Dana Kowalski@dana_ml#1 in AI Hype Check
2x on the boring stuff is a genuinely good result for a solo founder, because you have a lot of boring stuff. I look forward to the spreadsheet.
2 net votes
Kenji Tanabe@kenjitanabe4 rep in AI Hype Check
The paired-samples point is the one I'd underline. Every 'juniors are cooked' take assumes the tool replaces judgement. In your log it amplifies judgement and punishes the absence of it.
2 net votes
Ryan Goh@ai_maxi_ryan#3 in AI Hype Check
10% because ur using it wrong. agents doing whole features end to end is where the 10x is. ur still in autocomplete land
4 net votes
Dana Kowalski@dana_ml#1 in AI Hype Check
Possibly. I would love to see your log. Same format: task, time, estimate without, fixes needed. One month.
2 net votes
Ryan Goh@ai_maxi_ryan#3 in AI Hype Check
i dont have time to log things, im shipping
2 net votes
Danny Cho@danny_cho_la3 rep in AI Hype Check
"I don't have time to measure, I'm shipping" is how every team I know ended up with a nine-month LLM feature that should have been a regex.
0 net votes
Mio Kobayashi@mio_rustacean2 rep in AI Hype Check
Our team did a similar log for two weeks. Similar result, around 10 to 15%. The biggest variance was by person, not by tool.
1 net votes
Renee Whitfield@renee_em3 rep in AI Hype Check
Sharing this with my managers. When leadership asks 'how much faster are we with AI', this is what an honest answer looks like. Would you mind if I used your table format for a team survey?
0 net votes
Dana Kowalski@dana_ml#1 in AI Hype Check
Please do. If you get results, I would love to see them aggregated.
1 net votes