i logged every work hour for 4 weeks to answer dana's question. did AI 10x my output? (no. but.)

back in june i posted that AI tools 10x'd my output. dana (@dana_ml) asked what 10x meant. i said it FEELS like 10x. she said feelings are data with a very wide confidence interval. she was right and it annoyed me for two months. so.

the setup

• 4 weeks, mid july to early august. 19 work days (i took one day off after the july mess)
• odd days: AI allowed (claude code, cursor, chat). even days: no AI except plain editor autocomplete and google
• every morning i picked tasks from the backlog by rolling a die, so i couldn't save the easy stuff for AI days. yes, a real die. biscuit tried to eat it
• before starting a task i wrote down my estimate in hours. real time went into toggl
• "done" = shipped. if it came back as a bug within the 4 weeks it's "reopened"
• extra column suggested by karthi in june: "confidently wrong", every time the AI handed me something that looked right and wasn't

the numbers

• logged: 147 hours over 19 days (10 AI days, 9 no-AI days)
• tasks shipped: 31 on AI days, 17 on no-AI days
• reopened within the 4 weeks: 6 of the 31, and 2 of the 17
• shipped and stayed shipped: 25 vs 15
• per day: 2.5 vs 1.67. that's 1.5x. not 10x
• estimates: on AI days i finished in 64% of my estimate on average. on no-AI days, 108%
• "confidently wrong": 14 times. most cost me under 10 minutes. two cost me over an hour each (a shopify API field that doesn't exist in the API version i'm on, and a postgres index suggestion that made a query slower)

by type of work

• boilerplate, CRUD, migrations, tests, glue code: about 3x. this is real, and this is where the "10x" feeling comes from, because it's the work i hate, and hated work feels five times longer than it is
• writing (docs, customer emails, changelogs): about 2x
• debugging production: about 1x. sometimes worse, because it sends me down a confident wrong path first
• shopify API weirdness: worse than 1x. it doesn't know which API version i'm on and it doesn't ask
• deciding what to build: 0x. no change at all. still the hardest part, still all mine

so, did AI 10x my output?

no. on the work i measured it's about 1.5x. on the work i hate it's 3x. on the work that matters most it's nothing.

which still makes it the best ~$100 a month i spend. 1.5x for a solo founder is like hiring half a person who never sleeps and is occasionally drunk.

but i retract "10x". dana wins. i'm putting it in writing so she can link it forever.

caveats (dana will add more)

• n = 1 founder, 19 days
• i knew which kind of day it was, so i probably worked harder on no-AI days out of pure spite
• tasks weren't identical, the die only made them random
• 4 weeks is too short to see whether AI-written code costs more later
• the july billing migration happened before the log started, so it's not in the numbers. it's in my heart

spreadsheet available if anyone wants to check my math. biscuit has reviewed it and approves.