lambda cold starts with node 22: pay for provisioned concurrency or move the API to cloudflare workers?
Tiago Ferreira@tiago_lx
ok let me make a question pá. our API (US startup, i'm remote from lisboa) runs on AWS Lambda + API Gateway, node 22, bundled with esbuild, about 2 MB per function. users are mostly US east coast and brazil.
problem: p99 on cold starts is about 1.4s and the product people see it in the dashboard every monday. options on the table:
- provisioned concurrency on the hot functions (finance did not love the number)
- move everything to cloudflare workers, no cold starts, edge, all the nice words
catch: we use postgres on RDS in us-east-1 with prisma. is the workers idea dumb? what would you measure first?
onprem_or_die please don't answer this one ahah
Top answer
measure before you migrate. the lambda REPORT log line has an Init Duration field on cold starts, that's the part you can actually attack. in my experience on node it splits into: loading the bundle, initialising clients at module scope, and whatever your ORM does on startup. prisma's engine is often the biggest single chunk, so check that first.
cheap things to try before spending money:
- switch the functions to arm64. usually a bit faster to init and cheaper per ms
- lazy-load anything heavy that only some routes need
- make sure you're on the modular v3 AWS SDK and only importing the clients you use
on workers: the compute moves to the edge, your database doesn't. a user in são paulo hits a worker in são paulo, which then makes a round trip to virginia for every query. edge compute in front of a single-region database is a very fast way to wait for the database. you'd also need connection pooling (hyperdrive or a driver that works there) and prisma's driver adapters. it's doable, it's just not free, and it doesn't fix the latency you think it fixes.
if init is still bad after trimming: provisioned concurrency on the 2-3 latency-critical functions only, and schedule it for business hours with application auto scaling instead of 24/7. that changes the conversation with finance a lot.
Get the best of #edge-of-reason weekly
One email each Monday with the most useful posts. No spam; unsubscribe in one click.
Comments
Sign in to join the conversation.
Sign in to commentAlasdair Reid@onprem_or_die
ye asked me not to answer so here i am. a VPS with node on it. no cold starts since 2009. you're welcome, serverless child
Tiago Ferreira@tiago_lx
ahah knew it. pá, my VPS would also be in virginia, same latency for brazil 😄
Jordan Pike@jpike
measure before you migrate. the lambda REPORT log line has an Init Duration field on cold starts, that's the part you can actually attack. in my experience on node it splits into: loading the bundle, initialising clients at module scope, and whatever your ORM does on startup. prisma's engine is often the biggest single chunk, so check that first.
cheap things to try before spending money:
- switch the functions to arm64. usually a bit faster to init and cheaper per ms
- lazy-load anything heavy that only some routes need
- make sure you're on the modular v3 AWS SDK and only importing the clients you use
on workers: the compute moves to the edge, your database doesn't. a user in são paulo hits a worker in são paulo, which then makes a round trip to virginia for every query. edge compute in front of a single-region database is a very fast way to wait for the database. you'd also need connection pooling (hyperdrive or a driver that works there) and prisma's driver adapters. it's doable, it's just not free, and it doesn't fix the latency you think it fixes.
if init is still bad after trimming: provisioned concurrency on the 2-3 latency-critical functions only, and schedule it for business hours with application auto scaling instead of 24/7. that changes the conversation with finance a lot.
Tiago Ferreira@tiago_lx
this is very good, obrigado. init duration is 900ms of the 1.4s and i'm pretty sure most of it is prisma. 'a very fast way to wait for the database' ahah ok, workers is not magic. trying arm64 + trimming first
Harpreet Sandhu@hsandhu_ops
scheduled provisioned concurrency for business hours cut ours about 60% vs running it 24/7. scheduled actions on the alias, about 10 lines of terraform
Marcus Johnson@marcus_oncall
also check if the p99 cold start actually hurts users or just your dashboard. we had 1.2s cold starts on an admin api used by 6 people. nobody cared except the alert