Payout worker queue in Postgres: SKIP LOCKED is holding transactions open for 30s, better way?
We have a payouts table and 4 worker processes that pick pending payouts and call the bank API. Right now each worker does this:
begin;
select * from payouts
where status = 'pending'
order by created_at
limit 10
for update skip locked;
-- call bank API for each row (can take up to 30s)
update payouts set status = 'sent' where id = ...;
commit;
It works, no double picks. But the transaction stays open during the HTTP call, and under load we see connections piling up and "idle in transaction" alerts. We are behind PgBouncer.
Options I am thinking:
- pg_advisory_lock per payout id instead of the row lock
- a status column with some kind of lease/timeout
Has anyone done this at decent volume (we do ~50k payouts a day)? What is the cleanest pattern that doesnt risk sending the same payout twice?
Accepted answer
Written by
Arjun Mehta
Backend engineer at a lending startup in Bengaluru. Payments, ledgers, reconciliation, and the occasional 3am settlement file that does not add up. Postgres fan. Will talk about idempotency keys until you leave.
Get the best of #select-star-anonymous weekly
One email each Monday with the most useful posts. No spam; unsubscribe in one click.
Comments
Sign in to join the conversation.
Sign in to commentAccepted answer