Idempotent payment endpoints in Go and Postgres: the pattern that stopped our double charges

For six months we refunded somebody almost every Monday. Same story every time: customer on a bad connection taps Pay, the request reaches us, the PSP charges them, our response never reaches the phone, the app retries, we charge again. This is the pattern we use now. Zero double charges since February.

The rules

1. Every POST that moves money requires an Idempotency-Key header. No key, 400. The client generates it once per payment attempt: when the payment screen opens, not when the button is tapped.
2. Keys are scoped per merchant. Two merchants can send the same key and never see each other's responses.
3. We store a hash of the request body with the key. Same key with a different body is a client bug and gets a 422, not a silent replay.
4. A key has a state: in_progress or done. This is the part most blog posts skip.

The table

CREATE TABLE idempotency_keys (
  merchant_id   bigint      NOT NULL,
  key           text        NOT NULL,
  request_hash  bytea       NOT NULL,
  status        text        NOT NULL DEFAULT 'in_progress',
  response_code int,
  response_body jsonb,
  created_at    timestamptz NOT NULL DEFAULT now(),
  PRIMARY KEY (merchant_id, key)
);

The handler (pgx, trimmed)

func (s *Server) CreatePayment(w http.ResponseWriter, r *http.Request) {
	ctx := r.Context()
	key := r.Header.Get("Idempotency-Key")
	if key == "" || len(key) > 255 {
		http.Error(w, "Idempotency-Key required", http.StatusBadRequest)
		return
	}
	body, err := io.ReadAll(io.LimitReader(r.Body, 1<<20))
	if err != nil {
		http.Error(w, "bad body", http.StatusBadRequest)
		return
	}
	sum := sha256.Sum256(body)
	merchantID := merchantFrom(ctx)

	// Claim the key. Exactly one request wins this insert.
	tag, err := s.db.Exec(ctx,
		"INSERT INTO idempotency_keys (merchant_id, key, request_hash) VALUES ($1, $2, $3) ON CONFLICT DO NOTHING",
		merchantID, key, sum[:])
	if err != nil {
		http.Error(w, "db error", http.StatusInternalServerError)
		return
	}
	if tag.RowsAffected() == 0 {
		s.replay(ctx, w, merchantID, key, sum[:])
		return
	}

	code, resp := s.charge(ctx, merchantID, key, body)

	_, err = s.db.Exec(ctx,
		"UPDATE idempotency_keys SET status = 'done', response_code = $3, response_body = $4 WHERE merchant_id = $1 AND key = $2",
		merchantID, key, code, resp)
	if err != nil {
		log.Printf("idempotency: could not store response for %d/%s: %v", merchantID, key, err)
	}
	w.Header().Set("Content-Type", "application/json")
	w.WriteHeader(code)
	w.Write(resp)
}

func (s *Server) replay(ctx context.Context, w http.ResponseWriter, merchantID int64, key string, sum []byte) {
	var status string
	var hash, body []byte
	var code *int
	err := s.db.QueryRow(ctx,
		"SELECT status, request_hash, response_code, response_body FROM idempotency_keys WHERE merchant_id = $1 AND key = $2",
		merchantID, key).Scan(&status, &hash, &code, &body)
	if err != nil {
		http.Error(w, "db error", http.StatusInternalServerError)
		return
	}
	if !bytes.Equal(hash, sum) {
		http.Error(w, "Idempotency-Key reused with a different request", http.StatusUnprocessableEntity)
		return
	}
	if status != "done" || code == nil {
		w.Header().Set("Retry-After", "2")
		http.Error(w, "a request with this key is still in progress", http.StatusConflict)
		return
	}
	w.Header().Set("Content-Type", "application/json")
	w.WriteHeader(*code)
	w.Write(body)
}

The parts people get wrong

• The PSP call must be idempotent too. Pass your key, or something derived from it, downstream. Stripe accepts an Idempotency-Key header. With Paystack and most African PSPs, send your own unique transaction reference: they reject a duplicate reference.
• The server can die between the charge and the UPDATE. Then the key sits in in_progress forever and the client gets 409s. We run a reconcile job every minute: any key in progress for more than 5 minutes, ask the PSP what happened to that reference, then write the real outcome.
• Do not wrap the PSP call in the same DB transaction. You will hold a transaction open for 30 seconds while a bank thinks about life. Claim the key, commit, call out, then record.
• Expire keys. We keep them 72 hours, then a nightly job deletes them. A retry after 3 days is not a retry, it is a new payment.
• The ledger is still the source of truth. Idempotency stops duplicate requests. It does not replace double-entry bookkeeping and a daily reconciliation against the settlement file. That is another post.

Na so we stopped refunding people every Monday. If you write this in Rust it will take you three weeks and a lifetime of suffering, but the pattern is the same 😂