Idempotency keys in Go: the version that survives a double tap and a dead inverter

Customer taps Pay. The request goes out on a 3G connection in Onitsha, reaches our API, the charge succeeds, and the response dies somewhere on the way back. The app sees a timeout and retries. Without protection, that is two charges and one very angry customer calling the merchant, who calls us.

Same thing happens when NEPA takes light in the middle of a request and the client's inverter takes three seconds to kick in. Retries are not an edge case. Retries are Tuesday.

The contract

The client generates a UUID per payment attempt and sends it as Idempotency-Key. Same key means "this is the same request, do not do it again". We store the key with a hash of the request body, a status, and the response we sent.

CREATE TABLE idempotency_keys (
    merchant_id  bigint      NOT NULL,
    key          text        NOT NULL,
    request_hash bytea       NOT NULL,
    status       text        NOT NULL DEFAULT 'in_progress',
    status_code  int,
    response     jsonb,
    created_at   timestamptz NOT NULL DEFAULT now(),
    PRIMARY KEY (merchant_id, key)
);

Claim the key first, then do the work

func (h *Handler) CreatePayment(w http.ResponseWriter, r *http.Request) {
	ctx := r.Context()
	key := r.Header.Get("Idempotency-Key")
	if key == "" {
		http.Error(w, "Idempotency-Key required", http.StatusBadRequest)
		return
	}
	body, err := io.ReadAll(io.LimitReader(r.Body, 1<<20))
	if err != nil {
		http.Error(w, "bad body", http.StatusBadRequest)
		return
	}
	sum := sha256.Sum256(body)
	merchant := merchantID(ctx)

	tag, err := h.db.Exec(ctx,
		"INSERT INTO idempotency_keys (merchant_id, key, request_hash) VALUES ($1, $2, $3) ON CONFLICT DO NOTHING",
		merchant, key, sum[:])
	if err != nil {
		http.Error(w, "db error", http.StatusInternalServerError)
		return
	}

	if tag.RowsAffected() == 0 {
		h.replay(w, ctx, merchant, key, sum[:])
		return
	}

	// we own this key now. do the charge exactly once.
	code, resp := h.charge(ctx, merchant, key, body)

	_, _ = h.db.Exec(ctx,
		"UPDATE idempotency_keys SET status = 'done', status_code = $3, response = $4 WHERE merchant_id = $1 AND key = $2",
		merchant, key, code, resp)

	w.WriteHeader(code)
	w.Write(resp)
}

The INSERT ... ON CONFLICT DO NOTHING plus RowsAffected() is the whole trick. Two identical requests arriving at the same millisecond: Postgres lets exactly one insert win. No SELECT-then-INSERT race.

The replay path

func (h *Handler) replay(w http.ResponseWriter, ctx context.Context, merchant int64, key string, hash []byte) {
	var rec struct {
		hash   []byte
		status string
		code   *int
		body   []byte
	}
	err := h.db.QueryRow(ctx,
		"SELECT request_hash, status, status_code, response FROM idempotency_keys WHERE merchant_id = $1 AND key = $2",
		merchant, key).Scan(&rec.hash, &rec.status, &rec.code, &rec.body)
	if err != nil {
		http.Error(w, "db error", http.StatusInternalServerError)
		return
	}

	switch {
	case !bytes.Equal(rec.hash, hash):
		http.Error(w, "key reused with a different request", http.StatusUnprocessableEntity)
	case rec.status == "in_progress":
		w.Header().Set("Retry-After", "1")
		http.Error(w, "request in progress", http.StatusConflict)
	default:
		w.WriteHeader(*rec.code) // replay the ORIGINAL status, not 200
		w.Write(rec.body)
	}
}

Same key, different body: 422, because that is a client bug and you want them to see it. Same key while the first request is still running: 409 with Retry-After, so a client retrying in a loop gets told to wait instead of getting a second charge.

The part every tutorial skips

What if the process dies after the charge succeeded but before the UPDATE? The key is stuck at in_progress forever and you do not know if money moved. Two things fix this for us:

1. Pass the key down. The provider call uses our key as its reference, and most providers worth using accept their own idempotency key or reference. So even our own retries cannot double charge at the provider.
2. A sweeper. Every two minutes, keys in in_progress older than 5 minutes get looked up at the provider by reference, and the record is finished with the real outcome.

Plus a nightly delete of keys older than 72 hours, so the table does not grow forever.

If you want the long version of this idea, Brandur Leach wrote the classic piece on Stripe-style idempotency keys in Postgres at brandur.org/idempotency-keys. Our version is smaller, because our team is smaller, and because our inverter is louder 😂