Building APIs for users on 2G, prepaid data and phones from 2017: 20 things I learned the expensive way (1/20)

1/ Your users pay for every megabyte. Prepaid, in a currency that may have lost value this year. Every KB your API sends is money out of their pocket. Start there and most of this thread follows.

2/ Test like your users live. Chrome's "Slow 3G" preset is a holiday. On a Linux box between your test phone and the internet:

sudo tc qdisc add dev eth0 root netem delay 400ms 100ms loss 2% rate 400kbit
# remove it again
sudo tc qdisc del dev eth0 root

3/ Compress your JSON. I have seen production APIs that gzip the marketing site and send JSON raw. In nginx:

gzip on;
gzip_types application/json text/css application/javascript;
gzip_min_length 512;
gzip_comp_level 5;

Then check the response really has Content-Encoding. Do not assume.

4/ Let clients ask for less. GET /v1/transactions?fields=id,amount,status,created_at. Our transactions list went from 14KB to 1.1KB per page. Same screen.

5/ Cursor pagination, small default pages. Default 20, max 100. Offset pagination breaks when new transactions arrive while the user scrolls, and they always arrive.

6/ ETags. For a balance screen that people refresh nervously ten times a day, this is most of your traffic:

GET /v1/balance
If-None-Match: "a1b2c3"

HTTP/1.1 304 Not Modified

Zero bytes of body. The balance did not change. Nothing to send.

7/ Idempotency keys on everything that moves money. On a bad network the request reaches you, your response does not reach them, they tap again. Without a key you charge them twice.

POST /v1/payments
Idempotency-Key: 6f1c8e2a-4b1d-4f7a-9c3e-2d5b8a7f1e90

Store the key and the response, return the stored response on a repeat. Keep them for a day or three.

8/ Do not hold requests open. If you are waiting on a bank or a telco, return 202 Accepted with a status URL in a few hundred milliseconds instead of hanging for 60 seconds while the user's connection drops.

9/ Poll that status URL with backoff: 2s, 4s, 8s, then every 30s. Not every 500ms from a phone with 3% battery.

10/ One screen, one request. If your home screen makes 11 calls and the round trip is 800ms, users wait many seconds before anything appears, even with parallel requests. Build one endpoint for that screen. Call it a BFF, call it whatever, just build it.

11/ Reuse connections. A fresh TCP plus TLS 1.2 handshake is three round trips before the first byte. At 800ms RTT that is over two seconds of nothing. TLS 1.3, keep-alive, HTTP/2, session resumption. Boring settings, huge wins.

12/ Never send full size images in a list. A 48px avatar is 48px. Resize on the server, serve WebP, and put the size in the URL so caches work.

13/ SMS for the critical things. "Payment received" and OTPs should also go by SMS. When data dies, SMS very often still arrives.

14/ If your users have feature phones, your API is a USSD menu. Keep it shallow. USSD sessions time out, and every level of menu is a chance to lose them.

15/ Offline queues on the client, with the idempotency keys from 7/, so syncing twice is harmless.

16/ Error messages must say whether money moved. "Network error. Your payment was NOT sent. Try again." is worth more than any retry logic. "Error 500" is how you get a call centre full of crying people.

17/ Do not ship 2MB of JavaScript to show someone their balance.

18/ Cache public data. Exchange rates, product lists, bank lists: Cache-Control: public, max-age=300. Your bank list did not change in the last five minutes.

19/ Measure by country and by network type, p95 and p99, not a global average. Your global p50 is 180ms because half your requests come from your own office in Westlands.

20/ Test on a cheap phone. A real one. A Tecno, an Infinix, an itel, from the market, not the emulator. Sources and further reading: MDN's guide to HTTP caching, RFC 9110 for conditional requests and ETags, the IETF draft "The Idempotency-Key HTTP Header Field", and man tc-netem. Sawa, that is the thread. What did I miss?