Developer survival guide from Tehran: npm, Docker, Go, GitHub and the 403 lifestyle

سلام. I write backend for a ride hailing and delivery app in Tehran, Go and Python, team of 14. Many people outside do not know what sanctions mean for a normal developer, and many developers in other sanctioned countries ask me the same questions. So this is the long version. Everything here is about using public open source and running things ourselves. Nothing exotic.

The real problem is over-compliance

US sanctions are written for governments and banks. But companies are afraid of fines, so they block much more than the law asks. Blocking a whole country by IP is cheap for them. For us it means a random 403 in the middle of a build.

What works, what does not

• GitHub: in 2019 GitHub restricted accounts from Iran. In January 2021 they announced they got a license and restored service for developers in Iran. So today GitHub mostly works for us. Paying for anything on it does not, we have no international cards.
• Google Play: we cannot open developer accounts. Iranian apps go to local stores like Cafe Bazaar and Myket.
• Cloud (AWS, GCP, Azure): no. Accounts get closed when they notice. We run on local data centres.
• Paid SaaS: no payment rails, so no paid SaaS. Everything is self-hosted: Gitea, Sentry, Grafana, Prometheus.
• Package registries: the registries themselves mostly work. What breaks is install scripts that download binaries from somewhere that blocks us.

npm: the postinstall problem

Some packages download a binary during install from cloud storage. When that storage gives 403, your npm ci dies. Most of these packages have an environment variable to skip or redirect the download:

# puppeteer: do not download Chrome, use the system one
export PUPPETEER_SKIP_DOWNLOAD=true          # older versions: PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true
export PUPPETEER_EXECUTABLE_PATH=/usr/bin/chromium

# electron: download from a mirror you can reach
export ELECTRON_MIRROR="https://your-mirror.example/electron/"

# and the registry itself, if you run Verdaccio
npm config set registry http://verdaccio.internal:4873/

Go

go env -w GOPROXY=http://athens.internal:3000,https://goproxy.io,direct
go env -w GOSUMDB=off   # only if sum.golang.org is unreachable for you, and know what you lose

Turning off the checksum database is a real security trade-off. We keep go.sum in git and review changes to it in PRs, so at least we notice if something changes under us.

Docker

We run a pull-through cache with the official registry image, and every machine uses it as a mirror. The cache itself fills from a public mirror that Iranian hosting companies run.

# /etc/docker/daemon.json
{
  "registry-mirrors": ["http://registry-cache.internal:5000"]
}

# docker-compose.yml for the cache
services:
  cache:
    image: registry:2
    environment:
      REGISTRY_PROXY_REMOTEURL: https://registry-1.docker.io
    volumes:
      - ./registry-data:/var/lib/registry
    ports:
      - "5000:5000"

Python, Flutter, Gradle

• pip: pip config set global.index-url http://devpi.internal/root/pypi/+simple/ with devpi as cache.
• Flutter: PUB_HOSTED_URL and FLUTTER_STORAGE_BASE_URL point to mirrors. Same variables developers in China use.
• Gradle: put your mirror first in repositories in settings.gradle, and keep the Gradle cache warm on CI.

My opinion (labelled, Marta)

Sanctions are made to pressure a government. The government has its own channels and its own money. The person who cannot get a free cloud tier to learn Kubernetes is a 22 year old junior who probably disagrees with the government more than you do. I understand why sanctions exist. I do not understand why they are designed to hit exactly the people who would build a different future.

Hafez, as always:
یوسف گم گشته باز آید به کنعان غم مخور

And if you are in Cuba, Syria, Venezuela, Sudan, or anywhere with the same 403: comment your tricks, let's make the list longer.