Our software house lost five days of internet in July 2024. Here is everything we changed after
Actually I wanted to write this for two years. I will not write about politics in this post. Only about what a shutdown costs a small outsourcing company, and what we changed.
We are 34 people in Dhaka. Most clients are in Norway and Denmark. In July 2024 mobile internet went off first, then broadband. For about five days we had nothing. No Slack, no email, no GitHub, no way to tell clients why we disappeared.
The thing is, international voice calls still worked. So our CEO sat in his car with his personal phone and called every client one by one. Expensive calls. Some clients were very kind. One said "take care of yourselves first". One sent a notice the next week. We lost that contract.
আমরা অনেক কিছু হারিয়েছি. Not only money. But this post is about the money part, so here is what we changed.
- Contracts. Our force majeure clause now names "government ordered internet or telecom shutdown" explicitly, and says the SLA is paused during it. Before, it said "acts of God". God was not the problem.
- Phone tree. Every client has two phone numbers of ours and we have two of theirs. Written down on paper in the office. Yes, paper.
- A dead man's switch. A small VPS in Frankfurt runs a cron job. If nobody from our side has pushed a heartbeat for 24 hours, it emails every client contact: "We may be offline due to connectivity issues in Bangladesh. Here is what to expect and who to call." We test it every quarter.
- The client can deploy without us. All CI/CD secrets and pipelines live in the client's own cloud account, with a runbook. If we vanish for a week, they can still ship a hotfix. Some of our sales people hated this. It is the reason two clients stayed.
- Weekly handover notes. Every project has a one page "if we disappear tomorrow" document. Current branch, what is half done, what not to touch.
- Someone outside. We now have two contractors working with us from outside Bangladesh, mainly so that someone can answer the client in the first hours.
- VPN is not a plan. Brother, people kept telling us "just use a VPN". When there is no internet at all, a VPN connects to nothing.
Total cost of all this is small. The contract we lost was bigger than all of it.
If you run a team in a country where this can happen, and the list of such countries is longer than people think, do the list above before you need it.
Get the best of #remote_life weekly
One email each Monday with the most useful posts. No spam; unsubscribe in one click.
Comments
Sign in to join the conversation.
Sign in to comment