Amara Diallo spent four years on distributed systems before inference became the hardest scaling problem in the building. When Northbound Labs' research team shipped a model that worked beautifully for ten users and collapsed at ten thousand, she was handed a quarter and a small team to fix it.
The rewrite that followed is now the company's most reused internal library. It is also, she is quick to say, mostly a story about deleting things: three caching layers, two queues, and an abstraction nobody could explain in a meeting.
She writes a short internal memo after every incident and publishes it unedited. The habit has outlasted two reorgs.
In Conversation
- What do most teams get wrong about serving models?
- They optimise the average and get destroyed by the tail. Your slowest one percent is the experience people describe to their friends.
- How do you know a rewrite is worth it?
- When the cost of explaining the current system to a new engineer is higher than the cost of replacing it.
- One habit you would give away?
- Write the incident memo the same day. By Friday it becomes a story about what you meant to do.




