Amara Diallo spent four years on distributed systems before inference became the hardest scaling problem in the building. When Northbound Labs' research team shipped a model that worked beautifully for ten users and collapsed at ten thousand, she was handed a quarter and a small team to fix it.

The rewrite that followed is now the company's most reused internal library. It is also, she is quick to say, mostly a story about deleting things: three caching layers, two queues, and an abstraction nobody could explain in a meeting.

She writes a short internal memo after every incident and publishes it unedited. The habit has outlasted two reorgs.

In Conversation

What do most teams get wrong about serving models?
They optimise the average and get destroyed by the tail. Your slowest one percent is the experience people describe to their friends.
How do you know a rewrite is worth it?
When the cost of explaining the current system to a new engineer is higher than the cost of replacing it.
One habit you would give away?
Write the incident memo the same day. By Friday it becomes a story about what you meant to do.