Slack / Discord

How this problem is worked

Ten stages, from the questions you would ask an interviewer to the trade-offs you would defend. Each asks one question, and the simulator runs the architecture you draw against the requirements you wrote.

1. 01ClarificationsWhat would you ask before drawing a single box?
2. 02Functional reqsWhat must this system actually do?
3. 03Non-functionalWhat must it promise about speed, uptime and correctness?
4. 04Capacity estimationHow much load and data does this have to hold?
5. 05API designWhat does the outside world call, and what comes back?
6. 06Data modelWhat gets stored, and what is it looked up by?
7. 07Use-case breakdownHow does each requirement actually get served?
8. 08High-level designWhich components handle a request, and in what order?
9. 09Deep divesWhich part breaks first, and what do you do about it?
10. 10Trade-offsWhat did this design cost, and what breaks at 10×?

Primary sources for this problem

• Discord — How Discord Stores Trillions of Messages (Cassandra → ScyllaDB)
• Discord — How Discord Scaled Elixir to 5,000,000 Concurrent Users
• Discord — Maxjourney: 1M+ Online in a Single Server (relay tier)
• Discord — Using Rust to Scale Elixir for 11M Concurrent Users (SortedSet NIF)
• Slack — Flannel: an Application-Level Edge Cache to Make Slack Scale
• Slack — Real-time messaging (Channel Server + Gatewayserver)
• Slack — Scaling Datastores at Slack with Vitess
• Slack — Slack's Outage on January 4th, 2021 (TGW saturation)
• Slack — Slack's Incident on 2-22-22 (Consul / Vitess feedback)
• Slack — A Terrible, Horrible, No-Good, Very Bad Day (May 2020 HAProxy)
• Slack — Migration to a Cellular Architecture (InfoQ 2024)
• Slack — Tracing Notifications & How Slack Rebuilt Notifications
• Slack — Migrating Millions of Concurrent WebSockets to Envoy
• elixir-lang.org — Real-Time Communication at Scale with Elixir at Discord (2020)
• Cloudflare — July 17 2020 BGP outage post-mortem
• Google SRE Workbook, Ch. 5 (alerting on SLOs)