Build a pub/sub system (Google Pub/Sub / Redis Pub/Sub style)
One publisher, N subscribers, no shared queue. Build the fanout primitive that powers notifications, cache invalidation, and reactive UIs — with explicit knobs for at-most-once vs at-least-once delivery, durable vs ephemeral subscriptions, and what happens when a slow subscriber stalls the pipe.Enter to send · Shift+Enter for a new line
About Build a pub/sub system (Google Pub/Sub / Redis Pub/Sub style)
One publisher, N subscribers, no shared queue. Build the fanout primitive that powers notifications, cache invalidation, and reactive UIs — with explicit knobs for at-most-once vs at-least-once delivery, durable vs ephemeral subscriptions, and what happens when a slow subscriber stalls the pipe.
- Difficulty
- intermediate
- Time
- about 70 minutes
- Stages
- 9
- Topic
- Queues, Pub/Sub & Event Streaming
How this problem is worked
Nine stages, from what the thing is for to how it compares with the real implementations. Each asks one question, and the simulator runs the architecture you draw against the requirements you wrote.
- 01Purpose & invariantsWhat is this for, and what must always be true of it?
- 02Workload characterizationWho writes, who reads, and in what shapes?
- 03Data model & on-disk formatWhat does the data look like at rest?
- 04Core algorithmsHow do the write path and the read path actually work?
- 05Distribution & replicationHow does this scale out and survive losing a machine?
- 06Consistency & correctnessUnder concurrency and failure, what is guaranteed?
- 07Failure modes & recoveryWhat actually happens when each part fails?
- 08Operational characteristicsCan a human run this at three in the morning?
- 09Trade-offs & comparisonWhere does this sit against the alternatives?
Primary sources for this problem
- Google Cloud Pub/Sub architecture & overview docs
- Redis Pub/Sub vs Streams — antirez blog and Redis docs
- NATS architecture: subjects, queues, JetStream
- AWS SNS — Amazon Builders' Library on fanout
- Eugster et al. — The Many Faces of Publish/Subscribe (ACM CSUR 2003)
- Confluent — Pub/sub vs queues vs streams (technical comparison)
More in Queues, Pub/Sub & Event Streaming
Moving events between services without losing them: logs, work queues, fanout, change data capture, stream processing, and the delivery systems built on top.
- Build Build KafkaA partitioned, replicated, append-only log. The log is the database — internalize that, and a dozen product designs get easier.
- Build Build a Message Queue (RabbitMQ / SQS)A point-to-point work queue — the messaging primitive Kafka is NOT. Each message goes to one consumer, ack deletes, retries push to a dead-letter queue, and a poisoned message is everyone's problem. Internalize ack vs visibility timeout vs DLQ vs prefetch vs FIFO groups — and learn to tell when Kafka is the wrong tool and when a queue is.
- Build Build a CDC pipeline (Debezium + outbox)Your service writes to its DB and publishes to Kafka — and any crash between those two writes is permanent inconsistency. Build a Change Data Capture pipeline (modeled on Debezium + the outbox pattern) that closes the gap by making the database itself the event source.
- Ad Click AggregatorStream processing with watermarks and exactly-once.
- Notification SystemPush, email, SMS. Idempotent. Failover.
- Distributed Cron — Mass Scheduled EmailSingle trigger, 50M recipient idempotency, catch-up.
Browse the full problem catalog, or see what the simulator does and does not model.