Problems
#12Build Kafka
A partitioned, replicated, append-only log. The log is the database — internalize that, and a dozen product designs get easier.
Saved on this device
You are designing a distributed log: producers append messages, consumers read them in order, the system survives machine failures. This is the canonical "log is the database" system — once you have built it, you understand why partitions matter for ordering, why exactly-once is hard, what ISR means when a broker dies, and why log compaction has the trade-offs it does.
Resist the urge to "describe Kafka." Make decisions yourself, defend them, and let the AI push back. The point isn't to recreate Kafka byte-for-byte — it's to make every choice that the Kafka authors made, and feel why they made it.
Reading: Kreps — The Log: What every software engineer should know · Kafka original paper (LinkedIn, 2011) · Confluent — KRaft design doc
append-only log
partitioning
replication
ISR & quorum
log compaction
exactly-once semantics
controller / KRaft