Build a CRDT library

No scenes authored for this problem yet.

About Build a CRDT library

Conflict-free merging as a math problem. Build the canonical CRDTs — counters, sets, registers, sequences — and feel why a commutative-associative-idempotent merge function buys you offline-first sync without a coordinator. The price is paid in metadata growth, and that's the load-bearing trade.

Difficulty
advanced
Time
about 80 minutes
Stages
9
Topic
Consensus, Coordination & Durable Execution

How this problem is worked

Nine stages, from what the thing is for to how it compares with the real implementations. Each asks one question, and the simulator runs the architecture you draw against the requirements you wrote.

  1. 01Purpose & invariantsWhat is this for, and what must always be true of it?
  2. 02Workload characterizationWho writes, who reads, and in what shapes?
  3. 03Data model & on-disk formatWhat does the data look like at rest?
  4. 04Core algorithmsHow do the write path and the read path actually work?
  5. 05Distribution & replicationHow does this scale out and survive losing a machine?
  6. 06Consistency & correctnessUnder concurrency and failure, what is guaranteed?
  7. 07Failure modes & recoveryWhat actually happens when each part fails?
  8. 08Operational characteristicsCan a human run this at three in the morning?
  9. 09Trade-offs & comparisonWhere does this sit against the alternatives?

Primary sources for this problem

  • Shapiro, Preguiça, Baquero, Zawirski — A Comprehensive Study of Convergent and Commutative Replicated Data Types (INRIA 2011)
  • Shapiro et al. — Conflict-Free Replicated Data Types (SSS 2011)
  • Almeida, Shoker, Baquero — Delta state replicated data types (JPDC 2018)
  • Kleppmann — A Brief History of CRDTs (his talks and blog)
  • Yjs internals — YATA + the Yjs document model
  • Automerge documentation — RGA implementation in JavaScript/Rust
  • Riak DT — production CRDT implementations

More in Consensus, Coordination & Durable Execution

Getting N machines to agree, and getting one job to happen exactly once: Raft, coordination services, CRDTs, locks, leader election, schedulers and durable workflows.

Browse the full problem catalog, or see what the simulator does and does not model.