Arqly
Sign in
#13Build a stream processor (Flink / Kafka Streams style)
stub
Event time isn't processing time. Build a stream processor that tracks watermarks, windows by event time, holds keyed state with checkpoints, and recovers exactly-once after a node crash — and internalize why 'streaming SQL' is mostly the dataflow model in a different syntax.

Build a stream processor (Flink / Kafka Streams style). Event time isn't processing time. Build a stream processor that tracks watermarks, windows by event time, holds keyed state with checkpoints, and recovers exactly-once after a node crash — and internalize why 'streaming SQL' is mostly the dataflow model in a different syntax.

This problem is a stub. Suggested approaches haven't been authored yet — ask the staff engineer in the right panel for any stage and they'll generate one tuned to your draft.
Reading: Akidau et al. — The Dataflow Model (VLDB 2015) · Akidau, Chernyak, Lax — Streaming Systems (O'Reilly) · Apache Flink docs — Stateful Stream Processing & checkpointing · Kafka Streams architecture (Confluent docs) · Chandy & Lamport — Distributed snapshots paper (1985) · Kreps — The Log: What every software engineer should know
event time vs processing time vs ingestion time
watermarks as the imperfect 'no more late events' signal
windowing: tumbling, hopping, sliding, session
allowed lateness + late firing (the no-perfect-watermark trade)
keyed state vs operator state
state backends: in-memory, RocksDB-on-disk, remote
Chandy-Lamport checkpoints / asynchronous barrier snapshots
exactly-once via two-phase commit sinks + idempotent producers
joins on streams: interval joins, temporal joins (vs cartesian explosion)
the dataflow model: 'what / where / when / how' decomposition
No scenes authored for this problem yet.