AI Agent Platform

Long-running multi-step LLM agents — durable workflow + sandboxed execution + LLM gateway. Brain / hands / state are independently replaceable. The agent run is a workflow, not a request.

SavedSaved on this device

The problem

Long-running multi-step LLM agents — durable workflow + sandboxed execution + LLM gateway. Brain / hands / state are independently replaceable. The agent run is a workflow, not a request.

Build the platform that powers long-running multi-step LLM agents — Devin, Manus, OpenAI Codex agents, Cursor agents, Claude Code background agents, Replit Agent, Lindy. Users describe a task; the platform spawns an agent that plans, calls an LLM, calls tools, executes code in a sandbox, captures results, and iterates — sometimes for minutes, sometimes for hours, sometimes (Cursor's Solid→React migration) for three weeks. The architectural pressure here is not LLM inference (that's an external dependency) — it's durability, isolation, and cost containment under failure.

Three load-bearing concepts:

An agent run is a durable workflow, not a request. Brain (LLM gateway) / hands (sandbox) / state (orchestrator + history) are independently replaceable. When the sandbox host dies 90 minutes into a 2-hour Devin task, the workflow survives because Temporal's history is the source of truth. (Cognition's published architecture; Temporal's Replit case study.)
KV-cache-aware routing is not optional. Manus's blog calls KV-cache hit rate "the single most important metric for a production-stage AI agent" — Anthropic's prompt-caching read price is 10% of base, so a 90% cache hit rate cuts cost by ~80%. A naïve OpenAI-protocol-compatible passthrough cannot do this; the gateway must understand prefix locality.
Cost runaway is enforcement, not alerting. The published $47K LangChain incident (four agents in an unintended infinite loop, 11 days, $47K bill) is the canonical retro. Per-tenant kill-switches must fire at minute granularity, not at the daily billing job. Replit users have reported $30/hr → $360/day before their 2025 controls landed.

Reading: Cognition — Devin's 2025 Performance Review · Cognition — What We Learned Building Cloud Agents · Manus — Context Engineering for AI Agents (KV-cache hit rate) · Anthropic — Prompt Caching docs (5min/1h TTL) · Anthropic — Postmortem of Three Recent Issues (Aug/Sep 2025) · Anthropic — Claude Code sandboxing · Temporal — Replit Agent case study · Temporal — Of course you can build dynamic AI agents · Diagrid — Checkpoints Are Not Durable Execution · E2B — Firecracker vs QEMU (125 ms boot, 4 000 microVMs/host) · Modal — Top AI Code Sandbox Products 2025 · Simon Willison — The lethal trifecta for AI agents · OWASP — Top 10 for Agentic Applications (Dec 2025) · Anthropic — EscapeRoute MCP CVEs (CVE-2025-53109/53110) · Replit — Effort-based pricing recap (July 2025 billing bug) · OpenTelemetry — GenAI semantic conventions (gen_ai.operation.name)

durable execution (Temporal-style event-sourced workflow)
brain / hands / state separation
Firecracker microVM sandboxing
fork-from-snapshot
KV-cache-aware sticky LLM routing
Anthropic prompt caching (90% read discount)
MCP (Model Context Protocol) tool gateway
OAuth 2.1 PKCE for tool capabilities
per-tenant token-budget kill-switch
cost runaway circuit-breaker
SSE token streaming with resumable offsets
human-in-the-loop signal with timeout
indirect prompt-injection defense
snapshot+restore on sandbox crash
non-determinism error / workflow versioning

01Clarifications

What would you ask before drawing a single box?

Ambiguity you would resolve with the interviewer: scope, scale, who uses it, what counts as done.

About AI Agent Platform

Long-running multi-step LLM agents — durable workflow + sandboxed execution + LLM gateway. Brain / hands / state are independently replaceable. The agent run is a workflow, not a request.

Difficulty: advanced
Time: about 50 minutes
Stages: 10
Topic: Consensus, Coordination & Durable Execution

How this problem is worked

Ten stages, from the questions you would ask an interviewer to the trade-offs you would defend. Each asks one question, and the simulator runs the architecture you draw against the requirements you wrote.

01ClarificationsWhat would you ask before drawing a single box?
02Functional reqsWhat must this system actually do?
03Non-functionalWhat must it promise about speed, uptime and correctness?
04Capacity estimationHow much load and data does this have to hold?
05API designWhat does the outside world call, and what comes back?
06Data modelWhat gets stored, and what is it looked up by?
07Use-case breakdownHow does each requirement actually get served?
08High-level designWhich components handle a request, and in what order?
09Deep divesWhich part breaks first, and what do you do about it?
10Trade-offsWhat did this design cost, and what breaks at 10×?

Primary sources for this problem

Cognition — Devin's 2025 Performance Review
Cognition — What We Learned Building Cloud Agents
Manus — Context Engineering for AI Agents (KV-cache hit rate)
Anthropic — Prompt Caching docs (5min/1h TTL)
Anthropic — Postmortem of Three Recent Issues (Aug/Sep 2025)
Anthropic — Claude Code sandboxing
Temporal — Replit Agent case study
Temporal — Of course you can build dynamic AI agents
Diagrid — Checkpoints Are Not Durable Execution
E2B — Firecracker vs QEMU (125 ms boot, 4 000 microVMs/host)
Modal — Top AI Code Sandbox Products 2025
Simon Willison — The lethal trifecta for AI agents
OWASP — Top 10 for Agentic Applications (Dec 2025)
Anthropic — EscapeRoute MCP CVEs (CVE-2025-53109/53110)
Replit — Effort-based pricing recap (July 2025 billing bug)
OpenTelemetry — GenAI semantic conventions (gen_ai.operation.name)

Build the primitives this design leans on

Each one is an animated curriculum that constructs the system from scratch.

Build Build a production AI agent (from one API call up)A model API is a function from text to text — it remembers nothing, does nothing and fails without saying so. Everything an agent actually does lives in the program around it: the loop, the tools, the window, the budgets, the gates, the sandbox, the run log. Build that program, one mechanism at a time, until it can resolve a support ticket and issue a refund exactly once.

More in Consensus, Coordination & Durable Execution

Getting N machines to agree, and getting one job to happen exactly once: Raft, coordination services, CRDTs, locks, leader election, schedulers and durable workflows.

Browse the full problem catalog, or see what the simulator does and does not model.