Trace and span — stitching one user request

Each sidecar emits a span for its hop and stitches them into one trace via the shared traceparent header — but the trace silently breaks the moment an app forgets to copy that header onto an outbound request.

Previously

Two sidecars on every hop, all streaming live config from one control plane, are also the only fleet-wide observers of every request — so they are exactly where you stitch one user story together.

Scene 12

Trace and span — stitching one user request

  1. Watch
  2. Try it
  3. Predict
  4. Capture
TRACEone request · many spans, one shared trace idEvery hop forwards traceparent — four spans, one shared trace id, clean Gantt.RED METRICSsampling 100%req/s1200err2.0%p99340msSCproxyS1 · ingressserviceS1SCproxyS2 · ordersserviceS2SCproxyS3 · pricingserviceS3SCproxyS4 · inventoryserviceS4✓ traceparentpropagated✓ traceparentpropagated✓ traceparentpropagatedspanS1 · ingresst:trace-A…spanS2 · orderst:trace-A…spanS3 · pricingt:trace-A…spanS4 · inventoryt:trace-A…TRACE TIMELINEspans assembled by trace idTRACEt (ms)0100200300400tracetrace-A…S1 · ingr…360ms└ S2 · orde…300ms└ S3 · pric…220ms└ S4 · inve…130ms
↑ span — one hop's record (start, end, service)
↓ trace — all spans sharing one trace id
traceparent: 00-{trace-id}-{parent-span-id}-{flags} (W3C)
What to watch for

One request enters S1 and travels S1→S2→S3→S4. Each sidecar emits a span card as the dot crosses it. Below, the four spans assemble into one trace — same trace id on every span, nested by the parent it inherited from traceparent.

Implementation

Highlighted lines are the ones running in the diagram right now.

Sidecar.on_inbound_request
every inbound hop becomes one span, parented by traceparent
def on_inbound_request(req):
tp = parse(req.headers.get('traceparent'))
trace_id = tp.trace_id if tp else new_trace_id()
parent_id = tp.span_id if tp else None
span = start_span(
name=req.path,
trace_id=trace_id,
parent_id=parent_id,
)
resp = forward_to_app(req)
span.duration = elapsed(span.start)
emit_to_collector(span) # span is fire-and-forget
return resp
Sidecar.on_outbound_request
stamps traceparent on whatever request the app handed us
def on_outbound_request(req, current_span):
# W3C format: 00-<32 hex trace>-<16 hex span>-<2 hex flags>
req.headers['traceparent'] = (
f'00-{current_span.trace_id}'
f'-{current_span.span_id}'
f'-{flags_byte(sampled=current_span.sampled)}'
)
return req # sidecar can only stamp what's on the wire
Sidecar.emit (sampling vs RED)
metrics tick on every request; traces tick on flags & 0x01
def emit_to_collector(span):
# RED metrics: counted on EVERY request, no sampling.
metrics.requests.inc()
metrics.duration.observe(span.duration)
if span.errored: metrics.errors.inc()
# Traces: only written if the sampled bit is set.
if span.sampled: # head decision from flags byte
trace_store.write(span)
# else: span is dropped at emit time.

Where this sits in Build a Service Mesh (Envoy / Istio style)

Scene 11 of 13. Every sidecar emits a span tagged with the trace id from `traceparent`. One forgotten header rebuild breaks the trace silently — RED metrics keep flowing regardless.

Up next. Sidecar, listener, route, cluster, breaker, retry budget, rate limit, mTLS, control plane, trace — each is a knob; the next move is putting them all on one canvas and choosing settings for a concrete workload.

All 13 scenes in Build a Service Mesh (Envoy / Istio style) · Every curriculum

Built with Arqly
Every scene in Build a Service Mesh (Envoy / Istio style) builds on the one before it.All 13 Build a Service Mesh (Envoy / Istio style) scenes