fix(tracing): continue inbound W3C trace context at the ACP boundary

[NiteshDhanpal]

Stacked on #484.

Problem

After #484 (business↔obs correlation + Option A) and #485 (Temporal interceptor), an async turn's obs trace still detached from the ingress: the workflow + every activity started fresh traces, unlinked from the event/send API call.

Root cause (proven on infra-staging with per-hop traceparent probes): the ingress traceparent arrives at the agent's ACP server in the HTTP header, but FastACP never extracts it — the active OTel context stays empty. So start_workflow/signal (and the asyncio.create_task background dispatch) fire with no active span, the Temporal interceptor injects nothing, and the workflow/activities root fresh traces.

Fix

Extract + attach the inbound W3C context in the ASGI RequestIDMiddleware, which wraps the whole request (so the background task inherits it via create_task's context copy). Now the interceptor propagates the ingress trace across the Temporal boundary and the workflow/activity inherit it → one connected trace. Fail-open (obs never breaks a request).

Verified (infra-staging)

Trace 12c6290e…: 347 spans across all 4 services in one tree — gateway POST /agents/{id}/rpc → agent ACP → RunWorkflow → HandleSignal:receive_event → RunActivity → egp/identity/DB. Per-hop probes confirmed traceparent went from <none> → the ingress trace at every hop (ACP + worker). (Requires the control-plane header-forward fix, scaleapi/scale-agentex#396, to deliver the header.)

Tests

tests/test_trace_context_extraction.py — extract-makes-inbound-active, fail-open on missing header, safe detach. ruff clean.

🤖 Generated with Claude Code

Greptile Summary

The PR extracts and attaches inbound W3C trace context around each FastACP HTTP request so Temporal dispatch created during the request inherits the ingress trace, then safely detaches the request context.

• Adds fail-open OpenTelemetry extraction and cleanup helpers.
• Integrates context attachment into RequestIDMiddleware.
• Adds unit coverage for extraction, missing headers, and safe detachment.

Confidence Score: 4/5

The PR appears safe to merge, with a non-blocking interoperability issue for repeated baggage or tracestate header fields.

The request-scoped attach/detach lifecycle supports the intended Temporal propagation path, while dictionary conversion can omit portions of valid multi-line W3C propagation metadata.

Files Needing Attention: src/agentex/lib/sdk/fastacp/base/base_acp_server.py

Important Files Changed

Filename	Overview
src/agentex/lib/sdk/fastacp/base/base_acp_server.py	Adds request-scoped W3C context extraction and cleanup; repeated propagation header fields are collapsed before extraction.
tests/test_trace_context_extraction.py	Adds focused helper tests for valid traceparent extraction and fail-open cleanup behavior.

Sequence Diagram

sequenceDiagram
    participant Gateway
    participant Middleware as RequestIDMiddleware
    participant ACP as ACP Handler
    participant Temporal as Temporal Client
    participant Worker as Workflow/Activity Worker
    Gateway->>Middleware: HTTP request + traceparent
    Middleware->>Middleware: extract and attach context
    Middleware->>ACP: Handle JSON-RPC under ingress context
    ACP->>Temporal: Start workflow or signal
    Temporal->>Worker: Propagate trace context
    Worker-->>Worker: Continue ingress trace
    ACP-->>Middleware: Request handling returns
    Middleware->>Middleware: Detach context

Loading

Prompt To Fix All With AI

### Issue 1
src/agentex/lib/sdk/fastacp/base/base_acp_server.py:69
**Repeated propagation headers collapse**

Converting the raw ASGI header list to a dictionary retains only the final value of repeated `baggage` or `tracestate` fields, silently omitting earlier propagation metadata from downstream context.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

_{Reviews (1): Last reviewed commit: "fix(tracing): continue inbound W3C trace..." | Re-trigger Greptile}

Greptile also left 1 inline comment on this PR.

[greptile-apps]

Repeated propagation headers collapse

Converting the raw ASGI header list to a dictionary retains only the final value of repeated baggage or tracestate fields, silently omitting earlier propagation metadata from downstream context.

Prompt To Fix With AI

This is a comment left during a code review.
Path: src/agentex/lib/sdk/fastacp/base/base_acp_server.py
Line: 69

Comment:
**Repeated propagation headers collapse**

Converting the raw ASGI header list to a dictionary retains only the final value of repeated `baggage` or `tracestate` fields, silently omitting earlier propagation metadata from downstream context.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.