fix(exec,checks): bound isolate fetch concurrency + keyless-flow sampler walks to a keyless door - #9
Conversation
… test cannot corrupt siblings
A suite test bursting concurrent fetches (the apis.vin 429 rate-limit test)
exhausted workerd's ~6-connection isolate budget: unread response bodies
pinned connections, the runtime force-closed the least-recently-used open
body, and a SIBLING test's plain GET-and-parse read a truncated non-JSON
body ('Response closed due to connection limit') — the verdict reflected
runner plumbing, not target behavior.
Two guards, isolate-side (entrySource fetch wrapper) and parent-side
(gatewayFetch / createOutboundGateway):
- a FIFO semaphore bounds in-flight fetches to EXEC_MAX_CONCURRENT_FETCHES
(5, headroom under the budget); excess QUEUES, nothing is refused
- every non-stream response body is FULLY BUFFERED while the slot is held
(bufferResponse), so an unread body can never pin a connection;
text/event-stream passes through live (buffering would hang to the wall
breaker)
Suite semantics are untouched: status, statusText, headers, url, and body
bytes are preserved verbatim — only wire-level concurrency is shaped.
Tests pin both directions under a transport that models the workerd budget:
burst + sibling GET-and-parse both pass; a GENUINELY non-JSON listings body
still fails its test; the gateway never exceeds the bound; SSE stays live.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…or is found — fail only when none exists The old sampler drew 3 random endpoints and failed keyless-flow when all 3 answered 401 — but own-scope keyed doors (/keys/me, /usage, /dealer/leads) CORRECTLY 401 a keyless probe, so a keyed-door-heavy property (apis.vin) could fail while keyless-first held (/listings, /pricing answer 200). The sampler now: - honors the card's per-endpoint auth hint (agents.json interfaces.http auth 'none'/'keyless'/'public'): hinted-keyless doors are walked FIRST, and keyed-hinted doors stay excluded from the candidate pool - keeps the seeded first wave at MAX_KEYLESS_PROBES, then ESCALATES while no door has answered 2xx, up to MAX_KEYLESS_PROBE_TOTAL (12) — stopping at the first keyless success Escalation widens the search, never the acceptance: a surface with genuinely NO keyless access walks to the cap, every door refuses, and keyless-flow still FAILS (detail now names the keyed-door semantics). Tests pin both directions across seeds: many-keyed + one-keyless passes, hinted doors resolve inside the cheap wave, zero-keyless fails at the cap. Card-side follow-up (apis.vin / axp-faces): declare auth hints per door so the walk resolves in wave 1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cc54653aa4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| for (const path of probeOrder) { | ||
| if (keylessProbed >= MAX_KEYLESS_PROBE_TOTAL) break | ||
| if (keylessProbed >= MAX_KEYLESS_PROBES && keylessFound) break |
There was a problem hiding this comment.
When an endpoint-rich target needs all 12 attempts before finding a keyless success, this loop consumes nine more requests than the previous sampler before the offer, MCP OAuth, registry, and other fixed high-value probes run. With the default 32-request Observer budget, a target that also publishes three face alternates, pricing/probe metadata, typed-body candidates, and an offer reaches the budget before the MCP chain, producing status:null evidence and false failures for otherwise valid independent features. Cap this phase according to the remaining reserved budget or move escalation after the fixed probes.
Useful? React with 👍 / 👎.
| export async function bufferResponse(res: Response): Promise<Response> { | ||
| const ctype = (res.headers.get('content-type') ?? '').toLowerCase() | ||
| if (ctype.includes('text/event-stream')) return res | ||
| if (res.body === null) return res | ||
| const buf = await res.arrayBuffer() |
There was a problem hiding this comment.
For a streaming endpoint that is not labeled text/event-stream—for example an NDJSON feed, chunked download, or response with no content type—fetch() now waits for arrayBuffer() to reach EOF before returning any Response. A suite that only checks the response status previously completed as soon as headers arrived, but now hangs until the five-minute wall breaker and is falsely failed; large finite bodies also get unconditionally materialized in memory. The buffering guard needs a bounded or streaming-safe strategy rather than consuming every non-SSE body to completion.
Useful? React with 👍 / 👎.
| const ctype = (res.headers.get('content-type') ?? '').toLowerCase() | ||
| if (ctype.includes('text/event-stream')) return res | ||
| if (res.body === null) return res |
There was a problem hiding this comment.
When a suite opens several text/event-stream responses and leaves them live, this early return bypasses buffering while gatewayFetch still releases the semaphore slot in its finally block; the generated isolate wrapper does the same. Consequently any number of live SSE connections can remain open while later requests continue, so six concurrent streams can still exhaust the workerd connection limit and corrupt or close a sibling response—the exact failure this limiter is intended to prevent. Streaming bodies need to retain or otherwise account for their slot until cancellation or closure.
Useful? React with 👍 / 👎.
…v1); railLedger address in projection config Studio #9 alignment pass 2026-08-23. Adds railLedger (https://ledger.apis.ax/faces?face=api.qa) plus the §9.1 final-box record to the served G4 projection config. No prior blocked-on-rail-ledger record existed in this repo — the box is recorded registered here for the first time, not flipped. Registered via LEDGER.md door A: row in packages/rail-ledger/registry/faces.json @ draft/rail-ledger-v1; readout https://ledger.apis.ax/readouts/faces-payable (service built, deploy pending Batch-S). The hand-authored quartet is untouched. Gate: npm test — 45 files, 1285 passed / 2 skipped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 participant
Two runner-robustness fixes for the production verifier. Both are correctness fixes for what the checks MEASURE — neither makes the grader lenient, and tests pin both directions.
1. Hosted exec runner (api.qa/vitest@1): bound isolate fetch concurrency + buffer bodies
A suite test bursting concurrent fetches (e.g. a 429 rate-limit test whose response bodies are never read) exhausted workerd's ~6-connection isolate budget: unread bodies pinned connections, the runtime force-closed the least-recently-used open body, and a SIBLING test's plain GET-and-parse read a truncated non-JSON body (
Response closed due to connection limit). The verdict reflected runner plumbing, not target behavior.Two guards, isolate-side (entrySource fetch wrapper) and parent-side (gateway):
EXEC_MAX_CONCURRENT_FETCHES,createFetchLimiter) — excess queues FIFO, never refusedbufferResponse) — status/statusText/headers/url/bytes preserved verbatim;text/event-streamstays a live streamAssertions are unchanged; only wire concurrency/timing is shaped. Local runner untouched (Node has no connection budget).
Not lenient:
test/vitest-subset.test.tsproves a burst of 20 unread-body fetches no longer truncates a sibling's JSON read, AND that a listings body that is GENUINELY non-JSON still fails by parse error.2. keyless-flow: sampler walks candidates until a keyless door is found — fail only when none exists
The old sampler drew 3 random endpoints and failed keyless-flow when all 3 answered 401 — but own-scope keyed doors CORRECTLY 401 a keyless probe, so a keyed-door-heavy property could fail while keyless-first genuinely held.
The sampler now walks [card-hinted keyless doors (
agents.jsoninterfaces.httpauthnone/keyless/public) in seeded order] then [seeded shuffle of unhinted candidates]; probes 3 minimum, escalates up toMAX_KEYLESS_PROBE_TOTAL(12) while zero 2xx seen, stops at the first keyless success. Keyed-hinted doors leave the candidate pool.Verdict rule unchanged in direction: pass iff ≥1 sampled 2xx. Escalation widens the SEARCH, never the acceptance.
Not lenient:
test/checks.test.tsproves many-keyed + one-keyless passes on every seed, hinted doors resolve inside the cheap 3-probe wave, and a surface with genuinely NO keyless access walks to the cap, every door refuses, and keyless-flow still FAILS.Budget note: escalation can spend up to 9 extra requests of the 32-request politeness budget, only on surfaces whose first wave found no keyless door.
🤖 Generated with Claude Code