Skip to content

surface Codex usage limits instead of retrying them - #139

Open
null-topology wants to merge 1 commit into
raine:mainfrom
null-topology:codex-usage-limit-reset
Open

surface Codex usage limits instead of retrying them#139
null-topology wants to merge 1 commit into
raine:mainfrom
null-topology:codex-usage-limit-reset

Conversation

@null-topology

Copy link
Copy Markdown

What happens now

When a Codex subscription window is spent, upstream ends the stream with an error event that already carries the reset clock, twice over:

{"type":"error","status_code":429,
 "error":{"type":"usage_limit_reached","message":"The usage limit has been reached",
          "plan_type":"plus","resets_at":1788879437,"resets_in_seconds":9568},
 "headers":{"X-Codex-Primary-Window-Minutes":"300",
            "X-Codex-Primary-Used-Percent":"100",
            "X-Codex-Primary-Reset-After-Seconds":"9569",
            "X-Codex-Primary-Reset-At":"1788879438",
            "X-Codex-Secondary-Window-Minutes":"10080",
            "X-Codex-Secondary-Reset-After-Seconds":"596369"}}

Nothing reads it. classify_event_failure sees status 429 and classifies a retryable rate limit, then looks for a delay in error.retry_after, error.retry_after_seconds, retry_after_seconds and headers.retry-after — none of which this event carries. With no delay, live_stream_response falls back to its own backoff and spends MAX_RETRYABLE_LIVE_STREAM_RETRIES in full.

In a capture of the failure that is 11 upstream attempts over 2m54s, each refused in under a second, with the time spent entirely in the backoff between them. The 429 that finally reaches the client carries no reset information, so clients retry immediately against a window that reopens hours later.

The capture was taken on a fork carrying this same classification and retry path; the constants and the code involved are unchanged here.

What this changes

usage_limit_from_event recognises the event and reads the clock. Codex reports both windows on every limit error, so the one that actually ran out is identified by the countdown matching the error's own, then mapped to a window by its length — 300 minutes is the session window, 10080 the weekly one.

The live stream answers such an event once, before entering the retry loop, with the headers clients already read for rate limit state:

x-should-retry: false
anthropic-ratelimit-unified-status: rejected
anthropic-ratelimit-unified-reset: 1788879437
anthropic-ratelimit-unified-representative-claim: five_hour

Retry-After is deliberately absent. Clients sleep for its full value, and here that value is hours — a worse outcome than the present one. x-should-retry: false stops the retry loop instead, which is the honest signal: a spent window does not reopen on a backoff.

Testing

Verified against an account with the session window spent: 1.5s instead of 2m54s, a single upstream attempt instead of eleven, and the reset time reaching the client.

Four unit tests cover the recorded payload shape, including window attribution when the weekly window is the one that ran out, and the header fallback when the error body omits the clock.

cargo fmt --check, cargo clippy --all-targets and cargo test are clean (1032 tests).

Follow-up

A second change is prepared that publishes the quota reading Codex sends during a healthy stream (codex.rate_limits) through the same headers, so a client can warn before the allowance runs out rather than only when it has. It is kept separate from this one.

A spent subscription window arrives as an `error` event carrying
`usage_limit_reached` and the moment the window reopens. Nothing read it:
the failure classified as a retryable rate limit, and the delay lookup
covers `retry_after`, `retry_after_seconds` and `headers.retry-after`, none
of which this event carries. With no delay the live stream spent its whole
budget -- eleven upstream attempts over 2m54s in a capture of the failure,
each answered 429 in under a second -- and then returned 429 with no reset
information, so clients retried immediately against a window that reopens
hours later.

Recognise the event and answer it once, with the headers clients already
read for rate limit state: which window ran out and when it reopens.
Retry-After is deliberately absent, since clients sleep for its full value
and here that is hours; x-should-retry stops the retry loop instead.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant