surface Codex usage limits instead of retrying them - #139
Open
null-topology wants to merge 1 commit into
Open
Conversation
A spent subscription window arrives as an `error` event carrying `usage_limit_reached` and the moment the window reopens. Nothing read it: the failure classified as a retryable rate limit, and the delay lookup covers `retry_after`, `retry_after_seconds` and `headers.retry-after`, none of which this event carries. With no delay the live stream spent its whole budget -- eleven upstream attempts over 2m54s in a capture of the failure, each answered 429 in under a second -- and then returned 429 with no reset information, so clients retried immediately against a window that reopens hours later. Recognise the event and answer it once, with the headers clients already read for rate limit state: which window ran out and when it reopens. Retry-After is deliberately absent, since clients sleep for its full value and here that is hours; x-should-retry stops the retry loop instead.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What happens now
When a Codex subscription window is spent, upstream ends the stream with an
errorevent that already carries the reset clock, twice over:{"type":"error","status_code":429, "error":{"type":"usage_limit_reached","message":"The usage limit has been reached", "plan_type":"plus","resets_at":1788879437,"resets_in_seconds":9568}, "headers":{"X-Codex-Primary-Window-Minutes":"300", "X-Codex-Primary-Used-Percent":"100", "X-Codex-Primary-Reset-After-Seconds":"9569", "X-Codex-Primary-Reset-At":"1788879438", "X-Codex-Secondary-Window-Minutes":"10080", "X-Codex-Secondary-Reset-After-Seconds":"596369"}}Nothing reads it.
classify_event_failuresees status 429 and classifies a retryable rate limit, then looks for a delay inerror.retry_after,error.retry_after_seconds,retry_after_secondsandheaders.retry-after— none of which this event carries. With no delay,live_stream_responsefalls back to its own backoff and spendsMAX_RETRYABLE_LIVE_STREAM_RETRIESin full.In a capture of the failure that is 11 upstream attempts over 2m54s, each refused in under a second, with the time spent entirely in the backoff between them. The 429 that finally reaches the client carries no reset information, so clients retry immediately against a window that reopens hours later.
The capture was taken on a fork carrying this same classification and retry path; the constants and the code involved are unchanged here.
What this changes
usage_limit_from_eventrecognises the event and reads the clock. Codex reports both windows on every limit error, so the one that actually ran out is identified by the countdown matching the error's own, then mapped to a window by its length — 300 minutes is the session window, 10080 the weekly one.The live stream answers such an event once, before entering the retry loop, with the headers clients already read for rate limit state:
Retry-Afteris deliberately absent. Clients sleep for its full value, and here that value is hours — a worse outcome than the present one.x-should-retry: falsestops the retry loop instead, which is the honest signal: a spent window does not reopen on a backoff.Testing
Verified against an account with the session window spent: 1.5s instead of 2m54s, a single upstream attempt instead of eleven, and the reset time reaching the client.
Four unit tests cover the recorded payload shape, including window attribution when the weekly window is the one that ran out, and the header fallback when the error body omits the clock.
cargo fmt --check,cargo clippy --all-targetsandcargo testare clean (1032 tests).Follow-up
A second change is prepared that publishes the quota reading Codex sends during a healthy stream (
codex.rate_limits) through the same headers, so a client can warn before the allowance runs out rather than only when it has. It is kept separate from this one.