Skip to content

Arm request timeouts on an event loop - #2313

Open
pavel-ptashyts wants to merge 1 commit into
AsyncHttpClient:mainfrom
maygemdev:feature/event-loop-request-timeouts
Open

Arm request timeouts on an event loop#2313
pavel-ptashyts wants to merge 1 commit into
AsyncHttpClient:mainfrom
maygemdev:feature/event-loop-request-timeouts

Conversation

@pavel-ptashyts

@pavel-ptashyts pavel-ptashyts commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Problem

Request and read timeouts are armed on the client's HashedWheelTimer. That has two
properties that only show up on short deadlines:

  • A wheel quantizes. It fires on the first tick at or after the deadline, so a
    deadline near or below hashedWheelTimerTickDuration is rounded up to it.
  • One thread carries every expiry for the whole client, and HashedWheelTimer's
    default taskExecutor is ImmediateExecutor, so each expiry runs inline on the wheel
    thread — including future.completeExceptionally(...) and therefore whatever the caller
    chained onto the response future.

On a one-second budget the first costs 0.3% and nobody notices. On a budget of tens of
milliseconds a tick is a large fraction of it, and a burst of expiries has no headroom to
absorb before the wheel starts running late.

Measured

2000 timeouts armed as one burst on Netty 4.2.16, JDK 17, tasks doing nothing but
recording their own lag. This is the floor; real work on the firing thread only adds to it.

instrument deadline mean p50 p99 max
wheel, tick 5 ms 20 ms +2.7 +2 +5 +5
wheel, tick 1 ms 20 ms +1.3 +1 +2 +2
EventLoop.schedule 20 ms +0.0 +0 +0 +0
wheel, tick 5 ms 1000 ms +3.0 +3 +3 +3
wheel, tick 1 ms 1000 ms +1.9 +2 +2 +2
EventLoop.schedule 1000 ms +1.6 +2 +2 +2

An event loop shows zero overshoot because it schedules by deadline and derives its own
select() timeout from the nearest one. There is no quantum to round to.

This was a throwaway probe rather than JMH — client/src/jmh/java is not currently wired
into the build, so its benchmarks do not compile. Happy to add a proper benchmark if that
is fixed first, or as part of this.

Change

AsyncHttpClientConfig#isUseEventLoopTimeouts(), off by default, arms the request and
read timeouts on an event loop instead of the timer.

  • Channel affinity where it exists. On the pooled path the channel is already in hand,
    so its own loop is used and the timeout expires on the thread that would have to close
    it. On the connect path there is no channel yet — deliberately, so the timeout also bounds
    address resolution and the connect — and any loop will do, because what the wheel costs is
    a single thread and a rounded-up tick rather than the identity of the thread.
  • Arming allocates nothing extra. The cancellation handle lives on the task rather than
    in a wrapper, and the existing done flag stands in for the scheduler's already-expired
    flag, which the two schedulers spell differently.
  • Shutdown race closed. isShuttingDown() can return false and schedule reject
    immediately after. Netty answers a rejected timeout with a logged warning rather than an
    exception, which would leave the exchange with nothing to end it, so a rejection falls
    back to the timer.
  • The connection-pool cleaner stays on the timer either way.

Off by default because the expiry — and so whatever the caller chained onto the future —
then runs on an I/O thread, and blocking one stalls every connection it serves. The javadoc
says so and points callers at handleAsync.

Why not a wheel per event loop

That is how the Aerospike client solves the same problem: EventLoopBase owns a
HashedWheelTimer that is a Runnable the loop ticks itself. Deliberately not copied here.
A wheel arms in O(1) against O(log n) for a deadline queue, but at a few thousand timeouts
per loop that is a dozen comparisons, while the quantization it reintroduces costs
milliseconds on a 20 ms budget — the third row above is the whole point. A wheel also has to
be ticked forever, waking every loop even with nothing armed. Aerospike wrote its own because
its EventLoop abstracts over NIO, Netty and direct NIO and needed one timer; AHC is
Netty-only and gets a per-loop deadline queue for free.

API compatibility

No exception needed — revapi passes as-is. The change is additive: the existing
TimeoutsHolder constructor is kept and delegates, TimeoutTimerTask gains Runnable
without losing anything, and nothing is removed.

One place where narrowing was avoided on purpose: TimerTask#run declares throws Exception and Runnable#run does not. Rather than re-declaring the abstract method without
the throws clause, which would break an external subclass that declares it, the Runnable
entry point catches and logs.

Tests

EventLoopTimeoutTest asserts the switch itself rather than its side effects: which thread
onThrowable is called on. With the flag on it is one of the client's I/O threads; with it
off it is the timer thread. Both go through a real request against an endpoint that answers
well after the deadline.

Verification

mvnw clean verify — BUILD SUCCESS, 1466 tests, 0 failures, 0 errors, 21 skipped. Error
Prone, NullAway and Revapi all clean.

Caveat on the testing gate: AGENTS.md requires the build to run on JDK 11 and no JDK 11 is
installed on this machine, so it was run on JDK 17 (also in the CI matrix). The JDK 11
leg of CI on this PR is the real gate.

Claude Code on behalf of @pavel-ptashyts

🤖 Generated with Claude Code

A hashed wheel fires on the first tick at or after a deadline, so a
deadline near or below the tick duration is rounded up to it, and one
timer thread carries every expiry for the whole client. Both hurt short
deadlines: a tick is a large fraction of the budget, and a burst of
expiries has no headroom to absorb. Measured over 2000 timeouts armed
as one burst on Netty 4.2.16, a 20 ms deadline overshot by a mean of
2.7 ms and a p99 of 5 ms on a 5 ms wheel, 1.3/2 ms on a 1 ms wheel, and
0/0 ms scheduled on an event loop, which derives its select timeout
from the nearest deadline and so rounds nothing.

Add isUseEventLoopTimeouts(), off by default, which arms the request
and read timeouts on an event loop instead. On the pooled path the
channel is already in hand, so its own loop is used and the timeout
expires on the thread that would have to close it. On the connect path
there is no channel yet, deliberately, so that the timeout also bounds
address resolution and the connect: any loop will do there, since what
the wheel costs is a single thread and a rounded-up tick rather than
the identity of the thread.

Deliberately not a wheel per event loop, which is how the Aerospike
client solves this. A wheel arms in O(1) against O(log n) for a
deadline queue, but at a few thousand timeouts per loop that is a dozen
comparisons, while the quantization it reintroduces costs milliseconds
on a 20 ms budget; it also has to be ticked forever, waking every loop
even with nothing armed. Aerospike wrote its own wheel because its
EventLoop abstracts over NIO, Netty and direct NIO and needed one
timer; AHC is Netty-only and gets a per-loop deadline queue for free.

Arming allocates nothing beyond what the scheduler needs: the
cancellation handle lives on the task, and the existing done flag
stands in for the scheduler's already-expired flag, so no per-timeout
wrapper is required.

Left off by default because the expiry, and therefore whatever the
caller chained onto the response future, then runs on an I/O thread.
Blocking one stalls every connection it serves.

Claude Code on behalf of Pavel Ptashyts

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant