Add an opt-in virtual threads flag for blocking I/O thread pools on Java 25

[GGraziadei]

What is the purpose of the change

Now that master requires Java 25, this change lets operators opt in to Java virtual threads for Storm's blocking, I/O-bound thread pools through a new cluster setting, storm.virtual.threads.enabled (default false). With the flag off nothing changes apart from thread names.

A new helper, org.apache.storm.utils.StormThreadFactory, returns a virtual-thread ThreadFactory when the flag is on and a non-daemon, normal-priority platform-thread factory otherwise. The factory is injected into the existing pools, so core sizes, queues, rejection policy and error handling are untouched and the *.threads settings keep their meaning as a concurrency bound:

• Thrift server handlers for the SASL, TLS and simple transports (Nimbus, Supervisor and DRPC). With the simple transport an executor is now always supplied when the flag is on; without a configured queue size it uses the same unbounded queue THsHaServer builds by default.
• AsyncLocalizer download and task executors.
• Nimbus AssignmentDistributionService and Supervisor heartbeat pools.
• DRPCSpout background executor.
• The worker shared executor exposed through TopologyContext, read from the merged topology conf so a topology can override the flag.

Spout/bolt executor threads, worker transfer, JCQueue, Netty event loops and timers are deliberately left on platform threads: they are busy-poll hot loops that would lose throughput on virtual threads.

The javadoc of the new key documents the operational caveats: all virtual threads share one carrier scheduler sized to the core count, blocking file I/O (blob downloads) occupies a carrier, and with the simple transport and no configured queue size the effective handler concurrency goes from THsHaServer's fallback of 5 to *.threads.

What the benchmark shows

ThriftHandlerVirtualThreadsBench (added to examples/storm-perf) runs an in-process Nimbus Thrift server whose handler simulates blocking I/O and hammers it with N clients, each mode in its own JVM. On a 20-core machine:

clients / *.threads / io-ms	flag	calls/s	p50 ms	p99 ms	server OS threads	RSS after
200 / 64 / 10	off	6308	30.6	37.7	64	186 MB
200 / 64 / 10	on	6105	30.7	41.0	~23	175 MB
500 / 512 / 20	off	21401	21.0	32.1	512	268 MB
500 / 512 / 20	on	19946	22.5	35.4	~23	290 MB
2000 / 2000 / 50	off	28525	63.4	88.6	2000	531 MB
2000 / 2000 / 50	on	27257	61.2	121.9	~23	424 MB

At equal *.threads the throughput is unchanged (within a few percent) because the workload is bounded by the configured concurrency either way. The gain is in resources: server-side OS threads drop from *.threads to roughly the number of cores, and RSS drops at high concurrency. The tail latency grows slightly at very high concurrency. In short, this is a resource saving on the control plane that lets operators raise nimbus.thrift.threads / supervisor.thrift.threads to absorb bursts without paying for native threads; with the shipped defaults it is neutral, and it is off by default.

How was the change tested

• New unit tests: StormThreadFactoryTest (flag on/off/missing/unexpected type, thread naming, forced NORM_PRIORITY), WorkerStateTest (shared executor on virtual vs platform threads), AsyncLocalizerTest (download executor on virtual vs platform threads).
• New round-trip tests in AuthTest through a real ThriftServer and NimbusClient, asserting from inside the Nimbus handler that it runs on a virtual thread when the flag is on (simple transport with and without a queue size, digest SASL transport) and on a platform thread by default; the digest case also checks ReqContext still carries the principal on a virtual handler thread.
• Full storm-client and storm-server suites pass with the flag off (default): 650 and 487 tests respectively, 0 failures.
• Smoke test with the flag forced on for the whole JVM via -Dstorm.options=storm.virtual.threads.enabled=true: LocalNimbusTest passes and the "platform thread by default" assertion in AuthTest fails as expected, proving the flag reaches the handler pool.
• Benchmark runs above.