Fix concurrent ETKDG retry dispatch

[scal444]

An ETKDG worker can temporarily run out of dispatchable work while another
worker still has attempts in flight. The first worker currently treats that
empty dispatch as global completion. If any in-flight attempts subsequently
fail, their retries are never scheduled.

Track in-flight attempts in the scheduler and make production dispatch wait
until those attempts are recorded. Retry rounds now advance only after the
current round has completed. Cancellation wakes waiting workers, and exceptions
from dispatch threads are rethrown by embedMolecules() instead of being
silently discarded.

The scheduler also returns stable, per-molecule attempt IDs. This lets callers
vary retry initialization independently of thread scheduling and batch order.

Fixes #300

[greptile-apps]

Confidence Score: 5/5

The PR appears safe to merge, with no outstanding correctness or repository-rule violations identified.

Summary

• Prevents workers from declaring completion while retry-producing attempts remain in flight.
• Preserves per-molecule conformer and retry limits under concurrent dispatch.
• Adds focused tests for retries, stable attempt IDs, cancellation, and blocking dispatch behavior.

_{Reviews (4) · Last reviewed commit: "Stop ETKDG dispatch scan when batch is f..."}

[evasnow1992]

Changes look good to me. One minor comment but not blocking.