feat(routines): generation fencing, stagger offsets, and reconcile for the Action Scheduler bridge

[chubes4]

Closes #539.

What this adds

Generation fencing. Every WP_Agent_Routine_Action_Scheduler_Bridge::register() mints a generation (wp_generate_uuid4()), persists it in the non-autoloaded agents_routine_generation_<id> option, and stamps it into the scheduled action's args as a trailing metadata element so the logical args (array( 'routine_id' => ... )) remain stable for identity work — Action Scheduler matches args by exact JSON equality, so nothing in the bridge ever queries by stamped args. action_scheduler_stored_action_instance wraps every fetched routine action in WP_Agent_Generation_Fenced_Action, whose execute() no-ops (firing agents_routine_action_fenced) and whose get_schedule() reports a canceled schedule once the stamped generation is no longer current — so a superseded action neither wakes the routine nor spawns a recurrence successor. action_scheduler_stored_action additionally cancels successors stored with a stale generation (the race where an in-flight old-chain action finishes after re-registration and AS's repeat() clones the superseded args). The wake listener resolves the routine by logical args; legacy unstamped actions drain as fenced no-ops instead of double-firing beside their stamped replacements.

Stagger offsets. WP_Agent_Routine accepts stagger => bool|int (default true for interval routines, false for cron expressions, where the expression already is the slot; an int is an explicit max window in seconds). stagger_offset() computes crc32( 'agents_routine_stagger_' . $id ) % min( interval, window ), capped by the new MAX_STAGGER_SECONDS (3600) — deterministic, so re-registration always lands the routine back in the same slot, and co-scheduled interval routines no longer all fire in the same second. The bridge adds the offset to the first-run timestamp.

Reconcile. WP_Agent_Routine_Registry::reconcile( array $opts = [] ) repairs drift between the registry and the AS store: for every registered, non-paused routine it checks pending-action coverage by logical identity (hook + logical args + group) and enqueues a fresh schedule when missing; pending routine actions whose logical routine_id is not registered (or is durably paused) are unscheduled as orphans. Returns array( 'enqueued' => [ids], 'removed' => [ids], 'unchanged' => [ids], 'errors' => [id => message] ); $opts['dry_run'] reports the same shape without writing. Runs are serialized behind an add_option( 'agents_routine_reconcile_lock', time(), '', false ) CAS lock — add_option fails when the row exists; a lock older than 5 minutes is treated as stale and taken over; release happens in finally. To make "non-paused" well-defined across requests, pause state is now durable: the bridge maintains an agents_routine_paused option (a small, generic change to the previously stateless pause contract — the registry itself stays in-memory).

New public surface

• WP_Agent_Routine_Action_Identity — with_generation(), generation_from_args(), logical_args().
• WP_Agent_Generation_Fenced_Action (declared only when \ActionScheduler_Action exists, so the bootstrap can require the file unconditionally).
• WP_Agent_Routine — stagger arg, stagger_offset(), get_stagger_window(), MAX_STAGGER_SECONDS; to_array() round-trips the stagger window.
• WP_Agent_Routine_Action_Scheduler_Bridge — generation_option_name(), current_generation(), is_paused(), pending_routine_actions(), cancel_action_by_id(), register_generation_fence(), fence callbacks fence_stored_action / reconcile_stored_successor; unregister() now deletes the generation tombstone.
• WP_Agent_Routine_Registry — current_generation(), reconcile().
• agents/reconcile-routines ability (src/Routines/register-routine-abilities.php): input { dry_run?: bool }, output the reconcile report, show_in_rest: true, permission current_user_can( 'manage_options' ) filterable via agents_reconcile_routines_permission, annotations destructive: true, idempotent: true.
• Observability action: agents_routine_action_fenced (routine id, stale generation, action).

Testing

tests/routines-durability-smoke.php (registered in composer smoke) runs against a fake in-memory Action Scheduler: the as_* function shims are backed by an action-row store whose fetch_action applies action_scheduler_stored_action_instance and whose saves fire action_scheduler_stored_action, plus minimal ActionScheduler_Action / schedule / store class fakes and a plain-array option layer. 52 assertions cover: the stamp/strip identity round-trip, deterministic stagger slots (including window capping and disabled cases), re-registration advancing the generation and fencing the superseded claimed instance (no callback, fenced event, non-recurring schedule), current-generation execution still firing, stale-vs-current successor reconciliation, reconcile enqueued/unchanged/idempotency, orphan removal, paused routines neither enqueued nor left pending, dry-run writing nothing (no action, no generation advance, no lock), fresh-lock blocking + stale-lock takeover, and unregister teardown. The existing tests/routine-smoke.php passes unmodified — the bridge falls back to as_unschedule_all_actions when the granular store API is absent, which is what its fakes exercise.

Consumer

Extra-Chill/data-machine#3458 is the first consumer — it will register flows and system tasks as Routines and delete its private copies of this machinery.

Docs: the routines section of docs/channels-workflows-operations.md gains fencing, stagger, and reconcile subsections.

AI-generated with Claude Code via Homeboy worktree.

[chubes4]

Read the whole diff, not just the body. This is well-built: fencing sits at the right AS seam (stored_action_instance + successor reconcile), the fenced action returns a CanceledSchedule so AS won't re-spawn a dead chain, logical_args() preserves string keys, reconcile handles paused-as-orphan correctly, dry-run doesn't take the lock, and the layer-purity grep is clean. Approving with one change request and one note.

Blocking — register() is O(all pending routine actions) via unschedule_logical() → pending_routine_actions(), which fetches and hydrates every pending action under the hook/group on every single register() call.

The first consumer has ~700 routines. Bulk registration (activation, reconcile() after an AS prune, a consumer re-syncing on every request through register-routine-bridge-sync.php) becomes 700 × (query + 700 fetch_action()s) ≈ 490k store hydrations. The exact-args as_unschedule_all_actions( hook, $logical_args, group ) path was O(1) but is now only the fallback because stamped args won't exact-match logical args.

Fix without giving up correctness: query by logical identity, not by scanning. AS stores args as JSON in actionscheduler_actions.args (and hashes it in extended_args). Two options, either is fine:

1. Store the generation outside args — e.g. keep args purely logical (['routine_id' => ...]) so as_unschedule_all_actions/as_next_scheduled_action exact-match works, and stamp the generation onto the action via action_scheduler_stored_action → a post-insert update of a dedicated meta/option keyed by action id (agents_routine_action_generation_<action_id>). Fencing then reads by action id in fence_stored_action. Simpler identity, O(1) unschedule, no arg-shape magic.
2. Keep stamped args but add an indexed lookup: pass 'args' => $logical_args + 'partial_args_matching' => 'like' to as_get_scheduled_actions (AS ≥ 3.6 supports it) so the store does the filter.

I'd take (1) — it also removes WP_Agent_Routine_Action_Identity entirely, which is a net simplification for something headed to Core. Either way reconcile() may keep the single bulk scan (that's its job); register() may not.

Non-blocking, worth a follow-up issue: stagger uses crc32(id) % min(interval, window), which is fine, but abs((int) crc32(...)) on 64-bit PHP is a no-op and on 32-bit builds crc32 already returns non-negative — abs is dead. Cosmetic; leave or drop.

Also confirm in the PR body that the reconcile-routines ability permission (manage_options) runs through the same agents_access_permission-style filter the other abilities use so hosts can widen it — I see the annotation but didn't spot the filter call in register-routine-abilities.php.

[chubes4]

Recreated as an in-repo branch (no fork needed) — see the replacement PR linked below.

[chubes4]

Replacement: #541