A full apply or rollback force-restarts a thermally-held or manually-stopped rig

[VijitSingh97]

Title

apply / control-apply force-restarts a rig that is deliberately stopped (thermal hold or manual stop)

Summary

_apply_runtime (rigforge.sh) unconditionally runs sudo systemctl restart "$SERVICE_NAME" on
Linux, with no check for whether the miner is supposed to be stopped right now. Every full apply
— a local apply, a full-path control-apply (any change outside the #381 fast-path allowlist:
pools, DONATION, autotune, the watchdog enable/disable flag), and the rollback re-apply that
runs on any control-apply failure (fast-path or full-path) — goes through this same unconditional
restart. It does not know, and does not check, that the miner might currently be intentionally
offline.

Two ways a rig can be intentionally offline today:

• Watchdog thermal hold (watchdog(), opt-in via max_temp_c): when the reading exceeds the
  cutoff, the watchdog stops the service and drops a marker file under the worker root
  (watchdog.thermal-hold) so it knows to leave the miner stopped until the temperature drops back
  5 °C below the cutoff (or the cutoff is removed). The marker exists specifically so the watchdog's
  own health-check runs don't fight themselves.
• A manual stop — an operator runs the equivalent of a service stop by hand, for any reason, with
  no marker at all.

_apply_runtime consults neither. A systemctl restart on a stopped unit starts it — so any apply
that reaches this code path while the rig is thermally held or manually stopped silently overrides
that decision and starts the miner, independent of whether the temperature has actually come back
down or the operator's reason for stopping it still applies.

Worst-case scenario named during #381's security review

An operator raises max_temp_c while the rig is in a thermal hold, intending exactly what the
control path's #257 safety design allows: tune the thermal ceiling without disabling protection. If
that change reaches the full apply path — either because it's bundled with a non-fast-path key, or
because it's a fast-path change whose own success check fails for an unrelated reason and the
rollback re-apply (which always uses the full path) runs — _apply_runtime force-restarts the miner
regardless of the current temperature. The rig can come back up still over the old cutoff, or over
the new one, having bypassed the hold's cool-down entirely.

This is pre-existing on develop today — it is not introduced by #381's fast path. #381 actually
narrows one slice of it: the fast path's own success criterion (added in that same review round) now
explicitly leaves an already-stopped rig stopped instead of treating that as its own failure, so a
fast-path change alone can no longer trigger this. The full path and the rollback re-apply are
unaffected and still carry the hazard described here.

Suggested fix direction (not sized/scoped here — needs its own design pass)

Two directions surfaced during triage, in increasing order of scope:

1. Marker-aware: have _apply_runtime (or apply before calling it) check the watchdog's
   thermal-hold marker and skip the restart — regenerate the config as usual, log that the rig stays
   held, and let the watchdog's own next tick start it once the marker says it's safe. Small, but only
   covers the thermal-hold case, not a manual stop, and couples _apply_runtime to a state file that
   is conceptually the watchdog's own.
2. Run-state preservation (the same idea Control /apply: fast path for restart-free keys (from #344 item 1) #381's fast-path fix uses, generalized): capture whether
   $SERVICE_NAME was active before the apply begins, and restore that same state afterward instead
   of unconditionally restarting — was-active before → restart (today's behavior, unchanged); was NOT
   active before → regenerate the config and leave it stopped (config takes effect next start, whenever
   that is — the watchdog's tick, or the operator's own hand). This covers the manual-stop case too, and
   needs no knowledge of why the rig was stopped.

Direction 2 needs to be checked carefully against every caller of _apply_runtime before it lands:
tune/autotune call it in a loop and actively rely on the miner running to sample hashrate between
trials, so a blanket "preserve prior run-state" change applied indiscriminately could strand a tune
loop instead of a stopped miner. Scoping the fix to the apply/control-apply entry points specifically
(not the tune/autotune callers) is likely the right shape, but that's exactly the kind of judgment call
that belongs in its own issue and PR, not folded into an unrelated change.

Where to look

• _apply_runtime — the unconditional sudo systemctl restart on Linux.
• watchdog() — the thermal-hold stop/marker/resume logic (watchdog.thermal-hold).
• _control_do_apply / control_apply — the full control-apply path and its rollback re-apply, both
  of which call apply → _apply_runtime.