CAS improvements

docs/en/antalya/cas/architecture/backend.md

@@ -108,15 +108,17 @@ storage, and GC would silently stop reclaiming.
 `runCapabilityProbe` (`Backend/CasProbe.cpp`) runs a throwaway-key battery against every writable
 mount, described in full on the [bucket requirements](/antalya/cas/bucket-requirements) page. It is
 fail-closed: any check that does not pass throws `NOT_IMPLEMENTED` naming the specific failure, and
-the mount refuses to become writable. Two further gates run as the battery's opening steps, and one
+the mount refuses to become writable. The one tolerated exception is a versioning probe that cannot
+answer at all, described in the first bullet below. Two further gates run as the battery's opening steps, and one
 sits genuinely alongside it. The distinction matters: because the versioning check runs *inside* the
 battery, skipping the battery used to skip it too, which is exactly why the third gate exists.
 
-- `checkPoolPreconditions` — inside the battery. On the `GCS`-dialect combination only, requires bucket versioning to be
-  *verifiably* off. A confirmed `Enabled` and an inconclusive probe both throw: `CAS` cannot assume
-  the safe answer here, because what it would do on a versioned bucket is delete objects it believes
-  it reclaimed. A probe is inconclusive when the credential may not read the bucket's versioning
-  configuration, or when the backend cannot answer at all.
+- `checkPoolPreconditions` — inside the battery. On the `GCS`-dialect combination only, checks that
+  bucket versioning is off. A confirmed `Enabled` throws: what `CAS` would do on a versioned bucket is
+  delete objects it believes it reclaimed. An inconclusive probe — the credential may not read the
+  bucket's versioning configuration, or the backend cannot answer at all — logs a warning and lets
+  the mount proceed, since it is not evidence of a versioned bucket; verifying it then falls to the
+  operator, as it already does for soft delete.
 - `checkSkipAccessCheckSupport` — alongside the battery, in the skip branch of `Pool::open`, since it
   is the gate that decides whether the battery may be skipped at all. It asks whether the backend may serve a writable mount that skips the
   battery at all. The `GCS`-dialect combination refuses, so `skip_access_check = true` cannot reach a

docs/en/antalya/cas/architecture/garbage-collection.md

@@ -226,6 +226,7 @@ the user-facing configuration surface.
 | Setting | Default | Bounds |
 |---|---|---|
 | `cas_gc_meta_pool_size` | 16 | bounded pool for condemn-marker writes |
+| `cas_gc_read_concurrency` | 16 | bounded pool for the fold's read-ahead; `1` disables |
 
 ## Observability {#observability}
 

docs/en/antalya/cas/architecture/manifests-and-refs.md

@@ -101,7 +101,7 @@ swept for that root.
 flowchart TD
     A["LIST one page of cas/manifests/<br/>freeze candidates with exact GET"] --> B{"build-prefix eligible?<br/>durable watermark fact only"}
     B -->|"epoch less than lease epoch"| ELIG["eligible, old-epoch debris"]
-    B -->|"same epoch, min_active clears build_seq"| ELIG
+    B -->|"same epoch, min_active_build_sequence clears build_seq"| ELIG
     B -->|"no lease, or epoch ahead, or build may be live"| SKIP["skip"]
     ELIG --> C["protection view: committed manifests<br/>plus live precommits<br/>plus manifests with an unfolded minus-one"]
     C -->|"key protected"| SKIP2["skip"]

docs/en/antalya/cas/architecture/mounts-and-leases.md

@@ -9,7 +9,7 @@ doc_type: 'reference'
 
 Page 4 of 4 in the CAS architecture set. Covers server identity, the mount lease that fences
 writers, and the server-scoped control-plane objects. No external coordinator is involved: there
-is no ZooKeeper/Keeper client anywhere in this protocol — `MountLeaseKeeper` is a local lease
+is no ZooKeeper/Keeper client anywhere in this protocol — `MountLeaseRenewer` is a local lease
 *renewer*, not a Keeper client.
 
 ## `cas_server_root_id` — the identity {#server-root-id}
@@ -61,51 +61,55 @@ Two failure modes this closes:
   over, regardless of lease expiry.
 - A **same-uuid live twin** (two processes sharing one uuid file and `server_root_id`) is caught separately, by
   the mount claim's token-stability observation, and aborts with an operator-facing message rather
-  than corrupting the pool.
+  than corrupting the pool — this is the default behavior, with `cas_unsafe_remount_no_delay` off.
+  With it on, a same-uuid claim over such a slot reclaims at once instead of observing (see
+  `cas_unsafe_remount_no_delay` in the configuration reference).
 
 ## The mount lease {#mount-lease}
 
 One object, `gc/server-roots/<server_root_id>/mount`, carries **both** the liveness lease and the build
 watermark — there is no separate watermark object. `MountLease` fields: `server_uuid`,
 `writer_epoch`, `write_attempt_id`, `hostname`, `pid`, `started_at_ms`, renewal `seq`,
-`expires_at_ms`, `min_active` (the build-watermark floor), and `gc_fenced`.
+`expires_at_ms`, `min_active_build_sequence` (the build-watermark floor), and `gc_fenced`.
 
 - **Logical renewal identity.** Each holder-originated body has a fresh nonzero
   `write_attempt_id`. One logical renewal fixes one immutable `(key, bytes, expected token,
   write_attempt_id)` tuple before I/O. Every physical retry repeats it byte-for-byte; a later GC
   fence preserves the observed ID, while reclaim and successor bodies mint new IDs.
 - **Resolve before retry.** A transient or ambiguous conditional `PUT` is followed by one exact
-  `GET`. The keeper adopts the result only when the complete body, including `write_attempt_id`,
+  `GET`, except that an attempt whose transport error names a failed connection is reissued first
+  after a flat pause and settled by the reissue's own answer (a 2xx) or by the exact `GET` that
+  follows its `412`. The renewer adopts the result only when the complete body, including `write_attempt_id`,
   equals its immutable request. If the predecessor token is still current, another identical `PUT`
   may follow bounded backoff. A same-pair twin, GC-fenced body, successor, foreign holder, or absent
   body is never treated as this renewal.
 - **Absolute deadline.** Renewal uses `CLOCK_BOOTTIME`, not `CLOCK_MONOTONIC`, so a VM resumed from
   suspend correctly observes itself expired. Its absolute deadline is the minimum of the existing
   request-operation budget and the last confirmed lease deadline minus the safety margin. The
-  controller checks that one configured attempt still fits before each backend `PUT` or resolving
+  controller checks that one attempt envelope still fits before each backend `PUT` or resolving
   `GET`, after each interruptible backoff, and before accepting success. A retry, `GET`, response
   timestamp, or wall-clock step never extends authority.
-- **Cadence.** The runtime normally starts a logical renewal every `mount_renew_period` (default
-  10 s), with TTL `mount_lease_ttl_ms` (default 30 s, TTL/3 renewal ratio). The next beat is anchored
+- **Cadence.** The runtime normally starts a logical renewal every `cas_mount_renew_period_ms` (default
+  10 s), with TTL `cas_mount_lease_ttl_ms` (default 30 s, TTL/3 renewal ratio). The next beat is anchored
   at the committed body's pre-I/O BOOTTIME start. A slow recovery therefore causes an immediate
   catch-up beat when the nominal cadence has elapsed; it does not wait a fresh full period after the
   response.
 - **Per-write recheck.** Every durable write or delete captures the fence generation at admission
   and rechecks it immediately before the object-store call and on every conditional retry. Reads
   are not gated.
 - **Request-budget admission.** `refAppendFenceOk` refuses to *start* a ref-log attempt unless
-  `attempt_timeout + safety_margin` fits inside the remaining lease, rejecting with
-  `BAD_ARGUMENTS` at request-admission time rather than mid-flight.
+  `2 × envelope + safety_margin` fits inside the remaining lease (a write and its settlement read),
+  rejecting with `BAD_ARGUMENTS` at request-admission time rather than mid-flight.
 
-**Losing the lease is neither read-only mode nor a process abort.** `MountLeaseKeeper` is a
+**Losing the lease is neither read-only mode nor a process abort.** `MountLeaseRenewer` is a
 synchronous durable-slot state machine. A committed result advances its token, sequence, confirmed
 BOOTTIME deadline, and cadence anchor. Any admitted deterministic failure, confirmed conflict, or
 ambiguity left at the deadline/attempt limit moves it to `RenewalTerminal`; it cannot mint another
 body or publish a clean farewell. Owner cancellation before any request is the only
 `NotAttempted` result and leaves clean release possible. Cancellation after a request was sent is
 terminal because that request may still land.
 
-After the keeper call returns, `CasMountRuntime` consumes the result. A terminal result trips the
+After the renewer call returns, `CasMountRuntime` consumes the result. A terminal result trips the
 local fence (latches `lost`, bumps the fence generation, moves the in-process runtime to
 `TransientNotLive`) and latches one self-remount generation. A confirmed foreign/successor or
 same-pair conflict remains a typed fail-closed error; it is never adopted. A real fence still costs
@@ -115,9 +119,25 @@ inside authority already proved by the last confirmed lease.
 
 GC's own view of a dead server is symmetric and clock-skew-immune: a slot becomes fence-eligible
 only after the leader observes the *same* renewal token hold stable, on its own monotonic clock,
-for `TTL + TTL/20 + cadence` — the identical formula a re-mounting server uses to wait out a
-predecessor. The stamped `expires_at_ms` never participates in that decision; wall-clock `now` is
-audit-only.
+for `TTL + floor(TTL/20) + period` — close to, but not identical to, the threshold a re-mounting
+server uses to wait out a predecessor, which observes `TTL + floor(TTL/20) + max(1,
+floor(period/2))`. Both thresholds are evaluated purely on the observer's own clock and its own
+configured `TTL`/`period`; nothing about the writer's timing travels on the wire. The stamped
+`expires_at_ms` never participates in either decision — it is a writer-stamped diagnostic used by
+`system.cas_mounts` and by the non-authoritative decommission epoch-recovery precheck, never an
+authorization; local fencing is derived instead from the confirmed request's pre-I/O `BOOTTIME`
+anchor plus the TTL, and wall-clock `now` stays audit-only.
+
+Every server sharing a pool must therefore run the identical `cas_mount_lease_ttl_ms` and
+`cas_mount_renew_period_ms`: a member or GC leader configured with a shorter threshold than its
+peers can fence out a healthy peer whose token-update gap merely exceeds that shorter threshold —
+a peer renewing frequently stays live, one that missed a renewal does not. Change these values only
+with every member of the pool stopped; a graceful restart removes only that member's own startup
+observation and does not make mixed thresholds safe. With the defaults (TTL 30 s, period 10 s,
+margin 2 s), `TTL − margin − period − 2 × envelope = 4 s` is the scheduling-lateness budget before
+the first renewal attempt of a period can begin, where `envelope = attempt_timeout + 2 × cap` and
+`cap` is `attempt_timeout` when the disk's `connect_timeout_ms` is `0`, else
+`min(connect_timeout_ms, attempt_timeout)` (7 s with defaults).
 
 ## The two monotone counters {#counters}
 
@@ -133,9 +153,9 @@ into "not found".
 
 Global build ordering is the **pair** `(writer_epoch, build_seq)` compared lexicographically — the
 exact comparison GC uses for eligibility. The durable authority for both is the mount object
-itself: no mount means no deletion authority means nothing is swept. `min_active`, the oldest
+itself: no mount means no deletion authority means nothing is swept. `min_active_build_sequence`, the oldest
 in-flight `build_seq`, rides in the same mount object as the watermark floor; `UINT64_MAX` in
-`min_active` is the farewell/retired sentinel, not a real build.
+`min_active_build_sequence` is the farewell/retired sentinel, not a real build.
 
 ## Mount claim outcomes {#claim-outcomes}
 
@@ -153,9 +173,10 @@ a `MountClaimResult::Kind` together with a `MountPriorState` describing which ce
 | `MountPriorState` | Certificate that justified the reclaim |
 |---|---|
 | `None` | no reclaim needed (fresh claim or same-epoch refresh) |
-| `Clean` | the predecessor's own graceful farewell (`min_active == UINT64_MAX`) |
+| `Clean` | the predecessor's own graceful farewell (`min_active_build_sequence == UINT64_MAX`) |
 | `Fenced` | GC's own threshold-gated fence-out (`gc_fenced`) |
 | `UncleanObserved` | this claimant's own token-stability observation held for the full `TTL + drift` window |
+| `UncleanUnsafe` | the operator's explicit `cas_unsafe_remount_no_delay` authorization carried the slot's exact token — not a certificate of death |
 
 ## Behavioral mount-slot model {#mount-state-machines}
 
@@ -166,12 +187,13 @@ the claim outcomes above and is shown here as behavior, not as a type in the cod
 stateDiagram-v2
     [*] --> Absent
     Absent --> Live: claimMount putIfAbsent, seq=1
-    Live --> Live: keeper beat, putOverwrite seq+1
+    Live --> Live: renewer beat, putOverwrite seq+1
     Live --> Fenced: GC observes a stable token past threshold, gc_fenced=1, body preserved
-    Live --> Terminated: certified drain, terminal farewell (expires_at=now, min_active=MAX)
+    Live --> Terminated: certified drain, terminal farewell (expires_at=now, min_active_build_sequence=MAX)
     Fenced --> Live: same-uuid claim with a fresh writer_epoch, instant reclaim
     Terminated --> Live: same-uuid claim with a fresh writer_epoch, instant reclaim
     Live --> Live: same-uuid claim, proven-dead token via UncleanObserved
+    Live --> Live: same-uuid claim under cas_unsafe_remount_no_delay, no observation
     Fenced --> Fenced: same uuid and epoch claim, FencedSelf, no write
     Live --> Absent: decommission tail, mount then epoch then owner tombstone
     Terminated --> [*]
@@ -202,24 +224,24 @@ under a live mount is an operator-level event.
 
 **Writable open** runs in a strict order: bootstrap-residual proof, capability probe under a
 random per-mount prefix, pool-meta create-or-validate, `validateServerRootId`, owner claim,
-`allocateWriterEpoch`, mount claim and synchronous keeper start, materialization grace if the
-predecessor was unclean (default 30 s), arm the fence, then create and release the runtime-owned
-renewal and remount workers before the writable pool becomes externally visible. If the grace period
-consumed the TTL, one fresh synchronous renewal re-anchors the deadline before the fence is armed.
+`allocateWriterEpoch`, mount claim and synchronous renewer start, arm the fence, then create and
+release the runtime-owned renewal and remount workers before the writable pool becomes externally
+visible. If the claim consumed the TTL, one fresh synchronous renewal re-anchors the deadline
+before the fence is armed.
 Failure to construct either worker joins the partial pair, closes the fence, and fails the writable
 open. No incident path constructs a thread.
 
 The renewal and remount workers are separate and long-lived under one stable `CasMountRuntime`.
 `scheduleRemount` increments a requested-generation latch and wakes the persistent remount worker,
-including while an older generation is active. Before keeper replacement, remount requests
-`ParkRequested` and waits for the renewal driver to report `Parked`, which proves that no keeper call
+including while an older generation is active. Before renewer replacement, remount requests
+`ParkRequested` and waits for the renewal driver to report `Parked`, which proves that no renewer call
 is in flight. A successful remount handles only its snapshotted generation; a newer request is
 processed before renewal resumes.
 
 **Clean unmount:** request stop and join both persistent workers, drain the ref lanes, and only if
-the drain *certified* quiescence call `MountLeaseKeeper::release` on an `Active` keeper to write the
-terminal farewell (`expires_at_ms` already expired, `min_active = UINT64_MAX`). That sentinel is what
-lets a successor reclaim instantly. A `RenewalTerminal` keeper, an unresolved ref write, or a sent
+the drain *certified* quiescence call `MountLeaseRenewer::release` on an `Active` renewer to write the
+terminal farewell (`expires_at_ms` already expired, `min_active_build_sequence = UINT64_MAX`). That sentinel is what
+lets a successor reclaim instantly. A `RenewalTerminal` renewer, an unresolved ref write, or a sent
 renewal ambiguity writes no farewell — an unearned farewell would let a successor start mutating
 while a stale conditional request from the predecessor is still in flight.