Implement retention namespace generation transitions

[flyingrobots]

Goal

Implement the accepted keep.segment-store/v2 retention and one-way migration campaign tracked by #19 without weakening Keep's exact-byte, fail-closed storage law.

The affected invariant is that retention authority must be generation-checked, canonical, deterministic, bounded, and durable before any reader or writer may treat a version-2 state as admitted. This branch preserves every version-1 segment, catalog, and publication-head byte.

Plan

• Freeze the version-2 format, golden corpus, typed coordinates, and canonical retention and migration records.
• Implement deterministic closure verification, generation transition planning, publication preparation, ordered storage ports, and receipts.
• Implement bounded migration inventory, migration intent and receipt construction, verification-first execution, and filesystem migration authority observation.
• Implement one fresh writer-locked filesystem migration through all 21 ordered phases with exact record and namespace verification.
• Complete production filesystem retention publication and exact idempotent retry behavior.
• Complete restart recovery, immutable reader fencing, and model-based namespace transition evidence.
• Complete migration partial-prefix recovery, corruption coverage, and KEEP-CRASH-053..=073 process-death evidence.

Current status

This is an early Draft PR. Fresh migration now has a production filesystem adapter: it exclusively creates each fixed stage, synchronizes and hard-links it without replacement, retains byte-and-inode identity evidence, admits the version-2 namespace in order on the same filesystem and mount, reopens the complete view before the receipt, and leaves the admitted version-1 head, catalog, and segment bytes unchanged.

That adapter is intentionally a one-shot forward writer, not a restart engine. A process that opens a partial migration prefix still has no lawful continuation implementation, and the current crash matrix covers KEEP-CRASH-001..=035, not the migration boundaries 053..=073. Version 1 therefore remains the only admitted restart-safe production store.

The branch contains 52 commits spanning the #19 campaign and is four commits behind origin/main; no rebase or history rewrite has been performed.

Failure modes

• Stale or mismatched generations refuse before mutation.
• Missing, corrupt, excessive, changed, substituted, or noncanonical closure and migration evidence has typed refusal boundaries.
• Existing stages are never truncated; byte-equal substituted canonical files refuse by identity.
• Unknown or out-of-order version-2 namespace entries refuse before repair.
• Storage failures stop at the named publication or migration phase; partial-prefix recovery remains an explicit nonclaim.

Validation

• cargo fmt --all --check
• cargo check --workspace --all-targets --all-features --locked
• cargo check --workspace --all-targets --no-default-features --locked
• cargo clippy --workspace --all-targets --all-features --locked -- -D warnings
• cargo clippy --workspace --all-targets --no-default-features --locked -- -D warnings
• cargo test --workspace --all-features --locked
• cargo test --workspace --all-features --release --locked
• cargo test --workspace --doc --locked
• cargo +1.96.0 check --workspace --all-targets --all-features --locked
• cargo xtask golden-file-worldline-check
• cargo xtask conformance-check
• cargo xtask source-structure-check
• Debug and optimized cargo xtask durability-crash-matrix
• cargo xtask documentation-refusal-check
• cargo xtask documentation-integrity-check
• cargo deny check
• cargo audit
• Fuzz workspace format, check, and strict clippy gates
• Focused fresh-filesystem migration laws: 8 passed

Compatibility, recovery, performance, and security

• Format/API compatibility: version 2 preserves admitted version-1 immutable bytes; migration is one-way and does not yet authorize production admission.
• Recovery: the fresh forward writer is implemented; partial-prefix classification, continuation, and migration process-death evidence remain open.
• Benchmark impact: no performance claim is made.
• Security: capability-relative, no-follow filesystem admission, same-mount namespace creation, and repeated evidence revalidation are preserved. This work does not claim secure deletion or semantic/causal authority.

Alternatives rejected

• Inferring retention or migration completeness from path existence.
• Re-encoding version-1 immutable data in place.
• Treating a successful fresh execution as restart or crash-recovery evidence.
• Admitting byte-equal canonical substitutes without retained file identity.

Open questions

Peer review is most useful on partial-prefix recovery classification, retention publication ordering, and the immutable reader-fence aperture before those irreversible protocol slices are implemented.

Closes #19 only when every unchecked Plan item and the colocated requirements ledger are complete.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: fad637c2-261c-4b50-b425-db49f0ef5da7

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}