diff --git a/.skills/context-counts-ok b/.skills/context-counts-ok new file mode 100644 index 0000000..5c4b7af --- /dev/null +++ b/.skills/context-counts-ok @@ -0,0 +1,25 @@ +# Counts judged legitimate — check-counts.sh --ack-file default. +# First pass, 2026-09-09: this check post-dates the repo's last curation, so +# every entry below is an existing count read for the first time. Two counts +# were dropped rather than warranted (form 2, the policy-file default): the +# "three ways a skew has already failed" now reads "the ways", and the logging +# stack's "one formatter, two installers" now reads "its formatter, its +# installers" — both numbers were rhetorical and neither is checkable here. +# +# --- enumerated: the counted things are named in the same clause or the list +# --- immediately under it, so the number is checkable by reading the section. +enumerated :: **two backends** (`local`, `gcs`) +enumerated :: Three that bite, each symptomless until it matters: +enumerated :: Two env files, and the boundary between them +enumerated :: **Two blob backends, one seam.** +enumerated :: the worker retries the two it meets at runtime +enumerated :: **Three stream kinds, three sets of rules.** +# The three contracts and their four documents are the four `docs/contracts/` +# rows in ## Detail Docs, one line down the same file. +enumerated :: **Three normative contracts bound the wire and the roadmap** +enumerated :: four documents under `docs/contracts/` +# +# --- stable: structural, and cannot change without the sentence being rewritten. +# XAUTOCLAIM's reply shape is Redis wire format, and the ≥7.0 floor beside it is +# what the sentence exists to state; a fourth element would rewrite both. +stable :: `XAUTOCLAIM`'s three-element reply diff --git a/.skills/context-loss-ok b/.skills/context-loss-ok index 474baaa..7028143 100644 --- a/.skills/context-loss-ok +++ b/.skills/context-loss-ok @@ -21,7 +21,6 @@ duplicate :: `/etc/systemd/system/`** — the installed unit is a copy, not a sy duplicate :: `daemon-reload` alone silently re-reads the old file and the mismatch has no duplicate :: symptom until a directive matters. **Refuses to start off `main`, or off duplicate :: unpushed commits** (#37, #48) — `scripts/check_main_checkout.sh`; -duplicate :: `REPLICATOR_ALLOW_ANY_CHECKOUT=1` overrides; a dev worker asks the same duplicate :: question at the writer (#52). retarget :: Full lifecycle table and the dev-server invocation: @@ -58,4 +57,94 @@ disproven :: loop (CannObserv/watcher#275). Every `BlobStore` call from a corout # --- The style bullets -> docs/STYLE.md ------------------------------------- # The block moved verbatim; this bold pseudo-heading became the `## General` # heading of its own section, which is a heading rename the move forced. -rename :: **General:** + +# --- The 2026-09-09 curation (AGENTS.md 6,747 -> 5,969 exact tokens). --- +# Two shapes only. Every `tighten` line is a class-C rewrite whose claims all survive: +# prove-no-loss.sh --claims reports claims_dropped: 0, so no backticked identifier, +# issue reference or link target of the base file was dropped -- only words. The +# reasoning each rewrite stopped repeating was read in its destination first: +# docs/CONVENTIONS.md (OOM, ACL, the three fates, the dedupe keys), docs/STORAGE.md +# (the backend seam, to_thread inside the shutdown budget), docs/STREAMS.md (the +# compiled-in `local` default), docs/DEPLOYMENT.md (guard verdicts, the co-core pin). +# Some entries are re-wrapped fragments rather than whole claims -- a rewrite moves +# line boundaries, and whole-line matching cannot see the text survived. +# +# The `duplicate` five are the 'Where the reasoning lives' mini-index, which listed +# four docs the ## Detail Docs index below it already lists with fuller one-liners. +tighten :: Python ≥3.12, uv, pytest, ruff. `ty` is available as a **non-gating** type checker (`uv run ty check`) — advisory only; no pre-commit or CI gate. +tighten :: Auth is ADC. Pin the current minor — `>=0.13.1,<0.14` — and raise the **patch** floor with every co-core feature the code starts depending on: the reasoning, and the three ways a skew has already failed, in [docs/DEPLOYMENT.md](docs/DEPLOYMENT.md). +tighten :: **The manifest is a source, not the artifact.** Nothing re-embeds it — no hook, no CI step — so editing a `description` there changes what the repo says and not what `codebase_context_search` returns. Re-run `codebase_context_index` in the same change, or the highest-authority answer an agent gets stays the stale one (#19 CR #17). +tighten :: the `BlobStore` protocol — **two backends now** (`local`, `gcs`), selected by +tighten :: `REPLICATOR_BLOB_BACKEND` and defaulting to `local` (#7); `src/api/` is the dev-only +tighten :: `/health` app; `src/core/` holds config, logging, and the consume path's failure +tighten :: vocabulary. `tests/` mirrors +tighten :: `src/`. Every module with the job it owns: +tighten :: **Single-VM setup.** Code committed to main is the deployed code. Replicator shares the VM with archiver, watcher, and notifier. +tighten :: The worker binds no port; 8041 is the dev API port and 8040 is reserved. **Redis +tighten :: is Archiver-operated** — Replicator is a client, never ships a broker, never +tighten :: claims ownership — and server **≥ 7.0** is Replicator-critical because +tighten :: `claim_stale` reads `XAUTOCLAIM`'s three-element reply. `scripts/check_redis_floor.sh` +tighten :: guards it as an `ExecStartPre`. Ports, neighbours, and the redis-py pin: +tighten :: Three things that bite, each of which has no symptom until it matters: +tighten :: Every deploy situation with its command, the guard's full verdict table, and the +tighten :: dev-server invocation: [docs/DEPLOYMENT.md](docs/DEPLOYMENT.md). +tighten :: Every variable, which file carries it, and the reasoning behind each default: +tighten :: Replicator is a **consumer** first. Follow the conventions co-core and the archiver producer established: +tighten :: `command_id:occurred_at`), so nothing an issuer waits on can collapse — storage +tighten :: identity and correlation identity are not interchangeable. `info_source_id` +tighten :: rides both and is **echoed, never read**, as is replicate's +tighten :: `info_item_rep_spec_id`: each `test_boundaries.py` carve-out is one field +tighten :: wide, and adding one edits the charter (#28, #29). +tighten :: `gs://`; `local` is the compiled-in default **by decision, not by schedule** — +tighten :: the deployment flipped on 2026-08-20 and the default did not. Every `BlobStore` +tighten :: call from a coroutine goes through `asyncio.to_thread` — which puts it in the +tighten :: unit's shutdown budget, not just the handler's. Per-backend retention, ceilings +tighten :: and failure classification: [docs/STORAGE.md](docs/STORAGE.md), which is the +tighten :: authority and worth reading before touching either store. +tighten :: - **Store, then publish — never the reverse.** A fact pointing at bytes that are +tighten :: not there is unrepairable by the consumer; stored bytes with no fact repair +tighten :: themselves on the reclaim. +tighten :: *before* it — as `XADD .dlq` then `XACK`, which is the form broker#2's +tighten :: ACL grants and is now observed rather than inferred (#79). Retry cadence is +tighten :: `REPLICATOR_CLAIM_MIN_IDLE_MS`; a failing *cycle* is `run_loop`'s problem, not +tighten :: the message's. +tighten :: ceiling, the consume path keeps reading, acking and reclaiming throughout, and +tighten :: nothing is dropped or dead-lettered. The third, `XGROUP CREATE … MKSTREAM`, is +tighten :: boot-only and does **not** retry: `ensure_group` re-raises anything but +tighten :: `BUSYGROUP`, so a first boot against a capped broker exits and systemd +tighten :: restarts. Verified against a scratch broker this repo spawns, never the shared +tighten :: one. Never answer an OOM with a client-level retry — a re-sent `XADD` the +tighten :: broker already applied publishes twice. +tighten :: `fetch_failed(handler_error)`. The cost is deliberate: a grant nobody fixes +tighten :: retries forever rather than reaching `.dlq`. +tighten :: handler — so losing them costs one TTL window of re-fetches and never +tighten :: correctness. Endorsed as bus state rather than a role blur, with broker#1's four +tighten :: answers and the ACL grant they imply, in +tighten :: - **The replicate loop writes for `gcs` (#29)** — create-if-absent, `blob_uri` +tighten :: never resolved as a path, writers keyed by alias, refusals before credentials, +tighten :: provider failures classified by HTTP status. Read +tighten :: [docs/CONVENTIONS.md](docs/CONVENTIONS.md) before touching that path. +tighten :: requires `--production` for the one combination the live worker consumes — a +tighten :: frame there is fetched for real. +tighten :: - **Three normative contracts bound the wire and the roadmap** — four documents, +tighten :: all under `docs/contracts/`, linked from sibling repos and indexed below. +tighten :: `tests/test_boundaries.py` enforces the charter in CI; change a +tighten :: charter and its tests together. +duplicate :: Where the reasoning lives: +duplicate :: - What each stream carries — [docs/STREAMS.md](docs/STREAMS.md) +duplicate :: - The rules common to all of them, and the `replicator:cmd:*` keyspace — [docs/CONVENTIONS.md](docs/CONVENTIONS.md) +duplicate :: - Blob paths, modes, and the retention sweep — [docs/STORAGE.md](docs/STORAGE.md) +duplicate :: - Fakeredis's divergences, the keys an integration run may touch, and why production `co-gcs-replication` is refused from every test — [docs/TESTING.md](docs/TESTING.md) +tighten :: The logging stack — one formatter, two installers, and the journald lines that +tighten :: are deliberately not JSON: [docs/STYLE.md](docs/STYLE.md). +tighten :: classes and functions, small focused functions, and tests mirroring source — +tighten :: each with its rationale and its ruff gate in [docs/STYLE.md](docs/STYLE.md). +tighten :: - [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) — founding design and the module-by-module layout; read before changing one +tighten :: - [docs/CONVENTIONS.md](docs/CONVENTIONS.md) — the co-core/Redis Streams rules common to every stream: idempotency, validation, DLQ, `claim_stale`; and the `replicator:cmd:*` keys, this service's only non-stream footprint (#80) +tighten :: - [docs/STORAGE.md](docs/STORAGE.md) — blob paths and modes, the three populations under `REPLICATOR_BLOB_DIR`, TTL and ceiling semantics +tighten :: - [docs/DEPLOYMENT.md](docs/DEPLOYMENT.md) — VM topology, ports, the systemd unit's lifecycle, and the co-core pin +tighten :: - [docs/TESTING.md](docs/TESTING.md) — where fakeredis diverges from the live broker, which keys an integration run may create, and why production `co-gcs-replication` is unreachable from every test (#38) +tighten :: - [docs/SKILLS.md](docs/SKILLS.md) — vendored skill inventory, refresh procedure, and the doc-check sensitive-path list +tighten :: - [docs/SOCRATICODE.md](docs/SOCRATICODE.md) — the full tool table, the prefetch query, per-tool gotchas, and cross-repo search +tighten :: - [docs/contracts/content-fetch-issuer-reference.md](docs/contracts/content-fetch-issuer-reference.md) — its lookup half: the refusal list, the failure taxonomy, the silent conditions, trust posture +tighten :: - [docs/contracts/content-replicate-issuer-contract.md](docs/contracts/content-replicate-issuer-contract.md) — the replicate trust model and issuer obligations, settled ahead of the code (#34) diff --git a/.skills/context-metrics.jsonl b/.skills/context-metrics.jsonl index c7015a9..9758e2d 100644 --- a/.skills/context-metrics.jsonl +++ b/.skills/context-metrics.jsonl @@ -13,3 +13,5 @@ {"actions": ["baseline:scheduled"], "budget": 6000, "bytes": 14840, "delta_days": 5, "delta_tokens": 0, "docs_orphaned": 0, "docs_total": 15, "file": "AGENTS.md", "lines": 265, "links_dead": 0, "links_dead_anchors": 0, "no_loss": null, "no_loss_warrants": null, "note": null, "over_budget": false, "repo": "replicator", "repo_commit": "1330c82", "seams": 0, "seams_acked": 28, "skill_commit": "d710691", "skill_version": "1.11", "tokens": 5985, "tokens_exact": true, "tokens_live": 104362, "top_section": "Bus Conventions", "top_section_share": 25, "ts": "2026-08-27"} {"actions": ["baseline:scheduled"], "budget": 6000, "bytes": 14876, "delta_days": 5, "delta_tokens": 14, "docs_orphaned": 0, "docs_total": 15, "file": "AGENTS.md", "lines": 265, "links_dead": 0, "links_dead_anchors": 0, "no_loss": null, "no_loss_warrants": null, "note": null, "over_budget": false, "repo": "replicator", "repo_commit": "214df18", "seams": 3, "seams_acked": 28, "skill_commit": "662de71", "skill_version": "1.12", "tokens": 5999, "tokens_exact": true, "tokens_live": 105677, "top_section": "Bus Conventions", "top_section_share": 25, "ts": "2026-09-01"} {"actions": ["baseline:scheduled"], "budget": 6000, "bytes": 14878, "claims_dropped": null, "claims_warranted": null, "counts": null, "counts_acked": null, "delta_days": 7, "delta_tokens": 2, "docs_orphaned": 0, "docs_total": 15, "file": "AGENTS.md", "lines": 265, "links_dead": 0, "links_dead_anchors": 0, "no_loss": null, "no_loss_warrants": null, "note": null, "over_budget": true, "repo": "replicator", "repo_commit": "c3f0e31", "seams": 3, "seams_acked": 28, "skill_commit": "f603abf", "skill_version": "1.16", "tokens": 6001, "tokens_exact": true, "tokens_live": 109141, "top_section": "Bus Conventions", "top_section_share": 25, "ts": "2026-09-08"} +{"actions": ["baseline:pre-curation"], "budget": 6000, "bytes": 16709, "claims_dropped": null, "claims_warranted": null, "counts": null, "counts_acked": null, "delta_days": 1, "delta_tokens": 746, "docs_orphaned": 0, "docs_total": 15, "file": "AGENTS.md", "lines": 289, "links_dead": 0, "links_dead_anchors": 0, "no_loss": null, "no_loss_warrants": null, "note": null, "over_budget": true, "repo": "replicator", "repo_commit": "8d63a77", "seams": null, "seams_acked": null, "skill_commit": "798803e", "skill_version": "1.16", "tokens": 6747, "tokens_exact": true, "tokens_live": 116261, "top_section": "Bus Conventions", "top_section_share": 33, "ts": "2026-09-09"} +{"actions": ["tighten:Bus Conventions", "tighten:Detail Docs", "demote:Project Overview", "prune:Bus Conventions mini-index", "fix:count-precision"], "budget": 6000, "bytes": 14508, "claims_dropped": 0, "claims_warranted": 0, "counts": 0, "counts_acked": 9, "delta_days": 0, "delta_tokens": -759, "docs_orphaned": 0, "docs_total": 15, "file": "AGENTS.md", "lines": 263, "links_dead": 0, "links_dead_anchors": 0, "no_loss": "ok", "no_loss_warrants": 77, "note": "rewritten in place after CR 13-20: eight review fixes on top of the curation, measured on the tree that ships", "over_budget": false, "repo": "replicator", "repo_commit": "3aaa8cb", "seams": 0, "seams_acked": 32, "skill_commit": "798803e", "skill_version": "1.16", "tokens": 5988, "tokens_exact": true, "tokens_live": 115702, "top_section": "Bus Conventions", "top_section_share": 30, "ts": "2026-09-09"} diff --git a/.skills/context-seams-ok b/.skills/context-seams-ok index 724f520..15795d9 100644 --- a/.skills/context-seams-ok +++ b/.skills/context-seams-ok @@ -61,7 +61,6 @@ docs/ARCHITECTURE.md :: What each stream carries, and the reasoning behind each docs/DEPLOYMENT.md :: The GCS test bucket — the opposite grant, on purpose # The GCS test bucket stayed in DEPLOYMENT.md's Infrastructure section through the # #67 split — this pointer still resolves to what it names. -docs/TESTING.md :: The bucket and the SA are provisioned (#50) # 2026-08-19, #67 CR #14. The new ENVIRONMENT.md preamble points back at # DEPLOYMENT.md for the lifecycle and the unit — neither moved in this split. docs/ENVIRONMENT.md :: and the unit itself are in [DEPLOYMENT.md](DEPLOYMENT.md). @@ -74,3 +73,17 @@ docs/SKILLS.md :: case, which is why the rule in `AGENTS.md` → `## Code Explor # Provenance heading from #69, unrelated to this run and the same shape as the # ten provenance-headings already judged legitimate above. docs/contracts/replicator-boundaries.md :: Asked and answered — the derived-sidecar question (#69, 2026-08-20) + +# --- judged during the 2026-09-09 curation ------------------------------ +# All three SKILLS.md hits point at the *arrangement* rather than at a block: +# `## Project Layout` and `## Detail Docs` both still exist and still hold the +# content each sentence describes — this run tightened their prose and moved the +# command→fact diagram to docs/ARCHITECTURE.md, neither of which is what these +# lines name. Verified by reading all three against the sections they point at. +docs/SKILLS.md :: top-level package is the change that leaves AGENTS.md's Project Layout stale, +docs/SKILLS.md :: AGENTS.md's Detail Docs list is a by-name inventory of `docs/*.md`, so a new +docs/SKILLS.md :: doc does need an AGENTS.md line the gate will not ask for. A blanket `docs/` +# Provenance heading from #79 in a doc this curation did not move anything into +# or out of; same shape as the eleven already judged legitimate above, and +# renaming it would break inbound anchors for no gain this run can claim. +docs/TESTING.md :: Testing against a broker at its cap (#79) diff --git a/.skills/context-token-counts b/.skills/context-token-counts index eaec6cc..fb87157 100644 --- a/.skills/context-token-counts +++ b/.skills/context-token-counts @@ -3,18 +3,18 @@ # The offline estimators prefer a file's own last exact measurement to the # repo-wide figure in .skills/context-token-ratio, falling back to it for a # file never counted exactly or since drifted far from the size recorded here. -14878 6001 AGENTS.md -3753 1463 docs/ARCHITECTURE.md -10927 4251 docs/COMMANDS.md -13946 5164 docs/CONVENTIONS.md +14508 5988 AGENTS.md +4428 1681 docs/ARCHITECTURE.md +11213 4352 docs/COMMANDS.md +27269 9984 docs/CONVENTIONS.md 26650 9986 docs/DEPLOYMENT.md -21206 8005 docs/ENVIRONMENT.md +21446 8100 docs/ENVIRONMENT.md 22943 8612 docs/SKILLS.md 4977 1932 docs/SOCRATICODE.md 13176 4835 docs/STORAGE.md 24259 8747 docs/STREAMS.md 3759 1367 docs/STYLE.md -12139 4537 docs/TESTING.md +15909 5895 docs/TESTING.md 31972 11854 docs/contracts/content-fetch-issuer-contract.md 31801 11551 docs/contracts/content-fetch-issuer-reference.md 28660 10140 docs/contracts/content-replicate-issuer-contract.md diff --git a/AGENTS.md b/AGENTS.md index 086ffc6..5963de2 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -6,24 +6,17 @@ Be terse. Prefer fragments over full sentences. Skip filler and preamble. Sacrif Retrieval, fingerprinting, and temporary storage layer for the Cannabis Observer cluster. -Owns content fetching, temp storage, and fingerprinting — the network-bound, byte-handling work re-homed out of Watcher. Driven by **commands** on the Redis change bus; reports outcomes as **facts**. - -``` -content.fetch (command) → fetch → fingerprint → temp-store → blob_available (fact) - ↘ closed without bytes ───────────────→ fetch_failed (fact) -content.replicate (cmd) → guards → create-if-absent ────────→ replication_complete (fact) - ↘ refused / conflict ──────→ replication_failed (fact) -``` - **Worker-first.** Primary process = bus consumer (`src/worker/main.py`), not an HTTP API. The FastAPI app is a `/health` surface only, dev-only until a status endpoint is wanted. +The command → fact flow: [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md). + ## Development Methodology TDD required. Red → Green → Refactor. No production code without a failing test first. ## Environment & Tooling -Python ≥3.12, uv, pytest, ruff. `ty` is available as a **non-gating** type checker (`uv run ty check`) — advisory only; no pre-commit or CI gate. +Python ≥3.12, uv, pytest, ruff. `ty` is a **non-gating** type checker (`uv run ty check`) — advisory, no pre-commit or CI gate. **co-core comes from the wheelhouse, not PyPI.** `co-core` / `co-core-aio` resolve from `./.wheelhouse`, mirrored from the private GCS index `gs://co-gcs-pypi` by `scripts/sync_wheelhouse.py` via `[tool.uv] find-links`. Run the sync **before** `uv sync` on a fresh clone or after a version bump: @@ -31,7 +24,7 @@ Python ≥3.12, uv, pytest, ruff. `ty` is available as a **non-gating** type che uv run --no-project --with 'google-cloud-storage>=2,<4' python scripts/sync_wheelhouse.py ``` -Auth is ADC. Pin the current minor — `>=0.13.1,<0.14` — and raise the **patch** floor with every co-core feature the code starts depending on: the reasoning, and the three ways a skew has already failed, in [docs/DEPLOYMENT.md](docs/DEPLOYMENT.md). +Auth is ADC. Pin the current minor — `>=0.13.1,<0.14` — and raise the **patch** floor with every co-core feature the code starts depending on; the ways a skew has already failed are in [docs/DEPLOYMENT.md](docs/DEPLOYMENT.md). ## Code Exploration Policy @@ -58,7 +51,7 @@ Full tool table, prefetch query, per-tool guidance, cross-repo search: ## Code Exploration Notes (repo-specific) -**The manifest is a source, not the artifact.** Nothing re-embeds it — no hook, no CI step — so editing a `description` there changes what the repo says and not what `codebase_context_search` returns. Re-run `codebase_context_index` in the same change, or the highest-authority answer an agent gets stays the stale one (#19 CR #17). +**The manifest is a source, not the artifact.** Nothing re-embeds it, so re-run `codebase_context_index` in the same change as a `description` edit — otherwise the stalest answer is the one with the most authority (#19 CR #17). **`mcp-driver.mjs` lies twice — silently through the `skills/` symlink (skills#177), falsely from a worktree (skills#180).** Use `"$SOCRATICODE_DRIVER"`; disbelieve health findings outside the main checkout. Both in [docs/SKILLS.md](docs/SKILLS.md). @@ -67,23 +60,21 @@ Full tool table, prefetch query, per-tool guidance, cross-repo search: `src/worker/` is the primary process — the bus consumer, with the byte path, the failure fact, the retention sweep, the pacer, and the `content.fetch-policy` reader each behind their own seam. `src/storage/` is the content-addressed temp store behind -the `BlobStore` protocol — **two backends now** (`local`, `gcs`), selected by -`REPLICATOR_BLOB_BACKEND` and defaulting to `local` (#7); `src/api/` is the dev-only -`/health` app; `src/core/` holds config, logging, and the consume path's failure -vocabulary. `tests/` mirrors -`src/`. Every module with the job it owns: +the `BlobStore` protocol — **two backends** (`local`, `gcs`), selected by +`REPLICATOR_BLOB_BACKEND`, default `local` (#7). `src/api/` is the dev-only `/health` +app; `src/core/` holds config, logging, and the consume path's failure vocabulary; +`tests/` mirrors `src/`. Every module with the job it owns: [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md). ## Infrastructure -**Single-VM setup.** Code committed to main is the deployed code. Replicator shares the VM with archiver, watcher, and notifier. +**Single-VM setup.** Code committed to main is the deployed code; the VM is shared with archiver, watcher, and notifier. -The worker binds no port; 8041 is the dev API port and 8040 is reserved. **Redis -is Archiver-operated** — Replicator is a client, never ships a broker, never -claims ownership — and server **≥ 7.0** is Replicator-critical because -`claim_stale` reads `XAUTOCLAIM`'s three-element reply. `scripts/check_redis_floor.sh` -guards it as an `ExecStartPre`. Ports, neighbours, and the redis-py pin: -[docs/DEPLOYMENT.md](docs/DEPLOYMENT.md). +The worker binds no port; 8041 is the dev API port and 8040 is reserved. **Redis is +Archiver-operated** — Replicator is a client, never ships a broker — and server +**≥ 7.0** is Replicator-critical because `claim_stale` reads `XAUTOCLAIM`'s +three-element reply, guarded by `scripts/check_redis_floor.sh` as an `ExecStartPre`. +Ports, neighbours, and the redis-py pin: [docs/DEPLOYMENT.md](docs/DEPLOYMENT.md). ## Server Lifecycle @@ -91,7 +82,7 @@ guards it as an `ExecStartPre`. Ports, neighbours, and the redis-py pin: && uv sync --frozen && sudo systemctl restart replicator` — `git push` instead of the pull when the merge happened here. -Three things that bite, each of which has no symptom until it matters: +Three that bite, each symptomless until it matters: - **The service refuses to start off `main`, or off unpushed commits** (#37, #48). `REPLICATOR_ALLOW_ANY_CHECKOUT=1` overrides; a dev worker asks the same question @@ -101,8 +92,8 @@ Three things that bite, each of which has no symptom until it matters: - **The daily skills-refresh hook commits without pushing**, which is one of the states the checkout guard refuses. Check `git status -sb` before a restart. -Every deploy situation with its command, the guard's full verdict table, and the -dev-server invocation: [docs/DEPLOYMENT.md](docs/DEPLOYMENT.md). +Every deploy situation, the guard's verdict table, and the dev-server invocation: +[docs/DEPLOYMENT.md](docs/DEPLOYMENT.md). ## Environment Variables @@ -121,30 +112,27 @@ what keeps a sibling service from colliding. `BUILD_ID` is the one deliberate exception, stamped generically by the unit. For shell commands (dev only), load both — the snippet is under Common Commands. -Every variable, which file carries it, and the reasoning behind each default: +Every variable, which file carries it, and each default's reasoning: [docs/ENVIRONMENT.md](docs/ENVIRONMENT.md). ## Bus Conventions -Replicator is a **consumer** first. Follow the conventions co-core and the archiver producer established: +Replicator is a **consumer** first — follow what co-core and the archiver producer established: - **At-least-once ⇒ idempotent.** The command dedupes on `command_id`; both facts are keyed per *occurrence* (`content_fingerprint:command_id`, - `command_id:occurred_at`), so nothing an issuer waits on can collapse — storage - identity and correlation identity are not interchangeable. `info_source_id` - rides both and is **echoed, never read**, as is replicate's - `info_item_rep_spec_id`: each `test_boundaries.py` carve-out is one field - wide, and adding one edits the charter (#28, #29). + `command_id:occurred_at`). `info_source_id` and replicate's + `info_item_rep_spec_id` are **echoed, never read** — each `test_boundaries.py` + carve-out is one field wide, and adding one edits the charter (#28, #29). - **Two blob backends, one seam.** `local` announces `file://` and `gcs` announces - `gs://`; `local` is the compiled-in default **by decision, not by schedule** — - the deployment flipped on 2026-08-20 and the default did not. Every `BlobStore` - call from a coroutine goes through `asyncio.to_thread` — which puts it in the - unit's shutdown budget, not just the handler's. Per-backend retention, ceilings - and failure classification: [docs/STORAGE.md](docs/STORAGE.md), which is the - authority and worth reading before touching either store. -- **Store, then publish — never the reverse.** A fact pointing at bytes that are - not there is unrepairable by the consumer; stored bytes with no fact repair - themselves on the reclaim. + `gs://`; `local` stays the compiled-in default deliberately (#7). + Every `BlobStore` call from a coroutine goes through `asyncio.to_thread`, which + puts it in the unit's shutdown budget, not just the handler's. + [docs/STORAGE.md](docs/STORAGE.md) is the authority — read it before touching + either store. +- **Store, then publish — never the reverse.** A fact pointing at absent bytes is + unrepairable by the consumer; stored bytes with no fact repair themselves on the + reclaim. - **Read `count=1`.** `AsyncBusConsumer.read(count>1)` raises on a malformed frame *before* returning the well-formed ones, and `claim_stale` at `count>1` lets a poison entry jam recovery permanently. @@ -154,30 +142,25 @@ Replicator is a **consumer** first. Follow the conventions co-core and the archi `schema_version` first. - **Deterministic ⇒ DLQ; transient ⇒ retry; completed without bytes ⇒ fact + ack, no DLQ (#17).** `dead_letter` acks inside itself, so a fact is published - *before* it — as `XADD .dlq` then `XACK`, which is the form broker#2's - ACL grants and is now observed rather than inferred (#79). Retry cadence is - `REPLICATOR_CLAIM_MIN_IDLE_MS`; a failing *cycle* is `run_loop`'s problem, not - the message's. + *before* it — as `XADD .dlq` then `XACK`, the form broker#2's ACL grants + (#79). Retry cadence is `REPLICATOR_CLAIM_MIN_IDLE_MS`; a failing *cycle* is + `run_loop`'s problem, not the message's. - **A capped broker refuses only its `denyoom` commands, and the worker retries the two it meets at runtime (#79).** `XADD` and `SET` are refused and retried indefinitely — `OutOfMemoryError` is transient and exempt from the delivery - ceiling, the consume path keeps reading, acking and reclaiming throughout, and - nothing is dropped or dead-lettered. The third, `XGROUP CREATE … MKSTREAM`, is - boot-only and does **not** retry: `ensure_group` re-raises anything but - `BUSYGROUP`, so a first boot against a capped broker exits and systemd - restarts. Verified against a scratch broker this repo spawns, never the shared - one. Never answer an OOM with a client-level retry — a re-sent `XADD` the - broker already applied publishes twice. + ceiling — while the consume path reads, acks and reclaims throughout. The + third, `XGROUP CREATE … MKSTREAM`, is boot-only and does **not** retry: a first + boot against a capped broker exits and systemd restarts. Verified against a + broker the tests spawn, **never the shared one**. Never answer an OOM with a + client-level retry, which republishes an `XADD` the broker already applied. - **An ACL denial is transient too (#82).** `NoPermissionError` is the second `ResponseError` subclass in `_TRANSIENT_ERRORS`, so a grant broker#1's cutover got wrong backs off instead of closing valid commands with a terminal - `fetch_failed(handler_error)`. The cost is deliberate: a grant nobody fixes - retries forever rather than reaching `.dlq`. + `fetch_failed(handler_error)` — at the deliberate cost that a grant nobody + fixes retries forever. - **The `replicator:cmd:*` keys are the only non-stream keys on the broker (#80).** Per-stream dedupe — `SET NX EX` after a *completing* close, `EXISTS` before the - handler — so losing them costs one TTL window of re-fetches and never - correctness. Endorsed as bus state rather than a role blur, with broker#1's four - answers and the ACL grant they imply, in + handler — so losing them costs one TTL window of re-fetches, never correctness: [docs/CONVENTIONS.md](docs/CONVENTIONS.md#the-replicatorcmd-keys). - **Consumers must be idempotent; producers own the outbox.** Replicator has no DB — its durable record of intent is the consumer group's PEL. Do not add a @@ -187,24 +170,17 @@ Replicator is a **consumer** first. Follow the conventions co-core and the archi `content.blobs` and `content.artifacts` each carry both outcomes of their command; `content.fetch-policy` is read **groupless** — no group, no ack, no DLQ. -- **The replicate loop writes for `gcs` (#29)** — create-if-absent, `blob_uri` - never resolved as a path, writers keyed by alias, refusals before credentials, - provider failures classified by HTTP status. Read - [docs/CONVENTIONS.md](docs/CONVENTIONS.md) before touching that path. +- **The replicate loop writes for `gcs` (#29)** — create-if-absent, `blob_uri` never + resolved as a path, writers keyed by alias, refusals before credentials, provider + failures classified by HTTP status. Read + [docs/CONVENTIONS.md](docs/CONVENTIONS.md) first. - **Nothing but the seed script writes to `content.fetch`.** `scripts/seed_fetch.py` - requires `--production` for the one combination the live worker consumes — a - frame there is fetched for real. -- **Three normative contracts bound the wire and the roadmap** — four documents, - all under `docs/contracts/`, linked from sibling repos and indexed below. - `tests/test_boundaries.py` enforces the charter in CI; change a - charter and its tests together. - -Where the reasoning lives: - -- What each stream carries — [docs/STREAMS.md](docs/STREAMS.md) -- The rules common to all of them, and the `replicator:cmd:*` keyspace — [docs/CONVENTIONS.md](docs/CONVENTIONS.md) -- Blob paths, modes, and the retention sweep — [docs/STORAGE.md](docs/STORAGE.md) -- Fakeredis's divergences, the keys an integration run may touch, and why production `co-gcs-replication` is refused from every test — [docs/TESTING.md](docs/TESTING.md) + requires `--production` for the one combination the live worker consumes: a frame + there is fetched for real. +- **Three normative contracts bound the wire and the roadmap** — four documents + under `docs/contracts/`, linked from sibling repos and indexed below. + `tests/test_boundaries.py` enforces the charter in CI; change a charter and its + tests together. ## Common Commands @@ -258,32 +234,30 @@ from src.core.logging import get_logger logger = get_logger(__name__) ``` Entry points only: `configure_logging()` is called once inside the FastAPI `lifespan` or the worker's `run()`. Never in library modules. - -The logging stack — one formatter, two installers, and the journald lines that -are deliberately not JSON: [docs/STYLE.md](docs/STYLE.md). +The stack itself: [docs/STYLE.md](docs/STYLE.md). **Date & Time:** - All UTC - ISO 8601: `YYYY-MM-DDTHH:MM:SS.ffffffZ` (timestamps), `YYYY-MM-DD` (dates) **General:** imports at file top and explicit, docstrings on public modules, -classes and functions, small focused functions, and tests mirroring source — -each with its rationale and its ruff gate in [docs/STYLE.md](docs/STYLE.md). +classes and functions, small focused functions, and tests mirroring source — each +with its rationale and ruff gate in [docs/STYLE.md](docs/STYLE.md). ## Detail Docs -- [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) — founding design and the module-by-module layout; read before changing one +- [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) — founding design, the command → fact flow, module by module; read before changing one - [docs/STREAMS.md](docs/STREAMS.md) — what each stream carries, one bullet per rule `AGENTS.md` states in a line -- [docs/CONVENTIONS.md](docs/CONVENTIONS.md) — the co-core/Redis Streams rules common to every stream: idempotency, validation, DLQ, `claim_stale`; and the `replicator:cmd:*` keys, this service's only non-stream footprint (#80) -- [docs/STORAGE.md](docs/STORAGE.md) — blob paths and modes, the three populations under `REPLICATOR_BLOB_DIR`, TTL and ceiling semantics -- [docs/DEPLOYMENT.md](docs/DEPLOYMENT.md) — VM topology, ports, the systemd unit's lifecycle, and the co-core pin +- [docs/CONVENTIONS.md](docs/CONVENTIONS.md) — the rules common to every stream: idempotency, validation, DLQ, `claim_stale`; and the `replicator:cmd:*` keys (#80) +- [docs/STORAGE.md](docs/STORAGE.md) — blob paths and modes, the populations under `REPLICATOR_BLOB_DIR`, TTL and ceilings +- [docs/DEPLOYMENT.md](docs/DEPLOYMENT.md) — VM topology, ports, the unit's lifecycle, the co-core pin - [docs/ENVIRONMENT.md](docs/ENVIRONMENT.md) — every variable either env file carries, and the boundary between them -- [docs/TESTING.md](docs/TESTING.md) — where fakeredis diverges from the live broker, which keys an integration run may create, and why production `co-gcs-replication` is unreachable from every test (#38) +- [docs/TESTING.md](docs/TESTING.md) — fakeredis's divergences, the keys an integration run may create, why production `co-gcs-replication` is unreachable (#38) - [docs/STYLE.md](docs/STYLE.md) — the logging stack: formatter, installers, and the non-JSON journald lines - [docs/COMMANDS.md](docs/COMMANDS.md) — every runnable command, with flags -- [docs/SKILLS.md](docs/SKILLS.md) — vendored skill inventory, refresh procedure, and the doc-check sensitive-path list -- [docs/SOCRATICODE.md](docs/SOCRATICODE.md) — the full tool table, the prefetch query, per-tool gotchas, and cross-repo search +- [docs/SKILLS.md](docs/SKILLS.md) — vendored skill inventory, refresh procedure, doc-check sensitive paths +- [docs/SOCRATICODE.md](docs/SOCRATICODE.md) — full tool table, prefetch query, per-tool gotchas, cross-repo search - [docs/contracts/content-fetch-issuer-contract.md](docs/contracts/content-fetch-issuer-contract.md) — what a `content.fetch` producer must do; normative, linked from issuer repos -- [docs/contracts/content-fetch-issuer-reference.md](docs/contracts/content-fetch-issuer-reference.md) — its lookup half: the refusal list, the failure taxonomy, the silent conditions, trust posture +- [docs/contracts/content-fetch-issuer-reference.md](docs/contracts/content-fetch-issuer-reference.md) — its lookup half: refusal list, failure taxonomy, silent conditions, trust posture - [docs/contracts/replicator-boundaries.md](docs/contracts/replicator-boundaries.md) — what Replicator may become; run its three tests against any proposed capability -- [docs/contracts/content-replicate-issuer-contract.md](docs/contracts/content-replicate-issuer-contract.md) — the replicate trust model and issuer obligations, settled ahead of the code (#34) +- [docs/contracts/content-replicate-issuer-contract.md](docs/contracts/content-replicate-issuer-contract.md) — the replicate trust model and issuer obligations (#34) diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 07f80b0..3097d3e 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -8,6 +8,17 @@ bus stream carries is in [STREAMS.md](STREAMS.md), the rules common to all of them in [CONVENTIONS.md](CONVENTIONS.md), and blob-tree and retention rules in [STORAGE.md](STORAGE.md). +## What it owns, and what it emits + +Owns content fetching, temp storage, and fingerprinting — the network-bound, byte-handling work re-homed out of Watcher. Driven by **commands** on the Redis change bus; reports outcomes as **facts**. + +``` +content.fetch (command) → fetch → fingerprint → temp-store → blob_available (fact) + ↘ closed without bytes ───────────────→ fetch_failed (fact) +content.replicate (cmd) → guards → create-if-absent ────────→ replication_complete (fact) + ↘ refused / conflict ──────→ replication_failed (fact) +``` + ## Project Layout ```