refactor: V2: split the core model into machines, applications, and identities - #515
Open
passcod wants to merge 109 commits into
Open
refactor: V2: split the core model into machines, applications, and identities#515passcod wants to merge 109 commits into
passcod wants to merge 109 commits into
Conversation
passcod
changed the title
A many-to-many between devices and servers, trigger-maintained off every status insert. The identity-to-machine link is a single column on the machine, so it has nothing left to model. Dropped in the order Postgres will not enforce: trigger, then function, then table. A PL/pgSQL body is text, not a parsed dependency, so dropping the table first succeeds and leaves the trigger to fail on the next status push — every status push, for every reporter. statuses is partitioned and the trigger lived on the parent, so one DROP covers every partition and any created later. The whole slice goes with it: the model reader, the /api/devices/get_past_server_associations endpoint and its OpenAPI path, the generated TS, the "Past server associations" panel, the device-merge fix-up, and the seeder's truncate entry. The staleness anchor moves to machines.registered_at, joined in rather than fetched separately. Anchored on the box because the box is what gets backed up: anchoring on an application's registration would restart a machine's deadline every time a workload was added to it. Both cases now have tests. Behaviour change, not a like-for-like swap: min_first_seen was effectively "first status this application ever pushed", and enrolment precedes first push, so the anchor shifts earlier. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CheckSubject names the 18 checks that describe the box rather than the workload on it, and ingest grades and files each check at its own grain from one unified payload. Whole names, never prefixes: caddy_version is the box's while caddy_certs is the workload's, and ips is the box's addresses while ips_errors is a Tamanu error stream. Anything unrecognised is the application's. All five scouted traps were real: - raise_machine_event_with_state now takes the source. Its group and canopy-wide siblings assume canopy because they file what canopy determines for itself; a machine's checks come from alertd, and recording them under canopy would break per-source silences, the check_severities a push answers with, source staleness, and same-source recovery. - The debug assertion rejecting non-canopy filings outside application scope now admits a machine's. - silenced_health_checks_for_server covers the machine grain, so a silence on a machine check reaches the agent instead of holding only on canopy's side. - enqueue_incident_reeval keys on the application, so the machine path evaluates its incident inline. - Recovery bookkeeping is per grain. One shared previously-active set would make a check that moves grain read as unmentioned on the grain it left, closing and reopening it every push. The machine filing path carries device_id, so a machine issue records which reporter filed it. The detail-field split is now its own plan step: every figure read goes through server_reported_detail, and osName/osVersion/munin/bestoolVersion are machine-subject but read from the application's row, so the table split and the figure reads have to move together. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
server_reported_detail becomes application_reported_detail, and machine_reported_detail joins it. Existing rows are split in place: the box's fields move to its machine and come out of the application's body, so each fact is stored once. A host running two workloads records its platform once rather than once per workload. Reads did not change, which is what made this tractable. for_server returns an application's own detail merged with its machine's, so every figure consumer sees the view it saw before; the storage is what moved. The plan's worry — that every figure read goes through one table, with osName/osVersion/munin/ bestoolVersion machine-subject but read from the application's row — is answered by merging on read rather than by moving the readers. version stays with the application and has no machine counterpart: a version is what the workload runs. The agent's own version is a detail field like any other and goes to the box. The field list lives beside the check-subject list in commons_types::subject, since both answer the same question; the module is renamed from check_subject to say so. One regression introduced and fixed: merging on read made for_server error for an application that no longer exists, where it used to return nothing. A deleted application has no detail rather than being an error. An existing test caught it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Split the core model: machines, application servers, and identities (V2) An identity belongs to a machine, so which application a name request concerns is resolved from the name rather than from the credential. `authorise` now goes identity to live machine to the application on it declaring the name, and the entitlement answer carries one entry per workload on the box. Names were only ever declared as a side effect of an agent registering addresses, so there was nowhere for an operator to say which workload serves what. Adds `certificates/declare` and `certificates/release`. The operator refusal names the holder; the device-facing one never does. Ordering a certificate now declares the name it orders for, so an order exists only for a declared name. That invariant is what lets renewal and the expiry alert skip a released name: renewing past a release would order for a name another application now serves. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Split the core model: machines, application servers, and identities (V2) Maintenance windows landed on main while the split was in flight, written against `servers` before the rename. Rebasing carried them in; this moves them onto the grain the split creates. Taking a box down to patch it stops everything running on it, so a window naming one application left the others on the same host alerting through work that was always going to stop them. Naming the machine makes that one declaration with N consequences. Every pre-split window was over a server that is now an application on exactly one machine, so the backfill is that join. A filing now carries two different machine ids and conflating them widens what a window does: `machine_id` scopes machine-written silences, while `covering_machine` is the box whose window covers the filing, which for an application is the host it runs on. Passing the covering machine to `scoped_to` would have made every machine-scoped silence apply to that box's workloads. They travel as named fields on `FilingScope` rather than as four positional Option<Uuid>. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Split the core model: machines, application servers, and identities (V2) What a replica restores is a snapshot, and a snapshot is what a machine backed up. Naming an application was the only target available before the split: on a two-workload box it made the choice arbitrary, and a whole-group declaration expanded over applications, so such a box got two replicas of the same backup. The checks part along the same axis. restore-verification and redaction are the machine's, because what failed to restore is the box's backup. migration-test is the application's, because the version under test is that workload's. Three things the step turned up: The recovery gate is grain-specific. open_server_issue_active reads issues.application_id, so a machine-scoped check asking through it would never find its own open issue and never file the recovery. Added the machine-grain twins, and file_restore_check picks by the scope it files at. Silent if missed: the check files fine and never clears. The interleaving needed storing rather than deriving. A migration test is a machine's snapshot plus an application's candidate, so migration_tests gained an application_id; the worklist entry carries both ids and a report echoes the application back. Without it a two-workload box with different candidates could not have a verdict attributed. Snapshot authority reaches the machine through a join, because backup_runs still records the application that reported a run. That join goes away when the backup tables take the machine grain, now tracked as its own step. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Split the core model: machines, application servers, and identities (V2)
ShortStatus loses Away and Blip. Up, Down and Gone survive as reachable,
unreachable and never reported, with the wire values unchanged.
short_status now takes the target's own down threshold instead of fixed
2/10/30-minute bands, so the indicator and the reachability check are
graded on the same clock. They could previously disagree outright: a target
configured to five minutes showed a healthy dot while its own reachability
check had already failed. Application::reachability holds the threshold
lookup so the six call sites cannot drift apart on it again.
Two things found on the way. The status legend was already wrong, claiming
blip was "missed 2 checks" and away "last seen 2-10m ago" where the code
made blip 2-10m and away 10-30m; it now names the three states with no
durations, since each target is judged against its own. And away and down
never carried a health signal, StatusDot's reachable set being {up, blip},
so collapsing it to {up} makes that rule legible rather than incidental.
CHK gained a line separating the check's three results, which grade how
much of what should be reporting still is, from the target's reachability,
which is whether anything is. MCP's undefined "recent-activity window" now
points at the target's own threshold.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Split the core model: machines, application servers, and identities (V2) find_machines and get_machine are new, with figures split along the same axis as the reported detail: platform, hostname, processor count, memory, uptime, filesystems and addresses to the machine; version and database engine to the application. Each side names the other. An issue's target became one tagged scope rather than a row of nullable ids, so a client cannot read a machine's failure as unattributed because application_id was null. Three bugs behind it, all from machine checks having filed since the ingest step with nothing reading them. Issue::list excluded machine issues from the fleet list outright: its "not canopy-wide" guard was application_id OR server_group_id being non-null, and a machine issue has both null, so every maintenance, restore-verification, redaction and machine-subject finding was invisible. The group filter collected only a group's applications, dropping its machines' issues. And enrich_issues resolved names from application_id alone, so a machine issue rendered with no name, host or deployment; enrich_issue had its own copy of that logic and now delegates, the two having already drifted. machine_health_from_check_state is a sibling of the application rollup rather than a generalisation: the queries differ only in which column names the target, and both end at HealthState::from_results. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…tion Split the core model: machines, application servers, and identities (V2) W1 (PR #524) retires "deployment" for group, environment, and the Canopy instance, and adds an AGENTS.md grep rule to keep it retired. Sweeps the ~24 uses this branch introduced, leaving main's ~322 to W1 so the two open PRs do not both rewrite the same prose. billing.deployment keeps its spelling, being read outside Canopy. W1's GRP defines an environment as a group's servers at one rank, which stops meaning one thing once servers split: rank lives only on applications while group_id lives on both grains. So an environment is a set of applications, a machine belongs to a group without belonging to any of its environments, and a machine's stage is derived as the highest rank among the applications on it, which is what APP already says about billing. Settled in FLT because this branch is what introduces the distinction; the cross-reference to GRP goes in when W1 lands. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Split the core model: machines, application servers, and identities (V2) "An application is a piece of software running somewhere" is what the words already mean; it defined nothing about Canopy. The machine line was the same in weaker form. What is load-bearing is the sentence after them, which says machine facts belong to the machine and application facts to the application, and that survives. Drops the three uses of "site" this branch introduced along with them. It was a fourth word for a spec set W1 just reduced to three, and each place reads better without a noun there at all: an upgrade looks like a machine falling over, and what is production or test is the software serving that role. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…used Split the core model: machines, application servers, and identities (V2) "A box is not in an environment" was a claim about the world, and a wrong one: a machine does serve one environment in practice, since nobody puts a production workload and a demo one on the same box. What is true is only that Canopy records no relationship between them, because nothing it does turns on one, and a machine's stage is derived from its applications instead. The derivation is a safety net for a mixed box rather than a description of the normal case, and the spec now reads that way. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Split the core model: machines, application servers, and identities (V2) Checkpoint: all source compiles, tests do not yet. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The bar, stated mechanically: `bestool-canopy` generated from the public OpenAPI spec before and after this branch must not be a semver-breaking change. Comparing the generated specs shows `product` and `kind` never appeared on the public wire at all, so merging them into one type costs the agent-facing crate nothing. The one agent-facing surface that did carry them is the reserved tags, which are map keys rather than schema and so would have broken silently: both stay emitted, derived from the type, beside the new `canopy:type`. Two real breaks, both the same rename. `server_id` became `machine_id` on `VerificationArgs` and on `WorklistEntry`. Both keep the old name beside the new: the backfill gave every pre-split machine its application's id, so the two values are equal for anything that predates the split. A report naming both is refused rather than resolved by preference, since a reporter that disagrees with itself about what it restored has not been understood. Also restores the canonical-member rule the type merge had narrowed. A group's headline version comes from its highest-ranked version-tracked application, a central beating a facility on a rank tie, as it did when kind broke that tie. A group of facilities has a headline version; only a group of untracked types has none. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…he machine form One `ApplicationTypeChip` replaces `ServerProductChip` and `ServerKindChip`, since one type carries what two chips carried. `useProducts` becomes `useApplicationTypes`, keyed by type off the same catalogue endpoint. The sort helper orders by rank then type. The edit form shows an application's type rather than offering it, a type being reported and never entered, and gates the public-name field on that type's own capability. The server-create form becomes the machine-create form: the same form minus URL, product, kind, rank and public name. `MachineCreateArgs` widens to match `MachineUpdate` so creating and editing cannot disagree about what a field means. Binding a tailnet node at create time sets `device_id` without setting `registered_at` — naming a box is not the box arriving, and a backup deadline counts from arrival. Restores two behaviours the type merge had narrowed without needing to: - A group's billing attribution names a product again when its applications agree on one. They agree on software rather than on type, so a group holding a central and a facility names Tamanu, as it did when product was a field. - `SENAITE` keeps its own styling in the label the catalogue serves, rather than being sentence-cased into `Senaite`. Two items are outstanding rather than dropped, recorded in the plan: the machine form has no reachability switch, there being no machine-scoped silence to write; and it lands on the group page, there being no machine detail page yet. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The definition came out of this card but governs every change, so it lives in the specs rather than dying with the plan. A new API spec states it: the `bestool-canopy` crate generated from the public OpenAPI spec after a change must not be a semver-breaking change against the one generated before it, unless the break has been coordinated. Coordination is what makes a break permissible, and everything else is held to the definition. The spec is mechanical on purpose. "Keep the wire compatible" is a judgement each author makes differently; regenerating the crate and comparing is a check that passes or fails, so a break is found before it ships rather than reported from the field. It also covers the surface the definition cannot see. A tag key is not part of the schema, so withdrawing one breaks a consumer silently — those are held to the same rule, which is why an application still serves the software and the role beside its type. APP and FLT now say the system serves all three, which they did not. `AGENTS.md` carries the operative form alongside the repo's other standing constraints. Also drops a link to `servers/groups.md`, a file W1 introduces and this branch does not have. The plan had already settled that those links wait for W1; this one had gone in anyway. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`machines.get_detail` assembles what a box's page needs: its record, group and identity, the figures resolved across every source reporting on it, its own health and checks, the applications on it each carrying their own dot, and its billing labels. A version or a database engine is absent — those are a workload's rather than a box's, and each application's page carries them. Two pieces had to exist first. Machine reachability was nowhere: `Application::reachability` reads a `statuses` row and a machine has none, so `Machine::reachability` reads when the box last reported against the box's own threshold. A quiet machine and a quiet workload are two findings. And `consolidated_checks_latest` was application-only; rather than a second copy that would drift, it became `consolidated_checks_for(target: Scope, ..)` with two thin wrappers, so the catalog gate, silence pass, reachability fill-in and ordering stay one implementation. `ChecksTable` and `HealthIndicator` move out of `ServerDetail` into a component so both pages share one implementation. A machine's checks present without a silence control, which the table's `serverId: null` states rather than implies: a silence names an application, and none can name a machine yet. The group now presents its machines and the applications under each, bucketed by the rank a machine takes from its highest-ranked application. A machine carrying nothing appears as awaiting check-in rather than being absent — it was invisible before, so an operator who had just added a box had nothing to look at. An application links to its machine and the machine back, and a maintenance window's machine target links to the page instead of rendering as plain text. Fixes a bug found on the way: `suspended_targets` returns machine ids since maintenance took the machine grain, but the fleet listing tested them against the application id. Every machine that predates the split took its application's id, so the wrong read agreed with the right one on all existing data and parted company only for a machine created since. The test seeds deliberately unequal ids and fails on the old code. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
passcod
force-pushed
the
workhorse/v2
branch
from
September 1, 2026 06:04
4c221b2 to
928b487
Compare
The rule, now in CHK: a check that can be filed at a scope can be silenced at that scope. The scopes are the ones the check applies at — its own target and that target's group. Every reader of a silence must give the same answer. Silencing a check everywhere is the catalog ceiling rather than a silence, so no scope above the group is offered as one. The machine grain did not hold to it, and the gap was not where it looked. `ScopedCheckPolicy::silence` takes a `Scope` and has always written `machine_id`; the agent-facing set and the consolidated view both already read machine silences. What was missing was an operator surface — and underneath it a defect: `re_evaluate_incident_membership` resolved a silence by passing `issue.application_id.unwrap_or(Uuid::nil())`, so a machine-scoped issue consulted only its group. A machine silence therefore quieted the check in the consolidated view and in what the agent was told to run, while the same check still opened an incident. Three readers, two answers. `is_silenced` now takes the issue's own `Scope`, so the grain a check was filed at is the grain it is silenced at. `MachineSilencedRef` mirrors its two siblings, `silence_machine` / `unsilence_machine` / `list_for_machine` sit beside theirs, and `ChecksTable` takes a `CheckTarget` discriminated by grain rather than a nullable `serverId` — so the scopes it offers follow from what it is presenting instead of from a special case. Tests cover the invariant directly: one machine silence read at all three points, a machine silence not reaching the applications on that box, and a group silence reaching the machines in it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`StatusDot` carried two encodings because a server was two things: a fill for reachability and a ring for health. The machine owns reachability now, so the dot has one subject and spends its whole colourway on the application — healthy, warning, failing, never reported. The ring goes, and the unhealthy case stops being a fill-and-ring inversion and becomes a colour. `MachineEnclosure` is the machine: a pill around that box's dots, neutral, orange when the box's own checks are degraded, red when the box is down. Red on both grains is deliberate — red means down, and which element carries it says what went down. Orange belongs to the enclosure and light green to the dot, so each hue means one thing. A group card's dots become rank rows of enclosures rather than one flat strip with a triangle at the rank break. The triangle goes: the break is a rule now, and the enclosures say which dots share a box, which the strip never could. `FacilityServerStatus` gained the machine — id, name, its own reachability and health — since the card had nothing to group by. Two batch reads come with it, `Machine::get_many` and `MachineReportedDetail::latest_for_machines`, so a page of cards does not ask once per box. The create form's reachability switch is back too, writing the machine-scoped silence that now exists rather than the server-scoped one it used to. Its two e2e tests return with it, asserting on `scoped_check_policies.machine_id`. CHK gained "One subject per mark": a mark says one thing about one subject, an enclosure means nothing on its own, and no mark carries a second encoding for a second subject. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Maintenance arrived from main mid-split, after the status-page mockup was drawn, so neither the colourway nor the card type accounted for it: a box being worked on looked exactly like one that was not, on the page an operator watches. `FacilityServerStatus.machine_maintained` comes from the one batch `suspended_targets` read the card already had reason to make. The enclosure is what carries it. A window is declared over a machine or a group and never over an application, so the box shows it and the applications inside are suspended by their box rather than each saying so. A group's window marks every box on the card. The pill is hatched rather than cut. A mask on the enclosure would clip the dots inside it as well, which would say something about the applications, and the window is the box's. The hatch runs the same diagonal as the dot's maintenance cut, so the two read as one idea at either grain. The dot keeps its own cut for surfaces that draw applications without a box — a sibling strip — where it is the window's consequence for that application rather than a window of its own. CHK says which mark carries a window, and that a window's mark is distinguished from the mark for a target nobody is watching, so deliberate temporary work does not read as neglect. Also unticks the Frontend step, which was ticked early: the group card's three-band layout is still outstanding. The enclosures and rank rows are done; the bands are not. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Name and version, then the rank rows, then a status band carrying operators on the left and the incident right-aligned beside them. The band is omitted when there is neither, so a quiet card is two bands and the eye goes to the ones that have a third. `CardContent`'s padding goes with it: bands run edge to edge, so the card clips them and each band carries its own padding. Going back to the mockup for the band layout caught two deviations in what had already landed: - The enclosure was a solid fill where the mockup draws an outline with a wash. The pill is context for the dots inside it, not a competitor to them. - The rank rows had lost their watermark. The mockup spells the rank out behind its own row, faint enough to read only when looked for, and that is what replaces the triangle — rather than nothing replacing it. The test case for it is restored too; I had rewritten it into a weaker one when it went missing. Also removes a race from the machine-silence test, which flaked once under a full-suite run. It asserted on the popover's text, which re-renders off the same fetch the write triggers; it now waits on the silences section, which renders only once that write has landed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…name
`GET /machines/self` answers which box the caller is and what runs on it.
`GET /servers/self` is untouched: it asks which *application* the caller is and
409s on a box running two, which is the case the new endpoint exists for. They
answer different questions rather than the same one under two names, so they
are two handlers rather than one with an alias, and a fielded agent reading the
old one is unaffected.
That `server_id` had to keep meaning an application settled the shape:
`POST /status/{server_id}` resolves an application, so an agent that read its id
from `/servers/self` and pushes to it would break if the id changed meaning.
The `server` role is renamed rather than aliased through the code. `DeviceRole`
is not on the public wire at all, so the variant becomes `Machine` outright, a
migration rewrites the stored rows, and `FromStr` and serde accept `server` on
input. Two properties are pinned: a row still written as `server` reads as the
machine role and authenticates — every device in the fleet was written that
way, so that read is what stops the rename locking them out — and what Canopy
stores and presents is `machine`.
DID claimed the two endpoints reach the same answer, which stopped being true
once they were written. It now says what each does.
The step turned out to be pure addition: nothing on the public wire had been
renamed, so there was no old path to alias. The whole diff against the base
spec is one added path, with the one known widening from earlier unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
BAK has said since it was written that a backup is a machine's: a device request resolves identity → machine → group and never reaches the applications on the box, and a box shared by two workloads backs up once. The storage lagged that, keying every backup table on whichever application reported the run. Nothing on the wire moves. A device never names a target — it is resolved from the authenticated identity — so these columns were internal throughout, and the public spec is untouched. Five tables move, each column RENAMEd rather than added-and-dropped so it keeps its position: the models load positionally, and a column at the end silently misaligns the ones after it. `backup_requests` and `server_backup_capabilities` carry the moved column in their primary key, so two applications on one box collapse to one row — the capability is OR'd and dated from the first advertisement, the request keeps the operator's latest intent. `server_backup_capabilities` becomes `machine_backup_capabilities`. Four things were wrong rather than merely misnamed: - `resolve_server` in the public backup handler picked the device's single live *application*, which is precisely what BAK says must not happen. - The staleness scan was rooted at applications joined to capabilities, so a box with two workloads would have produced two staleness rows for its one backup, and alerted twice. - Staleness and reconcile filed at `Scope::Application`; they file at `Scope::Machine`, which is what makes a shared box's late backup one finding. - `latest_success_by_machine_type_for_group` reached the machine by joining applications. That join is gone. The backup panel moves off the application page onto the machine's, per FLT. Incomplete: 7 e2e tests in `backups.spec.ts` still point at the old panel location and cross-link. Rust is green at 1288; typecheck, clippy and biome are clean. The plan records what is left. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.