Skip to content

Commit 0acaeb9

Browse files
gustavobertoiclaude
andcommitted
docs: design v1 feature specs 09-14 and integrate cross-links
Add the remaining v1 (M6) feature specs that previously had only backlog entries: orchestration/onboarding (09), health & ordering (10), lifecycle hooks (11), service profiles (12), doctor & teardown (13), self-update & migration (14). Fixes from adversarial review applied: SDK read-only health probes, db gc in the CLI surface, profiles/groups key disambiguation, preUp + saga phase-count reconciliation. Wire spec links into ARCHITECTURE/FEATURES/ROADMAP/README; resolve Q-NAME (devstack); add Q-PROFILE/Q-SAGA-PARALLEL/Q-HEALTHWAIT/Q-HOOK-SCOPE/Q-DOCTOR-FIX. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 05da150 commit 0acaeb9

15 files changed

Lines changed: 922 additions & 54 deletions

CLAUDE.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,13 +7,13 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
77
`devstack` is a single static Go binary (CLI) that manages Docker-based dev
88
environments and **shares infrastructure across projects** — one warm
99
Postgres/Redis/MinIO on a tool-owned network for many repos, instead of a
10-
duplicate stack per repo. Repo folder is `devdock-go`; the module path is
11-
`github.com/open-source-cloud/devdock-go`; the binary built from `./cmd/devstack`
10+
duplicate stack per repo. The Go module and GitHub repo are
11+
`github.com/open-source-cloud/devstack` (local checkout folder `devdock-go`); the binary built from `./cmd/devstack`
1212
is `devstack`. The same binary is invocable under aliases (`rq`, `uranus`) via
1313
`argv[0]` dispatch.
1414

1515
The project is **spec-driven**: `docs/` (ARCHITECTURE, DECISIONS, ROADMAP,
16-
FEATURES, OPEN-QUESTIONS, and `docs/specs/01..08`) is the source of truth. When
16+
FEATURES, OPEN-QUESTIONS, and `docs/specs/01..14`) is the source of truth. When
1717
behavior is ambiguous, those docs win — read them before designing. Implementation
1818
is at the **M0 (foundations) stage**; see "Current status" below.
1919

README.md

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# devstack
22

3-
> Working name. The repo folder is `devdock-go`; the binary name is a placeholder you can change at any time, and the tool can be invoked under multiple aliases (e.g. `rq`, `uranus`) via the [alias system](docs/specs/07-cli-and-aliasing.md). These docs use **`devstack`** consistently because it matches the config schema (`devstack.yaml`, `apiVersion: devstack/v1`).
3+
> The binary is **`devstack`** (Go module + GitHub repo `github.com/open-source-cloud/devstack`; the local checkout folder is `devdock-go`). The same binary is invocable under aliases (e.g. `rq`, `uranus`) via the [alias system](docs/specs/07-cli-and-aliasing.md). Config files are `workspace.yaml` + `devstack.yaml` (`apiVersion: devstack/v1`).
44
55
A single-binary CLI that manages Docker-based development environments and **shares infrastructure across projects** — one warm Postgres / Redis / MinIO for many repos instead of a duplicate stack per project. It is a clean-slate, Go reimplementation of the ideas behind [`devdock`](../devdock) (the Python predecessor), redesigned around a *workspace* of shared services that independent project stacks attach to.
66

@@ -18,7 +18,7 @@ A developer working across 8 microservices today runs 8 Postgres containers, 8 R
1818

1919
## Status
2020

21-
📋 **Design / spec phase.** No code yet. This repository currently holds the architecture and specs that will drive implementation (spec-driven development). See the doc index below.
21+
🚧 **M0 foundations implemented; design ongoing.** The spine is in place and green (`make ci`): the CLI tree + `argv[0]` aliasing, the `flock` cross-process lock, the SQLite ledger, XDG/WSL2 handling, a read-only Docker client, and a real `doctor` preflight. The rest of the surface is spec-driven (see the doc index). Try it: `make build && make smoke`.
2222

2323
## How it works (one paragraph)
2424

@@ -42,6 +42,12 @@ A developer working across 8 microservices today runs 8 Postgres containers, 8 R
4242
6. [Multi-repo git](docs/specs/06-git.md)
4343
7. [CLI & aliasing](docs/specs/07-cli-and-aliasing.md)
4444
8. [State, locking & lifecycle](docs/specs/08-state-locking-and-lifecycle.md) — the concurrency spine
45+
9. [Orchestration & one-command onboarding](docs/specs/09-orchestration-and-onboarding.md) — the `up` saga
46+
10. [Health, readiness & dependency ordering](docs/specs/10-health-readiness-and-ordering.md)
47+
11. [Lifecycle hooks](docs/specs/11-lifecycle-hooks.md)
48+
12. [Service profiles & selective up](docs/specs/12-service-profiles-and-selective-up.md)
49+
13. [Doctor, diagnostics & teardown](docs/specs/13-doctor-diagnostics-and-teardown.md)
50+
14. [Self-update, notifications & migration](docs/specs/14-self-update-and-migration.md)
4551

4652
## Quickstart (target UX — not yet implemented)
4753

docs/ARCHITECTURE.md

Lines changed: 10 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -126,7 +126,7 @@ Standard Go layout: `/cmd/devstack/main.go` (thin), `/internal/*` (all implement
126126
| `internal/merge` | Layered deep-merge with explicit list strategy (replace default, `$merge: append` opt-in); guards shared-reference mutation. |
127127
| `internal/generate` | Compose-model build + `compose-go` validate/normalize, `${ref}` resolution against the workspace graph, secret-key emission, `writeIfChanged` + atomic rename, SHA-256 rebuild-hash ledger. |
128128
| `internal/docker` | `moby/moby/client` wrapper (read-only) + `docker compose` CLI driver with explicit `-p` and tool-owned labels; compose/version/context preflight. |
129-
| `internal/workspace` | **The differentiator:** shared-stack lifecycle, ref-counting + self-healing reconcile, port allocation, per-(engine,major-version) shared instances, profile/selective-up interaction. |
129+
| `internal/workspace` | **The differentiator:** shared-stack lifecycle, ref-counting + self-healing reconcile, port allocation, per-(engine,major-version) shared instances, profile/selective-up interaction (the reference-graph walk behind `--profile`, [spec 12](specs/12-service-profiles-and-selective-up.md)). |
130130
| `internal/state` | SQLite ledger (keyed by Docker context), WAL+busy_timeout, versioned migrations + backup, rolling event log, `doctor --rebuild-state` from live Docker labels. |
131131
| `internal/lock` | `gofrs/flock` cross-process advisory lock wrapping all global mutations; detects 9p/networked state dirs and warns. |
132132
| `internal/provision` | `pgx/v5` idempotent per-project Postgres role/db; redis index/prefix; minio bucket+key; ownership ledger + orphan gc. |
@@ -136,12 +136,13 @@ Standard Go layout: `/cmd/devstack/main.go` (thin), `/internal/*` (all implement
136136
| `internal/trust` | mkcert (or owned CA) host/Firefox trust install; WSL2 `certutil.exe` interop; trust status/install/uninstall. |
137137
| `internal/dns` | `*.localhost` guidance + idempotent marker-fenced `/etc/hosts` edits (sudo), removable on uninstall. |
138138
| `internal/tunnel` | cloudflared shared container, named-tunnel create/route/up/down, `Tunnel` interface; secret-bearing-service refusal guard. |
139-
| `internal/doctor` | Cross-platform capability-probe matrix with one-line remediations and `--fix`. |
140-
| `internal/hooks` | Lifecycle hooks (`postUp`/`preDown`/`firstRun`/`postPull`) with an idempotency ledger; powers migrations/seeding. |
141-
| `internal/health` | Typed healthchecks + `dependsOn: healthy` graph gating. |
142-
| `internal/orchestrate` | The multi-phase `up`/bootstrap **saga**: named resumable phases with durable phase-state + compensating rollback; TUI checklist + plain fallback. |
139+
| `internal/doctor` | Cross-platform capability-probe matrix with one-line remediations and `--fix`; also owns recovery (`--rebuild-state`) and teardown (`workspace destroy`/`uninstall`/`db gc`). See [spec 13](specs/13-doctor-diagnostics-and-teardown.md). |
140+
| `internal/hooks` | Lifecycle hooks (`preUp`/`postUp`/`preDown`/`firstRun`/`postPull`) with an idempotency ledger; powers migrations/seeding. See [spec 11](specs/11-lifecycle-hooks.md). |
141+
| `internal/health` | Typed healthchecks + `dependsOn: healthy` graph gating (cross-project read-only poll + compose-native ordering). See [spec 10](specs/10-health-readiness-and-ordering.md). |
142+
| `internal/orchestrate` | The multi-phase `up`/bootstrap **saga**: named resumable phases with durable phase-state + compensating rollback; TUI checklist + plain fallback. See [spec 09](specs/09-orchestration-and-onboarding.md). |
143143
| `internal/xdg` | XDG path resolution; WSL2 detection; refuse `/mnt/*` working dirs; template-cache GC/TTL. |
144-
| `internal/migrate` | `devstack import` reading an old devdock `project.yaml` → new `workspace.yaml` + `devstack.yaml` split (optional; high-leverage for the existing user base). |
144+
| `internal/migrate` | `devstack import` reading an old devdock `project.yaml` → new `workspace.yaml` + `devstack.yaml` split (optional; high-leverage for the existing user base); plus the three-artifact versioning policy + self-update choreography. See [spec 14](specs/14-self-update-and-migration.md). |
145+
| `internal/version` | Build-stamped identity (`Version`/`Commit`/`Date` via ldflags); consumed by `--version` and the update notifier. See [spec 14](specs/14-self-update-and-migration.md). |
145146
| `pkg/pluginsdk` | Public plugin contract — **deferred to v2** (out-of-tree secrets/tunnel providers). |
146147

147148
Every fast-moving or risky dependency (docker client, template engine, secrets provider, git, trust store) sits behind an internal interface so it can be swapped or vendored.
@@ -156,15 +157,15 @@ These emerged from adversarial review and are the things most likely to bite. Ea
156157
Two simultaneous `up` invocations (two terminals; IDE + terminal; a watch script) race on network-ensure (TOCTOU between inspect and create), port allocation, ref-count rows, and `CREATE ROLE`. SQLite file locking is unreliable on WSL2/9p. **Mitigation (build first, in M0):** a coarse `gofrs/flock` advisory lock around *every* operation that mutates the ledger or the shared stack; WAL + `busy_timeout`; idempotent guarded SQL; port allocation done inside the lock; `doctor` warns on 9p state dirs. Reads are lock-free snapshots. See [spec 08](specs/08-state-locking-and-lifecycle.md).
157158

158159
### 7.2 Partial-failure / rollback for the multi-phase `up`
159-
`up` runs ~8 phases (clone → network → shared services → provision DB → secrets → CA → generate → compose up). A failure in phase 4 after phase 3 started the shared stack and inserted ref rows leaves a half-up workspace and a lying ledger. **Mitigation:** model `up` as a **saga** with durable phase-state and compensating actions; `doctor --rebuild-state` reconstructs the ledger from live Docker labels.
160+
`up` runs ~8 phases (clone → network → shared services → provision DB → secrets → CA → generate → compose up). A failure in phase 4 after phase 3 started the shared stack and inserted ref rows leaves a half-up workspace and a lying ledger. **Mitigation:** model `up` as a **saga** with durable phase-state and compensating actions; `doctor --rebuild-state` reconstructs the ledger from live Docker labels. The saga's named phases, the `saga_phase` resumability table, and the lock-granularity rule are detailed in [spec 09](specs/09-orchestration-and-onboarding.md).
160161

161162
### 7.3 Teardown / uninstall
162-
devstack creates artifacts everywhere: the external network, shared volumes (Postgres data!), the SQLite DB, **root CA in host + Firefox + Windows trust stores**, alias symlinks, cloudflared creds, keyring entries, the cache. `devstack workspace destroy` / `devstack uninstall` must reverse **all** of it, with explicit data-loss confirmation for volumes, and must never orphan a CA (a security artifact) or dangling symlinks.
163+
devstack creates artifacts everywhere: the external network, shared volumes (Postgres data!), the SQLite DB, **root CA in host + Firefox + Windows trust stores**, alias symlinks, cloudflared creds, keyring entries, the cache. `devstack workspace destroy` / `devstack uninstall` must reverse **all** of it, with explicit data-loss confirmation for volumes, and must never orphan a CA (a security artifact) or dangling symlinks. The teardown order and the `--fix`-vs-destroy boundary are owned by [spec 13](specs/13-doctor-diagnostics-and-teardown.md).
163164

164165
### 7.4 Migration / versioning (three things evolve independently)
165166
1. **Config schema**`apiVersion: devstack/v1`; define the unknown-key / `additionalProperties` forward-compat policy.
166167
2. **State-DB schema** — versioned migrations table, backup-before-migrate, strictly additive within a major.
167-
3. **Generated compose format**`compose-go` re-normalization causes churny diffs; decide whether generated artifacts are committed (deterministic, reviewable) or gitignored (regenerated freely). Either way, golden output controls the churn.
168+
3. **Generated compose format**`compose-go` re-normalization causes churny diffs; decide whether generated artifacts are committed (deterministic, reviewable) or gitignored (regenerated freely). Either way, golden output controls the churn. The compatibility guarantees for all three artifacts (config `apiVersion`, state `schema_version`, generated-compose byte-stability) and the self-update choreography tying them together are specified in [spec 14](specs/14-self-update-and-migration.md).
168169

169170
### 7.5 Secrets ↔ generation coupling, and in-memory exposure
170171
The "host env auto-propagates into compose" assumption is **false** (verified). Each secret name must be emitted **per service** into the generated compose (as a valueless `environment: [NAME]` key) and the value passed via `exec.Cmd.Env` — so secrets and generation are coupled and need one owner plus a CI test asserting **no secret value ever lands in any generated file**. Resolved secrets also live as plain Go strings (GC'd, swappable, visible via `/proc/<pid>/environ` to same-user processes). Decision: document the threat model honestly rather than over-engineering `mlock` for a local dev tool. See [spec 04](specs/04-secrets.md).

docs/FEATURES.md

Lines changed: 14 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -18,27 +18,27 @@ Ordered by value. Effort is person-weeks.
1818

1919
### v1 (part of the full vision)
2020

21-
**1. One-command onboarding (`up` / `bootstrap`) · 3w** — the headline DX promise.
21+
**1. One-command onboarding (`up` / `bootstrap`) · 3w** — the headline DX promise. ([spec 09](specs/09-orchestration-and-onboarding.md))
2222
A single idempotent command takes a fresh checkout (just `workspace.yaml` + per-repo `devstack.yaml`) to a fully running stack: preflight doctor → parallel clone/pull all repos → ensure shared network + services → provision DB roles/buckets → resolve secrets → install local CA → generate compose+Dockerfiles → `compose up -d` every project. Each step is a named, **resumable** phase with a live checklist; re-running skips satisfied phases. Replaces devdock's `config init` + `git clone` + `config docker -g` + `docker up -g` dance with one verb. *This composes every subsystem into one proven path — the strongest reason a team adopts the tool.*
2323

24-
**2. `doctor` diagnostics + preflight · 3w** — the support-load killer.
24+
**2. `doctor` diagnostics + preflight · 3w** — the support-load killer. ([spec 13](specs/13-doctor-diagnostics-and-teardown.md))
2525
Checks the whole runtime contract and prints a categorized, actionable report: docker daemon + correct context (critical on WSL2), `docker compose` ≥ v2.20, git ≥ 2.30, disk for volumes, shared-network health, port conflicts (bind-test), CA trust state (host + Firefox/NSS + Windows on WSL2), secrets-provider reachability, stale ref rows vs live containers, `*.localhost`/resolver per platform. Every failure has a one-line remediation and, where safe, `--fix`. Includes `shared doctor` to reconcile the ledger. *Converts "it's broken" into "run doctor, it tells you exactly what to do."*
2626

27-
**3. Health/readiness gating with dependency ordering · 2w.**
27+
**3. Health/readiness gating with dependency ordering · 2w.** ([spec 10](specs/10-health-readiness-and-ordering.md))
2828
Each template declares a typed healthcheck (tcp/http/exec/`pg_isready`/redis PING) and `dependsOn … condition: healthy`. The orchestrator brings shared services up first, polls health, and only then starts dependents; `up` blocks (per-service spinners + timeout) until healthy or fails fast with the unhealthy service's last logs inlined. *Eliminates the "app crashed because the DB wasn't ready" flakiness; substrate for seed/migrate hooks.*
2929

30-
**4. Service profiles / groups (selective up) · 1.5w.**
30+
**4. Service profiles / groups (selective up) · 1.5w.** ([spec 12](specs/12-service-profiles-and-selective-up.md))
3131
Named groups (`core`, `frontend`, `payments`, `observability`) mapped onto Compose `profiles:` + the shared-service reference graph. `up --profile frontend` starts only that slice + the shared services it transitively `uses`. A `minimal` profile for low-RAM laptops. *On a 16GB laptop nobody runs 12 microservices at once; this leverages the shared-services graph directly.*
3232

33-
**5. Lifecycle hooks (pre/post up, migrate, first-run) · 2w.**
34-
Declarative hooks at `postUp`/`preDown`/`firstRun` (once per provisioned volume, tracked in the state DB so it survives restarts — unlike `initdb.d`)/`postPull`, run on the host or via `compose exec`, with the same `${ref}`/secret interpolation. Canonical uses: DB migrations, first-run seeding, app-key generation, `npm/composer install`. *The missing glue between "containers up" and "app actually works"; generalizes devdock's hard-coded entrypoint logic.*
33+
**5. Lifecycle hooks (pre/post up, migrate, first-run) · 2w.** ([spec 11](specs/11-lifecycle-hooks.md))
34+
Declarative hooks at `preUp`/`postUp`/`preDown`/`firstRun` (once per provisioned volume, tracked in the state DB so it survives restarts — unlike `initdb.d`)/`postPull`, run on the host or via `compose exec`, with the same `${ref}`/secret interpolation. Canonical uses: DB migrations, first-run seeding, app-key generation, `npm/composer install`. *The missing glue between "containers up" and "app actually works"; generalizes devdock's hard-coded entrypoint logic.*
3535

36-
**6. Update notifications + signed self-update · 1w.**
36+
**6. Update notifications + signed self-update · 1w.** ([spec 14](specs/14-self-update-and-migration.md))
3737
Throttled (~daily), opt-out background version check → one-line footer; `self update` via `go-selfupdate` against goreleaser releases with minisign/cosign verification; **detects Homebrew/dpkg-managed installs and refuses to self-replace**, directing to the package manager. *Painless, trustworthy updates matter for shipping security fixes given CVE-prone deps.*
3838

3939
### v2
4040

41-
**7. DB seed / snapshot / restore for shared services · 3w.**
41+
**7. DB seed / snapshot / restore for shared services · 3w.** (distinct from the v1 `db gc` orphan-reaper in [spec 13](specs/13-doctor-diagnostics-and-teardown.md); this item is the richer `db snapshot/restore/reset` data workflow.)
4242
`db snapshot [name]` (pg_dump/mongodump/redis RDB) into a content-addressed store; `db restore <name>`; `db reset` (drop+recreate role/db + replay seed). Per-project (using the per-project role isolation) so snapshots never clobber another project. Optional `db pull` from a sanitized remote dump feeds the `firstRun` hook. *Branch-switching and "I broke my local data" are daily pain; uniquely enabled by per-project-DB-on-shared-postgres.*
4343

4444
**8. Log aggregation + live TUI dashboard · 4w.**
@@ -67,12 +67,12 @@ Generalize the shared stack to run on a remote Docker host (DOCKER_HOST/SSH cont
6767

6868
| # | Feature | Effort | Milestone |
6969
|---|---|---|---|
70-
| 1 | One-command onboarding | 3w | v1 (M6) |
71-
| 2 | `doctor` diagnostics | 3w | v1 (M6) |
72-
| 3 | Health/readiness gating | 2w | v1 (M6) |
73-
| 4 | Service profiles | 1.5w | v1 (M6) |
74-
| 5 | Lifecycle hooks | 2w | v1 (M6) |
75-
| 6 | Self-update + notifications | 1w | v1 (M0/M6) |
70+
| 1 | One-command onboarding | 3w | v1 (M2/M6) · [spec 09](specs/09-orchestration-and-onboarding.md) |
71+
| 2 | `doctor` diagnostics | 3w | v1 (M0/M6) · [spec 13](specs/13-doctor-diagnostics-and-teardown.md) |
72+
| 3 | Health/readiness gating | 2w | v1 (M6) · [spec 10](specs/10-health-readiness-and-ordering.md) |
73+
| 4 | Service profiles | 1.5w | v1 (M6) · [spec 12](specs/12-service-profiles-and-selective-up.md) |
74+
| 5 | Lifecycle hooks | 2w | v1 (M6) · [spec 11](specs/11-lifecycle-hooks.md) |
75+
| 6 | Self-update + notifications | 1w | v1 (M0/M6) · [spec 14](specs/14-self-update-and-migration.md) |
7676
| 7 | DB snapshot/restore | 3w | v2 |
7777
| 8 | Log aggregation + dashboard | 4w | v2 |
7878
| 9 | Devcontainer/IDE integration | 2w | v2 |

0 commit comments

Comments
 (0)