You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs: design v1 feature specs 09-14 and integrate cross-links
Add the remaining v1 (M6) feature specs that previously had only backlog entries: orchestration/onboarding (09), health & ordering (10), lifecycle hooks (11), service profiles (12), doctor & teardown (13), self-update & migration (14). Fixes from adversarial review applied: SDK read-only health probes, db gc in the CLI surface, profiles/groups key disambiguation, preUp + saga phase-count reconciliation. Wire spec links into ARCHITECTURE/FEATURES/ROADMAP/README; resolve Q-NAME (devstack); add Q-PROFILE/Q-SAGA-PARALLEL/Q-HEALTHWAIT/Q-HOOK-SCOPE/Q-DOCTOR-FIX.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: README.md
+8-2Lines changed: 8 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,6 @@
1
1
# devstack
2
2
3
-
> Working name. The repo folder is `devdock-go`; the binary name is a placeholder you can change at any time, and the tool can be invoked under multiple aliases (e.g. `rq`, `uranus`) via the [alias system](docs/specs/07-cli-and-aliasing.md). These docs use **`devstack`** consistently because it matches the config schema (`devstack.yaml`, `apiVersion: devstack/v1`).
3
+
> The binary is **`devstack`** (Go module + GitHub repo `github.com/open-source-cloud/devstack`; the local checkout folder is `devdock-go`). The same binary is invocable under aliases (e.g. `rq`, `uranus`) via the [alias system](docs/specs/07-cli-and-aliasing.md). Config files are `workspace.yaml` + `devstack.yaml` (`apiVersion: devstack/v1`).
4
4
5
5
A single-binary CLI that manages Docker-based development environments and **shares infrastructure across projects** — one warm Postgres / Redis / MinIO for many repos instead of a duplicate stack per project. It is a clean-slate, Go reimplementation of the ideas behind [`devdock`](../devdock) (the Python predecessor), redesigned around a *workspace* of shared services that independent project stacks attach to.
6
6
@@ -18,7 +18,7 @@ A developer working across 8 microservices today runs 8 Postgres containers, 8 R
18
18
19
19
## Status
20
20
21
-
📋**Design / spec phase.**No code yet. This repository currently holds the architecture and specs that will drive implementation (spec-driven development). See the doc index below.
21
+
🚧**M0 foundations implemented; design ongoing.**The spine is in place and green (`make ci`): the CLI tree + `argv[0]` aliasing, the `flock` cross-process lock, the SQLite ledger, XDG/WSL2 handling, a read-only Docker client, and a real `doctor` preflight. The rest of the surface is spec-driven (see the doc index). Try it: `make build && make smoke`.
22
22
23
23
## How it works (one paragraph)
24
24
@@ -42,6 +42,12 @@ A developer working across 8 microservices today runs 8 Postgres containers, 8 R
|`internal/orchestrate`| The multi-phase `up`/bootstrap **saga**: named resumable phases with durable phase-state + compensating rollback; TUI checklist + plain fallback. |
139
+
|`internal/doctor`| Cross-platform capability-probe matrix with one-line remediations and `--fix`; also owns recovery (`--rebuild-state`) and teardown (`workspace destroy`/`uninstall`/`db gc`). See [spec 13](specs/13-doctor-diagnostics-and-teardown.md). |
140
+
|`internal/hooks`| Lifecycle hooks (`preUp`/`postUp`/`preDown`/`firstRun`/`postPull`) with an idempotency ledger; powers migrations/seeding. See [spec 11](specs/11-lifecycle-hooks.md). |
|`internal/migrate`|`devstack import` reading an old devdock `project.yaml` → new `workspace.yaml` + `devstack.yaml` split (optional; high-leverage for the existing user base). |
144
+
|`internal/migrate`|`devstack import` reading an old devdock `project.yaml` → new `workspace.yaml` + `devstack.yaml` split (optional; high-leverage for the existing user base); plus the three-artifact versioning policy + self-update choreography. See [spec 14](specs/14-self-update-and-migration.md). |
145
+
|`internal/version`| Build-stamped identity (`Version`/`Commit`/`Date` via ldflags); consumed by `--version` and the update notifier. See [spec 14](specs/14-self-update-and-migration.md). |
145
146
|`pkg/pluginsdk`| Public plugin contract — **deferred to v2** (out-of-tree secrets/tunnel providers). |
146
147
147
148
Every fast-moving or risky dependency (docker client, template engine, secrets provider, git, trust store) sits behind an internal interface so it can be swapped or vendored.
@@ -156,15 +157,15 @@ These emerged from adversarial review and are the things most likely to bite. Ea
156
157
Two simultaneous `up` invocations (two terminals; IDE + terminal; a watch script) race on network-ensure (TOCTOU between inspect and create), port allocation, ref-count rows, and `CREATE ROLE`. SQLite file locking is unreliable on WSL2/9p. **Mitigation (build first, in M0):** a coarse `gofrs/flock` advisory lock around *every* operation that mutates the ledger or the shared stack; WAL + `busy_timeout`; idempotent guarded SQL; port allocation done inside the lock; `doctor` warns on 9p state dirs. Reads are lock-free snapshots. See [spec 08](specs/08-state-locking-and-lifecycle.md).
157
158
158
159
### 7.2 Partial-failure / rollback for the multi-phase `up`
159
-
`up` runs ~8 phases (clone → network → shared services → provision DB → secrets → CA → generate → compose up). A failure in phase 4 after phase 3 started the shared stack and inserted ref rows leaves a half-up workspace and a lying ledger. **Mitigation:** model `up` as a **saga** with durable phase-state and compensating actions; `doctor --rebuild-state` reconstructs the ledger from live Docker labels.
160
+
`up` runs ~8 phases (clone → network → shared services → provision DB → secrets → CA → generate → compose up). A failure in phase 4 after phase 3 started the shared stack and inserted ref rows leaves a half-up workspace and a lying ledger. **Mitigation:** model `up` as a **saga** with durable phase-state and compensating actions; `doctor --rebuild-state` reconstructs the ledger from live Docker labels. The saga's named phases, the `saga_phase` resumability table, and the lock-granularity rule are detailed in [spec 09](specs/09-orchestration-and-onboarding.md).
160
161
161
162
### 7.3 Teardown / uninstall
162
-
devstack creates artifacts everywhere: the external network, shared volumes (Postgres data!), the SQLite DB, **root CA in host + Firefox + Windows trust stores**, alias symlinks, cloudflared creds, keyring entries, the cache. `devstack workspace destroy` / `devstack uninstall` must reverse **all** of it, with explicit data-loss confirmation for volumes, and must never orphan a CA (a security artifact) or dangling symlinks.
163
+
devstack creates artifacts everywhere: the external network, shared volumes (Postgres data!), the SQLite DB, **root CA in host + Firefox + Windows trust stores**, alias symlinks, cloudflared creds, keyring entries, the cache. `devstack workspace destroy` / `devstack uninstall` must reverse **all** of it, with explicit data-loss confirmation for volumes, and must never orphan a CA (a security artifact) or dangling symlinks. The teardown order and the `--fix`-vs-destroy boundary are owned by [spec 13](specs/13-doctor-diagnostics-and-teardown.md).
163
164
164
165
### 7.4 Migration / versioning (three things evolve independently)
2.**State-DB schema** — versioned migrations table, backup-before-migrate, strictly additive within a major.
167
-
3.**Generated compose format** — `compose-go` re-normalization causes churny diffs; decide whether generated artifacts are committed (deterministic, reviewable) or gitignored (regenerated freely). Either way, golden output controls the churn.
168
+
3.**Generated compose format** — `compose-go` re-normalization causes churny diffs; decide whether generated artifacts are committed (deterministic, reviewable) or gitignored (regenerated freely). Either way, golden output controls the churn. The compatibility guarantees for all three artifacts (config `apiVersion`, state `schema_version`, generated-compose byte-stability) and the self-update choreography tying them together are specified in [spec 14](specs/14-self-update-and-migration.md).
168
169
169
170
### 7.5 Secrets ↔ generation coupling, and in-memory exposure
170
171
The "host env auto-propagates into compose" assumption is **false** (verified). Each secret name must be emitted **per service** into the generated compose (as a valueless `environment: [NAME]` key) and the value passed via `exec.Cmd.Env` — so secrets and generation are coupled and need one owner plus a CI test asserting **no secret value ever lands in any generated file**. Resolved secrets also live as plain Go strings (GC'd, swappable, visible via `/proc/<pid>/environ` to same-user processes). Decision: document the threat model honestly rather than over-engineering `mlock` for a local dev tool. See [spec 04](specs/04-secrets.md).
A single idempotent command takes a fresh checkout (just `workspace.yaml` + per-repo `devstack.yaml`) to a fully running stack: preflight doctor → parallel clone/pull all repos → ensure shared network + services → provision DB roles/buckets → resolve secrets → install local CA → generate compose+Dockerfiles → `compose up -d` every project. Each step is a named, **resumable** phase with a live checklist; re-running skips satisfied phases. Replaces devdock's `config init` + `git clone` + `config docker -g` + `docker up -g` dance with one verb. *This composes every subsystem into one proven path — the strongest reason a team adopts the tool.*
Checks the whole runtime contract and prints a categorized, actionable report: docker daemon + correct context (critical on WSL2), `docker compose` ≥ v2.20, git ≥ 2.30, disk for volumes, shared-network health, port conflicts (bind-test), CA trust state (host + Firefox/NSS + Windows on WSL2), secrets-provider reachability, stale ref rows vs live containers, `*.localhost`/resolver per platform. Every failure has a one-line remediation and, where safe, `--fix`. Includes `shared doctor` to reconcile the ledger. *Converts "it's broken" into "run doctor, it tells you exactly what to do."*
26
26
27
-
**3. Health/readiness gating with dependency ordering · 2w.**
27
+
**3. Health/readiness gating with dependency ordering · 2w.** ([spec 10](specs/10-health-readiness-and-ordering.md))
28
28
Each template declares a typed healthcheck (tcp/http/exec/`pg_isready`/redis PING) and `dependsOn … condition: healthy`. The orchestrator brings shared services up first, polls health, and only then starts dependents; `up` blocks (per-service spinners + timeout) until healthy or fails fast with the unhealthy service's last logs inlined. *Eliminates the "app crashed because the DB wasn't ready" flakiness; substrate for seed/migrate hooks.*
29
29
30
-
**4. Service profiles / groups (selective up) · 1.5w.**
30
+
**4. Service profiles / groups (selective up) · 1.5w.** ([spec 12](specs/12-service-profiles-and-selective-up.md))
31
31
Named groups (`core`, `frontend`, `payments`, `observability`) mapped onto Compose `profiles:` + the shared-service reference graph. `up --profile frontend` starts only that slice + the shared services it transitively `uses`. A `minimal` profile for low-RAM laptops. *On a 16GB laptop nobody runs 12 microservices at once; this leverages the shared-services graph directly.*
Declarative hooks at `postUp`/`preDown`/`firstRun` (once per provisioned volume, tracked in the state DB so it survives restarts — unlike `initdb.d`)/`postPull`, run on the host or via `compose exec`, with the same `${ref}`/secret interpolation. Canonical uses: DB migrations, first-run seeding, app-key generation, `npm/composer install`. *The missing glue between "containers up" and "app actually works"; generalizes devdock's hard-coded entrypoint logic.*
Declarative hooks at `preUp`/`postUp`/`preDown`/`firstRun` (once per provisioned volume, tracked in the state DB so it survives restarts — unlike `initdb.d`)/`postPull`, run on the host or via `compose exec`, with the same `${ref}`/secret interpolation. Canonical uses: DB migrations, first-run seeding, app-key generation, `npm/composer install`. *The missing glue between "containers up" and "app actually works"; generalizes devdock's hard-coded entrypoint logic.*
35
35
36
-
**6. Update notifications + signed self-update · 1w.**
36
+
**6. Update notifications + signed self-update · 1w.** ([spec 14](specs/14-self-update-and-migration.md))
37
37
Throttled (~daily), opt-out background version check → one-line footer; `self update` via `go-selfupdate` against goreleaser releases with minisign/cosign verification; **detects Homebrew/dpkg-managed installs and refuses to self-replace**, directing to the package manager. *Painless, trustworthy updates matter for shipping security fixes given CVE-prone deps.*
38
38
39
39
### v2
40
40
41
-
**7. DB seed / snapshot / restore for shared services · 3w.**
41
+
**7. DB seed / snapshot / restore for shared services · 3w.** (distinct from the v1 `db gc` orphan-reaper in [spec 13](specs/13-doctor-diagnostics-and-teardown.md); this item is the richer `db snapshot/restore/reset` data workflow.)
42
42
`db snapshot [name]` (pg_dump/mongodump/redis RDB) into a content-addressed store; `db restore <name>`; `db reset` (drop+recreate role/db + replay seed). Per-project (using the per-project role isolation) so snapshots never clobber another project. Optional `db pull` from a sanitized remote dump feeds the `firstRun` hook. *Branch-switching and "I broke my local data" are daily pain; uniquely enabled by per-project-DB-on-shared-postgres.*
43
43
44
44
**8. Log aggregation + live TUI dashboard · 4w.**
@@ -67,12 +67,12 @@ Generalize the shared stack to run on a remote Docker host (DOCKER_HOST/SSH cont
0 commit comments