Fix '+ New fleet' first-creation gap — wire manifest creation through deploy_apply (blocks #104's k8s dispatch)

[brettchien]

Context

Discovered while investigating #104 (k8s console onboarding): traced deploy_provision's call chain (deploy_provision → t_provision → provision_from_library → oabctl::studio_api::redeploy) and found redeploy() requires an already-stored manifest (load_manifest() → S3 GetObject at manifests/{namespace}/{name}.yaml) — it only patches an existing manifest's image/bundle_from, it never creates a manifest's first version.

The only place a manifest's first version gets written to S3 is apply.rs::apply_manifests (called via deploy_apply → apply_deployment), which requires the caller to hand it a complete manifest YAML (networking, resources, secrets, runtime — everything create.rs's CLI wizard collects interactively).

The console's "+ New fleet" wizard (deploy.ts) only ever calls deploy_provision, never deploy_apply. If that's the full picture, the console currently has no way to create a genuinely new agent from scratch — it can only "redeploy" (bump image/bundle) an agent that already has a stored manifest, created some other way (today: the CLI's oabctl create wizard).

This is not k8s-specific — it's true for AWS today too. It surfaced during k8s work because adding a k8s branch to deploy_provision would just inherit the same limitation, so k8s support was blocked on deciding how to fix this properly rather than papering over it.

Decision (2026-08-27, Brett)

Of three options discussed (see #104's comment thread for the AWS-only "make K8sDriver mirror the missing behavior" and "cram creation into redeploy" alternatives, both rejected):

Go with: wire manifest creation through deploy_apply's existing create-or-update path (apply_manifests), not deploy_provision/redeploy. redeploy stays what its name says — patch-and-reapply an existing manifest. Creation is a distinct operation with its own entry point, matching the semantic boundary that already exists in the code (deploy_apply = give me an exact manifest, create-or-update; deploy_provision = compose-from-template convenience wrapper around redeploy).

Scope

1. Confirm the trace above is right — rule out a manifest-creation path this investigation missed, before building on the finding.
2. Manifest construction for a first deploy_provision call: compose_named/Bundle (what provision_from_library uses today) only produces files+image_tag, not a full OABServiceManifest (no networking/resources/secrets/runtime). Need either:
   • the console to collect the missing fields itself (bigger UI lift), or
   • sensible defaults ported from create.rs's wizard (same defaults an interactive CLI run would produce for a bare template), so console keeps asking only what it asks today (name/image/template) and the rest is filled in.
3. Orchestration: the console-facing call (still one call, to keep the wizard a single step) needs to branch — no stored manifest yet → build one (via the above) and persist+apply through the same path deploy_apply uses (apply_manifests), not a parallel mechanism; manifest already exists → today's redeploy patch-and-reapply, unchanged.
4. This is the prerequisite for K8s fleet onboarding — console UX for provider selection ('+ New fleet') #104's deploy_provision k8s dispatch — once this lands with a clean AWS-side "create" path through deploy_apply, adding k8s's provider dispatch happens at that same entry point, not redeploy.

Explicit non-goals

• Not re-litigating k8s's own dispatch mechanics (K8sDriver::apply already works standalone, per K8s fleet onboarding — console UX for provider selection ('+ New fleet') #104) — this issue is AWS-path-completion only. K8s dispatch resumes in K8s fleet onboarding — console UX for provider selection ('+ New fleet') #104 once this is done.
• Not redesigning create.rs's CLI wizard — reusing its defaults, not its interactive flow.

[brettchien]

Tick 1: verified the traced call chain (item 1) — confirmed correct, plus one more data point. create.rs's CLI wizard (the only place that creates a brand-new agent today) writes a full manifest.yaml to local disk, then calls crate::apply::run(...) (or tells the operator to run oabctl apply -f manifest.yaml) — same apply.rs::apply_manifests mechanism deploy_apply uses. So: apply.rs::apply_manifests genuinely is the one and only path that writes a manifest's first version, confirmed from a second angle.

Item 2 turned out bigger than scoped, stopping before implementing — see thread for the actual question.

[brettchien]

Progress (tick 1, part 2): opened #112 — made list_vpcs/select_subnets/list_security_groups reusable (pub) and added default_security_group (no-prompt equivalent of run()'s SG create-or-reuse default). Groundwork only, nothing calls these yet.

Note: VPC still has no zero-prompt default — run() always asks interactively, unlike subnet (auto-picked) and SG (defaults to create-new). Worth considering whether VPC should come from per-fleet config (mirroring how region/profile already work in fleets.toml) rather than auto-detected, since an account can have multiple VPCs with no obviously-correct one. Next tick: this question, plus still waiting on Brett's answer (posted to the Discord thread) on whether console should collect the Discord bot token itself or expect a pre-existing secret reference.

[brettchien]

Tick 2: dug into the Discord-token question further and it reframes things — posting before doing more speculative work.

create.rs's generate_config/generate_manifest are CLI-only helpers, unconditionally hardcoding a Discord secret ref. Studio's console never uses them — the console's compose flow (studio_compose::compose_named) builds config.toml from the operator's own authored Library (templates/overlays the operator sets up in Studio itself, persisted per-installation, starts empty). Whatever secrets that config.toml references (Discord or otherwise) is already fully handled by the existing, working push_bundle+inject_pre_seed_hook mechanism provision_from_library uses today — that doesn't need to change.

So the piece #111 is actually missing is narrower than "collect a Discord token": it's building the surrounding OABServiceManifest (networking/resources/runtime — VPC/subnet/SG per #112, capacity provider) when one doesn't exist yet, and applying it via apply_manifests. The open question is whether OABServiceManifest.spec.secrets (the ECS task-definition-level secret injection, distinct from config.toml's own [secrets.refs]) needs to be populated too, or whether agents resolve secrets themselves at runtime regardless of that field — I don't know which without reading manifest.rs's secrets field usage more closely, which I haven't done yet.

Pausing further #111 work rather than guess through two open design questions (this one, and the still-unanswered "does console collect the Discord token or expect a pre-existing secret ref" from tick 1). #112 (VPC/subnet/SG defaults) is safe standalone groundwork either way and stands regardless of how these resolve. Will pick back up once either gets an answer, or investigate manifest.rs's spec.secrets field next tick if nothing else comes in first.

[brettchien]

Tick 2, resolving my own open question: checked manifest.rs's expand() — OABServiceManifest.spec.secrets: HashMap<String, String> defaults to empty if the fleet template doesn't declare any (FleetTemplate.secrets has #[serde(default)]). An empty-secrets manifest is valid — it just means no ECS-level Secrets-Manager env-var injection happens for that service.

This means the Discord-token question doesn't actually block #111's core fix. The manifest-creation path can default spec.secrets to empty (same as it'd inherit from a stored manifest today via redeploy — this isn't a new limitation, just the existing one made explicit for the create-from-scratch case). Whether a given template's agent needs Discord/STT secrets to actually function is the operator's responsibility via their own template's config.toml + pre-existing Secrets Manager entries, same as it already is for every agent provisioned via create.rs today. Not a new gap #111 introduces, not something #111 needs to solve.

So both open questions from tick 1 turn out not to be blockers — resuming implementation. Unblocking finding, not asking for input this time.

[brettchien]

Tick 2 wrap-up: pushed a follow-up commit to #112 — default_vpc() (zero-prompt pick when the account has exactly one default VPC, errors otherwise rather than guessing). With this, #112 now covers no-prompt defaults for all of VPC/subnet/SG.

Both open questions from earlier this tick turned out not to be blockers (secrets: manifest.spec.secrets defaults to empty, valid; template content is the operator's own concern either way). Next tick: actually wire these into a "build a default OABServiceManifest, apply via apply_manifests" function and figure out how the console-facing call branches create-vs-redeploy — the real fix, everything so far has been groundwork.

[brettchien]

Tick 3: opened #113 (stacked on #112) — the actual fix. provision_from_library now checks load_manifest first: existing manifest → today's redeploy patch behavior unchanged; none yet → builds a fresh OABServiceManifest (build_default_manifest, using #112's VPC/subnet/SG/networking defaults) and applies it via a new provision_manifest helper. Console's "+ New fleet" can now genuinely create a new agent, not just redeploy an existing one.

This was the riskiest change of the whole effort (real AWS infra creation) — no local compile possible (same aws-sdk-ec2 OOM), so did a careful field-by-field manual check against manifest.rs's actual struct defs and OABServiceManifest::validate()'s requirements before pushing. Full detail in #113's description.

With #112 → #113, #111 is essentially done pending review/merge. Once #113 lands, #104's k8s deploy_provision dispatch resumes on this same branch point (load_manifest is provider-agnostic; only the "build fresh manifest" + "apply" halves need a Runtime::Kubernetes branch).