ci: harden workflows and add production execution plan

.github/workflows/ci.yml

@@ -4,19 +4,30 @@ on:
   push:
     branches: [main]
   pull_request:
+    branches: [main]
 
 permissions:
   contents: read
 
+concurrency:
+  group: ci-${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
 jobs:
   test:
     runs-on: ubuntu-latest
+    timeout-minutes: 10
     steps:
-      - uses: actions/checkout@v7
-      - uses: actions/setup-python@v5
+      - name: Checkout
+        uses: actions/checkout@v7
+
+      - name: Set up Python
+        uses: actions/setup-python@v6
         with:
           python-version: '3.12'
+
       - name: Unit tests
         run: python -m unittest discover -s src -p 'test_*.py' -v
+
       - name: Compile check
         run: python -m compileall -q src

.github/workflows/codeql.yml

@@ -12,18 +12,23 @@ permissions:
   contents: read
   security-events: write
 
+concurrency:
+  group: codeql-${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
 jobs:
   analyze:
     name: Analyze GitHub Actions
     runs-on: ubuntu-latest
+    timeout-minutes: 15
     steps:
       - name: Checkout
         uses: actions/checkout@v7
 
       - name: Initialize CodeQL
-        uses: github/codeql-action/init@v3
+        uses: github/codeql-action/init@v4
         with:
           languages: actions
 
       - name: Perform CodeQL Analysis
-        uses: github/codeql-action/analyze@v3
+        uses: github/codeql-action/analyze@v4

.github/workflows/dependency-review.yml

@@ -7,9 +7,14 @@ on:
 permissions:
   contents: read
 
+concurrency:
+  group: dependency-review-${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
 jobs:
   dependency-review:
     runs-on: ubuntu-latest
+    timeout-minutes: 10
     steps:
       - name: Checkout
         uses: actions/checkout@v7

EXEC-PLANNING.md

@@ -0,0 +1,165 @@
+# ZLoop Execution Planning
+
+This document is the implementation contract for moving ZLoop from the current reference runtime to a production-grade closed-loop platform.
+
+## Current baseline
+
+- Provider-neutral Python loop engine exists.
+- Loop lifecycle is bounded and explicit.
+- Independent verifier/reviewer roles are defined.
+- JSON schemas cover loop state and verification results.
+- JSONL persistence provides a reference memory backend.
+- CI, CodeQL, Dependency Review, and Dependabot are configured.
+
+## Execution order
+
+### EP-01 Durable state and recovery
+
+**Goal:** a run can survive process interruption without losing correctness.
+
+Deliverables:
+- `StateStore` interface with atomic load/save/update semantics.
+- SQLite backend for local use.
+- PostgreSQL backend contract for production.
+- run checkpoints after every state transition.
+- resume-from-checkpoint API.
+- cancellation/deadline state.
+- durable idempotency records for mutating actions.
+
+Acceptance gates:
+- restart during every non-terminal state resumes safely;
+- duplicate side effects are rejected/reconciled;
+- corrupted/incomplete checkpoints fail closed;
+- state transition tests cover all legal and illegal transitions.
+
+### EP-02 Model router and context governor
+
+**Goal:** make loops affordable and provider-neutral.
+
+Deliverables:
+- model provider adapter protocol;
+- per-role model profiles;
+- cheap-first routing with bounded fallback;
+- token/cost ledger;
+- context compaction;
+- structured-output schema validation;
+- provider failure classification.
+
+Acceptance gates:
+- hard budget cannot be exceeded silently;
+- retries do not repeat identical expensive context;
+- invalid structured output is repaired or escalated;
+- fallback cannot bypass policy or permissions.
+
+### EP-03 Isolated execution runtime
+
+**Goal:** mutating agents cannot collide or escape assigned scope.
+
+Deliverables:
+- worktree manager;
+- workspace ownership model;
+- command runner boundary;
+- timeout/resource limits;
+- action audit events;
+- mutation permission checks.
+
+Acceptance gates:
+- parallel workers cannot overwrite one another;
+- unauthorized mutations fail closed;
+- every external side effect has an idempotency key;
+- cancellation terminates active execution cleanly.
+
+### EP-04 Verification pipeline
+
+**Goal:** completion is decided by evidence, not by the maker.
+
+Deliverables:
+- acceptance criteria representation;
+- verifier adapter registry;
+- test/build/type/security check adapters;
+- reviewer severity model;
+- evidence fingerprints;
+- no-progress detector;
+- bounded repair planner.
+
+Acceptance gates:
+- executor cannot set SHIPPED directly;
+- INCONCLUSIVE never becomes PASS;
+- repeated failure signature triggers handoff;
+- regression verification is mandatory after repair.
+
+### EP-05 GitHub closed loop
+
+**Goal:** first real connector-backed vertical slice: fix a PR until required checks pass or escalation is required.
+
+Flow:
+`discover PR → inspect checks → plan smallest repair → worktree edit → test → push → observe checks → repair or ship`
+
+Deliverables:
+- GitHub read adapter;
+- scoped branch mutation adapter;
+- check-status adapter;
+- PR summary/update adapter;
+- approval boundary for high-impact changes.
+
+Acceptance gates:
+- never pushes to an unauthorized branch;
+- never merges when required checks/review gates are unmet unless explicit policy permits it;
+- stale head SHA is detected before mutation;
+- all mutations are auditable and idempotent.
+
+### EP-06 Fleet orchestration
+
+**Goal:** bounded specialist parallelism without uncontrolled fan-out.
+
+Deliverables:
+- specialist registry;
+- scoped child budgets;
+- max depth/fan-out policy;
+- worktree-per-mutating-agent;
+- merge/conflict coordinator;
+- independent QA role.
+
+Acceptance gates:
+- child agents cannot exceed parent scope or remaining budget;
+- fan-out limits are enforced;
+- overlapping diffs are detected before integration;
+- global verifier owns final acceptance.
+
+### EP-07 Production observability and security
+
+**Goal:** every loop can be operated and audited in production.
+
+Deliverables:
+- structured audit schema;
+- OpenTelemetry traces;
+- metrics for duration, retries, failures, tokens, cost, handoffs;
+- secret redaction;
+- RBAC/tenant boundaries;
+- SBOM/provenance/release evidence.
+
+Acceptance gates:
+- secrets do not appear in logs/state/artifacts;
+- every terminal state has a reason and evidence trail;
+- SLO-relevant metrics are queryable;
+- production mutation permissions are deny-by-default.
+
+## Definition of done
+
+No execution package is complete until:
+1. implementation is present;
+2. deterministic tests pass;
+3. failure paths are covered;
+4. security/permission boundaries are reviewed;
+5. docs and examples are updated;
+6. verification evidence is attached;
+7. rollback/handoff behavior is defined.
+
+## Non-goals for early releases
+
+- unbounded autonomous exploration;
+- unrestricted production mutation;
+- autonomous budget increases;
+- self-modifying acceptance criteria;
+- verifier bypasses;
+- unlimited subagent spawning.

ROADMAP.md

@@ -1,31 +1,89 @@
-# Roadmap
-
-This template is intended to stay generic while providing production-grade repository foundations.
-
-## Foundation
-
-- [x] Repository documentation baseline
-- [x] Security and contribution policies
-- [x] Issue and pull request templates
-- [x] CI and security workflow baseline
-- [x] Dependabot configuration
-- [x] Release workflow
-- [x] Docker and task-runner placeholders
-- [x] Architecture and development documentation structure
-
-## Future optional modules
-
-- [ ] Language-specific starter packs
-- [ ] Infrastructure-as-code starter packs
-- [ ] Kubernetes and Helm starter packs
-- [ ] SBOM and provenance workflows
-- [ ] Release signing and artifact attestation
-- [ ] OpenSSF Scorecard workflow
-- [ ] Container vulnerability scanning
-- [ ] Documentation site starter
-- [ ] Monorepo profile
-- [ ] Service/API profile
-- [ ] Web application profile
-- [ ] Library/SDK profile
-
-Generated repositories should adopt only the modules appropriate to their stack and threat model.
+# ZLoop Roadmap
+
+ZLoop is a bounded, verifiable loop-engineering runtime for agentic work. The roadmap prioritizes closed-loop reliability, independent verification, durable state, cost control, and auditable execution before broader autonomy.
+
+## Phase 0 — Repository and CI baseline
+
+- [x] Core loop engine and tests
+- [x] Agent role contracts
+- [x] Reusable skill contracts
+- [x] Loop state / verification schemas
+- [x] GitHub Actions CI
+- [x] Dependabot
+- [x] Dependency Review
+- [x] CodeQL
+- [x] Upgrade checkout to v7
+- [x] Upgrade setup-python to v6
+- [x] Upgrade CodeQL Action to v4
+- [x] Add workflow concurrency cancellation and timeouts
+
+## Phase 1 — Durable runtime
+
+- [ ] Replace JSONL-only persistence with a durable state-store interface
+- [ ] Add SQLite reference backend
+- [ ] Add PostgreSQL production backend
+- [ ] Add resumable runs and checkpoint recovery
+- [ ] Add cancellation and deadlines
+- [ ] Add durable idempotency records for side effects
+- [ ] Add retry backoff and failure classification
+- [ ] Add deterministic state-machine transition tests
+
+## Phase 2 — Model and context routing
+
+- [ ] Provider-neutral model adapter interface
+- [ ] Cheap-model-first routing policy
+- [ ] Fallback chains and provider health scoring
+- [ ] Per-stage model profiles
+- [ ] Token and cost forecasting
+- [ ] Context compaction and duplicate-read detection
+- [ ] Structured-output validation and repair
+
+## Phase 3 — Execution isolation and connectors
+
+- [ ] Git worktree manager
+- [ ] Sandbox execution boundary
+- [ ] GitHub connector adapter
+- [ ] CI/check adapter
+- [ ] Issue/ticket adapter boundary
+- [ ] Database/staging API adapter boundary
+- [ ] Capability-based permissions
+- [ ] Human approval gate for remote/high-impact mutations
+
+## Phase 4 — Verification and repair
+
+- [ ] Acceptance-criteria compiler
+- [ ] Deterministic verification matrix
+- [ ] Independent verifier enforcement
+- [ ] Security/static-analysis gate adapters
+- [ ] Reviewer severity model
+- [ ] Failure-signature catalog
+- [ ] No-progress detection using evidence fingerprints
+- [ ] Bounded repair planner
+- [ ] Regression verification before ship
+
+## Phase 5 — Fleet orchestration
+
+- [ ] Specialist/subagent registry
+- [ ] Scoped subgoals and per-agent budgets
+- [ ] Parallel worktree execution
+- [ ] Cross-agent conflict detection
+- [ ] Fleet-level cost governor
+- [ ] Quorum verification for critical operations
+- [ ] Dynamic delegation with hard fan-out limits
+
+## Phase 6 — Production operations
+
+- [ ] Structured audit event schema
+- [ ] OpenTelemetry traces
+- [ ] Metrics and SLOs
+- [ ] Cost ledger and budget dashboards
+- [ ] RBAC / tenant boundaries
+- [ ] Secret isolation
+- [ ] SBOM and provenance
+- [ ] Artifact attestation/signing
+- [ ] OpenSSF Scorecard
+- [ ] Container vulnerability scanning where applicable
+
+## Release gates
+
+A phase is complete only when its implementation has tests, documentation, failure-path coverage, security review, and observable evidence. ZLoop must never redefine acceptance criteria or bypass verification merely to reach a successful terminal state.