Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 6 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -201,15 +201,9 @@ checking its launcher and workspace boundary.

## Where this fits

Imp is one of four repositories that together run persistent agents with
AT Protocol accounts. Imp is the library: typed language-model programs, an
MCP client (`Imp.MCP`), and the ACP server side (`Imp.ACP`). Dwell hosts
residents on Imp and owns their capabilities and grants. Kite exposes one
AT Protocol account as MCP tools and delivers its notifications to Dwell.
Haven is the person's app: an ACP client to residents and to any other agent.
`ex_mcp` (our fork) is the one MCP and ACP implementation all four use.

Dependency direction: Haven → ex_mcp; Kite → ex_mcp; Dwell → Imp → ex_mcp.
Imp is never a service and never depends on the other three. Haven does not
compile against Imp; it launches an Imp program as an external ACP process.
Dwell inherits Imp's ex_mcp revision, so a bump here is a bump for Dwell.
Imp is a library: typed language-model programs, an MCP client (`Imp.MCP`), and
the ACP server side (`Imp.ACP`). It depends on `deepfates/ex_mcp`, a fork of
`ex_mcp`, which is the one MCP and ACP implementation Imp uses. Ordinary Imp
startup opens no protocol endpoint, and Imp is never a service. A host
application owns product lifetimes and decides when to launch an Imp program as
an external ACP process.
56 changes: 26 additions & 30 deletions benchmarks/authorities.json

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
"campaign_id": "local-mlx-sft-lifecycle-banking77-v2-deployment-continuation",
"status": "draft_for_principal_review_not_authorized",
"output_artifact": "benchmarks/results/local-mlx-lifecycle-banking77-v2-deployment-continuation.json",
"note": "Paths under benchmarks/results/ are run write targets; that directory is not tracked, so any predecessor artifact named here is unpublished history recorded for provenance, not a file in this repository.",
"purpose": "Complete only the deployment, base-versus-fused evaluation, persistence, fresh-process reload, identity, and cleanup measurements that stopped V2 left unobserved. This draft does not authorize a server launch.",
"closed_predecessor": {
"campaign_id": "local-mlx-sft-lifecycle-banking77-v2",
Expand Down
1 change: 1 addition & 0 deletions benchmarks/config/local-mlx-lifecycle-banking77-v2.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
"campaign_id": "local-mlx-sft-lifecycle-banking77-v2",
"status": "draft_for_principal_review_not_authorized",
"output_artifact": "benchmarks/results/local-mlx-lifecycle-banking77-v2.json",
"note": "Paths under benchmarks/results/ are run write targets; that directory is not tracked, so any predecessor artifact named here is unpublished history recorded for provenance, not a file in this repository.",
"purpose": "Run the already-approved one-model Banking77 base-versus-fused lifecycle probe with the JSONL that current public Imp actually renders. This is a draft only: it does not authorize training, inference, or a server.",
"closed_predecessor": {
"campaign_id": "local-mlx-sft-lifecycle-banking77-v1",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
"status": "preregistered_not_run",
"authorization": "not_launched",
"output_artifact": "benchmarks/results/support-ticket-lift-openrouter-free-v2-20260725.json",
"note": "Paths under benchmarks/results/ are run write targets; that directory is not tracked, so any predecessor artifact named here is unpublished history recorded for provenance, not a file in this repository.",
"purpose": "Repeat the stopped v1 preflight with enough bounded output room to remove the observed 64-token ceiling ambiguity. This is a measurement repair, not score-directed tuning and not C3 optimizer-effectiveness evidence.",
"predecessor": {
"campaign": "support-ticket-baseline-vs-labeled-few-shot",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
"status": "preregistered_not_run",
"authorization": "not_launched",
"output_artifact": "benchmarks/results/support-ticket-lift-openrouter-free-v3-20260725.json",
"note": "Paths under benchmarks/results/ are run write targets; that directory is not tracked, so any predecessor artifact named here is unpublished history recorded for provenance, not a file in this repository.",
"purpose": "Measure the already-frozen support-ticket baseline versus LabeledFewShot design on the one exact-free route selected by an independent synthetic typed-format canary. This preregistration is based only on schema completion and guarded route operation, not benchmark scores, and does not authorize launch.",
"closed_predecessors": [
{
Expand Down
16 changes: 8 additions & 8 deletions decisions.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,11 @@ not necessarily when it was made.
| 2026-09-13 | An ACP tool kind is derived from the MCP `ToolAnnotations` the server declares, not from a host table keyed by tool name; a tool that declares no hint gets `nil` rather than a guess, and an explicit `:tool_kinds` entry still wins. | `lib/imp/acp/tool_kind.ex` moduledoc, `test/acp_options_tool_kinds_test.exs`. A name table goes stale the moment a server publishes a tool it does not name, and that is exactly when a permission mode that would have asked does not ask. | In force. | Does not retire. |
| 2026-09-13 | On `session/load` and `session/resume` the session is installed with the `_meta` it was created with, not the `_meta` the request carried; both are passed to the program factory as `:meta` and `:requested_meta`. | `lib/imp/acp.ex` moduledoc, `test/acp_imp_acp_test.exs`. A session's history and transcript belong to the configuration that produced them, so a resume that silently adopts a different one replays one configuration's transcript as another's; only the factory knows which of its own `_meta` keys are identity-bearing, so it is given both and decides whether to refuse. | In force. | Does not retire. |
| 2026-09-13 | `Imp.ACP` (the ACP session/program adapter) is part of Imp; the separate `imp_acp` package is retired with no compatibility shim and no second implementation. | Retired `imp_acp` README, `AGENTS.md`. One protocol stack, one owner of the ExMCP dependency. | In force, and released in `v0.4.0`. | The "unreleased" caveat retired at `v0.4.0`. The absorption itself does not retire. |
| 2026-09-13 | Absorption renames: `IMP_ACP_PATH` becomes `IMP_PATH` (copied workspace example), `IMP_ACP_ROOT` becomes `IMP_ROOT` (Haven preset), launcher is `examples/workspace_agent/scripts/workspace-agent-acp`. Module names `Imp.ACP`, `Imp.ACP.Host`, `Imp.ACP.MCP` and the `deepfates.com/imp-acp` wire metadata namespace keep their names. | Retired `imp_acp` README. Saved executable and environment references are updated explicitly, not by rewriting contact history. | In force. | Does not retire. |
| 2026-09-13 | Absorption renames: `IMP_ACP_PATH` becomes `IMP_PATH` (copied workspace example), `IMP_ACP_ROOT` becomes `IMP_ROOT` (host application preset), launcher is `examples/workspace_agent/scripts/workspace-agent-acp`. Module names `Imp.ACP`, `Imp.ACP.Host`, `Imp.ACP.MCP` and the `deepfates.com/imp-acp` wire metadata namespace keep their names. | Retired `imp_acp` README. Saved executable and environment references are updated explicitly, not by rewriting contact history. | In force. | Does not retire. |
| 2026-09-13 | The workspace agent's session store keeps the `imp_acp/workspace_agent/sessions` directory name. | `examples/workspace_agent/README.md`. Preserves saved sessions; it is a storage location, not a dependency on the retired package. | In force. | Saved sessions are migrated, or the owner accepts losing restart continuity for them. |
| 2026-09-13 | Ordinary Imp boot starts no protocol listener, subprocess or remote connection and does not start the ExMCP application. Protocol entry points (`Imp.ACP.*`, non-empty `Imp.MCP.connect/2`) start it explicitly. Releases using them declare `applications: [ex_mcp: :load]`. | `mix.exs` dependency comment, `docs/PRODUCTION_OPERATIONS.md` "Protocol runtime in releases", commit `728f8c77`. Prediction and optimizer processes must not open listeners or acquire protocol boot output. | In force. | Does not retire while Imp is a library inside a host application. |
| 2026-09-13 (pin dated 2026-09-15) | ExMCP is the `deepfates/ex_mcp` fork at `7285330b490476cc153dd60fb9adac9cd39d4a94`, the one ref shared by kite, haven, dwell and imp. Do not move it independently of the other consumers. | `mix.exs` `ex_mcp_dependency/0`; the fork's `FORK.md` records each patch (byte-safe stdio frames, owner-bound subprocess cleanup, ACP delivery barriers, per-connection HTTP trust) with its failure. Workshop standing decision "one ref, not four". | In force. | Each patch is upstreamed or made unnecessary and the dependency becomes a released Hex version; until then the ref moves for all four repositories in one coordinated change. |
| 2026-09-13 | `mix.exs` resolves ExMCP three ways: a bundled `vendor/ex_mcp` if present, else `EX_MCP_PATH`, else the GitHub pin. | `mix.exs`. No reason is recorded beside the conditional; the bundled path is presumably for the source package and the env override for fork development (*unverified*, inferred from `test/acp_imp_acp_test.exs` and the workspace agent test passing `EX_MCP_PATH` through). | In force, reason unrecorded. Conflicts with the workshop rule that a new path deletes the old one; owner to rule. | The owner decides whether two of the three paths go, or records why all three stay. |
| 2026-09-13 (pin dated 2026-09-15) | ExMCP is the `deepfates/ex_mcp` fork at `7285330b490476cc153dd60fb9adac9cd39d4a94`, the one ref shared with the other consumers of the same fork. Do not move it independently of them. | `mix.exs` `ex_mcp_dependency/0`; the fork's `FORK.md` records each patch (byte-safe stdio frames, owner-bound subprocess cleanup, ACP delivery barriers, per-connection HTTP trust) with its failure. Owner ruling, 2026-09-11: one ref, not one per consumer. | In force. | Each patch is upstreamed or made unnecessary and the dependency becomes a released Hex version; until then the ref moves for every consumer of the fork in one coordinated change. |
| 2026-09-13 | `mix.exs` resolves ExMCP three ways: a bundled `vendor/ex_mcp` if present, else `EX_MCP_PATH`, else the GitHub pin. | `mix.exs`. No reason is recorded beside the conditional; the bundled path is presumably for the source package and the env override for fork development (*unverified*, inferred from `test/acp_imp_acp_test.exs` and the workspace agent test passing `EX_MCP_PATH` through). | In force, reason unrecorded. Conflicts with the standing rule (2026-09-11) that a new path deletes the old one; owner to rule. | The owner decides whether two of the three paths go, or records why all three stay. |
| 2026-09-13 | Known interoperability limit, not a feature: when an HTTP MCP server selects protocol version `2025-03-26`, the pinned fork's `notifications/initialized` can still carry the client's `2025-11-25` header, and a strict older server may reject the session. Compatibility with strict older HTTP servers is not claimed. | `docs/PRODUCTION_OPERATIONS.md`, commit `902a5546`. Inherited from the fork's connection manager. | Open defect, documented. | The fork settles the HTTP version before sending that notification and a test against a strict older server passes. |
| 2026-09-13 | Imported MCP tool calls disable generic transport retries, use ExMCP's `:safe_only` broken-stream policy, and never repeat an ambiguous write. The retired Imp-specific retry, backoff and Retry-After options are rejected, not ignored, and are not reimplemented above ExMCP. | `docs/PRODUCTION_OPERATIONS.md`. A timeout does not say whether the call ran; replay needs a real server idempotency contract, which is the application's to establish. | In force. | Does not retire. |
| 2026-09-13 | Tool errors return `{:error, {:mcp_tool_error, envelope}}` with the original content, codes and operation identifiers, so refusal, authorization refusal and indeterminate effect stay distinguishable. The old text-only error tuple is gone. | `docs/PRODUCTION_OPERATIONS.md`. A boundary declares its failure classes. | In force. | Does not retire. |
Expand All @@ -34,8 +34,8 @@ not necessarily when it was made.
| 2026-07-17 | Evidence-infrastructure tests (`:evidence_infrastructure`) are excluded from the default `mix test`. | `CONTRIBUTING.md`. They need full git history, pinned DSPy Python environments and sometimes provider credentials, none of which a fresh clone has. | In force. | Does not retire. |
| 2026-07-10 | ReqLLM is the provider transport boundary; upstream algorithm names keep upstream semantics; deliberate Elixir-native alternatives get a distinct contract and rationale; fixtures and symbol presence never become parity claims. | `CONTRIBUTING.md` "Design Standard". | In force. | Does not retire. |
| 2026-07-07 | `mix` aliases (`check`, `package.check`, ...) exist only in a source checkout (detected by `test/package_contract_test.exs` being present); the shipped package carries runtime sources only. | `mix.exs` `aliases/0` and its comment (detection entered in commit `c193433f`). Local benchmark and evidence control files stay out of the consumer's dependency tree. | In force. | Does not retire. |
| 2026-08-22 | `CONTRIBUTING.md` says "Tickets record unfinished work, not product truth", and `.tickets/` holds 57 tracked files. | `CONTRIBUTING.md`; `git ls-files .tickets`. | **Conflicts** with the workshop standing decision (2026-09-11) that no ticket file, log or diary belongs in a repository. Not acted on here; owner to rule. | The owner deletes `.tickets/` (and the sentence) or reopens the workshop decision. |
| 2026-09-11 (workshop) | No OAuth for now, neither MCP's nor ATProto's. | Workshop `AGENTS.md`. Imp carries `Imp.ACP.DemoMCPOAuthPlug` and a demo MCP HTTP server mix task used only by its own tests to exercise ExMCP's PKCE client; ordinary use has no OAuth path. | In force. | A multi-tenant reason appears; that is Haven's question, not Imp's. |
| 2026-09-11 (workshop) | Nothing leaves these repositories: no issues, pull requests or messages to upstream maintainers (`azmaveth/ex_mcp` included). Fixes live in the fork with their reason beside them. | Workshop `AGENTS.md`, the fork's `FORK.md`. | In force. | The owner says otherwise. |
| 2026-09-11 (workshop) | Tests falsify: revert the change and watch the named test fail before believing it. The owner's machine is a consumer, not a privileged insider: whatever a stranger must do to run this, the owner does the same. | Workshop `AGENTS.md`. The `stranger` CI job runs `deps.get` and `mix check` from a cold checkout for the same reason. | In force. | Does not retire. |
| 2026-09-11 (workshop) | Work is a pull request on a topic branch; working notes go in the pull request or nowhere. Adopting a new path deletes the old one in the same change. | Workshop `AGENTS.md`. | In force. | Does not retire. |
| 2026-08-22 | No ticket file, log or diary belongs in this repository; unfinished work is a pull request on a topic branch. | `CONTRIBUTING.md` "Unfinished work is a pull request on a topic branch; there is no ticket file in this repository." An earlier `.tickets/` directory and the `CONTRIBUTING.md` sentence that justified it were both removed. | Resolved: `git ls-files .tickets` is empty. | The owner reopens it. |
| 2026-09-11 | No OAuth for now, neither MCP's nor ATProto's. | Owner ruling, 2026-09-11. Imp carries `Imp.ACP.DemoMCPOAuthPlug` and a demo MCP HTTP server mix task used only by its own tests to exercise ExMCP's PKCE client; ordinary use has no OAuth path. | In force. | A multi-tenant reason appears; that is a host application's question, not Imp's. |
| 2026-09-11 | Outside contributions are welcome through GitHub issues and pull requests on this repository. Patches to the `deepfates/ex_mcp` fork are upstreamed to `azmaveth/ex_mcp` only by the owner; until then a fix lives in the fork with its reason beside it. | Owner ruling, 2026-09-11, amended 2026-09-17 when the repository went public; the fork's `FORK.md` records each patch with its failure. | In force. | The owner says otherwise. |
| 2026-09-11 | Tests falsify: revert the change and watch the named test fail before believing it. The owner's machine is a consumer, not a privileged insider: whatever a stranger must do to run this, the owner does the same. | Owner ruling, 2026-09-11. The `stranger.check` CI job in `.github/workflows/ci.yml` runs `deps.get` and `mix check` from a cold checkout for the same reason. | In force. | Does not retire. |
| 2026-09-11 | Work is a pull request on a topic branch; working notes go in the pull request or nowhere. Adopting a new path deletes the old one in the same change. | Owner ruling, 2026-09-11; `CONTRIBUTING.md` "Unfinished work is a pull request on a topic branch". | In force. | Does not retire. |
7 changes: 3 additions & 4 deletions docs/differentials/COMBEE_FIDELITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -217,10 +217,9 @@ on 2026-07-13 retained 8/8, 4/8, and 6/8 records respectively with 4, 1, and 3
calls.

Live mode is gated by `COMBEE_PREFLIGHT_MODE=live` and
`COMBEE_LIVE_PROVIDER=1`. The bounded run used pinned
`openai:gpt-4.1-mini-2025-04-14` and is stored at
`benchmarks/results/gepa-combee-preflight-live-20260713T231824Z.json`. All arms
retained 8/8 correct answers. Naive large-batch took 2.62 seconds, 1 call, 755
`COMBEE_LIVE_PROVIDER=1`. The bounded run on 2026-07-13 used pinned
`openai:gpt-4.1-mini-2025-04-14`; its result JSON is unpublished history and is
not retained in this repository. All arms retained 8/8 correct answers. Naive large-batch took 2.62 seconds, 1 call, 755
tokens, and $0.000450; ComBee took 4.25 seconds, 3 calls, 1,909 tokens, and
$0.001228; small-batch took 6.75 seconds, 4 calls, 1,266 tokens, and $0.000841.
Usage and cost came from ReqLLM telemetry.
Expand Down
11 changes: 5 additions & 6 deletions docs/differentials/CONFIDENCE_CALIBRATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,8 +100,8 @@ evidence gates and remains negative.

## July 13 Evidence

`benchmarks/results/confidence-calibration-live-20260713T225422Z.json` is the
fresh authoritative run. All gates passed:
The 2026-07-13 live run was the fresh authoritative run; its result JSON is
unpublished history and is not retained in this repository. All gates passed:

- calibration outcomes: 142 correct, 58 incorrect;
- calibration occupancy: 7 occupied bins, 6 supported bins;
Expand All @@ -121,10 +121,9 @@ those source sets are disjoint rather than paired.

The earlier July 13 support-routing fixture was all correct and used a one-bin
mapping. Its zero post-calibration Brier score and ECE are not evidence of
learned calibration. The retained
`benchmarks/results/confidence-calibration-assessment-20260713.json` records the
rejection and source-run digests; it is superseded as calibration evidence, not
rewritten as a successful result.
learned calibration. The 2026-07-13 assessment that recorded the rejection and
the source-run digests is unpublished history; that observation is superseded as
calibration evidence, not rewritten as a successful result.

This remains a narrow operational probe. It does not establish calibration on
other datasets, prompts, providers, model versions, or deployment populations.
Expand Down
7 changes: 4 additions & 3 deletions docs/differentials/MULTIMODAL_FIDELITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,10 @@ It records six dispatches in this run, zero resumed rows, six durable rows, and
All rows have distinct ReqLLM request IDs, OpenAI request IDs, and OpenAI
Responses IDs. The claim gate has no rejections.

`benchmarks/results/multimodal-quality-live-20260713T215119Z.json` is
invalidated and removed. Its checkpoint schema and pre-dispatch shape evidence
did not exclude forged minimal rows, so its result must not be used as proof.
The 2026-07-13 multimodal quality live run is invalidated and removed, and its
result JSON is unpublished history. Its checkpoint schema and pre-dispatch shape
evidence did not exclude forged minimal rows, so its result must not be used as
proof.

## Manifest Contract

Expand Down
8 changes: 4 additions & 4 deletions docs/differentials/PLAYBOOK_OPTIMIZER.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,10 +87,10 @@ without converting a failed lift into a success claim.

### Release Evidence

The committed seed-61 campaign in
`benchmarks/results/playbook-equation-balancer-live-v11.json` binds to commit
`37f58dc6e3110606b5b08d01124b2cfb1ef52566` and records SHA-256 identities for
all implementation sources. Its source/group-disjoint results were:
The seed-61 campaign bound to commit
`37f58dc6e3110606b5b08d01124b2cfb1ef52566` and recorded SHA-256 identities for
all implementation sources. Its result JSON is unpublished history and is not
retained in this repository. Its source/group-disjoint results were:

| Split | Baseline | Candidate | Solver calls |
|---|---:|---:|---:|
Expand Down
Loading
Loading