deepfates · deepfates · Sep 17, 2026 · Sep 17, 2026
@@ -4,6 +4,17 @@ User-visible changes to Imp are recorded here.
 
 ## Unreleased
 
+## 0.4.0 — 2026-09-17
+
+- `Imp.ACP` and `Imp.MCP.connect/2` are part of Imp. The separate `imp_acp`
+  package is retired: `Imp.ACP.start_link/1` and `Imp.ACP.run/1` expose an
+  ordinary Imp program to an ACP host, and `Imp.MCP.connect/2` imports
+  authorized MCP servers through ExMCP as ordinary tools with explicit
+  connection cleanup. There is no compatibility shim; a consumer that depended
+  on `imp_acp` depends on `imp` alone and changes the module prefix. ExMCP is
+  declared `runtime: false`, so an OTP release that uses either must list
+  `applications: [ex_mcp: :load]` in its release definition. Ordinary Imp
+  startup still starts no protocol endpoint.
 - A map or list value in a prompt, including a structured tool result, now
   renders the way DSPy renders a dict: `json.dumps(..., ensure_ascii=False)`
   with Python's default separators, complete. It was `inspect/1` at its
@@ -88,6 +99,24 @@ User-visible changes to Imp are recorded here.
   `{:mcp_tools_list_failed, server, {:invalid_mcp_tools_response, shape}}`
   rather than as a bare `{:invalid_mcp_tools_response, shape}` that named no
   server.
+- The repository is public. Installing at a tag needs no credentials, and the
+  README, docs, and livebooks no longer describe a private source release.
+- Removed the evidence-certification bookkeeping from the source checkout. It
+  never shipped in the package, so a consumer sees no change; the benchmark
+  harness it wrapped is unchanged.
+- Added [Benchmarks](https://github.com/deepfates/imp/blob/main/docs/BENCHMARKS.md)
+  and its [results table](https://github.com/deepfates/imp/blob/main/benchmarks/RESULTS.md).
+  Every number this repository publishes is one row in that table, carrying
+  the dataset and its license, the model, the provider, the date, the commit,
+  and the command that produced it; prose elsewhere cites a row rather than
+  restating a number. The benchmarks page says what each command needs from
+  you — key, Python environment, time, rough cost — and separates a row a
+  stranger can re-measure with an API key from one that only recomputes
+  statistics from committed rows, and from the claims that cannot be
+  re-measured at all. Neither page publishes an aggregate or a release score.
+  The tutorial's rows were re-measured live for this release, a month after the
+  first run, and both runs are recorded. They live in the repository, not in
+  the installed package.
 
 ## 0.3.2 — 2026-09-01
 

@@ -6,12 +6,6 @@ ordinary OTP application. It brings the central idea of
 [DSPy](https://dspy.ai)—programming behavior and improving it from examples
 rather than hand-editing prompts—to the BEAM.
 
-**Release state.** The released tag `v0.3.2` does not contain `Imp.ACP`. The
-integrated protocol adapters — `Imp.ACP` and `Imp.MCP.connect/2`, absorbed from
-the retired `imp_acp` package — exist only on `main` until the next tag.
-Installing `v0.3.2` gives you the library described below without them; using
-the adapters today means depending on `main` (see [Install](#install)).
-
 Here, “typed” means required inputs are checked and model outputs are parsed
 and validated against the signature before application code receives them.
 For DSPy compatibility, a supplied input whose value disagrees with its
@@ -135,13 +129,9 @@ Imp is not published to Hex. It is installed from this public repository, at
 an immutable tag, with no credentials:
 
 ```elixir
-{:imp, github: "deepfates/imp", tag: "v0.3.2"}
+{:imp, github: "deepfates/imp", tag: "v0.4.0"}
 ```
 
-That tag does not contain `Imp.ACP` or `Imp.MCP.connect/2`; a tag that does
-will be cut. Until then, depend on `{:imp, github: "deepfates/imp", branch:
-"main"}` if you need the adapters.
-
 ExMCP is declared `runtime: false`, so an OTP release that uses `Imp.ACP` or
 `Imp.MCP` must list `applications: [ex_mcp: :load]` in its release definition;
 see [protocol runtime in releases](docs/PRODUCTION_OPERATIONS.md#protocol-runtime-in-releases).
@@ -152,9 +142,9 @@ modules are ported from.
 Use `{:imp, path: "path/to/imp"}` only while developing against a local
 checkout. Imp requires Elixir `~> 1.19`. Commit your application's `mix.lock`;
 the Git tag fixes Imp's source, while normal Mix constraints may otherwise
-resolve newer compatible transitive versions. Version `0.3.2` contains the
-breaking `0.3` changes from `0.2.1`; see the [release notes](RELEASE_NOTES.md)
-when upgrading.
+resolve newer compatible transitive versions. Version `0.4.0` contains
+breaking changes from `0.3.2`; see the [release notes](RELEASE_NOTES.md) when
+upgrading.
 
 Imp uses [ReqLLM](https://hex.pm/packages/req_llm) for model providers. The
 examples use OpenAI, but programs are not tied to that provider. The
@@ -164,9 +154,6 @@ provider-free parts of the learning path and deployment example do too.
 
 ## Connect tools or expose a program
 
-Everything in this section is on `main` only, not in `v0.3.2` (see the release
-state at the top).
-
 `Imp.MCP.connect/2` imports authorized MCP servers through ExMCP, returning
 ordinary tools plus explicit connection cleanup. Source server/tool identities,
 schemas, and annotations remain in each tool's `metadata.mcp` even when names
@@ -209,8 +196,8 @@ Run `mix docs` for the exhaustive module and function reference.
 
 For an ordinary ACP workspace agent with bounded tools, see
 [examples/workspace_agent](examples/workspace_agent/README.md). It depends
-directly on this Imp checkout (`main` only, like the adapters it uses) and
-includes a provider-free mode for checking its launcher and workspace boundary.
+directly on this Imp checkout by path and includes a provider-free mode for
+checking its launcher and workspace boundary.
 
 ## Where this fits
 

@@ -1,101 +1,78 @@
-# Imp v0.3.2
+# Imp v0.4.0
 
 Imp is a framework for typed, optimizable language-model programs on the BEAM.
 Declare a task as named inputs and outputs, call it like any other Elixir
 program, measure it on examples, compile it with an optimizer, and run the
 selected program under OTP.
 
+This release absorbs the protocol adapters that previously lived on `main`
+only, and changes three published shapes. It is `0.4.0` rather than a patch
+because a program written against `v0.3.2` can need edits.
+
 ## Install
 
-`v0.3.2` is a Git source release from a public repository; no credentials are
+`v0.4.0` is a Git source release from a public repository; no credentials are
 required.
 
 ```elixir
-{:imp, github: "deepfates/imp", tag: "v0.3.2"}
+{:imp, github: "deepfates/imp", tag: "v0.4.0"}
 ```
 
 Imp is not published to Hex. Use a path dependency only while developing
 against a local checkout.
 
-This patch release makes the RLM controller's language guide match its
-restricted interpreter and preserves explicit zero-retry provider policy for
-all Req adapter forms, including MFA adapters with arguments.
-
-## What is included
-
-- Typed signatures with scalar, collection, enum, union, optional, default,
-  code, and constrained fields.
-- `Predict`, `ChainOfThought`, composed `Imp.Module` programs, retrieval,
-  ReActV2, CodeAct, RLM, tools, MCP, and provider streaming.
-- Examples, metrics, concurrent evaluation, disjoint train/selection/test
-  experiments, and optimizer reports.
-- Demonstration, instruction, prompt, ensemble, rule, playbook, and
-  weight-training optimizer families, including GEPA, MIPROv2, SIMBA, COPRO,
-  BootstrapFewShot, RandomSearch, KNNFewShot, BootstrapFinetune, BetterTogether,
-  Avatar, and Optimize Anything.
-- Checksummed whole-program and parameter artifacts that exclude credentials
-  and apply selected state to freshly constructed trusted code.
-- OTP-native operation with bounded tasks, cancellation, per-effect
-  authorization, redacted telemetry, caching, usage accounting, hot reload,
-  and failure propagation.
-- ReqLLM provider clients, explicit local/static test models, retriever and
-  trainer extension points, and local MLX/TRL integration boundaries.
-
-The [Learning Path](docs/LEARNING_PATH.md) builds one program from its first
-provider call through evaluation, optimization, tools, persistence, and
-deployment. The [deployment example](examples/deployment/README.md) shows a
-supervised two-stage program with parameter reload, concurrent calls, restart,
-timeouts, and crash containment. Five Livebooks cover the same system
-interactively.
-
-## What the BEAM changes
-
-Imp preserves DSPy's program/evaluate/optimize workflow without copying
-Python's object model. Programs are immutable values. Configuration can be
-explicit or process-scoped. Evaluation and tool work run in supervised tasks.
-Telemetry uses standard `:telemetry` events. Saved state is rebound to live
-providers and callbacks at application startup instead of serializing runtime
-authority.
-
-Provider output and optimizer search are stochastic. A compiled program is a
-candidate until it improves the metric that matters on data excluded from
-training and selection. Imp supplies that lifecycle; applications still own
-their data, metric, budget, promotion rule, and operational policy.
-
-DSPy's Python integration ecosystem is larger. Imp exposes extension points
-for providers, retrievers, adapters, tools, and trainers, but Python-only
-integrations do not automatically work on the BEAM. DSPy's Flex code optimizer
-is not included in this release.
-
-The supported center is the `Imp` facade, signatures, adapters, evaluation,
-static and ReqLLM execution, tools, telemetry, saving, and the deployment
-pattern. Generated docs place optimizer implementations, parameter artifacts,
-agent loops, training integrations, and `Imp.Run` in **Experimental optimizers
-and advanced workflows**. These are implemented and tested APIs, not release
-promises of effectiveness or pre-1.0 shape stability. In particular, GRPO is
-an external-training boundary rather than an in-process gradient engine.
-
-## Breaking changes from v0.2.1
-
-- `Imp.optimize/3`, `/4`, and `/5` return `{:ok, program}` or
-  `{:error, reason}`. Use the corresponding `Imp.optimize!` function when a
-  failure should raise.
-- `Imp.Adapters.Types` and its nested structs moved to `Imp.Adapter.Types`.
-- `Imp.Agent` and `Imp.Agent.Runtime` were removed. Use ReActV2 or RLM as the
-  program and ordinary Elixir supervision as the runtime. Use
-  `Imp.start_run/3` only when a host needs ordered events, addressable
-  cancellation, or explicit effect authorization.
+ExMCP is declared `runtime: false`, so an OTP release that uses `Imp.ACP` or
+`Imp.MCP` must list `applications: [ex_mcp: :load]` in its release
+definition; see [protocol runtime in
+releases](docs/PRODUCTION_OPERATIONS.md#protocol-runtime-in-releases).
+Ordinary Imp startup starts no protocol endpoint.
+
+## Headline changes
+
+- `Imp.ACP` and `Imp.MCP.connect/2` are in the tag. The separate `imp_acp`
+  package is retired with no compatibility shim: a consumer that depended on
+  it now depends on `imp` alone. `Imp.MCP.connect/2` also gains OAuth
+  credentials for remote HTTP servers (`Imp.MCP.OAuth`), `bearer_env`
+  descriptor auth, `on_failure: :drop` with an `unavailable` list, and a
+  per-dial timeout.
+- `:reasoning_effort` is the one reasoning option on `Imp.Clients.ReqLLM`.
+  `:openrouter_reasoning` is gone; the wire encoding is the separate
+  `:openrouter_reasoning_wire`.
+- ReActV2 sends the tool roster natively and no longer declares a `tools`
+  input field or writes its instructions into `signature.instructions`. Loop
+  guidance travels to the adapter through `:adapter_opts`.
+- The `:model_response` event's `metadata.cost` is a plain USD float or `nil`,
+  with any provider breakdown under `metadata.billing`.
+- Structured values in a prompt render complete, the way DSPy renders a dict,
+  instead of a truncated `inspect/1`.
+
+## Breaking changes from v0.3.2
+
+- Replace `openrouter_reasoning: ...` with `reasoning_effort: ...`. Saved
+  programs allowlist `:reasoning_effort` and `:openrouter_reasoning_wire` in
+  its place, so rebuild artifacts that carried the old key.
+- A caller that passed or read ReActV2's `tools` input field no longer has
+  one; the roster is sent natively.
+- A host that read `metadata.cost` as a provider billing map reads a number
+  now, and finds the map under `metadata.billing` when the provider sent one.
 
 ## Upgrade path
 
-1. Replace `Imp.Adapters.Types` references with `Imp.Adapter.Types`.
-2. Choose the returning or raising optimizer API explicitly.
-3. Replace `Imp.Agent` usage with a ReActV2/RLM program owned by your
-   supervision tree.
-4. Rebuild saved artifacts with `0.3.2` before promotion.
+1. Rename the reasoning option and rebuild saved artifacts with `0.4.0`.
+2. Drop any `tools` handling around ReActV2.
+3. Sum spend from `metadata.cost` as a number.
+4. If you depended on `{:imp, github: "deepfates/imp", branch: "main"}` for the
+   adapters, move to the tag.
 5. Run your held-out evaluation and application smoke test against the tagged
    dependency.
 
-Generated module documentation is the complete API reference. Start with
-`Imp`, `Imp.Signature`, `Imp.Module`, `Imp.Evaluate`, `Imp.Optimizer`,
-`Imp.Optimizer.Artifact`, `Imp.Run`, and `Imp.Telemetry`.
+New in this release: [Benchmarks](https://github.com/deepfates/imp/blob/main/docs/BENCHMARKS.md)
+and its [results table](https://github.com/deepfates/imp/blob/main/benchmarks/RESULTS.md)
+carry every number this repository publishes with the command that produces it,
+and the ticket-routing rows were re-measured live for this release, a month
+after the first run, with both runs recorded.
+
+The [CHANGELOG](CHANGELOG.md) records every user-visible change in this
+release. Generated module documentation is the complete API reference. Start
+with `Imp`, `Imp.Signature`, `Imp.Module`, `Imp.Evaluate`, `Imp.Optimizer`,
+`Imp.ACP`, `Imp.MCP`, and `Imp.Telemetry`.
@@ -18,19 +18,36 @@ for the claims that cannot be re-measured at all.
 
 | # | Number | Dataset (license) | Model | Provider | Date | Commit | Command |
 | --- | --- | --- | --- | --- | --- | --- | --- |
-| R1 | Zero-shot held-out accuracy `0.30`–`0.50` over 3 repeats (20 held-out of 60 tickets) | `priv/tutorial/support_tickets.json`, 60 rows, sha256 `7ea5ae7a…` (written for this repository; see [SUPPORT_TICKETS_LICENSE.md](../priv/tutorial/SUPPORT_TICKETS_LICENSE.md)) | `gpt-5.4-mini` | OpenRouter route `openai/gpt-5.4-mini` | 2026-08-22 | `88d61a9c` | `OPENAI_API_KEY=… mix run scripts/tutorial_ticket_routing_experiment.exs` |
-| R2 | `LabeledFewShot(k: 8)` held-out accuracy `0.95`–`1.00` over the same 3 repeats; per-repeat lift `+0.45`, `+0.65`, `+0.65` | same as R1 | `gpt-5.4-mini` | OpenRouter route `openai/gpt-5.4-mini` | 2026-08-22 | `88d61a9c` | same as R1 |
+| R1 | Zero-shot held-out accuracy `0.30`–`0.40` over 3 repeats (20 held-out of 60 tickets) | `priv/tutorial/support_tickets.json`, 60 rows, sha256 `7ea5ae7a…` (written for this repository; see [SUPPORT_TICKETS_LICENSE.md](../priv/tutorial/SUPPORT_TICKETS_LICENSE.md)) | `gpt-5.4-mini` | OpenRouter route `openai/gpt-5.4-mini` | 2026-09-17 | `7985ed2f` | `OPENAI_API_KEY=… mix run scripts/tutorial_ticket_routing_experiment.exs` |
+| R2 | `LabeledFewShot(k: 8)` held-out accuracy `0.90`–`0.95` over the same 3 repeats; per-repeat lift `+0.55`, `+0.55`, `+0.65` | same as R1 | `gpt-5.4-mini` | OpenRouter route `openai/gpt-5.4-mini` | 2026-09-17 | `7985ed2f` | same as R1 |
 
 R1 and R2 come from one execution of one command; they are two numbers from the
 same three repeats, not independent measurements. That run used 120 requests,
-44,365 tokens and `$0.038819` in provider-priced usage for all three repeats —
-about `$0.013` and 8–9 seconds per repeat. No row errored and the in-BEAM cache
-was cleared before each repeat, so all 120 calls were live.
+44,293 tokens and `$0.038488` in provider-priced usage for all three repeats —
+about `$0.013` and 8–13 seconds per repeat. No row errored and the in-BEAM cache
+was cleared before each repeat, so all 120 calls were live. The run's artifact
+is committed at
+[`benchmarks/data/tutorial-ticket-routing-2026-09-17.receipt.json`](data/tutorial-ticket-routing-2026-09-17.receipt.json),
+sha256 `2dcc1228…`; the dollar figure in it is the script's own pricing table
+(`$0.75`/`$4.50` per million tokens) applied to reported token counts, not a
+provider-billed amount, so treat it as an estimate and the token counts as the
+measurement.
+
+The same command was run a month earlier, on 2026-08-22 at commit `88d61a9c`,
+against the same dataset, model and route: zero-shot `0.30`–`0.50`, optimized
+`0.95`–`1.00`, per-repeat lift `+0.45`, `+0.65`, `+0.65`, 120 requests and
+`$0.038819` in provider-priced usage. Two independent runs a month apart agree
+on the thing worth claiming — every repeat improved, by 45 to 65 points — and
+disagree on the endpoints, which is what a twenty-row evaluation should do.
 
 Three repeats of a twenty-row evaluation is a coarse instrument. The gap between
-the two rows (45–65 points) is far larger than the instrument's resolution
-(one row is 5 points), which is why the direction is trustworthy while the exact
-endpoints are not.
+the two rows (55–65 points on 2026-09-17, 45–65 points on 2026-08-22) is far
+larger than the instrument's resolution (one row is 5 points), which is why the
+direction is trustworthy while the exact endpoints are not. The 2026-09-17 run
+put one optimized repeat at `0.90`, below the `0.95`–`1.00` the tutorial claimed
+from the first run alone, and one repeat at 13.1 seconds, above the 8–9 seconds
+it claimed; both claims were widened to the measured union rather than restated
+from the luckier run.
 
 ## Recomputable only