Skip to content

Release v0.4.0 - #131

Merged
deepfates merged 1 commit into
mainfrom
claude/release-0.4.0
Sep 17, 2026
Merged

deepfates merged 1 commit into
mainfrom
claude/release-0.4.0

Conversation

@deepfates

@deepfates deepfates commented Sep 17, 2026

Copy link
Copy Markdown
Owner

What this release contains

  • Imp.ACP and Imp.MCP.connect/2, absorbed from the retired imp_acp package, plus MCP OAuth credentials, bearer_env descriptor auth, on_failure: :drop, and per-dial timeouts.
  • :reasoning_effort replaces :openrouter_reasoning on Imp.Clients.ReqLLM, with :openrouter_reasoning_wire naming the encoding.
  • ReActV2 sends the tool roster natively: no tools input field, loop guidance through :adapter_opts, and a request shape a provider's prompt cache can key on.
  • metadata.cost is a plain USD float or nil, with any provider breakdown under metadata.billing.
  • The repository is public, the evidence-certification bookkeeping is gone from the source checkout, structured prompt values render complete instead of truncated by inspect/1, and the benchmark surface (docs/BENCHMARKS.md, benchmarks/RESULTS.md) ships.

Why 0.4.0

The Unreleased section is breaking. :openrouter_reasoning is removed, ReActV2 no longer declares a tools input field, and metadata.cost changed shape from a provider billing map to a number. A program and a host written against v0.3.2 can both need edits, so this is a minor bump, not 0.3.3.

Fresh measurement of R1 and R2

The ticket-routing tutorial was re-run live at 7985ed2f (the #130 merge), three repeats, OpenRouter route openai/gpt-5.4-mini. The artifact is committed at benchmarks/data/tutorial-ticket-routing-2026-09-17.receipt.json (sha256 2dcc1228…, which is also the runner's own filename for it). The prior 2026-08-22 artifact is not in the tree — benchmarks/runs/ is gitignored — so this one went next to the other retained receipts under benchmarks/data/.

Measured: baseline 0.35 / 0.40 / 0.30, optimized 0.90 / 0.95 / 0.95, per-repeat lift +0.55 / +0.55 / +0.65, 120 requests with usage, 44,293 tokens, 8.0–13.1 s per repeat, fresh-service check 4/4, no row errors, cache cleared before each repeat.

Two claims in the tutorial did not survive. The new run put one optimized repeat at 0.90, below the 0.951.00 the doc claimed, and one repeat at 13.1 s, above the claimed 8–9 s. Both claims came from the single 2026-08-22 run. They are widened to the measured union — 0.901.00, 8–13 s — in docs/TUTORIAL_TICKET_ROUTING.md, docs/BENCHMARKS.md, docs/EVIDENCE.md and the runner's own doc_claims_under_test, rather than restated from the luckier run. R1/R2 now carry the 2026-09-17 numbers with the 2026-08-22 run kept beside them, so the reader sees two runs a month apart agreeing that every repeat improved by 45–65 points and disagreeing on the endpoints.

On cost: the artifact does carry a usd field ($0.038488 for all three repeats), but it is the script's own price table ($0.75/$4.50 per million) applied to reported token counts, not a provider-billed amount. RESULTS.md states the token counts as the measurement and labels the dollar figure an estimate, matching how the existing row described "provider-priced usage". No test asserted the old numbers or the artifact digest; test/tutorial_parity_test.exs checks prose structure only.

Tag timing

examples/deployment/mix.exs and test/deployment_reference_test.exs assert tag: "v0.4.0", which will not exist until this merge is tagged. Safe for CI: the only step that builds the deployment example runs cd examples/deployment && IMP_PATH=../.. mix deps.get ... (.github/workflows/ci.yml), so the Git tag is never fetched. mix package.check's clean room stages the local package. No CI step or test does mix deps.get there without IMP_PATH.

Two notes

Verification

CI could not run: every job on this branch — and on the main push for #130 — was refused before its first step with "The job was not started because an Actions budget is preventing further use." So the checks below were run locally, including the full mix check that CI would have run.

  • mix format --check-formatted — clean.
  • MIX_ENV=test mix compile --warnings-as-errors — no warnings.
  • mix check (format + compile + full suite) — 54 doctests, 9 properties, 2815 tests, 0 failures, 13 skipped (220 excluded).
  • mix test on the contract set plus tutorial_parity, public_api_manifest, public_surface100 tests, 0 failures.
  • MIX_ENV=test mix imp.public_api --checkpublic API manifest is current.
  • MIX_ENV=test mix docs.check — docs rendered; livebooks Results: 5 passed, 0 failed.
  • mix package.checkclean-room package proof passed: tmp/package-clean-room (runs Hex resolution with HEX_OFFLINE=1; it does fetch the declared ex_mcp Git fork).

The artifact was scanned for key-shaped strings before committing: the only long tokens in it are the git sha, the dataset sha256 and the fresh-service artifact sha256.

Cut 0.4.0: the CHANGELOG's Unreleased section becomes the released section,
with entries added for the Imp.ACP / Imp.MCP.connect/2 absorption, the public
repository, the evidence bookkeeping removed from the source checkout, and the
benchmark surface. RELEASE_NOTES is rewritten for the tag. README drops the
"release state" caveat and the main-only notes, since the tag now carries the
adapters, and the install snippet, livebooks, docs, deployment example, and
the tests that assert the version move to v0.4.0. The public API manifest
records the new package version; nothing else in it moved.

Re-measured the ticket-routing tutorial live at 7985ed2 and committed the
artifact. Rows R1 and R2 now carry the 2026-09-17 run, with the 2026-08-22 run
kept beside them: two runs a month apart agree that every repeat improved by
45 to 65 points, and disagree on the endpoints. The new run put one optimized
repeat at 0.90 and one at 13.1 seconds, outside the 0.95-1.00 and 8-9 seconds
the tutorial claimed from the first run alone, so those claims are widened to
the measured union in the tutorial, BENCHMARKS, EVIDENCE and the runner's
recorded doc claims rather than restated from the luckier run.
@deepfates
deepfates merged commit ad64af1 into main Sep 17, 2026
17 of 20 checks passed
@deepfates
deepfates deleted the claude/release-0.4.0 branch September 17, 2026 21:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant