Release v0.4.0 - #131
Merged
Merged
Release v0.4.0#131
Conversation
deepfates
force-pushed
the
claude/release-0.4.0
branch
from
September 17, 2026 20:14
bfa4208 to
129637f
Compare
Cut 0.4.0: the CHANGELOG's Unreleased section becomes the released section, with entries added for the Imp.ACP / Imp.MCP.connect/2 absorption, the public repository, the evidence bookkeeping removed from the source checkout, and the benchmark surface. RELEASE_NOTES is rewritten for the tag. README drops the "release state" caveat and the main-only notes, since the tag now carries the adapters, and the install snippet, livebooks, docs, deployment example, and the tests that assert the version move to v0.4.0. The public API manifest records the new package version; nothing else in it moved. Re-measured the ticket-routing tutorial live at 7985ed2 and committed the artifact. Rows R1 and R2 now carry the 2026-09-17 run, with the 2026-08-22 run kept beside them: two runs a month apart agree that every repeat improved by 45 to 65 points, and disagree on the endpoints. The new run put one optimized repeat at 0.90 and one at 13.1 seconds, outside the 0.95-1.00 and 8-9 seconds the tutorial claimed from the first run alone, so those claims are widened to the measured union in the tutorial, BENCHMARKS, EVIDENCE and the runner's recorded doc claims rather than restated from the luckier run.
deepfates
force-pushed
the
claude/release-0.4.0
branch
from
September 17, 2026 20:28
129637f to
c9c5b78
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this release contains
Imp.ACPandImp.MCP.connect/2, absorbed from the retiredimp_acppackage, plus MCP OAuth credentials,bearer_envdescriptor auth,on_failure: :drop, and per-dial timeouts.:reasoning_effortreplaces:openrouter_reasoningonImp.Clients.ReqLLM, with:openrouter_reasoning_wirenaming the encoding.toolsinput field, loop guidance through:adapter_opts, and a request shape a provider's prompt cache can key on.metadata.costis a plain USD float ornil, with any provider breakdown undermetadata.billing.inspect/1, and the benchmark surface (docs/BENCHMARKS.md,benchmarks/RESULTS.md) ships.Why 0.4.0
The Unreleased section is breaking.
:openrouter_reasoningis removed, ReActV2 no longer declares atoolsinput field, andmetadata.costchanged shape from a provider billing map to a number. A program and a host written againstv0.3.2can both need edits, so this is a minor bump, not0.3.3.Fresh measurement of R1 and R2
The ticket-routing tutorial was re-run live at
7985ed2f(the #130 merge), three repeats, OpenRouter routeopenai/gpt-5.4-mini. The artifact is committed atbenchmarks/data/tutorial-ticket-routing-2026-09-17.receipt.json(sha2562dcc1228…, which is also the runner's own filename for it). The prior 2026-08-22 artifact is not in the tree —benchmarks/runs/is gitignored — so this one went next to the other retained receipts underbenchmarks/data/.Measured: baseline
0.35/0.40/0.30, optimized0.90/0.95/0.95, per-repeat lift+0.55/+0.55/+0.65, 120 requests with usage, 44,293 tokens, 8.0–13.1 s per repeat, fresh-service check 4/4, no row errors, cache cleared before each repeat.Two claims in the tutorial did not survive. The new run put one optimized repeat at
0.90, below the0.95–1.00the doc claimed, and one repeat at 13.1 s, above the claimed 8–9 s. Both claims came from the single 2026-08-22 run. They are widened to the measured union —0.90–1.00, 8–13 s — indocs/TUTORIAL_TICKET_ROUTING.md,docs/BENCHMARKS.md,docs/EVIDENCE.mdand the runner's owndoc_claims_under_test, rather than restated from the luckier run. R1/R2 now carry the 2026-09-17 numbers with the 2026-08-22 run kept beside them, so the reader sees two runs a month apart agreeing that every repeat improved by 45–65 points and disagreeing on the endpoints.On cost: the artifact does carry a
usdfield ($0.038488for all three repeats), but it is the script's own price table ($0.75/$4.50per million) applied to reported token counts, not a provider-billed amount. RESULTS.md states the token counts as the measurement and labels the dollar figure an estimate, matching how the existing row described "provider-priced usage". No test asserted the old numbers or the artifact digest;test/tutorial_parity_test.exschecks prose structure only.Tag timing
examples/deployment/mix.exsandtest/deployment_reference_test.exsasserttag: "v0.4.0", which will not exist until this merge is tagged. Safe for CI: the only step that builds the deployment example runscd examples/deployment && IMP_PATH=../.. mix deps.get ...(.github/workflows/ci.yml), so the Git tag is never fetched.mix package.check's clean room stages the local package. No CI step or test doesmix deps.getthere withoutIMP_PATH.Two notes
mainafter Public benchmark surface: one doc, one results table, honest claims #130 merged. README andmix.exskeep both sides: their "Read next" links andproduct_docs/extras, my Install snippet, deleted release-state paragraph, andversion: "0.4.0". (Public benchmark surface: one doc, one results table, honest claims #128 was auto-closed when its base branch went away with Remove evidence-certification bookkeeping; keep the benchmark harness #126; Public benchmark surface: one doc, one results table, honest claims #130 carried the same commits.)priv/public_api.jsonneeded regenerating — it pinspackage_version. That is the entire diff to it, which is also evidence the release changes no public API surface.Verification
CI could not run: every job on this branch — and on the
mainpush for #130 — was refused before its first step with "The job was not started because an Actions budget is preventing further use." So the checks below were run locally, including the fullmix checkthat CI would have run.mix format --check-formatted— clean.MIX_ENV=test mix compile --warnings-as-errors— no warnings.mix check(format + compile + full suite) —54 doctests, 9 properties, 2815 tests, 0 failures, 13 skipped (220 excluded).mix teston the contract set plustutorial_parity,public_api_manifest,public_surface—100 tests, 0 failures.MIX_ENV=test mix imp.public_api --check—public API manifest is current.MIX_ENV=test mix docs.check— docs rendered; livebooksResults: 5 passed, 0 failed.mix package.check—clean-room package proof passed: tmp/package-clean-room(runs Hex resolution withHEX_OFFLINE=1; it does fetch the declaredex_mcpGit fork).The artifact was scanned for key-shaped strings before committing: the only long tokens in it are the git sha, the dataset sha256 and the fresh-service artifact sha256.