Skip to content

Bump req_llm to 1.18.0 - #110

Open
deepfates wants to merge 1 commit into
mainfrom
claude/reqllm1180
Open

deepfates wants to merge 1 commit into
mainfrom
claude/reqllm1180

Conversation

@deepfates

Copy link
Copy Markdown
Owner

Two things in this repo pin the exact req_llm build, and both had to move.

Imp.BenchmarkTruth.MultimodalManifest carries the campaign's ReqLLM
dependency — package hash, upstream revision, version — and
runtime_dependency!/1 refuses to dispatch when the loaded version differs,
so the bump failed with ReqLLM runtime version must equal "1.17.1", got: "1.18.0". The pin now names 1.18.0, package SHA-256
d610d4de14c7ef697a2aa4c30eada4e1ae172bec08bb3dea3bbe25f20feb5d40, upstream
revision 9aa98a4e02da5083464d2eed268f48272f0d7180 (the commit tagged v1.18.0
in agentjido/req_llm). Both provider manifests carry the same dependency
block, so they were updated and their payload checksums recomputed.

That changes the manifests' identity. The OpenAI manifest now hashes to
5ab0c61f... instead of 04daa155..., and the Google one to c5ca53e2...
instead of 4b4c8049.... The admitted multimodal_live artifact records the
1.17.1 run and keeps its own manifest SHA, so it still validates and the
claim still stands; what it no longer is, is reproducible from the manifests
in the tree. Admitting a 1.18.0 re-run means a live OpenAI campaign, a new
artifact, and a new multimodal_manifest_sha256/0 in the reproduction
validator. MULTIMODAL_FIDELITY.md now says so.

test/campaign_budget_test.exs asserted that the pinned req_llm overwrites
a caller's max_retries: 0 with its default of 3. 1.18.0 honours the zero,
so the assertion is inverted to match and still fails loudly if upstream
changes again. The transport guard itself is untouched and the attempt count
is unchanged.

req stays at 0.6.3 here; dependabot PR #94 moved it to 0.7.2 in the same
lock, and that bump needs its own code changes (PR #109).

This supersedes dependabot PR #94.

mix check: 54 doctests, 9 properties, 2861 tests, 0 failures, 13 skipped (261 excluded).
mix quality.check: credo clean, no vulnerabilities found.

Two things in this repo pin the exact req_llm build, and both had to move.

`Imp.BenchmarkTruth.MultimodalManifest` carries the campaign's ReqLLM
dependency — package hash, upstream revision, version — and
`runtime_dependency!/1` refuses to dispatch when the loaded version differs,
so the bump failed with `ReqLLM runtime version must equal "1.17.1", got:
"1.18.0"`. The pin now names 1.18.0, package SHA-256
d610d4de14c7ef697a2aa4c30eada4e1ae172bec08bb3dea3bbe25f20feb5d40, upstream
revision 9aa98a4e02da5083464d2eed268f48272f0d7180 (the commit tagged v1.18.0
in agentjido/req_llm). Both provider manifests carry the same dependency
block, so they were updated and their payload checksums recomputed.

That changes the manifests' identity. The OpenAI manifest now hashes to
5ab0c61f... instead of 04daa155..., and the Google one to c5ca53e2...
instead of 4b4c8049.... The admitted multimodal_live artifact records the
1.17.1 run and keeps its own manifest SHA, so it still validates and the
claim still stands; what it no longer is, is reproducible from the manifests
in the tree. Admitting a 1.18.0 re-run means a live OpenAI campaign, a new
artifact, and a new `multimodal_manifest_sha256/0` in the reproduction
validator. MULTIMODAL_FIDELITY.md now says so.

`test/campaign_budget_test.exs` asserted that the pinned req_llm overwrites
a caller's `max_retries: 0` with its default of 3. 1.18.0 honours the zero,
so the assertion is inverted to match and still fails loudly if upstream
changes again. The transport guard itself is untouched and the attempt count
is unchanged.

req stays at 0.6.3 here; dependabot PR #94 moved it to 0.7.2 in the same
lock, and that bump needs its own code changes (PR #109).

This supersedes dependabot PR #94.

mix check: 54 doctests, 9 properties, 2861 tests, 0 failures, 13 skipped (261 excluded).
mix quality.check: credo clean, no vulnerabilities found.
@deepfates

Copy link
Copy Markdown
Owner Author

Holding this unmerged. The bump changes the multimodal benchmark manifest checksums, so the admitted artifact is no longer reproducible from the tree without a live re-run, and nothing needs req_llm 1.18.0 today. It moves when there is a reason and a budget for the live campaign.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant