Skip to content

Put a curated summary's companion contracts in the scene - #216

Open
shellygr wants to merge 11 commits into
masterfrom
shelly/curated-summary-scene
Open

Put a curated summary's companion contracts in the scene#216
shellygr wants to merge 11 commits into
masterfrom
shelly/curated-summary-scene

Conversation

@shellygr

@shellygr shellygr commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

OZ_BitMaps.spec summarizes BitMaps by rerouting its calls to a companion library of its own, OZ_BitMaps. CVL can only name a contract that is in the scene, and the scene is the files the conf lists. copy_summaries_folder copies the companion beside the spec and stops there, so nothing ever puts it in a conf.

The result is that any project whose methods match the bitmaps entry fails its warmup typecheck:

Found errors in certora/specs/summaries/OpenZeppelin/OZ_BitMaps.spec:
Error in spec file (OZ_BitMaps.spec:6:9): Variable `OZ_BitMaps` has not been declared.
...
RuntimeError: Cache warmup failed - stopping orchestration

bitmaps is the only entry in function_summaries.json that declares additional_contracts, and it has not changed since the package was imported from the AutoSetup repo, so I believe this has never worked. I hit it on a dev run whose main contract calls BitMaps.get/set/unset/setTo.

The base conf now carries an entry per top-level definition in each matched entry's additional_contracts. Per definition, not per file: a conf entry naming only the file puts just the stem-named contract in scene.

Two things worth a reviewer's attention:

  • curated_scene_contracts lives in summary_resolver.py rather than next to its caller in setup_summaries.py, because that module calls sys.exit(1) at import time when there is no ANTHROPIC_API_KEY or .env, so a unit test cannot import it at all.
  • A companion that was matched but not copied logs a warning and is skipped, matching what copy_summaries_folder already does for a missing bundled file. Happy to make it fatal instead.

Tests in Certora/Autosetup#TBD (six cases: the conf entry, multiple definitions per file, a summary with no companion, nothing matched, an unknown key, and the uncopied-companion warning).

🤖 Generated with Claude Code


Folded in (was #219): make the reroute actually attach.

Putting the companion in the scene was necessary but not sufficient. Two more things stopped OZ_BitMaps from doing anything:

  • Parameter matching compares canonicalId, which is "<file resolved under .certora_sources>|<qualified name>". A bundled companion declares its own copy of BitMaps.BitMap, and that is a different type from the project's however identical it looks. The companion is now generated per project from a template, importing the project's own copy of the library.
  • The bodies were require(false) tripwires. A body that always reverts leaves the summary nothing to attach to, so every call through it reverts and every rule over it passes vacuously. That reads as a green run. The bodies are neutral now.

Two smaller ones, both fixed here: the host functions have to be external, since DetectRerouteSummaries only sees external library functions, and the summary entries have to survive the prune pass, which drops entries naming a contract that is not in the scene index. That index comes from the project's own compilation, which happens before the companion exists.

Checked on a small project that uses BitMaps: every rule verifies under rule_sanity basic, and one of them is a bare satisfy, so a vacuous pass fails instead of looking like success. Tests are in the Autosetup PR.

A curated summary may reroute a library's calls to a companion of its own:
OZ_BitMaps.spec sends BitMaps.get to OZ_BitMaps.get. CVL can only name a
contract that is in the scene, and the scene is the files the conf lists, so
copying the companion beside the spec never made it nameable. Any project whose
methods match the bitmaps entry fails its warmup typecheck with
"Variable `OZ_BitMaps` has not been declared".

The base conf now carries an entry per top-level definition in each matched
entry's additional_contracts, which is also what keeps them nameable when one
file holds more than one.

`curated_scene_contracts` lives in summary_resolver rather than beside its
caller because setup_summaries exits at import time without an API key, and a
unit test cannot import it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@shellygr

shellygr commented Sep 5, 2026

Copy link
Copy Markdown
Contributor Author

Tests are in Certora/Autosetup#148.

shellygr and others added 7 commits September 5, 2026 19:23
That config exists to typecheck the summaries that were just set up, and it was
built with an empty additional-contracts list, so a summary rerouting through a
companion was typechecked against a scene the companion is missing from. It is
the config a real run died in.

Whether the run's own --additional-contracts belong there as well is still open,
so the TODO's question stands; only the companions move.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A third place decides a conf's scene: the call-resolution path rewrites `files`
from the main contract plus the additional contracts, which stripped the
companions back out again and left the summary naming a contract that is no
longer there.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The bundled OZ_BitMaps companion could never work. A reroute target must be an
external library function, and every function in it was internal; and even made
external, it declared its own copy of `struct BitMap`, which is a different type
from the project's however identical it looks — parameter matching compares
canonicalId, which names the file the type came from.

So the companion is now a template, filled in per project with the project's own
BitMaps.sol as the import and that library's pragma. Verified by hand first:
with the companion internal the typechecker gives exactly the production error,
and with it external and importing the project's own library it exits clean.

Two adjacent gaps this uncovered, both general rather than BitMaps-specific: a
file added to `files` needs its compiler_map entry or certoraRun rejects the
conf as unmatched, and a companion shipped as a template is emitted under its
untemplated name, which is the name the scene has to look for.

The spec's ghosts now key on calledContract rather than currentContract, which
is bound to the primary contract and so let two contracts holding a BitMap at
the same slot alias each other's bits.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every other entry in a conf is relative to the project root, and an absolute one
bakes the build machine's layout into a file that travels with the run.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The prune pass drops an internal-method entry whose receiver is absent from the
scene index, and that index is built from the project's own compilation — which
happens before a curated companion joins the conf. So every OZ_BitMaps entry was
commented out as "not in scene", leaving the reroute pointing at a companion
whose bodies revert on every call: a green run that proved nothing, which is
worse than the loud failure it replaced.

The exemption is per receiver and covers only contracts that are genuinely in
the conf and cannot be in an index that predates them. A receiver that is absent
for the ordinary reason is still dropped.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The four `require(false)` bodies were a tripwire for "summarization failed".
They guaranteed it instead: an unconditionally reverting body leaves nothing for
the summary to attach to, so every BitMaps call reverted, and rules over a
reverting call pass vacuously. The tripwire caused the failure it announced, and
made it silent.

Measured on a project that uses BitMaps, one variable changed:

  anchors            callsAreReachable   the four behavioural rules
  require(false)     VIOLATED            all SANITY_FAILED
  neutral            VERIFIED            all VERIFIED, none vacuous

setThenGetIsTrue verifying non-vacuously is what settles it: with neutral bodies
and no working summary, set stores nothing and get reads false, so that rule
fails. It passes, so the ghost is doing the work.

Neutral bodies also invert the failure mode. An unattached summary now shows up
as a violated rule rather than a green vacuous one, and rule_sanity catches the
rest.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ary-scene

# Conflicts:
#	certora_autosetup/setup/setup_summaries.py
@shellygr
shellygr requested a review from jar-ben September 11, 2026 23:07
@shellygr

Copy link
Copy Markdown
Contributor Author

Codex review:

  1. [P2] Companion can import the wrong BitMaps library. _project_library_file (worktrees/AutoProver/curated-summary-scene/certora_autosetup/setup/setup_summaries.py:830) selects the
    first matching library anywhere on disk. With multiple vendored OpenZeppelin copies, that can differ from the compiled contract’s dependency, recreating the canonical-type
    mismatch this PR fixes. Resolve the source from compilation metadata.

  2. [P2] Companions discovered during call resolution never enter the config. Companion injection (worktrees/AutoProver/curated-summary-scene/certora_autosetup/autosetup/
    autosetup.py:844) happens before call resolution. If a newly linked contract introduces BitMaps, _on_contracts_added generates and imports its summary but only adds the original
    library to files. The next typecheck encounters an undeclared OZ_BitMaps. Update the config after each summary batch.

  3. [P2] Failed companion generation leaves its summary enabled. This continue (worktrees/AutoProver/curated-summary-scene/certora_autosetup/setup/setup_summaries.py:797) skips
    generating the companion, but retains bitmaps in matched_functions and imports its reroute spec. A library excluded from discovery, or without a pragma, therefore produces a
    broken summary instead of the documented fallback. Drop the dependent summary or fail explicitly.

Three fixes on the companion path, all in the seam between which library the
build compiled and which conf the companion has to be in.

The companion's library was found by searching the filesystem for a file
declaring the name, which picks an arbitrary copy when a project vendors
several. That is the canonical-type mismatch the companion exists to avoid.
all_methods.json records the path the compiler read for each method, so the
library file comes from there now. find_libraries_used_by chose its file the
same way and now resolves the same way, so build_library_name_index goes away.

Companions matched during call resolution never reached the conf that call
resolution runs on, so the next typecheck met an undeclared OZ_BitMaps. They
are added after each summary batch.

A companion that could not be generated left its summary imported and its key
matched, which is a broken summary rather than the fallback of none that the
entry documents. copy_summaries_folder now generates the companions before it
copies anything, and returns the keys to drop.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@shellygr

Copy link
Copy Markdown
Contributor Author

Claude answers:

  1. Done. The library file now comes from all_methods.json, which records the originalFile the compiler read for each method, filtered to the compilation units in the scene, with the main contract's unit breaking a tie between two copies. find_libraries_used_by picked its file from the same kind of filesystem search and had the same hole, so it resolves the same way now and build_library_name_index is gone.

  2. Done. _on_contracts_added adds the companions to the conf after each summary batch. add_files_to_config dedupes and fills the compiler maps, so re-adding on every batch costs nothing.

  3. Done, dropped rather than fatal. copy_summaries_folder generates the companions before it copies anything and returns the keys that failed; the caller takes them out of matched_functions and the aggregator, and their spec is never copied. An unresolvable library, an ambiguous one, or one with no pragma now leaves the library unsummarized, which is the documented fallback.

Tests are on the matching branch of the tests repo: which copy wins, the drop all the way through to the aggregator, and the conf entry after a late batch. The BitMaps e2e passes locally against a project whose tree carries a second copy of the library.

The compiler-map pass over the additional contracts parsed them strictly, and a strict
parse treats an empty list as a caller error. Every project without a companion has an
empty list, so autosetup died at the summary-compilation step with "Must provide at
least one .sol file".

The files are also resolved against the project root now rather than the process CWD, so
each map entry reads the pragma of the file the conf names.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant