fix(learning): preserve optimizer selection and measurement integrity - #738
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Complete optimization methods could lose their selected winner when
selfImprove()ran them through the native proposal loop.An unchanged selection failed as a duplicate candidate; another selected surface could be replaced by the baseline using pooled training scores.
The method now owns search and selection, and a shared final comparison measures its selected surface without executing training again.
The change also repairs measurement and accounting failures:
defineAgentEval.evaluate()caches to surface content and execution revision.The complete method path retains actual method output and canonical measurement provenance.
The predictive-validity researcher documentation now describes its implemented recommendation behavior.
Process tests wait for child readiness and await cleanup; real-Git fixtures have an explicit startup allowance.
API migration in 0.174.0
SelfImproveResultis a union ofSelfImproveProposerResultandSelfImproveMethodResult.Use
modeto narrow before reading native generations.Method results expose
raw.method, combinedcost, receipt-onlyledgerCost, andtangle.method-improvementprovenance.Deferred method results have null final scores and no lift.
Native-only search controls are rejected in method mode.
Standalone premeasured baselines must use
surfaceDispatchRef(baselineSurface, dispatchRef)and the same judge revision.The examples and campaign documentation cover these contracts.
Validation
They make no paid provider calls and do not measure current model quality.
The shared ledger enforces limits on admitted calls.
Custom methods remain responsible for arbitrary external work they do not report.