Raise the vector over-fetch to 4x so every collection fills its budget - #177
Open
adamjohnwright wants to merge 1 commit into
Open
Raise the vector over-fetch to 4x so every collection fills its budget#177adamjohnwright wants to merge 1 commit into
adamjohnwright wants to merge 1 commit into
Conversation
3x still came up short on `complexes` for one of the 20 questions in tests/golden/questions.txt -- 8 distinct documents where 10 were asked for -- because a Reactome entity can occupy more rows than 3x leaves room to collapse. Measured across all four collections and all 20 questions: 4x clears every case, 6x gains nothing further. The cost is a larger k on a Chroma query that is already local, which is not measurable next to the LLM calls in the same request. Follows #169, which fixed the duplication itself. The constant carries the measurement so the next person does not have to redo it to know why it is 4. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #169, which fixed the duplication; this closes the last gap.
The gap
A Reactome entity occupies several CSV rows — a reaction appears once per pathway/input/output/catalyst combination — so the vector store is asked for more than we keep and the extras are collapsed by
st_id. At 3x, one of the 20 questions intests/golden/questions.txtstill came up short:After
All four collections, all 20 questions:
6x gains nothing over 4x, so 4 it is. The cost is a larger
kon a local Chroma query, which is not measurable beside the LLM calls in the same request.The constant carries the measurement in a comment, so the next person does not have to redo it to know why the number is 4.
🤖 Generated with Claude Code