fix(claude): normalize recovered tool outcomes - #79
Conversation
Transcript recovery marked every tool result approved and left matching PreToolUse spans pending. Turn and session cleanup then attached lifecycle errors, so denied calls looked approved and successful calls counted as failures. Reconcile transcript results with pending spans and derive approval, denial, failure, and cancellation from native result signals. This keeps error and approval analytics aligned with what actually happened.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 41e2adef95
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "Codex (@codex) review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".
Recovered tool output is arbitrary user-command data. Scanning successful output for words like "denied" or "aborted" could relabel an executed tool as rejected or cancelled, corrupting approval analytics in the opposite direction. Only use free-form denial and cancellation phrases when Claude marks the result as an error. Structured outcome fields remain authoritative on their own.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 159252e03f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "Codex (@codex) review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".
| @@ -851,11 +876,14 @@ impl ClaudeTranslator { | |||
| for id in ids { | |||
| if let Some(tool) = self.pending_tools.remove(&id) { | |||
| self.emitted_tools.insert(id); | |||
There was a problem hiding this comment.
When a turn contains multiple assistant request IDs and a later tool's PostToolUse hook is missing, emit_main processes only the first transcript segment and leaves the rest for drain_pending; this loop then marks every still-pending tool as emitted and cancelled. Consequently, the later segment's terminal tool_result is skipped by emit_parsed, losing its output and real success, denial, or failure outcome. Defer cancellation until all queued transcript segments have been reconciled.
Useful? React with 👍 / 👎.
2 participants
resolves https://linear.app/braintrustdata/issue/SDK-359/tool-outcome-fields-contradict-actual-results