feat(runtime): track Gemini thinking tokens in billing meter and session ledger - #239
Open
dembrane-sam-bot wants to merge 1 commit into
Open
feat(runtime): track Gemini thinking tokens in billing meter and session ledger#239dembrane-sam-bot wants to merge 1 commit into
dembrane-sam-bot wants to merge 1 commit into
Conversation
dembrane-sam-bot
enabled auto-merge
August 16, 2026 05:10
spashii
force-pushed
the
sam/update-track-thinking-tokens
branch
from
August 27, 2026 11:20
d7e062a to
ecb06ce
Compare
spashii
approved these changes
Aug 27, 2026
spashii
left a comment
Member
There was a problem hiding this comment.
Reviewed and approved by Sameer. Rebased onto main; the prior CI failure was a stale .trivyignore.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is this change?
This PR updates the billing meter and session ledger in
src/runtime/to track Gemini reasoning/thinking tokens (thoughts_token_count).Specifically:
_accumulate_usageinsrc/runtime/adk_runner.pyto countthoughts_token_countintooutput_tokens(since Vertex bills thinking tokens as output tokens) and tracks them independently asthoughts_tokens.tripleto aquad(prompt, output, thoughts, cache)to prevent dropped events on collision.thoughts_tokensintoAgentRunResult,SessionResult, andSessionLedgerEntryso thinking token consumption is visible and audited in/data/sessions.jsonl.What did Sam notice that led to this?
During our start-of-day token reconciliation pass, we noticed a consistent 3x-4x undercount on output tokens over the last 5 days when comparing our self-report ledger against Vertex AI's actual billed metrics (
aiplatform.googleapis.com/publisher/online_serving/token_countgrouped bytype). For example, yesterday (2026-08-15) GCM reported 30,124 output tokens whereas our ledger recorded only 7,036.By running a controlled test session with
Runner.run_async(), we discovered that Gemini 3.5 Flash generates thinking/reasoning tokens (thoughts_token_countwhich is billed as output tokens). However,_accumulate_usageonly readcandidates_token_countand completely ignoredthoughts_token_count. In our short test, thinking tokens (291) were 19x larger than candidates tokens (15).Tier
Tier 3 (Runtime/Substrate).
Confidence
100% confident. We verified the Vertex AI billing behavior via controlled local testing, and our tests in
tests/eval/pass cleanly.