Skip to content

fix(agents): preserve Gemini thought_signature across tool-calling turns - #396

Open
jirisacha wants to merge 1 commit into
AIDotNet:mainfrom
jirisacha:fix/gemini-thought-signature
Open

fix(agents): preserve Gemini thought_signature across tool-calling turns#396
jirisacha wants to merge 1 commit into
AIDotNet:mainfrom
jirisacha:fix/gemini-thought-signature

Conversation

@jirisacha

Copy link
Copy Markdown
Contributor

Problem

Gemini 3 models reason before they act. When that reasoning produces a function call, the
response carries an opaque signature:

"tool_calls": [{
  "id": "call_464538",
  "type": "function",
  "function": { "name": "ReadFile", "arguments": "{\"path\":\"README.md\"}" },
  "extra_content": { "google": { "thought_signature": "EukxCuYxARFNMg+6uxzTqh0j..." } }
}]

That signature has to come back with the same tool call on the next turn. The reasoning is
never returned as text, so the signature is the only thing carrying it forward, and Gemini
rejects any request that omits it:

400 INVALID_ARGUMENT
Function call is missing a thought_signature in functionCall parts. This is required for
tools to work correctly, and missing thought_signature may lead to degraded model
performance. Additional data, function call `default_api:ReadFile`, position 2.

Symptom in OpenDeepWiki: the first request of a conversation returns 200, the follow-up
returns 400. Since wiki generation is a long tool conversation, catalog generation dies on
its very first tool result and the repository ends up Failed.

This affects every Gemini 3 model, so it is not avoidable by picking a different one — I
tested gemini-3.1-pro-preview, gemini-3.7-flash, gemini-3.5-flash,
gemini-3-flash-preview and gemini-3.1-flash-lite-preview, and all five behave the same.
It matters for new users in particular, because the default model in the built-in
Google Gemini preset (BuiltinProviderPresets.json) is gemini-3-pro-preview — so
picking Google today gives a configuration where generation cannot complete.

Why the SDK does not handle it

extra_content is a Google extension outside the OpenAI schema. Requests go out through
OpenAIMicrosoft.Extensions.AIFunctionInvokingChatClient, and the outgoing
request is rebuilt from those libraries' own message objects, which have nowhere to keep a
provider-specific field on a tool call. So it is dropped coming in and never re-emitted
going out.

Upgrading does not help: OpenAI 2.8.0 and 2.13.0 and Microsoft.Extensions.AI.OpenAI
10.9.0 contain no reference to extra_content or thought_signature at all.

Disabling thinking is not a way out either — gemini-3.1-pro-preview answers
400 Budget 0 is invalid. This model only works in thinking mode, and the flash models
still require the signature.

The fix

A DelegatingHandler that carries the signature across turns at the HTTP layer, where the
raw JSON is still intact — the same approach FinishReasonNormalizingHandler already takes
for Gemini's non-OpenAI finish_reason values.

  • On the response: records each tool_calls[].idthought_signature pair as the SSE
    stream passes through. The stream itself is forwarded byte for byte; nothing is modified,
    and nothing beyond the current line is buffered. Streaming splits a tool call across
    chunks, so an indexid map handles chunks that carry the signature without an id.
  • On the request: re-attaches the signature to any tool call naming a known id that does
    not already carry extra_content.
  • Scope: only requests to generativelanguage.googleapis.com; every other provider is
    untouched.
  • Safety: all parsing is guarded, so a malformed chunk can never break a response. The
    cache is per-handler — AgentFactory builds one HttpClient per agent, so it spans a
    single conversation — and bounded at 512 entries, since a signature runs to several
    kilobytes.

Wired in as the outermost handler so it sees the request before it goes out and the
response after the retry logic below has settled.

Testing

dotnet build OpenDeepWiki.sln
dotnet test tests/OpenDeepWiki.Tests/OpenDeepWiki.Tests.csproj
  • 12 new xUnit tests in tests/OpenDeepWiki.Tests/Agents/ThoughtSignatureHandlerTests.cs,
    following the style of the neighbouring FinishReasonNormalizingHandlerTests (internal
    helpers called directly, no HTTP stack). They cover recording from streaming and
    non-streaming responses, index-based pairing when a chunk omits the id, restoring onto the
    request, not overwriting a caller-supplied extra_content, tolerating malformed JSON, the
    full round trip, and cache bounding.

  • Suite goes from 591 passed to 603 passed. The 7 failures in
    RepositoryAnalyzerSourceTests and RepositorySkillMarkdownBuilderTests are pre-existing
    — I confirmed they fail identically on a clean checkout of main (75840e5).

  • Verified against the live Gemini API with this handler compiled into a small harness
    that replays what the SDK does — send a tool call, strip the response to standard OpenAI
    fields, send the tool result back:

    turn 1 turn 2
    without the handler 200 400 missing a thought_signature
    with the handler 200 200 + the model's answer

    Also verified across a four-turn conversation with three sequential tool calls.

Gemini 3 models reason before they act, and when that reasoning produces a function call
the response carries an opaque signature at
tool_calls[].extra_content.google.thought_signature which must be sent back with the same
tool call on the next turn. The reasoning is never returned as text, so the signature is
the only thing carrying it forward, and Gemini rejects a request that omits it:

  400 INVALID_ARGUMENT - "Function call is missing a thought_signature in functionCall
  parts. This is required for tools to work correctly."

The field sits outside the OpenAI schema, so the OpenAI .NET SDK and
Microsoft.Extensions.AI drop it while remapping the response, and the follow-up request is
rebuilt without it. The first call of a conversation succeeds and the second fails, which
in practice means an agent dies on its first tool result -- catalog and document generation
never get past their opening step.

Handle it at the HTTP layer, where the raw JSON is still intact, following the pattern
FinishReasonNormalizingHandler already uses for a different Gemini/OpenAI mismatch on the
same layer. ThoughtSignatureHandler records each signature as the response streams past,
keyed by tool call id, and restores it on any later request naming the same call. Responses
are only read, never modified; only requests bound for Gemini's host are touched; parsing
failures are swallowed so a malformed chunk can never break a response; and the cache is
bounded, since a signature runs to several kilobytes.

Verified against the live API: without the handler the second turn of a tool conversation
returns 400 with the signature error, with it the same conversation returns 200 and the
model's answer.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant