[Issue]: Gemma4 tool-call parsing produces wrong name and arguments (v1.0.5)

[skarpov2]

Problem Description

Title: Gemma4 tool-call parsing produces wrong name and arguments (v1.0.5)

Body:

Hi FastFlowLM team,

First, thanks for the v1.0.5 release and for the recent improvements to Gemma4 tool-call argument parsing. Unfortunately, I'm still hitting a bug where tool calls are parsed incorrectly, and the resulting tool invocation is unusable by OpenAI-compatible clients (OpenClaw in my case).

Environment
FastFlowLM version: v1.0.5

Model: gemma4-it:12b (also reproducible with gemma4-it:e4b)

Server mode: flm serve gemma4-it:12b (OpenAI-compatible endpoint, /v1/chat/completions)

Client: OpenClaw 2026.9.3 (uses api=openai-completions, tool_choice: auto, stream: true)

Host: AMD Ryzen AI NPU (Strix Point)

What the model emits
The model produces a correct native Gemma4 tool call:

text
<|tool_call>call:tool_call{args:{query:"тренировки на пресс или ноги"},id:<|"|>memory_search<|"|>}<tool_call|>
Note: the actual function name is memory_search, and its argument is {"query": "..."}. The surrounding tool_call{...} wrapper is what Gemma4 emits, and the real tool identity lives in the id field.

What FLM parses it into
FLM's own log line shows the parsed result:

text
[FLM] Tool name: tool_call
[FLM] Tool args: {"args":{"query":"тренировки на пресс или ноги"},"id":"memory_search"}
[FLM] Tool JSON: {"name":"tool_call","arguments":"{"args":{"query":"тренировки на пресс или ноги"},"id":"memory_search"}"}
What the client receives
The OpenAI-compatible response contains:

json
{
"function": {
"name": "tool_call",
"arguments": "{"args":{"query":"тренировки на пресс или ноги"},"id":"memory_search"}"
},
"id": "call17890662170",
"type": "function"
}
Expected behavior
For the same model output, FLM should produce:

json
{
"function": {
"name": "memory_search",
"arguments": "{"query":"тренировки на пресс или ноги"}"
},
"id": "call17890662170",
"type": "function"
}
i.e.:

name should be the real tool name (memory_search), not the wrapper tool_call.

arguments should be the inner argument object ({"query": "..."}), not the whole {args, id} envelope.

Impact
Because the emitted name is tool_call (which does not exist in the client's tool list), the tool is never executed. The client sees an unknown function, drops the call, and the model then hallucinates / "negotiates" a response instead of returning real data. This completely breaks agentic tool use (memory search, web search, file operations, etc.) for Gemma4 models over the OpenAI-compatible endpoint.

Reproduction
Minimal reproduction with a single tool:

bash
curl http://127.0.0.1:52625/v1/chat/completions
-H "Content-Type: application/json"
-d '{
"model": "gemma4-it:12b",
"messages": [
{"role": "user", "content": "Find my press or leg workouts in memory"}
],
"tools": [
{
"type": "function",
"function": {
"name": "memory_search",
"description": "Search long-term memory",
"parameters": {
"type": "object",
"properties": {
"query": {"type": "string"}
},
"required": ["query"]
}
}
}
],
"tool_choice": "auto"
}'
The response will contain a tool call with name: "tool_call" instead of name: "memory_search".

Notes
I understand v1.0.5 improved Gemma4 tool-call argument parsing for edge cases. This bug looks like a separate issue: the function name mapping and the argument unwrapping are still wrong.

The same model, when served via a different OpenAI-compatible backend (e.g. llama.cpp with Qwen-family models), produces correct tool calls, so the model family and the client are not the problem — the parsing inside FLM is.

This was previously reported in the OpenClaw tracker as a Gemma4 tool-call recovery issue (openclaw#104868), but the fix needs to happen on the FLM side, since FLM is the component that produces the malformed OpenAI-format tool call.

Request
Could you please:

Map the Gemma4 call:{...} envelope so that name becomes the real tool name (from the id field) and arguments becomes the inner argument object.

Add a test case that covers this exact pattern, since it's the standard output shape for Gemma4 tool calls.

Happy to provide more logs or test a patched build if that helps.

Thanks!

Operating System

Linux Debian 13

CPU

24x AMD Ryzen AI 9 HX 370 w/ Radeon 890M

GPU

AMd Radeon 890M/XDNA RyzenAI-npu4

ROCm Version

ROCm 7.0.2 (rocm-core package)

Installation Method

Built from source

Installed ROCm Packages / Versions

amdgpu-install script (v7.0.2) with --usecase=rocm

wget https://repo.radeon.com/amdgpu-install/7.0.2/ubuntu/noble/amdgpu-install_7.0.2.70002-1_all.deb
dpkg -i amdgpu-install_7.0.2.70002-1_all.deb
amdgpu-install -y --usecase=rocm

ROCm Component

No response

Steps to Reproduce

2. Install ROCm

RUN echo "=== INSTALLING amdgpu-install ===" &&
apt-get update &&
apt-get install -y wget &&
wget -q https://repo.radeon.com/amdgpu-install/7.0.2/ubuntu/noble/amdgpu-install_7.0.2.70002-1_all.deb &&
dpkg -i amdgpu-install_7.0.2.70002-1_all.deb || true &&
apt-get install -f -y &&
echo "=== INSTALLING ROCm 7.0.2 via amdgpu-install ===" &&
amdgpu-install -y --usecase=rocm &&
apt-get clean && rm -rf /var/lib/apt/lists/* && rm amdgpu-install_*.deb

3. XRT

RUN mkdir -p /opt/xilinx/xrt/lib
/opt/xilinx/xrt/include/xrt/deprecated
/opt/xilinx/xrt/include/xrt/detail
/opt/xilinx/xrt/include/xrt/experimental

COPY xrt/xrt/lib/libxrt_coreutil.so* /opt/xilinx/xrt/lib/

RUN curl -sL https://raw.githubusercontent.com/Xilinx/XRT/master/src/runtime_src/core/include/xrt/xrt.h -o /opt/xilinx/xrt/include/xrt/xrt.h &&
curl -sL https://raw.githubusercontent.com/Xilinx/XRT/master/src/runtime_src/core/include/xrt/xrt_bo.h -o /opt/xilinx/xrt/include/xrt/xrt_bo.h &&
curl -sL https://raw.githubusercontent.com/Xilinx/XRT/master/src/runtime_src/core/include/xrt/xrt_device.h -o /opt/xilinx/xrt/include/xrt/xrt_device.h &&
curl -sL https://raw.githubusercontent.com/Xilinx/XRT/master/src/runtime_src/core/include/xrt/xrt_kernel.h -o /opt/xilinx/xrt/include/xrt/xrt_kernel.h &&
curl -sL https://raw.githubusercontent.com/Xilinx/XRT/master/src/runtime_src/core/include/xrt/xrt_uuid.h -o /opt/xilinx/xrt/include/xrt/xrt_uuid.h &&
curl -sL https://raw.githubusercontent.com/Xilinx/XRT/master/src/runtime_src/core/include/xrt/deprecated/xrt.h -o /opt/xilinx/xrt/include/xrt/deprecated/xrt.h &&
curl -sL https://raw.githubusercontent.com/Xilinx/XRT/master/src/runtime_src/core/include/xrt/deprecated/xclerr.h -o /opt/xilinx/xrt/include/xrt/deprecated/xclerr.h &&
curl -sL https://raw.githubusercontent.com/Xilinx/XRT/master/src/runtime_src/core/include/xrt/detail/xclbin.h -o /opt/xilinx/xrt/include/xrt/detail/xclbin.h &&
curl -sL https://raw.githubusercontent.com/Xilinx/XRT/master/src/runtime_src/core/include/xrt/detail/xrt_mem.h -o /opt/xilinx/xrt/include/xrt/detail/xrt_mem.h &&
curl -sL https://raw.githubusercontent.com/Xilinx/XRT/master/src/runtime_src/core/include/xrt/detail/pimpl.h -o /opt/xilinx/xrt/include/xrt/detail/pimpl.h &&
curl -sL https://raw.githubusercontent.com/Xilinx/XRT/master/src/runtime_src/core/include/xrt/detail/config.h -o /opt/xilinx/xrt/include/xrt/detail/config.h &&
curl -sL https://raw.githubusercontent.com/Xilinx/XRT/master/src/runtime_src/core/include/xrt/experimental/xrt_exception.h -o /opt/xilinx/xrt/include/xrt/experimental/xrt_exception.h &&
curl -sL https://raw.githubusercontent.com/Xilinx/XRT/master/src/runtime_src/core/include/xrt/experimental/xrt_xclbin.h -o /opt/xilinx/xrt/include/xrt/experimental/xrt_xclbin.h &&
curl -sL https://raw.githubusercontent.com/Xilinx/XRT/master/src/runtime_src/core/include/xrt/detail/abi.h -o /opt/xilinx/xrt/include/xrt/detail/abi.h &&
curl -sL https://raw.githubusercontent.com/Xilinx/XRT/master/src/runtime_src/core/include/xrt/detail/any.h -o /opt/xilinx/xrt/include/xrt/detail/any.h &&
curl -sL https://raw.githubusercontent.com/Xilinx/XRT/master/src/runtime_src/core/include/xrt/detail/param_traits.h -o /opt/xilinx/xrt/include/xrt/detail/param_traits.h &&
curl -sL https://raw.githubusercontent.com/Xilinx/XRT/master/src/runtime_src/core/include/xrt/detail/ert.h -o /opt/xilinx/xrt/include/xrt/detail/ert.h &&
curl -sL https://raw.githubusercontent.com/Xilinx/XRT/master/src/runtime_src/core/include/xrt/detail/span.h -o /opt/xilinx/xrt/include/xrt/detail/span.h &&
curl -sL https://raw.githubusercontent.com/Xilinx/XRT/master/src/runtime_src/core/include/xrt/experimental/xrt_fence.h -o /opt/xilinx/xrt/include/xrt/experimental/xrt_fence.h &&
curl -sL https://raw.githubusercontent.com/Xilinx/XRT/master/src/runtime_src/core/include/xrt/experimental/xrt_hw_context.h -o /opt/xilinx/xrt/include/xrt/experimental/xrt_hw_context.h &&
curl -sL https://raw.githubusercontent.com/Xilinx/XRT/master/src/runtime_src/core/include/xrt/xrt_hw_context.h -o /opt/xilinx/xrt/include/xrt/xrt_hw_context.h &&
curl -sL https://raw.githubusercontent.com/Xilinx/XRT/master/src/runtime_src/core/include/xrt/experimental/xrt_elf.h -o /opt/xilinx/xrt/include/xrt/experimental/xrt_elf.h &&
curl -sL https://raw.githubusercontent.com/Xilinx/XRT/master/src/runtime_src/core/include/xrt/experimental/xrt_ext.h -o /opt/xilinx/xrt/include/xrt/experimental/xrt_ext.h &&
curl -sL https://raw.githubusercontent.com/Xilinx/XRT/master/src/runtime_src/core/include/xrt/detail/bitmask.h -o /opt/xilinx/xrt/include/xrt/detail/bitmask.h &&
curl -sL https://raw.githubusercontent.com/Xilinx/XRT/master/src/runtime_src/core/include/xrt/experimental/xrt_module.h -o /opt/xilinx/xrt/include/xrt/experimental/xrt_module.h &&
curl -sL https://raw.githubusercontent.com/Xilinx/XRT/master/src/runtime_src/core/include/xrt/experimental/xrt_kernel.h -o /opt/xilinx/xrt/include/xrt/experimental/xrt_kernel.h

4. FLM

COPY flm-real /opt/fastflowlm/bin/flm-real
COPY flm /opt/fastflowlm/bin/flm
COPY lib /opt/fastflowlm/lib
COPY model_list.json /opt/fastflowlm/bin/
COPY model_info.json /opt/fastflowlm/bin/

RUN chmod +x /opt/fastflowlm/bin/flm-real /opt/fastflowlm/bin/flm &&
ln -s /opt/fastflowlm/bin/flm /usr/local/bin/flm

5. vars

ENV PATH=/usr/local/bin:/usr/bin:/bin:/opt/fastflowlm/bin:$PATH
LD_LIBRARY_PATH=/opt/fastflowlm/lib:/opt/xilinx/xrt/lib:/usr/lib/x86_64-linux-gnu

WORKDIR /root/.config/flm

ENTRYPOINT ["flm", "serve", "--host", "0.0.0.0", "--port", "52625"]

(Optional for Linux users) Output of rocminfo --support

rocminfo --support output

Paste output here

Additional Information

No response

[Cyronius]

The envelope you hit is real, but there is a second Gemma 4 bug underneath it that I think hurts agents more, and it is easy to check.

Tool results are truncated when thinking is off. With thinking off, the chat template ends a normal turn with an empty thought block (<|channel>thought\n<channel|>), and FLM trims those four tokens off the prefill and feeds them back during decode so its checkpoint lands before them. After a tool result the template ends the prompt at <tool_response|> and writes no thought block, but the trim still runs. It removes the last four tokens of the tool result and injects a thought block that was never in the prompt.

To reproduce, send a completed tool call whose result ends in a value the model has to repeat, and ask for it back verbatim at temperature 0:

{"role": "tool", "tool_call_id": "c1", "content": "{\"a_status\": \"open\", \"zz_code\": \"ZQX-7731\"}"}

On gemma4-it:12b:

• reasoning_effort: "none" — the server prefills 129 tokens for a prompt that renders to 133, and the answer is ZQX-773.
• reasoning_effort: "low" — it prefills all 135, and the answer is ZQX-7731.

The four dropped tokens are 1, <|"|>, }, <tool_response|>. A weather tool that returned 18 °C got reported as 15 °C the same way. So every tool loop in the default mode feeds the model a damaged result, and a short result loses its payload entirely. Trimming only when the rendered prompt actually ends with that block fixes it. One side effect worth knowing: once the prompt is whole, the model writes its own empty thought block in continuations, which then wants filtering so clients do not see a reasoning_content of one newline.

The envelope in this issue. I could not get the model to emit it in three tries — plain, an agent-style system prompt, and your Russian query all produced the direct form — so I cannot confirm the trigger. But the parser has no unwrap step, so when it does happen the client gets name: "tool_call" exactly as you saw. Recovering the inner name and arguments when the outer name is tool_call is small and cannot misfire unless someone really has a tool by that name.

One more, easy to hit. A tool whose parameter is typed ["string", "null"] — Pydantic optionals, most MCP servers — makes the template throw inside minja (get<T> not defined for this value type) and the whole request fails, even when the user never asks for that tool. Folding the type array to its first non-null member plus nullable: true keeps the template as Google shipped it.

All three are fixed and tested in the OpenFlowLM fork, with unit tests for the parser and integration tests for the trim, if the diffs are useful to you: Atomic-Germ/OpenFlowLM-Next#59. Happy to open the equivalent PR here if you would like it.