docs(embeddinggemma): quantization is bf16, not Q4_1

[Aitschend3251]

docs/docs/models/embeddinggemma.md line 14 declares Q4_1. The shipped model is not quantised.

What the repo itself says. src/model_list.json gives embed-gemma/300m:

"details": { "format": "NPU2", "family": "embed-gemma", "quantization_level": "none" }

It is the only entry in that file carrying none, and the docs page is the only place Q4_1 appears for this model.

What the artefact says. model.q4nx is 615,197,168 bytes for a ~308M-parameter model — 2.0 bytes per parameter, i.e. bf16 despite the .q4nx extension. Genuine 4-bit weights would be near 0.5 bytes/parameter. Verified independently on two machines and two FLM builds (v0.9.45 and v1.0.3), and a header parse reports 316 of 316 tensors as BF16; the same parser finds 197 quantised tensors in the Qwen3-1.7B-NPU2 container, so that is a finding rather than a parser artefact.

Why bf16 and not none. docs/docs/models/smolvla.md already uses - **Quantization:** bf16, so this follows existing usage on the docs side while agreeing with model_list.json.

Why it is worth fixing. This line sent #661 down a dead end: the opening report named coarse quantisation as the likely cause of an embedding defect, on the strength of this line, and two of us spent time ruling it out. The likely origin looks innocent — the v1.0.3 release notes upgrade the Qwen3.5 family and Qwen3.6-MoE "from Q4_1 to Q4_K", so Q4_1 was a real value for the LLM containers; it just does not describe this one.

One-line change, docs only.

[Atomic-Germ]

Is there a reason we aren't just using safetensors, since it is not quantized anyway? Other than hiding xclbin stuff I mean.

[Aitschend3251]

Not my call on the format, but two things from the artefacts, since I had them open.

The xclbins aren't hidden in the container — they ship as plain files next to it. Embedding-Gemma-300M-NPU2 has mm.xclbin, mv.xclbin, attn_full_mask.xclbin and sliding_attn.xclbin; Qwen3-1.7B-NPU2 has mm, attn, dequant, layer and lm_head. There'd be no room for them inside model.q4nx either: 615,197,168 B against ~307.6M bf16 parameters leaves about 34 kB of slack, and those four xclbins come to 829 kB.

And "not quantized anyway" is true of this model and no other. embed-gemma/300m is the only one of 38 entries in src/model_list.json with quantization_level: "none" — the rest are 25× Q4_1, 7× Q4_0, 5× Q4_K — and the same header parse that reports 316/316 BF16 here finds 197 quantised tensors at 5.000 bits/parameter in Qwen3-1.7B-NPU2. So whatever the container does, it has to keep doing it for the other 37, and safetensors for the single unquantised model would mean two load paths for one model. Whether it could carry the quantised ones I don't know — F4 and F6_E2M3 dtypes exist now, so "it can't" would be too strong a claim from me.

Worth saying that the premise of your question is this PR: until it lands, the docs still say Q4_1 for the one model that isn't quantised at all.

_{Drafted with Claude Code. The measurements are my own, and I re-checked every number above before posting.}

[Atomic-Germ]

Not my call on the format, but two things from the artefacts, since I had them open.

The xclbins aren't hidden in the container — they ship as plain files next to it. Embedding-Gemma-300M-NPU2 has mm.xclbin, mv.xclbin, attn_full_mask.xclbin and sliding_attn.xclbin; Qwen3-1.7B-NPU2 has mm, attn, dequant, layer and lm_head. There'd be no room for them inside model.q4nx either: 615,197,168 B against ~307.6M bf16 parameters leaves about 34 kB of slack, and those four xclbins come to 829 kB.

And "not quantized anyway" is true of this model and no other. embed-gemma/300m is the only one of 38 entries in src/model_list.json with quantization_level: "none" — the rest are 25× Q4_1, 7× Q4_0, 5× Q4_K — and the same header parse that reports 316/316 BF16 here finds 197 quantised tensors at 5.000 bits/parameter in Qwen3-1.7B-NPU2. So whatever the container does, it has to keep doing it for the other 37, and safetensors for the single unquantised model would mean two load paths for one model. Whether it could carry the quantised ones I don't know — F4 and F6_E2M3 dtypes exist now, so "it can't" would be too strong a claim from me.

Worth saying that the premise of your question is this PR: until it lands, the docs still say Q4_1 for the one model that isn't quantised at all.

_{Drafted with Claude Code. The measurements are my own, and I re-checked every number above before posting.}

But we already know precisely what the q4nx container is and isn't, how it's layed out, and what it does. They keep fiddling with small but annoying parts, and xclbins are per-family shape. It definitely does not ever hide anything but the tensors in it.

Importantly, embedding models aren't the same as other models no matter how it's done. It seems to Me there's no sense packing with no quants, it just requires unpacking. There are legitimately two paths.

[Aitschend3251]

Thanks for elaborating — I agree the q4nx container format itself isn't the issue, and I'm not proposing to change how this model is packed. My point is narrower: regardless of which of the two legitimate packing paths a model takes, the docs table should state which one was actually used for embed-gemma/300m. Right now it says Q4_1, and the model on disk is bf16 (per model_list.json's own quantization_level: "none", the file-size-per-parameter math, and the header parse showing 316/316 BF16 tensors).

That mismatch is what caused the confusion in #661 — anyone debugging VRAM/size expectations for this model would reasonably start from the documented Q4_1 and be off by roughly 4x (bf16 ≈ 2.0 bytes/param vs. ≈ 0.5 bytes/param for real 4-bit).

So: no objection to embedding models having their own packing path — just asking that the doc table reflect which path this particular model actually took. Happy to adjust the label if bf16 isn't your preferred wording (e.g. "unquantized"/"none", matching smolvla.md's existing "Quantization: bf16" convention) — the goal is just that it matches reality.