docs: add single-GPU PTQ announcement blog

[Fridah-nv]

What does this PR do?

Type of change: documentation

Adds an announcement blog at docs/source/announcements/single-gpu-ptq.rst — Quantizing a
1.5 TB Kimi-K3 Model on a Single GPU — covering the single-GPU PTQ workflow: disk/CPU offload
(#2008) plus layerwise calibration with per-layer shard export (#2136), with Kimi-K3 on a
single B200 (#2218) as the worked example.

Preview: https://NVIDIA.github.io/Model-Optimizer/pr-preview/pr-2440/announcements/single-gpu-ptq.html

Contents:

• The framing is the memory floor for calibration, not model size. Calibration has been
  all-or-nothing — the forward pass wants every layer resident even though it only reads one at
  a time — so the hardware floor for quantizing a model has tracked the floor for serving
  it. The post's claim is that layerwise calibration plus per-layer export drops that floor
  from one model to one layer, which makes it relevant to anyone allocation-constrained rather
  than only to people quantizing trillion-parameter checkpoints.
• The mechanism is explained as a loop interchange, with before/after pseudocode: data
  outside / depth inside becomes depth outside / data inside. The post makes the consequence
  explicit — after the swap a layer is finished when its inner loop ends, which is what makes
  calibrate(); quantize(); export(); release() well defined and a written shard a truthful
  completion record — and states the cost, namely that the calibration set's activations are
  held at the layer boundary.
• A copy-pasteable recipe and command, with layerwise.export_dir called out as the entire
  switch, and the pre-calibration refusals listed as a check-before-you-burn-a-session box.
• A brief interrupt-and-resume section backed by the SIGKILL-at-25/48 result from feat(export): export each decoder layer as layerwise calibration finishes it #2136.
• Results are Kimi-K3 only, plus an explanation of why peak GPU lands above
  --max_gpu_memory_gb and why host RSS spikes far above --max_cpu_memory_gb during load:
  both flags size accelerate's device map (example_utils.py:937-943), so they govern weight
  placement and not total allocation.
• "What you're trading" — four deliberate consequences of the design: sequential wall
  clock, the current calibration-algorithm restriction, calibration-memory-only scope, and the
  depth-not-width lower bound.

Also adds an announcement card to docs/source/index.rst, matching the shape used by recent
announcement PRs.

Usage

Documentation only; no API. The workflow the post documents:

python examples/hf_ptq/hf_ptq.py \
    --pyt_ckpt_path  <bf16_ckpt> \
    --recipe         models/moonshotai/Kimi-K3/ptq/nvfp4_experts-kv_none_layerwise_export \
    --export_path    <out> \
    --qformat nvfp4 --trust_remote_code --attn_implementation eager \
    --offload_folder <scratch> --max_gpu_memory_gb 140 --max_cpu_memory_gb 1700 \
    --calib_size 256 --batch_size 8 --skip_generate

Testing

Documentation only; no code paths change.

• nox -s docs succeeds: build succeeded under --fail-on-warning, which is the same gate
  the Docs workflow applies.
• Verified in the rendered HTML, not only the source: the page title, the announcement card
  in index.html, the reshaped Results list-table, and the MXFP4 caveat including its
  MXFP4 → BF16 → NVFP4 arrow chain.
• pre-commit run --files docs/source/announcements/single-gpu-ptq.rst docs/source/index.rst
  passes, including the three RST format hooks; no hook modified either file.
• CI Docs workflow green, and the PR preview deploys and serves the page (link above).

Two issues in noxfile.py surfaced while building these docs locally. Both are pre-existing
and out of scope here, but they cost an hour to diagnose and will hit anyone on a shared
machine, so flagging them for a separate PR:

• noxfile.py:33 hardcodes nox.options.envdir = "/tmp/.nox". On a shared box that directory
  is often already owned by another user, and every nox session then fails with
  Permission denied. Workaround: nox --envdir /tmp/.nox-$USER -s docs.
• noxfile.py:191-192 reuses -d /tmp/doctrees while noxfile.py:186-187 deletes
  docs/source/reference/generated on every run. The first build works; the second and later
  ones die with FileNotFoundError: .../generated/modelopt.deploy.rst does not exist, which
  looks like a content error but is a stale doctree cache. Workaround: rm -rf /tmp/doctrees.

Before your PR is "Ready for review"

• Is this change backward compatible?: N/A
• If you copied code from any other sources or added a new PIP dependency, did you follow guidance in CONTRIBUTING.md: N/A
• Did you write any new necessary tests?: N/A
• Did you update Changelog?: N/A — documentation only
• Did you get Claude approval on this PR?: ❌

Additional Information

Draft until #2218 merges and its Kimi-K3 figures are reconfirmed. Two items remain:

1. TODO(reconfirm) markers in the results table — wall clock, peak GPU, peak RSS, and the
   expert-projection count. feat(export): Kimi-K3 on layerwise fused export #2218 gated leaving draft on a fresh full-K3 run because its
   original numbers predate the rebase and feat(export): support multimodal and MTP models in layerwise export #2303's exporter redesign, so these should be filled
   from that run rather than carried forward.
2. No accuracy claim anywhere, by design. feat(export): Kimi-K3 on layerwise fused export #2218 records generation and accuracy as
   unvalidated. The post makes only the narrower equivalence claim from feat(export): export each decoder layer as layerwise calibration finishes it #2136 — per-layer export
   produces the same checkpoint as whole-model export — and states it separately so it cannot be
   read as an accuracy result. Worth preserving that separation through review.

Resolved: whether the "1.5 TB" figure was consistent with --pyt_ckpt_path <bf16_ckpt>. It
is: 1.5 TB is the published checkpoint, which stores its routed experts in MXFP4, and the run
dequantizes them to BF16 before calibrating. The end-to-end path is MXFP4 → BF16 → NVFP4. The
post now states this in a note after the command, along with the fact that the round trip cannot
recover what the source quantization discarded and that its accuracy cost is out of scope —
Kimi-K3 is there to demonstrate the workflow, not as a recommended recipe.

Deliberately out of scope: examples/kimi/kimi_k3/quantize_to_nvfp4.py, the calibration-free
MXFP4→NVFP4 shard-cast path. It is a different K3 workflow and mentioning it here would confuse
readers about which one to use.

Related: #2008, #2136, #2303 (merged); #2218 (draft).

🤖 Generated with Claude Code

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

PR Preview Action v1.8.1
🚀 View preview at https://NVIDIA.github.io/Model-Optimizer/pr-preview/pr-2440/
Built to branch gh-pages at 2026-09-15 23:18 UTC. Preview will be ready when the GitHub Pages deployment is complete.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 71.16%. Comparing base (c7ed23a) to head (5729586).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2440      +/-   ##
==========================================
- Coverage   71.42%   71.16%   -0.27%     
==========================================
  Files         590      590              
  Lines       64710    65361     +651     
==========================================
+ Hits        46222    46515     +293     
- Misses      18488    18846     +358     

Flag	Coverage Δ
unit	57.86% <ø> (+0.04%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.