Skip to content

Require PTQ recipe guidance before selecting quantization candidates #2478

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Zhehao-Hu wants to merge 1 commit into
base: main
Choose a base branch
from
+22 −0
Open

Require PTQ recipe guidance before selecting quantization candidates #2478

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions plugins/modelopt/skills/quant-recipe-search/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,24 @@ Default success rule: maximize the chosen performance objective while keeping
each benchmark within 1 percentage point of the matching BF16/FP16 baseline.
Near-threshold or noisy regressions require reruns before making a decision.

## Recipe Guidance

Before choosing a recipe, read `modelopt_recipes/ptq.md` in the source checkout
used for PTQ. Resolve all `modelopt_recipes/...` paths against that checkout.
If it is missing, follow
[Env-1. Get ModelOpt source](../common/environment-setup.md#env-1-get-modelopt-source).
Report if the guide cannot be read.
Comment on lines +62 to +66

@claude claude Bot Sep 18, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SUGGESTION] The "source checkout" requirement is the right call — I confirmed pyproject.toml ships only modelopt_recipes = ["**/*.yml", "**/*.yaml"] as package data, so ptq.md genuinely does not exist in a wheel and a pip-installed user must follow Env-1. But that same asymmetry creates a silent skew this paragraph doesn't close.

What happens: for a pip-installed user, ptq.md comes from a freshly cloned checkout while the YAMLs also exist in the installed wheel. modelopt/recipe/loader.py:_resolve_recipe_path resolves a relative --recipe modelopt_recipes/... filesystem-first (cwd-relative), then falls back to BUILTIN_RECIPES_LIB. So if the PTQ run's cwd is not that checkout, the guide and YAML you reviewed come from the clone while the YAML that actually calibrates comes from the installed library — potentially a different revision.

Why it matters: the fallback is silent for this case. The FutureWarning at modelopt/recipe/loader.py:89 only fires for deprecated tier prefixes, not for a checkout-vs-built-in divergence, so nothing surfaces the mismatch. The cited rationale would describe a recipe that never ran.

Line 70's "ensure the quantization run uses the recipe and imported configs reviewed" states the requirement but names no mechanism. Naming one makes it executable:

Suggested change
Before choosing a recipe, read `modelopt_recipes/ptq.md` in the source checkout
used for PTQ. Resolve all `modelopt_recipes/...` paths against that checkout.
If it is missing, follow
[Env-1. Get ModelOpt source](../common/environment-setup.md#env-1-get-modelopt-source).
Report if the guide cannot be read.
used for PTQ. Resolve all `modelopt_recipes/...` paths against that checkout, and
pass candidates to `ptq` as absolute paths under it (or run PTQ with that
checkout as the working directory) so a relative path cannot fall back to the
installed recipe library at a different revision.


Use the guide to choose quantization scope, KV-cache scheme, and calibration
method. Inspect the recipe YAML and its imported configs. When handing the
recipe to `ptq`, ensure the quantization run uses the recipe and imported configs
reviewed. Then perform the existing coverage, compatibility, and evaluation
checks.
Comment on lines +68 to +72

@claude claude Bot Sep 18, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SUGGESTION] "Inspect the recipe YAML and its imported configs" + "ensure the quantization run uses the recipe and imported configs reviewed" is not achievable as literally written for the general-recipe path, because the skill you delegate to steers away from YAMLs there.

plugins/modelopt/skills/ptq/SKILL.md:71 says: "General PTQ recipes in modelopt_recipes/general/ptq/ correspond to the same formats — --qformat is the simpler way to use them." A --qformat nvfp4 run loads no recipe YAML at all, so whenever this guidance leads to a general scheme (the common case for a first candidate — ptq.md §Choosing a general recipe), the delegated run can legitimately never touch the file whose sections you just cited. The instruction then either blocks the documented ptq path or gets quietly ignored.

Model-specific recipes are unaffected — ptq/SKILL.md:56 already prefers --recipe <path> and already requires inspecting include/exclude patterns, which composes cleanly with this section.

Worth splitting the two cases explicitly, e.g.:

Use the guide to choose quantization scope, KV-cache scheme, and calibration
method. Inspect the recipe YAML and its imported configs. When handing a recipe
to `ptq`, keep the reviewed artifacts authoritative: pass `--recipe` so the run
loads exactly what you inspected. `--qformat` is acceptable only for a general
scheme whose `modelopt_recipes/general/ptq/` YAML was the basis of the review —
note that substitution in the portfolio table. Then perform the existing
coverage, compatibility, and evaluation checks.


Cite the sections behind your choice and explain any departures from their
advice. Re-read the guide after pulling changes, switching branches or checkouts,
or editing it. Otherwise, reuse the guidance already read.

## Search Space

Keep the search space explicit. A candidate recipe is a tuple across these axes:
Expand Down Expand Up @@ -96,6 +114,7 @@ Do not collapse the search to one dimension such as numeric format only. Read
estimates.

3. **Pick baselines and first candidates**
- Apply [Recipe Guidance](#recipe-guidance) before selecting candidates.
- Always include BF16/FP16 and a near-lossless FP8/W8A8 baseline unless FP8
itself is the target.
- For ModelOpt work, start from `modelopt_recipes`: model-specific recipes
Expand Down Expand Up @@ -161,6 +180,9 @@ verbosity, positional exclusions, and decision.

## References

- For scope, KV-cache, and calibration guidance, read `modelopt_recipes/ptq.md`
in the PTQ source checkout before selecting candidates; see
[Recipe Guidance](#recipe-guidance) for when to re-read it.
- For recipe design, search-space details, sensitivity, and active-cost
accounting, read `references/recipe_iteration.md`.
- For a concrete prior case study, read `references/qwen36_case_study.md` only
Expand Down
Loading