Skip to content

SM80 (Ampere/A100/170HX) GSQ INT3 decode speed bottlenecked by upstream Humming kernel, not GSQ quality #8

Description

@yangfangyunyun

Summary

When serving a GSQ INT3 checkpoint (e.g. ISTA-DASLab/Qwen3.8-27B-3Bit-GSQ) via vLLM on an SM80 (Ampere) GPU — I tested on an NVIDIA CMP 170HX (64GB, ~1591 GB/s measured HBM bandwidth) — single-request decode throughput is noticeably lower than expected for the hardware's memory bandwidth (~60 tok/s), while the same GPU serving the equivalent FP8 checkpoint + speculative decoding reaches 120+ tok/s with >90% HBM bandwidth utilization.

This isn't a GSQ quantization-quality issue — the model output quality is excellent. It traces to the Humming GEMM kernel (the backend vLLM uses for low-bit weight-only inference), which currently gets only ~34-43% effective HBM bandwidth at M=1 (single-token decode) on SM80, confirmed both by my own kernel-level profiling and by inclusionAI's own published A800 benchmark data (humming_w3a16_int3_float16.json in their repo shows the same ~34% figure on A800).

I filed the detailed technical report upstream: inclusionAI/humming#72

Why I'm posting here too

Anyone deploying GSQ INT3/2bit checkpoints on an Ampere-class card (A100, A800, RTX 30-series, CMP 170HX, etc.) via vLLM will likely hit the same decode-speed ceiling regardless of GSQ calibration quality. Newer architectures (SM90/SM120/SM121) have received dedicated small-batch kernel tuning in Humming recently; SM80 hasn't (as of humming-kernels==0.1.10, the SM80 heuristics file's most recent change is an interface-only signature update).

Posting this here mainly so future SM80 users searching for "GSQ slow decode" or "vLLM GSQ low tok/s" can find the root cause and the upstream issue, rather than assuming it's a quantization problem with the GSQ checkpoint itself. Happy to share my full benchmark script/data if useful for calibration-vs-kernel disambiguation on your side.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions