Skip to content

Support paged stashing for GroupedLinear activations - #3423

Draft
lhb8125 wants to merge 1 commit into
NVIDIA:mainfrom
lhb8125:denliu/paged-stash-grouped-tensor
Draft

Support paged stashing for GroupedLinear activations#3423
lhb8125 wants to merge 1 commit into
NVIDIA:mainfrom
lhb8125:denliu/paged-stash-grouped-tensor

Conversation

@lhb8125

@lhb8125 lhb8125 commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Description

Enable paged stashing integrations to identify the activation storage saved by the device-initiated PyTorch GroupedLinear autograd function.

  • Mark input_to_save immediately before prepare_for_saving / save_for_backward.
  • Extend mark_grouped_tensor to support unquantized BF16/FP16 rowwise storage.
  • Preserve the existing quantized columnwise data and scale-inverse markers.
  • Add unit coverage for rowwise and columnwise storage.

This is a follow-up to #3224. The paired Megatron-LM integration is NVIDIA/Megatron-LM#6828.

Validation

  • Black 24.4.2 with the repository pre-commit arguments
  • Ruff
  • Full TE license check
  • Four-rank GB300 Megatron-LM integration test using HybridEP, MXFP8, device-initiated GroupedLinear, and GPU-only paged stash: TestPagedStashingGroupedTensor::test_forward_backward_without_op_fuser
    • all four ranks passed
    • activation metadata captured
    • stash/reload completed without overflow
    • output and input-gradient parity passed

@github-actions github-actions Bot added the community-contribution PRs from external contributor outside the core maintainers, representing community-driven work. label Aug 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-contribution PRs from external contributor outside the core maintainers, representing community-driven work.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant