Skip to content

From NVIDIA Megatron-LM for visibility - #18

Open
RaymondLi0 wants to merge 7709 commits into
bigcode-project:multi-query-attentionfrom
NVIDIA:main
Open

From NVIDIA Megatron-LM for visibility#18
RaymondLi0 wants to merge 7709 commits into
bigcode-project:multi-query-attentionfrom
NVIDIA:main

Conversation

@RaymondLi0

@RaymondLi0 RaymondLi0 commented Jan 24, 2023

Copy link
Copy Markdown
Collaborator

No description provided.

@RaymondLi0
RaymondLi0 changed the base branch from multi-query-attention to before-merge June 20, 2023 20:12
@RaymondLi0
RaymondLi0 changed the base branch from before-merge to multi-query-attention June 20, 2023 20:12
svcnemo-autobot and others added 27 commits July 29, 2026 19:37
@svcnemo-autobot
test(inference): MCORE-561 trust FP8 metadata in DeepSeek checkpoints (…
eab6551 
…#6049)

Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>
@balasaajay
ci: fix test notifications for weekly/release tests (#6055)
3f88509 
Signed-off-by: Ajay Balasa <abalasa@nvidia.com>
@tdene
Allow the MInf sync API to run serve() (#6123)
8f1c99c 
Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>
@nvcsathe @wdykas
Add disaggregated KV transfer backends (#5861)
77c8772 
Signed-off-by: Chaitra Sathe <csathe@nvidia.com>
Signed-off-by: Will Dykas <wdykas@nvidia.com>
Co-authored-by: Will Dykas <wdykas@nvidia.com>
@Phlip79
Reorganize a few docs (#6109)
9436656 
Signed-off-by: Philip Petrakian <ppetrakian@nvidia.com>
@fanshiqing
Perf: skip per-param copy_ dispatch in the MXFP8 param copy-back (#6094)
8e57bb6 
Signed-off-by: Shiqing Fan <shiqingf@nvidia.com>
@lmcafee-nvidia @shanmugamr1992 @claude
test: add async scheduling inference coverage (#6104)
354b5d2 
Signed-off-by: shanmugamr1992 <shanmugamr1992@gmail.com>
Signed-off-by: Lawrence McAfee <lmcafee@nvidia.com>
Co-authored-by: shanmugamr1992 <shanmugamr1992@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
@svcnemo-autobot
test(ci): AUT-1173 set functional tests to ft_launcher (#6128)
ab16414 
Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>
@AAnoosheh
Fix --freeze-all-layers for new model builder path + unit tests (#5926)
ff6b92a 
Signed-off-by: Asha Anoosheh <aanoosheh@nvidia.com>
@tdene @jalbericiola
Fix inference overflow into dummy blocks (#5950)
3fd8f33 
Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>
Co-authored-by: Jorge Albericio <jalbericiola@nvidia.com>
@ko3n1g
fix(tests): prefer staged release assets (#6149)
175b4cc 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@wujingyue @svcnvidia-nemo-ci
Fix Megatron FSDP optimizer under-update (#5976)
ce8a8a7 
Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>
Signed-off-by: svcnvidia-nemo-ci <svcnvidia-nemo-ci@nvidia.com>
Co-authored-by: svcnvidia-nemo-ci <svcnvidia-nemo-ci@nvidia.com>
@wujingyue @claude
Rebind MFSDP v2 sharded grads once per gradient reduction (#6041)
9d416ee 
Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@cuichenx
Fix MLA dynamic inference decode flag (#4902)
0081c84 
Signed-off-by: Chen Cui <chcui@nvidia.com>
@wujingyue
Normalize MFSDP data-parallel axes at fully_shard (#6067)
dc3c9ab 
Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>
@ISEEKYAN
Add Agent Compose package and runtime interface skeleton (#5937)
ab20a60 
Signed-off-by: Yan Bai <bayan@nvidia.com>
@prajwal1210 @claude
Fix GTP full-iteration CUDA-graph capture regression (#6077)
ec53920 
Signed-off-by: Prajwal Singhania <psinghania@nvidia.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
@wdykas
Harden dynamic prefix cache allocation - mamba mostly (#6091)
d04e7a0 
Signed-off-by: William Dykas <wdykas@nvidia.com>
@github-actions
Update copy-pr-bot.yaml [skip ci]
6513e3e
@svcnemo-autobot
ci(cache): AUT-1188 harden container cache selection (#6160)
8543ae4 
Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>
@svcnemo-autobot
ci(nvskills): AUT-1188 fix reusable workflow startup failure (#6162)
b19b1f4 
Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>
@wujingyue
Add MFSDP runtime schedule design (#6031)
d3519d5 
Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>
@utkarsh530 Utkarsh Utkarsh
@claude @shanmugamr1992 @guihong-nv
Feature/enable inference optimized qwen moe (#5700)
8339200 
Signed-off-by: Utkarsh Utkarsh <uutkarsh@nvidia.com>
Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com>
Signed-off-by: shanmugamr1992 <shanmugamr@nvidia.com>
Co-authored-by: Utkarsh Utkarsh <uutkarsh@nvidia.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com>
Co-authored-by: Guihong Li <guihongl@nvidia.com>
Co-authored-by: shanmugamr1992 <shanmugamr@nvidia.com>
@wdykas root
root root root root root root
Bs invar moe (#4871)
57deb43 
Signed-off-by: William Dykas <wdykas@nvidia.com>
Signed-off-by: root <root@nvl72103-T09.cm.cluster>
Signed-off-by: root <root@nvl72094-T13.cm.cluster>
Signed-off-by: root <root@nvl72098-T15.cm.cluster>
Signed-off-by: root <root@nvl72077-T17.cm.cluster>
Signed-off-by: root <root@nvl72005-T13.cm.cluster>
Signed-off-by: root <root@nvl72053-T14.cm.cluster>
Signed-off-by: root <root@nvl72151-T17.cm.cluster>
Co-authored-by: root <root@nvl72103-T09.cm.cluster>
Co-authored-by: root <root@nvl72094-T13.cm.cluster>
Co-authored-by: root <root@nvl72098-T15.cm.cluster>
Co-authored-by: root <root@nvl72077-T17.cm.cluster>
Co-authored-by: root <root@nvl72005-T13.cm.cluster>
Co-authored-by: root <root@nvl72053-T14.cm.cluster>
Co-authored-by: root <root@nvl72151-T17.cm.cluster>
@balasaajay
chore: update pytest version to 9.1.1 in pyproject.toml and uv.lock (#…
4ac5ad0 
…6170)

Signed-off-by: Ajay Balasa <abalasa@nvidia.com>
@lmcafee-nvidia @santhnm2
Fix LRU block accounting for resumed requests (#5995)
ee38649 
Signed-off-by: Lawrence McAfee <lmcafee@nvidia.com>
Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>
Co-authored-by: Keshav Santhanam <ksanthanam@nvidia.com>
@zhongbozhu
[Main] update moe single weight unit test (#6035)
d378bfb 
Signed-off-by: Zhongbo Zhu <zhongboz@nvidia.com>
balasaajay and others added 30 commits August 19, 2026 18:56
@balasaajay
ci: centralize container builds and add cache population workflow (#6646
e7b92ef 
)

Signed-off-by: Ajay Balasa <abalasa@nvidia.com>
@dimapihtar
tokenizers simplification (#6622)
e433171 
Signed-off-by: dimapihtar <dpykhtar@nvidia.com>
Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com>
@chtruong814
ci: Change copy-pr-bot update schedule to 9 a.m. UTC (#6669)
82894c4 
Signed-off-by: Charlie Truong <chtruong@nvidia.com>
@wujingyue
Support mixed-precision optimizers in MFSDP v2 (#6506)
fe1e8f8 
Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>
@svcnemo-autobot
ci: AUT-1606 bump release library to v1.8.10 (#6678)
4e1df1b 
Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>
@santhnm2
Add CUDA graph support for GDP dynamic inference (#6595)
3a56046 
Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>
@yashaswikarnati
Pass process groups to GDP projections (#6687)
a1580eb 
Signed-off-by: ykarnati <ykarnati@nvidia.com>
@rapatel @gautham-kollu
Fix fused MLA and MTP checkpointing for Megatron-FSDP. (#6271)
93924d5 
Signed-off-by: Ritesh Patel <ripatel@nvidia.com>
Co-authored-by: gautham-kollu <gkollu@nvidia.com>
@JF-D
fix: prevent local CUDA graph stream aliasing (#6643)
54b5e98 
Signed-off-by: Jiangfei Duan <jiangfeid@nvidia.com>
@wujingyue
Share MFSDP gradient completion hook (#6695)
cf3b8ee 
Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>
@kwangjunahn
Fix LayerWise buffer routing to respect use_muon = False (#6642)
1d82259 
Signed-off-by: Kwangjun Ahn <kwangjuna@nvidia.com>
@dimapihtar
remove unused dist_checkpointing code (#6663)
893eb05 
Signed-off-by: dimapihtar <dpykhtar@nvidia.com>
Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com>
@Phlip79
Add HybridModel MTP loss control (#6119)
db018b3 
Signed-off-by: Philip Petrakian <ppetrakian@nvidia.com>
@RPrenger @cspades
Add VLM support to the dynamic-batching inference server (#6260)
881f489 
Signed-off-by: rprenger <rprenger@nvidia.com>
Signed-off-by: Cory Ye <cye@nvidia.com>
Signed-off-by: Ryan Prenger <rprenger@nvidia.com>
Co-authored-by: Cory Ye <cye@nvidia.com>
@svcnemo-autobot
ci: AUT-1616 assign root build files to CI owners (#6710)
f300051 
Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>
@yashaswikarnati
Scope activation offload summaries to model ranks (#6652)
28797d0 
Signed-off-by: ykarnati <ykarnati@nvidia.com>
@ilml @claude
Add DP-balanced sequence-packing scheduler and wrap_data_iterator ent…
a9381be 
…ry point (#6684)

Signed-off-by: ilml <tolong@nvidia.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
@wdykas
Fix Refit Cache for mamba (#6672)
fbbc142 
Signed-off-by: wdykas <wdykas@nvidia.com>
@anlthms @codex
Add GDN dynamic inference (#6499)
606b9f4 
Signed-off-by: Anil Thomas <anthomas@nvidia.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
@YangFei1990
[NCCL EP] Add MXFP8 dispatch support (#6524)
7afe031 
Signed-off-by: YangFei1990 <feiw@nvidia.com>
@lauradang @claude
rl: add durable rollout bank for completed groups (#6352)
787649a 
Signed-off-by: Laura Dang <laurad@nvidia.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
@JF-D
fix: ensure GTP params are ready before CUDA graph replay (#6525)
b3c9100 
Signed-off-by: Jiangfei Duan <jiangfeid@nvidia.com>
@deepakn94
Reduce eval loss over the GTP-remat-inclusive data-parallel group (#6717
df197a8 
)

Signed-off-by: Deepak Narayanan <dnarayanan@nvidia.com>
@deepakn94 @fanshiqing
GTP+Muon: support 'blockwise' and 'distributed' tp_mode (#6380)
bb47df5 
Signed-off-by: Shiqing Fan <shiqingf@nvidia.com>
Signed-off-by: Deepak Narayanan <dnarayanan@nvidia.com>
Co-authored-by: Shiqing Fan <shiqingf@nvidia.com>
@chtruong814
ci: isolate tokenizer metadata writes (#6718)
57f56d4 
Signed-off-by: Charlie Truong <chtruong@nvidia.com>
@chtruong814
ci: prioritize baseline cache for merge groups (#6721)
41943e8 
Signed-off-by: Charlie Truong <chtruong@nvidia.com>
@cuichenx @jaredcasper
fix(moe): broadcast padding mask for expert bias counts (#6114)
723db5a 
Signed-off-by: Chen Cui <chcui@nvidia.com>
Co-authored-by: Jared Casper <155158+jaredcasper@users.noreply.github.com>
@github-actions
Update copy-pr-bot.yaml [skip ci]
5b2a621
@svcnemo-autobot
build: AUT-1615 add torch-memory-saver dev dependency (#6701)
ab5e2c9 
Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>
@yashaswikarnati
Apply precision configs to MIMO modalities (#6745)
b4a4c0c 
Signed-off-by: ykarnati <ykarnati@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.