fix: fix grad accumulation/overlap under ZeRO-2 - #213
Open
Chamberlain0w0 wants to merge 3 commits into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
背景
先前的实现
is_last_microbatch的处理过于草率,导致目前 ZeRO-2 同时开启梯度累积和overlap_grad_reduce时,当前实现会在第一个 microbatch backward 期间发起 reduce-scatter,并将grad_reduce_dispatched_设置为true。该状态直到
optimizer->step()调用FinishGradSync()后才会重置,因此存在两个问题:temp_full_grad_buffer,造成计算与通信之间的数据竞争。关闭
overlap_grad_reduce时不会触发该问题,因为梯度同步统一在所有 microbatch 完成后的optimizer->step()中执行。修改内容
参考 Megatron-LM 的
no_sync机制,引入真实的is_last_microbatch_控制:Module增加通用的no_sync()接口和 RAIINoSyncGuard。DistributedDataParallel::no_sync()在 guard 生命周期内将 bucket group 的is_last_microbatch_设置为false,退出时恢复为true。overlap_grad_reduce时保持原有行为,由optimizer->step()发起同步。no_sync_func_使用该机制,不直接依赖或包含 DDP 实现。NoSyncGuard。overlap_grad_reduce命令行参数,继续使用 DDP 配置中的默认行为。行为变化
开启梯度累积和
overlap_grad_reduce后,同一个 optimizer step 内的执行过程变为:optimizer->step():等待通信完成并更新参数。这样可以确保 reduce-scatter 读取的是所有 microbatch 累积后的完整梯度,同时避免通信期间继续修改 full gradient buffer。