Skip to content

Support use safetensors with paddle.MmapStorage to load model files - #2730

Closed
zeroRains wants to merge 2 commits into
PaddlePaddle:developfrom
zeroRains:loading
Closed

Support use safetensors with paddle.MmapStorage to load model files#2730
zeroRains wants to merge 2 commits into
PaddlePaddle:developfrom
zeroRains:loading

Conversation

@zeroRains

@zeroRains zeroRains commented Jul 7, 2025

Copy link
Copy Markdown
Contributor

pcard-71500

使用带有paddle.MmapStorage的safetensors,加速模型文件的加载。

单卡测试:ernie-4_5-21b-a3b-bf16-paddle

方法 文件加载时间 文件加载时间+模型参数导入模型时间 结论
baseline 28 44.64527976885438
MmapSotrage 4 22.69623290002346 加载时间缩短为原来的一半

8卡测试:ernie-4_5-300b-a47b-bf16-paddle

方法 文件加载时间 文件加载时间+模型参数导入模型时间 结论
baseline 175 222.6731412112713
MmapSotrage (get_slice) 168 226.66718888282776 基本不变
MmapSotrage (get_tensor) 86 131.25562098249793 加载时间缩短为原来的一半

前置PR:

提前使用方式:

  1. 编译最新develop的paddle
  2. 编译这个仓库git clone -b pp https://github.com/zeroRains/safetensors.git
  3. 拉取本pr的commit

@paddle-bot

paddle-bot Bot commented Jul 7, 2025

Copy link
Copy Markdown

Thanks for your contribution!

@paddle-bot paddle-bot Bot added the contributor External developers label Jul 7, 2025
@CLAassistant

CLAassistant commented Jul 7, 2025

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@zeroRains
zeroRains force-pushed the loading branch 3 times, most recently from 371ac7b to 52b4ac6 Compare July 7, 2025 12:31
Change-Id: I8f6faff3d86b682ccdccc31b38eb1d6b1db5e8a1
@paddle-bot paddle-bot Bot closed this Aug 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor External developers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants