Skip to content

V3 gpu flash attention triton solution fix - #29

Merged
Exorust merged 2 commits into
Exorust:mainfrom
jeniag:v3-gpu-flash-attention-triton-soln-fix
Sep 8, 2026
Merged

V3 gpu flash attention triton solution fix#29
Exorust merged 2 commits into
Exorust:mainfrom
jeniag:v3-gpu-flash-attention-triton-soln-fix

Conversation

@jeniag

@jeniag jeniag commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Existing solution in v3/gpu-systems/flash-attention-triton/flash-attention-triton_SOLN.ipynb only has a flash_attention_kernel not a full wrapper. This adds the wrapper.

To test:
from torchleet import check; check("flash-attention-triton", flash_attention_pytorch,flash_attention_triton=flash_attention_triton)

Passes with

torchleet: flash-attention-triton
   ✅ matches standard attention
   ✅ block size does not change the answer
   ✅ sequence not divisible by block
   ✅ numerically stable on large scores
   ✅ attention weights are a distribution
   ✅ gradients flow
   ✅ triton kernel
   all checks passed

True

@jeniag
jeniag requested a review from Exorust as a code owner September 7, 2026 19:05
@vercel

vercel Bot commented Sep 7, 2026

Copy link
Copy Markdown

@jeniag is attempting to deploy a commit to the exorust's projects Team on Vercel.

A member of the Team first needs to authorize it.

@review-notebook-app

Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@Exorust
Exorust merged commit dac1e91 into Exorust:main Sep 8, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants