-
Notifications
You must be signed in to change notification settings - Fork 44
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
IFU to v2.0.4 #14
IFU to v2.0.4 #14
Conversation
Add gpt-neox adoption
Follow xFormers's DISTPATCH_BOOL. Haven't tested it on Windows.
fixed cross attention typeerror
A new environment variable "FLASH_ATTENTION_INTERNAL_ENABLE_TIME_KERNEL" can switch the output of kernel running time |
[BUGs] Previously in older version of FA, we create tensors z and softmax_lse matrix of max sequence lengths with no padding for grouped gemm. But the strides for each batch for the tensors are different. This behaviour will cause wrong result from CK. Fixing it. |
…Platform/flash-attention into junhzhan/ifu-v2.0.0
Add MQA & GQA
Please remove *_hip.hpp |
d475794
to
02c234b
Compare
.gitignore
Outdated
@@ -24,7 +24,10 @@ var/ | |||
.vscode/settings. | |||
|
|||
# Generated files | |||
csrc/flash_attn_rocm/src/*hip* |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
better to use *_hip.*
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
makes sense
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, file names such as hip_flash_attention.cpp
or hip_hacks.hpp
are ignored too?
…Platform/flash-attention into junhzhan/ifu-v2.0.0
…Platform/flash-attention into junhzhan/ifu-v2.0.0
Current Unit Test Result: (PyTorch 2.0.0; ROCm 5.6)
3968 passed, 63 skipped
Current Performance on MI250: (docker pull rocm/pytorch:rocm5.7_ubuntu22.04_py3.10_pytorch_2.0.1)