Torch2 (#177) #178
Merged
Torch2 (#177) #178
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* make triton attn req mlri tagged triton * add comment * updt err * clean up req / install * updt * updt * exclude HazyR flash attn from pyright * lint * exclude flash_attn_triton.py from pyright * updt torch version * updt install instructions * updt * add extra install instructions for installing CMake * lint * updt * updt torch * updt * adding torch1.13 and torch2 testing matrix
sashaDoubov
reviewed
May 19, 2023
LGTM
|
mvpatel2000
approved these changes
May 19, 2023
Closed
dakinggg
added a commit
to dakinggg/llm-foundry
that referenced
this pull request
May 20, 2023
This reverts commit bb7f8bb.
vchiley
added a commit
to vchiley/llm-foundry
that referenced
this pull request
May 22, 2023
This reverts commit 89f56d2.
vchiley
added a commit
that referenced
this pull request
May 24, 2023
* fix and test * Revert "Revert "Torch2 (#177) (#178)" (#181)" This reverts commit 89f56d2. * updt import try except * updt hf model * updt imports * lint * add mpt hf model init / gen test * updt for temp testing * lint * rerun tests * Update .github/workflows/release.yaml Co-authored-by: Daniel King <43149077+dakinggg@users.noreply.github.com> * Update tests/test_hf_mpt_gen.py Co-authored-by: Daniel King <43149077+dakinggg@users.noreply.github.com> * add cpu test * updt tests / cpu img * updt cpu test install * rerun tests * fix hf import structure * fix test * pull_request -> pull_request_target * make onnx test smaller --------- Co-authored-by: Daniel King <daniel@mosaicml.com> Co-authored-by: Daniel King <43149077+dakinggg@users.noreply.github.com>
bmosaicml
pushed a commit
that referenced
this pull request
Jun 6, 2023
* Torch2 (#177) * make triton attn req mlri tagged triton * add comment * updt err * clean up req / install * updt * updt * exclude HazyR flash attn from pyright * lint * exclude flash_attn_triton.py from pyright * updt torch version * updt install instructions * updt * add extra install instructions for installing CMake * lint * updt * updt torch * updt * adding torch1.13 and torch2 testing matrix * Update pr-gpu.yaml * Update test_model.py * Update pr-cpu.yaml * Update pr-gpu.yaml * Update test_dataloader.py * Update pr-gpu.yaml
bmosaicml
pushed a commit
that referenced
this pull request
Jun 8, 2023
* fix and test * Revert "Revert "Torch2 (#177) (#178)" (#181)" This reverts commit 601d61a. * updt import try except * updt hf model * updt imports * lint * add mpt hf model init / gen test * updt for temp testing * lint * rerun tests * Update .github/workflows/release.yaml Co-authored-by: Daniel King <43149077+dakinggg@users.noreply.github.com> * Update tests/test_hf_mpt_gen.py Co-authored-by: Daniel King <43149077+dakinggg@users.noreply.github.com> * add cpu test * updt tests / cpu img * updt cpu test install * rerun tests * fix hf import structure * fix test * pull_request -> pull_request_target * make onnx test smaller --------- Co-authored-by: Daniel King <daniel@mosaicml.com> Co-authored-by: Daniel King <43149077+dakinggg@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
move #149 to main repo (from a fork)
uses #147 as a springboard to updt torch
In interactive instance, I installed torch2 req and everything works fine
125M models was getting good (the same) MFU from the same exact config in both torch1.13 and torch2
Note: torch2 version
pip listhas both triton version:doesn't seem to matter
Note: this does not use
torch.compile()(but there is no reason it shouldn't)Note:
flash-attnis still installed.xentropy-cuda-libis also still installed; I'm not settingloss_fnso mpt defaults to usingfused_crossentropyfor both settings.Biggest low probability risk: this old version of triton does not compile / work for H100s... 👀
Risk:
triton_pre_mlirhas no support and will never be updated.Still need to test at scale / convergencesee torch2 vs torch1.13 produce the same results here
cc @sashaDoubov (enables torch2 for muP dev)
cc @dskhudia enables torch2 and torch.compile() with triton attn impl
old pr commits:
make triton attn req mlri tagged triton
add comment
updt err
clean up req / install
exclude HazyR flash attn from pyright
lint
exclude flash_attn_triton.py from pyright
updt torch version & install instructions
add extra install instructions for installing CMake
lint
adding torch1.13 and torch2 testing matrix