Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement fused modules #747

Merged
merged 23 commits into from
Oct 21, 2023
Merged

Implement fused modules #747

merged 23 commits into from
Oct 21, 2023

Commits on Oct 19, 2023

  1. MLP: Memory saving

    @casper-hansen
    casper-hansen committed Oct 19, 2023
    Configuration menu
    Copy the full SHA
    cd1853e View commit details
    Browse the repository at this point in the history

  2. Remove RMSNorm restrictions

    @casper-hansen
    casper-hansen committed Oct 19, 2023
    Configuration menu
    Copy the full SHA
    fa635c6 View commit details
    Browse the repository at this point in the history

Commits on Oct 20, 2023

  1. Map packed weights to original

    @casper-hansen
    casper-hansen committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    ea409f9 View commit details
    Browse the repository at this point in the history

  2. FusedAttention module

    @casper-hansen
    casper-hansen committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    2ea0472 View commit details
    Browse the repository at this point in the history

  3. Simplify code

    @casper-hansen
    casper-hansen committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    ed26619 View commit details
    Browse the repository at this point in the history

  4. Move fused modules

    @casper-hansen
    casper-hansen committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    4b8ca18 View commit details
    Browse the repository at this point in the history

  5. Fix critical typo

    @casper-hansen
    casper-hansen committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    f4fc8e1 View commit details
    Browse the repository at this point in the history

  6. Split inplace

    @casper-hansen
    casper-hansen committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    5fd21f6 View commit details
    Browse the repository at this point in the history

  7. Add FFT config

    @casper-hansen
    casper-hansen committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    2ca2e1e View commit details
    Browse the repository at this point in the history

  8. Add validation of fused arguments

    @casper-hansen
    casper-hansen committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    6a5ac7f View commit details
    Browse the repository at this point in the history

  9. Add fused arguments to config

    @casper-hansen
    casper-hansen committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    856cec4 View commit details
    Browse the repository at this point in the history

  10. Update docs

    @casper-hansen
    casper-hansen committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    aa7176b View commit details
    Browse the repository at this point in the history

  11. Fix validation logic

    @casper-hansen
    casper-hansen committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    5fc8655 View commit details
    Browse the repository at this point in the history

  12. Add fused modules to flash attn

    @casper-hansen
    casper-hansen committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    c91dc99 View commit details
    Browse the repository at this point in the history

  13. Only fuse during training

    @casper-hansen
    casper-hansen committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    5550871 View commit details
    Browse the repository at this point in the history

  14. Remove timing

    @casper-hansen
    casper-hansen committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    7a6bd2b View commit details
    Browse the repository at this point in the history

  15. Formatting

    @casper-hansen
    casper-hansen committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    fac3ed1 View commit details
    Browse the repository at this point in the history

  16. Formatting

    @casper-hansen
    casper-hansen committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    0272261 View commit details
    Browse the repository at this point in the history

  17. Formatting

    @casper-hansen
    casper-hansen committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    779187c View commit details
    Browse the repository at this point in the history

  18. chore: lint

    @winglian
    winglian committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    7c9c535 View commit details
    Browse the repository at this point in the history

  19. chore: lint

    @winglian
    winglian committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    8e9a2f4 View commit details
    Browse the repository at this point in the history

  20. add e2e tests for fused llama

    @winglian
    winglian committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    fb07f6f View commit details
    Browse the repository at this point in the history

  21. no lora for tests

    @winglian
    winglian committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    3e34c4c View commit details
    Browse the repository at this point in the history