Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Mistral: Sliding Window Attention with Flash Attention and Sample Packing #732

Merged
merged 10 commits into from
Oct 16, 2023
Merged

Mistral: Sliding Window Attention with Flash Attention and Sample Packing #732

merged 10 commits into from
Oct 16, 2023

Commits on Oct 14, 2023

  1. Implement Mistral FA + SWA + Sample Packing

    @casper-hansen
    casper-hansen committed Oct 14, 2023
    Configuration menu
    Copy the full SHA
    9dfe60b View commit details
    Browse the repository at this point in the history

  2. Handle unbroadcastable tensor

    @casper-hansen
    casper-hansen committed Oct 14, 2023
    Configuration menu
    Copy the full SHA
    698764b View commit details
    Browse the repository at this point in the history

Commits on Oct 15, 2023

  1. chore: lint

    @winglian
    winglian committed Oct 15, 2023
    Configuration menu
    Copy the full SHA
    3347413 View commit details
    Browse the repository at this point in the history

Commits on Oct 16, 2023

  1. Simplify _prepare_decoder_attention_mask

    @casper-hansen
    casper-hansen committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    de0c3f8 View commit details
    Browse the repository at this point in the history

  2. Uncomment window size

    @casper-hansen
    casper-hansen committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    dca3f28 View commit details
    Browse the repository at this point in the history

  3. Upgrade flash-attn to minimum of 2.3.0 to support SWA

    @casper-hansen
    casper-hansen committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    b20fd6d View commit details
    Browse the repository at this point in the history

  4. Add original condition to avoid error during inference

    @casper-hansen
    casper-hansen committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    8393b73 View commit details
    Browse the repository at this point in the history

  5. chore: lint

    @winglian
    winglian committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    003ae63 View commit details
    Browse the repository at this point in the history

  6. use torchscript to prevent oom

    @winglian
    winglian committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    2628b22 View commit details
    Browse the repository at this point in the history

  7. chore: pylint

    @winglian
    winglian committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    54b038a View commit details
    Browse the repository at this point in the history