Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Phi2 multipack #1173

Merged
merged 11 commits into from
Jan 23, 2024
Merged

Phi2 multipack #1173

merged 11 commits into from
Jan 23, 2024

Commits on Jan 23, 2024

  1. phi2 multipack

    @winglian
    winglian committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    494dd38 View commit details
    Browse the repository at this point in the history

  2. update validation and examples for phi

    @winglian
    winglian committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    f4fc1b2 View commit details
    Browse the repository at this point in the history

  3. more updates to phi examples

    @winglian
    winglian committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    93b8f8e View commit details
    Browse the repository at this point in the history

  4. make sure to use the correct collator for phi multipack

    @winglian
    winglian committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    ea9ca67 View commit details
    Browse the repository at this point in the history

  5. phi needs attention mask now for multipack

    @winglian
    winglian committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    c9fa803 View commit details
    Browse the repository at this point in the history

  6. if the special token already exists in the tokenizer, don't require i… 

    …n lora modules to save
    @winglian
    winglian committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    06f1c55 View commit details
    Browse the repository at this point in the history

  7. fix qlora yml for phi, fix phi test validation

    @winglian
    winglian committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    ee608fe View commit details
    Browse the repository at this point in the history

  8. test qlora too

    @winglian
    winglian committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    929933f View commit details
    Browse the repository at this point in the history

  9. make sure flash attention is enabled for the test

    @winglian
    winglian committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    97e06f3 View commit details
    Browse the repository at this point in the history

  10. don't use remote code for phi anymore

    @winglian
    winglian committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    4350b95 View commit details
    Browse the repository at this point in the history

  11. reduce sequence len for sample packing phi

    @winglian
    winglian committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    faad02b View commit details
    Browse the repository at this point in the history