Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding support for Rotary Position Embeddings #675

Merged
merged 119 commits into from
Nov 6, 2023

Commits on Oct 5, 2023

  1. ..

    Shashank Rajput committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    aa9509e View commit details
    Browse the repository at this point in the history
  2. ..

    ShashankMosaicML committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    7354fcc View commit details
    Browse the repository at this point in the history
  3. ..

    ShashankMosaicML committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    6801142 View commit details
    Browse the repository at this point in the history
  4. ..

    ShashankMosaicML committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    eff6270 View commit details
    Browse the repository at this point in the history
  5. ..

    ShashankMosaicML committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    cdc6798 View commit details
    Browse the repository at this point in the history
  6. ..

    ShashankMosaicML committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    a74afb4 View commit details
    Browse the repository at this point in the history
  7. ..

    ShashankMosaicML committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    3c02585 View commit details
    Browse the repository at this point in the history
  8. ..

    ShashankMosaicML committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    47f5af6 View commit details
    Browse the repository at this point in the history
  9. ..

    ShashankMosaicML committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    3389f78 View commit details
    Browse the repository at this point in the history
  10. ..

    ShashankMosaicML committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    c9f2154 View commit details
    Browse the repository at this point in the history
  11. ..

    ShashankMosaicML committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    9db76a8 View commit details
    Browse the repository at this point in the history
  12. ..

    ShashankMosaicML committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    722eb0c View commit details
    Browse the repository at this point in the history

Commits on Oct 6, 2023

  1. ..

    ShashankMosaicML committed Oct 6, 2023
    Configuration menu
    Copy the full SHA
    4eb9f17 View commit details
    Browse the repository at this point in the history
  2. ..

    ShashankMosaicML committed Oct 6, 2023
    Configuration menu
    Copy the full SHA
    c675c22 View commit details
    Browse the repository at this point in the history
  3. ..

    ShashankMosaicML committed Oct 6, 2023
    Configuration menu
    Copy the full SHA
    de765c4 View commit details
    Browse the repository at this point in the history
  4. ..

    ShashankMosaicML committed Oct 6, 2023
    Configuration menu
    Copy the full SHA
    529ada8 View commit details
    Browse the repository at this point in the history
  5. ..

    ShashankMosaicML committed Oct 6, 2023
    Configuration menu
    Copy the full SHA
    7d39ffc View commit details
    Browse the repository at this point in the history
  6. ..

    ShashankMosaicML committed Oct 6, 2023
    Configuration menu
    Copy the full SHA
    bb92769 View commit details
    Browse the repository at this point in the history
  7. ..

    ShashankMosaicML committed Oct 6, 2023
    Configuration menu
    Copy the full SHA
    841becb View commit details
    Browse the repository at this point in the history
  8. ..

    ShashankMosaicML committed Oct 6, 2023
    Configuration menu
    Copy the full SHA
    e5d0e65 View commit details
    Browse the repository at this point in the history

Commits on Oct 7, 2023

  1. ..

    ShashankMosaicML committed Oct 7, 2023
    Configuration menu
    Copy the full SHA
    dabd231 View commit details
    Browse the repository at this point in the history
  2. ..

    ShashankMosaicML committed Oct 7, 2023
    Configuration menu
    Copy the full SHA
    7f1109a View commit details
    Browse the repository at this point in the history

Commits on Oct 8, 2023

  1. Configuration menu
    Copy the full SHA
    e98841d View commit details
    Browse the repository at this point in the history
  2. ..

    ShashankMosaicML committed Oct 8, 2023
    Configuration menu
    Copy the full SHA
    dea3b03 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    2927a8c View commit details
    Browse the repository at this point in the history
  4. ..

    ShashankMosaicML committed Oct 8, 2023
    Configuration menu
    Copy the full SHA
    d605fbf View commit details
    Browse the repository at this point in the history

Commits on Oct 9, 2023

  1. ..

    ShashankMosaicML committed Oct 9, 2023
    Configuration menu
    Copy the full SHA
    7b250f7 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    196b8e1 View commit details
    Browse the repository at this point in the history
  3. Merge pull request #2 from mosaicml/main

    Pulling the changes from main branch
    ShashankMosaicML authored Oct 9, 2023
    Configuration menu
    Copy the full SHA
    22212a1 View commit details
    Browse the repository at this point in the history

Commits on Oct 10, 2023

  1. ..

    ShashankMosaicML committed Oct 10, 2023
    Configuration menu
    Copy the full SHA
    0c3942e View commit details
    Browse the repository at this point in the history

Commits on Oct 12, 2023

  1. ..

    ShashankMosaicML committed Oct 12, 2023
    Configuration menu
    Copy the full SHA
    829b2a4 View commit details
    Browse the repository at this point in the history
  2. ..

    ShashankMosaicML committed Oct 12, 2023
    Configuration menu
    Copy the full SHA
    1629d1a View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    eb658a3 View commit details
    Browse the repository at this point in the history
  4. ..

    ShashankMosaicML committed Oct 12, 2023
    Configuration menu
    Copy the full SHA
    5cb95f6 View commit details
    Browse the repository at this point in the history
  5. ..

    ShashankMosaicML committed Oct 12, 2023
    Configuration menu
    Copy the full SHA
    30aa448 View commit details
    Browse the repository at this point in the history

Commits on Oct 13, 2023

  1. ..

    ShashankMosaicML committed Oct 13, 2023
    Configuration menu
    Copy the full SHA
    52119f5 View commit details
    Browse the repository at this point in the history
  2. ..

    ShashankMosaicML committed Oct 13, 2023
    Configuration menu
    Copy the full SHA
    048b886 View commit details
    Browse the repository at this point in the history
  3. ..

    ShashankMosaicML committed Oct 13, 2023
    Configuration menu
    Copy the full SHA
    c2ee0de View commit details
    Browse the repository at this point in the history
  4. ..

    ShashankMosaicML committed Oct 13, 2023
    Configuration menu
    Copy the full SHA
    9e8a1d6 View commit details
    Browse the repository at this point in the history
  5. ..

    ShashankMosaicML committed Oct 13, 2023
    Configuration menu
    Copy the full SHA
    9c2a2a6 View commit details
    Browse the repository at this point in the history
  6. ..

    ShashankMosaicML committed Oct 13, 2023
    Configuration menu
    Copy the full SHA
    6fa3037 View commit details
    Browse the repository at this point in the history
  7. ..

    ShashankMosaicML committed Oct 13, 2023
    Configuration menu
    Copy the full SHA
    5323520 View commit details
    Browse the repository at this point in the history
  8. Update llmfoundry/models/mpt/modeling_mpt.py

    Accepting the suggestion
    
    Co-authored-by: Vitaliy Chiley <6439018+vchiley@users.noreply.github.com>
    ShashankMosaicML and vchiley authored Oct 13, 2023
    Configuration menu
    Copy the full SHA
    b0960f7 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    f6632e1 View commit details
    Browse the repository at this point in the history

Commits on Oct 14, 2023

  1. ..

    ShashankMosaicML committed Oct 14, 2023
    Configuration menu
    Copy the full SHA
    df749ae View commit details
    Browse the repository at this point in the history

Commits on Oct 17, 2023

  1. ..

    ShashankMosaicML committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    8b886ba View commit details
    Browse the repository at this point in the history
  2. Merge pull request #3 from mosaicml/main

    Pulling from the main repository
    ShashankMosaicML authored Oct 17, 2023
    Configuration menu
    Copy the full SHA
    76a2095 View commit details
    Browse the repository at this point in the history
  3. ..

    ShashankMosaicML committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    dc58fc7 View commit details
    Browse the repository at this point in the history
  4. ..

    ShashankMosaicML committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    ed5a477 View commit details
    Browse the repository at this point in the history

Commits on Oct 19, 2023

  1. ..

    ShashankMosaicML committed Oct 19, 2023
    Configuration menu
    Copy the full SHA
    2a53de3 View commit details
    Browse the repository at this point in the history
  2. ..

    ShashankMosaicML committed Oct 19, 2023
    Configuration menu
    Copy the full SHA
    34e147c View commit details
    Browse the repository at this point in the history
  3. ..

    ShashankMosaicML committed Oct 19, 2023
    Configuration menu
    Copy the full SHA
    0a9d3af View commit details
    Browse the repository at this point in the history

Commits on Oct 20, 2023

  1. Configuration menu
    Copy the full SHA
    5981ade View commit details
    Browse the repository at this point in the history
  2. Merge pull request #4 from mosaicml/main

    merging from main
    ShashankMosaicML authored Oct 20, 2023
    Configuration menu
    Copy the full SHA
    b179b07 View commit details
    Browse the repository at this point in the history
  3. ..

    ShashankMosaicML committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    9afa082 View commit details
    Browse the repository at this point in the history
  4. ..

    ShashankMosaicML committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    9835acd View commit details
    Browse the repository at this point in the history

Commits on Oct 21, 2023

  1. ..

    ShashankMosaicML committed Oct 21, 2023
    Configuration menu
    Copy the full SHA
    1801677 View commit details
    Browse the repository at this point in the history

Commits on Oct 23, 2023

  1. Configuration menu
    Copy the full SHA
    0a38037 View commit details
    Browse the repository at this point in the history
  2. ..

    ShashankMosaicML committed Oct 23, 2023
    Configuration menu
    Copy the full SHA
    d86a1a5 View commit details
    Browse the repository at this point in the history
  3. ..

    ShashankMosaicML committed Oct 23, 2023
    Configuration menu
    Copy the full SHA
    7e336d2 View commit details
    Browse the repository at this point in the history

Commits on Oct 24, 2023

  1. added tests

    ShashankMosaicML committed Oct 24, 2023
    Configuration menu
    Copy the full SHA
    1897353 View commit details
    Browse the repository at this point in the history
  2. ..

    ShashankMosaicML committed Oct 24, 2023
    Configuration menu
    Copy the full SHA
    213cd14 View commit details
    Browse the repository at this point in the history
  3. ..

    ShashankMosaicML committed Oct 24, 2023
    Configuration menu
    Copy the full SHA
    68d03d3 View commit details
    Browse the repository at this point in the history
  4. ...

    ShashankMosaicML committed Oct 24, 2023
    Configuration menu
    Copy the full SHA
    c0da75c View commit details
    Browse the repository at this point in the history

Commits on Oct 25, 2023

  1. ..

    ShashankMosaicML committed Oct 25, 2023
    Configuration menu
    Copy the full SHA
    4600415 View commit details
    Browse the repository at this point in the history
  2. ..

    ShashankMosaicML committed Oct 25, 2023
    Configuration menu
    Copy the full SHA
    5ecda44 View commit details
    Browse the repository at this point in the history

Commits on Oct 26, 2023

  1. ..

    ShashankMosaicML committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    6441180 View commit details
    Browse the repository at this point in the history
  2. ..

    ShashankMosaicML committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    d71a2a0 View commit details
    Browse the repository at this point in the history
  3. Merge pull request #5 from ShashankMosaicML/rotary_dail_imp

    Adding the Dao AI Lab's rope implementation
    ShashankMosaicML authored Oct 26, 2023
    Configuration menu
    Copy the full SHA
    f952e5b View commit details
    Browse the repository at this point in the history
  4. ..

    ShashankMosaicML committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    f33ed5f View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    07eafb7 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    7efb6b1 View commit details
    Browse the repository at this point in the history
  7. minor change

    ShashankMosaicML committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    3a056a8 View commit details
    Browse the repository at this point in the history

Commits on Oct 27, 2023

  1. Configuration menu
    Copy the full SHA
    327dded View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    9c00106 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    999209c View commit details
    Browse the repository at this point in the history
  4. fixed tests

    ShashankMosaicML committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    a681b64 View commit details
    Browse the repository at this point in the history
  5. ..

    ShashankMosaicML committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    766fa75 View commit details
    Browse the repository at this point in the history
  6. Merge pull request #7 from mosaicml/main

    Pulling latest changes from mail
    ShashankMosaicML authored Oct 27, 2023
    Configuration menu
    Copy the full SHA
    869be97 View commit details
    Browse the repository at this point in the history
  7. ..

    ShashankMosaicML committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    21a4f31 View commit details
    Browse the repository at this point in the history
  8. temporary fix

    ShashankMosaicML committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    dbac1e0 View commit details
    Browse the repository at this point in the history
  9. ..

    ShashankMosaicML committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    5d62dfe View commit details
    Browse the repository at this point in the history
  10. ..

    ShashankMosaicML committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    ca57151 View commit details
    Browse the repository at this point in the history
  11. fixed a test

    ShashankMosaicML committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    99a81a1 View commit details
    Browse the repository at this point in the history
  12. ..

    ShashankMosaicML committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    b674e83 View commit details
    Browse the repository at this point in the history
  13. minor change

    ShashankMosaicML committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    8be09ab View commit details
    Browse the repository at this point in the history
  14. minor changes

    ShashankMosaicML committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    067439e View commit details
    Browse the repository at this point in the history

Commits on Oct 30, 2023

  1. added documentation

    ShashankMosaicML committed Oct 30, 2023
    Configuration menu
    Copy the full SHA
    b325097 View commit details
    Browse the repository at this point in the history
  2. added documentation

    ShashankMosaicML committed Oct 30, 2023
    Configuration menu
    Copy the full SHA
    1b35c0b View commit details
    Browse the repository at this point in the history

Commits on Oct 31, 2023

  1. temp commit

    ShashankMosaicML committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    3988b57 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    82ce2d9 View commit details
    Browse the repository at this point in the history
  3. minor changes

    ShashankMosaicML committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    d2930f9 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    b022b12 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    dbc0f84 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    54d8304 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    c05d34d View commit details
    Browse the repository at this point in the history
  8. minor changes

    ShashankMosaicML committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    c8f099d View commit details
    Browse the repository at this point in the history
  9. minor changes

    ShashankMosaicML committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    a48afa2 View commit details
    Browse the repository at this point in the history
  10. minor changes

    ShashankMosaicML committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    fa46318 View commit details
    Browse the repository at this point in the history
  11. minor changes

    ShashankMosaicML committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    5b10164 View commit details
    Browse the repository at this point in the history
  12. minor changes

    ShashankMosaicML committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    64d4a57 View commit details
    Browse the repository at this point in the history
  13. minor changes

    ShashankMosaicML committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    04452e6 View commit details
    Browse the repository at this point in the history
  14. ..

    ShashankMosaicML committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    1eff648 View commit details
    Browse the repository at this point in the history
  15. Configuration menu
    Copy the full SHA
    046fa08 View commit details
    Browse the repository at this point in the history

Commits on Nov 1, 2023

  1. Configuration menu
    Copy the full SHA
    cceca07 View commit details
    Browse the repository at this point in the history

Commits on Nov 2, 2023

  1. Configuration menu
    Copy the full SHA
    1e59de5 View commit details
    Browse the repository at this point in the history
  2. fixed tests

    ShashankMosaicML committed Nov 2, 2023
    Configuration menu
    Copy the full SHA
    ac0fd40 View commit details
    Browse the repository at this point in the history

Commits on Nov 3, 2023

  1. Configuration menu
    Copy the full SHA
    e59f784 View commit details
    Browse the repository at this point in the history
  2. minor changes

    ShashankMosaicML committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    c602a06 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    e59ec7e View commit details
    Browse the repository at this point in the history

Commits on Nov 4, 2023

  1. Update TUTORIAL.md

    Co-authored-by: Daniel King <43149077+dakinggg@users.noreply.github.com>
    ShashankMosaicML and dakinggg authored Nov 4, 2023
    Configuration menu
    Copy the full SHA
    5744a3e View commit details
    Browse the repository at this point in the history
  2. Update TUTORIAL.md

    Co-authored-by: Daniel King <43149077+dakinggg@users.noreply.github.com>
    ShashankMosaicML and dakinggg authored Nov 4, 2023
    Configuration menu
    Copy the full SHA
    0036ce7 View commit details
    Browse the repository at this point in the history
  3. Update TUTORIAL.md

    Co-authored-by: Daniel King <43149077+dakinggg@users.noreply.github.com>
    ShashankMosaicML and dakinggg authored Nov 4, 2023
    Configuration menu
    Copy the full SHA
    9ebd2e7 View commit details
    Browse the repository at this point in the history
  4. Update TUTORIAL.md

    Co-authored-by: Daniel King <43149077+dakinggg@users.noreply.github.com>
    ShashankMosaicML and dakinggg authored Nov 4, 2023
    Configuration menu
    Copy the full SHA
    4874713 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    e0d8b75 View commit details
    Browse the repository at this point in the history

Commits on Nov 6, 2023

  1. Configuration menu
    Copy the full SHA
    5ba6968 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    6b680d7 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    9204e31 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    828d2bb View commit details
    Browse the repository at this point in the history