Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DPO cleanup #1126

Merged
merged 23 commits into from
Jan 23, 2024
Merged

DPO cleanup #1126

merged 23 commits into from
Jan 23, 2024

Commits on Jan 23, 2024

  1. cleanup dpo to be a little more extensible, add zephyr/nectar strategy

    @winglian
    winglian committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    6a23720 View commit details
    Browse the repository at this point in the history

  2. fix eos slash

    @winglian
    winglian committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    2f7242d View commit details
    Browse the repository at this point in the history

  3. support for eval split

    @winglian
    winglian committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    74cbe2a View commit details
    Browse the repository at this point in the history

  4. fix kwargs

    @winglian
    winglian committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    1f28327 View commit details
    Browse the repository at this point in the history

  5. handle empty evals

    @winglian
    winglian committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    cb2f774 View commit details
    Browse the repository at this point in the history

  6. don't load peft model for dpo

    @winglian
    winglian committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    a91e0cb View commit details
    Browse the repository at this point in the history

  7. ensure dpo traning args gets bf16 for peft if applicable

    @winglian
    winglian committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    bcae290 View commit details
    Browse the repository at this point in the history

  8. fix duplicate kwargs for bf16

    @winglian
    winglian committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    a24c756 View commit details
    Browse the repository at this point in the history

  9. make sure to respect the configured lr scheduler

    @winglian
    winglian committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    d5e12dd View commit details
    Browse the repository at this point in the history

  10. supprt trainer callback to push config to wandb

    @winglian
    winglian committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    9a746fa View commit details
    Browse the repository at this point in the history

  11. set dataloader preload args

    @winglian
    winglian committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    60f566c View commit details
    Browse the repository at this point in the history

  12. ensure that we are loading the lora when merging

    @winglian
    winglian committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    c41391d View commit details
    Browse the repository at this point in the history

  13. Update src/axolotl/utils/data.py 

    Co-authored-by: Agus <agustin.piqueres@gmail.com>
    @winglian @plaguss
    winglian and plaguss committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    1cfd179 View commit details
    Browse the repository at this point in the history

  14. support local datasets for dpo 

    Co-authored-by: Agus <agustin.piqueres@gmail.com>
    @winglian @plaguss
    winglian and plaguss committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    02fc8f9 View commit details
    Browse the repository at this point in the history

  15. chore: lint

    @winglian
    winglian committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    7141fd1 View commit details
    Browse the repository at this point in the history

  16. dpo/kto/ipo smoke tests w lora, simplify dpo dataset type names

    @winglian
    winglian committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    44a6f2d View commit details
    Browse the repository at this point in the history

  17. add split to dpo tests

    @winglian
    winglian committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    7f3b7ce View commit details
    Browse the repository at this point in the history

  18. fix rebase/merging error

    @winglian
    winglian committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    52a227d View commit details
    Browse the repository at this point in the history

  19. handle edge case w logging

    @winglian
    winglian committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    064b20e View commit details
    Browse the repository at this point in the history

  20. use accelerator for dpo datasets so it doesn't break the logger

    @winglian
    winglian committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    c49315f View commit details
    Browse the repository at this point in the history

  21. missing args

    @winglian
    winglian committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    72fb877 View commit details
    Browse the repository at this point in the history

  22. validate checkpoint is an adapter for now

    @winglian
    winglian committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    29663d8 View commit details
    Browse the repository at this point in the history

  23. log warning when dataset strategy is not loadable

    @winglian
    winglian committed Jan 23, 2024
    Configuration menu
    Copy the full SHA
    76b5c2d View commit details
    Browse the repository at this point in the history