Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add basic support for the optimi adamw optimizer #1727

Merged
merged 7 commits into from
Jul 14, 2024
Merged

add basic support for the optimi adamw optimizer #1727

merged 7 commits into from
Jul 14, 2024

Commits on Jul 13, 2024

  1. add support for optimi_adamw optimizer w kahan summation

    @winglian
    winglian committed Jul 13, 2024
    Configuration menu
    Copy the full SHA
    3545486 View commit details
    Browse the repository at this point in the history

  2. pydantic validator for optimi_adamw

    @winglian
    winglian committed Jul 13, 2024
    Configuration menu
    Copy the full SHA
    903eff2 View commit details
    Browse the repository at this point in the history

  3. workaround for setting optimizer for fsdp

    @winglian
    winglian committed Jul 13, 2024
    Configuration menu
    Copy the full SHA
    251f15c View commit details
    Browse the repository at this point in the history

  4. make sure to install optimizer packages

    @winglian
    winglian committed Jul 13, 2024
    Configuration menu
    Copy the full SHA
    b137d00 View commit details
    Browse the repository at this point in the history

  5. make sure to have parity for model parameters passed to optimizer

    @winglian
    winglian committed Jul 13, 2024
    Configuration menu
    Copy the full SHA
    a3cc744 View commit details
    Browse the repository at this point in the history

  6. add smoke test for optimi_adamw optimizer

    @winglian
    winglian committed Jul 13, 2024
    Configuration menu
    Copy the full SHA
    7d89b05 View commit details
    Browse the repository at this point in the history

Commits on Jul 14, 2024

  1. don't use foreach optimi by default

    @winglian
    winglian committed Jul 14, 2024
    Configuration menu
    Copy the full SHA
    449e25f View commit details
    Browse the repository at this point in the history