Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feature: better device mapping for large models #918

Merged
merged 8 commits into from
Jan 5, 2024
Merged

feature: better device mapping for large models #918

merged 8 commits into from
Jan 5, 2024

Commits on Dec 29, 2023

  1. fix: improved memory handling when model is bigger than existing VRAM

    @kallewoof
    Karl-Johan Alm authored and kallewoof committed Dec 29, 2023
    Configuration menu
    Copy the full SHA
    27ba854 View commit details
    Browse the repository at this point in the history

  2. feature: add lora_on_cpu flag to do LoRA loading on CPU (RAM) 

    For big models where the models are taking up the entire GPU VRAM, the LoRA part will fail unless it is loaded on CPU only.
    @kallewoof
    Karl-Johan Alm authored and kallewoof committed Dec 29, 2023
    Configuration menu
    Copy the full SHA
    0957405 View commit details
    Browse the repository at this point in the history

  3. doc: add README

    @kallewoof
    Karl-Johan Alm authored and kallewoof committed Dec 29, 2023
    Configuration menu
    Copy the full SHA
    c1e23c7 View commit details
    Browse the repository at this point in the history

  4. fix: enable progress bars in do_merge_lora()

    @kallewoof
    Karl-Johan Alm authored and kallewoof committed Dec 29, 2023
    Configuration menu
    Copy the full SHA
    f60f9c2 View commit details
    Browse the repository at this point in the history

  5. doc: mention gpu_memory_limit and lora_on_cpu in merge part of README

    @kallewoof
    Karl-Johan Alm authored and kallewoof committed Dec 29, 2023
    Configuration menu
    Copy the full SHA
    76139bd View commit details
    Browse the repository at this point in the history

  6. Update src/axolotl/utils/models.py 

    Co-authored-by: Wing Lian <wing.lian@gmail.com>
    @kallewoof @winglian
    kallewoof and winglian committed Dec 29, 2023
    Configuration menu
    Copy the full SHA
    22247c6 View commit details
    Browse the repository at this point in the history

  7. fix: remove deletion of removed model_kwargs key

    @kallewoof
    kallewoof committed Dec 29, 2023
    Configuration menu
    Copy the full SHA
    088485c View commit details
    Browse the repository at this point in the history

  8. fix: validate that gpu_memory_limit and max_memory are not both set

    @kallewoof
    kallewoof committed Dec 29, 2023
    Configuration menu
    Copy the full SHA
    05ed3d1 View commit details
    Browse the repository at this point in the history