Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

NVIDIA GPU Support #11

Closed
wants to merge 6 commits into from
Closed

NVIDIA GPU Support #11

wants to merge 6 commits into from

Commits on Aug 17, 2023

  1. initial commit, working with 3080 10GB, albeit slowly

    @edicristofaro
    edicristofaro committed Aug 17, 2023
    Configuration menu
    Copy the full SHA
    a4926b8 View commit details
    Browse the repository at this point in the history

  2. Some cleanup, set a parameter for the number of GPU layers to offload… 

    … in the Dockerfiles
    @edicristofaro
    edicristofaro committed Aug 17, 2023
    Configuration menu
    Copy the full SHA
    8daac2e View commit details
    Browse the repository at this point in the history

  3. Backing off 70B GPU layers to fit in 10GB VRAM

    @edicristofaro
    edicristofaro committed Aug 17, 2023
    Configuration menu
    Copy the full SHA
    b676aa4 View commit details
    Browse the repository at this point in the history

  4. Offloading all of 7B to GPU

    @edicristofaro
    edicristofaro committed Aug 17, 2023
    Configuration menu
    Copy the full SHA
    e6c52d0 View commit details
    Browse the repository at this point in the history

Commits on Aug 21, 2023

  1. some flattening

    @edicristofaro
    edicristofaro committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    cfdc121 View commit details
    Browse the repository at this point in the history

  2. No more priviliged runtime

    @edicristofaro
    edicristofaro committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    19e95ce View commit details
    Browse the repository at this point in the history