New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix model parallel support for llama #271

Merged

diegofiori merged 2 commits into nebuly-ai:main from zhzou2020:fix_model_parallel

Mar 17, 2023

zhzou2020 commented Mar 17, 2023

#263 fix the issue of scores and kv_mask mismatch when using LLaMa 13B model.


          fix model parallel support for llama

6a3b871

zhzou2020 mentioned this pull request

[Chatllama] Supervised Finetune on LLaMA-13B #263

Closed


          init use_fairscale

d8acc23

Collaborator

diegofiori commented Mar 17, 2023

Hi @zhzou2020, thanks for solving the issue with fairscale multi-gpu setup! The PR looks good to me!

diegofiori merged commit 52a4e1b into nebuly-ai:main

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet