Skip to content

Latest commit

 

History

History

LLM-Finetuning

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Running LLM Finetuning using IPEX-LLM on Intel GPU

This folder contains examples of running different training mode with IPEX-LLM on Intel GPU:

  • LoRA: examples of running LoRA finetuning
  • QLoRA: examples of running QLoRA finetuning
  • QA-LoRA: examples of running QA-LoRA finetuning
  • ReLora: examples of running ReLora finetuning
  • DPO: examples of running DPO finetuning
  • common: common templates and utility classes in finetuning examples
  • HF-PEFT: run finetuning on Intel GPU using Hugging Face PEFT code without modification
  • axolotl: LLM finetuning on Intel GPU using axolotl without writing code

Verified Models

Model Finetune mode Frameworks Support
LLaMA 2/3 LoRA, QLoRA, QA-LoRA, ReLora HF-PEFT, axolotl
Mistral LoRA, QLoRA DPO
ChatGLM 3 LoRA, QLoRA HF-PEFT
Qwen-1.5 QLoRA HF-PEFT
Baichuan2 QLoRA HF-PEFT

Troubleshooting

  • If you fail to finetune on multi cards because of following error message:

    RuntimeError: oneCCL: comm_selector.cpp:57 create_comm_impl: EXCEPTION: ze_data was not initialized

    Please try sudo apt install level-zero-dev to fix it.

  • Please raise the system open file limit using ulimit -n 1048576. Otherwise, there may exist error Too many open files.

  • If application raise wandb.errors.UsageError: api_key not configured (no-tty). Please login wandb or disable wandb login with this command:

export WANDB_MODE=offline
  • If application raise Hugging Face related errors, i.e., NewConnectionError or Failed to download etc. Please download models and datasets, set model and data path, then set HF_HUB_OFFLINE with this command:
export HF_HUB_OFFLINE=1