You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Please check that this issue hasn't been reported before.
I searched previous Bug Reports didn't find any similar reports.
Expected Behavior
I should be able to use load_in_8bit: False + load_in_4bit: False + adapter: lora.
This issue was already observed here: #456 but closed without resolution.
Current behaviour
When you try this today, the process gets stuck, e.g.:
This isn't a supported configuration. We only officially support LoRA with 8bit and qLoRA with 4bit. if you wish to submit a fix to enable 16 bit lora fine-tuning, we would definitely welcome that PR.
Please check that this issue hasn't been reported before.
Expected Behavior
I should be able to use
load_in_8bit: False
+load_in_4bit: False
+adapter: lora
.This issue was already observed here: #456 but closed without resolution.
Current behaviour
When you try this today, the process gets stuck, e.g.:
(after this, nothing happens)
Steps to reproduce
.
Config yaml
base_model: mistralai/Mistral-7B-v0.1
model_type: MistralForCausalLM
tokenizer_type: LlamaTokenizer
is_mistral_derived_model: true
load_in_8bit: false
load_in_4bit: false
strict: false
datasets:
type: completion
dataset_prepared_path: last_run_prepared
val_set_size: 0
output_dir: ./mistral-7b-lora-out
adapter: lora
lora_model_dir:
sequence_len: 8192
sample_packing: true
pad_to_sequence_len: true
lora_r: 64
lora_alpha: 16
lora_dropout: 0.05
lora_target_linear: true
lora_fan_in_fan_out:
lora_target_modules:
wandb_project:
wandb_entity:
wandb_watch:
wandb_run_id:
wandb_log_model:
gradient_accumulation_steps: 4
micro_batch_size: 2
num_epochs: 3
optimizer: adamw_bnb_8bit
lr_scheduler: cosine
learning_rate: 0.0002
train_on_inputs: true
group_by_length: false
bf16: true
fp16: false
tf32: false
gradient_checkpointing: false
early_stopping_patience:
resume_from_checkpoint:
local_rank:
logging_steps: 1
xformers_attention:
flash_attention: true
warmup_steps: 10
eval_steps: 20
eval_table_size:
eval_table_max_new_tokens: 128
save_steps:
debug:
deepspeed:
weight_decay: 0.0
fsdp:
fsdp_config:
special_tokens:
bos_token: "
""eos_token: "
unk_token: ""
Possible solution
No response
Which Operating Systems are you using?
Python Version
3.10.13
axolotl branch-commit
latest docker
Acknowledgements
The text was updated successfully, but these errors were encountered: