You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Also are we expected to use this script to merge both reward model and sft model? At first glance, we should change forcausallm to forsequenceclassfication for reward model, correct?
The text was updated successfully, but these errors were encountered:
There is no more
_get_submoudles
inpeft
per huggingface/peft#279, so https://github.com/lvwerra/trl/blob/main/examples/stack_llama/scripts/merge_peft_adapter.py cannot use directly.Also are we expected to use this script to merge both reward model and sft model? At first glance, we should change
forcausallm
toforsequenceclassfication
for reward model, correct?The text was updated successfully, but these errors were encountered: