No optimizer.bin in Step 2

[mihara-bot]

Hi，
When I run "Step 2: Building the gradient datastore"
FileNotFoundError: [Errno 2] No such file or directory: '../out/llama2-7b-p0.05-lora-seed3/checkpoint-1688/optimizer.bin'
I check the folder "llama2-7b-p0.05-lora-seed3" generate from Step 1, only files optimizer.pt in checkpoint-1688.

I noticed that in other issues, some scholars had problem on generating optimizer.pt. I think my problem is different from these.
May I kindly ask for you help?

[xiamengzhou]

Could you confirm that you are using transformers==4.36.2? Transformers package gets frequently updated especially the part that supports distributed training :(

[pengming617]

i have the same error while using transformers==4.36.2

No such file or directory: 'out/llama2-7b-p0.05-lora-seed3/checkpoint-1688/optimizer.bin'

[xiamengzhou]

@pengming617 Please refer to this thread! I think you need to use fsdp to get optimizer.pt.