Optimal strategy to finetune

[rryltsev]

Dear CHGNet users!
I wonder if there is an optimal strategy to effectively finetune CHGNet with a small DFT dataset for a particular system. Specifically, the following questions arise:

1. How many configurations should I use for finetuninng?
2. How to set batch size and the number of epochs for a given number of configurations?
3. Should I freeze some layers of CHGNet during finetune, or is it better to retrain all the weights?

[janosh]

1. How many configurations should I use for finetuninng?

generally the more, the better but depending on your problem, even a single finetuning label can be enough

2. How to set batch size and the number of epochs for a given number of configurations?

small batch size (e.g. 8) probably preferred for fine tuning. do as many epochs as the validation loss keeps dropping

3 Should I freeze some layers of CHGNet during finetune, or is it better to retrain all the weights?

generally the fewer fine-tuning labels you have, the less layers should be unfrozen. if you have >10^4, i would unfreeze the whole model. f you have 1, only unfreeze the final layer. in between, it's a guessing game.