LLM Finetune Functionality

[dsikka]

Summary

• Enable LLM finetuning in sparsify by adding support for the llmfoundry library
• This is done by adding an additional task called finetune which provides support for all the steps required for llmfoundry finetuning integration
• Currently, this workflow launches finetuning given a llfoundry compliant yaml file (provided using the --data cli arg to the sparsify command)
• Exporting steps are currently skipped and as there is no sparseml support atm, in order to allow ddp training, finetune.py includes the steps which can be wrapped in launch_ddp and trainhook
• As the only essential arg is the data arg containing the yaml file, there are no finetune specific train args but this can be added in as greater finetuneing support is flushed out

Testing

• A sample yaml can be found under the new samples folder
• Tested locally using both 1 GPU and for DDP Training
• The following command was used for testing:

sparsify.run sparse-transfer --use-case finetune --data ../src/sparsify/auto/samples/finetune_llmfoundry_sample.yaml

This produces the following output in the running directory after training is complete.
sparse_transfer_finetune_2023_08_09_01_24_08

All paths from llmfoundry are updated to use the working directory as the root and therefore, the output directory shown above contains the checkpoints and all other llmfoundry-specific outputs.

[dsikka]

@eldarkurtic As part of your review, please pay special attention to the fsdp handling to ensure it is being applied how you expect it to. If you can pull down and test a model for accuracy/speed, that would also be appreciated.

[Satrat]

LGTM overall, just had a few nitpicky comments. I haven't tried running this on machine yet, will give it a go later today

[bfineran]

looks great overall, see few comments

addressed