Refactor existing HuggingFace support to a more robust one

[Anindyadeep]

Current status: WIP

The PR motivation started from this issue: #1018

What this PR do?

We have an existing HuggingFace model integration on DsPY. However developers can not use this class with much flexibility. So the goal of the PR is to provide a balanced abstraction such that developers can

1. add their own model object
2. or add just different configs and arguments while instantiating, example: quantization_config, lora_config directly in the dspy.HF class
3. make it also directly compatible with the finetuning procedures (might not be applicable now)

The reason is, HuggingFace is very popular for sometimes writing out entire lifecycle. So writing entre pipelines with huggingface and dspy might need to have almost all the supported configs and HF's own addons (like bitsandbytes, peft etc), so that they can focus more on dspy instead of finding wayaround on how to add this or that with dspy huggingface.

Todos

• Initial working draft implementation
• Testing the implementation passing HF models object
• Testing the implementation passing HF models object as string
• Testing the implementation passing HF models with quantization support
• Testing the implementation passing HF models with multi-gpu support (currently not have multi gpu settings)
• Refine the code (check once more, typos, check on some todos I wrote inside the code etc)
• Write good tutorial like documentation for this
• Fininshing (link etc)

cc: @okhat, let's discuss more about this approach and feel free to provide feedbacks :)

[Anindyadeep]

Hi would love to get your thoughts on this, so that according to the feedbacks I can move forward. Thanks

cc: @arnavsinghvi11

[Anindyadeep]

Current status: WIP

The PR motivation started from this issue: #1018

What this PR do?

We have an existing HuggingFace model integration on DsPY. However developers can not use this class with much flexibility. So the goal of the PR is to provide a balanced abstraction such that developers can

1. add their own model object
2. or add just different configs and arguments while instantiating, example: quantization_config, lora_config directly in the dspy.HF class
3. make it also directly compatible with the finetuning procedures (might not be applicable now)

The reason is, HuggingFace is very popular for sometimes writing out entire lifecycle. So writing entre pipelines with huggingface and dspy might need to have almost all the supported configs and HF's own addons (like bitsandbytes, peft etc), so that they can focus more on dspy instead of finding wayaround on how to add this or that with dspy huggingface.

Todos

• Initial working draft implementation
• Testing the implementation passing HF models object
• Testing the implementation passing HF models object as string
• Testing the implementation passing HF models with quantization support
• Testing the implementation passing HF models with multi-gpu support (currently not have multi gpu settings)
• Refine the code (check once more, typos, check on some todos I wrote inside the code etc)
• Write good tutorial like documentation for this
• Fininshing (link etc)

cc: @okhat, let's discuss more about this approach and feel free to provide feedbacks :)

Hi just wanted to check on whether we can move forward with this PR or not, else we can close it. Thanks :)

[arnavsinghvi11]

would eventually be added to pyproject.toml and/or poetry? also would be helpful to give the pip install command to avoid confusion

[arnavsinghvi11]

Hi @Anindyadeep , thanks for this PR refactor of HF! This looks great!

I anticipate this PR will eventually replace hf.py so it would be great to either build on the existing file if possible. It would also be easier to understand what’s being retained and any additional differences. Following the leftover todos and testing, let me know if you have any comments/additions to review.

Looking forward to the next stage of this PR!

[Anindyadeep]

Hi @Anindyadeep , thanks for this PR refactor of HF! This looks great!

I anticipate this PR will eventually replace hf.py so it would be great to either build on the existing file if possible. It would also be easier to understand what’s being retained and any additional differences. Following the leftover todos and testing, let me know if you have any comments/additions to review.

Looking forward to the next stage of this PR!

Hey thanks for the reply, on a second though I feel this PR is a very big addition so in that case, I could re-start from the PR implementation, where I can break into smaller parts (which could be easier for you all as well for review) and doing changes from hf.py. Does that sounds good to you?

[Anindyadeep]

Hi @Anindyadeep , thanks for this PR refactor of HF! This looks great!
I anticipate this PR will eventually replace hf.py so it would be great to either build on the existing file if possible. It would also be easier to understand what’s being retained and any additional differences. Following the leftover todos and testing, let me know if you have any comments/additions to review.
Looking forward to the next stage of this PR!

Hey thanks for the reply, on a second though I feel this PR is a very big addition so in that case, I could re-start from the PR implementation, where I can break into smaller parts (which could be easier for you all as well for review) and doing changes from hf.py. Does that sounds good to you?

Let me know your thoughts @arnavsinghvi11 whenever you find some bandwidth, thanks :)

[arnavsinghvi11]

I could re-start from the PR implementation, where I can break into smaller parts (which could be easier for you all as well for review) and doing changes from hf.py. Does that sounds good to you?

Yes that would be great!