Integrate Hugging Face Models with Azure ML: Download, Register, Deploy, and Test

[rdheekonda]

Description

This PR facilitates the process for users to download, register, and test Hugging Face models within AML, enabling PyRiT users to utilize uncensored HF models for setting up the RedTeamingBot.

This PR includes majorly three Jupyter notebooks that streamline the process from downloading models to deploying and testing them on AML. Each notebook is designed to facilitate various stages of the model lifecycle in AML.

Download and Register Hugging Face Model in AML (download_and_register_hf_model_aml.ipynb):

This notebook guides users through the process of downloading Hugging Face models and registering them within the AML model registry environment.

Deploy Hugging Face Model to AML Managed Online Endpoint (deploy_hf_model_aml.ipynb):

Once the models are registered, this notebook assists in deploying them to AML managed online endpoints. It covers the deployment process.

Programmatic Scoring of AML Endpoint (score_aml_endpoint):

To test the deployed models, the score_aml_endpoint script is provided for programmatic scoring. It demonstrates how to interact with the AML endpoint, send inference requests, and interpret the responses.

Tests

• no new tests required
• new tests added
• existing tests adjusted

Documentation

• no documentation changes needed
• documentation added or edited
• example notebook added or updated

[github-actions]

Test Results

95 tests  ±0   95 ✅ ±0   10s ⏱️ -1s
 1 suites ±0    0 💤 ±0 
 1 files   ±0    0 ❌ ±0 

Results for commit 3ccbe7a. ± Comparison against base commit 9de40a8.

♻️ This comment has been updated with latest results.

[rlundeen2]

I think this is really good documentation. But

One thing that I think we should change in this;

Remember with the notebook testing strategy that I'm working on, notebooks need to have runnable integration test code. Meaning it can run in our build pipeline.

Most of this code is runnable. But I think some of this code should move to somewhere else. E.g. we probably don't want to be actively running "Submit the pipeline job" in our code. For example, we could put some of it in a function and not actually call the function.

[rlundeen2]

Could you convert to the notebooks to percent format? I'm actively converting all our notebooks now for another story

I have a few comments on the notebooks themselves but I think they'll be easier once they're in percent format. One that comes to mind first, is if there's other documentation we should point to? A lot of this isn't documenting pyrit, and it would be cool to point to official AML deployment documentation as much as we can so we don't have to maintain it

[rlundeen2]

Should we move where these notebooks are? I could see them going in deployment or something instead of code

[rdheekonda]

Thank you, @rlundeen2, for the suggestion on percent and jupytext. Following our discussion and mutual agreement, I have moved the code to the deployment folder and converted it into the percent format as per your advice. Consequently, we now have .pct files alongside the .ipynb files. Additionally, I didn't add a dependency of Jupytext to our pyproject.toml, anticipating that you would handle its addition. It appears that for percent files, Jupytext recommends (reference) using the .pct.py extension, which I have incorporated accordingly. Also, added reference to official AML documentation wherever it's necessary.

[rlundeen2]

I think this is really good! My only ask pre-merge is for someone to go through the documentation and see if there are bugs as they're walking through. It sounds like Christian is doing/did this? If not, I'd like to (or someone to). But if he has already, feel free to merge!

(Also, omg, I don't know how to do a general comment you can resolve in github, I just wanted you to see this/resolve before you merged haha. STILL LEARNING GITHUB)

[rdheekonda]

Hi @rlundeen2 ,
I discussed with Christian, who had used these notebooks for model deployment. He encountered distinct issues. Following our conversation, I am planning to deploy those two models using our existing scripts. Therefore, I recommend proceeding with the merge for the time being. You've included these as comments, so there isn't a direct way to mark them as resolved. However, we can proceed with the merge once you remove the requested changes to "approve" if that works for you.