(proposal update) Add HuggingFace Space pushing capability to the HFModelPusher #178
Conversation
|
Thanks for the PR! 🚀 Instructions: Approve using |
|
|
||
| HuggingFace Space Hub comes with free resources to host prototype applications that use machine learning models. Currently supported application frameworks are Gradio and Streamlit. It is often a good idea to host current version of the model to the Huggingface Space, so it could be interated with real world before the production deployment. | ||
|
|
||
| By keeping these information in mind, `HFModelPusher` let you push a trained or blessed model to the HuggingFace Model Hub within a new branch within TFX pipeline. Then, if specified, it pushes an application to the HuggingFace Space Hub by injecting the current model information into the prepared template sources. |
There was a problem hiding this comment.
The hf_api has utilities for creating a PR too. Wouldn't be nice to create a PR with the newly created branch? We can, of course, parameterize that too.
There was a problem hiding this comment.
Do you have experiences?
I couldn't find any other ways but `upload_folder()' to create PR after commit/push changes.
Even if upload_folder() let us commit changes to a particular branch, it doesn't have a capability to create a new branch, so we need to create a branch beforehand. However, we have to create a new branch and push any changes to the branch to let the remote repository detect the newly created branch. So, maybe some kind of dummy files should be pushed beforehand which I want to avoid.
The 'create_pull_request()` seems like creating a PR, but it doesn't have any parameter to specify which branch the PR for.
There was a problem hiding this comment.
The revision argument in upload_folder() allows us to specify a branch name.
I'm not sure if it creates the revision when it doesn't exist, but there are workarounds for that (e.g.).
HF Hub has the best experience when the model is in main. If you expect users to use this component to create many variations of the same model, make sure you either push for one model per repo, or to move the best model to main after the exploration phase :)
There was a problem hiding this comment.
I see. Your suggestion is to create a branch with git_checkout() then call upload_folder(), right? but git_checkout() creates a branch only in the locally cloned repository. It means I have to push the branch, then upload the same files again with upload_folder(). I hoped there is APIs like create_pull_request() with revision parameter, or upload_folder() that creates a new branch.
| - `app_path` : path where the application templates are in the container that runs the TFX pipeline. This is expressed either apps.gradio.img_classifier or apps/gradio.img_classifier | ||
| - `repo_name` : the repository name to push the application to. The default value is same as the TFX pipeline name | ||
| - `space_sdk` : either `gradio` or `streamlit`. this will decide which application framework to be used for the Space repository. The default value is `gradio` | ||
| - `placeholders` : dictionary which placeholders to replace with model specific information. The keys represents describtions, and the values represents the actual placeholders to replace in the files under the `app_path`. There are currently two predefined keys, and if `placeholders` is set to `None`, the default values will be used. | ||
|
|
||
| ## Project Dependencies | ||
| - [tfx](https://pypi.org/project/tfx/) |
There was a problem hiding this comment.
To interact with the model hub it's better to use the hf_api.
There was a problem hiding this comment.
I think hf_api is part of huggingface-hub package?
There was a problem hiding this comment.
It is. I guess we should list that dependency here?
There was a problem hiding this comment.
huggingface-hub is already listed in the dependency section
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
|
how this can be merged? maybe I need /lgtm from others? |
|
/lgtm |
|
/merge |
integrated HuggingFace Space pushing capability into the HFModelPusher component proposal(#174)
cc: @sayakpaul