feat(docs): [SCv2] Add a section about loading custom HuggingFace models from Seldon CLI

[vtaskow]

What this PR does / why we need it:
Following #5103, we need a demo of how our users might use the Seldon CLI to load custom HuggingFace models and make predictions.

Testing

• created a local file with the following yaml contents(the same as the added file in this PR)

apiVersion: mlops.seldon.io/v1alpha1
kind: Model
metadata:
  name: custom-text-gen
spec:
  storageUri: "gs://seldon-models/scv2/samples/mlserver_1.3.5/custom-text-generation-huggingface"
  requirements:
    - huggingface
  memory: 3Gi

• ran seldon model load -f seldon-model.yaml and seldon model status custom-text-gen | jq

{
  "modelName": "custom-text-gen",
  "versions": [
    {
      "version": 1,
      "serverName": "mlserver",
      "kubernetesMeta": {},
      "modelReplicaState": {
        "0": {
          "state": "Available",
          "lastChangeTimestamp": "2023-08-29T16:25:07.815102498Z"
        }
      },
      "state": {
        "state": "ModelAvailable",
        "availableReplicas": 1,
        "lastChangeTimestamp": "2023-08-29T16:25:07.815102498Z"
      },
      "modelDefn": {
        "meta": {
          "name": "custom-text-gen",
          "kubernetesMeta": {}
        },
        "modelSpec": {
          "uri": "gs://seldon-models/scv2/samples/mlserver_1.3.5/custom-text-generation-huggingface",
          "requirements": [
            "huggingface"
          ],
          "memoryBytes": "3221225472"
        },
        "deploymentSpec": {
          "replicas": 1
        }
      }
    }
  ]
}

• inferred correctly by running seldon model infer custom-text-gen \ '{"inputs": [{"name": "args","shape": [1],"datatype": "BYTES","data": ["Once upon a time in a galaxy far away"]}]}'

  '{"inputs": [{"name": "args","shape": [1],"datatype": "BYTES","data": ["Once upon a time in a galaxy far away"]}]}'
{
	"model_name": "custom-text-gen_1",
	"model_version": "1",
	"id": "fad1b559-4181-42d9-ab61-86d1092bb7e1",
	"parameters": {},
	"outputs": [
		{
			"name": "output",
			"shape": [
				1,
				1
			],
			"datatype": "BYTES",
			"parameters": {
				"content_type": "hg_jsonlist"
			},
			"data": [
				"{\"generated_text\": \"Once upon a time in a galaxy far away the universe was never like this. There were no planets, but only small ones, and these little planets would grow to be a bit larger than life. Some of these planets have small civilizations, but the\"}"
			]
		}
	]
}

• ran seldon model unload custom-text-gen

{
  "modelName": "custom-text-gen",
  "versions": [
    {
      "version": 1,
      "serverName": "mlserver",
      "kubernetesMeta": {},
      "modelReplicaState": {
        "0": {
          "state": "Unloaded",
          "lastChangeTimestamp": "2023-08-29T16:29:09.304450147Z"
        }
      },
      "state": {
        "state": "ModelTerminated",
        "lastChangeTimestamp": "2023-08-29T16:29:09.304450147Z"
      },
      "modelDefn": {
        "meta": {
          "name": "custom-text-gen",
          "kubernetesMeta": {}
        },
        "modelSpec": {
          "uri": "gs://seldon-models/scv2/samples/mlserver_1.3.5/custom-text-generation-huggingface",
          "requirements": [
            "huggingface"
          ],
          "memoryBytes": "3221225472"
        },
        "deploymentSpec": {
          "replicas": 1
        }
      }
    }
  ],
  "deleted": true
}

Which issue(s) this PR fixes:
Fixes #

Special notes for your reviewer:

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[adriangonz]

👍