Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore,refactor(egen): minor updates to Automl Tabular Classification Model Evaluation Notebook #3218

Merged
merged 2 commits into from
Jul 10, 2024
Merged

chore,refactor(egen): minor updates to Automl Tabular Classification Model Evaluation Notebook #3218

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5bcf6b6
chore,refactor(Egen): Removed the google-cloud-pipeline-components pa…
Jul 5, 2024
faf64b3
chore(egen):renamed location variable to LOCATION
Jul 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
168 changes: 67 additions & 101 deletions notebooks/official/model_evaluation/automl_tabular_classification_model_evaluation.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,24 +32,26 @@
"# Vertex AI Pipelines: Evaluating batch prediction results from an AutoML Tabular classification model\n",
"\n",
"<table align=\"left\">\n",
"\n",
" <td>\n",
" <td style=\"text-align: center\">\n",
" <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model_evaluation/automl_tabular_classification_model_evaluation.ipynb\">\n",
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"> Run in Colab\n",
" <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Google Colaboratory logo\"><br> Open in Colab\n",
" </a>\n",
" </td>\n",
" <td>\n",
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model_evaluation/automl_tabular_classification_model_evaluation.ipynb\">\n",
" <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\">\n",
" View on GitHub\n",
" <td style=\"text-align: center\">\n",
" <a href=\"https://console.cloud.google.com/vertex-ai/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fvertex-ai-samples%2Fmain%2Fnotebooks%2Fofficial%2Fmodel_evaluation%2Fautoml_tabular_classification_model_evaluation.ipynb\">\n",
" <img width=\"32px\" src=\"https://cloud.google.com/ml-engine/images/colab-enterprise-logo-32px.png\" alt=\"Google Cloud Colab Enterprise logo\"><br> Open in Colab Enterprise\n",
" </a>\n",
" </td>\n",
" <td>\n",
" </td> \n",
" <td style=\"text-align: center\">\n",
" <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://github.com/raw/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/model_evaluation/automl_tabular_classification_model_evaluation.ipynb\">\n",
" <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\">\n",
" Open in Vertex AI Workbench\n",
" <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\"><br> Open in Workbench\n",
" </a>\n",
" </td>\n",
" <td style=\"text-align: center\">\n",
" <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/model_evaluation/automl_tabular_classification_model_evaluation.ipynb\">\n",
" <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\"><br> View on GitHub\n",
" </a>\n",
" </td> \n",
" </td>\n",
"</table>"
]
},
Expand Down Expand Up @@ -121,7 +123,7 @@
"- `PhotoAmt`: Total uploaded photos for this pet\n",
"- `Adopted`: Whether or not the pet was adopted (Yes/No).\n",
"\n",
"**Note**: This dataset is moved to a public Cloud Storage bucket from where it is accessed in this notebook."
"**Note**: This dataset is moved to a public Cloud Storage bucket from where it's accessed in this notebook."
]
},
{
Expand All @@ -146,159 +148,123 @@
{
"cell_type": "markdown",
"metadata": {
"id": "i7EUnXsZhAGF"
},
"source": [
"## Installation\n",
"\n",
"Install the following packages required to execute this notebook. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "2b4ef9b72d43"
"id": "f0316df526f8"
},
"outputs": [],
"source": [
"! pip3 install --upgrade --quiet google-cloud-aiplatform \\\n",
" google-cloud-pipeline-components==1.0.26 \\\n",
" matplotlib"
"## Get started"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "restart"
"id": "i7EUnXsZhAGF"
},
"source": [
"### Colab only: Uncomment the following cell to restart the kernel"
"### Install Vertex AI SDK for Python and other required packages"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "D-ZBOjErv5mM"
"id": "2b4ef9b72d43"
},
"outputs": [],
"source": [
"# Automatically restart kernel after installs so that your environment can access the new packages\n",
"# import IPython\n",
"\n",
"# app = IPython.Application.instance()\n",
"# app.kernel.do_shutdown(True)"
"! pip3 install --upgrade --quiet google-cloud-aiplatform \\\n",
" google-cloud-pipeline-components \\\n",
" matplotlib"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "yfEglUHQk9S3"
"id": "ff555b32bab8"
},
"source": [
"## Before you begin\n",
"\n",
"### Set your project ID\n",
"### Restart runtime (Colab only)\n",
"\n",
"**If you don't know your project ID**, try the following:\n",
"* Run `gcloud config list`.\n",
"* Run `gcloud projects list`.\n",
"* See the support page: [Locate the project ID](https://support.google.com/googleapi/answer/7014113)"
"To use the newly installed packages, you must restart the runtime on Google Colab."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "set_project_id"
"id": "f09b4dff629a"
},
"outputs": [],
"source": [
"PROJECT_ID = \"[your-project-id]\" # @param {type:\"string\"}\n",
"import sys\n",
"\n",
"# Set the project id\n",
"! gcloud config set project {PROJECT_ID}"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "region"
},
"source": [
"#### Region\n",
"if \"google.colab\" in sys.modules:\n",
"\n",
" import IPython\n",
"\n",
"You can also change the `REGION` variable used by Vertex AI. Learn more about [Vertex AI regions](https://cloud.google.com/vertex-ai/docs/general/locations)."
" app = IPython.Application.instance()\n",
" app.kernel.do_shutdown(True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"cell_type": "markdown",
"metadata": {
"id": "region"
"id": "4a2b7b59bbf7"
},
"outputs": [],
"source": [
"REGION = \"us-central1\" # @param {type: \"string\"}"
"<div class=\"alert alert-block alert-warning\">\n",
"<b>⚠️ The kernel is going to restart. Wait until it's finished before continuing to the next step. ⚠️</b>\n",
"</div>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "gcp_authenticate"
"id": "f82e28c631cc"
},
"source": [
"### Authenticate your Google Cloud account\n",
"### Authenticate your notebook environment (Colab only)\n",
"\n",
"Depending on your Jupyter environment, you may have to manually authenticate. Follow the relevant instructions below.\n",
"\n",
"**1. Vertex AI Workbench**\n",
"* Do nothing as you are already authenticated.\n",
"\n",
"**2. Local JupyterLab instance, uncomment and run:**"
"Authenticate your environment on Google Colab."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "ce6043da7b33"
"id": "46604f70e831"
},
"outputs": [],
"source": [
"# ! gcloud auth login"
"import sys\n",
"\n",
"if \"google.colab\" in sys.modules:\n",
"\n",
" from google.colab import auth\n",
"\n",
" auth.authenticate_user()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "0367eac06a10"
"id": "191d1345e064"
},
"source": [
"**3. Colab, uncomment and run:**"
"### Set Google Cloud project information\n",
"\n",
"To get started using Vertex AI, you must have an existing Google Cloud project. Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "21ad4dbb4a61"
"id": "set_project_id"
},
"outputs": [],
"source": [
"# from google.colab import auth\n",
"# auth.authenticate_user()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "c13224697bfb"
},
"source": [
"**4. Service account or other**\n",
"* See how to grant Cloud Storage permissions to your service account at https://cloud.google.com/storage/docs/gsutil/commands/iam#ch-examples."
"PROJECT_ID = \"[your-project-id]\" # @param {type:\"string\"}\n",
"LOCATION = \"us-central1\" # @param {type: \"string\"}"
]
},
{
Expand Down Expand Up @@ -344,7 +310,9 @@
},
"outputs": [],
"source": [
"BUCKET_URI = f\"gs://your-bucket-name-{PROJECT_ID}-unique\" # @param {type:\"string\"}"
"BUCKET_URI = (\n",
" f\"gs://model-evaluation-bucket-{PROJECT_ID}-unique\" # @param {type:\"string\"}\n",
")"
]
},
{
Expand All @@ -364,7 +332,7 @@
},
"outputs": [],
"source": [
"! gsutil mb -l $REGION $BUCKET_URI"
"! gsutil mb -l $LOCATION $BUCKET_URI"
]
},
{
Expand All @@ -375,7 +343,7 @@
"source": [
"#### Service Account\n",
"\n",
"You use a service account to create Vertex AI Pipeline jobs. If you do not want to use your project's Compute Engine service account, set `SERVICE_ACCOUNT` to another service account ID."
"You use a service account to create Vertex AI Pipeline jobs. If you don't want to use your project's Compute Engine service account, set `SERVICE_ACCOUNT` to another service account ID."
]
},
{
Expand All @@ -386,7 +354,7 @@
},
"outputs": [],
"source": [
"SERVICE_ACCOUNT = \"[your-service-account]\" # @param {type:\"string\"}"
"SERVICE_ACCOUNT = \"\" # @param {type:\"string\"}"
]
},
{
Expand Down Expand Up @@ -488,7 +456,7 @@
},
"outputs": [],
"source": [
"aiplatform.init(project=PROJECT_ID, location=REGION, staging_bucket=BUCKET_URI)"
"aiplatform.init(project=PROJECT_ID, location=LOCATION, staging_bucket=BUCKET_URI)"
]
},
{
Expand Down Expand Up @@ -985,8 +953,6 @@
},
"outputs": [],
"source": [
"import os\n",
"\n",
"# Delete model resource\n",
"model.delete()\n",
"\n",
Expand All @@ -999,10 +965,10 @@
"# Delete the evaluation pipeline\n",
"job.delete()\n",
"\n",
"# Delete Cloud Storage objects\n",
"delete_bucket = False\n",
"if delete_bucket or os.getenv(\"IS_TESTING\"):\n",
" ! gsutil -m rm -r $BUCKET_URI"
"# Delete the Cloud Storage bucket\n",
"delete_bucket = False # Set True for deletion\n",
"if delete_bucket:\n",
" ! gsutil rm -rf {BUCKET_URI}"
]
}
],
Expand Down