sagemaker_neo_compilation_jobs/imageclassification_caltech/Image-classification-fulltraining-highlevel-neo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model Optimization with an Image Classification Example\n",
    "1. [Introduction](#Introduction)\n",
    "2. [Prerequisites and Preprocessing](#Prequisites-and-Preprocessing)\n",
    "3. [Train the model](#Train-the-model)\n",
    "4. [Optimize trained model using SageMaker Neo and Deploy](#Optimize-trained-model-using-SageMaker-Neo-and-Deploy)\n",
    "5. [Request Inference](#Request-Inference)\n",
    "6. [Delete the Endpoints](#Delete-the-Endpoints)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "\n",
    "***\n",
    "\n",
    "Welcome to our model optimization example for image classification. In this demo, we will use the Amazon SageMaker Image Classification algorithm to train on the [Caltech-256 dataset](https://paperswithcode.com/dataset/caltech-256) and then we will demonstrate Amazon SageMaker Neo's ability to optimize models."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prequisites and Preprocessing\n",
    "\n",
    "***\n",
    "\n",
    "### Setup\n",
    "\n",
    "To get started, we need to define a few variables and obtain certain permissions that will be needed later in the example. These are:\n",
    "* A SageMaker session\n",
    "* IAM role to  give learning, storage & hosting access to your data\n",
    "* An S3 bucket, a folder & sub folders that will be used to store data and artifacts\n",
    "* SageMaker's specific Image Classification training image which should not be changed\n",
    "\n",
    "We also need to upgrade the [SageMaker SDK for Python](https://sagemaker.readthedocs.io/en/stable/v2.html) to v2.33.0 or greater and restart the kernel."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!~/anaconda3/envs/mxnet_p36/bin/pip install --upgrade sagemaker>=2.33.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sagemaker\n",
    "from sagemaker import session, get_execution_role\n",
    "\n",
    "role = get_execution_role()\n",
    "sagemaker_session = session.Session()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# S3 bucket and folders for saving code and model artifacts.\n",
    "# Feel free to specify different bucket/folders here if you wish.\n",
    "bucket = sagemaker_session.default_bucket()\n",
    "folder = \"DEMO-ImageClassification\"\n",
    "model_with_custom_code_sub_folder = folder + \"/model-with-custom-code\"\n",
    "validation_data_sub_folder = folder + \"/validation-data\"\n",
    "training_data_sub_folder = folder + \"/training-data\"\n",
    "training_output_sub_folder = folder + \"/training-output\"\n",
    "compilation_output_sub_folder = folder + \"/compilation-output\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker import session, get_execution_role\n",
    "from sagemaker.amazon.amazon_estimator import get_image_uri\n",
    "\n",
    "# S3 Location to save the model artifact after training\n",
    "s3_training_output_location = \"s3://{}/{}\".format(bucket, training_output_sub_folder)\n",
    "\n",
    "# S3 Location to save the model artifact after compilation\n",
    "s3_compilation_output_location = \"s3://{}/{}\".format(bucket, compilation_output_sub_folder)\n",
    "\n",
    "# S3 Location to save your custom code in tar.gz format\n",
    "s3_model_with_custom_code_location = \"s3://{}/{}\".format(bucket, model_with_custom_code_sub_folder)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.image_uris import retrieve\n",
    "\n",
    "aws_region = sagemaker_session.boto_region_name\n",
    "training_image = retrieve(\n",
    "    framework=\"image-classification\", region=aws_region, image_scope=\"training\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data preparation\n",
    "\n",
    "In this demo, we are using Caltech-256 dataset, pre-converted into `RecordIO` format using MXNet's [im2rec](https://mxnet.apache.org/versions/1.7/api/faq/recordio) tool. Caltech-256 dataset contains 30608 images of 256 objects. For the training and validation data, the splitting scheme followed is governed by this [MXNet example](https://github.com/apache/incubator-mxnet/blob/8ecdc49cf99ccec40b1e342db1ac6791aa97865d/example/image-classification/data/caltech256.sh). The example randomly selects 60 images per class for training, and uses the remaining data for validation. It takes around 50 seconds to convert the entire Caltech-256 dataset (~1.2GB) into `RecordIO` format on a p2.xlarge instance. SageMaker's training algorithm takes `RecordIO` files as input. For this demo, we will download the `RecordIO` files and upload it to S3. We then initialize the 256 object categories as well to a variable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import urllib.request\n",
    "\n",
    "\n",
    "def download(url):\n",
    "    filename = url.split(\"/\")[-1]\n",
    "    if not os.path.exists(filename):\n",
    "        urllib.request.urlretrieve(url, filename)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Dowload caltech-256 data files from MXNet's website\n",
    "download(\"http://data.mxnet.io/data/caltech-256/caltech-256-60-train.rec\")\n",
    "download(\"http://data.mxnet.io/data/caltech-256/caltech-256-60-val.rec\")\n",
    "\n",
    "# Upload the file to S3\n",
    "s3_training_data_location = sagemaker_session.upload_data(\n",
    "    \"caltech-256-60-train.rec\", bucket, training_data_sub_folder\n",
    ")\n",
    "s3_validation_data_location = sagemaker_session.upload_data(\n",
    "    \"caltech-256-60-val.rec\", bucket, validation_data_sub_folder\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class_labels = [\n",
    "    \"ak47\",\n",
    "    \"american-flag\",\n",
    "    \"backpack\",\n",
    "    \"baseball-bat\",\n",
    "    \"baseball-glove\",\n",
    "    \"basketball-hoop\",\n",
    "    \"bat\",\n",
    "    \"bathtub\",\n",
    "    \"bear\",\n",
    "    \"beer-mug\",\n",
    "    \"billiards\",\n",
    "    \"binoculars\",\n",
    "    \"birdbath\",\n",
    "    \"blimp\",\n",
    "    \"bonsai-101\",\n",
    "    \"boom-box\",\n",
    "    \"bowling-ball\",\n",
    "    \"bowling-pin\",\n",
    "    \"boxing-glove\",\n",
    "    \"brain-101\",\n",
    "    \"breadmaker\",\n",
    "    \"buddha-101\",\n",
    "    \"bulldozer\",\n",
    "    \"butterfly\",\n",
    "    \"cactus\",\n",
    "    \"cake\",\n",
    "    \"calculator\",\n",
    "    \"camel\",\n",
    "    \"cannon\",\n",
    "    \"canoe\",\n",
    "    \"car-tire\",\n",
    "    \"cartman\",\n",
    "    \"cd\",\n",
    "    \"centipede\",\n",
    "    \"cereal-box\",\n",
    "    \"chandelier-101\",\n",
    "    \"chess-board\",\n",
    "    \"chimp\",\n",
    "    \"chopsticks\",\n",
    "    \"cockroach\",\n",
    "    \"coffee-mug\",\n",
    "    \"coffin\",\n",
    "    \"coin\",\n",
    "    \"comet\",\n",
    "    \"computer-keyboard\",\n",
    "    \"computer-monitor\",\n",
    "    \"computer-mouse\",\n",
    "    \"conch\",\n",
    "    \"cormorant\",\n",
    "    \"covered-wagon\",\n",
    "    \"cowboy-hat\",\n",
    "    \"crab-101\",\n",
    "    \"desk-globe\",\n",
    "    \"diamond-ring\",\n",
    "    \"dice\",\n",
    "    \"dog\",\n",
    "    \"dolphin-101\",\n",
    "    \"doorknob\",\n",
    "    \"drinking-straw\",\n",
    "    \"duck\",\n",
    "    \"dumb-bell\",\n",
    "    \"eiffel-tower\",\n",
    "    \"electric-guitar-101\",\n",
    "    \"elephant-101\",\n",
    "    \"elk\",\n",
    "    \"ewer-101\",\n",
    "    \"eyeglasses\",\n",
    "    \"fern\",\n",
    "    \"fighter-jet\",\n",
    "    \"fire-extinguisher\",\n",
    "    \"fire-hydrant\",\n",
    "    \"fire-truck\",\n",
    "    \"fireworks\",\n",
    "    \"flashlight\",\n",
    "    \"floppy-disk\",\n",
    "    \"football-helmet\",\n",
    "    \"french-horn\",\n",
    "    \"fried-egg\",\n",
    "    \"frisbee\",\n",
    "    \"frog\",\n",
    "    \"frying-pan\",\n",
    "    \"galaxy\",\n",
    "    \"gas-pump\",\n",
    "    \"giraffe\",\n",
    "    \"goat\",\n",
    "    \"golden-gate-bridge\",\n",
    "    \"goldfish\",\n",
    "    \"golf-ball\",\n",
    "    \"goose\",\n",
    "    \"gorilla\",\n",
    "    \"grand-piano-101\",\n",
    "    \"grapes\",\n",
    "    \"grasshopper\",\n",
    "    \"guitar-pick\",\n",
    "    \"hamburger\",\n",
    "    \"hammock\",\n",
    "    \"harmonica\",\n",
    "    \"harp\",\n",
    "    \"harpsichord\",\n",
    "    \"hawksbill-101\",\n",
    "    \"head-phones\",\n",
    "    \"helicopter-101\",\n",
    "    \"hibiscus\",\n",
    "    \"homer-simpson\",\n",
    "    \"horse\",\n",
    "    \"horseshoe-crab\",\n",
    "    \"hot-air-balloon\",\n",
    "    \"hot-dog\",\n",
    "    \"hot-tub\",\n",
    "    \"hourglass\",\n",
    "    \"house-fly\",\n",
    "    \"human-skeleton\",\n",
    "    \"hummingbird\",\n",
    "    \"ibis-101\",\n",
    "    \"ice-cream-cone\",\n",
    "    \"iguana\",\n",
    "    \"ipod\",\n",
    "    \"iris\",\n",
    "    \"jesus-christ\",\n",
    "    \"joy-stick\",\n",
    "    \"kangaroo-101\",\n",
    "    \"kayak\",\n",
    "    \"ketch-101\",\n",
    "    \"killer-whale\",\n",
    "    \"knife\",\n",
    "    \"ladder\",\n",
    "    \"laptop-101\",\n",
    "    \"lathe\",\n",
    "    \"leopards-101\",\n",
    "    \"license-plate\",\n",
    "    \"lightbulb\",\n",
    "    \"light-house\",\n",
    "    \"lightning\",\n",
    "    \"llama-101\",\n",
    "    \"mailbox\",\n",
    "    \"mandolin\",\n",
    "    \"mars\",\n",
    "    \"mattress\",\n",
    "    \"megaphone\",\n",
    "    \"menorah-101\",\n",
    "    \"microscope\",\n",
    "    \"microwave\",\n",
    "    \"minaret\",\n",
    "    \"minotaur\",\n",
    "    \"motorbikes-101\",\n",
    "    \"mountain-bike\",\n",
    "    \"mushroom\",\n",
    "    \"mussels\",\n",
    "    \"necktie\",\n",
    "    \"octopus\",\n",
    "    \"ostrich\",\n",
    "    \"owl\",\n",
    "    \"palm-pilot\",\n",
    "    \"palm-tree\",\n",
    "    \"paperclip\",\n",
    "    \"paper-shredder\",\n",
    "    \"pci-card\",\n",
    "    \"penguin\",\n",
    "    \"people\",\n",
    "    \"pez-dispenser\",\n",
    "    \"photocopier\",\n",
    "    \"picnic-table\",\n",
    "    \"playing-card\",\n",
    "    \"porcupine\",\n",
    "    \"pram\",\n",
    "    \"praying-mantis\",\n",
    "    \"pyramid\",\n",
    "    \"raccoon\",\n",
    "    \"radio-telescope\",\n",
    "    \"rainbow\",\n",
    "    \"refrigerator\",\n",
    "    \"revolver-101\",\n",
    "    \"rifle\",\n",
    "    \"rotary-phone\",\n",
    "    \"roulette-wheel\",\n",
    "    \"saddle\",\n",
    "    \"saturn\",\n",
    "    \"school-bus\",\n",
    "    \"scorpion-101\",\n",
    "    \"screwdriver\",\n",
    "    \"segway\",\n",
    "    \"self-propelled-lawn-mower\",\n",
    "    \"sextant\",\n",
    "    \"sheet-music\",\n",
    "    \"skateboard\",\n",
    "    \"skunk\",\n",
    "    \"skyscraper\",\n",
    "    \"smokestack\",\n",
    "    \"snail\",\n",
    "    \"snake\",\n",
    "    \"sneaker\",\n",
    "    \"snowmobile\",\n",
    "    \"soccer-ball\",\n",
    "    \"socks\",\n",
    "    \"soda-can\",\n",
    "    \"spaghetti\",\n",
    "    \"speed-boat\",\n",
    "    \"spider\",\n",
    "    \"spoon\",\n",
    "    \"stained-glass\",\n",
    "    \"starfish-101\",\n",
    "    \"steering-wheel\",\n",
    "    \"stirrups\",\n",
    "    \"sunflower-101\",\n",
    "    \"superman\",\n",
    "    \"sushi\",\n",
    "    \"swan\",\n",
    "    \"swiss-army-knife\",\n",
    "    \"sword\",\n",
    "    \"syringe\",\n",
    "    \"tambourine\",\n",
    "    \"teapot\",\n",
    "    \"teddy-bear\",\n",
    "    \"teepee\",\n",
    "    \"telephone-box\",\n",
    "    \"tennis-ball\",\n",
    "    \"tennis-court\",\n",
    "    \"tennis-racket\",\n",
    "    \"theodolite\",\n",
    "    \"toaster\",\n",
    "    \"tomato\",\n",
    "    \"tombstone\",\n",
    "    \"top-hat\",\n",
    "    \"touring-bike\",\n",
    "    \"tower-pisa\",\n",
    "    \"traffic-light\",\n",
    "    \"treadmill\",\n",
    "    \"triceratops\",\n",
    "    \"tricycle\",\n",
    "    \"trilobite-101\",\n",
    "    \"tripod\",\n",
    "    \"t-shirt\",\n",
    "    \"tuning-fork\",\n",
    "    \"tweezer\",\n",
    "    \"umbrella-101\",\n",
    "    \"unicorn\",\n",
    "    \"vcr\",\n",
    "    \"video-projector\",\n",
    "    \"washing-machine\",\n",
    "    \"watch-101\",\n",
    "    \"waterfall\",\n",
    "    \"watermelon\",\n",
    "    \"welding-mask\",\n",
    "    \"wheelbarrow\",\n",
    "    \"windmill\",\n",
    "    \"wine-bottle\",\n",
    "    \"xylophone\",\n",
    "    \"yarmulke\",\n",
    "    \"yo-yo\",\n",
    "    \"zebra\",\n",
    "    \"airplanes-101\",\n",
    "    \"car-side-101\",\n",
    "    \"faces-easy-101\",\n",
    "    \"greyhound\",\n",
    "    \"tennis-shoes\",\n",
    "    \"toad\",\n",
    "    \"clutter\",\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train the model\n",
    "\n",
    "***\n",
    "\n",
    "Now that we are done with all the setup that is needed, we are ready to train our object detector. To begin, let us create a ``sagemaker.estimator.Estimator`` object. This estimator is required to launch the training job.\n",
    "\n",
    "We specify the following parameters while creating the estimator:\n",
    "\n",
    "* ``image_uri``: This is set to the training_image uri we defined previously. Once set, this image will be used later while running the training job.\n",
    "* ``role``: This is the IAM role which we defined previously.\n",
    "* ``instance_count``: This is the number of instances on which to run the training. When the number of instances is greater than one, then the image classification algorithm will run in distributed settings. \n",
    "* ``instance_type``: This indicates the type of machine on which to run the training. For this example we will use `ml.p3.8xlarge`.\n",
    "* ``volume_size``: This is the size in GB of the EBS volume to use for storing input data during training. Must be large enough to store training data as File Mode is used.\n",
    "* ``max_run``: This is the timeout value in seconds for training. After this amount of time SageMaker terminates the job regardless of its current status.\n",
    "* ``input_mode``: This is set to `File` in this example. SageMaker copies the training dataset from the S3 location to a local directory.\n",
    "* ``output_path``: This is the S3 path in which the training output is stored. We are assigning it to `s3_training_output_location` defined previously.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ic_estimator = sagemaker.estimator.Estimator(\n",
    "    image_uri=training_image,\n",
    "    role=role,\n",
    "    instance_count=1,\n",
    "    instance_type=\"ml.p3.8xlarge\",\n",
    "    volume_size=50,\n",
    "    max_run=360000,\n",
    "    input_mode=\"File\",\n",
    "    output_path=s3_training_output_location,\n",
    "    base_job_name=\"img-classification-training\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Following are certain hyperparameters that are specific to the algorithm which are also set:\n",
    "\n",
    "* ``num_layers``: The number of layers (depth) for the network. We use 18 in this samples but other values such as 50, 152 can be used.\n",
    "* ``image_shape``: The input image dimensions,'num_channels, height, width', for the network. It should be no larger than the actual image size. The number of channels should be same as the actual image.\n",
    "* ``num_classes``: This is the number of output classes for the new dataset. Imagenet was trained with 1000 output classes but the number of output classes can be changed for fine-tuning. For caltech, we use 257 because it has 256 object categories + 1 clutter class.\n",
    "* ``num_training_samples``: This is the total number of training samples. It is set to 15240 for caltech dataset with the current split.\n",
    "* ``mini_batch_size``: The number of training samples used for each mini batch. In distributed training, the number of training samples used per batch will be N * mini_batch_size where N is the number of hosts on which training is run.\n",
    "* ``epochs``: Number of training epochs.\n",
    "* ``learning_rate``: Learning rate for training.\n",
    "* ``top_k``: Report the top-k accuracy during training.\n",
    "* ``precision_dtype``: Training datatype precision (default: float32). If set to 'float16', the training will be done in mixed_precision mode and will be faster than float32 mode."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ic_estimator.set_hyperparameters(\n",
    "    num_layers=18,\n",
    "    image_shape=\"3,224,224\",\n",
    "    num_classes=257,\n",
    "    num_training_samples=15420,\n",
    "    mini_batch_size=128,\n",
    "    epochs=5,\n",
    "    learning_rate=0.01,\n",
    "    top_k=2,\n",
    "    use_pretrained_model=1,\n",
    "    precision_dtype=\"float32\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we setup the input ``data_channels`` to be used later for training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_data = sagemaker.inputs.TrainingInput(\n",
    "    s3_training_data_location, content_type=\"application/x-recordio\", s3_data_type=\"S3Prefix\"\n",
    ")\n",
    "\n",
    "validation_data = sagemaker.inputs.TrainingInput(\n",
    "    s3_validation_data_location, content_type=\"application/x-recordio\", s3_data_type=\"S3Prefix\"\n",
    ")\n",
    "\n",
    "data_channels = {\"train\": train_data, \"validation\": validation_data}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After we've created the estimator object, we can train the model using ``fit()`` API"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "ic_estimator.fit(inputs=data_channels, logs=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Optimize trained model using SageMaker Neo and Deploy\n",
    "\n",
    "***\n",
    "\n",
    "We will use SageMaker Neo's ``compile_model()`` API while specifying ``MXNet`` as the framework and the version to optimize the model. When calling this API, we also specify the target instance family, correct input shapes for the model and the S3 location to which the compiled model artifacts would be stored. For this example, we will choose ``ml_c5`` as the target instance family."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "optimized_ic = ic_estimator.compile_model(\n",
    "    target_instance_family=\"ml_c5\",\n",
    "    input_shape={\"data\": [1, 3, 224, 224]},\n",
    "    output_path=s3_compilation_output_location,\n",
    "    framework=\"mxnet\",\n",
    "    framework_version=\"1.8\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After compiled artifacts are generated and we have a ``sagemaker.model.Model`` object, we then create a ``sagemaker.mxnet.model.MXNetModel`` object while specifying the following parameters:\n",
    "* ``model_data``: s3 location where compiled model artifact is stored\n",
    "* ``image_uri``: Neo's Inference Image URI for MXNet\n",
    "* ``framework_version``: set to MXNet's v1.8.0\n",
    "* ``role`` & ``sagemaker_session`` : IAM role and sagemaker session which we defined in the setup\n",
    "* ``entry_point``: points to the entry_point script. In our example the script has SageMaker's hosting functions implementation\n",
    "* ``py_version``: We are required to set to python version 3\n",
    "* ``env``: A dict to specify the environment variables. We are required to set MMS_DEFAULT_RESPONSE_TIMEOUT to 500\n",
    "* ``code_location``: s3 location where repacked model.tar.gz is stored. Repacked tar file consists of compiled model artifacts and entry_point script "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.mxnet.model import MXNetModel\n",
    "\n",
    "optimized_ic_model = MXNetModel(\n",
    "    model_data=optimized_ic.model_data,\n",
    "    image_uri=optimized_ic.image_uri,\n",
    "    framework_version=\"1.8.0\",\n",
    "    role=role,\n",
    "    sagemaker_session=sagemaker_session,\n",
    "    entry_point=\"inference.py\",\n",
    "    py_version=\"py37\",\n",
    "    env={\"MMS_DEFAULT_RESPONSE_TIMEOUT\": \"500\"},\n",
    "    code_location=s3_model_with_custom_code_location,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now deploy this ``sagemaker.mxnet.model.MXNetModel`` using the ``deploy()`` API, for which we need to use an instance_type belonging to the target_instance_family we used for compilation. For this example, we will choose ``ml.c5.4xlarge`` instance as we compiled for ``ml_c5``. The API also allow us to set the number of initial_instance_count that will be used for the Endpoint. By default the API will use ``JSONSerializer()`` and ``JSONDeserializer()`` for ``sagemaker.mxnet.model.MXNetModel`` whose ``CONTENT_TYPE`` is ``application/json``. The API creates a SageMaker endpoint that we can use to perform inference.\n",
    "\n",
    "**Note**: If you compiled the model for a GPU `target_instance_family` then please make sure to deploy to one of the same target `instance_type` below and also make necessary changes in the entry point script `inference.py`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "optimized_ic_classifier = optimized_ic_model.deploy(\n",
    "    initial_instance_count=1, instance_type=\"ml.c5.4xlarge\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Request Inference\n",
    "\n",
    "***\n",
    "\n",
    "Once the endpoint is in ``InService`` we can then send a test image ``test.jpg`` and get the prediction result from the endpoint using SageMaker's ``predict()`` API. Instead of sending the raw image to the endpoint for prediction we will prepare and send the payload which is in a form acceptable by the API. Upon receiving the prediction result we will print the class label and probability."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import PIL.Image\n",
    "import numpy as np\n",
    "from IPython.display import Image\n",
    "\n",
    "test_file = \"test.jpg\"\n",
    "test_image = PIL.Image.open(test_file)\n",
    "payload = np.asarray(test_image.resize((224, 224)))\n",
    "Image(test_file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "result = optimized_ic_classifier.predict(payload)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "index = np.argmax(result)\n",
    "print(\"Result: label - \" + class_labels[index] + \", probability - \" + str(result[index]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Delete the Endpoint\n",
    "\n",
    "***\n",
    "\n",
    "Having an endpoint running will incur some costs. Therefore as an optional clean-up job, you can delete it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Endpoint name: \" + optimized_ic_classifier.endpoint_name)\n",
    "optimized_ic_classifier.delete_endpoint()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "conda_mxnet_p36",
   "language": "python",
   "name": "conda_mxnet_p36"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.13"
  },
  "notice": "Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved.  Licensed under the Apache License, Version 2.0 (the \"License\"). You may not use this file except in compliance with the License. A copy of the License is located at http://aws.amazon.com/apache2.0/ or in the \"license\" file accompanying this file. This file is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License."
 },
 "nbformat": 4,
 "nbformat_minor": 4
}