Chapter02/ch2_nb1_mnist_keras.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-28T15:33:40.275240Z",
     "start_time": "2019-04-28T15:33:40.269013Z"
    }
   },
   "source": [
    "<p style=\"border: 1px solid #e7692c; border-left: 15px solid #e7692c; padding: 10px; text-align:justify;\">\n",
    "    <strong style=\"color: #e7692c\">Tip.</strong> <a style=\"color: #000000;\" href=\"https://nbviewer.jupyter.org/github/PacktPublishing/Hands-On-Computer-Vision-with-TensorFlow-2/blob/master/Chapter02/ch2_nb1_mnist_keras.ipynb\" title=\"View with Jupyter Online\">Click here to view this notebook on <code>nbviewer.jupyter.org</code></a>. \n",
    "    <br/>These notebooks are better read there, as Github default viewer ignores some of the formatting and interactive content.\n",
    "    </p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<table style=\"font-size: 1em; padding: 0; margin: 0;\">\n",
    "    <tr style=\"vertical-align: top; padding: 0; margin: 0;background-color: #ffffff\">\n",
    "        <td style=\"vertical-align: top; padding: 0; margin: 0; padding-right: 15px;\">\n",
    "    <p style=\"background: #363636; color:#ffffff; text-align:justify; padding: 10px 25px;\">\n",
    "        <strong style=\"font-size: 1.0em;\"><span style=\"font-size: 1.2em;\"><span style=\"color: #e7692c;\">Hands-on</span> Computer Vision with TensorFlow 2</span><br/>by <em>Eliot Andres</em> & <em>Benjamin Planche</em> (Packt Pub.)</strong><br/><br/>\n",
    "        <strong>> Chapter 2: TensorFlow Basics and Training a Model</strong><br/>\n",
    "    </p>\n",
    "\n",
    "<h1 style=\"width: 100%; text-align: left; padding: 0px 25px;\"><small style=\"color: #e7692c;\">Notebook 1:</small><br/>A simple computer vision model using Keras</h1>\n",
    "<br/>\n",
    "<p style=\"border-left: 15px solid #363636; text-align:justify; padding: 0 10px;\">\n",
    "    In the second chapter of the book, we introduced the Keras API and how to build a simple model.\n",
    "    <br/>In this first notebook, we will therefore detail the related code snippets and results from the book.\n",
    "</p>\n",
    "<br/>\n",
    "<p style=\"border-left: 15px solid #e7692c; padding: 0 10px; text-align:justify;\">\n",
    "    <strong style=\"color: #e7692c;\">Tip.</strong> The notebooks shared on this git repository illustrate some notions from the book \"<em><strong>Hands-on Computer Vision with TensorFlow 2</strong></em>\" written by Eliot Andres and Benjamin Planche, published by Packt. If you enjoyed the insights shared here, <a href=\"https://www.amazon.com/Hands-Computer-Vision-TensorFlow-processing/dp/1788830644\" title=\"Learn more about the book!\"><strong>please consider acquiring the book!</strong></a>\n",
    "<br/><br/>\n",
    "The book provides further guidance for those eager to learn about computer vision and to harness the power of TensorFlow 2 and Keras to build efficient recognition systems for object detection, segmentation, video processing, smartphone applications, and more.</p>\n",
    "        </td>\n",
    "        <td style=\"vertical-align: top; padding: 0; margin: 0; width: 280px;\">\n",
    "    <a href=\"https://www.amazon.com/Hands-Computer-Vision-TensorFlow-processing/dp/1788830644\" title=\"Learn more about the book!\" target=\"_blank\">\n",
    "        <img src=\"../banner_images/book_cover.png\" width=280>\n",
    "    </a>\n",
    "    <p style=\"background: #e7692c; color:#ffffff; padding: 10px; text-align:justify;\"><strong>Leverage deep learning to create powerful image processing apps with TensorFlow 2 and Keras. <br/></strong>Get the book for more insights!</p>\n",
    "    <ul style=\"height: 32px; white-space: nowrap; text-align: center; margin: 0px; padding: 0px; padding-top: 10px;\">\n",
    "    <li style=\"display: block;height: 100%;float: left;vertical-align: middle;margin: 0 25px 10px;padding: 0px;\">\n",
    "        <a href=\"https://www.amazon.com/Hands-Computer-Vision-TensorFlow-processing/dp/1788830644\" title=\"Get the book on Amazon (paperback or Kindle version)!\" target=\"_blank\">\n",
    "        <img style=\"vertical-align: middle; max-width: 72px; max-height: 32px;\" src=\"../banner_images/logo_amazon.png\" width=\"75px\">\n",
    "        </a>\n",
    "    </li>\n",
    "    <li style=\"display: inline-block;height: 100%;vertical-align: middle;float: right;margin: -5px 25px 10px;padding: 0px;\">\n",
    "        <a href=\"https://www.packtpub.com/application-development/hands-computer-vision-tensorflow-2\" title=\"Get your Packt book (paperback, PDF, ePUB, or MOBI version)!\" target=\"_blank\">\n",
    "        <img style=\"vertical-align: middle; max-width: 72px; max-height: 32px;\" src=\"../banner_images/logo_packt.png\" width=\"75px\">\n",
    "        </a>\n",
    "    </li>\n",
    "    </ul>\n",
    "        </td>\n",
    "        </tr>\n",
    "        </table>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-05-16T11:43:45.756874Z",
     "start_time": "2019-05-16T11:43:44.798323Z"
    },
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Input data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-05-16T11:43:46.132993Z",
     "start_time": "2019-05-16T11:43:45.757994Z"
    },
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "num_classes = 10\n",
    "img_rows, img_cols = 28, 28\n",
    "num_channels = 1\n",
    "input_shape = (img_rows, img_cols, num_channels)\n",
    "\n",
    "(x_train, y_train),(x_test, y_test) = tf.keras.datasets.mnist.load_data()\n",
    "x_train, x_test = x_train / 255.0, x_test / 255.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Building a simple model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-05-16T11:43:46.150349Z",
     "start_time": "2019-05-16T11:43:46.134258Z"
    },
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model = tf.keras.models.Sequential()\n",
    "model.add(tf.keras.layers.Flatten())\n",
    "model.add(tf.keras.layers.Dense(128, activation='relu'))\n",
    "model.add(tf.keras.layers.Dense(num_classes, activation='softmax'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Launch training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-05-14T13:52:39.124524Z",
     "start_time": "2019-05-14T13:51:49.141572Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 60000 samples, validate on 10000 samples\n",
      "Epoch 1/25\n",
      "60000/60000 [==============================] - 2s 35us/sample - loss: 0.6397 - accuracy: 0.8418 - val_loss: 0.3560 - val_accuracy: 0.9042\n",
      "Epoch 2/25\n",
      "60000/60000 [==============================] - 2s 32us/sample - loss: 0.3369 - accuracy: 0.9059 - val_loss: 0.2958 - val_accuracy: 0.9187\n",
      "Epoch 3/25\n",
      "60000/60000 [==============================] - 2s 33us/sample - loss: 0.2899 - accuracy: 0.9183 - val_loss: 0.2651 - val_accuracy: 0.9241\n",
      "Epoch 4/25\n",
      "60000/60000 [==============================] - 2s 33us/sample - loss: 0.2603 - accuracy: 0.9262 - val_loss: 0.2417 - val_accuracy: 0.9316\n",
      "Epoch 5/25\n",
      "60000/60000 [==============================] - 2s 33us/sample - loss: 0.2378 - accuracy: 0.9330 - val_loss: 0.2222 - val_accuracy: 0.9359\n",
      "Epoch 6/25\n",
      "60000/60000 [==============================] - 2s 33us/sample - loss: 0.2192 - accuracy: 0.9384 - val_loss: 0.2074 - val_accuracy: 0.9405\n",
      "Epoch 7/25\n",
      "60000/60000 [==============================] - 2s 33us/sample - loss: 0.2029 - accuracy: 0.9436 - val_loss: 0.1947 - val_accuracy: 0.9450\n",
      "Epoch 8/25\n",
      "60000/60000 [==============================] - 2s 32us/sample - loss: 0.1892 - accuracy: 0.9469 - val_loss: 0.1814 - val_accuracy: 0.9479\n",
      "Epoch 9/25\n",
      "60000/60000 [==============================] - 2s 33us/sample - loss: 0.1771 - accuracy: 0.9505 - val_loss: 0.1719 - val_accuracy: 0.9502\n",
      "Epoch 10/25\n",
      "60000/60000 [==============================] - 2s 34us/sample - loss: 0.1668 - accuracy: 0.9528 - val_loss: 0.1627 - val_accuracy: 0.9532\n",
      "Epoch 11/25\n",
      "60000/60000 [==============================] - 2s 33us/sample - loss: 0.1574 - accuracy: 0.9554 - val_loss: 0.1559 - val_accuracy: 0.9542\n",
      "Epoch 12/25\n",
      "60000/60000 [==============================] - 2s 33us/sample - loss: 0.1492 - accuracy: 0.9583 - val_loss: 0.1491 - val_accuracy: 0.9560\n",
      "Epoch 13/25\n",
      "60000/60000 [==============================] - 2s 36us/sample - loss: 0.1420 - accuracy: 0.9599 - val_loss: 0.1439 - val_accuracy: 0.9581\n",
      "Epoch 14/25\n",
      "60000/60000 [==============================] - 2s 33us/sample - loss: 0.1352 - accuracy: 0.9622 - val_loss: 0.1382 - val_accuracy: 0.9597\n",
      "Epoch 15/25\n",
      "60000/60000 [==============================] - 2s 34us/sample - loss: 0.1292 - accuracy: 0.9639 - val_loss: 0.1335 - val_accuracy: 0.9615\n",
      "Epoch 16/25\n",
      "60000/60000 [==============================] - 2s 34us/sample - loss: 0.1236 - accuracy: 0.9657 - val_loss: 0.1286 - val_accuracy: 0.9626\n",
      "Epoch 17/25\n",
      "60000/60000 [==============================] - 2s 33us/sample - loss: 0.1186 - accuracy: 0.9668 - val_loss: 0.1245 - val_accuracy: 0.9628\n",
      "Epoch 18/25\n",
      "60000/60000 [==============================] - 2s 33us/sample - loss: 0.1135 - accuracy: 0.9684 - val_loss: 0.1223 - val_accuracy: 0.9645\n",
      "Epoch 19/25\n",
      "60000/60000 [==============================] - 2s 33us/sample - loss: 0.1095 - accuracy: 0.9697 - val_loss: 0.1172 - val_accuracy: 0.9648\n",
      "Epoch 20/25\n",
      "60000/60000 [==============================] - 2s 33us/sample - loss: 0.1053 - accuracy: 0.9713 - val_loss: 0.1153 - val_accuracy: 0.9663\n",
      "Epoch 21/25\n",
      "60000/60000 [==============================] - 2s 33us/sample - loss: 0.1017 - accuracy: 0.9717 - val_loss: 0.1128 - val_accuracy: 0.9670\n",
      "Epoch 22/25\n",
      "60000/60000 [==============================] - 2s 32us/sample - loss: 0.0983 - accuracy: 0.9725 - val_loss: 0.1095 - val_accuracy: 0.9681\n",
      "Epoch 23/25\n",
      "60000/60000 [==============================] - 2s 33us/sample - loss: 0.0949 - accuracy: 0.9735 - val_loss: 0.1066 - val_accuracy: 0.9693\n",
      "Epoch 24/25\n",
      "60000/60000 [==============================] - 2s 33us/sample - loss: 0.0917 - accuracy: 0.9747 - val_loss: 0.1054 - val_accuracy: 0.9695\n",
      "Epoch 25/25\n",
      "60000/60000 [==============================] - 2s 33us/sample - loss: 0.0891 - accuracy: 0.9758 - val_loss: 0.1040 - val_accuracy: 0.9696\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7f9c242a3c88>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.compile(optimizer='sgd',\n",
    "              loss='sparse_categorical_crossentropy',\n",
    "              metrics=['accuracy'])\n",
    "callbacks = [tf.keras.callbacks.TensorBoard('./keras')]\n",
    "model.fit(x_train, y_train, epochs=25, verbose=1, validation_data=(x_test, y_test), callbacks=callbacks)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Running with an estimator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-29T11:04:31.961895Z",
     "start_time": "2019-04-29T11:04:31.956592Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "W0429 13:04:31.959720 140496034522880 keras.py:460] You are creating an Estimator from a Keras model manually subclassed from `Model`, that was already called on some inputs (and thus already had weights). We are currently unable to preserve the model's state (its weights) as part of the estimator in this case. Be warned that the estimator has been created using a freshly initialized version of your model.\n",
      "Note that this doesn't affect the state of the model instance you passed as `keras_model` argument.\n"
     ]
    }
   ],
   "source": [
    "estimator = tf.keras.estimator.model_to_estimator(model, model_dir='./estimator_dir')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-04-29T11:05:35.649548Z",
     "start_time": "2019-04-29T11:04:32.510795Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tensorflow_estimator.python.estimator.estimator.Estimator at 0x7fc6e40b3160>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "BATCH_SIZE = 32\n",
    "def train_input_fn():\n",
    "    train_dataset = tf.data.Dataset.from_tensor_slices((x_train, y_train))\n",
    "    train_dataset = train_dataset.batch(BATCH_SIZE).repeat()\n",
    "    return train_dataset\n",
    "\n",
    "estimator.train(train_input_fn, steps=len(x_train)//BATCH_SIZE)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "tf2",
   "language": "python",
   "name": "tf2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}