07. Deep Learning Fundamentals/7.13 Classifying Fashion Products Using CNN.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Classifying Fashion Products Using CNN"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, we will import our required libraries as usual"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we will read the data. The dataset is available in the tensorflow.examples, so we can\n",
    "directly extract the data as follows,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting data/fashion/train-images-idx3-ubyte.gz\n",
      "Extracting data/fashion/train-labels-idx1-ubyte.gz\n",
      "Extracting data/fashion/t10k-images-idx3-ubyte.gz\n",
      "Extracting data/fashion/t10k-labels-idx1-ubyte.gz\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "fashion_mnist = input_data.read_data_sets('data/fashion/', one_hot=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " We will check what we got in our data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "No of images in training set (55000, 784)\n",
      "No of labels in training set (55000, 10)\n",
      "No of images in test set (10000, 784)\n",
      "No of labels in test set (10000, 10)\n"
     ]
    }
   ],
   "source": [
    "print(\"No of images in training set {}\".format(fashion_mnist.train.images.shape))\n",
    "print(\"No of labels in training set {}\".format(fashion_mnist.train.labels.shape))\n",
    "\n",
    "print(\"No of images in test set {}\".format(fashion_mnist.test.images.shape))\n",
    "print(\"No of labels in test set {}\".format(fashion_mnist.test.labels.shape))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "So we have 55000 data points in a training set and 10000 data points in a test set. Also we\n",
    "have 10 labels which means we have 10 categories.\n",
    "\n",
    "we have 10 categories of product and we will label all of them. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "labels = {\n",
    " 0: 'T-shirt/top',\n",
    " 1: 'Trouser',\n",
    " 2: 'Pullover',\n",
    " 3: 'Dress',\n",
    " 4: 'Coat',\n",
    " 5: 'Sandal',\n",
    " 6: 'Shirt',\n",
    " 7: 'Sneaker',\n",
    " 8: 'Bag',\n",
    " 9: 'Ankle boot'\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we look at some of our images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "y = 6 (Shirt)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7efe385e6e80>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAEuRJREFUeJzt3W1sVOeVB/D/AcyrccDYGJI6a9I4\nKBEIjCZo86IoqyoVXiGRfiAqHypWqkqVFGkbVclGfKBRopWS1RY2H1aVYIMgUps2UcuGRFG2UbRR\ntsrmxYmg0GV5CXGowRgbCJj3gM9+8KVywfecydyZueM9/58U2Z4z1/fxDP/M2Oc+zyOqCiKKZ1ze\nAyCifDD8REEx/ERBMfxEQTH8REEx/ERBMfxEQTH8REEx/ERBTajmyZqamrStra2apwzhypUrqTXv\nCs4JE+x/AlevXs10/MWLF1NrkydPNo+lr6+7uxsDAwNSzH0zhV9ElgF4AcB4AP+mqs9Z929ra0NX\nV1eWU9YkL2AiRT0XJTtx4kRqzQofAMyePdusnz592qw3Njaa9f3796fW2tvbzWPHjx9v1j1ZLl33\nnrO8n/M0hUKh6PuW/LZfRMYD+FcAnQDuArBKRO4q9fsRUXVl+Z1/KYCDqnpIVS8D+BWAFeUZFhFV\nWpbw3wLgTyO+7klu+wsiskZEukSkq7+/P8PpiKicsoR/tF9qbvhFSFU3qWpBVQvNzc0ZTkdE5ZQl\n/D0AWkd8/Q0AR7MNh4iqJUv4PwbQLiLzRGQigO8C2FGeYRFRpZXc6lPVKyKyFsB/YLjVt0VV/1i2\nkVVZltZN1raO10s/cOCAWX/99ddTa08++aR57KOPPmrWH3/8cbO+du1as/7WW2+l1jZv3mwe29nZ\nadbr6+vNeiXbbXm18sopU59fVd8E8GaZxkJEVcTLe4mCYviJgmL4iYJi+ImCYviJgmL4iYKq6nz+\nPFVyZ6Kenh6z/sYbb5j1L774wqw3NDSY9Y6OjtSa1yvftWuXWX///ffN+oULF8z6s88+m1qbOXOm\neeyGDRvMelNTk1m///77U2sLFy40j/3/0Mf38JWfKCiGnygohp8oKIafKCiGnygohp8oqDCtPo/X\n2rFWyH3llVfMY+vq6sz6ggULzPqsWbPM+vnz51Nrzz//vHns7t27zfrGjRvN+vr16836HXfckVr7\n/PPPzWMfeOABs370qL12zEcffVTyuZcvX27WvZWFa3V135H4yk8UFMNPFBTDTxQUw08UFMNPFBTD\nTxQUw08UVJg+f9a+6vbt21NrXk/Xq3/11VeZ6lOnTk2tffbZZ+axc+bMMetbt2416zt37jTrfX19\nqTXvOfHq3tit7eFOnTplHrtnzx6zvmjRIrM+FvCVnygohp8oKIafKCiGnygohp8oKIafKCiGnyio\nTH1+EekGMAjgKoArqloox6Dy4G2Tbc2Z9/rRXk/ZO96bzz80NJRamzJlinnsuXPnzLrVpy/GlStX\nUmve9Q/ec2L93N7x3mPuXR/h9flrYb6+pxwX+fyNqg6U4fsQURXxbT9RUFnDrwB+JyKfiMiacgyI\niKoj69v++1T1qIjMBvC2iPyvqr438g7J/xTWAMCtt96a8XREVC6ZXvlV9Wjy8TiA7QCWjnKfTapa\nUNVCc3NzltMRURmVHH4RmSYi0699DuDbAOypUERUM7K87W8BsD1paUwA8EtVfassoyKiiis5/Kp6\nCMDYn9Sc8Prdg4ODqTVrPj0AXL582ax76/p7a8R78/2znPvw4cNmvb6+3qyPG5f+5tI7d1aTJ09O\nrXl9+OPHj5t17zmdOHGiWa8FbPURBcXwEwXF8BMFxfATBcXwEwXF8BMFFWbpbk9vb69Zt6amWjUA\nGBiwJz3edNNNZt1ql3l1q90F+G1Eb0qwx5p2603p9VqB1jRrALh06VJqzXtMvefUOzdbfURUsxh+\noqAYfqKgGH6ioBh+oqAYfqKgGH6ioNjnT3h9fmsKqLeEtNev7u7uNuvz588369b0Uu/c3nUA3vFn\nzpwx6zNnzkyteb1079zetFyrz+9dvzBhgh2NI0eOmPUZM2aY9VrAV36ioBh+oqAYfqKgGH6ioBh+\noqAYfqKgGH6ioNjnT3h9fmv+94ULF8xjvbo3t9xbRtrql3tLTHtLb1+8eNGse9tonzx5MrXmzXn3\ntib/4IMPzLrViy8U7N3kvesAvDUaxgK+8hMFxfATBcXwEwXF8BMFxfATBcXwEwXF8BMF5fb5RWQL\ngOUAjqvqguS2RgC/BtAGoBvAI6p6qnLDrLxjx46ZdWvuuDev/PTp02bd2x68s7PTrHvXAVi86wC8\n7b+9df+zXIPgbX1+6NAhs249p3feead5bEtLi1k/ceKEWR8Linnl3wpg2XW3PQXgHVVtB/BO8jUR\njSFu+FX1PQDXX6a1AsC25PNtAB4u87iIqMJK/Z2/RVV7ASD5OLt8QyKiaqj4H/xEZI2IdIlIV39/\nf6VPR0RFKjX8fSIyFwCSj6l/cVLVTapaUNVCc3NziacjonIrNfw7AKxOPl8N4LXyDIeIqsUNv4i8\nDOC/AcwXkR4R+T6A5wA8JCIHADyUfE1EY4jb51fVVSmlb5V5LLnyevHeOu6WU6fsSyC8OfPefH9r\nn/us39tbl986NwDcdtttJX9vb62A8+fPm3XrOfPOPW/ePLPu7bUwFvAKP6KgGH6ioBh+oqAYfqKg\nGH6ioBh+oqC4dHfCm1ZrLXHtTem1tqkGgI6ODrPe09Nj1hsbG1Nr3s/ltcs83ve3WoHe4+Z97yVL\nlph1a2lvr9XnbV3uLcfutUC9n70a+MpPFBTDTxQUw08UFMNPFBTDTxQUw08UFMNPFFSYPr83PdTr\nKU+bNi215vV0vV66dx2ANzZr+Wxr6WwAOHv2rFmfPdtentH7/tbS33V1deax3jbY3vUR7777bmot\ny2MK+Euae9cBeMuSVwNf+YmCYviJgmL4iYJi+ImCYviJgmL4iYJi+ImCCtPn97aDHhwcNOtNTU2p\nNa9X7l0H4C0bbl1jAACXLl1KrU2cONE81rsGwZt3bq1zANhjmzRpknms97i2traa9b6+vpKP9ZZq\nz/rviX1+IsoNw08UFMNPFBTDTxQUw08UFMNPFBTDTxSU2+cXkS0AlgM4rqoLktueBvADAP3J3dap\n6puVGmQ5eH1Zb166xevpzpkzx6zPmDHDrHu9+KGhodSa10/21jk4cuSIWfeuQZg1a1bJ57Z+LsBf\nW9/6/t65vbo3X9+7RqGlpcWsV0Mxr/xbASwb5faNqro4+a+mg09EN3LDr6rvAThZhbEQURVl+Z1/\nrYj8QUS2iIi9DhUR1ZxSw/9zAN8EsBhAL4Cfpd1RRNaISJeIdPX396fdjYiqrKTwq2qfql5V1SEA\nmwEsNe67SVULqlpobm4udZxEVGYlhV9E5o748jsA9pRnOERULcW0+l4G8CCAJhHpAfBTAA+KyGIA\nCqAbwA8rOEYiqgA3/Kq6apSbX6zAWCrK68V7c+6tujcvfd++fWZ96dLU35oAAOPG2W/QrGsUvPn8\n3vr0N998s1n31r/3evWWixcvmnXvOWtoaEitedd1eOO21ikA/HX9awGv8CMKiuEnCorhJwqK4ScK\niuEnCorhJwoqzNLdXuvGW6rZag15y1t704mzLq/ttbws3tRVb0qwNzarTemd2/u5vHadtay41571\nllP3Wn1ZnpNq4Ss/UVAMP1FQDD9RUAw/UVAMP1FQDD9RUAw/UVBh+vzeFMssfX5vWuw999xj1r2e\nsXeNgtUv9/rN3jUI3hLVWZYV936uuro6s+4tj33vvfem1qztuwHgwIEDZt27TiDLUvDVwld+oqAY\nfqKgGH6ioBh+oqAYfqKgGH6ioBh+oqDC9Pm9ZaCz9Pm9fvXtt99u1r1eurcVtTW2rP1o73jvOgFr\nPr93jYA3Nu/cTU1NJY0LAA4fPmzWvd2nvGsQagFf+YmCYviJgmL4iYJi+ImCYviJgmL4iYJi+ImC\ncvv8ItIK4CUAcwAMAdikqi+ISCOAXwNoA9AN4BFVPVW5oWaTde18a878qVP2jz1t2jSzfvLkSbPu\nzWs/c+ZMas2bz++tJeBdg+Ctve89rhbvOfNYew54P5c333/hwoVm3fv+taCYV/4rAH6iqncC+GsA\nPxKRuwA8BeAdVW0H8E7yNRGNEW74VbVXVT9NPh8EsBfALQBWANiW3G0bgIcrNUgiKr+v9Tu/iLQB\n6ADwIYAWVe0Fhv8HAWB2uQdHRJVTdPhFpB7AbwD8WFXTf8m88bg1ItIlIl39/f2ljJGIKqCo8ItI\nHYaD/wtV/W1yc5+IzE3qcwEcH+1YVd2kqgVVLXiTIYioetzwy/AWtC8C2KuqG0aUdgBYnXy+GsBr\n5R8eEVVKMVN67wPwPQC7RWRncts6AM8BeEVEvg/gMICVlRlieZw7d86se9NmBwcHU2sHDx40j122\nbJlZ91pa3tLg1rLk3vbhXqvOm5rqHe+1KS3elF6vPmXKlNTakSNHzGO9Kd7ez5W1TVkNbvhV9fcA\n0v4Ffau8wyGiauEVfkRBMfxEQTH8REEx/ERBMfxEQTH8REGFWbrbmvYK+FNPraW/6+vrzWO9epZe\nuMeb0utd3+D1q70+v9Uv98bmbavu9fkbGhpSawMDA+axX375pVn3HpdKPqflwld+oqAYfqKgGH6i\noBh+oqAYfqKgGH6ioBh+oqDC9PlPnz5t1r0+v9XX9eaGe710r1fu9bO97aYt3vbiWZegtsbu9fGt\n+fiAv0ZDe3t7ai3r9Qve2McCvvITBcXwEwXF8BMFxfATBcXwEwXF8BMFxfATBRWmz+/x1sb3rhOw\neGsJeFt8T5o0yaxbvXqvT9/Y2GjWvX62d32Etf24N+e9paXFrO/fv9+sL1myJLU2d+5c89h9+/aZ\n9fPnz5t1zucnoprF8BMFxfATBcXwEwXF8BMFxfATBcXwEwXl9vlFpBXASwDmABgCsElVXxCRpwH8\nAEB/ctd1qvpmpQaalbfPfJb92O+++27z2JUrV5r1vXv3mvVDhw6Z9enTp6fWvPXnvbUAjh07Zta9\nPv+MGTNSa971C7t27TLr3nz/1tbW1FpHR4d57IcffmjWvfn+3joJtaCYi3yuAPiJqn4qItMBfCIi\nbye1jar6z5UbHhFViht+Ve0F0Jt8PigiewHcUumBEVFlfa3f+UWkDUAHgGvvidaKyB9EZIuIzEw5\nZo2IdIlIV39//2h3IaIcFB1+EakH8BsAP1bVMwB+DuCbABZj+J3Bz0Y7TlU3qWpBVQvNzc1lGDIR\nlUNR4ReROgwH/xeq+lsAUNU+Vb2qqkMANgNYWrlhElG5ueEXEQHwIoC9qrphxO0jp0V9B8Ce8g+P\niCqlmL/23wfgewB2i8jO5LZ1AFaJyGIACqAbwA8rMsIy8aZwdnd3l/y9C4VCyccCwPr16836pUuX\nzLo1bddb9tubyuxNXfVahVOnTk2tedNevbF5rUJLZ2enWX/11VdL/t6AP/ZaUMxf+38PQEYp1WxP\nn4h8vMKPKCiGnygohp8oKIafKCiGnygohp8oqDBLdy9fvtysz58/36w/88wzqbXHHnuspDEVy+tn\nZ+l3e2bOHHXKxpjX0NBg1p944gmzvmjRIrPubcteC/jKTxQUw08UFMNPFBTDTxQUw08UFMNPFBTD\nTxSUqGr1TibSD+CLETc1ARio2gC+nlodW62OC+DYSlXOsf2Vqha1Xl5Vw3/DyUW6VDXbShgVUqtj\nq9VxARxbqfIaG9/2EwXF8BMFlXf4N+V8fkutjq1WxwVwbKXKZWy5/s5PRPnJ+5WfiHKSS/hFZJmI\n7BORgyLyVB5jSCMi3SKyW0R2ikhXzmPZIiLHRWTPiNsaReRtETmQfMxlzm3K2J4WkSPJY7dTRP42\np7G1ish/isheEfmjiPx9cnuuj50xrlwet6q/7ReR8QD2A3gIQA+AjwGsUtX/qepAUohIN4CCqube\nExaRBwCcBfCSqi5IbvsnACdV9bnkf5wzVfUfamRsTwM4m/fOzcmGMnNH7iwN4GEAf4ccHztjXI8g\nh8ctj1f+pQAOquohVb0M4FcAVuQwjpqnqu8BOHndzSsAbEs+34bhfzxVlzK2mqCqvar6afL5IIBr\nO0vn+tgZ48pFHuG/BcCfRnzdg9ra8lsB/E5EPhGRNXkPZhQtybbp17ZPn53zeK7n7txcTdftLF0z\nj10pO16XWx7hH233n1pqOdynqksAdAL4UfL2lopT1M7N1TLKztI1odQdr8stj/D3AGgd8fU3ABzN\nYRyjUtWjycfjALaj9nYf7ru2SWry8XjO4/mzWtq5ebSdpVEDj10t7XidR/g/BtAuIvNEZCKA7wLY\nkcM4biAi05I/xEBEpgH4Nmpv9+EdAFYnn68G8FqOY/kLtbJzc9rO0sj5sau1Ha9zucgnaWX8C4Dx\nALao6j9WfRCjEJHbMPxqDwyvbPzLPMcmIi8DeBDDs776APwUwL8DeAXArQAOA1ipqlX/w1vK2B7E\n8FvXP+/cfO137CqP7X4A/wVgN4Ch5OZ1GP79OrfHzhjXKuTwuPEKP6KgeIUfUVAMP1FQDD9RUAw/\nUVAMP1FQDD9RUAw/UVAMP1FQ/wcmD5limwlfIgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7efe3861f940>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "img1 = fashion_mnist.train.images[41].reshape(28,28)\n",
    "# Get corresponding integer label from one-hot encoded data\n",
    "label1 = np.where(fashion_mnist.train.labels[41] == 1)[0][0]\n",
    "# Plot sample\n",
    "print(\"y = {} ({})\".format(label1, labels[label1]))\n",
    "plt.imshow(img1, cmap='Greys')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "That is a pretty good shirt isn't? We look one more image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "y = 8 (Bag)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7efe36dd5b70>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAEs1JREFUeJzt3W2MVGWWB/D/AboBsQWxGxp56R4I\nWTTo6lIYo2Z9mTg6K4kaI4EPhk0m02Mc4prMB41+GBOzETc7M/BhMxFXHEgcdBKGkQ9mHWM2wREz\nUqCC2q4jgjbSNO/QvNPdZz/0ZdJi33PKulV1S87/l5jurlNP1WN1/7nVfe59HlFVEFE8I/KeABHl\ng+EnCorhJwqK4ScKiuEnCorhJwqK4ScKiuEnCorhJwpqVC2frLm5Wdvb22v5lESh7Nq1CwcOHJBS\n7psp/CJyN4AVAEYC+G9VXWbdv729HcViMctTEpGhUCiUfN+y3/aLyEgA/wXgxwCuBrBYRK4u9/GI\nqLay/M5/A4DPVfULVT0L4BUA91ZmWkRUbVnCPxVA15Cvdye3fYOIdIhIUUSK+/fvz/B0RFRJWcI/\n3B8VvnV9sKquVNWCqhZaWloyPB0RVVKW8O8GMH3I19MA7Mk2HSKqlSzh3wxgtoj8QEQaASwCsKEy\n0yKiaiu71aeqfSKyFMAbGGz1rVLVjys2MyKqqkx9flV9HcDrFZoLEdUQT+8lCorhJwqK4ScKiuEn\nCorhJwqK4ScKqqbX8+fJ25lIxL4EemBgILU2YkR1/w09ffq0WZ8/f35qrbe31xw7deq3Lsf4hkmT\nJpn1trY2s75sWfpV3mPGjDHHUnXxyE8UFMNPFBTDTxQUw08UFMNPFBTDTxTURdPq81p5WXmtwCx6\nenrMemtrq1lfvHhxau2OO+4wxx47dsysHzx40KyvWLHCrL/xxhuptc7OTnOsJ2v7Njoe+YmCYviJ\ngmL4iYJi+ImCYviJgmL4iYJi+ImCkmr3x4cqFApar7v0ZnkdsvaTZ8yYYdZHjhxp1nfu3Jnp+avp\n6qvT9259+OGHzbGPPvpopadz0SsUCigWiyX9QPLITxQUw08UFMNPFBTDTxQUw08UFMNPFBTDTxRU\npuv5RWQXgF4A/QD6VLVQiUlVg7X0NuAvv22N9/r869atM+tdXV1mfeHChWbdUs0lywH/dbvllltS\na0899ZQ5ln3+6qrEYh63q+qBCjwOEdUQ3/YTBZU1/ArgzyKyRUQ6KjEhIqqNrG/7b1bVPSIyCcCb\nIvKpqm4ceofkH4UOwD+HnYhqJ9ORX1X3JB/3AVgP4IZh7rNSVQuqWmhpacnydERUQWWHX0TGiUjT\n+c8B/AjAR5WaGBFVV5a3/ZMBrE9aRaMA/F5V/6cisyKiqis7/Kr6BYB/rOBcqsrrZ3v98CzbcK9Z\ns6bssYB/3bulv7/frHtrBWT12GOPpdZeeOEFc+wrr7xi1hctWmTWre9ptdf0z3N9iFKx1UcUFMNP\nFBTDTxQUw08UFMNPFBTDTxTURbNFt8drn2RpiR09etQcu2HDBrP++OOPm/Xbb7/drFtGjcr2Lc7a\ndrKW7r7kkkvMsevXrzfrXqvPmnvWS5VrueR9tfDITxQUw08UFMNPFBTDTxQUw08UFMNPFBTDTxTU\nRdPnz7pEdZZLW3fs2FH2WABYtmxZpvHfV7Nnzzbr3pLnp06dMutjx45NrXl9/L6+PrNezZ+nWuGR\nnygohp8oKIafKCiGnygohp8oKIafKCiGnyioi6bPX+3ljrdv355ae+SRR8yx3nXrt956a9nPDQCH\nDx9OrbW1tZljs65zcOjQIbM+c+bM1NrJkyfNsePHjzfr999/v1lfvnx5am3OnDnm2KzrIHjyXFb8\nPB75iYJi+ImCYviJgmL4iYJi+ImCYviJgmL4iYJym5kisgrAAgD7VHVucttEAK8CaAewC8BCVU1v\nNlfIgQMHUmvPP/+8Odard3V1lTWnUowbN86sb9y4MdPjW+cRnDhxItNjjx492qxfc801Zt1aH7+1\ntdUca32/AeCzzz4z61dddVVqbcGCBebYZ555xqx71+t7r4vVy8+6NkWpSjny/w7A3Rfc9gSAt1R1\nNoC3kq+J6HvEDb+qbgRw4Wlc9wJYnXy+GsB9FZ4XEVVZub/zT1bVbgBIPk6q3JSIqBaq/gc/EekQ\nkaKIFPfv31/tpyOiEpUb/h4RmQIAycd9aXdU1ZWqWlDVQktLS5lPR0SVVm74NwBYkny+BMBrlZkO\nEdWKG34RWQvgXQD/ICK7ReQnAJYBuFNE/gbgzuRrIvoecfv8qro4pfTDCs/F3cd+yZIlqbWjR4+a\nY8eMGWPWm5ubzfrcuXNTa5s3bzbHeq6//nqz7q0HYPWcvbHeOQgTJkww601NTWbd6ll7z/3uu++a\n9e7ubrP+wAMPpNa2bdtmjr3rrrvM+uTJk8360qVLzXpHR0dqjdfzE1FVMfxEQTH8REEx/ERBMfxE\nQTH8REGJd/lgJc2bN0+t9o13+ai1nLK3zLO3TPS0adPM+mWXXZZaO3bsmDnWazN6La8zZ86Ydet7\nOHHixEyP7Z2V6S1xbb1unt7eXrN+9uzZsuvez4t3ObF1qTIAbNq0yaxbP49eq8/6fs+fPx/FYrGk\nXiGP/ERBMfxEQTH8REEx/ERBMfxEQTH8REEx/ERB1XSL7pMnT2Lr1q2p9YaGBnO8denrV199ZY49\ndeqUWT937pxZt5Yg886V8P6/vLl55z9Y5xFceuml5tjLL7/crHtLVHu9dmuLb29sY2OjWffOn7Au\n+fUe+4orrjDrR44cMeve614sFlNr8+fPN8dWCo/8REEx/ERBMfxEQTH8REEx/ERBMfxEQTH8REHV\ntM8/atQoTJqUvq2f12u3tj3eu3dv2fMC/Ov977nnntTajh07zLFeL93bato7T8BaT8C7Lt1betu7\n3t87B8H6nnrLih8/ftyse736mTNnlv3c3jkI3joJ7733nllfu3Ztas3r89dyi24iuggx/ERBMfxE\nQTH8REEx/ERBMfxEQTH8REG5fX4RWQVgAYB9qjo3ue1pAD8FcP4i9ydV9XXvsUaPHm32Xj3WNfve\n9dOeLNtkX3vtteZYry87a9Yss97X12fWrfUAvD0BvLl5z+2dw2CdPzFihH3s8db89/ZasM5/OHjw\noDnWWx/Cm/vs2bPN+vLly1Nrzz33nDnWO++jVKUc+X8H4O5hbv+Nql6X/OcGn4jqixt+Vd0I4FAN\n5kJENZTld/6lIrJNRFaJiP3ej4jqTrnh/y2AWQCuA9AN4FdpdxSRDhEpikjRWgePiGqrrPCrao+q\n9qvqAIAXANxg3HelqhZUteBt+khEtVNW+EVkypAv7wfwUWWmQ0S1Ukqrby2A2wA0i8huAL8EcJuI\nXAdAAewC8LMqzpGIqsANv6ouHubmF6swF9fp06dTa3PmzDHHfvLJJ2Z97NixZr2zszO19uGHH5pj\nFyxYYNa9cx/efvtts97c3Jxas9ZPAPy1CCZPnmzW9+zZY9Z7e3tTa966+6NG2T+eW7ZsMetHjx5N\nrXnnELzzzjtm3TN16lSzbq0HsG7dOnPsokWLyprThXiGH1FQDD9RUAw/UVAMP1FQDD9RUAw/UVA1\nXbq7r68Phw6Vf42QdZllW1ubOda7dHXbtm1m/cEHH0yt3XTTTeZYq90F+O02b7vo9vb21NqJEyfM\nsVb7FAB6enrM+vTp08261a7ztib/+OOPzbq3bLjVTvNONfeWU/fas14r0Zr7yy+/bI5lq4+IMmH4\niYJi+ImCYviJgmL4iYJi+ImCYviJgqppn//cuXPo7u4ue7zVm/WW7r7yyivN+s6dO836Sy+9lFqz\nlogG/K3HvXMQpkyZYtZfffXV1Fpra2umx37//ffN+owZM8y6tXS497PgLRs+fvx4s25t2+4tt37j\njTeade977m0Zb61qtWnTJnNspfDITxQUw08UFMNPFBTDTxQUw08UFMNPFBTDTxRUTfv8Z8+exZdf\nfplpfJpPP/3UHOtdX3348GGzbm3J7G2DnbVfbS1BDdjLc3vXvHtLb3vbh3trFVhbdHvLpXtz987t\nsLYP9869sJZDL8WRI0fMunV+xe7duzM9d6l45CcKiuEnCorhJwqK4ScKiuEnCorhJwqK4ScKyu3z\ni8h0AGsAtAIYALBSVVeIyEQArwJoB7ALwEJVNZvlI0aMcHu3Fqsv7G1j3dTUZNZHjhxp1q3rt72t\npr1+tre2vjc3q+71s/v7+826d/5DFt7cvHX9vfMfrHUSrHNGvLEAMGHCBLPubS9+5syZ1FqWjHwX\npRz5+wD8QlWvAnAjgJ+LyNUAngDwlqrOBvBW8jURfU+44VfVblXdmnzeC6ATwFQA9wJYndxtNYD7\nqjVJIqq87/Q7v4i0A7gewF8BTFbVbmDwHwgA6eeYElHdKTn8InIpgHUAHlNVewGzb47rEJGiiBS9\n852JqHZKCr+INGAw+C+r6h+Tm3tEZEpSnwJg33BjVXWlqhZUteD9kYSIascNvwz+2fNFAJ2q+ush\npQ0AliSfLwHwWuWnR0TVUsolvTcDeAjAdhH5ILntSQDLAPxBRH4C4CsA6XtYJ/r7+3H8+PFy5wpV\nNR/b4l1W621VbT1+Y2OjOdbbltybu3U5MeC3vCxeG9F7bq8l5o3PwmunWc/tvebbt283697W5N5l\n2tYy9N7PYqW44VfVvwBI+w7/sLLTIaJa4Rl+REEx/ERBMfxEQTH8REEx/ERBMfxEQdV06e6BgYFM\nfX6rp9zQ0FD245bCOsfAW77aOw/Au4QzS688zz68x5vbwMCAWffmbp3D4L3m3vkPEydONOveJcPW\n5cxZMvJd8MhPFBTDTxQUw08UFMNPFBTDTxQUw08UFMNPFFRN+/wNDQ2YNm1a2eOt5Y69ZaC9vq3X\nU7auHff6zVmXqPZY/XJvblmv18+6HoDFu+beYz239/32vmfeknTe+hFff/11aq2trc0cWyk88hMF\nxfATBcXwEwXF8BMFxfATBcXwEwXF8BMFVdM+/+jRozP1MK2trr013E+ePFn28wJ+XzgLby0Cr2ec\nZT+DrL30qA4cOGDW582bZ9afffbZ1NpDDz1U1py+Kx75iYJi+ImCYviJgmL4iYJi+ImCYviJgmL4\niYJy+/wiMh3AGgCtAAYArFTVFSLyNICfAji/0fiTqvq69ViNjY1ob29PrVv96qy6urrMurffurU2\n/969e82xs2bNMut5rp1PcZVykk8fgF+o6lYRaQKwRUTeTGq/UdX/rN70iKha3PCrajeA7uTzXhHp\nBDC12hMjour6Tu83RaQdwPUA/prctFREtonIKhG5PGVMh4gURaS4f//+4e5CRDkoOfwicimAdQAe\nU9VjAH4LYBaA6zD4zuBXw41T1ZWqWlDVQktLSwWmTESVUFL4RaQBg8F/WVX/CACq2qOq/ao6AOAF\nADdUb5pEVGlu+GVw+dYXAXSq6q+H3D5lyN3uB/BR5adHRNVSyl/7bwbwEIDtIvJBctuTABaLyHUA\nFMAuAD+rygwrxGvleZqamsqqEdWrUv7a/xcAwy3ebvb0iai+8ewSoqAYfqKgGH6ioBh+oqAYfqKg\nGH6ioBh+oqAYfqKgGH6ioBh+oqAYfqKgGH6ioBh+oqAYfqKgpJrLZX/ryUT2A/hyyE3NAOy9jvNT\nr3Or13kBnFu5Kjm3NlUtab28mob/W08uUlTVQm4TMNTr3Op1XgDnVq685sa3/URBMfxEQeUd/pU5\nP7+lXudWr/MCOLdy5TK3XH/nJ6L85H3kJ6Kc5BJ+EblbRP5PRD4XkSfymEMaEdklIttF5AMRKeY8\nl1Uisk9EPhpy20QReVNE/pZ8HHabtJzm9rSIfJ28dh+IyL/kNLfpIvK/ItIpIh+LyL8lt+f62hnz\nyuV1q/nbfhEZCeAzAHcC2A1gM4DFqvpJTSeSQkR2ASioau49YRH5ZwDHAaxR1bnJbf8B4JCqLkv+\n4bxcVR+vk7k9DeB43js3JxvKTBm6szSA+wD8K3J87Yx5LUQOr1seR/4bAHyuql+o6lkArwC4N4d5\n1D1V3Qjg0AU33wtgdfL5agz+8NRcytzqgqp2q+rW5PNeAOd3ls71tTPmlYs8wj8VQNeQr3ejvrb8\nVgB/FpEtItKR92SGMTnZNv389umTcp7Phdydm2vpgp2l6+a1K2fH60rLI/zD7f5TTy2Hm1X1nwD8\nGMDPk7e3VJqSdm6ulWF2lq4L5e54XWl5hH83gKEb500DsCeHeQxLVfckH/cBWI/623245/wmqcnH\nfTnP5+/qaefm4XaWRh28dvW043Ue4d8MYLaI/EBEGgEsArAhh3l8i4iMS/4QAxEZB+BHqL/dhzcA\nWJJ8vgTAaznO5RvqZefmtJ2lkfNrV287Xudykk/SylgOYCSAVar67zWfxDBEZCYGj/bA4Camv89z\nbiKyFsBtGLzqqwfALwH8CcAfAMwA8BWAB1W15n94S5nbbRh86/r3nZvP/45d47ndAuBtANsBDCQ3\nP4nB369ze+2MeS1GDq8bz/AjCopn+BEFxfATBcXwEwXF8BMFxfATBcXwEwXF8BMFxfATBfX/xkjX\nVRopgAUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7efe38656390>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "img1 = fashion_mnist.train.images[19].reshape(28,28)\n",
    "# Get corresponding integer label from one-hot encoded data\n",
    "label1 = np.where(fashion_mnist.train.labels[19] == 1)[0][0]\n",
    "# Plot sample\n",
    "print(\"y = {} ({})\".format(label1, labels[label1]))\n",
    "plt.imshow(img1, cmap='Greys')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "That's a good bag to hold!! <br><br>So now, we have to build a convolutional neural network which\n",
    "actually classifies all these images into their respective category. We define the\n",
    "placeholders for input images and output labels. As our input image is of size 784, we define\n",
    " placeholder for input x as follows"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x = tf.placeholder(tf.float32, [None, 784])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " We need to reshape the input to the format [p,q,r,s] where q and s are the actual size of\n",
    "an input image which is 28*28 and s is the channel number. As we have only grayscale\n",
    "images, the value of s is 1. p implies the number of training samples i.e batch size. Since we\n",
    "don't know the batch size we can set that as -1 and it will be dynamically changed during\n",
    "training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x_shaped = tf.reshape(x, [-1, 28, 28, 1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we have 10 different labels, we define placeholders for the output y as"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "y = tf.placeholder(tf.float32, [None, 10])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    " Now we define a function called conv2d which actually performs the convolutional\n",
    "operation i.e element wise multiplication of the input matrix (x) by the filter (w) with a\n",
    "stride of 1 and same padding.\n",
    "We set strides = [1,1,1,1]. The first and last values of strides are set to 1 which implies we\n",
    "don't want to move between training samples and different channels. The second and third\n",
    "values of strides are also set to 1 which implies we move the filter by 1 pixel in both height\n",
    "and width direction."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def conv2d(x, W):\n",
    "    return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We define a function called maxpool2d to perform the pooling operation. We perform max\n",
    "pooling with stride of 2 and same padding, ksize implies our pooling window shape."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def maxpool2d(x):\n",
    "    return tf.nn.max_pool(x, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we define weights and biases. We will build a convolutional network with two\n",
    "convolutional layers followed by a one fully connected layer and output layer, so we define the\n",
    "weights for all these layers. The weights are actually the filter in the convolutional layers.\n",
    "So the weight matrix will be initialized as [ filter_shape[0], fliter_shape[1],\n",
    "number_of_input_channel, filter_size ] "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " We will use a 5*5 filter and we set our filter size as 32. Since we use grayscale images, our\n",
    "input channel number will be 1. So our weight matrix of first convolution layer will be [5,5,1,32]. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "w_c1 = tf.Variable(tf.random_normal([5,5,1,32]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As the second convolutional layer takes the input from the first convolutional layer which\n",
    "has 32 channel output, the number of input channel to the next layer becomes 32."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "w_c2 = tf.Variable(tf.random_normal([5,5,32,64]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " Next, we initialize biases"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "b_c1 = tf.Variable(tf.random_normal([32]))\n",
    "b_c2 = tf.Variable(tf.random_normal([64]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " Now we perform operations on the first convolution layer i.e convolution operation on the\n",
    "input x with relu activations followed by a max pooling. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "conv1 = tf.nn.relu(conv2d(x_shaped, w_c1) + b_c1)\n",
    "conv1 = maxpool2d(conv1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " Now, the result of the first convolutional layer will be passed to the next convolutional layer\n",
    "where we perform\n",
    "convolution operation on the result of a first convolutional layer with relu activations\n",
    "followed by max pooling."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "conv2 = tf.nn.relu(conv2d(conv1, w_c2) + b_c2)\n",
    "conv2 = maxpool2d(conv2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After two convolution layers with convolution and pooling operation, our input image will\n",
    "be downsampled from 28 * 28 * 1 to 7 * 7 *1. We need to flatten this output before feeding to the\n",
    "fully connected layer. Then we feed this flattened output to the\n",
    "fully connected layer and we multiply with weights and add bias and apply relu\n",
    "activations. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x_flattened = tf.reshape(conv2, [-1, 7*7*64])\n",
    "w_fc = tf.Variable(tf.random_normal([7*7*64,1024]))\n",
    "b_fc = tf.Variable(tf.random_normal([1024]))\n",
    "fc = tf.nn.relu(tf.matmul(x_flattened,w_fc)+ b_fc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " Now we define the weights and bias for output layer. [number of neurons in the current\n",
    "layer, number of neurons layer in the next layer]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "w_out = tf.Variable(tf.random_normal([1024, 10]))\n",
    "b_out = tf.Variable(tf.random_normal([10]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " We can get output by multiplying the result of a fully connected layer with weight matrix and add bias.\n",
    "We will get the probabilities of the output by softmax activation function. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "output = tf.matmul(fc, w_out)+ b_out\n",
    "yhat = tf.nn.softmax(output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will define our loss function as cross entropy loss and We try to minimize our loss function\n",
    "using a optimizer called adam optimizer instead of gradient descent optimizer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "learning_rate = 0.01\n",
    "cross_entropy = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=output, labels=y))\n",
    "optimiser = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cross_entropy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " Next, we calculate the accuracy as follows,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(yhat, 1))\n",
    "accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "epochs = 10\n",
    "batch_size = 100"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " Now we will start the tensorflow session and build the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "init_op = tf.global_variables_initializer()\n",
    "\n",
    "with tf.Session() as sess:\n",
    "   sess.run(init_op)\n",
    "   total_batch = int(len(fashion_mnist.train.labels) / batch_size)\n",
    "    # For each epcoh\n",
    "   for epoch in range(epochs):\n",
    "        \n",
    "        avg_cost = 0\n",
    "        for i in range(total_batch):\n",
    "            batch_x, batch_y = fashion_mnist.train.next_batch(batch_size=batch_size)\n",
    "            \n",
    "            _, c = sess.run([optimiser, cross_entropy], \n",
    "                         feed_dict={x: batch_x, y: batch_y})\n",
    "            \n",
    "            avg_cost += c / total_batch\n",
    "            \n",
    "        print(\"Epoch:\", (epoch + 1), \"cost =\"\"{:.3f}\".format(avg_cost))\n",
    "        \n",
    "   print(sess.run(accuracy, feed_dict={x: mnist.test.images, y: mnist.test.labels}))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:universe]",
   "language": "python",
   "name": "conda-env-universe-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}