Intro to TensorFlow/Intro to Tensorflow.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Introduction to Tensorflow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In my experience, learning to anything useful in computer science has fallen at the strange intersection of theory and practice.  It's pretty easy to ignore the amount of depth that the lies under some of the things we code.  Machine learning takes that to an extreme. And everyone wants to be a Machine Learning Engineer these days.\n",
    "\n",
    "<img src=\"tensorflowmeme.png\">\n",
    "\n",
    "Elements of Statistical Learning is a fantastic book. If you can get through it you'll know quite a bit, but it doesn't mean much if you're unable to put any of into practice.  Tensorflow is a framework for building, training, and deploying machine learning models.  Keras is a wrapper built on top of Tensorflow making it a bit more accessible, easier, and cleaner to work with.\n",
    "\n",
    "The block of code below is the equivalent of \"Hello World\" in Tensorflow.  Despite being the first step into using Tensorflow, there is a ton to unpack here. So let's get to it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "import tensorflow as tf\n",
    "mnist = tf.keras.datasets.mnist\n",
    "\n",
    "(x_train, y_train),(x_test, y_test) = mnist.load_data()\n",
    "x_train, x_test = x_train / 255.0, x_test / 255.0\n",
    "\n",
    "model = tf.keras.models.Sequential([\n",
    "  tf.keras.layers.Flatten(input_shape=(28, 28)),\n",
    "  tf.keras.layers.Dense(512, activation=tf.nn.relu),\n",
    "  tf.keras.layers.Dropout(0.2),\n",
    "  tf.keras.layers.Dense(10, activation=tf.nn.softmax)\n",
    "])\n",
    "\n",
    "model.compile(optimizer='adam',\n",
    "              loss='sparse_categorical_crossentropy',\n",
    "              metrics=['accuracy'])\n",
    "\n",
    "model.fit(x_train, y_train, epochs=5)\n",
    "model.evaluate(x_test, y_test)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "mnist = tf.keras.datasets.mnist\n",
    "\n",
    "(x_train, y_train),(x_test, y_test) = mnist.load_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The MNIST dataset is composed of handwritten digits. 60,000 training images and 10,000 testing images. A **training set** is a set in which we are given the outcome of the feature we're observing.  A **feature** is a characteristic of an object that can be measured.  A more familiar term for a feature might be variable or attribute.\n",
    "\n",
    "By using the features of objects in the training set, our model will *learn* and then make predictions on similar objects that are new it.\n",
    "\n",
    "The **testing set** is what we will (you guessed it) test the model created through use of the training set.\n",
    "\n",
    "In the lines above, we are telling Tensorflow that we want files from the MNIST data set and we then load data into training and testing variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(60000, 28, 28)\n"
     ]
    }
   ],
   "source": [
    "print(x_train.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We already knew that our training set had 60,000 images. The 28,28 refers to the dimensions of each image, 28 pixels by 28 pixels. So let's explore the data some more."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAATEAAAD8CAYAAAAfZJO2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAF19JREFUeJzt3X2QXFWZx/HvjxATeVsDgRiSaHgJq7wswR3BFcsNurhoqQFfoVaNLkWwNohYlC9ka5WtLbbYXUBhVdawRMBFkOJFI1JGTGGhJSKBxUAML1mMMCYmBigIIpCZefaPvqM9092n70x3T/eZ/D7Wrek+z305XCfPnHvuuecqIjAzy9Vu3a6AmVkrnMTMLGtOYmaWNScxM8uak5iZZc1JzMyy5iRmZllzEjOzrDmJmVnWdp/Ig71M02I6e07kIc12KS/we16KF9XKPv72hD3jyacGS61777oXV0fESa0cr1UtJTFJJwGXAlOA/46IC1PrT2dPjtNbWzmkmSXcHWta3seTTw3y89WvKrXulNmPzkzFJc0DrgFeCQwBKyLiUknnA2cAvytWXR4RtxXbnAecDgwCZ0fE6tQxxp3EJE0BvgKcCPQD90haFRG/HO8+zaz7AhhiqF27GwDOjYj7JO0N3Cvp9iL2xYi4qHplSYcDpwJHAAcCP5R0WEQ0bBq20hI7FtgYEY8VB78eWAw4iZllLAh2Ns4ZY9tXxBZgS/F5h6QNwJzEJouB6yPiReBXkjZSyTV3NdqglY79OcATVd/761VO0lJJayWt3cmLLRzOzCbKUMn/jYWk+cAxwN1F0VmS1klaKWlGUVYqr1RrJYnV6zysmdcnIlZERF9E9E1lWguHM7OJEASDUW4BZg43Uoplab19StoLuAk4JyKeBS4HDgEWUmmpXTy8at0qJbRyOdkPzKv6PhfY3ML+zKxHDKXzRrXtEdGXWkHSVCoJ7NqIuBkgIrZWxa8Abi2+jjmvtNISuwdYIOkgSS+j0hm3qoX9mVkPCGCQKLU0I0nAlcCGiLikqnx21WqnAA8Wn1cBp0qaJukgYAHw89Qxxt0Si4gBSWcBq6kMsVgZEevHuz8z6x1jaIk1czzwYeABSfcXZcuB0yQtpJIzNwFnAkTEekk3ULlBOAAsS92ZhBbHiRXjOm5rZR9m1lsC2Nmmaesj4ifU7+dqmDci4gLggrLHmNAR+2bW+6LkpWKvcBIzs5ECBvPJYU5iZjZSZcR+PpzEzGwUMVi3G6s3OYmZ2QiVjn0nMTPLVGWcmJOYmWVsyC0xM8uVW2JmlrVADGY0c72TmJnV8OWkmWUrEC/FlG5XozQnMTMboTLY1ZeTZpYxd+ybWbYixGC4JWZmGRtyS8zMclXp2M8nNeRTUzObEO7YN7PsDXqcmJnlyiP2zSx7Q747aWa5qjwA7iRmZpkKxE4/dmRmuYrAg13NLGfyYFczy1fglpiZZc4d+2aWrUCeFNHM8lV5ZVs+qSGfmprZBNmFXp4raROwAxgEBiKirx2VsnxM2W/fZFx/tk/D2OPvPTC57QszIxk/9J9/kYwPPf98Mm71BbveiP0TImJ7G/ZjZj1il2mJmdnkE6FdqiUWwA8kBfC1iFjRhjqZWRdVOvZ3nceOjo+IzZIOAG6X9FBE3Fm9gqSlwFKA6ezR4uHMrPPymmO/pZpGxObi5zbgFuDYOuusiIi+iOibyrRWDmdmE6DSsa9SSzOS5km6Q9IGSeslfbIo31fS7ZIeLX7OKMol6TJJGyWtk/S6ZscYdxKTtKekvYc/A28DHhzv/sysdwyyW6mlhAHg3Ih4LfAGYJmkw4HPAWsiYgGwpvgO8HZgQbEsBS5vdoBWLidnAbdIGt7PNyPi+y3sz8x6QDtH7EfEFmBL8XmHpA3AHGAxsKhY7WrgR8Bni/JrIiKAn0l6haTZxX7qGncSi4jHgKPHu731ht2OfE0y/uh5L0/G//6onybj5+63esx1Kuu1sz6ejC/46L0dO/ZkN4YXhcyUtLbq+4pGN/gkzQeOAe4GZg0npojYUvSrQyXBPVG1WX9R1v4kZmaTUwTsHCqdxLaXGeQuaS/gJuCciHi2uIKru2q9KqX27SRmZiNULifbd3dS0lQqCezaiLi5KN46fJkoaTawrSjvB+ZVbT4X2Jzafz73Uc1swgwWz082W5pRpcl1JbAhIi6pCq0ClhSflwDfqSr/SHGX8g3AM6n+MHBLzMxGGR5i0SbHAx8GHpB0f1G2HLgQuEHS6cDjwPuL2G3AO4CNwPPAx5odwEnMzEZp3+VkRPyE+v1cAG+ts34Ay8ZyDCcxM6vhOfZtQun1RzWMbfxU+hm4H73py8n4/lPST1ns1qRb9XvPz2gYe+zFAxrGAJbNeDgZ/8abr0jG/+X1SxrG4p4Hktvuyip3J3edZyfNbJLx9NRmlj1fTppZttp8d7LjnMTMrMauNCmimU0yEWLASczMcubLSTPLlvvEbMym7L9/Mv7IpXOS8e++8asNYwdPndrk6K3Ntvv1Z+cl499+75saxoampeu27Nb0OLG+aYPJ+B9mNZ5GaHpyS3MSM7NseZyYmWXP48TMLFsRMFB+UsSucxIzsxq+nDSzbLlPzMyyF05iZpYzd+zbmPzmQwuS8fV/fWmTPTQbCzZ+/9NsHNjJb0zGBx9+pGFMxxwxrjpZZ0W4T8zMsiYGfXfSzHLmPjEzy5afnTSzvEWlXywXTmJmVsN3J80sW+GOfTPL3aS6nJS0EngnsC0ijizK9gW+BcwHNgEfiIinO1fNyW3Ouzd1bN83PvfKZPySR2pewjzCrM+kf5sHH350zHUa9vRR+4x7W+usnO5OlmkzXgWcNKrsc8CaiFgArCm+m9kkEFFJYmWWXtA0iUXEncBTo4oXA1cXn68GTm5zvcysi4ZCpZZeMN4+sVkRsQUgIrZISr+P3syyMqn6xFolaSmwFGA6e3T6cGbWokAMZXR3crw13SppNkDxc1ujFSNiRUT0RUTf1BZfSmFmEyNKLr1gvElsFbCk+LwE+E57qmNmXTfZOvYlXQfcBfy5pH5JpwMXAidKehQ4sfhuZpNFRk2xpn1iEXFag1B6gJGVd0b6MvvwZZ9Ixufd3vj9i3uu/21y25m/bjzfF0D6zY6teX5Wb/wlt1rtamU1GGd6PnAG8LtiteURcVsROw84ncqv3tkRsbrZMTxi38xGCGBoqG1/YK4CvgxcM6r8ixFxUXWBpMOBU4EjgAOBH0o6LCKSf0vzuQVhZhMjgFC5pdmu6o8zbWQxcH1EvBgRvwI2Asc228hJzMxqRJRbWnCWpHWSVkqaUZTNAZ6oWqe/KEtyEjOzWuU79mdKWlu1LC2x98uBQ4CFwBbg4qK8XtOuaap0n5iZjTKm4RPbI6JvLHuPiK1/PJJ0BXBr8bUfqH4zzVxgc7P9uSVmZrU6OMRieKB84RTgweLzKuBUSdMkHQQsAH7ebH9uifWAwY2/SsYP/VQ6njIw7i07b+frd3S7ClZPQLTp7mQxznQRlcvOfuALwCJJCytHYhNwJkBErJd0A/BLKr+6y5rdmQQnMTOrqz1JrME40ysT618AXDCWYziJmVmtHhmNX4aTmJnVchIzs2wND3bNhJOYmdXwpIhmlrf2PTvZcU5iZlZDbolZLh7//BuT8YE9mvw2N/uDndj8PQvuarJx2ln9i5Lxl3//voaxjP6NTrwemiusDCcxMxul3AwVvcJJzMxquSVmZlkb6nYFynMSM7ORPE7MzHLnu5NmlreMkpjnEzOzrLklloEp++yTjL9w7IKGsannbW0YA1j3mv8cV53+uH9NScZ3Np8OqqE7/rBHMt6/9FXJeAxsGPexd3W+nDSzfAV+7MjMMueWmJnlzJeTZpY3JzEzy5qTmJnlSuHLSTPL3WS6OylpJfBOYFtEHFmUnQ+cAfyuWG15RNzWqUrmTtOmJeMv/fVRyfinvvqNZPyEl69pGNs6+GJy2zv+MCMZ//wji5Px6464Khk/cPf0f3vK9N12JuOPfeAVyfjBD09vGBt64YVx1WlXkVNLrMyI/auAk+qUfzEiFhaLE5jZZNLBN4C3W9OWWETcKWl+56tiZj0hsz6xVp6dPEvSOkkrJaWvScwsLxm1xMabxC4HDgEWAluAixutKGmppLWS1u4k3T9jZr1BQ+WWXjCuJBYRWyNiMCKGgCuAYxPrroiIvojom8r4O3nNzOoZVxKTNLvq6ynAg+2pjpn1hIwuJ8sMsbgOWATMlNQPfAFYJGkhlf+MTcCZHayjmU2kzDr2y9ydPK1O8ZUdqEu2dpveeDwSwJMfPCYZ//G/XtbS8Y+47hMNY3PvSM/nNe179yTj+81+Lhm/bvVfJuPn7jf+Rvpx09LjxNZ9NH3e/uqJsxvGZl3zi+S2Q88/n4xPepMpiZnZLshJzMxyJXrnzmMZTmJmNlJmfWJ+UYiZ1WrT3cliMPw2SQ9Wle0r6XZJjxY/ZxTlknSZpI3FQPrXlamqk5iZ1WrfEIurqH32+nPAmohYAKwpvgO8HVhQLEupDKpvyknMzGoMzynWbGkmIu4EnhpVvBi4uvh8NXByVfk1UfEz4BWjxqTW5T6xklLT6Tx0yV8kt31ocWtDKBY/fHIyfth/PNYwNrh1W3Lb3efNTcaPXvV4Mv7p/X6ZjD8z9FLD2HE3nZvcdvZr0nVfc9S3kvG7/qnxef/gae9Mbrv9svT0SNOfTA//aGbKj+5rafuO62yf2KyI2AIQEVskHVCUzwGeqFqvvyjbktqZk5iZjRRjujs5U9Laqu8rImLFOI9cbybGpunUSczMapVviW2PiL4x7n2rpNlFK2w2MNzk7gfmVa03F9jcbGfuEzOzGu3qE2tgFbCk+LwE+E5V+UeKu5RvAJ4ZvuxMcUvMzGq1qU+swbPXFwI3SDodeBx4f7H6bcA7gI3A88DHyhzDSczMRmrjDBUNnr0GeGuddQNYNtZjOImZ2QgirxH7TmJmVsNJLEPaPX0qHv7S0Q1jD737K8lt+wfS03K/+2ufScbnr/y/ZHwgMRZs59+kp8o58t/+Nxn/wgH3JuNff/bVyfg3/vFdDWOH3vyz5LZTZu6XjC86sfEURAC//+AzDWO3HHNFctu5l7U2C/Gtv0/XfcVhB7e0/45zEjOzrDmJmVm2MpvFwknMzGo5iZlZzjwpopllzZeTZpavHnodWxlOYmZWy0ksP098uuFLzAF46N2XNoxtbjIO7P0XfjoZn//txvOBATz1loOS8fjQ3g1jNx7ZuN4A+09Jj4c64vr0WKzDVmxPxvd4+O5kPGVw+5PJ+D7XNYs3jr3vH9Jj82a979fJeFPnvqLJCutb238HecS+mWVPQ/lkMScxMxvJfWJmljtfTppZ3pzEzCxnbomZWd6cxMwsW2N721HXNU1ikuYB1wCvBIaovJLpUkn7At8C5gObgA9ExNOdq2pnXX7GV8e97fR6L5qq8q6P35mMzzk7fdqW7PPdsVapSpNxYN88Oxk/9Lx7kvHBgYEx16gXHPDVnybjMf5fh8JvWt1B1+Q2TqzM244GgHMj4rXAG4Blkg6n8avIzSx3EeWWHtA0iUXEloi4r/i8A9hA5a28jV5FbmaZ6/Ar29pqTH1ikuYDxwB30/hV5GaWs8k62FXSXsBNwDkR8azUpCPoT9stBZYCTGeP8dTRzCZYTh37pd4ALmkqlQR2bUTcXBRvLV5BzqhXkY8QESsioi8i+qY26WQ2s96goXJLL2iaxFRpcl0JbIiIS6pCjV5FbmY5C7Lq2C9zOXk88GHgAUn3F2XLafwq8izd+dxrkvHjpj3QMLZvk+lsls+8Pxlv5p0PvScZf/yuuQ1jB9/Y+LVlAIeuT7+SLTIdQmGt6ZVO+zKaJrGI+AmVoSP11LyK3MwmgcmUxMxs15LbYFcnMTMbKcKTIppZ5vLJYU5iZlbLl5Nmlq8AfDlpZlnLJ4c5iQ376QkHJuPH/d1bGsaeOfql5La7/25qMn7Yf6Wnbdn9t3Ufhvij+S880TDWI4OqLTO+nDSzrLXz7qSkTcAOYBAYiIi+ds5HWOrZSTPbhcQYlvJOiIiFEdFXfG/bfIROYmY2QmWwa5RaWtC2+QidxMys1lDJBWZKWlu1LK2ztwB+IOneqviI+QiBcc9H6D4xM6sxhlbW9qpLxEaOj4jNxcSpt0t6qLXajeSWmJmN1OY+sYjYXPzcBtwCHEvJ+QjLcBIzs1Eqz06WWZqRtKekvYc/A28DHqSN8xH6crIw+ORTyfisyxq/4mtWi8f2jF3Wc9o34eEs4JZiOvvdgW9GxPcl3UOb5iN0EjOzkdr48tyIeAw4uk75k7RpPkInMTOr1SNTT5fhJGZmtfLJYU5iZlZLQ/k8deskZmYjBVnNHOAkZmYjiJYfKZpQTmJmVstJzMyy5iRmZtlyn5iZ5c53J80sY+HLSTPLWOAkZmaZy+dq0knMzGp5nJiZ5S2jJNZ0UkRJ8yTdIWmDpPWSPlmUny/pN5LuL5Z3dL66ZtZxETA4VG7pAWVaYgPAuRFxXzFD472Sbi9iX4yIizpXPTPrioxaYk2TWPEmkuG3kuyQtAGY0+mKmVkXZZTExjTHvqT5wDHA3UXRWZLWSVopaUaDbZYOv85pJy+2VFkzmwABDEW5pQeUTmKS9gJuAs6JiGeBy4FDgIVUWmoX19suIlZERF9E9E1lWhuqbGadFRBD5ZYeUOrupKSpVBLYtRFxM0BEbK2KXwHc2pEamtnECnqm076MMncnBVwJbIiIS6rKZ1etdgqV1zCZ2WQQUW7pAWVaYscDHwYekHR/UbYcOE3SQip5exNwZkdqaGYTr0cSVBll7k7+BFCd0G3tr46ZdV/vtLLK8Ih9MxspAE/FY2ZZc0vMzPIVWd2ddBIzs5ECokfGgJXhJGZmtXpkNH4ZTmJmVst9YmaWrQjfnTSzzLklZmb5CmJwsNuVKM1JzMxGGp6KJxNjmk/MzHYRbZyKR9JJkh6WtFHS59pdVbfEzGyEAKJNLTFJU4CvACcC/cA9klZFxC/bcgDcEjOz0aKtkyIeC2yMiMci4iXgemBxO6vrlpiZ1Whjx/4c4Imq7/3Ace3aOUxwEtvB09t/GDf+uqpoJrB9IuswBr1at16tF7hu49XOur261R3s4OnVP4wbZ5ZcfbqktVXfV0TEiqrv9abxautdgwlNYhGxf/V3SWsjom8i61BWr9atV+sFrtt49VrdIuKkNu6uH5hX9X0usLmN+3efmJl11D3AAkkHSXoZcCqwqp0HcJ+YmXVMRAxIOgtYDUwBVkbE+nYeo9tJbEXzVbqmV+vWq/UC1228erluLYuI2+jgdPaKjJ6RMjMbzX1iZpa1riSxTj+G0ApJmyQ9IOn+UbeOu1GXlZK2SXqwqmxfSbdLerT4OaOH6na+pN8U5+5+Se/oUt3mSbpD0gZJ6yV9sijv6rlL1KsnzluuJvxysngM4RGqHkMATmvnYwitkLQJ6IuIro8pkvRm4Dngmog4sij7d+CpiLiw+AMwIyI+2yN1Ox94LiIumuj6jKrbbGB2RNwnaW/gXuBk4KN08dwl6vUBeuC85aobLbGOP4YwWUTEncBTo4oXA1cXn6+m8o9gwjWoW0+IiC0RcV/xeQewgcrI8a6eu0S9rAXdSGL1HkPopf8jA/iBpHslLe12ZeqYFRFboPKPAjigy/UZ7SxJ64rLza5c6laTNB84BribHjp3o+oFPXbectKNJNbxxxBadHxEvA54O7CsuGyyci4HDgEWAluAi7tZGUl7ATcB50TEs92sS7U69eqp85abbiSxjj+G0IqI2Fz83AbcQuXyt5dsLfpWhvtYtnW5Pn8UEVsjYjAq7/u6gi6eO0lTqSSKayPi5qK46+euXr166bzlqBtJrOOPIYyXpD2LDlck7Qm8DXgwvdWEWwUsKT4vAb7TxbqMMJwgCqfQpXMnScCVwIaIuKQq1NVz16hevXLectWVwa7FLeQv8afHEC6Y8ErUIelgKq0vqDzN8M1u1k3SdcAiKrMcbAW+AHwbuAF4FfA48P6ImPAO9gZ1W0TlkiiATcCZw31QE1y3NwE/Bh4Ahie9Wk6l/6lr5y5Rr9PogfOWK4/YN7OsecS+mWXNSczMsuYkZmZZcxIzs6w5iZlZ1pzEzCxrTmJmljUnMTPL2v8DljTucDXJ8UwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "plt.figure()\n",
    "plt.imshow(x_train[5])\n",
    "plt.colorbar()\n",
    "plt.grid(False)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For any given pixel of the image, the value assigned to that pixel can fall between 0 and 255. Which brings us to our next line of code:\n",
    "\n",
    "```python\n",
    "x_train, x_test = x_train / 255.0, x_test / 255.0\n",
    "```\n",
    "\n",
    "Which standardizes all of the values in our training and testing sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From C:\\Users\\Marlon\\AppData\\Roaming\\Python\\Python36\\site-packages\\tensorflow\\python\\keras\\_impl\\keras\\backend.py:1557: calling reduce_mean (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "keep_dims is deprecated, use keepdims instead\n",
      "WARNING:tensorflow:From C:\\Users\\Marlon\\AppData\\Roaming\\Python\\Python36\\site-packages\\tensorflow\\python\\keras\\_impl\\keras\\backend.py:1422: calling reduce_max (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "keep_dims is deprecated, use keepdims instead\n"
     ]
    }
   ],
   "source": [
    "model = tf.keras.models.Sequential([\n",
    "  tf.keras.layers.Flatten(input_shape=(28, 28)),\n",
    "  tf.keras.layers.Dense(512, activation=tf.nn.relu),\n",
    "  tf.keras.layers.Dropout(0.2),\n",
    "  tf.keras.layers.Dense(10, activation=tf.nn.softmax)\n",
    "])\n",
    "\n",
    "model.compile(optimizer='adam',\n",
    "              loss='sparse_categorical_crossentropy',\n",
    "              metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Okay, time for the cool stuff. \n",
    "\n",
    "Our first line assigns the model that we're using.\n",
    "```python\n",
    "tf.keras.models.Sequential\n",
    "```\n",
    "A sequential model can be thought of like a stack of pancakes.  Each layer is a pancake.  Since we're working our way to being fancy chefs, cooking up machine learning models, our pancakes have fillings.  The fillings try to be as good, if not better than their ingredients.  Our ingredients here are our training set.  \n",
    "The layers that follow the first can automatically recognize that the input shape of the model is sequential, which is why this done only once.\n",
    "\n",
    "```python\n",
    "tf.keras.layers.Flatten(input_shape=(28, 28)),\n",
    "```\n",
    "We're now taking the image being fed in and \"flattening\" it into a 2D array.  The input shape is the dimension of the image being fed into the layer.  Think of this as reformating the image for the model.\n",
    "\n",
    "```python\n",
    "tf.keras.layers.Dense(512, activation=tf.nn.relu)\n",
    "```\n",
    "This creates a densely connected neural layer.  \n",
    "Each input node in the layer is connected to an output node.  It received input from the previous layer, which is why it is *dense*.  The 512 represents the dimesionality of the output space, which at first glance means very little to most. An answer for what that means is a much deeper dive into neural networks than this notebook.  \n",
    "An activation function takes the input of a node in the network and generates the output that will be passed to the next layer.  Without a nonlinear activation function, this would just be a linear regression model.  The type of activation function used here is the RELU or Rectified Linear Unit, which is one of the most common activation functions used.\n",
    "\n",
    "```python\n",
    "tf.keras.layers.Dropout(0.2),\n",
    "```\n",
    "The dropout layer is a regularization technique where randomly selected neurons are ignored in training.  Here we have a 20% rate set, which will drop one of every five inputs for the next training cycle.\n",
    "\n",
    "```python\n",
    "tf.keras.layers.Dense(10, activation=tf.nn.softmax)\n",
    "```\n",
    "Similary to the RELU layer above, this layer uses a Softmax activation function.  The output of the Softmax activation function is similar to a categorial probability distribution, so it tells the probability of a class being true.\n",
    "\n",
    "```python\n",
    "model.compile(optimizer='adam',\n",
    "              loss='sparse_categorical_crossentropy',\n",
    "              metrics=['accuracy'])\n",
    "```\n",
    "\n",
    "Now we compile the model.  \n",
    "First of all, who the hell is Adam?  Not a person. Apparently it was derived from \"adaptive moment estimation\". A guy named Adam should have came up with this. This would be a better paragraph.  \n",
    "Anyway, the optimizer does exactly what it sounds like it does.  The Adam optimization algorithm is used to update network weights based on the training data.  \n",
    "The loss function is used to measure the variation between the value being predicted and what it actually is.  The goal is to minimize the loss function.\n",
    "\n",
    "```python\n",
    "model.fit(x_train, y_train, epochs=5)\n",
    "model.evaluate(x_test, y_test)\n",
    "```\n",
    "Finally we fit and evaluate our model.  Something to pay attention to is that an *epoch* is not the same thing as an iteration.  An epoch is a complete pass of the training data.  Having a model run for too few epochs would leave much to be desired in terms of its performance. Having a model run for too many epochs would risk it starting to \"remember\" the outputs it may want and therefore compromise its accuracy.\n",
    "\n",
    "The training of this model isn't great by any means, but this was just from the Tensorflow tutorial.\n",
    "\n",
    "And there you have it! A first go at Tensorflow.  Although it was a few lines of code, there is still a lot left to unpack but this would just turn into a book.  If you read all of this, I hope you learned something!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/5\n",
      "60000/60000 [==============================]60000/60000 [==============================] - 8s 126us/step - loss: 9.9728 - acc: 0.3813\n",
      "\n",
      "Epoch 2/5\n",
      "60000/60000 [==============================]60000/60000 [==============================] - 8s 127us/step - loss: 10.1151 - acc: 0.3724\n",
      "\n",
      "Epoch 3/5\n",
      "60000/60000 [==============================]60000/60000 [==============================] - 8s 127us/step - loss: 10.0101 - acc: 0.3789\n",
      "\n",
      "Epoch 4/5\n",
      "60000/60000 [==============================]60000/60000 [==============================] - 8s 126us/step - loss: 10.0149 - acc: 0.3786\n",
      "\n",
      "Epoch 5/5\n",
      "60000/60000 [==============================]60000/60000 [==============================] - 8s 127us/step - loss: 10.0893 - acc: 0.3740\n",
      "\n",
      "10000/10000 [==============================]10000/10000 [==============================] - 0s 32us/step\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[10.388112817382812, 0.3555]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.fit(x_train, y_train, epochs=5)\n",
    "model.evaluate(x_test, y_test)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}