CNN.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# CNN (Convolutional Neural Network)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Run the comment below only when using Google Colab\n",
    "# !pip install torch torchvision"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Google Colab에서 실행할 경우 위 주석을 해제하고 실행하면 됨. Colab에는 pytorch가 설치되어있지 않으므로 설치 필요."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import torch\n",
    "import torchvision\n",
    "import torch.nn as nn"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**torch**는 pytorch의 가장 상위에 있는 package로써 라이브러리에서 이것저것 불러오기 위해 import\n",
    "\n",
    "**torch.nn**은 모델을 정의할 때 사용하는 Class들을 포함하고 있음.\n",
    "\n",
    "**torchvision**은 Computer vision에서 사용하는 각종 테크닉들을 torch와 연동하여 구현한 라이브러리."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from torch.utils.data import DataLoader\n",
    "from torchvision import datasets\n",
    "from torchvision import transforms"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import datetime\n",
    "import os, sys"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**numpy**는 python으로 data science를 할 때 가장 기본이 되는 라이브러리 중 하나. 각종 행렬 연산에 필요한 함수들을 다수 포함하고 있음.\n",
    "**datetime** 시간찍을려고...\n",
    "**os, sys** os로부터 정보를 얻고 싶을 때 쓰는 라이브러리"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from matplotlib.pyplot import imshow, imsave\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**matplotlib**는 python visualization library"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "MODEL_NAME = 'CNN'\n",
    "DEVICE = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "GPU가 있다면 GPU를 통해 학습을 가속화하고, 없으면 CPU로 학습하기 위해 device를 정해준다.\n",
    "\n",
    "**torch.cuda.is_avaliable()**은 GPU가 사용가능한지를 판단하는 함수"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class HelloCNN(nn.Module):\n",
    "    \"\"\"\n",
    "        Simple CNN Clssifier\n",
    "    \"\"\"\n",
    "    def __init__(self, num_classes=10):\n",
    "        super(HelloCNN, self).__init__()\n",
    "        self.conv = nn.Sequential(\n",
    "            # (N, 1, 28, 28)\n",
    "            nn.Conv2d(1, 32, kernel_size=3, stride=1, padding=1),\n",
    "            nn.ReLU(),\n",
    "            nn.MaxPool2d(2, 2),\n",
    "            # (N, 32, 14, 14)\n",
    "            nn.Conv2d(32, 64, kernel_size=3, stride=1, padding=1),\n",
    "            nn.ReLU(),\n",
    "            nn.MaxPool2d(2, 2),\n",
    "            # (N, 64, 7, 7)\n",
    "        )\n",
    "        self.fc = nn.Sequential(\n",
    "            nn.Linear(7*7*64, 512),\n",
    "            nn.Dropout(p=0.5),\n",
    "            nn.Linear(512, num_classes),\n",
    "        )\n",
    "    \n",
    "    def forward(self, x):\n",
    "        y_ = self.conv(x) # (N, 64, 7, 7)\n",
    "        y_ = y_.view(y_.size(0), -1) # (N, 64*7*7)\n",
    "        y_ = self.fc(y_)\n",
    "        return y_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 모델 정의를 위한 코드\n",
    "\n",
    "**nn.Module**: 뉴럴넷 구현을 위한 base class. forward, parameter 등 모델을 만들고 사용할 때 필요한 부분들이 내부적으로 구현되어 있음.\n",
    "\n",
    "**__init__**: python class의 constructor. 필요한 멤버변수들을 초기화하고, **nn.Sequential** or **nn.ModuleList**를 이용하여 모델의 구성을 정의한다.\n",
    "\n",
    "**forward**: 모델의 input을 받고 output을 return하는 함수.\n",
    "\n",
    "**nn.Conv2d**: Convolutional Layer. 입력 채널과 출력 채널을 parameter로 받는다. kernel size가 3이므로 padding 1을 통해 같은 크기가 나오도록 한다.\n",
    "\n",
    "**nn.MaxPool2d**: max pooling을 수행한다. feature map size를 줄여주는 역할.\n",
    "\n",
    "**nn.Dropout**: Dropout. p는 drop probability"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model = HelloCNN().to(DEVICE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "정의한 모델을 메모리에 올리는 작업."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "transform = transforms.Compose([transforms.ToTensor(),\n",
    "                                transforms.Normalize(mean=(0.5, 0.5, 0.5),\n",
    "                                std=(0.5, 0.5, 0.5))]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**transforms** torchvision에서 제공하는 transform 함수들이 있는 패키지.\n",
    "\n",
    "**ToTensor**는 numpy array를 torch tensor로 변환.\n",
    "\n",
    "**Normalize**는 다음과 같이 계산함. input[channel] = (input[channel] - mean[channel]) / std[channel]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "mnist_train = datasets.MNIST(root='../data/', train=True, transform=transform, download=True)\n",
    "mnist_test = datasets.MNIST(root='../data/', train=False, transform=transform, download=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([1, 28, 28])"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mnist_train[0][0].shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**datasets**에는 여러 데이터들에 대해 다운로드하고 처리하는 클래스가 내장되어 있음. [참고](https://pytorch.org/docs/stable/torchvision/datasets.html)\n",
    "\n",
    "root 폴더에 없을 시에 download하고, 앞서 정의한 transform에 따라 전처리 된 데이터를 return함."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "batch_size = 64"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train_loader = DataLoader(dataset=mnist_train, batch_size=batch_size, shuffle=True, drop_last=True)\n",
    "test_loader = DataLoader(dataset=mnist_test, batch_size=100, shuffle=False, drop_last=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**DataLoader**는 pytorch에서 학습 시에 데이터를 배치 사이즈만큼씩 효율적으로 불러오도록 돕는 클래스. 잘 사용할수록 GPU의 사용률이 올라간다.\n",
    "\n",
    "**shuffle**: every epochs 마다 데이터의 순서를 랜덤하게 섞는다.\n",
    "\n",
    "**drop_last**: 데이터의 개수가 배치 사이즈로 나눠떨어지지 않는 경우, 마지막 배치를 버린다. 주로 학습시에만 사용."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "criterion = nn.CrossEntropyLoss()\n",
    "optim = torch.optim.Adam(model.parameters(), lr=0.001)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**nn.CrossEntropyLoss**: Cross entropy를 계산하는 Loss. softmax가 내부적으로 수행된다.\n",
    "\n",
    "**optim.Adam**: optim에는 여러 optimizer가 있고, Adam Optimizer는 대표적으로 많이 사용된다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "max_epoch = 5\n",
    "step = 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "최대 epoch 조정"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training\n",
    "\n",
    "첫번째 for문: 원하는 epoch만큼 반복\n",
    "\n",
    "두번째 for문: training datset에서 배치 사이즈 만큼씩 모두 샘플링 될 때까지 반복.\n",
    "\n",
    "**Line 2**: MNIST dataset은 DataLoader를 통해 image와 label을 return.\n",
    "\n",
    "**Line 4**: 각각 Device에 올린다 (GPU or CPU)\n",
    "\n",
    "**Line 5**: 모델에 이미지를 넣고 forward propagation 한다.\n",
    "\n",
    "**Line 7**: 결과값 y_hat과 실제 정답 y에 대한 loss를 계산한다.\n",
    "\n",
    "**zero_grad (Line 9)**: 모델의 gradient를 0으로 초기화한다.\n",
    "\n",
    "**backward (Line 10)**: loss를 계산하는 것까지 연결되어있는 graph를 따라 gradient를 계산한다.\n",
    "\n",
    "**step (Line 11)**: 계산된 gradient를 모두 parameter에 적용한다.\n",
    "\n",
    "**eval (Line 17)**: 모델을 evaluation mode로 바꿔준다 (dropout 조정, Batch normalization 조정 등)\n",
    "\n",
    "**torch.no_grad (Line 19)**: gradient를 계산하기 위해 추적하는 수고를 하지 않음\n",
    "\n",
    "**torch.max (Line 24)**: max value와 indices(즉, argmax)를 return.\n",
    "\n",
    "**train (Line 29)**: evaluation mode였던 모델을 train mode로 전환"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0/5, Step: 0, Loss: 2.318506956100464\n",
      "******************** Test ********************\n",
      "Step: 0, Loss: 2.70082426071167, Accuracy: 10.040000000000001 %\n",
      "**********************************************\n",
      "Epoch: 0/5, Step: 500, Loss: 0.03119628131389618\n",
      "Epoch: 1/5, Step: 1000, Loss: 0.07218557596206665\n",
      "******************** Test ********************\n",
      "Step: 1000, Loss: 0.010657086037099361, Accuracy: 98.33 %\n",
      "**********************************************\n",
      "Epoch: 1/5, Step: 1500, Loss: 0.04745262861251831\n",
      "Epoch: 2/5, Step: 2000, Loss: 0.02198915183544159\n",
      "******************** Test ********************\n",
      "Step: 2000, Loss: 0.021296314895153046, Accuracy: 98.8 %\n",
      "**********************************************\n",
      "Epoch: 2/5, Step: 2500, Loss: 0.010919615626335144\n",
      "Epoch: 3/5, Step: 3000, Loss: 0.0021269842982292175\n",
      "******************** Test ********************\n",
      "Step: 3000, Loss: 0.0012928677024319768, Accuracy: 99.0 %\n",
      "**********************************************\n",
      "Epoch: 3/5, Step: 3500, Loss: 0.006792597472667694\n",
      "Epoch: 4/5, Step: 4000, Loss: 0.019725322723388672\n",
      "******************** Test ********************\n",
      "Step: 4000, Loss: 0.017069529742002487, Accuracy: 98.88 %\n",
      "**********************************************\n",
      "Epoch: 4/5, Step: 4500, Loss: 0.001238018274307251\n"
     ]
    }
   ],
   "source": [
    "for epoch in range(max_epoch):\n",
    "    for idx, (images, labels) in enumerate(train_loader):\n",
    "        # Training Discriminator\n",
    "        x, y = images.to(DEVICE), labels.to(DEVICE) # (N, 1, 28, 28), (N, )\n",
    "        y_hat = model(x) # (N, 10)\n",
    "        \n",
    "        loss = criterion(y_hat, y)\n",
    "        \n",
    "        optim.zero_grad()\n",
    "        loss.backward()\n",
    "        optim.step()\n",
    "        \n",
    "        if step % 500 == 0:\n",
    "            print('Epoch: {}/{}, Step: {}, Loss: {}'.format(epoch, max_epoch, step, loss.item()))\n",
    "        \n",
    "        if step % 1000 == 0:\n",
    "            model.eval()\n",
    "            acc = 0.\n",
    "            with torch.no_grad():\n",
    "                for idx, (images, labels) in enumerate(test_loader):\n",
    "                    x, y = images.to(DEVICE), labels.to(DEVICE) # (N, 1, 28, 28), (N, )\n",
    "                    y_hat = model(x) # (N, 10)\n",
    "                    loss = criterion(y_hat, y)\n",
    "                    _, indices = torch.max(y_hat, dim=-1)\n",
    "                    acc += torch.sum(indices == y).item()\n",
    "            print('*'*20, 'Test', '*'*20)\n",
    "            print('Step: {}, Loss: {}, Accuracy: {} %'.format(step, loss.item(), acc/len(mnist_test)*100))\n",
    "            print('*'*46)\n",
    "            model.train()\n",
    "        step += 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test and Visualize"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "******************** Test ********************\n",
      "Step: 4685, Loss: 0.04823339357972145, Accuracy: 98.99 %\n",
      "**********************************************\n"
     ]
    }
   ],
   "source": [
    "# Test\n",
    "model.eval()\n",
    "acc = 0.\n",
    "with torch.no_grad():\n",
    "    for idx, (images, labels) in enumerate(test_loader):\n",
    "        x, y = images.to(DEVICE), labels.to(DEVICE) # (N, 1, 28, 28), (N, )\n",
    "        y_hat = model(x) # (N, 10)\n",
    "        loss = criterion(y_hat, y)\n",
    "        _, indices = torch.max(y_hat, dim=-1)\n",
    "        acc += torch.sum(indices == y).item()\n",
    "print('*'*20, 'Test', '*'*20)\n",
    "print('Step: {}, Loss: {}, Accuracy: {} %'.format(step, loss.item(), acc/len(mnist_test)*100))\n",
    "print('*'*46)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(torch.Size([1, 28, 28]), tensor(5))"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "idx = 7777 # 0 to 9999\n",
    "img, y = mnist_test[idx]\n",
    "img.shape, y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x2565434a9e8>"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADeNJREFUeJzt3X+MVfWZx/HPg4BGSiKs6YRQs4jBTdBYTCawiWTTdRXQ\nNIH+oRYTQyPpoFJc4mo0+oeYzUbTUKvyB8kYScFQCok/wCrbICFl12waRqP4qy2W0JQJDioGRBNw\n8Nk/5tAMMPd7L/eeX+PzfiWTufc899zz5GY+c86933Pu19xdAOIZU3UDAKpB+IGgCD8QFOEHgiL8\nQFCEHwiK8ANBEX4gKMIPBDW2zI2ZGacTAgVzd2vlcR3t+c1sgZn9ycw+MrOHOnkuAOWyds/tN7ML\nJP1Z0o2SDkraI2mxu3+QWIc9P1CwMvb8syV95O773f2kpN9IWtjB8wEoUSfhnyrpb8PuH8yWncHM\nesysz8z6OtgWgJwV/oGfu/dK6pU47AfqpJM9f7+ky4bd/162DMAo0En490iaYWaXm9l4ST+WtC2f\ntgAUre3DfncfNLOfSfqdpAskrXP393PrDECh2h7qa2tjvOcHClfKST4ARi/CDwRF+IGgCD8QFOEH\ngiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBTh\nB4IqdYpuFGPcuHENaxdddFGh277hhhuS9fnz5zesLVu2LO92zrBu3bqGtYGBgeS6e/bsSdZfe+21\nZP3EiRPJeh2w5weCIvxAUIQfCIrwA0ERfiAowg8ERfiBoDqapdfMDkj6QtIpSYPu3t3k8czS24a5\nc+cm66tWrWpYu/766zvatll6wtcyZ3muk8cffzxZf+SRR0rq5FytztKbx0k+/+run+bwPABKxGE/\nEFSn4XdJr5vZm2bWk0dDAMrR6WH/XHfvN7PvStphZn90993DH5D9U+AfA1AzHe353b0/+31Y0kuS\nZo/wmF537272YSCAcrUdfjObYGYTT9+WNE/Se3k1BqBYnRz2d0l6KRsKGivp1+7+37l0BaBwbYff\n3fdL+n6OvYR1xRVXJOtr165N1q+66qo824Gko0ePJuuvvPJKSZ0Uh6E+ICjCDwRF+IGgCD8QFOEH\ngiL8QFAdXdJ73hvjkt623HLLLcn65s2bC9t2nS/pPXbsWLK+f//+tp97xYoVyfobb7zR9nMXrdVL\netnzA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQTNFdA5MnT07Wly5dWlIn+du3b1/D2jvvvJNcd8eO\nHcn6J598kqy//PLLyXp07PmBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjG+Wtgzpw5yfq8efNK6uRc\np06dStbvv//+ZH3Tpk0NawMDA231hHyw5weCIvxAUIQfCIrwA0ERfiAowg8ERfiBoJqO85vZOkk/\nlHTY3a/Olk2WtFnSNEkHJN3q7p8X1+a3W52v17/99tuT9S1btpTUCfLWyp7/V5IWnLXsIUk73X2G\npJ3ZfQCjSNPwu/tuSUfOWrxQ0vrs9npJi3LuC0DB2n3P3+Xuh7LbH0vqyqkfACXp+Nx+d/fUHHxm\n1iOpp9PtAMhXu3v+ATObIknZ78ONHujuve7e7e7dbW4LQAHaDf82SUuy20skbc2nHQBlaRp+M9sk\n6f8k/ZOZHTSzpZKekHSjme2TdEN2H8AoYmXOr576bCCyxYsXJ+sbN24sqZNzff55+vSNI0fOHgg6\n06uvvtqwtmvXruS6W7dyQNkOd7dWHscZfkBQhB8IivADQRF+ICjCDwRF+IGgGOqrgZkzZybrq1ev\nTtYXLDj7osv8mKVHjTr5+xkcHEzWP/vss2T9+eefT9ZTQ4nbt29PrjuaMdQHIInwA0ERfiAowg8E\nRfiBoAg/EBThB4JinH8UGDs2/W1rK1eubFh79NFHk+tOmDAhWS9ynL9oqenFn3rqqeS6jz32WLJ+\n/PjxtnoqA+P8AJIIPxAU4QeCIvxAUIQfCIrwA0ERfiAoxvmDu+6665L1efPmJev33Xdfsj5mTOP9\ny8UXX5xct0pr1qxJ1u+9996SOjl/jPMDSCL8QFCEHwiK8ANBEX4gKMIPBEX4gaCajvOb2TpJP5R0\n2N2vzpatkvRTSZ9kD3vY3V9rujHG+cO55JJLGtbuuOOO5LrN6t3d3W311Iq9e/cm63PmzEnWT5w4\nkWc75yXPcf5fSRppVohfuvus7Kdp8AHUS9Pwu/tuSUdK6AVAiTp5z7/CzPaa2Tozm5RbRwBK0W74\n10qaLmmWpEOSftHogWbWY2Z9ZtbX5rYAFKCt8Lv7gLufcvdvJD0raXbisb3u3u3uxX06A+C8tRV+\nM5sy7O6PJL2XTzsAypL+TmhJZrZJ0g8kXWpmByU9KukHZjZLkks6IGlZgT0CKADX86O2urq6kvXd\nu3cn6zNmzMiznTNMnDgxWf/yyy8L23YzXM8PIInwA0ERfiAowg8ERfiBoAg/EFTTcX4U78orr0zW\n6/wV1/Pnz0/Wp06d2rDW7Ouvmw2XFTmctn379mS9ykt288KeHwiK8ANBEX4gKMIPBEX4gaAIPxAU\n4QeCYpw/M378+GT9pptuali76667Otr27NkNvwhJkjRpUnVfkWiWvjq02SXh/f39DWtbtmxJrnvP\nPfck67NmzUrWO7Fr165kfXBwsLBtl4U9PxAU4QeCIvxAUIQfCIrwA0ERfiAowg8EFearu8eOTZ/S\n8PTTTyfrd999d57tjBqdjvPX1Zo1a5L1Bx54IFk/efJknu3kiq/uBpBE+IGgCD8QFOEHgiL8QFCE\nHwiK8ANBNb2e38wuk7RBUpckl9Tr7k+b2WRJmyVNk3RA0q3u/nlxrXZm+fLlyXrUcfxvs2eeeaZh\n7cEHH0yuW+dx/Ly0sucflPQf7j5T0j9LWm5mMyU9JGmnu8+QtDO7D2CUaBp+dz/k7m9lt7+Q9KGk\nqZIWSlqfPWy9pEVFNQkgf+f1nt/Mpkm6VtIfJHW5+6Gs9LGG3hYAGCVa/g4/M/uOpBckrXT3Y8PP\n+XZ3b3Tevpn1SOrptFEA+Wppz29m4zQU/I3u/mK2eMDMpmT1KZIOj7Suu/e6e7e7d+fRMIB8NA2/\nDe3in5P0obs/Oay0TdKS7PYSSVvzbw9AUVo57L9O0h2S3jWzt7NlD0t6QtIWM1sq6a+Sbi2mxXwM\nDAxU3QJy1uyy3NRw3rdhiu1ONQ2/u/+vpEbXB/9bvu0AKAtn+AFBEX4gKMIPBEX4gaAIPxAU4QeC\nCvPV3WPGpP/P9fb2Jut33nlnnu2MGp1+dffRo0cb1jZs2JBcd/Pmzcl6X19fsh7hstyR8NXdAJII\nPxAU4QeCIvxAUIQfCIrwA0ERfiCoMOP8zVx44YXJ+qJFjb+f9Jprrkmue9tttyXr06dPT9aLtHr1\n6mT966+/TtaPHz+erKemPv/qq6+S66I9jPMDSCL8QFCEHwiK8ANBEX4gKMIPBEX4gaAY5we+ZRjn\nB5BE+IGgCD8QFOEHgiL8QFCEHwiK8ANBNQ2/mV1mZrvM7AMze9/M/j1bvsrM+s3s7ezn5uLbBZCX\npif5mNkUSVPc/S0zmyjpTUmLJN0q6bi7p78N4szn4iQfoGCtnuQztoUnOiTpUHb7CzP7UNLUztoD\nULXzes9vZtMkXSvpD9miFWa218zWmdmkBuv0mFmfmaXnVgJQqpbP7Tez70j6vaT/cvcXzaxL0qeS\nXNJ/auitQXJCOw77geK1etjfUvjNbJyk30r6nbs/OUJ9mqTfuvvVTZ6H8AMFy+3CHhuapvU5SR8O\nD372QeBpP5L03vk2CaA6rXzaP1fS/0h6V9I32eKHJS2WNEtDh/0HJC3LPhxMPRd7fqBguR7254Xw\nA8Xjen4ASYQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgmn6B\nZ84+lfTXYfcvzZbVUV17q2tfEr21K8/e/rHVB5Z6Pf85Gzfrc/fuyhpIqGtvde1Lord2VdUbh/1A\nUIQfCKrq8PdWvP2UuvZW174kemtXJb1V+p4fQHWq3vMDqEgl4TezBWb2JzP7yMweqqKHRszsgJm9\nm808XOkUY9k0aIfN7L1hyyab2Q4z25f9HnGatIp6q8XMzYmZpSt97eo243Xph/1mdoGkP0u6UdJB\nSXskLXb3D0ptpAEzOyCp290rHxM2s3+RdFzShtOzIZnZzyUdcfcnsn+ck9z9wZr0tkrnOXNzQb01\nmln6J6rwtctzxus8VLHnny3pI3ff7+4nJf1G0sIK+qg9d98t6chZixdKWp/dXq+hP57SNeitFtz9\nkLu/ld3+QtLpmaUrfe0SfVWiivBPlfS3YfcPql5Tfruk183sTTPrqbqZEXQNmxnpY0ldVTYzgqYz\nN5fprJmla/PatTPjdd74wO9cc919lqSbJC3PDm9ryYfes9VpuGatpOkamsbtkKRfVNlMNrP0C5JW\nuvux4bUqX7sR+qrkdasi/P2SLht2/3vZslpw9/7s92FJL2nobUqdDJyeJDX7fbjifv7O3Qfc/ZS7\nfyPpWVX42mUzS78gaaO7v5gtrvy1G6mvql63KsK/R9IMM7vczMZL+rGkbRX0cQ4zm5B9ECMzmyBp\nnuo3+/A2SUuy20skba2wlzPUZebmRjNLq+LXrnYzXrt76T+SbtbQJ/5/kfRIFT006Gu6pHeyn/er\n7k3SJg0dBn6toc9Glkr6B0k7Je2T9LqkyTXq7XkNzea8V0NBm1JRb3M1dEi/V9Lb2c/NVb92ib4q\ned04ww8Iig/8gKAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8E9f/KHJ5wsWXfUgAAAABJRU5ErkJg\ngg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2564f131550>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "imshow(img[0], cmap='gray')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "img = img.unsqueeze(dim=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**unsqueeze**: 원하는 dim에 차원을 높여준다. CNN forward를 위해서는 4D-Tensor여야함."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([5], device='cuda:0')"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample = img.to(DEVICE)\n",
    "out = model(sample)\n",
    "_, idx = out.max(dim=-1)\n",
    "idx"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Saving params.\n",
    "torch.save(model.state_dict(), 'model.pkl')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}