7-deep_convnet_vgg.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "deep-convnet-vgg.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# VGG\n",
        "[Very Deep Convolutional Networks for Large-Scale Image Recognition](https://arxiv.org/pdf/1409.1556.pdf)\n",
        "\n",
        "\"In this work we investigate the effect of the convolutional network depth on its\n",
        "accuracy in the large-scale image recognition setting.\" \n",
        "\n",
        "...\n",
        "\n",
        "\"a significant improvement\n",
        "on the prior-art configurations can be achieved by pushing the depth to 16–19\n",
        "weight layers\"\n",
        "\n",
        "...\n",
        "\n",
        "\"To this end, we fix other parameters of the architecture, and steadily increase the\n",
        "depth of the network by adding more convolutional layers, which is feasible due to the use of very\n",
        "small (\n",
        "3\n",
        "×\n",
        "3) convolution filters in all layers.\"\n",
        "\n",
        "---\n",
        "Segundo lugar no ImageNet em 2014 (atrás da complexa GoogLeNet), a VGG tenta explorar o limite da capacidade das arquiteturas de rede convolucional.\n",
        "\n",
        "Muitas arquiteturas foram propostas, AlexNet em 2012, alcançando um top 5 error de 15.3%, utilizando convoluções novamente, antes utilizada somente em 1989, por Yann Lecun em LeNet, para a leitura de digitos de 0 a 9. \n",
        "\n",
        "ZFNet em 2013, evoluindo a arquitetura e alcançando um top 5 error de 11.2%. \n",
        "\n",
        "Inceptionv1 (GoogLeNet) em 2014, alcançando um top 5 error de 6.67, com uma arquitetura extremamente complexa e paralelizada, com 22 camadas, com um processo extremamente único e regulado para treinar.\n",
        "\n",
        "E finalmente chegamos na VGG, proposta no mesmo ano de 2014, muito mais simples, com um top 5 error de 7.3, treinada em 4 GPUs por 2–3 semanas.\n",
        " \n",
        "\n",
        "Iremos construir a versão de 16 camadas, na coluna D:\n",
        "\n",
        "![image](https://user-images.githubusercontent.com/56324869/163493084-e6b9567e-a971-4cd5-831f-f501a95ecb35.png)\n"
      ],
      "metadata": {
        "id": "NikYfhCPH6Br"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BoTYmKGjEpJv",
        "outputId": "43b35cd4-f1f7-40de-d2ec-944f66abb0af"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Fri Apr 15 00:52:20 2022       \n",
            "+-----------------------------------------------------------------------------+\n",
            "| NVIDIA-SMI 460.32.03    Driver Version: 460.32.03    CUDA Version: 11.2     |\n",
            "|-------------------------------+----------------------+----------------------+\n",
            "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
            "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n",
            "|                               |                      |               MIG M. |\n",
            "|===============================+======================+======================|\n",
            "|   0  Tesla K80           Off  | 00000000:00:04.0 Off |                    0 |\n",
            "| N/A   74C    P8    80W / 149W |      0MiB / 11441MiB |      0%      Default |\n",
            "|                               |                      |                  N/A |\n",
            "+-------------------------------+----------------------+----------------------+\n",
            "                                                                               \n",
            "+-----------------------------------------------------------------------------+\n",
            "| Processes:                                                                  |\n",
            "|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |\n",
            "|        ID   ID                                                   Usage      |\n",
            "|=============================================================================|\n",
            "|  No running processes found                                                 |\n",
            "+-----------------------------------------------------------------------------+\n"
          ]
        }
      ],
      "source": [
        "!nvidia-smi"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import copy\n",
        "import os\n",
        "import time\n",
        "\n",
        "import torch\n",
        "import torch.nn as nn\n",
        "import torch.optim as optim\n",
        "import torch.nn.functional as F\n",
        "\n",
        "import torchvision.models as models\n",
        "import torchvision.transforms as transforms\n",
        "from torchvision.datasets import CIFAR10"
      ],
      "metadata": {
        "id": "Ztb71K4sEuXK"
      },
      "execution_count": 16,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "batch_size = 64\n",
        "\n",
        "transform = transforms.Compose(\n",
        "    [transforms.ToTensor(),\n",
        "     transforms.Resize((224,224)),\n",
        "     transforms.Normalize((0.485, 0.485, 0.406), (0.229, 0.224, 0.225))])\n",
        "\n",
        "image_datasets = {x: CIFAR10(root='./data', train=True if x==\"train\" else False ,download=True , transform=transform) for x in ['train','val']}\n",
        "dataloaders = {x: torch.utils.data.DataLoader(image_datasets[x], batch_size=batch_size, shuffle=True, num_workers=2) for x in ['train', 'val']}\n",
        "dataset_sizes = {x: len(image_datasets[x]) for x in ['train', 'val']}\n",
        "class_names = image_datasets['train'].classes"
      ],
      "metadata": {
        "id": "IBBEqQJR9tHi"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "class VGG16(nn.Module):\n",
        "\n",
        "  def __init__(self):\n",
        "      super().__init__()\n",
        "      \n",
        "      #2 conv3-64\n",
        "      self.conv1_1 = nn.Conv2d(in_channels=3, out_channels=64, kernel_size=3, padding=1)\n",
        "      self.conv1_2 = nn.Conv2d(in_channels=64, out_channels=64, kernel_size=3, padding=1)\n",
        "\n",
        "      #2 conv3-128\n",
        "      self.conv2_1 = nn.Conv2d(in_channels=64, out_channels=128, kernel_size=3, padding=1)\n",
        "      self.conv2_2 = nn.Conv2d(in_channels=128, out_channels=128, kernel_size=3, padding=1)\n",
        "      \n",
        "      #3 conv3-256\n",
        "      self.conv3_1 = nn.Conv2d(in_channels=128, out_channels=256, kernel_size=3, padding=1)\n",
        "      self.conv3_2 = nn.Conv2d(in_channels=256, out_channels=256, kernel_size=3, padding=1)\n",
        "      self.conv3_3 = nn.Conv2d(in_channels=256, out_channels=256, kernel_size=3, padding=1)\n",
        "\n",
        "      #3 conv3-512\n",
        "      self.conv4_1 = nn.Conv2d(in_channels=256, out_channels=512, kernel_size=3, padding=1)\n",
        "      self.conv4_2 = nn.Conv2d(in_channels=512, out_channels=512, kernel_size=3, padding=1)\n",
        "      self.conv4_3 = nn.Conv2d(in_channels=512, out_channels=512, kernel_size=3, padding=1)\n",
        "\n",
        "      #3 conv3-512\n",
        "      self.conv5_1 = nn.Conv2d(in_channels=512, out_channels=512, kernel_size=3, padding=1)\n",
        "      self.conv5_2 = nn.Conv2d(in_channels=512, out_channels=512, kernel_size=3, padding=1)\n",
        "      self.conv5_3 = nn.Conv2d(in_channels=512, out_channels=512, kernel_size=3, padding=1)\n",
        "\n",
        "      self.maxpool = nn.MaxPool2d(kernel_size=2, stride=2)\n",
        "\n",
        "      self.fc1 = nn.Linear(25088, 4096)\n",
        "      self.fc2 = nn.Linear(4096, 4096)\n",
        "      self.fc3 = nn.Linear(4096, 10)\n",
        "    \n",
        "  def forward(self, x):\n",
        "      x = F.relu(self.conv1_1(x))\n",
        "      x = F.relu(self.conv1_2(x))\n",
        "\n",
        "      x = self.maxpool(x)\n",
        "\n",
        "      x = F.relu(self.conv2_1(x))\n",
        "      x = F.relu(self.conv2_2(x))\n",
        "\n",
        "      x = self.maxpool(x)\n",
        "\n",
        "      x = F.relu(self.conv3_1(x))\n",
        "      x = F.relu(self.conv3_2(x))\n",
        "      x = F.relu(self.conv3_3(x))\n",
        "\n",
        "      x = self.maxpool(x)\n",
        "\n",
        "      x = F.relu(self.conv4_1(x))\n",
        "      x = F.relu(self.conv4_2(x))\n",
        "      x = F.relu(self.conv4_3(x))\n",
        "\n",
        "      x = self.maxpool(x)\n",
        "\n",
        "      x = F.relu(self.conv5_1(x))\n",
        "      x = F.relu(self.conv5_2(x))\n",
        "      x = F.relu(self.conv5_3(x))\n",
        "\n",
        "      x = self.maxpool(x)\n",
        "\n",
        "      x = torch.flatten(x, 1) # flatten all dims except batch\n",
        "      \n",
        "      x = F.relu(self.fc1(x))\n",
        "      x = F.dropout(x, 0.5) # dropout was included to combat overfitting\n",
        "      x = F.relu(self.fc2(x))\n",
        "      x = F.dropout(x, 0.5)\n",
        "      x = self.fc3(x)\n",
        "      return x\n",
        "     "
      ],
      "metadata": {
        "id": "uMQGYnzcHx5k"
      },
      "execution_count": 12,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Caso disponha do tempo necessário (muito tempo com a gpu do colab), treinamento seria feito com:\n",
        "epoches = 50\n",
        "device = torch.device(\"cuda\")\n",
        "model = VGG16()\n",
        "model.to(device)\n",
        "criterion = nn.CrossEntropyLoss()\n",
        "optimizer = optim.Adam(model.parameters(), lr=3e-4)\n",
        "\n",
        "best_model_wts = copy.deepcopy(model.state_dict())\n",
        "best_acc = 0.0\n",
        "model.to(device)\n",
        "start = time.time()\n",
        "\n",
        "for epoch in range(epoches):  \n",
        "\n",
        "    print(f\"\\nepoch: {epoch} / {(epoches-1)}\")\n",
        "    print(\"----------\")\n",
        "\n",
        "    epoch_loss = 0\n",
        "    model.train()\n",
        "\n",
        "    for x, y in dataloaders[\"train\"]:\n",
        "      x, y = x.to(device), y.to(device)\n",
        "      optimizer.zero_grad()\n",
        "      predictions = model(x)\n",
        "      loss = criterion(predictions, y)\n",
        "      loss.backward()\n",
        "      optimizer.step()\n",
        "      epoch_loss += loss.item()\n",
        "    loss = epoch_loss / len(dataloaders[\"train\"])\n",
        "    print(f\"epoch {epoch} training Loss: {loss:.4f}\")\n",
        "\n",
        "    with torch.no_grad():\n",
        "      model.eval()\n",
        "      correct = 0\n",
        "      samples = 0\n",
        "      for x, y in dataloaders[\"val\"]:\n",
        "        x, y = x.to(device), y.to(device)\n",
        "        predictions = model(x)\n",
        "        _, predictions = predictions.max(1)\n",
        "        correct += (predictions == y).sum()\n",
        "        samples += predictions.size(0)\n",
        "\n",
        "      val_acc = correct/samples\n",
        "      print(f\"val accuracy: {val_acc}\")\n",
        "\n",
        "      if val_acc > best_acc:\n",
        "        best_acc = val_acc\n",
        "        best_model_wts = copy.deepcopy(model.state_dict())\n",
        "  \n",
        "end = time.time()\n",
        "model.load_state_dict(best_model_wts)\n",
        "torch.save(model.state_dict(),\"vgg16.pth\")\n",
        "print(f\"Tempo de treinamento: {(end-start):.2f}\")\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 393
        },
        "id": "szgkbKZsE5z-",
        "outputId": "736c04a4-646d-42f8-cd4e-77398bcf9290"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "epoch: 0 / 49\n",
            "----------\n",
            "epoch 0 training Loss: 1.9267\n",
            "val accuracy: 0.4235000014305115\n",
            "\n",
            "epoch: 1 / 49\n",
            "----------\n"
          ]
        },
        {
          "output_type": "error",
          "ename": "KeyboardInterrupt",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-4-0dcc311b628e>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     26\u001b[0m       \u001b[0mloss\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbackward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     27\u001b[0m       \u001b[0moptimizer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstep\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 28\u001b[0;31m       \u001b[0mepoch_loss\u001b[0m \u001b[0;34m+=\u001b[0m \u001b[0mloss\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mitem\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     29\u001b[0m     \u001b[0mloss\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mepoch_loss\u001b[0m \u001b[0;34m/\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdataloaders\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"train\"\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     30\u001b[0m     \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34mf\"epoch {epoch} training Loss: {loss:.4f}\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mKeyboardInterrupt\u001b[0m: "
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Vejamos agora nossa implementação comparada a do torchvision:"
      ],
      "metadata": {
        "id": "ODCiSVLv99Lt"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from torchsummary import summary\n",
        "model = VGG16()\n",
        "model.to(device)\n",
        "summary(model,(3,224,244))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YQfzTRXuFAeu",
        "outputId": "295f422f-cea0-43e5-92ff-6f68d81b51c3"
      },
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "----------------------------------------------------------------\n",
            "        Layer (type)               Output Shape         Param #\n",
            "================================================================\n",
            "            Conv2d-1         [-1, 64, 224, 244]           1,792\n",
            "            Conv2d-2         [-1, 64, 224, 244]          36,928\n",
            "         MaxPool2d-3         [-1, 64, 112, 122]               0\n",
            "            Conv2d-4        [-1, 128, 112, 122]          73,856\n",
            "            Conv2d-5        [-1, 128, 112, 122]         147,584\n",
            "         MaxPool2d-6          [-1, 128, 56, 61]               0\n",
            "            Conv2d-7          [-1, 256, 56, 61]         295,168\n",
            "            Conv2d-8          [-1, 256, 56, 61]         590,080\n",
            "            Conv2d-9          [-1, 256, 56, 61]         590,080\n",
            "        MaxPool2d-10          [-1, 256, 28, 30]               0\n",
            "           Conv2d-11          [-1, 512, 28, 30]       1,180,160\n",
            "           Conv2d-12          [-1, 512, 28, 30]       2,359,808\n",
            "           Conv2d-13          [-1, 512, 28, 30]       2,359,808\n",
            "        MaxPool2d-14          [-1, 512, 14, 15]               0\n",
            "           Conv2d-15          [-1, 512, 14, 15]       2,359,808\n",
            "           Conv2d-16          [-1, 512, 14, 15]       2,359,808\n",
            "           Conv2d-17          [-1, 512, 14, 15]       2,359,808\n",
            "        MaxPool2d-18            [-1, 512, 7, 7]               0\n",
            "           Linear-19                 [-1, 4096]     102,764,544\n",
            "           Linear-20                 [-1, 4096]      16,781,312\n",
            "           Linear-21                   [-1, 10]          40,970\n",
            "================================================================\n",
            "Total params: 134,301,514\n",
            "Trainable params: 134,301,514\n",
            "Non-trainable params: 0\n",
            "----------------------------------------------------------------\n",
            "Input size (MB): 0.63\n",
            "Forward/backward pass size (MB): 125.11\n",
            "Params size (MB): 512.32\n",
            "Estimated Total Size (MB): 638.05\n",
            "----------------------------------------------------------------\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "torch_vgg = models.vgg16()\n",
        "torch_vgg.classifier[6] = nn.Linear(4096, 10) # Mudamos a camada final de 1000 probabilidades do ImageNet para 10 do Cifar-10 \n",
        "torch_vgg.to(device)\n",
        "summary(torch_vgg, (3,224,224))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "M-kkvsZM7f9v",
        "outputId": "9e4ee652-af62-40ce-e12f-f3fd968e474a"
      },
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "----------------------------------------------------------------\n",
            "        Layer (type)               Output Shape         Param #\n",
            "================================================================\n",
            "            Conv2d-1         [-1, 64, 224, 224]           1,792\n",
            "              ReLU-2         [-1, 64, 224, 224]               0\n",
            "            Conv2d-3         [-1, 64, 224, 224]          36,928\n",
            "              ReLU-4         [-1, 64, 224, 224]               0\n",
            "         MaxPool2d-5         [-1, 64, 112, 112]               0\n",
            "            Conv2d-6        [-1, 128, 112, 112]          73,856\n",
            "              ReLU-7        [-1, 128, 112, 112]               0\n",
            "            Conv2d-8        [-1, 128, 112, 112]         147,584\n",
            "              ReLU-9        [-1, 128, 112, 112]               0\n",
            "        MaxPool2d-10          [-1, 128, 56, 56]               0\n",
            "           Conv2d-11          [-1, 256, 56, 56]         295,168\n",
            "             ReLU-12          [-1, 256, 56, 56]               0\n",
            "           Conv2d-13          [-1, 256, 56, 56]         590,080\n",
            "             ReLU-14          [-1, 256, 56, 56]               0\n",
            "           Conv2d-15          [-1, 256, 56, 56]         590,080\n",
            "             ReLU-16          [-1, 256, 56, 56]               0\n",
            "        MaxPool2d-17          [-1, 256, 28, 28]               0\n",
            "           Conv2d-18          [-1, 512, 28, 28]       1,180,160\n",
            "             ReLU-19          [-1, 512, 28, 28]               0\n",
            "           Conv2d-20          [-1, 512, 28, 28]       2,359,808\n",
            "             ReLU-21          [-1, 512, 28, 28]               0\n",
            "           Conv2d-22          [-1, 512, 28, 28]       2,359,808\n",
            "             ReLU-23          [-1, 512, 28, 28]               0\n",
            "        MaxPool2d-24          [-1, 512, 14, 14]               0\n",
            "           Conv2d-25          [-1, 512, 14, 14]       2,359,808\n",
            "             ReLU-26          [-1, 512, 14, 14]               0\n",
            "           Conv2d-27          [-1, 512, 14, 14]       2,359,808\n",
            "             ReLU-28          [-1, 512, 14, 14]               0\n",
            "           Conv2d-29          [-1, 512, 14, 14]       2,359,808\n",
            "             ReLU-30          [-1, 512, 14, 14]               0\n",
            "        MaxPool2d-31            [-1, 512, 7, 7]               0\n",
            "AdaptiveAvgPool2d-32            [-1, 512, 7, 7]               0\n",
            "           Linear-33                 [-1, 4096]     102,764,544\n",
            "             ReLU-34                 [-1, 4096]               0\n",
            "          Dropout-35                 [-1, 4096]               0\n",
            "           Linear-36                 [-1, 4096]      16,781,312\n",
            "             ReLU-37                 [-1, 4096]               0\n",
            "          Dropout-38                 [-1, 4096]               0\n",
            "           Linear-39                   [-1, 10]          40,970\n",
            "================================================================\n",
            "Total params: 134,301,514\n",
            "Trainable params: 134,301,514\n",
            "Non-trainable params: 0\n",
            "----------------------------------------------------------------\n",
            "Input size (MB): 0.57\n",
            "Forward/backward pass size (MB): 218.77\n",
            "Params size (MB): 512.32\n",
            "Estimated Total Size (MB): 731.67\n",
            "----------------------------------------------------------------\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Veja que o número total de parâmetros da implementação oficial do TorchVision é a mesma de nossa implementação, 134,301,514, ou seja, são equivalentes. Este modelo possui 93.42% de precisão no CIFAR-10, e 74.4% de precisão (top 1) no ImageNet."
      ],
      "metadata": {
        "id": "AnQJOgO48vZn"
      }
    }
  ]
}