Lab 2 - solutions.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Lab 2 - solutions.ipynb",
      "version": "0.3.2",
      "views": {},
      "default_view": {},
      "provenance": [],
      "collapsed_sections": []
    }
  },
  "cells": [
    {
      "metadata": {
        "id": "dJ2vn2Yk5jGP",
        "colab_type": "text"
      },
      "source": [
        "## Preface from mike\n",
        "\n",
        "I modified the notebook from lab2 of the Berkley reinforcement bootcamp 2017 to run on google's colab. This save you from having to set up your own environment.\n",
        "\n",
        "Make a copy of this notebook, (so you don't give everyone the answers). Click file=>make a copy\n",
        "\n",
        "Run the first few cells to download the required code so you run it on a google cloud backend (for free for now). We can also share comments "
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {
        "id": "OojgG4UM5iKN",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "output_extras": [
            {
              "item_id": 1
            },
            {
              "item_id": 2
            }
          ],
          "base_uri": "https://localhost:8080/",
          "height": 389
        },
        "outputId": "3d37a998-5d1c-4433-ff8a-f1d4e9029a39",
        "executionInfo": {
          "status": "ok",
          "timestamp": 1509629010906,
          "user_tz": -480,
          "elapsed": 1279,
          "user": {
            "displayName": "Mike C",
            "photoUrl": "//lh3.googleusercontent.com/-xWyBhueI0Vo/AAAAAAAAAAI/AAAAAAAAIg8/rU-qliu5l-M/s50-c-k-no/photo.jpg",
            "userId": "110113503404408134511"
          }
        }
      },
      "source": [
        "%%bash\n",
        "echo \"downloading lab\"\n",
        "wget --no-check-certificate 'https://docs.google.com/uc?export=download&id=0B1BwaUH2mk-ELUVyUFFvVmcyUnM' -O lab2.zip"
      ],
      "cell_type": "code",
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "downloading lab\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "--2017-11-02 13:23:32--  https://docs.google.com/uc?export=download&id=0B1BwaUH2mk-ELUVyUFFvVmcyUnM\n",
            "Resolving docs.google.com (docs.google.com)... 74.125.141.102, 74.125.141.101, 74.125.141.138, ...\n",
            "Connecting to docs.google.com (docs.google.com)|74.125.141.102|:443... connected.\n",
            "HTTP request sent, awaiting response... 302 Moved Temporarily\n",
            "Location: https://doc-0o-0s-docs.googleusercontent.com/docs/securesc/ha0ro937gcuc7l7deffksulhg5h7mbp1/33ja8u79g6j11di8433ii8url0hpv5rk/1509624000000/16080780047704686699/*/0B1BwaUH2mk-ELUVyUFFvVmcyUnM?e=download [following]\n",
            "Warning: wildcards not supported in HTTP.\n",
            "--2017-11-02 13:23:32--  https://doc-0o-0s-docs.googleusercontent.com/docs/securesc/ha0ro937gcuc7l7deffksulhg5h7mbp1/33ja8u79g6j11di8433ii8url0hpv5rk/1509624000000/16080780047704686699/*/0B1BwaUH2mk-ELUVyUFFvVmcyUnM?e=download\n",
            "Resolving doc-0o-0s-docs.googleusercontent.com (doc-0o-0s-docs.googleusercontent.com)... 74.125.141.132, 2607:f8b0:400c:c06::84\n",
            "Connecting to doc-0o-0s-docs.googleusercontent.com (doc-0o-0s-docs.googleusercontent.com)|74.125.141.132|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 106566 (104K) [application/zip]\n",
            "Saving to: ‘lab2.zip’\n",
            "\n",
            "     0K .......... .......... .......... .......... .......... 48% 52.5M 0s\n",
            "    50K .......... .......... .......... .......... .......... 96% 49.2M 0s\n",
            "   100K ....                                                  100% 34.0M=0.002s\n",
            "\n",
            "2017-11-02 13:23:32 (49.9 MB/s) - ‘lab2.zip’ saved [106566/106566]\n",
            "\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "metadata": {
        "id": "vCyCCBlQ5i8d",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "output_extras": [
            {
              "item_id": 1
            },
            {
              "item_id": 2
            }
          ],
          "base_uri": "https://localhost:8080/",
          "height": 545
        },
        "outputId": "2f0bea5f-0df7-4721-efc2-e0a63dc6a401",
        "executionInfo": {
          "status": "ok",
          "timestamp": 1509629013535,
          "user_tz": -480,
          "elapsed": 1040,
          "user": {
            "displayName": "Mike C",
            "photoUrl": "//lh3.googleusercontent.com/-xWyBhueI0Vo/AAAAAAAAAAI/AAAAAAAAIg8/rU-qliu5l-M/s50-c-k-no/photo.jpg",
            "userId": "110113503404408134511"
          }
        }
      },
      "source": [
        "%%bash\n",
        "echo \"unzipping lab\"\n",
        "unzip -o lab2.zip\n",
        "mv -f lab2/* .\n",
        "rm lab2/ -rf\n",
        "\n",
        "# this notebook is python2, the code is python3, make it compatible with this\n",
        "echo \"\"\n",
        "echo \"python 3 to 2\"\n",
        "pasteurize -w *.py > pasteurize.log"
      ],
      "cell_type": "code",
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "unzipping lab\n",
            "Archive:  lab2.zip\n",
            "   creating: lab2/\n",
            "  inflating: lab2/.DS_Store          \n",
            "  inflating: lab2/Lab 2.ipynb        \n",
            "  inflating: lab2/LICENSE            \n",
            "  inflating: lab2/environment.yml    \n",
            "  inflating: lab2/lab2.md            \n",
            "   creating: lab2/.ipynb_checkpoints/\n",
            "  inflating: lab2/.ipynb_checkpoints/Lab 2-checkpoint.ipynb  \n",
            "\n",
            "python 3 to 2\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt\n",
            "root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt\n",
            "root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt\n",
            "root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt\n",
            "root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt\n",
            "root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt\n",
            "root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt\n",
            "root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt\n",
            "root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt\n",
            "root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt\n",
            "root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt\n",
            "root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt\n",
            "root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt\n",
            "root: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt\n",
            "RefactoringTool: Can't open *.py: [Errno 2] No such file or directory: '*.py'\n",
            "RefactoringTool: No files need to be modified.\n",
            "RefactoringTool: There was 1 error:\n",
            "RefactoringTool: Can't open *.py: [Errno 2] No such file or directory: '*.py'\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "metadata": {
        "id": "c2jQ5QPx5i_L",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "output_extras": [
            {
              "item_id": 1
            }
          ],
          "base_uri": "https://localhost:8080/",
          "height": 651
        },
        "outputId": "5cadb486-7cb5-455d-f081-8199ff2e2250",
        "executionInfo": {
          "status": "ok",
          "timestamp": 1509629020010,
          "user_tz": -480,
          "elapsed": 6227,
          "user": {
            "displayName": "Mike C",
            "photoUrl": "//lh3.googleusercontent.com/-xWyBhueI0Vo/AAAAAAAAAAI/AAAAAAAAIg8/rU-qliu5l-M/s50-c-k-no/photo.jpg",
            "userId": "110113503404408134511"
          }
        }
      },
      "source": [
        "%%bash\n",
        "echo \"install dependencies\"\n",
        "pip install gym chainer"
      ],
      "cell_type": "code",
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "install dependencies\n",
            "Collecting gym\n",
            "  Downloading gym-0.9.4.tar.gz (157kB)\n",
            "Collecting chainer\n",
            "  Downloading chainer-3.0.0.tar.gz (339kB)\n",
            "Requirement already satisfied: numpy>=1.10.4 in /usr/local/lib/python2.7/dist-packages (from gym)\n",
            "Requirement already satisfied: requests>=2.0 in /usr/local/lib/python2.7/dist-packages (from gym)\n",
            "Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from gym)\n",
            "Collecting pyglet>=1.2.0 (from gym)\n",
            "  Downloading pyglet-1.2.4-py2-none-any.whl (964kB)\n",
            "Collecting filelock (from chainer)\n",
            "  Downloading filelock-2.0.12.tar.gz\n",
            "Requirement already satisfied: mock in /usr/local/lib/python2.7/dist-packages (from chainer)\n",
            "Collecting nose (from chainer)\n",
            "  Downloading nose-1.3.7-py2-none-any.whl (154kB)\n",
            "Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python2.7/dist-packages (from chainer)\n",
            "Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python2.7/dist-packages (from requests>=2.0->gym)\n",
            "Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python2.7/dist-packages (from requests>=2.0->gym)\n",
            "Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python2.7/dist-packages (from requests>=2.0->gym)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python2.7/dist-packages (from requests>=2.0->gym)\n",
            "Requirement already satisfied: funcsigs>=1; python_version < \"3.3\" in /usr/local/lib/python2.7/dist-packages (from mock->chainer)\n",
            "Requirement already satisfied: pbr>=0.11 in /usr/local/lib/python2.7/dist-packages (from mock->chainer)\n",
            "Requirement already satisfied: setuptools in /usr/local/lib/python2.7/dist-packages (from protobuf>=3.0.0->chainer)\n",
            "Building wheels for collected packages: gym, chainer, filelock\n",
            "  Running setup.py bdist_wheel for gym: started\n",
            "  Running setup.py bdist_wheel for gym: finished with status 'done'\n",
            "  Stored in directory: /content/.cache/pip/wheels/ad/46/5a/1833d0ef8149a50e78fdb24a41f47e22a7f4a9dcd649b8db53\n",
            "  Running setup.py bdist_wheel for chainer: started\n",
            "  Running setup.py bdist_wheel for chainer: finished with status 'done'\n",
            "  Stored in directory: /content/.cache/pip/wheels/50/a6/5f/1b4d884fcc4e079cfc6f60f1ba0f5d1cffff63be09c111e273\n",
            "  Running setup.py bdist_wheel for filelock: started\n",
            "  Running setup.py bdist_wheel for filelock: finished with status 'done'\n",
            "  Stored in directory: /content/.cache/pip/wheels/68/e0/0c/ca5c34b3b61ef8d3caf4b7fbbfb2f9f840128ce4015ad52566\n",
            "Successfully built gym chainer filelock\n",
            "Installing collected packages: pyglet, gym, filelock, nose, chainer\n",
            "Successfully installed chainer-3.0.0 filelock-2.0.12 gym-0.9.4 nose-1.3.7 pyglet-1.2.4\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "TGyxx5xNh9ld",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "source": [
        "\"\"\"\n",
        "This project was developed by Peter Chen, Rocky Duan, Pieter Abbeel for the Berkeley Deep RL Bootcamp, August 2017.\n",
        "Bootcamp website with slides and lecture videos: https://sites.google.com/view/deep-rl-bootcamp/.\n",
        "\n",
        "Code adapted from Stanford CS231N materials: http://cs231n.stanford.edu/\n",
        "\"\"\"\n",
        "\n",
        "\n",
        "# note to properly run this lab, you should execute all code blocks sequentially\n",
        "import chainer\n",
        "import chainer.functions as F\n",
        "import chainer.links as L\n",
        "from chainer import training\n",
        "from chainer.training import extensions\n",
        "\n",
        "from collections import namedtuple, defaultdict, deque\n",
        "\n",
        "import numpy as np\n",
        "\n",
        "%matplotlib inline\n",
        "\n",
        "import matplotlib\n",
        "import matplotlib.pyplot as plt"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "HpYCNt7Ah9lg",
        "colab_type": "text"
      },
      "source": [
        "## Introduction to Chainer\n",
        "\n",
        "Chainer can be understood as Numpy plus the ability to record the computation graph of numerical operations to enable Automatic Differentiation. (Chainer actually also offers many other things; for example, a Numpy equivalent library that runs on GPU, but we will ignore them for now)"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {
        "id": "9UHVo1QJh9lh",
        "colab_type": "text"
      },
      "source": [
        "Let's illustrate how Chainer works by a simple 1D regression task.\n",
        "\n",
        "Suppose we have observations from the following model $y = w x + b + \\epsilon$ where $\\epsilon \\sim \\mathcal{N}(0, 0.1)$ and the task is to estimate the linear model parameters $w, b$ from data."
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {
        "id": "trRz8afmh9lh",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "output_extras": [
            {
              "item_id": 1
            }
          ],
          "base_uri": "https://localhost:8080/",
          "height": 266
        },
        "outputId": "c9bc13b6-34be-4e8c-8ecf-17ef3d491ae6",
        "executionInfo": {
          "status": "ok",
          "timestamp": 1509629022307,
          "user_tz": -480,
          "elapsed": 866,
          "user": {
            "displayName": "Mike C",
            "photoUrl": "//lh3.googleusercontent.com/-xWyBhueI0Vo/AAAAAAAAAAI/AAAAAAAAIg8/rU-qliu5l-M/s50-c-k-no/photo.jpg",
            "userId": "110113503404408134511"
          }
        }
      },
      "source": [
        "# first generate some observations\n",
        "true_a = 1.3\n",
        "true_b = 0.4\n",
        "data_x = (np.arange(100) / 99.0 - .5).astype(np.float32) # Chainer assumes all the cpu computation is done in float32\n",
        "data_y = (data_x * true_a + true_b + np.random.randn(*data_x.shape) * 0.1).astype(np.float32)\n",
        "_ = plt.scatter(data_x, data_y, c='b')"
      ],
      "cell_type": "code",
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAD4CAYAAADmWv3KAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3X+QHOV95/H37g67pdkfRCFrWC2L\nwC75CRjflXHqgizkH4DkQExRRyCmnEsVilXyD/4AVXIRlvYS5axgcQUIiJ0rVATzTzCiLgJMgYLA\npFwLsus4XHYddeSxY0uwWm3BBi9IuyNrtTtzf8yM1DvT3dPd0z0zPfN5Vbm8O9M70w+7+vYz3/4+\n36erUCggIiLtp7vZJyAiIslQgBcRaVMK8CIibUoBXkSkTSnAi4i0qUyzT6BsZuZEqst5Vq7MMjub\na/ZpNEwnjbeTxgoab9oMDw92eT2nGXxMMpmeZp9CQ3XSeDtprKDxthMFeBGRNqUALyLSphTgRUTa\nlAK8iEibUoAXEWlTCvAiIm1KAV5EpMFyOTh8uItcwuX3CvAiIg2yuAjj472sX59l7dp+1q/PMj7e\ny+JiMu/XMitZRUTa3c6dvezd23fm+8nJHvbuLS602rVrIfb30wxeRKQBcjk4cMB9Tn3gQCaRdI0C\nvIhIHYLm0995p4upKfeQe+xYN++849lSJjIFeBGRCMLm088/v8DoaN71uVWr8px/fvz9FhXgRUQi\nKOfTJyd7yOe7Svn0Pnbu7HU9PpuF665zj/7XXbdINhv/OSrAi4iEFDWfvnPnAlu2nGJsbImengJj\nY0ts2XKKnTvjv8EKqqIRkRTI5Yo57PPPLyQy0w0rSD79kkuqUy6ZTLFaZvv2hYaMRzN4EWlZja4b\nD6refHo2C5dckvzFSgFeRFpW2Dx3ozQjnx6FAryItKRm1I2H0eh8ehTKwYtIS4qa526UsPn0ZtxH\n0AxeRFpSXHXjSTf2qpVPb+Z9BAV4EWlJ9ea5W+UGbTPvI9SVojHGXA48A+yx1n674rlrgbuBJeB5\na+0363kvEek85Xz2gQMZjh3rZtWqPNddtxgoz+3V2Ov4cdi9e8HzAhFnKqXWfYTt273PIw6RZ/DG\nmH7g74AfeBzyEPBHwDpgozHmsqjvJSKdqZznnpjIcejQPBMTOXbtWiBTY2rqF1ifeKKXq66qns0n\nMeNvRv8Zp3pSNKeA64FjlU8YYz4M/NpaO2mtzQPPA9fU8V4i0sHC1o37BVbo4ujR6jRJEqmUZvSf\ncYqcorHWLgKLxhi3py8AZhzfvwt8xO/1Vq7Mksn0RD2dljA8PNjsU2ioThpvJ40V0j/e/n646CI4\ncsT/uIMH+8jloL9/kBde8D5mz56+yKmUm26CBx90e7yH1auT/e/cqDLJmp9DZmebXNRap+HhQWZm\nTjT7NBqmk8bbSWOF9hnvxo3Lc/BuJicLTE93MTU1x+RkP26hanKywBtvzAcuyazM4W/bBidP9lbd\nR9i2bYGZmdqvV4vfxTipAH+M4iy+bBSXVI6ISFLKN2Kfey5TStdUB++RkTz339/DM8+sIO+eSQmc\nSllcLKZ5Dhwovt/o6Nkbwo3sP+OUSJmktfYIMGSMudgYkwG+ABxM4r1ERNyUb9C++mqOL37xtOsx\nv/VbBf7+72FqqgevRMO6dcHustbK4Teq/4xT5Bm8MeaTwH3AxcBpY8zNwPeBw9bap4CvAd8rHb7P\nWvvzOs9VRCS0bBb27DnFuecWlqVJNmxY5MUXvUJggYGB4qz9ySd7efXVzJnZuFsFT7PLIb10FQrN\nW+rrNDNzojVOJKJ2yVsG1Unj7aSxQnuP15kff+edLtau7Sefd5u5F3Cb0W/Zcsp1c+zDh71fq6en\nwKFDwXP4YQ0PD3re49RKVhHpGM40iV8JY49HQZ9Xk7Nml0N6UYAXkY7k1wphacn9Z7wWJ7Vq+2B1\nkxSRjrVz5wIrVvSxf/9SVW5+crJ6Gu83G6+nrUJSFOBFpGNlMvDAA7B1a25ZCWMmA3v3Vgd4v9l4\no7fjC0IBXkQ6Xjk3X1bPbLzytZpJAV5EpEIrzsaj0E1WEWkLSWzs0YzFSXFSgBeRVIva5jfpnZ5a\ngQK8iCQm6SCay8HWrX2h2vxWXhA+9jGastNTIyjAi0jskt4ur/z669Zl2bfvHNdjvBYlVfaMOXKE\nhm2h12gK8CJSt8qZetL7kJZf369JmNuipFo9Y9otXaMALyKRuc3U77qrN9Eg6heknT70oTxDQ8vL\nFePaQi8t+XsFeBGJzG2m/uijfUxOJrcPqf92fGdNT3ezcePy1FA9PWNyOfjFL7rYti251FPcVAcv\nIpH4zaR7etz7ubjNqit3QKqlHKTdWgks7wJZTg0Vj9u1a+FMz5gwq1SdG3kUL1xnL1CVr99qNIMX\nkUj8ZtJezbqcs+rf/CbajVi/xl7ZrPsM3Jka2rlzgS1bTjE2tkRPT4GLLy62AfZaper8lOKV72/V\n/L36wceknXtou+mk8XbSWCH4eHM5WL8+6zmTLm+YMTfXhVtgvPzyRd54o/oTgFfPdef7Tk118cgj\n5/DSS2dbCaxbt8iTT/YG7sle/uRw+eUDzM+7j9d/jP6v3yjqBy8isfObSUMXc3PdzM11e86q33zT\nPWh6zYadN3TXr+/npZcybNiwyMTEPBMTOXbvXgiVXw+ySjVovr+ZPd/91JWDN8bsAa6kmPi6w1r7\nmuO524H/AiwB/8dae2c97yUirSfIxta5nPsEs1bP9crZcDlVUla8odtzpm8MhM+v1+Kf76//9ZMW\neQZvjPkMsMZauxb4MvCQ47kh4L8C6621VwGXGWOurPdkRaS1lIPr44+fpDtkNPHaNcltNhy0fr0y\nvz42tuSbX6/F/1NK/a+ftHpm8NcATwNYa980xqw0xgxZa48DC6X/DRhj5oAs8Ou6z1ZEWtLq1d4z\n3YGBQikPv9ylly655uDXrasOqEHq1y+5pJBIF0i31sEbNiyyefNpVq3yf/2wFUJxqyfAXwC87vh+\npvTYcWvtb4wxfwP8CjgJPGGt/bnfi61cmSWT8f8Y1OqGhwebfQoN1Unj7aSxAvT3DzI9DSMjBA5M\nN90EDz5Y/fimTd10d8Mzz8DkJIyNwY03wu7dGe66q/j4229Df3/x+Cef7OPHP+7jxhvh3nuLnxL6\n++Gii+DIkerXHxsr3iitPM/Vq4OPt9bv9+GHi8G6+N+kh2y2B+jzPH5xEf7iL86O7aKLWDaeRonz\nrc5cokspmu3AR4HjwMvGmP9orf2Z1w/PzrZgjVEIqrRoX5001sVFuOeeQfbvX2JqqpvR0bMbXdQK\nTNu2wcmTvVWbZHzjG8Wf3bp1+Wz2xAnYsaP4+LZtfezbd7aNwZEjxYvFyZNnK2o2blyegy/buPEU\n8/MLzM9HG3OY3+/QEMzPU/O9xseXn6vbeOLid3GqJ8AfozhjL1sFTJe+vhT4lbX23wGMMRPAJwHP\nAC8izZXLlQMtQPHTdJiFPLXSI347HR065F1Rs317cYFSK+556qbW/YLyeBqhnjLJg8DNAMaYK4Bj\n1tryZfAIcKkxZkXp+98DflHHe4lIQsJ0ZnT2YPHqxxJ2kwy//PrUVDdvvVVMDpQvIBMTOQ4dKpZG\n7tpV+5NFo8XV7yYOdS10MsbsBj4N5IHbgU8AH1hrnzLGfAXYBCwCh6y1f+n3WlrolC6dNN52H2tl\nOsFNd3eBW245zaFDPRw92k1/f/Gfay7XFSqN46bWgqkLL8xz/fXRX7+WuH+/fuMZG1tiYiIX6wze\nb6FTXf+5rLV3VTz0M8dzDwMP1/P6InJWEhUZQTszZrOFZTlyZ1VMvf1Y/PrDQBdHj7Z2v5dKUfrd\nJEUrWUVaXJJb0gVdqRlEPf1YyvXro6NLFNdNxvv6jRZ3PX5U6kUTk3b/GF+pk8bb7LF6pVC8erY4\nux/WqoQJkh656irvHi9OcfRjefPNLj73uf7A/WTikOTvtxF18OpFI5JSUXYgCrObkt9KzVtvXeCV\nV/x7vDjF0Y+lvGAqqddvtLA3nOOmAC/SwsJWZES7ILi3z73//mI5n/9y/bPiyC/7vVer9ntpZS1W\nYCQiTn7NrtxmtEGX9DtV1q8X2+cuT/04a9Cnps52iDx5siv2evS01LungQK8SAsLW5ER9oJQ+V7l\ndELlSk23RUyQTH45iX4ynUoBXqTFhZnRJl2iV7kaNckNLvxWvkowCvAiLS7sjDbuFEezOyJKdArw\nIikRdEYbV4ojTLmltCb9mkTaVL0pDrcdlNK0olRUJikiLqKUW0rrUYAXkSqt1BFRolOAF2kjQfrP\nBFEut3STxhWlnUoBXqQN1GpIFjbwa0Vpe9BNVpEWFrRE0euGaD4P3d1EqoTRitL0U4AXaUFhO0J6\n3RB94olzmJs7+0E9zi34pPUpRSPSgmp1hHSmXPxuiDo35nAKUwnT7I6IEl3kGbwxZg9wJcXu/HdY\na19zPDcGfA/oBX5irf1qvScq0ineew+efdb9n+bzz2c4fRpeeunszH7DhkXP/jNevBqPSXuJNIM3\nxnwGWGOtXQt8GXio4pD7gPustf8JWDLGXFTfaYq0v/KN0quvzjI97f5P8+jRbr773eUz+0cf7ePc\nc90D9cCA++PNroSJq9pH/EVN0VwDPA1grX0TWGmMGQIwxnQD64Hvl56/3Vr7dgznKtLWymmZ6eke\nwD210uMxSX///S42bareIu7WW0+7Ht+sSpio2w9KNFFTNBcArzu+nyk9dhwYBk4Ae4wxVwAT1tpv\n1HrBlSuzZDLBP2K2ouHhwWafQkN10niTHmsuBy+8UPu4pSX3wD893cOOHT18+9swPQ0jIz1ksz0s\nLkJ/PzzzDExOwtgY3Hgj3HtvH5lM9TaAZUmN9847Ye/es9+Xb/quWNHHAw8k8paBtOvfclxVNF0V\nX48CDwJHgOeMMX9orX3O7wVmZ9P9Wa3Z+3Y2WieNtxFjPXy4i8nJftxn7gVGRopVNC++mPHo9b5E\nJpNjfh6Ghor93Ms93XfsgK1bl5dbzs56n0tS483lYP/+LFB9/vv3L7F1a64pnyrS/rfsd3GKmqI5\nRnHGXrYKmC59/e/AW9baX1prl4AfAB+L+D4iHcFv5ejISJ6XXy7ujRp18VErVMKo/UHjRQ3wB4Gb\nAUppmGPW2hMA1tpF4FfGmDWlYz8J2HpPVKTdOG80+q0cveGGRc47r/h15f6p5Vx7Pb3eG3WzU+0P\nGi9SisZae8gY87ox5hCQB243xtwGfGCtfQq4E3isdMP1/wLPxnXCImnntYhpfLz2ylGvrfMmJ4Mv\nRMrlYGqqi0ceOWdZuWXSq1ST3m1KqnUVCq1x1ZyZOdEaJxJR2vN4YXXSeOMe6/j48rYCZVu2nGLX\nroXA7QnCbsjhPH5yshu3fP+WLad4+OG+xH63znOovIg1axORtP8tDw8Peua2FOBjkvY/krA6abxx\njjWXg/Xrs643SsfGlpiYCH6jsdaFIujxlefwr//aw/x8sr/bVtoGMO1/y34BXq0KpOM1Mg8d143G\nsBty+B1feQ7T0zUPq1sr3PTtBArw0rGasegmrhuNYS8UfsdXnsPISKBTkBRQgJeOVauhV1hBPgnE\n1Wc97IXC7/io5yCtTwFeOlKce46G/SQQR6lj2AuF3/FQf7mltCb1g5eOFCTFEbTTotdmG1Ddc718\nc3H79vr7rIfdkMPt+A0bFtm8+TSrVikf3o5URROTtN+JDyvt4w1TzeI31qCvE7akMexYwlwoah2f\n9t9tWGkfr6poRCrElQsPerMz7ny/U9iKFFWwdA4FeOlYceTCg9zsjDPfLxKGAry0vKTq1MvL/icm\nchw6NM/ERI5du8KlTIJ8ElCTLWkW3WSVlpVk3tqpnLKIqtbNzvIs373Nr5psSXIU4KVlhalOaSa3\nBmCV+e1PfWqJffvUZEsaSykaaUlpzFtX3rx01sc/+eQ5DAzkGRjIu+b7tUepJEEzeGlJcdapN0vl\nJ5C5uWKu/dZbT7F798KZ8snx8eTTUNKZ9CckLSnteev33oNnn3X/5/XqqxmgOHNPSxpK0kkpGmlJ\ncdWph1VvqqQ8I7/66izT0/6fQNKYhpJ00QxeWlbYpfj18KvYgeCrRStn5G7Kn0DaIQ0lrS1ygDfG\n7AGuBArAHdba11yO+Raw1lr72chnKB0rSHVKXLxSJfk89PfD/v3ZmjnyoD3Xy59A0p6GktYXKcAb\nYz4DrLHWrjXGXAo8CqytOOYy4NPA6brPUjpavXXqtfgF5ieeOIe5OYBiEPbLkfv3XC8wMpLnhhvO\nfirQHqWStKg5+GuApwGstW8CK40xQxXH3AfsqOPcRBrCLzCXK18queXI/doWjIzkefnl6pWycbRL\nEPESNUVzAfC64/uZ0mPHAYwxtwE/BI4EfcGVK7NkMtUzmTQZHh5s9ik0VBrGm8vB9DSMjOA5I+7v\nh4sugiNH3J51D/DHjvWwuDjI8PDyx2+6CR58sPr4P/7jHn73d93/ez38sPM8e8hmewD/PH7S0vC7\njVO7jjeum6xn/hUYY34b2ARcC4wGfYHZ2XSXDKS95WhYzRpv0JudYdscbNzofnN0YCDP3Fz17H7V\nqiUymRwzM8sf37YNTp7srboxvG3bQtWxlYaGYH6++L9m0t9yuvhdnKIG+GMUZ+xlq4DyVr1XA8PA\nBMVpyEeMMXustVsjvpdIqICdy8G2bX3s23e2FW+t+nKvip18Hh55pDrwe+XIG3ljWKSWSBt+GGM+\nBfyNtXaDMeYK4CFr7VUux10MPBakikYbfqRLo8c7Pu4+w96y5dSZgF2+CDz3XKaUU69Or1Ru5lGp\n8hPC4iLcc88g+/cvVZVqtutKU/0tp4vfhh+R/kSttYeMMa8bYw4BeeD2Ut79A2vtU9FOUzpRkJRL\nrQVB27cXl/0HqUGvVV9eWbGTycADD8DWrTnNyCV1Is9BrLV3VTz0M5djjgCfjfoe0r7CpFyCLAg6\n//xCoBr0qPXlSZdqiiRBrQqkKcJsYRdk1yT/GvSz1q1zb38QlbpASitTgJeGC9uDJUhfGr+LABTO\ntOp98sli+9677url3/6t/p4z69dnWbu2n/Xrs4yP97IY7/VDpC4K8NJwb70Vfgu7WguC/C4CH/3o\nEnNz3czNdZ/5tPDoo3186lPRAnMuB1u39iW2ibZIXNq0DkBakbPKJe8x2fbKkQcpP3QrddywYZEX\nX/T6M+8K1Z63ukqnmvOmr0izaQYvDVPOu09N9eC1QrSccvHKbVfumuTkton2V75yOlBu3i01VHkO\nQc5fm2hLK9EMXhrCv9NigQsvzHP99YuMjy/UvcORs+LFr2Ojk7N8srLC56KL4Oqre30+CZylLpDS\nShTgpSH8qly6u+Ef//Ekl15aqFrQVO8OR34dG52cgbmynv7IEXj00T6KnbH9qQuktBKlaKQh/Kpc\nRkfzrF5dSGyHI+cNWq8g7UwNeZ1Dj+c1osCFF6oLpLQeBXhpiCCljkEWNEXhzM2/+uo8mzZ5V+P4\nncPSkvvr33rrAq+8Ut0KWKTZ9OcoDVNrC76wOxwF7SxZls3CmjUF7rlngVzOvRrH7xwuvLBYlfPS\nS9Xnr8AurUh/ltIwtUodg+5wFLYVsBuv1gN+53D99Yvs2uV9cRBpNQrw0nB+fV2CbLTttX8qRLsR\nW+scxsa62Lhx+aIq9aWRNIjULjgJahecLkmP1yv9ksvB+vVZ1xRKrVbAUc/h8ssHmJ/X77ZdpX28\nfu2CdZNVElNPIy6vBU1J3YgNcw4iaaEAL7FLshFXkM6SIlKkAC+xC9MKOKwg5ZYiUqQAL7FKarGS\nU63OkiJSFLmKxhizB7iS4tLAO6y1rzme+xzwLWAJsMBma61Xs25pI0Fy5PVWoLiVWwJMTqp0UcQp\n0gzeGPMZYI21di3wZeChikP2Ajdba9cBg8Af1HWWkhpDQwXOP78xOfJsFsbGCtx9tzbeEHETNUVz\nDfA0gLX2TWClMWbI8fwnrbVHS1/PAOdFP0VJg/KN1Y0bs0xPu/9ZJZEjTzLfL5J2UVM0FwCvO76f\nKT12HMBaexzAGDMCbAT+W60XXLkySybj3/Gv1Q0PDzb7FBrKOd4774S9e6uP6eqC1avhxhvh3nv7\nyGT6qg+KKJeDF15wf+7gwT727OmL7YLSyb/bTtCu441rJWtV8bEx5kPAs8DXrbXv1XqB2dl071qc\n9sUSYTnHm8vB/v1ZoPoCfcEFSxw4kOO882B2Nt5zOHy4i8nJftw235icLPDGG/OxrDjt5N9tJ0j7\neP0uTlFTNMcoztjLVgHT5W9K6ZoDwLi19mDE9xCHehYN1fOzQfjdWH333W6OH09mhyPVxIv4ixrg\nDwI3AxhjrgCOWWudl8D7gD3W2n+u8/w6Xj2LhpJccOS8aDQr0KomXsRf5F40xpjdwKeBPHA78Ang\nA+AFYBb4kePwx621Lhnas9SLxl3lDkdlW7acKnU29G6ZW+tnoyh3cjx4sI+33y6c6eSYz8Mjj8T7\nXmHOx605WVwtfNP+ET4sjTdd/HrRqNlYTJL4I/FrrHXhhUtnepO7tcwN25QraG91r4vG5s2n6O52\n7wLZiF7pYXvDh5H2ABCWxpsufgFe7YJbmF9u++jRbr77Xe+WuX4/OzXVzVtvdXHppdUbTPv1Vvdb\npfrccxlefjnn2es9aWrhK1JNrQoaKOzNTr/cttf+oOV2AH4/m8/Dn/zJCsbHe/mrvwpeR+530Zie\n7ubqq7PcfXcvY2NaTSrSChTgGyDqzU6/m4he+4OW2wH4/Sx0cfRoMZA/8cQ5rkeULxRBb6ZCF9PT\nWmQk0koU4BugntWWbo21/uzPTjE2Vrtqpfyzo6NLFFsGVZubc0/fTU11s21b37KL0t139/L5z9cu\nwYnaVCzpck6RTqMAn7B6uyuWG2tNTOQ4dGieiYkcu3cvBCoPLP/s44+fpDvkbzqbLbBvX2/VRQko\nXTTA66IRduONJMs5RTqZAnzC4tqBqHJ3oTAtc1ev9k6tZLPhbky+8EKG7dsX+OlPYWQkntp39ZMR\nSYYCfMKSWgTkNrPftcu9JNEvH5/LdTEwkGdgIH/mQnHrrafI5dwvPOWL0u/8DtxwQ/BFRl7pl0b0\njxfpVArwCUt6tWXQfUOdM/7lqZUu5ua6mZvr5pZbFs6kgIJclIJ8iqiVfmnkHqsinUZ18A1QDnhu\ni4AapTzj37p1gauvzjI9XV1n+eqrGWDhzEWpXFfv5Jbj96t9L6dfyirr9cufcNwWZKmfjEh9NINv\ngDDpFKckqkqOH+/inXdqz5jD5Pi9PkUESb+on4xIcjSDb6Cgqy3DrC4NK+iMuda2eEEE3b6vFT7h\niLQjBfgWVCutUY8g6ZfKvi5jY4WqC85NN8G2bfhecOq5mGjmLlI/BfgWUyutsX37Qt3Bz2vGPD6+\nwPh49SeHyk6Rk5M9PPggnDzZ63vBCZrLdx6vfjIi8VE3yZj09w/yxhtzdc8+Dx/uYu3afvL56uqR\nnp4Chw7Fs0sRVM/UvTpFDgzkmZurTrW4daWs1Ih2vklLe7fBsDTedFE3yQSd7Y8Ob7/dX3e+vJFV\nJc4Zs98nB692Bs48uhelX0SaR1U0dSrny48cIZZVmM2qKvG7IeolzAUnaL2+iMRHAb4OSa3CDFOi\nGBe/FbcDA+5BXGWMIq0tcorGGLMHuJLissg7rLWvOZ67FrgbWAKet9Z+s94TbUVBywDDakZaw++G\n6K23nq7aremmm3rYtk1ljCKtLFKAN8Z8BlhjrV1rjLkUeBRY6zjkIeDzwBTwQ2PMP1lr/1/dZ9ti\nks6XN7qqxK8ePZNh2QVn9epBZmYadmoiEkHUFM01wNMA1to3gZXGmCEAY8yHgV9bayettXng+dLx\nbafdVmHWWnGrPLpIukRN0VwAvO74fqb02PHS/zvndu8CH4n4Pi2vPOs9eLCPyclCW6zCVD26SHuI\nq0zSr+VfoHaAK1dmyWQ8NhptcQ8/XLzhOj3dxchID9lsD1BdT95uhocHm30KDdNJYwWNt11EDfDH\nKM7Uy1YB0x7PjZYe8zU7m+7G38PDgwwNnWB+Hubnm302yUv74pAwOmmsoPGmjd/FKWoO/iBwM4Ax\n5grgmLX2BIC19ggwZIy52BiTAb5QOl4ctP+oiCQt0gzeWnvIGPO6MeYQkAduN8bcBnxgrX0K+Brw\nvdLh+6y1P4/lbFOisgWAk1unyA0bFtm8+TSrVgW7gen3+iIiZepFU0PQYDo8PMj09ImabX69+r1A\ngbEx/zYHSbYRDivtH2vD6KSxgsabNupFE0GUYFqrza/fylfoOnP88eOwe3d118gwbYQ1yxcRtSrw\nUA6mk5M9gXrMBGlbELTfyxNP9HLVVcv3Lg3aFqHWHqgi0jkU4F1E6TEzPU3NtgV+/V6W6+Lo0eUX\nlKCbU4e9MIlI+1KAdxE0mDqNjOAZvMttC/xWvnopX1D8Lg7l10+q+ZmIpJMCvIsgwbRS0LYFzk6R\nxT5t/soXlCCvH+XCJCLtSwHeRdQeM0Ha/Dr7vbz66jybNp1idNQ72DsvKLVeP8qFSUTal8okPYTd\nas5ZahW2giWXg23b+ti3rzpPvmXLqVAVMl5lmG6vU4+0l5aF0UljBY03bVQmGUE9PdnDNuvKZmHP\nnlOce27B9YIS5vX9Wv6KSGfRDD4mXrOAKLP5OOrXk66DT/usJ4xOGitovGmjGXwTRF11GlerXrX8\nFREF+Do4Z8mVwqw6FRFJgqpoInBbLXrnnYRedSoikiQF+AjcVos++CChV52KiCRJAT6kILNz1aOL\nSCtQgA8pyOy8WZtxaxMREXHSTdaQyrPzycnq/WMrV51CY+rRW6lPvIi0Dv3zD6k8Oy9XxDg5Z+f1\nLJQKSxU7IuKmbVM0SaYr3HrC3HEHvqtOk0zLqGJHRNxEmsEbY84BHgNWA0vAJmvtryqO+SLw5xT3\nbP2BtXZHfacaTCPSFW6z89WrB5mZief1wwhyT0ALnkQ6U9QZ/JeA9621VwF/C3zL+aQxJgvcA1wD\nrAWuNcZcVs+JBtXIDS+Snp0HoYodEfESNcBfAzxV+volYJ3zSWttDvi4tfaEtbYAvAecF/ksA+rE\ndEWzKnZEpPVFTVpcAMwAWGujAP5IAAAHt0lEQVTzxpiCMabXWnsmCW2tPQFgjPk4cDHwY78XXLky\nSyZTfeMyjF/+Eqam3J87dqyHxcVBhofregtyueL2fCMjVAXP4eHB+l48ou98B1asgGeegclJGBuD\nG2+Ee+/tI5Opbh0cl2aNtxk6aayg8baLmgHeGLMZ2Fzx8O9XfO+6NNMYswZ4HPiStfa03/vMztY/\nvc5kYHQ061HCuEQmk4ucJ6+V2292R7odO2Dr1uUdJGdnk3u/Zo+3kTpprKDxpo3fxalmgLfWPgI8\n4nzMGPMYxVn8z0o3XLucs/fSMRcCTwN/aq39afjTDi9oCWMUaShFVAdJEXGKmoM/CNxS+voG4F9c\njvkH4GvW2p9EfI9IgmybV0tliWUn5vZFJP2i5uD3ARuMMa8Ap4DbAIwxdwE/pHhTdT3w340x5Z+5\n31r7/brONoB6Fhh5pWFuu+10zVLE1atjHISISAwiBXhr7RKwyeXx3Y5vG16/UbmLUdh0hVcaZnGR\nQO0JRERaSVusZHXrzz4+3numP3sQfmmYF1/McO21KkUUkXRpi140YW6AVu7CVP661orQzZtPc845\n2sxaRNIj9QG+1g3Q7dsXyGaX59ePHu2mv79Q+vkuRkfzbNiw6JuGGR0tNKx5mIhIHFKfogm6e5Kz\nhUGh0MXcXDdzc91n2hk8+mgf557rnkt3pmFaoT2BiEgQqQ/wQXqx+M3ynd5/v4tNm+orsRQRaRWp\nT9EEWdx0+LD3LN9perqbr371NH/910rDiEj6pT7AQ+3dk4aGCpx/fp7paf9eN+UZv1aEikg7aIsA\n77W4qVw+eeBAhunp2jN4lTyKSDtpiwBfVjnzriyfPKvAwEDxuJMnu1TyKCJtqa0CvJPfjdWRkTwv\nv5xjxQqUaxeRttW2Ad6vfPLdd7s5fryL884rKNcuIm0r9WWSXrSVnYh0urYN8NrKTkQ6XdumaKB2\n+aRTZSdKEZG0a+sAH6Q3fK2t+ERE0qojQpjfwqU0bMUnIhJF2+bgg9BWfCLSziLN4EsbbT8GrAaW\ngE3W2l95HPs94JS19raI55iYIJ0oVUYpImkVdQb/JeB9a+1VwN8C33I7yBizAfhIxPdInEopRaSd\nRQ3w1wBPlb5+CVhXeYAxpg8YB3ZFfI/EqZRSRNpZ1JusFwAzANbavDGmYIzptdY670p+A/ifwPEg\nL7hyZZZMxr/bYxK+8x1YsQKeeQYmJ2FsDG68Ee69t49Mxq2Pjbfh4cGEzrI1ddJ4O2msoPG2i5oB\n3hizGdhc8fDvV3zfVfEza4Dfs9buNMZ8NsiJzM42747mjh2wdevyOvjZ2XCvMTw8yMzMiWROsAV1\n0ng7aayg8aaN38WpZoC31j4CPOJ8zBjzGMVZ/M9KN1y7KmbvfwhcZIz5MTAEDBtj/tJa+z/Cn35j\nqAe8iLSbqCmag8AtwAvADcC/OJ+01j4APABQmsHf1srBXUSkHUW9yboP6DHGvALcTjHfjjHmLmPM\n2rhOTkREousqFFojLTEzc6I1TiSitOfxwuqk8XbSWEHjTZvh4cEur+c6eiWriEg7U4AXEWlTCvAi\nIm2qZXLwIiISL83gRUTalAK8iEibUoAXEWlTCvAiIm1KAV5EpE0pwIuItCkFeBGRNhW1m2THa5d9\naYMIMlZjzBeBPwfywA+stTsafZ5xMMbsAa4ECsAd1trXHM9dC9xN8b/B89babzbnLONRY6yfo7gV\n5xJggc3WWvf9LVPCb7yOY74FrLXWfrbBp5cIzeCja4t9aQPyHasxJgvcQ3Erx7XAtcaYyxp+lnUy\nxnwGWGOtXQt8GXio4pCHgD+iuEXlxjSOsSzAWPcCN1tr1wGDwB80+BRjFWC8lH6fn270uSVJAT66\nttiXNiDfsVprc8DHrbUnrLUF4D3gvMaeYiyuAZ4GsNa+Caw0xgwBGGM+DPzaWjtZmsk+Xzo+rTzH\nWvJJa+3R0tczpPP36VRrvAD3Aan85OlFAT66ZfvSAgVjTG/FMaH2pW1hNcdqrT0BYIz5OHAx8OMG\nn2MczoyzZKb0mNtz7wIjDTqvJPiNFWvtcQBjzAiwkeIFLc18x2uMuQ34IXCkoWeVMOXgA2jUvrSt\nIMpYHT+7Bngc+JK19nQCp9donn22azyXRlXjMcZ8CHgW+Lq19r3Gn1KizozXGPPbwCbgWmC0aWeU\nAAX4ADplX1qIPFaMMRdS/Aj8p9banzbodON2DMesDlgFTHs8N1p6LK38xkopfXEA2GGtPdjgc0uC\n33ivBoaBCaAP+IgxZo+1dmtjTzF+StFEV96XFjz2pbXW/gdr7ZXA14HnWj24+/Ada8k/AF+z1v6k\nYWcVv4PAzQDGmCuAY+XUk7X2CDBkjLnYGJMBvlA6Pq08x1pyH7DHWvvPzTi5BPj9bv+Xtfay0r/V\n/wz8pB2CO6hdcGTGmB6KM901wCmKG4tPGmPuAn5orf2R49jPlp6/rRnnWq9aY6V4U/WnwP92/Nj9\n1trvN/xk62SM2U2xkiJPcb/hTwAfWGufMsZ8mmK1EMA/WWvvbdJpxsJrrMALwCzwI8fhj1tr9zb8\nJGPk97t1HHMx8Fi7lEkqwIuItCmlaERE2pQCvIhIm1KAFxFpUwrwIiJtSgFeRKRNKcCLiLQpBXgR\nkTb1/wGRpN54z1Df0AAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7f72f0de7e50>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "_RB5WmiLh9lm",
        "colab_type": "text"
      },
      "source": [
        "Chainer provides an abstraction called `Link` that describe some computation and keeps track of parameters for it. For instance, a `Linear` link describes a linear map on input and keeps track of `w` and bias `b`."
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {
        "id": "TTM77r6-h9lm",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "output_extras": [
            {
              "item_id": 1
            }
          ],
          "base_uri": "https://localhost:8080/",
          "height": 87
        },
        "outputId": "3e2d406b-a17c-4455-c304-1cb476560adf",
        "executionInfo": {
          "status": "ok",
          "timestamp": 1509629023674,
          "user_tz": -480,
          "elapsed": 711,
          "user": {
            "displayName": "Mike C",
            "photoUrl": "//lh3.googleusercontent.com/-xWyBhueI0Vo/AAAAAAAAAAI/AAAAAAAAIg8/rU-qliu5l-M/s50-c-k-no/photo.jpg",
            "userId": "110113503404408134511"
          }
        }
      },
      "source": [
        "model = L.Linear(in_size=1, out_size=1) # input is 1D data and output is also 1D data\n",
        "\n",
        "# Chainer will randomly initialize `w` and `b` for us.\n",
        "# we can take a look at their values\n",
        "print(\"w:\", model.W)\n",
        "print(\"b:\", model.b)\n",
        "\n",
        "# model.W and model.b have type `chainer.Variable`, \n",
        "#   which is a wrapper around Numpy array\n",
        "assert isinstance(model.W, chainer.Variable)\n",
        "\n",
        "# operations that involve `chainer.Variable` will produce\n",
        "#   `chainer.Variable` and this records the computation graph\n",
        "var_result = model.W + 123 # some random computation\n",
        "print(\"Operations on chainer.Variable: %s, type: %s\" % (var_result, type(var_result)))\n",
        "\n",
        "# the underlying numpy array can be accessed by `data` attribute\n",
        "print(\"numpy arrays:\", model.W.data, var_result.data)"
      ],
      "cell_type": "code",
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "('w:', variable W([[-2.68564606]]))\n",
            "('b:', variable b([ 0.]))\n",
            "Operations on chainer.Variable: variable([[ 120.31435394]]), type: <class 'chainer.variable.Variable'>\n",
            "('numpy arrays:', array([[-2.68564606]], dtype=float32), array([[ 120.31435394]], dtype=float32))\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "RYSYYK1Wh9lp",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "output_extras": [
            {
              "item_id": 1
            }
          ],
          "base_uri": "https://localhost:8080/",
          "height": 281
        },
        "outputId": "db206472-f289-4691-d7bb-cd1a06c899cf",
        "executionInfo": {
          "status": "ok",
          "timestamp": 1509629025385,
          "user_tz": -480,
          "elapsed": 713,
          "user": {
            "displayName": "Mike C",
            "photoUrl": "//lh3.googleusercontent.com/-xWyBhueI0Vo/AAAAAAAAAAI/AAAAAAAAIg8/rU-qliu5l-M/s50-c-k-no/photo.jpg",
            "userId": "110113503404408134511"
          }
        }
      },
      "source": [
        "# A chainer link is a callable object. calling it performs the\n",
        "#   forward computation. (in this case, it performs Wx + b)\n",
        "model_y = model(data_x[:, None]) # chainer's link usually assumes input is [Batch Size, Input Dimension]\n",
        "# `model_y` is a chainer variable so we use `.data` to access its numpy array for plotting\n",
        "\n",
        "# we can plot the model's current fit in red. it should be terrible because we haven't trained it yet\n",
        "_ = plt.plot(data_x, model_y.data[:,0], c='r')\n",
        "_ = plt.scatter(data_x, data_y, c='b')\n",
        "_ = plt.title(\"Initial model\")"
      ],
      "cell_type": "code",
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAEHCAYAAACk6V2yAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xl4U1X6wPFv0rRp04UWrAgFQf3J\nGfdtFFkKyqY4oCMOgjiiIIOKCiIiiCioyOICirsi4q6MIoICIojK4u6MK3PUGReWCijdU7olvz/S\nYGizN0tz836eh8c2ubn3HJO+9+Q9m8npdCKEEMJ4zPEugBBCiOiQAC+EEAYlAV4IIQxKArwQQhiU\nBHghhDAoCfBCCGFQEuBFQlBK/aSU6hngmAKl1NcNP1uVUiMbPx7g9e8qpf4emRL7vU4HpVTA8cnB\n1FkIfyzxLoAQkaK13gEc2/DrScBI4JlGjwuRNCTAi4SjlHoXWAEMAQ4D3gdGAJ2AH4AC4DUgRym1\nEbgE+EFrbVFKmYEHgH5AGrAJGK21rvVzvSXADqA7rhvFE8D/gAlANjBUa/2JUqo18ChwAlAPPK21\nntdwjtHADKAMeN7j3CbgFuBiIB1YDlyvta5v1v8kIZAUjUhcg4H+QBegD67gC4DWehdwE/CB1rqw\n0evOBwpxBeqjgFOAYUFcbyAwCDgTuBHI11ofB7wCjG84ZjZQrLVWQE9gnFKqp1IqD1gInN3wmvYe\n5/07cCFwGnBEw7+rgvkfIEQgEuBFonpFa12lta4EvgMODeZFWutXgT9rrWu11vuAT4DDg3jp2w3X\n+gbX383Khse/4o+A/Rfg4Ybr7AWWAQOArsD3WuutDcc97XHewcBirXWp1roOWITrm4kQzSYpGpGo\nSj1+rgdSgnmRUiofeEApdTLgAA4B7gvipeUAWmunUsoBVHi5dj5Q7PGaYlzBv3Wj8noekwvcoJQa\n2/C7BdgTTF2ECEQCvEg2dwK1wHFa62ql1POBXhCCXUAb4JeG39s0PFYMtPI4Lt/j553ACq31gxEs\nhxCApGiEcdXi6mQ1NXr8YOCrhuB+AtADyIrQNd8AxgIopQ7ClWp5E/jU9ZA6suG4Sz1e8zpwiVLK\n1vC6K5RSns8LETYJ8MKoNuFKj+zkwPTNvcCVSqmtwNXAJGCMUmpoBK45HchTSv0H18ieuVrrj7XW\nexqus65hPL72eM1yXPn8zxtedy7wVgTKIgQmWQ9eCCGMSVrwQghhUBLghRDCoCTACyGEQUmAF0II\ng2ox4+D37ClP6N7evDwbxcX2eBcjZpKpvslUV0iu+hqhrvn52Y2HAu8nLfgIsViCmkhpGMlU32Sq\nKyRXfY1eVwnwQghhUBLghRDCoCTACyGEQUmAF0IIg5IAL4QQBiUBXgghDEoCvBBCGJSxAnx9Pba7\n55D6zrp4l0QIIeLOUAHeVFGO7aH7yR0+hOzxV2EqLYl3kYQQIm4MFeCdrXIpfnMdtcefSPpLz5NX\n2JW0t9fEu1hCCBEXhgrwAPXHHEvJ6vVU3nQL5t9/o9XFF5J9zRWYSooDv1gIIQzEcAEegNRU7BMn\nU7xuI7UnnET60hddrfm3Vse7ZEIIETPGDPAN6o86mpLV66m4eQbm4r20umQY2eP+gal4b7yLJoQQ\nUWfoAA+AxULVhEmu1vxJJ5P+ysuu1vzqN+NdMiGEiCrjB/gG9X86ipI311Ex/TbMJcW0uvQisq+8\nHNPe3+NdNCGEiIqkCfCAqzU/fiLF6zdRe/IppC/7J60Lu5L25sp4l0wIISIuuQJ8g3r1J0reeJuK\nW27HVFZKq1EXk33laEy/S2teCGEcSRngAVdr/trrXK35U/5M+rJXaF14GmlvrIh3yYQQIiKSN8A3\nqO+iXK35GbMwlZfRavTfyR57mbTmhRAJL+kDPAApKVRdPZ7idzZT++fTSF++jNaFp5K2cnm8SyaE\nEGGTAO+h/sgulKx8i4rbZmOqqKDV5SPJHnMppt9+i3fRhBAiZBLgG0tJoeqqayjesJnaU7uSvuI1\nV2t+xWvxLpkQQoREArwP9UccScmKNVTcMQeT3U6rMZeSc/lITHv2xLtoQogEZrfDjz+asNujfy0J\n8P6kpFB1xdWu1nzXblhXLqd14alYX3sFnM54l04IkUDq6mD69DQKC21065ZJYaGN6dPTqKuL3jUl\nwAeh/vD/o+T11VTcOQ9TVRU5V4wmZ/QlmHbvjnfRhBBxFmyLfObMNB5/3Mq2bSk4HCa2bUvh8cet\nzJyZFrWySYAPltlM1T+uYu+GLdR064H1zRWu1vyyf0prXogkFEqL3G6H1astXs+zerUlaukaCfAh\nchx+BKWvvUn5nLsxVVeTc+Xl5Fx2Mfz6a7yLJoSIoVBa5Lt2mdixw3u43bnTzK5dpqiUUQJ8OMxm\n9l1+has1370n1tVvwDHHYH3lZWnNCxEFseyYDEaoLfK2bZ0UFDi8Ht++vYO2baMTNyTAN4PjsMMp\nXfYG5XPugepqcsb9g5xLL8K8S1rzQkRCPDomgxFqi9xmg4EDvRd64MA6bLaIFxGQAN98ZjP7Lh8L\nX35JTY9CrGtWkdfzNKxLX5TWvBDNFImOyWi0/kNtkdvtcOmltYwaVU3HjvWkpDjp2LGesWOrmTmz\nJnIFa6RZAV4pdaxS6r9KqWu8PNdPKfWxUuoDpdQtzblOQjj8cEpfXUn5vPmYamvJueYKci4ZhvnX\noniXTIiE1NyOSW+t/6lT0/jhB//BPpgbQrAtcs8yFBZmsm6dhf7969i4sZKNG+3MmlWDxXsVIyLs\nAK+UygQeANb7OGQhcAHQAxiglDo63GslDLOZfaPGsPe9D6jp2Qvr2jXkFXbF+vIL0poXIkTN7Zj0\n1vpfvNhK9+7eUz2hpoNmzqxh7Fj/LXJfZViyJDVqaRlPzWnBVwPnADsbP6GUOhzYq7XeprV2AKuA\nvs24VkJxdOpM6SsrKL9rAdTVkXPtldKaFyJEzemY9Nf6hz9SPTfc8MfxEydaQ0oHWSwwa1YNGzfa\n2bKlaYs8XkMjDyhjuC/UWtcBdUopb08fAnjO6d8NHOHvfHl5NiyWlHCL0yLk52cf+MDk62DoX2HM\nGKxr12Dt1RUWLIBLLwVTdIZFxVKT+hpYMtUVWk59hwyB++/39ngKnTr5LuN//ws7dgQ+//LlUFub\nzRtvwC+/eD9m7VorCxZY/ba4O3Vy/dduh6IiaNcOysp8l2HnzhTq6rLJzw9cxuaIYvbnAAGjWXFx\nCxn/FKb8/Gz27Clv+kRmG3hhGenPLiFzxs2YR42i+rkXqLh3IY72BbEvaIT4rK8BJVNdoWXVd8oU\nqKpKY/VqCzt3mmnf3sHAgXVMmVKDv2WhLBYoKLCxbZv/RuPPP8PDD/svwy+/OPn000qOOsr3N4a6\nOlc6ZvVqCzt2mCkocNC/fx0FBRavZWjfvh6Lxe63DsHydzOO1iianbha8W4FeEnlJA2TiX0jR1H8\n/ofU9D4T6/q3ySvsSvoLz0puXgg/AqVBfPHXCeopJYikgcMBF1+cESAf7z3X3qqV97/vaA6N9BSV\nAK+1/gnIUUp1VkpZgEHA2mhcK5E4Oh5K6dLllM9/AJxOsq+7mlYXXYB5x/Z4F02IFs1mg8MOc4YU\nFD07QcF7oK2vD+ZMJrZv952P95drLykxxXxopCeTM8wWpFLqFOBeoDNQC+wAVgA/aq1fU0r1AuY1\nHP6q1voef+fbs6c8oZuyoX6tNW/fRvak8aRtWI8jO4fK2+5k38UjEyY335K+xkdbMtUVjFdfux12\n7DCxaFEq69b9kerp37+O9eut/Pyzt1c58ZZZ7tixno0b7QfcaH780US3bpk4HE2PT0lxsmVLJW3b\nOtm1y0TbtqHdpIKRn5/tM2iEHeAjLdkCPABOJ+kvPkfmLTdhLi+j5ow+lM9/AEeHjtEpZAQZLQj4\nk0x1hcStr91OwCDa+Jg778z22onrK8C7A/Zhh/0Rrux2KCz0nu/3dkOINH8BXmayxpPJxL4Rl1C8\n8SNq+vQj7d13yOt1OunPLpHcvBBBCmX8umeqx26Hq66iSQpl9OhqOnYMfnhmvJYhCEasRtEIPxzt\nCyh98VWsL79A1vSpZE8aj3XFa67WfMdD4108IcIWTKu6ueefMsXKyy//kRt3jV93taZnzWqa6z5w\nxAsUFLhml44ZU0v79q5yWizsP4cnXwHbnVNvPNonVrl2XyRFEyGR+lprLtpJ1qTxWNetxZGZReWM\nO9h36egWl5tP1K/x4UimukJ49W0cyL0NG3QHvEhMzXef/803LQ2zXYPLl4Ortf/449Ymx48dW73/\nhuBZ/sYB21/5o31D80Zy8DEQ0SDgdO5vzZvLSqkpPIPyBQ/gOLRTZM4fAckU9JKprhBafX0FcocD\nFi1qGkSHD69m7tyaZgc/X0Hak9nsZMOGA8evh5ov9wzY4P3neKZgQAJ8TEQjCJiLdpJ1wwSsb7/1\nR2t+5Cgwx7/rJJmCXjLVFUKrr69Aa7M5sNu9fU6ddOjg4Jxz/mgNh9rq9RekA10rmBEvnh2ocOBN\nbPt2M5mZzoZymCL+zSQc/gK85OBbMEe79pQ9txTr0hddufkbJ2Jd6RpH7+jUOd7FE0nO3/hvu91X\nzHGPKU/B4XC1VUJN4/z8s+9FyHxdC1z5ePf6Nt5nl3pf38Y9icmtouKPugXK98db/JuCwj+Tieph\nIyje+BHVA84mbeN7tO7djfTFT7im2AkRJ/5WewxidRJeeik1pMW93KNlRozI8PPR954IcC/uFeqI\nF/+LljU9f0sjAT5BOA5pR9mzL1P20OM401LJnjqJVhcMxvzTj/EumjCgYNZE97faYzA8W8KeVq60\n8PvvTR93t6R37EghmBuIJ8/lhQ9c5he/s0v938S8n9+tJWwzKAE+kZhMVA8dTvHGj6k++xzSNm+k\n9RndSH/yMWnNi4ioq4PrriPoMeXBrPfiq1XtS1GRmT59Dryu/5a0kw4dgh+/7rm+jdb4Xd8m2JuY\n5/lb0jaDEuATkKPtIZQ9/SJljyzCabWSfdNkWp3/F8w//i/eRRMJzL0m+v334zNt4tkq9dyGrqDA\n93ovNpv3x7OyfAV+E0VFrutef33a/k5YXy1psxmef76KuXNrQkq/2GxwxBH47dgN9ibmef5IbDMY\nKdLJmqhMJqovuJCanr1dna+r36D1md2pvHkGVZdf0SJG2ojIitYY66ZjyptatcpCbS2sW+d9JMlZ\nZ9VRUWFi6dKmQWzEiNr9nameY8p9DaX09NJLaWzaZGHAgDqfnaMFBQ46dXKVJxoTjjzPuWOHef8N\nq6rK1OT8gTb5mDat+UNEQyHDJCMkrkPpnE6sy18l66YbMO/dS83p3Sm/7yEch/vdY6VZkmnoYLzr\nGu6koWBvCMGMKfe1NounMWOqvQZyX8Mh3fVaudJCUZH3yUqejj22jq+/blphzwlKbsHWPZT31teY\n+FAXHms8DLO5ZBx8DMQ7CACYdu8me8r1WN9cgTMjg8ppt1I15srgFr0OUUuob6zEu67BzLz0FMoN\nIdgx5SkpTurr/Qdg90QhCO2bxu+/Q58+NoqK/JehQ4d6+vevO2BFyOaOQY/0exuPhcdksbEk4Tz4\nYMoWP0vZE0tw2mxk3XITuecNJOW/38e7aCJM4ezrGUoOONhRIsGsm+4eSRLq2u1t2sDgwYHz3EVF\nZq68sjbkzT9iqaUtPCYB3mhMJqrPG8Le9z9m37nnk/rxh+Sd2YOMRx4MdncD0YL4C8C+huaFckPw\nP0ok8OgUT4E2wvbHPXTRX2et+/zhbP4RSwcOw4z9Jh+eJMAblDM/n/JFT1O66GmcWVlkzZhG7rln\nk/KDtOYTib8A7C2ghnpD8NfiHD68hk2b7Nx6aw3duwduHDSnheoeurh5s51hw2ojfv5YCnebwWiQ\nAG9wNeee72rNnzeE1E8+Iq9PDzIeWiit+QQR6lf+UG8I0LTF2bmzK79/1101zJ7tGs+9dGkqWVkO\nsrIcmM3O/T9HuoVqs8GCBdUtpgXcHC3hm4Z0skZIvDvigpG2cjnZU67H/Ntv1J5yKuULH6H+yC5h\nnSsR6hsp8a5rqEvXhtop6+YeJXLssVlUVpb7PI97RUiI/lrv0V6xMd7vbSTIKJoYSJQPium338i6\neTLpr72K02qlcsp0qq66JuSRNolS30hoKXUNdks6X/uPem5o4U9+fjY//1we123oYqWlvLfNIaNo\nxH7Ogw6i/LGnKF38HM7sHLJuv4XcQf1J+U7Hu2jCC8+Zo/6+8ntOjy8szGTdOtcuRRs2VNKvXx1v\nv22hZ8/gp82HmssXLZME+CRVM+hc9m76mH1DhpL62afk9e1JxsIFxGXBDHEAux2+/97ElCm+1zNp\nvJCVt6GRixdbGTcunaeeCn3afDi5fNHytKARpCLWnK3bUP7ok1Sfez7Zk68ja9YMrG++TvnCR6lX\nf4p38ZKOZ65927YDZ3a61x33toZ6//6uFro3W7d6T70Fmjbv7twNZV/SUMRja7tkJDn4CEn0XJ5p\n7+9kTbuR9GX/xJmWRuWN06gaNx5fY7sSvb6hiFVdg1kyICvLQUWF952SvE/19/64v2nz7vqGuy+p\nP9HeqzVURvgcSydrDBjhgwKQtvpNsiZfR8ruXdSeeBLl9z9C/VFHNznOKPUNRizqGso2dL4Ctrel\nBHw97q+jtHF9I9naDneET7QY4XMsnawiaDUD/0Lxxo/YN3Q4qf/+F3n9e2FbcLfk5qPAM48e7JIB\nvvia1nDUUd6fcKdZgtmUIlLjucNZdkE0jwR40YQzrzXlDz1O6XMv48hrTeacO8gd2JeUb7+Jd9EM\nwduGEI89lhrUxhK+11DH6+SjVauqmkwaGj26mr//vdZvJ240yMic2JMUTYQY4aueN6aSYrJuuYn0\nl1/AmZqKfdIU7NdOJL99a8PUN1AKItLvra80ha/lcMFJx47Br6HunojkWZfG4+Mbd+K6jR1bzWOP\nWaPy3sZjpcVAjPB3KykaETZnbh7lDzxK6fNLcbQ5iMy5s8g9uw98+WW8i9Zskd5aLZh0h780RUmJ\niVGjmra2Pdczuf32wItybd7c9Pw2Gzz9dOr+IZO+1l6PZqqkpa20mAzCbsErpRYAp+P6lE3QWn/i\n8dxPwDbAnQC8WGu9w9/5pAXf8plKS1yt+Zeeh9RUKidOxj5hEqSmxrtoYQm2wy/QexvKyJCtW02c\neab/DSHatnUG7NQM5jyeI2RCWfddaxM5OdH5LEdjZE5zGOHvNuKjaJRSvYHJWutBSqmjgMVa624e\nz/8EHKu1rgj2nBLgE0faurdoNfk62LGD2mOPd61pc+xx8S5WSEJJFwR6b4O5UTTdFi+0kS3NKT/4\n32mo8Wv/858UKiujP2qoJYyDN8LfbTRSNH2B5QBa661AnlIqJ8xziQRT0+8s+PprqkZcQurXX5I3\noDe2u+dATeKs9hepDr9gR4a4Z5ru2OE7PRJKmiKSq0yGW4bmaAkrLSYFp9MZ8r8uXbo83qVLl/M8\nft/YpUuXLh6//9SlS5fnunTpsqlLly5zu3TpYgp0ztraOqdIQKtXO50dOjid4HSecILT+a9/RfwS\nlZVO5w8/uP4byXN27uwqduN/nTsHf60ffnA6zWbv50lJ+aPcnTp5PwZcz02Y4HTW1oZWh9pa1+s6\nd3Zdq3Nn/+eZMMH79U2mwK8VLZrPuBqprFfjJsmtwBpgL66W/gXAK/5OUFyc2INgjfBVLxT763tK\nD0zvfkDmzOlkPPc0zlNPxT5hEvaJkyHN/3ongUR71uOAAd5TKwMGVFNZWUNlpet3f++txQIFBd5T\nJe3b12Ox2Pn0UxPbtmXireVuNjt55plKOnVy8vnnoacsbr4ZJk48MN1RXOz92ClToKrqwPx341Um\ni4uT67NshLrm52f7fC7cP5OdwCEev7cHity/aK2fcf+slFoFHEeAAC8SlzOnFRXzH6B60HlkTxpP\n5r3zsK5+k/IHHqHuuBPCPq87reHmXo8FiMisR/cGEt46/ILlb82Wbt3quO22NNauteDwkR1p397B\nkiWuoYvh3sTc6Y5A3DsNTZtW0yLy3yL6ws3BrwX+BqCUOhnYqbUub/i9lVLqLaWUu/nWG/i62SUV\nLV5tn34Uv/8hVZeMwvLt1+QOOAPb3Dugujrkc8Vi1qO3rdWmTath2zb/Qx0b89wRyXO3o6VL03jq\nKf9599xcZ1irPTaH5L+TR1gBXmu9BfhMKbUFWAhcrZS6TCl1vta6FFgFfKiU2gzsQVrvScOZnUPF\nvfdTsnQ5jnbtyZx/N3kDemP54l8hnSeWsx5tNujY0bl/e7pgluf15HmjGDq0looKc8OCYL7K+Mdm\n1qWlsR+PLpKHzGSNECPk8kIRTH1N5WVk3j6DjKefxJmSgv3aidgnTQGr/5mYEPtZj76GOo4ZU01m\nppVly+oDplCCHWtuNjvZsKGS9HR8Dl30t9pjtCXTZ9kIdZWZrCIunNk5VNy9gJJXVuBoX0DmffeQ\n178Xln9/HvC1/oYB9ugR/FTT5s4ufemlVO6/n6BSKMEuGFZQ4KBTJ6dsqiGiTgK8iLraXmdQ/N4H\nVI0ag+U/W8kd2JfMWTMD5ub95bb9LSsQzI5InvwF5oqK4FMooY41l6n7ItokwIuYcGZlUzFvPiXL\n3sBR0AHbwvnk9SukbsunIee2fbWkPdeW6dEjM+jOy2ADsydv/QD+ArZrwTDXCo+eo3Q8b2Keq0CG\nMpJHCF8kBx8hRsjlhaJZ9a2owHb7DDKXPEE9Zu7hBhZ1uJU+51hCzm175uOD2RGpXbt63nnHTps2\nBz7u67W+dlByXxcOHIPuba2VxmPNvWkpU/chuT7LRqir7OgUA0b4oISiOfW122HKFCu/vryFxYzm\ncH5kK39iFE9xwtiTmoxx97eOiucCXcHuiNSunauj1DPo+loEy9fyvBdeWE1mJj7Hr7ekgB2qZPos\nG6GuEuBjwAgflFCEU19vC27ZqGQONzGeB6jHzBPZ13HOJ1NwpmfsD5AQuAW/a1dwi2kd6I911n0F\n5ro6mDcve/8oGpvNVR5Xbt77eurx2HoukpLps2yEusooGtEieFtwy04mE1hIb97lJzpzZfl8Uk4t\nZMJpX+zvHJ09O42zzvLfGRlOHh2a5uY9JwHZ7bBtm4nZswl6jLuMXxctiQR4ERP+hiICvE9vjudL\nHk69loLy71i2uxfzHJPZs61mf27cX2dkoA7OQDwDc+ONQI45Bm6/PY0tWwKlf2TrOdGySIomQozw\nVS8UodY32PXIs7IcnFixmcWM5kh+QNOFUTzF9o5dm3Rqev7sK49+5pl1rF1r4ddfA2904Z5Y5Luz\n1onv2aku8dp6LpKS6bNshLpKikbEnf8Uimvq/vDh1djtJjZRyAl8wQKu40i+ZxM9mbh9Mnt+rvK7\nrAA0XVvmnntqOPfcwBOj3BOL/H3TSAncgJfx66JFkQAvosZzFqm/FMrw4TVs2mRn7tya/TeBKmxc\nzwJ68T4/8H9MdC7ghMu6Y/nwg/25fF9j3BsvpuU51txXusYdmP1Neqqv9/owvsa4CxFvEuBFxPna\nzHr6dO+TeubPr/E5s3MzPTmRf/P2cROw/PQ/cs87m1NfuJEMmvZk+urg9JwwtXlzZZONrT0Ds79v\nGh06OAJuih2PfUWF8EVy8BFihFxeKPzVN9Aepf7GiPvblDn9849IH3cVGb/8wPf8H6NZzCYK9782\nlAW6/JWhOeU3gmT6LBuhrjIOPgaM8EEJha/6/v479Oljo6ioeatA+gqiVXurWHXqPMaWLwDgAa5l\nGrOxk+lzlmqoGt9kOnY0MWBAdcR2kmrpkumzbIS6SieriDp3WsYV3Ju/jruvTSkyWmfwr4tm04PN\nfEcXJrCQLziBQt6nqMjMgAG+FxULVuONQL75Bkm/iIQkAT5BBLPsbTReGyx3x6er5e49iEdqCdyZ\nM2s4fuzJDC74lHlM5jB+5H16s5Dx7N1WFbEdkWTnI5HoJMC3cL46LINpoTbntYF43jQCTWJyi9QQ\nQncLe+U6J/e3m0N3tvAtR3EtD/Ilx9Obd2VGqRCEv+m2iJFAG0/76/ALZdPqYDsO3fnptWvhl18y\nKShw0KNHnZ+NLlyLew0eHNpm1sEoKzOxa5eZIrpyMp8zg9u4kbt4lzN5ePtV7PnxVjodkxnRawqR\nSKSTNUKC6awJdfSFv2VyO3Sop3//Or+rGfp6rWdnpGeHYqAt6cD3CBObzYHd3jTIR6rj0xtvdTyV\nj1nCZRzNVuo6dKJi4UPU9uzVrOsYoSMuFMlUXyPUVTpZ4yzcVIm/STfbt5v9bmjh77VFRWb69HGV\n4dZb/U8aCjYVY7d7/4wNHlwXdHAPta/A27j5TziNk/mc1SfeQMrObeQOGUTWjROhoiK4kwphIJKi\niYFQUiWe3JNuvLXCU1K8z6xcvdrCtGk1fl8LJoqKXGXIyvI+qWfVKgu1tQeud+4/FeMZ4A9chjeQ\nUL9FeHKf/8Bx8yZOnHkrJV/9hewJ48hY8iRp76yjfMGD1Bb2DlgeIYxCUjQR4uurXrC7EfkS6sJX\nwS2aFfg8vh73tcORJ29pmeZMLAqGz/NXV2ObPw/bwgWY6uupuvRyKmfcjjMrO6jzgjG+xocimepr\nhLpKiiaO/KVKghkX7m3PztGjq+nY0XvL23Moovu17dr5XoPFl2AW1vJl924zZWWuegVKT/lL+4Qy\nEsbnkEarFftNt1Ky5h3qjjqajKefJK/X6aS+tyHM2gmROCTAR5m/tU0CjQt3bzgxbdqBKyTOnVvj\nc+Euz6GI7uGE77xjp107Xys5er/B+FpYy243MWxYDYceCr5uGgfeZPzn+Jt7AwxW3QknUbz2PSqv\nn4y5aCe5Q88ja9J4TOVlETm/EC2RBPgo87eKoq9x4d5avbNnp9Gxo/cVEr0tmuWpTRtXZ2dgrmV7\nR46s5pBDvN8Q2rZ1MHNmNVu3wrBhtX7rFUzrvDk3wJBZrdin3kLJWxuoO/pYMp5d4mrNb1gfuWsI\n0YKkzJw5M95lAMBur5kZ7zI0h8lk5bvvarFaITX1wOd69aqnvBx27zZRWWmiQwcHw4fXMnNmDWYv\nt1j3yJayMjNOp4myMjOffWbUzRXWAAAZg0lEQVShvBz69HE1rc1m18+XXFLL8OG1XHttLWedVe/1\nfJ5l2LnT5HM/UZMJBgyoY9MmV4elt2MqKkysWOF6fsYMO5WVvuu1fbuJBQusOJ1Nz1NZaWL48FoO\nPtj1LeWzz5reCIYPd9Up0hxtD2HfiEvAZCJt/dtkLH0Rc9FOarv1AGt6k+MzM63Y7cmzDHAy1dcI\ndc3MtN7m6znpZG2mPyb+WPnlF6ffESDBjINvbqdsIP4WA/Pdgeq9wzXQ6orB1sXfCpLRXv/F8tUX\nZI8fh+Wbr6hvX0D5/IXU9ul/wDFG6IgLRTLV1wh1lU7WKHLnmH/6Ca85Zk/BrG0S7Zx08OmaP/j6\nVuBOs/iqV7DpqcaLe8VybfW6406g+K0NVE6+CfPuXeQOv4CsCeMwlZZE/+JCRFnYAV4ptUAp9YFS\naotS6tRGz/VTSn3c8PwtzS9myxSpESCeYpGT9pa/d2+X543DR/9suKOAfPUVxG1xr7Q07JNvonjt\ne9QedwIZLz5HXq/TSVv3VowLIkRkhRXglVK9gSO11t2Ay4GFjQ5ZCFwA9AAGKKWOblYpW6hotLbD\n6ZQNlbcWs+d2eY35GjIZzA0nnq3zUNUfexwla96hcsrNmH/bQ6sRQ8kefxWUSGteJKZwW/B9geUA\nWuutQJ5SKgdAKXU4sFdrvU1r7QBWNRxvONFqbYfS6m0OzxazvxvLUUd57+gM5YaTMEvvpqZinzTF\n1Zo//kTSX3oejjmGtLfXxLtkQoQs3HbUIcBnHr/vaXisrOG/ezye2w0cEeiEeXk2LJZmzK6JkyFD\n4P77mz7ep08K+fnZfjtTi4qgXTvX7+6f3cc/9pjnMSnYbCmAtclrIxkwH3oIMjLg9ddh2zbo2BHO\nOw/mzrUwdWrTx++5x4rFEmimbII6oxt8+jHcdRfcdhutLr4QRo6E++6DvLx4ly7q8vODn+mb6Ixc\n10h9UfaXiwgqT1Fc3HIX7/Y3+mXKFKiq+mMUjc3marU/+6yJd95pOhrEc8TI9u1mMjOdDdcwUVDg\noH//OsaMqaV9e9e1cnKgstL1rzlrtgTr5pth4sQD61te3vTxTp0Sf/RBUMaOJ//cc6n9+0hSn3mG\n+rfWUnHP/dScNTDeJYsaI4wsCZYR6urvBhVuimYnrpa6W3ugyMdzBQ2PJZxgVoF055i/+QaGDq2l\nosJMRYXZ54gaz5mdTqepyfGLF1vp3t37tQLNCvXUnF2c/I2KSYg0S6Qddxwlq9dTOe1WzHt/p9Ul\nw8i+eiym4r3xLpkQfoUb4NcCfwNQSp0M7NRalwNorX8CcpRSnZVSFmBQw/EJJ5SACrBli/cUk3tE\nTbA7H4H3ZXuDGbETzV2cklpqKvbrbqB43UZqTzyJ9H++RF5hV9LWrIp3yYTwKawAr7XeAnymlNqC\na8TM1Uqpy5RS5zccchXwIrAReFlr/V1EShtDoQ6BLCoi4Igaf6NufHFfK9gRO6HelERo6o86mpJV\n66mYPhNzSTGtRg4n+6ox0poXLZLMZPXhxx9NdOuWicPhf0let8zMbP70p3q/szbB98xOX9zXatvW\nGXBWqL/zR2IWrCcj5C6D5auuKfo/ZI+/ktR/fY4j/2DK776PmnMGxaGEkSXvbWKRmaxhCHUIpL9h\nhj161AU8xhf3tYIZHx+rlRmFS736EyVvrqPiltsxlZXS6rIRZF85GtPe3+NdNCEACfA+BRNQG3dk\neo5fN5udZGU5yMpysHRp2v5c+PTp3o/xtfSu51jzQOPjY7oyo3CxWKi69jqK12+i9pQ/k77sFVr3\nPI20N1bEu2RCSIrGH1+LYE2fXsOsWQcOVxwyJIUpU8r3b3g9ZYqVl19umvf2tkAXwI4dJhYtSmXd\nusALbkV7d6RgGOGrbbCCrmt9PRmPPkTm3DswVVez769DqJhzL85o7DgeRfLeJhZ/KRoJ8EFoHFAD\nBdHmrAgZzIqT/soZyo2iOYzwhxGsUOua8v13ZE8YR+qnH+M46CDK582nZvBfo1jCyJL3NrFIDr6Z\nPMd/BzO6pjm58HDGmnsOjSwszGTdOgv9+9excWPLXvvFqOqP7ELJyreouG02pooKWl0+kuwxl2L6\n7bd4F00kGUMH+OZM9vElmOAd61y4t6GRixdbWbIkNfkmJbUUKSlUXXUNxRs2U3tqV9JXvEbrwlNJ\nW/FavEsmkoghA3w0J/sEE7xjsSKkWzSWLBaRU3/EkZSsWEPF7bMxVVbSasyl5Fw+EtOePYFfLEQz\nGTLAR3uyT/fugVdXjNWKkDI0MgGkpFB1ZUNr/rTTsa5cTutep2F9fRm0kD4wYUyG62SN1pZ3vhYJ\nq6oy0b79gaNoGpcn3E7TYER7iz9fjNA5FayI1rW+noxFj5I5+3ZMVVVUDzqP8rn34jz44MicPwLk\nvU0sSdXJGqkWbdMx7t4XCRs61LWZxX334bUjM9oLdMUyHSQiICWFqiuuZu+GLdSc3h3rG6+7WvOv\nvSKteRFxhgvwOTlO2rYNv4PTW/5+6tQ0n3nuzZvjPzwlVukgETmOw4+gdPkqKu6ch2nfPnKuGE3O\nqL9j2rUr3kUTBmKYFI1nCmXbNjPelqH3NckomDHurpmmvtelOe20rLh/1Yt2OsiTEb7aBivadTX/\n+D+yr7uatA8248jLo2L23VQPGQqm+PSfyHubWPylaOLf/IwQdwqlKScdO/4xA3X6dP+bbbz9tvf/\nJSkpUO+lb7UlLQHgTgeJxOI47HBKX3uT9KeeIOuOGeRcNYbq11+j4u4FONoeEvgEQvhgiADvb6hg\nu3YO1q6106ZN09Z5RcUfNz7X2PEUfK0J4y24g+S5RYSYzey7/Apq+g4g+7qrsa55k9QPN1Nx511U\n/21Y3FrzIrEZIgfvr2N1924zZWUmfv8dVq4MfD9L8bGSb4cODkaNkjy3iC5H58MoXfYG5XPvxVRT\nS87VY8kZORzzr0WBXyxEI4ZowbsnH3kbKtiunYNHH01lzRoLRUWB72e+WurnnFPXkL+viVmeWyQp\ns5l9o/9BTd/+ZF9/Lda3VpP64QdUzJpL9YUXSWteBM0QLXh/QwVzc5089ZSVoqIUgtn/O1BLPWn3\nJRUx5+jUmdJ/vk75XQugro6ca68k55Jh0poXQTNECx7YH4A9l/b112nqi7TURYtiNrPvssup6dOP\n7InXYl27htTCrlTcMYfqYSOkNS/8MswwSTfPoYK7dvnedg+c2GxOzOY/ZqP271/HmDG1tG8felA3\nwnCrUCRTfVtMXZ1O0p9dQuaMmzFXVlDdbwAV9y7E0a59RC/TYuobA0aoa1LNZPVMofhbGKxdOwef\nfVbJ119X8v77lfTr52rt9+wZ2cXJhIgYk4l9I0dR/P6H1PQ+E+u6teQVdsX64nMyC1Z4ZbgA78lf\nbn7w4DratHEd8/TTqTz1VPQWJxMikhwdD6V06XLK710IDgc5E8bR6qILMO/YHu+iiRbG0AEeAk/j\nl+V2RUIymdh3yWWu1vwZfUh7Zx15vU4n/flnpDUv9jN8gLdYYNYs14JgW7Y03eFIltsViczRoSOl\nL79G+YIHAcieeA2thg+R1rwAkiDAu/ka3hjr3ZeEiDiTiX0Xj3S15s/sS9qG9eQVdiX9uaelNZ/k\nkibA+yLL7QqjcBR0oPSlZZTf9xCYTGRffy2tLvwr5u3b4l00ESdJH+BBltsVBmIysW/EJRRv/Ijq\nfgNIe2+DKzf/zFPSmk9ChhsH3xzNWW7XCONpQ5FM9U3YujqdWF9+gazpUzGXlVJTeAbl9z2Io+Oh\nfl+WsPUNgxHqmlTj4JtDliEQhmIyUT38Yldrvv9ZpG1819WaX/IkOLz3OwljCWupAqVUKrAE6ATU\nA6O01v9rdEwtsNnjob5aax9LeQkhosXRrj1lzy3FuvRFsqZPJfvGiVhXLqd8/gM4OnWOd/FEFIXb\ngh8BlGitewJ3AnO8HFOqtT7D458EdyHixWSietgIV2v+rIGkbXyP1r27kb74CWnNG1i4Ab4v8FrD\nz+uAHpEpjhAimhyHtKPsmZcoe+hxnGmpZE+dRKsLBmP+6cd4F01EQVidrEqptcBkrfUXDb9vA47Q\nWtd4HFMBrMCVxnlVaz3f3znr6uqdFouP3TaEEJFXVARXXQWvv+7qgJo3D8aNA7N0zSWY8PdkVUqN\nAcY0erhrEBe4AXgO1x547yul3tdaf+rrOsXFib0mgBF640ORTPU1bF0tWfD4M1jP/idZ0yZjvvZa\nal54ibRnn2ZPzsHxLl1MGOG9zc/P9vlcwACvtV4ELPJ8TCm1BDgE+KKhw9Xk2XpveN2jHsevB44D\nfAZ4IUQcmExUX3AhNT17uzpfV78Bxx9P+vSZ7Bs9VlrzCS7cd28tMLTh58HABs8nlcsLSimTUsqC\nK0f/TfjFFEJEk7NtW8qWPE/Zo09CejrZ026k1fl/wfy//8a7aKIZwg3wLwMpSqlNwNXATQBKqalK\nqW5aaw1sAz7GNVRyldb640gUWAgRJSYT1UOGwrffUn3OYNI+2EzrM7uT8fjDMtImQclM1ggxQi4v\nFMlU32SqKzTUd3cZ1teXkTV1Eua9e6nt2o3y+x+i/vD/i3fxIsoI763MZBVChMZkovqvF7B34ydU\nDzqP1I8+IO/MHmQ8+iDUy5SWRCEBXgjhkzM/n7LFz1K66GmcNhtZt04j99yzSfnv9/EumgiCBHgh\nREA1557P3o2fsO/c80n95CNXa/4Rac23dBLghRBBcR50EOWLnqb0yWdwZmWRNWMauYPPIuUHac23\nVBLghRAhqRn8V1dr/vwLSP30Y/L69CDjoYXSmm+BJMALIULmbNOG8seeonTxczizssm6bTq5gwaQ\n8v138S6a8CABXggRtppB57J348fsG/I3Uj/7xNWaf+A+qPO+DaaILQnwQohmcbZpQ/mjiyld8gLO\nnFZk3XEruYP6k6L/E++iJT0J8EKIiKg5ZxB7N37EvgsuJPXzz8jr25OMhfOlNR9HEuCFEBHjbN2G\n8kcWUfr0izhy88iaNZPcc/qS8p+t8S5aUpIAL4SIuJqBf6F440fs+9swUv/9L/L6FWK77x5pzceY\nBHghRFQ481pT/vATlD7zEo7cPDJn307uwL6kbP023kVLGhLghRBRVXP2Oa7W/IUXkfpFQ2t+wd1Q\nWxvvohmeBHghRNQ581pT/uBjlD73Mo42B5E55w5Xa/5b2SYimiTACyFipmbAQIrf/5B9w0aQ+uW/\nyevfC9u986Q1HyUS4IUQMeXMzaP8gUcpfeGfOA7KJ3PeneSe3YeUr7+Kd9EMRwK8ECIuavqdRfH7\nH1J10d9J/eoL8gb0xnb3HKipCfxiERQJ8EKIuHG2yqXi/ocpeelVHAe3JfPuOeSddSYpX30Z76IZ\nggR4IUTc1fbp72rNXzwSyzdfkXfWGdjm3Smt+WaSAC+EaBGcOa2oWPAgJS8tw9H2EDLvnUfegDOw\nfPnveBctYUmAF0K0KLV9+rla85dchuXbr8k960xsc2dJaz4MEuCFEC2OMzuHinsXUrJ0OY5D2pE5\n/y7y+vfG8sW/4l20hCIBXgjRYtWe0aehNT8Ky9ZvyD27D7Y5t0N1dbyLlhAkwAshWjRXa/5+Sv75\nOo72BWQuuIe8Ab2x/PvzeBetxZMAL4RICLW9z6T4vQ+ouvRyLFu/JXdgX2yzpTXvjwR4IUTCcGZl\nU3H3AkpeXYmjoAOZ991DXr9CLP/6LN5Fa5EkwAshEk5tYW/2vvsBVaPGYNH/IXdgXzJnzYR9++Jd\ntBZFArwQIjFlZVExbz4ly97A0eFQbAvnu1rzn30S75K1GGEHeKVUb6XUbqXUIB/PX6yU+kQp9ZFS\n6vLwiyiEEL7V9uzF3ne3UDX6H1i+0+T+pT+Zt98qrXnCDPBKqSOA64HNPp7PBG4F+gFnABOVUq3D\nLKMQQviXlUXF3HspWb4KR8dDsT14H3l9e2L59ON4lyyuwm3BFwFDgFIfz3cFPtFal2qtq3DdCHqE\neS0hhAhKbfee7H33A+z/uBLL99+RO2gAmTOnQ1VVvIsWF5ZwXqS1tgMopXwdcgiwx+P33UA7f+fM\ny7NhsaSEU5wWIz8/O95FiKlkqm8y1RUSvL752fD4I/D3izCNHo3t4YXY1q2Bp56C7t2bHp7IdQ0g\nYIBXSo0BxjR6eIbW+q0QrmMKdEBxsT2E07U8+fnZ7NlTHu9ixEwy1TeZ6goGqu9RJ8H6zWTOvo2M\nJx6Fnj2puuJqKqdOB5sNMEZd/d2gAgZ4rfUiYFGI19yJqxXvVgB8GOI5hBCieWw2KmfNo3rQX8m+\nbhy2Rx8kbe1qyu9/hLqup8e7dFEXrWGSHwGnKqVylVJZuPLvG6N0LSGE8Kvu9G4Uv7MZ+5XXkPLj\n/8g99ywyb5kK9sTOHAQS7iiavyil3gXOBuYopdY2PD5VKdWtoWN1KvAWsA64TWvtq0NWCCGiz2aj\n8vbZlKxcS/3hR2B77GE44QRSP9wS75JFjcnpdMa7DADs2VPeMgoSJiPk8kKRTPVNprpCktS3qorM\nOXdge+whnEDVP66kctqM/bn5RJKfn+2zj1Nmsgohkk9GBpW3z4ZNm1yt+ccfofUZ3QzXmpcAL4RI\nXt27u3Lz48Zj/uVnWp03kMybb4TKyniXLCIkwAshkltGBpUzZ1Hyxlrq/+9IbE886mrNb9kU75I1\nmwR4IYQA6v58GsXrN2G/5jrM234h96/nkHXTDVBREe+ihU0CvBBCuKWnU3nr7ZSsWkddF0XGk4/T\n+ozupG56P94lC4sEeCGEaKTu5D9TvG4j9vHXY97+C7lDBpE15fqEa81LgBdCCG/S06mcPpOS1eup\n+9NRZDy1yJWb3/hevEsWNAnwQgjhR91Jp1D89vtUXncD5h3byb1gMFmTJ2KqaPlzBSTACyFEIFYr\n9mm3UrLmHeqOOpqMp58kr9fppL63Id4l80sCvBBCBKnuhJMoXvselddPxly0k9yh55E1aQKm8rJ4\nF80rCfBCCBEKqxX71FsoeWsDdUcdQ8azT7la8xvWx7tkTUiAF0KIMNQdfyLFb79H5fU3Yv61iNxh\n55M1aXyLas1LgBdCiHClpWGfOt3Vmj/6WDKeXeJqzb+zLt4lAyTACyFEs9UdfyLFa9+lctIUzLt+\nJXf4ELImXoOpLL6rpEuAF0KISEhLwz7lZorfepe6Y44j4/lnyCvsStr6tXErkgR4IYSIoPrjjne1\n5m+chnnPblpd9DeyJozDVFoS87JIgBdCiEhLTcV+w1SK175H7XEnkPHic67W/Lq3YloMCfBCCBEl\n9cceR8mad6iccjPm33+j1YihZF97JaaS4phcXwK8EEJEU2oq9klTKH77fWqPP5H0l18gr39vqK6O\n+qUtUb+CEEII6o8+hpLV68l45EEsW7+B1NSoX1MCvBBCxEpqKlXjJ8bscpKiEUIIg5IAL4QQBiUB\nXgghDEoCvBBCGJQEeCGEMCgJ8EIIYVAS4IUQwqAkwAshhEGZnE5nvMsghBAiCqQFL4QQBiUBXggh\nDEoCvBBCGJQEeCGEMCgJ8EIIYVAS4IUQwqAkwAshhEHJhh9hUkqlAkuATkA9MEpr/T8fx74IVGut\nL4tZASMsmPoqpYYBkwAHsF5rfXOsy9lcSqkFwOmAE5igtf7E47l+wGxc9V+ltb4jPqWMjAB1PROY\ng6uuGhijtXbEpaAR4q++HsfMAbpprc+IcfGiQlrw4RsBlGitewJ34vpjaEIp1R84IpYFixK/9VVK\n2YB5QF+gG9BPKXV0zEvZDEqp3sCRWutuwOXAwkaHLAQuAHoAAxKtfp6CqOvjwN+01j2AbODsGBcx\nooKoLw3vZ69Yly2aJMCHry/wWsPP63D90R9AKWUFpgOzYliuaPFbX621HThOa12utXYCvwNtYlvE\nZusLLAfQWm8F8pRSOQBKqcOBvVrrbQ0t2VUNxycqn3VtcIrWenvDz3tIvPeysUD1BbgXSLhvnf5I\ngA/fIbg++DT8wTuVUmmNjrkJeAQoi3HZoiFgfbXW5QBKqeOAzsCHMS5jc+2vY4M9DY95e2430C5G\n5YoGf3VFa10GoJRqBwzAdUNLZH7rq5S6DHgP+CmmpYoyycEHQSk1BhjT6OGujX43NXrNkcCftdYz\nlVJnRLF4ERdOfT1eeyTwAjBCa10bheLFktc6BvFcImpSH6XUwcBKYJzW+vfYFymq9tdXKdUaGAX0\nAwriVqIokAAfBK31ImCR52NKqSW4WgBfNHRAmrTWNR6H/AU4VCn1IZAD5CulbtRa3xWjYoctzPqi\nlOqA62vwJVrrf8eouJG0E49WHdAeKPLxXEHDY4nKX11pSF+sBm7WWq+NcdmiwV99+wD5wEbAChyh\nlFqgtZ4Y2yJGnqRowrcWGNrw82Bgg+eTWuv7tNbHa61PB8YBbyZCcPfDb30bPAlcpbX+PGaliqy1\nwN8AlFInAzvdaSet9U9AjlKqs1LKAgxqOD5R+axrg3uBBVrrNfEoXBT4e29f0Vof3fC3ej7wuRGC\nO8hywWFTSqXgauUeCVQDl2mttymlpgLvaa0/8Dj2jIbnL4tHWSMhUH1xdar+G/jY42XztdYrYl7Y\nZlBKzcU1ksIBXA2cBJRqrV9TSvXCNVII4FWt9T1xKmZE+Kor8BZQDHzgcfgLWuvHY17ICPL33noc\n0xlYYpRhkhLghRDCoCRFI4QQBiUBXgghDEoCvBBCGJQEeCGEMCgJ8EIIYVAS4IUQwqAkwAshhEH9\nP+LyF6Lo3R4HAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7f72f0d9c950>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "nexItAIIh9ls",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "output_extras": [
            {
              "item_id": 1
            }
          ],
          "base_uri": "https://localhost:8080/",
          "height": 52
        },
        "outputId": "081ca4b2-23e0-43fe-e777-f9fd7387d8bd",
        "executionInfo": {
          "status": "ok",
          "timestamp": 1509629026335,
          "user_tz": -480,
          "elapsed": 684,
          "user": {
            "displayName": "Mike C",
            "photoUrl": "//lh3.googleusercontent.com/-xWyBhueI0Vo/AAAAAAAAAAI/AAAAAAAAIg8/rU-qliu5l-M/s50-c-k-no/photo.jpg",
            "userId": "110113503404408134511"
          }
        }
      },
      "source": [
        "# now let's walk through how to perform forward computation\n",
        "#  and use AD to get gradients\n",
        "\n",
        "# first we clear the gradients that are stored in the model\n",
        "model.cleargrads()\n",
        "# as we have seen we can perform forward computation by calling the link\n",
        "model_y = model(data_x[:, None])\n",
        "\n",
        "# remember that `model_y` is a chainer variable. to operate on chainer variable\n",
        "#    we will use functions from chainer.functions to operate on those objects.\n",
        "loss = F.mean(F.square(model_y - data_y[:, None]))\n",
        "# `loss` is a scalar chainer variable\n",
        "assert isinstance(loss, chainer.Variable)\n",
        "print(\"loss\", loss)\n",
        "# calculating gradients d loss /d params is as simple as\n",
        "loss.backward()\n",
        "\n",
        "# we can inspect the gradient of loss with respect to W\n",
        "print(\"dloss/dW\", model.W.grad)"
      ],
      "cell_type": "code",
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "('loss', variable(1.5272977352142334))\n",
            "('dloss/dW', array([[-0.67883784]], dtype=float32))\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "bmutokMYh9lu",
        "colab_type": "text"
      },
      "source": [
        "Now that we know how to calculate gradients, we can code up a simple loop to perform gradient descent to train this model:\n",
        "\n",
        "(Hint: if you run into weird problems, maybe the state has been messed up and you can try re-runing all the code blocks from the beginning)"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {
        "id": "muy2zwTQh9lv",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "output_extras": [
            {
              "item_id": 1
            },
            {
              "item_id": 2
            },
            {
              "item_id": 3
            }
          ],
          "base_uri": "https://localhost:8080/",
          "height": 615
        },
        "outputId": "5f7704e9-505d-4920-e8d8-f50a04cfbd3d",
        "executionInfo": {
          "status": "ok",
          "timestamp": 1509629325804,
          "user_tz": -480,
          "elapsed": 1269,
          "user": {
            "displayName": "Mike C",
            "photoUrl": "//lh3.googleusercontent.com/-xWyBhueI0Vo/AAAAAAAAAAI/AAAAAAAAIg8/rU-qliu5l-M/s50-c-k-no/photo.jpg",
            "userId": "110113503404408134511"
          }
        }
      },
      "source": [
        "# now we can perform gradient descent to improve this model\n",
        "model = L.Linear(in_size=1, out_size=1)\n",
        "losses = []\n",
        "\n",
        "for i in range(100):\n",
        "    model.cleargrads()\n",
        "    loss = F.mean(F.square(model(data_x[:, None]) - data_y[:, None]))\n",
        "    losses.append(float(loss.data))\n",
        "    ## *** YOUR CODE HERE TO PERFORM GRADIENT DESCENT ***\n",
        "    ## Hint: you could access gradients with model.W.grad, model.b.grad\n",
        "    ## Hint2: you could write data into a parameter with model.W.data[:] = some_numpy_array\n",
        "    ## Hint3: if your model doesn't learn, remember to try different learning rates\n",
        "    lr = 0.1 # learning rate\n",
        "    model.cleargrads()\n",
        "    loss.backward()\n",
        "    model.W.data[:] -= model.W.grad[0] * lr\n",
        "    model.b.data[:] -= model.b.grad[0] * lr\n",
        "    \n",
        "    if i % 25 == 0:\n",
        "        print(\"Itr\", i, \"loss:\", loss)\n",
        "plt.plot(np.array(losses))\n",
        "plt.title(\"Learning curve\")\n",
        "plt.figure()\n",
        "plt.plot(data_x, model(data_x[:, None])[:,0].data, c='r')\n",
        "plt.scatter(data_x, data_y, c='b')\n",
        "_ = plt.title(\"Trained model fitness\")\n"
      ],
      "cell_type": "code",
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "('Itr', 0, 'loss:', variable(0.427589476108551))\n",
            "('Itr', 25, 'loss:', variable(0.11692220717668533))\n",
            "('Itr', 50, 'loss:', variable(0.05454133823513985))\n",
            "('Itr', 75, 'loss:', variable(0.0280784722417593))\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAEHCAYAAACzy817AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xl4nOV97vHvLJqRNNo3W5b37cEL\nxhvGxhgDBgIk4ISQZnEWEtokLT2lyTltOc3SZjmENDQ0SxtC05SszUYhUIeEkABhMYttvCI/3mTL\ntmQt1r4vM+ePGSlj2bLHsqTRO+/9ua65Zntn5vfzWLcePe/miUQiiIiIc3mTXYCIiFwcBbmIiMMp\nyEVEHE5BLiLicApyERGHU5CLiDicglzGnTEmYoyZmoTPfYcx5rvj/bkiY82f7AJExou19jHgsWTX\nITLaFOQyYRhjgsBXgJuAAPCwtfa+2HNrgG8CISAM/JW19hljzEzgZeCnwHJr7XpjTAT4IPBJYDLw\nT9baB40xdwLvt9Zeb4x5BDgKXAnMB/YDG621HcaYtwDfAdqAB4EHgCXW2iND6l0BPAxkA9XAndba\nitjnT7PWHo8tFwGmAXOB+4DjQC8wD/iytfbR2HJvB+611q42xmwEvhjr9yDwPmtt/Sj8M0sK0tSK\nTCR/CywELgUWAXcYY94We+5h4CvW2kuA+4GH4l5XBOyw1q6Pe2yRtXYZcBtwnzHGd5bPexfwbmAO\nUAy8I7bc94CPWmsXEA3b0DD1/gT4tLV2PtGR/jcT6HEZ8JC1dhPwi1h9A94B/MwYMxv4AfBea+1s\n4Nkh/YqcRkEuE8mtwL9Za7utte3A94HbY88tBX4Wu/0CMDvudWmcOWXyg9j1diAdKDnL52221jZY\na/uA3cB0oqPzoLX2qdgy3+AsPyfGmPlAUdxy3wTemUCPndba38du/wK4xRjjM8b4gbcCPyf6F8lz\n1to9seUeAm4b5peRiKZWZELJAx40xtwXux8EXovd3gT8lTEmG/ABnrjX9VtrW4a8VzOAtbbfGEPs\nNUM1x79HbJl8oDHu8aphai2Kf33sl0HfMMvGa4h7zWFjzDGi0ztp0YfsMWNMHnC1MWbfkFoLgdoE\nPkNcRkEuE0kV8IC19n/iHzTGlAH/Dlxhrd1hjJlHdE57LLQAWXH3Jw+zXD1QYIzxWmvDxpg0oCw2\njx4m9ovDGJN/ns8bmF4J8se/OKqAZ6y1d4ysBXEbTa3IRPJL4E9jUw0eY8ynjTE3EZ2/bgf2xaYg\nPgpgjMk6x3uN1AEgzRhzTez+x4GzHSL0ANGVlgNTP3cRnceH6IrPy2K3P0I02IfzC+B64G1Ep1UA\nfgOsi82VY4xZZYz52gV3Iq6hIJdkec4Ysy/uchXwr0S3JNkL7AMWAC8CO4FfER2FbwGeBF4Bnh/t\noqy13cCfA48YY3bEPjPMkDC31kaIriz9lDHmAPC+2OsAPgV8K/b6dqKj/OE+bz/Rn8MT1tqq2GPV\nwJ8BjxljyonOv/901JqUlOPR8chFhmeMCRHdDDHPWtt8vuVFkkEjcpEhjDGvG2PeHbv7bqBcIS4T\nmVZ2ipzpE8C/GmO+QHRa5ENJrkfknDS1IiLicJpaERFxuHGfWqmrax3xnwD5+Zk0NnaMZjmO4Ma+\n3dgzuLNvN/YMF953cXG2Z7jnHDUi9/vduYeyG/t2Y8/gzr7d2DOMbt+OCnIRETmTglxExOEU5CIi\nDqcgFxFxOAW5iIjDKchFRBxOQS4i4nCOCfJTzV088j976e7pT3YpIiITimOCfMfBeh599iB7KhrO\nv7CIiIs4JsizMtIAaG7vTnIlIiITi2OCPDcUAKC5rSfJlYiITCzOCfKsWJC3K8hFROI5J8hDQQCa\n2zS1IiISzzFBnhH0EfB7NSIXERnCMUHu8XjIy0lXkIuIDOGYIAfIzw7S0t5DWKenExEZ5Lgg7w9H\naO/sTXYpIiIThrOCPCcd0JYrIiLxEjpnpzHmQWA1EAHusda+fpZlvgSssdZeM6oVxsnP/mOQTy0e\nq08REXGW847IjTHrgXnW2jXAXcDXz7LMQuDq0S/vdPnZ0U0QW7RTkIjIoESmVjYAjwNYa8uBfGNM\nzpBl/hn41CjXdoaBINfUiojIHyUS5JOBurj7dbHHADDG3Ak8DxwZzcLOZmCOvEk7BYmIDEpojnwI\nz8ANY0wB8GHgeqAskRfn52fi9/tG8LFAYycA3X0RiouzR/YeDuW2fsGdPYM7+3ZjzzB6fScS5FXE\njcCBKUB17PZ1QDHwAhAE5hhjHrTWfmK4N2ts7BhhqZCXnwlAbUM7dXWtI34fpykuznZVv+DOnsGd\nfbuxZ7jwvs8V+olMrTwN3AFgjFkOVFlrWwGstb+w1i601q4G3gFsP1eIX6w0v49Qul9z5CIicc4b\n5Nbal4FtxpiXiW6xcrcx5k5jzDvGvLqzyM0K6sBZIiJxEpojt9beO+ShnWdZ5ghwzcWXdG65oQBV\n9e309oVJ8ztqfyYRkTHhuCQcOC55i6ZXREQAJwZ5SCeYEBGJ58AgH9gpSPPkIiLgyCDXuTtFROI5\nL8h17k4RkdM4L8g1Ry4ichrnBXmWTsIsIhLPcUEeSvfj83q0+aGISIzjgtzj8ZCbFaBJKztFRAAH\nBjlE58mb23uI6CTMIiJODfIgff1hOrv7kl2KiEjSOTLIc7TliojIIEcG+cAmiJonFxFxaJDnDewU\npE0QRUScGeSFudFzd9Y1dyW5EhGR5HNkkBfnZQBQFzuHp4iImzkyyItyM/B4oPYizv8pIpIqHBnk\naX4vBdnp1DZpRC4i4sggByjJz6CprYfu3v5klyIiklSODnKAOo3KRcTlHB/ktVrhKSIu59wgz1OQ\ni4iAk4M8PxNAKzxFxPUcG+TFebGdgrQJooi4nGODPD3gJzcU0IhcRFzPsUEOUJyfwanmbvr6w8ku\nRUQkaRwd5JPyMghHIpxq0TFXRMS9HB3kxdoEUUTE2UGubclFRJwe5HmxTRAV5CLiYs4Ocu2mLyLi\n7CDPykgjM+inRtuSi4iLOTrIIToqr2vqIhyJJLsUEZGkSIkg7+sP09Sq83eKiDulRJCDVniKiHs5\nPsgHzt+peXIRcSvHB/mUohAAx2vbk1yJiEhyOD7IpxVn4fV4OFrTmuxSRESSwvFBHkjzMaUoRGVt\nK+GwtlwREfdxfJADzJicRU9vmOpTml4REffxJ7KQMeZBYDUQAe6x1r4e99yfAXcB/cBO4G5r7bgO\njWdOzuGl3Sc5crKVsuKs8fxoEZGkO++I3BizHphnrV1DNLC/HvdcJvAeYJ21di1wCbBmjGod1ozJ\n2QAcPal5chFxn0SmVjYAjwNYa8uBfGNMTux+h7V2g7W2NxbqucDJMat2GNNKsvB40ApPEXGlRKZW\nJgPb4u7XxR5rGXjAGHMvcA/wL9baw+d6s/z8TPx+3whKjSouzj7r49MnZXOsto2Cwix8Xs+I33+i\nGq7vVObGnsGdfbuxZxi9vhOaIx/ijJS01t5vjPka8CtjzIvW2peGe3HjRey4U1ycTV3d2UfdZUUh\njp5sZY+tGdy2PFWcq+9U5caewZ19u7FnuPC+zxX6iUytVBEdgQ+YAlQDGGMKjDFXA1hrO4GngLUJ\nVzaKNE8uIm6VSJA/DdwBYIxZDlRZawfSMg14xBgzsKnIKsCOepUJmDkQ5JonFxGXOe/UirX2ZWPM\nNmPMy0AYuNsYcyfQbK19zBjzeeBZY0wf0c0PnxjTiocxvSQbjweOaEQuIi6T0By5tfbeIQ/tjHvu\nEeCR0StpZIIBH6WFISprWglHIng9qbfCU0TkbFJiz84BMyZl09XTT02DjoQoIu6RWkGuFZ4i4kIp\nFeSzSqNBfuhEy3mWFBFJHSkW5DkEAz72HmlIdikiIuMmpYLc7/OyYHo+Jxs6qG/Wqd9ExB1SKsgB\nFs0qAGBvhUblIuIOKRfki2NBvkdBLiIukXJBXpKfQVFuOuVHGukPh5NdjojImEu5IPd4PCyeVUBH\ndx8V1doMUURSX8oFOWieXETcJSWDfMGMfLwej4JcRFwhJYM8Mz2N2VNyOFzVQkdXb7LLEREZUykZ\n5BCdXglHIpQfbUx2KSIiYyplg3xgM8QdB+uTXImIyNhK2SCfNSWHgpwg22wd3b39yS5HRGTMpGyQ\nez0e1iyaTFdPP28cqEt2OSIiYyZlgxzgysXRU41u2VOT5EpERMZOSgd5aWGIWaU57Kk4RXNbd7LL\nEREZEykd5BAdlUci8MqbGpWLSGpK+SBftaAEn9fDy3tOJrsUEZExkfJBnp0ZYMmcQo7VtlFZo2Ov\niEjqSfkgB7hycSkAL+3WqFxEUo8rgnzJnEJyQwFe2FWlXfZFJOW4IsjT/F5uvHwaXT39/H77iWSX\nIyIyqlwR5ADXLCsjI+jnt1uP0aM9PUUkhbgmyDOCfq5bXkZrRy8v7q5OdjkiIqPGNUEOcMPKaaT5\nvfz61UqdBk5EUoargjwnFGDdklLqm7t4rbw22eWIiIwKVwU5wE2rpuP1eHjixQr6+jUqFxHnc12Q\nF+VlcO2yMmoaO/ndtuPJLkdE5KK5LsgBNq6bRSjdzxMvVdDS3pPsckRELoorgzwrI423r5tNZ3c/\n//2Hw8kuR0TkorgyyAGuWTaFsqIQL+ys0jFYRMTRXBvkPq+X91w/jwjww6f3Ew5Hkl2SiMiIuDbI\nARbNLGDlJSUcPNHM068fS3Y5IiIj4uogB3j/jfPJyUzjv/9wmBN1bckuR0Tkgrk+yHMyA3zo5kvo\n6w/znc3l2rZcRBzH9UEOsGxeMWsXT+boyVY2bzma7HJERC6IgjzmvdfPpyAnyBMvVVB+pCHZ5YiI\nJCyhIDfGPGiM2WKMedkYc/mQ5641xrxijHnJGPNdY4wjfzlkpvv5+MbFeD0eHnpiLw0tXckuSUQk\nIecNXWPMemCetXYNcBfw9SGLPAzcYa1dC2QDN416leNkblku79kwj9aOXr71+B7Nl4uIIyQyet4A\nPA5grS0H8o0xOXHPr7DWDhy0pA4oHN0Sx9d1y8tYvWgSh6pa+K/fHUh2OSIi5+VPYJnJwLa4+3Wx\nx1oArLUtAMaYUuBG4DPnerP8/Ez8ft+IigUoLs4e8WsT9b83reRvvvECz24/wbzpBdy6bvaYf+b5\njEffE40bewZ39u3GnmH0+k4kyIfyDH3AGFMCPAn8hbX21Lle3NjYMYKPjCouzqaubnx2p/+LjYv4\n4g+28e+/3E26z8PSeUXj8rlnM559TxRu7Bnc2bcbe4YL7/tcoZ/I1EoV0RH4gCnA4LnSYtMsTwGf\nttY+nXBVE1xRXgb33LGENJ+Xh57Yw5GTLckuSUTkrBIJ8qeBOwCMMcuBKmtt/K+RfwYetNb+egzq\nS6pZpTl87LZF9PaG+drPd1F7EX9NiIiMFU8kcv6DRRlj7geuBsLA3cAyoBn4DdAIbIlb/MfW2oeH\ne6+6utYRH50qWX+CPbP1GD9+5gBFuen83/evID87OK6f78Y/Pd3YM7izbzf2DCOaWjljWntAQnPk\n1tp7hzy0M+72+KZaEly/chodXX08/mIFD/zkDe7dtJzszECyyxIRAbRnZ8JuXTuTGy+fRvWpDr76\ns510dPUmuyQREUBBnjCPx8O7r5vL1ZeVcvRkKw/8ZAftCnMRmQAU5BfA4/HwwZsuYd2SUo6cbOWB\n/9pBW6fCXESSS0F+gbweDx+6+ZLoyLymlQd+8gatHTqBs4gkj4J8BLyxkfn6pVOorGnj/h9tp7G1\nO9lliYhLKchHyOvx8MG3mMEVoF/64TZtZy4iSaEgvwgDK0Dfvm4W9c1dfOmH26mscd/2sCKSXAry\ni+TxeLht7Szee/08mtt7uP9H23ViChEZVwryUXLDyml87LZF9PWH+erPdvLKmyeTXZKIuISCfBRd\nsXASn/iTpQTSvDz8xJts3nKERA6BICJyMRTko2zBjHzu3RQ9Hsujzx/mP5/apzMNiciYUpCPgWkl\nWXzmQyuZMTmbF3dV89WfaschERk7CvIxkpcV5N73LWfZvCL2VTbxxe9vpaq+PdlliUgKUpCPoWDA\nx923X8pb18ygtrGTL35/KzsO1ie7LBFJMQryMeb1eHjn+jl87LZFhMMRvvGLXTz5UgVhrQQVkVGi\nIB8nVyycFD0pRU6Qx16o4JuP7qajqy/ZZYlIClCQj6MZk7P57J2Xs2BGPjsO1vOF773O8bq2ZJcl\nIg6nIB9nOZkBPvnuy7j5iunUNHbyxe9t5cVd1ed/oYjIMBTkSeDzennXtXP5y9svxefz8t1flfPd\nzeV09/YnuzQRcaCEztkpY2P5/GKmlmTxrcf28OLuaiqqW/jYxkVMLc5Kdmki4iAakSdZSV4Gf/+B\nFWxYPpUT9e184Xtbee6NE9q1X0QSpiCfANL8XjbdOJ//dfulBPxevv8by789tkd7g4pIQhTkE8iy\n+cV87iOrMNPy2La/js/8x6vsqTiV7LJEZIJTkE8wBTnp/M17l3HHNXNo6+jlqz/dycOP79aKUBEZ\nloJ8AvJ6Pdyyegaf/uBKSgszefKFw/zjf77OoRPNyS5NRCYgBfkENmNyNv9w5+VsvHoOtQ0d3PfD\nbTz6/CF6+zQ6F5E/UpBPcIE0H3+6cTF/+75lFOaks3nLUY3OReQ0CnKHMNPz+fxdq9iwYirVpzq4\n7wfb+K9nDtDVo+O1iLidgtxB0gN+Nt0wn3s3LackP4Pfbj3GZ77zGrsOacsWETdTkDvQ/Gl5fO4j\nq3jrmhk0tXXzLz/fybef2EtzW3eySxORJNAu+g4VSPPxzvVzWLVgEo88tY9X36xh16F6br96Dtcu\nK8Pr9SS7RBEZJxqRO9y0kiw+9YEVfODG+YCHH/12P1/43latDBVxEQV5CvB6PVy7fCr3fXQ1axZN\n4mhNK//vB9v47uZymtt7kl2eiIwxBXkKyQ0F+LNbF3HvpuVMLc7ixd3V/P3DW/j1q5X09YeTXZ6I\njBEFeQqaPy2Pf/jwSjbdMB+vx8PPnj3Ip7/zKm8cqNNRFUVSkFZ2piif18uGFVO5YuEkfvliBc9u\nP8E3Ht3NJdPz+JPr5jJzck6ySxSRUaIReYrLykhj0w3z+dxdq1gyp5B9lU18/pGtPPzkXuqbOpNd\nnoiMAo3IXaKsKMRfv+syyo808NNnD/LK3hpeL6/l2uVlvO3KmeRkBpJdooiMkEbkLrNgZgGfvfNy\nPnrrQvKzgzyz9Th/99AWHn/hMB1dOpGFiBMlNCI3xjwIrAYiwD3W2tfjnksHvg0sstauHJMqZVR5\nPR5WL5rMyktKeH5HFU++VMETLx3hma3HuemK6WxYMZWMoP5YE3GK847IjTHrgXnW2jXAXcDXhyzy\nFWDHGNQmY8zvi64Q/fLHr+SOa+bg8cB//+Ewf/fQFjZvOUJntw7IJeIEiUytbAAeB7DWlgP5xpj4\nTR7+HnhsDGqTcRIM+Lhl9Qy+/PEreftVswiHIzz6fDTQ/+flI3R0KdBFJjLP+bYrNsY8DGy21v4y\ndv8F4C5r7f64ZWYCv0hkaqWvrz/i9/suqmgZW+2dvTz54mEef/4Q7Z29hNL9vPWq2dy2bja5WcFk\nlyfiVsMeQGkkE6EXdTSmxsaOEb+2uDiburrWi/l4R0pG3xuWTuHKBSU8+8YJfvNaJT97Zj+PP3eQ\ndUum8JZV0yjKyxjTz9d37R5u7BkuvO/i4uxhn0skyKuAyXH3pwDVCX+6OFZG0M8tq2ewYcVU/rCz\niqdfq+R324/z7BsnuHxBCW9ZNU07FolMAIkE+dPA54BvG2OWA1XWWvf9+nSxYJqPG1ZO49plZbxW\nXsNTr1by6ps1vPpmDWZaHm9ZNZ0lcwvxenToXJFkOO8cOYAx5n7gaiAM3A0sA5qttY8ZY34OTAMW\nAduAh621Px7uverqWkd8sA/9CTYxRCIR9lY08JvXKtl7pBGAkrwMNqyYylVLSkdl08WJ1vN4cWPf\nbuwZRjS1MuxIKaEgH00K8gs3kfs+XtvGb7ceY8veGvr6wwQDPtYunsx1y6cypSg04vedyD2PJTf2\n7caeYXSDXHt9yEWZWpLFh29ZwDuvmcPzO6p47o0T/H579HLJ9DyuXT6VZfOK8Pu0E7HIWFGQy6jI\nyQxw65UzuWX1dHYcqOd3246zr7KJfZVN5IQCrFtSyrolpZTkZya7VJGUoyCXUeXzellhSlhhSqg+\n1c7zO6p4aXc1m7ccZfOWoyyYkc+6y0pZPq+YQJr2JxAZDQpyGTOlhSHes2Eet189m222jud3VlF+\ntJHyo41kBv2sWjiJqy4tZVZpNh5t8SIyYgpyGXOBNB9rFk9mzeLJVJ9q58Xd1by85yTPvXGC5944\nQWlhJlcunszqhZMpzE1PdrkijqOtVhwgFfvuD4fZW9HAy3tOsn1//eA5RedPy2P1oknctHY2Xe3d\nSa5y/KXid30+buwZtNWKpACf18uSOUUsmVNER1cvW20dW/acxB5rYv+xJn782/0snFnAFQsmsXRe\nkQ6rK3IO+umQpMtMT+Pqy6Zw9WVTaGjp4tU3a9i6v45dh06x69Ap0vxeFs8qYOUlJSydq1AXGUo/\nETKhFOSkc/PqGXzw1sXs2neS18prea28hjcO1PPGgXr8Pg+LZhaw3BSzdG4R2TpFnYiCXCau0sIQ\nG6+axcarZnGivp1ttpat+2rZeegUOw+dwuvxMG9qLsvmFbF0fjElY3xERpGJSkEujlBWFKKsaBa3\nrZ1FTWMH2/fXsd3Wsf9YE/ZYEz/5/UHKikIsmVvIZXOKmFuWi9erTRrFHRTk4jiT8jO5+YoZ3HzF\nDJrbutlxMDrtUn60kadeqeSpVyoJpftZPLuQJbMLWTS7gBxNwUgKU5CLo+VmBVm/tIz1S8vo7u2n\n/GgjOw/Ws+vQqcFD7QLMmJTN4tkFLJ5VwJyyXB37RVKKglxSRjDNx9K5RSydW0QkEuFEfTu7Dp1i\nz+FTHDjezNGaVjZvOUowzYeZnsfCGfksmFlAWXFIx1IXR1OQS0ryeDxMLc5ianEWt6yeQVdPH/sq\nm3izooG9RxoGN20EyM5Mw0zPZ8H0PMz0fEoLM3XIAHEUBbm4QnrAPzhaB2ho6Ro87kv50Ua27otu\nEQOQk5nG/Gl5g5epxVlacSoTmoJcXKkgJ521l5ay9tJSIpEItY2d7KtsZF9lE7ayka22jq22Doie\nu3RuWS5zp+YytyyX2aU5BAM6cqNMHApycT2Px8OkgkwmFWSyfmkZkUiEuuYu9ldGDxdw4EQzuw+f\nYvfh6FSM1+NhWkkWc8pymDMll9llOZTkZWg6RpJGQS4yhMfjoSQvg5K8DK5aUgpAc3sPB483c6iq\nmYPHmzlysoWjNa38fvsJAELpfmaV5gxeZpZmk5cVTGYb4iIKcpEE5IYCrDDFrDDFAPT2hTlW28ah\nqmYOnWjmSHUreyoa2FPRMPia/OwgMyZlM2Ny7DIpm7ysgEbuMuoU5CIjkOb3MntKDrOn5HDDymkA\ntHb0UFHdypGTLRypbqXiZAs7Dtaz42D94OuyM9OYPimb6SVZTItdJhdm4vNqu3YZOQW5yCjJzgyw\nZE4hS+YUDj7W1NbN0ZOtHK1ppbKmjcqaVvZWNLA3buTu93mZUpjJ1JIszMxC8jL9TC3O0uhdEqYg\nFxlDeVlB8uYGuSy22SNAe1cvx2vbqKxt41hNG8fr2jhR305lbRsv7zk5uFxm0M+UohBTijKZUhii\ntCjElMIQBTlBBbycRkEuMs5C6dEdkMz0/MHHwuEINY0dtPaEKT9Uz/G6Nqrq26mobuHgiebTXh9M\n8zGpIIPSwhCTCzIHL5MKMkgP6EfajfSti0wAXq+H0sIQS4qzmV+aPfh4X3+Ykw0dVJ/qoLq+napT\n7dHbpzqorGk7431yswJMys9kUn4GJfkZTMrPpDgvelsn5Ehd+mZFJjC/zzt4qIF44UiEhuYuqhs6\nqGno4GTsuqaxkwOx0+UNlZWRRkl+BsV5GRTlpg9eF+VlUJAd1IHEHExBLuJAXo+HorwMivIyuHR2\n4WnP9faFqWvqpLapk9rGTmoaO6hr6qSusZOjJ1s5XNVyxvt5PNH5/MLcdIpy0ynMiV4KctIpzAlS\nkJOuEf0Epm9GJMWk+b2xlaShM54LhyM0tnZT3xwN+lPNXdQ1dVHf3Mmpli4OnYju8HQ2GUEfBdnp\n5GcHKcgJkpcVJD87ehm4nZWRphWxSaAgF3ERr9dDYW46hbnpp61sHdDXH6aptZtTLV00tHRT39JF\nY0sXDbHHGlu6OVHfPuz7+30eckNB8rID0S12QkFysgLkhQLkZgXIDQXJCQXIzkzTVM4oUpCLyCC/\nzzs4ZTOcrp4+Glu7T7s0tf3xuqmthyPVrfSHz5zCiZeVkUZOKEBRXgbpaV5yMgNkx0I+JzN6nR27\nzgz6NdI/BwW5iFyQ9ICf0kI/pYVnTt0MCEcitHX00tTWTUt7D01tPTS3d9Pc3kNLew/NbT20dPTQ\n3NZN1TlG+AN8Xg+hjDSyM9LIGrhkRq9D6bHrDP/g/VBGGqF0v2tG/QpyERl1Xo+HnFCAnND5z5Wa\nlx+iorKB1o5oyLd29NLSEb1uHbju7KGts4+mtnNP7QwVTPORme4nlO4nMz0tdu0nMxi9nZHuJzM4\n8JifjGDsOt1PRsDvmOPQK8hFJKnS/N7BlaaJ6A+Hae/qo72zl9aOXtq7emnr7KW9s4/2rl7aO2P3\nu6L3O7r6ONXSzfG6xH8BDAim+cgI+sgI+kkP+KO3A37S467TA37SA77YxT94HQz4SE/zkR70EUzz\njelfBwpyEXEUnzc6n56TGaC08PzLDwiHI3R099HR3UdnXMh3dPfR2d1He1f0+vRLP53dfbR19lLX\n1EVff3jEdft9Xt6zYS7XLZ864vcY9r1H/R1FRCYgr9czOL8+Ur19Ybp6+ujs6aeru4+unv7o/e5+\nunvjHuvtH3yuuyf6XE9fmPwxOka9glxEJEFpfi9p/gDZmcmu5HTuWKUrIpLCFOQiIg6nIBcRcbiE\n5siNMQ8Cq4EIcI+19vW4564H7gP6gV9Za78wFoWKiMjZnXdEboxZD8yz1q4B7gK+PmSRrwPvBNYC\nNxpjFo56lSIiMqxEplY2AI85b+LMAAAEdUlEQVQDWGvLgXxjTA6AMWY20GCtPWatDQO/ii0vIiLj\nJJGplcnAtrj7dbHHWmLXdXHP1QJzzvVm+fmZ+P2+Cyzzj4qLs8+/UApyY99u7Bnc2bcbe4bR63sk\n25Gf6+AD5z0wQWNjxwg+Mqq4OJu6utYRv96p3Ni3G3sGd/btxp7hwvs+V+gnEuRVREfeA6YA1cM8\nVxZ77FzFXNRRaPSb2z3c2DO4s2839gyj13cic+RPA3cAGGOWA1XW2lYAa+0RIMcYM9MY4wfeFlte\nRETGiScSiZx3IWPM/cDVQBi4G1gGNFtrHzPGXA18Obboo9baB8aqWBEROVNCQS4iIhOX9uwUEXE4\nBbmIiMMpyEVEHE5BLiLicI45scS5DtyVaowx/wSsI/r9fAl4HfgB4CO6Df8HrLXdyatwbBhjMoA9\nwBeA3+GOnjcBfwv0AZ8FdpHCfRtjsoDvA/lAEPgccBL4FtGf7V3W2j9PXoWjyxizGPgl8KC19pvG\nmGmc5fuN/T/4a6JbBj5srf2PC/kcR4zIEzhwV8owxlwLLI71ehPwL8DngX+11q4DDgIfSWKJY+nT\nQEPsdsr3bIwpBP4BuIroPhgbSf2+7wSstfZaovunfI3o//F7rLVrgVxjzM1JrG/UGGNCwDeIDkoG\nnPH9xpb7LHA9cA3wCWNMwYV8liOCnHMcuCsF/QF4V+x2ExAi+uU+EXvsSaJfeEoxxlwCLAQ2xx66\nhhTvmWhPz1hrW6211dbaj5L6fdcDA6dMzif6i3tW3F/YqdRzN3ALp+/tfg1nfr9XAK9ba5uttZ3A\nS0SPJpswpwT50INzDRy4K+VYa/utte2xu3cRPaJkKO7P61qgNCnFja1/Bj4Zd98NPc8EMo0xTxhj\nXjDGbCDF+7bW/gSYbow5SHTQ8n+AxrhFUqZna21fLJjjne37PdvBBy/o38ApQT7URR2vxQmMMRuJ\nBvlfDnkq5Xo3xnwQ2GKtrRhmkZTrOcZDdHR6O9Eph//k9F5Trm9jzPuBSmvtXOA64IdDFkm5ns9h\nuF4v+N/AKUF+rgN3pRxjzFuATwE3W2ubgbbYikBI4MBkDvRWYKMx5hXgT4HPkPo9A9QAL8dGboeA\nVqA1xfteC/wGwFq7E8gAiuKeT8We453t//UFH3xwKKcE+bAH7ko1xphc4CvA26y1Ayv+niF6FiZi\n179ORm1jxVr7bmvt5dba1cB3iG61ktI9xzwNXGeM8cZWfGaR+n0fJDonjDFmBtFfXuXGmKtiz99O\n6vUc72zf76vA5caYvNhWPWuBFy7kTR1zrJWhB+6K/TZPOcaYjwL/COyPe/hDRAMuHTgKfNha2zv+\n1Y09Y8w/AkeIjtq+T4r3bIz5GNEpNIAvEt3UNGX7jgXVd4FJRDev/QzRzQ+/TXRg+aq19pPDv4Nz\nGGNWEF33MxPoBU4Am4BHGPL9GmPuAP6G6CaY37DW/uhCPssxQS4iImfnlKkVEREZhoJcRMThFOQi\nIg6nIBcRcTgFuYiIwynIRUQcTkEuIuJw/x+60DdcrjrVlQAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7f72f0e5ce10>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAEHCAYAAACk6V2yAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xl8VNXZwPHfLCSZSQJGjBAggFo8\nolar1gUVFxArKmKpC+6gEAGtQGuLRdriW2v1rRawFkyIoEUFXBC0mILg8iKotVi3Vo9VQQOkmEIk\ny0yWWd4/ZiZMktm3TGae7+fjx+TOnTvnJOG55z733OcY3G43QgghMo+xuxsghBAiOSTACyFEhpIA\nL4QQGUoCvBBCZCgJ8EIIkaEkwAshRIYyd3cDRM+mlFoCnO/99ihgD2D3fn+q1rohimN9Cpyrtd6b\noLY5gO9orXcm4nghPud6YIrW+rwQ+wwFPtdad/k3p5S6D5gE3I3nZ/ms1volpdTVQJXWuj4Z7RaZ\nTwK8iIvWerrva6XUTuB6rfWbMR7rmAQ1q6e5GrhBa70ZWO63/R5gKyABXsREArxIKqXU63iC1ATg\nFuAL4AlgKJAL/FFr/Qfvvm6gFPgO8DvgdeByIA+YpLV+QymVC/weuAjIASq01vd53z8W+CPQBiwL\n0aadwEPAZGAgMB0Y7T1mLTBWa12nlDoBWAL0BZqBOVrrDUopI/AwcBnwH+ANv2Mf4m3D6Xj+ff1G\na+0ftDu35SlgMLBMKXUvcB1QCYwCFPC6UmoSMAX4CjgTOBr4DBivtbYppY71trMEaAEma63/rpQq\nAFYAx3h/1puBGd6vu2zXWrcFa6fomSQHL1LhFOA4rfU2YB6wwztaHw38TilVGuA9JwFva62HA4u9\n7wP4OXAs8F3gOOAKpdSlSikT8BieQDUccAGmEG06Xmt9MvAbPMHuWTwnFiMwwRvEVwGPeNs6BVip\nlCrEcyK40NuOc4Fz/I77kPezj8ET5O9RSh0frBFa6+uA3cB1Wuulfttv9n55nt8V0ZV4RvtHAcXA\nD73tXAv8WWt9NDANWKeUMgM3Ad96fx5HAw7vzyzYdpFhJMCLVHhZa+3yfn0H8GMArfWXeEbARwR4\nT4PWep336/fwjHIBxgGLtdYtWusm4M94rg6GAXla643e/R4P06a13v9/BNi11q9rrd3AP4EB3jb1\nxxPk0Vr/Hc8I+lQ8AX291rpRa20HnvE77jhgkdbapbWuBdZ425cI67XW+7XWDm+7B+M5kRyO94pF\na70Vz1XImcA3wAil1IWASWs9XWv9fojtIsNIikakwn6/r0/FM2ofDDjxpBUCDTQO+H3t5OBo/BBg\ngffGJHhSDH8DDqVjrrouTJt8N3+dQGOAzyrGM8r1L9ZUhyeYHornZnKgzzoEeMZ7gxfAgufqIBEC\n/UwOAazAJ0op32u9gb5a62eVUofiuUo5Rin1JPCTENtbEtROkSYkwItUexJYADyqtXYrpXZH+f49\nwINa67/4b1RKDccT2HyK42sme4FDlVIGvyDf17u9DugT5LP2AJdrrT/u1L6hcbYnmD1AfbAb1Frr\ncqBcKTUQeB64EVgabHuS2ii6iaRoRKodDmz3BvebgHygIIr3rwOmKKVMSimDUmqeUuoi4HPAoZQ6\nz7vfZCCeUqk7gV14ct4opc7Ek7L5G/AW8AOllFUpZcWTG/dv3zTve8xKqQVKqZNjbIMDzwg9lK+A\nXUqpK7yfeZhSaqVSKl8p9Uul1M0AWuvdwA7AHWx7jG0UaUwCvEi1XwIvKKU+xBPYy4GlSqmjInz/\nn/AEtX8CnwLDgTe9M0DK8MxG+QTPjc7GoEcJwztqnwjc7j3ew8CV3rz/S3hmBmk8M2he7tS/Pkop\n7W2jCfgwxmY8A2xTSl0VYTs/Bf4P2Oxt5wrgBqWU9r7W6t0WbLvIMAapBy+EEJlJRvBCCJGhJMAL\nIUSGkgAvhBAZSgK8EEJkqLSZB19b29Cj7/YWFVmpq7N1dzNSJpv6m019hezqbyb0tbi40BDsNRnB\nJ4jZHKrsSebJpv5mU18hu/qb6X2VAC+EEBlKArwQQmQoCfBCCJGhJMALIUSGkgAvhBAZSgK8EEJk\nKAnwQgiRoSTACyFECtlssGOHAVsKnq+SAC+EECngcMC8eTmMHGllxIh8Ro60Mm9eDg5H+PfGKm1K\nFQghRCabPz+Hiorc9u+rq01UVHiepL333takfKaM4IUQIslsNqiqCjyerqoyJy1dIwFeCCHiEElO\nfe9eA7t3Bw63e/YY2bs3aL2wuEiAF0KIGESTU+/Xz83Aga6AxxkwwEW/fskppisBXgghYuDLqVdX\nm3C5DN6cei7z5+d02ddqhbFjA99NHTvWgdWanDZKgBdCiCjFklOfP7+VsrIWSkudmExuSkudlJW1\nMH9+cm6wgsyiEUL0ADabJ4/dr587aaPdaESSUz/iiI5pF7PZM1tm7tzWlPVFRvBCiLTVHXPHIxFP\nTt1qhSOOSM2JSgK8ECJtRZPnTqXuyqlHSwK8ECItddfc8UhFk1NPZXkCf5KDF0KkpVjy3KkUSU7d\n4fBchVRVmdm928jAgS7GjnUwf34r5hREXwnwQoi05MtzV1d3XRg7mrnjyb5B68upB9Id5Qn8xZWi\nUUodr5T6Qil1e4DXLlBK/U0p9ZZS6pfxfI4QIvvEm+cOdIP2rrty+Pzz0KmSRKVT0iHFFHOAV0rl\nA38ENgfZ5WHgR8BZwIVKqWNj/SwhRHaKZ+54oBu0y5blcuaZgWfjJHrGTneVJ/AXT4qmBbgYmNP5\nBaXUkcB+rXW19/uXgdHAv+L4PCFElol17nio0TMY2lMlFgvcfbdna6LTKYlKMcUj5gCvtXYADqVU\noJf7A7V+338DHBXqeEVFVszmrj+InqS4uLC7m5BS2dTfbOorpGd/hwyJfN8vvoDdu8Pvt24d3Hef\np68bNgTeZ+PGXBYsyI0pfz9hAixaFGi7iSFDkv8zTtVN1rDXInV13TznKU7FxYXU1jZ0dzNSJpv6\nm019hczor9kMAwdaA46e/VVXw8cfN9LcDNXV+QQKVdXVbj7+uCmiGTudb+jOmQN2u2cWzZ49RgYM\n8MyimTOnldrasIeLSKiTcbIC/B48o3ifgd5tQgiRdL4btL4USzCDBsGjj/Zi40YzrsAPpkaUTgk1\nHTLV5Qn8JSXAa613KqV6K6WGAruAS4HrkvFZQggRiO9GbFWVmepqI4FG50VFsHx5bpft/s46K/xd\n1nD5+1BTKZPJ4HbH9qFKqVOAh4ChQBuwG3gR2KG1fkEpdQ7wgHf357XWD4Y6Xm1tQ/c9sZAAmXBZ\nG41s6m829RUyr782G+zebaCyshebNh1MlYwZ42Dz5ly++irQu9wUFLi97zeEfEDJZoORIwOng0pL\nnWzZYkvqqL24uDBoCjzmAJ9oEuB7lmzqbzb1FTK7v/458r17DYwYURAkNeMm0Ii/rKyly4yaHTsM\njBiRj8vVdX+Tyc22bZHl72MVKsBLLRohRNbwr+TYr5+bwYMD72cKkroP9IBSd63WFAkJ8EKIrGS1\nwvjxgV9zOgNvD/SAUjpXlpRaNEKIrPXgg2C3t3SYxjhmjINXXjFH9YCS/w1d/+mQyVytKRIS4IUQ\nWSvYk7JmMwGnWAYbkXfHak2RkAAvhMh6nacxxjoi767pkMFIgBdCiE6SOiJvbsbQ1IS7b98EHTA4\nuckqhMgIyVg1KZHrpxr/U4P1d/9D3+8dQ99TjoPm5vgPGoaM4IUQPVqsqybZbJ6iZGYzSc2Xm/+x\nHUvFEnLXrcHgcOAqKsI2607IDf0EbUI+O+mfIIQQSWKzwZw5uaxefXAR7nBlfjueEDxFyRK+jJ7D\nQc7LL2EtX0yvd9/xbFLHYC+bQfOPrkruGcWPBHghRNIka7k8X5Bev94cdFGNqiozc+e2dvncZC6j\nZ/i2jrwVT2BZVoFp9y4AWsb8AHvZDNrOOQ8MyV/kw58EeCFE3DoH8mQvNt05SAcSaGHucMvoBToh\nRML0mcay9FHynl2JwWbDbc3HfvNU7FOn4TxqWPQHTBAJ8EKImAUL5C4XVFYmZ5QcerWmgw4/3EXv\n3h2nLEayjF7E0xzdbnq9thlrxWJyXt0EgLN0MPafldF8/Y24+xwS2XGSSAK8ECJmwdIdBQWBa7PE\nM0r2CRWk/dXUGLnwwo759XiW0fNdpfQvbCR/zSqslY9i2akBaDt9BLayGbSOvYTEJfLjJ9MkhRAx\nCTWSbmwMnGvevdvIV1/Fl4cOVdzLUwXSx7f2ai7z53tuwsZSN8a3GPdVI/bx2um/oeDYY+k7bzam\nnV/yvPV6fvvDbfz3hQ20jhufVsEdJMALIWIU6Ujan8sF111nYd68HBzeOBvt/PVQQdpqDTwC968C\nOX9+K2VlLZSWOjGZPDXby8paAj+l6nbz52nbGVVxI1trvsMc/hcHZu7hVwzma66wrWDeCyPaTyDp\nRurBJ0gm19AOJJv6m019hcj7G2qhi2D11P1NmdKC0UhM89cDLeBx1lkOnnkmJ+K67DYbOByFmM0N\nXUfura3kvrSW3EcXk/vBewC8z4ksYiYruYYW8jrsnoqFPYIJVQ8+va4nhBA9Ruh1T/1jTuBgv2pV\nLxobD14BRDd/3XNCGDPGwZQpbQwY4AncW7dGXgXSaoXiYjosfm3Ytw/LiuXkLVuK6T81uA0GXuQy\n/sBs3uDcgP2AGG7QpkhcAV4ptQA4A89vcKbW+l2/124DrgecwN+11rPi+SwhRPrxpTUOzkePPL8e\nLE8fzfz1ZctM7XVjIPgJJ1xddtMn/8KydAl5z63G0NyMq6AQ260zqLu2jNuvPz7IVcpB3b2wRzAx\n5+CVUucCw7TWI4BbgIf9XusN/AwYqbU+GzhWKXVGvI0VQqQXX3B9+mk7xgTd0Qu0qEa4+euB8+vu\n0Pl1lwvWr6fPFeM59NwzsDz5BK5+/Wm89372f/AJTb+5n5zhRwbN9/vr7oU9golnBD8aWAugtf5E\nKVWklOqtta4HWr3/FSilGgErsD/u1goh0tKQIcGnHxYUuAOO1oNtj2f+ekRVIBsbyVv9FJalj8KX\nX5ADtJ59Dvap02m98KIu6/X5lw7evdvYfiPXbjeELCOcrKd4oxFPgO8PbPf7vta7rV5r3ayUugf4\nErADq7TWn4U6WFGRFbM59GVQuisuLuzuJqRUNvU3m/oKkJ9fSE0NlJREXjZlwgRYtKjr9smTjRiN\nsG4dVFdDaalnqTyXy8gf/9h1/5oaE2PHFjJ+vGfFJbMZ8vNh8GDYubPr/qWlBo4/vqBLO4cM6bTj\nzp3wyCNQWQkHDniKfU2eDLNmkXPCCYSaB1Ne7gnYnp+JwdtOKCkxYbWagIOpI4cD7rzT09+vv/a0\n278vqZTIj2s/FXtTNHOBo4F64FWl1Ila6w+CvbmuLoE1PruBzLTIXNnUV4cDHnigkDVrnFGXGJgz\nB+z2nC6LZPziF573zp7dtZxBS4tn/+rqjvn7nTs9Jwu7vaU9v37hhYHLE1x4YQtNTa00NQVolNtN\nr7e3YalYQk7VXzC4XDgP70fznNux33gzhx17pOd3G+Hvt3dv2j/H93Xnz503r2M7A/UlkUINPuIJ\n8HvwjNh9BgA13q+HA19qrf8LoJTaApwCBA3wQojudbAyI4DnajqaEgPh0iOdVzvy7T97diujRlmp\nqel6Be9/wzWqVZZaWshd+zyWiiX0+sgTdtpO+B72W2fQMn4C5CRn3nqyat3EKp4AvxG4ByhXSp0M\n7NFa+06DO4HhSimL1toOfB94Oa6WCiGSIprKjHBwFO7/dahAHk59vYG9exOTXzfU1mJ54jEsyysx\n1n6D22ik5dLx2Mum03b6iKRXc0xorZsEiDnAa623KaW2K6W2AS7gNqXUJOCA1voFpdTvgdeUUg5g\nm9Z6S2KaLIRIpEgqM+7ebWTOnFy2bTOxa5eR/HxPkLLZDHFXigxVHybQDddAJxDTRx9iXbqE3DXP\nYmhtxdW7D7bpP8Z+SxmuwZ2T8ckTT62bZIgrB6+1vqvTpg/8XisHyuM5vhDioGTMyoi0MqPV6u6w\nqIb/7Jd4K0WGemAqUMGwdk4nORuqsCxdQs5Wz/jRceRR2KdOp/nqa6GgIOq2xCtUX7pjKqU8ySpE\nmotnSbpwJ4RY6skEE0+O2T+/3vGGq6HLCcTQUE/e0yuwLC3H9PVOAFrPPR/7rTNoHTWGhE3Ij1FU\n9wqSTGrRJEg2zbSA7Opvd/e186wMn7KywLMyojkhhKsnM2iQi7PPDl7jxV+gei/R2rePoDdcz+6v\nWX/RQgqeewpjYwPuvDyar7zGs6jGMcNj+rxk/m5TNQ8+VC0aqSYpRBqL9AlOf76cenW1CZera8lc\nf6EqM06c2Mqbb9q4//7WEOV5D0pEjrnrDVc35/Mq67iMN/4znN6PP4q7sJDGu3/Nvvc/ofGhRTEH\n92Tz3SvozidcJcALkcYimZXhL7YTQsfH+4cO9Vwd/OEPnnRLqJOAv0TkmH03KXNpZjLL+IATeZXR\nXMZLfJDzfWofXsb+v3+EfeZPcR/aN74PywKSgxcijUU7KyOWaXqdpx8ef3wBTU0dUz+xPq4frYKG\n/7DksGWcVl1JMf/FgYlVXM1CZnHipJO4d2Lq89g9mQR4IdJYtLMy4pmm559S6Px0ZqA56JC4HLP5\ng39gKV9M7ro1XNLWRlNuEX/K+Tm/b7oN18CB3XaTsqeTAC9EmotmVkayp+l1noMe10M7Dgc5VX/B\nWrGEXu+85dl0tPJMc7xyIpdi5dS9Bvr1656FNDKBBHgh0lxEFRL9JHqaXqJngxi+rSPvqRVYHivH\ntKsagJbRY7CXzaDtvFHtT5taifMEIiTAC9FTRFoCINoTQjCxzr8PxvT5vz2Laqx+GoPNhttqxT55\nCvap03F+Z1j0BxRhSYAXIkNFWxOms0ArKEX9xKrbTa/XX8VSsZjcza8A4BxUiv3OX9B8/Y24DymK\nuX0iPAnwQogu4q6KaLOR99xqLEuXYNafAtB22hnYyqbTevG41BdGz1LyUxZCdBFrVUTjnt1Yli0l\nb8VyjHV1uM1mmn90Ffay6ThOOiXZzRadSIAXIoMk6oZotNMtzdvf9aRhXlyLwenE1bcvTT/5Gc2T\npuDqXxJ7Q0RcJMALkQHC3RCNNvBHNN2yrY3c9S9iKV9Mr+3vetox/DjsZdNpnnAlWCwJ7qWIlgR4\nIdJYpIE52A1Rl8tTXDGWmTDBplvec0cNlkWPY1m2FFPNHtwGAy0XXYx96nTazj4n6YtqiMhJgBci\nDUVbETLYDdFVq3rR2Hgwlx7PEnyDDvyLohWPkvf9lRiam3HlF2Cbciv2KdNwHXlUfB0WSSEBXog0\nFG6Kov/IPtQNUf+FOfxFXLvd5eKQba9QUr6YnDdeA8A5eCj2qbfSfM31uHv3ib5zImViDvBKqQXA\nGYAbmKm1ftfvtVJgJZADvKe1nhZvQ4XIFvv2wUsvBf6n+fLLZtraYNOmgyP7MWMcQW+IBhN2fdDG\nRvJWP42l8lHMX3wOQOuZZ2Mvm0HrD8aCKfLPEt0npnLBSqlzgWFa6xHALcDDnXZ5CHhIa30a4FRK\nDY6vmUJkPofDs7iHZ8GLwP80d+0ysnx5x1rvy5bl0qdP4EBdUBB4e7DCY8bqr+FnP6Pv94ZT+Is7\nMVV/TfPE66jbvIUDa1+m9eJLExLcbTbYscMQsHyxSJxYR/CjgbUAWutPlFJFSqneWut6pZQRGAlc\n4339tsQ0VYjMFsni1yYTOJ1dt3/7rYHJk1vYtKnjDVGXCyorux6zQ+ExtxvzO29jXbqEnPUvgssF\nxYfT9PO52G+8Gffhhyegdx6JLn8gQov1R9of2O73fa13Wz1QDDQAC5RSJwNbtNa/CHfAoiIrZnPP\nvuwrLi7s7iakVDb1N9l9tdlgw4bw+zmdgXPqNTUm7r7bxCOPQE0NlJSYsFpNOByQnw/r1kF1NZSW\nwvjx8OCDuZhdBnjmGVi4ELZ7/zmfdBLMmoXx6qvJz80lP4F9BJg1CyoqDn7vu7dgseSycGGCPyxC\nmfx3nKhzpqHT1wOBRcBOYL1S6hKt9fpQB6ir69nXat29bmeqZVN/U9HXHTsMVFfn0/Gfko+bkhLP\nSPeVV8xBHj5yYjbbaGqC3r099dx9Nd3vvhtmzz54Uza/qZaWecswLK/E9M1e3EYjrZdchv3WGbSd\nPoLiw3t7+5vY+us2G6xZYwW6tn/NGiezZ6e+LHAm/B2HOkHFGuD34Bmx+wwAarxf/xf4Smv9BYBS\najNwHBAywAuRzUI9OVpS4uLVV2307euZuhhLrXerFb7T9CGWXywhb82zGFpacBX2xjbtduy3lOEa\nMjSBvQks1vIHInaxrsm6EbgCwJuG2aO1bgDQWjuAL5VSvvqfpwA63oYKkWn8bzSGWvd03DgHfb3L\nj3ZeP7W01ElZWUvwWu9OJzl/fZk+Ey7l0FFnYVn5JM4BA2m473/Z/8En1N51H1+4jkjJzU7fSSyQ\nRCzYLbqKaQSvtd6mlNqulNoGuIDblFKTgANa6xeAWcDj3huuHwEvJarBQvR0wW40zpsXfqGOYEvn\nVVd3fNrV0FBP3sonsVSWY9q5A4DWkedhv3U6rRf8AIfLGPRmZ7Ike7Up0ZXB7U6Ps2ZtbUN6NCRG\nmZDLi0Y29TfRfZ03L/BsmbKyli4PMYUKeoFOFDec9Rm/KHwE68oVGBsbcOfl0XzlROxTpuEcfizg\nuXKYMyeX1atzArahvDw3ab9b/zZ3Pol1xyyaTPg7Li4uDFobQgJ8gmTCH0o0sqm/ieyrzQYjR1oD\n5tpLS51s2RL5jcaDJwo35/E6M1nEZbyIETfO/iU03zwV+w2TcXvzO77gun692ZsL7xoXSkudfPqp\niaam5P5uE70MYKwy4e84VICXmaci66Uy2CTqRqPNBq++7GASTzOLhZzIhwD8jVP586Ez+dmWsVj7\n9Orwnkjm2e/ZY6SmxjMTJ5niXW1KREYCvMha3fHQTbR11gMx7N2LeeFjbN31GIdTiwMTq7mKhczi\nbc7AdABu3N/EEX5Pt4YqSNa5DSUlpvYplqJnkwAvslZC1hz1E8mVQDw3Gs0fvo+lfDG5a5/H0NbG\nt8Yi7nfN4U/cxi5K2/cbMMDZ5UQR6sqhaxskwGcKCfAiK8W95qifaK8EgtVZDziDxeEgp2o91orF\n9HrnLc+mYUdjnzqdX35yI48sP7TLWwKdKEJdOYCbQYNcXHyxrw2h0zii55AAL7JSIh+6ieZKwDfK\nnzu341THzgHZcOBb8p5ageWxckzVXwPQOuoCbGXTaTtvNBiNzHVAa6+WiE4Uoa4cJk5s5f77Iz+h\niZ5DArzISonIhUPkVwKRjvJNX/wbS2U5eSufwmBrwm21Yr/pFuxl03EOO7rD8QPNiQ+96lPwKwcp\n9JWZ5NcqslKiHrqJ9Eog5Cj/Ny30euM1z6LVmzYC4Bw4CPtP59B8/Y24i7qmYTr3JZKrjWhPCKLn\nkwAvslZUufAgIrkSCDbKz8POIc/+mT6vLSTn358C0Hbq6dhunUHrxeNI1rBapihmDwnwIu0la556\nIka0kVwJ7NjRcZQ/gN3cxp+4lXL61u3HVW+mecKV2Mum4zj5+/F2S4h2EuBF2krVPPV4R7ThrgR8\no/z+1duZxUKu5Fl64eC/9OWPhXfxw1cmkXvkgIT0RQh/EuBF2kr0PPVkCXkl0NaG9aUX2WQr52je\nBuBjjmMhs3iK67jxGiMTj0yfvojMEmu5YCGSKtzslHRcy9N3JWC1gqFuP7kLF2A6+gSKfzyZo/e9\nTZXpEi7N28j3jB+ysXQyN5YZ20f5skapSAYZwYu01FMXhzDpT7EsfZS8Z1disNtpJJ+H+TF/5Md8\n7hwGTpg4saV93rlvoW1Zo1Qkg/wJibSUqHnqKeFykfPaJizli8l5/VUAWgcO4b7621nYMIUDHNJh\n961bzfiWw+spaSjRM0mKRqSlUCscJXNxiKhSJU1N5C2vpOjsU+lzzRXkvP4qLaefyZIxqznS+Rn3\nNPy0S3CHg1cgPTENJXoWGcGLtJWIeeqRCjVjBzpO1SzYX43lsQrynnwC44FvcffqRfNV12Avm85d\nz5wWtiSv7wqkp6ahRM8Rc4BXSi0AzgDcwEyt9bsB9vkdMEJrfV7MLRRZK5VPXgZLlbhckJ8Pa563\nMHj3O9yVt5CxzS9gcjtxHVZM0513Yb/pFtz9+kVcktd3BdKj0lCiR4opwCulzgWGaa1HKKWGA8uA\nEZ32ORY4B2iLu5UiqyX7yctQgfn5lW7GNj3FcyziNN4FO/yD7/HBuTO4ZMXlkJfXvm/okrxuSkpc\njBt38KpA1igVyRbrCH40sBZAa/2JUqpIKdVba13vt89DwN3A/PiaKERyBQrMh1FLGRXc1vQnBlCD\nCwMvcDkLmcX/cQ6lX7o432XDPwaHGpGXlLh49VUb3tXz2qUyDSWyT6wBvj+w3e/7Wu+2egCl1CTg\nDWBnpAcsKrJiNgeqVd1zFBcXdncTUqon9Ndmg5oaKCkh6Ig4Px8GD4adO+F4PmImi7ieJ8mjhXoK\n+QOzeYTb2cGR7e/Zs8eEw1FIcXHHY02YAIsWdf2Mq64yccwxgX9e5eX+7TRhtZro7prsPeF3myiZ\n3NdE3WRtX/RVKXUoMBm4ABgY6QHq6nr2lIFMWLw3Gt3V30jr0kRV5sDl4k61mWN2/onReKY5fs5R\nPMwdPJd/IzVNXWfCDBjgxGy2UVvbcfucOWC353QZkc+Z09pl385694amJrp9NaVs+lvOhL6GOkHF\nGuD34Bmx+wwAarxfjwKKgS14hiFHKaUWaK1nx/hZQkQVsG02mDMnl9Wrc9q3BZpfbmhsIHfVU1iW\nPsptO74EYGvu+fy+bSbvDxjLDy52c4kLKiu7tidYjlxK8op0EmuA3wjcA5QrpU4G9mitGwC01s8B\nzwEopYYCj0twF/GK5IEg30lg/Xpz0JudVVVmfnXDZxQ9WU7e0yswNtTjzs3Ffu0N2KdOZ9ARxzN3\nr4F+/VranzTNz89lzRpnVDlyKckr0oHB7Y7tj1ApdT+eWTIu4DbgJOCA1voFv32G4gnw54U7Xm1t\nQ4/+15AJl3rRSFR/I0m52GySFcCgAAAbqUlEQVQwcqQ14M3L0lInW7bYsFo9j/wHn4Pu5hz+j1ks\n4nLjOgwuF87D+9F881TsN96M+7DDgraxuLiQr75qyJoReTb9LWdCX4uLCw3BXos5B6+1vqvTpg8C\n7LMTOC/WzxCZK5qUSyQPBPXr5w441TGHFiayilks5CTeB6Dl+JNomTaDlst+CDk5Xd4TiIzIRU8k\nT7KKbhFNDZZIHgjqfBLox3+YxqNM41H6sxcnRp7hSv45ejo3V56CNT/ooCcqyVqMRIhEkFo0IuWi\nrcESSV0a30ngJN7jcW7iK4Ywn3vIpYXf81O+a/2cWwpWce9r5zHynHzuuiuHzz+PvTyvrwrkyJFW\nRozIZ+RIK/Pm5eAI3EwhuoWM4EXKffVV9DVYQj4Q5HRyyGsvs8mxhKN5E4BPUSxiJn/mRgYdnctn\nnx38U6+uNrFsmYlly3IoLY2+PG+ks3SE6G4S4EXK+M9ycbkC7xOsBkug6Yf5jgPkVa7AUlmO6euv\n6AP8c9AYHmi+g5X7x1IyECaOcfDKK8HSMYaoAnOks3Tmzm2VdI1ICxLgRcp0zrsH4ku5BMttW63w\nHfe/ybu3nLyVT2FsasRtsWC/8WbsU6dxuDqG39jg9r329tz844+Hv5EaKDD72pCfH3n7pQqkSCcS\n4EVKhK606GbQIBcXX+xg3rzWwCsc/boFy1tvYFm6hJyNf8XgduMsGUDjrJ/SfMMk3IceLPLiP+Ml\n1A1af/6BufMMn8GDYdSoHF55Jfw/F6kCKdKJBHiREqGmOhqN8NRTdoYPd3eZy15b3Yqz4ilczy7k\nkLp/AtB2yqnYb51ByyWXQa9eIT83VMVGf/6BufNIfedOWLYsF09l7NCkCqRIJxLgRUqEGkkPHOhi\nyBB3h1F+CXuYwWKm8SiHsY+2OjNNl11B6/TpOE45NarP9r9BW11txK90Ujv/1FCwKw2TCZzOQK8c\nvAKRKpAincg0SZESkUx13LvXQMmu7TzJdXzFEObxWwy4uY9f8B3jl/zz7uVRB3c4eIN2yxYbW7c2\nMXlyC6WlTkwmN6WlTsrKWtoDc6grjcDBHSZObOXNN23ce68slC3Si/w5ipQJOtVxno2cF1/ihCWL\necf9DgD/5FgWMZMnuR47VkoHOunXr+Ok9WgfMrJaYdgwNw880IrNFrgYWKgrjUGDXIwZ42DTpq5T\nNSWwi3Qkf5YiZTpPdSzJ20/Rc09gGVGBafcuAD4qvYifVP+ETVyAfyrFP7cdVSngIIKVHgiVs7/4\nYgf33hv85CBEupEAL1KucPdnHL50CXnPrMRgs+G25mO/eSr2qdM4bMgwjpyfQ2mVK2j1xmjKHMSi\n85VGaamBCy88mMaRujSip4i5mmSiSTXJniXq/rpc9Hp9M9aKJeS8ugkAZ+lg7LfcSvN1N+Du03FR\njWDpl0grSyaCrw3HH19AU5P8bjNRJvQ1KdUkhQjHZoPanTaOeHMlfZ5YgvnfnwHQesaZ2KdOp3Xs\nJQTLqQQbJUdSWTJRo2tfG6zW7l9lSYhYSIAXCedwwKI79zL4pQomNlRyKHU4jL2w/ehqWqbNwHHi\nSTEfO5LKkkIID5kmKRLH7cb87jvsPvtm7n36GGY0/B4HZu7hV5S6vmJ23yfiCu4Q2XRLIYSHjOBF\n/FpbyX1pLZaKxfT6x3t8H/iAE1jILFZyDS3kAVBV5UxIIa6QlSWFEO1iDvBKqQXAGXie356ptX7X\n77Xzgd8BTkADU7TWQeoHip7KsG8flhXLyVu2FNN/anAbDNSNvJgr3/wJm92j6PzEaKJy5IEqSwJU\nV8vURSH8xZSiUUqdCwzTWo8AbgEe7rRLBXCF1vosoBC4KK5WirRi+uRfUFZG35OGk3/f/2BobMRW\nNp39b/+DvRWr+Ff/8whUDiDROXKrFUpL3dx3nyy8IUQgsY7gRwNrAbTWnyilipRSvbXW9d7XT/H7\nuhboG+ggogdxucjZtAFL+RJytrzu2TRkKPap02i+5nraLL3bHz6qqQk8bkhGjjzZc+KF6MliDfD9\nge1+39d6t9UD+IK7UqoEuBD4ZbgDFhVZMZtDV/xLd8XFhd3dhMRraIAnnoBFi+Dzzz3bzj8fZs3C\ndMklFJhMFACzZkFFRde3GwwwZAiMHw8PPpiL2Ry6nno0bDbYsCHwaxs35rJgQW7CTigZ+bsNIZv6\nm8l9TdRN1i7X40qpw4GXgBla633hDlBXF+PimGkiEx6Y8Gf8aieWxyrIe3oFxvoDuHNyaL7meuxT\np+M8/rsd+muzwZo1VqDrCbp/fydVVTb69oW6usS2cccOA9XV+QRKB1VXu/n446aEzInPtN9tONnU\n30zoa6gTVKwBfg+eEbvPAKDG941SqjdQBdyttd4Y42cIP9EW1orpvW43vd55C0v5YnKq/oLB5cJ5\neD+apt2G/aZbcBcXB3xbqIePvvnGSH29gb59Ez8/XebECxFarPPgNwJXACilTgb2aK39T4MPAQu0\n1n+Ns31Zz+GAefNiu4kY8XtbWshd/TSHjDmXQy67iNz1L+I4/gTqHyln//aPsd15V5fgbrN5RtA2\n28FAG0gyA63MiRcitJhr0Sil7gfOAVzAbcBJwAFgA1AHvOW3+9Na6wAZ2oOkFk1gnVc48ikra/FW\nNgw+Og/3XkNtLZYnHsOyvBJj7Te4jUZaLx6HrWwGjtPP8CTQO/FVcty4MZevv3a3V3J0uaCyMvhn\nJYt/ZclklfDNhMv4aGRTfzOhr6Fq0UixsQRJxh9KqMJagwY522uTByqZG+q9P+i3nedGLiD/xWcx\ntLbi6t2Hhok3sWNsGUUnDQ458g120pgypQWjMfDDR6molR5PCiucTAgC0cim/mZCXyXAp0Ay/lB2\n7DAwYkQ+Lleg35+bQDcXfSPmzu814mQcLzGTRZzP6wA4jvoOTTdP45efT2LtpkPC1lYPddIoKXHy\n6qs2LJbkBdrukglBIBrZ1N9M6KtUk0wT0Y4yQ91EDLY+aFWVmblzW9vf+211EzezjDt4mCPZAcAr\nXMCKvndgHXUBzh1GKpdHNo881M3Umhojo0ZZGTdOVjgSIl1IsbEUiPVGaaibiMHWB/WVAyjY+yXL\n+8xkNwNZyGxKqKGcMo7jYy7kFVbsG0f5UgurVvUKeJyqKjM2W+Q3U8FATY2Jiopc5s/PCd0xIURK\nSIomQUJd6oW72RlKoJuIY8Y4eOUVc4CRvZurijez/MQFWDZXYXC7qcsfyKOmGTxYX8Z+DgvwCYFT\nPUajmyuvbGPbNlOH1E2wm6n+Yl14I5l59HhkwmV8NLKpv5nQV8nBp0CwP5RErUDUOfj5nzRyaeZa\nnmYmiziRDwFoO/kU7GUzaBl3OZ98nsP550eXyy8ocNHY2PUCz3czdf36XHbvDvxek8nNtm2RP2SU\niDVWkykTgkA0sqm/mdBXycF3o0StQNR5haP581vpbath4IuVXFtfweHU4jSYsI+bQPO0GTi+f1r7\nvkOGBM/lW61ubLagfx9dbNhgZssWG7/9bS4nnOCipib+h4yknowQySE5+CRLxkNA5vffo+iOqfzv\n6qOZVf9bDu3t4MD0n/DtPz6msfLxDsEdQufybTYDBQUuCgpcmExuSkudTJzYEjTo+05Khx0G48ZF\n/pCRfy6/8/aqqsDjDN99ACFEbGQEn2S+4OobkfqL6mlLh4Ocqr9gLV9Mr7+97dl0tMJeNoPmK64m\n3IH8F8morjZyMLVioLHR8/XEiS3cf79nv61bA+X4O56UIll4I1z6JZVrrAqRbSTAp0A8KxAZvq0j\n78k/Y1lWgWlXNQAto8dgnzqdtvNHB3zaNBDfIhmzZ7cyapQ1YGpl61Yz0BrxSSnQwhudzzPh0i9S\nT0aI5JEAnwKRBMLOTJ//m15LlmB99mlMzTbcViv2Sbd4qjkOOzrmttTXG9i7N/yIOZqTUuf7Az7h\n0i++5fsScoUjhOhCAnwKBQuE7dxuer3+KnkVS8jb7CnC+TWlrOjzK/57+U38/L78uGeVRDpiDrcs\nXiQiTb/IGqtCJIcE+HRgs5H33GosS5dg1p8C8CZnsZBZrOVynAfM8ATYcuMv3BXJiLnzlMzSUneX\nPPqECTBnDiFPOPGcTGTkLkT8JMB3I+Oe3ViWLSVvxXKMdXW4e/Wi8YdXM3HbbNbvPb3L/v5pjXgE\nGzHPm9fKvHldb4h2friputrEokVgt+eEPOFEm34Je4UjhIiKPOiUIPn5hXz8cWNEo0/z3/+GpWIx\nuS+tw+B04urbF/tNN9M8aQpf2AcELTAW7QNE4YR6eMpfsIeeInlQKxXlfJMtEx6GiUY29TcT+ioP\nOiXRwfro8PXX+cGfwmxrI/cv67BULKbX9r973jv8OOy3zqB5wpWQlwdAP1vqZpX4j5hD3RD1TaPs\nLJJpjJJ+EaL7SICPU8dpgIYu0wAN+/eR9+QTWB6rwFSzB7fBQMtFF3umOZ59Tpdpjt01qyTUDdFg\nojnhSPpFiNSTAB+HUKPez1/8jLyGBRSsXYXBbseVX4Bt6jTst9yK68ijQh63O2aVhLohWlDgDjiK\nl2mMQqS3mAO8UmoBcAaealUztdbv+r12AXAf4ARe1lr/Jt6GpqPOo14DLi7ir8xiIRf+5xVYCc7B\nQ7FPKaP52htw9+4T0XG7I60R6sph4sS2Lqs1TZhgYs4cmcYoRDqLKcArpc4FhmmtRyilhgPLgBF+\nuzwM/ADYDbyhlHpea/2vuFubZnyj3v3Vdm7iCe7gYRSfAfBW7jkc9fA0DJeN9azOEYNUpzVCXTmY\nzXQ44QwZUkhtbcqaJoSIQawj+NHAWgCt9SdKqSKlVG+tdb1S6khgv9a6GkAp9bJ3/4wL8AX7vqay\n6DFOq17GIRyghRyWM4lFzOT0m4Zz7w971gg33JWD5NGF6FliDfD9ge1+39d6t9V7/+8/tvsGCJ10\nBoqKrJjNsY10U8rthq1bYeFCeOEFLnS5qLf2Y0HObH5fP43cwf0YPx4efBDM5tALY6SzIUPC71Nc\nXJj8hqSJbOorZFd/M7mvibrJGqriVUTVsOrq0rwubGsruevWYKlYQq8P/gFA23dPxD51Gi0/vIJb\nDz2Msz5upF+/BqxWqKvr5vYmWSbMH45UNvUVsqu/mdDXUCeoWAP8HjwjdZ8BQE2Q1wZ6t/VIhv/+\nF8sTj5G3vBLTN3txG420XDwO+60zaDvjzPZpjpK+EEKkm1gD/EbgHqBcKXUysEdr3QCgtd6plOqt\nlBoK7AIuBa5LRGNTyfTxR1iWLiFvzbMYWlpwFfbGNu127LeU4RoyNOR7I1lbNF3XHxVCZI6YArzW\neptSartSahvgAm5TSk0CDmitXwCmAyu9u6/WWn+WkNYmm9NJzsa/Ylm6hJw3/w8AxxFHYi+bTsvV\n1+IuCJ2ri2Rt0UD7jBnjYMqUNgYMiCzYy8lBCBEJqUUDGBrqyVv5JJalj2L6aicAreecj71sGq0X\n/ABbszFsQC0uLuTWW1sC1nIpKztYBTJYvRdwU1oauk5LOi1OnQm5y0hlU18hu/qbCX2VWjRBGHfu\nwPJYOXlPrcDY2IA7Lw/7dTd6FtU49jhPQP1VZAE1ksUtfF8HdrDMQX093H9/16qR0SxOLaN8IUT2\nBXi3m17b3sRSvpicDS9jcLtx9utP0x2zsd8wGXffvu27RhNQa2oIu7gFBN/H36pVObz5ppmLLz54\nMol0daR0GuULIbpX9vyTb24m94XnsFYswfzPjwBoO+lk7GUzaBl3OeTkdNg90oDqU1JCRFUgg+3T\nkYFduzqeTCJfHSnyk5IQIrNFVz6wBzLs3Yv1gd/S9+Rj6T1zBqZP/0Xz+AnUrX+Fb//6Gi0/uqpL\ncIfIlpvz56vlEoivKFeofYKpqjJjsx0sixCI7wQS7qRkS/NHDYQQiZWxI3jzh+9jqVhC7gvPYWhr\nw3XIIdhun4X95qm4BpWGfX+ky835i6QKpP8+1dVGwj0H5j86D1dGeMeOyEb5QojskHEBPmfTBiwP\nLyDn7W0AOIYdjX3qdJqvnAj5+REfJ5a67JFUgfTfZ/duA5WVvdi40ewNzF2Dvf/JJNwJJJaTkhAi\nc2VUgDfs30efa68EoHXUBdjKZtB23igwxpaJirUueyRPtVqtMGyYmwceaOXXv25lzpxcVq/umiry\nP5lEUgysOxYLEUKkp4wK8O5D+/LtqudxDRqM82gV9/FSVZfdaoUFC1ro08cd0ckk1AmkOxYLEUKk\nJ3nQKUGCPTAR7Xz0RM1fT/Y8+Ex4QCRS2dRXyK7+ZkJf5UGnbhDrfPREFS2T4mdCCAnwcfAfJXcm\n89GFEN0t4+fBJ4PD4akpM3KklREj8hk50sqsWZ7tEP4hKZmPLoRIBQnwMfCNzqurTbhcnhoyixZ5\ntkP0D0kJIUQySICPUiSj80ieOk1W23bsMMgVghACkAAftUhG55GULUikQCmjefNy2lNGQojsJDdZ\noxTp06KpnI8uN3SFEIHICD5KkY7OfQ9JbdliY9u2JrZssXHvvYkv2Ss3dIUQwcQUbpRSvYDHgSGA\nE5istf6y0z5XAz/Fs6TfZq313fE1NXrJetgn0Oh8wgQTc+ZE99RpIkRaRlgIkX1iHcFfC3yrtT4b\n+C3wO/8XlVJW4AFgNDACuEApdWw8DY1GsnPSgUbnCxfSLQtqdNcNXSFE+os1wI8GXvB+vQk4y/9F\nrbUN+K7WukFr7Qb2AX1JkUDTGCsqctunMSaKb3TenUW8Un1DVwjRc8Qa4PsDtQBaaxfgVkp1iJ5a\n6wYApdR3gaHA27E3M3LZmJOeP7+VsrIWSkudmExuSkudlJW1SIExIbJc2KSCUmoKMKXT5tM7fR/w\nyR2l1DDgaeBarXVbqM8pKrJiNodbyi68L76A3bsDv7ZnjwmHo5Di4tiPb7N51l8tKaHL6Li4uDD2\nA8epvNy/bSasVhOQG/Z98ejO/qZaNvUVsqu/mdzXsAFea10JVPpvU0o9jmcU/4H3hqtBa93aaZ9B\nwFrgBq31++E+p64uMUNrsxkGDrQGmcboxGy2UVsb/XHDFQ9Ll6p0vXtDU5Pnv2RKl/6mQjb1FbKr\nv5nQ11AnqFhTNBuBK71fjwNeC7DPY8B0rfV7MX5GTJKVk05VXl8IIRIl1nkfq4ExSqk3gRZgEoBS\n6i7gDTw3VUcC/6NU+8Ibf9BavxhXayOUiIeMOleKDJXXnztXct1CiPST0Qt+xDIPPlAq5qyzHDzz\nTA4uV9dbDSaTm23bmjjttIIef6kXjUy4tI1UNvUVsqu/mdDXrFrwo3NQj/Yhn0CP/a9aZaKgwEVj\nY+hFsYUQIp1kTKmCaB5uClZ1MdQUy2BkrrkQIl1lzAg+koJbgdIvY8Y4mDKljQED3CEf+7fZDFx9\ndSvbtplkMWshRI+QEQE+3MNNc+e2YrUGPgksW2Zi2bIcSks9wT5YpciBA1088EALkNzFrIUQIlEy\nIkUTScGt0OkXgzfY59KnT+B8ui8Vkw7lCYQQIhIZEeAjKbgV6iTg79tvDUyeLI/9CyF6voxI0fge\nbvLl3P35Rt6hFurwV1NjZNq0Nn7961ZJxQgherSMGMFDZAW3zjzTGfY4vhG/pGKEED1dRozg4WCN\n9rlzO468fdMnq6rM7NplpKDAk8rxzGnvOq9dpj0KITJFxgR4n84PN3WeOeN7WOmqq1rIz4dNm5K/\nZqoQQnSHjAvw/kLNnHnrLTNbttgk1y6EyFgZk4MPJJLpk5JrF0JkqowO8LJeqRAim2V0gJf1SoUQ\n2Syjc/AQXW34WMoLCyFEusr4AB9s+qS/cMvxCSFET5Q14StUbfhIKlEKIURPE1OA9y60/TgwBHAC\nk7XWXwbZdyXQorWeFGMbkyrSSpRCCNHTxHqT9VrgW6312cBvgd8F2kkpNQY4KsbPSIlIplIKIURP\nFGuAHw284P16E3BW5x2UUrnAPODeGD8jJWQqpRAiU8Ua4PsDtQBaaxfgVkrldNrnF8ASoD725iWf\nTKUUQmSqsDl4pdQUYEqnzad3+r5DHkMpNQz4vtZ6vlLqvEgaUlRkxWwOXco3Wf70J7BYYN06qK6G\n0lIYPx4efDAXszk3/AG8iosLk9jK9JNN/c2mvkJ29TeT+2pwu6NPQSilHgdWaq03eG+47tRaD/R7\nfRZwM2ADegPFwO+11v8b7Ji1tQ3dnguJZx58cXEhtbUNyWlYGsqm/mZTXyG7+psJfS0uLgx6ozDW\naZIbgSuBDcA44DX/F7XWC4GFAN4R/KRQwT1dhJpKKYQQPU2sOfjVgEkp9SZwG558O0qpu5RSIxLV\nOCGEELGLKUWTDOmQoolHJlzqRSOb+ptNfYXs6m8m9DVUiiaji40JIUQ2kwAvhBAZSgK8EEJkKAnw\nQgiRodLmJqsQQojEkhG8EEJkKAnwQgiRoSTACyFEhpIAL4QQGUoCvBBCZCgJ8EIIkaEkwAshRIaK\ntVxw1sukhccjEUl/lVJXAz8FXMBmrfXdqW5nvJRSC4AzADcwU2v9rt9rFwD34en/y1rr33RPKxMj\nTF/Px7PWshPQwBTv6m09Vqj++u3zO2CE1vq8FDcvKWQEH7uMWXg8QiH7q5SyAg/gWa93BHCBUurY\nlLcyDkqpc4FhWusRwC3Aw512eRj4EZ41iC/saf3zF0FfK4ArtNZnAYXARSluYkJF0F+8v89zUt22\nZJIAH7uMWXg8QiH7q7W2Ad/VWjdord3APqBvapsYt9HAWgCt9SdAkVKqN4BS6khgv9a62juSfdm7\nf08VtK9ep2itd3m/rqXn/S47C9dfgIeAHnfVGYoE+NhlzMLjEQrbX611A4BS6rvAUODtFLcxXu19\n9Kr1bgv02jdASYralQyh+orWuh5AKVUCXIjnhNaTheyvUmoS8AawM6WtSjLJwUcgVQuPp4tY+uv3\n3mHA08C1Wuu2JDQvlYIupBDmtZ6oS3+UUocDLwEztNb7Ut+kpGrvr1LqUGAycAEwMOg7eiAJ8BHQ\nWlcClf7bvAuP9wc+8N6ANGitW/12uQQYrJR6G+/C40qpn/eEtWlj7C9KqUF4LoNv0Fq/n6LmJtIe\n/EZ1wACgJshrA73beqpQfcWbvqgC7tZab0xx25IhVH9HAcXAFiAXOEoptUBrPTu1TUw8SdHEzrfw\nOARZeFxrfYLW+gxgBrC+JwT3EEL21+sxYLrW+r2UtSqxNgJXACilTgb2+NJOWuudQG+l1FCllBm4\n1Lt/TxW0r14PAQu01n/tjsYlQajf7XNa62O9/1Z/CLyXCcEdpFxwzJRSJjyj3GFACzBJa12tlLoL\neENr/Zbfvud5X5/UHW1NhHD9xXNT9X3gb35v+4PW+sWUNzYOSqn78cykcOFZUP4k4IDW+gWl1Dl4\nZgoBPK+1frCbmpkQwfoKbADqgLf8dn9aa12R8kYmUKjfrd8+Q4HHM2WapAR4IYTIUJKiEUKIDCUB\nXgghMpQEeCGEyFAS4IUQIkNJgBdCiAwlAV4IITKUBHghhMhQ/w98v15j/MiJggAAAABJRU5ErkJg\ngg==\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7f72eecd1750>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "0yPW3udd6EyG",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "output_extras": [
            {
              "item_id": 1
            }
          ],
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "25016a4a-590f-4dc8-ebd8-a01f481b5c13",
        "executionInfo": {
          "status": "ok",
          "timestamp": 1509629152921,
          "user_tz": -480,
          "elapsed": 772,
          "user": {
            "displayName": "Mike C",
            "photoUrl": "//lh3.googleusercontent.com/-xWyBhueI0Vo/AAAAAAAAAAI/AAAAAAAAIg8/rU-qliu5l-M/s50-c-k-no/photo.jpg",
            "userId": "110113503404408134511"
          }
        }
      },
      "source": [
        ""
      ],
      "cell_type": "code",
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([-0.22414903], dtype=float32)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 18
        }
      ]
    },
    {
      "metadata": {
        "id": "S7e1bXzRh9lx",
        "colab_type": "text"
      },
      "source": [
        "## Train your first deep model\n",
        "\n",
        "Now we have learned the basics of Chainer. We can use it to train a deep model to classify MNIST digits. We will train a model on the MNIST dataset because the dataset is small.\n",
        "\n",
        "First we load the data and see what the images look like:"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {
        "id": "BF2lwVp-h9ly",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "output_extras": [
            {
              "item_id": 5
            },
            {
              "item_id": 6
            },
            {
              "item_id": 7
            }
          ],
          "base_uri": "https://localhost:8080/",
          "height": 650
        },
        "outputId": "ce8472df-30f8-40cb-9c7a-5ea93a809261",
        "executionInfo": {
          "status": "ok",
          "timestamp": 1509629264181,
          "user_tz": -480,
          "elapsed": 68556,
          "user": {
            "displayName": "Mike C",
            "photoUrl": "//lh3.googleusercontent.com/-xWyBhueI0Vo/AAAAAAAAAAI/AAAAAAAAIg8/rU-qliu5l-M/s50-c-k-no/photo.jpg",
            "userId": "110113503404408134511"
          }
        }
      },
      "source": [
        "train, test = chainer.datasets.get_mnist()\n",
        "# use train[data_point_index] to access data\n",
        "print(\"train[i][0] is the ith image that's flattened, and has shape:\", train[12][0].shape)\n",
        "print(\"train[i][1] is the ith image's label, such as:\", train[12][1])\n",
        "# here we visualize two of them\n",
        "plt.imshow(train[12][0].reshape([28, 28,]))\n",
        "plt.title(\"Label: %s\" % train[12][1])\n",
        "plt.figure()\n",
        "plt.imshow(train[42][0].reshape([28, 28,]))\n",
        "_ = plt.title(\"Label: %s\" % train[42][1])"
      ],
      "cell_type": "code",
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading from http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz...\n",
            "Downloading from http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz...\n",
            "Downloading from http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz...\n",
            "Downloading from http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz...\n",
            "(\"train[i][0] is the ith image that's flattened, and has shape:\", (784,))\n",
            "(\"train[i][1] is the ith image's label, such as:\", 3)\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPoAAAEHCAYAAACHl1tOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAESxJREFUeJzt3XuwVeV5x/EvchGKF7wR9KAwTJgn\nIDoOOBptIZjEymgMZY4MmVq1ak3rhIydjuNBU+tloGIY5iRFqpPRiNFG5dKKxksTrBG8jHFEM9HC\no4YUp2AEpKJ4OQKH/rE3uPfh7LVva+29Oc/v89de691rreds+fmu+9tv7969iEjfdkizCxCR7Cno\nIgEo6CIBKOgiASjoIgEo6CIBDGh2AZIOM9sLnOju/1vFMv8D/JW7P1fFMkuAt919bpnvtQM3AoOB\nbcDfufvrlW5H0qUeXVJnZicBdwHT3f0rwDLgp82tKjb16H2cmf0JcC9wGjAIWOHu1xZ85etmtgg4\nFrjP3f8xv9x0YC4wFHgb+Et339Zj3bcBG939rh6b3ZX//sb89NPAren+ZVINBb3vuxo4HPgKMAx4\ny8weKdhdnwScDhwDrDezZcBHwP3A2e7+upldT66Hvqhwxe5+fW8bdPd3gXcBzGwA8NfAypT/LqmC\ndt37OHdfSG4Xeq+7/x/wBjCm4Cv/5u573H0L8CxwFjAN+HXBMfVdwLfNrH812zaza4D3gMlAR51/\nitRBQe/jzGws8O9m9paZrSfXexf+d99a8HkHcBS5nn+Kma3PL/Nivu2Yarbt7j8md0jwI+AFMxtS\n+18i9dCue9+3GHgF+At332Nmz/doP7rg81HAdqALWOXuF/X4LmZWdoNmNg5oc/dV7r4XeNDM7gAM\neK22P0PqoR697xsOvJoP+bnAWOCwgvbvmNkhZjac3C72GuA/gclmNgbAzM4wsx9Xsc3jgJ+Z2Qn5\n5f8UGAhsqP/PkVqoR+9bfm1muwum/4bcmfNOM/sn4BHgFuBWM3s1/52Xgd+Q+x9Cp7v/N4CZXQX8\nh5kNIndy7u97bqzUWXd3X21m84BVZnYIuT2E77j7hyn+rVKFfnoeXaTv0667SAAKukgACrpIAAq6\nSACNOuuuM34i2etXqqHmoJtZJ/BVciG+xt1frnVdIpKtmnbdzexrwFh3Pwu4EviXVKsSkVTVeoz+\nDXI3X+Du64CjzOyI1KoSkVTVGvQRFD8MsTU/T0RaUFpn3UueBBCR5qs16Jsp7sFPIP+iARFpPbUG\n/Zfk3zZiZhOBze7+UWpViUiqan6oxczmA1OAbuB77v7bhK/rOrpI9koeQjfq6TUFXSR7JYOuW2BF\nAlDQRQJQ0EUCUNBFAlDQRQJQ0EUCUNBFAlDQRQJQ0EUCUNBFAlDQRQJQ0EUCUNBFAlDQRQJQ0EUC\nUNBFAlDQRQJQ0EUCUNBFAlDQRQJQ0EUCUNBFAlDQRQJQ0EUCUNBFAlDQRQJQ0EUCUNBFAlDQRQIY\n0OwC6vX5558ntj/zzDOJ7UOGDCmanjJlCqtXr94//fzzz5dcdseOHYnrXrRoUWL7jBkzEttHjhxZ\nND1//nzmzJmTuExa2traEtunT59eNH3SSSfxzjvvFE1L66gp6GY2FVgGvJGf9Tt3/35aRYlIuurp\n0Z9194tSq0REMqNjdJEA+u3du7fqhfK77v8KvA0cDdzi7r9KWKT6jYhItfqVbKgx6G3AnwFLgTHA\nM8CX3b3UmbHMgq6TcdnQybiDUsmg13SM7u6bgIfzk783sz8CbcAfalmfiGSrpmN0M7vYzK7Nfx4B\nfAnYlGZhIpKeWnfdDwd+DgwDBpE7Rn8iYZHMdt0XLFiQ2N7R0VHV+rq7uznkkNY8R9lKtfWsY/fu\n3QwY8MUO4umnn15y2auuuipx3e3t7Yntw4YNq6DCkFLfdf8IuLDmckSkoVqjexCRTCnoIgEo6CIB\nKOgiASjoIgHUdHmtBpltZPz48Ynt69evr2p91VzCGj58eGL75MmTq9p2OcuWLWPmzJkVfXfcuHGJ\n7evWrUts37JlS2L7mjVriqbTvPS3du3axPbTTjstle30QSUvr6lHFwlAQRcJQEEXCUBBFwlAQRcJ\nQEEXCUBBFwngoH/d83PPPZfYXvjWk9709iaUbdu2VbTtQYMGJbYfdthhFa2nGsuWLUt9nb3p6upK\nbD/55JMPmDdmzJj9nzds2FDztsv9jbqOXj316CIBKOgiASjoIgEo6CIBKOgiASjoIgEo6CIBHPTP\no0s2Cker6c3UqVOLpqt5Hn3w4MGJ7eWelR81alRF2wlIz6OLRKagiwSgoIsEoKCLBKCgiwSgoIsE\noKCLBHDQP48uvduzZ09i+0033ZTY3tnZmWY5Rd58883E9pEjR2a27agqCrqZTQBWAp3ufoeZnQjc\nD/QH3gUucffkNxWISNOU3XU3s6HAIuDpgtm3AovdfTLwNnBFNuWJSBoqOUbvAs4HNhfMmwo8mv/8\nGPDNdMsSkTSV3XV3993AbjMrnD20YFd9C3B8BrVJHfr375/YPnfu3Lrae9Pd3V31MtIYaZyMK3kj\nvTRP1ifjPv3006Lpah5qKffCTp2MS1+tl9d2mtmQ/Oc2infrRaTF1Br0VUB7/nM78FQ65YhIFso+\nj25mk4CFwGhgF7AJuBhYAgwGNgKXu/uuhNXoefQMJI39fs899yQuu3Dhwrq2PXDgwKLprq4uDj30\n0P3TK1asKLnstGnTEtc9YIBu76hRycPoSk7GvULuLHtP59ZRkIg0kG6BFQlAQRcJQEEXCUBBFwlA\nQRcJQNcxWtjGjRuLpkeNGlU075RTTim5bLk74+rV211whfOS7m7r1083UzaaenSRABR0kQAUdJEA\nFHSRABR0kQAUdJEAFHSRAHQdvYU99NBDRdMdHR1F87K+Vp6kq+vAl/4Wzps4cWLJZc8555zEdc+a\nNSux/cILL0xsP/54vdmsJ/XoIgEo6CIBKOgiASjoIgEo6CIBKOgiASjoIgGUfd1zSvS65xps2LCh\naHrMmDFF85JGW1m1alXiut977736iuuhmpFa6lVuO/PmzSua7ujo4Pbbbwdg9uzZicsOHTq0vuKa\nq+SD/urRRQJQ0EUCUNBFAlDQRQJQ0EUCUNBFAlDQRQLQdfQ+aseOHYntH374YWL79u3bE9sffPDB\noun58+czZ86c/dMLFiwouWyD/s3tV3iNf8aMGYnfXb58eWJ7i7+TvvZhkwHMbAKwEuh09zvMbAkw\nCXg//5UF7v54vVWKSDbKBt3MhgKLgKd7NF3v7r/IpCoRSVUlx+hdwPnA5oxrEZGMVHyMbmY3A9sK\ndt1HAIOALcBsd9+WsLiO0UWyV98xei/uB95399fMbA5wM5D8tIA0lE7GfSHQybiSagq6uxcerz8K\n3JlOOSKShZquo5vZCjMbk5+cCryeWkUikrqyx+hmNglYCIwGdgGbyJ2FnwN8AuwELnf3LQmr0TF6\nMKtXry7ZNnfu3MRlyz1LX61qnpVfsmRJYvull16aQkWZqf0Y3d1fIddr97SijoJEpIF0C6xIAAq6\nSAAKukgACrpIAAq6SAAaNlkyMWXKlJJtTz31VOKy7e3tie0rV66sqaZKrFu3LrN1N5N6dJEAFHSR\nABR0kQAUdJEAFHSRABR0kQAUdJEAdB1dGq7cI6NnnnlmYnuW19EnTJiQ2bqbST26SAAKukgACrpI\nAAq6SAAKukgACrpIAAq6SAC6jp6hnTt3JrY/8MADie2nnnpq0fTZZ5/NCy+8UDR9MOru7k5sX7t2\nbWbbHjAg+Z/8GWeckdm2m0k9ukgACrpIAAq6SAAKukgACrpIAAq6SAAKukgAFV1HN7MfApPz378N\neBm4H+gPvAtc4u5dWRXZqspdJz/33HMT21966aXE9k8++eSAeRMnTixfWAv4+OOPS7bdeeedicsu\nX7487XL2mzRpUmL72LFjM9t2M5Xt0c3sHGCCu58FTAN+BNwKLHb3ycDbwBWZVikidalk1301MDP/\n+QNgKLnx0h/Nz3sM+GbqlYlIasruurv7HmDfftiVwBPAeQW76luA47MpT0TS0G/v3r0VfdHMpgM3\nAH8OvOXuw/Pzvwz8zN2TbryubCMiUo9+pRoqPRl3HvADYJq77zCznWY2xN0/BdqAzenUeXBp9Mm4\nwYMH89lnnxVNt6p6TsZdd911qdbS3d29/4WU5V48+eKLL6a67VZRycm4I4EFwLfcfXt+9ipg35CX\n7UDy8Jgi0lSV9OizgGOBpWa2b95lwN1m9rfARuC+bMprbR0dHYnt5Xrsct5///2i6ba2tqJ5w4cP\nL7nswIED69r2rl27Etvvvvvuoumrr766qKe+4YYbSi67Y8eOumord7h55JFHHjDviCOOAOC++0L+\nU63oZNxPgJ/00pS8XyoiLUN3xokEoKCLBKCgiwSgoIsEoKCLBKCgiwRQ8S2wdeqTt8A++eSTie0X\nXHBBqtsrvMMLYPLkySW/e9xxx9W1ra1btya2r1mzJrG2LO27Jl5Kz7vbxo0bx7p16/Z/7sNK3gKr\nHl0kAAVdJAAFXSQABV0kAAVdJAAFXSQABV0kAF1Hr8MHH3yQ2H7jjTcmti9evLiq7TXyWnW1qqmt\n3NDF8+bNS2yfOXNmYvvo0aMrqqMP0nV0kcgUdJEAFHSRABR0kQAUdJEAFHSRABR0kQAqGqlFejds\n2LDE9s7OzsT2WbNmJbY//vjjB8wrfJf8hAkTSi67dOnSxHWXM378+KqXKawt6Vn8cte5R44cWfW2\nJZl6dJEAFHSRABR0kQAUdJEAFHSRABR0kQAUdJEAKnoe3cx+CEwmd939NuDbwCRg32DdC9z9wIu+\nX+iTz6OLtJiSz6OXvWHGzM4BJrj7WWZ2DPAq8F/A9e7+i/RqFJGsVHJn3GrgN/nPHwBDgf6ZVSQi\nqavqVVJm9l1yu/B7gBHAIGALMNvdtyUsql13kezV/yopM5sOXAnMBu4H5rj714HXgJvrLFBEMlTR\nQy1mdh7wA2Cau+8Ani5ofhS4M4PaRCQlZXt0MzsSWAB8y9235+etMLMx+a9MBV7PrEIRqVslPfos\n4FhgqZntm3cv8LCZfQLsBC7PpjwRSYPe6y7Sd+i97iKRKegiASjoIgEo6CIBKOgiASjoIgEo6CIB\nKOgiASjoIgEo6CIBKOgiASjoIgEo6CIBKOgiATRq2OSSj8+JSPbUo4sEoKCLBKCgiwSgoIsEoKCL\nBKCgiwSgoIsE0Kjr6PuZWSfwVXKvgL7G3V9udA29MbOpwDLgjfys37n795tXEZjZBGAl0Onud5jZ\nieSGw+oPvAtc4u5dLVLbEqobSjvL2noO8/0yLfC7pTD8eM0aGnQz+xowNj8E8zjgp8BZjayhjGfd\n/aJmFwFgZkOBRRQPf3UrsNjdl5nZPwNX0IThsErUBi0wlHaJYb6fpsm/W7OHH2/0rvs3gEcA3H0d\ncJSZHdHgGg4WXcD5wOaCeVPJjXUH8BjwzQbXtE9vtbWK1cDM/Od9w3xPpfm/W291NWz48Ubvuo8A\nXimY3pqf92GD6yhlvJk9ChwN3OLuv2pWIe6+G9hdMAwWwNCCXc4twPENL4yStQHMNrN/oLKhtLOq\nbQ/wcX7ySuAJ4Lxm/24l6tpDg36zZp+Ma6V74N8CbgGmA5cB95jZoOaWlKiVfjtosaG0ewzzXaip\nv1uzhh9vdI++mVwPvs8J5E6ONJ27bwIezk/+3sz+CLQBf2heVQfYaWZD3P1TcrW1zK6zu7fMUNo9\nh/k2s5b43Zo5/Hije/RfAhcBmNlEYLO7f9TgGnplZheb2bX5zyOALwGbmlvVAVYB7fnP7cBTTayl\nSKsMpd3bMN+0wO/W7OHHGzWa6n5mNh+YAnQD33P33za0gBLM7HDg58AwYBC5Y/QnmljPJGAhMBrY\nRe5/OhcDS4DBwEbgcnff1SK1LQLmAPuH0nb3LU2o7bvkdoHfLJh9GXA3TfzdStR1L7ld+Mx/s4YH\nXUQar9kn40SkARR0kQAUdJEAFHSRABR0kQAUdJEAFHSRAP4fd8L/YmeydroAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7f72eec94750>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPoAAAEHCAYAAACHl1tOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAADs9JREFUeJzt3XusHOV5x/GvTTAXg22uBlxLFqr1\n0AipVYyqGNXEJBAo4lIJUyMM4tYmqnCUiiKEBaoAWRCFUre1EQiFco0FOKhgAnIT7Aba/AHIYAQU\nHiAKSMUEcwvGcOTiS//YtXvO8dk5e/bsDb/fz18z7zuz+5y1fp7Zd2b2nbBz504k7d0m9roASZ1n\n0KUCGHSpAAZdKoBBlwpg0KUCfK3XBag9ImInMDMz/2cM+7wNXJiZ/zWGfe4B3srMpRXbLACG9wcw\nJTM/a/a91D4GXW2XmT8DfrZrPSL+ElhoyHvHoO/lIuJA4G7gT4BJwCOZedWgTb4dEcuBw4F7M/O6\n+n7nUDsqTwbeAi7IzA+HvfbNwDuZeUfF++9ff50/b99fpbEy6Hu/vwEOBo4DpgFvRsSjg07X5wAn\nAIcBr0fEKuAz4H7gxMx8JSKWAHcACwa/cGYuaeL9Lwd+nZm/actfo5Y4GLeXy8xbgXMyc2dmfgK8\nChw7aJOfZub2zNwEPA3MBU4HfpWZr9S3uQM4OyL2Gct7R8RE4O+Afxjv36Hx8Yi+l4uI2cA/RsRx\nwHZgJrVT+V0+GLT8KXAIMAE4KSJeH9Z32Bjffi6wJTNfHXPhaiuDvve7DVgP/EVmbo+IXw/rP3TQ\n8iHAx8BW4KnMXDBsWyJiLO99JvDk2MpVJ3jqvvc7EnixHvJTgdnAQYP6z4+IiRFxJDAP+E/g34F5\nEXEsQET8aUT8cwvv/cfAa+MrX+3gEX3v8quI2DZo/a+ojXgvi4i/Bx4FbgBujIgX69s8DzxH7T+E\nZZn53wAR8dfAv0XEJGqDc387/M2aGHX/A+B34/+zNF4TfB5d2vt56i4VwKBLBTDoUgEMulSAbo26\nO+Indd6ERh0tBz0ilgHfpBbiH2bm862+lqTOaunUPSK+BczOzLnUHlr4l7ZWJamtWv2O/h1qN1+Q\nma8Bh0TElLZVJamtWg36UQx9GOKDepukPtSuUfeGgwCSeq/VoG9k6BH8GOC98ZcjqRNaDfovqP/a\nSER8A9jo74FJ/avlh1oi4kfAScAO4IrMfKlic6+jS53X8Ct0t55eM+hS5zUMurfASgUw6FIBDLpU\nAIMuFcCgSwUw6FIBDLpUAIMuFcCgSwUw6FIBDLpUAIMuFcCgSwUw6FIBDLpUAIMuFcCgSwUw6FIB\nDLpUAIMuFcCgSwUw6FIBDLpUAIMuFcCgSwUw6FIBDLpUAIMuFcCgSwUw6FIBvtbKThExH1gFvFpv\nejkzf9CuoiS1V0tBr3s6Mxe0rRJJHeOpu1SA8RzRvx4Rq4FDgRsy85dtqklSm03YuXPnmHeKiBnA\nnwEPA8cC/wH8YWb+b4Ndxv4mksZqQsOOVoI+XEQ8ByzMzN822MSgS53XMOgtfUePiEURcVV9+Shg\nOvBua7VJ6rRWT90PBlYC04BJ1L6jP1mxi0d0qfM6e+reBIMudV57T90lfbUYdKkABl0qgEGXCmDQ\npQKM5xZYfYWNdrVly5Ytlf1r1qwZsn7eeeexatWq3ev33Xdfw31feumlytd++eWXK/unTp1a2a89\neUSXCmDQpQIYdKkABl0qgEGXCmDQpQIYdKkAPr32FbZ58+aGfevWravc96677qrsf+KJJ8ZUy44d\nO5g4sbnjxuTJkyv733///cr+Aw88sOm6CuPTa1LJDLpUAIMuFcCgSwUw6FIBDLpUAIMuFcDn0Xto\n48aNlf033XTTkPUVK1awePHi3etV18K3bt1a+dqzZ8+u7L/++usr+7dt27ZH23XXXbd7eenSpQ33\nXbhwYeVre528/TyiSwUw6FIBDLpUAIMuFcCgSwUw6FIBDLpUAJ9HH4fXX3+9sv/ss8+u7H/33eop\n5QcGBoasD3/me8mSJQ33veSSSypfe9asWZX9++67b2X/8NpnzJgxpG3mzJkN912+fHnla19xxRWV\n/Wqo4fPoTd0wExHHA48ByzJzRUTMBO4H9gHeAy7KzOo7NCT1zKin7hExGVgOrB3UfCNwW2bOA94C\nLutMeZLaoZnv6FuBM4DB92vOB1bXlx8HTmlvWZLaadRT98zcBmyLiMHNkwedqm8Cju5AbX3vuOOO\nq+x/44032v6eO3bsaPtrtmLGjBmVbf1Sp2ra8VBLwwGAvZ2Dcf/Pwbj+1urltS0RcUB9eQZDT+sl\n9ZlWg/4UcG59+VxgTcW2knps1FP3iJgD3ArMAr6MiAXAIuCeiPg+8A5wbyeL7FeffPJJZf+8efMq\n+w866KDK/gsvvHCPtmeffXb38gknnNBw3wkT+vcb1Wi/6672a2Ywbj21UfbhTm17NZI6wltgpQIY\ndKkABl0qgEGXCmDQpQL4mKpaMvzS3wMPPDCkbeXKlQ33/eKLLypfe//99x9fceVy2mSpZAZdKoBB\nlwpg0KUCGHSpAAZdKoBBlwrgtMlqydtvv91Um/qDR3SpAAZdKoBBlwpg0KUCGHSpAAZdKoBBlwrg\ndXR1xMknn9ywb9KkSV2sROARXSqCQZcKYNClAhh0qQAGXSqAQZcKYNClAngdXSPavHlzZf/69esr\n2y655JKG+06c6PGl25oKekQcDzwGLMvMFRFxDzAH+Ki+yS2Z+URnSpQ0XqMGPSImA8uBtcO6lmTm\nzztSlaS2auYcaitwBrCxw7VI6pBRj+iZuQ3YFhHDuxZHxJXAJmBxZn7YgfrUI1OmTKnsHxgYaKpN\n/aHVwbj7gY8yc0NEXANcDyxuW1XqudEG46ZPnz5kfWBggAMOOGD3etVg3O233z6u2jR2LQU9Mwd/\nX18N+C8n9bGWrnNExCMRcWx9dT7wStsqktR2zYy6zwFuBWYBX0bEAmqj8A9FxBfAFuDSThap7lu3\nbl1l/9atWyvbrrzyyrbXpNY1Mxi3ntpRe7hH2l6NpI7wFiWpAAZdKoBBlwpg0KUCGHSpAD6mqhGt\nXTv8GaahRnrUdHDbkUce2faa1DqP6FIBDLpUAIMuFcCgSwUw6FIBDLpUAIMuFcDr6BrRxo3VPxF4\n4oknVrZNnTq17TWpdR7RpQIYdKkABl0qgEGXCmDQpQIYdKkABl0qgEGXCmDQpQIYdKkABl0qgEGX\nCmDQpQIYdKkABl0qQFPPo0fEj4F59e1vBp4H7gf2Ad4DLsrMPefRVd8aadrjwdasWVPZf9ZZZ+3R\ndswxx4yrJnXOqEf0iDgZOD4z5wKnA/8E3AjclpnzgLeAyzpapaRxaebU/RngvPry74HJ1OZLX11v\nexw4pe2VSWqbUU/dM3M78Hl99XLgSeC0Qafqm4CjO1OepHZo+jfjIuIcakH/LvDmoK4J7S5Knbff\nfvtV9n/++eeV/SN58MEHWy1HHdbsYNxpwLXA6Zn5aURsiYgDMnMAmAFU/5Kg+s5og3GHHnpoZf/w\nwbgHH3yQ888/f8i6+kczg3FTgVuAMzPz43rzU8C59eVzgeohWkk91cwRfSFwOPBwROxquxj4SUR8\nH3gHuLcz5alTnn322cr+gYGByv6rr766qTb1h2YG4+4E7hyh69T2lyOpE7wzTiqAQZcKYNClAhh0\nqQAGXSqAQZcK4LTJhbr33vHd+jB9+vSm2tQfPKJLBTDoUgEMulQAgy4VwKBLBTDoUgEMulQAr6Nr\nRNOmTavsnzJlSlNt6g8e0aUCGHSpAAZdKoBBlwpg0KUCGHSpAAZdKoDX0Qv1wgsvVPYfccQRlf0H\nH3xwU23qDx7RpQIYdKkABl0qgEGXCmDQpQIYdKkABl0qQFPX0SPix8C8+vY3A2cDc4CP6pvckplP\ndKRCtWTlypWV/Rs2bKjsv/baa9tZjnps1KBHxMnA8Zk5NyIOA14E1gFLMvPnnS5Q0vg1c0R/Bniu\nvvx7YDKwT8cqktR2E3bu3Nn0xhHxPWqn8NuBo4BJwCZgcWZ+WLFr828iqVUTGnU0fa97RJwDXA58\nFzgB+CgzN0TENcD1wOJxFqk2Gu07+qJFiyr7R/uOvnTp0jHXpN5pdjDuNOBa4PTM/BRYO6h7NXB7\nB2qT1CajXl6LiKnALcCZmflxve2RiDi2vsl84JWOVShp3Jo5oi8EDgcejohdbXcDD0XEF8AW4NLO\nlKdWvf/+++Pa/4ILLmhTJeoHowY9M+8E7hyha3wTbEvqGu+Mkwpg0KUCGHSpAAZdKoBBlwpg0KUC\njOle93HwXnep8xre6+4RXSqAQZcKYNClAhh0qQAGXSqAQZcKYNClAnRr2uSG1/ckdZ5HdKkABl0q\ngEGXCmDQpQIYdKkABl0qgEGXCtCt6+i7RcQy4JvUnlH/YWY+3+0aRhIR84FVwKv1ppcz8we9qwgi\n4njgMWBZZq6IiJnA/dQmuXwPuCgzt/ZJbffQJ1NpjzDN9/P0wefWy+nHuxr0iPgWMLs+BfMfAf8K\nzO1mDaN4OjMX9LoIgIiYDCxn6PRXNwK3ZeaqiLgJuIweTIfVoDbog6m0G0zzvZYef269nn6826fu\n3wEeBcjM14BDImJKl2v4qtgKnAFsHNQ2n9pcdwCPA6d0uaZdRqqtXzwDnFdf3jXN93x6/7mNVFfX\nph/v9qn7UcD6Qesf1Ns2d7mORr4eEauBQ4EbMvOXvSokM7cB2wZNgwUwedAp5ybg6K4XRsPaABZH\nxJU0N5V2p2rbDnxeX70ceBI4rdefW4O6ttOlz6zXg3H9dA/8m8ANwDnAxcBdETGptyVV6qfPDmrf\nga/JzG8DG6hNpd0zg6b5Hj6dd08/t2F1de0z6/YRfSO1I/gux1AbHOm5zHwXeKi++puI+B0wA/ht\n76raw5aIOCAzB6jV1jenzpnZN1NpD5/mOyL64nPr5fTj3T6i/wJYABAR3wA2ZuZnXa5hRBGxKCKu\nqi8fBUwH3u1tVXt4Cji3vnwusKaHtQzRL1NpjzTNN33wufV6+vFu/dzzbhHxI+AkYAdwRWa+1NUC\nGoiIg4GVwDRgErXv6E/2sJ45wK3ALOBLav/pLALuAfYH3gEuzcwv+6S25cA1wO6ptDNzUw9q+x61\nU+A3BjVfDPyEHn5uDeq6m9opfMc/s64HXVL39XowTlIXGHSpAAZdKoBBlwpg0KUCGHSpAAZdKsD/\nAYIhAHdutaTEAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7f72f0d9c8d0>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "qyWpO4hPh9l1",
        "colab_type": "text"
      },
      "source": [
        "Next we will provide some boilerplate code and train a linear classifier as an example:"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {
        "id": "kFJ_OMDNh9l2",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "output_extras": [
            {
              "item_id": 23
            }
          ],
          "base_uri": "https://localhost:8080/",
          "height": 475
        },
        "outputId": "f4147165-7452-49af-9fec-f3353e3a9576",
        "executionInfo": {
          "status": "ok",
          "timestamp": 1509629273026,
          "user_tz": -480,
          "elapsed": 8789,
          "user": {
            "displayName": "Mike C",
            "photoUrl": "//lh3.googleusercontent.com/-xWyBhueI0Vo/AAAAAAAAAAI/AAAAAAAAIg8/rU-qliu5l-M/s50-c-k-no/photo.jpg",
            "userId": "110113503404408134511"
          }
        }
      },
      "source": [
        "def run(model, batchsize=16, num_epochs=2):\n",
        "    \n",
        "    optimizer = chainer.optimizers.Adam() # we will use chainer's Adam implementation instead of writing our own gradient based optimization\n",
        "    optimizer.setup(model)\n",
        "    \n",
        "    stats = defaultdict(lambda: deque(maxlen=25))\n",
        "    for epoch in range(num_epochs):\n",
        "        train_iter = chainer.iterators.SerialIterator(train, batchsize, repeat=False, shuffle=True)\n",
        "        test_iter = chainer.iterators.SerialIterator(test, batchsize, repeat=False, shuffle=False)\n",
        "        \n",
        "        for itr, batch in enumerate(train_iter):\n",
        "            xs = np.concatenate([datum[0][None, :] for datum in batch])\n",
        "            ys = np.array([datum[1] for datum in batch])\n",
        "\n",
        "            logits = model(xs)\n",
        "\n",
        "            loss = F.softmax_cross_entropy(logits, ys)\n",
        "            model.cleargrads()\n",
        "            loss.backward()\n",
        "            optimizer.update()\n",
        "            \n",
        "            # calculate stats\n",
        "            stats[\"loss\"].append(float(loss.data))\n",
        "            stats[\"accuracy\"].append(float((logits.data.argmax(1) == ys).sum() / batchsize))\n",
        "            if itr % 300 == 0:\n",
        "                print(\"; \".join(\"%s: %s\" % (k, np.mean(vs)) for k, vs in stats.items()))\n",
        "\n",
        "# try a simple linear model\n",
        "run(L.Linear(None, 10))"
      ],
      "cell_type": "code",
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "loss: 2.32925891876; accuracy: 0.0\n",
            "loss: 0.637445527315; accuracy: 0.08\n",
            "loss: 0.480117873549; accuracy: 0.04\n",
            "loss: 0.421319420338; accuracy: 0.16\n",
            "loss: 0.374452891648; accuracy: 0.16\n",
            "loss: 0.339015575051; accuracy: 0.2\n",
            "loss: 0.332811718583; accuracy: 0.36\n",
            "loss: 0.389967683554; accuracy: 0.16\n",
            "loss: 0.355354978442; accuracy: 0.16\n",
            "loss: 0.362367997468; accuracy: 0.24\n",
            "loss: 0.306577712595; accuracy: 0.32\n",
            "loss: 0.353270591199; accuracy: 0.16\n",
            "loss: 0.386706850827; accuracy: 0.16\n",
            "loss: 0.328302717209; accuracy: 0.2\n",
            "loss: 0.330680179298; accuracy: 0.2\n",
            "loss: 0.297317226529; accuracy: 0.32\n",
            "loss: 0.306289359778; accuracy: 0.24\n",
            "loss: 0.342978409827; accuracy: 0.28\n",
            "loss: 0.29750328362; accuracy: 0.28\n",
            "loss: 0.348949042261; accuracy: 0.24\n",
            "loss: 0.311050440073; accuracy: 0.2\n",
            "loss: 0.267931166291; accuracy: 0.08\n",
            "loss: 0.25495399639; accuracy: 0.52\n",
            "loss: 0.305251099467; accuracy: 0.2\n",
            "loss: 0.222549476624; accuracy: 0.32\n",
            "loss: 0.268471718431; accuracy: 0.28\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "-s4rTjKNh9l3",
        "colab_type": "text"
      },
      "source": [
        "Next we will try to improve performance by training an MLP instead. A partial implementation is provided for you to fill in:"
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {
        "id": "Z3gdlPs0h9l4",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "output_extras": [
            {
              "item_id": 26
            }
          ],
          "base_uri": "https://localhost:8080/",
          "height": 475
        },
        "outputId": "167239eb-36c3-4df0-ec92-a5e434ed8397",
        "executionInfo": {
          "status": "ok",
          "timestamp": 1509629324478,
          "user_tz": -480,
          "elapsed": 51435,
          "user": {
            "displayName": "Mike C",
            "photoUrl": "//lh3.googleusercontent.com/-xWyBhueI0Vo/AAAAAAAAAAI/AAAAAAAAIg8/rU-qliu5l-M/s50-c-k-no/photo.jpg",
            "userId": "110113503404408134511"
          }
        }
      },
      "source": [
        "class MLP(chainer.Chain):\n",
        "\n",
        "    def __init__(self, n_units, n_out):\n",
        "        super(MLP, self).__init__()\n",
        "        with self.init_scope():\n",
        "            # the size of the inputs to each layer will be inferred\n",
        "            self.l1 = L.Linear(None, n_units)  # n_in -> n_units\n",
        "            self.l2 = L.Linear(None, n_units)  # n_units -> n_units\n",
        "            self.l3 = L.Linear(None, n_out)  # n_units -> n_out\n",
        "\n",
        "    def __call__(self, x):\n",
        "        # *** YOUR CODE HERE TO BUILD AN MLP W/ self.l1, self.l2, self.l3 ***\n",
        "        # \n",
        "        # Hint: you should make use of non-linearities / activation functions\n",
        "        #     https://docs.chainer.org/en/stable/reference/functions.html#activation-functions\n",
        "        x = F.relu(self.l1(x))\n",
        "        x = F.relu(self.l2(x))\n",
        "        x = self.l3(x)\n",
        "        return x\n",
        "#         raise NotImplementedError()\n",
        "run(MLP(200, 10))"
      ],
      "cell_type": "code",
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "loss: 2.19567322731; accuracy: 0.0\n",
            "loss: 0.289126877189; accuracy: 0.12\n",
            "loss: 0.230725547671; accuracy: 0.24\n",
            "loss: 0.245554587245; accuracy: 0.28\n",
            "loss: 0.187953667641; accuracy: 0.4\n",
            "loss: 0.20777797401; accuracy: 0.36\n",
            "loss: 0.151928565502; accuracy: 0.48\n",
            "loss: 0.14529140085; accuracy: 0.6\n",
            "loss: 0.116416935623; accuracy: 0.56\n",
            "loss: 0.112564135492; accuracy: 0.44\n",
            "loss: 0.151220483184; accuracy: 0.48\n",
            "loss: 0.2010664922; accuracy: 0.28\n",
            "loss: 0.0842502993345; accuracy: 0.6\n",
            "loss: 0.0990357077122; accuracy: 0.68\n",
            "loss: 0.0542347553372; accuracy: 0.72\n",
            "loss: 0.153207380772; accuracy: 0.48\n",
            "loss: 0.0910018110275; accuracy: 0.48\n",
            "loss: 0.0606182068586; accuracy: 0.76\n",
            "loss: 0.094375398159; accuracy: 0.72\n",
            "loss: 0.0803616392612; accuracy: 0.68\n",
            "loss: 0.137596744299; accuracy: 0.36\n",
            "loss: 0.0812636685371; accuracy: 0.64\n",
            "loss: 0.129164605439; accuracy: 0.52\n",
            "loss: 0.0433241149783; accuracy: 0.8\n",
            "loss: 0.0538192850351; accuracy: 0.68\n",
            "loss: 0.0591419115663; accuracy: 0.76\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "BhFCDhs3h9l6",
        "colab_type": "text"
      },
      "source": [
        "Next you should try to implement logging test loss and see if the model is overfitting."
      ],
      "cell_type": "markdown"
    },
    {
      "metadata": {
        "id": "0QbIzZNSh9l6",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          },
          "output_extras": [
            {
              "item_id": 32
            }
          ],
          "base_uri": "https://localhost:8080/",
          "height": 580
        },
        "outputId": "75c0e248-1267-47a3-acab-5d5fdc0ab829"
      },
      "source": [
        "def better_run(model, batchsize=16, num_epochs=2):\n",
        "\n",
        "    optimizer = chainer.optimizers.Adam() # we will use chainer's Adam implementation instead of writing our own gradient based optimization\n",
        "    optimizer.setup(model)\n",
        "    \n",
        "    stats = defaultdict(lambda: deque(maxlen=25))\n",
        "    for epoch in range(num_epochs):\n",
        "        train_iter = chainer.iterators.SerialIterator(train, batchsize, repeat=False, shuffle=True)\n",
        "        for itr, batch in enumerate(train_iter):\n",
        "            xs = np.concatenate([datum[0][None, :] for datum in batch])\n",
        "            ys = np.array([datum[1] for datum in batch])\n",
        "\n",
        "            logits = model(xs)\n",
        "\n",
        "            loss = F.softmax_cross_entropy(logits, ys)\n",
        "            model.cleargrads()\n",
        "            loss.backward()\n",
        "            optimizer.update()\n",
        "            \n",
        "            # calculate stats\n",
        "            stats[\"loss\"].append(float(loss.data))\n",
        "            stats[\"accuracy\"].append(float((logits.data.argmax(1) == ys).sum() / batchsize))\n",
        "            if itr % 30 == 0:\n",
        "                test_iter = chainer.iterators.SerialIterator(test, batchsize, repeat=False, shuffle=False)\n",
        "                # *** YOUR CODE implement logging of stats on test set ***\n",
        "                for batch_test in test_iter:\n",
        "                    xs = np.concatenate([datum[0][None, :] for datum in batch_test])\n",
        "                    ys = np.array([datum[1] for datum in batch_test])\n",
        "\n",
        "                    logits = model(xs)\n",
        "\n",
        "                    loss = F.softmax_cross_entropy(logits, ys)\n",
        "                    # no need to learn, when we are testing\n",
        "                    # calculate stats\n",
        "                    stats[\"val_loss\"].append(float(loss.data))\n",
        "                    stats[\"val_accuracy\"].append(float((logits.data.argmax(1) == ys).sum() / batchsize))\n",
        "                    \n",
        "                print(\"; \".join(\"%s: %s\" % (k, np.mean(vs)) for k, vs in stats.items()))\n",
        "\n",
        "better_run(MLP(200, 10))"
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "loss: 2.38229012489; val_accuracy: 0.0; val_loss: 2.29367769241; accuracy: 0.0\n",
            "loss: 1.55391245842; val_accuracy: 0.0; val_loss: 1.13345803261; accuracy: 0.0\n",
            "loss: 0.646526403427; val_accuracy: 0.0; val_loss: 0.822305475473; accuracy: 0.08\n",
            "loss: 0.527407506108; val_accuracy: 0.0; val_loss: 0.691670941114; accuracy: 0.0\n",
            "loss: 0.466817269325; val_accuracy: 0.08; val_loss: 0.612925790548; accuracy: 0.04\n",
            "loss: 0.393106015325; val_accuracy: 0.04; val_loss: 0.580229463577; accuracy: 0.04\n",
            "loss: 0.353550501466; val_accuracy: 0.0; val_loss: 0.639054588675; accuracy: 0.16\n",
            "loss: 0.325949620903; val_accuracy: 0.0; val_loss: 0.54268496573; accuracy: 0.16\n",
            "loss: 0.493765832782; val_accuracy: 0.04; val_loss: 0.598064358234; accuracy: 0.08\n",
            "loss: 0.441802020669; val_accuracy: 0.08; val_loss: 0.474574801922; accuracy: 0.12\n",
            "loss: 0.315273371339; val_accuracy: 0.16; val_loss: 0.465327720046; accuracy: 0.24\n",
            "loss: 0.358024782836; val_accuracy: 0.08; val_loss: 0.472458402514; accuracy: 0.28\n",
            "loss: 0.322830531597; val_accuracy: 0.04; val_loss: 0.486205202937; accuracy: 0.24\n",
            "loss: 0.336920343041; val_accuracy: 0.16; val_loss: 0.497796416879; accuracy: 0.16\n",
            "loss: 0.202015681267; val_accuracy: 0.04; val_loss: 0.466192596555; accuracy: 0.48\n",
            "loss: 0.307852499485; val_accuracy: 0.08; val_loss: 0.449665151238; accuracy: 0.12\n",
            "loss: 0.269956387281; val_accuracy: 0.12; val_loss: 0.506647168398; accuracy: 0.24\n",
            "loss: 0.307980099916; val_accuracy: 0.2; val_loss: 0.430230525136; accuracy: 0.2\n",
            "loss: 0.229430580735; val_accuracy: 0.04; val_loss: 0.412149706483; accuracy: 0.28\n",
            "loss: 0.270627126098; val_accuracy: 0.08; val_loss: 0.40500033319; accuracy: 0.32\n",
            "loss: 0.186080741286; val_accuracy: 0.12; val_loss: 0.386364367008; accuracy: 0.44\n",
            "loss: 0.22999602288; val_accuracy: 0.16; val_loss: 0.432844335437; accuracy: 0.28\n",
            "loss: 0.250118284822; val_accuracy: 0.12; val_loss: 0.374633293748; accuracy: 0.28\n",
            "loss: 0.174835678637; val_accuracy: 0.08; val_loss: 0.440627692938; accuracy: 0.44\n",
            "loss: 0.248081167936; val_accuracy: 0.16; val_loss: 0.361101129651; accuracy: 0.24\n",
            "loss: 0.247025899291; val_accuracy: 0.16; val_loss: 0.347476246953; accuracy: 0.2\n",
            "loss: 0.189303216934; val_accuracy: 0.2; val_loss: 0.358281924129; accuracy: 0.36\n",
            "loss: 0.290047548115; val_accuracy: 0.32; val_loss: 0.296972821355; accuracy: 0.32\n",
            "loss: 0.209366900921; val_accuracy: 0.12; val_loss: 0.341278240383; accuracy: 0.2\n",
            "loss: 0.257778088152; val_accuracy: 0.2; val_loss: 0.327720628977; accuracy: 0.28\n",
            "loss: 0.225090374351; val_accuracy: 0.12; val_loss: 0.43271810472; accuracy: 0.28\n",
            "loss: 0.153858321309; val_accuracy: 0.2; val_loss: 0.37645216465; accuracy: 0.6\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "nUEQ9fr97jc2",
        "colab_type": "code",
        "colab": {
          "autoexec": {
            "startup": false,
            "wait_interval": 0
          }
        }
      },
      "source": [
        ""
      ],
      "cell_type": "code",
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "IbawtwqUh9l8",
        "colab_type": "text"
      },
      "source": [
        "Try different variants!\n",
        "\n",
        "- Does using a ConvNet improve performance (reduce overfitting?)\n",
        "- Try changing the learning rate and observe the effect\n",
        "- Does the model train if you give it correlated gradients? (consecutively sample many batches of \"1\", then many batches of \"2\", ... etc"
      ],
      "cell_type": "markdown"
    }
  ]
}