dqn_navigation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Deep Q Learning Network\n",
    "\n",
    "This project will create a **Deep Q-Learning Network** to run through an *Unity Environment* (the one seen on the gif below) and will be trained to maximize future reward. Our reward structure is simple: **+1** if the agent collects *yellow* bananas and **-1** if it collects *blue* bananas. Thus, the agent will be trained to pick as many *yellow* bananas as it can, while avoiding the *blue* ones. \n",
    "\n",
    "![alt-text](images/banana.gif)\n",
    "\n",
    "First we'll instantiate the framework and `Unity` agent **Banana.app**. It's also declared the function `unwrap_env_info` which receives an `env_info` object and returns the encapsulated `state`, `reward` and `done` information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-10-19T21:13:49.474232Z",
     "start_time": "2019-10-19T21:13:49.109418Z"
    }
   },
   "outputs": [],
   "source": [
    "from unityagents import UnityEnvironment\n",
    "import numpy as np\n",
    "from collections import deque\n",
    "from src.agent import Agent\n",
    "import torch\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def unwrap_env_info(obj):\n",
    "    return obj.vector_observations[0], obj.rewards[0], obj.local_done[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-10-19T21:15:10.171104Z",
     "start_time": "2019-10-19T21:15:07.016789Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:unityagents:\n",
      "'Academy' started successfully!\n",
      "Unity Academy name: Academy\n",
      "        Number of Brains: 1\n",
      "        Number of External Brains : 1\n",
      "        Lesson number : 0\n",
      "        Reset Parameters :\n",
      "\t\t\n",
      "Unity brain name: BananaBrain\n",
      "        Number of Visual Observations (per agent): 0\n",
      "        Vector Observation space type: continuous\n",
      "        Vector Observation space size (per agent): 37\n",
      "        Number of stacked Vector Observation: 1\n",
      "        Vector Action space type: discrete\n",
      "        Vector Action space size (per agent): 4\n",
      "        Vector Action descriptions: , , , \n"
     ]
    }
   ],
   "source": [
    "env =  (file_name=\"Banana.app\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Examine the State and Action Spaces\n",
    "\n",
    "To create the `Agent` object (more information about the `Agent` class please look on our [Github Wiki]()) it's necessary to inform the number of possible *states* and the number of possible *actions*, and then the *agent* will create the *local* and *target* identical networks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-10-19T21:15:25.187787Z",
     "start_time": "2019-10-19T21:15:25.159880Z"
    }
   },
   "outputs": [],
   "source": [
    "brain_name = env.brain_names[0]\n",
    "brain = env.brains[brain_name]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of agents: 1\n",
      "Number of actions 4\n",
      "States look like: [1.         0.         0.         0.         0.84408134 0.\n",
      " 0.         1.         0.         0.0748472  0.         1.\n",
      " 0.         0.         0.25755    1.         0.         0.\n",
      " 0.         0.74177343 0.         1.         0.         0.\n",
      " 0.25854847 0.         0.         1.         0.         0.09355672\n",
      " 0.         1.         0.         0.         0.31969345 0.\n",
      " 0.        ]\n",
      "States have length 37\n"
     ]
    }
   ],
   "source": [
    "env_info = env.reset(train_mode=True)[brain_name]\n",
    "\n",
    "print(f'Number of agents: {len(env_info.agents)}')\n",
    "\n",
    "action_size = brain.vector_action_space_size\n",
    "print(f'Number of actions {action_size}')\n",
    "\n",
    "state = env_info.vector_observations[0]\n",
    "print('States look like:', state)\n",
    "\n",
    "state_size = len(state)\n",
    "print(f'States have length {state_size}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "agent = Agent(state_size=state_size, action_size=action_size, seed=25)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training the model\n",
    "\n",
    "The model is trained based on the [Multi-Layer Perceptron](https://cs.stanford.edu/~quocle/tutorial1.pdf) that is defined on the `Agent` object. The training uses `Experience Replay`, that randomizes over the training data (the training episodes), thereby removing correlations in the observation sequence and smoothing over changes in the distribution. It is also used an iterative update that adjusts the action-values (**Q**) towards target values that are only periodically updated, thereby reducing correlations with the target. Both of those techniques are explained on the [Original DQN Paper](https://storage.googleapis.com/deepmind-media/dqn/DQNNaturePaper.pdf). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def dqn(n_episodes=2000, max_t=1000, eps_start=1.0, eps_end=0.01, eps_decay=0.995):\n",
    "    \"\"\" Deep Q-Learning\n",
    "    Params\n",
    "    =====\n",
    "        n_episodes (int): maximum number of training episodes\n",
    "        max_t (int): maximum number of timesteps per episode\n",
    "        eps_start (float): first value of epsilon (for epsilon-greedy selection)\n",
    "        eps_end (float): minimum value of epsilon\n",
    "        eps_decay (float): the decay rate of epsilon per episode\n",
    "    \"\"\"\n",
    "    \n",
    "    scores = []\n",
    "    scores_window = deque(maxlen=100)\n",
    "    eps = eps_start\n",
    "    \n",
    "    for i_episode in range(1, n_episodes+1):\n",
    "        env_info = env.resetin_mode=True)[brain_name]\n",
    "        state = env_info.vector_observations[0]\n",
    "        score = 0\n",
    "        \n",
    "        for t in range(max_t):\n",
    "            action = agent.act(state, eps)\n",
    "            env_info = env.step(action)[brain_name]\n",
    "            next_state, reward, done = unwrap_env_info(env_info)\n",
    "            agent.step(state, action, reward, next_state, done)\n",
    "            state = next_state\n",
    "            score += reward\n",
    "            \n",
    "            if done:\n",
    "                break\n",
    "        scores_window.append(score)\n",
    "        scores.append(score)\n",
    "        eps = max(eps_decay*eps, eps_end)\n",
    "        print(f'\\rEpisode {i_episode}\\tAverage Score: {np.mean(scores_window):.2f}', end='')\n",
    "        if i_episode % 100 == 0:\n",
    "            print(f'\\rEpisode {i_episode}\\tAverage Score: {np.mean(scores_window):.2f}')\n",
    "        \n",
    "        if np.mean(scores_window) >= 25:\n",
    "            print(f'\\nEnvironment Solved in {i_episode-100} episodes!\\tAverag Score: {np.mean(scores_window):.2f}')\n",
    "            torch.save(agent.qnetwork_local.state_dict(), 'checkpoint.pth')\n",
    "            break\n",
    "            \n",
    "    return scores"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When plotting our score over the episodes trained, we can see that the training is resulting on a better mean score over time. On **DQN**'s it is normal to training data's score looks like noise, and we understand that the training is improving over time when we look at a moving mean of this plot. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Episode 100\tAverage Score: 0.99\n",
      "Episode 200\tAverage Score: 3.94\n",
      "Episode 300\tAverage Score: 6.97\n",
      "Episode 400\tAverage Score: 10.30\n",
      "Episode 500\tAverage Score: 13.03\n",
      "Episode 600\tAverage Score: 14.47\n",
      "Episode 700\tAverage Score: 14.70\n",
      "Episode 717\tAverage Score: 15.01\n",
      "Environment Solved in 617 episodes!\tAverage Score: 15.01\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO2dd5gcxbX23zOzUTkHFBCSABEF0iIMCCyCiRdwJBgTbGxsYy7m2h++wglw5DqAsY0N2OBsMGCMwWREzkgESQgkhJCQQFko7Era3Zmp74/u6qmuqeowMz0zu3N+z7PPzlSnM+mt06dOnSIhBBiGYZj6IVVtAxiGYZjKwsLPMAxTZ7DwMwzD1Bks/AzDMHUGCz/DMEyd0VBtA6IwbNgwMWHChGqbwTAM06OYN2/eBiHEcL29Rwj/hAkTMHfu3GqbwTAM06MgohWmdg71MAzD1Bks/AzDMHUGCz/DMEydwcLPMAxTZ7DwMwzD1BmJCT8RjSOix4hoERG9TkRfdduvIKL3iOhV9+/EpGxgGIZhCkkynTMD4OtCiJeJqD+AeUT0sLvtGiHEzxK8NsMwDGMhMY9fCLFaCPGy+3gbgDcAjEnqegzDJEdnJovb564El3HvHVQkxk9EEwAcCOAFt+kiIppPRDcT0WDLMRcQ0Vwimrt+/fpKmMkwjIWrH16CS++Yj4cWra22KUwZSFz4iagfgH8CuEQIsRXAbwFMAnAAgNUAfm46TghxoxCiTQjRNnx4wYxjhmEqyPptnQCAbTszVbaEKQeJCj8RNcIR/b8JIe4EACHEWiFEVgiRA/A7ADOStIFhGIbxk2RWDwG4CcAbQoirlfbRym4fA7AwKRsYhmGYQpLM6jkMwNkAFhDRq27bNwGcSUQHABAAlgP4YoI2MAzDMBqJCb8Q4mkAZNh0X1LXZBiGYcLhmbsMwzBFsnVnN7K5npfiysLPMAxTBF2ZHPa/4iF89989b5iShZ9hGKYIurI5AMBdr7xXZUviw8LPMAxTZ7DwMwzDlEDPi/Cz8DMMw9QdLPwMwzBFkHML1vXEunUs/AzDMEUgctW2oHhY+BmGYSJw7SNv4Tt3LcRtL63EF/48F8KN7pM2TbUrk8PJv3oaz769oQpWRiPJkg0MwzC9hmseWeJ7Ludt6aGe9zbvwIL3tuCbdy7A45ceWSHr4sEeP8MwTBHYFqXpCYvVsPAzDMMUQVilBtJjQDUECz/DMEwRSM9eaJn8te/vs/AzDMMURZjA166/z8LPMAxTFDlrjL/ChhQBCz/DMAUIIfD3F97Ftp3diV+rvTODW158tyYGRR9etBbL1rdH2rcHVmP2YOFnGKaAuSs+wDf/tQDfuSv5ksPfvWshLrtzAV58Z1Pi1wrjC3+ei6N+/kSkfWuhoyoWFn6GYQrY0ZUFAGxo70r8WuvbO51rdmcTv1Y5seu+u6GGg/ws/AzDFCAzEfWMlWSvWcNKacBWq6cnhIBY+BmGKYCq4K72tNCJzVzZIdRyN8bCzzCMlUpocU/z9CVZy5vTE9bgZeFnGKYAL9RTAQ3raZ6+JD+BS293/tdyh8bCzzBMAVKyKhnjV1m3dSdyCXrOa7fuDNy+dWc3Ojozgfu4S+6iK5NDu7JvmMe/ZUe3N3geRHtnBvNWfIAtO8qfUsvCzzBMIRX0+HXPeM2WnZjxozn4xZy3ErnenS+vwsE/moN5Kz6w7rP/FQ/h4B/NCTxPJpcvyL/v5Q96j20TuyRTr3wIR//88VA7l6zdhk/89lm88q7dzmJh4WcYpqaQHu79C1Yncn45X2DJ2m2B+7WHePw5y0IsUQZ3398SfMcB5ENJqQRCRiz8DMMUILN6qhHoaW1MA0BoqKVUSr2bsQ3ulitCJc/Dws8wTEWgfJC/aoR53MVSLh3NWlz+co1NyLGCVAJjxCz8DMMUUM18FBkq6YgwAFoKpQ5cZy2hHtudQFy8kFFP8viJaBwRPUZEi4jodSL6qts+hIgeJqK33P+Dk7KBYZjiEN7/yrn88kpS8JLLhy+PkNrsy6dzlnZ+eZ50Ai5/kh5/BsDXhRB7A/gQgK8Q0d4AZgOYI4TYHcAc9znDMDWEFB3deU0i516XtR4w/wmAXfjL1WHlRA8M9QghVgshXnYfbwPwBoAxAE4F8Cd3tz8B+GhSNjBMPfHj+9/Axbe8AgD43j2L8P9uf63oc1knJwUc8/yyjTjkx3NCB2VveOJtnH3TC6HXLoZrH3kLX/zL3Ej7CgF8q4QKpJ/RXsNRP38c2ZxQsnr8in3SL5/CbXNX+to+/6e5mDD7Xjyw0J/B9OnfPY+bn37HOU9PCvWoENEEAAcCeAHASCGEfJVrAIy0HHMBEc0lornr16+vhJkM06O54YlluPu19wEANz/zDu6Yt6roc+U8j193+e3H/N8Db2L1lp14c01wmuSP738TT721wbq9lBj5NY8swYOvrw3cR9XRv73wLv7y/Iqir6eybH0HdnRnrXn8r7+/Fd+4Y76v7ZE3HFv19mff3ojHFju616M8fgkR9QPwTwCXCCG2qtuE860yvktCiBuFEG1CiLbhw4cnbSbDMAo5i8cfNjmpLNe2DJomTTnCWIS8/WoHE3buPk0N1m09Lp2TiBrhiP7fhBB3us1riWi0u300gHVJ2sAwTHxsJYcTlX3hv3bS2GrslHpO0x1LWNi/T3Pauq1HCT85gambALwhhLha2XQ3gHPdx+cC+HdSNjAMUxze4K6lvRLXTgqbjJbjstmcMHr3mZDbmD5NduFPotab/f6idA4DcDaABUT0qtv2TQBXAbiNiM4HsALAaQnawDBMEdgyUxL1xqkC1wigHKEeIYQxvz8s06dPo12Kk0jnTEz4hRBPw965Hp3UdRmGKZ2cJZ+zp4R6sjkRLpgJvDY1q0clk5OTscx3NK0BHn+PCvUwDBMNIQT++Mw72Lw9+fVtw+jMZPGj+97Azc84qYQFGlWEKN/z2vu4/om3jduyOYEnlviz9uKkwa/6YLuXIvnuxu1ee1cm73YvXdeOe9xsJ8AeOinHjcbld7+OF97ZWNAuyzg0WDqjxnReivU7jySyepIM9TAME4FXV27GFfcswrNvb8SN57RV1ZYbn1iGG59c5j0vx3qy/+3OLThzxngMbG30bfunIeU0Tsjl9Buex3ubd+CUqbvgY795xmvvzGQ9L/qYq58AAJw8dZfAc5VjlvJ/5ufz8dX8+4xXd4dgurcY0jf/vujvcY/N42cYxk6n651u3l7+BTfisnWn3wZdDEuJg5uKl5kWGYkz83X9tk7v8caO/B2T6vHb0K+T5NCCvJYefmpMO8/VcI5uV4/M42cYJphaWqBP19xypnOaYt9dhpHQOHcVNi+9M0D45Yxa/drlFn71c836PP488rVmlBetv08c42eYXky1ljn02RAi/KWUoTEd220Q/jh3FfkZxv72IOH39unWhD/B999WYlm2q3dDhR4/Cz/DMAmie5uFefzFi6PpWFX4pfCquhdW216eU7fbFOrRz6V3DmX3+BW9zhhCPao9qsevTwBLJaDSLPwMw3gkUX3TO7ehrTtrmuWab+sOmfjkefxae2emsJa/mlIJGEI9gVcqDVOMXxV41cvXOyj2+BmmF5JE1kYQnZksVm/ZYdymO9jvbuzQtgts3t6FLdu78d7mHXh7fXtBZ7Fy03bjAK0pxr9COb+Mvecsgrh6yw5P0Ds6M1i3Lb9u7aZ2fyqs0ePXrt/Z7e8cPuhILp3WFOPP+jz+nPdeVCLUw+mcDFMjVGrC6iW3vor7F67Bsh+diFRKH2z0G9HRlcWzb+craQoBHPC9h337XH3aVHx82tj8+f/xKt5atw2XHjdFO7ffji07un2VNGWoRzVBeunZnMAhP34UJ+43Cr85azo+/ptnsVhZLP2Inz7mO7cpxu95/O5z3eM//CePwUapd0KyZIMv1KOc88HX1+LB19finotmYuTAZt+xnNXDML2QCjv8ePD1NQDMHrgppL5EKbNs2j5/1ZaCtqcNZZcLYuyaxy2jOj6PP5sXfgB4eJHTUaiib8Lk8We1sFKctNFiFldRP1f52oLSNgHnrkaPbnEeP8P0Yiqd02OqIhnm2ZoyX0yhCFM9f/3U+vXlM1UQZYw/bsaNKU1Uv55tzVwTpS6qJT1+daDWNHzRr6WhcHCXPX6G6X1UK4/fpPGmuwDV4zQdYxImL01RzdAJybzJicJj5HniRlrMg7vO9eTriVMTqJj6QeoKXFkvzGQe3JX0bWrgwV2GqSeSyKgJOqd5ALZwP1/KpeF8+jiBcx5nP7UccZjwC8NdQiZrTtcMwxjqyekev3qdYPe/1I/GE34qbFNJEXEeP8MwpREUojDH+Avbtu7Mr6FrEkCTLsn9fGmKoZOs7B5/3FCLcXBXj/ErL6Y9ZJ3gUiuGmrJ6bO8/5/EzDFMSQYOSphizSd+2KfV7TGczeaRSvPzX1zz+rB7qkf/9aY56GwC0NtrLGAPR0jnVkMq2neUXftMELlMZBxWh2QWwx88wFeG6x5bi5F89Hfu46594G8f/4knr9tteWokZP3ykIFxi+12v39aJyd+8Dy+/+4H1nFfe8zrOvukFX9uE2fd6j/e74kFfvPuSW18xCqzE1KaK4tUPLynY/tvH38ZNT7/ja1u6rh37fPcB37G6zn38N8/6nguDXcdc/SQ6OjMQmo73bQ7ORLelc/7m8aX447PLC66jF6eTXHH36zjn5hdLHtz1RD4k1PPzhxbjo9c942vjPH6GqQA/fXBxUcdddf+bgdu/8c/5AJwffEO68Mesy8CL72xCJidw01PvYNpZg43n/MMzywOv2ZnJYVNHF0YPbAUA3PVqvi69Oaun8BzbdnaHCu33/7MIB44f5Gvr6Mpi+Yb8BK0wr9mUxw8A72zowJhBjv1ycHSPkf2wob0TNkzCn80J/OSBxcrz/DZTlVAAXidRzPiLekjW4PGb3o+nDGmwSaT7ssfPMBbC6sTERf6AMwXnNf+y5docxeSQ+85jUY4oHv9eowc4XnuRJqiHhVRfsN6JZHOFce8wogzuqtfZElISu5jPwB+yKozxRz0nh3oYpoJ0dAXHfW3YOgz5Ay6sSGn2dNPuqF5c0Su4riURPEqMf2BrA9o7M0UPbvoHd0M8fuO4gCOa3rHWztOPKZ0zKKtns8XjlxTT95oGqX2TuiK+p5zHzzAVJGzAz4Zp8hCQ9+v17BLb779cHr+NKB7/wNZGbNuZCXX4mxvMUqKeL2rfUTDRKyeMbUGYPH69s/B5/CHCX1yoJ3+M7Lz9uf3Of9tyjBL2+BmmAvRxl+wrVvhtteDl71evOGnTMM/jL1H4bXcgUQqpOcLfHSrafSyLhavHhXm4+QlcusefM4Z/gjDH+HPac8XjDwn1FOfxK8LvTR4rvH6TpdOU2O7YSoGFn2E0+rkDmdssmR5h2Jb9k95eocfvhnq0/WVsvmThtxxu0mH9ZmVgayO27gwP9fRpMg/+qraHvYp8Vk/hOUxtQZhj/PZzbNkRXJmzmFCXaqL+mavnDBP+JGDhZ2qGBau24O7X3g/fMWH6tUjhj+bx7+zO4tpH3vKeyyJoKlu2d3shIF0E7B6/I/xPL92ABW4htDtfXoXX3y8sihaETbR+8cgS7OhyYuF3v/Y+/veO+Xhj9VbfPgNbG9GVyYWuaPXe5h145d3NBe36koJBn++371qIVR9sx60vvetrv/XFld5dS1cmh8cWrwuN8ZuE/xotFVV9X8I9/vjCv3RdO666/03klDGKN5WCd1+77VUAQGOahZ+pY07+9dO4+JZXqm2GNzloR3fhAKGJ3zz+Nq55JC8q375rYcE+370736aHerwCZJq4qCV8T/61M6/ga7e9hpN+GW+OgU207nzlPVw7x+mwfvPYUvxj7kq8t9lfp7/V9eS3FznQrZd7CPp8t3dl8ciitQXVPu9dsNr31nz2Dy8VhG0KzmX47J5bttH3XO07wjr5Yu+6rn/ibaz8YLtxwZkla9sBAI1JjN6GwMLPMBoyDhvVy9PLC5toV4Ql7uBuqQS9jPZOx9O1DUjL6QbFCp8q/FFOYRPguDH+KGE69RxhnbxJuKPSmckF1gJqYI+fYapPKmZsPe7gm57O6RUni3WW6AR1YPI1mmLQQP61FSv86nl3RuggN203x9rjC3/4HYp6ThnyspEJm4QQQGd3LjA01WiYzJc0LPwMoyHL9kYN68a9U9dFwFZrvlxZnEHnkcJs80hlJxgWU7eh3kmEpUwC9uUPCzN9gu1R77Bsn49P+EM6JVvHGIWubDbwjqFXxfiJ6GYiWkdEC5W2K4joPSJ61f07ManrM0yxpGKGN+LmWesiGzXrptiZxIEev7utO2TSWbHXVu9utu4I98I3WQZZ9cuH2aOGemwrWKkfQ5jHXzjpLjphoZ6k5mkEkWRX80cAxxvarxFCHOD+3Zfg9RmmKKRMRI3xx10ar8Djt1xHv74+KByVIJHMh3osMX5XIYr1+H3CHyHuvjliqCfMno6urLE+ju+cMWL8xb5+wBF+W8ca5dpJkJjwCyGeBLApqfMzTFJECfW8vb7dEzVTLZytO7u9DJktO7qxestOb5ttcFf+39jeiXXbdhZcXz1u8ZptkTNtcgJ4f/MOo/BmQmL88r14d9P2SNfSUUMcUUI9myyhnqByCzbad2bQnbXH17NxYvwlhHrWbNmJZ5cWFl+T7Owu/m6iWKpRnfMiIjoHwFwAXxdCGGvOEtEFAC4AgPHjx1fQPKbe8UI9FuVfuWk7jv75Ezh/5m74zn/tbYwhn/TLp7By0w4sv+okHHfNk1izNS/81nROl+k/eAQA8PfPH+xrV8XnuIDyzzo5IXDoVY96FS5V5ALktrsJGeopunxFJl6M31YsTRfeKB54e1fGS1c1oXr8tqym/PWKF+fL7lwQuH3S8L6BlUaToNKjCr8FMAnAAQBWA/i5bUchxI1CiDYhRNvw4cMrZR/DeDNsbaEe+SOdu9y5odWzevq3NGDlpnw+vCr6gGECl6spegegX73oUI/7OvQcfSDfuZk82tcuP7bklFL/7Nhw4beFPQrq7EQQ/lxOYN4Ke9AhTvE7m8f/qeljI5/DxH5jBmLaruaS20lSUeEXQqwVQmSFEDkAvwMwo5LXZ5goeHn8FnGRzVLw9cHdsLCAHk+37V0Q1y4y3BCkb04BNGH0oAe2NpZcIEztrLZGEH6bJ6+/Z5mcCE2DdNY9sEtcnNm4tsHdof2aI5/DxLB+TcY7xrDCbaVSUeEnotHK048BKJziyDBVxstksWbbCN9+uraEhQX0gT4pQLoOFcT4S/T4TWRyIjBeXqrwZ2PG+K3nMcT4WxqCl1/MCREooHHeTluHVA59Nr3H6YSFP7EYPxHdAmAWgGFEtArA5QBmEdEBcJyc5QC+mNT1mZ6LECJ2pkw5SYWUQ9ZX0dN/uGGzPAs8fsvuupdZrMcfFBXJ5czevqRU4VfPHcXjt6F3llkh0NyYwraA0HhOiMAc+ThplDaPvxwlk3uV8AshzjQ035TU9ZjegxDJLDcXlbAYv+7xx8/jt6dzBqU/JuPx5wJz1EuN8as2l+bxF5ZUbg71+GFc4tI7R4xQT1ip7VKohvDzzF2m5ih1xalSCavV43n87n5xf6SFWT151OwZfcJT0J1E0EIhYXn8QXcSpd55ZcoU6jG99pbGYPnK5gQaUgEx/hgev134y+HxF7YlHePnxdaZmqPYZf5s3Pz0O7jqgTcxdexAjBnUil+ccSAeWLgaX/rryzhqygg8+uY6AMCb3z8eLY1p78f8txfexY/uexOThvfFnK/P8s6ne/xxf/uZrEDbDx7GhnYnZ10uYiIE8LfnV3j76aGRc29+0XrO3S6zz4UM0reOzixm/OgR63bber1RufWllZHsCMPUOdnWAJCccO1TGNSn0bo9joPxHUPF1cY0lSfGbzhJOqDDKgfs8TM1Rwkp00a+959F6Mrk8NLyD3DXq049+F89uhQAPNEHgLVu2qX8Ha7Y6Exaent9h98+L6tH2htP0bqzOU/0AaccsWS1kvqph3rWBQW0AwjqSNe3dxZ40/1bGnD/Vw8HkH+NlebGs6f7npvCXD/91P6YOXlY4HmC6uzLgec44v3rTx+IBy45HN86cS/86bMzfGGasYNb8fmZu0U/mYupb026fA8LP1NzlNvjNxHkyIbpQE7z+FXd189r6hRsVSoFnEqOklJCIwC8CVtBHZNpxuphk4Zhr9EDAPjjz+WoIrm3e94g+jc34Nh9RvnadI+/X3MDpowagK8es3uk6/ZrbihYF1gOPOsx9j1H9ree54g9hmPKqAH4whETcejkYb5O45Spu2B4/2jpncfsNcJ7bIrxB4WoygELP1NzVCLGTwZ5l5cNG6yVwk+e8Oft1cXF9FraO+3lAbqyOa/ziFLULApBWTsdhrIPquao70U5Bhyj6JlpQFbPwOnvrpIW1aJsrjDDJ2sR/qCPv1F7AWqMP05UTBV209vKg7tM3SEqULok6HcV9gOWEmQq5tbS6M80MaUM2hYJEUKgszuLoX0drzFKUbMgpLYEZQOZ+liyiH2p8X7A3OHqmCZd6QPinvBHtCmTyxV0KBnDAuhAcMevn0PdN0XkdbJhd0fqecwePws/U2dUxOOPKWK+RcOFPzasZkPqk4pMYaugujdd2RyG9WsCUHqoRwpK3NWj0hYvNu6CMyaivO2mpQgLPX5n0DaqSd3ZwgyfnJs2LG3q1+x0JkHeti7I6ushIi81NixUo57HJPzleK+DYOFnao5qxfhlh2O6vFpsTDqfKUOoR08xjOPxA06Mv39LA5oaUqV7/HIRlZjCr2pOusyhnihnMHr8hhg/EN6BqyabvHBVdOVdhE10G1JUcD3SHsv3Omj+gLM9ONTDHj/Tq8nmBH724GJfHfZHFq1N9JrPLt2AV97dbLQFMNfO6czk4/JS6Oe8uQ7PLt3gGzzVJxXpGUEA0N5p9vjf37wDzy3biKaGFAa0NJQc45caFXfily98Ue5QT4RzmETzjnmrfM/7Njvvc5g+qv2u6bxpIq+jl52J7Zym4313REReSCpsVS21EzKnc9aI8BPRTCL6rPt4OBHFz1tiGI1H3liLXz+2FN/7zyKvbXZIGdtS+fTvXzC2e8JvcPl9Hr+y/dO/f8EnLrrHf9oNzxWcyxbq2eq2r9i4HS2N6dAa8WFIAe+yTD5SaUyTl3FDWtzae2wQo5mTh2H6roNx9JQR+OxhEwAA1316Gg6ZONR4nWihHuc9POeQXXHAuEEAgDdWb/XtIzvYqLOmbzh7ulGMiYDfn9uGI/ccjgGtTvjI1sGZwjfqmEWK8h7/aW3jsM8u+QymKaP8mUKqsJs6Q7k9qRnskYSfiC4H8L8ALnObGgH8NRmTmHpCpjbGjUMnQbDHnxdPPUtGHZNo1gZ3pejO2G0Ill91EvYc2T+0tn13NucOFJY2yp3yPP7w9/b+rx6BXQa1+I7TH5sEcfYJU/DPLx+Km847CJefvA+WX3USTtp/NL7/0X2N14kW6nH2+t6p++ILh0807xPTIz5un1HGEg/pFOHw3YfjD5+d4b1Wm9iGhW+I8iU3Rg1oxr0XH+5t09+PsKwe2aHpWUTlIupZPwbgFAAdACCEeB+APdmVYSLixUQTvrWNQiZnj/Grwq/H7UVAVo9Oc2N47D6TFUhR6Yut52P84R1Ic0PKWHvIl9Vj+IxsHrft44wW6gkWRWefQlvDaGoolDvT8TYbjR5/weCujPH799W/3+p7aRzcdZvCOptiiSr8XcL5dgsAIKK+iVjD1B3Sq60F4ZeFwEx62xUg/FlfjN/yk3J3aUqnQj3+TE6UKYMmelZPc0PKEyNbHr/J+bQ5pKVUrfRlvFgHWlOB1zfRbAj1qKeXYRtbqCcsRTOlZPU0adfSw0zyTALmzk1+Fkn9LqK+bbcR0Q0ABhHRFwA8AmchFYYpCellJ+XZxCHIMVaX5itYDcoX4zd7/HJcwOR16mTcUE+p5EM94R5/k+Lx22L8JkG05eXb7A8qJicJS3UElBh45Clczt2WjqljsXUm5sFdf+qrvLvS99Wf63cKtvOGDRIXS6QibUKInxHRRwBsBbAngO8KIR5OxCKmrpDectJT1KMgBdIkTp1KmQXd4/elc1qEXe5jvSPw2SHKUvxL6kk0jz/tiWDa4uUbRdIa0olup44qdjbdk51DnPdJ98Kd46OHr8Li7SnKv9e6YOvf77AOVd4hJuUQhQo/EaUBPCKEOBIAiz1TVuQPJSh9bWN7J7Z3ZTFuSB/rPrmcwML3t2D/sYO8tk0dXWiPsUj4K+9uxvbOrDHVc8XG7TjYzVQpEH411GMpFSx3ieTx50RZF/gwrbWr09SQgtQYax6/0TO1XNvyeUYZtlDFzhpvd4U1zkQ8o8fvc72df7bvotHj184lQz16WEh/HlbqoTsXbSJYsYSeVQiRBZAjooGJWMDUNdmc+YeiMuNHc3D4Tx4LPM9fX1iBU379DJ56a73X9uGfPIYjfvpYZO/zpw8uxuf/PNeYZ3/7vHx54YIYv8/jN4d69FLOQZzWNjZwcZSoSHH5+wvvhu6bTpEn1n5RCh7ctQlvVE/80EmFaZ/+jJfgeHuc/tF0rh2GWkXTx5sXPw8b3AWAo6Y4xdd21wq96YO9XgaRxVY5M/ljB46x7FEaUbuTdgALiOgmIvql/EvEIqauyHv8pS2Rt8ydKPXW2navbZtlolQxqJ66HuMXkWL8zv//Piq8muQPPrpf4ALpUbH1pd//6L54/rKjcc3pU7X9g7N64oRFbAOk+uv60+dmFOyjhsPCcurl9Yf2bcJN57YZ95WYOlNfZVX3//QJg3H47oXlnqMM7p5+0DjMv+JYTBrez39sQFaPiQEtDVh45XH42kf2CNyvWKIuxHKn+8cwZUWKeqklf1vdxUy2Gzw4QrQQQxBq56OXOVa32VaFkjH+wX3tC4MAbsglRYkt6QcA/ZrTGDWwBaMGtPra04a4uS/sY/L4Lde23Qnon4Np8DLl62zM58+nc8rrAePdUGA6RUZnwTSRTR1IVk1uNXTgplIS/lCP87oHtBR+xoUef/AHTETeTOIkiDq4+yciagIgu3k9aV8AACAASURBVJ/FQojSCokwDPLec6lT1Pu4P9Qdllr3paImxhRm9dhLNuT3cf73N4iCivRwy7GknzX+7m6QtWm8di+d0+zlmwd3Y4Z6ItzKqD6AbawgKKsnRYDpW2BbPrEAYb7LNFliC4vp6OMDSZdkCCOS8BPRLAB/ArAczusfR0TnCiGeTM40ph7w0t9K/CHkPf7CnzwRRRKcINQ4vr7wd1CRNomM8fdtCp7gJQUhStpjGGGplrpnmu90lH19tXoM1wjpXIpBL3VsoiGl26oLcDSPX0V9v6JWiPXX6rHvp2cEpS13GpUi6r3EzwEcK4RYDABEtAeAWwBMDzyKYULIBsyWjYNcf7XU+jY2VA9QDxWr/YDN45fHx6kmWSq2S0nR0T1+2W7L5DEP7pqvUYrw+weUzfs0GO5OvGMs147s8SPauBKg3QXE8PjV9yfOXIRyEXVwt1GKPgAIIZbAqdfDMJERQhR4snJwN24Nfv1c8nfUkZDw5wI8fn+tnuAYfxiexx/XQANhYZh+eqjHMLjrj/dHD/VQ8ATmQFSxt3WUaZnOadhm6zyjFKsDHBujCr9KkHwXlmwo/lzlIKrwzyWi3xPRLPfvdwDmJmkYkzxn3PgcPnX9s4H7HHvNE7jo7y97zyfMvhfXPbYUh/54Di69/bXI12rvzGC/Kx7Cl/46z9cuMy2iLFh+zcNLMGH2vQCAWT97HCdc+5S3TcbdTel5xfyITee47rGluOjvLxfE+NWSwTaPP2q/Vs7Yry1RSp8VKj1/KUa+MgaK8BoHYq2dS2H7pOF9I70PoZObAAzr2+TblyifUTV+qLmizJjBrcZ2ibyUEMCogS0F28NMN73mXdzz6B3YoFbH/lEDg21KiqjC/2UAiwBc7P4tctuYHszzyzbhpeUfBO6zZG07/jN/ta/tpw8uxvtbduJ2rUZ6EGu27EB7ZwYPvu6vtS+9sCge/7Vz3gLgdBIrNm7Hm2u2eduybgeSVJXPbE5g0eqteHXlZnQEpImGZfUAwL8uPBT7jjEvOu6Jh/IyTtg3v/C4Ke/dhk2U1dTUP39uBu7/qlNFstEwKUrtiFoaU7j6tKn46tH5lFR7qMf//Obz2nDrBYdABMjnl2dNco4NSSH92aem4vh9/YuxE4BxQ/rg+s9Mw68/faDx/NecdgAuP3lv6/XVS33/1MLqouqaDKaDTH32v75yGP742YMK2g+eOATXnnEAvvtfdnuSJKrwNwC4VgjxcSHExwH8EkDwKBXT6yhlwHGrZQatrIETZ75Sl2Fnqffl8O5N5IQAhNNRBRVZs+Xxq1YdOH4w9t3FPB/S5PF/cvpY7/HoGB6iLUyiFis7Yo/hGDvYSYOUwq/a6jeH8PFpY/GlD09SrmG+ti7YR00ZieH9mwPtld6xL7xkUKhPTBtTuBKW+/T4fUdjUKs5Cj2wTyNO2m90oA0AICDQt7kBh012Olm5toBp/Ei1wvRejBzQgll7jihoTxHh1APGeEkJlSaq8M8BoH7jWuEUamN6MbrQl6KpNrGUXlSc5RZNg3Qy7l5qDXsb2ZxATgh0ZXOBE8NstXj012dLUzQuvK0IdbyZquZ22ziEvBNQyziHFTGLE+oBgkNestP21QoKKGBmIyhcFlT1VA6yShvl84FuR7Kzu/C7FVZsrRgb9fMmQVThbxFCeFMi3cf2wilMr0AX2FLWwrWtM+uFemL0KqZbbhl3T0j3kROOIHR25wLXzLXm8Wt22dJXTYKgTm6LowfWUE/abKO0SZ3h6ss+IUOb9doxDHWRX4GwUI+KKXQUJMBxso3krgNanTGQnYY5Iv4VuMon/EkTVfg7iGiafEJEbQACKz8R0c1EtI6IFiptQ4joYSJ6y/1vLorB1AT6giGlpFzKYmn6DN3OIoTflJ2RdWM9FfH4A0I9tmqK+t2T7YdvyupprJDHL+8s1HES0+BqmEce1B70HZKORTokbm5CFeAgUS1m3WDp8ZsmBxpqvEXC1KFWkqjCfwmA24noKSJ6CsCtAC4KOeaPAI7X2mYDmCOE2B1O+Gh2DFuZCqMLXGkev3MumW8vkcIf59ymW245OJxUjD+bE16a3+btdo/fJjq6WTaP39QcpT69CZvnaypPDOQ7ZbXzNAmbf9KSJZ2zKI/fLWSnvN44ufd5mwK2BSiePq4u378g4Y96bp2a9viJ6CAiGiWEeAnAFAD/ANAN4AEA7wQd687q3aQ1nwpnBjDc/x8txmimvDy/bCP+9sKKgnZd+KNosxACVz+0GMs3dGjncsRSz3opxuPXb7mvuv9N/OIRJ+OnK6Gsnpwyb2BDe6d1P5sQxo3xq3cIfo8/hvBb2m2loWXhs4zyHpouRxHc3Ki1elTkV0A9NCiDCjB/J4M6x1gdp/tfLsIe9v2PFeoJq9UT+UzFETZz9wYAx7iPDwHwTQD/DeAAADcC+GTM640UQsjcwDUARtp2JKILAFwAAOPHj495GSYOZ9z4PADgrIN39bXrBc+ieOWrPtiBXz66FPcuWI05X5/ltcusHl3fuxSPf+KwvlimdRgm1Bh/dzaH659427itXOwxsh82b+/2fvgmL/S4fUZCCGCcJVf8t5/xT3K3LephSmtVw0c2R3Hi8L5ehVLTcSq2AWjp8atZU03pFBrTZE2TDXJcPz5tDO58+T1fm+zQpozqj0+1jQMAfP0je6BvcwM+duAYPLN0A86fuZu3/wHj8+srHLnn8IJc/JEDWnDcPiNxwRH5TKMgAdY9bdOrkjbKsY7WxjROaxuLk6fuUrCveraj97LKWQHVXnco7PJpIYT02k8HcKMQ4p9CiO8AmFzKhdU1fC3bbxRCtAkh2oYPH17KpZgiKWZwV3ruulDIuwd9olaXK9TZnECfZmfQccyg4JRFNdSTyernK3+Mv23CEOSEsL7+Yf2accPZbbjxnDZjBcd9xwzA9F39w1k28fVKWChtah14k6ZNHTcIowYUTjhSve6bz8uXLLZ5/PLOwu/xE351pjkvHggW2atPO8C67cuzJnkC/99H747PzdwNg/s24S/nH4wR/fOvpbkh7c1jOGn/XfCDj+7nO086Rbjh7Dbf+xuY1RNgr3y/5KuXs8D7tzTiJ5+cisN3L9QhebpPTR8bq5pmWKgnmfvWPKHCT0Ty1RwN4FFlWzE1Q9cS0WgAcP+vK+IcTIXQRTRKNCYfH/W3y1CP7tHmQz35W+mw+QJqqKdbG8wtJiYcRprIi/GbCFtO0XRbbxNfvSMD9Kweg2AIESokaraRLfOo0RvcNb+H5rBP4GWtxAmLyH2jLj8bGOPXtpHlMQBsd8NMA1rsUuelgEYzTbGjhmP8cAqxPUFE/4aTxfMUABDRZABbirje3QDOdR+fC+DfRZyDqRC6iEaZwGVbaUp6/HosXw31eMIfco1Ke/yyvrut4wsTflO823aMKSspyoLbYdVN1Y7GtvaBDA3pd2tBH3uxAhbnMFMaafD+0UM9QWxXPH77xZx/cfMe/NU5C22qaoxfCPFDIpoDYDSAh0T+l5+CE+u3QkS3AJgFYBgRrQJwOYCrANxGROcDWAHgtNLMZ5JEiqgUlShfbtMAHQBvOUM91KMO7sotYSElNbsio3mn0uZylDWWpIjcPH7zOcPW0TWJTVNIFU+VMFEXlmv4rhdhgFjuY0uJNd1txNV9+RYW4/GXg0iD466NcoxLr2LqO1+RdhSTVlpOQsM1QojnDW1LIhx3pmXT0RHsYmoAKaIyAyVKjN/u8ZtDPWqtHnls2GXUAVy9fIPcVs6aPemU2zFZThkm/MYUTYvXLe32XUvZ1RZwMgq/sqstd99kk34XFfROxi0pLO2Pk80o9y0lnTgKuhbLGL9exbQcBM0grgRVHltm4rJ0XTvWb7OnE9rY2Z3Fqys3xzpGimhXJoct27uNoY7Fa7Zh5abt3nO5TzYnMG9FPptXhnp2due8BdE7OjOe955ThHXdtk48tnidtba+Gup59u2Nvm054dwFlHMiVypFTsdkkUBbKEYO9pm8TFu4xbzyU7hImBYCV+215e6ryM4jToy/WP2Kk5YqnYikZmXryPdNOiVBHr9+TDFUowtg4e9hHHP1EzjsqkfDd9S47M4F+Oh1z2DNlp2Rj1Hj5Wf87nljqOO4XzyJw3/ymPdc/gDe2dCBT/z2Obz+vjMU1KGkhp5904vYvL0Ld76ST/VzQj3583/2Dy/h67e/arRLLb38jTvmF2zvzOTK6/ETIZcTVuGxhWJk64qNhSmqJqEG8ncw6nsRli0iRHiop7kx7RUdszFxmLNA+En7+wuZ7TmqPwDguH3yFTFHuAXXwsIwYwe3+jq5YkI9spMo1uMfNyRaYbsT3QJuk4c7r/fMGU4auW0wXLWtlDScvUYXVmotx9KbQbDw90BM1SnDeM319ttDJsSoqIO7b6zeGimrR/dYN7R3Gdt3dufQ6Xr744a0+gZ38zab8wdslT49G4TwYv9nf2jXwH1t6IuMB3n8trDNV49xyhebloO0efymwenWpjS+fdJegfZGifH/+XMH460fnmDdZ9TAFiz+wfEF79mk4f2w+AfH46MHjvHaHr90FuZfcWxoyOLx/zcLb3wvP4HfGyRMONSz5Af513nvxYdHOua0tnFY/IPjMX6oU4bshx/d13ceE/JllOJmTB7hvL+VJLll3JmawpsOH+MHV5jOGT2PXz9Gb8/kct62xlQqMF1SZ0tAyQTAqdsji7bJAltxcQZ08+MVQtjTWW3eu5zxaUrRtB0j0d9qmQVk+ggEwtM5mxpSSKcI6ZDAgs271dv10hs29HkNtjGgILxQTwx1Vcdd4qznrL7OVIrQFLGKZqnJBEF3FUnAHn+dIH80QQKhf3n1WbBRhF9fnQpuNkxOC0fklLz9dCo4a0ZHLx6nkxXCi1O3FPmD8q0C5dptKyth8977uLXWTeMN+l3CoD4hK5kGCKUQZnFT386wlNOKE8fjd00vNtSTZM58GSI9VaHGvg1MUuQieFq2HHvnuIjpnAZxlE2qQDoev/O4IZ1yQj3acTZTt+wIEf6c8F5LlGwWE6SFeoDC1FGJzXtvdRdlMY036APCppm3JmwfQVjIJcrgbiVJ2uNXSTJcXo2F0stBbX0bmMSIItq2WbWAI85RzqF7/AJ5EVbFTpY5dtqdCVJR3aYw4c/khCe2thWxwjBVoCy4m3FJWzz+oNWVdA99iLuGrNWewK3h4Yxqpw9Kiovxy3kkxSl/TxTnpC1m4a8Tovxo9IiE6vE3pqi4GH8uf7ehep1q3n5DylwSweaphYV6cjnhhVfKE+px/pti9YDz3qjICqStAZ2OHvuOGjM3xvgjZPXUDEVN4HL+m+4m4xyfJAlPMSg7PLhbJ8jfjPyCfuOO1wr2OeXXT+Phr33Ye96Z9Xv8YcJ/3DVPYvHabb42ARg9/tOuf87LzvFCPYbzm9pWbgpcAwgnXvuUd7dSdKhHeZz3+M2hngHaGq/7jx2EF9/ZFDhgN1A7ZsQA/3q0cYRkwrA+1pmgTelUUVlgSRMn/LKLW7RvSL/gNXttBHUyk0f2K+qcEnmntktIYcFag4W/TpCiLf/fNndVwT5vrWv3Pe/WYvyy8/jwHsPxxJL13jbp4eqiDzjCLUNI6oCmmpLZmCbs7DZ4/KDIcd1Lj9sT27syuO6xt31r4habLWEa3DWFeg4YNwgXzvIXqv3dOW145d0PMKy/PXwzeUQ/3HxeG/YaPQAPL1qLY/cehdPbxoUP8mrcfF4bZuw2FL+a46xHcNbB49GVyeH2ec7n+/ils7B6S3BHWUnyoZ7oyv/5wydi3JA+XpXOuNgu9ZfzZ2C/MeZF76Ny+O7DcP1npuGoKdFLMtcCLPx1gtSsOJkRqtARked9f6ptLDZ2dGLhe1sxddwgLFvfbjsFciJ/i24LR8jUSZNpURdo2WNkf+OMU33hl6iYB3cLbTnv0AkYNdA/MDuwtRGz9hwROhYhxeKcQyYAQMF5bPaYziEn/PRracDUsYNw+7xVEMLxRGvJG5XfoTjRl3SKvMlVxWCbDGUqs1zMuY/ft3jb7Ccu/ylVOMZfJ3h1cGIcowopQSnABvJEsG9T2hr7BvwZNqbvMpEr/LnCCVJE9o5Kr4+TTpk7Ft3jj+pomhb8NoV6gs6XTAql+f2Q5lIPGcqslcHmeoWFv06QAhonM0Jfgk+dBCbFvLUxjUwuZz1vJpfzQj0mz0uIfBgpjsffrA2OpoiMmS16jn3UyTxqKEI+NtkSFLIoJYUybgaLtKPKRR9DKSarhyk/LPx1Qj7UE/0Y/6Lb5AkzEXli3tKYRndWGMsSAE4OuzyN7beeTplDPc5dRlSPn4xepJ49EzX7xV+ywflvivEHru9aRnUL8+PltXqKniZdi4YJhmP8PYhSpoXrg7tRUCceqSKsevwynLHZEs/OZPMef9DC3LYOyVYYTQ+jpImMbozu4TuTraJkuRg8fkNIq9Lzomwfn3yZtZ5VWEyRNqb8sMdfQ7y3eQe++++F1hmiuji+uWYrfvLAm9E6BHeXi295BX99fkXgrqu3OHaoE7jWbevEVfe/CcD50eZnxjox9A86uozn6s4Jb3DX9lN3ZgUXpnMu37gdX/n7y8ZjdI8/lSKjN6/PkC3O43ee6Ms8ApXzXMMuk5/kpJYRqL1uoJh6/Ez5YeGvIS659RX8+bkVeG2VuW6+7q2fcePz+M3j/vRFG/LYJWvb8e27Fgbu+4075uPPz63AG6u3Yg8lz/nppRsAOLVTfndOG86cMQ4T3EqGW4M8fin8FvWSWT0mr19eU8cU6jGVTmhIEy6cNQmfmDYWMycPw96GErgqk4b3dW31n9t5LfFCPQDw1aN3xx/OOyhwHxPySp+YNhYAcOoBu+Ck/Ubjax/Zw7h/yif2+U6g1mCPH/jiERON7RcfvXvRKatxYeGvIbbucAS8tdEcgdMHF2WefZSfULF1TnYf0b8gXEJE2Gv0APz44/t7AtxhifFnsvk8frvH7xZpi+GhFnj8RMawS2MqhW8cPwU/P20q/vr5gzFyQPAkoHMPneCdTz03UFjSwtkWbOf/fGQPHDllRPBOAXz9WEfo+zQ14LqzpmGEpaZPOerCM5Xh/Jm7Gdu/9pE9cFQJ35U4sPDXEHKxEttsU113hPY/iDixfdUzb0hTQbgk5dvuCr/lrqM7l8uHeqwxfljz+G3o3n06RUhbPH6VsAFX+dpMwm+yLynPNa637tmYgC1JUM8efxRPLemkXBb+GkJmxthi9iaPE4hWwySOkKjaaAqfqF9JWadGXWFLxe/xB4R6YtTj120EnMFdU6pmgfCHCI5pDMDUV/RtkpU3ky2HEHneQYn1bCqFF+qpY+WphZkWdfz21x7Sa7Zpie61y6dRZrfG8fh9Hn2KCkIwcTx+f4zffL18Pf7IJhYIeCplFvXGlH63EnbewjbT2ET/Fqe0gq3DK5W4A7Oyw6px3feoZ4+/Fl46C38NIbNobCKte3NSHGx3Ar5jYwl//nFDujDVUt0uJ0h1dFry+APWqpXkJ4cV1zkB7uCuoTyy3haaD2/4VZo6A7myl+11Vwu1w6jlPqCes3qCXnqlPjPO468R3t+cL6SVzQm8tXYbWpvSGDu4D7Z3ZbBg1RbsPrK/7xip5WHCCkT/Qs1ftRlL1uZr7zSmUwUH+8YAUsEe/7NvbwwsTww4Yrt1R3foWrp+G/zP02SewKWPT0RNiwxrGyA9/hhrGCcJKeMQteBR2vBq9dSykQkT5bUn/fawx18jHHrVo95jIYCPXPMkZv7fYwCAS2+fj9NvfN7XOahE8fijOvyn/PoZvLtpu/fcHOrJP5bZNdssov3ays24+uElAICj9zJnLKQofAF1HT0Wn0pZYvwxXUtjjN/wKzneTbs7aLchsc4fFVm4TS/fbKOneNDFVOcsNxOG9sHQkIVvkqQWPir2+GsQXcjfWL0VQKG4yr2SHNAzrbyletb9mp2v0MaOztBzfWjiUJw/cyKmff9h//mKEAFdoNNExvPoJRvkLl//yB7492vvY6lWilq+NnWA3eShfWjiULzz4xMT81y/cuRkfOXIyeE7ukgril2XttJUU/wev/TIKl69Nu7I2OOvQawxfks+Z9TSxcXQmC5ceUvV3P4tjvBvaDfP3FVJk3l2bTHiqR9ji/HbSKUIOwxzD0yes2mRkxRRTYUr1FBPT6CuB3eDur0KfX4s/DWIzYO3dQhRQj3F0pBKGb6L+S+ujHVvspRsULGFY4oJU+jH2Eo2FOLsQ2TOyJEir75mY9y/xn45pjINtdgJ5Av9VdeOqhIpjz9ZauzrywD2lDzds5c/8mRDPVQY6lG+lf08jz881JO2iHMx3l9BVg+ZSzboyMNSRNhuyMgxDRCb+pOgNQiqgfT4c6I2Ysg2vFo9PWVQIgFqodOrSoyfiJYD2AYgCyAjhGirhh21SkG+vq1dhnoSdO30evaAX3RlqMdUlrmlMYWd3TnfcSbhL2ah8EKP3xySCTretBatqRMyhXSSDK8Vg1erpxbdfIV8rZ7q2lFNauGlV3Nw90ghhLkCV51jE5XurLlDSFKETF60Ko6N6VSBwEsG92nC6i07vefpFBnFuRgPSBdjAiEdI8Zvi7Oa6v0YPf4oObQVhGq4MJuJWpi9Wi2CxoYqVVGVQz0ReHzxOkyYfS/eMiwmHsZNT7+DCbPvxbad/uqVE2bfay05fNbvX/A9l18TW3mAXA7YtrMbE2bfi98/tQwAcNr1z2Hfyx/EhNn34tO/ez623RKTx69/b/s0mf2HQX38KXO2XPtiQj1jtHVkicy26sg9iIBxQwrXotUHSVsaU0ab+zbXVkKcF+P36Ubt9QLSoloId1SLSCNRvTSPXwB4iIjmEdEFph2I6AIimktEc9evX19h8/w8+PoaAMBLyz+Ifaysfb9uW2EM/N75q2OdqyvjF355W58VAhvdrJo/P+dc78Xlm9DuTi569u2Noec+ck/zwtNRhNq2fsBobfFwefNwjJbPX8xt/yenO+WKh/Ztwt+/cDCG9WtGc0MaN5/Xhr+cP8N6nJxMls0J3PnlwzBz8jDfdvWO5J6LZuKJS4/02ffxaWPwl/NnYMqo4PLOleCJS2d5j6WJtViD30Qd635NZDRVS/hnCiGmATgBwFeI6Ah9ByHEjUKINiFE2/DhZlGqFPmBs/g/qnLGXnWPXw315Gu1FHed4/cdhanjBhW0G1MZI35rRvT3l0CWNh40wT/pKe4PYeLwvt65BrY24tBJefE+aspIHDJxqPVYORjd0ZnB8P7NmDpuoNFGAYH9xg7EyAEtvlvzpnQKh+9e3e+jZNehfb3H/oVYajfsU4s2VZooX/ek36eqCL8Q4j33/zoA/wJgd9FqAC/Fryjhz2dblIoe45fkhLDc6kfHtli5yePX47O2mKUc+JXI91EfzI2bD6/urZdkMJ1fRU44a++UlVC1c4fk8dfsBCnXxFrP6vHoEUZWnkqNfVRc+ImoLxH1l48BHAsgeEmoKiN1pJhB1JThbsEWGjGhdjaFoZ68XaXclQD2gVfz5CW7jSr9mv3lBmQnUlBZs4jvukynNE3aCupIZGxe1tjRLQ+r1ZNwFeai6Tmhnlq3L3mC/JxKfX7VGKEaCeBf7o+zAcDfhRAPVMGOyFAJXrv8kNUkEFMaoQ11X+vgrsiva1uS8EdMtYzqoUf1+OOGeojIW//W5PEHIYW/3Z28pb9dXqhHaVfNq9V0Se89rE3zPLwJXHXs8kd57UkPA1Rc+IUQywBMrfR1S6GU+LnJ49c99yDUBc+DsnqynvDHu6OQ2EoeRJnQFDnUIz1+Q4G1OBDyy05GyeRR6dfsDO6275Qev32tAVNbrYZ6PAejRu2TcFZPbbz22spJq1FKCvWkCo9VxfyN1Vt9z3X+OW8Vlm3oAAB0WWL8n7npBXzh8N0AAOu3deJ/bnutYB+i4Pi/rY6OrU6Nis0LlouVeMe559fPGTfUQ5R/P+N6/DL81Jkx19E32aIOZtfYhF2PnrL0Yj1P3JLUwlvAwh+BoMW2ox6rTvhRPf6zb3ohsMDZlfcsMh6nc9er73uP73nt/YLtowa0+CZTFdhpi/EbJzRF++oOaGnAjWdPxw/vewMHjBuEkW6Wj37O2KEeEGbsNgRnHTweF1oqWM4+YQr2GzOwoH36roNxziG74vMzJzoNllCPzb5a9ajzHn++rRYt/dvnP4Q75q2sSFnk6z8zPVIpkUoTOIGrQh8aC38EUoa4b1Tkh6xm5Ni8zTCC1nft7A4+52lt43DtnLes2xsi1NEZ0rcJmzq6AhdNV9+jfi0NOHTyMBy7zyjrOZ3ngaYbr9OQTuGHH9vPus+XPjzJ2J5OEb536r7ec9vgrr9IW/5xrcb4JULJ8KpF9hzVH986ae+KXEuumVBrRJrAxYutVx/pCReX1eP8z2TNoZ44OhIo/CHjBnLBFBspS4xf7QwaLB2g7Nz0OwY91GM6p3p8LaDmw0tU+2qsUoMH9ZBQD1MbMX4W/hgECa8NKSTdllBPnB9qKcLfHCL8toVM1IFXGU8vLBbnPNcFXR/c9a5VhqyecqF78GHjHEkWxCsFz+zaNI9RqAVHh4U/AlK042TjSMI8/jgx4zBxDyLM47ctUaiKnsygsdkcVfjLkcdfLoJKTufb8o21GuqRoYFaHYNgolGpT4+FPwJStIsRXtm7qymWXUWHeor/WjSFZL+kUmT80qliLr1/PeIlX6Mu6M0N5kXWdYcn/uBu+SiI8RtcZ9W8GqvG7JEvDZJvq9VOigmn1+Xx9yQWrNqCzkwWL76zCUC48L+0fBN2HdoHI/o7xcne2dCBJW5Fz3sXrEZWCJy032gtxh/9x9mtHPeQWzguKmEefzpFRndDFWVPEiOGemyUOnBVzh9FwQSukHBXrXrUah5/DUQSmBqHPf4ATv710/jk9c9hwXtbAAA7QzJnPnX9czjlV894z4/8fcelXwAAE/VJREFU2ePYvN0px/yf+atx0d9fwcOL1hYd49+pZANd8Jd5MY6MJvymGvPpFOGzh00AAHz2MGeuwEit6qbktDanYubUsQMD7zB0YTLFzffZZYAXWjpzxvjA40tBz/wwnVvtz2phAZajp4xAS6P+/vLgbm9AFjA8af/RiV6HPf4YbNtZuEarzpqt9lx5ANjY0eX70aqat+fI/lis1Pzv25RGh7KylexEikEV4l+eeSAuvuUVTBnVH2+uca6XJjKuopVOAZefvA8uP3kfAMBnPrSr9RrH7TMqUqqerq2qmI4f0gdPfuPIgmNuefFd5fjyKf+M3YZg+VUnYcLse63n9sf4y3bpornpvIMK2oyhngrZw5SPySP6YflVJyV+Hfb4Y6AvpqIStUyCEEBntznUo3ubLY3+GPnm7eELmttoVDx+meGjhi3SKcIOg/DHib9HDvVou6lrBkcJpSQZyjBVOfXH+GtTTvOlmEVd18FhosHCH4Mgjz/OwK9aeC1IRvQUzA9K8PibFY9fhn1UTztl9fiji0iUxc4d/OdUQz1JLhwfBVOtsx4xc9f9X5vWMbUGC38MtnXahTdqqieR3+MPEhI9Lr9lRwmhHoPHr166IU3YYRjDiOPxR9X9II8/Sp58kv6sKcfaX48/wYuXQP5OJW9gjfZRTA3Awh+Dcnj8Qmgef8CP05YOWQwm4VdF1vH4C19fEh5/QYxf9fijiFWCsR7TmX0ef40qv6/UBEd6mBB6vfDL2a6ZbA5dmRw2tndCCGGcBdudzQWmV27e3g0hBIQQ6MrkvFBJt3vuKGRzudDsIElYJk4c/MKfX3dWkk4RtneWFuqJHuP376dqaRRhTVLXUkqs3Lue8jHUbKjHUKSNYWz06qye5Rs6MOtnj+Onn9wfl94x32ufOXkYnl66ATef14ajpowEAKzbthMzfjgH3z91H5x9yATrOW95cSVWfbAdv3n8bQDAa5cfi6lXPoRPuYt/h/Gdf7/ue67eKeiCGFZmIQ5Nhhi/KrINKcK+YwbiuWX+hdnjhHpss3/DUO3Y11BRU2ev0cktdG5O51RLNiR26ZJQB3eH9XWqoO4xsl81TWI0Jgztg+Ubt1fbDAC93OOXqYp3vvyer/3ppRsAAA8uXOu1Ld/gfCD/frWwpDEA3PGlQwAAG9o78ZfnV3jt67c5ZV9vn7eqTFY7/OG8gzyBHtq3CdecPhU/+cT+OHrKCG+fC2dNwr8uPBQXWUoTq6gev6y54wv1pAg3nDMd//7KYfjtWdO89kQ8fu25vPM4ab/RuE65to3LT06uuqM51JN/XKuzYfOT64D9xg7EbV88BN84fkpVbWL8/OvCw3DvxTOrbQaAXu7xS2wDhl2GMgrNBRNjgEMmDsX0XQcDcEJGzQ1pbEN4Tn9cVNGZPKKfJ9anHLALPnagc0exbEMH5ry5DgBw4n6jse+YgXhp+abQc6sev/TM1WhXmggDWhoxddwgTB03yNcelWLTOeXnc+D4Qd5i6EHoaa5lJcTjr/VQj1xRbMZuQ6poDWNicN8mDK7AOgRR6NUev0xus+XYq3Xx5WMpkKpn15AmEBEa04TunPCFYEy576VZ65BOkXcdVXxVbZVCG2VQ1eTx53wev/m4yBmaiB7qsWX1xLm7SApTPX5fHn+NlmU2lZNmGBu9XPgdMpYRLzWtUqYymgY+GxSBdTz+/Nu2oaN8K/wIbUJVk2tLWqmTr4qQtCvKurOq8Mu6+9lcvq6LrfOII8ZR187VJxhJj78WhF8NmUjSPcHjd//Xqn1MbdHLhd/5OdiqWqqhHrn4thRItbNocD3khjShOyt8IrphW/mEP6elV8oORvWk1bCDFNqGCOvOqqGexlR+cFee2xbSiRXqibqvHupxP4a4VTqTwJTH3xNCPfI9rVXzmNqiVwu/9NqtoR7F45c5+s2GWa3So25Mp5DJ+T3+jR3Fl1HQz6/emDgevxvqUbxxVZjydyLhgql60/IOIiuE5+nbQz0xPP6Iwq3vVUuhHpMFPaEsM3GRNiYGvXpwtyvrhG/eWtdu3P7i8k248G/zsLG9CxOG9gXgePVPLlmP+xas9vaT4tiVyeGvz7/rO8dV979ZcN7p3384VocwsLUJG9o7fQKeJkJjmDeeyndIYagdhjxfQ4rQmE5hR3fWKhhRvHgnJBV9sLtgBa4YHVjSmF6u6b2rNWT4rlbtY2qL3i38ESZV3bfAqWsv0zK7MgIPvr4Gt7600ttH/qjaO8PFrW3XwZi74oPINu4xsh8unDUZi9duwxkHjcOHf/o4AMcDb3azV1Rdb1dmD0sBPSggg+OQiUPx4T2HAwC++OGJ6NPYgNamNC45ZnecsO9opAi457X30d+STRPF47/lCwfjntfex4DWaF+nQycNw/kzd0O/5gYcNGEI9h0zAM0NKZxywC6Rjk+CO750COav2uI9t6Vt3njO9EqZFIvp4wfjC4fvhs/N3K3apjA9gF4t/GFlFMYMasV7m3cAANa3u8KfzRXM6m00xEEG9WksKJN878Uz0ZRO4SPXPOlr18stq/zpczMwemBrQXs6RWh1hb9LGaPY0V0o/GMGFR4vmTpuEL704UkAgMtO2Mtrv+SYPbzHXzt2T+vxUcIvu4/sH3gO0zm/81/+XPwrTtkn8vFJ0DZhCNomDMGWkEJ4u7p3hrVGKkWRSmIzDNDLY/xhHv8QJadWxvi7MtmC8ssNhqwZ03qyA1oa0b+lsaA9aEDQJqwpIrQ2OcK/Q6mh066UVYiSxlnqnT+HDhim91G3Hn9jmjCwtVCkOzM5dGghHVMMvX9zI4Ad/raWBuO+gcIfEL/v6wq/Wi5ZtS2KKJcq23Hy+BmG6RlU5WdNRMcT0WIiWkpEs5O6TpDwN6VTRq+9K5MrqMJp0tc+TYWzR/s1Nxjbg1LsrPnzRGhtcuxThV8dZ0hHyN8vlXr1+Dk7hunNVFz4iSgN4DoAJwDYG8CZRJRIcDIo1NPcmDYKf6dB+E2Y9LAhnTLmgQd5/BSQRtnH8/jz9qgef5QsmJJDPTWQaVNRvBlcVbWCYRKlGqGeGQCWCiGWAQAR3QrgVACLyn0htSSDjkxl1JkXIyMnKkG530Gy2moI9agef5S8+VKX4TN1ZL2ZOnu5TJ1SjVDPGAArleer3DYfRHQBEc0lornr168v6kJBHv/O7iz2HxteAhgA9jaUAf7wHsN9z3cZ2OI9njTcn/lxwn6jrOfWC45dfPTu3uP93BLFZx083mv74hGTvMeqx3/pceasmuP2sV87iEuO2T18pwoycZjznn551qSQPUtDZlJ9VXv96RThvEMnJHpthqkUVOkys0T0SQDHCyE+7z4/G8DBQoiLbMe0tbWJuXPnxr5WLiewsaMLB/3wEQDAI187Av/7zwWYt+IDr+5+Nicw6Zv3AQBev/I4nHrdM1iqTPi6/jPTcPy+owEAE2bfCwCY++1jMKxfs/d82Y9OtOa7Z3MC6RR5+0pOmboLfnnmgbFfk2rH8qtOsm4DgIVXHhep2iXDML0TIponhGjT26vh8b8HYJzyfKzbVnZSKfLF8RtSKa8GT35yVF6w+zSlC+LmplWwWjUvPWiSky1GXomQQr0OzDIME0w1hP8lALsT0W5E1ATgDAB3J3UxXx36NCHr1tU1rW4VVqBLUs6VsZKEUzEZhjFR8TiAECJDRBcBeBBAGsDNQojXQw4rGtUbb0ynvEqQUcXblG4ZpRpmLcAeP8MwJqoSABZC3AfgvkpftyGV9/ijLmSeVDpjJYZW6i4Vk2GYSPQM17VMNKTzMf6mtHn5Pj3c05PFs95SMRmGiUZdCX9zQ8qrs29aWxcApo131puV5RzUmbjD+jWXzZYpo/sXfezw/uWzg2GY+qMucv0euORwtO/MoKUxjUxWevx54X/qG0d6Mf/vnrw3zjhoPIb0a8KSNduwzy75HP6H/+cIfLA9X2f/6f89MlKhNAB4/rKjsWx9O/Yc1R/vbtqOqWMHhR9k4aFL/HYY9/mfI4o+P8MwvZu6EP4po/LibfL4xw3p4z1ubkhjP3dil17ueHDfJgxWKnqOHdwHURk1sAWj3EleQ0u8c9DtMLHHyOLvKBiG6d3UVagHyC/s3dRDMnMYhmHKTd2pX1abwMUwDFNv1K3ws8fPMEy9Unfq5wl/D5l9yzAMU27qTv1kNczGCixiwjAMU4vURVaPyq0XfAiPvLEWfZp650u/7tPT0LeZxy8YhrHTO9UvgMkj+mHyiH7VNiMxTtp/dLVNYBimxqm7UA/DMEy9w8LPMAxTZ7DwMwzD1Bks/AzDMHUGCz/DMEydwcLPMAxTZ7DwMwzD1Bks/AzDMHUGiUos/loiRLQewIoiDx8GYEMZzUmSnmJrT7ET6Dm29hQ7gZ5ja0+xE0jO1l2FEMP1xh4h/KVARHOFEG3VtiMKPcXWnmIn0HNs7Sl2Aj3H1p5iJ1B5WznUwzAMU2ew8DMMw9QZ9SD8N1bbgBj0FFt7ip1Az7G1p9gJ9Bxbe4qdQIVt7fUxfoZhGMZPPXj8DMMwjAILP8MwTJ3Rq4WfiI4nosVEtJSIZlfZlpuJaB0RLVTahhDRw0T0lvt/sNtORPRL1+75RDStgnaOI6LHiGgREb1ORF+tYVtbiOhFInrNtfVKt303InrBtekfRNTktje7z5e62ydUylb3+mkieoWI/lPjdi4nogVE9CoRzXXbau7zd68/iIjuIKI3iegNIjqk1mwloj3d91L+bSWiS6pqpxCiV/4BSAN4G8BEAE0AXgOwdxXtOQLANAALlbafAJjtPp4N4P/cxycCuB8AAfgQgBcqaOdoANPcx/0BLAGwd43aSgD6uY8bAbzg2nAbgDPc9usBfNl9fCGA693HZwD4R4W/A18D8HcA/3Gf16qdywEM09pq7vN3r/8nAJ93HzcBGFSrtro2pAGsAbBrNe2s6Iuu8Bt8CIAHleeXAbisyjZN0IR/MYDR7uPRABa7j28AcKZpvyrY/G8AH6l1WwH0AfAygIPhzIBs0L8HAB4EcIj7uMHdjypk31gAcwAcBeA/7o+65ux0r2kS/pr7/AEMBPCO/t7Uoq3KNY8F8Ey17ezNoZ4xAFYqz1e5bbXESCHEavfxGgAj3cc1YbsbYjgQjiddk7a64ZNXAawD8DCcu7zNQoiMwR7PVnf7FgBDK2TqLwB8A0DOfT60Ru0EAAHgISKaR0QXuG21+PnvBmA9gD+4IbTfE1HfGrVVcgaAW9zHVbOzNwt/j0I4XXvN5NYSUT8A/wRwiRBiq7qtlmwVQmSFEAfA8ahnAJhSZZMKIKL/ArBOCDGv2rZEZKYQYhqAEwB8hYiOUDfW0OffACd8+lshxIEAOuCETDxqyFa4YzinALhd31ZpO3uz8L8HYJzyfKzbVkusJaLRAOD+X+e2V9V2ImqEI/p/E0LcWcu2SoQQmwE8BidkMoiIGgz2eLa62wcC2FgB8w4DcAoRLQdwK5xwz7U1aCcAQAjxnvt/HYB/welQa/HzXwVglRDiBff5HXA6glq0FXA60peFEGvd51WzszcL/0sAdnczJ5rg3GLdXWWbdO4GcK77+Fw48XTZfo47uv8hAFuUW8JEISICcBOAN4QQV9e4rcOJaJD7uBXOWMQbcDqAT1psla/hkwAedT2tRBFCXCaEGCuEmADne/ioEOKsWrMTAIioLxH1l4/hxKQXogY/fyHEGgAriWhPt+loAItq0VaXM5EP80h7qmNnJQc2Kv0HZ3R8CZy477eqbMstAFYD6IbjqZwPJ247B8BbAB4BMMTdlwBc59q9AEBbBe2cCeeWcz6AV92/E2vU1v0BvOLauhDAd932iQBeBLAUzm11s9ve4j5f6m6fWIXvwSzks3pqzk7Xptfcv9fl76YWP3/3+gcAmOt+B+4CMLgWbQXQF85d20ClrWp2cskGhmGYOqM3h3oYhmEYAyz8DMMwdQYLP8MwTJ3Bws8wDFNnsPAzDMPUGSz8TK+GiLJaZcTAKq1E9CUiOqcM111ORMOKOO44IrrSrdx4f6l2MIyJhvBdGKZHs0M4JR0iIYS4PkljInA4nIldhwN4usq2ML0U9viZusT1yH9CTt35F4lostt+BRH9P/fxxeSsSzCfiG5124YQ0V1u2/NEtL/bPpSIHiJnXYDfw5mEI6/1GfcarxLRDUSUNthzults7mI4Bd1+B+CzRFRrs82ZXgALP9PbadVCPacr27YIIfYD8Gs4YqszG8CBQoj9AXzJbbsSwCtu2zcB/NltvxzA00KIfeDUtxkPAES0F4DTARzm3nlkAZylX0gI8Q84lVAXujYtcK99SikvnmFMcKiH6e0EhXpuUf5fY9g+H8DfiOguOOUAAKekxScAQAjxqOvpD4Cz0M7H3fZ7iegDd/+jAUwH8JJTBgmtyBfj0tkDwDL3cV8hxLYIr49hYsPCz9QzwvJYchIcQT8ZwLeIaL8irkEA/iSEuCxwJ2eJw2EAGohoEYDRbujnv4UQTxVxXYaxwqEepp45Xfn/nLqBiFIAxgkhHgPwv3BKI/cD8BTcUA0RzQKwQTjrFTwJ4NNu+wlwioUBThGuTxLRCHfbECLaVTdECNEG4F4Ap8JZku9bQogDWPSZJGCPn+nttLqes+QBIYRM6RxMRPMBdMIpmauSBvBXIhoIx2v/pRBiMxFdAeBm97jtyJfVvRLALUT0OoBnAbwLAEKIRUT0bTgrWqXgVGf9CoAVBlunwRncvRDA1YbtDFMWuDonU5e4i6K0CSE2VNsWhqk0HOphGIapM9jjZxiGqTPY42cYhqkzWPgZhmHqDBZ+hmGYOoOFn2EYps5g4WcYhqkz/j/h+M4uHS4QqgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "scores = dqn()\n",
    "\n",
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(111)\n",
    "plt.plot(np.arange(len(scores)), scores)\n",
    "plt.ylabel('Score')\n",
    "plt.xlabel('Episode #')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "env.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Watch the agent play!\n",
    "\n",
    "On the code below we let our trained agent play the *Banana game*, and we can see that even though we've trained only on **CPU**, our `agent` performs a score of **14**, near the mean score over 100 episodes on training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:unityagents:\n",
      "'Academy' started successfully!\n",
      "Unity Academy name: Academy\n",
      "        Number of Brains: 1\n",
      "        Number of External Brains : 1\n",
      "        Lesson number : 0\n",
      "        Reset Parameters :\n",
      "\t\t\n",
      "Unity brain name: BananaBrain\n",
      "        Number of Visual Observations (per agent): 0\n",
      "        Vector Observation space type: continuous\n",
      "        Vector Observation space size (per agent): 37\n",
      "        Number of stacked Vector Observation: 1\n",
      "        Vector Action space type: discrete\n",
      "        Vector Action space size (per agent): 4\n",
      "        Vector Action descriptions: , , , \n"
     ]
    }
   ],
   "source": [
    "env = UnityEnvironment(file_name='Banana.app')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "brain_name = env.brain_names[0]\n",
    "brain = env.brains[brain_name]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Score: 14.0\n"
     ]
    }
   ],
   "source": [
    "env_info = env.reset(train_mode=False)[brain_name]\n",
    "\n",
    "state = env_info.vector_observations[0]\n",
    "state_size = len(state)\n",
    "action_size = brain.vector_action_space_size\n",
    "\n",
    "agent = Agent(state_size=state_size, action_size=action_size, seed=25)\n",
    "agent.qnetwork_local.load_state_dict(torch.load('checkpoint_Oct-20-2019_1941.pth'))\n",
    "\n",
    "score = 0\n",
    "\n",
    "while True:\n",
    "    action = agent.act(state)\n",
    "    env_info = env.step(action)[brain_name]\n",
    "    next_state, reward, done = unwrap_env_info(env_info)\n",
    "    score += reward\n",
    "    state = next_state\n",
    "    \n",
    "    if done:\n",
    "        break\n",
    "        \n",
    "print(f'Score: {score}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "env.close()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "drlnd",
   "language": "python",
   "name": "drlnd"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  },
  "latex_envs": {
   "LaTeX_envs_menu_present": true,
   "autoclose": false,
   "autocomplete": true,
   "bibliofile": "biblio.bib",
   "cite_by": "apalike",
   "current_citInitial": 1,
   "eqLabelWithNumbers": true,
   "eqNumInitial": 1,
   "hotkeys": {
    "equation": "Ctrl-E",
    "itemize": "Ctrl-I"
   },
   "labels_anchors": false,
   "latex_user_defs": false,
   "report_style_numbering": false,
   "user_envs_cfg": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}