Sutton-Book/ch_02/K-Armed-Bandit.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <span style=\"color:#2595bc\"><i>K</i>-Armed Bandit Problem</span>\n",
    "\n",
    "\n",
    "<a id=\"1\"></a>\n",
    "## <span style=\"color:#2595bc\"> Basics and Definitions </span>\n",
    "\n",
    "Imagine you have a slot machine with $K$-arms or levers. Whenever you pull down one lever, you are given a reward based upon which lever you pulled. Each lever arm has a reward associated with it. When the probabilistic distribution of rewards associated with a lever do not change with time, ie they are fixed, it's <span style=\"color:#f31818\"><i>stationary problem</i></span>. When the rewards change with use of levers, it's a <span style=\"color:#f31818\"><i>non-stationary</i></span> one. Your objective is to maximize the expected total reward over some time period, for example, over 1000 action selections, or <i>time steps</i>\n",
    "\n",
    "\n",
    "\n",
    "Each of the $k$ actions have an expected or mean reward that we expect to get when we select that action. We denote the action selected on time step $t$ as $A_{t}$ and the corresponding reward as $R_{t}$. The <b><span style=\"color:#f31818\">action value</span></b> then of an arbitrary action $a$, denoted $q_{*} (a)$, is the expected reward given that a is selected:<br>\n",
    "    $$q_{*} (a) = \\mathbb{E}[R_{t}|A_{t}=a]$$\n",
    "\n",
    " \n",
    "Simply put, we <b>estimate</b> how much reward we ought to get when we choose action $a$ based on our previous experiences when we chose the same action.If you knew the value of each action, then it would be trivial to solve the k-armed bandit problem: you would always select the action with highest value in order to maximize the objective. We assume that you do not know the action values with certainty, although we may make estimates as we play the game. Based on our ever-evolving estimates we will choose our option that is most likely to give us the most reward.\n",
    "<a id=\"2\"></a>\n",
    "## <span style=\"color:#2595bc\"> Explorations VS Exploitation</span>\n",
    "\n",
    "If you maintain estimates of the action values, then at any time step there is at\n",
    "least one action whose estimated value is greatest. We call these the <b><i>greedy</i></b> actions.\n",
    "When you select one of these actions, we say that you are <b><i><span style=\"color:#f31818\">exploiting</span></i></b> your current\n",
    "knowledge of the values of the actions. If instead you select one of the <b><i>nongreedy</i></b> actions, then we say you are <b><i><span style=\"color:#f31818\">exploring</span></i></b>, because this enables you to improve your\n",
    "estimate of the nongreedy action’s value. Exploitation is the right thing to do to\n",
    "maximize the expected reward on the one step, but exploration may produce the\n",
    "greater total reward in the long run.\n",
    "\n",
    "Reward is lower in the short run, during exploration, but higher in the long run because after you have discovered the better actions, you can exploit them many times. Because it is not possible both to explore and to exploit with any single action selection, one often refers to the “conflict” between exploration and exploitation.\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "<a id=\"3\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"images/eq1.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "#### <span style=\"color:#2595bc\"> Which action to take( or which lever to pull )? </span>\n",
    "\n",
    "\n",
    "The simplest action selection rule is to select the action (or one of the actions)\n",
    "with <span style=\"color:#f31818\">highest estimated action value</span>, that is, to select at step $t$ one of the greedy\n",
    "actions, $ A_{t}^* $ , for which $$Q_{t}(A_{t}^*) = max_{a} Q_{t}(a)$$.\n",
    "\n",
    "All that the above equation says is that, at a given state $t$, the action $a$ will be our preferred choice i.e $a$ will be $A_{t}^*$ if the action value associated with $a$ , which is $Q_{t}(a)$ is the maximum among all action values for all possibilites of $a$.\n",
    "\n",
    "\n",
    "The above <i><span style=\"color:#f31818\">greedy action selection</span></i> method can be also be written mathematically as :- \n",
    "$$A_{t} = argmax_a Q_t(a)$$\n",
    "where $argmax_a$ denotes the value of $a$ at which the expression that follows is maximized\n",
    "(with ties broken arbitrarily)\n",
    "\n",
    "\n",
    "#### <span style=\"color:#2595bc\">Okay. But what about exploration?</span>\n",
    "\n",
    "Greedy action selection always exploits current knowledge to maximize immediate reward; it spends no time at all sampling apparently\n",
    "inferior actions to see if they might really be better. A simple alternative is to\n",
    "behave greedily most of the time, but every once in a while, say with small probability\n",
    "$\\varepsilon$, instead to select randomly from amongst all the actions with equal probability\n",
    "independently of the action-value estimates.We call methods using this near-greedy\n",
    "action selection rule <i><span style=\"color:#f31818\">$\\varepsilon-greedy$ methods</span></i>. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Import dependencies\n",
    "import numpy as np\n",
    "import random\n",
    "import matplotlib\n",
    "from matplotlib import pyplot as plt\n",
    "%matplotlib inline\n",
    "from IPython.display import Image\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a id=\"4\">Sample Reward Distribution</a>\n",
    "\n",
    "Sample Example - Let's see for a given 10-armed-Bandit problem what do the probability distribution of the rewards look like"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAGeCAYAAACU8HYuAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJztvXm8bGdVp/+sGs4599whISQMmbjQ\nKINhToBuQJGhGYxM0kpEBUUj3aiA0FGwuxV/dis0zdDYLc08gwjSoNJgkElaBpOQMCWtDAlJSEgC\nGe50zqmqvX5/7F331t2nhj28wz6115PPyT1DVe23du29vu8a3vWKqmIYhmEYndgDMAzDMJqBCYJh\nGIYBmCAYhmEYGSYIhmEYBmCCYBiGYWSYIBiGYRiACUJrEZG3icgf1Xj+mSJyUES6jsbzehH5j9n3\njxSRa1y8bvZ6jxCR/+fq9XKv/VQRuTo7Fw/w8Pp/ICLvyr4/7pyLyB1F5LMickBE/pukvFVEbhaR\nL7keS9MQkf8jIs+KPY5lwgTBMSJypYgcyW7c6zPDuyf2uMogIs8WkVH2Hg6KyHcyQ/Oj48eo6ndV\ndY+qjgq81ucWHVNVn6uq/5+j8auI3H3itf9eVe/h4rWn8ErgN7Jz8WVPxwCmnvPzgZuAfar6IuDh\nwGOB01X1wT7HMo3s2n9MgcfdVUQSEfmzEq99VBjHqOoTVPXtVcZqTMcEwQ8/rap7gPsDDwBeEmsg\nItKr+NTPZ+/hBOAxwBHgYhE5y9ngMlx5GZG4C/D1Kk908L7vAnxDj60uvQtwpaoeqjCWqtdJFX4J\nuBn4ORFZDXhcYxGqal8Ov4ArgcdM/PwK4G8mfl4lnVV+F/g+8HpgV/a3zwA/k33/MECBn8p+fjRw\nafb9vwA+CfyAdIb4buDE3Bh+B/gKsAn0SIXpEuAA8OfA+4A/mvEeng18bsrv/xr4QPb9/mx8vYnn\nfDt7/e8AzwTuBWwAI+AgcEv22LcBfwZ8FDhEKjhvG48HeCRwDfDS7P1dCTxzYhyfBn512niBz2bj\nOpQd8+fGrzfx+Htlr3ELqTF/0sTf3gb8D+BvsvfyReBfTDkXq9nrj4/1rYKvfdz7nvK6d82ugwPA\nhcCfAu/Kn/PstQbAVjaOX8+d65dlzzkXuDQbzz8A911wnZwKfBC4Mfscf2vi8X8AvB94Rza+rwNn\nZ397J5CQThwOAhfMuLYE+Bbwb0mv/6fn/v5j2fv+Yfb3lwKPz97nIHvty/LXAenk9j8AVwE3ZGM8\nIXfenkV6390E/N7EMR8MXATclh3zVbHtSDT7FXsAy/bFhCAApwNfBV478fdXAx8BTgL2An8F/HH2\ntz8EXpd9/9Lsxnn5xN9em31/d9LQwCpwCqkRfE1uDJcCZwC7gJXsRnkh0Aeent1cZQXhV4DvZ9+P\nb7IesDu7me6R/e3OwI/Nei1SY3Yrqeh1gDW2C8IQeFX2Hn+C1ICOX/+oIZh2jGxcd5/4+ZFkgpC9\n/29m53cFeBSpcbvHxNh+kBmJHqnYvm/O5330WAVf+7j3PeX1Pj/xvn88e/42QZh4vT+acx4eQGoc\nHwJ0SQ3ilcDqjOukA1wM/Kds/HcjFfnHZY//A1LReWL2en8MfGHatT/nfD2CVHxuB7wO+KuJv+0F\nrgNelF0Te4GHTBz7XbnXOnodkF6b38zGvAf4S+CdufP2xux93i8bw70mzvkvZt/vAR4a247E+rKQ\nkR/+t4gcAK4mvSF/H0BEhDTu+0JV/aGqHgD+C/CM7HmfITV+kBqDP574+Seyv6Oq31TVC1V1U1Vv\nJDUg48eN+e+qerWqHgEeSmqsXqOqA1X9APCPFd7X90iFbBoJcJaI7FLV61R1URjlw6r6f1U1UdWN\nGY/5j9l7/AzpjP1nK4w5z0NJb/o/UdUtVf0kqedz3sRjPqSqX1LVIakg3N/ha8983yJyJnAOx973\nZ0knDFU5H/hfqvpFVR1pGm/fzMY5ZvI6OQc4RVX/MBv/t0mN6DMmHv85Vf2opnmMd5Ia1zI8C/g/\nqnoz8B7g8SJyh+xv5wLXq+p/U9UNVT2gql8s+LrPJJ3Zf1tVD5KGaZ+RC4W9TFWPqOplwGUTYx8A\ndxeRk1X1oKp+oeR7WhpMEPzwFFXdSzozvSdwcvb7U4B10lj8LSJyC/Cx7PeQzlR+VETuSGqE3gGc\nISInk85YPwtHq0veJyLXishtwLsmjjHm6onvTwWu1WwKlHFVhfd1Gqkrfxyaxqx/DngucJ2I/I2I\n3HPBa1294O836/Gx8KtI30ddTgWuVtUk99qnTfx8/cT3h0mNvKvXnve+T2X6+67KXYAXja+17Ho7\ng+PP49W5x5+ae/xLgTtOPCZ/btaK5h9EZBfwb0hFFlX9PGkI5+ezh5xB6hVX4VSOP1dXkXp488Y+\n/lyfA/wocIWI/KOInFtxDDseEwSPZDPbt5HmDCCNXR4hDaecmH2doGnyFlU9TOqyPx/4mqpukcZ9\nf5s0Rn1T9jr/hdQFvo+q7gN+gTQ2e9zhJ76/Djgt81DGnFnhLT0V+PsZ7/XjqvpY0nDRFaQzy/w4\nZo1vGrcTkd0TP59J6qFAGj5an/jbnRa81iTfIxXZyWv/TODaEq9R57Xnve/rmP6+q3I18J8nrrUT\nVXVdVd87YzxXA9/JPX6vqj6x4PEWfaZPBfYB/zOrwLueVCzHpaNXk4Z8qrz290gFbcyZpGHH7y94\nHqr6z6p6HnAH4OXAB3KfQWswQfDPa4DHisj9spnjG4FXj91kETlNRB438fjPAL+R/QtpnHTyZ0hj\nqweBW0XkNODfLxjD50lvjt8Skb6IPI3U41iIiHSzMsHXkXo8L5vymDuKyJOzm2gzG9t4lvx94HQR\nWSlyvBwvE5EVEXkEaTjhL7LfXwo8TUTWs/LS5+Se931mG5Yvks4OL8jOxSOBnyZNstel1mur6lWk\nyc3x+3549vyqvBF4rog8JFujsFtEfkpE9s54/JeAAyLyOyKyK/vszxKRcwoeb955h9TwvwW4D6kH\nfH/SfMr9ROQ+pOG1O4vIC0RkVUT2ishDJl57f05sJ3kv8MLsWt1DOmn68yzsNxcR+QUROSW7P2/J\nfp3Me86yYoLgmSzG/w7SRB2kVR3fBL6QhXs+AUzWyH+G1OB/dsbPkBrlB5ImKP+GNIE2bwxbwNNI\nk44/JA3vzH0O8C9F5CBpsvjTpDO7c1T1q1Me2yH1Yr6Xvf5PkFaRQFoN9XXgehG5acpzZ3E9aWni\n90hDDM9V1Suyv72atOrk+8Dbs79P8gfA27Owx3F5h+xc/DTwBFKP7X8CvzTx2pVx9No/T5oE/iFp\n7ukdNcZzEfBrpJVKN5Ned8+e8/gRqfDen7TC6CbgTaSlx0X4Y+A/ZOf9xZN/yCYujybNY10/8XUx\nadj0WVlO7bGk5/B64J+Bn8xeYjwZ+IGIXDLl2G8hzWl8Nhv7BvCbBcf9eODr2fX+WuAZWU6ldcjx\nYWXDMAyjrZiHYBiGYQAmCIZhGEaGCYJhGIYBmCAYhmEYGSYIhmEYBmCCYBiGYWSYIBiGYRiACYJh\nGIaRYYJgGIZhACYIhmEYRoYJgmEYhgGYIBiGYRgZJgiGYRgGYIJgGIZhZJggGIZhGIAJgmEYhpFh\ngmAYhmEAJgiGYRhGhgmCYRiGAZggGIZhGBkmCIZhGAZggmAYhmFkmCAYhmEYgAmCYRiGkWGCYBiG\nYQAmCIZhGEaGCYJhGIYBmCAYhmEYGSYIhmEYBmCCYBiGYWSYIBiGYRiACYJhGIaR0Ys9gDKcfPLJ\nun///tjDMAzD2FFcfPHFN6nqKYset6MEYf/+/Vx00UWxh2EYhrGjEJGrijzOQkaGYRgGYIJgGIZh\nZJggGIZhGIAJgmEYhpFhgmAYhmEAJgiGYRhGhgmCYRiGAZggGIZhGBkmCIZhGAZggmAYhmFkmCAY\nhmEYgAmCYRiGkWGCYBgG3/3B4dhDMBqACYIRlVGiXH7dbVx+3W1ce8uR2MNpJapq594Adlj7a2P5\nGIwSrr05NUZbw4TTTtwVeUTtQxUUjT0MowGYh2BEZTBKpn5vhEOP/s9oOyYIRlSGo2OWaDAyqxQD\nVTvvRooJghGVQXLMKxgm5iHEQDEHwUgxQTCiMukhDM1DiIJq+mUYJghGVCZFYJQoSWKWKQaWVDbA\nBMGIzFYukZz/2fCPouYhGIAJghGZfN5gaB5CcNKyU8MwQTAik88bDM1DCE6aQzBJMEwQjMhYyCg+\nFjIyxpggGFEZDI8XAFuLEJ5EITFFMDBBMCKTFwALGYUnUfMQjBQTBCMq+XYV1r4iPGMxsJJfwwTB\niEaSKKOcEdoamlEKzTihbGfeiC4IItIVkS+LyF/HHosRlmkJZPMQwnPUQ7C4UeuJLgjA84HLYw/C\nCM8042+CEJ6xEJgeGFEFQUROB34KeFPMcRhxmFZRZFVG4UnMQzAyYnsIrwEuAGZOC0XkfBG5SEQu\nuvHGG8ONzPDONG/A1iGEZ7xa3LwzI5ogiMi5wA2qevG8x6nqG1T1bFU9+5RTTgk0OiMEW8MpIaNh\nYqtmAzPIEvnWbdaI6SE8DHiSiFwJvA94lIi8K+J4jMBsThEEMC8hNOM9KQa2H0XriSYIqvoSVT1d\nVfcDzwA+qaq/EGs8RnimeQjzfm/4YewZmIdgxM4hGC1mVszaBCEs48/BBMHoxR4AgKp+Gvh05GEY\ngZkVGrKQUVjGLcftvBvmIRjRsJBRMxj3j8qvGjfahwmCEQ0ThGYwXvthZaeGCYIRheEomTkjnVV9\nZPhhHCqykJFhgmBEYZ7xMcMUjuEoObonxZGtUeTRGLExQTCisDmYbfTn/c1wy+HBMRHYGIysBXbL\nMUEwojAvLLQ5tJlqKDYmvAJV2LBz32pMEIwozEscD0dqM9VAHM6FiSxs1G5MEIwoLPICLLEchrwg\n5H822oUJghGFRQbfSk/DcGQwmvuz0S5MEIwoLBIEyyOEIR8ispBRuzFBMKJgIaP4JImyMbCQkXEM\nEwQjCos9BBME3xyeEh46MhjafhQtxgTBCM5wlDBa0FnTQkb+Obgx3Pa7JIFD5iW0FhMEIzhFZv/m\nIfjnwMZg6u+nCYXRDkwQjOAUMfb52LbhngOb0w3/wc3pQmEsPyYIDWHWbG0ZKRIOMg/BP7M8gdvM\nQ2gtJggNQFW5+VB7BGGjQK+i0UiP9uk33LM5HM1c62Eho/ZigmAEp2jC2LwEf8wz+lvDxJL6LcUE\nwQhO0W6mlkfwx4EFXoB5Ce3EBMEITlFDbx6CPw7OSCgX/buxnJggGMEpauhNEPyxyOAv8iCM5cQE\nwQhKkmjhxnUWMvJDkiiHt+Yb/EPmIbQSEwQjKGVm/SYIfjgyGJEs+BgOb42shUULMUEwglLGyBcp\nTzXKU2T2P0rUWmG3EBMEIyhltmi07Rz9UDRhbInl9mGCYASlzKx/NFIGtjjNOUUN/aFNE+S2YYLQ\nAFRBaUe8tmxewCqN3FNcEMxDaBsmCEZQygqCJZbdkiRaeFc0Cxm1DxMEIyhlE8UmCG45PBhRtHjo\n8JZtltM2TBAaQlvuu7I9cqzSyC2L1h9MkiR2/ttGL/YADKJmD7aGCcOJovRd/S4i4uVYw1HCcMFO\naXnMQ3BL0T5SRx8/HLFrpetpNEbTMEFoOd+84SDfu+XI0Z8f/iMns9b3YwCqJIgtqewW89CMeVjI\nqCHE8hLyBqLsDLLcsaoIgnkILilr4O38twsThAYQM3GXN9I+DUCV8I9PgWojZT9f89DahQlCQ4gl\nCtsFoVkewiixxWkusSovYx4mCA1AiRMyShJlENBDqPraNkt1h3kIxjxMEBpAohql7HTaze4ziVg1\n/LNps1QnbA2ThV1O85iH0C5MEBpAKgbhFWHabNHnjLCqcbFZqhuqNAvcGia2OK1FmCA0hBj33DRv\nwOdsvKphN0FwQ9k1IJBel4MKzzN2JiYIDSBtbheeaTuXbXpM4FZNDltS2Q3DsvGijMQ8hNZggtAA\nFI1y021NMbSjkZIk7scyHCWVvaCiW24a86moB4w8XA9GMzFBaACJxgkZzTK004SiLsMaRqXOc41j\njCpeZFWfZ+w8TBAagEaqMpoVivEhCHVec2ghIydU9fx8eIxGMzFBaACJxonTzjLS+bUJLqiS0Bzj\nQ6DaSNXQj4WM2oMJQgOI5iGEDBnV8hD8npytYcJXr7nV6zGaQOWQkQlCazBBaACxPIRZFUWDofux\n1BEZ31VGiWormrhVDf1YDqE9mCA0gFGiwW+6JFFGM2beWyP3xrHOLF/Vbx4hVtlvaKom581DaA8m\nCA0gDRmFvenmzdi3PHgIVWvgjz3f3/lR/JTaNo2ql5g5CO3BBKEBpCGjsMecZ2B9hGjqvj+fIbUY\n598wmogJQgMYqQZ3y+eFYHzMxusadJ+nR1XRiEGj4Sjh4KZtaG/ExwShASRJ+JXK8/rTePEQar6k\nT2MZa2HgmB8e2uIL3/qBLcAzomOC0AA0gkGaF9P3UeZZdwbu1VZGFoRB9uZ8l9caxiJ6sQdgxAkZ\nzSstrZsAnkbe4L7i41fMffwFj7tn7gUcD2iCROP0khozDt8NkoRddL0dR8TbSxtLgnkIDWCUJOEF\nYYGH4DpEU/fl/CaV4wrCOHzXVA/BhKQ9mIfQAIZJ6iGoKhLo7ltkfIaJ0u+6G0ve4G7zABbgOWIU\ndR3C2CPz4ZkZRhnMQ2gAY+McMqm4KHHsOrFc95359hBiVviMP3/biMaIjQlCAxiHi0KGjRaJj2vj\nVNfg+rTXSZJ+xRKFsfjOWjkeG6uGbQ8mCA1gHCoIuTNYaA+hbiis4zGSNm4bEqvqcyzOvru6mmE3\nFmGC0ADGIYOQHsKiFteuBaGuQe94zK0kETy0SY56CLYOwYhMNEEQkTNE5FMi8g0R+bqIPD/WWGIz\nNgQhcwiLZqOuO57WNeheBeGohxBLEMY5BEsqG3GJWWU0BF6kqpeIyF7gYhG5UFW/EXFMURgGnqGq\n6sIqo3llqVWoa9DF49RlfNpjCIKqHvXWNm3vaCMy0TwEVb1OVS/Jvj8AXA6cFms8MQmdQyiSMHYe\nMqp5pfn0EGIk9cdMemqz9rg2jFA0IocgIvuBBwBfnPK380XkIhG56MYbbww9NO+MEj3a5yfUwqQi\nxr55ISNHA5lCEjGpPCkC3pPKrdj1wahDdEEQkT3AB4EXqOpt+b+r6htU9WxVPfuUU04JP0DPTBrn\ncB7C4uO4Nk6NTiqPBSGGhzAhCINh0siOpw0ckuGJqIIgIn1SMXi3qv5lzLHE4riQQSBBKHKcppWd\n+lzAfTRkFMHy5cN3oa4Bozkc2hxy/a0bHNwcxh5K1CojAd4MXK6qr4o1jthMln+GWqm6OVhsdDYG\nbrfRrDvD73pUhJi9hPJ5A595hKp6F7PPUxu44cAmX7v2Vq6/dSP2UKJ6CA8DfhF4lIhcmn09MeJ4\nojApAqGSikU2lB+O3HZg7dWIGYlA12MSYXzeYyR18/tX+5wUVP08TRD8Mkqasw4lWtmpqn4OaH0f\nxeNiyIHCBRsFPARIx7ZrxU075n63+tyj2xGvTf/G5z1vnEOQLzX1KUpVDXsD7NRS06R1KNGTym0n\nRg6haL17EU+iKL0anVPriEkRtiKuA9iWQ/AqCFWfZ4rgkxjNLWdhghCZyVnBaKRBKl2KGnqXBrJf\nYyFCnXDTIkbJsdBYjG6j23IIHicFVQ17E0IZy8zR9ufmIRh5NzGEl1DYQygYWipCHQ+h59FDOG4d\nQBQPIVzIyHIIzWQYoXXNLEwQIpM3CL7jiMNRUrjNssuQUZ2ksE8PIbYg5I/p8/OvathND/wy/syb\nsGOe7ZhGuiBppOo9Vj2NkEnFacebR9HkcxHqnNs63sUiNicSyaGTepPhqjE+PcRJw15mT2sLGfnl\naMiyATvmmYcA3HpkwDdvOBjl2Pm4te84dpn1BRuOPYSqaQSfQj15vkeJBo3jThP/RW3J61DVsMdY\nsNcmJjdIir1S3TwEUlc6xixostPlGN8ewpESgnBky20ZZq/TYavCLChUyAjSGbrPnEX+WHk2PQrS\npGEvs6d1jJYebWE4Spi8JbZGCas9N6XeVTAPgfRGiSHM08I3vmvhy4SBtoaJU6Fc6VW73Fb7/m6Q\nvOi5FsF5TBN/n5VmVbfotJCRP/KJ5Nh5hIUegoisAj8D7J98vKr+ob9hhUU1TiXFtJj1luMuo3nK\ntqTYGIzYverGkVztdagSmFvxOGM/vDXM/Tzi9t6OdjyzchZbo4S1jlsRrGPUTRD8kfcSYy9OK3Kn\nfxi4FbgY2PQ7nDgkqlHipNNmiL7LTsuEjMCtIFT3EPwJQr6h2KGtcA3G5gqCY69oWCNh6fveGPfx\n2X/yOne/w16vx2oa20LGO0AQTlfVx3sfSURGSZxkzrQEsu8cQlkPoayAzKNqbNSXh7A1TLa56Ic2\nw4WMZhUQ+Agb1Clg8V0fPzaCvr3jJhK6qGQRRe60fxCR+3gfSUTSkFH4406tMvFZh55o6cVmLrue\nrlb1ECo+bxH5cNGs3/li1qy9zmy+7LGK4DupPGxQL5/QbFuHFHnXvCIewsOBZ4vId0hDRgKoqt7X\n68gCEqvKaFoC2eVisDxVykhdrkWoYthXeh1vje2m9Z/fHCQMRkmQNSmzPAEf12IdjVFNRaHjqdrr\n6MKsBtThh2Yn5hCe4H0UkRklGiWpPM3YJgneDFIV4+7SQ6iSQ6iadyjC4RkVRYc3R5yw7l8QZhl+\nHyGjusZ2mCgrngWhjSGjfJQgRoPFSRZe9ap6FXAi8NPZ14nZ75aGROMsz5/14fu6KKoYd7ceQvkc\ngq9wEaQ7VU39faCw0eyQkfuLsW5i2JcHraoT3T7b5yHk70mfEYIiLLzbROT5wLuBO2Rf7xKR3/Q9\nsJDEChnN+vA3He9Wdux45W+4zeHIWcK9inH3uUhnpocQShBmJpXdG8a6ttaXBz0ZMokdLolBfsLl\ncgJWhSIho+cAD1HVQwAi8nLg88DrfA4sJMORRpmd7AQPQTUdj4syyE5HWO13SiW2XW3Qk2eU6MxF\naKEqjWZ5Aj48hLoG3Vfp6aQoJkkqhqFWijeBfF5vPAHzuSHUPIqceQEmRz1iyXY6Gybp8vGQS/Tn\ndR31JQhVX9fleHaVFJZ1T4IwLyw0K5TkmlmG34e3Wvc11dN8KeR+EE1jMMUGJEncPEIRD+GtwBdF\n5EPZz08B3uxvSOGZ7EfuK3GWZ96F72stQtUE8eZgBLv6Tsawa6XLLYcHhR/veoHWmIMbs43+4a0R\no0S97uMMx/bSzeMjdFJ3gu/LQ8gbv81BwvqKl0M1jnkRAl/X/SIWCoKqvkpEPk1afgrwy6r6Za+j\nCsx49hQyjzAvbOIrsVRVEFzGNct6CGUfX5RpJaf5v5/gSASnkSQ6M67vxUOoadB95RC2J1Xb4yHM\nuh9dTsDKMlMQRGSfqt4mIicBV2Zf47+dpKo/9D+8MBxf5RBGmedd+D5uirS1c7Wb2mUb7PWV4m0w\nel3xVnYaWxDm5QmamEPwFU7d5iFErrIJySxBiJlYnnd3vgc4l7SH0eTVINnPd/M4rqBE8RDmXPgu\na//H1AlDuAxhlZnx+/IOYH7IqMjf6zLPQPswvnVf09etkb8P2uUhTH+vLtvFlGWmIKjqudm/dw03\nnDgc3eQ6oCDMq2TZHCTOY9h1knUuz8vaSvEZv68Ko83haKHIHdwsnueoQngPod7zQ+UQfEyGmkrs\ndTDTKLIO4e+K/G4nE8NDODKY/6G7roWvs/rVZV38aq9bWOi85Q8KzP4PePYQ5u1N4KV1RUNDRm3O\nIcwy/IcDNljMM1MQRGQtyx+cLCK3E5GTsq/9wGmhBuibtNNp+n3IhTGzFkWNcb1RSx2j7roDY9GZ\nvy8PYVH+AFIB9TlbnbfuJVb33Xn4KItX1e2tGyIvzApFMmcdzMZgFHQr10nm5RB+HXgBcCpwycTv\nbwP+1OegQjIpAqF2KxqOkoUX/iLBKMugxgzP9aK93Su9QrP03SUS0GUoOvs/uDn0Vv63KAQzSpRe\n150VrmvQfSyU2hgk28phN4cjr430msLhwWhuKfChrREn7Aq/QG/mEVX1tVn+4MWqeteJr/up6lIK\nQigPoUjSyHUcsU5bXdfnpejMf33V06K0ggvPfCaWF4WFXOcRpIFrSadd46pxY+ihOLzgGgzZhn2S\nIlOwW0Xkl/K/VNV3eBhPcCbDIaE2pygy+3ceMqrVD99t++PdBQx9tyte+hipamGDUyS0VJVF3qjr\nuv/aHoKbYRzHLGE+vDVi71qcOvxQLLq2Qq2Wz1NEEM6Z+H4NeDRpCGlJBCG8h1BEEJyHjGqKnct9\nfousRfAVLtoYJIUbvfks/1vkAcTeOSuPjxzCrEq7g5tD7uj+cI1i0f0dcue+SYqsVD6us6mInAi8\nz9uIAjOZ1AonCIvVf2vodqOWupUrLmesuwuEjHz1MCrjivucpS261ly3L6lrz32EnGZ9FjGrbELR\nVA+hirU5BCzN2oTJGzNUY62is3+XXkJde+4ygtHrdhauQN696sdDKHNOhyP1NklYZPBdX4uRmmfO\nZZZRXPYcgqounJgcGYyitORfeNeJyF9xbKVyF7gX8H6fgwpJjBxC0dj0IYftE5TmeAiQ5hHmGcUi\nXkQVyoaBDnuq9lhUb+++wWE9RXAtKJvD0cw8yuGtYdQW0L45MhgtDFuOk+v7AudSikzDXjnx/RC4\nSlWv8TSe4Bxfdpp4vxCPbI3mLkqaxGVSs7aH4GYYR1lf6XHzodmrgX2tQSjrdR3ZGnnpabTI4Lv2\nTOqXnboZx5h5YaEkSY1mmb5XO4mi1WuHNhsoCKr6GRG5E/BgUrvwLe+jCsika66aJvv6Duu/85Qx\n8k4FIffzKz5+xdzHX/C4ex7/fMeKsChH4MsYlC3n8xW+WBQScu0hrNTMRdV9fp6FzQU3hssrCCUi\nBKEp0rriV4EvAU8Dng58QUR+xffAQpF3W30nlksJgsM6+NorXx0LwjwPYLXf8bIXgWr51ceuy3/H\n41i0LsR1C4e6XWNdd5299cj8XlG3eW4dEpOiFUQHIyTXi0jwvwceoKo/ABCR2wP/ALzF58BCkReA\nwVDB4wYdZYz81jBhczhyUo8o59ryAAAgAElEQVSft+d5D2Dx890qwrzZn68KozIlp2Ncl/9CsYSx\n64lJ3Wo1V9VuYxatFj+w4be5YEyKTgp9d9ydRpFP+QfAgYmfD2S/WwryN6fvSqOyYSBX9chNqjKC\n+Y3rdvWbES6q+pxFFAkHOQ8Z1Zjhi7gVhFGyuMpmWT2EpMB7HxOjp9G8DXJ+O/v2m6RbaH6YdKL5\nZOArAcbmnSTRbQlenyGjMhfDmIMbQ07a7cJlqbmnroMRTNLtCKv9ztSeTk1JKMOx0lOXBrGIsR+M\nEqerw+vkxVyHiw5uDBdOMAbDhI3BKNpWkr44tLX4vR/3+M0RJ6yH62k0byq2N/v3WxyfSP6wv+GE\nZZo34FMQyl4MAAcc9eWvWznlo9fY+kp3qiD4ChlV9f42h24FoYgwqaY71blKrNZJCrsOF91WMBx0\nYMNfc8FYlM1JHdoacsJ6uEqjeRvkvCzYKCIxzfj7FIQqVUOuQkadmoLgoxR3V7/HzWw3Dr48hKph\nmK1hAqvuxlF0LcShTXeCICL0e51KTQ5D5w/G3LYx4JS9Dk98Ayi7Dib0hkHzQkavUdUX5BamHUVV\nn+R1ZAGYthBta+hvcVqVkIWrGHZdc+7DQ5hl+Nc9zQprCYJDil4Hriuc+l1hwb5MU1l1HDIq4yEs\nG2UFIfR2mvOmH+/M/n3lnMfsaEJ7CFVucFcx7LplnHU9jGlMCw31ex16jmekY6qGjNwLQjFDd7iK\n9Z7Daq9bqU+QyxxCmTxajCob38zaR3n24xsiCKp6sYh0gfNV9ZkBxxSMaTe6T0Go+uFuDEa1BaGu\nPfchCNPiw762zYQaHoLDa6LMWgjXJa+7V7vcfKjK89xVfR0u0LZhzMZg5DyhH5uyk8KyAlKXuWda\nVUfAXUTEY2V+PKYZf59lp1U/XBduY91ulT66eUwLRbgOT0zShJBRqfbbrgWhYj7CZV+psrP+WF0/\nfVF2UrgxGAXdTrXIFfJt4P+KyEdIO50CoKqv8jaqQEzLIfhqcJckyuawooewVd8gNTFktNrrIHL8\nGgdvW1YmWrl7pMtJQpmc0MbA7XaSVWf6LltIlC2sOLAx5MT15ZiPbg2T0tegalrlFqraqsgnPS47\n7XCsFLVZu3dUZGoOYeinwd3mcPv+sUXZqCgkk9QPGdUewjZE0l3RJmdNvjyEOrN8lx5CmTCQauod\nugrZVCnn7fcWtyovQ1lB8LlrXWiqevpHtsKtxyhypX1DVf9i8hci8m88jScos2Z+g5Gy0nNrAeuE\nfVyEDup6CD56C0Hat2hSEHxd+E0RhCrtt10Jwlq/S7crhbvtAuxxvK91m0NGlSMEDiaERSki/S8p\n+Lsdx6yabB+J5VqC4CCH0O9Un+V1u+KtJfhark+TLw9hc1T9HG7VeG6esgbOdR5hT0lxcRkuGoyS\n0jH0A0skCIv20Xb9vCrMW4fwBOCJwGki8t8n/rSPdF+EHc+sfIEXQahxY7sQhG6N1gU9T94BpB7C\nvJ9dUWf3qSTBWSy/ykpVl6yvdLn1cPHV72UFZB5VZvujkXJka+RtsWJIKgtCwJ3T5n3a3wMuAp4E\nXDzx+wPAC30OKhSzDL+PSqM6IjMaaW2D1K/x3F4N72IReQ8h/7Mr6t5TiSqdmpVaSaKVQkYuKVtp\n5LKNSOUY+mBJBKFsq92MUcXnVWHeOoTLgMtE5D2qOgAQkdsBZ6jqzaEG6It5VSc+Ko1ib0dQJwfg\nc8OgSY+g3+s4q6jJk9RUhJFqoYTbPI4MRqWvA9fdVveslXsXLtcgVC27rhp7bxpVvdSQHkKRqd+F\nIrJPRE4CLgHeKCKv9jwu78ybsVfp97KIunsS131+ndW/vhLKcHyfHJ/CU3fDchel4FXCP5uDxGkL\n5L0lBGGl13Ga5K++MDPs4ixfVJ1ohswhFLESJ6jqbaQ7pr1DVR8CPNrvsPwzLyzkI4dQ16DUFYQ6\nxtbnStHJcbnepnGSuuevrqBA9TySy342q71u4TzNPsd7SddZqb8MLIuH0BOROwM/C/y15/EEY54X\n4COHUNcg1RWUOrP8cB5CcwWh7vOheuda13mEohu3l/EmilB1W1DfgnDNzYf5zk2HuPaWI16PsxNy\nCEXuwD8EPg58U1X/UUTuBvyz32H5Z5775iOHENsg1Sk79WmoV4IJQs3nO7gnj1RsVudaEIoa+qLC\nUZSmhoy+c9MhvnXDQa66qUKjpxJU9RB8dU+YxsIrI1uU9hcTP38b+BmfgwrBvMVGrrtbwvak8Cs+\nfsXcx+f3PK47Qe10qvfD99lfqNMRuh1hlGijcwgjBx5CVcPuenFW0VCQSw9hOEoqx8J9J5XHIRnf\noZmd0N5h3jqEC1T1FSLyOqbvh/BbXkfmmXkXmY8LsG6DKhchi5VuNUFwvYVinn63wyip39F1HnUF\noX7IT6fuDleEqqGWWRQx9K4TynXew3CkDEeJl7boqse20a0a0lkm5l0Zl2f/XhRiIKGZd4Fueehn\nlLdHeQ+g7POrsNLrcGizwvM8tx/udQUGadmpL2on9Wt+AHXcftdFDuPE8jyBcp1QrlspM0wUH0tU\nJr0ClwsQdyrz1iH8Vfbv230dXEQeD7wW6AJvUtU/8XWsPPMEQTVNLK96WiQVi6qhnxAeQvqvvxtR\nazrsdfW4zuzTR9nhvrU+Nw5mzw5cJ5Trzr5dVHkVed1hoqy0WBDm3uki8iwRuUREDmVfF4nIL7k4\ncLb5zv8AngDcGzhPRO7t4rWLsCgs5NpNr1up46J9RFXD7lsQxh5IncT3ImLvB1EnL+WjDHqRwXct\nCHVzML7i+/lz2/aw0bwcwrOAFwC/TbogTYAHAv9VRFRV3znruQV5MGnl0rez470PeDLwjZqvW4hF\nN6jrxHJdg+6i9LNK6KfT8Vv9A8feW51+S4uoa9DrCkqdkNF4Vb3L8t9FK5ZdVxjVTuoH9BDazLw7\n/d8CT1XVT6nqrap6i6p+krTC6HkOjn0acPXEz9dkvzsOETk/80wuuvHGGx0cNr0IFrnh5iFkz+n6\nD5uNz43PJnq1BaHm8+vO8l17CfMMfq8rztuQ1w17+RKEvFCHXBXcROZZiH2qemX+l9nv9vka0JTj\nvUFVz1bVs0855RQnr1mkimjT8WKYug3iXMwOq+QQfHUfnWTsgPjYlW1M7ZBRzeM3TRDW+t00mT+F\nvY69A2jGSvEir2sho9nMW7bnYknftcAZEz+fnv3OO0XCQa5XK9daKexoP4IqHSN9bno/ppuJpc8V\n0fnTV3YdSF2xqru4yMfipL1rPW4+tL0Vtuv8AdQPxbhYBzKNvAD4Eh6oP6kIwbxP/l4i8pUpvxfg\nbg6O/Y/Aj4jIXUmF4BnAzzt43YUUCQdVrRmfRZ1wiKtQSpXW0mshPITM2HZ9eggWMtrG3rX+VEFw\nuQfCmNrdZj2FcvIhoiaGjEIKyVxB8HlgVR2KyG+QtsXoAm9R1a/7POaYIsbedf+UWe55EVzNnDsd\nWVh/nifEXq6dTmpwfdZ/52f4ZdeB1A05NTGpOsvw+/AQ6o6+btnwLPKei8+kctVJha/dCqcxbx3C\nVb4PrqofBT7q+zh5iuxRuuG8yqgZvYTW+t1SghAmZCRew0XgYJYV2cPwkV+ZZvg7nfKb6Oxk8kLr\nt5Fctc8woB4Uam63dBRpQzwYJk5nZU3pNlrWwIfwEIIIQm2DXPP4kddBTGP3So/8PGV9pdeqlbr5\nUJzPRnKVPQS3w5hLe6YCExQNB20MRs52jKqzuMtl64iyBj6IIIh4zR9Afbe77vPzhrdsUtvH6el0\nhF393nHN83zkD5rMdg+heTmEkLTSQygaDnKZR6jTMdRlYrfMa630Ot5n7pCGQ3zHSeuugq6b2K8b\n8qnrYcwiX3kWYgLQJPJVRj6S92OqfoIhQ0bzVip/lTm5IFW9r5cReWaUaOGOny7zCHVm+S4Xh5UJ\nGYXa2FykfkhmEXWS+lDfS8sfvWxS29f5yV8P60uwmX0Zti1M85pUriwJTscxj3n+4bnZv+NVyeNW\nFc/0Nxz/lJn1V93ycBq19iNw6CGsl0gYhkgoQ3qj+PYQ6ghCp1O/Aqp+2asnDyH3GYf6zJvCthyC\nh71QxlRvgd+ADXLGVUYi8lhVfcDEn35XRC4Bftf34HxQRhBcl56uVhQEtzmEDp1OsR3AXOVPFtEJ\n4CHUOYcuqrxq5zBqj2A6ayvHv7dQXmFTyAuCj+1zx1R1PkKmNYpc6SIiD5v44V8VfF4jKbNhueuN\ncqomll16CCJS2EvYHcg4hMgh1NlcpW7bEahv0H219Zi8Fjodv7vj1cFHDmWU6LaJ0XCktTezmkXV\n1w2Z6C5iGX4FeKuInJD9fEv2ux1Jmf1ZXe/lWnk/AsfdRtdXuhzcWLwt43ogD0HEf+KsTlLYxT4N\ntZPKnuz0ZIhordf1Jsy1y3Y9DGtWAnkwUlZ6fgRokqKVZi52SyzK3DteRDrA3VX1fmNBUNVbg4zM\nE2VDRi53TqsiCN2OON86MJ0VLt46bT1QPLkj4rWxHdQL+7g4/3WT2r72iuh2hJVeh61hwppXj7B5\n6zBmhYcGo8TLHiBV+zGF7Lc3VxBUNRGRC4D373QhGFMmDKSa9j1yVYpXZQc2Hy787tXF49i10g22\nQClElVG3I4VzJ3lc9JKqm4eoKyjz2LXSZWuYeK0wamJSfVY+z1fpaV4PilaaNcZDyPiEiLwY+HPg\n0PiXqvpDb6PySNkw0ObAoSBUyAWsepilF8khhCw/7HU6nLxnNchxtioogoukcl1R8blXxAPPvB2J\nqtfFgfXXYbhn1qpk15tjjamaC/DV6XUaRQTh57J/JzfFUdx0PA2KqpZOFKePd9MfvkpJn48ywCLG\nPlSFEaSz91NP3OX9OL2usLU4dTL1efWPXV1UXLU/n/n6HaHruda9iUn1WeFj15tjjak60/eV5J7G\nwrteVe8aYiAh2BolpUMGLhPLVTwNH2WA/W7naNx4Fsu4QGmt3+XwZvnKMRceYp3EtM+9pkMRu/34\nNGYZftfl5mOqCkKS4DSXOY9C00AROQu4N7A2/p2qvsPXoHxRxbgX6YxalH63Q68rpXqu+9qPYPdq\nd64gLGPHyyr7QaTPq/8Z1GkB4jN/EIrYO9ZNI6SHMK3EtQy+Kp/yLLzrReT3gUeSCsJHgScAnwN2\nnCBU2RbT9Wxhrd/l4Kh43MLXytH1lem7ZR39e4HE806j6noOJx5CjVm+z/xBKGpPbj2cgln3tg8P\noW6iepgkrARY/lXkCE8HHg1cr6q/DNwPOGH+U5pJJQ/B8VqEprSfnucBdLtSqSKq6VQ9ly4qvTo1\nWny7LjveifhYmDbLE/DlIdTBZ4+lSYrEBY5k5adDEdkH3MDxeyHvGKqEf1zPFsrkBHyuHJ3nASxj\nuAiqhX56XXfrQHpdqWQYlsFDyJcwx27/nSQ6M2S6OXS7/gjqb80ZamvPInf+RSJyIvBG4GLgIPB5\nr6PyRJV9kreGCUmizmryy8Sxfa4cnWf0lzGhDNU8BJceWq/TYZMKvawa2k6iDE2LGM3zApIkLUBx\n6SUPaq4uG3rssTRJkSqjf5d9+3oR+RiwT1W/4ndYfqjam2hrlLDWcXNx5JuJzcPHGoSj45jT5C5k\nyWlIYgvCSk84tHiB+DaWwUPIz2tK72nteGK0qKfZxpZbQag7wx8EChkttE4i8k4R+TURuaeqXrlT\nxQCqLzhxGVNsSvvpeU3uQjW1C003a0FeBpdVXlWb5LncUzsWTasyOrxgQcrhQYUFK3OonVQO5CEU\nudLeAtwZeJ2IfFtEPigiz/c8Li9UbW3rcil7GSNfpMVEHWaFhpa5BXLZPELVUtVpVDXsSxEyatg6\nhEV7nRx2uBcK1E8K+9zreZIiIaNPichngXOAnwSeC/wY8FrPY3NKkmhlt83lUvZuR1jtdwrlM3wb\n5lnitMybpKz1uxwo0Ol18vGuqFpHvhQho9rPd3sOFhl8l5tjgYvVxg0RBBH5O2A3aSL574FzVPUG\n3wNzTZ2NL1w3u1pf6bE52Cr0OJ9MM3b9XmepyxzLiqxLcWyzh1DXnrtu/71IEA5tug0Z7RSKnOav\nAFvAWcB9gbNExH/jGcfUEQTXza6KVvH4nqlPM47L7B1A+ffn0kurKgjLkENoUnM7VeXIghzBYU/t\nK5pOkZDRCwFEZC/wbOCtwJ0A/+0pHVJnr1TXC1WKCMJav1ur3UERphlHX60ymkKZktpeV5zOzqu2\noFgGQagdMnKYRNgcLu5pNhqljTBdVRqF6EPkgiIho98AHgE8CLiSNMn8936H5Z4mhYyKzDpDJHan\nhYyW3UMoE4ZzHbKrsvNdp1OvD1JTqGsQXZ6CouGgI1vuBKE+Ya6BIlf8GvAq4GJV3bGBtTphH/ch\no2bsRzC5W9YYX60ymsK89Rd5XH8G1TZIWo7Po0lJ5aIVRIe2Rpy47uywO4KFUxZVfSXphgC/CCAi\np4jIjmuJXWeW77rkq8gsPNRq4bwnsswlp5DOVIuKnutzUaUNSVM3vS9Lk8pODxXcFONwCxPLRRam\n/T7wO8BLsl/1gXf5HJQP6hh11yGjbmexUQplmPN19svuIUDxUJDrnk6dSgvjluPzaFIM/VDBPTEO\nmiBM5anAk8i2z1TV7wF7fQ7KB3X2JVUPNcCLDH6oBnP5pGmVOPdOI2aVV9kZv3kIKS53TCuaQ3C5\nOK3uWpI6GyyVocjVtqXpqgoFEJHdfofkhzrrQmr2pZrKvFXIIuGSu5OCIBLuwotJ0XPrY0+IsjP+\nZckh1DWIrhLrg1FSOCd4ZGtUu231mLoh4FD2oIggvF9E/hdwooj8GvAJ4E1+h+WeOh4CpCudXbLe\nn+0BrPW7zrqrLmJSAPrdTqNce18UuTl7XfFS7lnaQ1iSMuA6M3wRd4JQdsFZ0XzDIuqGgEOFkIus\nQ3iliDwWuA24B/CfVPVC7yNzTF17nqjScVjpMO8DDpnYnfQQlmJFbAGKVXn5CdmVFQSXvZRiUsdD\ncFl2e6hkGOjw5oh9a/3ax01b2VePVITyEApd9ZkAXAggIh0ReaaqvtvryBxT10NwnUWYFzIKuUHN\nZM5gGRZAFWGt31l4c/qq8irb0nxZPIQ6Rt2pIJT0EFwlljtZIUmVHkkh28nMtDzZ7mjPA04DPkIq\nCM8DXgxcBuwoQcg3lyq7Y9MoUVyK9LwZQ8gNaia9gmVJYC5CRNjV785NGvry0sp2W12WJL9IuoVo\nlZi8S0Eomyhe1Ca7DFUFIaQ9mDcVfSdwM2lTu18FXkq6vuQpqnppgLE5pXazQcd0OrONUsiQUb+F\nHgKk53iuIHhy0ct8tqv9TrBcUggqC4LDvFbZtQUuK4129bvcXPF5oZgnCHdT1fsAiMibgOuAM1V1\nI8jIHJO/Dsvu2FQ35DSNWUYp5Iyg3z22crctOQRYbJh9fQZlYsm+u92GpupMv2oPqDxpU7tyBt5l\nG+yq11TICeI8CzAYf6OqI+CanSoG4KDKyIOHMW2rypAlp2O62U5ey9B3vyjzqrzA30049gyL4HuD\npNBUFYRuxZ3m8hwZjEpHCkaJsuGo82nVMuaQOcV5R7qfiNyWfS/AruxnAVRV93kf3ZIzrYJkrd8N\nXvrZ6wgDlqOJWlHmGfxuR7zW/6+v9gqFIhaJ1k5jtdfhYIXnucqjVA3/HN4aOVkxfsKuatVKVZ9X\nhZlXnKou1fSkbnzcx4KttZXtY4rRqmAsBG3yEGKW/RYNHfhYGBeTqiFJV5VWhwu2rNj2vK0hJ+1e\nqX381V6X9QW5q23P6XcaEzJaKuoa9L4jt3WSaaGDGO2nx0LQJg9hfc559v0ZFBaEJWs0WNXrcuYh\nLNgUZxYu8wj7Ss72Q3oH0CpBqP5Wu13xUu0xzRuIsUFNp4WC0Mn2tp6Gb0NcJFnc6Szf3hRVy5qd\neQg1QkauOHG9nIE/cVd9z6QMJggF8FUL3u92tlVQxGg/3UYPAWYbft9hu6I75i1bG5HKIaOum8+j\nanLYVVIZys/4zUPwRB2j7jO2njc+MWaFx3IIrbkcgNkhDN+iXGR71JCVJaGo6iG4Koeu2gK/zm6L\nefas9ugWDF93OrB3Lex10BoLUKeWuWwP+zLkBSBGUnksBG3zEGad6xCfwSIvYdnyB1DDQ3Bw/6lq\n5X3VXe6HIiKFZ/371vrBFya2RhCaGDKC42ejnU6c9hHjt9c2QZjlCZRtL1GFRXmE9SlrVHY6VZLK\nPUf5uzqz/CSBoUNROLGgIJTNN7igNYJQx6j7bOkwuRZhtRcnbjxuTdw2QZhm+HtdCdJIbFFJ6e4l\n9BC6FXaMcxVCrbsNrsuwUWEPIXD+AFokCP1edWPnaun8NJrQfrojQsvSB8CsKq8whnhRjmBZ97Yu\n6325+jyKboozi8HQXauCoobeRdvtsrTGDNTxEHyGjI4ThEjN5bodcbpF4U4hpiDMM/i9rt+V0jEp\nK3SuhLFuHmBz5K7SqN/tLPQQ1/rdKPnE1ghCr9upnBz2meDL71gWAxG3e9buFKaFMEKtA5kXElq2\npnaTlDVyrjYIqu0h1Aw55Vk0+9+3K8410BpBgOpx2WlN6FzRhJBRtyOtyx+MyYcwQu1Q1ut2oi2M\ni0nZnMC09i5VqLs3sustdBflEUKvPxjTKkGo4n76rvyZbIkRK2TUEaGFDgKwfcYa0k2fZfh9TkBi\nU3bVsavPo645d939/oQFFUQmCAGo4orv6ve8Vv50OnI0aR0zqexyE5KdxHZBCPcZzLoezUM4xrLs\nKZ1nz0pvplcuAnsjJJShdYJQ/uIKcXOOhcBHR9UinLje56zTTohy7NjkRTikKM+6tpZZEMrM+Lsd\ncfZ55LfQLf18x7uqdzoycxXyntXZYuGbVglClZBREEHIQkWxPIR+t7PUYYp5bBOEgGG7WbPlGNUl\noeh3O4VbN7hqagfNCxnB7NBgzHuxVVZgXsvjWYSoB9+3q88w0VZtYdkUJgWg2wmzKG3MtNLSbkeW\nfm/r1V6HwwXKOJdZGGH2hCDmGpRWCUKv22Gl1ylVghaiBPBH77jX+zGM6UwKQmhDPG0G7HJW3FTW\n+t1Cm9W4zB/UneF7cBAaGTJc/qsvR9mTvczxXCNu2e9qr7OtumtZF6RNUtTQu03w18wheIgZzfIE\nYm6dGkUQROS/isgVIvIVEfmQiJwY6thlZvyxms0Z4YgpCCLbk6YxNkgKTdH36DZkVC9J66PScGYO\nydHaiyrEOvKFwFmqel/gn4CXhDpwmfic75JTIz6Ti/JirAPJG71WeAgFDb1LQah7G/uwAuMQ9iTd\nyG1LogiCqv6tqo43OP0CcHqoY5epg17WBmPG8YxvyhhJ/bwH2g4PIXzIqG5rFl/zwryNib1tahOS\nyr8C/Hmog5Ux8pY/aAcrvQ5HtkbmIQSiqKF3eS7y9vwVH79i7uMveNw9c8/3owi7+l1uZXD059g2\nx5sgiMgngDtN+dPvqeqHs8f8HjAE3j3ndc4Hzgc488wza4+rzAmPrdZGGGKuA8l7CG2oMipi6Htd\nt/21aoeMPHkIeXGMPSHwJgiq+ph5fxeRZwPnAo/WOSl8VX0D8AaAs88+u3aqf7yx/bBA90ILGbWD\no3tKR1gpni91jdXPKiTdrF3LvHvQtWHMz/DzHsDC53u6NLq5jUhiN5mMVWX0eOAC4Emqejj08YvO\n/M1DaAfjmzBGP6e8CC37orQxiwy+a0+pvofg59rodfKffwsFAfhTYC9woYhcKiKvD3nwoqWnJgjt\nYCwIoTc0h+O73YrEnyGGYpHBd93UrolVRrB9QhD784+SVFbVu8c47phdBep81/rdKAbCCE/MPaUn\nDULIthmxWbS+p2m5FH8ho+NfuBd5L9tmnfVA7CrgIVj+oD1EDRlNGIB+iyYgC0NGDVsQ6qO5HWwX\ngNgeQrPOeiCKbPTdhnpwI6V3NGQU4dgTHkJsYxCSRfeX66SyL4Nel+05JBOE4KwWyA3ELv8ywtGJ\n6iG0NWQUNqncVPJJ5diTgnac9RxFSvvaUP5npHQj5hBE5Oj+ALFnhyFZ5CE0bac0Xw5G/pozQYhA\nv7t4D2Hbm6A9dDrpV6y+VeNKo9jGICTzPIROpz33X359RN0WG3Vpx1nPMa3LZJ6mJbUMf3RFot6I\nRxfGRa4wCclKrzMzZ9M078AnTeud2Z4rMMeiBUBtmaEY6awspiCMHYO2RSlnGf625A9g+/qG2ALR\nnjOfY5HBN0EwQjH2ENrWan2W4W9iQYevTyY/EbGQUSTmhYQ6nfa0EDDSWVnM+zBmlVNMZhl+H3sp\nN7W5Xf51Y18BrbV68wTBxMAIyXhWGHt2GJpZht9H/q6u9+Wr/XV+XLG9xNZavpXu7FmIlZwaIRl7\nBi3KKQOzDb8PD6HJTGpA7ClByy7BY/R7s0+95Q+MkMjRpHJscxCWWYbfR5eAumfW58T9OEGwpHIc\n5pX4WcioXUj2XyyOdluNbQ0CM8vwNzKH4GYY019bxkUFFjKKxrxVoTE2SjHaS3sFYbvh73bEy4Ss\ntuB7/GialENqsSDMfuttWiBkxGccKWpZxIh+t3O0bccYX2sQ6p5bn8Z6PLYG6EF7BWGeF2BJ5fYR\n82aUBs0QQ5NPLPtKKNf1OnyGkTtHQ0bxP//WWr7+HC/AQkYtI3Yir8WCkBcAX20r6haK+GxlM/7U\nm+AhtlYQOh2ZWdVhgmCE5GjIqIV3Y14AfO1DUlcQfEYNOg3KIbXwEjzGLMM/z3swDNc0KWQQmrwA\nFNmrpAr97uxmeovodcXrdrrjV27Cx99qyzcreWweghESaWlSGbYLgM/QzLzFqHOf53ldUio4zShm\nWby58BIzq/TU1iEYIWlzDiEvAF4FoddhYzAq/TzfrfAfdJeTvL5+GVpt+WZtWZjf1s4wfNKZWJjU\nNvKzb5+dTqsa9qqexU6k3YIwxfCLtGtvWyM+x9YhtE8RJo20753Sqr52m/ZnaM87ncK00FDb+skY\n8Vlb6XKnE9Za1/4a0mYaD6MAAArlSURBVOqd8dv2PRPfs1otQr674vN2Iq0WhGnG3/IHRmj2rfU5\n67QTvFayNJXJ7Wx9z8SrCkLV5+1EWm39piWVLX9gGGEZ5w18J2/3rJkgLKI973QK03IFlj8wjLCc\ns/92QY7T73ZY7XfYHCSFn7O+0m1VGLnV1m+aN2AeQvsQid69otWIyNEv35Sd7bcpfwAmCNt/Z4vS\nWocc/Z+x7JQVhKphpp1KuwVhSnjIksrtQyTuBjlGOMoa+L3mIbSHaUnlNsULjRShnYvC2oh5CPNp\ntSBMLTttQD8RIyyWQ2gPu1d6hZvcdbvCLk/N9ppKq63fNOOf38HJWH6EMAlNIz6djrBntV/osfvW\neq27LlotCJ2ObJst9C1k1DrSzc1jj8IIxd6CYaC9a8WEY5lotSAAdHOKYDmEdmKfenvYt6uYoS8q\nHMtE6wUh7xHYwrT2YR5CuzAPYTatt355j8AWprWPjhSPKxs7nz0FEsvdjrB7pV0JZTBB2OYR2MK0\n9tHvdrjHnfbGHoYRiE5H2L0y30vY28KEMpggbPMImrCNnWEYflm0vqBt6w/GtN76TYaMRCypbBht\nYJGHsN43QWglk60qLKFsGO1gfXV+fmDR35eV1lvASY/AEsqG0Q7WF3gIizyIZaX1gjApAhYuMox2\nsN7vziw17nRgrUX7KE/Sznc9wWRVkXkIhtEOOh1hbUafol39dlYYgQnCcVVF5iEYRntYn7HOYNbv\n20DrBWGyytRKTg2jPczaDW13SxPKYIJwXMdT8xAMoz2s9aYb/lmhpDbQekGYbHdtq5QNoz2s9Kab\nv1m/bwPtfecZVmVkGO1kluFf7ZqH0FomRcB2SzOM9mAewnba+84zjqsyspCRYbSGlRmdCUwQWky3\nI0cXqHRbWntsGG2k35Vti9O6XWl16Lj1ggDpIhWwHIJhtAkR2eYNrLa8n1m7333GOLFsK5UNo130\ncwLQ5nARmCAAx0JFHRMEw2gVeQFoe8fjdr/7jI55CIbRSrZvkNVuG2CCwLGLwHIIhtEu8u1q8iGk\nttHud59hSWXDaCf9XKl527sVmCAw4SFY2alhtIp8zsBCRhERkReJiIrIyTHH0ZG0HtmSyobRLrbl\nECxkFAcROQP418B3Y41hTK/li1EMo63kQ0T9ltuBmHL4auACQCOOAYCT96yy//a7Yw/DMIzA5JPK\nbZ8YRtlJWkSeDFyrqpct2qpORM4Hzgc488wzvYzn5D2rnLxn1ctrG4bRXLYnldsdMvImCCLyCeBO\nU/70e8BLScNFC1HVNwBvADj77LOjexOGYSwPeY8gLxBtw5sgqOpjpv1eRO4D3BUYewenA5eIyINV\n9Xpf4zEMw8iTX3fQ9m10g4eMVPWrwB3GP4vIlcDZqnpT6LEYhtFu8lVGbfcQ2i2HhmG0ml63c6z9\nfVdYlNNcdqIklSdR1f2xx2AYRnvpdTsMhontmIh5CIZhtJzx2oO2t60AEwTDMFpOv9eh07HGdtCA\nkJFhGEZMztl/UuwhNAaTRMMwDAMwQTAMwzAyTBAMwzAMwATBMAzDyDBBMAzDMAATBMMwDCPDBMEw\nDMMATBAMwzCMDBMEwzAMAzBBMAzDMDJMEAzDMAzABMEwDMPIMEEwDMMwABMEwzAMI0NUNfYYCiMi\nNwJXeXr5k4GY+zrb8e34dvz2Ht/3GO6iqqcsetCOEgSfiMhFqnq2Hd+Ob8e347d1DBYyMgzDMAAT\nBMMwDCPDBOEYb7Dj2/Ht+Hb8iEQfg+UQDMMwDMA8BMMwDCPDBMEwDMMATBAQkbeIyA0i8rVIxz9D\nRD4lIt8Qka+LyPMDH39NRL4kIpdlx39ZyONPjKMrIl8Wkb+OcOwrReSrInKpiFwU4fgnisgHROQK\nEblcRP5lwGPfI3vf46/bROQFoY6fjeGF2bX3NRF5r4isBT7+87Njfz3Ee59mc0TkJBG5UET+Ofv3\ndr7HMY3WCwLwNuDxEY8/BF6kqvcGHgo8T0TuHfD4m8CjVPV+wP2Bx4vIQwMef8zzgcsjHHfMT6rq\n/SPVgb8W+Jiq3hO4HwHPg6r+v+x93x94EHAY+FCo44vIacBvAWer6llAF3hGwOOfBfwa8GDSc3+u\niNzd82Hfxnab87vA36nqjwB/l/0cnNYLgqp+FvhhxONfp6qXZN8fIDUGpwU8vqrqwezHfvYVtNJA\nRE4Hfgp4U8jjNgEROQH4ceDNAKq6paq3RBrOo4FvqaqvbgCz6AG7RKQHrAPfC3jsewFfVNXDqjoE\nPgM8zecBZ9icJwNvz75/O/AUn2OYResFoUmIyH7gAcAXAx+3KyKXAjcAF6pq0OMDrwEuAJLAxx2j\nwN+KyMUicn7gY98VuBF4axYye5OI7A48hjHPAN4b8oCqei3wSuC7wHXArar6twGH8DXgESJyexFZ\nB54InBHw+GPuqKrXZd9fD9wxwhhMEJqCiOwBPgi8QFVvC3lsVR1lIYPTgQdnbnQQRORc4AZVvTjU\nMafwcFV9IPAE0pDdjwc8dg94IPBnqvoA4BARwgUisgI8CfiLwMe9Hens+K7AqcBuEfmFUMdX1cuB\nlwN/C3wMuBQYhTr+jDEpgb30MSYIDUBE+qRi8G5V/ctY48hCFZ8ibE7lYcCTRORK4H3Ao0TkXQGP\nP56loqo3kMbPHxzw8NcA10x4ZR8gFYjQPAG4RFW/H/i4jwG+o6o3quoA+EvgX4UcgKq+WVUfpKo/\nDtwM/FPI42d8X0TuDJD9e0OEMZggxEZEhDR+fLmqvirC8U8RkROz73cBjwWuCHV8VX2Jqp6uqvtJ\nQxafVNVgM0QR2S0ie8ffA/+aNIwQBFW9HrhaRO6R/erRwDdCHX+C8wgcLsr4LvBQEVnP7oVHE7i4\nQETukP17Jmn+4D0hj5/xEeBZ2ffPAj4cYQz0Yhy0SYjIe4FHAieLyDXA76vqmwMO4WHALwJfzeL4\nAC9V1Y8GOv6dgbeLSJd0gvB+VQ1e+hmROwIfSm0RPeA9qvqxwGP4TeDdWdjm28Avhzx4JoSPBX49\n5HEBVPWLIvIB4BLSirsvE76FwwdF5PbAAHie76T+NJsD/AnwfhF5DmmL/5/1OYaZY7PWFYZhGAZY\nyMgwDMPIMEEwDMMwABMEwzAMI8MEwTAMwwBMEAzDMIwMEwTDyCEiTxERFZF7Lnjcs0Xk1Imf3xS4\nMaFhOMUEwTC2cx7wuezfeTybtN0CAKr6q6oaY1GZYTjBBMEwJsh6Sj0ceA4TbZhF5HeyPRMuE5E/\nEZGnA2eTLii7VER2icinReTs7PHnZY//moi8fOJ1DorIf85e5wsiEqWJmWFMwwTBMI7nyaR7E/wT\n8AMReZCIPCH7/UOyfSNeoaofAC4CnpntJ3Bk/AJZGOnlwKNI95g4R0TG7Yx3A1/IXuezpL34DaMR\nmCAYxvGcR9pkj+zf80gbsL1VVQ8DqOqi/TPOAT6dNWwbAu8m3fMAYAsYtwa5GNjvbuiGUY/W9zIy\njDEichLprP4+IqKku3cpbltCD/RYv5gRdg8aDcI8BMM4xtOBd6rqXVR1v6qeAXwHuBX45WwDlbFw\nABwA9k55nS8BPyEiJ2dNA88j3YnLMBqNCYJhHOM8tu8n/EHSjrAfAS7KOtK+OPvb24DXj5PK4ydk\nO1/9LuneEpcBF6tqlHbGhlEG63ZqGIZhAOYhGIZhGBkmCIZhGAZggmAYhmFkmCAYhmEYgAmCYRiG\nkWGCYBiGYQAmCIZhGEbG/w9ug0u9ihqAsQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f608036d358>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# An example bandit problem \n",
    "\n",
    "def example_bandit_rewards_plot():\n",
    "    # fake data\n",
    "    fs = 10  # fontsize\n",
    "    pos = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]\n",
    "    \n",
    "    \n",
    "    data = [np.random.normal(0, std, size=100) for std in pos]\n",
    "\n",
    "    fig, axes = plt.subplots(nrows=1, ncols=1, figsize=(6, 6))\n",
    "    \n",
    "    #Mean of the rewards. For the arms, the means of the rewards lie in a normal distribution\n",
    "    #with mean 0 and standard deviation 1\n",
    "    optimal_action_values = np.random.normal(0, 1, size=10)\n",
    "    \n",
    "    #The actual Reward for a particular arm lies in a normal distribution\n",
    "    #with mean being the optimal_action_value for the particular arm and \n",
    "    #standard deviation 1\n",
    "    rewards = [np.random.normal(optimal_action_value,1,size=100) for optimal_action_value in optimal_action_values]\n",
    "\n",
    "    axes.violinplot(rewards, pos, points=20, \n",
    "                          showmeans=True, showextrema=False, showmedians=False)\n",
    "    #axes.set_title('Reward Distribution for different Actions', fontsize=fs)\n",
    "\n",
    "\n",
    "    axes.set_xlabel(\"Action\")\n",
    "    axes.set_xticks(pos)\n",
    "    axes.set_ylabel(\"Reward Distribution\")\n",
    "    fig.suptitle(\"Reward Distribution for different Actions\")\n",
    "    #fig.subplots_adjust(hspace=0.4)\n",
    "    plt.show()\n",
    "\n",
    "example_bandit_rewards_plot()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a id=\"5\">Greedy Bandit Implementation</a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# In order to standardise test set and compare various methods later,\n",
    "# in subsequent examples/problems we will assume the rewards lie in normal distribution\n",
    "# whose mean is given by\n",
    "list_optimal = [0.5,0.7,0.9,1.1,2.4,0.1,0.9,0.1,0.3,0.7]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epsilon :  0.1 \t average reward value 2.108155864694302\n",
      "epsilon :  0 \t average reward value 1.0599881866791852\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7f607e256a90>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfIAAAF3CAYAAABe7Mm7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XmcFNW5//HPMzOsIrINyg6iiIgi\niihqFFyJayQSF6LiEo3RqPGnxqhRr/Eak5gYTYiGKCJGjV70KhqCGkXFixsqKrgiAQVZBkRWWWbm\n+f1xqmEYh6GB6a7pqu/79epXd1dX9zw1jX7nnDp1jrk7IiIiUpiK4i5AREREtp6CXEREpIApyEVE\nRAqYglxERKSAKchFREQKmIJcRESkgCnIRURECpiCXEREpIApyEVERAqYglxERKSAlcRdwJZq06aN\nd+3aNe4yREQKyltvvbXI3Uu38TPalpSU3AP0Rg3BfKoEppWXl5+37777Lqz+YsEFedeuXZkyZUrc\nZYiIFBQzm72tn1FSUnLPTjvttHtpaemSoqIiLdSRJ5WVlVZWVtZr/vz59wAnVH9df1GJiEi2epeW\nli5TiOdXUVGRl5aWLiX0hHz79TzXIyIihatIIR6P6PdeY2YryEVEJDUefPDBHa655pqdAC6//PL2\n119//Y65+lmTJk1q2qNHj16dO3fuPXz48E6VlZXf2uedd95pvPfee/ds2LDhPltbi4JcRERSY9iw\nYUtvueWW+fn4WT/5yU+63HXXXbNnzZo1bebMmY3Hjh3bvPo+bdu2Lb/jjjs+v+CCCxZs7c9RkIuI\nSMH4y1/+0mrPPffcvWfPnr1OP/30LuXl5TRt2rTvueee22mXXXbZY8CAAT2+/PLLEoCbb765bffu\n3ffo0aNHr+OOO25ngDvvvLP1mWee2bn6506ePLlJnz59evbo0aPXkUce2b2srKwYoH///rtdeOGF\nHfbcc8/du3bt2nvChAnNsqlz9uzZDVasWFF0+OGHrywqKmLYsGGLn3jiiZbV9+vQoUP5oYceuqpB\ngwZbfcqi4Eati4hIPXDOOZ2YNq1pnX5m796rGDXqi029/PbbbzceO3ZsqylTpnzUqFEj/+EPf9j5\n7rvvbv3NN98U9evXb+W99977xRVXXNHu6quvbj9mzJjP77zzzp1mz579fpMmTXzRokXFtf3o4cOH\nd7v99ts/P/bYY1dcdtll7X/+85+3HxXVUl5ebu+///6HjzzyyA433XRT+8GDB3/y7rvvNjrllFO6\n1/RZr7zyysezZ89u0K5du3WZbV26dFk7b968Blv7q6mNglxERArChAkTtp82bVrTPn367A6wevXq\norZt25YXFRVx3nnnfQVwzjnnLB4yZMguALvttts3J510UrcTTjjh62HDhn29qc9dvHhx8fLly4uP\nPfbYFQA/+tGPFg8dOnTnzOtDhw5dAnDggQeuvPLKKxsC9OnTZ81HH330Qe6ONnsKchER2XK1tJxz\nxd1t6NChi0eMGDG36vY77rijXdXnZgbAxIkTP/3Xv/61/ZNPPrnDbbfd1u7jjz+evjU/t3Hjxg5Q\nUlJCRUWFAWyuRd6lS5d1VVvgs2fPbli1hV6XdI5cREQKwuDBg5c9/fTTLefOnVsCsGDBguJPPvmk\nYWVlJffdd19LgNGjR7fu37//8oqKCj777LOGxx9//PIRI0bMXbFiRfHSpUtr7F5v3bp1RfPmzSsy\n57/vvffe1gMGDFhRWy2ZFnlNtzZt2lR06dJlXbNmzSqff/757SorK3nwwQdbn3jiiZvsFdgWapGL\niEhB2HfffVdfd911cw8//PAelZWVNGjQwO+8887PmzRpUvnGG29s97vf/a5969at1z3++OMzy8vL\n7fTTT++2fPnyYne38847b2GbNm0qNvXZ9913338uvPDCLpdccklR586d1zz88MOztrXeESNGzD73\n3HO7rV692gYNGrRs6NChSwF++9vflgJcddVVZZ9//nnJfvvt12vlypXFZuZ//etfd/zwww+ntWrV\n6tvXqm2CuRfWtf39+vVzTdEqIrJlzOwtd++3LZ/x7rvvzurTp8+iuqqprjRt2rTvqlWr3om7jlx7\n99132/Tp06dr9e1qkYuI1EeVlbBsGXz1VbgtWQIdO8Luu8ddmdQzCnIRkVxas2ZDEFcN5c09/vrr\nEOZVXXUV/OY38RxHPZaG1nhtFOQiIptTWRmCdfHiELKLF4fQXbIkbK/6uHoor1q16c8tKoIWLaBV\nq3Br2RK6d9/wOLM987xbt/wdsxQMBbmIpIt7CNmFC2HBAli0KARz1ZBetGjDLRPatY0n2m67EMiZ\n8O3efeMg3tTj5s1DmItsAwW5iBS+b76BsrIQzpu6LViw4XF5ec2f06RJCNjSUmjTBjp3htatw61V\nq40ft2oVwrtFC2jYML/HK1KFglxE6p/KyhC88+aF4M2EdE33ZWWwYhOX/DZuDG3bwo47QocOsM8+\n4XFmW9u2IbQz4dykSX6PU6QOKMhFJL/Ky0NAz5kDc+eG++q3uXNrbjU3aLAhfNu2hV133fh55pZ5\nvt12EM3yJQJhGdPp06c3ueWWW+Zffvnl7Zs1a1Zx0003bfXKY7WZNGlS03PPPbfr6tWriw477LCl\no0aN+qIoB6dSFOQiUrdWrIDZs+Hzz2u+nzv326OxGzeGTp1Cq/ngg8Pjjh2hffvQcs4Ec/PmCmbZ\nJsOGDVsKLM3Hz8osYzpo0KCVAwcO3HXs2LHNf/CDHyyr65+jIBeR7FVWhi7t2oJ6yZKN31NSEkK5\nSxcYODCcd+7cOWzL3Fq2VEBLVv7yl7+0uuuuu3Zct26d7bPPPivHjBkzu3nz5n1PO+20RS+99FLz\n0tLSdY899tjM9u3bl998881t77vvvtLi4mLv0aPH6qeffnrmnXfe2XrKlCnbjRkz5vOqnzt58uQm\nF154YZdvvvmmqEuXLmseeuihWaWlpRX9+/ffbd99913xyiuvNF++fHnx3XffPWvw4MG1Tt8KGy9j\nCqxfxlRBLiK5tXo1fPFFzSGdua1du/F7tt8+hHSXLjBgQLjv3HnDfbt2UFzrCpJSgM45h07TplGn\ny5j27s2qUaPQMqZbSEEukibl5fCf/8BHH4X76oG9oNqpQrMQxJ07w777wkknfTuoW7SI51gkdbSM\nac0U5CJJtHQpfPppCOyqt08/3bhF3bjxhq7u447bOKC7dAnd3rq0SmpQW8s5V7SMac0U5CKFasUK\n+OSTENAffxwef/YZzJwZJjHJKC4OE5T07AnHHhvue/YM20pLdW5aCsbgwYOXDRkyZJdrrrlmQYcO\nHcoXLFhQvHTp0uLMMqbnn3/+kpqWMT3qqKNWdOrUqVU2y5gOHjx4xZYsY7qp19u0aVORWcZ00KBB\nKx988MHWF1100cJt/R3UJGdBbmadgDHAjoADI939jmr7GHAHcAywChju7m/nqiaRgrR8Obz/Pkyb\nFu4//DAE95w5G/YpKgot6F12gZNPDiHdvXtYYKN7d7WqJRGSsoxpXcvZMqZm1g5o5+5vm9n2wFvA\n99z9gyr7HAP8lBDk+wN3uPv+tX2uljGVRFuwAN55Z+PbjBkbXm/WDHr1gt12C63qzP0uu0CjRvHV\nLfWeljEtfHlfxtTd5wHzosfLzexDoANQtSviRGCMh78mXjOzFmbWLnqvSHK5h8Fm1UN7XpV/+t26\nQd++cNZZ0KcP9O4dWt2am1tEqsjLOXIz6wr0BV6v9lIH2GjAxJxom4JcksM9TILyf/8Hr74aAnvq\n1LDWNIRz2LvvDkccEYK7b1/Ye2+NBhfJUhpa47XJeZCbWTPgMeAyd9+qC+HN7HzgfIDOnTvXYXUi\nObByZQjsyZPhzTfDLXNZV5MmoXU9bNiGwO7dW3N8i8hWy2mQm1kDQog/6O6P17DLXKBTlecdo20b\ncfeRwEgI58hzUKrI1lu6FF55BV5+OdymTAnXa5uF89dHHw39+sH++4dFO0p0sYgUrMrKykorKirS\n/4fzrLKy0oDKml7L5ah1A+4FPnT3P2xit3HAxWb2D8Jgt6U6Py713qJFMGkSvPRSCO6pU0P3eYMG\n0L8/XHklHHIIHHhgmBtcJDmmlZWV9SotLV2qMM+fyspKKysr2wGYVtPruWwaHAScAbxvZlOjbdcA\nnQHc/W5gPGHE+gzC5Wdn57Aeka3z5ZcbWtsvvQQfROM1mzQJU5LecEMI7v33h6Z1OmOlSL1SXl5+\n3vz58++ZP39+b0CjLvOnEphWXl5+Xk0v5uzys1zR5WeSU+4wa9aG4H755Q2Xf22/PRx0EBx6aAju\nfv10fbYUjLq4/EzqJ52sE/n6a3juORg/Hp5/PiwaAtCqFXznO3DhhSG8+/TR+W0RqXf0fyVJH/cw\nQ9r48eE2eTJUVISlNI84An7+89Di3mMPXbMtIvWeglzSobw8dJM//jg8+eSG6U379oWrr4ZjjgkD\n1dTiFpECo/9rSXKtWwcTJ8LYsfC//xtGmzdpAoMHw3/9V7hv3z7uKkVEtomCXJJl3bpwvnvsWHji\nCViyJMxPfvzx8P3vw3e/q5HlIpIoCnIpfO7wxhvw97/DI49AWVm4fvvEE8NKYEcdFdbdFhFJIAW5\nFK5PPgkt71GjwjrcjRrBCSfAD38YZlPTamAikgIKciksX30FDz8Mo0eHqVABBg2Ca6+FIUNghx1i\nLU9EJN8U5FL/VVSE67tHjQqD1tauDYuN3H576D7v1i3uCkVEYqMgl/prxQq47z64447Qdd6qFfz4\nx3D22SHIRUREQS710OzZMGIEjBwZVhYbMAD++7/he9/TeW8RkWoU5FJ/TJ4cussffzwsATpkCFx+\nORxwQNyViYjUWwpyiVdFBYwbB7/7Hbz6apgm9Yor4KKLoHPnuKsTEan3FOQSj+XL4a9/hT/9CT7/\nPAxY+9Ofwvnv7baLuzoRkYKhIJf8WrAgBPaIEWHVscMOgz/8IZz/Li6OuzoRkYKjIJf8mDsXfv1r\nuOeecPnYSSeFxUr22y/uykRECpqCXHJr/ny49Va4++5wPnz4cLjySujRI+7KREQSQUEuubFqFfzq\nV+Ea8LVr4ayz4LrrNHmLiEgdU5BL3aqshDFj4Prr4YsvwrznN9wAu+wSd2UiIomkIJe6M2UKXHwx\nvP469O8fViM75JC4qxIRSbSiuAuQBFi8OEyd2r8/zJoVWuSvvaYQFxHJAwW5bL2KinAteI8eYTT6\npZfCxx/DGWeEmdlERCTn1LUuW+e110I3+ltvwaGHwp//DL17x12ViEjqqEUuW2bJkjD72oAB8OWX\n8NBDMHGiQlxEJCZqkUv2nnoKLrgAysrCZC7XXgvNmsVdlYhIqinIZfMWL4af/QweeAD23BP++U/o\n2zfuqkREBHWty+Y8+yz06gUPPwy//GW4xEwhLiJSbyjIpWYVFSG4Bw+Gtm3h7bfhppugYcO4KxMR\nkSrUtS7fNn8+nH56GMR2zjlhtbKmTeOuSkREaqAgl429+CKceiosWwb33RcWORERkXpLXesSVFaG\nZUYPPxxatAjTrCrERUTqPbXIBZYuDauTPflkaI2PHAnbbx93VSIikgUFedpNnw5DhsDMmfDHP8Il\nl2h6VRGRAqIgT7NHHw2D2Zo1gxdegO98J+6KRERkC+kceRqVl8MVV8App8Bee4VLyxTiIiIFSS3y\ntFm4EE47LbTAL7oI/vAHXRsuIlLAFORp8sILIcSXLtWlZSIiCaGu9bS45x448kho3TpMs6oQFxFJ\nBAV50rnDzTfDj34Ugvz117XkqIhIgqhrPckqKuDSS2HECDjjDLj3XmjQIO6qRESkDqlFnlSLF8P3\nvhdC/MorYfRohbiISAKpRZ5E8+bBEUfAjBlhwZOLL467IhERyREFedLMnh3mS58/HyZMgEGD4q5I\nRERySEGeJJ98Elriy5fDv/8NBxwQd0UiIpJjCvKk+OgjGDgwrGI2cSLsvXfcFYmISB4oyJPg4483\ndKG/9BLsvnu89YiISN4oyAvdJ5+EEK+ogBdfVIiLiKSMgryQffjhxt3pvXrFXZGIiOSZgrxQzZgR\nRqcXFYXu9J49465IRERioCAvRJlLzNauVYiLiKScgrzQfPllCPFly8JqZnvsEXdFIiISIwV5IVm4\nMIT4ggXhOvG+feOuSEREYqYgLxRLl8LRR4du9QkTYP/9465IRETqAQV5IaishB/+EKZNg6eegkMO\nibsiERGpJxTkheC22+Dpp8MCKIMHx12NiIjUI1rGtL575RW45hoYOhQuuijuakREpJ5RkNdnZWVw\nyinQrRvccw+YxV2RiIjUM+par68qKsJ58cWLYfx4aN487opERKQeUpDXV7fcAs8+CyNHQp8+cVcj\nIiL1lLrW66PnnoMbbwwt8vPOi7saERGpxxTk9c3778PJJ4cZ2+66S+fFRUSkVjkLcjMbZWYLzWza\nJl4faGZLzWxqdLs+V7UUjLlz4ZhjoFmzcF68WbO4KxIRkXoul+fIRwN/BsbUss8kdz8uhzUUjpUr\n4dhjwwxukyZBx45xVyQiIgUgZ0Hu7i+bWddcfX7iXHIJvPdeaIlrcJuIiGQp7nPkA8zsXTP7l5ml\ndxmvf/wDRo0KE79o5jYREdkCcV5+9jbQxd1XmNkxwBPArjXtaGbnA+cDdO7cOX8V5sMHH8D558OA\nAWGkuoiIyBaIrUXu7svcfUX0eDzQwMzabGLfke7ez937lZaW5rXOnFq7Fk49FZo0gUcegRJd1i8i\nIlsmtuQws52ABe7uZtaf8EfF4rjqicWvfhUuNxs3Djp1irsaEREpQDkLcjN7GBgItDGzOcANQAMA\nd78bOBm40MzKgW+AU93dc1VPvfPWW/DrX8OZZ8Lxx8ddjYiIFCgrtOzs16+fT5kyJe4yts2aNbDv\nvrBkSVhjvGXLuCsSkYQzs7fcvV/cdUjd00nZONx0E0yfDv/8p0JcRES2SdyXn6XPm2/CrbfC8OFh\nFjcREZFtoCDPp5UrQ4C3awe33x53NSIikgDqWs+nCy6Ajz4Ks7e1aBF3NSIikgBqkefLo4/Cgw/C\nDTfA0UfHXY2IiCSEgjwf5s+HH/8Y9t8/TMMqIiJSRxTk+XDddbBiBdx/v2ZvExGROqUgz7X33gsL\novz0p7DbbnFXIyIiCaMgz7WrrgoD2669Nu5KREQkgdTPm0vPPBNuf/gDtGoVdzUiIpJAapHnSkUF\nXHkl7Lwz/OQncVcjIiIJpRZ5rtx/f1jZ7NFHoVGjuKsREZGEUos8F1asCCPVDzgATj457mpERCTB\n1CLPhd//HubNg7FjwSzuakREJMHUIq9r8+bBb38LQ4fCgQfGXY2IiCScgryuXX89rFsHv/513JWI\niEgKKMjr0vvvh8lfLr4YunePuxoREUkBBXlduuoqaN48DHQTERHJg00OdjOzPwG+qdfd/ZKcVFSo\nnn0WJkwIA900+YuIiORJbS3yKcBbQGNgH+DT6LY30DD3pRUQ99Aa79YNLroo7mpERCRFNtkid/f7\nAczsQuBgdy+Pnt8NTMpPeQViwgR4910YPVqTv4iISF5lc468JdC8yvNm0TYBqKyEm2+GDh3gtNPi\nrkZERFImmwlhbgXeMbOJgAGHADfmsqiC8re/weTJYbR6Q51xEBGR/DL3TY5nw8wM6AisA/aPNr/u\n7vPzUFuN+vXr51OmTInrx29s+XLYZZewzvhLL2kWNxGpt8zsLXfvF3cdUvdqbZG7u5vZeHffE3gy\nTzUVjltvhYUL4emnFeIiIhKLbM6Rv21m++W8kkLz+edhnfFhw2A//XpERCQe2Zwj3x8YZmazgZWE\n8+Tu7nvltLL67pe/DJed3XJL3JWIiEiKZRPkR+e8ikLz3nvwwANwxRXQuXPc1YiISIptNsjdfTaA\nmbUlTA6Tbu4hwFu0gF/8Iu5qREQk5TZ7jtzMTjCzT4H/AC8Bs4B/5biu+uv+++G55+BXv4KWupxe\nRETilc1gt18BBwCfuHs34HDgtZxWVV/NnQuXXQbf+Q5ceGHc1YiIiGQV5OvcfTFQZGZF7j4RSOe1\niNdcA6tXh8lfirRwnIiIxC+bwW5fm1kz4GXgQTNbSBi9ni7PPANjxsDVV4dJYEREROqBWmd2AzCz\n7YBvCK33YcAOwINRKz3vYpnZrbw8zN7WsCG8/TY0aZLfny8iso00s1tyZdMiPxV42d0/Be7PcT31\n0+OPw8yZ8NhjCnEREalXsgnyzsBfzawbYY3yl4FJ7j41p5XVF+7wu9/BrrvCiSfGXY2IiMhGsrmO\n/AYAM2sC/Ai4EvgjUJzb0uqJceNgyhQYORKK03HIIiJSODYb5GZ2HXAQYR3yd4ArgEk5rqv+uPFG\n6NkTzjwz7kpERES+JZuu9SFAOfBPwoQwr7r7mpxWVV9MnRpuf/4zNGoUdzUiIiLfstmLod19H+AI\n4A3gSOB9M3sl14XVC/fdF0aqn3Za3JWIiIjUKJuu9d7Ad4BDCRPBfEEautZnzw4TvwwZAq1axV2N\niIhIjbLpWr+VENx3Am+6+7rcllRP/OY34frxW2+NuxIREZFNymbU+nHRiPXOqQnx1avh4YdDa7xL\nl7irERER2aRsVj87HpgKTIie721m43JdWKyefBK+/hrOPjvuSkRERGqVzcofNwL9ga8BoolguuWw\npviNHg2dOsGgQXFXIiIiUqtsVz9bWm1b7RO0F7K5c+HZZ+GsszQBjIiI1HvZDHabbmanA8Vmtitw\nCTA5t2XF6LHHoLISzjgj7kpEREQ2K5sW+U+BPYA1wEPAMuCyXBYVqyefhF69oEePuCsRERHZrGwm\nhFnl7te6+37R7VqgbR5qy7+vvoKXXtLiKCIiUjBqDXIzG2BmJ5tZ2+j5Xmb2EPB/eaku38aPh4oK\nBbmIiBSMTQa5mf0OGAV8H/inmd0MPAu8Duyan/Ly7MknoV072G+/uCsRERHJSm2D3Y4F+rr7ajNr\nSZiatbe7z8pLZfm2Zg1MmADDhkFRNkMHRERE4ldbYq1299UA7r4E+DSxIQ4wcSKsWAEnnBB3JSIi\nIlmrrUW+c7UZ3LpVfe7uyUq8p5+Gpk3hsMPirkRERCRrtQV59RFfv89lIbF79tkwk1vjxnFXIiIi\nkrVNBrm7v5TPQmI1Zw58+in8+MdxVyIiIrJFspnZLfkmTgz3mlt9m6xdC998EybG25pbRUVYeK5Z\ns3CWo+r2qvc1bauoCKvOlpeDWbhVVoL7hvdkHsOGfczC8zVrYN26cFu5csN+tcl8bk2ftymexeTG\nVX8nJSXQoEG4z7y/6rHEcV9Xn1FcHG4lJeG+qGjDa1t6y/xu8v3eLX1/9d9h5t9udVX/HVV9fMEF\ncNVVm/83JOmiIIcQ5C1bQp8+ef/Rmf+I3cN/sMuWhSBp3BgWL4ZVq0I4Zm5Vny9YEOawyQRYJvwy\nz9etq/lxtq9BeLxmTQjYVas21Jn5n0vVx2vX5v3XJ9so8/0VFW3Z/da8p+o9bPi3mvkjrLJy4z+I\ntvRW9Xjq6r1Vj7cufnb1z8v8MVM1rKv+sVf9Dz+tqiw1yTrIzaypu6/KZTGxmTgRBg6sk8vOVq+G\nzz6D5s2hrCwE7XvvwRdfhIBevhzmzQu3lSvDGi1byyz8nEyL5uuvwx8ADRpsaMVVbdHV9Lhx49AC\nrv5aSUn4/JISaNQo7Ne0adi2qVbI9tuHfTKtqy25Zf4n17hx+L2sWhWeZz6r+n1N2xo0CI8z9WU+\ns3qQVK8dwjE2bBg+I3MM2fz+M/9kqn7e5lrl2bye+Z971T+uYNvCs6Z7ESl8mw1yMzsQuAdoBnQ2\nsz7ABe7+k1wXlxezZoXb5Zdv8VvXrIHnn4fJk+HLL2H6dJgypeZu2WbNNtx22ik0/hs3hm7dNv6L\nvEmTsM+qVWG/pk3DtiZNNn7cpAm0bh3CR0RE0iubFvntwNHAOAB3f9fMDslpVfmU5fnxysow8dv/\n/A9MnQqLFoXu7RUrwus77QS77w4/+xnsuWdoVbZvDy1aQPfuYXlzERGRupZV17q7f2Eb98PVMDxj\nY2Y2CjgOWOjuvWt43YA7gGOAVcBwd387m3rq1MSJUFoKe+wBhK7Rv/8dbr8dDj44dC1Pnw6vvhq6\nxUtLQzD36xdCfOjQMDV706Z5r1xERCSrIP8i6l53M2sAXAp8mMX7RgN/BsZs4vXvEuZs3xXYH7gr\nus8f9w3nx6M/VK66Cm67Lbz8zjvhvkuXsDz5wQeH4C7REEEREaknsomkHxNazh2AuYSFUy7a3Jvc\n/WUz61rLLicCY9zdgdfMrIWZtXP3eVnUVDc++yxcQx51q99/fwjxQw6BO+6ADh3C+euWLTUwSERE\n6qfNBrm7LwKG5eBndyAsxJIxJ9qWvyB/7rlwP2gQL78Mw4fDPvvAM89ogjcRESkM2Yxav7OGzUuB\nKe7+ZN2XVGMN5wPnA3Tu3LnuPvjRR6FnTx6bthsnDw2bHn5YIS4iIoUjmwunGwN7A59Gt72AjsC5\nZvbHbfjZc4GqY7k7Rtu+xd1Huns/d+9XWlq6DT+yiuXL4cUXeXOP4Zw13NhzT/j3v6FHj7r5eBER\nkXzI5hz5XsBB7l4BYGZ3AZOAg4H3t+FnjwMuNrN/EAa5Lc3r+fG//Y3VNOJ7z19M27Ywfjx07Ji3\nny4iIlInsgnyloTJYJZGz7cDWrl7hZmt2dSbzOxhYCDQxszmADcADQDc/W5gPOHSsxmEy8/O3spj\n2CqzXpzFScVv8OXX2/HsowpxEREpTNkE+W+BqWb2ImDAIcAtZrYd8O9NvcndT6vtQ6PR6psd/Z4L\n5eVw/gunMrViL0aPhiOPjKMKERGRbZfNqPV7zWw80D/adI27fxk9vjJnleXQA/eV89zKA/nFgS9y\n1lkD4y5HRERkq2W7SshqwmVhS4BdCn2K1n8/sYIOzOG/L8rfKXkREZFcyObys/MIs7l1BKYCBwCv\nAofltrTcmfnxOnbjY6zPXnGXIiIisk2yaZFfCuwHzHb3QUBf4OucVpVjn33ZmJ2LP4eePeMuRURE\nZJtkE+Sr3X01gJk1cvePgN1yW1buLF8OZd9sT/eOa7JbdFpERKQey2bU+hwzawE8ATxnZkuA2bkt\nK3c+eXsF0Iyd99wu7lJERETpSJEFAAAQ8UlEQVS2WTaj1k+KHt5oZhOBHYAJOa0qhyY+NA/YlYOH\ndYm7FBERkW1Wa5CbWTEw3d17Arj7S3mpKof+8/xMWtGK9gd1i7sUERGRbVbrOfJoWtaPzawOVyqJ\n14IvK9iRBWGNUhERkQKX7RSt083sDWBlZqO7n5CzqnJowdoW7NihARRlewm9iIhI/ZVNkP8y51Xk\nSeWir/igYjdO2umLze8sIiJSADbbLI3Oi88CGkSP3wTeznFdOfGfO5/iK1pzwJ4rN7+ziIhIAdhs\nkJvZj4CxwF+jTR0Il6IVnJlfNgZg1x/0jbkSERGRupHNieKLgIOAZQDu/inQNpdF5cqshU0B6Naz\nUcyViIiI1I1sgnyNu6/NPDGzEsBzV1KOrFtH2VOvAtC2nWZ0ExGRZMgmyF8ys2uAJmZ2JPA/wFO5\nLSsHystZQksa8w2NG8ddjIiISN3IJsivBsqA94ELgPHAdbksKleW0JKWLIm7DBERkTqTzeVn3wPG\nuPvfcl1MTrlXCfL2cVcjIiJSJ7JpkR8PfGJmD5jZcdE58oK0jOY0D2P2REREEiGb68jPBnYhnBs/\nDfjMzO7JdWF1zp21NKQRa+KuREREpM5k1bp293Vm9i/CaPUmhO7283JZWJ2Lgnx7lsddiYiISJ3J\nZkKY75rZaOBT4PvAPcBOOa4rJ9bSkIa7d4+7DBERkTqTTYv8TOAR4AJ3L9x+6ahF3rC4Iu5KRERE\n6sxmg9zdT6v63MwOBk5z94tyVlWOrKUhDUsq4y5DRESkzmR1jtzM+gKnA0OB/wCP57KonFjfIl8V\ndyUiIiJ1ZpNBbmY9CKPUTwMWEbrXzd0H5am2urU+yFfEXYmIiEidqa1F/hEwCTjO3WcAmNnP8lJV\njqhrXUREkqa2UetDgHnARDP7m5kdDlh+ysoBDXYTEZEE2mSQu/sT7n4q0BOYCFwGtDWzu8zsqHwV\nWGfcWUcDtchFRCRRspnZbaW7P+TuxwMdgXeAn+e8shxQ17qIiCRNNnOtr+fuS9x9pLsfnquCcqWi\n3KmgRF3rIiKSKFsU5IWsfJ0DUFLsMVciIiJSd1IT5JVRj3pRao5YRETSID2x5qElbqhFLiIiyZGa\nIPdMflvhXkEnIiJSXXqCvDJqkZta5CIikhzpCfIov00tchERSZDUBHmGclxERJIkNUGe6VoXERFJ\nktQFuVrkIiKSJOkJ8vXnyOOtQ0REpC6lJsjXX0euIBcRkQRJTZDrHLmIiCRReoJcXesiIpJA6Qly\nDXYTEZEESk2Q6xy5iIgkUWqC3IkSXEkuIiIJkp4gr9TqZyIikjzpCfLMYLfUHLGIiKRBemJt/Xrk\nIiIiyZGaIF9/HbnOkYuISIKkJ8iV4yIikkApDHINdhMRkeRITZBvuI5cTXIREUmO1AS5qyEuIiIJ\nlJ4g1xStIiKSQApyERGRApaaIM9QkIuISJKkJsi1HrmIiCRReoI8mtPNitQkFxGR5EhPkGvRFBER\nSaDUBPn668jTc8QiIpICOY01MxtsZh+b2Qwzu7qG14ebWZmZTY1u5+Wqlg3XkatrXUREkqMkVx9s\nZsXACOBIYA7wppmNc/cPqu36iLtfnKs6MnT5mYiIJFEuW+T9gRnuPtPd1wL/AE7M4c+rlRZNERGR\nJMplkHcAvqjyfE60rbrvm9l7ZjbWzDrlsB5AQS4iIskS99Cvp4Cu7r4X8Bxwf007mdn5ZjbFzKaU\nlZVt1Q/SdeQiIpJEuQzyuUDVFnbHaNt67r7Y3ddET+8B9q3pg9x9pLv3c/d+paWlW1XM+nPkuo5c\nREQSJJdB/iawq5l1M7OGwKnAuKo7mFm7Kk9PAD7MVTE6Ry4iIkmUs1Hr7l5uZhcDzwDFwCh3n25m\nNwFT3H0ccImZnQCUA18Bw3NVT4aZuthFRCQ5chbkAO4+Hhhfbdv1VR7/AvhFLmvY8LMyj9QkFxGR\n5Ih7sFve6DpyERFJIgW5iIhIAUtNkGdornUREUmS1MSaa4ybiIgkUHqCfH3XuvrWRUQkOdIT5LqO\nXEREEig1Qb5+PXIFuYiIJEhqgnz9OXIFuYiIJEjqglznyEVEJEnSE+TrB7tp+LqIiCRHaoIcNGpd\nRESSJzVB7h4FuIJcREQSJD1BnulaR13rIiKSHOkJ8sxgtyK1yEVEJDlSE+Trz5GrRS4iIgmSmiD3\nyuiBzpGLiEiCpCfINUWriIgkUIqCXFO0iohI8qQmyNFgNxERSaDUBLnWIxcRkSRKT5BXqmtdRESS\nJz1BrsFuIiKSQKkJ8gydIxcRkSRJTZBnutZFRESSJD1Brq51ERFJIAW5iIhIAUtNkKMJYUREJIFS\nE+TrryNXkouISIKkLsiV4yIikiTpCXJNCCMiIgmUmiDP0HXkIiKSJKkJcs21LiIiSZSeIFfXuoiI\nJFB6gpyQ4OpaFxGRJElNkOs6chERSaLUBLnmWhcRkSRKT5Cra11ERBIoPUGuwW4iIpJAqQnyDAW5\niIgkSWqCXNeRi4hIEqUnyDNd6zpHLiIiCZKeINeiKSIikkCpCfIMBbmIiCRJaoJ8/XXkSnIREUmQ\n9AR55jpy5biIiCRIeoJc15GLiEgCpSbI18+1rlHrIiKSIKkJ8kzXuoiISJKkJ8jVtS4iIgmUniDX\noikiIpJAqQlyrUcuIiJJlJogd022LiIiCZSiIFfXuoiIJE96glyD3UREJIFSE+QZCnIREUmS1AT5\n+lPkSnIREUmQ1AW5clxERJIkPUFeqSlaRUQkeVIT5BlqkYuISJKkJsh1GbmIiCRR6oJcXesiIpIk\nOQ1yMxtsZh+b2Qwzu7qG1xuZ2SPR66+bWddc1aLBbiIikkQ5C3IzKwZGAN8FegGnmVmvarudCyxx\n912A24Hf5KoezbUuIiJJlMsWeX9ghrvPdPe1wD+AE6vtcyJwf/R4LHC4WW6idv0pciW5iIgkSC6D\nvAPwRZXnc6JtNe7j7uXAUqB1LorxynCvHBcRkSQpiMFuZna+mU0xsyllZWVb9Rk7dmvKUa2n0KxN\n4zquTkREJD65DPK5QKcqzztG22rcx8xKgB2AxdU/yN1Huns/d+9XWlq6VcUcekkfnlnUjy4Hd9r8\nziIiIgUil0H+JrCrmXUzs4bAqcC4avuMA86KHp8MvOBaOFxERCRrJbn6YHcvN7OLgWeAYmCUu083\ns5uAKe4+DrgXeMDMZgBfEcJeREREspSzIAdw9/HA+Grbrq/yeDUwNJc1iIiIJFlBDHYTERGRminI\nRURECpiCXEREpIApyEVERAqYglxERKSAKchFREQKmIJcRESkgCnIRURECpiCXEREpIApyEVERAqY\nFdoaJWZWBszeyre3ARbVYTmFQMecDjrmdNiWY+7i7lu3fKTUawUX5NvCzKa4e7+468gnHXM66JjT\nIY3HLJunrnUREZECpiAXEREpYGkL8pFxFxADHXM66JjTIY3HLJuRqnPkIiIiSZO2FrmIiEiipCbI\nzWywmX1sZjPM7Oq466krZtbJzCaa2QdmNt3MLo22tzKz58zs0+i+ZbTdzOzO6PfwnpntE+8RbB0z\nKzazd8zs6eh5NzN7PTquR8ysYbS9UfR8RvR61zjr3lpm1sLMxprZR2b2oZkNSMF3/LPo3/Q0M3vY\nzBon7Xs2s1FmttDMplXZtsXfq5mdFe3/qZmdFcexSHxSEeRmVgyMAL4L9AJOM7Ne8VZVZ8qB/+fu\nvYADgIuiY7saeN7ddwWej55D+B3sGt3OB+7Kf8l14lLgwyrPfwPc7u67AEuAc6Pt5wJLou23R/sV\nojuACe7eE+hDOPbEfsdm1gG4BOjn7r2BYuBUkvc9jwYGV9u2Rd+rmbUCbgD2B/oDN2TCX9IhFUFO\n+Mc9w91nuvta4B/AiTHXVCfcfZ67vx09Xk74H3wHwvHdH+12P/C96PGJwBgPXgNamFm7PJe9Tcys\nI3AscE/03IDDgLHRLtWPN/N7GAscHu1fMMxsB+AQ4F4Ad1/r7l+T4O84UgI0MbMSoCkwj4R9z+7+\nMvBVtc1b+r0eDTzn7l+5+xLgOb79x4EkWFqCvAPwRZXnc6JtiRJ1J/YFXgd2dPd50UvzgR2jx0n4\nXfwRuAqojJ63Br529/LoedVjWn+80etLo/0LSTegDLgvOp1wj5ltR4K/Y3efC9wGfE4I8KXAWyT7\ne87Y0u+14L9v2TZpCfLEM7NmwGPAZe6+rOprHi5NSMTlCWZ2HLDQ3d+Ku5Y8KgH2Ae5y977ASjZ0\ntwLJ+o4Boq7hEwl/xLQHtiOFrcykfa+SG2kJ8rlApyrPO0bbEsHMGhBC/EF3fzzavCDTnRrdL4y2\nF/rv4iDgBDObRThFchjh/HGLqAsWNj6m9ccbvb4DsDifBdeBOcAcd389ej6WEOxJ/Y4BjgD+4+5l\n7r4OeJzw3Sf5e87Y0u81Cd+3bIO0BPmbwK7RiNeGhEEz42KuqU5E5wHvBT509z9UeWkckBm9ehbw\nZJXtZ0YjYA8Allbpxqv33P0X7t7R3bsSvscX3H0YMBE4Odqt+vFmfg8nR/sXVAvH3ecDX5jZbtGm\nw4EPSOh3HPkcOMDMmkb/xjPHnNjvuYot/V6fAY4ys5ZRT8ZR0TZJC3dPxQ04BvgE+Ay4Nu566vC4\nDiZ0vb0HTI1uxxDODz4PfAr8G2gV7W+EEfyfAe8TRgXHfhxbeewDgaejxzsDbwAzgP8BGkXbG0fP\nZ0Sv7xx33Vt5rHsDU6Lv+QmgZdK/Y+C/gI+AacADQKOkfc/Aw4QxAOsIPS/nbs33CpwTHfsM4Oy4\nj0u3/N40s5uIiEgBS0vXuoiISCIpyEVERAqYglxERKSAKchFREQKmIJcRESkgCnIJdHMzM3s91We\nX2FmN9bB5zYys3+b2VQzO6XaawdEK3BNjVYquzHaPtDMDtzWny0iUlXJ5ncRKWhrgCFm9mt3X1SH\nn9sXwN33ruG1+4EfuPu70cp7mYlcBgIrgMl1WIeIpJxa5JJ05cBI4GfVXzCzrmb2QrS28/Nm1rmG\nfVqZ2RPRPq+Z2V5m1hb4O7Bf1OruXu1tbQmTfODuFe7+QbSgzY+Bn0Xv+Y6ZlZrZY2b2ZnQ7KPqZ\nN5rZA2b2arS+9I+i7e3M7OXo/dPM7Dt192sSkUKlIJc0GAEMi5YDrepPwP3uvhfwIHBnDe/9L+Cd\naJ9rCMtILgTOAya5+97u/lm199wOfGxm/2tmF5hZY3efBdxNWEt7b3efRJgj/nZ33w/4PtGyrJG9\nCPPIDwCuN7P2wOnAM1EvQB/CLH4iknLqWpfEc/dlZjYGuAT4pspLA4Ah0eMHgN/W8PaDCSGLu79g\nZq3NrPlmft5NZvYgYc7r04HTCN3q1R0B9KqybHbzaBU7gCfd/RvgGzObCPQnrBkwKlok5wl3V5CL\niFrkkhp/JMxjvV0+fpi7f+budxEW++hjZjWtjV0EHBC10Pd29w7uviLzEd/+SH8ZOISwstVoMzsz\nZwcgIgVDQS6p4O5fAY8SwjxjMmEFNYBhwKQa3jopeg0zGwgs8mrrvVdnZsfahmb2rkAF8DWwHNi+\nyq7PAj+t8r6qA+dONLPG0R8AA4E3zawLsMDd/0boht+ntjpEJB0U5JImvwfaVHn+U+BsM3sPOAO4\ntIb33AjsG+1zKxuWl6zNGYRz5FMJXfbD3L0CeAo4KTPYjdDV3y8aSPcBYTBcxnuEJTtfA37l7l8S\nAv1dM3sHOIVwjl1EUk6rn4nUM9F15yvc/ba4axGR+k8tchERkQKmFrmIiEgBU4tcRESkgCnIRURE\nCpiCXEREpIApyEVERAqYglxERKSAKchFREQK2P8Hz6ibBMmOc54AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f60aadd72b0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Multi Armed bandit \n",
    "\n",
    "\n",
    "def greedy_bandit(arms=10,std_dev_reward=1,init_expectation = 0,iterations=1000,epsilon=0.1):\n",
    "    #std_dev_reward - Standard devation of rewards about their means\n",
    "    #Iterations - No of iterations to cary one experiment\n",
    "    #epsilon - parameter\n",
    "    #init_expectaton -- Initial expectation value\n",
    "    \n",
    "    \n",
    "    no_of_actions = arms\n",
    "    optimal_action_values = list_optimal\n",
    "    \n",
    "    #Initiate expectation counter\n",
    "    expect = [init_expectation]*no_of_actions\n",
    "    #Keeps reconrd of number of occurence of action\n",
    "    act_occur = [0]*no_of_actions\n",
    "    #Average reward at each stage of iterations\n",
    "    avg_rward_list = [0]\n",
    "    #Average reward value\n",
    "    avg_rward_value = 0\n",
    "\n",
    "    def get_maxvalue_index(list_):\n",
    "        #Get list of indexes which contain maximum value in a list\n",
    "        indexes_of_max_values = np.argwhere(list_ == np.amax(list_)).flatten().tolist()\n",
    "        #return an index randomly\n",
    "        return random.choice(indexes_of_max_values)\n",
    "\n",
    "    \n",
    "    \n",
    "\n",
    "\n",
    "    counter = 0\n",
    "    #print (optimal_action_values)\n",
    "   \n",
    "    for t in range(iterations):\n",
    "\n",
    "        counter += 1\n",
    "       \n",
    "        # Explore with prob of epsilon\n",
    "        if random.random()<epsilon:\n",
    "            #act_index --> Index of arm upon which action is made\n",
    "            act_index = random.randint(0,no_of_actions-1)\n",
    "            #print (\"*\")\n",
    "        else:\n",
    "            act_index = (get_maxvalue_index(expect))\n",
    "        \n",
    "        #Increment counter\n",
    "        act_occur[act_index]+=1\n",
    "        #Reward at state t\n",
    "        rward_t = np.random.normal(optimal_action_values[act_index],std_dev_reward)\n",
    "        #Update expectation\n",
    "        expect[act_index] = (((expect[act_index]*(act_occur[act_index]-1))+rward_t)/(act_occur[act_index]))\n",
    "\n",
    "        avg_rward_value = ((avg_rward_value*(counter-1))+rward_t)/(counter)\n",
    "        avg_rward_list.append(avg_rward_value)\n",
    "        \n",
    "    return avg_rward_list,avg_rward_value\n",
    "\n",
    "\n",
    "def greedy_experiment(arms=10,std_dev_reward=1,init_expectation = 0,iterations=1000,epsilon=0.1,no_of_exp = 500):\n",
    "    \n",
    "    #no_of_ex --> No of experiments to carry to obtain average data\n",
    "    \n",
    "    #Average over experiments\n",
    "    rward_list = [0]\n",
    "    rward_value = 0\n",
    "    for x in range(no_of_exp):\n",
    "        avg_lists, avg_values = greedy_bandit(arms,std_dev_reward,init_expectation,iterations,epsilon)\n",
    "        if x == 0:\n",
    "            rward_list = avg_lists\n",
    "            rward_value = avg_values\n",
    "        else:\n",
    "            #Update averages\n",
    "            rward_list = np.add(np.dot(x,rward_list),avg_lists)/(x+1)\n",
    "            rward_value = ((x*rward_value)+avg_values)/(x+1)\n",
    "  \n",
    "    print (\"epsilon : \",epsilon,\"\\t average reward value\",rward_value)\n",
    "    return (rward_list)\n",
    "\n",
    "fig, axes = plt.subplots(nrows=1, ncols=1, figsize=(6, 6))\n",
    "axes.plot(greedy_experiment(epsilon=0.1),color='r', label=\"epsilon=0.1\")\n",
    "axes.plot(greedy_experiment(epsilon=0),color='b',label=\"epsilon=0\")\n",
    "axes.set_xlabel(\"No of Steps\")\n",
    "\n",
    "axes.set_ylabel(\"Average Reward\")\n",
    "axes.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## <a id=\"6\">Optimistic Initial Values</a>\n",
    "\n",
    "Initial action values can also be used as a simple way of encouraging exploration. The initial action-values $Q_1(a)$ acts as \"bias\" to our expectations. For eg, if we set the initial action-values to a high number, then when the real reward will be less than our expectation, our agent will be disappointed, the expectation from that action will decrease and our agent and will move on to explore other actions. Thus <i><span style=\"color:#f31818\">exploration is \"induced\" initially</span></i> by setting a high initial action-value.\n",
    "\n",
    "Conversely setting a low initial expectation will lead to more exploitation that exploration as our agent would be quite delighted initially because the results would be higher than our initial estimates\n",
    "\n",
    "<a id=\"6a\"></a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epsilon :  0 \t average reward value 2.3075967578038545\n",
      "epsilon :  0 \t average reward value 1.0601421046249047\n",
      "epsilon :  0 \t average reward value 0.7503940338068164\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7f607e22aeb8>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAdcAAAF3CAYAAAD3mAEvAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XmcXHWZ7/HPU1W9d9Ld2UP2kECI\nAcK+6mUMDCIQVBAFZ2BQRAcZUeY66oyODHNfc90FBlwQEXBcuIKDiGEYFEZwQQgKJIQACSQkIfvS\n6U5vtTz3j3Oqu9J0d6q761Slu75vXvWqOms9pyv0t3/n/Op3zN0RERGRwomVugAREZHRRuEqIiJS\nYApXERGRAlO4ioiIFJjCVUREpMAUriIiIgWmcBURESkwhauIiEiBKVxFREQKTOEqIiJSYIlSFzBY\nEyZM8NmzZ5e6DBGREeWZZ57Z4e4Th7mPSYlE4nZgEeXdOMsAK1Op1JXHHXfctr5WGHHhOnv2bJYv\nX17qMkRERhQzWz/cfSQSidunTJlyxMSJE3fHYrGyHZg+k8nY9u3bF27ZsuV2YGlf65TzXx4iIjI4\niyZOnLi3nIMVIBaL+cSJE5sJWvB9r1PEekREZGSLlXuwZoU/h34zVOEqIiIjxtq1ayuWLFly6KxZ\nsxZNnz79yMsuu2xme3u7bdmyJX7SSScdVltbe8xll102czD7vPnmm8c3NTUdvWDBgoULFixY+PWv\nf33CcOtUuIqIyIiQyWR417veNW/p0qV71q9fv3LdunUrOjo67Oqrr55eW1vrN9xwwxvXX3/9xqHs\n+/zzz9+9evXqVatXr1513XXX7RhurQpXEREZEX7xi1+Mqaqqylx77bU7ARKJBN/+9rc33HfffePd\nnbPPPru1uro6U+o6YQT2FhYRkdL71L3PzXh5S0ttIfd52JQxbV+56OgN/S1fsWJFzdFHH92WO2/c\nuHGZadOmdb3wwgtVp556antf25177rlz165dW917/jXXXLP1mmuu2Qnw0EMPNR522GH1c+fO7bjl\nlls2zJs3LzmcY1G4iojIqPbLX/7y1YGWX3zxxXs+/OEP76qpqfGvfOUrE/7qr/5qzpNPPvnycN5T\n4SoiIoM2UAszKosWLWq///77m3Ln7dq1K7Zjx47EUUcd1dHfdgdquU6ZMiWdnffJT35yxw033DB9\nuLUqXEVEZERYunRpy+c+97nYLbfcMv6aa67ZmUqluPrqq2d88IMf3FZfX9/vV4QO1HJdv359xaxZ\ns5IAP/rRjxrnzp3bb1DnS+EqIiIjQiwW4/77719z1VVXzfrKV74yddeuXYnzzz9/95e+9KUtANOm\nTTuytbU1nkwm7eGHH25ctmzZy8cdd9wBg/LLX/7ypIcffrgxHo97Y2Nj6s4771w33FoVriIiMmLM\nmzcv+eijj64BeOSRR+ouv/zyub/97W9rTz/99LZNmzatGMo+b7311k3ApkLWqXAVEcmTu5NxSGec\njDupjAevM07ag9fZhztk3HHCZw+2d3q2z66Tya6bfZ0Jnj1nWcadGeNqOXRifal/DAeNs846a98b\nb7wxpECNmsJVRAaUSmfoSmdIhSGS8Z55XakM6UxPyKRzQiaV7gmgVLh9OuMk0z3bZEMp070t+wVV\n7uvs9qm0k8707G//50y4PNx/TuDlhmHPOm+uP9MrJIP66K6jlD76vw7lM+csKGkNkh+Fq0iJpDNO\nRzJNezJNe1d6v9dtyTQdXcF0RzJDZypNMp0hmXY6U5ngdfY5J7xS6SAwkulgXjK9f5hlwy0VPidT\nGbrSPYGyX7B1P5f6JwUxg0QsRjxmJGJGPB4+x2z/+dl5cSNuRizW81wRj1FdkV0vtv9+wnUSsZ5t\n4jn7i1nvZRCPxYjHINZr3ey+YmbELFhuBmaGQc/8PtYJpoN5tt96wfTksW/q8CoHKYWrlB13pyud\nCQMtEwZYuvu5IyfQsi2cZMZJZwOsV5glM0ELLvvoTGWCfWSfc/aZfY/OZNDyG6qKeBAWiZhRmYiR\niMVIhEGR2G9+sF5VRYzaMFBiZlQlYj37iO8fCtlgyb6uTMS695UbJJWJGJXZ7XsHTE7QZOuJh7X0\nDsJ47n7NiMXoY54V8F+ASPQUrjIipDPOvq4UbZ1BC64zlWZvR4q97cnu59bOFPs6U93P+zrT+81r\n6UjR0pGkrStNqkDNsZhBIh6jKh7rDqHKRIyaijhVFXGqEzHG1VVSnYhTXRGjuiKe84hRWxmnJpyu\nCV/XVMSpzn1dEQ/CMAyzirhhprAROZgpXKVo3J3WzhQ7WrvY3NzOrn1d7N7Xxa59SXa3dXUHZDYQ\nWzpStHSmaO1I0Z5MH/gNCMKurjJBXVWCuqo49VUJaisTzBxXy5jqCsZUB/NrKxNBeHWHW0/wZQMt\nt+WXbW0FrcPYfqcdRUR6U7jKkLk721s72bS7nW0tnTS3J9m1r4tNu9vZ3dZFa2fQomxuT9LcHrzu\n71TomOoEDTUV1FclqK9K0FRXyfRxtYwJp+vC59qqOJVhK7GhpoKxNRWMra5gbHWC+uogMNWqExm9\n1q5dW3HVVVfNXLNmTU06nba3v/3tzd/5znc2NDc3xy644IJDV6xYUXfRRRftvPvuu1/Pd5/t7e12\n0UUXzVmxYkVtY2Nj6qc//emrhx9+eNdw6lS4Sr9S6QybmzvYtKedTbvb938OH12pN4fl2OoE4+ur\nugNzakMNY2sqaKipYFxdBePrqpjaUM24+krG1VXSVFtJRVw3aBKRgWVvOXfllVduu/baa9emUiku\nvfTSWVdfffX0m266adMNN9zwxnPPPVezcuXKmsHs96abbprQ0NCQev3111fedtttTdddd930A43q\ndCAK1zLm7qzd3sr6nW1s3dvJpj1t+4Xolr0db+opOqG+kmmNNRwxdQxnHjGJaY01TGuqZcrYahpr\nK2iqq6S+Sv+sRKTw+rvl3OzZs4+68cYbN5199tmtL730UtVg9/vggw82Xn/99W8AXHHFFbs//elP\nz8xkMsRiQ/+jX78Fy4C78/quNl7e2sprO1pZvbmFtTv28er2Vlo6Ut3rxWPGlLHVTGuq4eS545nW\nVBOGZ/B8SGMN1RXxEh6JiBw07v/YDLatKugt55i0sI133Vr0W85t3bq1cs6cOV0AFRUV1NfXp7du\n3ZqYOnVq6s17y4/CdRTava+L5zbu4dkNweO5DXvY3dZza8IpY6s5dFId71o8jSOnNXDYlDFMGlPF\npDFVJHR6VkRGmeGe4h0KhesI15lKs+qNvTy3oSdM1+0M/rAzg/mT6jlr4WQWz2jiiKljmD2+jqa6\nyhJXLSIj3gAtzKhEdcu5yZMnd7322muVhx56aDKZTNLa2hqfPHnykFutoHAdcTIZ57mNe3j4ha08\n+epOVr2xt7sH7qQxVSye0cjFJ8xg8YxGjpreqOufIjJqRHXLuXPPPXfPHXfcMf7MM8/c9/3vf7/p\nlFNOaRnO9VZQuI4I7s6Tr+7ivj9t5PGXt7OtpZNEzFg8o5ErTpvN4hmNLJ7ZyNSGQXWQExEZUaK6\n5dy1116748ILL5wzc+bMRQ0NDel77rln7XBrVbgexPa0dXHvMxv50VOv8+r2fYytTvDW+RNZcsQk\nliyYTENtRalLFBEpqihuOVdbW+sPPfRQQa/LKlwPQi+80cx3H3+VZSu30JXKcOzMRr723qM596ip\n6q0rIhLSLeckL8+s382tj63h0dXbqK9K8L7jZ3DpSTM5YurYUpcmIiKDoHAtMXfn0dXb+N5vX+P3\na3fSWFvBdWcdxuWnzNZpXxGREUrhWkIbdrVxw4OreGTVViaPreKf3nkEl540kzr18BURGdH0W7wE\n3J17nt7ADQ+uwh0+e84CPnT6HA3gICIySihci2xvR5Jrf/xnHntpO6ceOp6vvvdoDmnUV2hEREYT\nNZWK6MlXd3LOjU/wxCs7+Jelb+E/PnSSglVEZBDWrl1bsWTJkkNnzZq1aPr06UdedtllM9vb223L\nli3xk0466bDa2tpjLrvsspnDeY8TTzzx8NmzZy9asGDBwgULFizctGnToBuiCtci6Eyl+b/LXuSS\n7z5JRdy45yMnc/mps3WjbRGRQcjecm7p0qV71q9fv3LdunUrOjo67Oqrr55eW1vrN9xwwxvXX3/9\nxkK819133/3q6tWrV61evXrVtGnTBj0UosI1Yvs6U1x+x1N85/FXef8JM/nlx9/KcbPGlbosEZER\np79bzt13333j3Z2zzz67tbq6+s03mS4BXXONUGtniiu+/xTPrN/N1y8+mvccO73UJYmIFMTnf/f5\nGWt2rynoLefmNc1r+9fT/rXot5zra5srr7xydiwWIxxecfNgxxpWuEakpSPJ5Xc8xXMbm7n5kmM4\n76hDSl2SiEhZGuwt5+65555X58yZk9y9e3fsvPPOO/Sb3/zm+P5CuD8K1wi0hqeCn9/YzC2XHMM5\nR04tdUkiIgU1UAszKlHccu6jH/3ozkWLFi0EeMc73rHnxhtvfGPOnDlJgKampsz73ve+XU899VQd\noHAtpX2dKT74/ad5TsEqIlJQUd1ybvXq1auyr5PJJDt27EhMnTo11dnZacuWLWt4+9vf3jLYWtWh\nqYA6kmkuu+Mplq/fxY3vW6xgFREpoOwt5372s581zZo1a1FTU9PiWCxG7i3nPv/5z8+49957x0+e\nPPmoZ5555k2t1QNpb2+PnXnmmfMPO+ywhW95y1sWTp06NXnddddtH+x+1HItoC8+tJpn1u/mlkt1\njVVEJApR3HIu19ixYzMvvPDCi8Pdj8K1QB5dvZU7f7+OD542R8EqIlIEB/Mt53RauAC2tXTwqZ8+\nz4IpY/iHdxxe6nJERKTE1HIdpkzG+dRPn6e1M8VPrjpZNzMXERG1XIfrzt+v4zcvb+dz5y1k/uQx\npS5HRCRKmUwmo3FbgfDn0O9oUArXYXh5awtf/K/VnHnEJP7qpGGNEy0iMhKs3L59e0O5B2wmk7Ht\n27c3ACv7W0enhYconXH+4d7nqauM88ULj8KsrP+tiUgZSKVSV27ZsuX2LVu2LKK8G2cZYGUqlbqy\nvxUUrkP0yKqtPLthD1+/+Ggm1FeVuhwRkcgdd9xx24Clpa5jJIjsLw8zm2Fmj5nZKjN7wcyu7WMd\nM7ObzWyNmT1vZsdGVU8huTt3/PY1Jo+tYunR+tqNiIjsL8pmfQr4e3dfCJwMfMzMFvZa5xxgfvi4\nCvhWhPUUzO/W7OSpdbu4+ox5JOLlfGZERET6ElkyuPtmd/9T+LoFeBGY1mu1C4C7PfAk0GhmB/2Y\ngTf+6mWmNlTzvhNmlLoUERE5CBWl2WVms4FjgD/2WjQNyL2zwkbeHMAHleXrdrF8/W4+8ra5+k6r\niIj0KfJwNbN64D7gE+6+d4j7uMrMlpvZ8u3bBz1+ckH9x5PrGVOd4GK1WkVEpB+RhquZVRAE6w/d\n/Wd9rLIJyE2p6eG8/bj7be5+vLsfP3HixGiKzcOeti6WrdzCu4+ZRm2lOlqLiEjfouwtbMD3gBfd\n/ev9rPYAcFnYa/hkoNndN0dV03D955830ZXK8P4TNGCEiIj0L8rm12nAXwMrzOzZcN4/AjMB3P3b\nwDLgncAaoA24IsJ6hsXd+clTGzh6egMLDxlb6nJEROQgFlm4uvtvgQGHLXJ3Bz4WVQ2FtHLTXl7a\n2sK/vfvIUpciIiIHOX1JM0+PrNpCzOCcRVNKXYqIiBzkFK55+tWL2zhuVhNNdZWlLkVERA5yCtc8\nbNrTzqrNeznziMmlLkVEREYAhWseHn1xKwBLFK4iIpIHhWsefvXiNmaPr+XQiXWlLkVEREYAhesB\ntHel+cPanSw5YrLu2SoiInlRuB7Anzfspiud4bR540tdioiIjBAK1wN46rVdmMFxs8aVuhQRERkh\nFK4H8PS6XRwxZSwNNRWlLkVEREYIhesAulIZnlm/mxPnqNUqIiL5U7gO4IU3mulIZhSuIiIyKArX\nATy/sRmAxTMaS1yJiIiMJArXAazY1Mz4ukqmNlSXuhQRERlBFK4DWLmpmSOnN+j7rSIiMigK1360\nd6V5ZVsrR05rKHUpIiIywihc+7Fq817SGWeRwlVERAZJ4dqPlZuCzkxquYqIyGApXPuxYlMzE+rV\nmUlERAZP4dqPF97Yy8JD1JlJREQGT+Hah3TGWbu9lcMn15e6FBERGYEUrn14fVcbXakM8yePKXUp\nIiIyAilc+/Dy1hYA5k9Sy1VERAZP4dqHNdtaAdRyFRGRIVG49uHlrS1Ma6yhvipR6lJERGQEUrj2\n4eWtrczTKWERERkihWsvmbCnsK63iojIUClce9m8t4OuVIa5ExWuIiIyNArXXtbv2AfA7PG1Ja5E\nRERGKoVrL+t2tgEwa0JdiSsREZGRSuHay/qd+6hMxJg6VmMKi4jI0Chce9mwu43pjTXEYhpTWERE\nhkbh2sum3e1Ma6opdRkiIjKCKVx72bi7nekKVxERGQaFa472rjQ793UxrVHhKiIiQ6dwzbFpTzsA\n05v0NRwRERk6hWuOzc1BuE5tUE9hEREZOoVrjs3NHQAcotPCIiIyDArXHJv3BOE6aWxViSsREZGR\nTOGaY8vedibUV1GViJe6FBERGcEUrjk2N3foequIiAybwjXHluYOpihcRURkmBSuOdRyFRGRQlC4\nhtq6UjS3J9VyFRGRYVO4hrq/htOgr+GIiMjwKFxDW8JwVctVRESGS+EayrZcdc1VRESGS+Ea2tYS\nhOvEMRpAQkREhkfhGtrV2kVtZZzaykSpSxERkRFO4Rraua+L8fWVpS5DRERGAYVraEdrJ+PqdEpY\nRESGT+Ea2tnaxYQ6tVxFRGT4FK6hXTotLCIiBaJwBdydnfs6GV+v08IiIjJ8Cldgb0eKZNoZr9PC\nIiJSAApXglPCgE4Li4hIQUQWrmZ2h5ltM7OV/Sw/w8yazezZ8PHPUdVyIDtbOwEYr97CIiJSAFGO\nmHAncAtw9wDrPOHu50VYQ152tAYt13E6LSwiIgUQWcvV3R8HdkW1/0LauS9ouU5QhyYRESmAUl9z\nPcXMnjOzh8zsLaUqYpdariIiUkClHEj3T8Asd281s3cC9wPz+1rRzK4CrgKYOXNmwQvZua+LsdUJ\nKhOl/ltDRERGg5KlibvvdffW8PUyoMLMJvSz7m3ufry7Hz9x4sSC17JrXxdNarWKiEiBlCxczWyK\nmVn4+sSwlp2lqKWlI8nY6opSvLWIiIxCkZ0WNrMfA2cAE8xsI/AFoALA3b8NXAT8rZmlgHbg/e7u\nUdUzkNbOFPVVutWciIgURmSJ4u6XHGD5LQRf1Sm5lo4UM8fVlroMEREZJfoNVzP7d6DflqS7fzyS\nikqgpSNFfbVariIiUhgDXXNdDjwDVAPHAq+Ej8XAqOr9o2uuIiJSSP0219z9LgAz+1vgdHdPhdPf\nBp4oTnnRc3ddcxURkYLKp7dwEzA2Z7o+nDcqtHWlyTiM0WlhEREpkHwS5YvAn83sMcCAtwHXR1lU\nMbV2pgB0zVVERApmwEQJv4f6K+Ah4KRw9qfdfUvUhRVLS0cSgDG65ioiIgUyYLi6u5vZMnc/Evh5\nkWoqqpaOoOU6RtdcRUSkQPK55vonMzsh8kpKpDtcdVpYREQKJJ9EOQn4gJmtB/YRXHd1dz8q0sqK\nRNdcRUSk0PJJlLMjr6KEdM1VREQK7YDh6u7rAcxsEsGAEqNK9rSwvucqIiKFcsBrrma21MxeAV4D\nfgOsI+g9PCooXEVEpNDy6dD0r8DJwMvuPgdYAjwZaVVF1NqZoq4yTjxmpS5FRERGiXzCNenuO4GY\nmcXc/THg+IjrKpqWjqSut4qISEHlcy50j5nVA48DPzSzbQS9hkeF1k7dEUdERAorn5brBUAb8Eng\nv4C1wPlRFlVMLR0pfcdVREQKKp9UeT/wuLu/AtwVcT1Fp3AVEZFCy6flOhP4jpm9ZmY/NbO/M7PF\nURdWLK2dKd3LVURECuqA4eruX3D3twMLCe7j+imCm6iPCi0dSX0NR0RECuqAqWJmnwNOI7iP65+B\n/80oull6q04Li4hIgeWTKu8BUsAvCQaR+IO7d0ZaVZGkM86+rrR6C4uISEHlc1r4WOBM4CngLGCF\nmf026sKKITtov77nKiIihZTPaeFFwFuB/0UweMQGRslp4e5B+3XNVURECiifVPkiQZjeDDzt7slo\nSyqefZ1pAOoUriIiUkD53BXnPDOrAWaOpmAFaOsKTgvXVsZLXImIiIwm+dwV53zgWYLRmTCzxWb2\nQNSFFUN7Mmi51ihcRUSkgPIZROJ64ERgD4C7PwvMibCmomnvCsO1QuEqIiKFk+9dcZp7zfMoiim2\ntjBcdVpYREQKKZ+ePC+Y2aVA3MzmAx8Hfh9tWcWh08IiIhKFfFqufwe8BegEfgTsBT4RZVHFotPC\nIiIShXx6C7cB/xQ+ADCzmcDrEdZVFNmWa22lvoojIiKFM2DL1cxOMbOLzGxSOH2Umf0I+F1RqotY\n9pprVSKfBryIiEh++k0VM/sKcAdwIfBLM/s/wH8DfwTmF6e8aHUk09RUxInFrNSliIjIKDLQ+dBz\ngWPcvcPMmgiGPVzk7uuKUlkRtHWl1JlJREQKbqDzoR3u3gHg7ruBV0ZTsEJwWlidmUREpNAGarnO\n7TUS05zcaXdfGl1ZxdGRTKvlKiIiBTdQuF7Qa/prURZSCm1daQ0gISIiBddvuLr7b4pZSCm0d6Wp\n1mlhEREpsLL+Dkp7Ui1XEREpvPIOV3VoEhGRCOQdrmZWG2UhpdDWpQ5NIiJSePncz/VUM1sFrA6n\njzazb0ZeWRHotLCIiEQhn5brN4CzgZ0A7v4c8LYoiyoWnRYWEZEo5HVa2N039JqVjqCWospknPZk\nmhoN2i8iIgWWT7JsMLNTATezCuBa4MVoy4peZyoD6HZzIiJSePm0XD8KfAyYBmwCFofTI1r3jdIr\nyrrDtIiIRCCf+7nuAD5QhFqKqjMV3m5OLVcRESmwA4armd3cx+xmYLm7/7zwJRVHV3hauDKulquI\niBRWPslSTXAq+JXwcRQwHfiQmd0YYW2Ryl5zrdJpYRERKbB8OjQdBZzm7mkAM/sW8ARwOrAiwtoi\npZariIhEJZ9kaQLqc6brgHFh2HZGUlUR6JqriIhEJZ+W65eBZ83sfwAjGEDi38ysDvhVhLVFqlMt\n14ObO6STYAaZFHS1QXIfpLogFoNYInhYDDwTPjzndThN73m91wtf97le7ro5y/Fw3yGz7ItgWSYd\n1OxpyGRyXqeCZZ7pOUbPhMt6bZOd1/2cydk+d18OsXjwc8jW4Zmebbp/BgM89z6m3GPr+UD6+Fn1\nNZ3POjnvl+8+891fru7PJfvZ9DW/17Lsse73c+jr5+L7PfW7zVBkjzf7WXf/uwNO+wSc9S9D37cU\nTT69hb9nZsuAE8NZ/+jub4SvPxVZZREb9ddcu3/phL84Uh3BI9kePHftg0wyWJ5sh67W4NHZCsm2\nMLziPb/0M6lgvbZd0L4L2nYG0xYLf7nHg+fsL/ZMKicQwiDYL0Cy83PWyQZJqhPSI/akSOFYrOfn\nGkuEr3P+sIiF//tmfwlD8LnnfiYGYGGg9PUc63/Zm+oJl1ls/233m86+tw2wTu50PvscxPrdcsKt\nr1Dsd5n1HCv0P53POm8K8QG4v3n97s8w1nOcM0/Of59SUvkOT9QBbCbo3DTPzOa5++PRlRW9ol9z\ndYd924NH+x5IdwX/syRzQi+5L3jubA3WS3eGv0AtWGfvG2EIhuuZQcfeXi2aMLiG85fzQKrGQk0T\n1I6DirrgvVKdPe9tub/841BRkxMMiZ6gyH3uXhZOxyshUQXxiuAwYnGorIOK2mD+fiGdDpbv94s2\n3vMLN/eXb+8Q2G95H/P7XJdev8j7aNW86bhiOT+DeK+gyAZS7ja9WqIiMuLk81WcKwlGZZoOPAuc\nDPwBeHu0pUUr23KtHkrLtastCL62XbBxOYw9JAg7gFQ77HgFmjcGgdi6FXaugdZtwXS+apqCMEkn\ng+lEFYyZAtWNMHYaJKoBh+oGiFX0/ELO/SVt2WPzYPtEDVRUB9tW1gXbeQYqa6GyPnhU1ee8r+cE\nYUVP4ImIyIDyableC5wAPOnuf2FmC4B/i7as6PW0XA/QoWnHGljzSBCiW1fCng1BC/NAqhuCVl7d\nRJh+YhDA9ZODgKybEIRWuitYL1ETtPAqanue4xrzWERkpMrnN3iHu3eYGWZW5e6rzezwA21kZncA\n5wHb3H1RH8sNuAl4J9AG/I27/2mQ9Q9ZT2/hXi1Xd1j/e3hpGbz6P0GgQhCGkxfBsZeFrbuaoBU5\n46SglZqo6jltOf7QoBWo03oiImUpn3DdaGaNwP3AI2a2G1ifx3Z3ArcAd/ez/Bxgfvg4CfhW+FwU\nfV5z3fEK3PYX0NUSTE9cAO/4Ihx+DjTMDK6d9WXKm/52EBGRMpZPb+F3hy+vN7PHgAbgv/LY7nEz\nmz3AKhcAd7u7A0+aWaOZTXX3zQcue/j26y2cScOvvgC///dg4ZIvwGFnw+S3FKMUEREZZQYMVzOL\nAy+4+wIAd/9NAd97GpB7n9iN4bw3hauZXQVcBTBz5syCvHm25Vr18i/hPz8cdFCqaYKlt8AR5xXk\nPUREpDwN2FU2HIXpJTMrTKINkbvf5u7Hu/vxEydOLMg+O1NpJsX2Ev/53wbBesZn4VOvKlhFRGTY\n8rnm2gS8YGZPAd3dZN196TDfexMwI2d6ejivKJLJLm6v+ErQY/fKX8P044v11iIiMsrlE66fj+i9\nHwCuMbOfEHRkai7W9VaAUzfczlG2Fs77dwWriIgUVD4dmn5jZrOA+e7+KzOrBQ442r2Z/Rg4A5hg\nZhuBLwAV4T6/DSwj+BrOGoKv4lwx1IMYisObnwhevOXdA68oIiIySPmM0PRhgs5E44BDCTodfRtY\nMtB27n7JAZY78LG8Ky2kfTuY3LGO71W8nw9VjSlJCSIiMnrlM/bfx4DTgL0A7v4KMCnKoiK3+kFi\nZPhjRdG+VisiImUkn3DtdPeu7ISZJYhsVPgiWfUA2xKHsLFyXqkrERGRUSifcP2Nmf0jUGNmZwE/\nBX4RbVkRymTg9T/wbPUJVOpzj1L2AAAT6UlEQVRG6SIiEoF8wvUzwHZgBfARgo5In4uyqEjtfg2S\nbbwan01VYpTey1VEREoqn6/ivItgmMLvRl1MUay4FzCejR9JpcJVREQikE+6nA+8bGY/MLPzwmuu\nI9frf4Api1jvU6hK6LSwiIgU3gHD1d2vAOYRXGu9BFhrZrdHXVgkMhnY9CeYdjxdqfSbbzcnIiJS\nAHmli7sngYeAnwDPEJwqHnn2rIfOZjjkGDpTGariClcRESm8A6aLmZ1jZncCrwAXArcDUyKuKxp7\nXg+ex82hK5VRy1VERCKRz/XTy4B7gI+4e2fE9USrObzDXcMMOlOv7H+jdBERkQLJ55rrJe5+fzZY\nzex0M7s1+tIisGcDYDB2WthyVYcmEREpvLx6/prZMcClwHuB14CfRVlUZJo3wJipkKikM5VWy1VE\nRCLRb7ia2WEEvYMvAXYQnBo2d/+LItVWeHs3QcM0UukMGUeDSIiISCQGarmuBp4AznP3NQBm9smi\nVBWV9t1QP4XOVAZAg0iIiEgkBkqX9wCbgcfM7LtmtgSw4pQVkfbdUNNIVxiuarmKiEgU+k2XsBPT\n+4EFwGPAJ4BJZvYtM/vLYhVYUO3NUN2Y03JVhyYRESm8fHoL73P3H7n7+cB04M/ApyOvrNAy6WAA\nCbVcRUQkYoNKF3ff7e63ufuSqAqKTDoZPCeq6UylAV1zFRGRaJRRuvTc371TLVcREYlQ2aRLOhME\n6rMbm9VbWEREIlU26ZIMTwU/tHIrezuCU8S65ZyIiEShbMI11xXffxpQy1VERKJRNuniHlxzzV55\nnTuhjiOmjildQSIiMmrlNbbwaOAeXGd1jAuPnc7XLj66xBWJiMhoVTbhmvUXCyZzqoJVREQiVDbh\nmj0tXIzxG/d07KEt1Ubc4rze8jp7O/fSnm6nKl7FuOpx1CZqiVmMmWNnErc4yUySVCYFwI72HdRV\n1NFU3URVvAp3pyPdQXuqnY5UBx3pDipjlVTGK3F3WpOtpDIpquJVVCeqScQSVMYrqYxVEo/F6Up3\n0ZHqoDPdSTKTJO1pMplM8OwZMt7zOvfZMOIWJx6LE7MYMWLEY3EqYhVkPENFvIKEJUhlUpgF62af\nYxbrfmSnzYx0Jth/VbyKqngVnelOMmRIWIJ4LE7CEphZ988jmQ7qdRx3x8y6183uF8BzvmaV/Zy7\np/tYtt+88HXv7XrP617vAPsbcNs+5vW1zYHq617uDGr9/d6zr20H+/PJc9v93/bA9fX1XrnMjOx/\nMYuB0T3d3/r97muQvxH6+tz6XfcAxzGUfY6vHs/kusl5ry+lU0bhCt9qHMskWoa9r45UB3s697B6\n12reaH2DZ7Y+w4aWDWxv305LVwud6cLcU74mUROETCZZkP2JyMj2wUUf5JPHjez7p5SLsgnXjlQ7\n32xqpCrzKBcOctuWrhb+Z8P/sGLHCp7e8jRr96zd76/NCTUTOLzpcBaOX8iYyjGMrx5PQ1UDyUyS\n8TXjmV4/nap4FftS+9jdsZuudBdd6S7W7V1HIpYgYcHHYGZMrJnYvd6ezj0kYgnqK+qpq6ijNlFL\nVbyKrkywPcCYyjFUxCpoT7XTle4imUkG+890kc6kg1ZioorqeNCqjVucWKynRdnfs+PdrdwMQQs3\nlUl1t1SzrcpELNHdAs59pD2Nu+/XQo7H4hhBy7Qz3UlVvArDSHuatKeDPyIcKuIVVMQquus1DDPD\n3Ul5inQm3b1NtuWR2zrpntfHst7r5C7va15f2wz0Xv1u29d7DLC/A27bx7whv1fOLgY6nr72N6Rt\n8/jZ9vVenv3Pfb9WbsYz/bb+BmwV5rTeB9WCHdSq+a2c73ozx87M/82lpMomXLP/i3WS6nN5Z7qT\nP27+I82dzexL7sMw1res53ebfseGlg0kM0kM45RDTuGsWWcxoWYCk2oncXjT4UyqnUQ8pu/MiohI\noHzCNewtnNWZ7mTD3g3cteouVu5YyZo9a960TSKWYEHTAi467CJOn3Y6px1ymkJUREQOqHzCNdMT\nrl99+qvcveru7lNGM8fMZF7jPC5/y+XELMZhTYfRWNXI2Mqx1FbUlqpkEREZoconXHN67t216i7m\nNsxlbsNc3jb9bbx7/rtLWJmIiIw25ROuvTo2XHnklZx/6PklqkZEREazsglXwmuu062RG/7y6xw7\n6dgSFyQiIqNV2YRrJrzmekpsDidMOaHE1YiIyGhWNgP3Zw68ioiISEGUT7hmgvu5Dna4MxERkcEq\nm3DNDiMx0DijIiIihVA24erdJ4YVriIiEq2yCdeMslVERIqkbMI1O/yhrrmKiEjUyiZcs/2FFa4i\nIhK1sglXz2RHaFK4iohItMonXMNndRYWEZGolU24Zlw9mkREpDjKJlyzd8XR91xFRCRq5ROu+p6r\niIgUSfmEa/hF17I5YBERKZmyyZqeu7mq5SoiItEqn3BVhyYRESmS8gnX7MD9ClcREYlY2YRrJjuI\nhLJVREQiVjbhqrGFRUSkWMomXNH3XEVEpEjKJ1xDarmKiEjUIg1XM3uHmb1kZmvM7DN9LP8bM9tu\nZs+GjyujqiXj6ah2LSIisp9EVDs2szhwK3AWsBF42swecPdVvVa9x92viaqOrOzwh+rRJCIiUYuy\n5XoisMbdX3X3LuAnwAURvt+AsgP365qriIhELcpwnQZsyJneGM7r7UIze97M7jWzGVEV09NuVbiK\niEi0St2h6RfAbHc/CngEuKuvlczsKjNbbmbLt2/fPrR3UstVRESKJMpw3QTktkSnh/O6uftOd+8M\nJ28HjutrR+5+m7sf7+7HT5w4cUjFZEdoUsNVRESiFmW4Pg3MN7M5ZlYJvB94IHcFM5uaM7kUeDGq\nYnoGaFK6iohItCLrLezuKTO7BngYiAN3uPsLZnYDsNzdHwA+bmZLgRSwC/ib6OrRwP0iIlIckYUr\ngLsvA5b1mvfPOa8/C3w2yhp63ljDH4qISHGUukNT0ajlKiIixVI+4Ro+x9RbWEREIlY+4aoRmkRE\npEjKKFzD08LKVhERiVj5hGt4YlinhUVEJGrlE646LSwiIkVSNuGKWq4iIlIkZROumXCIJlfLVURE\nIlY24Zq95qpBJEREJGrlE64aoUlERIqkfMK1exgJERGRaJVNuBL2Ftb9XEVEJGplE64993Mtm0MW\nEZESKZukyX7PVe1WERGJWvmFq04Li4hIxMonXAl7CytcRUQkYuUTrtlLrjoxLCIiESujcNX3XEVE\npDjKJ1y7ewsrXEVEJFplE66o5SoiIkVSNuGqa64iIlIs5ROuYW9hnRYWEZGolU24Zi+5xtRyFRGR\niJVNuKpDk4iIFEv5hKvrrjgiIlIcZROume7ewmVzyCIiUiJllDQaW1hERIqjbMK1+6s4ClcREYlY\n+YRrtuWq3sIiIhKxsgnX7AhNylYREYla2YRrBnVoEhGR4iifpOm+5lo+hywiIqVRNknTPYiEiIhI\nxMomXLPdhWPqLSwiIhErm3DV8IciIlIs5ROu+p6riIgUSfmEK7pZuoiIFEfZhGu26apwFRGRqJVN\nuGa6xxYum0MWEZESKZ+kUctVRESKpOzCtYyOWERESqRsoibTPYiEWq4iIhKtsgnX7kEkyuiQRUSk\nNMomaVw3SxcRkSIpm3BF4SoiIkVSNuHqrmuuIiJSHGUUrsGzWq4iIhK18glXsh2aFK4iIhKtsglX\nNEKTiIgUSdkkTcaDgft1yzkREYla2YRrloY/FBGRqJVNuGZ7C6vhKiIiUSufcO0e/jBe0jpERGT0\nK5tw1SASIiJSLGUTrtnTwjGFq4iIRCzScDWzd5jZS2a2xsw+08fyKjO7J1z+RzObHV01armKiEhx\nRBauZhYHbgXOARYCl5jZwl6rfQjY7e7zgG8AX4qqHjT8oYiIFEmULdcTgTXu/qq7dwE/AS7otc4F\nwF3h63uBJRZR0zL8lqtariIiErkow3UasCFnemM4r8913D0FNAPjI6kmHERC4SoiIlEbER2azOwq\nM1tuZsu3b98+pH1MaJjBkZ0VNNRPKHB1IiIi+0tEuO9NwIyc6enhvL7W2WhmCaAB2Nl7R+5+G3Ab\nwPHHH++9l+fjvUuu4b1LrhnKpiIiIoMSZcv1aWC+mc0xs0rg/cADvdZ5ALg8fH0R8Kj33HhVRERk\nRIqs5eruKTO7BniYYFikO9z9BTO7AVju7g8A3wN+YGZrgF0EASwiIjKiRXlaGHdfBizrNe+fc153\nAO+NsgYREZFiGxEdmkREREYShauIiEiBKVxFREQKTOEqIiJSYApXERGRAlO4ioiIFJjCVUREpMAU\nriIiIgWmcBURESkwhauIiEiB2UgbJ9/MtgPrh7j5BGBHAcsZCXTM5UHHXB6Gc8yz3H1iIYuR/o24\ncB0OM1vu7seXuo5i0jGXBx1zeSjHYx6pdFpYRESkwBSuIiIiBVZu4XpbqQsoAR1zedAxl4dyPOYR\nqayuuYqIiBRDubVcRUREIlc24Wpm7zCzl8xsjZl9ptT1FIqZzTCzx8xslZm9YGbXhvPHmdkjZvZK\n+NwUzjczuzn8OTxvZseW9giGxsziZvZnM3swnJ5jZn8Mj+seM6sM51eF02vC5bNLWfdQmVmjmd1r\nZqvN7EUzO6UMPuNPhv+mV5rZj82serR9zmZ2h5ltM7OVOfMG/bma2eXh+q+Y2eWlOBbZX1mEq5nF\ngVuBc4CFwCVmtrC0VRVMCvh7d18InAx8LDy2zwC/dvf5wK/DaQh+BvPDx1XAt4pfckFcC7yYM/0l\n4BvuPg/YDXwonP8hYHc4/xvheiPRTcB/ufsC4GiCYx+1n7GZTQM+Dhzv7ouAOPB+Rt/nfCfwjl7z\nBvW5mtk44AvAScCJwBeygSwl5O6j/gGcAjycM/1Z4LOlriuiY/05cBbwEjA1nDcVeCl8/R3gkpz1\nu9cbKQ9gOsEvnbcDDwJG8MX6RO/PG3gYOCV8nQjXs1IfwyCPtwF4rXfdo/wzngZsAMaFn9uDwNmj\n8XMGZgMrh/q5ApcA38mZv996epTmURYtV3r+R83aGM4bVcJTYccAfwQmu/vmcNEWYHL4ejT8LG4E\n/gHIhNPjgT3ungqnc4+p+3jD5c3h+iPJHGA78P3wVPjtZlbHKP6M3X0T8FXgdWAzwef2DKP7c84a\n7Oc64j/v0ahcwnXUM7N64D7gE+6+N3eZB3/Ojopu4WZ2HrDN3Z8pdS1FlACOBb7l7scA++g5VQiM\nrs8YIDyteQHBHxaHAHW8+fTpqDfaPtdyUi7hugmYkTM9PZw3KphZBUGw/tDdfxbO3mpmU8PlU4Ft\n4fyR/rM4DVhqZuuAnxCcGr4JaDSzRLhO7jF1H2+4vAHYWcyCC2AjsNHd/xhO30sQtqP1MwY4E3jN\n3be7exL4GcFnP5o/56zBfq6j4fMedcolXJ8G5oc9DSsJOkY8UOKaCsLMDPge8KK7fz1n0QNAttfg\n5QTXYrPzLwt7Hp4MNOecgjrouftn3X26u88m+BwfdfcPAI8BF4Wr9T7e7M/honD9EdUScPctwAYz\nOzyctQRYxSj9jEOvAyebWW34bzx7zKP2c84x2M/1YeAvzawpbPH/ZThPSqnUF32L9QDeCbwMrAX+\nqdT1FPC4Tic4bfQ88Gz4eCfB9aZfA68AvwLGhesbQc/ptcAKgt6YJT+OIR77GcCD4eu5wFPAGuCn\nQFU4vzqcXhMun1vquod4rIuB5eHnfD/QNNo/Y+BfgNXASuAHQNVo+5yBHxNcU04SnKH40FA+V+CD\n4bGvAa4o9XHp4RqhSUREpNDK5bSwiIhI0ShcRURECkzhKiIiUmAKVxERkQJTuIqIiBSYwlVGNTNz\nM/tazvT/NrPrC7DfKjP7lZk9a2bv67Xs5PDOLM+Gd7C5Ppx/hpmdOtz3FpGDX+LAq4iMaJ3Ae8zs\n/7r7jgLu9xgAd1/cx7K7gIvd/bnwjkzZwR/OAFqB3xewDhE5CKnlKqNdCrgN+GTvBWY228weDe+N\n+Wszm9nHOuPM7P5wnSfN7CgzmwT8B3BC2Do9tNdmkwgGBsDd0+6+KrypwkeBT4bbvNXMJprZfWb2\ndPg4LXzP683sB2b2h/D+nB8O5081s8fD7Vea2VsL92MSkUJSuEo5uBX4gJk19Jr/78Bd7n4U8EPg\n5j62/Rfgz+E6/wjc7e7bgCuBJ9x9sbuv7bXNN4CXzOw/zewjZlbt7uuAbxPci3Sxuz9BMCbyN9z9\nBOBC4PacfRxFMG7yKcA/m9khwKUEt1hbTHBP12eH9uMQkajptLCMeu6+18zuJrj5dnvOolOA94Sv\nfwB8uY/NTycIPtz9UTMbb2ZjD/B+N5jZDwnGeL2U4H6bZ/Sx6pnAwmDoXADGhnc3Avi5u7cD7Wb2\nGMFNsJ8G7ghv1HC/uytcRQ5SarlKubiRYNzWumK8mbuvdfdvEQw4f7SZ9XVv0RhwctiSXezu09y9\nNbuLN+/SHwfeRnDHkzvN7LLIDkBEhkXhKmXB3XcB/48gYLN+T3BnHYAPAE/0sekT4TLM7Axgh/e6\nX25vZnau9TRH5wNpYA/QAozJWfW/gb/L2S63c9QFZlYdhvIZwNNmNgvY6u7fJTiFfOxAdYhI6Shc\npZx8DZiQM/13wBVm9jzw18C1fWxzPXBcuM4X6bkV2ED+muCa67MEp5s/4O5p4BfAu7MdmghOUx8f\ndpZaRdDhKet5gturPQn8q7u/QRCyz5nZn4H3EVyzFZGDkO6KI3KQCb8X2+ruXy11LSIyNGq5ioiI\nFJhariIiIgWmlquIiEiBKVxFREQKTOEqIiJSYApXERGRAlO4ioiIFJjCVUREpMD+P96Pb8hgoLji\nAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f607e28f438>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, axes = plt.subplots(nrows=1, ncols=1, figsize=(6, 6))\n",
    "axes.plot(greedy_experiment(init_expectation=5,epsilon=0), label=\"Q1=5\")\n",
    "axes.plot(greedy_experiment(init_expectation=0,epsilon=0),label=\"Q1=0\")\n",
    "axes.plot(greedy_experiment(init_expectation=-5,epsilon=0),label=\"Q1=-5\")\n",
    "axes.set_xlabel(\"No of Steps\")\n",
    "\n",
    "axes.set_ylabel(\"Average Reward\")\n",
    "axes.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is necessary to note however that setting optimistic initial values is <span style=\"color:#f31818\">not well suited to nonstationary problems</span> because its drive for exploration is inherently <span style=\"color:#f31818\"><b>temporary</b></span>. If the task changes, creating a renewed need for\n",
    "exploration, this method cannot help. Indeed, any method that focuses on the initial\n",
    "state in any special way is unlikely to help with the general nonstationary case"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"7\"></a>\n",
    "## <span style=\"color:#2595bc\">Upper-Confidence-Bound Action Selection</span>\n",
    "\n",
    "Exploration is needed because the estimates of the action values are uncertain. The\n",
    "greedy actions are those that look best at present, but some of the other actions\n",
    "may actually be better. ε-greedy action selection forces the non-greedy actions to\n",
    "be tried, but indiscriminately, with no preference for those that are nearly greedy or\n",
    "particularly uncertain. It would be better to select among the non-greedy actions\n",
    "according to their potential for actually being optimal, taking into account both how\n",
    "close their estimates are to being maximal and the uncertainties in those estimates.\n",
    "One effective way of doing this is to select actions as -\n",
    "\n",
    "\n",
    "$$\\begin{equation*}\n",
    "A_t = argmax_{a}\\Bigl[Q_t(a) + c\\sqrt{\\frac{\\log t}{N_t(a)}}\\Bigr]\n",
    "\\end{equation*}\n",
    "$$\n",
    "\n",
    "where $\\log t$ denotes the natural logarithm of $t$ (the number that e ≈ 2.71828 would\n",
    "have to be raised to in order to equal $t$), $N_t(a)$ denotes the number of times that\n",
    "action $a$ has been selected prior to time t (the denominator), and the number\n",
    "c > 0 controls the degree of exploration. If $N_t(a)$ = 0, then a is considered to be a\n",
    "maximizing action.\n",
    "\n",
    "The idea of this <span style=\"color:#f31818\"><i>upper confidence bound</i> (UCB)</span> action selection is that the squareroot\n",
    "term is a <b>measure of the uncertainty or variance</b> in the estimate of $a$’s value.\n",
    "The quantity being max’ed over is thus a sort of upper bound on the possible true\n",
    "value of action $a$, with the $c$ parameter determining the confidence level. \n",
    "\n",
    "Each time\n",
    "$a$ is selected the uncertainty is reduced; $N_t(a)$ is incremented and, as\n",
    "it appears in the denominator of the uncertainty term, the term is decreased. On\n",
    "the other hand, each time an action other than a is selected $t$ is increased but Nt(a)\n",
    "is not; as $t$ appears in the numerator the uncertainty estimate is increased. The\n",
    "use of the natural logarithm means that the increase gets smaller over time, but is\n",
    "unbounded; <span style=\"color:#f31818\">all actions will eventually be selected, but as time goes by it will be\n",
    "a longer wait</span>, and thus a lower selection frequency, for actions with a lower value\n",
    "estimate or that have already been selected more times\n",
    "<a id=\"7a\"></a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epsilon :  0.1 \t average reward value 2.1211933201110056\n",
      "epsilon :  0  c:  2 \t average reward value 2.270963060078824\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7f607e1d16a0>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgoAAAF3CAYAAADElWPXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xl4VOX9/vH3J4SwgyxhlQAKiBRl\nFUXcV1xwoVIVrHtd+bpVq7UWLfVq/bW1KqJQ64ri0kZrtXWpCyqIGwgCArLv+xYgECDJ8/vjmSEh\nJmEG5szJZO7XdZ1rZs6cmfkM03ruPOdZzDmHiIiISHkywi5AREREqi4FBREREamQgoKIiIhUSEFB\nREREKqSgICIiIhVSUBAREZEKKSiIiIhIhRQUREREpEIKCiIiIlIhBQURERGpUGbYBcSrWbNmrn37\n9mGXISKSUqZMmbLeOZd9gO/RPDMz82mgG/pDs7ooBmYWFhZe27t377XlHZByQaF9+/ZMnjw57DJE\nRFKKmS050PfIzMx8umXLlodnZ2dvysjI0EJB1UBxcbGtW7eu6+rVq58GzivvGCVCERGJVbfs7Owt\nCgnVR0ZGhsvOzs7DtxKVf0wS6xERkdSWoZBQ/UR+0wrzgIKCiIiklXHjxjW69957WwLccccdrYcP\nH94iqM+aMGFC3c6dO3fNycnpduWVV7YtLi7+0TFTp06t3aNHjy5ZWVm9gqglNze3Yfv27bvl5OR0\ni37veCgoiIhIWhk6dGjeH/7wh9XJ+Kybbrqp3ejRo5csXrx45sKFC2vn5uY2LHtM8+bNCx977LGl\n119//ZpEf35hYSG33357zjvvvDN37ty537/++utNpkyZUjue91BQEBGRlPHkk082OeKIIw7v0qVL\n1yFDhrQrLCwEoG7duj2vueaath07dvxJv379Oq9cuTIT4MEHH2x+6KGH/qRz585dzz333EMARo4c\n2fTyyy/PKfvekyZNqtO9e/cunTt37nr66acfum7duhoAffv2PezGG29sc8QRRxzevn37bu+99179\nWGpdsmRJzW3btmWceuqp+RkZGQwdOnTDm2++2bjscW3atCk88cQTt9esWbPSyzq5ubkNu3btevhh\nhx3WtV+/fp1jqeGTTz6p165du51du3bdVbt2bTdo0KCNubm5B8Xy2qiUG/UgIiJVwNVXt2XmzLoJ\nfc9u3bbz7LPLKnr622+/rZ2bm9tk8uTJc2rVquUuu+yynDFjxjQdNmzYhh07dmT06dMn/5lnnll2\n5513trrnnntajx07dunIkSNbLlmyZEadOnXc+vXra1T28VdeeWWHRx55ZOk555yz7bbbbmt99913\nt342Uk9hYaHNmDFj9muvvdZoxIgRrQcMGDD3u+++q3XxxRcfWt57TZw48YclS5bUbNWq1e7ovnbt\n2u1atWpVzf35p1m5cmXmsGHD2n/yySdzunTpsmvNmjU1AN5+++0Gd911V9uyx9epU6d46tSpc5Yt\nW5bVpk2bXdH9Bx988K6vvvoqpqATpaAgIiIp4b333mswc+bMut27dz8coKCgIKN58+aFABkZGVx7\n7bUbAa6++uoNgwYN6ghw2GGH7bjwwgs7nHfeeZuHDh26uaL33rBhQ42tW7fWOOecc7YB/OIXv9gw\nePDgQ6LPDx48eBPAsccem3/XXXdlAXTv3n3nnDlzZgX1fUv75JNP6vXt23drly5ddgG0aNGiCGDg\nwIFbBw4cGGgNCgoiIhK/Sv7yD4pzzgYPHrzhiSeeWLGvY80MgPHjx8979913G/z73/9u9Je//KXV\nDz/88P3+fHbt2rUdQGZmJkVFRQawrxaFdu3a7S7dgrBkyZKs0i0MibCvFoW2bdvuWrFiRVZ0//Ll\ny/dqYYiF+iiIiEhKGDBgwJb//Oc/jVesWJEJsGbNmhpz587NAiguLua5555rDPD888837du379ai\noiIWLFiQNXDgwK1PPPHEim3bttXIy8sr9/JD06ZNixo2bFgU7X/wzDPPNO3Xr9+2yuqJtiiUtzVr\n1qyoXbt2u+vXr1/80Ucf1SsuLmbcuHFNzz///ApbNSpz0kkn5X/99dcN5syZkxX97uBbFMr7/KlT\np84BOPHEE/MXL15ce86cOVkFBQX2xhtvNPnpT38aVw1qURARkZTQu3fvgvvuu2/Fqaee2rm4uJia\nNWu6kSNHLu3cufOuOnXqFH/99df1/vznP7du2rTp7jfeeGNhYWGhDRkypMPWrVtrOOfs2muvXdus\nWbOiit7/ueeeW3TjjTe2u+WWWzJycnJ2vvLKK4sPtOYnnnhiyTXXXNOhoKDATj755C2DBw/OA/jT\nn/6UDfCrX/1q3dKlSzOPOuqorvn5+TXMzP3tb39rMXv27JlNmjTZM5aydevWhSNHjlx84YUXdiwu\nLqZp06a7J02aNG9fn1+zZk0efvjhpQMGDOhcVFTEkCFD1vfp06cgnu9gzqXW3Bl9+vRxmsJZRCQ+\nZjbFOdfnQN7ju+++W9y9e/f1iaopkerWrdtz+/btU8OuI1V99913zbp3796+vOfUoiAiUsXs2gVb\ntsDWrSW3+fnQrh106RJ2dZJuFBRERA5AYaE/kW/b5m/L3q/sufz8H2/btvmgUJ6774aHHkru90sV\nak0IjoKCiKSdggLYvPnHf7WXvS27lXfCL4jxam9GBtSvDw0a+K1+fb+1bAn16u29NWzoj4neNmjg\n97f9Ud92keApKIhISios9Cf7TZtg48bYbqP3Yzm5Z2SUnNCjJ+sGDaBZsx/vi+Vx3boQGbEnklIU\nFEQkNM75v8xjPdGXvt2ypfL3rl8fGjeGJk387WGH+dvovkaN/Fb6L/fSf8HrxC7iKSiIyAFzzl9f\n37AB1q6FNWv8yTyWv/AjU/WXq2bNkhN9kybQujV067Z3ACjv9qCDICur4vcVkdgpKIhIufLz/Qk/\nukUDwNq1Jdu6dT4cbNxYcQc8M/+Xe+kTeU5O5Sf66K3+qpcgjBs3rtH3339f5w9/+MPqO+64o3X9\n+vWLRowYkfCVG8EvM33NNde0LygoyDjllFPynn322WUZGYmZ6/D6668/+H//+1+jmjVrunbt2u18\n5ZVXFlc2T8T+UlAQSSPbtu19sl+1ym+rV/vb6HPr1vmgUJ7GjaF5c7917gxNm/qTevS2RQv/XLNm\n/thGjaBGpUvxiCTX0KFD84C8ZHxWdJnpk08+Of+kk07qlJub2/BnP/vZPi6cxebMM8/cMmrUqOU1\na9bkxhtvbPPb3/625ejRo/c5vXW8FBREUpxzsH49rFzpT/jRrWwIWLXKB4WyzCA72/e+b9ECOnXy\nj1u0KNkX3bKz1aQv4XryySebjB49usXu3butV69e+WPHjl2SmZlJ3bp1e1566aXrP/3004bZ2dm7\nX3/99YWtW7cufPDBB5s/99xz2TVq1HCdO3cu+M9//rNw5MiRTSdPnlxv7NixS0u/96RJk+rceOON\n7Xbs2JHRrl27nS+//PLi7Ozsor59+x7Wu3fvbRMnTmy4devWGmPGjFk8YMCASqd3hr2XmQb2LDNd\nNigsW7Ys8+qrr263dOnSWgCjRo1acvrpp1cQ1UsMGjRoz/v069cvPzc390dLWCeCgoJIFbZ9uw8A\nK1b4rbz7K1eW3+xfvz60auW3Xr1K7pc+6bdq5f/6z9R/CSROV19N25kzSegy0926sf3ZZ0m7ZaZv\nuOGGnOOPP37r8OHDFxQWFhJdj6J3796H5efn/6jmhx56aNkFF1ywtfS+559/vtlFF120sbLvt7/0\nnweREBQV+Sb+sif/so83l7N0S7160KaN3/r3L7nfuvXeYaBeveR/L5EgVddlpidNmtQgNzd3EfjV\nKZs2bVoEMGXKlB9ief3dd9/dskaNGu6GG25QUBBJJfn5sGABzJ9fss2b5/etXOnDQmk1avim/jZt\nfPP/SSeVBIBoGGjTxg/dUwc/CVtlf/kHJd2WmY6lRWHkyJFN33///YMmTJgwN1GdJMtSUBA5AFu3\n7h0ESm8rV+59bHZ2SQDIydk7ALRu7VsB1OlPpGIDBgzYMmjQoI733nvvmjZt2hSuWbOmRl5eXo3O\nnTvvii4zfd11120qb5npM844Y1vbtm2bxLLM9IABA7bFs8x0Rc83a9asKLrM9Mknn5w/bty4pjff\nfPPassf1799/65///Ofs4cOHr41eemjatGnRvloUcnNzGz722GMtJ0yY8EODBg2KKzv2QCgoiOzD\n7t3+xD93LixbBjNn+m3+fD9KoLSWLaFjRzjjDB8KOnb026GH+t7/IrL/qtMy06WNHj166ZVXXtmu\nc+fOzTIyMhg1atSS0047bZ+dGe+4446cXbt2ZZxyyimdAXr16rXt5ZdfXrqv18VLy0yLROTnww8/\n+AAwezbMmgXff+8Dwu5SjYUHHQRHHrl3EOjUyYeB+vXDq1+kMlpmWiqjZaZFyli/HqZO3XubO9cP\nNQTfB6BDB/jJT2DgQOjaFQ4/3F8yyM5WHwERSR8KClKtOQdLl/44FCxfXnJMTg707AmXXuqnB462\nFNRN6MAvEQmSWhOCo6Ag1UZRkb90UDYUbNrkn8/I8AsDnXCCDwY9e0KPHn5GQRERKZ+CgqSs1avh\niy9g0iS/TZvmJygCqFXL9yMYPNiHgZ49/WO1EogckOLi4mLLyMhIrc5tUqni4mIDKhw1oaAgKaGo\nyI80iIaCSZNg4UL/XFYW9O4Nv/iFn4GwVy/o0kWzDYoEYOa6deu6Zmdn5yksVA/FxcW2bt26RsDM\nio7Rf0qlSsrLgy+/LAkFX33l5ywAP99A//5w001w7LE+GNSqFW69InFzzv+POi+vZNu8uWQr/Tj6\n/ODBcM01oZVcWFh47erVq59evXp1NyCY2X0k2YqBmYWFhddWdICCglQJa9bAp5/CZ5/5beZM/9/R\njAx/yeDnP/eh4NhjoX17jTqQkDnnr3NFT+alT/ax7tuyBYr3MUdO7dp+PG6jRn6raC3vJOndu/da\n4LxQi5CkU1CQUGzf7oPBhx/6bfp0v79ePR8GLrrItxr07eunLBZJOOdgxw7YuNH3eN20Kfb7mzf/\neA7usjIySk7w0a19+5L7pQNAo0Yla3IfdFDJc2oqkypAQUGS5ocf4N13/fbpp7Bzp//v4HHHwR//\nCKec4i8jqG+BxMU5/9f5xo2wYUPJbfR+dCvvhF/ZX+gZGf6E3bgxNGnibzt08LfRk3pFJ/1Gjfzs\nW2r6kmpA/0mWwBQXwzffwJtvwr/+5YMC+I6GN90EAwbA8cdDnTrh1ilVSHGxP4mXPuGvX1+ylX4c\nvb9hQ+V/3Tdo4MfARk/wXbuWnPhLh4Cy9xs29GFBJM0pKEhCOec7Ib78Mrzxhl8YKTPTL4Q0bBic\ne65vfZU0U1AAq1b5Ma2rVlV8f+3aik/6mZnQrJnfmjb1U2U2a+ZP7k2b+q3s/caNoWbN8t9PRGKi\noCAJsWwZvPACPP+8X0a5Vi04+2y48EIfDho3DrtCSTjn/LX6yk780fubN//49RkZfghLy5bQqpWf\n7KJFC2jevOREXzoYNGyopnyRECgoyH5bv973N3j1VXjvPd9qfPLJcN99MGiQ/++6pKhdu0pO9itX\n+jmvV6zwt9FtxQrfUlBWnTr+xN+qlV8s47TT/P1oIIjez87WutoiKUBBQeKydCmMGwf/+Y+fFdE5\naNMG7r0Xrr7a9/WSFLB5MyxatPe2eLH/gVet8imwrJo1/Y998MFw1FFwwQX+cdkQ0KCB/vIXqUYU\nFGSfiop8y8HTT/uAUFQEffrA/ff7ywo9e6rPV5WzY4c/8ZcNA9Gt7KWAhg19ymvf3o9Pbd265MTf\nurUPB82a6YcWSUMKClKhjRvhmWfgiSdgyRJ/+fjOO+HGG6Fdu7CrS3NFRb7pf9EiP5d1dIsGgdWr\n9z6+dm0fAjp0gGOOgUMO8fejmzqRiEgFFBTkR5YuhYcf9i0I27f7EQsPPwznnacO5ElXUODHlc6a\nBd9/729nzfKhYPfukuMyMqBtWx8Azj577xDQoYNPeWoNEJH9oKAggO9rMGGCbz144w2/b+hQ+OUv\n4Ygjwq0tLezY4QNBNAxEbxcsKJnmt0YN6NjRzwNwwQUlrQKHHAI5OUpxIhKIwIKCmbUFxgItAAc8\n5Zx7rMwxBjwGnA1sB650zn0bVE3yY87Bxx/7mRE/+shPMHfLLXDrrf7cIwlWUABz5vjFLEoHgoUL\n9w4EnTr5RS4uucSPHOjaFTp31pS+IpJ0QbYoFAK/dM59a2YNgClm9oFzblapY84COkW2o4HRkVsJ\nmHMwfjwMHw6ff+6Hrj/yCFx3HdStG3Z11YBzvg/B1Kl+++47Hw7mzy8JBJmZPhD06AFDhuwdCLKy\nwq1fRCQisKDgnFsFrIrc32pms4E2QOmgcD4w1jnngC/N7CAzaxV5rQTkiy/gnnv8Ko2tW8OTT8JV\nV/n+brIfioth3rySUBDdokMMzfwlgyOOgIsv9oGgWzcfEhQIRKSKS0ofBTNrD/QEvirzVBtgWanH\nyyP7FBQCMGsW/Pa3vg9Cixbw+ONw7bUKCHFxDubOha++gsmTYcoU31qQn++fr1nTh4DzzvPjRnv2\n9JcQtASmiKSowIOCmdUHXgduc85t2c/3uA64DiBHF87jtnixn/PgpZf8Ms4jRsDtt/vF7WQfCgp8\nGPj8c79NmlTSUlCvng8CV19dEgq6dlUrgYhUK4EGBTOriQ8J45xzb5RzyAqgbanHB0f27cU59xTw\nFECfPn1cAKVWS7t3+2GNI0b4P4TvuAPuvtvPmyMVWLfOh4FoMJg8uWQp4k6d/AxT/ftDv35+GUxN\nQSwi1VyQox4MeAaY7Zz7awWHvQUMM7NX8Z0Y89Q/ITFmzYKf/xy+/dYvzPTYY36YvZSxeTN88okf\n8vHRRzB7tt+flQW9e/shIP37+9kKmzcPtVQRkTAE2aLQH/g5MMPMpkX23QvkADjnxgDv4IdGzscP\nj7wqwHrSQn4+/OY3fj6Egw6C3Fz46U/DrqoKKSjwLQXRYDB5su+MWLcuHH88XHGFDwZ9+qjzhogI\nwY56mAhUujJMZLTDzUHVkG6++sq3Isyb54c5/v73+iOY4mI/AuGDD+DDD31IKCjwlwyOPtqnqtNO\n8/c1R4GIyI9oZsZqoLDQT5j0u9/5xfzGj/fTLqetJUt8MPjgA99qsGGD39+tG9xwA5x6KpxwgtbB\nFhGJgYJCilu0CC67zPe/GzrUX3Jo1CjsqpJsyxbfz+B///PhYO5cv791a9/58PTTfTho2TLUMkVE\nUpGCQopyDl58EYYN8/P5jBvnJ/dLC875qY//+1+//vXnn/tmlbp14eST4aabfDg4/HD/jyMiIvtN\nQSEFFRTA9dfD2LG+/92LL6bBss87d8Knn8Lbb8Nbb/klLgG6d/drX59xhh+ZoH4GIiIJpaCQYlau\n9MMdv/4aHngA7ruvGg/lX7cO3nnHh4P334dt26BOHd9acN99fjnlNm3CrlJEpFpTUEghX33lQ8LW\nrfCvf/mVhqudNWv8mM5XX/WXFJzzfQ2GDoWBA+GUU3xYEBGRpFBQSBFjx/ohj61b+z573bqFXVEC\nbdvmk89LL/khjMXFfgGl4cNL1kxQXwMRkVAoKFRxzvlLDCNG+H56//wnNG0adlUJUFzsRyo895xf\npWr7dmjfHn79a7j0Ur/CooiIhE5BoQorLvbrMzz2mF93aMwYvzhhSlu3Dp5/Hp56CubP92M5hw71\nM0X17w8ZGWFXKCIipSgoVFHO+bmB/v53uO02+OtfU7j13Tn47DMYOdJ3TNy92w/XuP9+uOgiTZUs\nIlKFKShUUffd50PCvffCgw+maEjIz/cTPIwaBTNmQJMm8H//B9dc45djFhGRKk9BoQp6/HH4wx98\n58WUDAnz5sGTT/r+B3l50KOHTz1DhvhJkUREJGUoKFQx//433HqrH/r4xBMpFBKKi/0siY8/7uc8\nyMyEwYPh5pv9REgp80VERKQ0BYUqZMECv8pxr17w8sv+XFvlFRTACy/AX/7iOye2auVXp/rFL/x9\nERFJaalwKkoLmzb5yZQyMvx8Q1V+TqHt2/3lhb/8xU+SdNRRfpKkQYOqwdAMERGJUlCoArZuhbPO\ngh9+8OsctW8fdkWVKCz0fQ/uvx9WrYLTTvPNHyefrMsLIiLVkIJCyHbsgPPPh8mT4fXX/Xm3SnLO\nL8b061/D7NnQrx/84x9w3HFhVyYiIgHS7DYhcg6uvNJPUDh2rA8MVY5zMH68n/fgggt8p8U33vDr\nMCgkiIhUewoKIXr8cf9H+R//6EcOVjlffOHDwCmn+J6WY8bAzJm+M4UuM4iIpAUFhZB8/jn88pd+\nzaO77gq7mjIWLYJLLvHDGhcu9J0WFy2C669PkaEYIiKSKPqvfgjWrvVTDLRv70cWVpnlDbZs8Ss2\nPvmkDwS//S386ldQv37YlYmISEgUFJKsuNjPlbBxI7z3Hhx0UNgVRfz3v77FYOVKPwfC8OHQpk3Y\nVYmISMiqyt+yaeOxx3xAePRROPLIsKvBz4EwZAicey40buz7JfztbwoJIiICKCgk1bx5fpGngQP9\nH++hKi726y906eLHZd5/P0yZAkcfHXJhIiJSlejSQ5IUF8O110KtWn7wQKiDBpYuhauugo8/hhNP\n9AV16RJiQSIiUlUpKCTJ3/4Gn30GzzwDrVuHWMibb/qQUFgITz3l04uGOoqISAV06SEJ1qyBu++G\n00/35+hQ7NoFt93m50Do2BGmTfOdFhUSRESkEgoKSfDAA36q5lGjQjovL1rkJ0567DG/hvXEiXDo\noSEUIiIiqUaXHgI2e7bvM3jjjdC5c5I/3Dl46SW4+WY/WcPrr/vVHUVERGKkFoWA3X031KvnpyVI\nqq1bYehQuPxy6N7dX2pQSBARkTgpKATok0/g7bf9kMjs7CR+8OzZ0LevX0jiwQd9IVV67WoREamq\ndOkhIM7BnXdC27Zwyy1J/ODcXN9jsm5d+PBDOOmkJH64iIhUN2pRCMg77/j5i0aMgDp1kvCBhYV+\nXYbBg6FbN//hCgkiInKA1KIQkIcegpwc300gcGvXwsUX+0sMN98Mf/0rZGUl4YNFRKS6U1AIwMSJ\nfhs5EmrWDPjDvvwSLroINmzwS1FefnnAHygiIulElx4C8NBD0KwZXHNNgB/iHIweDSec4OeF/uIL\nhQQREUk4BYUEmz7dr9h8yy2+P2Eg8vPhyivhppv8dI+TJ0OPHgF9mIiIpDMFhQT705+gfn0YNiyg\nD1i92s+y+OKLfsrHt9/2y0OLiIgEQH0UEmjRInj1Vb+kQiDn7oUL4YwzYNUq32xx1lkBfIiIiEgJ\nBYUEGjnSr+Vw220BvPn06XDmmX5xp48+gmOOCeBDRERE9qagkCDbtsGzz/oBCAcfnOA3/+47OPFE\nf01jwgTo2jXBHyAiIlI+BYUEefFF2LIlgFkYFy+Gs8/2IWHiRE3FLCIiSaWgkADOwZgx0LNngq8I\nrFoFp53m16j+9FOFBBERSToFhQT45hvfhWD0aN9HISE2b4YBA/woh48+giOOSNAbi4iIxE5BIQH+\n/nc/Z8KQIQl6w+3bYeBAvwrkO+/A0Ucn6I1FRETio6BwgNauhZdegssug4YNE/CGu3fDz34Gn38O\nr73mLz2IiIiEREHhAI0aBTt3wi9/mYA3Ky728z7/97/+OsbgwQl4UxERkf2nmRkPwM6dvhPjwIHQ\npcsBvplzcOedfvjE738PN9yQkBpFREQOhILCAfjXv2DdOr/kwgF76CF45BG49Vb4zW8S8IYiIiIH\nTkHhADz7rB+xePrpB/hG//wn3HsvDB0Kf/1rAodOiIiIHBgFhf0UHbV42WWQcSD/ijNm+JUgjz3W\nJ48DejMREZHE0llpP/3jH77v4QENidy0CS68EBo1gtxcyMpKWH0iIiKJoFEP++nll6FHDzj88P18\ng6IiuPRSWLrUz7rYqlVC6xMREUkEtSjshwUL4KuvDrA14be/hfff9+Mr+/VLWG0iIiKJpKCwH155\nxd9ecsl+vkFuLvzxj3DddX4TERGpohQU4uScv+xw/PHQtu1+vMHs2b7zYr9+MHJkossTERFJqMCC\ngpk9a2ZrzWxmBc+fZGZ5ZjYtsg0PqpZEmj7dn+v367JDQQFcfLFfGCI3F2rVSnh9IiIiiRRkZ8bn\ngVHA2EqOmeCcOzfAGhLu5ZchMxMuumg/XvyrX/nhkP/9L7RunfDaREREEi2wFgXn3GfAxqDePwzF\nxb5/wplnQrNmcb74k0/g8cfhllvg7LODKE9ERCThwu6j0M/MvjOzd83sJyHXsk+ffw7Llu3HZYf8\nfL/Y06GH+k6MIiIiKSLMeRS+Bdo557aZ2dnAm0Cn8g40s+uA6wBycnKSV2EZL7/suxecd16cL7z3\nXli40M+XULduILWJiIgEIbQWBefcFufctsj9d4CaZlZug75z7innXB/nXJ/s7Oyk1hm1a5efjfH8\n86F+/TheOGGCH90wbBiccEJg9YmIiAQhtKBgZi3N/OpHZtY3UsuGsOrZl48/ho0b45w7Yft2uPpq\n6NBBlxxERCQlBXbpwcxeAU4CmpnZcuB+oCaAc24McBFwo5kVAjuAS5xzLqh6DtQbb0CDBnDGGXG8\n6Pe/h/nzfcqIqxlCRESkaggsKDjnLt3H86PwwyervOJieOstOOssqF07xhd9/z08/DBccQWcfHKg\n9YmIiAQl7FEPKeHbb2HNGjg31hkfior8KIeGDeHPfw60NhERkSBp9cgY/O9//nbAgBhfMGaMXzXq\npZcgpM6XIiIiiaAWhRhMnAhdu8Z4zs/PhxEj4KSTDnB5SRERkfBV2KJgZo8DFXYudM7dEkhFVUxR\nEUya5JdoiMmoUbB2LfzrX+AHdYiIiKSsyloUJgNTgNpAL2BeZOsBZAVfWtXw/feQlwfHHRfDwVu2\nwJ/+5KdoPvbYwGsTEREJWoUtCs65FwDM7EbgOOdcYeTxGGBCcsoL38SJ/jamoPDoo36yhREjAq1J\nREQkWWLpo9AYaFjqcf3IvrQwcaJf6LF9+30cuGGDHw554YXQu3cyShMREQlcLKMeHgKmmtl4wIAT\ngAeCLKoqmTjRtybss7vBPfcKbdliAAAap0lEQVT4mRjVmiAiItVIpS0KkSmWPwSOBv4FvAH0i16W\nqO6WLvWrRe7zssP8+fDcc3DzzdCtW1JqExERSYZKWxScc87M3nHOHQH8O0k1VRkx90/43e8gK8u3\nKoiIiFQjsfRR+NbMjgq8kipo4kS/RMMRR1Ry0OzZfv3pYcOgZcuk1SYiIpIMsfRROBoYamZLgHx8\nPwXnnDsy0MqqgIkToV8/yKzsX+nBB6FOHbjrrqTVJSIikiyxBIUzA6+iCtq0CWbOhIsuquSgefPg\n1Vfhl7/UVM0iIlIt7TMoOOeWAJhZc/zkS2nhiy/AuX30T3joIahVywcFERGRamiffRTM7Dwzmwcs\nAj4FFgPvBlxX6CZOhBo14OijKzhg1Sq/6NNVV0GLFkmtTUREJFli6cz4e+AYYK5zrgNwKvBloFVV\nARMnQq9eUK9eBQeMGgW7d8Pttye1LhERkWSKJSjsds5tADLMLMM5Nx7oE3Bdodq1C77+upLLDtu2\nwejRfhbGjh2TWpuIiEgyxdKZcbOZ1Qc+A8aZ2Vr86Idqa+5c2LkT+lQUhx591Pd2/NWvklqXiIhI\nssXSonA+sB24HXgPWAAMDLKosM2Z428PP7ycJ/Py/JoO551XSQcGERGR6iGWFoVLgM+cc/OAtJi6\nefZsf9u5czlPPvMMbN4Mw4cntSYREZEwxBIUcoC/mVkHYDL+EsQE59y0QCsL0ezZkJNTTkdG52DM\nGDj2WK0QKSIiaWGflx6cc/c7504BugITgLuAKUEXFqapU6FHj3KeGD/eT7J0/fVJr0lERCQMscyj\ncJ+ZvQv8D+gI3AkcHHRhYdm6FX74oYIGg9GjoXFjGDw46XWJiIiEIZZLD4OAQuC/+AmXvnDO7Qy0\nqhBNm+avMPwoKHz7LeTmwt13+7UdRERE0kAslx56AacBXwOnAzPMbGLQhYVl8mR/+6OgMHasn6pR\niz+JiEga2WeLgpl1A44HTsRPtLQM31ehWpo6FVq3LrNi9IYN8NRTcNll0LRpaLWJiIgkWyyXHh7C\nB4ORwDfOud3BlhSu6dPhyLILaL/yCuzYAXfcEUpNIiIiYYll9chzzawOkFPdQ8Lu3X5o5BlnlHni\n+eehZ89yEoSIiEj1Fsuoh4HANPysjJhZDzN7K+jCwjBvnl/nYa88MHMmTJkCV1wRWl0iIiJhiWUK\n5weAvsBmgMhESx0CrCk006f72yOOKLXzhRcgMxOGDAmlJhERkTDFunpkXpl9LohiwjZjhh/Y0KVL\nZEdhIbz0Epx9NmRnh1qbiIhIGGLpzPi9mQ0BaphZJ+AWYFKwZYVjxgw47DCoVSuy44MPYPVquPLK\nMMsSEREJTSwtCv8H/ATYCbwMbAFuC7KosEyfXuayw/PP++GQ55wTVkkiIiKhimXCpe3Oud84546K\nbL8BmiehtqTasgWWLCnVkXHrVnjrLbj4YsjKCrU2ERGRsFQaFMysn5ldZGbNI4+PNLOXgc+TUl0S\nzZzpb/e0KLz9NhQUwCWXhFaTiIhI2CoMCmb2Z+BZ4KfAf83sQfzCUF8BnZJTXvLMmOFv9wSFf/wD\n2rSB/v1Dq0lERCRslXVmPAfo6ZwrMLPG+KmbuznnFielsiSbNw9q14acHCAvD959F266CTJi6cYh\nIiJSPVV2FixwzhUAOOc2AfOqa0gAHxQOPTSSC/75Tz/z0s9+FnZZIiIioaqsReGQMjMwdij92Dl3\nXnBlJd/8+dCpE36N6ZEjoXt3OOaYsMsSEREJVWVB4fwyjx8OspAwFRfDggVw1ln4daZnzIDRo8Es\n7NJERERCVWFQcM59msxCwrR8OezcGWlRGDMG6tSBSy8NuywREZHQqace/rIDQMf6q+HFF+Hqq6FR\no3CLEhERqQIUFCgJCp1ee9AvAHX33eEWJCIiUkXEHBTMrG6QhYRp3jyoVctx8PvPwLXXQtu2YZck\nIiJSJewzKJjZsWY2C5gTedzdzJ4MvLIkWrQI2mfnk7GrAM49N+xyREREqoxYWhQeAc4ENgA4574D\nTgiyqGRbuhTa2VKoWxdOqFZfTURE5IDEdOnBObeszK6iAGoJzbKlxbRd+TWcf76fnlFERESAyudR\niFpmZscCzsxqArcCs4MtK3l27oTVazJoyyK4rVquni0iIrLfYmlRuAG4GWgDrAB6RB5XCytW+Nuc\nJvlw1FHhFiMiIlLF7LNFwTm3HhiahFpCsWypA4y2fVpoJkYREZEy9hkUzGxkObvzgMnOuX8nvqTk\nWvrVKqA1bU/oEHYpIiIiVU4slx5q4y83zItsRwIHA9eY2aMB1pYUy7701x7annNkyJWIiIhUPbF0\nZjwS6O+cKwIws9HABOA4YEaAtSXFitl5NLZN1O3eKexSREREqpxYWhQaA/VLPa4HNIkEh52BVJUs\nxcWsXrqLVvW3qn+CiIhIOWJpUfgTMM3MPgEMP9nSH8ysHvBhgLUFb9o0Vu9oRMuuNcOuREREpEqK\nZdTDM2b2DtA3sute59zKyP27AqssGSZPZg2n0PfQBmFXIiIiUiXFuihUAbAK2AR0NLN9znNsZs+a\n2Vozm1nB82ZmI81svplNN7NesZedINOns5qWtDy0XtI/WkREJBXEsijUtcBnwPvA7yK3D8Tw3s8D\nAyp5/iygU2S7Dhgdw3sm1LbvFpBPfVq2Uv8EERGR8sTSonArcBSwxDl3MtAT2LyvFznnPgM2VnLI\n+cBY530JHGRmrWKoJ2HWfL8egBYtkvmpIiIiqSOWoFDgnCsAMLNazrk5wGEJ+Ow2QOnFppZH9iXH\n8uWs3pQFQMuWSftUERGRlBLLqIflZnYQ8CbwgZltApYEW9bezOw6/OUJcnJyEvOm48ezGp8QFBRE\nRETKF8uohwsjdx8ws/FAI+C9BHz2CqBtqccHR/aVV8NTwFMAffr0cQn4bPj2W1bXzIHdCgoiIiIV\nqfTSg5nVMLM50cfOuU+dc28553Yl4LPfAi6PjH44Bshzzq1KwPvGZto01jTvRkYGNGuWtE8VERFJ\nKZW2KDjniszsBzPLcc4tjeeNzewV4CSgmZktB+4HakbedwzwDnA2MB/YDlwVf/n7yTn47jtWtxhB\ndiHUqJG0TxYREUkpsfRRaAx8b2ZfA/nRnc658yp7kXPu0n0874CbYyky4davh02bWN2mjS47iIiI\nVCKWoPDbwKtItvnzAVi9qwkttbq0iIhIhWLpzPipmbUDOjnnPjSzukBqN9ZHgsKabfU4XHMoiIiI\nVCiWmRl/AeQCf4vsaoMfKpm65szBWQar12fq0oOIiEglYplw6WagP7AFwDk3D2geZFGBe/VV8o48\nnl27TLMyioiIVCKWoLCz9HBIM8sEEjOXQRh27ICFC9l8yiAAGjcOuR4REZEqLJag8KmZ3QvUMbPT\ngX8CbwdbVoCW+Ekl85oeAkCjRmEWIyIiUrXFEhTuAdYBM4Dr8fMf3BdkUYGa4+ePigaFhg3DLEZE\nRKRqi2V45AX4VR7/HnQxSTFrFgBbmrQH1KIgIiJSmVhaFAYCc83sRTM7N9JHIXXNmgVt25K3uy6g\nFgUREZHK7DMoOOeuAjri+yZcCiwws6eDLiwwU6fCT37Cli3+oVoUREREKhZLiwLOud3Au8CrwBT8\n5YjUs2aNb1E45RTy8vwuBQUREZGKxTLh0llm9jwwD/gp8DSQmtMUrVvnb9u3Jy8PMjOhdu1wSxIR\nEanKYulvcDnwGnC9c25nwPUEK3q9oWFDtmzxrQlm4ZYkIiJSlcWy1sNeq0Ca2XHApc65cFZ+PBCl\nrjfk5akjo4iIyL7ENILBzHoCQ4DBwCLgjSCLCkypFoWtW6FBg3DLERERqeoqDApm1hk/yuFSYD3+\n8oM5505OUm2JV6pFYccOqFs33HJERESquspaFOYAE4BznXPzAczs9qRUFZRSLQo7dqgjo4iIyL5U\nNuphELAKGG9mfzezU4HU7vqXl+d7L9avT0EB1KkTdkEiIiJVW4VBwTn3pnPuEqALMB64DWhuZqPN\n7IxkFZhQW7b4Hoxm7NihoCAiIrIvsczMmO+ce9k5NxA4GJgK3B14ZUHIy9szw5IuPYiIiOxbTDMz\nRjnnNjnnnnLOnRpUQYGKtiiALj2IiIjEIK6gkPK2boX69QF06UFERCQG6RUUdu7cc71Blx5ERET2\nLb2Cwq5dkJXFrl3+0oMmXBIREalcegWFnTuhVi1Wr/YPW7UKtxwREZGqLqYpnKuLYUvuYt36dpz2\nvn+soCAiIlK5tAoKT2waApvgH9f5xz/5Sbj1iIiIVHXpdemhlBEjoEOHsKsQERGp2tKqRSHq0Udh\n2LCwqxAREan60qpFoY2tAODWW6FGjZCLERERSQFpFRQasJWfHTYt7DJERERSRloFhZ0ui6yaLuwy\nREREUkb6BIWiInaRRVbNsAsRERFJHekTFHbvZhdZ1KpZHHYlIiIiKSN9goJzvkUhU0FBREQkVmkV\nFHZSS0FBREQkDmkVFHaRRa3MorArERERSRlpExSKiqCYGmpREBERiUPaBIXdu/ywyMwMBQUREZFY\npU1QcMU+KJiFXIiIiEgKUVAQERGRCikoiIiISIUUFERERKRC6RcU0uYbi4iIHLi0OW26yFpQalEQ\nERGJXdoEhWhSUE4QERGJXdoEhWiLgpoUREREYpc+QUGdGUVEROKmoCAiIiIVUlAQERGRCqVPUNCo\nBxERkbilT1BQi4KIiEjcFBRERESkQukTFHTpQUREJG5pExT2TLikoCAiIhKztAkK0UsPSgoiIiKx\nS7ugoJwgIiISu0CDgpkNMLMfzGy+md1TzvNXmtk6M5sW2a4Nqhb1URAREYlfZlBvbGY1gCeA04Hl\nwDdm9pZzblaZQ19zzg0Lqo4otSiIiIjEL8gWhb7AfOfcQufcLuBV4PwAP69SalEQERGJX5BBoQ2w\nrNTj5ZF9Zf3UzKabWa6ZtQ2qGLUoiIiIxC/szoxvA+2dc0cCHwAvlHeQmV1nZpPNbPK6dev264P2\nBIWwv7GIiEgKCfK0uQIo3UJwcGTfHs65Dc65nZGHTwO9y3sj59xTzrk+zrk+2dnZ+1VMSYuCmhRE\nRERiFWRQ+AboZGYdzCwLuAR4q/QBZtaq1MPzgNlBFeOwyGcG9QkiIiLVT2CjHpxzhWY2DHgfqAE8\n65z73sxGAJOdc28Bt5jZeUAhsBG4Mqh6NDOjiIhI/AILCgDOuXeAd8rsG17q/q+BXwdZw57Pis7M\nKCIiIjFLm6596swoIiISv7Q5baozo4iISPzSJyhowiUREZG4pU9Q0IRLIiIicUufoBBtUchQUhAR\nEYlV+gSFaIsCGv0gIiISq/QJCmpREBERiVvaBAVNuCQiIhK/tAkKeyZcUlAQERGJWfoEhT3DI5UU\nREREYpWGQSHcOkRERFJJ+gQFzaMgIiISNwUFERERqVD6BIU9wyPDrUNERCSVpM1pU4tCiYiIxC99\ngoI6M4qIiMQtbYLCngmXNDOjiIhIzNImKDgt8SAiIhK39AkKGvUgIiISt/QJCuqjICIiErf0CQrF\n6qMgIiISr/QJCmpREBERiVv6BAW1KIiIiMQtfYKCWhRERETipqAgIiIiFUqfoKDhkSIiInFLm6Cg\nmRlFRETilzZBQTMzioiIxC99goJGPYiIiMQtfYKCOjOKiIjELf2CgloUREREYpY+QSF66QF1VhAR\nEYlV+gQFtSiIiIjELX2CguZREBERiVv6BAW1KIiIiMQtbYLCngmXlBNERERiljZBYc+ES0oKIiIi\nMUu7oKCcICIiErv0CQrqzCgiIhK39AkK6swoIiISt/QJCmpREBERiVv6BAW1KIiIiMQtDYNCuHWI\niIikkrQ5bZaMelCLgoiISKzSJihowiUREZH4pU1Q0IRLIiIi8UufoKBRDyIiInFLn6CgUQ8iIiJx\nS6OgoBYFERGReKVRUPAJQS0KIiIisUufoKA+CiIiInFLn6Cg1SNFRETiln5BQZceREREYpY+QSF6\n6UFBQUREJGZpExREREQkfmkTFNSiICIiEr9Ag4KZDTCzH8xsvpndU87ztczstcjzX5lZ+6Bq2TOD\ns4KCiIhIzAILCmZWA3gCOAvoClxqZl3LHHYNsMk51xF4BPh/QdXjiiN17YkMIiIisi9Btij0BeY7\n5xY653YBrwLnlznmfOCFyP1c4FQLaB1ojXoQERGJX5BBoQ2wrNTj5ZF95R7jnCsE8oCmQRSzZwpn\nBQUREZGYpURnRjO7zswmm9nkdevW7dd7tGhflzOaTqZ+s9oJrk5ERKT6CjIorADalnp8cGRfuceY\nWSbQCNhQ9o2cc0855/o45/pkZ2fvVzEn3tqD99f3oV3/g/fr9SIiIukoyKDwDdDJzDqYWRZwCfBW\nmWPeAq6I3L8I+NhFrxGIiIhI6DKDemPnXKGZDQPeB2oAzzrnvjezEcBk59xbwDPAi2Y2H9iIDxMi\nIiJSRQQWFACcc+8A75TZN7zU/QJgcJA1iIiIyP5Lic6MIiIiEg4FBREREamQgoKIiIhUSEFBRERE\nKqSgICIiIhVSUBAREZEKKSiIiIhIhRQUREREpEIKCiIiIlIhBQURERGpkKXaGkxmtg5Ysp8vbwas\nT2A5qUDfOT3oO6eHA/nO7Zxz+7f8rqS1lAsKB8LMJjvn+oRdRzLpO6cHfef0kI7fWcKnSw8iIiJS\nIQUFERERqVC6BYWnwi4gBPrO6UHfOT2k43eWkKVVHwURERGJT7q1KIiIiEgc0iYomNkAM/vBzOab\n2T1h15MoZtbWzMab2Swz+97Mbo3sb2JmH5jZvMht48h+M7ORkX+H6WbWK9xvsH/MrIaZTTWz/0Qe\ndzCzryLf6zUzy4rsrxV5PD/yfPsw695fZnaQmeWa2Rwzm21m/dLgN7498r/pmWb2ipnVrm6/s5k9\na2ZrzWxmqX1x/65mdkXk+HlmdkUY30Wqr7QICmZWA3gCOAvoClxqZl3DrSphCoFfOue6AscAN0e+\n2z3AR865TsBHkcfg/w06RbbrgNHJLzkhbgVml3r8/4BHnHMdgU3ANZH91wCbIvsfiRyXih4D3nPO\ndQG64797tf2NzawNcAvQxznXDagBXEL1+52fBwaU2RfX72pmTYD7gaOBvsD90XAhkghpERTw/+eZ\n75xb6JzbBbwKnB9yTQnhnFvlnPs2cn8r/gTSBv/9Xogc9gJwQeT++cBY530JHGRmrZJc9gExs4OB\nc4CnI48NOAXIjRxS9vtG/x1ygVMjx6cMM2sEnAA8A+Cc2+Wc20w1/o0jMoE6ZpYJ1AVWUc1+Z+fc\nZ8DGMrvj/V3PBD5wzm10zm0CPuDH4UNkv6VLUGgDLCv1eHlkX7USaW7tCXwFtHDOrYo8tRpoEblf\nHf4tHgV+BRRHHjcFNjvnCiOPS3+nPd838nxe5PhU0gFYBzwXudzytJnVoxr/xs65FcBfgKX4gJAH\nTKF6/85R8f6uKf97S9WWLkGh2jOz+sDrwG3OuS2ln3N+aEu1GN5iZucCa51zU8KuJYkygV7AaOdc\nTyCfkuZooHr9xgCRpvPz8SGpNVCPNPwrubr9rpKa0iUorADalnp8cGRftWBmNfEhYZxz7o3I7jXR\n5ubI7drI/lT/t+gPnGdmi/GXkE7BX78/KNJEDXt/pz3fN/J8I2BDMgtOgOXAcufcV5HHufjgUF1/\nY4DTgEXOuXXOud3AG/jfvjr/zlHx/q7V4feWKixdgsI3QKdIj+ksfKeot0KuKSEi12GfAWY75/5a\n6qm3gGjv5yuAf5faf3mkB/UxQF6pZs4qzzn3a+fcwc659vjf8WPn3FBgPHBR5LCy3zf673BR5PiU\n+gvNObcaWGZmh0V2nQrMopr+xhFLgWPMrG7kf+PR71xtf+dS4v1d3wfOMLPGkZaYMyL7RBLDOZcW\nG3A2MBdYAPwm7HoS+L2OwzdNTgemRbaz8ddnPwLmAR8CTSLHG34EyAJgBr5XeejfYz+/+0nAfyL3\nDwG+BuYD/wRqRfbXjjyeH3n+kLDr3s/v2gOYHPmd3wQaV/ffGPgdMAeYCbwI1KpuvzPwCr4Pxm58\ny9E1+/O7AldHvvt84Kqwv5e26rVpZkYRERGpULpcehAREZH9oKAgIiIiFVJQEBERkQopKIiIiEiF\nFBRERESkQgoKUq2ZmTOzh0s9vtPMHkjA+9Yysw/NbJqZXVzmuWMiKxhOi6z0+EBk/0lmduyBfraI\nSDJl7vsQkZS2ExhkZn90zq1P4Pv2BHDO9SjnuReAnznnvousXBqdKOkkYBswKYF1iIgESi0KUt0V\nAk8Bt5d9wszam9nHZjbdzD4ys5xyjmliZm9GjvnSzI40s+bAS8BRkVaDQ8u8rDl+Eh2cc0XOuVmR\nBbtuAG6PvOZ4M8s2s9fN7JvI1j/ymQ+Y2Ytm9oWZzTOzX0T2tzKzzyKvn2lmxyfun0lEpHwKCpIO\nngCGRpZrLu1x4AXn3JHAOGBkOa/9HTA1csy9+GV+1wLXAhOccz2ccwvKvOYR4Acz+5eZXW9mtZ1z\ni4ExwCOR10zAr1HxiHPuKOCnRJbNjjgSv45FP2C4mbUGhgDvR1oxuuNn4RQRCZQuPUi155zbYmZj\ngVuAHaWe6gcMitx/EfhTOS8/Dn8Sxzn3sZk1NbOG+/i8EWY2Dj/n/hDgUvxlh7JOA7r6pQwAaBhZ\nBRTg3865HcAOMxsP9MWvWfJsZBGwN51zCgoiEji1KEi6eBQ/j369ZHyYc26Bc240fjGj7mbWtJzD\nMoBjIi0MPZxzbZxz26Jv8eO3dJ8BJ+BXBnzezC4P7AuIiEQoKEhacM5tBP6BDwtRk/ArUAIMBSaU\n89IJkecws5OA9c65LZV9lpmdYyXNBJ2AImAzsBVoUOrQ/wH/V+p1pTtGnm9mtSMB4yTgGzNrB6xx\nzv0df5miV2V1iIgkgoKCpJOHgWalHv8fcJWZTQd+DtxazmseAHpHjnmIkuV/K/NzfB+FafhLGkOd\nc0XA28CF0c6M+EshfSIdJWfhOztGTccvqfwl8Hvn3Ep8YPjOzKYCF+P7OIiIBEqrR4pUMZF5F7Y5\n5/4Sdi0iImpREBERkQqpRUFEREQqpBYFERERqZCCgoiIiFRIQUFEREQqpKAgIiIiFVJQEBERkQop\nKIiIiEiF/j8yuMhPNTWHQgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f607e1f99e8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "\n",
    "def UCB_bandit(arms=10,std_dev_reward=1,init_expectation = 0,iterations=1000,epsilon=0.1,c=2):\n",
    "    #Initiations. Same as previous\n",
    "    no_of_actions = arms\n",
    "    \n",
    "    optimal_action_values = list_optimal\n",
    "    expect = [init_expectation]*no_of_actions\n",
    "    act_occur = [0]*no_of_actions\n",
    "    \n",
    "    def get_maxvalue_index(list_):\n",
    "        #Get list of indexes which contain maximum value in a list\n",
    "        indexes_of_max_values = np.argwhere(list_ == np.amax(list_)).flatten().tolist()\n",
    "        #return an index randomly\n",
    "        return random.choice(indexes_of_max_values)\n",
    "\n",
    "    avg_rward_list = [0]\n",
    "    avg_rward_value = 0\n",
    "\n",
    "\n",
    "    counter = 0\n",
    "    #print (optimal_action_values)\n",
    "   \n",
    "    for t in range(iterations):\n",
    "\n",
    "        if t==0:\n",
    "            #act_index --> Index of arm upon which action is made\n",
    "            act_index = get_maxvalue_index(expect)\n",
    "        else:\n",
    "            #c*Sqrt terms \n",
    "            sqrt_term = c*(np.sqrt(np.log(t)/np.add(act_occur,1e-4)))   #1e-4 to avoid division by zero\n",
    "            #The list of values over which the max value is takes to choose index\n",
    "            decision_list = expect + sqrt_term\n",
    "            act_index = get_maxvalue_index(decision_list)\n",
    "        \n",
    "        counter += 1\n",
    "        \n",
    "        \n",
    "        act_occur[act_index]+=1\n",
    "        rward_t = np.random.normal(optimal_action_values[act_index],std_dev_reward)\n",
    "\n",
    "        expect[act_index] = (((expect[act_index]*(act_occur[act_index]-1))+rward_t)/(act_occur[act_index]))\n",
    "\n",
    "        avg_rward_value = ((avg_rward_value*(counter-1))+rward_t)/(counter)\n",
    "        avg_rward_list.append(avg_rward_value)\n",
    "        #print  (act_index)\n",
    "    #plt.plot(avg_rward_list,color=colr)\n",
    "    #print (\"Average Reward : Greedy : \\t\",avg_rward_value)\n",
    "    return avg_rward_list,avg_rward_value\n",
    "\n",
    "\n",
    "def UCB_experiment(arms=10,std_dev_reward=1,init_expectation = 0,iterations=1000,epsilon=0.1,c=2,no_of_exp = 500):\n",
    "    rward_list = [0]\n",
    "    rward_value = 0\n",
    "    for x in range(no_of_exp):\n",
    "        avg_lists, avg_values = UCB_bandit(arms,std_dev_reward,init_expectation,iterations,epsilon,c)\n",
    "        if x == 0:\n",
    "            rward_list = avg_lists\n",
    "            rward_value = avg_values\n",
    "        else:\n",
    "            rward_list = np.add(np.dot(x,rward_list),avg_lists)/(x+1)\n",
    "            rward_value = ((x*rward_value)+avg_values)/(x+1)\n",
    "  \n",
    "    print (\"epsilon : \",epsilon,\" c: \",c,\"\\t average reward value\",rward_value)\n",
    "    return (rward_list)\n",
    "\n",
    "\n",
    "fig, axes = plt.subplots(nrows=1, ncols=1, figsize=(6, 6))\n",
    "axes.plot(greedy_experiment(epsilon=0.1),color='r', label=\"epsilon=0.1 c=0\")\n",
    "axes.plot(UCB_experiment(epsilon=0,c=2),color='b',label=\"epsilon=0 c=2\")\n",
    "axes.set_xlabel(\"No of Steps\")\n",
    "\n",
    "axes.set_ylabel(\"Average Reward\")\n",
    "axes.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"8\"></a>\n",
    "## <span style=\"color:#2595bc\">Incremental Implementation</span>\n",
    "Let $R_i$ denote the reward after $i$th selection of a specific action $a$. Let $Q_n$ denote the estimate of it's action value after $a$ has been selected $n-1$ times, then we know that\n",
    "$Q_n = \\frac{ R_1+R_2+.+.+R_{n-1}}{n-1}$\n",
    "\n",
    "There is some math involved in the subsequent step. If you are not able to get it, fret not. Jump down the equations where I have explained the result.\n",
    "So, the expression for $Q_n$ can be further simplified in a way that we don't need to store all the $R_{i}$'s to compute the sum. Given $Q_n$  and the $n$th reward, $R_n$, the new average of all $n$ rewards can be computed by - "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"images/eq.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The <span style=\"color:#f31818\"><i>$\\alpha$</i></span> here is known as <span style=\"color:#f31818\"><i>step size</i></span>. Note that as far we have studies, $\\alpha$ changes with time and is not constant. The equation can also be expressed as - \n",
    "\n",
    "$$\n",
    "NewEstimate = OldEstimate + StepSize \\Biggl[Target - OldEstimate\\Biggr]\n",
    "$$\n",
    "\n",
    "The expression $[Target - OldEstimate]$ is an <span style=\"color:#f31818\"><i>error</i></span> in the estimate. At each step we try to predict the $Action Values$ accurately so that we can make a proper decision. But the <i>error</i> reflects how much our prediction$(Q_n)$ differs from the actual rewards we obtain. We need to reduce this error. It is by modifying our prediction by taking a step toward the “Target.” The target is presumed to indicate a desirable direction in which to move. The actual reward $R$ is our desired output as our prediction ideally should give this output. Therefore we change our $OldEstimate$ by a value $[Target - OldEstimate]$ which is nothing but the error."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"9\"></a>\n",
    "## <span style=\"color:#2595bc\">Non-Stationary Problem</span>\n",
    "\n",
    "The averaging methods discussed so far are appropriate in a stationary environment,\n",
    "but not if the bandit is changing over time. In such cases\n",
    "it makes sense to weight recent rewards more heavily than long-past ones. One of\n",
    "the most popular ways of doing this is to use <b>constant step-size</b>.\n",
    "\n",
    "So, we can take a constant $\\alpha$ and modify our update rule as \n",
    "$$Q_{n+1} = Q_{n} + \\alpha\\Biggl[R_n - Q_n\\Biggr]$$\n",
    "\n",
    "where the step-size $\\alpha \\in (0,1]$ i.e $\\alpha$ can vary from 0 to 1(excluding 0 and including 1)\n",
    "\n",
    "This results in $Q_{n+1}$ taking a very interesting form. $Q_{n+1}$ becomes a <b><i>weighted average</i></b> of past rewards and the initial estimate $Q_1$\n",
    "\n",
    "$$Q_{n+1} = Q_n + \\alpha[R_n - Q_n]$$\n",
    "\n",
    "<span style=\"color:#f31818\"> $$= \\alpha R_n + (1-\\alpha)Q_n$$ </span>\n",
    "\n",
    "This intermediate step gives very vital insight to what we are doing. Initially when we were simply taking the average of all the rewards we got, we gave <b>equal weightage</b> to each reward we got for computing the expectation/prediction, disregarding at which time step we got the reward. However here, we assign a weight of $\\alpha$ to the latest reward we get, which is $R_n$ and a weight of $1-\\alpha$ to the previous estimate $Q_n$ Now we will explore what weights we give to the returns we get at an arbitrary time step $R_t$ under this method\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"images/3.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"images/average.png\">\n",
    "<a id=\"9a\"></a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "alpha :  0.2 \t epsilon :  0.0 \t average reward value 2.170798746883593\n",
      "alpha :  0.2 \t epsilon :  0.1 \t average reward value 2.484326257845963\n",
      "alpha :  0.0 \t epsilon :  0.1 \t average reward value 0.6657153504816635\n",
      "alpha :  0.0 \t epsilon :  0.0 \t average reward value 0.6669434270554765\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7f607e152ba8>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAioAAAF3CAYAAACLyGAHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzs3XlcVXX++PHXuZftsiq7gICCoICS\nQhRWrmMxYVYqWVKNpVmW45hl+rNyZvzaNNlikqY1uWQ5VkOOVqO0mJqmZWghKuDKVVERENm3e+/5\n/XFwZzPZ1Pfz8eBx7z3nc895n9s13nzO5/N5K6qqIoQQQgjRHunaOgAhhBBCiPpIoiKEEEKIdksS\nFSGEEEK0W5KoCCGEEKLdkkRFCCGEEO2WJCpCCCGEaLckURFCCCFEuyWJihBCCCHaLUlUhBBCCNFu\nSaIihBBCiHbLqq0DuFLu7u5qYGBgW4chhBDXlB07duSrqupxlcfwtLKy+gCIQP7QFc3DAuw2mUzj\noqKiTtXV4JpLVAIDA0lNTW3rMIQQ4pqiKIrxao9hZWX1gbe3dw8PD49CnU4nheLEVbNYLEpeXl7Y\nyZMnPwCG1dVGMmIhhBBNFeHh4VEsSYpoLjqdTvXw8ChC66Wru00rxiOEEOLappMkRTS32u9UvfmI\nJCpCCCFuKCtWrHCZMWOGN8CUKVN8Zs6c6dVS59q8ebN9SEhImL+/f8SYMWM6WyyW33WcrKwsm27d\nuoVfbZvf45133nELCAiICAgIiHjnnXfc6mrz5JNP+nXp0iU8JCQkbMiQIUH5+fn65jq/JCpCCCFu\nKImJiUX/+Mc/TrbGuZ5++umAhQsXGrOzs3cfOnTILjk52bk1zttccnNz9a+99prP9u3bM1JTUzNe\ne+01n7y8vMuSkLvuuqt43759e/bt27c3ODi48uWXX/ZurhgkURFCCHHNePfdd1179uzZo3v37mGj\nR48OMJlMANjb2/ceO3Zs5+Dg4PDY2NiQ48ePWwHMnj3bMygoKDwkJCRs6NChXQGSkpLcHn30Uf9L\nj71161ZDZGRk97O9Amd/IcfExIROmDDBt2fPnj0CAwMjUlJSHJsSq9FotC4tLdUNHjy4TKfTkZiY\nWLB69eqODb0nKyvLJioqKjQsLKxHWFhYj2+//dbh0jZJSUlugwcPDoqJiQkNCAiIeO655zqd3Wc2\nm3nwwQcDgoODw2+77bZupaWlCsCbb77pHhER0SM0NDTsrrvuCiopKWnS7//Vq1e79OvXr9jLy8vs\n4eFh7tevX/GqVatcLm03fPjwYmtrawBiY2PLcnJybJpy/Ka45mb9CCGEaHtTk9M67ztZYt+cxwzx\ndip/fWTk0fr279y50y45Odk1NTU109bWVn344Yf9Fy1a5DZx4sSCiooKXXR0dNnixYuPPv/8852m\nT5/us3z58iNJSUneRqMx3WAwqI3djhgzZkyXuXPnHomPjy+dPHmyz7Rp03yWLFlyFMBkMinp6ekZ\nn376qcusWbN84uLi9qWlpdmOGjUqqK5jbdmyJctoNFp36tSp5uy2gICA6hMnTlg3FIOPj49p8+bN\n++zt7dX09HTbhx56qOvu3bszLm23a9cuh/T09D2Ojo6W3r17h917771FXl5epiNHjth9/PHHh/r2\n7Wu8++67uy5fvrzj008/fToxMbHwueeeyweYNGmST1JSkvuLL754auHCha7z5s27rPcjMDCwMiUl\n5VBOTo61n59f9dntvr6+1Tk5OQ1ew7Jly9xHjhx5uqE2V0ISFSGEENeElJQUp927d9tHRkb2AKis\nrNR5enqaAHQ6HePGjTsN8PjjjxcMHz48GCA0NLTi/vvv7zJs2LAziYmJZ+o7dkFBgb6kpEQfHx9f\nCvDEE08UJCQkdD27PyEhoRCgb9++ZVOnTrUBiIyMrMrMzNzbnNdYXV2tjB07NmDv3r0GnU6H0Wi0\nravd7bffXuzt7W0GiI+PL9y4caPjqFGjzvj6+lb17du3AqB3797l2dnZtgA7duwwzJw507ekpERf\nVlam79+/fxHAhAkTTk+YMKHZkopp06Z56/V69amnnmr/iYqiKJ2B5YAXoALvq6o675I2A4A1wOHa\nTatUVZ3VUjEJIYRoHg31fLQUVVWVhISEggULFuQ01lZRFAA2bNiwf926dU5r1qxxeeONNzplZWXt\n+T3ntrOzUwGsrKwwm80KQGM9KgEBATUX9qAYjUabC3tY6vLKK694eXp61nz++eeHLRYLBoMhqqHr\nu/S1jY3NuVlZer1eraio0AGMHz++S3Jy8oHY2NiKpKQkt02bNjkBNNaj4uvrW3O2LUBOTo5N//79\nS+qKKSkpye3rr7/usHnz5n06XfONLGnJHhUT8JyqqjsVRXECdiiK8q2qqpdmn5tVVR3agnEIIYS4\nDsTFxRUPHz48eMaMGbm+vr6m3NxcfVFRkT4kJKTaYrGwdOnSjuPHjy9ctmyZW0xMTInZbObgwYM2\n99xzT8mdd95Z2rlzZ9eioqI6b/+4ubmZnZ2dzSkpKY5xcXGlixcvdouNjS1tKJ7GelTc3d3Njo6O\nlvXr1zsMHDiwbMWKFW7PPPNMnauvnlVUVKT38/Or1uv1zJ8/381sNtfZbsuWLc65ubl6BwcHy9q1\nazt88MEH2Q0dt7y8XOfv719TVVWlfPLJJ65nE6bGelTuu+++olmzZvmeHa+zadMm57lz5x67tF1y\ncrLzvHnzvDdv3pzl5OT0+6Y21aPFBtOqqnpCVdWdtc9LgAzAt6XOJ4QQ4voWFRVV+dJLL+UMHjw4\nJCQkJGzQoEEhR48etQYwGAyW7du3O3Tr1i38hx9+cHr11VdPmEwmZfTo0V1CQkLCIiIiwsaNG3fK\n3d297t/8wNKlSw9PmzbNLyQkJGzXrl2Gf/7zn8evNuYFCxYYn3rqqcCAgICIwMDAqoSEhCKAOXPm\neMyZM+eykgaTJ08+tXLlSrfQ0NCwzMxMO4PBUOcv/V69epUNGzYsKDw8PPyee+4p7NevX3lDcUyf\nPv14TExMj+jo6O7dunWrbGr8Xl5e5qlTpx6PiorqERUV1eOFF1447uXlZQYYNWpUwA8//GAPMGXK\nFP+ysjL9oEGDQmoHOl82WPn3UlS15dfuURQlEPgBiFBVtfiC7QOAz4FjwHHgeVVVG+yWi46OVmUJ\nfSGEuDKKouxQVTX6ao6RlpaWHRkZmd9cMTUne3v73uXl5b+2dRytISkpyS01NdVh+fLlR9o6luaS\nlpbmHhkZGVjXvhYfTKsoiiNaMjL5wiSl1k4gQFXVUkVR7gZWA93qOMZ4YDyAv3+zJWlCCHFtKDoG\nBQfBxQ/c6hwSIcR1q0UTFUVRrNGSlBWqqq66dP+FiYuqqmsVRXlXURR3VVXzL2n3PvA+aD0qLRmz\nEEK0CxYLHNkGm/4Jh3/Qtt08DuLfbNu42qkbpTcFYNKkSQVAQVvH0VpactaPAiwGMlRVfaueNt5A\nrqqqqqIoMWhjZm6YD18IIS5Tlg/b5sOuz6A4B2xdYMAM8I6ArgPaOjohWl1L9qjcBjwCpCuK8lvt\nthmAP4CqqouAkcAERVFMQAXwoNoag2aEEKI9OXMUfloIx3+FE2lgqoBud8If/gahfwRbp8aOIMR1\nq8USFVVVtwBKI23mA/NbKgYhhGjXik/A97Nh16eACj69tcSk/zTwCGnr6IRoF2RlWiGEaA0WC1hq\ntF6TA9/B/m+03hMrO+jzCNzxnDZYVghxESlKKIQQLammAra9C3O6wGwvWHIX/PA6KDqIeRKe2ABD\n50qS0opWrFjhMmPGDG+AKVOm+MycOdOrpc61efNm+5CQkDB/f/+IMWPGdLZYLl8W5ddff7W76aab\nutvY2PT5PbHExMSEnl3P5GraXKnMzEybXr16dff394+Ij4/vWllZedldlP/+97/O4eHhPUJCQsLC\nw8N7fPHFF1d8H1N6VIQQoiVUl8NP78LWd6DyDHj1hF6jwCsMAu+QacZtKDExsQgoao1zPf300wEL\nFy40Dhw4sGzAgAHdkpOTnR944IGLlurw9PQ0zZs370hycnKDlZXbmylTpvhNnDgxd/z48YWjR4/2\nnzdvnvu0adPyLmzj6elZ87///e9AYGBgzS+//GIXHx8fcurUqV1Xch7pURFCiKtVfhpOH4aKM9pU\n4k1z4J0+8P3/gf+t8Kev4KnNcPcciBojScpVePfdd1179uzZo3b10wCTyQRoC76NHTu2c3BwcHhs\nbGzI8ePHrQBmz57tGRQUFB4SEhI2dOjQrqAtmPboo49etijX1q1bDZGRkd1DQkLChgwZEnR22fiY\nmJjQCRMm+Pbs2bNHYGBgREpKimNTYjUajdalpaW6wYMHl+l0OhITEwtWr159WTLi6+tr6t+/f7m1\ntXWDk0kSExP9IyIiegQHB4c/++yzPnW1qe9zAFi5cmXHS68hKyvLJioqKjQsLKxHWFhYj2+//dah\nKddmsVjYtm2b02OPPVYIWiHIL7/8ssOl7W677baKwMDAGtBWFq6qqtJVVFQ0OH71UtKjIoQQV0pV\n4dgvcPRnOLQJDq4H9ZIuff9YuG8hBA1smxhb2upnOnNqb7PeSsAzrJz7FtRb7HDnzp12ycnJrqmp\nqZm2trbqww8/7L9o0SK3iRMnFlRUVOiio6PLFi9efPT555/vNH36dJ/ly5cfSUpK8jYajekGg0HN\nz8+vs87PWWPGjOkyd+7cI/Hx8aWTJ0/2mTZtms+SJUuOAphMJiU9PT3j008/dZk1a5ZPXFzcvsaK\nEhqNRusLixAGBARUX1ik8Eq99dZbOV5eXmaTyUTfvn1Df/75Z8Mtt9xScWGb+j6H+q7Bx8fHtHnz\n5n329vZqenq67UMPPdR19+7dGYWFhbrY2NjudcWxYsWKQz4+PiYnJyeztbV2OYGBgdW5ubk2DcX/\n4YcfdgwPDy83GAxXNLtXEhUhhGgqVYX8fbBuGhzaoG2zcYLoseDaRetZ8Y+FTpHgeFkZF3GVUlJS\nnHbv3m0fGRnZA6CyslLn6elpAtDpdIwbN+40aH/dDx8+PBggNDS04v777+8ybNiwM4mJiWfqO3ZB\nQYG+pKREHx8fXwrwxBNPFCQkJHQ9uz8hIaEQoG/fvmVTp061gcaLEja3Dz/80HXZsmXuJpNJycvL\ns05LS7O7NFGp73Oo7xqqq6uVsWPHBuzdu9eg0+kwGo22AB07drQ0dG0nTpy4ovwhNTXVbubMmb4p\nKSn7r+R9IImKEEI0rKYCsrdoM3R2faolKlZ2cNc/tLVOXIOgGUvaXzMa6PloKaqqKgkJCQULFizI\naayttuYobNiwYf+6deuc1qxZ4/LGG290ysrKarCeXH3s7OxUACsrK8xmswLQWI9KQEBAzYU9KEaj\n0ebCHpYrkZmZaTN//nyvHTt2ZHh4eJhHjBgRWFlZ2egX7+znUN81vPLKK16enp41n3/++WGLxYLB\nYIgCaKxHpXfv3pUlJSX6mpoarK2tyc7OtvHy8qquq/3BgwetR44cGbx48eLD4eHhVVd67ZKoCCFE\nXc4cgQPrYdsCKKj9I9CjO9w+BaL+BB0D2zS8G1FcXFzx8OHDg2fMmJHr6+trys3N1RcVFelDQkKq\nLRYLS5cu7Th+/PjCZcuWucXExJSYzWYOHjxoc88995TceeedpZ07d3YtKiqq8/aPm5ub2dnZ2ZyS\nkuIYFxdXunjxYrfY2NjShuJprEfF3d3d7OjoaFm/fr3DwIEDy1asWOH2zDPPnPo9115YWKg3GAwW\nV1dX89GjR602btzo0r9//5JL29X1OTR03KKiIr2fn1+1Xq9n/vz5bmazVly6sR4VgFtvvbXk7LmW\nLFniNnTo0Mt6rPLz8/V33313t7///e/H7rzzzrIrvGxAEhUhhDjPVA3pn2krxW6ZC+YqLTkZuRQC\nbgOnFpvFKpogKiqq8qWXXsoZPHhwiMViwdraWk1KSjoSEhJSbTAYLNu3b3d4/fXXfdzc3GpWrVp1\nyGQyKaNHj+5SUlKiV1VVGTdu3Cl3d3dzfcdfunTp4QkTJgRMmjRJ5+/vX7Vy5crsq415wYIFxrFj\nx3aprKxUBg4cWJyQkFAEMGfOHA+AF154Ie/IkSNWN998c1hZWZleURT1vffe88rIyNjt6up6buBT\nbGxsRURERHlQUFBEp06dqqOioupMour6HBqKb/LkyadGjBgR9Mknn7gNGjSoyGAwXD5/uh5vvvnm\nsVGjRgXNnj3bNzw8vPwvf/lLPmjTv3/55ReHt99++/icOXM8jxw5Yvvqq6/6vPrqqz4A69ev3+fr\n62tq6nmUa23F+ujoaDU1NbWtwxBCXA8sFsjLhIIDsP9r2P8dlJ7U9gUNhrteAffQ6+LWjqIoO1RV\njb6aY6SlpWVHRkbmN96y9dnb2/e+kQoT1uda/RzS0tLcIyMjA+vaJz0qQogbh6qComiF/35eBDuX\nQ2muts/KAL594J55WgFAZ1+trRCiTUmiIoS4vlnMkLYSNr8Fpae02TgluVBTBr5RMHgmuIeAVwTY\nNO9sW9F6rsVehJZwPX4OkqgIIa5PpirY8Ar8sgSqa8cT+sdqM3b8YuCOKeAR2rYxCiEaJYmKEOL6\nUF0GP7+nFfurqYATv2nbI0ZAyB8h/H7Qy//yhLjWyL9aIcS1w1QNpw+BoYNW1O/wD1CcA4VGyPgS\nyk6BtT1499Lq6oQPh9C4to5aCHEVJFERQrRP5hrIWgvGbdqaJtUlcGKXVuDvUrbO0DkG+r0A/re0\nfqxCiBZz7c+5E0JcG0xV2nTgS50+BFkp2u0agKPbYcOr8HYv+OxR2P4+5GdBVSl0GwIDX4TbJsMf\n/gZ/+hImbIPpR+DhzyVJEU2yYsUKlxkzZngDTJkyxWfmzJkttkDO5s2b7UNCQsL8/f0jxowZ09lS\nx7+BX3/91e6mm27qbmNj0+f3xBITExP6ww8/NDgSvCltrlRmZqZNr169uvv7+0fEx8d3raysbJFp\nctKjIoRoOeWntTVKTh+GdS9ovSFu3cDGQZv6W3z8gunBduDgAUW1K7N3ugniXoWu/cFwWcFZIX63\nxMTEIqCoNc719NNPByxcuNA4cODAsgEDBnRLTk52fuCBB4ovbOPp6WmaN2/ekeTk5Gvqiz5lyhS/\niRMn5o4fP75w9OjR/vPmzXOfNm1aXnOfR3pUhBDNpygH9q6B9bPg4xEwNxwWD4H/jtd6VHqNAs/u\n4OCu3a4JGgR3vQoPfATRj2tThP/wN/jzTnjiewi/T5IUcZF3333XtWfPnj26d+8eNnr06ACTSVvg\n1N7evvfYsWM7BwcHh8fGxoYcP37cCmD27NmeQUFB4SEhIWFDhw7tCpCUlOT26KOP+l967K1btxoi\nIyO7h4SEhA0ZMiQoLy9PD1pvxIQJE3x79uzZIzAwMCIlJcWxKbEajUbr0tJS3eDBg8t0Oh2JiYkF\nq1evvuwL7evra+rfv3+5tbV1gyuwJiYm+kdERPQIDg4Of/bZZ33qalPf5wCwcuXKjpdeQ1ZWlk1U\nVFRoWFhYj7CwsB7ffvutQ1OuzWKxsG3bNqfHHnusELQCiF9++WWHprz3SkmPihDi97NYtMGsGV/C\nrx/BqdrSIDor8OgBvR6A4CHg3Am8IxuedRM2rHViFs3i5R9f7nyg8ECz3koI7hhc/n+3/V+9xQ53\n7txpl5yc7Jqampppa2urPvzww/6LFi1ymzhxYkFFRYUuOjq6bPHixUeff/75TtOnT/dZvnz5kaSk\nJG+j0ZhuMBjU/Pz8Ouv8nDVmzJguc+fOPRIfH186efJkn2nTpvksWbLkKIDJZFLS09MzPv30U5dZ\ns2b5xMXF7WusKKHRaLS+sAhhQEBA9YVFCq/UW2+9lePl5WU2mUz07ds39OeffzZcWj25vs+hvmvw\n8fExbd68eZ+9vb2anp5u+9BDD3XdvXt3RmNFCX18fExOTk5ma2vtcgIDA6tzc3Ntfu+1NUQSFSFE\n3coK4GSadgtGbw2HNmm3bnL3QvYP4HczHP9V+wFw9II//B263AGe4WBt17bxi+tOSkqK0+7du+0j\nIyN7AFRWVuo8PT1NADqdjnHjxp0G7a/74cOHBwOEhoZW3H///V2GDRt2JjExsY6R2JqCggJ9SUmJ\nPj4+vhTgiSeeKEhISOh6dn9CQkIhQN++fcumTp1qA40XJWxuH374oeuyZcvcTSaTkpeXZ52WlmZ3\naaJS3+dQ3zVUV1crY8eODdi7d69Bp9NhNBptofGihCdOnGi1/EESFSFuBDUVkPk/sHPREoyyPK0q\ncNExCLxdGzOSvx8qTmu3b/RWcCINKhu4jX8yHZx8tIGtrl0h7F5t2rC4ITTU89FSVFVVEhISChYs\nWJDTWFultvzBhg0b9q9bt85pzZo1Lm+88UanrKysPb/n3HZ2diqAlZUVZrNZAWisRyUgIKDmwh4U\no9Foc2EPy5XIzMy0mT9/vteOHTsyPDw8zCNGjAisrKxsdPiGckEZiLqu4ZVXXvHy9PSs+fzzzw9b\nLBYMBkMUQGM9Kr17964sKSnR19TUYG1tTXZ2to2Xl1f177m2xkiiIsT1SFVhzyotEaksgp8WaUvG\nA6CATq89WmrgwLfn39chQFufxN5VSz58o8HWSRvgGjFCO65fNDh6asvR2zpLz4loNXFxccXDhw8P\nnjFjRq6vr68pNzdXX1RUpA8JCam2WCwsXbq04/jx4wuXLVvmFhMTU2I2mzl48KDNPffcU3LnnXeW\ndu7c2bWoqKjO2z9ubm5mZ2dnc0pKimNcXFzp4sWL3WJjY+usUHxWYz0q7u7uZkdHR8v69esdBg4c\nWLZixQq3Z5555tTvufbCwkK9wWCwuLq6mo8ePWq1ceNGl/79+5dc2q6uz6Gh4xYVFen9/Pyq9Xo9\n8+fPdzObteLSjfWoANx6660lZ8+1ZMkSt6FDh9bbY3U1JFER4lp35Cett+TINsjZAV36afVtsjfX\nNlAgaKDWc6LotLVITmXAfe9qC6PlZ4HFpBXhc3Bv+nkdPVvkcoSoT1RUVOVLL72UM3jw4BCLxYK1\ntbWalJR0JCQkpNpgMFi2b9/u8Prrr/u4ubnVrFq16pDJZFJGjx7dpaSkRK+qqjJu3LhT7u7u5vqO\nv3Tp0sMTJkwImDRpks7f379q5cqV2Vcb84IFC4xjx47tUllZqQwcOLA4ISGhCGDOnDkeAC+88ELe\nkSNHrG6++eawsrIyvaIo6nvvveeVkZGx29XV9dxc5tjY2IqIiIjyoKCgiE6dOlVHRUXVmUTV9Tk0\nFN/kyZNPjRgxIuiTTz5xGzRoUJHBYKhjDYG6vfnmm8dGjRoVNHv2bN/w8PDyv/zlLy1SWVtR1QYH\nGbc70dHRampqaluHIUTbMVVrY0b2/FcrtJebrm3X1f7dYTGBjRMMfhm63QnWBnDybrt4RbugKMoO\nVVWjr+YYaWlp2ZGRkS3yy+hq2dvb974eC/JdqWv1c0hLS3OPjIwMrGuf9KgI0R7t+gy2zNXWFQm/\nH1Dh2A5tDInxRzCbzt/KGfD/oM+fwFwFHQOh+IQ25sTOuS2vQAghmoUkKkK0B2ZT7bgRYN002P7e\n+X2HN2mPBldw9tHGnHiGQ+SDWhLTofPFx3Lu1DoxC9GOXIu9CC3hevwcJFERoi2ZqmHrPK3qr6ka\nqmpn2fQYBiM+gIoz2tokNo7g2+d8MiOEEDcISVSEaEtbk+D72dotG88wKC/QBr4OfAmsbMHJS/sR\nQogblCQqQrSmqhKt9k3BQa2n5JfF2sqtDye3dWSinas2WdApYKWXyifixiKJihAtzVwDB76D47/B\nbyvOF90DcPCEP/y17WIT7V5ReQ0fbDnE/3adYPQt/oy7o2vjbxLiOiKpuRDNqegYZHylrWvy36fg\n3b7wzwBY+SBs+qd2a6ffVBjyf/D4NzB5F3j3bOuoRTukqipfpB1n4Jsbmb/hAE4Ga4I9m1QLTzRi\nxYoVLjNmzPAGmDJlis/MmTNb7P7q5s2b7UNCQsL8/f0jxowZ09liafIyJRfJysqy6datW/jVtvk9\n3nnnHbeAgICIgICAiHfeecetuY/fGOlREaIu1WWQuRaCB2urtF7q+G/wxZ+1pehveRJ6PgA7lkLq\nEi0ZuZBnGPR+BHoMBbsOMm1YNGrL/nzmfJ3JrmNFdPN05KOxMYT7uLR1WNeNxMTEIqCB+hDN5+mn\nnw5YuHChceDAgWUDBgzolpyc7PzAAw8Ut8a5m0Nubq7+tdde89mxY8denU5H7969wx588MEzHh4e\n9S6c19wkURECoKZS6w2xc9ZWdP11BRxcr+1z8Qe3rloNnJ4jobIY0v4NFYXa/u/+pv2gaL0jtz8L\nelvwCIGA2xuuGCzEBU6XVfPWt1l8/NMROrsamDOyFyP6+KHXKY2/+Qbx7rvvui5cuNCrpqZG6dOn\nT9ny5cuNVlZW2Nvb937ooYfyN23a5Ozh4VHz+eefH/Lx8THNnj3bc+nSpR56vV4NCQmp/Oqrrw4l\nJSW5paamOpytKnzW1q1bDRMmTAioqKjQBQQEVP373//O9vDwMMfExIRGRUWVbtmyxbmkpES/aNGi\n7Li4uAaX1wcwGo3WpaWlusGDB5cBJCYmFqxevbpjQ4lKVlaWzejRo7tUVFToAObNm3dkyJAhZRe2\nSUpKcluzZk2HkpISq9zcXOuRI0cWvPnmmycAzGYzDz74YEBqaqqjl5dX9ddff33A0dFRffPNN92X\nLl3qUVNTowQGBlYlJycfdnJyarR7Z/Xq1S79+vUr9vLyMgP069eveNWqVS5PPvnk6cbe21zk/6Di\nxqaq8M1LsG3+5fv8btaWnd/5ERzaqG3b+Kr22DEQEpPBpzfs/xZ+nAexz2i9JkJcoez8Ml5avZst\nB7RFX8fd3oXn7wrFzrr9Tkc/PuPFzlX799s35zFtu3Ur9/nHK/UWO9y5c6ddcnKya2pqaqatra36\n8MMP+y9atMht4sSJBRUVFbro6OiyxYsXH33++ec7TZ8+3Wf58uVHkpKSvI1GY7rBYFDz8/Mb/EDH\njBnTZe7cuUfi4+NLJ0+e7DNt2jSfJUuWHAUwmUxKenp6xqeffuoya9Ysn7i4uH2NFSU0Go3WFxYh\nDAgIqL6wSGFdfHx8TJs3b973HwaNAAAgAElEQVRnb2+vpqen2z700ENdd+/enXFpu127djmkp6fv\ncXR0tPTu3Tvs3nvvLfLy8jIdOXLE7uOPPz7Ut29f49133911+fLlHZ9++unTiYmJhc8991w+wKRJ\nk3ySkpLcX3zxxVMLFy50nTdv3mVLVwcGBlampKQcysnJsfbz8ztXbNDX17c6JyenwWtobpKoiBuL\nxawNbD19SCuqty9Fm33TKRKc/SCgr3bbx9YJYp7Qlqof/Fett8VUCcd+gfDh2tThs1VJQ+O0HyGu\nkNmi8v4Ph3j7u33Y6HX8ZXA3+oV4EBXQsa1Da5dSUlKcdu/ebR8ZGdkDoLKyUufp6WkC0Ol0jBs3\n7jTA448/XjB8+PBggNDQ0Ir777+/y7Bhw84kJibWWzSvoKBAX1JSoo+Pjy8FeOKJJwoSEhLOjVxO\nSEgoBOjbt2/Z1KlTbaDxooS/R3V1tTJ27NiAvXv3GnQ6HUaj0baudrfffnuxt7e3GSA+Pr5w48aN\njqNGjTrj6+tb1bdv3wqA3r17l2dnZ9sC7NixwzBz5kzfkpISfVlZmb5///5FABMmTDg9YcKEVusd\n+T0kURE3jkMbYfXTUHxBhXhHb4h8CO5dUP9iaopyfvVX924tHqa4Mfx6pJA3vsnixwMF3BXuxd+H\nReDtcu1Uom6o56OlqKqqJCQkFCxYsCCnsbZK7R8SGzZs2L9u3TqnNWvWuLzxxhudsrKy9vyec9vZ\n2akAVlZWmM1mBaCxHpWAgICaC3tQjEajzYU9LHV55ZVXvDw9PWs+//zzwxaLBYPBENXQ9V362sbG\n5lwBP71er569hTR+/PguycnJB2JjYyuSkpLcNm3a5ATQWI+Kr69vzdm2ADk5OTZ1VW1uSZKoiOtT\nTQX87zktOSnLB7cgrecEIHaitvS8a9e6B8oK0UJUVSXtWBGvrctk26ECbKx0TIvrzoQBdf6uE5eI\ni4srHj58ePCMGTNyfX19Tbm5ufqioiJ9SEhItcViYenSpR3Hjx9fuGzZMreYmJgSs9nMwYMHbe65\n556SO++8s7Rz586uRUVFdf5F4ubmZnZ2djanpKQ4xsXFlS5evNgtNja2wXEojfWouLu7mx0dHS3r\n1693GDhwYNmKFSvcnnnmmVMNHbOoqEjv5+dXrdfrmT9/vpvZXPeY1S1btjjn5ubqHRwcLGvXru3w\nwQcfZDd03PLycp2/v39NVVWV8sknn7ieTZga61G57777imbNmuWbl5enB9i0aZPz3LlzjzV0ruYm\niYq4vuTu0ZKUr56Fk7vOb7eyhdv+olUT9u8LOpmZL1pPaZWJpPX72bw/n4wTxbgYrJkW152EaD/c\nHevs2Rd1iIqKqnzppZdyBg8eHGKxWLC2tlaTkpKOhISEVBsMBsv27dsdXn/9dR83N7eaVatWHTKZ\nTMro0aO7lJSU6FVVVcaNG3fK3d293tkqS5cuPTxhwoSASZMm6fz9/atWrlyZfbUxL1iwwDh27Ngu\nlZWVysCBA4sTEhKKAObMmeMB8MILL+Rd2H7y5MmnRowYEfTJJ5+4DRo0qMhgMNQ54LVXr15lw4YN\nCzp58qTNyJEjC/r161eelZVlU18c06dPPx4TE9PD1dXV1KdPn9LS0tImDYDy8vIyT5069XhUVFSP\n2niPnx1YO2rUqIBnnnkmr1+/fuVN+zR+H0VV1cZbtSPR0dFqampqW4ch2puyfPjoPjiZrr22cYTo\nx2DQy4ACVvX++xWiRWWdLGHSyl/Jyi2hm6cj90T6kBDtRycXQ6vGoSjKDlVVo6/mGGlpadmRkZH5\nzRVTc7K3t+99PRbkq0t9s5auZWlpae6RkZGBde2THhVx7ftpIXz9IqhmCImDoEEQdp/UyBFtymJR\n+WDLId74eh9OdlYsGRPNoO7ynRTiSkmiIq4tVaVaIT9re23cyfevQF4G2DpD9OMw8EXpPRFtLudM\nBS+v3s33maf4Qw9PXhvRCze5xdOibpTeFIBJkyYVAAWNNrxOSKIi2rd938DOD7UZOT594Lt66uJM\nTgdDh9aNTYgLmMwWMk+W8MHmQ3y16wQ6ncLLQ8MYe3uXtg5NiGuaJCqi/frhdfh+9vnXe9doj49+\noSUlxq3a+icBfdsmPiGAtKNn+MfaDH47eoYqkwUHGz2PxAbw+G1d6OzarOuhtQcWi8Wi6HS6a2tw\no2jXLBaLAtS7Sq4kKqJ92jQHNrwCfjEwYJo2OPbINujzp/NTijtFtm2M4oalqirf7s1lyY+H+enQ\nadwcbHgoxh/fDgYeiO6Mi32rLtzZmnbn5eWFeXh4FEmyIpqDxWJR8vLyXIDd9bWRREW0L2UF8K8B\ncOYI9HoQ7lt4fiqx/61tGpoQADVmC9OSd7Hq1xzcHGx4/LYuTB7SDWe76zY5OcdkMo07efLkBydP\nnowAZI6/aA4WYLfJZBpXX4MWS1QURekMLAe8ABV4X1XVeZe0UYB5wN1AOTBGVdWdLRWTaOeqy+DD\ne7Qk5fYp2sBYWe9EtCPbD5/mb1/sYe+JYv48KJhJg7thrb9xvqNRUVGngGFtHYe4sbRkj4oJeE5V\n1Z2KojgBOxRF+VZV1QtX8fsj0K325xZgYe2juBH98Aac2gN3v6HV2RGiHbBYVDbtz2P1rzl8mXYc\nBxsr3h51E/f19m3r0IS4IbRYoqKq6gngRO3zEkVRMgBf4MJE5V5guaqtOveToigdFEXpVPtecaMw\n18D6WVoF48iHJEkR7UZljZkpn/3G2vSTONtZMaZvFyYNDqaDvUyBF6K1tMoYFUVRAoHewM+X7PIF\nLixsdax2myQqNwJV1Qr+pX2irY3SfSjEv9nWUQmBqqq8u/Eg/9p8iDPlNUyL687jtwdia9WkVceF\nEM2oxRMVRVEcgc+ByaqqFv/OY4wHxgP4+/s3Y3SiVZlNsOsTSP+PVo/nVAZU1X4lXLvCqI+1xEWI\nNqKqKhuz8vjoJyPfZ55icHdPxvfryi1d3do6NCFuWC2aqCiKYo2WpKxQVXVVHU1ygM4XvPar3XYR\nVVXfB94HrdZPC4QqWkrBQUhbqa2JUp+bEqHPo5KkiDaVW1zJq2szWP3bcRQFJg4M5rk7Q1DkeylE\nm2rJWT8KsBjIUFX1rXqafQFMVBTlE7RBtEUyPuU6Un4alsVDSe1/0l4PgnswRD0GlUXaEvhCtLHc\n4kr+/fMRFm06SJXJwjMDg3iyf9ANMd1YiGtBS/ao3AY8AqQrivJb7bYZgD+AqqqLgLVoU5MPoE1P\nfqwF4xGt7ZPRUJYHoz8Dv5vPL9QG4ODednEJAeSXVvH3L/eyLv0EJovKkDAv/jwomF5+UopBiPak\nJWf9bAEa7DOtne3zTEvFINqIqsJnj2gryd71Dwi5q60jEje4KpMZY0E5+aVVdHF3YGNWHn/9Yg8A\nj90WyOhbAuji7tDGUQoh6iIr04rmVV0Oe1dDxpfaEvc317vYoBAtbsv+fP6XfoIfD+Rz5HT5Rftu\n6eLKjLt7ENlZelCEaM8kURHNJ2cnrHwISk9qNXoeT9GqHgvRCnbnFPHRNiMZJ4ux0es4UVRJzpkK\nnGyt6Oxqz4g+fkQFdOREUQWeznYkxvij08lAWSHaO0lURPMwboWlf9Se3zwO+k+XJEU0G7NFRVVV\nMk6UsCvnDI62VhwrrGBt+gnyS6sorjBRUWPGxkqHp5MtHk62+HXUCgQ+NaCrrH8ixDVMEhVx9cpP\nw4Z/aM//vFNm84hmscNYyIqfjew0FpJbXEWVyYzlksUJevm5ENbJmSAPRwLc7Bl2ky8uBpmtI8T1\nRBIVcXWqSuGj++BEmjZwVpIUUQeLRWXp1mwqa8x083TExWCNlV5HXkklO4yFHMwr465wL8I6ufD5\nzmP8eCCf/adKsbPW0cuvAwNCPTHY6Al0syfC14W8Em1QbICbDIAV4noniYq4cpVFsG0B6K1h+7+g\n9BSMWAw9R7Z1ZKIdUFWVY4UV7DAWsu1gAZkni8ktruJkcWWd7XUKuBis+T7z1LnX3TydeG5ICI/2\nDZQeEiFucJKoiCv3n8fg4Przrwf8P0lSbnA1ZgvJO47xfeYpfjt6hrySKgAcba24qXMH/N0c6NfN\nnQGhnuScqaC4oobKGjNeznaEejtha6Uj40QJh/PLiPB1lp4SIcQ5kqiIpqsqhfV/15IUzzAYMB30\nNtDtzraOTLQAVVUxWVSs9ToASqtMWFSVvceL2X74NDFdXNl17AxfpB1nd45Ws8m3g4E7gt3p7d+B\nqABXQr2d0F8ys8bDybbO84X5OBPm49yyFyWEuOZIoiKaJmcnLLkLzNXg6KWtNtuhc+PvE+1CZY2Z\nr/ecxNnOmj4BHSkorWL5NiOnSirpG+SOk50VB/PKOHq6HJNFxcnOiq0H8skuKMevo4FTJVWYLSrm\nS0ezAj4udvTyc2Hs7V0YFukjtXGEEM1KEhXRuNw98K+B2vO+f4Y7ngNDx7aNSTTIYlFZk5ZDTmEF\npVVmlm/LprzaDGi1H610CmaLikWFteknz73P29mO/NIqDDZ6qk0WogI6Yq3X2g4I9URRoJunI2Gd\nnMk8WUKIlxOxQVJZWAjRciRREQ0rzIbkx7Xn/abCoJfaNJwbQVF5DX/9YjffZZwisrMLM4eGE+rt\ndG6/qqrnei3Kq01syMzjy7Tj/HS4gDPlNfTyc8Far2OHsRDQBqfeFux+LqHIPFFC1skSXk/oRTdP\nJ3YfL6LaZKGTix1dPRzPHf/C89Tllq6SoAghWp6ildu5dkRHR6upqaltHcaNobIY/jUICvZD/Ftw\n89i2jui6U1JZw1vf7mND5ik8nGxxtrNmfe3sF3sb/UW9IDZ6HX4dDZwpr6GnnwsZJ7TZNGc52Ohx\nsrPmZHElHeytmR7XnYHdPbHSKbg51j0uRNw4FEXZoapqdFvHIcSVkh4VcbmaCig4CKsnaEnKyKUQ\nMbyto2p1jfUo/B5mi0pZtQknW+2f3pvf7GPZ1mw8nWzJOllCcaUJnQKLHo7iznBvMk4UsyHrFAdy\nS3E2WGMsKONgXh4bs/Lw7WCgq7sDM+8J47Zgd6pMFhxtrSiprMHGSiersQohrguSqIjLrXwIDm3Q\nng/4fzdkkvLNnpNM+3wXvfw60MvPhZsDXTlWWMHwPr7YWev5PjOXl1dr1Xen/7E790T6UF5t4tu9\nuew5XkyQhwOH8soA0OsUjhVWUFljJuNkMcfPaOuJnB2Y+uDNnfnniF4AZJwoJsjDERsrbaZNj07O\n9Oh0+UwYk9mCVe1snLPOzs5xspN1R4QQ1w+59SMulrcPFtysPQ+7F+5/D6wNbRtTKziUV8p3GbmU\nVZn5Zm8uGSe06bauDjacLqu+qK2nky2nSqou2mZrpaPGbLlsifcLGaz16HUKfYPcOHK6nJJKE0PC\nvHh2SIgsaiZanNz6Edcq6VER56kqrBoH1g4wcTu4+LV1RC1OVVX+8slvfJF2/KLtLgZrPnsyllBv\nJ9KOniFlz0k+3mbEz9Weru4OOBuseHZICA42Vrz/wyGSvt/P4O5eDO3ViaiAjvx8+DQ1Zgveznb4\ndjTg72p/rsfj0nVFhBBC1E96VMR5BQfhnT4Q9xrc+lRbR9OiTpdVs2nfKRZvOczunGLuvcmHYZE+\n9PbviKuDDWaLekUJxemyajraW8saIqLdkh4Vca2SHhUB1eWw6gnI3a29DhrUtvG0oNTs07yQvItD\n+dr4ETtrHSOj/Jgzohe6CxKTK+31cHWwadY4hRBCaCRREbBuKmR+pT23dQb3bm0bTwsorTIxc81u\nVu3MobOrgQkDgojwceEPYZ4yO0YIIdoxSVRudCUnYc8acPKB8nwY9LK2aMd14Id9eby78QA7jIUo\nikK1yUKErzNzH7iJbl5OjR9ACCFEm5NE5UZWU6nV71EtMPpTrSflOpnh896mg7y6LvPc6/he3oyO\n8ee2YPc2jEoIIcSVkkTlRvbbx9oS+aM/g0692jqaq1ZSWcPSH7M5cKqUL9KO09XdgWWPxeDqaIOj\nrXzVhRDiWiT/975RFeXA/54DZz8Iuauto7kqe44XsfTHbJJ3HDu37Yk7uvD8XaEy/kQIIa5xkqjc\nqLa/pz2GxrVtHA0wmS28v/kQPx7Ix9/VnkduDeRfmw+x/fBpAt3tubWLG78ePcP3tbVxAOaP7s0d\n3TxkATUhhLhOSKJyIyo6BmmfgHdPbc2UduhkUSVjlm4n82QJAD9SwMrtRwHo6u7ATuMZfjxQcK79\nWw9EEhfhjb2NfKWFEOJ6Iv9Xv9GoKqydCqW5cN9C0Levr0CVycxf1+zhk1+OYrDW8/LQMPp1cyc9\np4j//prD0F6dGBnVmRqzhXW7T9DR3obbg90vq3sjhBDi+tC+fkuJlnfwe8haC/1egODBbR3NRSqq\nzTyy+GdSjYUEutmz6JEountrBfm6eTkxvM/5Jf31Oj33977+l/gXQogbnSQqN5KiHPi4thLy7c+2\nbSx1eC0lk1RjIeP7dWXG3T3aOhwhhBDtgCQqN4riEzA3THseMQJs7NskjMoaM2VVJlwdbM7VxTlR\nVMGflmxnX24pj9waIEmKEEKIcyRRuRFsfQe+eUl7HjEChn/Qaqf+aFs2X6adYHv2aQLd7MkuKD+3\nr4u7A0dPa69NFpX/98fujLuja6vFJoQQov2TROV6ZzHDlre15xEjYcQHrbZE/vJt2cxcs+fcaztr\nPUEeDlRUmzleVMnh2sKAAPf39uXJ/kGtEpcQQohrhyQq17t9X2s1fP44B255slVOqaoqCzYc4I1v\n9nFzYEeWjLkZGyvdRYuvmS0qOgUqayzYWOmuuFqxEEKIG4MkKte7I1tBZw19Hm2V0+3LLWHNbzks\n2HAQa73CWw/chJPd5YuvnU1MDDaycqwQQoj6SaJyPcveAr+ugMDbW7zYYFFFDc999hvfZWirxEYF\ndOQ/T8aik54SIYQQV0ESleuRqQo2/hO2vKW9vuWpFj2dqqrM+24/32WcopunIw/G+JMQ7SdJihBC\niKsmicr1oKoEVo2HXg9A+P3w3d/gp3fP7+/av1lPp6oqxwor6OxqT2WNmYn/3sl3Gae4o5s7H429\npVnPJYQQ4sYmicr1YNdn2mqzWWvhP2POb390DTh4NPttn3+szeBfmw9jZ62jssYCnK9WLIQQQjQn\nSVSuB8YftUdre6ipXadkyP9B1wHNeprN+/OY//0Bfj58GuBckjLj7u6M7ydTi4UQQjQ/SVSuZRYL\n7P8G9vwXQuPhoX/DmaNQmA0BfZvtNJU1Zv44b/O5dU+s9QprJ91BkIcjJ4or8e3QsgN1hRBC3Lgk\nUWnvzhyFDa+A381w89jz27O3wLrpkJuuvT67RkqHztpPMzl+poK/fPLruSTlnYd6MyTMCztrbVqx\nJClCCCFakiQq7d1/nwLjFkhbCd/PhueytJVll8Wfb/PHOc0+YLbKZGbXsSKeXrGTvJIqRt/iz8vx\nYbLuiRBCiFYliUp7ZbHA7mQtSenzKOxcDhWnIXWx1psC2pL4Ixc3+6lnf7WXD7YcPvd6/ujexPfs\ndK6IoBBCCNFaJFFpLUU5YK4C1yYU3fvu7+fXQAEY+BIM/iu8HgQp089vH5bU7GHuPV58UZKyYtwt\n3Bbs3uznEUIIIZpCEpXWUFMJc8PA0BGmZV++v+AgrHsBAu+Avn+GH98+v+/+98DJS3seOxG2zQcr\nAzzxPdg4NGuYZovK+I9ScbS1InlCLFY6hWBPp2Y9hxBCCHElJFFpaWeOwrcztecVhdqPoeP5/TUV\nMD8aVAsc+A6++6u2PfpxuHkceIWfbztklratY2CzVkBWVZX/7DhGSaWJY4UVvJEQSXdv52Y7vhBC\nCPF7tViioijKEmAocEpV1Yg69g8A1gBn7zOsUlV1VkvF0+ryD8Dnj8OJtIu3FxrPJyqqCh/8QUtS\nzvaWnDVgBjh6XPxenR5cuzR7qHO+zmLhxoPnTx3q0UBrIYQQovXoWvDYy4C4RtpsVlX1ptqf6ydJ\nAUiZdnGScuds7XFJ7UdydDv8exTk7tZ6SIbMgsm7YfBMbUXZS5OUFpKdX8bK7UfOvQ71csLd0bZV\nzi2EEEI0psV6VFRV/UFRlMCWOn67djJdu41z1osnoeIMfPMSmCq050viQDVr+x9ZrfWWdOgMdzzX\namHml1Zxz/wtWCwqn46/lVRjIffe5NNq5xdCCCEa09ZjVGIVRUkDjgPPq6q6p43juTrHUrUkpDz/\n/LYu/bVaO9YGuPVprVjgumnnk5TgIVqPShtY9mM25dVmVk3oS2TnDtzS1a1N4hBCCCHq05aJyk4g\nQFXVUkVR7gZWA93qaqgoynhgPIC/v3/rRXilPhisPQ6uHTz7XBYYXM/vj3sVjFth1yfn9zt5t26M\ntVb/msP8DQcIdLMnsnOHNolBCCGEaEy9iYqiKO8Aan37VVWddDUnVlW1+ILnaxVFeVdRFHdVVfPr\naPs+8D5AdHR0vTG1G+tngYt/3UlI2L1w4jctgWmjJKW82sT8DQcAmHpX9zaJQQghhGiKhgbTpgI7\nADugD7C/9ucmwOZqT6woirdSu9SpoigxtbEUXO1x20xe1sWvu8fX3c7/Vu2x64CWjKZB/1yXyYFT\npTzVP4j4Xp3aLA4hhBCiMfX2qKiq+iGAoigTgNtVVTXVvl4EbG7swIqirAQGAO6KohwD/gpY1x57\nETASmKAoigmoAB5UVbX995bUZ+s7F7++dULd7QL6arN6PMNaPqY6bNmfz/JtRuLCvZn+R+lNEUII\n0b41ZYxKR8AZOF372rF2W4NUVX2okf3zgfkNtbmmlF1yx8rFr/62XQe0ZCT1WrDhAK9/rfX8vDUq\nsk1iEEIIIa5EUxKVfwK/KoqyAVCAfsDfWjKoa07ap7BvHfR6EPb8F3z7aNON24l16SeYsGLnudeP\n39YFe5u2nvAlhBBCNK7B31a1Y0i+A9YBt9Runqaq6smWDuya8tO72uNNoyH+zWZd3v5q1ZgtPP3v\n80nKf56K5eZA1wbeIYQQQrQfDSYqqqqqiqKsVVW1J9py96Iuhg7g0QO69m/rSC6zMSsPVYVHYwP4\n+7BwlHaURAkhhBCNacoS+jsVRbm5xSNpT84cbXpbU7W2Eq33ZeWM2oWfDhWg1ynMuLuHJClCCCGu\nOU1JVG4BtimKclBRlF2KoqQrirKrpQNrM7v+A29HgHEbVJWCqar+tqoKc7pAeQF0al+DUy0WlVfX\nZbB4y2EGhHhgZ91+xswIIYQQTdWUEZV3tXgU7cmhjdpjXiYsjQOf3jB+4/n9/3sOcvfC4+u0asfV\npdr2ngmtHGj9ThVX8sRHO0g7egaA8f26tnFEQgghxO/TaKKiqqoRQFEUT7TF365vpgrtUantbDr+\n68X7f/lAe9z1GWSt05634SqzoA2YfW1dJoN7eHFT5w78Y23GuSTF3kZPtAyeFUIIcY1qNFFRFGUY\n8CbgA5wCAoAMILxlQ2tjhYfPPzduBRtH8Ag9vy3zf3AqQ3v+wPLWje0SX+85yQdbDvPBlvMxd/Vw\nYO2kO1AU0OtkbIoQQohrU1Nu/fwfcCvwnaqqvRVFGQg83LJhtSGldizHzo/Ob1v6R+3xvkXntxXn\nQMVpGPgSdLmj9eKrw+Z9l5VHYsqQEBmXIoQQ4prXlESlRlXVAkVRdIqi6FRV3aAoytstHllrK8kF\niwnMtYNnyy//5U/mV9pjn0dhZ20vin3b3laxWFS2HsrntmA3Voy7lWqThdIqE64OV12OSQghhGhz\nTZn1c0ZRFEfgB2CFoijzgLKWDasNfPYIzA3DXF5Yf5vMr8CnD4TefX5bPVN+q00Wvt5zElVVUVWV\n17/O5L1NB39XaBbL5SWQVFXlp0MFdJ2xlqOnK4iL0IoL2ljpJEkRQghx3WhKj8q9aEUDnwUSARdg\nVksG1SaO/gxAxbHdHLR0JVJ3qO52t/0F3ILPv/bpfe5pWZUJk1nF2WDF+I9S2ZiVd9nbx/frekXr\nmXyy/QjTV6XzxwhvFozug652vMkb32SxYIOW+EQFdOSRWwOafEwhhBDiWtGUROVB4AdVVfcDH7Zw\nPG3O0XSaErXTudcW1yB0py/oCekYoCUqYfeC380XJSqD39zEyeJKPh1/a51JCkBucRXeLk2bPFVR\nbeafKZkArNt9krC/prDn73EUlFWdS1K+nHg7od5OV3qZQgghxDWhKbd+/IH3FEU5rCjKfxRF+bOi\nKDe1dGCtztb53NMqzt86ec7854vbOXhqt3seWA59L953srgSgOyCi++MPXFHF16I02YMHS+qqPP0\nB/NKKa6sAbQqx13/3//oMTOFM+U1LHtMWxi4ssbC4i2H6DdnAwAzh4bR088FG6um/GcUQgghrj2N\n/oZTVfWvqqoOAsKAzcBUYEdLB9baLMr5GTJVWJ97viXXGga+eL6hg0ed7884UXzu+bTP0wG4pYsr\nf4oN4MX4MPp109439T9pF73PbFGJe/sHBr+5icFvbuJIQTmvf53F2WEpQR4O9A/xIG3mnQD8Y20m\nlTUW+ga58fjtXX7/BQshhBDXgKaso/IScBvgCPwKPI+WsFz7zhzRpiMf2Yau8vwg2mqsKFA64qYW\nUoEt9H8BNryi7bSqe6Dq29/tu+i1j4sdnz4Ze/51BwMAB/PKqDFbsNbrUFWVkYu2knmyBIC8kir6\nva71lsR2deNPfQOJi9AWknOxt+bhW/35+Kcj3B7szrsP92mez0AIIYRox5oyRmU4YAL+B2wCtqmq\n2kABnGvI2z3r3GyFhfsqZzJEt5NS7Pnsl6M8MOk3sDbU2X7Tvjy+3pN70bYlj11cx9HVwYYxfQNZ\ntjWbX4+c4buMXN7/QRuwa61X2PXXu/jTku1szz4NwKKHo3Cxt77oGDOHhvPwrQGEejlJgUEhhBA3\nhKbc+ukD/AHYDgwB0hVF2dLSgbUlCwpHVS+WmLWF3l74fBdzd9SgOnqxYMMBZvw3/aL2L9a+/kMP\nr3PbgjwcLzvus38IQQOQWz0AACAASURBVFHgq13HzyUpABunDsRgo2f52BhWP3Mb3z7b77IkBbSp\nx929nSVJEUIIccNoyq2fCOAOoD8QDRzlern1Uw97e3sounjbvPX7ub+3L69/nQXAxIHB527ndO5o\nz7HCChYk9mbUez/h6WSLtf7yHNDF3pru3s4s32YE4I5u7ng72+FTOwvIzlrPTZ07tOCVCSGEENeW\nptz6+SdaYpIE/KKqak3LhtQGnP1YW+TP3cpWADKcb7ssUQE4crr83PMPNh/mmYFBRM3+DoCEKD9s\nrfSsmtC3vjXgAG2AbcaJYmytdCwdczNWdSQ0QgghhNA0pXryUEVRDID/dZmkAMVFp3m6ag4d+BMG\nqunlHAbkXtYuaf3+c88raszM/GLPudd3hGizenSNFACM6eLKsq3ZhHg5SZIihBBCNKLR35SKotwD\n/Aak1L6+SVGUL1o6sNbkrGg9JWdw4gRu2Fidn6q87IJBsanGQuxt9ET4OrP3eBEbMk+d23dPr/OL\nxDXkDz28GHt7F16M79FM0QshhBDXr6bc+vkbEANsBFBV9TdFUa7rBTwqa8znnvt1vHimz5QhIZRX\nm3nr2/PTkb/68+1NHuBqY6Xj5aFhzROoEEIIcZ1ryr2HGlVVLx2xcXmVvOuIAtx7kw8Avh3suaOb\n+7l9d3TzIMTr/JL1s+4NJ8LXpbVDFEIIIW4ITUlU9iiKMhrQK4rSTVGUd4CtLRxXqzCp5y//gWi/\nc88dbK14fWQkm1/Qpg1/NPaWc/u8nG0v6mWpMV/XOZsQQgjRppqSqPwZCAeqgH8DxfD/27vz+KjK\ne4/jn98sWSEhIWEJ+yabAipuiBYUl9aFi9rWpdVal3qrdbna1mq1au116SK1tbZWrdh63VDABeuG\ninVBENkXAUH2JSQkEMgyM8/9Y05CWBMgk5nJfN+v17ycOefMzO/kqPnmeZ7zPNwYy6KahXMELFL3\n8sELBvOjk3sCcMHRnUkL+OiSn7XH23Izg7ssAnhcj/zY1yoiIpKizLkDbxEws67OuZUxqKdBQ4cO\ndTNmzDj0DwpVw73RO3XCzvDfvWW/h0/9chPZ6X6O7hYNJgvWlpObFaRTm73PVisikkjM7HPn3NB4\n1yFyoPY7mNbMTgA6AVOdcxvNbBBwK9EJ4Lo0Q32xE46uAvBi6GR+F/oO0xo4/OTDdl2McEBRzj6O\nFBERkaayz64fM/st8CRwPvC6md0LvAVMA/o0T3kxVFMJwFzXgw2o+0ZERCQR7a9F5SzgSOdcpZnl\nEZ06/3Dn3IpmqSzGwhXF+IFS17rBY0VERCQ+9jeYttI5VwngnCsFlrSUkAIQ3hqdrK0Y3VosIiKS\nqPbXotJztxloe9R/7Zw7N3ZlxV6ofCNpQIeOnZkweli8yxEREZG92F9QGb3b69/HspDmVlVRShYw\nclBvjuyaF+9yREREZC/2GVSccx80ZyHNraKigjwgN0djVERERBJVyi7fu2NHdCHCvFzdZiwiIpKo\nUjaoVHlBpU1rtaiIiIgkqkYHFTPbcz75JOZClYScj4yMtHiXIiIiIvvQYFAxs2FmtgBY5L0ebGZ/\niXllsRaqpIogGUF/vCsRERGRfWhMi8pDwBnAZgDn3Gzg5FgW1SxCVVQRJD2Qsr1fIiIiCa9Rv6Wd\nc6t22xSOQS3NykKVVJFGml9BRUREJFHtd1FCzyozGwY4MwsCNwALY1tWjC18jcKtC6kkiJnFuxoR\nERHZh8Y0J1wDXEt0FeU1wBDvdfJ6/hLa71hKjQXjXYmIiIjsR4MtKs65YuCSZqil2VVaZrxLEBER\nkf1oMKiY2cN72VwGzHDOTWr6kppPlWXEuwQRERHZj8Z0/WQQ7e5Z4j0GAZ2BK8xsbAxri7kqn1pU\nREREElljBtMOAk50zoUBzOxR4ENgODA3hrXFXLWCioiISEJrTItKHtCq3utsIN8LLlX7epOZPWlm\nG81s3j72m5k9bGZLzWyOmR11QJU3gVCgRU22KyIi0uI0Jqg8CMwys3+Y2VPAF8BvzSwbeGc/73sK\nOHM/+78J9PEeVwOPNqbgphQOZDf3V4qIiMgBaMxdP0+Y2WTgWG/Tbc65td7zn+7nfVPNrPt+Pno0\n8LRzzgGfmlkbM+vonFvXuNIPknN1T2vStHKyiIhIImvstKyVwDqgFOhtZk0xhX4noP6Mt6u9bbEV\n2Tmpbnlml5h/nYiIiBy8xtyefCXR2Wg7A7OA44FPgFNiW9ouNVxNtHuIrl27HtJnuUgNtXPRbmnd\n7xArExERkVhqTIvKDcAxwNfOuZHAkcCWJvjuNUD9Jo3O3rY9OOcec84Ndc4NLSwsPKQvDYdCAPw5\nNJqa/D6H9FkiIiISW40JKpXOuUoAM0t3zi0C+jbBd78CXOrd/XM8UBbz8SlAOBwNKqWuFd3a6q4f\nERGRRNaYeVRWm1kbYCLwtpmVAl839CYzexYYARSY2WrgV0AQwDn3V2Ay8C1gKbAduPxgTuBA1QaV\nMH6O7NqmOb5SREREDlJj7voZ4z29y8zeA3KBfzfifRc1sN8Rh8UNI6EaAEYN6Ei3tro9WUREJJHt\nN6iYmR+Y75zrB+Cc+6BZqoqh2hYV82vlZBERkUS33zEq3uyzi83s0G61SSC1g2nN749zJSIiItKQ\nxoxRyQPmm9lnQEXtRufcuTGrKoYitS0qvsacuoiIiMRTY35b3xHzKppRXdePTy0qIiIiia4xg2k/\nMLNuQB/n3DtmlgUk7W/52hYV1KIiIiKS8BqcR8XMrgLGA3/zNnUieqtyUoqE1KIiDatYtYKZd9xA\nxdw5uHrrQzUXV1NDZMcOXDjc8MEiIi1YY5oVriW6IOE0AOfcEjNrF9OqYigSid6ebP69n3pNuIbJ\nyydzTq9z8Fljl0KSZFZVspnFL/+DkrXLSevQkfDseRS8O5tMYOWLb1HeNpPM4SfQOqMNnW+4mWB+\nPtWrVlH68stk9OpF65Ej8WUf3K3uzjkqpk6lbOYMsvocRs6IkVR++SVLfno96Ws2U9G7I4MeHUda\nF61LJSKpqTFBpco5V20WXSHHzAJA8/+J2UQi3l+o+xpM+/jcx/nL7L+QEcjgjO5nNGdp0sxcJMLm\nWZ+x7JqryCkP0X63/a+f1obCVu0Z/OpigpOmUAksff5ltvbtROaydQRCEQC2divgiEfH4YtEWDX9\nfcqXLabvOZcQzG9LID8fFwrhwmHM5yNcXk54yxYyBg4En48Fjz6I7+GnACgH1tfWFoSvC6Hb0nUs\nO+10tncrhE4dyDlsAOltCyk882zSC9tT8fEnbF46j+pQFR36HUV6166k9+7dTD9BEZHYa0xQ+cDM\nbgMyzew04MfAq7EtK3Yi4WiLim8ftyeXz/mCF+4L8VXvtdC9GQuTmHPOsXTy86z/40NkFleQvT0a\nWnOALy//BrlDhtLK0tkWjFBRs42bRl2D3+dn/c9WMeHF38DbU+m53rG9Yg2uAywc0JrDFm1jwMpi\nVnzrrLrvyQRW/uu1BuuJ+AwijsVd/Uy78jhY/BVHf7ieLdkG117K90fexNMT7yE06Q06rdpExpeb\nSP9kLiEH5Q89Aj7wh3b+zbCGJwCo6NmBnlf9hJVTXsWXk0OvMd+neOk8tq9cTqezLyBnwBFN+nMV\nEYkla6j/3cx8wBXA6YABbwKPu3h03ANDhw51M2bMOOj3L5s5hV6vjOHzkx7n6FO/vcs+5xyTLjuV\nvp+tY811o+ly8eXkpufSIbvDoZYtcVCyYjEbZ3xE6fLF+LOzsadfptWWKja3Nsry06jJTqPi8J4U\nHDOMM865vsHPqw5XU1JZQtiF2VGzg955vYm4CK+9+WeC9/2VNUOK4PC+pHcoouz9KfT4fC1l2UZV\nACpy0wlWhcmqCLGhUxYDFlXQfgss6Oqj4+23M+r4iwGYVzyPnrk9yQpm7fK9K8tXUhWuYt6amcwd\n/xjdFm8hEDFCaX4yf3Axbvt2yj/+iII5qxm0IvqfZrUf0vYyxGXb8QPo9sP/ZuWbE8kt6k7vH92I\nBTS4vKUzs8+dc0PjXYfIgWpMUDkPeN05V9U8Je3foQaVJdPfos/r3+aLEf9gyDeiqwOYGb/8zy+Z\nuXEmNzy3na5zNrDoujO5s/U7+MzH7EtnN1X5coBCJaU4HwTb5DX6PZFQiE/+dAf5f9t1zHdFOizo\nn80Rv7yfIw8f1aR1llWVkZOWQ20X6S71uAg+81EdrqYqXEXrtNZU1FRQWllKuj+dwqxDWxG8vh2h\nHfzr1d9QUbyeXiedRcmKxYSWfUWgbQFb1n9N1ifzOH5uNYHIzvdsLcwmeNYougwZTlanrmQOGKgJ\nEVsgBRVJVo35M+oc4CEzmwo8D/zbOReKbVmxE6k3hf4ZL51Bmj+N18a8RuULEzh2B1SFcwBYu34J\nv5gcZklRBC6NZ8WpqaakhM9u/zH570VDYkn7LLo++Duq5s5j84YVdBh5Bt2GnQ5A5bq1lMz/ghV/\nuJ9A8RZCAR/5JdVUBWDhuQPZunUzhw0/m64nns6lnWPT7ZGbnrvPfbWDstP8aaT50wDIDmaTHWz6\ntaYyA5lcNebenRv67bo/fH2Yl8bfy7YpUyj8weWsfX0ivd9dTNFTk9jEJACq0v1sGdqLSGE+LhJh\n8OU3UdB/SJPXKiLSGA22qACYWRD4JvBdYDjwtnPuyhjXtleH2qKy6KNX6Pf295lz2nNM/vvNRAxu\n+8cCFvbrD8D83mkMXFrNy8OM8z6O/mz6L1rYqM9eUbaCilAFA9sOZO22tYyd/gfuGn4PWcEspq2b\nRro/nSHtUut/+M45iEQa/Rf6htcmsPbuu8nYGm3A254OZVnQsXTX4yIGGw7vSGZZJW1W7rpzRccA\n7qyRjLruftIyspD9K9lRwsfTXqJ04WyqykrI/XAe3dfU0Koyuj8CVGX62dg9F7p0JKN9R0KREEde\nfgu52flUbCkmo20h6+Z8Svm6lfgzMvHltCZgAaq2lREqLye/qAfL33yJ9PYdGHzJ9WS2bbpWJGkc\ntahIsmpUx7RzrsbM3iB6t08m8F9AXILKoYpEvLt+/AFGfxoNIusr1tftb1tcDcCwBTsD3NTVUzm5\n88kNfvaVT55NRg3847r3+OV9p/HTlyJceeEbbE83vvdemPI2aQx5IXW6kcrXrWT6NRdT8FUJaTWO\n1acfQYcTRpCZk0+v087D0tJ2OX7R//0Nd89YMoAvDguSfd1VnDbyCjIDmbw78Y+UPjWO7YEwHW66\nmfK/P8nh09bVvXf2ie2pOrIfo39wL/1bFTTzmSa3/Mx8zh5xFYzwNtwKy4oXs3L5HMIuTPETj9N9\n+hqy126hYGEJMB+Atc++z1qDQL1xMPWX+gwRnRnST3Ttjdo5DZY8Np7ivh1wndqR1rUrVZUVOBfh\n+OvupnXeznuvwuEQCz9/i9INK8nMK6Bdt/7k5rRj67Zi8vKKyM7adyuWiLQcjRmjUtuSMgJ4H3gB\neCte3T+H2qIy//0XGfj+lcz/1gR8l14LwFXX+/n7w/ueWOu7t/r59JLPdhnguDvnHIv6DwBg+WO3\n0OPq3+31uM6zptE6I+eg608GzjlmPfUQGQ/8HYCNudCubM/jNl18Kj1PG0N2+04svvpycldvYXWB\nwf/+jFEnXbbX8R71v2POqs9YvXQW/QafQq+2fWJ1OgLURGoIWIB5c6YQtgilC2ZTPmkSNVlphNu1\nJWfRakInHU3B0GHRieq2lLOdarLTW+Pycti8Zhm5BUX4fH7Wvfgs+QtW067E7TLYtzzL2HRiX2q2\nbyNveQn5G7YTrLc/YoCLzlK5Kc9H+fmn0GPkOfQ/chQ+n+Y8aohaVCRZNSaoPEt0bMobiTCg9lCD\nyrwpz3L41GtYcM5r2CVXN+o987vC3ZcE+M+F/6F4RzHbarYxuHDwLsdMWDKBfufcBsBr3+nG2S98\nvdfPsvF/o9/hDbfOJJtwRQUzf38H1Z/NwKqqyVsVTSbLbz6PM6/8NetLVrFp+XyWvTMR3ycz6bu4\nYo/P+M8p7Tj+57/liG7HNnf5EgclWzeyuWw9rYOt2Lh4FmvG/p5Oi0sI+yDiN9YNbE/20UNp3aEz\nLhyhYuliQi5EqKyM1tMXk18SnWpged9civ7nFoZ844I4n1FiU1CRZNWYtX4uqv/azIYDFznnro1Z\nVTHk6s2jsq+ItqF7Lu1X7GwCGLgSXrgvRPEZa7jh6e+QXel4/s75dfs3bd/Eu3/9Zd24xVMm7hpS\ner7+GovmfUDaz3/L/MUfJm1QiUQi+Hw+Pn3mIba+NIHDfvM7uvU/lpLpnzD/np9TsGRT3bFzj2hF\nr7vu41sDo3fXFLXtRlHbbgwe+i1qIjWs2bKKGZMeI+2FfxOOhKkY3IurHkzalRnkIOS3bkd+62iH\nUIf2PRl08nl1d0g1xDnH+kUzmfPsIxRN+IT0H93Bq/0epKp9Hm2OPYFRV9wV4+pFpLk0aoyKmR0J\nXAx8G1gOvBzLomIpEonel+n3B9hX31VlbgawZ19F2Zrl/PbJaFv0W5e+xendo3ed/GbSjVz7+s77\nPbOqwQUD9Jr0Ci5UQ3qvXuSWrWYH0P7+f8H5tzflKcWEcw4zIxKJsHTuVFbfeCMd10Ub1HK9x/Yx\nl1E7zLgA+PywAEc98hTO4JwOA8gMZO71s4O+IN3ze9L98vvh8vub43QkSTR22Qozo2P/o+l4z5Ns\nu2kjHz/wU/I+mk3ukpUEPljJa29Pod//3EHvY0+LccUiEmv7DCpmdhhwkfcoJtr9Y865kc1UW2zU\ntajsO6NV+/fe1vLx3Mmc6j3fMGk83HA65dXlnPHEvD2O7TzuH6T37FH3uqjHESwD2m6Fzds20rZV\n4i6X9PJNY8j7bAnrRwyg8zvzKChzdNztmC9PP4zWs5fTcUMN8wZk0+rmn3DuMWPISWvZ428k8bTK\na8fp948DoKZqB+/eeRVFb3xOzaXXMzPDR1nH1kQG96fghJMYdO7l+x37JCKJZ39/viwCTgHOds4N\nd879CUj6pVwjkWg7im8/M3EG6k1L7hvzzbrnaa++V/c868s1TF8/nRFPDyOtInqnUOe/Plq3v/Xh\ng3b5zLS8fMJXfAeA2VNfOoQziI3NOzYTioSYcMsF9H9jER02hxny0lwKyhxreuZQdu91tJr8HL1m\nf07/RQsZ/fAkTn5nBuVP/YZzX/iIb554mUKKxF0wPZMzH/gXXV6dyMLRR7BiQD41VTsomvgpaT//\nLVNPOZqZkx6Pd5kicgD21/VzHnAh8J6Z/Rt4jugU+knNeYsSsp+7BIJu576OF36PNRPeAOCk+dEA\nszU3je3FG3jun7fxzBPRz9sy/HD6jxhBq1NG4s/L2+PWW4DDrriBJU++QPF77/CXtM1YxHHCkHP4\n6yd/YEcQHvzGb2mfvfvSeLH3ytgb6PLEW6wYmE+/WSVs7JhJYPSZbP98JieMHUf/tnuvKRBM47jj\nz2vmakUaVtitL+c98AIAoXCI+TPe4Kt/Pkb+F8sJ3PZ73vnsE9r2H0S/keeT2alznKsVkf1pzF0/\n2cBool1ApwBPAxOcc2/Fvrw9HepdPzMnjOWo2b9iycVTCZ17Yd32vp/PYPHR0QHxNmo4Oa0KKJs4\nkT6ffMySE4bVHee75Dy2zJ9F5fKvCPt2TkRWeM+vKPjOhTTk/W+eQPXWLXQphu1psC4fennTuDx/\nXgG33zuFoC+4/w9pIlU1lYy/eQxHvbVil+2tXh5HlwG680Zano0blvPZ1RfSa3F53bbSwkyyf3Q5\ngy65rkV3C+muH0lWDY5cc85VOOf+zzl3DtAZ+AL4ecwrixHnTfgWjuw6lNaXnU3exdEbnDqMOouO\nv76H3u+/RyAvj05jx9YdV3DBt+HEY2hXtjOkWKtWtD1/1wUO98WOGUSX4ujzrOqdIQXguy8X8+7s\nCXyw6gOeWfgMzjk279jM7E2zaeo1ICu2l/HCT87iqLdWsLldBhu/F707p6RtmkKKtFjt2vfgrImf\nkvvBa6z93x8x75wBhKsqSbv3L3w6bAhTfn0tsyY8TtXWLfEuVUQ8B7RkqnOuFHjMeyQl5631U1FV\nxu7Tt3W4807aXnklwaIiAIIdoqsm55x5Bmu8Y3KLepDTvVfde8qO7s0xj/yz0VPEdz3iRCpfmApA\nTV4rgqXb6Dl5MqGaKlaOHkPFz37F1zlGr3WOG2/7gKqPPmX4nBDvXXk2N4558OBPvJ6aSA0v3jKG\n496Pzux64gczceEws4rGcsQZjQtcIsnKzChq34ui826E826komobb//1NvKee5eOz0wBpjDtd39k\ne+e2pPXsyeDvXkMwM5ucvgPjXbpISkq9td1dtEWltHTtHkEFqAspe8jIgMpK/Dk5dOl1FKu8zVVD\nB+Bv06bRX9/5xFEs5T4qzj6JIbc/QGjjJtJ79iDd2z9gFeDN8HLNLR/WvW/Tfa8y57gLGVR0VKO/\nq1ZNpIaacA1ZwSzKyjfyxvUXcNyn0TlPSn98AWaGBQIc9cNbDvizRZJddnor/uuGhwldF2LJvA/5\n+vP3CD41gcCajbSbs4GNEz4BYHa/9uQOO4muI8+idd8BhMvLSeus8S0isZZyQcV5XT7bN0Sjhu/Y\nIym67IoG39frtVfxpadjZmQf1rdu++C+3zig7w8WFdHn44/wt26NBYME8vLq9nV98glW/nDPWtKu\nu5LCPz/Oq3dczhGPz9prP/qMrz+mZ2Ff8rPa7rHv/245l6GTV9T189XOqdv6ucfpP+TEA6pfpKUK\n+AP0HzyS/oNHwg/vAWD2/CksnDKBqjWr6fL+YoJPjmfdk+OpXWWqtGsbcseMoefIc8no27dFj3ER\niZdGrZ6cSA51MO1n//oVxy4dyyuFP6LPn16lzQtP0XHQcQf8OTvmzad6+VfknH12k/7PacecOVR/\nvZJWI0cQ3rKFcFkZGX36MPu4o0nfEWLzv37D8KG73mnzwbtPUXDdA8w7dyDffWD8LvtmT3uVtMt+\ntsf39J0/D18ju6tEJDoD9acfv0jpfz7At70Sh6PnlCXkb4vu39y3PQP+dyz5AxNzhXQNppVklXIt\nKuZCPNomh6p1c+kDZOQf3Eq7mYcPJPPwpu+zzhw0iMxB0TlY/K1agde03P6OX7LltrvY8O9XYeh5\nvD/9RebOncKAEWMoe/5Z2jkoen8BU7+YyBG9hrF842KWTX0N98U8jgAqLzmbyKZi8o4+jtyjjlFI\nETlAhVmFnDPqxzDqx3Xb1m5ZzWf/eZ6yTz6i3+SFrLvgImYd24Mel11D58En4ktPx5eZ2egxbCKy\np9RrURn3Cx7+6hWO+dJx+heO7h9/SOZBhpXm9smwwVBZzfjhPq58K9LwGzxr+xVw6sQPGz5QRA7a\n/PnvM/8P99Bzxjqy6y3fGjHYnh3AF46w5aTD6f+Dn1B01PBmr08tKpKsUq5FhUiI25/f+Us+LatV\nHIs5MHbskeT+e9peQ0rGn+/Ht20H22+9e499/e56oDnKE0lpAweOYOATI1hXvJyZr4+j5qvlVEWq\niGyrwIpLab1xGz3fmkPZW1exIc0o7V9EzrHHE85Mh+xMAjk5dB54HO0PG9zwl4mkkJQLKhapqXse\nAXzp6fs+OMEcc++fWd35T2z78D8EhxxBr5/eQbhkM/j8pHXuBMC6wnYsv+NW2v3ydgiFqCovpdOQ\nYQ18sog0lY4FPTjrsrv22O6cY/6891jw3N9IX7aWgmVryZ6963IaxQZze+cRykrDl9eGQN8+ZLTK\nJadNe/qcfgHpuXl7fK5IS5dyXT+fP/bfZP3hfQCqgjBk7sL9v0FEJAYqKrfy+bRJZGW0IiOYybbN\nG1g76QVaL1yNL+LILa0mu3Ln8RGD9RecyKm/Pri1itT1I8kq9VpUwjs7j2uCjVtSXkSkqWVntObk\nb3xv142nXVr3tDpUxYZVi1i/chFlJeso/uAdCgce1sxVisRfSgeVUFBzHohIYkoLpNOlx2C69PDG\nrIy5Mb4FicRJ6jUphHe2pdYEdcugiIhIIku9oBLaGVTC6QoqIiIiiSwFg0r1zqeZaXEsRERERBqS\nckHFRXaOUQlnJc+tySIiIqko5YJK/RYVl5URx0JERESkIakXVCI7g0qNun5EREQSWuoFlfDOmWnz\n8zrGsRARERFpSMoFlUi9mXgHDhkVx0pERESkISkXVHBhAHb060qb88+PczEiIiKyPykXVFwkuvLw\n1jOPx/yaR0VERCSRpV5Q8bp+fIGUWz1AREQk6aRcUCES7frxBYJxLkREREQaEtOgYmZnmtliM1tq\nZrfuZf8PzGyTmc3yHlfGsh6o36KioCIiIpLoYtb/YWZ+4BHgNGA1MN3MXnHOLdjt0Oedc9fFqo7d\nuYiCioiISLKIZYvKscBS59xXzrlq4DlgdAy/r1EcGqMiIiKSLGIZVDoBq+q9Xu1t2935ZjbHzMab\nWZcY1hPlonf9qEVFREQk8cV7MO2rQHfn3CDgbWDc3g4ys6vNbIaZzdi0adMhfaGXUxRUREREkkAs\ng8oaoH4LSWdvWx3n3GbnXO1yxo8DR+/tg5xzjznnhjrnhhYWFh5aVWpRERERSRqxDCrTgT5m1sPM\n0oALgVfqH2Bm9RfbORdYGMN6orwWFX9QCxKKiIgkupiNKHXOhczsOuBNwA886Zybb2b3ADOcc68A\n15vZuUAIKAF+EKt6AIhEanMKfrWoiIiIJLyY3vrinJsMTN5t2531nv8C+EUsa9i1oDC1axKq60dE\nRCTxxXswbbNy4RoizgB1/YiIiCSDlAoqkUi4LqgEAgoqIiIiiS6lgko4FKoLKr6gun5EREQSXcoF\nldp5VPxqUREREUl4KRVUIpGdY1QCGqMiIiKS8FIqqIRrQnhrEqpFRUREJAmkVFCJREI43fUjIiKS\nNFIqqITCNXVjVAKB9PgWIyIiIg1KqaDiwhG1qIiIiCSRlAoq4XBN3cy0waBaVERERBJdSgWVSDgM\nEW8eFU2hLyIiTJhFjAAADD5JREFUkvBSKqi4SIRQ3Vo/MV3mSERERJpASgWVSCREmGiLCn5/fIsR\nERGRBqVUUHHhMOGIETEwX0qduoiISFJKqd/WkUgIQlAdtHiXIiIiIo2QYkElgoWMGgUVERGRpJBS\nQcWFw9GgkpZSpy0iIpK0Uuo3touE8YWMUFADaUVERJJBSgWVSCSEvwZCaQoqIiIiySClgooLhwnW\noK4fERGRJJFSv7EjkTDBEITVoiIiIpIUUiqouEgYf9iIaLI3ERGRpJBiQSWCOcCfUqctIiKStFLq\nN7aLhPBFwCmoiIiIJIWU+o3tXAR/BK3zIyIikiRSK6iEw/gUVERERJJGagWVSEgtKiIiIkkkpYIK\nEa9FJaCgIiIikgxSKqhE1PUjIiKSVFIqqIRcDQEFFRERkaSRUkEl4t2ebIFAvEsRERGRRkipoBIK\nVxOIgAWC8S5FREREGiG1gkqoBlCLioiISLJIraASVlARERFJJikVVCKhakBdPyIiIskiJYOKTy0q\nIiIiSSGlgkrY6/rxpaXFuRIRERFpjJQKKhFvMK0voKAiIiKSDFIqqIRDIQB8QY1RERERSQYpFVQI\nR4OKX10/IiIiSSGlgkok4gUVdf2IiIgkhdQKKqEwAP60jDhXIiIiIo2RWkGlrutHY1RERESSQUoF\nFed1/QSC6XGuRERERBojtYJKuLbrRy0qIiIiySA1g4oG04qIiCSFmAYVMzvTzBab2VIzu3Uv+9PN\n7Hlv/zQz6x7LeqgNKkEFFRERkWQQs6BiZn7gEeCbwADgIjMbsNthVwClzrnewEPAA7GqB8BF1KIi\nIiKSTGLZonIssNQ595Vzrhp4Dhi92zGjgXHe8/HAqWZmsSpoeyQ6hb5aVERERJJDLINKJ2BVvder\nvW17PcY5FwLKgLaxKOaNsTczbOJ6QEFFREQkWSTFYFozu9rMZpjZjE2bNh3cZ1RV1T1XUBEREUkO\nsQwqa4Au9V539rbt9RgzCwC5wObdP8g595hzbqhzbmhhYeFBFVNU0LvuueZRERERSQ6xDCrTgT5m\n1sPM0oALgVd2O+YV4DLv+QXAFOeci0UxwbyCuueBgIKKiIhIMgjE6oOdcyEzuw54E/ADTzrn5pvZ\nPcAM59wrwBPAP81sKVBCNMzERKCgkFDtc3X9iIiIJIWYBRUA59xkYPJu2+6s97wS+HYsa6iVVtCh\nLqj409SiIiIikgySYjBtU8hoU6/rR2NUREREkkLqBJWcNnXPg8GMOFYiIiIijZUyQSUrM7PuuQVj\n2uMlIiIiTSRlgkrQv/NUze+PYyUiIiLSWCkTVOqzgFpUREREkkFKBhUUVERERJJCSgYV86XkaYuI\niCQd/cYWERGRhKWgIiIiIglLQUVEREQSloKKiIiIJCwFFREREUlYCioiIiKSsBRUREREJGEpqIiI\niEjCUlARERGRhKWgIiIiIglLQUVEREQSloKKiIiIJKyUWka409iHiFRWxrsMERERaaSUCio5Z54Z\n7xJERETkAKjrR0RERBKWgoqIiIgkLAUVERERSVgKKiIiIpKwFFREREQkYSmoiIiISMJSUBEREZGE\npaAiIiIiCUtBRURERBKWgoqIiIgkLAUVERERSVgKKiIiIpKwFFREREQkYZlzLt41HBAz2wR8fZBv\nLwCKm7CcZKBzTg0659RwKOfczTlX2JTFiDSHpAsqh8LMZjjnhsa7juakc04NOufUkIrnLKKuHxER\nEUlYCioiIiKSsFItqDwW7wLiQOecGnTOqSEVz1lSXEqNUREREZHkkmotKiIiIpJEUiaomNmZZrbY\nzJaa2a3xrqepmFkXM3vPzBaY2Xwzu8Hbnm9mb5vZEu+fed52M7OHvZ/DHDM7Kr5ncHDMzG9mX5jZ\na97rHmY2zTuv580szdue7r1e6u3vHs+6D5aZtTGz8Wa2yMwWmtkJKXCNb/L+nZ5nZs+aWUZLu85m\n9qSZbTSzefW2HfB1NbPLvOOXmNll8TgXkVhJiaBiZn7gEeCbwADgIjMbEN+qmkwIuNk5NwA4HrjW\nO7dbgXedc32Ad73XEP0Z9PEeVwOPNn/JTeIGYGG91w8ADznnegOlwBXe9iuAUm/7Q95xyeiPwL+d\nc/2AwUTPvcVeYzPrBFwPDHXOHQ74gQtpedf5KeDM3bYd0HU1s3zgV8BxwLHAr2rDjUhLkBJBheh/\nvEudc18556qB54DRca6pSTjn1jnnZnrPtxL9BdaJ6PmN8w4bB/yX93w08LSL+hRoY2Ydm7nsQ2Jm\nnYGzgMe91wacAoz3Dtn9fGt/DuOBU73jk4aZ5QInA08AOOeqnXNbaMHX2BMAMs0sAGQB62hh19k5\nNxUo2W3zgV7XM4C3nXMlzrlS4G32DD8iSStVgkonYFW916u9bS2K19x9JDANaO+cW+ftWg+09563\nhJ/FWOBnQMR73RbY4pwLea/rn1Pd+Xr7y7zjk0kPYBPwD6+763Ezy6YFX2Pn3Brgd8BKogGlDPic\nln2dax3odU366y2yP6kSVFo8M2sFvATc6Jwrr7/PRW/tahG3d5nZ2cBG59zn8a6lGQWAo4BHnXNH\nAhXs7A4AWtY1BvC6LkYTDWlFQDYp2ErQ0q6ryMFIlaCyBuhS73Vnb1uLYGZBoiHlGefcy97mDbXN\n/d4/N3rbk/1ncSJwrpmtINqFdwrR8RttvC4C2PWc6s7X258LbG7OgpvAamC1c26a93o80eDSUq8x\nwChguXNuk3OuBniZ6LVvyde51oFe15ZwvUX2KVWCynSgj3fHQBrRQXmvxLmmJuH1wz8BLHTO/aHe\nrleA2tH/lwGT6m2/1LuD4HigrF4zc8Jzzv3COdfZOded6HWc4py7BHgPuMA7bPfzrf05XOAdn1R/\noTrn1gOrzKyvt+lUYAEt9Bp7VgLHm1mW9+947Tm32Otcz4Fe1zeB080sz2uJOt3bJtIyOOdS4gF8\nC/gSWAbcHu96mvC8hhNtGp4DzPIe3yLaP/8usAR4B8j3jjeid0AtA+YSvasi7udxkOc+AnjNe94T\n+AxYCrwIpHvbM7zXS739PeNd90Ge6xBghnedJwJ5Lf0aA3cDi4B5wD+B9JZ2nYFniY7BqSHacnbF\nwVxX4IfeuS8FLo/3eemhR1M+NDOtiIiIJKxU6foRERGRJKSgIiIiIglLQUVEREQSloKKiIiIJCwF\nFREREUlYCirSopmZM7Pf13t9i5nd1QSfm25m75jZLDP77m77jvdW8J3lrXR8l7d9hJkNO9TvFhFJ\nJYGGDxFJalXAeWZ2n3OuuAk/90gA59yQvewbB3zHOTfbW7m7dqK2EcA24OMmrENEpEVTi4q0dCHg\nMeCm3XeYWXczm2Jmc8zsXTPrupdj8s1sonfMp2Y2yMzaAf8CjvFaTXrt9rZ2RCfxwjkXds4t8BaM\nvAa4yXvPSWZWaGYvmdl073Gi9513mdk/zewTM1tiZld52zua2VTv/fPM7KSm+zGJiCQmBRVJBY8A\nl5hZ7m7b/wSMc84NAp4BHt7Le+8GvvCOuQ142jm3EbgS+NA5N8Q5t2y39zwELDazCWb2IzPLcM6t\nAP4KPOS950OiaxQ95Jw7BjgfeLzeZwwiuo7RCcCdZlYEXAy86bXiDCY6C7GISIumrh9p8Zxz5Wb2\nNHA9sKPerhOA87zn/wQe3MvbhxMNETjnpphZWzPLaeD77jGzZ4iuuXIxcBHRbp/djQIGRJeyASDH\nWwUbYJJzbgeww8zeA44lumbVk94ilBOdcwoqItLiqUVFUsVYouuoZDfHlznnljnnHiW6mN5gM2u7\nl8N8wPFeC8sQ51wn59y22o/Y8yPdVOBkoivjPmVml8bsBEREEoSCiqQE51wJ8ALRsFLrY6IrMANc\nAny4l7d+6O3DzEYAxc658v19l5mdZTubSfoAYWALsBVoXe/Qt4Cf1Htf/YG5o80swws4I4DpZtYN\n2OCc+zvRbqKj9leHiEhLoKAiqeT3QEG91z8BLjezOcD3gRv28p67gKO9Y+4HLmvE93yf6BiVWUS7\nlC5xzoWBV4ExtYNpiXZFDfUG6i4gOti21hzgPeBT4NfOubVEA8tsM/sC+C7RMS4iIi2aVk8WSTDe\nvCvbnHO/i3ctIiLxphYVERERSVhqUREREZGEpRYVERERSVgKKiIiIpKwFFREREQkYSmoiIiISMJS\nUBEREZGEpaAiIiIiCev/AX9aqBAwnr+OAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f607e198c88>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def weighted_non_stationary(arms=10,mean_optimal=0,std_dev_optimal=1,std_dev_reward=1,init_expectation = 0,iterations=1000,epsilon=0.1, alpha=0.5):\n",
    "    #alpha - step size\n",
    "    #epsilon - Exploration coeff\n",
    "    \n",
    "    #mean_optimal -- Mean value about which the true optimal value of the arms(by average method) lies\n",
    "    #Std_dev_optimal -- Standard deviation of the optimal values of the rewards about the mean_optimal\n",
    "    \n",
    "    #Initiations , same as prev\n",
    "    no_of_actions = arms\n",
    "    optimal_action_values = list_optimal\n",
    "    expect = [init_expectation]*no_of_actions\n",
    "    act_occur = [0]*no_of_actions\n",
    "\n",
    "    def get_maxvalue_index(list_):\n",
    "        #Get list of indexes which contain maximum value in a list\n",
    "        indexes_of_max_values = np.argwhere(list_ == np.amax(list_)).flatten().tolist()\n",
    "        #return an index randomly\n",
    "        return random.choice(indexes_of_max_values)\n",
    "\n",
    "    avg_rward_list = [0]\n",
    "    avg_rward_value = 0\n",
    "\n",
    "\n",
    "    counter = 0\n",
    "    #print (optimal_action_values)\n",
    "   \n",
    "    for t in range(iterations):\n",
    "        \n",
    "        \n",
    "        counter += 1\n",
    "        #Change Reward function at every 1/5th of the experiment\n",
    "        if counter%200==0:\n",
    "            #Set seed so that for every condition and experiment the changes are same\n",
    "            #Helps in comparison\n",
    "            np.random.seed(counter)\n",
    "            changes = np.random.normal(mean_optimal,std_dev_optimal,no_of_actions)\n",
    "            optimal_action_values = np.add(optimal_action_values,changes)\n",
    "        \n",
    "       #Exploration\n",
    "        if random.random()<epsilon:\n",
    "            #act_index --> Index of arm upon which action is made\n",
    "            act_index = random.randint(0,no_of_actions-1)\n",
    "           \n",
    "        else:\n",
    "            act_index = (get_maxvalue_index(expect))\n",
    "        act_occur[act_index]+=1\n",
    "        rward_t = np.random.normal(optimal_action_values[act_index],std_dev_reward)\n",
    "        \n",
    "        #Update expectation\n",
    "        expect[act_index] = ((rward_t*alpha) + ((1-alpha)*(expect[act_index])))\n",
    "        avg_rward_value = ((avg_rward_value*(counter-1))+rward_t)/(counter)\n",
    "        avg_rward_list.append(avg_rward_value)\n",
    "        #print  (act_index)\n",
    "    #plt.plot(avg_rward_list,color=colr)\n",
    "    #print (\"Average Reward : Greedy : \\t\",avg_rward_value)\n",
    "    return avg_rward_list,avg_rward_value\n",
    "\n",
    "\n",
    "def weighted_non_stationary_experiment(arms=10,mean_optimal=0,std_dev_optimal=1,std_dev_reward=1,init_expectation = 0,iterations=1000,epsilon=0.1,alpha=0.5,no_of_exp = 500):\n",
    "    rward_list = [0]\n",
    "    rward_value = 0\n",
    "    for x in range(no_of_exp):\n",
    "        avg_lists, avg_values = weighted_non_stationary(arms,mean_optimal,std_dev_optimal,std_dev_reward,init_expectation,iterations,epsilon,alpha)\n",
    "        if x == 0:\n",
    "            rward_list = avg_lists\n",
    "            rward_value = avg_values\n",
    "        else:\n",
    "            rward_list = np.add(np.dot(x,rward_list),avg_lists)/(x+1)\n",
    "            rward_value = ((x*rward_value)+avg_values)/(x+1)\n",
    "  \n",
    "    print (\"alpha : \",alpha, \"\\t epsilon : \",epsilon,\"\\t average reward value\",rward_value)\n",
    "    return (rward_list)\n",
    "                                           \n",
    "fig, axes = plt.subplots(nrows=1, ncols=1, figsize=(6, 6))\n",
    "axes.plot(weighted_non_stationary_experiment(epsilon=0.,alpha=0.2), label=\"epsilon=0. alpha=0.2\")\n",
    "axes.plot(weighted_non_stationary_experiment(epsilon=0.1,alpha=0.2), label=\"epsilon=0.1 alpha=0.2\")\n",
    "axes.plot(weighted_non_stationary_experiment(epsilon=0.1,alpha=0.), label=\"epsilon=0.1 alpha=0\")\n",
    "axes.plot(weighted_non_stationary_experiment(epsilon=0.,alpha=0.), label=\"epsilon=0. alpha=0.\")\n",
    "#axes.plot(greedy_experiment(epsilon=0),color='b',label=\"epsilon=0\")\n",
    "axes.set_xlabel(\"No of Steps\")\n",
    "\n",
    "axes.set_ylabel(\"Average Reward\")\n",
    "axes.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"10\"></a>\n",
    "## <span style=\"color:#2595bc\">GRADIENT BANDIT ALGORITHMS</span>\n",
    "\n",
    "Uptill now we have considered methods that estimate action values and\n",
    "use those estimates to select actions. This is often a good approach, but it is not the\n",
    "only one possible. In this section we consider learning a <span style=\"color:#f31818\">numerical preference $H_t(a)$</span>\n",
    "for each action a. The larger the preference, the more often that action is taken, but\n",
    "the preference has no interpretation in terms of reward. Only the relative preference\n",
    "of one action over another is important; if we add 1000 to all the preferences there is\n",
    "no effect on the action probabilities,\n",
    "\n",
    "Long story short, we associate some numerical probability of us taking an action. Based upon what reward we get for the action, we change the probability associated with that action. Unlike. previous algorithms, we do not compute action values rather <span style=\"color:#f31818\">deal with direct probability of choosing a particular action.</span> \n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"images/prob.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <span style=\"color:#2595bc\">EXPLAINATION</span>\n",
    "\n",
    "The $\\bar R_t$ term  <span style=\"color:#f31818\">serves\n",
    "as a baseline</span> with which the reward is compared. If the reward is higher than the\n",
    "baseline, then the probability of taking $A_t$\n",
    "in the future is increased, and if the reward\n",
    "is below baseline, then probability is decreased. The non-selected actions move in\n",
    "the opposite direction.\n",
    "\n",
    "(Look up the derivation on your own)\n",
    "<a id=\"10a\"></a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "alpha :  0.1 \t average reward value 1.9473769141864785\n",
      "alpha :  0.2 \t average reward value 1.9473165592505024\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7f607e036ef0>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAdsAAAF3CAYAAADtpOGhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xl4XWW5/vHvk6FJm7ZpmqRz00EK\ntBRaSpihVJmRWTzAQUUFixxE5SgCehQOnJ+K4kEZBHqwIiqDMlmVWYYiY1MoUNpCS1vahg7plDRz\nd/L8/lgrdDd22EmzspKd+3Nd68rea9j72dmQu+9a73pfc3dEREQkOhlxFyAiIpLuFLYiIiIRU9iK\niIhETGErIiISMYWtiIhIxBS2IiIiEVPYioiIRExhKyIiEjGFrYiISMQUtiIiIhHLiruAjlRUVOSj\nR4+OuwwRkW5j7ty56929eA9fY1BWVtbdwER6ZiOuGZifSCQuPuigg9btaIe0CtvRo0dTVlYWdxki\nIt2GmX20p6+RlZV195AhQ8YXFxdvysjI6HED7jc3N1tFRcWENWvW3A2cvqN9euK/QEREpGNNLC4u\nruqJQQuQkZHhxcXFlQQt+x3vE9Wbm9lIM3vezBaY2Xtm9q0d7GNmdouZLTGzd8xsStK2C81scbhc\nGFWdIiKyxzJ6atC2CD//TjM1ypZtAviOu08ADgMuM7MJrfY5GRgXLtOBOwDMbCBwLXAocAhwrZkV\nRFiriIj0QLfeemvhqFGjJo4aNWrirbfeWrijfWbOnFmw11577ZeRkXHQ7Nmz+7TnfSILW3df7e5v\nho+3AAuB4a12OwO41wOvAQPMbChwIvCMu290903AM8BJUdUqIiI9z9q1azNvvPHGYW+88cbCsrKy\nhTfeeOOwioqKzNb7TZ48ue7hhx9eUlpaWt3e9+qUa7ZmNho4EHi91abhwMqk56vCdTtbv6PXnm5m\nZWZWVlFR0VEli4hIN3PbbbcV7r333hP22WefCWeeeeaY3e3/2GOP5U+dOrVq8ODBTcXFxU1Tp06t\neuSRR/Jb7zdlypT6SZMmNexJbZH3RjazvsDDwLfdvaqjX9/dZwAzAEpLS3v0NQMRkdh99asjmT+/\nXadad2rixFpmzly5q13Kyspyb7rppqGvvvrqoqFDhybWrl2beccddwz81a9+NaT1vqNHj65/8skn\nl5aXl2ePGDGisWX98OHDG8vLy7M7tPZQpGFrZtkEQftHd39kB7uUAyOTno8I15UD01qtfyGaKkVE\npLt76qmn+p922mmbhg4dmgAYPHhw06WXXrrx0ksv3Rh3bRBh2JqZAb8BFrr7/+5kt1nAN8zsAYLO\nUJXuvtrMngJ+nNQp6gTgmqhqFRGRDrKbFmhn2l3Ldvjw4VtffPHFfi3ry8vLex1zzDFboqglypbt\nkcAXgXfNbF647vtACYC73wk8DpwCLAFqga+E2zaa2Q3AnPC46929S/zrREREup4TTzyx6pxzztnr\nBz/4wZohQ4Y0rV27NnN3Ldszzzyz8vrrrx/e0inqxRdf7H/zzTeviqK+yMLW3f8J2G72ceCynWyb\nCcyMoDQREUkzpaWl9d/5zndWH3300ftmZGT4xIkTax9++OHluzpm8ODBTVdeeeXHBx100HiA733v\nex8PHjy4CeDcc88dddlll1VMnTq19t577x1w5ZVXlmzatCnrrLPOGjd+/Pjaf/7zn4vbUp8FeZce\nSktLXcM1ioikzszmunvpnrzG22+/vXzSpEnrO6qm7urtt98umjRp0ugdbUursZFFRNKNOyQSULcl\nQd2GWuo311O3sY66zQ3UVzZQV7UV3PnMlQfFXarsgsJWRKSN3KG+HmproaYyQe2GOmo21lO7qZGa\nTY3UVm6lpqqJ2i0JarY4tTVOTY1TW2s0NjgNDU59QwZ1DUZ9vVHXmEFdYyb1iSzqEr2oa+5FfXMv\n6ppzqPNcmskk+HPdP1y2N8gqWHtlZ/8WpC0UtiKSlhoboboaampgyxaoqnQq1zVQs6Ge2s3bQrG2\nKkFtdTM1W5qpq22moc5pbGimti6DmvpMqhuyqG7oRU2iFzWJHGqbcqhtbglACP6M9guXnetFA72p\nI5d6stlKLvX0zmggNzNB76x6CrIT5OY20Ts7Qe9eTeT2aqZ3rtM718ntbfTuY/TOyyA3L4PefbPI\n7ZtF7/7Z9O6fTV5hDrBHs+RJxBS2IhI7d6irg8rK7ZeqqvBnpbNlQyM1G+qp2byV6soENVXNn4Rp\nTZ1RXZ9FTUMW1Y1BKCa89Z83A3LDZXuZJMijht7UkUMDvWikj9WTl1lNv+x6hmRvpW9eI31ymsjL\nbSKvj9Ont5PXz+jTN4O8vhnk9c8kr18GffKzyRuQ/cnPvIE59BnQi6x+vSE3F3L7Bj+z9Oe3J9G3\nLSIdIpGATZuCZePGYNnu8UZn07qtVK7fSuWmJio3O5VbMqisyaKyrheJ5n8ZkjaJATnk4PRlK3k0\nkEcNfakmjxqGUkNfqyWv11by+iTom9tEXp9m8vKMvL5G/3wjvyCDfgVZ9BuQSZ8BvYIQHJhLn8Le\nZOf3gb59w6UQ+vSBDM1AKh1HYSsiJBJBIG7YEATk5s3blk9al0k/qyqbqd6coGZLM5VVRmV1JrUN\nu/5z0p8qBrCZfCrJp5JhVDI+fJxPJfk5DeT3aSQ/r4n8fs3k50N+QQb5AzPpX5xD3yF9ySougAED\nCDbmQ/8BkD8qeJyTA7bLuw1FYqOwFUkj7sFp1Q0b2rI4lZW7DqksS5CfUU1/qyK/eRP9misZFLYq\n85MCcyAbKciqZmD/BAMLnILCDAYOymLAkNwgKAsLWy3jgp8FBZC5q5atSDRuvfXWwptuumkowHe/\n+93Vl19++YbW+1xyySUjnn766fzs7GwfNWpUw/3337+8qKioqS3vo7AV6YKam4NW5Pr1QSC2nJJt\nOS3b0vpMPk0btEydxsadB2f/7FoKs6sotI0UNq9nXGINhVvXUMiGT5aBbKSATUErtJ+TX5RNTlE/\nrLho+6AcODB8PHr79X36qIUp3ULLFHtz585dkJGRwYEHHjjhvPPO21xcXLxdkJ544olVt91226rs\n7GwuvfTS4T/84Q+H3HHHHeVteS+FrUgnqq6GtWthzZpty+rVUF4OH38cLGvXBgHb3Lzz18nLbqAg\nu4aCzCoKMzayT/N6ChrXUdj4MYWs3y48PwnRvEayC/sHIVlcDEVF25biIVA0cft1hYXQq1fn/XJE\n9tBtt91WeMsttww2M8aPH1/32GOPLdvV/slT7AG0TLF3ySWXbDfE49lnn/3JjHWHH354zUMPPVTQ\n+rV2R2Ersoeam4MW6OrV2wJz9ertA7Vlqan51+MzM5oZ0reaYbkb2StzDUfaxxTmlVNU8xFFzWu3\na222tDh7ZWXBwMJtp2ALC2HQoCBEB42C4tJgXUtoFhUFPWBFIvbVrzJy/nw6dIq9iROpnTmTLjHF\n3j333FN0zjnntHmsfoWtyC64B63MlSthxYptS8vzlSuDEE0k/vXYgXn1DO69hSHZGzjE1jIkbxWD\nM5YzpHoJQ/xjhrCGwayluLmCzKpmyBoIxUNhaLgMGwZDDoLBg4MQTQ7PPh07XahId9cZU+xdddVV\nQzIzM/3rX/+6wlakLVrCdNmyYFm6NPi5fDl89FEQqHV12x+Tk93EyIJqSvI2cmzv1QwbuYKhdR8y\ndPMihtcvYRhBkObUNEKtBUGZHKBDh8PQ0lahOiToTSvSze2uBdqZOnKKvVtuuaXwqaeeGvDSSy99\nkNGO28I0EYGkvdraIDxbgjQ5VJctC0YXSlbUt47R/TYwKns1Jc3LKan/gJKq+ZQ0LqaEFRRTsW06\nq8GDYeTInS9DhkD2Ls9KicQqXSYiKCsryz3nnHP2eu211xa2TLHXci12Z9auXZs5ZcqUCfPmzVsA\nMHny5AlvvvnmgtbHPfTQQ/2vuuqqkS+99NL7w4YN28F5rIAmIpC0V10NH3yw/dISqGvWbL9v7+yt\njOm/kbG9VnFM7mLGbn2bMfULGMtSRrOcftXVUJsBw4fDqBIYMSJ4PPzg4OeIEcEydKg6EIl0EVFO\nsfef//mfJY2NjRmf+cxn9gaYMmVK9X333beiLfWpZSvdSnU1vPdesCxYsG356KPt9xuZX8mn+qxm\nLEsZU7uAsVXzGOMfMpalDGId1qsXjB4NY8fCpz61LVBHjQqW4cM1nJ70COnSsu0K1LKVbmfrVli4\nEObNg3ffhUWLglBdunTbPjnZTexbtJ4jei/ja8PeZJ91L7FPYj57sYTelfXQe0gQpmPGBD/HHhf+\nHBtcJ9VwfCLSSRS2ErsNG2DOnGCZOzcI1qVLg8CFIFT3LqjgoMwlfHlgGftvns3E5ncYs3UZmaub\ng+uiEyfCv0+GSZ+FAw4IWqt5efF+MBGRkMJWOlVVFbz11rZwnTMnuK4KYObsM3QL++V9xJlD3mXS\nhueYXPsy47YuJmu9w/jxwbLPBNj3bNhnH9h772BcXBGRLkxhK5HZsAFeeQWefz4I2A8+CAZ8aDFq\nUC0HFy7l63u/zsFr/spBVc/R/+MtQaejSZPglCmw/zdgwgQoLYV+u54vVERi09zc3GwZGRnp0wmo\njZqbmw3Y6bhvClvpMOXl8MILwTJ7dhCuADk5zpRPVXL88I/Zp/BtJn38BAdveILidethQybstx98\n/mAo/RkcfDDsv796+Yp0L/MrKiomFBcXV/bEwG1ubraKiop8YP7O9lHYSrvV1gat1ieegGee2Rau\n+fnO0ftt5Ktj3uaw9X/n0LdnkLugOtg4dCic+Gk45L+C1uqBB2o0JJFuLpFIXLxmzZq716xZMxHo\niT0Pm4H5iUTi4p3toFt/JGXuQY/gZ58NAvaFF6ChAfr0caYdWMWxA99i2ur7mfTO78lsDIddmjAB\nTjkFjj02CNZBgzQjjEgX0hG3/sjuqWUru+QOb78NDzwAf/rTts5M+46u4z8mv8XJtQ8zdclMcl7e\nHGzYf3+47OswbRoccUQwlq+ISA+nsJUdWrYM7r0X7rsvOD2cmekcd8A6vn/Ukxy76HbGLJ8DKzJg\nyhT42pfg8MODcC0pibt0EZEuR2Ern6iqgocegt/9LujgZOYcs1c539nvEc5efCNFb30M/fvDSSfB\nqZfDySer5SoikgKFbQ/X1AT/+EcQsI8+GsxwM27gev6n6A98Yf3NjFq8AvbaCy47F049FY46Sj2F\nRUTaSGHbQy1YEJwm/sMfglt2BuQ1cmHR01y48v9x6KY3sOOPg2Mvg9NPDwaPUKcmEZF2U9j2IM3N\n8OST8JOfwD//GVyHPaVkPr/MvZHTav5MTsEguPYiuOhPwfRwIiLSISILWzObCZwKrHP3iTvYfiVw\nQVId44Fid99oZsuBLUATkFC39D1TVwf33AO33BKMOzyiqJ5f7Pt7vrDoBwxatRnOOw8ufibo4KSZ\nbkREOlyUf1nvAW4D7t3RRnf/OfBzADM7DbjC3Tcm7fJpd+/xUzbtiUQCfvtbuO66YJjEA0dv4o9j\nfsE5y35Gr6xC+O9vwPTpwUD+IiISmcjC1t1nm9noFHc/H7g/qlp6mkQiuCf2uutg8WI4YuQK/jjg\nW0xb/ljQ2en/fg1f/CLk5MRdqohIjxD7sFpm1gc4CXg4abUDT5vZXDObHk9l3U8iAbfdFuTpBRdA\nbnUFj/X7Iv9cOYppRzfB3/8enEe++GIFrYhIJ4o9bIHTgJdbnUI+yt2nACcDl5nZ1J0dbGbTzazM\nzMoqKiqirrXLevLJYDTEyy+Hkj4VPFJ8CfNWD+aMQ9dgb7wBs2YFwyZmZsZdqohIj9MVwvY8Wp1C\ndvfy8Oc64FHgkJ0d7O4z3L3U3UuLi4sjLbQrWrUKzjorGF+irqqRhyffwIsLB3HW8DfIePaZYIaA\ngw+Ou0wRkR4t1rA1s3zgGOAvSevyzKxfy2PgBHYxbVFP1dQUnDKeMAGeesr56VF/Y0H5AM7+8OfY\nzTcHs7Ife2zcZYqICNHe+nM/MA0oMrNVwLVANoC73xnudhbwtLvXJB06GHjUgkEUsoD73P3JqOrs\njj76KLhb57XX4IQJK7ljzdmMfXkuXHQR/M//wODBcZcoIiJJouyNfH4K+9xDcItQ8rqlwKRoqur+\nZs2Cr3wFEg0J/jj8+5y/4OfY0UfDL8uCSQFERKTL0QgG3URtLXzve3D77XBg8UoerDmWcYMSwYDG\nZ5yh4RRFRLqwrtBBSnZjxQo49NAgaK/In8mrFXsx7psnw7vvwplnKmhFRLo4tWy7uA8+gOOOc6oq\nGngq4yxOGLAQZj0DU3d6N5SIiHQxCtsubN48OOG4ZmxLJS80fprJZ38K7n4LCgriLk1ERNpAp5G7\nqJdfhmlHJ8jdtJqXmMrku/4jmNldQSsi0u2oZdsFPf00nHVaghGNS3mm5CJKZv0BJqmDtohId6WW\nbRfz8MNw6ilNjGucz+yjvk/JW39R0IqIdHMK2y7kt3c18m+fb6a06XWeP+tWBv/jPhg4MO6yRERk\nDylsuwB3+H/f28xXv96LY/1ZnrnmeQr+PAN69Yq7NBER6QC6ZhuzrVvhC2dU8acnBvCFrAe4+6EB\n5Jzxg7jLEhGRDqSwjZE7fPOrW/jTE/35Sd4NXPX657D9JsRdloiIdDCFbYxuvamBO//Qj6tybubq\n188GBa2ISFrSNduYPPFoPVd8L4szeZQfPzIe9tsv7pJERCQiCtsYvDdvK+f+WzOTeJs//DZBxikn\nxV2SiIhESGHbyTZuhNM+vYW8RCWzflhG3pc/H3dJIiISMYVtJ2pqgvNP3UL55jwePeVuRlw/Pe6S\nRESkEyhsO9EPr97K06/247Z+3+ewe/8j7nJERKSTKGw7yd/+Bj+5KZuvMYOvzToNCgvjLklERDqJ\nwrYTbN4M07/SyP68y60XzoVp0+IuSUREOpHus+0EV13RyNr1mcwafA05N/8+7nJERKSTqWUbsZdf\nhhn39OIK+xWlj3xf89GKiPRAatlGKJGAS79UTQkb+O+r6+CII+IuSUREYqCwjdCdv27m3aV9ebj4\nO+T98JdxlyMiIjFR2EakogJ+eM1WjmM2Z912HPTuHXdJIiISE12zjcj1369nS20mv5pyL/b5c+Iu\nR0REYqSwjcCmTfCbezL4kv2BCb/5DpjFXZKIiMRIYRuBO39WSV2iF9/+fDlMnhx3OSIiEjOFbQdb\nuxZ+/IscTuVvHHDjBXGXIyIiXYDCtoPd+b+1VG/N5abTZ8Po0XGXIyIiXUBkYWtmM81snZnN38n2\naWZWaWbzwuVHSdtOMrP3zWyJmV0dVY0drbER7vx1EyfzOPv8+MK4yxERkS4iypbtPcDuZkV/yd0n\nh8v1AGaWCdwOnAxMAM43swkR1tlhHv5jPWuq+3H5oXNgv/3iLkdERLqIyMLW3WcDG9tx6CHAEndf\n6u6NwAPAGR1aXERuuWEz4/iAE39+XNyliIhIFxL3NdvDzextM3vCzFqagsOBlUn7rArXdWllrzfx\n2rIhXDbyr2QcpWEZRURkmzjD9k1glLtPAm4FHmvPi5jZdDMrM7OyioqKDi2wLe68ejl5VPPl68fq\nvloREdlObGHr7lXuXh0+fhzINrMioBwYmbTriHDdzl5nhruXuntpcXFxpDXvTGN9Mw/PLuLsAc+T\n/6VuccZbREQ6UWxha2ZDzIImoJkdEtayAZgDjDOzMWbWCzgPmBVXnal4+hfvsLk5n3Mv6gsZcZ+Z\nFxGRriayiQjM7H5gGlBkZquAa4FsAHe/EzgHuNTMEkAdcJ67O5Aws28ATwGZwEx3fy+qOjvCgzOq\nGGCbOf5Hh8ddioiIdEEW5Ft6KC0t9bKysk59z/plqxk0No9zJi5i5ruHdOp7i4jsKTOb6+6lcdeR\n7nTOcw898V8vs4X+nHtFl+8wLSIiMVHY7ommJh58rBeF2ZV85osKWxER2TGF7R6o+fPj/LX2WD43\nbQPZ2XFXIyIiXZXCdg/8/YY3qSWPc68cFXcpIiLShSls2+vdd3lwwUQG963hmM9kxl2NiIh0YQrb\ndtpy1308zimcc24mmcpaERHZhcjus01rzc3M+kMV9fTmXM2kJyIiu6GWbXu89RYPVp7I8IIajjwy\n7mJERKSrU9i2Q81jz/A0J/C5z5lGZxQRkd3SaeR2+Mf962ggl9POjbsSERHpDtQua6slS/j7h/vQ\nN6eRqVPjLkZERLoDhW0bNT/5NH/lNE76dCO9esVdjYiIdAc6jdxGr/1pBasZxtlfTJ8JHEREJFpq\n2bZFIsEjrw+nV8ZWPnuqxV2NiIh0EwrbNvDX3+CRxs9y3KT19O8fdzUiItJdKGzbYOmfyljGWE49\nv1/cpYiISDeisG2D2Y9XAzD1lL4xVyIiIt2JwjZVlZW89OFQBvauZfz4uIsREZHuRGGbqhde4BU/\nnCMPrNWoUSIi0iaKjRStf7KM99mXI08ZEHcpIiLSzShsU/TqP5sAOGKqbk0WEZG2Udimwp1XFheT\nnZGgtDTuYkREpLtR2KZixQpeaZjClJIN9O4ddzEiItLdKGxT0Dz3LeZyEAeXaohGERFpO12ATMHi\n51ZSQ18OOi477lJERKQbUss2BW++1gjAlMNzYq5ERES6I4VtCt78oC85GY0azEJERNpFYbs7Gzfy\n5pa9mDR8Pdk6iywiIu2gsN0Nn/c2bzKFKQc0xV2KiIh0U5GFrZnNNLN1ZjZ/J9svMLN3zOxdM3vF\nzCYlbVserp9nZmVR1ZiKVS9+yGYKOODo/DjLEBGRbizKlu09wEm72L4MOMbd9wduAGa02v5pd5/s\n7rEOI7HojSoAxh+qCWxFRKR9Irv1x91nm9noXWx/Jenpa8CIqGrZEwsXBj/VOUpERNqrq1yzvQh4\nIum5A0+b2Vwzmx5TTdDczKLyfgzIqWXQoNiqEBGRbi72QS3M7NMEYXtU0uqj3L3czAYBz5jZInef\nvZPjpwPTAUpKSjq2uBUrWJT4FPuO3IJZn459bRER6TFibdma2QHA3cAZ7r6hZb27l4c/1wGPAofs\n7DXcfYa7l7p7aXFxcccWOH8+CxmvU8giIrJHdtqyNbNbCU7n7pC7f3NP3tjMSoBHgC+6+wdJ6/OA\nDHffEj4+Abh+T96rvTbP/ZA1nMq+h9TG8fYiIpImdnUaueWWmyOBCcCD4fPPAwt298Jmdj8wDSgy\ns1XAtUA2gLvfCfwIKAR+bWYAibDn8WDg0XBdFnCfuz/Zpk/VQd6fE/RE3vdAnUIWEZH222nYuvvv\nAMzsUoJrqInw+Z3AS7t7YXc/fzfbLwYu3sH6pcCkfz2i8y1cEDTs99035kJERKRbS+WabQGQfJNp\n33BdenNnUXlfsjMSjB0bdzEiItKdpdIb+afAW2b2PGDAVOC6KIvqEsrLWdQ4lnFDqsjKGhh3NSIi\n0o3tMmwtuHD6LME9sIeGq69y9zVRFxa7ZctYxL5MGLs17kpERKSb22XYurub2ePhkIp/6aSaugRf\nuYoVHMRnx9XFXYqIiHRzqVyzfdPMDo68ki5m0+L11NGHEfvkxV2KiIh0c6lcsz0UuMDMPgJqCK7b\nursfEGllMVv1fg0AI/bKjbkSERHp7lIJ2xMjr6ILWrU8AcCILjk9goiIdCe7DVt3/wggHKe4xzTz\nVn0cnGFX2IqIyJ7a7TVbMzvdzBYTzD/7IrCc7WfoSUur1ueSYc0MHRp3JSIi0t2l0kHqBuAw4AN3\nHwMcSzD/bPpqbGRVTQFD+1aTFfu8SCIi0t2lErZbwxl5Mswsw92fB0ojrite5eWsYjgjiuvjrkRE\nRNJAKu22zWbWF5gN/NHM1hH0Sk5fS5eyghL21/VaERHpAKm0bM8AaoErgCeBD4HToiwqbs1LlrKc\n0YwZnxN3KSIikgZSadmeB8x298XA7yKup0tY824FDeQyZv9ecZciIiJpIJWwLQHuMrMxBHPczgZe\ncvd5kVYWo2ULg2u1o8em0vAXERHZtd2mibtf6+6fIZhA/iXgSmBu1IXFaVl5NgBjxsRciIiIpIXd\ntmzN7L+AIwnmsX0L+C4pTB7fnS1bH0zfO3p0vHWIiEh6SOU08tlAAvg7waAWr7p7Q6RVxWzZliKG\n9qkkNzc/7lJERCQNpHIaeQpwHPAGcDzwrpn9M+rCYlNXx7LGYYwpqoq7EhERSROpnEaeCBwNHEMw\nmMVK0vk08rJlLGMMR41MxF2JiIikiVROI/+UIFxvAea4+9ZoS4rX1sXLWcmJjBm3Ou5SREQkTaQy\n68+pZtYbKEn3oAVYubCaZjIZM6HHTHAkIiIRS2XWn9OAeQSjR2Fmk81sVtSFxWX5kuD08ZiJfWOu\nRERE0kUqozZcBxwCbAYIB7NI2ztQV5c3AzBsrFq2IiLSMVKd9aey1TqPopiuYENFELZFRTEXIiIi\naSOVsH3PzP4dyDSzcWZ2K/BKxHXFZv3GDIxmBgyIuxIREUkXqYTt5cB+QANwH1AFfDvKouK0oSqb\ngdnVZGbGXYmIiKSLVHoj1wI/CBcAzKwEWBFhXbFZX9Obwj61QP+4SxERkTSxy5atmR1uZueY2aDw\n+QFmdh/wcqdU19nc2dCQR1Hf+rgrERGRNLLTsDWznwMzgc8Bfzez/wGeBl4HxqXy4mY208zWmdn8\nnWw3M7vFzJaY2TtmNiVp24VmtjhcLmzLh2q3zZtZ74UUDmjqlLcTEZGeYVenkT8LHOju9WZWQDBM\n40R3X96G178HuA24dyfbTyYI7nHAocAdwKFmNhC4lmB4SAfmmtksd9/Uhvduu3Xr2EAhUwobI30b\nERHpWXZ1Grne3esBwpBb3Magxd1nAxt3scsZwL0eeA0YYGZDgROBZ9x9Y/jezwAnteW926WigvUU\nUTRYvaNERKTj7KplO7bVSFFjkp+7++kd8P7DCVrMLVaF63a2PlK1KzdQT28Kh/aK+q1ERKQH2VXY\nntHq+S+iLKS9zGw6MB2gpKRkj15r/bItABSV9NnjukRERFrsNGzd/cVOeP9yYGTS8xHhunJgWqv1\nL+zoBdx9BjADoLS0dI9GttqwshaAwpK8PXkZERGR7aQyqEWUZgFfCnslHwZUuvtq4CngBDMrCDtn\nnRCui1T5ymCoxqEl2VG/lYii9+2iAAAVc0lEQVSI9CCpzGfbbmZ2P0ELtcjMVhH0MM4GcPc7gceB\nU4AlQC3wlXDbRjO7AZgTvtT17r6rjlYdYuXq4NcxYkTU7yQiIj1JymFrZn3C0aRS5u7n72a7A5ft\nZNtMgvt8O83CtQPJtq0MGaKWrYiIdJxU5rM9wswWAIvC55PM7NeRVxaDZysmcdygdzQusoiIdKhU\nrtneTHDf6wYAd38bmBplUXFZt3UAowduibsMERFJMyl1kHL3la1Wpd14hk0JZ6MXUDww7T6aiIjE\nLJWwXWlmRwBuZtlm9l1gYcR1dbqNH23ByaCo2OIuRURE0kwqYft1gk5Mwwnuf53MTjo1dWcVSyoB\nKB6iC7YiItKxUpnPdj1wQSfUEqv1y6sBKBqmoRpFRKRj7TZszeyWHayuBMrc/S8dX1I8KlYGc9gW\nl/SOuRIREUk3qZxGziU4dbw4XA4gGD7xIjP7ZYS1dar1HwfT6hWN6RdzJSIikm5SGdTiAOBId28C\nMLM7gJeAo4B3I6ytU61fF/RCLvpUfsyViIhIukmlZVsA9E16ngcMDMO3IZKqYlCxIZN+VJEzSGEr\nIiIdK5WW7c+AeWb2AmAEA1r82MzygGcjrK3TrFoFs5cOp9jWQ/bYuMsREZE0k0pv5N+Y2ePAIeGq\n77v7x+HjKyOrrBONHAkwgkOy34y7FBERSUOpTrFXD6wGNgF7mVlaDteYn10XdwkiIpKGUrn152Lg\nWwQ9kOcBhwGvAp+JtrTOl5ezNe4SREQkDaXSsv0WcDDwkbt/GjgQ2BxpVTHJy9G4yCIi0vFSCdt6\nd68HMLMcd18E7BNtWfHI662wFRGRjpdKb+RVZjYAeAx4xsw2AR9FW1Y8+vbxuEsQEZE0lEpv5LPC\nh9eZ2fNAPvBkpFXFpE9fzfgjIiIdb5dha2aZwHvuvi+Au7/YKVXFJLt3dtwliIhIGtrlNdtwlKj3\nzaykk+qJVe/+ClsREel4qVyzLQDeM7M3gJqWle5+emRVxSRvcN/d7yQiItJGqYTtDyOvoovoM7R/\n3CWIiEgaSqWD1ItmNgoY5+7PmlkfIDP60jpf3tjBcZcgIiJpaLf32ZrZ14CHgLvCVcMJbgNKG30y\ng4njjzw+L+ZKREQkHaUyqMVlwJFAFYC7LwYGRVlUZzthyDsckLOIYcPirkRERNJRKmHb4O6NLU/M\nLAtIq9Ef3HV/rYiIRCeVsH3RzL4P9Daz44E/A3+NtqzOZ+n17wcREelCUgnbq4EK4F3gEuBx4L+i\nLKqzBTGr1q2IiEQjlVt/zgTudff/i7qYuLibWrYiIhKZVFq2pwEfmNnvzezU8JptSszsJDN738yW\nmNnVO9h+s5nNC5cPzGxz0rampG2zUn3P9nHMFLYiIhKNVO6z/YqZZQMnA+cDt5vZM+5+8a6OC8dV\nvh04HlgFzDGzWe6+IOm1r0ja/3KCuXJb1Ln75DZ9mnZSBykREYlSKi1b3H0r8ATwADCX4NTy7hwC\nLHH3pWFv5geAM3ax//nA/anUEwWdRhYRkaikMqjFyWZ2D7AY+BxwNzAkhdceDqxMer4qXLej9xgF\njAGeS1qda2ZlZvaamaUS7u3mDuogJSIiUUnl+uuXgAeBS9y9IaI6zgMeCmcZajHK3cvNbCzwnJm9\n6+4ftj7QzKYD0wFKSto/OZGu2YqISFR227J19/Pd/bGWoDWzo8zs9hReuxwYmfR8RLhuR86j1Slk\ndy8Pfy4FXmD767nJ+81w91J3Ly0uLk6hrB28RruOEhERSU1K12zN7EAz+7mZLQduABalcNgcYJyZ\njTGzXgSB+i+9is1sX4Jp/F5NWldgZjnh4yKC4SIXtD62ozi69UdERKKz09PIZrY3Qael84H1BKeS\nzd0/ncoLu3vCzL4BPEUwS9BMd3/PzK4Hyty9JXjPAx5w9+S0Gw/cZWbNBP8g+GlyL+YO57piKyIi\n0dnVNdtFwEvAqe6+BMDMrtjF/v/C3R8nGHEqed2PWj2/bgfHvQLs35b32hNq04qISJR2dRr5bGA1\n8LyZ/Z+ZHUsaNwDVQUpERKKy07ANO0WdB+wLPA98GxhkZneY2QmdVWBn0KAWIiISpVR6I9e4+33u\nfhpBj+K3gKsir6xTefo22UVEJHYp9UZu4e6bwlttjo2qoDgoakVEJEptCtt05a5rtiIiEh2FbUj3\n2YqISFQUtqiDlIiIREthG1LciohIVBS2hINaKG1FRCQiCtuQrtmKiEhUFLZoPlsREYmWwpZw1h/d\n+iMiIhFR2EI464/CVkREoqGwRbP+iIhItBS2IV2xFRGRqChs0aAWIiISLYVtSB2kREQkKgpbdM1W\nRESipbAlOI2sE8kiIhIVhW1Ip5FFRCQqClt0GllERKKlsA3pNLKIiERFYYtu/RERkWgpbEO6Zisi\nIlFR2NJyzVatWxERiYbClnDWH3WTEhGRiChsIZj1R6eRRUQkIgpbdBpZRESipbAN6TSyiIhEJdKw\nNbOTzOx9M1tiZlfvYPuXzazCzOaFy8VJ2y40s8XhcmGUdbqbGrYiIhKZrKhe2MwygduB44FVwBwz\nm+XuC1rt+qC7f6PVsQOBa4FSgrO8c8NjN0VWb1QvLCIiPV6ULdtDgCXuvtTdG4EHgDNSPPZE4Bl3\n3xgG7DPASRHVqRPIIiISqSjDdjiwMun5qnBda58zs3fM7CEzG9nGYzuEY+qNLCIikYm7g9RfgdHu\nfgBB6/V3bX0BM5tuZmVmVlZRUdG+KlynkUVEJDpRhm05MDLp+Yhw3SfcfYO7N4RP7wYOSvXYpNeY\n4e6l7l5aXFzcrkLVphURkShFGbZzgHFmNsbMegHnAbOSdzCzoUlPTwcWho+fAk4wswIzKwBOCNdF\nRqeRRUQkKpH1Rnb3hJl9gyAkM4GZ7v6emV0PlLn7LOCbZnY6kAA2Al8Oj91oZjcQBDbA9e6+MbJa\ndRJZREQiFFnYArj748Djrdb9KOnxNcA1Ozl2JjAzyvq2vRmY8lZERCISdwepLkEtWxERiZLClqCD\nlOJWRESiorANqYOUiIhERWFLODayiIhIRBS2IcWtiIhERWFLOKiF0lZERCKisA1pPlsREYmKwpaW\na7Zq2oqISDQUtmjWHxERiZbCFtCwFiIiEiWFLYpaERGJlsI2pNPIIiISFYUt6iAlIiLRUtiG1LIV\nEZGoKGxBd9iKiEikFLaEt/7EXYSIiKQthW1Ik8eLiEhUFLZo1h8REYmWwjakDlIiIhIVhS3qICUi\nItFS2IZ0IllERKKisCUcrlFpKyIiEVHYEnSQUtaKiEhUFLYhdZASEZGoKGxp6SCltq2IiERDYRsy\n9UkWEZGIKGxRBykREYmWwjakrBURkagobNFwjSIiEq1Iw9bMTjKz981siZldvYPt/2lmC8zsHTP7\nh5mNStrWZGbzwmVWlHU6pokIREQkMllRvbCZZQK3A8cDq4A5ZjbL3Rck7fYWUOrutWZ2KfAz4Nxw\nW527T46qvu25OkiJiEhkomzZHgIscfel7t4IPACckbyDuz/v7rXh09eAERHWs1PqICUiIlGKMmyH\nAyuTnq8K1+3MRcATSc9zzazMzF4zszOjKDCZTiOLiEhUIjuN3BZm9gWgFDgmafUody83s7HAc2b2\nrrt/uINjpwPTAUpKStr1/q5mrYiIRCjKlm05MDLp+Yhw3XbM7DjgB8Dp7t7Qst7dy8OfS4EXgAN3\n9CbuPsPdS929tLi4uN3FKm5FRCQqUYbtHGCcmY0xs17AecB2vYrN7EDgLoKgXZe0vsDMcsLHRcCR\nQHLHqg6lW39ERCRKkZ1GdveEmX0DeArIBGa6+3tmdj1Q5u6zgJ8DfYE/W3DRdIW7nw6MB+4ys2aC\nfxD8tFUv5o6tFV2zFRGR6ER6zdbdHwceb7XuR0mPj9vJca8A+0dZW2ua9UdERKKiEaRQBykREYmW\nwjak08giIhIVhS0tLVulrYiIRENhG9JwjSIiEhWFLeGtP2rYiohIRBS2hLf+xF2EiIikLYVtSLf+\niIhIVBS2qIOUiIhES2EbUstWRESiorBFLVsREYmWwpYgbNWyFRGRqChsQ2rXiohIVBS26D5bERGJ\nlsI2pKwVEZGoKGxBAzWKiEikFLYhzfojIiJRUdii+WxFRCRaCltabv2JuwoREUlXCtuQwlZERKKi\nsEWnkUVEJFoK25BGkBIRkagobNHYyCIiEi2FbUgtWxERiYrClnC4RrVsRUQkIgpbghGk1LIVEZGo\nKGxDateKiEhUFLaEHaR0o62IiEREYRsyTUcgIiIRUdjS0rKNuwoREUlXkYatmZ1kZu+b2RIzu3oH\n23PM7MFw++tmNjpp2zXh+vfN7MQo6wzeL+p3EBGRniqysDWzTOB24GRgAnC+mU1otdtFwCZ33wu4\nGbgxPHYCcB6wH3AS8Ovw9SKh4RpFRCRKUbZsDwGWuPtSd28EHgDOaLXPGcDvwscPAceamYXrH3D3\nBndfBiwJXy8SjiluRUQkMlGG7XBgZdLzVeG6He7j7gmgEihM8dgOpdPIIiISlW7fQcrMpptZmZmV\nVVRUtOs1jh08n73Hd/tfhYiIdFFZEb52OTAy6fmIcN2O9lllZllAPrAhxWMBcPcZwAyA0tLSdt2/\nM2vNoe05TEREJCVRNufmAOPMbIyZ9SLo8DSr1T6zgAvDx+cAz7m7h+vPC3srjwHGAW9EWKuIiEhk\nImvZunvCzL4BPAVkAjPd/T0zux4oc/dZwG+A35vZEmAjQSAT7vcnYAGQAC5z96aoahUREYmSBQ3J\n9FBaWuplZWVxlyEi0m2Y2Vx3L427jnSnXkEiIiIRU9iKiIhETGErIiISMYWtiIhIxBS2IiIiEVPY\nioiIRExhKyIiEjGFrYiISMQUtiIiIhFT2IqIiEQsrYZrNLMK4KN2Hl4ErO/AcroDfeb019M+L+gz\nt9Uody/uyGLkX6VV2O4JMyvraeOD6jOnv572eUGfWbomnUYWERGJmMJWREQkYgrbbWbEXUAM9JnT\nX0/7vKDPLF2QrtmKiIhETC1bERGRiPX4sDWzk8zsfTNbYmZXx11PRzGzkWb2vJktMLP3zOxb4fqB\nZvaMmS0OfxaE683Mbgl/D++Y2ZR4P0H7mVmmmb1lZn8Ln48xs9fDz/agmfUK1+eEz5eE20fHWXd7\nmdkAM3vIzBaZ2UIzOzzdv2czuyL873q+md1vZrnp9j2b2UwzW2dm85PWtfl7NbMLw/0Xm9mFcXwW\n6eFha2aZwO3AycAE4HwzmxBvVR0mAXzH3ScAhwGXhZ/tauAf7j4O+Ef4HILfwbhwmQ7c0fkld5hv\nAQuTnt8I3OzuewGbgIvC9RcBm8L1N4f7dUe/Ap50932BSQSfPW2/ZzMbDnwTKHX3iUAmcB7p9z3f\nA5zUal2bvlczGwhcCxwKHAJc2xLQ0sncvccuwOHAU0nPrwGuibuuiD7rX4DjgfeBoeG6ocD74eO7\ngPOT9v9kv+60ACMI/gh9BvgbYAQ3+2e1/s6Bp4DDw8dZ4X4W92do4+fNB5a1rjudv2dgOLASGBh+\nb38DTkzH7xkYDcxv7/cKnA/clbR+u/20dN7So1u2bPuftsWqcF1aCU+bHQi8Dgx299XhpjXA4PBx\nuvwufgl8D2gOnxcCm909ET5P/lyffOZwe2W4f3cyBqgAfhueOr/bzPJI4+/Z3cuBm4AVwGqC720u\n6f09t2jr99rtv+900dPDNu2ZWV/gYeDb7l6VvM2Df+qmTXd0MzsVWOfuc+OupRNlAVOAO9z9QKCG\nbacWgbT8nguAMwj+oTEMyONfT7emvXT7XtNdTw/bcmBk0vMR4bq0YGbZBEH7R3d/JFy91syGhtuH\nAuvC9enwuzgSON3MlgMPEJxK/hUwwMyywn2SP9cnnzncng9s6MyCO8AqYJW7vx4+f4ggfNP5ez4O\nWObuFe6+FXiE4LtP5++5RVu/13T4vtNCTw/bOcC4sBdjL4JOFrNirqlDmJkBvwEWuvv/Jm2aBbT0\nSLyQ4Fpuy/ovhb0aDwMqk05XdQvufo27j3D30QTf5XPufgHwPHBOuFvrz9zyuzgn3L9btRTcfQ2w\n0sz2CVcdCywgjb9ngtPHh5lZn/C/85bPnLbfc5K2fq9PASeYWUF4RuCEcJ10trgvGse9AKcAHwAf\nAj+Iu54O/FxHEZxiegeYFy6nEFyr+gewGHgWGBjubwQ9sz8E3iXo6Rn759iDzz8N+Fv4eCzwBrAE\n+DOQE67PDZ8vCbePjbvudn7WyUBZ+F0/BhSk+/cM/DewCJgP/B7ISbfvGbif4Jr0VoIzGBe153sF\nvhp+9iXAV+L+XD110QhSIiIiEevpp5FFREQip7AVERGJmMJWREQkYgpbERGRiClsRUREIqawlR7J\nzNzMfpH0/Ltmdl0HvG6OmT1rZvPM7NxW2w4LZ52ZF87Oc124fpqZHbGn7y0iXVfW7ncRSUsNwNlm\n9hN3X9+Br3sggLtP3sG23wH/5u5vhzNOtQxEMQ2oBl7pwDpEpAtRy1Z6qgQwA7ii9QYzG21mz4Xz\ngv7DzEp2sM9AM3ss3Oc1MzvAzAYBfwAODluvn2p12CCCQQpw9yZ3XxBOEvF14IrwmKPNrNjMHjaz\nOeFyZPie15nZ783s1XBu0q+F64ea2ezw+PlmdnTH/ZpEpCMobKUnux24wMzyW62/Ffidux8A/BG4\nZQfH/jfwVrjP94F73X0dcDHwkrtPdvcPWx1zM/C+mT1qZpeYWa67LwfuJJiHdbK7v0QwnvPN7n4w\n8Dng7qTXOIBgzOfDgR+Z2TDg3wmmk5tMMJ/tvPb9OkQkKjqNLD2Wu1eZ2b0EE5HXJW06HDg7fPx7\n4Gc7OPwogiDE3Z8zs0Iz67+b97vezP5IMD7tvxPMNTptB7seB0wIhv0FoH84exPAX9y9Dqgzs+cJ\nJgSfA8wMJ554zN0VtiJdjFq20tP9kmDM2bzOeDN3/9Dd7yAYPH+Sme1oXtUM4LCwpTvZ3Ye7e3XL\nS/zrS/psYCrBbC73mNmXIvsAItIuClvp0dx9I/AngsBt8QrBrEEAFwAv7eDQl8JtmNk0YL23mi+4\nNTP7rG1rro4DmoDNwBagX9KuTwOXJx2X3NnqDDPLDUN6GjDHzEYBa939/whOOU/ZVR0i0vkUtiLw\nC6Ao6fnlwFfM7B3gi8C3dnDMdcBB4T4/Zdu0Z7vyRYJrtvMITk9f4O5NwF+Bs1o6SBGc1i4NO18t\nIOhA1eIdgqnkXgNucPePCUL3bTN7CziX4JqviHQhmvVHpJsI78utdveb4q5FRNpGLVsREZGIqWUr\nIiISMbVsRUREIqawFRERiZjCVkREJGIKWxERkYgpbEVERCKmsBUREYnY/we8LNZQ/V6NYQAAAABJ\nRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f607e276b70>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "\n",
    "def gradient_bandit(arms=10,std_dev_reward=1,init_pref = 0,iterations=1000,epsilon=0.,alpha=0.1):\n",
    "    #alpha - step size parameter\n",
    "    #epsilon - exploration parameter\n",
    "    #init pref -- initial numerical preference, == 0\n",
    "    \n",
    "    no_of_actions = arms\n",
    "    #optimal_action_values = np.random.normal(mean_optimal, std_dev_optimal, size=no_of_actions)\n",
    "    optimal_action_values = list_optimal\n",
    "    \n",
    "    num_preferences = [init_pref]*no_of_actions\n",
    "    \n",
    "    #Equal probabilities\n",
    "    probabilites = [1./no_of_actions]*no_of_actions\n",
    "    act_occur = [0]*no_of_actions\n",
    "\n",
    "    def get_maxvalue_index(list_):\n",
    "        #Get list of indexes which contain maximum value in a list\n",
    "        indexes_of_max_values = np.argwhere(list_ == np.amax(list_)).flatten().tolist()\n",
    "        #return an index randomly\n",
    "        return random.choice(indexes_of_max_values)\n",
    "   \n",
    "    \n",
    "    def softmax(list_a):\n",
    "        #Returns softmax probabilities of a list\n",
    "        list_b = list_a - (np.max(list_a))\n",
    "        prob = (np.exp(list_b)/np.sum(np.exp(list_b)))\n",
    "        return prob\n",
    "\n",
    "    avg_rward_list = [0]\n",
    "    avg_rward_value = 0\n",
    "\n",
    "\n",
    "    counter = 0\n",
    "    \n",
    "   \n",
    "    for t in range(iterations):\n",
    "\n",
    "        counter += 1\n",
    "       #Exploration\n",
    "        if random.random()<epsilon:\n",
    "            #act_index --> Index of arm upon which action is made\n",
    "            act_index = random.randint(0,no_of_actions-1)\n",
    "            \n",
    "        else:\n",
    "            act_index = (get_maxvalue_index(num_preferences))\n",
    "        \n",
    "        act_occur[act_index]+=1\n",
    "        rward_t = np.random.normal(optimal_action_values[act_index],std_dev_reward)\n",
    "        \n",
    "        #Update average reward\n",
    "        avg_rward_value = ((avg_rward_value*(counter-1))+rward_t)/(counter)\n",
    "        \n",
    "        #temp variable to update preference of \n",
    "        temp_variable = num_preferences[act_index] + (alpha*(rward_t-avg_rward_value)*(1-probabilites[act_index]))\n",
    "        num_preferences = num_preferences - np.dot(alpha,np.dot((rward_t-avg_rward_value),probabilites))\n",
    "        num_preferences[act_index] = temp_variable\n",
    "        probabilites = softmax(num_preferences)\n",
    "        avg_rward_list.append(avg_rward_value)\n",
    "        \n",
    "        \n",
    "        \n",
    "    \n",
    "    return avg_rward_list,avg_rward_value\n",
    "\n",
    "\n",
    "def gradient_experiment(arms=10,std_dev_reward=1,init_pref = 0,iterations=1000,epsilon=0.,alpha=0.1,no_of_exp = 500):\n",
    "    rward_list = [0]\n",
    "    rward_value = 0\n",
    "    for x in range(no_of_exp):\n",
    "        avg_lists, avg_values = gradient_bandit(arms,std_dev_reward,init_pref,iterations,epsilon, alpha)\n",
    "        if x == 0:\n",
    "            rward_list = avg_lists\n",
    "            rward_value = avg_values\n",
    "        else:\n",
    "            #Update averages\n",
    "            rward_list = np.add(np.dot(x,rward_list),avg_lists)/(x+1)\n",
    "            rward_value = ((x*rward_value)+avg_values)/(x+1)\n",
    "  \n",
    "    print (\"alpha : \",alpha,\"\\t average reward value\",rward_value)\n",
    "    return (rward_list)\n",
    "\n",
    "fig, axes = plt.subplots(nrows=1, ncols=1, figsize=(6, 6))\n",
    "axes.plot(gradient_experiment(alpha=0.1),color='r', label=\"c=0.1\")\n",
    "axes.plot(gradient_experiment(alpha=0.2),color='b',label=\"c=0.2\")\n",
    "axes.set_xlabel(\"No of Steps\")\n",
    "\n",
    "axes.set_ylabel(\"Average Reward\")\n",
    "axes.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  },
  "latex_envs": {
   "LaTeX_envs_menu_present": true,
   "autocomplete": true,
   "bibliofile": "biblio.bib",
   "cite_by": "apalike",
   "current_citInitial": 1,
   "eqLabelWithNumbers": true,
   "eqNumInitial": 1,
   "hotkeys": {
    "equation": "Ctrl-E",
    "itemize": "Ctrl-I"
   },
   "labels_anchors": false,
   "latex_user_defs": false,
   "report_style_numbering": false,
   "user_envs_cfg": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}