Stats_NLP_Project_Questions.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "6Rc_ahEnTF9m"
   },
   "source": [
    "# Predict tags on StackOverflow with linear models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "E5hmogPBTF9p"
   },
   "source": [
    "In this assignment you will learn how to predict tags for posts from [StackOverflow](https://stackoverflow.com). To solve this task you will use multilabel classification approach.\n",
    "\n",
    "### Libraries\n",
    "\n",
    "In this task you will need the following libraries:\n",
    "- [Numpy](http://www.numpy.org) — a package for scientific computing.\n",
    "- [Pandas](https://pandas.pydata.org) — a library providing high-performance, easy-to-use data structures and data analysis tools for the Python\n",
    "- [scikit-learn](http://scikit-learn.org/stable/index.html) — a tool for data mining and data analysis.\n",
    "- [NLTK](http://www.nltk.org) — a platform to work with natural language."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "zSq4Uzh6TF9q"
   },
   "source": [
    "### Data\n",
    "\n",
    "You can find all data required for this assignment into the folder `/data`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "2V1gba1KTF9r"
   },
   "source": [
    "### Text preprocessing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "WnvsWcP9TF9t"
   },
   "source": [
    "For this assignment you will need to use a list of stop words. It can be downloaded from *nltk*:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 51
    },
    "colab_type": "code",
    "id": "R-RxVRvsTF9u",
    "outputId": "bb9bb432-14c0-4fb5-878c-e02bd816423e"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package stopwords to\n",
      "[nltk_data]     C:\\Users\\IC016726\\AppData\\Roaming\\nltk_data...\n",
      "[nltk_data]   Package stopwords is already up-to-date!\n"
     ]
    }
   ],
   "source": [
    "import nltk\n",
    "nltk.download('stopwords')\n",
    "from nltk.corpus import stopwords"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ErHWjimITF91"
   },
   "source": [
    "In this task you will deal with a dataset of post titles from StackOverflow. You are provided a split to 3 sets: *train*, *validation* and *test*. All corpora (except for *test*) contain titles of the posts and corresponding tags (100 tags are available). The *test* set doesn't contain answers. Upload the corpora using *pandas* and look at the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "-ufeO-nfVGha"
   },
   "outputs": [],
   "source": [
    "from ast import literal_eval\n",
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "pDahiblaU-e7"
   },
   "source": [
    "Literal_eval package takes care of the preprocessing for the string so that it can be used in python. To know more on literal_eval please see the below documentation <br>\n",
    "https://kite.com/python/docs/ast.literal_eval"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "bubX7TSLVLXR"
   },
   "source": [
    "## Task 1: Create training, testing and validation data from the files given. Use title to be the independent variable and tags to be the dependent variable ( 5 points)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "KBFMf3bhVPb_"
   },
   "source": [
    "Note: Ensure you apply literal_eval function on the tags column to ensure all the tags are readable in python"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "x8iH7o77TF96"
   },
   "outputs": [],
   "source": [
    "train = pd.read_csv(\"train.tsv\",header=0,delimiter=\"\\t\",quoting=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "validation = pd.read_csv(\"validation.tsv\",header=0,delimiter=\"\\t\",quoting=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "HPaUfSVvTF99"
   },
   "outputs": [],
   "source": [
    "test = pd.read_csv(\"test.tsv\",header=0,delimiter=\"\\t\",quoting=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "      <th>tags</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>How to draw a stacked dotplot in R?</td>\n",
       "      <td>['r']</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>mysql select all records where a datetime fiel...</td>\n",
       "      <td>['php', 'mysql']</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                               title              tags\n",
       "0                How to draw a stacked dotplot in R?             ['r']\n",
       "1  mysql select all records where a datetime fiel...  ['php', 'mysql']"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.head(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "USlubitlTF-G"
   },
   "outputs": [],
   "source": [
    "train['tags'] = train['tags'].apply(literal_eval)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "validation['tags'] = validation['tags'].apply(literal_eval)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "      <th>tags</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>How to draw a stacked dotplot in R?</td>\n",
       "      <td>[r]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>mysql select all records where a datetime fiel...</td>\n",
       "      <td>[php, mysql]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                               title          tags\n",
       "0                How to draw a stacked dotplot in R?           [r]\n",
       "1  mysql select all records where a datetime fiel...  [php, mysql]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.head(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "      <th>tags</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Why odbc_exec always fail?</td>\n",
       "      <td>[php, sql]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Access a base classes variable from within a c...</td>\n",
       "      <td>[javascript]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                               title          tags\n",
       "0                         Why odbc_exec always fail?    [php, sql]\n",
       "1  Access a base classes variable from within a c...  [javascript]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "validation.head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Bcb3kXUSTF-M"
   },
   "source": [
    "## Task 2 (Pre-processing). Implement the function *text_prepare* following the instructions. After that, run the function *test_test_prepare* to test it on tiny cases. (10 points)\n",
    "\n",
    "One of the most known difficulties when working with natural data is that it's unstructured. For example, if you use it \"as is\" and extract tokens just by splitting the titles by whitespaces, you will see that there are many \"weird\" tokens like *3.5*, *?*,  *{}*, etc. To prevent the problems, it's usually useful to prepare the data in a custom way\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "PBSchmxtTF-N"
   },
   "outputs": [],
   "source": [
    "import re\n",
    "from nltk.tokenize import word_tokenize "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "iI1plv6WTF-R"
   },
   "outputs": [],
   "source": [
    "REPLACE_BY_SPACE_RE = re.compile('[/(){}\\[\\]\\|@,;]')\n",
    "BAD_SYMBOLS_RE = re.compile('[^0-9a-z #+_]')\n",
    "STOPWORDS = set(stopwords.words('english'))\n",
    "\n",
    "def text_prepare(text): ### The function will take in text and lower case it remove the stopwords, symbols and return it.\n",
    "    text = text.lower() ### Write a code which can change the input text to lowercase.\n",
    "    text = re.sub(REPLACE_BY_SPACE_RE, '', text) ### Write a code which replaces REPLACE_BY_SPACE_RE (above mentioned) symbols by space in text\n",
    "    text = re.sub(BAD_SYMBOLS_RE, '', text) ### Write a code which deletes symbols which are in BAD_SYMBOLS_RE (above mentioned) from text\n",
    "    ### Write a code which deletes stopwords from text\n",
    "    word_tokens = word_tokenize(text)\n",
    "    filtered_sentence = [] \n",
    "    for w in word_tokens: \n",
    "        if w not in STOPWORDS: \n",
    "            filtered_sentence.append(w)\n",
    "    text = \" \".join(filtered_sentence)\n",
    "    return text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "t5LQEgMATF-Y"
   },
   "outputs": [],
   "source": [
    "def test_text_prepare():\n",
    "    examples = [\"SQL Server - any equivalent of Excel's CHOOSE function?\",\n",
    "                \"How to free c++ memory vector<int> * arr?\"]\n",
    "    answers = [\"sql server equivalent excels choose function\", \n",
    "               \"free c++ memory vectorint arr\"]\n",
    "    for ex, ans in zip(examples, answers):\n",
    "        if text_prepare(ex) != ans:\n",
    "            return \"Wrong answer for the case: '%s'\" % ex\n",
    "    return 'Basic tests are passed.'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "6t2DpralVokG"
   },
   "source": [
    "Execute the test_text_prepare function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Basic tests are passed.'"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_text_prepare()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ettHAZ8eVt1R"
   },
   "source": [
    "*Note: You should pass the above test to ensure the text preprocessing is done before our analysis*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "QV5JC6HXTF-k"
   },
   "source": [
    "Now we can preprocess the titles using function *text_prepare* and  making sure that the headers don't have bad symbols:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_train = train[\"title\"]\n",
    "y_train = train[\"tags\"]\n",
    "x_validation = validation[\"title\"]\n",
    "y_validation = validation[\"tags\"]\n",
    "x_test = test[\"title\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_train = x_train.apply(text_prepare)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_validation = x_validation.apply(text_prepare)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_test = x_test.apply(text_prepare)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "AfWls32MV5Uu"
   },
   "source": [
    "Print the top 5 elements in x_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "3BP3LiGsTF-o"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0                               draw stacked dotplot r\n",
       "1    mysql select records datetime field less speci...\n",
       "2                       terminate windows phone 81 app\n",
       "3         get current time specific country via jquery\n",
       "4                           configuring tomcat use ssl\n",
       "Name: title, dtype: object"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0                                odbc_exec always fail\n",
       "1      access base classes variable within child class\n",
       "2           contenttype applicationjson required rails\n",
       "3                  sessions sinatra used pass variable\n",
       "4    getting error type json exist postgresql rake ...\n",
       "Name: title, dtype: object"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_validation.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    warning mysql_query expects parameter 2 resour...\n",
       "1    get click coordinates input typeimage via java...\n",
       "2              implement cloud storage media assets zf\n",
       "3              catcomplete jquerys autocomplete plugin\n",
       "4            error building android app cordova 31 cli\n",
       "Name: title, dtype: object"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_test.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "At_ZTeqTTF-t"
   },
   "source": [
    "## Task 2 (WordsTagsCount) - Find 3 most popular tags and 3 most popular words in the train data. - 5 points"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "x2mT1f7gWKSu"
   },
   "source": [
    "Note: The words which appear the most are considered as popular in this case!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "q-eQoxipTF-v"
   },
   "outputs": [],
   "source": [
    "tags=[]\n",
    "for item in train['tags']:\n",
    "    tags.append(item)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "from collections import Counter\n",
    "words=[]\n",
    "for word in tags:\n",
    "    for item in word:\n",
    "        words.append(item)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "c = Counter(words)\n",
    "tags_counts = c"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('javascript', 19078), ('c#', 19077), ('java', 18661)]"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "most_popular_tag = {} \n",
    "most_popular_tag = c.most_common()\n",
    "most_popular_tag[:3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "words_frequency = {}\n",
    "titlesWords = []\n",
    "for doc in x_train:\n",
    "    words = doc.split()\n",
    "    for word in words: \n",
    "        titlesWords.append(word)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "c = Counter(titlesWords)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('using', 8274), ('c', 5610), ('php', 5422)]"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "most_popular_Words = {} \n",
    "most_popular_Words = c.most_common()\n",
    "most_popular_Words[:3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "xkwg8_WNTF-3"
   },
   "source": [
    "We are assuming that *tags_counts* and *words_counts* are dictionaries like `{'some_word_or_tag': frequency}`. After applying the sorting procedure, results will be look like this: `[('most_popular_word_or_tag', frequency), ('less_popular_word_or_tag', frequency), ...]`.\n",
    "\n",
    "eg: \n",
    "Tag 1 - 100 Tag 2 - 65 Tag 3 - 250 <br>\n",
    "after sorting looks like, <br>\n",
    "Tag 3 - 250 Tag 1 - 100 Tag 2 - 65"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "RSCPa54uTF-4"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Xk7M4uJ_TF-8"
   },
   "source": [
    "## Task - 3 Transforming text to a vector (10 points)\n",
    "\n",
    "Machine Learning algorithms work with numeric data and we cannot use the provided text data \"as is\". There are many ways to transform text data to numeric vectors. In this task you will try to use two of them.\n",
    "\n",
    "#### Bag of words\n",
    "\n",
    "One of the well-known approaches is a *bag-of-words* representation. To create this transformation, follow the steps:\n",
    "1. Find *N* most popular words in train corpus and numerate them. Now we have a dictionary of the most popular words.\n",
    "2. For each title in the corpora create a zero vector with the dimension equals to *N*.\n",
    "3. For each text in the corpora iterate over words which are in the dictionary and increase by 1 the corresponding coordinate.\n",
    "\n",
    "Let's try to do it for a toy example. Imagine that we have *N* = 4 and the list of the most popular words is \n",
    "\n",
    "    ['hi', 'you', 'me', 'are']\n",
    "\n",
    "Then we need to numerate them, for example, like this: \n",
    "\n",
    "    {'hi': 0, 'you': 1, 'me': 2, 'are': 3}\n",
    "\n",
    "And we have the text, which we want to transform to the vector:\n",
    "\n",
    "    'hi how are you'\n",
    "\n",
    "For this text we create a corresponding zero vector \n",
    "\n",
    "    [0, 0, 0, 0]\n",
    "    \n",
    "And iterate over all words, and if the word is in the dictionary, we increase the value of the corresponding position in the vector:\n",
    "\n",
    "    'hi':  [1, 0, 0, 0]\n",
    "    'how': [1, 0, 0, 0] # word 'how' is not in our dictionary\n",
    "    'are': [1, 0, 0, 1]\n",
    "    'you': [1, 1, 0, 1]\n",
    "\n",
    "The resulting vector will be \n",
    "\n",
    "    [1, 1, 0, 1]\n",
    "   \n",
    "Implement the described encoding in the function *my_bag_of_words* with the size of the dictionary equals to 5000. To find the most common words use train data. You can test your code using the function *test_my_bag_of_words*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "6ER1oq48TF-9"
   },
   "outputs": [],
   "source": [
    "DICT_SIZE = 5000\n",
    "most_frequent_word = most_popular_Words[:DICT_SIZE]\n",
    "WORDS_TO_INDEX = {} ####### YOUR CODE HERE #######\n",
    "count = 0\n",
    "for word, frequency in most_frequent_word:\n",
    "    WORDS_TO_INDEX[word] = count\n",
    "    count += 1\n",
    "INDEX_TO_WORDS = {} ####### YOUR CODE HERE #######\n",
    "for word, index in WORDS_TO_INDEX.items():\n",
    "    INDEX_TO_WORDS[index]=word\n",
    "    \n",
    "ALL_WORDS = WORDS_TO_INDEX.keys()\n",
    "\n",
    "    \n",
    "def my_bag_of_words(text, words_to_index, dict_size):\n",
    "        \n",
    "    ### Write a function which returns a vector which is a bag-of-words representation of 'text'\n",
    "    result_vector = [0]*dict_size\n",
    "    for word in text.split():\n",
    "        if word in words_to_index:\n",
    "            result_vector[words_to_index[word]] = 1\n",
    "    return np.array(result_vector)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "qnLYWnsaTF_A"
   },
   "outputs": [],
   "source": [
    "def test_my_bag_of_words():\n",
    "    words_to_index = {'hi': 0, 'you': 1, 'me': 2, 'are': 3}\n",
    "    examples = ['hi how are you']\n",
    "    answers = [[1, 1, 0, 1]]\n",
    "    for ex, ans in zip(examples, answers):\n",
    "        if (my_bag_of_words(ex, words_to_index, 4) != ans).any():\n",
    "            return \"Wrong answer for the case: '%s'\" % ex\n",
    "    return 'Basic tests are passed.'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "mOlSiUqEW3jD"
   },
   "source": [
    "Execute the test_text_prepare function <br>\n",
    "*<u>Note:</u> You should pass the above test to ensure BOW is working correctly!*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "cqWhoF24TF_D"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Basic tests are passed.'"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_my_bag_of_words()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "BXW3ALCITF_H"
   },
   "source": [
    "Now apply the implemented function to all samples (this might take up to a minute):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "m39xHB2yTF_H"
   },
   "outputs": [],
   "source": [
    "from scipy import sparse as sp_sparse"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "6DZwPWB1TF_K"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X_train shape  (100000, 5000)\n",
      "X_val shape  (30000, 5000)\n",
      "X_test shape  (20000, 5000)\n"
     ]
    }
   ],
   "source": [
    "X_train_mybag = sp_sparse.vstack([sp_sparse.csr_matrix(my_bag_of_words(text, WORDS_TO_INDEX, DICT_SIZE)) for text in x_train])\n",
    "X_val_mybag = sp_sparse.vstack([sp_sparse.csr_matrix(my_bag_of_words(text, WORDS_TO_INDEX, DICT_SIZE)) for text in x_validation])\n",
    "X_test_mybag = sp_sparse.vstack([sp_sparse.csr_matrix(my_bag_of_words(text, WORDS_TO_INDEX, DICT_SIZE)) for text in x_test])\n",
    "print('X_train shape ', X_train_mybag.shape)\n",
    "print('X_val shape ', X_val_mybag.shape)\n",
    "print('X_test shape ', X_test_mybag.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "WPXTUeNSTF_N"
   },
   "source": [
    "As you might notice, we transform the data to sparse representation, to store the useful information efficiently. There are many types: of such representations, however sklearn algorithms can work only with  csr matrix, so we will use this one.<br>\n",
    "<u>Documentations on sparse matrix:</u> <br>\n",
    "(https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.csr_matrix.html#scipy.sparse.csr_matrix) \n",
    "(https://docs.scipy.org/doc/scipy/reference/sparse.html)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "i8AMzsVxTF_Q"
   },
   "source": [
    "\n",
    "For the 11th row in *X_train_mybag* find how many non-zero elements it has. In this task the answer (variable *non_zero_elements_count*) should be a number, e.g. 20."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "bOetzSLzTF_S"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<1x5000 sparse matrix of type '<class 'numpy.int32'>'\n",
       "\twith 6 stored elements in Compressed Sparse Row format>"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "eleventhElement = X_train_mybag[11]\n",
    "eleventhElement"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([0, 0, 0, 0, 0, 0]), array([  23,   68,   86,  796, 1227, 2823]))"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nonZeroElement = eleventhElement.nonzero()\n",
    "nonZeroElement"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nonZeroElement[1].shape[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "6 Non Zero elements"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "GYAFKGfKTF_V"
   },
   "source": [
    "## Task 4 - TF-IDF (5 points)\n",
    "\n",
    "The second approach extends the bag-of-words framework by taking into account total frequencies of words in the corpora. It helps to penalize too frequent words and provide better features space. \n",
    "\n",
    "Implement function *tfidf_features* using class  from *scikit-learn*. Use *train* corpus to train a vectorizer. Don't forget to take a look into the arguments that you can pass to it. We suggest that you filter out too rare words (occur less than in 5 titles) and too frequent words (occur more than in 90% of the titles). Also, use bigrams along with unigrams in your vocabulary. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "1s3uEKmjYpuh"
   },
   "source": [
    "## Write a function which takes x_train, x_val and x_test as input and return the tf-idf features of the same and the vocabulary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "lkz1RYsBTF_Z"
   },
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import TfidfVectorizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "o6V7FPirTF_i"
   },
   "outputs": [],
   "source": [
    "def tfidf_features(X_train, X_val, X_test):\n",
    "    tfidf_vectorizer = TfidfVectorizer(ngram_range=(1,2), max_df=0.9,min_df=5,token_pattern= '(\\S+)')\n",
    "    X_train_tfidf = tfidf_vectorizer.fit_transform(X_train)\n",
    "    X_val_tfidf = tfidf_vectorizer.transform(X_val)\n",
    "    X_test_tfidf = tfidf_vectorizer.transform(X_test)\n",
    "    return X_train_tfidf, X_val_tfidf, X_test_tfidf, tfidf_vectorizer.vocabulary_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "ySnV2zCBTF_p"
   },
   "outputs": [],
   "source": [
    "X_train_tfidf, X_val_tfidf, X_test_tfidf, tfidf_vocab = tfidf_features(x_train, x_validation, x_test)\n",
    "tfidf_reversed_vocab = {i:word for word,i in tfidf_vocab.items()}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "o_aopTTNL5G1"
   },
   "source": [
    "Print the index of string \"C#\" in the vocabulary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "kZ-3wqTTMCRi"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "index of string C# is 1914\n"
     ]
    }
   ],
   "source": [
    "print('index of string C# is',tfidf_vocab['c'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\"# is removed as part of preprocessing\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ZYYO2kdiTF_6"
   },
   "source": [
    "## Task 5: Classification (15 points)\n",
    "MultiLabel classifier\n",
    "\n",
    "As we have noticed before, in this task each example can have multiple tags. To deal with such kind of prediction, we need to transform labels in a binary form and the prediction will be a mask of 0s and 1s. For this purpose it is convenient to use MultiLabelBinarizer from sklearn. <br>\n",
    "<u>Documentation:</u> <br>\n",
    "http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.MultiLabelBinarizer.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "ojeCe_e_TF_7"
   },
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import MultiLabelBinarizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "p8JYQr8YTF_-"
   },
   "outputs": [],
   "source": [
    "mlb = MultiLabelBinarizer(classes=sorted(tags_counts.keys()))\n",
    "y_train = mlb.fit_transform(y_train)\n",
    "y_val = mlb.fit_transform(y_validation)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "qN1NUA3DTGAB"
   },
   "source": [
    "In this task we suggest to use One-vs-Rest approach, which is implemented in [OneVsRestClassifier](http://scikit-learn.org/stable/modules/generated/sklearn.multiclass.OneVsRestClassifier.html) class. In this approach *k* classifiers (= number of tags) are trained. As a basic classifier, use [LogisticRegression](http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html). It is one of the simplest methods, but often it performs good enough in text classification tasks. It might take some time, because a number of classifiers to train is large.\n",
    "\n",
    "**OneVsRest multi-label strategy**\n",
    "\n",
    "The Multi-label algorithm accepts a binary mask over multiple labels. The result for each prediction will be an array of 0s and 1s marking which class labels apply to each row input sample.\n",
    "\n",
    "**Logistic Regression & SVM**\n",
    "\n",
    "OneVsRest strategy can be used for multi-label learning, where a classifier is used to predict multiple labels for instance. LR & SVM supports multi-class, but we are in a multi-label scenario, therefore, we wrap classifiers in the OneVsRestClassifier.\n",
    "\n",
    "*If you want to learn more about OneVsRest, check out these links:*\n",
    "- *https://towardsdatascience.com/multi-label-text-classification-with-scikit-learn-30714b7819c5*\n",
    "- *https://towardsdatascience.com/journey-to-the-center-of-multi-label-classification-384c40229bff*\n",
    "- *https://medium.com/coinmonks/multi-label-classification-blog-tags-prediction-using-nlp-b0b5ee6686fc*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "2Pj6MDkbTGAC"
   },
   "outputs": [],
   "source": [
    "from sklearn.multiclass import OneVsRestClassifier\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.svm import SVC, LinearSVC"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "eUHZR2TWTGAF"
   },
   "outputs": [],
   "source": [
    "def train_classifier(X_train, y_train):\n",
    "    # Create and fit LogisticRegression and LinearSVC wraped into OneVsRestClassifier.\n",
    "\n",
    "    ######################################\n",
    "    ######### YOUR CODE HERE #############\n",
    "    ###################################### \n",
    "    clf = OneVsRestClassifier(LogisticRegression()).fit(X_train,y_train)\n",
    "    SVC = OneVsRestClassifier(LinearSVC()).fit(X_train,y_train)\n",
    "    return clf  ### clf is the model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "k660NfRFTGAL"
   },
   "source": [
    "Train the classifiers for different data transformations: *bag-of-words* and *tf-idf*.\n",
    "classifier_mybag = model for "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "jXyhN-yDTGAP"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:433: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n",
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:433: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n"
     ]
    }
   ],
   "source": [
    "classifier_mybag = train_classifier(X_train_mybag, y_train)\n",
    "classifier_tfidf = train_classifier(X_train_tfidf, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ijQZCd9cTGAU"
   },
   "source": [
    "Now you can create predictions for the data. You will need two types of predictions: labels and scores."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "0QT7uWddTGAV"
   },
   "outputs": [],
   "source": [
    "y_val_predicted_labels_mybag = classifier_mybag.predict(X_val_mybag)\n",
    "y_val_predicted_scores_mybag = classifier_mybag.decision_function(X_val_mybag)\n",
    "\n",
    "y_val_predicted_labels_tfidf = classifier_tfidf.predict(X_val_tfidf)\n",
    "y_val_predicted_scores_tfidf = classifier_tfidf.decision_function(X_val_tfidf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "p9n0TEtHTGAj"
   },
   "source": [
    "Now, we would need to compare the results of different predictions, e.g. to see whether TF-IDF transformation helps or to try different regularization techniques in logistic regression. For all these experiments, we need to setup evaluation procedure. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "RVqAxDxqTGAk"
   },
   "source": [
    "## Evaluation (10 points)\n",
    "\n",
    "To evaluate the results we will use several classification metrics:\n",
    " - [Accuracy](http://scikit-learn.org/stable/modules/generated/sklearn.metrics.accuracy_score.html)\n",
    " - [F1-score](http://scikit-learn.org/stable/modules/generated/sklearn.metrics.f1_score.html)\n",
    " - [Area under ROC-curve](http://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_auc_score.html)\n",
    " - [Area under precision-recall curve](http://scikit-learn.org/stable/modules/generated/sklearn.metrics.average_precision_score.html#sklearn.metrics.average_precision_score) \n",
    " \n",
    "Make sure you are familiar with all of them. If you want a refresher, you can click the link to their documentation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "_YmNhVsU9K9-"
   },
   "source": [
    "## Import the necessary libraries for the above metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "tBvrkOUYTGAl"
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.metrics import f1_score\n",
    "from sklearn.metrics import roc_auc_score \n",
    "from sklearn.metrics import average_precision_score\n",
    "from sklearn.metrics import recall_score"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "vNkYBl_wTGAp"
   },
   "source": [
    "Define the function *print_evaluation_scores* which takes y_val and predicted as input calculates and prints the following output:\n",
    " - *accuracy*\n",
    " - *F1-score - Average = 'weighted'* \n",
    " - *Precision - Average = 'macro'*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "-4nevWnhTGAq"
   },
   "outputs": [],
   "source": [
    "def print_evaluation_scores(y_val, predicted):\n",
    "    print(\"accuracy\", accuracy_score(y_val, predicted))\n",
    "    print(\"f1 score\", f1_score(y_val, predicted, average='weighted'))\n",
    "    print(\"average precision\", average_precision_score(y_val, predicted, average='macro'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "G92GIO4dTGAt"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Bag-of-words\n",
      "accuracy 0.3555\n",
      "f1 score 0.6447918889440271\n",
      "average precision 0.34154662087620574\n",
      "Tfidf\n",
      "accuracy 0.32986666666666664\n",
      "f1 score 0.607820453826557\n",
      "average precision 0.297127679909274\n"
     ]
    }
   ],
   "source": [
    "print('Bag-of-words')\n",
    "print_evaluation_scores(y_val, y_val_predicted_labels_mybag)\n",
    "print('Tfidf')\n",
    "print_evaluation_scores(y_val, y_val_predicted_labels_tfidf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "kT7oso8ZTGAv"
   },
   "source": [
    "You might also want to plot some form of the [ROC curve](http://scikit-learn.org/stable/modules/model_evaluation.html#receiver-operating-characteristic-roc) for the case of multi-label classification. The input parameters for the roc curve are:\n",
    " - true labels\n",
    " - decision functions scores\n",
    " - number of classes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "x4hbW3SeBVxz"
   },
   "source": [
    "Import the roc_auc function from the metrics.py file provided"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "G_9M0AI4TGAv"
   },
   "outputs": [],
   "source": [
    "from metrics import roc_auc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "sZaymN4uTGA0"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEWCAYAAAB42tAoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nOydd3hUxRbAf7PpISGh9xJ6CyDSFBAQRbELKoriQxEEREREUR6I8kQBEQUpAZ7Cs3eBh6igiIg8egvSpJMAAQIJKSTZ7M77Y26S3c1usoFsdhPm93377Z17z5177pY5d87MnCOklGg0Go1G4wqTtxXQaDQajW+jDYVGo9FoCkQbCo1Go9EUiDYUGo1GoykQbSg0Go1GUyDaUGg0Go2mQLSh0JR6hBDjhRD/9sJ17xdCnBRCpAohrivp65cGhBDHhBC3FHD8RyHEP66g3teEEJ9cnXYad9GGwocQQnQVQmwQQiQLIS4IIf4UQnTwtl5XihBiiRDiDU9fR0r5ppTyKU9fxwkzgJFSyjAp5Q7Hg0IIKYRIMwxJvBBiphDCz0HmLiHEZkMuUQjxqRCitoNMDSHEB0KI00KIFCHEfiHE60KIck6uWd+4rn+x3+1V4qxxl1L2kVL+x1s6adxDGwofQQhRHlgBvA9UBGoBrwOZ3tRLUyD1gL8KkWkjpQwDugP9gSdzDgghHgA+A2YBlYGWqO97vRCigiFTEfgfEALcIKUMB24FIoGGxXo3Go0rpJT65QMvoD2QVMBxEzABOA6cBT4CIoxj9QEJPAGcBC4Cw4AOwG4gCZjjUN+TwD5D9megXgHX7gxsMOrZBfQw9lcE4oC7jXIYcAh4HBgKmIEsIBX4ryFTE/gWOAccBUbZXOc14Cvj3lJQjXB7m+PjgHjj2AGgl815n9jI3WOcmwSsBZrbHDsGjDU+l2TgSyC4KJ85EGTckwTSgMMuzpdAI5vyV8BcY1sY9b7k5Jp7gMlG+Q0gFjC5+Ts6YVw31XjdUNBvx8n5PYzv9CVD9jRwH3AHcBC4AIy3kV8CvOF4vsPnfQtwu/FbMBt67TKOrwWeKuB+WgKrjesm5FzbyXf+NXDG+E7XAS1tjt0B7DV+N/HAWGN/ZdTDWZJR/x/ufs7X2svrCuiX8UVAeSAR+A/QB6jgcPxJVCPcANUgfwd8bByrbzQOMUAw0BvIAJYCVVG9k7NAd0P+PqOu5oC/0YhscKFXLUOvO4wG51ajXMU43tv4g1YFFgHf2Jzr2IiYgG3Aq0CgcS9HgNuM468Zet8B+AFvARuNY01RRrCmzT03tDnvE2O7CarxvhUIQDV4h4BA4/gxYDPKYFVEGcthLu7d5WduHLczBE7Ozz0ONEM1us/blCUQ5eS814H/GdsbgdeL8DvK+S34u3sfDuf3ALKN7ygAGIIy6p8B4aiGOwNo4OI77oETQ+H4PdkcX4sLQ2Fc7zTwAup3HQ50claXcY/hKCP+HrDT5thpoJuxXQFoZ2y/hfrPBBivboDwdlvgiy/tevIRpJSXgK6oP/ki4JwQYrkQopoh8igwU0p5REqZCrwCPOzgi/6XlDJDSrkK1Vh+LqU8K6WMRz0t5Qy4Pg28JaXcJ6XMBt4E2goh6jlR7TFgpZRypZTSKqVcDWxFNeYY1/oa+BW406jbFR1QBmaylDJLSnnEuNeHbWTWG9eyAB8DbYz9FlQj0EIIESClPCalPOzkGv2BH6SUq6WUZtQ4Qghwo43MbCnlKSnlBeC/QFsX+rrzmRfGdiFEGsogrQXmGfsrG++nnZxz2uZ4JRcyRaGo92EGphif3xeGLrOklClSyr9QvbXWV6mTO9wFnJFSvmP8rlOklJucCUopPzSOZ6KMSBshRIRx2Iz63ZSXUl6UUm632V8D1Zs2Syn/kIYF0dijDYUPYTTcg6SUtYFWqKfe94zDNVGugxyOo3oD1Wz2JdhsX3ZSDjO26wGzhBBJQoicbrdA9R4cqQc8mCNryHdF/cFyWGjou1hKmVjALdYDajrUNd7hHs7YbKcDwUIIfynlIWA0qhE4K4T4QghR08k17D4nKaUV1ROxvTfHa4ThHHc+88JoZ9TfH+gE5AxAnzfeazg5p4bN8UQXMkWhqPeRaBhqUL8bcP1bKjaEEH8ZA/+pQohuQB3A2cOA43l+QoipQojDQohLqF4M5BnbfqgHm+NCiN+FEDcY+99G9bRWCSGOCCFeLtYbKkNoQ+GjSCn3o7r1rYxdp1ANbQ51US6CBIrOSeBpKWWkzStESrnBhezHDrLlpJRTQf1JgQUov/dwIUQj29twUtdRh7rCpZR3uKO0lPIzKWVX1OcggWlOxOw+JyGEQDU48e5co6C6uMLPXCq+Qg1Kv2rsPoAaC3jQVlYIYUI1bL8au34B7jf2u3U5J/uK87fjSBoQalOuXoBsgU/rUsqWUs0gC5NS/oH6vbgzYD8AuBc1FhKBcr+BevhBSrlFSnkvyj26FDVWhNEDeUFK2QC4GxgjhOjlxvWuObSh8BGEEM2EEC/kTI0UQtQBHkH5qAE+B54XQkQJIcJQ7qIvDddRUYkBXhFCtDSuFSGEeNCF7CfA3UKI24wnt2AhRA+bKZzjjfcnUW6ej2ymgCag/OI5bAYuCSHGCSFCjPpauTMFWAjRVAhxsxAiCOUjv4xyRznyFXCnEKKXECIA5d/ORA3GF5Xi/MwBpgJDhRDVDRfHWGCCEGKA8XlUB/6NGq961zhnplH+T45rUAhRy5hq68z9cw6wYv+5F/d92LITuEMIUdHQf3QBsglA/SIYvRVAdSHEaCFEkBAiXAjRyYlcOOo7TkQZrTdzDgghAoUQjwohIgxX2iWM340xNbmR8TCRs9/Zb+qaRxsK3yEF5ZrYZPi0N6Jmv7xgHP8Q5bNfh5otlAE8eyUXklJ+j3oa/8Loqu9BDaA7kz2Jelobj2qETgIvAiYhxPXAGOBxw1UxDfXUmNOF/wDlG04SQiw1ZO5GjQkcRblX/o16CiyMIFRDe568wfPxjkJSygOocZX3Ddm7UbOysty4hiPF9pkbusUCv6M+P6SUXwIDgecNXfeixlO65LjwjHGUG1H+9E1CiBRUbyMZ5TZxvEY6MAX40/jcOxf3fTjwMWom3DFgFWoWmSu+Nt4ThRDbC5AD1BM/alLC3ajv/G+gpxPRj1DutHjUZ7jR4fhA4JjxWx+G+n0ANEb12FJRvb15Usq1hel1LSL02I1Go9FoCkL3KDQajUZTINpQaDQajaZAtKHQaDQaTYFoQ6HRaDSaAvG5CJOFUblyZVm/fn1vq6HRaDSlim3btp2XUla5knNLnaGoX78+W7du9bYaGo1GU6oQQhwvXMo52vWk0Wg0mgLRhkKj0Wg0BaINhUaj0WgKRBsKjUaj0RSINhQajUajKRBtKDQajUZTIB6bHiuE+BCVoeqslLKVk+MClVT+DlTymEE2mac0Go3G+2RZ1Msq1SvIH0JcNJunUiHbquInWyXUCQf//M/iMjkD6/4LmIL8EEJA+SCIygugnJmZzYULl7FYJJZDFwm6nE31iqEgJTStCBWCc2UPHbpAYmI61svZWPaco3ndSCqVD4JAP7jRWR6yK8OT6yiWAHNQIYCd0QcV5rcxKrz2fONdo9F4mrPpkJqlGsFso1GLCMovdzkb1hwHs1U1gv4muK+x8zr/iIO1J1RDabHCTXXI7FUvN1WfHW9thKQMlTnDYoVXb4TI4PxyOxLgvW1gtarGt01VeCmvmcjIyOb8+TSysqxcfmczIXvO08DfHywSpnSDLnmN5caNccTHX8J8MYO45YdpExJINSHUdWN658qlpGQyf/5WzGYrlhOXCD2dytjQEHWwXxN4sFmu7NKl+/nuu31YLFaykzO5PyCAh4OMz3Feb6iYd0+DBi3lbEJabnlJeDmqmkzQrprdPe3bvIPVC77OLTfxM/FYsFHnA02gUcXcY599Gcv+fXmf8CM7Amnu7wflAqFGu9z95iyzs2/BbTwaZlwIUR9Y4aJHsQBYK6X83CgfAHpIKQvMD9y+fXupF9xpfBIpVWOaacl7Eq3uImPowQsQew7SsyEjG1pVgRucZXYFJv6hGvYfjkB0ZfjkLqgUkl9u3UkYsVpdN9MKN9WGj+90XufAH+Cno3nlxX3gLifJ5BLSoNXivHKVENg72P62gdek5NT2BExbTvN7r/oEmK3saVHJTq73ihTqnri6BktTdDZ9N4nEuN2cOrBum5Sy/ZXU4c2V2bVQSXByiDP25TMUQoihwFCAunXrlohymjJO4mXYegYyLJCZrRreXvWcy87ZDhtPqUb9shle7gzd6+SXu5gBTT+w33dupPM6VxxRT9U5PNvOtaH48Sgcv6S2tyZASpZzQ2GRkJAOwOxHuxPbpBbMu+C8zhtuUK8cTuBa9vUB9mVXcgRipR7dVktA0HmtKzlNSVKhZjP++n3hVdXhTUMhnOxz2r2RUi4EFoLqUXhSKY0PcjoVDl5UbpDULGgQqbrrtlileuoe+Qv8bjx/BPnB/x6FOuXz17k3ER77Ia/ctZZrQ7HrHPx8LK98Lt25XKCffTk0IE83k8PPPcRB1lxABs4ABz/35Ww4cEH5q4Fffz3CmTOpXNx2hovp6TwdFqqMhA8SbVnFqKxHSvaiodUgvDZHjlzkwoXLubuj/PyoJIRyp7WqnLvfYpXs2H4ak5BYpImvtkfzdsXy6jvsUgt65D2s/rb2KMuWHcgtdxBVeTSrEwgBn98JVfN6lNdfv5Bjx5MANYtoU2QEDUx+MLAFTLwxV27jxjjuf+RnTH7++KVm0bVcEJ+1NR5MJt5o90Dxyiu/8OuvR/GTYDqSxOTm1akRksT2rJM89vt0AOTwERyffCdRUVFX/BF601DEoZLe51AblQReU5rJssCFy5CcpRr1OuWhamh+uYxsGPKzejq+lAXZFlg3IL8cKJfLK+vyykPb5DcUJgHVy+UZCVAuoFQXro5gh4b6cgHpox0HLzNdNOpBDnXmfBarjsHDzR1kHeo0W+2KW7bE88MPf3PhwmVSO0XjN9Dmr/Kb8f5rzhN7pHo1qg1vt+efNvU8Xb437RO2Ode3GDmRHKEMIlCnQojxFJgA148Bf+Wn37r1FD//fJgpdAOgXYMK9GlXUzWq9cvbGdqYhTuYvSoKKVVNy6b1pklUJIQHQr28gd+9e88xcuRKAgJMBAX40apxJd4c31XVGR4IFWpBiDKoP87dzP8OxuHvbyLAT/DEoLbc2KWuemQVNobcYmW/aQ+BgX4EBJjo2dsf7nA+LnNdwwwaPJBBQICSDQ0NUOMDTti0exwmk8Dk+NDgQOeelTh9pk2BMjm89dYtudvp6em88cYbvP322/j5+dH50FAaNWqEEIKrDaTqTUOxHBgphPgCNYidXNj4hMZLXMyAX46r95QsCPWH4dc5l526Cd63mbw2sycMbJlfLtDP3kcOeYOljjg21JcyXet6fTXYlpBXXnEYmlfKLxfsUGdGEQxFljIUs1ekEOvoc3d003xxGajmxF1TqxCXTghUbQ1VwcH8uE20ZZWdkTCbgrBmWwgyZbNoRwcqRAZz9z1N851nMqfz0W/V+c/3efqMfq4zffs2zydLQBhBdX8gK8tKcLA/4eGBHDnyHGFh+RvLy/I4p/7eQ/nyQapBbV8T+jhvgHtVS+TmCRAU5EdQkD9VqoSCX/7fRotuzVmz6yZ3Pg6eeaYjzzzTsVA5Pz8Tjz3W2q06IyODiXQ2CO8Ef2e/7WLixx9/5JlnnuHoUfWfGjx4MJUqOfndXyGenB77OdADqCyEiAMmAQEAUsoYYCVqauwh1PTYJzyli8YFu8/BoYvK/30qFUa1c+6mOZ2qBklzaFzBtaFwJCXL+X6TgHIBkGbT0Kab1VRBR0Idfqau6oT8jXpdJ/cDajbKrfWUwQj2h9rhzht+gHrR8Hp0XvkyBfjpPUOOy2Z9zS4khqgGoCIQCJxLN2M5nkTQ5XT2na1Kpz73c2PzKqi/99dgCoC6N3PuPNSqNRM/P0FoaACtWlXlgTmDnV6vmjzIrRGnCQryJzDQj4bd60Pt6k5l4+ObERERREBAwSatW7d6dOvmwr3nQOPGxdfIlWXi4+MZPXo033zzDQCtW7cmJiaGG2zHn4oBjxkKKWWBjkippls946nrX9OkZsGJSxCXohrBm5wMvAK88BvsPJtXvruhc0PhOG0ytYCG2pGCGvXwQPcMRY0wNYYQGqCMi6PbyZYp3dQ9h/grvR0Mx4ULlzl9OoUTJ5I51q06ffs2p1o15UeOvYLGP7puAOfWr2X24m1ErnuAOy7/wZDYRZhNAUgbd0atlHjqpMYVWl9sjRuwAH7mTG5Nf5M0EQLVy/HLwUq0HmumocmfDoDtp/Svf/3Onj3nqFQphMiKwVRp0RacNLQ1akiysiYU2qAD3HVXE+66q4kbnwBUruzEtagpEZ555hmWLVtGaGgokydP5rnnnsPfv/ib9VKXj0KDmnduEvZ+1RxWH4MBK/LKveq5NhS1w+0NhatBWkdDUVDj72+CyiHqnPBAte2K93upe4gINGRdNDida8L397usJj3dzIED50lMvExSUgYhIf7ceafzRu7BB79mzZo8l1fDhhXZnyXtehKLRlTk/Pl0qlR5O3dfREQQSUkvYwGWAp9cTuTWg98w4pdhUAX+9RKw4nXX92rDOw1HcK5GHb650InOravzaLlA+viHQlgNcvotR49eZOjinURGBlDhoomabaLoYXL+d504sbtb1xVCuGUkNL5NdnZ2rjGYNm0aAQEBvPPOOx6dEerRdRSe4JpdRxGzE7YnqNk/hy7CloHqSduRnQlwa95iHVpWhrUPO6/z1fUwf2de+V9dYVjb/HJSKtdTeBBEGq8RbrqerpJTp1I4eDCRo0cv0r9/K+XbNnDpKroKoqteYlSnY/xtsTL0gx1QMYSnav1GRkR5Bu/5ECsCs18AQZb8xjI5sDwRWZc4GtSR2q26EdDkAXsBkx97ztZECn8qVw4lPDyIcuUC1OpcjaYQkpOTmTBhAgcPHuSnn34q8u9GCFEq11FobEnKUDN0aoc7P77sbzWHPodDSc4NRdVy9uWzafllcriuGtxST003rRMOXWs7lxMC5vd2fuwqsVqly1kg3bsvYd26vKRc7dvXJDo6z+1UnEYid9rmceC4ChfwWyRq5fBJclf8mJB2RiKxZhcSowdTv+UgIow/bkGTEFs5d/NrNC6RUvL1118zevRoTp8+jZ+fHzt37uS660rmYQ20ofAuK4/AF/vUPP1TqXBPI/jgduey9SMcDMVF6OakYa8coqZp1giDuuEqhoyUzt1U9zdWrxJk+/bTvPHGOk6fTiUu7hKtW1fjhx+cT4ut2/MGnup/T2559h/AH/nHEd56KNjOTVSlSihnz74ImZfAYsyQWj8eYv/tUq+1tbuzlvwunEgg2JqNX3BFKje6l9CwmvjVuVn9cfwCqCRM6GFXjac4fPgwI0eO5KeffgLghhtuICYmhtat3ZuVVVxoQ+FNzqWrVbc5/OU0Ko7CJmgYAPGpzuUC/eDkMOeGwYPs2XOWv/9O5NChCxw6dIGpU2+hQoX84xOXLmXy/ff7c8vVHUJc2LqTgqsWngc+um4AFSuGEB1dlTrVBDfV3UeL6n/DO67v/0hEFJf9Q2iQfIS+93zHT1F9CAIygQeBdkBv412j8RYzZsxg4sSJZGRkEBkZybRp03jqqacwmUo+6Lc2FJ4g26ri+GyIh3VxMKSNcvE4cm8j+OcfeQu4jiSpWUDlAvLL9q6vFpQ1rgjN7CNI5sMLPu8+fT4lLu5Sbnnw4HZ07Jh/dXC9evYG7/x5+wF0R3dS3N7jnF6/iaZNK/P4463zBqmTjkDSIbV9Anb/axUcWupUt3MhlTFJK+WzLlFvyHFOh9XkRWAQsAy1TkEP8Wp8jfT0dDIyMhg4cCAzZsygatWqXtNFGwpPMPRn+O/hvHKnms4NRWQw3BYFyw+p2UKNI1Uvo1xEftm21dSrBNm//zz/+99Jdu48w/r1J+nUqRbz5jkPMtemTTU7Q3H48AWnhqJ27fKYTAKrsYrXNqSCLYv6HiZ712L8mgcjHhCQvh72PQjHq0DmRbC6XiC3uXoHTpWrySctHuPbxv1oLgT7gG+BHUDJfooajXucO3eOAwcO0LVrVwDGjRtHjx49uOkm9xYUehI968kTxOyEievzyg80cT0Y/PdF1aNoUiF/rKASIDU1i+Bgf6erRufM2cyzz/6YW+7UqRYbNz6VW/bErKMcFl12z/O/qt6tudthWanc0XclycGRVAFeNF4ajS9jtVr58MMPeemll/D392f//v1UrFix8BOLiJ71VJKkm1XoiW0JanGXMxxnDx246Lq+xhWKTzc3+e03Fcjs558PU+/mG6jTsr5zQVMjnpr7rN2uISWwIjnasiqv0HYk5oj6LAEOSMkv9W7hVFhNJILzIZVz3Ww1UCuVewLfe1xDjaZ42LNnD8OGDePPP/8E4NZbbyU9Pd0jhuJq0IbCXeJTYNpmWHZIGQuA/s2gtZMB1xaVoFEktKisIj3e5GLaqZcYPfpndu9WM6i6Plv/quqKrnyBUQHPw5EVhQs7EmYTVvtyItTqAtXakx1el9/q7uPnik2Z5mK85UaUQWgCVMF5KGKNxldJS0tj8uTJzJw5k+zsbKpVq8Z7771H//79fXJdjXY9uUtqFnT7XIXFyKF/M5hzi+tzShgpJbGxZ/nhh4PEBdYmK8TJWIcLFo1w/gTz/Qffc+uFJwk1pUJAOUzm5MIrMwWA1QxhtcHGNZSLOQ1umQch9u6l48BiwNX65srAXpRh0GhKM3369MldNDd8+HCmTJlCZGSkR6+pXU8lQVigioT60PK8fd8fhNe7OE8iU0LkHycIgAgn0VoLwHoxEeW4scGcBv9uyP3pCSp4PoCtkajeEUKNWRjSCkERcP3zUL1DUW+BrYCzsyoBzVHRIp8scq0aje8ybtw4EhISmD9/Pp06+X4GaG0obMmywK/HoU8D58d71oWHm6kpr3c2gH+08qqRgIJXJ9eLtDJhQOV8+y0WK888s5LevRvSs2d9KlRorBblmdMg+QgcXw2/j7U/qf1Y6DAOTP4QUA78nEzhLQLJwAjgMyfH7gVeQSdQ15QNsrOzef/99zl27BizZs0CoEePHmzdutUrayKuBG0ocvgzHsavU5nPPr0TersIxDCtu5qd5MHY8o5kZVlYv/4EP/74N9b6zbkkyuWTWTSiIo899h2ffhqbuy8m5k6Uw8YePz8TMTF35e1IiYclzSErJZ8sdXrCfcsg0EVokSJyAVyuZL4ZWE1eB0ajKe1s3ryZp59+mp07VUy1oUOH0rKl6vGXFiMB2lAoJv4BMbvyyi+vUzOXQp08NTvb50EmLD5DwuVAIAKinLsXo+sqnR54oAXffbePm26qx333NaN374b5haUVErZBdoYq/zgQLh3PL9f8Mah3C7R4vFgW8ElgOLDAYX9VlHEo2YAEGo1nSUpKYvz48cTExCClpF69esyZMyfXSJQ2tKEA6FIbFuzKy9h9MgXe2wbjO5eoGs7XJeTPFNa4muClfvmn1d51VxNSUl7Bz0kmMKSE/Z/BysdcK9BtKnQcV0StC+c8+QegWwG/ogyFRlOW+OKLLxg9ejQJCQn4+/vzwgsvMHHiRMqVy+8JKC1oQwFwexS80hne3KjKA5rD4OiCzykmzGYLn34aS/XqYcSecD7zKCX+DF++mRc6/IMP7gHyGwq7RXPSChkqkTvHV8MPDqHGhQlqGgndTQHwwCo1/lCMfAIMdLI/Dsi/ZlujKRusWrWKhIQEunTpwvz584mOLpm2xJNoQ5HD6OtV+Izbo1wn+ikGnPca6sKRvJLjVNV33tnP79XD6NOnEX36NOLWW524lGyxZsP75SHbeXgMBu6Equ4lb79SpgATHPZdD2xCx1XSlC0yMzOJj4+nQQM1CWb69Ol069aNf/zjH6VqHKIgrq11FKlZKgS3qyxfrsJxFyOFrWyOrhvAqLvsB46zsiz4+5tc5m3IRVphpsO9BUWq/ZYMuGUBtBp0BVoXjgR2AdOALxyOfQvcj14Upyl7rFmzhuHDh2Mymdi1axeBgfldxb6CXkfhDhcz4OH/qpXSr3VxLlOCKyIXjahInTrv2gXS27TpKTp2zO9+CnQnBtSfk2DjZPt9LR6HPv+5WlXdojzgLPD5QVQSII2mLJGQkMDYsWP55JNPAGjWrBlxcXG5vYqyxrVhKFKzlJHYnqBeHWvAHZ79QrOyLHz11V+Ehwdy773NAOV2sqVjx1rExV2iYsUQHnqoBTWcZaxzhZQqrPaGSap8Pm9aLFVaw8AdahzCwywEnnbYF2LsH4Ce6qopW1itVhYtWsTLL79MUlISwcHBTJgwgRdffNGnexNXS9k3FFYJI39RBiKHZ3+BFv1V1rhiJjMzm08/jWXy5N85fjyZfi/1Y0W8vbspZzrrM8904PHHW3PbbY0IDnbjq5BWSNwHPz8BZ7Y4l3nyb6jQ6Gpvo1A2AM76ZRa0cdCUXe6//36WL1fRGW677Tbmzp1Lw4aFjBmWAcq+oRCo3NArj+RNf21cQaUM9QDp6WZGjPiBTCMZUYV6Ne2O245B3HxzQdmVbUg6Ap92gAwX4xtd34QGd6rYSiGejzo5BnjXYd9MYDR6HEJTtunbty+bN29m1qxZPPjggz4ZwM8TXDuD2WuOw9OroFYYrOinYjd5iAEDvuXzz/fQe/jd1G1VH3AddK9QVg6EfZ/k33/dKLhuJFQomREACfQA1jnsHwu8nU9aoykbLF++nLi4OEaMGAGowJupqamEhxdPpIKSRA9mu8PN9WDVQ+AnisVIZGRku3QXDR58HYmRUblGIqpSEYyxlJASB7H/zj843XYkdH8b/AtIg+ohWgL7HPalAEUYVdFoSg0nTpxg1KhRLFu2jKCgIG6//XYaNGiAEKJUGomr5doxFABRVz8mceZMKhMmrGHp0v28smgo+09bnUhFUreVChnsbLqrfYVbYc2zxmI3AfF/OJd7NgUCvdMsf4i9kYhFrazWaMoaZrOZ2bNnM2nSJNLS0ggPD+eNN96gXj0nqYyvIa4tQ6TqMk4AACAASURBVHGVvPfeRiZO/I3U1CwAF0Yij0KNxJ4lamDaGcKkBq8fXq9WUHvJF1oFFYIjByt6HEJTNtm4cSNPP/00u3fvBuDBBx/k3XffpVYtHUeg7BmK85eVe6lC8btnEhJSc42ELUUef5ASfn4S/lqSt6/tM9D0IbUdXhci6l+xnsXBVFSob1s2o42EpuwyceJEdu/eTVRUFHPmzOGOO+7wtko+Q9kzFBP+gFXH4Ok2MKwtRAQVW9Xjx3djyZJdnDnjbGmZG2Slwu6FanD67I68/YMPQ6RvLNSROJ/eWrqmPGg0hSOlJCUlhfLlywMwZ84cPvroI/75z38SGhrqZe18i7I15X1/Inx3EFKyYMYWaPcf2H2u2KoPDw/irbd6UaFCMLNn3170CuZWgN9fsDcSz1z0GSNxmPw/iG/QRkJT9jhw4AC33HILffv2JWfmZ9OmTZkyZYo2Ek4oW4bi/e32rVrtcGjpKk2Oc5KTM/jhh4Mujz/+eBsOHRqFjGrhXoXmdFj3MrwjVLA+AL8g6D4DnrkAwZ7Nk1uoesB7KJeS4zI9K9CvxDXSaDxHRkYGkyZNonXr1qxZs4adO3dy7Ngxb6vl85Qt11ODSIgMgqRMVR7bAZzlZnDBn3+eoH//bzh7No1t24YSHV0tn4zJJPhkQ3ZuBNicVdZOORcLHzlJyfPcZa8NTtvyJ9DVyf5HgY/R4xGassXq1asZMWIEhw4dAuDJJ59k+vTpVKpUtIfJaxGPGgohxO3ALFRk6X9LKac6HK8L/AeINGRellKuvOILvtBBjU18ug9+OVakeE5z525mzJhVZGWpFdVPPLGMjRufys3x4Cw8uMtZTWlnYNNbsGN23r5yNeC+5VD9ita7FDv7sDcSNYHJwJNoA6EpW0gpGTx4MIsXLwagRYsWxMTE0K1bNy9rVnrwmKEQQvgBc4FbUblqtgghlksp99qITQC+klLOF0K0AFYC9a/qwmGBylg8XbR8CwEBfrlGAmDbttMsWLCVZ57pCOCekYjfAMvuhcvn7fff/wM08J0ZFBcAW8fZd6gw4BpNWUQIQf369QkJCeHVV19lzJgxZTqAnyfwZI+iI3BISnkEQAjxBXAvYGsoJCpCNUAEcMqD+hTIkCHtWL/+BB9/rOZQv/DCDQwe3C6fXL6psFLCiTUq7/SqwfbH6t8GXf4F1Tt4Su0iIwHbjvY0tJHQlD127tzJ6dOn6dOnDwDjxo1j4MCBREW5GV9NY4cnDUUt4KRNOQ7o5CDzGrBKCPEsUA64xVlFQoihwFCAunXrFruixjWYP/9Ojh5NYuTIDvTv78ba47+/g+VOhnt7zIQ2I8C/+KbmXi2ZgOPKkkeAl7ygi0bjKVJSUpg0aRKzZs2iUqVK7N+/n4oVKxIUFKSNxFXgSUPhzNXtONPyEWCJlPIdIcQNwMdCiFZSSrslz1LKhagUB7Rv395jszXLlQtk3bpBhUeEtGbDN7fCybX2+5s+DI37QtMHPaXiFbEbcHTE1QU+84IuGo0nkFKydOlSRo0aRVxcHCaTiQEDBhAQUMBkE43beNJQxAG2yadrk9+1NBi4HUBK+T8hRDBQGTjr9lUOXlCL6qqVc0v8778TCQsLpEYN56E1bI2E0/zWC2pBqsNtlEAO6itFYm8kAlCZ6LSHVlNWOH78OCNHjmTFihUAtG/fngULFtCuXX7XsebK8OQ6ii1AYyFElBAiEHgYWO4gcwLoBSCEaI7yjhRthdybG6HVYnjkv/D935CR7VL0r7/O0rPnf+jefQkHDya6lJu9IoUh8y7kH8C2rLI3EgHl1II5HzUSR7H/gj8HstBGQlN2kFLSr18/VqxYQfny5ZkzZw4bN27URqKY8ViPQkqZLYQYCfyMmvr6oZTyLyHEZGCrlHI58AKwSAjxPOrhd5AsSoKMTAv8cERt/3JcvVY/CG3zr384fPgCvXp9REJCGgCdO/+bb799iJ498/stbQ1EtP9mRqX0sRd4ZAOUrwdhNfFVfsV+wKcXylJrNGUBq9WKyWRCCMGMGTOIiYnh3XffpUaNGt5WrUzi0XUUxpqIlQ77XrXZ3ovzjJrusfm0fblaKLSu6lR09eojuUYC4OLFDBYv3kmPHvXzuZtyWHTZYSHOvcug3q0Q4JnseMXBSuBOh30vAtO9oItGU9wkJiby8ssvA7Bo0SIAevToQY8ePbyoVdmndK/MTs2C6uXgjGEAaoaByflA9LBh7Tl2LIlp0/4EYOjQdsybd2e+gevcFdeWVfYVjLGo0N8+ihnnLqX1XI0l1mh8AyklH330EWPHjuX8+fMEBgYyadIkateu7W3VrglKt6Ho0wBuj4K/EmHc7/BswX7Jt97qxYULl0lLMzN//l2YTML5gDUwKusR6L8OqraFQN/OaGUlv5GIAZ5C+fw0mtLMvn37GD58OL///jugehDz58/XRqIEKd2GAlTMpFaV4YfCw9flrJXIzrZiMnoezoxEtGUVPLEfKjYtdnWLm01AZ5tyRVSiIR2GQ1PakVLy6quvMm3aNMxmM5UrV+add95h4MCBhU9h1xQrpd9QFBE/PxN+fiYwpzE75r/g1xtwGI8YY/WJoH2FkY29kXgEvTZCU3YQQhAfH4/ZbGbIkCFMnTqVihWLmCRMUyz4rtP9KpBScvly/p6CLbMXrSLWMBJ24xHPZ5cKIwHGUnWDj9FGQlP6OXXqVG4qUoDp06ezfv16Fi5cqI2EFymThuKzz2K57roFbNvmOnRULN0BiA7czqhh98KodHhBgql0ePUbA4uN7VbAY17URaO5WiwWC3PmzKF58+Y8/PDDZGWplMOVK1emSxc9HcPblDlDkZycwfPP/8yBA4l07vwBb775BxaL1V7or//kbo76x41quqsPT3m1JSeo3yGjPBMVokOjKa1s376dzp078+yzz3Lp0iUaNmzIpUuXvK2Wxga3DIUQIlAI4ZgAzXukZsGDy2DK/+CHwxCft/Zh+vQ/OXcuHYDsbCuvvbaW48eT1UGLGX4bzew1NsH6AkpX2sOuqDDhOTyPHrjWlE4uXbrEc889R4cOHdi6dSu1a9fmu+++Y/ny5VSuXNnb6mlsKHQwWwhxJ+rBNRCIEkK0BSZJKb0XnfrABVh7Ur0AGkTApoFIKfnzz5N2oq+80pUGDSqowoZXYfssYkPUmr/oyhdQ84RKB/8CNtiUra4ENRofR0rJTTfdxK5du/Dz82PMmDG89tprhIf79lT0axV3ehSTUeHBkwCklDvJn165ZNnrEKephXr6EELw22//4NtvH6JFiypUq1aOl14y/JtSwma7BHuMesh3OkmFkQy8alO+gO5JaEovQgief/55OnbsyNatW3nnnXe0kfBh3Jkea5ZSJjnMW/ZYqG+3OHjBvtw8r1cghKBv3+bce29TDh26QLlyaina7C/3ExviOhCgr3OrzfYhoIK3FNForoCsrCxmzpyJn58fL774IgCPP/44jz32GH5+pWMCybWMO4ZinxDiIcAkhIgCngM2elatQnioGTSqAEeS4GgyXF89n4ifn4mmTfP8nLEX7AMFRtctPXHqJ6FC8QLcCDT0oi4aTVH5448/GDZsGHv37iUoKIjHH3+catWqIYTQRqKU4I6hGInyelhR6ZV/Bl7xpFKFEl1FvdzlwNcY0cxZdMsaaPKAZ/TyAD+ifH85rPCWIhpNETl//jwvvfQSixeridyNGzdm3rx5VKuWP7qzxrdxZ4ziNinlOCnldcbrZaBPoWf5ELPX2iQ1KkVGAuAOm+0EtMtJ4/tIKVm8eDHNmjVj8eLFuQH8du/ezS23OM12rPFx3DEUE5zs+2dxK3I1JCdnuD6YfpZYiwp0ER0ZV0IaFQ/32my/BTgPoK7R+B6ffPIJiYmJ3HzzzezevZvXXnuN4GDHrO2a0oJL15MQ4jZUmtJaQoiZNofK40MzM1NSMqlVayYdO9biscda069fcyIijB+klPDvhuB/HIBRDzTwoqZF4yfs0wG+7C1FNBo3SE9PJzk5mRo1aiCEYN68eWzZsoVHH31UB/ArAxTUozgL7AEygL9sXqvwIdfTqlWHSUsz89tvxxg8eDkdO/6b3CR56//JbLEoTzgwzDtKFhEr9h9wmitBjcYH+PHHH2nVqhUDBw7M/e81bdqUxx57TBuJMoLLHoWUcgewQwjxqZSyAN9OCWORamV2mJr2unz5QbvDd97ZWP04Lx2HzW/lTomNrlN6opXYzgP5AShda8c11wrx8fGMHj2ab775BoDw8HASExP1quoyiDuznmoJIaYALYBcJ6OUsonHtCqI06kQtRAig6BWOJsP26dDveeepmC1wKL6dvtH3R1ZgkpeOS/YbAdjP5it0fgCFouFuXPnMmHCBFJSUihXrhyTJ09m1KhR+Ptfc5kLrgnc+VaXAG8AM1AekSfw5hhFtlUtSU7KhKRM/oq5g90NI1iz5igbNpyky4214N3S+2O1HQxK9ZoWGo1zrFYr3bt3588/VUrh++67j1mzZlG3bl0va6bxJO60qKFSyp+FEDOklIeBCUKIPzytmEvMVru8n6aa4bRtW522baszZswNcOK3vIMRDSCr5FW8UmbbbMeh05hqfA+TyUTv3r05ceIEc+bM4Z577vG2SpoSwB3HfaZQI1KHhRDDhBB3482ZmhLwsxkgq17O/vh3t+duzq6+o2R0KgZOo5a851DLW4poNDZIKfnyyy/59ttvc/eNGzeOvXv3aiNxDeFOj+J5IAwYBUwBIoAnPalUgTSpAJuGQ2IGJKRBbZuZTKc2gsXoQjQbQOzxbKB0hOuoabP9X69podHkcfjwYUaMGMGqVauoUqUKN998MxUqVCAoKIigoKDCK9CUGQo1FFLKTcZmCjAQQAhR25NKFYqfCaqGqlcOUsLnN+SVb/sQFqqJpaPu8u2olM1ttp8E7vKWIhoNkJmZydtvv82UKVPIyMigQoUKTJkyhYiICG+rpvESBRoKIUQHlBdkvZTyvBCiJTAOuBnwrrFw5MPGedt9VzL7p9IxOBEMZNqUP/CWIhoNsHbtWoYPH87+/fsBGDhwIDNmzKBqVR0X4FqmoJXZbwH9gF2oAezvUW70acCwklHPNe+/v4m0NDPt29ekQ+N0IpIO5x2M6kPsjyoUuS+7nVpgbyTM3lJEo0FNex0xYgT79++nadOmzJ8/n549e3pbLY0PUFCP4l6gjZTyshCiInDKKB8oGdUKJiZmG3v3ngNgfr//MizH6zQ6007OV91ObYF9NmXvJvjQXKtYrVYyMjIIDQ3Fz8+P+fPns27dOl566SU9DqHJpaBZTxlSyssAUsoLwH5fMRJms4WDB/OSEJ26ZBiDqu3AL9DFWb5DO1Q3LQeLtxTRXNPExsbSrVs3nn322dx93bt3Z+LEidpIaOwoqEfRQAjxnbEtgPo2ZaSUfT2qmSvOpXNk1lays/PW/E2+ba3aaNIPgNkrUrygmHvcAthO2k3DvTnKGk1xkZaWxuTJk5k5cybZ2dkcPXqUixcvUqGCDmKvcU5BhqKfQ3mOJxVxm1OpRLy1iemhoexrWZG/A2yex8PU6oPYE8rb72vjE4nArzblTOzWDmo0Hue///0vI0eO5MSJEwghGDFiBFOmTCEysnSEuNF4h4KCAv7q6pi3qW4y8WJoCAztAA+E5U0VajHQrjfhS+MTZsA2VJoV1U3TaEqC7Oxs+vfvz3ffKadA27ZtWbBgAR07dvSyZprSQOn2ekQGwfL71bYwgTD5ZG/iHPY9hyFoI6EpWfz9/YmIiCAsLIx3332XLVu2aCOhcRuPGgohxO1CiANCiENCCKe5d4QQDwkh9goh/hJCfFZopZVD4aGm0CcK6obBud1q/42v24n5Um+is832HcBCbymiuabYtGkTmzZtyi2//fbb7Nu3j9GjR+sor5oi4favRQgRJKXMLFwyV94PmAvciopxt0UIsVxKuddGpjHwCtBFSnlRCFH4qp5aYTD3VrW9xiY6UqfxPjmInQwcMbbvA773oi6aa4OkpCReeeUVFixYQLNmzdi5cyeBgYFUqlTJ26ppSimF9iiEEB2FELHA30a5jRDifTfq7ggcklIekVJmAV9gnwYalBdmrpTyIoCU8qzbmqclwA4j3mpIZZ90O0nAdojwa28porkmkFLy2Wef0axZM2JiYvDz8+Oee+7BYtETsDVXhzuup9mo8EOJAFLKXYA7yzVrASdtynHkD4raBGgihPhTCLFRCHE7biClhKM/5u14Yr/dcV9xO7W32R5MEbpvGk0R+fvvv+nduzePPvooCQkJdOnShR07djB16lRCQkK8rZ6mlONO22WSUh53yH3rziOKs/FaxwXI/kBjoAcqdtQfQohWUsoku4qEGAoMBahcuQ6hoW9y+Y0JABzlBqJCfK9LvQbYblP+t7cU0ZR5zGYzN998M3FxcVSsWJHp06fzxBNPYDKV7rkqGt/BnV/SSSFER0AKIfyEEKOBg4WdhOpB1LEp10aFAXGUWSalNEspjwIHUIbDDinlQilleyll++DgcoT5JeceO3hBBej2pfGJLKCXTTnbW4poyjRSqueugIAApkyZwqBBg9i/fz+DBw/WRkJTrLjzaxoOjAHqAgmoSTzD3ThvC9BYCBElhAgEHgaWO8gsxXBjCSEqo1xRRygAs9nCc9025pa3lRsF+NYiu2ib7cXoTHWa4iUhIYGBAwfyxhtv5O57/PHHWbx4MVWqVPGiZpqyijuup2wp5cNFrVhKmS2EGAn8jGorP5RS/iWEmAxslVIuN471FkLsRbmzXpRSJrquFcyJl3mygwqCsTO+GrXb2sfI9/b4xEzyulvBwCDvqaIpY1itVhYtWsTLL79MUlISkZGRjB49mvBw3xiT05Rd3DEUW4QQB4Avge+klG77eKSUK4GVDvtetdmWqN7KGHfrjJKCmhGpAIS2e4xbbm7g7qke53ngPZtygRZPoykCu3btYtiwYWzcqHrTt99+O3PnztVGQlMiFOp6klI2BN4ArgdihRBLhRBF7mEUGwF5CYmaPPgqNWuG+8T4xLPYG4njQKgLWY3GXcxmM2PHjuX6669n48aN1KhRg6+++oqVK1fSoIHvPCRpyjZujXhJKTdIKUehImRfAj71qFYF4Wcz4SqoPOD98QkL9hETT6EGdDSaq8Xf358dO3ZgtVp59tln2bdvHw8++CAOsxA1Go9SqOtJCBGGWij3MCq98zLgRg/r5ZpqxgzbKvkzS3trfGKQzXY8UMMrWmjKCidOnMBisRAVFYUQgpiYGJKTk2nfvn3hJ2s0HsCdHsUe1Eyn6VLKRlLKF6SUmwo7yWP4GU9SwaleU8GRT4z3pkBNbyqiKdWYzWZmzJhB8+bNGTJkSO7018aNG2sjofEq7gxmN5BSWgsXKyGk4Xqq1xvw/vqJQzbba7ymhaa087///Y9hw4axe7cKclmxYkXS09MpV66clzXTaAowFEKId6SULwDfCiHypXT2VoY7a4YyDEcT/Ak5k+r18YnrbLZ1b0JTVC5evMjLL7/MwoUqpnBUVBRz586lT58+XtZMo8mjoB7Fl8a7b2S2M8hUdoEnx/xF7RtWE3xDd8A74xNzgRwHWIcSv7qmtJOZmUnbtm05ceIEAQEBvPjii/zzn/8kNFTPl9P4FgVluNtsbDaXUtoZC2MhnVcy4IUEqIAYCanlaFMphDRvKGEw0mZ7tde00JRWgoKCGDx4ML/++ivz58+nRYsW3lZJo3GKO4PZTzrZN7i4FXEXs0WpnJoZSKVK3ouKaTsysgWIcCWo0RhkZGQwadIkPvssLz/X+PHjWbt2rTYSGp+moDGK/qgpsVFCiO9sDoUDSc7P8jwmY9JT0woRpFZr5LWIe+VttvV8FE1hrF69mhEjRnDo0CGqVq3K/fffT0hIiM40pykVFPQr3YyKQlEb5Y7PIQXY4UmlCsLPiFS+ekBHhmSrHkVJD2Tbzg12zMSk0dhy5swZxowZw+effw5Ay5YtiYmJ0TkiNKWKgsYojgJHgV9KTh03MBkTsAICc3eV5EC2xD4H9tISu7KmNGGxWFiwYAHjx48nOTmZkJAQJk2axPPPP09gYGDhFWg0PkRBrqffpZTdhRAXsU84JFDx/Cp6XLuCqFYBvLC6w3ZQZ1bJX15TSrBYLLz//vskJydzxx13MGfOHKKiorytlkZzRRTkespJd1q5JBQpMsOuh3kXSvSSvzuUR5Xo1TW+TkpKChaLhcjISAIDA1m0aBEJCQn07dtXx2bSlGpcznqyWY1dB/CTUlqAG4CnAe8uFw3wzuUftdn2naXqGm8jpeS7776jefPmvPDCC7n7u3btSr9+/bSR0JR63JkeuxSVBrUh8BEqMOBnBZ/iYYR3csbFG+9f4TwhuOba49ixY9xzzz3069eP+Ph49uzZQ0ZGhrfV0miKFXcMhVVKaQb6Au9JKZ8FanlWrYJJSbMwYMC3JXrNF222dXAFjdlsZtq0abRo0YIVK1ZQvnx55syZw4YNGwgODva2ehpNseJWKlQhxIPAQOA+Y59XE1NnZEoSI6NKzP+VDsywKYeV0HU1vkl6ejqdO3cmNjYWgIcffpiZM2dSo4YOMK8pm7hjKJ4ERqDCjB8RQkQBn3tWrYKxWAV1W9UHSmYNxb9str220lDjM4SGhtK+fXvS09OZN28evXv39rZKGo1HKdRQSCn3CCFGAY2EEM2AQ1LKKZ5XzTVWmTdCUBJrKH423uujQ3Vci0gp+eijj2jYsCFdu3YF4N133yUwMFAvnNNcE7iT4a4b8DFqLFcA1YUQA6WUf3paOVdUKMHB7B/IW4buVeuo8Qr79u1j+PDh/P777zRv3pydO3cSGBhIRIR+ZNBcO7jjenoXuENKuRdACNEcZTi8FuIopHxyiVxHArYJV/uVyFU1vsDly5eZMmUK06dPx2w2U6VKFV555RUCArw6PKfReAV3DEVgjpEAkFLuE0J4NwbBuepQ1/OX+cRmexkQ5PlLanyAn376iWeeeYYjR44AMGTIEKZOnUrFit4NRqDReAt3DMV2IcQCVC8C1LozrwUFBKB6+cJlioHHbbbvKZErarxNamoqAwcO5Pz587Rq1YqYmBi6dOnibbU0Gq/ijqEYhopW8RJqjGId8L4nlSqUSmH2CSE8gG308o9dSmnKAhaLBavVSkBAAGFhYcyaNYu4uDief/557WrSaCjEUAghooGGwPdSyuklo5IbmDwfw3+8zfZjHr+axlts27aNp59+mnvvvZeJEycCMGDAAC9rpdH4Fi5XZgshxqPCdzwKrBZCOMt05xWyszwbIsEKvG1sN/LolTTe4tKlSzz33HN07NiRbdu28fHHH2M2m72tlkbjkxQUwuNRoLWU8kGgAzC8ZFQqnHPHjni0ftsE4Ws8eiVNSSOl5Ouvv6ZZs2bMnj0bIQRjxoxh+/bt2s2k0bigIB9OppQyDUBKeU4I4U5cqBLhrzNVoJrn6h9rvAegQudqygYpKSn079+fH3/8EYBOnToRExND27ZtvayZRuPbFGQoGtjkyhZAQ9vc2VLKvh7VrABsV2YXe91AjgNiiceuovEGYWFhZGZmEhERwdSpUxk6dCgmk888/2g0PktBhsJxfdkcp1JeYH3Uex6r23Y6V3+PXUVTUqxbt44aNWrQuHFjhBB8+OGHBAcHU62aB7ukGk0Zo6Cc2b+WpCJF4XSEmtfuiYCAo222vZP1QlMcnD9/npdeeonFixfTq1cvVq9ejRCCevXqeVs1jabUUar73cUdENB2LpV3MzNprhSr1cqHH35I06ZNWbx4MYGBgXTr1g2LxeJt1TSaUotHDYUQ4nYhxAEhxCEhxMsFyD0ghJBCCK/FjwJ4x2b7Ya9poblS/vrrL3r06MHgwYO5cOECvXr1IjY2lkmTJuHv7/m1NxpNWcXtf48QIkhKmVkEeT9gLnArEAdsEUIst40bZciFo1Z+b3K3bk8xwXhvgk51WtpITk6mc+fOpKamUrVqVWbOnMmAAQN0vmqNphgotEchhOgohIgF/jbKbYQQ7oTw6IjKXXFESpkFfAHc60TuX8B07D0/JY6tq2mCSymNryGlBCAiIoJx48YxbNgw9u/fz6OPPqqNhEZTTLjTo5iNira9FEBKuUsI0dON82oBJ23KcUAnWwEhxHVAHSnlCiHEWFwghBgKDAW4vrYbV74CXrfZHuiZS5RKzGYzcXFxZGR41Y7nIzs7m4sXLxISEkJYmEpO26+fmqh35swZzpw54031NBqvERwcTO3atYt1Aak7hsIkpTzu8HTmzsigs8c5mXtQLeB7FxhUWEVSyoXAQoD2dYQsRLzIJAIHje3XCxK8BomLiyM8PJz69ev7xBO6lJKzZ88SHx9PaGgoQUFBNGvWzCd002i8jZSSxMRE4uLiiIqKKrZ63TEUJ4UQHQFpjDs8S167WhBx2C9srg2csimHA62AtcafvDqwXAhxj5Ryq6tKz4riu3lQlquyTfmFYq299JORkeEzRiItLY3jx4+Tnp4OQGRkJHXr1vUJ3TQaX0AIQaVKlTh37lyx1uuOoRiOcj/VBRKAX3Av7tMWoLEQIgqVRvVhIDcsp5QyGZs2WgixFhhbkJEAuCxULopyllTg6hPJ7LLZHgWUu+oayx7ebogtFgvx8fGcPXsWgMDAQOrWrUtkZKRX9dJofBFP/F8LNRRSyrNcwWxRKWW2EGIk8DNq7dqHUsq/hBCTga1SyuVF1taGhqknKI40d9cZ7wKYddW1aTyBEIJLly4BUL16dWrUqIGfn14OqdGUFO7MelokhFjo+HKncinlSillEyllQynlFGPfq86MhJSyR2G9CVs61Lj6xXZxNtvDrro2TXGSkZFBdrZKH2UymYiKiqJFixbUrl3bqZFYvnw5U6dOLWk1fY61a9cSERHBddddR7NmzRg71n6OyNKlS2ndujXNmjUjOjqapUuX3eTTHwAAIABJREFU2h2fMWMGzZo1o1WrVrRp04aPPvqoJNV3i/fee88n9cohMzOT/v3706hRIzp16sSxY8ecys2aNYtWrVrRsmVL3nsvLyxR//79adu2LW3btqV+/fq5QStjY2MZNGhQCdyBE6SUBb5QIY9yXv9AzX56v7DzPPWqXLeNfGpuopTbz8ir5V6bijXO2bt3r/2Oyu/bv1zxn1h7ued/det6FotFxsfHy61bt8qjR49eueIFYLVapcVi8Ujd7pCdne2xun/77Td55513SimlTE9Pl02bNpXr16+XUkq5c+dO2bBhQ3nkyBEppZRHjhyRDRs2lLt27ZJSSjl//nzZu3dvmZycLKWUMikpSS5ZsqRY9bvaezebzTI6OlqazeYinVOSzJ07Vz799NNSSik///xz+dBDD+WTiY2NlS1btpRpaWnSbDbLXr16yYMHD+aTGzNmjHz99ddzy7169ZLHjx8vVId8/1spJcqTc0XtbqE9Cinllzav/wB9gRaeM11uUgxuuGXGe42rr0pTDKSkpLB3715OnTqV+wM9evQozZo146mnnqJVq1Y8+uij/PLLL3Tp0oXGjRuzefNmAJYsWcLIkSMBSEhI4P7776dNmza0adOGDRs2cOzYMZo3b86IESNo164dJ0+e5PPPPyc6OppWrVoxbtw4pzodO3aMbt260a5dO9q1a8eGDRsA9dS3cuXKXLlBgwbx7bffYrFYePHFF+nQoQOtW7dmwYIFgHrS79mzJwMGDCA6OhqA++67j+uvv56WLVuycGFeJ/2DDz6gSZMm9OjRgyFDhuTe17lz5+jXrx8dOnSgQ4cO/PnnnwV+niEhIbRt25b4+HhA9RbGjx+fOxsmKiqKV155hbffVmm63nzzTebNm0f58mocMCIign/84x/56j106BC33HILbdq0oV27dhw+fJi1a9dy11135cqMHDmSJUuWAFC/fn0mT55M165dmT59Oh07drT7fFu3bg2obIPdu3fn+uuv57bbbuP06dP5rr1mzRratWuXu9J+0aJFdOjQgTZt2tCvX7/ciQ6DBg1izJgx9OzZk3HjxpGWlsaTTz5Jhw4duO6661i2bFmB3+/VsGzZstzP7YEHHuDXX3/NXe+Tw759++jcuTOhoaH4+/vTvXt3vv/+ezsZKSVfffUVjzzySO6+u+++my+++OKqdSwyRbUsqNSoh67UMl3tK7dHcSK5UKvqCquUsqZNpeuvuKayT0n0KLKysuSRI0fkli1b5JYtW2RsbGzuU+3R/7d35vExXe8f/5wkiNQSBE1DQiSyTDKTRSKhxBpaa4Sg1FJrUV+UomiV/mqp0tpLq2qpKLUVRVXsUbFERBCtRCxBkEQSss7z++PO3MxkZpJJmkwyyXm/Xvf1usu55z73zJ373LN9nrg4MjU1paioKMrLyyMvLy8aOXIkyeVy2rdvH/Xp04eIiH766SeaOHEiERGFhITQihUriEj4gk1JSaG4uDhijFF4eDgRET18+JCaNm1KT58+pZycHOrYsSPt3btXw7aMjAx6/fo1ERHFxsaSt7c3ERHt2bOHhg0bRkREWVlZ1KRJE3r16hV9//33tHDhQiIiyszMJG9vb7p79y6FhYWRhYWF+DVPRPT8+XMiEr78JRIJPXv2jB4+fEh2dnb0/Plzys7Oprffflu8r8GDB9OZM2eIiOjevXvk7OysYa9qjeLFixfk5eVFiYmJRETk6elJkZGRaukjIyPJ09OTXr58SZaWljp/I1V8fX1pz549RET0+vVrysjIULsuEdHEiRPpp59+IiIiOzs7WrJkiXhMJpPRv//+S0REixcvpoULF1J2djb5+/vT06dPiYgoNDSURo4cqXHtzz77jFauXCluP3v2TFyfM2eOeGz48OHUo0cPsQYze/Zs2rp1KxERJScnk6OjI6Wnp+v8fQvy9ttvk0wm01j+/PNPjbQSiYTu378vbtvb21NSUpJampiYGHJ0dKRnz55RRkYG+fn50aRJk9TSnDp1SsOes2fPUs+ePbXaWDD/guA/1CiK7MxmjCUjf/6DCYAXAHTqNhmMpnVKfGon5I/TrQagbWnYwykROTk5uHHjBnJzc8EYg7W1Nd588021OBHNmzcXv8IlEgk6d+4Mxhjc3d21tv+eOHFCbMM2NTVF3bp1kZycDDs7O/j5+QEAIiIi0KFDBzRs2BAAMGTIEJw+fRp9+/bVsG/SpEmIjIyEqakpYmOFkeHvvPMOJk+ejKysLBw5cgTt27dHzZo1cezYMURFRWH37t0ABGmRO3fuoHr16vD19VUb275y5UrxK/L+/fu4c+cOHj9+jICAANSvL4zoGzBggHjN48ePIyYmXwHn5cuXSEtLQ+3a6v11Z86cgVQqxe3btzFr1iy8+eabAISPwoIjYpT7tB3TRlpaGh4+fIigoCAAwuQufRg4MF+0PyQkBL/++itmzZqFnTt3YufOnbh9+zaio6PRtWtXAMJIN2trzbp+YmIiXFxcxO3o6GjMnTsXKSkpSE9PR7du3cRjAwYMEPuzjh07hgMHDmDZsmUAhD6whIQEvPXWW1p/34KcOXNGr/sE8tUCVClYti4uLpg5cya6du2KWrVqQSaTaeiR7dixQ602AQCNGjXCo0ePYGgKdRRMuDsZhOGtACAnbaVgZJxUWU8vLyOMlaRJ+qUb5iYsRVCtWjVYWloiOzsbtra2Wl88NWrUENdNTEzEbRMTE7HDWx/eeCN/8LOux3jv3r344gth2uUPP/yAgwcPonHjxrh27Rrkcrlon7m5OTp06ICjR49i586d4h+aiLBq1Sq1FxYgND2pXv/kyZM4fvw4wsPDYWFhgQ4dOiAzM1OnXYCgjBseHo6aNWsWep/t2rXDwYMHERsbi7fffhtBQUHw8PCARCLBpUuXxKYeALhy5QpcXV1Rp04dvPHGG7h79y7s7e115q3LPjMzM8jlcnG74Ex+1XsfOHAgBgwYgH79+oExBkdHR1y/fh0SiQTh4eGF3lvNmjXV8h4xYgT27dsHmUyGzZs34+TJk1qvSUT47bff4OTkpJbf/Pnztf6+BWnXrh3S0tI09i9btgxdunRR29ekSRPcv38fTZo0QW5uLlJTU0XHr8qoUaMwatQoAMCnn36KJk3yZSdyc3OxZ88eXL58We2czMzMIn//sqDQPgqFU9hLRHmKxeidxEGV9UcAqpeXIVWUvLw8PHjwQO1PZ2trC0dHR72/Touic+fOWLdunXg95dBaVVq3bo1Tp07h2bNnyMvLw44dOxAQEICgoCBERkYiMjISrVq1QmpqKqytrWFiYoKtW7eqyZUPGjQIP/30E86cOSM6hm7dumHdunXIyRHiJMbGxiIjI0Pj+qmpqahXrx4sLCxw69YtXLhwAQDg6+uLU6dOITk5Gbm5ufjtt9/EcwIDA7F6dX78sMjIyELLoWXLlpg9ezaWLFkCAJg+fToWLVok1sLi4+Px1Vdf4eOPhWmms2fPxsSJE8XyevnypVrfCQDUqVMHTZo0EUdLZWVl4dWrV7Czs0NMTAyysrKQmpqKv/7SHc6mRYsWMDU1xcKFC8WahpOTE5KSkkRHoaxpFsTFxQX//POPuJ2WlgZra2vk5ORg+/btOq/ZrVs3rFq1SnR0V69eBYBCf19Vzpw5Iz4XqktBJwEAvXv3xs8//wwA2L17Nzp16qS1tqacF5SQkIA9e/ao1R6OHz8OZ2dnNecBCM+Tm1vRH2CljT4y4xcZY15lbkkx+f332yU6r5fKOu/ENiwpKSm4ceMGHj9+jISEBPFPa2JiUqqThL777juEhYXB3d0d3t7eWl841tbWWLRoETp27Ch2yvbpo6lZOWHCBPz888/w8/NDbGys2ldqYGAgTp8+jS5duqB6deGTY/To0XB1dYWXlxfc3Nwwbtw4rbWe7t27Izc3F1KpFPPmzRObxGxsbPDpp5+idevW6NKlC1xdXVG3bl0AQlOVskbg6uqK9evXF1kW48ePx+nTpxEXFwcPDw8sWbIEvXr1grOzM3r16oWlS5eKwy8//PBDdOzYET4+PnBzc0NAQAAsLCw08ty6dStWrlwJqVSKNm3a4PHjx2jatClCQkIglUoxZMgQeHp6apynysCBA7Ft2zaEhIQAECZR7t69GzNnzoRMJoOHh4fWjuV33nkHp0+fFrcXLlyI1q1bo2vXrnB2dtZ5vXnz5iEnJwdSqRRubm6YN28egMJ/35IyatQoPH/+HA4ODli+fLk4bPvRo0d49913xXTBwcFwdXVFr169sGbNGtSrV088FhoaqtHsBABhYWHo0aPHf7axuDBdlQTGmBkJk+auA3AB8C+ADAjjjYiIysV5NLTzoL4zT6Bbo8fo3794g69ykF+D+ArA7NI2rhJy8+ZNtTbhkpCdnY2EhASkpKQAACwsLGBnZ1cqf8rKSHp6OmrVqoXc3FwEBQXhgw8+EPsEOEBQUBCWLl0KR0fH8jbFoGRlZSEgIABnz54tMr6Ktv8tY+wyEZUo5k9hV7sIwAtA30LSlBumpsX/An1bZV37YEhOaUJEePLkCR49egS5XA4TExPY2NigUaNG5S4LUpGZP38+jh8/jszMTAQGBmp0sFd1Fi9ejMTExCrnKBISErB48eJyCcJV2BUZABDRvwaypViYPHlVrPQEwfMBgCOMPAaskZCXl4fHjx9DLpejXr16aNq0qdhEw9GNcmQORztOTk4andJVAUdHx3JzjoU5ioaMsWm6DhLR8jKwR28CfW2KlX6lyvpvOlNx/iu5ubkwMTGBiYkJzMzMYGdnB8YYF/DjcIyYwhyFKYBaqKBRQWuaF6/6NUVl3b10TeFAaGZ68eIF7t+/j0aNGuGtt94CALUOOg6HY5wU9rZNJKIFBrOkuBSjjVt1cOSh0rekypOZmYl79+6JQ17T09P1nsDF4XAqPkX2UVRYmtXVO+knKuvvlL4lVRa5XI7Hjx8jMTFRmOZvZoYmTZqgQYMG3ElwOJWIwvp0OxvMipJQU7+mp7sAvles+6Ciez/jQTkhSing16BBA0gkElhZWXEnUYEwNTWFh4cH3Nzc0KtXL3GIMgDcuHEDnTp1QsuWLeHo6IiFCxeqzbz+448/0KpVK7i4uGiVLK8IXL16FaNHjy5vMwpl0aJFcHBwgJOTE44ePao1jVLs0M3NDcOHD9eYexMREQFTU1NRGiYpKQndu3cvc9uV6HQURPTCYFaUIS1U1nlgov8OY1+AsS9QvfpXkEr3wNzcHE5OTmjevLlaMPcNGy6LaRn7AmPH/l6OVheOrtm4leH6NWvWRGRkJKKjo1G/fn2sWbMGAPD69Wv07t0bs2bNQmxsLK5du4bz589j7dq1AAQNpUmTJmHbtm24efMmoqOjC5X2KAnFkV/RxVdffYWPPvrIoNcsDjExMQgNDcWNGzdw5MgRTJgwQeP3lsvlGD58OEJDQxEdHQ07OztxZjcgPB8zZ85Uk4Vp2LAhrK2ti1QQLi2MdpTo8+dFD49VlfdaAMC/zKyp/BCR1ji8rq6uGqJ0pUl8fLxeMuMXL15EmzZt4OnpiTZt2uD2bWHmfl5eHqZPnw53d3dIpVKsWrUKgLr09a5duxAZGQk/Pz9IpVIEBQUhOTlZqz3apMHXrVuHTz7Jb+DcvHmz+PLatm0bfH194eHhgXHjxokviVq1auGzzz5D69atER4ejgULFogzoseOHSt+2UdEREAqlcLf3x8zZswQ5Rt0yZkXhr+/vyg5/ssvv6Bt27YIDAwEIEyCXL16tTiLeOnSpZgzZ44429nMzAwTJkzQyDM9PR0jR44Uy1cpOVKrVi0xze7du8WAO6ry3zNmzECzZs3UajkODg548uSJXpLqaWlpiIqKgkwmA6D7Gdi8eTMGDBiAXr16iff79ddfi2X3+eefi3nqkn4vKfv378egQYNQo0YNNG/eHA4ODuIzq+T58+eoUaMGWrZsCQDo2rWrmnTLqlWrEBwcjEaNGqmd17dv30JlS0qVksrOlteilBk/eTJOm7quuqyuysIpGTExMZSRkUExMTEUERFBwHy1RRfff39JLd2YMQdKdH19ZcZTU1PFADV//vkn9evXj4iI1q5dS/369ROPKaW9C0pfu7u708mTJ4mIaN68efS///1Pqz3apMGfPn1KLVq0ENN0796dzpw5QzExMdSzZ0/Kzs4mIqIPP/yQfv75ZyIiAkA7d+7UyJeIaOjQoXTggFBeEomEzp07R0REM2fOJIlEQkSkU868IG+88QYRCXLr/fv3pz/++IOIiKZOnUrffvutRnpLS0tKTU3VKkmujU8++UStrF68eKF2XSKiXbt20fDhw4lIU/578uTJtGnTJiIiunDhAnXu3JmI9JNUP3HihPg7E+l+Bn766SeysbERy/jo0aM0ZswYMYBVjx496NSpU0Sk/fctyJQpU7RKji9atEgj7cSJE0V5cyKiDz74gHbt2qWWRi6Xk62tLUVERIhl4ubmRkREDx48oPbt21Nubi4NHz5c7dwHDx6I6QpicJnxioqJSeHt4E9V1ruWrSmVlvT0dCQnJ4uidqpNS4ZEH5nx1NRUDB8+HHfu3AFjTBTlO378OMaPHy/OZlVV8VQK0qWmpiIlJQUBAQEAgOHDh2PAgAFabdEmDe7n5wd7e3tcuHABjo6OuH37Ntq2bYs1a9bg8uXL8PHxASA09yi/Ck1NTREcHCzmGxYWhqVLl+LVq1d48eIFJBKJqFjapk0bAMB7772HgwcFWUtdcuaqMubKa3p4eCA+Ph7e3t6ijDcVMiqtOH1Mx48fVwuko89waFX574EDB2LBggUYOXIkQkNDxd9EH0n1xMREUSYe0P0MAMJXuvK3P3bsGI4dOybqUaWnp+POnTto37691t+3QYMGavavWLFCv8KBfpLjjDGEhoZi6tSpyMrKQmBgoPi8TpkyBUuWLNEa/teQkuNG6SjMn1yCiXPhMxQbq6z/UbbmVEr27duHjz76CBs3boSVlRUaNWoEGxsbEMn0On/sWG+MHetdKrboIzM+b948dOzYEXv37kV8fDw6dOgAoPAXYlFaU/fv30evXoKM5Pjx4+Hs7KxVGhwQXni//vornJ2dERQUJMZ4GD58OBYtWqSRt7m5ufjnz8zMxIQJE3Dp0iU0bdoU8+fPL1JynHTImRdE2UeRmpqKnj17Ys2aNZg8eTIkEomauB4A3L17F7Vq1ULt2rUhkUhw+fJlsVmnMDu0la/qvsIkx/39/fHPP/8gKSkJ+/btw9y5cwHoJ6leUHJc1zNQ8JpEhNmzZ2PcuHFq+emSfi/I1KlTERYWprF/0KBBmDVLPVSPUnJcyYMHD8Q5Rqr4+/uLMS+OHTsmxsW4dOkSBg0aBAB49uwZDh8+DDMzM/Tt29egkuNG2Ufhf3EJ6mfp7gBU/XuNhjBzkKM/Dx8+xKBBg/DgwQNUr14dLi4usLW11fpVU1FITU2FjY0wW18ZghMQFF7Xr18vOpQXLzTHaNStWxf16tUT/6hbt25FQEAAmjZtKspJjx8/Xqc0OAD069cP+/btw44dO8Sv4s6dO2P37t2inPSLFy9w7949jesrX0ZWVlZIT08Xawn16tVD7dq1xeuofrnrK2eueo8rV67EsmXLkJOTgyFDhuDs2bM4fvw4AKHmMXnyZLGvZcaMGfjqq6/EF5ZcLsfy5ZpiDAWlz5V9O40bN8bNmzchl8s1QnyqwhhDUFAQpk2bBhcXF/HrXR9J9YKS47qegYJ069YNmzZtQnq6EI3m4cOHePr0aaG/ryorVqzQKjle0EkAguR4aGgosrKyEBcXhzt37qiFglWifEaysrKwZMkSjB8/HgAQFxeH+Ph4xMfHo3///li7dq2o/WVIyXGjdBTv1agBlxaagUCUqIovryp7cyoFOTk54hesjY0N/u///g8rV67Em2++aRQqr5988glmz56Ntm3bqo0qGT16NGxtbSGVSiGTyfDLL79oPf/nn3/GjBkzIJVKERkZic8++0wjjS5pcEB4qbu6uuLevXvii8DV1RVffvklAgMDIZVK0bVrV61xoC0tLTFmzBi4u7ujb9++YlMVIMTPHjt2LPz9/UFEouS4vnLmqnh6ekImkyE0NBQ1a9bE/v378eWXX8LJyQnu7u7w8fER43NLpVJ8++23GDx4MFxcXODm5qbV9rlz5yI5ORlubm6QyWTil/bixYvRs2dPdOrUSWukOlWUkuOqUfD0kVR3dnZGamqqONFT1zNQkMDAQLz33nvw9/eHu7s7+vfvj7S0tEJ/35IikUgQEhICV1dXdO/eHWvWrBE/uN59912x6ejrr7+Gi4sLpFIpevXqhU6dOhWZtyElx3XKjFdUGtp5UJJHE2DZVsBRe3uostJbE0DxpAOrJufPn8f48eMxY8YMvP/++2rHSkNmnFNylJLjQL5q6nff8YHeSlasWIHatWtX+LkUZUH79u2xf/9+rf1CpS0zbpQ1CjR+A6hedDPI1wYwxZh58eIFxo0bh7Zt2+L69etYu3Ztoe3iHMNz6NAhccLcmTNnxDZ8jsCHH36o1odVVUhKSsK0adMMpqVmnDWKX0OA1p9qPZ4MQNkolQ6g4jeaGB4iwrZt2/Dxxx8jKSkJ1apVwyeffII5c+ZodI7xGgWHY3wYMnBRBUb38D3VOZrcSWjy5MkTDB48WGxLDggIwLp167gz4HA4OjHOpqdCHIXu8RUcQOg4TUxMhJWVFTZv3oywsDDuJDgcTqEYZY3ip5+vIaBhMuzt1dvnXiK/8/ong1tVcfnzzz/h5eWFBg0aoEaNGti1axesra01JhJxOByONoyyRhFz8znS0rI09qt+F7+vcbTqkZiYiMGDByMwMBAzZ+ZHCXdzc+NOgsPh6I1ROgo5Ma2zQZWT2bujak+yy8vLw9q1a+Hs7CyOmXdycjLaEU2VXSpbF4MHD4ZUKtVbMkJViK80ISJMnjwZDg4OkEqluHLlitZ0r1+/RkBAQLmr8RbGkSNH4OTkBAcHB1EAsSAJCQno2LEjPD09IZVKcfjwYfFYVFQU/P39IZFI4O7uLk6W7NKli04hyUpBSUWiymuxspXR1PaBFHUyXk3w6opKomStMllVg8uXL5OPjw9BmKBOPXr0oLi4uBLnp01czNCoCswNGzaMvvzySyIShNvs7e3p6NGjRESUkZFB3bt3p9WrVxMR0fXr18ne3p5u3rxJREQ5OTm0Zs2aUrVNKUJX2iQmJpKtrW2xzlEtp9Lk0KFD1L17d5LL5RQeHk6+vr5a061evVqr0KAulKJ8hiI3N5fs7e3p33//paysLJJKpXTjxg2NdGPGjKG1a9cSEdGNGzfIzs6OiITf2t3dXRRLfPbsmShuuHnzZvG5rAiUtiigUdYoRtUwh52ludq+ISrrloY1p8IQHx8PX19fREREwMbGBr/99ht+//13NGvWrFTyZ2W0FIfKJpWdmZkpXtvT01McjRYYGIinT5/Cw8NDlBZR8uTJEwQFBUEmk0Emk+H8+fMa99O5c2d4eXnB3d0d+/fvBwBkZGSgR48ekMlkcHNzw86dOwEAs2bNgqurK6RSqdYa1/79+zFs2DAwxuDn54eUlBSts7S3b9+OPn36FGpDfHw8XFxcMGHCBHh5eeH+/fs4duwY/P394eXlhQEDBojSGrqk10vKxYsX4eDgAHt7e1SvXh2DBg0S7VKFMYaXL4UAyqmpqaI207Fjx8QZ/gDQoEEDcZZ17969sWPHjv9kX4WmpB5GnwVCK9BtAP8AmKXl+DQAMQCiAPwFwK6oPK1sZUTdg4n+za83pKskGFoyB1xpGD16NE2dOpVevnxZKvmpfpmU1YNSFJVZKnvZsmU0YsQIIiK6efMmNW3alF6/fk1xcXGipHhBQkJCaMWKFWKZpKSkqNmbk5NDqampRESUlJRELVq0ILlcTrt376bRo0eL+aSkpNDz58+pZcuWJJfLiYgoOVmzPt6jRw/xPoiIOnXqJEpiK8nKyqLGjRuL27psiIuLI8YYhYeHi8fatWtH6enpRES0ePFi+uKLL4hIt/S6Ktu2bdMq+R0cHKyRdteuXTRq1Chxe8uWLTRx4kSNdI8ePSI3NzeysbEhS0tLunTpEhERrVixgoYOHUqBgYHk6empJlNPROTg4KBVlrw8MBqZccaYKYA1EFS+HwCIYIwdIKIYlWRXAbQioleMsQ8BLAUwUDO3AhT4sIhSWd/038w2KuLj4/HRRx9h+vTpokT2hg0byiwUaXn1cFRmqeyzZ8+KQY6cnZ1hZ2eH2NhY1KlTR+e1T5w4gS1btgAQ+m+U+k9KiAiffvopTp8+DRMTEzx8+BBPnjyBu7s7pk+fjpkzZ6Jnz55o164dcnNzYW5ujtGjR6NHjx7o2bOnxvVIy5d8wfJ99uwZLC3z6/K6bAAAOzs7UUfpwoULiImJQdu2bQEA2dnZ8PcXQoxpk15XqvkqGTJkCIYMGQJ90Oc+AGDHjh0YMWIEPv74Y4SHh+P9999HdHQ0cnNzcfbsWURERMDCwgKdO3eGt7c3OncWokYrZb8r40CRsmx68gXwDxHdJaJsAKEA+qgmIKIwIlKOaL0AoIleOQe1BKzyZxD/qXKofCImGJacnBwsWbIErq6uOHjwoJpqZWWMV62Uyr537x6ys7PFcJ4SiQSXLl1SS6tNKrsodDmckkpl9+vXD0C+VLZSXfThw4ca0QC1vbz+K9u3b0dSUhIuX76MyMhING7cGJmZmWjZsiUuX74Md3d3zJ49GwsWLICZmRkuXryI4OBg7Nu3T2scZn2ksgtKfuuyAdCU/O7atatYRjExMfjxxx9F6fXdu3fj+vXrGDNmjFbJ7+3bt8PDw0Nj6d+/f4nuAxCEGENCQgAIv21mZiaePXuGJk2aICAgAFZWVrCwsMC7776r1rFvSNlvQ1OWjsIGwH2V7QeKfboYBR2hIxhjYxljlxjRvo37AAAbD0lEQVRjwlvBsy5QJ1/fRRnIsI+WcysbZ8+ehaenJ2bNmoXXr19j0KBB2LNnT3mbZRAqo1R2+/btxXCWsbGxSEhIgJOTU6Hl0LlzZ6xbtw6AMMJN2Z6uJDU1FY0aNUK1atUQFhYmSps/evQIFhYWGDp0KKZPn44rV64gPT0dqampePfdd/Htt99qtbF3797YsmULiAgXLlxA3bp1NRRh69Wrh7y8PPFlrsuGgvj5+eHcuXOiXPirV68QGxurU3q9IEOGDNEq+a0tvY+PD+7cuYO4uDhkZ2cjNDQUvXv31khna2uLv/76C4AghZGZmYmGDRuiW7duiIqKwqtXr5Cbm4tTp07B1dUVgODwHj9+XGr9gRWOkrZZFbUAGADgB5Xt9wGs0pF2KIQaRY2i8rWylRH9nd82+K/KwV/0ar0zTl68eEGjRo0SRzO1aNFCHO1TllS0UU9ERD179qQtW7YQEVFUVBQFBARQy5YtqUWLFjR//nyxvZ2I6PfffycvLy9ydnYmFxcXmj59ukb+aWlpNGzYMJJIJCSVSum3334jIqFN297engICAmjixIlqfRQFw1kKYWJBmzdvFvclJSVRSEgIubu7k4uLC40bN07j2q9fv6bhw4eTm5sbeXh40IkTJ4iICu2jePz4MfXu3Zvc3NxIJpPR+fPn1copKSmJ/Pz8yNvbm0aNGkXOzs4UFxdHR44cIXd3d5LJZNSqVSuKiIigR48ekY+PD7m7u5Obm5ua/UrkcjlNmDCB7O3tyc3NTaN/QskHH3xAf/75Z6E2aLuvv/76i1q1akXu7u7k7u5O+/fvJyKiOXPmUIsWLahz5840YsQI+vzzz7VetzgcOnSIHB0dyd7eXm2U0rx588Tr3rhxg9q0aUNSqZRkMpna/2zr1q3k6upKEomEZsyYIe6PiIhQC8ta3pR2H0VZOgp/AEdVtmcDmK0lXRcANwE00idfK1sZvTi1knJyhGF1PVQOVmaePXtGVlZWVK1aNZo3bx69evXKINetCI6CYxxcuXKFhg6tmsNJJk+eTMePHy9vM0SMaXhsBABHxlhzxlh1AIMAHFBNwBjzBPA9gN5E9FRLHlr5ePpx/PuvEKnskGKf7jBGxsutW7eQlSXMQG/QoAG2b9+OqKgoLFiwoNK2hXKMF09PT3Ts2LFCT7grK9zc3MRO7cpImTkKIsoFMAnAUQg1hl+J6AZjbAFjTNkw+DWAWgB2McYiGWMHdGSnRh4JZmer7DtVapaXP69evcKcOXMglUqxdOlScX9gYKA4J4DDqYh88MEHFTpkblkxZsyY8jahTClTUUAiOgzgcIF9n6msdylJvnK5IOERq7LPMJFjy54jR45gwoQJiIuLAyAMO+RwOJzyxChnZtdhJjB7kgHlPNfahaY2Dh49eoSQkBC88847iIuLg7u7O86dO8fDXnI4nHLHKGXG11jUAuwssUqx3aJcrfnvxMbGolWrVkhLS4OFhQXmz5+PKVOmoFq1qjArhMPhVHSM0lGAhIlQyg7sd8rPklLB0dERPj4+eOONN7Bq1SrY2dmVt0kcDocjYpRNT1B0Ziul0jSnzFRsXr58iSlTpogTwRhjOHDgAA4cOMCdhBa4zHj5yozfunUL/v7+qFGjBpYtW6YzHRGhU6dOGhMAKxLKmekODg6YPHmy1pnxycnJCAoKglQqha+vL6KjowEA9+/fR8eOHeHi4gKJRKLWLDx9+nScOHHCYPdhcEo6rra8FitbGdHt3ZQjl4s7k4o3xLjckMvl9Ouvv5K1tTUBoG7dupW3SUVSEeZRcJlx/SgrmfEnT57QxYsX6dNPP6Wvv/5aZ7qDBw/SlClTipW3UljRUPj4+ND58+dJLpdT9+7d6fDhwxpppk+fTvPnzyciQaixU6dORCSIBV6+fJmIiF6+fEmOjo6iTHl8fDx17drVQHdRNMY0j6LsYCb4TUWHx6ocTdGXu3fvokePHggJCUFiYiL8/PywZMmS8jareHzDymYpBlxm3PAy440aNYKPj0+RfWaqMuMA0LdvX3h7e0MikWDDhg3i/lq1auGzzz5D69atER4ejsuXLyMgIADe3t7o1q2bKGG+ceNG+Pj4QCaTITg4GK9evdK4ZnFITEzEy5cv4e/vD8YYhg0bhn379mmki4mJEedEODs7Iz4+Hk+ePIG1tTW8vLwAALVr14aLi4v4LNrZ2eH58+d4/Pjxf7KxomKcfRTMRBSFquieLjs7G8uWLcPChQuRmZkJS0tLLF68GGPGjIGJSUW3vmKRl5eHv/76C6NGjQIgNDt5e3urpWnRogXS09Px8uVLREdH4+OPPy4y34ULF6Ju3bq4fv06AOgVqSw2NhbHjx+HqampqAU1cuRI/P3332jWrBkaN26M9957D1OnTsXbb7+NhIQEdOvWDTdv3lTLRylweP36ddy6dQuBgYGIjY3FgQMH0LNnT63aS5MnT0ZAQAD27t2LvLw8MX6DEnNzc+zduxd16tTBs2fP4Ofnh969e+PIkSN46623cOiQME01NTUVL168wN69e3Hr1i0wxtQcXnE5d+4cvv/+e3F706ZNqF+/Pl6/fg0fHx8EBwejQYMGyMjIgJubGxYsWICcnBwEBARg//79aNiwIXbu3Ik5c+Zg06ZN6Nevnzg/Ye7cufjxxx9FpV0lYWFhmDp1qoYtFhYWGg704cOHaNIkX3e0SZMm4oteFZlMhj179uDtt9/GxYsXce/ePTx48ACNGzcW08THx+Pq1ato3bq1uM/Lywvnzp1DcHBwMUuu4mOUjuLXHZdx4dNegKkJepS3MUVw//59LFiwAFlZWRgyZAi++eYbtQfOqPi4fITGucy4OoaWGdeXFy9eqN3bypUrRTHF+/fv486dO2KwH+XL9Pbt24iOjhZ/07y8PFFwMDo6GnPnzkVKSgrS09PRrVs3jWt27NhRqzPVBmnpj9D2nMyaNQv/+9//4OHhIdb0zMzyX5Xp6ekIDg7Gt99+q/Y7KWXGKyNG6Si2hd7B7XnC1/iAcrZFG8nJybC0tARjDC1atMB3330HBweHSj3FvyxRyoynpqaiZ8+eWLNmDSZPngyJRILTp0+rpdUmM66MSKYLXQ6npDLjc+fOBZAvM16Y3Iq2l9d/RVXiu1q1amjWrJmazPjhw4cxe/ZsBAYG4rPPPsPFixfx119/ITQ0FKtXry5xp6yZmRnkcjlMTExw8uRJHD9+HOHh4bCwsECHDh3EMjQ3NxedLBFBIpEgPDxcI78RI0Zg3759kMlk2Lx5M06ePKmRpjg1iiZNmuDBgwfiti6Z8Tp16uCnn34S7WvevDmaN28OQJD4Dw4OxpAhQ0Q5eSVcZryC8dolv/rYq5B0hkYul2PTpk1wcHDAtm3bxP3jxo3jTqIU4DLjAoaWGdcXJycn3L17V7ShXr16sLCwwK1bt3DhwgWd5yQlJYmOIicnBzdu3AAApKWlwdraGjk5OWIZFURZoyi4FHQSAGBtbY3atWvjwoULICJs2bJFrU9FSUpKCrKzBYGgH374Ae3bt0edOnVARBg1ahRcXFwwbdo0jfNiY2Ph5lZZNCIKUNJe8PJarGxl1HHpygqnGBsdHU3t2rUTZcAHDx5c3iaVChVt1BMRlxk3tMx4YmIi2djYUO3atalu3bpkY2MjhjlVZcGCBbRx40YiIsrMzKTu3buTu7s79e/fnwICAigsLEzNTiVXr16ldu3akVQqJVdXV9qwYQMREa1du5aaNWtGAQEBNGnSJLH8/wsREREkkUjI3t6eJk6cKD4r69ato3Xr1hER0fnz58nBwYGcnJwoKChIDI175swZAiCWoUwmo0OHDhERUXZ2Njk7O5fZKLjiYjQy42W1WNnKaMChXwlE1KekpViKZGRk0KxZs8jMzIwAUKNGjWj79u1qLytjpiI4Co5x8OjRI+rSpUt5m1Eu7Nmzh+bOnVveZogYTczssiTPWui88yhnO2JjY9GtWzfEx8eDMYbx48fjq6++0qszlMOpbFhbW2PMmDF4+fJloZ3xlZHc3Fy9RtgZK0bpKB41FkZWlPeIJzs7O5ibm0Mmk2H9+vViwHgOp6qijDVd1RgwoCIOqyk9jLIzO48JIyYMXaPIzc3F6tWr8fz5cwBAjRo1cOTIEVy6dIk7CQ6HU2kxTkdhIjgKQ2qrXrx4Eb6+vvjoo48wc+ZMcb+dnZ3aGGsOh8OpbBilozCV52GQga6VmpqKSZMmwc/PD1evXoWtra3WIXUcDodTWTFKR/GqmgUalPE1iAihoaFwdnbGmjVrYGpqik8++QQxMTHo1asizd7gcDicssUoHYWcmSDtVU6ZXuPatWsYPHgwHj9+jDZt2uDKlStYsmSJ2oxcjmHgMuPlKzO+fft2SKVSSKVStGnTBteuXdOajqhyy4xnZmbC19cXMpkMEokEn3/+uXjOoEGDcOfOHYPdh8Ep6bja8lqsbGXk/CyG9j/LKNH44sIoKHk8depU2rhxI+Xl5ZX6tYyFijCPgsuM60dZyYyfO3dOnHR2+PBh8vX11ZqussuMy+VySktLIyJhgp2vry+Fh4cTEdHJkydp9OjRBrqLouHzKAAQGJrKS1cjJywsDBMmTMD333+P9u3bA4BWuYeqzJi1L8ok340T6hedSIG/vz+ioqIA6JYZ79ChAyZOnFgsmfGPPvoIly5dAmMMn3/+OYKDg1GrVi1RmXX37t04ePAgNm/ejBEjRqB+/fq4evUqPDw8sHfvXkRGRsLS0hKAIDN+7tw5mJiYYPz48UhISAAAfPvtt2jbtq3atTMzM/Hhhx/i0qVLMDMzw/Lly9GxY0c1mfFVq1ahXbt24jlPnjzB+PHjRbmMdevWoU2bNmr306dPHyQnJyMnJwdffvkl+vTpg4yMDISEhODBgwfIy8vDvHnzMHDgQMyaNQsHDhyAmZkZAgMDNYITqebt5+enppekyvbt2zF27Fhxu2/fvrh//z4yMzPxv//9TzxWq1YtTJs2DUePHsU333yDmjVrYtq0aUhPT4eVlRU2b94Ma2trbNy4ERs2bEB2djYcHBywdetWWFhYaH8w9EBVZhyAKDP+zjvqMTJjYmIwe/ZsAOoy440bNxZrbTk5OcjJyRH1wNq1a4cRI0YgNze3Ug5uMco7kjMTvPX8NdDwvzcDPX36FDNmzBDVOJcvXy46Ck7FgsuMC5SnzPiPP/6o8WJVUhVkxvPy8uDt7Y1//vkHEydOFGXGTUxM4ODggGvXrmk8k5UBo3QUxBjqvVH9P+Uhl8vx448/YubMmUhOTkaNGjUwd+5czJgxo5SsrHwU58u/NOEy4+qUl8x4WFgYfvzxR5w9e1br8aogM25qaorIyEikpKQgKCgI0dHRohCgUmacO4oKguXDdFRvqCkPrC9xcXEYOnSo+MURGBiINWvWwMHBobRM5JQiXGa8eJSFzHhUVBRGjx6NP/74Q1THLUhVkBlXYmlpiQ4dOuDIkSOio+Ay4xWMF2/VAYrxxViQOnXqIDY2Fm+++SZCQ0Nx5MgR7iSMAC4zLmBomfGEhAT069cPW7duRcuWLXXaVdllxpOSksSmudevX+P48eNi/xcg/H4SiURn+Rg1Je0FL6/FylZGM+JjiLKLN1riyJEjlJmZKW6fP3+eUlJSipVHVaSijXoi4jLjhpYZHzVqFFlaWorS2t7e3lrtquwy49euXSMPDw9yd3cniURCX3zxhZjv48ePycfH5z/bV1pwmXFbGc1PidO7wBISEqhv374EgBYuXKj3eRyBiuAoOMZBVZYZX758Of3www/lbYZIaTsKo2x6esKKNjs3NxfLly+Hi4sL9u3bh1q1aqF+/fLpjOVwqgKqMuNVDUtLSwwfPry8zSgzjLIz2xmF909cuHAB48ePF2eQBgcH47vvvoONjY0hzONwqixVVWZ85MiR5W1CmWKUjqJGIR3Zf//9N9q0aQMiQrNmzbB69Wr06FHekSuMGypkGCqHw6lYUBmMpDNKR5FVSNOTr68vunXrBk9PT8ydO/c/zeTkCEMZnz9/jgYNGnBnweFUcIgIz58/h7m5eanma5SOoumDNKBFHlDNFHfu3MHUqVOxfPlytGzZEowxHDp0CCYmRtn9UuFQjj1PSkoqb1M4HI4emJubq81ALw2M0lE0mXUaWWsbY/HG77Bo0SJkZWXB3Nwcu3fvBgDuJEqRatWqaUw24nA4VYsydRSMse4AvgNgCuAHIlpc4HgNAFsAeAN4DmAgEcUXle+1BzEY2t4Xsf8Ksr4jR47E0qVLS9l6DofD4QAAK4uODwBgjJkCiAXQFcADABEABhNRjEqaCQCkRDSeMTYIQBARDSwsX/M36lPWK2HmrIuLC9avX89F/DgcDqcIGGOXiahVSc4tyzYaXwD/ENFdIsoGEAqg4Hz5PgB+VqzvBtCZFdFjmvUqBeam1fDVZwsQGRnJnQSHw+GUMWVZo+gPoDsRjVZsvw+gNRFNUkkTrUjzQLH9ryLNswJ5jQWgFLp3AxBdJkYbH1YAnhWZqmrAyyIfXhb58LLIx4mIahedTJOy7KPQVjMo6JX0SQMi2gBgAwAwxi6VtPpU2eBlkQ8vi3x4WeTDyyIfxtilkp5blk1PDwA0VdluAuCRrjSMMTMAdQGUTRg1DofD4ZSIsnQUEQAcGWPNGWPVAQwCcKBAmgMAlAIp/QGcoLJqC+NwOBxOiSizpiciymWMTQJwFMLw2E1EdIMxtgCCiuEBAD8C2MoY+wdCTWKQHllvKCubjRBeFvnwssiHl0U+vCzyKXFZlFlnNofD4XAqB3wKM4fD4XAKhTsKDofD4RRKhXUUjLHujLHbjLF/GGOztByvwRjbqTj+N2OsmeGtNAx6lMU0xlgMYyyKMfYXY8yuPOw0BEWVhUq6/owxYoxV2qGR+pQFYyxE8WzcYIz9YmgbDYUe/xFbxlgYY+yq4n/ybnnYWdYwxjYxxp4q5qhpO84YYysV5RTFGPPSK+OShsYrywVC5/e/AOwBVAdwDYBrgTQTAKxXrA8CsLO87S7HsugIwEKx/mFVLgtFutoATgO4AKBVedtdjs+FI4CrAOopthuVt93lWBYbAHyoWHcFEF/edpdRWbQH4AUgWsfxdwH8AWEOmx+Av/XJt6LWKMpE/sNIKbIsiCiMiF4pNi9AmLNSGdHnuQCAhQCWAsg0pHEGRp+yGANgDRElAwARPTWwjYZCn7IgAHUU63WhOaerUkBEp1H4XLQ+ALaQwAUAlowx66LyraiOwgbAfZXtB4p9WtMQUS6AVAANDGKdYdGnLFQZBeGLoTJSZFkwxjwBNCWig4Y0rBzQ57loCaAlY+wcY+yCQs25MqJPWcwHMJQx9gDAYQAfGca0Ckdx3ycAKm48ilKT/6gE6H2fjLGhAFoBCChTi8qPQsuCMWYCYAWAEYYyqBzR57kwg9D81AFCLfMMY8yNiFLK2DZDo09ZDAawmYi+YYz5Q5i/5UZE8rI3r0JRovdmRa1RcPmPfPQpCzDGugCYA6A3EWUZyDZDU1RZ1IYgGnmSMRYPoQ32QCXt0Nb3P7KfiHKIKA7AbQiOo7KhT1mMAvArABBROABzCIKBVQ293icFqaiOgst/5FNkWSiaW76H4CQqazs0UERZEFEqEVkRUTMiagahv6Y3EZVYDK0Co89/ZB+EgQ5gjFlBaIq6a1ArDYM+ZZEAoDMAMMZcIDiKqhjf9wCAYYrRT34AUokosaiTKmTTE5Wd/IfRoWdZfA2gFoBdiv78BCLqXW5GlxF6lkWVQM+yOAogkDEWAyAPwAwiel5+VpcNepbFxwA2MsamQmhqGVEZPywZYzsgNDVaKfpjPgdQDQCIaD2E/pl3AfwD4BWAkXrlWwnLisPhcDilSEVteuJwOBxOBYE7Cg6Hw+EUCncUHA6HwykU7ig4HA6HUyjcUXA4HA6nULij4FQ4GGN5jLFIlaVZIWmb6VLKLOY1TyrUR68pJC+cSpDHeMbYMMX6CMbYWyrHfmCMuZaynRGMMQ89zpnCGLP4r9fmVF24o+BURF4TkYfKEm+g6w4hIhkEscmvi3syEa0noi2KzREA3lI5NpqIYkrFynw710I/O6cA4I6CU2K4o+AYBYqawxnG2BXF0kZLGglj7KKiFhLFGHNU7B+qsv97xphpEZc7DcBBcW5nRQyD6wqt/xqK/YtZfgyQZYp98xlj0xlj/SFobm1XXLOmoibQijH2IWNsqYrNIxhjq0poZzhUBN0YY+sYY5eYEHviC8W+yRAcVhhjLEyxL5AxFq4ox12MsVpFXIdTxeGOglMRqanS7LRXse8pgK5E5AVgIICVWs4bD+A7IvKA8KJ+oJBrGAigrWJ/HoAhRVy/F4DrjDFzAJsBDCQidwhKBh8yxuoDCAIgISIpgC9VTyai3QAuQfjy9yCi1yqHdwPop7I9EMDOEtrZHYJMh5I5RNQKgBRAAGNMSkQrIWj5dCSijgopj7kAuijK8hKAaUVch1PFqZASHpwqz2vFy1KVagBWK9rk8yDoFhUkHMAcxlgTAHuI6A5jrDMAbwARCnmTmhCcjja2M8ZeA4iHIEPtBCCOiGIVx38GMBHAagixLn5gjB0CoLekORElMcbuKnR27iiucU6Rb3HsfAOCXIVqhLIQxthYCP9rawgBeqIKnOun2H9OcZ3qEMqNw9EJdxQcY2EqgCcAZBBqwhpBiYjoF8bY3wB6ADjKGBsNQVb5ZyKarcc1hqgKCDLGtMY3UWgL+UIQmRsEYBKATsW4l50AQgDcArCXiIgJb2297YQQxW0xgDUA+jHGmgOYDsCHiJIZY5shCN8VhAH4k4gGF8NeThWHNz1xjIW6ABIV8QPeh/A1rQZjzB7AXUVzywEITTB/AejPGGukSFOf6R9T/BaAZowxB8X2+wBOKdr06xLRYQgdxdpGHqVBkD3Xxh4AfSHESNip2FcsO4koB0ITkp+i2aoOgAwAqYyxxgDe0WHLBQBtlffEGLNgjGmrnXE4ItxRcIyFtQCGM8YuQGh2ytCSZiCAaMZYJABnCCEfYyC8UI8xxqIA/AmhWaZIiCgTgrrmLsbYdQByAOshvHQPKvI7BaG2U5DNANYrO7ML5JsMIAaAHRFdVOwrtp2Kvo9vAEwnomsQ4mPfALAJQnOWkg0A/mCMhRFREoQRWTsU17kAoaw4HJ1w9VgOh8PhFAqvUXA4HA6nULij4HA4HE6hcEfB4XA4nELhjoLD4XA4hcIdBYfD4XAKhTsKDofD4RQKdxQcDofDKZT/B7kZbW7NRen8AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "roc_auc(y_val, y_val_predicted_scores_mybag,len(y_val[1]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "XwNpUEVLTGA2"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "9S3Ddz7LTGA3"
   },
   "source": [
    "## Task 4 (MultilabelClassification) - Optional \n",
    "** Once we have the evaluation set up, we suggest that you experiment a bit with training your classifiers. We will use *F1-score weighted* as an evaluation metric. Our recommendation:\n",
    "- compare the quality of the bag-of-words and TF-IDF approaches and chose one of them.\n",
    "- for the chosen one, try *L1* and *L2*-regularization techniques in Logistic Regression with different coefficients (e.g. C equal to 0.1, 1, 10, 100).\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Ers40KuOTGA4"
   },
   "outputs": [],
   "source": [
    "######################################\n",
    "######### YOUR CODE HERE #############\n",
    "######################################"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "mdqTO269TGA8"
   },
   "source": [
    "When you are happy with the quality, create predictions for *test* set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "M51F10f9TGBB"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "Stats_NLP_Project_Questions.ipynb",
   "provenance": [],
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}