Multi-Label-Classification-of-Pubmed-Articles.ipynb

{"cells":[{"source":"<a href=\"https://www.kaggle.com/code/owaiskhan9654/multi-label-classification-of-pubmed-articles?scriptVersionId=148000148\" target=\"_blank\"><img align=\"left\" alt=\"Kaggle\" title=\"Open in Kaggle\" src=\"https://kaggle.com/static/images/open-in-kaggle.svg\"></a>","metadata":{},"cell_type":"markdown"},{"cell_type":"markdown","id":"5e1cfb6a","metadata":{"papermill":{"duration":0.017444,"end_time":"2023-10-25T18:37:42.05241","exception":false,"start_time":"2023-10-25T18:37:42.034966","status":"completed"},"tags":[]},"source":["# **<span style=\"color:#F32321;\">Connect on Linkedin if you have any doubts</span>** - [Contact](https://www.linkedin.com/in/owaiskhan9654/)"]},{"cell_type":"markdown","id":"8ff8220f","metadata":{"papermill":{"duration":0.01509,"end_time":"2023-10-25T18:37:42.083371","exception":false,"start_time":"2023-10-25T18:37:42.068281","status":"completed"},"tags":[]},"source":["\n","\n","# <p style=\"background-color:#1a0a36;font-family:newtimeroman;color:#FFF9ED;font-size:120%;text-align:center;border-radius:10px 10px 10px 10px;\">MultiLabel Classification of PubMed Articles using Deep Learning</p>\n","## This Notebook Got Selected in November 2022 Kaggle ML Research Spotlight🎉\n","\n","<img src=\"https://gitlab.com/Owaiskhan9654/Multi-Label-Classification-of-Pubmed-Articles/-/raw/main/Kaggle%20ML%20Research%20Spotlight%20Winners2.JPG\" height=200px width=700px>\n","\n","Read Announcements [Here](https://www.kaggle.com/discussions/general/370095) and [Here](https://www.kaggle.com/kaggle-ml-research-spotlight-winners). \n","\n","\n","<center><a href=\"https://www.kaggle.com/code/owaiskhan9654/multi-label-classification-of-pubmed-articles\"><img src=\"https://gitlab.com/Owaiskhan9654/Multi-Label-Classification-of-Pubmed-Articles/-/raw/main/images/presentation_gif.gif\"></a></center>\n","\n","\n","1. The traditional machine learning models give a lot of pain when we do not have sufficient labeled data for the specific task or domain we care about to train a reliable model.\n","\n","2. Transfer learning allows us to deal with these scenarios by leveraging the already existing labeled data of some related task or domain. We try to store this knowledge gained in solving the source task in the source domain and apply it to our problem of interest.\n","\n","3. In this work, I have utilized Transfer Learning utilizing **BIO BERT** model and Default **BERT-BASE Uncased**. \n","\n","4. Also Applied **Roberta For Sequence Classification** and **XLNet For Sequence Classification** models class for Fine-Tuning the Model. \n","\n","5. All the model performance for comparision has been logged to Weight and Biases. Check them out [here](https://wandb.ai/owaiskhan9515/Multi%20Label%20Classification%20of%20PubMed%20Articles%20(Paper%20Night%20Presentation)?workspace=) \n","\n","6. Model upload to Hugging Face Hub <img src=\"https://huggingface.co/front/assets/huggingface_logo-noborder.svg\" style=\"Height: 18px;width:18px\"> [Link](https://huggingface.co/owaiskha9654/Multi-Label-Classification-of-PubMed-Articles)\n","<br>\n","\n","7. This Model has been Connected to a Live application which is Build using Gadio and runnong on HuggingFace Spaces. All the code used to make it live is present in this notebook only:). Check it out [here](https://huggingface.co/spaces/owaiskha9654/Multi-Label-Classification-of-Pubmed-Articles)\n","\n"," <a id='top'></a>\n"," <div class=\"list-group\" id=\"list-tab\" role=\"tablist\">\n"," <p style=\"background-color:#1a0a36;font-family:newtimeroman;color:#FFF9ED;font-size:200%;text-align:center;border-radius:10px 10px;\">TABLE OF CONTENTS</p>   \n","    \n","* [1. IMPORTING LIBRARIES](#1)\n","    \n","* [2. LOADING DATA](#2)\n","    \n","* [3. DATA VISUALIZATION](#3)\n","    \n","* [4. Tokenizations](#4)   \n","    \n","* [5. Creating the Data Loaders](#5) \n","      \n","* [6. Loading the pretrained model](#6)\n","    \n","* [7. Training the model](#7)\n","    \n","* [8. Visualizing The results](#8)  \n","    \n","* [9. Evaluating the model](#9)\n","    \n","* [10. Classification Report](#10)\n","    \n","* [11. References](#11)\n"]},{"cell_type":"markdown","id":"9bb9afcb","metadata":{"papermill":{"duration":0.015441,"end_time":"2023-10-25T18:37:42.113975","exception":false,"start_time":"2023-10-25T18:37:42.098534","status":"completed"},"tags":[]},"source":["<center><h1 style = \"font-size:25px;font-family: Comic Sans MS\"> Firstly installing the Transformers Library and GitHub Large file system to push code to GitHub and Model to Huggingface Platform </h1></center>\n","\n","\n","\n","\n","- [GitHub Code Link](https://github.com/Owaiskhan9654/Multi-Label-Classification-of-Pubmed-Articles) <img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAHoAAAB6CAMAAABHh7fWAAAAaVBMVEX///8AAABJSUn19fX5+fnLy8vDw8Pj4+Pq6urc3NzR0dE5OTk0NDTx8fHU1NR7e3u3t7eGhoawsLBycnIkJCQpKSmZmZkvLy8dHR0LCwuMjIxgYGBUVFShoaFPT0+9vb1CQkJpaWkWFhZ285W/AAAFlElEQVRogdVb69qyKhCV8piZSb2Wlpnd/0V+kVoiMwg6tZ+9fqqxFIY1J3Kc/xfcMPL8Pa+e4Hvfi0L3F6zhhmer7bFgAxTH7Srjm/CLtMkuu/4xFH/XbJd8g3ddpxraN31ar4mJd2kxzdtNf7ojJOZHU94WR07Dm+TGHzz49Hz5qrvV2Z5Y4Fwt3HH7eB6xQLxfQBxc5hMLXIK5zPkyYoF8FnG4YK4/iGdoHKcgFrDdaO6Bipmxg5WpBzc6ZsZuFta2O1EyM3YyltYNLbHAxozZo2dmzDNh9r/BzJj/nzEbcH9ltltMzHk000+Z4BzpmF0S8cQQ67Tl+k1mxq44M4Gr0gN1ZIqUnNMymyupxaXM08f4KiItiSKfqbi83jf2vKn3Wlfllyc4ZFOdVb8dgtJK1O9V76Rr5d4BYla15Ph5xYQb235cfww5VINZQFnWW+WpZnjfHQfjRfE4nx/FaPBRIKoGd1s1O8nU9x/Zo5t215+53X4ThesnibsOo83+mQF2t9LR1i3VUbMxcwRMnaJ8UXwvayS2T+ryHit6BXmE8UNQ2EuQNobAsBf5kR3wCCPIGdfQuHLMAiooQa3AhcaV9BRaaRJq8Kul1YZD329NeKuSLSBjeIKgODI5MrD7BOrl1KqSvlD29wEhe0HZ/PYAhErgLWlYJHhdbGcuFnv0St4g9xXdsQa8c9jbPYRYUWq1WM6SOzL0X2toWPxbUFg4VgJq/QOWzxrmSXpg+VsbMiD2nU4MaogUHn0r7oGe4zndRAXHBJlyYcJ7+Na8AgwAJMIWlS14Rh5kJeZQiYhfEOsJB5tg5DgPsBmfnoIDrwWJebeAjbxwEcGJCWvqCRxHR4iAX6ZHNAdc7vSR0hyZfQvANs6R6waVD3PAE5sje4uyh4CIVuo04PXZ1WQIAUjROLD5/YA6RhTlB9QnBy4Z/YD67MBi9gPqwgEv/4KaObBfIZRwTMQfDty4+4GkHJHwqKKkrkCK2IHDVaLArAUsmCvErWhKivaAU5AL8kpbSn8Nr2mKLASl/0BC3gprpRHaGfJxHEsz73TUSN7lo8ng4iyzR4TmAEhEShciIfP9jEgRh/10LERAzjnEDpqPsSU99wGQvOolWli3mOizsbqB6C0j245ofyEr3QkHkl+TOG3EVXf5NbrY36wgdf4J72IuTjfx3n8bEIT4MZ+FvhNnPnbpe4M+wS4L5hyf7U9bRde91fdAdUB3jkDf28BqpC2yWfWcBDVegfg9JlIZ7nCsrMmTXN+MfleGpd3XeJGXjxqZp9yqqBOWU+fxBooxMMU2PNmMu6iHOjGyODfkzQSvvGkHTvvWMagKeEv5hL4FPDVq/EqmO9gHPTccQ/zdEGnPb8anAOVwV9oJXMd9R2xubX7WYBRySsLTdR4gd1qgc44FQgrG8iy7mM74G/V3mrjJ9LiB8vLy3m7VPVA/RLPLkMbSGKXyQzlH6Br7SnSj9WVGZ9SgvEZW8gM0FxNH5owO5oFncmSabu+NFlC7sdGQZAB1ugVcacr7qfWHoqgvnYJtWxlbRBJlN9cblMuvouhR3C/5VLA2LSpoFikt1mAbhUEQhAbua1JVNKYi2eg01RgrjLKDdn8M3xu2iAXUsfbHUjfOOvXRU091CyUBs80/tNS4+veQ6muWR5211AZVQCkjKA42/6nQURsVAUex8enS5Lz2PH+fNxPpiIba6Byp7vTsfS61cc0VbbavZlJb5BEJMsh9FrVd99+Fj1bMorY+6AE6X70ewdQz/n4SAQPZU6/mpYtqnGdNPbv6thuPtbWjXi0p8tZyxniyoT4vPcNUDfsyf+bUD4La13pwhHSC+pPrxZzmgIdb99lgo3+wryRca8L/EUbZK1yd8KGv/GObkdWzO7h+xSf9Z8gr/yd/nFyIfyAPRSrI9IUxAAAAAElFTkSuQmCC\" style=\"Height: 45px;width:45px\"> \n","\n","\n","- [Model Link](https://huggingface.co/owaiskha9654/Multi-Label-Classification-of-PubMed-Articles)<img src=\"https://huggingface.co/front/assets/huggingface_logo-noborder.svg\" style=\"Height: 45px;width:45px\"> \n"]},{"cell_type":"code","execution_count":1,"id":"938f95c1","metadata":{"execution":{"iopub.execute_input":"2023-10-25T18:37:42.147071Z","iopub.status.busy":"2023-10-25T18:37:42.146618Z","iopub.status.idle":"2023-10-25T18:38:30.934267Z","shell.execute_reply":"2023-10-25T18:38:30.933234Z"},"papermill":{"duration":48.807596,"end_time":"2023-10-25T18:38:30.937003","exception":false,"start_time":"2023-10-25T18:37:42.129407","status":"completed"},"tags":[]},"outputs":[{"name":"stdout","output_type":"stream","text":["\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\r\n","cached-path 1.1.3 requires huggingface-hub<0.8.0,>=0.0.12, but you have huggingface-hub 0.16.4 which is incompatible.\r\n","allennlp 2.9.3 requires transformers<4.19,>=4.1, but you have transformers 4.24.0 which is incompatible.\u001b[0m\u001b[31m\r\n","\u001b[0m\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\r\n","\u001b[0m\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\r\n","\r\n","\r\n","\r\n","The following NEW packages will be installed:\r\n","  git-lfs\r\n","0 upgraded, 1 newly installed, 0 to remove and 35 not upgraded.\r\n","Need to get 3316 kB of archives.\r\n","After this operation, 11.1 MB of additional disk space will be used.\r\n","Get:1 http://archive.ubuntu.com/ubuntu focal/universe amd64 git-lfs amd64 2.9.2-1 [3316 kB]\r\n","Fetched 3316 kB in 5s (670 kB/s)\r\n","Selecting previously unselected package git-lfs.\r\n","(Reading database ... 108264 files and directories currently installed.)\r\n","Preparing to unpack .../git-lfs_2.9.2-1_amd64.deb ...\r\n","Unpacking git-lfs (2.9.2-1) ...\r\n","Setting up git-lfs (2.9.2-1) ...\r\n","Processing triggers for man-db (2.9.1-1) ...\r\n","Error: Failed to call git rev-parse --git-dir: exit status 128 \r\n","Git LFS initialized.\r\n"]}],"source":["! pip install -q transformers==4.24.0\n","\n","!pip install -q gradio\n","!sudo apt-get install git-lfs\n","!git lfs install"]},{"cell_type":"markdown","id":"4339561d","metadata":{"papermill":{"duration":0.019417,"end_time":"2023-10-25T18:38:30.975895","exception":false,"start_time":"2023-10-25T18:38:30.956478","status":"completed"},"tags":[]},"source":["<a id=\"1\"></a>\n","## <p style=\"background-color:#1a0a36;font-family:newtimeroman;color:#FFF9ED;font-size:140%;text-align:center;border-radius:10px 10px;\">IMPORTING LIBRARIES</p>\n","#### [Top ↑](#top)"]},{"cell_type":"code","execution_count":2,"id":"cb3e948b","metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","execution":{"iopub.execute_input":"2023-10-25T18:38:31.014926Z","iopub.status.busy":"2023-10-25T18:38:31.014568Z","iopub.status.idle":"2023-10-25T18:38:43.256733Z","shell.execute_reply":"2023-10-25T18:38:43.255755Z"},"papermill":{"duration":12.264905,"end_time":"2023-10-25T18:38:43.259519","exception":false,"start_time":"2023-10-25T18:38:30.994614","status":"completed"},"tags":[]},"outputs":[],"source":["import os\n","import wandb\n","import torch\n","import pickle\n","import numpy as np\n","%matplotlib inline\n","import pandas as pd\n","import gradio as gr\n","import seaborn as sns\n","import tensorflow as tf\n","from typing import Dict\n","from ast import literal_eval\n","from torch.optim import AdamW\n","from tqdm import tqdm, trange\n","import matplotlib.pyplot as plt\n","from kaggle_secrets import UserSecretsClient\n","from torch.nn import BCEWithLogitsLoss, BCELoss\n","from sklearn.model_selection import train_test_split\n","from sklearn.preprocessing import MultiLabelBinarizer\n","from keras.preprocessing.sequence import pad_sequences\n","from torch.utils.data import TensorDataset, DataLoader, RandomSampler, SequentialSampler\n","from sklearn.metrics import classification_report, confusion_matrix, multilabel_confusion_matrix, f1_score, accuracy_score\n","from transformers import XLNetForSequenceClassification, XLNetTokenizer,BertForSequenceClassification,BertTokenizer, RobertaForSequenceClassification,RobertaTokenizer\n","\n","# pd.set_option('Display.max_colwidth',None)"]},{"cell_type":"code","execution_count":3,"id":"fe889158","metadata":{"execution":{"iopub.execute_input":"2023-10-25T18:38:43.300597Z","iopub.status.busy":"2023-10-25T18:38:43.299897Z","iopub.status.idle":"2023-10-25T18:38:43.306191Z","shell.execute_reply":"2023-10-25T18:38:43.304836Z"},"papermill":{"duration":0.028947,"end_time":"2023-10-25T18:38:43.308683","exception":false,"start_time":"2023-10-25T18:38:43.279736","status":"completed"},"tags":[]},"outputs":[],"source":["def warn(*args, **kwargs):\n","    pass\n","import warnings\n","warnings.warn = warn"]},{"cell_type":"code","execution_count":4,"id":"e02359e4","metadata":{"execution":{"iopub.execute_input":"2023-10-25T18:38:43.347051Z","iopub.status.busy":"2023-10-25T18:38:43.346725Z","iopub.status.idle":"2023-10-25T18:38:43.354832Z","shell.execute_reply":"2023-10-25T18:38:43.35388Z"},"papermill":{"duration":0.029241,"end_time":"2023-10-25T18:38:43.357117","exception":false,"start_time":"2023-10-25T18:38:43.327876","status":"completed"},"tags":[]},"outputs":[{"data":{"text/plain":["'1.11.0'"]},"execution_count":4,"metadata":{},"output_type":"execute_result"}],"source":["torch.__version__"]},{"cell_type":"code","execution_count":5,"id":"1c971857","metadata":{"execution":{"iopub.execute_input":"2023-10-25T18:38:43.395288Z","iopub.status.busy":"2023-10-25T18:38:43.394555Z","iopub.status.idle":"2023-10-25T18:38:51.162372Z","shell.execute_reply":"2023-10-25T18:38:51.161073Z"},"papermill":{"duration":7.789592,"end_time":"2023-10-25T18:38:51.164677","exception":false,"start_time":"2023-10-25T18:38:43.375085","status":"completed"},"tags":[]},"outputs":[{"name":"stdout","output_type":"stream","text":["Found GPU at: /device:GPU:0\n"]}],"source":["device_name = tf.test.gpu_device_name()\n","if device_name != '/device:GPU:0':\n","    raise SystemError('GPU device not found')\n","print('Found GPU at: {}'.format(device_name))"]},{"cell_type":"code","execution_count":6,"id":"14bc8598","metadata":{"execution":{"iopub.execute_input":"2023-10-25T18:38:51.202718Z","iopub.status.busy":"2023-10-25T18:38:51.202395Z","iopub.status.idle":"2023-10-25T18:38:51.213526Z","shell.execute_reply":"2023-10-25T18:38:51.212652Z"},"papermill":{"duration":0.032053,"end_time":"2023-10-25T18:38:51.215634","exception":false,"start_time":"2023-10-25T18:38:51.183581","status":"completed"},"tags":[]},"outputs":[{"data":{"text/plain":["'Tesla T4'"]},"execution_count":6,"metadata":{},"output_type":"execute_result"}],"source":["device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n","n_gpu = torch.cuda.device_count()\n","torch.cuda.get_device_name(0)"]},{"cell_type":"markdown","id":"e3b8c156","metadata":{"papermill":{"duration":0.018116,"end_time":"2023-10-25T18:38:51.252001","exception":false,"start_time":"2023-10-25T18:38:51.233885","status":"completed"},"tags":[]},"source":["\n","<img src=\"https://github.com/camo/dd842f7b0be57140e68b2ab9cb007992acd131c48284eaf6b1aca758bfea358b/68747470733a2f2f692e696d6775722e636f6d2f52557469567a482e706e67\">\n","\n","> I will be integrating W&B for visualizations and logging artifacts and comparisons of different models!\n","> \n","> [Multi Label Classification of PubMed Articles (Paper Night Presentation)]\n","> https://wandb.ai/owaiskhan9515/Multi%20Label%20Classification%20of%20PubMed%20Articles%20(Paper%20Night%20Presentation)\n","\n","\n","> \n","> - To get the API key, create an account in the [website](https://wandb.ai/site) .\n","> - Use secrets to use API Keys more securely "]},{"cell_type":"code","execution_count":7,"id":"3b86d6dd","metadata":{"execution":{"iopub.execute_input":"2023-10-25T18:38:51.292515Z","iopub.status.busy":"2023-10-25T18:38:51.292188Z","iopub.status.idle":"2023-10-25T18:38:55.39402Z","shell.execute_reply":"2023-10-25T18:38:55.392928Z"},"papermill":{"duration":4.124012,"end_time":"2023-10-25T18:38:55.396116","exception":false,"start_time":"2023-10-25T18:38:51.272104","status":"completed"},"tags":[]},"outputs":[{"name":"stderr","output_type":"stream","text":["\u001b[34m\u001b[1mwandb\u001b[0m: W&B API key is configured. Use \u001b[1m`wandb login --relogin`\u001b[0m to force relogin\n","\u001b[34m\u001b[1mwandb\u001b[0m: \u001b[33mWARNING\u001b[0m If you're specifying your api key in code, ensure this code is not shared publicly.\n","\u001b[34m\u001b[1mwandb\u001b[0m: \u001b[33mWARNING\u001b[0m Consider setting the WANDB_API_KEY environment variable, or running `wandb login` from the command line.\n","\u001b[34m\u001b[1mwandb\u001b[0m: Appending key for api.wandb.ai to your netrc file: /root/.netrc\n","\u001b[34m\u001b[1mwandb\u001b[0m: Currently logged in as: \u001b[33mowaiskhan9515\u001b[0m. Use \u001b[1m`wandb login --relogin`\u001b[0m to force relogin\n"]},{"data":{"text/html":["wandb version 0.15.12 is available!  To upgrade, please run:\n"," $ pip install wandb --upgrade"],"text/plain":["<IPython.core.display.HTML object>"]},"metadata":{},"output_type":"display_data"},{"data":{"text/html":["Tracking run with wandb version 0.12.18"],"text/plain":["<IPython.core.display.HTML object>"]},"metadata":{},"output_type":"display_data"},{"data":{"text/html":["Run data is saved locally in <code>/kaggle/working/wandb/run-20231025_183852-2hua4iyd</code>"],"text/plain":["<IPython.core.display.HTML object>"]},"metadata":{},"output_type":"display_data"},{"data":{"text/html":["Syncing run <strong><a href=\"https://wandb.ai/owaiskhan9515/Multi%20Label%20Classification%20of%20PubMed%20Articles%20%28Paper%20Night%20Presentation%29/runs/2hua4iyd\" target=\"_blank\">42.Biobert-base-cased-v1.2-Run-27</a></strong> to <a href=\"https://wandb.ai/owaiskhan9515/Multi%20Label%20Classification%20of%20PubMed%20Articles%20%28Paper%20Night%20Presentation%29\" target=\"_blank\">Weights & Biases</a> (<a href=\"https://wandb.me/run\" target=\"_blank\">docs</a>)<br/>"],"text/plain":["<IPython.core.display.HTML object>"]},"metadata":{},"output_type":"display_data"},{"data":{"text/html":["<button onClick=\"this.nextSibling.style.display='block';this.style.display='none';\">Display W&B run</button><iframe src=\"https://wandb.ai/owaiskhan9515/Multi%20Label%20Classification%20of%20PubMed%20Articles%20%28Paper%20Night%20Presentation%29/runs/2hua4iyd?jupyter=true\" style=\"border:none;width:100%;height:420px;display:none;\"></iframe>"],"text/plain":["<wandb.sdk.wandb_run.Run at 0x7c0c100a5c50>"]},"execution_count":7,"metadata":{},"output_type":"execute_result"}],"source":["try:\n","    from kaggle_secrets import UserSecretsClient\n","    user_secrets = UserSecretsClient()\n","    secret_value_0 = user_secrets.get_secret(\"wandb_api\")\n","    wandb.login(key=secret_value_0)\n","    anony=None\n","except:\n","    anony = \"must\"\n","    print('If you want to use your W&B account, go to Add-ons -> Secrets and provide your W&B access token. Use the Label name as wandb_api. \\nGet your W&B access token from here: https://wandb.ai/authorize')\n","    \n","    \n","    \n","wandb.init(project=\"Multi Label Classification of PubMed Articles (Paper Night Presentation)\",name=f\"42.Biobert-base-cased-v1.2-Run-27\")"]},{"cell_type":"markdown","id":"caabbbcd","metadata":{"papermill":{"duration":0.019428,"end_time":"2023-10-25T18:38:55.435518","exception":false,"start_time":"2023-10-25T18:38:55.41609","status":"completed"},"tags":[]},"source":["<a id=\"2\"></a>\n","## <p style=\"background-color:#1a0a36;font-family:newtimeroman;color:#FFF9ED;font-size:150%;text-align:center;border-radius:10px 10px;\"> Reading BioASQ Processed Dataset</p>\n","#### [Top ↑](#top)"]},{"cell_type":"code","execution_count":8,"id":"04a99166","metadata":{"execution":{"iopub.execute_input":"2023-10-25T18:38:55.482772Z","iopub.status.busy":"2023-10-25T18:38:55.482432Z","iopub.status.idle":"2023-10-25T18:38:57.956192Z","shell.execute_reply":"2023-10-25T18:38:57.955258Z"},"papermill":{"duration":2.497264,"end_time":"2023-10-25T18:38:57.958822","exception":false,"start_time":"2023-10-25T18:38:55.461558","status":"completed"},"tags":[]},"outputs":[{"data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>Title</th>\n","      <th>abstractText</th>\n","      <th>meshMajor</th>\n","      <th>pmid</th>\n","      <th>meshid</th>\n","      <th>meshroot</th>\n","      <th>A</th>\n","      <th>B</th>\n","      <th>C</th>\n","      <th>D</th>\n","      <th>E</th>\n","      <th>F</th>\n","      <th>G</th>\n","      <th>H</th>\n","      <th>I</th>\n","      <th>J</th>\n","      <th>L</th>\n","      <th>M</th>\n","      <th>N</th>\n","      <th>Z</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>Expression of p53 and coexistence of HPV in pr...</td>\n","      <td>Fifty-four paraffin embedded tissue sections f...</td>\n","      <td>['DNA Probes, HPV', 'DNA, Viral', 'Female', 'H...</td>\n","      <td>8549602</td>\n","      <td>[['D13.444.600.223.555', 'D27.505.259.750.600....</td>\n","      <td>['Chemicals and Drugs [D]', 'Organisms [B]', '...</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>Vitamin D status in pregnant Indian women acro...</td>\n","      <td>The present cross-sectional study was conducte...</td>\n","      <td>['Adult', 'Alkaline Phosphatase', 'Breast Feed...</td>\n","      <td>21736816</td>\n","      <td>[['M01.060.116'], ['D08.811.277.352.650.035'],...</td>\n","      <td>['Named Groups [M]', 'Chemicals and Drugs [D]'...</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>1</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>[Identification of a functionally important di...</td>\n","      <td>The occurrence of individual amino acids and d...</td>\n","      <td>['Amino Acid Sequence', 'Analgesics, Opioid', ...</td>\n","      <td>19060934</td>\n","      <td>[['G02.111.570.060', 'L01.453.245.667.060'], [...</td>\n","      <td>['Phenomena and Processes [G]', 'Information S...</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                                               Title  \\\n","0  Expression of p53 and coexistence of HPV in pr...   \n","1  Vitamin D status in pregnant Indian women acro...   \n","2  [Identification of a functionally important di...   \n","\n","                                        abstractText  \\\n","0  Fifty-four paraffin embedded tissue sections f...   \n","1  The present cross-sectional study was conducte...   \n","2  The occurrence of individual amino acids and d...   \n","\n","                                           meshMajor      pmid  \\\n","0  ['DNA Probes, HPV', 'DNA, Viral', 'Female', 'H...   8549602   \n","1  ['Adult', 'Alkaline Phosphatase', 'Breast Feed...  21736816   \n","2  ['Amino Acid Sequence', 'Analgesics, Opioid', ...  19060934   \n","\n","                                              meshid  \\\n","0  [['D13.444.600.223.555', 'D27.505.259.750.600....   \n","1  [['M01.060.116'], ['D08.811.277.352.650.035'],...   \n","2  [['G02.111.570.060', 'L01.453.245.667.060'], [...   \n","\n","                                            meshroot  A  B  C  D  E  F  G  H  \\\n","0  ['Chemicals and Drugs [D]', 'Organisms [B]', '...  0  1  1  1  1  0  0  1   \n","1  ['Named Groups [M]', 'Chemicals and Drugs [D]'...  0  1  1  1  1  1  1  0   \n","2  ['Phenomena and Processes [G]', 'Information S...  1  1  0  1  1  0  1  0   \n","\n","   I  J  L  M  N  Z  \n","0  0  0  0  0  0  0  \n","1  1  1  0  1  1  1  \n","2  0  0  1  0  0  0  "]},"execution_count":8,"metadata":{},"output_type":"execute_result"}],"source":["dataset_Name='../input/pubmed-multilabel-text-classification/PubMed Multi Label Text Classification Dataset Processed.csv'\n","\n","df= pd.read_csv(dataset_Name)\n","df.head(3)"]},{"cell_type":"code","execution_count":9,"id":"4d59d158","metadata":{"execution":{"iopub.execute_input":"2023-10-25T18:38:58.00053Z","iopub.status.busy":"2023-10-25T18:38:58.000177Z","iopub.status.idle":"2023-10-25T18:38:58.00644Z","shell.execute_reply":"2023-10-25T18:38:58.005449Z"},"papermill":{"duration":0.029118,"end_time":"2023-10-25T18:38:58.008406","exception":false,"start_time":"2023-10-25T18:38:57.979288","status":"completed"},"tags":[]},"outputs":[{"name":"stdout","output_type":"stream","text":["Total number of Articles extracted from Bioasq dataset are = 50000\n"]}],"source":["print(\"Total number of Articles extracted from Bioasq dataset are =\",len(df))"]},{"cell_type":"code","execution_count":10,"id":"4dddd1d4","metadata":{"execution":{"iopub.execute_input":"2023-10-25T18:38:58.052322Z","iopub.status.busy":"2023-10-25T18:38:58.051698Z","iopub.status.idle":"2023-10-25T18:39:01.116148Z","shell.execute_reply":"2023-10-25T18:39:01.115177Z"},"papermill":{"duration":3.089323,"end_time":"2023-10-25T18:39:01.118795","exception":false,"start_time":"2023-10-25T18:38:58.029472","status":"completed"},"tags":[]},"outputs":[{"name":"stdout","output_type":"stream","text":["Average Article length:  192.05284\n","Stdev Article length:  76.74764082329723\n"]}],"source":["print('Average Article length: ', df.abstractText.str.split().str.len().mean())\n","print('Stdev Article length: ', df.abstractText.str.split().str.len().std())"]},{"cell_type":"code","execution_count":11,"id":"8ed063cc","metadata":{"execution":{"iopub.execute_input":"2023-10-25T18:39:01.161195Z","iopub.status.busy":"2023-10-25T18:39:01.160527Z","iopub.status.idle":"2023-10-25T18:39:01.168245Z","shell.execute_reply":"2023-10-25T18:39:01.167161Z"},"papermill":{"duration":0.031193,"end_time":"2023-10-25T18:39:01.170433","exception":false,"start_time":"2023-10-25T18:39:01.13924","status":"completed"},"tags":[]},"outputs":[{"name":"stdout","output_type":"stream","text":["Mesh Labels Root Class: \"\n","\" ['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'L', 'M', 'N', 'Z']\n","\n","\n","Number of Labels:  14\n"]}],"source":["cols = df.columns\n","cols = list(df.columns)\n","mesh_Heading_categories = cols[6:]\n","num_labels = len(mesh_Heading_categories)\n","print('Mesh Labels Root Class: \"\\n\"',mesh_Heading_categories)\n","print(\"\\n\")\n","print('Number of Labels: ' ,num_labels)\n"]},{"cell_type":"markdown","id":"187bc5b2","metadata":{"papermill":{"duration":0.019988,"end_time":"2023-10-25T18:39:01.210365","exception":false,"start_time":"2023-10-25T18:39:01.190377","status":"completed"},"tags":[]},"source":["Orginal Version of this Dataset contains **15,559,157 Articles** from [BioASQ Task 9A](http://participants-area.bioasq.org/datasets/).\n","More details about the format of the data and the task are available in the [Guidelines for task 9a](http://participants-area.bioasq.org/general_information/Task9a/)\n","\n","This dataset which I am using currently is a preprocessed version and currently consists of a approx **50k** collection of research articles from [**PubMed**](https://pubmed.ncbi.nlm.nih.gov/) repository. Originally these documents are manually annotated by Biomedical Experts with their MeSH labels and each articles are described in terms of 10-15 MeSH labels. In this Dataset we have huge numbers of labels present as a MeSH major which is raising the issue of extremely large output space and severe label sparsity issues. To solve this Issue Dataset has been Processed and mapped to its root as Described in the Below Figure.\n","![Mapped Image not Fetched](https://gitlab.com/Owaiskhan9654/Gene-Sequence-Primer/-/raw/main/Capture111.PNG)\n","![Tree Structure](https://gitlab.com/Owaiskhan9654/Gene-Sequence-Primer/-/raw/main/Capture22.PNG)\n","\n","\n","\n","\n","For more information on the attributes visit [here](https://www.kaggle.com/datasets/owaiskhan9654/pubmed-multilabel-text-classification).\n","\n","<a id=\"3\"></a>\n","## <p style=\"background-color:#1a0a36;font-family:newtimeroman;color:#FFF9ED;font-size:150%;text-align:center;border-radius:5px 5px;\">DATA VISUALIZATION</p>\n","#### [Top ↑](#top)\n","\n","#### In order to, get a full grasp of what steps should I be taking to utilizing this dataset. Let us have a look at the information in data. "]},{"cell_type":"code","execution_count":12,"id":"d09ee3e7","metadata":{"execution":{"iopub.execute_input":"2023-10-25T18:39:01.259778Z","iopub.status.busy":"2023-10-25T18:39:01.259104Z","iopub.status.idle":"2023-10-25T18:39:01.503506Z","shell.execute_reply":"2023-10-25T18:39:01.502687Z"},"papermill":{"duration":0.268975,"end_time":"2023-10-25T18:39:01.5056","exception":false,"start_time":"2023-10-25T18:39:01.236625","status":"completed"},"tags":[]},"outputs":[{"name":"stdout","output_type":"stream","text":["CPU times: user 5.99 ms, sys: 0 ns, total: 5.99 ms\n","Wall time: 5.41 ms\n"]},{"data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>Root Label</th>\n","      <th>number of Abstract</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>A</td>\n","      <td>23263</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>B</td>\n","      <td>46577</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>C</td>\n","      <td>26453</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>D</td>\n","      <td>31074</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>E</td>\n","      <td>39202</td>\n","    </tr>\n","    <tr>\n","      <th>5</th>\n","      <td>F</td>\n","      <td>8885</td>\n","    </tr>\n","    <tr>\n","      <th>6</th>\n","      <td>G</td>\n","      <td>33609</td>\n","    </tr>\n","    <tr>\n","      <th>7</th>\n","      <td>H</td>\n","      <td>6069</td>\n","    </tr>\n","    <tr>\n","      <th>8</th>\n","      <td>I</td>\n","      <td>5595</td>\n","    </tr>\n","    <tr>\n","      <th>9</th>\n","      <td>J</td>\n","      <td>5531</td>\n","    </tr>\n","    <tr>\n","      <th>10</th>\n","      <td>L</td>\n","      <td>7503</td>\n","    </tr>\n","    <tr>\n","      <th>11</th>\n","      <td>M</td>\n","      <td>21363</td>\n","    </tr>\n","    <tr>\n","      <th>12</th>\n","      <td>N</td>\n","      <td>22919</td>\n","    </tr>\n","    <tr>\n","      <th>13</th>\n","      <td>Z</td>\n","      <td>8049</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["   Root Label  number of Abstract\n","0           A               23263\n","1           B               46577\n","2           C               26453\n","3           D               31074\n","4           E               39202\n","5           F                8885\n","6           G               33609\n","7           H                6069\n","8           I                5595\n","9           J                5531\n","10          L                7503\n","11          M               21363\n","12          N               22919\n","13          Z                8049"]},"execution_count":12,"metadata":{},"output_type":"execute_result"}],"source":["%%time\n","\n","counts = []\n","for mesh_Heading_category in mesh_Heading_categories:\n","    counts.append((mesh_Heading_category, df[mesh_Heading_category].sum()))\n","df_count = pd.DataFrame(counts, columns=['Root Label', 'number of Abstract'])\n","df_count"]},{"cell_type":"code","execution_count":13,"id":"a773c459","metadata":{"execution":{"iopub.execute_input":"2023-10-25T18:39:01.550121Z","iopub.status.busy":"2023-10-25T18:39:01.549291Z","iopub.status.idle":"2023-10-25T18:39:01.979692Z","shell.execute_reply":"2023-10-25T18:39:01.978674Z"},"papermill":{"duration":0.455158,"end_time":"2023-10-25T18:39:01.981941","exception":false,"start_time":"2023-10-25T18:39:01.526783","status":"completed"},"tags":[]},"outputs":[{"data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAA80AAAI3CAYAAAChyLk+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAAByKElEQVR4nO3dd3RUVf/24XtSIQkhhRZ66FjoiEjvVUIVpAsKIk2K0qsggjQBRRBpSpMSeuiggCAdQUR6DwRJKCmEJDPvH7zMY37JwaCZTEw+11qsxZy995nvTii55+yzj8lisVgEAAAAAAAScLB3AQAAAAAApFaEZgAAAAAADBCaAQAAAAAwQGgGAAAAAMAAoRkAAAAAAAOEZgAAAAAADDjZuwAAAP6NmjVr6ubNm3/bb/HixapQoUIKVPQ/M2fO1KxZs9SrVy/17t07Wc65Zs0aDRkyJN4xk8kkDw8P5c+fXzVq1FCnTp3k4eGRLO9nbzdu3NCyZct08OBB3bhxQ+Hh4XJ3d1e+fPn02muvqVmzZipUqFC8/rVq1VKuXLm0a9cuO1YOAEgrCM0AgDShcuXKypo1q2F7lixZUrAa28uSJYuqVKkiSYqNjdXNmzd18uRJnTp1SuvXr9fSpUvl6+trl9qSK7h+9dVX+uqrrxQTEyMvLy+98sor8vLy0sOHD/Xbb79p3rx5mj9/voYOHaoOHTok4wwAAPgfQjMAIE3o1q1bil9JtqcCBQros88+i3fs7Nmz6tixo65cuaKZM2dq9OjR9ikuGUycOFHz58+Xh4eHPvnkEwUEBMjB4X93lVksFu3fv19Tp07VtWvX7FgpACCt455mAADSiGLFiumdd96RJO3Zs8e+xfwLP//8s+bPny9nZ2fNnz9fzZo1ixeYpadL0itXrqwVK1aoYcOGdqoUAJAecKUZAJAu/fzzz9q+fbuOHj2qO3fuKCIiQlmzZtVrr72mbt26qWDBgoZj9+7dqxUrVujEiRO6f/++MmfOrDx58qhmzZrq2LGjMmTIkGDMn3/+qRkzZmjPnj0KDQ1VtmzZ1LBhQ/Xu3Vuurq7JNq9ixYpZ3y8xx44d04IFC3Ts2DE9ePBAmTNnVtmyZdWlSxeVKlUq0TGRkZFatGiRgoKCrFd18+XLpwYNGqhTp07KmDGjte/gwYMVGBgoSbp586aKFi1qbUvqcu3Zs2dLktq0aaOSJUs+t6+zs7NKly79t+eUpJMnT2rLli365ZdfdPv2bT18+FBeXl4qU6aM4fzj4uL0ww8/aN26dTp//ryio6Pl6emp7Nmzq0KFCurWrZt8fHys/S9duqQ5c+bo0KFDunv3rpydneXt7a3ixYurSZMmqlevXpJqBQCkHoRmAEC6NGrUKN2+fVuFCxdWuXLlJEnnz5/X2rVrtXXrVs2bN896/BmLxaLRo0dr+fLlkqRXXnlFr732mh48eKCLFy9qypQpatiwoXLnzh1vXHBwsJo3by6LxaLSpUsrIiJCR48e1TfffKMLFy7o66+/TrZ5PXr0SFLi93AvXbpUn3zyicxms1599VW9/vrrunr1qrZu3art27drzJgxeuutt+KNCQ0NVadOnXTu3DllzpxZlStXliT98ssvmjZtmoKCgrRo0SJ5eXlJksqWLavIyEht3bpVbm5u8UKit7f339b/4MEDHTlyRJLUrFmzf/Q1MDJt2jQdOnRIhQoVUokSJeTi4qLLly9r69at2rFjh6ZMmaIGDRrEGzNs2DAFBgYqQ4YMKlu2rLy9vRUWFqZr165pwYIFql+/vjU0//HHH3r77bcVERGhAgUKqEaNGjKZTLpz54727dunx48fE5oB4D+I0AwASJcGDRqk1157TZ6entZjFotFK1as0KhRozRy5Eht2rRJJpPJ2r5o0SItX75cWbJk0ZdffhnvyqTFYtHBgweVOXPmBO+1evVqtWrVSiNHjpSLi4sk6eLFi2rZsqV2796to0ePqmzZsskyr927d0uSqlevHu/42bNnNX78eEnS9OnT44XDTZs2aeDAgRo7dqxKlSqlIkWKWNvGjBmjc+fOqVy5cpo9e7b16/XgwQN1795dx48f19ixYzV16lRJUqtWrVSxYkVt3bpV3t7eCe67/jtnzpyR2WyWs7NzvKvUyaFLly6aPHlygg8Udu3apT59+mj06NGqXr269cr5zZs3FRgYKD8/P61atSrBuN9//13ZsmWzvl64cKEiIiLUv39/de/ePV7fiIgInTt3LlnnAwBIGdzTDABIEzp27KiiRYsm+uv/XjGWpNq1a8cLzNLT+2TbtGmj0qVL6+LFi7pw4YK1LTY21npFeMKECQmW8ppMJlWsWFGZMmVK8F5+fn4aPny4NTBLUsGCBRUQECBJOnDgwD+et/R0CfHVq1c1ceJEbd68WUWKFFGfPn3i9Vm8eLFiY2PVsGHDBFdTGzVqpPr16ysmJkaLFy+2Hr9586a2bt0qBwcHffLJJ/G+XpkzZ9a4cePk4OCgoKAgBQcH/6s5PBMWFiZJ8vLykpNT8n62X7Vq1USvwNesWVP16tXT/fv39csvv1iP37t3T5L00ksvJTquePHi8XYof9a/atWqCfq6u7sneRk5ACB14UozACBNeN4jpxK7x1iSbt++rT179ujSpUsKDw+X2WyW9L/7ga9cuaLChQtLkk6fPq2wsDDlyJEj0VD0PK+//nqiNRQoUECSFBIS8kLnk6RDhw4leiW2cuXKmj17dryALkmHDx+WJDVv3jzR87Vo0UKbN2/WoUOHrMeOHDkii8WiUqVKWWv9q0KFCqlkyZI6fvy4Dh8+rCZNmrzwPFJaaGio9uzZo/Pnz+vhw4eKi4uT9HRpvvT0e/5MgQIF5O7urh9//FFff/213nzzTeXKlcvw3CVKlNCPP/6oUaNGqW/fvipfvnyC7wMA4L+H0AwASBNe9JFTM2bM0Jw5cxQbG2vYJzw83Pr7W7duSZL8/f1fuDY/P79Ej3t4eEiSnjx58sLn/OtzmqOiovTHH3/o8uXL2rdvn6ZMmaIhQ4bE63/nzh1JSnC/9TN58uSJ1y8pY561HT9+PN64f+PZfc/3799XXFycHB0dk+W8krR8+XJ99tlnioqKMuzz1++5h4eHPv30Uw0dOlTTpk3TtGnTlD17dpUqVUrVq1dXo0aN4m3i1rVrVx05ckQHDhxQly5d5OLiouLFi6t8+fJq0qRJsi83BwCkDEIzACDd2bp1q7788ku5u7tr9OjRev3115U1a1br1eABAwZo48aNslgsyfJ+//dxSckhsec0L1myRGPHjtXChQtVsWLFBPc1S4p3j3ZS/ZMx/9RLL70kBwcHxcTE6OzZs3r55ZeT5by//vqrRo8eLScnJ3388ceqUaOGcuTIoYwZM8pkMmnq1KmaM2dOgu95/fr19cYbb2jnzp06fPiwjh07pq1bt2rr1q2aNWuWlixZYv1QJGPGjFq4cKFOnjypvXv36tixYzp+/LhOnjypefPmqXfv3urVq1eyzAcAkHK4pxkAkO5s2bJFktS/f3+1atVKefLkibd8+urVqwnG5MyZU5J0+fLllCnyH2jXrp31PunPPvss3lX07NmzS5KuX7+e6Nhnx5/1S8oYSbpx40aCcf/Gs0dgSbI+uio5bNu2TRaLRR06dFDXrl1VoEABubm5WT8QSOx7/oynp6eaNWumTz/9VFu2bNH27dtVoUIF3bx5U5MnT07Qv2TJkurVq5fmz5+vX375RRMmTJCTk5NmzZqlS5cuJducAAApg9AMAEh3Hjx4IEnKkSNHgraLFy/q999/T3D85Zdflre3t27fvq29e/favMZ/asCAAcqQIYMuX76s9evXW4+XL19ekrR27dpEx61Zs0aS9Nprr1mPlStXTiaTSSdPnkz0w4KLFy/q5MmTcnBwsJ5fevrsZEnPXfr+PM92nl6+fLl+/fXX5/aNjY3ViRMn/vacz/ueh4aG6ueff05yfXnz5lWPHj0kPd2V/HlcXFzUvHlzlSxZUhaLRX/88UeS3wcAkDoQmgEA6c6zTa1WrlwZ737ie/fuadCgQYmGPWdnZ2uYGzJkSIIw9+yRU8+ek2wv2bNnV8eOHSVJs2fPts6lY8eOcnJy0qZNm7R9+/Z4Y4KCghQUFCRnZ2d16NDBejxXrlyqW7euzGazRo4cGW9uDx8+1MiRI2U2m9WgQYN49237+PjI2dlZ9+7ds4bVF1GlShV16tRJMTExeuedd7R27VrrJm3PWCwWHThwQG3atNGmTZv+9pzPvufr1q1TRESE9Xh4eLiGDh2qhw8fJhhz5swZbd68WY8fP07QtmvXLkn/W4EgPV0en9iV5OvXr1t3Yn/eRmIAgNSJe5oBAGnC3Llzn7uct3HjxqpcubIkqVOnTlq7dq327NmjunXrqkSJEoqOjtahQ4fk5+en2rVra8eOHQnO0blzZ128eFErV67UW2+9pVdeeUX58uXT/fv3dfHiRQUHB2vnzp2JPnYqJb333ntasWKFrl27prVr16ply5YqVqyYhg4dqk8++US9evVSyZIllSdPHl27dk2//vqrHBwcNGLEiASbVY0ePVqXL1/WoUOHVLt2beuV6F9++UUPHjxQsWLFNHLkyHhjnJ2dVb16dW3fvl3NmjVT6dKllSFDBnl7e2vgwIFJmsOQIUPk6emp2bNna9CgQZowYYJKlCghT09PhYeH67ffftPdu3fl6OioZs2a/e35mjdvrkWLFum3335T7dq1VbZsWVksFh05ckTOzs5q0aKFVq9eHW/MrVu31K9fP2XMmFEvvfSS/Pz8FBMTozNnzuj69etyd3eP92ivH374QWPHjlWePHlUuHBhubm56c8//9TRo0cVExOjRo0aqUSJEkmaPwAg9SA0AwDShH379j23vVixYtbQnCdPHgUGBmratGk6evSodu/erezZs6t169bq2bOnxo8fn+g5TCaTxo0bp1q1almXDp89e1ZeXl7Kmzev2rdvb/jYq5Tk6empbt266fPPP9fs2bPVtGlTOTk5qV27dipWrJgWLFigY8eO6bffflPmzJlVt25ddenSJdHnCPv4+Gj58uVatGiRgoKC9NNPP0mS8uXLpy5duqhjx45yc3NLMO6TTz5R5syZtW/fPm3ZskWxsbHKlStXkkOzyWRSr169FBAQoGXLlungwYM6efKkwsPD5e7urvz586tZs2Zq3rx5knY0z5w5s1avXq0vvvhC+/fv1549e+Tr66s6deqoT58+WrFiRYIxJUuW1IABA3To0CFdunRJv/32m5ydneXn56cuXbqoffv28a4cf/jhh9q9e7d+/fVXHT9+XOHh4cqSJYtee+01tWrVSvXq1UvS3AEAqYvJklxbgwIAAAAAkMZwTzMAAAAAAAYIzQAAAAAAGCA0AwAAAABggNAMAAAAAIABQjMAAAAAAAYIzQAAAAAAGOA5zS8gLCxCZjNP6AIAAACAtMTBwSRvb/dE2wjNL8BsthCaAQAAACAdYXk2AAAAAAAGCM0AAAAAABggNAMAAAAAYIDQDAAAAACAAUIzAAAAAAAGCM0AAAAAABggNAMAAAAAYIDQDAAAAACAASd7F4D04fr1a+rUqY2qV6+lkSM/kSSFhYXpiy8m68CBfTKZHFSxYiWNGjVOkjR+/Ght375Fzs7O1nNs2bJHjo6O2rYtSJ9//qn1uNlsVnR0tObN+07FihXXgAF99Ouvx63tMTExyps3nxYvXpFCswUAAACQVhCakSKmTp2oYsVeinds2LCPVLz4S1q9epMyZMigS5cuxGtv27ajunX7IMG56tZtoLp1G1hfb968QQsXzlPRosUkSVOmzIjXv1evbipbtnxyTQUAAABAOsLybNjcjh1b5eGRKV5wPXTooEJC7uiDD/rKw8NDTk5OKlKk2D86f1DQRtWv30gmkylBW3DwLf366wnVr9/oH9cPAAAAIP0iNMOmIiLCNW/eHPXu3S/e8d9+O6W8efNp/PjRatiwlt59t6OOHz8ar09g4Co1aFBTXbq01549OxM9/+3bwTp58rhhKN6yZZNKlCglP7+cyTMhAAAAAOkKoRk29c03X6tx4ybKli17vOMhISE6dOigypQpq/Xrt6pNm3YaMmSA7t+/L0lq2bKNli9fow0btum9997X+PFj9OuvJxKc/1kozpkzV6Lvv2XLJjVs+GZyTwsAAABAOkFohs2cP/+Hjhw5pNat2yVoc3V1lZ9fTjVu3FROTk6qXbuesmXLrlOnTkiSihYtpsyZveTk5KSKFSurbt36+vHH3QnOs2XLJjVo0DjR9z958oRCQ++pevVayTovAAAAAOkHG4HBZo4fP6rbt2+pRYunoTYqKlJxcWZduXJJzZu30v79e+P1T+ye5PhtlnjHfv31hP78865q1Eg8FG/ZslFVq9aQm5vbv5sIAAAAgHSLK82wmSZNmmvFirVasGCJFixYooCAFnrjjUqaMmWWqlatoUePHiooaKPi4uK0e/cOhYSE6NVXS0mSdu/eocjISJnNZh06dFBbtwapUqWq8c4fFLRJ1arVlJube4L3jo5+rF27trM0GwAAAMC/wpVm2EyGDBmUIUMG6+uMGTPKxcVV3t7ekqSJE6dqypTPNHXqROXNm1+ffTZFXl5ekqSVK5frs88+kcUi+fnl1KBBw1SmTDnruaKjo7V793aNGzcp0ff+6ac98vDIFG8MAAAAALwok8Visfx9N0jSvXvhMpv5cgEAAABAWuLgYJKvr0fibSlcCwAAAAAA/xmEZgAAAAAADHBPM/4Vn8yucnRxsXcZySLuyROFPoi2dxkAAAAAUhFCM/4VRxcX3fyyp73LSBa5en4pidAMAAAA4H9Yng0AAAAAgAFCMwAAAAAABgjNAAAAAAAYIDQDAAAAAGCA0AwAAAAAgAFCMwAAAAAABgjNAAAAAAAYIDQDAAAAAGCA0AwAAAAAgAFCMwAAAAAABgjNAAAAAAAYIDQDAAAAAGCA0AwAAAAAgAFCMwAAAAAABgjNAAAAAAAYIDQDAAAAAGCA0AwAAAAAgAFCMwAAAAAABgjNAAAAAAAYIDQDAAAAAGCA0AwAAAAAgAFCMwAAAAAABgjNAAAAAAAYIDQDAAAAAGCA0AwAAAAAgAFCMwAAAAAABgjNAAAAAAAYIDQDAAAAAGCA0AwAAAAAgAFCMwAAAAAABgjNAAAAAAAYIDQDAAAAAGCA0AwAAAAAgAFCMwAAAAAABgjNAAAAAAAYIDQDAAAAAGCA0AwAAAAAgAFCMwAAAAAABgjNAAAAAAAYIDQDAAAAAGCA0AwAAAAAgAFCMwAAAAAABgjNAAAAAAAYIDQDAAAAAGCA0AwAAAAAgAFCMwAAAAAABgjNAAAAAAAYIDQDAAAAAGCA0AwAAAAAgAFCMwAAAAAABgjNAAAAAAAYIDQDAAAAAGCA0AwAAAAAgAFCMwAAAAAABgjNAAAAAAAYIDQDAAAAAGCA0AwAAAAAgAFCMwAAAAAABgjNAAAAAAAYIDQDAAAAAGAgVYTmWbNmqWjRojp37pwk6cSJE2rSpInq1aunLl266N69e9a+tmgDAFsYO3aEAgLqqW7damrTprk2bFhrbduwYa1at26qOnWqqH//3vrzz7vWtqVLF6tDh7dUp05VtWrVREuXLo533uDgW+rdu7tq1aqktm1b6PDhX6xtQUEb1aVLe9WtW03NmjXUV199odjYWJvPFQAAIK2ye2j+7bffdOLECeXKlUuSZDab9dFHH2nkyJHaunWrypUrp8mTJ9usDQBspX37zlq5coO2bftREydO1TffzNbZs7/r2LEjmjPnS02YMEWbN+9Szpw5NXr0MOs4i8Wi4cPHKihol6ZMmaHVq3/Qjh1bre2jRw9TkSJFtXnzTnXr9oFGjBiksLAwSdLjx4/Vt+8Abdq0Q3PnLtSRI4e1bNn3KT53AACAtMKuofnJkycaO3asRo8ebT12+vRpubq6qly5cpKkNm3aaMuWLTZrAwBbKVCgoFxcXCRJJtPTXzdv3tDPP+9TjRq1VaBAQTk7O6tz53d14sQx3bx5Q5LUrl0nFS1aTE5OTsqbN7+qVKmmU6dOSpKuXbuqc+fOqmvX7nJ1zaDq1WupQIFC+vHHnZKkZs1aqmTJ0nJ2dlbWrNlUt25961gAAAC8OLuG5i+++EJNmjRR7ty5rceCg4OVM2dO62sfHx+ZzWbdv3/fJm0AYEuTJ3/2/5dRt5SvbxZVrFhJ0tOryc88+/2lSxcSjLdYLDp58rj8/QtIki5fvqScOXPJzc3d2qdQocK6fPlSou//17EAAAB4cU72euPjx4/r9OnTGjhwoL1KeGG+vh72LgE2ljVrJnuXgDRm4sTx+vTTsTp+/LgOHTqknDl9VK9eLfXr10/vvNNB+fPn18yZi2QymeTiYkrwZ3DGjBlycDCpU6d2cnFxkZOTWV5emeP1y5bNV3fu3EkwdtWqVTp37qwmTfpMPj782QYAAPgn7BaaDx8+rIsXL6pWrVqSpNu3b6tr167q0KGDbt26Ze0XGhoqBwcHeXl5yc/PL9nbXsS9e+Eymy1/3zEdSWsh8+7dR/YuAWlUvnxF9cMPq/XNNwvVqlUbvfNON/Xs2UsRERF666235ebmpgwZPOP9GVy9eoXWrAnUl19+owcPoiVFKzbWQQ8ePIzX7+7dUDk4OMc79tNPezR58hRNn/6V4uKc+bMNAADwHA4OJsOLpHZbnt2tWzft27dPu3bt0q5du5QjRw59++23evfdd/X48WMdOXJEkrR8+XLVr19fkvTKK68kexsApJS4uDjrfcstWryl5csDtWHDNlWrVlNxcXHy9y9o7btx4zp9//0iTZ/+lbJly2497u9fQLdu3VRkZIT12IUL5+MtwT548GdNmjROEydOVcGChVJgZgAAAGmX3a40G3FwcNCkSZM0atQoRUdHK1euXPr8889t1gYAthAWFqqjRw/rjTeqyNXVVUeOHNKOHVs1evR4RUdH6+bN6/L3L6g7d+5o0qTxatXqbXl6ekqStm0L0ty5X2nGjK+VK1fueOfNmzefChUqovnzv9F77/XQwYM/6+LF8xo3bpIk6ejRwxo7doQ+/fRzvfTSKyk+bwAAgLTGZPnrbjR4LpZnJ5Q1aybd/LKnvctIFrl6fskSViSbsLAwjRgxSBcunJPZbFGOHDnUsmUbNWnSTI8ePVKvXu/p5s0bcnNzV8OGb+q993rI0dFRktSqVROFhNyx7rwtSXXrNtBHHw2V9PQ5zePHj9aZM78pe/bs6t9/kMqXryBJ6t27u3799US8sSVKlNaUKTNScPYAAAD/Lc9bnk1ofgGE5oQIzQAAAAD+61LlPc0AAAAAAKR2hGYAAAAAAAykuo3AAOC/ILOXs1ycM9i7jGTxJOaxHtyPsXcZAAAAqRKhGQD+ARfnDJq4vJ69y0gWg9pslURoBgAASAzLswEAAAAAMEBoBgAAAADAAKEZAAAAAAADhGYAAAAAAAwQmgEAAAAAMEBoBgAAAADAAKEZAAAAAAADhGYAAAAAAAwQmgEAAAAAMEBoBgAAAADAAKEZAAAAAAADhGYAAAAAAAwQmgEAAAAAMEBoBgAAAADAAKEZAAAAAAADhGYAAAAAAAwQmgEAAAAAMEBoBgAAAADAAKEZAAAAAAADhGYAAAAAAAwQmgEAAAAAMEBoBgAAAADAAKEZAAAAAAADhGYAAAAAAAwQmgEAAAAAMEBoBgAAAADAAKEZAAAAAAADhGYAAAAAAAwQmgEAAAAAMEBoBgAAAADAAKEZAAAAAAADhGYAAAAAAAwQmgEAAAAAMEBoBgAAAADAAKEZAAAAAAADhGYAAAAAAAwQmgEAAAAAMEBoBgAAAADAAKEZAAAAAAADhGYAAAAAAAwQmgEAAAAAMEBoBgAAAADAgJO9CwAAIK0ZO3aEjh49pKiox/Lx8VW7dh315ptNdfnyJY0bN0o3b96QJBUtWlwffjhQ/v4FrGP/+OOsZsyYonPnzipDhozq0OEdvfXW25Kk4OBb+vTTMTpz5rSyZ8+hfv0+VvnyFSRJT5480ddfz9TOndsVHR2t2rXr6cMPB8rJif/qAQD4N/ifFACAZNa+fWcNHjxCLi4uunr1inr37q7ChYsqV67cGjduonLk8JPZbNaaNSs1evRQLVq0XJJ0//59DRjQW3369Ff16rUUGxujkJAQ63lHjx6mV155VZMnf6EDB/ZrxIhBWrYsUN7e3vr++4U6e/Z3fffdCsXFmTVoUD8tWvStunbtbq8vAwAAaQLLswEASGYFChSUi4uLJMlkevrr5s0bypQpk/z8cspkMsliscjBwUE3bly3jluxYokqVHhddes2kIuLi9zc3JU/v78k6dq1qzp37qy6du0uV9cMql69lgoUKKQff9wpSdq/f69atmwjT8/M8vb2VsuWbbRp0/qUnzwAAGkMV5oBALCByZM/U1DQBkVHR6tIkaKqWLGSta1+/eqKioqS2WyOdyX4t99OqUCBQnr//S66ceO6XnrpFfXvP0g5cuTQ5cuXlDNnLrm5uVv7FypUWJcvX7K+tlgsf6nAopCQOwoPD5eHh4dN5woAQFpGaAYAwAYGDhysfv0+0unTp3T8+BHrlWdJ2rJlj6KiohQUtFE5cvhZj4eEhOjcubOaNu1LFShQSLNnz9CYMUM1e/Z8RUVFyt09fvh1d/fQn3/elSRVqFBRq1YtU5ky5WQ2x2nlyqdLvh8/fkxoBgDgX2B5NgAANuLo6KiSJUvp7t0QBQauiteWMWNGNW3aQuPGjVJYWKgkydXVVVWr1lDx4i/L1dVV77zznk6d+lXh4eHKmNFNkZER8c4RGRkhNzc3SVKnTl1UuHBRvfNOW73/fldVrVpdTk5O8vHxSZnJAgCQRhGaAQCwsbi4OOuO2X9lNpv1+PFj3b37dLOvQoUKxWs3mUzW3/v7F9CtWzfjBecLF85bd952dc2g/v0Hae3aIK1cuU6enplVtGhxOTjwXz0AAP8G/5MCAJCMwsJCtWPHVkVGRiouLk6//HJAO3ZsVbly5XX48EGdO3dWcXFxiogI16xZ05QpUybly/d0s6+GDZvop5/26Pz5PxQbG6uFC+epRIlS8vDwUN68+VSoUBHNn/+NoqOj9eOPu3Xx4nlVq1ZLknT3boj+/POuLBaLTp8+xc7ZAAAkE+5pBgAgWZm0du1qTZ48QWazRTly5FCfPgNUuXI17dq1Q9Omfa67d0Pk6uqq4sVf1pQpM+Xq6ipJKlu2vLp376mPPvpQjx8/VokSpTRq1DjrmceM+VTjx49WgwY1lT17dn3yyUR5e3tLero797Ol3tmyZdf77/fSa6+9bpevAAAAaYnJEn+rTTzHvXvhMpv5cv1V1qyZdPPLnvYuI1nk6vml7t59ZO8y8B+RNWsmTVxez95lJItBbbbyZx8AAKRrDg4m+fomvnEmy7MBAAAAADBAaAYAAAAAwAD3NAMA8IIyeWVQBmdne5eRLB7HxOjR/cf2LgMAgFSL0AwAwAvK4OysRmum27uMZLGp+Yd6JEIzAABGWJ4NAAAAAIABQjMAAAAAAAYIzQAAAAAAGCA0AwAAAABggNAMAAAAAIABQjMAAAAAAAYIzQAAAAAAGEhyaL569ap++umneMdOnjyp999/X23atNGKFSuSvTgAAAAAAOzJKakdJ0+erPv376tq1aqSpNDQUL333nuKjIyUq6urRo8eLV9fX9WuXdtmxQIAAAAAkJKSfKX59OnTeuONN6yvN23apPDwcK1Zs0YHDhxQyZIltWjRIpsUCQAAAACAPSQ5NIeGhipbtmzW13v37lWZMmVUpEgRubi4qGHDhrp48aJNigQAAAAAwB6SHJozZsyoR48eSZLi4uJ09OhRlStXztqeIUMGhYeHJ3+FAAAAAADYSZJDc+HChbV27VqFhYXphx9+UGRkpCpVqmRtv3nzpnx8fGxSJID/prFjRyggoJ7q1q2mNm2aa8OGtZKkmJgYDR/+sVq2fFOVK5fTsWNH4o2zWCz66qsZatiwlho2rKWvvpohi8UiSTp58rjq1KkS71flyuW0Z8/OBO/ft28PVa5cTrGxsTafKwAAANKmJG8E1rVrV33wwQfW+5qLFy8e70rz/v379dJLLyV/hQD+s9q376zBg0fIxcVFV69eUe/e3VW4cFEVLFhIJUqUUqtWbTVy5KAE49atW6O9e/do4cKlMplM6tevp3LmzKmmTVuqZMnS2r59r7XvsWNHNGhQf1Wo8Ea8c2zbFkRYBgAAwL+W5CvN1atX16JFi9SpUyf17NlT8+fPl8lkkiSFhYUpR44cat68uc0KBfDfU6BAQbm4uEiSTKanv27evCFnZ2e99VZblSxZSg4OjgnGbdmySW3atFe2bNmVNWs2tWnTTps3b0z0PbZs2aQaNWopY8aM1mPh4eGaP/8b9ejRxzYTAwAAQLqR5CvNklS+fHmVL18+wXFvb2/NmjUr2YoCkHZMnvyZgoI2KDo6WkWKFFXFipX+dszlyxdVqFAR6+tChYro8uVLCfpFRUVp9+6dmjhxarzjc+Z8qWbNWsjX1/ffTwAAAADpWpKvND8TGRmpn3/+WevXr9eff/5pi5oApCEDBw7Wtm0/6csv56lq1RrWK8/PExUVJQ8PD+trd3cPRUVFWu9rfubHH3fJy8tLpUuXtR47e/aMTp06qRYtWiffJAAAAJBuvVBoXrp0qapWraouXbpo0KBBOn/+vCTp3r17evXVV/XDDz/YpEgA/22Ojo4qWbKU7t4NUWDgqr/tnzFjRkVE/G83/oiICGXM6Ga9JeSZoKBNql+/kfW42WzWlCmfqW/fAXJyeqGFNAAAAECikhyat27dqrFjx6pChQoaN25cvCs+vr6+qlKlinbs2GGTIgGkDXFxcbp588bf9vP3L6gLF85bX1+4cE7+/gXi9blz57ZOnDiq+vUbWY9FRETo7NnfNWrUUDVpUk/vvddRktS8eSOdPHk8mWYBAACA9CTJofnbb79VhQoV9OWXX6pWrVoJ2l955RXrlWcACAsL1Y4dWxUZGam4uDj98ssB7dixVeXKPd0X4cmTJ4qOjpYkxcbGKjo62vphXP36DbVixRLdvRuiP/+8q+XLl6hhw8bxzr9162a98koJ5cqV23rMw8NDa9cGacGCJVqwYIk+//wLSdK3336nl156JSWmDQAAgDQmyesXz507p4EDBxq2Z82aVffu3UuWogCkBSatXbtakydPkNlsUY4cOdSnzwBVrlxNktS2bQvdvh0sSerfv5ckaeXK9fLzy6mAgBa6deumOnZsI0l6880ABQS0iHf2LVs2qW3bDvHf0WSSr28W6+snT55Ikry9fViuDQAAgH8kyT9FOjg4yGw2G7aHhITEe+QLgPTt6a76cw3bV63aYNhmMpn0wQd99cEHfQ37LF26+m9r8PPLqX37jvxtPwAAAMBIkpdnFytWTPv27Uu0zWw2a8uWLXr11VeTrTAAAAAAAOwtyVea27dvr/79+2v69Olq2rSpJMlisejSpUuaNm2aLly48Nzl2wDSFi8vZzk7Z7B3GckiJuax7t+PsXcZAAAASIWSHJobNmyoP/74Q19//bXmzn265PLdd9+VxWKRxWJRr169VK1aNZsVCiB1cXbOoGUL69m7jGTxduetkgjNAAAASOiFdsbp16+f6tatqw0bNujSpUuyWCzKly+fAgICWJoNAAAAAEhzXng72Zdfflkvv/yyLWoBAAAAACBVSfJGYAAAAAAApDeGV5pnzZr1wiczmUzq2bPnvyoIAAAAAIDUgtAMAAAAAIABw9C8c+dOm7/5Bx98oBs3bsjBwUFubm4aMWKEihcvrsuXL2vw4MG6f/++vLy8NHHiROXPn1+SbNIGAAAAAEBiDENzrly5bP7mEydOVKZMmSRJO3bs0NChQxUYGKhRo0apbdu2CggI0Lp16zRy5EgtXrxYkmzSBgAAAABAYpK8EVhsbKzCw8MN28PDwxUbG/tCb/4sMD8bbzKZdO/ePZ05c0aNGzeWJDVu3FhnzpxRaGioTdoAAAAAADCS5EdOffbZZ9q7d6+2bt2aaHuLFi1Uo0YNDR48+IUKGDZsmPbv3y+LxaJ58+YpODhY2bNnl6OjoyTJ0dFR2bJlU3BwsCwWS7K3+fj4vFC9AAAAAID0I8mhed++fapbt65he7169bRjx44XDs3jx4+XJK1du1aTJk1S3759X2h8SvL19bB3CbCxrFkz/X0npEnp/Xuf3uef3vH9BwDAWJJD8+3bt5U3b17D9jx58ig4OPgfF9K0aVONHDlSOXLk0J07dxQXFydHR0fFxcUpJCREfn5+slgsyd72Iu7dC5fZbPnHc0yL0toPWnfvPrJ3Cf8Z6f17n97nn97x/QcAIG1xcDAZXiRN8j3Nzs7OCgkJMWy/e/euHBySfDpFRETEC9m7du1S5syZ5evrq+LFi2vjxo2SpI0bN6p48eLy8fGxSRsAAAAAAEaSfKW5WLFi2rJli9577z25uLjEa4uJiVFQUJCKFi2a5DeOiopS3759FRUVJQcHB2XOnFlff/21TCaTRo8ercGDB+urr76Sp6enJk6caB1nizYAAAAAABKT5NDcvn179e3bV927d1f//v1VtGhRmUwmnT17VtOmTdOFCxc0ZcqUJL9xlixZ9MMPPyTaVrBgQa1cuTLF2gAAAAAASEySQ3O9evXUvXt3zZkzR2+99ZZMJpNMJpPMZrMsFovee+89NWzY0Ja1AgAAAACQopIcmiWpX79+qlWrltavX69r165JkvLnz6/GjRurRIkSNikQAAAAAAB7eaHQLEklSpQgIAMAAAAA0oWkb3cNAAAAAEA6Y3iledasWTKZTOrRo4ccHBw0a9asvz2ZyWRSz549k7VAAAAAAADs5W9D87NHTBGaAQAAAADpjWFo3rlzpyRZn8n87DUAAAAAAOmFYWjOlSvXc18DAAAAAJDWJXkjsI4dO+rAgQOG7QcPHlTHjh2TpSgAAAAAAFKDJIfmQ4cO6c8//zRsDw0N1eHDh5OlKAAAAAAAUoNke+TUw4cPrfc/AwAAAACQFhje0yxJZ8+e1dmzZ62vjxw5ori4uAT97t+/r2XLlqlgwYLJXyEAAAAAAHby3NC8Y8cO66OmTCaTVqxYoRUrViTa193dXcOGDUv+CgEAAAAAsJPnhuZmzZrptddek8ViUadOndS9e3dVqlQpXh+TySQ3NzcVKlRIrq6uNi0WAAAAAICU9NzQnCtXLuujpiZMmKDy5csrd+7cKVIYAAAAAAD2lqSNwCIiIjR06FCtW7fO1vUAAAAAAJBqJCk0u7u7y9PTU76+vrauBwAAAACAVCPJj5yqUKECz2EGAAAAAKQrSQ7NH330kY4ePaoZM2YoPDzcljUBAAAAAJAqPHcjsL/q3LmzoqOjNXv2bM2ePVs+Pj7KkCFDvD4mk0k7duxI9iKB/6InT55oypTPdOTIIT18+FC5cuVW9+49VbHi0x3oHz9+rFmzpmv37u2KjY1VoUJF9OWX38Q7R0xMjDp3fluRkZEKDNxsPV65cjllyJBBJpNJklSrVl0NHjxCkrRixRKtWvWDHjy4r4wZM6pWrTr64IO+cnJK8l93AAAAAP9fkn+Kzpkzpy3rANKcuLg4ZcuWXbNmzVX27Dl04MB+jRw5RIsXL5efX05NmjRecXGx+v77VfL09NT58+cSnGPp0sXy8vJWZGRkgraFC5cpd+48CY5XrlxNDRs2UaZMmfTw4QMNHz5Iq1YtV5s27W0yTwAAACAtS3Jo/u6772xZB5DmZMyYUV27dre+rlSpinLmzKk//vhdT5480b59PykwcJPc3T0kScWKFY83/tatm9q2LUi9evXTpEnjk/y+uXL977FwFotFJpODbty48S9nAwAAAKRPSb6nOSlOnDiRnKcD0pTQ0Hu6fv2a/P0L6syZ08qRI4e+/XaOGjWqpY4dW2vPnp3x+k+f/rm6despV1fXRM/Xs+d7atKknoYO/UjBwbfitW3btkV161ZTo0a1dfHiOQUENLfZvAAAAIC07F+H5tDQUM2fP1+NGjXS22+/nRw1AWlObGysxowZofr1Gylfvvy6ezdEly5dlLu7h9au3aJ+/T7W+PGjdeXKZUnSjz/uVlycWdWq1Uj0fLNmzdWqVRu0ZMkqZcmSRR9//KFiY2Ot7XXr1te2bT9q2bI1CghoIR8fnxSZJwAAAJDW/KPQbDabtXv3bvXq1UtVq1bVpEmTdP/+fbVq1Sq56wP+88xmsz75ZIScnZ3Uv/8gSZKrq6ucnJzUqVNXOTs7q3TpsipdupwOHTqoqKgozZ49Qx9+ONDwnKVKlZGzs7MyZcqkvn0HKjj4lq5evZKgX548eeXvX0BTpky01fQAAACANO2FttO9cuWKVq9erbVr1+rPP/+UJNWpU0cdOnRQuXLlrDv5AnjKYrHos88+UWhoqCZP/sK6g3XBgoUT9H321+fGjWsKDr6lnj3fk/R0B+2IiHA1aVJPc+YskJ9fwk35TCaTLBZLojXExcXp5k3uaQYAAAD+ib8NzY8fP1ZQUJBWrVqlY8eOydHRUdWrV9frr7+ucePGqXHjxipfvnxK1Ar850yePEFXrlzW9OlfydX1f49oK1WqjLJnz6Hvv1+o9u0768yZ0zp27Kg++KCvcuXKrTVrNln7nj79q6ZOnaT587+Xl5e3Ll26qLi4WBUoUEjR0dH65pvZypIlq/Ln95ckbdiwVpUrV5W3t48uX76k775bqAoVXk/xuQMAAABpwXND8/DhwxUUFKSIiAi99NJLGjp0qBo3bixvb29du3YtpWoE/pNu3w7WunVr5OLiooCAetbjH300VHXrNtCECVM0ceI4ff/9QuXI4afhw8coX778kiRf3yzW/pkyecrBwcF6LCwsVJMnT9DduyHKkCGjXn21hCZNmm69in3q1EnNnfuVoqIi5eXlrRo1auvdd99PuYkDAAAAachzQ/OqVauUL18+LViwQCVKlEipmoA0IUcOP+3bd8SwvUCBgpozZ8HfnqdMmXIKDNxsfV22bHktW7bGsP/QoaNerFAAAAAAhp67Edirr76qq1evqkuXLho+fLiOHDEOAAAAAAAApDXPvdK8cuVKnT9/XitXrtSGDRu0evVq5cqVS02bNlWZMmVSqkYAAAAAAOzibzcCK1y4sIYOHaqPPvpIO3bs0OrVq/XVV1/JYrHIZDLp+PHjKl++vLy9vVOiXiBV8c7sIicXV3uXkSxin0Qr7METe5cBAAAApCpJfuSUs7OzGjRooAYNGuj27dtavXq1AgMDtWDBAn333XcqW7as6tatq3bt2tmyXiBVcXJx1d5vGtu7jGRR5b2NkgjNAAAAwF89955mIzly5FDPnj21Y8cOLViwQPXq1dOJEyc0bty45K4PAAAAAAC7SfKVZiMVK1ZUxYoV9ejRI61fvz45agIAAAAAIFX4R1eaE5MpUyaWZgMAAAAA0pRkC80AAAAAAKQ1hGYAAAAAAAwQmgEAAAAAMEBoBgAAAADAgGForlWrlnbu3Gl9PWvWLJ07dy5FigIAAAAAIDUwDM3BwcGKiIiwvp41a5b++OOPFCkKAAAAAIDUwDA0Z8+ePcGVZZPJZPOCAAAAAABILZyMGmrVqqV58+Zp7969ypw5syRp9uzZ+uGHHwxPZjKZtGjRouSvEgAAAAAAOzAMzQMHDpSnp6d+/vln3bp1SyaTSaGhoYqKikrJ+gAAAAAAsBvD0JwhQwb16dNHffr0kSQVK1ZMQ4cO1ZtvvplixQEAAAAAYE9JfuTUhAkTVLp0aVvWAgAAAABAqmJ4pfn/atasmfX3YWFhunHjhiQpd+7c8vb2Tv7KAAAAAACwsySHZkk6e/asxo0bp6NHj8Y7Xq5cOQ0bNkzFihVL1uIAAAAAALCnJIfmc+fO6e2339aTJ09Uq1YtFSpUSJJ04cIF7d69W+3atdPy5ctVuHBhmxULAAAAAEBKSnJonjFjhpydnbVs2bIEV5TPnTun9u3ba8aMGZo5c2ayFwkAAAAAgD0kOTQfPnxYbdu2TXQJdpEiRfT2229r+fLlyVrcf92TJ080ZcpnOnLkkB4+fKhcuXKre/eeqlixki5fvqRx40bp5s2n94YXLVpcH344UP7+BSRJS5cuVlDQRt2+fVteXl5q1qyl2rbtGO/8P/ywTCtXLlNYWKiyZ8+hCROmKG/efDp27IimT/9cISF35ODgqJIlS6t//4+VNWu2FP8aAAAAAMB/WZJDc1RUlLJmzWrYni1bNp7h/H/ExcUpW7bsmjVrrrJnz6EDB/Zr5MghWrx4ubJkyapx4yYqRw4/mc1mrVmzUqNHD9WiRU8/eLBYLBo+fKwKFiykW7duqF+/XsqWLbtq164nSdqwYa02bVqnSZOmK39+f926dVOZMmWSJOXP76+pU2cpS5asevLkiebNm63Jkydo4sRpdvtaAAAAAMB/UZIfOZUnTx7t3r3bsH337t3KkydPshSVVmTMmFFdu3aXn19OOTg4qFKlKsqZM6f++ON3ZcqUSX5+OWUymWSxWOTg4KAbN65bx7Zr10lFixaTk5OT8ubNrypVqunUqZOSJLPZrAULvlHv3v3l719AJpNJuXLllqdnZkmSj4+vsmT53wccDg6O1ivaAAAAAICkS/KV5oCAAE2dOlUDBgzQ+++/rwIFni4jvnjxoubMmaP9+/drwIABNis0LQgNvafr16/J37+g9Vj9+tUVFRUls9msrl27JzrOYrHo5MnjCghoLkkKCQlRSMgdXbp0UZ9+OkaOjo6qX7+R3nnnPTk4PP0c5Pbt2+rcuY0iIiLk4OCgQYOG236CAAAAQDr0vNsyT58+pXnzZuuPP87K0dFBpUqV1YcffqQsWbJIkh49eqQvvpisgwd/liQ1a9YyXi745pvZ2rt3j65evaKOHbvEa7NYLFq8eL7WrVuj8PBwVaz4hj7+eJjc3T1ScvppXpJDc9euXXXmzBlt2rRJmzdvtoYzs9ksi8WiBg0aqEuXLjYr9L8uNjZWY8aMUP36jZQvX37r8S1b9igqKkpBQRuVI4dfomPnz58rs9mihg2bSJLu3r0jSTp8+KAWLVqu8PBH6tevl7JmzaYmTZ4+TztHjhzasmWPHj58oPXrA5U3b/5Ezw0AAADg33nebZmPHj1UkybNVaHC63J0dNLUqRP16adjNHXq0w2UZ86cqsePH2vVqg0KCwtV3749lCOHnxo1evqzf+7cedSjRx+tW7c6wftu2bJJW7du1uzZ3ypTJk+NHTtc06Z9ruHDx6To/NO6JC/PdnR01PTp0/Xtt9+qTZs2euONN/TGG2/o7bff1vz58zVt2jRrkEZ8ZrNZn3wyQs7OTurff1CC9owZM6pp0xYaN26UwsJC47WtXr1CW7Zs0uefT5eLi4skydXVVZLUtm1H6zLvgIDmOnBgf4Jze3pmVoMGjTVkyADFxsbaYHYAAABA+va82zIrVqykmjVry93dQxkyZFCLFq2tt11K0v79P6ldu47KkCGD/PxyqnHjAG3atN7a3qBBY1WsWElubu4J3nf//p/UuHGAsmfPITc3N7Vr10m7dm3X48ePU2Te6UWSrzQ/U6lSJVWqVMkWtaRJFotFn332iUJDQzV58hdyckr8S242m/X48WPdvRsib28fSdLGjev0/feLNGvWXGXLlt3aN2/e/HJ2dpbJZLIe+8tvE4iLi1NYWKgiIyOs9z0DAAAAsI3Ebst85uTJY9Yn5jxjsVji/f7SpYtJfq//O/bJkye6fv2aChcu8g8qR2K4NGxjkydP0JUrlzVx4jS5umawHj98+KDOnTuruLg4RUSEa9asacqUKZPy5fOXJG3bFqS5c7/StGlfKleu3PHOmSFDBtWsWUdLly5WZGSEQkLuaP36QFWqVFmS9OOPu3Tt2hWZzWaFhYVp5sxpKlKkKIEZAAAAsDGj2zIl6cKF81qwYJ569uxrPVahwhv6/vtFioyM0I0b17Vp03pFRyftSnGFCm9ow4Z1Cg6+pfDwcC1ZskiSkjweSfPCV5qRdLdvB2vdujVycXFRQEA96/GPPhoqJydnTZv2ue7eDZGrq6uKF39ZU6bMtC69/uab2Xrw4L7ee+9/z2auW7eBPvpoqCSpf/+PNWnSeAUENFCmTJn05ptN1ahRgCTp7t0QzZo1XWFhoXJzc1fp0mX06aeTU3DmAAAAQPrzvNsyb9y4roED+6hv3wEqWbK09fiHHw7UtGmfq02b5vL0zKzatetpx46tSXq/Ro2aKCTkjnr37q64uDi1adNO+/fvVdas2ZJ1XukdodmGcuTw0759Rwzba9asbdi2cuV6wzZJcnf30JgxExJta9myjVq2bJO0IgEAAAD8a8+7LfP27WB9+OEH6ty5q+rXbxRvnKdnZo0aNc76es6cL1W8+MtJek8HBwd17drduqP2oUMHlTVrNkJzMmN5NgAAAAD8S0a3Zd69G6I+fd5X8+ZvqWnTlgnG3bx5Qw8e3FdcXJwOHNiv9evXqFOnrtb22NhYRUdHy2w2Ky4uTtHR0YqLi5MkPXz4QDdv3pDFYtHly5c0c+ZUde78Lhs0JzOuNAMAAADAv/C82zJv3LiuW7duasGCuVqwYK61bfv2vZKks2d/14wZUxQe/kh58uTTyJHjVKDA/zYQmzhxnIKCNlpfL148X0OHjlLDhm/q/v37GjSon0JC7sjLy1utWrVRQEDzFJhx+pKk0Pz48WNt2bJF/v7+KlmypK1r+k/xyZxBji7O9i4jWcQ9iVHoAzYNAAAAAF7E392W2aVLN8O2WrXqqFatOobtw4aN1rBhoxNty5s3n5YtW5PkOvHPJCk0u7i4aPjw4Ro2bBih+f9wdHHW3dnf27uMZJG1R3tJhGYAAAAAeCZJi90dHBzk5+en8PBwW9cDAAAAAECqkeR7mps2bar169erU6dOcnFxsWVNAAAAAJBq+GR2k6OLo73LSBZxT+IU+iDS3mX8pyQ5NJcpU0bbt29XQECA2rZtq3z58iljxowJ+pUvXz5ZCwQAAAAAe3J0cdTtz6/au4xkkeOjfPYu4T8nyaH5nXfesf5+/PjxMplM8dotFotMJpN+//335KsOAAAAAAA7SnJonjBhgi3rAAAAAAAg1UlyaG7WrJkt6wAAAADwH/XkyRNNmfKZjhw5pIcPHypXrtzq3r2nKlaspJiYGI0ZM0xnz/6u27eDNWPG1ypTppx17IoVS7Rq1Q968OC+MmbMqFq16uiDD/rKyel/UeWHH5Zp5cplCgsLVfbsOTRhwhTlzZtPx44d0fTpnysk5I4cHBxVsmRp9e//sbJmzWaPLwPSqCTtng0AAAAARuLi4pQtW3bNmjVXW7fu0Xvv9dDIkUMUHHxLklSiRCmNGPGJfH19E4ytXLma5s//Xtu2/ajvvluhCxfOa9Wq5db2DRvWatOmdZo0abq2b9+rSZOmy8vLS5KUP7+/pk6dpS1b9mjt2iDlyZNHkyezQhbJ64VCc3BwsIYMGaKqVavqlVde0YEDByRJoaGhGjJkiH799VebFAkAAAAg9cqYMaO6du0uP7+ccnBwUKVKVZQzZ0798cfvcnZ21ltvtVXJkqXk4JBwB+pcuXIrU6ZMkp7tk+SgGzduSJLMZrMWLPhGvXv3l79/AZlMJuXKlVuenpklST4+vsqSJav1XA4Ojrp580YKzBjpSZKXZ1+/fl2tW7dWdHS0SpUqpZ9//tna5uPjo9OnT2vVqlUqUaKETQoFAAAA8N8QGnpP169fk79/wST137ZtiyZPnqDIyAh5eXmpV68PJUkhISEKCbmjS5cu6tNPx8jR0VH16zfSO++8JweHp9f/bt++rc6d2ygiIkIODg4aNGi4raaFdCrJoXn69OlycHDQxo0b5erqqjfeeCNee7Vq1bR79+5kLxAAAADAf0dsbKzGjBmh+vUbKV++/EkaU7dufdWtW1/Xr1/Tli2b5OPjI0m6e/eOJOnw4YNatGi5wsMfqV+/XsqaNZuaNHm651KOHDm0ZcsePXz4QOvXBypv3qS9J5BUSV6e/fPPP+vtt9+Wn59fgsdNSVLOnDl1+/btZC0OAAAAwH+H2WzWJ5+MkLOzk/r3H/TC4/PkySt//wKaMmWiJMnV1VWS1LZtR2XKlEl+fjkVENBcBw7sTzDW0zOzGjRorCFDBig2NvbfTQT4iySH5vDwcGXLZrwLXUxMjOLi4pKlKAAAAAD/LRaLRZ999olCQ0M1fvykeLtfv4i4uDjrfcl58+aXs7NzvIt2iVy/izc2LCxUkZER/+i9gcQkOTT7+fnp/Pnzhu0nT55U3rx5k6UoAAAAAP8tkydP0JUrlzVx4jS5umaI1/bkyRNFR0dLerp8Ozo6WhaLRdLT3bHDwkIlSZcvX9J33y1UuXLlJUkZMmRQzZp1tHTpYkVGRigk5I7Wrw9UpUqVJUk//rhL165dkdlsVlhYmGbOnKYiRYpaNwoDkkOSP/6pU6eOli9frpYtWypr1qc71D37xGfr1q3asmWLevfubZsqAQAAAKRat28Ha926NXJxcVFAQD3r8Y8+Gqq6dRuobdsWun07WJLUv38vSdLKlevl55dTp06d1Ny5XykqKlJeXt6qUaO23n33fes5+vf/WJMmjVdAQANlypRJb77ZVI0aBUiS7t4N0axZ0xUWFio3N3eVLl1Gn346OQVnjvQgyaG5R48e2rNnj9566y2VK1dOJpNJ33zzjaZNm6Zff/1VxYsXV5cuXWxZKwAAAIBUKEcOP+3bd8SwfdWqDYZtQ4eOeu653d09NGZM4s9ebtmyjVq2bJO0IoF/KMnLsz08PLRixQq1bNlSp0+flsVi0f79+3X58mW1bdtWixcvtt6oDwAAAABAWvBCd+d7eHho+PDhGj58uEJDQ2WxWOTj45PobtoAAAAAAPzX/bMt7STrs9MAAAAApH3eXu5yck7yQtVULTbGrLD77LCNpHnh0Lx582bt2LFD169flyTlyZNHtWvXVsOGDZO9OAAAAACpg5Ozgw4uDLF3Gcni9c7Gj9IF/q8kh+bIyEj17NlTBw8elMVikaenpyTp1KlTCgoK0ooVKzR79my5ubnZrFgAAAAAAFJSktdXTJs2TQcOHFD79u21d+9eHTp0SIcOHdLevXvVvn17/fLLL5o2bZotawUAAAAAIEUlOTQHBQWpfv36GjZsmPU5zZKUNWtWDRs2THXr1lVQUJBNigQAAAAAwB6SHJrDw8NVoUIFw/bXX39d4eHhyVIUAAAAAACpQZJDc9GiRXX16lXD9qtXr6pIkSLJUhQAAAAAAKlBkkPzhx9+qB9++EG7du1K0LZjxw6tXLlS/fr1S9biAAAAAACwJ8Pds4cMGZLgWO7cudWzZ0/5+/urYMGCkqSLFy/q8uXLKlKkiDZs2KCKFSvarloAAAAAAFKQYWgODAw0HHTp0iVdunQp3rE//vhD586d06effpqkNw4LC9PHH3+sa9euycXFRfny5dPYsWPl4+OjEydOaOTIkYqOjlauXLn0+eefy9fXV5Js0gYAAAAAQGIMl2efPXv2hX/9/vvvSX5jk8mkd999V1u3btWGDRuUJ08eTZ48WWazWR999JFGjhyprVu3qly5cpo8ebIk2aQNAAAAAAAjSb6nObl5eXnF2427VKlSunXrlk6fPi1XV1eVK1dOktSmTRtt2bJFkmzSBgAAAACAEbuF5r8ym81atmyZatasqeDgYOXMmdPa5uPjI7PZrPv379ukDQAAAAAAI4b3NCfm2LFjWrJkia5evar79+/LYrHEazeZTNqxY8cLF/HJJ5/Izc1N7du31/bt2194fErx9fWwdwk2lzVrJnuXYFfMP/3OPz3PXWL+6R3ffwDpUXr+ty89z/2fSHJo/uGHHzRq1Cg5OzvL399ffn5+yVLAxIkTdfXqVX399ddycHCQn5+fbt26ZW0PDQ2Vg4ODvLy8bNL2Iu7dC5fZHP+DgrT2B+7u3Ucv1J/5p9/5p+e5S8w/veP7DyA9Ss//9qXnuacXDg4mw4ukSQ7NX3/9tYoXL6558+bJx8cnWQqbOnWqTp8+rblz58rFxUWS9Morr+jx48c6cuSIypUrp+XLl6t+/fo2awMAAAAAwEiSQ/O9e/fUtWvXZAvM58+f15w5c5Q/f361adNG0tPnQH/55ZeaNGmSRo0aFe/xUJLk4OCQ7G0AAAAAABhJcmguWLCgHj58mGxvXLhwYf3xxx+JtpUpU0YbNmxIsTYAAAAAABKT5N2z33//fS1dulR37tyxZT0AAAAAAKQaSb7SXLduXUVFRalRo0aqVauWcuXKJQeH+JnbZDKpZ8+eyV4kAAAAAAD2kOTQfPnyZc2YMUPh4eFat25don0IzQAAAACAtCTJoXnMmDEKDQ3VsGHDVK5cOXl6etqyLgAAAAAA7C7JofnEiRPq2rWrOnToYMt6AAAAAABINZK8EZiHh0eyPW4KAAAAAID/giSH5gYNGmjbtm22rAUAAAAAgFQlyaG5TZs2ioiI0AcffKADBw7o+vXrunXrVoJfAAAAAACkFUm+p7lRo0YymUw6ffq0du/ebdjv999/T5bCAAAAAACwtySH5p49e8pkMtmyFgAAAAAAUpUkh+bevXvbsg4AAAAAAFKdJN/TDAAAAABAepPkK82HDx9OUr/y5cv/42IAAAAAAEhNkhyaO3TokKR7mtkIDAAAAACQViQ5NE+YMCHBsdjYWF2/fl1r1qxR7ty51bp162QtDgAAAAAAe0pyaG7WrJlhW9euXZ/bDgAAAADAf1GybASWOXNmtWrVSvPmzUuO0wEAAAAAkCok2+7Znp6eun79enKdDgAAAAAAu0uW0BwdHa3169crS5YsyXE6AAAAAABShSTf0zxkyJBEjz948EAnTpxQaGioPv7442QrDAAAAAAAe0tyaA4MDEz0eObMmeXv768hQ4bozTffTLbCAAAAAACwtySH5rNnz9qyDgAAAAAAUp1k2wgMAAAAAIC0htAMAAAAAICB5y7Pfv/991/oZCaTSbNnz/5XBQEAAAAAkFo8NzTv2bPnhU5mMpn+TS0AAAAAAKQqzw3NSdn869ChQ/r888916tQpZc2aNdkKAwAAAADA3pK8e/b/de7cOU2ePFl79+6Vu7u7+vbtq3feeSc5awMAAAAAwK5eODQHBwfriy++0IYNG+Tg4KAOHTqoR48e8vb2tkV9AAAAAADYTZJD84MHD/T1119r6dKlevLkiRo1aqQPP/xQuXPntmV9AAAAAADYzd+G5idPnmjhwoWaN2+eHj58qEqVKmngwIEqXrx4StQHAAAAAIDdPDc0r1y5UrNmzVJISIheeuklDRw4UBUrVkyp2gAAAAAAsKvnhuYRI0bIZDLplVdeUYMGDXT27Nnn7qhtMpnUuXPn5K4RAAAAAAC7+Nvl2RaLRadOndKpU6f+9mSEZgAAAABAWvLc0Lx48eKUqgMAAAAAgFTnuaH5tddeS6k6AAAAAABIdRzsXQAAAAAAAKkVoRkAAAAAAAOEZgAAAAAADBCaAQAAAAAwQGgGAAAAAMAAoRkAAAAAAAOEZgAAAAAADBCaAQAAAAAwQGgGAAAAAMAAoRkAAAAAAAOEZgAAAAAADBCaAQAAAAAwQGgGAAAAAMAAoRkAAAAAAAOEZgAAAAAADBCaAQAAAAAwQGgGAAAAAMAAoRkAAAAAAAOEZgAAAAAADBCaAQAAAAAwQGgGAAAAAMAAoRkAAAAAAAOEZgAAAAAADBCaAQAAAAAwQGgGAAAAAMAAoRkAAAAAAAOEZgAAAAAADBCaAQAAAAAwQGgGAAAAAMAAoRkAAAAAAAOEZgAAAAAADBCaAQAAAAAwQGgGAAAAAMAAoRkAAAAAAAOEZgAAAAAADBCaAQAAAAAwQGgGAAAAAMAAoRkAAAAAAAOEZgAAAAAADBCaAQAAAAAwQGgGAAAAAMAAoRkAAAAAAAOEZgAAAAAADBCaAQAAAAAwQGgGAAAAAMAAoRkAAAAAAAOEZgAAAAAADBCaAQAAAAAwQGgGAAAAAMAAoRkAAAAAAAOEZgAAAAAADBCaAQAAAAAwQGgGAAAAAMAAoRkAAAAAAAOEZgAAAAAADNgtNE+cOFE1a9ZU0aJFde7cOevxy5cvq3Xr1qpXr55at26tK1eu2LQNAAAAAAAjdgvNtWrV0pIlS5QrV654x0eNGqW2bdtq69atatu2rUaOHGnTNgAAAAAAjNgtNJcrV05+fn7xjt27d09nzpxR48aNJUmNGzfWmTNnFBoaapM2AAAAAACex8neBfxVcHCwsmfPLkdHR0mSo6OjsmXLpuDgYFkslmRv8/Hxsc9EAQAAAAD/CakqNKd2vr4e9i7B5rJmzWTvEuyK+aff+afnuUu2n/+NGzc0ZswYnThxQi4uLqpXr56GDh0qJycnHThwQJMmTdLVq1fl7e2tbt26qXXr1tax3333nRYsWKD79+8rf/78Gjp0qMqVKydJmjlzpr7++mu5uLhY+69fv1558uSx6XzSmvT+5x9A+pSe/+1Lz3P/J1JVaPbz89OdO3cUFxcnR0dHxcXFKSQkRH5+frJYLMne9qLu3QuX2WyJdyyt/YG7e/fRC/Vn/ul3/ul57hLzf1HDho2Qt7ePAgODFB7+SP369dTcuQvUrFlL9ezZUz169FFAQHOdPXtGffq8rzx5Cqlw4SL67bfTmjx5smbN+kZFixbT2rWr1bNnT61bt1WOjo6KiIhWzZp1NHLkJyk6H77/ANKj9PxvX3qee3rh4GAyvEiaqh455evrq+LFi2vjxo2SpI0bN6p48eLy8fGxSRsAIGUEB99SzZq15erqKl/fLKpQ4Q1dvnxRDx8+UEREhOrXbySTyaTixV9Wvnz+unLlkiTp9u1b8vcvoGLFistkMql+/Ua6f/++wsLC7DwjAACQXtgtNI8bN05Vq1bV7du39c4776hRo0aSpNGjR+v7779XvXr19P3332vMmDHWMbZoAwDYXqtWb2vHjm16/Pix7t4N0cGD+1Whwhvy8fFV7dr1tGnTesXFxen06V91+3awSpQoJUl6/fU3FBdn1m+/nVZcXJw2bVqnwoWLyNfX13ru/ft/UoMGNdW+/VsKDFxlpxkCAIC0ym7Ls4cPH67hw4cnOF6wYEGtXLky0TG2aAMA2F6pUmW0fn2g6tWrpri4ODVo0FhVq1aXJNWuXU8TJ47TjBlTJEkDBgxW9uw5JElubu6qXr2mPvigqyTJw8NDkyfPkMlkkiTVrFlHAQHN5e3tozNnTmv48I/l4eGhOnXqp/wkAQBAmpSqlmcDANIes9msAQN6q1q1Gtq+fa82bdqhR48eavbsGbp69YpGjx6q4cPHaPfuA/ruuxVaunSxfv55nyRp48Z12rx5g7777gft3n1AI0Z8oo8/7qc//7wrSfL3L6AsWbLK0dFRr75aUi1bvq09e3bac7oAACCNITQDAGzq4cOHunPntlq0aC0XFxdlzuylhg2b6MCB/bp06YLy5MmrChUqysHBQXnz5lfFipV18OB+SdL583/ojTcqK2/efHJwcNDrr78hX98sOnXqZKLvZTJJFkuiTQAAAP8IoRkAYFNeXl7y88ulwMBVio2N1aNHjxQUtFEFCxZWkSLFdOPGdR09elgWi0U3b97Qzz/vVcGChSVJxYu/rJ9/3q+bN2/IYrHo8OGDun79qgoUKCRJ2rt3jx4+fCiLxaIzZ05r1aoVqlKlmv0mCwAA0pxU9cgpAEDa9Omnk/TFF1O0ZMkiOTo6qEyZ8urTp798fHw1ePBITZ/+uW7fvm29H/nNN5tKkurXb6SbN2+od+/uevTokbJly6aPPhqqfPnyS5J27NimCRM+UUzME2XNmk3t2nVSgwaN7TdRAACg4OBbmjLlM50+fUouLi6qXr2m+vQZICcnJ50//4cmTPhEV69eVr58/hoyZIQKFy4ab3xMTIw6d35bkZGRCgzcbD2+b99PmjNnlm7fDlbBgoU1aNBw+fsXsPl8CM0AAJsrXLioZs2am2hbrVp1VKtWnUTbTCaT3n33fb377vuJto8Z82my1QgAAJLHlCmfydvbR+vWbVF4+CP169dTgYGr1LRpCw0ePEBvvfW2mjVrpXXr1mjw4AFavjxQzs7O1vFLly6Wl5e3IiMjrceuX7+msWNH6PPPv9DLL7+iZcu+0+DB/bVkySo5Odk21rI8GwAAAACQbIKDb6lmzdpydXWVr28WVajwhi5fvqjjx48qLi5Ob73VVi4uLmrVqo0sFouOHj1sHXvr1k1t2xak9u07xzvnoUMHVLJkKZUsWUpOTk5q166T7t69qxMnjtl8PlxpBgC8sExeLsrg7GrvMpLF45hoPbr/xN5lAACQZrRq9bZ27Nim0qXL6dGjhzp4cL/efbeHLl++qIIFC1sfHSlJhQoV1uXLl/T6629IkqZP/1zduvWUq2vCnzMsf9nt8+nvLbp06aLKlXvNpvMhNAMAXlgGZ1c1WNfJ3mUki6CARXokQjMAAMmlVKkyWr8+UPXqVVNcXJwaNGisqlWra9Gib+Xh4R6vr7u7hyIjIyRJP/64W3FxZlWrVkPHjh2J169cuQqaPXumjh07oldfLaklSxYpJiZG0dGPbT4fQjMAAACQzOrUqRLvdXR0tJo1a6l+/T5WcPAttWrVRBkzZrS2t2vXSZ07vytJevLkiSZPnqA9e3YpQ4YMatu2g9q0aS9Junz5ksaNG6WbN29IkooWLa4PPxyYIpshAUlhNps1YEBvNWnSTF9/PV9RUZGaMGGsZs+eIV/fLIqIiIjXPyIiQm5u7oqKitLs2TP0+edfJHrefPnya9iw0Zo2bZLu3bununUbKH9+f2XNms3mcyI0AwAAAMls+/a91t9HRkYqIKCeatSoHa9PUNDuRDcwmj9/rm7cuK5VqzYoNPSe+vR5X/nzF9Drr7+hLFmyaty4icqRw09ms1lr1qzU6NFDtWjRcpvPCUiKhw8f6s6d22rRorVcXFzk4uKihg2b6JtvvlLv3v21fPkSWSwW6xLtixfPq3nzVrpx45qCg2+pZ8/3JD3dQTsiIlxNmtTTnDkL5OeXUzVq1Lb+PXr06JE2bVqn4sVftvmc2AgMAAAAsKEff9wlLy8flSxZOkn9g4I2qnPnd+Xp6an8+f315ptNFRS0QZKUKVMm+fnllMlkksVikYODg27cuG7L8oEX4uXlJT+/XAoMXKXY2Fg9evRIQUEbVbBgYZUuXVYODg5auXK5njx5otWrV0iSypYtL3//glqzZpMWLFiiBQuWaPDg4fL29tGCBUuULVt2SdLZs78rLi5OYWFhmjRpvCpVqmp9DKUtcaUZAAAAsKGgoI2qX79hvM2PJKllyzdlMplUvnwFffBBX3l5eenhw4e6d+9PFSpU2NqvUKHC2rt3T7yx9etXV1RUlMxms7p27W77SQAv4NNPJ+mLL6ZoyZJFcnR0UJky5dWnT385OztrwoTJ+uyzcfr661nKnz+/JkyYbH3clK9vFus5MmXylIODQ7xjX3wxWRcunJeTk5Nq1Kil3r37p8h8CM0AAACAjdy+HawTJ45p8OAR1mOZM3tp3rzFKlSoiB4+fKCpUydq7Njhmjp1lqKinj6X1t3dw9rfw8Mj3vNqJWnLlj2KiopSUNBG5cjhlzKTAZKocOGimjVrbqJtRYoU0/z53//tOcqUKafAwM3xjs2e/W2y1PeiWJ4NAAAA2MiWLZtUokQp5cyZy3rMzc1NxYq9JCcnJ/n4+Kpfv4916NBBRUZGKGNGN0my7iYsPdsoyS3BuTNmzKimTVto3LhRCgsLtf1kgHSK0AwAAADYyJYtm1W/fqPn9nm2bNtstsjT01O+vll04cI5a/uFC+cNd8c2m816/Pix7t4NSb6iAcTD8mwAAADABk6dOqk//wxRzZrxd83+7bfTypTJQ7lz59WjRw81ffpklS5dVh4eT5dk16/fSIsWzVfRoi8pLCxUGzYEasiQUZKkw4cPKnNmLxUsWFiPH0fpm29mK1OmTMqXzz/F54f0wydzRjm6pI3oGPckVqEPol5oTNqYOQAAAJDKBAVtVLVqNeTm5h7v+K1bNzR37lcKCwuVu7u7ypWroNGjx1vbu3btrsmTJ6hlyzfl6uqqdu066vXX35AkPXoUrmnTPtfduyFydXVV8eIva8qUmXJ1dU3RuSF9cXRx0p0ZP9m7jGSRvU/VFx5DaAYAAABs4OOPhyV6vE6d+qpTp77hOBcXFw0dOkpDh45K0FazZu0EV64B2Bb3NAMAAAAAYIDQDAAAAACAAZZnAwAAAEng5eUuZ+e0cc0pJsas+/cj/r4jAEIzAAAAkBTOzg5atOauvctIFp2aZ7V3CcB/Rtr4qAwAAAAAABsgNAMAAJvbsWOr2rVrqdq1K+uttwJ08uRxSdKRI4fUtm0L1apVSb17d9ft28Hxxh0+/Iu6dGmn2rUrq1mzhtq5c7u1bd++n9Shw1uqU6eK3n+/iy5fvpSicwIApA+EZgAAYFOHDx/U7NkzNWTIKG3b9pO+/PIb5cyZS/fv39ewYR/p3Xd7aPPmXSpW7CWNHDnEOu7y5UsaM2a43nvvA23ZskcLFy5VsWLFJUnXr1/T2LEjNHDgUAUF7ValSlU0eHB/xcbG2muaAIA0itAMAABs6ttv5+qdd97VK6+8KgcHB2XNmk1Zs2bTjz/ukr9/QdWsWVuurq7q0qWbLlw4r6tXr0iSFi36VgEBzVWxYiU5OTkpc2Yv5cqVW5J06NABlSxZSiVLlpKTk5Pateuku3fv6sSJY3acKQAgLSI0AwAAm4mLi9PZs2cUFnZfrVs3VbNmDTV16kRFRz/W5cuXVKhQYWvfjBkzKleuXLp8+aIk6bffTkuSOnZsrYCAeho7doQePnxg7W+xWP7P7y26dOliykwMAJBuEJoBAIDNhIWFKjY2Vnv27NSXX87TggVLdf78H1q48FtFRUXK3d0jXn8PDw9FRkZKku7evaOtWzdr3LhJWrYsUNHRjzVt2ueSpHLlKujEiWM6duyIYmJi9N13CxQTE6Po6McpPse/06tXN9Ws+Ybq1KmiOnWq6O23m0uSjh07oipVyluP16lTRUFBG63jrly5rD593le9etXUunVT/fjjbmtbcPAtVa5cLt7YhQvnpfjcACA94JFTAADAZlxcXCVJLVu2VpYsWSRJrVu306JF36pkyTKKiIj/nNiIiAi5ublJklxdXdWw4ZvKmzefJKlDhy768MMPJEn58uXXsGGjNW3aJN27d0916zZQ/vz+ypo1W0pN7YX06/ex3nyzaYLjWbJkVWDg5gTHY2NjNXjwADVt2lzTpn2pEyeOadCgfvL3X2L9ekhSUNBuOTml3h/nevXqpjNnTsvR0VHS0/kuW7ZGx44dUd++PZQhQwZr3/79B6lBg8aSpLFjR+jo0UOKinosHx9ftWvX0fr1i4mJ0Zgxw3T27O+6fTtYM2Z8rTJlyqX43ACkH6n3X1kAAPCf5+npqWzZssc7ZjKZJEn+/gW0Zcv/rqxGRUXp5s0b8vcvKEkqWLCwte9fxz1To0Zt1ahRW5L06NEjbdq0TsWLv2yTeaS0a9eu6N69u2rdup1MJpPKli2vV18tqa1bN+u993rYu7wX8qIfGEhS+/adNXjwCLm4uOjq1Svq3bu7Chcuat0IrkSJUmrVqq1Gjhxky9IBQBLLswEAgI01bPimVq/+QWFhoXr48KFWrFiqN96ooqpVa+jSpYvas2enoqOjtWDBNypYsLDy5ctvHbd58wbdvHlDjx8/1vffL1SlSpWt5z179nfFxcUpLCxMkyaNV6VKVa1jU5s5c2apUaNa6tGji44dO2I9HhYWqjffrKtWrZpoxowpioqKMjyHxZLwnu2WLd9Us2YN9emnY3T//n1blZ/iChQoKBcXF0mSyfT0182bNyRJzs7OeuuttipZspQcHBztWSaAdILQDAAAbKpz53dVrNhLevvt5mrfvpWKFCmqjh27yNvbW+PGTdLcuV+pQYOaOnPmtMaM+dQ6rnHjANWv30jdu3dWy5aN5eLirL59P7K2f/HFZNWvX0Nt27ZQpkyZNGjQcHtM72/16NFHP/ywToGBQWrSpLkGDeqvmzdvKF++/FqwYKnWrduiL76YrT/+OKuZM6dKkvLmzS8vLx8tXbpYsbGxOnTooE6cOGa9ZztzZi/Nm7dYq1Zt0LfffqfIyAiNHZs65/9PPzCYPPkz1apVSW3btpSvbxZVrFgppUsHAEkszwYAADbm5OSkgQMHa+DAwQnaypevoKVLVxuO7dq1u7p27Z5o2+zZ3yZbjbb08suvWH/foEFjbd++VQcO7FPLlm3k6/v0Pu+cOXOpR48+GjToQ3388TA5OTlpwoTJmj79cy1ZsljFihVXzZp15OzsLElyc3NTsWIvSZJ8fHzVr9/HCgior8jICLm5uaf8JA306NFH/v7+cnJy1s6d2zRoUH8tXLjU+oFBvnz5dft2sMaPH62ZM6fq44+HWccOHDhY/fp9pNOnT+n48SPWK88AkNK40gwAAJCCTCaT/vK0rHjHzeb/NRQqVFizZs3V5s07NXXqLN26ddPwnu1n93v/dXxq8PLLr8jNzV0uLi5q0KCxXn21pA4c2Cdf3yzy9y8gBwcH6wcGP/64K8F4R0dHlSxZSnfvhigwcJUdZgAAhGYAAACbefTokX755YCio6MVGxurbduCdPLkMVWoUFHHjh3R7dvBslgsunPntr7+eqaqVKlmHXvhwnlFR0fr8ePHWrr0O92796caNnxT0tNnWF+7dkVms1kPHtzX9OmTVbp0WXl4eBiVkiok9QOD/ysuLs56TzMApDSWZwMAgCTL5JVBGf7/EuH/uscxMXp037bPdY6NjdU338zW1atX5OjooLx582vChMnKmzeffv55r8aOHaFHjx4qc2YvVa1aXd26fWAdu3XrZm3YsFZxcbEqUaK0pk370rpE+datG5o79yuFhYXK3d1d5cpV0OjR4206lxf16NEjnTlzWqVKlZGjo6N27dqukyePqW/fATp27Ihy5syl7NlzKCTkTrwPDMLCQnX06GG98UYVubq66siRQ9qxY2u8+T158kSW/5++Y2NjFR0dLRcXlwQ7rANAciA0AwCAJMvg7KzGq5bYu4xksbFlOz2SbUOzt7e35s1bnGhbmzbt1aZNe8OxPXv2Vc+efRNtq1OnvurUqZ8sNdrKP//AwKS1a1dr8uQJMpstypEjh/r0GaDKlf93Fb5t2xa6fTtYktS/fy9J0sqV6+XnlzOlpwkgHSA0AwAAINn90w8MvL29NWvW3Oeee9WqDf+6PgBIKu5pBgAAAADAAFeaAQAAkiiTV0ZlcE4bPz49jonVo/tRf98RANK5tPGvPgAAQArI4OykgFVb7V1GsljXsp4eveCYTF5uyuDsaJN6UtrjmDg9uh9p7zIA/AcQmgEAAJAkGZwd9dbqs/YuI1n80KLYC39oACB94p5mAAAAAAAMEJoBAAAAADBAaAYAAAAAwAChGQAAAAAAA4RmAAAAAAAMEJoBAAAAADBAaAYAAAAAwAChGQAAAAAAA4RmAAAAAAAMEJoBAAAAADBAaAYAAAAAwAChGQAAAAAAA4RmAAAAAAAMEJoBAAAAADBAaAYAAAAAwAChGQAAAAAAA4RmAAAAAAAMEJoBAAAAADBAaAYAAAAAwAChGQAAAAAAA4RmAAAAAAAMEJoBAAAAADBAaAYAAAAAwAChGQAAAAAAA4RmAAAAAAAMEJoBAAAAADBAaAYAAAAAwAChGQAAAAAAA4RmAAAAAAAMEJoBAAAAADBAaAYAAAAAwAChGQAAAAAAA4RmAAAAAAAMEJoBAAAAADBAaAYAAAAAwAChGQAAAAAAA4RmAAAAAAAMEJoBAAAAADBAaAYAAAAAwAChGQAAAAAAA4RmAAAAAAAMEJoBAAAAADBAaAYAAAAAwAChGQAAAAAAA4RmAAAAAAAMEJoBAAAAADBAaAYAAAAAwAChGQAAAAAAA4RmAAAAAAAMpKvQfPnyZbVu3Vr16tVT69atdeXKFXuXBAAAAABIxdJVaB41apTatm2rrVu3qm3btho5cqS9SwIAAAAApGLpJjTfu3dPZ86cUePGjSVJjRs31pkzZxQaGmrnygAAAAAAqZWTvQtIKcHBwcqePbscHR0lSY6OjsqWLZuCg4Pl4+OTpHM4OJgSP57JPdnqtDejOT6PY6akff3+C/7J/F09stmgEvt40fm7e2S3USUp75987z3d0/f8s2XMYoNK7OMfzd/N0waV2MeLzj+bW/r+fy+bWwYbVGIf/2T+Wd2cbVCJffyT+bu7pZ1rTv/s5570O39HT0cbVZLy/sn33iGTqw0qsY/E5v+8r4nJYrFYbFlQanH69GkNGjRImzZtsh5r2LChPv/8c7388st2rAwAAAAAkFqlnY+K/oafn5/u3LmjuLg4SVJcXJxCQkLk5+dn58oAAAAAAKlVugnNvr6+Kl68uDZu3ChJ2rhxo4oXL57kpdkAAAAAgPQn3SzPlqSLFy9q8ODBevjwoTw9PTVx4kQVKFDA3mUBAAAAAFKpdBWaAQAAAAB4EelmeTYAAAAAAC+K0AwAAAAAgAFCMwAAAAAABgjNAAAAAAAYIDQDAAAAAGDAyd4F4PkePHigKlWq6K233tLw4cPtXU6KqlmzplxcXOTq6qro6GiVK1dOo0aNkrOzs71LSxExMTH66quvtHnzZrm4uMjR0VGvv/66BgwYkOa/Bs++9y4uLoqKilKhQoX03nvvqUyZMvYuLUX89c/+M19++aVy585tx6pSzv+df4UKFTR06FA7V5VyYmJi9PXXX2vjxo1ycnKSo6Oj8ufPrz59+qhQoUL2Ls+matasqa+//lpFihSxHmvevLkGDRqkChUq2LGylJXY1yE9Sa/zT8/zfvLkiX788Uc5OjpKktasWaMhQ4ZoxIgRat++vZ0rtL2aNWvKzc1N69evl4ODg/VYevjzsHPnTs2YMSPesXv37slisWj//v12qiohQnMqt3HjRpUsWVKbNm3Sxx9/LBcXF3uXlKJmzJihIkWKKC4uTu3atdP27dvVsGFDe5eVIoYMGaLo6GitXr1aHh4eio2N1erVq/XkyZM0H5ql/33vJWnbtm3q1q2bvv32W5UsWdLOlaWMv84/PUrP8x8yZIgeP36slStXytPTUxaLRT/++KMuX76c5kMzgPQpW7Zs2rdvn6pVqyZJCgwM1Msvv2znqlJWZGSk1q1bp2bNmtm7lBRVq1Yt1apVy/o6NDRULVq00EcffWTHqhJieXYqt3r1an3wwQcqWrSodu7cae9y7CY6OlrR0dHy9PS0dykp4sqVK9qxY4fGjRsnDw8PSZKTk5Nat24td3d3O1eX8urWras2bdro22+/tXcpgE09+7s/fvx46793JpNJ1atXV506dexcHQDYRrNmzbRmzRpJ0vXr1xUZGZnuPjjt1auXZs2apSdPnti7FLuJi4tTv379VL9+/VR3kYwrzanY2bNndf/+fb3++uu6e/euVq9erQYNGti7rBTVp08fubq66tq1a6pcubIqV65s75JSxJkzZ5QvXz5lzpzZ3qWkGiVLltSuXbvsXUaKefZnX5IcHR2tP0ykF3+d/8CBA1WlShU7V5Qy+Lsf/3svPf0gAUDa9tprr2np0qV68OCBAgMD1bRpU/3222/2LitFvfLKK3r55Ze1bNkyderUyd7l2MWUKVNksVg0cOBAe5eSAKE5FVu1apUCAgJkMplUt25djRs3Tnfu3FH27NntXVqKebZEMzo6Wr1799bChQvVuXNne5cFO7BYLPYuIUWl5+XJEvN/5sKFCxowYIAeP36sKlWqpIu9Lf7v97558+Z2rAZASjCZTGrQoIE2bdqkTZs2afny5ekuNEvShx9+qI4dO6ply5b2LiXFbdmyRZs3b9bq1aut97anJizPTqWePHmijRs3avXq1apZs6YaNmyomJiYdHe16RlXV1dVr15dP//8s71LSREvvfSSrl69qgcPHti7lFTj1KlTKly4sL3LAGzq2d/9hw8fSpIKFSqkdevWqUOHDgoPD7dzdQBgO82aNbN+aObt7W3vcuyiQIECqlatmhYsWGDvUlLUxYsXNWrUKM2YMUO+vr72LidRhOZUaufOnfL399dPP/2kXbt2adeuXZo/f74CAwPtXZpdmM1mHT58WPnz57d3KSkif/78qlmzpkaOHGn9QTkuLk4rV65URESEnatLeTt27NCyZcvUpUsXe5cC2FT+/PlVq1YtDR8+XI8ePbIej4yMtGNVAGB7efLkUb9+/fTBBx/YuxS76t27t5YuXZpuft4LDw9Xz5491b9/f5UoUcLe5RhieXYqtXr1ar355pvxjpUuXVpms1mHDh3Sa6+9ZqfKUtaze9tiYmJUuHBh9ezZ094lpZjPPvtMX375pVq0aCFnZ2eZzWZVq1Yt3eyg3qdPH+sjpwoWLKi5c+emm52zpYT3dY4bN06vvvqqHStCSpkwYYK++uortWzZUk5OTvL09FS2bNnUrVs3e5cGpIjY2Nh4//6lJ++88068pakbNmxIV3sctG7d2t4l2F2OHDkUEBCg+fPn27uUFLF06VJdu3ZNS5cu1dKlS+O1LVmyxLohrr2ZLOntRkEAAACkSiEhIWrQoIH279+vDBky2LscAJDElWYAAACkAosXL9bSpUs1aNAgAjOAVIUrzQAAAAAAGGAjMAAAAAAADBCaAQAAAAAwQGgGAAAAAMAAoRkAANhUhw4dVLNmTZucu2jRoho8eLBNzg0AgERoBgAg2f3yyy8qWrRovF+lS5dWs2bNtHDhQsXGxqZIHTt27NDMmTNfaEyHDh1UunRpG1UEAMB/D4+cAgDARho3bqyqVavKYrHozz//1Lp16zRhwgRdvHhRn3zyic3ff8eOHQoMDFTv3r1t/l4AAKRVhGYAAGzkpZdeUkBAgPV127Zt1aBBA61cuVL9+vWTj4+PHasDAABJwfJsAABSiJubm0qWLCmLxaJr167Fazt79qx69uypChUq6NVXX1XDhg31zTffKC4uLsF5ktK3Q4cOCgwMlKR4y8TXrFmTLHPZt2+fPvzwQ9WqVUslSpRQuXLl1KVLFx06dMhwzPXr19WjRw+VLVtWZcqUUc+ePXX9+vUE/SwWi5YuXarmzZurZMmSKl26tDp06KCDBw8mS+0AALwIrjQDAJCCnoXEzJkzW4+dOnVKHTp0kJOTk9q1a6csWbJo9+7dmjx5ss6ePaspU6a8cN/3339fZrNZR44c0aRJk6zjy5QpkyzzCAwM1IMHD9S0aVPlyJFDd+7c0cqVK9W5c2ctXrxY5cqVi9c/MjJSHTp0UIkSJdS/f39dvXpVS5cu1cmTJxUYGKisWbNa+3700UfatGmT6tWrp+bNm+vJkyfasGGDunTpopkzZ6pWrVrJMgcAAJKC0AwAgI1ERUUpNDRUknT37l0tX75cZ86cUYkSJeTv72/tN378eD158kTLly9XsWLFJEnt27fXhx9+qI0bN6ply5aqWLHiC/WtVKmSNmzYoCNHjsRbIp5cPvnkE7m5ucU71qZNGzVq1Ehz5sxJEJrDwsLUsWNHDRs2zHqsfPny6tWrl2bOnKmxY8dKkrZv364NGzZo7Nixat26tbVvx44d9dZbb2n8+PGqWbOmTCZTss8JAIDEsDwbAAAbmTlzpipWrKiKFSuqSZMmWrp0qerWrauvvvrK2ufevXs6fvy4atasaQ3BkmQymdSjRw9JT4Pki/a1tb8G5oiICIWFhcnBwUElS5bUr7/+muiYbt26xXtdp04d+fv7a+fOndZj69evl7u7u2rXrq3Q0FDrr4cPH6pmzZq6efOmrly5YpM5AQCQGK40AwBgI61bt1b9+vUVExOjc+fOad68ebp9+7ZcXV2tfW7cuCFJKlSoUILxBQoUkIODg3VJ94v0tbVr165p2rRp2rdvnx4+fBivLbGrwJ6envGWYD9TsGBB7dixQ5GRkXJzc9PFixcVERGhN954w/C97927F+9KPQAAtkRoBgDARvLly2cNf9WqVVPZsmXVtm1bjRo1StOmTbNzdf9cRESE2rVrp6ioKHXq1ElFihSRu7u7HBwcNGfOnH+1YZfFYpGPj0+8+7j/r8KFC//j8wMA8KIIzQAApJAyZcooICBAa9euVYcOHVSmTBnlzp1bknThwoUE/S9duiSz2aw8efJI0gv1lRK/4pscDhw4oJCQEH366adq0aJFvLbp06cnOubhw4e6e/dugqvNFy9elK+vr3W5d758+XTlyhWVLFlS7u7uNqkfAIAXwT3NAACkoA8++ECOjo6aMWOGJMnX11elS5fW7t27de7cOWs/i8WiuXPnSnp67++L9pX+d9/x/fv3k3UOjo6O1vf9q3379unkyZOG457V+Mz27dt1+fJl1a5d23qsadOmMpvNmjp1aqLn+PPPP/9p2QAA/CNcaQYAIAXly5dPDRs2tO5sXa5cOQ0bNkwdOnRQu3bt1LZtW2XNmlW7d+/Wvn371LhxY+vO2ZJeqG/JkiX1/fffa8yYMapWrZqcnZ1VokSJeFejExMTExNvs7K/qlu3rsqWLausWbNq4sSJunnzpnLkyKHff/9d69atU5EiReIF+me8vb21fft2hYSE6LXXXrM+cipLlizq1auXtV/9+vXVvHlzff/99/rtt99Uo0YNeXt76/bt2zpx4oSuXr0ab+MwAABsjdAMAEAK69GjhzZt2qQvvvhC3333nV599VUtX75cM2bM0LJlyxQZGak8efJo4MCB6tKlS7yxL9K3cePG+v3337Vp0yZt2bJFZrNZEyZMSFJo/uKLLxJty5cvnwoVKqR58+bp888/1/fff6/Y2Fi98sor+uabb7Rq1apEQ7Obm5sWLVqkTz/9VFOmTJHFYlGVKlU0ePBgZcuWLV7fCRMmqEKFCvrhhx80Z84cxcTEKGvWrHrppZc0YMCApHyJAQBINibL/11bBQAAAAAAJHFPMwAAAAAAhgjNAAAAAAAYIDQDAAAAAGCA0AwAAAAAgAFCMwAAAAAABgjNAAAAAAAYIDQDAAAAAGCA0AwAAAAAgAFCMwAAAAAABgjNAAAAAAAY+H8mX7nBh8tp+QAAAABJRU5ErkJggg==\n","text/plain":["<Figure size 1152x648 with 1 Axes>"]},"metadata":{},"output_type":"display_data"},{"name":"stdout","output_type":"stream","text":["CPU times: user 551 ms, sys: 239 ms, total: 790 ms\n","Wall time: 421 ms\n"]}],"source":["%%time\n","sns.set(font_scale = 1)\n","plt.figure(figsize=(16,9))\n","ax= sns.barplot(mesh_Heading_categories, df.iloc[:,6:].sum().values)\n","plt.title(\"Each Root Class\", fontsize=22)\n","plt.ylabel('Number of Articles', fontsize=18)\n","plt.xlabel('Root Label ', fontsize=18)\n","\n","#adding the text labels\n","rects = ax.patches\n","labels = df.iloc[:,6:].sum().values\n","for rect, label in zip(rects, labels):\n","    height = rect.get_height()\n","    ax.text(rect.get_x() + rect.get_width()/2, height + 5, label, ha='center', va='bottom', fontsize=12)\n","\n","plt.show()"]},{"cell_type":"code","execution_count":14,"id":"bd95f666","metadata":{"execution":{"iopub.execute_input":"2023-10-25T18:39:02.025914Z","iopub.status.busy":"2023-10-25T18:39:02.025602Z","iopub.status.idle":"2023-10-25T18:39:02.051883Z","shell.execute_reply":"2023-10-25T18:39:02.050973Z"},"papermill":{"duration":0.050698,"end_time":"2023-10-25T18:39:02.053888","exception":false,"start_time":"2023-10-25T18:39:02.00319","status":"completed"},"tags":[]},"outputs":[{"name":"stdout","output_type":"stream","text":["(40000, 20)\n","(10000, 20)\n"]}],"source":["df_train, df_test = train_test_split(df, random_state=32, test_size=0.20, shuffle=True)\n","\n","print(df_train.shape)\n","print(df_test.shape)\n","del(df)"]},{"cell_type":"code","execution_count":15,"id":"e6f6947d","metadata":{"execution":{"iopub.execute_input":"2023-10-25T18:39:02.097493Z","iopub.status.busy":"2023-10-25T18:39:02.096931Z","iopub.status.idle":"2023-10-25T18:39:02.138037Z","shell.execute_reply":"2023-10-25T18:39:02.136942Z"},"papermill":{"duration":0.065849,"end_time":"2023-10-25T18:39:02.140724","exception":false,"start_time":"2023-10-25T18:39:02.074875","status":"completed"},"tags":[]},"outputs":[{"data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>Title</th>\n","      <th>abstractText</th>\n","      <th>meshMajor</th>\n","      <th>pmid</th>\n","      <th>meshid</th>\n","      <th>meshroot</th>\n","      <th>A</th>\n","      <th>B</th>\n","      <th>C</th>\n","      <th>D</th>\n","      <th>...</th>\n","      <th>F</th>\n","      <th>G</th>\n","      <th>H</th>\n","      <th>I</th>\n","      <th>J</th>\n","      <th>L</th>\n","      <th>M</th>\n","      <th>N</th>\n","      <th>Z</th>\n","      <th>one_hot_labels</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>31112</th>\n","      <td>Neurofibromatosis associated with a coronary a...</td>\n","      <td>A case of a patient with type 1 neurofibromato...</td>\n","      <td>['Aged', 'Coronary Aneurysm', 'Humans', 'Male'...</td>\n","      <td>11144803</td>\n","      <td>[['M01.060.116.100'], ['C14.280.647.250.250', ...</td>\n","      <td>['Named Groups [M]', 'Diseases [C]', 'Organism...</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>...</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>[0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0]</td>\n","    </tr>\n","    <tr>\n","      <th>48911</th>\n","      <td>Phenolic constituents from Perovskia atriplici...</td>\n","      <td>Perovskoate, an isorinic acid derivative (1) a...</td>\n","      <td>['Caffeic Acids', 'Catechols', 'Cholinesterase...</td>\n","      <td>16644529</td>\n","      <td>[['D02.241.223.200.054'], ['D02.455.426.559.38...</td>\n","      <td>['Chemicals and Drugs [D]', 'Analytical, Diagn...</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>...</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>[0, 1, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0]</td>\n","    </tr>\n","    <tr>\n","      <th>16394</th>\n","      <td>Motion artifact and background noise suppressi...</td>\n","      <td>Optical coherence tomography (OCT) is a techni...</td>\n","      <td>['Algorithms', 'Angiography', 'Artifacts', 'Ar...</td>\n","      <td>25089975</td>\n","      <td>[['G17.035', 'L01.224.050'], ['E01.370.350.700...</td>\n","      <td>['Phenomena and Processes [G]', 'Information S...</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>...</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>[0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 1, 0]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","<p>3 rows × 21 columns</p>\n","</div>"],"text/plain":["                                                   Title  \\\n","31112  Neurofibromatosis associated with a coronary a...   \n","48911  Phenolic constituents from Perovskia atriplici...   \n","16394  Motion artifact and background noise suppressi...   \n","\n","                                            abstractText  \\\n","31112  A case of a patient with type 1 neurofibromato...   \n","48911  Perovskoate, an isorinic acid derivative (1) a...   \n","16394  Optical coherence tomography (OCT) is a techni...   \n","\n","                                               meshMajor      pmid  \\\n","31112  ['Aged', 'Coronary Aneurysm', 'Humans', 'Male'...  11144803   \n","48911  ['Caffeic Acids', 'Catechols', 'Cholinesterase...  16644529   \n","16394  ['Algorithms', 'Angiography', 'Artifacts', 'Ar...  25089975   \n","\n","                                                  meshid  \\\n","31112  [['M01.060.116.100'], ['C14.280.647.250.250', ...   \n","48911  [['D02.241.223.200.054'], ['D02.455.426.559.38...   \n","16394  [['G17.035', 'L01.224.050'], ['E01.370.350.700...   \n","\n","                                                meshroot  A  B  C  D  ...  F  \\\n","31112  ['Named Groups [M]', 'Diseases [C]', 'Organism...  0  1  1  0  ...  0   \n","48911  ['Chemicals and Drugs [D]', 'Analytical, Diagn...  0  1  0  1  ...  0   \n","16394  ['Phenomena and Processes [G]', 'Information S...  0  0  0  0  ...  0   \n","\n","       G  H  I  J  L  M  N  Z                              one_hot_labels  \n","31112  0  0  0  0  0  1  0  0  [0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0]  \n","48911  1  0  0  0  0  0  0  0  [0, 1, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0]  \n","16394  1  0  0  0  1  0  1  0  [0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 1, 0]  \n","\n","[3 rows x 21 columns]"]},"execution_count":15,"metadata":{},"output_type":"execute_result"}],"source":["df_train['one_hot_labels'] = list(df_train[mesh_Heading_categories].values)\n","df_train.head(3)"]},{"cell_type":"code","execution_count":16,"id":"f23048bd","metadata":{"execution":{"iopub.execute_input":"2023-10-25T18:39:02.188823Z","iopub.status.busy":"2023-10-25T18:39:02.188521Z","iopub.status.idle":"2023-10-25T18:39:02.198892Z","shell.execute_reply":"2023-10-25T18:39:02.197987Z"},"papermill":{"duration":0.035453,"end_time":"2023-10-25T18:39:02.201049","exception":false,"start_time":"2023-10-25T18:39:02.165596","status":"completed"},"tags":[]},"outputs":[],"source":["labels = list(df_train.one_hot_labels.values)\n","Article_train = list(df_train.abstractText.values)"]},{"cell_type":"markdown","id":"57f89f79","metadata":{"papermill":{"duration":0.021274,"end_time":"2023-10-25T18:39:02.244238","exception":false,"start_time":"2023-10-25T18:39:02.222964","status":"completed"},"tags":[]},"source":["<a id=\"4\"></a>\n","## <p style=\"background-color:#1a0a36;font-family:newtimeroman;color:#FFF9ED;font-size:150%;text-align:center;border-radius:10px 10px;\"> Tokenizations</p>\n","#### [Top ↑](#top)"]},{"cell_type":"code","execution_count":17,"id":"66600dbf","metadata":{"execution":{"iopub.execute_input":"2023-10-25T18:39:02.291441Z","iopub.status.busy":"2023-10-25T18:39:02.291068Z","iopub.status.idle":"2023-10-25T18:45:39.498118Z","shell.execute_reply":"2023-10-25T18:45:39.497034Z"},"papermill":{"duration":397.256873,"end_time":"2023-10-25T18:45:39.522694","exception":false,"start_time":"2023-10-25T18:39:02.265821","status":"completed"},"tags":[]},"outputs":[{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"606737504cee43808715f680c9cd066c","version_major":2,"version_minor":0},"text/plain":["Downloading (…)solve/main/vocab.txt:   0%|          | 0.00/213k [00:00<?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"12cf03bde07949e399f0d4f9ad8c517b","version_major":2,"version_minor":0},"text/plain":["Downloading (…)lve/main/config.json:   0%|          | 0.00/1.11k [00:00<?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"name":"stdout","output_type":"stream","text":["tokenizer outputs:  dict_keys(['input_ids', 'token_type_ids', 'attention_mask'])\n","CPU times: user 6min 36s, sys: 801 ms, total: 6min 37s\n","Wall time: 6min 37s\n"]}],"source":["%%time\n","max_length = 128\n","#tokenizer = RobertaTokenizer.from_pretrained('distilroberta-base', do_lower_case=False)  # tokenizer\n","tokenizer = BertTokenizer.from_pretrained('dmis-lab/biobert-base-cased-v1.2', do_lower_case=True) \n","#tokenizer = XLNetTokenizer.from_pretrained('xlnet-base-cased', do_lower_case=False) \n","\n","encodings = tokenizer.batch_encode_plus(Article_train,max_length=max_length,padding=True,truncation=True) # tokenizer's encoding method\n","print('tokenizer outputs: ', encodings.keys())\n","\n","input_ids = encodings['input_ids'] # tokenized and encoded sentences\n","attention_masks = encodings['attention_mask'] # attention masks"]},{"cell_type":"code","execution_count":18,"id":"3f93e18d","metadata":{"execution":{"iopub.execute_input":"2023-10-25T18:45:39.568292Z","iopub.status.busy":"2023-10-25T18:45:39.567478Z","iopub.status.idle":"2023-10-25T18:45:45.896284Z","shell.execute_reply":"2023-10-25T18:45:45.895264Z"},"papermill":{"duration":6.35431,"end_time":"2023-10-25T18:45:45.898774","exception":false,"start_time":"2023-10-25T18:45:39.544464","status":"completed"},"tags":[]},"outputs":[{"name":"stdout","output_type":"stream","text":["df label indices with only one instance:  [49950, 49878, 49876, 49838, 49759, 49739, 49696, 49590, 49585, 49574, 49546, 49535, 49528, 49487, 49470, 49419, 49404, 49395, 49314, 49245, 49150, 49080, 49071, 49058, 49004, 48991, 48982, 48922, 48919, 48757, 48708, 48686, 48679, 48677, 48602, 48594, 48583, 48531, 48454, 48438, 48411, 48388, 48375, 48350, 48337, 48298, 48213, 48185, 48123, 48085, 48075, 48056, 48020, 48001, 47876, 47840, 47828, 47817, 47776, 47671, 47495, 47435, 47369, 47363, 47362, 47359, 47356, 47321, 47293, 47244, 47239, 47159, 47104, 47089, 47061, 47050, 47002, 46955, 46887, 46874, 46837, 46747, 46706, 46651, 46635, 46598, 46446, 46365, 46247, 46223, 46199, 46170, 46033, 46028, 45955, 45946, 45756, 45755, 45712, 45682, 45671, 45573, 45509, 45430, 45391, 45317, 45294, 45226, 45216, 45190, 45134, 45132, 45105, 45052, 45035, 45025, 44971, 44953, 44867, 44851, 44827, 44777, 44763, 44625, 44606, 44598, 44586, 44579, 44574, 44568, 44499, 44479, 44462, 44296, 44206, 44181, 44116, 44105, 44100, 44090, 43970, 43957, 43831, 43792, 43736, 43721, 43684, 43623, 43574, 43553, 43350, 43307, 43287, 43281, 43220, 43198, 43162, 43156, 43136, 43106, 43091, 43053, 42943, 42845, 42831, 42720, 42703, 42661, 42625, 42600, 42561, 42454, 42381, 42370, 42364, 42287, 42048, 42044, 42010, 41953, 41896, 41881, 41814, 41813, 41737, 41671, 41579, 41466, 41413, 41364, 41283, 41248, 41188, 41180, 41145, 41046, 40974, 40905, 40841, 40826, 40820, 40744, 40732, 40715, 40673, 40666, 40609, 40556, 40548, 40533, 40529, 40446, 40417, 40408, 40367, 40363, 40234, 40217, 40204, 40129, 40127, 40034, 39998, 39997, 39964, 39960, 39908, 39890, 39860, 39827, 39813, 39796, 39775, 39769, 39752, 39743, 39729, 39717, 39695, 39682, 39627, 39588, 39568, 39556, 39550, 39540, 39495, 39375, 39200, 39145, 39097, 39072, 39038, 39000, 38997, 38976, 38874, 38832, 38809, 38751, 38705, 38670, 38659, 38648, 38644, 38609, 38522, 38437, 38418, 38414, 38313, 38287, 38246, 38215, 38207, 38144, 38130, 38125, 38064, 37954, 37921, 37798, 37758, 37709, 37704, 37703, 37686, 37670, 37536, 37532, 37531, 37448, 37438, 37432, 37382, 37333, 37198, 37083, 37074, 36980, 36971, 36921, 36828, 36803, 36762, 36754, 36746, 36679, 36660, 36641, 36639, 36578, 36512, 36483, 36442, 36400, 36367, 36323, 36310, 36304, 36281, 36240, 36239, 36233, 36215, 36202, 36197, 36189, 36186, 36107, 36103, 36068, 36024, 35962, 35833, 35832, 35781, 35738, 35697, 35617, 35565, 35564, 35555, 35488, 35460, 35411, 35409, 35330, 35281, 35242, 35211, 35164, 35112, 35067, 35043, 34978, 34965, 34910, 34868, 34863, 34859, 34823, 34802, 34731, 34669, 34668, 34636, 34598, 34560, 34549, 34465, 34450, 34377, 34164, 34141, 34107, 33979, 33959, 33854, 33812, 33673, 33639, 33473, 33465, 33390, 33322, 33261, 33257, 33226, 33214, 33087, 33015, 32975, 32962, 32960, 32943, 32917, 32863, 32848, 32846, 32820, 32778, 32771, 32759, 32525, 32461, 32241, 32217, 32166, 32144, 32135, 32010, 31904, 31899, 31873, 31860, 31833, 31827, 31751, 31732, 31695, 31663, 31654, 31584, 31504, 31497, 31465, 31426, 31357, 31335, 31298, 31288, 31177, 31150, 31116, 31071, 30949, 30858, 30847, 30807, 30699, 30645, 30599, 30585, 30578, 30431, 30370, 30355, 30347, 30306, 30296, 30262, 30174, 30080, 30067, 30052, 30018, 29945, 29906, 29859, 29820, 29770, 29756, 29752, 29743, 29713, 29710, 29495, 29402, 29363, 29356, 29346, 29289, 29276, 29265, 29263, 29048, 29029, 28993, 28965, 28881, 28879, 28817, 28776, 28753, 28673, 28646, 28638, 28574, 28562, 28537, 28435, 28396, 28381, 28332, 28268, 28265, 28217, 28134, 28123, 28109, 28087, 27953, 27917, 27899, 27881, 27862, 27853, 27810, 27750, 27716, 27687, 27659, 27639, 27608, 27575, 27559, 27399, 27383, 27348, 27267, 27242, 27136, 27047, 26998, 26986, 26883, 26868, 26860, 26812, 26766, 26722, 26676, 26652, 26633, 26620, 26552, 26529, 26506, 26496, 26392, 26382, 26353, 26279, 26258, 26252, 26225, 26202, 26177, 26138, 25981, 25945, 25906, 25892, 25887, 25872, 25789, 25740, 25733, 25637, 25551, 25533, 25502, 25400, 25309, 25269, 25144, 25095, 25047, 25039, 24892, 24724, 24702, 24582, 24567, 24540, 24507, 24466, 24465, 24458, 24449, 24426, 24424, 24416, 24338, 24331, 24265, 24201, 24108, 24065, 24047, 24009, 23968, 23967, 23963, 23912, 23901, 23873, 23851, 23798, 23784, 23766, 23764, 23605, 23601, 23550, 23508, 23485, 23459, 23437, 23394, 23227, 23226, 23200, 23173, 23170, 23114, 23082, 23071, 23025, 22938, 22865, 22842, 22788, 22739, 22705, 22697, 22666, 22557, 22501, 22463, 22408, 22319, 22288, 22139, 22059, 22017, 22011, 21996, 21966, 21960, 21950, 21903, 21754, 21742, 21656, 21632, 21613, 21577, 21566, 21563, 21539, 21332, 21320, 21257, 21232, 21205, 21201, 21174, 21137, 21115, 21081, 21071, 21048, 21031, 20942, 20802, 20733, 20726, 20715, 20710, 20686, 20517, 20481, 20459, 20455, 20447, 20334, 20324, 20320, 20305, 20206, 20178, 20168, 20165, 20130, 20116, 20084, 20072, 20060, 19967, 19936, 19915, 19905, 19897, 19852, 19814, 19804, 19749, 19740, 19725, 19678, 19631, 19621, 19575, 19515, 19501, 19459, 19350, 19307, 19289, 19258, 19207, 19201, 19194, 19192, 19135, 19036, 19024, 19008, 18995, 18951, 18941, 18890, 18851, 18810, 18796, 18755, 18735, 18714, 18627, 18613, 18583, 18573, 18531, 18507, 18502, 18501, 18496, 18448, 18437, 18355, 18263, 18241, 18239, 18219, 18217, 18203, 18182, 18112, 18061, 18041, 18006, 17989, 17981, 17957, 17884, 17840, 17827, 17826, 17819, 17686, 17565, 17547, 17514, 17471, 17390, 17317, 17239, 17215, 17201, 17162, 17148, 17094, 17065, 17016, 16938, 16914, 16885, 16792, 16681, 16677, 16651, 16632, 16503, 16495, 16467, 16423, 16326, 16319, 16167, 16159, 16142, 16132, 16044, 16000, 15966, 15965, 15823, 15799, 15792, 15778, 15763, 15758, 15711, 15683, 15644, 15638, 15621, 15532, 15525, 15520, 15490, 15427, 15418, 15414, 15363, 15360, 15334, 15330, 15324, 15288, 15255, 15233, 15178, 15177, 15128, 15055, 15040, 15035, 14969, 14906, 14889, 14820, 14811, 14801, 14779, 14767, 14731, 14679, 14664, 14632, 14608, 14424, 14389, 14276, 14268, 14098, 14049, 13979, 13951, 13948, 13941, 13892, 13845, 13829, 13757, 13719, 13656, 13648, 13637, 13599, 13529, 13497, 13399, 13305, 13300, 13205, 13188, 13111, 13103, 13088, 13080, 13063, 13028, 12952, 12918, 12846, 12825, 12680, 12667, 12664, 12570, 12564, 12536, 12506, 12433, 12427, 12392, 12325, 12292, 12239, 12155, 12151, 12086, 12018, 12009, 12008, 11945, 11918, 11903, 11864, 11849, 11786, 11784, 11781, 11709, 11681, 11656, 11641, 11618, 11603, 11575, 11518, 11504, 11498, 11489, 11484, 11444, 11390, 11373, 11285, 11257, 11208, 11054, 11040, 10952, 10947, 10911, 10893, 10869, 10770, 10673, 10624, 10591, 10555, 10443, 10434, 10430, 10357, 10284, 10274, 10257, 10127, 10106, 10104, 10094, 10088, 10065, 10005, 9986, 9885, 9878, 9732, 9729, 9711, 9691, 9688, 9654, 9632, 9613, 9509, 9482, 9477, 9416, 9399, 9377, 9320, 9282, 9241, 9185, 9184, 9159, 9052, 9018, 9004, 8994, 8973, 8956, 8952, 8941, 8912, 8863, 8860, 8843, 8799, 8798, 8767, 8757, 8654, 8624, 8569, 8564, 8560, 8542, 8425, 8388, 8281, 8172, 8159, 8097, 8069, 7925, 7857, 7770, 7755, 7651, 7634, 7626, 7625, 7614, 7484, 7393, 7276, 7179, 7177, 7043, 6938, 6923, 6886, 6875, 6830, 6826, 6809, 6744, 6637, 6553, 6501, 6463, 6446, 6442, 6417, 6415, 6379, 6361, 6314, 6251, 6056, 6032, 5961, 5920, 5904, 5897, 5893, 5858, 5842, 5833, 5732, 5714, 5690, 5678, 5644, 5602, 5541, 5533, 5531, 5481, 5446, 5368, 5243, 5223, 5134, 5037, 4987, 4873, 4809, 4730, 4707, 4692, 4685, 4682, 4635, 4576, 4548, 4488, 4437, 4405, 4355, 4315, 4282, 4244, 4169, 4147, 4107, 4087, 4067, 4034, 4029, 3993, 3963, 3943, 3903, 3867, 3795, 3744, 3720, 3707, 3679, 3674, 3654, 3631, 3585, 3525, 3448, 3427, 3394, 3392, 3357, 3209, 3171, 3161, 3118, 3114, 3109, 3080, 3071, 3065, 3052, 3043, 2983, 2942, 2912, 2899, 2876, 2799, 2767, 2711, 2691, 2679, 2610, 2420, 2387, 2361, 2327, 2274, 2254, 2170, 2162, 2146, 2091, 2050, 2038, 2033, 2018, 1921, 1754, 1688, 1668, 1613, 1591, 1555, 1521, 1448, 1411, 1407, 1348, 1347, 1313, 1249, 1215, 1186, 1184, 1051, 941, 925, 913, 907, 884, 806, 752, 680, 545, 507, 458, 388, 364, 360, 223, 213, 200, 193, 184, 171, 158, 105, 103, 96, 50]\n"]}],"source":["# Identifying indices of 'one_hot_labels' entries that only occur once - this will allow me to stratify split our training data later\n","label_counts = df_train.one_hot_labels.astype(str).value_counts()\n","one_freq = label_counts[label_counts==1].keys()\n","one_freq_idxs = sorted(list(df_train[df_train.one_hot_labels.astype(str).isin(one_freq)].index), reverse=True)\n","print('df label indices with only one instance: ', one_freq_idxs)"]},{"cell_type":"code","execution_count":19,"id":"1b3b3e43","metadata":{"execution":{"iopub.execute_input":"2023-10-25T18:45:45.944436Z","iopub.status.busy":"2023-10-25T18:45:45.944106Z","iopub.status.idle":"2023-10-25T18:45:47.657237Z","shell.execute_reply":"2023-10-25T18:45:47.656087Z"},"papermill":{"duration":1.738623,"end_time":"2023-10-25T18:45:47.659445","exception":false,"start_time":"2023-10-25T18:45:45.920822","status":"completed"},"tags":[]},"outputs":[{"name":"stderr","output_type":"stream","text":["/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py:6: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at  /usr/local/src/pytorch/torch/csrc/utils/tensor_new.cpp:207.)\n","  \n"]}],"source":["\n","train_inputs, validation_inputs, train_labels, validation_labels, train_masks, validation_masks = train_test_split(input_ids, labels, attention_masks,\n","                                                            random_state=2020, test_size=0.20)\n","\n","# Convert all of our data into torch tensors, the required datatype for our BERT Pytorch model\n","train_inputs = torch.tensor(train_inputs)\n","train_labels = torch.tensor(train_labels)\n","train_masks = torch.tensor(train_masks)\n","\n","validation_inputs = torch.tensor(validation_inputs)\n","validation_labels = torch.tensor(validation_labels)\n","validation_masks = torch.tensor(validation_masks)"]},{"cell_type":"markdown","id":"73fe689f","metadata":{"papermill":{"duration":0.023344,"end_time":"2023-10-25T18:45:47.706855","exception":false,"start_time":"2023-10-25T18:45:47.683511","status":"completed"},"tags":[]},"source":["<a id=\"5\"></a>\n","## <p style=\"background-color:#1a0a36;font-family:newtimeroman;color:#FFF9ED;font-size:150%;text-align:center;border-radius:10px 10px;\"> Creating the Data Loaders</p>\n","#### [Top ↑](#top)"]},{"cell_type":"code","execution_count":20,"id":"5157f7b5","metadata":{"execution":{"iopub.execute_input":"2023-10-25T18:45:47.753155Z","iopub.status.busy":"2023-10-25T18:45:47.752773Z","iopub.status.idle":"2023-10-25T18:45:47.760441Z","shell.execute_reply":"2023-10-25T18:45:47.759489Z"},"papermill":{"duration":0.033499,"end_time":"2023-10-25T18:45:47.762834","exception":false,"start_time":"2023-10-25T18:45:47.729335","status":"completed"},"tags":[]},"outputs":[],"source":["batch_size = 64\n","\n","# Create an iterator of our data with torch DataLoader. This helps save on memory during training because, unlike a for loop, \n","# with an iterator the entire dataset does not need to be loaded into memory\n","\n","train_data =     TensorDataset(train_inputs, train_masks, train_labels,)\n","train_sampler = RandomSampler(train_data)\n","train_dataloader = DataLoader(train_data, sampler=train_sampler, batch_size=batch_size)\n","\n","validation_data = TensorDataset(validation_inputs, validation_masks, validation_labels,)\n","validation_sampler = SequentialSampler(validation_data)\n","validation_dataloader = DataLoader(validation_data, sampler=validation_sampler, batch_size=batch_size)"]},{"cell_type":"code","execution_count":21,"id":"23a92f1b","metadata":{"execution":{"iopub.execute_input":"2023-10-25T18:45:47.813368Z","iopub.status.busy":"2023-10-25T18:45:47.812457Z","iopub.status.idle":"2023-10-25T18:45:47.977594Z","shell.execute_reply":"2023-10-25T18:45:47.976629Z"},"papermill":{"duration":0.191961,"end_time":"2023-10-25T18:45:47.980161","exception":false,"start_time":"2023-10-25T18:45:47.7882","status":"completed"},"tags":[]},"outputs":[],"source":["torch.save(validation_dataloader,'validation_data_loader')\n","torch.save(train_dataloader,'train_data_loader')"]},{"cell_type":"markdown","id":"c8e43338","metadata":{"papermill":{"duration":0.021556,"end_time":"2023-10-25T18:45:48.024092","exception":false,"start_time":"2023-10-25T18:45:48.002536","status":"completed"},"tags":[]},"source":["<a id=\"6\"></a>\n","## <p style=\"background-color:#1a0a36;font-family:newtimeroman;color:#FFF9ED;font-size:150%;text-align:center;border-radius:10px 10px;\"> Loading the pretrained model</p>\n","#### [Top ↑](#top)"]},{"cell_type":"code","execution_count":22,"id":"83f322a4","metadata":{"execution":{"iopub.execute_input":"2023-10-25T18:45:48.073049Z","iopub.status.busy":"2023-10-25T18:45:48.072675Z","iopub.status.idle":"2023-10-25T18:45:52.474987Z","shell.execute_reply":"2023-10-25T18:45:52.473919Z"},"papermill":{"duration":4.427238,"end_time":"2023-10-25T18:45:52.477116","exception":false,"start_time":"2023-10-25T18:45:48.049878","status":"completed"},"tags":[]},"outputs":[{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"a0f1fcec8f184e238219e80bbcf8129a","version_major":2,"version_minor":0},"text/plain":["Downloading pytorch_model.bin:   0%|          | 0.00/436M [00:00<?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"name":"stderr","output_type":"stream","text":["Some weights of the model checkpoint at dmis-lab/biobert-base-cased-v1.2 were not used when initializing BertForSequenceClassification: ['cls.predictions.decoder.bias', 'cls.seq_relationship.weight', 'cls.predictions.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.dense.bias', 'cls.predictions.decoder.weight']\n","- This IS expected if you are initializing BertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n","- This IS NOT expected if you are initializing BertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n","Some weights of BertForSequenceClassification were not initialized from the model checkpoint at dmis-lab/biobert-base-cased-v1.2 and are newly initialized: ['classifier.bias', 'classifier.weight']\n","You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"]},{"name":"stdout","output_type":"stream","text":["Model Pushed to Cuda for Training\n","CPU times: user 1.81 s, sys: 983 ms, total: 2.79 s\n","Wall time: 4.4 s\n"]}],"source":["%%time\n","#Tried Several Models Locally XLNet was performing Best. Note If you are changing the model then change the Tokenizer also\n","# model = RobertaForSequenceClassification.from_pretrained('distilroberta-base', num_labels=num_labels)\n","model = BertForSequenceClassification.from_pretrained(\"dmis-lab/biobert-base-cased-v1.2\", num_labels=num_labels)\n","# model = XLNetForSequenceClassification.from_pretrained(\"xlnet-base-cased\", num_labels=num_labels)\n","model.cuda()\n","print('Model Pushed to Cuda for Training')"]},{"cell_type":"code","execution_count":23,"id":"942dafd3","metadata":{"execution":{"iopub.execute_input":"2023-10-25T18:45:52.524046Z","iopub.status.busy":"2023-10-25T18:45:52.523701Z","iopub.status.idle":"2023-10-25T18:45:52.532764Z","shell.execute_reply":"2023-10-25T18:45:52.531827Z"},"papermill":{"duration":0.035059,"end_time":"2023-10-25T18:45:52.534897","exception":false,"start_time":"2023-10-25T18:45:52.499838","status":"completed"},"tags":[]},"outputs":[],"source":["param_optimizer = list(model.named_parameters())\n","no_decay = ['bias', 'gamma', 'beta']\n","optimizer_grouped_parameters = [\n","    {'params': [p for n, p in param_optimizer if not any(nd in n for nd in no_decay)],\n","     'weight_decay_rate': 0.01},\n","    {'params': [p for n, p in param_optimizer if any(nd in n for nd in no_decay)],\n","     'weight_decay_rate': 0.0}\n","]"]},{"cell_type":"code","execution_count":24,"id":"2ef5d401","metadata":{"execution":{"iopub.execute_input":"2023-10-25T18:45:52.58029Z","iopub.status.busy":"2023-10-25T18:45:52.579992Z","iopub.status.idle":"2023-10-25T18:45:52.586116Z","shell.execute_reply":"2023-10-25T18:45:52.585184Z"},"papermill":{"duration":0.031412,"end_time":"2023-10-25T18:45:52.588347","exception":false,"start_time":"2023-10-25T18:45:52.556935","status":"completed"},"tags":[]},"outputs":[],"source":["optimizer = AdamW(optimizer_grouped_parameters,lr=6e-6)\n","# optimizer = AdamW(model.parameters(),lr=4e-5)  # Default optimization #XL-NET"]},{"cell_type":"code","execution_count":25,"id":"3ea6038a","metadata":{"execution":{"iopub.execute_input":"2023-10-25T18:45:52.635547Z","iopub.status.busy":"2023-10-25T18:45:52.635225Z","iopub.status.idle":"2023-10-25T18:45:52.640769Z","shell.execute_reply":"2023-10-25T18:45:52.639808Z"},"papermill":{"duration":0.031809,"end_time":"2023-10-25T18:45:52.642895","exception":false,"start_time":"2023-10-25T18:45:52.611086","status":"completed"},"tags":[]},"outputs":[],"source":["os.environ['TF_FORCE_GPU_ALLOW_GROWTH']='true'"]},{"cell_type":"markdown","id":"8a3d3258","metadata":{"papermill":{"duration":0.022288,"end_time":"2023-10-25T18:45:52.687835","exception":false,"start_time":"2023-10-25T18:45:52.665547","status":"completed"},"tags":[]},"source":["<a id=\"7\"></a>\n","## <p style=\"background-color:#1a0a36;font-family:newtimeroman;color:#FFF9ED;font-size:150%;text-align:center;border-radius:10px 10px;\"> Training the model</p>\n","#### [Top ↑](#top)"]},{"cell_type":"code","execution_count":26,"id":"379d4d19","metadata":{"execution":{"iopub.execute_input":"2023-10-25T18:45:52.734265Z","iopub.status.busy":"2023-10-25T18:45:52.733949Z","iopub.status.idle":"2023-10-25T20:00:04.828045Z","shell.execute_reply":"2023-10-25T20:00:04.827081Z"},"papermill":{"duration":4452.119715,"end_time":"2023-10-25T20:00:04.830119","exception":false,"start_time":"2023-10-25T18:45:52.710404","status":"completed"},"tags":[]},"outputs":[{"name":"stderr","output_type":"stream","text":["Epoch :   0%|          | 0/6 [00:00<?, ?it/s]"]},{"name":"stdout","output_type":"stream","text":["Train loss: 0.4172029095888138\n"]},{"name":"stderr","output_type":"stream","text":["Epoch :  17%|█▋        | 1/6 [12:14<1:01:10, 734.12s/it]"]},{"name":"stdout","output_type":"stream","text":["F1 Validation Accuracy:  83.58620689655173\n","Flat Validation Accuracy:  9.375\n","\n","\n","Train loss: 0.32697912526130674\n"]},{"name":"stderr","output_type":"stream","text":["Epoch :  33%|███▎      | 2/6 [24:36<49:15, 738.83s/it]  "]},{"name":"stdout","output_type":"stream","text":["F1 Validation Accuracy:  85.91160220994475\n","Flat Validation Accuracy:  14.0625\n","\n","\n","Train loss: 0.30155076694488525\n"]},{"name":"stderr","output_type":"stream","text":["Epoch :  50%|█████     | 3/6 [36:59<37:02, 740.71s/it]"]},{"name":"stdout","output_type":"stream","text":["F1 Validation Accuracy:  86.37602179836513\n","Flat Validation Accuracy:  15.625\n","\n","\n","Train loss: 0.28567135289311407\n"]},{"name":"stderr","output_type":"stream","text":["Epoch :  67%|██████▋   | 4/6 [49:22<24:43, 741.68s/it]"]},{"name":"stdout","output_type":"stream","text":["F1 Validation Accuracy:  86.96844993141289\n","Flat Validation Accuracy:  20.3125\n","\n","\n","Train loss: 0.27252469077706337\n"]},{"name":"stderr","output_type":"stream","text":["Epoch :  83%|████████▎ | 5/6 [1:01:45<12:22, 742.20s/it]"]},{"name":"stdout","output_type":"stream","text":["F1 Validation Accuracy:  86.3697705802969\n","Flat Validation Accuracy:  15.625\n","\n","\n","Train loss: 0.2615785965025425\n"]},{"name":"stderr","output_type":"stream","text":["Epoch : 100%|██████████| 6/6 [1:14:08<00:00, 741.42s/it]"]},{"name":"stdout","output_type":"stream","text":["F1 Validation Accuracy:  87.3972602739726\n","Flat Validation Accuracy:  21.875\n","\n","\n"]},{"name":"stderr","output_type":"stream","text":["\n"]},{"data":{"text/html":["Waiting for W&B process to finish... <strong style=\"color:green\">(success).</strong>"],"text/plain":["<IPython.core.display.HTML object>"]},"metadata":{},"output_type":"display_data"},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"62a7889621f640b5a15685ab562919a4","version_major":2,"version_minor":0},"text/plain":["VBox(children=(Label(value='0.000 MB of 0.000 MB uploaded (0.000 MB deduped)\\r'), FloatProgress(value=1.0, max…"]},"metadata":{},"output_type":"display_data"},{"data":{"text/html":["<style>\n","    table.wandb td:nth-child(1) { padding: 0 10px; text-align: left ; width: auto;} td:nth-child(2) {text-align: left ; width: 100%}\n","    .wandb-row { display: flex; flex-direction: row; flex-wrap: wrap; justify-content: flex-start; width: 100% }\n","    .wandb-col { display: flex; flex-direction: column; flex-basis: 100%; flex: 1; padding: 10px; }\n","    </style>\n","<div class=\"wandb-row\"><div class=\"wandb-col\"><h3>Run history:</h3><br/><table class=\"wandb\"><tr><td>train_loss</td><td>█▄▃▂▁▁</td></tr><tr><td>val_f1_accuracy</td><td>▁▅▆▇▆█</td></tr><tr><td>val_flat_accuracy</td><td>▁▄▅▇▅█</td></tr></table><br/></div><div class=\"wandb-col\"><h3>Run summary:</h3><br/><table class=\"wandb\"><tr><td>train_loss</td><td>0.26158</td></tr><tr><td>val_f1_accuracy</td><td>87.39726</td></tr><tr><td>val_flat_accuracy</td><td>21.875</td></tr></table><br/></div></div>"],"text/plain":["<IPython.core.display.HTML object>"]},"metadata":{},"output_type":"display_data"},{"data":{"text/html":["Synced <strong style=\"color:#cdcd00\">42.Biobert-base-cased-v1.2-Run-27</strong>: <a href=\"https://wandb.ai/owaiskhan9515/Multi%20Label%20Classification%20of%20PubMed%20Articles%20%28Paper%20Night%20Presentation%29/runs/2hua4iyd\" target=\"_blank\">https://wandb.ai/owaiskhan9515/Multi%20Label%20Classification%20of%20PubMed%20Articles%20%28Paper%20Night%20Presentation%29/runs/2hua4iyd</a><br/>Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s)"],"text/plain":["<IPython.core.display.HTML object>"]},"metadata":{},"output_type":"display_data"},{"data":{"text/html":["Find logs at: <code>./wandb/run-20231025_183852-2hua4iyd/logs</code>"],"text/plain":["<IPython.core.display.HTML object>"]},"metadata":{},"output_type":"display_data"},{"name":"stdout","output_type":"stream","text":["CPU times: user 1h 13min 48s, sys: 25.1 s, total: 1h 14min 13s\n","Wall time: 1h 14min 12s\n"]}],"source":["%%time\n","\n","# For Storing our loss and accuracy for plotting\n","train_loss_set = []\n","val_f1_accuracy_list,val_flat_accuracy_list,training_loss_list,epochs_list=[],[],[],[]\n","\n","# Number of training epochs (recommend between 5 and 10)\n","epochs = 6\n","\n","# trange is a tqdm wrapper around the normal python range\n","for _ in trange(epochs, desc=\"Epoch \"):\n","    # Training\n","\n","    # Set our model to training mode (as opposed to evaluation mode)\n","    model.train()\n","\n","    # Tracking variables\n","    tr_loss = 0 #running loss\n","    nb_tr_examples, nb_tr_steps = 0, 0\n","  \n","    # Train the data for one epoch\n","    for step, batch in enumerate(train_dataloader):\n","        # Add batch to GPU\n","        batch = tuple(t.to(device) for t in batch)\n","        # Unpack the inputs from our dataloader\n","        b_input_ids, b_input_mask, b_labels= batch\n","        # Clear out the gradients (by default they accumulate)\n","        optimizer.zero_grad()\n","\n","        # Forward pass for multilabel classification\n","        # https://pytorch.org/docs/stable/generated/torch.nn.BCELoss.html\n","        # https://pytorch.org/docs/stable/generated/torch.nn.BCEWithLogitsLoss.html\n","        # Creates a criterion that measures the Binary Cross Entropy between the target and the input probabilities\n","        # Also This loss combines a Sigmoid layer and the BCELoss in one single class. This version is more numerically stable \n","        # than using a plain Sigmoid followed by a BCELoss as, by combining the operations into one layer, we take advantage of the \n","        # log-sum-exp trick for numerical stability.\n","        outputs = model(b_input_ids, token_type_ids=None, attention_mask=b_input_mask)\n","        logits = outputs[0]\n","        loss_func = BCEWithLogitsLoss()  \n","        loss = loss_func(logits.view(-1,num_labels),b_labels.type_as(logits).view(-1,num_labels)) #convert labels to float for calculation\n","        \n","        train_loss_set.append(loss.item())    \n","\n","        # Backward pass\n","        loss.backward()\n","        # Update parameters and take a step using the computed gradient\n","        optimizer.step()\n","        # scheduler.step()\n","        # Update tracking variables\n","        tr_loss += loss.item()\n","        nb_tr_examples += b_input_ids.size(0)\n","        nb_tr_steps += 1\n","\n","    print(\"Train loss: {}\".format(tr_loss/nb_tr_steps))\n","    training_loss_list.append(tr_loss/nb_tr_steps)\n","\n","    ###############################################################################\n","\n","    # Validation\n","\n","    # Put model in evaluation mode to evaluate loss on the validation set\n","    model.eval()\n","\n","    # Variables to gather full output\n","    logit_preds,true_labels,pred_labels,tokenized_texts = [],[],[],[]\n","\n","    # Predict\n","    for i, batch in enumerate(validation_dataloader):\n","        batch = tuple(t.to(device) for t in batch)\n","        # Unpack the inputs from our dataloader\n","        b_input_ids, b_input_mask, b_labels = batch\n","        with torch.no_grad():\n","            # Forward pass\n","            outs = model(b_input_ids, token_type_ids=None, attention_mask=b_input_mask)\n","            b_logit_pred = outs[0]\n","            pred_label = torch.sigmoid(b_logit_pred)\n","\n","        b_logit_pred = b_logit_pred.detach().cpu().numpy()\n","        pred_label = pred_label.to('cpu').numpy()\n","        b_labels = b_labels.to('cpu').numpy()\n","\n","    tokenized_texts.append(b_input_ids)\n","    logit_preds.append(b_logit_pred)\n","    true_labels.append(b_labels)\n","    pred_labels.append(pred_label)\n","\n","    # Flatten outputs\n","    pred_labels = [item for sublist in pred_labels for item in sublist]\n","    true_labels = [item for sublist in true_labels for item in sublist]\n","\n","    # Calculate Accuracy\n","    threshold = 0.50\n","    pred_bools = [pl>threshold for pl in pred_labels]\n","    true_bools = [tl==1 for tl in true_labels]\n","    val_f1_accuracy = f1_score(true_bools,pred_bools,average='micro')*100\n","    val_flat_accuracy = accuracy_score(true_bools, pred_bools)*100\n","\n","    print('F1 Validation Accuracy: ', val_f1_accuracy)  \n","    print('Flat Validation Accuracy: ', val_flat_accuracy)\n","    print('\\n')\n","    val_f1_accuracy_list.append(val_f1_accuracy)\n","    val_flat_accuracy_list.append(val_flat_accuracy)\n","    epochs_list.append(epochs)  \n","    \n","    wandb.log({\"train_loss\":tr_loss/nb_tr_steps,\"val_f1_accuracy\":val_f1_accuracy,\"val_flat_accuracy\":val_flat_accuracy,})\n","wandb.finish()"]},{"cell_type":"code","execution_count":27,"id":"76da3ba5","metadata":{"execution":{"iopub.execute_input":"2023-10-25T20:00:04.880715Z","iopub.status.busy":"2023-10-25T20:00:04.880388Z","iopub.status.idle":"2023-10-25T20:00:04.88668Z","shell.execute_reply":"2023-10-25T20:00:04.885786Z"},"papermill":{"duration":0.033468,"end_time":"2023-10-25T20:00:04.888712","exception":false,"start_time":"2023-10-25T20:00:04.855244","status":"completed"},"tags":[]},"outputs":[],"source":["num_epochs = np.arange(1,len(training_loss_list)+1)\n","df_train_results=pd.DataFrame({'Epochs':num_epochs,'F1 Validation Accuracy':val_f1_accuracy_list,\\\n","                               'Flat Validation Accuracy':val_flat_accuracy_list,'Train loss':training_loss_list})"]},{"cell_type":"markdown","id":"f8d9aa89","metadata":{"papermill":{"duration":0.024427,"end_time":"2023-10-25T20:00:04.938048","exception":false,"start_time":"2023-10-25T20:00:04.913621","status":"completed"},"tags":[]},"source":["<a id=\"8\"></a>\n","## <p style=\"background-color:#1a0a36;font-family:newtimeroman;color:#FFF9ED;font-size:150%;text-align:center;border-radius:10px 10px;\"> Visualizing The results</p>\n","\n","#### [Top ↑](#top)"]},{"cell_type":"code","execution_count":28,"id":"b664ecc7","metadata":{"execution":{"iopub.execute_input":"2023-10-25T20:00:04.991395Z","iopub.status.busy":"2023-10-25T20:00:04.991041Z","iopub.status.idle":"2023-10-25T20:00:05.266833Z","shell.execute_reply":"2023-10-25T20:00:05.26585Z"},"papermill":{"duration":0.304953,"end_time":"2023-10-25T20:00:05.268961","exception":false,"start_time":"2023-10-25T20:00:04.964008","status":"completed"},"tags":[]},"outputs":[{"data":{"text/plain":["Text(0.5, 1.0, 'Training Loss vs Number of Epochs for Bert-Base')"]},"execution_count":28,"metadata":{},"output_type":"execute_result"},{"data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAm8AAAFWCAYAAAA/q3pwAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAABmpklEQVR4nO3deVhUddsH8O/MwLDv6wACgoKIbImahqKgYgri8pKKZuXWolmalW2a+lQPpfakZWa5ZC5pai64pai4Iea+IKkIguyCyCbbMO8fwOSIy6AwM8D3c11exZkz59yHe5ab33YEMplMBiIiIiJqFoTqDoCIiIiIlMfijYiIiKgZYfFGRERE1IyweCMiIiJqRli8ERERETUjLN6IiIiImhEWb9Qobt26BXd3dyxevPipjzFz5ky4u7s3YlTUWgUFBeHll19WdxhPpbq6GosXL0ZwcDA6duzYot4TW7Zsgbu7O+Lj41VyvrVr12LAgAHo1KkT3N3dcevWLZWcl6ipaak7AGoaDfnAj4mJgYODQxNG07y4u7ujd+/e+Omnn9Qdika5desWgoODAQBvvPEGpk2bVm+foKAg6OvrIzo6WtXhtRh//vknvv/+ewwfPhxdunSBUPj4v7GDgoKQnp7+yMe//vprhIeHN3aYGu/EiROYO3cugoODMXHiRGhpacHc3LzJzxsfH4+xY8cqbBOLxbC2tkbXrl0xYcIEuLq6Ntm5T548iVdeeQXGxsZKPWfx4sX4/vvvFbYZGBjA1tYW/fr1w2uvvQZTU9MmiJaeBYu3Furrr79W+Pn06dPYsGEDRowYgc6dOys81hgfaPb29rhw4QJEItFTH2PevHmYM2fOM8dCTe/XX3/F6NGjYW1tre5QWpzjx4/DyMgIX3zxBQQCgVLPsbW1xfTp0x/62HPPPdeY4TUbx48fBwB8+eWXaik+QkND0atXLwBAeXk5/vnnH/zxxx/Yu3cvduzYAXt7+0Y/58mTJ/H9999j6NChShdvdaZOnSr/I76oqAjx8fFYunQpYmNjsWXLlif+EUGqxeKthXrwL22pVIoNGzbA19f3iX+FFxcXw9DQsEHnEwgE0NHRaXCc99PW1n6m55NqdOrUCZcuXcL333+PuXPnqjsctZPJZCgtLYWBgUGjHC83NxfGxsZKF24AYGRk1Cpb1x4nNzcXABq9cCsrK4OWlha0tB7/9dmxY8d6OXFycsIXX3yBffv24dVXX220mJ7mM/tBvXr1gpeXl/znMWPGYMqUKdi3bx8SExPRsWPHZw2TGhFL6VaubmxQQkICxo8fj86dO2Pw4MEAaj4Qvv32W0RERKBbt27o1KkT+vXrh/nz5+PevXsKx3nYmLf7tx08eBDDhw+Hl5cXAgICEBUVhaqqKoVjPGzMW922oqIizJ49G927d4eXlxdGjhyJ8+fP17ueO3fu4KOPPkK3bt3g5+eHsWPHIiEhAS+//DKCgoIa69cmv773338fPXr0QKdOndC3b18sXLiw3u+moKAAX375Jfr27QsvLy9069YNw4YNwy+//KKw39atW/F///d/8Pf3h6+vL4KDg/Hee+8hPz//kTFIpVIEBARg6NChD338999/h7u7O/bv3w+gpgVg8eLFCAkJgY+PD/z9/REWFoaoqCilr9vHxwf9+vXD5s2bcePGjSfu/6jxZ/Hx8XB3d8eWLVvk2+rGRMXFxeH7779Hnz594O3tjYiICJw7dw5ATevCqFGj4Ovri4CAAPzwww+PPPfly5cxduxY+Pn5oWvXrvjwww+Rl5dXb7+KigosXboUgwYNgpeXF/z9/fHGG28gISHhkTGvXbsWAwcOhJeXF1asWPHE38Mff/yBoUOHwtvbG507d8a4ceNw6tSpeseOj49Heno63N3d4e7ujpkzZz7x2Mqqex+kpaXhzTffROfOnfHcc89h8uTJSEtLq7d/aWkpFixYgL59+6JTp0544YUX8MEHHzy0m1Ymk2Hjxo2IiIiAn58f/Pz8EBYWhu+++67evtXV1Vi+fLn8uCEhIfjzzz/r7Xfo0CGMGTMG3bp1g7e3N3r37o0pU6YgOTn5kddY97lT97qq+z3e/xpMTEzE5MmT0a1bN3h5eWHgwIH4+eefIZVKFY5V9/mTn5+Pjz76CD169ICvry+ysrIe/Ut+jLqW6of9obpr1y6MGjUKfn5+8PHxQUREBPbs2VNvv7rXRFxcnHz/N998EzNnzpR3fwYHB8uv+1nGIT8s3oZ8L1RXV2PVqlUICwuDn58fnnvuOYSEhODjjz9GZWWlwr4XL16U56TuNfHjjz/W+56gGmx5I2RkZOCVV17BgAED0L9/f5SWlgIAsrOzsWnTJvTv3x+hoaHQ0tLCyZMn8csvv+DKlStYvny5UsePjY3FunXrMHLkSAwfPhwxMTFYsWIFTExM8MYbbyh1jPHjx8Pc3ByTJ09GQUEBVq5ciUmTJiEmJkb+F2dFRQVee+01XLlyBcOGDYOXlxf++ecfvPbaazAxMXm6X84jpKenIyIiAkVFRYiMjISTkxNOnjyJn376CWfOnMGqVavkf5m/8847OHXqFEaOHAl3d3eUlZUhKSkJJ0+exIQJEwDUFG4ffvgh/P39MXXqVOjq6iIzMxOxsbHIy8t7ZNe2SCTC4MGDsXz5cly7dg3t27dXeHzr1q0wMzNDYGAgAGDOnDnYvHkzhgwZAj8/P0ilUqSkpDR4APn06dNx4MABLFy4sN54mcYwf/58VFdXY+zYsaisrMSKFSswbtw4fP311/jkk0/w0ksvISwsDLt378aiRYvg4OBQr5UjKysLr776Kvr374+QkBAkJCRg8+bNuHTpEjZt2gQ9PT0AQGVlJcaPH4+zZ88iPDwco0ePRnFxMTZu3IhRo0ZhzZo1Ci0SQE23cUFBASIiImBlZQVbW9vHXs8333yDX375Bd7e3pg+fbr8+K+88gqWLFmCwMBAuLq64uuvv8bSpUvlf4QAgKOj4xN/X1Kp9JFFvpmZmUIrXmlpKV5++WV5LDdv3sS6detw/vx5/Pnnn7CyslL4vZw5cwYhISF47bXXcPPmTaxfvx7Hjh3D5s2bFa77/fffx44dO+Dj44M33ngDRkZGuHHjBvbu3Yt33nlHIaZvv/0WZWVlGDFiBMRiMdavX4+ZM2fC0dFRPqzj5MmTePPNN9G+fXu8/vrrMDIyQk5ODuLi4pCamoq2bds+9HrNzc3x9ddfY+PGjTh16pR8CImlpSWAmiLh5ZdfhpaWFkaPHg1LS0scPHgQ8+fPR2JiIhYsWFDvmK+99hosLS3x1ltvobS0FPr6+k/Myb179+Q5KS8vx9WrV/Htt9/CzMwM/fv3r/f7WLp0KXr27Il33nkHQqEQ+/btwzvvvINZs2Zh9OjRCvtfunQJe/fuxUsvvST/w619+/YoLi7Gvn378NFHH8HMzAyA8uOfi4uL5fEWFxfj5MmT2LJlCzp37ox27drJ92vI98KPP/6IRYsWoU+fPhg5ciREIhFu3bqFAwcOoKKiQl4UHjp0CFOmTIGTkxPGjRsHExMTnDt3DosWLcKVK1ewaNEipa6hVZFRq7B582aZm5ubbPPmzQrb+/TpI3Nzc5Nt3Lix3nPKy8tlFRUV9bZ/++23Mjc3N9n58+fl29LS0mRubm6yRYsW1dvm4+MjS0tLk2+vrq6WDRo0SPbCCy8oHPfDDz+Uubm5PXTb7NmzFbbv2rVL5ubmJlu/fr1825o1a2Rubm6yJUuWKOxbt71Pnz71ruVh3NzcZJMmTXrsPtOnT5e5ubnJDh06pLD9v//9r8Lvs7Cw8KHxP2jy5MkyPz8/WWVlpVIx3u/q1asyNzc3WVRUlML2mzdvytzc3GTz5s2Tb+vSpYtswoQJDT6HTPZvPufMmSOTyWSyTz/9VObm5iY7e/asfJ8+ffrIBg0apPC8Pn36yMaMGVPveCdOnKj3mqx7nQ4ZMkRWXl4u375//36Zm5ubrGPHjrILFy7It5eXl8teeOEF2UsvvVTvnG5ubrKVK1cqbF+5cqXMzc1N9tNPP9XbdvjwYYV9i4qKZIGBgQqx18XcpUsX2e3btx/1q1KQlJQkc3d3l40cOVLhmrKysmSdO3eW9enTR1ZVVSXfPmbMGKVfq/df66P+5eXlKRzbzc1N9p///EfhGH/99ZfMzc1N9tlnn8m3bdiw4aGvq4MHD8rc3NxkM2bMkG/buXOnfJtUKlXY//6f6/IbHh5e73fh6ekpmzZtmnzbl19+KXNzc1P69/ygh32eyGQy2YgRI2QeHh6yK1euyLdVV1fLpk6dKnNzc5MdP3683jHee+89pc9b9xp52L+BAwfKrl+/rrD/pUuXZG5ubrIFCxbUO9abb74p8/PzkxUVFcm31R3r2LFj9fZftGiRzM3NTeHz9knqnvOwf2+++aasuLhYYf+GfC8MGTJE9uKLLz72/GVlZbIePXrIIiMj633+1b03T5w4ofT1tBbsNiWYmppi2LBh9baLxWL5X0ZVVVW4e/cu8vPz0aNHDwB4aLflwwQHByvMZhUIBOjWrRtyc3NRUlKi1DEeHB/y/PPPAwBu3rwp33bw4EGIRKJ6M70iIiJgZGSk1HmUUV1djQMHDqBjx47yFq06r7/+OoRCobybUkdHB2KxGBcuXHjsMgVGRkYoKyvDoUOHIJPJGhRP+/bt4enpiR07dqC6ulq+fevWrQCAIUOGyLcZGhri+vXruHr1aoPO8TBTpkyBnp4evvnmm2c+1oNGjRoFsVgs/9nf3x8A4O3trdAKJhaL4eXlhZSUlHrHMDQ0RGRkpMK2yMhIGBoaYt++ffJt27dvh4uLCzw9PZGfny//V1FRgR49euD06dMoKytTOE54eDgsLCyUupaYmBjIZDJMmDBB4ZpsbGwwbNgwpKen1+uebSh7e3usXLnyof8e9tqfNGmSws/9+vVD27ZtERMTI9+2b98+CIVCvP766wr79u7dGx4eHoiJiZG/3nbs2AEA+PDDD+sNbH/YQPfIyMh6v4u2bdsq5LEu7r179zZa11leXh7Onj2LoKAgdOjQQb5dIBDgzTffBACF10ad8ePHN/hcI0aMkOdg6dKlmDFjBu7cuYNJkyYpdDvv2LEDAoEAQ4YMUXj95efnIygoCCUlJfIhA3U6dOgg/xxuLLNmzZLHu2jRIrz66qs4fPgwpk6dioqKCvl+DfleMDQ0RHZ2tsLwgAcdO3YMt2/fxrBhw1BYWKhw/XUTPo4dO9ao19oSsNuU0KZNm0fOEl27di1+//13XL9+XaEwAIC7d+8qffwH1Q0iLigoUGqg94PHqOsSKCgokG+7desWrK2t6x1PLBbDwcEBhYWFSsX7JPn5+SgtLVXoSqhjamoKKysr+fghsViMjz/+GF988QWCg4PRrl07PP/88+jbty+6d+8uf97rr7+Ov//+G5MnT4apqSm6du2KXr164cUXX1RqIPLQoUPxn//8B8ePH0dAQABkMhm2b9+O9u3bo1OnTvL9Pv74Y3zwwQcICwtDmzZt0K1bN/Tp0wdBQUENnk1mY2ODV155BUuXLsWBAwcadUzhg/mu6/Z+2JI2JiYmCq+D+49xf4EA1OSjTZs2CuO7kpKSUFZWppCPB925cwcSiUT+s7OzszKXAQDyov3BLu37t6WlpdXrmm0IfX19pb/MjY2N5V2j93N1dcX+/fvl3YJ176eHDTlo164drly5gjt37sDCwgI3b96ElZWVvGvySR71mXB/UTN69GjExMRgzpw5mD9/Pjp37oyePXsiNDT0qWfI1+XiYe9dFxcXCIXCh479a0i+6zg5OSnkpE+fPujatSteeuklzJ8/H99++y2AmtefTCbDiy+++Mhj3b59+5niqaioqPd5ra+vr/BZ+eAfRiEhIbCwsMCCBQuwefNmjBo1Sv6Yst8L06dPx+TJk+Uz07t27YrevXsjJCRE/t5MSkoCUPPZ9CgPXj+xeCNAPvbnQStXrsR///tfBAQEYOzYsbC2toa2tjays7Mxc+ZMpVuIHrd8yLMeo6GtVOowatQoBAcHIzY2FidPnsTevXuxZs0aDBw4UP4B7uzsjF27diEuLg5xcXE4efIkPv30UyxatAhr16594rinQYMGISoqClu3bkVAQABOnz6NtLQ0zJgxQ2G/vn374sCBA4iNjcXff/+N48ePY9OmTfD398fKlSvrFTtPMnHiRPz+++/49ttv0bt37wY998HB4fd7VCH5LEvRPIpMJoObm5t8jNnDPFgsPOo9Q8pR5g8FMzMzbNq0CadOncLx48fx999/46uvvsLixYuxbNky+Pn5qSDSGo2Vbx8fHxgZGeHEiRPybTKZDAKBAD///PMjX98PFpsNjefs2bP1eiSmTJmCt99++7HP69mzJxYsWIATJ07Ii7eGfC/4+flh3759OHr0KOLj4xEfH4/o6Gj8+OOPWLduHUxNTeX7f/DBB/Dw8HhoHFySqD4Wb/RI27Ztg729PX7++WeFD9vDhw+rMapHs7e3R1xcHEpKShT+oqysrMStW7cavO7Ro5ibm8PAwADXr1+v99jdu3eRm5tb70PI2toaERERiIiIgFQqxQcffIDo6Gi89tpr8Pb2BlDTKhQYGCjvio2NjcWkSZOwcuVKzJ49+4kx9erVC/v370dJSQm2bt0KoVAonzl8P1NTU4SHhyM8PBwymQzz58/HL7/8gpiYmMf+9f8whoaGePPNN/HVV189dLZg3fke1jL2sBaOxpSWloaKigqFgrSiogJpaWlwcXGRb3NycsKdO3fw/PPPN8laVnWtTNeuXatXhNe9hh7WEtVUCgsLkZubW6/1LSkpCRYWFvLB+G3atMGRI0dQWFhY772TlJQEQ0NDeQu4s7MzYmJicPv2baVb35QhEonQrVs3dOvWDUDNLNHhw4fjxx9/xLJlyxp8vLqW24e9d2/cuIHq6uomz4VUKlXohnR2dsaRI0dgZ2f3zIv3Pmp5mQ4dOmDlypUK25S5zroZofcPb2no94KBgQFCQkIQEhICoKbVbu7cudi0aRMmTJggb0XU09Nr9K7gloxj3uiRhEIhBAKBwl9SVVVV+Pnnn9UY1aMFBQVBKpVi9erVCts3btyIoqKiRjuPUChEnz59kJCQUO8Da9myZaiurkbfvn0B1Mw4e3D6vEgkks8Aq+tieNhMwbp1lZTtnh46dCju3buH7du3Y8+ePejRowdsbGzkj0ul0npdxwKBoMHneVBkZCTs7e2xePFihS+lOs7OzkhOTkZ2drZ8W0VFBdauXftU51NWcXEx1q1bp7Bt3bp1KC4ulucHqBkTmJubW+/Lrc6zdtkEBQVBIBBg+fLlCssj5OTkYMuWLbC3t1f5GloPFj779u1DcnKywu+lb9++qK6urrdvbGwsEhISFLraw8LCANTMqn2wG+1pW8cf9p5wcXGBjo7OU79WLSws4Ofnh4MHDyqM+5TJZPLr7Nev31MdWxnHjh1DaWkpPD095dvq/sBauHDhQ1ujG/L6qyu8H/z9mJiYoEePHgr/lCne6sZA3h9vQ74XHpbDumPVxRgQEAALCwv8/PPPD/0jr6ysDMXFxU+MtbVhyxs90oABA7BgwQJMnDgR/fr1Q3FxMaKjo5+4OKW6RERE4Pfff8f//vc/pKamypcK2bNnD5ycnBo06PnmzZtYsmTJQx979dVXMX36dBw/fhyTJ09GZGQkHB0dcerUKezatQtdunSRT99PSUnBmDFj0K9fP7Rv3x7Gxsa4ceMG1q9fDwcHB/lA/PHjx8PIyAj+/v6QSCQoLCzEn3/+CYFAoPTiq4GBgTA1NcX8+fNRXFxcb+23kpISBAQEICgoCB07doS5uTlu3bqF9evXw8TEBH369FH693M/sViMd955Bx988AGA+ouijh49Gjt37sSrr76KkSNHorKyEtu2bWvyrkdHR0f88MMPuHbtGjw9PXH58mVs3rwZLi4uCmt+jR07FsePH8fXX3+NEydO4Pnnn4ehoSEyMjJw4sQJiMVi/Pbbb08dh4uLC8aPH49ffvkFY8aMwYsvvoiSkhJs3LgRpaWlmD9//jN3BxcVFWHbtm0Pfczd3V1hcL6ZmRn27duHnJwcdO3aVb5UiKWlJaZMmSLfb+jQofjzzz/x888/Iz09Hf7+/khNTZXve/8dHV588UX89ddf2Lp1K27evImgoCAYGxsjJSUFR48efarbpX322WfIyspCQEAA7OzsUFZWht27d6OkpOSZFiT+5JNP8PLLL2P06NGIjIyElZUVDh48iKNHjyI0NPSxYx8bIiEhQZ6TiooKXL9+HRs3boS2tjbeffdd+X7e3t54++23sXjxYgwZMgQhISGwsbFBTk4OLl++jMOHD+PSpUtKndPHxwdAzVI7YWFh0NHRQfv27eHm5vbE5x4+fFi+bmNxcTHOnDmDnTt3wtbWVqHLtSHfCwMHDoSvry+8vb1hbW2N3Nxc+e9g0KBBAGoKzqioKEyePBkDBgzA8OHD4eTkhMLCQty4cQP79u3D999/L299pRqa+S1MGmH8+PGQyWTYtGkTvvjiC1hZWeHFF1/E8OHDMXDgQHWHV49YLMavv/6Kr7/+GjExMdi9eze8vb2xatUqfPLJJ/VmDD5OcnLyQxcXBWqKRHt7e2zcuBGLFi3C9u3bUVRUBBsbG7z++ut488035R9ktra2GD58OOLj47F//35UVFTAxsYGERERmDhxoryAGTVqFHbv3o0NGzbg7t27MDU1hYeHBz799FP5zFplrj80NBRr1qyBoaGhQisKAOjq6uKVV16Rj6srKSmBtbU1goKC8Prrryu00jXU4MGDsXLlSly5cqXeY507d8Z///tfLF26FN988w2sra0xatQodOrUqVFXmX+Qra0t/ve//yEqKgo7d+6EtrY2wsLC8OGHHyqs06WtrY2ffvoJ69atw7Zt2+SLmlpbW8PLy+uRCyA3xPvvvw8nJyesW7cOCxYsgLa2Nnx8fLBgwQJ5Af8ssrKy5MXzg9544w2F4k1fXx+//vorvvzySyxYsAAymQw9e/bEzJkzFcYWaWtrY/ny5fjxxx+xa9cu7Nu3D0ZGRhgwYADeffddhQkcAOTXsmnTJvzwww8QCoVwcHDAgAEDnuqawsPDsWXLFvz555/Iz8+HoaEh2rVrh0WLFsm74J6Gl5cXfv/9dyxatAjr169HaWkp2rRpgxkzZmDcuHFPfdwHRUdHy4tWoVAIU1NTvPDCC5g0aZJ8qESdKVOmoFOnTvjtt9+wevVqlJaWwsLCAu3bt8cnn3yi9Dk7d+6MGTNm4Pfff8dnn32GqqoqTJkyRani7f611LS0tGBjY4MRI0Zg8uTJCjOrG/K9MG7cOMTGxuK3335DUVERLCws4OPjg9dff13hNdmzZ09s2rQJy5Ytw/bt23Hnzh0YGxvD0dERr776aoPu1d1aCGTNYcQ30TOQSqV4/vnn4e3trfTCwkQt0csvv4z09HQcOHBA3aEQ0TPgmDdqUR7Wuvb777+jsLAQL7zwghoiIiIialzsNqUW5dNPP0VFRQX8/PwgFotx9uxZREdHw8nJCS+99JK6wyMiInpmLN6oRQkICMDatWsRFxcnHzcSERGBd955R6nFbomIiDQdx7wRERERNSMc80ZERETUjLB4IyIiImpGWtWYtzt3SlBd3XS9xBYWhsjL40rQmoZ50TzMiWZiXjQPc6KZmjovQqEAZmYGj3y8VRVv1dWyJi3e6s5Bmod50TzMiWZiXjQPc6KZ1JkXdpsSERERNSMqK96Sk5MxYsQIhISEYMSIEUhJSXnkvjdu3ICPjw+ioqLk2+bMmYMBAwZg8ODBGDlyJC5evKiCqImIiIg0i8qKt9mzZyMyMhJ79+5FZGQkZs2a9dD9pFIpZs+eXe++jL169cKOHTuwfft2vP7665g2bZoqwiYiIiLSKCop3vLy8pCQkIDQ0FAAQGhoKBISEpCfn19v32XLlqF3795wdnZW2N6nTx9oa2sDAHx9fZGVlYXq6uomj52IiIhIk6hkwkJmZiZsbGwgEokAACKRCNbW1sjMzIS5ubl8v8TERBw9ehSrV6/GkiVLHnm8tWvXonfv3hAKG1Z7Wlg0/Qr7VlZGTX4OajjmRfMwJ5qJedE8zIlmUmdeNGa2aWVlJT777DN89dVX8iLvYXbu3IkdO3Zg7dq1DT5HXl5xk84OsbIyQm5uUZMdn54O86J5mBPNxLxoHuZEMzV1XoRCwWMbnFRSvEkkEmRnZ0MqlUIkEkEqlSInJwcSiUS+T25uLlJTUzFp0iQAQGFhIWQyGYqLizFv3jwAwL59+/Dtt99i1apVsLS0VEXoSom7nIUtsUnILyyHubEOhgW6orunrbrDIiIiohZIJcWbhYUFPDw8EB0djfDwcERHR8PDw0Ohy9TOzg7x8fHynxcvXozS0lJ8+OGHAICDBw/iq6++wsqVK+Hg4KCKsJUSdzkLv+5OREVVzfi7vMJy/Lo7EQBYwBEREVGjU9ls088//xxr1qxBSEgI1qxZgzlz5gAAJk6cqNSyHx999BEqKysxdepUhIeHIzw8HHfu3GnqsJ9oS2ySvHCrU1FVjS2xSWqKiIiIiFoygUwmazVLNzfFmLdx/z3wyMdWzAxq1HPR0+GYEc3DnGgm5kXzMCeaSd1j3niHhWdkYazToO1EREREz4LF2zMaFugKsVb9XyPHuxEREVFTYPH2jLp72uKVFzvAwlgHAgDmRjow0tdG3OUslJZVqjs8IiIiamE0Zp235qy7py26e9rK+8BvZBTiy99O47e/ruL1wZ7qDo+IiIhaELa8NQEXO2OEBzgjPiEbcZez1B0OERERtSAs3prIoO7OaOdggjV//YPbBffUHQ4RERG1ECzemohQKMCk0I4AgJ+jE5r0tlxERETUerB4a0KWpnoY088d127dxc4TN9UdDhEREbUALN6a2POeNujqYY3tR5ORnFmo7nCIiIiomWPx1sQEAgHGhrjDxFCMZdsvo6yiSt0hERERUTPG4k0F9HW1MTG0I3Lu3MPvMdfUHQ4RERE1YyzeVMTd0QwvPu+Ew+czcfqfXHWHQ0RERM0UizcVGtKzLZxsjPDrnkTcKSpXdzhERETUDLF4UyEtkRCTBndERaUUK3YmoFrG5UOIiIioYVi8qZjEwgAjg9vjcsod7D91S93hEBERUTPD4k0NAn3t4NvOEpsOJeFWTrG6wyEiIqJmhMWbGggEArw6sAP0dbXw047LqKySqjskIiIiaiZYvKmJsb4Y4wd5ID23BH8cSlJ3OERERNRMsHhTIy8XCwR3dsD+U7dwKTlP3eEQERFRM8DiTc0iervC3tIAy6OvoKi0Qt3hEBERkYZj8aZmYm0RJg32RElZJVbtToSMy4cQERHRY7B40wBtrA0xPNAVZ6/dxuHzGeoOh4iIiDQYizcN0a9LG3R0NsP6mGvIyi9VdzhERESkoVRWvCUnJ2PEiBEICQnBiBEjkJKS8sh9b9y4AR8fH0RFRcm33bt3D++++y769euHAQMG4ODBgyqIWnWEAgHGD+oIbZEQy7ZfRpW0Wt0hERERkQZSWfE2e/ZsREZGYu/evYiMjMSsWbMeup9UKsXs2bPRt29fhe3Lly+HoaEh9u3bh6VLl+LTTz9FSUmJKkJXGTMjHbz6YgekZBVh29FkdYdDREREGkglxVteXh4SEhIQGhoKAAgNDUVCQgLy8/Pr7bts2TL07t0bzs7OCtt3796NESNGAACcnZ3RqVMnHD58uMljV7XO7tYI8JZgV9xNXE0rUHc4REREpGG0VHGSzMxM2NjYQCQSAQBEIhGsra2RmZkJc3Nz+X6JiYk4evQoVq9ejSVLligcIyMjA/b29vKfJRIJsrKyGhSHhYXhM1yFcqysjJ75GFNHPoekjENYvusKFr3XB4Z62o0QWevWGHmhxsWcaCbmRfMwJ5pJnXlRSfGmjMrKSnz22Wf46quv5EVeY8vLK0Z1ddMtxWFlZYTc3KJGOda4gR3w1W9n8N2605g02LNRjtlaNWZeqHEwJ5qJedE8zIlmauq8CIWCxzY4qaR4k0gkyM7OhlQqhUgkglQqRU5ODiQSiXyf3NxcpKamYtKkSQCAwsJCyGQyFBcXY968ebCzs0N6erq8pS4zMxPdunVTRfhq4WpngsEBzth6JBnerhZ43tNW3SERERGRBlDJmDcLCwt4eHggOjoaABAdHQ0PDw+FLlM7OzvEx8fjwIEDOHDgAF555RW89NJLmDdvHgBgwIAB2LBhAwAgJSUFFy9eRM+ePVURvtoM6u6Edg4m+O2vf3C74J66wyEiIiINoLLZpp9//jnWrFmDkJAQrFmzBnPmzAEATJw4ERcvXnzi88ePH4/CwkL069cPr7/+OubOnQtDw6Yfw6ZOIqEQE0M7QiYDfolOaNIuXyIiImoeBLJWdD+m5jTm7X7HL2Xil+grGNbLBaE9nBv9+C0dx4xoHuZEMzEvmoc50UzqHvPGOyw0A909bdHVwxrbjiYjObNQ3eEQERGRGrF4awYEAgFeDnGHiaEYy7ZfRllFlbpDIiIiIjVh8dZMGOhqY2JoR+TcuYffY66rOxwiIiJSExZvzYi7oxkGPO+Iw+czcOZqrrrDISIiIjVg8dbMDO3pAicbI6zanYg7ReXqDoeIiIhUjMVbM6MlEmLS4I6oqJRixa4rqG49k4WJiIgILN6aJYmFAUYEt8fl5HzEnLql7nCIiIhIhVi8NVO9fe3g284SfxxKwq2cYnWHQ0RERCrC4q2ZEggEeHVgB+jrauGnHZdRWSVVd0hERESkAizemjFjfTHGDfRAem4JNh26oe5wiIiISAVYvDVz3q4WCO7sgH2n0nApOU/d4RAREVETY/HWAkT0doW9pQGWR19BUWmFusMhIiKiJsTirQUQa4swMawjSsoqsWp3ImRcPoSIiKjFYvHWQjjaGGFYL1ecvXYbRy5kqjscIiIiaiIs3lqQ/l3bwMPJDOv2X0VWfqm6wyEiIqImwOKtBREKBJgQ2hHaIiF+3nEZVdJqdYdEREREjYzFWwtjZqSDVwZ0QHJmEbYfS1Z3OERERNTIWLy1QP4drBHgLcHOuJu4mlag7nCIiIioEbF4a6Ei+7aHlYkeft6RgNKyKnWHQ0RERI2ExVsLpSvWwsTBHXGnqBxr9v2j7nCIiIiokbB4a8Fc7UwwOMAZJy5n48TlLHWHQ0RERI2AxVsLN6i7E9rZm+C3v/7B7bv31B0OERERPSMWby2cSCjExLCOkMmAX3YkoLqad18gIiJqzlRWvCUnJ2PEiBEICQnBiBEjkJKSUm+fzZs3IywsDOHh4QgLC8Pq1avlj+Xl5WHSpEkICwvDiy++iM8//xxVVRyIrwwrUz2M7ueGq7fuYnf8TXWHQ0RERM9AZcXb7NmzERkZib179yIyMhKzZs2qt09ISAi2b9+Obdu2Yf369Vi5ciUSExMBAEuXLoWrqyt27NiB7du34/Lly/jrr79UFX6z16OTLbp6WGPrkWQkZxaqOxwiIiJ6Siop3vLy8pCQkIDQ0FAAQGhoKBISEpCfn6+wn6GhIQQCAQCgrKwMlZWV8p8FAgFKSkpQXV2NiooKVFZWwsbGRhXhtwgCgQAvh7jDxFCMZTsSUF4hVXdIRERE9BS0VHGSzMxM2NjYQCQSAQBEIhGsra2RmZkJc3NzhX1jYmKwcOFCpKam4r333oO7uzsA4K233sLbb7+NgIAA3Lt3D6NHj0bnzp0bFIeFhWHjXNBjWFkZNfk5npYVgBmj/fHJ0mPYejwFUyJ81R2SymhyXlor5kQzMS+ahznRTOrMi0qKt4YIDg5GcHAwMjIyMHnyZPTq1QsuLi7Ys2cP3N3d8euvv6KkpAQTJ07Enj17MGDAAKWPnZdX3KQD9q2sjJCbW9Rkx28MtiY6GNDNEbtP3ISbnTH83KzUHVKTaw55aW2YE83EvGge5kQzNXVehELBYxucVNJtKpFIkJ2dDam0pqtOKpUiJycHEonkkc+xs7ODl5cXDh06BABYs2YNBg8eDKFQCCMjIwQFBSE+Pl4V4bc4Q3u6wMnGCCt3J6KguFzd4RAREVEDqKR4s7CwgIeHB6KjowEA0dHR8PDwqNdlmpSUJP///Px8xMfHw83NDQDg4OCAw4cPAwAqKioQFxeH9u3bqyL8FkdLJMSkwR1RUSnFip1XUC3j8iFERETNhcpmm37++edYs2YNQkJCsGbNGsyZMwcAMHHiRFy8eBEAsGHDBgwaNAjh4eF49dVXMWbMGAQEBAAAPv74Y5w+fRphYWEYMmQInJ2d8dJLL6kq/BZHYmGAEcHtcSk5HzGnb6k7HCIiIlKSQCZrPc0uHPOmSCaTYfHmi7iUnI9Zr/jDwbrpJ3SoQ3PLS2vAnGgm5kXzMCeaqVWMeSPNJBAI8OqLHaCvI8KyHZdRWcXlQ4iIiDQdi7dWzthAjHGDOuJWbgk2x95QdzhERET0BCzeCN6uFgh+zgF//Z2GS8l56g6HiIiIHoPFGwEAIvq4ws7SAMt3XkFRaYW6wyEiIqJHYPFGAACxtgiTwjqiuLQSv+75B61oHgsREVGzwuKN5BxtjDA80BVnrubiyIVMdYdDRERED8HijRT079oGHk5mWLf/KrLzS9UdDhERET2AxRspEAoEGD/IA9oiIZbtuIwqabW6QyIiIqL7sHijesyNdfHKgA5IzizC9mMp6g6HiIiI7sPijR7Kv4M1Arwk2BmXgqtpBeoOh4iIiGqxeKNHGtW3PaxM9PDzjgSUllWpOxwiIiICizd6DD0dLUwM64g7ReVYu+8fdYdDREREYPFGT+Bqb4LBLzgj7nI2TiRkqTscIiKiVo/FGz3RoB5OaGdvgt/2XsXtu/fUHQ4REVGrxuKNnkgkFGJCWEfIZDL8En0F1dW8+wIREZG6sHgjpVib6mF0PzdcTSvA7vib6g6HiIio1WLxRkrr0ckWXTpYY+uRZCRnFqo7HCIiolaJxRspTSAQYOwAdxgbiLFsRwLKK6TqDomIiKjVYfFGDWKgq40JoR2Rk1+KDQeuqTscIiKiVofFGzWYh5MZBnRzxKFzGTh7NVfd4RAREbUqLN7oqQzt5QJHG0Os3J2Iu8Xl6g6HiIio1Xiq4u3EiRM4efJkY8dCzYiWSIhJYZ6oqJRi+c4rqJZx+RAiIiJVUKp4GzNmDE6fPg0AWLZsGaZPn4733nsPS5cuVfpEycnJGDFiBEJCQjBixAikpKTU22fz5s0ICwtDeHg4wsLCsHr1aoXHd+3ahbCwMISGhiIsLAy3b99W+vzU+OwsDTAiqB0uJecj5vQtdYdDRETUKmgps9O1a9fg6+sLAPjjjz+wevVqGBgYYNSoUXjjjTeUOtHs2bMRGRmJ8PBwbNu2DbNmzapXnIWEhGDYsGEQCAQoLi5GWFgYunbtig4dOuDixYv4/vvv8euvv8LKygpFRUUQi8UNu1pqdL397HEhKQ9/HEyCh5MZHKwM1R0SERFRi6ZUy1t1dTUEAgFSU1Mhk8nQrl07SCQS3L17V6mT5OXlISEhAaGhoQCA0NBQJCQkID8/X2E/Q0NDCAQCAEBZWRkqKyvlP69atQrjxo2DlZUVAMDIyAg6OjrKXSU1GYFAgNcGekBfR4Rl2xNQWcXlQ4iIiJqSUsVb586dMXfuXERFRaFfv34AgNTUVJiZmSl1kszMTNjY2EAkEgEARCIRrK2tkZmZWW/fmJgYDBo0CH369MGECRPg7u4OAEhKSkJaWhpGjx6NoUOHYsmSJZBxnJVGMDYQY9wgD9zKLcbm2BvqDoeIiKhFU6rb9KuvvsLKlSthbm6OCRMmAABu3LiBsWPHNnpAwcHBCA4ORkZGBiZPnoxevXrBxcUFUqkU//zzD1auXImKigpMmDABdnZ2GDJkiNLHtrBo+i49KyujJj+HJgq2MsK1jCLsPJaMAD8H+LlbqzskBa01L5qMOdFMzIvmYU40kzrzolTxZmZmhunTpyts6927t9InkUgkyM7OhlQqhUgkglQqRU5ODiQSySOfY2dnBy8vLxw6dAguLi6ws7PDgAEDIBaLIRaLERwcjAsXLjSoeMvLK27Sm6pbWRkhN7eoyY6v6cKed8SZxGwsXHcac8d3g6GetrpDAsC8aCLmRDMxL5qHOdFMTZ0XoVDw2AYnpbpNV65ciStXrgAAzp07h969eyMoKAhnz55VKggLCwt4eHggOjoaABAdHQ0PDw+Ym5sr7JeUlCT///z8fMTHx8PNzQ1AzTi5o0ePQiaTobKyEidOnECHDh2UOj+phlhbhNcHe6KotBKrdieyW5uIiKgJKNXytmrVKvzf//0fAGDBggV49dVXYWBggC+//BJ//PGHUif6/PPPMXPmTCxZsgTGxsaIiooCAEycOBFTp06Fl5cXNmzYgGPHjkFLSwsymQxjxoxBQEAAAGDQoEG4dOkSBg4cCKFQiICAAHlMpDkcbYwwPNAVGw9ex5ELmejlY6fukIiIiFoUgUyJ5pHnnnsOZ86cQXFxMYKCghAXFweRSAR/f3+cOnVKFXE2Cnabqka1TIYFv5/DjYxCfP5aF9iY66s1HuZF8zAnmol50TzMiWZqFt2mEokEZ86cwa5du+Dv7w+RSITi4mL57FGi+wkFAowf5AEtkQDLdiSgSlqt7pCIiIhaDKWKtw8++ABTp07F0qVL8dZbbwEADh48CC8vryYNjpovc2NdvDKgA5IzC7H9WIq6wyEiImoxlBrzFhgYiKNHjypsGzBgAAYMGNAkQVHL4N/BGi942WJnXAo6tTWHWxtTdYdERETU7ClVvAFASkoKoqOjkZOTA2tra4SGhsLZ2bkJQ6OWILKvG66mFeCX6AR8/lpX6Osq/ZIjIiKih1Cq2/TAgQMYNmwYkpOTYWJiguTkZAwfPhwxMTFNHR81c3o6WpgU5on8wnKs3fePusMhIiJq9pRqBvn222+xZMkSPP/88/Jt8fHxmDdvHoKDg5ssOGoZXO1NEPaCM7YdTYa3qyW6dbRRd0hERETNllItb1lZWfD391fY1rlzZ2RlZTVJUNTyhPZwgqu9MVbv/Qd5d8vUHQ4REVGzpVTx1qFDB6xYsUJh28qVK+Hh4dEkQVHLIxIKMTHMEzKZDL9EJzTpentEREQtmVLdpp9//jnefPNNrF69GhKJBJmZmdDT08PSpUubOj5qQaxN9TC6nxuW77yC3fE3Mai7s7pDIiIianaUKt5cXV2xa9cunDt3Tj7b1NvbG9nZ2U0dH7UwPTrZ4nxSHrYeSYZnW3M42xqrOyQiIqJmRaluUwDQ0tKCv78/Bg4cKB//1r9//yYLjFomgUCAsSHuMDYQY9n2BJRXSNUdEhERUbOidPH2MErcFpWoHkM9bUwI7Yjs/FJsOHBN3eEQERE1K89UvAkEgsaKg1oZDyczhHRzxKFzGTh7LVfd4RARETUbz1S8ET2LoT1d4GhjiJW7EnG3uFzd4RARETULj52wEBgY+MjWNXaZ0rPS1hJiUpgn5qz6G8t3XcG0CB+25hIRET3BY4u3b775RlVxUCtlZ2mAEUHtsOavq4g5fQt9/duoOyQiIiKN9tjirWvXrqqKg1qxPn72uJCUh40Hk+DhZAZ7K0N1h0RERKSxOOaN1E4gEGDcQA/o64jw0/YEVFZVqzskIiIijcXijTSCsYEY4wZ54FZuMTbHJqk7HCIiIo3F4o00hrerJYKes8dff6fhckq+usMhIiLSSCzeSKO81KcdJBb6WB6dgOJ7leoOh4iISOModW/T999//6FLOIjFYtja2qJv377o0KFDowdHrY9YW4TXB3ti3q+n8OvuRLw1tBOXDyEiIrqPUi1vRkZGiImJgUwmg62tLWQyGQ4cOAChUIikpCSMGDECW7dubeJQqbVwtDHCsEAXnL6ai6MXMtUdDhERkUZRquUtJSUFy5YtQ+fOneXbzp49i0WLFmHlypU4fPgwvvzySwwZMuSRx0hOTsbMmTNRUFAAU1NTREVFwdnZWWGfzZs3Y9WqVRAKhaiurkZERATGjh2rsM+NGzcwdOhQREZG4sMPP1T+SqlZCenqiEs38rFu/zW4OZrCxkxf3SERERFpBKVa3s6fPw8fHx+FbZ06dcKFCxcAAD179kR2dvZjjzF79mxERkZi7969iIyMxKxZs+rtExISgu3bt2Pbtm1Yv349Vq5cicTERPnjUqkUs2fPRt++fZUJm5oxoUCA8YM8oCUSYNn2BFRJuXwIERERoGTx5uHhgW+//Rbl5TX3nywvL8d3330nH+d269YtmJiYPPL5eXl5SEhIQGhoKAAgNDQUCQkJyM9XnFFoaGgoH99UVlaGyspKhfFOy5YtQ+/eveu12FHLZG6si7EDOiA5sxA7jqWoOxwiIiKNoFS36X//+1/MmDEDnTt3homJCe7evYtOnTrJb59VUFCA2bNnP/L5mZmZsLGxgUgkAgCIRCJYW1sjMzMT5ubmCvvGxMRg4cKFSE1NxXvvvQd3d3cAQGJiIo4ePYrVq1djyZIlT3WxFhZNv3K/lZVRk5+jNRloZYSr6XexMy4FAc85oGNbi6c6DvOieZgTzcS8aB7mRDOpMy9KFW8ODg74/fffkZmZiZycHFhZWcHOzk7+uJeXV6MFFBwcjODgYGRkZGDy5Mno1asX2rRpg88++wxfffWVvAB8Gnl5xaiuljVarA+ysjJCbm5Rkx2/tRoW0BYXruXim99O4fPXukJfV6mXrRzzonmYE83EvGge5kQzNXVehELBYxucGvQtKBaLYW5uDqlUirS0NABAmzZPvpG4RCJBdnY2pFIpRCIRpFIpcnJyIJFIHvkcOzs7eHl54dChQxgwYABSU1MxadIkAEBhYSFkMhmKi4sxb968hlwCNUN6OlqYGOaJ/645g7X7rmJiWEd1h0RERKQ2ShVvhw8fxieffILc3FyF7QKBAFeuXHni8y0sLODh4YHo6GiEh4cjOjoaHh4e9bpMk5KS4OrqCgDIz89HfHw8+vfvDzs7O8THx8v3W7x4MUpLSznbtBVpZ2+CsBecse1oMnzaWaCrh426QyIiIlILpYq3uXPn4q233sLQoUOhq6v7VCf6/PPPMXPmTCxZsgTGxsaIiooCAEycOBFTp06Fl5cXNmzYgGPHjkFLSwsymQxjxoxBQEDAU52PWp7QHk64lJyH1Xv+gaudCSxMnu61SERE1JwJZDLZEweBde3aFfHx8c1+pXuOeWv+cgruYfaKk3C2McL7o/wgFD75Ncm8aB7mRDMxL5qHOdFM6h7zptRSIcOHD8fmzZsbLSiip2VtqofRfd3wT1oB9pxMVXc4REREKqdUt+n58+fx22+/4eeff4alpaXCY2vXrm2SwIge5QUvW1y4kYc/D99AR2czONsaqzskIiIilVGqeIuIiEBERERTx0KkFIFAgLEh7khKv4tl2xMw+7Uu0NF++iVkiIiImhOlirehQ4c2dRxEDWKop40Jgzww//dz2HDgOsaGuKs7JCIiIpV4ZPG2detW+Y3mN23a9MgD/N///V+jB0WkDA9nc4R0c8Se+FR4u1jAt73lk59ERETUzD2yeNu5c6e8eNu2bdtD9xEIBCzeSK2G9nRBQnI+Vuy6gnnju8LEUEfdIRERETUppZYKaSm4VEjLlHG7BHNW/Q13R1NMi/Cpt6QN86J5mBPNxLxoHuZEMzWLpULq5OXlIS0tTeEfkbrZWRpgRFA7XLqRjwNn0tUdDhERUZNSye2xiJpaHz97XEjKw8aD19HB0RT2Vo/+i4WIiKg5U6rlre72WOfOnUNiYqL8Hws30hQCgQCvDfSArliEn7YnoLKqWt0hERERNQmlirfCwkKMHDnyqe9rSqQKJgZijBvogVu5xdhyOEnd4RARETUJ3h6LWhSfdpbo85w99p5Mw+WUfHWHQ0RE1Oh4eyxqcV7q0w6JN+9gyZ8XoSvWQkFROcyNdTAs0BXdPW3VHR4REdEz4e2xqMXR0Rahe0cbbDmSjHvlUgBAXmE5ft2dCAAs4IiIqFnj7bGoRYo9n1FvW0VVNbbEJrF4IyKiZo23x6IWKa+w/JHbq2UyCB9YyJeIiKi54O2xqEWyMNZ5ZAH30U9x6OVjhwAvCW+nRUREzQ5vj9WIeBsTzRF3OQu/7k5ExX3rvYm1hHjB2xaZt0uRmFoAkVAA33aWCPS1Q8e25myNUyG+VzQT86J5mBPNpO7bYyk15u1+MpkM99d7QmGD7rBFpBJ149q2xCYhv7D+bNPMvBIcOZ+JoxczcfpqLixNdNGztjXOzIitcUREpLmUannLzs7G3LlzcerUKRQWFio81pzussCWt9bpcXmprKrG2Wu5iD2XgSs370AoEMCnnQUCfe3Rqa05hEK2xjUFvlc0E/OieZgTzdQsWt5mz54NXV1drFq1CmPGjMHatWuxePFiBAYGNlqgROqgrSVEVw8bdPWwQfadUhw+n4FjFzJx9tptWBjroKe3HQK8JTA35t1FiIhIMyjV8tatWzccPHgQ+vr68Pf3x6lTp1BQUICRI0diz549qoizUbDlrXVqaF6qpNU4d+02Ys+l43LKHQgEgI+rJXr52sHbxYKtcY2A7xXNxLxoHuZEMzWLljehUAgtrZpdjY2NkZ+fD0NDQ2RnZzdOlEQaREskhH8Ha/h3sEZOwT0cOZ+BIxcyce76bZgZ6aCntwQ9ve1gYcLWOCIiUj2lijcfHx/ExsaiX79+CAgIwLvvvgtdXV106tRJ6RMlJydj5syZKCgogKmpKaKiouDs7Kywz+bNm7Fq1SoIhUJUV1cjIiICY8eOBQD88MMP2LVrF4RCIbS1tTFt2jT07NlT+SslegrWpnoYHuiK8IC2OH/9NmLPZ2DHsRTsOJ4CLxcLBPrYwbudBUScuENERCqiVLdpYWEhqqurYWpqirKyMqxYsQIlJSV45ZVXYG1trdSJxo4di+HDhyM8PBzbtm3D5s2bsXr1aoV9iouLYWBgAIFAgOLiYoSFheHHH39Ehw4dcOTIEfj7+0NPTw+JiYkYM2YMjh49Cl1d5Vs/2G3aOjV2Xm4X3MPhC5k4ciEDd4srYGooRoC3HXp5S2Bpqtdo52nJ+F7RTMyL5mFONJO6u02f2FwglUrxxRdfQF9fHwCgq6uLt956C++//77ShVteXh4SEhIQGhoKAAgNDUVCQgLy8/MV9jM0NISgdq2tsrIyVFZWyn/u2bMn9PRqvhjd3d0hk8lQUFCg1PmJGpOlqR6G9XLB/Ld64O1hXnC0McLO4yn4cGkcFm48h9P/5KJKWv3kAxERET2FJ3abikQiHDt2TF5EPY3MzEzY2NhAJBLJj2ltbY3MzEyYm5sr7BsTE4OFCxciNTUV7733Htzd3esdb+vWrXB0dIStbcPuUfm4KraxWFkZNfk5qOGaKi+2Nibo/4ILcu6UYv/JVPwVfxM//HkRZkY66NvVEf27OcHWwqBJzt3c8b2imZgXzcOcaCZ15kWpMW+vvPIKFi9ejLfffhva2tpNGlBwcDCCg4ORkZGByZMno1evXnBxcZE/fvLkSXz33XdYsWJFg4/NbtPWSRV5EQDo95w9gnwluHgjH4fPZWDTgWv4I+YaPJ3NEOhrD9/2ltAScWwcwPeKpmJeNA9zopnU3W362OItOjoaoaGhWLNmDW7fvo2VK1fC3NxcoRXu0KFDTwxCIpEgOzsbUqkUIpEIUqkUOTk5kEgkj3yOnZ0dvLy8cOjQIXnxdvbsWbz//vtYsmSJQkFHpClEQiF821nCt50l8gvLcPRCJg5fyMCSrZdgrK+NF7wl6OVjBxszfXWHSkREzdRji7dZs2YhNDQU33zzzTOdxMLCAh4eHoiOjkZ4eDiio6Ph4eFRr8s0KSkJrq6uAID8/HzEx8ejf//+AIALFy5g2rRpWLRoETw9PZ8pHiJVMDfWxeCAtgjt4YxLyXmIPZeBvfFp2H0iFR5OZgj0tYNfeytoa7E1joiIlPfY2aZ+fn44e/Zso5woKSkJM2fORGFhIYyNjREVFQUXFxdMnDgRU6dOhZeXF7788kscO3YMWlpakMlkiIiIwMsvvwwAGD58ONLT02FjYyM/5tdff/3QMXGPwm7T1kmT8nKnqBxHL2bi8LkM5BWWwVBPGwFeEvTytYOteetpjdOknNC/mBfNw5xoJnV3mz62ePPx8cFPP/2Ex60m0r1792eLUIVYvLVOmpiXapkMCcn5iD2XgXPXb0NaLUMHR1P08rVDZzfrFt8ap4k5IeZFEzEnmkndxdtju00rKirwySefPLJ4EwgEiImJebYIiVohoUCATi4W6ORigbvFta1x5zOwbHsCDPWuoUcnWwT62kHCmapERPSAxxZvenp6LM6ImpiJoQ4GdXfGi8874crNO4g9l4GY07fw199pcHMwQaCvPTq7W0GsLVJ3qEREpAGUWiqEiJqeUCCAp7M5PJ3NcbekAscvZiL2fAZ+jk7Auv1a6N7JFoE+drC3avr1ComISHM9tnhT4s5ZRNQETAzEePF5J4R0c8Q/N+8g9nwGDp5Jx/5Tt9DOwQSBPnbw72ANHbbGERG1Oo8t3hprpikRPR2hQAAPZ3N4OJujsLQCxy9mIfZ8BpbvvIJ1+6+hh2fN2DgHa7bGERG1Fuw2JWomjPXFGNDNESFd2+BqWgFiz2Ug9nw6Ys7cgqudMXr52qFrBxvoiNkaR0TUkrF4I2pmBAIB3B3N4O5ohsh7bvKxcSt3JeL3mGt43rNmbJyjDe+HSETUErF4I2rGDPW00b+rI/p1aYNrt+4i9lwGjpzPxMEz6WgrMUagrx26elhDV8y3OhFRS8FPdKIWQCAQwK2NKdzamGJU3/aIu5yFw+cysGp3ItbHXEP3jjYI9LWHky1b44iImjsWb0QtjKGeNvr5t0Hfzg5ISi9E7Pl0HL+UhUPnMuBka4RAXzt087CBng7f/kREzRE/vYlaKIFAgHYOJmjnYIJRwe0RdzkbsefSsXrPP9gQcx3dOtog0NcOzrZGEAgE6g6XiIiUxOKNqBXQ19VGcGcHBD1njxuZhYg9l4ETCVk4fD4DjtaGNa1xHW2hr8uPBCIiTcdPaqJWRCAQwNXOBK52JhgZ1B7xCVmIPZeB3/66ig0Hr6OrR01rnIvEmK1xREQaisUbUSulr6uFPs85oLefPVKyihB7LgPxCdk4eiETDlY1rXHdPW2gr6ut7lCJiOg+LN6IWjmBQIC2EmO0lRhjRFA7xF/JRuy5DKzddxV/HLyOLh2sEehrD1d7tsYREWkCFm9EJKeno4Xevvbo7WuPm1lFiD2fgROXs3DsUhbsLQ3Qy9cOPTrZwoCtcUREasPijYgeysnWCGNt3fFSH1ecvJKD2HMZWL//GjYdSoK/uzUCfe3Q3sGErXFERCrG4o2IHktXrIVePnbo5WOH1Ox/W+PiLmdBYqGPQB879PCSwFCPrXFERKrA4o2IlOZoY4SX+7vjpd7tcDIxG4fPZ+D3A9exKfbf1ji3NqZsjSMiakIs3oiowXTEIvT0tkNPbzvcyilG7PkMHL+UhRMJ2bAxr2mNe8HLFkb6YnWHSkTU4ghkMplM3UGoSl5eMaqrm+5yrayMkJtb1GTHp6fDvKhGeaUUpxJzEHs+A9dv3YWWSIDn3KwQ6GMHdyczCAUCxF3OwpbYJOQXlsPcWAfDAl3R3dNW3aFTLb5XNA9zopmaOi9CoQAWFoaPfJwtb0TUKHS0RXjBS4IXvCRIz61pjYu7lIWTV3JgbaYHZ1sjnL12G5VV1QCAvMJy/Lo7EQBYwBERNYBQ3QEQUctjb2WIyL5uWDjlBUwM6whTQx2cvJIjL9zqVFRVY0tskpqiJCJqnlRWvCUnJ2PEiBEICQnBiBEjkJKSUm+fzZs3IywsDOHh4QgLC8Pq1avlj0mlUsyZMwd9+/ZFv3798Mcff6gqdCJ6StpaInT3tMXM0c89cp+8wnIkpOSjvFKqwsiIiJovlXWbzp49G5GRkQgPD8e2bdswa9YsheIMAEJCQjBs2DAIBAIUFxcjLCwMXbt2RYcOHbBjxw6kpqbir7/+QkFBAYYMGYLu3bvDwcFBVZdARM/AwlgHeYXlD31s/u/nIBIK0NbOGB0cTeHexgzt7E2gIxapOEoiIs2nkpa3vLw8JCQkIDQ0FAAQGhqKhIQE5OfnK+xnaGgoX2KgrKwMlZWV8p937dqFiIgICIVCmJubo2/fvtizZ48qwieiRjAs0BViLcWPHLGWEK8McMe7ET7o37UNqqtl2BWXigUbzmHK/w7jy99OY3NsEi4l56GsokpNkRMRaRaVtLxlZmbCxsYGIlHNX9EikQjW1tbIzMyEubm5wr4xMTFYuHAhUlNT8d5778Hd3V1+DDs7O/l+EokEWVlZDYrjcTM3GouVlVGTn4MajnlRv8G9jWBspIvVu6/g9p17sDTTw9gXPdC7cxsAQPDzzgCA0rJKJKbcwcWk27iUdBt74lOxM+4mREIB2rUxhZerJTq5WsDD2Rz6vE1Xo+N7RfMwJ5pJnXnRuNmmwcHBCA4ORkZGBiZPnoxevXrBxcWlUY7NpUJaJ+ZFc3g6miLq9e4KOXlYbtpY6KGNRRsM7NoGZRVVuJ5+F/+kFuCf1AL8eeg6Nh24BqFAACdbo5puVkcztHcwgZ6Oxn2kNSt8r2ge5kQztYqlQiQSCbKzsyGVSiESiSCVSpGTkwOJRPLI59jZ2cHLywuHDh2Ci4sLJBIJMjIy4O3tDaB+SxwRtUy6Yi10amuBTm0tAADlFVJcz7iLf1Lv4J/UAvz1dxp2x6dCIACcbY3g7mgG9zamaO9gCn1dFnNE1PKo5JPNwsICHh4eiI6ORnh4OKKjo+Hh4VGvyzQpKQmurq4AgPz8fMTHx6N///4AgAEDBuCPP/5A//79UVBQgP3792Pt2rWqCJ+INIiOWARPZ3N4Otd8fpRXSpEkb5m7g/2n0rCntphzsjGCe23LnJuDCbtZiahFUNmfpZ9//jlmzpyJJUuWwNjYGFFRUQCAiRMnYurUqfDy8sKGDRtw7NgxaGlpQSaTYcyYMQgICAAAhIeH4/z58/JibvLkyWjTpo2qwiciDaWjLUJHZ3N0rC3mKiqlSMoolLfMxZy+hb0n0yBAzb1Za4o5U7i1MYUBizkiaoZ4e6xGxLEJmol50TyqzElFpRQ3MgrxT1pNy9z19EJUSashANDG2hDujmbo4GiK9m1MYajXuos5vlc0D3OimVrFmDciInURa4vQwckMHZzMALRFZVVtMZdagH/SCnDoXDr2nappmXOwNoR7m5puVndHFnNEpJlYvBFRq6KtJaotzswAAJVV1UjOrOlmTUwtwOHzGdh/+hYAwMHKQN4y59bGFEb6YnWGTkQEgMUbEbVy2lpCuLWpKc7CXgCqpHXFXE0365ELGYipLebsrQzg3sYUHRzN4NbGFMYGLOaISPVYvBER3UdLJER7h5qlRkJ7OKNKWo2UrCJ5y9yxi1k4cCYdAGBnaQB3x3+LORMWc0SkAizeiIgeQ0skRDt7E7SzN8Gg7jUtczezivBPWgESU+/g+KUsHKwt5iQW+vJuVvc2pjAx1FFz9ETUErF4IyJqAC2REK72JnC1N8HA550gra7GzaxiecvcictZOHS2ppizNdevGS/naAr3NmYwM2IxR0TPjsUbEdEzEAmFcLEzhoudMV6sLeZSs4uRWLvOXPyVbBw6lwEAsDHXrx0zVzOjlcUcET0NFm9ERI1IJBSircQYbSXGeLGbE6qrZUjNKULizZoJEH8n5uDw+ZpiztpMr7aLtWZpEnNjXTVHT0TNAYs3IqImJBQK4GxrDGdbYwzo5ojqahnScv5tmTuVmIvD5zMBANamenBzNJUXdBYmLOaIqD4Wb0REKiQUCuBkawQnWyOEdK0p5m7lFiOxdmmSs1dzcfRCTTFnaaKLDrULBrs7msLSRE/N0RORJmDxRkSkRkKhAI42RnC0MUL/Lm1QLZPhVk6x/A4QZ6/l4ujFmmLOwlhXPl6uppjThUAgUPMVEJGqsXgjItIgQsG/xVy/2mIuI7dE3s16PikPxy5lAQAsjHVqCrk2pnB3MoMVizmiVoHFGxGRBhMKBHCwNoSDtSH6+tcWc7dL5HeAuJCUh+O1xZyZkY5Cy5y1qR6LOaIWiMUbEVEzIhQI4GBlCAcrQwR3doBMJkNGXql8nbnLyfmIu5wNoKaYc69dMLiDoxmszVjMEbUELN6IiJoxgUAAe0sD2FsaIOi5mmIus7aY+yetAAkpd3CitpgzMRTXTIBoUzMBwtZcHwKBAHGXs7AlNgn5heUwN9bBsEBXdPe0VfOVEdGjsHgjImpBBAIB7CwNYGdpgD61xVxWfin+Sa25nVdi6h3EJ9QWcwZiWJjo4GZWMaTVMgBAXmE5ft2dCAAs4Ig0FIs3IqIWTCAQQGJhAImFAXr72UMmkyH7zj0kpt7B1do7QMhkis+pqKrG+v3X4N7GFGZGOuxqJdIwLN6IiFoRgUAAW3N92Jrro7evPU7UtsI9qPheJWYsOQ4TAzGcbY3QVmIMZ4kxnCVGMNYXqzhqIrofizciolbMwlgHeYXl9bYbG4gR2t0JyZlFSMkqxIWkPMjkz9FFW4kRnCXGaGtrBCdbY+jr8uuESFX4biMiasWGBbri192JqKiqlm8TawkxIqidwpi3e+VVuJlVhJSsIiRnFiI5sxCn/smVP25jro+2EiO0ta1pnXO0MYKOtkil10LUWrB4IyJqxeoKtCfNNtXT0UIHJzN0cDKTbyu+V4mU2kIuJasIiTf/ndkqrJ044Syp6XJtKzGCg5UhtERC1V0cUQvF4o2IqJXr7mmL7p62sLIyQm5ukdLPM9TTRicXC3RysZBvu1NUjpSswpru1sxChXu1aokEaGNtWDN2rnYcnZ2FAYRCToggagiVFW/JycmYOXMmCgoKYGpqiqioKDg7Oyvs88MPP2DXrl0QCoXQ1tbGtGnT0LNnT/nzZ82ahcLCQlRUVGDgwIF4++23VRU+EREpwcxIB2ZGVvBrbwUAkMlkuH23rKZ1rnb8XNylLBw8kw4AEGsL4WRTNyGiptuViwkTPZ7KirfZs2cjMjIS4eHh2LZtG2bNmoXVq1cr7OPt7Y1x48ZBT08PiYmJGDNmDI4ePQpdXV188803CAkJwZgxY1BSUoLQ0FAEBgbC29tbVZdAREQNJBAIYGWqBytTPXT1sAEAVMtkyMor/beFLqsQB8+mo/LvmnF3+jpacJYYwdm2pru1rcSYS5YQ3UclxVteXh4SEhKwcuVKAEBoaCjmzZuH/Px8mJuby/era2UDAHd3d8hkMhQUFMDW1hYCgQBFRTXN+WVlZRAIBArPJSKi5kF430LCPTpJAABV0mpk3C5RmBCx92SqfPFg4/uXLKn9r7EBlyyh1kklxVtmZiZsbGwgEtXMPBKJRLC2tkZmZuYjC7CtW7fC0dERtrY1g2Y//vhjvPHGG1i3bh0KCwvxwQcfwMHBQRXhExFRE9MSCeFoUzNLtZePHQCgskqK1JxipGQWySdFXFRYskQHzrWzW+uKOn1dbfVdBJGKaOSEhZMnT+K7777DihUr5Ns2bNiA8PBwTJgwATk5OXj55ZfRqVMn+Pj4KH1cCwvDpghXgZWVUZOfgxqOedE8zIlm0rS82ElM8fx9H/OlZZVISr+L62kFuJZWgGtpd3D66r9LlthZGqB9GzO0a2OK9m1M4WpvAl0djfyqU5qm5YRqqDMvKnlFSyQSZGdnQyqVQiQSQSqVIicnBxKJpN6+Z8+exfvvv48lS5bAxcVFvv23337D/v37AQDW1tZ4/vnn8ffffzeoeMvLK0Z1tezJOz6lhs7UItVgXjQPc6KZmktebI11YOtpgwDPmjF0xfcqFWa4Xriei9iztwAAAkFNQde2dvycs8QYDlaG0NZqHkuWNJectDZNnRehUPDYBieVFG8WFhbw8PBAdHQ0wsPDER0dDQ8Pj3pdphcuXMC0adOwaNEieHp6Kjzm4OCAI0eOYMiQISguLsbp06cRFBSkivCJiEiDGeppo1NbC3Rq+++SJQXF5fLu1uSsQpy7fhtHL9YsWSIS/rtkSdva8XMSS32IhM2joCMSyGQP3pK4aSQlJWHmzJkoLCyEsbExoqKi4OLigokTJ2Lq1Knw8vLC8OHDkZ6eDhsbG/nzvv76a7i7u+PSpUv4z3/+g9LSUlRVVWHgwIGYMmVKg2Jgy1vrxLxoHuZEM7XkvMhkMuTdLUNyVpF8YeGb2UW4Vy4FULNkiaPNv3eIaCupWbJEqOYZri05J82ZulveVFa8aQIWb60T86J5mBPN1NryUi2TITu/VGFCRGp2kfxWYXo6WnC2NZKvP+csMYKFsa5KlyxpbTlpLtRdvDXvUZxERERPSSgQQGJhAImFAbp3qlnZQFpdjfTcmiVLalroivDXyTT5kiVG+tryma3OEmO0lRjDhEuWkIqxeCMiIqolEj58yZK0nJLa1rmaO0Xcv2SJmZGO/P6tdUuXGHDJEmpCLN6IiIgeQ1tLBBc7Y7jYGcu3lVVUITW7WL6gcEpWEc7ct2SJtZmewoLCTjZG0BGL1BE+tUAs3oiIiBpIV6wFtzamcGtjKt9WUlYpv39rcmYRrqYVID4hG0DtkiUWBvfd9ssYbaybz5IlpFlYvBERETUCA11teLY1h2fbf5fBultcLp/hmpJVhAtJeTh2MQtAzZIlDlaG8vXnnG2NYG9lAJFQiLjLWdgSm4T8wnKYG+tgWKArunvaquvSSMOweCMiImoiJoY68G2nA992lgBqlywpLKuZ4Vo7fi7+SjYOncsAAIi1hDA11MHtu2Worl0MIq+wHL/uTgQAFnAEgMUbERGRyggEAlia6MHSRA/+HawB1CxZknPnXs3YucwiHDx7S1641amoqsavexKRX1gGe0tD2FsZwMJEV+3r0JF6sHgjIiJSI6FAAFtzfdia66O7py32nUp76H4VldXYHHtD/rOOtgh2lgawtzKAg6UB7K0MYWdpAFNDsUrXoiPVY/FGRESkQSyMdZBXWP7Q7XPGdUPG7RLcul2MjNwSpN8uwYXrt3H0QqZ8PwNdLdjXFnP2Vgby/zfU4/IlLQWLNyIiIg0yLNAVv+5OlN/pAagZCzcs0BX6ulpo52CCdg4mCs8pLKlA+u0SpOcW1xZ3JTiRkI175VXyfUwMxLXFXG1RZ2UAOwsD6OmwFGhumDEiIiINUjcpoSGzTY0NxDA2EMPDyUy+TSaT4U5ReW1RV1PYpd8uQey5dIXC0NJE977u15rCTmKhD20trkunqVi8ERERaZjunrbo7mn7TPfQFAgEMDfWhbmxLrxcLOTbq6tluH33HtJza1ro6lrrLifny28DJhAANmb6Ct2u9pYGsDHXg0jItenUjcUbERFRKyIUCmBtpg9rM334uVnJt1dJq5F9515NC13teLpbOcU4czUXdZNftUQC2JobwKGu27W2sLPkzFeVYvFGRERE0BIJa1rZLA0Aj3+3V1RKkZlXivTb/xZ1124V4ETt3SMAQKxd99x/x9PZWxpy5msTYfFGREREjyTWFsHJ1ghOtkYK2++VVyHjdm0LXW1r3YUbeTh68d+Zr/o6WrXFXE23q0Nta52RvljVl9GisHgjIiKiBtPT0YKrvQlc7RVnvhaVVtTMeK1tpUvPLcbJhGyU3jfz1dhAXDuWzgAOtYWdnSVnviqLvyUiIiJqNEb6Yrg7iuHuqDjztaC44t+u19wSpN8uxuHzGaio/Hfmq4WxjryVrq7rVWKhD7E2Z77ej8UbERERNSmBQAAzIx2YGemgU9v7Zr7KZLh9t6x2weGawu5WbgkSUvJRJf135qu1mT4cLA3kS5rYWxnCxkwPWqLWOfOVxRsRERGphVAggLWpHqxN9eDb3lK+vUpajZw79+TdrnVr1Z259u/MV5FQAImFvvy2YA61hZ2lqV6Ln/nK4o2IiIg0ipZICLvalrYuHazl2yurame+5v57i7Ck9LuIf2Dmq52FgeLdJCwNYGak02JmvrJ4IyIiomZBW0sERxsjONo8ZOZrXol8PF3G7WJcSs7HsYtZ8n30ame+Oty36LCdlQGMm+HMVxZvRERE1Kzp6WjB1c4ErnaKM1+L71UqdLum5xbj78QcHDqXId/HWF9bcZJE7f8/bOZr3OWsBt22rKmweCMiIqIWyVBPG+6OZvVmvt4tqZAXc7dqC7sjFzJRXimV72durKPQ7XqnsAzRcTfl94XNKyzHr7sTAUDlBZzKirfk5GTMnDkTBQUFMDU1RVRUFJydnRX2+eGHH7Br1y4IhUJoa2tj2rRp6Nmzp/zx3377DWvXroW2tjaEQiG2bdumqvCJiIioBRAIBDA11IGpoQ4825rLt1fLZMi/Wya/32tda92Vm3dQJa1+6LEqqqqxJTap5RZvs2fPRmRkJMLDw7Ft2zbMmjULq1evVtjH29sb48aNg56eHhITEzFmzBgcPXoUurq6+Ouvv7Bnzx5s2rQJhoaGuH37tqpCJyIiohZOKBDA0lQPlqZ68G3378xXaXXNzNdPfo5/6PPyCstVFaKcShZIycvLQ0JCAkJDQwEAoaGhSEhIQH5+vsJ+PXv2hJ6eHgDA3d29ZlG/ggIAwIoVKzBlyhQYGhoCACwtLUFERETUlERCISQWBrAw1nno44/a3pRU0vKWmZkJGxsbiEQ1KySLRCJYW1sjMzMT5ubmD33O1q1b4ejoCFvbmqbIpKQknD9/Ht999x0qKiowcuRIvPTSSw2Kw8LC8NkuRAlWVkZP3olUjnnRPMyJZmJeNA9zohleDfXE93+cVxgXp6MtwquhnirPkUZOWDh58iS+++47rFixQr5NKpUiMzMT69atw507dzBq1Ci0bdsWXbp0Ufq4eXnFqK6WNUXIAGreYLm5RU12fHo6zIvmYU40E/OieZgTzeHpaIqxA9zrzTb1dDRt9BwJhYLHNjippHiTSCTIzs6GVCqFSCSCVCpFTk4OJBJJvX3Pnj2L999/H0uWLIGLi4t8u52dHUJDQyEUCmFhYYEePXrgwoULDSreiIiIiJ5Wd09bdPe0VXtRrZIxbxYWFvDw8EB0dDQAIDo6Gh4eHvW6TC9cuIBp06Zh0aJF8PT0VHgsNDQUR44cAQCUlpbi9OnT6NChgyrCJyIiItIYAplM1nT9iPdJSkrCzJkzUVhYCGNjY0RFRcHFxQUTJ07E1KlT4eXlheHDhyM9PR02Njby53399ddwd3dHWVkZPvvsMyQkJAAAwsPDMWnSpAbFwG7T1ol50TzMiWZiXjQPc6KZmjovT+o2VVnxpglYvLVOzIvmYU40E/OieZgTzaTu4k0l3aZERERE1DhYvBERERE1IyzeiIiIiJoRjVznrakIhYIWcQ5qOOZF8zAnmol50TzMiWZqyrw86ditasICERERUXPHblMiIiKiZoTFGxEREVEzwuKNiIiIqBlh8UZERETUjLB4IyIiImpGWLwRERERNSMs3oiIiIiaERZvRERERM0IizciIiKiZqRV3R6rqURFRWHv3r1IT0/Hjh074Obmpu6QWr07d+7ggw8+QGpqKsRiMZycnDB37lyYm5urO7RW76233sKtW7cgFAqhr6+Pzz77DB4eHuoOiwB8//33WLx4MT/HNERQUBDEYjF0dHQAADNmzEDPnj3VHFXrVl5eji+//BJxcXHQ0dGBr68v5s2bp/I4WLw1guDgYIwdOxajR49WdyhUSyAQYMKECejWrRuAmgJ7/vz5+PLLL9UcGUVFRcHIyAgAsH//fnz88cf4888/1RwVXb58GefOnYO9vb26Q6H7LFq0iIW0Bvnmm2+go6ODvXv3QiAQ4Pbt22qJg92mjcDf3x8SiUTdYdB9TE1N5YUbAPj6+iIjI0ONEVGdusINAIqLiyEQ8Kbb6lZRUYG5c+fi888/V3coRBqrpKQEW7duxTvvvCP/3LK0tFRLLGx5oxavuroa69evR1BQkLpDoVqffPIJjh07BplMhl9++UXd4bR63333HQYPHgwHBwd1h0IPmDFjBmQyGTp37ozp06fD2NhY3SG1WmlpaTA1NcX333+P+Ph4GBgY4J133oG/v7/KY2HLG7V48+bNg76+PsaMGaPuUKjWF198gUOHDmHatGn4+uuv1R1Oq3b27FlcunQJkZGR6g6FHrB27Vps374dmzdvhkwmw9y5c9UdUqsmlUqRlpaGjh07YsuWLZgxYwbefvttFBcXqzwWFm/UokVFReHmzZv43//+B6GQL3dNM2TIEMTHx+POnTvqDqXV+vvvv5GUlITg4GAEBQUhKysL48ePx9GjR9UdWqtXNxxHLBYjMjISZ86cUXNErZtEIoGWlhZCQ0MBAD4+PjAzM0NycrLKY+G3GbVYCxcuxKVLl/DDDz9ALBarOxxCzZiRzMxM+c8HDhyAiYkJTE1N1RdUKzdp0iQcPXoUBw4cwIEDB2Bra4vly5cjICBA3aG1aqWlpSgqKgIAyGQy7Nq1i7Oy1czc3BzdunXDsWPHAADJycnIy8uDk5OTymMRyGQymcrP2sL85z//wV9//YXbt2/DzMwMpqam2Llzp7rDatWuXbuG0NBQODs7Q1dXFwDg4OCAH374Qc2RtW63b9/GW2+9hXv37kEoFMLExAQffvghPD091R0a1QoKCsLSpUs5w1HN0tLS8Pbbb0MqlaK6uhqurq749NNPYW1tre7QWrW0tDR8/PHHKCgogJaWFt59910EBgaqPA4Wb0RERETNCLtNiYiIiJoRFm9EREREzQiLNyIiIqJmhMUbERERUTPC4o2IiIioGWHxRkRERNSMsHgjohZp5syZ+Pbbb9VybplMho8++ghdunTB//3f/6klhgctXrwYM2bMUHcYRNQIeGN6IlKJoKAg3Lt3DzExMdDX1wcA/PHHH9i+fTt+++03NUfXuE6fPo1jx44hNjZWfq3327JlCz755BP5AtJ19uzZAxsbG1WFSUTNFIs3IlKZ6upqrF69Gm+88Ya6Q2kQqVQKkUik9P7p6emwt7d/aOFWx9fXF+vXr2+M8IiolWG3KRGpzPjx47FixQoUFhbWe+zWrVtwd3dHVVWVfNvLL7+MP/74A0BNa9XIkSPx5Zdfwt/fH8HBwThz5gy2bNmCwMBAdO/eHX/++afCMe/cuYPXXnsNfn5+GDNmDNLT0+WPJSUl4bXXXkPXrl0REhKCXbt2yR+bOXMmZs+ejYkTJ8LX1xfx8fH14s3OzsYbb7yBrl27ol+/fti4cSOAmtbETz/9FOfOnYOfnx8WLVrU4N9TUFAQfvrpJwwcOBBdunTBRx99hPLycvnjGzduRL9+/dC1a1e88cYbyM7Olj927do1+XX16NEDS5culT9WWVmJDz74AH5+fhg0aBAuXrwof2zZsmXo2bMn/Pz8EBISgri4uAbHTUSqweKNiFSmU6dO6Nq1K5YvX/5Uz79w4QLc3d0RHx+P0NBQTJ8+HRcvXsS+ffvwzTffYO7cuSgpKZHvv2PHDrz11luIj49Hhw4d5GO+SktLMW7cOISGhuL48eP49ttvMWfOHFy/fl3+3OjoaLzxxhs4c+YMOnfuXC+W6dOnw9bWFkeOHMGiRYuwcOFCxMXFISIiAnPmzIGvry/Onj2LqVOnPtW17tixA8uXL8e+ffuQnJyMJUuWAADi4uKwYMEC/O9//8PRo0dhb2+P6dOnAwCKi4vx2muvoWfPnjhy5Aj++usvdO/eXX7MAwcOYNCgQTh16hSCgoIwb948AMCNGzewdu1abNq0CWfPnsXy5cthb2//VHETUdNj8UZEKjV16lSsWbMG+fn5DX6ug4MDhg8fDpFIhIEDByIzMxOTJ0+GWCxGQEAAxGIxUlNT5fv37t0bXbp0gVgsxrRp03Du3DlkZmbi0KFDsLe3x/Dhw6GlpYWOHTsiJCQEe/bskT83ODgYnTt3hlAohI6OjkIcmZmZOHPmDGbMmAEdHR14eHggIiIC27ZtU/pazp8/D39/f/m/vn37Kjw+evRoSCQSmJqa4s0338TOnTsB1BR1w4cPh6enJ8RiMaZPn45z587h1q1bOHToECwtLTFu3Djo6OjA0NAQPj4+8mN27twZgYGBEIlECA8PR2JiIgBAJBKhoqICSUlJqKyshIODAxwdHZVPDBGpFMe8EZFKubm5oXfv3li2bBlcXV0b9FwLCwv5/9cN9re0tJRv09HRUWh5s7W1lf+/gYEBTExMkJOTg/T0dFy4cAH+/v7yx6VSKQYPHiz/WSKRPDKOnJwcmJiYwNDQUL7Nzs4Oly5dUvpafHx8Hjvm7f7z29nZIScnR35uT09PhesyNTVFdnY2MjMzH1t03f+70tXVRXl5OaqqquDk5ISPP/4YixcvxvXr1xEQEICZM2dy8gSRhmLLGxGp3NSpU7Fx40aFsVp1g/vLysrk23Jzc5/pPFlZWfL/Lykpwd27d2FtbQ2JRIIuXbrg1KlT8n9nz57FnDlzlDqutbU17t69i+LiYvm2zMzMRi12MjMz5f+fkZEBa2tr+bnvH7tXWlqKgoIC2NjYQCKRIC0t7anOFxYWhvXr1+PgwYMQCASYP3/+s10AETUZFm9EpHJOTk4YOHCgwhIh5ubmsLGxwbZt2yCVSrFp06anLkTqxMbG4tSpU6ioqMB3330HHx8fSCQS9O7dGykpKdi6dSsqKytRWVmJCxcuICkpSanjSiQS+Pn5YeHChSgvL0diYiI2bdqk0HL3rNatW4esrCwUFBRg6dKlGDhwIAAgNDQUW7ZswZUrV1BRUYGFCxfC29sbDg4O6N27N3Jzc7Fq1SpUVFSguLgY58+ff+K5bty4gbi4OFRUVEAsFkNHRwdCIb8eiDQV351EpBaTJ09GaWmpwrZ58+Zh+fLl6NatG65fvw4/P79nOkdoaCh++OEHdOvWDZcvX8Y333wDADA0NMTy5cuxa9cu9OzZEwEBAZg/fz4qKiqUPvbChQuRnp6Onj17YsqUKXj77bfRo0cPpZ9fNxv1/n8XLlxQiH3cuHHo27cvHB0d8eabbwIAevTogXfeeQdvv/02AgICkJaWJl+M2NDQECtWrMDBgwfxwgsvICQk5KEzZR9UUVGBBQsWoFu3bggICEB+fr58EgQRaR6BTCaTqTsIIiL6V1BQEP7zn/80qBgkotaDLW9EREREzQiLNyIiIqJmhN2mRERERM0IW96IiIiImhEWb0RERETNCIs3IiIiomaExRsRERFRM8LijYiIiKgZ+X8QjzJUpuufTgAAAABJRU5ErkJggg==\n","text/plain":["<Figure size 720x360 with 1 Axes>"]},"metadata":{},"output_type":"display_data"}],"source":["fig, ax = plt.subplots(figsize=(10, 5));\n","ax.plot(num_epochs, np.array(training_loss_list) ,'bo-',label=\"Train Loss\")\n","ax.set_xlabel(\"Number of Epochs\")\n","ax.set_ylabel(\"Training Loss\")\n","ax.set_title(\"Training Loss vs Number of Epochs for Bert-Base\",fontsize=18)"]},{"cell_type":"code","execution_count":29,"id":"545a4ffa","metadata":{"execution":{"iopub.execute_input":"2023-10-25T20:00:05.323807Z","iopub.status.busy":"2023-10-25T20:00:05.323488Z","iopub.status.idle":"2023-10-25T20:00:05.585275Z","shell.execute_reply":"2023-10-25T20:00:05.584258Z"},"papermill":{"duration":0.291636,"end_time":"2023-10-25T20:00:05.58758","exception":false,"start_time":"2023-10-25T20:00:05.295944","status":"completed"},"tags":[]},"outputs":[{"data":{"text/plain":["(0.0, 100.0)"]},"execution_count":29,"metadata":{},"output_type":"execute_result"},{"data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAmsAAAHDCAYAAAB76136AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAABQa0lEQVR4nO3dfXzN9f/H8ec5m22GmWGzRlS+m2vDoiIxamSuhUQKIURJkhRR+hEqUlKipCTXV7mmIolcJbpwmYthLnchm+18fn/MTjt2Nmc426f2uN9uu+2cz+dzPp/X2ftzzp7n/Xl/PsdiGIYhAAAAmJI1rwsAAABA1ghrAAAAJkZYAwAAMDHCGgAAgIkR1gAAAEyMsAYAAGBihLV8okuXLoqMjHSYNmTIEIWFhbn0+GPHjiksLEyTJk1yR3kKCwvTkCFD3LJuwIycvSb/TWbNmqUmTZqoSpUqCgsL07Fjx/K6pFtiy5YtCgsL0/z583NleytWrFCLFi1UrVo1hYWFacuWLbmyXfy7eOZ1AWa3ZcsWPf7441nO/+qrrxQeHi5JWr58ub7//nv9+uuvOnDggFJSUrR27VqVLl36utvZsGGDevXqpccff1wvv/xylstNnDhRkydP1rhx49S8efMcP5+8EhcXp08//VS1a9dWnTp18rqc63rrrbf08ccfq2zZslq1alVel4McSP8AEh0drfHjx2ea36VLF+3Zs0c7duzI7dL+M3788UeNHDlSjRo10lNPPSVPT08FBARkuXyXLl30008/ZTl/wIAB6tOnjztKNbVDhw7p+eefV3h4uF555RV5eXnprrvucvt2jx07pkaNGjlMK1CggEqWLKmqVauqe/fuql69ulu2vW/fPq1Zs0atW7d26X+jJM2fP18vvfSSw7SCBQsqKChI999/v7p3767g4GB3lGsahDUXRUdHq379+pmm33777fbbX375pXbt2qUKFSqoTJkyOnTokMvrv//++xUYGKglS5Zo8ODBKlCgQKZlDMPQwoUL5efnp4ceeujGnkgGo0aN0muvvXbT63FFXFyc3nvvPfXr189pWNu9e7esVnN09KakpGjRokW6/fbbdeTIEf3000+qXbt2XpeFHFq2bJl69OihihUr5nUp/zk//PCDJGn06NHy9/d36TFeXl56/fXXnc7Lr230008/KSUlRUOHDlXlypVzfft169ZVy5YtJUnJyck6fPiw5syZo7Vr1+rLL79UtWrVbvk29+3bp/fee0+1a9d2Oayl69Kli6pWrSpJSkxM1C+//KJZs2Zp9erVWrZsmQoXLnzL6zULwpqLKlWqZN+pszJmzBgFBgbK09NTI0eOzFFY8/DwUJs2bTRlyhStX7/eaRj78ccfdfz4cXXq1Ene3t45fg7XchYI88qteD63yoYNGxQbG6sZM2bo+eef17x58/4VYS0hIeE//WaVE6GhoTp8+LDGjRunadOm5XU5pnAr94/Y2FhJcjmoSZKnp+d130Pzm/S/Y9GiRW/peq9cuSKbzXbd99Vy5cplapOaNWuqT58+Wrx48S0Na7di/4uIiFCTJk0cpvn5+WnGjBnavHmzHnzwwZtav5mZoyvjP+K2226Tp+eN59+2bdvKYrFo3rx5TufPnTvXvpyUdti1d+/eatCggapUqaI6deqoT58++u2331zaXlZj1rZt26aOHTuqWrVquu+++zRy5EhdunQp03I2m00ffPCBHnvsMdWtW1dVqlRRgwYNNHz4cJ0/f96+3JYtW+xd7u+9957CwsIUFhbmMF4nqzFrX3/9tVq3bq1q1aqpVq1a6tatm7Zt25ZpufTH79ixQ507d1Z4eLjq1Kmjl19+WYmJiS79PdLNnTtXZcqU0T333KPmzZtr5cqVSkhIcLrs3r171b9/f913332qUqWKHnjgAQ0cOFB//fWXw3I//vijevbsqTp16qhq1apq1KiRhg4dqnPnztn/RlmNk3HWTunjnY4ePar+/furdu3aqlWrliTX2yWjlStXqkuXLoqIiFD16tUVFRWl119/XcnJydq7d6/CwsL09ttvO31sz549VbNmTaf7SLpHHnlE9913n1JSUjLN+/777xUWFqYZM2bY658xY4aaN2+uGjVqqGbNmoqKitLQoUN15cqVLLeR0W233aZOnTpp48aN2rx583WXz2r8mLOxmhnbatasWYqKilLVqlXVvHlzrV+/XpL0+++/q3v37qpZs6bq1Kmj119/Pcvajx49qqefflq1atVSzZo11bdvXx09ejTTcoZh6IsvvlCbNm1UvXp11ahRQ126dNGPP/6YZc3Lly9XmzZtVK1atSx7tTJas2aNOnbsqPDwcNWoUUMdO3bUmjVrMq07fT9Nfy136dLluut2Vfr+fu7cOQ0ePFh16tRReHi4unbtql9//TXT8ikpKZo6daoefvhhVa1aVXXq1FHfvn31+++/O11/dvv6tebNm6dmzZqpSpUqatiwoT766KNMy2zfvl09evRQ3bp1VbVqVd1///166qmntHPnzmyfZ8b9qlGjRpneE48dO6YXXnjB/t7SuHFjTZgwQX///bfDeiZNmqSwsDD9+eefevPNN1W/fn1Vq1btutvPSmBgoCTnH+Z/+OEHdevWTREREfZ9/ssvv8y0XGRkpLp06aK9e/eqe/fuqlWrllq0aKFJkybZD2c+/vjj9v3nZsYrO6s3p++BCxcuVLt27RQREaHw8HA1atRIzz//vP39Od3hw4f1wgsvqF69eqpSpYoiIyM1ZsyYbN/7bhV61lz0999/Z2o4Ly+vW9qTcfvtt+vuu+/W999/r9OnT9t3QkmKj4/XmjVrVKFCBVWpUkWS9Pnnn8vf31/t27dXyZIl9ddff2nOnDl69NFHtWDBApUrVy7HNezatUtPPvmkChUqpKeeekpFihTR8uXL9eKLL2Za9sqVK5o2bZoeeughNWrUSAULFtQvv/yiefPmafv27Zo3b559DMZLL72kN998Uw8++KD900+hQoWyrSV93Fi1atU0cOBAJSQkaM6cOeratavef/99PfDAAw7L79u3T71791abNm0UHR2tn376SXPnzpXVatWoUaNcev6xsbH6/vvv9fTTT8tisah169aaMWOGli1bpg4dOjgsu379ej3zzDPy9fVVu3btVLZsWcXGxmrjxo36448/7IfIZ8+erREjRigoKEgdO3ZUSEiITpw4ofXr1+vUqVPZjvXJTmJiojp37qyaNWvq2Wefte+frrZLurfffltTpkxR+fLl9cQTT9j3pVWrVql///6qVKmSKleurAULFqh///7y8PCwP/bUqVPauHGj2rZtK19f3yxrbdWqlUaOHKnvv/9eDRs2dJi3cOFCeXp62sdgfvDBB5o4caIaNmyojh07ysPDQ8eOHdO6deuUnJzsco9w7969NW/ePL311luaN2+eLBaLy39bV8yaNUtxcXF65JFH5OXlpZkzZ6pfv3569913NWzYMEVHR6tx48batGmTZs6cqYCAgEzjsi5duqQuXbrY9/EjR47oiy++0K5du7RgwQKVLFnSvuwLL7ygZcuWKSoqSm3atFFycrKWLFmibt26adKkSZnGIK1Zs0YzZ87Uo48+qo4dO173vWrWrFkaOXKk7rzzTnudCxYsUN++fTVy5Eh16NBBAQEBGjt2rObMmaNt27Zp7NixkqQSJUq49De79j00nZ+fX6YPuj169FDRokXVr18/nTlzRp9//rk6d+6sr776SqGhofblBg0apG+++UZ169bVo48+qjNnzmjWrFnq2LGjZs2apUqVKtmXvd6+nvF1MXv2bJ05c0bt2rWTn5+fFi9erHHjxqlUqVL2ffXgwYPq1q2bSpQooccff1zFixfX2bNn9fPPP+u3336zj2d2ZuzYsVq9erVWr16tl156ScWKFbO/Jx4/flyPPPKI4uPj1alTJ5UtW1Y//fSTPvzwQ23fvl0zZszI9PcaNGiQfHx81K1bN0ly2HeykpSU5PC+cfjwYU2YMEEFCxbM1OP21Vdfafjw4QoPD1fv3r1VsGBB/fDDDxoxYoT++uuvTP8jTpw4oa5du6pJkyZ66KGHdOnSJd17772KjY3VV199pd69e+vOO++U5DicKDuJiYn2ei9duqQ9e/Zo2rRpKlu2rO655x77cjl5D1y4cKFefPFFRUREqH///vLx8VFMTIy+/fZbnT171v7+vGfPHnXt2lV+fn7q0KGDgoKC9Ntvv2nmzJnasWOHZs6c6d6jVQay9eOPPxqhoaFOf5599tksH/faa68ZoaGhxtGjR3O0vQULFhihoaHG1KlTHaZ/+eWXRmhoqPHpp5/apyUmJmZ6/P79+43KlSsbw4cPd5jeuXNno2HDhg7TXnzxRSM0NNRhWocOHYzKlSsbBw8etE9LSkoy2rZta4SGhhoTJ060T7fZbMbff/+dqYY5c+YYoaGhxrJly+zTjh49munxGYWGhhovvvii/f6BAweMsLAwo2PHjkZSUpJ9+smTJ41atWoZDRs2NFJSUhweHxYWZuzcudNhvU899ZRRqVIlIyEhwel2r/Xhhx8aYWFhxl9//WWf1rJlS6Ndu3YOy126dMmoU6eOcc899xgnT57MtJ7U1FTDMAwjJibGqFy5stG0aVPj4sWLWS6Xvp/Nmzcv0zLO2qlz585GaGioMWHChEzL56Rddu3aZYSGhhpdunQxLl++nGk9NpvNMAzDmD17thEaGmps2LDBYZn333/fCA0NNXbt2pVpexmdP3/eqFy5stG/f3+H6fHx8Ub16tWNXr162ae1atXKaNq0abbry05oaKjRs2dPwzAM44MPPjBCQ0ONpUuX2ud37tzZCA8Pd3iMs9eHYTjfb9Pbql69ekZcXJx9+r59++z74cqVKx3W07p1a6Nu3bqZthkaGmq8/vrrDtNXrVplhIaGGq+88kqmabNnz3ZY9sqVK0br1q2Nhg0b2tsqveZKlSoZ+/fvz/oPlcGFCxeM8PBwo3HjxkZ8fLx9enx8vNGoUSMjPDzcYf91tk9mJ/25ZvWze/fuTOvu27ev/TkZhmH88ssvRlhYmNGtWzf7tI0bNxqhoaHGgAEDHJbdt2+fUbFiRePRRx+1T3N1X09v37p16zq0b/prvn379vZpn376qUv7f1YmTpzo9P/EwIEDnb7e/u///s8IDQ015syZk2kdnTt3Nq5cueLSdtP3EWc/9evXN37++WeH5U+dOmVUqVLFGDhwYKZ1jRo1yqhQoYLDe2bDhg0z1Zlu3rx5RmhoqPHjjz+6VGvGxzj76dixo3H69GmH5XPyHti3b1+jRo0a1/3bNW/e3IiKinJ4fRjGP69NZ+/dtxKHQV3UoUMHTZ8+3eHn6aefvuXbadKkiYoUKZLpcNj8+fPl5eWlFi1a2Kel92QYhqGEhASdO3dOxYoV0x133KHdu3fneNtnz57Vjh07FBkZqTvuuMM+3cvLS0888USm5S0Wi3x8fCRJqampiouL07lz5+yfcG6khnRr166VYRjq0aOHw6fdoKAgtWnTRsePH9fevXsdHhMeHp7pDKZ77rlHKSkpOn78uEvbnTdvniIiIlSmTBn7tNatW2v37t36888/7dM2btyo8+fP68knn1RQUFCm9aSfLLFixQpduXJF/fr1k5+fX5bL3aju3btnmpaTdlm8eLEk6fnnn880vsVisdh7o6Kjo+Xr62s/FC+l7Xfz5s1TaGjodce2+Pv7KzIyUuvXr1dcXJx9+sqVK/X333+rdevW9mmFCxfWqVOnnB7uzqmuXbsqMDBQ77zzjsuHUF3Vpk0bFSlSxH6/QoUKKly4sAIDAzONOa1Zs6ZiY2OdHpLv2bOnw/0HH3xQd9xxh9auXWuftnjxYhUqVEiNGzfWuXPn7D9xcXGKjIzU8ePHdfjwYYf1PPDAAy6fWbhp0yZ7L1/GHrjChQurS5cuunTpkv2kghvl7e2d6T00/Sfj+026Hj16OPSGVqlSRXXr1tXmzZvtf8fVq1dLSutFzbhshQoV1LBhQ/3888/2nhhX9/V0bdu2dWjfggULKjw83OHvnD5/7dq1SkpKyvHfxBmbzaZ169apUqVKmY4e9OrVS1ar1eHQdLquXbvmeBhOo0aN7G0wdepUDRs2TAUKFNDTTz/t8P66cuVKJScnq127dg7737lz5xQZGSmbzZZp//D391ebNm1yVM/19O3b117v5MmT7Ye7e/XqpYsXL9qXy8l7YJEiRXT58mVt2LBBhmE43e7vv/+u33//XdHR0UpOTnZ4/rVq1ZKvr682bdp0S5/rtTgM6qKyZcvqvvvuc/t2fHx81KxZM82ePVs7duxQjRo1tH//fu3atUtNmzZ1GNC7d+9evfvuu/rpp58yHTPP6Vk2kuzjZNK7pjMqX76808csX75c06dP1759+zL9M8z44smp9Gs2/e9//8s0L33a0aNH7WcGSXIIWOnS/14XLly47ja3bdumw4cPq0WLFjpy5Ih9evXq1WW1WjV37lz7eIv0N+yMh1icSV/OHWe7BQQEOA2AkuvtcuTIEVksFlWoUCHbbRUqVEjR0dFasGCBzp07p4CAAG3ZskVHjx7V0KFDXaq3VatWWrlypb755hv7IeWFCxeqaNGiDodGBw4cqL59++qxxx5TYGCgateurQYNGigqKsohuLuiYMGCeuaZZ/TKK69o9uzZt3RslbPXWNGiRVWqVCmn06W0/TDj4X8/Pz+nh6vuuusurVmzRpcuXZKvr68OHDigxMTEbN+Dzp496xB6cjIMwtXX283w8PDI0Xuos6B51113aePGjTpx4oT+97//6dixY7JarU6XLV++vNasWaNjx44pICDA5X09nbP29ff3d3gvadasmRYvXqwpU6ZoxowZql69uurVq6dmzZopJCTE5eea0blz53Tp0iWn77n+/v4qWbKk07a4kWEvpUqVytQmkZGRatKkiUaMGKE5c+ZIkg4cOCBJTj+0pztz5ozD/TJlyjgMmbie1NTUTIfJfXx8HAJzaGioQ72NGzdW+fLl9dxzz+mjjz7SoEGD7PNcfQ/s1auXtm7dqr59+8rf31+1a9dW/fr11bRpU/sHl/TnP2nSpCyvNXrt87/VCGsm1LZtW82ePVvz589XjRo17CcctGvXzr7MiRMn9Nhjj6lw4cJ6+umndeedd6pgwYKyWCwaPXp0rgx4XLVqlZ577jlVq1ZNQ4cOVXBwsLy9vZWamqoePXpk+SnFXbJ7Y3CllvReo4kTJ2rixImZ5i9evFiDBg1yy7iE7MZTORuUL6UFEWdy2i7OehWcad++vebMmaOFCxeqW7dumjt3rry8vFw+w69+/foKCAjQwoUL1aFDB504cUJbt25Vx44dHUJYjRo1tHr1am3cuFFbtmzRli1btHTpUn3wwQf64osvcnQGopT2epo+fbo++OADhx48V6SmpmY5L6v97Wb3w6weFxAQ4PS6cemuDVpZ7R/5mav7upR9O6bz8vLS9OnTtXv3bn3//ffatm2bJk6cqPfee0/jx4/P1bMT03uSblZISIjuvPNO7dq1y/5hIX2/Tb/igTPXfljO6f4XExOTadxl69at9X//93/ZPq5evXqS5HCiTU7eA8uVK6fly5dr8+bN2rx5s3766ScNGzZMEydO1KxZsxzG03Xr1k3333+/0zqy+uB8qxDWTKhatWoKDQ21D+xfvHixbrvtNodPFKtXr9alS5f0wQcfOAyslNI+vee0B0L655PkwYMHM83bv39/pmmLFi2St7e3PvvsM4cXZvqnkIxyOrg7/YX/559/Zhp8ml6Ls560G5WQkKCVK1eqbt26at++fab5v//+u95//32tW7dOUVFR9h6Mffv22d8snEn/tLtv3z6nh3rSpfe8OOuNzOmV4XPSLuXKldN3332n33777bqHMqtWrapKlSpp7ty5ateunVatWqXGjRu7HJ48PT0VHR2tzz77TEePHtXSpUtlGIbTAFWoUCFFRUUpKipK0j+D3+fOnasePXq4tL10Hh4eev7559W3b1998sknTpfx9/d3eqbhzfYmXU9cXJxiY2Mz9a4dOHBAxYsXtw91KFu2rA4fPqzq1atf98ScG5Hx9Xbvvfc6zHPH680VBw4cyDRA/8CBA/Lw8NBtt91mr8lms+nAgQOZeszS9/f097Wc7Os5Va1aNfs6Y2Ji1KpVK73zzjs3FNYCAgJUqFAhp++5Fy9eVGxsrNuvS5f+ATE9rKW/jxUrVuymjzBl9b+gZMmSmj59usO0rIKhs1ozDjHIyXuglBa6H3jgAfth52+//VY9e/bU9OnTNXz4cJUtW1ZS2rCV3DjC5gxj1kyqXbt2SkhI0Msvv6wzZ86odevWDuOb0j/1XftJfc6cOfZr9+RUiRIlFB4ernXr1jlcIy45Odl+WYWMPDw8ZLFYZLPZ7NMMw9AHH3yQadn0fzquHhqNjIyUxWLRtGnTHLqwT58+rfnz5yskJOS6hyBzYvny5bp06ZI6duyoJk2aZPrp2bOnChYsaO/lrFu3rooVK6bp06fr9OnTmdaX3i5NmjRRgQIFNHnyZKeX/0hfrnTp0vL09Mw07mP79u05PgU/J+2SflbbhAkTnF664Nr965FHHtGBAwc0atQoJSUl6ZFHHslRbenBbOHChVq0aJHuuOOOTOMMnZ0xmH7B0Bs9tN64cWPVqFFD06dP19mzZzPNL1eunBITEx3GsqRfQsTdpk6d6nB/9erVOnTokBo3bmyf1qpVK9lsNk2YMMHpOm72EEzdunXl6+urzz//3GE/TUhI0Oeffy5fX1/VrVv3praRUx9//LHD/vfrr7/qhx9+0L333msPrOl/o6lTpzos+8cff2jdunWqVauW/Wy+nO7rrnC2r5YqVUoBAQE3vK9arVY1bNhQe/fu1Xfffecwb+rUqbLZbA77xq22f/9+HT58WEFBQfazfJs2bSovLy9NmjRJly9fzvSY+Ph4p39TZ7L6X+Dt7a377rvP4Ser4TcZpY/fy3hR4Zy8Bzprw/T/Lek1VqpUSaGhoZo9e7bTD3ApKSkuDbW5GfSs3UJbt27V1q1bJaWd5iul9QikH3PPydeptGjRQm+99ZZWrFghi8WSaaBm/fr1VbBgQQ0ePFidO3eWn5+ftm/fru+++0633357todvsjNkyBB16dJFjz76qB577DH7pTucrS8qKkorV65U165d1apVK6WkpGjNmjWZrgMkpX0iK1u2rJYtW6YyZcqoRIkSKliwYJbfjXjnnXeqe/fu+vjjj9W5c2c1bdpUiYmJmjNnji5duqRx48blaDzE9cydO1cFCxbMsou7YMGCql+/vtasWaNTp04pKChIb7zxhgYMGKDmzZvbL91x7tw5bdy4UU888YQaN26sUqVKaejQoRo5cqSaN2+uli1bKiQkRKdOndLatWs1evRoVaxYUYUKFVLr1q319ddfa+DAgapdu7aOHDmi+fPnKywszOVr50k5a5dq1arpqaee0kcffaQ2bdqoadOmKlmypI4dO6aVK1fq66+/dujeT98vFy9erNKlS2fqhbme9De9GTNmKCEhQQMHDsy0zMMPP6zw8HBVq1ZNgYGBio2N1Zw5c1SgQAE1a9YsR9vLaNCgQXrsscd04MCBTJcZad++vaZPn66+ffvq8ccfV4ECBbRy5cobfh25qlixYlq9erVOnz5tb/MvvvhCJUqUUL9+/ezLNWnSRG3atNHnn3+uX3/9VQ0bNlSxYsV08uRJ7dy5U0eOHHE4ISGn/Pz8NGjQII0cOVLt27e3h+oFCxboyJEjGjlypMPYoRuR/s0gzpQpU0Y1a9Z0mHbixAl1795dkZGRio2N1eeffy4fHx+98MIL9mXq1q2rpk2batmyZbp48aIaNmyo2NhYffHFF/L29tawYcPsy+Z0X3fFBx98oE2bNqlBgwYqXbq0DMPQ+vXrdfDgwRz3AGc0cOBA/fDDD+rbt686deqk22+/Xdu2bdPy5ct199135/hwflYOHz5sb5OUlBT99ddf+uqrr5SSkuIw/qtUqVIaMWKEhg0bpocfflgtWrRQSEiIzp07pz/++ENr1qzRsmXLXBorXbVqVVmtVk2ZMkUXL16Ur6+vSpcu7dLXW23bts1+Isfly5f166+/av78+fbhQOly8h7YvXt3FSlSRBEREQoODlZcXJwWLFggi8ViH+JhsVg0duxYde3aVS1atFDbtm1Vvnx5Xb58WUeOHNHq1as1cODAW35CRUaEtVvoxx9/1HvvvecwLeNhl5yEtWLFiqlx48b65ptvVKdOnUwvgttvv10fffSRJkyYoClTpsjDw0M1a9bUzJkzNWrUKJfPfrxWeu/D+PHjNXXqVBUpUkRRUVF69NFHM30XabNmzZSYmKgZM2ZozJgx9oHizz//vNOvlBo3bpxGjx6tt99+W3///bdCQkKy/SLrF154QWXLltUXX3yh8ePHq0CBAqpevbrGjx+viIiIG3p+zvz555/atWuXHnrooWzHWTz00ENauXKlFixYoN69e6tRo0b64osvNGXKFM2dO1eJiYkqUaKEatWq5XAR2/Q322nTpmnmzJlKTk5WYGCg7r33XofB6C+99JIMw9CaNWu0du1aVa5cWR988IHmzJmTo7CW03YZNGiQKlSooM8//9zem1GqVCnVr18/0ziYwoULq2nTppo3b57atGlzQ9cua926tcaMGSOr1epwdnO6bt266dtvv9XMmTMVHx+v4sWLq3r16urVq5fLg8OdiYiIUGRkpNatW5dpXpkyZTR58mRNmDBB7777rvz9/dWyZUu1bdtWTZs2veFtXo+vr68+/fRTjR49WuPHj5dhGLr//vs1ZMiQTIeA3nzzTdWpU0dz5szRhx9+qCtXrqhkyZKqVKmSnn/++ZuuJf2EjmnTpmny5MmS0s6qnDx58i3pyUlOTtbgwYOdzmvevHmmsPbxxx/rzTfftPfmVK9eXYMHD860D4wbN06VKlXSggUL9H//93/y9fXV3XffrQEDBmS6mHRO9nVXNG7cWLGxsVqxYoXOnDkjHx8flS1bVq+//rrDGOOcCgkJ0Zw5czRx4kQtXrxY8fHxCgoKUq9evfT000/f1MXXM9q0aZP9LEaLxSI/Pz9VrVpV3bp1y9ST2rZtW5UrV06ffPKJvvrqK8XHx8vf31933HGHBgwY4NJ13aS0C1aPHj1aH330kV577TVduXJFrVu3dimszZw5037bw8NDxYsXV9OmTdWnTx+HYSY5eQ989NFH9c033+irr77SxYsX5e/vr4oVK2rYsGEOQ4wqVqyoBQsW6MMPP9S6des0e/ZsFSpUSCEhIWrdunWOP7jmlMXI7VHgAP7V0s8SW7dundMzH4GbMWTIEC1YsCDLbyAA8iPGrAFwWXx8vBYvXqz69esT1AAgl+RKWBszZowiIyMVFhamP/74wz790KFD6tChg6KiotShQweHiw1mNw9A7vrjjz+0cOFCPfPMM7p06ZJ69eqV1yUBQL6RK2GtUaNGmjVrVqaLBA4fPlydOnXSypUr1alTJ7366qsuzQOQu1auXKkXX3xRBw8e1PDhw1WjRo28LgkA8o1cHbMWGRmpKVOmKDQ0VGfPnlVUVJS2bNkiDw8Ppaamqk6dOlq1apUMw8hy3o1+6TUAAMC/UZ6dDRoTE6OgoCD75Rc8PDwUGBiomJgYGYaR5TzCGgAAyE84wQAAAMDE8qxnLTg4WKdOnVJqaqr9UOfp06cVHBwswzCynJdT588nymZz35He4sUL6+zZzFemR96iXcyHNjEf2sScaBfzyY02sVotKlbM+dfJ5VlYK168uCpWrKilS5eqZcuWWrp0qSpWrGg/zJndvJyw2Qy3hrX0bcB8aBfzoU3MhzYxJ9rFfPKyTXLlBIPXX39dq1at0pkzZ1SsWDH5+/tr2bJlOnDggIYMGaK4uDj5+flpzJgxuvPOOyUp23k5cfZsglv/wCVLFlFsbLzb1o8bQ7uYD21iPrSJOdEu5pMbbWK1WlS8eGGn8/7z32BAWMufaBfzoU3MhzYxJ9rFfPI6rHGCAQAAgIkR1gAAAEyMsAYAAGBihDUAAAATI6wBAACYGGENAADAxAhrAAAAJkZYAwAAMDHCGgAAgIkR1gAAAEyMsAYAAGBihDUAAAATI6wBAACYGGENAADAxAhrAAAAJkZYAwAAMDHCGgAAgIkR1gAAAEyMsAYAAGBihDUAAAATI6wBAACYGGENAADAxAhrAAAAJkZYAwAAMDHCGgAAgIkR1gAAAEyMsAYAAGBihDUAAAATI6wBAACYGGENAADAxAhrAAAAJkZYAwAAMDHCGgAAgIkR1gAAAEyMsAYAAGBihDUAAAATI6wBAACYGGENAADAxAhrAAAAJkZYAwAAMDHPvC4AAADAjOJ+/EFn5s/TH+fPybNYgEq0aSu/e+7L9ToIawCQz5jlHxBgZnE//qBTn82QkZwsSUo5d1anPpshSbn+eiGsAUA+YqZ/QMDNMlJSZEtOlpGcfPV3kmzJV67+TpueNi8pwzIZfzufbiQn68rZM5JhOG4vOVln5s8jrAEArs+w2WSkpshISZVSU2WkpFy9nyIj/X76vAzTT8/+0h7U7OtKTtbp2V/K6usri4enLB4esnh6pv14eEr2+x6Z5svDQxaLJY/+CjArhxB15ervpOTMIepKsowkZ6EpScaVK84fkz4vOVlKTc15cR4esnp5yeLldfW3t/2+Z5Eisly9feVMrNOHp5w7e5N/nZwjrAFwq3/DITeH4JMedlwIPukBSSmp9scbqRkfk5LhMamOj8mwTNr2snrMP78zPubaT/w3y5YQrxMT37mxB3t4/BPg7L89JSfhzuLhIXlcDX7py3t42sOgnCxvubq8rll/xu3Jyfbt6/P0cFheVqspA2ZuvFaMlBTZrjj2PDkEoisZepiuDVFXkp0/5tp5NxqirFZZvb0dQpSlQAFZvbzkUbiIrN7XBKwCBa7e95bF2ytDAEt7nMXL2/ljPF2LPn///rvTYOYZUDznz+0mWQzjFr/iTebs2QTZbO57iiVLFlFsbLzb1o+cSX+zSzFxMPivMgwjLUzYbDJS04JN3E9bdObr2TKuXLEvZylQQMWaRqtQxUr/BJ8MAemGgo9DeMr74OMgPchc2xvlEDb+ue8QOjKEmsyPSQ8+jr1glgy9YM4ec3zSO0q9eDFzmUWLKqTfAKd/d8e/13XaxaEtnS+ftn9kXv6fUHx1ve5isTgJeY7tISd/a/vf3MnymdsvPSRmDK/XrC9D+ybs+UXnFi90fK14FpB/VBP5lv+fY2hKytBblZycOXhlNe9mQlTGIHRtr5S319VQlTFoZQhRBbycP8brn3lWLy+XQ1RuuXbIgCRZvLwU9PgTbvm/YrVaVLx4YafzCGs3ibBmHrn9wsoJw2aTroaY9CBj2NJvX51uS++5SevlcVj+Rh5rS037x2dzYf3XhCyH25kemypd3b6Rakv7x3r1+bmdyYLPdQ8RWq2yWM11hSQzv04yMgwjbR91CHNOej7t4e/ansprw5+zXtHUf8JmpsPI/4RGp8s7mXdDQehmWCwZeqKuDVFemQJWVtMzz/N2vG+yEJWbcrMDgLBGWPtPM1JSlJoQryOjRjjtMbD6+iqgWfObDCQZApIt8zIZQ1PGdaYHIrf23GTlau+BPDxksVqvBgtrWpiwpocP69XDQv+EoPTlZb0afKyOy6Stw/Of6enLZwhSurq92Nmzsiwv5LlB//rg829FD7R72D+UZeiRTA9+aSEvcw9jegiNeX9SlustM/SVa0JUWi8W4wVzT278r88urOXfuAzTMgxDtsuXlRp3Ualx8UqJj0u7HR+vlLg4pcbHKTUu7SclPk62xMRs12e7dElnvv7KcWJ6sEgPJfYQkhZk7CHEHjzSblsLFJCs3jkMQVeXsWZY5toQ5Hl1eoZtXXf9DtvydHysSYLN+VUrshzzUahylTyoCFLaWZ9+99zHh81bLO39xHpDPVGxAcWzfK0UvPOuW1Ee/sUIa8gVRmqqUuPjlRoflxa4roYuh9vx8VcDWlyW41WshQrJs4ifPPz85FW6tAoW8ZOnX9r9Mwvmy5aQ+R+PZ7FiKjdq9NXwY97Bxf9FJdq0dXrIrUSbtnlYFWA+vFaQHcIabohhGDKSLislLv5qT9fFLG5f7f1KSHC6Hounpzyuhi+PIn7yvu02efgVlUeRIldDWIbbhYtk+4nV6u3t/M2u7SOy+hS85X8DXF/6oTUOuQHZ47WC7DBm7Sb9lw4jGDabY+9Xethydjs+PtO1mtJZfQvJw6+IvQfMw88v7fbV+55+/9y2Fix4S3u5GItjXv+l18p/BW1iTrSL+TBmDW5lS0q6GrLSxnw53L7mMGRqQoLzgfAeHv8ErCJF5B18mzz8isijiJ88/Yqm3fYrmna/SPa9X+7GWBwAwH8NYe1fxrDZlJqY4DDAPtPt+Li0gflxF7Pu/SpY8GovV1F5lSolj/+FZur1Sr9t9fVljBcAAHmEsHaDbuWVpm3JyZnGfNnPfLx2IH5CvPPeL6v1ak9XWtAqEBiU1utVxC/tkGSG2x5F/NLOagQAAKZHWLsB1/siZMNmky0xMXOvl8O4r3h7QDOSLjvdjtXHxz7AvkBgoAqWL5953NfVgGb19TXFpRoAAMCtRVi7AWfmz3P6Rcgnp09T7NdfKTU+3vnV3K1WeRS5OtariJ8K3Fn+6uD7q2O+7IPy0wKa1csrl54RAAAwK8LaDXB24UJJUmqqClcP/6f365qzIa2FCtH7BQAAcoSwdgM8s7nSdNDjT+ZBRQAA4L+Kbp4bUKJNW1muOUTJlaYBAIA70LN2A7jSNAAAyC2EtRvExVcBAEBu4DAoAACAiRHWAAAATIywBgAAYGKENQAAABMjrAEAAJiYKcLa+vXr1apVK7Vs2VItWrTQqlWrJEmHDh1Shw4dFBUVpQ4dOujw4cN5WygAAEAuy/NLdxiGocGDB2vWrFkKDQ3Vb7/9pkcffVSNGzfW8OHD1alTJ7Vs2VKLFi3Sq6++qs8++yyvSwYAAMg1puhZs1qtio9Pu1ZZfHy8AgMDdf78ee3du1fR0dGSpOjoaO3du1fnzp3Ly1IBAAByVZ73rFksFr3zzjvq06ePfH19lZiYqKlTpyomJkZBQUHy8PCQJHl4eCgwMFAxMTEKCAjI46oBAAByR56HtZSUFH344Yd6//33VatWLf3888969tlnNXbs2Fuy/uLFC9+S9WSnZMkibt8Gco52MR/axHxoE3OiXcwnL9skz8Pavn37dPr0adWqVUuSVKtWLRUsWFDe3t46deqUUlNT5eHhodTUVJ0+fVrBwcE5Wv/Zswmy2Qx3lC5JfN2USdEu5kObmA9tYk60i/nkRptYrZYsO5jyfMxaqVKldPLkSR08eFCSdODAAZ09e1Zly5ZVxYoVtXTpUknS0qVLVbFiRQ6BAgCAfCXPe9ZKliypESNGaMCAAbJYLJKk0aNHy9/fXyNGjNCQIUP0/vvvy8/PT2PGjMnjagEAAHKXxTAM9x0jNAEOg+ZPtIv50CbmQ5uYE+1iPvn+MCgAAACyRlgDAAAwMcIaAACAiRHWAAAATIywBgAAYGKENQAAABMjrAEAAJgYYQ0AAMDECGsAAAAmRlgDAAAwMcIaAACAiRHWAAAATIywBgAAYGKENQAAABMjrAEAAJgYYQ0AAMDECGsAAAAmRlgDAAAwMcIaAACAiRHWAAAATIywBgAAYGKENQAAABMjrAEAAJgYYQ0AAMDECGsAAAAmRlgDAAAwMcIaAACAiRHWAAAATIywBgAAYGKENQAAABMjrAEAAJgYYQ0AAMDECGsAAAAmRlgDAAAwMcIaAACAiRHWAAAATIywBgAAYGKENQAAABMjrAEAAJgYYQ0AAMDECGsAAAAmRlgDAAAwMZfC2vnz591dBwAAAJxwKaw1bNhQTz/9tFasWKHk5GR31wQAAICrXApr69at07333quPPvpI9erV0yuvvKJt27a5uzYAAIB8z6WwFhAQoMcff1zz5s3T7NmzFRAQoMGDB6tRo0Z69913dfz4cXfXCQAAkC/l+ASDM2fO6MyZM0pMTNTtt9+uU6dOqXXr1po6dao76gMAAMjXPF1Z6M8//9TixYu1dOlSFSxYUK1atdKiRYtUqlQpSVKfPn3UokUL9ezZ063FAgAA5DcuhbXOnTurWbNmevfdd1WtWrVM80uXLq2uXbve8uIAAADyO5fC2saNG1WgQIFslxkwYMAtKQgAAAD/cGnM2pgxY7R9+3aHadu3b9cbb7zhlqIAAACQxqWwtnTpUlWpUsVhWpUqVbR06VK3FAUAAIA0LoU1i8UiwzAcpqWmpspms7mlKAAAAKRxKaxFRETonXfesYczm82mSZMmKSIiwq3FAQAA5HcunWDw8ssvq1evXqpXr55uu+02xcTEqGTJkpoyZYq76wMAAMjXXAprpUqV0oIFC7Rr1y6dPHlSwcHBqlatmqzWHF9TFwAAADngUliTJKvVqho1arizFgAAAFzDpbCWkJCgSZMmaevWrTp//rzDyQYbNmxwV20AAAD5nkvHMUeMGKG9e/eqT58+unDhgoYNG6bg4GA98cQTbi4PAAAgf3OpZ23Tpk1avny5ihUrJg8PDzVu3FhVq1ZV7969CWwAAABu5FLPms1mU5EiRSRJvr6+io+PV8mSJXXkyBG3FgcAAJDfudSzVqFCBW3dulX33nuvIiIiNGLECBUqVEjlypVzc3kAAAD5m0s9a6+//rpCQkIkpV1zzcfHR3FxcRo7dqxbiwMAAMjvrtuzlpqaqvnz5+vpp5+WJBUvXpwvcAcAAMgl1+1Z8/Dw0BdffCFPT5cvyQYAAIBbxKXDoK1atdKXX37p7loAAABwDZe6y3bv3q3PP/9c06ZNU6lSpWSxWOzzZs2a5bbiAAAA8juXwlr79u3Vvn17txWRlJSk0aNHa/PmzfL29lZ4eLhGjRqlQ4cOaciQIbpw4YL8/f01ZswYzkAFAAD5ikthrXXr1m4t4q233pK3t7dWrlwpi8WiM2fOSJKGDx+uTp06qWXLllq0aJFeffVVffbZZ26tBQAAwExcCmtz587Ncl67du1uqoDExEQtXLhQ3377rf3waokSJXT27Fnt3btX06dPlyRFR0dr1KhROnfunAICAm5qmwAAAP8WLoW1RYsWOdw/c+aMjh49qho1atx0WDt69Kj8/f313nvvacuWLSpUqJAGDBggHx8fBQUFycPDQ1LaWamBgYGKiYkhrAEAgHzDpbA2c+bMTNPmzp2rAwcO3HQBqampOnr0qCpVqqQXX3xRu3btUu/evfXuu+/e9LolqXjxwrdkPdkpWbKI27eBnKNdzIc2MR/axJxoF/PJyza54YuntWnTRvfcc49efPHFmyogODhYnp6eio6OliRVr15dxYoVk4+Pj06dOqXU1FR5eHgoNTVVp0+fVnBwcI7Wf/Zsgmw246ZqzE7JkkUUGxvvtvXjxtAu5kObmA9tYk60i/nkRptYrZYsO5hc/iL3jD+JiYn66quv7F/ufjMCAgJUp04dbdq0SZJ06NAhnT17VuXKlVPFihW1dOlSSdLSpUtVsWJFDoECAIB8xaWetUqVKjlcW02SgoKCNHLkyFtSxGuvvaahQ4dqzJgx8vT01NixY+Xn56cRI0ZoyJAhev/99+Xn56cxY8bcku0BAAD8W1gMw7juMcLjx4873C9YsOC/poeLw6D5E+1iPrSJ+dAm5kS7mE9eHwZ1qWfN09NTPj4+Klq0qH3axYsXdfnyZQUFBd2aKgEAAJCJS2PW+vTpo5MnTzpMO3nypPr16+eWogAAAJDGpbB26NAhhYWFOUwLCwvTwYMH3VIUAAAA0rgU1ooXL64jR444TDty5Ij8/f3dURMAAACucimstW3bVs8884zWr1+v/fv3a926derfv78eeeQRd9cHAACQr7l0gkHPnj3l6empMWPG6OTJkwoODla7du305JNPurs+AACAfM2lsGa1WtWjRw/16NHD3fUAAAAgA5cOg06dOlW7d+92mLZ792599NFHbikKAAAAaVwKa5999pnKly/vMO2uu+7Sp59+6paiAAAAkMalsHblyhV5ejoeMS1QoICSk5PdUhQAAADSuBTWKleurC+++MJh2uzZs1WpUiW3FAUAAIA0Lp1g8NJLL+nJJ5/U4sWLVaZMGR09elSxsbGaPn26u+sDAADI11wKa//73/+0cuVKbdiwQTExMXrooYfUoEEDFSpUyN31AQAA5GsuhTVJKlSokJo1ayZJunDhghYuXKgFCxZo7ty5bisOAAAgv3M5rKWkpGjDhg1auHChvv32W5UqVUodOnRwZ20AAAD53nXD2p49e7Rw4UItXbpUqampevDBB+Xt7a3Zs2erePHiuVEjAABAvpVtWIuOjtbRo0f1wAMPaOTIkWrQoIG8vLz03Xff5VZ9AAAA+Vq2l+74+++/ZbVa5e3tLR8fHxUoUCC36gIAAICu07O2du1abd26VQsWLNBzzz0nb29vNW3aVElJSbJYLLlVIwAAQL513Yvi3n333Ro9erQ2bdqkF198UYcOHVJiYqK6dOmiWbNm5UaNAAAA+ZZL32AgST4+PmrZsqU++eQTrV+/Xi1atCCsAQAAuJnLYS2joKAg9erVS8uXL7/V9QAAACCDGwprAAAAyB2ENQAAABMjrAEAAJiYy183lc5msznct1rJewAAAO7iUlj79ddfNXLkSP3+++9KSkqSJBmGIYvFon379rm1QAAAgPzMpbA2ZMgQNWzYUKNHj5aPj4+7awIAAMBVLoW148eP67nnnuNbCwAAAHKZSwPOHnzwQW3cuNHdtQAAAOAaLvWsJSUlqV+/fqpVq5ZKlCjhMG/s2LFuKQwAAAAuhrXy5curfPny7q4FAAAA13AprPXr18/ddQAAAMAJl6+ztmXLFi1cuFCnT59WYGCgWrZsqXvuucedtQEAAOR7Lp1g8PXXX+vZZ59VyZIl9eCDDyowMFDPP/+85syZ4+76AAAA8jWXetY+/vhjTZ8+XRUqVLBPa9q0qfr376/27du7rTgAAID8zqWetQsXLuiuu+5ymHbnnXfq4sWLbikKAAAAaVwKazVr1tT//d//6e+//5YkXbp0SWPHjlWNGjXcWhwAAEB+59Jh0Ndee03PPfecIiIiVLRoUV28eFE1atTQ+PHj3V0fAABAvuZSWAsMDNSsWbMUExOj2NhYBQYGqlSpUu6uDQAAIN/LMqwZhmH/LlCbzSZJCgoKUlBQkMM0q9WlI6kAAAC4AVmGtVq1amn79u2SpEqVKmX6Evf0MLdv3z73VggAAJCPZRnWli1bZr+9du3aXCkGAAAAjrI8hhkcHGy/vWLFCoWEhGT6WbVqVa4UCQAAkF+5NOBs8uTJTqd/8MEHt7QYAAAAOMr2bNDNmzdLSjuZ4Mcff5RhGPZ5x44dU6FChdxbHQAAQD6XbVh7+eWXJUlJSUkaOnSofbrFYlHJkiU1bNgw91YHAACQz2Ub1tatWydJGjx4sMaOHZsrBQEAAOAfLo1ZI6gBAADkDZe+wSAhIUGTJk3S1q1bdf78eYexaxs2bHBXbQAAAPmeSz1rI0aM0N69e9WnTx9duHBBw4YNU3BwsJ544gk3lwcAAJC/udSztmnTJi1fvlzFihWTh4eHGjdurKpVq6p3794ENgAAADdyqWfNZrOpSJEikiRfX1/Fx8erZMmSOnLkiFuLAwAAyO9c6lmrUKGCtm7dqnvvvVcREREaMWKEChUqpHLlyrm5PAAAgPzNpZ61119/XSEhIZLSrr3m4+OjuLg4zhIFAABwM5d61sqUKWO/Xbx4cb3xxhtuKwgAAAD/yDKszZ0716UVtGvX7pYVAwAAAEdZhrVFixY53N++fbtKlCih4OBgxcTE6MyZM6pZsyZhDQAAwI2yDGszZ8603x41apQaNWrkcJmOTz/9VEePHnVrcQAAAPmdSycYLF68WF26dHGY1rlz50y9bwAAALi1XAprJUqUsH+pe7r169crICDALUUBAAAgjUtngw4bNkzPPPOMpk2bplKlSikmJkb79+/Xu+++6+76AAAA8jWXwlrdunW1Zs0afffddzp9+rQaNGigBx54QMWKFXN3fQAAAPmaS2FNkgICAtSqVSs3lgIAAIBrZRnWunfvrmnTpkmSOnXqJIvF4nS5WbNmuacyAAAAZB3WMvaiPfLII7lRCwAAAK6RZVhr3ry5/Xbr1q1zpRgAAAA44uumAAAATMzlr5tyxmKx3NKw9t5772nSpElasmSJQkNDtXPnTr366qtKSkpSSEiI3nrrLRUvXvyWbQ8AAMDsXPq6qdzw66+/aufOnQoJCZEk2Ww2vfDCC3rzzTcVERGh999/X+PGjdObb76Zq3UBAADkJZe+wSAjwzBks9nsP7dCcnKyRo4cqREjRtin7dmzR97e3oqIiJAkdezYUStWrLgl2wMAAPi3cOk6a6dOndLIkSO1bds2xcXFOczbt2/fTRfx7rvvqkWLFipdurR9WkxMjG677Tb7/YCAANlsNl24cEH+/v4ur7t48cI3Xd/1lCxZxO3bQM7RLuZDm5gPbWJOtIv55GWbuBTWhg8fLh8fH82YMUOdO3fWrFmzNGnSJD3wwAM3XcCOHTu0Z88eDRo06KbX5czZswmy2Qy3rFtKa7zY2Hi3rR83hnYxH9rEfGgTc6JdzCc32sRqtWTZweRSWNuxY4fWr18vX19fWSwWVahQQW+88YY6duyo9u3b31RxW7du1YEDB9SoUSNJ0smTJ9W9e3d16dJFJ06csC937tw5Wa3WHPWqAQAA/Nu5NGbNarXK0zMt1/n5+encuXPy9fXVqVOnbrqAnj17auPGjVq3bp3WrVunUqVKadq0aerRo4cuX76sbdu2SZJmz56tJk2a3PT2AAAA/k1c6lmrXr26vv32Wz344IOqV6+enn32Wfn4+KhKlSpuK8xqtWrs2LEaPny4w6U7AAAA8hOLYRhZDujav3+/ypcvr7i4ONlsNvn7++vy5cv65JNPlJiYqK5duyowMDA3680xxqzlT7SL+dAm5kObmBPtYj6mHrPWqlUrhYWFqXXr1mrWrJkkycfHR3369Ln1VQIAACCTbMesff/992rdurUWLVqk+vXrq2/fvlqzZo1SUlJyqz4AAIB8LdvDoBkdPHhQixYt0pIlS3Tp0iU1a9ZMLVu2VLVq1dxd403hMGj+RLuYD21iPrSJOdEu5pPXh0Fd/gaDO++8U88995zWrVun8ePHa/369erQocMtKxIAAACZuXQ2aLqdO3dq4cKF+uabb1SkSBH17dvXXXUBAABALoS148ePa9GiRVq0aJHOnj2rqKgoTZ482f6dnQAAAHCfbMNa586dtXPnTtWpU0f9+vXTgw8+KB8fn9yqDQAAIN/LNqzVr19f48ePV1BQUG7VAwAAgAyyDWs9e/bMrToAAADghMtngwIAACD3EdYAAABMjLAGAABgYjcc1gzD0NatW29lLQAAALjGDYe1K1eu6PHHH7+VtQAAAOAa2Z4NunDhwiznXbly5VbXAgAAgGtkG9ZeeuklVa5cWV5eXpnmufj97wAAALgJ2Ya1smXLatCgQbrnnnsyzUtKSlL16tXdVhgAAACuM2atdu3aOnjwoPMHWq26++673VIUAAAA0mTbszZy5Mgs5xUoUEAzZ8685QUBAADgH9n2rMXGxuZWHQAAAHAi27AWFRXlcL9fv35uLQYAAACOsg1r157x+dNPP7m1GAAAADjKNqxZLJbcqgMAAABOZHuCQWpqqn788Ud7D1tKSorDfUm699573VshAABAPpZtWCtevLiGDh1qv+/v7+9w32KxaO3ate6rDgAAIJ/LNqytW7cut+oAAACAEzf8Re4AAABwP8IaAACAiRHWAAAATIywBgAAYGKENQAAABMjrAEAAJgYYQ0AAMDECGsAAAAmRlgDAAAwMcIaAACAiRHWAAAATIywBgAAYGKENQAAABMjrAEAAJgYYQ0AAMDECGsAAAAmRlgDAAAwMcIaAACAiRHWAAAATIywBgAAYGKENQAAABMjrAEAAJgYYQ0AAMDECGsAAAAmRlgDAAAwMcIaAACAiRHWAAAATIywBgAAYGKENQAAABMjrAEAAJgYYQ0AAMDECGsAAAAmRlgDAAAwMcIaAACAiRHWAAAATIywBgAAYGKENQAAABMjrAEAAJgYYQ0AAMDECGsAAAAmRlgDAAAwMc+8LuD8+fMaPHiw/vrrL3l5eals2bIaOXKkAgICtHPnTr366qtKSkpSSEiI3nrrLRUvXjyvSwYAAMg1ed6zZrFY1KNHD61cuVJLlixRmTJlNG7cONlsNr3wwgt69dVXtXLlSkVERGjcuHF5XS4AAECuyvOw5u/vrzp16tjvh4eH68SJE9qzZ4+8vb0VEREhSerYsaNWrFiRV2UCAADkiTw/DJqRzWbTl19+qcjISMXExOi2226zzwsICJDNZtOFCxfk7+/v8jqLFy/shkodlSxZxO3bQM7RLuZDm5gPbWJOtIv55GWbmCqsjRo1Sr6+vurcubNWr159S9Z59myCbDbjlqzLmZIliyg2Nt5t68eNoV3MhzYxH9rEnGgX88mNNrFaLVl2MJkmrI0ZM0ZHjhzRlClTZLVaFRwcrBMnTtjnnzt3TlarNUe9agAAAP92eT5mTZImTJigPXv2aPLkyfLy8pIkValSRZcvX9a2bdskSbNnz1aTJk3yskwAAIBcl+c9a3/++ac+/PBDlStXTh07dpQklS5dWpMnT9bYsWM1fPhwh0t3AAAA5Cd5Htb+97//6ffff3c6r2bNmlqyZEkuVwQAAGAepjgMCgAAAOcIawAAACZGWAMAADAxwhoAAICJEdYAAABMjLAGAABgYoQ1AAAAEyOsAQAAmBhhDQAAwMQIawAAACZGWAMAADAxwhoAAICJEdYAAABMjLAGAABgYoQ1AAAAEyOsAQAAmBhhDQAAwMQIawAAACZGWAMAADAxwhoAAICJEdYAAABMjLAGAABgYoQ1AAAAEyOsAQAAmBhhDQAAwMQIawAAACZGWAMAADAxwhoAAICJEdYAAABMjLAGAABgYoQ1AAAAEyOsAQAAmBhhDQAAwMQIawAAACZGWAMAADAxwhoAAICJEdYAAABMjLAGAABgYoQ1AAAAEyOsAQAAmBhhDQAAwMQIawAAACZGWAMAADAxwhoAAICJEdYAAABMjLAGAABgYoQ1AAAAEyOsAQAAmBhhDQAAwMQIawAAACZGWAMAADAxwhoAAICJEdYAAABMjLAGAABgYoQ1AAAAEyOsAQAAmBhhDQAAwMQIawAAACZGWAMAADAxwhoAAICJEdYAAABMjLAGAABgYoQ1AAAAEyOsAQAAmBhhDQAAwMQIawAAACZGWAMAADAx04e1Q4cOqUOHDoqKilKHDh10+PDhvC4JAAAg15g+rA0fPlydOnXSypUr1alTJ7366qt5XRIAAECu8czrArJz9uxZ7d27V9OnT5ckRUdHa9SoUTp37pwCAgJcWofVanFnibm2DeQc7WI+tIn50CbmRLuYj7vbJLv1mzqsxcTEKCgoSB4eHpIkDw8PBQYGKiYmxuWwVqxYIXeWKEkqXryw27eBnKNdzIc2MR/axJxoF/PJyzYx/WFQAACA/MzUYS04OFinTp1SamqqJCk1NVWnT59WcHBwHlcGAACQO0wd1ooXL66KFStq6dKlkqSlS5eqYsWKLh8CBQAA+LezGIZh5HUR2Tlw4ICGDBmiuLg4+fn5acyYMbrzzjvzuiwAAIBcYfqwBgAAkJ+Z+jAoAABAfkdYAwAAMDHCGgAAgIkR1gAAAEzM1N9gYGZjxozRypUrdfz4cS1ZskShoaF5XVK+d/78eQ0ePFh//fWXvLy8VLZsWY0cOZJLveSxPn366NixY7JarfL19dUrr7yiihUr5nVZkPTee+9p0qRJvIeZRGRkpLy8vOTt7S1JGjRokO6///48rgpJSUkaPXq0Nm/eLG9vb4WHh2vUqFG5WgNh7QY1atRIjz/+uB577LG8LgVXWSwW9ejRQ3Xq1JGUFqjHjRun0aNH53Fl+duYMWNUpEgRSdKaNWs0dOhQLViwII+rwq+//qqdO3cqJCQkr0tBBhMnTiQ4m8xbb70lb29vrVy5UhaLRWfOnMn1GjgMeoMiIiL4JgWT8ff3twc1SQoPD9eJEyfysCJIsgc1SUpISJDFwhdU57Xk5GSNHDlSI0aMyOtSAFNLTEzUwoULNWDAAPt7V4kSJXK9DnrW8J9ks9n05ZdfKjIyMq9LgaSXX35ZmzZtkmEY+vjjj/O6nHzv3XffVYsWLVS6dOm8LgXXGDRokAzDUK1atTRw4ED5+fnldUn52tGjR+Xv76/33ntPW7ZsUaFChTRgwABFRETkah30rOE/adSoUfL19VXnzp3zuhRIeuONN7RhwwY999xzGjt2bF6Xk6/t2LFDe/bsUadOnfK6FFxj1qxZWrx4sebNmyfDMDRy5Mi8LinfS01N1dGjR1WpUiXNnz9fgwYN0jPPPKOEhIRcrYOwhv+cMWPG6MiRI3rnnXdktbKLm0mrVq20ZcsWnT9/Pq9Lybe2bt2qAwcOqFGjRoqMjNTJkyfVvXt3bdy4Ma9Ly/fSh9Z4eXmpU6dO2r59ex5XhODgYHl6eio6OlqSVL16dRUrVkyHDh3K1Tr4T4b/lAkTJmjPnj2aPHmyvLy88rqcfC8xMVExMTH2++vWrVPRokXl7++fd0Xlcz179tTGjRu1bt06rVu3TqVKldK0adNUr169vC4tX7t06ZLi4+MlSYZhaPny5Zw1bQIBAQGqU6eONm3aJEk6dOiQzp49q7Jly+ZqHXw36A16/fXXtWrVKp05c0bFihWTv7+/li1bltdl5Wt//vmnoqOjVa5cOfn4+EiSSpcurcmTJ+dxZfnXmTNn1KdPH/3999+yWq0qWrSoXnzxRVWuXDmvS8NVkZGRmjJlCmcg5rGjR4/qmWeeUWpqqmw2m+666y4NGzZMgYGBeV1avnf06FENHTpUFy5ckKenp5599lk98MADuVoDYQ0AAMDEOAwKAABgYoQ1AAAAEyOsAQAAmBhhDQAAwMQIawAAACZGWAPwnzFkyBC9/fbbebJtwzD00ksv6e6771a7du3ypIZrTZo0SYMGDcrrMgDcJL4bFIDbREZG6u+//9batWvl6+srSfr666+1ePFizZw5M4+ru7V+/vlnbdq0Sd9++639uWY0f/58vfzyy/ZrAKZbsWKFgoKCcqtMAP9ChDUAbmWz2fTZZ5+pd+/eeV1KjqSmpsrDw8Pl5Y8fP66QkBCnQS1deHi4vvzyy1tRHoB8hMOgANyqe/fu+uSTTxQXF5dp3rFjxxQWFqaUlBT7tC5duujrr7+WlNYb1bFjR40ePVoRERFq1KiRtm/frvnz5+uBBx7QvffeqwULFjis8/z583ryySdVo0YNde7cWcePH7fPO3DggJ588knVrl1bUVFRWr58uX3ekCFDNHz4cD311FMKDw/Xli1bMtV76tQp9e7dW7Vr19aDDz6oOXPmSErrLRw2bJh27typGjVqaOLEiTn+O0VGRurDDz/Uww8/rLvvvlsvvfSSkpKS7PPnzJmjBx98ULVr11bv3r116tQp+7w///zT/rzuu+8+TZkyxT7vypUrGjx4sGrUqKFmzZrpl19+sc+bOnWq7r//ftWoUUNRUVHavHlzjusG4H6ENQBuVaVKFdWuXVvTpk27ocfv3r1bYWFh2rJli6KjozVw4ED98ssvWr16td566y2NHDlSiYmJ9uWXLFmiPn36aMuWLapQoYJ9zNalS5fUrVs3RUdH64cfftDbb7+t1157Tfv377c/dunSperdu7e2b9+uWrVqZapl4MCBKlWqlL7//ntNnDhREyZM0ObNm/XII4/otddeU3h4uHbs2KH+/fvf0HNdsmSJpk2bptWrV+vQoUN6//33JUmbN2/W+PHj9c4772jjxo0KCQnRwIEDJUkJCQl68skndf/99+v777/XqlWrdO+999rXuW7dOjVr1kzbtm1TZGSkRo0aJUk6ePCgZs2apblz52rHjh2aNm2aQkJCbqhuAO5FWAPgdv3799fnn3+uc+fO5fixpUuXVtu2beXh4aGHH35YMTEx6tu3r7y8vFSvXj15eXnpr7/+si/foEED3X333fLy8tJzzz2nnTt3KiYmRhs2bFBISIjatm0rT09PVapUSVFRUVqxYoX9sY0aNVKtWrVktVrl7e3tUEdMTIy2b9+uQYMGydvbWxUrVtQjjzyiRYsWufxcdu3apYiICPtP48aNHeY/9thjCg4Olr+/v55++mn79w0vWbJEbdu2VeXKleXl5aWBAwdq586dOnbsmDZs2KASJUqoW7du8vb2VuHChVW9enX7OmvVqqUHHnhAHh4eatmypX777TdJkoeHh5KTk3XgwAFduXJFpUuX1u233+56wwDINYxZA+B2oaGhatCggaZOnaq77rorR48tXry4/Xb64PwSJUrYp3l7ezv0rJUqVcp+u1ChQipatKhOnz6t48ePa/fu3YqIiLDPT01NVYsWLez3g4ODs6zj9OnTKlq0qAoXLmyfdtttt2nPnj0uP5fq1atnO2Yt4/Zvu+02nT592r7typUrOzwvf39/nTp1SjExMdmGrIx/Kx8fHyUlJSklJUVly5bV0KFDNWnSJO3fv1/16tXTkCFDONkBMCF61gDkiv79+2vOnDkOY63SB+NfvnzZPi02NvamtnPy5En77cTERF28eFGBgYEKDg7W3XffrW3bttl/duzYoddee82l9QYGBurixYtKSEiwT4uJibml4SYmJsZ++8SJEwoMDLRvO+PYu0uXLunChQsKCgpScHCwjh49ekPba968ub788kutX79eFotF48aNu7knAMAtCGsAckXZsmX18MMPO1yyIyAgQEFBQVq0aJFSU1M1d+7cGw4e6b799ltt27ZNycnJevfdd1W9enUFBwerQYMGOnz4sBYuXKgrV67oypUr2r17tw4cOODSeoODg1WjRg1NmDBBSUlJ+u233zR37lyHnrmb9cUXX+jkyZO6cOGCpkyZoocffliSFB0drfnz52vfvn1KTk7WhAkTVK1aNZUuXVoNGjRQbGysZsyYoeTkZCUkJGjXrl3X3dbBgwe1efNmJScny8vLS97e3rJa+ZcAmBGvTAC5pm/fvrp06ZLDtFGjRmnatGmqU6eO9u/frxo1atzUNqKjozV58mTVqVNHv/76q9566y1JUuHChTVt2jQtX75c999/v+rVq6dx48YpOTnZ5XVPmDBBx48f1/33369+/frpmWee0X333efy49PPFs34s3v3bofau3XrpsaNG+v222/X008/LUm67777NGDAAD3zzDOqV6+ejh49ar/4b+HChfXJJ59o/fr1qlu3rqKiopyeyXqt5ORkjR8/XnXq1FG9evV07tw5+0kLAMzFYhiGkddFAEB+FxkZqddffz1H4Q9A/kDPGgAAgIkR1gAAAEyMw6AAAAAmRs8aAACAiRHWAAAATIywBgAAYGKENQAAABMjrAEAAJgYYQ0AAMDE/h/nykcvEUDYygAAAABJRU5ErkJggg==\n","text/plain":["<Figure size 720x504 with 1 Axes>"]},"metadata":{},"output_type":"display_data"}],"source":["fig, ax = plt.subplots(figsize=(10, 7));\n","ax.plot(num_epochs, np.array(val_f1_accuracy_list),'ro-' ,label=\"F1 Validation Accuracy\")\n","ax.set_xlabel(\"Number of Epochs\")\n","ax.set_ylabel(\"F1 Validation Accuracy\")\n","ax.set_title(\"F1 Validation Accuracy vs Number of Epochs for Bert-Base\",fontsize=18)\n","ax.set_ylim(0, 100)"]},{"cell_type":"code","execution_count":30,"id":"a98dc487","metadata":{"execution":{"iopub.execute_input":"2023-10-25T20:00:05.644796Z","iopub.status.busy":"2023-10-25T20:00:05.644452Z","iopub.status.idle":"2023-10-25T20:00:05.913796Z","shell.execute_reply":"2023-10-25T20:00:05.912784Z"},"papermill":{"duration":0.300191,"end_time":"2023-10-25T20:00:05.915896","exception":false,"start_time":"2023-10-25T20:00:05.615705","status":"completed"},"tags":[]},"outputs":[{"data":{"text/plain":["(0.0, 100.0)"]},"execution_count":30,"metadata":{},"output_type":"execute_result"},{"data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAnAAAAFWCAYAAADpDqAVAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAABUqklEQVR4nO3dd1hTZ/8G8Dthyh6yxFW1QRBXQbFKq4KKA8VRxVlnrbXu+nPV1llbcM/XOmpt63gtiAOtW1t3ta7XSlulVlFZshSQlZzfH5hTQgADEpLo/bkuLs3JyTnf8CThzvM85xyJIAgCiIiIiMhgSHVdABERERGVDwMcERERkYFhgCMiIiIyMAxwRERERAaGAY6IiIjIwDDAERERERmY1zbAeXh4YMaMGbouo1IEBARgyJAhKsuGDBmCgIAAjR5/8eJFeHh4YPfu3ZVe24MHD+Dh4YHVq1dX+raJ9FVJ70lDoVAosHr1agQGBsLLywseHh66LqnS7N69Gx4eHrh48WKV7G/btm3o3LkzvL294eHhgQcPHlTJfuPi4jB27Fi0atXqlfpbR6qMdV1AZbp48SLef//9Uu//73//i2bNmlXKvr799lvY2Nigd+/eL1x327ZtmD9/PmbOnIlhw4aVut706dOxZ88e7NixA2+99Val1FkVHjx4gKioKHTo0AGenp66LueFJk2ahJ9++gmtWrXC1q1bdV0OaejBgwcIDAwEAIwZMwaTJ09WWycgIAAWFhaIjo6u6vJeGVFRUVizZg369OmDFi1aQCot+3t+QEAAHj58WOr94eHhCAkJqewy9d6FCxcwf/58BAYG4oMPPoCxsTEcHByqZN8zZ87En3/+iTFjxqB69eqoXbt2lex3yJAh+PXXX1WW2dnZoXbt2ujduzf69esHIyMjrex79erV8PT0RIcOHTR+TPEvJ0ZGRnB0dISHhweGDRsGf3//yi6zUr1SAU4pODgY7777rtryynwRf/fdd3B3d9cowHXv3h1hYWHYvXt3qQEuMzMThw8fRr169SolvG3evPmlt6Gphw8fYs2aNXB3d1cLcO7u7rhx44bW3rTllZaWhmPHjqF27dq4ePEiHjx4gJo1a+q6LCqnrVu3YtCgQXB2dtZ1Ka+cc+fOwdraGl988QUkEolGj3F1dcWUKVNKvM+QvoxWpnPnzgEAFi1aBDs7uyrbb15eHi5fvozBgwdj5MiRVbZfJVNTUyxcuBAAIAgCUlJScODAAcydOxexsbGYPXu2Vva7Zs0a9OrVq1wBDgA8PT0xfPhwAEBBQQEePnyIH3/8ESNHjsTq1avRqVMnbZRbKV7JAOfl5aVX3/hsbGzQsWNHREdH4/fff0ejRo3U1vnpp5/w7Nkz9OnTp1L2aWpqWinbeVkSiQRmZma6LkO0b98+FBQUYPny5ejfvz92796NCRMm6LqsF8rMzISVlZWuy9AL3t7euHnzJtasWYP58+fruhydEwQB2dnZsLS0rJTtJScnw8bGRuPwBgDW1tZ69ZmrD5KTkwGg0sNbTk4OjI2NYWxc8p/vx48fQxAE2NraVup+Ac0+h4yNjdVeC4MHD0ZgYCCioqIqNcDl5+dDoVC81N8YFxcXtXo7deqEkJAQREVF6XWAe23nwJXk4MGDGDNmDNq1awdvb2/4+flh7Nix+OOPP1TW8/DwwMOHD/Hrr7/Cw8ND/ClrfsN7770HAIiMjCzx/sjISBgbG6Nnz54ACoddR4wYgXfeeQfe3t7w9/fH1KlTNZ5DUdocuGPHjqFnz55o3Lgx2rZtixUrVqCgoEBtvczMTCxfvhx9+/aFn58fvL290bFjRyxZsgTPnj0T19u9e7c4bD1z5kzxd6Gc/1PaHLiCggJs2LABXbt2RePGjeHn54ePP/4Yf/75p8p6RR9/8uRJ9OnTB40bN4a/vz/CwsJKrL0sERERaNmyJby9vdGuXTtERUVBoVCUuO6FCxcwevRo+Pn5oXHjxggMDMSsWbOQmpqqst7hw4cxZMgQ+Pr6omnTpggKCsLChQuRl5cn/o5Km3dTUjsp50/dunULI0eOhI+PD3r06AFA83ZREgQBu3btQt++fdG8eXM0b94c3bt3x8qVKwEAR48ehYeHB3bt2lXi76Bbt27o2LEjSrvinlwuh7+/P3r16lXi/Tt37oSHhweOHTsGAMjNzcXq1asRFBSEpk2bwtfXV+yh1lTTpk3RsWNHREZG4u+//37h+qXNRytp7qeyrc6fP481a9agffv2aNKkCfr27Ytr164BAH799VcMGDAAzZo1g7+/P9auXVvqvn///Xe8//77aN68OVq2bInp06cjJSVFbb28vDysX78e3bp1Q+PGjeHr64sxY8bg1q1bpda8bds28f3zzTffvPD38OOPP6JXr15o0qQJfHx8MGLECFy+fFlt2xcvXsTDhw/F93Jlzp9Svt7j4uLw0UcfwcfHB2+99RY+/vhjxMXFqa2fnZ2NpUuXokOHDvD29kabNm0wbdq0EodsX/RaL0qhUGDz5s3idoOCghAVFaW23qlTpzB48GD4+fmhSZMmaNeuHcaNG4e7d++W+hyVn1nK11Xxz0QA+OOPP/Dxxx+Lny1du3bFxo0bIZfLVbY1Y8YMeHh4IDU1FTNnzkTr1q3RrFkzJCQklLjvGTNmoH379gAKe6SU+1Z+9lTkc/fgwYPo3bs3mjRpIvaslZeZmRns7OxgYmKidl9SUhLmzJkj/t319/fHZ599pvY+Wb16NTw8PHD79m18+eWXePfdd9GkSRMcOHBAHAqNiopS+ZtcUcqe/eL1njlzBpMmTUJgYCCaNGkCX19fjBgxQm3YGABu376NCRMmiH/D27RpgyFDhuDUqVMq62n63i/JK9kD9+zZM7U/sqampi/85vDDDz/Azs4O/fr1g5OTE+7fv49du3ZhwIABiIqKQt26dQEUzun48ssvYW9vjzFjxoiPL2t+Q6tWrVCzZk1ER0djxowZKj1kd+/exdWrVxEYGIjq1asDAL755hs0a9YMQ4YMgZ2dHf766y9ERETgwoUL2L9/P+zt7cv7a8HRo0cxfvx4uLu74+OPP4aRkRF2796Nn3/+WW3dxMREREREoFOnTggODoaxsTF+/fVXbNq0CTExMeIQbYsWLTBmzBisX78eoaGh8PHxAQDxeZRm6tSp+Omnn9CmTRsMGDAAjx8/xrZt29C/f39s27YNXl5eKuv//PPP2L59O/r3748+ffrg+PHj+Oabb2Bra6vSBmW5ceMG/vrrL3z11VcAgF69euHo0aM4d+6c2lyHnTt3Yu7cuXBxcUH//v3h7u6OR48e4eTJk0hMTBTbevny5Vi/fj0aNGiAYcOGia+bI0eOYMKECRXuCX306BGGDh2Kzp07o1OnTsjOzgagebso/d///R/279+Ppk2bYsyYMbC2tsbff/+Nw4cPY+LEiWjfvj2cnJwQGRmJfv36qTz22rVruHPnDiZPnlxqb4yRkRF69OiBzZs34/bt23jzzTdV7t+zZw/s7e3Rtm1bAMC8efMQGRmJnj17onnz5pDL5fjnn3/KPal8ypQpOHHiBJYtW4Y1a9aU67GaWLJkCRQKBd5//33k5+fjm2++wYgRIxAeHo5PP/0U/fr1Q/fu3fHTTz9h1apVqFmzptq3+ISEBAwbNgydOnVCUFAQbt26hcjISNy8eRMRERGoVq0agMJehJEjR+Lq1asICQnBoEGDkJmZKX72/PDDD2jcuLHKtrdu3Yr09HT07dsXTk5OcHV1LfP5LF68GJs2bUKTJk0wZcoUcftDhw7FunXr0LZtW9SvXx/h4eFYv3490tLSMHPmTACaTT2Ry+Vqn7lK9vb2Kq+f7OxsDBkyRKzl3r172L59O65fv46oqCg4OTmp/F6uXLmCoKAgDB8+HPfu3cOOHTtw9uxZREZGqjzvF73Wi1q+fDlycnIQGhoKU1NT7NixAzNmzEDt2rXFz7Bff/0VH330Ed588018+OGHsLa2RlJSEs6fP4/79+/jjTfeKPH5Ojg4IDw8HLt27cLly5cRHh4O4N/PxP/9738YMmQIjI2NMWjQIFSvXh0nT57EkiVL8Mcff2Dp0qVq2xw+fDiqV6+OsWPHIjs7GxYWFiXuOzQ0FA0bNsSXX36Jjh07omPHjgCA+vXrAyj/5+6xY8fw/fffY8CAAejfv7/GowBFXwupqamIiorC7du31T6rHz16hNDQUOTn5+O9995D7dq1xTa+ePEiIiMjYW1trfKYqVOnwtzcHCNGjABQOOIWHh6OadOmwdfXV+1z7EUKCgrEeuVyOR49eoR169bByMhI7HhRioqKQkZGBnr27AlXV1ckJibixx9/xLBhw/Ddd9/B19cXQOFUnaFDhwIA+vfvjxo1aiAtLQ03b97E9evX0a5dOwAVe++rEF4hFy5cEGQyWYk/kyZNUllXJpMJ06dPV1mWlZWlts07d+4IjRo1EubMmaOyvH379sLgwYPLVd+aNWsEmUwmHDhwQGX5kiVLBJlMJhw7dqzMWs6dOyfIZDJhw4YNL6xl8ODBQvv27cXbBQUFQtu2bYWWLVsKKSkp4vInT54I7dq1E2QymRAZGSkuz83NFfLy8tRqWL58uSCTyYTr16+Ly5S/96KPV4qLixNkMpmwatUqcdmZM2cEmUwmTJw4UVAoFOLymJgYwdPTUxgwYIDa45s2bSrExcWJyxUKhdCtWzehTZs2avsszeeffy40a9ZMyMzMFARBEPLy8oRWrVoJEydOVFkvPj5eaNSokdClSxchIyNDbTtyuVwQBEG4fv26IJPJhCFDhgg5OTkq6ygUCvG5RUZGCjKZTLhw4YLatoq3kyAUtqdMJhN27dqltn552uXAgQOCTCYTpk6dKtZc/DkIgiAsXbpUkMlkwu3bt1XW+fTTTwVPT08hISFBbX9F/fXXX4JMJhPCwsJUlt+7d0+QyWTCggULxGUtWrQQRo0aVeb2SqN8LcybN08QBEGYPXu2IJPJhKtXr4rrtG/fXujWrZvK40p7r5b0ulW2Vc+ePYXc3Fxx+bFjxwSZTCZ4eXkJN27cEJfn5uYKbdq0Efr166e2T5lMJmzZskVl+ZYtWwSZTCZ8/fXXast++eUXlXWfPn0qtG3bVqV2Zc0tWrQQHj9+XNqvSkVsbKzg4eEh9O/fX+U5JSQkCD4+PkL79u2FgoICcXlJr8myKJ9raT9FP28GDx4syGQyYeHChSrbOHLkiCCTyYTPPvtMXPbf//63xNfVyZMnxde1kqavdWX7hoSEqP0uGjVqJEyePFlctmjRIkEmk2n8ey5u+vTpgkwmU1seGhoqeHp6CjExMeIyhUIhTJgwQZDJZMK5c+fUtvHJJ59ovN+SPnMFoWKfu15eXsKdO3c03reyfYv/eHp6qtUjCIIwZswYoVWrVkJ8fLzK8hs3bqg9ZtWqVYJMJhMGDx4s5Ofnq22rpL/pL1Laa7ZFixYqf4+VSvq7nJycLLRs2VLlc035eVH8b31x5Xnvl+SVHEINDQ3Fli1bVH4++uijFz5O+a1GEARkZmYiNTUV9vb2eOONN3Djxo2Xrqt3796QSqUqQzZyuRx79uyBk5OT2EtRtBaFQoGnT58iNTUVHh4esLa2rlAtv//+O+Lj49G7d2+VnkJra2v0799fbX1TU1Ox+7igoAAZGRlITU1F69atAQDXr18vdw1KR48eBVB4JGHRb+YNGzZE+/bt8dtvv6l9mw8MDFQ52EAikcDPzw/JycnIysp64T5zcnJw4MABdOrUSZwrZGJigu7du+P48eNIT08X1z106BDy8/Mxbtw42NjYqG1LeVTevn37AACffPKJ2hwMiURSrjlExdnZ2ZV4gEx52mX//v0ACo9uLn4kYdHbffv2hUQiQUREhLgsOzsbBw8exLvvvgsXF5cya33zzTfRqFEj7N+/X2U4es+ePQAgTgsAACsrK9y5cwd//fVXmdvUxLhx41CtWjUsXrz4pbdV3IABA1R6T5XfrJs0aaLyjdjU1BSNGzfGP//8o7YNKysrDBw4UGXZwIEDYWVlJb4HgMLXUb169dCoUSOkpqaKP3l5eWjdujV+++035OTkqGwnJCQEjo6OGj2X48ePQxAEjBo1SuU5ubi4oHfv3nj48KFGwzVlcXd3V/vMVf4U70EBgNGjR6vc7tixI9544w0cP35cXHb06FFIpVJ8+OGHKuu2a9cOnp6eOH78uPh60/S1rjRw4EC138Ubb7yh0o7Kug8fPlzuqRqlSUlJwdWrVxEQEICGDRuKyyUSifg3quhrQ6kyDkaoyOeusme2PMzMzFTaf/HixQgMDMSaNWtUesufPn2KU6dOISAgAKampiqvfXd3d9SuXRtnz55V2/7QoUNLnf9XEU2bNhVr3bRpExYsWIAaNWpgypQpOH36tMq6RXs+s7KykJaWBqlUiqZNm6r8XVa+dk6fPo3MzMxS912R935Rr+QQap06dcQ/aOVx69YtrFy5Er/++qs4ZKVUGUcqurm5oU2bNjh79iwSExPh4uKCM2fOICkpCaNGjVJ5UZ4/fx7r1q3D9evXkZubq7KdjIyMcu9bOb+kXr16aveV9gbdtm0bdu7ciTt37qjNE6tIDUoPHjyAVCotcb8NGjTAsWPH8ODBA5WgWatWLbV1lZOD09PTXziB+9ChQ3j69ClatGiBe/fuictbtGiBrVu3Yt++feJcPuWH+ItOiXLv3j1IJBKVD+LKUqtWrVKP3NW0Xe7duwcnJ6cXDmfXqlULrVu3xt69e/HJJ5/AxMQEP/30E7KystSGEErTq1cvLFy4UByOFgQB+/btw5tvvglvb29xvVmzZmHatGno3r07atWqBT8/P7Rv3x4BAQEvPF1FcS4uLhg6dCjWr1+PEydOaHzeQ00Uf70pJ4SX9Dlga2ur8gWg6DaKD6GbmpqiVq1aKvO9YmNjkZOTg7fffrvUetLS0uDm5ibeVk7n0IRy3mzx4e2iy+Li4soeqnkBCwsLjT9zbWxsxGHSourXr49jx46JQ4QPHjyAs7NziZPxGzRogJiYGKSlpcHR0VHj17pSaZ8nRefWDRo0CMePH8e8efOwZMkS+Pj44J133kFwcHCFTweibIsGDRqo3VevXj1IpdIS5wKWp73L2nd5P3crsl8jIyO110KPHj0watQorFmzBp07d0aDBg1w9+5dKBQKREREqHx5LKqkdipvTenp6cjPz1dZVvT1Z29vr1Zv165d0alTJ8yePRvHjh0TvzTfv38fy5cvx5kzZ/DkyROVxxQNxS1btkTPnj2xe/du7N+/H97e3mjdujW6du2q0vYVee8X9UoGuIp49OgRBg0aBCsrK3z00UeoV68eqlWrBolEgkWLFqkFuorq06cPTp8+jaioKIwZM0Y8qKHo0ac3btzAyJEjUbt2bXzyySeoWbMmzM3NIZFIMHny5FInlFemLVu24KuvvoK/vz/ef/99ODs7w8TEBImJiZgxY0aV1FBUWach0aQW5e/5008/LfX+ss4hWBpNetrKur+0b/bK+VHFaatd+vXrh4kTJ+LEiRMICgpCREQEnJycxLkaL9KtWzeEhYVhz5498Pf3x2+//Ya4uDhMnTpVZb0OHTrgxIkT+Pnnn3Hp0iWcO3cOERER8PX1xZYtW8o9Z/CDDz7Azp07sXz5co1rVSo+Ybyo0sKkNk6HIwgCZDKZOOesJMUDQ2mvD9KMJl8W7O3tERERgcuXL+PcuXO4dOkSvvzyS6xevRobNmxA8+bNq6DSQrpq78rcr7+/P06fPo2LFy+iQYMG4mdVjx49Sj0IqqSjS83Nzcu13/Hjx6sdZFD8oI3irKys0KxZMxw/fhz37t1DgwYNkJWVhUGDBuHZs2cYOnQoZDIZLC0tIZVK8fXXX+PChQsq2wgLC8PIkSPxyy+/4PLly9iyZQvWr1+PWbNmYfDgwQAq9t4vigHuuaNHjyI7Oxv/+c9/0KpVK5X70tPTK+20HIGBgbCzs0NUVBRCQ0Nx4sQJvPXWWyo9Y9HR0ZDL5di4caPKN5Ds7Gy11K8p5XZKOmovNjZWbdnevXvh7u6OjRs3qnzY/fLLL2rrlneosFatWlAoFIiNjVXrvVLWUpnnZrt//z4uXbqE7t27l3iOoPPnz2Pnzp24efMmvL29xW94MTExpU5UBgq/Cf7yyy/4448/0KRJk1LXU/YglNRr+eDBgxKPzCpNedqlbt26OH78OB4/fvzCnonAwEA4OjoiIiICb775Jq5cuSKefFQTDg4OePfdd3Hs2DFkZWVhz549kEql4tGzRdnZ2SEkJAQhISEQBAFLlizBpk2bcPz4cXTp0kWj/Skpv3B9+eWXJR5FqNxfST1kJfV0VKa4uDjk5eWpfHbk5eUhLi5O5f1ep04dpKWloVWrVuXuhdSE8r1/+/ZttQMS7ty5o7JOVXjy5AmSk5PVeuFiY2Ph6OgoDlPVqlULp0+fxpMnT9SmMsTGxsLKyko8mKs8r/XyMDIygp+fH/z8/AAUHj3ap08f/Oc//8GGDRvKvT3l55ry917U33//DYVCobW2qOrP3eKUX1aVU15q164NiUSC/Pz8Co2YaWr69OkV+rtZvN7z588jKSkJixYtUjvd14oVK0rchkwmg0wmw6hRo/DkyRP07dsXS5cuxaBBgyCRSF76vf9KzoGrCOU36+I9GLt27RLP51OUpaVliX8UXsTU1BQhISH4559/MHfuXPHom5JqKe7rr78u9ZQXL9KoUSO4urpi9+7dKvMcMjMzsXPnTrX1pVIpJBKJyu+joKAAGzduVFtX+YGr6bCqMkRt2LBBZft//fUXTpw4AR8fn0o9Y3lkZCQEQcDw4cPRuXNntZ8PPvhAXA8AOnfuDBMTE6xdu7bE+QvKmrt37w4AWLZsmXjKkJLWUwZC5Yk9laKjo5GUlFSu51KedlHWt3jxYrXXTfHXuYmJCXr16oUzZ86Ip8XQdPhUqVevXnj27Bn27duHQ4cOoXXr1irz5+RyeYnDDsoj3yo6LD9w4EC4u7tj9erVJbZD3bp1cffuXSQmJorL8vLysG3btgrtT1OZmZnYvn27yrLt27cjMzNT5YtEz549kZycjC1btpS4ncePH79UHQEBAZBIJNi8ebPKUFJSUhJ2794Nd3d3taMPta14+Dl69Cju3r2r8nvp0KEDFAqF2ro///wzbt26pTLsXp7XuqZKOqq2Xr16MDMzq/Br1dHREc2bN8fJkydV5oEKgiA+T+WRo5Wtqj93ixIEQZzfqJxSoTw6/ejRo+Ipeoo/prQjm0tiYWFR4t9k5fBl0Z8XSU1NxdWrV2FmZiYOOZeWEc6cOaM2Jzw9PV3tdWhjY4OaNWvi2bNn4rSol33vswfuuXfffRfVqlXDtGnTMHjwYNjY2ODKlSv45ZdfULt2bbXhlqZNmyIiIgIrVqxA/fr1IZVK0b59+1IP7y7qvffew9atW3Ho0CFYWFio9Tp06NAB3377LT744AOEhobCxMQEZ8+exZ9//lmh04cAhS++mTNnYtKkSejbt694SZPIyEjY2dnh0aNHKut37twZS5cuxQcffICOHTsiMzMT0dHRJfbINGjQAJaWlti+fTvMzc1hY2MDBweHUsf127Rpgy5duuDAgQPIyMhA+/btkZycjO3bt8PMzKxST/Qol8vFP1IlnUAZKPzW2ahRI/EUL66urpg1axbmz5+P7t27IyQkBO7u7khMTMTx48exaNEieHp6okmTJvjggw+wceNG9O7dG126dIGTkxMePHiAw4cP48cff4SNjQ3q1auH1q1b47///S8EQYCnpydiYmJw7Ngx1KlTp1wTpMvTLl26dMGRI0ewZ88e3Lt3DwEBAbCxscE///yDM2fOqF1uql+/fti8eTOio6PRsmXLcs81adu2Lezs7LBkyRJkZmaqDYtkZWXB398fAQEB8PLygoODAx48eIAdO3bA1tZWPH9VeZmammLixImYNm0aAPUTpw4aNAgHDhzAsGHD0L9/f+Tn52Pv3r1aH5aqXbs21q5di9u3b6NRo0b4/fffERkZiXr16qmcE+z999/HuXPnEB4ejgsXLqBVq1awsrLCo0ePcOHCBZiamuL777+vcB316tXDyJEjsWnTJgwePBhdunRBVlYWdu3ahezsbCxZsuSlh4afPn2KvXv3lnifh4eHSo+Pvb09jh49iqSkJLRs2VI8jUj16tUxbtw4cb1evXohKioKGzduxMOHD+Hr64v79++L6xa98kN5X+ua+Oyzz5CQkAB/f3/UqFEDOTk54tzQlzlp8aeffoohQ4Zg0KBBGDhwIJycnHDy5EmcOXMGwcHBZc6HehlV9blbUFCg8lpITU3FkSNHcOXKFfj7+6s8v7lz52LgwIEYPHgwQkJC4OXlBYVCgbi4OBw/fhw9e/bE+PHjNdpvs2bNcP78eWzYsAE1atSARCJBt27dXvi4xMREsV6FQoFHjx4hMjIST548weTJk8VTp/j4+MDJyQlhYWF4+PAhXF1dERMTg71790Imk6kE8j179mDr1q3o0KED6tSpA2NjY1y6dAlnzpxBly5dxGHgl33vM8A9V7t2bWzcuBHLli3D+vXrYWRkhLfeegvff/89FixYoHbiyMmTJyMjIwPbt2/HkydPxG8YmgQ4mUyGJk2a4MaNG+jSpYvaY3x8fLB69WqsW7cOK1euhJmZGVq3bo0ffvhBHDuviM6dO2PVqlVYu3YtVq9eDUdHR/Tq1QstWrQQz6mjNHLkSAiCgIiICHzxxRdwcnJCly5d0KdPH3Tt2lVlXXNzcyxfvhwrVqzAokWLkJeXh5YtW5b5QbRkyRJ4eXkhKioKX331FSwsLNCiRQtMnDixUi+effr0aSQlJYmXSilNUFAQli1bhiNHjqB79+4YOHAgateujc2bN+P7779HXl4enJ2d8fbbb6uce2rq1Klo2LAhfvjhB2zatAmCIMDV1RXvvvuuylyN8PBwLFiwAPv378e+ffvg4+OD7777DnPnzi3zOpLFladdAGDp0qXw9fVFREQE1q5dC6lUipo1a6Jz585q69apUwd+fn64cOFCha4IYmpqiuDgYPzwww+wsrJSG642NzfH0KFDcf78eZw/fx5ZWVlwdnZGQEAAPvzwwxce7VqWHj16YMuWLYiJiVG7z8fHB1999RXWr1+PxYsXw9nZGQMGDIC3t3eZ1yZ+Wa6urlixYgXCwsJw4MAB8ajn6dOnq7znTUxM8PXXX2P79u3Yu3eveNJrZ2dnNG7cuNT5QeXxf//3f6hTpw62b9+OpUuXwsTEBE2bNhVfHy8rISFBDNDFjRkzRiXAWVhYYOvWrVi0aBGWLl0KQRDwzjvvYMaMGSqXRjMxMcHmzZvxn//8BwcPHsTRo0dhbW2Nzp07Y9KkSWoTu8vzWtdESEgIdu/ejaioKKSmpsLKygoNGjTAqlWrEBQUVKFtAkDjxo2xc+dOrFq1Cjt27EB2djZq1aqFqVOnqn0OV7aq+NzNy8tTeS2YmZmhTp06mDx5MkaMGKEy5cbNzQ2RkZHYuHEjTpw4gX379sHMzAxubm5o3759uaZUzJkzB/Pnz8f69evFYU9NAlxMTIxKvZaWlvD09MQnn3yi8ngbGxts2rQJixcvxg8//ICCggJ4e3tj48aNiIiIUAlwfn5+iImJwalTp5CcnCy+FqdPn67yN/xl3/sSoapnoxOR3vrggw9w7do1nD59utyThYleZMiQIXj48CFOnDih61KIDB7nwBERgMLTjpw5cwY9evRgeCMi0nNVEuDCwsIQEBAADw8PlW7Gu3fvIjQ0FEFBQQgNDVU5iWJZ9xFR5bl+/Tp2796NSZMmwcTE5IXDzUREpHtVEuACAwOxbds2uLu7qyyfM2cOBg4ciMOHD2PgwIH4/PPPNbqPiCrPjh07MGvWLGRmZmLJkiVaPZUAERFVjiqdAxcQEID169dDJpMhJSUFQUFBuHjxIoyMjCCXy+Hn54cjR45AEIRS79PWYc5EREREhkJnR6HGx8fDxcVFPHzdyMgIzs7OiI+PhyAIpd7HAEdERESvOx7EQERERGRgdNYD5+bmhsTERMjlcnGYNCkpCW5ubhAEodT7yistLQsKhfZGiR0drZCSon62ftIttov+YZvoH7aJfmK76J+qaBOpVAJ7e0uN19dZgHN0dISnpyeio6MREhKC6OhoeHp6ikOkZd1XHgqFoNUAp9wH6R+2i/5hm+gftol+YrvoH31rkyo5iGHhwoU4cuQIHj9+DHt7e9jZ2eHAgQOIjY3FjBkzxIsVh4WFiRd5Luu+8khJydTqL93JyRrJyU+1tn2qGLaL/mGb6B+2iX5iu+ifqmgTqVQCR0crjdd/5a/EwAD3emK76B+2if5hm+gntov+0ccAx4MYiIiIiAwMAxwRERGRgWGAIyIiIjIwDHBEREREBoYBjoiIiMjAMMARERERGRgGOCIiIiIDwwBHREREZGAY4IiIiIgMDAMcERERkYFhgCMiIiIyMAxwRERERAaGAY6IiIjIwDDAERERERkYBjgiIiIiA8MAR0RERGRgGOCIiIiIDAwDHBEREZGBYYAjIiIiMjAMcEREREQGhgGOiIiIyMAwwBEREREZGAY4IiIiIgPDAEdERERkYBjgiIiIiAwMAxwRERGRgWGAIyIiIjIwDHBEREREBoYBjoiIiMjAMMARERERGRgGOCIiIiIDwwBHREREZGAY4IiIiIgMDAMcERERkYFhgCMiIiIyMAxwRERERAaGAY6IiIjIwDDAERERERkYBjgiIiIiA8MAR0RERGRgGOCIiIiIDAwDHBEREZGBYYAjIiIiMjAMcEREREQGhgGOiIiIyMAwwBEREREZGL0IcCdPnkTPnj0REhKCHj164MiRIwCAu3fvIjQ0FEFBQQgNDcU///yj20KJiIiI9ICxrgsQBAHTpk3Dtm3bIJPJ8Mcff2DAgAHo0KED5syZg4EDByIkJAR79+7F559/ju+++07XJRMRERHplF70wEmlUjx9+hQA8PTpUzg7OyMtLQ23bt1CcHAwACA4OBi3bt1CamqqLkslIiIi0jmd98BJJBKsWLECY8eOhYWFBbKysrBhwwbEx8fDxcUFRkZGAAAjIyM4OzsjPj4eDg4OGm/f0dFKW6WLnJystb4PKj+2i/5hm+gftol+YrvoH31rE50HuIKCAnz99ddYt24dfHx88Ntvv2HSpEkIDw+vlO2npGRCoRAqZVslcXKyRnLyU61tnyqG7aJ/2Cb6h22in9gu+qcq2kQqlZSr00nnQ6gxMTFISkqCj48PAMDHxwfVqlWDmZkZEhMTIZfLAQByuRxJSUlwc3PTZblEREREOqfzAOfq6oqEhAT8/fffAIDY2FikpKSgTp068PT0RHR0NAAgOjoanp6e5Ro+JSIiInoV6XwI1cnJCXPnzsXEiRMhkUgAAIsWLYKdnR3mzp2LGTNmYN26dbCxsUFYWJiOqyUiIiLSPYkgCNqbIKYHOAfu9cR20T9sE/3DNtFPbBf9wzlwRERERPTSGOCIiIiIDAwDHBEREZGBYYAjIiIiMjAaBbi0tDRt10FEREREGtIowLVv3x4fffQRDh06hLy8PG3XRERERERl0CjAnThxAm+//TY2btwIf39/fPbZZ7h8+bK2ayMiIiKiEmgU4BwcHPD+++8jMjISO3fuhIODA6ZNm4bAwECsXLkSDx8+1HadRERERPRcuQ9iePz4MR4/foysrCzUrl0biYmJ6NWrFzZs2KCN+oiIiIioGI0upXX79m3s27cP0dHRqFatGnr27Im9e/fC1dUVADB27Fj06NEDo0eP1mqxRERERKRhgBs8eDC6deuGlStXokmTJmr316xZE0OHDq304oiIiIhInUYB7syZMzAxMSlznYkTJ1ZKQURERERUNo3mwIWFheHKlSsqy65cuYIvvvhCK0URERERUek0CnDR0dHw9vZWWebt7Y3o6GitFEVEREREpdMowEkkEgiCoLJMLpdDoVBopSgiIiIiKp1GAc7X1xcrVqwQA5tCocDq1avh6+ur1eKIiIiISJ1GBzF8+umn+PDDD+Hv748aNWogPj4eTk5OWL9+vbbrIyIiIqJiNApwrq6uiIqKwvXr15GQkAA3Nzc0adIEUmm5zwNMRERERC9JowAHAFKpFM2bN9dmLURERESkAY0CXGZmJlavXo1Lly4hLS1N5YCGU6dOaas2IiIiIiqBRmOgc+fOxa1btzB27Fikp6dj9uzZcHNzw7Bhw7RcHhEREREVp1EP3NmzZ3Hw4EHY29vDyMgIHTp0QOPGjTFmzBiGOCIiIqIqplEPnEKhgLW1NQDAwsICT58+hZOTE+7du6fV4oiIiIhInUY9cA0bNsSlS5fw9ttvw9fXF3PnzoWlpSXq1q2r5fKIiIiIqDiNeuAWLlwId3d3AIXnhDM3N8eTJ08QHh6u1eKIiIiISN0Le+Dkcjl2796Njz76CADg6OjIi9gTERER6dALe+CMjIywfft2GBtrfMo4IiIiItIijYZQe/bsiR07dmi7FiIiIiLSgEbdajdu3MAPP/yAzZs3w9XVFRKJRLxv27ZtWiuOiIiIiNRpFOD69euHfv36absWIiIiItKARgGuV69e2q6DiIiIiDSkUYCLiIgo9b733nuv0oohIiIiohfTKMDt3btX5fbjx48RFxeH5s2bM8ARERERVTGNAtz333+vtiwiIgKxsbGVXhARERERlU2j04iUpHfv3oiMjKzMWoiIiIhIAxr1wCkUCpXbz549w759+8QL3BMRERFR1dEowHl5eamc+w0AXFxcMH/+fK0URURERESl0yjAHT9+XOV2tWrV4ODgoJWCiIiIiKhsGgU4Y2NjmJubw9bWVlyWkZGBnJwcuLi4aK04IiIiIlKn0UEMY8eORUJCgsqyhIQEjBs3TitFEREREVHpNApwd+/ehYeHh8oyDw8P/P3331opioiIiIhKp1GAc3R0xL1791SW3bt3D3Z2dtqoiYiIiIjKoFGA69OnD8aPH4+TJ0/izp07OHHiBCZMmIC+fftquz4iIiIiKkajgxhGjx4NY2NjhIWFISEhAW5ubnjvvfcwfPhwbddHRERERMVoFOCkUilGjRqFUaNGabseIiIiInoBjYZQN2zYgBs3bqgsu3HjBjZu3FgpReTm5mLOnDno1KkTunfvjs8++wxA4cEToaGhCAoKQmhoKP75559K2R8RERGRIdMowH333Xdo0KCByrL69etj69atlVLE4sWLYWZmhsOHD2P//v2YOHEiAGDOnDkYOHAgDh8+jIEDB+Lzzz+vlP0RERERGTKNAlx+fj6MjVVHW01MTJCXl/fSBWRlZWHPnj2YOHGieLmu6tWrIyUlBbdu3UJwcDAAIDg4GLdu3UJqaupL75OIiIjIkGk0B65Ro0bYvn07hg0bJi7buXMnvLy8XrqAuLg42NnZYc2aNbh48SIsLS0xceJEmJubw8XFBUZGRgAAIyMjODs7Iz4+vlyX8XJ0tHrpGl/Eycla6/ug8mO76B+2if5hm+gntov+0bc20SjAzZw5E8OHD8e+fftQq1YtxMXFITk5GVu2bHnpAuRyOeLi4uDl5YXp06fj+vXrGDNmDFauXPnS2waAlJRMKBRCpWyrJE5O1khOfqq17VPFsF30D9tE/7BN9BPbRf9URZtIpZJydTppFODefPNNHD58GKdOnUJ8fDw6deqEdu3awdLSssKFKrm5ucHY2FgcKm3atCns7e1hbm6OxMREyOVyGBkZQS6XIykpCW5ubi+9TyIiIiJDptEcOACwtLREt27dMGrUKLRp0wZ79uzBe++999IFODg4wM/PD2fPngVQeORpSkoK6tatC09PT0RHRwMAoqOj4enpWa7hUyIiIqJXkUY9cABQUFCAU6dOYc+ePfj555/h6uqK0NDQSili3rx5mDVrFsLCwmBsbIzw8HDY2Nhg7ty5mDFjBtatWwcbGxuEhYVVyv6IiIiIDNkLA9zNmzexZ88eREdHQy6Xo2PHjjAzM8POnTvh6OhYKUXUqlUL33//vdry+vXr48cff6yUfRARERG9KsoMcMHBwYiLi0Pbtm0xf/58tGvXDqampvjll1+qqj4iIiIiKqbMOXDPnj2DVCqFmZkZzM3NYWJiUlV1EREREVEpyuyBO378OC5duoSoqChMnjwZZmZm6NKlC3Jzc8WT7hIRERFR1XrhUagtWrTAokWLcPbsWUyfPh13795FVlYWhgwZgm3btlVFjURERERUhManETE3N0dISAi++eYbnDx5Ej169GCAIyIiItIBjQNcUS4uLvjwww9x8ODByq6HiIiIiF6gQgGOiIiIiHSHAY6IiIjIwDDAERERERkYjS+lpaRQKFRuS6XMgERERERVSaMA9/vvv2P+/Pn4888/kZubCwAQBAESiQQxMTFaLZCIiIiIVGkU4GbMmIH27dtj0aJFMDc313ZNRERERFQGjQLcw4cPMXnyZF59gYiIiEgPaDSBrWPHjjhz5oy2ayEiIiIiDWjUA5ebm4tx48bBx8cH1atXV7kvPDxcK4URERERUck0CnANGjRAgwYNtF0LEREREWlAowA3btw4bddBRERERBrS+DxwFy9exJ49e5CUlARnZ2eEhISgVatW2qyNiIiIiEqg0UEMP/74IyZNmgQnJyd07NgRzs7O+OSTT7Br1y5t10dERERExWjUA7dp0yZs2bIFDRs2FJd16dIFEyZMQL9+/bRWHBERERGp06gHLj09HfXr11dZVq9ePWRkZGilKCIiIiIqnUYB7q233sJXX32FZ8+eAQCys7MRHh6O5s2ba7U4IiIiIlKn0RDqvHnzMHnyZPj6+sLW1hYZGRlo3rw5li5dqu36iIiIiKgYjQKcs7Mztm3bhvj4eCQnJ8PZ2Rmurq7aro2IiIiISlBqgBMEQbz2qUKhAAC4uLjAxcVFZZlUqtEoLBERERFVklIDnI+PD65cuQIA8PLyUruQvTLgxcTEaLdCIiIiIlJRaoA7cOCA+P/jx49XSTFERERE9GKljn+6ubmJ/z906BDc3d3Vfo4cOVIlRRIRERHRvzSawLZ27doSl//nP/+p1GKIiIiI6MXKPAr1/PnzAAoPWLhw4QIEQRDve/DgASwtLbVbHRERERGpKTPAffrppwCA3NxczJo1S1wukUjg5OSE2bNna7c6IiIiIlJTZoA7ceIEAGDatGkIDw+vkoKIiIiIqGwazYFjeCMiIiLSHxpdiSEzMxOrV6/GpUuXkJaWpjIX7tSpU9qqjYiIiIhKoFEP3Ny5c3Hr1i2MHTsW6enpmD17Ntzc3DBs2DAtl0dERERExWnUA3f27FkcPHgQ9vb2MDIyQocOHdC4cWOMGTOGIY6IiIioimnUA6dQKGBtbQ0AsLCwwNOnT+Hk5IR79+5ptTgiIiIiUqdRD1zDhg1x6dIlvP322/D19cXcuXNhaWmJunXrark8IiIiIipOox64hQsXwt3dHUDhueHMzc3x5MkTHp1KREREpAMa9cDVqlVL/L+joyO++OILrRVERERERGUrNcBFRERotIH33nuv0oohIiIiohcrNcDt3btX5faVK1dQvXp1uLm5IT4+Ho8fP8Zbb73FAEdERERUxUoNcN9//734/wULFiAwMFDllCFbt25FXFycVosjIiIiInUaHcSwb98+DBkyRGXZ4MGD1XrpiIiIiEj7NApw1atXFy9sr3Ty5Ek4ODhopSgiIiIiKp1GR6HOnj0b48ePx+bNm+Hq6or4+HjcuXMHK1eurNRi1qxZg9WrV2P//v2QyWS4du0aPv/8c+Tm5sLd3R2LFy+Go6Njpe6TiIiIyNBo1APXpk0bHDt2DP3794eXlxcGDBiAY8eOwd/fv9IK+f3333Ht2jXxfHMKhQL/93//h88//xyHDx+Gr68vlixZUmn7IyIiIjJUGvXAAYCDgwN69uyplSLy8vIwf/58LF26FO+//z4A4ObNmzAzM4Ovry8AoH///ggMDMSXX36plRqIiIiIDEWpAW7kyJHYvHkzAGDgwIGQSCQlrrdt27aXLmLlypXo0aMHatasKS6Lj49HjRo1xNsODg5QKBRIT0+HnZ3dS++TiIiIyFCVGuCK9rb17dtXawVcvXoVN2/exNSpU7WyfUdHK61stygnJ2ut74PKj+2if9gm+odtop/YLvpH39qk1ADXvXt38f+9evXSWgGXLl1CbGwsAgMDAQAJCQkYOXIkhgwZgkePHonrpaamQiqVlrv3LSUlEwqFUJklq3ByskZy8lOtbZ8qhu2if9gm+odtop/YLvqnKtpEKpWUq9NJ55fSGj16NEaPHi3eDggIwPr169GgQQPs2rULly9fhq+vL3bu3InOnTu/1L6IiIiIXgUaX0qrJBKJRGuX0pJKpQgPD8ecOXNUTiNCRERE9LqTCIKgvfFFPcAh1NcT20X/sE30D9tEP7Fd9I9BDaGWRhAEFM18UqlGp5IjIiIiokqiUYBLTEzE/PnzcfnyZTx58kTlvpiYGK0URkREREQl06j7bM6cOTAxMcG3334LCwsLREVFISAgAPPmzdN2fURERERUjEY9cFevXsXJkydhYWEBiUSChg0b4osvvkD//v3Rr18/bddIREREREVo1AMnlUphbFyY9WxsbJCamgoLCwskJiZqtTgiIiIiUqdRD1zTpk3x888/o2PHjvD398ekSZNgbm4Ob29vbddHRERERMWUGeDu3LmDBg0aIDw8HAqFAgAwa9YsfPPNN8jKysLQoUOrpEgiIiIi+leZAa5nz57w8PBAr1690K1bNwCAubk5xo4dWyXFEREREZG6MufAnT59Gr169cLevXvx7rvv4uOPP8axY8dQUFBQVfURERERUTEaX4nh77//xt69e7F//35kZ2ejW7duCAkJQZMmTbRd40vhlRheT2wX/cM20T9sE/3EdtE/+nglBo0vo1CvXj1MnjwZJ06cwNKlS3Hy5EmEhoZWqEgiIiIiqrhyXUrr2rVr2LNnD3766SdYW1vj448/1lZdRERERFSKFwa4hw8fYu/evdi7dy9SUlIQFBSEtWvXwtfXtyrqIyIiIqJiygxwgwcPxrVr1+Dn54dx48ahY8eOMDc3r6raiIiIiKgEZQa4d999F0uXLoWLi0tV1UNEREREL1BmgBs9enRV1UFEREREGtL4KFQiIiIi0g8McEREREQGRqMAl5ycXK7lRERERKQ9GgW4oKCgEpcrr49KRERERFVHowBX0tW2MjMzIZFIKr0gIiIiIipbmUehtm3bFhKJBLm5uWjXrp3Kfenp6eyBIyIiItKBMgPc4sWLIQgCRo8ejfDwcHG5RCKBo6Mj6tWrp/UCiYiIiEhVmQGuZcuWAIALFy6gWrVqVVIQEREREZVNo4vZV6tWDTExMbh8+TLS0tJU5sRNnDhRa8URERERkTqNDmL473//iwEDBuDChQvYuHEj/vrrL2zZsgX379/Xdn1EREREVIxGAW7Tpk3YtGkT1q5dC3Nzc6xduxYrV66EsbFGHXhEREREVIk0CnApKSnw9fUtfIBUCoVCgbZt2+LkyZNaLY6IiIiI1GnUhebq6ooHDx6gZs2aqFu3Lo4fPw57e3uYmJhouz4iIiIiKkajADdq1CjExsaiZs2aGDt2LCZOnIj8/Hx8+umn2q6PiIiIiIrRKMD17t1b/H/btm3x66+/Ij8/H5aWllorjIiIiIhKVmqAUygUpT/I2BjGxsZQKBSQSjWaRkdERERElaTUAOfl5VXmtU4FQYBEIkFMTIxWCiMiIiKikpUa4I4fP16VdRARERGRhkoNcO7u7khOToaTk1NV1kNEREREL1DmBLagoCCV2+PGjdNqMURERET0YmUGuKLXPAWAX3/9VavFEBEREdGLlRngyjqIgYiIiIh0o8zzwMnlcly4cEHsiSsoKFC5DQBvv/22diskIiIiIhVlBjhHR0fMmjVLvG1nZ6dyWyKR8GhVIiIioipWZoA7ceJEVdVBRERERBrS6FJaRERERK+bXxOuYF/sIaTnpsPOzA496ndGS9e3dF0WAAY4IiIiIjW/JlzB9j8ika/IBwCk5aZj+x+RAKAXIY4XMiUiIiJC4enT0nLScSvlT+z6a68Y3pTyFfnYF3tIR9WpYg8cERERvVYEQUBG3hPEZyYiPisB8VlJiM9KREJ2Ip4V5JT52LTc9Kop8gUY4IiIXjP6PK+HqDKJQS0rEQlZSc/DWiLis1SDmpWJJdwsXdDC5S24WbrAzdIZ397aifTcDLVt2pvZVeEzKJ3OA1xaWhqmTZuG+/fvw9TUFHXq1MH8+fPh4OCAa9eu4fPPP0dubi7c3d2xePFiODo66rpkIiKDpe/zeogqQhAEPMl7Koazoj/PCp6J6ymDmq9L8+dBrfDH2tRKbZsh9buovFcAwERqgh71O1fJc3oRiVD8ellVLD09HX/++Sf8/PwAAGFhYcjIyMDChQsRFBSEL7/8Er6+vli3bh3i4uLw5Zdflmv7KSmZUCi09xSdnKyRnPxUa9unimG76B+2iW7lFOQgNScdK69+jcz8LLX7rUwsMfmtj+Bgbg9TIxMdVEhKfK+UThAEPM3PfD70majSo5ZdJKhZGlvA1dIFblaFAa2GpQvcLF1LDGplqcreaqlUAkdHzevTeYAr7vDhw9ixYwemTJmCWbNmITo6GgCQmpqKwMBAXL16tVzbY4B7PbFd9A/bRLtyCnKRmpOG1Jw0PM5JReqzNKTkpCE1JxUpOWnIys/WeFs2ptZwNLeHg7k9HKs5FP6r/L+ZHUwY8LSK75VCT/MyEZ+VgEfK3rTMwjlqRV/LFsbVivSkuRb+a+UCaxOrSr0caFW0SXkDnM6HUItSKBTYsWMHAgICEB8fjxo1aoj3OTg4QKFQID09HXZ2dhpvszy/jIpycrLW+j6o/Ngu+odtUnE5Bbl4nJWKpKwUJGelICnrMZKzUgv/n52Cp7mZKuubGJnA2cIRTpYO8HCuBydLRzhbOuLbqz8iPeeJ2vZtzWwwpFlvJD/ffnJ2Ch5kPcK1xzchV8hV1rU3t4WTZeG2ldt1tqwOJ0tHVLewZ8CrBK/Te+VJzlPEPYlHXMYjPMiIx4Mn8Yh7Eq/ymrY0qYaatjXwdq23UNPWDbVsa6CmjRvszG2q7Lrt+tYmehXgFixYAAsLCwwePBhHjx6tlG2yB+71xHbRP2yTsuXJ85CaU9hrlvKseE9aqtqwp7HUWOwla+rYCI7mDnCoZv98mQNsTEvugQip17XEeT0963eFp6UXPC1V11cICmTkPnleV+q/Neak4c+kv3E+9woUgkLlMbamNnCsVlhbdbGuwp48e3M7mEj16k+P3nlV3yuZeVlFhjz/PaCg6Gu7mrE53Cxd0MTRC26WrnC1dIabpQtsTdWDWkEm8Dgzs/hutII9cGUICwvDvXv3sH79ekilUri5ueHRo0fi/ampqZBKpeXqfSMi0hd58nwx/KTmpKqFtKf5qn+IjCVG4hBmTWvvwiFMc3s4VHOAo7k9rE2tIJWU/1Seyvk7ms7rkUqksDe3g725HRrYvaF2v0JQID03Q3w+Kc+HbFOfpeFuxn1cSbqhEvAkkMDWzOb5sKwDHMXQWXjb3twWxgx4Bi0zP0sc7lQOfcZnJaq8xs2Nnge16o3gZulcOPxpVXJQo5Lpxbtk2bJluHnzJjZs2ABTU1MAgLe3N3JycnD58mX4+vpi586d6NxZP478ICIqLr9IQEt5PhdN2WP1OCcVT/NUA5qRxAgO5nZwNHdAEycvOJg7PJ9nVhhmbEytKxTQNNHS9S20dH2rUnoVpBIpHJ4HsJLIFXKk5z4R5+Ipw11KTipiM+7icuJVCPh3lEQZ8FTD3b+/G3szOxhJjV6qZqocWfnZJRz1maDyWjc3MoOrpQu8q3uqHPVpZ2bLoPaSdH4Qw+3btxEcHIy6devC3NwcAFCzZk2sXbsWV65cwZw5c1ROI1K9evVybZ9DqK8ntov+MfQ2yVcUFB4k8KxIL5M43JmKjDzV52YkMYK9uZ3Yc/bvwQCFwUSbAU1T+tAmhQEvo0i4Uwa9wl7K9NwMtYBnZ2b7PNwVPcCi8Ladma3BBzx9aJeisvOfqR3xGZ+ViCdFXvNmRqaFR30WOaCgxisU1PRxCFXnAU7bGOBeT2wX/aPvbZKvKEBaTjpSihzBmZKTKoa0jDzVif9SiRQOZnbikKZjsSM2bc1sdB7QXkTf2wQoDHhpuRnisHPRcJeaox7wpBJpYcATh2ULh52rP+/JszOz0fuAp6t2eVbwTGXIU/lT9LVvamQKNwsX8WhPN0sXuFq4wMHc7pUIaqXRxwCnF0OoRETaVqAoQFpOxvPes6IhrTAIZOQ+UQsC9maFPWiejrLnIc0Bjs8DmyEEtFeBkdQI1as5oHo1B6CEUdoCRYE4B0813KXiz7Q7pbSrrdgb6lDt+YEWz3vx7MxsX/l2LQxqhQcRJDy/hFR8VqLKVQdMpSZwtXRBQ4c3VYY+7c3tXvnfj6FggCOiV0JhT0368z/kqUWO5iz8f1l/yBvavykewansSbM11f+eGio8Grd6NUdUr1byVXrKCu5/pN1GRkLpwV013BlecH9WkIOEYkd8Fg9qJlITuFk6Q2ZfX2X404FBTe8xwBGRQSjvUJoEEnEOmod9A5UjOB3N7V+JuVL0YsZSYzhZOMLJouSApxw6Vx50UnToPCblrzKHzpXDssoDT3QV8HIKcguP+Cw29Fn0ousmUmO4WjjjTbv6hVcleD786WBuz6BmoBjgiEirNL0UzctMZpfZ11eZzO5g7gB7BjTSgInUGM4W1eFsUfIBcvnyfLWeXeW8yN9T/ij14JWiw7KaHrzyovdKTkEuErOTnl+ZIEGcr1Y8qLlYOKOB3RsqPWqO1RjUXjU8iOElGcIk4NcR20U/FL9wOgAYS4zh5+YDWzMblSM603MzSjxfmGOR4auip5OwM+P5wl4W3ycvr6zTx6TkpKkcqQn8e34/hyJfNhzN7ZGUnYyj939Wea8YSYzQ0P5NQAIkZCUiJSft3+1IjeFi4aR6CSlLF1Sv5sCgpgX6eBADA9xL4gegfmK7VIwgCJALcuTJ85Arz0OeIv/f/5d0W56PPEXh7XzxMXnIlecjX56H+08fQF7sLP1Kqid0LRrSeELXqsL3ifaVdgJn5ZUtip/AuSTuVm4qBxK4WrqgurkDe5irkD4GOH46EhkYuUIuBqXCMJX/PFzlieEqX56P3OK3xXWKhDCFMog9344iT+2ySC9iJDGCqZEpTKUmMDMyLfy/kSnMjMxKDW8AsLzdF7ykEr3yTI1M4GrpDFdL5xLvV15CbcHFpaVuY1bLydoqjwwYPz3plaLpfCttkivkKj1V+Yp8scdK7KkqEqTyxNslhat8td4wuSB/cRFFSCXSwmAlNRHDlanUFGZGprA2tYKpkYl4u/D+wvXMpMVuG5nCRGoKsyK3TaWmZfYCzD67SGV+jpK9Ga+HSQQUnlfN1dIF9mZ2pb5XiErCT1B6ZRSfb5WWm47tf0QCgEqIUwgKsccpX1HycKBKmCqhlypPnlcsfP0bxgrKGbAkkBTrufo3bFmaWKr1bJkWCVHiY0oIW8pt6HIYskf9ziVeOL1HfV4Wj6govleovBjgyKDlFOQg9fnZ83/8a6/Khx8A5Cvy8X3MLuyLPSQOM+YrCsq1DwkkJfRKFf7YmVSDadFlamGr2G0jE7VtGEuMXtkzmJf3wulEryu+V6i8eBDDS+IkYO3KKcgtvP7k8wuCK0/CqTw5a1Z+tkbbaeXmKw75KcOY6rBh8bD1b2Azlhq/sgGrKvG9on/YJvqJ7aJ/eBADUTG5zyfwKg+7Vw1pacjMz1JZ30RqIp4hvY5N7X+PWqxmj43/+17lDONK9mZ2GOLZr6qeEhERkdYxwJFWKY+w+veyRkVDWqpaQDOWGovXnKxtU1M8a77ybOfWJlal9oaF1O/COSRERPRaYICjl1LaOY6UIa34OY6MpcZwMLeDo7kDall7ixeTVoY0a1PLCp+EknNIiIjodcEAR2Uq6yzjj3NS8TSvWEArcpbxJk5ecDB/fr3A5ydptTa10upZwlu6voWWrm9xDgkREb3SGOBec/mKgsKDBEq4zl9qTmqJ1/lT9qA1dvQSr/NXvVrhdf/Kus4fERERVQ4GuFdcvqIAac9Ps6E8OCAlJ1UMaRl5T1TWl0qk4qWNGjk2FOeeKUMaAxoREZHuMcAZuAJFAdJyMp73nhUNaYU9aRm5TyDg39OoSCVS2JvZwdHcHl6OHkWO4iwc4rQ1s2FAIyIi0nMMcHpOrpAjLTcdKUWGOJXDmymlBjRbOJjbo6H9m3CsVnhEp/JUG7amNrwAMhERkYFjgNOxwoCWIR7BqRrS0pCem6ES0CSQwN68sAfNw75B4Wk2nveeOZg7wM6MAY2IiOhVxwBXQZpeNF2ukCM9N+PfYc1nqeI8tJRnJQc0OzNbOFazh8y+vjgfTdmTZmdmy4BGRET0mmOAq4CSLpq+LSYCd9LvwtbMRuWIzvTcDCgEhfhYZUBzMLfHm/b1/j1J7fOQZm9mx4BGREREZWKAq4B9sYfULppeIBTg7KOLkEACWzMbOJjbo77tG897zp4fKGDuAHtzWxhL+WsnIiKiimOSqIC03PRS71ve7guYMKARERGRFvF8ERVgb2ZX6nKGNyIiItI2BrgK6FG/M0ykJirLeNF0IiIiqirsLqoAXjSdiIiIdIkBroJ40XQiIiLSFQ6hEhERERkYBjgiIiIiA8MAR0RERGRgGOCIiIiIDAwDHBEREZGBYYAjIiIiMjAMcEREREQGhgGOiIiIyMAwwBEREREZGAY4IiIiIgPDAEdERERkYBjgiIiIiAwMAxwRERGRgWGAIyIiIjIwDHBEREREBoYBjoiIiMjAMMARERERGRi9D3B3795FaGgogoKCEBoain/++UfXJRERERHplN4HuDlz5mDgwIE4fPgwBg4ciM8//1zXJRERERHplLGuCyhLSkoKbt26hS1btgAAgoODsWDBAqSmpsLBwUGjbUilEm2WWGX7oPJju+gfton+YZvoJ7aL/tF2m5R3+3od4OLj4+Hi4gIjIyMAgJGREZydnREfH69xgLO3t9RmiQAAR0crre+Dyo/ton/YJvqHbaKf2C76R9/aRO+HUImIiIhIlV4HODc3NyQmJkIulwMA5HI5kpKS4ObmpuPKiIiIiHRHrwOco6MjPD09ER0dDQCIjo6Gp6enxsOnRERERK8iiSAIgq6LKEtsbCxmzJiBJ0+ewMbGBmFhYahXr56uyyIiIiLSGb0PcERERESkSq+HUImIiIhIHQMcERERkYFhgCMiIiIyMAxwRERERAZGr6/EoM/CwsJw+PBhPHz4EPv374dMJtN1Sa+9tLQ0TJs2Dffv34epqSnq1KmD+fPn87QzOjZ27Fg8ePAAUqkUFhYW+Oyzz+Dp6anrsgjAmjVrsHr1an6G6YmAgACYmprCzMwMADB16lS88847Oq6KcnNzsWjRIpw/fx5mZmZo1qwZFixYoOuyGOAqKjAwEO+//z4GDRqk61LoOYlEglGjRsHPzw9AYchesmQJFi1apOPKXm9hYWGwtrYGABw7dgyzZs1CVFSUjqui33//HdeuXYO7u7uuS6EiVq1axTCtZxYvXgwzMzMcPnwYEokEjx8/1nVJADiEWmG+vr68IoSesbOzE8MbADRr1gyPHj3SYUUEQAxvAJCZmQmJhBfp1rW8vDzMnz8fc+fO1XUpRHotKysLe/bswcSJE8XPrurVq+u4qkLsgaNXkkKhwI4dOxAQEKDrUgjAp59+irNnz0IQBGzatEnX5bz2Vq5ciR49eqBmzZq6LoWKmTp1KgRBgI+PD6ZMmQIbGxtdl/Rai4uLg52dHdasWYOLFy/C0tISEydOhK+vr65LYw8cvZoWLFgACwsLDB48WNelEIAvvvgCp06dwuTJkxEeHq7rcl5rV69exc2bNzFw4EBdl0LFbNu2Dfv27UNkZCQEQcD8+fN1XdJrTy6XIy4uDl5eXti9ezemTp2K8ePHIzMzU9elMcDRqycsLAz37t3DihUrIJXyJa5PevbsiYsXLyItLU3Xpby2Ll26hNjYWAQGBiIgIAAJCQkYOXIkzpw5o+vSXnvKaTmmpqYYOHAgrly5ouOKyM3NDcbGxggODgYANG3aFPb29rh7966OK2OAo1fMsmXLcPPmTaxduxampqa6Lue1l5WVhfj4ePH2iRMnYGtrCzs7O90V9ZobPXo0zpw5gxMnTuDEiRNwdXXF5s2b4e/vr+vSXmvZ2dl4+vQpAEAQBBw8eJBHa+sBBwcH+Pn54ezZswCAu3fvIiUlBXXq1NFxZbwWaoUtXLgQR44cwePHj2Fvbw87OzscOHBA12W91m7fvo3g4GDUrVsX5ubmAICaNWti7dq1Oq7s9fX48WOMHTsWz549g1Qqha2tLaZPn45GjRrpujR6LiAgAOvXr+eRjzoWFxeH8ePHQy6XQ6FQoH79+pg9ezacnZ11XdprLy4uDrNmzUJ6ejqMjY0xadIktG3bVtdlMcARERERGRoOoRIREREZGAY4IiIiIgPDAEdERERkYBjgiIiIiAwMAxwRERGRgWGAIyIiIjIwDHBE9MqYMWMGli9frpN9C4KAmTNnokWLFnjvvfd0UkNxq1evxtSpU3VdBhFpAS9mT0RaExAQgGfPnuH48eOwsLAAAPz444/Yt28fvv/+ex1XV7l+++03nD17Fj///LP4XIvavXs3Pv30U/Ek00qHDh2Ci4tLVZVJRK8IBjgi0iqFQoHvvvsOY8aM0XUp5SKXy2FkZKTx+g8fPoS7u3uJ4U2pWbNm2LFjR2WUR0SvOQ6hEpFWjRw5Et988w2ePHmidt+DBw/g4eGBgoICcdmQIUPw448/Aijsterfvz8WLVoEX19fBAYG4sqVK9i9ezfatm2Lt99+G1FRUSrbTEtLw/Dhw9G8eXMMHjwYDx8+FO+LjY3F8OHD0bJlSwQFBeHgwYPifTNmzMCcOXPwwQcfoFmzZrh48aJavYmJiRgzZgxatmyJjh07YteuXQAKexVnz56Na9euoXnz5li1alW5f08BAQH4+uuv0bVrV7Ro0QIzZ85Ebm6ueP+uXbvQsWNHtGzZEmPGjEFiYqJ43+3bt8Xn1bp1a6xfv168Lz8/H9OmTUPz5s3RrVs3/O9//xPv27BhA9555x00b94cQUFBOH/+fLnrJiLdYIAjIq3y9vZGy5YtsXnz5go9/saNG/Dw8MDFixcRHByMKVOm4H//+x+OHj2KxYsXY/78+cjKyhLX379/P8aOHYuLFy+iYcOG4hyw7OxsjBgxAsHBwTh37hyWL1+OefPm4c6dO+Jjo6OjMWbMGFy5cgU+Pj5qtUyZMgWurq44ffo0Vq1ahWXLluH8+fPo27cv5s2bh2bNmuHq1auYMGFChZ7r/v37sXnzZhw9ehR3797FunXrAADnz5/H0qVLsWLFCpw5cwbu7u6YMmUKACAzMxPDhw/HO++8g9OnT+PIkSN4++23xW2eOHEC3bp1w+XLlxEQEIAFCxYAAP7++29s27YNERERuHr1KjZv3gx3d/cK1U1EVY8Bjoi0bsKECfjhhx+Qmppa7sfWrFkTffr0gZGREbp27Yr4+Hh8/PHHMDU1hb+/P0xNTXH//n1x/Xbt2qFFixYwNTXF5MmTce3aNcTHx+PUqVNwd3dHnz59YGxsDC8vLwQFBeHQoUPiYwMDA+Hj4wOpVAozMzOVOuLj43HlyhVMnToVZmZm8PT0RN++fbF3716Nn8v169fh6+sr/nTo0EHl/kGDBsHNzQ12dnb46KOPcODAAQCFwa5Pnz5o1KgRTE1NMWXKFFy7dg0PHjzAqVOnUL16dYwYMQJmZmawsrJC06ZNxW36+Pigbdu2MDIyQkhICP744w8AgJGREfLy8hAbG4v8/HzUrFkTtWvX1rxhiEinOAeOiLROJpOhXbt22LBhA+rXr1+uxzo6Oor/Vx4AUL16dXGZmZmZSg+cq6ur+H9LS0vY2toiKSkJDx8+xI0bN+Dr6yveL5fL0aNHD/G2m5tbqXUkJSXB1tYWVlZW4rIaNWrg5s2bGj+Xpk2bljkHruj+a9SogaSkJHHfjRo1UnlednZ2SExMRHx8fJnBq+jvytzcHLm5uSgoKECdOnUwa9YsrF69Gnfu3IG/vz9mzJjBAyqIDAR74IioSkyYMAG7du1SmbulnPCfk5MjLktOTn6p/SQkJIj/z8rKQkZGBpydneHm5oYWLVrg8uXL4s/Vq1cxb948jbbr7OyMjIwMZGZmisvi4+MrNfDEx8eL/3/06BGcnZ3FfRedy5ednY309HS4uLjAzc0NcXFxFdpf9+7dsWPHDpw8eRISiQRLlix5uSdARFWGAY6IqkSdOnXQtWtXldOHODg4wMXFBXv37oVcLkdERESFw4jSzz//jMuXLyMvLw8rV65E06ZN4ebmhnbt2uGff/7Bnj17kJ+fj/z8fNy4cQOxsbEabdfNzQ3NmzfHsmXLkJubiz/++AMREREqPXgva/v27UhISEB6ejrWr1+Prl27AgCCg4Oxe/duxMTEIC8vD8uWLUOTJk1Qs2ZNtGvXDsnJyfj222+Rl5eHzMxMXL9+/YX7+vvvv3H+/Hnk5eXB1NQUZmZmkEr5J4HIUPDdSkRV5uOPP0Z2drbKsgULFmDz5s3w8/PDnTt30Lx585faR3BwMNauXQs/Pz/8/vvvWLx4MQDAysoKmzdvxsGDB/HOO+/A398fS5YsQV5ensbbXrZsGR4+fIh33nkH48aNw/jx49G6dWuNH688SrXoz40bN1RqHzFiBDp06IDatWvjo48+AgC0bt0aEydOxPjx4+Hv74+4uDjxhMVWVlb45ptvcPLkSbRp0wZBQUElHkFbXF5eHpYuXQo/Pz/4+/sjNTVVPDCCiPSfRBAEQddFEBG97gICArBw4cJyBUIien2xB46IiIjIwDDAERERERkYDqESERERGRj2wBEREREZGAY4IiIiIgPDAEdERERkYBjgiIiIiAwMAxwRERGRgfl/PnCePORQaVYAAAAASUVORK5CYII=\n","text/plain":["<Figure size 720x360 with 1 Axes>"]},"metadata":{},"output_type":"display_data"}],"source":["fig, ax = plt.subplots(figsize=(10, 5));\n","ax.plot(num_epochs, np.array(val_flat_accuracy_list),'go-', label=\"Flat Validation Accuracy\")\n","ax.set_xlabel(\"Number of Epochs\")\n","ax.set_ylabel(\"Flat Validation Accuracy\")\n","ax.set_title(\"Flat Validation Accuracy vs Number of Epochs for for Bert-Base\",fontsize=18)\n","ax.set_ylim(0, 100)"]},{"cell_type":"code","execution_count":31,"id":"33c8a48b","metadata":{"execution":{"iopub.execute_input":"2023-10-25T20:00:05.973879Z","iopub.status.busy":"2023-10-25T20:00:05.973283Z","iopub.status.idle":"2023-10-25T20:00:06.000993Z","shell.execute_reply":"2023-10-25T20:00:06.000055Z"},"papermill":{"duration":0.059763,"end_time":"2023-10-25T20:00:06.003066","exception":false,"start_time":"2023-10-25T20:00:05.943303","status":"completed"},"tags":[]},"outputs":[{"data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>Title</th>\n","      <th>abstractText</th>\n","      <th>meshMajor</th>\n","      <th>pmid</th>\n","      <th>meshid</th>\n","      <th>meshroot</th>\n","      <th>A</th>\n","      <th>B</th>\n","      <th>C</th>\n","      <th>D</th>\n","      <th>...</th>\n","      <th>F</th>\n","      <th>G</th>\n","      <th>H</th>\n","      <th>I</th>\n","      <th>J</th>\n","      <th>L</th>\n","      <th>M</th>\n","      <th>N</th>\n","      <th>Z</th>\n","      <th>one_hot_labels</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>35083</th>\n","      <td>Expression of N-methyl-d-aspartate receptor 1 ...</td>\n","      <td>High levels of glutamate can be toxic to retin...</td>\n","      <td>['Analysis of Variance', 'Animals', 'Cell Deat...</td>\n","      <td>17942238</td>\n","      <td>[['E05.318.740.150', 'N05.715.360.750.125', 'N...</td>\n","      <td>['Analytical, Diagnostic and Therapeutic Techn...</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>...</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>[1, 1, 1, 1, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0]</td>\n","    </tr>\n","    <tr>\n","      <th>9005</th>\n","      <td>Protection of pregnant swine by vaccination ag...</td>\n","      <td>The protection conferred on pregnant gilts by ...</td>\n","      <td>['Animals', 'Antibodies, Bacterial', 'Bacteria...</td>\n","      <td>7150130</td>\n","      <td>[['B01.050'], ['D12.776.124.486.485.114.107', ...</td>\n","      <td>['Organisms [B]', 'Chemicals and Drugs [D]', '...</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>...</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>[0, 1, 1, 1, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0]</td>\n","    </tr>\n","    <tr>\n","      <th>23836</th>\n","      <td>An examination of Escherichia coli strains iso...</td>\n","      <td>Ninety-five strains of Escherichia coli isolat...</td>\n","      <td>['Animals', 'Antigens', 'Antigens, Bacterial',...</td>\n","      <td>6135266</td>\n","      <td>[['B01.050'], ['D23.050'], ['D23.050.161'], ['...</td>\n","      <td>['Organisms [B]', 'Chemicals and Drugs [D]', '...</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>...</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>[1, 1, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","<p>3 rows × 21 columns</p>\n","</div>"],"text/plain":["                                                   Title  \\\n","35083  Expression of N-methyl-d-aspartate receptor 1 ...   \n","9005   Protection of pregnant swine by vaccination ag...   \n","23836  An examination of Escherichia coli strains iso...   \n","\n","                                            abstractText  \\\n","35083  High levels of glutamate can be toxic to retin...   \n","9005   The protection conferred on pregnant gilts by ...   \n","23836  Ninety-five strains of Escherichia coli isolat...   \n","\n","                                               meshMajor      pmid  \\\n","35083  ['Analysis of Variance', 'Animals', 'Cell Deat...  17942238   \n","9005   ['Animals', 'Antibodies, Bacterial', 'Bacteria...   7150130   \n","23836  ['Animals', 'Antigens', 'Antigens, Bacterial',...   6135266   \n","\n","                                                  meshid  \\\n","35083  [['E05.318.740.150', 'N05.715.360.750.125', 'N...   \n","9005   [['B01.050'], ['D12.776.124.486.485.114.107', ...   \n","23836  [['B01.050'], ['D23.050'], ['D23.050.161'], ['...   \n","\n","                                                meshroot  A  B  C  D  ...  F  \\\n","35083  ['Analytical, Diagnostic and Therapeutic Techn...  1  1  1  1  ...  0   \n","9005   ['Organisms [B]', 'Chemicals and Drugs [D]', '...  0  1  1  1  ...  0   \n","23836  ['Organisms [B]', 'Chemicals and Drugs [D]', '...  1  1  1  1  ...  0   \n","\n","       G  H  I  J  L  M  N  Z                              one_hot_labels  \n","35083  1  0  0  0  0  0  1  0  [1, 1, 1, 1, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0]  \n","9005   1  0  0  0  0  0  1  0  [0, 1, 1, 1, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0]  \n","23836  1  0  0  0  0  0  0  0  [1, 1, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0]  \n","\n","[3 rows x 21 columns]"]},"execution_count":31,"metadata":{},"output_type":"execute_result"}],"source":["df_test['one_hot_labels'] = list(df_test[mesh_Heading_categories].values)\n","df_test.head(3)"]},{"cell_type":"code","execution_count":32,"id":"8cd16fdc","metadata":{"execution":{"iopub.execute_input":"2023-10-25T20:00:06.060694Z","iopub.status.busy":"2023-10-25T20:00:06.059966Z","iopub.status.idle":"2023-10-25T20:00:06.066265Z","shell.execute_reply":"2023-10-25T20:00:06.065373Z"},"papermill":{"duration":0.037174,"end_time":"2023-10-25T20:00:06.068265","exception":false,"start_time":"2023-10-25T20:00:06.031091","status":"completed"},"tags":[]},"outputs":[],"source":["test_labels = list(df_test.one_hot_labels.values)\n","Articles_test = list(df_test.abstractText.values)\n","test_mesh_categories = list(df_test.columns[6:20])"]},{"cell_type":"code","execution_count":33,"id":"73ff3530","metadata":{"execution":{"iopub.execute_input":"2023-10-25T20:00:06.124912Z","iopub.status.busy":"2023-10-25T20:00:06.12462Z","iopub.status.idle":"2023-10-25T20:01:45.657116Z","shell.execute_reply":"2023-10-25T20:01:45.655871Z"},"papermill":{"duration":99.564967,"end_time":"2023-10-25T20:01:45.660012","exception":false,"start_time":"2023-10-25T20:00:06.095045","status":"completed"},"tags":[]},"outputs":[],"source":["# Encoding input data\n","test_encodings = tokenizer.batch_encode_plus(Articles_test,max_length=max_length,padding=True,truncation=True)\n","test_input_ids = test_encodings['input_ids']\n","test_attention_masks = test_encodings['attention_mask']"]},{"cell_type":"code","execution_count":34,"id":"4e653790","metadata":{"execution":{"iopub.execute_input":"2023-10-25T20:01:45.721763Z","iopub.status.busy":"2023-10-25T20:01:45.72088Z","iopub.status.idle":"2023-10-25T20:01:46.12946Z","shell.execute_reply":"2023-10-25T20:01:46.128602Z"},"papermill":{"duration":0.441523,"end_time":"2023-10-25T20:01:46.131827","exception":false,"start_time":"2023-10-25T20:01:45.690304","status":"completed"},"tags":[]},"outputs":[],"source":["# Make tensors out of data\n","test_inputs = torch.tensor(test_input_ids)\n","test_labels = torch.tensor(test_labels)\n","test_masks = torch.tensor(test_attention_masks)\n","# Create test dataloader\n","test_data = TensorDataset(test_inputs, test_masks, test_labels,)# test_token_types)\n","test_sampler = SequentialSampler(test_data)\n","test_dataloader = DataLoader(test_data, sampler=test_sampler, batch_size=batch_size)\n","# Save test dataloader\n","torch.save(test_dataloader,'test_data_loader')"]},{"cell_type":"markdown","id":"4dffd91c","metadata":{"papermill":{"duration":0.026765,"end_time":"2023-10-25T20:01:46.18609","exception":false,"start_time":"2023-10-25T20:01:46.159325","status":"completed"},"tags":[]},"source":["<a id=\"9\"></a>\n","## <p style=\"background-color:#1a0a36;font-family:newtimeroman;color:#FFF9ED;font-size:150%;text-align:center;border-radius:10px 10px;\"> Evaluating the model</p>\n","#### [Top ↑](#top) "]},{"cell_type":"code","execution_count":35,"id":"31630d64","metadata":{"execution":{"iopub.execute_input":"2023-10-25T20:01:46.242243Z","iopub.status.busy":"2023-10-25T20:01:46.241657Z","iopub.status.idle":"2023-10-25T20:02:58.304716Z","shell.execute_reply":"2023-10-25T20:02:58.303512Z"},"papermill":{"duration":72.124867,"end_time":"2023-10-25T20:02:58.33812","exception":false,"start_time":"2023-10-25T20:01:46.213253","status":"completed"},"tags":[]},"outputs":[{"name":"stdout","output_type":"stream","text":["CPU times: user 1min 12s, sys: 35.9 ms, total: 1min 12s\n","Wall time: 1min 12s\n"]}],"source":["%%time\n","\n","# Test\n","\n","# Put model in evaluation mode to evaluate loss on the validation set\n","model.eval()\n","\n","#track variables\n","logit_preds,true_labels,pred_labels,tokenized_texts = [],[],[],[]\n","\n","# Predict\n","for i, batch in enumerate(test_dataloader):\n","    batch = tuple(t.to(device) for t in batch)\n","    # Unpack the inputs from our dataloader\n","    b_input_ids, b_input_mask, b_labels, = batch\n","    with torch.no_grad():\n","        # Forward pass\n","        outs = model(b_input_ids, token_type_ids=None, attention_mask=b_input_mask)\n","        b_logit_pred = outs[0]\n","        pred_label = torch.sigmoid(b_logit_pred)\n","\n","        b_logit_pred = b_logit_pred.detach().cpu().numpy()\n","        pred_label = pred_label.to('cpu').numpy()\n","        b_labels = b_labels.to('cpu').numpy()\n","\n","    tokenized_texts.append(b_input_ids)\n","    logit_preds.append(b_logit_pred)\n","    true_labels.append(b_labels)\n","    pred_labels.append(pred_label)\n","\n","# Flatten outputs\n","tokenized_texts = [item for sublist in tokenized_texts for item in sublist]\n","pred_labels = [item for sublist in pred_labels for item in sublist]\n","true_labels = [item for sublist in true_labels for item in sublist]\n","# Converting flattened binary values to boolean values\n","true_bools = [tl==1 for tl in true_labels]"]},{"cell_type":"markdown","id":"83f0e73d","metadata":{"papermill":{"duration":0.027061,"end_time":"2023-10-25T20:02:58.394134","exception":false,"start_time":"2023-10-25T20:02:58.367073","status":"completed"},"tags":[]},"source":["<a id=\"10\"></a>\n","## <p style=\"background-color:#1a0a36;font-family:newtimeroman;color:#FFF9ED;font-size:150%;text-align:center;border-radius:10px 10px;\"> Classification Report</p>\n","#### [Top ↑](#top)\n"]},{"cell_type":"code","execution_count":36,"id":"8b97b7ad","metadata":{"execution":{"iopub.execute_input":"2023-10-25T20:02:58.45078Z","iopub.status.busy":"2023-10-25T20:02:58.449718Z","iopub.status.idle":"2023-10-25T20:02:58.73581Z","shell.execute_reply":"2023-10-25T20:02:58.734562Z"},"papermill":{"duration":0.317119,"end_time":"2023-10-25T20:02:58.738141","exception":false,"start_time":"2023-10-25T20:02:58.421022","status":"completed"},"tags":[]},"outputs":[{"name":"stdout","output_type":"stream","text":["Test F1 Accuracy:  0.8526193413263993\n","Test Flat Accuracy:  0.1765 \n","\n","              precision    recall  f1-score   support\n","\n","           A       0.84      0.74      0.79      4609\n","           B       0.96      0.99      0.98      9250\n","           C       0.91      0.84      0.88      5206\n","           D       0.92      0.92      0.92      6259\n","           E       0.82      0.96      0.88      7778\n","           F       0.81      0.74      0.78      1767\n","           G       0.83      0.90      0.86      6799\n","           H       0.60      0.13      0.22      1221\n","           I       0.66      0.62      0.64      1068\n","           J       0.77      0.50      0.61      1110\n","           L       0.72      0.44      0.55      1491\n","           M       0.87      0.91      0.89      4232\n","           N       0.83      0.77      0.80      4602\n","           Z       0.73      0.70      0.72      1558\n","\n","   micro avg       0.86      0.84      0.85     56950\n","   macro avg       0.81      0.73      0.75     56950\n","weighted avg       0.86      0.84      0.84     56950\n"," samples avg       0.87      0.85      0.84     56950\n","\n"]}],"source":["pred_bools = [pl>0.50 for pl in pred_labels] #boolean output after thresholding\n","# Print and save classification report\n","Test_F1_Accuracy=f1_score(true_bools, pred_bools,average='micro')\n","Test_Flat_Accuracy= accuracy_score(true_bools, pred_bools)\n","print('Test F1 Accuracy: ',Test_F1_Accuracy )\n","print('Test Flat Accuracy: ',Test_Flat_Accuracy,'\\n')\n","\n","df_test=pd.DataFrame({'Test F1 Accuracy':Test_F1_Accuracy, 'Test Flat Accuracy':Test_Flat_Accuracy},index=[0])\n","\n","print(classification_report(true_bools,pred_bools,target_names=test_mesh_categories))\n","clf_report = classification_report(true_bools,pred_bools,target_names=test_mesh_categories,output_dict=True)\n","df_report=pd.DataFrame(clf_report).transpose()\n","\n"]},{"cell_type":"code","execution_count":37,"id":"0ea13be0","metadata":{"execution":{"iopub.execute_input":"2023-10-25T20:02:58.796504Z","iopub.status.busy":"2023-10-25T20:02:58.795642Z","iopub.status.idle":"2023-10-25T20:02:58.805209Z","shell.execute_reply":"2023-10-25T20:02:58.804392Z"},"papermill":{"duration":0.04073,"end_time":"2023-10-25T20:02:58.807372","exception":false,"start_time":"2023-10-25T20:02:58.766642","status":"completed"},"tags":[]},"outputs":[],"source":["df_report.to_csv('Classification_Report.csv',index=False)"]},{"cell_type":"code","execution_count":38,"id":"681bfc2a","metadata":{"execution":{"iopub.execute_input":"2023-10-25T20:02:58.864642Z","iopub.status.busy":"2023-10-25T20:02:58.864322Z","iopub.status.idle":"2023-10-25T20:02:59.531727Z","shell.execute_reply":"2023-10-25T20:02:59.530647Z"},"papermill":{"duration":0.697957,"end_time":"2023-10-25T20:02:59.534169","exception":false,"start_time":"2023-10-25T20:02:58.836212","status":"completed"},"tags":[]},"outputs":[{"data":{"text/plain":["('./Multi_label_Classification_Save/tokenizer_config.json',\n"," './Multi_label_Classification_Save/special_tokens_map.json',\n"," './Multi_label_Classification_Save/vocab.txt',\n"," './Multi_label_Classification_Save/added_tokens.json')"]},"execution_count":38,"metadata":{},"output_type":"execute_result"}],"source":["model.save_pretrained('./Multi_label_Classification_Save/')\n","tokenizer.save_pretrained('./Multi_label_Classification_Save/')"]},{"cell_type":"code","execution_count":39,"id":"c762fd41","metadata":{"execution":{"iopub.execute_input":"2023-10-25T20:02:59.593172Z","iopub.status.busy":"2023-10-25T20:02:59.592845Z","iopub.status.idle":"2023-10-25T20:03:00.02533Z","shell.execute_reply":"2023-10-25T20:03:00.024463Z"},"papermill":{"duration":0.46368,"end_time":"2023-10-25T20:03:00.027808","exception":false,"start_time":"2023-10-25T20:02:59.564128","status":"completed"},"tags":[]},"outputs":[],"source":["user_secrets = UserSecretsClient()\n","secret_value_0 = user_secrets.get_secret(\"Hugging_Face_model_Push_Secret\") ##Has kept it private. Please use your own token"]},{"cell_type":"code","execution_count":40,"id":"812dd735","metadata":{"execution":{"iopub.execute_input":"2023-10-25T20:03:00.087197Z","iopub.status.busy":"2023-10-25T20:03:00.086868Z","iopub.status.idle":"2023-10-25T20:03:00.095357Z","shell.execute_reply":"2023-10-25T20:03:00.09416Z"},"papermill":{"duration":0.040954,"end_time":"2023-10-25T20:03:00.097669","exception":false,"start_time":"2023-10-25T20:03:00.056715","status":"completed"},"tags":[]},"outputs":[],"source":["#Converting Labels to categorical before pushing it to Hugging Face Hub\n","model.config.label2id= {\n","\"Anatomy [A]\": 0,\n","\"Organisms [B]\": 1,\n","\"Diseases [C]\": 2,\n","\"Chemicals and Drugs [D]\": 3,\n","\"Analytical, Diagnostic and Therapeutic Techniques, and Equipment [E]\": 4,\n","\"Psychiatry and Psychology [F]\": 5,\n","\"Phenomena and Processes [G]\": 6,\n","\"Disciplines and Occupations [H]\": 7,\n","\"Anthropology, Education, Sociology, and Social Phenomena [I]\": 8,\n","\"Technology, Industry, and Agriculture [J]\": 9,\n","\"Information Science [L]\": 10,\n","\"Named Groups [M]\": 11,\n","\"Health Care [N]\": 12,\n","\"Geographicals [Z]\": 13,\n","}\n","\n","\n","model.config.id2label={\n","    \"0\": \"Anatomy [A]\",\n","    \"1\": \"Organisms [B]\",\n","    \"2\": \"Diseases [C]\",\n","    \"3\": \"Chemicals and Drugs [D]\",\n","    \"4\": \"Analytical, Diagnostic and Therapeutic Techniques, and Equipment [E]\",\n","    \"5\": \"Psychiatry and Psychology [F]\",\n","    \"6\": \"Phenomena and Processes [G]\",\n","    \"7\": \"Disciplines and Occupations [H]\",\n","    \"8\": \"Anthropology, Education, Sociology, and Social Phenomena [I]\",\n","    \"9\": \"Technology, Industry, and Agriculture [J]\",\n","    \"10\": \"Information Science [L]\",\n","    \"11\": \"Named Groups [M]\",\n","    \"12\": \"Health Care [N]\",\n","    \"13\": \"Geographicals [Z]\"\n","}\n","    "]},{"cell_type":"code","execution_count":null,"id":"137989b3","metadata":{"papermill":{"duration":0.027419,"end_time":"2023-10-25T20:03:00.152861","exception":false,"start_time":"2023-10-25T20:03:00.125442","status":"completed"},"tags":[]},"outputs":[],"source":[]},{"cell_type":"code","execution_count":41,"id":"c9f2076d","metadata":{"execution":{"iopub.execute_input":"2023-10-25T20:03:00.209891Z","iopub.status.busy":"2023-10-25T20:03:00.209582Z","iopub.status.idle":"2023-10-25T20:03:15.607351Z","shell.execute_reply":"2023-10-25T20:03:15.606406Z"},"papermill":{"duration":15.429077,"end_time":"2023-10-25T20:03:15.609995","exception":false,"start_time":"2023-10-25T20:03:00.180918","status":"completed"},"tags":[]},"outputs":[{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"31ed99b17f984fab8d427c6ed249adcd","version_major":2,"version_minor":0},"text/plain":["pytorch_model.bin:   0%|          | 0.00/433M [00:00<?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"data":{"text/plain":["CommitInfo(commit_url='https://huggingface.co/owaiskha9654/Multi-Label-Classification-of-PubMed-Articles/commit/d61fa2729a91b611cd13bf0a3554d6a7185ca426', commit_message='Upload BertForSequenceClassification', commit_description='', oid='d61fa2729a91b611cd13bf0a3554d6a7185ca426', pr_url=None, pr_revision=None, pr_num=None)"]},"execution_count":41,"metadata":{},"output_type":"execute_result"}],"source":["model.push_to_hub(repo_id='owaiskha9654/Multi-Label-Classification-of-PubMed-Articles',use_auth_token=secret_value_0)"]},{"cell_type":"code","execution_count":42,"id":"ca3dba54","metadata":{"execution":{"iopub.execute_input":"2023-10-25T20:03:15.669955Z","iopub.status.busy":"2023-10-25T20:03:15.669602Z","iopub.status.idle":"2023-10-25T20:03:15.970106Z","shell.execute_reply":"2023-10-25T20:03:15.969104Z"},"papermill":{"duration":0.333172,"end_time":"2023-10-25T20:03:15.972425","exception":false,"start_time":"2023-10-25T20:03:15.639253","status":"completed"},"tags":[]},"outputs":[{"data":{"text/plain":["CommitInfo(commit_url='https://huggingface.co/owaiskha9654/Multi-Label-Classification-of-PubMed-Articles/commit/7a8d76717744b1b7b348862ab08eb7d9d51e07b2', commit_message='Upload tokenizer', commit_description='', oid='7a8d76717744b1b7b348862ab08eb7d9d51e07b2', pr_url=None, pr_revision=None, pr_num=None)"]},"execution_count":42,"metadata":{},"output_type":"execute_result"}],"source":["tokenizer.push_to_hub(repo_id='owaiskha9654/Multi-Label-Classification-of-PubMed-Articles',use_auth_token=secret_value_0)"]},{"cell_type":"code","execution_count":43,"id":"fbcc9338","metadata":{"execution":{"iopub.execute_input":"2023-10-25T20:03:16.032906Z","iopub.status.busy":"2023-10-25T20:03:16.032587Z","iopub.status.idle":"2023-10-25T20:03:16.583814Z","shell.execute_reply":"2023-10-25T20:03:16.583024Z"},"papermill":{"duration":0.583795,"end_time":"2023-10-25T20:03:16.586049","exception":false,"start_time":"2023-10-25T20:03:16.002254","status":"completed"},"tags":[]},"outputs":[{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"7c7640a7196d4769ab36d3f8e430d2d3","version_major":2,"version_minor":0},"text/plain":["Downloading (…)solve/main/vocab.txt:   0%|          | 0.00/213k [00:00<?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"216ed18f243b454e8c1c1f0f223baf5a","version_major":2,"version_minor":0},"text/plain":["Downloading (…)cial_tokens_map.json:   0%|          | 0.00/125 [00:00<?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"5f579d9b57c341dea27617c96a38011a","version_major":2,"version_minor":0},"text/plain":["Downloading (…)okenizer_config.json:   0%|          | 0.00/388 [00:00<?, ?B/s]"]},"metadata":{},"output_type":"display_data"}],"source":["tokenizer = BertTokenizer.from_pretrained('owaiskha9654/Multi-Label-Classification-of-PubMed-Articles', do_lower_case=True) \n"]},{"cell_type":"code","execution_count":44,"id":"357ad13e","metadata":{"execution":{"iopub.execute_input":"2023-10-25T20:03:16.647173Z","iopub.status.busy":"2023-10-25T20:03:16.646873Z","iopub.status.idle":"2023-10-25T20:03:22.731484Z","shell.execute_reply":"2023-10-25T20:03:22.730531Z"},"papermill":{"duration":6.118339,"end_time":"2023-10-25T20:03:22.734206","exception":false,"start_time":"2023-10-25T20:03:16.615867","status":"completed"},"tags":[]},"outputs":[{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"0bd477a12dcc46ef8a4ff1d863e275bf","version_major":2,"version_minor":0},"text/plain":["Downloading (…)lve/main/config.json:   0%|          | 0.00/1.81k [00:00<?, ?B/s]"]},"metadata":{},"output_type":"display_data"},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"3f64dfbac4e24d75ae50eda9ab7854d3","version_major":2,"version_minor":0},"text/plain":["Downloading pytorch_model.bin:   0%|          | 0.00/433M [00:00<?, ?B/s]"]},"metadata":{},"output_type":"display_data"}],"source":["num_labels=14\n","model = BertForSequenceClassification.from_pretrained(\"owaiskha9654/Multi-Label-Classification-of-PubMed-Articles\", num_labels=num_labels)"]},{"cell_type":"markdown","id":"5e47b284","metadata":{"papermill":{"duration":0.029548,"end_time":"2023-10-25T20:03:22.795614","exception":false,"start_time":"2023-10-25T20:03:22.766066","status":"completed"},"tags":[]},"source":["![GradioDemo](https://github.com/Owaiskhan9654/Multi-Label-Classification-of-Pubmed-Articles/blob/25e17ebe22bfc4e004066d7fa0bbcd62eeebce4b/docs/GradioDemo.png?raw=true)"]},{"cell_type":"code","execution_count":45,"id":"14ab7262","metadata":{"execution":{"iopub.execute_input":"2023-10-25T20:03:22.85769Z","iopub.status.busy":"2023-10-25T20:03:22.857304Z","iopub.status.idle":"2023-10-25T20:03:22.866568Z","shell.execute_reply":"2023-10-25T20:03:22.865603Z"},"papermill":{"duration":0.042916,"end_time":"2023-10-25T20:03:22.868721","exception":false,"start_time":"2023-10-25T20:03:22.825805","status":"completed"},"tags":[]},"outputs":[],"source":["# def Multi_Label_Classification_of_Pubmed_Articles(model_input: str) -> Dict[str, float]:\n","    \n","#     # Encoding input data\n","#     dict_custom={}\n","#     Preprocess_part1=model_input[:len(model_input)]\n","#     Preprocess_part2=model_input[len(model_input):]\n","#     dict1=tokenizer.encode_plus(Preprocess_part1,max_length=1024,padding=True,truncation=True)\n","#     dict2=tokenizer.encode_plus(Preprocess_part2,max_length=1024,padding=True,truncation=True)\n","    \n","#     dict_custom['input_ids']=[dict1['input_ids'],dict1['input_ids']]\n","#     dict_custom['token_type_ids']=[dict1['token_type_ids'],dict1['token_type_ids']]\n","#     dict_custom['attention_mask']=[dict1['attention_mask'],dict1['attention_mask']]\n","    \n","#     outs = model(torch.tensor(dict_custom['input_ids']), token_type_ids=None, attention_mask=torch.tensor(dict_custom['attention_mask']))\n","#     b_logit_pred = outs[0]\n","#     pred_label = torch.sigmoid(b_logit_pred)\n","    \n","#     ret ={\n","#         \"Anatomy [A]\": float(pred_label[0][0]),\n","#         \"Organisms [B]\": float(pred_label[0][1]),\n","#         \"Diseases [C]\": float(pred_label[0][2]),\n","#         \"Chemicals and Drugs [D]\": float(pred_label[0][3]),\n","#         \"Analytical, Diagnostic and Therapeutic Techniques, and Equipment [E]\": float(pred_label[0][4]),\n","#         \"Psychiatry and Psychology [F]\": float(pred_label[0][5]),\n","#         \"Phenomena and Processes [G]\": float(pred_label[0][6]),\n","#         \"Disciplines and Occupations [H]\": float(pred_label[0][7]),\n","#         \"Anthropology, Education, Sociology, and Social Phenomena [I]\": float(pred_label[0][8]),\n","#         \"Technology, Industry, and Agriculture [J]\": float(pred_label[0][9]),\n","#         \"Information Science [L]\": float(pred_label[0][10]),\n","#         \"Named Groups [M]\": float(pred_label[0][11]),\n","#         \"Health Care [N]\": float(pred_label[0][12]),\n","#         \"Geographicals [Z]\": float(pred_label[0][13])}\n","#     return ret\n","\n","\n","# model_input = gr.Textbox(\"Input text here (Note: This model is trained to classify Medical Articles(Still in Progress phase))\", show_label=False)\n","# model_output = gr.Label(\"Multi Label MeSH(Medical Subheadings) Result\", num_top_classes=6, show_label=True, label=\"MeSH(Medical Subheadings) Labels assigned to this article\")\n","\n","\n","# examples = [\n","#     (\n","#         \"A case of a patient with type 1 neurofibromatosis associated with popliteal and coronary artery aneurysms is described in which cross-sectional\",\n","#         \"imaging provided diagnostic information.\",\n","#         \"The aim of this study was to compare the exercise intensity and competition load during Time Trial (TT), Flat (FL), Medium Mountain (MM) and High \",\n","#         \"Mountain (HM) stages based heart rate (HR) and session rating of perceived exertion (RPE).METHODS: We monitored both HR and RPE of 12 professional \",\n","#         \"cyclists during two consecutive 21-day cycling races in order to analyze the exercise intensity and competition load (TRIMPHR and TRIMPRPE).\",\n","#         \"RESULTS:The highest (P<0.05) mean HR was found in TT (169±2 bpm) versus those observed in FL (135±1 bpm), MM (139±3 bpm), HM (143±1 bpm)\"\n","#     ),\n","#     (\n","#         \"The association of body mass index (BMI) with blood pressure may be stronger in Asian than non-Asian populations, however, longitudinal studies \",\n","#         \"with direct comparisons between ethnicities are lacking. We compared the relationship of BMI with incident hypertension over approximately 9.5 years\",\n","#         \" of follow-up in young (24-39 years) and middle-aged (45-64 years) Chinese Asians (n=5354), American Blacks (n=6076) and American Whites (n=13451).\",\n","#         \"We estimated risk differences using logistic regression models and calculated adjusted incidences and incidence differences. \",\n","#         \"To facilitate comparisons across ethnicities, standardized estimates were calculated using mean covariate values for age, sex, smoking, education\",\n","#         \"and field center, and included the quadratic terms for BMI and age. Weighted least-squares regression models with were constructed to summarize\",\n","#         \"ethnic-specific incidence differences across BMI. Wald statistics and p-values were calculated based on chi-square distributions. The association of\",\n","#         \"BMI with the incidence difference for hypertension was steeper in Chinese (p<0.05) than in American populations during young and middle-adulthood.\",\n","#         \"For example, at a BMI of 25 vs 21 kg/m2 the adjusted incidence differences per 1000 persons (95% CI) in young adults with a BMI of 25 vs those with\",\n","#         \"a BMI of 21 was 83 (36- 130) for Chinese, 50 (26-74) for Blacks and 30 (12-48) for Whites\"\n","#     )\n","# ]\n","\n","# title = \"Multi Label Classification of Pubmed Articles (Paper Night July Edition at Thoucentric)\"\n","# description = \"The traditional machine learning models give a lot of pain when we do not have sufficient labeled data for the specific task or domain we care about to train a reliable model. Transfer learning allows us to deal with these scenarios by leveraging the already existing labeled data of some related task or domain. We try to store this knowledge gained in solving the source task in the source domain and apply it to our problem of interest. In this work, I have utilized Transfer Learning utilizing BIO BERT model to fine tune on Pubmed MultiLabel classification Dataset.\"\n","# text1 = (\n","#     \"<center> Author: Owais Ahmad Data Scientist at <b> Thoucentric </b> <a href=\\\"https://www.linkedin.com/in/owaiskhan9654/\\\">Visit Profile</a> <br></center>\"\n","    \n","#     \"<center> Model Trained Kaggle Kernel <a href=\\\"https://www.kaggle.com/code/owaiskhan9654/multi-label-classification-of-pubmed-articles\\\">Link</a> <br></center>\"\n","        \n","#     \"<center> Kaggle Profile <a href=\\\"https://www.kaggle.com/owaiskhan9654\\\">Link</a> <br> </center>\"\n","    \n","#     \"<center> HuggingFace Model Deployed Repository <a href=\\\"https://huggingface.co/owaiskha9654/Multi-Label-Classification-of-PubMed-Articles\\\">Link</a> <br></center>\"\n","# )\n","\n","\n","\n","# app = gr.Interface(\n","#     Multi_Label_Classification_of_Pubmed_Articles, \n","# \tinputs=model_input,\n","#     outputs=model_output, \n","#     examples=examples,\n","#     title=title,\n","# \tdescription=description,\n","#     article=text1,\n","#     allow_flagging='never',\n","#     analytics_enabled=False,\n","# )\n","\n","\n","# app.launch(inline=True,share=True)"]},{"cell_type":"markdown","id":"1a7bb994","metadata":{"papermill":{"duration":0.030261,"end_time":"2023-10-25T20:03:22.929378","exception":false,"start_time":"2023-10-25T20:03:22.899117","status":"completed"},"tags":[]},"source":["<a id=\"11\"></a>\n","## <p style=\"background-color:#1a0a36;font-family:newtimeroman;color:#FFF9ED;font-size:150%;text-align:center;border-radius:10px 10px;\"> References</p>\n","1. [Attention Is All You Need](https://arxiv.org/abs/1706.03762)\n","2. [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://arxiv.org/abs/1810.04805)\n","2. https://github.com/google-research/bert\n","3. https://github.com/huggingface/transformers\n","4. [BCE WITH LOGITS LOSS Pytorch](https://pytorch.org/docs/stable/generated/torch.nn.BCEWithLogitsLoss.html#torch.nn.BCEWithLogitsLoss)\n","5. [Transformers for Multi-Label Classification made simple by \n","Ronak Patel](https://towardsdatascience.com/transformers-for-multilabel-classification-71a1a0daf5e1)\n","\n","\n","\n","\n","#### [Top ↑](#top)\n"]},{"cell_type":"markdown","id":"c0e6f621","metadata":{"papermill":{"duration":0.030268,"end_time":"2023-10-25T20:03:22.990658","exception":false,"start_time":"2023-10-25T20:03:22.96039","status":"completed"},"tags":[]},"source":["<center><h1 style = \"font-size:25px;font-family: Comic Sans MS\">  Feel free to comment if you have any queries:)</h1></center>\n","\n","<center><h1 style = \"font-size:25px;font-family: Comic Sans MS\">  Also currently this notebook needs lots of improvements and I am open to suggestions.</h1></center>"]},{"cell_type":"markdown","id":"b2e0ad7c","metadata":{"papermill":{"duration":0.02899,"end_time":"2023-10-25T20:03:23.04907","exception":false,"start_time":"2023-10-25T20:03:23.02008","status":"completed"},"tags":[]},"source":[" ### This Notebook is Created by [**Owais Ahmad**](https://www.linkedin.com/in/owaiskhan9654/) for Multi label Classification of PubMed Articles incorporating Transfer Learning Techniques.\n"," ### This is for Tutorial/Research Purpose only\n"," \n"," \n"," \n","- **Email owaiskhan9654@gmail.com**\n","- **Contact +919515884381**"]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.7.12"},"papermill":{"default_parameters":{},"duration":5153.965376,"end_time":"2023-10-25T20:03:26.115242","environment_variables":{},"exception":null,"input_path":"__notebook__.ipynb","output_path":"__notebook__.ipynb","parameters":{},"start_time":"2023-10-25T18:37:32.149866","version":"2.3.4"},"widgets":{"application/vnd.jupyter.widget-state+json":{"state":{"035c38cd79c74a83be73fcd749d068ea":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"062cf6508b1741bca9eab2a114e716ca":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"06676756fd054adcb53db9b3c6cd0baf":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"0bd477a12dcc46ef8a4ff1d863e275bf":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_74af4dfd4bf1493091944a8b5b89177f","IPY_MODEL_d870c3700b8c4a76a4c2ed8a8115dd2c","IPY_MODEL_6484b6b4c9434922b611bb1e3b3b7774"],"layout":"IPY_MODEL_b3f8ea639fd24889acb2d396d0d32b2a"}},"0c05e41f4b1c4735a4dc0df28b3dd1da":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_60d851c900ce49e6a6b76d1ea8ca60df","placeholder":"​","style":"IPY_MODEL_2f3f5b9c50354dd69715d7b237b19def","value":" 213k/213k [00:00&lt;00:00, 4.60MB/s]"}},"10d9e0b4980546b69f3738dd9c752d27":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"11c6c4695b6d446fb57b8984089f5491":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"12cf03bde07949e399f0d4f9ad8c517b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_b16bee2d37a245d9a1f5c6dcb8362d6c","IPY_MODEL_93c4ceae5cec4a65953ae9d78cf4b7a7","IPY_MODEL_ee97c6512c0b41efab0e7318400dd60d"],"layout":"IPY_MODEL_7d47ef8c725443efa071897989a7fdeb"}},"15362668e2bc451bab619de7768e46cf":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"","description":"","description_tooltip":null,"layout":"IPY_MODEL_557c7b3c4ce04bd889d2dc4b7a7fc970","max":1.0,"min":0.0,"orientation":"horizontal","style":"IPY_MODEL_f2ebd518f2ac430488e9a9bc5c883b6d","value":0.0}},"157b80a7bd0447c09fbec31d60ba9262":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_9d97c5d180354ca29f1d600c7749105b","placeholder":"​","style":"IPY_MODEL_fcf6aede78ff4cd78004624f2dd1876a","value":" 433M/433M [00:12&lt;00:00, 40.9MB/s]"}},"18179143feca4fecbb00713a8401d18d":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"1c40cce323004c0a8d23bbb5d856ddb4":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_25f3610c25644dd7b5ad67f5f25ae190","max":433355117.0,"min":0.0,"orientation":"horizontal","style":"IPY_MODEL_a36bb19a354243afba9bd2c24209a740","value":433355117.0}},"1d2ded027fd740749cffd61eff772973":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"216ed18f243b454e8c1c1f0f223baf5a":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_4a33229660da42cc8b5233dbadd79673","IPY_MODEL_641921a5bf6345b3a4fcfcc5057df33b","IPY_MODEL_322c7fc8827041da86cec0f17b601db3"],"layout":"IPY_MODEL_f8596d36d9224047a91c5ec6293b4219"}},"25f3610c25644dd7b5ad67f5f25ae190":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"2644f4a3cff2459fb1f5ececb6579d41":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_ff0d51e58f4148838ea8bfd47910e189","max":433355117.0,"min":0.0,"orientation":"horizontal","style":"IPY_MODEL_abd64a93beb14b6faa7f52ea1351c6ac","value":433355117.0}},"27cc7b42763f494e9665441ca9a7f421":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"2cffbbe38e9d4757924df4f5f760fd7d":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"2eb375bd5c7a4bcd97d37bfadfff719a":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"2f3f5b9c50354dd69715d7b237b19def":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"31ed99b17f984fab8d427c6ed249adcd":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_b5ab143dc0a3453aaadbdc053dc83f37","IPY_MODEL_1c40cce323004c0a8d23bbb5d856ddb4","IPY_MODEL_157b80a7bd0447c09fbec31d60ba9262"],"layout":"IPY_MODEL_544bb89554be461db92dbe6c8ad83fc7"}},"322c7fc8827041da86cec0f17b601db3":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_3a413837db45404e8ae117604f996012","placeholder":"​","style":"IPY_MODEL_6bb145deb5eb485aa37e3381fb3461df","value":" 125/125 [00:00&lt;00:00, 4.38kB/s]"}},"32ffa834f69849a2ae231f50be59ecab":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"3320fbf123794ec7854a25ec9082c948":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"3369ec11ad5f4d45840a6c6543678686":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"34565de5b51d46fbbd3e821281568bcb":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"35d891113d2c4543a2e93c2cb3c7a5ea":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_aa4911cb6eae4385bc32a7ca4bdf0985","max":388.0,"min":0.0,"orientation":"horizontal","style":"IPY_MODEL_aac053d9e4b74346ab95374bc0a5e6d5","value":388.0}},"372f8a3963b849b1a3e6ad491c1b3a5a":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"3a413837db45404e8ae117604f996012":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"3c4b987cd94e4b138418d0be9684fe48":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"3f64dfbac4e24d75ae50eda9ab7854d3":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_a10e82c067034256827d36644fbf2e72","IPY_MODEL_2644f4a3cff2459fb1f5ececb6579d41","IPY_MODEL_b3419538ce264a84899f2773e918a37d"],"layout":"IPY_MODEL_52c6999ba11043bd842bcb5d0c1d9571"}},"43d696ae359b455da80b265d20d85c01":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"4850ec4e3ded4fd5bc7b379ca672e9dd":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"4a33229660da42cc8b5233dbadd79673":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_ca0f07e7c70c4bf4a4dd52a5ca933dc2","placeholder":"​","style":"IPY_MODEL_c5c56d29be454102b8ddc43e50b0ba3b","value":"Downloading (…)cial_tokens_map.json: 100%"}},"4ad308a497b347f69cc7391c04e92dff":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"4d52c49bf3c94ee1bd6b171301d609af":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"4d63fb73f67b4577b35c22b93f5bcedc":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"5112e3407541452e99ab6e7a58ca49e2":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_8afe4e849dde44f08f5a3cf76c388296","max":435783451.0,"min":0.0,"orientation":"horizontal","style":"IPY_MODEL_11c6c4695b6d446fb57b8984089f5491","value":435783451.0}},"52c6999ba11043bd842bcb5d0c1d9571":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"53b7e7c1b2e6479f82ab202a184dfad0":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"544bb89554be461db92dbe6c8ad83fc7":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"557c7b3c4ce04bd889d2dc4b7a7fc970":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5a4c65f2b0c8469998c4235836b41cb2":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5c605011b61f413f9d4aabd0763be249":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"VBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"VBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"VBoxView","box_style":"","children":["IPY_MODEL_f7c3889acaae42568af89d1d2b5713bc","IPY_MODEL_15362668e2bc451bab619de7768e46cf"],"layout":"IPY_MODEL_c63afa70cf13447e9dc491a278db59ed"}},"5f579d9b57c341dea27617c96a38011a":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_920b422d13264936ba4c82eee2e20360","IPY_MODEL_35d891113d2c4543a2e93c2cb3c7a5ea","IPY_MODEL_9e87b0be8610449fadccce5a30ca735f"],"layout":"IPY_MODEL_9a3d2121a27d49eb99134c80996fbb0e"}},"606737504cee43808715f680c9cd066c":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_64e0795445f2459d91291742e93cca2d","IPY_MODEL_ff9efac04c8240a79b3348ffc35b1c07","IPY_MODEL_9dacaf7f695e482a80664e623054e1a7"],"layout":"IPY_MODEL_4ad308a497b347f69cc7391c04e92dff"}},"60d851c900ce49e6a6b76d1ea8ca60df":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"62a7889621f640b5a15685ab562919a4":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"VBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"VBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"VBoxView","box_style":"","children":["IPY_MODEL_672eb407b5744aae81794567b198bebb","IPY_MODEL_c1dea86f7bdb49259b63c942aa302d55"],"layout":"IPY_MODEL_e0f391d8720c4766bcd08098fd07a6f8"}},"63db95e522ea408f9c726b776185e5e8":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"641921a5bf6345b3a4fcfcc5057df33b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_c4f6f3dd9db94b2185f782b78390b6e2","max":125.0,"min":0.0,"orientation":"horizontal","style":"IPY_MODEL_035c38cd79c74a83be73fcd749d068ea","value":125.0}},"6425b904d65b4788966b1d07ed31dd50":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"6484b6b4c9434922b611bb1e3b3b7774":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_c044d530742c44b8b858745194e534ef","placeholder":"​","style":"IPY_MODEL_43d696ae359b455da80b265d20d85c01","value":" 1.81k/1.81k [00:00&lt;00:00, 61.3kB/s]"}},"64e0795445f2459d91291742e93cca2d":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_5a4c65f2b0c8469998c4235836b41cb2","placeholder":"​","style":"IPY_MODEL_63db95e522ea408f9c726b776185e5e8","value":"Downloading (…)solve/main/vocab.txt: 100%"}},"6609eb60db62477fb91ef91d972d736a":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"672eb407b5744aae81794567b198bebb":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"LabelModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"LabelModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"LabelView","description":"","description_tooltip":null,"layout":"IPY_MODEL_34565de5b51d46fbbd3e821281568bcb","placeholder":"​","style":"IPY_MODEL_d458f9c293334b5b9948dc1e34465d52","value":"0.026 MB of 0.026 MB uploaded (0.000 MB deduped)\r"}},"695aa2506c134418a6d1ca6f8f10c6d8":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"6bb145deb5eb485aa37e3381fb3461df":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"71d737f41ecb421bb98c691ec5daaaec":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"73a7259337d94f33b32512f47c823e4e":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"74af4dfd4bf1493091944a8b5b89177f":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_e2209db84c5b4a38b82e4a1dc484041c","placeholder":"​","style":"IPY_MODEL_4850ec4e3ded4fd5bc7b379ca672e9dd","value":"Downloading (…)lve/main/config.json: 100%"}},"782dffbdb78f46449bc2d7eb116cad75":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"7c7640a7196d4769ab36d3f8e430d2d3":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_e2b8c93062b24aafb01e148a4d97b50b","IPY_MODEL_84f3d676cf9e4a06a5cd0e5563350c79","IPY_MODEL_0c05e41f4b1c4735a4dc0df28b3dd1da"],"layout":"IPY_MODEL_b330cbffafcd4d5d8b0c520ba12cbb02"}},"7d47ef8c725443efa071897989a7fdeb":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"84f3d676cf9e4a06a5cd0e5563350c79":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_bbf49b145db940b2bc09aa92a4fdbd96","max":213450.0,"min":0.0,"orientation":"horizontal","style":"IPY_MODEL_95d0d53cfefb4436880c1894b8abacb5","value":213450.0}},"8afe4e849dde44f08f5a3cf76c388296":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"8b6365ee8a504ef0945886eec8eb1c31":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"920b422d13264936ba4c82eee2e20360":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_782dffbdb78f46449bc2d7eb116cad75","placeholder":"​","style":"IPY_MODEL_18179143feca4fecbb00713a8401d18d","value":"Downloading (…)okenizer_config.json: 100%"}},"93c4ceae5cec4a65953ae9d78cf4b7a7":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_9f91a0b47bc341fba1f74a235eff4b16","max":1110.0,"min":0.0,"orientation":"horizontal","style":"IPY_MODEL_bb262c8fc9804fff9fd7d8e37c126a3c","value":1110.0}},"95d0d53cfefb4436880c1894b8abacb5":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"9a3d2121a27d49eb99134c80996fbb0e":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9ba8aeab68f54a85b390af043b67a6bb":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9d97c5d180354ca29f1d600c7749105b":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"9dacaf7f695e482a80664e623054e1a7":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_ebb67a659d84457fb6a929187bd0c819","placeholder":"​","style":"IPY_MODEL_3320fbf123794ec7854a25ec9082c948","value":" 213k/213k [00:00&lt;00:00, 2.90MB/s]"}},"9e6eaf01d9f94bd9b67f88c54800d44f":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"9e87b0be8610449fadccce5a30ca735f":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_1d2ded027fd740749cffd61eff772973","placeholder":"​","style":"IPY_MODEL_3369ec11ad5f4d45840a6c6543678686","value":" 388/388 [00:00&lt;00:00, 13.5kB/s]"}},"9f91a0b47bc341fba1f74a235eff4b16":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a0f1fcec8f184e238219e80bbcf8129a":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_fb420d04d1a542b998d149e7284f6231","IPY_MODEL_5112e3407541452e99ab6e7a58ca49e2","IPY_MODEL_b7ee217c8b0546e7898609fd931de77e"],"layout":"IPY_MODEL_53b7e7c1b2e6479f82ab202a184dfad0"}},"a10e82c067034256827d36644fbf2e72":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_06676756fd054adcb53db9b3c6cd0baf","placeholder":"​","style":"IPY_MODEL_fc09d628e8584c39880706a7be2770bc","value":"Downloading pytorch_model.bin: 100%"}},"a36bb19a354243afba9bd2c24209a740":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"aa4911cb6eae4385bc32a7ca4bdf0985":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"aac053d9e4b74346ab95374bc0a5e6d5":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"abd64a93beb14b6faa7f52ea1351c6ac":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"b16bee2d37a245d9a1f5c6dcb8362d6c":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_4d52c49bf3c94ee1bd6b171301d609af","placeholder":"​","style":"IPY_MODEL_32ffa834f69849a2ae231f50be59ecab","value":"Downloading (…)lve/main/config.json: 100%"}},"b330cbffafcd4d5d8b0c520ba12cbb02":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"b3419538ce264a84899f2773e918a37d":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_372f8a3963b849b1a3e6ad491c1b3a5a","placeholder":"​","style":"IPY_MODEL_2cffbbe38e9d4757924df4f5f760fd7d","value":" 433M/433M [00:04&lt;00:00, 102MB/s]"}},"b3f8ea639fd24889acb2d396d0d32b2a":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"b5ab143dc0a3453aaadbdc053dc83f37":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_cd86358528574fb4a2695d6c9f88381c","placeholder":"​","style":"IPY_MODEL_695aa2506c134418a6d1ca6f8f10c6d8","value":"pytorch_model.bin: 100%"}},"b7ee217c8b0546e7898609fd931de77e":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_062cf6508b1741bca9eab2a114e716ca","placeholder":"​","style":"IPY_MODEL_4d63fb73f67b4577b35c22b93f5bcedc","value":" 436M/436M [00:02&lt;00:00, 204MB/s]"}},"bb262c8fc9804fff9fd7d8e37c126a3c":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"bbf49b145db940b2bc09aa92a4fdbd96":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c044d530742c44b8b858745194e534ef":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c1dea86f7bdb49259b63c942aa302d55":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"","description":"","description_tooltip":null,"layout":"IPY_MODEL_9ba8aeab68f54a85b390af043b67a6bb","max":1.0,"min":0.0,"orientation":"horizontal","style":"IPY_MODEL_eb5e1f006a014c6899a3a9f94ccebfe3","value":1.0}},"c4f6f3dd9db94b2185f782b78390b6e2":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c5c56d29be454102b8ddc43e50b0ba3b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"c5dbed5426844c10b8ccece463fbdeaf":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c63afa70cf13447e9dc491a278db59ed":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ca0f07e7c70c4bf4a4dd52a5ca933dc2":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"cd86358528574fb4a2695d6c9f88381c":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d458f9c293334b5b9948dc1e34465d52":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"d7d6e1cddad64885aea3702f8e389e15":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d870c3700b8c4a76a4c2ed8a8115dd2c":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_10d9e0b4980546b69f3738dd9c752d27","max":1814.0,"min":0.0,"orientation":"horizontal","style":"IPY_MODEL_6609eb60db62477fb91ef91d972d736a","value":1814.0}},"e0f391d8720c4766bcd08098fd07a6f8":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e2209db84c5b4a38b82e4a1dc484041c":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"e2b8c93062b24aafb01e148a4d97b50b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_c5dbed5426844c10b8ccece463fbdeaf","placeholder":"​","style":"IPY_MODEL_2eb375bd5c7a4bcd97d37bfadfff719a","value":"Downloading (…)solve/main/vocab.txt: 100%"}},"eb5e1f006a014c6899a3a9f94ccebfe3":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"ebb67a659d84457fb6a929187bd0c819":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ee97c6512c0b41efab0e7318400dd60d":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_8b6365ee8a504ef0945886eec8eb1c31","placeholder":"​","style":"IPY_MODEL_71d737f41ecb421bb98c691ec5daaaec","value":" 1.11k/1.11k [00:00&lt;00:00, 37.1kB/s]"}},"f2ebd518f2ac430488e9a9bc5c883b6d":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"f7c3889acaae42568af89d1d2b5713bc":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"LabelModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"LabelModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"LabelView","description":"","description_tooltip":null,"layout":"IPY_MODEL_3c4b987cd94e4b138418d0be9684fe48","placeholder":"​","style":"IPY_MODEL_6425b904d65b4788966b1d07ed31dd50","value":""}},"f8596d36d9224047a91c5ec6293b4219":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"fb420d04d1a542b998d149e7284f6231":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_73a7259337d94f33b32512f47c823e4e","placeholder":"​","style":"IPY_MODEL_9e6eaf01d9f94bd9b67f88c54800d44f","value":"Downloading pytorch_model.bin: 100%"}},"fc09d628e8584c39880706a7be2770bc":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"fcf6aede78ff4cd78004624f2dd1876a":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"ff0d51e58f4148838ea8bfd47910e189":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"ff9efac04c8240a79b3348ffc35b1c07":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_d7d6e1cddad64885aea3702f8e389e15","max":213450.0,"min":0.0,"orientation":"horizontal","style":"IPY_MODEL_27cc7b42763f494e9665441ca9a7f421","value":213450.0}}},"version_major":2,"version_minor":0}}},"nbformat":4,"nbformat_minor":5}