13_LR_Regularization.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "13_LR_Regularization.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyOBOJ2j7rR2RWq4fqoSqdFZ",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/sandipanpaul21/Logistic-regression-in-python/blob/main/13_LR_Regularization.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Regularization**\n",
        "\n",
        "They work by penalizing (regularizing) the magnitude of coefficients of features along with minimizing the error between predicted and actual observations.\n",
        "\n",
        "**Ridge (L2) and Lasso (L1) regression** \n",
        "- They are powerful techniques generally used for creating parsimonious models in presence of a ‘large’ number of features. \n",
        "- Here ‘large’ can typically mean either of two things:\n",
        "  1. *Large enough to enhance the tendency of a model to overfit* (as low as 10 variables might cause overfitting)\n",
        "  2. *Large enough to cause computational challenges.* With modern systems, this situation might arise in case of millions or billions of features\n",
        "\n",
        "Though Ridge and Lasso might appear to work towards a common goal, the inherent properties and practical use cases differ substantially. \n",
        "- They work by penalizing the magnitude of coefficients of features along with minimizing the error between predicted and actual observations. \n",
        "- These are called **‘regularization’ techniques**. The key difference is in how they assign penalty to the coefficients:\n",
        "\n",
        "**Ridge Regression:**\n",
        "\n",
        "- Performs L2 regularization\n",
        "- i.e. adds penalty equivalent to square of the magnitude of coefficients\n",
        "  \n",
        "    Minimization objective = LS Obj + α * (sum of square of coefficients)\n",
        "\n",
        "**Lasso Regression:**\n",
        "- Performs L1 regularization\n",
        "- i.e. adds penalty equivalent to absolute value of the magnitude of coefficients\n",
        "\n",
        "    Minimization objective = LS Obj + α * (sum of absolute value of coefficients)\n",
        "\n",
        "**Note** that here ‘LS Obj’ refers to ‘least squares objective’, i.e. the linear regression objective without regularization.\n",
        " \n",
        "**Why Penalize the Magnitude of Coefficients?**\n",
        "\n",
        "Lets try to understand the impact of model complexity on the magnitude of coefficients. \n",
        "\n",
        "As an example, I have simulated a sine curve (between 60° and 300°) and added some random noise:"
      ],
      "metadata": {
        "id": "hvDEPAfrK1r-"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Importing libraries\n",
        "\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "import random\n",
        "import matplotlib.pyplot as plt\n",
        "%matplotlib inline\n",
        "from matplotlib.pylab import rcParams\n",
        "rcParams['figure.figsize'] = 12, 10\n",
        "\n",
        "#Define input array with angles from 60deg to 300deg converted to radians\n",
        "x = np.array([i*np.pi/180 for i in range(60,300,4)])\n",
        "np.random.seed(10)  #Setting seed for reproducibility\n",
        "y = np.sin(x) + np.random.normal(0,0.15,len(x))\n",
        "data = pd.DataFrame(np.column_stack([x,y]),columns=['x','y'])\n",
        "plt.plot(data['x'],data['y'],'.')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 610
        },
        "id": "1mWldIWiMHQZ",
        "outputId": "171e8046-5d72-4279-c370-01909e7956b1"
      },
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7fe123b4ecd0>]"
            ]
          },
          "metadata": {},
          "execution_count": 1
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAssAAAI/CAYAAACBC5TUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAfCklEQVR4nO3da2zs+X3X8c/PZ45bVC41zqpdsllvrEaIqqAWW2FQEQLaorRESbkUpRxQi2KtkBJRxAMUVAlEH5UnXB4cQVenESlYTauWqAuNCL0cVFUwbTwlvSSh4mB1lF0C2bpDoeLiM50fD463Ot34l2Mfz8x/Lq+XdLS2Z+T/b1Z+8PbP3/n9S601AADA59vqegEAALCsxDIAADSIZQAAaBDLAADQIJYBAKBBLAMAQEOv6wW0vOlNb6ovvPBC18sAAGDNDYfDX621PnPZY0sbyy+88EJOTk66XgYAAGuulDJqPWYMAwAAGsQyAAA0iGUAAGgQywAA0CCWAQCgQSwDAECDWAYAgAaxDAAADWIZAAAaxDIAADSIZQAAaBDLAADQIJYBAKBBLAMAQINYBgCABrEMAAANYhkAABrEMgAANIhlAABoEMsAANAglgEAoEEss3aGo3Hu3n+Q4Wjc9VIAgBXX63oBMEvD0Th37g1yPplmu7eV46N+DvZ2ul4WALCi7CyzVganZzmfTDOtycPJNIPTs66XBACsMLHMWunv72a7t5VbJbnd20p/f7frJQEAK8wYBmvlYG8nx0f9DE7P0t/fNYIBANyIWGbtHOztiGQAYCaMYQAAQINYBgCABrEMAAANYhkAABrEMgAANIhlAABoEMsAANAglgEAoEEsAwBAg1iGKxiOxrl7/0GGo3HXSwEAFsjtruEJhqNx7twb5HwyzXZvK8dHfbfTBoANYWcZnmBwepbzyTTTmjycTDM4Pet6SQDAgohleIL+/m62e1u5VZLbva3093e7XhIAsCDGMN5gOBpncHqW/v7u0v6pfRXWuE4O9nZyfNT3/xwANpBYfswqzKauwhrX0cHejv/PALCBjGE8ZhVmU1dhjQAA60IsP2YVZlNXYY3z4vg2AGDRSq216zVc6vDwsJ6cnCz8uqswD7wKa5w14ycAwLyUUoa11sPLHjOz/AarMJu6CmuctcvGTzbt/wEAsHjGMFgJmzx+AgB0x84ySZZ/tMPxbQBAF8QyKzMPvInjJwBAt4xh4Dg6AIAGsYx5YACABmMYXGseeNlnmwEAZmkmsVxK+WCSdyb5XK31qy55vCT5x0m+Kcn/TvLttdafm8W1mY2rzAOvymwzAMCszGoM458leccXePwbk7zt4t+LSf7JjK7bmU28m5zZZgBg08xkZ7nW+lOllBe+wFPeneT76qPbBQ5KKV9aSnm21vrZWVx/0TZ1h/X12eaHk+kTZ5uNawAA62BRM8tvTvKZxz5/5eJrKxnLm3o3uavONm/qLxMAwPpZqjf4lVJezKMxjTz//PMdr6btOjus6+Yqs82b+ssEALB+FhXLryZ5y2OfP3fxtd+m1vpSkpeS5PDwsC5madfnbnJf2Cb/MgEArJdFxfLLSd5fSvlwkj+S5NdXdV75de4m1+aXCQBgXczq6LjvT/InkryplPJKkr+b5HaS1Fr/aZKP5tGxcQ/y6Oi4vzqL67K8/DIBAKyDWZ2G8a1PeLwmed8srgUAAIvidtd0bhPPrAYAVsNSnYbB5nHMXDecgw0AVyOW6ZRj5hbPLygAcHXGMOjU68fM3SpxzNyCuG05AFydneUlsol/GnfM3OI5BxsArq48Oqhi+RweHtaTk5Oul7Ew/jTOIm3iL2YA0FJKGdZaDy97zM7ykjC7yyI5BxsArsbM8pIwuwsAsHzsLC8Js7vr4TrjDUYhAGD5ieUl4k/jq+06c+dm1AFgNRjDgBm5zpFsjm8DgNUglmFGrjN3bkYdAFaDo+NghswsA8DqcXQcLMh15s7NqAPA8jOGAQAADWIZAAAaxDIAADSIZQAAaBDLAADQIJYBAKBBLAMAQINYXmPD0Th37z/IcDTueikAACvJTUnW1HA0zp17g5xPptnubeX4qO8GGAAA12RneU0NTs9yPplmWpOHk2kGp2ddLwkAYOWI5TXV39/Ndm8rt0pyu7eV/v5u10sCAFg5xjDW1MHeTo6P+hmcnqW/v2sEAwDgKYjlNXawtyOSAQBuwBgGAAA0iGUAAGgQywAA0CCWAQCgQSwDAECDWAZWmtu6AzBPjo4DVpbbugMwb3aWgZXltu4AzJtYBlaW27oDMG/GMNhow9F4bW4Jvk6v5arc1h2AeRPLbKx1mnddp9dyXW7rDsA8GcNgY63TvOs6vRYAWCZimY21TvOu6/RaAGCZlFpr12u41OHhYT05Oel6Gay5dZrzXafXAgCLVEoZ1loPL3vMzDIbbZ3mXdfptQDAsjCGAQAADWIZAAAaxDIAADSIZQAAaBDLAADQIJYBAKBBLAMAQINYBgCABrEMAAANYhkAABrEMgAANIhlAABoEMsAANAglgEAoEEsAwBAg1gGAIAGsQwAAA1iGQAAGsQyAAA0iOUFGI7GuXv/QYajcddLAQDgGnpdL2DdDUfj3Lk3yPlkmu3eVo6P+jnY2+l6WQAAXIGd5TkbnJ7lfDLNtCYPJ9MMTs+6XhIAAFckluesv7+b7d5WbpXkdm8r/f3drpcEAMAVGcOYs4O9nRwf9TM4PUt/f9cIBgDAChHLC3CwtyOSAQBWkDEMAABoEMsAANAglmHJOacbALpjZhmWmHO6AaBbdpZhiTmnGwC6JZZhiTmnGwC6ZQwDlphzugGgW2IZlpxzugGgO8YwAACgQSwDAECDWAYAgAaxDAAADWIZAAAaxDIAADSIZQAAaBDLAADQIJYB3mA4Gufu/QcZjsZdLwWAjrmDH8BjhqNx7twb5HwyzXZvK8dHfXdQBNhgdpYBHjM4Pcv5ZJppTR5OphmcnnW9JAA6JJYBHtPf3812byu3SnK7t5X+/m7XSwKgQ8YwgI0xHI0zOD1Lf3+3OVpxsLeT46P+E58HwGYQy8BGuM4s8sHejkgGIIkxDGBJzfpECrPIADwNO8vA0pnHiRSvzyI/nEzNIgNwZWIZWDqX7QLfNJbNIgPwNMQysHTmtQtsFhmA6xLLwNKxCwzAshDLwFKyCwzAMnAaBgAANIhlAABoEMtA06zPOgaAVWNmGbjUPM46BoBVY2cZuJQ73gGAWAYaXj/r+FaJO94BsLGMYQCXctYxAMxoZ7mU8o5Syi+XUh6UUj5wyePfXkp5rZTyiYt/R7O4LjBfB3s7ed+f/AqhDMDGuvHOcinlVpK7Sb4hyStJPl5KebnW+qk3PPUHaq3vv+n1AABgUWaxs/z2JA9qrae11vMkH07y7hl8XwAA6NQsYvnNST7z2OevXHztjf58KeUXSik/VEp5ywyuCwAAc7Wo0zD+VZIXaq1/KMmPJfnQZU8qpbxYSjkppZy89tprC1oaAABcbhax/GqSx3eKn7v42m+ptZ7VWv/fxaf3khxc9o1qrS/VWg9rrYfPPPPMDJYGAABPbxax/PEkbyulvLWUsp3kPUlefvwJpZRnH/v0XUk+PYPrAgDAXN34NIxa66SU8v4kH0tyK8kHa62fLKV8V5KTWuvLSf56KeVdSSZJfi3Jt9/0ugAAMG+l1tr1Gi51eHhYT05Oul4GAABrrpQyrLUeXvaY210DAECDWAYAgAaxDAAADWIZAAAaxDIAADSIZQAAaBDLAADQIJYBAKBBLAMAQINYBgCABrEMAAANYhlgzoajce7ef5DhaNz1UgC4pl7XCwBYZ8PROHfuDXI+mWa7t5Xjo34O9na6XhYAV2RnGWCOBqdnOZ9MM63Jw8k0g9OzrpcEwDWIZYA56u/vZru3lVslud3bSn9/t+slAXANxjAA5uhgbyfHR/0MTs/S3981ggGwYsQywJwd7O2IZIAVZQwDAAAaxDIAADSIZQAAaBDLAADQIJYBAKBBLAMAQINYBgCABrEMAAANYhkAABrEMgAANIhl4MaGo3Hu3n+Q4Wjc9VIAYKZ6XS8AWG3D0Th37g1yPplmu7eV46N+DvZ2ul4WAMyEnWXgRganZzmfTDOtycPJNIPTs66XBAAzI5aBG+nv72a7t5VbJbnd20p/f7frJQHAzBjDAG7kYG8nx0f9DE7P0t/fNYIBwFoRy8CNHeztXDmSh6OxsAZgZYhlYGG8GRCAVWNmGVgYbwYEYNWIZWBhvBkQgFVjDANYGG8GBGDViGVgoa7zZkAA6JoxDAAAaBDLAE9pOBrn7v0HGY7GXS8FgDkxhgHwFByDB7AZ7CwDPAXH4AFsBrEM8BQcgwewGYxhADwFx+ABbAaxDPCUHIMHsP6MYQAAQINYBgCABrEMAAANYhkAABrEMgAANIhlAABoEMsAANAglgEAoEEsAwBAg1gGAIAGsQwAAA1iGQAAGsQyAAA0iGUAAGgQywAA0CCWAQCgQSwDAECDWAYAgAaxDAAADWIZAAAaxDIAADSIZQAAaBDLAADQIJYBVtBwNM7d+w8yHI27XgrAWut1vQAArmc4GufOvUHOJ9Ns97ZyfNTPwd5O18sCWEt2lgFWzOD0LOeTaaY1eTiZZnB61vWSANaWWAZYMf393Wz3tnKrJLd7W+nv73a9JIC1ZQwDYMUc7O3k+KifwelZ+vu7RjAA5kgsA6ygg70dkQywAMYwAACgQSwDAECDWAYAgAaxDAAADWIZAAAaxDIAADSIZQAAaBDLAADQIJYBlshwNM7d+w8yHI27XgoAcQc/gKUxHI1z594g55NptntbOT7qu0sfQMfsLAMsicHpWc4n00xr8nAyzeD0rOslAWw8sQywJPr7u9nubeVWSW73ttLf3+16SQAbzxgGwJI42NvJ8VE/g9Oz9Pd3jWAALAGxDLBEDvZ2RDLAEjGGAQAADWIZAAAaxDIAADSIZQAAaBDLAADQIJYBAKBBLAMAQINYBgCABrEMAAANYhkAABrEMgAANIhlAABoEMsAANAglgEAoGEmsVxKeUcp5ZdLKQ9KKR+45PEvKqX8wMXjP1NKeWEW1wUAgHm6cSyXUm4luZvkG5N8ZZJvLaV85Rue9t4k41rrVyT5h0n+/k2vCwAA8zaLneW3J3lQaz2ttZ4n+XCSd7/hOe9O8qGLj38oydeVUsoMrg0AAHMzi1h+c5LPPPb5Kxdfu/Q5tdZJkl9PsjuDawMAwNws1Rv8SikvllJOSiknr732WtfLAQBgw80ill9N8pbHPn/u4muXPqeU0kvye5KcvfEb1VpfqrUe1loPn3nmmRksDQAAnt4sYvnjSd5WSnlrKWU7yXuSvPyG57yc5NsuPv4LSX6y1lpncG0AAJib3k2/Qa11Ukp5f5KPJbmV5IO11k+WUr4ryUmt9eUk35vkn5dSHiT5tTwKagAAWGo3juUkqbV+NMlH3/C1v/PYx/83ybfM4loAALAoS/UGPwAAWCZiGQAAGsQywBobjsa5e/9BhqNx10sBWEkzmVkGYPkMR+PcuTfI+WSa7d5Wjo/6Odjb6XpZACvFzjLAmhqcnuV8Ms20Jg8n0wxOP+94ewCeQCwDrKn+/m62e1u5VZLbva3093e7XhLAyjGGAbCmDvZ2cnzUz+D0LP39XSMYAE9BLAOssYO9HZEMcAPGMAAAoEEsAwBAg1gGoHPOgwaWlZllADrlPGhgmdlZBqBTzoMGlplYBqBTzoMGlpkxDAA65TxoYJmJZQA65zxoYFkZwwAAgAaxDAAADWIZAAAaxDIA1+IGIsAm8QY/AK7MDUSATWNnGYArcwMRYNOIZQCuzA1EgE1jDAOAK3MDEWDTiGUArsUNRIBNYgwDAAAaxDIASRwJB3AZYxgAOBIOoMHOMgCOhANoEMsAOBIOoMEYBgCOhANoEMsAJHEkHMBljGEAAECDWAYAgAaxDAAADWIZAAAaxDIAADSIZQAAaBDLAADQIJYBAKBBLAMAQINYBgCABrEMAAANYhkAABrEMgAANIhlAABoEMsAANAglgEAoEEsAwBAg1gGAIAGsQwAAA1iGQAAGsQyAAA0iGUAAGgQywAA0CCWAZiL4Wicu/cfZDgad70UgKfW63oBAKyf4WicO/cGOZ9Ms93byvFRPwd7O10vC+Da7CwDMHOD07OcT6aZ1uThZJrB6VnXSwJ4KmIZgJnr7+9mu7eVWyW53dtKf3+36yUBPBVjGADM3MHeTo6P+hmcnqW/v2sEA1hZYhmAuTjY2xHJwMozhgEAAA1iGYCV4Tg6YNGMYQCwEhxHB3TBzjIAK8FxdEAXxDIAK8FxdEAXjGEAsBIcRwd0QSwDsDIcRwcsmjEMALgCJ3HAZrKzDABP4CQO2Fx2lgFYO7PeBXYSB2wuO8sArJV57AK/fhLHw8nUSRywYcQyAGvlsl3gm8aykzhgc4llANbKvHaBncQBm0ksA7BW7AIDsySWAVg7doGBWXEaBgAANIhlAABoEMsAANAglgEAoEEsAwBAg1gGAIAGsQwAAA1iGQAAGsQyAAA0iGUAAGgQywAA0CCWAQCgQSwDAECDWAYAgAaxDAAADWIZAAAaxDIAADSIZQAAaBDLAADQIJYBAKBBLAMAQINYBgCABrEMAAANYhkAABrEMgAANIhlAABouFEsl1J+bynlx0op//nivzuN5/1mKeUTF/9evsk1AQBgUW66s/yBJD9Ra31bkp+4+Pwy/6fW+tUX/951w2sCAMBC3DSW353kQxcffyjJN9/w+wEAwNK4aSx/Wa31sxcf/7ckX9Z43heXUk5KKYNSiqAGAGAl9J70hFLKjyf58kse+s7HP6m11lJKbXybvVrrq6WU/SQ/WUr5xVrrf7nkWi8meTFJnn/++ScuHgAA5umJsVxr/frWY6WU/15KebbW+tlSyrNJPtf4Hq9e/Pe0lPLvknxNks+L5VrrS0leSpLDw8NWeAMAwELcdAzj5STfdvHxtyX5kTc+oZSyU0r5oouP35Tka5N86obXBYCZGI7GuXv/QYajcddLAZbQE3eWn+C7k/xgKeW9SUZJ/mKSlFIOk/y1WutRkj+Q5HtKKdM8ivPvrrWKZQA6NxyNc+feIOeTabZ7Wzk+6udg79JTUIENdaNYrrWeJfm6S75+kuTo4uN/n+QP3uQ6ADAPg9OznE+mmdbk4WSawemZWAZ+G3fwA2Bj9fd3s93byq2S3O5tpb+/2/WSgCVz0zEMAFhZB3s7OT7qZ3B6lv7+rl1l4POIZQA22sHeTmeRPByNhTosObEMAB3w5kJYDWaWAaADl725EFg+YhkAOuDNhbAajGEAQAe8uRBWg1gGgI50+eZC4GqMYQAAQINYBgCABrEMAAANYhkAABrEMgAANIhlAABoEMsAANAglgEAoEEsAwBAg1gGACBJMhyNc/f+gwxH466XsjTc7hoAgAxH49y5N8j5ZJrt3laOj/puxx47ywAAJBmcnuV8Ms20Jg8n0wxOz7pe0lIQywAApL+/m+3eVm6V5HZvK/393a6XtBSMYQAAkIO9nRwf9TM4PUt/f9cIxgWxDABAkkfBLJJ/O2MYAADQIJYBAKBBLAMAQINYBgCABrEMAAANYhkAABrEMgAANIhlAABoEMsAANAglgEAoEEsAwBAg1gGAIAGsQwAAA1iGQBmaDga5+79BxmOxl0vBZiBXtcLAIB1MRyNc+feIOeTabZ7Wzk+6udgb6frZQE3YGcZAGZkcHqW88k005o8nEwzOD3reknADYllAJiR/v5utntbuVWS272t9Pd3u14ScEPGMABgRg72dnJ81M/g9Cz9/V0jGLAGxDIAzNDB3o5IhjViDAMAABrEMgAANIhlAABoEMsAANAglgEAoEEsA8CScwtt6I6j4wBgibmFNnTLzjIALDG30IZuiWUAWGJuoQ3dMoYBAEvMLbShW2IZAJacW2hDd4xhAABAg1gGAIAGsQwAAA1iGQAAGsQyAAA0iGUAAGgQywAA0CCWAQCgQSwDAECDWAYA4FqGo3Hu3n+Q4Wjc9VLmzu2uAQC4suFonDv3BjmfTLPd28rxUX+tb8duZxkAgCsbnJ7lfDLNtCYPJ9MMTs+6XtJciWUAAK6sv7+b7d5WbpXkdm8r/f3drpc0V8YwAAC4soO9nRwf9TM4PUt/f3etRzASsQwAwDUd7O2sfSS/zhgGAAA0iGUAAGgQywAA0CCWAQCgQSwDAECDWAaADbRJtyuGm3B0HABsmHW7XfFwNN6YM38ft6mve9HEMgBsmMtuV7yqsbVu4X9Vm/q6u2AMAwA2zDrdrviy8N8Em/q6u2BnGQA2zKrcrvgqYwavh//DyXTlw/86NvV1d6HUWrtew6UODw/ryclJ18sAgJWxTjOs1xkzWKfXfR2b+rrnoZQyrLUeXvaYnWUAWAPrNsN6nbnqg72dlX6tT2tTX/eimVkGgDWwbjOs6zRXfV2O9VsudpYBYA10PcM665GAVZmrnrV1+wvBOhDLALAGuozLeQXeJo4ZrNOxfutCLAPAmugqLgXe7HT9FwI+n1gGAG5E4M3Opo6fLDNHxwEAN+YYM1aZo+MAgLnaxPliNoOj4wAAoEEsAwBAg1gGAIAGsQwAAA1iGQCAuViHW3c7DQMAaJr1kXDrdsTcur2eWVqXW3eLZQDgUrOOnXWJp9etyuvpKuive2fHZf3FQywDAJea9W2s1+222KvweroM+uvc2XGZf/EQywDApWZ9G+t1uy32KryeLoP+OrfuXuZfPMQyAHCp68ROF9+va6vweroO+qve2bHrdX4hpdba9RoudXh4WE9OTrpeBgDASlvWWeA36nKdpZRhrfXwssfsLAMArLGr7u52bVnX6ZxlAABoEMsAANBwo1gupXxLKeWTpZRpKeXSOY+L572jlPLLpZQHpZQP3OSaAADztg53nmM2bjqz/EtJ/lyS72k9oZRyK8ndJN+Q5JUkHy+lvFxr/dQNrw0AcGVXfQPZMp/5y+LdKJZrrZ9OklLKF3ra25M8qLWeXjz3w0nenUQsAwALcZ0AXuYzf1m8RcwsvznJZx77/JWLrwEALMRlAdzy+pm/t0qW7sxfFu+JO8ullB9P8uWXPPSdtdYfmeViSikvJnkxSZ5//vlZfmsAYINd56YXq3CzERbnibFca/36G17j1SRveezz5y6+dtm1XkryUvLopiQ3vC4AQJLrB/CynvnL4i3ipiQfT/K2Uspb8yiS35PkLy3gugAAv0UA8zRuenTcny2lvJLkjyb50VLKxy6+/vtKKR9NklrrJMn7k3wsyaeT/GCt9ZM3WzYAAMzfTU/D+EiSj1zy9f+a5Jse+/yjST56k2sBAMCiuYMfAAA0iGUAAGgQywAA0CCWAQCgQSwDAECDWAYAgAaxDAAADWIZAAAaxDIAADSIZQAAaBDLAADQIJYBAOZsOBrn7v0HGY7GXS+Fa+p1vQAAgHU2HI1z594g55NptntbOT7q52Bvp+tlcUV2lgEA5mhwepbzyTTTmjycTDM4Pet6SVyDWAYAmKP+/m62e1u5VZLbva3093e7XhLXYAwDAGCODvZ2cnzUz+D0LP39XSMYK0YsAwDM2cHejkheUcYwAACgQSwDAECDWAYAgAaxDAAADWIZAAAaxDIAADSIZQAAaBDLAADQIJYBAKBBLAMAQINYBgCABrEMAAANYhkAABrEMgAANIhlAABoEMsAANAglgEAoEEsAwBAg1gGAIAGsQwAAA1iGQAAGkqttes1XKqU8lqS0Rwv8aYkvzrH789m8fPErPhZYpb8PDFL6/zztFdrfeayB5Y2luetlHJSaz3seh2sBz9PzIqfJWbJzxOztKk/T8YwAACgQSwDAEDDJsfyS10vgLXi54lZ8bPELPl5YpY28udpY2eWAQDgSTZ5ZxkAAL6gjYvlUsoHSymfK6X8UtdrYbWVUt5SSrlfSvlUKeWTpZTv6HpNrK5SyheXUn62lPLzFz9Pf6/rNbH6Sim3Sin/sZTyr7teC6utlPIrpZRfLKV8opRy0vV6FmnjxjBKKX88yW8k+b5a61d1vR5WVynl2STP1lp/rpTyu5IMk3xzrfVTHS+NFVRKKUm+pNb6G6WU20l+Osl31FoHHS+NFVZK+ZtJDpP87lrrO7teD6urlPIrSQ5rret6znLTxu0s11p/Ksmvdb0OVl+t9bO11p+7+Ph/Jfl0kjd3uypWVX3kNy4+vX3xb7N2M5ipUspzSf5MkntdrwVW2cbFMsxDKeWFJF+T5Ge6XQmr7OJP5p9I8rkkP1Zr9fPETfyjJH8rybTrhbAWapJ/W0oZllJe7HoxiySW4YZKKb8zyQ8n+Ru11v/Z9XpYXbXW36y1fnWS55K8vZRiVIynUkp5Z5LP1VqHXa+FtfHHaq1/OMk3JnnfxVjrRhDLcAMXs6U/nOS41vovu14P66HW+j+S3E/yjq7Xwsr62iTvupgz/XCSP1VK+RfdLolVVmt99eK/n0vykSRv73ZFiyOW4SldvCHre5N8utb6D7peD6utlPJMKeVLLz7+HUm+Icl/6nZVrKpa69+utT5Xa30hyXuS/GSt9S93vCxWVCnlSy7eyJ5Sypck+dNJNuZUsY2L5VLK9yf5D0l+fynllVLKe7teEyvra5P8lTzasfnExb9v6npRrKxnk9wvpfxCko/n0cyy476AZfBlSX66lPLzSX42yY/WWv9Nx2tamI07Og4AAK5q43aWAQDgqsQyAAA0iGUAAGgQywAA0CCWAQCgQSwDAECDWAYAgAaxDAAADf8fDguDvKEe9NQAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 864x720 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "This resembles a sine curve but not exactly because of the noise. \n",
        "\n",
        "Let’s try to estimate the sine function using polynomial regression with powers of x from 1 to 15. \n",
        "\n",
        "Let’s add a column for each power upto 15 in our dataframe. "
      ],
      "metadata": {
        "id": "q4N-tDcCK1xQ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "for i in range(2,16):  #power of 1 is already there\n",
        "    colname = 'x_%d'%i      #new var will be x_power\n",
        "    data[colname] = data['x']**i\n",
        "print(data.head())"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dI3Do9x-MdEU",
        "outputId": "46e921a0-21cb-4320-d7c9-dcd82f84b2bc"
      },
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "          x         y       x_2  ...       x_13       x_14       x_15\n",
            "0  1.047198  1.065763  1.096623  ...   1.821260   1.907219   1.997235\n",
            "1  1.117011  1.006086  1.247713  ...   4.214494   4.707635   5.258479\n",
            "2  1.186824  0.695374  1.408551  ...   9.268760  11.000386  13.055521\n",
            "3  1.256637  0.949799  1.579137  ...  19.486248  24.487142  30.771450\n",
            "4  1.326450  1.063496  1.759470  ...  39.353420  52.200353  69.241170\n",
            "\n",
            "[5 rows x 16 columns]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now that we have all the 15 powers, let’s make 15 different linear regression models with each model containing variables with powers of x from 1 to the particular model number. For example, the feature set of model 8 will be – {x, x_2, x_3, … ,x_8}.\n",
        "\n",
        "First, we’ll define a generic function which takes in the required maximum power of x as an input and returns a list containing – [ model RSS, intercept, coef_x, coef_x2, … upto entered power ]. \n",
        "\n",
        "Here RSS refers to ‘Residual Sum of Squares’ which is nothing but the sum of square of errors between the predicted and actual values in the training data set.\n",
        "\n",
        "Note that this function will not plot the model fit for all the powers but will return the RSS and coefficients for all the models. \n",
        "\n",
        "Now, we can make all 15 models and compare the results. For ease of analysis, we’ll store all the results in a Pandas dataframe and plot 6 models to get an idea of the trend. "
      ],
      "metadata": {
        "id": "2S5LF7KeMeHG"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#Import Linear Regression model from scikit-learn.\n",
        "from sklearn.linear_model import LinearRegression\n",
        "import warnings\n",
        "warnings.filterwarnings(\"ignore\")\n",
        "def linear_regression(data, power, models_to_plot):\n",
        "    #initialize predictors:\n",
        "    predictors=['x']\n",
        "    if power>=2:\n",
        "        predictors.extend(['x_%d'%i for i in range(2,power+1)])\n",
        "    \n",
        "    #Fit the model\n",
        "    linreg = LinearRegression(normalize=True)\n",
        "    linreg.fit(data[predictors],data['y'])\n",
        "    y_pred = linreg.predict(data[predictors])\n",
        "    \n",
        "    #Check if a plot is to be made for the entered power\n",
        "    if power in models_to_plot:\n",
        "        plt.subplot(models_to_plot[power])\n",
        "        plt.tight_layout()\n",
        "        plt.plot(data['x'],y_pred)\n",
        "        plt.plot(data['x'],data['y'],'.')\n",
        "        plt.title('Plot for power: %d'%power)\n",
        "    \n",
        "    #Return the result in pre-defined format\n",
        "    rss = sum((y_pred-data['y'])**2)\n",
        "    ret = [rss]\n",
        "    ret.extend([linreg.intercept_])\n",
        "    ret.extend(linreg.coef_)\n",
        "    return ret\n",
        "\n",
        "#Initialize a dataframe to store the results:\n",
        "col = ['rss','intercept'] + ['coef_x_%d'%i for i in range(1,16)]\n",
        "ind = ['model_pow_%d'%i for i in range(1,16)]\n",
        "coef_matrix_simple = pd.DataFrame(index=ind, columns=col)\n",
        "\n",
        "#Define the powers for which a plot is required:\n",
        "models_to_plot = {1:231,3:232,6:233,9:234,12:235,15:236}\n",
        "\n",
        "#Iterate through all powers and assimilate results\n",
        "for i in range(1,16):\n",
        "    coef_matrix_simple.iloc[i-1,0:i+2] = linear_regression(data, power=i, models_to_plot=models_to_plot)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 297
        },
        "id": "mVVO9UeSMej-",
        "outputId": "d1af115a-0f8c-4966-8581-ac529bc4bb50"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaUAAAEYCAYAAAD8hukFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydd3xUVfqHn/femUnoLQgCgQAiTQwlBFAQxAKCim0VsICi/HTdXdfe11173bWsriIqKESwizQrSJSOgPQeDKB0BASSzNzz++PeIZMhPZNMO8/nM2Rm7r3nvsn9ct5z3vOec0QphUaj0Wg0kYARbgM0Go1Go/GjnZJGo9FoIgbtlDQajUYTMWinpNFoNJqIQTsljUaj0UQM2ilpNBqNJmKIOKckIrNF5MYQlSUi8o6I7BeRhaEoUxOZaN1oyorWTGQSFqckIlkiclREDovIThEZJyI1y1hGiogoEXEVc1pv4DygmVIqvUJGRykicrKITBGRHc7fKyXcNpUXrZuqQ0TOFpEVInJARPaKyKci0jTcdpUVrZmqRUQaikiGiPzuOOiJZS0jnD2li5RSNYGuQBrwUCXcowWQpZT6o6wXliDAiKCUNlrATODySjanqtC6qSCltHE1MEApVRdoAmwA/lephlUeWjMVpAw2fgL8BjQHTgKeL+u9wh6+U0ptB2YApwUfExFDRB4Ska0isktE3hWROs7hOc7PA04rqFfQtaOAsUAv5/i/nO9vEpGNIrLP6UE0CbhGicitIrIB+z9hsD3+FtNop+fxq4jcFXA8QURedI7tcN4nOMe+F5HLnfdnOuUMdj6fIyLLAsq5QUTWOC2NL0WkRWltLOTvu1Mp9RqwqKRzowmtmyrRzY6Ar3zAKSVdF8lozVSuZkTkfCAZuFsp9btSKk8ptbSk605AKVXlLyALONd5nwysAh5zPs8GbnTe3wBsBFoBNbG98HvOsRRAAa5i7jMS+CHgc39gD3aLKQF4BZgTcFwBXwP1gWqFlOe/5/tADaATsDvgd3kUmI/dQmgIzA34vR4FXnHePwBsAp4JOPaS836I8zu3B1zYrbq5xdkIHAB6l/A3dznXpoTjmWvdRJ9usFu7B7B723nAyHBrQGsmcjUD/AP4EpgA7MVuCPct8zMLo1AOO7/gVuC1gF86UCjfAn8OuK6t85/DVU6hvAU8G/C5plNeSsBD6F9Mef57tgv47lngLef9JmBQwLEB2F16gHOAn533M4EbgfnO5++By5z3M4BRAWUYwBGgRWlsLMb2WHFKWjdVqBvn2vrAvUDPcGtAayZyNQOMca4ZBbiBoc7fPakszyyc4btLlFJ1lVItlFJ/VkodLeScJthC8rMVWySNynnPAuUppQ5je/TAAdzsUpQTeM5Wp9wTyg86Ng84VUQaAZ2Bd4FkEUkC0skPEbQAXhJ7gPkAsA+QctgYq2jdVLFulFL7gPHA5xIF4x+FoDVTNZo5iu0Y31J26G6Sc/2ZZSgj/GNKJbAD+w/npzngBXZie+QKlSciNYAGwPaAc0pTbnKQTf7Ye2H27gBQSh0BlgC3ASuVUrnYXe47gE1KqT3ONdnA/zn/ifyvakqpuWW0MZ7Rugm9blzYoaLaFSgjktGaqbhmfi7k/DL/7SLdKb0P3C4iLcVO43wSmKyU8mLHVy3sGHBZyrteRDo7g4JPAguUUllltOthEakuIh2B64HJAeU/JHZaZBJ2jHVCwHXfA39xfoIdPgj8DPA6cL9TNiJSR0T+VEb7CiAiidhxbYAE53Mso3VTQd2IyGUi0tZJAGgI/BtY6vSaYhGtmYrXNZ8C9URkhIiYInIF0Az4sUylhDqGW8rYYxbOgF0hx2aTH+c1sP/Y2djCmADUCzj3Uef7AxQS7yYozut8dzN2PHYfMBV7XoH/mAJOKcbuFOec0ditkt+AewKOJwIvA786r5eBxIDjA5zr+zqfT3M+XxV0n2uBFcBB53d/uzgbsWPmfYqxWwW/wvHctW6iRzfAX4EtwB+OvZNwxhqi6aU1U+V1TR+nvMPA4uLOLeolTkGaUiD2xNMtgFvZLSiNpkS0bjRlJZ41E+nhO41Go9HEEdopaTQajSZi0OE7jUaj0UQMuqek0Wg0moghLBPhkpKSVEpKSjhurXFYsmTJHqVUw3DbURa0bsJPtOlGayYyKItuwuKUUlJSWLx4cThurXEQka0lnxVZaN2En2jTjdZMZFAW3ejwnUaj0WgihohwStn7joTbBE08kr0QMl+wf2o0pUFrptIJ++KK7y/8hX9OWcWY69Loe2rUhKo10U72Qhh/MfhywfTAiCmQHLcbhmpKg9ZMlRD2ntKAjo1p3bAmN41fzLdrdobbHE28kJVpVy7KZ/9cnqFbwJri0ZqpEsLulOrX8JBxUw/anVyLmycsYebK38JtkiYaKCqMUtrwSkofu7UrJhgmLM2A756wW8K6koldKqIbrZkqIezhO4C61T1MuLEHI99eyK0ZP/HiVZ25KLVJyRdqohYReRu4ENillDphe+piKSqMEvz9wKfh6F67MgkMs2QvtFu9/uO/b4Ml4/NbwFmZOiwToYRVNwCdh2JvOYTWTCUREU4JoHaim3dH9eCGdxZx26Sl5PksLuvarHQX+yuZwkSkiVTGAf/F3oCsbASGUbw5MPsp6Hd/ge+VNwc17U6UsvBh8k3CucyteR5N61bjpi23Y6o8xF8xASx7P79SSukTyt9TE1rGERLdHLPDb8npJ+pp+p1gWWAYMOgFSBtZuOMyPVozlUDEOCWAmgkuxt3QnRvHL+bOD5fj9Smu7J5c/EV68DEqUUrNcVZCLjv+MIo3B7Bg82zYOg8GPo0y3VhesBQYyocpCgOLgTkzOD/nK77f2xVULiIWPm8Oh9fMos7599q60Q2biKfCujFM8PkAZYffUoejWvRGmW7EBwpBLB+CQlkWMu0O+G25fX3geNLRvVozlUREOSWA6h4Xb4/szv+9t4R7Pv6ZHJ/FtT1bFH1B8OCj7kbHPsnpdoUw+ynbISkLfLn8unYBP/rO4pjXh7tZZ/60+1WULwdBIYCBj/4sAUOwlEGucjF6TiK9jPXcenYa7qJ0o3visUFyOnS5Bha/AyiU5WXRZ/9lyYGabD16NfXlMPtUTR5zj8OFDxFQyoda/DYKE8Mw7MCdv2eUnF68HrRuykX4nVIhDy7RbfJmfx9fHPqGiZ+vJ897CTf0bln49f5Ws7+nVK2BPWCphRATiMho7I3OaN68ef6B5HQ7ZLd1Hvhy8WJQf8MHXCoWkuDBGHwfMNAO0Sx51260AIIFCsRwcazfE5y0vTsvfrOB79buYkw/i8b7F9ka8o8pgO6JRxlFagYgdRhqWQaWNxevJaTumUY3sVCJblac8x5G8x7sWNua5Ln/QCm7x2RgOyefT/FD7UHU6XkdnZt1948s2XXY8gxAIHVY4eNUWjelJiROqdyDj8UMPHomXMplvlwuSnTxj+nZLNhcjR5nDyl+wLpaA5h5nxZCDKGUGgOMAUhLSyu4pL3TY8r85lO2blrHMNcsTCyw8mxd9LnTPqdxZ2ecwAnb2AVTTw7xyrAuDDqtMRkff0Sdjx5FkYfYXgvMBOg8TPfEo4ziNPOTasOr6h908P3MOU1ySd31OaIsUF66WCsheQAk3wrtuh9v0Chl95oMYOXh2jz/hZf2i3/g9nPbcF6trcj4C21tACydCCOn6ghOBQhVT2kc5Rl8LGHAWpQPN4onPONQmxTeLW/gajsAap4EjVNPdEBaCHHHO7805F/renNX+9MwsucW3mNOGwmNOtiVzNIMsLwFBqcv6HQyPXf8gWee1x5LALui8uUCSg9oxwhZe/5g6BvzaVynA3ffdC3t8tbC+BlFR1mcBo1MvxOUhWEm8H/XjKDRnqb8b/YmRr+3hCeSvmK4Ly+/1+Svd4IjOFo3pSYkTqncg4/FDFj7H6iIYCgLwUJZFmrtVABEDFAKUKUXgo7xRgwi8j7QD0gSkW3AI0qpt8pSxndrd/Lo1NUM6NiIW64ehGxvYz/fonrMyemQOrxQDdTr0B+16D94vTmYSmGJgWF67POLuEZT9VRENylJNXj4og5cdPrJ1K3uAdLzG7NFacbfoHGevzs5nT+lwKVdmvLB4m189eUWrlAmHrF3LJfA8SadCFEuqmxMqdA4bxED1vy2LH8+QONUZOZ9KO8xEHW8RaKUld86MVwlC0HHeCMKpdSwily/bf8Rbp+8nPaNa/PS0C6YhuQ7nswXiu4xFzU4nZyOjPgCtWkOY5ce5MDe3+jVdwi9A68rCt3YqTIqqpsTkqbKqRmXaTC8R3MGdRrNmx/Vo+76j6lb3UPXi26hSUlaA62ZYqgyp1RknDdowPr4TGl/iGXEMBgxBVmegfppAsrKs4cFjnskgS7DCwoI7Ae+c3X+gLUO7cUMSinumLwcy1K8dnVXEt1mwRPKEjoJqhzcyelcfYaX695ayJhvDvBxta84PW9F0ZWHbuzEBmUNtzm6qZvSh79cdzXfrD6Xuz5ajvejY7y9fybpskprppyEP/sOCvZwgmfXL8+AOsmQOhxJHY5alsHaTZtotX8uLrEwXAlI6vD8svwP3B8S9A9Y68luMYOI8ODg9uz9I4eUpBonnlDa0EkRlUN1j4u3RnTn4f++TZuZD6HElz/RNrgs3diJDcoSbitEN+d2SGfa3/rw4jsT6PTtA1jixXAlaM2Ug8hwSpDf1c1emD+7/oRe0xTkohdppxTvffgRvy3/muot+3JL0+4cbyv7HziW/dkfEtST3WKK1OS6xZ9Q0hwSKLZyqFPdzWOdD+Ce60WUhfLlIoVVHnpAO3YojWagSN00rVuNJ7v8jjHbi4GFz5uLsSUT0ZopEyFZkNUZfJwHtBWRbSIyqtyF+Vss/R+0J7pZ3oIP374f1/7pClz97uL5NXW584NleH2OE/I/cP+vJkbByW7+NOHC0HulxBeBC2wWUjnUaX82hunBqwxyceFr3vvEMgL1qsMw8UExunG3PgvDlYAPg1xlMnZbU5QqOJNBa6Z4QpV9V6HBxxMorNcU9PBFhDvOO5UEl8FzX66j2eGV3N5mJ2arswpm1AROgixuUq2O80Yv5R00Lilkk5yOMfILVmRO5V8r6tFnfT1uL2xxkdK2sDWRRWXoJjkdGTEFY0smk3Y044llNdlZew0PDm6PiBQsQ2umUCInfFcYpYjz3nr2KTQ7vILzF98B27yozOeREVPsHpGf0jgcHeeNTiramCipckhO5/Rh3UlxLefl7zaQ3rI+Z56SVHG7NeGlMnWTnI4kpzNSKbKqrWLsD1tIqpXAzX1bh8b2GCfs+ymVSEkhN2BI3c0kiBcTC8ubQ96mOQVPKMzhBFNCKEcToZTm2ZaHgFCuiPDEJZ1o3bAmt09ext7DOaG5hyZ8VJFuHrmoIxeefjJPz1jL58u2h+YeMU7kO6XSkNIHw5WAhUmucvHk6gYczfUVOF6iw9Fx3uikMhoT/lZ0wAZu1TwmLw/twoEjedz78c8njhNooosq0o1hCC9cmUp6y/rc/dHPLMs+UPH7xDiR5ZTKm2jgOBTjnAeZ2/ttxmWfxPXjFvJHjrfA8RIdTil6ZZoIozIaE0W0ojs0qc09A9vyzZpdfLh4W+HX6mSZ6KAKdZPgMnn9mm6cVCuB0e8uZvehoJ621kwBImdMKUQx3nOAFxtu544PljPi7YWMPx9q7Jhnt4QCx5k0sUOoB42LSdm94cyWfLNmJ//6YhV9q2+h0b5F+ceD19bTPe7Ipgp1U7+GhzevS+PS137klfETeaTTPjspa+dqe7FgZdnzKbVmIsgphTDRYEjnprgMg3GTJ+N674niJz9qNMEUk2BjGMJzV6Ry74tjqfvR4yi8iGECkr+AK+hkmXikhMSs9ifX5rWzfPT64V5klhfmPGOvXO9sq4IvR2uGSHJKIZ5QNvj0kzl1/e+YK7z2Yq5FTX4sCr02VXxTWCva0URySh/ubrsbc30eIhb458j5HRKik2XilWDdBNUj/RPX4RN7cq3yOduk+BFDa4ZIckqVsKpumx4X4FvzGl5vLl5Mchr1oE5hJwYKB3QYRnMiQeHl1AFPkbfBDSoP0+VCEFsvhmlP+vZv9qaJXwobknCSsrzeHHwYeEwT8eWBYcCgF7RmiCSnBKGP8SanY478gi2LZ3L/T3XYO9XHxJuOcdKBnws6Ib9wdBhGUxRB4WXj2D72Xf4hGR9k4Gt8JvcMbKd71pqCFDYk0edOZMQUdi77ir/Nq8F5HRpzc4sdWjcBRJZTqgyS02mZnM5tp+9l1PhFPPraOF7Je8RunZieoJ1FdRhGUwSFhJdPTk4ncU9TnvtyHV0OteC8PrpS0QRQ1JBEcjpNk9PpZq7h6TmbOa3HefRO1hOy/URWSngl0qt1A969IZ1TjizD5w1ovfh3FhUTTHfBuQtp1+vQncamiBTi0We1om2jWvxzyiqO5HoLXqNTfeObEtLO7zjvVFol1eCBT1fkz6vUmomDnlIAaSn1qX3Rn8ib+jHgxTDdGME7i4IOw2gKp5Dwsts0ePzS0/jT6/N46dsN3H9Be/uAXktRA8UOSSS6TR6/9DSGv7mAl7/bwL0dD2rNEGdOCeDUtHPYZExm2hcf8jOdeCCxA60a1jxhUcUC6Ew8TTF0T6nPlWnNeCtzC3/q1oxTTqql11LUlIozWidxRbdmvDlnM6PUcpK0ZuInfBdI6679OW/0MyxVbbhqzHw27DxU9MmFLB2iiWOKCK/cO7Ad1T0m//h8lb0EkV5LURNIMWG5+y+wtfPypkYorZn46yn5ae9dy5dpi7lncW2GjlFMvKkH7RrXPvFE3eLV+AkMyQWlfjeomcDdA9vx8Gcrmfrzr1yUGvopDpooJXA3bH/qd9rI44fztePl/PPford7bVxrJi57Sn6RJC18jrH8kwfUmzz5xnhWbv/9xHN1i1fjJ7iBsvidAr3n4enN6dikNlOmfkbu7Ofsa/RaipqsTNshYdlz2abfeUKPaXh6cy5ruJ2Vc6eT0+yMuNZMfDqlgMrF8OVymfUVY9SjPP3muwVX8fWPJQ18Wq8ergnY1di/WZsqsPCmaQjP98zh5dxHcM1+Uod7NTYpfewekh9lnbBVhrl9Ec8deZgb8zIw3hsS17qJT6cUVLkIigTx0ce9lmvGLmBx1r6CY0kz74vr7rTGwZ/imzbSXjyzkN5z+5zleI4vIxPCfXo00Utyuh2yM1z2UkJmwokRl6xMTCsPl1iIL4/D62aFx9YIID7HlPyVS8ByQmJ6uOKyoUye5uO6txcyo+siWuixJE0w/hTfwGkEgbpI6YPh8uDNy8USFx4d7tWA3ZBp1KHoMUanoax8ueQpk/d+bc4tYTE0/MSnU4JCK5cGyelManqMq8cu4N4ltZnocWNa6LEkzYkUNf8kOR0Z8QWLvvuM59Y25M6cVpxZ9dZpIpEStlBnxBQkK5OMbU157udanLvzEG0a1apaGyOA+AzfBRK0sd9JtRN5f3RPDjTowrCcB9jU6e96LElTNpLT6TL8MXbXS+Wxqavx+iw9U19TMk5ddOnFl1HD4+Kjzz+JS81op1QISTUTeP+mnhxp1JWBS9L46mDz/IO6ctGUgkS3yQMXtGftb4f45qupeq6bptTUr+HhkS6H+fv2u1DfPR53mtFOqQjq1fAw8caedGhShz9P/IlpP/+qJ9JqysTA0xpzTdPfqL3geZQ354RtsjWaohgic0iQXETFX8KMdkrFUKeamwmj0umcXJe3J01i55RH7N0hdeWiKQWybRGPHniAdPUzYDmZV3p8UlMC2Qtx/5yBAEqBEjOuNKOdUgnUSnTz3vlChucpGuyah1K6ctGUkqxMDCsPlyh8SjjarI8en9SUTFYmWD4EsESY6T4H1ax7uK2qMkLilERkoIisE5GNInJfKMqMJKrtmItHvMcrlx0NeujKJQTEum6Op/mKSR5uXvJdrjUTAuJFN4iJMhIYe7AHX63eGW6rqowKOyURMYFXgQuADsAwEelQ0XIjipQ+iFO5+AwPf9k+gHG/NAy3VVFNXOjGn+bb/0GmdXmD1zcnMXfTnnBbFdXEk27sVWQ+Z3+DLrz4zQYsS5V8bQwQip5SOrBRKbVZKZULTAKGhKDcyCGgcjFGTCG9ZX12TX+Kz7/4NNyWRTOxrxs4nuZ74eAhNK1bjcenrsEXJ5VLJRFXunG16MnfzjmFNb8e5MtVv4XbqiohFE6pKZAd8Hmb810BRGS0iCwWkcW7d+8OwW2rGEckbtPg3l33cqf7Q85fPJqPPvsk3JZFK/GhG4dEt8kz6Ufpt+s9Zn3zRbjNiWZK1E2saMbPxfW382DtGcycOSUuektVluiglBqjlEpTSqU1bBjFoa+sTMSXi4mFR7xsXjSTf3+93t5DR89hCjkxo5vshZw5dxR3uj+k99xR/LFpboFjWjehI2Y0A5C9EPO9IYzKncjThx9i7vfTCxyLRd2EYpmh7UBywOdmznexiX8Q0peLYXpo1SSZzbOfZ+aOVgzMfhGJ862My0B86SagMaOUlwWzp9C39Rl62/SyE3e6wZeLgYVbvKybP4Mz+g7C2L4oZnUTCqe0CGgjIi2xxTEUGB6CciMT/yBkViZy7CCXz3sR5fbh22ygxEICtzOIEZFUEvGlm4DGjCUuvtqSR+qXT1M3b6feRLJsxK1uxHCz4ZCHdR/9i/bVf49Z3VTYKSmlvCLyF+BLwATeVkqtqrBlkYz/4b9zAWJ57Q0wxMKnDJQoDNOD6DlMxRJ3uglozOT8vpd/Lnodc54Fptve0kAv/Fsq4lY3yzOQQ7t4dN27mKt9KNOFxKhuQrJKuFJqOjC9xBNjiaxMsKzjH8Uw+Tr5LlZu3ELNVv24uWl3PTO5BOJON05jptY7F6DEnhypfF4kbQTUSdZ7dpWSuNMNwLJJGN5juFH2pFqfD0m7LiZ1E79bV1SUlD7gSrC3OTYMZNALXNBtBGu+Xs+z321k40fLee6KVExDSi5LEz84jRkBFOBDkE7DMFv0CLdlmkjFv1M2duadhZCLiafTMIwY1I12SuUlIBzjb6kIcOf5bXGbBv/+ej1en+LfV6biMnWfSeMQ0JhRYvBQzgg6/daYq1uE2zBNxFJgXMlka/Kl3LGuPdcfSOaiGNSNdkoVIXDTruyFxx3U385Jx+MyeHrGWvJ8Fi8N7YLHpR2ThoKJMi16s3mGxZdfruPCTk2oU90dbus0kUhQAzi5aXcOvTiHl7/dwKBOJ8dcNEbXlKEgeEuLxeO42fiMV3rnMWPlb/x54hJyvL5wW6mJFJyJ2NK8B8+kH2N47kd88NnH4bZKE8kEbEZqbl/Ey8mzqLX7J6at+DXcloUc3VMKBf6Yr/LZY0zT7wTL4iLDIKnrAzy3dD1TXsng4kuuJKFlr3Bbq4kUshfSctow7nDlkLvuU379dj8ne47E3MC1JoQ4DeB2vlwyEkxenb4L6/ckjJaxoxntlEJBQMwXEbB8gALLoteaJ/iwmok64MX37kSOXfs5ia20Y9JwvDFjYuGWPBpmPogSEP9kSP852klp/DiaEeXDg+Jvx96A75Q9TumfQBswlBCNutFOKRQExnyrNXB6Sl77mLIwlQWiwMrjg08mcfFfu1MzQf/p457AAWzE2UNHgfcY/PgibPwuJmfsaypAoGZEMCwfBgrlzUFmPwXth8DM+6JaN7pmDBWBSQ9gOyZl2RMjEbC8iOnmk/0t+eitBYy7IZ3aiXpgO64JTHpIrI932l0YeAGFrJsJKFtDMTZjX1MBAjVTrQHMuBevNwdTLNg8G7bMsTUTxbrRTqkySBsJjTrkd6EBsjIxU/pw4+/J/PX9pVw7dgHv3tBDZ1zFO05jxgAObVpMvTUTMPz7YBsGIDE3Y19TQQIawMZJ7Vn63n2k5i21IzKKqNeNdkqVRXDPyXk/MBlev8bglgk/MezN+Uy4sQf1a3jCZKQmkmhw5gjy1n2IZeVhuDwYFzwDR/dG7diApvKR5j2QfveR+9U1JIgPw+WBgU9HtW60U6psggcdsxdyzp5MJg86jaEzDjN0zDwm3NiDk2olhttSTbhJTif3ms8YnzGBDdU782yX63DrideaEuh8xvk8/NPTNN6/mFHDrqNa6+hOpNKKr0wKmb/k/9xl1gg+HGySve8oQ8fM57ffj4XbWk0EUKP1GbS69GE+3d2UsZlbwm2OJgoQES4bchnPH7mQMVuSwm1OhdFOqTIJnL/ky4U1nxf4fLp3BeNvSGfn78e4asw8th84Gm6LNRHAgI6NuaX1Ho58+yzbV8wOtzmaKKBr83rc0nov1pzn2b/uh3CbUyG0U6pM/OmbYto/2w8p+DmlD+kt6/PejT3YdziXq96YR/a+IxW7Z4zuRhlPyLZF3LPzHm4zPyDp4yuxti7Qz1VTPNkLuXvn3fxVPqDGpMvydRKFutFjSpVJIYu2FsjKcwYhuzavx8SbenDtWwu56o15TLypJy2TapT9fnoX09ggKxPx5eHCApXHuq/G0H7nNP1cNUWTlYnhy8MQC6+Vx56V35IEUVkf6J5SZROwZlWhnx1Ob1aXjJt6cMxrcdUb89i463DZ7xUcLszKDMEvoKlynB62EhOf4WZZ9gGUfq6a4gjQTJ64eHVLY9SW6KwPtFOqSkroSndsUodJo3tiKRg6Zh7rfjtUtvKDw4VROEdBw/EetvR/kD+u+pgZZj9ylYmqzOcahWEeTQABmvm2+5u888tJLFAdKr8+qATd6PBdVREcWitiLsGpjWox+f96MvzN+QwdM4/3RvXgtKZ1SnePwsKFmujEmedWHxh++SkMm+jltlN20vf8y0L/XHXYNzZwNDPQZ9F2/Q/cOc/Lt8M/JXH73MqpDypJN7qnVFUUtpK4P1U8qJXRumFNPvi/XlT3uBj+5nyWZx8o/X2KCA9qopeBp53MKd36M3JjH3441ir0N9Bh35jCZRo8OqQj2w8c5T9r61ZefVBJutFOqaoIDK0ZhrM+VdEPs0WDGkwa3ZM61d1cM3YBS7buD4PRmoggeyGPJ33FRfW28ffJy9h1MMRz2nTYN+bo4d7E6y1ms+THL1m14/fKuUkl6UY7parCH2effAgAACAASURBVFrr/yAMegHMhBIfZnL96kwe3YukWglc99YCFmzeW/r76TGC2MAJkXi+f4oXc/9Bm5zV3JrxE7leK3T3CNSmDt1FP45mBux6iwnuJxibMblyNhmtJN3oMaWqJHA9vOAFWzNfKDTu2+TQCr7ovID7f6rDyHdg7Ig0zjwlaNZ2IUsZ6TGCGCEgRGL44MnWK/lww0rGTf6F0VcPDd19gtdq1EQvgXsuCXQ7MJN54zbTb0AljEdWgm60UwoX/ge5PAOWZtj7LwUnQOxcDdPvpKayeNn0cHvtx7hhnOKNa7vRr+1J9vWLx+Vvk2Em5Cc6BMd6dYUTnQRuIGmYtMz+jLvceeSs/5Rp0xIZPPiScFuoiTQCNGMYJle55yDZs/CNewdz5BcRXxdopxQu/L0Z7zHs9ebJT4BQCgzT3sFW2d1u8eXyZNcDrF/ZntHvLuG1q7tybq2tBTcU9OXk95j8FZkeI4huAjMqf98GS8ZjYOERL6vnTceV0pMBHRtXyq2VUohIpZStqUSCNONaMh4RC68vl4NrZ1E7wp2SHlMKF/7ejN8hAQj5jsjf0zl+zKD6qf14/6aeXJK0jWUZD7P1u7FgBY4tiF1xgS3KbtdB52F2j0uPL0Uv/ozK1GHHB5YNVwJ7ktL5a8ZS5m7cE7p7OWORB9b/wCWv/qgTbKKVAM2If1KtcvHoinr8keMN7b1CPH5doZ6SiPwJ+CfQHkhXSi0OhVFxQWBvRsTuHSmLAk4KALF7TYNeAKDOd3fzzOEMLFce3s0GPtOFaXntMsSAJeNh2ft2GHDZJLv3hWUf84f3IrylpCmCwF1HU/rw0LE82n70Pq+M3wAjhnNG8FhjWXF678qXSzXloprvQRJcnUJiuiZM+CfVZmWySjqyZfoaZvzvboZcchXulJ4VL78Sxq8rGr5bCVwGvFHBcuKPQsIytkMS8h2TAa37Qb/77d7OOxeA5UNQmNinTsrtS+ppnTit5kG7jOAVyXF6UlG8PbImgICxyFpLM7je8jLcNBkxTvHH8KGc16GRXVEszwDE7l0V9byDE2SyMlHOALmpFP/otJ8OpZ24rYlcnGSEtMXj+DDhcdR+H77xE8kbMSXfMfm1UK1ByRsEBuomQDPKl4uEoH6pkFNSSq0BdNy5vPgzV7IX2r0bZzAbJD/xod/99rmBY0cACIYrgfUNL+Sh5Y1482wf55rv57dY2g+BrfOCekp6fCnqCRqLFCBBYESNeSzNWEXiaW3oveFZxJdrn790IoycemJFUUiCjNW8N15cGEqhTDcdeg2q4l9OU2lkL4Tpd2IoLwgoK5fvPniJM7vNoUadk2DmfSVHVfyNnaUZKMuLV1y87B7Fny0TN4o8ZfKPxbVJS/yFIZ2bkug2y2VqlSU6iMhoYDRA8+bNq+q20UHw8kBQsAWb+ULBsSMxodsIJHUY9zfuRtZ7S7jxu938r+8bXFBz44krkpem9aOJDk4YixTEMLnA+x0DXHlYawQlPo43EwvrHTsVVGCCTO7G7/nrtrPZfex+/tZ6J33Pv1RrJZbIyixQhxhi0PePrzAzZ2KJgYFFsVGV7IWo8RejvMcQpzEkliK51jG+aPc67Y4tZ321VJZtTuLLaWsY0rlpuU0t0SmJyDdAYek9DyqlPi/tjZRSY4AxAGlpacEDJ5rgfP/A9yl9wJVgt2QMwx5fchxOIjDmum7cOvEnbvl+Fw9feBmjklsWXqYm+glKEafN+XDoV2THUkwsRMBSgojzX8z0IP6Gjj/s8vu2AhWUpYQZPy5mzxEXFw6+hL7NdyFbf4Bda3RjJlYIqkOM5HQ8W+chKLyWBSKIM3QgAVGVY5vnsWP51+zK3kSaNwcXCoXdJDJNgyv7pELa5cDlnJ69kMvNDA7n+Ej8rV65NVOiU1JKnVuukjWho7CeVMDgYsKIKbx2dRq3TVrKY1NXk+u1uKVf6/DaXAw6QaYCBGqhWoOCYRecdFrTZIGrOxuPVGeb2YaB0/5Hy4Tnqb1tNmJ5wTBRpgvl89rtYwWDvV9xYbXZmIn14N1ShHLCgNZNBQjWzYy7EcfBmAIWCp8yecs3iMYJubgNg33jXmWo9wtaYNEUE2WYWAgigvgTs2beZzeQAcYNRny51AJYM7nwsHEp0POUooXAXk/mCydMjvUkp/PKsC7c8cFynpm5llyvxd/OOSVSx/t0gkxF8GvBrwMsAhNkDKXo0fs8POZpXPn11bh/zcu/VsDrU0zyns0OlUTrhANcan1th2+svEhPkNG6qQiBurGc+Y/OvyYKMaBHIxcdd3+F6cvFQIHY5xhiId1GQJ3k/MSsQG0A+AJ0VgHNVDQl/FLgFaAhME1ElimlBlSkTE0pKGJyrMs0+M9VnXGbBv/5Zj25Ph93nd824hyTTpAJEcGhvIAEGWnZh65ZmYDXX/OgAAvBEjc10q/h3M5nk6rWY7w3JyoSZLRuQkQRujFMD12a14PdXoKnpogYkDr8xMSsQG2YbqdBQ4U0U9Hsu0+BTytShqYcFLNvkmkIz11xOh6X8OqsTeT5FPdf0C5q/yPrBJliKClBBgpUFGK4ka7X4kkdxqXHNdPjRC1FeYKM1kwJFKcbsB3O8ZBwwDzJwN2zC6t/Rk4r3VSEEhClqj7nIC0tTS1erMPBlYllKf75xSrenbeVkWek8MhFHQo4JhFZopRKq6z7lyZBRkRmA3eVdmxA66YcBM9ZggptAhltutGaKQeFzVmCKtONHlOKUQxD+NfFHfGYBmN/2ILXsnj8kqqbna8TZCKEwLHIKFg9XusmAgjO2q1i3WinFMOICA8Obk+C26B5/erhNkcTbvTq8ZryUMW60Quyxjgiwt0D2nFV98iJrYvIpSKyDeiFnSDzZbhtiguifIdZrZswUcW60T0lTZWjE2TCRDEJMtGA1k2YqGLdaKek0cQTepUPTXmoQt2EJftORHYDWyv5NklACDeaqVTCYWsLpVTDKr5nhdC6OQGtmxLQmjmBcNlaat2ExSlVBSKyuDJTV0NJNNka60TTs4gmW2OZaHoO0WCrTnTQaDQaTcSgnZJGo9FoIoZYdkpjwm1AGYgmW2OdaHoW0WRrLBNNzyHibY3ZMSWNRqPRRB+x3FPSaDQaTZShnZJGo9FoIoaYckoikiwis0RktYisEpHbwm1TSYiIKSJLRWRquG2JV7RuNOUh2nQTLZqJtRUdvMCdSqmfRKQWsEREvlZKrQ63YcVwG7AGqB1uQ+IYrRtNeYg23USFZmKqp6SU+lUp9ZPz/hD2A2gaXquKRkSaAYOBseG2JZ7RutGUh2jSTTRpJqacUiAikgJ0ARaE15JieRG4B3uLR00EoHWjKQ9RoJuo0UxMOiURqQl8DPxdKXUw3PYUhohcCOxSSi0Jty0aG60bTXmIdN1Em2ZizimJiBtbIBOVUp+E255iOBO4WESygElAfxGZEF6T4hetG015iBLdRJVmYmryrIgIMB7Yp5T6e7jtKS0i0g+4Syl1YbhtiUe0bjTlIRp1Ew2aibWe0pnAtdgtgWXOa1C4jdJEPFo3mvKgdVMJxFRPSaPRaDTRTaz1lDQajUYTxWinpNFoNJqIQTsljUaj0UQM2ilpNBqNJmLQTkmj0Wg0EYN2ShqNRqOJGLRT0mg0Gk3EoJ2SRqPRaCIG7ZQ0Go1GEzFop6TRaDSaiEE7JY1Go9FEDNopaTQajSZiiDinJCKzReTGEJUlIvKOiOwXkYWhKFMTGWidaMqD1k3kExanJCJZInJURA6LyE4RGefs3liWMlJERImIq5jTegPnAc2UUukVMjpKcf7jPCgiv4jIQRGZJCK1w21XadA6qTpE5GQRmSIiO5y/V0rQ8edFZIOIHBKRtSJyXXgsLRmtm6qjFLoZJyK5zrPwv8ziygxnT+kipVRNoCuQBjxUCfdoAWQppf4o64UliDEiKKWN12Hv+XIm0ASoBrxSmXaFGK2TClJKGy1gJnB5Ecf/AC4C6gAjgJdE5IzQWFgpaN1UkBDpBuBZpVTNgJevuALDHr5TSm0HZgCnBR8TEUNEHhKRrSKyS0TeFZE6zuE5zs8DjvftFXTtKGAs0Ms5/i/n+5tEZKOI7HM8fJOAa5SI3CoiG4ANhdjjbz2NdloGv4rIXQHHE0TkRefYDud9gnPsexG53Hl/plPOYOfzOSKyLKCcG0RkjRMW+FJEWpTWxkK4CHhLKZWtlDoMPANcJSLVS3FtxKB1Urk6UUrtVEq9Biwq4vgjSqm1SilLKbUAyAR6FXZuJKF1E17dlIewOyURSQYGAUsLOTzSeZ0NtAJqAv91jp3l/KzreN95gRcqpd4CbgbmOccfEZH+wFPAlcDJwFbsPesDuQToAXQoxuyzgTbA+cC9InKu8/2DQE+gM5AKpJPfQvse6Oe87wtsDvgd+jrHEZEhwAPAZUBD7P/87xdno4gcEJHexdgrQe8THPujBq2T458rUyelQkSqAd2BVRUtq7LRujn+OZy6+bPjpJf4HWexKKWq/AVkAYeBA9gP7jWgmnNsNnCj8/5b4M8B17UF8gAXkAIowFXMfUYCPwR8fgu7K+n/XNMpL8X5rID+xZTnv2e7gO+exe6JAGwCBgUcG4DdvQc4B/jZeT8TuBGY73z+HrjMeT8DGBVQhgEcAVqUxsZCbL4RWO/YXgeY4pTRKxzPXuskMnUSUI7LuTalmHPGO7ZJuDWidRPZusEOnzZwjg8CDgFnFldWOHtKlyil6iqlWiil/qyUOlrIOU2wReVnK/Yv16ic9yxQnrLDWXuBpgHnZJeinMBztjrlnlB+0LF5wKki0gi7pfMukCwiSdgtHn+4oAV2vP6AiBwA9mH3bspqo5+3sVtCs7FbtrOc77eVoYxwonVSNTopFSLyHHYo7Erl1DoRitZNBOhGKfWTUmqvUsqrlJoOTMTupRVJ2MN3JbAD+4/opzngBXZie+UKlSciNbC9+PaAc0pTbnKQTTuKsXcHgFLqCLAEuA1YqZTKBeYCdwCblFJ7nGuygf9z/kP5X9WUUnPLaCPOfS1ljwekKKWaYTum7RT8naMdrZMK6qQ0OOMmFwDnK6UOhrLsMKF1UwW6CUJRcDjhBCLdKb0P3C4iLcVO6XwSmKyU8gK7sTM/WpWxvOtFpLMzQPgksEAplVVGux4Wkeoi0hG4HpgcUP5DItLQaaH8A5gQcN33wF+cn2D3XgI/A7wO3O+UjYjUEZE/ldG+44hIfRFpLTYdgH8DjyqlrPKWGYFonVRQJ04ZidjjjQAJzmf/sfuB4cC5Sqm9FblPBKF1U/m6uUJEaoqdVHI+cA32EELRlDV2GIoXdsz33CKOzSY/5mtg/+GzsUUyAagXcO6jzvcHgJ6FlDWSgJiv893N2LHZfcBU7DkG/mMKOKUYu1Occ0Zjt1B+A+4JOJ4IvAz86rxeBhIDjg9wru/rfD7N+XxV0H2uBVYAB53f/e3ibMSOn/cpwuZTgXXYceOtwB3heOZaJ5Gtk4BrCryCjuU4ZfhfD4RbI1o3Ea+bTOB3517LgaElPT9xLtSUArEnhm0B3MpuTWk0J6B1oikPWjc2kR6+02g0Gk0coZ2SRqPRaCIGHb7TaDQaTcSge0oajUajiRjCsihgUlKSSklJCcetNQ5LlizZo5RqGG47yoLWTfiJNt1ozUQGZdFNWJxSSkoKixcvDsetNQ4isrXksyILrZvwE2260ZqJDMqim9gI32UvhMwX7J8aTWnRutGUFa2ZSifi9/QokeyFMP5i8OWC6YERUyA5Lvfb0pQFrRtNWdGaqRKiv6eUlWmLRPnsn1mZ4bZIEw1o3WjKitZMlRD9Timlj91qEdP+Wa2B7l5rSkbrJm6xrHJOg9GaqRLCHr6bP2cGK36YxvCrhlOjdcDuytkL7ZZISp+iu8j+cwY+DUf32iKZeZ/uXscB3q3zObrhe2q1PbvgM9a60RTBzoPHePiVt/lr69/odOaFZdcNQOehgEDjVK2ZSiIkTklE3gYuBHYppU7YdrhIshfS/fuRpHlzkYwPYeQX9oMNjt06lcfhk3sxbX8zFmXtp23eGq7feBumykP8oiise62FEpGUWzMA2QvxvXMx1cmD+f/JrxCK0M2hxj15aV09fty0l+TDK/iv95+4VR7iStC6iTIqopuG+5fzUt4jeNZ6YeOYEnVTwEEFn9MZrZlKIlQ9pXHY2wi/W6arsjIxrDxELHy+XJj9FPS7v2Al4c2B6XdiWRZuDOp4O9PTXR+fUqByEbGwvLkY/laO6ckXTkqfEP16mkpgHOXRDEBWJm7yMLBQ3hykGN0oy8KDQZq3M73qNsZT3cD4PQ/BwufNwdo0B3frs7RuoodxlFM3xi8/4MGLiQXeY7A8w3YkhegGywLDgF5/gcTa5O79BZc3B8PRzfKt++lsujF8aM2EmJA4JaXUHGeF27KR0gcxPXi9x2yhbJ4NW+fZLRV/JSGCZVkYWHiwGOBa7GwTZaJE8CkhV5mM39qEG89Iw+Vv+RbWDS9tF11T6ZRbM3C88eH15mBK0bpRloUE6EYOc1w3ljLIVS6eXd2Ae8/sRqLWTVRQUd0oMVGWBaKQpRmQOvy4npQvFwvA8mKCfd6PL2IBhjLwIhgYeMXF0792AbpwT9vdpPW7uHBdaN2UiypLdBCR0SKyWEQW79692/4yOR1GTGFr7e5YSkBZdoXy2zI7dtttBEtPf4gc5cJyNisU/0v5MJQPwzCZ1fIOnl5Zm5sn/MSxxt2gz52FVyzjL4bvnrB/6sHJqKAo3ciIKfzkSrV1UYhutp3xGMeUC1+AbgBQPls7hsHK0+9nXPZJ3PTuYnJPTtO6iREK1QxAcjq7Wl+Rv/Wp5bV7S1mZHOj7KGNcw3godyQKs8D2qCZgioVLQLqNIHHUNF6992ZqtzmDK1b14tM9TYJN0LqpAFXmlJRSY5RSaUqptIYNA1abSE5nT9od5OJGiQmGCUszYMm7WEszeHKx8ETS01hdRiKG+4RyRSkGtfbw6JCOfLNmJ7dMWEJu1jw7K2bxuPzsGJ3OGZUUpRtp3oP17f5CripEN8syeGS+4raEf5Fz+rWF6gal6H6S4pnLTydzwx7u/2QF6pcFWjcxQJF1DVC753Xk4MEiXzPquydI/PZB5uS246qb/4Hrwn8jhguQ445JAEMUZr1kSE6nYa0E/ndNN3q0rM/kTz5m3+RbYert+c5H66bchD37DuDUbudw9YwHuLvdbno1OApLxoPyYfkU51bbwFWj/oOruge6DrdbNod3w4av7ZaOE8+9LjkFl2Hw0Wcfwy9PoZQ9boAYYCYUDO3oGHBM0K33AK7+6QFubfkb5zTJzdeNV9FJrWDgLc9SvXFt6H5tkbq5MjmZHQeOMufbaXjXPoXbygOtm5il5iln8Lcaj9PHs5Y/nQIsGY8oHy6leLLLAVok14XkkdCog+1Ijh2Eef+1e+NmQoHn7zYNXu/no/r7j+JZ47V7V0snwsipeny7AkSEU6pXw4NKTufWX47wdXoi9Zdl4PMq8pSLi3t1ou6SV/LjsoHZMEHx2uE9mnPqhr0YG+zkCQWIP7RzdG9+ppWO8cYE7U+uTaee5zFq3lY+bG/SzczAcnRzYc+OnLLhTcgrWTe3ndOG1mu3I7vzQCz7PK2bmKXGKb3457IW9G9dg3pLbc0ow02LZsl2Dzm4rmk3uMjnX2/XAqzjQWLye0V97tS6KSch2U9JRN4H+gFJwE7gEaXUW0Wdn5aWpoIXSdy46zAX//cHGtVOpOXRVZyW9zOXnnE6LRc/Vra5ANkLyXvnQsSXiykK8bd49TyCAojIEqVUWhjvXybNQOG6OZLrZdBLmWTtPUJXWc/Zies5t1t72i97sky6ydk8D969GFPlad0UQ7TppjDNbN59mMEv/0C7k2tRa/dPnGGuZeS5XUn85sGyzzvKXogaN9i+DsBMQEZO1ZoJoiy6CVX23bCKlnHKSTV55vLT+dcXq3Cn9OCsPsNomf1O2ecCJKfjGvkFX07/iNm/+PhTh+p061tEdkwwOlumygiFZgCqe1x8dMsZzF63m50H23JFt2Y0Wv5qmXWT0KoXu6/8hMkfZpDjrsutPeuT2Kav1k2EEQrdtGpYkwcHt+ehz1bStG5HHr9pFImr/1e+eUfJ6cjIaWyb/Taz1+2iUa+RnKc1UyEiInzn56LUJlyUGpDJYpYhLhvwkKV5D869qTufTPyJy1fu5KXTmjKEEkSgF1uMWpJqJnBFt2b5X5Qlnh+gm4Yd+pB+XUeuHjufZVsa8HbfNNwlVR5aN1HJ1T2aUyPBpEfLBjSpW63sY0BBumh6TXc+ePVH9i3L5ex283H98qPWTDmJKKd0Ak7KeKmWjQl6yK7kdF4e1oXr3l7IhA8/5MLEpzCtvKJFoGf1xw4V0E16y3SeuLQT93z0M2MzJnPz1tuR4ioPrZuoRES4tEtAQ6a0moFCdSPJ6dx2ThtefTcDxj8NStc15SXyF2RNTi98/kggRaRfJrpNxo5IY3DtTShvCemZwYst6myZ6KYCurkyLZk/92vNobWzUN4crZt4oTSagSJ107/dSVxSbwtYuq6pCJHvlEpDMQ+5dqKbSy69Cq+48CoDy3AXLgJ/S6n/g7o7HS8Uo5u7B7Qloc1ZzsTtYioPrZv4owjdiAgdzhhEntZMhYis8F15B/9K6HrXPbU3e676hI8/mcQPee24z2xLx6LK0QKJPipBNyLCn68dzvNv+TC3/kjnnhdyflFla91EJ5Wgm25nDuDeRU/S6o9lXH/1NSRozZSZyHFKFR38K+ohO8JLSunDoFueY/wb87hm7AImje5F28a1Qme/JjxUom7cWZncef4ZjJ7Vntdm7+axOllc2yslZKZrwkgl6Ua2LeLWlr9x+4JT8G5J4i8tQ2hznBA54bvKWJYjaP2p5D9WknFTTzwug6vHzmfjrsMVv4cmvFSybjwTLuWNfj7ObX8SD3++ijfnbK54+ZrwU4m6abH8P0xKfIofZ81gx4GjFS83zogcp1QZg3+FCC8lqQYZN/UEhOFvzidrzx8Vv48mfFSBbhK2zeW1q7sxuNPJPDF9Df/+ej2hmHSuCSOVrBs3XrrLKv4+aRl5PqviZccRkeOUKmPwrwjhtW5Yk4ybepDns7h67ILCWzPZC/VWx9FAFenG4zJ4eVgXrkxrxsvfbuCJaWsKd0xaN9FBJetGTA9p/S5mYdY+Hpmyqvgt2LVmChCSZYbKSmFLf1QaxQxmrtj2O8PfnE+/Glt4tttBqp3azz6wPMNecdq/cGcMZsiEe7mY8hAJurEsxaNTVzNubhYPnX6IUc22Iy2dVrbWTcRRpZqBE3Tz1Iw1vPH9Zvqe2pBHh3SkRYMaBc/ZudreVNC/4GsMagbCsMxQRFPYgKQjik4pfZg0yKDV9IfxZHpRc5+zF6v35WLvJIie3BavFKEbIyuTRzr3pvmRwwxb81fUei+YLq0bjU2Qbu7reJAL9s/nydVJ9H1uN1ectJ0nDz2ES+WB6cKwfHaYGMCXozVDPDilYIKybjp2HoYlXgwsLF8eAOKvWBA9uU1jE6AbMT1cnzoUFaAbe5VorRtNANkLkXeH0NmXy6REN592+h95m1ZjqDxnW/U8LFH5YyhiaM0Qj04pOPkBheFKwPLmkqcE0xBcWPYGYF2ugdRhcd9y0XCCbkQAVwI+bw5eZeAyDUzl07rR5BOgGcMHl9ffAl2Ho8ZPQvlyURh4lcIUH6ZhIoNe0JohHp1S8MKLqcMhdTiyJZM3sxrz3ZpdPJt2kFPSL9AC0eRTiG4kdThq8xwe/bkeG3ce5qWeh2mcep7WjcamsEVek9MRZ+KtK6UPK37Zz3czP2FvvXQe6zoCM9w2RwDx55SKmI0tyenc2MvHzNfncumKI0w/+zSSw2yqJoIoQjeu5HRu75bDkP/+yOUrYXr/LtQJs6maCKGolR8Cxp26JENW9Y68Mnk5bedvZcQZKeGzN0KInJTwqqSIhRcT3Sb/u7obKPj75GV4/fMLdMqmBorUTVLNBF67uis7Dx7jvk9+zk8V17rRlGKR10s6N6VPmySe/3Id+9Zlxr1m4tMpFUNy/eo8cVknlmzdz39nbTxhVYh4FoumaFKT63LXgLbMWPkbHy3ZpnWjKTUiwr8u7kg73xpqTbo87jUTv06pmFbsxalNuLRLU/773UZ2/vx16Jcj0UQvxehmdJ9WpKfU57Gpqzm8bpbWjSafEnrNrRrWZGST7YiVF/eaiU+nFNiKHTcYpt5+glgeuagDdat7eGZtQ5Te+0QDJerGMISnL+9Ejtfi1S2N9Z45Ghu/br59HN65ABaPK/S0jr0GkUcJ217EAfHplILTwhe/c0J3uW51D49f0pFPdjfl89TX9N4nmlLpplXDmvztnDb8b1MSP509XutGY+vGmwNY9mof0+8stMeU0uVsnm30LP8zhuK99rO41Ux8OiV/qqYz5RFUod3lAbV/4YXG3zB50TZ2dPpz3IpE41BK3dzUcjcP1p7BWz9sIafX37Vu4p2UPmAEVLXKKjw0l72Q4Sf9wrdH2zD7j/jd8yI+nZI/VTNtpL3eVGEhFmc29mW/j+dteYyMjz4Km7maCKGUuvFMuJQbcyfy/JGHmTljStjM1UQIyekw6AUwXPaqDWbCiaE5J8R3yqqXyEh4kiU/zgyPrRFA/M1T8uOfK5A6vPAFW51QjSgfCQLWlkwWbB5Ej1YNwmezJvyUVjdYeMRL1pIvOXDOYOpW94TPZk34SRsJjToUvdNtQH3jRmFu/ZG9h4fSoGZCWMwNJ/HrlPwUtfNowGxsMT1scHfm+6mr+eIvvTEMOfF8TXxRGt0Ybr4/2pbDszby4OAOVW+jJrIobgv0IN3MzWlPvWU7GNU7/sJ48Rm+Kw0B+63IiCkMHjSEVTsO8vny7eG2TBPJBOjGGPkFrbr0Z/y8rfz6u96BVFMMQbrJOTmNKcviX4m3kQAAFFdJREFUs67RPaXiCGjZXNxU8WbmZmbMmMJFB4/ganWWHsDWFE6Abm6rcYTPl23nsy8+45YWOwoP3Wg0UEA3QzpvYuaMKeyb+QP1O/aPK81op1RKDEN4Iu0Ibb98BGOWFzKf16m+mhJJrl+duzsc5Nr1f0Nt8iExuvmfJrRc1nAH13qeJGG+Fxa/GFea0eG70pK9kM6bXscj9l4oKo5nXGvKQPZCRuS+j4c8JM5n6mtKT9KmT0iQ3Lisa3RPqTT4Z2R7czBQeJWgDBfuOJ1xrSkljm4SvDkoUfiUIC43htaNpjiyF8LSCQigFCjDROJIM7qnVBr8M/mxEAzWVOvKDdbDHGzYJdyWaSKZAN2AwVzViTda/CduwjCacpKVCZYPwa5xltQfFFeaCYlTEpGBIrJORDaKyH2hKDOi8KdrigmuBKqf/xCZx1ox/sescFsW1cSTbsSVwJq2t/LvtXXZcUBn4lWEeNKNz/Dw2v7u+dvoxAEVdkoiYgKvAhcAHYBhIhJbkzIC0jUZMYXWXftzbvuTGPvDFg4dywu3dVFJPOpm0KCLUQrGzNkcbsuilnjTzU/9xjPrj5b8sHFPuK2qMkLRU0oHNiqlNiulcoFJwJAQlBtZBG7Wlb2Qxxt8Retjqxg/NyvclkUrcaebZodX8kryLNYs/IZdh46F27JoJa500+XM8zmr2mZ+/+qZuNlfKRSJDk2B7IDP24AewSeJyGhgNEDz5s1DcNsw4QxeN/bl8n6ii5syXRw+syU1E3TOSBmJS90M9OXSzzT5cMbJXHflleG2KhopUTcxoxkg4dclvMVjyJ481PgJyIgvYn58qcoSHZRSY5RSaUqptIYNG1bVbUNPwPYFHryclruCCfO32sf09tchJ9Z0I8qHR7zsXfktvx9xQr9aNyElZjQDkJWJCy8usVDevIKp4TGqm1A077cDyQGfmznfxSZBa+I1OKkxR2c/T25iGp6vH7Qdlp4gWRriVzeGm525NVg5+RHO7HQqzLxP66b0xJ1uxPTg8+aQh4knsT5G5gtQrUHM6iYUTmkR0EZEWmKLYygwPATlRib+QcisTKjWgJEz7kVZucjMyYCy90rxT3aLEZFUEnGrG6NaAx6ddg/G1jxUtoEoS+um9MSlbtbPm874ZQd5YuZ9YOWBiK2ZGNRNhZ2SUsorIn8BvgRM4G2l1KoKWxbJ+Neomno7pi8HxD+h1kSQuN7KuLTEu27cKhdBYVkghgFaN6UiXnVzqqW4bO094MvBbvwazsaBsaebkIzOK6WmA9NDUVbU4My6BoUCfMpkUbsH6HUyetHNUhLPuhFHN3mYyIBn8OQe0LopJXGnm+yFmO8NIc06hiiFEgNxJcDAp+Ho3pjTjU4ZKy/OrGsbYVa183hscxdmXdYPj0svlKEpgiDdfOg9izxvf67vE3/75mhKiZMkY6DwirDUPJ3O1zyDO6VnuC2rFHTtWV4KzNZPpGGf69l+4CifLt0Wbss0kUyQbtY0Gszr32/iWJ6v5Gs18UmgZswEnjpyCWO2JIXbqkpD95TKS2DCQ0ofujbrzulLf+TVWZu4vGszXKb295pCCNLNoJxWTBy7gA+XbOPani3CbZ0mEgnQjJnSh0bfm7z07QbOPCWJzsl1w21dyNE1Z0UImK0v2xbx75O/JWn/Mj5btiPclmkimQDdnOHZxONJXzHn22nkeHVvSVMEAZp5Jv0YdyRO5ZXxE9l1MPZWBtE9pVDgzNZv7cv9//buPz6K+s7j+Oszs9mAKPJTqBBF8VC0EjkTRJEWuVoQBK2ed4ICenp6bb2TM9bS6ql3Uk69ov3hw+OsVUJJKKggiihagxhaS5oAQUTkTgQXEJAfgkLI7sx874/ZJEsMAZLd7M7yeT4ePOCRbHa+y7wz35nvT0pzbX69ZAfuV92wdXda1ZxIBTLrGm5yarnehKiaf5DLviFZ13GtkihSQcd513OnG2WSZzPj6a3cUdiJDv2GZU1mtFJKhsTZ+hgm1z6DLPVgmQ2jpkOP8+uba7IlOCoJ6nKDR1hiFK6bhlln/CHimhvVlIRrTa4Y7qqZgVVuMH/6L2TgzZA/ruF1Ac2NVkrJkDhbXwTLc7AA4znIa/eAFQLPybqZ16qVEnODnxsRwPNAc6OakpAZSwTxPCS+O62pfB5ZNRuQQOdG+5SSIWGpeRk1HZEQxoBAw4xr3QpbNZaQG2v0dIzYmLrvaW5UUxK3Qxk1HQnl4iFg8Oe+ubHA50aflJKlbrY+fjjcRUUIHpad46/yUHfnkkUzr1USJORm+96D9Fj+ALYYLDsEmhvVlITM0ON8rOpS3JUlGNfBiE2ObQU6N1oppYAU3Mq6WC/eWPQi5xVcxdj8XoFu41Vto/eVP+Dnm0/B/nQ5k268mS4dcjU3qnnxCsrOH0/ZGy/x1MaePDTmAvLd9wObG22+S5EBl36X98++jQdXdmB/94ENGwQq1Yy/vfY6nnav4fEPTj18Y0mlmpM3iMtumcaeLhcx+U9hDg2eHNjcaKWUQj8eeR5fHIwx452P010UFRB9unVgwuA+zKuMsH77/nQXRwVIuxybqddeyCe7DvDb5Z+kuzgtppVSCn2z16ncfe5e7D8+yefrgtfhqNLjX/7mHC5vt5HVJQ9iPl2R7uKooIhUcPn2Yv6xzy6eLd/IgVon3SVqEe1TSqVIBXdvLcKzongvLIB/WOR/XfsJVDM67V7NczIV9kfxiudg3/Kq/w3NjTqS+AR+3Cg/sXKoqpnC7D/35c6zdwcuN1oppdKmciw3hiUejhdjZ/nznLZxQVbuFqmSaFM5tokh4uG4UWJVJeSsnau5UUcWn1SLcbE8uPG0zbz+7uvcUT4VCVhutPkuleIT3YzYOBKiYtNeTKrnEEQqoHy6/7cKpvgW2EZsYibEmq37Uj/3RHMTbAkriWOHOfeSUfQ/tAbj1AYuN/qklErxiW6yqZyVbn+eW/IRI9uX+f/pqZhDkPAIH6Q7I9VIQm5mbvoGb6/fybx2r2N5aG5U0xqtPj+gdyG/Wb6R2MEF5IobqNxopZRq8XkElxpDh4+7MnHzA8y4vIaO/a9I/i9+wiN8/Z2RXlyCKZ6bcRdH+e0Ty7jvpEd47OL9qVnkV3OTHQ6bwA+Fl49k3Ks1PDXkIKfnXxmY3GjzXRsREaZ970JWmX7867bhmN6FyT9Io0f4IM7mVofrdFKYR675Ji/u7MUM79rUVBaam6x03V/34qOc/jxZc3WgcqOVUhvKO7CWWf3K2fvRcuav3JqCAySsi6VNMFnjqk4RftmrjHfeXsS6bSmYu6S5yT6RCk75y6+466/28uqabeyriSX/GCnKjTbftZV4+2uBG2VObojbXgkxuO/t9OrUPrnHSVwXSwVfPDdj3SgjQjZTSk7i0cm30S7HTu5xNDfZI6Gv504rh7ecKby8qj+TLuuT1MO4nuHfKtsz8oKJfCuve9LeV5+U2krinkvicLV5h2XPTsHZ/Od0l0xlssNy41Kw7w2WP/9THSWnjuyw4eExbu+4AnfZ9KRPxF66fielKz7lh6Uriew5mLT31UqprSS0v4plc0PoXf7uy1mY4rF6gVFHlpAby7K5Meddhm19BnfmGM2NalpiX49lMyL2NhMPzcZL8rWmZMVmup0cBgP3zFudtPfVSqmtJLa/DrwZ27iExEPcGP9X8Xryj6fzTrJD49zgERJ/U7c9H5Ql/3iam+BrlBkrfq1J5lylyJ6DvLPhc8ZfciYPDTxAYWQm+zYsT8p7a59SW6prt49UwOo5GDeKi82D1Z15+PIv6dfjlFYf4g/rdvDa4oVM+/J+wuJghXIR7bwOtoTcSDw3jrF5eE1npl3hcHJu636No47HE29t4LO1y3jswAPk4iChXB30EGSNMuM6UaLGZn+XQnok4e1fqIwgwIReO+jy0vcxoSgydyHc8mqrM6NPSulQNzly+P189ffzaZdjU/abKez+sIm7mGO5c42/5qWF87l9ViUXxt4nLA42HsYJ5u6TqgkJudkwooRt+2p4/ekfNd0veYy5OfD24zzw1HPMWPYxQ0LrCeHUb6+tuckC8cwcuOzHTHTuZ+Hqbc3n4hhz03nlU4w/fTvdd1dgebH6Vp9kZEaflNIlfjfRtbqUZ00JnuPgzJ3HgSv/kw7ePr9deMc6WFzkb41tH+HOtXImLC7C81xGmRw293uSm64Yj1XyAq5TS9TY7D71Ynq3/SdUqRA//wOqS5mbW4L5wsGbWYIZ/RhSs+e4cmMWF9HOc/l3k8N1VxUz+OxxeMWlOE4U19jk6nyl7JA3iI55g7jp058xesMPMP9r/CfhkY9Cze6G+UXVpbCqtGHX2sa5iVRAdSlmVQkTHAeic6H942CHcZxaPAkRTkJmtFJKl7phm84hLAyWACZG6K37MAJi2eC5/mxpALf26zOmIxWwuAjjOVhAWBwmn7MD66yb/buj9Uu5pSzMd/fm8U/p+Iwq+RJyY2NAwPGieIuKsI4jN2ZxEXgONtBOHAZbH0JeEdakV6koe5nH13fnv08dkJSmHpUBIhVcs+0JwEUA4xxCFheBMWDZgIAbxWAQ+PoKDQm5A0NIwBgHanYjk17hhXklrLYv5DFdZijA6oZtYuq/ZIlgjIdgMK4fnnpiNdzRRCr8n9+3Bc9zseLvYlkWclb8NXmD6AiMqfodn7y/DL7dt00+lkqxRrkxgCUcV24O7dpMjudix39erITX5A2iZ/4BBm+Yw5r33uTKEWPb7KOpFNpUjngeiF8PGQHxXMCA6zVURgY8hJiE+KJLIT0SrjWJufMQrLpVHPIGsf6ck4hUleG9Ox3rrNZtk9GqSklEbgAeBvoDg4wxla15vxNK3bBNNwriJ8UyHkaMf9rjwQHx735HTW8YJFE8Nj5IwsI1NiFxsSwbufSuw9t0i8cyyaklVlNK9OVqwhfflBEd15qbVmiUGzEGjIeIwRj8iw4giH8H3ERuxAiOsRHxsCwLGuXmzEXjuCenFvPefIhNgPxxmpug6zMUQrkYpxY3XjGJePFvmvrrjSs2q7qN4Ze7CrBerGamNRXLi/lZskIYD6KesLnLEPr1bbjRHZK7kSnyCJTFcMXCGj0dKbi1RUVt7ZPSWuA64H9a+T4nnsRVffdtgapiwCD+/S4i4BhhTXggtUN+xEWdO9N+0WTcbdVYTi2CB8awqusYCvMHIB26wRtTGlbsvWgcuFEsPMJ4sLoY1s7NlBFVmpuWaiI3Eq+GjPh3uw7Cts6X0GX0Q/7IvEWTcbZWY8Wbim0s9px7I6flnQPtu34tN+JGCeFhjIepfB5ZPUdzE3QJK8+z91OslbMQwIvfyFjiX31CBRMpzB/Prz4oY+1fysA95P+8B1w8kTe3hHl3i8sjX86GquUQz8YFsTXkEMPCYIwLi++FHhe0KDOtqpSMMR+Cv9ioaoFGQ8Rxo2DZfsXkOWDl8Ix1AzuXfMSc8FSMOFjx1j5XBAmFueTaHyJnXOKPmElcsRfj7+UUbwMWTMasAK25aaWj5MYlxOTtI5HnVlAankoYB7s+NxZWKMxpQ2/13+MIufGcQ4jmJrvEcxOKVMCauf4IS9vPDZ6D2GHoeREUj6WLU8tQ/CcpAxjLZk/f6/nn977i13lLsXbEDlsdvNP5wzGVT2DifVYYr8WZabM+JRG5A7gD4IwzzmirwwZDo71QAGRTOaE+Q3nq9AIir0wlpzp+suPNM5w1DHv4TxtOemKzjh2G/PGQPx6pLsWpmo1lvIY24ADR3DTjCLnJ7TOU/7DOZe+bj5Kz+fDcWGcPQ674yVFz460sxayeTQjPv1gFKDeamaNIeGqSuvNal6H6Pkuvvm/SRXjR+TZTf38QzxguGno1LJjVkJk+Q+mQN4jPvvUzev7xgYZRny3MjBhjmn+ByB+Ank18635jzML4a94B7j3WNt6CggJTWanNwccsUgEzR8fDgn/Cr3q8YThn4giZunA1HqXX6OsiUmWMKUhVkTU3GUBzo5k5XvWj7GoBD8TCs8PM7vR9utsHOKtgBOcVfue4MgPHl5ujPikZY75z/J9MJVXeILjlNX8eAQI98w/vB6hr7z/SSs9pWAFac5MBNDfqeCU+fbfvCjW7sdp3ZWJdbpb8Dnq+ktLM6JDwoEg82Y37ATKgvV9lKM2NOl6NK5Y2zk2rlhkSke+JyBbgUuA1EVmSnGKpZgV8p1DNTZpoblRLtHFuWjv6bgGwIEllUceqcQd3wO52NTdporlRLdHGudHmu6DSnUJVS2huVEu0YW50lXCllFIZ46hDwlNyUJHPgc0pPkw3YFeKj5Es6SjrmcaY7m18zFbR3HyN5uYoNDNfk66yHnNu0lIptQURqUzlfIpkClJZs12QzkWQyprNgnQeglBWbb5TSimVMbRSUkoplTGyuVJ6Jt0FOA5BKmu2C9K5CFJZs1mQzkPGlzVr+5SUUkoFTzY/KSmllAoYrZSUUkpljKyqlEQkT0SWisg6EflARO5Od5mORkRsEVklIovSXZYTleZGtUTQchOUzGTbMkMOUGSMWSkipwBVIvKWMWZdugvWjLuBD4GO6S7ICUxzo1oiaLkJRGay6knJGPOZMWZl/N9f4p+AXukt1ZGJSG9gNPBsustyItPcqJYIUm6ClJmsqpQSiUgfYCCwIr0ladYvgPsAL90FUT7NjWqJAOQmMJnJykpJRE4GXgImG2P2p7s8TRGRq4GdxpiqdJdF+TQ3qiUyPTdBy0zWVUoikoMfkBJjzPx0l6cZQ4CxIrIJ+D0wXERmp7dIJy7NjWqJgOQmUJnJqsmzIiJAMbDHGDM53eU5ViIyDLjXGHN1ustyItLcqJYIYm6CkJlse1IaAkzAvxNYHf8zKt2FUhlPc6NaQnOTAln1pKSUUirYsu1JSSmlVIBppaSUUipjaKWklFIqY2ilpJRSKmNopaSUUipjaKWklFIqY2ilpJRSKmP8P24yYBiRVo1RAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 6 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We would expect the models with increasing complexity to better fit the data and result in lower RSS values. This can be verified by looking at the plots generated for 6 models.\n",
        "\n",
        "This clearly aligns with our initial understanding. As the model complexity increases, the models tends to fit even smaller deviations in the training data set. Though this **leads to overfitting**, lets keep this issue aside for some time and come to our main objective, i.e. the impact on the magnitude of coefficients.\n",
        "\n"
      ],
      "metadata": {
        "id": "vzt4JtBRMeuA"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#Set the display format to be scientific for ease of analysis\n",
        "pd.options.display.float_format = '{:,.2g}'.format\n",
        "coef_matrix_simple"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 578
        },
        "id": "m5lNtk8BNxtD",
        "outputId": "d1043c10-39e2-4d05-8c6d-58de217efbc5"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-0cc61585-2f84-48c0-a03c-e884ff1d0904\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>rss</th>\n",
              "      <th>intercept</th>\n",
              "      <th>coef_x_1</th>\n",
              "      <th>coef_x_2</th>\n",
              "      <th>coef_x_3</th>\n",
              "      <th>coef_x_4</th>\n",
              "      <th>coef_x_5</th>\n",
              "      <th>coef_x_6</th>\n",
              "      <th>coef_x_7</th>\n",
              "      <th>coef_x_8</th>\n",
              "      <th>coef_x_9</th>\n",
              "      <th>coef_x_10</th>\n",
              "      <th>coef_x_11</th>\n",
              "      <th>coef_x_12</th>\n",
              "      <th>coef_x_13</th>\n",
              "      <th>coef_x_14</th>\n",
              "      <th>coef_x_15</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>model_pow_1</th>\n",
              "      <td>3.3</td>\n",
              "      <td>2</td>\n",
              "      <td>-0.62</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>model_pow_2</th>\n",
              "      <td>3.3</td>\n",
              "      <td>1.9</td>\n",
              "      <td>-0.58</td>\n",
              "      <td>-0.006</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>model_pow_3</th>\n",
              "      <td>1.1</td>\n",
              "      <td>-1.1</td>\n",
              "      <td>3</td>\n",
              "      <td>-1.3</td>\n",
              "      <td>0.14</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>model_pow_4</th>\n",
              "      <td>1.1</td>\n",
              "      <td>-0.27</td>\n",
              "      <td>1.7</td>\n",
              "      <td>-0.53</td>\n",
              "      <td>-0.036</td>\n",
              "      <td>0.014</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>model_pow_5</th>\n",
              "      <td>1</td>\n",
              "      <td>3</td>\n",
              "      <td>-5.1</td>\n",
              "      <td>4.7</td>\n",
              "      <td>-1.9</td>\n",
              "      <td>0.33</td>\n",
              "      <td>-0.021</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>model_pow_6</th>\n",
              "      <td>0.99</td>\n",
              "      <td>-2.8</td>\n",
              "      <td>9.5</td>\n",
              "      <td>-9.7</td>\n",
              "      <td>5.2</td>\n",
              "      <td>-1.6</td>\n",
              "      <td>0.23</td>\n",
              "      <td>-0.014</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>model_pow_7</th>\n",
              "      <td>0.93</td>\n",
              "      <td>19</td>\n",
              "      <td>-56</td>\n",
              "      <td>69</td>\n",
              "      <td>-45</td>\n",
              "      <td>17</td>\n",
              "      <td>-3.5</td>\n",
              "      <td>0.4</td>\n",
              "      <td>-0.019</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>model_pow_8</th>\n",
              "      <td>0.92</td>\n",
              "      <td>43</td>\n",
              "      <td>-1.4e+02</td>\n",
              "      <td>1.8e+02</td>\n",
              "      <td>-1.3e+02</td>\n",
              "      <td>58</td>\n",
              "      <td>-15</td>\n",
              "      <td>2.4</td>\n",
              "      <td>-0.21</td>\n",
              "      <td>0.0077</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>model_pow_9</th>\n",
              "      <td>0.87</td>\n",
              "      <td>1.7e+02</td>\n",
              "      <td>-6.1e+02</td>\n",
              "      <td>9.6e+02</td>\n",
              "      <td>-8.5e+02</td>\n",
              "      <td>4.6e+02</td>\n",
              "      <td>-1.6e+02</td>\n",
              "      <td>37</td>\n",
              "      <td>-5.2</td>\n",
              "      <td>0.42</td>\n",
              "      <td>-0.015</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>model_pow_10</th>\n",
              "      <td>0.87</td>\n",
              "      <td>1.4e+02</td>\n",
              "      <td>-4.9e+02</td>\n",
              "      <td>7.3e+02</td>\n",
              "      <td>-6e+02</td>\n",
              "      <td>2.9e+02</td>\n",
              "      <td>-87</td>\n",
              "      <td>15</td>\n",
              "      <td>-0.81</td>\n",
              "      <td>-0.14</td>\n",
              "      <td>0.026</td>\n",
              "      <td>-0.0013</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>model_pow_11</th>\n",
              "      <td>0.87</td>\n",
              "      <td>-75</td>\n",
              "      <td>5.1e+02</td>\n",
              "      <td>-1.3e+03</td>\n",
              "      <td>1.9e+03</td>\n",
              "      <td>-1.6e+03</td>\n",
              "      <td>9.1e+02</td>\n",
              "      <td>-3.5e+02</td>\n",
              "      <td>91</td>\n",
              "      <td>-16</td>\n",
              "      <td>1.8</td>\n",
              "      <td>-0.12</td>\n",
              "      <td>0.0034</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>model_pow_12</th>\n",
              "      <td>0.87</td>\n",
              "      <td>-3.4e+02</td>\n",
              "      <td>1.9e+03</td>\n",
              "      <td>-4.4e+03</td>\n",
              "      <td>6e+03</td>\n",
              "      <td>-5.2e+03</td>\n",
              "      <td>3.1e+03</td>\n",
              "      <td>-1.3e+03</td>\n",
              "      <td>3.8e+02</td>\n",
              "      <td>-80</td>\n",
              "      <td>12</td>\n",
              "      <td>-1.1</td>\n",
              "      <td>0.062</td>\n",
              "      <td>-0.0016</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>model_pow_13</th>\n",
              "      <td>0.86</td>\n",
              "      <td>3.2e+03</td>\n",
              "      <td>-1.8e+04</td>\n",
              "      <td>4.5e+04</td>\n",
              "      <td>-6.7e+04</td>\n",
              "      <td>6.6e+04</td>\n",
              "      <td>-4.6e+04</td>\n",
              "      <td>2.3e+04</td>\n",
              "      <td>-8.5e+03</td>\n",
              "      <td>2.3e+03</td>\n",
              "      <td>-4.5e+02</td>\n",
              "      <td>62</td>\n",
              "      <td>-5.7</td>\n",
              "      <td>0.31</td>\n",
              "      <td>-0.0078</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>model_pow_14</th>\n",
              "      <td>0.79</td>\n",
              "      <td>2.4e+04</td>\n",
              "      <td>-1.4e+05</td>\n",
              "      <td>3.8e+05</td>\n",
              "      <td>-6.1e+05</td>\n",
              "      <td>6.6e+05</td>\n",
              "      <td>-5e+05</td>\n",
              "      <td>2.8e+05</td>\n",
              "      <td>-1.2e+05</td>\n",
              "      <td>3.7e+04</td>\n",
              "      <td>-8.5e+03</td>\n",
              "      <td>1.5e+03</td>\n",
              "      <td>-1.8e+02</td>\n",
              "      <td>15</td>\n",
              "      <td>-0.73</td>\n",
              "      <td>0.017</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>model_pow_15</th>\n",
              "      <td>0.7</td>\n",
              "      <td>-3.6e+04</td>\n",
              "      <td>2.4e+05</td>\n",
              "      <td>-7.5e+05</td>\n",
              "      <td>1.4e+06</td>\n",
              "      <td>-1.7e+06</td>\n",
              "      <td>1.5e+06</td>\n",
              "      <td>-1e+06</td>\n",
              "      <td>5e+05</td>\n",
              "      <td>-1.9e+05</td>\n",
              "      <td>5.4e+04</td>\n",
              "      <td>-1.2e+04</td>\n",
              "      <td>1.9e+03</td>\n",
              "      <td>-2.2e+02</td>\n",
              "      <td>17</td>\n",
              "      <td>-0.81</td>\n",
              "      <td>0.018</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-0cc61585-2f84-48c0-a03c-e884ff1d0904')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-0cc61585-2f84-48c0-a03c-e884ff1d0904 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-0cc61585-2f84-48c0-a03c-e884ff1d0904');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "              rss intercept coef_x_1  ... coef_x_13 coef_x_14 coef_x_15\n",
              "model_pow_1   3.3         2    -0.62  ...       NaN       NaN       NaN\n",
              "model_pow_2   3.3       1.9    -0.58  ...       NaN       NaN       NaN\n",
              "model_pow_3   1.1      -1.1        3  ...       NaN       NaN       NaN\n",
              "model_pow_4   1.1     -0.27      1.7  ...       NaN       NaN       NaN\n",
              "model_pow_5     1         3     -5.1  ...       NaN       NaN       NaN\n",
              "model_pow_6  0.99      -2.8      9.5  ...       NaN       NaN       NaN\n",
              "model_pow_7  0.93        19      -56  ...       NaN       NaN       NaN\n",
              "model_pow_8  0.92        43 -1.4e+02  ...       NaN       NaN       NaN\n",
              "model_pow_9  0.87   1.7e+02 -6.1e+02  ...       NaN       NaN       NaN\n",
              "model_pow_10 0.87   1.4e+02 -4.9e+02  ...       NaN       NaN       NaN\n",
              "model_pow_11 0.87       -75  5.1e+02  ...       NaN       NaN       NaN\n",
              "model_pow_12 0.87  -3.4e+02  1.9e+03  ...       NaN       NaN       NaN\n",
              "model_pow_13 0.86   3.2e+03 -1.8e+04  ...   -0.0078       NaN       NaN\n",
              "model_pow_14 0.79   2.4e+04 -1.4e+05  ...     -0.73     0.017       NaN\n",
              "model_pow_15  0.7  -3.6e+04  2.4e+05  ...        17     -0.81     0.018\n",
              "\n",
              "[15 rows x 17 columns]"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "It is clearly evident that the size of coefficients increase exponentially with increase in model complexity. Compare Power 2 with Power 15 Coefficients.\n",
        "\n",
        "**What does a large coefficient signify?(Power 2 or 3 or 4)**\n",
        "\n",
        "- It means that we’re putting a lot of emphasis on that feature, i.e. the particular feature is a good predictor for the outcome. \n",
        "- When it becomes too large, the algorithm starts modelling intricate relations to estimate the output and ends up overfitting to the particular training data."
      ],
      "metadata": {
        "id": "b0fyONkdMfGq"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Ridge Regression**\n",
        "\n",
        "- Ridge regression performs ‘L2 regularization‘, i.e. it adds a factor of sum of squares of coefficients in the optimization objective. \n",
        "\n",
        "- Thus, ridge regression optimizes the following:\n",
        "      Objective = RSS + α * (sum of square of coefficients)\n",
        "\n",
        "  Here, α (alpha) is the parameter which balances the amount of emphasis given to minimizing RSS vs minimizing sum of square of coefficients. \n",
        "  \n",
        "  α can take various values:\n",
        "  \n",
        "  α = 0: The objective becomes same as simple linear regression. We’ll get the same coefficients as simple linear regression.\n",
        "\n",
        "  α = ∞: The coefficients will be zero. Why? Because of infinite weightage on square of coefficients, anything less than zero will make the objective infinite.\n",
        "\n",
        "  0 < α < ∞: The magnitude of α will decide the weightage given to different parts of objective.\n",
        "The coefficients will be somewhere between 0 and ones for simple linear regression.\n",
        "\n",
        "This gives some sense on how α would impact the magnitude of coefficients. One thing is for sure that any non-zero value would give values less than that of simple linear regression. \n",
        "\n",
        "Note the ‘Ridge’ function used here. \n",
        "  - It takes ‘alpha’ as a parameter on initialization. \n",
        "  - Also, keep in mind that normalizing the inputs is generally a good idea in every type of regression and should be used in case of ridge regression as well.\n",
        "\n",
        "Now, lets analyze the result of Ridge regression for 10 different values of α ranging from 1e-15 to 20. These values have been chosen so that we can easily analyze the trend with change in values of α. These would however differ from case to case.\n",
        "\n",
        "**Note** \n",
        "  - Each of these 10 models will contain all the 15 variables and only the value of alpha would differ\n",
        "  - This is different from the simple linear regression case where each model had a subset of features."
      ],
      "metadata": {
        "id": "17KNhEWQK13O"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "dUb6YeuhKxF1",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 297
        },
        "outputId": "85633f1e-78f7-4333-9baa-aac7bc097656"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaUAAAEYCAYAAAD8hukFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydeXxU5fX/3+fOJAHZdwQCYRdUFokRlyhYV9TiVguKglqX2lWp/dmqrVpxaUu/2lattsqiAooLsuOKREBD2JRVQIJh30RAIMnMfX5/3DtkMkyWSSaZJef9Yl7JzL3PvWdyP5zzPOfZxBiDoiiKosQDVqwNUBRFUZQAGpQURVGUuEGDkqIoihI3aFBSFEVR4gYNSoqiKErcoEFJURRFiRuqFZREZL6I/CwahojDOBH5TkRyo3HNCu43XkQej/a5SvmoZpSKUI3UbSoMSiKSLyJHReSwiOxy/5ANI7mJiGSIiBERbzmnnQdcDHQwxmRFcv1ERkQGi8gnIvK9iORX81qpIvKW+8yMiAwKOf6IiBS7zzLw6lKde5Zhh2qmBqllzYiIPC0i+9zX0yIi1bmne13VSA0S6XMTkRtFZIuI/CAi00SkedCx5iLyrntsi4jcGEHZX4pInogUisj4ythe2ZbSVcaYhsAZQCbwUCXLRUInIN8Y80OkBSsQZbzzA/AKcH+UrvcZMALYWcbxN4wxDYNe30TpvqGoZmqO2tTMncDVQF+gD3AVcFeU7qsaqTkq/dxE5FTgReBmoA1wBHg+6JTngCL32E3AC26ZypTdDjyOo9dKEVH6zhizDZgDnBZ6TEQsEXnIjZi7RWSiiDRxDy9wfx5wa0Znh5S9HfgfcLZ7/FH38ztEZKOI7BeR6SLSLqiMEZFfiMgGYEM4e0VkqojsdGuUCwJ/yDDnDRKRrSLyRxHZ69bibgo5rZmIzBKRQyLyhYh0DSr/rIgUiMhBEVkqItnl/R2DMcbkGmNeBcIGBxE5RUQ+cP8G60XkhnKuVWSMecYY8xngr6wNNYlqJuE1MxIYa4zZ6j7LscCoytpaGVQj0dcIkT23m4AZxpgFxpjDwMPAtSLSSEQaANcBDxtjDrs6mY4ThMotC2CMeccYMw3YV1nDIwpKIpIODAGWhzk8yn0NBroADYF/u8fOd382dWvni4MLGmNeBu4GFrvH/ywiFwJPAjcAJwNbgCkh97waOAvoXYbJc4DuQGtgGfB6OV+vLdASaI/zQF8SkZ5Bx4cBjwLNgI3AmKBjS4B+QHNgEjBVROoBiMh5InKgnPuWiSuID9xrtnZteF5Eyvq+leEq9z/jahH5eTWuUylUMwmvmVOBlUHvV7qfRQ3VSI1oJJLnVupcY8wmnJZRD/flM8Z8Xca1yitbNYwx5b6AfOAwcADnAT4P1HePzQd+5v7+EXBPULmeQDHgBTIAA3jLuc8o4LOg9y8Dfw1639C9Xob73gAXVmR/UPmmbpkm7vvxwOPu74MAH9Ag6Pw3cWoHgXP/F3RsCLCunHt9B/StrG1umYtw0gzBn/0UyAn57EXgz5W43lZgUMhnvYF2gAc4B9gBDI/Ezkp+F9VM8mjGD5wS9L67+zcR1Uj8aiSS5+b+je8O+Wyba382sDPk2B3A/IrKhnz2ODC+MrZXtqV0tTGmqTGmkzHmHmPM0TDntMMRV4AtOMJpU8l7lHs94zQN9+HUOAIUlFVYRDwi8pSIbBKRgzj/CcCptYTjO1M677zFtSFAcL79CI6YA/f6nYisdZvzB4Am5dwnEjoBZ4nIgcALp7ncVkQ6StCAhcpczBizxhiz3RjjN8YsAp4Fro+CneFQzSSBZnACR+Og942Bw8b1NNVENVJzGonkuYWeGzj/UAXHKipbJaLZkbcd5z9EgI44tYRdlH7gVbqem5ZogROFA5T3H+NGYChubRLngX4HlDUCpZmINAgSUEdgVUVGunne3wM/AlYbY2wRKe8+kVAAfGqMubiM4xGNVgqDITp2VhXVTPxrZjVOZ3lgOHVf97PaQjVSNY1E8twC5wbu3QVIA74GbMArIt2NMYE+tuBrlVe2SkRz8uxk4F4R6SzO0M4ncEZ6+YA9OF8ukuHHk4FbRaSfiKS51/vCGJNfyfKNgEKcWtBJbvmKeFScIbLZwJXA1EreJ/AdvSLyJ06sOZSJOB259YAU563UE5FU9/BMoIeI3CwiKe7rTBHpVc710gJ5ZyDVvZ64x4aKSDNxyAJ+DbxXWVtrANVMnGsGmAjcJyLtxRkQMBon7VRbqEaqoBEie26v4/Q1Z7tB+jHgHWPMITeYvgM8JiINRORcnKD8akVlwRnB6GrLA3hcbZXbGIpmUHrFNXQBsBk4BvwKwBhzBKcDb6GbUhhY0cWMMR/ijOR4G6fvoytOp2BlmYjTVN4GrAE+r+D8nTg1nu04f+i7jTHrKnGfecBcnJrBFpzvfbz57z6s8lIl5wNHgdk4taijwPsA7oO9BOd7b3dtfBqnJlIW691rtHdtO0pJzXAYTmfqIZy/z9PGmAmV+I41hWom/jXzIjAD+Aqnhj/L/ay2UI1UTSPlPjc3hZsNYIxZjTMg5HVgN05AvCfoWvcA9d1jk4Gfu2UqU/YhHD09gDPt4CgVDP2X6KSGExtxJgy+ZozpEGtblMRANaNUhGqkaujad4qiKErcoEFJURRFiRs0facoiqLEDdpSUhRFUeKGmCw42LJlS5ORkRGLWyc8S5cu3WuMaRVrO2oT1Uv1UM0okRBrvcQkKGVkZJCXlxeLWyc8IrKl4rOSC9VL9VDNKJEQa71o+k5RFEWJGxIzKBXkQs5Y56eiVITqRYkU1UzMSLxNrApyYcKPwV8EnlQYOR3S68yGkkqkqF6USFHNxJTEaynl5zhiMX7wFcL8J7U2o5RNsF78RbByktaAlfJRHxNTYhqUjDHsXLWAovl/K3noFTWbM7Kd2gsWYMM3851ajYqmbhBOH+VpJqAX8YDlgeWT4OMxqpm6QqR6AfUxMSYq6TsReQVn9dvdxpgTtjQui7VLPqLzrOF4xQefjYXLnoK5D5Q0my97Co7uw9fxXNZ5e2Ebw6m2wdNvGOz4ErYvB2M75+fnaBM7QaiqXsKmVaDkM8sD/UdA3+GltdBvGMdX/F86oaTVpJpJGKqkmYr0EuRjyMg+UTPqY2JCtPqUxuNsUTwxkkLdjizHwoeFDb5jsHxi6Wbz7NEY24/B4rWiUXxt0pmU9gSCD8vjBSsFbJ8jroxItq9XYsx4qqCX0mmVY04qrkl6UHrOD3njYPlrTnBq2/fESo4nteS9aiaRGE+kmgmXhmuWcYKPwbbBsmDIWGjTu3QlR31MrROVoGSMWSAiGZGWS+16AYXzn8KDjWCcmonldXZIEcHYfgSDFz9PpI5jZ9vBpOx0gpjt92Nl3uI4pdBajhLXVFUvZGQ7jsLvB4yTirv8r47D8B1zPsNg/EWQ9woGC8E42vIXOTXikdMdZ6WaSSiqpJlAGs5XyPE0nOUt5WOwXS3ZNsy6F3oOKQlaNjBAfUxtE9uBDulZLG9+Rck2j7Yf2p4GA0ay49zH8Rnr+NaoFjbtdn6CJYKNRaHxMEsGQfbo8GLRIZ3JR3oWu7v9BBNIxdk+2LkC+g2jsOtl+CQVPwLG0YwYG4zBj1AsXg62GehoJZxmVC/JR3qWUwnpOgiDBcbG7y9mLZ1401zIg0WjKHZ9DIAxNva6WRixnD5ITyr0vVH1UsvU2pBwEbkTuBOgY8eOxz8/fMr1FC6cQz3xIdiwfTlm1xrGpDxCO+8d/MF+ycnpAmAjBrC8TG3+Cx75LIUmPfZyXr1vnFQO4vQngA7pTHDC6cVvGx77tg9/Yypp+EE8mGWvg+3DNl7+XHwz5zfezqXH5mFhI1Kyr/WfCkcwe/IPPDZ0G0NbbFO9JCHhNLPgaGdm7ruCR81npFCMRww9/RvoKls41uc55v3QgiFb/gqBqo4x+G0/mzpcR8aPbictPcsJPPk5UL+F09qu36J0Wlj1ElVqLSgZY14CXgLIzMw83jhK7zOYmz75I893eJ+2ez8HY2P7Cul4bBnn3/Yksr+Xm/d1m9mAGMOw0xrwalFD/jd5CufKo4i/yLng8teh/02lhwFrB2XCEU4vHkv4/c9u4dHXvDTfnUs72cswzyd4xSZVfNyf3YoWl/8D8sY7qRhjI4BHhPvObcX6/AZMeONNrkgbg4dixwmpXpKGcJrx24Yv6cG8zBe5dPd4vFtzEGOTio9bTv4Wsv8Iee1g1r0Y41RkBMN7my3ee+Mofx84m7M/u60kBSiW8zK2DoCoIWI+T6lb64ZsOek0/mVfj/GkYeOhyHg5rVsXBm4b73Q83joHMkeBJ+14szq16wW8MGIAfXxfYfzFJRf0FwGmZBiwdlAmFR1bnMQTv7qdH//q7/S45E4sbypGPHg8Xlr4dzu12sxRcMX/OX0HYoEnjVan/4g37jqbezrvRIwvkABUvSQ5g3q2Yu5vz+fqq66l/iUPlfgQywPfby2lF3H1YnnrcdmV19Okfgo5H0zDHwhI4AQiOxCcVC81QVT2UxKRycAgoCWwC/izMeblss7PzMw0wYslTs79lj+88xW/7vEd9uYFNGzWhruO/Ndp/QQ3jwPN6KBOx/fnTeeCRbeSKq6j8aTBqJnOhZOwQ1tElhpjMmNtR3Worl5KUZDrpOKWTyoZJVWOXsy3X+AfdwUeu9jpePKkIUmsF1DNlCICvfj8Nu/NnMaQZXeRIsV4MIhbySlzKHkSEGu9RGv03fDqlP9pZjrvLNvKcxuFrIzbGNctB8kJk04JvIK4+JKreHrLP+hYMJ3LT2tLs3NGlpxT1gCIJHU+iUJ19VKK9Cznedq+SulFOp6F97bZLJ/xPKu3H6R+/xFcp3qJe6KmmQj04vVYXDf0Wta1a8xHc95hj92A2/o1puMZl5atBdVLtYmLte8sS5h8x0D8xpDm9UCBBYvGlj+fxH34kpHNHTcO49Jn2jNheyrTfX7ScsaGF4WuaZWcBIb+VjT/KMhh9Pv5OMZNWcH0xdvp2HwuZ7L6RM2oXpKTCPVySkY2jX49lpGv5DIl7whvtDpA3/wwPkb1EhXiIiiBUys5bkxgKGdZNY6Qh99i5HT+9pM+/Gv861gTnwR84UURug6adlAmBxXpBU7QjIyczl+vH0DK9iWc9tGDGPEjoZpRvSQnVdBL+5HTmXLnQMb8ZwI95j2IsXyIJ031UgPETVA6gTCpl+OEefiDs7Pwd96JbCsGKWNUTGVrSEriUZ5eIKxm6qVn8fBp+0lZ7EOMjfEXIcGaUb0kL1XQS8v0LP7S7wCpn6leapKYj76rEsGLbAY9/OyLr8YnXnxYGE/KiaII1JAufFCb1nWNMjTTtPeFiCcVn7Hwibe0ZlQvdZcy9NKw5yDEm4rPncB/uO3ZJWVUL1EhflpKkXQQltH8Tut8NluvncrUtyazt3EWj7TLJCVcWRVL4hNph3JZKZv0LDyjZjBrxlTGbWvPw/Sgb2g51UtyEAUfQ3oW1sgZbF/+PvfmNqTxwhT+29VgWVJSTvVSLeIjKFWlg7CMh9+hzyC62N15dsoK6s1Zx8NX9q4ho5WYUdUO5bIcRnoW2bf15/H/W8DoqSuZ+avzqJfiib7dSuyIoo8hPYsO6Vlc1SqfP723mhcXfMPPB3WtGbvrIPGRvgvXQVgV3PWohrbYxqhzMnj5s828t2JbdG1VYk+09ALHNdN4z3Keuq4PG3cf5rlPNkbPViU+iLKPoSCXmwd2YsjpbRn7/npWFhyIrr11mPhoKUWjgzCkJvTQzdNYs705/+/tL+nWuiGntmsSfbuV2BCtDuUQzVwwcjrX9m/PC/M3cWWfdvRs2yi6diuxowZ8jIyczpPX9GfFtwu4780VzPp1trawo0B8tJSi0UEYUhPyfruQ5246g2YnpXLnxKXs/6Go9Pm6ym/iEq0O5TC154eu7E2jel4efPcrbDtotRPVS2JTAz6G/ByanJTC09f3YdOeH/i/D74ufb5qpkrER0sJqt9BGKYm1KpRGv8ZMYCfvLiYf45/nT+dvh+rczbsWuMs8mpsZ8kQHSmTeESjQzmMZpo3SOUPQ3rx+7e+5OMPZ3BR/Q1w7CAs/rfqJdGpAR8DkN29FcOz0lm6cB47zTu07XOx+phqED9BqbqUMVqmb3pTXjjfxzkLfw8f+8DjdVYcN36nnL9QJ7nVVcrQzE8GdGDFwvc5b9EDGNwtVQKoXuou5Uy6ffD0w3i/HEPKEh9m2TOI+pgqkzxBCU6sCblDQH9UtBW/BHasLaZkWy+c1X51klvdJVgzQUtX/abbLry5PkTs0uerXuo2ZfiYht9vxabEx4j6mCqTXEEpmOBOScuD5UnB5y/Gj0Wqx0L8PrAsGDJWazDKCZ3YbS57iiIrBZ9dhEcMgjjbHahelAAhPkY8Kfj9xfiwSPV4EH+x+pgqkLxBKbhT0gYZcAv7rdb8clEDzuzQjPt77tGVfJUSQjuxj+7j2PB3GTfpNRo0bcXtZzRBOqtelCDC+Ji90pqfL6zPVX3acWv7repjqkDyBqXQTsm+N9I6PYsL6m/kb/PWc+pZFzMk/eRYW6nEC2E6sRunZ9HoonY8NnMNXVufyeD01rG2UoknwviYNulZdDu6kieWb+PCi++mU4sGsbYy4YiPIeE1QRlDQO86vwuntW/MI9NXc7jQ55yrQzeVMvQyYmAnMlqcxJjZa/H53f4l1YsCZWrmd5f0xGtZjH3fHSKueomI5G0pQdghoF6PxeNXn841zy/kmQ++5qG+h3UPFMUhjF5SvRYPXN6Lu19bytvLtvLTtjtVL0oJYTTTunE9bjsvg+c+2cRvT/mOLrNuVL1EQPK2lMqhX3pTfpqZzvhF+Xy3+uPoLVmjJCWXntqG/h2b8o8PvqZ40wLVi1Ihd57flSb1U1j26QzVS4Qkf1Aqo+l83yU9SPVavLytfdgl6pU6Shi9iAh/uLwXuw4WMutgF9WLUpowmmlSP4U7z+/CpF0dsa0U1UsEJHf6LjBk01dYMjQzcxQArRvV42fZXfjnR36uuWESXX9YriNl6johQ3zpPwL6Dof0LLI6N2dQz1Y8suIAF9/0Dg22L1a9KOX6mJHnZPC/nFMZ0+ppHj51n+qlkiR3Syk/xxELNtg+Z9mPoNrMXV32cm+9GbyZVwDZo1UwdZ3QYeF54xyH42rmd5f0pMvR1SxbMEMdjOJQjo9pmOblob6HSStYxMaT+qleKklyB6WMbKf2EsDYJTndglwaTL6WX/EGv932OzYt+zg2NirxQ2CIL+6GbZhS/QCn2euZUu9Jzs7/D2bCVTqaSqnQx1z71c+5L2UqHWcOV71UkuQOSulZTnPa8jpLfXjSSnK6bq3YwiYFH6sXzY6trUrsCQzxzRzlaCW0HyA/hxR8eMXG9hVrp7VSoY8RfzFebCy7mD2rPoqtrQlCcvcpgeNg2vQ+cRHFoIlvxvIyYXt7+u07QscWJ8XUXCXGBIb49r0xrGbEk4rfV0iR8eBvezYNY2utEg9U4GOMv4hi42HSrnR+E1NDE4PkD0oQfsn6oBV/D7bM4svXDvHKws088uNTY2OjEl+Uo5l9X33I3Tn1OX9LS37bPTbmKXFGOXqR/BwmbW3Pv75qzPUHjtK+af3Y2Jgg1I2gVBaukFoAV/ZZwdd5H1HYaDZp3S7QTkklPOlZtE7PosXePPI+m0eh5z3Vi1I2ro+57MBRnvjqE96fO13XxKuAuh2UgvhFt320X/MXUj71wcKxOvNaKZf7e39P+qbHVC9KpWjftD6/6LafYWvvxaz3I7q6Q5kk90CHCOi2YyZp4gx8MDrzWqmAHjtnkibFqhel0ow8aTFpFCG6ukO5aFACZ6jm8tcQwBgw4tGZ10rZHNeLwRiwVS9KRRTk0mLDm4iAAYzlVc2UgQYlcGosth8BbBFyGlyqzWqlbEL0MtO6ELv9mbG2SolngjVjhG87Xq0+pgyiEpRE5DIRWS8iG0XkgWhcs1YJDA8XD7aVyr/2Z7L74LFYW5XUJLRmgvSCJ40JPwzkg7W7Ym1VUpPQeoGS4eHioVhSePHAWbG2KG6pdlASEQ/wHHA50BsYLiK9q3vdWiVoX5Td17xJnr87U5dujbVVSUvCayZIL+aW99jVpC8v52yOtVVJS8LrBUqGh1/4IB9k/pdJO9qyatv3sbYqLolGSykL2GiM+cYYUwRMAYZG4bq1S3oWZI+m/emDGNilOVOWfIttm1hblawkvmZcvXg7DeTWczPIzd/PyoIDsbYqWUl8vcBxzVxw0RWclOph/KL8WFsUl0QjKLUHCoLeb3U/K4WI3CkieSKSt2fPnijctoYoyOVPTefS6ruVLNq0L9bWJCsVaiZh9AIMb7eT36bN4KMPZsbalGQlqXxM4z3Lebb9x3y7cj77DhfG2py4o9bmKRljXgJeAsjMzIzPJoi7DH0vfxGvp3l4PqcV53Uf4XweuoSIUqMkhF7g+MK+v5ZCCre8zb61nWjRME31EgMSQjOuj7nIX8R5Hg+zPkzn+quvVR8TRDSC0jYgPeh9B/ezxMNdpFWMn1QMbM7h4NedafzmdbqdcXRJOs0EFvb9dv7LtNg3V/USXZJOL2L8pIph76qP8PVvh/fVq1UzLtFI3y0BuotIZxFJBYYB06Nw3donaFSVeFLZ42/Awbl/AX+hbmccXZJSM0Y8eHd9hfGpXqJMUuoFy0vDYzvY/uk43TI9iGoHJWOMD/glMA9YC7xpjFld3evGhMCoqgG3YHW/iMdSJ3Ly/s8xxnaXpdftjKNBUmqm52V4xKa32QioXqJJsurFMn6Gez+m7Tdvu1tf6JbpEKU+JWPMbCB5NiRaMQV8x0jBOKs8YEGXQTDoD3W6WR1Nkk4zX89DjB9LwA9YnQchg1Uv0SJZ9eIBjPFx8JQRNG7bWfuU0BUdTiSwJTZOP6lthGJJ0YCklE1+Dti2W4EB23hYd8ovVC9KeFy9gLvkEBZTfedB9mjVDBqUTiSkX2lxs6u41X6YI23OiLVlSrySkQ3eNMACy8sYuZ3nNjaPtVVKvBKkF7G8TG37W55Z34wjRb5YWxYX6NYVoQRt/kdGNqn+biz8z2JmfrmDGzLTKy6v1D2CNCMZ2XhWNmTuonx2HTxGm8b1Ym2dEm+E+Jhuvm4cenExM1Zu56dndoy1dTFHW0rhcGdek55FprWBhxrPYdnCebG2SolnApoBfpkynT6sZ9IX38bYKCVuCfIxZ3o28Oemc1mSMxdj4nN6VW2iLaXyKMhFJg7lNl8hxYWT+e6N5TQ7Z6TmfZXwuBMjm/mLmJzm5e+Ld+JLbYW3y/mqGSU8ro8Z6Suk+KjF3jdW0OrcUXVaLxqUyiNoYmQqNilrX4cNbztN78BxHS2jBAgMkjF+UjD83v4v1icGcv4Olz0FR/epXpTShPiYFusmwcZ36rSP0aBUHoFBD75jIAYLg/EVInMfgJ2rwPbpDGylhIBe/EWICJbtx8KA7yjMus85R/WiBBPWxxxDFj4DGz+uk6s8aJ9SeQQ6JDNHYaxUfEYAG7Yt01UelBMJ2tJChozFiNfZyRgcrahelFBK+RhHL2Bg/dw662O0pVQR6VmQnoX0Gc7KV/8f/YpX4CHQGSk6A1spjasXAH/BMqwVE4JqfqoXJQyuZiwEO2+c01oyBrEs6qJmtKVUSaTjWRw4czRFJgU7sBxI5q11qlmtREZa5gh8Vho+Y2FUL0pF9B2O8Th6sT2pMGQsXPhgndOMtpQi4NzBQ7j78z9zVZNvuO7aYXVKKEoVSM9i+9A3eHPqZLoPuIxrrrw21hYp8Ux6FuaW9/jvxInsbJbJo5mjYm1RTNCWUgTUS/HQZ+DF/G7nj9hc/9RYm6MkABn9BrO80238fU1TfH471uYocY6300BM9n1M2NqG9TsPxdqcmKBBKUJGnN2JMz0b+fqtR515KYpSAb895TuGHppCbs7cWJuiJADDzuzIWd6NbJ72WJ30MZq+i5DWB77ktZQxWDuKMRMmIJc9rfNPlLIpyCVrwa0MSCnEP38adHW3TK+D80+UytF8/wpePe5jJtY5H6NBKVLyc0jBh4iN31eEZ/ZoMKbOzSVQKkl+DuIvwosNppg9n42j1aZ36+T8E6WSBPkY21eE1DEfo+m7SMnIRjyp+LGwjWBsu07OJVAqiTs50oiHYvGydsch3WVUKZ8gH+On7vkYDUqR4k52++6s3/OwbxQ+KyW6O0YW5ELO2DqZS05KXL3IhQ8ytffzPLt3ALYnippRvSQfrma29ruPh4tG4bdS65SP0fRdVUjPomV6FkcOLOeWNRn894KjNOw5uPrNandBT03tJBnu5MhLDx7j8ZWf8N/Oz3BXx+3V7yNQvSQv6Vl07HAmqwo+4zdHevCvs3/A6hyFPqUE0Iy2lKrBvRf3YIm/G08dHhKdBxu0oGddaarXJdo0rsd1Azowdm1TdveNws60qpekRkS4Z1A3Zn2Xzqwmw+uMj9GgVA06H13NC53ms27JR2zac7j6Fwza9bauLS1SV/h1j/3cwbvMmv1e9S+mekl6Lmv8LQ81nsPHH8zEtqOw11ICaEbTd1XFbQZf5C/iPK+Hv7/dlIfvHlW9a4bsSBlvzWqlmhTkcvK0n3Kft5Cite+yf107mp9SDaegekluCnKxXnX2cyssfIPPF3TgnEFDqnfNBNCMBqWq4jaDxfhJE+i8dTob395Ct6zLq/eggxb0VJIMVzMebFIoZtusx2jeYIzqRQlP0F5LKVJMSs7T+Du3wNPprOpdN841o+m7qhLUDBaPhxu8C8j46hnMhB/H7agWJcYENIOFRwydDy7BnnCV6kUJT7BeMJzhW1kn/IsGpaoSvHdO/xGk4MeLje0rjMvOQyUOCGim6yACgcn44rOzWYkDgvUijl6wiyjc+GmsLatRNChVh/QsyB4NfYcj3jT8WBQZL+vq9a3yJXM27OGtaW+z+d26ue5V0pOeBYP+gHjTsPFQZLx86T29WpfcfegYe9fmxPXcE6WKBPTiSXMmYBsvU3Z3qv5143iukvYpRQO3RuPb+Cn/XLiXZu+/S3rzk2jQ9ZyILvO/nG+YPfs9Xk99ghR82Kuewxo1I/qQLB4AACAASURBVK7zv0oVcPXi37SAfy/aS9NPptOzbSPSOp8d0WWOFft5aNoqvln2Ma+nPoEtPixvWlzOPVGqQWACdn4O4zafzMdfbueyhmNo0+fiKj3n/etzaDr1esRfjMThXCUNStEiPYu0XWu43/8Uxvbjf20q5tYZSMeznNrIykmAQN/w8w0WfjKb/R9O47etjlDvkB8xNj5/EWzOwYojwShRIj2LlF1rGF38JMb2Y098E26b6WijINdJ6dVvUeZCnIWbFzPt3Tf4Zk8nftllJ6nbfFjYGH8Rkp8TV05GiQLu4IRRnpe545vHsJbYmOX/QgIBpSIfU5DL92s/5oX8k7G+Xch9niK8Ep960aAULQpyYfZoLOMDAWMX8fX7/6Vn28mw7DWwi53zlr8Oo2aWiKAgl+8/n0jmqskMTLGxfvAilhfbD8XGwzdpfdCdm5KQMHrZ/ukrtGs6CZZPAn8xYINY4Cnd+vEvGYdn1mh+Ymyur5+Gt//TmF2p+HxF2HhJjcO5J0oUKMjlpA9/j8GPALbvGDL/Seg1FObc70yGhRN9TN547FmjaWj8/MaksKDHaMhPwecvxocHT/q5pMTsS52IBqVokZ8DtrOJmwEQIaPgXcxWP0LQpLfALGq3dmMm/JhGvmOIGATA9sOAWyg6qT03fejlgkMZGpSSkTB6abHxLQy+0noxdinNmG+/wMwajcf4EQFMERzdh4ycwdzpb/LWvgxeaX+mdhYnI65mBEczYgz2pvlYmxc4fiNAiF7sWfdh2X4sgXri49LOKTB4Jhty5/D/8hozbG97bsiI0XcKg2o3WmRkgzcNsBDxIC264RUnIJWahx2YRV2Qi/3JkxjfMazgM0SgbT/qdb+AHzf5hpO+ejVuOySVahCsF8uLr10WKRSfqBexHM3Ub4G9YCxL3nsejO0EpAD1W0B6FvV7XMBFRR/z/dRfql6SkSDNgICAhY1t+8P6mMMbFrF2yh/BdiowBmfpIr7fCkC3My/j9kZfUG/e7/Bv+aL2v08ZiDFVX7pCRH4CPAL0ArKMMXmVKZeZmWny8ip1amIRyOu66ReDu/21ASMWVqezoVVPSGuMWfxvjO1DAscsy9kzBQOWFxBsfxFiDIiFuB3Y0vGspcaYzFh+zepQFc0ktV4CfUdz7scE0i+AEQ+c/UusooOw52vMt59jjE2x8eCxBI8JtKgEvPXgsqew59yP+Iocf+VJO57CEZGE1Yz6mBACPmbZa5hAl4ABW0DwYHfIoii1KbsPHuPkvZ/hMT5nKDmCiOVUcowNlgeMca5hwHhSsW6dFRd6qW5LaRVwLbAgCrYkPulZ0CQdbB9gI05CDhGwjcG35XPsvHGYhc+A7cMCjAjSZRCccYtzDWM7/Qn+ouMtKMGO28UTq4BqJkBgSsHRfU5t1v1YANs2TP9iHcV5r2G2LATbj4UhRWw8Z4xAug52HAzG0cba97D8xYhTgVa9JCsBH2Ps43pBwAPYxoaCL6i/aQ6d9nxCKsV4xSBYjl4GjHT8i/E7PsYuRnD8k2UXx41eqtWnZIxZC26TUHEIzML2F4HlQRCnRSSAsbEwGMPx5rRleWDwH5yyKyYfLweC8ReD2NhYWHG6eGKkqGbCENCMrxDBxogFViqN66dgHfY5zue4Xizod6NTbsviki0Ieg2F/M+Ot7ZE9ZK8BPkYCfgK24clAsZH6b+UOCm/QWF8jDElA7DiSC+1NtBBRO4E7gTo2LFjbd229gld8BCQ/Bw89Vtg5j6A8RWC2BjEEdSQsSWjZMKUM/WbYx3dH7eLJ9YUdUYvUFoz9VsgR/fhzcjmQoDx7x8fVSVSjl7Ss6BNb6SCqQfJTJ3RTBk+Rtw08PFReFYKnHFzaS2ElKtoqkosqLBPSUQ+BNqGOfSgMeY995z5wO/qfL63IkLnn5QzD6UsYp3vrQzR1kyd1QuUnn/Stm/EeoH414z6mCgSOl8JIl4RPNZ6qbClZIy5qDYMqRMEr86bADtAVhXVTBQJaEb1olSGJPAxOiQ8ViTADpBKHKF6USIlQTVTraAkIteIyFbgbGCWiMyLjll1gATYAbImUM1UEdWL6iVSElQz1R199y7wbpRsqVskwA6QNYFqpoqoXpRISVDN6DJDsSTOd4BU4gzVixIpCaiZaq3oUOWbiuwBttTybVsCe2v5ntWhLHs7GWNa1bYxsUT1UinKs1c1Uzski2ZiqpeYBKVYICJ58TwsNpREszfZSLS/f6LZm4wk2jOIV3t19J2iKIoSN2hQUhRFUeKGuhSUXoq1ARGSaPYmG4n29080e5ORRHsGcWlvnelTUhRFUeKfutRSUhRFUeIcDUqKoihK3JBUQUlE0kXkExFZIyKrReQ3Yc4ZJCLfi8gK9/WnWNgaZE++iHzl2nLCssbi8E8R2SgiX4rIGbGwMxlRvSiRkmiaSUS9JNuKDj5gtDFmmYg0ApaKyAfGmDUh5+UYY66MgX1lMdgYU9aku8uB7u7rLOAF96dSfVQvSqQkomYSSi9J1VIyxuwwxixzfz8ErAXax9aqajMUmGgcPgeaisjJsTYqGVC9KJGShJqJO70kVVAKRkQygP7AF2EOny0iK0VkjoicWquGnYgB3heRpe7OmaG0BwqC3m8lsf8TxCWqFyVSEkQzCaeXZEvfASAiDYG3gd8aYw6GHF6Gs7bTYREZAkzDabrGivOMMdtEpDXwgYisM8YsiKE9dQ7VixIpCaSZhNNL0rWURCQFRyyvG2PeCT1ujDlojDns/j4bSBGRlrVsZrA929yfu3GW6A9d0ncbkB70voP7mRIFVC9KpCSSZhJRL0kVlEREgJeBtcaYf5RxTlv3PEQkC+dvsK/2rCxlSwO3sxQRaQBcAqwKOW06cIs7SmYg8L0xZkctm5qUqF6USEkkzSSqXpItfXcucDPwlYiscD/7I9ARwBjzH+B64Oci4gOOAsNM7Ja1aAO86+rXC0wyxswVkbuD7J0NDAE2AkeAW2NkazKielEiJZE0k5B60WWGFEVRlLghqdJ3iqIoSmKjQUlRFEWJGzQoKYqiKHGDBiVFURQlbtCgpCiKosQNGpQURVGUuEGDkqIoihI3aFBSFEVR4gYNSoqiKErcoEFJURRFiRs0KCmKoihxgwYlRVEUJW6odlASkfki8rNoGOMunz5ORL4TkdxoXLOC+40Xkcejfa5SeVQ/Sk2gukpcKhWURCRfRI6KyGER2eX+IRpGciMRyRARIyLlbZdxHnAx0MEYE7oZVdLiiv5pEdnnvp4O7MdSxvk3isgWEflBRKaJSPOgY81F5F332BYRuTHo2MkiMl1EtrvPIqNmv9nx+6p+apAo6+eXIpInIoUiMr5WvkAVUV3VLCLyiIgUu3/fwKtLTd83kpbSVcaYhsAZQCbwUA3Y0wnIN8b8EGnBCkQV79wJXA30BfoAVwF3hTtRRE4FXsTZ06UNzh4ozwed8hxQ5B67CXjBLQNgA3OB66L/FSpE9VNzRFM/24HHgVdq0N5oorqqWd4wxjQMen1T0zeMOH3nbq87Bzgt9JiIWCLykFsL2y0iE0WkiXs4sC/8ATfinh1S9nbgf8DZ7vFH3c/vEJGNIrLfreW3CypjROQXIrIB2BDOXhGZKiI7ReR7EVkQ5KBDzxskIltF5I8istethd0UclozEZklIodE5AsR6RpU/lkRKRCRgyKyVESyy/s7hjASGGuM2er+fccCo8o49yZghjFmgbvl8sPAtSLSSJzdJa8DHjbGHDbGfIazs+TNAMaYXcaY54ElEdgWVVQ/8asfAGPMO8aYacRod92qorqqEV3FhIiDkoik4+xUuDzM4VHuazDQBWgI/Ns9dr77s6kbcRcHFzTGvAzcDSx2j/9ZRC4EngRuAE4GtgBTQu55NXAW0LsMk+cA3YHWwDLg9XK+XlugJdAe5z/6SyLSM+j4MOBRoBnOTo1jgo4tAfoBzYFJwFQRqQcgIueJyIFy7nsqsDLo/Ur3swrPNcZswmkZ9XBfPmPM15W8Vq2j+olr/SQsqqsa0RXAVW7gXS0iP6/g3OhgjKnwBeQDh4EDOA/geaC+e2w+8DP394+Ae4LK9QSKcbbizQAM4C3nPqOAz4Levwz8Neh9Q/d6Ge57A1xYme/gnt/ULdPEfT8eeNz9fRDgAxoEnf8mTqsjcO7/go4NAdaVc6/vgL6VtMsPnBL0vrtrp4Q59yPg7pDPtrn2ZwM7Q47dAcwP+czrXj+jsn+76rxUP4mhn5DPHgfG14Y+VFdxq6veQDvAA5wD7ACG1/RzjaSldLUxpqkxppMx5h5jzNEw57RzxRFgi/vg20RwnzKvZ5x0wz6cGkOAgrIKi4hHRJ4SkU0ichBHxODUOsLxnSmdN97i2hBgZ9DvR3DEGLjX70RkrdscPwA0Kec+oRwGGge9bwwcNq4yKjg3cP6hCo7FGtVP/OsnEVFd1ZCujDFrjDHbjTF+Y8wi4Fng+sqUrQ7Rnqe0HadTMEBHnCi/C6cmUK3ruX0mLXBqdgHKu+6NwFDgIpyHkRG4VBnnN3PvEaCja0O5uHna3+M055sZY5oC35dzn1BW43RSB+jrflbhueKMhkkDvnZfXhHpXslrxRuqn9jqJ1lRXVVNV6GYapStNNEOSpOBe0WkszhDM5/AGb3hA/bgjP6KZEjhZOBWEeknImnu9b4wxuRXsnwjoBCnFnOSW74iHhWRVPeBXglMreR9At/RKyJ/4sTaaHlMBO4TkfZuh+lonGZ5OF7HyfNmu0J9DHjHGHPIrU29AzwmIg1E5Fwc8b8aKOzmk9Pct2mB/HKcoPqJoX7AGS3masIDeESkniT+CDLVVRV0JSJDRaSZOGQBvwbeq2z5qhLtoPQKjgNcAGwGjgG/AjDGHMHpgFsoIgdEZGBFFzPGfIgzOuhtnHxmV5xOvcoyEaepuw1YA3xewfk7cXKu23H+895tjFlXifvMwxlq/bV7v2MENd9dB3C4nPIvAjOAr4BVwCz3s0D5w4FRM8aY1Tgdr68Du3GEd0/Qte4B6rvHJgM/d8sEOIqTwgFY576PF1Q/sdfPQziaeAAY4f5eE8OsaxPVVdV0NQxn4MQh1+anjTETKnHfaiHh0851DxEZBLxmjOkQa1uUxEP1o9QEdVFXuvadoiiKEjdoUFIURVHiBk3fKYqiKHGDtpQURVGUuCEmQz1btmxpMjIyYnHrhGfp0qV7jTGtYm1HbaJ6qR6qGSUSYq2XmASljIwM8vLyYnHrhEdEtlR8VnKheqkeqhklEmKtl8RM3xXkQs5Y56eiVITqRYkU1UzMSLyZ2gW5MOHH4C8CTyqMnA7pdWbfrTrPsWI/9VI8lS+gelEiRTUTUxKvpZSf44jF+J2f+TmxtkipRe55fRm3j1/C+p2VXD9U9aJEimompiReUMrIdmov4gHLA99v1SZ2HcG2DZkZzcjN38/lzy7gwXe/Yv8PReUXCtaLJxXqt9C0TF2iKmk49TExJfbpu4JcpyaSke00kUPfh6PfMDi8BzZ8AEsnwIrJ2sSuA1iWcE/X/dzqX8nruzry5JICZn21g7+ddYwf1d+A1bkMzfQbBgi07QtzH9C0TF2hrDSc+pi4JipBSURewVm5drcx5oTtiMskVDSXPVXaaVz2FBzdVzpgBc4XAWM7r0ATWwWTEFRXL/X9RfzMk8olP53Ciws2cd6iBzDiw/Z4sfqPgL7DT9SLJ9XZfzM0LaOaSQiqpJmy0nChPmfnCkAc3QQfVx8TE6LVUhqPs73wxEgKFW9agNdfhBg/tu8Yhxe9TCP3vfEVIrNHg2074uh5OTRsXSIyY4FlAeKIKyPut55XShhPFfRSysn4jtGxYBp/6ZsOH/uwsDG+IkzeOFj+GtL9Eji0A/yFJU4F42gl4JBUM4nEeCLVTCAN5y8qScOtnBykoUKYdZ/zOzgtop6Xq4+JMVEJSsaYBSKSEWm5LY36k24LqYBgqLdvNUUIHiyMCB7bhwUYA7JuppPjBRALPGkntqSCqUwTXYkJVdULGdmOc/H7AQPLJ2Fd/lfwpmF8x0AMgsH4izDrZgbtRmY5TqXvjc4rnC5UL3FNlTSTnuWk3FZOguWTnKBjecDyOjsoiYDtD7qJH9bNco5D+GxNANVLjVFrfUoicidwJ0DHjh0BaHPq+exefT0dvnkDwZCC4WCzU9nZoCdrTGeu3DYWMX7E9S7GrdGI5XXEkjkq/M10SGfCE04vpGdB/xGQNw4wYPuc1Eu/YcjhPZgNH2D7ixBjjmvGuRiOXgIaCFeBUb0kPGVqJj/H0Yrxg99A+/5wcl+nj3HO/W4rOoBxWtYDRpZOA+fnwK41ToCq30L7JmuQWgtKxpiXgJcAMjMzDUCjeik0Gnw7fPse+AoRbJocWEWTQxvoOXI67GqPmXXf8WCEcSs3tkGO7CupCRfkOrWhQF44XC5ZRZNQhNML4DzfFZNLUjLLJzkOx5OKXP5XZOcK7KWvgvEd1wvgOJMAqpekpEzNBNJ4vkLAhu3LnQAzcjiMmgULn4F1synZwdxAkw6l+yUDZcVyXtrXVGPEfkh4oInddVDJww486MxRyG1zkcxboecV2J5UfFgUGg+PfNmMgv1HHNGMv8KpPee9AuOvdGoywcOANRecPAT0cuGDTqvpeA24yAk8Vz6DddscjnW5HFssfEbwSQqm03lOedVL3SOcj/EVwvwnnePDJsGVzzhpu0DXQEADgQoLtvPe2G4/t6V6qSFiPyQcHNEM+gNsWVzSJA7MJ8nIhiufQQBPQS725hzm/9Cdtz9P461nFjD1tMX09heXXCvgnEZO15xvspKeVVKLDW41BeaTpGdR/5YpFG5ezNyZbzFhewf6rGjInzoYrPwcUL3UPYJ9TKDV88185/3I6U5XQJveJ2ogtJVVmf5spVpEZT8lEZkMDAJaAruAPxtjXi7r/MzMTBN2scRA7rYSOdttB47ym8nLsb/9gjfTxuCh2EnnedJg1MykFYqILDXGZMbajuoQNb1ASSouKI0XrBfbNoyZvZaXP9vM1f3a8fezC/FOvKqkHyHJ9QKqmVIU5DotpG/mO60e8Tit7uzRZd882C9VFIiSYABErPUSrdF3w6NxneM14JyxFeb42zetz+Q7BzJmVhNuWAy/bpnHed1b4u1/U/liSALRJDpR0wuE6cgurRfLEh66ohfNG6Tyt3nrOVrchn/fPIOUVVM43qekeol7oupjQrMyZaXfgp99cNAKrBIRbkSeDpipNvGRvgsleH5BWaIpyCUlP4dH+mczrsXVjJrRg3Mat2BcsZ+0cIJxy6hokpAK9CIi/GJwNzodWcWaRS/xtyPn8rvb/0Gq1+1SLSvwqF6Sk0AfU3mVjfJWgyhLEzpgJirEZ1CqSDQhwrh15HQa/6Qvr789FV59EoMPCedEVDTJSSWdzJXL72ZISiGF29/l2fHCfbfdjGfbEnUydZFAVqYsynr25WmiMpVppULiMyhB+aIJI4zrskfTY8MBPGuLEbEx/iIk1ImoaJKXSjoZC5s08WM2f8Yf3+nHU61zEHUySihlPfvyNFGZypFSIfEblMqjDGGcfu4V+L5+AZ+/GCNeUkKdiIqm7hKkGcuTSvrpF/P8kgL6ZHbkJnUydYNI+gfLevYVaaKiypFSIfETlKIkGO+tM/lwzts8v7ktP93Zlp+mhymrokl8Ih2AEKKZYR3O5Ev5igdzC2g56CUubbBBnUwyU5X+wXDPXge+1DjxEZSiLJjBl17LhA/h4WmrOaVtY/qmN60525Xap6oDEII0I8Bfhp5Gg93L+PKzRbS+/Dr6q5NJXqLRP6gDX2qF2K/oANHZ6TEgmI/H4Hl1KM9l+2jVKI1fTFrG90eLKy6vJA5R2hnUuz2PB/c+wH3eqZzywQi2fTU/unYq8UPoZo9V6R/UHWlrhfgISjUgmMa7Puefw/uz4/tj/PHdr4jGJGElToiGXgDynUEOHmxS8DFnxlscLvRF11YlPghenqqqLZxIdVeVXW+VOEnfRaNDOczghwHpzbjv4h78bd56Lu7Vhqv7ty85X3PDiUu0BiAEaUasFOb+0I28N1fywogzkFLLjKN6SQaq2z9YmakqgWO71sDs0c6qEZ40TfVFQHwEJagxwdx9QVc+Xrebt957hx/tO0KjnoNVMMlANAYgBGnGk5HNpZtbMmb2Wl5ZmM/t53UucTLHDsLif6telBN1F25pNMvj7NMU2N3AX6hz3CIgfoJSNAgjGE9+Dv/p1ZCGOx8j9TMfZvHfERWMEiBIMz8zX9Di5I+YPGcD2fVOp8fcm0oW4gygelECBA98KLV1uk3JNhg4i7jqHLdKk1xBKZggwbQSwRY/FgbbX4yoYJRQCnKRiUO5xl/EkBQPc+cMprtdhAQHJFC9KCUE92MHb51ueZyf/mLnsyFjtRITAckblEIEI5YHn23jxyLV40FUMEowrl7E+EkT+KHQR3GKl1QM7t7ZjrNRvSQUa7Yf5JP1uzmlbSNOObkx7ZrUO7G/sKqE9mMHb2cB2gdZRZI3KIUIRi57il07tvKrRQ0Y2q89I9sVqGCUEoIHPXhS8Z5+I8OWZDOm//f06pKhe+ckKMu+/Y6/zVt//H2T+in0OrkRp7Zrwuntm3B6hyZ0btEAy6pCoKrM6g6gg2QiJHmDUhjBtAe6HlvJ4yu2kX3hXXRp1TDWVirxQohermuXyaQdixm+7gfev/x8WjeqF2sLlSowYmAnhvZrx9e7DrFmxyHW7jjImu0Hee3zLRT6nNRsozQvp3doQv+OTTmjYzPO6NiMZg1SK3eDigbc6ITbiEneoARhBfP7y05h7qqdjJm1lpdHnel8qDUZBUrpxQuM/UlfrvhnDn98ZxX/vWVASdpH9ZJQNKqXwoBOzRnQqfnxz3x+mw27D/PV1u9ZufUAK7ce4D+ffoPfdvqbu7ZqQFbn5pzVuQUDu7SgbZMqVkp0pfmISe6gFIZWjdL4xYXdeGrOOhZt3Ms5ad9oTUYJS7fWDbn/0p48Pmst01duZ2i/9lrzTRK8HoteJzem18mNueFMZ4HMo0V+Vm49wNIt35GXv5+ZX+5gcm4BAF1aNeC8bi05r1tLzunWkoZpQa6zvEqKrjQfMckflMIIZtQ5Gby6eAtjZq9lRr9cLK3JKAFC9HLruZ2Z+eUOHpm+mvO6taSF1nyTlvqpHgZ2cVpGAH7bsHbHQRZv2sfCTXuZmreViYu3kOIRzsxozo96teGKpgW0nXaDM3UgMHAqc1TJRXWl+YhJ7qAUqNWGCKZeioffXdqDe99YyaLTTuE8rckoULoVZHmg/wg8fYfz1+v7cMU/cxgzay3/OEdrvnUFjyWc1r4Jp7Vvwh3nd6HQ52fplu/4dP0ePlm/m7/MXMM+z3uMTjmGB4OxbWT2aGjTWwc5VIPkDkr5OSWTH23bWcXBFcyP+7bnxU+/4aGlfj68eRrebxeqcOo6pVpBfsgbBysm02PkdO6+oCv/+ngj155xFudpzbdOkub1cE7XlpzTtSV/GNKLb/cdYcXiIkzeWxjjRwT8tp+ln7xHxtV9aP39l5rqrQLxsSBrTZGR7U5oczH28ZV9PZbwlzOOMOTAZOav3w3Zo1UwdZ1A/p/A8GBzPEX3i8Hd6NTiJKa88xa+bxZoQFLo2OIkfnzlNXiv/Ad4vNhY+EjhqbUtGfjkR7z19mRsX6GuKh4hyd1SSs9yUnbB69wF0i0FuWQuGMUZKYUUL3qX4h4zSMkYGFt7ldgSyP+vnATLJ4HtO56iq5fi4Zlzijjl/YexPvFBzt+15qs4ZI5C2vRG8nNIy8jm7/V689bSrczJy+cK4yVFfBjxcrTNQBrH2tYEILmDEjidjm16n5huCdq2wBgfqxbNop8GJSUwLLzvjSdopr+9Cr/4sLAx/iJEBzkoAYKmE3TBmXriu7gHeQu78O3SeUzZ3Yk1rx7kJwNWcef5XUhvflJs7Y1jkj8oQfgJbm6qxviL8OPh+fyTec5vk+JJ7oymUknK0Ix4U/EVF2GLl1Qd5KCUg9djMfD8yxl4/uX033WI/+VsZl3eh0zJW0O97hcw9Mpr6NhCg1ModSMohcNN1Uh+DqvkNN6f6ePdZduOz1lQlBNIz8IaOYPPP5rG39e34vfFXdG2tVIZurdpxNNnHcOsexLjK6Lwm3e55R97OOXMi/jNRd1p2TAt1ibGDXW7WZCeBdmjOePcSzitfWM+/XgW9gLdKVIph/QszrjpL+xq0pc33n1b9aJUnpWTEV8hFjb1LD93dNzO5NxvGfS3+bz46SaKfHbF16gD1N2WUhAiwh9PP0T/Tx6Gj33g1Y3clLKpl+Lh6ayjDPj0QdWLUjkKcmH5awT2WRLLyyVDrmdevd48MWstT85Zx9SlW3nimtPJ6ty8/GslOXW7pRTEwEPvkyZFxzuxdfimUh7n/vCB6kWpPPk5zm60AAj0vxHSs+jaqiEvjzqTV0ZlcrTIzw0vLuZP763iSJEvpubGEg1KAAW5WCteRwBjwIhHZ+orZVOQiwTpxY9u/FfTiMhlIrJeRDaKyAOxtidiAnPgxAPees7oziAuPKUNH9x3Preem8Grn2/hin9+xpdbD8TI2NiiQQmO12IEsEX4uN7FmopRyiZIL0aEt/wXsLtpn1hblbSIiAd4Drgc6A0MF5HesbUqQgJz4C58sMxU70mpXv581alMvmMghcV+rnthEa98thljTJgLJi9RCUrJVIuxrVSe/+5MVm//PtZWJTUJrZlStd403vZn86+PNsbaqmQmC9hojPnGGFMETAGGxtimyHEHVgGQU/YAmYFdWjD7N9lc0KM1j81cw2+mrOBYsT/suclItYNSstViCm+cxvqUXvwvZ3OsrUpaEl4zQXqxRs6g55k/YnLut+Tv/SHWliUr7YGCoPdb3c9KISJ3ikieiOTt2bOn1oyLiMCivx+PcX6WEZianpTKSzcP4P5LezLjy+3cZrqElQAAE/JJREFU8OJidh86VsvGxoZotJSSqhbTMM3LPzt8wtYv57Pj+6OxtipZSXzNBGq96Vnc1+sA93jf4+3p78baqjqNMeYlY0ymMSazVatWsTYnPOG2PgEnOIW0nixL+MXgbrx0cyYbdh3m2ucXsbkOVHyiEZSSrhZz4Y7/MtE7hg/fnxlri5KVCjWTEHoBKMil+dSf8FvrTe7Zci+bl38Sa4uSkW1A8Kz2Du5niUdw6tfywPdbIW98ua2ni3u3YcqdAzlS5Ocn/1nMup0HY2N7LVFrAx0SqRYjxk+q+Ni/+mNnaGaYWoxSsySEXuC4ZixsUsTHsk+nq16izxKgu4h0FpFUYBgwPcY2VY1A6rfnZc4Q8aXjnQWj/eWvJt43vSlv3nU2XksY/tLnrN2RvIEpGkEpOWsxnlR2FNWnYOLdMP7KCnPASkQkp2bEQ4O9X2KPu0L1EkWMMT7gl8A8YC3wpjFmdWytqiZfz3OCkLGd4CTWcb9T1vSCbq0b8sZdA6mX4mHE/75g057DtWx07RCNoJR8tZgLH8S6/CkeTX2V7lvfwlRQi1EiJvk0M+AWPJbFxd6liB2mz0CpFsaY2caYHsaYrsaYMbG2p1rk5zibjgawPM4WO+UMFw/QqUUDXvvZWQDc/L8v2HUw+QY/VDsoJV0tJj3LWQ167XRS8WERmCMguv11lEhKzSDuVigG55/qRSmDjGxnaSossLxOQMocVemNRru2asiE27I4cLSY28Yv4YfC5Fr9ISpr3xljZgOzo3GtmBMYsuluo+5H8OMlNfNm6DtcJ9VGiaTTjLuumQGKxcOnJ13KxcPvVb0oJxJoXYfu8RYBp7VvwnM3ncHt45fwu6kree7GM7AsqbhgAqArOoQSGLKJjWCxtelZDCt8kI1Zj6mDUcITtK6ZIGxqfw137B/BoqIuMTZMiVuCphRUdWDM4J6t+cPlvZizaicvfLqphgytfTQohRIyW7/JkIdZZZ3C+EX5sbZMiVdC1jXrctHPaNM4jWc+2FDnlohRIiR4Mu34K2DmvREFp59ld+bKPicz9v31LN60rwYNrT00KIUSskZV0x7n8eN+7Xh76Ta+P1oca+uUeCREM2mdz+aeQd3Izd/PoiRxFEoNETqZNm9cyajNSrSgRISnrutDRosG3PvGCg4cKapF42sGDUrhCGla33/STHr51jI1r6DiskrdJGRds2En76Bt43o8+6G2lpRyCLSyCfQHGfAdg4XPVGo5IoCGaV6eHdafvYcLeWjaqloxuybRoFQebtO6Td5YpqSNocX8B/Bv+SLWVinxSlAqJu31a/hH1+VkFozjq88/iLVlSrwSaGVnjgIrxf3QwPq5FU6oDeb0Dk347UXdmfnlDuau2lHjZtckuvNseQQ1rVPwM9Q3DzPhEzhjBLTtC0f3VXn0jJKEBKdifIWcvf5JslL8MO8tzOFfIvWaqF6UE0nPOj6tgLxxOHMKDFgWkUxFueuCrsxZtZOHpq3m7K4taVI/pcIy8YgGpfIINK19xwCDJWDsIsh7xTkuFnh0K2zFJaAXfxGIILYfLwZj/LDwWdWL8v/bO/vgKOo0j3+e7pmELAJKeDUmoIgisosChsAdigouKivrvqmgLtbd6npn1elxeytaWlfl3h5rrZ6u3rnr3e0tqEDVniAUb4IgCwpu5CWsCGJEIgGDAgoISjLT/bs/uodMhgDOSzLdk+dTlUpmuqfzTOabfvp5+nme3+kZehvUzPH0YxfBhBlpXfhGbYtfff9b3PTsG/z61R089t0h7WB07lGndDoSofWW2cjm2bhOE2JMUvrXbQ6t9SSjJPeflJTCkmkYN44IXgu26kU5HTnqX7pzVH9mrq/jlivKGVLWLfd2tjHqlM5EIrQeOpnYxpdg84tEcLzOfbG0a19pyYlUjIcsmYbrOogxGLEQ1YtyOlL0kwkPjL+IhVs+5rFF25h7dxUi4WqqVaf0dSmvpLi8kuePjeKL917nJxNG0NU9ovcIlFMzYir0Hoyzcw2Pr9nPBZ2Pc9uPpqhelDNTX51xxNStJMoD4wbyyIJ3Wb7tE759aZ82MrJtUKeUJtdeN5Fr3+mC9eVAHhh/Ub7NUYJOeSXR8kr6RnYxfdE2zo8NoCrfNinBJlHFmeG9JYDbKiv433V1PLF8B+Mu6Y0dohFEWhKeJgN6nsW1g3rx4lsfcTzm5NscJSRMHllBry7F/PuK9/NtihJ0Uqo4WTIt7aVQIrbFP46/iPc/OcrCLeFaFUadUgbcP+gQtxz/I2tWLc63KUoYqK+m01tP8ehlR/nzrs9Yt/NAvi1SgkyL1Wktr0Amg6VQbhjSl0v6duWZVR/guEkN3AFfhFLTd+lSX82Q1+5gcLSR2Pr5uN0fxzr+md5bUlonKRVzo13E/LMeYfHifYy67BByvmpGaYXUKs5lDzan8tIokrEs4b6rL+TvZ29i6dYGJn7r3JNTgwFsT1CnlC51a/11c1yMicGSfwJMYD9gJc8kpWLEaeKfyzZTUb8AVsW9NXVUM0prJFfh9R6ccdHDhCF9uKBnZ/7j9Z3c+M2+SOqsvQC2J2j6Ll380NqIjRHLa4zM5SqjAQ+tlTRJTsXYRQzo1ZmoxBFcTC40o3opfJJncaaJbQn3XHkB2xuOeFPEU/QYxPYEjZTSxQ+tpW4tGxoMl737K4otBysXH3AIQmslTVIaIiNAvGYucacJsaPY2WhG9VL4ZFEanmDSZWU8vmwH//3GLkZPzb5Bt61Rp5QJfmh9eczh73aUcF3nWm794eTsP+AQhNZKBqQ2RP54Ab9/8QW2Fw/l12VXYGd6XNVLYZN80WHZcPntGa1+3Slqc3tVP55eWUvdgWP0z0GDblui6bss6BS1GT32eh78dDxvOxdmf8AQhNZK9kT6VdFn4sPMP1DGgposynVVL4XN6dZaSpPJIyuwLWF29e42MDS3qFPKhvpqfuzM4+rOdTy5PAf9JymLxQX5akbJEP8e0MSz67n03K48sfx9GuMZ9rupXgqb1tZayvA+ZO+unbhucG/+uKE+8P2Vmr7LFD+0jjpN/JdE+NGu6az74EJGX9gju+MGPLRWsiApHWPZRfzymllMWhhj5ro67r5yQGbHVL0ULkkDodk8G9x4VhHxlJH9WLp1H6++u49JpXsDe19JnVKmJIXWNnB7yTpq5+2i6pYpWP1G5ts6JYikdOoP3fkcf9N/Es+sivCD4eV071yU9a+YU72bmONyR1W/0A3iVFohaSB0tk5k9IBSys4uYcv6FUw6+GBgC2Q0fZcpSfl8sWwmsZopx17AzTDnq3QATqRjLMCFD1fz8IGfc3FsO08s35H14eOOy29W1vLa9k/VIRUaibJwgEX3w6IHWp5n6qtbfz5pu/Xmk9w38CAle9d57Qi5bGXJIRopZUpyqe/hPVgbZyLiEnebaPzgTxRnc+WRgzJQJYAkNLP63+DD1WBcLCfGvf0a+Nvq3dxWWZH5+jf11dRVL6Xvka5M/s6tOTVbCQj11fCHGz1HArDpBRh2h7cK9tKfNT+/+SWYuqj53LHhD978PONyi13EQ+4U4hIhCoEskFGnlA2J0Lq+GqmZg3GaiLk2cz+p4K5Mj6m9J4VNeSWMnQ4frT/xGY+8ZhKlc4/x8CtbmXfv6PQnOvuaOT/eyOziCJEuo4C+bWK+kkfq1oITa37sxryKPMsGN6l4Ibk9oL7ac0huHADLaWLIOXF+bv2CJyu/COSFr6bvckGiofaah1l63v0c2LqS995+reU+Z+q8T2zfMudEaJ2Tjn8leCRXzU2YwVkN63lydBNb6g8xa31d836n00zytrq1GH/0VRSHyO432+udKO1J/zFgR1OeNC0dEjRHP/XVXlTeYrsw/Oxj1B04xp5uw70iilOl/PKERkq5orwSPtnGzQ3TcCMOscXz+cqZQUnsEBw/Auuf9ab92inzzuqrT1TXGDdO3Fi4RrCxiGOzr8sw+uf1jSltgq+XRFpljF3Mo33v4fNXF7I/NoyeR7e3rLhK1kxSOga7GCbMIEYEyxjsaDRw6ZhsEJEfAv8CXAJUGmM25NeiPFJeCVMXw5tPwXtLAdffYADx+tUqqqDkHHjzaahd4UdWie0WiMWgj+czp+gV7HkG8B1Wasovj6hTyhV+mCxuHBswpgmW/QyDQU6IB3AaW4bWM2+C+HF/PxBj2Fz6HaLd+/GfdX14Z7HDyxVfUt79G/l6Z0pbkJJWkfhx7jr8LI7lYK+ei0EQ/OUGTpOOwWnk84P7uLvpIe7p18C4678fiBNLDtkKfA/4Xb4NCQTllVA2HHYsgxOrUQhgwBjY/ZZXwNACCwaMhXP6w8aZiHGIiv+aBAGaCKJOKVfUrQW32flYIhjjImIS1ykeYrUIrY3T6J18DLgiWJEiqm6+D4AZ3Vcy7e3tbPhokDqlQiNFLwDiOkQwGAMiiROGeJFSSamXrju8p+XrxOK5ur5ss3sz7MrhzeneAJxccoExZjug1YTJJKo4E+OHEO8iReTkVB7iTaMfO917WDMHnCZcLFzXJSqOd24KUMFDVk5JQ+sk+o/xPvz4ccAgGES8ixfED6DFhhue8PafeRMm3gi4xI1gJAIDxxPt1sdL6yx7kJ5OEzMtG9n7IfRIf+ZVEFHN+JzQS6N3MoHmK1xfN64VwR5+p1ddlVhTx7K9+wpODETY12csG3Z9xr+O6kL3/5vqHc+yPJ2NmJqvd5cXRORu4G6AioqKPFvThqQM+QWa115KrsKzol51XvK8PP91n5VewU9f2Mgvzn+Hwed2y2imXluRbaSkoXWClHJfMa6XkBMvLecg7C4dQ1HDHjrVLKJ7/DiCIW6Ehu4jKRt9K9ar0z1BifirTbqI43gVNjVzvOOHH9UMnNRSwMaZJzYJXtQ8N3YV55WM46rNT5y42MEFht8JgLv5JXp8vIo5xWuIMMVzSLheJLVkmrcOT0BONKdDRF4D+rSy6WFjzIKvexxjzPPA8wAjRowwZ9g93KRO8khee2nLbEA8RwMto2f/qxfQ2Fd4xBnJyxNHt6flZyQrp6ShdQop5b7ih9bGjeMY4dz9b2Dv/xO2n8s1AlYkSvn3HmvZ7W8sfxlkgxdjZT7zKmioZpJIaimgZk6zUxELsYs51u1SqtbciRH/vhOAFYGhk9m/dSXnOHEiuNjiIBaeZhKpPeMG5h7BmTDGjMu3DQVDsrM6Q3vJ+MG9eXplLQePNlJ6VnGeDD6Zdrun1JFDa6lbS/RQPWyahZjke0yCXD6lWSiJPLFdBBNmwL6anMy8CiMdRi9w8vLXXx1E+o/hrp1rsFc7yeM4+fC8Sfx+Qwm1G7/BrEjEc0h2kTeGps9lLavyOpBelFY4w9ImV1/ci6deq2Vt7QG+e3lZHg1tyRmdkobWGdBKaC311bBlLsQbvWo8sbyrXsS7okl1ZonX52DmVXuTC810KL1Aq4NVowBro16/GhAzNut3HqSWVfQbOpZjQ4bR6UB1szbKK7NaOjuIiMjNwDNAT2CxiNQYY76dZ7PCQXJBRCsXtd8s60Zp5yJW7/g0XE5JQ+sckXo1nIiCNs5svl/U2sTnEE6BVs3kCL8vRbbMhqP7idauYIq1min2OqRqFJSPAcac/JqQ6eV0GGPmA/PzbUcoOdWFro9lCVdd1JPXd3yK45r0J4m0ETrRoT1JDFUcMRW6lXtpuYAORVQCQnklTHwKyoYhbhwxDqJ6Ub4uiXPOKS5Urrq4J59/GeMvew61s2GnJiunJCI3i8geYBReaP1qbszqAHTQVUNVMxnSQfWitC1XD+rFnJ9Ucem5GQ4CbgOyrb7T0DpTzhBaFyqqmQzpoHpR2paunaKMGlCabzNaoBMd8kmB5f+VNkb1onQA9J6SoiiKEhjEmPavthWR/cBH7fxrewAH2vl3ZsOp7O1njOnZ3sbkE9XL1+J09qpm2odC0Uxe9ZIXp5QPRGSDMWZEvu34uoTN3kIjbH//sNlbiITtMwiqvZq+UxRFUQKDOiVFURQlMHQkp/R8vg1Ik7DZW2iE7e8fNnsLkbB9BoG0t8PcU1IURVGCT0eKlBRFUZSAo05JURRFCQwF5ZREpFxEXheRbSLyroj8Qyv7jBWRwyJS4389mg9bk+ypE5F3fFtOWhpcPH4jIh+IyF9EZFg+7CxEVC9KuoRNM2HUS6GNGYoD04wxm0SkC7BRRFYYY7al7LfWGDMxD/adiquNMadqurseGOh/jQSe878r2aN6UdIljJoJlV4KKlIyxjQYYzb5P38BbAeCs3pVZkwCZhmPt4CzRaRvvo0qBFQvSroUoGYCp5eCckrJiEh/4HLgz61sHiUiW0RkqYhc2q6GnYwBlovIRn8J8FTKgPqkx3sI9z9BIFG9KOkSEs2ETi+Flr4DQETOAl4G7jfGHEnZvAlvttNREbkBeAUvdM0Xf22M2SsivYAVIvKeMWZNHu3pcKhelHQJkWZCp5eCi5REJIonlpeMMfNStxtjjhhjjvo/LwGiItKjnc1Mtmev//1TvHWGUtcm2AuUJz0+z39OyQGqFyVdwqSZMOqloJySiAjwP8B2Y8yTp9inj78fIlKJ9zc42H5WtrCls3+zFBHpDFwHbE3ZbSFwp18lUwUcNsY0tLOpBYnqRUmXMGkmrHoptPTdXwF3AO+ISI3/3ENABYAx5rfAD4B7RSQOfAXcavI31qI3MN/XbwSYbYxZJiI/TbJ3CXAD8AHwJXBXnmwtRFQvSrqESTOh1IuOGVIURVECQ0Gl7xRFUZRwo05JURRFCQzqlBRFUZTAoE5JURRFCQzqlBRFUZTAoE5JURRFCQzqlBRFUZTA8P/kfGrx230//gAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 6 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ],
      "source": [
        "from sklearn.linear_model import Ridge\n",
        "def ridge_regression(data, predictors, alpha, models_to_plot={}):\n",
        "    #Fit the model\n",
        "    ridgereg = Ridge(alpha=alpha,normalize=True)\n",
        "    ridgereg.fit(data[predictors],data['y'])\n",
        "    y_pred = ridgereg.predict(data[predictors])\n",
        "    \n",
        "    #Check if a plot is to be made for the entered alpha\n",
        "    if alpha in models_to_plot:\n",
        "        plt.subplot(models_to_plot[alpha])\n",
        "        plt.tight_layout()\n",
        "        plt.plot(data['x'],y_pred)\n",
        "        plt.plot(data['x'],data['y'],'.')\n",
        "        plt.title('Plot for alpha: %.3g'%alpha)\n",
        "    \n",
        "    #Return the result in pre-defined format\n",
        "    rss = sum((y_pred-data['y'])**2)\n",
        "    ret = [rss]\n",
        "    ret.extend([ridgereg.intercept_])\n",
        "    ret.extend(ridgereg.coef_)\n",
        "    return ret\n",
        "\n",
        "#Initialize predictors to be set of 15 powers of x\n",
        "predictors=['x']\n",
        "predictors.extend(['x_%d'%i for i in range(2,16)])\n",
        "\n",
        "#Set the different values of alpha to be tested\n",
        "alpha_ridge = [1e-15, 1e-10, 1e-8, 1e-4, 1e-3,1e-2, 1, 5, 10, 20]\n",
        "\n",
        "#Initialize the dataframe for storing coefficients.\n",
        "col = ['rss','intercept'] + ['coef_x_%d'%i for i in range(1,16)]\n",
        "ind = ['alpha_%.2g'%alpha_ridge[i] for i in range(0,10)]\n",
        "coef_matrix_ridge = pd.DataFrame(index=ind, columns=col)\n",
        "\n",
        "models_to_plot = {1e-15:231, 1e-10:232, 1e-4:233, 1e-3:234, 1e-2:235, 5:236}\n",
        "for i in range(10):\n",
        "    coef_matrix_ridge.iloc[i,] = ridge_regression(data, predictors, alpha_ridge[i], models_to_plot)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "- Here we can clearly observe that as the value of alpha increases, the model complexity reduces. \n",
        "- Though higher values of alpha reduce overfitting, significantly high values can cause underfitting as well (eg. alpha = 5). \n",
        "- Thus alpha should be chosen wisely. A widely accept technique is cross-validation, i.e. the value of alpha is iterated over a range of values and the one giving higher cross-validation score is chosen.\n",
        "\n"
      ],
      "metadata": {
        "id": "HZVaWbqdK18h"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#Set the display format to be scientific for ease of analysis\n",
        "pd.options.display.float_format = '{:,.2g}'.format\n",
        "coef_matrix_ridge"
      ],
      "metadata": {
        "id": "e_XKJQx8K1Zo",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 423
        },
        "outputId": "53978e33-b3b8-44a0-e4b1-24bf6132da79"
      },
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-09bb328f-951e-403f-a8e7-8252cdaf3582\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>rss</th>\n",
              "      <th>intercept</th>\n",
              "      <th>coef_x_1</th>\n",
              "      <th>coef_x_2</th>\n",
              "      <th>coef_x_3</th>\n",
              "      <th>coef_x_4</th>\n",
              "      <th>coef_x_5</th>\n",
              "      <th>coef_x_6</th>\n",
              "      <th>coef_x_7</th>\n",
              "      <th>coef_x_8</th>\n",
              "      <th>coef_x_9</th>\n",
              "      <th>coef_x_10</th>\n",
              "      <th>coef_x_11</th>\n",
              "      <th>coef_x_12</th>\n",
              "      <th>coef_x_13</th>\n",
              "      <th>coef_x_14</th>\n",
              "      <th>coef_x_15</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>alpha_1e-15</th>\n",
              "      <td>0.87</td>\n",
              "      <td>95</td>\n",
              "      <td>-3e+02</td>\n",
              "      <td>3.8e+02</td>\n",
              "      <td>-2.3e+02</td>\n",
              "      <td>63</td>\n",
              "      <td>2.1</td>\n",
              "      <td>-5</td>\n",
              "      <td>0.6</td>\n",
              "      <td>0.17</td>\n",
              "      <td>-0.031</td>\n",
              "      <td>-0.0051</td>\n",
              "      <td>0.00085</td>\n",
              "      <td>0.00024</td>\n",
              "      <td>-6.1e-05</td>\n",
              "      <td>4.5e-06</td>\n",
              "      <td>-9.3e-08</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>alpha_1e-10</th>\n",
              "      <td>0.92</td>\n",
              "      <td>11</td>\n",
              "      <td>-29</td>\n",
              "      <td>31</td>\n",
              "      <td>-15</td>\n",
              "      <td>2.9</td>\n",
              "      <td>0.17</td>\n",
              "      <td>-0.091</td>\n",
              "      <td>-0.011</td>\n",
              "      <td>0.002</td>\n",
              "      <td>0.00064</td>\n",
              "      <td>2.4e-05</td>\n",
              "      <td>-2e-05</td>\n",
              "      <td>-4.2e-06</td>\n",
              "      <td>2.2e-07</td>\n",
              "      <td>2.3e-07</td>\n",
              "      <td>-2.3e-08</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>alpha_1e-08</th>\n",
              "      <td>0.95</td>\n",
              "      <td>1.3</td>\n",
              "      <td>-1.5</td>\n",
              "      <td>1.7</td>\n",
              "      <td>-0.68</td>\n",
              "      <td>0.039</td>\n",
              "      <td>0.016</td>\n",
              "      <td>0.00016</td>\n",
              "      <td>-0.00036</td>\n",
              "      <td>-5.4e-05</td>\n",
              "      <td>-2.9e-07</td>\n",
              "      <td>1.1e-06</td>\n",
              "      <td>1.9e-07</td>\n",
              "      <td>2e-08</td>\n",
              "      <td>3.9e-09</td>\n",
              "      <td>8.2e-10</td>\n",
              "      <td>-4.6e-10</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>alpha_0.0001</th>\n",
              "      <td>0.96</td>\n",
              "      <td>0.56</td>\n",
              "      <td>0.55</td>\n",
              "      <td>-0.13</td>\n",
              "      <td>-0.026</td>\n",
              "      <td>-0.0028</td>\n",
              "      <td>-0.00011</td>\n",
              "      <td>4.1e-05</td>\n",
              "      <td>1.5e-05</td>\n",
              "      <td>3.7e-06</td>\n",
              "      <td>7.4e-07</td>\n",
              "      <td>1.3e-07</td>\n",
              "      <td>1.9e-08</td>\n",
              "      <td>1.9e-09</td>\n",
              "      <td>-1.3e-10</td>\n",
              "      <td>-1.5e-10</td>\n",
              "      <td>-6.2e-11</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>alpha_0.001</th>\n",
              "      <td>1</td>\n",
              "      <td>0.82</td>\n",
              "      <td>0.31</td>\n",
              "      <td>-0.087</td>\n",
              "      <td>-0.02</td>\n",
              "      <td>-0.0028</td>\n",
              "      <td>-0.00022</td>\n",
              "      <td>1.8e-05</td>\n",
              "      <td>1.2e-05</td>\n",
              "      <td>3.4e-06</td>\n",
              "      <td>7.3e-07</td>\n",
              "      <td>1.3e-07</td>\n",
              "      <td>1.9e-08</td>\n",
              "      <td>1.7e-09</td>\n",
              "      <td>-1.5e-10</td>\n",
              "      <td>-1.4e-10</td>\n",
              "      <td>-5.2e-11</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>alpha_0.01</th>\n",
              "      <td>1.4</td>\n",
              "      <td>1.3</td>\n",
              "      <td>-0.088</td>\n",
              "      <td>-0.052</td>\n",
              "      <td>-0.01</td>\n",
              "      <td>-0.0014</td>\n",
              "      <td>-0.00013</td>\n",
              "      <td>7.2e-07</td>\n",
              "      <td>4.1e-06</td>\n",
              "      <td>1.3e-06</td>\n",
              "      <td>3e-07</td>\n",
              "      <td>5.6e-08</td>\n",
              "      <td>9e-09</td>\n",
              "      <td>1.1e-09</td>\n",
              "      <td>4.3e-11</td>\n",
              "      <td>-3.1e-11</td>\n",
              "      <td>-1.5e-11</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>alpha_1</th>\n",
              "      <td>5.6</td>\n",
              "      <td>0.97</td>\n",
              "      <td>-0.14</td>\n",
              "      <td>-0.019</td>\n",
              "      <td>-0.003</td>\n",
              "      <td>-0.00047</td>\n",
              "      <td>-7e-05</td>\n",
              "      <td>-9.9e-06</td>\n",
              "      <td>-1.3e-06</td>\n",
              "      <td>-1.4e-07</td>\n",
              "      <td>-9.3e-09</td>\n",
              "      <td>1.3e-09</td>\n",
              "      <td>7.8e-10</td>\n",
              "      <td>2.4e-10</td>\n",
              "      <td>6.2e-11</td>\n",
              "      <td>1.4e-11</td>\n",
              "      <td>3.2e-12</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>alpha_5</th>\n",
              "      <td>14</td>\n",
              "      <td>0.55</td>\n",
              "      <td>-0.059</td>\n",
              "      <td>-0.0085</td>\n",
              "      <td>-0.0014</td>\n",
              "      <td>-0.00024</td>\n",
              "      <td>-4.1e-05</td>\n",
              "      <td>-6.9e-06</td>\n",
              "      <td>-1.1e-06</td>\n",
              "      <td>-1.9e-07</td>\n",
              "      <td>-3.1e-08</td>\n",
              "      <td>-5.1e-09</td>\n",
              "      <td>-8.2e-10</td>\n",
              "      <td>-1.3e-10</td>\n",
              "      <td>-2e-11</td>\n",
              "      <td>-3e-12</td>\n",
              "      <td>-4.2e-13</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>alpha_10</th>\n",
              "      <td>18</td>\n",
              "      <td>0.4</td>\n",
              "      <td>-0.037</td>\n",
              "      <td>-0.0055</td>\n",
              "      <td>-0.00095</td>\n",
              "      <td>-0.00017</td>\n",
              "      <td>-3e-05</td>\n",
              "      <td>-5.2e-06</td>\n",
              "      <td>-9.2e-07</td>\n",
              "      <td>-1.6e-07</td>\n",
              "      <td>-2.9e-08</td>\n",
              "      <td>-5.1e-09</td>\n",
              "      <td>-9.1e-10</td>\n",
              "      <td>-1.6e-10</td>\n",
              "      <td>-2.9e-11</td>\n",
              "      <td>-5.1e-12</td>\n",
              "      <td>-9.1e-13</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>alpha_20</th>\n",
              "      <td>23</td>\n",
              "      <td>0.28</td>\n",
              "      <td>-0.022</td>\n",
              "      <td>-0.0034</td>\n",
              "      <td>-0.0006</td>\n",
              "      <td>-0.00011</td>\n",
              "      <td>-2e-05</td>\n",
              "      <td>-3.6e-06</td>\n",
              "      <td>-6.6e-07</td>\n",
              "      <td>-1.2e-07</td>\n",
              "      <td>-2.2e-08</td>\n",
              "      <td>-4e-09</td>\n",
              "      <td>-7.5e-10</td>\n",
              "      <td>-1.4e-10</td>\n",
              "      <td>-2.5e-11</td>\n",
              "      <td>-4.7e-12</td>\n",
              "      <td>-8.7e-13</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-09bb328f-951e-403f-a8e7-8252cdaf3582')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-09bb328f-951e-403f-a8e7-8252cdaf3582 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-09bb328f-951e-403f-a8e7-8252cdaf3582');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "              rss intercept coef_x_1  ... coef_x_13 coef_x_14 coef_x_15\n",
              "alpha_1e-15  0.87        95   -3e+02  ...  -6.1e-05   4.5e-06  -9.3e-08\n",
              "alpha_1e-10  0.92        11      -29  ...   2.2e-07   2.3e-07  -2.3e-08\n",
              "alpha_1e-08  0.95       1.3     -1.5  ...   3.9e-09   8.2e-10  -4.6e-10\n",
              "alpha_0.0001 0.96      0.56     0.55  ...  -1.3e-10  -1.5e-10  -6.2e-11\n",
              "alpha_0.001     1      0.82     0.31  ...  -1.5e-10  -1.4e-10  -5.2e-11\n",
              "alpha_0.01    1.4       1.3   -0.088  ...   4.3e-11  -3.1e-11  -1.5e-11\n",
              "alpha_1       5.6      0.97    -0.14  ...   6.2e-11   1.4e-11   3.2e-12\n",
              "alpha_5        14      0.55   -0.059  ...    -2e-11    -3e-12  -4.2e-13\n",
              "alpha_10       18       0.4   -0.037  ...  -2.9e-11  -5.1e-12  -9.1e-13\n",
              "alpha_20       23      0.28   -0.022  ...  -2.5e-11  -4.7e-12  -8.7e-13\n",
              "\n",
              "[10 rows x 17 columns]"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "This straight away gives us the following inferences:\n",
        "\n",
        "- The RSS increases with increase in alpha, this model complexity reduces\n",
        "- An alpha as small as 1e-15 gives us significant reduction in magnitude of coefficients. How? Compare the coefficients in the first row of this table to the last row of simple linear regression table.\n",
        "- High alpha values can lead to significant underfitting. Note the rapid increase in RSS for values of alpha greater than 1\n",
        "- Though the coefficients are very very small, they are NOT zero.\n",
        "\n",
        "The first 3 are very intuitive. But #4 is also a crucial observation. Let’s reconfirm the same by determining the number of zeros in each row of the coefficients data set:\n",
        "\n"
      ],
      "metadata": {
        "id": "3oz0KcDCPhqd"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "coef_matrix_ridge.apply(lambda x: sum(x.values==0),axis=1)\n",
        "\n",
        "# Inference\n",
        "# This confirms that all the 15 coefficients are greater than zero in magnitude \n",
        "# (can be +ve or -ve). "
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mIGf1hmRPaVE",
        "outputId": "b94a2ec1-9e88-4ffa-e22c-8d181d92f9c1"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "alpha_1e-15     0\n",
              "alpha_1e-10     0\n",
              "alpha_1e-08     0\n",
              "alpha_0.0001    0\n",
              "alpha_0.001     0\n",
              "alpha_0.01      0\n",
              "alpha_1         0\n",
              "alpha_5         0\n",
              "alpha_10        0\n",
              "alpha_20        0\n",
              "dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Lasso Regression**\n",
        "\n",
        "- LASSO stands for Least Absolute Shrinkage and Selection Operator. \n",
        "- There are 2 key words here – ‘absolute‘ and ‘selection‘.\n",
        "- Lasso regression performs L1 regularization, i.e. it adds a factor of sum of absolute value of coefficients in the optimization objective. \n",
        "- Thus, lasso regression optimizes the following:\n",
        "\n",
        "      Objective = RSS + α * (sum of absolute value of coefficients)\n",
        "\n",
        "    Here, α (alpha) works similar to that of ridge and provides a trade-off between balancing RSS and magnitude of coefficients. \n",
        "    Like that of ridge, α can take various values.\n",
        "    Lets iterate it here briefly:\n",
        "\n",
        "    α = 0: Same coefficients as simple linear regression\n",
        "\n",
        "    α = ∞: All coefficients zero (same logic as before)\n",
        "    \n",
        "    0 < α < ∞: coefficients between 0 and that of simple linear regression\n",
        "\n",
        "Notice the additional parameters defined in Lasso function – ‘max_iter‘. \n",
        "\n",
        "This is the maximum number of iterations for which we want the model to run if it doesn’t converge before. \n",
        "\n",
        "This exists for Ridge as as well but setting this to a higher than default value was required in this case. "
      ],
      "metadata": {
        "id": "Uqx_IIyaPh6X"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.linear_model import Lasso\n",
        "def lasso_regression(data, predictors, alpha, models_to_plot={}):\n",
        "    #Fit the model\n",
        "    lassoreg = Lasso(alpha=alpha,normalize=True, max_iter=1e5)\n",
        "    lassoreg.fit(data[predictors],data['y'])\n",
        "    y_pred = lassoreg.predict(data[predictors])\n",
        "    \n",
        "    #Check if a plot is to be made for the entered alpha\n",
        "    if alpha in models_to_plot:\n",
        "        plt.subplot(models_to_plot[alpha])\n",
        "        plt.tight_layout()\n",
        "        plt.plot(data['x'],y_pred)\n",
        "        plt.plot(data['x'],data['y'],'.')\n",
        "        plt.title('Plot for alpha: %.3g'%alpha)\n",
        "    \n",
        "    #Return the result in pre-defined format\n",
        "    rss = sum((y_pred-data['y'])**2)\n",
        "    ret = [rss]\n",
        "    ret.extend([lassoreg.intercept_])\n",
        "    ret.extend(lassoreg.coef_)\n",
        "    return ret\n",
        "\n",
        "#Initialize predictors to all 15 powers of x\n",
        "predictors=['x']\n",
        "predictors.extend(['x_%d'%i for i in range(2,16)])\n",
        "\n",
        "#Define the alpha values to test\n",
        "alpha_lasso = [1e-15, 1e-10, 1e-8, 1e-5,1e-4, 1e-3,1e-2, 1, 5, 10]\n",
        "\n",
        "#Initialize the dataframe to store coefficients\n",
        "col = ['rss','intercept'] + ['coef_x_%d'%i for i in range(1,16)]\n",
        "ind = ['alpha_%.2g'%alpha_lasso[i] for i in range(0,10)]\n",
        "coef_matrix_lasso = pd.DataFrame(index=ind, columns=col)\n",
        "\n",
        "#Define the models to plot\n",
        "models_to_plot = {1e-10:231, 1e-5:232,1e-4:233, 1e-3:234, 1e-2:235, 1:236}\n",
        "\n",
        "#Iterate over the 10 alpha values:\n",
        "for i in range(10):\n",
        "    coef_matrix_lasso.iloc[i,] = lasso_regression(data, predictors, alpha_lasso[i], models_to_plot)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 297
        },
        "id": "kj2u_6LIPiGN",
        "outputId": "cf9386f3-2b51-4148-f07a-a6dec8cf0df0"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaUAAAEYCAYAAAD8hukFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydeXxU5fWHn3NnkoDs+zoQQGSrghCCCyjaRcS12iqLCq5Va1sttXW3Vq22v1KXulTrAiig4oosKopIcGNHZBOQYABBRAERSDJz398f9w6ZDJNJJplktvN8HMPMvfe9Z3K/Oeddzvu+YoxBURRFUZIBK9EGKIqiKEoQDUqKoihK0qBBSVEURUkaNCgpiqIoSYMGJUVRFCVp0KCkKIqiJA01CkoiMk9EroiHIeLwrIh8LyIL41FmJfebICL3xPtcJTqqGaUyVCOZTaVBSUQKReSAiOwTkR3uL7JhLDcRkVwRMSLijXLaYODnQEdjTH4s5acyInKKiLwvIntEpLCGZWWLyMvuMzMiMjTsuIjIP0Rkl/v6h4hITe5ZgR2qmVoknppxy/upiKwVkf1uuZ1Djk0QkRL3WQZfnjjcUzVSi8T6ty4io0Rks4j8KCKvi0jzkGPNReQ199hmERkVw7XXichiESkWkQlVsb2qLaWzjDENgf5AHnBbFa+Lhc5AoTHmx1gvrESUyc6PwDPAjXEqbwFwEbA9wrGrgHOBvsAxwFnAb+J033BUM7VH3DQjIi2BV4HbgebAYuDFsNP+aYxpGPIK1PS+LqqR2qPKf+si0gd4ArgYaAPsBx4LOeVRoMQ9Nhp43L2mKtduA+7B0WuViKn7zhizFZgN/CT8mIhYInKbGzG/EZFJItLEPTzf/bnbrRkdH3bt5cBTwPHu8bvcz68UkQ0i8p2ITBeR9iHXGBH5rYisB9ZHsldEponIdrdGOT/4i4xw3lAR2SIit4jIt24tbnTYac1EZKaI/CAin4pIt5DrHxKRIhHZKyJLRGRItN9jKMaYhcaY54AvK7Ctp4jMcX8H60TkgihllRhjHjTGLAAiOY4xwHhjzBb3WY4HxlbV1uqgmkluzQDnAauMMdOMMQeBvwJ9RaRnVe2pKaqR+GuE2P7WRwNvGmPmG2P24VRQzhORRiLSADgfuN0Ys8/1LdNxglDUawGMMa8aY14HdlXV8JiCkoj4gOHAsgiHx7qvU4CuQEPgEffYSe7Ppm5N6+PQC40xTwNXAx+7x+8UkVOB+4ALgHbAZuCFsHueCwwCeldg8mygO9AaWApMjvL12gItgQ44D/RJEekRcnwEcBfQDNgA3BtybBHQD6emOQWYJiL1AERksIjsjnLfCnEFMccts7Vrw2MiUtH3rYw+wIqQ9yvcz2oN1UzSa6acJtwWxUbK6+Ja14EvEZHzq2NXJTarRuKvkVj+1sM1sBGnZXSU+/IbY76ooKxo11YPY0zUF1AI7AN24zzAx4D67rF5wBXuv98Drg25rgdQCniBXMAA3ij3GQssCHn/NE63QfB9Q7e8XPe9AU6tzP6Q65u61zRx308A7nH/PRTwAw1Czn8Jp3YQPPepkGPDgbVR7vU90LeqtrnX/AynmyH0swuBgrDPngDurEJ5W4ChYZ8FgJ4h77u7vxOJxVbVTPpoxv2d3R/22YfAWPff/YEW7jMZDvwAnKgaSW6NEMPfuvs7vjrss62u/UOA7WHHrgTmVXZt2Gf3ABOqYntVW0rnGmOaGmM6G2OuNcYciHBOexxxBdmMI5w2VbxH1PKM0zTchVPjCFJU0cUi4hGR+0Vko4jsxfkjAKfWEonvTfl+582uDUFCx2j244g5eK8/icgatzm/G2gS5T6x0BkYJCK7gy+c5nJbEekkIYPPVSxvH9A45H1jYJ9xVRNnVDOpoZlwTeC+/wHAGLPUGLPLGOM3xszCaRWcFwc7QTUCtaeRWP7Wo2kgqj6qcDxm4jlPaRvOH0SQTji1hB04EbpG5bndEi1wonCQaOWOAs7BqU02walVAVSUgdLMvUeQTq4NUXH7ef+M0x3QzBjTFNgT5T6xUAR84P7hBl8NjTHXGGO+MiGDz1UsbxXOwGeQvu5niUI1k3jNlNOE+326UbEuTJzsrCqqkeppJJa/9XANdAVygC/cl1dEuldQVrRrq0U8g9JU4AYR6SJOauffgReNMX5gJ2Dj9AnHUt6lItJPRHLc8j41xhRW8fpGQDFOLegI9/rKuEuctOohwJnAtCreJ/gdvSJyB4fXHCpEnIHcekCW81bqiUi2e3gGcJSIXCwiWe5roIj0ilJeTrDfGch2ywsKeRLwRxHpIM7g7jicLoREoZpJvGZeA34iIue7Zd4BfGaMWeve61ci0tC95y9wMjunV9XWOKAaqYZGiO1vfTJwlogMcQPo34BXjTE/uK28V4G/iUgDETkRJyg/V9m14GQwurryAB5Xq1GzGuMZlJ5xDZ0PbAIOAr8DMMbsxxnA+9DtUjiussKMMe/iZHK8AnyNU3sbEYM9k3CayluB1cAnlZy/HafPdhvOL/rq4B9mJbwNvIVTM9iM870PNf/dhxWte+0k4AAwC6cWdQB4B8B9sL/A+d7bXBv/gVMTqYh1bhkdXNsOUFYzfAJ4E1gJfA7MdD9LFKqZBGvGGLMTJ7vqXve7DKL87+wPOL+P3cD/AVcaY+ZV4TvGC9VI9TQS9W/d7cIdAmCMWYWTEDIZ+AYnIF4bUta1QH332FTgGveaqlx7G44+b8Kp0BygktR/qZ3hhNRCnEmmzxtjOibaFiU1UM0olaEaqR669p2iKIqSNGhQUhRFUZIG7b5TFEVRkgZtKSmKoihJQ0IWHGzZsqXJzc1NxK1TniVLlnxrjGmVaDvqEtVLzVDNKLGQaL0kJCjl5uayePHiRNw65RGRzZWflV6oXmqGakaJhUTrRbvvFEVRlKQhNYNS0UIoGO/8VJTKUL0osaKaSRipt4lV0UKYeDYESsCTDWOmgy9jNpRUYkX1osSKaiahpF5LqbDAEYsJgL8Y5t2ntRmlYkL1EiiBFVO0BqxER31MQkl8UApvJlfWbM4d4tResAAbvpzn1GpUNJlBJH1E00xQL+IBywPLpsDce1UzmUKsegH1MQkmLt13IvIMzuq33xhjDtvSuELCm8nD7oe3bir//sAuRyShzed+I+Drz2DbMjC2c35hgTaxU4S46WWMu1h18DPLA8deBH1HHq6X4Ir/SyaWtZpUMylBrelFfUxSEq8xpQk4WxRPiumqcs3kg7BsUvlm86xxYNtgWTB8PLTpXd4BWVlg+x1x5cayfb2SYCYQD72smAJNfCHdcwFY/AwsnXS4XoIOyJNd9l41kypMoJp6MYESxASw/cXsnfU3aJZLE/cz9THJSVyCkjFmvojkxnxh7hDnwQcCgHFqJpYXbDAiYAcQDMa2kZk3QI/hZQ7IBgZc4jil8FqOktTETS/LpsDp/3Qchv8gBuO0h2w/zPwjtO8HgeKymu6BXU5tubBANZNC1EQvAfGCbePBptG2Bfi3fUwJgkcsQLDsABbGCUzqY5KCxGbf+fKd7pbFzwIG2w6wOedIlpT4WHzQx91ZE/ASQASMsbHXzkTEcl6ebOg7qmKhFC1U55NuhOsl4Oejgvf4Xk6mnv0tp8hSPNiuXgKwdYl7oauXoBYi6UH1kn748tl7wSt45v+Dxts+dLSBzTeNerHB040Pf2zPH0v+h4T4GNbORCzXLUbzMaqXWqPOgpKIXAVcBdCpU6dDnweOHoFZOgXsYjzGplPxWjrIl2Qd/wSf+330XfE3wEYAC4OxA/jFw/JeN9K/w0A84AhkxRRAnPEE0JTOFCeSXowxLG82jN4yGY9dSgBh4J7ZeAlge7JY2m40A76eihinhS2AMWCLYVmvv3Cs6iWtiaSZ5j2HQIMc5/n6i7GwabtvDW09XzJ4zHR2buxKi3k3EfQxBoNt2zDgEqx+bkAKBqD6LZzWdv0W5ce+VS9xpc6CkjHmSeBJgLy8PAOw4ZsfuH56CdkHbuL2htPpV7ocDzYe/JzT9EsYMg58zdx+X6fLxhKwMby/dB13f/0hjw7x45v+a0cgAMsmw7Gjy6cB6wBlyhFJLwA3LaxPe+sOxnbcwjGNfqDpmqmIcTST36sLnD7bCThLJmGMUwMGmLt0DbcVFfDgiSX0fGuk6iUNqUgz+PKdwDHvPieTLiRxodXQcdAwBzPzBoxxAlPA2Ly4zjBoUC+ODCZL+IsBG8RyXsbWBIhaIqEp4Y3qZXGgJMBlIy+k38X3I94cJ3XXk+3URgrcgcdLZ0PeWMTjHPd4czjup+ew9fsDvPTyVOxAaVmhgRLAlKUB6wBl2iAiPHHxAP578zWcfPn9NDt+jNMtF0z33rPFOfHMB+GMfzvdMGJheesx+Ofn8sNBP9PfeEn1kon48mHozeD6kEN6KVro+JYzHkAsL0Ys8OTwzo9HMvzhBSz5YDomUIIzwIQTiOxgcFK91AZx2U9JRKYCQ4GWwA7gTmPM0xWdn5eXZ4KLJdq2wbLc6mxoMzlS8zisH3fnD8U8Mmkyt3xzI9nid2zx5MDYGU55adjnKyJLjDF5ibajJtREL4cR7IpbNqUsS6oCvewr9jPhxRe5cuPvM0YvkPqaiVUvEEUzVdTLzqZ9+csrn7F73QJeqHcfWaYUCbaUPDkVp5KnAYnWS7yy70ZW99pDAQnKBqELxkfuTgkbpG7VKIfbrx7Lsy8eQf3VL9G1VQMGnXsdnuA5OkCZlNREL4fhy3eep+2vVC8Nc7xcd8lo3nunMdvnP0uTI7I58fzraKZ6SWoSoZdWwFOX5PHf+c0Y8Q6c23QTvzzxaBrZe6NrQfVSY5Jz7bvgjOpo80nch+/NHcIVIy/kgXf7c9t767nxg41cm7sA6RJBFLqmVXpSFb3AIc38tNcQCro8wVWTltB+hp+XWUCzbz493JGoXtKTKurF2rqIaz0FDBjWmzFzejLhk/pMvmIQ7faudCrOqpdaITmDUnBgsqIaR9jDlzHT+ePP82mzZwXnrfwD5ku/Mz4VLorwddB0gDI9qEwvcJhmhoyZzoRLB/Lgs5OpP/VujASc8alQzahe0pMY9TLIk82rZz3PhTOLufvxCTzq/ysSKD088Khe4kLi176rCF++m30X4aFGevjAqNZfkS1+LGxsf9nnhwhdB00HKNOLaHqBiJoZ1LUF9/bfjdf4ERNwBrRDNaN6SV9i1Evv4hVMuCyf7vuXO74lzPcAqpc4kZwtpcqooPktXYZgeXMI+IspMR6+qt+XHqHXVaWGpKQnFWima94w/CsfxR8owYiXrFBHonrJXCLoZYCvGVnDz6dk9qtO5deTjahe4k7yBKVYBggrevi+fGTMdErWz+OGTxvy2RxhZq8SmjXILn+tiiX1iXVAOYpmvJe+yXuzX+HRTW0Z+11HzvaFXad6SQ/i4GOOOe4XLNg/gY/ee4Oc3JP4fceBSPh1qpcakRxBqToDhBU9fF8+9X35/PaoPZz/+Efc+PJn/O+SAYjI4ecqqUl1B5SjaOaky/N4/MlPuOmVzzimQxNyWzaIv91K4oijjxl86hkssY/igXe/oMlHhYw9sUstGZ2ZJMeYUgVjRDETsk/K0R2b8OdhPXh3zQ5eWFQUX3uVxFILesnyWDw88li8lvCHF5dTGrDja7OSWOKsmd91/46f9WrDPTPXsGTz9/G1NcNJjqAUjwHCYE0oZAO3y07swgndWnDPjNVs3X0g/nYriaGW9NK+aX3+ft7RrCjazZPzv4y/3UriiLNmrOfO4aHBJbRrWo/fT13GngOllV+vVInkCErB/ttTb61+bn+EmpBlCf84/xgMcPOrKym3ekVlu08qyUst6QXgzGPac8bR7Xjo3fWs3/FD2fmql9SmFjTTYNvH/Gdkf3bsPchtr39++PmqmWqRHGNKUPMBwgqyq3zNj+Cm03vy+vTXWDNtJr2PPwN2rHYWeTW2s2SITnJLPWpJLwB3ndOHDzd+y4QXX+LufruxivfCx4+oXlKdWtBMP19T/vDT7oyf8wUXttnG4Ky1znnqY6pN8gSlmhIlHXN0+x1cmPN3PKv9mHWPIXbAqe2AswmcTnLLPKLopWXDHMYfV8wJH94Ec0uBkBa26iVzqUAz1wztRtHKD8ibf7MzCdvyOLsaqI+pFukTlODwmpCbAurZswVLAoixsQOlSKiTEUsnuWUqFeiF3CGcWu8LbPE7e3hBWdqv6iWziaAZb2EBt7TfhPd7v7Noa8AG9THVJr2CUiihKaCWB7G8BAIGv7HI8lpYAT9YFgwfrzUY5fClq4bdj3iy8fuL8YgbliyP6kUpI0QzTS0PfsuD3wbxePGIQKBUfUw1SN+gFDooaQMDLuFg/fb8pqA+vVo15tY+3+qsa6WM8MSHA7uwxr7JOzOmUbDV5k+DW9Gsz6mqF6WMcB9z7CU8+7mfZVYfHriwHzlbPlIfUw3SNyiFD0r2HUUDXz6DvRu5f/ZaTvnZGZzga5loK5VkIVLigy+f/qOP4Y//msf3u1rxmG9Aoq1UkokwzXiPHUXv3l2596lP6b2pJdedOi7RFqYkyZESXhtUkAI69oRcOjStz32z15aliGvqplKBXlo3rsdVJ3Vl1srtLC/a7ZyrelEgomZOPLIlp/+kLY++v5FtwbmRqpeYSN+WEkRMAa2X5eH6n3Xnxpc/Y/bn2xnetEj3QFEcKkgZvmJIVyZ9vJl/vb2O508T1YtSRgTN3DK8F++t/YZ/vb2Of59QonqJkfRtKUXhvP4dObJ1Q8a/sw57U5yWH1HSloY5Xq4d2o0FG75l85K3VS9KVHzNj+DywV14ddlWtq+Yo3qJkfQPShGazh5L+OPPj2Ljzh8pKO2pe6AoZVTQ1XLRcZ1p3SiHJ4vaY1QvSigRNHPt0G60aJDNY4XtVC8xkt7dd8GUTX9xWWpm3lgAhvVpS8+2jbhrmc07F7+O96sPNVMm0wmbRsCxF0HfkeDLp16Wh2uHduOvbxZz4TmTOMa/UvWiVOhjGtXL4nenHslf3yzhl2dN5Fj7c9VLFUnvllJhgSMWbLD9zrIfbm3GsoS/HruPYd9P4aONu6LvQqlkBuFp4YufPbRYK8CI/E78tEEhqz+apQ5GcYjiY0YO6sRpjTezYsEMTOfBqpcqkt5BKXeIU3sJYuyyPt2ihQwquIxxWdPInz8We/OnibFRSR6CKb6H1m8w5cYB6m1fwhPmLn61dyL2hLM0m0qJ6mNyvl7Co/67uGj/89ghlRslOukdlHz5TnPa8jpLfXhyyvp0CwuQQAkebLzGz4ZFbyXWViXxBFN888Y6WgkfBygswGP8eMXG6KC1ApX6GI8pPaQXe5PqpSqk95gSOA6mTe/DF950a8UmUIIfD09t6cA/jNEdajOdYIpv31ERNSOebGx/CSXGwzeNBtA5sdYqyUAUHxPUS6nxsCTQk8EJNTQ1SP+gBJHnn7i1YiksYN6+7rz0gYdzv9zFCd10lQeFqJop/mIel82rR7t1Tfl3v8SYpyQZUfTCpgL+8klD1i5vyFsnGyxLK77RyIygVBGukIaWBmi55H3emzODE7bt0kFspWJ8+dT35dNz/yo+/2QOe5u/TeOep6helMj48rF8+fysyTamT13GJ/Nnc4JnjfqYKGR2UHKpl+Xhpp/s4YxlN2J2BBCdea1UwtVdd9F48b3kfOiHTx9QvShROePodrzz9nT6z7vd2XNJfUyFpHeiQwycaT4ghxJEZ14rVaDNptfIkVIsNOlBqRyPJVzfagnZRn1MZWhQAihaSL3PpyICxoCxPDrzWqmYooWw7HkEgzFgo3pRKqFoId22vO74GHCy9VQzEdGgBE6NxQ4ggI2wouWZ2qxWKiZULyK8aZ2C3WFgoq1SkpnCAiSoGSN80+189TEVEJegJCLDRGSdiGwQkZviUWadEpw0KR4CVjYP7hzAwdJAoq1Ka1JaMyF6wZPDpP3H897abxJtVVqT0nqBsiko4qFUsvjf3uMSbVHSUuOgJCIe4FHgdKA3MFJEete03DolZF+Utb94nnn7uzB9xbZEW5W2pLxmQvRiLnmDrxsfw8SPChNtVdqS8nqBsikop97KG8f8l/8VtmT9jh8SbVVSEo+WUj6wwRjzpTGmBHgBOCcO5dYtvnwYMo6jj/s5Pdo0YsKHhWWbACrxJvU14+rF2/k4LjquMws2fKtOpvZIfb3AIc384rSzqJ/l4cn5XybaoqQkHkGpA1AU8n6L+1k5ROQqEVksIot37twZh9vWDrJlEf9s8y71ti9m4abvEm1OulKpZlJFLwAXd9jB77Km896cGYk2JV1JKx/T7Lvl/KfjXApXvM+OvQcTbU7SUWfzlIwxTwJPAuTl5SVnE8Rdhv6YQAmTczw8Orclg7pe7HwevoSIUqukhF4AihbS+KXzud5TTMn6V9m3oTMNc7yqlwSQEppxfcxPAyWc6PHwytsduOjXv1YfE0I8gtJWwBfyvqP7Werhbl0gJkA2YAoL+HZNLi1f+bVuZxxf0k4zHmyy8FP4/tP02jFT9RJf0k4vYgJki2HXqvc40L8j9af+UjXjEo/uu0VAdxHpIiLZwAhgehzKrXtCsqrEm8Uu05Af3robAsW6nXF8SUvN2GJhvl6BUb3Em7TUi1heWvi/YfPcp3TL9BBqHJSMMX7gOuBtYA3wkjFmVU3LTQjBrKoBl2Ad+TPuzpqEb89CjLHdZel1O+N4kJaa6TEMrxh6BNY7e+qoXuJGuurFMgFGeufSZevrGMurW6a7xGVMyRgzC5gVj7KSguUvgP8gXow7odZCug6FoTdndLM6nqSdZr54GzEBLIEAgqfrUNVLHElHvWACeABjAhR1/hWduhylY0roig6HE9wS21kMBBuhFK86GKViCgvAthEc1djGYkf/G1QvSmRcvYCjFyMWT+x20sVVMxqUDie0z9eTzboO5zOy+BY+t3ok2jIlWckdAt4cwMJYXu7wj2VCUetEW6UkKyF6EcvLJz1vZvK2tny2ZXeiLUsKNCiFEzJbn7EzaT/6v6z29uS5jzcn2jIlWQlq5qe3YV06m297jOLFRUW6VJUSmRC9cOls+p17PQ1zvDxVsCnRliUFGpQi4StrSjfZtYwH289l84r32bO/NNGWKclKUDPALY1mkbv/c2at/DrBRilJS4iPabRzGQ93nMvXn3/A13sOJNqyhKOb/EXDneh2WqCEUyxh4+TPaDLsN9rvq0TG1UtuoIQpOR6eeX8X/NhMB6+VinE1c0qghBO9wqrJK2h31jUZrRcNStEoN9ENem59GTNxJjJmetlxdThKkHKTrw1X7n0MMxdnl9Fh98OBXaoXpTxhPqbfN69iJs7OaB+jQSkawaQH/0HAYAG2vxh56ybY/jnYfp2BrZQR1EugBBHBCgQQY8B/AGb+0TlH9aKEEtHHHEQ+fBA2zM3IVR50TCkawQHJvLHgycGPINiwdamu8qAcTkiSjAwfjy1eZydjcLSielHCCfExYmVhwNnReN1bGetjtKVUGb588OUjfUdR9PJtdNq9EI8E13oUnYGtlMfVC8C+LxfTZNVzITU/1YsSgUOaEVj8LILBNgaxLDJRM9pSqiq+fBoPu50SsgjgLiGTd2lGNauV2Gh2/CWUSg5+LIzqRamMviPBWw8/FqVkwfDxztSUDNOMtpRioEWvIYzv9G+8RR/ym9FjqNf1+ESbpCQzvnyWnTqR+W+/xkmnnMtxJw9PtEVKMuPLR8ZMZ/kH0/n7qhbc1voc+ndqlmir6hxtKcXIyT89gwcOnsXL37RPtClKCjBw8DDeaDSCf6/NPOeiVANfPr1+/Vc21uvDkx9k5s60GpRiZEDnZvyq9TYOzv0/7M2fJtocJcnxWMKNfXaT99WzbFw6N9HmKClAgxwvf+6zh67rnmDrynmJNqfO0e67GJEti7h/320QKMFMmgan/0PnnygVU7SQc1ZcQ8Bbgv3m69DK3TI9A+efKFWkaCGj1l5HwFOC/errUPzPjPIxGpRipbAAjylFxCYQKIZZ48CYjJtLoFSRwgIkUIpXbPx2KT8ufI4Ga6Zl5PwTpYqU00wJZtY4Z75bhuhFu+9iJXcI4snGxoNtLIxtZ+RcAqWKuJMjjXgoxcva7T/oLqNKdEI0Y7AwdmbNcdOgFCvuZLeSk27mbnMZpZIV3x0jixZCwXjnp5L6uHqRU2/l8c4P8O9vBmA8cdSM6iX9CNHMq+2up9hkYTLIx2j3XXXw5VPPl4/3x9WM/qQjT518kCa9Tq15s9pdnFG7dtIMd3Lkad328PDDC3jl2Mf5VYtNNR8jUL2kL65m8nru46IHGvG7rjsYetp5GeFjtKVUA648qQvLOYp/7T8zPg82uOttBjXVM4k+7Ztwas/W3PtZQ37M/0PNNaN6SXu6tWpIx2OGcs3moexs2rfmBaaAZjQo1YB2e1fyn47vs27xe3yz92DNCwzZ9TbTlhbJFP7cZw8ji19mztvTa16Y6iX9KVrInU3foo+9lsfmbah5eSmgGe2+qy4hey2d7PEweUYbrhg1omZlBhdn1HTh9KRoIT3fvojuWcWULn2NH/u0pUG3E6pfnuolvXF9TPNACVOyvVz0qbBlcBc6Njui+mWmgGY0KFWXcvugGI5YM42972ymca9TavagQxb0VNIMVzMebIwpZfubf6XB+feoXpTIhHS1ZWH4nfUy015vwQ2XXlSzcpNcM9p9V11CmsHi8XK+NY8GH93vDCImaVaLkmCCmsHCIwbf9wsxE89SvSiRCdGLYHOitZKrC29g3eJ3E21ZraJBqbqE7J1jHXsRWWI7NeAkHTxUkoCgZroNJRiYbL/qRamAUL2IhYUhS/x8Mnc6AdtUenmqokGpJvjyYcg46DsS8Wbjx6LEeDGdB9es3CSfR6DUAF8+DL0Z8eYQwEOJ8bImJw5ZVaqZ9MTVC54ct1cmmze+78LkTzfXrNwk1ouOKcUDXz4y5k1WFcxg6uf7GP3RTI4Wibnf1hjDyk/m0HvOxc5SRkk6j0CpIW4NOLDhAx768Ftavfc63Vo3IDu3mluhpMDcE6UGhCQnWJ0H03fGanbNvo+vG42i3U9Ojr28JNeLBqV44cvn6O6r6In8F4kAACAASURBVPPFOGStjb3hCawxbzoPu2ghrJgCiLORVwQBbF05j3dnv4pn7xZ6e0oQsbEDJViFBUklGCVO+PLJ3rGaPwfux5QGsCe+BJfNKNNLYQHUb1HxQpxFC9n1+XtM+NqHVVjA76xivG73sahm0o/gDtiLJ3DHt3/ClgD+l1/D32g63s7HVe5jgprKHQKFBRh/MYJNwF/MR3Ne48jzj6Zdk/oJ+WrhaFCKF0ULsWb/CSGAAAF/MYFlU/CsmAJLnwe71Dlv2WQYO6NMNEUL+fqDZ2m5fhqjJQBZXizLiz/gp9T2sKfZQNom7EsptUbRQpg1Dsv4QcDYJWye+xSdW0yBZVMgUArYIJbTdRNSmzWLn8XM/BNN7QDXksVs3x8IbMsCuxRjeclKwrknShxwNSO2H4+AMcUUvnw7R548Gmbf6LR84HAfs3iCu3C0jfFkM6XZtZxnvGSJHz9eHtrYhn+V2gn7WuFoUIoXhQVg2whgAGMEs+x5LONHCBmUDCZCuDVi/7Nn0TpQjCUGcU6AYy9hX047Lp+XQ7/NLbn1Jwn5Rkpt4uoFHL0gQtsvX8FsCpTXi7HLaebglx/jnTEOjwlgCdQTP+f1rI857U1mTJ/Gc9t9PNT4aNol5EsptUqIZgA8Al32LiIwcwkeExJUwnwMs8aB7QfA9hfz9fatzOj/X37ZfBP1up7ExNb9OSLbU9ffpkI00SFe5A4Bbw5gIeLh+/qdsWwnIJXLk3FnUfs3f8Kml29DAsV4JOQMEWjbj8Y9T2F0myICi56ldN6/knJAUqkBoXqxvNgd8smSCHoRy9FM/RZsn3kvc6Y+CMZGxAlmAlC/BdJpEPmnnM25Mp/CCb9RvaQjIZpxnzweMRg7gA1lugmu1FC0EObd56wyjrPDjkEY08fLr/N8eLueBCum0GDOjciWRQn4QpERY6qfWigivwb+CvQC8o0xi6tyXV5enlm8uEqnphbBft1lUzDB7hfXcxgspPNx2C17sHV/Fu3XPI24tV3EQsRyVIMBywuIk15uDEYsLK/ThSOdBi0xxuQl9HvWgOpoJq31Ehw7mn2j87wBDNhisaXXFTSzDhDYsZbG3y4BY/CLhyzLwrJLOSQubz0Ydn/5Mjw5iNuFIyIpqxn1MWEEfYw7JBD03rYBIxal7QZSv0krAMz6OW6XniFgALHwWB7E2GB5HH8THFbw5Bzq8ku0XmraUvocOA+YHwdbUh9fPjTxge1HsBEE9z9sY/AXfopZPAHf6ifxEMAjIAjSdSj0v8Qpw9jOeEKgxOnGEbCwk3bxxGqgmgkSnFJwYBfYAbfu6zSWjYGClRvJ+XwqTXcuwjI2HjFki43VfzR0O8VpRWEcbax5AwKlCI7eRPWSngR9jNtdJ+7/PQIYQ9a2RZi1MzBrZ0CgGMEQQNjZ6gQ8A8Y4AckEHB8TDEiQVP6lRmNKxpg1ACJS2amZQ3AWdqAELA+CYGw/IoJlAlhu98yh35jlceYhACyfeug6ENfJ2GVdOGkwgK2aiUBQM25GFGLh8eZwavfWZG0oC1aA06LuO8p5s/njsrTeXudA4YKywW7VS/oSwcdg+/FYrq9xT3P6XQSPtx7tzrnL+XDFC2U+plxLKXn0UmeJDiJyFXAVQKdOnerqtnVP+IKHgBQW4KnfAt66qczxII4who8vy5IJu67StOA0JmP0AuU14z5vyR1Ce4BNr5QFGomiF18+tOld6dSDdCZjNBPBx1BYgNRvgYRk4YmVBf0vLq+F8OuSUC+VjimJyLsQMSv5VmPMG+4584A/ZXx/b2WEzz+pRsBJdH9vVYi3ZjJWL1B+/knbvtWqoCS7ZtTHxJHw+UoQ84rgidZLpS0lY8zP6sKQjCB0dd4kn1VdE1QzcSSoGdWLUhXSwMdoSniiSIEdIJUkQvWixEqKaqZGQUlEfikiW4DjgZki8nZ8zMoAUmAHyNpANVNNVC+ql1hJUc3UNPvuNeC1ONmSWaTADpC1gWqmmqhelFhJUc3oMkOJJMl3gFSSDNWLEispqJkarehQ7ZuK7ARquCFIzLQEvq3je9aEiuztbIxpVdfGJBLVS5WIZq9qpm5IF80kVC8JCUqJQEQWJ3NabDipZm+6kWq//1SzNx1JtWeQrPZq9p2iKIqSNGhQUhRFUZKGTApKTybagBhJNXvTjVT7/aeavelIqj2DpLQ3Y8aUFEVRlOQnk1pKiqIoSpKjQUlRFEVJGtIqKImIT0TeF5HVIrJKRP4Q4ZyhIrJHRJa7rzsSYWuIPYUistK15bBljcXhYRHZICKfiUj/RNiZjqhelFhJNc2kol7SbUUHPzDOGLNURBoBS0RkjjFmddh5BcaYMxNgX0WcYoypaNLd6UB39zUIeNz9qdQc1YsSK6momZTSS1q1lIwxXxtjlrr//gFYA3RIrFU15hxgknH4BGgqIu0SbVQ6oHpRYiUNNZN0ekmroBSKiOQCxwKfRjh8vIisEJHZItKnTg07HAO8IyJL3J0zw+kAFIW830Jq/xEkJaoXJVZSRDMpp5d0674DQEQaAq8A1xtj9oYdXoqzttM+ERkOvI7TdE0Ug40xW0WkNTBHRNYaY+Yn0J6MQ/WixEoKaSbl9JJ2LSURycIRy2RjzKvhx40xe40x+9x/zwKyRKRlHZsZas9W9+c3OEv0hy/puxXwhbzv6H6mxAHVixIrqaSZVNRLWgUlERHgaWCNMebfFZzT1j0PEcnH+R3sqjsry9nSwB0sRUQaAL8APg87bTpwiZslcxywxxjzdR2bmpaoXpRYSSXNpKpe0q377kTgYmCliCx3P7sF6ARgjPkv8CvgGhHxAweAESZxy1q0AV5z9esFphhj3hKRq0PsnQUMBzYA+4FLE2RrOqJ6UWIllTSTknrRZYYURVGUpCGtuu8URVGU1EaDkqIoipI0aFBSFEVRkgYNSoqiKErSoEFJURRFSRo0KCmKoihJgwYlRVEUJWnQoKQoiqIkDRqUFEVRlKRBg5KiKIqSNGhQUhRFUZIGDUqKoihK0lDjoCQi80TkingY4y6f/qyIfC8iC+NRZiX3myAi98T7XKXqqH6UeKOaSm2qFJREpFBEDojIPhHZ4f4yGsZyIxHJFREjItG2yxgM/BzoaIwJ34wqbXGF/w8R2eW+/hHcj6WC80eJyGYR+VFEXheR5iHHmovIa+6xzSIyKuRYOxGZLiLb3GeRW7vf7NB9VT+1SJz1c52ILBaRYhGZUCdfoBqopmoXETlFRN4XkT0iUliX946lpXSWMaYh0B/IA26rBXs6A4XGmB9jvbASYSU7VwHnAn2BY4CzgN9EOlFE+gBP4Ozp0gZnD5THQk55FChxj40GHnevAbCBt4Dz4/8VKkX1U3vEUz/bgHuAZ2rR3nihmqo9fsTRwI11feOYu+/c7XVnAz8JPyYilojc5tbCvhGRSSLSxD0c3Bd+t1u7OT7s2suBp4Dj3eN3uZ9fKSIbROQ7t5bfPuQaIyK/FZH1wPpI9orINBHZ7kb8+SEOOvy8oSKyRURuEZFv3ZrY6LDTmonITBH5QUQ+FZFuIdc/JCJFIrJXRJaIyJBov8cwxgDjjTFb3N/veGBsBeeOBt40xsx3t1y+HThPRBqJs7vk+cDtxph9xpgFODtLXgxgjNlhjHkMWBSDbXFF9ZO8+gEwxrxqjHmdBO2uWx1UU/HXlDFmoTHmOeDLql4TL2IOSiLiw9mpcFmEw2Pd1ylAV6Ah8Ih77CT3Z1NjTENjzMehFxpjngauBj52j98pIqcC9wEXAO2AzcALYfc8FxgE9K7A5NlAd6A1sBSYHOXrtQVaAh1w/tCfFJEeIcdHAHcBzXB2arw35NgioB/QHJgCTBORegAiMlhEdke5bx9gRcj7Fe5nlZ5rjNmI0zI6yn35jTFfVLGsOkf1k9T6SUlUU7WiqcRhjKn0BRQC+4DdOA/hMaC+e2wecIX77/eAa0Ou6wGU4mzFmwsYwBvlPmOBBSHvnwb+GfK+oVtervveAKdW5Tu45zd1r2nivp8A3OP+eyjgBxqEnP8STqsjeO5TIceGA2uj3Ot7oG8V7QoAPUPed3ftlAjnvgdcHfbZVtf+IcD2sGNXAvPCPvO65edW9XdXk5fqJzX0E/bZPcCEutCHair5NBVyzc9wui/r7NnG0lI61xjT1BjT2RhzrTHmQIRz2rsCCbLZffhtYrhPheUZp7thF06tIUhRRReLiEdE7heRjSKyF0fI4NQ8IvG9Kd93vNm1Icj2kH/vxxFk8F5/EpE1bpN8N9Akyn3C2Qc0DnnfGNhnXFVUcm7w/B8qOZZoVD/Jr59UQzVVe5pKGPGep7QNZ2AwSCecSL8DpzZQo/LcMZMWODW7INHKHQWcgxPtm+DUjAAqykxq5t4jSCfXhqi4fbV/xmnSNzPGNAX2RLlPOKtwBqmD9HU/q/RcEekK5ABfuC+viHSvYlnJhuonsfpJR1RT1dNUwoh3UJoK3CAiXcRJz/w78KIxxg/sxMn+6hpjeZeKSD8RyXHL+9QYU1jF6xsBxTg1mSPc6yvjLhHJdh/qmcC0Kt4n+B29InIHh9dGozEJ+KOIdHAHTcfhNM0jMRk4S0SGuGL9G/CqMeYHt0b1KvA3EWkgIifi/AE8F7zY7VPOcd/mBPuYkwTVTwL1A07GmKsJD+ARkXqS2llkqqlqaEqcBJF6QJbzVuqJSHZVr68J8Q5Kz+A4wPnAJuAg8DsAY8x+nEG4D0Vkt4gcV1lhxph3cbKDXgG+BrrhDOxVlUk4zd2twGrgk0rO347T77oN54/3amPM2irc522cVOsv3PsdJKQJ7zqAfVGufwJ4E1gJfA7MdD8LXr8vmDljjFmFM/g6GfgGR3zXhpR1LVDfPTYVuMa9JsgBnC4cgLXu+2RB9ZN4/dyGo4mbgIvcf9dGqnVdoZqqnqZOwnn2s3BaZweAd6pw3xojkbudMw8RGQo8b4zpmGhblNRD9aPEm0zVlK59pyiKoiQNGpQURVGUpEG77xRFUZSkQVtKiqIoStKQkFTPli1bmtzc3ETcOuVZsmTJt8aYVom2oy5RvdQM1YwSC4nWS0KCUm5uLosXL07ErVMeEdlc+VnpheqlZqhmlFhItF60+05RFCWcooVQMN75qdQpCQ1K/oDNA3O+4LsfS2K7UAWTsbz1+XY+2vhtbBepXpRYKFoIE8+Gufc6P1U3dUpClw9Z+tVuHnl/AxM/LuTm03tyQZ4PqXjDTIegYAIl4MmGMdPBlzEbQmY0wUrM9r0Hee3aE+jaqgobjapelFgpLHD0YgLOz8IC1UwdktCWUn6X5sz6/RCOatOIv7yyktFPfcqW7/dHvyhUMP5imHef1mQyBK/H4qkxeXgs4YqJi9mzv7TyiyI5GEWJRu4QpwIjHrA8sGeL+pg6JOFjSj1K1/BCr494cmiAFUW7+cuDT7PqxTsrFkH9FiCCs9itDV/O0yZ2BuFrfgRPXDyALd8f4JrJSygN2NEvCHUwnmxHP9qVlzlE6rqtSnduvxHQ43RAYMlE9TF1SFy670TkGZzVa78xxhy2JXGFuF0rVqCEX3iyKfjZ32gw9248q0spWfs4DLuP7JLdjmPx5Tvnv3UT2LYTmAxgbG1iZxJFCxlYVMATQ3tw6Xu7uOONVfw9bz+yeUGZTsLpNwIQaNvX0Y925aUUNfUv5Z43lP9s2P1wYFd5HxM8LuL4F/UxdUq8xpQm4GwxPCmmq8p1xR2k+boXMOJHsPHbJTD7RgzGGWfqcTo0bO2cjw3GcprWxjjiyq3y9vNKgomHkznFk83dA/7Da4veJfD5fXiN39HDsRdB35GHOxhPtrMxtI4VpCITqLF/cbv6m+WW/2zWuLJKbqiPMQHXx1iAqI+pQ+ISlIwx80UkN+YLc4c4jiQQAAx8/RliecEGSwRj/AhO3JG1M8DKAvd4xFqOkipMIA6VmNH1PqRhOwt2lYLYjo4WPwNLJ8Hx18H2zyBQXFbTxa3ABIOUOpmUoEb+xZPtBJ9gV7/lLfMhImC7vscAa2cAbmUXovuYooWOHtX3xJ06y74TkauAqwA6derkfOjLd2q2i58FjCOQDsdAu75YbftizxqHsf1lWyXapdBhALTrW1YbrggVTdISr0qMtXwKw39xH/7ZXsSUYolxtGL74cMHQy60HAfTd5TziqQL1UvKc5iP8eU7XXbz7nMCkrEh4IcOxzo+pG1ft6XkDynFBiMwYEzFPkYzOmuVOkt0MMY8aYzJM8bktWoVsoJF35HgreeaYsO2ZbB8KrTpjTV8PIjHGTpyKzOHjodStBBmXA8zbnD+rfMMUh4RuUpEFovI4p07dzofBisxwWqK7Sdn50oCR49gvgwkgBV5H2rBqfH68p3XkHGHByTVS8oT0cf48mHozeDJIZKPwfUxYSVBk45lGgkmRiye4PxcMVUzOmuRxG9zHLE24z7oIeOQNr3xL5tC0aqP6XRwHR5sTKAECY4HFC2ECWe43TPAsslw7GgdO0hxjDFPAk8C5OXllcWaviMdhxIocVpNy6bQwPZzsieL/xWfweXeWXiwkfDwdGBX2b+LFsKKKYA45em8lPSmEh9Dm95Oy3rdWzhdvDllXbvBCkuwC1Cs8l2A2g0cdxIflKCsNrP547ImcTB1N3cI3rMfpFPfTwlMOAtjl2LEi7fzYKe+XFgAgZD5Kjp2kN4EHUxhgTN/ZMlEMAEsG07ueyQXLL2dyztuZfhRRyAfP+o4oHAnE16JOf2fqpd0J9zHhM4/8uXDiCmRu3CDFRbcqQfGdoYZBlwCTXza3VsLxCslfCowFGgpIluAO40xT8dUSKizqd/isNRdT+dBWJe+ydy3XuXRTW05ZnlD7vQZJHcIeLLKnExlYwdK6hPshitaWK7V1LP+HkbkH81vPz2KG3v34LeXnlmBkwmrxBzYVaY91UvSERf/AmU+ZsUUWDbFqdAsn1o2JhR8hRKeLCEh45Oqk1ohXtl3I+NRziFRFIyP2J0inQZx6pX5fDxzDU8t2ESWR7hl+EBk7Mzy3TFBsVQ0SKnOJ6HUppP5tWcqW3v8i/97ex1dW/bn9CHjyl8TqRIT1ILqISmJm38B5xkXFjjJDVXprg2vLFeW7av+pcYkR/ddOMHaSYTuFBHh1jN60WbvZ3z/0YO8VHwaF553/uGDkpFEoVkzSUFtOhkJlHBdl+3MP9CFG15aTsdmR3B0xybOuUGHcfr/wfblHKrEQHTNqJNJL6L4l3KEPvvQyk1FPkb9S1xIzqAUWjuJ4AxkyyKu+PJ67KxiSla8xqz6Xoaffk7lotAB7fQkzMlkdTuJJwf05dxHP+SKSYuYft1g2uz5LLI2omlGnUx6Uol/ASp+9tE0of4lLiR87bsKiZS6G6SwAAmU4MEmW/x8/uFMZq38uvLFN8PXQdMB7fQg6GROvfWQk2jVKIenxuTxw0E/V0xcTOnG+ZG1EU0zuphr+hLNv0DFzz6aJtS/xIXkbClVRkjN2PJks7vZIK5/cTldzu5Lr2jN8qrUkJTUJMKYUK92jXl4xLFc+dxiHqzfmj95spFwbUTryqlqN4+SflT07KNpQv1LXBBjIk43rFXy8vLMYVsVx9p3H3L+9837cd7jH7HnQCmzfplD2+8Xpa0oRGSJMSYv0XbUJRH1EgNPzt/I32et5b6B+xnZ+qvYVnNIgzEl1YxLDXxMJq3+kWi9JEdLqTp99yE142bAM2MHcscjzzDzzXWMuGAUDdJQLEoIMTiGK4d0Zf2Ofdy8aAtHjBjBOb4O5U+IlnmnWXnpQQ19TLly0jggJQPJEZTiMEDY5cAqJnjuwRwsIfD8SwTGvomn86BaMlhJKDE6GBHh3l8ezebv9nPjy5/ha34E/Ts1UweTScQjCUETX+qE5Eh0iMcAYWEBHrsUr9h47FI+eu/1+NupJAfVSEDI9lr896IBtG1cj6smLWHn6gJd7y6TiJOP0cSX2ic5glKE7KmYcUVnxINtZfHA+ta8s2p7xedXZfdJJTmppoNp3iCbZ8bmUewPMGP6S5hYHIzqJbWJo4+psu5UM9UiObrvoOZ9967opLAA0/EE/DMDjJu2gjfbNCK3ZYPyXTU7VjtL1gfXRdNmeGpRgyynI1s34tFR/Xl44hpGZ3vJEpCKHExQMwf3wsePqF5SnTj5mColxaiPqTbJE5TigSu6HODRUfu55eGnmff0DEYN7Uf2nFvLFmK0A04NGZxN4HSSW+pRAwdz0lGtKDzzXEZMN/y+2w6GnnZexat/BNc8C6J6yWzCdRcMRKHrdaqPqRHpFZRC8P34uZP4sL8EZk8GMe6S9TaEbmsgls4/yUAuOT6X9Tt+zthPNvPPfu24wEf5mm746tBBVC9KkNDEBxHHv6iPqTFpG5SCiQ+Ijd8ItniwRNytjsVZKdqynE2+tAaTkdx5Vm8Kd/3Ira+tpE9gLX3mXFyWWTXs/vKrQ+NqR/WiBAlNfDCW409QH1NT0jcoBRMfAiUE8HCvGcsNJ7akcc9TnOOaCpzxeD0Wj4zqzy8f+5C5b71Ib0qQYOJD6HYWVVkdWsk8wld3GHZ/mU5AfUw1Sd+gFJL48F3TPF6cdoB1hU15/qcDsSxRoSgANKmfxTNjBnLHoyspNh5yQhMfdOKsEo3KEh9UO9UifYMSHHIq7YA7D37FX15ZyTMfbuKKIV3Ln6eTKDOa3JYNuOaiUVz8TIBft9zMeeddiDeaDlQvSpCqVFxULzGR3kEphAvyfMxZ/Q3/fHsdQ7q3okfbRs4BnaWtAMd3a8Gvzj2PP7+yktUrGvHXzhWcqHpRYkH1EjPJMXm2DhAR7j//aBrleBk3bTmlATerSmdpKy4XDuzE5YO7MOGjQp77ZHPkk1QvSjjRJsmqXmIm/VtKRQsPbZXesu9I7jn3J1wzeSmPz9vI73/aXbcnUMpxy9E/0GvDuyydMZfCr5uSm3da+Zqt6kUJJXQ+WzDTLm9s2XHVS8ykd1AqWggTznDnmwDLJnP62Bmc1bc9/5m7ntP6tKWH7oGiBClaiOe5czjfX8z5Xht7hWCvegRrzJtlulC9KKEUFpRNG7BtZxWHNr1VLzUgvbvvCgucuQJB3ObzX8/qTeN6WTw99QXs+eOdY9F2oVQyA7erRdwJsx4Mtr+EA+vnlZ2jg9ZKKLlD3PlJLsYu30WneomZ9G4p5Q4BT1ZZS8ltPrdomMNDJ5Yw4INbYK4fvLo2lUJZV4u/GMHGYFFqvNy3uiW3n2yTtW2xDlor5fHlO112oevcBbvoNMmhWqR3UPLlw9iZh8aU6DvykChO9K7FFj8WNiZQgujaVEpoV0v9FsiBXXxysAeT3hNK31jF31sWONup12RPHiX9yBvrdNmFt4jisYdTBpLeQQkqnEcgXYYg3mz8pSUExEt258FIAsxTkowwvZwCXBNYy+PzNnLikG6cqYPWSiQi+RlNcqgW6R+UKsKXjzXmTZZ+MJ2/r2rB2O99nN0p0UYpyciNv+jBlzv38fsFO2h35kQG2J/rGIFSOZrkUC3SO9GhMnz5HDvqbkrbD+TuGav5ceNHuimXchiWJTxwYT96tm3MmHfgi6OuchyMbuKmVIYvv3wSlWqmUjI7KAEeS7j3lz/B9+NKsp8/V7fHViJyRLaXp8bkUT/bw+UTF7HniwW6nboSG8HEB9VMVDI+KAEc07EpN7Vbjscu1pnXSoW0b1qf/12Sxzd7i/nktUcx/oOqF6XqrJgKqplK0aAEULSQgbtnOft0Acby6qCkEpF+vqY8farN0P3vAMbZyk31olRG0UJY9jyHNv9TzVSIBiWAwgLEDiCAbYQvO56jg5JKhQzOWkuW2AhgEDh2lOpFiU5hgbNFOoBqJipxyb4TkWHAQ4AHeMoYc388yq0zQjYE9OPhni39ePhgKY3qZSXasrQlpTWTOwTx5hDwF1NivKxsPAx1L7VLdfVy15urWL1tb63aVhW6lzTlduPFi8FPFndvOpr1T3ycaLMA6N2+MXee1SfRZhyixkFJRDzAo8DPgS3AIhGZboxZXdOy64yQDQG/qt+P918+yCNzN3Dz8F6JtiwtSXnN+PKRMdOxN87njmVNmPkuvNJ9L73aNU60ZWlJyusFWJ/dm7tb3Eefks9YlX0M67N7J9qkpCUeLaV8YIMx5ksAEXkBOAdIGcEAhya/dS9ayGOd3uLZj9bz5UAfXVs1TLRl6Ujqa8aXT5Yvnz/1P8j3Dz/DB0+/Sdtfj6RZj8GJtiwdqbZekqkFAMc7P3Q9vKjEIyh1AIpC3m8BBoWfJCJXAVcBdOqUpLNU3ZTN0wMlnOLx8MCrzbjlN2MSbVU6UqlmUkIvQJs9n/GkuQvbX0Jg6ksUj5lOTpfjE21WupF2PkbXw6uYOkt0MMY8aYzJM8bktWrVqq5uGxvBVaJNgBwJ4P3qQxas/1YnvCWAlNALQGEBll2KV2w8xs+cWa9gvvpU9ZIAUkIzFW36pz7mEPFoKW0FfCHvO7qfpR4ha1WJJxuT1Yzd067DBOYitl9rNvEjLTWDWHi/Xkbg2Ql4sVUv8SM99WJ5YM8WWDwB3rpJW08u8WgpLQK6i0gXEckGRgDT41Bu3RNcq+rUW5Fh9/MnewKnF78FAZ1UG2fSTzMDLsFjWfzCuwSPXap6iS9pqRcQWDLR2fZCfcwhahyUjDF+4DrgbWAN8JIxZlVNy00YvnynNrPmDSy7BI84k90Moiv9xom01AyCBEqwMCDOfDfbylK9xIF01cuhbjzbBrFAPOpjiNM8JWPMLGBWPMpKOMGByJCN3kqMxZq2Z9PvzGsyulkdT9JOM2Gz9V/np8ySodzb5BjaJNS49CCt9eLJgtP/CQd2aUYeuqLD4QQHIrEBC+k2lEc6P8SIbRfwdeOjE22dkoyEzdaX/hfT84qn+KikK1dNWszB0kDUy5UMI9LqDnljy68mnsFoUAonOBApHmeb9KE3c8G552HbMP6dLxJtnZKM0BxflAAAD7NJREFUlNNMPeg7it7tG/Pghf34bOse/jRtBcaYRFupJAsR9KKUkbmb/FVEhI25fMCYEzrz1IJNXHZiF3q315n7Sghh26gHB6p/0SefP5/Wk3+8tZYjWzfk+p8dlWBDlaRAN/+LigalSIRubezOvv7DUcfz0uIs/vHWWiZepiJSwgjqJWxi5NUnD2T9Nz/w4Lvr6d66EWcc0y6xdirJQQQfowHKQYNSNEJmXze0PExtdza3rf8JC9Z3ZXD3lom2Tkk2widGrpiCFBZwf/4JfLWrGeOmLcfXvD7HdGyaaEuVZCF0hQfLA8deBH1HZnRw0qAUjXJOJkCvba8wNed13p32KfbPT8M6+J3WbpQywidGLpsCgVKyLYuJA67muV27eWjCl9z7u8tp26Reoq1VkoEwH8PiZ2DpJBg+Htr0zsgWlAalaASdjP8gzkwlQzZ+Ti+ejcyc7cwt8ORk/AxsxSV0rCA4Ux8bbJsGix7hN1iMMV5ufzqbu6+7jPrZnkRbrCSaMB8DgO2HmX90NgLMwJVkNPsuGkEnkzfWCT4IAJa4x42tM7CV8vjyndTeviPBKv/nJdjkWAFaf7eIcdOWY9uakZfxhPoYCamkBH1LBq7yoEGpMnz5cOaDMHYG5I1FPDnYWGBwfuoMbCUSvnynC8byEqzMIBaWJ5seg05n1srtPPiuTjFQKPMxZ/zb0YtYzoTaYNp4hvkY7b6rKsFsmb6jkE0FPL10Dwf27OSykRfTIEOa1UqM5I0tGxeo3+LQjP2zOw7kw+LPeHjuBrq1bsg5/Tok2lIlGQjVSzAI6ZiSUim+fMSXz6CuezjrkQUcWN+cP3dLtFFK0hKa+usiwD3nHk3ht/u58eXP6NT8CI7t1Cwx9inJRbheMigYBdHuu2pydMcmnN23PU8v2MT2PQcTbY6SzETYKyfba/HfiwfQpnEOV05awrbdBxJooKIkDxqUqoPrZG495gdsY3hgjo4NKBUQnIcy917nZ9HCQ/pp/t1ynh4zkIOlAa6YuJj9Jf5EW6skCxm86Z9238VKyGS3Np5s/n3k71m7/HU2HzmKzv2GJto6JdmIMKGW5S8cWvXhqDHT+c+oY7l8wiJueHE5j48egHUovVPJSMK3TB92f0atIK4tpVgJdTL+Ys4sGs8N3mm0feOCjKzVKJUQuvimJ5ty++i4qb6n9GjNrWf05u1VOxg/Z12iLVYSTZiPYda48i3tNEeDUqyEOhnLQoyNFxuPXUrhkrdrXn4GN9vTkpDdjBkz3Zm/FCHV97ITcxmZ7+PR9zfy+rIYdvpWvaQfYT4GY8d3vlKSa0a772IlfEXot27CBErw4+GhjW34l23wVLf7JbzZnkGzuNOa8IyqCCtEiwh3nf0TNn37I39+5TN8zY9gQOdKMvJUL+lJBB9z6BnXdL5SCmhGg1J1CHUybXojhQUsKu3Ja+/A8Uu3cEGer3rlho8/FBYknWCUOBAhTRycjLzHRw/g3Mc+5DfPLeb1355Ix2ZHVFyO6iV9CfMxcZuvlAKa0e67muIuKzP4lOEc26kp//f2On4srmYWVfj4QwbN4s4YKuk6adYgm6fH5FFcanPFxMXRtaR6yQyCS1fFI3ikgGY0KNWEEAcjItx+Zm92/lDMY/M2VK+88PGHJKvBKDUkUnp4BI5s3YhHRvfnix0/cP2LUdbIU72kP/Ee/0kBzWj3XXWJ0Dfbv1M+5/Zrz/8KNnFhXic6tYjS9VIRFXTtKGlAeFbVvPtg6M0Rn/fJR7XijjN789c3V/PPt9dx0+k9I5epeklfamv8J8k1oy2l6hJp/knBeG7vuw+vJfxtxuq43KbEb8elHCUJCHadYAE2fDkvaotpzAm5jBrUif9+sJFXlmypS0uVZCBSJSZJM+biiQal6lIubdPd0G3uvbR45dfcm7efd9fs4L01O2p0ix+L/Qy8912mfPpVnIxWEkqw66TbUGcl6Eq2PnEy8vpwQrcW3PzqSpZs/q7ispM8zVepBjFWYtIFDUrV5f/bO/sYqcorDj/nzvItkYooirtSKlar7SJdlxKL0WIpUKy10aRSi5A2apMmbWJtq01sk9qWpqEa1KobG7/1Hz9JBAWMLVtDRVi2gFAspVggapVSFF13mbmnf7z37tyZncUddnfux54nWXZ27r27Z2YOv3Pe9z3vudG52fOudjfjCkZNlx6/m6knHcctz75GR1eh+t8dCMzml1/gUMcRPjVhzMDbb8RDfbObssuN6NNi87Ccxx++NZ1Tx43kuoc3se/ghz1P6uNalZEyqkxiqiLBSYwFpf4QvaFbpKKlbsqF3Pr1c9n/vw5uq/aeORGBaW5dwsVj9tA0+YTBsd+Ih2hCM3epE5lK4hAIx7gD7dx3zfl05l1F3uHOfKmoVCrzNbJBlUlMn0h4EmOFDgNB+Wa3Pa3MmAxXNTdwX+tu5n/2FKbx+tH3GoTicmhft8B4qiyc+Maxb8Y1kkt9M7y93bWQUd+JTriQvXeDW6Pc/Fj37bDPuGYFdy2czpIHXuXJlltZdPAOJLxu7lInVgO1wdJIFlF9CT/b1mU9tSTUkEoaEz2W8L1KFpQGigoic8slv6Jh9Ca2P/QEjYUVRRGJVtGUC5CXA68OvwBHyHFy4yXxvi5jcNi7wfmKH+xDKnQ6Pwh9odAFBKXggXBcOKuZ5Rd0MWfDcpBC8bqOAxW7RBgZIqyY2/hAUWO8Ord00HhV7wkOlF6TgiTGgtJAUS4y+Y8YtfrHXO8XIK/dd8Sm0FnMTMJhdP4jugXIB52+iEd2+GzgMyxv/nIcr8YYbPa0gh+trJSewSh8PiIc88f+E198JDhLxCsNROHUnQWm7NEjkemCjffD5kfAL7iRD/TUmPLkJ+FJjK0pDRQ9RAbwCwguIIUiQigi4TUlIuQEaOfEBTxz8JN8Z9J+vP2v1uwl1AIRuVJEXhMRX0Sa4rYnNibPgroRgOcy3oYZFX2BpiVOQABalyGjx+PVjaSAUFCPg6d9yR0LE5wXb4X757ns2MgWlTQGhcKRYkCCosbs3eDKyP0Kx+qbi1N5WVpTEpErgV8AZwPNqrpxIIxKJaHIdI96SnfhK1BQjzdn/pL6MIM5tNcJko+btps6Bx0zgda/rOPREXczclcBdt9THKInLKM5RrYB3wDujduQWClfh1x1IyU+Ix7M+53re/by7bDzeXc8NwKZu5T8vjak/THG/nst/oPr8KYtdHtZ8J1wrbzBXZtynzGNidBDY0KifpOD+cvc4we/FviEAuI0Zub3nc+9vd01es13uk7k85dB0+KavZSj0d/pOxOYkFBk/vQbt59AfcLxUfjvy7nz2bp+Kws/uIkTtrQU54U/fw1MbHQdx/OdLFYhJ+rWoAoFN0Rvf7yYMacYVd0Bbg/OkCdcJ2hdVprNhrzV7oJVoav4XDD9MmL86aj4iPrk813k8z7DPa+YSaufuAXsY8Q0JqSixkBxHkbg03Od32x+yPmKy3hdWfnZlxU7josEPqeJS2L6FZRMYMoIyzffWO8+eC8HCPh5xMsxi81cUNhArl1DF3JzvcefBh0H0EIXHj6+COLlwA9HXDrkSn1F5FrgWoCGhoaYrRlkwk2S4UhHvMgNAY+UnhuZ/pXccPxCF0f8HLe9PZ2fzJtGbtWPigvaCVvAPhZMY8o4isbg5eD11eBHfEY85wsX3VRadaeeC0wajLISlMTUrNBhyIhMpfLNoNTb2/QgHsWAFK4xfXjqTDbu+S8zNEcdilc3HJn72yDjKZYFp0VkRGQtMLHCoZ+p6rN9+R2q2gK0ADQ1NfXSkTQjlE/lhbe+BreI3T1S8uDMuSXXeHtaeen9M2hZV4c0TOGmJasSu4A92JjG7HOzKlGmXFTaXzFadTfjOlh/Z+KSmI8NSgMhMDAERSYqCOEaUvvj3dmwj+CrsLZrGi33vUKbnsmVJ/+an571LuPPnV28vnFh6kRGVa2OvVp6a5K5+DlXJn74HfjHGti5Cna9WCz5rW9mnipXd23j3j/vZupJjVwx64ba298PTGOOgd40JprE5Ea4KbtoRWZ51d1ZX02cvnxsUDKBGSAiDiGjxiNvtSObH2WOtDF7+Fa2zX6Yz838bs+Nsgnv6GsMMtF1p52rKm54FBF+fuk5/OvdD7j5qa1MHj86VV1ATGMGiPrmYhKDdK9T9+gyXh7MEqYvVhJeS8K2RE2L4fh6xC/g4TNM85znbxsSnRtE5HIR2QfMBJ4TkRfitikVfMzN2YblPO5aOJ1JnxjFjU9sIV+w7vJDkvpmWHA7LLjNTQOnsP1Uf0vCLwfuACbgBKZdVb8yIJZlnVBkErqrerBQ1aeBp+O2I3VUmnopY9xod9daX6Eul4180zSmH6RUY/pbfWcCc6z0QWQMo4Q+TLVMmXBcjYypDaYx/SClGmNthuIkgfO5hmFkiBRqTDbG+IZhGEYmENXaV06KyDvAGzX+sycC79b4b/aH3uw9XVUn1NqYODF/6RNHs9d8pjZkxWdi9ZdYglIciMhGVU1NA9C02Zs10vb+p83eLJK2zyCp9tr0nWEYhpEYLCgZhmEYiWEoBaWWuA2okrTZmzXS9v6nzd4skrbPIJH2Dpk1JcMwDCP5DKWRkmEYhpFwLCgZhmEYiSFTQUlE6kXkJRHZLiKvicgPKpxzkYgcEpH24OuWOGyN2LNHRLYGtvS41bM4lovILhHZIiLT47Azi5i/GNWSNp9Jo79krc1QHrhBVdtEZCywSUTWqOr2svNaVXVBDPb1xsWq2tumu3nA1OBrBnB38N3oP+YvRrWk0WdS5S+ZGimp6puq2hY8fh/YAUyK16p+cxnwkDr+CowTkVPiNioLmL8Y1ZJBn0mcv2QqKEURkcnAecArFQ7PFJG/icgqETmnpob1RIHVIrIpuJ1zOZOAvZGf95Hu/wSJxPzFqJaU+Ezq/CVr03cAiMhxwJPAD1X1vbLDbbjeTodFZD7wDG7oGhdfVNX9InISsEZE/q6q62K0Z8hh/mJUS4p8JnX+krmRkogMwznLo6r6VPlxVX1PVQ8Hj1cCw0TkxBqbGbVnf/D9P7j7xpT3md8P1Ed+Pi14zhgAzF+MakmTz6TRXzIVlEREgD8CO1T1972cMzE4DxFpxr0HB2pnZYktY4LFUkRkDDAH2FZ22gpgUVAl8wXgkKq+WWNTM4n5i1EtafKZtPpL1qbvLgC+DWwVkfbguZuBBgBVvQe4AvieiOSBDuCbGl9bi5OBpwP/rQMeU9XnReT6iL0rgfnALuBDYElMtmYR8xejWtLkM6n0F2szZBiGYSSGTE3fGYZhGOnGgpJhGIaRGCwoGYZhGInBgpJhGIaRGCwoGYZhGInBgpJhGIaRGCwoGYZhGInh/85cXCMeaOZOAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 6 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "This again tells us that the model complexity decreases with increase in the values of alpha. \n",
        "\n",
        "But notice the straight line at alpha=1. Appears a bit strange to me. Let’s explore this further by looking at the coefficients:"
      ],
      "metadata": {
        "id": "8mtV-ckZPh9T"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#Set the display format to be scientific for ease of analysis\n",
        "pd.options.display.float_format = '{:,.2g}'.format\n",
        "coef_matrix_lasso"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 423
        },
        "id": "nmXOcN9PQrtf",
        "outputId": "90f91d7d-aba2-49c7-e0c3-898d476374db"
      },
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-28a68ffd-7b05-474b-bad5-c119fd0aaa15\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>rss</th>\n",
              "      <th>intercept</th>\n",
              "      <th>coef_x_1</th>\n",
              "      <th>coef_x_2</th>\n",
              "      <th>coef_x_3</th>\n",
              "      <th>coef_x_4</th>\n",
              "      <th>coef_x_5</th>\n",
              "      <th>coef_x_6</th>\n",
              "      <th>coef_x_7</th>\n",
              "      <th>coef_x_8</th>\n",
              "      <th>coef_x_9</th>\n",
              "      <th>coef_x_10</th>\n",
              "      <th>coef_x_11</th>\n",
              "      <th>coef_x_12</th>\n",
              "      <th>coef_x_13</th>\n",
              "      <th>coef_x_14</th>\n",
              "      <th>coef_x_15</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>alpha_1e-15</th>\n",
              "      <td>0.96</td>\n",
              "      <td>0.22</td>\n",
              "      <td>1.1</td>\n",
              "      <td>-0.37</td>\n",
              "      <td>0.00089</td>\n",
              "      <td>0.0016</td>\n",
              "      <td>-0.00012</td>\n",
              "      <td>-6.4e-05</td>\n",
              "      <td>-6.3e-06</td>\n",
              "      <td>1.4e-06</td>\n",
              "      <td>7.8e-07</td>\n",
              "      <td>2.1e-07</td>\n",
              "      <td>4e-08</td>\n",
              "      <td>5.4e-09</td>\n",
              "      <td>1.8e-10</td>\n",
              "      <td>-2e-10</td>\n",
              "      <td>-9.2e-11</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>alpha_1e-10</th>\n",
              "      <td>0.96</td>\n",
              "      <td>0.22</td>\n",
              "      <td>1.1</td>\n",
              "      <td>-0.37</td>\n",
              "      <td>0.00088</td>\n",
              "      <td>0.0016</td>\n",
              "      <td>-0.00012</td>\n",
              "      <td>-6.4e-05</td>\n",
              "      <td>-6.3e-06</td>\n",
              "      <td>1.4e-06</td>\n",
              "      <td>7.8e-07</td>\n",
              "      <td>2.1e-07</td>\n",
              "      <td>4e-08</td>\n",
              "      <td>5.4e-09</td>\n",
              "      <td>1.8e-10</td>\n",
              "      <td>-2e-10</td>\n",
              "      <td>-9.2e-11</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>alpha_1e-08</th>\n",
              "      <td>0.96</td>\n",
              "      <td>0.22</td>\n",
              "      <td>1.1</td>\n",
              "      <td>-0.37</td>\n",
              "      <td>0.00077</td>\n",
              "      <td>0.0016</td>\n",
              "      <td>-0.00011</td>\n",
              "      <td>-6.4e-05</td>\n",
              "      <td>-6.3e-06</td>\n",
              "      <td>1.4e-06</td>\n",
              "      <td>7.8e-07</td>\n",
              "      <td>2.1e-07</td>\n",
              "      <td>4e-08</td>\n",
              "      <td>5.3e-09</td>\n",
              "      <td>2e-10</td>\n",
              "      <td>-1.9e-10</td>\n",
              "      <td>-9.3e-11</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>alpha_1e-05</th>\n",
              "      <td>0.96</td>\n",
              "      <td>0.5</td>\n",
              "      <td>0.6</td>\n",
              "      <td>-0.13</td>\n",
              "      <td>-0.038</td>\n",
              "      <td>-0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>7.7e-06</td>\n",
              "      <td>1e-06</td>\n",
              "      <td>7.7e-08</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-7e-11</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>alpha_0.0001</th>\n",
              "      <td>1</td>\n",
              "      <td>0.9</td>\n",
              "      <td>0.17</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0.048</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>9.5e-06</td>\n",
              "      <td>5.1e-07</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-4.4e-11</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>alpha_0.001</th>\n",
              "      <td>1.7</td>\n",
              "      <td>1.3</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0.13</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1.5e-08</td>\n",
              "      <td>7.5e-10</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>alpha_0.01</th>\n",
              "      <td>3.6</td>\n",
              "      <td>1.8</td>\n",
              "      <td>-0.55</td>\n",
              "      <td>-0.00056</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>alpha_1</th>\n",
              "      <td>37</td>\n",
              "      <td>0.038</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>alpha_5</th>\n",
              "      <td>37</td>\n",
              "      <td>0.038</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>alpha_10</th>\n",
              "      <td>37</td>\n",
              "      <td>0.038</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "      <td>-0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-28a68ffd-7b05-474b-bad5-c119fd0aaa15')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-28a68ffd-7b05-474b-bad5-c119fd0aaa15 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-28a68ffd-7b05-474b-bad5-c119fd0aaa15');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "              rss intercept coef_x_1  ... coef_x_13 coef_x_14 coef_x_15\n",
              "alpha_1e-15  0.96      0.22      1.1  ...   1.8e-10    -2e-10  -9.2e-11\n",
              "alpha_1e-10  0.96      0.22      1.1  ...   1.8e-10    -2e-10  -9.2e-11\n",
              "alpha_1e-08  0.96      0.22      1.1  ...     2e-10  -1.9e-10  -9.3e-11\n",
              "alpha_1e-05  0.96       0.5      0.6  ...         0        -0    -7e-11\n",
              "alpha_0.0001    1       0.9     0.17  ...        -0        -0  -4.4e-11\n",
              "alpha_0.001   1.7       1.3       -0  ...         0         0         0\n",
              "alpha_0.01    3.6       1.8    -0.55  ...         0         0         0\n",
              "alpha_1        37     0.038       -0  ...        -0        -0        -0\n",
              "alpha_5        37     0.038       -0  ...        -0        -0        -0\n",
              "alpha_10       37     0.038       -0  ...        -0        -0        -0\n",
              "\n",
              "[10 rows x 17 columns]"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Apart from the expected inference of higher RSS for higher alphas, we can see the following:\n",
        "\n",
        "1. For the same values of alpha, the coefficients of lasso regression are much smaller as compared to that of ridge regression (compare row 1 of the 2 tables).\n",
        "2. For the same alpha, lasso has higher RSS (poorer fit) as compared to ridge regression\n",
        "3. Many of the coefficients are zero even for very small values of alpha\n",
        "\n",
        "Inferences #1,2 might not generalize always but will hold for many cases. The real difference from ridge is coming out in the last inference. "
      ],
      "metadata": {
        "id": "hmVoR602Q_9e"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "coef_matrix_lasso.apply(lambda x: sum(x.values==0),axis=1)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "43M_fKAvQ3iK",
        "outputId": "242be1eb-672b-450a-f011-d90b30d76b9c"
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "alpha_1e-15      0\n",
              "alpha_1e-10      0\n",
              "alpha_1e-08      0\n",
              "alpha_1e-05      8\n",
              "alpha_0.0001    10\n",
              "alpha_0.001     12\n",
              "alpha_0.01      13\n",
              "alpha_1         15\n",
              "alpha_5         15\n",
              "alpha_10        15\n",
              "dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We can observe that even for a small value of alpha, a significant number of coefficients are zero. \n",
        "\n",
        "This also explains the horizontal line fit for alpha=1 in the lasso plots, its just a baseline model! \n",
        "\n",
        "This phenomenon of most of the coefficients being zero is called ‘sparsity‘. \n",
        "\n",
        "Although lasso performs feature selection, this level of sparsity is achieved in special cases only which we’ll discuss towards the end.\n",
        "\n",
        "This has some really interesting implications on the use cases of lasso regression as compared to that of ridge regression."
      ],
      "metadata": {
        "id": "WgiW_7hNRNBV"
      }
    }
  ]
}