Linear_Regression.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.2"
    },
    "colab": {
      "name": "Copy of Linear Regression.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "include_colab_link": true
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/alokproc/Verzeo/blob/master/Linear_Regression.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oBjrrWbT-kfR",
        "colab_type": "text"
      },
      "source": [
        "# Linear Regression"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-CoX51IE-kfU",
        "colab_type": "text"
      },
      "source": [
        "## What is Regression Analysis?\n",
        "\n",
        "Regression analysis is a form of predictive modelling technique which investigates the relationship between a dependent (target) and independent variable (s) (predictor). This technique is used for forecasting, time series modelling and finding the causal effect relationship between the variables. For example, relationship between rash driving and number of road accidents by a driver is best studied through regression.\n",
        "\n",
        "\n",
        "Regression analysis is an important tool for modelling and analyzing data. Here, we fit a curve / line to the data points, in such a manner that the differences between the distance of data points from the curve or line is minimized. The topic will be explained in detail in coming sections."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UXRm0V9s-kfX",
        "colab_type": "text"
      },
      "source": [
        "## Why do we use Regression Analysis?\n",
        "\n",
        "As mentioned above,Regression analysis estimates the relationship between two or more variables. Let’s understand this with an easy example:\n",
        "\n",
        "Let’s say, you want to estimate growth in sales of a company based on current economic conditions. You have the recent company data which indicates that the growth in sales is around two and a half times the growth in the economy. Using this insight, we can predict future sales of the company based on current & past information.\n",
        "\n",
        "There are multiple benefits of using Regression analysis. They are as follows:\n",
        "\n",
        "   * It indicates the significant relationships between dependent variable and independent variable.\n",
        "   * It indicates the strength of impact of multiple independent variables on dependent variable.\n",
        "\n",
        "Regression analysis also allows us to compare the effects of variables measured on different scales, such as the effect of price changes and the number of promotional activities. These benefits help Market Researchers / Data Analysts / Data Scientists to eliminate and evaluate the best set of variables to be used for building predictive models."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YLcL7VD8-kfZ",
        "colab_type": "text"
      },
      "source": [
        "## Linear Regression\n",
        "\n",
        "It is one of the most widely known modeling technique. Linear regression is usually among the first few topics which people pick while learning predictive modeling. In this technique, the dependent variable is continuous, independent variable(s) can be continuous or discrete, and nature of regression line is linear.\n",
        "\n",
        "Linear Regression establishes a relationship between dependent variable (Y) and one or more independent variables (X) using a best fit straight line (also known as regression line).\n",
        "\n",
        "It is represented by an equation Y=a+b*X + e, where a is intercept, b is slope of the line and e is error term. This equation can be used to predict the value of target variable based on given predictor variable(s)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iZkcGRwh-kfb",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# imports\n",
        "import pandas as pd\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "# this allows plots to appear directly in the notebook\n",
        "%matplotlib inline"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r-TDI9FA-kfl",
        "colab_type": "text"
      },
      "source": [
        "## Example: Advertising Data\n",
        "\n",
        "Let's take a look at some data, ask some questions about that data, and then use Linear regression to answer those questions!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Q1553bju-kfn",
        "colab_type": "code",
        "outputId": "ffbd24f8-dae4-44fd-bffa-efa03f41a6c9",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 413
        }
      },
      "source": [
        "# read data into a DataFrame\n",
        "data = pd.read_csv('Advertising.csv', index_col=0)\n",
        "data.columns = ['TV','Radio','Newspaper','Sales']"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "error",
          "ename": "FileNotFoundError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mFileNotFoundError\u001b[0m                         Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-2-3f9c148fe10e>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mdata\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread_csv\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Advertising.csv'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mindex_col\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      2\u001b[0m \u001b[0mdata\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcolumns\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m'TV'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m'Radio'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m'Newspaper'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m'Sales'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/pandas/io/parsers.py\u001b[0m in \u001b[0;36mparser_f\u001b[0;34m(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, skipfooter, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, doublequote, escapechar, comment, encoding, dialect, tupleize_cols, error_bad_lines, warn_bad_lines, delim_whitespace, low_memory, memory_map, float_precision)\u001b[0m\n\u001b[1;32m    700\u001b[0m                     skip_blank_lines=skip_blank_lines)\n\u001b[1;32m    701\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 702\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0m_read\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfilepath_or_buffer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    703\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    704\u001b[0m     \u001b[0mparser_f\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__name__\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/pandas/io/parsers.py\u001b[0m in \u001b[0;36m_read\u001b[0;34m(filepath_or_buffer, kwds)\u001b[0m\n\u001b[1;32m    427\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    428\u001b[0m     \u001b[0;31m# Create the parser.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 429\u001b[0;31m     \u001b[0mparser\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mTextFileReader\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfilepath_or_buffer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    430\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    431\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mchunksize\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0miterator\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/pandas/io/parsers.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, f, engine, **kwds)\u001b[0m\n\u001b[1;32m    893\u001b[0m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0moptions\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'has_index_names'\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mkwds\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'has_index_names'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    894\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 895\u001b[0;31m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_make_engine\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mengine\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    896\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    897\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mclose\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/pandas/io/parsers.py\u001b[0m in \u001b[0;36m_make_engine\u001b[0;34m(self, engine)\u001b[0m\n\u001b[1;32m   1120\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m_make_engine\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mengine\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'c'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1121\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mengine\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;34m'c'\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1122\u001b[0;31m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_engine\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mCParserWrapper\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mf\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0moptions\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1123\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1124\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mengine\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;34m'python'\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/pandas/io/parsers.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, src, **kwds)\u001b[0m\n\u001b[1;32m   1851\u001b[0m         \u001b[0mkwds\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'usecols'\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0musecols\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1852\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1853\u001b[0;31m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_reader\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mparsers\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mTextReader\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msrc\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1854\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0munnamed_cols\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_reader\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0munnamed_cols\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1855\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32mpandas/_libs/parsers.pyx\u001b[0m in \u001b[0;36mpandas._libs.parsers.TextReader.__cinit__\u001b[0;34m()\u001b[0m\n",
            "\u001b[0;32mpandas/_libs/parsers.pyx\u001b[0m in \u001b[0;36mpandas._libs.parsers.TextReader._setup_parser_source\u001b[0;34m()\u001b[0m\n",
            "\u001b[0;31mFileNotFoundError\u001b[0m: [Errno 2] File b'Advertising.csv' does not exist: b'Advertising.csv'"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WxaAEYMXB8x7",
        "colab_type": "code",
        "outputId": "973db52c-3710-46ec-debf-56a32fc36402",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "pwd"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'/content'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 1
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "c0DrLHNH-kfu",
        "colab_type": "code",
        "outputId": "fb9ab250-f722-4f84-fdf5-2ed4220ded22",
        "colab": {}
      },
      "source": [
        "data.head()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>TV</th>\n",
              "      <th>Radio</th>\n",
              "      <th>Newspaper</th>\n",
              "      <th>Sales</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>230.1</td>\n",
              "      <td>37.8</td>\n",
              "      <td>69.2</td>\n",
              "      <td>22.1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>44.5</td>\n",
              "      <td>39.3</td>\n",
              "      <td>45.1</td>\n",
              "      <td>10.4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>17.2</td>\n",
              "      <td>45.9</td>\n",
              "      <td>69.3</td>\n",
              "      <td>9.3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>151.5</td>\n",
              "      <td>41.3</td>\n",
              "      <td>58.5</td>\n",
              "      <td>18.5</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>180.8</td>\n",
              "      <td>10.8</td>\n",
              "      <td>58.4</td>\n",
              "      <td>12.9</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "      TV  Radio  Newspaper  Sales\n",
              "1  230.1   37.8       69.2   22.1\n",
              "2   44.5   39.3       45.1   10.4\n",
              "3   17.2   45.9       69.3    9.3\n",
              "4  151.5   41.3       58.5   18.5\n",
              "5  180.8   10.8       58.4   12.9"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": true,
        "id": "Rkg-jI1z-kgF",
        "colab_type": "code",
        "outputId": "5b702b76-cdcc-4d11-a525-002ef26f8c5d",
        "colab": {}
      },
      "source": [
        "data.corr()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>TV</th>\n",
              "      <th>Radio</th>\n",
              "      <th>Newspaper</th>\n",
              "      <th>Sales</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>TV</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.054809</td>\n",
              "      <td>0.056648</td>\n",
              "      <td>0.782224</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Radio</th>\n",
              "      <td>0.054809</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.354104</td>\n",
              "      <td>0.576223</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Newspaper</th>\n",
              "      <td>0.056648</td>\n",
              "      <td>0.354104</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.228299</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Sales</th>\n",
              "      <td>0.782224</td>\n",
              "      <td>0.576223</td>\n",
              "      <td>0.228299</td>\n",
              "      <td>1.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                 TV     Radio  Newspaper     Sales\n",
              "TV         1.000000  0.054809   0.056648  0.782224\n",
              "Radio      0.054809  1.000000   0.354104  0.576223\n",
              "Newspaper  0.056648  0.354104   1.000000  0.228299\n",
              "Sales      0.782224  0.576223   0.228299  1.000000"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VvjCEHnQ-kgc",
        "colab_type": "code",
        "outputId": "590b0779-54c5-42a5-823a-26cc5b74e56e",
        "colab": {}
      },
      "source": [
        "import seaborn as sns\n",
        "sns.heatmap(data.corr())"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.axes._subplots.AxesSubplot at 0x18865954240>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAD8CAYAAACGsIhGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAGBZJREFUeJzt3X2wZVV55/HvrxGkBYUyMBnlRUzSkgGkGmhQRFSUcTAilEoNML7hONPgG4kJiSQgQ0EZq2LwBQVMa3XwFfElWo3DVAPGFhTB7gBpXhSmqyV2BxMEBEZBoe995o+92z5e7737Xjj3nk3z/VC7OHvttddZe6P3Oc9a6+yTqkKSpOksGHUHJEn9Z7CQJHUyWEiSOhksJEmdDBaSpE4GC0lSJ4OFJG1lkixPcneSW6Y4niTnJ1mXZG2SA7vaNFhI0tbnYuCoaY6/CljUbkuBi7oaNFhI0lamqq4G7pumyrHAZ6pxHbBzkmdN1+ZThtnBrcmj96z3q+2thc8+fNRdUM88+MHXjLoLvfK0P/5EHm8bs/mbs92uv38yTUaw2bKqWjaLt9sN2DCwv7Et+8lUJxgsJOkJpg0MswkOE00W3KYNVgYLSeqD8bH5fLeNwB4D+7sDd013gnMWktQHY5tmvj1+K4A3t6uiXgg8UFVTDkGBmYUk9ULV+NDaSnIJ8DJglyQbgf8FbNu8T30CuBz4I2Ad8BDw1q42DRaS1AfjwwsWVXVix/EC3jmbNg0WktQHQ8ws5oLBQpL6YH4nuGfNYCFJfWBmIUnqUsNZ5TRnDBaS1AdDnOCeCwYLSeoDh6EkSZ2c4JYkdTKzkCR1coJbktTJCW5JUpcq5ywkSV2cs5AkdXIYSpLUycxCktRp7NFR92BaBgtJ6gOHoSRJnRyGkiR1MrOYH0l+B/hmu/sfgTHgp+3+e6tq5UDdPwGeV1XvmN9eStIUDBbzo6ruBRYDJDkb+HlV/W2Sk4ETgJUD1U8A/nzeOylJU6ieT3AvGHUH5sFXgKOTPBUgyV7As4HvjLBPkvSbanzm2whs9cGizTi+DxzVFp0AXFpVNbpeSdIE4+Mz30Zgqw8WrUtoggTtvy+ZrFKSpUnWJFnzqc9MWkWS5kbPM4utZs6iw9eBDyU5EFhYVTdMVqmqlgHLAB69Z72Zh6T54wT36FXVz5OsApYzRVYhSSPl9yx64xLgH9gyHCVJ/bHJHz+ad1V19iRlXwMy/72RpBkws5AkdXLOQpLUycxCktTJzEKS1MnMQpLUydVQkqROPX8CkcFCkvrAOQtJUieDhSSpkxPckqROY2Oj7sG0niyPKJekfhvi71kkOSrJ7UnWJTl9kuN7JvlWkhuTrE3yR11tGiwkqQ+GFCySbANcALwK2Ac4Mck+E6qdCXypqg6gebjqhV3dM1hIUh8M78ePDgHWVdX6qnoE+CJw7MR3A57Rvt4JuKurUecsJKkHanzm37NIshRYOlC0rP3xNoDdgA0DxzYCL5jQxNnAFUneDewAHNn1ngYLSeqDWSydHfxVz0lM9lMMEyPRicDFVXVekkOBzybZr2rqtMVgIUl9MLzVUBuBPQb2d+e3h5neBhwFUFXfS7I9sAtw91SNOmchSX0wvNVQq4FFSZ6bZDuaCewVE+r8GHgFQJL/BGwP/HS6Rs0sJKkPhvQN7qralORdwEpgG2B5Vd2a5BxgTVWtAP4M+GSS99AMUZ1UNf3DqQwWktQHQ3yQYFVdDlw+oeysgde3AYfNpk2DhST1gc+GkiR1msXS2VEwWExh4bMPH3UXeuPhu64ZdRd649wl7xt1F3rhJ5/80ai70Cu//8dDaKTnz4YyWEgzZKDQXCqHoSRJnRyGkiR18vcsJEmdzCwkSZ02OcEtSeriMJQkqZPDUJKkLi6dlSR1M7OQJHUyWEiSOvm4D0lSl9n8BvcoGCwkqQ8MFpKkTq6GkiR1MrOQJHUyWEiSutSYw1CSpC5mFpKkLi6dlSR1M1hIkjr1e8rCYCFJfVCb+h0tDBaS1Af9jhUGC0nqg75PcC8YdQcmSjKW5KYktyS5LMnOszz/7CSnta/PSXLk3PRUkoZofBbbCPQuWAAPV9XiqtoPuA9452NtqKrOqqqrhtc1SZobNV4z3kahj8Fi0PeA3QCS7Jjkm0luSHJzkmM3V0pyRpLbk1wF7D1QfnGS49rXr0hyY3vu8iRPne+LkaQpmVk8Nkm2AV4BrGiLfgm8tqoOBI4AzkvjIOAE4ADgdcDBk7S1PXAxcHxVPZ9mrubtk9RbmmRNkjXj47+Yg6uSpMnVpplvo9DHYLEwyU3AvcAzgSvb8gB/nWQtcBVNxvG7wOHA16rqoap6kC3BZdDewI+q6o52/9PASyZWqqplVbWkqpYsWLDDUC9KkqZT4zPfRqGPweLhqloMPAfYji1zFm8AdgUOao//O7B9e6xrEC9z0VFJGhqHoR6bqnoAOBU4Lcm2wE7A3VX1aJIjaIIJwNXAa5MsTPJ04DWTNPdDYK8kf9Duvwn49txegSTNXN8zi15/z6KqbkzyzzRzEp8HLkuyBriJJgBQVTckubQt+xfgmkna+WWStwJfTvIUYDXwiXm6DEnqNKogMFO9CxZVteOE/cFM4dApznk/8P5Jyk8aeP1NmklwSeqdGuv3aHnvgoUkPRn1PbPo7ZyFJD2Z1HhmvHVJclT73bN1SU6fos5/TXJbkluTfKGrTTMLSeqBYWUW7XfULgD+M7ARWJ1kRVXdNlBnEfCXwGFV9bMk/6GrXTMLSeqBqsx463AIsK6q1lfVI8AXgWMn1PmfwAVV9bPmvevurkYNFpLUA7NZOjv4tIl2WzrQ1G7AhoH9jW3ZoOcBz0vy3STXJTmqq38OQ0lSD4zPYjVUVS0Dlk1xeLKGJn5x+SnAIuBlwO7ANUn2q6r7p3pPg4Uk9cBMJq5naCOwx8D+7sBdk9S5rqoeBX6U5Haa4LF6qkYdhpKkHhjiaqjVwKIkz02yHc2Xmic+M+/rNA9kJckuNMNS66dr1MxCknqghvQzFVW1Kcm7gJXANsDyqro1yTnAmqpa0R57ZZLbgDHgz6vq3unaNVhIUg8McRiKqrocuHxC2VkDrwv403abEYOFJPXADJbEjpTBQpJ6YMxnQ0mSuphZSJI6DXPOYi4YLCSpB4a1GmquGCwkqQfMLCRJncbG+/0daYOFJPWAw1CSpE7jroaSJHVx6awkqZPDUHrCO3fJ+0bdhd5435pzR92FXnj1Ae8YdRd65YohtOEwlLSVMFBoLrkaSpLUqeejUAYLSeoDh6EkSZ1cDSVJ6jQ+6g50MFhIUg8UZhaSpA6bHIaSJHUxs5AkdXLOQpLUycxCktTJzEKS1GnMzEKS1KXnv6pqsJCkPhg3s5AkdfFBgpKkTk5wS5I6jcdhKElSh7FRd6CDwUKSesDVUJKkTq6GkiR1cjWUJKmTw1CSpE59Xzq7oKtCkkpy3sD+aUnOntNeSdKTzFhmvo1CZ7AAfgW8Lskuc92Z+ZTErEpSb4zPYhuFmQSLTcAy4D0TDyTZNclXk6xut8Pa8puT7JzGvUne3JZ/NsmRSfZN8v0kNyVZm2RRkr2S/DDJp9uyryR5WnveWW37tyRZljTfXkmyKslHklzbHjukLd8hyfL2nBuTHNuWn5Tky0kuA64Yxg2UpGEYZrBIclSS25OsS3L6NPWOa0ePlnS1OZNgAXAB8IYkO00o/yjw4ao6GHg98Km2/LvAYcC+wHrg8Lb8hcB1wCnAR6tqMbAE2Nge3xtYVlX7Aw8C72jLP15VB1fVfsBC4OiBPuxQVS9q6y5vy84A/rHt1xHAB5Ps0B47FHhLVb184kUmWZpkTZI14+O/mOGtkaTHrzLzbTpJtqH5m/0qYB/gxCT7TFLv6cCpwPUz6d+MgkVVPQh8pm140JHAx5PcBKwAntF24BrgJe12EfD8JLsB91XVz4HvAX+V5L3Ac6rq4ba9DVX13fb154AXt6+PSHJ9kpuBl9MEoc0uaft4dfv+OwOvBE5v+7UK2B7Ys61/ZVXdN8V1LquqJVW1ZMGCHSarIklzYoiZxSHAuqpaX1WPAF8Ejp2k3rnA3wC/nEn/ZppZAHwEeBsw+Fd0AXBoVS1ut92q6v8BV9NkE4fT/LH+KXAcTRChqr4AHAM8DKxMsvlT/sSlxpVke+BC4Liqej7wSZo//kx1DhDg9QP92rOqftAeN2WQ1Dtjs9gGR0HabelAU7sBGwb2N7Zlv5bkAGCPqvrGTPs342DRfhr/Ek3A2OwK4F0DHVjc1t0A7AIsqqr1wHeA02iDRZLfA9ZX1fk0Gcn+bRN7Jjm0fX1ie97mwHBPkh1pgs6g49s2Xww8UFUPACuBdw/MbRww0+uUpFEYz8y3wVGQdls20NRkA1W//lCdZAHwYeDPZtO/2WQWAOfRBIHNTgWWtBPSt9HMRWx2PXBH+/oamsj2nXb/eOCWdpjoD2mGuAB+ALwlyVrgmcBFVXU/TTZxM/B1YPWEPv0sybXAJ9gSyM4FtgXWJrml3Zek3hriMNRGYI+B/d2Buwb2nw7sB6xKcifNXPKKrknuzuWjVbXjwOt/B542sH8P7Sf7Sc5708DraxkITFX1AeADg/WTPAMYr6rBgLO5/pnAmVN08atV9ZcT6j8MnDxJOxcDF0/RjiSNzBCXxK4GFiV5LvCvwAnAf9t8sB19+fWH/iSrgNOqas10jc42s5AkzYGaxTZtO1WbaKYHVtKM1nypqm5Nck6SYx5r/3rzxbSqupMmNZrNOS+bk85I0jwb5rOhqupy4PIJZWdNUfdlM2mzN8FCkp7M/PEjSVKn8Z4/pNxgIUk90PenzhosJKkH+p1XGCwkqRfMLCRJnTal37mFwUKSeqDfocJgIUm94DCUJKmTS2clSZ36HSoMFpLUCw5DSZI6jfU8tzBYSFIPmFlIkjqVmYUkqYuZhSSpk0tnJUmd+h0qDBaS1Aubeh4uDBaS1ANOcD9BPfjB14y6C73xk0/+aNRd6IUfv+ztvP2BIf5Q8hPY/77xwlF3YavjBLe0lTBQaC6ZWUiSOplZSJI6jZWZhSSpg9+zkCR1cs5CktTJOQtJUieHoSRJnRyGkiR1cjWUJKmTw1CSpE5OcEuSOjlnIUnq5DCUJKlTOcEtSeoyZmYhSeriMJQkqVPfh6EWjLoDkqQms5jp1iXJUUluT7IuyemTHP/TJLclWZvkm0me09WmwUKSeqBm8c90kmwDXAC8CtgHODHJPhOq3Qgsqar9ga8Af9PVP4OFJPXAWNWMtw6HAOuqan1VPQJ8ETh2sEJVfauqHmp3rwN272rUYCFJPTCbYagkS5OsGdiWDjS1G7BhYH9jWzaVtwH/p6t/TnBLUg/MZjVUVS0Dlk1xOJOdMmnF5I3AEuClXe9psJCkHhjiaqiNwB4D+7sDd02slORI4AzgpVX1q65GezsMleSMJLe2s/U3JXnBNHUvTnLcfPZPkoZpiKuhVgOLkjw3yXbACcCKwQpJDgD+Djimqu6eSf96mVkkORQ4Gjiwqn6VZBdguxF3S5LmzLAeJFhVm5K8C1gJbAMsr6pbk5wDrKmqFcAHgR2BLycB+HFVHTNdu70MFsCzgHs2p0ZVdQ9AkrOA1wALgWuBk2tC7pbkIOBDNDfiHuCkqvpJklOBU4BNwG1VdcJ8XYwkdRmr4T2kvKouBy6fUHbWwOsjZ9tmX4ehrgD2SHJHkguTbJ58+XhVHVxV+9EEjKMHT0qyLfAx4LiqOghYDry/PXw6cEC7rviUebkKSZqhqprxNgq9DBZV9XPgIGAp8FPg0iQnAUckuT7JzcDLgX0nnLo3sB9wZZKbgDPZsn54LfD5dvZ/02TvO7gcbfm1tw37siRpSsP8Bvdc6OswFFU1BqwCVrXB4WRgf5pvHW5Icjaw/YTTAtxaVYdO0uSrgZcAxwDvS7JvVf1G0BhcjvbQR0/p94NaJG1V+v7jR73MLJLsnWTRQNFi4Pb29T1JdgQmW/10O7BrO0FOkm2T7JtkAbBHVX0L+AtgZ5o5DUnqhfGqGW+j0NfMYkfgY0l2phkyWkczJHU/cDNwJ83ysN9QVY+0S2jPT7ITzfV9BLgD+FxbFuDDVXX/fFyIJM1E3zOLXgaLqvon4EWTHDqz3SbWP2ng9U00w00TvXhY/ZOkYRvmaqi50MtgIUlPNqMaXpopg4Uk9YDDUJKkTmYWkqROZhaSpE5jNTbqLkzLYCFJPTCqx3jMlMFCknpgVI/xmCmDhST1gJmFJKmTq6EkSZ1cDSVJ6uTjPiRJnZyzkCR1cs5CktTJzEKS1MnvWUiSOplZSJI6uRpKktTJCW5JUieHoSRJnfwGtySpU98ziwWj7oD0RHHRTv3+P7Oe2MarZryNQvoezZ7skiytqmWj7kcfeC8a3octvBfzx8yi/5aOugM94r1oeB+28F7ME4OFJKmTwUKS1Mlg0X+Ox27hvWh4H7bwXswTJ7glSZ3MLCRJnQwWkqROBoseSPI7SW5qt39L8q8D+/9lQt0/SXLhqPo6DEnG2mu7JcllSXae5flnJzmtfX1OkiPnpqezk6SSnDewf1qSs0fYpSekJGckuTXJ2vZ/Jy+Ypu7FSY6bz/49WRkseqCq7q2qxVW1GPgE8OH29UXACROqnwBcMt99HLKH2+vdD7gPeOdjbaiqzqqqq4bXtcflV8Drkuwy6o4MU5J5eyxQkkOBo4EDq2p/4Ehgw3y9v6ZmsOi3rwBHJ3kqQJK9gGcD3xlhn4bte8BuAEl2TPLNJDckuTnJsZsrtZ82b09yFbD3QPmvP1kmeUWSG9tzl2++b/NoE83qnPdMPJBk1yRfTbK63Q5ry29OsnMa9yZ5c1v+2SRHJtk3yffbT9hrkyxKsleSHyb5dFv2lSRPa887q23/liTLkqQtX5XkI0mubY8d0pbv0N6r1e29O7YtPynJl5NcBlwxL3ev8Szgnqr6FUBV3VNVd011XYOSHJTk20n+KcnKJM9qy09Nclt7r744j9eyVTFY9FhV3Qt8HziqLToBuLS2kiVsSbYBXgGsaIt+Cby2qg4EjgDOa/+IHkRz7QcArwMOnqSt7YGLgeOr6vk0D8l8+5xfxG+7AHhDkp0mlH+UJmM8GHg98Km2/LvAYcC+wHrg8Lb8hcB1wCnAR9tMcwmwsT2+N7Cs/fT9IPCOtvzjVXVwm7UtpPmUvtkOVfWitu7ytuwM4B/bfh0BfDDJDu2xQ4G3VNXLH9uteEyuAPZIckeSC5O8tC2f7rpIsi3wMeC4qjqI5vre3x4+HTigvVenzMtVbIUMFv13CVuGoraGISiAhUluAu4Fnglc2ZYH+Oska4GraDKO36X5A/q1qnqoqh5kS3AZtDfwo6q6o93/NPCSObyGSbX9+wxw6oRDRwIfb697BfCMJE8HrqHp50tohh2fn2Q34L6q+jlN5vVXSd4LPKeqHm7b21BV321ffw54cfv6iCTXJ7kZeDlNENrskraPV7fvvzPwSuD0tl+rgO2BPdv6V1bVfY/vjsxOe80H0TzG46fApUlOYvrrgua//37Ale21nAns3h5bC3w+yRtpsj89Bj6ivP++DnwoyYHAwqq6YdQdGoKHq2px++n7GzRzFucDbwB2BQ6qqkeT3EnzxwvofNj/bw1LjNBHgBuAvx8oWwAcOvDHHoAkV9Nc/540n/JfCxxHE0Soqi8kuR54NbAyyf+gyUAm3o9qs6sLgSVVtaGdXN9+sM7Ec2ju2+ur6vYJ/XoB8IvZXPSwVNUYTeBa1QaHk4H9mfq6oLmOW6vq0EmafDVNMD4GeF+SfavKoDFLZhY9137SWkWTVm8NWcWvVdUDNJ/AT2uHEXYC7m4DxRHAc9qqVwOvTbKw/TT+mkma+yGwV5I/aPffBHx7bq9gcu2n8S8BbxsovgJ41+adJIvbuhuAXYBFVbWeZj7qNNpgkeT3gPVVdT5NRrJ/28SeaSaDAU5sz9v8B/SeJDvSBJ1Bx7dtvhh4oL3/K4F3D8xtHPD4rv7xSbJ3kkUDRYuBzYFsquuirbPr5nuSZNt2vmcBsEdVfQv4C2BnYMe5u4Ktl5nFE8MlwD/w2yujnvCq6sYk/0xzbZ8HLkuyBriJJgBQVTckubQt+xfaP6QT2vllkrcCX06zemc1zcqyUTmPgeBAExQvaIfYnkITADePn18PbNO+vgb4AFsWMRwPvDHJo8C/AecAzwB+ALwlyd8B/xe4qKoeSvJJ4GbgTpp7MOhnSa5tz//vbdm5NJnQ2jZg3MmE+YB5tiPwsXaIbBOwjmZI6n6mvi6q6pE0Cx3ObzPWp9Bc1x3A59qy0Mwb3T8fF7K18XEf0hNMmlVx32gne2d6zirgtKpaM0fd0lbOYShJUiczC0lSJzMLSVIng4UkqZPBQpLUyWAhSepksJAkdfr/IV2KwMH0HFQAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 2 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lN1ObhkQ-kgo",
        "colab_type": "text"
      },
      "source": [
        "What are the **features**?\n",
        "- TV: Advertising dollars spent on TV for a single product in a given market (in thousands of dollars)\n",
        "- Radio: Advertising dollars spent on Radio\n",
        "- Newspaper: Advertising dollars spent on Newspaper\n",
        "\n",
        "What is the **response**?\n",
        "- Sales: sales of a single product in a given market (in thousands of widgets)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JzvJ7SVv-kgq",
        "colab_type": "code",
        "outputId": "d4995dec-8dc0-4d19-bac9-d18b7e69acc8",
        "colab": {}
      },
      "source": [
        "# print the shape of the DataFrame\n",
        "data.shape"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(200, 4)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HWTm9qLs-kgy",
        "colab_type": "text"
      },
      "source": [
        "There are 200 **observations**, and thus 200 markets in the dataset."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cX9N_Z1r-kg0",
        "colab_type": "code",
        "outputId": "a53f8ed2-803d-4b2a-fcf6-8e130688c2bf",
        "colab": {}
      },
      "source": [
        "# visualize the relationship between the features and the response using scatterplots\n",
        "fig, axs = plt.subplots(1, 3, sharey=True)\n",
        "data.plot(kind='scatter', x='TV', y='Sales', ax=axs[0], figsize=(16, 8))\n",
        "data.plot(kind='scatter', x='Radio', y='Sales', ax=axs[1])\n",
        "data.plot(kind='scatter', x='Newspaper', y='Sales', ax=axs[2])"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.axes._subplots.AxesSubplot at 0x29a05148940>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA7AAAAHjCAYAAAAaOPOyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3X+QHOd93/nPMzO7gxUWBJAFSRM/SDpZ0QqgELCEC6VAp4hk+colknBypJkKKTF1iaWkitTpzjaB3NkIzbBSZZKyUyeTtkPLvpg2nTMF5Az+8NVdygQjgScxByoLhIAVEedYBEAVKe4BIBZazO7M9P2x6MXMbM9Md0//eJ7u96tKJWL2x/T2PN/v08/zfPtp43meAAAAAACwXSXvAwAAAAAAIAwGsAAAAAAAJzCABQAAAAA4gQEsAAAAAMAJDGABAAAAAE5gAAsAAAAAcAIDWAAAAACAExjAAgAAAACcwAAWAAAAAOCEWt4HEMaGDRu8G2+8Me/DAGCZN954433P867O+ziSQq4DEIRcB6AMwuY6JwawN954o44cOZL3YQCwjDHm+3kfQ5LIdQCCkOsAlEHYXEcJMQAAAADACQxgAQAAAABOYAALAAAAAHACA1gAAAAAgBMYwAIAAAAAnMAAFgAAAADgBAawAAAAAAAnMIAFAAAAADiBASwAAAAAwAkMYAEAAAAATmAACwAAAABwAgNYAAAAAIATGMACAAAAAJzAABYAAAAA4AQGsAAAAAAAJzCABQAAAAA4gQEsAAAAAMAJDGABJGJ2rqGjp85pdq6R96EAgDXIjbAVbROuquV9AADcd3DmjPYeOKaxSkWL7baeuPtm7d6xKe/DAoBckRthK9omXMYKLICRzM41tPfAMV1abOtCo6lLi23tOXCMGV0ApUZuhK1om3AdA1gAIzl9dl5jle5UMlap6PTZ+ZyOCADyR26ErWibcB0DWAAj2bx+Qovtdtdri+22Nq+fyOmIACB/5EbYirYJ1zGABTCSqcm6nrj7Zq0aq2hNvaZVYxU9cffNmpqs531oAJAbciNsRduE69jECcDIdu/YpF3TG3T67Lw2r5+gEwQAkRthL9omXMYAFkAipibrdIAA0IPcCFvRNuEqSogBAAAAAE5gAAsAAAAAcAIDWAAAAACAExjAAgAAAACcwAAWQCZm5xo6euqcZucaeR8KgBIjFwEgD7iNXYgBpO7gzBntPXBMY5WKFtttPXH3zdq9Y1PehwWgZMhFAMgD7mMFFkCqZuca2nvgmC4ttnWh0dSlxbb2HDjGrCeATJGLAJAHioEBLIBUnT47r7FKd6oZq1R0+ux8TkcEoIzIRQDIA8XAABZAqjavn9Biu9312mK7rc3rJ3I6IgBlRC4CQB4oBgawAFI1NVnXE3ffrFVjFa2p17RqrKIn7r5ZU5P1vA8NQImQiwCQB4qBTZwApG73jk3aNb1Bp8/Oa/P6CToKALkgFwEgD7iPASyATExN1ukkAOSOXASAPOA2SogBAAAAAE5gAAsAAAAAcAIDWAAAAACAExjAAkjN7FxDR0+d4wHhAHJBDgLKgVgvFzZxApCKgzNntPfAMY1VKlpst/XE3Tdr945NeR8WgJIgBwHlQKyXDyuwQIElOSMZ5XfNzjW098AxXVps60KjqUuLbe05cEyzcw1mSQGkLigH/eL+Yzr57oXA7007J5H30sc5TsYo5zGPz2DQ9QaKixVYoKCSnJGM+rtOn53XWKWiS2ovvzZWqei519/Wb756kllSAKkKykELzbY++9Vv6is/u30572SxcsPqUPo4x8kY5Tzm9Rn0u944fXaex+QUGCuwQAElOSMZ53dtXj+hxXa767WFVltPH3qLWVIAqQvKQZK00PK6qkHSXrlhdSh9nONkjHIe8/wMgmJ9sd3W5vUTqb838sMAFiggf0aykz8jmcXvmpqs64m7b9aqsYrW1GtaNVbRQ7dOa7xaTeSYAGAQPweN11Ze5vh5J8k82U8W71F2nONkjHIe8/wMgq43nrj7ZlZfC44SYqCAkpyRjPu7du/YpF3TG3T67Pzy9z796slEjgkAhtm9Y5O2XneVPvvVb2qh5S2/3pl30l65YXUofZzjZIxyHvP+DHqvNxi8Fl9qK7DGmC3GmEPGmD83xhw3xnz58uu/Yow5Y4yZufy/z6Z1DEBZJTkjOcrvmpqsa/uWdZqarDNLCiBz09eu0Vd+dntg3skiJ5H30sc5TsaofX3en0Hn9QaKz3ieN/y74vxiY66TdJ3ned8xxqyR9IakvyPpXklznud9Jezv2rlzp3fkyJFUjhMostm5RmIzkkn9riSPyRjzhud5O0f6JRYh1wHpGJR3ksxJcd4/DHLdcFl8jmUwynnkM8Cowua61EqIPc/7gaQfXP7vC8aYP5fElnBAhvxVBpt+V5LHBABhDMo7WeQk8l76OMfJGOU88hkgK5ls4mSMuVHST0p6/fJLDxljjhljfs8Ysz6LYwAAAAAAuC31AawxZlLSAUn/g+d5H0j6LUl/TdIOLa3Q/lqfn/uiMeYtY8wP33777bQPEwByQa4DUAbkOgBJSXUAa4wZ09Lg9TnP8/6tJHme967neS3P89qSfkfS3wz6Wc/znvE878Oe5119/fXXp3mYgFVm5xo6euocz7ArCXIdUHzk9eRzHecUKK/U7oE1xhhJvyvpzz3P+/WO16+7fH+sJP1dSW+mdQyAaw7OnNHeA8c0Vqlosd3WE3ffrN07uHUcAFxFXk8e5xQotzRXYHdJ+ryk23oemfOEMeY/GWOOSbpV0v+Y4jEAzpida2jvgWO6tNjWhUZTlxbb2nPgGLPLAOAo8nryOKcA0tyF+LAkE/ClP03rPQGXnT47r7FKRZd05WHgY5WKTp+dZ1c/AHAQeT15nFMAmexCDGC4zesntNhud7222G5r8/qJnI4IADAK8nryOKcAGMAClpiarOuJu2/WqrGK1tRrWjVW0RN33+zkjDKbawBIi0v5pUh53RacU/e5FMOwU2olxACi271jk3ZNb9Dps/PavH7CyQ6ZzTUApMXF/FKEvG4bzqm7XIxh2IcBLGCZqcm6s51x5+Ya/v1Jew4c067pDc7+TQDs4HJ+cTmv24pz6h6XYxh2oYQYwApxy3v8zTU6+ZtrAMAoRs0vlC0C+Zmda+jQd99T1XTv78o1AuJgBRZAl1HKe9hcA0BaRskvlC0C+fHjr1YxurjQ6voa1wiIgxVYAMtGfb4em2sASEvc/MJzQ4H8dMbfXOPK4HX1eJVrBMTGCiyAZUk8X4/NNQCkJU5+4bmhQH6C4m91vapH79qmWz9yDTGIWBjAAliWVAkwm2sASEvU/MKtDUB+guKv1fYYvGIklBADWEYJMICiIa8B+SH+kAZWYIGMzc41rC6vzasE2PbzAiAZecS6Tbc2kOuQFVvamk3xV2a2tIckMIAFMuTKTphZlwC7cl4AjCbPWLfh1gZyHbJiW1uzIf7KzLb2MCpKiIEEDXrOIDthBuO8AOVQ9lifnWtoz/7y/v3IjuuxxjObk+V6ewjCCiyQkGGzW+yEGYzzApRD2WP9udffVqPZvZlNmf5+ZMflWCvaSqENXG4P/bACCyQgzOwWO2EG47wA5VDmWJ+da+jpQ2+teH2hVY6/H9lyNdaKuFJoA1fbwyAMYIEE+LNbnfzZLR878QXjvADlUOZYP312XuPV6orXH7p1uhR/P7LlaqyFuZZCdK62h0EoIQYSEHZ2i534gnFegHIoa6wH9RH1mtF9t1yf0xGh6FyMtSKuFNrCxfYwCCuwQAKizG5NTda1fcs655NH0jgvQDmUMdaD+ogn79leqnOA7LkWa0VcKbSJa+1hEFZggYQUbXYLAJAc+ghgOOIEYTCABRJkw3POivSgagCIw9Y8aEMfgWTY2saKgDjBMAxggQJh+3kAZUceRNpoY0C+uAcWKAi2nwdQduRBpI02BuSPASxQEGw/D6DsyINIG20MyB8DWKAg2H4eQNmRB5E22hiQPwawQEGw/TyAsiMPIm20MSB/bOIEFAjbzwMoO/Ig0kYbA/LFABYoGLafB1B25EGkjTYG5IcSYgAAAACAExjAAgAAAACcwAAWSNnsXENHT50b+Iy4MN8DAAhv1LxKXkZYrrQVV44TGIZ7YIEUHZw5o70HjmmsUtFiu60n7r5Zu3dsivw9AIDwRs2r5GWE5UpbceU4gTBYgQVSMjvX0N4Dx3Rpsa0LjaYuLba158CxrpnPMN8DAAhv1LxKXkZYrrQVV44TCIsBLDCifiU5p8/Oa6zSHWJjlYpOn52P9D0AgPBGzauj/DwlmuXiSh+e13ESD0gLJcTACAaV5GxeP6HFdrvr+xfbbW1eP7H87zDfAwAIL2penZ1rdD3PM25epkSzfJLqw3vbYNLyuNYgHpAmVmCBmIaV5ExN1vXE3Tdr1VhFa+o1rRqr6Im7b+7qnKYm67r345u7fu+9OzfzbDkAiClM7vUdnDmjXY+/os997XXtevwVvTBzJtLP+yjRLKc4baVXUBu08TijIB6QNlZggZj8kpxLujKr6Zfk+J3C7h2btGt6Q9+Z1dm5hp5/43TXa88fOa0v335TqoPYtGd7AWCQtHPQsNzrH4N/ke3n8T0HjmnX9IZQP98pTH+AYoraVjoNaoNJt5tRjjMqV+KBayF3MYAFYgpbkjM1We+bGOMk+VETLmU9API0KAcleUE5KPdKw/PvsJ/vxO0g5RalrXTKeqAX9zijsi0egvIK10Juo4QYiCmJkpyoSX7UUiPKegDkaVAOyqKUslOSF9lZl2iiGGwb6CXFpngIyitcC7mPFVhgBKOW5PhJfk/PLGDUcrew7+tKWQ+AYuqXg46/cz6zUkpflPwbRpYlmiiGpNugTWyIh37XTc98/uNcCzmOASwwolFLcsIm+SQGn0Wd7QXghn45SDK5XFAmfZGdVYkmisOGgV5a8o6HftdNkuFayHGUEAMWmJqsa/uWdQMTfRKDT5vKegCUT78ctG3jVbldUIbJv0CaaIPp6HfdtG3jVVwLOY4VWMARUcuN+83mFnm2F4D9+uUgP79VjdFiq619d2wlP8F57HSbn0HXTVwLuY0BLJCjqB1bmIQbZme9vMt6AJRbUA7avWOTLlxq6tGXTmi8VtFjL5/QmlW1xHcGZUCBrKS50y3tOJxB101cC7mLASyQk7gd26CEm+Uz5QAgSbNzDT328gktNNtaaC69lnT+4tEZyEqa/THtOBoGqsXDPbBADtLawt3fsKCTvxEKANgs7fzFozOQpbTaM+0YYAAL5CKtjo1dhgG4Ku38xQQfspRWe6YdAwxggVzE6dhm5xo6eurcwFlWdhkG4LIHPzOtei2d/MUEH7KURn88O9fQ+flFLbRoxyg37oEFchD14eVR7ndxaWc9NqEA0uFabHXmOMnTFz/9V3XfLdcneuxR8y4wqiT7484YabXbGqsarapVE2vHruUMlBsDWCAnYTu2OBtBuLBhAZtQAOlwLbaCctzTr57Ufbdcn/h7uTTBh2JIoj8OipF6TXr6/p/Uto1r2RQKpUMJMZCjMA8vL+L9LmxCAaTDxdjKOseFybuATYJiZLxa1dqJ8UQHx67kDIABLNBHmHtOs1DE+7aKOCgHbOBibLmU42zpF1AuacaITTmD+EJYlBADAWwqpynifVsuXbACLnExtlzJcTb1CyiXNGPElpxBfCEKBrBAjzQfPh5X0e7bcuWCFXCNq7Fle46zsV9AuaQVIzbkDOILUTGABXr45TR+EpWulNPkmUhd2JgpCtsvWAFXuRpbNuc4W/sFlEtaMZJ3ziC+EBUDWKCHLeU0ZWDzBSvgMmIrWfQLKLo8cwbxhajYxAnokcbDxwEA7qJfANJDfCEqVmCBAHmX07iCB58D9iNOk0G/AKSnM75Wj1d1caGl2bkGcYZADGCBPijBG4wdAwH7EafJol8A0jM1Wdfhk++TszAUJcTAEC49lyyrY03qwecunVvAdr3xlFScxn1/V7h63GV08t0L2n/klE6+eyHw63yWbss6Z8FdrMACA7i0epHlsSaxY6BL5xawXVA83TC1OrOdPV2NZ1ePu4z+2Z/8Jz377beX//3AJ6/XP/+Zv7H8bz5L97EbMcJiBRbow6WZwKyPddQdA106t4Dt+sXT6vFqJjt7uhrPrh53GZ1890LX4FWSnv3W28srsXyWxcBuxAiLASzQhz8T2MmfCbRN1sc66o6BLp1bwHb94uniQiuTnT1djWdXj7uMZk6dG/g6n2UxsBsxwqKEGOjDpZnAPI51lB05XTq3gO0GxdP2LetS3znX1Xh29bjLaMeWdQNf57MsDnb7RhiswAJ9uDQTmNexTk3WtX3Lusjv49K5BWw3LJ7ixmlS728rV4+7jKavXaMHPnl912sPfPJ6TV+7RhKfZdGknbPgPuN5Xt7HMNTOnTu9I0eO5H0YKCmXnqHo0rFKox+vMeYNz/N2pnBouSDXYRR5x3/e7x+XC8dNrlty8t0Lmjl1Tju2rFsevHZy4bME0F/YXEcJMTCES8/9c+lYJfeOF7BZ3vGU9/vH5epxl9H0tWsCB64+PkugHCghBkokyjPyeJ4eUAz9YpkYB8IJEyvEE5AdVmABy6RVAhXlGXk8Tw8ohn6xPGqMU6qJsggTK7b2mcQpiiq1AawxZoukZyX9mKS2pGc8z/tfjDF/RdIfS7pR0l9KutfzvLNpHQfgkrQ6wc5n5PkPCN9z4Jh2TW9Y0alF+V4A9uoXy1uvu2qkGLf1Yh1IWpj+0NY+kzhFkaVZQtyU9Aue5/11SZ+Q9KAxZqukfyrpzzzP+7CkP7v8b6D00nwQe5Rn5PE8PaAY+sXyzKlzsWM8zTwF2CZMf2hjn0mcouhSG8B6nvcDz/O+c/m/L0j6c0mbJP2MpN+//G2/L+nvpHUMgEvS7ASjPCOP5+kBxdAvlndsWRc7xm28WAfSEqY/tLHPJE5RdJls4mSMuVHST0p6XdK1nuf9QFoa5Eq6JotjAGyXZicY5Rl5PE8PKIZ+sTx97ZrYMW7jxTqQljD9oY19JnGKokv9ObDGmElJ/17Sv/A8798aY855nreu4+tnPc9bH/BzX5T0sKR1V1999Yb33nsv1eMEbPDCzBntSfGelaANHfpt8uDC5g9FeDYiuQ5xhY3RpGM87TyFlch14aTVb4X5vbb1mcQpXBQ216U6gDXGjEl6SdL/6Xner19+7T9L+ozneT8wxlwn6VXP835i0O+J+8BrwEXDOsEkO0nXN3kowkVdJ3IdwsoydqNMfCEd5LrhXO/P0kCcRsP5yl/YXJfmLsRG0u9K+nN/8HrZC5L+gaRfvfz/B9M6BsBFgx7EnmQHbevOiQAGyzJ2++WcQXkKyBr9WTDiNDwmQNyS5j2wuyR9XtJtxpiZy//7rJYGrj9ljHlL0k9d/jeQKpceMN7vWJPeVZBNHgA3ZRW7w3KOC3nVhWNEfP7ne/ydD+jPElam2GHXZvektgLred5hSabPl29P632BXi7Nqg06Vv+i1Z9dlq500HFmWNnkAXBTVrE7KOccPvm+9XnVpdyP6Do/34VWWy36s8SULXaSvr5C+jLZhRjIi0uzasOONemLVht3TgQwXFax2y/nrB6vWp9XXcr9iK7382002zLGqF4z9GcjKmPsMKHvntRWYAEbuDSrdvyd86qY7qKFzmP1L1p7dxUc5e/YvWOTdk1vYNMCwDFZxG6/nHNxoWV9XnUp9yO6oM93Va2qp+//Sa2dGKc/G0GesZPXJkppXF8hXQxgUWiuzKodnDmjPfuPqtHs3hW891jTuGhlkwfATVnEblDOmZ1r6FKz1fV9l5otq/KqK7kf8fT7fLdtXEt/NqK8YifvsmUm9N1CCTEKzYUyWb9cp3fwWq8FH+vUZF3bt6yz6m8AUFxBOaf3EXxpP1M+KhdyP+Lj801PHufWlrJlrq/cwQosrJFW6Uias2pJHHNQuc6Hxqr67c9/XJ++6eqkDhUAEnH67Lwmxmq60GguvzYxVgssMczzuYpp5X6eFWkHVszSk/W5tbHknzi3GwNYWCHt0pE0Su2SOuagcp22PG3beFVShwoAiQlbYph3SaCUfO634W/CFdwCk54sz+3m9ROaX2x2vTa/2Myt5J84tx8lxMidLaUjUSR5zJRCAXBJmJzlYl4fpoh/E2AL07OJZe+/s0Kcu4EVWOQuj9KRUUtDwh5z2PehFAqAC/yctmt6g17be1vfnJVmXs+rtM/GMkdgkCRiJYt4O312XqtqVS22rqzCrqpVc4kt4twNDGCRu6x3vEuiNCTMMUd9H0qhANgsSk5LK6/nWdrHzsZwSRKxklW82RRbNh0L+qOEGLnLsoQ2qdKQYcecRwnK7FxDR0+dG/geYb4HgDv8mD757oVUYztqTksjr+dd2sftHsWTRZ+YR7+bRKxkGW82xZZNx4L+WIGFFbIqoU2yNGTQMWddghJmlpRNCYBi8WPaa3tqtDytGluak04jtuPktKTzug2lfdzuURxZ9Il59btJxErW8WZTbNl0LAjGABbWyKKENunSkH7HnGUJSucsqd/R7DlwTLumNwSuCPf7HgDu6Ixpn//facR23JyWZF63pbSP2z3cl0WfmGe/m0Ss5BFvNsWWTceClSghRqlkVRqSZQmKP0vayZ8ljfI9ANwRFNO+NGLbhrI6G44BxZBFn5hnv5tErBBvsBkrsCidrEpDsnqfMLOktqxcAEhGUEz70optG8rqbDgGuC+LPjHvfjeJWCHeYCtWYFFKU5N1bd+yLvVknMX7hJklZSYVKJbOmK5Xl56XuGqsknpsZ5U7bT8GuC2LPtGGfjeJWCHeYCNWYOGkvJ4DaKsws6TMpALF0hnTq8erurjQIrZF/4BwsugT6XfDI24RBQNYOMf13XTTStJhNhxgUwKgWKLGdNEvEl3vH5Cu3vafRZ9IvzsccYuoGMDCKa7vpkuSBpCXoucf1/sHpKvo7d9VxC3i4B5YOMX23XQHPbA8y4eCA0CnQflnUN5yie39A/JT9v7X5hgnbhEHK7BwSt67+g0ybHY364eCA4CvX/557vW39ZuvnizEqpTN/QPyVeb+1/aVZ+IWcbACC6fYsKtfkDCzuyRpAHkJyj8LrZaePnSyMKtStvYPyF9Z+18XVp6JW8TBCiycY+OufmFmd/0kvadnJtSG4wdQbEH558HPTOuZb/yFGs3irErZ2D8gf2Xtf11ZeSZuERUDWGQuiV0ww+zql+Vum2Fnd4cl6aLvEAogPcPyR2/+kaSnXz3Z9T1ZrkrlsSM7ObZcOj9vVwZJSbZRG1aew/497NaMKBjAIlNZ3YuR9T0fUWZ3+yVp2+9TAWCvsPmjN//ktSqVR74jx5ZLv8/b5kFS0m0075VnYg5pMZ7n5X0MQ+3cudM7cuRI3oeBEc3ONbTr8Vd0afHKbOCqsYpe23tbosk0q/fp995xZk7zPGaXGWPe8DxvZ97HkRRyHeIYNX9kvSqZR75zPceS66Jx8fNO85jzqDxw8TNA/sLmOjZxQmay2io9zy3Zpybr2r5lXeTkzDbyAOIaNX/EzVtx5ZHvyLHl4uLnneYxZx3jkpufAdzBABaZyepeDBvu+YjKxWMGYAfX8kcex+vaOcJoXPy8XTzmQYr298AuDGCRmay2SndxS3YXjxmAHVzLH3kcr2vnCKNx8fN28ZgHKdrfA7twDywyl9W9GC7uNuniMeeJ+8KAK1zLH3ndl+fSOfKR6+Jx8fN28ZgHKdrfg3SFzXXsQozMZbVVuotbsmd9zHQsQPLyiivXcl4ex+vaOcJoXPy8/WOenWvo6KlzzvfPLn4GsB8DWKCk2N4eSB5xBWBU5BFgMO6BBUpodq6hvQeO6dJiWxcaTV1abGvPgWOanWvkfWiAs4grAKMijwDDMYAFcuaXCWXZOUXd3j6PYwRsFhQTozw2ghgDgg2LjaLFDo+fAYajhBhOseWezaSOI68yoSjb21PKBHTrFxNxHxuRZozZkDNtOAa4aVhsBH191/QGpzeK5PEzwHAMYOEMWwZSSR1HZ5nQJS11VnsOHNOu6Q2pX+T529vv6fk7et83z2MEbDQsJsLEVZTfNwobcqYNxwA3DYuNoK///PMzqlYqGq+m297SbNdx8ghQNgxgkauwM5i2DKSSPA6/TMj/PdKVMqEs/qbdOzYNnanO+xgB2wyLiTBxFeX3xWVDzsz7GFj5dduw2Aj6erMtNdttNZrptbcs2nXUPJIm4gg2YgCL3ESZwUx7IBU2QSd5HDaUCQ3b3t6GYwRsEiYmojw2YvV4VY1ma+DviyOLyadheTPPCTBWft03LNaCvt4rjfaWVbtO8vEzcQehWcQRA2TEwSZOyEXUXfbSHEgdnDmjXY+/os997XXtevwVvTBzpu/3JnkcU5N13fvxzV2v3btzs1UJ3C9lWjVW0Zp6TavGKpQyodSSjImDM2d051OHVakYSVK9ahKLsbQnn8LkzbwmwNjFtRiGxVrv1+s1o7Gq6fodabQ31yZ2o1zjdMoijuIeG8AKLHIRdQYzrXtCopYCJXkcs3MN/fGR012vPX/ktL58+01WDRBtKmUCbJBETHTmHp9njJ77h39TY7WqZucaI8VamvfRhc2bed3Lx60PxTEs1nq//trJ90dqb2FWA126R3WUcucsKt/yvs0B7mIAi1zEmcGMetEYpiOKk6CTGtA99/rby/fphH3vvCRZygQUwagxEZR7Kka672uvq16rJlKuN0quGpQ/o+TNPCbAXFshw2DDYq3z66O0tyjlsq5M7I4yCE0ijpLKI0AvBrDIRdwZzLAXjWE7orgJetSL19m5hp4+9NaK1xdaXGQBZRCUe/zV2IVWU1IyqxFxctWw/Bk1b2Y9AebSChmSF6e9xVkNdGFid5RB6KhxlHQeAToxgEVu0prBjNIR5VniNl6tqtFsdr3+0K3T1neIAEbXm3sarbaM56nR8pa/J4/ViDD504UBoisrZLBDUVcDR43VuHFUlDwCezGARSRJ7xaXxgxm1I7IlhK3es3ovluuT/29AdihM/esHq/qzqcOSx0D2DxWI8LmTxcGiHmskLGjqpuKvBo4aqxGiSO//Z+fXyi6X+ACAAAgAElEQVRMHoGdGMAiNFceSxCnI6LEDUAeOnOPDTkhSv50oYQyS670kVip6H1yFrHa2f4XWi21ve6vk0eQJAawCMWl3eLy6oh6Z96HzcQz8wiUQ1AuCHrNhpxQ9Av5tLjURyJYb0XExYXWyLuBJ8nm1f2g9l+rSPVaReNV8giSxwAWobh2f0jWF4K9M+/37tys54+cHjoTz8wjUGxBq3Ke1HelzoacYMNA2jWu9ZEINjVZ1+GT71u3km776n5Q+58Yq+np+z+mtRNj5BEkjgEsQnHx/pCsLgSDZh6f/dbbksRMPFBiQbnh4f1HJRk1mnav1NkwkHaJi30kVrJxJd3GY+rVr/1v23iVNceIYqnkfQBwg19WtmqsojX1mlaNVSgHucyfeRzEn4kHUB5BuaFqKqpWTNdr5Af30UcWQ1DM5h2fNh5TL9o/ssYKLEKjrCxY0MxjL2bigfIJyg0try153QNY8kMx0Ee6z8aVdBuPKQjtH1liBRaRTE3WtX3LOhJTh6CZxwc+eT0zkUDJBeWGJ+/ZrifvYaWiqOgj3WbjSqKNx9QP7R9ZYQUWSEDQzOOXb7+JmUig5PqtSrBSAdjJxpVEG48JyBMDWCAhvZuesAkKACk4F5AfAHvZGJ82HhOQF0qIASRudq6ho6fOaXaukfehoARobwBwBTkRRccKLCKz+WHayJ/tz6tDsdDeQJ8EXFH0nEi8Q2IAi4iKnhgxGheeV4fioL2BPgm4oug5kXiHjxJihNaZGC80mrq02NaeA8eGlqhQytIt7fOR5/nu97y64+98QBtA4kZ5PmKScUKOi2+Ucxe3T0JxjNp+sojbLPODC8+Mjcu2eCfv54sVWITmJ0Z/Vk+6khj7zewxW9Yt7fOR9/kOel7d/GJTX3j2iMartAEka/P6Cc0vNrtem19sDn0+YpJxknfMuWzUcxenT0JxjNJ+sorbrPODK8+MjcOmeCfv548VWIQWNTHaNluWt7TPhw3nu/d5dfWakTFGjSZtAOkwxgz8d68k48SGmHNVEueuyBfrGGyU9pNV3OaRH1x6ZmxUtsQ7ed8ODGARWtTEWORSljjSPh+2nO/dOzbptb236Q9/7hb9zgM7tapWzf2YUEynz86vaF+ratUV7auz1CvJOLEl5lyUxLkr8sU6Bhul/WQVt1HeJ8ly1M4++LW9txVmZdCWeCfv24ESYkQS5WHatsyWJWXUne/SPh9xyynT4D+vbnauUag2ALuEianeUq99d25NrE0Gvf9Cq6Xz8wuanWswkBogqXwYpU9CccRtP7NzDZ2fX9BCqxX5Z9M6xjTKUYv6zFgb4r1o17auYgUWkU1N1rV9y7qhicOW2bIkHJw5o12Pv6LPfe117Xr8Fb0wcyby78jifEQtp0xbkdoA7DOsfQWVej320gntu2NrIm2y9/1rFantSQ8+9x9j54mySDI3hO2TUBxx2o/fjz/43H9U25NqFaXaL4U5RspRo8s73rmusQMrsEhVnNky257xdfLdC3p4/zEtNEffln7U2cNB58Yvp1xsXVmF9csp8zyPNsyYorgGta9+m358dNNavbb3Nh1/57wko20brxr5/Y+/84G+8OwRNZrt5RhM+/EVtuXKqMgNGEWU9hP0eJl6raKn7/+Ytm28KpW2NzvX0A1Tq/XSQ5/SxYXWimOcnWvo0HffU63SPdFs20ZkrueZNJC78scAFqmLUspi285uB2fO6OGvH9VCy+t6fZQOJm5pz7BzY3NZS1HLmWCHfu1rUEwcPvl+YrlmarKumVPn1Gh2v1eaF6K25cq4yA0YRdj2EzSZNV6taO3EWGbxuX3LuhVfrxqjiwvplzPHVZQ8kwZyV74oIYY1bCul8Y+nd/AqZd/BhDk3lLUA3frFhKREc83sXENPH3prxesLrXTyhG25ErBdlhO8w+Kz8+udg9fV9apV/TZ5BjZjBbbEbCsLsekZX/2OR5LGa9l3MGHPDWUtQLegmDh66lyiueb02XmNV6tqNLs3UXvo1ulUYtC2XOkq2/pApMefzNrTs5qYR3wGfX31eFWP3rVNt37kGmvaYpQ8QywhawxgS8rGshDbSmCDjme8avSnX/qUpq9dk/ux9Ds3lLUA3XpjIulcE/T76jWj+265Ptbvi/N+NpUdusDGPhDpymqCd1h8Bn295XlWDV6lfHdRBoahhLiEsnyId5TnmtlWAht0PF/52e3Lg9ckn9sW51hsKTMCXBM2nsLGeNDve/Ke7anFp835IMu8GBelkeU1aAfbpNrusPi0OX47ubSLsgt5B8liBbaEsig/izsjZ1sJbL/jyWPG0bZzA7hsWDxFjfGs49PGfODKSgwl2OiVdNsdFp82xm+QYcdpQyy5kneQrNQGsMaY35N0p6T3PM/76OXXfkXSFyT98PK3/c+e5/1pWseAYGmXnwVtVx/lcRK2lcD2Hs+of1+SxwIgvn7xFDfGs45Pm/JBnnkxKkqw0SmttjssPm2K30EGHWfeseRS3kGy0iwh/teSfjrg9X/ped6Oy/9j8JqDtMtX/Bm5Tv6MXBS2loQk9feVha2fI+xhWxspaoyneZ5dOmeulHAiviht3aW2a5u8Y4nPrrxSW4H1PO8bxpgb0/r9GE2a5StJzMhlWRISdvc8//tWj1eZvQ+J0h4MY2MbWT1eVaOZ7LMZ896lM+3znPdKTFSulHAiuqhtffP6CV3qifdLzVaqbTdOPsg7h/STZyy5lneQnMgDWGNMRdKk53kfxHzPh4wxD0g6IukXPM87G/P3YERpla+Mul19liUhYTu63u+7d+dmPX/kdOrb8buM0h4MY2Mb8WO9UjFSy1O9amQqZqQYz3uQnsV5zvIxJUlxpYQT4cVt657nDfx3kuLkg7xzyDB5xZKLeQfJCDWANcb8kaR/Iqkl6Q1Ja40xv+553pMR3++3JD0mybv8/78m6R/2ec8vSnpY0rqrr7464tsgb6PMyGW1KUDYji7o+54/clovPfQpXVxoZTbjaOvsa5DZuYYOffc91Sqm63U2SlmpzLnu+DvnVTHZtJEw8dMZ6z7PGL38UPxHZ9kwSM8qp7KqiUGyyHVx2vrps/OaGKvpQuPKc5wnxmqx42NQromTD2zIIYPkfW1C3imnsCuwWz3P+8AYc7+kP5W0V0sD2UgDWM/z3vX/2xjzO5JeGvC9z0h6RpJ27tyZ3lSY4/JOHIPEnZHLqiQkbEfX7/suLrS0fcu6RI+pH9tnXzv5x1o1RhcXki3DLKKy5rqDM2e0Z/9RNZrdf3IabWRY/Ph59Pz8wopYr1crK9pxFDbs0pllmR2rmoPZ3GenLYtcF6etx/mZfp/jsFwTd4CdVw4Z1l5tuTYh75RP2AHsmDFmTNLfkfSU53mLxpjIyccYc53neT+4/M+/K+nNqL8DV9iSOJKWVUlI2E4rj/tjOtk++9opaAVLklbXq2q1PUp7IOlKO+kdvNZryW8AMix+OvPoQquldk/PNupAz4Z7tCizs0NR+2ybxGnrUX+m3+cYpq/OaoCdhDATf65cm6B4wg5g/5Wkv5R0VNI3jDE3SBp4D6wx5t9I+oykDcaY05IekfQZY8wOLZUQ/6WkfxzrqBGYOH7x60e19bqrYpe62aS3JESSjp46l+isdZROK8v7Y3rZsIITVtCxrh6v6tG7tunWj1xj3fEiH0Ht5ENjVf325z+uT9+UbGnhoPiRtCKP1ipLA+nxajIDvTB5JotVOcrskhflc+NiPztx2nrYnwn6HB/ef0zrPjQuyRvaV6cxwE4jf4Rpry5dm6B4Qg1gPc/7qqSvdrz0fWPMrUN+5u8HvPy7EY4NAwQljoWWp8/+xmF95Z5izOr6JSGjzFoPS+xhOq2k74+JyoYVnLCCjrXleQxe0SWonbTladvGqzJ5Lz9+gvLoxFhNT9//Ma2dGIt8Qdgv3wzKM1muylFml5yonxsX+9mK09bD/EzQ59hotvVP/uANtbx2qAqOJAfYaeWPfu31+DsfLOdGl65NUDyhngNrjLnWGPO7xpj/4/K/t0r6B6keGQYKShyStNBsa8+BY9Y8T3FUnbOAFxpNXVoM//cdnDmjXY+/os997XXtevwVvTBzJvD7pibr2r5lnbUP6s77OWtRuHSsyE+W7WTQe/WL7W0brxqYE4IMyzdBeWaU/Ib8xPnc8u5HkIx+114/Wmyp0fTkeZ7qteF5bdh1R5Den0kzfwT9nfOLTX3h2SPLOe61k+/T3yM3YUuI/7Wk/1XSL13+9/ck/bFYUc2Nf1H2i18/qoVW95RfkWZ1485ahyl/CVt2Y8P9Yy6V/7l0rMhPlu2k33v1i20p2i0LcctDWZVzU5zPzYZ+BKPr/BwrxuhHPRu8jVLB0U+/a5U080dve/X3B2g022o0r+S41/beptf23kZ/j8yFHcBu8DzveWPM/yRJnuc1jTHxt2VEInbv2KSt112lz/7GYS00rySwIs3qxp21HpbYo5bd2DAoc6n8z6VjRX6ybCf93qs3tg+ffF+7Hn8lUkle3AtJVuXcFPdzs6Efwej8z/H4O+f1hWePdG1G51dwJPXZDrpWSTt/dLbX8/MLevC5/6jF1pVbqfwcF3UlGUhCqBJiSReNMVNa2nxJxphPSDqf2lEhtOlr1+gr9xS3hCNuqeGgxB637CZOyQ8A+/mxLSlWboh7IUnJvZtG+dzoR4pharKuT990jZ68Z3tq8TvsWiWL/OG3120b1zLZBquEXYH9eUkvSPprxpjXJF0t6Z7UjgqR2Darm/SOeHH+vkHlWkdPnaNsD8AKcVdSRykPTTN/l/mZo2mzrd9FssLGTprtIEw+yqodUgIP24Tdhfg7xpi/LeknJBlJ/9nzvMVUjwyR2FKymdaOeHH+vn6JPWi1pNFqa/V4deTjBGCXKIO4UUryRrmQTCN/88zR9NnS75ZRmpMzUWMnrXYQNh9l1Q6ZtIFNBpYQG2P+W/9/knZraQB7k6S7Lr8GLLNxR82gcq3OsptVY0shYDxPdz51uO9OxQDcE3Ynct+oJXm2lIfamIuBpESN6yhsih0bbzGwJccBw1Zg7xrwNU/Sv03wWGCROLObLu2oubwB1le/KUlqtDyp5fFgeaAg4u4MXIRVhkG52P+6q38byi1uXIe9prHtOqYI+QhIw8ABrOd5/11WBwJ7xC09c21HzYsLLdVrVS0E7KpHJwG4bZQLUddLQ/vl4jfPnNffe+ZblBXDWXHiOso1jY3XMa7nIyANYXchljHmDmPMHmPMP/P/l+aBIR+jlM/YWO4yiI0dFYBklDm+g3Lxvju36rGXT1hRGgnEFTWuo17TuHYdA5RVqE2cjDG/LelDkm6V9DUt7UD8H1I8LuRk1PIZl8pd2FUPKK6yx3dvLratNBKII2pcx2n3Ll3HAGUV9jE6f8vzvJuNMcc8z3vUGPNr4v7XQkpi1cKlchc6KqC4yh7fvbm4rCvSKJYocT3KM5rLli8Al4QtIZ6//P8/MsZslNSU9OPpHBLyVMbyGXbVA4qL+F5SxtyO4gob17R7oJjCrsC+ZIxZJ+kJSW9cfu1r6RwS+snqofRlX7UAgCIitw+XVT+L7JSl3dN2USYDB7DGmP9K0inP8x67/O9JSf9J0ncl/cv0Dw++rB9KT/kMABQPub2/rPtZZKfo7Z62i7IZVkL8ryQtSJIx5tOSfvXya+clPZPuocFn04O1AQAoGvpZuIq2izIaNoCtep73/13+778n6RnP8w54nrdP0nS6hwafv4tep86H0hfJ7FxDR0+dI/H2CDovnCvYym+bJ9+9kHkbJS4QR5n62bLqzA1FyhOutt0ifQbI3rB7YKvGmJrneU1Jt0v6YoSfRULK8jxDSmCCBZ0XT+JcwUp+e/XanhotT6vGli6ssmij5BDEVZZ+tqw6c8OlZkue52lirFaIPOFi2yVXY1TDVmD/jaR/b4w5qKWdiL8pScaYaS2VESMDo+6i58IsFyUwwYLOy8P7j2rPfs4V7NPZXhstT5J0abGdSRslh9jN9n6I3WqLqzc3LLY8NdsqTJ7Is+3GiWtyNZIwcBXV87x/YYz5M0nXSfq/PM/zLn+pIulLaR8croi7i17Ws1xxdsGbnWvo0HffU61iul4f9rDxMgh6CHvVVKTuU9V1rtiJEFnz29z5+cUV7dWXdjwHxUqS7+n/javHq7q40CK+InBltaUsu9WWTVBu6FStGB367nu69SPXDPzMbe5b82i7ceM67VyNchhaBux53rcDXvteOoeDQaLuotc5y+Unij0HjmnX9IZUkkScZOb/TNUYXVxodX3N9hKYLASVBrW8tuR1j2D9c+XKhSKKo7PNLbTaarWDLxLTjuc0y+j8v1FaWlGuV41MxRBfIWTdD42q6LvVltHm9RO61Gz1/frFRkuPvHBcv3zwzb4x7ULfmmXbHSWuXSx5hn2GlRDDYVne2B+nJKTzZzoHr6vrVcq3LgsqDXrynu168p6V5UKSKMtBpnrjvtFsyxijes2oXl2aZFk1VskkntMqo+u6UFtcuuhqtDziKyRXN5hBsVwpILziQ+NX2uXFhVbfmKbkdaVR4ppyfSSBjZgKLMtZrjglIUE/s3q8qkfv2ja0lKdM+pUG9b529NQ5ynKQqaAYXlWr6un7f1JrJ8YzL7dNo4xuUPkh8TUcqy3I2+mz85oYq+lCo7n82up6Vf9o14/rdw//l64J9KCYpuR1pVHjmnJ9jIoV2ALLcpYrTjILLo/1GLwGmJqsa/uWdV3npfc1LhSRtX5tbtvGtdq+ZZ2mr12zot2mLShWRhH0N/qIr+FYbUHeAq812p52b9+oVs/KbFBM07eulERcJ52rUS4MYAtu945Nem3vbfrDn7tFLz30Kd0wtTqVspeoyczfDGHfHVutu7BJY7fMLHbg5EIRWRvU5qK2eRt3qe3NU/4jgepVY/1Onzbp7Ide23ubdfcOotj65anpa9eE6jPT7FuTiO288kPScZ13nsv7/RGNCbovwDY7d+70jhw5kvdhOC2rDQjC7NLXeyz77tyqj25cm2sZiX/cb545r8dePpHoeXJhJ+isJXWMxpg3PM/bmeCh5SrPXDfKZ9L7s/3afL/3sHGDlH55KuuyaBvPDbJHroumN9fMzjV0/J0PJHnatnFtV+yGzX1J961JxHZR8kPef0fe748rwuY6BrAlMDvX0K7HX1negERa2ljltb23ZT7AselYfH7iqlWM5hrdOxWOemw2/r15S7Kj4KIuGUl+Jv3a/L47tgZODtkYI7Ycky3HgfyR68LrzWf37tys54+ctmpwkkRsFyU/5P135P3+6BY211FCXAI27QJp07FI3bsL9g5epdGPzba/N44ky2rYzTF/vZ9n0p9JUJuvGqNHXzoR+B42xogtx2TLcQCuCMpnz37rbev6nCRiO+n8kFcJbd55Lu/3RzzsQlwCNm1AYNOxSMMfcD7qsdn290aVdFkNuznmK+jzvGFqdaKfSWCbb7U1Xqto4comoMvvYWOM2HJMthwH4IphfbpkR5+TRGwnmR/yLKHNO8/l/f6IhxVYx4WZMbNpc588jmXQOeq3w+jq8WSeRWvTuY8qjdVSOor89Ps8V49XE/1Mgtr8I3dtU7PdfbvKQqut8/OLkmRdjNgSt7YcRxxsiII8DNo13Ddqn5NE205qF98k8kNalVFhz1PeeS7v90c8rMA6ImjzgCgzZjY9cyvLYxl2jvzEtSfFTaWy+nuT3mAijdXSoPNNR5GNfp/nxYVW388kbpsKavNrVtWW3+NSs6VWu60Hn/vO8vu9tvc2K/LToL+hzMcRRZi+yYXN5uCeoD4m6B7YuG0u7kplUHtPIraT+B1p9PVRz1PeeS7v9y+qNPM8mzg5ICgR7JrewE3nQ0S5Md/1i6k0yn/S3NiAXYiDpZnrhn2eYXcSHvUYjr9zXl949ogazSt9D7mrOMLkDXb8jI5cF03QLsSj9jlx+0Tb23vSfT2bIkGK3+7ZxKkg+pV2HH/nfC43nbtUFhblxnyXH6idVvlPmmU1Lp9vVw37PDs/kzTb1NqJcY1Xq12vJ5W7XMpPRTUs77KRG7LQ28ck0efE2ewnyfaeVn5Luq9nUyRkkecpIbZcv9IOyWR+L6Hts4i9ynK/ZVAbqcjo+Dsf6NM3XT3S76asplgGfZ6dKxRpbraVVly6lp+Katjny0ZucFWc3JVUe4+a36KuOCfZ15fl2gv9ZZHnWYG1XL9EsG3jVZnedO7irHlZbswPaiM/WmzpC88e0QszZ0b+/ayWFkvQ53lw5ox2Pf6KPve117Xr8Vf05jvnU7sASSMuXcxPRTXs8+XiFq6Kk7uSaO9R81tvPg97HZBUX1+Way/0l0WeZwXWcoM2vclydSyrWfOk70W1ZQUx7N8V5+/328jD+4923VfYaC51crumN9BxoK/OiyM/vh976YT23bFV//yl46rIqOm1te+OrYm1o6TjklU9uwz6fNnIDS4LatuD+u1+7V2Sjp46F+p3RMlvQfk8j+sAW669kI8s8jwDWAcMuxjIIjFkMZuSVglgVueon7B/1yh//+4dm7TuQ2P6J3/4Hf1oobX8OhfxGKbfxdHsxQU1W55alzf6+5UXj2vNqlpiZblJxuXm9ROaX2x2vTa/2GRVL0eDPl8ubuGyzrYdpt/ube+HT76vXY+/cmWX5I9v1vNvnO77O6Jcf9k0mZf3tRfylXaep4TYEXmXcaZdElLUEsCwf1cSf/+2jWvV7tlVnNI8DBN0cbTQaumpV95Sq6M5LbY8Pbzf3pg0xgz8N+ySd58GjCpKv+23d0krfubZb7898HdEuf6iRB82STPPswJbAFk9AibN2RSbZg2TFPbvSuLvT6Nkw/XHC2Ew//Pdd+dWPfbSieV28+BnpvVbr/6/Wmi1ur6/WjFWxuTps/NaVatqsXVlFXZVrWrlsaaFWAXSExRfcfrtoJ/pFfQ7wl5/UaKPsmAA67isd970S0L87dyTesZaUWcNw/5do/z9nec+yUkGdnUttue+/X09+uJxjVUranme9t2xVR/dtHa5zT116K0VP9Nqe1bGZFD8NJotrR6v9vmJlVweABKrwJI04rhffG1eP6FLze5JvkvN1sAcGZSrevXr+8OW5Ea9Txfp4/wnjwGsw/K6Wb83mQ+7fyOMos4ahv274v79/TrWUc+bLRtBIB3Pffv7+qU/eVOSlldZH3v5RNeD5p+8Z7t+/vkZNS9fa41VjZ68x86Y7Iwfr+2p0fJUqRjd+dThUPnI5QEgsQosSSOOB8WXJHk9t+30/rtXUF9/787Nev7I6USvfaLep4v0cP7TwQDWEnFmZ/Iouw1K5s9++21JGvniqagbe4T9u6L+/WleuMZpW8ww2qvzs5GkR186seJ7qqa7PNhvj8ff+UCSp20b11r9ue7esUlbr7tKn/2Nw5I8XVoMFxOuDgD9z/T8/EKm/QBxDhulFceD+kJJmhir6ULjyq0LE2O15djrjJWzFxc0c+qcdmxZF9jXf/n2m1KJK1fzW1Fw/tPDANYCcWdn4pSvjCru/Rthpblrnd+ZrB6v6uJCK5WOYtBW+mHeq/P7hl0opjmBEbWkmRlGe/V+Ng9+ZlrjVaOF7k17tdhqa/V4tevWgKnJuj5909UDf39vO81zgHNxoaV6taKF5mj3pNl+/33nZ7rQaqnds+iT1u0XxDlsFRTHUe/ZD8pdw/rCfl/rjJWLC82uGH3gk9frn//M30j0iRL98m4W+Y1Jrf5c7F9cwQA2Z6POzkQtXxnVKPdv5MnvTPzywlVjSxtwJ3UBlvSFXZjfl+Z9w1FKmplhtFfQZ/PUoZOSVuaJn7x+ne586nCkNpzG7QSjiBMTrt1/H/SZ1ipSvVbReDW92y+Ic9gsKI4vNlp688z55d1/Bxl0O86gvrDfM157Y6XTs996Ww984kZNX7smgb988PVC2vmNSa3BXOtfXMJjdDLib3rUu726PzvTqbM8ZZDTZ+c1MdY9B+GXr6QlaDv3Bz55fde/9925VafPzlvzuI3OC6/G5eeCXFpsJ/aonqQfART0+x7ef3TF70v70Ua7d2zSa3tv0x/+3C16be9tfTulUdow0hX02dSqRv/wUz++4ntf/y9nI7XhoHba+ziIX9x/TCffvZD439VPnJhIO46SFvSZTozV9DsP7Bwaq0m/L3Hev29HtqYm69p359YVrz/28gnNzjUGfk7D+vBd0xv0zOc/rqfv/9iK+ArqJ4NipdfMqXMj/sXhjj3N/Hby3Qt6+OtHC/f4wyS51r+4hBXYDKQ1O5blzM6wnW79+zfePHO+63EcNszGDSp7TqKUI+kSkaDf12h6+qPX39aXbv9w1/emfd9wmLImZhjt1W9VQp70obGKfrQY/1aAMLcTLDTb+uxXv6mv/Oz2VPPAqDtxu3T/fb9427bxqlSPmzhfidUnu3x049rlW4R8Y5WKnnv9bf3mqydXfE6D7iP39wQ4fPL9oZ9xbz8ZplJtR4hV4TDCXH+kkd8OzpzRw/uPaaHVXc1DeexKLvUvLmEFNmVpzo5lNbNzcOaMdj3+ij73tde16/FX9MLMGU1Ndj+ceGqyrs3rJ/TYyyesm40b1JkkcQGW9IXd5vUT+lHvDYpaKv0c9ID0vJIiM4z26rcq8cw3/mLg4FWKV3obZKHlpZoHwuSnMPKOo7DyijfivFvSlTcY3eb1E2r13Ea10Grp6UMnV3xOz337+8t54wvPHlmxn8jFhZZe/4vZWJ9xb6xUTPfXH/jk9YmVD4e9/kgyv/ltv3OvgUHvDXf6F5ewApuytGfHkpjZGXQDfpT7nmy9WT3oERud98AmsV19530wC62ljXJGYYyRejriWsQNKbLEDKO9glYlWgG3yt+7c5NeOPqD0I9y6Pc4iP/t/zm94sImrTzQLz9tve6qVDZqs0Ve8UacX2Frf1dmQTnpwc9M65lv/IUaHTmpWgcpZiAAACAASURBVDF69MXjWmh5y59f1az8fV/5d9/TWM/oM+xn3BsrnbsQJzV4la5MUj764gmNVY1abS8wdye50VK/6pvxqin1pBayxQA2ZVFmx+IG/Sg/O6wEKkonbXOJWWdnksYuxP7vf+71t/X0obf0zDf+Qk+/ejJWSdnps/NaNVbRXKN7RnixZce57CfNHaQRX9CqRK/V9aruv+VG7f3pvz5y6e0Dn7hRn/3qN7tKy9LKA0H5yWt7+uxvHFa9WuyyzrzijThfYnN/V2a9OUmSnn71ZNf3LLY8jVUry8/AlqR6rSpP0vxiR/lx1WixGf8z7oyVqcl6ogNX38GZM5dv21o61kfu2rYi3yVd6h7U9sdrFf3plz6Vyt8IBKGEOGVJll0FbUIwygYSYUqgonTStpeY+SUc09euSa2U4zdfPalG0xuppGzz+gk1e5+LIemRu7ZZcy7hjs64XF2vBn5Ps+UtD0BHLb2dvnaNvvKz2zPJA0H5qdHytNDMrqxzdq6hb3zvPX3jez+kfLREbO/vyqwzJwV9To/ctXXFpF7La6vd+1rb0yN3bev7Gee1gZf/viffvbB8DXdxoaWFlre8aVXn9yZd6h50Tr9yz80MXpEpVmAzkETZVdAMmieNNKsWZnU1yuNU4vytRXp+WFIlZZ3nvFoxWmx5euSurbr/lhvSOGyUgB+Xh777nh554XhXObEkPXTrdKIlZ1mVmvbmp0azpUrF6NJiNmWdB2fO6Be/flSLl1ebaxXp1+/dUcgVX6xESbUbgiqw9t25dcWGk5JWXOvs3rFJP/3RH1vxGee1gVfn+zZabRlv8CZKaZW60/aRNwawGRml7CroPq+H9x+VZNRoxn8mX9jV1aiJKuzfWrQdHJMsKaNzQNKmJuu69SPX6JcPvtn1er1mdN8t13e9lkRsZlVq2ntxeudTh7u+nubO7Hv2H1sevEpSs72Um3k2anlQUu2Gqcn6ih2F992xVR/dtLarjw3qd3s/47yeiRz0vr16813az4un7SMvlBA7IOiZYlVTUbXP5gJhRSmBSnoHtSLu4Jh0SRm71iFpQW30yXu29704cyU2O28PyKqs8/TZ+RU5WFrKzWV/Nipgm6C89tjLJ1ZMEIfpd/N6JnLQ+64aq2i8avrmO0rdUVSswDogaAat5bUlr/viKc6sWtq7GPf73qDnrhVhB0dWTmG7YW00qOSsWjE69N33dOtHrhnYpm24JSCrGNy8fkKtgHvVWx4b+QC2SbKUNuqqZhJ5cXauofPzi1porVx5/dP//r8euDEl1yUoIgawDuh3H6q08n6NOIkpzV2M+33vQqul3mu/ouzgSFkNbDeojQZdnF1stPTIC8f1ywff7BvjNt0SkEUMTk3W9eQ9N+sXeu6B7V3RBpC/JEtpo+wNkkRe7PwdrXZbY1WjVbXq8u8Ls3kS1yUoGuMNebyCDXbu3OkdOXIk78MYKIuVh973mJ1r6Pg75yUZbdt4VebJaXauoV2Pv9K1YcqqsYpe23tb4IYwvd9bq0jVSkXjBX/cBdJjjHnD87ydeR9HUmzJdS/MnFneROxiz+OcgmI8Si6wzai5O+88XHY2rPpngVw3Oj+v9Q4mO9uQpMQ2oUwiLwb9jnrN6Hce2KltG9cWus2jnMLmOlZgE/Dct7+vR186obGKLu8Yu033fyL5HWM7Z9BsWO2IUpIT9L0TYzU9ff/HtHZirPAXH4BLBu1aHBTjae10mbakNqv69E3XpHSEGDRICPr8KJVEvzYTVErb2YYuNVvyPE8TY7VQ+WDYqmYSeTHod4xXq1o7MU777lCWiSxcwQB2RM99+/v6pT9Z2tVz4fJrv/Qnb0pGqT32JK8d8HpFKcnp972sWAB26rdrcVCMp7nTZVpsyaPob9AEQ9Dn9/PPz1DVU3LDJqU6B539dvW90GhKGj0fJJEXXcytWbNhQQfZYxfiEczONfToi8cDv/boiydS27UzzR3wojyYO+ouxuyEB7glbNxGie8oOSZNee0kinCG7YYd9Pk121Kj6c7u2RhdZz6JuoN6UBvqNGo+SOK6h2unwVzcNR/JYAV2BKfPzmusurQhUa+xqkmtfC7OjJxfXuE/xDtsOdawWawou9uVZSc8SllgqzhtM2zchvm+7o3c2nro1mndd8v1fX9nmrHEyobdhpVfBn1+vVwoY0d8vdcsD35mOlLJ7rA2FDcfdOatQXkxbH4Lm4PLeO3h6u0rGB0D2BFsXj+hVp9NsFptL7ULoSg74ElXkrwkXVpsq141MhUztBwrbPlMlN3tir4THqUssNUobTNs3A76vqAc82v/7nt66tBbevKe7SuOJe1YippHka1hEwy9n5+/s72/I3Tv96NYgvLJU4fekhT+8YK9bSjoHtio+aBf3hp1wWBYDi7rtQcTkeXFAHYEncmv3fa00PJUr1VkjFK7EPJn2HZNb9Bre28LNSPnJ3lfo+VJLa9rgOr6LJYNM4/cUwdb2dA2g3KMJDWa3opjSet4e/NEWapCXBRmgqH383vt5PtMSJREv82Nvvjpv6qnXz25og2E3djJ/91x8kHYvJV0fovz+2y4ZkoCE5HlxQB2RJ3Jr7c8N+kEEWeGrd9FozS8HGux3dbq8aqOnjpndZKzZebR9UkAFNfps/Pyeh687LW90G0ziVw2qFyvN07SiKVBKyPEp52CBhe9/VHn58eERHn0u2a575brdd8t1/fdaXjYxk7+v+MIm7eSzm/H3/lAlZ6V50G/L+w1kyuDXOK+nBjAJiDoAijpQVXcGbtBF42DyrEW223d+/HNuvOpw7kPDAexYWXJRykLbLV6vLpUedGh0fK0erw69GeTymV+jnl4/zE1moPjJOlYsilPIBq/fw3bDpmQKIdhK29pV3MECZu3ksxvB2fOaE+InOoLez5sWRgIi7gvH3YhTkEau6LF3TGzcwe7VWNLP1+vmsCd7Hbv2KTX9t6mP/y5W/TSQ5/S82+cznxnt9m5hr7xvff0je/9MNR72bSTKLsFwjZL8fRDHT75Q9Vr3TP0q8YqXc937ffzSeay3Ts26f/+p7fpF37qJtVrpm+cJB1LNuUJRMdOowjSec3y2t7bAgdYQbFfrZhUYj/Oru2rx6sarxrtu2Nr7JLl3sFrvWb65sswuZB4gwtYgU1BGuVvo8zYDSpz7uXPYh09dS7zctiDM2f0i18/urwJR60i/fq9OwbO+tm26kkpC2xxcOaMfuH5GTWDCzAkaWicpJHLpibr+tLtH15R5tcryViyLU8gGm7PQD/DVt6CYv9io6U3z5zX9i3rEj+eKLu2X7jU1KMvndB4raLHXj6hNatqkVY5g+LiQ+NV/fbnPqZP33RN4M+EyYXEG1zACmwK0rhYGnVFYmqyru1b1mn62jXavmXd0J/L+oJvdq6hPfuPde0g2WxLD+8/OnDWz8ZVT/9ck+iRl6V4Oho4eF09Xg0dJ2nmgTBxklQs2ZgnEB4TEIhrarKufXduXfH6Yy+fSG1FMUzemp1r6LGXT2ih2dZcoxVrlTMoLtqep20b1w48tmG5kHiDC1iBTcHUZF377tiqX3nhTVUrFXnyErlYynJ1L+ud3U6fnVe1Yla8XjXds35Bmwqw6glcMTvX0KHvvicjI6n7vteJsaoe3b1Nt37kmlBxUqQdHl3PE725z5UNVpJQpHaI7H1049rl6jNfWiuKg+Ky82tJrHJ2xkW1YrTY8rTvzuGlyMNyIfEGF6Q2gDXG/J6kOyW953neRy+/9lck/bGkGyX9paR7Pc87m9Yx5OXgzBk98sKbaraXZq3GqisHZnFleaN6lhd8m9dPqNVe+Uzdlndl1m/QpgLcwA9ciZFaxehSwPJr2/NCD159rg/8OrmaJ3pz3707N+v5I6ed2WAlCUVqh8jW5vUTannd1xdprCgOukbp/dq+O7Ymssq5XIr84nGNVSt67KUTWlMfXoo8LBcSb7BdmiXE/1rST/e89k8l/ZnneR+W9GeX/10oQaV7iy1PD+938wb4rMphpybrevKem7sG+7WK9OQ925dXG9hUAOivM0bmGis3ZxqrGj15T7xZdMri8xOU+5791tulzIW0Q8SRxS0Eg65Rgr722MsntO/OrSMf03IpcsvTxYV4pcj9EG+wWWorsJ7nfcMYc2PPyz8j6TOX//v3Jb0qaW9ax5CH02fnVTUVSd0XkP6ud1mWq7jGn/E7/s55SUbbNl61/DexqQAwWFCMrK5X9fB/8xP6q1ev1raNa62OlSLlsiQNepa3j1wIXJHHrUaDrlH8/+792kc3rtVre28b6Zi4NkJZZX0P7LWe5/1AkjzP+4ExJnibNIctlaqsvNBotb1My1VcNTVZD9w9j00FgMGCYqTV9nTX9o3WX8gUMZclZdCzvH3kQmBJXrcaDbtG6fe1UY+JayOUlbW7EBtjvmiMecsY88O3334778MJbakUdrtqHWd2lNK9fspWUssuoiiqpHKdqzFStlwWVdDn+sAnr3fucwbSvq7LM5cMyr9p5mZX8z4wqqxXYN81xlx3efX1Oknv9ftGz/OekfSMJO3cuXPl7j4Wu1IK+4EkL3TpXmfZi6TA/06ypLbf+9ma+NhUAEWUZK5LMkbilPTG+Zl+uez4O+e1dmK869nV/veXLf6DPtcv335TKc8F3JX2dV1QLqlWjF48+k4qt1H05rtB+TfN65eg351V/gbykvUA9gVJ/0DSr17+/4MZv39mlkphrw79/Z1lL5eaLXmep4mxmuYXmzLGaFWt2lUOM2rZSL/3s718z9VdRIGsJBEjcUp645YBB+WyS82WvvDsEcmTGi1Pq8YqarU9Z/JUGno/V3Ih0C0ol1xstPQrL56QtFQN92s/uz2RvNEv3w2KyzRjtvN3Z5m/gbykVkJsjPk3kr4l6SeMMaeNMf9ISwPXnzLGvCXppy7/u5Bm5xo6eupcqNKV3rKXxZanZlu60GguPYqn5a0ohxmlbGTQ+1G+B7gvSv4J+tmoZXijlO715rJ6rSLP89Roemq0lhZpLi22yVMa7XMFiq4zl6yuV1d8PaknQgzbcTipGI3zu7LO30Be0tyF+O/3+dLtab1nGoJKKoaVWUSdyQqzy6Svs0w4bknKsPfz2h472AGOOjhzRnv2H1XVVNTy2nrynmgrDnFuTxj1lobOXHZ+flEPPvcdXWg0B/5MUXfa7Ne/lH2FhPJG+Aa1BT+XHPrue/rlP3lzxfOwK8bo0Hffi/w87E798t1zr7+t33z1ZCIxGjfe88jfQB6yLiF2SlAC8aSBSaVzJstPBnsOHNOu6Q19E0GYXSZ9vWXCcUpShr1fo+Vp9fjK2UsAdpuda+gXnp+5/BzqpUd5/fzzMwPzT684tycksROmn8tm5xqh8mERd9rsd9Eap18pkrIP3nFFmLYwNVnXrR+5RoutlXlkfrGlR144rl8++GbsdhSU7xZaLT196KQazdFjdJR4zyt/A1mzdhfivAWVVDy8/5j27D86sMzCn8nq1PkssCC9JXRjVaNaRVpTr6lWWbpvY1CZcNQyE//96rXgj79eM7q40Ar8GgA7zc419OLRd9Sz4KBmW5c3lAsnzu0JSe6E2fm76lUjSVo1VunKi0XcaXNQGV+cfqUoKG+EL2pbqFRM4OsXF1qh21HQ9VVQvnvo1g9rvJpMjI4S73nnbyArrMAquByl32528oz8lQ1pZZlF3Jms3nJg/xg6/9vfjdO/B1aKPzO9e8cmrfvQuP7xHxzR/GL38XqemHkDHOLngeDLNUmKtuFnnNsTwvxM2DLQzt9Vll2IB5XxlXmFxKbyRsqY8zM719Ch776nWs+gtF9bOH12fmnzy1b/WxGGtaNB11dB12xPv3qy6+fjxuio8Z5W/nYZsVs8pR/A9ktQQQmk1fbUeyEYVNL7xN03a0/P7wwTMEG7TPoOn3x/xXHumt4wUlnZto1XBV7WPnLXNgIccETnqkSQsarRto1rI//eOLcnDPqZqJNt/X5XUXPToIvWUfoV19kyeKeMOT/+ua+aldVh/dpCmFuzBrWjMGW8vTkqqRhNIt6Tzt8uI3aLqdQD2GEJat+dW/Xoiyc0VjVqtT09cffNy98zKKkkPZPV7zif+fzHR5qZ7kyS1YrRYsvTI3dt1f233DDS8QLITr9N2fxbBJ68J/+BTtnv4Qxj2EWriyskSax62DB4p/3mp98E3ep6dfm6rN9EV2+7uXfnZj1/5HSodhRn5T/JGHUx3vPWb9NVYreYSj2AHZSgDp98X4+9dEJjFaPFZluP3LVtecYmTFJJciar33FKZuSZaZIk4LaglYZ6raLfeWCntm28yoqYtqkM1GbD8rFLKyRJrnrk3U/RfvMTdO5Xj1f16F3bhu4kHNRuvnz7TaHaUdyV/yRj1KV4z1u/fEPsFlepN3Hql6BWj1eXZ2wuLrS00PL02Msnlm/in5qsa/uWdZk1/n7HuW3jVYnceJ/13wMgOUEbcDx5z8369E1XWxPTtpSBuqAI+TiNjZfyPC+03/wE3s7leaEfg9PbbsK2IzY2csegfEPsFlepV2D7lSZdXGhZNWMzqIQq75lpAPmzPQ/YUAaK7BRt1YP2m588z73teRVLBuWb7VvWEbsFVeoBrBScoIKeQ5j3jM2gREqZCQDb8wAXg+VRxFUP2m9+8jz3tudVDM83xG4xlbqE2BdUYmJj6UgRSssAlBc5rBxs7UNHRfvND+ce/YTJN7Sf4in9Cmw/Qc8h7Hz+qq141hVQDsQ68hKm7bHqARulkTfJxfkj35QPA9gBpibrgc9ftfX5UTzrCigHYh15idL2KL+ETdLIm+Rie5BvyoUSYi3Nnh09dW7FDolp7KQY5f2j/o4wx5rEewHIT1CsP7w/+bzU+57kDWTVJ2aJtl0OabTdtK+7aJvlw2ceXulXYAfNnmWxk2KY2bsw5SlhjpWZQsB9QbHeaLb1R6+/rS/d/uHE38/PG7WK0ULL0yN3bdX9t9yQ+PvAfkXbXZg+sTzSaLtRr7sWWm09dOu07rvl+qHvSdssHz7zaEq9Ajts9iztnRTDzN4dnDmjXY+/os997XXtevwVvTBzJvB3DTvWIs6cA2W0ef2EFlrtFa8/deitVKpD/Lwx12hpodnWL/3vb+q5b38/0feBG4q0uzB9Yrmk0XajXnc1mm392r/7nv7Wr/5Z32u5oJ+jbRYfn3l0pR7A+rNnnfzZM2mpnn7fnVs1Xqtodb2a+E6Kw94/SoMetgvbsPcC4IapyboeunV6xevj1Wri8Xz67LxqFbPi9UdfPE7HmjEbSsuKtLswfWK5pNF241x3SVKj6Q0cnOTVNm3IMWVFPoqu1CXEw2bPDs6c0WMvndBYxWix2dYjd20beTm/sxx42PtHLXkZtAtbkWbOgbK775br9dSht9RoesuvpRHPS6u93orXx6rpl42ys+cVNpWWFWW3T/rE8km67c7ONXTD1Gq99NCndHGhFeq6yzfoWi6PtmlTjikj8lF0pV6BHTR71rn6eXGhpYWWp8dePjHSzFRvOfBrJ98fOHsXp0H3e9ZVkWbOgbKbmqzryXu2px7PU5N1PXLX1hWvtzwv9YupMLdOlIGNpWVFeKYifWI5JdV2O3PUnU8d1vdnL/a97qrXVl5qD7qWy7pt2phjyoZ8FF2pV2Cl/jNySd/w35kg/N+558Axvbb3Nr2297bAGUG/Qe/pmRWL26CLMnMOILt4vv+WGyRvqWx4rFpRy/Myu5jqzJW7pjeUMmcVbeMkm9AnIo4oOcpvY3/0+tt66tBbGq9WQ13LZdk2yTF2IB9FU/oBrBT87Kikl/MHJYhBs4FJN2iekwUUR1bxfP8nbtBPf/THuJjKAaVl6aJPRFRRc9TUZF1fuv3Duu+W6yPl0KzaJjnGHuSj8EpdQjxI0sv5oySIIpRrAXBbVnmIi6lulJYBdombo2y9liPHwEWswA6Q5OpnUDnwvju3du14DABJc20zpKRvnSgCSssAewzLUa7lXIkcA/cwgL2sX8JJcjm/M0G8eeb85R2O2fENQDpc3VmSi6mVylBa5uKFP8qpX47yc27VGC22lp5ecf8nbsj5aMMpQ45BcTCAVbYXeX5y+HvPfItNSgCkxvXNkLiYKhdXJ1tQXr05qjPn+n7pT96UzOXN8AAkpvT3wOaxfTgPLAaQNvIMXMFjPFAEp8/Oq2rMitcffXG0RzACWKn0A9g8LvLYpARA2sgzcAWTLSiCzesntNhqr3h9rGpoy0DCSj+AzeMijx3fAKSNPANXMNmCIpiarOuRu7ateL3V9mjLQMJKfw9sXjteskkJgLSRZ+ACdp5GUdz/iRsks1Q2PFY1arU92jKQgtIPYKUrF3nH3/lAkqdtG9dG/h1xdk9kkxIAaRuWZ9j5NT7OXXKYbIEtBsV1mJi//5Yb9NPbfoy2DKSIAexlh0++H3sHRHZPBOAicld8nLvkMamLvA2K6ygxT1sG0lX6e2Cl0XZAZPdEAC4id8XHuQOKZ1BcE/OAXRjAarQdENk9EYCLyF3xce6A4hkU18Q8YBdKiDXaDojsngjAReSu+Dh3QPEMi2tiHrBHKVdgZ+caOnrq3HLpR5jHTfT+jI9HVQCwTb981YncFR/nLjlh2iqQhUFxbVvMZxE3xCZsZjzPy/sYhtq5c6d35MiRRH7XoJvw++0uF+bGfXajBLJnjHnD87ydeR9HUpLIdVE3FyJ3xce5Gw0bYYVHrsvOqLsQpy2LuCE2kZewua40JcSzcw0df+cD7dl/TI1mW5e0VAqy58Ax7ZresDzD1puQTr57QQ9//agWWl7gz/jYcQ5A3jo3GhmUrzrZlLtsuDiMwqZz55o4bdVVrrXrshsU11nGfFC7ySJuyhSb/RCz9ivFANafSarIqNHsvofBvwk/qIEenDmjh/cf00LLC/0zAJAXf6MR/6JDcidfMeNfLi631Sho14ijX7vJIm7KEpv9ELNuKPw9sJ0zST9abK34er+b8P2fW+gZ8A76GQDIk6ubC/GIivJxta1GQbtGHIPaTRZxU4bY7IeYdUfhB7BBW59L0ofGqytuwu+8Yb3fz41XDZt1ALCSbRuNhFXER1SwAcpgrrbVKIrYrpG+Qe0mi7iJ8x5FyXfErDsKX0IcNJNUrxn99uc+pm0b1y4HZG/JwL47t674ubGq0R/93C3a+eNTmR0/AESxe8cm7ZreoOPvnJdktG3jVXkf0lBFm/GnBC0cv60W9V6zorVrDJfEvZPD2k0WcRPlPYqU74hZdxR+BTZoJunJe7br0zddE3hTvF8y8NhLJ7Tvjq1aNVbRqrGl01SR9Lnf+w96YeZMjn8RAAx2+OT7+uIfvKEHn/uOdj3+ivU5q0ircZSgRTM1Wdf2Leuc/KyHKVK7xnAHZ85o1+Ov6HNfe32kvBum3WQRN2Heo2j5jph1R+FXYKXhM0n9blj/6Ka1eumhT+mzX/2mJKnR8qSWV7rd2AC4w9UdJIuyGlf2DVDQrSjtGoMlnXddaTdFzHeunPuyK8UAVhq89fnm9RNaaHVv8OSXDJw+O696raqFVnP5a64HJ4DicvmCIihPu/Y4A0rQ0IvHLRXfsLwbJ4+50G6Kmu9cOPdlV5oB7CCHT76vdseTcmoVdZUMFDE4ARRTkS4oXLy3yi9B29Nz3FwMAcU1KO+6mMfCIt8hL6UZwPab/fLLPhY7nvVarVS0a3qDJIITgFuKkrPSKIXOajWXErRkubYKj/IJyrv77tiq4+98oD37j6nRdOuWjiiubBz4gSRP2zauzfuQUAKlGMAOmv0KKvtoNNv6o9ff1pdu/7AkLkYAuKUIOSvpUuisV0EoQUtGkVevUCydeff/b+/+g+wq6zuOf753d7NZkpiEJDKSBRETUdFkidGCUdoAo2BtUk1mxNJKO0wdW6w/KiVaZhwtM51pLEUdEIuWgtaihaigdBREKP6oQIBkifyQFG2zAQ0sm5jVzbKb++0f57nJZtnd3Lt77z3Pc+77NXNn7zn33L3f+9znfO/53vOcc3bs3qfLb3tEJZmGR8dd0SKRQzpq8cOdz7KeoqkKfxbio50hLTv+tfyC51111xNHnEWtyGdKBFA8qeeseg6FLtqZMlsFnxtSs2hup7oXduny2x7RgZGyfjty8AXLpHpIx2RYT5GHwhewR7so8aK5nXr/2mUveN6stjYuXAwAOann5Qy4OH2a+NyQoon6rSQdM6utkJdlYT1FHgo/hLiaX/H/6HdO1FV3PaHhUZ90GQBAc9VrKHSRTmzVSvjckKKJ+m1nu+nzf7xKpx4/v1DFq8R6inwUfg9stReE/tTGlVy4GAAiU4+h0FycPk18bkjRRP32UxtX6sxXvLiQfZf1FHkwdz/6UjlbvXq1b926dUb/o5qzGHKmQyAtZvaAu6/OO456qUeuw+TI8WnicyPXpajV+m2rvV80RrW5rvBDiCuqOSMkZ40EgOIix6eJzw0parV+22rvF/kq/BBiAAAAAEAxUMACAAAAAJJAAQsAAAAASAIFrLIDz7fv2stFlwG0LPIgAMSLHA0c1jIncZrMLdt2a9OWXnWUShopl7V5wwqt61mad1gA0DTkQQCIFzkaOFJL74HtHxzWpi29OjBS1v7hUR0YKevSLb38ugWgZZAHASBe5GjghVqmgJ1o6EXfwJA6Skc2QUeppL6BoWaHBwC5IA+mj6GFaCWt1t/J0cALtcQQ4omGXqxZtlj7hkb0/MHyEcuOlMvqXtiVU6QA0FzdC7s0Um6NPNg/OKy+gSF1L+wqzPUKGVqIVtKK/T32HF3EvIr4Fb6AHTv04oCyBPCRm7arZNKstjYdLJfV0Waa3d52KBmyAgJoFYvmdmrzhhW6dNxGYdHyYBE3fCf6frt0S6/WLFtcuM8PaNX+HnOOLmJeRRoKX8D2DQypvWRHzBs56JKk4dFRSVJnu3T1Bafp1OPnR5EQAKCZ1vUsQq0PfAAAE6lJREFU1Zpliwv7K3pRN3wrQwsr70nKhhb+9Kl9mt81q5CfJVrXZP29b2Co8P28ETl6pntOi5pXkYbCF7A7du/T4PDBKZeZ1dam+V2zWOEAtKxFczsLmwOLuuE70dDCA6MH9edf2qpZbW3sEUGhxD6UttHqmaPrsee0qHkVaSj0SZz6B4d1+W2PvGB+25E7ZFsqAQJAqynqhm9laOHsjpLmdbars70kd9fwqHO2UhTO+P4+u6MUzVDalNTrrMZFzatIQ6H3wPYNDKnNjqxW58xq0/t+9+W66q4n1GYlHfR4jiUAgFpxAo2ji/kYspkaO7Rw39CILv7Kg9o/PHro8cn2iNBvkKKZDqWNpd/nGUe99pwWOa8ifoUuYHfs3qffPH/k8OGD7jp2zixJJpkktwmfCwCx4wQa1Svycb6VoYX9g8NV7RGh3yBl0x1KG0u/zzuOeu45LXJeRdwKO4R4suHDf33OK3T5bY9oeLSs3z5/UMOjDLECkB4ubl+7RXM7tfKEBYXdyKpmiCX9Bq0oln4fQxz1Hopd9LyKOBV2D+xEQyTmdLbp2DmzOOgcQPI4gQYmcrQ9IvQbtKJY+n0scbDnFKnLpYA1s19I2i/poKRRd19d79eYaIjEwbKr54QFHHQOIHmcQAOTmWqIJf0GrSiWfh9LHFKxzzyP4stzCPFad+9pRPEqTT5EYtlx8ziLHYDkcUZOTAf9Bq0oln4fSxxA6szdm/+i2R7Y1e7+bDXLr1692rdu3Tqt15rsTG+xnIkOwPSZ2QON+hEsD9PJdeQyTAf9Ji3kuvqIpd/HEgcQm2pzXV7HwLqk283MJf2zu187fgEze6+kv5G0YMmSJdN+ocmGSDB0AkAMZprryGWYDvoNmq1e23UzEUu/jyUOIFV5DSFe4+6rJJ0n6WIzO3P8Au5+rbsvd/clJ5544oxfsH9wWNt37eVMiwCiUu9cNx65D0AMZprryGUAKnLZA+vuT4W/e8zsG5LeIOmeRr1e3tfcAoA8kPsAFAG5DMBYTd8Da2ZzzGxe5b6kt0ja0ajXi+GaWwDQbOQ+AEVALgMwXh5DiI+T9EMz2y7pPkm3uft3GvVilWtujVW55hYAFBW5D0ARkMsAjNf0IcTu/qSklc16vZiuuQUAzULuA1AE5DIA4+V5Hdim4JpbAFoRuQ9AEZDLAIyX12V0mmpdz1KtWbaYa24BaCnkPgBFQC4DMFZLFLAS19wC0JrIfQCKgFwGoKLwQ4gBAAAAAMVAAQsAAAAASAIFLAAAAAAgCRSwAAAAAIAkUMACAAAAAJJAAQsAAAAASAIFLAAAAAAgCRSwAAAAAIAkUMACAAAAAJJAAQsAAAAASAIFLAAAAAAgCRSwAAAAAIAkUMACAAAAAJJAAQsAAAAASAIFLAAAAAAgCRSwAAAAAIAkFLqA7R8c1vZde9U/OJx3KACAGpC/AdSKvAG0hva8A2iUW7bt1qYtveoolTRSLmvzhhVa17M077AAAEdB/gZQK/IG0DoKuQe2f3BYm7b06sBIWfuHR3VgpKxLt/TyixwARI78DaBW5A2gtRSygO0bGFJH6ci31lEqqW9gKKeIAADVIH8DqBV5A2gthSxguxd2aaRcPmLeSLms7oVdOUUEAKgG+RtArcgbQGspZAG7aG6nNm9YodkdJc3rbNfsjpI2b1ihRXM78w4NADAF8jeAWpE3gNZS2JM4retZqjXLFqtvYEjdC7tIYgCQCPI3gFqRN4DWUdgCVsp+kSOBAUB6yN8AakXeAFpDIYcQAwAAAACKhwIWAAAAAJAEClgAAAAAQBIoYAEAAAAASaCABQAAAAAkgQIWAAAAAJAEClgAAAAAQBIoYAEAAAAASaCABQAAAAAkgQIWAAAAAJAEClgAAAAAQBIoYAEAAAAASaCABQAAAAAkgQIWAAAAAJAEClgAAAAAQBIoYAEAAAAASaCABQAAAAAkoXAFbP/gsLbv2qv+weG8QwEAAHXG9zymg34DFEd73gHU0y3bdmvTll51lEoaKZe1ecMKretZmndYAACgDviex3TQb4BiKcwe2P7BYW3a0qsDI2XtHx7VgZGyLt3Syy9tAAAUAN/zmA76DVA8hSlg+waG1FE68u10lErqGxjKKSIAAFAvfM9jOug3QPEUpoDtXtilkXL5iHkj5bK6F3blFBEAAKgXvucxHfQboHgKU8AumtupzRtWaHZHSfM62zW7o6TNG1Zo0dzOvEMDAAAzxPc8poN+AxRPoU7itK5nqdYsW6y+gSF1L+wiOQEAUCB8z2M66DdAsRSqgJWyX9pITAAAFBPf85gO+g1QHIUZQgwAAAAAKDYKWAAAAABAEihgAQAAAABJoIAFAAAAACSBAhYAAAAAkAQKWAAAAABAEihgAQAAAABJoIAFAAAAACSBAhYAAAAAkAQKWAAAAABAEihgAQAAAABJoIAFAAAAACSBAhYAAAAAkAQKWAAAAABAEihgAQAAAABJoIAFAAAAACSBAhYAAAAAkARz97xjOCoze0bS/1a5+GJJzzYwnHqIPUbim7nYY4w9Pqm6GF/q7kuaEUwz1JjrpDQ+x6kQf76IP3/VvodWznWpfM7EWV/EWV+pxFlVrkuigK2FmW1199V5xzGV2GMkvpmLPcbY45PSiDFvqbcR8eeL+PNXhPfQaKm0EXHWF3HWVypxVoshxAAAAACAJFDAAgAAAACSUMQC9tq8A6hC7DES38zFHmPs8UlpxJi31NuI+PNF/PkrwntotFTaiDjrizjrK5U4q1K4Y2ABAAAAAMVUxD2wAAAAAIACooAFAAAAACShUAWsmZ1rZo+b2U4z+2je8UiSmf3CzB42s21mtjXMO9bM7jCzJ8LfhU2O6Toz22NmO8bMmzAmy3w2tGmvma3KKb5PmNnu0I7bzOxtYx77WIjvcTN7axPiO8HM7jKzR83sp2b2wTA/ijacIr6Y2nC2md1nZttDjJ8M819mZveGNvyamc0K8zvD9M7w+EmNjjF2Mea7qdSSd2JU63ofm1rXuViZWZuZPWRm3w7TycQf4/ZA7GLNc6nlgxTWGzNbYGY3m9ljoV3PiLE9zezD4TPfYWY3htyae3vW8h1rmaZu29dbYQpYM2uTdLWk8yS9WtK7zezV+UZ1yFp37xlz/aWPSrrT3ZdLujNMN9P1ks4dN2+ymM6TtDzc3ivpmpzik6QrQzv2uPt/SlL4jM+XdGp4zudCX2ikUUkfcfdXSTpd0sUhjljacLL4pHjacFjSWe6+UlKPpHPN7HRJ/xBiXC5pQNJFYfmLJA24+zJJV4blWlbk+W4y16v6vBOjWtf72NS6zsXqg5IeHTOdWvyxbQ9EK/I8l1o+SGG9+Yyk77j7KyWtVBZvVO1pZkslfUDSand/jaQ2ZdtPMbTn9Yp7276uClPASnqDpJ3u/qS7Py/pq5LW5xzTZNZLuiHcv0HSHzbzxd39HknPVRnTeklf8sxPJC0ws5fkEN9k1kv6qrsPu/vPJe1U1hcaxt2fdvcHw/39ypLsUkXShlPEN5k82tDdfTBMdoSbSzpL0s1h/vg2rLTtzZLONjNrZIyRSynfSao570RnGut9VKaxzkXHzLol/b6kL4ZpU0LxTyKJ/pOTaPNcSvkghfXGzF4k6UxJ/yJJ7v68u+9VhO0pqV1Sl5m1SzpG0tOKoD1j37avtyIVsEsl7Roz3aepN9qbxSXdbmYPmNl7w7zj3P1pKUuCkl6cW3SHTRZTTO36/jDU4boxw0hyjc+yoaynSbpXEbbhuPikiNowDGnaJmmPpDsk/Y+kve4+OkEch2IMj++TtKjRMUYspvVyJmLMhUdV5XofnRrXuRh9WtKlksphepHSij+V7YFYJJHnEsgHKaw3J0t6RtK/hqHOXzSzOYqsPd19t6R/lPR/ygrXfZIeUHztWRHddmm9FKmAnWhvTAzXCFrj7quU7a6/2MzOzDugGsXSrtdIermyoW9PS7oizM8tPjObK2mLpA+5+6+nWnSCeQ2PcYL4ompDdz/o7j2SupX90v6qKeKIpR/GgvbISQ3rfXRqXOeiYmZvl7TH3R8YO3uCRaOMP0h9e6DZov98Y88HCa037ZJWSbrG3U+T9BvFM/z6kPDD/3pJL5N0vKQ5ytbn8fJuz6OJsQ/UpEgFbJ+kE8ZMd0t6KqdYDnH3p8LfPZK+oWyj4VeVXfXh7578IjxkspiiaFd3/1XY+CpL+oIOD3HNJT4z61D2pfUVd/96mB1NG04UX2xtWBGGCd2t7DiiBWFYzvg4DsUYHp+v6oeZF1EU62UdxJgLJ1Xjeh+tKte52KyRtM7MfqFsKOlZyvYspRJ/StsDsYg6zyWSD1JZb/ok9bl7ZbTYzcoK2tja8xxJP3f3Z9x9RNLXJb1R8bVnRTTbpfVWpAL2fknLw5nAZik7qPrWPAMyszlmNq9yX9JbJO0IcV0YFrtQ0i35RHiEyWK6VdJ7whnLTpe0rzIcoZnGjc1/h7J2rMR3vmVnqX2ZsgPS72twLKbsOI1H3f2fxjwURRtOFl9kbbjEzBaE+13KvhQelXSXpI1hsfFtWGnbjZK+7+5J/VpYZ9Hlu2mKMRdOaBrrfVSmsc5Fxd0/5u7d7n6Ssv7+fXe/QInEn9j2QCyizXOp5INU1ht3/6WkXWZ2Sph1tqRHFFl7Khs6fLqZHRP6QCXOqNpzjCi2SxvC3Qtzk/Q2ST9TdlzPZRHEc7Kk7eH200pMyo4/uFPSE+HvsU2O60ZlQ0hHlP0Kc9FkMSkbZnB1aNOHlZ15LY/4vhxev1fZiveSMctfFuJ7XNJ5TYjvTcqGWvRK2hZub4ulDaeIL6Y2XCHpoRDLDkkfD/NPVlY875R0k6TOMH92mN4ZHj+5GetKzLfY8l0V8Vadd2K81brex3ardZ2L+Sbp9yR9O6X4Fen2QOy3WPNcivkg9vVG2eFNW0ObflPSwhjbU9InJT0W8uiXJXXG0J61fMcqh237et8svBEAAAAAAKJWpCHEAAAAAIACo4AFAAAAACSBAhYAAAAAkAQKWAAAAABAEihgAQAAAABJoIBFlMxskZltC7dfmtnuMdNvHbfsh8zsc3nFCgBTMbODIXftMLNvVa7HWsPzP2Fml4T7f2dm5zQmUgDImJmb2RVjpi8xs0/kGBJwCAUsouTu/e7e4+49kj4v6cpw/xplF+Me63xl178CgBgNhXz2GknPSbp4uv/I3T/u7t+rX2gAMKFhSe80s8V5B1JPZtaedwyYOQpYpOZmSW83s05JMrOTJB0v6Yc5xgQA1fpvSUslyczmmtmdZvagmT1sZusrC5nZZWb2uJl9T9IpY+Zfb2Ybw/2zzeyh8NzrKnkRAOpgVNK1kj48/gEzW2JmW8zs/nBbE+Y/bGYLLNNvZu8J879sZueY2almdl8YkdJrZsvN7CQze8zMbgjzbjazY8LzPh7+/w4zu9bMLMy/28w+bWY/Do+9IcyfE3Lh/SE3rg/z/9TMbjKzb0m6vSmth4aigEVS3L1f0n2Szg2zzpf0NXf3/KICgKMzszZJZ0u6Ncw6IOkd7r5K0lpJV4QNv9cpy22nSXqnpNdP8L9mS7pe0rvc/bWS2iX9RcPfBIBWcrWkC8xs/rj5n1E2Mu71kjZI+mKY/yNJaySdKulJSW8O80+X9BNJ75P0mTCibrWkvvD4KZKudfcVkn4t6S/D/Kvc/fVh9EqXpLePiWGOu78xLHtdmHeZpO+HuNZK+pSZzQmPnSHpQnc/a3pNgZhQwCJFN+rwMGKGDwOIXZeZbZPUL+lYSXeE+Sbp782sV9L3lO2ZPU7ZRt833P237v5rHS54xzpF0s/d/Wdh+gZJZzbwPQBoMSH/fEnSB8Y9dI6kq0Jeu1XSi8xsnqQfKMtDZyo75Ou1ZrZU0nPuPqhsBMrfmtkmSS9196Hw/3a5+4/C/X+T9KZwf62Z3WtmD0s6S1lhXHFjiPGe8PoLJL1F0kdDXHdLmi3pxLD8He7+3MxaBLGggEWKvinpbDNbJanL3R/MOyAAmMJQ2OPwUkmzdPgY2AskLZH0uvD4r5RtcEnS0UaVWCMCBYBxPi3pIklzxswrSTqjcq4Sd1/q7vsl3aPsB7g3Kysgn5G0UVlhK3f/d0nrJA1J+q6ZVfaGjs93HkaZfE7SxjDK5As6nB8nfI6yvLhhTFwnuvuj4fHfTO/tI0YUsEhO+BXvbmVDRtj7CiAJ7r5P2Z6MS8ysQ9J8SXvcfcTM1iorcKVsI/AdZtYV9mr8wQT/7jFJJ5nZsjD9J5L+q7HvAECrCXst/0NZEVtxu6T3VybMrCcsu0vSYknL3f1JZecnuUShgDWzkyU96e6fVbbndkX4Fyea2Rnh/rvD8yrF6rNmNldZITzWu8L/fJOkfSG/flfSX405Vva0mb17xIoCFqm6UdJKSV/NOxAAqJa7PyRpu7LDH74iabWZbVW2N/axsMyDkr4maZukLQobf+P+zwFJfybppjC8rqzsjO0AUG9XKCtMKz6gLHf1mtkjyo5trbhXUuXQhh8oOzSicqLNd0naEYb4vlLZ8GRJelTSheFwimMlXePue5XtdX1Y2ci7+8fFNGBmP1aW9yrF9eWSOiT1mtmOMI0CMs59AwAAAKDZwtUkvh1O1FTtc+6WdIm7b21QWIgce2ABAAAAAElgDywAAAAAIAnsgQUAAAAAJIECFgAAAACQBApYAAAAAEASKGABAAAAAEmggAUAAAAAJOH/Aeqxyg7URfJMAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x29a050432e8>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rwvhBsSz-kg9",
        "colab_type": "text"
      },
      "source": [
        "## Questions About the Advertising Data\n",
        "\n",
        "Let's pretend you work for the company that manufactures and markets this widget. The company might ask you the following: On the basis of this data, how should we spend our advertising money in the future?\n",
        "\n",
        "These general questions might lead you to more specific questions:\n",
        "1. Is there a relationship between ads and sales?\n",
        "2. How strong is that relationship?\n",
        "3. Which ad types contribute to sales?\n",
        "4. What is the effect of each ad type of sales?\n",
        "5. Given ad spending in a particular market, can sales be predicted?\n",
        "\n",
        "We will explore these questions below!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W8hvXN90-kg_",
        "colab_type": "text"
      },
      "source": [
        "## Simple Linear Regression\n",
        "\n",
        "Simple Linear regression is an approach for predicting a **quantitative response** using a **single feature** (or \"predictor\" or \"input variable\"). It takes the following form:\n",
        "\n",
        "$y = \\beta_0 + \\beta_1x$\n",
        "\n",
        "What does each term represent?\n",
        "- $y$ is the response\n",
        "- $x$ is the feature\n",
        "- $\\beta_0$ is the intercept\n",
        "- $\\beta_1$ is the coefficient for x\n",
        "\n",
        "Together, $\\beta_0$ and $\\beta_1$ are called the **model coefficients**. To create your model, you must \"learn\" the values of these coefficients. And once we've learned these coefficients, we can use the model to predict Sales!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "F7m-aEGz-khB",
        "colab_type": "text"
      },
      "source": [
        "## scikit-learn"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nLEZ__Kc-khD",
        "colab_type": "text"
      },
      "source": [
        "### Introduction\n",
        "\n",
        "Since its release in 2007, scikit-learn has become one of the most popular open source Machine Learning libraries for Python. scikit-learn provides algorithms for Machine Learning tasks including classification, regression, dimensionality reduction, and clustering. It also provides modules for extracting features, processing data, and evaluating models.\n",
        "\n",
        "Conceived as an extension to the SciPy library, scikit-learn is built on the popular Python libraries NumPy and matplotlib. NumPy extends Python to support efficient operations on large arrays and multidimensional matrices. matplotlib provides visualization tools, and SciPy provides modules for scientific computing.\n",
        "\n",
        "scikit-learn is popular for academic research because it has a well-documented, easy-to-use, and versatile API. Developers can use scikit-learn to experiment with different algorithms by changing only a few lines of the code. scikit-learn wraps some popular implementations of machine learning algorithms, such as LIBSVM and LIBLINEAR. Other Python libraries, including NLTK, include wrappers for scikit-learn. scikit-learn also includes a variety of datasets, allowing developers to focus on algorithms rather than obtaining and cleaning data.\n",
        "\n",
        "Licensed under the permissive BSD license, scikit-learn can be used in commercial applications without restrictions. Many of scikit-learn's algorithms are fast and scalable to all but massive datasets. Finally, scikit-learn is noted for its reliability; much of the library is covered by automated tests."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uWSP8TOr-khG",
        "colab_type": "code",
        "outputId": "04546fd6-56ed-40c4-8347-30292da646f4",
        "colab": {}
      },
      "source": [
        "# create X and y\n",
        "feature_cols = ['TV']\n",
        "X = data[feature_cols]\n",
        "y = data.Sales\n",
        "\n",
        "# follow the usual sklearn pattern: import, instantiate, fit\n",
        "from sklearn.linear_model import LinearRegression\n",
        "lm = LinearRegression()\n",
        "lm.fit(X, y)\n",
        "\n",
        "# print intercept and coefficients\n",
        "print(lm.intercept_)\n",
        "print(lm.coef_)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "7.032593549127693\n",
            "[0.04753664]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d3ElHCQT-khP",
        "colab_type": "text"
      },
      "source": [
        "## Interpreting Model Coefficients\n",
        "\n",
        "How do we interpret the TV coefficient ($\\beta_1$)?\n",
        "- A \"unit\" increase in TV ad spending is **associated with** a 0.047537 \"unit\" increase in Sales.\n",
        "- Or more clearly: An additional $1,000 spent on TV ads is **associated with** an increase in sales of 47.537 widgets.\n",
        "\n",
        "Note that if an increase in TV ad spending was associated with a **decrease** in sales, $\\beta_1$ would be **negative**."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BTf1iSi8-khQ",
        "colab_type": "text"
      },
      "source": [
        "## Using the Model for Prediction\n",
        "\n",
        "Let's say that there was a new market where the TV advertising spend was **$50,000**. What would we predict for the Sales in that market?\n",
        "\n",
        "$$y = \\beta_0 + \\beta_1x$$\n",
        "$$y = 7.032594 + 0.047537 \\times 50$$"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3k8sWTJx-khS",
        "colab_type": "code",
        "outputId": "55700a5f-3431-4ef8-e50d-32fb2640fc3f",
        "colab": {}
      },
      "source": [
        "# manually calculate the prediction\n",
        "7.032594 + 0.047537*50"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "9.409444"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Simjthgu-khc",
        "colab_type": "text"
      },
      "source": [
        "Thus, we would predict Sales of **9,409 widgets** in that market."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XgUKdTRt-khe",
        "colab_type": "code",
        "outputId": "a4fdf2f5-97c5-4be8-a0e5-3534904dbcf5",
        "colab": {}
      },
      "source": [
        "# you have to create a DataFrame since the Statsmodels formula interface expects it\n",
        "X_new = pd.DataFrame({'TV': [50]})\n",
        "X_new.head()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>TV</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>50</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   TV\n",
              "0  50"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WvQsyWTy-kho",
        "colab_type": "code",
        "outputId": "f74ac18e-d361-40d8-969f-876b29bd2ae0",
        "colab": {}
      },
      "source": [
        "# use the model to make predictions on a new value\n",
        "lm.predict(X_new)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([9.40942557])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W_kpWg_H-kh0",
        "colab_type": "text"
      },
      "source": [
        "## Plotting the Least Squares Line\n",
        "\n",
        "Let's make predictions for the **smallest and largest observed values of x**, and then use the predicted values to plot the least squares line:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zMUfuHmG-kh3",
        "colab_type": "code",
        "outputId": "93ce5c81-cd3f-454d-eece-39d5388f2484",
        "colab": {}
      },
      "source": [
        "# create a DataFrame with the minimum and maximum values of TV\n",
        "X_new = pd.DataFrame({'TV': [data.TV.min(), data.TV.max()]})\n",
        "X_new.head()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>TV</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0.7</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>296.4</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "      TV\n",
              "0    0.7\n",
              "1  296.4"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NY4naSjt-kiI",
        "colab_type": "code",
        "outputId": "f0bb2790-5b52-41c9-b3ab-2b9bcbe1d038",
        "colab": {}
      },
      "source": [
        "# make predictions for those x values and store them\n",
        "preds = lm.predict(X_new)\n",
        "preds"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([ 7.0658692 , 21.12245377])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DOdN7V4u-kiT",
        "colab_type": "code",
        "outputId": "cdc737a5-e916-4490-9f42-9f3d3b546b19",
        "colab": {}
      },
      "source": [
        "# first, plot the observed data\n",
        "data.plot(kind='scatter', x='TV', y='Sales')\n",
        "\n",
        "# then, plot the least squares line\n",
        "plt.plot(X_new, preds, c='red', linewidth=2)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x29a071cb400>]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 11
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEKCAYAAAAfGVI8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJztnXmcVPWV6L+nqqsXaDYbRVZBEbVhEJWISkRAx41FDbhBayaZyOQ9nYkvi2RmnnEc3sxEjea9xCxjEicxjQuCCoIxUQEV4hI0gDQoMoqyRaRZG+jq7qrf++NWdVVX13JruVW3qs738+lPd//qVt3zu/fWOb/fOed3fmKMQVEURSlfPIUWQFEURSksaggURVHKHDUEiqIoZY4aAkVRlDJHDYGiKEqZo4ZAURSlzFFDoCiKUuaoIVAURSlz1BAoiqKUORWFFsAO/fv3N8OHDy+0GIqiKEXFO++8s88Yc2Kq44rCEAwfPpx169YVWgxFUZSiQkQ+sXOcuoYURVHKHDUEiqIoZY4aAkVRlDJHDYGiKEqZo4ZAURSlzFFDoCiKkgOaW/xs2HGQ5hZ/oUVJm6JIH1UURXEzS9fvYv6Sjfg8HtqDQe6fNZaZ4wYXWizb6IxAURQlC5pb/MxfspHW9iBH/B20tge5a8nGopoZqCFQFEXJgp0HjuPzdFWlPo+HnQeOF0ii9FFDoCiKkgVD+tXQHgx2aWsPBhnSr6ZAEqWPGgJFUZQsqKut4v5ZY6n2eehVVUG1z8P9s8ZSV1tVaNFso8FiRVGULJk5bjATR/Zn54HjDOlXU1RGANQQKIqi5IS62irbBqC5xe8qo6GGQFEUJY+4MdVUYwSKoih5wq2ppmoIFEVR8oRbU03VECiKouQJt6aaqiFQFEVxmHAdIsCVqaYaLFYUpahxOgMn28+PFxxeO39q52cCbNhxsKAZRGoIFEUpWpzOwMn286ODw61YLqG7lmxk7fypnD20r2syiBxzDYnIUBFZJSJbRKRJRL4Rav8XEdklIutDP1c7JYOiKKWL0xk4ufj8ZMFhN2UQORkj6AC+ZYw5C7gAuF1E6kOv/dAYMy7084KDMiiKUqI4nYGTi89PFhx2UwaRY4bAGLPHGPNu6O8jwBageAp0K4riapzOwMnF5yerQ+SmDKK8ZA2JyHDgHOCtUNMdIrJRRB4VkX75kEFRlMLg1M5dThd7y9Xnzxw3mLXzp9L4tQmsnT+1MwbgpmJ1Yoxx9gQitcCrwL8ZY54RkQHAPsAAC4CBxpivxnnfPGAewLBhw8775JNPHJVTUZTck49gqNuzhgr5+SLyjjFmfMrjnDQEIuIDlgO/N8Y8FOf14cByY8yYZJ8zfvx4s27dOkdkVBTFGZpb/Ey8byWt7RH3R7XPw9r5UwueN18u2DUETmYNCfArYEu0ERCRgVGHXQdsckoGRVEKh5uCoYWiWDa0d3IdwUTgFuA9EVkfavsn4GYRGYflGtoO/J2DMiiKUiDcFAy1Q65dNG5ZI2AHxwyBMWYNIHFe0nRRRSkDwsHQu2KUoRvdQrlW2okWkk0c2d+V/deVxYqiOEYx7NzlhNIOu8XCnwcRt5gbr4EaAkVRMsKuKyWdnbsKgRNKO1dusXztZKaGQFEUID2lU0z+71Q4EcvIhVssn9fY8XUEuUDTRxXFWdJROqWYFrps/a5uSjsXSjfTEX2urrHd9FGdEShKmZOuj7zY/N92cCqWkalbLN/XWA2BopQ56SqdYksLtYubYhn5vsa6Q5milDnpKp1C18gplkVa2ZDva6wxAkVRMvKR5yujJZpSClLbIdtr7IpaQ7lCDYGiOE8hFHs6lGKQ2mk0WKwoSlq4yUcej1IMUrsFjREoilIUlGqQ2g2oIVAUJSnpBmeLdSOackZdQ4pSxqSKC6QbnHU6mFsMtYuKETUEilKmpFLa6S40i3f8dxZvyHnFTbfHMooRdQ0pShkSrbSP+DtobQ9y15KNXdw56W4sE+94f4fh8bc+tS2TEy6lclh3kC06I1CUMsROBk66wdkh/WpoCwS6tT+8ahtzJgxLOop3yqVUbusOMkVnBIpShthR8ukGZ+tqq7hjyund2iu9ybentDM7yQSnPrcU0RmBopQhdsskpxucnTNhGA+v2oa/I2JkUqV4Zro+IFWgW9cd2EcNgaKUKXaVvJ3gbLRSfmB2enX4M1kfYMflo+sO7KOGQFHKmFxk4MRTymvnT00rxfP2ySN5eNWHVHq9KY2H3WymYtozudCoIVAUJWMSKeW186dy9tC+Kd8fbURAmDfp1JSB5XRcPsW27qBQ9Z7UEChKCVAoBZKNHz6eEfnJaivDKBmZlM12uwGAwmY4adaQohQ5S9fvYuJ9K2n45VtMvG8ly9bvytu5s/HDp7tOIUwplpoodIaTzggUpYhJd/VvrsnGD5+NESk2l08qEs2sdm//C3Uj+kPPno6eXw2BohQxbkiRzFQpZxvMLRaXjx2ijaIv0M7kj95h1pbVjLnvT/DjH8Nttzl6fjUEilLExBtVtwUCHDreRnOLP2+KMlOlnK4RcdvmObmSp65nJb88zc+OH/+Cq7a8Tt/jR6wXRGDLlhxJmxjdoUxRipzobSaPt3cgIlRXeEuupILbykXkRJ7334eFC62fjz/ubP5gwAiWjpnCuO98ncuv+ELGMupWlYpSRjS3+GnafZjbHlvXZVVvqWzl6LZtKrOS57PP4MknobERovRaYNBgHj3lQpaceQnvnzQivc9MgG5VqShlRF1tFX1qfFR6PV0MQamUVHBDLCQreVpa4LnnLOX/0ksQduf17g2zZ0NDA5tGjOVH/7WOI/4Oe5+ZQ9QQKEqJUMolFdLtWza+ezvvtSVPRwe8/LKl/J99Fo4ds9p9Ppg+HRoarN811nuGtPgLdv8cW0cgIkNFZJWIbBGRJhH5Rqj9BBF5SUQ+DP3u55QMilJOuDW/Phf7AaTTt2zWVdh9b0J5elZa7p4774TBg+Gqqyz//7FjcNFF8NOfwp49sHQpXH99pxFIt4+5xrEYgYgMBAYaY94VkV7AO8C1wN8A+40x3xeR7wL9jDHzk32WxggUxT5uyqzJdYA3Vd+y8d1n8t6wPMMO/YV+zy22Rv8ffBA5YNQouOUWmDMHTj01J31Mh4LHCIwxe4A9ob+PiMgWYDBwDTA5dNhvgNVAUkOgKIp93JJf78Rit1R9s+O7T6Ro0/b7NzdT9/TT1DU2wtq1kfaTToKbb7ZcP+edZ6WA5rCPTpCXGIGIDAfOAd4CBoSMBMaYPSJyUj5kUBQlv6SrWHMxEk7lu082Q7Hl9z9+HJYvt9w9L7wA7e1We48ecN11lvK/7DKoKK7wq+PSikgtsAS40xhzWGxaRxGZB8wDGDYseREqRSl23OTOyRXpBHhz5UJKtlo51Qwl4Xt7+GDVKsvts3gxHD5snczjgSuusJT/tddCbW36F8klOGoIRMSHZQQWGmOeCTV/JiIDQ7OBgcDeeO81xjwCPAJWjMBJORWlkLhtoVQumXrGSbyw6S+d/98wfkg3Q5drF1Ki1cp2ZijR7z1l93/T94kfwYzHYefOyAnOO89S/jfdBCefnLZ8bsQxQyDW0P9XwBZjzENRLy0Dvgx8P/R7qVMyKIrbKXTROKdYun4Xdy3egL+j6xhu0bqdfOPSUdn55m0Qz89ua4aycyd1Tzxh+f03boy0Dx8Oc+daP2edlZFMbsbJGcFE4BbgPRFZH2r7JywDsEhE/hb4FLjeQRkUJS3y7aIp1EIpJ/sZNm6xRgDi9y1f6x8Sun4CrfBfj1uun1WrIJxJ2a8f3HijpfwvushyBZUoTmYNrQESBQQudeq8ipIphXDRFGIRmNP9jGfcwsTrWz63lAy7fnZ9dojh76yh9398E5Ytg9ZW64CqKpgxw3L9XHml9X8ZUFyhbUVxiEK5aFIFN3M9as+kn+nKEc+4AVRVJF4gZacKabpydDveGHjjDeoaG6lbtAiamyMHT55sKf9Zs6Bv6i02Sw01BIpCYWvZxFOCTo3a0+1nJnLEGre2QIA7ppyeci/iZPnz6coRffyQzz/l/3Y0ccYrz8NHH0UOGj3aWux1881Q5pmJaggUhcLX6YlWgk7OTtLpZzZy5HIHsXTlaG7xc99jr3HTxtVcu3kV4/Z8GHlx0CBrlW9DA4wdm/Zir1JFDYGikL2fOpduHCdnJ+n0M1s5crVC1rYcR4/C0qX4fvlfvLZ6JRXGOv5IZQ2vnPVF/mr+7Zx2w3TwerOWqdRQQ6AoITIdxebajeP07MRuP1PJka8Mq6RydHTAypVWxs8zz8DRo/QG2j1eXjrtfJ6rn8LLI89HetSwdsZUNQIJUEOgKFGkO4p1qp6O01k08foZq9iTyZHPDKtucgQC/Ge9UPe9f4QnnoC/RBasceGF0NDAK2ddzJ0rd+LzeBAHs5BKBd2hTFGyYMOOgzT88q0um4n0qqqg8WsTOHtodtkn+VzTkEyxx8pRqN3CDjR9QOtvfsuJSxdTsTWqwufpp1s+/7lz4bTTOptLsWxHuhS8+qiiuJVcKohs3Dip5MhXFUo7NXgyWQmck+u8fz88/TQ0NtJvzZpI+4knWiUeGhrgC1+IG/R1SxXWYkANgVJW5Nqlkakbx031hdINCtsxfln1r7UVVqyw/P4rVkQqfNbUWMXdGhrgr//a2ulLyQlqCJSywam0zHSDzG6rL5TurCaV8cuof8EgvP66pfyffhoOHbLaPR5L6Tc0WGWee/XKTaeVLqghUMoGp9My7X6G2zZiz2RWk8z4pdW/piZL+S9cCDt2RNrPPTdS4XPgwJz1VeMG8VFDoJQNhV405jY5oskkdTaR8UvZv927rWyfxkZYvz5y0CmnRCp81tdn1Z94uMkd5zZKt5yeosRQyM3B3ShHPLnOHto3YznCm9QD3fr30BUjqFv8hLV715Ah8O1vW0agb1+YNw9ee80q//Bv/+aIEYh2Vx3xd9DaHuSuJRtpbvHb6lOq44odnREoZUUuSx+EycTd4IQcqXDSLRJvtL32mxdz6LnlDF7xDFU/eD5S4bOyEqZPt1w/V1+dlwqfmbjjymkGoYZAKTvsLKaySzbKItP0xkxkdVKpdQkOmwDn7P6Aw1/7Kf22v0ld877IgZMmWcp/9myr1n8eSdcd57aAvtOoIVDKnkyVZCGURSayOi3nzgPHGbF/N1dseIVrm1Yz/OCeyIv19ZbynzPHigEUiHQD4m4L6DuNGgKlrMlGSWarLDKpr5+JrI4ptc8/h6eeYvDPf8XvmiJB389qT2DpWZewYuxUHv3h16jrVZ35OXJIOu44Nwb0nUQNgeJa8pHql42SzEZZZDKyz1TWnCq1Y8dg6VIr3fPFFyEQoD/QUlnDi6Mu4tnRU3hj2F8R9HjpVVXBzoOtrjEEYN8dl89d09yAGgLFleQrUJeNksxUWWQ6ss9UVjtyxjO6nW29K6n70x/ht7+1Kny2tFhv8no5PPVyFvQ9l+eHj6fV11XhF/sIuhAB/UKhhkBxHfn0vWc78stEWWQ6ss9G1mRyxjO6xhh+/dPnuHbTKgZuWg0t+yMfNmGC5fe/4Qbae/Th+ZgCdJB8W8piolzqFakhUFxHvgN12Y780lUW2cxCspE1UbZUtNEdfGgvH/6vp5i5aRXP7vu087hP+g3kxK9/lR5f+bJV7TP8mZDRtpSKu1BDoLiOQgTq8jnyy3YWkktZdx44Tp3/KBdvfJXrNq9mwo5Nna811/Tm+bMmsbR+MttGjKbxtgviltYuJxdKqaKGQHEd5RCoK7jy9PvhhRc489ePsWrFcioD1n4KrRWVvDzqQpaNmczKYefQ4bVURLUxSQ1xubhQ0qGY6hqpIVBcSVhRNu0+BAijB/UutEg5J+/KMxiENWusjJ9Fi+DgQaoAI8LaEeN4YeylvDjqQu6ZcwHTgdeWbKSmRA2x0xTbqmQ1BIprWbNtX9ZfpmIalTnG5s2RCp+fRvz+jBsHDQ3ITTdxZp/+1B44zjejrlOyGUuhr2uhz5+MVMkObpRdDYHiSnKROVRso7KcsmdPpMLnn/8caR86NFLhc8yYzuY66HZdE81YCn1dF775Cfc+34TP6yFgjOvua7Jkh1wMbpxADYHiSnKxarecasUAcOQIPPuspfxfecVyBQGB3r1pv24W1V/5Mlx8sbXZS4YU+roufPMT/vk5K6DdFgjk/fx2SJTs0LPS69pnUg2B4kqyzRzKVQqqG6fxXWhvh5despT/c8/B8eMABH0+dl781zxw4nj+OGoCRz0V3N9nJDOzMALgTGqv3Wvc3OLn3uWbu7V7RVxVAyhRssPRtoBr6xelbQhExAPUGmMOOyCPUkJko0SzzRzKRQpqoV0gCTEG/vQnS/k/+aRV8yfEvnMm8KOB5/Ny/cXs9vawGgNAIJiT0WeuU3vTucY7Dxyn0iu0dXRtbw+4bwVzvKyw5ha/a+sX2TIEIvI48HWsR+odoI+IPGSMecBJ4ZTiJRdKNNMUy7ABuntaPQtWbM7IkBTaBRKX//5vK+Db2AgffhhpP/NMuOUWDlwziy8++VG3Vb5hwqNPoGAGOpp0r/GQfjV0BE239ntmjC74iDoesTEWN6dF250R1BtjDovIXOAFYD6WQUhoCETkUWA6sNcYMybU9i/AbUB4CPNPxpgXMpRdcSm5VKLppljGGqC7p9czZlCftJWea8oQ79tnpXo2NsIbb0TaBwywSjs3NMA554AIn+44iM+zvYvM0bQHg2zadYgbH3mjIAY6lnSvcbQi9XqE9oDhnhn1zJ1QuPLW6VLw9SMJsGsIfCLiA64FHjbGtItId9PclV8DDwOPxbT/0Bjzg/TEVIqJQinReAZowfLNrJ0/Ne3zxnOB+ANWwM9xjh+HZcss5f/ii9Bh+UJMz54cuGI6Fbc20HvalVDR9esbT2aAap8HYwx3T69nwfLNBTHQ8cjEzeRWRWqHaFdpvBXahcRu5Og/ge1AT+A1ETkFSBojMMa8BuxPdoxSmhSqlnvYAEUT7Q5Jh/Dos9rnodpnfaYYw/SH17Bs/a6cyNuFQMDK9PnKV6zR/k03wfLlVjzgqqtY9+8/5tzbf8slY77C+eu8LNv0WVKZe1VVUOEBn1fwiADC/pa2nF2fXFBXW8Xd0+qp9Ao9K722926uq81ub+V8Eb3f8dL1u5h430oafvkWE+9b6cwzlAW2ZgTGmB8BP4pq+kREpmR4zjtE5FZgHfAtY8yBDD9HyTG5ypAplC801wZo5rjB1A/szdU/eh0Af8BAwOQuVmAMbNxojfwffxx27468dv75ltvnxhtp7tGHhvtW0ipB8Fuzg0QyRFZkH+a2x9bh7wjSHkqzfHjVNqDrRL6Qwcql63exYMVmKis8tIXcPK4IxueAaBdlWyBA0EB7wLgn3hSD3WDxAODfgUHGmKtEpB64EPhVmuf7GbAA62lcADwIfDXBOecB8wCGDRuW5mmUdIkX3M1mCl6IKbwTBuhoW4CqCi9tgUiqis/joWn3IfrUVGbWtx07LMXf2AibIkXeOPVUS/nPnQujRnU279xxMG1fep8aHyJd20Xg9smn85PV2woerIx244VZsHwzV44+2TXKMVPiuShjcUvaaBi7MYJfA/8F/HPo/63AU6RpCIwxnfNZEfkFsDzJsY8AjwCMHz8+VTxCyYJ4D+43F63H6/FQ6c08qFiIQmS5NkDxZhmtHQFue2wdlV6v/Wtz8CAsWWIp/9WrO5uDJ9Sxf/q1+G69hT5TJxGrvZtb/Bw63ta5eCpMqpF8z0pvt+yh1vYgV405mTkThhXcx+6aYLwDxOtbLG5JGw1j1xD0N8YsEpF/BDDGdIhIINWbYhGRgcaY8M7W1wGbkh2v5Id4D25HEDqCQfwd7pzKJiOXBih2ltEWCBIIBvEHwN+R3E2D3w+/+52V8vn889b/ANXVMHMmb150Fbd9dgL4qmhffYz763Z3MSjRs7SggQoP1PgqbI3kj7YFqPKK5c4KUeUVjrYFGDmg8JVCS3lP4Hh9q/DQbWBV6HsQjV1DcFRE6gg5GEXkAuBQsjeIyBPAZKC/iOwE7gEmi8i40OdsB/4uM7GVXJIo2yQat4/WnFwBHD3LOHS8ndsXvssRf1dXUee1CQbhj3+0Rv6LFsGBUAhMBKZOtVw/X/oSzd5q/ua+lbQG4/v9483Sqio8/GTuuYwe1DtlH4f0q0E8AlGGQDziGkXr5pz6bEnUNzdnO9k1BN8ElgGnicha4ERgdrI3GGNujtOcbkxByQPdR72R4FYYp0ZruVDg+VgBHJ5lJFodOmzvp/CfP7BG/9u3R14cO9ZS/jffDEOGdDan8vvHm6VVeIW9h1ttleQuBkVbzKmgqUjUN7f2UYyx534XkQrgDECAD4wx7U4KFs348ePNunXr8nW6siVaKa/dtq+bEsm1cs2FAm9u8TMxZs/cap8no7UDdlm2fhd3LdnIyccOcvl7q/mfO9+k75aNkQOGDIlU+Pyrv8pI7nivg+X7T6fiputrJSmOIiLvGGPGpzou6YxARL6U4KVRIoIx5pmMpFNcSbRv3enRWq5WH+c96NjSwsxNq7jyzcfwrXwFCc8OeveG66+3Rv+TJqWs8JlqxB67ivao3wrJHW1Lr+Km7hym2CGVa2hGktcMoIaghHFSieRKgecl6NjRAS+/bPn9n30Wjh2jEsDngxkzLOU/bRrUpHfOVMY2/Pqq9/dyz7KmTiMA7o/ZKMVFUkNgjPlKvgRR8ocb3AW5UuCO+cKNgXXrIhU+9+6NvDZxoqX8r78e6uqyOk0qY1tXW8WUM0/ify/tmmBXKhk2ijuwXYZaRKYBo4HqcJsx5l+dEEpxDreUVs6lAs+pG+ujj2DhQgKP/RbvtqgKn2ecYSn/OXOshV95pBgCv5mQaEDihoFKuWErWCwiPwd6AFOAX2JlDL1tjPlbZ8Wz0GBxbsh1YDUXX1hXfOmbmyMVPv/4x87mfT37sWL0JIb/wzwumXNVt8Ve+cYV1ypHJBqQuGWgUirkJFgcxUXGmLEistEYc6+IPIjGB4qOXAZWc/WFzVUcIm0lefy4VdStsRFeeKGzwmegpgfLT5vAkrMms3b4OAIeL9XvC2uPthVc+ZZK4DdRokD9wN62EwhKySi6AbuGIFye8JiIDMKqKjrCGZEUp8iVX95tm7bYNkrBILz6qqX8Fy+Gw6ECuh4Pn100mQdO/AKrz7iQfVLZ5W0amM0tiQYk623WVNJZQ+6xW4Z6uYj0Be7H2pDmY+BJx6RSHCG2TLHdsr+xxCv37BGhaXfSxeaOEG2Ujvg7aG23tmRsbvFHDnrvPZg/H045xVrd++ijcPgwx8aew9H7fsD+rR9xydS7WHzmJd2MALg/MBtd7rgYzpNoQDJuaN+UAxVb91tJm1TrCL4A7DDGLAj9Xwu8B7wP/NB58ZRck4vAarwv8rE2qxDbA7PPzuvoLNHo8rPN26hbvcIa/b/3XuQNw4fzwWXXcGdFPTtPOoX2I0Fu394Wt0hYj0ovwdDiLTfNBqLdImu27cvL6DiXo/BEwe+RA3qlDIqXarG6Qru6kgaLReRd4DJjzH4RmYQ1C/h7YBxwljEmaZmJXKHB4vxh94Fctn4X31m8sbMoXRinV/XGEh0A7+U/ypUfrGXW5tVM+PQ9JPxs9+sHN94IDQ00jz2Pifev6hIwr6rwAAZ/R1SBtgrhF7eOZ/SgPo73JR0l0LXOvVUAL/oWOHH9nVq9nUnWUD5XkudLOTvp6spVsNhrjAnvMnYj8IgxZgmwRETWZyukkn+SPdzpPJAzxw2mb49Kvv7bdzjWXriFTnWVwm/q/sL+Rx5l6ta3qAqEKp9UVcHMmVaZh6uugkrL5ROvxk+l18O8Sad2q9M/adRJjsufzjUvVJ17p0bhiYLfidrDz254y00nU2nzFYdwS7wtpSEQkQpjTAdwKaGNYmy+V3EZyR7uTB7I0YN6E3Rwx6uERssYayP3xkZ46ikm7LfGKkaE9kmX4PvyrTBrFvTp0+0zE/mn50wYlvc6/ele80LVuXdDyejYZ/fuafWMGdzH1eVP7OAWV1eqYPETwKsishQrc+h1ABEZSYoy1Iq7SBVky2S/31wFn+MRd4/XDz6A730PRo60Vvf+7Gewfz+MGQP33Yd88gm+V1fDV78a1wikkrmuNrIXbj4CsOle83gK2ecVqiok59c/mvA1q6oQevi8VFWIrfPk6hrGe3bvfb6JnpVeR5RlLve+ToUbjCykLjHxbyLyCjAQ+IOJBBQ8WLECpUhINfLI9IF0ojhd9Be/9mgzs7a8zvBH74TdWyMHDRpkuX0aGqxSz2mQSuZ8uQXSveaFrHNvffHFqj1sUi+sy+U1jPfstgUMV/94DT+Ynft7k0/l7JZV4yndO8aYN+O0bY13rOJeUj3csdUu2wOGu6fX23ogc73QafeufVzbtJor17/CF7f/mQpjyR2o7YX3+tmW8r/kEvB6Mz5HMj90vtwCmSiBQtS5D1+T6MSAZNck19cw0cZJbR1Bx+7N7ZNH8vCqbXnZUcwN+zKon79MsKN0Zo4bzJHWDu59vgmf18OC5ZvpVVWRn3TQjg545RVobGTMs8/y/aNHAWj3eHnptPNZMXYqf//g/6LFG9ow3ut1JKsj3z7bTJRAvlcYp3tNcnkNOwPE0+q59/km2gJdY1K5vjfRMxkwzJt0KnMmDHP8ehd61bgaghLBjlJMpXSaW/wsWLGZtoDp3Czd0QwGY+Ddd62g7xNPwGefAZb3Yf/Y8/jxoAm8POYSPq/uxQ3jhzDtV+90GrEbzhvCond25tx9UwifbbQSCN/HnpVejrYF8hq4TvRcpHtNEh3fs9LLhh0Hbfcp1r307cvP4AcvbaUtamaS6+SE2JnMT1ZvY86EYbbfX6xlL9QQuBw7D1c6/thkI49EwbCcj4Y//hgef9wyAO+/H2k//XTL7TN3Liecdhp3tPi5NqQUpz+8pssX9LE3PwXoVqsmW+VZSJ9t+D6aoMEfMFT7rICl0yUUUj0/6V6TeMffMH4I0x9e07kV6h1TTk860o6nlB96eSv3TK9nwQpnUkezmckUe9kLNQQuxs7DlUt/bM9Kb7etEVvbrZFc1uzfD08/bSncj0p1AAAYFElEQVT/NWsi7SeeaO3n29AA48eDiGX8QiPHs4f2ZUOc3P94XP2j16mq8Gb9RSyEzzb6PoYJ/+3krMzu85PuNYk+Pp4hf/ClrTy8ahsPJAj2JlLKYwb3Ye38qY7cm0xng25ZC5ANaghcit2HK9NRTLyZxtG2AJVe6eKHrfJKl52x0qK1FVasoO3Xj1Hx+9/haQ8t9qqpgeuus5T/ZZdZO32FiGf8Jo7sHzdY2OVUIaXZFrCqiGb7RXSDHz6MkzGKeOcN142KXVCX7jUJH79hx0FMsHsFA3+SYG8ypezUvcl0NuiWtQDZoIbApdh9uDIZxSSaaWzadahbME48kp4PNhiE116LVPg8dIhKICAe1px6Lj2+civnfuOr0KtXt7cmMn5r50+N62pYtM6KEfgDQcRY7pRk1yoRbvDtJsqMAWdjFPmoG9Wz0tvl3kST6D4VykWXyWzQLWsBskENgUux+3Cl+4VJVgt+wYrN3Y6/e5q9FFI2bbKU/+OPw44dnc1NJ4/kmfrJLDtrEp/XnkC138NaqSTeBo/JjF+8L+g3Lh3VxfVAlLKx+0V0i283+j7GixE4pQDD542tG+XvMDlzbxxtC1Dt83RzO0Ly+1SotMpMZj5uWAuQDWoIXEo6D1c6X5h0asH3rPIyZnD8FboA7NplZfs0NsKGDZH2U06BuXN5f+oMbnrtEEf8HV3OlWikbmetQ2x2Tbi/dq9V9PsAV/l2Y/3q+coacrpuVCJFb2eFcqHTKu3ihrUA2aCGIM9EpwfuPtQKmIQVLtN5uOx+YRIp2+F1PfB3dI0FBIKm+5f48GF45hlL+a9caaWAglXh84YbrNW+EyeCx8OJLX7aV6/sdq50V87aXfmb6lrFvu/2ySNd59stlOJzsm5U7H1tCwS5Y8rIvOTn55NiMVrxsLVncaEplTLUsemBYXxe4cHr81fHf9n6XV397aGcfLCCrlVeQTwScZO0t8Pvf28p/6VLrSAwWBU9Z8ywgr5XXWVV/ExxrvBnpio1nOsyxPHeV1UhgHRxieS7jLabSHSvcoUbYjHlRq73LFayJF56YJj2gOE7i/PnkkiU2hfGiLDi9omM/LgJ7vgPePJJa4P3MJdcYin/WbNo9vWwvtztQHv3L3q8kbqdvPVkax0yGcXHe1+l1xu3/HS5KqmZ4wZTP7A363ccZNzQvowc0D2gnw3FPGIuddQQ5IlUJYS9Hulc0JWPUVP4s1e9v5cKT6SI2Ij9u7jhg9cY2ng7fPJx5A319XDLLTBnDgyzVlpaCv1Nq18dAYwx1Pgquin3WN9+omC1HZ94phkahSo/XUyjYLcEzpX8o4YgTyRLDwTLH79p1yFufOSNvHwRw196rwjVB5qZ9f7rXNe0inF7ouoJDhxoKf6GBjj7bJCIwUi0SUo4MJwo6BrPIJqgVUmyymtvZXQmGRqp3hfPBWXtwSyMHtQ7IyVeTIq1FBZFKZmjhiBPxEsPDOPzCt+bYe26lI8vYnOLn3uefJvLN7/BtU2rmPTxu50VPo9W1tB8xXSG/cM8mDIlYYXPVDOcRO6aeAbRuhams4ZMqn5nmqFh931L1+/i209voD10jyo88NAN49JS4sWmWEthUZSSOY4ZAhF5FJgO7DXGjAm1nQA8BQwHtgM3GGMOOCWD24j1zUdnDe08cByvdK3znvMvYiAAK1ci//koa59fSs82yxXV7vGyeuT5VP/NrZz+tTkMO6lfyo9KNcNJ5K6JHZn7OwJ4PNIlRmGn35n6m1O9r7nFz12LN3YaAYCOIHxn8YakZZdjjUuxKdZSWBSlZI6TM4JfAw8Dj0W1fRd4xRjzfRH5buj/+Q7K4DqiFVF0MO7FTX/pVsrheHtH9l9EY2D9+kiFzz17OCH00ruDzuDZ0VNYcebFHOvTj7X/MBXAVoXIWIUeL0ZgZ0TfuRgsikIqoJ0HjuP1dN94xSvxlXgi90+xKdZSWBSlZI5jhsAY85qIDI9pvgaYHPr7N8BqyswQxKO5xc+/Lm/q1i6SeieohGzfbq3yXbgQNketGB45EhoaeOWcS7n9T0e6fOnXbNuXlk871tUC9gPd0QbRTQpoSL8aAnHq4gRMdyWeyv3jpn7ZodgXRSmZk+8YwQBjzB4AY8weETkp1RvKAcst5AG6zgh83jRdCQcO0PJbS/nXvv1GZ3Owrj+em2/i0JduYPtpYxhyQg8ura1i7dSuq2zDefbp+LRjXS3Rf8e6TBJl0LhJAdXVVvHA7LF8KyZG8MDss20FvqPdP27ql100xbM8cW2wWETmAfMAhg2ztzFEoUik4OymDg7pV0PAdPe3x13ZG4vfDytWQGMjgeUrqG1vA6C1opKXTr+A3519Ka+NOIcvnT+cRSt34lv9dpfRfliueKWes/FpR7tM2gJBLh5Zx6tbP6eywkvAmLTWDuSbsAJPlTVkx/3jpn4pSiLybQg+E5GBodnAQGBvogONMY8Aj4C1sjhfAqZLIh9xupvF3Dh+aOdmK2CtK3hgdgJXQjBo1fRvbLRq/B88CIAgvH7KOJ4bPYXfj7qQlqoeoePjb+QSPdqPp9TaApn5tOO5TF5+/3MA2tvysPNZDqirrepWhjneMcXm/lGUeOTbECwDvgx8P/R7aZ7Pnzapyh0kWhyVTupgc4u/s8RDmAoPTBzZv+uBmzdbyn/hQvg0YjQ45xx2TZ/FLS0j+KgySZG4KDwITbsPM2nUiUBEqUW7RALBIGu37Us79z1VaimAV8S1GTTpUIzuH0WJxcn00SewAsP9RWQncA+WAVgkIn8LfApc79T5c0GqUX3T7kN44qR8xqvkme5m35Ver3X84WarxENjI/z5z5E3DRtmFXibOxdGj6a6xc/u+1ZCnBIW8TjWHq45H+nTxJH9iU6Y6QhmNnJPlVoK0J7hbMONqPtHKXY8Tn2wMeZmY8xAY4zPGDPEGPMrY0yzMeZSY8zpod/7nTp/tkSP9o/4O2htt3ZTam7xA5aRuO2xdRyLSflsDwYZN7RvVpt99/QfY9qf/8BZX54FQ4fCt75lGYE+feC22+DVV619f//932H0aCAyoq/2eehVVYHPK1R4oFdVBdU+D7deOCxUZC1CeIeocJ92HjhOZcwCsrABS4ewLFUViR+ve2aMVuWpKC7BtcHiQpMsIwSsOvb+jq6hi6oKD/fPGsvIAb3S3uz7gZlnsfzB33DNplVM+eBNajr8oZP64JprrDIPV18N1dUJZU6VznnZWQP4euO7XYxX9Ewll7nvYVkef+tT/t8rWwkX+PQK/Ou1Y5g74ZS0P1NRFGdQQ5CAZEoxnpGorvDw3avO7PTr2/IdGwNvvw2Njcx48klm7NsXee3iiy3lP3s2nHBC9/cmIFk65+hBfQiaxDXnEwU/wd4is3iy/P2lpzNnwrCs6/YoiuIcuh9BEpat38V3Fm/AKx4CJti5h2u82vYAtVVeOoLdUyO7sW2bFfBtbLT+DnPWWZbynzMHhg+3JaPdFNXwcZt2H2LB8s1Js5miPzOdRWbFVGlTUcoB3Y8gB1gmUkAAE/GvR4+cPcCxkEFo8SdJjfz8c1i0yFL+b74ZaT/55EiFz3HjulT4TIXdFNXY4+6eVs+YwX0SKuzwrCKdwmnFVGlTUZSuOBYsLnbCStDfEeRYW6BbYHXmuMHcPa2etjjlCDpjCceOwVNPWbt4DRoEd9xhGYGePeHWW+EPf4CdO+HBB+Gcc5IageYWPxt2HOw8f6pgdmw/oo9bsGKzrVF72AUWt28pzhFPFkVR3InOCBKQqnxAc4ufBSs20xHoagg8wQDjt23gjO82wrLnoKXFesHrtYK9DQ0wc6ZlDGwSb7R9Sl1PWymq2VTBtBs8LrZKm4qidEUNQQJSKcEuys8Y6vd+zLVNq7hmy6sMaInKij3/fEv533gjnJR+aaVE7pnGr56PP5BaSWeTCWR35WyxVdpUFKUraggSEK0EvSK0ByzfenRJhv4H/kLDxlVc27SKM/ZFrfQ99VRL+c+dC6NGZSVHolW6c375FmFHUrXPct/EU9LZlkGwk/2kpRYUpbjRrKEULHzzE+5dvplKr9ARNDx0+Slc/cEfraDvq692HnegpjcHZ1zHiDv/Di64IK2gbzISZShFU+kVXviHi5NuNp6PjB7NGlIUd6FZQzkgHAeg1c9FH63j2qZVXPYfb0PA2peX6mr802awa9qX6HPNNEackFgRZ0q3Hb0CQcR03eqyqsLbbVObeJ/jtHLWUguKUpyoIUhEMMiBF1/hX3/3Ey5veo2+rVbQN4hw4MKL6Tfvq/ClL1HVuzenZvDx6Yye4+7oFWUI1B+vKEo2qCGIZcuWzgqfIz/5hJGh5s0njeDZ+iksq5/EwX4n8cC4sczs3TujU2SSc+/WHb0URSl+NEYAsGdPpMLnu+9G2ocM4cPLZvJN32jeO6FrbZxqn4e186emrYDj+fwz+Sz1xyuKkgqNEaSipYUjTyyCxoXUrlmNhNIfg71747n+eivrZ9IkTvd4+M7WvQmLtUHXwm6pFHSucu7VH68oSq4oL0PQ0QEvvQSNjXQ88yy9Wi1F3uapYPWoCbw4diovj/wC/+fG8V1cNYmKtW3adYgbH3mj00Vzw/ghLFq3M6nLR3PuFUVxG6XvGjIG1q2z3D5PPgl7I7tjvj2knudGT2HFGV/kUE0k4yeeq2bhW59w7/Ob8XmFQNBw9/R6FizfnDStM5HLZ9n6Xd18/FqXR1GUXKOuofZ2+P73LQOwdWuk/Ywz2DNzNl9pPZ33e/SP+9ZYV83S9btCFTuF9o4g98wYzZhBfVJux5jI5aPbGyqK4iZKt+hcRYU1A9i6FQYMgDvvtGYGW7ZQ+b272d4ncbmHaFdNdImHo20B2gKGBSs207PSm3o7xiQun7raKs4e2leNgKIoBad0DYGINSN48UWrwucPfwjnnQci1NVWcfe0eiq9Qs9KLxUe8Hmlc1vH6HTMnQeOY2IqjJqg4WhboHNryJ5V3ngScPf0elX0iqK4ntJ1DYFV/jkOS9fvYsGKzVRWeGgLGO69ZgxXjj45rqumZ6W3yypeAH/A0LPS2+niWfX+Xu5Z1tRldW/PSi9jBvVxpl+Koig5pHRnBAmIdvW0+AO0dQRZsHwzQFxXze5D8TduD7fX1VYx5cyTCMQE3QPGaCaQoihFQdkZArubrURIVDyu+45l1T5PXPeSoiiKmylt11Ac0s3jHz2oNxUe6Ih6S4XHao9GM4EURSlWym5GkMno/RuXjqLSCz18XqoqhIduGJewLr9mAimKUmyU3YwA7I/eo4vDiXj4H5NPY86EYaroFUUpKcpuRhCmrraKIf1q2HngeNxN1mM3ZPd3BPnJ6m0FkFRRFMVZynJGAIlLQYeLxh063qYbsiuKUhaUpSFobvFz1+KN+Du6bgh/pLWDBSs24/N4aAsEiFlHpsXhFEUpScrOEDS3+HnwD1vxd3TNHPJ6hHufb6ItYDqNQ4UHqio8VHp1AxhFUUqXsjIES9fv6pwJxNLeYaxKpVF4RfjFrePpU+PTlFBFUUqWggSLRWS7iLwnIutFxMGtxyKEg7/xjADAnAlDaIt5yR8wDOpTrSmhiqKUNIXMGppijBlnp1Z2Loi3ojhMVYUw9cwBVPu6vl7t83SpH6QoilKKlE36aLwVxWBVHf3ejNGMTlAgToPDiqKUOoUyBAb4g4i8IyLz4h0gIvNEZJ2IrPv888+zPmHsimKfV/AKVHo9LFi+mbXb9mm9IEVRypKCbFUpIoOMMbtF5CTgJeDvjTGvJTo+q60qY2hu8dO0+xC3PbYOf0ek7+FtJQGtF6QoSklgd6vKgswIjDG7Q7/3As8C5+fr3HW1VfSpqaTS23UzmejFYhocVhSlnMi7IRCRniLSK/w3cDmwKZ8ypFuBVFEUpZQpxIxgALBGRDYAbwMrjDEv5lMA3T9AURQlQt4XlBljPgLOzvd5Y9H9AxRFUSzKamVxLHW1VWoAFEUpe8pmHYGiKIoSHzUEiqIoZY4aAkVRlDJHDYGiKEqZU1aGoLnFz4YdB+NuTakoilKulE3WUKKtKRVFUcqdspgRxG5E39oe5K4lG3VmoCiKQpkYgnh7EYRrCymKopQ7ZWEItLaQoihKYsrCEGhtIUVRlMSUTbBYawspiqLEp2wMAWhtIUVRlHiUhWtIURRFSYwaAkVRlDJHDYGiKEqZo4ZAURSlzFFDoCiKUuaUtCHQInOKoiipKdn0US0ypyiKYo+SnBFokTlFURT7lKQh0CJziqIo9ilJQ6BF5hRFUexTkoZAi8wpiqLYp2SDxVpkTlEUxR4lawhAi8wpiqLYoSRdQ4qiKIp91BAoiqKUOWoIFEVRyhw1BIqiKGWOGgJFUZQyR4wxhZYhJSLyOfBJBm/tD+zLsTiFpJT6U0p9gdLqTyn1BUqrP+n25RRjzImpDioKQ5ApIrLOGDO+0HLkilLqTyn1BUqrP6XUFyit/jjVF3UNKYqilDlqCBRFUcqcUjcEjxRagBxTSv0ppb5AafWnlPoCpdUfR/pS0jECRVEUJTWlPiNQFEVRUlCyhkBErhSRD0Rkm4h8t9DypIuIbBeR90RkvYisC7WdICIviciHod/9Ci1nIkTkURHZKyKbotriyi8WPwrdq40icm7hJO9Ogr78i4jsCt2f9SJyddRr/xjqywcickVhpE6MiAwVkVUiskVEmkTkG6H2ors/SfpSlPdHRKpF5G0R2RDqz72h9hEi8lbo3jwlIpWh9qrQ/9tCrw/P6MTGmJL7AbzAfwOnApXABqC+0HKl2YftQP+YtvuB74b+/i5wX6HlTCL/JOBcYFMq+YGrgd8BAlwAvFVo+W305V+Ab8c5tj70vFUBI0LPobfQfYiRcSBwbujvXsDWkNxFd3+S9KUo70/oGteG/vYBb4Wu+SLgplD7z4H/Efr7fwI/D/19E/BUJuct1RnB+cA2Y8xHxpg24EngmgLLlAuuAX4T+vs3wLUFlCUpxpjXgP0xzYnkvwZ4zFi8CfQVkYH5kTQ1CfqSiGuAJ40xfmPMx8A2rOfRNRhj9hhj3g39fQTYAgymCO9Pkr4kwtX3J3SNW0L/+kI/BpgKLA61x96b8D1bDFwqIpLueUvVEAwGdkT9v5PkD4cbMcAfROQdEZkXahtgjNkD1hcAOKlg0mVGIvmL9X7dEXKVPBrlpiuqvoRcCedgjTyL+v7E9AWK9P6IiFdE1gN7gZewZi0HjTEdoUOiZe7sT+j1Q0BduucsVUMQzyIWW3rURGPMucBVwO0iMqnQAjlIMd6vnwGnAeOAPcCDofai6YuI1AJLgDuNMYeTHRqnzVV9itOXor0/xpiAMWYcMARrtnJWvMNCv3PSn1I1BDuBoVH/DwF2F0iWjDDG7A793gs8i/VAfBaekod+7y2chBmRSP6iu1/GmM9CX9gg8Asi7oWi6IuI+LAU50JjzDOh5qK8P/H6Uuz3B8AYcxBYjRUj6Csi4R0lo2Xu7E/o9T7Yd2N2UqqG4E/A6aFIeyVWEGVZgWWyjYj0FJFe4b+By4FNWH34cuiwLwNLCyNhxiSSfxlwayg75QLgUNhF4VZifOTXYd0fsPpyUyibYwRwOvB2vuVLRsiH/CtgizHmoaiXiu7+JOpLsd4fETlRRPqG/q4BLsOKe6wCZocOi7034Xs2G1hpQpHjtCh0lNypH6xMh61Y/rV/LrQ8acp+KlZmwwagKSw/lu/vFeDD0O8TCi1rkj48gTUlb8catfxtIvmxprc/Cd2r94DxhZbfRl9+G5J1Y+jLODDq+H8O9eUD4KpCyx+nP1/Ech9sBNaHfq4uxvuTpC9FeX+AscCfQ3JvAr4Xaj8Vy2BtA54GqkLt1aH/t4VePzWT8+rKYkVRlDKnVF1DiqIoik3UECiKopQ5aggURVHKHDUEiqIoZY4aAkVRlDJHDYGi2EBE6qIqWf4lprLlFTHH3ikiPy2UrIqSLmoIFMUGxphmY8w4Yy39/znww9DfP8NasBjNTVhrDxSlKFBDoCjZsRiYLiJV0Fn4bBCwpoAyKUpaqCFQlCwwxjRjrei8MtQUrgmvKzWVokENgaJkzxNE3EPqFlKKDjUEipI9z2FtCHIuUGNCG6UoSrGghkBRssRYO0qtBh5FZwNKEaKGQFFywxPA2VjboipKUaHVRxVFUcocnREoiqKUOWoIFEVRyhw1BIqiKGWOGgJFUZQyRw2BoihKmaOGQFEUpcxRQ6AoilLmqCFQFEUpc/4/bJ9DiHopCz4AAAAASUVORK5CYII=\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x29a071cb828>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pNfJvfPo-kig",
        "colab_type": "code",
        "outputId": "613d59bb-9e40-4fe2-98df-678be1de6f76",
        "colab": {}
      },
      "source": [
        "# print the p-values for the model coefficients\n",
        "lm.pvalues"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Intercept    1.406300e-35\n",
              "TV           1.467390e-42\n",
              "dtype: float64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bBcoZeCm-kiq",
        "colab_type": "text"
      },
      "source": [
        "## How Well Does the Model Fit the data?\n",
        "\n",
        "The most common way to evaluate the overall fit of a linear model is by the **R-squared** value. R-squared is the **proportion of variance explained**, meaning the proportion of variance in the observed data that is explained by the model, or the reduction in error over the **null model**. (The null model just predicts the mean of the observed response, and thus it has an intercept and no slope.)\n",
        "\n",
        "R-squared is between 0 and 1, and higher is better because it means that more variance is explained by the model. Here's an example of what R-squared \"looks like\":"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NCf6qe_u-kis",
        "colab_type": "text"
      },
      "source": [
        "<img src=\"images/08_r_squared.png\">"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iPcTLzJb-kiv",
        "colab_type": "text"
      },
      "source": [
        "You can see that the **blue line** explains some of the variance in the data (R-squared=0.54), the **green line** explains more of the variance (R-squared=0.64), and the **red line** fits the training data even further (R-squared=0.66). (Does the red line look like it's overfitting?)\n",
        "\n",
        "Let's calculate the R-squared value for our simple linear model:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pnMYD-y3-kiw",
        "colab_type": "code",
        "outputId": "5a98160e-799c-484f-caa6-6c0251b5a612",
        "colab": {}
      },
      "source": [
        "# print the R-squared value for the model\n",
        "lm.rsquared"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.611875050850071"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tDcVNSKj-ki9",
        "colab_type": "text"
      },
      "source": [
        "Is that a \"good\" R-squared value? It's hard to say. The threshold for a good R-squared value depends widely on the domain. Therefore, it's most useful as a tool for **comparing different models**."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6WkLu94--ki_",
        "colab_type": "text"
      },
      "source": [
        "## Multiple Linear Regression\n",
        "\n",
        "Simple linear regression can easily be extended to include multiple features. This is called **multiple linear regression**:\n",
        "\n",
        "$y = \\beta_0 + \\beta_1x_1 + ... + \\beta_nx_n$\n",
        "\n",
        "Each $x$ represents a different feature, and each feature has its own coefficient. In this case:\n",
        "\n",
        "$y = \\beta_0 + \\beta_1 \\times TV + \\beta_2 \\times Radio + \\beta_3 \\times Newspaper$\n",
        "\n",
        "Let's use Statsmodels to estimate these coefficients:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZBpBgBjt-kjC",
        "colab_type": "code",
        "outputId": "0a32feb0-19c7-4373-af49-300be89dfa3b",
        "colab": {}
      },
      "source": [
        "# create X and y\n",
        "feature_cols = ['TV', 'Radio', 'Newspaper']\n",
        "X = data[feature_cols]\n",
        "y = data.Sales\n",
        "\n",
        "lm = LinearRegression()\n",
        "lm.fit(X, y)\n",
        "\n",
        "# print intercept and coefficients\n",
        "print(lm.intercept_)\n",
        "print(lm.coef_)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "2.9388893694594067\n",
            "[ 0.04576465  0.18853002 -0.00103749]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XNK9lF-Y-kjJ",
        "colab_type": "text"
      },
      "source": [
        "How do we interpret these coefficients? For a given amount of Radio and Newspaper ad spending, an **increase of $1000 in TV ad spending** is associated with an **increase in Sales of 45.765 widgets**.\n",
        "\n",
        "A lot of the information we have been reviewing piece-by-piece is available in the model summary output:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tkqevaSa-kjM",
        "colab_type": "code",
        "outputId": "ba47f1f5-43a7-498b-e8bc-912b62b0047b",
        "colab": {}
      },
      "source": [
        "lm = smf.ols(formula='Sales ~ TV + Radio + Newspaper', data=data).fit()\n",
        "lm.conf_int()\n",
        "lm.summary()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<table class=\"simpletable\">\n",
              "<caption>OLS Regression Results</caption>\n",
              "<tr>\n",
              "  <th>Dep. Variable:</th>          <td>Sales</td>      <th>  R-squared:         </th> <td>   0.897</td>\n",
              "</tr>\n",
              "<tr>\n",
              "  <th>Model:</th>                   <td>OLS</td>       <th>  Adj. R-squared:    </th> <td>   0.896</td>\n",
              "</tr>\n",
              "<tr>\n",
              "  <th>Method:</th>             <td>Least Squares</td>  <th>  F-statistic:       </th> <td>   570.3</td>\n",
              "</tr>\n",
              "<tr>\n",
              "  <th>Date:</th>             <td>Wed, 23 May 2018</td> <th>  Prob (F-statistic):</th> <td>1.58e-96</td>\n",
              "</tr>\n",
              "<tr>\n",
              "  <th>Time:</th>                 <td>16:17:10</td>     <th>  Log-Likelihood:    </th> <td> -386.18</td>\n",
              "</tr>\n",
              "<tr>\n",
              "  <th>No. Observations:</th>      <td>   200</td>      <th>  AIC:               </th> <td>   780.4</td>\n",
              "</tr>\n",
              "<tr>\n",
              "  <th>Df Residuals:</th>          <td>   196</td>      <th>  BIC:               </th> <td>   793.6</td>\n",
              "</tr>\n",
              "<tr>\n",
              "  <th>Df Model:</th>              <td>     3</td>      <th>                     </th>     <td> </td>   \n",
              "</tr>\n",
              "<tr>\n",
              "  <th>Covariance Type:</th>      <td>nonrobust</td>    <th>                     </th>     <td> </td>   \n",
              "</tr>\n",
              "</table>\n",
              "<table class=\"simpletable\">\n",
              "<tr>\n",
              "      <td></td>         <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n",
              "</tr>\n",
              "<tr>\n",
              "  <th>Intercept</th> <td>    2.9389</td> <td>    0.312</td> <td>    9.422</td> <td> 0.000</td> <td>    2.324</td> <td>    3.554</td>\n",
              "</tr>\n",
              "<tr>\n",
              "  <th>TV</th>        <td>    0.0458</td> <td>    0.001</td> <td>   32.809</td> <td> 0.000</td> <td>    0.043</td> <td>    0.049</td>\n",
              "</tr>\n",
              "<tr>\n",
              "  <th>Radio</th>     <td>    0.1885</td> <td>    0.009</td> <td>   21.893</td> <td> 0.000</td> <td>    0.172</td> <td>    0.206</td>\n",
              "</tr>\n",
              "<tr>\n",
              "  <th>Newspaper</th> <td>   -0.0010</td> <td>    0.006</td> <td>   -0.177</td> <td> 0.860</td> <td>   -0.013</td> <td>    0.011</td>\n",
              "</tr>\n",
              "</table>\n",
              "<table class=\"simpletable\">\n",
              "<tr>\n",
              "  <th>Omnibus:</th>       <td>60.414</td> <th>  Durbin-Watson:     </th> <td>   2.084</td>\n",
              "</tr>\n",
              "<tr>\n",
              "  <th>Prob(Omnibus):</th> <td> 0.000</td> <th>  Jarque-Bera (JB):  </th> <td> 151.241</td>\n",
              "</tr>\n",
              "<tr>\n",
              "  <th>Skew:</th>          <td>-1.327</td> <th>  Prob(JB):          </th> <td>1.44e-33</td>\n",
              "</tr>\n",
              "<tr>\n",
              "  <th>Kurtosis:</th>      <td> 6.332</td> <th>  Cond. No.          </th> <td>    454.</td>\n",
              "</tr>\n",
              "</table>"
            ],
            "text/plain": [
              "<class 'statsmodels.iolib.summary.Summary'>\n",
              "\"\"\"\n",
              "                            OLS Regression Results                            \n",
              "==============================================================================\n",
              "Dep. Variable:                  Sales   R-squared:                       0.897\n",
              "Model:                            OLS   Adj. R-squared:                  0.896\n",
              "Method:                 Least Squares   F-statistic:                     570.3\n",
              "Date:                Wed, 23 May 2018   Prob (F-statistic):           1.58e-96\n",
              "Time:                        16:17:10   Log-Likelihood:                -386.18\n",
              "No. Observations:                 200   AIC:                             780.4\n",
              "Df Residuals:                     196   BIC:                             793.6\n",
              "Df Model:                           3                                         \n",
              "Covariance Type:            nonrobust                                         \n",
              "==============================================================================\n",
              "                 coef    std err          t      P>|t|      [0.025      0.975]\n",
              "------------------------------------------------------------------------------\n",
              "Intercept      2.9389      0.312      9.422      0.000       2.324       3.554\n",
              "TV             0.0458      0.001     32.809      0.000       0.043       0.049\n",
              "Radio          0.1885      0.009     21.893      0.000       0.172       0.206\n",
              "Newspaper     -0.0010      0.006     -0.177      0.860      -0.013       0.011\n",
              "==============================================================================\n",
              "Omnibus:                       60.414   Durbin-Watson:                   2.084\n",
              "Prob(Omnibus):                  0.000   Jarque-Bera (JB):              151.241\n",
              "Skew:                          -1.327   Prob(JB):                     1.44e-33\n",
              "Kurtosis:                       6.332   Cond. No.                         454.\n",
              "==============================================================================\n",
              "\n",
              "Warnings:\n",
              "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
              "\"\"\""
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tDe03Byv-kjU",
        "colab_type": "text"
      },
      "source": [
        "What are a few key things we learn from this output?\n",
        "\n",
        "- TV and Radio have significant **p-values**, whereas Newspaper does not. Thus we reject the null hypothesis for TV and Radio (that there is no association between those features and Sales), and fail to reject the null hypothesis for Newspaper.\n",
        "- TV and Radio ad spending are both **positively associated** with Sales, whereas Newspaper ad spending is **slightly negatively associated** with Sales. (However, this is irrelevant since we have failed to reject the null hypothesis for Newspaper.)\n",
        "- This model has a higher **R-squared** (0.897) than the previous model, which means that this model provides a better fit to the data than a model that only includes TV."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "86IyinkK-kjb",
        "colab_type": "text"
      },
      "source": [
        "## Feature Selection\n",
        "\n",
        "How do I decide **what features has to be included** in a linear model? Here's one idea:\n",
        "- Try different models, and only keep predictors in the model if they have small p-values.\n",
        "- Check whether the R-squared value goes up when you add new predictors.\n",
        "\n",
        "What are the **drawbacks** in this approach?\n",
        "- Linear models rely upon a lot of **assumptions** (such as the features being independent), and if those assumptions are violated (which they usually are), R-squared and p-values are less reliable.\n",
        "- Using a p-value cutoff of 0.05 means that if you add 100 predictors to a model that are **pure noise**, 5 of them (on average) will still be counted as significant.\n",
        "- R-squared is susceptible to **overfitting**, and thus there is no guarantee that a model with a high R-squared value will generalize. Below is an example:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-O-dnsWS-kje",
        "colab_type": "code",
        "outputId": "ada3a455-d995-405f-bb3c-1a06d9a6fa0e",
        "colab": {}
      },
      "source": [
        "# only include TV and Radio in the model\n",
        "lm = smf.ols(formula='Sales ~ TV + Radio', data=data).fit()\n",
        "lm.rsquared"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.8971942610828957"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pL0o33Sf-kjm",
        "colab_type": "code",
        "outputId": "034a9239-79d2-4464-b202-adfb9986e4bd",
        "colab": {}
      },
      "source": [
        "# add Newspaper to the model (which we believe has no association with Sales)\n",
        "lm = smf.ols(formula='Sales ~ TV + Radio + Newspaper', data=data).fit()\n",
        "lm.rsquared"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.8972106381789522"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KFJrEti3-kjv",
        "colab_type": "text"
      },
      "source": [
        "**R-squared will always increase as you add more features to the model**, even if they are unrelated to the response. Thus, selecting the model with the highest R-squared is not a reliable approach for choosing the best linear model.\n",
        "\n",
        "There is alternative to R-squared called **adjusted R-squared** that penalizes model complexity (to control for overfitting), but it generally [under-penalizes complexity](http://scott.fortmann-roe.com/docs/MeasuringError.html).\n",
        "\n",
        "So is there a better approach to feature selection? **Cross-validation.** It provides a more reliable estimate of out-of-sample error, and thus is a better way to choose which of your models will best **generalize** to out-of-sample data. There is extensive functionality for cross-validation in scikit-learn, including automated methods for searching different sets of parameters and different models. Importantly, cross-validation can be applied to any model, whereas the methods described above only apply to linear models."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gNqbiXwH-kjy",
        "colab_type": "text"
      },
      "source": [
        "## Handling Categorical Predictors with Two Categories\n",
        "\n",
        "Up until now, all the predictors have been numeric. What if one of the predictors was categorical?\n",
        "\n",
        "Let's create a new feature called **Size**, and randomly assign observations to be **small or large**:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "p6nNxX9l-kj0",
        "colab_type": "code",
        "outputId": "ebdd501e-0e96-4f72-dbbd-8a05e0b530b3",
        "colab": {}
      },
      "source": [
        "import numpy as np\n",
        "\n",
        "# set a seed for reproducibility\n",
        "np.random.seed(12345)\n",
        "\n",
        "# create a Series of booleans in which roughly half are True\n",
        "nums = np.random.rand(len(data))\n",
        "mask_large = nums > 0.5\n",
        "\n",
        "# initially set Size to small, then change roughly half to be large\n",
        "data['Size'] = 'small'\n",
        "data.loc[mask_large, 'Size'] = 'large'\n",
        "data.head()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>TV</th>\n",
              "      <th>Radio</th>\n",
              "      <th>Newspaper</th>\n",
              "      <th>Sales</th>\n",
              "      <th>Size</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>230.1</td>\n",
              "      <td>37.8</td>\n",
              "      <td>69.2</td>\n",
              "      <td>22.1</td>\n",
              "      <td>large</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>44.5</td>\n",
              "      <td>39.3</td>\n",
              "      <td>45.1</td>\n",
              "      <td>10.4</td>\n",
              "      <td>small</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>17.2</td>\n",
              "      <td>45.9</td>\n",
              "      <td>69.3</td>\n",
              "      <td>9.3</td>\n",
              "      <td>small</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>151.5</td>\n",
              "      <td>41.3</td>\n",
              "      <td>58.5</td>\n",
              "      <td>18.5</td>\n",
              "      <td>small</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>180.8</td>\n",
              "      <td>10.8</td>\n",
              "      <td>58.4</td>\n",
              "      <td>12.9</td>\n",
              "      <td>large</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "      TV  Radio  Newspaper  Sales   Size\n",
              "1  230.1   37.8       69.2   22.1  large\n",
              "2   44.5   39.3       45.1   10.4  small\n",
              "3   17.2   45.9       69.3    9.3  small\n",
              "4  151.5   41.3       58.5   18.5  small\n",
              "5  180.8   10.8       58.4   12.9  large"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ssOJg41B-kkA",
        "colab_type": "text"
      },
      "source": [
        "For scikit-learn, we need to represent all data **numerically**. If the feature only has two categories, we can simply create a **dummy variable** that represents the categories as a binary value:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "t9xfVaGo-kkC",
        "colab_type": "code",
        "outputId": "964b321c-03b6-48bc-d629-b551eb7a4b6c",
        "colab": {}
      },
      "source": [
        "# create a new Series called IsLarge\n",
        "data['IsLarge'] = data.Size.map({'small':0, 'large':1})\n",
        "data.head()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>TV</th>\n",
              "      <th>Radio</th>\n",
              "      <th>Newspaper</th>\n",
              "      <th>Sales</th>\n",
              "      <th>Size</th>\n",
              "      <th>IsLarge</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>230.1</td>\n",
              "      <td>37.8</td>\n",
              "      <td>69.2</td>\n",
              "      <td>22.1</td>\n",
              "      <td>large</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>44.5</td>\n",
              "      <td>39.3</td>\n",
              "      <td>45.1</td>\n",
              "      <td>10.4</td>\n",
              "      <td>small</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>17.2</td>\n",
              "      <td>45.9</td>\n",
              "      <td>69.3</td>\n",
              "      <td>9.3</td>\n",
              "      <td>small</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>151.5</td>\n",
              "      <td>41.3</td>\n",
              "      <td>58.5</td>\n",
              "      <td>18.5</td>\n",
              "      <td>small</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>180.8</td>\n",
              "      <td>10.8</td>\n",
              "      <td>58.4</td>\n",
              "      <td>12.9</td>\n",
              "      <td>large</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "      TV  Radio  Newspaper  Sales   Size  IsLarge\n",
              "1  230.1   37.8       69.2   22.1  large        1\n",
              "2   44.5   39.3       45.1   10.4  small        0\n",
              "3   17.2   45.9       69.3    9.3  small        0\n",
              "4  151.5   41.3       58.5   18.5  small        0\n",
              "5  180.8   10.8       58.4   12.9  large        1"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ObgQBijV-kkI",
        "colab_type": "text"
      },
      "source": [
        "Let's redo the multiple linear regression and include the **IsLarge** predictor:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bBXBvuRp-kkJ",
        "colab_type": "code",
        "outputId": "52bd04a3-d2f0-4d6e-b263-f1a8f35c8540",
        "colab": {}
      },
      "source": [
        "# create X and y\n",
        "feature_cols = ['TV', 'Radio', 'Newspaper', 'IsLarge']\n",
        "X = data[feature_cols]\n",
        "y = data.Sales\n",
        "\n",
        "# instantiate, fit\n",
        "lm = LinearRegression()\n",
        "lm.fit(X, y)\n",
        "\n",
        "# print coefficients\n",
        "zip(feature_cols, lm.coef_)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<zip at 0x29a071d45c8>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bas66Kcv-kkO",
        "colab_type": "text"
      },
      "source": [
        "How do we interpret the **IsLarge coefficient**? For a given amount of TV/Radio/Newspaper ad spending, being a large market is associated with an average **increase** in Sales of 57.42 widgets (as compared to a Small market, which is called the **baseline level**).\n",
        "\n",
        "What if we had reversed the 0/1 coding and created the feature 'IsSmall' instead? The coefficient would be the same, except it would be **negative instead of positive**. As such, your choice of category for the baseline does not matter, all that changes is your **interpretation** of the coefficient."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "k0cEVJxg-kkR",
        "colab_type": "text"
      },
      "source": [
        "## Handling Categorical Predictors with More than Two Categories\n",
        "\n",
        "Let's create a new feature called **Area**, and randomly assign observations to be **rural, suburban, or urban**:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KbvHBMQC-kkS",
        "colab_type": "code",
        "outputId": "b6a378ce-7bdb-4fbb-d04a-d9f27e3f5fbd",
        "colab": {}
      },
      "source": [
        "# set a seed for reproducibility\n",
        "np.random.seed(123456)\n",
        "\n",
        "# assign roughly one third of observations to each group\n",
        "nums = np.random.rand(len(data))\n",
        "mask_suburban = (nums > 0.33) & (nums < 0.66)\n",
        "mask_urban = nums > 0.66\n",
        "data['Area'] = 'rural'\n",
        "data.loc[mask_suburban, 'Area'] = 'suburban'\n",
        "data.loc[mask_urban, 'Area'] = 'urban'\n",
        "data.head()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>TV</th>\n",
              "      <th>Radio</th>\n",
              "      <th>Newspaper</th>\n",
              "      <th>Sales</th>\n",
              "      <th>Size</th>\n",
              "      <th>IsLarge</th>\n",
              "      <th>Area</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>230.1</td>\n",
              "      <td>37.8</td>\n",
              "      <td>69.2</td>\n",
              "      <td>22.1</td>\n",
              "      <td>large</td>\n",
              "      <td>1</td>\n",
              "      <td>rural</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>44.5</td>\n",
              "      <td>39.3</td>\n",
              "      <td>45.1</td>\n",
              "      <td>10.4</td>\n",
              "      <td>small</td>\n",
              "      <td>0</td>\n",
              "      <td>urban</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>17.2</td>\n",
              "      <td>45.9</td>\n",
              "      <td>69.3</td>\n",
              "      <td>9.3</td>\n",
              "      <td>small</td>\n",
              "      <td>0</td>\n",
              "      <td>rural</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>151.5</td>\n",
              "      <td>41.3</td>\n",
              "      <td>58.5</td>\n",
              "      <td>18.5</td>\n",
              "      <td>small</td>\n",
              "      <td>0</td>\n",
              "      <td>urban</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>180.8</td>\n",
              "      <td>10.8</td>\n",
              "      <td>58.4</td>\n",
              "      <td>12.9</td>\n",
              "      <td>large</td>\n",
              "      <td>1</td>\n",
              "      <td>suburban</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "      TV  Radio  Newspaper  Sales   Size  IsLarge      Area\n",
              "1  230.1   37.8       69.2   22.1  large        1     rural\n",
              "2   44.5   39.3       45.1   10.4  small        0     urban\n",
              "3   17.2   45.9       69.3    9.3  small        0     rural\n",
              "4  151.5   41.3       58.5   18.5  small        0     urban\n",
              "5  180.8   10.8       58.4   12.9  large        1  suburban"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EySO-Ahj-kkb",
        "colab_type": "text"
      },
      "source": [
        "We have to represent Area numerically, but we can't simply code it as 0=rural, 1=suburban, 2=urban because that would imply an **ordered relationship** between suburban and urban (and thus urban is somehow \"twice\" the suburban category).\n",
        "\n",
        "Instead, we create **another dummy variable**:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fJnyRZDJ-kkd",
        "colab_type": "code",
        "outputId": "31f2df97-7879-4df7-97f6-b4a490d3e818",
        "colab": {}
      },
      "source": [
        "# create three dummy variables using get_dummies, then exclude the first dummy column\n",
        "area_dummies = pd.get_dummies(data.Area, prefix='Area').iloc[:, 1:]\n",
        "\n",
        "# concatenate the dummy variable columns onto the original DataFrame (axis=0 means rows, axis=1 means columns)\n",
        "data = pd.concat([data, area_dummies], axis=1)\n",
        "data.head()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>TV</th>\n",
              "      <th>Radio</th>\n",
              "      <th>Newspaper</th>\n",
              "      <th>Sales</th>\n",
              "      <th>Size</th>\n",
              "      <th>IsLarge</th>\n",
              "      <th>Area</th>\n",
              "      <th>Area_suburban</th>\n",
              "      <th>Area_urban</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>230.1</td>\n",
              "      <td>37.8</td>\n",
              "      <td>69.2</td>\n",
              "      <td>22.1</td>\n",
              "      <td>large</td>\n",
              "      <td>1</td>\n",
              "      <td>rural</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>44.5</td>\n",
              "      <td>39.3</td>\n",
              "      <td>45.1</td>\n",
              "      <td>10.4</td>\n",
              "      <td>small</td>\n",
              "      <td>0</td>\n",
              "      <td>urban</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>17.2</td>\n",
              "      <td>45.9</td>\n",
              "      <td>69.3</td>\n",
              "      <td>9.3</td>\n",
              "      <td>small</td>\n",
              "      <td>0</td>\n",
              "      <td>rural</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>151.5</td>\n",
              "      <td>41.3</td>\n",
              "      <td>58.5</td>\n",
              "      <td>18.5</td>\n",
              "      <td>small</td>\n",
              "      <td>0</td>\n",
              "      <td>urban</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>180.8</td>\n",
              "      <td>10.8</td>\n",
              "      <td>58.4</td>\n",
              "      <td>12.9</td>\n",
              "      <td>large</td>\n",
              "      <td>1</td>\n",
              "      <td>suburban</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "      TV  Radio  Newspaper  Sales   Size  IsLarge      Area  Area_suburban  \\\n",
              "1  230.1   37.8       69.2   22.1  large        1     rural              0   \n",
              "2   44.5   39.3       45.1   10.4  small        0     urban              0   \n",
              "3   17.2   45.9       69.3    9.3  small        0     rural              0   \n",
              "4  151.5   41.3       58.5   18.5  small        0     urban              0   \n",
              "5  180.8   10.8       58.4   12.9  large        1  suburban              1   \n",
              "\n",
              "   Area_urban  \n",
              "1           0  \n",
              "2           1  \n",
              "3           0  \n",
              "4           1  \n",
              "5           0  "
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5ZUd7N8L-kkl",
        "colab_type": "text"
      },
      "source": [
        "Here is how we interpret the coding:\n",
        "- **rural** is coded as Area_suburban=0 and Area_urban=0\n",
        "- **suburban** is coded as Area_suburban=1 and Area_urban=0\n",
        "- **urban** is coded as Area_suburban=0 and Area_urban=1\n",
        "\n",
        "Why do we only need **two dummy variables, not three?** Because two dummies captures all of the information about the Area feature, and implicitly defines rural as the baseline level. (In general, if you have a categorical feature with k levels, you create k-1 dummy variables.)\n",
        "\n",
        "If this is confusing, think about why we only needed one dummy variable for Size (IsLarge), not two dummy variables (IsSmall and IsLarge).\n",
        "\n",
        "Let's include the two new dummy variables in the model:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_wSpgAoZ-kko",
        "colab_type": "code",
        "outputId": "1d9ff3bb-03a5-4767-b5c1-e09df3cd7745",
        "colab": {}
      },
      "source": [
        "# create X and y\n",
        "feature_cols = ['TV', 'Radio', 'Newspaper', 'IsLarge', 'Area_suburban', 'Area_urban']\n",
        "X = data[feature_cols]\n",
        "y = data.Sales\n",
        "\n",
        "# instantiate, fit\n",
        "lm = LinearRegression()\n",
        "lm.fit(X, y)\n",
        "\n",
        "# print coefficients\n",
        "print(feature_cols, lm.coef_)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "['TV', 'Radio', 'Newspaper', 'IsLarge', 'Area_suburban', 'Area_urban'] [ 0.04574401  0.1878667  -0.0010877   0.07739661 -0.10656299  0.26813802]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-FEr8NE2-kku",
        "colab_type": "text"
      },
      "source": [
        "How do we interpret the coefficients?\n",
        "- Holding all other variables fixed, being a **suburban** area is associated with an average **decrease** in Sales of 106.56 widgets (as compared to the baseline level, which is rural).\n",
        "- Being an **urban** area is associated with an average **increase** in Sales of 268.13 widgets (as compared to rural).\n",
        "\n",
        "**A final note about dummy encoding:** If you have categories that can be ranked (i.e., strongly disagree, disagree, neutral, agree, strongly agree), you can potentially use a single dummy variable and represent the categories numerically (such as 1, 2, 3, 4, 5)."
      ]
    }
  ]
}