15_Factor_Analysis.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "15 Factor Analysis.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyOiZ6QJk5vT3Yie41EbKwXZ",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/sandipanpaul21/Dimensionality-Reduction-in-Python/blob/master/15_Factor_Analysis.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PNJv59MuPYa1"
      },
      "source": [
        "#### **Factor Analysis**\n",
        "\n",
        "- Factor analysis can only be used to reduce continuous variables of the dataset. Therefore, we will be removing categorical variables. \n",
        "- Factor analysis is a linear statistical model. \n",
        "- It is used to explain the variance among the observed variable and condense a set of the observed variable into the unobserved variable called factors.\n",
        "- Observed variables are modeled as a linear combination of factors and error terms \n",
        "- Factor or latent variable is associated with multiple observed variables, who have common patterns of responses.\n",
        "- Each factor explains a particular amount of variance in the observed variables. It helps in data interpretations by reducing the number of variables.\n",
        "- Factor analysis is a method for investigating whether a number of variables of interest X1, X2,……., Xi, are linearly related to a smaller number of unobservable factors F1, F2,..……, Fk."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Xs-1J0xdP5yi"
      },
      "source": [
        "#### **Assumptions:**\n",
        "1. There are no outliers in data.\n",
        "2. Sample size should be greater than the factor.\n",
        "3. There should not be perfect multicollinearity.\n",
        "4. There should not be homoscedasticity between the variables."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cDZws3SbQUR3"
      },
      "source": [
        "#### **Types of Factor Analysis**\n",
        "1. **Exploratory Factor Analysis :** Its basic assumption is that any observed variable is directly associated with any factor.\n",
        "2. **Confirmatory Factor Analysis (CFA):** Its basic assumption is that each factor is associated with a particular set of observed variables. CFA confirms what is expected on the basic.\n",
        "\n",
        "**How does factor analysis work?**\n",
        "- The primary objective of factor analysis is to reduce the number of observed variables and find unobservable variables. \n",
        "- These unobserved variables help the market researcher to conclude the survey. - This conversion of the observed variables to unobserved variables can be achieved in two steps:\n",
        "  1. **Factor Extraction:** In this step, the number of factors and approach for extraction selected using variance partitioning methods such as principal components analysis and common factor analysis.\n",
        "  2. **Factor Rotation:** In this step, rotation tries to convert factors into uncorrelated factors — the main goal of this step to improve the overall interpretability. There are lots of rotation methods that are available such as: Varimax rotation method, Quartimax rotation method, and Promax rotation method."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qXVXkx6XqNTx"
      },
      "source": [
        "**What is a factor?**\n",
        "- A factor is a latent variable which describes the association among the number of observed variables. \n",
        "- The maximum number of factors are equal to a number of observed variables. Every factor explains a certain variance in observed variables. \n",
        "- The factors with the lowest amount of variance were dropped. \n",
        "- Factors are also known as latent variables or hidden variables or unobserved variables or Hypothetical variables.\n",
        "\n",
        "**What are the factor loadings?**\n",
        "- The factor loading is a matrix which shows the relationship of each variable to the underlying factor. \n",
        "- It shows the correlation coefficient for observed variable and factor. \n",
        "- It shows the variance explained by the observed variables.\n",
        "\n",
        "**What is Eigenvalues?**\n",
        "\n",
        "Eigenvalues represent variance explained each factor from the total variance. It is also known as characteristic roots.\n",
        "\n",
        "**What are Communalities?** \n",
        "\n",
        "- Commonalities are the sum of the squared loadings for each variable. It represents the common variance. \n",
        "- It ranges from 0-1 and value close to 1 represents more variance.\n",
        "\n",
        "**What is Factor Rotation?**\n",
        "\n",
        "- Rotation is a tool for better interpretation of factor analysis. Rotation can be orthogonal or oblique. \n",
        "- It re-distributed the commonalities with a clear pattern of loadings.\n",
        "\n",
        "**Choosing the Number of Factors**\n",
        "- Kaiser criterion is an analytical approach, which is based on the more significant proportion of variance explained by factor will be selected. \n",
        "- The eigenvalue is a good criterion for determining the number of factors. Generally, an eigenvalue greater than 1 will be considered as selection criteria for the feature.\n",
        "- The graphical approach is based on the visual representation of factors' eigenvalues also called scree plot. This scree plot helps us to determine the number of factors where the curve makes an elbow."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "isa06YR_q4UF"
      },
      "source": [
        "**Factor Analysis Vs. Principle Component Analysis**\n",
        "- PCA components explain the maximum amount of variance while factor analysis explains the covariance in data.\n",
        "- PCA components are fully orthogonal to each other whereas factor analysis does not require factors to be orthogonal.\n",
        "- PCA component is a linear combination of the observed variable while in FA, the observed variables are linear combinations of the unobserved variable or factor.\n",
        "- PCA components are uninterpretable. In FA, underlying factors are labelable and interpretable.\n",
        "- PCA is a kind of dimensionality reduction method whereas factor analysis is the latent variable method.\n",
        "- PCA is a type of factor analysis. PCA is observational whereas FA is a modeling technique"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BsXOA7JBrfji"
      },
      "source": [
        "**BARTLETT’S TEST OF SPHERICITY**\n",
        "- Bartlett’s test checks whether the correlation is present in the given data. - It tests the null hypothesis (H0) that the correlation matrix is an Identical matrix. \n",
        "- The identical matrix consists of all the diagonal elements as 1.\n",
        "-  So, the null hypothesis assumes that no correlation is present among the variables.\n",
        "- We want to reject this null hypothesis because factor analysis aims at explaining the common variance i.e. the variation due to correlation among the variables. \n",
        "- If the p test statistic value is less than 0.05, we can decide that the correlation is not an Identical matrix i.e. correlation is present among the variables with a 95% confidence level."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "S4xaiKFitIXr"
      },
      "source": [
        "**Pros and Cons of Factor Analysis**\n",
        "- Factor analysis explores large dataset and finds interlinked associations. \n",
        "- It reduces the observed variables into a few unobserved variables or identifies the groups of inter-related variables, which help the market researchers to compress the market situations and find the hidden relationship among consumer taste, preference, and cultural influence. \n",
        "- Also, It helps in improve questionnaire in for future surveys. Factors make for more natural data interpretation.\n",
        "- Results of factor analysis are controversial. \n",
        "- Its interpretations can be debatable because more than one interpretation can be made of the same data factors. \n",
        "- After factor identification and naming of factors requires domain knowledge."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YH6xariz48hl"
      },
      "source": [
        "# Libraries\n",
        "\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "import matplotlib.pyplot as plt\n",
        "%matplotlib inline\n",
        "from sklearn import datasets\n",
        "#!pip install factor_analyzer\n",
        "from factor_analyzer import FactorAnalyzer"
      ],
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1govJ-HS42LW",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "outputId": "11497d3b-4513-4437-fc8a-8919ec4f6a89"
      },
      "source": [
        "# Load the Datasets \n",
        "\n",
        "# Boston Dataset\n",
        "boston = datasets.load_boston()\n",
        "boston_pd = pd.DataFrame(boston.data)\n",
        "boston_pd.columns = boston.feature_names\n",
        "boston_pd[\"MEDV\"] = boston.target\n",
        "boston_pd.head()"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>CRIM</th>\n",
              "      <th>ZN</th>\n",
              "      <th>INDUS</th>\n",
              "      <th>CHAS</th>\n",
              "      <th>NOX</th>\n",
              "      <th>RM</th>\n",
              "      <th>AGE</th>\n",
              "      <th>DIS</th>\n",
              "      <th>RAD</th>\n",
              "      <th>TAX</th>\n",
              "      <th>PTRATIO</th>\n",
              "      <th>B</th>\n",
              "      <th>LSTAT</th>\n",
              "      <th>MEDV</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0.00632</td>\n",
              "      <td>18.0</td>\n",
              "      <td>2.31</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.538</td>\n",
              "      <td>6.575</td>\n",
              "      <td>65.2</td>\n",
              "      <td>4.0900</td>\n",
              "      <td>1.0</td>\n",
              "      <td>296.0</td>\n",
              "      <td>15.3</td>\n",
              "      <td>396.90</td>\n",
              "      <td>4.98</td>\n",
              "      <td>24.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0.02731</td>\n",
              "      <td>0.0</td>\n",
              "      <td>7.07</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.469</td>\n",
              "      <td>6.421</td>\n",
              "      <td>78.9</td>\n",
              "      <td>4.9671</td>\n",
              "      <td>2.0</td>\n",
              "      <td>242.0</td>\n",
              "      <td>17.8</td>\n",
              "      <td>396.90</td>\n",
              "      <td>9.14</td>\n",
              "      <td>21.6</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0.02729</td>\n",
              "      <td>0.0</td>\n",
              "      <td>7.07</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.469</td>\n",
              "      <td>7.185</td>\n",
              "      <td>61.1</td>\n",
              "      <td>4.9671</td>\n",
              "      <td>2.0</td>\n",
              "      <td>242.0</td>\n",
              "      <td>17.8</td>\n",
              "      <td>392.83</td>\n",
              "      <td>4.03</td>\n",
              "      <td>34.7</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0.03237</td>\n",
              "      <td>0.0</td>\n",
              "      <td>2.18</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.458</td>\n",
              "      <td>6.998</td>\n",
              "      <td>45.8</td>\n",
              "      <td>6.0622</td>\n",
              "      <td>3.0</td>\n",
              "      <td>222.0</td>\n",
              "      <td>18.7</td>\n",
              "      <td>394.63</td>\n",
              "      <td>2.94</td>\n",
              "      <td>33.4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0.06905</td>\n",
              "      <td>0.0</td>\n",
              "      <td>2.18</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.458</td>\n",
              "      <td>7.147</td>\n",
              "      <td>54.2</td>\n",
              "      <td>6.0622</td>\n",
              "      <td>3.0</td>\n",
              "      <td>222.0</td>\n",
              "      <td>18.7</td>\n",
              "      <td>396.90</td>\n",
              "      <td>5.33</td>\n",
              "      <td>36.2</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "      CRIM    ZN  INDUS  CHAS    NOX  ...    TAX  PTRATIO       B  LSTAT  MEDV\n",
              "0  0.00632  18.0   2.31   0.0  0.538  ...  296.0     15.3  396.90   4.98  24.0\n",
              "1  0.02731   0.0   7.07   0.0  0.469  ...  242.0     17.8  396.90   9.14  21.6\n",
              "2  0.02729   0.0   7.07   0.0  0.469  ...  242.0     17.8  392.83   4.03  34.7\n",
              "3  0.03237   0.0   2.18   0.0  0.458  ...  222.0     18.7  394.63   2.94  33.4\n",
              "4  0.06905   0.0   2.18   0.0  0.458  ...  222.0     18.7  396.90   5.33  36.2\n",
              "\n",
              "[5 rows x 14 columns]"
            ]
          },
          "metadata": {},
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "iBrHtoWEsFz6",
        "outputId": "57cc2dde-7183-4155-f9eb-07502648f41b"
      },
      "source": [
        "from factor_analyzer.factor_analyzer import calculate_bartlett_sphericity\n",
        "chi_square_value,p_value=calculate_bartlett_sphericity(boston_pd)\n",
        "print(\"P value of Barlett Test :\",p_value)\n",
        "print(\"In this Bartlett ’s test, the p-value is 0. The test was statistically significant\")\n",
        "print(\"So indicating that the observed correlation matrix is not an identity matrix.\")"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "P value of Barlett Test : 0.0\n",
            "In this Bartlett ’s test, the p-value is 0. The test was statistically significant\n",
            "So indicating that the observed correlation matrix is not an identity matrix.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "awP6w6NKsD1W"
      },
      "source": [
        "**Kaiser-Meyer-Olkin (KMO)**\n",
        "- Test measures the suitability of data for factor analysis. \n",
        "- It determines the adequacy for each observed variable and for the complete model. \n",
        "- KMO estimates the proportion of variance among all the observed variable. Lower proportion id more suitable for factor analysis. \n",
        "- KMO values range between 0 and 1. \n",
        "- Value of KMO less than 0.6 is considered inadequate."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6GBK8Vuost_x",
        "outputId": "e5f600b6-6487-40dd-f508-87db11226c64"
      },
      "source": [
        "from factor_analyzer.factor_analyzer import calculate_kmo\n",
        "kmo_all,kmo_model=calculate_kmo(boston_pd)\n",
        "print(\"KMO Test :\",kmo_model)\n",
        "print(\"The overall KMO for our data is more than 0.84, which is excellent.\")\n",
        "print(\"This value indicates that you can proceed with your planned factor analysis.\")"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "KMO Test : 0.8527194101902166\n",
            "The overall KMO for our data is more than 0.84, which is excellent.\n",
            "This value indicates that you can proceed with your planned factor analysis.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GdP592jLuMrL",
        "outputId": "0ff88c69-8b50-44b0-8a62-10173627dd5b"
      },
      "source": [
        "# Choosing the Number of Factors\n",
        "# For choosing the number of factors, you can use the Kaiser criterion and scree plot. Both are based on eigenvalues.\n",
        "\n",
        "# Create factor analysis object and perform factor analysis\n",
        "fa = FactorAnalyzer()\n",
        "fa.fit(boston_pd)\n",
        "fa"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "FactorAnalyzer(bounds=(0.005, 1), impute='median', is_corr_matrix=False,\n",
              "               method='minres', n_factors=3, rotation='promax',\n",
              "               rotation_kwargs={}, svd_method='randomized', use_smc=True)"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0pVv9Dgeurr8",
        "outputId": "603752fc-1890-49d8-fd48-6055de716654"
      },
      "source": [
        "# Check Eigenvalues\n",
        "ev, v = fa.get_eigenvalues()\n",
        "print(ev)\n",
        "print(\"Here, you can see only for 3-factors eigenvalues are greater than one.\")\n",
        "print(\"It means we need to choose only 3 factors (or unobserved variables).\")"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[6.54598958 1.64953191 1.34890592 0.88653987 0.85089944 0.66001077\n",
            " 0.5354108  0.40307658 0.27726358 0.25225744 0.21279025 0.1829875\n",
            " 0.1340097  0.06032666]\n",
            "Here, you can see only for 3-factors eigenvalues are greater than one.\n",
            "It means we need to choose only 3 factors (or unobserved variables).\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 363
        },
        "id": "EYmdnUzAutKW",
        "outputId": "2761a990-d099-4c61-b629-d0e68907fcbb"
      },
      "source": [
        "# Create scree plot using matplotlib\n",
        "plt.scatter(range(1,boston_pd.shape[1]+1),ev)\n",
        "plt.plot(range(1,boston_pd.shape[1]+1),ev)\n",
        "plt.title('Scree Plot')\n",
        "plt.xlabel('Factors')\n",
        "plt.ylabel('Eigenvalue')\n",
        "plt.grid()\n",
        "plt.show()\n",
        "print(\"The scree plot method draws a straight line for each factor and its eigenvalues.\")\n",
        "print(\"Number eigenvalues greater than one considered as the number of factors.\")\n",
        "print(\"Here, you can see only for 3-factors eigenvalues are greater than one.\")\n",
        "print(\"It means we need to choose only 3 factors (or unobserved variables).\")"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAEWCAYAAABsY4yMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZRcZZ3/8fe3q/euTjpJL2SDkO4QZIcEQRFMcAREEGZ+OsOM67hkNlHmIDA4zrjM6HhkZJSfOv5UHFCQqICggCwjHUQUTMJOFrIBWSCdkK27093prv7+/rjVS0Iv1em6dbtufV7n9Klbt+6t++2c9KduPfe5z2PujoiIxE9R1AWIiEg4FPAiIjGlgBcRiSkFvIhITCngRURiSgEvIhJTCniRkJnZR8zsd1HXIYVHAS95x8zeZma/N7O9ZrbLzB4zs9MjrukLZtZtZm1mtidd31sO432WmdnHw6hRCo8CXvKKmU0C7gH+LzAVmAl8Eega4/sUZ786furuSaAO+B1wp5lZCMcRyYgCXvLNMQDufpu7p9y9w90fdPdn+zYws0+Y2WozazWzVWZ2Wnr9S2Z2jZk9C7SbWbGZnZk+295jZs+Y2aJB7zPZzG40s1fNbKuZ/buZJUYr0N27gZuBI4Bph75uZm81s+XpbyDLzeyt6fVfBs4GvpX+JvCtcf1LScFTwEu+eRFImdnNZvYuM5sy+EUzex/wBeBDwCTgPcDrgzb5S+DdQA3QANwL/DvBt4HPAHeYWV1625uAHqAJOBU4Dxi1+cTMyoCPAJvdfechr01NH/MGgvC/HrjXzKa5+z8DjwKfdPeku38yg38PkWEp4CWvuPs+4G2AA98HdpjZL82sIb3Jx4GvuftyD6x395cHvcUN7r7Z3TuADwD3uft97t7r7g8BK4AL0+93IXCFu7e7ewvwX8BlI5T352a2B9gMLAD+dIht3g2sc/cfu3uPu98GrAEuPsx/EpFhhdEOKRIqd19NcIaMmR0L3AJ8g+DsfDawYYTdNw9aPgp4n5kNDtcSoDn9Wgnw6qBm9KJD9j/Uz9z9A6OUPwN4+ZB1LxNcSxDJKgW85DV3X2NmNwF/k161GWgcaZdBy5uBH7v7Jw7dyMymE1y4rXX3niyVC7CN4MNjsCOB+4eoT2Rc1EQjecXMjjWzK81sVvr5bIIz98fTm/wA+IyZLbBAk5kdGqh9bgEuNrPzzSxhZuVmtsjMZrn7q8CDwNfNbJKZFZlZo5m9fZy/wn3AMWb2V+mLvH8BHEfQMwhgOzB3nMcQARTwkn9agTOAJ8ysnSDYnweuBHD3nwNfBn6S3vYugguob+Dum4FLgM8COwjO6K9i4O/iQ0ApsArYDdwOTB9P8e7+OnBRut7XgauBiwZdjP0m8F4z221mN4znWCKmCT9EROJJZ/AiIjGlgBcRiSkFvIhITCngRURiakL1g6+trfU5c+ZEXcYbtLe3U1VVFXUZh0W1R0O1516+1g3jq33lypU73b1uqNcmVMDPmTOHFStWRF3GGyxbtoxFixZFXcZhUe3RUO25l691w/hqN7ND74zupyYaEZGYUsCLiMSUAl5EJKYU8CIiMaWAFxGJqQnVi+Zw3PXUVq57YC3b9nQwo6aCq86fz6WnamhtEZG8Dvi7ntrKtXc+R0d3CoCtezq49s7nABTyIlLw8rqJ5roH1vaHe5+O7hTXPbA2oopERCaOvA74bXs6xrReRKSQ5HXAz6ipGNN6EZFCktcBf9X586koSRy0rqIkwVXnz4+oIhGRiSOvL7L2XUj917ufZ19nD9MnlXPNu47VBVYREfI84CEI+eryYj528wq+/YHTOO3IKVGXJCIyIeR1E02fxrokABta2iKuRERk4ohFwM+aUkFpoogNO9qjLkVEZMKIRcAXJ4qYU1vJhh06gxcR6ROLgIegmUZNNCIiA2IV8C/v2s+Bnt6oSxERmRDiE/D1VaR6nVd2qR1eRARiFPBNddUArG9RwIuIQIwCfm5dMCO5LrSKiARiE/BVZcVMn1yugBcRSQs14M2sxsxuN7M1ZrbazN4S5vHUk0ZEZEDYZ/DfBO5392OBk4HVYR6ssa6KDTvacfcwDyMikhdCC3gzmwycA9wI4O4H3H1PWMcDaKxP0tbVQ0trV5iHERHJCxbW2a6ZnQJ8D1hFcPa+Evi0u7cfst0SYAlAQ0PDgqVLlx72MVe9nuJryzu5+vRyjpuWGH2HDLW1tZFMJrP2frmk2qOh2nMvX+uG8dW+ePHile6+cMgX3T2UH2Ah0AOckX7+TeDfRtpnwYIFPh6v7e3wo665x3/0+03jep9DNTc3Z/X9ckm1R0O1516+1u0+vtqBFT5MpobZBr8F2OLuT6Sf3w6cFuLxqK8uI1lWrEHHREQIsQ3e3V8DNptZ3/RK7yBorgmNmaUvtKonjYhI2BN+XA7camalwEbgr0M+Ho11Sf6w8fWwDyMiMuGFGvDu/jRBW3zONNYnufOprbR19ZAsy/sJq0REDlts7mTt0ze70ya1w4tIgYtdwDfVa0waERGIYcAfObWKRJEp4EWk4MUu4EuLizhqqqbvExGJXcADzK1Lsl6DjolIgYtlwDfWV/HSzv30pDR9n4gUrngGfF2SA6letuzuiLoUEZHIxDLgm+qDrpJqhxeRQhbLgG+sVcCLiMQy4CdXllCbLGODJuAWkQIWy4CHYHan9TqDF5ECFt+Arw+6Srqm7xORAhXfgK9Lsrejm13tB6IuRUQkErEN+IGeNGqHF5HCFNuAb6zToGMiUthiG/AzJldQXlLEBg1ZICIFKrYBX1RkzK1NqieNiBSs2AY8BD1p1EQjIoUq3gFfV8WW3R10dqeiLkVEJOdiHfBN9UncYdNO9aQRkcIT64Dvm59VzTQiUoiKw3xzM3sJaAVSQI+7LwzzeIc6urYKMzQmjYgUpFADPm2xu+/MwXHeoLwkwawpFepJIyIFKdZNNBA006gvvIgUIgtzMC4z2wTsBhz4f+7+vSG2WQIsAWhoaFiwdOnSrNZw2+oumjf38N13VlJkdljv0dbWRjKZzGpduaLao6Hacy9f64bx1b548eKVwzZ/u3toP8DM9GM98AxwzkjbL1iwwLPt1sdf9qOuucc372o/7Pdobm7OXkE5ptqjodpzL1/rdh9f7cAKHyZTQ22icfet6ccW4BfAm8M83lA06JiIFKrQAt7Mqsysum8ZOA94PqzjDad/0DG1w4tIgQmzF00D8AsL2r2LgZ+4+/0hHm9IU6tKqaksUV94ESk4oQW8u28ETg7r/TNlZjTWBbM7iYgUkth3k4SgmUZt8CJSaAok4JPsbOti7/7uqEsREcmZggj4/p40O9VMIyKFoyACvn/QMbXDi0gBKYiAnzWlgtJEkdrhRaSgFETAFyeKmFNbqZ40IlJQCiLgIWim2ai+8CJSQAoq4F/etZ8DPb1RlyIikhMFE/BN9UlSvc4ru9QOLyKFoWACvq8nzXrN7iQiBaJgAn5u36BjaocXkQJRMAFfVVbM9Mnl6gsvIgWjYAIe0tP36QxeRApEgQV8MOiYhzhNoYjIRFFYAV+fpK2rh5bWrqhLEREJXUEFfJPGpBGRAlJQAd/YPz+rAl5E4q+gAr6+uoxkWbHGpBGRglBQAR9M36fZnUSkMBRUwIO6SopI4Si8gK9P8ureTtq6eqIuRUQkVIUX8OmeNJvUTCMiMZdRwJtZg5ndaGa/Tj8/zsw+luG+CTN7yszuGU+h2dJUrzFpRKQwZHoGfxPwADAj/fxF4IoM9/00sHpsZYXnyKlVJIpMAS8isZdpwNe6+8+AXgB37wFSo+1kZrOAdwM/OOwKs6y0uIijpmr6PhGJv+IMt2s3s2mAA5jZmcDeDPb7BnA1UD3cBma2BFgC0NDQwLJlyzIs6fBNLurk2Zf2Z3ystra2nNQVBtUeDdWee/laN4RYu7uP+gOcBjxGEOqPETTRnDTKPhcB30kvLwLuGe04CxYs8Fz4yn2rfN5n7/PunlRG2zc3N4dbUIhUezRUe+7la93u46sdWOHDZGpGZ/Du/qSZvR2YDxiw1t27R9ntLOA9ZnYhUA5MMrNb3P0DY/sIyr6muiQHUr1s2d3BnNqqqMsREQlFRgFvZh86ZNVpZoa7/2i4fdz9WuDa9P6LgM9MhHCHg8ekUcCLSFxl2gZ/+qDlcuAdwJPAsAE/kTXWDgT8O97UEHE1IiLhyLSJ5vLBz82sBlia6UHcfRmwbCyFhWlyZQm1yTL1pBGRWDvcO1nbgaOzWUiuadAxEYm7TNvgf0W6iyTBh8JxwM/CKioXGuuT3Pvsq7g7ZhZ1OSIiWZdpG/x/DlruAV529y0h1JMzTXVJ9nZ0s6v9ANOSZVGXIyKSdZm2wT8SdiG5NtCTpl0BLyKxNGIbvJm1mtm+IX5azWxfrooMQ2OdBh0TkXgb8Qze3YcdYiDfzZhcQXlJkXrSiEhsZdoGD4CZ1RP0gwfA3V/JekU5UlRkzK3V7E4iEl+Zjgf/HjNbB2wCHgFeAn4dYl050VivgBeR+Mq0H/y/AWcCL7r70QR3sj4eWlU50lhXxZbdHXR2jzrysYhI3sk04Lvd/XWgyMyK3L0ZWBhiXTnRVJ/EHTbt1A1PIhI/mbbB7zGzJPBb4FYzayG4mzWv9c3PumFHG2+aPiniakREsivTM/hLgP3APwL3AxuAi8MqKleOrq3CDPWkEZFYyvQM/m+An7r7VuDmEOvJqfKSBLOmVGhMGhGJpUzP4KuBB83sUTP7pJnFZozdxrokG3QGLyIxlFHAu/sX3f144B+A6cAjZva/oVaWI411STbubKO310ffWEQkj4x1uOAW4DXgdaA+++XkXlN9ks7uXrbt7Yi6FBGRrMr0Rqe/N7NlwG+AacAn3P2kMAvLlYGeNGqHF5F4yfQi62zgCnd/OsxiotA/6FhLG28/pi7iakREsifT4YKvNbOEmc0YvE8+j0XTZ2pVKTWVJazXkAUiEjOZzuj0SeALwHagN73agbxvpjEz9aQRkVjKtInmCmB+eriC2Gmsq+LhNTuiLkNEJKsy7UWzGdg7ljc2s3Iz+6OZPWNmL5jZF8deXm401SfZ2dbF3v3dUZciIpI1mZ7BbwSWmdm9QFffSne/foR9uoBz3b3NzEqA35nZr919wo1C2d+TZmcbpx05JeJqRESyI9Mz+FeAh4BSgrta+36G5YG+hu2S9M+EvJuoP+DVDi8iMWLumWeumVW6+/4xbJ8AVgJNwLfd/ZohtlkCLAFoaGhYsHTp0ozryZZUr/M3D+3nvDkl/Pn80je83tbWRjKZzHld2aDao6Hacy9f64bx1b548eKV7j708O3uPuoP8BZgFfBK+vnJwHcy2Te9fQ3QDJww0nYLFizwqLzz+mX+sZuWD/lac3NzbovJItUeDdWee/lat/v4agdW+DCZmmkTzTeA8wmGKMDdnwHOyfQTxt33pAP+gkz3ybXGuiQb1RdeRGIk47Fo3H3zIatGnOfOzOrMrCa9XAG8E1gz5gpzpLEuycu79nOgp3f0jUVE8kDG3STN7K2Am1mJmX0GWD3KPtOBZjN7FlgOPOTu94yj1lA11SdJ9Tqv7NKYNCISD5l2k/xb4JvATGAr8CDB0MHDcvdngVPHVV0O9fWkWd/STlP9iB2ERETyQqZj0ewE3h9yLZGa2zfomNrhRSQmMh2L5oYhVu8luHp7d3ZLikZVWTHTJ5erL7yIxEambfDlwCnAuvTPScAs4GNm9o2Qasu5xrqkzuBFJDYybYM/CTjL3VMAZvbfwKPA24DnQqot5xrrqrjjya24O2YWdTkiIuOS6Rn8FGDwbVZVwNR04HcNvUv+aapP0tbVQ0trbH4lESlgmZ7Bfw14Oj1tnxHc5PQVM6sCYjH5Nhw8Jk3DpPKIqxERGZ9Me9HcaGb3AW9Or/qsu29LL18VSmURaKxPd5Xc0cZbm2ojrkZEZHxGbKIxs2PTj6cR3Li0Of1zRHpdrNRXl5EsK1ZPGhGJhdHO4K8EPgF8fYjXHDg36xVFKJi+r4oNO3Q3q4jkvxED3t0/kX5cnJtyotdYl+QPG2M5M6GIFJjRmmiuHrT8vkNe+0pYRUWpsT7Jq3s7aevqiboUEZFxGa2b5GWDlq895LUJO/TvePT1pNmkZhoRyXOjBbwNszzU81hoqteYNCISD6MFvA+zPNTzWDhyahWJImO9etKISJ4brRfNyWa2j+BsvSK9TPp5LO8EKi0u4qiplTqDF5G8N1ovmkSuCplI5mrQMRGJgYyn7CskjfVVvLRzPz0pTd8nIvlLAT+EprokB1K9bNndEXUpIiKHTQE/hL4xadRMIyL5TAE/hMbavvlZFfAikr8U8EOYXFlCbbJMZ/AiktcU8MPQoGMiku9CC3gzm21mzWa2ysxeMLNPh3WsMDTWJ1nf0oZ7LO/nEpECkOmMToejB7jS3Z80s2pgpZk95O6rQjxm1jTVJdnb0c2u9gNRlyIiclhCO4N391fd/cn0ciuwGpgZ1vGybaAnjZppRCQ/WS6aIMxsDvBb4AR333fIa0uAJQANDQ0Lli5dGno9mdjZ0ctnHungI8eXsnBKF8lkcvSdJqC2tjbVHgHVnnv5WjeMr/bFixevdPeFQ77o7qH+AElgJfBno227YMECnyhSqV6f/7n7/Eu/esGbm5ujLuewqfZoqPbcy9e63cdXO7DCh8nUUHvRmFkJcAdwq7vfGeaxsq2oyJhbqzFpRCR/hdmLxoAbgdXufn1YxwlTY70CXkTyV5hn8GcBHwTONbOn0z8Xhni8rGuqS7JldwcHUuoqKSL5J7Ruku7+O/J81qfG+ircYft+BbyI5B/dyTqCvvlZt7Vp2GARyT8K+BE8t2UvAE+/up+zvvowdz21NeKKREQyF+adrHntrqe28vlfvgDAri5j694Orr3zOQAuPTVv7tcSkQKmM/hhXPfAWjq6U0AQ8AAd3Smue2BtlGWJiGRMAT+MbXsGZnPa0TlwrXjrHs3yJCL5QQE/jBk1Ff3L75yZOui1Ty99it9v2Elvr3rXiMjEpYAfxlXnz6eiJAHASVODIC8rLuLsplqa17TwV99/gsVfX8a3m9fTsq8zylJFRIaki6zD6LuQGrS5tzKzpoKrzp/PpafOpLM7xf3Pv8bS5a9w3QNruf6hFzn32HouO302bz+mjuKEPjdFJHoK+BFceupMLj11JsuWLePy9y/qX19ekuh/bdPOdn66fDO3r9zCQ6u20zCpjPctmM2fL5zNkdMqoyteRAqeAn6cjq6t4p/edSxXnncMD69p4afLN/OdZev5VvN63tZUy1+cPpvzjm+grDgRdakiUmAU8FlSkiji/OOP4Pzjj2Dbng5uX7mFny7fzOW3PcWUyhL+9NRZXPbm2RzTUB11qSJSIBTwIZhRU8Gn3jGPTy5u4rENO1m6fDM/fvwlfvjYJk47sobLTj+Sd580nYdWbee6B9aybU8HMwa18YuIZIMCPkRFRcbZ8+o4e14dr7d18YuntrJ0+WauvuNZ/uXu5+lJOan0jFpb9+hOWRHJLnX3yJFpyTI+fvZcHvrHc7jj795Cosj6w72P7pQVkWxSwOeYmbHgqKl0HEgN+brulBWRbFHAR2TwnbKDJYqMh9dsz3E1IhJHCviIDL5Ttk9poojaZCkfvWkFH795BZt37Y+oOhGJAwV8RC49dSb/8WcnMrOmAgNm1lTwtfeexKNXn8tnLzyW32/YyZ9c/wg3/GYdnd1DN+eIiIxEvWgi1Hc37KGWnNPIxSfP4N/vXc31D73IHU9u4QvvOZ7F8+sjqFJE8pXO4Ceo6ZMr+PZfncYtHzuDRJHx1/+znCU/WsGW3Wq2EZHMKOAnuLfNq+X+T5/DNRccy6Prgmabbz28jq4eNduIyMhCC3gz+6GZtZjZ82Edo1CUFhfxd4sa+c2Vb+fcY+v5zwdf5IJvPMojL+6IujQRmcDCPIO/CbggxPcvODNqKvjO+xfwo4++GYAP//CP/N0tK9V3XkSGFFrAu/tvgV1hvX8hO+eYOu6/4myuOn8+zWtb+JOvP8J3lq3nQE9v1KWJyARi7uFNO2dmc4B73P2EEbZZAiwBaGhoWLB06dLQ6jlcbW1tJJPJqMsY0s6OXm5bc4CV21McUWV88E1lHF870L9+Itc+GtUejXytPV/rhvHVvnjx4pXuvnCo1yIP+MEWLlzoK1asCK2ew7Vs2TIWLVoUdRkjal7bwhd++QIvv76fd584ndPnTOH7j27istmtLN1cnZcjVebDv/twVHvu5WvdML7azWzYgFc/+JhYPL+et1wxje//diM3PLyOe597FYDUzOyPVHnXU1s1zLFIHlDAx0h5SYLL3zGPWx5/me2tXQB844WguaajO8WVP3+G7/12I1VlCSpKi6kqTVBZWkxVWfqxNEFFaYKqsmIqSxNUlRZTWRY89u2zbE0L/3bPKjrT7f0a5lhk4got4M3sNmARUGtmW4DPu/uNYR1PBrSkwx3gzHrn8RYDINXrzKipYP+BHvZ2dPPqng72H0ix/0AP7QdSh32RtqM7xZfvXc3FJ88gUWRZ+R1EZPxCC3h3/8uw3ltGNqOmor/r5FkNvTzeEnSWmllTwQ8+PGRTHQDdqd6BwO8KHg99fs0dzw257462Lk750oO8ec5Uzpw7jTPnTuO4GZMU+CIRUhNNDF11/nyuvfM5OgYNUlZRkuCq8+ePuF9JoojJFUVMrigZdpsbfrN+yH73UypLuOCEI3hi4y5+s6YFgOqyYk4/eipnHB2E/vEzJlGc0M3TIrmigI+hvrbwYHaoVmZm8ULocB8en7/4+P73376vk8c3vs4Tm3bx+MbXeTgd+MmyYhbOmcKZc6dxxtFTOXHmZAW+SIgU8DHVN1LlsmXLuPz9i7L6vsCIvWgaJpVzySkzueSUYF3Lvs7+sH9i0y6++us1AFSVJlg4ZypnzA3O8E+cOZmSRFF/L53LZrfyz199WL10RA6TAl7GbLhhjodTP6mci0+ewcUnzwBgR2sXT2x6nSc2BqH/tfuDeWgrSxPMnlrJhpY2enqdnhC6eIoUEgW85FxddRkXnTSDi04KAn9nWxd/TJ/h/+SJV+jpDW6+u2FQF8/P3fU8Pb1OU32SpvokyTL91xUZjf5KJHK1yTIuPHE6F544nR//4eX+9WcM6uLZ1tXDZ37+TP9r0yeX01SfZF59dfDYkKSpLsmUqtIRj6WbtKSQKOBlQhmui+eMyeX8+ONnsL6lrf9nXUsrt/3xlYMu+NYmS2msGwj8eQ3BB0B9dRl3P73toAvEav6RuFPAy4QyXC+dqy84lsa6JI11Sc4/fmD73l5n654O1u9oY/32geC/++lttHb29G9XXV5MZ3eK7tTBYy91dKe47oG1CniJJQW8TChj7eJZVGTMnlrJ7KmVB81Z6+7saO1i3aCz/Vsef2XI9+g7k5/fkOSYI6qZ31DNtGRZ1n83kVxTwMuEk40unmZG/aRy6ieVc1ZTLQDNa3YMeZNWaaKI+557ldv+2N2/rjZZyjEN1RzTUM2xR1RzzBHB8mgXd9XFUyYSBbwUjOGaf/7jz07kklNmsKO1i7XbW1n7Wisvbm9l7fY2frp880Hbz6ypYH467OcfkWR+wyTm1lVRXpLgrqe2Drz/bLXxS/QU8FIwRrtJq++M/+x5df379PY6W3Z3sHZ7OvTT4f/ouh397fmJImPOtEq27umgszsYsO31zmB/tfFLlBTwUlDGepNWUZFx5LRKjpxWyTuPa+hf353q5aWd7f1n/Gtfa2XDjvb+129aN/CntXVPB9c/9CLz0t05j66toqw4gUjYFPAih6EkUcS8hmrmNVRz0UnBurO++nB/G/+7ZqX49ZYgxBNFxrceXkf6/i2KDOZMq+rvv9/Xl7+xLklF6fDBrz78MlYKeJEsGdzGf9wU59dbBtr4LzjhCDbuaE9352xlXUsb61raeHhNS/+du2Ywa0oF8+qrmZe+Y7evH///rtoeeh9+XSCOHwW8SJaM1sXzuBmTOG7GpIP2OdDTy8uvtweBvz3ozrm+pY3frdvJgdTABCwJM1L+xj78X/zVC1SWJigvSVBWXBQ8lhRRVpygPP3Yt36ksfnDvkCsbx/RUMCLZNFYu3iWFg809XDiwPqeVC+bd3ewLn22H3xovNHu/d0s+fHKjGorLrKBD4HiIsoGPa7etq//A+W+zcHdwx3dKf7l7ufZ1X6AZHkxybJiqsqKSZYF0zpWlQ6sKy0eftjngz48CO/DQ9883kgBLzIBFSeKOLq2iqNrqzjvePjJE68M2Ye/vrqMGz98Ol09Kbp6eunsDh67elJ0dvfS1d23vneIbQaWB39b2NY+cKbf2tnDl+5ZNWq9pcVF6bBP9Ad/sjwI/+Y1LQd1NYWBbx9VZcVUlCSoKA0+eCpKEv2PFaXBB5BZdN888p0CXiQPDNeH/7MXvokTZ00e9/sPvkD88WNTfP25IBpmTC7n3k+dTVtXD+0Hemjr7AmWu1K0d/Ut99CWfi1YF7y2q/0Ar+zaz/4DqSGPuXt/N5/40YoR6zKD8uIg7IPwLxq0nGD5S7v6u6Y2bxv45vGFXwZNV5MqSphUXsLkyhImpb+FjPSBMZR8bl5SwIvkgUwmWhmPkcYAmlJVOuoonSMZ/OExWH11GT/8yOl0dKfoOJCioztF56Dlju4UnQdSdPb0vmFdR3eKtq6e/nAHeH73QHDv6Ri66arI6A/9SRXFQfgPej65ouSg15/ZvJfvPrKBrvSE9Pn2DUEBL5InxtqHf6zvDbmd5vGzF76JE2aO79vH4A+Py48f+ObRUF3GjR85nX0d3ezr7GZfRw97+5e708s97OvoZn1LW/82hzYlDaWjO8U1dzzLb9ftoK66jPrq8vRjWf9jpt8Uwr5+oIAXESDaaR4P13AfHtce5odHV0+K1nTw7+3o5k+/8/thtuvliY272NHaddD1i8E1HBr69ZPKqUuWUTepjLpkGU++vJuv3Leazp7e0K4fhBrwZnYB8E0gAfzA3b8a5vFEZGIK69tHtr95lBUnKEsmqE2PJjpz0PwEg82sqeCxfzoXd2dvRzctrV3saO2ipbUzeNzX1b/uxe2tPLZ+J/Q0nGUAAAcqSURBVPsGDV99qKUbBmYvy+bQFqEFvJklgG8D7wS2AMvN7JfuPvoleRGRDIX1zQOG/4Zw1fnzgWDU0prKUmoqg9FHR9LZnUp/CATB/7e3DFwjmFrmbN0fNOlsG+ID5XCFeQb/ZmC9u28EMLOlwCWAAl5E8kI2m5fKSxL9cxfAwd8OzpvVy3O707OX1VRkqXowP+TuuKy9sdl7gQvc/ePp5x8EznD3Tx6y3RJgCUBDQ8OCpUuXhlLPeLS1tZFMJqMu47Co9mio9tzLt7r3dHSzdXcHve40VMD2DigyY+aUCmoqSjJ+n8WLF69094VDvRb5RVZ3/x7wPYCFCxf6okWLoi1oCMuWLWMi1pUJ1R4N1Z57+Vj34F40SzdX51Uvmq3A7EHPZ6XXiYgI4V4/ABh+AInxWw7MM7OjzawUuAz4ZYjHExGRQUI7g3f3HjP7JPAAQTfJH7r7C2EdT0REDhZqG7y73wfcF+YxRERkaGE20YiISIQU8CIiMRVaP/jDYWY7gJejrmMItcDOqIs4TKo9Gqo99/K1bhhf7Ue5e91QL0yogJ+ozGzFcDcSTHSqPRqqPffytW4Ir3Y10YiIxJQCXkQkphTwmfle1AWMg2qPhmrPvXytG0KqXW3wIiIxpTN4EZGYUsCLiMSUAn4EZjbbzJrNbJWZvWBmn466prEws4SZPWVm90Rdy1iYWY2Z3W5ma8xstZm9JeqaMmVm/5j+v/K8md1mZuVR1zQcM/uhmbWY2fOD1k01s4fMbF36cUqUNQ5nmNqvS/+fedbMfmFmNVHWOJyhah/02pVm5mZWm41jKeBH1gNc6e7HAWcC/2Bmx0Vc01h8GlgddRGH4ZvA/e5+LHAyefI7mNlM4FPAQnc/gWCQvcuirWpENwEXHLLun4DfuPs84Dfp5xPRTbyx9oeAE9z9JOBF4NpcF5Whm3hj7ZjZbOA84JVsHUgBPwJ3f9Xdn0wvtxIETfZnDg6Bmc0C3g38IOpaxsLMJgPnADcCuPsBd98TbVVjUgxUmFkxUAlsi7ieYbn7b4Fdh6y+BLg5vXwzcGlOi8rQULW7+4Pu3jez9eMEc1BMOMP8uwP8F3A1kLWeLwr4DJnZHOBU4IloK8nYNwj+s/RGXcgYHQ3sAP4n3bz0AzOrirqoTLj7VuA/Cc7AXgX2uvuD0VY1Zg3u/mp6+TWgIcpixuGjwK+jLiJTZnYJsNXdn8nm+yrgM2BmSeAO4Ap33xd1PaMxs4uAFndfOerGE08xcBrw3+5+KtDOxG0mOEi6vfoSgg+pGUCVmX0g2qoOnwd9qPOuH7WZ/TNB8+qtUdeSCTOrBD4L/Gu231sBPwozKyEI91vd/c6o68nQWcB7zOwlYClwrpndEm1JGdsCbHH3vm9KtxMEfj74E2CTu+9w927gTuCtEdc0VtvNbDpA+rEl4nrGxMw+AlwEvN/z5yafRoKTgmfSf7OzgCfN7IjxvrECfgRmZgRtwavd/fqo68mUu1/r7rPcfQ7BRb6H3T0vziTd/TVgs5nNT696B7AqwpLG4hXgTDOrTP/feQd5coF4kF8CH04vfxi4O8JaxsTMLiBolnyPu++Pup5Muftz7l7v7nPSf7NbgNPSfwvjooAf2VnABwnOgJ9O/1wYdVEF4HLgVjN7FjgF+ErE9WQk/a3jduBJ4DmCv68Je/u8md0G/AGYb2ZbzOxjwFeBd5rZOoJvJF+NssbhDFP7t4Bq4KH03+p3Iy1yGMPUHs6x8udbjIiIjIXO4EVEYkoBLyISUwp4EZGYUsCLiMSUAl5EJKYU8BJ7ZpYa1M316fSwE2PZ/9I8G2ROBAhuCxeJuw53P2Uc+18K3MMYbrgys+JBA1+JREL94CX2zKzN3ZODnicJ7tCcApQAn3P3u9OvfQj4DMEYLM8C/00Q7nvTP/+H4Gaa7xKMFrkB+Ki77zazZcDTwNuA2wjubP08kCIYeOyc0H9ZkUEU8BJ7ZpYiuLMUYBPwPqDS3felJ1Z4HJgHHAf8Aniru+80s6nuvsvMbgLucffb0+/3LHC5uz9iZl8CJrn7FemAX+Xuf5/e7jngAnffamY1eTbsscSAmmikEBzURJMeQO4rZnYOwXDKMwmGxT0X+Lm77wRw9zeM2Z0er77G3R9Jr7oZ+PmgTX46aPkx4CYz+xnBwGMiOaWLrFKI3g/UAQvSwb8dyNbUeu19C+7+t8DngNnASjOblqVjiGREAS+FaDLBePndZrYYOCq9/mHgfX1BbGZT0+tbCdrdcfe9wG4zOzv92geBRxiCmTW6+xPu/q8Ek5jMDuW3ERmGmmikEN0K/CrdRr4CWAPg7i+Y2ZeBR9Lt9k8BHyEYU//7ZvYp4L0Ew+h+Nz1Rw0bgr4c5znVmNg8wgvlNszpbj8hodJFVRCSm1EQjIhJTCngRkZhSwIuIxJQCXkQkphTwIiIxpYAXEYkpBbyISEz9fwXdbpJPBkz8AAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          }
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "The scree plot method draws a straight line for each factor and its eigenvalues.\n",
            "Number eigenvalues greater than one considered as the number of factors.\n",
            "Here, you can see only for 3-factors eigenvalues are greater than one.\n",
            "It means we need to choose only 3 factors (or unobserved variables).\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DAXZJSUYvRlp",
        "outputId": "3d4dc8d6-5e0f-4273-bd24-a6e4ff9e1d49"
      },
      "source": [
        "# Performing Factor Analysis\n",
        "\n",
        "# Create factor analysis object and perform factor analysis\n",
        "fa = FactorAnalyzer(n_factors=3)\n",
        "fa.fit(boston_pd)\n",
        "fa.loadings_"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[ 0.00681393,  0.62752005, -0.03747586],\n",
              "       [-0.6864742 ,  0.15409256,  0.21429705],\n",
              "       [ 0.57923372,  0.29165741, -0.12918081],\n",
              "       [ 0.22773115, -0.05249549,  0.22701669],\n",
              "       [ 0.69503037,  0.28719269,  0.01927342],\n",
              "       [-0.14855496,  0.2187204 ,  0.81465728],\n",
              "       [ 0.78266901,  0.04952037, -0.05658295],\n",
              "       [-0.91421884, -0.09769615, -0.1311503 ],\n",
              "       [-0.02472132,  1.06885318,  0.18159069],\n",
              "       [ 0.06796636,  0.94426231,  0.06523967],\n",
              "       [ 0.01384139,  0.25091612, -0.37124322],\n",
              "       [-0.05072981, -0.45037014,  0.02968354],\n",
              "       [ 0.36173886,  0.0743171 , -0.59322145],\n",
              "       [-0.06415291,  0.01811887,  0.94045888]])"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 483
        },
        "id": "AS6x05b7wj89",
        "outputId": "10c62657-1a7e-404f-9303-57613d1e43c6"
      },
      "source": [
        "# Making it into readable structure\n",
        "loadings = fa.loadings_\n",
        "Variable_Names =['crim','zn','indus','chas','nox','rm','age','dis','rad','tax','ptratio','black','lstat','MedV'] \n",
        "Matrix = pd.DataFrame(loadings,index = Variable_Names,\n",
        "                      columns = ['Factor_1','Factor_2','Factor_3'])\n",
        "Matrix\n",
        "\n",
        "# Inference\n",
        "# Loadings indicate how much a factor explains a variable. \n",
        "# The loading score will range from -1 to 1.\n",
        "# Values close to -1 or 1 indicate that the factor has an influence on these variables. \n",
        "# Values close to 0 indicates that the factor has a lower influencer on the variable.\n",
        "\n",
        "# For example, \n",
        "# in Factor 1, we can see that the features ‘age’ and ‘nox’  have high loadings than other variables. \n",
        "# From this, we can see that Factor 0, explains the common variance "
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Factor_1</th>\n",
              "      <th>Factor_2</th>\n",
              "      <th>Factor_3</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>crim</th>\n",
              "      <td>0.006814</td>\n",
              "      <td>0.627520</td>\n",
              "      <td>-0.037476</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>zn</th>\n",
              "      <td>-0.686474</td>\n",
              "      <td>0.154093</td>\n",
              "      <td>0.214297</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>indus</th>\n",
              "      <td>0.579234</td>\n",
              "      <td>0.291657</td>\n",
              "      <td>-0.129181</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>chas</th>\n",
              "      <td>0.227731</td>\n",
              "      <td>-0.052495</td>\n",
              "      <td>0.227017</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>nox</th>\n",
              "      <td>0.695030</td>\n",
              "      <td>0.287193</td>\n",
              "      <td>0.019273</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>rm</th>\n",
              "      <td>-0.148555</td>\n",
              "      <td>0.218720</td>\n",
              "      <td>0.814657</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>age</th>\n",
              "      <td>0.782669</td>\n",
              "      <td>0.049520</td>\n",
              "      <td>-0.056583</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>dis</th>\n",
              "      <td>-0.914219</td>\n",
              "      <td>-0.097696</td>\n",
              "      <td>-0.131150</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>rad</th>\n",
              "      <td>-0.024721</td>\n",
              "      <td>1.068853</td>\n",
              "      <td>0.181591</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>tax</th>\n",
              "      <td>0.067966</td>\n",
              "      <td>0.944262</td>\n",
              "      <td>0.065240</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>ptratio</th>\n",
              "      <td>0.013841</td>\n",
              "      <td>0.250916</td>\n",
              "      <td>-0.371243</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>black</th>\n",
              "      <td>-0.050730</td>\n",
              "      <td>-0.450370</td>\n",
              "      <td>0.029684</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>lstat</th>\n",
              "      <td>0.361739</td>\n",
              "      <td>0.074317</td>\n",
              "      <td>-0.593221</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>MedV</th>\n",
              "      <td>-0.064153</td>\n",
              "      <td>0.018119</td>\n",
              "      <td>0.940459</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "         Factor_1  Factor_2  Factor_3\n",
              "crim     0.006814  0.627520 -0.037476\n",
              "zn      -0.686474  0.154093  0.214297\n",
              "indus    0.579234  0.291657 -0.129181\n",
              "chas     0.227731 -0.052495  0.227017\n",
              "nox      0.695030  0.287193  0.019273\n",
              "rm      -0.148555  0.218720  0.814657\n",
              "age      0.782669  0.049520 -0.056583\n",
              "dis     -0.914219 -0.097696 -0.131150\n",
              "rad     -0.024721  1.068853  0.181591\n",
              "tax      0.067966  0.944262  0.065240\n",
              "ptratio  0.013841  0.250916 -0.371243\n",
              "black   -0.050730 -0.450370  0.029684\n",
              "lstat    0.361739  0.074317 -0.593221\n",
              "MedV    -0.064153  0.018119  0.940459"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 159
        },
        "id": "GHdU3ITnxAEq",
        "outputId": "81011fac-ef32-4cc7-e819-18cae87a417e"
      },
      "source": [
        "# Also compute the proportional variance and cumulative variance of the 3-factor solution\n",
        "\n",
        "print(\"Total 58% cumulative Variance explained by the 3 factors.\")\n",
        "pd.DataFrame(fa.get_factor_variance(),\n",
        "             index = ['SS Loading','Population Variance','Cummulative Variance'],\n",
        "             columns = ['Factor_1','Factor_2','Factor_3'])\n",
        "\n",
        "# Inference : \n",
        "# The first row represents the variance explained by each factor. \n",
        "# Population variance is the variance explained by a factor out of the total variance. \n",
        "# Cumulative variance is nothing but the cumulative sum of proportional variances of each factor. \n",
        "# In our case, the 3 factors together are able to explain 58% of the total variance.\n",
        "\n",
        "# In unrotated cases, the variances would be equal to the eigenvalues. \n",
        "# Rotation changes the distribution of proportional variance but the cumulative variance will remain the same. \n",
        "# Oblique rotations allow correlation between the factors while the orthogonal rotations keep the factors uncorrelated."
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Total 58% cumulative Variance explained by the 3 factors.\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Factor_1</th>\n",
              "      <th>Factor_2</th>\n",
              "      <th>Factor_3</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>SS Loading</th>\n",
              "      <td>2.955135</td>\n",
              "      <td>2.953383</td>\n",
              "      <td>2.212301</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Population Variance</th>\n",
              "      <td>0.211081</td>\n",
              "      <td>0.210956</td>\n",
              "      <td>0.158021</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Cummulative Variance</th>\n",
              "      <td>0.211081</td>\n",
              "      <td>0.422037</td>\n",
              "      <td>0.580058</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                      Factor_1  Factor_2  Factor_3\n",
              "SS Loading            2.955135  2.953383  2.212301\n",
              "Population Variance   0.211081  0.210956  0.158021\n",
              "Cummulative Variance  0.211081  0.422037  0.580058"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2aDeuDKZzVGp",
        "outputId": "909faa7d-c981-4ff9-88c4-7cb58dc7b94f"
      },
      "source": [
        "# COMMUNALITIES\n",
        "# Communality is the proportion of each variable’s variance that can be explained by the factors. \n",
        "# Rotations don’t have any influence over the communality of the variables.\n",
        "\n",
        "print(pd.DataFrame(fa.get_communalities(),index=boston_pd.columns,columns=['Communalities']))\n",
        "\n",
        "# Inference :\n",
        "# The proportion of each variable’s variance that is explained by the factors can be inferred. \n",
        "# For example, we could consider the variable ‘CRIM’ about 39% of its variance is explained by all the factors together."
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "         Communalities\n",
            "CRIM          0.395232\n",
            "ZN            0.540915\n",
            "INDUS         0.437263\n",
            "CHAS          0.106154\n",
            "NOX           0.565918\n",
            "RM            0.733574\n",
            "AGE           0.618225\n",
            "DIS           0.862541\n",
            "RAD           1.176033\n",
            "TAX           0.900507\n",
            "PTRATIO       0.200972\n",
            "B             0.206288\n",
            "LSTAT         0.488290\n",
            "MEDV          0.888907\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZA75GQ0Mz5An"
      },
      "source": [
        "That’s all about the factor analysis which can be used to find the underlying variance due to correlation among the observed variables like my heart which finds the creamy layer of the Oreo despite the unappetizing outer crunchy biscuits."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 159
        },
        "id": "006sv4SQxL-L",
        "outputId": "b2cca66d-cbbc-4e33-8f57-03bd6c251110"
      },
      "source": [
        "# Also compute the proportional variance and cumulative variance of the 4-factor solution\n",
        "# Performing Factor Analysis\n",
        "\n",
        "# Create factor analysis object and perform factor analysis\n",
        "fa = FactorAnalyzer(n_factors=4)\n",
        "fa.fit(boston_pd)\n",
        "fa.loadings_\n",
        "\n",
        "print(\"Total 62% cumulative Variance explained by the 4 factors.\")\n",
        "pd.DataFrame(fa.get_factor_variance(),\n",
        "             index = ['SS Loading','Population Variance','Cummulative Variance'],\n",
        "             columns = ['Factor_1','Factor_2','Factor_3','Factor_4'])"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Total 62% cumulative Variance explained by the 4 factors.\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Factor_1</th>\n",
              "      <th>Factor_2</th>\n",
              "      <th>Factor_3</th>\n",
              "      <th>Factor_4</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>SS Loading</th>\n",
              "      <td>2.937030</td>\n",
              "      <td>2.823889</td>\n",
              "      <td>2.163093</td>\n",
              "      <td>0.790065</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Population Variance</th>\n",
              "      <td>0.209788</td>\n",
              "      <td>0.201706</td>\n",
              "      <td>0.154507</td>\n",
              "      <td>0.056433</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Cummulative Variance</th>\n",
              "      <td>0.209788</td>\n",
              "      <td>0.411494</td>\n",
              "      <td>0.566001</td>\n",
              "      <td>0.622434</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                      Factor_1  Factor_2  Factor_3  Factor_4\n",
              "SS Loading            2.937030  2.823889  2.163093  0.790065\n",
              "Population Variance   0.209788  0.201706  0.154507  0.056433\n",
              "Cummulative Variance  0.209788  0.411494  0.566001  0.622434"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 159
        },
        "id": "fcWRarAlxLO6",
        "outputId": "c844783c-f190-4421-a796-950a647adcd3"
      },
      "source": [
        "# Also compute the proportional variance and cumulative variance of the 2-factor solution\n",
        "\n",
        "# Performing Factor Analysis\n",
        "\n",
        "# Create factor analysis object and perform factor analysis\n",
        "fa = FactorAnalyzer(n_factors=2)\n",
        "fa.fit(boston_pd)\n",
        "fa.loadings_\n",
        "\n",
        "print(\"Total 50% cumulative Variance explained by the 2 factors.\")\n",
        "pd.DataFrame(fa.get_factor_variance(),\n",
        "             index = ['SS Loading','Population Variance','Cummulative Variance'],\n",
        "             columns = ['Factor_1','Factor_2'])"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Total 50% cumulative Variance explained by the 2 factors.\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Factor_1</th>\n",
              "      <th>Factor_2</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>SS Loading</th>\n",
              "      <td>5.223245</td>\n",
              "      <td>1.915309</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Population Variance</th>\n",
              "      <td>0.373089</td>\n",
              "      <td>0.136808</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Cummulative Variance</th>\n",
              "      <td>0.373089</td>\n",
              "      <td>0.509897</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                      Factor_1  Factor_2\n",
              "SS Loading            5.223245  1.915309\n",
              "Population Variance   0.373089  0.136808\n",
              "Cummulative Variance  0.373089  0.509897"
            ]
          },
          "metadata": {},
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pYwz7V9z5aGI",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "outputId": "60fdc02f-b767-4d88-b18d-790b22fe6c3c"
      },
      "source": [
        "# Another Method\n",
        "# Removing Response Variable\n",
        "\n",
        "Factor1 = boston_pd[['CRIM', 'ZN', 'INDUS', 'CHAS', 'NOX', 'RM', 'AGE', 'DIS', 'RAD', \n",
        "                    'TAX','PTRATIO', 'B', 'LSTAT']]\n",
        "Factor1.head()"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>CRIM</th>\n",
              "      <th>ZN</th>\n",
              "      <th>INDUS</th>\n",
              "      <th>CHAS</th>\n",
              "      <th>NOX</th>\n",
              "      <th>RM</th>\n",
              "      <th>AGE</th>\n",
              "      <th>DIS</th>\n",
              "      <th>RAD</th>\n",
              "      <th>TAX</th>\n",
              "      <th>PTRATIO</th>\n",
              "      <th>B</th>\n",
              "      <th>LSTAT</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0.00632</td>\n",
              "      <td>18.0</td>\n",
              "      <td>2.31</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.538</td>\n",
              "      <td>6.575</td>\n",
              "      <td>65.2</td>\n",
              "      <td>4.0900</td>\n",
              "      <td>1.0</td>\n",
              "      <td>296.0</td>\n",
              "      <td>15.3</td>\n",
              "      <td>396.90</td>\n",
              "      <td>4.98</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0.02731</td>\n",
              "      <td>0.0</td>\n",
              "      <td>7.07</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.469</td>\n",
              "      <td>6.421</td>\n",
              "      <td>78.9</td>\n",
              "      <td>4.9671</td>\n",
              "      <td>2.0</td>\n",
              "      <td>242.0</td>\n",
              "      <td>17.8</td>\n",
              "      <td>396.90</td>\n",
              "      <td>9.14</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0.02729</td>\n",
              "      <td>0.0</td>\n",
              "      <td>7.07</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.469</td>\n",
              "      <td>7.185</td>\n",
              "      <td>61.1</td>\n",
              "      <td>4.9671</td>\n",
              "      <td>2.0</td>\n",
              "      <td>242.0</td>\n",
              "      <td>17.8</td>\n",
              "      <td>392.83</td>\n",
              "      <td>4.03</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0.03237</td>\n",
              "      <td>0.0</td>\n",
              "      <td>2.18</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.458</td>\n",
              "      <td>6.998</td>\n",
              "      <td>45.8</td>\n",
              "      <td>6.0622</td>\n",
              "      <td>3.0</td>\n",
              "      <td>222.0</td>\n",
              "      <td>18.7</td>\n",
              "      <td>394.63</td>\n",
              "      <td>2.94</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0.06905</td>\n",
              "      <td>0.0</td>\n",
              "      <td>2.18</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.458</td>\n",
              "      <td>7.147</td>\n",
              "      <td>54.2</td>\n",
              "      <td>6.0622</td>\n",
              "      <td>3.0</td>\n",
              "      <td>222.0</td>\n",
              "      <td>18.7</td>\n",
              "      <td>396.90</td>\n",
              "      <td>5.33</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "      CRIM    ZN  INDUS  CHAS    NOX  ...  RAD    TAX  PTRATIO       B  LSTAT\n",
              "0  0.00632  18.0   2.31   0.0  0.538  ...  1.0  296.0     15.3  396.90   4.98\n",
              "1  0.02731   0.0   7.07   0.0  0.469  ...  2.0  242.0     17.8  396.90   9.14\n",
              "2  0.02729   0.0   7.07   0.0  0.469  ...  2.0  242.0     17.8  392.83   4.03\n",
              "3  0.03237   0.0   2.18   0.0  0.458  ...  3.0  222.0     18.7  394.63   2.94\n",
              "4  0.06905   0.0   2.18   0.0  0.458  ...  3.0  222.0     18.7  396.90   5.33\n",
              "\n",
              "[5 rows x 13 columns]"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jH-_ofTI64-5",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 452
        },
        "outputId": "84654f2c-40a3-4a20-aea7-babe37e4b8c8"
      },
      "source": [
        "# Creating Correlation Matrix for the above dataset\n",
        "\n",
        "# To perform factor analysis we first create a correlation matrix using the above dataset.\n",
        "# We can also manually analyse this matrix as this will give us an idea of the variables \n",
        "# that are highly correlated to each other.\n",
        "corrm = Factor1.corr()\n",
        "corrm"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>CRIM</th>\n",
              "      <th>ZN</th>\n",
              "      <th>INDUS</th>\n",
              "      <th>CHAS</th>\n",
              "      <th>NOX</th>\n",
              "      <th>RM</th>\n",
              "      <th>AGE</th>\n",
              "      <th>DIS</th>\n",
              "      <th>RAD</th>\n",
              "      <th>TAX</th>\n",
              "      <th>PTRATIO</th>\n",
              "      <th>B</th>\n",
              "      <th>LSTAT</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>CRIM</th>\n",
              "      <td>1.000000</td>\n",
              "      <td>-0.200469</td>\n",
              "      <td>0.406583</td>\n",
              "      <td>-0.055892</td>\n",
              "      <td>0.420972</td>\n",
              "      <td>-0.219247</td>\n",
              "      <td>0.352734</td>\n",
              "      <td>-0.379670</td>\n",
              "      <td>0.625505</td>\n",
              "      <td>0.582764</td>\n",
              "      <td>0.289946</td>\n",
              "      <td>-0.385064</td>\n",
              "      <td>0.455621</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>ZN</th>\n",
              "      <td>-0.200469</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-0.533828</td>\n",
              "      <td>-0.042697</td>\n",
              "      <td>-0.516604</td>\n",
              "      <td>0.311991</td>\n",
              "      <td>-0.569537</td>\n",
              "      <td>0.664408</td>\n",
              "      <td>-0.311948</td>\n",
              "      <td>-0.314563</td>\n",
              "      <td>-0.391679</td>\n",
              "      <td>0.175520</td>\n",
              "      <td>-0.412995</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>INDUS</th>\n",
              "      <td>0.406583</td>\n",
              "      <td>-0.533828</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.062938</td>\n",
              "      <td>0.763651</td>\n",
              "      <td>-0.391676</td>\n",
              "      <td>0.644779</td>\n",
              "      <td>-0.708027</td>\n",
              "      <td>0.595129</td>\n",
              "      <td>0.720760</td>\n",
              "      <td>0.383248</td>\n",
              "      <td>-0.356977</td>\n",
              "      <td>0.603800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>CHAS</th>\n",
              "      <td>-0.055892</td>\n",
              "      <td>-0.042697</td>\n",
              "      <td>0.062938</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.091203</td>\n",
              "      <td>0.091251</td>\n",
              "      <td>0.086518</td>\n",
              "      <td>-0.099176</td>\n",
              "      <td>-0.007368</td>\n",
              "      <td>-0.035587</td>\n",
              "      <td>-0.121515</td>\n",
              "      <td>0.048788</td>\n",
              "      <td>-0.053929</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>NOX</th>\n",
              "      <td>0.420972</td>\n",
              "      <td>-0.516604</td>\n",
              "      <td>0.763651</td>\n",
              "      <td>0.091203</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-0.302188</td>\n",
              "      <td>0.731470</td>\n",
              "      <td>-0.769230</td>\n",
              "      <td>0.611441</td>\n",
              "      <td>0.668023</td>\n",
              "      <td>0.188933</td>\n",
              "      <td>-0.380051</td>\n",
              "      <td>0.590879</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>RM</th>\n",
              "      <td>-0.219247</td>\n",
              "      <td>0.311991</td>\n",
              "      <td>-0.391676</td>\n",
              "      <td>0.091251</td>\n",
              "      <td>-0.302188</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-0.240265</td>\n",
              "      <td>0.205246</td>\n",
              "      <td>-0.209847</td>\n",
              "      <td>-0.292048</td>\n",
              "      <td>-0.355501</td>\n",
              "      <td>0.128069</td>\n",
              "      <td>-0.613808</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>AGE</th>\n",
              "      <td>0.352734</td>\n",
              "      <td>-0.569537</td>\n",
              "      <td>0.644779</td>\n",
              "      <td>0.086518</td>\n",
              "      <td>0.731470</td>\n",
              "      <td>-0.240265</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-0.747881</td>\n",
              "      <td>0.456022</td>\n",
              "      <td>0.506456</td>\n",
              "      <td>0.261515</td>\n",
              "      <td>-0.273534</td>\n",
              "      <td>0.602339</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>DIS</th>\n",
              "      <td>-0.379670</td>\n",
              "      <td>0.664408</td>\n",
              "      <td>-0.708027</td>\n",
              "      <td>-0.099176</td>\n",
              "      <td>-0.769230</td>\n",
              "      <td>0.205246</td>\n",
              "      <td>-0.747881</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-0.494588</td>\n",
              "      <td>-0.534432</td>\n",
              "      <td>-0.232471</td>\n",
              "      <td>0.291512</td>\n",
              "      <td>-0.496996</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>RAD</th>\n",
              "      <td>0.625505</td>\n",
              "      <td>-0.311948</td>\n",
              "      <td>0.595129</td>\n",
              "      <td>-0.007368</td>\n",
              "      <td>0.611441</td>\n",
              "      <td>-0.209847</td>\n",
              "      <td>0.456022</td>\n",
              "      <td>-0.494588</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.910228</td>\n",
              "      <td>0.464741</td>\n",
              "      <td>-0.444413</td>\n",
              "      <td>0.488676</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>TAX</th>\n",
              "      <td>0.582764</td>\n",
              "      <td>-0.314563</td>\n",
              "      <td>0.720760</td>\n",
              "      <td>-0.035587</td>\n",
              "      <td>0.668023</td>\n",
              "      <td>-0.292048</td>\n",
              "      <td>0.506456</td>\n",
              "      <td>-0.534432</td>\n",
              "      <td>0.910228</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.460853</td>\n",
              "      <td>-0.441808</td>\n",
              "      <td>0.543993</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>PTRATIO</th>\n",
              "      <td>0.289946</td>\n",
              "      <td>-0.391679</td>\n",
              "      <td>0.383248</td>\n",
              "      <td>-0.121515</td>\n",
              "      <td>0.188933</td>\n",
              "      <td>-0.355501</td>\n",
              "      <td>0.261515</td>\n",
              "      <td>-0.232471</td>\n",
              "      <td>0.464741</td>\n",
              "      <td>0.460853</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-0.177383</td>\n",
              "      <td>0.374044</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>B</th>\n",
              "      <td>-0.385064</td>\n",
              "      <td>0.175520</td>\n",
              "      <td>-0.356977</td>\n",
              "      <td>0.048788</td>\n",
              "      <td>-0.380051</td>\n",
              "      <td>0.128069</td>\n",
              "      <td>-0.273534</td>\n",
              "      <td>0.291512</td>\n",
              "      <td>-0.444413</td>\n",
              "      <td>-0.441808</td>\n",
              "      <td>-0.177383</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>-0.366087</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>LSTAT</th>\n",
              "      <td>0.455621</td>\n",
              "      <td>-0.412995</td>\n",
              "      <td>0.603800</td>\n",
              "      <td>-0.053929</td>\n",
              "      <td>0.590879</td>\n",
              "      <td>-0.613808</td>\n",
              "      <td>0.602339</td>\n",
              "      <td>-0.496996</td>\n",
              "      <td>0.488676</td>\n",
              "      <td>0.543993</td>\n",
              "      <td>0.374044</td>\n",
              "      <td>-0.366087</td>\n",
              "      <td>1.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "             CRIM        ZN     INDUS  ...   PTRATIO         B     LSTAT\n",
              "CRIM     1.000000 -0.200469  0.406583  ...  0.289946 -0.385064  0.455621\n",
              "ZN      -0.200469  1.000000 -0.533828  ... -0.391679  0.175520 -0.412995\n",
              "INDUS    0.406583 -0.533828  1.000000  ...  0.383248 -0.356977  0.603800\n",
              "CHAS    -0.055892 -0.042697  0.062938  ... -0.121515  0.048788 -0.053929\n",
              "NOX      0.420972 -0.516604  0.763651  ...  0.188933 -0.380051  0.590879\n",
              "RM      -0.219247  0.311991 -0.391676  ... -0.355501  0.128069 -0.613808\n",
              "AGE      0.352734 -0.569537  0.644779  ...  0.261515 -0.273534  0.602339\n",
              "DIS     -0.379670  0.664408 -0.708027  ... -0.232471  0.291512 -0.496996\n",
              "RAD      0.625505 -0.311948  0.595129  ...  0.464741 -0.444413  0.488676\n",
              "TAX      0.582764 -0.314563  0.720760  ...  0.460853 -0.441808  0.543993\n",
              "PTRATIO  0.289946 -0.391679  0.383248  ...  1.000000 -0.177383  0.374044\n",
              "B       -0.385064  0.175520 -0.356977  ... -0.177383  1.000000 -0.366087\n",
              "LSTAT    0.455621 -0.412995  0.603800  ...  0.374044 -0.366087  1.000000\n",
              "\n",
              "[13 rows x 13 columns]"
            ]
          },
          "metadata": {},
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ol4hvN5F7BkJ",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "d9126671-a388-481f-f3b4-74708f49f502"
      },
      "source": [
        "# Finding Eigen Values\n",
        "\n",
        "# We will now find the eigenvalues to decide the number of factors \n",
        "# that will correctly group the features on the level of their similarity \n",
        "# allowing us to manually select features from each of these groups.\n",
        "eigen_values = np.linalg.eigvals(corrm)\n",
        "eigen_values_cumvar = (eigen_values/12).cumsum()\n",
        "eigen_values_cumvar\n",
        "\n",
        "# Inference : \n",
        "# Clearly, the four factors explain approximately 80% of the variance. \n",
        "# Therefore, the number of factors will be equal to 4 in our case."
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([0.51057074, 0.63001033, 0.73356172, 0.80502631, 0.87459431,\n",
              "       0.92937824, 0.97399124, 1.00699935, 1.01229179, 1.0353704 ,\n",
              "       1.04947898, 1.06498018, 1.08333333])"
            ]
          },
          "metadata": {},
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1VtocZY77Qhb",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "e2a838cf-3041-475b-c385-4aa1692c7e26"
      },
      "source": [
        "# Using Factor Analyzer to perform factor analysis\n",
        "\n",
        "# Compute the factor loadings to group the variables based on their correlation values\n",
        "Factor_Analysis = FactorAnalyzer(n_factors=4,rotation='varimax',method='ml')\n",
        "Factor_Analysis.fit(Factor1)\n",
        "\n",
        "# Here, we have used rotation equal to varimax to get maximum variance \n",
        "# and the method deployed for factor analysis is maximum likelihood.\n",
        "# R and Python use methods – maximum likelihood or minres. "
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "FactorAnalyzer(bounds=(0.005, 1), impute='median', is_corr_matrix=False,\n",
              "               method='ml', n_factors=4, rotation='varimax', rotation_kwargs={},\n",
              "               svd_method='randomized', use_smc=True)"
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GjzXDGi0-QBc",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 452
        },
        "outputId": "ce850c43-2b8f-40ac-e655-db99f04519f3"
      },
      "source": [
        "# Compute Factor Loading\n",
        "\n",
        "loadings = Factor_Analysis.loadings_\n",
        "Variable_Names =['crim','zn','indus','chas','nox','rm','age','dis','rad','tax','ptratio','black','lstat'] \n",
        "Matrix = pd.DataFrame(loadings,index = Variable_Names,\n",
        "                      columns = ['Factor_1','Factor_2','Factor_3','Factor_4'])\n",
        "Matrix\n",
        "\n",
        "# Select variables for each of these groups which will help us decreasing the features \n",
        "# and decrease the chances of multicollinearity."
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Factor_1</th>\n",
              "      <th>Factor_2</th>\n",
              "      <th>Factor_3</th>\n",
              "      <th>Factor_4</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>crim</th>\n",
              "      <td>0.585458</td>\n",
              "      <td>0.146890</td>\n",
              "      <td>0.149510</td>\n",
              "      <td>0.121240</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>zn</th>\n",
              "      <td>-0.069552</td>\n",
              "      <td>-0.750364</td>\n",
              "      <td>-0.143407</td>\n",
              "      <td>-0.352882</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>indus</th>\n",
              "      <td>0.552113</td>\n",
              "      <td>0.573599</td>\n",
              "      <td>0.302909</td>\n",
              "      <td>0.067669</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>chas</th>\n",
              "      <td>-0.016890</td>\n",
              "      <td>0.150731</td>\n",
              "      <td>-0.114095</td>\n",
              "      <td>-0.151607</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>nox</th>\n",
              "      <td>0.567399</td>\n",
              "      <td>0.659297</td>\n",
              "      <td>0.235815</td>\n",
              "      <td>-0.166511</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>rm</th>\n",
              "      <td>-0.120800</td>\n",
              "      <td>-0.123509</td>\n",
              "      <td>-0.758943</td>\n",
              "      <td>-0.231940</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>age</th>\n",
              "      <td>0.351947</td>\n",
              "      <td>0.715909</td>\n",
              "      <td>0.223779</td>\n",
              "      <td>-0.031676</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>dis</th>\n",
              "      <td>-0.366249</td>\n",
              "      <td>-0.833285</td>\n",
              "      <td>-0.077686</td>\n",
              "      <td>0.003268</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>rad</th>\n",
              "      <td>0.897665</td>\n",
              "      <td>0.196578</td>\n",
              "      <td>0.031521</td>\n",
              "      <td>0.243988</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>tax</th>\n",
              "      <td>0.908120</td>\n",
              "      <td>0.231359</td>\n",
              "      <td>0.139312</td>\n",
              "      <td>0.185353</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>ptratio</th>\n",
              "      <td>0.298806</td>\n",
              "      <td>0.143480</td>\n",
              "      <td>0.208084</td>\n",
              "      <td>0.674383</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>black</th>\n",
              "      <td>-0.444489</td>\n",
              "      <td>-0.145959</td>\n",
              "      <td>-0.129541</td>\n",
              "      <td>-0.006842</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>lstat</th>\n",
              "      <td>0.401955</td>\n",
              "      <td>0.360062</td>\n",
              "      <td>0.650686</td>\n",
              "      <td>0.097368</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "         Factor_1  Factor_2  Factor_3  Factor_4\n",
              "crim     0.585458  0.146890  0.149510  0.121240\n",
              "zn      -0.069552 -0.750364 -0.143407 -0.352882\n",
              "indus    0.552113  0.573599  0.302909  0.067669\n",
              "chas    -0.016890  0.150731 -0.114095 -0.151607\n",
              "nox      0.567399  0.659297  0.235815 -0.166511\n",
              "rm      -0.120800 -0.123509 -0.758943 -0.231940\n",
              "age      0.351947  0.715909  0.223779 -0.031676\n",
              "dis     -0.366249 -0.833285 -0.077686  0.003268\n",
              "rad      0.897665  0.196578  0.031521  0.243988\n",
              "tax      0.908120  0.231359  0.139312  0.185353\n",
              "ptratio  0.298806  0.143480  0.208084  0.674383\n",
              "black   -0.444489 -0.145959 -0.129541 -0.006842\n",
              "lstat    0.401955  0.360062  0.650686  0.097368"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ly5yvaiZ_oB7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 142
        },
        "outputId": "e0cb8bfe-a8e8-4dc3-8ad0-c15b9aea7b93"
      },
      "source": [
        "# Also compute the proportional variance and cumulative variance of the 4-factor solution\n",
        "\n",
        "pd.DataFrame(Factor_Analysis.get_factor_variance(),\n",
        "             index = ['SS Loading','Population Variance','Cummulative Variance'],\n",
        "             columns = ['Factor_1','Factor_2','Factor_3','Factor_4'])"
      ],
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Factor_1</th>\n",
              "      <th>Factor_2</th>\n",
              "      <th>Factor_3</th>\n",
              "      <th>Factor_4</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>SS Loading</th>\n",
              "      <td>3.326160</td>\n",
              "      <td>2.856880</td>\n",
              "      <td>1.339279</td>\n",
              "      <td>0.807530</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Population Variance</th>\n",
              "      <td>0.255858</td>\n",
              "      <td>0.219760</td>\n",
              "      <td>0.103021</td>\n",
              "      <td>0.062118</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Cummulative Variance</th>\n",
              "      <td>0.255858</td>\n",
              "      <td>0.475618</td>\n",
              "      <td>0.578640</td>\n",
              "      <td>0.640758</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                      Factor_1  Factor_2  Factor_3  Factor_4\n",
              "SS Loading            3.326160  2.856880  1.339279  0.807530\n",
              "Population Variance   0.255858  0.219760  0.103021  0.062118\n",
              "Cummulative Variance  0.255858  0.475618  0.578640  0.640758"
            ]
          },
          "metadata": {},
          "execution_count": 19
        }
      ]
    }
  ]
}