examples/encoding_categorical_features.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### Encoding categorical features\n",
    "The ```categories``` and ```dummies``` arguments in the ```df_to_pandas``` method can be used to encode the categorical features. In this example, we create an encoding template for the ```ACSIncome``` task, namely ```ACSIncome_categories```, based on Appendix B.1 of the original [folktables paper](https://arxiv.org/abs/2108.04884). Note that the encoding of categorical features is subject to change and that any updates or additional information can be found in the [ACS PUMS documentation](https://www.census.gov/programs-surveys/acs/microdata/documentation.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from folktables import ACSDataSource, ACSIncome, generate_categories"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "The ```categories``` argument in the ```df_to_pandas``` method expects a nested dict with columns of categorical features and their corresponding encodings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "ACSIncome_categories = {\n",
    "    \"COW\": {\n",
    "        1.0: (\n",
    "            \"Employee of a private for-profit company or\"\n",
    "            \"business, or of an individual, for wages,\"\n",
    "            \"salary, or commissions\"\n",
    "        ),\n",
    "        2.0: (\n",
    "            \"Employee of a private not-for-profit, tax-exempt,\"\n",
    "            \"or charitable organization\"\n",
    "        ),\n",
    "        3.0: \"Local government employee (city, county, etc.)\",\n",
    "        4.0: \"State government employee\",\n",
    "        5.0: \"Federal government employee\",\n",
    "        6.0: (\n",
    "            \"Self-employed in own not incorporated business,\"\n",
    "            \"professional practice, or farm\"\n",
    "        ),\n",
    "        7.0: (\n",
    "            \"Self-employed in own incorporated business,\"\n",
    "            \"professional practice or farm\"\n",
    "        ),\n",
    "        8.0: \"Working without pay in family business or farm\",\n",
    "        9.0: \"Unemployed and last worked 5 years ago or earlier or never worked\",\n",
    "    },\n",
    "    \"SCHL\": {\n",
    "        1.0: \"No schooling completed\",\n",
    "        2.0: \"Nursery school, preschool\",\n",
    "        3.0: \"Kindergarten\",\n",
    "        4.0: \"Grade 1\",\n",
    "        5.0: \"Grade 2\",\n",
    "        6.0: \"Grade 3\",\n",
    "        7.0: \"Grade 4\",\n",
    "        8.0: \"Grade 5\",\n",
    "        9.0: \"Grade 6\",\n",
    "        10.0: \"Grade 7\",\n",
    "        11.0: \"Grade 8\",\n",
    "        12.0: \"Grade 9\",\n",
    "        13.0: \"Grade 10\",\n",
    "        14.0: \"Grade 11\",\n",
    "        15.0: \"12th grade - no diploma\",\n",
    "        16.0: \"Regular high school diploma\",\n",
    "        17.0: \"GED or alternative credential\",\n",
    "        18.0: \"Some college, but less than 1 year\",\n",
    "        19.0: \"1 or more years of college credit, no degree\",\n",
    "        20.0: \"Associate's degree\",\n",
    "        21.0: \"Bachelor's degree\",\n",
    "        22.0: \"Master's degree\",\n",
    "        23.0: \"Professional degree beyond a bachelor's degree\",\n",
    "        24.0: \"Doctorate degree\",\n",
    "    },\n",
    "    \"MAR\": {\n",
    "        1.0: \"Married\",\n",
    "        2.0: \"Widowed\",\n",
    "        3.0: \"Divorced\",\n",
    "        4.0: \"Separated\",\n",
    "        5.0: \"Never married or under 15 years old\",\n",
    "    },\n",
    "    \"SEX\": {1.0: \"Male\", 2.0: \"Female\"},\n",
    "    \"RAC1P\": {\n",
    "        1.0: \"White alone\",\n",
    "        2.0: \"Black or African American alone\",\n",
    "        3.0: \"American Indian alone\",\n",
    "        4.0: \"Alaska Native alone\",\n",
    "        5.0: (\n",
    "            \"American Indian and Alaska Native tribes specified;\"\n",
    "            \"or American Indian or Alaska Native,\"\n",
    "            \"not specified and no other\"\n",
    "        ),\n",
    "        6.0: \"Asian alone\",\n",
    "        7.0: \"Native Hawaiian and Other Pacific Islander alone\",\n",
    "        8.0: \"Some Other Race alone\",\n",
    "        9.0: \"Two or More Races\",\n",
    "    },\n",
    "}\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "We use the ```dummies``` argument to indicate that dummy variables for all the categorical features (which we have encoded via the ```categories``` argument) should be created. Note that the default value is ```False``` which just returns a pandas dataframe with the encoded categorical features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "   AGEP    OCCP  POBP  RELP  WKHP  \\\n0  30.0  9610.0   6.0  16.0  40.0   \n1  21.0  1970.0   6.0  17.0  20.0   \n2  65.0  2040.0   6.0  17.0   8.0   \n3  33.0  9610.0  36.0  16.0  40.0   \n4  18.0  1021.0   6.0  17.0  18.0   \n\n   COW_Employee of a private for-profit company orbusiness, or of an individual, for wages,salary, or commissions  \\\n0                                                0.0                                                                \n1                                                0.0                                                                \n2                                                0.0                                                                \n3                                                1.0                                                                \n4                                                0.0                                                                \n\n   COW_Employee of a private not-for-profit, tax-exempt,or charitable organization  \\\n0                                                0.0                                 \n1                                                0.0                                 \n2                                                1.0                                 \n3                                                0.0                                 \n4                                                1.0                                 \n\n   COW_Federal government employee  \\\n0                              0.0   \n1                              0.0   \n2                              0.0   \n3                              0.0   \n4                              0.0   \n\n   COW_Local government employee (city, county, etc.)  \\\n0                                                0.0    \n1                                                0.0    \n2                                                0.0    \n3                                                0.0    \n4                                                0.0    \n\n   COW_Self-employed in own incorporated business,professional practice or farm  \\\n0                                                0.0                              \n1                                                0.0                              \n2                                                0.0                              \n3                                                0.0                              \n4                                                0.0                              \n\n   ...  SEX_Male  RAC1P_Alaska Native alone  RAC1P_American Indian alone  \\\n0  ...       1.0                        0.0                          0.0   \n1  ...       1.0                        0.0                          0.0   \n2  ...       1.0                        0.0                          0.0   \n3  ...       1.0                        0.0                          0.0   \n4  ...       0.0                        0.0                          0.0   \n\n   RAC1P_American Indian and Alaska Native tribes specified;or American Indian or Alaska Native,not specified and no other  \\\n0                                                0.0                                                                         \n1                                                0.0                                                                         \n2                                                0.0                                                                         \n3                                                0.0                                                                         \n4                                                0.0                                                                         \n\n   RAC1P_Asian alone  RAC1P_Black or African American alone  \\\n0                0.0                                    0.0   \n1                0.0                                    0.0   \n2                0.0                                    0.0   \n3                0.0                                    0.0   \n4                0.0                                    0.0   \n\n   RAC1P_Native Hawaiian and Other Pacific Islander alone  \\\n0                                                0.0        \n1                                                0.0        \n2                                                0.0        \n3                                                0.0        \n4                                                0.0        \n\n   RAC1P_Some Other Race alone  RAC1P_Two or More Races  RAC1P_White alone  \n0                          1.0                      0.0                0.0  \n1                          0.0                      0.0                1.0  \n2                          0.0                      0.0                1.0  \n3                          0.0                      0.0                1.0  \n4                          0.0                      0.0                1.0  \n\n[5 rows x 53 columns]",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>AGEP</th>\n      <th>OCCP</th>\n      <th>POBP</th>\n      <th>RELP</th>\n      <th>WKHP</th>\n      <th>COW_Employee of a private for-profit company orbusiness, or of an individual, for wages,salary, or commissions</th>\n      <th>COW_Employee of a private not-for-profit, tax-exempt,or charitable organization</th>\n      <th>COW_Federal government employee</th>\n      <th>COW_Local government employee (city, county, etc.)</th>\n      <th>COW_Self-employed in own incorporated business,professional practice or farm</th>\n      <th>...</th>\n      <th>SEX_Male</th>\n      <th>RAC1P_Alaska Native alone</th>\n      <th>RAC1P_American Indian alone</th>\n      <th>RAC1P_American Indian and Alaska Native tribes specified;or American Indian or Alaska Native,not specified and no other</th>\n      <th>RAC1P_Asian alone</th>\n      <th>RAC1P_Black or African American alone</th>\n      <th>RAC1P_Native Hawaiian and Other Pacific Islander alone</th>\n      <th>RAC1P_Some Other Race alone</th>\n      <th>RAC1P_Two or More Races</th>\n      <th>RAC1P_White alone</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>30.0</td>\n      <td>9610.0</td>\n      <td>6.0</td>\n      <td>16.0</td>\n      <td>40.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>21.0</td>\n      <td>1970.0</td>\n      <td>6.0</td>\n      <td>17.0</td>\n      <td>20.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>65.0</td>\n      <td>2040.0</td>\n      <td>6.0</td>\n      <td>17.0</td>\n      <td>8.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>33.0</td>\n      <td>9610.0</td>\n      <td>36.0</td>\n      <td>16.0</td>\n      <td>40.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>18.0</td>\n      <td>1021.0</td>\n      <td>6.0</td>\n      <td>17.0</td>\n      <td>18.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n    </tr>\n  </tbody>\n</table>\n<p>5 rows × 53 columns</p>\n</div>"
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_source = ACSDataSource(survey_year='2018', horizon='1-Year', survey='person')\n",
    "ca_data = data_source.get_data(states=[\"CA\"], download=True)\n",
    "\n",
    "ca_features, ca_labels, _ = ACSIncome.df_to_pandas(ca_data, categories=ACSIncome_categories, dummies=True)\n",
    "\n",
    "ca_features.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "outputs": [
    {
     "data": {
      "text/plain": "   PINCP\n0  False\n1  False\n2  False\n3  False\n4  False",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>PINCP</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>False</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>False</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>False</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>False</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>False</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ca_labels.head()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Automatic generated categorical feature encoding\n",
    "The ```categories``` argument can automatically be generated using the ```get_definitions``` and ```generate_categories``` function to respectively download the definitions available [here](https://www2.census.gov/programs-surveys/acs/tech_docs/pums/data_dict/) and create the categories dictionary.\n",
    "Note that this feature only works for ```survey_year```>='2017' and categories not needing the externally deffined ```PUMA``` codes. [puma codes](https://www2.census.gov/geo/docs/reference/puma/2010_PUMA_Names.txt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "   AGEP  WKHP  \\\n0  30.0  40.0   \n1  21.0  20.0   \n2  65.0   8.0   \n3  33.0  40.0   \n4  18.0  18.0   \n\n   COW_Employee of a private for-profit company or business, or of an individual, for wages, salary, or commissions  \\\n0                                                0.0                                                                  \n1                                                0.0                                                                  \n2                                                0.0                                                                  \n3                                                1.0                                                                  \n4                                                0.0                                                                  \n\n   COW_Employee of a private not-for-profit, tax-exempt, or charitable organization  \\\n0                                                0.0                                  \n1                                                0.0                                  \n2                                                1.0                                  \n3                                                0.0                                  \n4                                                1.0                                  \n\n   COW_Federal government employee  \\\n0                              0.0   \n1                              0.0   \n2                              0.0   \n3                              0.0   \n4                              0.0   \n\n   COW_Local government employee (city, county, etc.)  \\\n0                                                0.0    \n1                                                0.0    \n2                                                0.0    \n3                                                0.0    \n4                                                0.0    \n\n   COW_Self-employed in own incorporated business, professional practice or farm  \\\n0                                                0.0                               \n1                                                0.0                               \n2                                                0.0                               \n3                                                0.0                               \n4                                                0.0                               \n\n   COW_Self-employed in own not incorporated business, professional practice, or farm  \\\n0                                                1.0                                    \n1                                                0.0                                    \n2                                                0.0                                    \n3                                                0.0                                    \n4                                                0.0                                    \n\n   COW_State government employee  \\\n0                            0.0   \n1                            1.0   \n2                            0.0   \n3                            0.0   \n4                            0.0   \n\n   COW_Working without pay in family business or farm  ...  SEX_Male  \\\n0                                                0.0   ...       1.0   \n1                                                0.0   ...       1.0   \n2                                                0.0   ...       1.0   \n3                                                0.0   ...       1.0   \n4                                                0.0   ...       0.0   \n\n   RAC1P_Alaska Native alone  RAC1P_American Indian alone  \\\n0                        0.0                          0.0   \n1                        0.0                          0.0   \n2                        0.0                          0.0   \n3                        0.0                          0.0   \n4                        0.0                          0.0   \n\n   RAC1P_American Indian and Alaska Native tribes specified; or American Indian or Alaska Native, not specified and no other races  \\\n0                                                0.0                                                                                 \n1                                                0.0                                                                                 \n2                                                0.0                                                                                 \n3                                                0.0                                                                                 \n4                                                0.0                                                                                 \n\n   RAC1P_Asian alone  RAC1P_Black or African American alone  \\\n0                0.0                                    0.0   \n1                0.0                                    0.0   \n2                0.0                                    0.0   \n3                0.0                                    0.0   \n4                0.0                                    0.0   \n\n   RAC1P_Native Hawaiian and Other Pacific Islander alone  \\\n0                                                0.0        \n1                                                0.0        \n2                                                0.0        \n3                                                0.0        \n4                                                0.0        \n\n   RAC1P_Some Other Race alone  RAC1P_Two or More Races  RAC1P_White alone  \n0                          1.0                      0.0                0.0  \n1                          0.0                      0.0                1.0  \n2                          0.0                      0.0                1.0  \n3                          0.0                      0.0                1.0  \n4                          0.0                      0.0                1.0  \n\n[5 rows x 816 columns]",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>AGEP</th>\n      <th>WKHP</th>\n      <th>COW_Employee of a private for-profit company or business, or of an individual, for wages, salary, or commissions</th>\n      <th>COW_Employee of a private not-for-profit, tax-exempt, or charitable organization</th>\n      <th>COW_Federal government employee</th>\n      <th>COW_Local government employee (city, county, etc.)</th>\n      <th>COW_Self-employed in own incorporated business, professional practice or farm</th>\n      <th>COW_Self-employed in own not incorporated business, professional practice, or farm</th>\n      <th>COW_State government employee</th>\n      <th>COW_Working without pay in family business or farm</th>\n      <th>...</th>\n      <th>SEX_Male</th>\n      <th>RAC1P_Alaska Native alone</th>\n      <th>RAC1P_American Indian alone</th>\n      <th>RAC1P_American Indian and Alaska Native tribes specified; or American Indian or Alaska Native, not specified and no other races</th>\n      <th>RAC1P_Asian alone</th>\n      <th>RAC1P_Black or African American alone</th>\n      <th>RAC1P_Native Hawaiian and Other Pacific Islander alone</th>\n      <th>RAC1P_Some Other Race alone</th>\n      <th>RAC1P_Two or More Races</th>\n      <th>RAC1P_White alone</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>30.0</td>\n      <td>40.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>21.0</td>\n      <td>20.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>65.0</td>\n      <td>8.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>33.0</td>\n      <td>40.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>18.0</td>\n      <td>18.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>...</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>0.0</td>\n      <td>1.0</td>\n    </tr>\n  </tbody>\n</table>\n<p>5 rows × 816 columns</p>\n</div>"
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "definition_df = data_source.get_definitions(download=True)\n",
    "categories = generate_categories(features=ACSIncome.features, definition_df=definition_df)\n",
    "\n",
    "ca_features, ca_labels, _ = ACSIncome.df_to_pandas(ca_data, categories=categories, dummies=True)\n",
    "\n",
    "ca_features.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "outputs": [
    {
     "data": {
      "text/plain": "   PINCP\n0  False\n1  False\n2  False\n3  False\n4  False",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>PINCP</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>False</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>False</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>False</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>False</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>False</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ca_labels.head()"
   ],
   "metadata": {
    "collapsed": false,
    "pycharm": {
     "name": "#%%\n"
    }
   }
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.9.7 ('base')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "c4f92193806e2908606a5f23edd55a5282f2f433b73b1c504507f9256ed9f0b4"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}