notebooks/bias_reduction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "W7Jzla7ErAbD"
   },
   "source": [
    "# Addressing Bias and Fairness in Machine Learning: A Practical Guide and Hands-on Tutorial\n",
    "## KDD 2023 Hands-on Tutorial\n",
    "### Rayid Ghani, Kit Rodolfa, Pedro Saleiro, Sérgio Jesus"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KTypj7J8rAbJ"
   },
   "source": [
    "# <font color=red>Exploring Bias Reduction Strategies</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BBNQKdiirAbK"
   },
   "source": [
    "## 1. Install dependencies, import packages and data\n",
    "This is needed every time you open this notebook in **colab** to install dependencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "24sTaU89rAbL"
   },
   "outputs": [],
   "source": [
    "# packages not installed in colab\n",
    "!pip install aequitas==0.42.0\n",
    "!pip install fairlearn==0.4.6\n",
    "!pip install hyperparameter-tuning\n",
    "!pip install fairgbm==0.9.14\n",
    "\n",
    "import yaml\n",
    "import os\n",
    "import math\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "from aequitas.group import Group\n",
    "from aequitas.bias import Bias\n",
    "from aequitas.fairness import Fairness\n",
    "import aequitas.plot as ap\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "import fairlearn\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")\n",
    "sns.set() \n",
    "DPI = 200\n",
    "DATAPATH = 'https://github.com/dssg/fairness_tutorial/raw/master/data/'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "v2U-ALesrAbN"
   },
   "source": [
    "## What has already happened?\n",
    "\n",
    "We've already cleaned data, generated features, created train-test sets, built 1000s of models on each training set and scored each test set with them, and calculated various evaluation metrics. \n",
    "\n",
    "As described earlier, the goal here is to select top 1000 project submissions that are likely to not get funded in order to prioritize resource allocation. That corresponds to the metric **Precision at top 1000**.\n",
    "\n",
    "### <font color=red>We audited the \"best\" model at precision at top 1000 and found that it has disparities for True Positive Rate for all attributes that we care about (poverty_level of the school, sex, and school_location_type)</font>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1vkzSfw8rAbO"
   },
   "source": [
    "## What do we want to do now?\n",
    "\n",
    "1. Could we have picked a different model that was similar enough in \"precision at top 1000\" but less biased?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "eDO35R5grAbP"
   },
   "source": [
    "## Load predictions, labels, and attributes for all models that were built to audit\n",
    "\n",
    "We have trained 400 models using random sampling of hyperparameters for the following algorithms: RandomForest, Logistic Regression, MLP and LightGBM. The `evals_df` contains a table with the performance metrics for each model on the holdout test set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "muY_xli8rAbR",
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "evals_df = pd.read_csv(DATAPATH +'split2_evals.csv.gz', compression='gzip')\n",
    "# Let's sort the models by Precision at top 1000 predicted positives (our performance metric of interest for this case study)\n",
    "evals_df.sort_values('model_precision', ascending = False)\n",
    "evals_df.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RiX_PwwMrAbS"
   },
   "source": [
    "### Spot-check the model with highest precision at 1000 to see what type it is"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "VUcUjGTSrAbS"
   },
   "outputs": [],
   "source": [
    "print('Highest precision model classpath and hyperparameters: \\n\\n' , evals_df['model_classpath'][0], '\\n', evals_df['hyperparameters'][0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "CFMeQTljrAbT"
   },
   "source": [
    "## Load pre-computed Aequitas audit results \n",
    "\n",
    "The `aequitas_df` contains a table with the bias audit results for the 400 models for all the attributes we care about (we pre-selected the metrics of interest for this tutorial). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "P4WA-GNQrAbU"
   },
   "outputs": [],
   "source": [
    "aequitas_df = pd.read_csv(DATAPATH + 'split2_aequitas.csv.gz', compression='gzip')\n",
    "aequitas_df.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "igzvCivmrAbV"
   },
   "source": [
    "Combining the `evals_df` with the `aequitas_df` allow us to get a clearer picture how the different models compare with each other in both performance and bias of our groups of interest. \n",
    "\n",
    "The `create_scatter_disparity_performance` method will help us to easily plot the 400 models bias-performance tradeoffs and it's going to be used throughout the notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Gfzhh8EnrAbV"
   },
   "outputs": [],
   "source": [
    "def create_scatter_disparity_performance(evals_df, aequitas_df,  attr_col, group_name, \n",
    "                                         performance_col='model_precision', bias_metric='tpr', flip_disparity=False, \n",
    "                                         mitigated_tags=[], mitigated_bdfs=[], mitigated_performances=[], ylim=None):\n",
    "    disparity_df = aequitas_df.loc[(aequitas_df['attribute_name']==attr_col) & (aequitas_df['attribute_value']==group_name)].copy()\n",
    "    disparity_metric = bias_metric + '_disparity'\n",
    "    scatter_schema = ['model_uuid', performance_col, 'attribute_name', 'attribute_value', bias_metric, disparity_metric, 'model_tag']\n",
    "    if flip_disparity:\n",
    "        disparity_df[disparity_metric]= disparity_df.apply(lambda x: 1/x[disparity_metric] , axis=1)\n",
    "    scatter = pd.merge(evals_df, disparity_df, how='left', on=['model_uuid'], sort=True, copy=True)\n",
    "    scatter = scatter[['model_uuid', performance_col, 'attribute_name', 'attribute_value', bias_metric, disparity_metric]].copy()\n",
    "    scatter['model_tag'] = 'Other Models'\n",
    "    scatter.sort_values('model_precision', ascending = False, inplace=True, ignore_index=True)\n",
    "    scatter['model_tag'] = scatter.apply(lambda x: 'Highest Precision at 1000' if int(x.name) < 1 else x['model_tag'], axis=1)\n",
    "    mitigated_points = []\n",
    "    scatter_final = pd.DataFrame()\n",
    "    if mitigated_bdfs and mitigated_performances:\n",
    "        for i in range(len(mitigated_bdfs)):\n",
    "            if not mitigated_bdfs[i].empty and mitigated_performances[i] !=None:\n",
    "                mitigated_bdfs[i][performance_col] = mitigated_performances[i]\n",
    "                mitigated_bdfs[i]['model_tag'] = mitigated_tags[i]\n",
    "                new_disparity_df = mitigated_bdfs[i].loc[(mitigated_bdfs[i]['attribute_name']==attr_col) & (mitigated_bdfs[i]['attribute_value']==group_name)].copy()\n",
    "                if flip_disparity:\n",
    "                    new_disparity_df[disparity_metric]= new_disparity_df.apply(lambda x: 1/x[disparity_metric] , axis=1)\n",
    "                scatter_new = new_disparity_df[[c for c in new_disparity_df.columns if c in scatter_schema]].copy()\n",
    "                mitigated_points.append(scatter_new)\n",
    "        scatter_final = pd.concat(mitigated_points , axis=0)\n",
    "\n",
    "    ax = sns.scatterplot(\n",
    "        x='model_precision', y=disparity_metric, hue='model_tag',\n",
    "        data=scatter,\n",
    "        alpha=0.4, s=20, palette = np.array(sns.color_palette()[0:2])\n",
    "\n",
    "\n",
    "    )\n",
    "    if not scatter_final.empty:\n",
    "        ax1 = sns.scatterplot(\n",
    "            x='model_precision', y=disparity_metric, hue='model_tag',\n",
    "            data=scatter_final,\n",
    "            alpha = 0.95, s=20,  palette = np.array(sns.color_palette()[2:3] + sns.color_palette()[4:])\n",
    "        )\n",
    "\n",
    "    if ylim:\n",
    "        plt.ylim(0, 10)\n",
    "    flip_placeholder = 'Flipped' if flip_disparity else ''\n",
    "    ax.set_title('{} {} vs.{} for {}:{}'.format(flip_placeholder, disparity_metric, performance_col, attr_col,group_name ), y=1., fontsize='xx-small')\n",
    "    #plt.gcf().set_size_inches((4, 3))\n",
    "    plt.legend(loc='upper left', fontsize='xx-small', prop={'size': 5})\n",
    "    plt.gcf().set_dpi(DPI)\n",
    "    plt.show() "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1leh3zoHrAbW"
   },
   "source": [
    "\n",
    "\n",
    "\n",
    "## Let's see if we could have picked a better model for fairness in Poverty Level\n",
    "\n",
    "Each point in the scatterplot represents a model. We highlight the model we picked before (highest global performance)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "2r82K6SXrAbW"
   },
   "outputs": [],
   "source": [
    "create_scatter_disparity_performance(evals_df, aequitas_df, 'poverty_level', 'highest', flip_disparity=True )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "c_mO5_rRrAbX"
   },
   "source": [
    "## Let's see if we could have picked a better model for fairness in Metro Type\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "3XwF6FfKrAbX"
   },
   "outputs": [],
   "source": [
    "create_scatter_disparity_performance(evals_df, aequitas_df, 'metro_type', 'urban', flip_disparity=True, ylim=10 )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "83cXAfpZrAbY"
   },
   "source": [
    "## Let's see if we could have picked a better model for fairness in Teacher sex\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "E2iLNVwjrAbY"
   },
   "outputs": [],
   "source": [
    "create_scatter_disparity_performance(evals_df, aequitas_df, 'teacher_sex', 'female', flip_disparity=False )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hz6HP95UrAbZ"
   },
   "source": [
    "# Bias Reduction Strategies\n",
    "We will try a few different bias reduction strategies now and compare the audit results with the original models\n",
    "\n",
    "0. Unawareness\n",
    "1. Re-Sampling\n",
    "2. Regularization (using Fairlearn package)\n",
    "3. Post-hoc adjustment\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <font color=\"red\">Bias Reduction Strategy 0: Unawareness</font>\n",
    "### Can removing the protected attribute from the dataset help improve the fairness of our models? "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wsqFpwP1rAbZ"
   },
   "source": [
    "### What has already happened?\n",
    "\n",
    "We've already cleaned data, generated features, created train-test sets, built 1000s of models on each training set and scored each test set with them, and calculated various evaluation metrics. We then used these results to pick a \"best\" model in terms of performance on the \"accuracy\" metric we care about: **Precision at the top 1000** (corresponding to our goal of selecting 1000 project submissions that are most likely to not get funded in order to prioritize resource allocation).\n",
    "\n",
    "When we audited this selected model with Aequitas, however, we found biases across many attributes, including the poverty level of the schools. Here, we explore several methods of bias mitigation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We will first load the dataset\n",
    "traindf = pd.read_csv(DATAPATH + 'train_20120501_20120801.csv.gz', compression='gzip')\n",
    "testdf = pd.read_csv(DATAPATH + 'test_20121201_20130201.csv.gz', compression='gzip')\n",
    "train_attrdf = pd.read_csv(DATAPATH + 'train_20120501_20120801_protected.csv.gz', compression='gzip')\n",
    "test_attrdf = pd.read_csv(DATAPATH + 'test_20121201_20130201_protected.csv.gz', compression='gzip')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "traindf.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get the best model from the previous run"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('Highest precision model classpath and hyperparameters: \\n\\n' , evals_df['model_classpath'][0], '\\n', evals_df['hyperparameters'][0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Let's load the hyperparameters of the highest performance model to a dictionary\n",
    "import ast\n",
    "hyperparameters= ast.literal_eval(evals_df['hyperparameters'][0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Since the model with highest performance is a RandomForestClassifier let's create a classifier with the fetched hyperparameters\n",
    "rf = RandomForestClassifier(**hyperparameters)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rf.__dict__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Defining label, group of interest and reference group for the attribute column we care about\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "label_col = 'quickstart_label'\n",
    "attribute_col = 'poverty_level'\n",
    "\n",
    "group_of_interest = 'highest'\n",
    "reference_group = 'lower'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Look at training data distributions\n",
    "\n",
    "#### Poverty_level=Highest\n",
    "label_pos_poverty_highest =  P(poverty_level=highest | not_funded)\n",
    "\n",
    "label_neg_poverty_highest =  P(poverty_level=highest | funded)\n",
    "\n",
    "\n",
    "#### Poverty_level=Lower\n",
    "label_pos_poverty_lower =  P(poverty_level=lower | not_funded)\n",
    "\n",
    "label_neg_poverty_lower =  P(poverty_level=lower | funded)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_attrdf[attribute_col].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_group_stats(df, attrdf, label_col, attribute_col, group_name):\n",
    "    \"\"\"Calculates group-wise stats: prevalence, group_size, num label pos, num label neg\n",
    "       Returns: prevalence, group_size, df of group label pos, df of group label neg\n",
    "    \"\"\"\n",
    "    group_size = len(df.loc[attrdf[attribute_col]==group_name])\n",
    "    label_pos = df.loc[(attrdf[attribute_col]==group_name) & (df[label_col] > 0)].copy()\n",
    "    label_neg = df.loc[(attrdf[attribute_col]==group_name) & (df[label_col] < 1.0)].copy()\n",
    "    assert (group_size == len(label_pos) + len(label_neg)), \"label pos + label neg is different from group size!\"\n",
    "    prev = len(label_pos) / group_size\n",
    "    print(\"{} == '{}'\\n\\t group size: {} \\n\\t prevalence: {:.3f}\".format(attribute_col,group_name, group_size, prev))\n",
    "    return prev, group_size, label_pos, label_neg"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "_ = get_group_stats(traindf, train_attrdf, label_col, attribute_col, group_of_interest)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "_ = get_group_stats(traindf, train_attrdf, label_col, attribute_col, reference_group)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### What type of disparities do we see in the data distribution here?\n",
    "\n",
    "\n",
    "1.\n",
    "\n",
    "2.\n",
    "\n",
    "3.\n",
    "\n",
    "4.\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "...\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('Recapping the group of interest and reference we defined above: \\n\\n Attribute: {}\\n Group of interest: {}\\n Reference Group: {}'.format(attribute_col, group_of_interest, reference_group))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Let's now try Unawareness\n",
    "\n",
    "For this, we will remove the information regarding the group of interest in the dataset (before passing it to train)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Removing the protected attribute of poverty level (they are OHE)\n",
    "protected_attributes = [\n",
    "    'project_features_entity_id_all_poverty_level_highpoverty',\n",
    "    'project_features_entity_id_all_poverty_level_highestpoverty',\n",
    "    'project_features_entity_id_all_poverty_level_lowpoverty',\n",
    "    'project_features_entity_id_all_poverty_level_moderatepoverty',\n",
    "]\n",
    "\n",
    "traindf_blind = traindf.drop(columns=protected_attributes)\n",
    "testdf_blind = testdf.drop(columns=protected_attributes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This method will get a given model and retrain it on a new dataset, and output the predictions of a test set\n",
    "def retrain_and_test(model, label_col, new_traindf, testdf):\n",
    "    y_train = new_traindf[label_col].values\n",
    "    model.fit(new_traindf.drop(['entity_id','as_of_date',label_col], axis = 1), y_train)\n",
    "    y_pred = model.predict_proba(testdf.drop(['entity_id','as_of_date',label_col], axis = 1))[:,1]\n",
    "    new_preds = testdf[['entity_id','as_of_date',label_col]].copy()\n",
    "    new_preds['predict_proba'] = y_pred\n",
    "    new_preds = new_preds.sort_values('predict_proba', ascending = False).reset_index(drop=True).copy()\n",
    "    new_preds['score'] = new_preds.apply(lambda x: 1.0 if int(x.name)  < 1000 else 0.0, axis=1)\n",
    "    return new_preds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We will now retrain the best performing model without the protected attribute columns\n",
    "new_preds_0 = retrain_and_test(rf, label_col, traindf_blind, testdf_blind)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "j2a_ozRlrAbk"
   },
   "source": [
    "## Before and After: Run Aequitas\n",
    "\n",
    "\n",
    "Let's define an audit util method to use throghout the notebook when audit different models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This method will help auditing the predictions we made with Aequitas\n",
    "def aequitas_audit(input_df, ref_groups_dict):\n",
    "    g = Group()\n",
    "    b = Bias()\n",
    "    xtab, _ = g.get_crosstabs(input_df[['score','label_value'] + list(ref_groups_dict.keys())].copy())\n",
    "    bdf = b.get_disparity_predefined_groups(xtab, input_df, ref_groups_dict)\n",
    "    return bdf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LSNc5sdDrAbk"
   },
   "source": [
    "#### Define the configurations for running the bias audits:\n",
    "\n",
    "    - a dictionary for reference groups in the format attribute:group\n",
    "    - the metrics of interest\n",
    "    - the fairness threshold (the ratio of each group value compared to reference group on a given metric)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Definitions for the audit\n",
    "ref_groups_dict={'poverty_level':'lower', 'metro_type':'suburban_rural', 'teacher_sex':'male'}\n",
    "attributes = ref_groups_dict.keys()\n",
    "metrics = ['tpr']\n",
    "fairness_threshold = 1.3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We will create a dataframe with the predictions and perform an audit on it\n",
    "df_0 = pd.merge(new_preds_0, test_attrdf, how='left', on=['entity_id','as_of_date'], sort=True, copy=True)\n",
    "df_0 = df_0.rename(columns = {label_col:'label_value'})\n",
    "bdf_0 = aequitas_audit(df_0, ref_groups_dict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# The disparities of unawareness in the removed protected attribute.\n",
    "ap.disparity(bdf_0, metrics, 'poverty_level', fairness_threshold = 1.3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The disparities of unawareness in the other protected attributes.\n",
    "ap.disparity(bdf_0, metrics, 'metro_type', fairness_threshold = 1.3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ap.disparity(bdf_0, metrics, 'teacher_sex', fairness_threshold = 1.3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mitigated_precision_0 = df_0.loc[df_0['score']==1]['label_value'].mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mitigated_precision_0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_configs = {\n",
    "    'evals_df':evals_df, \n",
    "    'aequitas_df':aequitas_df,\n",
    "    'attr_col':'poverty_level', \n",
    "    'group_name':'highest',\n",
    "    'performance_col':'model_precision',\n",
    "    'bias_metric':'tpr', \n",
    "    'flip_disparity':True, \n",
    "    'mitigated_tags':['Unawareness'],\n",
    "    'mitigated_bdfs':[bdf_0],\n",
    "    'mitigated_performances':[mitigated_precision_0],\n",
    "    'ylim':None\n",
    "}\n",
    "create_scatter_disparity_performance(**plot_configs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "epoavJ9arAbZ"
   },
   "source": [
    "## <font color=\"red\">Bias Reduction Strategy 1: Re-Sampling</font>\n",
    "### Can resampling approaches help improve the fairness of our models?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "b4NnbfyBrAbh"
   },
   "source": [
    "### Let's now try resampling\n",
    "\n",
    "We can perform three types of resampling:\n",
    "\n",
    "1. Change the training data such that different poverty levels are distributed more uniformly but keep the distribution of labels the same within each poverty level P(poverty_level = highest) = P (poverty_level=lower)\n",
    "\n",
    "\n",
    "2. Change the training data such that different poverty levels have more uniform label distributions P(poverty_level = highest | not funded ) = P(poverty_level=lower | not funded)\n",
    "\n",
    "\n",
    "3. Change both\n",
    "\n",
    "The following method implements these 3 approaches:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "D7ntjXjhrAbh"
   },
   "outputs": [],
   "source": [
    "def sample_groups_equal(df, \n",
    "                  attrdf, \n",
    "                  label_col, \n",
    "                  attribute_col, \n",
    "                  group_of_interest, \n",
    "                  reference_group, \n",
    "                  equal_group_prevalence=True, \n",
    "                  equal_group_size=False, \n",
    "                  random_seed=42):\n",
    "    print(\"Input stats for each group:\\n\")\n",
    "    prev_a, size_a, lp_a,ln_a = get_group_stats(df, attrdf, label_col, attribute_col, group_of_interest)\n",
    "    prev_ref, size_ref, lp_ref,ln_ref = get_group_stats(df, attrdf, label_col, attribute_col, reference_group)\n",
    "    if equal_group_prevalence:\n",
    "        # create largest possible sample based on prevalence ratio (minimize number of label positives removed)\n",
    "        # group_prevalence_ratio = prev_group_of_interest / prev_ref_group\n",
    "        if prev_a < prev_ref:\n",
    "            # create subsample\n",
    "            num_lp_a = len(lp_a)\n",
    "            target_prev = prev_ref\n",
    "            new_num_ln_a = math.floor((num_lp_a / target_prev) - num_lp_a)\n",
    "            ln_a = ln_a.sample(n=new_num_ln_a, replace=False, random_state=random_seed)\n",
    "        elif prev_ref < prev_a:\n",
    "            # create subsample\n",
    "            new_num_lp_ref = len(lp_ref)\n",
    "            target_prev = prev_a\n",
    "            new_num_ln_ref = math.floor((num_lp_ref / target_prev) - num_lp_ref)\n",
    "            ln_ref = ln_ref.sample(n=new_num_ln_ref, replace=False, random_state=random_seed)\n",
    "    adf = pd.concat([lp_a, ln_a], axis=0)\n",
    "    refdf = pd.concat([lp_ref, ln_ref], axis=0)\n",
    "    size_a = len(adf)\n",
    "    size_ref = len(refdf)\n",
    "    if equal_group_size:\n",
    "        # create largest possible sample based on size ratio\n",
    "        if size_a > size_ref:\n",
    "            adf = adf.sample(n=size_ref, replace=False, random_state=random_seed)\n",
    "        elif size_ref > size_a:\n",
    "            refdf = refdf.sample(n=size_a, replace=False, random_state=random_seed)\n",
    "    new_traindf = pd.concat([adf, refdf], axis=0)\n",
    "    print(\"\\n\\nNew sample stats for each group:\\n\")\n",
    "    _ = get_group_stats(new_traindf, attrdf, label_col, attribute_col, group_of_interest)\n",
    "    _ = get_group_stats(new_traindf, attrdf, label_col, attribute_col, reference_group)\n",
    "    return new_traindf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RfUt1Ih9rAbi"
   },
   "source": [
    "### Let's create 3 new different training dataframes by applying the 3 approaches described above"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Mg85rJ2qrAbi"
   },
   "source": [
    "#### Sampling so we get equal group sizes (randomly undersampling largest group)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "PYbIiVU8rAbi"
   },
   "outputs": [],
   "source": [
    "new_traindf_1 = sample_groups_equal(traindf, train_attrdf, label_col, attribute_col, group_of_interest, reference_group, equal_group_prevalence=False, equal_group_size=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rKyTHY9erAbi"
   },
   "source": [
    "#### Sampling so we get equal group prevalence in the two groups (undersampling label negatives of group with lower prevalence)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "STMcTnfqrAbi"
   },
   "outputs": [],
   "source": [
    "new_traindf_2 = sample_groups_equal(traindf, train_attrdf, label_col, attribute_col, group_of_interest, reference_group, equal_group_prevalence=True, equal_group_size=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BAxsvF9CrAbi"
   },
   "source": [
    "#### Sampling so we get both equal group prevalence and group sizes in the two groups "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "iJvuUSVhrAbj"
   },
   "outputs": [],
   "source": [
    "new_traindf_3 = sample_groups_equal(traindf, train_attrdf, label_col, attribute_col, group_of_interest, reference_group, equal_group_prevalence=True, equal_group_size=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kViIXWoGrAbj"
   },
   "source": [
    "#### Rebuild model for each of the resampled training dataframes and predict on the test set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "qUN-7nsUrAbj"
   },
   "outputs": [],
   "source": [
    "new_preds_1 = retrain_and_test(rf, label_col, new_traindf_1, testdf)\n",
    "new_preds_2 = retrain_and_test(rf, label_col, new_traindf_2, testdf)\n",
    "new_preds_3 = retrain_and_test(rf, label_col, new_traindf_3, testdf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HOZTDXu_rAbj"
   },
   "source": [
    "#### Calculate precision at 1000 for each of the 3 approaches"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "jd8_zaSyrAbk"
   },
   "outputs": [],
   "source": [
    "print('Approach 1 Model Precision: ', new_preds_1[new_preds_1['score'] > 0][label_col].mean())\n",
    "print('Approach 2 Model Precision: ', new_preds_2[new_preds_2['score'] > 0][label_col].mean())\n",
    "print('Approach 3 Model Precision: ', new_preds_3[new_preds_3['score'] > 0][label_col].mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "y87lRSXDrAbl"
   },
   "source": [
    "For reference, let's start with looking at the \"best\" model we chose in terms of overall precision at 1,000:\n",
    "\n",
    "#### Now let's start with loading the predictions from the \"best\" model chosen earlier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "FJzS2qXIrAbl"
   },
   "outputs": [],
   "source": [
    "id_col='entity_id'\n",
    "date_col = 'as_of_date'\n",
    "top_k= 1000\n",
    "old_preds = pd.read_csv(DATAPATH + 'predictions_c598fbe93f4c218ac7d325fb478598f1.csv.gz', compression='gzip')\n",
    "old_attrdf = pd.read_csv(DATAPATH + 'test_20121201_20130201_protected.csv.gz', compression='gzip')\n",
    "\n",
    "old_df = pd.merge(old_preds, old_attrdf, how='left', on=[id_col, date_col], sort=True, copy=True)\n",
    "\n",
    "old_df = old_df.sort_values('predict_proba', ascending=False)\n",
    "old_df = old_df.rename(columns = {label_col:'label_value'}) # naming for Aequitas\n",
    "\n",
    "# create a \"score\" column with the predicted class (named \"score\" for use with Aequitas below)\n",
    "old_df['score'] = old_df.apply(lambda x: 1.0 if x.name in old_df.head(top_k).index.tolist() else 0, axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "B5bFcifTrAbl"
   },
   "source": [
    "### Before: Run Aequitas for the \"best\" model chosen earlier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Szqgc5N9rAbs"
   },
   "outputs": [],
   "source": [
    "bdf_old = aequitas_audit(old_df, ref_groups_dict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "D5BLdwH6rAbt"
   },
   "outputs": [],
   "source": [
    "ap.disparity(bdf_old, metrics, 'poverty_level', fairness_threshold)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9GDd3-P2rAbt"
   },
   "source": [
    "### After: Audit for Bias for the new mitigated models  (keeping the attributes, reference groups, bias metric, and tolerance the same as before)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "eLbuoshorAbt"
   },
   "source": [
    "#### Approach 1: Equalizing group sizes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "m1_wC1xyrAbu"
   },
   "outputs": [],
   "source": [
    "df_1 = pd.merge(new_preds_1, test_attrdf, how='left', on=['entity_id','as_of_date'], sort=True, copy=True)\n",
    "df_1 = df_1.rename(columns = {label_col:'label_value'})\n",
    "bdf_1 = aequitas_audit(df_1, ref_groups_dict)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "pRJthZOWrAbu"
   },
   "outputs": [],
   "source": [
    "ap.disparity(bdf_1, metrics, 'poverty_level', fairness_threshold = 1.3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "7cxAs0F1rAbu",
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "ap.disparity(bdf_1, metrics, 'metro_type', fairness_threshold = 1.3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "XGO_exqzrAbu",
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "ap.disparity(bdf_1, metrics, 'teacher_sex', fairness_threshold = 1.3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oRiTI8_MrAbv"
   },
   "source": [
    "#### Approach 2: Equalizing group prevalence"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "wzRiRqM3rAbv"
   },
   "outputs": [],
   "source": [
    "df_2 = pd.merge(new_preds_2, test_attrdf, how='left', on=['entity_id','as_of_date'], sort=True, copy=True)\n",
    "df_2 = df_2.rename(columns = {label_col:'label_value'})\n",
    "bdf_2 = aequitas_audit(df_2, ref_groups_dict)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "dEvgTNKJrAbv"
   },
   "outputs": [],
   "source": [
    "ap.disparity(bdf_2, metrics, 'poverty_level', fairness_threshold = 1.3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "CCfldYQIrAbv"
   },
   "outputs": [],
   "source": [
    "ap.disparity(bdf_2, metrics, 'metro_type', fairness_threshold = 1.3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "knVsyzwrrAbv"
   },
   "outputs": [],
   "source": [
    "ap.disparity(bdf_2, metrics, 'teacher_sex', fairness_threshold = 1.3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fMYpa2fyrAbv"
   },
   "source": [
    "#### Approach 3: Equalizing both group prevalence and group sizes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "di8NttYArAbw"
   },
   "outputs": [],
   "source": [
    "df_3 = pd.merge(new_preds_3, test_attrdf, how='left', on=['entity_id','as_of_date'], sort=True, copy=True)\n",
    "df_3 = df_3.rename(columns = {label_col:'label_value'})\n",
    "bdf_3 = aequitas_audit(df_3, ref_groups_dict)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "QCirzmStrAbx"
   },
   "outputs": [],
   "source": [
    "ap.disparity(bdf_3, metrics, 'poverty_level', fairness_threshold = 1.3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "wzq3aLOCrAbx"
   },
   "outputs": [],
   "source": [
    "ap.disparity(bdf_3, metrics, 'metro_type', fairness_threshold = 1.3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "zF-egkRzrAby"
   },
   "outputs": [],
   "source": [
    "ap.disparity(bdf_3, metrics, 'teacher_sex', fairness_threshold = 1.3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jr3xZierrAby"
   },
   "source": [
    "### Adding to the model selection (tradeoff) graph\n",
    "\n",
    "Finally, let's look at how this new option stacks up against what we plotted in our model selection process:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "E82Q9IkgrAby"
   },
   "outputs": [],
   "source": [
    "mitigated_precision_1 = df_1.loc[df_1['score']==1]['label_value'].mean()\n",
    "mitigated_precision_2 = df_2.loc[df_2['score']==1]['label_value'].mean()\n",
    "mitigated_precision_3 = df_3.loc[df_3['score']==1]['label_value'].mean()\n",
    "\n",
    "plot_configs = {\n",
    "    'evals_df':evals_df, \n",
    "    'aequitas_df':aequitas_df,\n",
    "    'attr_col':'poverty_level', \n",
    "    'group_name':'highest',\n",
    "    'performance_col':'model_precision',\n",
    "    'bias_metric':'tpr', \n",
    "    'flip_disparity':True, \n",
    "    'mitigated_tags':['Resampling 1: eq group size','Resampling 3: eq group prev', 'Resampling 3: eq group prev + size'],\n",
    "    'mitigated_bdfs':[bdf_1, bdf_2, bdf_3],\n",
    "    'mitigated_performances':[mitigated_precision_1, mitigated_precision_2, mitigated_precision_3],\n",
    "    'ylim':None\n",
    "}\n",
    "create_scatter_disparity_performance(**plot_configs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "37nxDqY-rAbz"
   },
   "source": [
    "## <font color=red>Bias Reduction Strategy 2: Regularization (using FairGBM and the Fairlearn package)</font>\n",
    "### <font color=red>In-Processing Fairness Improvement</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "lzh5ZyxdrAbz"
   },
   "source": [
    "### 1. Import packages and data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "TXPOZUPGrAbz"
   },
   "outputs": [],
   "source": [
    "# Let's use the methods \n",
    "from fairlearn.reductions import ExponentiatedGradient, GridSearch, DemographicParity, TruePositiveRateDifference\n",
    "from fairlearn.metrics import selection_rate_group_summary"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2CRf_3SArAbz"
   },
   "source": [
    "### What has already happened?\n",
    "\n",
    "We've already cleaned data, generated features, created train-test sets, built 1000s of models on each training set and scored each test set with them, and calculated various evaluation metrics. We then used these results to pick a \"best\" model in terms of performance on the \"accuracy\" metric we care about: **Precision at the top 1000** (corresponding to our goal of selecting 1000 project submissions that are most likely to not get funded in order to prioritize resource allocation).\n",
    "\n",
    "When we audited this selected model with Aequitas, however, we found biases across many attributes, including the poverty level of the schools. Here, we explore a method of using in-processing to train a fairness-aware classifier in order to reduce this bias."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BoTdEbWPrAbz"
   },
   "source": [
    "### <font color=green>FairLearn - a reductions approach</font>\n",
    "\n",
    "[Paper](https://arxiv.org/pdf/1803.02453.pdf): _A Reductions Approach to Fair Classification_, 2018\n",
    "\n",
    "> We present a systematic approach for achievingfairness in a binary classification setting. Whilewe focus on two well-known quantitative defini-tions of fairness, our approach encompasses manyother  previously  studied  definitions  as  specialcases. The key idea is to __reduce fair classification__ to a __sequence  of  cost-sensitive__  classification problems, whose solutions yield a randomized classifier with the __lowest (empirical) error__ subject to  the  __desired  constraints__.   We  introduce  two reductions that work for any representation of the cost-sensitive  classifier  and  compare  favorably to prior baselines on a variety of data sets, while overcoming several of their disadvantages.\n",
    "\n",
    "[FairLearn Documentation](https://fairlearn.github.io/user_guide/mitigation.html#id17)\n",
    "\n",
    "\n",
    "\n",
    "### TLDR; \n",
    "\n",
    "- This approach poses Fair Learning as a constrained optimization problem: minimize the empirical error, subject to linear constraints of the fairness (e.g., TPR difference, demographic parity).\n",
    "- Solve the constrained optimization as a __cost-sensitive__ classification problem.\n",
    "- Obtain a __randomized classifier__, which implies they will create multiple base estimators.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jkZwmXderAb0"
   },
   "source": [
    "### Load train and test matrices as well as protected attributes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Wf3bIZRCrAb0"
   },
   "outputs": [],
   "source": [
    "traindf = pd.read_csv(DATAPATH + 'train_20120501_20120801.csv.gz', compression='gzip')\n",
    "testdf = pd.read_csv(DATAPATH + 'test_20121201_20130201.csv.gz', compression='gzip')\n",
    "train_attrdf = pd.read_csv(DATAPATH + 'train_20120501_20120801_protected.csv.gz', compression='gzip')\n",
    "test_attrdf = pd.read_csv(DATAPATH + 'test_20121201_20130201_protected.csv.gz', compression='gzip')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Zs3itOxgrAb0"
   },
   "source": [
    "### Set up some parameters we'll need below and create matrices"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "toLhuG9xrAb0"
   },
   "outputs": [],
   "source": [
    "label_col = 'quickstart_label'\n",
    "date_col = 'as_of_date'\n",
    "id_col = 'entity_id'\n",
    "attr_col = 'poverty_level'\n",
    "exclude_cols = [label_col, date_col, id_col]\n",
    "\n",
    "top_k = 1000\n",
    "\n",
    "# aequitas parameters\n",
    "metrics = ['tpr']\n",
    "disparity_threshold = 1.3\n",
    "protected_attribute_ref_group = {attr_col:'lower'}\n",
    "\n",
    "\n",
    "X_train, y_train, A_train = traindf[[c for c in traindf.columns if c not in exclude_cols]].values, traindf[label_col].values, train_attrdf[[attr_col]]\n",
    "X_test,   y_test,   A_test   = testdf[[c for c in testdf.columns if c not in exclude_cols]].values,   testdf[label_col].values  , test_attrdf[[attr_col]]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yGan6jWdrAb0"
   },
   "source": [
    "### Setting up a fairness-improving classifier\n",
    "\n",
    "To account fairness during model training, we'll use the **Exponentiated Gradient** provided by the `fairlearn` module.\n",
    "\n",
    "\n",
    "Its hyperparameters are: \n",
    "- `estimator`: an estimator that implements the methods `fit(X, y, sample_weight)` and `predict(X)`.\n",
    "- `constraints`: fairness constraints.\n",
    "- `eps: float`: fairness threshold, i.e., how much constraint violation we support (defaults to 0.01). \n",
    "- `T: int`: maximum number of iterations (defaults to 50).\n",
    "- `nu: float`: convergence threshold for duality gap (defaults to None).\n",
    "- `eta_0: float`: initial learning rate (defaults to 2).\n",
    "- `run_linprog_step: bool`: whether to apply saddle point optimization to the convex hull of classifiers obtained so far, after each exponentiated gradient step (defaults to True)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "NxeWsgoSrAb1"
   },
   "outputs": [],
   "source": [
    "# NOTE: Exponentiated Gradient has a stoachastic component\n",
    "np.random.seed(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vqmYPIWirAb1"
   },
   "source": [
    "Notice that we're using `TruePositiveRateDifference` for our fairness constraint here since we care about equalizing **recall** (aka **tpr** aka **equality of opportunity**) across our subgroups:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "OEXSkKkIrAb1"
   },
   "outputs": [],
   "source": [
    "# Step 1. Define the constraint\n",
    "constraint = TruePositiveRateDifference()\n",
    "\n",
    "# Step 2. Define the base estimator (any estimator providing 'fit' and 'predict')\n",
    "# Note: we could have used other algorithm such as logistic regression or random forest\n",
    "base_estimator = DecisionTreeClassifier(max_depth=20, min_samples_leaf=10)\n",
    "\n",
    "# Step 3. Define the bias reducer algorithm you want to apply\n",
    "bias_reducer = ExponentiatedGradient(base_estimator, constraint, T=50)\n",
    "\n",
    "# Step 4. Fit the data (and provide the sensitive attributes)\n",
    "bias_reducer.fit(X_train, y_train, sensitive_features=A_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xEIWe-LmrAb1"
   },
   "source": [
    "### Predict on our test set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "9fTP_bLCrAb2"
   },
   "outputs": [],
   "source": [
    "# Step 5. Use the mitigator to make predictions \n",
    "y_pred_4 = bias_reducer.predict(X_test)\n",
    "new_preds_4 = testdf[['entity_id','as_of_date','quickstart_label']].copy()\n",
    "new_preds_4['score'] = y_pred_4"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "G7AYAIpdrAb2"
   },
   "source": [
    "### Look at the output\n",
    "Notice that unlike many classifiers, the `ExponentiatedGradient` doesn't have a method for predicting a continuous score, just predicted classes of 0 or 1. How many projects did this model predict are at risk of going unfunded (that is, predicted class of 1)?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "d764wl5yrAb2"
   },
   "outputs": [],
   "source": [
    "new_preds_4['score'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pM4yuF7VrAb2"
   },
   "source": [
    "It looks like about 6,500 projects are predicted as being at risk of going unfunded by this classifier, but unfortunately our program is resource-constrained and can only help 1,000 of them. At this point, if we wanted to pick out the 1,000 highest-risk projects (subject to our fairness constraint), we're a bit stuck: **the classifier doesn't give us any method for distinguishing higher-risk vs lower-risk projects!**\n",
    "\n",
    "Given this limitation, we might posit that a reasonable approach would be to pick 1,000 projects to intervene with from among these 6,500. Let's see what would happen if we did that..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rgWF9BsirAb3"
   },
   "source": [
    "### Before and After: Run Aequitas\n",
    "\n",
    "For reference, let's start with looking at the \"best\" model we chose in terms of overall precision at 1,000:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1xDzw1ubrAb3"
   },
   "source": [
    "#### Load the predictions from the \"best\" model chosen earlier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "jaGaIYzUrAb3"
   },
   "outputs": [],
   "source": [
    "old_preds = pd.read_csv(DATAPATH + 'predictions_c598fbe93f4c218ac7d325fb478598f1.csv.gz', compression='gzip')\n",
    "old_attrdf = pd.read_csv(DATAPATH + 'test_20121201_20130201_protected.csv.gz', compression='gzip')\n",
    "\n",
    "old_df = pd.merge(old_preds, old_attrdf, how='left', on=[id_col, date_col], sort=True, copy=True)\n",
    "\n",
    "old_df = old_df.sort_values('predict_proba', ascending=False)\n",
    "old_df = old_df.rename(columns = {label_col:'label_value'}) # naming for Aequitas\n",
    "\n",
    "# create a \"score\" column with the predicted class (named \"score\" for use with Aequitas below)\n",
    "old_df['score'] = old_df.apply(lambda x: 1.0 if x.name in old_df.head(top_k).index.tolist() else 0, axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RkXsZCq0rAb3"
   },
   "source": [
    "#### Before: Run Aequitas for the \"best\" model chosen earlier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "kH7af0IerAb4"
   },
   "outputs": [],
   "source": [
    "def aequitas_audit(input_df, ref_groups_dict):\n",
    "    g = Group()\n",
    "    b = Bias()\n",
    "    xtab, _ = g.get_crosstabs(input_df[['score','label_value'] + list(ref_groups_dict.keys())].copy())\n",
    "    bdf = b.get_disparity_predefined_groups(xtab, input_df, ref_groups_dict)\n",
    "    return bdf\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "-HyyEUvnrAb4"
   },
   "outputs": [],
   "source": [
    "bdf_old = aequitas_audit(old_df, protected_attribute_ref_group)\n",
    "\n",
    "ap.disparity(bdf_old, metrics, attr_col, fairness_threshold=disparity_threshold)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "OHff1L-UrAb4"
   },
   "source": [
    "#### After: Run Aequitas for the new, fairness-aware model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "IJ2_VF0srAb4"
   },
   "source": [
    "Remember here that we would choose 1,000 at random from the 6,500 with predicted class 1, so the expected value for the recall disparity of this randomly-selected set would just be the value of full set (that is, the recall of each subgroup would, on average, be proportionally lower in the sub-sample):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "xJw1h7_8rAb4"
   },
   "outputs": [],
   "source": [
    "df_4 = pd.merge(new_preds_4, test_attrdf, how='left', on=['entity_id','as_of_date'], sort=True, copy=True)\n",
    "df_4 = df_4.rename(columns = {label_col:'label_value'})\n",
    "\n",
    "bdf_4 = aequitas_audit(df_4, ref_groups_dict)\n",
    "\n",
    "ap.disparity(bdf_4, metrics, attr_col, fairness_threshold=disparity_threshold)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4OpNRgzArAb5"
   },
   "source": [
    "That looks pretty good! The new model appears to have reduced the disparity across poverty levels considerably relative to what we saw when training a model without a fairness constraint and choosing based on precision alone.\n",
    "\n",
    "However, the natural question here is where there is a fairness-accuracy trade-off here: What cost did we incur in terms of model performance, that is overall precision?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "f77b9mpTrAb5"
   },
   "source": [
    "#### Precision of the \"best\" model chosen earlier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "xZyYHBLurAb5"
   },
   "outputs": [],
   "source": [
    "old_df.loc[old_df['score']==1]['label_value'].mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-06IOWldrAb5"
   },
   "source": [
    "#### Precision of the new, fairness-aware model\n",
    "\n",
    "As above, we would be sampling from the 6,500 down to 1,000 but the expected value of precision in this sample would just be the mean label value in the full population:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "cTjI7AYyrAb5"
   },
   "outputs": [],
   "source": [
    "df_4.loc[df_4['score']==1]['label_value'].mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "v573tZWMrAb6"
   },
   "source": [
    "So, compared to the old model, this method has resulted in **quite a large trade-off in model performance to acheive fairness**. This is certainly in part a result of the lack of flexibility in the method not allowing us to provide a score threshold or top k size that we're interested in equalizing our fairness metric around and instead using a built-in threshold that yields 6,500 predicted positives when we're only able to intervene on 1,000. Unfortunately, this sort of inflexibility appears to be a common attribute of many in-processing methods available today.\n",
    "\n",
    "For context, the overall base rate (`df['label_value'].mean()`) is 0.338, so the drop-off in precision here is about half way from our previous model to simply choosing at random."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "CGpzioL0rAb6"
   },
   "source": [
    "### Adding to model selection\n",
    "\n",
    "Finally, let's look at how this new option stacks up against what we plotted in our model selection process:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "9fdo113irAb6"
   },
   "outputs": [],
   "source": [
    "mitigated_precision_4 = df_4.loc[df_4['score']==1]['label_value'].mean()\n",
    "plot_configs = {\n",
    "    'evals_df':evals_df, \n",
    "    'aequitas_df':aequitas_df,\n",
    "    'attr_col':'poverty_level', \n",
    "    'group_name':'highest',\n",
    "    'performance_col':'model_precision',\n",
    "    'bias_metric':'tpr', \n",
    "    'flip_disparity':True, \n",
    "    'mitigated_tags':['Fairlearn Reductions'],\n",
    "    'mitigated_bdfs':[bdf_4], \n",
    "    'mitigated_performances':[mitigated_precision_4], \n",
    "    'ylim':None\n",
    "}\n",
    "create_scatter_disparity_performance(**plot_configs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using FairGBM + Hyperparameter optimization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the following cells we will setup hyperparameter optimization, with FairGBM models.\n",
    "\n",
    "FairGBM is an algorithm based on LightGBM with fairness constraints applied to the loss function. It was presented earlier this year in ICLR. [link](https://openreview.net/pdf?id=x-mXzBgCX3a)\n",
    "\n",
    "### <font color=green>FairGBM - Gradient Boosting with Fairness Constraints</font>\n",
    "\n",
    "> Tabular data is prevalent in many high-stakes domains, such as financial services or public policy. Gradient Boosted Decision Trees (GBDT) are popular in these settings due to their scalability, performance, and low training cost. While fairness in these domains is a foremost concern, existing in-processing Fair ML methods are either incompatible with GBDT, or incur in significant performance losses while taking considerably longer to train. We present FairGBM, a dual ascent learning framework for training GBDT under fairness constraints, with little to no impact on predictive performance when compared to unconstrained GBDT. Since observational fairness metrics are non-differentiable, we propose smooth convex error rate proxies for common fairness criteria, enabling gradient-based optimization using a ``proxy-Lagrangian'' formulation. Our implementation shows an order of magnitude speedup in training time relative to related work, a pivotal aspect to foster the widespread adoption of FairGBM by real-world practitioners.\n",
    "\n",
    "### TLDR; \n",
    "\n",
    "- Similar to Fairlearn, this approach poses Fair Learning as a constrained optimization problem.\n",
    "- The method uses the algorithm of boosting to adjust the errors of certain instances depending on the constraints.\n",
    "- We obtain a single final GBM classifier. \n",
    "\n",
    "Due to having only a model (and the computation being easier than EG), it is possible to train multiple models quickly. We will use a wrapper of `Optuna` for the purpose.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We define the hyperparameters to search on in this dictionary.\n",
    "\n",
    "HYPERPARAM_SPACE = {\n",
    "    \"FairGBM\": {\n",
    "        \"classpath\": \"fairgbm.FairGBMClassifier\",\n",
    "        \"kwargs\": {\n",
    "            \"boosting_type\": \"gbdt\",\n",
    "            \"n_estimators\": 26,\n",
    "            \"num_leaves\": 63,\n",
    "            \"min_child_samples\": 96,\n",
    "            \"max_depth\": 72,\n",
    "            \"learning_rate\": 0.1380440731273914,\n",
    "            \"constraint_type\": \"tpr\",\n",
    "            \"multiplier_learning_rate\": {\n",
    "                \"type\": \"float\",\n",
    "                \"range\": [0.01, 1.0],\n",
    "                \"log\": True,\n",
    "            },\n",
    "            \"proxy_margin\": {\n",
    "                \"type\": \"float\",\n",
    "                \"range\": [0.01, 1.0],\n",
    "                \"log\": True,\n",
    "            },\n",
    "            \"global_constraint_type\": \"fpr,fnr\",\n",
    "            \"global_target_fnr\": {\n",
    "                \"type\": \"float\",\n",
    "                \"range\": [0.05, 0.25],\n",
    "                \"log\": False,\n",
    "            },\n",
    "            \"global_target_fpr\": {\n",
    "                \"type\": \"float\",\n",
    "                \"range\": [0.02, 0.15],\n",
    "                \"log\": False,\n",
    "            }\n",
    "        }\n",
    "    }\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from hpt.tuner import ObjectiveFunction, OptunaTuner\n",
    "\n",
    "obj_func = ObjectiveFunction(\n",
    "    X_train, y_train, X_test, y_test,\n",
    "    hyperparameter_space=HYPERPARAM_SPACE,\n",
    "    eval_metric=\"precision\",\n",
    "    other_eval_metric=\"tpr_ratio\",  # Fairness metric (Equal Opportunity)\n",
    "    s_train=(A_train[\"poverty_level\"]==\"highest\").astype(int).values,  # Casting to numpy and as int (required by package)\n",
    "    s_val=(A_test[\"poverty_level\"]==\"highest\").astype(int).values,\n",
    "    ppr=1000/17677,  # We have to define the target predictive positive rate (which is 1000 of the total instances in the dataset)\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tuner = OptunaTuner(\n",
    "    objective_function=obj_func,\n",
    "    direction=\"maximize\",\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Just to silence the logging of the method\n",
    "import logging, sys\n",
    "logging.disable(sys.maxsize)\n",
    "\n",
    "tuner.optimize(n_trials=50, n_jobs=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get the best model\n",
    "bias_reducer = obj_func.reconstruct_model(obj_func.best_trial)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred_5 = bias_reducer.predict_proba(X_test)\n",
    "new_preds_5 = testdf[['entity_id','as_of_date','quickstart_label']].copy()\n",
    "new_preds_5['score'] = y_pred_5[:, 1]\n",
    "threshold = new_preds_5['score'].sort_values().values[-1000]\n",
    "new_preds_5['score'] = (new_preds_5['score'] >= threshold).astype(int)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_5 = pd.merge(new_preds_5, test_attrdf, how='left', on=['entity_id','as_of_date'], sort=True, copy=True)\n",
    "df_5 = df_5.rename(columns = {label_col:'label_value'})\n",
    "\n",
    "bdf_5 = aequitas_audit(df_5, ref_groups_dict)\n",
    "\n",
    "ap.disparity(bdf_5, metrics, attr_col, fairness_threshold=disparity_threshold)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_5.loc[df_5['score']==1]['label_value'].mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Now let's add to the plot:\n",
    "\n",
    "mitigated_precision_5 = df_5.loc[df_5['score']==1]['label_value'].mean()\n",
    "plot_configs = {\n",
    "    'evals_df':evals_df, \n",
    "    'aequitas_df':aequitas_df,\n",
    "    'attr_col':'poverty_level', \n",
    "    'group_name':'highest',\n",
    "    'performance_col':'model_precision',\n",
    "    'bias_metric':'tpr', \n",
    "    'flip_disparity':True, \n",
    "    'mitigated_tags':['FairGBM'],\n",
    "    'mitigated_bdfs':[bdf_5], \n",
    "    'mitigated_performances':[mitigated_precision_5], \n",
    "    'ylim':None\n",
    "}\n",
    "create_scatter_disparity_performance(**plot_configs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "J3BHUfzxrAb6"
   },
   "source": [
    "## <font color=red>Bias Reduction Strategy 3: Post-Hoc Disparity Mitigation</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HrhZcuD9rAb6"
   },
   "source": [
    "### What has already happened?\n",
    "\n",
    "We've already cleaned data, generated features, created train-test sets, built 1000s of models on each training set and scored each test set with them, and calculated various evaluation metrics. We then used these results to pick a \"best\" model in terms of performance on the \"accuracy\" metric we care about: **Precision at the top 1000** (corresponding to our goal of selecting 1000 project submissions that are most likely to not get funded in order to prioritize resource allocation).\n",
    "\n",
    "When we audited this selected model with Aequitas, however, we found biases across many attributes, including the poverty level of the schools. Here, we explore a method of using post-hoc disparity mitigation to reduce this bias in the selected model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jbMduPAgrAb6"
   },
   "source": [
    "### <font color=green>Intro to Post-Hoc Bias Mitigation</font>\n",
    "\n",
    "![Diagram of Post-Hoc Adjustments](post_hoc_adj.png \"Post-Hoc Adjustments\")\n",
    "\n",
    "One approach to improving the fairness of our model is to make post-hoc adjustments to the thresholds used for each subgroup to choose the 1,000 projects on which to intervene. Because our fairness metric here (**recall** aka **tpr** aka **equality of opportunity**) is monatonically increasing with the depth of the score, we should be able to find score thresholds for each subgroup that will equalize this metric across the groups, subject to the constraint that we want to choose a total of 1,000 projects for our intervation.\n",
    "\n",
    "In short, here's how this will work (see the references below for a more detailed discussion):\n",
    "1. Train the model as usual on a training set, predict scores on a test set\n",
    "2. Split this test set by subgroups on our protected attribute (here, poverty level)\n",
    "3. Sort each subgroup by score and calculate the cumulative tpr/recall up to and including each row in the set, storing this \"rolling within-subgroup recall\" value\n",
    "4. Recombine the subgroups, and sort the entire set by this new value\n",
    "5. Take the top 1,000 projects from this re-ordered list and use it to calculate \"top k\" sizes for each subgroup that equalize recall\n",
    "6. Then, on a future test set, use these calculated subgroup list sizes to assess the impact of disparities and overall precision\n",
    "\n",
    "References:\n",
    "- Hardt, et al, [Equality of Opportunity in Supervised Learning](http://papers.nips.cc/paper/6373-equality-of-opportunity-in-supervised-learning)\n",
    "- Rodolfa, et al, [Case Study: Predictive Fairness to Reduce Misdemeanor Recidivism Through Social Service Interventions](https://dl.acm.org/doi/abs/10.1145/3351095.3372863?casa_token=zc196JJrqkkAAAAA:bPmqmKrA91esJhIxHPT4K1crWWb5JGcflVFDkTgODctMzLpUX50_56Kyyh4NJ2GTd_QSydqhNpjT)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6xdNWoWRrAb7"
   },
   "source": [
    "### Load the train, test, and protected attributes from the first split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "qp0sSsSerAb7"
   },
   "outputs": [],
   "source": [
    "split1_traindf = pd.read_csv(DATAPATH + 'train_20111101_20120201.csv.gz', compression='gzip')\n",
    "split1_testdf = pd.read_csv(DATAPATH + 'test_20120601_20120801.csv.gz', compression='gzip')\n",
    "split1_attrdf = pd.read_csv(DATAPATH + 'test_20120601_20120801_protected.csv.gz', compression='gzip')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "EzJSFvOgrAb7"
   },
   "source": [
    "Let's just take a quick look at the data to make sure it makes sense:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "MLYr-KKJrAb7"
   },
   "outputs": [],
   "source": [
    "split1_traindf.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "T2_aZeR_rAb7"
   },
   "source": [
    "### Set up some parameters we'll need below\n",
    "\n",
    "Note that the classifier type and hyperparameters here are from the best-performing model we chose above based on precision on the top 1000."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "8H8fzeVFrAb7"
   },
   "outputs": [],
   "source": [
    "hyperparameters = {\n",
    "    'criterion': 'gini',\n",
    "    'max_depth': 30,\n",
    "    'max_features': 'sqrt',\n",
    "    'min_samples_leaf': 44,\n",
    "    'min_samples_split': 3,\n",
    "    'n_estimators': 87,\n",
    "    'n_jobs': -1,\n",
    "    'random_state': 213500298\n",
    "}\n",
    "clf = RandomForestClassifier(**hyperparameters)\n",
    "\n",
    "top_k = 1000\n",
    "\n",
    "label_col = 'quickstart_label'\n",
    "entity_col = 'entity_id'\n",
    "date_col = 'as_of_date'\n",
    "exclude_cols = [label_col, entity_col, date_col] # columns to exclude from the X matrices for the classifier\n",
    "\n",
    "protected_attribute_col = 'poverty_level'\n",
    "\n",
    "# Parameters for Aequitas\n",
    "\n",
    "metrics = ['tpr']\n",
    "disparity_threshold = 1.3\n",
    "protected_attribute_ref_group = {protected_attribute_col:'lower'}\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zIbdJJ9erAb8"
   },
   "source": [
    "### Train the model and predict on the test set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "v-1FMnEDrAb8"
   },
   "outputs": [],
   "source": [
    "# train\n",
    "y_train = split1_traindf[label_col].values\n",
    "clf.fit(split1_traindf.drop(exclude_cols, axis = 1), y_train)\n",
    "\n",
    "# test set predictions\n",
    "split1_preds = split1_testdf[[entity_col, date_col, label_col]].copy()\n",
    "split1_preds['predict_proba'] = clf.predict_proba(split1_testdf.drop(exclude_cols, axis = 1))[:,1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6Z2cNQEzrAb8"
   },
   "source": [
    "Let's take a quick look at the predictions to make sure they look good:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ZQzMEARlrAb8"
   },
   "outputs": [],
   "source": [
    "split1_preds.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PghyKbGBrAb8"
   },
   "source": [
    "### Combine predictions with protected attributes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "E26kYW0VrAb8"
   },
   "outputs": [],
   "source": [
    "df = pd.merge(split1_preds, split1_attrdf, how='left', on=[entity_col,date_col], sort=True, copy=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "nb0GFlGlrAb9"
   },
   "source": [
    "### Sort by score, then split by protected attribute (poverty level)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "SjDW283hrAb9"
   },
   "outputs": [],
   "source": [
    "protected_attribute_groups = df[protected_attribute_col].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "xRmXcAjtrAb9"
   },
   "outputs": [],
   "source": [
    "df = df.sort_values('predict_proba', ascending=False)\n",
    "subgroup_dfs = []\n",
    "for grp in protected_attribute_groups:\n",
    "    subgroup_dfs.append(df[df[protected_attribute_col]==grp].copy())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9GhbKz0LrAb9"
   },
   "source": [
    "### Calculate within-subgroup cumulative recall\n",
    "\n",
    "Here, we calculate the recall up to and including each row within the highest-poverty and lower-poverty subsets of the test set. Doing this allows us to recombine and sort the sets in a way that will let find recall-equalizing \"top k\" list sizes for each subgroup."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "IHFVwb8hrAb9"
   },
   "outputs": [],
   "source": [
    "for subgrp_df in subgroup_dfs:\n",
    "    subgrp_df['cumsum_recall'] = subgrp_df[label_col].cumsum() / subgrp_df[label_col].sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "McJ2GzYVrAb-"
   },
   "source": [
    "### Recombine the subgroup sets and sort by this cumulative recall"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "b64e0VEirAb-"
   },
   "outputs": [],
   "source": [
    "recall_df = pd.concat(subgroup_dfs, axis=0).sort_values('cumsum_recall', ascending=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7ltEQv_LrAb-"
   },
   "source": [
    "### Find subgroup sizes, holding the overall list size (1000) constant\n",
    "\n",
    "Now we can simply threshold this re-sorted list by `top_k` (here, 1000) to identify how many individuals from each group we should apply in the future.\n",
    "\n",
    "Notice here that each subgroup will still be ordered by their predicted score, but the scores will no longer be perfectly ordered across subgroups."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "p0S6xAbJrAb_"
   },
   "outputs": [],
   "source": [
    "new_pp = recall_df.head(top_k).copy()\n",
    "new_pp.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "vi0iKqfTrAcB"
   },
   "outputs": [],
   "source": [
    "new_pp[protected_attribute_col].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0JQqE2u6rAcB"
   },
   "source": [
    "Let's just store this to re-use below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "eH5QmW0wrAcC"
   },
   "outputs": [],
   "source": [
    "subgroup_k = {} \n",
    "for grp in protected_attribute_groups:\n",
    "  subgroup_k[grp] = new_pp[protected_attribute_col].value_counts()[grp]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FssBodUHrAcC"
   },
   "source": [
    "### Apply these subgroup-specific sizes to future test set data\n",
    "\n",
    "Now we have calculated the number of projects we need to select from each poverty level, we can apply these to the most recent split to assess how well this method reduces the recall disparities we saw initially and whether this has any impact on the overall precision of the model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "e6wcWvFhrAcC"
   },
   "source": [
    "### Load the predictions and protected attributes from the future test set\n",
    "\n",
    "Note that the predictions here correspond to the same model + hyperparameters we specified above"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "cIYTYLyErAcD"
   },
   "outputs": [],
   "source": [
    "split2_preds = pd.read_csv(DATAPATH + 'predictions_c598fbe93f4c218ac7d325fb478598f1.csv.gz', compression='gzip')\n",
    "split2_attrdf = pd.read_csv(DATAPATH + 'test_20121201_20130201_protected.csv.gz', compression='gzip')\n",
    "\n",
    "df2 = pd.merge(split2_preds, split2_attrdf, how='left', on=[entity_col, date_col], sort=True, copy=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "aBpVLOZqrAcD"
   },
   "source": [
    "Take a quick look to make sure the data loaded without any issue:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "lUtxFwM8rAcD"
   },
   "outputs": [],
   "source": [
    "df2.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "dwcOovWYrAcE"
   },
   "source": [
    "### Split the test set by poverty level to apply the thresholds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "cuT-p3fPrAcE"
   },
   "outputs": [],
   "source": [
    "df2 = df2.sort_values('predict_proba', ascending=False)\n",
    "new_subgroup_dfs = {}\n",
    "for grp in protected_attribute_groups:\n",
    "    new_subgroup_dfs[grp] = df2[df2[protected_attribute_col]==grp].copy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pZHFdav5rAcE"
   },
   "source": [
    "### Choose the number of projects from each subgroup found above\n",
    "\n",
    "Notice here that we're choosing the \"top k\" individuals within each subgroup based on their predicted score -- in a deployment, we wouldn't know the true labels to calculate recall values, which is why we had to go one step back in time to find these group sizes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "7dPkwdcVrAcF"
   },
   "outputs": [],
   "source": [
    "pp_dfs = []\n",
    "for grp in protected_attribute_groups:\n",
    "    pp_dfs.append(new_subgroup_dfs[grp].head(subgroup_k[grp]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "am65wwNwrAcF"
   },
   "source": [
    "### Recombine and create a predicted class label for this resulting set\n",
    "That is, 1,000 projects with a label 1 chosen by this process and 0 otherwise"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "eZ-s1JQ9rAcF"
   },
   "outputs": [],
   "source": [
    "new_pp2 = pd.concat(pp_dfs, axis=0).sort_values('predict_proba', ascending=True)\n",
    "\n",
    "mitigated_df = df2.copy()\n",
    "mitigated_df = mitigated_df.rename(columns = {label_col:'label_value'}) # naming for Aequitas\n",
    "\n",
    "# create a \"score\" column with the predicted class (named \"score\" for use with Aequitas below)\n",
    "mitigated_df['score'] = mitigated_df.apply(lambda x: 1.0 if x.name in new_pp2.index.tolist() else 0, axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6CgOyZhnrAcF"
   },
   "source": [
    "for comparison, let's also look at the unmitigated result again..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ZHleARoErAcF"
   },
   "outputs": [],
   "source": [
    "unadjusted_df = df2.sort_values('predict_proba', ascending=False).copy()\n",
    "unadjusted_df = unadjusted_df.rename(columns = {label_col:'label_value'}) # naming for Aequitas\n",
    "\n",
    "# create a \"score\" column with the predicted class (named \"score\" for use with Aequitas below)\n",
    "unadjusted_df['score'] = unadjusted_df.apply(lambda x: 1.0 if x.name in unadjusted_df.head(top_k).index.tolist() else 0, axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HO4fVQpdrAcF"
   },
   "source": [
    "### Running Aequitas - Before and After"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "mSWlF1i6rAcG"
   },
   "source": [
    "#### For the original score, without post-hoc adjustment for equity"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ybg9apkhrAcG"
   },
   "outputs": [],
   "source": [
    "bdf_unadjusted = aequitas_audit(unadjusted_df, protected_attribute_ref_group)\n",
    "ap.disparity(bdf_unadjusted, metrics, protected_attribute_col, fairness_threshold=disparity_threshold)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-N-WtYo7rAcG"
   },
   "source": [
    "#### For the score, with post-hoc disparity mitigation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "p7uQQPtxrAcG"
   },
   "outputs": [],
   "source": [
    "bdf_mitigated = aequitas_audit(mitigated_df, ref_groups_dict)\n",
    "\n",
    "ap.disparity(bdf_mitigated, metrics, protected_attribute_col, fairness_threshold=disparity_threshold)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-tVKDd-ErAcH"
   },
   "source": [
    "So, it looks the post-hoc adjustments have actually manage to mitigate the existing disparity pretty well (perhaps even over-shooting somewhat, though still within our fairness threshold of 1.3).\n",
    "\n",
    "However, the natural question here is where there is a fairness-accuracy trade-off here: What cost did we incur in terms of model performance, that is overall precision?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MzvV4AqBrAcH"
   },
   "source": [
    "#### Before: Precision of the original, unadjusted score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "4AhwXfukrAcH"
   },
   "outputs": [],
   "source": [
    "unadjusted_df.loc[unadjusted_df['score']==1]['label_value'].mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "reI9ombCrAcH"
   },
   "source": [
    "#### After: Precision of the new, disparity-mitigated score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "y4ASkfUZrAcH"
   },
   "outputs": [],
   "source": [
    "mitigated_df.loc[mitigated_df['score']==1]['label_value'].mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tFawIxgVrAcI"
   },
   "source": [
    "Somewhat surprisingly, we actually don't seem to see any trade-off with the disparity mitigation here!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "t2fr1FSxrAcI"
   },
   "source": [
    "### Adding to the model selection (tradeoff) graph\n",
    "\n",
    "Finally, let's look at how this new option stacks up against what we plotted in our model selection process:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "q04Ql5LgrAcI"
   },
   "outputs": [],
   "source": [
    "mitigated_precision_6 = mitigated_df.loc[mitigated_df['score']==1]['label_value'].mean()\n",
    "plot_configs = {\n",
    "    'evals_df':evals_df, \n",
    "    'aequitas_df':aequitas_df,\n",
    "    'attr_col':'poverty_level', \n",
    "    'group_name':'highest',\n",
    "    'performance_col':'model_precision',\n",
    "    'bias_metric':'tpr', \n",
    "    'flip_disparity':True, \n",
    "    'mitigated_tags':['Equalizing Recall'],\n",
    "    'mitigated_bdfs':[bdf_mitigated], \n",
    "    'mitigated_performances':[mitigated_precision_6], \n",
    "    'ylim':None\n",
    "}\n",
    "create_scatter_disparity_performance(**plot_configs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2Be3njfMrAcI"
   },
   "source": [
    "# Wrapping-up\n",
    "\n",
    "## Overview of all mitigation strategies on the model selection (tradeoff) graph\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "BZDyviDOrAcI"
   },
   "outputs": [],
   "source": [
    "plot_configs = {\n",
    "    'evals_df':evals_df, \n",
    "    'aequitas_df':aequitas_df,\n",
    "    'attr_col':'poverty_level', \n",
    "    'group_name':'highest',\n",
    "    'performance_col':'model_precision',\n",
    "    'bias_metric':'tpr', \n",
    "    'flip_disparity':True, \n",
    "    'mitigated_tags':['Unawareness', 'Resampling 1: eq group size','Resampling 3: eq group prev', 'Resampling 3: eq group prev + size','Fairlearn Reductions', 'FairGBM', 'Equalizing Recall'],\n",
    "    'mitigated_bdfs':[bdf_0, bdf_1, bdf_2, bdf_3, bdf_4, bdf_5, bdf_mitigated], \n",
    "    'mitigated_performances':[mitigated_precision_0, mitigated_precision_1, mitigated_precision_2, mitigated_precision_3,mitigated_precision_4, mitigated_precision_5, mitigated_precision_6], \n",
    "    'ylim':None\n",
    "}\n",
    "create_scatter_disparity_performance(**plot_configs)"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "name": "bias_reduction.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}