docs/source/notebooks/DiCE_feature_importances.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Estimating local and global feature importance scores using DiCE\n",
    "\n",
    "Summaries of counterfactual examples can be used to estimate importance of features. Intuitively, a feature that is changed more often to generate a proximal counterfactual is an important feature. We use this intuition to build a feature importance score. \n",
    "\n",
    "This score can be interpreted as a measure of the **necessity** of a feature to cause a particular model output. That is, if the feature's value changes, then it is likely that the model's output class will also change (or the model's output will significantly change in case of regression model).  \n",
    "\n",
    "Below we show how counterfactuals can be used to provide local feature importance scores for any input, and how those scores can be combined to yield a global importance score for each feature."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.compose import ColumnTransformer\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.preprocessing import StandardScaler, OneHotEncoder\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "\n",
    "import dice_ml\n",
    "from dice_ml import Dice\n",
    "from dice_ml.utils import helpers  # helper functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preliminaries: Loading the data and ML model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset = helpers.load_adult_income_dataset().sample(5000)  # downsampling to reduce ML model fitting time\n",
    "helpers.get_adult_data_info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "target = dataset[\"income\"]\n",
    "\n",
    "# Split data into train and test\n",
    "datasetX = dataset.drop(\"income\", axis=1)\n",
    "x_train, x_test, y_train, y_test = train_test_split(datasetX,\n",
    "                                                    target,\n",
    "                                                    test_size=0.2,\n",
    "                                                    random_state=0,\n",
    "                                                    stratify=target)\n",
    "\n",
    "numerical = [\"age\", \"hours_per_week\"]\n",
    "categorical = x_train.columns.difference(numerical)\n",
    "\n",
    "# We create the preprocessing pipelines for both numeric and categorical data.\n",
    "numeric_transformer = Pipeline(steps=[\n",
    "    ('scaler', StandardScaler())])\n",
    "\n",
    "categorical_transformer = Pipeline(steps=[\n",
    "    ('onehot', OneHotEncoder(handle_unknown='ignore'))])\n",
    "\n",
    "transformations = ColumnTransformer(\n",
    "    transformers=[\n",
    "        ('num', numeric_transformer, numerical),\n",
    "        ('cat', categorical_transformer, categorical)])\n",
    "\n",
    "# Append classifier to preprocessing pipeline.\n",
    "# Now we have a full prediction pipeline.\n",
    "clf = Pipeline(steps=[('preprocessor', transformations),\n",
    "                      ('classifier', RandomForestClassifier())])\n",
    "model = clf.fit(x_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "d = dice_ml.Data(dataframe=dataset, continuous_features=['age', 'hours_per_week'], outcome_name='income')\n",
    "m = dice_ml.Model(model=model, backend=\"sklearn\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Local feature importance\n",
    "\n",
    "We first generate counterfactuals for a given input point. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "exp = Dice(d, m, method=\"random\")\n",
    "query_instance = x_train[1:2]\n",
    "e1 = exp.generate_counterfactuals(query_instance, total_CFs=10, desired_range=None,\n",
    "                                  desired_class=\"opposite\",\n",
    "                                  permitted_range=None, features_to_vary=\"all\")\n",
    "e1.visualize_as_dataframe(show_only_changes=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These can now be used to calculate the feature importance scores. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "imp = exp.local_feature_importance(query_instance, cf_examples_list=e1.cf_examples_list)\n",
    "print(imp.local_importance)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Feature importance can also be estimated directly, by leaving the `cf_examples_list` argument blank."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "imp = exp.local_feature_importance(query_instance, posthoc_sparsity_param=None)\n",
    "print(imp.local_importance)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Global importance\n",
    "\n",
    "For global importance, we need to generate counterfactuals for a representative sample of the dataset. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cobj = exp.global_feature_importance(x_train[0:10], total_CFs=10, posthoc_sparsity_param=None)\n",
    "print(cobj.summary_importance)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Convert the counterfactual output to json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "json_str = cobj.to_json()\n",
    "print(json_str)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Convert the json output to a counterfactual object"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "imp_r = imp.from_json(json_str)\n",
    "print([o.visualize_as_dataframe(show_only_changes=True) for o in imp_r.cf_examples_list])\n",
    "print(imp_r.local_importance)\n",
    "print(imp_r.summary_importance)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}