13 - Explore Differential Privacy.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Explore Differential Privacy\n",
    "\n",
    "A machine learning project typically involves an iterative process of data analyses in order to gain insights into the data and determine which variables are most likely to help build predictive models. Analyzing data usually involves aggregative and statistical functions that provide insights into the statistical distribution of variables and the relationships between them. With large volumes of data, the aggregations provide a level of abstraction; but with smaller amounts of data, or with repeated analyses, even aggregated results may reveal details about individual observations.\n",
    "\n",
    "*Differential privacy* is a technique that is designed to preserve the privacy of individual data points by adding \"noise\" to the data. The goal is to ensure that enough noise is added to provide privacy for individual values while ensuring that the overall statistical makeup of the data remains consistent, and aggregations produce statistically similar results as when used with the original raw data.\n",
    "\n",
    "## Install the SmartNoise SDK\n",
    "\n",
    "[*SmartNoise*](https://smartnoise.org/) is a toolkit from OpenDP; a joint project between researchers at Microsoft, Harvard University, and other contributors that aims to provide building blocks for using differential privacy in data analysis and machine learning projects.\n",
    "\n",
    "> **Note**: SmartNoise is in the early stages of development at this time.\n",
    "\n",
    "Let's start by installing the SmartNoise Python SDK package."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!pip install opendp-smartnoise"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load data\n",
    "\n",
    "Now, let's take a look at some raw data. In this case, we have a set of 10,000 records for patients who have been tested for diabetes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "data_path = 'data/diabetes.csv'\n",
    "diabetes = pd.read_csv(data_path)\n",
    "diabetes.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The output from the code above shows the key summary statistics for the variables in the diabetes dataset.\n",
    "\n",
    "## Perform an analysis\n",
    "\n",
    "You can use SmartNoise to create an analysis in which noise is added to the source data. The underlying mathematics of how the noise is added can be quite complex, but SmartNoise takes care of most of the details for you. However, there are a few concepts it's useful to be aware of.\n",
    "\n",
    "- **Upper and lower bounds**: *Clamping* is used to set upper and lower bounds on values for a variable. This is required to ensure that the noise generated by SmartNoise is consistent with the expected distribution of the original data.\n",
    "- **Sample size**: To generate consistent differentially private data for some aggregations, SmartNoise needs to know the size of the data sample to be generated.\n",
    "- **Epsilon**: Put simplistically, *epsilon* is a non-negative value that provides an inverse measure of the amount of noise added to the data. A low epsilon results in a dataset with a greater level of privacy, while a high epsilon results in a dataset that is closer to the original data. Generally, you should use epsilon values between 0 and 1. Epsilon is correlated with another value named *delta*, that indicates the probability that a report generated by an analysis is not fully private.\n",
    "\n",
    "With these concepts in mind, examine and run the following code, which creates an analysis and reports the mean **Age** value from the differentially private data. The actual mean value from the original raw data is also shown for comparison."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import opendp.smartnoise.core as sn\n",
    "\n",
    "cols = list(diabetes.columns)\n",
    "age_range = [0.0, 120.0]\n",
    "samples = len(diabetes)\n",
    "\n",
    "with sn.Analysis() as analysis:\n",
    "    # load data\n",
    "    data = sn.Dataset(path=data_path, column_names=cols)\n",
    "    \n",
    "    # Convert Age to float\n",
    "    age_dt = sn.to_float(data['Age'])\n",
    "    \n",
    "    # get mean of age\n",
    "    age_mean = sn.dp_mean(data = age_dt,\n",
    "                          privacy_usage = {'epsilon': .50},\n",
    "                          data_lower = age_range[0],\n",
    "                          data_upper = age_range[1],\n",
    "                          data_rows = samples\n",
    "                         )\n",
    "    \n",
    "analysis.release()\n",
    "\n",
    "# print differentially private estimate of mean age\n",
    "print(\"Private mean age:\",age_mean.value)\n",
    "\n",
    "# print actual mean age\n",
    "print(\"Actual mean age:\",diabetes.Age.mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Explore data distribution with histograms\n",
    "\n",
    "It's common when analyzing data to examine the distribution of a variable using a histogram.\n",
    "\n",
    "For example, let's look at the true distribution of ages in the diabetes dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "%matplotlib inline\n",
    "\n",
    "ages = list(range(0, 130, 10))\n",
    "age = diabetes.Age\n",
    "\n",
    "# Plot a histogram with 10-year bins\n",
    "n_age, bins, patches = plt.hist(age, bins=ages, color='blue', alpha=0.7, rwidth=0.85)\n",
    "plt.grid(axis='y', alpha=0.75)\n",
    "plt.xlabel('Age')\n",
    "plt.ylabel('Frequency')\n",
    "plt.title('True Age Distribution')\n",
    "plt.show()\n",
    "print(n_age.astype(int))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's compare that a differentially private histogram of Age."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "with sn.Analysis() as analysis:\n",
    "    data = sn.Dataset(path = data_path, column_names = cols)\n",
    "\n",
    "    age_histogram = sn.dp_histogram(\n",
    "            sn.to_int(data['Age'], lower=0, upper=120),\n",
    "            edges = ages,\n",
    "            upper = 10000,\n",
    "            null_value = -1,\n",
    "            privacy_usage = {'epsilon': 0.5}\n",
    "        )\n",
    "    \n",
    "analysis.release()\n",
    "\n",
    "plt.ylim([0,7000])\n",
    "width=4\n",
    "agecat_left = [x + width for x in ages]\n",
    "agecat_right = [x + 2*width for x in ages]\n",
    "plt.bar(list(range(0,120,10)), n_age, width=width, color='blue', alpha=0.7, label='True')\n",
    "plt.bar(agecat_left, age_histogram.value, width=width, color='orange', alpha=0.7, label='Private')\n",
    "plt.legend()\n",
    "plt.title('Histogram of Age')\n",
    "plt.xlabel('Age')\n",
    "plt.ylabel('Frequency')\n",
    "plt.show()\n",
    "\n",
    "print(age_histogram.value)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The histograms are similar enough to ensure that reports based on the differentially private data provide the same insights as reports from the raw data.\n",
    "\n",
    "## Calculating covariance\n",
    "\n",
    "Another common goal of analysis is to establish relationships between variables. SmartNoise provides a differentially private *covariance* function that can help with this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with sn.Analysis() as analysis:\n",
    "    sn_data = sn.Dataset(path = data_path, column_names = cols)\n",
    "\n",
    "    age_bp_cov_scalar = sn.dp_covariance(\n",
    "                left = sn.to_float(sn_data['Age']),\n",
    "                right = sn.to_float(sn_data['DiastolicBloodPressure']),\n",
    "                privacy_usage = {'epsilon': 1.0},\n",
    "                left_lower = 0.,\n",
    "                left_upper = 120.,\n",
    "                left_rows = 10000,\n",
    "                right_lower = 0.,\n",
    "                right_upper = 150.,\n",
    "                right_rows = 10000)\n",
    "analysis.release()\n",
    "print('Differentially private covariance: {0}'.format(age_bp_cov_scalar.value[0][0]))\n",
    "print('Actual covariance', diabetes.Age.cov(diabetes.DiastolicBloodPressure))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this case, the covariance between **Age** and **DisatolicBloodPressure** is positive, indicating that older patients tend to have higher blood pressure.\n",
    "\n",
    "## Use SQL queries\n",
    "\n",
    "In addition to the **Analysis** functionality, SmartNoise enables you to use SQL queries against data sources to retrieve differentially private aggregated results.\n",
    "\n",
    "First, you need to define the metadata for the tables in your data schema. You can do this in a .yml file, such as the **diabetes.yml** file in the **/metadata** folder. The metadata describes the fields in the tables, including data types and minimum and maximum values for numeric fields."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from opendp.smartnoise.metadata import CollectionMetadata\n",
    "\n",
    "meta = CollectionMetadata.from_file('metadata/diabetes.yml')\n",
    "print (meta)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the metadata defined, you can create *readers* that you can query. In the following example, we'll create a **PandasReader** to read the raw data from a Pandas dataframe, and a **PrivateReader** that adds a differential privacy layer to the **PandasReader**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "from opendp.smartnoise.sql import PandasReader, PrivateReader\n",
    "\n",
    "reader = PandasReader(meta, diabetes)\n",
    "private_reader = PrivateReader(meta, reader)\n",
    "print('Readers ready.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you can submit a SQL query that returns an aggregated resultset to the private reader."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "query = 'SELECT Diabetic, AVG(Age) AS AvgAge FROM diabetes.diabetes GROUP BY Diabetic'\n",
    "\n",
    "result_dp = private_reader.execute_typed(query)\n",
    "print(result_dp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's compare the result to the same aggregation from the raw data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "result = reader.execute_typed(query)\n",
    "print(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can customize the behavior of a **PrivateReader** by specifying an explicit epsilon value.\n",
    "\n",
    "Let's try a reader with a high epsilon (low privacy) value, and another with a low epsilon (high privacy) value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "low_privacy_reader = PrivateReader(meta, reader, 5.0)  # large epsilon, less privacy\n",
    "result = low_privacy_reader.execute_typed(query)\n",
    "print(result)\n",
    "print()\n",
    "\n",
    "high_privacy_reader = PrivateReader(meta, reader, 0.1)  # smaller epsilon, more privacy\n",
    "result = high_privacy_reader.execute_typed(query)\n",
    "print(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the results of the high epsilon (low privacy) reader are closer to the true results from the raw data than the results from the low epsilon (high privacy) reader.\n",
    "\n",
    "## Learn more\n",
    "\n",
    "To learn more about differential privacy with SmartNoise, see [https://smartnoise.org](https://smartnoise.org/)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.6 - AzureML",
   "language": "python",
   "name": "python3-azureml"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}