week02-probability-and-distributions.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Probability and distributions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "tags": [
     "hide_input"
    ]
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import numpy as np\n",
    "from scipy import stats\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Focus questions\n",
    "\n",
    "Goals for student knowledge at the end of the lesson:\n",
    "\n",
    "1. Give examples of situations where the following distributions are an appropriate model for observed data:\n",
    "    - Poission\n",
    "    - Weibull\n",
    "    - log-normal\n",
    "    - normal\n",
    "2. Apply Bayes' theorem to determine probability in cases where trials are not independent.\n",
    "3. How does a “prior probability” differ from a “posterior probability” in applications of Bayes’ theorem?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Probability Density functions\n",
    "\n",
    "__Probability Functions__\n",
    "\n",
    "$P(x)$ = probability mass function (for discrete data)\n",
    "\n",
    "_Example:_ The probability an number $x$ coming up on a dice role. The PMF is plotted as vertical lines because the probability of rolling a non-integer (e.g. 3.5) is equal to zero.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "tags": [
     "hide_input"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAh4klEQVR4nO3dfZQdVZnv8e+PhAgkhIi0AkkgQTJgQEewDXhRBhFHAkLwihoUGEANGUVA4SJyhyvOoOPMQkQUCRGChLfAAHojkxFRXhQHMB2IYBIyNgFMk4R0LoRXJWTy3D9qN6mc7O4+nXT1STe/z1q9+lTtvaueqnNOPVW76lQpIjAzM6u1VaMDMDOzLZMThJmZZTlBmJlZlhOEmZllOUGYmVmWE4SZmWU5QfQySYdIatvEtmMkhaTBnZSfJ+nKXF1J/yHp7zY98i2XpLdJ+rWkFyV9p4/n/ZKkPSqa9j9LOrOKaWfm9TZJiyS9qYs620r6maTnJf1bX8S1OSQ9Kemw9PoCSddt4nQOkvTH9F4f06tBbobyMknaLcU3qC9jcILoRyLiWxHxuU7KJkbENQCSTpJ0X99GV6kpwCpgeEScVdVMJN0jaYP1GxHDImJJBfNqAk4ErkjDh0halzYCL0paLOnkVNaxM/BQzTR2krRG0pOlcU9K+nOaTsffrhHxDHA3xbrszLHA24C3RMQneneJt2j/CPwgvdc/bXQwORHxpxTff/flfJ0geqizvXur1O7AwhhYv+o8CZgTEX8ujVsWEcOA4cBXgR9JGl8qHypp39Lwp4EnMtM+Km1MOv6WpfHXA6d2EdPuwH9FxNqeLkxvfy/6+Hu2O7CgD+fXbzhB8Ppe19ckLZT0nKSrJW2Tyg6R1Cbpq5JWAFdLepOkSyQtS3+X1B66p+6gVWnanymNP1LSw5JekLRU0gWZkE5J010u6axS204Pozv2fiW9A5gGvC/tPa6W9F5Jz5S/dJI+Lml+J9P6saQfpm6rlyT9VtLOaTmfk/SYpP1K9c+V9Hja810o6WOlsj0l3Zu6LVZJuimNl6TvSlqZyh6p2fi9Hgvwd8A5KZbDUnwXlups0K2X1vnZaZrPS7qp4/1M5ZMkzU/vweOSDpf0TeADwA/SfH6Q6oakPdPrHSTNlNQu6SlJ/yBpq1R2kqT7JF2U1tETkibm1m8yEbg3VxCFnwLPAeUEcW1aFx1OBGZ2MY9aDwJ7SNq9tkDSN4D/A3wqLf9nJW2VlvGp9D7NlLRDqt9xVPNZSX8C7spM882Sbk/r67n0elRnwaX37auSHgFeljRY0tGSFqTP8T3p891jkj4vqVXSs5JmS9o1jX8c2AP4WVrujbrgJL0jzXt1iuXoUtmPJV0m6d/T5/9BSW8vle8t6c4038WSPtlFjGPTd+VFSXcCO5XKaruUd5B0lYptxNOSLlQV3U8R8Yb/A54E/gCMBnYEfgtcmMoOAdYC/wK8CdiW4pD0AeCtQBPwn8A/1dS/ONX/G+BlYK9S+TspkvO7gGeAY1LZGCCAG4GhqV47cFgqvwC4rqbu4DR8D/C59Pok4L6aZVwITCwN/wQ4q5P18WOKLp33ANtQfPmfoNggDQIuBO4u1f8EsGtapk+l5d0lld0I/O9Utg3w/jT+I8A8YAQg4B0dbTqJ58Iuhg8B2mrez9+lmHYEFgFTU9kE4HngwymmkcDeteuwNK0A9kyvZwL/F9g+rf//Aj5bWuevAZ9P6+jvgWWAOlmmduC9uWVIcX0sTW+v0ns9Bliapv8OYDFwGPBkzbIf1sVn/RHg6E7KLiB9vtLwKUArxQZ0GHAbcG3N528mxWd128z03gJ8HNgurbN/A37azfdwPsX3cFvgryg+Sx8GtgbOSfEMqV3W2thrpnsoxed5f4rv5PeBX9ezztJ8W4HzgCFpWi+y/vv8Y+BZis/VYIqjtFmpbGh6v05OZfunOPbpZF73s367cXCaT2ff959SdE8OpdgO/Q44tde3jb09wf74lz4gU0vDRwCPp9eHAGuAbUrljwNHlIY/0vElZX2CGFoqvxk4v5N5XwJ8t+ZDsHep/F+Bq2q/BJkPzD10nSC+ClyfXu8IvELXG+QflYa/BCwqDb8TWN3F+pwPTEqvZwLTgVE1dQ6l2MAeCGzVzfvzY3qeII6vWYfT0usrOtZ3Zj6vr8PSuAD2pNgovwqML5WdCtxTWuetpbLtUtudO5nXazXv8yHAOmA1xQZnPjC59r0Gfknxefs2ReLNJYiX0nRWU7NBptj5ObGTmF7/fKXhXwFfKA3vleIeXIppjx58z94NPNfN9/CU0vD5wM2l4a2Ap4FDSvXrSRBXAf9aGh6WlmNM7XQybT8ArKD0GaXY6bmg9Fm8slR2BPBYev0p4Dc107sC+HpmPrux8XbjBjLfd4rzRK9SSsrAcZR22nrrz11M6y0tvX6KYu+zQ3tE/KU0vGuq01n95yLi5Vy5pAMk3Z0Ou58HplI6lKwjlk11HXCUpGHAJyk+uMu7qP9M6fWfM8PDOgYknZi6bFZLWg3sy/plOofiCOF36fD8FICIuAv4AXAZ8Iyk6ZKGb9YSbmhF6fUrpXhHUyT4ntqJYg+y9n0fmZtnRLySXg4j7zmKveqyZRExIiJ2jIh3R8SsTLuZFMnoOIr3NOeYNJ0REXFMTdn2FImjHrnPeccGqsNSOiFpO0lXpC6qF4BfAyO66QopT2+D+UfEulQ+srZRN2qn8xLw/+qczq7A0jTvDp2+72z4WdsdOKDje5G+G58Bdu5kPrntRs7uFEc2y0vTvYLiSKJXOUGsN7r0ejeK7oEOUVN3GcWb1Fn9N0sa2kn5DcBsYHRE7EBxvkA9iKUetfESEU9THMJ+DDiBoj97s6X+7B8Bp1Fc/TKCortOab4rIuLzEbErxR73Dzv69CPi0oh4D7APRXfC/6pzti9T7KF3yH3hOrMUeHsnZRutt5JVFHudte/70z2Yd9kjFMvcU7cCRwJLIqKzDUhW6r/eE/h9nU1yn/O1bLiz0NU6O4viqOOAiBhO0W0CG3/ey8rT22D+kkTx3ejpOq+dzlCK7q96prMMGN1xrimp931fCtxbStYjorho4O8zdZeT3250Nt1XgZ1K0x0eEfvUEVOPOEGs90VJoyTtSNHfeFMXdW8E/kFSk6SdKE7u1e7NfUPSEEkfAD5K0f8KxR7csxHxF0kTKK5EqXV+2vvah6L/sqtYcp4BRkkaUjN+JsUe/TspzkH0hqEUX+p2ABWXZr5+slnSJ0onJp9Ldf9bxYnzAyRtTbHB/wtQ7yV884EjJO0oaWfgzB7EexVwsqQPpZOwIyXtncqeoehv30gUlxfeDHxT0vYpMX6FzvfiuzOH4vxUj6Q9zEOB7OXO3ZhA0R1Vb2K5EfhyOnk6DPgWcFPUf5XT9hRHm6vT9+rrPYz3ZuDI9F5tTZFwXqU459cTN1C85+9OJ6G/BTwYEU/W0fZBis/nOZK2lnQIcBSQO7qrdTvwV5JOSG23Tp/7jU60p/ekhfXbjfen+WwkHfn/AviOpOHpc/x2ST3+PHXHCWK9GyhW+pL0d2EXdS+keDMfAR4FHqqpv4JiY7iM4qTV1Ih4LJV9AfhHSS9SJJabM9O/l+LE2K+AiyLiFz1clrsoLttbIWlVafxPKPakflJzKLvJImIh8B2Ko5NnKJLPb0tV3gs8KOkliiOnMyLiCYpLOX9EsZ6eojjkv6jO2V5LsRf8JMV7VncCjYjfUSTd71KcrL6X9XuX3wOOVXHFzaWZ5l+i2FgsAe6j+MzMqHfeNWZSJLlte9owIloiYlO6yT5DccRarxkU6/rXFBcp/IViHdTrEoqTzasoLur4eQ/aEhGLgeMpTiqvothgHhURa3o4nV9RnM+4lWJP/e3A5DrbrgGOprjqbBXwQ4pzOI912bBo+yLwt2leyyi2Cx0Xu+R8GjiA4hzU1+n6CrUTKbo8F1J8h24Bdul+iXpG6QTHG5qKHxp9LiJ+2ehYqpYu6zv1jbCsWzpJ3wJWRsQlfTCvt1Ikw/1qzqeZdco/+noDkfRxii6eja5Zt74XEef14bxWUlwaa1Y3J4g3CEn3UPzo6oSaKzLMzLLcxWRmZlk+SW1mZlkDqotpp512ijFjxjQ6DDOzfmPevHmrIqIpVzagEsSYMWNoaWlpdBhmZv2GpE5/F+MuJjMzy3KCMDOzLCcIMzPLcoIwM7MsJwgzM8tygjAzsywnCDMzy3KCMDOzLCcIMzPLGlC/pO4rF1yQfz1QePn6Ny9f/7WlLduAuptrc3Nz9MWtNlR6ou4AWn2v8/L1b16+/qsRyyZpXkQ058rcxWRmZllOEGZmluUEYWZmWZUmCEmHS1osqVXSuZnyvSXdL+lVSWfXlI2QdIukxyQtkvS+KmM1M7MNVXYVk6RBwGXAh4E2YK6k2RGxsFTtWeB04JjMJL4H/DwijpU0BNiuqljNzGxjVR5BTABaI2JJRKwBZgGTyhUiYmVEzAVeK4+XNBw4GLgq1VsTEasrjNXMzGpUmSBGAktLw21pXD32ANqBqyU9LOlKSUNzFSVNkdQiqaW9vX3zIjYzs9dVmSCUGVfvlb2Dgf2ByyNiP+BlYKNzGAARMT0imiOiuakp+1hVMzPbBFUmiDZgdGl4FLCsB23bIuLBNHwLRcIwM7M+UmWCmAuMkzQ2nWSeDMyup2FErACWStorjfoQsLCLJmZm1ssqu4opItZKOg24AxgEzIiIBZKmpvJpknYGWoDhwDpJZwLjI+IF4EvA9Sm5LAFOripWMzPbWKU364uIOcCcmnHTSq9XUHQ95drOB7L3BzEzs+r5l9RmZpblBGFmZllOEGZmluUEYWZmWU4QZmaW5QRhZmZZThBmZpblBGFmZllOEGZmluUEYWZmWU4QZmaW5QRhZmZZThBmZpblBGFmZllOEGZmluUEYWZmWZUmCEmHS1osqVXSuZnyvSXdL+lVSWdnygdJeljS7VXGaWZmG6ssQUgaBFwGTATGA8dJGl9T7VngdOCiTiZzBrCoqhjNzKxzVR5BTABaI2JJRKwBZgGTyhUiYmVEzAVeq20saRRwJHBlhTGamVknqkwQI4GlpeG2NK5elwDnAOu6qiRpiqQWSS3t7e09DtLMzPKqTBDKjIu6GkofBVZGxLzu6kbE9IhojojmpqamnsZoZmadqDJBtAGjS8OjgGV1tj0IOFrSkxRdU4dKuq53wzMzs65UmSDmAuMkjZU0BJgMzK6nYUR8LSJGRcSY1O6uiDi+ulDNzKzW4KomHBFrJZ0G3AEMAmZExAJJU1P5NEk7Ay3AcGCdpDOB8RHxQlVxmZlZfSpLEAARMQeYUzNuWun1Coqup66mcQ9wTwXhmZlZF/xLajMzy3KCMDOzLCcIMzPLcoIwM7MsJwgzM8tygjAzsywnCDMzy3KCMDOzLCcIMzPLcoIwM7MsJwgzM8tygjAzsywnCDMzy3KCMDOzLCcIMzPLcoIwM7OsShOEpMMlLZbUKuncTPneku6X9Kqks0vjR0u6W9IiSQsknVFlnGZmtrHKnignaRBwGfBhoA2YK2l2RCwsVXsWOB04pqb5WuCsiHhI0vbAPEl31rQ1M7MKVXkEMQFojYglEbEGmAVMKleIiJURMRd4rWb88oh4KL1+EVgEjKwwVjMzq1FlghgJLC0Nt7EJG3lJY4D9gAc7KZ8iqUVSS3t7+6bEaWZmGVUmCGXGRY8mIA0DbgXOjIgXcnUiYnpENEdEc1NT0yaEaWZmOVUmiDZgdGl4FLCs3saStqZIDtdHxG29HJuZmXWjygQxFxgnaaykIcBkYHY9DSUJuApYFBEXVxijmZl1orKrmCJiraTTgDuAQcCMiFggaWoqnyZpZ6AFGA6sk3QmMB54F3AC8Kik+WmS50XEnKriNTOzDVWWIADSBn1OzbhppdcrKLqeat1H/hyGmZn1Ef+S2szMspwgzMwsywnCzMyynCDMzCzLCcLMzLKcIMzMLMsJwszMspwgzMwsywnCzMyynCDMzCyrrgQh6VZJR0pyQjEze4Ood4N/OfBp4I+Svi1p7wpjMjOzLUBdCSIifhkRnwH2B54E7pT0n5JOTs9tMDOzAabuLiNJbwFOAj4HPAx8jyJh3FlJZGZm1lB13e5b0m3A3sC1wFERsTwV3SSppargzMysceo9grgyIsZHxD93JAdJbwKIiObOGkk6XNJiSa2Szs2U7y3pfkmvSjq7J23NzKxa9SaICzPj7u+qgaRBwGXARIqnxB0naXxNtWeB04GLNqGtmZlVqMsupvRI0JHAtpL2Y/1T3oYD23Uz7QlAa0QsSdOaBUwCFnZUiIiVwEpJR/a0rZmZVau7cxAfoTgxPQq4uDT+ReC8btqOBJaWhtuAA+qMa3PamplZL+gyQUTENcA1kj4eEbf2cNq5Z0pHb7eVNAWYArDbbrvVOXkzM+tOd11Mx0fEdcAYSV+pLY+IizPNOrQBo0vDo4BldcZVd9uImA5MB2hubq43AZmZWTe662Iamv4P24RpzwXGSRoLPA1Mpvg1dtVtzcysF3TXxXRF+v+Nnk44ItZKOg24AxgEzIiIBZKmpvJp6SR4C8VJ73WSzgTGR8QLubY9jcHMzDadIjrvlZF0aVeNI+L0Xo9oMzQ3N0dLS/W/21PpDEkXq6/f8vL1b16+/qsRyyZpXme/Z+uui2leBfGYmVk/UM9VTGZm9gbU3VVMl0TEmZJ+RuYy04g4urLIzMysobrrYro2/b+oy1pmZjbgdNfFNC/9v1fSEIo7ugawOCLW9EF8ZmbWIPXe7vtIYBrwOMWvnMdKOjUi/qPK4MzMrHHqShDAd4APRkQrgKS3A/8OOEGYmQ1Q9d7ue2VHckiWACsriMfMzLYQ3V3F9D/TywWS5gA3U5yD+ATF7TDMzGyA6q6L6ajS62eAv0mv24E3VxKRmZltEbq7iunkvgrEzMy2LPVexbQN8FlgH2CbjvERcUpFcZmZWYPVe5L6WmBniifM3UvxfIYXqwrKzMwar94EsWdEnA+8nO7PdCTwzurCMjOzRqs3QbyW/q+WtC+wAzCmkojMzGyLUO8P5aZLejNwPjCb4glz51cWlZmZNVxdRxARcWVEPBcR90bEHhHx1o6nzXVF0uGSFktqlXRuplySLk3lj0jav1T2ZUkLJP1B0o3pRLmZmfWRuhKEpLdI+r6khyTNk3SJpLd002YQcBkwERgPHCdpfE21icC49DcFuDy1HQmcDjRHxL4Ujx2d3IPlMjOzzVTvOYhZFLfW+DhwLLAKuKmbNhOA1ohYku78OguYVFNnEjAzCg8AIyTtksoGA9tKGgxsByyrM1YzM+sF9SaIHSPinyLiifR3ITCimzYjgaWl4bY0rts6EfE0xTMo/gQsB56PiF/kZiJpiqQWSS3t7e11Lo6ZmXWn3gRxt6TJkrZKf5+kuJtrV5QZV/tUumyddEJ8EjAW2BUYKun43EwiYnpENEdEc1NTUzchmZlZvbpMEJJelPQCcCpwA7Am/c0CvtzNtNuA0aXhUWzcTdRZncOAJyKiPSJeA24D/kc38zMzs17UZYKIiO0jYnj6v1VEDE5/W0XE8G6mPRcYJ2lsehrdZIpLZMtmAyemq5kOpOhKWk7RtXSgpO0kCfgQsGiTltDMzDZJvb+DQNLRwMFp8J6IuL2r+hGxVtJpwB0UVyHNiIgFkqam8mnAHOAIoBV4BTg5lT0o6RbgIWAt8DAwvScLZmZmm0cRtacFMpWkbwPvBa5Po44D5kXERr9taKTm5uZoaWmpfD4qnTmpY/X1O16+/s3L1381YtkkzYuI5lxZvUcQRwDvjoh1aYLXUOzVb1EJwszMek+9VzHBhpe17tDLcZiZ2Ram3iOIbwEPS7qb4tLUg4GvVRaVmZk1XLcJQtJWwDrgQIrzEAK+GhErKo7NzMwaqNsEERHrJJ0WETez8WWqZmY2QNV7DuJOSWdLGi1px46/SiMzM7OGqvccxCkUt8n4Qs34PXo3HDMz21LUmyDGUySH91Mkit8A06oKyszMGq/eBHEN8AJwaRo+Lo37ZBVBmZlZ49WbIPaKiL8uDd8t6fdVBGRmZluGek9SP5xupgeApAOA31YTkpmZbQnqPYI4gOKuq39Kw7sBiyQ9CkREvKuS6MzMrGHqTRCHVxqFmZltcepKEBHxVNWBmJnZlqUnN+szM7M3ECcIMzPLqjRBSDpc0mJJrZI2enZEetTopan8EUn7l8pGSLpF0mOSFkl6X5WxmpnZhipLEJIGAZcBEyl+iX2cpPE11SYC49LfFODyUtn3gJ9HxN7AX+NnUpuZ9akqjyAmAK0RsSQi1gCzgEk1dSYBM6PwADBC0i6ShlM8c+IqgIhYExGrK4zVzMxqVJkgRgJLS8NtaVw9dfYA2oGrJT0s6UpJQ3MzkTRFUouklvb29t6L3szsDa7KBKHMuNrHcHdWZzCwP3B5ROwHvEwnz7+OiOkR0RwRzU1NTZsTr5mZlVSZINqA0aXhUcCyOuu0AW0R8WAafwtFwjAzsz5SZYKYC4yTNFbSEGAyGz+RbjbFLTyU7vX0fEQsT48zXSppr1TvQ8DCCmM1M7Ma9d5qo8ciYq2k04A7gEHAjIhYIGlqKp8GzAGOAFqBV4CTS5P4EnB9Si5LasrMzKxiiqg9LdB/NTc3R0tLS+XzUenMyQBafa/z8vVvXr7+qxHLJmleRDTnyvxLajMzy3KCMDOzLCcIMzPLcoIwM7MsJwgzM8tygjAzsywnCDMzy3KCMDOzLCcIMzPLcoIwM7MsJwgzM8tygjAzsywnCDMzy3KCMDOzLCcIMzPLcoIwM7OsShOEpMMlLZbUKuncTLkkXZrKH5G0f035IEkPS7q9yjjNzGxjlSUISYOAy4CJwHjgOEnja6pNBMalvynA5TXlZwCLqorRzMw6V+URxASgNSKWRMQaYBYwqabOJGBmFB4ARkjaBUDSKOBI4MoKYzQzs05UmSBGAktLw21pXL11LgHOAdZ1NRNJUyS1SGppb2/frIDNzGy9KhOEMuNqH8OdrSPpo8DKiJjX3UwiYnpENEdEc1NT06bEaWZmGVUmiDZgdGl4FLCszjoHAUdLepKia+pQSddVF6qZmdWqMkHMBcZJGitpCDAZmF1TZzZwYrqa6UDg+YhYHhFfi4hRETEmtbsrIo6vMFYzM6sxuKoJR8RaSacBdwCDgBkRsUDS1FQ+DZgDHAG0Aq8AJ1cVj5mZ9Ywiak8L9F/Nzc3R0tJS+XxUOnMygFbf67x8/ZuXr/9qxLJJmhcRzbky/5LazMyynCDMzCzLCcLMzLKcIMzMLMsJwszMspwgzMwsywnCzMyynCDMzCzLCcLMzLKcIMzMLMsJwszMspwgzMwsywnCzMyynCDMzCzLCcLMzLKcIMzMLKvSBCHpcEmLJbVKOjdTLkmXpvJHJO2fxo+WdLekRZIWSDqjyjjNzGxjlSUISYOAy4CJwHjgOEnja6pNBMalvynA5Wn8WuCsiHgHcCDwxUxbMzOrUJVHEBOA1ohYEhFrgFnApJo6k4CZUXgAGCFpl4hYHhEPAUTEi8AiYGSFsZqZWY0qE8RIYGlpuI2NN/Ld1pE0BtgPeDA3E0lTJLVIamlvb9/cmM3MLKkyQSgzrvYx3F3WkTQMuBU4MyJeyM0kIqZHRHNENDc1NW1ysGZmtqEqE0QbMLo0PApYVm8dSVtTJIfrI+K2CuM0M7OMKhPEXGCcpLGShgCTgdk1dWYDJ6armQ4Eno+I5ZIEXAUsioiLK4zRzMw6MbiqCUfEWkmnAXcAg4AZEbFA0tRUPg2YAxwBtAKvACen5gcBJwCPSpqfxp0XEXOqitfMzDZUWYIASBv0OTXjppVeB/DFTLv7yJ+fMDOzPuJfUpuZWZYThJmZZTlBmJlZlhOEmZllOUGYmVmWE4SZmWU5QZiZWZYThJmZZTlBmJlZlhOEmZllOUGYmVmWE4SZmWU5QZiZWZYThJmZZTlBmJlZlhOEmZllVZogJB0uabGkVknnZsol6dJU/oik/etta2Zm1aosQUgaBFwGTATGA8dJGl9TbSIwLv1NAS7vQVszM6tQlUcQE4DWiFgSEWuAWcCkmjqTgJlReAAYIWmXOtuamVmFqnwm9UhgaWm4DTigjjoj62wLgKQpFEcfAC9JWrwZMffETsAqDdwnZ3v5+jcvX//V18u2e2cFVSaI3OJFnXXqaVuMjJgOTO9ZaJtPUktENPf1fPuKl69/8/L1X1vSslWZINqA0aXhUcCyOusMqaOtmZlVqMpzEHOBcZLGShoCTAZm19SZDZyYrmY6EHg+IpbX2dbMzCpU2RFERKyVdBpwBzAImBERCyRNTeXTgDnAEUAr8Apwcldtq4p1E/V5t1Yf8/L1b16+/muLWTZFZLv2zczsDc6/pDYzsywnCDMzy3KC6CFJMyStlPSHRsdSBUmjJd0taZGkBZLOaHRMvUnSNpJ+J+n3afm+0eiYepukQZIelnR7o2PpbZKelPSopPmSWhodT2+TNELSLZIeS9/B9zU0Hp+D6BlJBwMvUfwCfN9Gx9Pb0i/Zd4mIhyRtD8wDjomIhQ0OrVdIEjA0Il6StDVwH3BG+iX/gCDpK0AzMDwiPtroeHqTpCeB5ohY1ehYqiDpGuA3EXFluoJzu4hY3ah4fATRQxHxa+DZRsdRlYhYHhEPpdcvAosoftk+IKTburyUBrdOfwNmL0nSKOBI4MpGx2I9I2k4cDBwFUBErGlkcgAnCOuCpDHAfsCDDQ6lV6UumPnASuDOiBhIy3cJcA6wrsFxVCWAX0ial26zM5DsAbQDV6cuwislDW1kQE4QliVpGHArcGZEvNDoeHpTRPx3RLyb4hf6EyQNiK5CSR8FVkbEvEbHUqGDImJ/ijs9fzF1+Q4Ug4H9gcsjYj/gZaChjzpwgrCNpL75W4HrI+K2RsdTlXT4fg9weGMj6TUHAUenfvpZwKGSrmtsSL0rIpal/yuBn1Dc+XmgaAPaSke0t1AkjIZxgrANpJO4VwGLIuLiRsfT2yQ1SRqRXm8LHAY81tCgeklEfC0iRkXEGIrb09wVEcc3OKxeI2lounCC1PXyt8CAuZowIlYASyXtlUZ9CGjoxSFV3qxvQJJ0I3AIsJOkNuDrEXFVY6PqVQcBJwCPpn56gPMiYk7jQupVuwDXpIdSbQXcHBED7nLQAeptwE+KfRgGAzdExM8bG1Kv+xJwfbqCaQnp9kON4stczcwsy11MZmaW5QRhZmZZThBmZpblBGFmZllOEGZmluUEYVYxSS91X8tsy+MEYWZmWU4QZj0k6V8kfaE0fIGkr0v6laSH0vMKJmXaHVJ+RoOkH0g6Kb1+j6R7003o7ki3XTdrKCcIs56bBXyqNPxJ4GrgY+lGch8EvpNuW9KtdO+r7wPHRsR7gBnAN3s3ZLOe8602zHooIh6W9FZJuwJNwHPAcuC76e6i6yieofE2YEUdk9wL2Be4M+WUQWl6Zg3lBGG2aW4BjgV2pjii+AxFsnhPRLyW7qi6TU2btWx41N5RLmBBRDT08ZJmtdzFZLZpZlHcMfVYimSxA8WzGF6T9EFg90ybp4Dxkt4kaQeKu3UCLAaaOp4/LGlrSftUvgRm3fARhNkmiIgF6dbTT0fEcknXAz+T1ALMJ3ML8YhYKulm4BHgj8DDafwaSccCl6bEMZjiyXAL+mRhzDrhu7mamVmWu5jMzCzLCcLMzLKcIMzMLMsJwszMspwgzMwsywnCzMyynCDMzCzr/wPQJWblMgnrZwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# generate a uniform distribution\n",
    "# see https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.randint.html#scipy.stats.randint\n",
    "low = 1\n",
    "high = 7\n",
    "rv = stats.randint(low, high)\n",
    "\n",
    "# plot the probability mass function\n",
    "x = np.arange(low,high)\n",
    "plt.vlines(x, 0, rv.pmf(x), colors='b', linestyles='-', lw=3)\n",
    "plt.xlim(0.5,6.5)\n",
    "plt.ylim(0,0.175)\n",
    "plt.title('probability mass function (PMF) for a roll of one die')\n",
    "plt.xlabel('value')\n",
    "plt.ylabel('probability');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$F(x)$ = cumulative distribution function (CDF)\n",
    "\n",
    "The cumulative distribution is the probability of occurrence of a value of $x$ or lower. For the example of one die roll, the CDF is a step function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "tags": [
     "hide_input"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAdzklEQVR4nO3de5gcZZ328e9NAgSSQBSyQg4QVBQCCwgj4qsCrr7KQYi6gByUBQ/Iqyheui7qioyKirvGRQSNWUQRxICAGDTKogKuiwcSjsaIRhbMmEASTjJ4gJDf+8fzjHQ6T0/3HGq6M3N/rquvmeo6/aq7uu6qp6qrFRGYmZnV26zdBZiZWWdyQJiZWZEDwszMihwQZmZW5IAwM7MiB4SZmRU5IJqQdJKknwxh/O9J+qfhrKnBfG6U9Nb8/wmS/msYp71U0sH5/25Jlw7jtD8k6cLhml7dtM+WtFbS/VVMv5/5zpN0ZkXTni1pcRXTbjC/d0s6p8kwL5H0W0m9kl47QqUNiqSDJfXUdN8r6ZWDnFZb1q9mapdpyJ+viPCjnwdwEvCTFoftBi5tU503Am8d4DhfBc4e4DiDXkbgYKBnhF6PmcCfgb/rlPVjmOZ3FXBs3XPHA4uBXmAV8D3gpTXv15PAY/nxG+B8YMe692V9Hr/vcW3uNwHo6e91BH4InD5Sr8EQX78N1kHgXuCVnbp+DXIZB7VMpYePIKwhSePbXcMQ7Aw8GBGr213IcJG0I/By4Jqa594LnAt8EngWsBPwBWBOzaiXR8Rk4JnA64AdgCV5en1WRsSkmscRABHxF1LgnNhPaTsDSwe5TMO6jo3gOjvq1q+idqddCyl9NbAGeBA4Pz/fTc1eLDALCGB87r4ROBu4mbw3BGwHfB34I3ALMKs0bs34b83/n0TNHiLwOWBFns4S4GX5+UOAJ0h7a73AHbXTArYEHgH2rJnWVGr2QoDXALfn4W4G9urntfm/wK+BR0l7hDeVagYE/AewOg97J7AncEqu9Qk23GO8FzgjD/dXYDw1eyT5tb8SuJy0R3orsHdNXQE8t6b7q/m9mJiXtXZPdVrhvTyStLF5JL92u9ftGf1zru3RXMOEwmvzyrp5fZXC0Uthua4AvpaXaynQ1d+6COwO/AV4Ks/nkdplrhn3bcBy4CFgITCt7vU6Ffgt8DBwAaAG7/mJwA9qurfN8z26n/Vkg9c3PzcOuAP4TO7e6LWpG/4E4IYG/X6XX+c/51q2zO/rwry8y4G31dVzJXAp6TO00VEvcDhwW+6/Aujup7aDSUc4ZwD3A5fkGs4FVubHucCWpWWln73t/Pp+Lb/n9wEfJjXLb7R+NRh/0O878GZgWe53HbBzP6/Bm3J9DwL/ysbrde3n6wDStuWRvA4c3N82uGOPICSNA75DWvBZwHRgwQAmcSzphZsOPAf4KfAV0l7UMuCsQZZ2C7BPns5lwDclTYiI75P24i6PtAe2d+1IEfFX0gbmuJqnjwFuiojVkvYFLgLeTgqzLwELJW1ZX4Ck7UlNDR8Gtid9SF/SoN5XAQcCzwOmAG8g7fnMJwXmv0XNHmN2HOlDOiUi1hWmOQf4Zs1rcI2kzRvMv2/5HwcOZcM91ZV1y/U84BvAe0jhuQi4VtIWNYMdQwrjXYC9SGFYP68f1M1ro2EaOJK0jk0hfaDPz3UV18WIWEb6kP80z2dK/QQl/QPwqVz3jnka9evxa4AXAnvn4V7doL6/B+6u6X4xqQnoWy0uHwAR8RTwbeBlLY6yLNdWmtZzgN8DR+TX4K+k97CHFBRHAZ+U9Iqa0eaQQmIKaR2s9zgpDKeQ1sP/1+Tcxg6kdXFn0o7Pv5I2hPvkuvcnfVYG6vOkkHg2cFCu6eRW1q+hvO95WT8EvJ70Ofhv0mu6EUmzgS+StnXTSNuOGQ2GnQ58l7TD9kzSztZVkqY2egE6NiBIb+o04P0R8XhE/CUiBnKy+CsR8buIeJR0iPy7iPhB3uB9E3jBYIqKiEsj4sGIWBcRc0l7K89vcfTL2DAgjs/PQdrb+FJE/DwinoqIi0l78AcUpnMY8KuIuDIiniTtITU6UfYkMBnYjbSHsiwiVjWp87yIWBERf27Qf0nNvD9L2kiV6hyoNwDfjYjr87Q/A2wF/J+62lZGxEOkI8N9hmG+fX4SEYvyBvQSnt4oDmVdPAG4KCJuzRvPDwIvljSrZphzIuKRiPg9cAONl2kK6eimz3bA2gYh3sxK0kaizzRJj9Q8jqnp9xhpQ9mUpJnAS4Ez8ut0O3AhaQPW56cRcU1ErC+tYxFxY0TclfvfSdo4HtTPbNcDZ0XEX/P0TgA+FhGrI2IN8NG6+beyHONI6+MHI+KxiLgXmDuA6QzlfX878Kn8WV1H2vHcR9LOhfkcBXwnIn6c53Mm6fUoeSOwKK/j6yPietK5q8MaLUQnB8RM4L5BrvwAD9T8/+dC96TBTFTS+yQtk/SopEdIH5ztWxz9R8BWkl6U3+x9eHrvb2fgfbUfUtJrMK0wnWmkQ28AIh07rigMR0T8iLQnfAHwgKT5krZpUmdxWqX+EbGep/cWh2oaaU+rdtorSHvsfWqD8E8M8n1soH7aE3Kb9lDWxfpl6iU1BQxmmR4mhX2fB4HtB9nuPp3U9NFnZURMqXlcUdNvMqlJrxXTgIciojbI7mPD5e13/cqfjxskrZH0KOkorb/P2JpI50pqa7ivpvs+Br5+bg9sUZjO9PLgGxnK+74z8Lma7cBDpKbi0rzrtwWP5/mU7AwcXbeNeSnpCKeokwNiBbBTg5X/cWDrmu4dhjCfx/PfptOT9DJSW+cxwDNyk8KjpDcPUrtiQ3mDdwXpKOJ4UvL3fZBWAJ+o+5BuHRGlQ8tVpI1WX12q7S7M97yI2A/Yg9TU9P4m9Ta7xW/tvDcjHdL2NRf9icavZbPpriStxH3T7luuPzQZrxUbrDN5D7HhoXWd/tbFgS7TRNKe/2CW6U7S+9fnp6RzIK8dyETye3YEqemiFbuT2qtbsRJ4pqTaINuJDZe32Wt2GamJb2ZEbAvM4+nPWEn99DZ4zfP8VzIwa0lH3/XTafV9G8r7vgJ4e922YKuIuLkwbP22YOs8n0bTvaRuuhMjouFlzJ0cEL8gLfw5kiZKmiCpr539duBASTtJ2pZ0+DYo+RD0D8AbJY2T9GbSOYuSycA60kmr8ZI+AtTujT8AzMofwEYuIx26nsDTzUsA/wmcmveelJf58LoPWp/vAntIen3eaL2bxqH2wjzNzUkbyb6Tqn31PrufWhvZr2be7yE1hf0s97sdOD6/loewYdPAA8B2+T0ruQI4XNIrcr3vy9MufTAG6jekI4LD87Q/TGoebEV/6+IDwIy68yS1LgNOlrRPPp/0SeDnuclioK4H9pU0ASA3n34EuEDSayVtLWlzSYdK+rf6kXO/3UlNNjuQmgdbcRCpmbapiFhBer8+lV+nvYC3UD7X0Mhk0lHIXyTtT9qZGohvAB+WNDWfr/sI6aR4y3Iz4xXAJyRNzkf87x3AdIbyvs8DPihpDwBJ20o6usGwVwKvkfTSvA5+jMbb9UuBIyS9On8+Jyh9L6R4zoJ+JtR2+Q06Angu6SRYD2nDSm47u5y0R7WEdAJxKN5G2qt+kLSX3WiDdB3pg/Ib0uHjX9jwcPmb+e+Dkm4tTSAifk7aUE+j5kMXEYtzHeeTmhKWUzgBm4ddCxwNnJNr3hX4nwY1b0MKn4d5+kqHz+R+XwZm58PNaxqMX/Jt0nvxMKlN9vX5nAHA6aT37RFSCP5tuhHxa9KH9548zw0O+yPiblI76edJe3BHkE5+PjGA2oryxvQdpPbwP5Deg55+R3p63IbrIqnZcClwv6S1hXF/SGoXvooUMs8hXUAxmGV4IM9vTs1znyVtuD5M2nFZAZxGzesOvEFSL+k9WUhaB/aLuosESnIYHQZcPIBSjyOdzF9JakI9K39mW/UO4GOSHiNt3K9oMny9s0lt63cCd5GutDt7gNMAeBdpPbkH+Alpo39RKyMO5X2PiG8BnwYWSPoj8EvSSfHSsEuBd+baVpE+k8X1Oof3HNIJ8L515f30kwNKzddmtinIV61cDOwfI/DhlfQuUlPPv1Q9L+s8DggzMyuqrIlJ0kWSVkv6ZYP+knSepOWS7lT6HoCZmXWIKs9BfJX0haZGDiW1ne9K+nLLFyusxczMBqiygIiIH7Phddb15gBfi+RnwBRteG8YMzNro3bejG06G14B1JOf2+hbvpJOIR1lMHHixP122223ESnQzKwKt90G6xt937kyS9ZGRKvf/QHaGxClL74Uz5hHum/QfICurq5YvHjEbodvZjbs1N/X/qqb633Nh9lQOwOihw2//Vv7bVwzszFhpC4kHUwotfOLcguBE/PVTAcAj7ZwEzkzMxshlR1BSPoG6d7r2yv9xN9ZwOYAETGPdCvnw0jfGP4TcHJVtZiZ2cBVFhARcVyT/kH6iriZmXWgjr0Xk5mZtZcDwszMihwQZmZW5IAwM7MiB4SZmRU5IMzMrMgBYWZmRQ4IMzMrckCYmVmRA8LMzIocEGZmVuSAMDOzIgeEmZkVOSDMzKzIAWFmZkUOCDMzK3JAmJlZkQPCzMyKHBBmZlbkgDAzsyIHhJmZFTkgzMysyAFhZmZFDggzMytyQJiZWZEDwszMihwQZmZW5IAwM7MiB4SZmRU5IMzMrMgBYWYdZ+5cmDwZpNH52FQ4IMys43R3Q29vu6uo3qRJ7a6gfw4IM+s4YyUcurvbXUX/xre7ADOz/kS0u4Kxy0cQZmZWVGlASDpE0t2Slkv6QKH/tpKulXSHpKWSTq6yHjMza11lASFpHHABcCgwGzhO0uy6wd4J/Coi9gYOBuZK2qKqmszMrHVVHkHsDyyPiHsi4glgATCnbpgAJksSMAl4CFhXYU1mZtaiKgNiOrCiprsnP1frfGB3YCVwF3B6RKyvn5CkUyQtlrR4zZo1VdVrZmY1qgyI0tdB6q9HeDVwOzAN2Ac4X9I2G40UMT8iuiKia+rUqcNdp5mZFVQZED3AzJruGaQjhVonA1dHshz4X2C3CmsyM7MWVRkQtwC7Stoln3g+FlhYN8zvgVcASHoW8HzgngprMjOzFlX2RbmIWCfpNOA6YBxwUUQslXRq7j8P+DjwVUl3kZqkzoiItVXVZGZmrav0m9QRsQhYVPfcvJr/VwKvqrIGMzMbHH+T2szMihwQZmZW5IAwM7MiB4SZmRU5IMzMrMgBYWZmRQ4IMzMrckCYmVmRA8LMzIocEGZmVuSAMDOzIgeEmZkVOSDMzKzIAWFmZkUOCDMzK3JAmJlZkQPCzMyKHBBmZlbkgDAzsyIHhJmZFTkgzMysyAFhZmZFDgizTdTcuTB5Mkij72GdwQFhtonq7obe3nZXUa1Jk9pdwdjmgDDbRI2FcOjubncVY9v4dhdgZkMX0e4KbDTyEYSZmRU5IMzMrMgBYWZmRQ4IMzMrckCYmVmRA8LMzIocEGZmVuSAMDOzokoDQtIhku6WtFzSBxoMc7Ck2yUtlXRTlfWYmVnrWgoISVdJOlxSy4EiaRxwAXAoMBs4TtLsumGmAF8AjoyIPYCjW52+mZlVq9UN/heB44HfSjpH0m4tjLM/sDwi7omIJ4AFwJy6YY4Hro6I3wNExOoW6zEzs4q1FBAR8YOIOAHYF7gXuF7SzZJOlrR5g9GmAytqunvyc7WeBzxD0o2Slkg6sTQhSadIWixp8Zo1a1op2czMhmggTUbbAScBbwVuAz5HCozrG41SeK7+lmLjgf2Aw4FXA2dKet5GI0XMj4iuiOiaOnVqqyWbmdkQtHQ3V0lXA7sBlwBHRMSq3OtySYsbjNYDzKzpngGsLAyzNiIeBx6X9GNgb+A3LdZvZmYVafUI4sKImB0Rn+oLB0lbAkREV4NxbgF2lbSLpC2AY4GFdcN8G3iZpPGStgZeBCwb8FKYmdmwazUgzi4899P+RoiIdcBpwHWkjf4VEbFU0qmSTs3DLAO+D9wJ/IIURL9stXgzM6tOv01MknYgnVjeStILePq8wjbA1s0mHhGLgEV1z82r6/534N8HULOZmY2AZucgXk06MT0D+GzN848BH6qoJjMz6wD9BkREXAxcLOkfI+KqEarJzMw6QLMmpjdGxKXALEnvre8fEZ8tjGZmZqNAsyamifnvpKoLMTOzztKsielL+e9HR6YcMzPrFM2amM7rr39EvHt4yzEzs07RrIlpyYhUYWZmHaeVq5jMzGwMatbEdG5EvEfStWx8oz0i4sjKKjMzs7Zq1sR0Sf77maoLMTOzztKsiWlJ/ntTvuHebqQjibvzjwCZmdko1ertvg8H5gG/I92PaRdJb4+I71VZnJmZtU9LAQHMBV4eEcsBJD0H+C7ggDAzG6Vavd336r5wyO4B/PvRZmajWLOrmF6f/10qaRFwBekcxNGkHwQyM7NRqlkT0xE1/z8AHJT/XwM8o5KKzIbJ3LnQ3Q29ve2uxGzT1OwqppNHqhCz4TZWwmGSb6VpFWn1KqYJwFuAPYAJfc9HxJsrqstsyMZKOHR3t7sKG61avYrpEuDXpF+Y+xhwAul3ps02CbHRfQDMrJlWr2J6bkScCTye7890OPD31ZVlZmbt1mpAPJn/PiJpT2BbYFYlFZmZWUdotYlpvqRnAGcCC0m/MHdmZVWZmVnbtRQQEXFh/vcm4NnVlWNmZp2ipSYmSdtJ+rykWyUtkXSupO2qLs7MzNqn1XMQC0i31vhH4ChgLXB5VUWZmVn7tXoO4pkR8fGa7rMlvbaCeszMrEO0egRxg6RjJW2WH8eQ7uZqZmajVLOb9T1GujmfgPcCl+ZemwG9wFmVVmdmZm3T7F5Mk0eqEDMz6yytnoNA0pHAgbnzxoj4TjUlmZlZJ2j1MtdzgNOBX+XH6fk5MzMbpVo9gjgM2Cci1gNIuhi4DfhAVYWZmVl7tXoVE8CUmv+3HeY6zMysw7R6BPFJ4DZJN5CuaDoQ+GBlVZmZWds1DQhJmwHrgQOAF5IC4oyIuL/i2szMrI2aNjHl8w6nRcSqiFgYEd9uNRwkHSLpbknLJTU8XyHphZKeknTUAGo3M7MKtXoO4npJ/yxppqRn9j36G0HSOOAC4FBgNnCcpNkNhvs0cN0Aazczswq1eg7izaRvVL+j7vn+bv29P7A8Iu4BkLQAmEO6TLbWu4CrSM1XZmbWIVo9gphNOhq4A7gd+DywR5NxpgMrarp78nN/I2k68DpgXn8TknSKpMWSFq9Zs6bFks3MbChaDYiLgd2B80jhsHt+rj8qPFf/0/Hnkk54P9XfhCJifkR0RUTX1KlTW6vYzMyGpNUmpudHxN413TdIuqPJOD3AzJruGcDKumG6gAWSALYHDpO0LiKuabEuMzOrSKtHELdJOqCvQ9KLgP9pMs4twK6SdpG0BXAs6fes/yYidomIWRExC7gSeIfDwcysM7R6BPEi4ERJv8/dOwHLJN0FRETsVT9CRKyTdBrp6qRxwEURsVTSqbl/v+cdzMysvVoNiEMGM/GIWAQsqnuuGAwRcdJg5mFmZtVoKSAi4r6qCzEzs84ykJv1mZnZGOKAMDOzIgeEmZkVOSDMzKzIAWFmZkUOCDMzK3JAjGFz58LkySCNzoeZDY0DYgzr7obe3nZXUb1Jk9pdgdmmyQExho2VcOjubncVZpumVm+1YaNc1N+I3czGPB9BmJlZkQPCzMyKHBBmZlbkgDAzsyIHhJmZFTkgzMysyAFhZmZFDggzMytyQJiZWZEDwszMihwQZmZW5IAwM7MiB4SZmRU5IMzMrMgBYWZmRQ4IMzMrckCYmVmRA8LMzIocEGZmVuSAMDOzIgeEmZkVOSDMzKzIAWFmZkWVBoSkQyTdLWm5pA8U+p8g6c78uFnS3lXWY2ZmrassICSNAy4ADgVmA8dJml032P8CB0XEXsDHgflV1WNmZgNT5RHE/sDyiLgnIp4AFgBzageIiJsj4uHc+TNgRoX1mJnZAFQZENOBFTXdPfm5Rt4CfK/UQ9IpkhZLWrxmzZphLNHMzBqpMiBUeC6KA0ovJwXEGaX+ETE/Iroiomvq1KnDWKKZmTUyvsJp9wAza7pnACvrB5K0F3AhcGhEPFhhPWZmNgBVHkHcAuwqaRdJWwDHAgtrB5C0E3A18KaI+E2FtZiZ2QBVdgQREesknQZcB4wDLoqIpZJOzf3nAR8BtgO+IAlgXUR0VVWTmZm1ThHF0wIdq6urKxYvXtzuMkYF1Zwl2sRWAzMbIElLBroD7m9Sm5lZkQOiH3PnwuTJaU97ND7MzPrjgOhHdzf09ra7iupNmtTuCsysEzkg+jFWwqG7u91VmFknqvJ7EKOKT+Ka2VjjIwgzMytyQJiZWZEDwszMihwQZmZW5IAwM7MiB4SZmRU5IMzMrMgBYWZmRQ4IMzMrckCYmVmRA8LMzIocEGZmVuSAMDOzIgeEmZkVOSDMzKzIAWFmZkUOCDMzK3JAmJlZkQPCzMyKHBBmZlbkgDAzsyIHhJmZFTkgzMysyAFhZmZFDggzMytyQJiZWZEDwszMihwQZmZW5IAwM7MiB4SZmRVVGhCSDpF0t6Tlkj5Q6C9J5+X+d0rat8p6zMysdZUFhKRxwAXAocBs4DhJs+sGOxTYNT9OAb5YVT1mZjYwVR5B7A8sj4h7IuIJYAEwp26YOcDXIvkZMEXSjv1NdMkSkEbmYWY2lo2vcNrTgRU13T3Ai1oYZjqwqnYgSaeQjjAAekF3D2+p/doeWDuKA2N7YG27i6iQl2/TNZqXDUZ++XYe6AhVBkRpkxqDGIaImA/MH46iBkrS4ojoase8R4KXb9M2mpdvNC8bbBrLV2UTUw8ws6Z7BrByEMOYmVkbVBkQtwC7StpF0hbAscDCumEWAifmq5kOAB6NiFX1EzIzs5FXWRNTRKyTdBpwHTAOuCgilko6NfefBywCDgOWA38CTq6qniFoS9PWCPLybdpG8/KN5mWDTWD5FLFRk7+ZmZm/SW1mZmUOCDMzK3JANCDpIkmrJf2y3bUMN0kzJd0gaZmkpZJOb3dNw0nSBEm/kHRHXr6PtrumKkgaJ+k2Sd9pdy3DTdK9ku6SdLukxe2uZ7hJmiLpSkm/zp/DF7e7phKfg2hA0oFAL+mb3nu2u57hlL+tvmNE3CppMrAEeG1E/KrNpQ0LSQImRkSvpM2BnwCn52/rjxqS3gt0AdtExGvaXc9wknQv0BURo/KLcpIuBv47Ii7MV3luHRGPtLmsjfgIooGI+DHwULvrqEJErIqIW/P/jwHLSN9gHxXyrVt6c+fm+TGq9oQkzQAOBy5sdy02MJK2AQ4EvgwQEU90YjiAA2LMkzQLeAHw8zaXMqxy88vtwGrg+ogYVcsHnAv8C7C+zXVUJYD/krQk32pnNHk2sAb4Sm4ivFDSxHYXVeKAGMMkTQKuAt4TEX9sdz3DKSKeioh9SN/O31/SqGkmlPQaYHVELGl3LRV6SUTsS7rj8ztzk+9oMR7YF/hiRLwAeBzY6OcQOoEDYozKbfNXAV+PiKvbXU9V8qH7jcAh7a1kWL0EODK30y8A/kHSpe0taXhFxMr8dzXwLdLdoUeLHqCn5qj2SlJgdBwHxBiUT+J+GVgWEZ9tdz3DTdJUSVPy/1sBrwR+3daihlFEfDAiZkTELNItbH4UEW9sc1nDRtLEfPEEuenlVcCouZowIu4HVkh6fn7qFUBHXiBS5d1cN2mSvgEcDGwvqQc4KyK+3N6qhs1LgDcBd+V2eoAPRcSi9pU0rHYELs4/WrUZcEVEjLpLQUexZwHfSvsxjAcui4jvt7ekYfcu4Ov5CqZ76MzbDPkyVzMzK3MTk5mZFTkgzMysyAFhZmZFDggzMytyQJiZWZEDwmwYSOptPpTZpsUBYWZmRQ4IswJJn5b0jprubklnSfqhpFvzbxXMKYx3cO3vM0g6X9JJ+f/9JN2Ub0B3Xb7tulnHckCYlS0A3lDTfQzwFeB1+SZyLwfm5tuWNJXvffV54KiI2A+4CPjE8JZsNrx8qw2zgoi4TdLfSZoGTAUeBlYB/5HvLLqe9BsazwLub2GSzwf2BK7PmTIuT8+sYzkgzBq7EjgK2IF0RHECKSz2i4gn891UJ9SNs44Nj8z7+gtYGhEd+dOSZiVuYjJrbAHpbqlHkcJiW9LvMDwp6eXAzoVx7gNmS9pS0rakO3UC3A1M7fvtYUmbS9qj8iUwGwIfQZg1EBFL822n/xARqyR9HbhW0mLgdgq3EI+IFZKuAO4Efgvclp9/QtJRwHk5OMaTfhVu6YgsjNkg+G6uZmZW5CYmMzMrckCYmVmRA8LMzIocEGZmVuSAMDOzIgeEmZkVOSDMzKzo/wOfdAXnA2fLYgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# make a step function from points, function adapted from http://stackoverflow.com/a/41185239\n",
    "def get_x_y_steps(x, y, where=\"post\"):\n",
    "    zipxx = list(zip(x, x))\n",
    "    zipyy = list(zip(y, y))\n",
    "    if where == \"post\":\n",
    "        x_step = [x[0]] + [_x for tup in zipxx[1:] for _x in tup]\n",
    "        y_step = [_y for tup in zipyy[:-1] for _y in tup] + [y[-1]]\n",
    "    elif where == \"pre\":\n",
    "        x_step = [_x for tup in zipxx[:-1] for _x in tup] + [x[-1]]\n",
    "        y_step = [y[0]] + [_y for tup in zipyy[1:] for _y in tup]\n",
    "    return x_step, y_step\n",
    "\n",
    "# create step function for CDF\n",
    "xcdf = np.hstack([0,x,x[-1]+1])\n",
    "ycdf = np.hstack([0,rv.cdf(x),1])\n",
    "(x_step,y_step) = get_x_y_steps(xcdf,ycdf)\n",
    "\n",
    "# plot CDF\n",
    "plt.plot(x_step, y_step,lw=3,color='b')\n",
    "plt.xlim(0.5,6.5)\n",
    "plt.ylim(0,1)\n",
    "plt.title('cumulative distribution function (CDF) for a roll of one die')\n",
    "plt.xlabel('value')\n",
    "plt.ylabel('probability');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Formal features of Probabilities\n",
    "__Mutiplication Principle__ - If two events $A$ and $B$ are unrelated, the probability of joint occurrence $P(A,B)$, i.e. both events happening, is multiplication of two probabilities,\n",
    "\n",
    "$P(A,B)  = p(B)p(A)$.\n",
    "\n",
    "This is a simplification of __Bayes' theorem__ for the special case where events are independent. In the more general case where the probability of events are not independent, \n",
    "\n",
    "$P(A,B) = p(B|A)p(A) = p(A|B)p(B)$,\n",
    "\n",
    "where $p(A|B)$ is the probability of $A$ occuring, given that $B$ has already occurred. In this case, the two events depend on each other, which is not the case for dice.\n",
    "\n",
    "__Additive principle__ - Probability of one event or another mutually exclusive event is the sum of the probabilities. Mutually exclusive means that the events cannot occur at the same time.\n",
    "\n",
    "$P(C|D) = P(C) + P(D)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bayes theorem – diachronic interpretation\n",
    "\n",
    "__Diachronic__: happening over time\n",
    "\n",
    "Updating a hypothesis (H) given new data (D):\n",
    "\n",
    "$P(H|D) = \\frac{P(H)P(D|H)}{P(D)}$\n",
    "\n",
    "* $P(H)$ - probability of a hypothesis before seeing data (“prior”)\n",
    "\n",
    "\n",
    "* $P(H|D)$  - probability of a hypothesis after seeing data (“posterior”)\n",
    "\n",
    "\n",
    "* $P(D|H)$ - probability of the data given the hypothesis (“likelihood”)\n",
    "\n",
    "\n",
    "* $P(D)$ - probability of the data under any hypothesis (“normalizing constant”)\n",
    "\n",
    "$P(D)$ is often the hardest to understand and quantify\n",
    "\n",
    "Problems can often be simplified by laying out a suite of hypotheses that are:\n",
    "\n",
    "1. Mutually exclusive\n",
    "\n",
    "2. Collectively exhaustive\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sum of two dice\n",
    "\n",
    "From the rules above, we can calculate the probability mass function and cumulative distribution function for the sum of two dice. From the multiplication rule, the probability of rolling a particular sequence of dice is (1/6) $\\times$ (1/6) = 1/36. This is the probability of a joint occurrence of two independent events. For example, the combinations (1 and 1), (1 and 2), (2 and 1) and (1 and 3) are all joint occurrences of two independent events.\n",
    "\n",
    "The probability of obtaining a certain sum, which may be arrived at through several possible sequences of dice rolls, can be calculated from the addition rule. There are three ways of obtaining a sum of four: (1 and 3) or (2 and 2) or (3 and 1). These combinations of rolls are mutually exlusive. Therefore the probability of rolling a four is (1/36) + (1/36) + (1/36) = 1/12.\n",
    "\n",
    "### The Monty Hall problem\n",
    "\n",
    "A classic application of Bayes theorem, showing how our intuition about probability can lead us astray."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Distributions\n",
    "\n",
    "### Normal Distribution\n",
    "\n",
    "\n",
    "![Normal dist](images/Standard_deviation_diagram.png)\n",
    "\n",
    "* __Mean__ - 1st moment\n",
    "* __Variance__ - 2nd moment\n",
    "* __Skew__ - 3rd moment, Describes the spread (Normal Distribution, skew = 0)\n",
    "* __Kurtosis__ - 4th moment, Describes how pointy the distribution is. Low value would be flat (compared to a Normal distribution (k=3))\n",
    "\n",
    "### Non-normal distributions\n",
    "\n",
    "__Example: Life expectancy__\n",
    "\n",
    "Negative skew towards later life (spike at birth). The cumulative probability function is useful because we are not necessarily interested in the probability of dying at a very specific age, like exactly 67 years. We are more interested in the probability of living until a certain age or less. The cumulative probability distribution tapers off to one for very high ages. The probability of living to 120 years or less is equal to one.\n",
    "\n",
    "![life expectancy](images/life_expectancy_1.png)\n",
    "\n",
    "source: http://www.cureffi.org/2013/06/19/how-censoring-by-life-expectancy-affects-age-of-onset-distributions/\n",
    "\n",
    "\n",
    "### Poisson Distribution\n",
    "  * Often used as a model for count data\n",
    "  * Assumptions:\n",
    "    * One event does not affect the probability of the next (they are independent)\n",
    "    * Events cannot occur in the same time and place in the interval\n",
    "    * Events occur at a constant rate<br>\n",
    "\n",
    "$k$ = # of events counted per interval (ie floods per century)\n",
    "\n",
    "$\\lambda$ = expected values, true mean\n",
    "    \n",
    "$P(k) = \\frac{\\lambda^k e^{-\\lambda}}{k!}$\n",
    "\n",
    "Count uncertainty  = $\\sqrt{k}$ - More counts => less uncertainty\n",
    "\n",
    "__Exercise__: if on average 10 flood occur per century. What is probability of counting 15 floods occuring in a century?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Weibull Distribution \n",
    "\n",
    "A theoretical distribution that is often fit to wind speed data. Wind speed has only positive values, so the probability of a negative wind speed is zero. The distribution of wind speed is generally __skewed__, since very high values do occur, but are rare. This theoretical distribution is useful for engineering wind power projects, since it can be defined by a few parameters. The parameters obtained from a fit to data at a given location can be used as inputs to models for designing equipment with maximum efficiency.\n",
    "\n",
    "![weibull distribution](images/weibull.png)\n",
    "\n",
    "Source: http://www.wind-powemr-program.com/wind_statistics.htm\n",
    "\n",
    "\n",
    "### Log-Normal Distribution \n",
    "\n",
    "Many biological data, such as growth rates, follow a log-normal distribution. The distributions are skewed, with a peak close to zero but a long tail of rare high values. These data can be transformed by taking the logarithm, giving a distribution that is more symmetric and closer to a normal distribution.\n",
    "\n",
    "![log normal](images/log_normal.png)\n",
    "\n",
    "source: Landry, M.R. and B.M. Hickey (eds.) (1989) Coastal Oceanography of Washington and\n",
    "Oregon, 607 pp., Elsevier Science, Amsterdam, The Netherlands."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Back to Normal Distributions\n",
    "\n",
    "Random instrument error is often normally distributed. The mean of errors tends to be zero if error is randomly distributed and __accurate__, i.e. not biased.\n",
    "\n",
    "![satellite error](images/instr_error_dist.png)\n",
    "\n",
    "source: Connolly and Lentz (2014)\n",
    "\n",
    "In this example, error is defined as: $T_{sat} - T_{mooring}$ units of [$^o$C]. There is a fairly large spread in the data. We therefore cannot be extremely confident in any particular satellite image. However, we can be confident that the mean of a large number of errors is close to zero.\n",
    "\n",
    "Normal distributions can be \"standardized\", where the\n",
    "* mean = 0\n",
    "* standard deviation = 1\n",
    "* Unitless\n",
    "\n",
    "The __z-score__ transforms the data so that each data point is unitless and described in terms of standard deviations away from the mean.\n",
    "\n",
    "$z_i = \\frac{x_i - \\bar{x}}{ s }$\n",
    "\n",
    "The z-score can be helpful for identifying outliers (large Z-scores). Outliers are often classified as having a z-score of $\\pm$ 3. However, the basic rules of scientific integrity require that you have a good reason for excluding outliers from your analysis, and report that they have been excluded. Never delete outliers from the original data set."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "data-book",
   "language": "python",
   "name": "data-book"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}