pydata-tutorial-simpsons-SESSION.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# D’oh! Unevenly spaced time series analysis of The Simpsons in Pandas\n",
    "#### [PyData Seattle 2017 tutorial](https://pydata.org/seattle2017/schedule/presentation/104/)\n",
    "#### [Joe McCarthy](http://interrelativity.com/joe/), Data Scientist, [Indeed](https://www.indeed.com)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Motivations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Unevenly spaced opportunities to do unevenly spaced time series data analysis\n",
    "\n",
    "<p><a href=\"http://pandas.pydata.org/\"><img src=\"http://pandas.pydata.org/_static/pandas_logo.png\" style=\"margin: 0px 0px 10px 20px; width: 150px; float: right;\" title=\"pandas\" alt=\"pandas_logo.png\" /></a>I occasionally need to analyze gaps between events of interest, but not so often that I remember how to do it. I end up searching through my notebooks to remind myself which Pandas methods to use, and how to use them. I wanted to create a protypical example notebook that I can more easily reference .. and wanted to share it with others, in case they might also find it useful.</p>\n",
    "\n",
    "### Data.world\n",
    "\n",
    "<p><a href=\"https://data.world\"><img src=\"https://d2ogkq1rg66kh0.cloudfront.net/site-resources/images/fb-image-share.7b15f964.jpg\" style=\"margin: 0px 12px 5px 20px; width: 125px; float: right;\" title=\"data.world\" alt=\"data_world_logo.png\" /></a>The last time I hunted down a notebook containing unevenly spaced time series analysis was around the time I first heard about the [data.world](https://data.world) platform for creating, sharing and collaborating on data sets .. I think of it as a GitHub for data.</p>\n",
    "\n",
    "<p><a href=\"https://www.dataquest.io\"><img src=\"http://data-science-hack.com/wp-content/uploads/2016/04/dataquest-io-1-1.png\" style=\"margin: 0px 0px 5px 20px; width: 125px; float: right;\" title=\"dataquest.io\" alt=\"dataquest-io-logo.png\" /></a>I was further inspired about using data.world after reading a great tutorial created by Josh Devlin at [Dataquest.io](https://www.dataquest.io) on [Turbocharge Your Data Acquisition using the data.world Python Library](https://www.dataquest.io/blog/datadotworld-python-tutorial/). This is also where I discovered the dataset about The Simpsons episodes, which I think is more entertaining and generally accessible than the datasets I typically work with at Indeed (although those are interesting, too).</p>\n",
    "\n",
    "### PyData redemption\n",
    "\n",
    "I gave a tutorial on [Python for Data Science](https://pydata.org/seattle2015/schedule/presentation/8/) at PyData Seattle 2015, wherein I tried to squeeze a 4-hour tutorial into a 2-hour slot. It didn't work out so well, though I did make the [Jupyter Notebook from the tutorial](http://nbviewer.jupyter.org/github/gumption/Python_for_Data_Science/blob/master/Python_for_Data_Science_all.ipynb) publicly available, so attendees could go back over some of the things we rushed through or didn't even get to. \n",
    "\n",
    "For PyData Seattle 2017, I decided to propose a topic that I estimated would take more like an hour, so that we can take a somewhat more leisurely approach .. and, with any luck, we'll be able to get to lunch before the other morning tutorials let out. \n",
    "\n",
    "And I will be making this notebook publicly available as well.\n",
    "\n",
    "### We're hiring\n",
    "\n",
    "At the risk of shameless promotion, we have a number of [open positions in Indeed's Seattle office](https://www.indeed.jobs/career/JobListSeattle), including\n",
    "* [Product Scientist](https://www.indeed.jobs/career/JobDetail/Product-Scientist/3949)\n",
    "* [Data Science Manager](https://www.indeed.jobs/career/JobDetail/Data-Science-Manager/4357)\n",
    "\n",
    "We have a number of Software Engineering and User Experience positions as well."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using Python and Pandas in a Jupyter Notebook"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Among the prerequisites for this tutorial are to have Python, Pandas and Jupyter Notebook installed on your computer. In case anyone has a computer that does not yet meet these requirements, I recommend downloading and installing [Anaconda](https://www.continuum.io/downloads), an open data science platform for Python, which includes Pandas, Jupyter Notebook and a variety of other useful open source Python libraries. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Python 2 vs. Python 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are 2 major versions of Python in widespread use: [Python 2](https://docs.python.org/2/) and [Python 3](https://docs.python.org/3/). Python 3 has some features that are not backward compatible with Python 2, and some Python 2 libraries have not been updated to work with Python 3. I have been using Python 2, primarily because I use some of those Python 2[-only] libraries, but an increasing proportion of them are migrating to Python 3, and I anticipate shifting to Python 3 in the near future.\n",
    "\n",
    "For more on the topic, I recommend a very well documented IPython Notebook, which includes numerous helpful examples and links, by [Sebastian Raschka](http://sebastianraschka.com/), [Key differences between Python 2.7.x and Python 3.x](http://nbviewer.ipython.org/github/rasbt/python_reference/blob/master/tutorials/key_differences_between_python_2_and_3.ipynb), the [Cheat Sheet: Writing Python 2-3 compatible code](http://python-future.org/compatible_idioms.html) by Ed Schofield ... or [googling Python 2 vs 3](https://www.google.com/q=python%202%20vs%203).\n",
    "\n",
    "In order to make this notebook compatible with Python 2 *or* Python 3, we will import the [`print_function`]((https://docs.python.org/3/library/functions.html#print)) from the  [**`__future__`**](https://docs.python.org/2/library/__future__.html) module (in Python 2, `print` is a [statement](https://docs.python.org/2/reference/simple_stmts.html#print) not a function). We will also import the `division` module from the `future`, as I find [the use of `/` for \"true division\"](https://www.python.org/dev/peps/pep-0238/) - and the use of `//` for \"floor division\" - to be more aligned with my intuition.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from __future__ import print_function, division"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will also import - using the customary aliases - commonly used libraries in the python scientific stack, and specify that any plots will appear inline within the cell of this notebook (vs. popping up a separate window)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using pandas\n",
    "\n",
    "This tutorial is designed for people who are already familiar with the `pandas` data analysis library.\n",
    "\n",
    "A thorough review of `pandas` concepts is beyond the scope of this session, but for participants who are not sufficiently familiar with `pandas`, I highly recommend the pandas [tutorial (\"10 Minutes to pandas\")](https://pandas.pydata.org/pandas-docs/stable/10min.html) and [Intro to Data Structures](https://pandas.pydata.org/pandas-docs/stable/dsintro.html).\n",
    "\n",
    "Here are a few basic concepts:\n",
    "\n",
    "* A [`DataFrame`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.html) is a tabular (2-dimensional) data structure with 2 axes: rows (`axis=0`) and columns (`axis=1`)\n",
    "* The columns in a `DataFrame` are one-dimensional arrays called [`Series`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.html)\n",
    "* Elements of a `DataFrame` can be accessed using labels, positions or boolean vectors\n",
    "  * The labels for a `Series` or the rows of a `DataFrame` are accessible via their `index` attribute\n",
    "  * The labels for each column in a `DataFrame` are accessible via its `columns` attribute\n",
    "  * Individual labels can be specified using brackets or - in most cases - dotted notation\n",
    "  * Positions can be specified using the [slice operators](https://docs.python.org/2/tutorial/introduction.html) for python sequences\n",
    "  \n",
    "Examples of these concepts will be highlighted when we first encounter them.\n",
    "\n",
    "Other important `pandas` concepts and operations used throughout this tutorial include [grouping (split-apply-combine)](http://pandas.pydata.org/pandas-docs/stable/groupby.html) and [merging](http://pandas.pydata.org/pandas-docs/stable/merging.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using the data.world Python Library"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this section, I will borrow heavily - with permission - from Josh Devlin's tutorial, [Turbocharge Your Data Acquisition using the data.world Python library](https://www.dataquest.io/blog/datadotworld-python-tutorial/).\n",
    "\n",
    "There are additional elements incorporated from the [Python SDK documentation](https://data.world/nrippner/explore-the-data-world-python-sdk/workspace/file?filename=ddw_SDK.ipynb) the [Data Wrangling tutorial](https://data.world/nrippner/python-data-wrangling-tutorial/workspace/file?filename=datadotworld_wrangling_tutorial.ipynb) at data.world."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Installing the data.world library"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data.world Python SDK can be installed via the command line using the [`conda install` utility](https://conda.io/docs/using/pkgs.html#install-a-package-from-anaconda-org):\n",
    "\n",
    "`conda install -c conda-forge datadotworld-py`\n",
    "\n",
    "Alternately, you can use `pip` to install the SDK:\n",
    "\n",
    "`pip install git+git://github.com/datadotworld/data.world-py.git`\n",
    "\n",
    "Next, you need to retrieve and store your data.world API token.\n",
    "\n",
    "* From your data.world account, go to [Settings > Advanced](https://data.world/settings/advanced) and get your API token (the image below is from Josh's tutorial):\n",
    "\n",
    "<a href=\"https://data.world/settings/advanced\"> <img src=\"https://www.dataquest.io/blog/images/data.world_tutorial/api_token.png\" style=\"width: 600px;\" title=\"data.world Advanced Settings\" alt=\"data_world_api_token.png\" /></a>\n",
    "\n",
    "* If you installed the data.world Python SDK in a virtualenv or Conda env, activate that environment. \n",
    "* Run `dw configure` on the command line, which will prompt you for your token:\n",
    "```\n",
    "~ (datadotworld) $ dw configure\n",
    "API token (obtained at: https://data.world/settings/advanced): _\n",
    "```\n",
    "When you enter your token, a `~/.dw/` directory will created in your home directory and your token will be stored in your `~/.dw/config` file.\n",
    "\n",
    "Finally, if you used `pip install` above, you will need to run `pip install datadotworld[PANDAS]` on the command line."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The Simpsons"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the benefit of anyone who is not familiar with The Simpsons, here is some context from the [Wikipedia page](https://en.wikipedia.org/wiki/The_Simpsons) for the television show:\n",
    "\n",
    "<blockquote>\n",
    "<p><a href=\"https://en.wikipedia.org/wiki/The_Simpsons\"><img src=\"https://upload.wikimedia.org/wikipedia/en/thumb/0/0d/Simpsons_FamilyPicture.png/220px-Simpsons_FamilyPicture.png\" style=\"margin: 0px 0px 10px 20px; width: 150px; float: right;\" title=\"The Simpsons\" alt=\"Simpsons_FamilyPicture.png\" /></a>The Simpsons is an American animated sitcom created by Matt Groening for the Fox Broadcasting Company. The series is a satirical depiction of working-class life epitomized by the Simpson family, which consists of Homer, Marge, Bart, Lisa, and Maggie. The show is set in the fictional town of Springfield and parodies American culture, society, television, and the human condition.</p>\n",
    "\n",
    "<p>...</p>\n",
    "\n",
    "<p>Since its debut on December 17, 1989, 618 episodes of The Simpsons have been broadcast. Its 28th season began on September 25, 2016. It is the longest-running American sitcom and the longest-running American animated program, and, in 2009, it surpassed Gunsmoke as the longest-running American scripted primetime television series. The Simpsons Movie, a feature-length film, was released in theaters worldwide on July 27, 2007, and grossed over $527 million. On November 4, 2016, the series was renewed for a twenty-ninth and thirtieth season of 22 episodes each, extending the show to 2019.</p>\n",
    "</blockquote>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The Simpsons data set at data.world"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Josh provides a brief history of this data set in [his tutorial](https://www.dataquest.io/blog/datadotworld-python-tutorial/):\n",
    "\n",
    "> For this tutorial, we’ll be working with a data set of information on the TV show, [The Simpsons](https://en.wikipedia.org/wiki/The_Simpsons). The dataset was scraped by Tod Schenider for his post [The Simpsons by the Data](http://toddwschneider.com/posts/the-simpsons-by-the-data/), for which he made the scraper [available on GitHub](https://github.com/toddwschneider/flim-springfield). [Kaggle user William Cukierski used the scraper to upload the data set](https://www.kaggle.com/wcukierski/the-simpsons-by-the-data), which was then [rehosted on data.world](https://data.world/data-society/the-simpsons-by-the-data).\n",
    "\n",
    "[The data set page on data.world](https://data.world/data-society/the-simpsons-by-the-data) shows four CSV files in the data set:\n",
    "\n",
    "* `simpsons_characters.csv`: Every character appearing in The Simpsons.\n",
    "* `simpsons_episodes.csv`: Every episode of the The Simpsons.\n",
    "* `simpsons_locations.csv`: Every location appearing in The Simpsons.\n",
    "* `simpsons_script_lines.csv`: Most lines from most scripts of the Simpsons.\n",
    "\n",
    "In _this_ tutorial, we'll only be using the data from `simpsons_episodes.csv` and `simpsons_script_lines.csv`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using data.world’s Python library to explore the data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Continuing with [Josh's tutorial](https://www.dataquest.io/blog/datadotworld-python-tutorial/):\n",
    "\n",
    "> First, let’s import the `datadotworld` library:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import datadotworld as dw"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> We’re going to use the `load_dataset()` function to take a look at the data. When we use `load_dataset()` for the first time, it:\n",
    "> \n",
    "> * Downloads the data set from data.world and caches it in our `~/.dw/` directory\n",
    "> * Returns a `LocalDataset` object representing the data set\n",
    "> \n",
    "> Caching the data set locally is a really neat feature - it allows for quicker subsequent loading, lets you work on the data offline, ensures that your source data is the same each time you run your code, and in the future will support data set versioning. After the first time you call `load_dataset()` for a given dataset, it will load the dataset from the cached version. You can pass `True` to the optional `force_update` parameter if you wish to force a reload from the remote version and overwrite the changes.\n",
    "> \n",
    "> `load_dataset()` has one required parameter, `dataset_key` which you can extract from the URL of the data set on data.world. As an example, our simpsons data set has the URL https://data.world/data-society/the-simpsons-by-the-data, which makes its ID `data-society/the-simpsons-by-the-data`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "lds = dw.load_dataset('data-society/the-simpsons-by-the-data')  # , force_update=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After loading the dataset, we can verify that it has been locally cached."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "total 35060\r\n",
      "-rw-r--r-- 1 joem staff   222548 Jul  3 15:30 simpsons_characters.csv\r\n",
      "-rw-r--r-- 1 joem staff   125999 Jul  3 15:30 simpsons_episodes.csv\r\n",
      "-rw-r--r-- 1 joem staff   182776 Jul  3 15:30 simpsons_locations.csv\r\n",
      "-rw-r--r-- 1 joem staff 35363962 Jul  3 15:30 simpsons_script_lines.csv\r\n"
     ]
    }
   ],
   "source": [
    "! ls -l ~/.dw/cache/data-society/the-simpsons-by-the-data/latest/data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Getting to know our data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> To take a closer look at our `LocalDataset` object , we can use the `LocalDataset.describe()` method, which returns a JSON object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{u'description': u'Contains the characters, locations, episode details, and script lines for approximately 600 Simpsons episodes.\\n\\nOriginally, this dataset was scraped by Tod Schenider for his post [The Simpsons by the Data](http://toddwschneider.com/posts/the-simpsons-by-the-data/), for which he made the scraper [available on GitHub](https://github.com/toddwschneider/flim-springfield). [Kaggle user William Cukierski used the scraper to upload the data set](https://www.kaggle.com/wcukierski/the-simpsons-by-the-data), which has been rehosted here.\\n\\nAn additional column, `chalkboard_gag`, has been added to the `simpsons_episodes.csv` file, scraped from [Simpsons Wiki](http://simpsons.wikia.com/wiki/List_of_chalkboard_gags).',\n",
      " u'homepage': u'https://data.world/data-society/the-simpsons-by-the-data',\n",
      " u'keywords': [u'data society',\n",
      "               u'simpsons',\n",
      "               u'text mining',\n",
      "               u'characters',\n",
      "               u'script lines',\n",
      "               u'regression',\n",
      "               u'classification'],\n",
      " u'name': u'data-society_the-simpsons-by-the-data',\n",
      " u'resources': [{u'format': u'csv',\n",
      "                 u'name': u'simpsons_characters',\n",
      "                 u'path': u'data/simpsons_characters.csv'},\n",
      "                {u'format': u'csv',\n",
      "                 u'name': u'simpsons_episodes',\n",
      "                 u'path': u'data/simpsons_episodes.csv'},\n",
      "                {u'format': u'csv',\n",
      "                 u'name': u'simpsons_locations',\n",
      "                 u'path': u'data/simpsons_locations.csv'},\n",
      "                {u'format': u'csv',\n",
      "                 u'name': u'simpsons_script_lines',\n",
      "                 u'path': u'data/simpsons_script_lines.csv'},\n",
      "                {u'bytes': 214869,\n",
      "                 u'format': u'csv',\n",
      "                 u'mediatype': u'text/csv',\n",
      "                 u'name': u'original/simpsons_characters.csv',\n",
      "                 u'path': u'original/simpsons_characters.csv'},\n",
      "                {u'bytes': 125398,\n",
      "                 u'format': u'csv',\n",
      "                 u'mediatype': u'text/csv',\n",
      "                 u'name': u'original/simpsons_episodes.csv',\n",
      "                 u'path': u'original/simpsons_episodes.csv'},\n",
      "                {u'bytes': 178199,\n",
      "                 u'format': u'csv',\n",
      "                 u'mediatype': u'text/csv',\n",
      "                 u'name': u'original/simpsons_locations.csv',\n",
      "                 u'path': u'original/simpsons_locations.csv'},\n",
      "                {u'bytes': 35185146,\n",
      "                 u'format': u'csv',\n",
      "                 u'mediatype': u'text/csv',\n",
      "                 u'name': u'original/simpsons_script_lines.csv',\n",
      "                 u'path': u'original/simpsons_script_lines.csv'}],\n",
      " u'title': u'The Simpsons by the Data'}\n"
     ]
    }
   ],
   "source": [
    "# We use pprint as it makes our output easier to read \n",
    "import pprint as pp\n",
    "\n",
    "pp.pprint(lds.describe())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The JSON object has additional information since Josh created his tutorial, which has been inserted below:\n",
    "\n",
    "> Our JSON object has 6 key/value pairs at the top level: \n",
    "* `description`\n",
    "* `homepage` \n",
    "* `keywords` \n",
    "* `name` \n",
    "* `resouces` \n",
    "* `title`\n",
    "\n",
    "> `resources` is a list that contains information about each file in our data.world data set:\n",
    "* `name` \n",
    "* `format`\n",
    "* `path`\n",
    "* `mediatype`\n",
    "* `bytes`\n",
    "\n",
    "> Along with the `LocalDataset.describe()` function, there are three key attributes of our LocalDataset object which we can use to access the data itself:  \n",
    "* `LocalDataset.dataframes`\n",
    "* `LocalDataset.tables`\n",
    "* `LocalDataset.raw_data`\n",
    "\n",
    "> Each of these attributes work the same way, but return the data in a different format."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{u'simpsons_episodes': LazyLoadedValue(<pandas.DataFrame>), u'simpsons_characters': LazyLoadedValue(<pandas.DataFrame>), u'simpsons_script_lines': LazyLoadedValue(<pandas.DataFrame>), u'simpsons_locations': LazyLoadedValue(<pandas.DataFrame>)} \n",
      "\n",
      "{u'simpsons_episodes': LazyLoadedValue(<list of rows>), u'simpsons_characters': LazyLoadedValue(<list of rows>), u'simpsons_script_lines': LazyLoadedValue(<list of rows>), u'simpsons_locations': LazyLoadedValue(<list of rows>)} \n",
      "\n",
      "{u'simpsons_episodes': LazyLoadedValue(<bytes>), u'original/simpsons_locations.csv': LazyLoadedValue(<bytes>), u'simpsons_characters': LazyLoadedValue(<bytes>), u'original/simpsons_episodes.csv': LazyLoadedValue(<bytes>), u'simpsons_locations': LazyLoadedValue(<bytes>), u'simpsons_script_lines': LazyLoadedValue(<bytes>), u'original/simpsons_script_lines.csv': LazyLoadedValue(<bytes>), u'original/simpsons_characters.csv': LazyLoadedValue(<bytes>)} \n",
      "\n"
     ]
    }
   ],
   "source": [
    "for i in [lds.dataframes, lds.tables, lds.raw_data]:\n",
    "    print(i, '\\n')  # pprint does not workon lazy-loaded dicts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> `LocalDataset.dataframes` returns a dictionary of pandas DataFrame objects, where as `LocalDataset.tables` and `LocalDataset.raw_data` returns the data in dictionaries of Python lists and bytes format respectively. Lists can be useful if we don’t want to use pandas, and bytes is great if we have binary data like images or database files.\n",
    ">\n",
    "> Because of the power of the pandas library, let’s use `LocalDataset.dataframes` to explore and have some fun with our data!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exploratory analysis of The Simpsons data using `pandas`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "data.world provides a number of ways of working with data, including their own version of SQL (dwSQL), SPARQL and an R SDK (see their [Intro to data.world Dataset tutorial](https://data.world/jonloyens/an-intro-to-dataworld-dataset)), but in this tutorial we will focus only on using the Python SDK and using the `pandas DataFrames` in their datasets.\n",
    "\n",
    "We will create a `DataFrame` variable, `df`, to work with data from the `simpsons_script_lines` CSV file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/joem/anaconda/lib/python2.7/site-packages/datadotworld/models/dataset.py:192: UserWarning: Unable to set data frame dtypes automatically using simpsons_script_lines schema. Data types may need to be adjusted manually. Error: Integer column has NA values in column 7\n",
      "  'Error: {}'.format(resource_name, e))\n",
      "/Users/joem/anaconda/lib/python2.7/site-packages/datadotworld/util.py:136: DtypeWarning: Columns (4,5,6) have mixed types. Specify dtype option on import or set low_memory=False.\n",
      "  return self._loader_func()\n"
     ]
    }
   ],
   "source": [
    "df = lds.dataframes['simpsons_script_lines']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The warnings show that we have some columns with mixed types in this `DataFrame`.\n",
    "\n",
    "```\n",
    "/Users/joem/anaconda/lib/python2.7/site-packages/datadotworld/models/dataset.py:192: UserWarning: Unable to set data frame dtypes automatically using simpsons_script_lines schema. Data types may need to be adjusted manually. Error: Integer column has NA values in column 7\n",
    "  'Error: {}'.format(resource_name, e))\n",
    "/Users/joem/anaconda/lib/python2.7/site-packages/datadotworld/util.py:136: DtypeWarning: Columns (4,5,6) have mixed types. Specify dtype option on import or set low_memory=False.\n",
    "  return self._loader_func()\n",
    "```\n",
    "\n",
    "We can use the [`pandas.DataFrame.info()`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.info.html) function to get more information on the contents of these columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 158271 entries, 0 to 158270\n",
      "Data columns (total 13 columns):\n",
      "id                    158271 non-null int64\n",
      "episode_id            158271 non-null int64\n",
      "number                158271 non-null int64\n",
      "raw_text              158271 non-null object\n",
      "timestamp_in_ms       158271 non-null object\n",
      "speaking_line         158271 non-null object\n",
      "character_id          140750 non-null object\n",
      "location_id           157864 non-null float64\n",
      "raw_character_text    140749 non-null object\n",
      "raw_location_text     157863 non-null object\n",
      "spoken_words          132112 non-null object\n",
      "normalized_text       132087 non-null object\n",
      "word_count            132112 non-null object\n",
      "dtypes: float64(1), int64(3), object(9)\n",
      "memory usage: 15.7+ MB\n"
     ]
    }
   ],
   "source": [
    "df.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "9 columns have type `object`, which is usually an indicator that the column contains a mix of data types. \n",
    "\n",
    "Sometimes this happens when columns include null (`NaN`) values, so let's show the number of null values along with the number of data types, and include the number of unique values in each column. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " 0: id:                  158271 unique values,      0 null values,  1 data type(s)\n",
      " 1: episode_id:             564 unique values,      0 null values,  1 data type(s)\n",
      " 2: number:                 395 unique values,      0 null values,  1 data type(s)\n",
      " 3: raw_text:            148280 unique values,      0 null values,  1 data type(s)\n",
      " 4: timestamp_in_ms:       2704 unique values,      0 null values,  2 data type(s)\n",
      " 5: speaking_line:            5 unique values,      0 null values,  2 data type(s)\n",
      " 6: character_id:          7185 unique values,  17521 null values,  2 data type(s)\n",
      " 7: location_id:           4459 unique values,    407 null values,  1 data type(s)\n",
      " 8: raw_character_text:    6765 unique values,  17522 null values,  2 data type(s)\n",
      " 9: raw_location_text:     4498 unique values,    408 null values,  2 data type(s)\n",
      "10: spoken_words:        122000 unique values,  26159 null values,  2 data type(s)\n",
      "11: normalized_text:     119804 unique values,  26184 null values,  2 data type(s)\n",
      "12: word_count:             114 unique values,  26159 null values,  2 data type(s)\n"
     ]
    }
   ],
   "source": [
    "for i, column_label in enumerate(df.columns):\n",
    "    print('{:2d}: {:20} {:6d} unique values, {:6d} null values, {:2d} data type(s)'.format(\n",
    "        i,\n",
    "        column_label + ':', \n",
    "        df[column_label].nunique(),\n",
    "        df[column_label].isnull().sum(),\n",
    "        len(df[column_label].apply(lambda x: type(x)).value_counts())))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also see what kinds of values are stored in the columns that have mixed data types."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4 timestamp_in_ms\n",
      "<type 'int'>    131072\n",
      "<type 'str'>     27199\n",
      "Name: timestamp_in_ms, dtype: int64\n",
      "\n",
      "5 speaking_line\n",
      "<type 'bool'>    131072\n",
      "<type 'str'>      27199\n",
      "Name: speaking_line, dtype: int64\n",
      "\n",
      "6 character_id\n",
      "<type 'float'>    133921\n",
      "<type 'str'>       24350\n",
      "Name: character_id, dtype: int64\n",
      "\n",
      "8 raw_character_text\n",
      "<type 'str'>      140749\n",
      "<type 'float'>     17522\n",
      "Name: raw_character_text, dtype: int64\n",
      "\n",
      "9 raw_location_text\n",
      "<type 'str'>      157863\n",
      "<type 'float'>       408\n",
      "Name: raw_location_text, dtype: int64\n",
      "\n",
      "10 spoken_words\n",
      "<type 'str'>      132112\n",
      "<type 'float'>     26159\n",
      "Name: spoken_words, dtype: int64\n",
      "\n",
      "11 normalized_text\n",
      "<type 'str'>      132087\n",
      "<type 'float'>     26184\n",
      "Name: normalized_text, dtype: int64\n",
      "\n",
      "12 word_count\n",
      "<type 'str'>      132112\n",
      "<type 'float'>     26159\n",
      "Name: word_count, dtype: int64\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for i, column_label in enumerate(df.columns):\n",
    "    if len(df[column_label].apply(lambda x: type(x)).value_counts()) > 1:\n",
    "        print(i, column_label)\n",
    "        print(df[column_label].apply(lambda x: type(x)).value_counts())\n",
    "        print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Preparation & Cleaning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are a number of columns which have more than one data type, e.g., `str` and `float`. There are some other data quality issues in the `DataFrame` created from `simpsons_script_lines.csv`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index([u'id', u'episode_id', u'number', u'raw_text', u'timestamp_in_ms',\n",
       "       u'speaking_line', u'character_id', u'location_id',\n",
       "       u'raw_character_text', u'raw_location_text', u'spoken_words',\n",
       "       u'normalized_text', u'word_count'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>episode_id</th>\n",
       "      <th>number</th>\n",
       "      <th>raw_text</th>\n",
       "      <th>timestamp_in_ms</th>\n",
       "      <th>speaking_line</th>\n",
       "      <th>character_id</th>\n",
       "      <th>location_id</th>\n",
       "      <th>raw_character_text</th>\n",
       "      <th>raw_location_text</th>\n",
       "      <th>spoken_words</th>\n",
       "      <th>normalized_text</th>\n",
       "      <th>word_count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>9549</td>\n",
       "      <td>32</td>\n",
       "      <td>209</td>\n",
       "      <td>Miss Hoover: No, actually, it was a little of ...</td>\n",
       "      <td>848000</td>\n",
       "      <td>True</td>\n",
       "      <td>464</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Miss Hoover</td>\n",
       "      <td>Springfield Elementary School</td>\n",
       "      <td>No, actually, it was a little of both. Sometim...</td>\n",
       "      <td>no actually it was a little of both sometimes ...</td>\n",
       "      <td>31</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>9550</td>\n",
       "      <td>32</td>\n",
       "      <td>210</td>\n",
       "      <td>Lisa Simpson: (NEAR TEARS) Where's Mr. Bergstrom?</td>\n",
       "      <td>856000</td>\n",
       "      <td>True</td>\n",
       "      <td>9</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Lisa Simpson</td>\n",
       "      <td>Springfield Elementary School</td>\n",
       "      <td>Where's Mr. Bergstrom?</td>\n",
       "      <td>wheres mr bergstrom</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>9551</td>\n",
       "      <td>32</td>\n",
       "      <td>211</td>\n",
       "      <td>Miss Hoover: I don't know. Although I'd sure l...</td>\n",
       "      <td>856000</td>\n",
       "      <td>True</td>\n",
       "      <td>464</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Miss Hoover</td>\n",
       "      <td>Springfield Elementary School</td>\n",
       "      <td>I don't know. Although I'd sure like to talk t...</td>\n",
       "      <td>i dont know although id sure like to talk to h...</td>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>9552</td>\n",
       "      <td>32</td>\n",
       "      <td>212</td>\n",
       "      <td>Lisa Simpson: That life is worth living.</td>\n",
       "      <td>864000</td>\n",
       "      <td>True</td>\n",
       "      <td>9</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Lisa Simpson</td>\n",
       "      <td>Springfield Elementary School</td>\n",
       "      <td>That life is worth living.</td>\n",
       "      <td>that life is worth living</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>9553</td>\n",
       "      <td>32</td>\n",
       "      <td>213</td>\n",
       "      <td>Edna Krabappel-Flanders: The polls will be ope...</td>\n",
       "      <td>864000</td>\n",
       "      <td>True</td>\n",
       "      <td>40</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Edna Krabappel-Flanders</td>\n",
       "      <td>Springfield Elementary School</td>\n",
       "      <td>The polls will be open from now until the end ...</td>\n",
       "      <td>the polls will be open from now until the end ...</td>\n",
       "      <td>33</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     id  episode_id  number  \\\n",
       "0  9549          32     209   \n",
       "1  9550          32     210   \n",
       "2  9551          32     211   \n",
       "3  9552          32     212   \n",
       "4  9553          32     213   \n",
       "\n",
       "                                            raw_text timestamp_in_ms  \\\n",
       "0  Miss Hoover: No, actually, it was a little of ...          848000   \n",
       "1  Lisa Simpson: (NEAR TEARS) Where's Mr. Bergstrom?          856000   \n",
       "2  Miss Hoover: I don't know. Although I'd sure l...          856000   \n",
       "3           Lisa Simpson: That life is worth living.          864000   \n",
       "4  Edna Krabappel-Flanders: The polls will be ope...          864000   \n",
       "\n",
       "  speaking_line character_id  location_id       raw_character_text  \\\n",
       "0          True          464          3.0              Miss Hoover   \n",
       "1          True            9          3.0             Lisa Simpson   \n",
       "2          True          464          3.0              Miss Hoover   \n",
       "3          True            9          3.0             Lisa Simpson   \n",
       "4          True           40          3.0  Edna Krabappel-Flanders   \n",
       "\n",
       "               raw_location_text  \\\n",
       "0  Springfield Elementary School   \n",
       "1  Springfield Elementary School   \n",
       "2  Springfield Elementary School   \n",
       "3  Springfield Elementary School   \n",
       "4  Springfield Elementary School   \n",
       "\n",
       "                                        spoken_words  \\\n",
       "0  No, actually, it was a little of both. Sometim...   \n",
       "1                             Where's Mr. Bergstrom?   \n",
       "2  I don't know. Although I'd sure like to talk t...   \n",
       "3                         That life is worth living.   \n",
       "4  The polls will be open from now until the end ...   \n",
       "\n",
       "                                     normalized_text word_count  \n",
       "0  no actually it was a little of both sometimes ...         31  \n",
       "1                                wheres mr bergstrom          3  \n",
       "2  i dont know although id sure like to talk to h...         22  \n",
       "3                          that life is worth living          5  \n",
       "4  the polls will be open from now until the end ...         33  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another [notebook](pydata-tutorial-simpsons-data-cleaning.ipynb) in the repository contains a more detailed data analysis and cleaning of the `DataFrame`.\n",
    "\n",
    "For now, we'll just import a module with some utility functions and run a function to do the cleaning.\n",
    "\n",
    "If there is time - and interest - toward the end of the tutorial, we can review that notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/joem/anaconda/lib/python2.7/site-packages/pandas/core/indexing.py:517: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
      "  self.obj[item] = s\n"
     ]
    }
   ],
   "source": [
    "import pydata_simpsons\n",
    "\n",
    "df = pydata_simpsons.clean_simpsons_script_lines(df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>episode_id</th>\n",
       "      <th>number</th>\n",
       "      <th>raw_text</th>\n",
       "      <th>timestamp_in_ms</th>\n",
       "      <th>speaking_line</th>\n",
       "      <th>character_id</th>\n",
       "      <th>location_id</th>\n",
       "      <th>raw_character_text</th>\n",
       "      <th>raw_location_text</th>\n",
       "      <th>spoken_words</th>\n",
       "      <th>normalized_text</th>\n",
       "      <th>word_count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>(Street: ext. street - establishing - night)</td>\n",
       "      <td>8000</td>\n",
       "      <td>False</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td></td>\n",
       "      <td>Street</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>(Car: int. car - night)</td>\n",
       "      <td>8000</td>\n",
       "      <td>False</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td></td>\n",
       "      <td>Car</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Marge Simpson: Ooo, careful, Homer.</td>\n",
       "      <td>8000</td>\n",
       "      <td>True</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Marge Simpson</td>\n",
       "      <td>Car</td>\n",
       "      <td>Ooo, careful, Homer.</td>\n",
       "      <td>ooo careful homer</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Homer Simpson: There's no time to be careful.</td>\n",
       "      <td>10000</td>\n",
       "      <td>True</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>Homer Simpson</td>\n",
       "      <td>Car</td>\n",
       "      <td>There's no time to be careful.</td>\n",
       "      <td>theres no time to be careful</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>Homer Simpson: We're late.</td>\n",
       "      <td>10000</td>\n",
       "      <td>True</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>Homer Simpson</td>\n",
       "      <td>Car</td>\n",
       "      <td>We're late.</td>\n",
       "      <td>were late</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id  episode_id  number                                       raw_text  \\\n",
       "0   1           1       0   (Street: ext. street - establishing - night)   \n",
       "1   2           1       1                        (Car: int. car - night)   \n",
       "2   3           1       2            Marge Simpson: Ooo, careful, Homer.   \n",
       "3   4           1       3  Homer Simpson: There's no time to be careful.   \n",
       "4   5           1       4                     Homer Simpson: We're late.   \n",
       "\n",
       "   timestamp_in_ms  speaking_line  character_id  location_id  \\\n",
       "0             8000          False             0            1   \n",
       "1             8000          False             0            2   \n",
       "2             8000           True             1            2   \n",
       "3            10000           True             2            2   \n",
       "4            10000           True             2            2   \n",
       "\n",
       "  raw_character_text raw_location_text                    spoken_words  \\\n",
       "0                               Street                                   \n",
       "1                                  Car                                   \n",
       "2      Marge Simpson               Car            Ooo, careful, Homer.   \n",
       "3      Homer Simpson               Car  There's no time to be careful.   \n",
       "4      Homer Simpson               Car                     We're late.   \n",
       "\n",
       "                normalized_text  word_count  \n",
       "0                                         0  \n",
       "1                                         0  \n",
       "2             ooo careful homer           3  \n",
       "3  theres no time to be careful           6  \n",
       "4                     were late           2  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will also use data from the `simpsons_episodes.csv` file to map episodes to seasons, so we will create a separate `DataFrame` for working with the episodes data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "600"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_episodes = lds.dataframes['simpsons_episodes']\n",
    "len(df_episodes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 600 entries, 0 to 599\n",
      "Data columns (total 14 columns):\n",
      "id                        600 non-null int64\n",
      "image_url                 596 non-null object\n",
      "imdb_rating               597 non-null float64\n",
      "imdb_votes                597 non-null float64\n",
      "number_in_season          600 non-null int64\n",
      "number_in_series          600 non-null int64\n",
      "original_air_date         600 non-null datetime64[ns]\n",
      "original_air_year         600 non-null int64\n",
      "production_code           600 non-null object\n",
      "season                    600 non-null int64\n",
      "title                     600 non-null object\n",
      "us_viewers_in_millions    594 non-null float64\n",
      "video_url                 596 non-null object\n",
      "views                     596 non-null float64\n",
      "dtypes: datetime64[ns](1), float64(4), int64(5), object(4)\n",
      "memory usage: 65.7+ KB\n"
     ]
    }
   ],
   "source": [
    "df_episodes.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>image_url</th>\n",
       "      <th>imdb_rating</th>\n",
       "      <th>imdb_votes</th>\n",
       "      <th>number_in_season</th>\n",
       "      <th>number_in_series</th>\n",
       "      <th>original_air_date</th>\n",
       "      <th>original_air_year</th>\n",
       "      <th>production_code</th>\n",
       "      <th>season</th>\n",
       "      <th>title</th>\n",
       "      <th>us_viewers_in_millions</th>\n",
       "      <th>video_url</th>\n",
       "      <th>views</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>7.4</td>\n",
       "      <td>1511.0</td>\n",
       "      <td>10</td>\n",
       "      <td>10</td>\n",
       "      <td>1990-03-25</td>\n",
       "      <td>1990</td>\n",
       "      <td>7G10</td>\n",
       "      <td>1</td>\n",
       "      <td>Homer's Night Out</td>\n",
       "      <td>30.3</td>\n",
       "      <td>http://www.simpsonsworld.com/video/275197507879</td>\n",
       "      <td>50816.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>12</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>8.3</td>\n",
       "      <td>1716.0</td>\n",
       "      <td>12</td>\n",
       "      <td>12</td>\n",
       "      <td>1990-04-29</td>\n",
       "      <td>1990</td>\n",
       "      <td>7G12</td>\n",
       "      <td>1</td>\n",
       "      <td>Krusty Gets Busted</td>\n",
       "      <td>30.4</td>\n",
       "      <td>http://www.simpsonsworld.com/video/288019523914</td>\n",
       "      <td>62561.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>14</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>8.2</td>\n",
       "      <td>1638.0</td>\n",
       "      <td>1</td>\n",
       "      <td>14</td>\n",
       "      <td>1990-10-11</td>\n",
       "      <td>1990</td>\n",
       "      <td>7F03</td>\n",
       "      <td>2</td>\n",
       "      <td>Bart Gets an \"F\"</td>\n",
       "      <td>33.6</td>\n",
       "      <td>http://www.simpsonsworld.com/video/260539459671</td>\n",
       "      <td>59575.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>17</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>8.1</td>\n",
       "      <td>1457.0</td>\n",
       "      <td>4</td>\n",
       "      <td>17</td>\n",
       "      <td>1990-11-01</td>\n",
       "      <td>1990</td>\n",
       "      <td>7F01</td>\n",
       "      <td>2</td>\n",
       "      <td>Two Cars in Every Garage and Three Eyes on Eve...</td>\n",
       "      <td>26.1</td>\n",
       "      <td>http://www.simpsonsworld.com/video/260537411822</td>\n",
       "      <td>64959.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>19</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>8.0</td>\n",
       "      <td>1366.0</td>\n",
       "      <td>6</td>\n",
       "      <td>19</td>\n",
       "      <td>1990-11-15</td>\n",
       "      <td>1990</td>\n",
       "      <td>7F08</td>\n",
       "      <td>2</td>\n",
       "      <td>Dead Putting Society</td>\n",
       "      <td>25.4</td>\n",
       "      <td>http://www.simpsonsworld.com/video/260539459670</td>\n",
       "      <td>50691.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id                                          image_url  imdb_rating  \\\n",
       "0  10  http://static-media.fxx.com/img/FX_Networks_-_...          7.4   \n",
       "1  12  http://static-media.fxx.com/img/FX_Networks_-_...          8.3   \n",
       "2  14  http://static-media.fxx.com/img/FX_Networks_-_...          8.2   \n",
       "3  17  http://static-media.fxx.com/img/FX_Networks_-_...          8.1   \n",
       "4  19  http://static-media.fxx.com/img/FX_Networks_-_...          8.0   \n",
       "\n",
       "   imdb_votes  number_in_season  number_in_series original_air_date  \\\n",
       "0      1511.0                10                10        1990-03-25   \n",
       "1      1716.0                12                12        1990-04-29   \n",
       "2      1638.0                 1                14        1990-10-11   \n",
       "3      1457.0                 4                17        1990-11-01   \n",
       "4      1366.0                 6                19        1990-11-15   \n",
       "\n",
       "   original_air_year production_code  season  \\\n",
       "0               1990            7G10       1   \n",
       "1               1990            7G12       1   \n",
       "2               1990            7F03       2   \n",
       "3               1990            7F01       2   \n",
       "4               1990            7F08       2   \n",
       "\n",
       "                                               title  us_viewers_in_millions  \\\n",
       "0                                  Homer's Night Out                    30.3   \n",
       "1                                 Krusty Gets Busted                    30.4   \n",
       "2                                   Bart Gets an \"F\"                    33.6   \n",
       "3  Two Cars in Every Garage and Three Eyes on Eve...                    26.1   \n",
       "4                               Dead Putting Society                    25.4   \n",
       "\n",
       "                                         video_url    views  \n",
       "0  http://www.simpsonsworld.com/video/275197507879  50816.0  \n",
       "1  http://www.simpsonsworld.com/video/288019523914  62561.0  \n",
       "2  http://www.simpsonsworld.com/video/260539459671  59575.0  \n",
       "3  http://www.simpsonsworld.com/video/260537411822  64959.0  \n",
       "4  http://www.simpsonsworld.com/video/260539459670  50691.0  "
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_episodes.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The only field we need from the episodes data is `season`, and we'll need the `id` field to map to the `episode_id` field in the `simpsons_script_lines` data, so we'll [merge](http://pandas.pydata.org/pandas-docs/stable/merging.html) those fields from `df_episodes` with `df` (using [`pandas.DataFrame.merge()`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.merge.html)), and then drop the extraneous `id` column (`axis=1`) from `df_episodes` (using [`pandas.DataFrame.drop()`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.drop.html))."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "df = df.merge(\n",
    "    df_episodes[['id', 'season']],\n",
    "    left_on='episode_id',\n",
    "    right_on='id',\n",
    "    how='left',\n",
    "    suffixes=['', '_y']).drop('id_y', axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>episode_id</th>\n",
       "      <th>number</th>\n",
       "      <th>raw_text</th>\n",
       "      <th>timestamp_in_ms</th>\n",
       "      <th>speaking_line</th>\n",
       "      <th>character_id</th>\n",
       "      <th>location_id</th>\n",
       "      <th>raw_character_text</th>\n",
       "      <th>raw_location_text</th>\n",
       "      <th>spoken_words</th>\n",
       "      <th>normalized_text</th>\n",
       "      <th>word_count</th>\n",
       "      <th>season</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>(Street: ext. street - establishing - night)</td>\n",
       "      <td>8000</td>\n",
       "      <td>False</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td></td>\n",
       "      <td>Street</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>(Car: int. car - night)</td>\n",
       "      <td>8000</td>\n",
       "      <td>False</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td></td>\n",
       "      <td>Car</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Marge Simpson: Ooo, careful, Homer.</td>\n",
       "      <td>8000</td>\n",
       "      <td>True</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Marge Simpson</td>\n",
       "      <td>Car</td>\n",
       "      <td>Ooo, careful, Homer.</td>\n",
       "      <td>ooo careful homer</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Homer Simpson: There's no time to be careful.</td>\n",
       "      <td>10000</td>\n",
       "      <td>True</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>Homer Simpson</td>\n",
       "      <td>Car</td>\n",
       "      <td>There's no time to be careful.</td>\n",
       "      <td>theres no time to be careful</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>Homer Simpson: We're late.</td>\n",
       "      <td>10000</td>\n",
       "      <td>True</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>Homer Simpson</td>\n",
       "      <td>Car</td>\n",
       "      <td>We're late.</td>\n",
       "      <td>were late</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id  episode_id  number                                       raw_text  \\\n",
       "0   1           1       0   (Street: ext. street - establishing - night)   \n",
       "1   2           1       1                        (Car: int. car - night)   \n",
       "2   3           1       2            Marge Simpson: Ooo, careful, Homer.   \n",
       "3   4           1       3  Homer Simpson: There's no time to be careful.   \n",
       "4   5           1       4                     Homer Simpson: We're late.   \n",
       "\n",
       "   timestamp_in_ms  speaking_line  character_id  location_id  \\\n",
       "0             8000          False             0            1   \n",
       "1             8000          False             0            2   \n",
       "2             8000           True             1            2   \n",
       "3            10000           True             2            2   \n",
       "4            10000           True             2            2   \n",
       "\n",
       "  raw_character_text raw_location_text                    spoken_words  \\\n",
       "0                               Street                                   \n",
       "1                                  Car                                   \n",
       "2      Marge Simpson               Car            Ooo, careful, Homer.   \n",
       "3      Homer Simpson               Car  There's no time to be careful.   \n",
       "4      Homer Simpson               Car                     We're late.   \n",
       "\n",
       "                normalized_text  word_count  season  \n",
       "0                                         0       1  \n",
       "1                                         0       1  \n",
       "2             ooo careful homer           3       1  \n",
       "3  theres no time to be careful           6       1  \n",
       "4                     were late           2       1  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The new `season` field will be in the rightmost column of the `DataFrame`, but I would like to see it appear right after the `id` field, so will re-order the columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "df = df[[u'id', u'season', u'episode_id', u'number', u'raw_text', u'timestamp_in_ms',\n",
    "       u'speaking_line', u'character_id', u'location_id',\n",
    "       u'raw_character_text', u'raw_location_text', u'spoken_words',\n",
    "       u'normalized_text', u'word_count']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>season</th>\n",
       "      <th>episode_id</th>\n",
       "      <th>number</th>\n",
       "      <th>raw_text</th>\n",
       "      <th>timestamp_in_ms</th>\n",
       "      <th>speaking_line</th>\n",
       "      <th>character_id</th>\n",
       "      <th>location_id</th>\n",
       "      <th>raw_character_text</th>\n",
       "      <th>raw_location_text</th>\n",
       "      <th>spoken_words</th>\n",
       "      <th>normalized_text</th>\n",
       "      <th>word_count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>(Street: ext. street - establishing - night)</td>\n",
       "      <td>8000</td>\n",
       "      <td>False</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td></td>\n",
       "      <td>Street</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>(Car: int. car - night)</td>\n",
       "      <td>8000</td>\n",
       "      <td>False</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td></td>\n",
       "      <td>Car</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Marge Simpson: Ooo, careful, Homer.</td>\n",
       "      <td>8000</td>\n",
       "      <td>True</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Marge Simpson</td>\n",
       "      <td>Car</td>\n",
       "      <td>Ooo, careful, Homer.</td>\n",
       "      <td>ooo careful homer</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Homer Simpson: There's no time to be careful.</td>\n",
       "      <td>10000</td>\n",
       "      <td>True</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>Homer Simpson</td>\n",
       "      <td>Car</td>\n",
       "      <td>There's no time to be careful.</td>\n",
       "      <td>theres no time to be careful</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>Homer Simpson: We're late.</td>\n",
       "      <td>10000</td>\n",
       "      <td>True</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>Homer Simpson</td>\n",
       "      <td>Car</td>\n",
       "      <td>We're late.</td>\n",
       "      <td>were late</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id  season  episode_id  number  \\\n",
       "0   1       1           1       0   \n",
       "1   2       1           1       1   \n",
       "2   3       1           1       2   \n",
       "3   4       1           1       3   \n",
       "4   5       1           1       4   \n",
       "\n",
       "                                        raw_text  timestamp_in_ms  \\\n",
       "0   (Street: ext. street - establishing - night)             8000   \n",
       "1                        (Car: int. car - night)             8000   \n",
       "2            Marge Simpson: Ooo, careful, Homer.             8000   \n",
       "3  Homer Simpson: There's no time to be careful.            10000   \n",
       "4                     Homer Simpson: We're late.            10000   \n",
       "\n",
       "   speaking_line  character_id  location_id raw_character_text  \\\n",
       "0          False             0            1                      \n",
       "1          False             0            2                      \n",
       "2           True             1            2      Marge Simpson   \n",
       "3           True             2            2      Homer Simpson   \n",
       "4           True             2            2      Homer Simpson   \n",
       "\n",
       "  raw_location_text                    spoken_words  \\\n",
       "0            Street                                   \n",
       "1               Car                                   \n",
       "2               Car            Ooo, careful, Homer.   \n",
       "3               Car  There's no time to be careful.   \n",
       "4               Car                     We're late.   \n",
       "\n",
       "                normalized_text  word_count  \n",
       "0                                         0  \n",
       "1                                         0  \n",
       "2             ooo careful homer           3  \n",
       "3  theres no time to be careful           6  \n",
       "4                     were late           2  "
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Time Series Analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From [Wikipedia](https://en.wikipedia.org/wiki/Time_series):\n",
    "\n",
    "> A **time series** is a series of data points indexed (or listed or graphed) in time order. Most commonly, a time series is a sequence taken at successive equally spaced points in time. Thus it is a sequence of discrete-time data. Examples of time series are heights of ocean tides, counts of sunspots, and the daily closing value of the Dow Jones Industrial Average.\n",
    ">\n",
    "> **Time series *analysis*** comprises methods for analyzing time series data in order to extract meaningful statistics and other characteristics of the data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Time Series data from The Simpsons"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In [The Simpsons dataset on data.world](https://data.world/data-society/the-simpsons-by-the-data), we will use two `DataFrames`, each loaded from its corresponding CSV file.\n",
    "\n",
    "Each data point (row) in the DataFrame we loaded from **`simpsons_script_lines.csv`** represents a distinct script line, indexed by an **`id`** field that indicates the order in which the line occured.\n",
    "\n",
    "Other fields in this `DataFrame` include\n",
    "* **`episode_id`**: an identifier for each episode\n",
    "* **`number`**: an identifier for the position of each line within an episode\n",
    "* **`raw_text`**: the raw text extracted from ???\n",
    "* **`timestamp_in_ms`**: the number of milliseconds since the start of the season when the line occurred\n",
    "* **`speaking_line`**: a boolean indicator of whether the line was a speaking line\n",
    "* **`character_id`**: a unique identifier for each character\n",
    "* **`location_id`**: a unique identifier for each location\n",
    "* **`raw_character_text`**: the full text (name) of the character\n",
    "* **`raw_location_text`**: the raw text (name) of the location\n",
    "* **`spoken_words`**: the text spoken in the line (if any)\n",
    "* **`normalized_text`**: a lowercase version of the `spoken_words`, with most punctuation removed\n",
    "* **`word_count`**: the number of space-delimited tokens in `normalized_text`\n",
    "\n",
    "Each data point in the `DataFrame` loaded from **`simpsons_episodes.csv`** represents a distinct episode, which is also indexed by an **`id`** field that indicates the order in which the episode occured. This `id` field maps to the `episode_id` in `simpsons_script_lines.csv`. The only other field from this `DataFrame` we will use is **`season`**, which is a number indicating the season in which the episode originally aired."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that although there are 600 `id` values in `simpsons_episodes.csv`, there are only 564 `episode_id` values in `simpsons_script_lines.csv`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(600, 564, 36)"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_episodes.id.nunique(), df.episode_id.nunique(), len(df_episodes[~df_episodes.id.isin(df.episode_id)])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>image_url</th>\n",
       "      <th>imdb_rating</th>\n",
       "      <th>imdb_votes</th>\n",
       "      <th>number_in_season</th>\n",
       "      <th>number_in_series</th>\n",
       "      <th>original_air_date</th>\n",
       "      <th>original_air_year</th>\n",
       "      <th>production_code</th>\n",
       "      <th>season</th>\n",
       "      <th>title</th>\n",
       "      <th>us_viewers_in_millions</th>\n",
       "      <th>video_url</th>\n",
       "      <th>views</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>161</th>\n",
       "      <td>424</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>7.1</td>\n",
       "      <td>757.0</td>\n",
       "      <td>4</td>\n",
       "      <td>424</td>\n",
       "      <td>2008-11-02</td>\n",
       "      <td>2008</td>\n",
       "      <td>KABF16</td>\n",
       "      <td>20</td>\n",
       "      <td>Treehouse of Horror XIX</td>\n",
       "      <td>12.48</td>\n",
       "      <td>http://www.simpsonsworld.com/video/691895363987</td>\n",
       "      <td>21509.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>189</th>\n",
       "      <td>447</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.7</td>\n",
       "      <td>543.0</td>\n",
       "      <td>6</td>\n",
       "      <td>447</td>\n",
       "      <td>2009-11-22</td>\n",
       "      <td>2009</td>\n",
       "      <td>LABF18</td>\n",
       "      <td>21</td>\n",
       "      <td>Pranks and Greens</td>\n",
       "      <td>7.03</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>209</th>\n",
       "      <td>550</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>7.7</td>\n",
       "      <td>705.0</td>\n",
       "      <td>20</td>\n",
       "      <td>550</td>\n",
       "      <td>2014-05-04</td>\n",
       "      <td>2014</td>\n",
       "      <td>RABF21</td>\n",
       "      <td>25</td>\n",
       "      <td>Brick Like Me</td>\n",
       "      <td>4.39</td>\n",
       "      <td>http://www.simpsonsworld.com/video/311243331763</td>\n",
       "      <td>65613.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>216</th>\n",
       "      <td>570</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>6.7</td>\n",
       "      <td>334.0</td>\n",
       "      <td>18</td>\n",
       "      <td>570</td>\n",
       "      <td>2015-04-19</td>\n",
       "      <td>2015</td>\n",
       "      <td>TABF11</td>\n",
       "      <td>26</td>\n",
       "      <td>Peeping Mom</td>\n",
       "      <td>3.23</td>\n",
       "      <td>http://www.simpsonsworld.com/video/430426691868</td>\n",
       "      <td>40157.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>229</th>\n",
       "      <td>573</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>6.8</td>\n",
       "      <td>335.0</td>\n",
       "      <td>21</td>\n",
       "      <td>573</td>\n",
       "      <td>2015-05-10</td>\n",
       "      <td>2015</td>\n",
       "      <td>TABF15</td>\n",
       "      <td>26</td>\n",
       "      <td>Bull-E</td>\n",
       "      <td>2.77</td>\n",
       "      <td>http://www.simpsonsworld.com/video/442879555692</td>\n",
       "      <td>43978.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>230</th>\n",
       "      <td>576</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>6.7</td>\n",
       "      <td>395.0</td>\n",
       "      <td>2</td>\n",
       "      <td>576</td>\n",
       "      <td>2015-10-04</td>\n",
       "      <td>2015</td>\n",
       "      <td>TABF17</td>\n",
       "      <td>27</td>\n",
       "      <td>Cue Detective</td>\n",
       "      <td>6.02</td>\n",
       "      <td>http://www.simpsonsworld.com/video/770595395906</td>\n",
       "      <td>178.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>231</th>\n",
       "      <td>580</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>6.8</td>\n",
       "      <td>335.0</td>\n",
       "      <td>6</td>\n",
       "      <td>580</td>\n",
       "      <td>2015-11-08</td>\n",
       "      <td>2015</td>\n",
       "      <td>TABF21</td>\n",
       "      <td>27</td>\n",
       "      <td>Friend with Benefit</td>\n",
       "      <td>3.48</td>\n",
       "      <td>http://www.simpsonsworld.com/video/770437187884</td>\n",
       "      <td>175.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>232</th>\n",
       "      <td>583</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>8.4</td>\n",
       "      <td>698.0</td>\n",
       "      <td>9</td>\n",
       "      <td>583</td>\n",
       "      <td>2015-12-13</td>\n",
       "      <td>2015</td>\n",
       "      <td>VABF02</td>\n",
       "      <td>27</td>\n",
       "      <td>Barthood</td>\n",
       "      <td>5.97</td>\n",
       "      <td>http://www.simpsonsworld.com/video/770423875541</td>\n",
       "      <td>182.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>233</th>\n",
       "      <td>585</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>6.8</td>\n",
       "      <td>309.0</td>\n",
       "      <td>11</td>\n",
       "      <td>585</td>\n",
       "      <td>2016-01-10</td>\n",
       "      <td>2016</td>\n",
       "      <td>VABF04</td>\n",
       "      <td>27</td>\n",
       "      <td>Teenage Mutant Milk-Caused Hurdles</td>\n",
       "      <td>8.33</td>\n",
       "      <td>http://www.simpsonsworld.com/video/770430019813</td>\n",
       "      <td>172.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>234</th>\n",
       "      <td>598</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2</td>\n",
       "      <td>598</td>\n",
       "      <td>2016-10-02</td>\n",
       "      <td>2016</td>\n",
       "      <td>VABF18</td>\n",
       "      <td>28</td>\n",
       "      <td>Friends and Family\"[203]</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>235</th>\n",
       "      <td>599</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3</td>\n",
       "      <td>599</td>\n",
       "      <td>2016-10-09</td>\n",
       "      <td>2016</td>\n",
       "      <td>VABF17</td>\n",
       "      <td>28</td>\n",
       "      <td>The Town\"[205]</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>236</th>\n",
       "      <td>600</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4</td>\n",
       "      <td>600</td>\n",
       "      <td>2016-10-16</td>\n",
       "      <td>2016</td>\n",
       "      <td>VABF16</td>\n",
       "      <td>28</td>\n",
       "      <td>Treehouse of Horror XXVII\"[207]</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>237</th>\n",
       "      <td>588</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>6.6</td>\n",
       "      <td>293.0</td>\n",
       "      <td>14</td>\n",
       "      <td>588</td>\n",
       "      <td>2016-02-21</td>\n",
       "      <td>2016</td>\n",
       "      <td>VABF06</td>\n",
       "      <td>27</td>\n",
       "      <td>Gal of Constant Sorrow</td>\n",
       "      <td>3.10</td>\n",
       "      <td>http://www.simpsonsworld.com/video/770451523845</td>\n",
       "      <td>144.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>238</th>\n",
       "      <td>590</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>7.3</td>\n",
       "      <td>262.0</td>\n",
       "      <td>16</td>\n",
       "      <td>590</td>\n",
       "      <td>2016-03-13</td>\n",
       "      <td>2016</td>\n",
       "      <td>VABF09</td>\n",
       "      <td>27</td>\n",
       "      <td>The Marge-ian Chronicles</td>\n",
       "      <td>3.07</td>\n",
       "      <td>http://www.simpsonsworld.com/video/770891331927</td>\n",
       "      <td>201.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>239</th>\n",
       "      <td>593</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>7.1</td>\n",
       "      <td>229.0</td>\n",
       "      <td>19</td>\n",
       "      <td>593</td>\n",
       "      <td>2016-04-24</td>\n",
       "      <td>2016</td>\n",
       "      <td>VABF12</td>\n",
       "      <td>27</td>\n",
       "      <td>Fland Canyon</td>\n",
       "      <td>2.77</td>\n",
       "      <td>http://www.simpsonsworld.com/video/770894403718</td>\n",
       "      <td>192.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>240</th>\n",
       "      <td>595</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>6.4</td>\n",
       "      <td>227.0</td>\n",
       "      <td>21</td>\n",
       "      <td>595</td>\n",
       "      <td>2016-05-15</td>\n",
       "      <td>2016</td>\n",
       "      <td>VABF13</td>\n",
       "      <td>27</td>\n",
       "      <td>Simprovised</td>\n",
       "      <td>2.80</td>\n",
       "      <td>http://www.simpsonsworld.com/video/770892355976</td>\n",
       "      <td>228.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>501</th>\n",
       "      <td>579</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>6.8</td>\n",
       "      <td>385.0</td>\n",
       "      <td>5</td>\n",
       "      <td>579</td>\n",
       "      <td>2015-10-25</td>\n",
       "      <td>2015</td>\n",
       "      <td>TABF18</td>\n",
       "      <td>27</td>\n",
       "      <td>Treehouse of Horror XXVI</td>\n",
       "      <td>6.75</td>\n",
       "      <td>http://www.simpsonsworld.com/video/770433091920</td>\n",
       "      <td>195.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>502</th>\n",
       "      <td>584</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>6.6</td>\n",
       "      <td>356.0</td>\n",
       "      <td>10</td>\n",
       "      <td>584</td>\n",
       "      <td>2016-01-03</td>\n",
       "      <td>2016</td>\n",
       "      <td>VABF03</td>\n",
       "      <td>27</td>\n",
       "      <td>The Girl Code</td>\n",
       "      <td>4.41</td>\n",
       "      <td>http://www.simpsonsworld.com/video/770434627559</td>\n",
       "      <td>166.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>504</th>\n",
       "      <td>575</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>5.9</td>\n",
       "      <td>532.0</td>\n",
       "      <td>1</td>\n",
       "      <td>575</td>\n",
       "      <td>2015-09-27</td>\n",
       "      <td>2015</td>\n",
       "      <td>TABF14</td>\n",
       "      <td>27</td>\n",
       "      <td>Every Man's Dream</td>\n",
       "      <td>3.28</td>\n",
       "      <td>http://www.simpsonsworld.com/video/770426947744</td>\n",
       "      <td>187.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>505</th>\n",
       "      <td>572</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>6.2</td>\n",
       "      <td>330.0</td>\n",
       "      <td>20</td>\n",
       "      <td>572</td>\n",
       "      <td>2015-05-03</td>\n",
       "      <td>2015</td>\n",
       "      <td>TABF13</td>\n",
       "      <td>26</td>\n",
       "      <td>Let's Go Fly a Coot</td>\n",
       "      <td>3.12</td>\n",
       "      <td>http://www.simpsonsworld.com/video/438408259807</td>\n",
       "      <td>40783.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>506</th>\n",
       "      <td>569</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>6.7</td>\n",
       "      <td>374.0</td>\n",
       "      <td>17</td>\n",
       "      <td>569</td>\n",
       "      <td>2015-03-15</td>\n",
       "      <td>2015</td>\n",
       "      <td>TABF10</td>\n",
       "      <td>26</td>\n",
       "      <td>Waiting for Duffman</td>\n",
       "      <td>3.59</td>\n",
       "      <td>http://www.simpsonsworld.com/video/420981315819</td>\n",
       "      <td>37874.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>507</th>\n",
       "      <td>581</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>6.3</td>\n",
       "      <td>328.0</td>\n",
       "      <td>7</td>\n",
       "      <td>581</td>\n",
       "      <td>2015-11-22</td>\n",
       "      <td>2015</td>\n",
       "      <td>TABF20</td>\n",
       "      <td>27</td>\n",
       "      <td>Lisa with an 'S'</td>\n",
       "      <td>5.64</td>\n",
       "      <td>http://www.simpsonsworld.com/video/770417219938</td>\n",
       "      <td>145.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>509</th>\n",
       "      <td>587</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>6.3</td>\n",
       "      <td>252.0</td>\n",
       "      <td>13</td>\n",
       "      <td>587</td>\n",
       "      <td>2016-02-14</td>\n",
       "      <td>2016</td>\n",
       "      <td>VABF07</td>\n",
       "      <td>27</td>\n",
       "      <td>Love Is in the N2-O2-Ar-CO2-Ne-He-CH4</td>\n",
       "      <td>2.89</td>\n",
       "      <td>http://www.simpsonsworld.com/video/770439747712</td>\n",
       "      <td>158.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>510</th>\n",
       "      <td>571</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>6.9</td>\n",
       "      <td>347.0</td>\n",
       "      <td>19</td>\n",
       "      <td>571</td>\n",
       "      <td>2015-04-26</td>\n",
       "      <td>2015</td>\n",
       "      <td>TABF12</td>\n",
       "      <td>26</td>\n",
       "      <td>The Kids Are All Fight</td>\n",
       "      <td>3.33</td>\n",
       "      <td>http://www.simpsonsworld.com/video/434682947703</td>\n",
       "      <td>42123.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>514</th>\n",
       "      <td>586</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>6.7</td>\n",
       "      <td>271.0</td>\n",
       "      <td>12</td>\n",
       "      <td>586</td>\n",
       "      <td>2016-01-17</td>\n",
       "      <td>2016</td>\n",
       "      <td>VABF05</td>\n",
       "      <td>27</td>\n",
       "      <td>Much Apu About Something</td>\n",
       "      <td>3.95</td>\n",
       "      <td>http://www.simpsonsworld.com/video/770449475567</td>\n",
       "      <td>170.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>518</th>\n",
       "      <td>591</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>6.5</td>\n",
       "      <td>246.0</td>\n",
       "      <td>17</td>\n",
       "      <td>591</td>\n",
       "      <td>2016-04-03</td>\n",
       "      <td>2016</td>\n",
       "      <td>VABF10</td>\n",
       "      <td>27</td>\n",
       "      <td>The Burns Cage</td>\n",
       "      <td>2.32</td>\n",
       "      <td>http://www.simpsonsworld.com/video/770893891557</td>\n",
       "      <td>190.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>519</th>\n",
       "      <td>574</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>6.9</td>\n",
       "      <td>335.0</td>\n",
       "      <td>22</td>\n",
       "      <td>574</td>\n",
       "      <td>2015-05-17</td>\n",
       "      <td>2015</td>\n",
       "      <td>TABF16</td>\n",
       "      <td>26</td>\n",
       "      <td>Mathlete's Feat</td>\n",
       "      <td>2.82</td>\n",
       "      <td>http://www.simpsonsworld.com/video/446649411760</td>\n",
       "      <td>47429.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>520</th>\n",
       "      <td>589</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>6.3</td>\n",
       "      <td>233.0</td>\n",
       "      <td>15</td>\n",
       "      <td>589</td>\n",
       "      <td>2016-03-06</td>\n",
       "      <td>2016</td>\n",
       "      <td>VABF08</td>\n",
       "      <td>27</td>\n",
       "      <td>Lisa the Veterinarian</td>\n",
       "      <td>3.09</td>\n",
       "      <td>http://www.simpsonsworld.com/video/770452547696</td>\n",
       "      <td>174.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>521</th>\n",
       "      <td>582</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>7.1</td>\n",
       "      <td>305.0</td>\n",
       "      <td>8</td>\n",
       "      <td>582</td>\n",
       "      <td>2015-12-06</td>\n",
       "      <td>2015</td>\n",
       "      <td>VABF01</td>\n",
       "      <td>27</td>\n",
       "      <td>Paths of Glory</td>\n",
       "      <td>5.53</td>\n",
       "      <td>http://www.simpsonsworld.com/video/770422339536</td>\n",
       "      <td>170.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>522</th>\n",
       "      <td>577</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>7.2</td>\n",
       "      <td>404.0</td>\n",
       "      <td>3</td>\n",
       "      <td>577</td>\n",
       "      <td>2015-10-11</td>\n",
       "      <td>2015</td>\n",
       "      <td>TABF19</td>\n",
       "      <td>27</td>\n",
       "      <td>Puffless</td>\n",
       "      <td>3.31</td>\n",
       "      <td>http://www.simpsonsworld.com/video/772103747997</td>\n",
       "      <td>160.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>548</th>\n",
       "      <td>578</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>7.5</td>\n",
       "      <td>462.0</td>\n",
       "      <td>4</td>\n",
       "      <td>578</td>\n",
       "      <td>2015-10-18</td>\n",
       "      <td>2015</td>\n",
       "      <td>TABF22</td>\n",
       "      <td>27</td>\n",
       "      <td>Halloween of Horror</td>\n",
       "      <td>3.69</td>\n",
       "      <td>http://www.simpsonsworld.com/video/770425923947</td>\n",
       "      <td>196.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>549</th>\n",
       "      <td>594</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>6.7</td>\n",
       "      <td>193.0</td>\n",
       "      <td>20</td>\n",
       "      <td>594</td>\n",
       "      <td>2016-05-08</td>\n",
       "      <td>2016</td>\n",
       "      <td>VABF14</td>\n",
       "      <td>27</td>\n",
       "      <td>To Courier with Love</td>\n",
       "      <td>2.52</td>\n",
       "      <td>http://www.simpsonsworld.com/video/772102211921</td>\n",
       "      <td>192.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>564</th>\n",
       "      <td>441</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>7.2</td>\n",
       "      <td>648.0</td>\n",
       "      <td>21</td>\n",
       "      <td>441</td>\n",
       "      <td>2009-05-17</td>\n",
       "      <td>2009</td>\n",
       "      <td>LABF12</td>\n",
       "      <td>20</td>\n",
       "      <td>Coming to Homerica</td>\n",
       "      <td>5.86</td>\n",
       "      <td>http://www.simpsonsworld.com/video/729848899646</td>\n",
       "      <td>6477.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>574</th>\n",
       "      <td>592</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>6.4</td>\n",
       "      <td>228.0</td>\n",
       "      <td>18</td>\n",
       "      <td>592</td>\n",
       "      <td>2016-04-10</td>\n",
       "      <td>2016</td>\n",
       "      <td>VABF11</td>\n",
       "      <td>27</td>\n",
       "      <td>How Lisa Got Her Marge Back</td>\n",
       "      <td>2.55</td>\n",
       "      <td>http://www.simpsonsworld.com/video/770893891750</td>\n",
       "      <td>188.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>575</th>\n",
       "      <td>596</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>6.9</td>\n",
       "      <td>210.0</td>\n",
       "      <td>22</td>\n",
       "      <td>596</td>\n",
       "      <td>2016-05-22</td>\n",
       "      <td>2016</td>\n",
       "      <td>VABF15</td>\n",
       "      <td>27</td>\n",
       "      <td>Orange Is the New Yellow</td>\n",
       "      <td>2.54</td>\n",
       "      <td>http://www.simpsonsworld.com/video/772111939897</td>\n",
       "      <td>276.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>576</th>\n",
       "      <td>597</td>\n",
       "      <td>http://static-media.fxx.com/img/FX_Networks_-_...</td>\n",
       "      <td>6.6</td>\n",
       "      <td>104.0</td>\n",
       "      <td>1</td>\n",
       "      <td>597</td>\n",
       "      <td>2016-09-25</td>\n",
       "      <td>2016</td>\n",
       "      <td>VABF20</td>\n",
       "      <td>28</td>\n",
       "      <td>Monty Burns' Fleeing Circus</td>\n",
       "      <td>3.36</td>\n",
       "      <td>http://www.simpsonsworld.com/video/772654659902</td>\n",
       "      <td>994.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      id                                          image_url  imdb_rating  \\\n",
       "161  424  http://static-media.fxx.com/img/FX_Networks_-_...          7.1   \n",
       "189  447                                                NaN          6.7   \n",
       "209  550  http://static-media.fxx.com/img/FX_Networks_-_...          7.7   \n",
       "216  570  http://static-media.fxx.com/img/FX_Networks_-_...          6.7   \n",
       "229  573  http://static-media.fxx.com/img/FX_Networks_-_...          6.8   \n",
       "230  576  http://static-media.fxx.com/img/FX_Networks_-_...          6.7   \n",
       "231  580  http://static-media.fxx.com/img/FX_Networks_-_...          6.8   \n",
       "232  583  http://static-media.fxx.com/img/FX_Networks_-_...          8.4   \n",
       "233  585  http://static-media.fxx.com/img/FX_Networks_-_...          6.8   \n",
       "234  598                                                NaN          NaN   \n",
       "235  599                                                NaN          NaN   \n",
       "236  600                                                NaN          NaN   \n",
       "237  588  http://static-media.fxx.com/img/FX_Networks_-_...          6.6   \n",
       "238  590  http://static-media.fxx.com/img/FX_Networks_-_...          7.3   \n",
       "239  593  http://static-media.fxx.com/img/FX_Networks_-_...          7.1   \n",
       "240  595  http://static-media.fxx.com/img/FX_Networks_-_...          6.4   \n",
       "501  579  http://static-media.fxx.com/img/FX_Networks_-_...          6.8   \n",
       "502  584  http://static-media.fxx.com/img/FX_Networks_-_...          6.6   \n",
       "504  575  http://static-media.fxx.com/img/FX_Networks_-_...          5.9   \n",
       "505  572  http://static-media.fxx.com/img/FX_Networks_-_...          6.2   \n",
       "506  569  http://static-media.fxx.com/img/FX_Networks_-_...          6.7   \n",
       "507  581  http://static-media.fxx.com/img/FX_Networks_-_...          6.3   \n",
       "509  587  http://static-media.fxx.com/img/FX_Networks_-_...          6.3   \n",
       "510  571  http://static-media.fxx.com/img/FX_Networks_-_...          6.9   \n",
       "514  586  http://static-media.fxx.com/img/FX_Networks_-_...          6.7   \n",
       "518  591  http://static-media.fxx.com/img/FX_Networks_-_...          6.5   \n",
       "519  574  http://static-media.fxx.com/img/FX_Networks_-_...          6.9   \n",
       "520  589  http://static-media.fxx.com/img/FX_Networks_-_...          6.3   \n",
       "521  582  http://static-media.fxx.com/img/FX_Networks_-_...          7.1   \n",
       "522  577  http://static-media.fxx.com/img/FX_Networks_-_...          7.2   \n",
       "548  578  http://static-media.fxx.com/img/FX_Networks_-_...          7.5   \n",
       "549  594  http://static-media.fxx.com/img/FX_Networks_-_...          6.7   \n",
       "564  441  http://static-media.fxx.com/img/FX_Networks_-_...          7.2   \n",
       "574  592  http://static-media.fxx.com/img/FX_Networks_-_...          6.4   \n",
       "575  596  http://static-media.fxx.com/img/FX_Networks_-_...          6.9   \n",
       "576  597  http://static-media.fxx.com/img/FX_Networks_-_...          6.6   \n",
       "\n",
       "     imdb_votes  number_in_season  number_in_series original_air_date  \\\n",
       "161       757.0                 4               424        2008-11-02   \n",
       "189       543.0                 6               447        2009-11-22   \n",
       "209       705.0                20               550        2014-05-04   \n",
       "216       334.0                18               570        2015-04-19   \n",
       "229       335.0                21               573        2015-05-10   \n",
       "230       395.0                 2               576        2015-10-04   \n",
       "231       335.0                 6               580        2015-11-08   \n",
       "232       698.0                 9               583        2015-12-13   \n",
       "233       309.0                11               585        2016-01-10   \n",
       "234         NaN                 2               598        2016-10-02   \n",
       "235         NaN                 3               599        2016-10-09   \n",
       "236         NaN                 4               600        2016-10-16   \n",
       "237       293.0                14               588        2016-02-21   \n",
       "238       262.0                16               590        2016-03-13   \n",
       "239       229.0                19               593        2016-04-24   \n",
       "240       227.0                21               595        2016-05-15   \n",
       "501       385.0                 5               579        2015-10-25   \n",
       "502       356.0                10               584        2016-01-03   \n",
       "504       532.0                 1               575        2015-09-27   \n",
       "505       330.0                20               572        2015-05-03   \n",
       "506       374.0                17               569        2015-03-15   \n",
       "507       328.0                 7               581        2015-11-22   \n",
       "509       252.0                13               587        2016-02-14   \n",
       "510       347.0                19               571        2015-04-26   \n",
       "514       271.0                12               586        2016-01-17   \n",
       "518       246.0                17               591        2016-04-03   \n",
       "519       335.0                22               574        2015-05-17   \n",
       "520       233.0                15               589        2016-03-06   \n",
       "521       305.0                 8               582        2015-12-06   \n",
       "522       404.0                 3               577        2015-10-11   \n",
       "548       462.0                 4               578        2015-10-18   \n",
       "549       193.0                20               594        2016-05-08   \n",
       "564       648.0                21               441        2009-05-17   \n",
       "574       228.0                18               592        2016-04-10   \n",
       "575       210.0                22               596        2016-05-22   \n",
       "576       104.0                 1               597        2016-09-25   \n",
       "\n",
       "     original_air_year production_code  season  \\\n",
       "161               2008          KABF16      20   \n",
       "189               2009          LABF18      21   \n",
       "209               2014          RABF21      25   \n",
       "216               2015          TABF11      26   \n",
       "229               2015          TABF15      26   \n",
       "230               2015          TABF17      27   \n",
       "231               2015          TABF21      27   \n",
       "232               2015          VABF02      27   \n",
       "233               2016          VABF04      27   \n",
       "234               2016          VABF18      28   \n",
       "235               2016          VABF17      28   \n",
       "236               2016          VABF16      28   \n",
       "237               2016          VABF06      27   \n",
       "238               2016          VABF09      27   \n",
       "239               2016          VABF12      27   \n",
       "240               2016          VABF13      27   \n",
       "501               2015          TABF18      27   \n",
       "502               2016          VABF03      27   \n",
       "504               2015          TABF14      27   \n",
       "505               2015          TABF13      26   \n",
       "506               2015          TABF10      26   \n",
       "507               2015          TABF20      27   \n",
       "509               2016          VABF07      27   \n",
       "510               2015          TABF12      26   \n",
       "514               2016          VABF05      27   \n",
       "518               2016          VABF10      27   \n",
       "519               2015          TABF16      26   \n",
       "520               2016          VABF08      27   \n",
       "521               2015          VABF01      27   \n",
       "522               2015          TABF19      27   \n",
       "548               2015          TABF22      27   \n",
       "549               2016          VABF14      27   \n",
       "564               2009          LABF12      20   \n",
       "574               2016          VABF11      27   \n",
       "575               2016          VABF15      27   \n",
       "576               2016          VABF20      28   \n",
       "\n",
       "                                     title  us_viewers_in_millions  \\\n",
       "161                Treehouse of Horror XIX                   12.48   \n",
       "189                      Pranks and Greens                    7.03   \n",
       "209                          Brick Like Me                    4.39   \n",
       "216                            Peeping Mom                    3.23   \n",
       "229                                 Bull-E                    2.77   \n",
       "230                          Cue Detective                    6.02   \n",
       "231                    Friend with Benefit                    3.48   \n",
       "232                               Barthood                    5.97   \n",
       "233     Teenage Mutant Milk-Caused Hurdles                    8.33   \n",
       "234               Friends and Family\"[203]                     NaN   \n",
       "235                         The Town\"[205]                     NaN   \n",
       "236        Treehouse of Horror XXVII\"[207]                     NaN   \n",
       "237                 Gal of Constant Sorrow                    3.10   \n",
       "238               The Marge-ian Chronicles                    3.07   \n",
       "239                           Fland Canyon                    2.77   \n",
       "240                            Simprovised                    2.80   \n",
       "501               Treehouse of Horror XXVI                    6.75   \n",
       "502                          The Girl Code                    4.41   \n",
       "504                      Every Man's Dream                    3.28   \n",
       "505                    Let's Go Fly a Coot                    3.12   \n",
       "506                    Waiting for Duffman                    3.59   \n",
       "507                       Lisa with an 'S'                    5.64   \n",
       "509  Love Is in the N2-O2-Ar-CO2-Ne-He-CH4                    2.89   \n",
       "510                 The Kids Are All Fight                    3.33   \n",
       "514               Much Apu About Something                    3.95   \n",
       "518                         The Burns Cage                    2.32   \n",
       "519                        Mathlete's Feat                    2.82   \n",
       "520                  Lisa the Veterinarian                    3.09   \n",
       "521                         Paths of Glory                    5.53   \n",
       "522                               Puffless                    3.31   \n",
       "548                    Halloween of Horror                    3.69   \n",
       "549                   To Courier with Love                    2.52   \n",
       "564                     Coming to Homerica                    5.86   \n",
       "574            How Lisa Got Her Marge Back                    2.55   \n",
       "575               Orange Is the New Yellow                    2.54   \n",
       "576            Monty Burns' Fleeing Circus                    3.36   \n",
       "\n",
       "                                           video_url    views  \n",
       "161  http://www.simpsonsworld.com/video/691895363987  21509.0  \n",
       "189                                              NaN      NaN  \n",
       "209  http://www.simpsonsworld.com/video/311243331763  65613.0  \n",
       "216  http://www.simpsonsworld.com/video/430426691868  40157.0  \n",
       "229  http://www.simpsonsworld.com/video/442879555692  43978.0  \n",
       "230  http://www.simpsonsworld.com/video/770595395906    178.0  \n",
       "231  http://www.simpsonsworld.com/video/770437187884    175.0  \n",
       "232  http://www.simpsonsworld.com/video/770423875541    182.0  \n",
       "233  http://www.simpsonsworld.com/video/770430019813    172.0  \n",
       "234                                              NaN      NaN  \n",
       "235                                              NaN      NaN  \n",
       "236                                              NaN      NaN  \n",
       "237  http://www.simpsonsworld.com/video/770451523845    144.0  \n",
       "238  http://www.simpsonsworld.com/video/770891331927    201.0  \n",
       "239  http://www.simpsonsworld.com/video/770894403718    192.0  \n",
       "240  http://www.simpsonsworld.com/video/770892355976    228.0  \n",
       "501  http://www.simpsonsworld.com/video/770433091920    195.0  \n",
       "502  http://www.simpsonsworld.com/video/770434627559    166.0  \n",
       "504  http://www.simpsonsworld.com/video/770426947744    187.0  \n",
       "505  http://www.simpsonsworld.com/video/438408259807  40783.0  \n",
       "506  http://www.simpsonsworld.com/video/420981315819  37874.0  \n",
       "507  http://www.simpsonsworld.com/video/770417219938    145.0  \n",
       "509  http://www.simpsonsworld.com/video/770439747712    158.0  \n",
       "510  http://www.simpsonsworld.com/video/434682947703  42123.0  \n",
       "514  http://www.simpsonsworld.com/video/770449475567    170.0  \n",
       "518  http://www.simpsonsworld.com/video/770893891557    190.0  \n",
       "519  http://www.simpsonsworld.com/video/446649411760  47429.0  \n",
       "520  http://www.simpsonsworld.com/video/770452547696    174.0  \n",
       "521  http://www.simpsonsworld.com/video/770422339536    170.0  \n",
       "522  http://www.simpsonsworld.com/video/772103747997    160.0  \n",
       "548  http://www.simpsonsworld.com/video/770425923947    196.0  \n",
       "549  http://www.simpsonsworld.com/video/772102211921    192.0  \n",
       "564  http://www.simpsonsworld.com/video/729848899646   6477.0  \n",
       "574  http://www.simpsonsworld.com/video/770893891750    188.0  \n",
       "575  http://www.simpsonsworld.com/video/772111939897    276.0  \n",
       "576  http://www.simpsonsworld.com/video/772654659902    994.0  "
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_episodes[~df_episodes.id.isin(df.episode_id)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise Set A"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have our base datasets, we can do some time series analyses on the data. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise A1: What are the minimum, maximum and mean number of script lines per episode?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You may find it helpful to use the following `pandas` functions:\n",
    "* [`pandas.DataFrame.groupby()`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.groupby.html)\n",
    "* [`pandas.Series.nunique()`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.nunique.html)\n",
    "* [`pandas.Series.min()`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.min.html), [`pandas.Series.max()`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.max.html), [`pandas.Series.mean()`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.mean.html) .. and/or [`pandas.core.groupby.DataFrameGroupBy.agg()`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.core.groupby.DataFrameGroupBy.agg.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<pandas.core.groupby.DataFrameGroupBy object at 0x10ce82710>"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby('episode_id')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "episode_id\n",
       "1    377\n",
       "2    294\n",
       "3    250\n",
       "4    341\n",
       "5    323\n",
       "dtype: int64"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby('episode_id').size().head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>season</th>\n",
       "      <th>number</th>\n",
       "      <th>raw_text</th>\n",
       "      <th>timestamp_in_ms</th>\n",
       "      <th>speaking_line</th>\n",
       "      <th>character_id</th>\n",
       "      <th>location_id</th>\n",
       "      <th>raw_character_text</th>\n",
       "      <th>raw_location_text</th>\n",
       "      <th>spoken_words</th>\n",
       "      <th>normalized_text</th>\n",
       "      <th>word_count</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>episode_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>377</td>\n",
       "      <td>377</td>\n",
       "      <td>377</td>\n",
       "      <td>377</td>\n",
       "      <td>377</td>\n",
       "      <td>377</td>\n",
       "      <td>377</td>\n",
       "      <td>377</td>\n",
       "      <td>377</td>\n",
       "      <td>377</td>\n",
       "      <td>377</td>\n",
       "      <td>377</td>\n",
       "      <td>377</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>294</td>\n",
       "      <td>294</td>\n",
       "      <td>294</td>\n",
       "      <td>294</td>\n",
       "      <td>294</td>\n",
       "      <td>294</td>\n",
       "      <td>294</td>\n",
       "      <td>294</td>\n",
       "      <td>294</td>\n",
       "      <td>294</td>\n",
       "      <td>294</td>\n",
       "      <td>294</td>\n",
       "      <td>294</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>250</td>\n",
       "      <td>250</td>\n",
       "      <td>250</td>\n",
       "      <td>250</td>\n",
       "      <td>250</td>\n",
       "      <td>250</td>\n",
       "      <td>250</td>\n",
       "      <td>250</td>\n",
       "      <td>250</td>\n",
       "      <td>250</td>\n",
       "      <td>250</td>\n",
       "      <td>250</td>\n",
       "      <td>250</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>341</td>\n",
       "      <td>341</td>\n",
       "      <td>341</td>\n",
       "      <td>341</td>\n",
       "      <td>341</td>\n",
       "      <td>341</td>\n",
       "      <td>341</td>\n",
       "      <td>341</td>\n",
       "      <td>341</td>\n",
       "      <td>341</td>\n",
       "      <td>341</td>\n",
       "      <td>341</td>\n",
       "      <td>341</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>323</td>\n",
       "      <td>323</td>\n",
       "      <td>323</td>\n",
       "      <td>323</td>\n",
       "      <td>323</td>\n",
       "      <td>323</td>\n",
       "      <td>323</td>\n",
       "      <td>323</td>\n",
       "      <td>323</td>\n",
       "      <td>323</td>\n",
       "      <td>323</td>\n",
       "      <td>323</td>\n",
       "      <td>323</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             id  season  number  raw_text  timestamp_in_ms  speaking_line  \\\n",
       "episode_id                                                                  \n",
       "1           377     377     377       377              377            377   \n",
       "2           294     294     294       294              294            294   \n",
       "3           250     250     250       250              250            250   \n",
       "4           341     341     341       341              341            341   \n",
       "5           323     323     323       323              323            323   \n",
       "\n",
       "            character_id  location_id  raw_character_text  raw_location_text  \\\n",
       "episode_id                                                                     \n",
       "1                    377          377                 377                377   \n",
       "2                    294          294                 294                294   \n",
       "3                    250          250                 250                250   \n",
       "4                    341          341                 341                341   \n",
       "5                    323          323                 323                323   \n",
       "\n",
       "            spoken_words  normalized_text  word_count  \n",
       "episode_id                                             \n",
       "1                    377              377         377  \n",
       "2                    294              294         294  \n",
       "3                    250              250         250  \n",
       "4                    341              341         341  \n",
       "5                    323              323         323  "
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby('episode_id').count().head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "episode_id\n",
       "1    377\n",
       "2    294\n",
       "3    250\n",
       "4    341\n",
       "5    323\n",
       "Name: id, dtype: int64"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby('episode_id')['id'].nunique().head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "episode_id\n",
       "1    377\n",
       "2    294\n",
       "3    250\n",
       "4    341\n",
       "5    323\n",
       "Name: id, dtype: int64"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby('episode_id').id.nunique().head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "episode_id\n",
       "1    377\n",
       "2    294\n",
       "3    250\n",
       "4    341\n",
       "5    323\n",
       "Name: number, dtype: int64"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(df.groupby('episode_id')['number'].max() + 1).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we want to restrict this to only speaking lines, we can constrain the rows that we look at."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "episode_id\n",
       "1    333\n",
       "2    271\n",
       "3    207\n",
       "4    304\n",
       "5    283\n",
       "Name: id, dtype: int64"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[df.speaking_line].groupby('episode_id')['id'].nunique().head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "79"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby('episode_id')['id'].nunique().min()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "395"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby('episode_id')['id'].nunique().max()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "280.57978723404256"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby('episode_id')['id'].nunique().mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "min      79.000000\n",
       "mean    280.579787\n",
       "max     395.000000\n",
       "Name: id, dtype: float64"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby('episode_id')['id'].nunique().agg(['min', 'mean', 'max'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise A2: Show (plot) the distribution of the number of script lines per episode."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You may find it helpful to use the [`pandas.DataFrame.hist()`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.hist.html) function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x11a29c450>"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEACAYAAABfxaZOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFvpJREFUeJzt3X+MZXV9xvH3s2wgIgmzVtlpdpHBQPkVzWDCErM2jlUR\n24YlmlC0bRipTROkEkladukf2x8miImUViWpYp3FQpGaFpbE4kLgpNHyw1S2rO5KtzG74pYdkB8q\nMTGsfPrHPcNchhnm7Pneud/7nfO8kpu958y9c5757tnP3HnunbuKCMzMrBvW5A5gZmbD46FvZtYh\nHvpmZh3ioW9m1iEe+mZmHeKhb2bWIcsOfUnHSXpY0qOS9kjaXu9fJ2mXpMclfVPSiX332SZpv6R9\nki5YyS/AzMyaU5PX6Us6PiJ+IekY4NvAJ4APAc9ExGckXQOsi4itks4GbgXOAzYC9wGnh38hwMws\nu0b1TkT8or56HLAWCGALsKPevwO4uL5+EXB7RByJiAPAfmDToAKbmVl7jYa+pDWSHgUOA/dGxHeA\n9RExCxARh4GT6ptvAJ7ou/uhep+ZmWXW9JH+SxFxLr26ZpOkc+g92n/FzQYdzszMBmvt0dw4In4m\nqQIuBGYlrY+IWUnjwFP1zQ4BJ/fdbWO97xUk+ZuEmVkLEaG2923y6p03zr0yR9LrgPcB+4CdwHR9\ns8uAu+rrO4FLJR0r6VTgNOCRJYIXe9m+fXv2DM6fP0cX85ecfTXkT9Xkkf6vAzskraH3TeJrEfEN\nSQ8Bd0i6HDgIXFIP8r2S7gD2Ai8CV8Qgko6YAwcO5I6QxPnzKjl/ydmh/Pyplh36EbEHePsi+58F\n3rvEfa4DrktOZ2ZmA+XfyG1peno6d4Qkzp9XyflLzg7l50/V6JezVuTA0mpsfczMVpQkYiWfyLXF\nVVWVO0IS58+r5PwlZ4fy86fy0Dcz6xDXO2ZmBXG9Y2ZmjXnot1R6L+j8eZWcv+TsUH7+VB76ZmYd\n4k7fzKwg7vTNzKwxD/2WSu8FnT+vkvOXnB3Kz5/KQ9/MrEPc6ZuZFcSdvpmZNeah31LpvaDz51Vy\n/pKzQ/n5U3nom5l1iDt9M7OCuNM3M7PGPPRbKr0XdP68Ss5fcnYoP38qD30zsw5xp29mVhB3+mZm\n1piHfkul94LOn1fJ+UvODuXnT7U2dwAza2d8fILZ2YNZjr1+/SkcPnwgy7EtjTt9s0JJAnL9GxL+\n95uHO30zM2vMQ7+l0ntB58+r7PxV7gBJyl77dB76ZmYdsmynL2kjcAuwHngJ+GJEfE7SduCPgafq\nm14bEffU99kGXA4cAa6KiF2LfF53+mYJ3Ol3U2qn32TojwPjEbFb0gnAfwFbgN8Dfh4RNyy4/VnA\nbcB5wEbgPuD0hRPeQ98sjYd+N634E7kRcTgidtfXXwD2ARvmjr/IXbYAt0fEkYg4AOwHNrUNOKpK\n7wWdP6+y81e5AyQpe+3THVWnL2kCmAQernddKWm3pJslnVjv2wA80Xe3Q8x/kzAzs4wav06/rnYq\n4G8i4i5JbwJ+EhEh6VP0KqCPSfoc8GBE3Fbf72bgGxHxrws+n+sdswSud7optd5p9Bu5ktYCXwe+\nGhF3AUTE0303+RJwd339EHBy38c21vteZXp6momJCQDGxsaYnJxkamoKmP8RzNve9vbS2/PmtqeG\ntN3LkPvr78J2VVXMzMwAvDwvUzR6pC/pFnqP6q/u2zceEYfr658EzouIj0g6G7gVOJ9erXMvq/CJ\n3P4TvkTOn9cg8ud7pF8B7y72kX7p586KP9KXtBn4fWCPpEfpnWXXAh+RNEnvZZwHgD8BiIi9ku4A\n9gIvAlcUPd3NzFYRv/eOWaHc6XeT33vHzMwa89Bv6dVPpJXF+fMqO3+VO0CSstc+nYe+mVmHuNM3\nK5Q7/W5yp29mZo156LdUei/o/HmVnb/KHSBJ2WufzkPfzKxD3OmbFcqdfje50zczs8Y89FsqvRd0\n/rzKzl/lDpCk7LVP56FvZtYh7vTNCuVOv5vc6ZuZWWMe+i2V3gs6f15l569yB0hS9tqn89A3M+sQ\nd/pmhXKn303u9M3MrDEP/ZZK7wWdP6+y81e5AyQpe+3TeeibmXWIO32zQrnT7yZ3+mZm1piHfkul\n94LOn1fZ+avcAZKUvfbpPPTNzDrEnb5Zodzpd5M7fTMza8xDv6XSe0Hnz6vs/FXuAEnKXvt0Hvpm\nZh3iTt+sUO70u8mdvpmZNbbs0Je0UdL9kr4vaY+kT9T710naJelxSd+UdGLffbZJ2i9pn6QLVvIL\nyKX0XtD58yo7f5U7QJKy1z5dk0f6R4CrI+Ic4B3AxyWdCWwF7ouIM4D7gW0Aks4GLgHOAj4A3KTe\nz6FmZpbZUXf6ku4EPl9f3hURs5LGgSoizpS0FYiIuL6+/b8DfxkRDy/4PO70zRK40++moXb6kiaA\nSeAhYH1EzAJExGHgpPpmG4An+u52qN5nZmaZrW16Q0knAF8HroqIFyQt/DZ/1N/2p6enmZiYAGBs\nbIzJyUmmpqaA+d5tVLdvvPHGovI6/2htDyr/vLntqSFsVy9nGJX1PJrt/rUbhTxN8s7MzAC8PC+T\nRMSyF3rfHO6hN/Dn9u2j92gfYBzYV1/fClzTd7t7gPMX+ZxRsgceeCB3hCTOn9cg8gMBkeHyQJT8\n77f0c6de+0aze7FLo05f0i3ATyLi6r591wPPRsT1kq4B1kXE1vqJ3FuB8+nVOvcCp8eCA7nTN0vj\nTr+bUjv9ZYe+pM3AfwB76J1hAVwLPALcAZwMHAQuiYjn6/tsA/4IeJHeTwe7Fvm8HvpmCTz0u2nF\nn8iNiG9HxDERMRkR50bE2yPinoh4NiLeGxFnRMQFcwO/vs91EXFaRJy12MBfDV7dqZbF+fMqO3+V\nO0CSstc+nX8j18ysQ/zeO2aFcr3TTX7vHTMza8xDv6XSe0Hnz6vs/FXuAEnKXvt0HvpmZh3iTt+s\nUO70u8mdvpmZNeah31LpvaDz51V2/ip3gCRlr306D30zsw5xp29WKHf63eRO38zMGvPQb6n0XtD5\n8yo7f5U7QJKy1z6dh76ZWYe40zcrlDv9bnKnb2ZmjXnot1R6L+j8eZWdv8odIEnZa5/OQ9/MrEPc\n6ZsVyp1+N7nTNzOzxjz0Wyq9F3T+vMrOX+UOkKTstU/noW9m1iHu9M0K5U6/m9zpm5lZYx76LZXe\nCzp/XmXnr3IHSFL22qfz0Dcz6xB3+maFcqffTe70zcysMQ/9lkrvBZ0/r7LzV7kDJCl77dMtO/Ql\nfVnSrKTH+vZtl/RjSd+tLxf2fWybpP2S9km6YKWCm5nZ0Vu205f0TuAF4JaIeFu9bzvw84i4YcFt\nzwJuA84DNgL3AacvVt6707fVYnx8gtnZg5mO7k6/a1a804+IbwHPLXbsRfZtAW6PiCMRcQDYD2xq\nG86sBL2BHxkuZkcvpdO/UtJuSTdLOrHetwF4ou82h+p9q07pvaDz51blDpCgyh0gSfnnTpq1Le93\nE/DXERGSPgV8FvjY0X6S6elpJiYmABgbG2NycpKpqSlg/i9mVLd37949UnmcP2+e+UHYdHv3Ud5+\nqW2W+fhKbffWYFTWfzVvV1XFzMwMwMvzMkWj1+lLOgW4e67TX+pjkrYCERHX1x+7B9geEQ8vcj93\n+rYq5Hu9vF+n30XDep2+6OvwJY33feyDwPfq6zuBSyUdK+lU4DTgkbbhzMxssJq8ZPM24D+B35D0\nI0kfBT4j6TFJu4F3AZ8EiIi9wB3AXuAbwBWr9eH83I9fpXL+3KrcARJUuQMkKf/cSbNspx8RH1lk\n91de4/bXAdelhDIzs5Xh994xS+RO34bJ771jZmaNeei3VHov6Py5VbkDJKhyB0hS/rmTxkPfzKxD\n3OmbJXKnb8PkTt/MzBrz0G+p9F7Q+XOrcgdIUOUOkKT8cyeNh76ZWYe40zdL5E7fhsmdvpmZNeah\n31LpvaDz51blDpCgyh0gSfnnThoPfTOzDnGnb5bInb4Nkzt9MzNrzEO/pdJ7QefPrcodIEGVO0CS\n8s+dNB76ZmYd4k7fLJE7fRsmd/pmZtaYh35LpfeCzp9blTtAgip3gCTlnztpPPTNzDrEnb5ZInf6\nNkzu9M3MrDEP/ZZK7wWdP7cqd4AEVe4ASco/d9J46JuZdYg7fbNE7vRtmNzpm5lZYx76LZXeCzp/\nblXuAAmq3AGSlH/upPHQNzPrkGU7fUlfBn4XmI2It9X71gFfA04BDgCXRMRP649tAy4HjgBXRcSu\nJT6vO31bFdzp2zANo9P/CvD+Bfu2AvdFxBnA/cC2OszZwCXAWcAHgJvU+xdhZmYjYNmhHxHfAp5b\nsHsLsKO+vgO4uL5+EXB7RByJiAPAfmDTYKKOltJ7QefPrcodIEGVO0CS8s+dNG07/ZMiYhYgIg4D\nJ9X7NwBP9N3uUL3PzMxGwNoBfZ5W5d709DQTExMAjI2NMTk5ydTUFDD/3XhUt+f2jUoe58+dp6r/\nbLo9t6/t/ee2WebjK7Hd2zdK638021NTUyOVZ7ntqqqYmZkBeHlepmj0y1mSTgHu7nsidx8wFRGz\nksaBByLiLElbgYiI6+vb3QNsj4iHF/mcfiLXVgU/kWvDNKxfzlJ9mbMTmK6vXwbc1bf/UknHSjoV\nOA14pG24UTb3nbhUzp9blTtAgip3gCTlnztplq13JN1G7+e5X5P0I2A78GngXyRdDhyk94odImKv\npDuAvcCLwBV+OG9mNjr83jtmiVzv2DD5vXfMzKwxD/2WSu8FnT+3KneABFXuAEnKP3fSeOibmXWI\nO32zRO70bZjc6ZuZWWMe+i2V3gs6f25V7gAJKuA4JA39Mj4+kZ6++HMnzaDehsHMOuWX5KiWZmf9\npr2p3OmbJepqp5/ra+763HCnb2ZmjXnot1R6L+j8uVW5AySocgdIUv65k8ZD38ysQ9zpmyVypz/c\n43Z9brjTNzOzxjz0Wyq9F3T+3KrcARJUuQMkKf/cSeOhb2bWIe70zRK50x/ucbs+N9zpm5lZYx76\nLZXeCzp/blXuAAmq3AGSlH/upPHQNzPrEHf6Zonc6Q/3uF2fG+70zcysMQ/9lkrvBZ0/typ3gARV\n7gBJyj930njom5l1iDt9s0Tu9Id73K7PDXf6ZmbWmId+S6X3gs6fW5U7QIIqd4Ak5Z87aTz0zcw6\nxJ2+WSJ3+sM9btfnRmqnvzbx4AeAnwIvAS9GxCZJ64CvAacAB4BLIuKnKccxM7PBSK13XgKmIuLc\niNhU79sK3BcRZwD3A9sSjzGSSu8FnT+3KneABFXuAEnKP3fSpA59LfI5tgA76us7gIsTj2FmZgOS\n1OlL+iHwPPAr4B8i4mZJz0XEur7bPBsRb1jkvu70bVVwpz/c43Z9bmTt9IHNEfGkpDcBuyQ9zqvP\nhG7/DZmZjZCkoR8RT9Z/Pi3pTmATMCtpfUTMShoHnlrq/tPT00xMTAAwNjbG5OQkU1NTwHzvNqrb\nN954Y1F5nX9lt+d77qbbNwKTR3H7pbZZ5uMrsV317RvG8fq3662Ev6/+Tn9Uzp/l8s7MzAC8PC9T\ntK53JB0PrImIFyS9HtgF/BXwHuDZiLhe0jXAuojYusj9i653qqrq+wdfHucfnHb1TsX8QGt95BbH\nHYQKeHemY6fXO6N07rSRWu+kDP1TgX+j9ze/Frg1Ij4t6Q3AHcDJwEF6L9l8fpH7Fz30zea40x/u\ncbs+N7IN/VQe+rZaeOgP97hdnxt+w7VMSn+tr/PnVuUOkKDKHSBJ+edOGg99M7MOcb1jlsj1znCP\n2/W54XrHzMwa89BvqfRe0Plzq3IHSFDlDpCk/HMnjYe+mVmHuNM3S+ROf7jH7frccKdvZmaNeei3\nVHov6Py5VbkDJKhyB0hS/rmTxkPfzKxD3OmbJXKnP9zjdn1uuNM3M7PGPPRbKr0XdP7cqtwBElS5\nAyQp/9xJk/o/Z5mZDdFxdZ02XOvXn8LhwweGftyV4E7fVo3x8QlmZw9mOro7/dV+3FGZV34/fbNa\n955Q9dAf5nFHZV75idxMSu8FnT+3KneABFXuAImq3AGy8tA3M+sQ1zu2arje6cKxXe+43jEzs8Y8\n9FsqvVN2/tyq3AESVLkDJKpyB8jKQ9/MrEPc6duq4U6/C8d2p+9O38zMGvPQb6n0Ttn5c6tyB0hQ\n5Q6QqModICsPfTOzDnGnb6uGO/0uHNudvjt9MzNrbMWGvqQLJf1A0v9IumaljpNL6Z2y8+dW5Q6Q\noModIFGVO0BWKzL0Ja0BPg+8HzgH+LCkM1fiWLns3r07d4Qkzp9byflLzg7t8vfex3/Yl/HxiUF/\n8Sv2n6hsAvZHxEEASbcDW4AfrNDxhu7555/PHSGJ8+dWcv6Ss0O7/L8kx3MJs7OD/w9jVmrobwCe\n6Nv+Mb1vBFk888wz7NixY6Cf88EHH+SGG25Y9nbnn38+mzdvHuixzcxai4iBX4APAV/s2/4D4O8X\n3CaG5Qtf+ELQ+zY99Mtb3/qOoX2dC61ff0qWr3nNmuOzrTdEhkub416W6biDuFxW2FoPYu3zfb0L\n1ftoe1mpR/qHgDf3bW+s971Cjv/rctj27HmwE19nv5de+kXGo+da6zbHHcRPnznPrZLWeqE2a5/n\n6x30/FiR1+lLOgZ4HHgP8CTwCPDhiNg38IOZmVljK/JIPyJ+JelKYBe9Vwh92QPfzCy/bL+Ra2Zm\nwze038iVdEDSf0t6VNIj9b51knZJelzSNyWdOKw8y5H0ZUmzkh7r27dkXknbJO2XtE/SBXlSz1si\n/3ZJP5b03fpyYd/HRia/pI2S7pf0fUl7JH2i3l/E+i+S/0/r/aWs/3GSHq7/re6RtL3eP/Lr/xrZ\ni1j7OZLW1Dl31tuDW/uUZ4GP5gL8EFi3YN/1wJ/X168BPj2sPA3yvhOYBB5bLi9wNvAovbpsAvhf\n6p+iRiz/duDqRW571ijlB8aByfr6CfSeHzqzlPV/jfxFrH+d6fj6z2OAh+i95LqU9V8sezFrX+f6\nJPBPwM56e2BrP8z33hGv/sliC/NPo+8ALh5intcUEd8Cnluwe6m8FwG3R8SRiDgA7Cfj7yXAkvlh\n8ZcgbGGE8kfE4YjYXV9/AdhH7xVgRaz/Evk31B8e+fUHiIi5l2AdR2+gBOWs/2LZoZC1l7QR+G3g\n5r7dA1v7YQ79AO6V9B1JH6v3rY+IWej9QwFOGmKeNk5aIu/CX0Y7xPw/8lFzpaTdkm7u+xFxZPNL\nmqD3E8tDLH2+lJD/4XpXEetf1wuPAoeBeyPiOxSy/ktkh0LWHvhb4M+Y/2YFA1z7YQ79zRHxdnrf\nwT4u6Td55RfFItujrrS8NwFviYhJev8gPps5z2uSdALwdeCq+hFzUefLIvmLWf+IeCkizqX3E9Ym\nSedQyPovkv1sCll7Sb8DzNY/Kb7WC/Rbr/3Qhn5EPFn/+TRwJ70fQWYlrQeQNA48Naw8LS2V9xBw\nct/tFv1ltNwi4umoi0DgS8z/GDhy+SWtpTcwvxoRd9W7i1n/xfKXtP5zIuJn9N6W8kIKWn94ZfaC\n1n4zcJGkHwL/DPyWpK8Chwe19kMZ+pKOrx/1IOn1wAXAHmAnMF3f7DLgrkU/QT7ild9tl8q7E7hU\n0rGSTgVOo/cLabm9In99ssz5IPC9+voo5v9HYG9E/F3fvpLW/1X5S1l/SW+cqz8kvQ54H73nJUZ+\n/ZfI/oNS1j4iro2IN0fEW4BLgfsj4g+BuxnU2g/pmehT6b2f6aP0hv3Wev8bgPvovbphFzA2jDwN\nM98G/B+9t9f7EfBRYN1SeYFt9J453wdcMKL5bwEeq/8u7qTXE45cfnqPdn7Vd858l94jzSXPl0Ly\nl7L+b60z767z/kW9f+TX/zWyF7H2C76WdzH/6p2Brb1/OcvMrEP83yWamXWIh76ZWYd46JuZdYiH\nvplZh3jom5l1iIe+mVmHeOibmXWIh76ZWYf8P2ymlAyN0TD+AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11fb4bcd0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df.groupby('episode_id')['id'].nunique().hist()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise A3: What are the minimum, maximum and mean number of episodes per season?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "min     13.000000\n",
       "mean    21.692308\n",
       "max     25.000000\n",
       "Name: episode_id, dtype: float64"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby('season')['episode_id'].nunique().agg(['min', 'mean', 'max'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise A4: Show (plot) the distribution of the number of episodes per season."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x11a81f950>"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEACAYAAACj0I2EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFAZJREFUeJzt3X+M5HV9x/HnGw4tlOJCjXepl7BiQmyb6NrWH6lEJ+IP\nbFPgD/+R1rqY+IepP2IMxR9pqP9QJG2tsfEPI7BopK1FI9DWSomOCVW0AgsoJ5rU41B7azDQBm2s\neu/+sXN363J7szPfmfnO+zvPR7LJfr4zO5/XDft9M/vaue9FZiJJqu+UtgNIkibDgS5JHeFAl6SO\ncKBLUkc40CWpIxzoktQRQwd6RFwXERsRcf8JbntnRByJiHOmE0+StFu7eYV+A/Dq7QcjYj/wSuDh\nSYeSJI1u6EDPzDuBx05w0weAKyaeSJI0lrE69Ii4GHgkMx+YcB5J0pj2jPoFEXE68B4265ZjhyeW\nSJI0lpEHOvBsYBm4LyIC2A/cHREvzMwfbL9zRHixGEkaQ2aO9GJ5t5VLDD7IzK9n5r7MPC8znwV8\nF3j+iYb5llBlP6666qrWMyxq/srZzd/+R/X849jN2xZvAr4EnB8RhyLi8u3zmg5XLgcPHmw7QiOV\n81fODuZvW/X84xhauWTmZUNuP29ycSRJ4/Jvig6xurradoRGKuevnB3M37bq+ccR43Y1u94gIqe9\nhyR1TUSQU/ql6MLq9/ttR2ikcv7K2cH8bauefxwOdEnqCCsXSZpDVi6StMAc6ENU7+Eq56+cHczf\ntur5x+FAl6SOsEOXpDlkhy5JC8yBPkT1Hq5y/srZwfxtq55/HA50SeoIO3Sp4/btW2ZjY3b/9O/e\nvedy+PDBme3XVeN06A50qeM2/x2aWZ6DMfb1vHWcvxSdguo9XOX8lbND/fzQbztAI/Wf/9E50CWp\nI6xcpI6zcqnJykWSFpgDfYjqPVzl/JWzQ/38duj1ONAlqSPs0KWOs0OvyQ5dkhaYA32I6j1c5fyV\ns0P9/Hbo9Qwd6BFxXURsRMT9W45dGxEHImI9Ij4VEWdNN6YkaZihHXpEXAA8AXwsM587OPYK4POZ\neSQirgEyM9+9w9fboUstskOvaSodembeCTy27dgdmXlksLwL2D/KppKkyZtEh/5G4LMTeJy5VL2H\nq5y/cnaon98OvZ49Tb44It4L/DQzbzrZ/VZXV1leXgZgaWmJlZUVer0ecPxJn9f1+vr6XOVZtPyu\nJ7M+7ui6N+U1jfIu4rrf77O2tgZwbF6OalfvQ4+Ic4Hbjnbog2OrwJuAl2fmT07ytXboUovs0Gsa\np0Pf7Sv0GHwc3egi4ArgpScb5pKk2dnN2xZvAr4EnB8RhyLicuBDwJnAv0XEPRHx4SnnbM2Tf2St\npXL+ytmhfn479HqGvkLPzMtOcPiGKWSRJDXgtVykjrNDr8lruUjSAnOgD1G9h6ucv3J2qJ/fDr0e\nB7okdYQdutRxdug12aFL0gJzoA9RvYernL9ydqif3w69Hge6JHWEHbrUcXboNdmhS9ICc6APUb2H\nq5y/cnaon98OvR4HuiR1hB261HF26DXZoUvSAnOgD1G9h6ucv3J2qJ/fDr0eB7okdYQdutRxdug1\n2aFL0gJzoA9RvYernL9ydqif3w69Hge6JHWEHbrUcXboNdmhS9ICc6APUb2Hq5y/cnaon98OvZ6h\nAz0irouIjYi4f8uxsyPi9oh4KCI+FxFPm25MSdIwQzv0iLgAeAL4WGY+d3Ds/cAPM/PaiLgSODsz\n37XD19uhSy2yQ69pKh16Zt4JPLbt8CXAjYPPbwQuHWVTSdLkjduhPyMzNwAy8zDwjMlFmi/Ve7jK\n+Stnh/r57dDr2TOhxznpz1erq6ssLy8DsLS0xMrKCr1eDzj+pM/ren19fa7yLFp+15NZH3d03Zvy\nmkZ5F3Hd7/dZW1sDODYvR7Wr96FHxLnAbVs69ANALzM3ImIf8IXM/PUdvtYOXWqRHXpN03wfegw+\njroVWB18/gbgllE2lSRN3m7etngT8CXg/Ig4FBGXA9cAr4yIh4ALB+tOevKPrLVUzl85O9TPb4de\nz9AOPTMv2+GmV0w4iySpAa/lInWcHXpNXstFkhaYA32I6j1c5fyVs0P9/Hbo9TjQJakj7NCljrND\nr8kOXZIWmAN9iOo9XOX8lbND/fx26PU40CWpI+zQpY6zQ6/JDl2SFpgDfYjqPVzl/JWzQ/38duj1\nONAlqSPs0KWOs0OvyQ5dkhaYA32I6j1c5fyVs0P9/Hbo9TjQJakj7NCljrNDr8kOXZIWmAN9iOo9\nXOX8lbND/fx26PU40CWpI+zQpY6zQ6/JDl2SFpgDfYjqPVzl/JWzQ/38duj1NBroEfGOiPh6RNwf\nEZ+IiKdMKpgkaTRjd+gR8WvAncBzMvP/IuIfgH/OzI9tu58dutQiO/SaxunQ9zTc81TglyPiCHAG\n8P2GjydJGtPYlUtmfh/4K+AQ8D3g8cy8Y1LB5kX1Hq5y/srZoX5+O/R6xn6FHhFLwCXAucB/AzdH\nxGWZedP2+66urrK8vAzA0tISKysr9Ho94PiTPq/r9fX1ucqzaPldT2Z93NF1b8prGuVdxHW/32dt\nbQ3g2LwcVZMO/bXAqzPzTYP164EXZeZbtt3PDl1qkR16TbN+H/oh4MUR8Uux+R1zIXCgweNJkhpo\n0qF/FbgZuBe4DwjgIxPKNTee/CNrLZXzV84O9fPbodfT6F0umfk+4H0TyiJJasBruUgdZ4dek9dy\nkaQF5kAfonoPVzl/5exQP78dej0OdEnqCDt0qePs0GuyQ5ekBeZAH6J6D1c5f+XsUD+/HXo9DnRJ\n6gg7dKnj7NBrskOXpAXmQB+ieg9XOX/l7FA/vx16PQ50SeoIO3Sp4+zQa7JDl6QF5kAfonoPVzl/\n5exQP78dej0OdEnqCDt0qePs0GuyQ5ekBeZAH6J6D1c5f+XsUD+/HXo9DnRJ6gg7dKnj7NBrskOX\npAXmQB+ieg9XOX/l7FA/vx16PY0GekQ8LSL+MSIORMQ3IuJFkwomSRpNow49ItaAL2bmDRGxBzgj\nM/9n233s0KUW2aHXNE6HPvZAj4izgHsz89lD7udAl1rkQK9p1r8UfRbwaETcEBH3RMRHIuL0Bo83\nl6r3cJXzV84O9fPbodezp+HX/hbwJ5n5tYj4G+BdwFXb77i6usry8jIAS0tLrKys0Ov1gONP+ryu\n19fX5yrPouV3PZn1cUfXvSmvaZR3Edf9fp+1tTWAY/NyVE0ql73AlzPzvMH6AuDKzPyDbfezcpFa\nZOVS00wrl8zcAB6JiPMHhy4EHhz38SRJzTR9H/rbgE9ExDrwPODq5pHmy5N/ZK2lcv7K2aF+fjv0\nepp06GTmfcALJpRFktSA13KROs4OvSav5SJJC8yBPkT1Hq5y/srZoX5+O/R6HOiS1BF26FLH2aHX\nZIcuSQvMgT5E9R6ucv7K2aF+fjv0ehzoktQRduhSx9mh12SHLkkLzIE+RPUernL+ytmhfn479Hoa\nXctFkubBvn3LbGw8PNM99+49l8OHD850z2Hs0KWOW4QOffZ/Rpj2n9MOXZIWmAN9iOo9XOX8lbND\n/fzVO/T6+UfnQJekjrBDlzrODn1qu9qhS5Kmw4E+RPUetHL+ytmhfv76HXS/7QAz50CXpI6wQ5c6\nzg59arvaoUuSpsOBPkT1HrRy/srZoX7++h10v+0AM9d4oEfEKRFxT0TcOolAkqTxNO7QI+IdwG8D\nZ2XmxSe43Q5dapEd+tR27VaHHhH7gd8DPtrkcSRJzTWtXD4AXMHs/9c4M9V70Mr5K2eH+vnrd9D9\ntgPM3NjXQ4+I3wc2MnM9InrAjj8arK6usry8DMDS0hIrKyv0ej3g+Df9vK7X19fnKs+i5Xc9mfVx\nR9e9Ka9plHf+/3yTzd/r9ej3+6ytrQEcm5ejGrtDj4irgT8CfgacDvwK8OnM/ONt97NDl1pkhz61\nXeeuQ5/IXyyKiJcB7/SXotL8caBPbde5G+i+D32IJ/9IV0vl/JWzQ/389TvoftsBZm4i/6ZoZn4R\n+OIkHkuSNB6v5SJ1nJXL1Ha1cpEkTYcDfYjqPWjl/JWzQ/389TvoftsBZs6BLkkdYYcudZwd+tR2\ntUOXJE2HA32I6j1o5fyVs0P9/PU76H7bAWbOgS5JHWGHLnWcHfrUdrVDlyRNhwN9iOo9aOX8lbND\n/fz1O+h+2wFmbiLXcpG0O/v2LbOx8XDbMdRRdujSDLXV9dqhT2VXO3RJ0nQ40Ieo3oNWzl85O9TP\nX7+D7rcdYOYc6JLUEXbo0gzZoU9pRzt0wFfoktQZDvQhqveglfNXzg7189fvoPttB5g5B7okdYQd\nujRDduhT2tEOHfAVuiR1hgN9iOo9aOX8lbND/fz1O+h+2wFmbuyBHhH7I+LzEfGNiHggIt42yWCS\npNGM3aFHxD5gX2auR8SZwN3AJZn5zW33s0OXBuzQp7SjHTrQ4BV6Zh7OzPXB508AB4Bnjvt4kqRm\nJtKhR8QysAJ8ZRKPN0+q96CV81fODvXz1++g+20HmLnG10Mf1C03A28fvFJ/ktXVVZaXlwFYWlpi\nZWWFXq8HHP+mn9f1+vr6XOVZtPzTXLd3bfI+0NvyOTNYM+T26ew3q/+eW3acUP7drSf55+n3+6yt\nrQEcm5ejavQ+9IjYA/wT8NnM/OAO97FD11xajD67jT3t0Cfy6C28D/164MGdhrkkaXaavG3xJcAf\nAi+PiHsj4p6IuGhy0eZD9R60cv7K2Tf12w7QUL/tAA312w4wc2N36Jn578CpE8wiSWrAa7loYdmh\nT28/O/QJPLrXcpGkxeVAH6J6j1s5f+Xsm/ptB2io33aAhvptB5g5B7okdYQduhaWHfr09rNDn8Cj\n26FL0uJyoA9RvcetnL9y9k39tgM01G87QEP9tgPMnANdkjrCDl0Lyw59evvZoU/g0e3QJWlxNb58\nbptuvfVf+OhHPzHVPR59dIOnP33vsfX+/fv40Ieu5dRTa1z1oN/vH7tUZzWVs2/qc/yyqxX1MX8t\npQf69df/HbfddhZwwRR3eRD4jWOrPXvexNVX/xlLS0tT3PMXtXHd7r17z+Xw4YMz3VNd8dRBBaJZ\nK92hX3rp67nlllcBr5/K45/IU56yxMbGwZkO9C72g/PADr0r+7W3px26JGkqHOhD9dsO0FC/7QBj\n833obeu3HaChftsBZs6BLkkd4UAfqtd2gIZ6bQcYW+13uEDl535Tr+0ADfXaDjBzDnRJ6ggH+lD9\ntgM01G87wNjs0NvWbztAQ/22A8ycA12SOsKBPlSv7QAN9doOMDY79Lb12g7QUK/tADPnQJekjmg0\n0CPiooj4ZkR8KyKunFSo+dJvO0BD/bYDjM0OvW39tgM01G87wMyNPdAj4hTgb4FXA78JvC4injOp\nYPNjve0ADdXNv75eN/sm87erev7RNXmF/kLg25n5cGb+FPh74JLJxJonj7cdoKG6+R9/vG72TeZv\nV/X8o2sy0J8JPLJl/d3BMUlSC0pfPvepTz2N00//a0477ZNT2+PHP76XM864+9j6Rz/6X045pdLv\nkg+2HWBsBw8ebDtCQwfbDtDQwbYDNHSw7QAzN/blcyPixcCfZ+ZFg/W7gMzM92+7X7evwSpJUzLq\n5XObDPRTgYeAC4H/Ar4KvC4zD4z1gJKkRsauXDLz5xHxFuB2Nrv46xzmktSeqf+LRZKk2Zjob/ci\n4rqI2IiI+7ccuzYiDkTEekR8KiLOmuSek3Si/Ftue2dEHImIc9rINsxO2SPirYPn/4GIuKatfMPs\n8L3zvIj4ckTcGxFfjYjfaTPjyUTE/oj4fER8Y/Bcv21w/OyIuD0iHoqIz0XE09rOut0Jsr91cLzE\nubvTc7/l9nk/d3fMP/L5m5kT+2DzX2teAe7fcuwVwCmDz68B/mKSe047/+D4fuBfge8A57Sdc4Tn\nvsdmJbZnsH562zlHzP854FWDz18DfKHtnCfJvw9YGXx+Jpu/X3oO8H7gTwfHrwSuaTvrCNlLnLs7\n5R+sK5y7Oz3/I5+/E32Fnpl3Ao9tO3ZHZh4ZLO8aPMFz6UT5Bz4AXDHjOCPZIfub2RwgPxvc59GZ\nB9ulHfIfAY6+ol0CvjfTUCPIzMOZuT74/AngAJvf65cANw7udiNwaTsJd7ZD9mdWOXd3yj+4ucK5\nu1P+kc/fWb+h+o3AZ2e8ZyMRcTHwSGY+0HaWMZwPvDQi7oqIL8xzZbGDdwB/GRGHgGuBd7ecZ1ci\nYpnNnzbuAvZm5gZsnrjAM9pLNtyW7F/ZdlOJc3dr/orn7rbnf+Tzd2Z/sSgi3gv8NDNvmtWeTUXE\n6cB7gFduPdxSnHHsAc7OzBdHxAuATwLntZxpFG8G3p6Zn4mI1wLX84v/LeZORJwJ3Mxm7idO8Pcw\n5vZdCNuzbzle4tzdmh/4OcXO3RN874x8/s7kFXpErAK/B1w2i/0m6NnAMnBfRHyHzR85746IuX6V\ntcUjwKcBMvM/gCMR8avtRhrJGzLzMwCZeTOb1w+aW4MT8Gbg45l5y+DwRkTsHdy+D/hBW/lOZofs\nZc7dE+Qvde7u8PyPfP5OY6AHW/5PGBEXsdlhXZyZP5nCfpN2LH9mfj0z92XmeZn5LDavV/P8zJzL\nk5Jtzz3wGeDlABFxPnBaZv6wjWC7tD3/9yLiZQARcSHwrVZS7d71wIOZ+cEtx24FVgefvwG4ZfsX\nzYknZS927v5C/oLn7om+d0Y/fyf829qbgO8DPwEOAZcD3wYeBu4ZfHy47d8qj5J/2+3/yfz+pvxE\nz/0e4OPAA8DXgJe1nXPE/L87yH0v8GU2T8jWs+6Q/yVs/pi/Psh7D3ARcA5wB5vvXLgdWGo76y6z\nv6bKubvTc7/tPvN87u70vXPaqOevf7FIkjqi0mUDJUkn4UCXpI5woEtSRzjQJakjHOiS1BEOdEnq\nCAe6JHWEA12SOuL/AU6yhOYqgvcZAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11a5b4a50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df.groupby('season')['episode_id'].nunique().hist()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise A5: What are the minimum, maximun and mean number of characters appearing per episode?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Depends on what we mean by \"character\": `character_id` or `raw_character_text`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "min     12.000000\n",
       "mean    38.734043\n",
       "max     69.000000\n",
       "Name: character_id, dtype: float64"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby('episode_id')['character_id'].nunique().agg(['min', 'mean', 'max'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "min     12.000000\n",
       "mean    39.099291\n",
       "max     69.000000\n",
       "Name: raw_character_text, dtype: float64"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby('episode_id')['raw_character_text'].nunique().agg(['min', 'mean', 'max'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise A6: Show (plot) the distribution of the number of characters per episode."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x11b0fc350>"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEACAYAAAC9Gb03AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAF4lJREFUeJzt3W+MpWV5x/HvD1YtqDCLDTspWxlQqWiqoxa00capgKKm\nwIvGvzUMTX2jDcYaw0JfUF9U/iQNMW19YaTMasA/aCxrQwUJ3LbUKiqMKLuspHYBrTtWpComgsDV\nF+dZdnbOrGfOc84993Pu8/skk537OXPmvi7P81yc+c2ZoyICMzOr1xGlCzAzs7w86M3MKudBb2ZW\nOQ96M7PKedCbmVXOg97MrHIDB72kqyWtSLp7nds+IOlJScetOnaxpPsk7ZH0+nEXbGZmw9nIM/pr\ngDesPShpO3AWcP+qY6cCbwFOBd4IfFSSxlOqmZm1MXDQR8TtwMPr3HQV8ME1x84FPh0Rj0fEPuA+\n4PRRizQzs/ZaZfSSzgEejIjvrLnpBODBVesfNsfMzKyQLcPeQdJRwCX0YhszM+u4oQc98DxgDvh2\nk79vB+6UdDq9Z/DPXfW125tjfST5TXbMzFqIiKF+97nR6EbNBxHx3YiYjYiTI+Ik4AfAyyLix8Au\n4K2Sni7pJOD5wB2/odhqPy699NLiNbg/9zeN/dXcW0S758cbeXnldcBXgVMkPSDpgrXzetV/BHYD\nnwV2AzcC74m2lU24ffv2lS4hK/c32Wrur+be2hoY3UTEOwbcfvKa9WXAZSPWZWZmY+K/jM1kcXGx\ndAlZub/JVnN/NffWlkolK5KmNdUxM2tNEpHpl7E2pJRS6RKycn+Treb+au6tLQ96M7PKOboxM5sg\njm7MzKyPB30mteeE7m+y1dxfzb215UFvZlY5Z/RmZhPEGb2ZmfXxoM+k9pzQ/U22mvurube2POjN\nzCrnjN5sgNnZOVZW7h/8hRls23Yi+/fvK7K3dVObjN6D3myA3v+/TqlzVa3fg9zq5F/GdkjtOaH7\nm2w191dzb2150JuZVc7RjdkAjm6sSxzdmJlZHw/6TGrPCTezv9nZOSQV+6hRzednzb215UFvndd7\naWNs8sdtqz43m2zO6K3zymbkAM7orTuc0ZuZWR8P+kxqzwlr7w9S6QKyqvnxq7m3tgYOeklXS1qR\ndPeqY1dK2iNpWdLnJR2z6raLJd3X3P76XIWbmdnGDMzoJb0GeAT4RES8pDl2JnBrRDwp6XIgIuJi\nSS8CrgVOA7YDtwAvWC+Md0ZvG+WM3teJHZQlo4+I24GH1xy7JSKebJZfozfUAc4BPh0Rj0fEPuA+\n4PRhCjIzs/EaR0b/58CNzecnAA+uuu2HzbGpU3tOWHt/zugnV829tbVllDtL+mvg1xHxqTb3X1xc\nZG5uDoCZmRnm5+dZWFgADj5Yk7peXl7uVD2jrEu+Te9Bqfl3YZPWy5u83+HWzcrn59SuU0osLS0B\nPDUvh7Wh19FLOhH44oGMvjm2CLwbeF1EPNoc20Evr7+iWX8JuDQivr7O93RGPyGmOyMvvb8zejtU\nztfRq/k4sNHZwAeBcw4M+cYu4G2Sni7pJOD5wB3DFGRmZuO1kZdXXgd8FThF0gOSLgD+HngW8GVJ\nd0r6KEBE7AY+C+yml9u/Z1qfttefE6bSBWSWSheQVc3nZ829tTUwo4+Id6xz+Jrf8PWXAZeNUpSZ\nmY2P3+vGBnJG74zeusPvdWNmZn086DOpPydMpQvILJUuIKuaz8+ae2vLg97MrHLO6G0gZ/TO6K07\nnNGbmVkfD/pM6s8JU+kCMkulC8iq5vOz5t7a8qA3M6ucM3obyBm9M3rrDmf0ZmbWx4M+k/pzwlS6\ngMxS6QKyqvn8rLm3tjzozcwq54zeBnJG74zeusMZvZmZ9fGgz6T+nDCVLiCzVLqArGo+P2vurS0P\nejOzyjmjt4Gc0Tujt+5wRm9mZn086DOpPydMpQvILJUuIKuaz8+ae2vLg97MrHLO6G0gZ/TO6K07\nnNGbmVkfD/pM6s8JU+kCMkulC8iq5vOz5t7aGjjoJV0taUXS3auObZV0s6S9km6SdOyq2y6WdJ+k\nPZJen6twMzPbmIEZvaTXAI8An4iIlzTHrgAeiogrJV0EbI2IHZJeBFwLnAZsB24BXrBeGO+MfnI4\no3dGb92RJaOPiNuBh9ccPhfY2Xy+Eziv+fwc4NMR8XhE7APuA04fpiAzMxuvthn98RGxAhAR+4Hj\nm+MnAA+u+rofNsemTv05YSpdQGapdAFZ1Xx+1txbW1vG9H1a/Wy5uLjI3NwcADMzM8zPz7OwsAAc\nfLAmdb28vNypekZdHxx8B9bLa9Zrb5/0dVf6a1Y+P6d2nVJiaWkJ4Kl5OawNvY5e0onAF1dl9HuA\nhYhYkTQL3BYRp0raAUREXNF83ZeASyPi6+t8T2f0E8IZvTN6646cr6NX83HALmCx+fx84IZVx98m\n6emSTgKeD9wxTEFmZjZeG3l55XXAV4FTJD0g6QLgcuAsSXuBM5o1EbEb+CywG7gReM+0Pm2vPydM\npQvILJUuIKuaz8+ae2trYEYfEe84zE1nHubrLwMuG6UoMzMbH7/XjQ3kjN4ZvXWH3+vGzMz6eNBn\nUn9OmEoXkFkqXUBWNZ+fNffWlge9mVnlnNHbQM7ondFbdzijNzOzPh70mdSfE6bSBWSWSheQVc3n\nZ829teVBb2ZWOWf0NpAzemf01h3O6M3MrI8HfSb154SpdAGZpdIFZFXz+Vlzb2150JuZVc4ZvQ3k\njN4ZvXWHM3ozM+vjQZ9J/TlhKl1AZql0AVnVfH7W3FtbHvRmZpVzRm8DOaN3Rm/d4YzezMz6eNBn\nUn9OmEoXkFkqXUBWNZ+fNffWlge9mVnlnNHbQM7ondFbdzijNzOzPh70mdSfE6bSBWSWSheQVc3n\nZ829tTXSoJf0fknflXS3pGslPV3SVkk3S9or6SZJx46rWDMzG17rjF7S7wC3Ay+MiMckfQa4EXgR\n8FBEXCnpImBrROxY5/7O6CeEM3pn9NYdJTL6I4FnStoCHAX8EDgX2NncvhM4b8Q9zMxsBK0HfUT8\nD/B3wAP0BvzPIuIWYFtErDRfsx84fhyFTpr6c8JUuoDMUukCsqr5/Ky5t7a2tL2jpBl6z95PBH4G\nXC/pnfT/jHvYnzsXFxeZm5sDYGZmhvn5eRYWFoCDD9akrpeXlztVz6jrg4PvwHp5zXrt7ZO+7kp/\nzcrn59SuU0osLS0BPDUvhzVKRv+nwBsi4t3N+l3Aq4DXAQsRsSJpFrgtIk5d5/7O6CeEM3pn9NYd\nm53RPwC8StJvqTcJzgB2A7uAxeZrzgduGGEPMzMb0SgZ/R3A54C7gG/Te9rzMeAK4CxJe+kN/8vH\nUOfEqT8nTKULyCyVLiCrms/Pmntrq3VGDxARHwI+tObwT4EzR/m+ZmY2Pn6vGxvIGb0zeusOv9eN\nmZn18aDPpP6cMJUuILNUuoCsaj4/a+6tLQ96M7PKOaO3gZzRO6O37nBGb2ZmfTzoM6k/J0ylC8gs\nlS4gq5rPz5p7a8uD3syscs7obSBn9M7orTuc0ZuZWR8P+kzqzwlT6QIyS6ULyKrm87Pm3tryoDcz\nq5wzehvIGb0zeusOZ/RmZtbHgz6T+nPCVLqAzFLpArKq+fysube2POjNzCrnjN4GckbvjN66wxm9\nmZn18aDPpP6cMJUuILNUuoCsaj4/a+6tLQ96M7PKOaO3gZzRO6O37nBGb2ZmfTzoM6k/J0ylC8gs\nlS4gq5rPz5p7a2ukQS/pWEnXS9oj6R5Jr5S0VdLNkvZKuknSseMq1szMhjdSRi9pCfhKRFwjaQvw\nTOAS4KGIuFLSRcDWiNixzn2d0U8IZ/TO6K072mT0rQe9pGOAuyLieWuO3wu8NiJWJM0CKSJeuM79\nPegnhAe9B711x2b/MvYk4CeSrpF0p6SPSToa2BYRKwARsR84foQ9Jlb9OWEqXUBmqXQBWdV8ftbc\nW1tbRrzvy4H3RsQ3JV0F7KD/qc9hn44sLi4yNzcHwMzMDPPz8ywsLAAHH6xJXS8vL4/t+83OzrGy\ncj9lpebfhebf5TXrtbdP+ror/TWrDp+fXuddp5RYWloCeGpeDmuU6GYb8J8RcXKzfg29Qf88YGFV\ndHNbRJy6zv0d3WyQo5Np3t/RjR1qU6ObJp55UNIpzaEzgHuAXcBic+x84Ia2e5iZ2ehGfR39hcC1\nkpaBlwIfBq4AzpK0l97wv3zEPSZS/TlhKl1AZql0AVnVfH7W3Ftbo2T0RMS3gdPWuenMUb6vmZmN\nj9/rZgI4o5/m/Z3R26H8XjdmZtbHgz6T+nPCVLqAzFLpAhrPQFKxj9nZudL/Awyt/mtveB70Zp32\nKL3YaNwft23o68r//YaNgzP6CeCMfpr3L9+7r9NucUZvZmZ9POgzqT8nTKULyCyVLiCzVLqAbOq/\n9obnQW9mVjln9BPAGf0071++d1+n3eKM3szM+njQZ1J/TphKF5BZKl1AZql0AdnUf+0Nz4PezKxy\nzugngDP6ad6/fO++TrvFGb2ZmfXxoM+k/pwwlS4gs1S6gMxS6QKyqf/aG54HvZlZ5ZzRTwBn9NO8\nf/nefZ12izN6MzPr40GfSf05YSpdQGapdAGZpdIFZFP/tTc8D3ozs8o5o58Azuinef/yvfs67RZn\n9GZm1seDPpP6c8JUuoDMUukCMkulC8im/mtveCMPeklHSLpT0q5mvVXSzZL2SrpJ0rGjl2lmZm2N\nnNFLej/wCuCYiDhH0hXAQxFxpaSLgK0RsWOd+zmj3yBn9NO8f/nefZ12y6Zn9JK2A28CPr7q8LnA\nzubzncB5o+xhZmajGTW6uQr4IIc+5dgWESsAEbEfOH7EPSZS/TlhKl1AZql0AZml0gVkU/+1N7wt\nbe8o6c3ASkQsS1r4DV962J/7FhcXmZubA2BmZob5+XkWFnrf6sCDNanr5eXlsX6/gxdmV9bLHatn\n3Ouu9MeA29uuN9pfs+rY9TVN65QSS0tLAE/Ny2G1zuglfRj4M+Bx4Cjg2cAXgD8AFiJiRdIscFtE\nnLrO/Z3Rb5Az+mnev3zvvk67ZVMz+oi4JCKeGxEnA28Dbo2IdwFfBBabLzsfuKHtHmZW2jOQVOxj\ndnau9P8AVcjxOvrLgbMk7QXOaNZTp/6cMJUuILNUuoDM0ga/7lF6P1GU+VhZuX/4zqq/9obXOqNf\nLSK+Anyl+fynwJnj+L5mZjY6v9fNBHBGP837T3Pvvf09Jw7l97oxM7M+HvSZ1J8TptIFZJZKF5BZ\nKl1ANvVfe8PzoDczq5wz+gngjH6a95/m3nv7e04cyhm9mZn18aDPpP6cMJUuILNUuoDMUukCsqn/\n2hueB72ZWeWc0U8AZ/TTvP80997b33PiUM7ozcysjwd9JvXnhKl0AZml0gVklkoXkE39197wPOjN\nzCrnjH4COKOf5v2nuffe/p4Th3JGb2ZmfTzoM6k/J0ylC8gslS4gs1S6gGzqv/aG50FvZlY5Z/QT\nwBn9NO8/zb339vecOJQzejMz6+NBn0n9OWEqXUBmqXQBmaXSBWRT/7U3PA96M7PKOaOfAM7op3n/\nae69t7/nxKGc0ZuZWR8P+kzqzwlT6QIyS6ULyCyVLiCb+q+94bUe9JK2S7pV0j2SviPpwub4Vkk3\nS9or6SZJx46vXDMzG1brjF7SLDAbEcuSngV8CzgXuAB4KCKulHQRsDUidqxzf2f0G+SMfpr3n+be\ne/t7ThxqUzP6iNgfEcvN548Ae4Dt9Ib9zubLdgLntd3DzMxGN5aMXtIcMA98DdgWESvQ+48BcPw4\n9pg09eeEqXQBmaXSBWSWSheQTf3X3vC2jPoNmtjmc8D7IuIRSWt/zjrsz12Li4vMzc0BMDMzw/z8\nPAsLC8DBB2tS18vLy2P9fgcvzK6slztWz7jXXemPAbe3XW+0v1z7b3TdrDp2fW/mOqXE0tISwFPz\nclgjvY5e0hbgX4B/jYiPNMf2AAsRsdLk+LdFxKnr3NcZ/QY5o5/m/ae5997+nhOHKvE6+n8Cdh8Y\n8o1dwGLz+fnADSPuYWZmIxjl5ZWvBt4JvE7SXZLulHQ2cAVwlqS9wBnA5eMpdbLUnxOm0gVklkoX\nkFkqXUA29V97w2ud0UfEfwBHHubmM9t+XzMzGy+/180GzM7OsbJyf+Eqpjunnd79p7n33v6TMic2\nS5uM3oN+A/zLUO/vQV9u/0mZE5vFb2rWKal0AZml0gVklkoXkFkqXUA2zuj7edCbmVXO0c0GOLrx\n/o5uyu0/KXNiszi6MTOzPh702aTSBWSWSheQWSpdQGapdAHZOKPv50FvZlY5Z/Qb4Ize+zujL7f/\npMyJzeKM3szM+njQZ5NKF5BZKl1AZql0AZml0gVk44y+38jvR29mls8zmui0jG3bTmT//n3F9h8X\nZ/Qb4Ize+zujn979uzannNGbmVkfD/psUukCMkulC8gslS4gs1S6gIxS6QI6x4PezKxyzug3wBm9\n93dGP737d21OOaM3M7M+HvTZpNIFZJZKF5BZKl1AZql0ARml0gV0zkS8jv6RRx7hV7/6VekyzMwm\nUucz+ieeeILjjpvlscfK1Pnkk7/mscd+Tumc0PtP6/7T3Hs39q8ho+/8M/qI4Be/eJiIxwtVcAfw\nykJ7m5mNLltGL+lsSfdK+p6ki3Lt012pdAGZpdIFZJZKF5BZKl1ARql0AZ2TZdBLOgL4B+ANwIuB\nt0t6YY69umu5dAGZub/JVnN/NffWTq5n9KcD90XE/RHxa+DTwLmZ9uqo/ytdQGbub7LV3N84e+u9\nqVqpj9nZubF0kSujPwF4cNX6B/SGv5nZBHmUkr8MXlkZzzt3dv6XsdD7LfOzn/0nRfZ+4omH+eUv\n29xz35gr6Zp9pQvIbF/pAjLbV7qAjPaVLqBzsry8UtKrgL+JiLOb9Q4gIuKKVV/TrdcsmZlNiGFf\nXplr0B8J7AXOAH5E7zWKb4+IPWPfzMzMfqMs0U1EPCHpL4Gb6f3C92oPeTOzMor9ZayZmW2OTXlT\nM0lXS1qRdPeqY1sl3Sxpr6SbJB27GbWMm6Ttkm6VdI+k70i6sDleS3/PkPR1SXc1/V3aHK+ivwMk\nHSHpTkm7mnU1/UnaJ+nbzWN4R3Ospv6OlXS9pD3NdfjKWvqTdErzuN3Z/PszSRcO299mvXvlNfT+\neGq1HcAtEfF7wK3AxZtUy7g9DvxVRLwY+EPgvc0fh1XRX0Q8CvxxRLwMmAfeKOl0KulvlfcBu1et\na+rvSWAhIl4WEQde5lxTfx8BboyIU4GXAvdSSX8R8b3mcXs58Argl8AXGLa/iNiUD+BE4O5V63uB\nbc3ns8C9m1VL5j7/GTizxv6Ao4FvAqfV1B+wHfgysADsao7V1N9/A89Zc6yK/oBjgP9a53gV/a3p\n6fXAv7fpr+T70R8fESsAEbEfOL5gLWMhaY7es96v0XsQquiviTXuAvYDX46Ib1BRf8BVwAc59C9j\nauovgC9L+oakv2iO1dLfScBPJF3TxBsfk3Q09fS32luB65rPh+qvS//HIxP9W2FJzwI+B7wvIh6h\nv5+J7S8inoxedLMdOF3Si6mkP0lvBlYiYpnee+IezkT213h19H70fxO9aPGPqOTxo/fKwZcD/9j0\n+Et6sUYt/QEg6WnAOcD1zaGh+is56FckbQOQNAv8uGAtI5G0hd6Q/2RE3NAcrqa/AyLi5/TeGvBs\n6unv1cA5kr4PfAp4naRPAvsr6Y+I+FHz7//SixZPp57H7wfAgxHxzWb9eXqDv5b+Dngj8K2I+Emz\nHqq/zRz04tBnTLuAxebz84Eb1t5hgvwTsDsiPrLqWBX9SfrtA7/Rl3QUcBawh0r6i4hLIuK5EXEy\n8Dbg1oh4F/BFKuhP0tHNT5tIeia9nPc71PP4rQAPSjqlOXQGcA+V9LfK2+k9ETlgqP425XX0kq6j\n94uu5wArwKX0nllcD/wucD/wloiYuLfUk/Rq4N/oXTzRfFxC76+BP8vk9/f7wE56TwqOAD4TEX8r\n6Tgq6G81Sa8FPhAR59TSn6ST6L1KI+jFHNdGxOW19Acg6aXAx4GnAd8HLgCOpJ7+jqbXw8kR8Yvm\n2FCPn/9gysyscl36ZayZmWXgQW9mVjkPejOzynnQm5lVzoPezKxyHvRmZpXzoDczq5wHvZlZ5f4f\n2MZvX0Qi0KAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11b2006d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df.groupby('episode_id')['character_id'].nunique().hist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x11a8f8a50>"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEACAYAAAC9Gb03AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAF8dJREFUeJzt3WuQpFV9x/HvD9cLeGFWU+xU3MiASkQrOmpcTWnKiVxE\nrYAvUl5jMaTiG01pGctiIS+ILyKXqpRlJfGFJTKrBV7QMqwpIkjBMTFGUWEE2WWlYhbQuGNEomKV\nIPDPi36WmZ2etaef7jPn6dO/T1WXc56envP/2/386fl1T68iAjMzq9cxpQswM7O8POjNzCrnQW9m\nVjkPejOzynnQm5lVzoPezKxyAwe9pMslrUi6bYPr3i/pUUlPX3PsAkl3Sdov6cxxF2xmZsPZzDP6\nK4DXrj8oaSdwBnD3mmOnAm8CTgVeB3xUksZTqpmZtTFw0EfE14D7N7jqw8AH1h07B/hMRDwcEQeB\nu4BdoxZpZmbttcroJZ0N3BsRt6+76pnAvWvWP2qOmZlZIduGvYGkY4EL6cU2ZmbWcUMPeuDZwBzw\n3SZ/3wncImkXvWfwz1rzvTubY30k+UN2zMxaiIihXvvcbHSj5kJEfC8iZiPi5Ig4Cfgh8OKI+Amw\nF3izpCdIOgl4DnDzbym22stFF11UvAb35/6msb+ae4to9/x4M2+vvAr4OnCKpHsknbd+Xq/5j8A+\n4HPAPuBa4F3RtrIJd/DgwdIlZOX+JlvN/dXcW1sDo5uIeNuA609et74YuHjEuszMbEz8l7GZLC4u\nli4hK/c32Wrur+be2lKpZEXStKY6ZmatSSIyvRhrQ0oplS4hK/c32Wrur+be2vKgNzOrnKMbM7MJ\n4ujGzMz6eNBnUntO6P4mW8391dxbWx70ZmaVc0ZvnTc7O8fKyt2DvzGTHTtO5NChg8X2N1urTUbv\nQW+d1/vsvJKPFbX+jBGzcfOLsR1Se05Ye3+QSheQVc33X829teVBb2ZWOUc31nmObsxWOboxM7M+\nHvSZ1J4T1t6fM/rJVXNvbXnQm5lVzhm9dZ4zerNVzujNzKyPB30mteeEtfd3ZEb/RCQVuczOzuXp\nruL7r+be2hr4b8aa2YOUio5WVob6Dd1sQ87orfO6kNGX29+vD9iRnNGbmVkfD/pMas8Ja+/P76Of\nXDX31tbAQS/pckkrkm5bc+wySfslLUv6gqSnrbnuAkl3NdefmatwMzPbnIEZvaRXAQ8An4yIFzbH\nTgdujIhHJV0CRERcIOn5wJXAy4CdwA3AczcK453R22Y5o/d5YquyZPQR8TXg/nXHboiIR5vlN+gN\ndYCzgc9ExMMRcRC4C9g1TEFmZjZe48jo/wK4tvn6mcC9a677UXNs6tSeE9benzP6yVVzb22N9D56\nSX8D/CYiPt3m9ouLi8zNzQEwMzPD/Pw8CwsLwOqdNanr5eXlTtUz6f2tDt6tWi9v8X5HWzerCb//\nvG6/TimxtLQE8Ni8HNam3kcv6UTgS4cz+ubYIvBO4DUR8WBzbDe9vP7SZv1l4KKI+OYGP9MZvW2K\nM3qfJ7Yq5/vo1VwOb3QW8AHg7MNDvrEXeIukJ0g6CXgOcPMwBZmZ2Xht5u2VVwFfB06RdI+k84B/\nAJ4CfEXSLZI+ChAR+4DPAfvo5fbvmtan7Yd/9apV7f05o59cNffW1sCMPiLetsHhK37L918MXDxK\nUWZmNj7+rBvrPGf0Pk9slT/rxszM+njQZ1J7Tlh7f87oJ1fNvbXlQW9mVjln9NZ5zuh9ntiqNhm9\n/4UpG2h2do6VlbtLl2FmLTm6yaSmnLA35GPd5aYNjuW6lJAK7bs1anp8rldzb2150JuZVc4ZvQ00\n3Rl56f2d0duR/D56MzPr40GfSf05YSpdQGapdAFZ1fz4rLm3tjzozcwq54zeBnJG74zeusMZvZmZ\n9fGgz6T+nDCVLiCzVLqArGp+fNbcW1se9GZmlXNGbwM5o3dGb93hjN7MzPp40GdSf06YSheQWSpd\nQFY1Pz5r7q0tD3ozs8o5o7eBnNE7o7fucEZvZmZ9POgzqT8nTKULyCyVLiCrmh+fNffW1sBBL+ly\nSSuSbltzbLuk6yUdkHSdpOPXXHeBpLsk7Zd0Zq7CzcxscwZm9JJeBTwAfDIiXtgcuxS4LyIuk3Q+\nsD0idkt6PnAl8DJgJ3AD8NyNwnhn9JPDGb0zeuuOLBl9RHwNuH/d4XOAPc3Xe4A3Nl+fDXwmIh6O\niIPAXcCuYQoyM7PxapvRnxARKwARcQg4oTn+TODeNd/3o+bY1Kk/J0ylC8gslS4gq5ofnzX31ta2\nMf2cVr9bLi4uMjc3B8DMzAzz8/MsLCwAq3fWpK6Xl5c7Vc+o69XBd3i9vG69/vpJX3elv2blx+fU\nrlNKLC0tATw2L4e1qffRSzoR+NKajH4/sBARK5JmgZsi4lRJu4GIiEub7/sycFFEfHODn+mMfkI4\no3dGb92R8330ai6H7QUWm6/PBa5Zc/wtkp4g6STgOcDNwxRkZmbjtZm3V14FfB04RdI9ks4DLgHO\nkHQAOK1ZExH7gM8B+4BrgXdN69P2+nPCVLqAzFLpArKq+fFZc29tDczoI+JtR7nq9KN8/8XAxaMU\nZWZm4+PPurGBnNE7o7fu8GfdmJlZHw/6TOrPCVPpAjJLpQvIqubHZ829teVBb2ZWOWf0NpAzemf0\n1h3O6M3MrI8HfSb154SpdAGZpdIFZFXz47Pm3tryoDczq5wzehvIGb0zeusOZ/RmZtbHgz6T+nPC\nVLqAzFLpArKq+fFZc29tedCbmVXOGb0N5IzeGb11hzN6MzPr40GfSf05YSpdQGapdAFZ1fz4rLm3\ntjzozcwq54zeBnJG74zeusMZvZmZ9fGgz6T+nDCVLiCzVLqArGp+fNbcW1se9GZmlXNGbwM5o3dG\nb93hjN7MzPp40GdSf06YSheQWSpdQFY1Pz5r7q2tkQa9pPdJ+p6k2yRdKekJkrZLul7SAUnXSTp+\nXMWamdnwWmf0kn4X+BrwvIh4SNJngWuB5wP3RcRlks4HtkfE7g1u74x+Qjijd0Zv3VEio38c8GRJ\n24BjgR8B5wB7muv3AG8ccQ8zMxtB60EfEf8D/D1wD70B//OIuAHYERErzfccAk4YR6GTpv6cMJUu\nILNUuoCsan581txbW9va3lDSDL1n7ycCPweulvR2+n/HPervnYuLi8zNzQEwMzPD/Pw8CwsLwOqd\nNanr5eXlTtUz6np18B1eL69br79+0tdd6a9Z+fE5teuUEktLSwCPzcthjZLR/xnw2oh4Z7N+B/AK\n4DXAQkSsSJoFboqIUze4vTP6CeGM3hm9dcdWZ/T3AK+Q9CT1JsFpwD5gL7DYfM+5wDUj7GFmZiMa\nJaO/Gfg8cCvwXXpPez4GXAqcIekAveF/yRjqnDj154SpdAGZpdIFZFXz47Pm3tpqndEDRMQHgQ+u\nO/wz4PRRfq6ZmY2PP+vGBnJG74zeusOfdWNmZn086DOpPydMpQvILJUuIKuaH58199aWB72ZWeWc\n0dtAzuid0Vt3OKM3M7M+HvSZ1J8TptIFZJZKF5BVzY/Pmntry4PezKxyzuhtIGf0zuitO5zRm5lZ\nHw/6TOrPCVPpAjJLpQvIqubHZ829teVBb2ZWOWf0NpAzemf01h3O6M3MrI8HfSb154SpdAGZpdIF\nZFXz47Pm3tryoDczq5wzehvIGb0zeusOZ/RmZtbHgz6T+nPCVLqAzFLpArKq+fFZc29tedCbmVXO\nGb0N5IzeGb11hzN6MzPr40GfSf05YSpdQGapdAFZ1fz4rLm3tkYa9JKOl3S1pP2S7pD0cknbJV0v\n6YCk6yQdP65izcxseCNl9JKWgK9GxBWStgFPBi4E7ouIyySdD2yPiN0b3NYZ/YRwRl9y/ycBDxba\nG3bsOJFDhw4W29/6tcnoWw96SU8Dbo2IZ687fifw6ohYkTQLpIh43ga396CfEB70ZV+MLd27z9Nu\n2eoXY08CfirpCkm3SPqYpOOAHRGxAhARh4ATRthjYtWfE6bSBWSWSheQWSpdQDb1n3vD2zbibV8C\nvDsivi3pw8Bu+p9+HPXpwOLiInNzcwDMzMwwPz/PwsICsHpnTep6eXm5U/WMul4dDIfXy+vW66+f\n9HVX+mPA9W3Xm+2vWXXs8ThN65QSS0tLAI/Ny2GNEt3sAP4zIk5u1q+iN+ifDSysiW5uiohTN7i9\no5sJ4ejG0Y11x5ZGN008c6+kU5pDpwF3AHuBxebYucA1bfcwM7PRjfo++vcAV0paBl4EfAi4FDhD\n0gF6w/+SEfeYSPXnhKl0AZml0gVklkoXkE39597wRsnoiYjvAi/b4KrTR/m5ZmY2Pv6sGxvIGb0z\neusOf9aNmZn18aDPpP6cMJUuILNUuoDMUukCsqn/3BueB72ZWeWc0dtAzuid0Vt3tMnoR3rXjW2N\n2dk5VlbuLl2GmU0oRzeZjDMn7A35KHjZsMOx9ddNqXQBmaXSBWTjjL6fB72ZWeWc0U8AZ+TTvH/5\n3n2edovfR29mZn086DOpPydMpQvILJUuILNUuoBs6j/3hudBb2ZWOWf0E8AZ/TTvX753n6fd4oze\nzMz6eNBnUn9OmEoXkFkqXUBmqXQB2dR/7g3Pg97MrHLO6CeAM/pp3r987z5Pu8UZvZmZ9fGgz6T+\nnDCVLiCzVLqAzFLpArKp/9wbnge9mVnlnNFPAGf007x/+d59nnaLM3ozM+vjQZ9J/TlhKl1AZql0\nAZml0gVkU/+5N7yRB72kYyTdImlvs94u6XpJByRdJ+n40cs0M7O2Rs7oJb0PeCnwtIg4W9KlwH0R\ncZmk84HtEbF7g9s5o98kZ/TTvH/53n2edsuWZ/SSdgKvBz6+5vA5wJ7m6z3AG0fZw8xKeiKSil1m\nZ+dK/x9QhVGjmw8DH+DIpxw7ImIFICIOASeMuMdEqj8nTKULyCyVLiCztMnve5CS/15x799LHrKz\n6s+94W1re0NJbwBWImJZ0sJv+daj/t63uLjI3NwcADMzM8zPz7Ow0PtRh++sSV0vLy+P9eetnphd\nWS93rJ5xr7vSHwOub7vebH+59t/sull17PzeynVKiaWlJYDH5uWwWmf0kj4E/DnwMHAs8FTgi8Af\nAgsRsSJpFrgpIk7d4PbO6DfJGf007z/Nvff295w40pZm9BFxYUQ8KyJOBt4C3BgR7wC+BCw233Yu\ncE3bPczMbHQ53kd/CXCGpAPAac166tSfE6bSBWSWSheQWSpdQDb1n3vDa53RrxURXwW+2nz9M+D0\ncfxcMzMbnT/rZgI4o5/m/ae5997+nhNH8mfdmJlZHw/6TOrPCVPpAjJLpQvILJUuIJv6z73hedCb\nmVXOGf0EcEY/zftPc++9/T0njuSM3szM+njQZ1J/TphKF5BZKl1AZql0AdnUf+4Nz4PezKxyzugn\ngDP6ad5/mnvv7e85cSRn9GZm1seDPpP6c8JUuoDMUukCMkulC8im/nNveB70ZmaVc0Y/AZzRT/P+\n09x7b3/PiSM5ozczsz4e9JnUnxOm0gVklkoXkFkqXUA29Z97w/OgNzOrnDP6CeCMfpr3n+bee/t7\nThzJGb2ZmfXxoM+k/pwwlS4gs1S6gMxS6QKyqf/cG54HvZlZ5ZzRTwBn9NO8/zT33tvfc+JIzujN\nzKyPB30m9eeEqXQBmaXSBWSWSheQTf3n3vBaD3pJOyXdKOkOSbdLek9zfLuk6yUdkHSdpOPHV66Z\nmQ2rdUYvaRaYjYhlSU8BvgOcA5wH3BcRl0k6H9geEbs3uL0z+k1yRj/N+09z7739PSeOtKUZfUQc\niojl5usHgP3ATnrDfk/zbXuAN7bdw8zMRjeWjF7SHDAPfAPYEREr0PuPAXDCOPaYNPXnhKl0AZml\n0gVklkoXkE39597wto36A5rY5vPAeyPiAUnrf8866u9di4uLzM3NATAzM8P8/DwLCwvA6p01qevl\n5eWx/rzVE7Mr6+WO1TPudVf6Y8D1bdeb7S/X/ptdN6uOnd9buU4psbS0BPDYvBzWSO+jl7QN+Bfg\nXyPiI82x/cBCRKw0Of5NEXHqBrd1Rr9Jzuinef9p7r23v+fEkUq8j/4TwL7DQ76xF1hsvj4XuGbE\nPczMbASjvL3ylcDbgddIulXSLZLOAi4FzpB0ADgNuGQ8pU6W+nPCVLqAzFLpAjJLpQvIpv5zb3it\nM/qI+A/gcUe5+vS2P9fMzMbLn3WzCbOzc6ys3F24iunOaad3/2nuvbf/pMyJrdImo/eg3wS/GOr9\nPejL7T8pc2Kr+EPNOiWVLiCzVLqAzFLpAjJLpQvIxhl9Pw96M7PKObrZBEc33t/RTbn9J2VObBVH\nN2Zm1seDPptUuoDMUukCMkulC8gslS4gG2f0/Tzozcwq54x+E5zRe39n9OX2n5Q5sVXaZPQjf3ql\nmVk+T2yeaJWxY8eJHDp0sNj+4+LoJptUuoDMUukCMkulC8gslS5gkx6k9xvFMJebWtxm40v5v4gf\nDw96M7PKOaPfBGf03t8Z/fTu37U55ffRm5lZHw/6bFLpAjJLpQvILJUuILNUuoCMUukCOseD3sys\ncs7oN8EZvfd3Rj+9+3dtTjmjNzOzPh702aTSBWSWSheQWSpdQGapdAEZpdIFdM5E/GXsAw88wK9/\n/evSZZiZTaTOZ/SPPPIIT3/6LA89VKbORx/9DQ899AtK54Tef1r3n+beu7F/DRl955/RRwS//OX9\nRDxcqIKbgZcX2tvMbHTZMnpJZ0m6U9L3JZ2fa5/uSqULyCyVLiCzVLqAzFLpAjJKY/xZvQ9VK3WZ\nnZ0bSxdZBr2kY4B/BF4LvAB4q6Tn5diru5ZLF5CZ+5tsNfc3zt7afKja+C7j+lC1XM/odwF3RcTd\nEfEb4DPAOZn26qj/K11AZu5vstXcX829tZNr0D8TuHfN+ofNMTMz22KdfzEWeq8yP/Wpf1pk70ce\nuZ9f/arNLQ+OuZKuOVi6gMwOli4gs4OlC8joYOkCOifL2yslvQL424g4q1nvBiIiLl3zPd16z5KZ\n2YQY9u2VuQb944ADwGnAj+m9R/GtEbF/7JuZmdlvlSW6iYhHJP0VcD291wEu95A3Myuj2F/GmpnZ\n1tiSDzWTdLmkFUm3rTm2XdL1kg5Iuk7S8VtRy7hJ2inpRkl3SLpd0nua47X090RJ35R0a9PfRc3x\nKvo7TNIxkm6RtLdZV9OfpIOSvtvchzc3x2rq73hJV0va35yHL6+lP0mnNPfbLc3//lzSe4btb6s+\nvfIKen88tdZu4IaI+H3gRuCCLapl3B4G/joiXgD8EfDu5o/DqugvIh4E/iQiXgzMA6+TtItK+lvj\nvcC+Neua+nsUWIiIF0fEruZYTf19BLg2Ik4FXgTcSSX9RcT3m/vtJcBLgV8BX2TY/iJiSy7AicBt\na9Z3Ajuar2eBO7eqlsx9/jNweo39AccB3wZeVlN/wE7gK8ACsLc5VlN//w08Y92xKvoDngb81wbH\nq+hvXU9nAv/epr+Sn0d/QkSsAETEIeCEgrWMhaQ5es96v0HvTqiivybWuBU4BHwlIr5FRf0BHwY+\nwJEfk1hTfwF8RdK3JP1lc6yW/k4Cfirpiibe+Jik46inv7XeDFzVfD1Uf136h0cm+lVhSU8BPg+8\nNyIeoL+fie0vIh6NXnSzE9gl6QVU0p+kNwArEbFM7zNxj2Yi+2u8Mnq/+r+eXrT4x1Ry/9F75+BL\ngH9qevwVvVijlv4AkPR44Gzg6ubQUP2VHPQrknYASJoFflKwlpFI2kZvyH8qIq5pDlfT32ER8Qt6\nHw14FvX090rgbEk/AD4NvEbSp4BDlfRHRPy4+d//pRct7qKe+++HwL0R8e1m/QV6g7+W/g57HfCd\niPhpsx6qv60c9OLIZ0x7gcXm63OBa9bfYIJ8AtgXER9Zc6yK/iT9zuFX9CUdC5wB7KeS/iLiwoh4\nVkScDLwFuDEi3gF8iQr6k3Rc89smkp5ML+e9nXruvxXgXkmnNIdOA+6gkv7WeCu9JyKHDdXflryP\nXtJV9F7oegawAlxE75nF1cDvAXcDb4qIifvYOUmvBP6N3slz+PNFL6T318CfY/L7+wNgD70nBccA\nn42Iv5P0dCroby1JrwbeHxFn19KfpJPovUsj6MUcV0bEJbX0ByDpRcDHgccDPwDOAx5HPf0dR6+H\nkyPil82xoe4//8GUmVnluvRirJmZZeBBb2ZWOQ96M7PKedCbmVXOg97MrHIe9GZmlfOgNzOrnAe9\nmVnl/h/bcMDJpyo6QgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11a0fd7d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df.groupby('episode_id')['raw_character_text'].nunique().hist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise Set B"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some of the interesting questions we might ask about The Simpsons are focused on the characters.\n",
    "\n",
    "In order to simpify analysis of The Simpsons characters, it will be useful to construct a new `DataFrame` containing data associated with each character (vs. data associated with each episode or script line, although we will use data from both sources in constructing the new `DataFrame`).\n",
    "\n",
    "The following exercises will focus primarily on using some standard `pandas` functions, and set the stage for doing some additional time series analysis."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise B1: Determine the number of lines associated with each character"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "_(We will do this together)_\n",
    "\n",
    "First, it is important to note that we have two fields in the `DataFrame` extracted from `simpsons_script_lilnes.csv` that are associated with characters:\n",
    "* `character_id`: a unique identifier for each character\n",
    "* `raw_character_text`: the full text (name) associated in the script with each appearance of a character\n",
    "\n",
    "And it is further important to note that there are more distinct `raw_character_text` values than there are distinct `character_id` values. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(6764, 6720)"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.raw_character_text.nunique(), df.character_id.nunique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we group the script lines by `character_id` (using [`pandas.DataFrame.groupby()`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.groupby.html)) and then gather the distinct `raw_character_text` values associated with each `character_id` (using [`pandas.Series.unique()`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.unique.html)), the first few rows indicate why there are more `raw_character_text` values than `character_id` values.\n",
    "\n",
    "A given character can have different personae within the context of a show (e.g., \"Marge Simpson\" and \"Young Marge\")."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "character_id\n",
       "0                                                   []\n",
       "1    [Marge Simpson, Young Marge, Marge's Thoughts,...\n",
       "2    [Homer Simpson, Homer's Brain, Young Homer, Te...\n",
       "3                                    [Seymour Skinner]\n",
       "4                                              [JANEY]\n",
       "Name: raw_character_text, dtype: object"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby('character_id').raw_character_text.unique().head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we group by both `character_id` and `raw_character_text`, and then find the number of rows in each group (using [`pandas.core.groupby.GroupBy.size()`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.core.groupby.GroupBy.size.html)), and then convert the resulting `Series` back into a `DataFrame` (using [`pandas.Series.reset_index()`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.reset_index.html)), we can create a new DataFrame with a row for each `raw_character_text` value, wherein each row has \n",
    "* `character_id`: a non-unique value which may be shared across other `raw_character_text` rows\n",
    "* `raw_character_text`: a unique full text (name) for the character\n",
    "* a column indicating the number of script lines associated with that `raw_character_text` (labeled `0`)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>character_id</th>\n",
       "      <th>raw_character_text</th>\n",
       "      <th>0</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td></td>\n",
       "      <td>17521</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>Marge Simpson</td>\n",
       "      <td>14157</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>Marge's Thoughts</td>\n",
       "      <td>23</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>Teenage Marge</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>Young Marge</td>\n",
       "      <td>72</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2</td>\n",
       "      <td>Homer Simpson</td>\n",
       "      <td>29839</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2</td>\n",
       "      <td>Homer's Brain</td>\n",
       "      <td>93</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2</td>\n",
       "      <td>Homer's Thoughts</td>\n",
       "      <td>44</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2</td>\n",
       "      <td>Teenage Homer</td>\n",
       "      <td>34</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2</td>\n",
       "      <td>Young Homer</td>\n",
       "      <td>95</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   character_id raw_character_text      0\n",
       "0             0                     17521\n",
       "1             1      Marge Simpson  14157\n",
       "2             1   Marge's Thoughts     23\n",
       "3             1      Teenage Marge     11\n",
       "4             1        Young Marge     72\n",
       "5             2      Homer Simpson  29839\n",
       "6             2      Homer's Brain     93\n",
       "7             2   Homer's Thoughts     44\n",
       "8             2      Teenage Homer     34\n",
       "9             2        Young Homer     95"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby(['character_id', 'raw_character_text']).size().reset_index().head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively, we could use [`pandas.Series.nunique()`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.nunique.html) to count the number of unique `id` values within each group."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>character_id</th>\n",
       "      <th>raw_character_text</th>\n",
       "      <th>id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td></td>\n",
       "      <td>17521</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>Marge Simpson</td>\n",
       "      <td>14157</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>Marge's Thoughts</td>\n",
       "      <td>23</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>Teenage Marge</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>Young Marge</td>\n",
       "      <td>72</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2</td>\n",
       "      <td>Homer Simpson</td>\n",
       "      <td>29839</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2</td>\n",
       "      <td>Homer's Brain</td>\n",
       "      <td>93</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2</td>\n",
       "      <td>Homer's Thoughts</td>\n",
       "      <td>44</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2</td>\n",
       "      <td>Teenage Homer</td>\n",
       "      <td>34</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2</td>\n",
       "      <td>Young Homer</td>\n",
       "      <td>95</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   character_id raw_character_text     id\n",
       "0             0                     17521\n",
       "1             1      Marge Simpson  14157\n",
       "2             1   Marge's Thoughts     23\n",
       "3             1      Teenage Marge     11\n",
       "4             1        Young Marge     72\n",
       "5             2      Homer Simpson  29839\n",
       "6             2      Homer's Brain     93\n",
       "7             2   Homer's Thoughts     44\n",
       "8             2      Teenage Homer     34\n",
       "9             2        Young Homer     95"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby(['character_id', 'raw_character_text'])['id'].nunique().reset_index().head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can rename the column showing the number of script lines (using [`pandas.DataFrame.rename()`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.rename.html)), and assign the resulting `DataFrame` to a new variable, `df_characters`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "df_characters = df.groupby(\n",
    "    ['character_id', 'raw_character_text'])['id'].nunique().reset_index().rename(\n",
    "    columns={'id': 'num_lines'})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>character_id</th>\n",
       "      <th>raw_character_text</th>\n",
       "      <th>num_lines</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td></td>\n",
       "      <td>17521</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>Marge Simpson</td>\n",
       "      <td>14157</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>Marge's Thoughts</td>\n",
       "      <td>23</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>Teenage Marge</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>Young Marge</td>\n",
       "      <td>72</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2</td>\n",
       "      <td>Homer Simpson</td>\n",
       "      <td>29839</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2</td>\n",
       "      <td>Homer's Brain</td>\n",
       "      <td>93</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2</td>\n",
       "      <td>Homer's Thoughts</td>\n",
       "      <td>44</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2</td>\n",
       "      <td>Teenage Homer</td>\n",
       "      <td>34</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2</td>\n",
       "      <td>Young Homer</td>\n",
       "      <td>95</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   character_id raw_character_text  num_lines\n",
       "0             0                         17521\n",
       "1             1      Marge Simpson      14157\n",
       "2             1   Marge's Thoughts         23\n",
       "3             1      Teenage Marge         11\n",
       "4             1        Young Marge         72\n",
       "5             2      Homer Simpson      29839\n",
       "6             2      Homer's Brain         93\n",
       "7             2   Homer's Thoughts         44\n",
       "8             2      Teenage Homer         34\n",
       "9             2        Young Homer         95"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_characters.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can sort the characters based on the number of lines they have (using [`pandas.DataFrame.sort_values()`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.sort_values.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>character_id</th>\n",
       "      <th>raw_character_text</th>\n",
       "      <th>num_lines</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2</td>\n",
       "      <td>Homer Simpson</td>\n",
       "      <td>29839</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td></td>\n",
       "      <td>17521</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>Marge Simpson</td>\n",
       "      <td>14157</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>8</td>\n",
       "      <td>Bart Simpson</td>\n",
       "      <td>13776</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>9</td>\n",
       "      <td>Lisa Simpson</td>\n",
       "      <td>11500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>15</td>\n",
       "      <td>C. Montgomery Burns</td>\n",
       "      <td>3172</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>17</td>\n",
       "      <td>Moe Szyslak</td>\n",
       "      <td>2863</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>3</td>\n",
       "      <td>Seymour Skinner</td>\n",
       "      <td>2443</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>11</td>\n",
       "      <td>Ned Flanders</td>\n",
       "      <td>2145</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49</th>\n",
       "      <td>31</td>\n",
       "      <td>Grampa Simpson</td>\n",
       "      <td>1886</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    character_id   raw_character_text  num_lines\n",
       "5              2        Homer Simpson      29839\n",
       "0              0                           17521\n",
       "1              1        Marge Simpson      14157\n",
       "17             8         Bart Simpson      13776\n",
       "21             9         Lisa Simpson      11500\n",
       "29            15  C. Montgomery Burns       3172\n",
       "32            17          Moe Szyslak       2863\n",
       "10             3      Seymour Skinner       2443\n",
       "25            11         Ned Flanders       2145\n",
       "49            31       Grampa Simpson       1886"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_characters.sort_values('num_lines', ascending=False).head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can exclude the null character by restricting the rows to those for which `character_id>0` or those for which `raw_character_text!=''`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>character_id</th>\n",
       "      <th>raw_character_text</th>\n",
       "      <th>num_lines</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2</td>\n",
       "      <td>Homer Simpson</td>\n",
       "      <td>29839</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>Marge Simpson</td>\n",
       "      <td>14157</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>8</td>\n",
       "      <td>Bart Simpson</td>\n",
       "      <td>13776</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>9</td>\n",
       "      <td>Lisa Simpson</td>\n",
       "      <td>11500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>15</td>\n",
       "      <td>C. Montgomery Burns</td>\n",
       "      <td>3172</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>17</td>\n",
       "      <td>Moe Szyslak</td>\n",
       "      <td>2863</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>3</td>\n",
       "      <td>Seymour Skinner</td>\n",
       "      <td>2443</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>11</td>\n",
       "      <td>Ned Flanders</td>\n",
       "      <td>2145</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49</th>\n",
       "      <td>31</td>\n",
       "      <td>Grampa Simpson</td>\n",
       "      <td>1886</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42</th>\n",
       "      <td>25</td>\n",
       "      <td>Milhouse Van Houten</td>\n",
       "      <td>1862</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    character_id   raw_character_text  num_lines\n",
       "5              2        Homer Simpson      29839\n",
       "1              1        Marge Simpson      14157\n",
       "17             8         Bart Simpson      13776\n",
       "21             9         Lisa Simpson      11500\n",
       "29            15  C. Montgomery Burns       3172\n",
       "32            17          Moe Szyslak       2863\n",
       "10             3      Seymour Skinner       2443\n",
       "25            11         Ned Flanders       2145\n",
       "49            31       Grampa Simpson       1886\n",
       "42            25  Milhouse Van Houten       1862"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_characters[df_characters.character_id>0].sort_values(\n",
    "    'num_lines', ascending=False).head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise B2: Determine the number of episodes in which each character appears."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "More specifically: add a new column to `df_characters`, named `num_episodes`, containing a count of the number of `episode_id` values associated with each `raw_character_text` value.\n",
    "\n",
    "It may be useful to break this down into two steps:\n",
    "* Create a new `DataFrame` containing the number of `episode_id` values associated with each `raw_character_text` value (hint: you might use  [`pandas.Series.nunique()`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.nunique.html) again)\n",
    "* [Merge](http://pandas.pydata.org/pandas-docs/stable/merging.html) that `DataFrame` with the existing `df_characters` (using [`pandas.DataFrame.merge()`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.merge.html))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "raw_character_text\n",
       "                               564\n",
       "\"For Dummies\" Author             1\n",
       "\"Just Stamp the Ticket\" Man      5\n",
       "\"Mario\" #2                       1\n",
       "\"Shorts\" Bart                    1\n",
       "Name: episode_id, dtype: int64"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby('raw_character_text')['episode_id'].nunique().head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As before, we can convert the resulting `Series` object into a `DataFrame` using [`pandas.Series.reset_index()`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.reset_index.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>raw_character_text</th>\n",
       "      <th>episode_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td></td>\n",
       "      <td>564</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>\"For Dummies\" Author</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>\"Just Stamp the Ticket\" Man</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>\"Mario\" #2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>\"Shorts\" Bart</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            raw_character_text  episode_id\n",
       "0                                      564\n",
       "1         \"For Dummies\" Author           1\n",
       "2  \"Just Stamp the Ticket\" Man           5\n",
       "3                   \"Mario\" #2           1\n",
       "4                \"Shorts\" Bart           1"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby('raw_character_text')['episode_id'].nunique().reset_index().head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can verify that the two `DataFrame` objects have the same length (one row per `raw_character_text` value)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(6764, 6764)"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(df_characters), len(df.groupby('raw_character_text')['episode_id'].nunique().reset_index())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can add this new column to `df_characters` using [`pandas.DataFrame.merge()`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.merge.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>character_id</th>\n",
       "      <th>raw_character_text</th>\n",
       "      <th>num_lines</th>\n",
       "      <th>episode_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td></td>\n",
       "      <td>17521</td>\n",
       "      <td>564</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>Marge Simpson</td>\n",
       "      <td>14157</td>\n",
       "      <td>563</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>Marge's Thoughts</td>\n",
       "      <td>23</td>\n",
       "      <td>17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>Teenage Marge</td>\n",
       "      <td>11</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>Young Marge</td>\n",
       "      <td>72</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   character_id raw_character_text  num_lines  episode_id\n",
       "0             0                         17521         564\n",
       "1             1      Marge Simpson      14157         563\n",
       "2             1   Marge's Thoughts         23          17\n",
       "3             1      Teenage Marge         11           3\n",
       "4             1        Young Marge         72           4"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_characters.merge(\n",
    "    df.groupby(\n",
    "        'raw_character_text')['episode_id'].nunique().reset_index(),\n",
    "    on='raw_character_text',\n",
    "    how='left').head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are several ways we could rename the new column (which defaults to `episode_id`, the column selected in the `groupby` call. \n",
    "\n",
    "Here is the way I usually do it, using [`pandas.DataFrame.rename()`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.rename.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>character_id</th>\n",
       "      <th>raw_character_text</th>\n",
       "      <th>num_lines</th>\n",
       "      <th>num_episodes</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td></td>\n",
       "      <td>17521</td>\n",
       "      <td>564</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>Marge Simpson</td>\n",
       "      <td>14157</td>\n",
       "      <td>563</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>Marge's Thoughts</td>\n",
       "      <td>23</td>\n",
       "      <td>17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>Teenage Marge</td>\n",
       "      <td>11</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>Young Marge</td>\n",
       "      <td>72</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   character_id raw_character_text  num_lines  num_episodes\n",
       "0             0                         17521           564\n",
       "1             1      Marge Simpson      14157           563\n",
       "2             1   Marge's Thoughts         23            17\n",
       "3             1      Teenage Marge         11             3\n",
       "4             1        Young Marge         72             4"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_characters.merge(\n",
    "    df.groupby(\n",
    "        'raw_character_text')['episode_id'].nunique().reset_index(),\n",
    "    on='raw_character_text',\n",
    "    how='left').rename(\n",
    "    columns={'episode_id': 'num_episodes'}).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "During the session, participants suggested other ways to do this.\n",
    "\n",
    "One way is to use `rename()` _before_ doing the `reset_index()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>character_id</th>\n",
       "      <th>raw_character_text</th>\n",
       "      <th>num_lines</th>\n",
       "      <th>num_episodes</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td></td>\n",
       "      <td>17521</td>\n",
       "      <td>564</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>Marge Simpson</td>\n",
       "      <td>14157</td>\n",
       "      <td>563</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>Marge's Thoughts</td>\n",
       "      <td>23</td>\n",
       "      <td>17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>Teenage Marge</td>\n",
       "      <td>11</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>Young Marge</td>\n",
       "      <td>72</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   character_id raw_character_text  num_lines  num_episodes\n",
       "0             0                         17521           564\n",
       "1             1      Marge Simpson      14157           563\n",
       "2             1   Marge's Thoughts         23            17\n",
       "3             1      Teenage Marge         11             3\n",
       "4             1        Young Marge         72             4"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_characters.merge(\n",
    "    df.groupby(\n",
    "        'raw_character_text')['episode_id'].nunique().rename(\n",
    "        'num_episodes').reset_index(),\n",
    "    on='raw_character_text',\n",
    "    how='left').head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another approach is to use the `name` argument to [`pandas.Series.reset_index()`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.reset_index.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>character_id</th>\n",
       "      <th>raw_character_text</th>\n",
       "      <th>num_lines</th>\n",
       "      <th>num_episodes</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td></td>\n",
       "      <td>17521</td>\n",
       "      <td>564</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>Marge Simpson</td>\n",
       "      <td>14157</td>\n",
       "      <td>563</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>Marge's Thoughts</td>\n",
       "      <td>23</td>\n",
       "      <td>17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>Teenage Marge</td>\n",
       "      <td>11</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>Young Marge</td>\n",
       "      <td>72</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   character_id raw_character_text  num_lines  num_episodes\n",
       "0             0                         17521           564\n",
       "1             1      Marge Simpson      14157           563\n",
       "2             1   Marge's Thoughts         23            17\n",
       "3             1      Teenage Marge         11             3\n",
       "4             1        Young Marge         72             4"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_characters.merge(\n",
    "    df.groupby(\n",
    "        'raw_character_text')['episode_id'].nunique().reset_index(\n",
    "        name='num_episodes'),\n",
    "    on='raw_character_text',\n",
    "    how='left').head()"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "In any event, we can assign the merged `DataFrame` to `df_characters`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "df_characters = df_characters.merge(\n",
    "    df.groupby(\n",
    "        'raw_character_text')['episode_id'].nunique().reset_index(),\n",
    "    on='raw_character_text',\n",
    "    how='left').rename(\n",
    "    columns={'episode_id': 'num_episodes'})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>character_id</th>\n",
       "      <th>raw_character_text</th>\n",
       "      <th>num_lines</th>\n",
       "      <th>num_episodes</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td></td>\n",
       "      <td>17521</td>\n",
       "      <td>564</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>Marge Simpson</td>\n",
       "      <td>14157</td>\n",
       "      <td>563</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>Marge's Thoughts</td>\n",
       "      <td>23</td>\n",
       "      <td>17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>Teenage Marge</td>\n",
       "      <td>11</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>Young Marge</td>\n",
       "      <td>72</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   character_id raw_character_text  num_lines  num_episodes\n",
       "0             0                         17521           564\n",
       "1             1      Marge Simpson      14157           563\n",
       "2             1   Marge's Thoughts         23            17\n",
       "3             1      Teenage Marge         11             3\n",
       "4             1        Young Marge         72             4"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_characters.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise B3: Determine the first and last episodes in which each character appears"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "More specifically, add the following columns to `df_characters`:\n",
    "* `first_episode_id`: the first (smallest) `episode_id` in which the associated `raw_character_text` value appears\n",
    "* `last_episode_id`: the last (largest) `episode_id` in which the associated `raw_character_text` value appears\n",
    "\n",
    "You can either follow the pattern in the previous exercise - twice - or combine the two into a single grouping (using the [`pandas.core.groupby.DataFrameGroupBy.agg()`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.core.groupby.DataFrameGroupBy.agg.html)) before merging."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>character_id</th>\n",
       "      <th>raw_character_text</th>\n",
       "      <th>num_lines</th>\n",
       "      <th>num_episodes</th>\n",
       "      <th>first_episode_id</th>\n",
       "      <th>last_episode_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td></td>\n",
       "      <td>17521</td>\n",
       "      <td>564</td>\n",
       "      <td>1</td>\n",
       "      <td>568</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>Marge Simpson</td>\n",
       "      <td>14157</td>\n",
       "      <td>563</td>\n",
       "      <td>1</td>\n",
       "      <td>568</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>Marge's Thoughts</td>\n",
       "      <td>23</td>\n",
       "      <td>17</td>\n",
       "      <td>87</td>\n",
       "      <td>561</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>Teenage Marge</td>\n",
       "      <td>11</td>\n",
       "      <td>3</td>\n",
       "      <td>184</td>\n",
       "      <td>510</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>Young Marge</td>\n",
       "      <td>72</td>\n",
       "      <td>4</td>\n",
       "      <td>34</td>\n",
       "      <td>411</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2</td>\n",
       "      <td>Homer Simpson</td>\n",
       "      <td>29839</td>\n",
       "      <td>564</td>\n",
       "      <td>1</td>\n",
       "      <td>568</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2</td>\n",
       "      <td>Homer's Brain</td>\n",
       "      <td>93</td>\n",
       "      <td>47</td>\n",
       "      <td>35</td>\n",
       "      <td>535</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2</td>\n",
       "      <td>Homer's Thoughts</td>\n",
       "      <td>44</td>\n",
       "      <td>26</td>\n",
       "      <td>278</td>\n",
       "      <td>561</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2</td>\n",
       "      <td>Teenage Homer</td>\n",
       "      <td>34</td>\n",
       "      <td>7</td>\n",
       "      <td>68</td>\n",
       "      <td>510</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2</td>\n",
       "      <td>Young Homer</td>\n",
       "      <td>95</td>\n",
       "      <td>10</td>\n",
       "      <td>43</td>\n",
       "      <td>516</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   character_id raw_character_text  num_lines  num_episodes  first_episode_id  \\\n",
       "0             0                         17521           564                 1   \n",
       "1             1      Marge Simpson      14157           563                 1   \n",
       "2             1   Marge's Thoughts         23            17                87   \n",
       "3             1      Teenage Marge         11             3               184   \n",
       "4             1        Young Marge         72             4                34   \n",
       "5             2      Homer Simpson      29839           564                 1   \n",
       "6             2      Homer's Brain         93            47                35   \n",
       "7             2   Homer's Thoughts         44            26               278   \n",
       "8             2      Teenage Homer         34             7                68   \n",
       "9             2        Young Homer         95            10                43   \n",
       "\n",
       "   last_episode_id  \n",
       "0              568  \n",
       "1              568  \n",
       "2              561  \n",
       "3              510  \n",
       "4              411  \n",
       "5              568  \n",
       "6              535  \n",
       "7              561  \n",
       "8              510  \n",
       "9              516  "
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_characters = df_characters.merge(\n",
    "    df.groupby('raw_character_text')['episode_id'].agg(\n",
    "        ['first', 'last']).reset_index().rename(\n",
    "    columns={'first': 'first_episode_id', 'last': 'last_episode_id'}),\n",
    "    on='raw_character_text',\n",
    "    how='left')\n",
    "df_characters.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise B4: Determine the number of seasons, the first season and the last season in which a character appears"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "More specifically, add the following columns to `df_characters`:\n",
    "* `num_seasons`: the number of distinct `seasons` in which the associated `raw_character_text` appears\n",
    "* `first_season`: the first `season` in which the associated `raw_character_text` value appears\n",
    "* `last_episode_id`: the last `season` in which the associated `raw_character_text` value appears\n",
    "\n",
    "You should be able to modify the code you wrote for Exercises 2 and 3 above to address this task."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>character_id</th>\n",
       "      <th>raw_character_text</th>\n",
       "      <th>num_lines</th>\n",
       "      <th>num_episodes</th>\n",
       "      <th>first_episode_id</th>\n",
       "      <th>last_episode_id</th>\n",
       "      <th>num_seasons</th>\n",
       "      <th>first_season</th>\n",
       "      <th>last_season</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td></td>\n",
       "      <td>17521</td>\n",
       "      <td>564</td>\n",
       "      <td>1</td>\n",
       "      <td>568</td>\n",
       "      <td>26</td>\n",
       "      <td>1</td>\n",
       "      <td>26</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>Marge Simpson</td>\n",
       "      <td>14157</td>\n",
       "      <td>563</td>\n",
       "      <td>1</td>\n",
       "      <td>568</td>\n",
       "      <td>26</td>\n",
       "      <td>1</td>\n",
       "      <td>26</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>Marge's Thoughts</td>\n",
       "      <td>23</td>\n",
       "      <td>17</td>\n",
       "      <td>87</td>\n",
       "      <td>561</td>\n",
       "      <td>12</td>\n",
       "      <td>5</td>\n",
       "      <td>26</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>Teenage Marge</td>\n",
       "      <td>11</td>\n",
       "      <td>3</td>\n",
       "      <td>184</td>\n",
       "      <td>510</td>\n",
       "      <td>3</td>\n",
       "      <td>9</td>\n",
       "      <td>24</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>Young Marge</td>\n",
       "      <td>72</td>\n",
       "      <td>4</td>\n",
       "      <td>34</td>\n",
       "      <td>411</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2</td>\n",
       "      <td>Homer Simpson</td>\n",
       "      <td>29839</td>\n",
       "      <td>564</td>\n",
       "      <td>1</td>\n",
       "      <td>568</td>\n",
       "      <td>26</td>\n",
       "      <td>1</td>\n",
       "      <td>26</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2</td>\n",
       "      <td>Homer's Brain</td>\n",
       "      <td>93</td>\n",
       "      <td>47</td>\n",
       "      <td>35</td>\n",
       "      <td>535</td>\n",
       "      <td>15</td>\n",
       "      <td>2</td>\n",
       "      <td>25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2</td>\n",
       "      <td>Homer's Thoughts</td>\n",
       "      <td>44</td>\n",
       "      <td>26</td>\n",
       "      <td>278</td>\n",
       "      <td>561</td>\n",
       "      <td>13</td>\n",
       "      <td>13</td>\n",
       "      <td>26</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2</td>\n",
       "      <td>Teenage Homer</td>\n",
       "      <td>34</td>\n",
       "      <td>7</td>\n",
       "      <td>68</td>\n",
       "      <td>510</td>\n",
       "      <td>7</td>\n",
       "      <td>4</td>\n",
       "      <td>24</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2</td>\n",
       "      <td>Young Homer</td>\n",
       "      <td>95</td>\n",
       "      <td>10</td>\n",
       "      <td>43</td>\n",
       "      <td>516</td>\n",
       "      <td>10</td>\n",
       "      <td>3</td>\n",
       "      <td>24</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   character_id raw_character_text  num_lines  num_episodes  first_episode_id  \\\n",
       "0             0                         17521           564                 1   \n",
       "1             1      Marge Simpson      14157           563                 1   \n",
       "2             1   Marge's Thoughts         23            17                87   \n",
       "3             1      Teenage Marge         11             3               184   \n",
       "4             1        Young Marge         72             4                34   \n",
       "5             2      Homer Simpson      29839           564                 1   \n",
       "6             2      Homer's Brain         93            47                35   \n",
       "7             2   Homer's Thoughts         44            26               278   \n",
       "8             2      Teenage Homer         34             7                68   \n",
       "9             2        Young Homer         95            10                43   \n",
       "\n",
       "   last_episode_id  num_seasons  first_season  last_season  \n",
       "0              568           26             1           26  \n",
       "1              568           26             1           26  \n",
       "2              561           12             5           26  \n",
       "3              510            3             9           24  \n",
       "4              411            4             2           19  \n",
       "5              568           26             1           26  \n",
       "6              535           15             2           25  \n",
       "7              561           13            13           26  \n",
       "8              510            7             4           24  \n",
       "9              516           10             3           24  "
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "column_label = 'season'\n",
    "\n",
    "df_characters = df_characters.merge(\n",
    "    df.groupby('raw_character_text')[column_label].nunique().reset_index().rename(\n",
    "        columns={column_label: 'num_{}s'.format(column_label)}),\n",
    "    on='raw_character_text',\n",
    "    how='left')\n",
    "\n",
    "df_characters = df_characters.merge(\n",
    "    df.groupby('raw_character_text')[column_label].agg(['first', 'last']).reset_index().rename(\n",
    "        columns={'first': 'first_{}'.format(column_label), \n",
    "                 'last': 'last_{}'.format(column_label)}),\n",
    "    on='raw_character_text',\n",
    "    how='left')\n",
    "\n",
    "df_characters.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Your `DataFrame` should look similar to the one below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "df_characters = pydata_simpsons.create_simpsons_characters_dataframe(df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>character_id</th>\n",
       "      <th>raw_character_text</th>\n",
       "      <th>num_lines</th>\n",
       "      <th>num_episodes</th>\n",
       "      <th>first_episode_id</th>\n",
       "      <th>last_episode_id</th>\n",
       "      <th>num_seasons</th>\n",
       "      <th>first_season</th>\n",
       "      <th>last_season</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td></td>\n",
       "      <td>17521</td>\n",
       "      <td>564</td>\n",
       "      <td>1</td>\n",
       "      <td>568</td>\n",
       "      <td>26</td>\n",
       "      <td>1</td>\n",
       "      <td>26</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>Marge Simpson</td>\n",
       "      <td>14157</td>\n",
       "      <td>563</td>\n",
       "      <td>1</td>\n",
       "      <td>568</td>\n",
       "      <td>26</td>\n",
       "      <td>1</td>\n",
       "      <td>26</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>Marge's Thoughts</td>\n",
       "      <td>23</td>\n",
       "      <td>17</td>\n",
       "      <td>87</td>\n",
       "      <td>561</td>\n",
       "      <td>12</td>\n",
       "      <td>5</td>\n",
       "      <td>26</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>Teenage Marge</td>\n",
       "      <td>11</td>\n",
       "      <td>3</td>\n",
       "      <td>184</td>\n",
       "      <td>510</td>\n",
       "      <td>3</td>\n",
       "      <td>9</td>\n",
       "      <td>24</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>Young Marge</td>\n",
       "      <td>72</td>\n",
       "      <td>4</td>\n",
       "      <td>34</td>\n",
       "      <td>411</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2</td>\n",
       "      <td>Homer Simpson</td>\n",
       "      <td>29839</td>\n",
       "      <td>564</td>\n",
       "      <td>1</td>\n",
       "      <td>568</td>\n",
       "      <td>26</td>\n",
       "      <td>1</td>\n",
       "      <td>26</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2</td>\n",
       "      <td>Homer's Brain</td>\n",
       "      <td>93</td>\n",
       "      <td>47</td>\n",
       "      <td>35</td>\n",
       "      <td>535</td>\n",
       "      <td>15</td>\n",
       "      <td>2</td>\n",
       "      <td>25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2</td>\n",
       "      <td>Homer's Thoughts</td>\n",
       "      <td>44</td>\n",
       "      <td>26</td>\n",
       "      <td>278</td>\n",
       "      <td>561</td>\n",
       "      <td>13</td>\n",
       "      <td>13</td>\n",
       "      <td>26</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2</td>\n",
       "      <td>Teenage Homer</td>\n",
       "      <td>34</td>\n",
       "      <td>7</td>\n",
       "      <td>68</td>\n",
       "      <td>510</td>\n",
       "      <td>7</td>\n",
       "      <td>4</td>\n",
       "      <td>24</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2</td>\n",
       "      <td>Young Homer</td>\n",
       "      <td>95</td>\n",
       "      <td>10</td>\n",
       "      <td>43</td>\n",
       "      <td>516</td>\n",
       "      <td>10</td>\n",
       "      <td>3</td>\n",
       "      <td>24</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   character_id raw_character_text  num_lines  num_episodes  first_episode_id  \\\n",
       "0             0                         17521           564                 1   \n",
       "1             1      Marge Simpson      14157           563                 1   \n",
       "2             1   Marge's Thoughts         23            17                87   \n",
       "3             1      Teenage Marge         11             3               184   \n",
       "4             1        Young Marge         72             4                34   \n",
       "5             2      Homer Simpson      29839           564                 1   \n",
       "6             2      Homer's Brain         93            47                35   \n",
       "7             2   Homer's Thoughts         44            26               278   \n",
       "8             2      Teenage Homer         34             7                68   \n",
       "9             2        Young Homer         95            10                43   \n",
       "\n",
       "   last_episode_id  num_seasons  first_season  last_season  \n",
       "0              568           26             1           26  \n",
       "1              568           26             1           26  \n",
       "2              561           12             5           26  \n",
       "3              510            3             9           24  \n",
       "4              411            4             2           19  \n",
       "5              568           26             1           26  \n",
       "6              535           15             2           25  \n",
       "7              561           13            13           26  \n",
       "8              510            7             4           24  \n",
       "9              516           10             3           24  "
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_characters.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>season</th>\n",
       "      <th>episode_id</th>\n",
       "      <th>number</th>\n",
       "      <th>raw_text</th>\n",
       "      <th>timestamp_in_ms</th>\n",
       "      <th>speaking_line</th>\n",
       "      <th>character_id</th>\n",
       "      <th>location_id</th>\n",
       "      <th>raw_character_text</th>\n",
       "      <th>raw_location_text</th>\n",
       "      <th>spoken_words</th>\n",
       "      <th>normalized_text</th>\n",
       "      <th>word_count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>(Street: ext. street - establishing - night)</td>\n",
       "      <td>8000</td>\n",
       "      <td>False</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td></td>\n",
       "      <td>Street</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>(Car: int. car - night)</td>\n",
       "      <td>8000</td>\n",
       "      <td>False</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td></td>\n",
       "      <td>Car</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Marge Simpson: Ooo, careful, Homer.</td>\n",
       "      <td>8000</td>\n",
       "      <td>True</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Marge Simpson</td>\n",
       "      <td>Car</td>\n",
       "      <td>Ooo, careful, Homer.</td>\n",
       "      <td>ooo careful homer</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Homer Simpson: There's no time to be careful.</td>\n",
       "      <td>10000</td>\n",
       "      <td>True</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>Homer Simpson</td>\n",
       "      <td>Car</td>\n",
       "      <td>There's no time to be careful.</td>\n",
       "      <td>theres no time to be careful</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>Homer Simpson: We're late.</td>\n",
       "      <td>10000</td>\n",
       "      <td>True</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>Homer Simpson</td>\n",
       "      <td>Car</td>\n",
       "      <td>We're late.</td>\n",
       "      <td>were late</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id  season  episode_id  number  \\\n",
       "0   1       1           1       0   \n",
       "1   2       1           1       1   \n",
       "2   3       1           1       2   \n",
       "3   4       1           1       3   \n",
       "4   5       1           1       4   \n",
       "\n",
       "                                        raw_text  timestamp_in_ms  \\\n",
       "0   (Street: ext. street - establishing - night)             8000   \n",
       "1                        (Car: int. car - night)             8000   \n",
       "2            Marge Simpson: Ooo, careful, Homer.             8000   \n",
       "3  Homer Simpson: There's no time to be careful.            10000   \n",
       "4                     Homer Simpson: We're late.            10000   \n",
       "\n",
       "   speaking_line  character_id  location_id raw_character_text  \\\n",
       "0          False             0            1                      \n",
       "1          False             0            2                      \n",
       "2           True             1            2      Marge Simpson   \n",
       "3           True             2            2      Homer Simpson   \n",
       "4           True             2            2      Homer Simpson   \n",
       "\n",
       "  raw_location_text                    spoken_words  \\\n",
       "0            Street                                   \n",
       "1               Car                                   \n",
       "2               Car            Ooo, careful, Homer.   \n",
       "3               Car  There's no time to be careful.   \n",
       "4               Car                     We're late.   \n",
       "\n",
       "                normalized_text  word_count  \n",
       "0                                         0  \n",
       "1                                         0  \n",
       "2             ooo careful homer           3  \n",
       "3  theres no time to be careful           6  \n",
       "4                     were late           2  "
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise Set C"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have `df_characters` constructed, we can do some additional analysis (more easily)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise C1: What are the minimum, maximum and mean number of script lines per character?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "min         1.000000\n",
       "mean       23.395476\n",
       "max     29839.000000\n",
       "Name: num_lines, dtype: float64"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_characters['num_lines'].agg(['min', 'mean', 'max'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Unevenly spaced time series analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "As noted earlier (from the Wikipedia article on Time series):\n",
    "\n",
    "> Most commonly, a time series is a sequence taken at successive equally spaced points in time\n",
    "\n",
    "Unevenly spaced time series analysis involves data points that are spaced over a sequence of unequal time intervals. The sizes of the intervals themselves are often a focus of such analysis.\n",
    "\n",
    "Examples of such analyses I've done at Indeed include\n",
    "* identifying \"repeat business\" from Indeed Hire clients, based on different definitions of \"repeat\", by analyzing the intervals between the dates on which new job requisitions from the same client were created\n",
    "* determining how much time Indeed Hire recruiters spend reviewing candidate resumes and cover letters, by analyzing the intervals between candidate profile dispositions\n",
    "\n",
    "Within the context of this tutorial, we'll restrict our focus to unevenly spaced time series analysis of The Simpsons."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Identify scene changes (an unevenly spaced time series analysis)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One potential indicator of scene changes is changes of location.\n",
    "\n",
    "One way to identify changes of location is to create a new column with the location id of the previous script line and then look for rows in which the previous location id is different from the [current] location id."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is important to remember that there is noise in the data, including script lines for which no location information is available. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(407, 407, 407)"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(len(df[df.location_id==0]), \n",
    " len(df[df.raw_location_text=='']), \n",
    " len(df[(df.location_id==0) & (df.raw_location_text=='')]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>season</th>\n",
       "      <th>episode_id</th>\n",
       "      <th>number</th>\n",
       "      <th>raw_text</th>\n",
       "      <th>timestamp_in_ms</th>\n",
       "      <th>speaking_line</th>\n",
       "      <th>character_id</th>\n",
       "      <th>location_id</th>\n",
       "      <th>raw_character_text</th>\n",
       "      <th>raw_location_text</th>\n",
       "      <th>spoken_words</th>\n",
       "      <th>normalized_text</th>\n",
       "      <th>word_count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>4576</th>\n",
       "      <td>4588</td>\n",
       "      <td>2</td>\n",
       "      <td>16</td>\n",
       "      <td>0</td>\n",
       "      <td>Marge Simpson: (CLEARING HER THROAT) Hello, ev...</td>\n",
       "      <td>5000</td>\n",
       "      <td>True</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Marge Simpson</td>\n",
       "      <td></td>\n",
       "      <td>Hello, everyone. You know, Halloween is a very...</td>\n",
       "      <td>hello everyone you know halloween is a very st...</td>\n",
       "      <td>83</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12759</th>\n",
       "      <td>12786</td>\n",
       "      <td>3</td>\n",
       "      <td>43</td>\n",
       "      <td>0</td>\n",
       "      <td>Homer-Ape: (INSPIRED) Oooh.</td>\n",
       "      <td>37000</td>\n",
       "      <td>True</td>\n",
       "      <td>625</td>\n",
       "      <td>0</td>\n",
       "      <td>Homer-Ape</td>\n",
       "      <td></td>\n",
       "      <td>Oooh.</td>\n",
       "      <td>oooh</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12760</th>\n",
       "      <td>12787</td>\n",
       "      <td>3</td>\n",
       "      <td>43</td>\n",
       "      <td>1</td>\n",
       "      <td>Homer-Ape: (ADJUSTING HIMSELF) Ahhh.</td>\n",
       "      <td>95000</td>\n",
       "      <td>True</td>\n",
       "      <td>625</td>\n",
       "      <td>0</td>\n",
       "      <td>Homer-Ape</td>\n",
       "      <td></td>\n",
       "      <td>Ahhh.</td>\n",
       "      <td>ahhh</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14586</th>\n",
       "      <td>14613</td>\n",
       "      <td>3</td>\n",
       "      <td>49</td>\n",
       "      <td>0</td>\n",
       "      <td>Bret: Hey everybody, Bret Gunsilman here in pi...</td>\n",
       "      <td>35000</td>\n",
       "      <td>True</td>\n",
       "      <td>704</td>\n",
       "      <td>0</td>\n",
       "      <td>Bret</td>\n",
       "      <td></td>\n",
       "      <td>Hey everybody, Bret Gunsilman here in pivotal ...</td>\n",
       "      <td>hey everybody bret gunsilman here in pivotal w...</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15222</th>\n",
       "      <td>15249</td>\n",
       "      <td>3</td>\n",
       "      <td>51</td>\n",
       "      <td>0</td>\n",
       "      <td>Teenager: Hey, what gives?</td>\n",
       "      <td>30000</td>\n",
       "      <td>True</td>\n",
       "      <td>628</td>\n",
       "      <td>0</td>\n",
       "      <td>Teenager</td>\n",
       "      <td></td>\n",
       "      <td>Hey, what gives?</td>\n",
       "      <td>hey what gives</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          id  season  episode_id  number  \\\n",
       "4576    4588       2          16       0   \n",
       "12759  12786       3          43       0   \n",
       "12760  12787       3          43       1   \n",
       "14586  14613       3          49       0   \n",
       "15222  15249       3          51       0   \n",
       "\n",
       "                                                raw_text  timestamp_in_ms  \\\n",
       "4576   Marge Simpson: (CLEARING HER THROAT) Hello, ev...             5000   \n",
       "12759                        Homer-Ape: (INSPIRED) Oooh.            37000   \n",
       "12760               Homer-Ape: (ADJUSTING HIMSELF) Ahhh.            95000   \n",
       "14586  Bret: Hey everybody, Bret Gunsilman here in pi...            35000   \n",
       "15222                         Teenager: Hey, what gives?            30000   \n",
       "\n",
       "       speaking_line  character_id  location_id raw_character_text  \\\n",
       "4576            True             1            0      Marge Simpson   \n",
       "12759           True           625            0          Homer-Ape   \n",
       "12760           True           625            0          Homer-Ape   \n",
       "14586           True           704            0               Bret   \n",
       "15222           True           628            0           Teenager   \n",
       "\n",
       "      raw_location_text                                       spoken_words  \\\n",
       "4576                     Hello, everyone. You know, Halloween is a very...   \n",
       "12759                                                                Oooh.   \n",
       "12760                                                                Ahhh.   \n",
       "14586                    Hey everybody, Bret Gunsilman here in pivotal ...   \n",
       "15222                                                     Hey, what gives?   \n",
       "\n",
       "                                         normalized_text  word_count  \n",
       "4576   hello everyone you know halloween is a very st...          83  \n",
       "12759                                               oooh           1  \n",
       "12760                                               ahhh           1  \n",
       "14586  hey everybody bret gunsilman here in pivotal w...          14  \n",
       "15222                                     hey what gives           3  "
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[df.location_id==0].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As with character data, some `location_id` values are associated with more than one `raw_location_text` value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(4460, 4499, 4492)"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(df.location_id.nunique(), \n",
    " df.raw_location_text.nunique(), \n",
    " df.raw_location_text.str.lower().nunique())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Simpson Home                       35056\n",
       "Springfield Elementary School       7092\n",
       "Moe's Tavern                        4626\n",
       "Springfield Nuclear Power Plant     3593\n",
       "Kwik-E-Mart                         1476\n",
       "Name: raw_location_text, dtype: int64"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[df.location_id>0].raw_location_text.value_counts().head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DUMP                 1\n",
       "MISTY FOREST         1\n",
       "Vermont              1\n",
       "PLYMOUTH, ENGLAND    1\n",
       "SPRINGFIELD - DAY    1\n",
       "Name: raw_location_text, dtype: int64"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[df.location_id>0].raw_location_text.value_counts().tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the context of this tutorial, we will simply forge ahead, noting that the results below are, at best, approximate."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise Set D"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise D1: Add a column, `prev_location_id`, to contain the previous  `location_id` for each script line within a given season"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The [`pandas.DataFrame.shift()`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.shift.html) function can be used to return the rows of a `DataFrame` shifted by some offset (the default offset is 1 row). Positive offsets can be interpreted as shifting \"down\", which allows us to easily access previous rows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    1\n",
       "1    2\n",
       "2    2\n",
       "3    2\n",
       "4    2\n",
       "Name: location_id, dtype: int64"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.location_id.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    NaN\n",
       "1    1.0\n",
       "2    2.0\n",
       "3    2.0\n",
       "4    2.0\n",
       "Name: location_id, dtype: float64"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.location_id.shift().head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also use negative offset values to shift \"up\" for next rows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    2.0\n",
       "1    2.0\n",
       "2    2.0\n",
       "3    2.0\n",
       "4    3.0\n",
       "Name: location_id, dtype: float64"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.location_id.shift(-1).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll want to first group by episode."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    NaN\n",
       "1    1.0\n",
       "2    2.0\n",
       "3    2.0\n",
       "4    2.0\n",
       "Name: location_id, dtype: float64"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby(['episode_id'])['location_id'].shift().head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And to keep things consistent with the `location_id` field, we can substitute zero for null (`NaN`) values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    0\n",
       "1    1\n",
       "2    2\n",
       "3    2\n",
       "4    2\n",
       "Name: location_id, dtype: int64"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby('episode_id').location_id.shift().fillna(0).astype('int64').head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "df.loc[:, 'prev_location_id'] = df.groupby(\n",
    "    'episode_id').location_id.shift().fillna(0).astype('int64')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>location_id</th>\n",
       "      <th>prev_location_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   location_id  prev_location_id\n",
       "0            1                 0\n",
       "1            2                 1\n",
       "2            2                 2\n",
       "3            2                 2\n",
       "4            2                 2"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[['location_id', 'prev_location_id']].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise D2: How many scene changes are there across all episodes?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we conveniently ignore the fact that we have rows with missing locations (where `location_id==0`), we can compute this by simply counting the number of rows where the `location_id` and `prev_location_id` differ."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "17122"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(df[df.location_id != df.prev_location_id])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Most script lines take place within the same scene (location)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "141125"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(df[df.location_id == df.prev_location_id])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}