cord19kg/examples/notebooks/Glucose is a risk facor for COVID-19 (3000 papers).ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prerequisites and installation instructions for Colab"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you want to run this notebook in Google Colab, run the following cell. Otherwise follow [installation instructions](https://github.com/BlueBrain/BlueGraph/blob/master/README.rst#installation) to install BlueGraph and its dependencies locally."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Install bluegraph\n",
    "! git clone https://github.com/BlueBrain/BlueGraph\n",
    "! cd BlueGraph && pip install .[cord19kg]\n",
    "\n",
    "# Install graph-tool using condacolab\n",
    "!pip install -q condacolab\n",
    "import condacolab\n",
    "condacolab.install()\n",
    "!conda install -c conda-forge graph-tool==2.58\n",
    "\n",
    "\"\"\"\n",
    "# Install graph-tool on Debian & Ubuntu\n",
    "!echo \"deb [ arch=amd64 ] https://downloads.skewed.de/apt bionic main\" >> /etc/apt/sources.list\n",
    "!apt-key adv --keyserver keys.openpgp.org --recv-key 612DEFB798507F25\n",
    "!apt-get update\n",
    "!apt-get install python3-graph-tool=2.37 python3-cairo python3-matplotlib\n",
    "\"\"\"\n",
    "\n",
    "DATA_PATH = \"BlueGraph/cord19kg/examples/data/\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prerequisites and installation instructions for a local environment\n",
    "\n",
    "In order to run this notebook `graph-tool` must be installed manually (it cannot be installed as a part of `pip install bluegraph`, as it is not an ordinary Python library, but a wrapper around a C++ library). Please, see [graph-tool installation instructions](https://git.skewed.de/count0/graph-tool/-/wikis/installation-instructions#native-installation) (currently, BlueGraph supports :code:`graph-tool<=2.37`.)\n",
    "\n",
    "We recommend using `conda` for installing `graph-tool`. For example:\n",
    "\n",
    "```\n",
    "conda install -c conda-forge graph-tool==2.37\n",
    "```\n",
    "\n",
    "or as a part of a new `conda` environment:\n",
    "\n",
    "```\n",
    "conda create --name <your_environment> -c conda-forge graph-tool==2.37\n",
    "conda activate <your_environment>\n",
    "```\n",
    "\n",
    "\n",
    "BlueGraph and the set of dependecies supporting custom tools for CORD-19 analysis can be installed using:\n",
    "\n",
    " ```\n",
    " pip install bluegraph[cord19kg]\n",
    " ```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Topic-centered co-occurrence network analysis of CORD-19\n",
    "\n",
    "In this notebook we will perform interactive exploration and analysis of a topic-centered subset of the [CORD-19](https://www.kaggle.com/allen-institute-for-ai/CORD-19-research-challenge) dataset using the `cord19kg` package. The exploration and analysis techniques presented here focus on named entities and their co-occurrence in the scientific articles constituting the dataset.\n",
    "\n",
    "The input data for this notebook contains the named entities extracted from the 3000 most relevant articles to the query _\"Glucose is a risk factor of COVID-19\"_ obtained using the article search model [BlueSearch](https://github.com/BlueBrain/Search). The entity extraction was performed using the Named Entity Recognition (NER) techniques also included in [BlueSearch](https://github.com/BlueBrain/Search). The entities represent 10 different types (i.e. proteins, chemicals, drugs, diseases, conditions, organs, organisms, pathways, cell types, cell compartments). \n",
    "\n",
    "The interactive literature exploration through the named entity co-occurrence analysis consisting of the following steps:\n",
    "\n",
    "1. __Data preparation__ step converts raw mentions into aggregated entity occurrence statistics.\n",
    "2. __Data curation__ step allows the user to manage extracted entities: modify, filter them and link to the ontology.\n",
    "3. __Network generation__ step allows creating entity co-occurrence networks based on paper-, section- and paragraph-level co-occurrence relations between entities. These entity relations are quantified using mutual-information-based scores (pointwise mutual information and its normalized version).\n",
    "4. __Network visualization and analysis__ step allows the user to perform interactive network visualization, edit network elements and perform its analysis (spanning tree, mutual-information based shortest paths between entities, etc)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "import os\n",
    "import zipfile\n",
    "\n",
    "import pandas as pd\n",
    "\n",
    "import dash_cytoscape as cyto\n",
    "\n",
    "from kgforge.core import KnowledgeGraphForge\n",
    "\n",
    "from cord19kg.utils import (generate_curation_table,\n",
    "                           link_ontology,\n",
    "                           generate_cooccurrence_analysis,\n",
    "                           download_from_nexus)\n",
    "from cord19kg.apps.curation_app import curation_app\n",
    "from cord19kg.apps.visualization_app import visualization_app"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "try:\n",
    "    print(f\"Data path: '{DATA_PATH}'\")\n",
    "except NameError:\n",
    "    DATA_PATH = \"../data/\"\n",
    "    print(f\"Data path: '{DATA_PATH}'\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cyto.load_extra_layouts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Data preparation\n",
    "\n",
    "The input dataset contains occurrences of different terms in paragraphs of scientific articles from the CORD-19 dataset previously extracted by means of a NER model.\n",
    "The dataset is stored in Blue Brain Nexus."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Blue Brain Nexus bucket to download data from\n",
    "\n",
    "nexus_bucket = \"covid19-kg/data\"\n",
    "nexus_endpoint = \"https://bbp.epfl.ch/nexus/v1\"\n",
    "nexus_config_file = f\"{DATA_PATH}../config/data-download-nexus.yml\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "download_from_nexus(\n",
    "    uri=f\"{nexus_endpoint}/resources/{nexus_bucket}/_/1e01e1a2-133f-4833-9fe0-93230384b95f\",\n",
    "    output_path=DATA_PATH, config_file_path=nexus_config_file,\n",
    "    nexus_endpoint=nexus_endpoint, nexus_bucket=nexus_bucket, unzip=True)\n",
    "\n",
    "data = pd.read_csv(f\"{DATA_PATH}/Glucose_risk_3000_papers.csv\")\n",
    "print(\"Done.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.sample(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "On the first preparation step, we group and aggregate the input data by unique entities."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "print(\"Prepating curatation data...\")\n",
    "curation_input_table, factor_counts = generate_curation_table(data)\n",
    "print(\"Done.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The resulting dataframe contains a row per unique named entity together with the following occurrence data: \n",
    "- sets of unique paragraphs, papers, sections, where the corresponding entity is mentioned (`paper`, `section`, `paragraph` columns);\n",
    "- number of total entity occurrences (the `raw_frequency` column);\n",
    "- number of unique papers where it occurs (the `paper_frequency` column);\n",
    "- unique entity types assigned by the NER model (the `entity_type` column, multiple types are possible).\n",
    "- raw entity types assigned by the NER model with the multiplicity of thier occurrence (the `raw_entity_types` column).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "curation_input_table.sample(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The second output of the data preparation step outputs the counts of different instances of occurrence factors: number of distinct papers/sections/paragraphs in the input corpus."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "factor_counts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Data curation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loading the NCIT ontology linking data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To group synonymical entities in the previously extracted table (e.g. `ace2`, `ace-2`, `angiotensin-converting enzyme 2`), as well as assign additional semantics to these entities (e.g. human-readable definition, taxonomy, etc), we peform further _linking_ of the entities to the terms from the [NCIT ontology](https://ncithesaurus.nci.nih.gov/ncitbrowser/).\n",
    "\n",
    "To be able to perform such ontology linking, we load some additional (pre-computed using ML-based linking models) data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "print(\"Loading the ontology linking data...\")\n",
    "\n",
    "download_from_nexus(\n",
    "    uri=f\"{nexus_endpoint}/resources/{nexus_bucket}/_/4fde1f8f-ee7f-435e-95a8-abb79139db93\",\n",
    "    output_path=DATA_PATH, config_file_path=nexus_config_file,\n",
    "    nexus_endpoint=nexus_endpoint, nexus_bucket=nexus_bucket, unzip=True)\n",
    "print(\"\\tLoading the linking dataframe in memory...\")\n",
    "ontology_linking = pd.read_csv(f\"{DATA_PATH}/NCIT_ontology_linking_3000_papers.csv\")\n",
    "\n",
    "\n",
    "print(\"\\tLoading ontology type mapping...\")\n",
    "ontology_linking_type_mapping_data = download_from_nexus(\n",
    "    uri=f\"{nexus_endpoint}/resources/{nexus_bucket}/_/92bc2a04-6003-4f4d-85e1-dcc5f2352df2\", \n",
    "    output_path=DATA_PATH, config_file_path=nexus_config_file,\n",
    "    nexus_endpoint=nexus_endpoint, nexus_bucket=nexus_bucket)\n",
    "with open(f\"{DATA_PATH}/{ontology_linking_type_mapping_data.distribution.name}\", \"rb\") as f:\n",
    "    type_mapping = json.load(f)\n",
    "print(\"Done.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The ontology linking table contains the following columns:\n",
    "- `mention` entity mentioned in the text\n",
    "- `concept` ontology concept linked to the entity mention\n",
    "- `uid` unique identifier of the ontology concept\n",
    "- `definition` definition of the concept\n",
    "- `taxonomy` taxonomy of semantic types associated with the concept"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "ontology_linking.sample(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Interactive curation of  entity occurrence data\n",
    "\n",
    "The package provides an interactive entity curation app that allows the user to visualize the entity occurrence data, modify it, perform ontology linking (see `Link to NCIT ontology` button), filter short or unfrequent entities.\n",
    "\n",
    "The field `Keep` allows specifying a set of entities that must be kept in the dataset at all times (even if they don't satisfy the selected filtering criteria).\n",
    "\n",
    "Finally the value specified in the `Generate Graphs from top N frequent entities` field corresponds to the number of top entities (by the frequency of their occurrence in papers) to be included in the co-occurrence network."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We load the prepared data table into the curation app as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "curation_app.set_table(curation_input_table.copy())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can specify the default entities to keep."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "default_entities_to_keep = [\"glucose\", \"covid-19\"]\n",
    "curation_app.set_default_terms_to_include(default_entities_to_keep)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we set the ontology linking callback to be fired upon a click on the `Link to NCIT ontology` button."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "curation_app.set_ontology_linking_callback(lambda x: link_ontology(ontology_linking, type_mapping, x))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Launch the curation app"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The application can be launched either inline (inside the current notebook) as below. Note that if you run this notebook in Colab, you may want to set a lower number of entities to include, in order to avoid long generation time ('Generate Graphs from top N frequent entities' field). Current default value is 200."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "curation_app.run(port=8074, mode=\"inline\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or it can be opened externally (by the URL that you can open in a separate tab of your browser, try uncommenting, executing and doing Ctrl+Click on the displayed URL)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# curation_app.run(port=8070, mode=\"external\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Co-occurrence network generation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Current curation table displayed in the curation app can be extracted using the `get_curated_table` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "curated_occurrence_data = curation_app.get_curated_table()\n",
    "curated_occurrence_data.sample(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we can proceed we need to convert paper/section and paragraph columns into `set`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "curated_occurrence_data[\"paper\"] = curated_occurrence_data[\"paper\"].apply(set)\n",
    "curated_occurrence_data[\"paragraph\"] = curated_occurrence_data[\"paragraph\"].apply(set)\n",
    "curated_occurrence_data[\"section\"] = curated_occurrence_data[\"section\"].apply(set)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also retreive current values of the `Keep` field (these entities will be also included in the resulting co-occurrence network)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "curation_app.get_terms_to_include()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Generating co-occurrence networks\n",
    "\n",
    "In the cell below we generate a paper-based entity co-occurrence network. Along with the network generation the `generate_comention_analysis` function:\n",
    "\n",
    "- computes node centrality metrics (such as degree, RageRank)\n",
    "- computes co-occurrence statistics (such as frequency, pointwise mutual information and normalized pointwise mutual information) and assignes them as weights to the corresponding edges\n",
    "- performs entity community detection based on different co-occurrence statistics\n",
    "- computes mutual-information-based minimum spanning trees."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we run the co-occurrence analysis, we will create a dictionary with backend configurations for the analytics: we set metrics (centalities) computation to use `graph_tool`, community detection to use `networkx` and, finally, path search to use `graph_tool` as well."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "backend_configs = {\n",
    "    \"metrics\": \"graph_tool\",\n",
    "    \"communities\": \"networkx\",\n",
    "    \"paths\": \"graph_tool\"\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "type_data = curated_occurrence_data[[\"entity_type\"]].rename(columns={\"entity_type\": \"type\"})\n",
    "\n",
    "graphs, trees = generate_cooccurrence_analysis(\n",
    "    curated_occurrence_data,  factor_counts,\n",
    "    n_most_frequent=curation_app.n_most_frequent,\n",
    "    type_data=type_data, \n",
    "    factors=[\"paragraph\"],\n",
    "    keep=curation_app.get_terms_to_include(),\n",
    "    cores=8,  # here set up the number of cores\n",
    "    backend_configs=backend_configs)\n",
    "print(\"Done.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Network visualization and analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loading the generated graph into the visualization app"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First of all, we set a backend for the visualization app (currently two backends are supported: based on `NetworkX` and `graph-tool`, in this example we use the latter)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "visualization_app.set_backend(\"graph_tool\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# #  Run the following use NetworkX as the backend for the visualization app\n",
    "# visualization_app.set_backend(\"networkx\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "visualization_app.add_graph(\n",
    "    \"Paragraph-based graph\", graphs[\"paragraph\"],\n",
    "    tree=trees[\"paragraph\"], default_top_n=100)\n",
    "\n",
    "visualization_app.set_current_graph(\"Paragraph-based graph\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loading papers' meta-data into the app\n",
    "\n",
    "We now load an additional dataset containing some meta-data on the papers where the entities analyzed in this notebook occur."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "paper_medata = download_from_nexus(uri=f\"{nexus_endpoint}/resources/{nexus_bucket}/_/8fc1e60c-1ebe-4173-82c0-9775a4917041\",\n",
    "                         output_path = DATA_PATH, config_file_path=nexus_config_file, nexus_endpoint=nexus_endpoint, nexus_bucket=nexus_bucket)\n",
    "paper_data = pd.read_csv(f\"{DATA_PATH}/{paper_medata.distribution.name}\")\n",
    "paper_data = paper_data.set_index(\"id\")\n",
    "paper_data.head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We pass a callback for the lookup of paper meta-data to the visualization app using the `set_list_papers_callback` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def list_papers(paper_data, selected_papers, limit=200):\n",
    "    selected_paper_data = paper_data.loc[[int(p) for p in selected_papers]].head(200)\n",
    "    return selected_paper_data.to_dict(\"records\")\n",
    "\n",
    "visualization_app.set_list_papers_callback(lambda x: list_papers(paper_data, x))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The ontology linking process described above is noisy, therefore, we would like to keep a possibility of accessing, the raw entities that were linked to particular ontology concepts. For this we define the function `get_aggregated_entities` that retreives such raw entities and we pass it to the visualization app using the `set_aggregated_entities_callback` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def top_n(data_dict, n, smallest=False):\n",
    "    \"\"\"Return top `n` keys of the input dictionary by their value.\"\"\"\n",
    "    df = pd.DataFrame(dict(data_dict).items(), columns=[\"id\", \"value\"])\n",
    "    if smallest:\n",
    "        df = df.nsmallest(n, columns=[\"value\"])\n",
    "    else:\n",
    "        df = df.nlargest(n, columns=[\"value\"])\n",
    "    return(list(df[\"id\"]))\n",
    "\n",
    "\n",
    "def get_aggregated_entities(entity, n):\n",
    "    if \"aggregated_entities\" in curated_occurrence_data.columns:\n",
    "        aggregated = curated_occurrence_data.loc[entity][\"aggregated_entities\"]\n",
    "    else:\n",
    "        aggregated = [entity]\n",
    "    if curation_input_table is not None:\n",
    "        df = curation_input_table.set_index(\"entity\")\n",
    "        if entity in curated_occurrence_data.index:\n",
    "            freqs = df.loc[aggregated][\"paper_frequency\"].to_dict()\n",
    "        else:\n",
    "            return {}\n",
    "    else:\n",
    "        df = data.copy()\n",
    "        df[\"entity\"] = data[\"entity\"].apply(lambda x: x.lower())\n",
    "        freqs = df[df[\"entity\"].apply(lambda x: x.lower() in aggregated)].groupby(\"entity\").aggregate(\n",
    "            lambda x: len(x))[\"entity_type\"].to_dict()\n",
    "    if len(freqs) == 0:\n",
    "        return {}\n",
    "    return {e: freqs[e] for e in top_n(freqs, n)}\n",
    "\n",
    "visualization_app.set_aggregated_entities_callback(\n",
    "    lambda x: get_aggregated_entities(x, 10))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we create a dictionary `definitions` that will serve the visualization app as the lookup table for accessing the definitions of different ontology concepts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "definitions = ontology_linking[[\"concept\", \"definition\"]].groupby(\n",
    "    \"concept\").aggregate(lambda x: list(x)[0]).to_dict()[\"definition\"]\n",
    "visualization_app.set_entity_definitons(definitions)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Launching the visualization app"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As before, the interactive graph visualization app can be launched in two modes: inline and external. Here we recommend the external mode for better user experience."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "visualization_app.run(port=8081, mode=\"external\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "bg",
   "language": "python",
   "name": "bg"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}