Copia_de_malaria_detection_with_fastai_v2.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "version": "3.6.6",
      "mimetype": "text/x-python",
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "pygments_lexer": "ipython3",
      "nbconvert_exporter": "python",
      "file_extension": ".py"
    },
    "colab": {
      "name": "Copia de malaria-detection-with-fastai-v2.ipynb",
      "provenance": [],
      "include_colab_link": true
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/danielsoy/data/blob/main/Copia_de_malaria_detection_with_fastai_v2.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_uuid": "94b4605e05a89d3a06778dfecfbd894a835341ee",
        "id": "dA77L-7Q-v2W"
      },
      "source": [
        "# Malaria Detection\n",
        "\n",
        "Malaria is a life-threatening disease caused by parasites that are transmitted to people through the bites of infected female Anopheles mosquitoes. It is preventable and curable.\n",
        "* In 2017, there were an estimated 219 million cases of malaria in 90 countries.\n",
        "* Malaria deaths reached 435 000 in 2017.\n",
        "* The WHO African Region carries a disproportionately high share of the global malaria burden. In 2017, the region was home to 92% of malaria cases and 93% of malaria deaths.\n",
        "\n",
        "Malaria is caused by Plasmodium parasites. The parasites are spread to people through the bites of infected female Anopheles mosquitoes, called \"malaria vectors.\" There are 5 parasite species that cause malaria in humans, and 2 of these species – P. falciparum and P. vivax – pose the greatest threat.\n",
        "\n",
        "\n",
        "***Diagnosis of malaria can be difficult***:\n",
        "\n",
        "* Where malaria is not endemic any more (such as in the United States), health-care providers may not be familiar with the disease. Clinicians seeing a malaria patient may forget to consider malaria among the potential diagnoses and not order the needed diagnostic tests. Laboratorians may lack experience with malaria and fail to detect parasites when examining blood smears under the microscope.\n",
        "* Malaria is an acute febrile illness. In a non-immune individual, symptoms usually appear 10–15 days after the infective mosquito bite. The first symptoms – fever, headache, and chills – may be mild and difficult to recognize as malaria. If not treated within 24 hours, P. falciparum malaria can progress to severe illness, often leading to death.\n",
        "\n",
        "***Microscopic Diagnosis***\n",
        "\n",
        "Malaria parasites can be identified by examining under the microscope a drop of the patient’s blood, spread out as a “blood smear” on a microscope slide. Prior to examination, the specimen is stained  to give the parasites a distinctive appearance. This technique remains the gold standard for laboratory confirmation of malaria. However, it depends on the quality of the reagents, of the microscope, and on the experience of the laboratorian.\n",
        "\n",
        "![malaria](https://cdn1.sph.harvard.edu/wp-content/uploads/2015/03/Malaria-cells_CDC.jpg)\n",
        "\n",
        "references :\n",
        "* [1] https://www.who.int/news-room/fact-sheets/detail/malaria \n",
        "* [2] https://www.cdc.gov/malaria/diagnosis_treatment/diagnosis.html \n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "61VUBLfD-v2c"
      },
      "source": [
        "### Install dependencies"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "_uuid": "a673b6e72f7d6357035fe757d865b4e35e7d37cc",
        "collapsed": true,
        "jupyter": {
          "outputs_hidden": true
        },
        "trusted": true,
        "id": "t0BHcYzS-v2d"
      },
      "source": [
        "%%bash\n",
        "pip install git+https://github.com/fastai/fastai2\n",
        "pip install torch torchvision feather-format pillow=='6.2.0' kornia pyarrow --upgrade   > /dev/null"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1RGat-w6-v2e"
      },
      "source": [
        "## EDA"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5",
        "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
        "trusted": true,
        "id": "ALMVim-y-v2f"
      },
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "from fastai2.data.all import *\n",
        "from fastai2.vision.data import *\n",
        "from fastai2.vision.all import *\n",
        "\n",
        "from sklearn.metrics import auc, roc_curve, precision_recall_curve, classification_report\n",
        "\n",
        "import os\n",
        "print(os.listdir(\"../input/cell_images/cell_images/\"))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_uuid": "693a80046f72af973f371188ef4f5f68337c5e45",
        "id": "j2kTduQj-v2f"
      },
      "source": [
        "## Dataset"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "_uuid": "5fb06be4bf9e7a552dd652a675526c88ba6d366b",
        "trusted": true,
        "id": "iSfVtuge-v2g"
      },
      "source": [
        "path=Path('../input/cell_images/cell_images/')\n",
        "path.ls()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "trusted": true,
        "id": "hkcc927N-v2h"
      },
      "source": [
        "splitter = RandomSplitter()\n",
        "item_tfms = [Resize(224)]\n",
        "batch_tfms=[*aug_transforms(flip_vert=True,max_zoom=1.2, max_warp=0), Normalize.from_stats(*imagenet_stats)]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "_uuid": "dc60646595963ce3147a8c5ad5f537808b9e8e37",
        "trusted": true,
        "id": "npPGaUs8-v2h"
      },
      "source": [
        "data_block = DataBlock(blocks=[ImageBlock, CategoryBlock],\n",
        "                  get_items=get_image_files,\n",
        "                  get_y=parent_label,\n",
        "                  splitter=splitter,\n",
        "                  item_tfms=item_tfms,\n",
        "                  batch_tfms=batch_tfms)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "trusted": true,
        "id": "z_RGWVFE-v2i"
      },
      "source": [
        "data = data_block.dataloaders(path, bs=64)\n",
        "data.show_batch()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_uuid": "84d059588de5f03210855f2c62012f02c96355f7",
        "id": "0pIyGcW0-v2j"
      },
      "source": [
        "## Model XResNet34"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "_uuid": "43592a27e6723f9846e60bd1cf7e6442fb61e81a",
        "collapsed": true,
        "jupyter": {
          "outputs_hidden": true
        },
        "trusted": true,
        "id": "ctLwNIcS-v2j"
      },
      "source": [
        "learn = cnn_learner(data, xresnet34, metrics=[accuracy]).to_fp16()\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "trusted": true,
        "id": "WCsM9EWF-v2j"
      },
      "source": [
        "learn.fine_tune(15)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "trusted": true,
        "id": "DXlUm3UJ-v2k"
      },
      "source": [
        "learn.save('stage-1')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_uuid": "0f2bd331893e3a2eec9e7bb707bdf2468fb32789",
        "id": "cQXbB2Yk-v2k"
      },
      "source": [
        "## Confusion Matrix"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "_uuid": "0a6fe6a1fc8f67ab07d0de5d6ec9a872523f714b",
        "trusted": true,
        "id": "nTGoilka-v2k"
      },
      "source": [
        "\n",
        "interp = ClassificationInterpretation.from_learner(learn)\n",
        "interp.plot_confusion_matrix(figsize=(10,8))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "trusted": true,
        "id": "6f-FXtIn-v2l"
      },
      "source": [
        "preds, y_true=learn.get_preds()\n",
        "y_true.shape, preds.shape"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "trusted": true,
        "id": "mANBmdaS-v2l"
      },
      "source": [
        "y_true=y_true.numpy() \n",
        "preds=np.argmax(preds.numpy(), axis=-1)\n",
        "y_true.shape, preds.shape"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "trusted": true,
        "id": "HORqiB4u-v2l"
      },
      "source": [
        "classes = list(data.vocab)\n",
        "report = classification_report(y_true, preds, target_names=classes)\n",
        "print(report)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "trusted": true,
        "id": "1SwFF7qL-v2m"
      },
      "source": [
        "learn.show_results()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "_uuid": "f270038f25daea75fc74b8ddd17d7c57897d7798",
        "trusted": true,
        "id": "kMq4x2w1-v2m"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}