Day #13 Imbalanced_Datasets_in_PyTorch.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "view-in-github"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/skyprince999/100-Days-Of-ML/blob/master/Day_13_Imbalanced_Datasets_in_PyTorch.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "aHmKv5xEpqs8"
   },
   "source": [
    "In many machne learning competitions we come across imbalanced datasets. Say in a rare disease identification challenge, the target variable may just be 0.1% of the entire dataset. Traditionally this is a challenge for most ML practitioners. \n",
    "\n",
    "Normally the way to overcome this is by using either one of the two ways of balancing the dataset:\n",
    "\n",
    "\n",
    "*   Undersampling the majority class\n",
    "*   Oversampling the minority class\n",
    "\n",
    "When you undersample the majority class, it generally leads to a loss of information. \n",
    "\n",
    "While oversampling the minority class can lead to overfitting to those particular examples. \n",
    "\n",
    "In this colab notebook, we use PyTorchs `ImbalancedDatasetSampler` to do the following:\n",
    "\n",
    "\n",
    "\n",
    "1.   Rebalanced the class distributions when sampling from the imbalanced dataset\n",
    "2.   Estimate the sampling weights automatically\n",
    "3. Avoid creating a balanced dataset\n",
    "4. Mitigate overfitting by the use of data augmentations\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 119
    },
    "colab_type": "code",
    "id": "sxSQrTcSragx",
    "outputId": "66f26649-c049-4c29-dbe6-b517b69cb25e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cloning into 'imbalanced-dataset-sampler'...\n",
      "remote: Enumerating objects: 12, done.\u001b[K\n",
      "remote: Counting objects:   8% (1/12)\u001b[K\r",
      "remote: Counting objects:  16% (2/12)\u001b[K\r",
      "remote: Counting objects:  25% (3/12)\u001b[K\r",
      "remote: Counting objects:  33% (4/12)\u001b[K\r",
      "remote: Counting objects:  41% (5/12)\u001b[K\r",
      "remote: Counting objects:  50% (6/12)\u001b[K\r",
      "remote: Counting objects:  58% (7/12)\u001b[K\r",
      "remote: Counting objects:  66% (8/12)\u001b[K\r",
      "remote: Counting objects:  75% (9/12)\u001b[K\r",
      "remote: Counting objects:  83% (10/12)\u001b[K\r",
      "remote: Counting objects:  91% (11/12)\u001b[K\r",
      "remote: Counting objects: 100% (12/12)\u001b[K\r",
      "remote: Counting objects: 100% (12/12), done.\u001b[K\n",
      "remote: Compressing objects:   9% (1/11)\u001b[K\r",
      "remote: Compressing objects:  18% (2/11)\u001b[K\r",
      "remote: Compressing objects:  27% (3/11)\u001b[K\r",
      "remote: Compressing objects:  36% (4/11)\u001b[K\r",
      "remote: Compressing objects:  45% (5/11)\u001b[K\r",
      "remote: Compressing objects:  54% (6/11)\u001b[K\r",
      "remote: Compressing objects:  63% (7/11)\u001b[K\r",
      "remote: Compressing objects:  72% (8/11)\u001b[K\r",
      "remote: Compressing objects:  81% (9/11)\u001b[K\r",
      "remote: Compressing objects:  90% (10/11)\u001b[K\r",
      "remote: Compressing objects: 100% (11/11)\u001b[K\r",
      "remote: Compressing objects: 100% (11/11), done.\u001b[K\n",
      "Unpacking objects:   3% (1/30)   \r",
      "Unpacking objects:   6% (2/30)   \r",
      "Unpacking objects:  10% (3/30)   \r",
      "Unpacking objects:  13% (4/30)   \r",
      "Unpacking objects:  16% (5/30)   \r",
      "Unpacking objects:  20% (6/30)   \r",
      "Unpacking objects:  23% (7/30)   \r",
      "Unpacking objects:  26% (8/30)   \r",
      "Unpacking objects:  30% (9/30)   \r",
      "Unpacking objects:  33% (10/30)   \r",
      "Unpacking objects:  36% (11/30)   \r",
      "Unpacking objects:  40% (12/30)   \r",
      "Unpacking objects:  43% (13/30)   \r",
      "Unpacking objects:  46% (14/30)   \r",
      "Unpacking objects:  50% (15/30)   \r",
      "Unpacking objects:  53% (16/30)   \r",
      "Unpacking objects:  56% (17/30)   \r",
      "Unpacking objects:  60% (18/30)   \r",
      "Unpacking objects:  63% (19/30)   \r",
      "Unpacking objects:  66% (20/30)   \r",
      "Unpacking objects:  70% (21/30)   \r",
      "Unpacking objects:  73% (22/30)   \r",
      "Unpacking objects:  76% (23/30)   \r",
      "Unpacking objects:  80% (24/30)   \r",
      "remote: Total 30 (delta 0), reused 5 (delta 0), pack-reused 18\u001b[K\n",
      "Unpacking objects:  83% (25/30)   \r",
      "Unpacking objects:  86% (26/30)   \r",
      "Unpacking objects:  90% (27/30)   \r",
      "Unpacking objects:  93% (28/30)   \r",
      "Unpacking objects:  96% (29/30)   \r",
      "Unpacking objects: 100% (30/30)   \r",
      "Unpacking objects: 100% (30/30), done.\n"
     ]
    }
   ],
   "source": [
    "#First git clone the repo to your local drive \n",
    "\n",
    "!git clone https://github.com/ufoym/imbalanced-dataset-sampler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "9xS81J9cryci",
    "outputId": "4c4a0879-a4d0-412b-c242-5ff4ce515777"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "imbalanced-dataset-sampler  sample_data\n"
     ]
    }
   ],
   "source": [
    "!ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 51
    },
    "colab_type": "code",
    "id": "Dd5aucjsrziB",
    "outputId": "6db578ff-1d9a-4839-e5bd-64cd0a9dcb16"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "examples  MANIFEST.in  requirements.txt  torchsampler\n",
      "LICENSE   README.md    setup.py\n"
     ]
    }
   ],
   "source": [
    "!ls imbalanced-dataset-sampler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 204
    },
    "colab_type": "code",
    "id": "1Opv6ViXtnOG",
    "outputId": "9ea292ac-d17f-44a5-9257-e6ec83229751"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Partial import during the build process.\n",
      "running install\n",
      "running bdist_egg\n",
      "running egg_info\n",
      "creating torchsampler.egg-info\n",
      "writing torchsampler.egg-info/PKG-INFO\n",
      "writing dependency_links to torchsampler.egg-info/dependency_links.txt\n",
      "writing requirements to torchsampler.egg-info/requires.txt\n",
      "writing top-level names to torchsampler.egg-info/top_level.txt\n",
      "writing manifest file 'torchsampler.egg-info/SOURCES.txt'\n",
      "error: package directory 'torchsampler' does not exist\n"
     ]
    }
   ],
   "source": [
    "!python imbalanced-dataset-sampler/setup.py install"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 238
    },
    "colab_type": "code",
    "id": "CcHlxFdgt40k",
    "outputId": "09a02f09-4222-4645-e69e-b96e0dc5440a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Processing ./imbalanced-dataset-sampler\n",
      "Requirement already satisfied: torch in /usr/local/lib/python3.6/dist-packages (from torchsampler==0.1) (1.4.0)\n",
      "Requirement already satisfied: torchvision in /usr/local/lib/python3.6/dist-packages (from torchsampler==0.1) (0.5.0)\n",
      "Requirement already satisfied: pillow>=4.1.1 in /usr/local/lib/python3.6/dist-packages (from torchvision->torchsampler==0.1) (6.2.2)\n",
      "Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from torchvision->torchsampler==0.1) (1.17.5)\n",
      "Requirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from torchvision->torchsampler==0.1) (1.12.0)\n",
      "Building wheels for collected packages: torchsampler\n",
      "  Building wheel for torchsampler (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
      "  Created wheel for torchsampler: filename=torchsampler-0.1-cp36-none-any.whl size=3634 sha256=a2b4ad34d4ceb35792f249a52eae9f50190bdd2723de505d54e4bdb5281d1ffd\n",
      "  Stored in directory: /root/.cache/pip/wheels/38/2b/6a/c92da1292ef596800afc50058a85ca91c768176288a586ecbe\n",
      "Successfully built torchsampler\n",
      "Installing collected packages: torchsampler\n",
      "Successfully installed torchsampler-0.1\n"
     ]
    }
   ],
   "source": [
    "!pip install imbalanced-dataset-sampler/."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "LhEQGF3suEt7"
   },
   "outputs": [],
   "source": [
    "import torch\n",
    "from torchsampler import ImbalancedDatasetSampler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "RvT8MCoTuPKI"
   },
   "outputs": [],
   "source": [
    "train_loader = torch.utils.data.DataLoader(\n",
    "    train_dataset, \n",
    "    sampler=ImbalancedDatasetSampler(train_dataset),\n",
    "    batch_size=args.batch_size, \n",
    "    **kwargs\n",
    ")"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "authorship_tag": "ABX9TyMWXlZdVVvYv8T+OgK1HDGy",
   "collapsed_sections": [],
   "include_colab_link": true,
   "name": "Day #13_Imbalanced Datasets in PyTorch.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}