project.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TntJ6WXFZPdo"
      },
      "source": [
        "\n",
        "\n",
        "<h2 align=\"center\"> NSDC Data Science Projects</h2>\n",
        "  \n",
        "<h2 align=\"center\">Project: Sentiment Analysis of Movie Reviews\n",
        "</h2>\n",
        "\n",
        "<h3 align=\"center\"> Name: Shamura Ahmad</h3>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RKnVUiwQZPdq"
      },
      "source": [
        "### **Please read before you begin your project**\n",
        "\n",
        "**Instructions: Google Colab Notebooks:**\n",
        "\n",
        "Google Colab is a free cloud service. It is a hosted Jupyter notebook service that requires no setup to use, while providing free access to computing resources. We will be using Google Colab for this project.\n",
        "\n",
        "Certain parts of this project will be completed individually, while other parts are encouraged to be completed with the rest of your team. In order to work within the Google Colab Notebook, **please start by clicking on \"File\" and then \"Save a copy in Drive.\"** This will save a copy of the notebook in your personal Google Drive. Each member of your team should work on their personal copy.\n",
        "\n",
        "Please rename the file to \"DSP - Movie Reviews Analysis - Your Full Name.\"\n",
        "\n",
        "You can now start working on the project. :)\n",
        "\n",
        "**Project Description:**\n",
        "\n",
        "This project will introduce students to an array of skills as they strive to create a sentiment analysis model to classify a given review as positive or negative. Sentiment Analysis leverages both Natural Language Processing and Machine Learning skills - how to represent text in a machine-understandable format so as to classify the text and extract sentiment. We will also cover visualizations and how to deploy models in the real world."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gQpujHnvZPdr"
      },
      "source": [
        "\n",
        "\n",
        "<h3 align = \"center\">\n",
        "    Day #1\n",
        "</h3>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e417MmhSZPdr"
      },
      "source": [
        "NOTE: These steps are to be completed **individually**, not as a team.\n",
        "GOAL: The main goal of this milestone is to set up your environment, install the required packages, learn how to acces data and do some basic exploratory data analysis."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pmouMnaaZPdr"
      },
      "source": [
        "**Step 1:**\n",
        "\n",
        "Setting up libraries and installing packages\n",
        "\n",
        "To install a library:\n",
        "```python\n",
        " import <library> as <shortname>\n",
        "```\n",
        "We use a *short name* since it is easier to refer to the package to access functions and also to refer to subpackages within the library.\n"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(\"Moview Review Analysis\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "eEdlXxUAeMeu",
        "outputId": "c46cbc0b-634c-4c7a-ac98-bc3fb130d11b"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Moview Review Analysis\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [],
      "metadata": {
        "id": "3zOkzLqdetB2"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import pandas as pd\n",
        "import numpy as nm\n",
        "import matplotlib.pyplot as mp\n",
        "import nltk.tokenize as nt\n",
        "from sklearn.feature_extraction.text import CountVectorizer\n",
        "from sklearn.preprocessing import LabelBinarizer\n",
        "from sklearn.naive_bayes import MultinomialNB\n",
        "from sklearn.metrics import accuracy_score\n",
        "import seaborn as sb"
      ],
      "metadata": {
        "id": "5r1L14EMfAAH"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qy1yhZqNZPdt"
      },
      "source": [
        "These are the libraries that will help us throughout this project. It is not necessary that you know what each library does, but you can always look it up.\n",
        "\n",
        "We encourage you to read more about the important and most commonly used packages like Pandas and Natural Language Toolkit (NLTK) and write a few lines in your own words about what they do. [You may use the Data Science Resource Repository (DSRR) to find resources to get started!](https://nebigdatahub.org/nsdc/data-science-resource-repository/)\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dInyzwncZPdt"
      },
      "source": [
        "**Step 2:**\n",
        "\n",
        "Let’s access our data. We will be using the Internet Movie Database (IMDb) as our dataset. The dataset contains 50,000 movie reviews from the Internet Movie Database. Reviews have been pre-labeled with sentiment polarity (positive/negative).  \n",
        "\n",
        "\n",
        "[The IMDb Movie Reviews dataset is available at this link](https://github.com/raw/meghjoshii/NSDC_DataScienceProjects_SentimentAnalysis/main/IMDB%20Dataset.csv). It is better to use the link provided directly within the read_csv function.\n",
        "\n",
        "\n",
        "\n",
        "We will use pandas to read the data from the csv file using the `read_csv` function. This function returns a pandas dataframe. We will store this dataframe in a variable called `df`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "aBDt0Qvzndex"
      },
      "outputs": [],
      "source": [
        "url = \"https://github.com/raw/meghjoshii/NSDC_DataScienceProjects_SentimentAnalysis/main/IMDB%20Dataset.csv\""
      ]
    },
    {
      "cell_type": "code",
      "source": [
        " data = pd.read_csv(url)"
      ],
      "metadata": {
        "id": "OB29TWAJkAq9"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "721YSkyhZPd3"
      },
      "source": [
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nhFDTTsSZPd4"
      },
      "source": [
        "**Step 3:**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4BRrwQ2yZPd4"
      },
      "source": [
        "Let's see what the data looks like. We can use the `head` function which returns the first 5 rows of the dataframe."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "24duiI4RpNDN"
      },
      "outputs": [],
      "source": [
        "data.head()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jnZloVlvZPd4"
      },
      "source": [
        "There are 2 columns in the dataframe - `review` and `sentiment`. The `review` column contains the text of the review and the `sentiment` column contains the sentiment of the review.\n",
        "\n",
        "The `describe()` function gives us a summary of the data."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "IfzpsKR2BIFy"
      },
      "outputs": [],
      "source": [
        "data.describe()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aW0psSI_ZPd5"
      },
      "source": [
        "We can see that we have 50,000 reviews in our dataset. The `sentiment` column has 2 unique values - `positive` and `negative`."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eT9zPK-tZPd5"
      },
      "source": [
        "Individual columns can be accessed using the `[]` operator. For example, `df['review']` returns the `review` column of the dataframe."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "LyqiBi-OZPd5"
      },
      "outputs": [],
      "source": [
        "print(data['review'])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uxbGYJZIZPd5"
      },
      "source": [
        "Let's see how many positive and negative reviews we have in our dataset. We can use the `value_counts()` function to get the count of each unique value in the `sentiment` column."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8DbvKXDBBWvW"
      },
      "outputs": [],
      "source": [
        "sentiment_count = data['sentiment'].value_counts()\n",
        "print(sentiment_count)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rwbdUzT8ZPd5"
      },
      "source": [
        "We can see that we have 25,000 positive reviews and 25,000 negative reviews in our dataset. They are evenly distributed and we do not have to worry about class imbalance.\n",
        "\n",
        "[Follow this link to learn more about class imbalance](https://machinelearningmastery.com/what-is-imbalanced-classification/)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cT26TL00ZPd5"
      },
      "source": [
        "\n",
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "p0mDeS4HZPd5"
      },
      "source": [
        "**Step 4:**\n",
        "Let's take a short break from coding and do some reading that is imperative to understand the concepts of this project."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mJ2xYujeZPd6"
      },
      "source": [
        "\n",
        "The **objective** of our machine learning model will be to predict the sentiment of a review given the text of the review. So, the model needs to learn the relationship between the text of the review and the sentiment of the review. Hence, this is a supervised learning problem where the input is text and the output is a label.\n",
        "\n",
        "[Click here to watch introductory videos and learn more about supervised machine learning](https://www.youtube.com/playlist?list=PLNs9ZO9jGtUCiGTo3iP0qmI9_qi8oYaRN).\n",
        "\n",
        "\n",
        "\n",
        "Since we are going to be using text as input, we cannot directly use the text because computers do not understand text. We need to convert the text into a format that is useful for our classification model.\n",
        "\n",
        "Count vectorization is a method to convert text into a format that is useful for classification models. It converts the text into a matrix of token counts meaning that each row in the matrix represents a review and each column represents a word. The value in each cell is the number of times that word occurs in that review. So, by learning the frequency of each word in each review, the model can learn the relationship between the text and the sentiment of the review. The intuition behind this is that positive reviews will have more positive words and negative reviews will have more negative words.\n",
        "\n",
        "Now that we have established the intuition behind count vectorization, let's look at features of the count vectorizer. The features of the count vectorizer are the words that we want to consider. We would only want to use words that are relevant to the sentiment of the review. For example, if we are classifying reviews of movies, we would not want to consider words like `the`, `a`, `an` etc. because they are not relevant to the sentiment of the review. Also, we would want to consider words that occur frequently in the reviews. For example, if a word occurs only once in the entire dataset, it is not very useful for our model.\n",
        "\n",
        "To remove words that are not relevant to the sentiment of the review, first we need to tokenize the text.\n",
        "\n",
        "Tokenization is the process of splitting a string into a list of tokens. This helps us to break down the text into smaller chunks which are easier to work with. What we essentially want to do  is remove all the punctuation and special characters from the text because they do not add any value to the text. We also want to convert all the text to lowercase so that the model does not treat the same word with different cases as different words."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Z3NohCaoZPd6"
      },
      "source": [
        "\n",
        "\n",
        "<h3 align = \"center\">\n",
        "    Day #2\n",
        "</h3>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f9hFI6uKZPd6"
      },
      "source": [
        "NOTE: These steps are to be completed **individually**, not as a team.\n",
        "\n",
        "GOAL: The main goal of this milestone is to learn natural langauge processing and how to use the NLTK library to preprocess text. We will also learn how to use the CountVectorizer class to convert text into a format that is useful for classification models."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "A1uBbf33ZPd6"
      },
      "source": [
        "**Step 1:**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GtYBV0jQZPd6"
      },
      "source": [
        "We will use the `nltk` library to perform these preprocessing steps. First, we will use the `word_tokenize` function to tokenize the text."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "0NFQtgeAZPd6"
      },
      "outputs": [],
      "source": [
        "import nltk\n",
        "from nltk.tokenize import word_tokenize\n",
        "nltk.download('punkt')\n",
        "data['review'] = data['review'].apply(word_tokenize)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "-4TsDmstZPd6"
      },
      "outputs": [],
      "source": [
        "data['review'][1]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3uwzXgWcZPd6"
      },
      "source": [
        "We see that the text has been tokenized into a list of words. Also, the list contains punctuation and special characters which we do not want."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "g4GGKW28ZPd7"
      },
      "source": [
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EtqjXDhAZPd7"
      },
      "source": [
        "**Step 2:**\n",
        "\n",
        "\n",
        "Let's clean the text by removing punctuations, special characters and converting the text to lowercase. We will use the `isalpha` function to check if a word is an alphabet. If it is not an alphabet, we will remove it from the list. We will also convert the text to lowercase using the `lower` function. Next, we will remove the stopwords from the list. Stopwords are words that do not add any value to the text. For example, `the`, `a`, `an` etc. are stopwords. We will use the `stopwords` function from the `nltk.corpus` package to get a list of stopwords. We will then use the `remove` function to remove the stopwords from the list."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "UmMAl_YnZPd7"
      },
      "outputs": [],
      "source": [
        "data['review'] = data['review'].apply(lambda x: [item for item in x if item.isalpha()])\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "V_J_DyzyZPd7"
      },
      "outputs": [],
      "source": [
        "print(\" \".join(data['review'][1]))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "egtHLhV_ZPd7"
      },
      "outputs": [],
      "source": [
        "data['review'] = data['review'].apply(lambda x: [item.lower() for item in x])"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import nltk\n",
        "nltk.download('stopwords')\n",
        "from nltk.corpus import stopwords\n",
        "stop_words = set(stopwords.words('english'))"
      ],
      "metadata": {
        "id": "ACEl3_P79Wl4"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "5IKcYhFjZPd7"
      },
      "outputs": [],
      "source": [
        "data['review'] = data['review'].apply(lambda x: [item for item in x if item not in stop_words])\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OOvMGYcCZPd7"
      },
      "source": [
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QqEiNV5uZPd8"
      },
      "source": [
        "**Step 3:**\n",
        "\n",
        "Now that we have cleaned the text, we need to use a stemmer to stem the words. Stemming is the process of reducing a word to its root form. For example, the root form of the word `running` is `run`. Stemming helps us to reduce the number of unique words in the text. We will use the `PorterStemmer` function from the `nltk.stem` package to stem the words."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4NqS01znZPd8"
      },
      "outputs": [],
      "source": [
        "from nltk.stem import PorterStemmer"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "86zNoUceZPd8"
      },
      "outputs": [],
      "source": [
        "ps = PorterStemmer()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "IRDONNgaZPd8"
      },
      "outputs": [],
      "source": [
        "data['review'] = data['review'].apply(lambda x: [ps.stem(item) for item in x])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "xeOaUm0MZPd8"
      },
      "outputs": [],
      "source": [
        "data['review'] = data['review'].apply(lambda x: \" \".join(x))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "grdrqrdoZPd8"
      },
      "source": [
        "---\n",
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BhRdCZx7ZPd8"
      },
      "source": [
        "\n",
        "\n",
        "<h3 align = \"center\">\n",
        "    Day #3\n",
        "</h3>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oiA7qc95ZPd8"
      },
      "source": [
        "NOTE: These steps are to be completed **individually**, not as a team.\n",
        "\n",
        "GOAL: The main goal of this milestone is to split the dataset into training and testing sets. We will also learn how to use the CountVectorizer class to convert text into a format that is useful for classification models. We will also learn how to use the MultinomialNB class to train a Naive Bayes classifier."
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Training and Testing Data:\n",
        "\n",
        "Machine learning uses algorithms to learn from data in datasets. They find patterns, develop understanding, make decisions, and evaluate those decisions.\n",
        "\n",
        "In machine learning, datasets are split into two subsets:\n",
        "\n",
        "The first subset is known as the **training data** - it’s a portion of our actual dataset that is fed into the machine learning model to discover and learn patterns. In this way, it trains our model.\n",
        "\n",
        "The other subset is known as the **testing data**.\n",
        "\n",
        "Once your machine learning model is built (with your training data), you need unseen data to test your model. This data is called testing data, and you can use it to evaluate the performance and progress of your algorithms’ training and adjust or optimize it for improved results.\n",
        "\n",
        "Testing data has two main criteria. It should:\n",
        "\n",
        "1. Represent the actual dataset\n",
        "2. Be large enough to generate meaningful predictions\n"
      ],
      "metadata": {
        "id": "ep1BAYMi4qmf"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4TGKhOnjZPd8"
      },
      "source": [
        "**Step 1:**\n",
        "\n",
        "Now, the data is tokenized, cleaned and reduced to its root form.\n",
        "The next step is to split the data for training and testing. We split the data because we need to train our model on some data and test it on some data. We have a total of 50,000 reviews, so let's split it into 40,000 reviews for training and 10,000 reviews for testing.\n",
        "To do this, we can use the slice operator `:`. For example, `df[:30000]` returns the first 30,000 rows of the dataframe. Similarly, `df[30000:]` returns the last 20,000 rows of the dataframe.\n",
        "\n",
        "Name the training data as `train_reviews` and testing data as `test_reviews`. Remember, we are only splitting the reviews column and will do the same for sentiment in the next step."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9CwLp2sDCL8M"
      },
      "outputs": [],
      "source": [
        "train_reviews = data.review[:40000]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cDlrQ-1DCPrS"
      },
      "outputs": [],
      "source": [
        "test_reviews = data.review[40000:]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "p596V8gXZPd9"
      },
      "source": [
        "Now let us do the same for the sentiment column. Name the training data as `train_sentiments` and testing data as `test_sentiments`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "l0GW7b7CZPd9"
      },
      "outputs": [],
      "source": [
        "train_sentiments = data.sentiment[:40000]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "JPmpK8iIqqdp"
      },
      "outputs": [],
      "source": [
        "train_sentiments = data.sentiment[40000:]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Nvy4Ng5IZPd9"
      },
      "source": [
        "---\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vYIHq7__ZPd9"
      },
      "source": [
        "**Step 2:**\n",
        "\n",
        "We need to make a few changes to the data before we can use it to train our model. First, we need to convert the data into a format that is useful for our model. We will use the `CountVectorizer` function from the `sklearn.feature_extraction.text` package to convert the text into a matrix of token counts.\n",
        "\n",
        "For the sentiment column, we need to convert the labels into numbers. We will use the `LabelEncoder` function from the `sklearn.preprocessing` package to convert the labels into numbers."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qs5GfcJRZPd9"
      },
      "source": [
        "[To read more about Count Vectorizer, follow this link](https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.CountVectorizer.html). [You can also use this link to read more about Label Encoder](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.LabelEncoder.html). Please go through the parameters of both these functions to better understand the code below."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "nxzTuYbmCShZ"
      },
      "outputs": [],
      "source": [
        "cv = CountVectorizer(min_df=0, max_df=1, binary = False, ngram_range = (1,3))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_x_082TdZPd9"
      },
      "source": [
        "To transform the data, we will use the `fit_transform` function. The `fit_transform` function fits the model to the data and then transforms the data into a matrix of token counts. We will use the `fit_transform` function on the training data and the `transform` function on the testing data. This is because we only want to fit the model to the training data and not the testing data."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "43Thg7oeZPd9"
      },
      "outputs": [],
      "source": [
        "cv_train_reviews = cv.fit_transform(train_reviews)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cQP_nC4CZPd9"
      },
      "outputs": [],
      "source": [
        "cv_test_reviews = cv.transform(test_reviews)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vWxxbASGZPd9"
      },
      "source": [
        "Again, for the sentiment column, we will use the `fit_transform` function on the training data and the `transform` function on the testing data."
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "lb = LabelBinarizer()"
      ],
      "metadata": {
        "id": "LPOjdR-pKmJk"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "m9xLTsunCcdI"
      },
      "outputs": [],
      "source": [
        "lb_train_sentiments = lb.fit_transform(train_sentiments)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "GKCUVO5oZPd-"
      },
      "outputs": [],
      "source": [
        "lb_test_binary = lb.fit_transform(test_sentiment)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FkxY_9XRZPd-"
      },
      "source": [
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-1RZAqvQZPd-"
      },
      "source": [
        "**Step 3:**\n",
        "\n",
        "Model Building: In this step, we will build our model. We will use the `MultinomialNB` function from the `sklearn.naive_bayes` package to build our model. The Multinomial Naive Bayes classifier is suitable for classification with discrete features (e.g., word counts for text classification). The multinomial distribution normally requires integer feature counts. Bag-of-Word counts are an example of integer-valued discrete features.\n",
        "\n",
        "[Please read about the Multinomial Naive Bayes classifier here](https://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.MultinomialNB.html) and write about it in the comments below."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mJilZDoDZPd-"
      },
      "source": [
        "<h4 style=\"color:orange\">\n",
        "    TO-DO\n",
        "</h4>\n",
        "\n",
        "Write a few lines about the following:\n",
        "\n",
        "- **Machine Learning Classifiers:**\n",
        "\n",
        "- **Naive Bayes Classifier:**\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cWxBnsi-ZPd-"
      },
      "source": [
        "The first part is to train and fit the Multinomial Naive Bayes classifier to the training data. We will use the `fit` function to train the model on the training data."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rI4TpFtOCke6"
      },
      "outputs": [],
      "source": [
        "mnb = MultinomialNB()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vK7QbHmUZPd-"
      },
      "outputs": [],
      "source": [
        "mnb_bow = mnb.fit(cv_train_reviews, lb_train_sentiments)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Oj9NLo4ZCr7V"
      },
      "outputs": [],
      "source": [
        "mnb_bow_predict = mnb.predict(cv_test_reviews)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "950v5Y4YCveb"
      },
      "outputs": [],
      "source": [
        "mnb_bow_score = accuracy_score(lb_test_binary, mnb_bow_predict)\n",
        "print(\"Accuracy :\", mnb_bow_score)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-fKVYr6zZPd-"
      },
      "source": [
        "If you have completed all the steps correctly, you should see an accuracy of ~70-80%. This is decent, but not a very good accuracy since language is a very complex thing. But it's a great start!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Jhsah2fiZPd-"
      },
      "source": [
        "---\n",
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PQLLgAgcZPd_"
      },
      "source": [
        "<h3 align = \"center\">\n",
        "    Milestone #4\n",
        "</h3>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "m6E5VHGMZPd_"
      },
      "source": [
        "NOTE: This milestone is to be completed as a **group**.  Each group member should try a different classifier and you must discuss the results with your teammates. Please attend Office Hours or ask your questions on Slack.\n",
        "\n",
        "GOAL: The main goal of this milestone is to understand how to use different classifiers to train a model and how to evaluate the performance of the model."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6kVj97SAZPd_"
      },
      "source": [
        "**Step 1:**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QFRvyZl8ZPd_"
      },
      "source": [
        "[Please read about the different classifiers here](https://scikit-learn.org/stable/supervised_learning.html#supervised-learning).\n",
        "Each team member should try at least one **different** classifier. You can try more than one classifier if you want. Please write about the classifier you have chosen in the comments below."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "hBGkNG__ZPd_"
      },
      "outputs": [],
      "source": [
        "svm = SVC()\n",
        "svm_bow = svm.fit(cv_train_reviews, lb_train_sentiment)\n",
        "svm_predict = svm.predict(cv_test_reviews)\n",
        "svm_score = accuracy_score(lb_test_sentiment, svm_predict)\n",
        "print(\"Accuracy (SVM) :\", svm_score)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sTCklEYkZPd_"
      },
      "source": [
        "**Step 2:**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yB5Wc7FiZPd_"
      },
      "source": [
        "Data Visualizations are a great way to understand the data. [If you are interested in finding additional resources on Data Visualization, we recommend leveraging the DSRR resources, linked here.](https://nebigdatahub.org/nsdc/data-science-resource-repository/) We will be using word clouds to visualize the data. Word clouds are a great way to visualize the most frequent words in a text. We will use the `WordCloud` function from the `wordcloud` package to visualize the most frequent words in the reviews.\n",
        "\n",
        "[Please read about the WordCloud function here](https://amueller.github.io/word_cloud/generated/wordcloud.WordCloud.html).\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "zfajmPP3C6ZZ"
      },
      "outputs": [],
      "source": [
        "# word cloud for positive review words in the entire dataset\n",
        "from wordcloud import WordCloud, STOPWORDS\n",
        "import matplotlib.pyplot as plt\n",
        "%matplotlib inline\n",
        "\n",
        "#join all the positive reviews\n",
        "positive_words = ' '.join(list(df[df['sentiment'] == 'positive']['review']))\n",
        "\n",
        "#word cloud for positive words\n",
        "wordcloud = WordCloud(width=800, height=500, random_state=21, max_font_size=110).generate(positive_words)\n",
        "\n",
        "plt.figure(figsize=(10, 7))\n",
        "plt.imshow(wordcloud, interpolation=\"bilinear\")\n",
        "plt.axis('off')\n",
        "plt.show()\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bhEL3IO3C9O4"
      },
      "outputs": [],
      "source": [
        "# TODO: Word cloud for negative reviews in the dataset"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5-yTKusaZPd_"
      },
      "source": [
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UT8Rpdt4ZPeA"
      },
      "source": [
        "<h3 align = 'center' >\n",
        "Thank you for completing the project!\n",
        "</h3>\n",
        "\n",
        "Please do reach out to us if you have any questions or concerns. We are here to help you learn and grow.\n"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": [],
      "gpuType": "T4"
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.8"
    },
    "vscode": {
      "interpreter": {
        "hash": "58fcf84e0ebbaad2b6cc744ed9d48691de4e147b5be28c3707516d96647d7374"
      }
    },
    "accelerator": "GPU"
  },
  "nbformat": 4,
  "nbformat_minor": 0
}