09_BagofWords_N_Gram.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<small><small><i>\n",
    "All the IPython Notebooks in **[Python Natural Language Processing](https://github.com/milaan9/Python_Python_Natural_Language_Processing)** lecture series by **[Dr. Milaan Parmar](https://www.linkedin.com/in/milaanparmar/)** are available @ **[GitHub](https://github.com/milaan9)**\n",
    "</i></small></small>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "view-in-github"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/milaan9/Python_Python_Natural_Language_Processing/blob/main/09_BagofWords_N_Gram.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "p3vxf3uti2hp"
   },
   "source": [
    "# 09 Bag Of Words and N-Grams\n",
    "\n",
    "## **Bag Of Words**\n",
    "\n",
    "- Bag of words is a Natural Language Processing technique of **text modelling**. In technical terms, we can say that it is a method of **feature extraction** with text data. This approach is a simple and flexible way of extracting features from documents.\n",
    "\n",
    "- A bag of words is a representation of text that describes the occurrence of words within a document. We just keep track of word counts and **disregard the grammatical details and the word order**.\n",
    "-  It is **called a “bag”** of words because any information about the order or structure of words in the document is discarded.\n",
    "- The model is only concerned with whether known words occur in the document, not where in the document.\n",
    "\n",
    "### **Why is the Bag-of-Words algorithm used?**\n",
    "-  The dataset is **messy and unstructured and not fixed-length inputs**, and machine learning algorithms prefer **structured, well defined fixed-length inputs** and by using the Bag-of-Words technique we can convert variable-length texts into a fixed-length vector.\n",
    "\n",
    "- The machine learning models work with **numerical data** rather than textual data. By using the bag-of-words (BoW) technique, we convert a text into its equivalent **vector of numbers**.\n",
    "\n",
    "Sentence 1: ”The Cat sat here” \n",
    "\n",
    "Sentence 2: “The Cat sat in the hat” \n",
    "\n",
    "Sentence 2: “The Cat is with the hat”\n",
    "\n",
    "**Step 1:** Convert the above sentences in lower case as the case of the word does not hold any information.\n",
    "\n",
    "**Step 2:** Remove special characters and stopwords from the text.\n",
    "\n",
    "After applying the above steps, the sentences are changed to\n",
    "\n",
    "Sentence 1: ”the cat sat” \n",
    "\n",
    "Sentence 2: “the cat sat in the hat” \n",
    "\n",
    "Sentence 2: “the cat with the hat”\n",
    "\n",
    "Although the above sentences do not make much sense the maximum information is contained in these words only.\n",
    "\n",
    "**Step 3:** Go through all the words in the above text and make a list of all of the words in our model vocabulary.\n",
    "- [\"the\", \"cat\", \"sat\",\"in\",\"hat\",\"with\"]\n",
    "\n",
    "Now as the vocabulary has only 6 words, we can use a fixed-length document-representation of 6, with one position in the vector to score each word. \n",
    "\n",
    "Sentence 1: [1,1,1,0,0,0] \n",
    "\n",
    "Sentence 2: [2,1,1,1,1,0] \n",
    "\n",
    "Sentence 2: [2,1,0,0,1,1]\n",
    "\n",
    "<img src = \"https://miro.medium.com/max/1400/1*3IACMnNpwVlCl8kSTJocPA.png\"></img>\n",
    "\n",
    "\n",
    "Disadvantages\n",
    "- In bag of words we consider all words as same weighted.\n",
    "ex : His brother is good.\n",
    "here GOOD is highly weighted word and brother is less weighted. In Bag of word we consider all words as same weighted.\n",
    "- It disregard the grammatical details and the word order when storing words.\n",
    "\n",
    "##  **CountVectorizer** \n",
    "Used to count the number of words present in a sentence."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "u9hXx1ohBl7j"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: sklearn in /srv/conda/envs/notebook/lib/python3.7/site-packages (0.0)\n",
      "Requirement already satisfied: scikit-learn in /srv/conda/envs/notebook/lib/python3.7/site-packages (from sklearn) (1.0.1)\n",
      "Requirement already satisfied: joblib>=0.11 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from scikit-learn->sklearn) (1.1.0)\n",
      "Requirement already satisfied: numpy>=1.14.6 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from scikit-learn->sklearn) (1.21.4)\n",
      "Requirement already satisfied: scipy>=1.1.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from scikit-learn->sklearn) (1.7.2)\n",
      "Requirement already satisfied: threadpoolctl>=2.0.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from scikit-learn->sklearn) (3.0.0)\n"
     ]
    }
   ],
   "source": [
    "!pip install sklearn\n",
    "\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "# list of text documents\n",
    "text = [\"The quick brown fox jumped over the lazy dog.\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "id": "RalxdJEgByB4",
    "outputId": "e26e9688-14ca-4527-a752-488643781579"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'the': 7, 'quick': 6, 'brown': 0, 'fox': 2, 'jumped': 3, 'over': 5, 'lazy': 4, 'dog': 1}\n"
     ]
    }
   ],
   "source": [
    "# create the transform\n",
    "vectorizer = CountVectorizer()\n",
    "# tokenize and build vocab\n",
    "vectorizer.fit(text)\n",
    "# summarize\n",
    "print(vectorizer.vocabulary_)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 68
    },
    "id": "4r-Obvt-B0E2",
    "outputId": "0464c586-4d9c-49b0-8a02-d84d95ad96a4"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1, 8)\n",
      "<class 'scipy.sparse.csr.csr_matrix'>\n",
      "[[1 1 1 1 1 1 1 2]]\n"
     ]
    }
   ],
   "source": [
    "# encode document\n",
    "vector = vectorizer.transform(text)\n",
    "# summarize encoded vector\n",
    "print(vector.shape)\n",
    "print(type(vector))\n",
    "print(vector.toarray())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "UdmHqUnHjYTJ"
   },
   "source": [
    "In above array, there is 2 in the end bcz \"the\" is present twice in that sentence."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "id": "u5PdY09-B4Y-"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import re\n",
    "\n",
    "'''The first function we will implement is to extract the words from a document using regular expressions.\n",
    " As we do so, we will be converting all words to lower case and exclude our stop words.'''\n",
    "\n",
    "def tokenize_sentences(sentences):\n",
    "    words = []\n",
    "    for sentence in sentences:\n",
    "        w = extract_words(sentence)\n",
    "        words.extend(w)\n",
    "        \n",
    "    words = sorted(list(set(words)))\n",
    "    return words"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "id": "OwK13TqNB611"
   },
   "outputs": [],
   "source": [
    "'''Next, we implement our tokenize_sentences function. This function builds our vocabulary by looping through\n",
    " all our documents (sentences), extracting the words from each, removing duplicates using the set function and \n",
    " returning a sorted list of words.'''\n",
    "\n",
    "def extract_words(sentence):\n",
    "    ignore_words = ['a']\n",
    "    words = re.sub(\"[^w]\", \" \",  sentence).split() #nltk.word_tokenize(sentence)\n",
    "    words_cleaned = [w.lower() for w in words if w not in ignore_words]\n",
    "    return words_cleaned   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "id": "bKrBOqp8B8Dl"
   },
   "outputs": [],
   "source": [
    "def bagofwords(sentence, words):\n",
    "    sentence_words = extract_words(sentence)\n",
    "    # frequency word count\n",
    "    bag = np.zeros(len(words))\n",
    "    for sw in sentence_words:\n",
    "        for i,word in enumerate(words):\n",
    "            if word == sw: \n",
    "                bag[i] += 1\n",
    "                \n",
    "    return np.array(bag)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "id": "8S8NULCRB-F1"
   },
   "outputs": [],
   "source": [
    "'''Our last function is the implementation of the bag of words model. This function takes an input of a sentence \n",
    "and words (our vocabulary). It then extracts the words from the input sentence using the previously defined function. \n",
    "It creates a vector of zeros using numpy zeros function with a length of the number of words in our vocabulary.'''\n",
    "\n",
    "\n",
    "sentences = [\"Machine learning is great\",\"Natural Language Processing is a complex field\",\n",
    "\"Natural Language Processing is used in machine learning\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 68
    },
    "id": "fuR2rb3OCDB1",
    "outputId": "c969d717-c231-4aa8-fc0a-33a5b299cb29"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0],\n",
       "       [1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0],\n",
       "       [0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1]])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vocabulary = tokenize_sentences(sentences)\n",
    "bagofwords(\"Machine learning is great\", vocabulary)\n",
    "\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "\n",
    "vectorizer = CountVectorizer(analyzer = \"word\", tokenizer = None, preprocessor = None, stop_words = None, max_features = 5000) \n",
    "\n",
    "train_data_features = vectorizer.fit_transform(sentences)\n",
    "\n",
    "vectorizer.transform([\"Machine learning is great\",\"Natural Language Processing is a complex field\",\n",
    "\"Natural Language Processing is used in machine learning\"]).toarray()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 129
    },
    "id": "kdZkFZG9C9HH",
    "outputId": "1a34e1b2-8792-46a1-c3a3-1800be47741b"
   },
   "source": [
    "________________________________********************__________________________"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "n7lypA70S8el"
   },
   "source": [
    "## **Bag Of Words - KN**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "id": "vfIU6n8SW14L"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading collection 'popular'\n",
      "[nltk_data]    | \n",
      "[nltk_data]    | Downloading package cmudict to\n",
      "[nltk_data]    |     /home/jovyan/nltk_data...\n",
      "[nltk_data]    |   Package cmudict is already up-to-date!\n",
      "[nltk_data]    | Downloading package gazetteers to\n",
      "[nltk_data]    |     /home/jovyan/nltk_data...\n",
      "[nltk_data]    |   Package gazetteers is already up-to-date!\n",
      "[nltk_data]    | Downloading package genesis to\n",
      "[nltk_data]    |     /home/jovyan/nltk_data...\n",
      "[nltk_data]    |   Package genesis is already up-to-date!\n",
      "[nltk_data]    | Downloading package gutenberg to\n",
      "[nltk_data]    |     /home/jovyan/nltk_data...\n",
      "[nltk_data]    |   Package gutenberg is already up-to-date!\n",
      "[nltk_data]    | Downloading package inaugural to\n",
      "[nltk_data]    |     /home/jovyan/nltk_data...\n",
      "[nltk_data]    |   Package inaugural is already up-to-date!\n",
      "[nltk_data]    | Downloading package movie_reviews to\n",
      "[nltk_data]    |     /home/jovyan/nltk_data...\n",
      "[nltk_data]    |   Package movie_reviews is already up-to-date!\n",
      "[nltk_data]    | Downloading package names to\n",
      "[nltk_data]    |     /home/jovyan/nltk_data...\n",
      "[nltk_data]    |   Package names is already up-to-date!\n",
      "[nltk_data]    | Downloading package shakespeare to\n",
      "[nltk_data]    |     /home/jovyan/nltk_data...\n",
      "[nltk_data]    |   Package shakespeare is already up-to-date!\n",
      "[nltk_data]    | Downloading package stopwords to\n",
      "[nltk_data]    |     /home/jovyan/nltk_data...\n",
      "[nltk_data]    |   Package stopwords is already up-to-date!\n",
      "[nltk_data]    | Downloading package treebank to\n",
      "[nltk_data]    |     /home/jovyan/nltk_data...\n",
      "[nltk_data]    |   Package treebank is already up-to-date!\n",
      "[nltk_data]    | Downloading package twitter_samples to\n",
      "[nltk_data]    |     /home/jovyan/nltk_data...\n",
      "[nltk_data]    |   Package twitter_samples is already up-to-date!\n",
      "[nltk_data]    | Downloading package omw to /home/jovyan/nltk_data...\n",
      "[nltk_data]    |   Package omw is already up-to-date!\n",
      "[nltk_data]    | Downloading package wordnet to\n",
      "[nltk_data]    |     /home/jovyan/nltk_data...\n",
      "[nltk_data]    |   Package wordnet is already up-to-date!\n",
      "[nltk_data]    | Downloading package wordnet31 to\n",
      "[nltk_data]    |     /home/jovyan/nltk_data...\n",
      "[nltk_data]    |   Package wordnet31 is already up-to-date!\n",
      "[nltk_data]    | Downloading package wordnet_ic to\n",
      "[nltk_data]    |     /home/jovyan/nltk_data...\n",
      "[nltk_data]    |   Package wordnet_ic is already up-to-date!\n",
      "[nltk_data]    | Downloading package words to\n",
      "[nltk_data]    |     /home/jovyan/nltk_data...\n",
      "[nltk_data]    |   Package words is already up-to-date!\n",
      "[nltk_data]    | Downloading package maxent_ne_chunker to\n",
      "[nltk_data]    |     /home/jovyan/nltk_data...\n",
      "[nltk_data]    |   Package maxent_ne_chunker is already up-to-date!\n",
      "[nltk_data]    | Downloading package punkt to\n",
      "[nltk_data]    |     /home/jovyan/nltk_data...\n",
      "[nltk_data]    |   Package punkt is already up-to-date!\n",
      "[nltk_data]    | Downloading package snowball_data to\n",
      "[nltk_data]    |     /home/jovyan/nltk_data...\n",
      "[nltk_data]    |   Package snowball_data is already up-to-date!\n",
      "[nltk_data]    | Downloading package averaged_perceptron_tagger to\n",
      "[nltk_data]    |     /home/jovyan/nltk_data...\n",
      "[nltk_data]    |   Package averaged_perceptron_tagger is already up-\n",
      "[nltk_data]    |       to-date!\n",
      "[nltk_data]    | \n",
      "[nltk_data]  Done downloading collection popular\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#!pip install ntlk\n",
    "\n",
    "import nltk\n",
    "nltk.download('popular')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "id": "nPqM5qxpTCmx"
   },
   "outputs": [],
   "source": [
    "paragraph =  \"\"\"I have three visions for India. In 3000 years of our history, people from all over \n",
    "               the world have come and invaded us, captured our lands, conquered our minds. \n",
    "               strong not only as a military power but also as an economic power. Both must go hand-in-hand. \n",
    "               My good fortune was to have worked with three great minds. Dr. Vikram Sarabhai of the Dept. of \n",
    "               space, Professor Satish Dhawan, who succeeded him and Dr. Brahm Prakash, father of nuclear material.\n",
    "               I was lucky to have worked with all three of them closely and consider this the great opportunity of my life. \n",
    "               I see four milestones in my career\"\"\"\n",
    "               "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "id": "oxBVC9hUTDi8"
   },
   "outputs": [],
   "source": [
    "               \n",
    "# Cleaning the texts\n",
    "import re # used for regular expression\n",
    "from nltk.corpus import stopwords\n",
    "from nltk.stem.porter import PorterStemmer\n",
    "from nltk.stem import WordNetLemmatizer\n",
    "\n",
    "ps = PorterStemmer()\n",
    "wordnet=WordNetLemmatizer()\n",
    "sentences = nltk.sent_tokenize(paragraph)\n",
    "corpus = []\n",
    "for i in range(len(sentences)):\n",
    "\n",
    "  # here we'll substitute or replace everything(i.e ',','/','\\' etc.) except word containing a to z and A to Z ([^a-zA-Z]).\n",
    "    review = re.sub('[^a-zA-Z]', ' ', sentences[i])\n",
    "\n",
    "    review = review.lower()\n",
    "    review = review.split()\n",
    "    \n",
    "    review = [ps.stem(word) for word in review if not word in set(stopwords.words('english'))]\n",
    "    review = ' '.join(review)\n",
    "    corpus.append(review)\n",
    "    \n",
    "# Creating the Bag of Words model\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "cv = CountVectorizer(max_features = 1500,ngram_range = (1,1))\n",
    "# cv = CountVectorizer(max_features = 1500,ngram_range = (1,1),stop_words = 'english' )\n",
    "X = cv.fit_transform(corpus).toarray()\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "DhpJvXYbTDPX",
    "outputId": "43624a41-cc4b-4bed-a935-79e851069973"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0,\n",
       "        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0,\n",
       "        0, 1, 0, 0, 0],\n",
       "       [0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1,\n",
       "        1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1,\n",
       "        0, 0, 0, 1, 1],\n",
       "       [1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "        0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0,\n",
       "        0, 0, 0, 0, 0],\n",
       "       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 2, 0, 0, 0,\n",
       "        0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "        0, 0, 0, 0, 0],\n",
       "       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0,\n",
       "        0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0,\n",
       "        0, 0, 1, 0, 0],\n",
       "       [0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "        0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0,\n",
       "        1, 0, 0, 0, 0],\n",
       "       [0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
       "        0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 1, 1, 0, 1, 0, 1, 0, 1, 0, 0,\n",
       "        0, 0, 0, 0, 0],\n",
       "       [0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0,\n",
       "        0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0,\n",
       "        0, 0, 1, 0, 0],\n",
       "       [0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0,\n",
       "        0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,\n",
       "        0, 0, 0, 0, 0]])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-5qwSUgHN3yv"
   },
   "source": [
    "# **What are N-Grams?**\n",
    "\n",
    "**Sentence 1: “This is a good job. I will not miss it for anything”** \n",
    "\n",
    "**Sentence 2: ”This is not good at all”** \n",
    "\n",
    "For this example, let us take the vocabulary of 5 words only. The five words being- \n",
    "**[ good, job, miss, not, all ]** \n",
    "\n",
    "So, the respective vectors for these sentences are:\n",
    "\n",
    "“This is a good job. I will not miss it for anything”= **[1,1,1,1,0]** \n",
    "\n",
    "”This is not good at all”= **[1,0,0,1,1]**\n",
    "\n",
    "Can you guess what is the problem here? Sentence 2 is a negative sentence and sentence 1 is a positive sentence. Does this reflect in any way in the vectors above? Not at all. So how can we solve this problem? Here come the N-grams to our rescue.\n",
    "\n",
    "- **An N-gram is an N-token sequence of words**: a 2-gram (more commonly called a **bigram**) is a two-word sequence of words like “really good”, “not good”, or “your homework”, and a 3-gram (more commonly called a **trigram**) is a three-word sequence of words like “not at all”, or “turn off light”.\n",
    "\n",
    "For example, the bigrams in the first line of text in the previous section: “This is not good at all” are as follows:\n",
    "\n",
    "“This is” \n",
    "\n",
    "“is not”  \n",
    "\n",
    "“not good” \n",
    "\n",
    "“good at” \n",
    "\n",
    "“at all”"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "f0dHbgU7TDCi"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "include_colab_link": true,
   "name": "9_BagofWords_N-Gram.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}