Fake_News_Prediction.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "This is a binary classification project which uses textual data of the news for predicting if it is real or fake.\n",
        "  - So, we will use logistic regression model for training data "
      ],
      "metadata": {
        "id": "wsaD0MNEKCG9"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Dataset information : \n",
        "\n",
        "\n",
        "1. id: unique id for a news article\n",
        "2. title: the title of a news article\n",
        "3. author: author of the news article\n",
        "4. text: the text of the article; could be incomplete\n",
        "5. label: a label that marks whether the news article is real or fake:\n",
        "           1: Fake news\n",
        "           0: real News\n",
        "\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "8OmNjT3KK6vy"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Importing Dependencies"
      ],
      "metadata": {
        "id": "942N_QTFNrx5"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "sNeHETUzJrl6"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "import re\n",
        "from nltk.corpus import stopwords\n",
        "from nltk.stem.porter import PorterStemmer\n",
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "from sklearn.metrics import accuracy_score\n",
        "import matplotlib.pyplot as plt"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import nltk\n",
        "nltk.download('stopwords')"
      ],
      "metadata": {
        "id": "sdJfq8ITOs04",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "007f161e-3a3e-48da-ac70-2b9bb98fbaf8"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[nltk_data] Downloading package stopwords to /root/nltk_data...\n",
            "[nltk_data]   Package stopwords is already up-to-date!\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# printing the stopwords in English\n",
        "print(stopwords.words('english'))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_46Z_Ad4Xj2Y",
        "outputId": "287c30a1-ac5f-41f5-b4eb-a860f97e6b00"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "['i', 'me', 'my', 'myself', 'we', 'our', 'ours', 'ourselves', 'you', \"you're\", \"you've\", \"you'll\", \"you'd\", 'your', 'yours', 'yourself', 'yourselves', 'he', 'him', 'his', 'himself', 'she', \"she's\", 'her', 'hers', 'herself', 'it', \"it's\", 'its', 'itself', 'they', 'them', 'their', 'theirs', 'themselves', 'what', 'which', 'who', 'whom', 'this', 'that', \"that'll\", 'these', 'those', 'am', 'is', 'are', 'was', 'were', 'be', 'been', 'being', 'have', 'has', 'had', 'having', 'do', 'does', 'did', 'doing', 'a', 'an', 'the', 'and', 'but', 'if', 'or', 'because', 'as', 'until', 'while', 'of', 'at', 'by', 'for', 'with', 'about', 'against', 'between', 'into', 'through', 'during', 'before', 'after', 'above', 'below', 'to', 'from', 'up', 'down', 'in', 'out', 'on', 'off', 'over', 'under', 'again', 'further', 'then', 'once', 'here', 'there', 'when', 'where', 'why', 'how', 'all', 'any', 'both', 'each', 'few', 'more', 'most', 'other', 'some', 'such', 'no', 'nor', 'not', 'only', 'own', 'same', 'so', 'than', 'too', 'very', 's', 't', 'can', 'will', 'just', 'don', \"don't\", 'should', \"should've\", 'now', 'd', 'll', 'm', 'o', 're', 've', 'y', 'ain', 'aren', \"aren't\", 'couldn', \"couldn't\", 'didn', \"didn't\", 'doesn', \"doesn't\", 'hadn', \"hadn't\", 'hasn', \"hasn't\", 'haven', \"haven't\", 'isn', \"isn't\", 'ma', 'mightn', \"mightn't\", 'mustn', \"mustn't\", 'needn', \"needn't\", 'shan', \"shan't\", 'shouldn', \"shouldn't\", 'wasn', \"wasn't\", 'weren', \"weren't\", 'won', \"won't\", 'wouldn', \"wouldn't\"]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Data Preprocessing"
      ],
      "metadata": {
        "id": "uzIEqkUYXwV3"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# loading the dataset into pandas dataframe\n",
        "\n",
        "dataset = pd.read_csv(\"train.csv\")\n",
        "dataset.head(10)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 363
        },
        "id": "qu4y8hPCXjyu",
        "outputId": "51e0cb8a-7e7d-4fe6-f254-283c7cbb59c9"
      },
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   id                                              title  \\\n",
              "0   0  House Dem Aide: We Didn’t Even See Comey’s Let...   \n",
              "1   1  FLYNN: Hillary Clinton, Big Woman on Campus - ...   \n",
              "2   2                  Why the Truth Might Get You Fired   \n",
              "3   3  15 Civilians Killed In Single US Airstrike Hav...   \n",
              "4   4  Iranian woman jailed for fictional unpublished...   \n",
              "5   5  Jackie Mason: Hollywood Would Love Trump if He...   \n",
              "6   6  Life: Life Of Luxury: Elton John’s 6 Favorite ...   \n",
              "7   7  Benoît Hamon Wins French Socialist Party’s Pre...   \n",
              "8   8  Excerpts From a Draft Script for Donald Trump’...   \n",
              "9   9  A Back-Channel Plan for Ukraine and Russia, Co...   \n",
              "\n",
              "                         author  \\\n",
              "0                 Darrell Lucus   \n",
              "1               Daniel J. Flynn   \n",
              "2            Consortiumnews.com   \n",
              "3               Jessica Purkiss   \n",
              "4                Howard Portnoy   \n",
              "5               Daniel Nussbaum   \n",
              "6                           NaN   \n",
              "7               Alissa J. Rubin   \n",
              "8                           NaN   \n",
              "9  Megan Twohey and Scott Shane   \n",
              "\n",
              "                                                text  label  \n",
              "0  House Dem Aide: We Didn’t Even See Comey’s Let...      1  \n",
              "1  Ever get the feeling your life circles the rou...      0  \n",
              "2  Why the Truth Might Get You Fired October 29, ...      1  \n",
              "3  Videos 15 Civilians Killed In Single US Airstr...      1  \n",
              "4  Print \\nAn Iranian woman has been sentenced to...      1  \n",
              "5  In these trying times, Jackie Mason is the Voi...      0  \n",
              "6  Ever wonder how Britain’s most iconic pop pian...      1  \n",
              "7  PARIS  —   France chose an idealistic, traditi...      0  \n",
              "8  Donald J. Trump is scheduled to make a highly ...      0  \n",
              "9  A week before Michael T. Flynn resigned as nat...      0  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-df13fcf0-faa0-4ca1-9291-26b3a42952d5\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>id</th>\n",
              "      <th>title</th>\n",
              "      <th>author</th>\n",
              "      <th>text</th>\n",
              "      <th>label</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>House Dem Aide: We Didn’t Even See Comey’s Let...</td>\n",
              "      <td>Darrell Lucus</td>\n",
              "      <td>House Dem Aide: We Didn’t Even See Comey’s Let...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>FLYNN: Hillary Clinton, Big Woman on Campus - ...</td>\n",
              "      <td>Daniel J. Flynn</td>\n",
              "      <td>Ever get the feeling your life circles the rou...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2</td>\n",
              "      <td>Why the Truth Might Get You Fired</td>\n",
              "      <td>Consortiumnews.com</td>\n",
              "      <td>Why the Truth Might Get You Fired October 29, ...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>3</td>\n",
              "      <td>15 Civilians Killed In Single US Airstrike Hav...</td>\n",
              "      <td>Jessica Purkiss</td>\n",
              "      <td>Videos 15 Civilians Killed In Single US Airstr...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>4</td>\n",
              "      <td>Iranian woman jailed for fictional unpublished...</td>\n",
              "      <td>Howard Portnoy</td>\n",
              "      <td>Print \\nAn Iranian woman has been sentenced to...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>5</td>\n",
              "      <td>Jackie Mason: Hollywood Would Love Trump if He...</td>\n",
              "      <td>Daniel Nussbaum</td>\n",
              "      <td>In these trying times, Jackie Mason is the Voi...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>6</td>\n",
              "      <td>Life: Life Of Luxury: Elton John’s 6 Favorite ...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Ever wonder how Britain’s most iconic pop pian...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>7</td>\n",
              "      <td>Benoît Hamon Wins French Socialist Party’s Pre...</td>\n",
              "      <td>Alissa J. Rubin</td>\n",
              "      <td>PARIS  —   France chose an idealistic, traditi...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>8</td>\n",
              "      <td>Excerpts From a Draft Script for Donald Trump’...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Donald J. Trump is scheduled to make a highly ...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>9</td>\n",
              "      <td>A Back-Channel Plan for Ukraine and Russia, Co...</td>\n",
              "      <td>Megan Twohey and Scott Shane</td>\n",
              "      <td>A week before Michael T. Flynn resigned as nat...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-df13fcf0-faa0-4ca1-9291-26b3a42952d5')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-df13fcf0-faa0-4ca1-9291-26b3a42952d5 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-df13fcf0-faa0-4ca1-9291-26b3a42952d5');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# rows and columns in datasets\n",
        "dataset.shape"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jis-qZXKXjlf",
        "outputId": "98b40230-101b-4c5d-97c6-803363688269"
      },
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(20800, 5)"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# checking the number of missing values in dataset\n",
        "\n",
        "dataset.isnull().sum()\n",
        "# column wise missing data"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "49R8l3SvYCbG",
        "outputId": "c3636b70-621f-43c0-c0b3-cfec4993c393"
      },
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "id           0\n",
              "title      558\n",
              "author    1957\n",
              "text        39\n",
              "label        0\n",
              "dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# as we have more than 20800 we can drop rows having missing values\n",
        "\n",
        "# replacing null values with empty string\n",
        "\n",
        "dataset1 = dataset.fillna(\"\") "
      ],
      "metadata": {
        "id": "A5jLKWclYCYS"
      },
      "execution_count": 14,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "dataset1.isnull().sum()\n",
        "#  there is no null value in dataset1 as we replaced null values with empty string"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TngBUK_VYWZh",
        "outputId": "0653cf68-831c-4881-fa00-908cbe5c1b5e"
      },
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "id        0\n",
              "title     0\n",
              "author    0\n",
              "text      0\n",
              "label     0\n",
              "dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# merging the author name and news title\n",
        "dataset1['content'] = dataset1['author'] +' '+dataset1['title']\n"
      ],
      "metadata": {
        "id": "TJtQOxY6YWXY"
      },
      "execution_count": 18,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(dataset1['content'].head())"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "p5D3eMUwYWVA",
        "outputId": "4f01d49c-5f1c-4ae5-84e2-bd638b11fbb3"
      },
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "0    Darrell Lucus House Dem Aide: We Didn’t Even S...\n",
            "1    Daniel J. Flynn FLYNN: Hillary Clinton, Big Wo...\n",
            "2    Consortiumnews.com Why the Truth Might Get You...\n",
            "3    Jessica Purkiss 15 Civilians Killed In Single ...\n",
            "4    Howard Portnoy Iranian woman jailed for fictio...\n",
            "Name: content, dtype: object\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "1KhpFNksYWSn"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Separating the data and label\n",
        "X = dataset1.drop('label', axis =1)\n",
        "Y = dataset1['label']\n",
        "X.head()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "lHe0u_6FYCV_",
        "outputId": "9660ce9f-67e6-42ff-8d53-a5285c36eb12"
      },
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   id                                              title              author  \\\n",
              "0   0  House Dem Aide: We Didn’t Even See Comey’s Let...       Darrell Lucus   \n",
              "1   1  FLYNN: Hillary Clinton, Big Woman on Campus - ...     Daniel J. Flynn   \n",
              "2   2                  Why the Truth Might Get You Fired  Consortiumnews.com   \n",
              "3   3  15 Civilians Killed In Single US Airstrike Hav...     Jessica Purkiss   \n",
              "4   4  Iranian woman jailed for fictional unpublished...      Howard Portnoy   \n",
              "\n",
              "                                                text  \\\n",
              "0  House Dem Aide: We Didn’t Even See Comey’s Let...   \n",
              "1  Ever get the feeling your life circles the rou...   \n",
              "2  Why the Truth Might Get You Fired October 29, ...   \n",
              "3  Videos 15 Civilians Killed In Single US Airstr...   \n",
              "4  Print \\nAn Iranian woman has been sentenced to...   \n",
              "\n",
              "                                             content  \n",
              "0  Darrell Lucus House Dem Aide: We Didn’t Even S...  \n",
              "1  Daniel J. Flynn FLYNN: Hillary Clinton, Big Wo...  \n",
              "2  Consortiumnews.com Why the Truth Might Get You...  \n",
              "3  Jessica Purkiss 15 Civilians Killed In Single ...  \n",
              "4  Howard Portnoy Iranian woman jailed for fictio...  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-94e165b5-d743-4729-a2be-6a4376614cca\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>id</th>\n",
              "      <th>title</th>\n",
              "      <th>author</th>\n",
              "      <th>text</th>\n",
              "      <th>content</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>House Dem Aide: We Didn’t Even See Comey’s Let...</td>\n",
              "      <td>Darrell Lucus</td>\n",
              "      <td>House Dem Aide: We Didn’t Even See Comey’s Let...</td>\n",
              "      <td>Darrell Lucus House Dem Aide: We Didn’t Even S...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>FLYNN: Hillary Clinton, Big Woman on Campus - ...</td>\n",
              "      <td>Daniel J. Flynn</td>\n",
              "      <td>Ever get the feeling your life circles the rou...</td>\n",
              "      <td>Daniel J. Flynn FLYNN: Hillary Clinton, Big Wo...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2</td>\n",
              "      <td>Why the Truth Might Get You Fired</td>\n",
              "      <td>Consortiumnews.com</td>\n",
              "      <td>Why the Truth Might Get You Fired October 29, ...</td>\n",
              "      <td>Consortiumnews.com Why the Truth Might Get You...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>3</td>\n",
              "      <td>15 Civilians Killed In Single US Airstrike Hav...</td>\n",
              "      <td>Jessica Purkiss</td>\n",
              "      <td>Videos 15 Civilians Killed In Single US Airstr...</td>\n",
              "      <td>Jessica Purkiss 15 Civilians Killed In Single ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>4</td>\n",
              "      <td>Iranian woman jailed for fictional unpublished...</td>\n",
              "      <td>Howard Portnoy</td>\n",
              "      <td>Print \\nAn Iranian woman has been sentenced to...</td>\n",
              "      <td>Howard Portnoy Iranian woman jailed for fictio...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-94e165b5-d743-4729-a2be-6a4376614cca')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-94e165b5-d743-4729-a2be-6a4376614cca button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-94e165b5-d743-4729-a2be-6a4376614cca');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "Y.head()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mDQ3ybC2egL0",
        "outputId": "ed272a7f-4d6c-4252-d775-6fe2ef39847c"
      },
      "execution_count": 26,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0    1\n",
              "1    0\n",
              "2    1\n",
              "3    1\n",
              "4    1\n",
              "Name: label, dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(X.shape,Y.shape)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YuJZxTBGegJf",
        "outputId": "d61d147b-557c-4250-a822-81630c2dd9b0"
      },
      "execution_count": 27,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(20800, 5) (20800,)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Stemming\n",
        "\n",
        "> Steeming is the process of reducing a word to its rootword\n",
        "\n",
        "ex: \n",
        "  actor, actess, acting --> act\n",
        "\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "tHzzvJR_fJSi"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "port_stem = PorterStemmer()"
      ],
      "metadata": {
        "id": "pwQKhn5LegG2"
      },
      "execution_count": 28,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "def stemming(content):\n",
        "  stemmed_content = re.sub('[^a-zA-Z]', ' ', content)\n",
        "  # sub substitues some values, numbers and space will be substituted by space in content\n",
        "  # it is a regular expression syntax\n",
        "  stemmed_content = stemmed_content.lower()\n",
        "  # makes words to lowercase\n",
        "  stemmed_content = stemmed_content.split()\n",
        "  # splits the content \n",
        "  stemmed_content = [port_stem.stem(word) for word in stemmed_content if not word in stopwords.words('english')]\n",
        "  stemmed_content = ' '.join(stemmed_content)\n",
        "  return stemmed_content\n"
      ],
      "metadata": {
        "id": "NHK95mZmegEC"
      },
      "execution_count": 34,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "dataset1['content'] = dataset1['content'].apply(stemming)"
      ],
      "metadata": {
        "id": "e853JYhYhrAe"
      },
      "execution_count": 35,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(dataset1['content'])\n",
        "# now dataset has all lowercase letters with no numerical values\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lUmfqabvhq9w",
        "outputId": "70e6aeba-5aaf-4548-d4db-cabc3ab1dd3d"
      },
      "execution_count": 37,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "0        darrel lucu hous dem aid even see comey letter...\n",
            "1        daniel j flynn flynn hillari clinton big woman...\n",
            "2                   consortiumnew com truth might get fire\n",
            "3        jessica purkiss civilian kill singl us airstri...\n",
            "4        howard portnoy iranian woman jail fiction unpu...\n",
            "                               ...                        \n",
            "20795    jerom hudson rapper trump poster child white s...\n",
            "20796    benjamin hoffman n f l playoff schedul matchup...\n",
            "20797    michael j de la merc rachel abram maci said re...\n",
            "20798    alex ansari nato russia hold parallel exercis ...\n",
            "20799                            david swanson keep f aliv\n",
            "Name: content, Length: 20800, dtype: object\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# seperating the data and label\n",
        "X = dataset1['content'].values\n",
        "Y = dataset1['label'].values"
      ],
      "metadata": {
        "id": "6SSqoH-DegB3"
      },
      "execution_count": 38,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(X)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zr4poRO2YCTm",
        "outputId": "84e76c93-32d6-478d-df1d-f83f3cbf94c3"
      },
      "execution_count": 43,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "['darrel lucu hous dem aid even see comey letter jason chaffetz tweet'\n",
            " 'daniel j flynn flynn hillari clinton big woman campu breitbart'\n",
            " 'consortiumnew com truth might get fire' ...\n",
            " 'michael j de la merc rachel abram maci said receiv takeov approach hudson bay new york time'\n",
            " 'alex ansari nato russia hold parallel exercis balkan'\n",
            " 'david swanson keep f aliv']\n",
            "[1 0 1 ... 0 1 1]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(Y)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SXjyqitPYCRC",
        "outputId": "ddb5905f-5d35-4c87-efc6-3d33bfc36cf2"
      },
      "execution_count": 47,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[1 0 1 ... 0 1 1]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "Y.shape\n",
        "# shape for Y"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kLLTB12oYCOf",
        "outputId": "19a29ebb-f886-408e-b3f8-69a9c1c47f88"
      },
      "execution_count": 46,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(20800,)"
            ]
          },
          "metadata": {},
          "execution_count": 46
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# converting the textual data to numeric data\n",
        "# Tf means term frequency and idf is inverse document frequency\n",
        "vector = TfidfVectorizer()\n",
        "vector.fit(X)\n",
        "\n",
        "X = vector.transform(X)"
      ],
      "metadata": {
        "id": "Xg0po1ZilqEZ"
      },
      "execution_count": 50,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(X)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "57s3wgI4lqBt",
        "outputId": "a68a4d49-3b03-4e1b-b2ce-e7ff0e1be959"
      },
      "execution_count": 51,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "  (0, 15686)\t0.28485063562728646\n",
            "  (0, 13473)\t0.2565896679337957\n",
            "  (0, 8909)\t0.3635963806326075\n",
            "  (0, 8630)\t0.29212514087043684\n",
            "  (0, 7692)\t0.24785219520671603\n",
            "  (0, 7005)\t0.21874169089359144\n",
            "  (0, 4973)\t0.233316966909351\n",
            "  (0, 3792)\t0.2705332480845492\n",
            "  (0, 3600)\t0.3598939188262559\n",
            "  (0, 2959)\t0.2468450128533713\n",
            "  (0, 2483)\t0.3676519686797209\n",
            "  (0, 267)\t0.27010124977708766\n",
            "  (1, 16799)\t0.30071745655510157\n",
            "  (1, 6816)\t0.1904660198296849\n",
            "  (1, 5503)\t0.7143299355715573\n",
            "  (1, 3568)\t0.26373768806048464\n",
            "  (1, 2813)\t0.19094574062359204\n",
            "  (1, 2223)\t0.3827320386859759\n",
            "  (1, 1894)\t0.15521974226349364\n",
            "  (1, 1497)\t0.2939891562094648\n",
            "  (2, 15611)\t0.41544962664721613\n",
            "  (2, 9620)\t0.49351492943649944\n",
            "  (2, 5968)\t0.3474613386728292\n",
            "  (2, 5389)\t0.3866530551182615\n",
            "  (2, 3103)\t0.46097489583229645\n",
            "  :\t:\n",
            "  (20797, 13122)\t0.2482526352197606\n",
            "  (20797, 12344)\t0.27263457663336677\n",
            "  (20797, 12138)\t0.24778257724396507\n",
            "  (20797, 10306)\t0.08038079000566466\n",
            "  (20797, 9588)\t0.174553480255222\n",
            "  (20797, 9518)\t0.2954204003420313\n",
            "  (20797, 8988)\t0.36160868928090795\n",
            "  (20797, 8364)\t0.22322585870464118\n",
            "  (20797, 7042)\t0.21799048897828688\n",
            "  (20797, 3643)\t0.21155500613623743\n",
            "  (20797, 1287)\t0.33538056804139865\n",
            "  (20797, 699)\t0.30685846079762347\n",
            "  (20797, 43)\t0.29710241860700626\n",
            "  (20798, 13046)\t0.22363267488270608\n",
            "  (20798, 11052)\t0.4460515589182236\n",
            "  (20798, 10177)\t0.3192496370187028\n",
            "  (20798, 6889)\t0.32496285694299426\n",
            "  (20798, 5032)\t0.4083701450239529\n",
            "  (20798, 1125)\t0.4460515589182236\n",
            "  (20798, 588)\t0.3112141524638974\n",
            "  (20798, 350)\t0.28446937819072576\n",
            "  (20799, 14852)\t0.5677577267055112\n",
            "  (20799, 8036)\t0.45983893273780013\n",
            "  (20799, 3623)\t0.37927626273066584\n",
            "  (20799, 377)\t0.5677577267055112\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Splitting the data into training and test data"
      ],
      "metadata": {
        "id": "6pBMoOK5mffy"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "X_train, X_test, Y_train, Y_test = train_test_split(X,Y, test_size = 0.2, stratify = Y, random_state =2)"
      ],
      "metadata": {
        "id": "SaIZKhGAlp_J"
      },
      "execution_count": 75,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Training model using logistic regression model"
      ],
      "metadata": {
        "id": "8o8XSUcDoqaS"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "model = LogisticRegression()"
      ],
      "metadata": {
        "id": "HXKHX40ilp8V"
      },
      "execution_count": 76,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model.fit(X_train, Y_train)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "h0Ff3rX4oual",
        "outputId": "99e14957-cd79-4093-994a-41d4ebb38811"
      },
      "execution_count": 77,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "LogisticRegression()"
            ]
          },
          "metadata": {},
          "execution_count": 77
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Model Evaluation"
      ],
      "metadata": {
        "id": "_GNf5O_Tp7Fy"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# accuracy score on training data\n",
        "X_train_prediction = model.predict(X_train)\n",
        "training_data_accuracy = accuracy_score(X_train_prediction, Y_train )"
      ],
      "metadata": {
        "id": "K4ok7RxJouYN"
      },
      "execution_count": 78,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(\" Accuracy score for the traing data : \",training_data_accuracy)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3BZQKUVhouWD",
        "outputId": "e7f867e8-09f0-4d41-fa10-7d5b47297ab9"
      },
      "execution_count": 79,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " Accuracy score for the traing data :  0.9865985576923076\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# accuracy score on test data\n",
        "X_test_prediction = model.predict(X_test)\n",
        "test_data_accuracy = accuracy_score(X_test_prediction, Y_test )"
      ],
      "metadata": {
        "id": "pWeWDnJulp6H"
      },
      "execution_count": 80,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(\" Accuracy score for the testing data : \",test_data_accuracy)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "X2IhwC3Wlp3L",
        "outputId": "cc554156-9044-4d86-8cdb-29d680d82cfa"
      },
      "execution_count": 81,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " Accuracy score for the testing data :  0.9790865384615385\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Making a predicting system "
      ],
      "metadata": {
        "id": "fs49QuyTsAvW"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "i =0\n",
        "X_new = X_test[i]\n",
        "\n",
        "prediction = model.predict(X_new)\n",
        "print(\"Prediction Result : \",prediction)\n",
        "\n",
        "if(prediction[i] == 0):\n",
        "  print(\"This is a Real news\")\n",
        "else:\n",
        "  print(\"This news is Fake!!!\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "QxZJoQ1blp0j",
        "outputId": "feec4300-65d1-41a0-deea-c15d90493856"
      },
      "execution_count": 106,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Prediction Result :  [1]\n",
            "This news is Fake!!!\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(\"Actual state of news : \", Y_test[i])\n",
        "\n",
        "if(prediction[i] == Y_test[i]):\n",
        "  print(\"Both Predicted and Actual values are same !!!\")\n",
        "else:\n",
        "  print(\"Predicted value differs from Actual values\")\n",
        "# if Y_test[0] is equal to predicted output by the model then the prediction made by model is correct\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lxnbopkGYCL_",
        "outputId": "49827328-dab7-44bf-fad0-9057a04e4bd8"
      },
      "execution_count": 107,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Actual state of news :  1\n",
            "Both Predicted and Actual values are same !!!\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [],
      "metadata": {
        "id": "thjYMo54KA61"
      }
    }
  ]
}