2_PreTrained_Model_for_Tagging.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "2. PreTrained Model for Tagging.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/navneetkrc/Flair_SOTA_NLP/blob/master/2_PreTrained_Model_for_Tagging.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "metadata": {
        "id": "CQpqE1VyNHlt",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Install Dependencies"
      ]
    },
    {
      "metadata": {
        "id": "rVbCDs6wMdk-",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "!pip install torch_nightly -f https://download.pytorch.org/whl/nightly/cu92/torch_nightly.html\n",
        "!pip install flair"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "1Nj7O9_2sumr",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "from flair.data import Sentence\n",
        "from flair.models import SequenceTagger"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "NnfZaO83NB50",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "#1. NLP Base Types and tasks"
      ]
    },
    {
      "metadata": {
        "id": "PT__1hBNNlzt",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "##Check for some basics tasks"
      ]
    },
    {
      "metadata": {
        "id": "tRNSgtNbNTPe",
        "colab_type": "code",
        "outputId": "44454d49-2612-4119-8686-5052ba42d134",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "# make a sentence\n",
        "sentence = Sentence('I love Amsterdam .')\n",
        "\n",
        "# load the NER tagger\n",
        "tagger = SequenceTagger.load('ner')\n",
        "\n",
        "# run NER over sentence\n",
        "tagger.predict(sentence)"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[Sentence: \"I love Amsterdam .\" - 4 Tokens]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        }
      ]
    },
    {
      "metadata": {
        "id": "RCosXqIXNkFk",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Done! The Sentence now has entity annotations. Print the sentence to see what the tagger found.\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "2yq-9aAqNs6Z",
        "colab_type": "code",
        "outputId": "f15dbc3e-a9e9-492c-d676-0e1701c7d49a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        }
      },
      "cell_type": "code",
      "source": [
        "print(sentence)\n",
        "print('The following NER tags are found:')\n",
        "\n",
        "# iterate over entities and print\n",
        "for entity in sentence.get_spans('ner'):\n",
        "    print(entity)"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Sentence: \"I love Amsterdam .\" - 4 Tokens\n",
            "The following NER tags are found:\n",
            "LOC-span [3]: \"Amsterdam\"\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "CvaCgDSiN1OD",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "This should print:\n",
        "\n",
        "Sentence: \"I love Amsterdam .\" - 4 Tokens\n",
        "\n",
        "The following NER tags are found:\n",
        "\n",
        "LOC-span [3]: \"Amsterdam\""
      ]
    },
    {
      "metadata": {
        "id": "mBBfJAsstiqb",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "\n",
        "\n",
        "---\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "UCI0qLKStlNi",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "##Basic1- Creating a Sentence"
      ]
    },
    {
      "metadata": {
        "id": "X4GnV6DRNx4F",
        "colab_type": "code",
        "outputId": "58001c3b-4260-44b0-a5c3-ee1dd8a85223",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "# The sentence objects holds a sentence that we may want to embed or tag\n",
        "from flair.data import Sentence\n",
        "\n",
        "# Make a sentence object by passing a whitespace tokenized string\n",
        "sentence = Sentence('The grass is green .')\n",
        "\n",
        "# Print the object to see what's in there\n",
        "print(sentence)\n",
        "\n",
        "#expected output-> Sentence: \"The grass is green .\" - 5 Tokens"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Sentence: \"The grass is green .\" - 5 Tokens\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "eJ4Nm174txiC",
        "colab_type": "code",
        "outputId": "7d0567d6-1c87-4612-e331-3bf53580d036",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "cell_type": "code",
      "source": [
        "##The print-out tells us that the sentence consists of 5 tokens. You can access the tokens of a sentence via their token id or with their index:\n",
        "\n",
        "# using the token id\n",
        "print(sentence.get_token(4))\n",
        "\n",
        "# using the index itself\n",
        "print(sentence[3])"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Token: 4 green\n",
            "Token: 4 green\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "RM4q50pxul97",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "For both the cases it prints:\n",
        "\n",
        "Token: 4 green\n",
        "\n",
        "Token: 4 green"
      ]
    },
    {
      "metadata": {
        "id": "kjOzaUrTuh3B",
        "colab_type": "code",
        "outputId": "7a3cca12-eb2a-4ca4-9e75-c93c2a33c644",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "cell_type": "code",
      "source": [
        "for token in sentence:\n",
        "    print(token)"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Token: 1 The\n",
            "Token: 2 grass\n",
            "Token: 3 is\n",
            "Token: 4 green\n",
            "Token: 5 .\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "v2eaZhhFu0_0",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "This should print:\n",
        "\n",
        "Token: 1 The\n",
        "\n",
        "Token: 2 grass\n",
        "\n",
        "Token: 3 is\n",
        "\n",
        "Token: 4 green\n",
        "\n",
        "Token: 5 ."
      ]
    },
    {
      "metadata": {
        "id": "oOG7eJmlvNBZ",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "##Tokenization"
      ]
    },
    {
      "metadata": {
        "id": "6mH4Oyw7vUkM",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "In some use cases, you might not have your text already tokenized. For this case, we added a simple tokenizer using the lightweight segtok library.\n",
        "\n",
        "Simply use the use_tokenizer flag when instantiating your Sentence with an untokenized string:\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "hM31xEUvvO2n",
        "colab_type": "code",
        "outputId": "40799d7d-7f40-4d17-8bfe-0a5564078da5",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "from flair.data import Sentence\n",
        "\n",
        "# Make a sentence object by passing an untokenized string and the 'use_tokenizer' flag\n",
        "sentence = Sentence('The grass is green.', use_tokenizer=True)\n",
        "\n",
        "# Print the object to see what's in there\n",
        "print(sentence)"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Sentence: \"The grass is green .\" - 5 Tokens\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "quIeE-uYvcgd",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "This should print \n",
        "\n",
        "Sentence: \"The grass is green .\" - 5 Tokens"
      ]
    },
    {
      "metadata": {
        "id": "6QnosUJnvjMX",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "###Adding Tags to tokens"
      ]
    },
    {
      "metadata": {
        "id": "MV8czKK5vYw6",
        "colab_type": "code",
        "outputId": "c7e4fb52-e402-4eaf-b1a0-f85e79e79184",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "# add a tag to a word in the sentence\n",
        "sentence[3].add_tag('ner', 'color')\n",
        "\n",
        "# print the sentence with all tags of this type\n",
        "print(sentence.to_tagged_string())"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "The grass is green <color> .\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "0ox4yCx5vrUH",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "this should print:\n",
        "\n",
        "The grass is green <color> ."
      ]
    },
    {
      "metadata": {
        "id": "yaoM1bgiv6BI",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Each tag is of class Label which next to the value has a score indicating confidence. Print like this:\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "HgxsseWUvpwf",
        "colab_type": "code",
        "outputId": "70bb1abc-91d1-414d-d2bc-046638226f18",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "from flair.data import Label\n",
        "\n",
        "tag: Label = sentence[3].get_tag('ner')\n",
        "\n",
        "print(f'\"{sentence[3]}\" is tagged as \"{tag.value}\" with confidence score \"{tag.score}\"')"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\"Token: 4 green\" is tagged as \"color\" with confidence score \"1.0\"\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "1aUW3t8pwEQj",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "This should print:\n",
        "\n",
        "\"Token: 4 green\" is tagged as \"color\" with confidence score \"1.0\"\n",
        "\n",
        "Also our color tag has a score of 1.0 since we manually added it. If a tag is predicted by our sequence labeler, the score value will indicate classifier confidence.\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "bN-AcmmhwQny",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "###Adding Labels to Sentences"
      ]
    },
    {
      "metadata": {
        "id": "wdor7-v0wXGK",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "A Sentence can have one or multiple labels that can for example be used in text classification tasks. For instance, the example below shows how we add the label 'sports' to a sentence, thereby labeling it as belonging to the sports category."
      ]
    },
    {
      "metadata": {
        "id": "X7ic6mu3v-rS",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "sentence = Sentence('France is the current world cup winner.')\n",
        "\n",
        "# add a label to a sentence\n",
        "sentence.add_label('sports')\n",
        "\n",
        "# a sentence can also belong to multiple classes\n",
        "sentence.add_labels(['sports', 'world cup'])\n",
        "\n",
        "# you can also set the labels while initializing the sentence\n",
        "sentence = Sentence('France is the current world cup winner.', labels=['sports', 'world cup'])\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "_msAzDtRwfJp",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Labels are also of the Label class. So, you can print a sentence's labels like this:\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "JR7ECzktwgRL",
        "colab_type": "code",
        "outputId": "edf79ad0-b52e-45f6-87a3-da64969d2de8",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        }
      },
      "cell_type": "code",
      "source": [
        "sentence = Sentence('France is the current world cup winner.', labels=['sports', 'world cup'])\n",
        "\n",
        "print(sentence)\n",
        "for label in sentence.labels:\n",
        "    print(label)"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Sentence: \"France is the current world cup winner.\" - 7 Tokens\n",
            "sports (1.0)\n",
            "world cup (1.0)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "N-nYr7Rrwpga",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "This should print:\n",
        "\n",
        "sports (1.0)\n",
        "\n",
        "world cup (1.0)\n",
        "\n",
        "**This indicates that the sentence belongs to these two classes, each with confidence score 1.0.**\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "i0pWs5aoyAyb",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "\n",
        "\n",
        "---\n",
        "\n",
        "**In next column we will check about the use of pre-trained model for tagging our text in the next segment.**"
      ]
    },
    {
      "metadata": {
        "id": "W15ZynHSyKlq",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "#2. Use of Pretrained model to tag your text"
      ]
    },
    {
      "metadata": {
        "id": "aFGl3XVhD153",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "##Tutorial 2: Tagging your Text\n",
        "Here, we show how to use our pre-trained models to tag your text data.\n",
        "\n",
        "Tagging with Pre-Trained Sequence Tagging Models\n",
        "Let's use a pre-trained model for named entity recognition (NER). \n",
        "\n",
        "This model was trained over the English CoNLL-03 task and can recognize 4 different entity types."
      ]
    },
    {
      "metadata": {
        "id": "OuqEEpBLD28t",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "from flair.models import SequenceTagger\n",
        "\n",
        "tagger = SequenceTagger.load('ner')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "h8Oolc26EJhf",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "All you need to do is use the predict() method of the tagger on a sentence. \n",
        "\n",
        "This will add predicted tags to the tokens in the sentence. Lets use a sentence with two named entities:\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "HgV_hQ9AFAk0",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "3272c722-319b-4ba8-eae8-9f72daa23c91"
      },
      "cell_type": "code",
      "source": [
        "sentence = Sentence('George Washington went to Washington .')\n",
        "\n",
        "# predict NER tags\n",
        "tagger.predict(sentence)\n",
        "\n",
        "# print sentence with predicted tags\n",
        "print(sentence.to_tagged_string())"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "George <B-PER> Washington <E-PER> went to Washington <S-LOC> .\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "KTQpBWCRFHlf",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "This should print:\n",
        "\n",
        "George <B-PER> Washington <E-PER> went to Washington <S-LOC> . \n"
      ]
    },
    {
      "metadata": {
        "id": "eAXhravDGIKy",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "###Getting Annotated Spans\n",
        "Many sequence labeling methods annotate spans that consist of multiple words, such as \"George Washington\" in our example sentence. You can directly get such spans in a tagged sentence like this:\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "QlPhWjGqGSVb",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        },
        "outputId": "71f48f54-471d-4573-fc60-6e216c5e72e7"
      },
      "cell_type": "code",
      "source": [
        "for entity in sentence.get_spans('ner'):\n",
        "    print(entity)"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "PER-span [1,2]: \"George Washington\"\n",
            "LOC-span [5]: \"Washington\"\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "MboeMjBRGVnv",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "This should print:\n",
        "\n",
        "PER-span [1,2]: \"George Washington\"\n",
        "\n",
        "LOC-span [5]: \"Washington\"\n",
        "\n",
        "Which indicates that \"George Washington\" is a person (PER) and \"Washington\" is a location (LOC). Each such Span has a text, a tag value, its position in the sentence and \"score\" that indicates how confident the tagger is that the prediction is correct. You can also get additional information, such as the position offsets of each entity in the sentence by calling:\n"
      ]
    },
    {
      "metadata": {
        "id": "xyVwVDGBGqKf",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 54
        },
        "outputId": "940a50e4-4b03-4aec-fe2b-257e2a3cd890"
      },
      "cell_type": "code",
      "source": [
        "print(sentence.to_dict(tag_type='ner'))"
      ],
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "{'text': 'George Washington went to Washington .', 'labels': [], 'entities': [{'text': 'George Washington', 'start_pos': 0, 'end_pos': 17, 'type': 'PER', 'confidence': 0.9999276995658875}, {'text': 'Washington', 'start_pos': 26, 'end_pos': 36, 'type': 'LOC', 'confidence': 0.9988662004470825}]}\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "8W2AkLhFG2Sa",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "\n",
        "\n",
        "This should print:\n",
        "\n",
        "{'text': 'George Washington went to Washington .',\n",
        "    'entities': [      \n",
        "        {'text': 'George Washington', 'start_pos': 0, 'end_pos': 17, 'type': 'PER', 'confidence': 0.999},        \n",
        "        {'text': 'Washington', 'start_pos': 26, 'end_pos': 36, 'type': 'LOC', 'confidence': 0.998}\n",
        "    ]}"
      ]
    },
    {
      "metadata": {
        "id": "uy4vD58PJm3u",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "##Tagging Multilingual Text\n",
        "If you have text in many languages (such as English and German), you can use our new multilingual models:\n",
        "\n",
        "Same approach I will try to use the same for HINGLISH (Hindi+English) Dataset as well\n"
      ]
    },
    {
      "metadata": {
        "id": "wx6wq8pfFEe-",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "00fad159-d378-429b-b534-2a9d910c0f3b"
      },
      "cell_type": "code",
      "source": [
        "# load model\n",
        "tagger = SequenceTagger.load('pos-multi')\n",
        "\n",
        "# text with English and German sentences\n",
        "sentence = Sentence('George Washington went to Washington . Dort kaufte er einen Hut .')\n",
        "\n",
        "# predict PoS tags\n",
        "tagger.predict(sentence)\n",
        "\n",
        "# print sentence with predicted tags\n",
        "print(sentence.to_tagged_string())"
      ],
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "George <PROPN> Washington <PROPN> went <VERB> to <ADP> Washington <PROPN> . <PUNCT> Dort <ADV> kaufte <VERB> er <PRON> einen <DET> Hut <NOUN> . <PUNCT>\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "Ad8dDhdtKWOi",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "This should print:\n",
        "\n",
        "George <PROPN> Washington <PROPN> went <VERB> to <ADP> Washington <PROPN> . <PUNCT>\n",
        "\n",
        "Dort <ADV> kaufte <VERB> er <PRON> einen <DET> Hut <NOUN> . <PUNCT>\n",
        "So, both 'went' and 'kaufte' are identified as VERBs in these sentences.\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "ydVYuyg7K2vj",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "##Experimental: Semantic Frame Detection\n",
        "For English, we provide a pre-trained model that detects semantic frames in text, trained using Propbank 3.0 frames. This provides a sort of word sense disambiguation for frame evoking words, and we are curious what researchers might do with this.\n",
        "\n",
        "Here's an example:\n"
      ]
    },
    {
      "metadata": {
        "id": "1LAyt6BOK__m",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        },
        "outputId": "7a93739e-bf36-400d-ff9d-c2cbc9bf1ec2"
      },
      "cell_type": "code",
      "source": [
        "# load model\n",
        "tagger = SequenceTagger.load('frame')\n",
        "\n",
        "# make German sentence\n",
        "sentence_1 = Sentence('George returned to Berlin to return his hat .')\n",
        "sentence_2 = Sentence('He had a look at different hats .')\n",
        "\n",
        "# predict NER tags\n",
        "tagger.predict(sentence_1)\n",
        "tagger.predict(sentence_2)\n",
        "\n",
        "# print sentence with predicted tags\n",
        "print(sentence_1.to_tagged_string())\n",
        "print(sentence_2.to_tagged_string())"
      ],
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "George returned <return.01> to Berlin to return <return.02> his hat .\n",
            "He had <have.LV> a look <look.01> at different hats .\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "gJ8eg5xCLDSI",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "This should print:\n",
        "\n",
        "George returned <return.01> to Berlin to return <return.02> his hat .\n",
        "\n",
        "He had <have.LV> a look <look.01> at different hats .\n"
      ]
    },
    {
      "metadata": {
        "id": "sZSopQEmLgdZ",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "\n",
        "\n",
        "\n",
        "As we can see, the frame detector makes a distinction in sentence 1 between two different meanings of the word 'return'. 'return.01' means returning to a location, while 'return.02' means giving something back.\n",
        "\n",
        "Similarly, in sentence 2 the frame detector finds a light verb construction in which 'have' is the light verb and 'look' is a frame evoking word.\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "LQ6b3zlrLseG",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "##Tagging a List of Sentences\n",
        "Often, you may want to tag an entire text corpus. In this case, you need to split the corpus into sentences and pass a list of Sentence objects to the .predict() method.\n",
        "\n",
        "For instance, you can use the sentence splitter of segtok to split your text:\n"
      ]
    },
    {
      "metadata": {
        "id": "-WJM2IigKVcv",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        },
        "outputId": "865d7bb6-b4bc-458f-b3f9-6371647f762c"
      },
      "cell_type": "code",
      "source": [
        "# your text of many sentences\n",
        "text = \"This is a sentence. This is another sentence. I love Berlin.\"\n",
        "\n",
        "# use a library to split into sentences\n",
        "from segtok.segmenter import split_single\n",
        "sentences = [Sentence(sent, use_tokenizer=True) for sent in split_single(text)]\n",
        "\n",
        "# predict tags for list of sentences\n",
        "tagger: SequenceTagger = SequenceTagger.load('ner')\n",
        "tagger.predict(sentences)\n"
      ],
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[Sentence: \"This is a sentence .\" - 5 Tokens,\n",
              " Sentence: \"This is another sentence .\" - 5 Tokens,\n",
              " Sentence: \"I love Berlin .\" - 4 Tokens]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 24
        }
      ]
    },
    {
      "metadata": {
        "id": "4eK7GKu1L-g6",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "\n",
        "Using the mini_batch_size parameter of the .predict() method, you can set the size of mini batches passed to the tagger. Depending on your resources, you might want to play around with this parameter to optimize speed.\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "U468xUPVM3V_",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "##Tagging with Pre-Trained Text Classification Models\n",
        "Let's use a pre-trained model for detecting positive or negative comments. This model was trained over the IMDB dataset and can recognize positive and negative sentiment in English text.\n"
      ]
    },
    {
      "metadata": {
        "id": "03ddv7XbKVlR",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 122
        },
        "outputId": "0f4c9c7b-666a-4ae7-8c1b-ecf24bbbab88"
      },
      "cell_type": "code",
      "source": [
        "\n",
        "from flair.models import TextClassifier\n",
        "\n",
        "classifier = TextClassifier.load('en-sentiment')\n",
        "\n",
        "#All you need to do is use the predict() method of the classifier on a sentence.\n",
        "#This will add the predicted label to the sentence. Lets use a sentence with negative sentiment:\n",
        "\n",
        "sentence = Sentence('This film hurts. It is so bad that I am confused.')\n",
        "\n",
        "# predict NER tags\n",
        "classifier.predict(sentence)\n",
        "\n",
        "# print sentence with predicted labels\n",
        "print(sentence.labels)\n",
        "\n",
        "#This should print:\n",
        "#[NEGATIVE (1.0)]"
      ],
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "2019-01-24 23:32:26,139 https://s3.eu-central-1.amazonaws.com/alan-nlp/resources/models-v0.4/TEXT-CLASSIFICATION_imdb/imdb.pt not found in cache, downloading to /tmp/tmpese0hsa6\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "100%|██████████| 2794252905/2794252905 [04:00<00:00, 11602723.62B/s]"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "2019-01-24 23:36:28,181 copying /tmp/tmpese0hsa6 to cache at /root/.flair/models/imdb.pt\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "2019-01-24 23:36:52,960 removing temp file /tmp/tmpese0hsa6\n",
            "[NEGATIVE (1.0)]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "ETuvGZ8kNfq-",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "\n",
        "\n",
        "---\n",
        "\n",
        "**In next column we will check about the use of word embeddings to embed our text**"
      ]
    },
    {
      "metadata": {
        "id": "rsX-n4ucybqR",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "#3. Use of word Embeddings"
      ]
    },
    {
      "metadata": {
        "id": "0aikssTCyc2c",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "#4. Using Bert, Elmo and Flair Embeddings"
      ]
    },
    {
      "metadata": {
        "id": "zoxEHlxRyd3W",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "#5. Using Document Embeddings"
      ]
    },
    {
      "metadata": {
        "id": "7f9A2uygyd_g",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "#6. Loading your own Corpus"
      ]
    },
    {
      "metadata": {
        "id": "4mtiOsqyyd9l",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "#7. Training your own Model"
      ]
    },
    {
      "metadata": {
        "id": "RGCyj7DRyd7W",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "#8. Optimizing our models"
      ]
    },
    {
      "metadata": {
        "id": "Rj2fT7amzRQV",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "#9. Training your own Flair Embeddings"
      ]
    },
    {
      "metadata": {
        "id": "gGp7lLdszZI4",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "A6yRoTpnzX1I",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "\n",
        "\n",
        "---\n",
        "\n"
      ]
    }
  ]
}