Emotion Analysis of Writers and Readers of Japanese Tweets on Vaccinations.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "[WASSA 2022]  Emotion Analysis of Writers and Readers of Japanese Tweets on Vaccinations.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# Emotion Analysis of Writers and Readers of Japanese Tweets on Vaccinations\n",
        "\n",
        "Patrick Ramos\n",
        "\n",
        "NAPI 2022 Intern\n",
        "\n",
        "---\n",
        "\n",
        "This notebook performs emotional analysis on a collection of Japanese tweets about vaccines and compares the extracted emotions to vaccination rates in Japan. Emotion mining is performed by fine-tuning a pre-trained BERT on a Japanese emotional analysis dataset and using the fine-tuned model and a dataset of tweets about vaccines to infer the emotions of Japanese SNS users towards vaccines.\n",
        "\n",
        "Acknowledgements:\n",
        "\n",
        "* PhD Candidate Kiki Ferawati for the vaccine-related SNS post dataset\n",
        "* Dr. Aramaki, Dr. Wakamiya, Dr. Kong, Ms. Kiki, Mr. Lean, and Mr. Kiyomoto for advice and guidance regarding the project"
      ],
      "metadata": {
        "id": "XebTJRXt-I3y"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Installations, imports and downloads"
      ],
      "metadata": {
        "id": "fkPQjv1j-0cb"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Downloads"
      ],
      "metadata": {
        "id": "Vpk4Yxk5pca2"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "The [WRIME dataset](https://github.com/ids-cv/wrime) will be used for fine-tuning BERT on emotion analysis. The dataset will be explained in more detail later."
      ],
      "metadata": {
        "id": "qdZGQ-3r_ny-"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "zcKYUJCu3YGZ"
      },
      "outputs": [],
      "source": [
        "!wget https://github.com/raw/ids-cv/wrime/master/wrime-ver1.tsv"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Vaccination data is downloaded from [Our World in Data](https://github.com/owid/covid-19-data/tree/master/public/data/vaccinations)."
      ],
      "metadata": {
        "id": "XOL148uYAIZU"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!wget https://github.com/raw/owid/covid-19-data/master/public/data/vaccinations/vaccinations.csv"
      ],
      "metadata": {
        "id": "GXDv0VoeAIIG"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Installations"
      ],
      "metadata": {
        "id": "P-PFo0CIpfME"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "We will need to install [HuggingFace](https://huggingface.co/) for the BERT model. We might have to restart the runtime after this due to updating matplotlib"
      ],
      "metadata": {
        "id": "ru5Au1OIBRnr"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "pip install -U transformers['ja'] datasets matplotlib"
      ],
      "metadata": {
        "id": "9Sk8NszZ6sUF"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Imports"
      ],
      "metadata": {
        "id": "XZ5MFOFUpg57"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "We use several libraries for this project, including PyTorch and HuggingFace for the model training, pandas for cleaning and organizing our data, and plotly for visualizing our results."
      ],
      "metadata": {
        "id": "qeY9ytq_Bukv"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from datasets import load_dataset\n",
        "import numpy as np\n",
        "from scipy.stats import pearsonr, ks_2samp\n",
        "from sklearn import metrics\n",
        "import torch\n",
        "from transformers import BertJapaneseTokenizer, BertForSequenceClassification, Trainer, TrainingArguments, DataCollatorWithPadding, pipeline\n",
        "from transformers.pipelines.base import KeyDataset\n",
        "from tqdm.notebook import tqdm\n",
        "import matplotlib.patches as mpatches\n",
        "import matplotlib.pyplot as plt\n",
        "import pandas as pd\n",
        "\n",
        "from itertools import chain, product\n",
        "from pathlib import Path"
      ],
      "metadata": {
        "id": "w4JmmLeXq1YW"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "We can load tensorboard to view model metrics after training."
      ],
      "metadata": {
        "id": "TZNiGRqoPS1Q"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%load_ext tensorboard"
      ],
      "metadata": {
        "id": "NvbHiTLQPSek"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Setup for saving"
      ],
      "metadata": {
        "id": "9_brio0zUZWM"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "If we would like to save the BERT checkpoints and other datasets, we can mount our Google Drive on this runtime."
      ],
      "metadata": {
        "id": "DI6kv0wH-3uh"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from google.colab import drive\n",
        "\n",
        "drive.mount('/content/drive')"
      ],
      "metadata": {
        "id": "d2XNShWBHvKn"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Fill out the form below to choose were to save our datasets and model checkpoints."
      ],
      "metadata": {
        "id": "VtJQx6D_CRn_"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "save_root = '/content/'#@param {'type':'string'}\n",
        "save_root = Path(save_root)"
      ],
      "metadata": {
        "cellView": "form",
        "id": "MWZa0NS4QwKZ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Fine-tuning BERT on emotion analysis"
      ],
      "metadata": {
        "id": "FK2_v7kTCWjN"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "As stated earlier, we will use the [WRIME dataset](https://aclanthology.org/2021.naacl-main.169.pdf) to fine-tune BERT on emotion analysis. WRIME (Dataset of **W**riters’ and **R**eaders’ **I**ntensities of E**m**otion\n",
        "for their **E**stimation) is a dataset of 40,000 SNS posts annotated with the \"subjective\" emotional intensities of the posts' writers and the \"objective\" emotional intensities of the posts' readers.\n",
        "\n",
        "Each post is annotated with following eight emotions (following Plutchik's framework) for both writers and readers:\n",
        "\n",
        "* joy\n",
        "* sadness\n",
        "* anticipation\n",
        "* surprise\n",
        "* anger\n",
        "* fear\n",
        "* disgust\n",
        "* trust\n",
        "\n",
        "Each post can exhibit one or several of the emotions and the intesities are integers from 0 to 3, inclusive (respectively indicating \"none\", \"weak\", \"medium\", and \"strong\"). An example showing the annotation of a writer and the annotations of three readers, taken from the paper's Github, is shown below.\n",
        "\n",
        "\"タイヤがパンクしてた。。いたずらの可能性が高いんだって。。\"\n",
        "\n",
        "(\"The tire of my car was flat. I heard that it might be mischief.\")\n",
        "\n",
        "||Joy\t|Sadness\t|Anticipation\t|Surprise\t|Anger\t|Fear\t|Disgust\t|Trust|\n",
        "|---|---|---|---|---|---|---|---|---|\n",
        "|Writer\t |0\t|3\t|0\t|1\t|3\t|0\t|0\t|0|\n",
        "|Reader 1|0\t|3\t|0\t|3\t|1\t|2\t|1\t|0|\n",
        "|Reader 2|0\t|2\t|0\t|2\t|0\t|0\t|0\t|0|\n",
        "|Reader 3|0\t|2\t|0\t|2\t|0\t|1\t|1\t|0|"
      ],
      "metadata": {
        "id": "_7pb_c_7CZCT"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "As there are three annotators for reader emotion, we average their scores to create only one set of reader emotions per data point. We also split the dataset into train, test, and validation sets using the splits provided by the dataset."
      ],
      "metadata": {
        "id": "tqlOUnO40FBl"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "emotion_names = ('joy', 'sadness', 'anticipation', 'surprise', 'anger', 'fear', 'disgust', 'trust')\n",
        "wrime_tsv_targets = [f'Writer_{emotion.title()}' for emotion in emotion_names] + \\\n",
        "                    [f'Reader{i}_{emotion.title()}' for i, emotion in product(range(1, 4), emotion_names)]\n",
        "\n",
        "wrime_df = pd.read_csv('wrime-ver1.tsv', sep='\\t')\n",
        "wrime_df = wrime_df.rename(\n",
        "    columns=dict([('Train/Dev/Test', 'split'), ('Sentence', 'text')]+\\\n",
        "    [(col, col.replace('Saddness', 'Sadness')) for col in wrime_df.columns if 'Saddness' in col])\n",
        ")\n",
        "wrime_df = wrime_df[['split', 'text', *wrime_tsv_targets]]\n",
        "\n",
        "for emotion in emotion_names:\n",
        "  wrime_df[f'Reader_{emotion.title()}'] = wrime_df[[f'Reader{i}_{emotion.title()}' for i in range(1, 4)]].mean(axis=1)\n",
        "annotators = ('writer', 'reader')\n",
        "wrime_df['label_ids'] = wrime_df[[f'{annotator.title()}_{emotion.title()}' for annotator, emotion in product(annotators, emotion_names)]].values.tolist()\n",
        "wrime_df = wrime_df[['split', 'text', 'label_ids']]\n",
        "\n",
        "for df_split, dataset_split in zip(['train', 'dev', 'test'], ['train', 'validation', 'test']):\n",
        "  wrime_df[wrime_df.split==df_split].drop(columns='split').to_csv(save_root/f'wrime_{dataset_split}.csv', index=False)"
      ],
      "metadata": {
        "id": "6ggCXpHm0FBm"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "We can now load the dataset from the splits."
      ],
      "metadata": {
        "id": "B53IhtZH0FBm"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "wrime = load_dataset('csv', data_files={split:str(save_root/f'wrime_{split}.csv') for split in ['train', 'validation', 'test']})"
      ],
      "metadata": {
        "id": "24miXVsm0FBn"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "To use BERT, the inputs must be tokenized. We use a [tokenizer pre-trained on Japanese text by the Tohoku NLP Lab](https://huggingface.co/cl-tohoku/bert-base-japanese-v2) to tokenize WRIME."
      ],
      "metadata": {
        "id": "Z2KPNhLL0FBo"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "tokenizer = BertJapaneseTokenizer.from_pretrained('cl-tohoku/bert-base-japanese-v2')\n",
        "\n",
        "def preprocess_function(examples):\n",
        "  examples['label_ids'] = [eval(label_ids) for label_ids in examples['label_ids']]\n",
        "  examples = tokenizer(examples['text'], truncation=True, max_length=512)\n",
        "  return examples\n",
        "\n",
        "tokenized_wrime = wrime.map(preprocess_function, batched=True)"
      ],
      "metadata": {
        "id": "EgEN0KEM0FBo"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "data_collator = DataCollatorWithPadding(tokenizer=tokenizer, max_length=512)"
      ],
      "metadata": {
        "id": "MchIPtju0FBq"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now we can actually load the BERT model. The BERT model we use is also from the [Tohoku NLP Lab](https://huggingface.co/cl-tohoku/bert-base-japanese-v2) and is pre-trained on Japanese Wikipedia."
      ],
      "metadata": {
        "id": "5gmIwN1v0FBq"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "model = BertForSequenceClassification.from_pretrained(\n",
        "    'cl-tohoku/bert-base-japanese-v2',\n",
        "    num_labels=16,\n",
        "    problem_type='regression'\n",
        ")"
      ],
      "metadata": {
        "id": "mJ3WxwbtDzBt"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "While we will fine-tune using mean squared error as our loss function, we can also report mean absolute error during training to measure performance."
      ],
      "metadata": {
        "id": "hBRp3xet0FBq"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def compute_metrics(eval_pred):\n",
        "  \"\"\"Computes metrics\"\"\"\n",
        "  logits, labels = eval_pred\n",
        "  return {\n",
        "      'mae' : metrics.mean_absolute_error(y_true=labels, y_pred=logits)\n",
        "  }"
      ],
      "metadata": {
        "id": "cWq9OYAr0FBq"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now we can begin training."
      ],
      "metadata": {
        "id": "SYhxdaeP0FBr"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "training_args = TrainingArguments(\n",
        "    output_dir=save_root / 'bert-base-japanese-v2-wrime-fine-tune',\n",
        "    evaluation_strategy='epoch',\n",
        "    per_device_train_batch_size=32,\n",
        "    per_device_eval_batch_size=32,\n",
        "    learning_rate=2e-5,\n",
        "    weight_decay=0.01,\n",
        "    adam_beta1=0.9,\n",
        "    adam_beta2=0.999,\n",
        "    num_train_epochs=3,\n",
        "    lr_scheduler_type='linear',\n",
        "    warmup_ratio=0.01,\n",
        "    save_strategy='epoch',\n",
        "    report_to='tensorboard'\n",
        ")\n",
        "\n",
        "trainer = Trainer(\n",
        "    model=model,\n",
        "    args=training_args,\n",
        "    data_collator=data_collator,\n",
        "    train_dataset=tokenized_wrime['train'].shuffle(seed=42),\n",
        "    eval_dataset=tokenized_wrime['test'].shuffle(seed=42),\n",
        "    tokenizer=tokenizer,\n",
        "    compute_metrics=compute_metrics\n",
        ")"
      ],
      "metadata": {
        "id": "8nzE-Z8LSKfd"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "trainer.train()"
      ],
      "metadata": {
        "id": "1ZI8j8R70FBr"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "We can also look at our metrics through Tensorboard."
      ],
      "metadata": {
        "id": "vbdmLOqA0FBr"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%tensorboard --logdir \"$save_root/bert-base-japanese-v2-wrime-fine-tune/\""
      ],
      "metadata": {
        "id": "-IIYSppe0FBs"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Inferring emotions from vaccine tweets"
      ],
      "metadata": {
        "id": "tiPe_roxf1bl"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Thanks to PhD Candidate Kiki Ferawati for providing the vaccine dataset. The dataset is composed of 20,254 tweets containing the keywords `'ワクチン'`, `'モデルナ'`, `'ファイザー'` and/or `オミクロン` scraped from December 1, 2021 to December 31, 2021. For each keyword, 15 random minutes were sampled from each day in December 2021, and all tweets containing the keyword in those minutes were scraped.  We preprocess this dataset by removing `'\\r'` characters, which can affect parsing into a dataset compatible with BERT.\n",
        "\n",
        "We provide the IDs of the Tweets. Scrape the Tweets using your favorite scraper and save them into a file called `vaccine_tweets.csv`. This csv should have the Tweet text in a column called `\"text\"`."
      ],
      "metadata": {
        "id": "DaHorKS2ko7N"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "We can load the preprocessed dataset as follows."
      ],
      "metadata": {
        "id": "Xn2WHpqjlkfT"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "vaccine_tweets = load_dataset('csv', data_files={'unsupervised':'vaccine_tweets.csv'})['unsupervised']"
      ],
      "metadata": {
        "id": "u5o-DvmvFxKw"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Again, the emotions we will be extracting from the vaccine-related tweets are:\n",
        "\n",
        "* joy\n",
        "* sadness\n",
        "* anticipation\n",
        "* surprise\n",
        "* anger\n",
        "* fear\n",
        "* disgust\n",
        "* trust"
      ],
      "metadata": {
        "id": "mmIbT5NYp5Pe"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "emotion_names = ('joy', 'sadness', 'anticipation', 'surprise', 'anger', 'fear', 'disgust', 'trust')\n",
        "annotators = ('writer', 'reader')\n",
        "labels = [f'{annotator}_{emotion}' for annotator, emotion in (product(annotators, emotion_names))]"
      ],
      "metadata": {
        "id": "46gIylp_zHwc"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "We load the same tokenizer but use the BERT model we trained earlier."
      ],
      "metadata": {
        "id": "_x0QSznbqLwl"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "model = BertForSequenceClassification.from_pretrained(\n",
        "    save_root/'bert-base-japanese-v2-wrime-fine-tune/checkpoint-3750',\n",
        "    num_labels=16,\n",
        "    id2label={i:label for i, label in enumerate(labels)},\n",
        "    max_length=512\n",
        ")\n",
        "tokenizer = BertJapaneseTokenizer.from_pretrained('cl-tohoku/bert-base-japanese-v2')"
      ],
      "metadata": {
        "id": "jISnr1UxrzLg"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "The pipeline shown below will allow the easy inference of emotion scores from input text."
      ],
      "metadata": {
        "id": "tDRedjgpqnJG"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "pipe = pipeline(\n",
        "    'text-classification',\n",
        "    model=model,\n",
        "    tokenizer=tokenizer,\n",
        "    return_all_scores=True,\n",
        "    function_to_apply='none',\n",
        "    device=0 if torch.cuda.is_available() else -1\n",
        ")"
      ],
      "metadata": {
        "id": "B6jq0Dzozmdc"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "We iterate over the vaccine tweets and infer the emotion scores from each tweet."
      ],
      "metadata": {
        "id": "c5mLs_RcqSj2"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "emotion_scores = pd.DataFrame()\n",
        "for out in tqdm(pipe(KeyDataset(vaccine_tweets, 'text'), batch_size=32)):\n",
        "  emotion_scores = emotion_scores.append(\n",
        "      pd.DataFrame([dict([tuple(label_score.values()) for label_score in out])]),\n",
        "      ignore_index=True\n",
        "  )"
      ],
      "metadata": {
        "id": "QdMv5L0t9Rnl"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "We attach the emotion scores to each the vaccine tweet dataset to more easily view the emotion scores per tweet."
      ],
      "metadata": {
        "id": "Cy3lyv7fqhoH"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "vaccine_tweets_df = pd.read_csv('vaccine_tweets.csv')\n",
        "vaccine_tweets_emotions = pd.concat((vaccine_tweets_df, emotion_scores), axis='columns')\n",
        "vaccine_tweets_emotions.to_csv(save_root/'vaccine_tweets_emotions.csv', index=False)"
      ],
      "metadata": {
        "id": "wYghA6DpDs8D"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "vaccine_tweets_emotions = pd.read_csv(save_root/'vaccine_tweets_emotions.csv')\n",
        "vaccine_tweets_emotions.created_at = pd.to_datetime(vaccine_tweets_emotions.created_at)"
      ],
      "metadata": {
        "id": "Z4fER1JRzhvN"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "We can make a simple summary of the mined emotions by showing the average, standard deviation, and sum of each emotion column."
      ],
      "metadata": {
        "id": "pRoz-tmurCBX"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "vaccine_tweets_emotions = pd.read_csv(save_root/'vaccine_tweets_emotions.csv')\n",
        "vaccine_tweets_emotions.created_at = pd.to_datetime(vaccine_tweets_emotions.created_at)\n",
        "vaccine_tweets_emotions.sort_values('created_at')\n",
        "emotion_summary = {'avg':[], 'st_dev':[], 'total':[]}\n",
        "for emotion in labels:\n",
        "  emotion_summary['avg'].append(vaccine_tweets_emotions[emotion].mean())\n",
        "  emotion_summary['st_dev'].append(vaccine_tweets_emotions[emotion].std())\n",
        "  emotion_summary['total'].append(vaccine_tweets_emotions[emotion].sum())\n",
        "emotion_summary = pd.DataFrame(emotion_summary, index=labels)\n",
        "emotion_summary"
      ],
      "metadata": {
        "id": "TO9mV2weLUlq"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Preparing vaccination data"
      ],
      "metadata": {
        "id": "rod744UAlSRD"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "The vaccination data from [Our World in Data](https://ourworldindata.org/) contains daily information regarding vaccinations, people vaccinated, and people fully vaccinated. A person is considered vaccinated in the dataset if they have received at least one dose and fully vaccinated if they have received at least two.\n",
        "\n",
        "To preprocess the vaccination data, we filter the data to only include entries in Japan from December 1, 2021 to December 31, 2021, the same timeframe as the collected tweets. Note that some rows have missing information, which we will have to ingore during our visualizations."
      ],
      "metadata": {
        "id": "J52jxr8wlWLm"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "vaccinations = pd.read_csv('vaccinations.csv')\n",
        "vaccinations = vaccinations[(vaccinations.iso_code=='JPN') & (vaccinations.date > '2021-12-01') & (vaccinations.date <= '2021-12-31')]\n",
        "vaccinations.to_csv(save_root/'vaccinations.csv')"
      ],
      "metadata": {
        "id": "-vXAPohZlRpE"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "vaccinations = pd.read_csv(save_root/'vaccinations.csv')\n",
        "vaccinations.date = pd.to_datetime(vaccinations.date)"
      ],
      "metadata": {
        "id": "x2-E6FSdlVp9"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "We will look into three vaccination measures:\n",
        "\n",
        "* daily vaccinations - the number of doses administered on particular day\n",
        "* people vaccinated - the number of people who received at least one dose\n",
        "* people fully vaccinated - the number of people who received at least two doses"
      ],
      "metadata": {
        "id": "1nCmfRFKIfjp"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "vaccine_stats = ['daily_vaccinations', 'people_vaccinated', 'people_fully_vaccinated']"
      ],
      "metadata": {
        "id": "DOHuDAKrhnW5"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Visualizing results"
      ],
      "metadata": {
        "id": "Uw2dhVmIlqbS"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "emotions = ('joy', 'sadness', 'anticipation', 'surprise', 'anger', 'fear', 'disgust', 'trust')\n",
        "annotators = ('writer', 'reader')\n",
        "labels = [f'{ann}_{emo}' for ann, emo in product(annotators, emotions)]\n",
        "writer_labels = [f'writer_{emo}' for emo in emotions]\n",
        "reader_labels = [f'reader_{emo}' for emo in emotions]"
      ],
      "metadata": {
        "id": "5oYlCrJ4to8H"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Distributions of predicted emotion scores for writers and readers"
      ],
      "metadata": {
        "id": "4d0CY1YCmAFg"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "labels_no_collate = list(chain.from_iterable(zip(writer_labels, reader_labels)))\n",
        "positions = np.arange(len(emotions))\n",
        "width = 0.2\n",
        "\n",
        "fig, ax = plt.subplots()\n",
        "\n",
        "for sublabels in (writer_labels, reader_labels):\n",
        "  boxplot = ax.boxplot(\n",
        "      vaccine_tweets_emotions[sublabels[::-1]],\n",
        "      showmeans=True,\n",
        "      sym='',\n",
        "      vert=False,\n",
        "      widths=0.4,\n",
        "      positions=(positions+width) if sublabels==writer_labels else (positions-width),\n",
        "      patch_artist=True,\n",
        "      boxprops={'color':'black', 'facecolor':'tab:blue' if sublabels==writer_labels else 'tab:orange'},\n",
        "      whiskerprops={'color':'black'},\n",
        "      capprops={'color':'black'},\n",
        "      medianprops={'color':'black'},\n",
        "      meanprops={'marker':'.', 'markerfacecolor':'black', 'markeredgecolor':'black'}\n",
        "  )\n",
        "\n",
        "writer_patch = mpatches.Patch(color='tab:blue', label='Writer')\n",
        "reader_patch = mpatches.Patch(color='tab:orange', label='Reader')\n",
        "ax.legend(handles=[writer_patch, reader_patch])\n",
        "\n",
        "ax.set_ylabel('Emotion', fontsize=12)\n",
        "ax.set_xlabel('Score', fontsize=12)\n",
        "ax.set_yticks(positions, [emo.title() for emo in emotions[::-1]], fontsize=12)\n",
        "ax.grid(axis='x')\n",
        "\n",
        "plt.show()"
      ],
      "metadata": {
        "id": "JAS4t_YqyRmz"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Distributions of predicted emotions for writers and readers for all keywords and for each keyword"
      ],
      "metadata": {
        "id": "9FNEnxhXY4AV"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "x = np.arange(len(emotions))\n",
        "width = 0.4\n",
        "\n",
        "fig, axs = plt.subplots(3, 2, figsize=(8, 8))\n",
        "gs = axs[0, 0].get_gridspec()\n",
        "for ax in axs[0, :]:\n",
        "  ax.remove()\n",
        "ax_summary = fig.add_subplot(gs[0, :])\n",
        "\n",
        "writer_proportions = vaccine_tweets_emotions[writer_labels].sum() / vaccine_tweets_emotions[writer_labels].sum().sum() * 100\n",
        "reader_proportions = vaccine_tweets_emotions[reader_labels].sum() / vaccine_tweets_emotions[reader_labels].sum().sum() * 100\n",
        "\n",
        "writer_bars = ax_summary.bar(x - width/2, height=writer_proportions, width=width, label='Writer')\n",
        "reader_bars = ax_summary.bar(x + width/2, height=reader_proportions, width=width, label='Reader')\n",
        "\n",
        "ax_summary.set_title('All keywords', fontsize=14)\n",
        "ax_summary.legend()\n",
        "\n",
        "keywords_jp = ['ワクチン', 'モデルナ', 'ファイザー', 'オミクロン']\n",
        "keywords_en = ['Vaccine', 'Moderna', 'Pfizer', 'Omicron']\n",
        "\n",
        "for keyword_jp, keyword_en, ax in zip(keywords_jp, keywords_en, axs[1:].flat):\n",
        "  filtered_df = vaccine_tweets_emotions[vaccine_tweets_emotions.text.str.contains(keyword_jp)]\n",
        "  writer_proportions = filtered_df[writer_labels].sum() / filtered_df[writer_labels].sum().sum() * 100\n",
        "  reader_proportions = filtered_df[reader_labels].sum() / filtered_df[reader_labels].sum().sum() * 100\n",
        "\n",
        "  writer_bars = ax.bar(x - width/2, height=writer_proportions, width=width, label='Writer')\n",
        "  reader_bars = ax.bar(x + width/2, height=reader_proportions, width=width, label='Reader')\n",
        "\n",
        "  ax.set_title(keyword_en, fontsize=14)\n",
        "\n",
        "for ax in list(axs[1:].flat)+[ax_summary]:\n",
        "  ax.set_xticks(x, [emo.title() for emo in emotions], rotation=45, fontsize=14)\n",
        "  ax.set_yticks(np.arange(0, 35, 5))\n",
        "  ax.set_axisbelow(True)\n",
        "  ax.grid(axis='y')\n",
        "\n",
        "fig.supxlabel('Emotion', fontsize=14)\n",
        "fig.supylabel('Proportion', fontsize=14)\n",
        "plt.tight_layout()\n",
        "\n",
        "plt.show()"
      ],
      "metadata": {
        "id": "R__w1X2y99Uj"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Average score for each emotion and the number of new vaccinations, people vaccinated, and people fully vaccinated for several days in December 2021"
      ],
      "metadata": {
        "id": "EOhbqQjMvaKC"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "daily_emotion_stats = vaccine_tweets_emotions.groupby(pd.Grouper(key='created_at', freq='D')).mean()\n",
        "colors = ['orange', 'cornflowerblue', 'pink', 'plum', 'crimson', 'gold', 'seagreen', 'silver']\n",
        "\n",
        "fig, axs = plt.subplots(3, 2, sharex=True, figsize=(12, 6))\n",
        "gs = axs[0, 1].get_gridspec()\n",
        "for ax in axs[:, 0]:\n",
        "    ax.remove()\n",
        "ax_emo = fig.add_subplot(gs[:, 0])\n",
        "\n",
        "for stat, ax in zip(vaccine_stats, axs[:, 1].flat):\n",
        "  filtered_df = vaccinations[['date', stat]]\n",
        "  if stat.startswith('people_'):\n",
        "    filtered_df = filtered_df.diff()\n",
        "  ids = []\n",
        "  vals = []\n",
        "  latest_ids = []\n",
        "  latest_vals = []\n",
        "  for i, row in filtered_df.iterrows():\n",
        "    if not pd.isnull(row[stat]) and not vaccinations.date.loc[i].isoformat().startswith('2021-11-30'):\n",
        "      latest_vals.append(row[stat])\n",
        "      latest_ids.append(vaccinations.date.loc[i])\n",
        "    else:\n",
        "      if latest_vals:\n",
        "        vals.append(latest_vals)\n",
        "        ids.append(latest_ids)\n",
        "        latest_vals = []\n",
        "        latest_ids = []\n",
        "  if latest_vals:\n",
        "    vals.append(latest_vals)\n",
        "    ids.append(latest_ids)\n",
        "\n",
        "  nan_ids = [[ids_subset0[-1], ids_subset1[0]] for ids_subset0, ids_subset1 in zip(ids[:-1], ids[1:])]\n",
        "  nan_vals = [[vals_subset0[-1], vals_subset1[0]] for vals_subset0, vals_subset1 in zip(vals[:-1], vals[1:])]\n",
        "\n",
        "  for ids_subset, vals_subset in zip(ids, vals):\n",
        "    ax.plot(ids_subset, vals_subset, '-|', c='black')\n",
        "  for ids_subset, vals_subset in zip(nan_ids, nan_vals):\n",
        "    ax.plot(ids_subset, vals_subset, ':|', c='black')\n",
        "\n",
        "  ax.set_ylabel(stat.replace('daily_', '').replace('_v', '\\nv').replace('_', ' ').title(), fontsize=12)\n",
        "  ax.tick_params(axis='x', rotation=45, labelsize=12)\n",
        "  ax.grid()\n",
        "\n",
        "for i, label in enumerate(labels):\n",
        "  ax_emo.plot(daily_emotion_stats.index.date, daily_emotion_stats[label], '-' if label.startswith('writer') else ':', color=colors[i%8], label=label.replace('_', ' ').title())\n",
        "\n",
        "ax_emo.set_ylabel('Average Emotion Score', fontsize=12)\n",
        "ax_emo.tick_params(axis='x', rotation=45, labelsize=12)\n",
        "ax_emo.grid()\n",
        "\n",
        "fig.supxlabel('Date', fontsize=12)\n",
        "fig.legend(loc='upper left', bbox_to_anchor=(1, 1))\n",
        "fig.tight_layout()\n",
        "\n",
        "plt.show()"
      ],
      "metadata": {
        "id": "_axU31GkstM6"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Correlation analysis of emotion scores and vaccination statistics\n",
        "\n",
        "Choose the vaccination measure in the dropdown box below"
      ],
      "metadata": {
        "id": "h4SuJKmeOWxm"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "daily_emotion_stats = vaccine_tweets_emotions.groupby(pd.Grouper(key='created_at', freq='D')).sum()\n",
        "daily_emotion_stats.index = daily_emotion_stats.index.tz_localize(None)\n",
        "daily_vaccine_stats = vaccinations.set_index('date')[vaccine_stats]\n",
        "data = pd.concat((daily_emotion_stats, daily_vaccine_stats), axis=1)\n",
        "\n",
        "stat = 'daily_vaccinations' #@param ['daily_vaccinations', 'people_vaccinated', 'people_fully_vaccinated']\n",
        "\n",
        "fig, axs = plt.subplots(4, 4,  figsize=(10, 10))\n",
        "\n",
        "for label, ax in zip(labels, axs.flat):\n",
        "  reformatted_label = label.replace('_', ' ').title()\n",
        "  reformatted_stat = stat.replace('daily_', '').replace('_', ' ').title()\n",
        "\n",
        "  filtered_data = data.loc[:, labels+[stat]]\n",
        "  if stat.startswith('people_'):\n",
        "    filtered_data[stat] = filtered_data[stat].diff()\n",
        "  filtered_data = filtered_data.dropna()\n",
        "  \n",
        "  slope, intercept = np.polyfit(filtered_data[stat], filtered_data[label], 1)\n",
        "  endpoint_xs = np.array([filtered_data[stat].min(), filtered_data[stat].max()])\n",
        "\n",
        "  r, p_value = pearsonr(filtered_data[stat], filtered_data[label])\n",
        "\n",
        "  ax.scatter(x=filtered_data[stat], y=filtered_data[label], c='tab:green', alpha=0.7)\n",
        "  ax.plot(endpoint_xs, slope*endpoint_xs+intercept, color='black')\n",
        "\n",
        "  ax.set_title(f'{reformatted_label}\\nr={r:.2f}\\np-value={p_value:.3f}', fontsize=18)\n",
        "\n",
        "  ax.grid()\n",
        "\n",
        "fig.supxlabel(reformatted_stat, fontsize=24)\n",
        "fig.supylabel('Total Emotion Score', fontsize=24)\n",
        "fig.tight_layout()\n",
        "\n",
        "plt.show()"
      ],
      "metadata": {
        "id": "0DgVbK4zOeGe"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Goodness-of-fit tests"
      ],
      "metadata": {
        "id": "xgtNYQ0vU88Q"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Distributions of emotions between vaccine brands\n",
        "\n",
        "We conduct a Kolmogorov-Smirnov test to compare the distributions of each writer and reader emotion across two vaccine brands, Moderna and Pfizer."
      ],
      "metadata": {
        "id": "iL9NgJvWiEmp"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "moderna_tweets = vaccine_tweets_emotions[vaccine_tweets_emotions.text.str.contains('モデルナ')]\n",
        "pfizer_tweets = vaccine_tweets_emotions[vaccine_tweets_emotions.text.str.contains('ファイザー')]\n",
        "ks_results = {'stat':[], 'p':[]}\n",
        "for label in labels:\n",
        "  ks_result = ks_2samp(moderna_tweets[label].values, pfizer_tweets[label].values)\n",
        "  ks_results['stat'].append(ks_result.statistic)\n",
        "  ks_results['p'].append(ks_result.pvalue)\n",
        "ks_results = pd.DataFrame(ks_results, index=labels)\n",
        "ks_results"
      ],
      "metadata": {
        "id": "iKmOqfRxjR3Z"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Q-Q plots of predicted writer and reader emotions"
      ],
      "metadata": {
        "id": "6SX8gxAHucFg"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "fig, axs = plt.subplots(2, 4, figsize=(8, 4))\n",
        "\n",
        "for emotion, ax in zip(emotions, axs.flat):\n",
        "  writer_subset = vaccine_tweets_emotions[f'writer_{emotion}']\n",
        "  writer_quantiles = np.array([writer_subset.quantile(i/100) for i in range(100)])\n",
        "  reader_subset = vaccine_tweets_emotions[f'reader_{emotion}']\n",
        "  reader_quantiles = np.array([reader_subset.quantile(i/100) for i in range(100)])\n",
        "  \n",
        "  ax.plot(writer_quantiles, reader_quantiles, c='black')\n",
        "  ax.set_xticks([0, 1, 2, 2.5], ['0', '1', '2', ''])\n",
        "  ax.set_yticks([0, 1, 2, 2.5], ['0', '1', '2', ''])\n",
        "  ax.grid()\n",
        "  ax.set_title(emotion.title(), fontsize=14)\n",
        "\n",
        "fig.supxlabel('Writer Quantiles', fontsize=14)\n",
        "fig.supylabel('Reader Quantiles', fontsize=14)\n",
        "fig.tight_layout()\n",
        "\n",
        "plt.show()"
      ],
      "metadata": {
        "id": "1LTD11Btudly"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}