Multiclass_Logistic_Regression_Gender/model_training_testing.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c0ccaff5",
   "metadata": {},
   "source": [
    "## Training a multi-class logistic regression model to predict gender of Chinese name\n",
    "\n",
    "To train a logistic regression, the names and genders in the train set are converted into word vectors and numerical values respectively. \n",
    "\n",
    "As there are three gender classes available in the dataset, namely M (male), F (female), and U (undefined), the logistic regression model is a multi-class one. In other words, to enable the model to distinguish among these three gender classes, we need to train three separate logistic regression models that can evaluate the probablity of one gender class at a time. During the training, M, F and U are represented by 0, 1 and 2 respectively. \n",
    "\n",
    "As for the Chinese names, they are converted into word vectors based on the one-hot encoding approach. That is, the total number of unique characters seen in the training set is first calculated so that the dimension of the word vector can be determined. For example, suppose we have 3 unique characters in total, say `'春'`, `'杰'`, `'乐'`, we can use `(1, 0, 0)`, `(0, 1, 0)` and `(0, 0, 1)` respectively to represent these three characters. To represent `'春杰'`, we can combine two vector together (vector addition) and use `(1, 1, 0)`. More concretely, to be able to represent any character, seen or unseen, we need to add an extra dimension so that our model can deal with names with unseen characters. For example, the example just above, we can then use `(1, 0, 0, 0)` to represent `'春'` such that when `'春'` comes with a unseen character, say `A`, we can use `(1, 0, 0, 1)` to represent `'春A'`. However, to make the model simpler, no matter how many unseen characters there are, they will only be represented by the extra one in the additional dimension for the one-hot word vector. Therefore, `'春ABCDEFGAAAAAA'` will still be `(1, 0, 0, 1)`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be275460",
   "metadata": {},
   "source": [
    "## Overview\n",
    "\n",
    "- [Data Loading and Preprocessing](#data)\n",
    "    - [Data Loading](#loading)\n",
    "    - [Preprocessing](#prep)\n",
    "- [Model Training](#)\n",
    "- [Testing](#test)\n",
    "    - [Testing against the train/dev sets](#train_dev)\n",
    "    - [Testing against the test set](#test_set)\n",
    "- [A problem](#problem)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1acb694c",
   "metadata": {},
   "source": [
    "<a name='data'></a>\n",
    "## Data Loading and Preprocessing\n",
    "\n",
    "It turns out there are over 6500 unique characters for the 633857 in original train set spilted previously (see <ins>train_dev_test_split.ipynb</ins> within this directory), which means for every training example in the train set, the word vector for a any given name has over 6500 dimensions. That means, for the entire train set, the names as a matrix will have at least 6500 by 633857 dimensions, which turn out to be overwhelmingly large to train because of the enormous calculations involved!\n",
    "\n",
    "However, it also turns out that the performance of the logistic regression in predicting gender of Chinese names soon plateau after fed with some ten thousand examples. Therefore without changing the algorithms, for example by adding some extra hidden layers (i.e., nerual networks), more examples do not make any worthy differences. As a tradeoff, here I only selected the first 100,000 examples from the original train set. The rest of the original train set will be treated as an additional dev set to test the performance of the model on unseen names. \n",
    "\n",
    "**Note**: With more hidden layers betwen the input names and the output genders, one-hot word vectors can be even more expensive to train. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c064e466",
   "metadata": {},
   "source": [
    "<a name='loading'></a>\n",
    "### Data Loading "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "72d7b90a",
   "metadata": {},
   "outputs": [],
   "source": [
    "from utils import *\n",
    "\n",
    "# data_loader (from utils) will return [[name, gender], [name, gender]...]\n",
    "train_ds_all = data_loader('data/train_ds.txt')\n",
    "# the model will only be trained on train_ds (the first 100000 examples of the entire train set)\n",
    "train_ds, train_ds_dev = train_ds_all[:100000], train_ds_all[100000:]\n",
    "assert(len(train_ds_all) == len(train_ds) + len(train_ds_dev))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "1ae8f7f9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['莹暂', 'F']\n",
      "['荣辉', 'M']\n",
      "['泽彬', 'M']\n",
      "['二庄', 'M']\n",
      "['治权', 'M']\n"
     ]
    }
   ],
   "source": [
    "# check the first 5 examples\n",
    "for i in range(5):\n",
    "    print(train_ds[i])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ffbc5a7",
   "metadata": {},
   "source": [
    "<a name='prep'></a>\n",
    "### Preprocessing "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "090a85a3",
   "metadata": {},
   "outputs": [],
   "source": [
    "from collections import defaultdict\n",
    "\n",
    "\n",
    "# make the char_dict that will be useful in converting name vectors \n",
    "def char_dict(ds):\n",
    "    '''Returns a char dict that stores the indices for the seen\n",
    "    characters in the selected train set and returns a constant index\n",
    "    for all unseen characters.\n",
    "    \n",
    "    Param:\n",
    "        ds: dataset --> [list, list,...] where list=[name, gender]\n",
    "    '''\n",
    "    dic = {}\n",
    "    idx = 1\n",
    "    for item in ds:\n",
    "        for char in item[0]:\n",
    "            if char not in dic:\n",
    "                dic[char] = idx\n",
    "                idx += 1\n",
    "                \n",
    "    # the size of the dic = idx, which all includes the one index \n",
    "    # reserved for all unseen characters\n",
    "    dic['size'] = idx\n",
    "    dic = defaultdict(lambda :idx, dic)\n",
    "    return dic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "8af49c92",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(4376,\n",
       " defaultdict(<function __main__.char_dict.<locals>.<lambda>()>,\n",
       "             {'莹': 1,\n",
       "              '暂': 2,\n",
       "              '荣': 3,\n",
       "              '辉': 4,\n",
       "              '泽': 5,\n",
       "              '彬': 6,\n",
       "              '二': 7,\n",
       "              '庄': 8,\n",
       "              '治': 9,\n",
       "              '权': 10,\n",
       "              '哧': 11,\n",
       "              '天': 12,\n",
       "              '彦': 13,\n",
       "              '成': 14,\n",
       "              '其': 15,\n",
       "              '荷': 16,\n",
       "              '志': 17,\n",
       "              '峰': 18,\n",
       "              '诚': 19,\n",
       "              '晔': 20,\n",
       "              '紫': 21,\n",
       "              '俊': 22,\n",
       "              '絮': 23,\n",
       "              '雨': 24,\n",
       "              '洪': 25,\n",
       "              '君': 26,\n",
       "              '世': 27,\n",
       "              '佳': 28,\n",
       "              '讯': 29,\n",
       "              '谨': 30,\n",
       "              '德': 31,\n",
       "              '会': 32,\n",
       "              '川': 33,\n",
       "              '嘉': 34,\n",
       "              '艺': 35,\n",
       "              '贤': 36,\n",
       "              '晓': 37,\n",
       "              '晴': 38,\n",
       "              '银': 39,\n",
       "              '芳': 40,\n",
       "              '琼': 41,\n",
       "              '荫': 42,\n",
       "              '绣': 43,\n",
       "              '莉': 44,\n",
       "              '红': 45,\n",
       "              '丹': 46,\n",
       "              '泰': 47,\n",
       "              '山': 48,\n",
       "              '昕': 49,\n",
       "              '瑶': 50,\n",
       "              '仪': 51,\n",
       "              '泠': 52,\n",
       "              '迎': 53,\n",
       "              '姣': 54,\n",
       "              '引': 55,\n",
       "              '沂': 56,\n",
       "              '艳': 57,\n",
       "              '琳': 58,\n",
       "              '潞': 59,\n",
       "              '诩': 60,\n",
       "              '莎': 61,\n",
       "              '灏': 62,\n",
       "              '瀚': 63,\n",
       "              '岚': 64,\n",
       "              '心': 65,\n",
       "              '语': 66,\n",
       "              '彤': 67,\n",
       "              '歆': 68,\n",
       "              '媛': 69,\n",
       "              '帅': 70,\n",
       "              '斌': 71,\n",
       "              '故': 72,\n",
       "              '幸': 73,\n",
       "              '敏': 74,\n",
       "              '闻': 75,\n",
       "              '珊': 76,\n",
       "              '远': 77,\n",
       "              '滔': 78,\n",
       "              '庆': 79,\n",
       "              '煜': 80,\n",
       "              '苗': 81,\n",
       "              '利': 82,\n",
       "              '雪': 83,\n",
       "              '智': 84,\n",
       "              '鹏': 85,\n",
       "              '达': 86,\n",
       "              '玉': 87,\n",
       "              '新': 88,\n",
       "              '亚': 89,\n",
       "              '铸': 90,\n",
       "              '玫': 91,\n",
       "              '瑛': 92,\n",
       "              '伊': 93,\n",
       "              '菀': 94,\n",
       "              '梓': 95,\n",
       "              '韦': 96,\n",
       "              '忠': 97,\n",
       "              '华': 98,\n",
       "              '美': 99,\n",
       "              '啸': 100,\n",
       "              '文': 101,\n",
       "              '冰': 102,\n",
       "              '彳': 103,\n",
       "              '人': 104,\n",
       "              '石': 105,\n",
       "              '坚': 106,\n",
       "              '述': 107,\n",
       "              '萍': 108,\n",
       "              '靖': 109,\n",
       "              '裕': 110,\n",
       "              '子': 111,\n",
       "              '翔': 112,\n",
       "              '广': 113,\n",
       "              '磊': 114,\n",
       "              '鹦': 115,\n",
       "              '旭': 116,\n",
       "              '墨': 117,\n",
       "              '慈': 118,\n",
       "              '含': 119,\n",
       "              '斐': 120,\n",
       "              '然': 121,\n",
       "              '锦': 122,\n",
       "              '勇': 123,\n",
       "              '佑': 124,\n",
       "              '鸽': 125,\n",
       "              '凤': 126,\n",
       "              '舞': 127,\n",
       "              '楚': 128,\n",
       "              '倩': 129,\n",
       "              '薇': 130,\n",
       "              '晨': 131,\n",
       "              '水': 132,\n",
       "              '境': 133,\n",
       "              '宣': 134,\n",
       "              '玄': 135,\n",
       "              '康': 136,\n",
       "              '春': 137,\n",
       "              '道': 138,\n",
       "              '森': 139,\n",
       "              '乐': 140,\n",
       "              '懿': 141,\n",
       "              '滟': 142,\n",
       "              '凯': 143,\n",
       "              '朕': 144,\n",
       "              '云': 145,\n",
       "              '博': 146,\n",
       "              '彩': 147,\n",
       "              '龙': 148,\n",
       "              '瑞': 149,\n",
       "              '洁': 150,\n",
       "              '来': 151,\n",
       "              '姿': 152,\n",
       "              '煊': 153,\n",
       "              '妍': 154,\n",
       "              '先': 155,\n",
       "              '运': 156,\n",
       "              '吉': 157,\n",
       "              '香': 158,\n",
       "              '菱': 159,\n",
       "              '绮': 160,\n",
       "              '柏': 161,\n",
       "              '青': 162,\n",
       "              '财': 163,\n",
       "              '祥': 164,\n",
       "              '飘': 165,\n",
       "              '宇': 166,\n",
       "              '韬': 167,\n",
       "              '朦': 168,\n",
       "              '慧': 169,\n",
       "              '菡': 170,\n",
       "              '玲': 171,\n",
       "              '李': 172,\n",
       "              '淑': 173,\n",
       "              '娥': 174,\n",
       "              '惠': 175,\n",
       "              '富': 176,\n",
       "              '彪': 177,\n",
       "              '正': 178,\n",
       "              '漪': 179,\n",
       "              '建': 180,\n",
       "              '余': 181,\n",
       "              '晗': 182,\n",
       "              '玮': 183,\n",
       "              '凡': 184,\n",
       "              '甫': 185,\n",
       "              '缨': 186,\n",
       "              '润': 187,\n",
       "              '静': 188,\n",
       "              '金': 189,\n",
       "              '泯': 190,\n",
       "              '兆': 191,\n",
       "              '杨': 192,\n",
       "              '冉': 193,\n",
       "              '炜': 194,\n",
       "              '凌': 195,\n",
       "              '瞳': 196,\n",
       "              '逸': 197,\n",
       "              '映': 198,\n",
       "              '羽': 199,\n",
       "              '贝': 200,\n",
       "              '翎': 201,\n",
       "              '阳': 202,\n",
       "              '可': 203,\n",
       "              '维': 204,\n",
       "              '烨': 205,\n",
       "              '兹': 206,\n",
       "              '明': 207,\n",
       "              '景': 208,\n",
       "              '鸿': 209,\n",
       "              '恒': 210,\n",
       "              '一': 211,\n",
       "              '平': 212,\n",
       "              '才': 213,\n",
       "              '屿': 214,\n",
       "              '娴': 215,\n",
       "              '大': 216,\n",
       "              '为': 217,\n",
       "              '鑫': 218,\n",
       "              '钇': 219,\n",
       "              '岩': 220,\n",
       "              '娇': 221,\n",
       "              '娃': 222,\n",
       "              '战': 223,\n",
       "              '杰': 224,\n",
       "              '钊': 225,\n",
       "              '仁': 226,\n",
       "              '永': 227,\n",
       "              '焱': 228,\n",
       "              '雁': 229,\n",
       "              '灵': 230,\n",
       "              '矜': 231,\n",
       "              '丽': 232,\n",
       "              '占': 233,\n",
       "              '自': 234,\n",
       "              '佩': 235,\n",
       "              '莲': 236,\n",
       "              '兴': 237,\n",
       "              '冠': 238,\n",
       "              '伟': 239,\n",
       "              '字': 240,\n",
       "              '煦': 241,\n",
       "              '巳': 242,\n",
       "              '娣': 243,\n",
       "              '琴': 244,\n",
       "              '清': 245,\n",
       "              '树': 246,\n",
       "              '纱': 247,\n",
       "              '洋': 248,\n",
       "              '宝': 249,\n",
       "              '霖': 250,\n",
       "              '珣': 251,\n",
       "              '连': 252,\n",
       "              '璧': 253,\n",
       "              '海': 254,\n",
       "              '双': 255,\n",
       "              '牛': 256,\n",
       "              '言': 257,\n",
       "              '卓': 258,\n",
       "              '靓': 259,\n",
       "              '婷': 260,\n",
       "              '阿': 261,\n",
       "              '屯': 262,\n",
       "              '伦': 263,\n",
       "              '璟': 264,\n",
       "              '琪': 265,\n",
       "              '崇': 266,\n",
       "              '杜': 267,\n",
       "              '炯': 268,\n",
       "              '民': 269,\n",
       "              '昭': 270,\n",
       "              '雅': 271,\n",
       "              '延': 272,\n",
       "              '英': 273,\n",
       "              '迪': 274,\n",
       "              '睿': 275,\n",
       "              '舒': 276,\n",
       "              '启': 277,\n",
       "              '八': 278,\n",
       "              '国': 279,\n",
       "              '光': 280,\n",
       "              '木': 281,\n",
       "              '素': 282,\n",
       "              '卿': 283,\n",
       "              '园': 284,\n",
       "              '芬': 285,\n",
       "              '聪': 286,\n",
       "              '增': 287,\n",
       "              '奕': 288,\n",
       "              '璐': 289,\n",
       "              '琦': 290,\n",
       "              '彝': 291,\n",
       "              '安': 292,\n",
       "              '嫦': 293,\n",
       "              '浩': 294,\n",
       "              '遂': 295,\n",
       "              '义': 296,\n",
       "              '星': 297,\n",
       "              '婧': 298,\n",
       "              '蕊': 299,\n",
       "              '中': 300,\n",
       "              '茹': 301,\n",
       "              '槐': 302,\n",
       "              '炼': 303,\n",
       "              '友': 304,\n",
       "              '蓓': 305,\n",
       "              '儿': 306,\n",
       "              '张': 307,\n",
       "              '哲': 308,\n",
       "              '茗': 309,\n",
       "              '万': 310,\n",
       "              '家': 311,\n",
       "              '喜': 312,\n",
       "              '传': 313,\n",
       "              '林': 314,\n",
       "              '瑜': 315,\n",
       "              '承': 316,\n",
       "              '健': 317,\n",
       "              '细': 318,\n",
       "              '诺': 319,\n",
       "              '立': 320,\n",
       "              '根': 321,\n",
       "              '涵': 322,\n",
       "              '宁': 323,\n",
       "              '思': 324,\n",
       "              '勉': 325,\n",
       "              '宗': 326,\n",
       "              '虎': 327,\n",
       "              '冬': 328,\n",
       "              '鲁': 329,\n",
       "              '欣': 330,\n",
       "              '烜': 331,\n",
       "              '楣': 332,\n",
       "              '苹': 333,\n",
       "              '果': 334,\n",
       "              '沛': 335,\n",
       "              '铃': 336,\n",
       "              '熙': 337,\n",
       "              '亿': 338,\n",
       "              '瑾': 339,\n",
       "              '昱': 340,\n",
       "              '柯': 341,\n",
       "              '梅': 342,\n",
       "              '洛': 343,\n",
       "              '北': 344,\n",
       "              '雷': 345,\n",
       "              '联': 346,\n",
       "              '坤': 347,\n",
       "              '烽': 348,\n",
       "              '朋': 349,\n",
       "              '燃': 350,\n",
       "              '耀': 351,\n",
       "              '振': 352,\n",
       "              '宏': 353,\n",
       "              '渝': 354,\n",
       "              '妤': 355,\n",
       "              '涛': 356,\n",
       "              '郡': 357,\n",
       "              '祎': 358,\n",
       "              '秀': 359,\n",
       "              '至': 360,\n",
       "              '开': 361,\n",
       "              '雄': 362,\n",
       "              '月': 363,\n",
       "              '生': 364,\n",
       "              '游': 365,\n",
       "              '剑': 366,\n",
       "              '强': 367,\n",
       "              '曦': 368,\n",
       "              '毅': 369,\n",
       "              '霞': 370,\n",
       "              '露': 371,\n",
       "              '颖': 372,\n",
       "              '弈': 373,\n",
       "              '楷': 374,\n",
       "              '卉': 375,\n",
       "              '依': 376,\n",
       "              '舟': 377,\n",
       "              '妙': 378,\n",
       "              '麒': 379,\n",
       "              '长': 380,\n",
       "              '荥': 381,\n",
       "              '雯': 382,\n",
       "              '壬': 383,\n",
       "              '常': 384,\n",
       "              '桃': 385,\n",
       "              '优': 386,\n",
       "              '珏': 387,\n",
       "              '璇': 388,\n",
       "              '小': 389,\n",
       "              '松': 390,\n",
       "              '馨': 391,\n",
       "              '圳': 392,\n",
       "              '元': 393,\n",
       "              '琛': 394,\n",
       "              '屹': 395,\n",
       "              '发': 396,\n",
       "              '学': 397,\n",
       "              '羿': 398,\n",
       "              '珺': 399,\n",
       "              '舁': 400,\n",
       "              '翰': 401,\n",
       "              '赋': 402,\n",
       "              '昊': 403,\n",
       "              '仲': 404,\n",
       "              '轩': 405,\n",
       "              '营': 406,\n",
       "              '铭': 407,\n",
       "              '栩': 408,\n",
       "              '缘': 409,\n",
       "              '玛': 410,\n",
       "              '孝': 411,\n",
       "              '甄': 412,\n",
       "              '书': 413,\n",
       "              '纯': 414,\n",
       "              '芷': 415,\n",
       "              '漓': 416,\n",
       "              '珈': 417,\n",
       "              '颍': 418,\n",
       "              '爽': 419,\n",
       "              '腊': 420,\n",
       "              '畅': 421,\n",
       "              '葳': 422,\n",
       "              '丞': 423,\n",
       "              '倚': 424,\n",
       "              '入': 425,\n",
       "              '欢': 426,\n",
       "              '沫': 427,\n",
       "              '刚': 428,\n",
       "              '本': 429,\n",
       "              '嫣': 430,\n",
       "              '悦': 431,\n",
       "              '珍': 432,\n",
       "              '钰': 433,\n",
       "              '飞': 434,\n",
       "              '祺': 435,\n",
       "              '泉': 436,\n",
       "              '筱': 437,\n",
       "              '辰': 438,\n",
       "              '良': 439,\n",
       "              '益': 440,\n",
       "              '氧': 441,\n",
       "              '镇': 442,\n",
       "              '霏': 443,\n",
       "              '兰': 444,\n",
       "              '姝': 445,\n",
       "              '耘': 446,\n",
       "              '容': 447,\n",
       "              '济': 448,\n",
       "              '榕': 449,\n",
       "              '缇': 450,\n",
       "              '若': 451,\n",
       "              '泓': 452,\n",
       "              '碧': 453,\n",
       "              '越': 454,\n",
       "              'M': 455,\n",
       "              '桐': 456,\n",
       "              '杏': 457,\n",
       "              '蒙': 458,\n",
       "              '千': 459,\n",
       "              '里': 460,\n",
       "              '晋': 461,\n",
       "              '诗': 462,\n",
       "              '芸': 463,\n",
       "              '孟': 464,\n",
       "              '禹': 465,\n",
       "              '袄': 466,\n",
       "              '汀': 467,\n",
       "              '鹂': 468,\n",
       "              '韵': 469,\n",
       "              '政': 470,\n",
       "              '芝': 471,\n",
       "              '福': 472,\n",
       "              '恩': 473,\n",
       "              '娜': 474,\n",
       "              '芹': 475,\n",
       "              '钟': 476,\n",
       "              '薰': 477,\n",
       "              '悠': 478,\n",
       "              '也': 479,\n",
       "              '盈': 480,\n",
       "              '郝': 481,\n",
       "              '燕': 482,\n",
       "              '玥': 483,\n",
       "              '军': 484,\n",
       "              '贺': 485,\n",
       "              '磷': 486,\n",
       "              '恺': 487,\n",
       "              '波': 488,\n",
       "              '胜': 489,\n",
       "              '苑': 490,\n",
       "              '笑': 491,\n",
       "              '希': 492,\n",
       "              '铎': 493,\n",
       "              '叶': 494,\n",
       "              '童': 495,\n",
       "              '珑': 496,\n",
       "              '瑚': 497,\n",
       "              '鸣': 498,\n",
       "              '怡': 499,\n",
       "              '涯': 500,\n",
       "              '茄': 501,\n",
       "              '楠': 502,\n",
       "              '箐': 503,\n",
       "              '贞': 504,\n",
       "              '淮': 505,\n",
       "              '影': 506,\n",
       "              '顺': 507,\n",
       "              '豪': 508,\n",
       "              '韩': 509,\n",
       "              '锡': 510,\n",
       "              '风': 511,\n",
       "              '闫': 512,\n",
       "              '妮': 513,\n",
       "              '全': 514,\n",
       "              '行': 515,\n",
       "              '键': 516,\n",
       "              '亲': 517,\n",
       "              '超': 518,\n",
       "              '秋': 519,\n",
       "              '柔': 520,\n",
       "              '育': 521,\n",
       "              '枫': 522,\n",
       "              '璃': 523,\n",
       "              '霜': 524,\n",
       "              '憬': 525,\n",
       "              '徽': 526,\n",
       "              '淇': 527,\n",
       "              '朵': 528,\n",
       "              '潇': 529,\n",
       "              '航': 530,\n",
       "              '勤': 531,\n",
       "              '业': 532,\n",
       "              '令': 533,\n",
       "              '锋': 534,\n",
       "              '绍': 535,\n",
       "              '帆': 536,\n",
       "              '南': 537,\n",
       "              '淳': 538,\n",
       "              '城': 539,\n",
       "              '婕': 540,\n",
       "              '在': 541,\n",
       "              '尚': 542,\n",
       "              '陵': 543,\n",
       "              '甲': 544,\n",
       "              '有': 545,\n",
       "              '珂': 546,\n",
       "              '棣': 547,\n",
       "              '坷': 548,\n",
       "              '潼': 549,\n",
       "              '厘': 550,\n",
       "              '杉': 551,\n",
       "              '渊': 552,\n",
       "              '颔': 553,\n",
       "              '铫': 554,\n",
       "              '衡': 555,\n",
       "              '钧': 556,\n",
       "              '邦': 557,\n",
       "              '祯': 558,\n",
       "              '力': 559,\n",
       "              '满': 560,\n",
       "              '萌': 561,\n",
       "              '溪': 562,\n",
       "              '羚': 563,\n",
       "              '汉': 564,\n",
       "              '与': 565,\n",
       "              '田': 566,\n",
       "              '旸': 567,\n",
       "              '守': 568,\n",
       "              '颂': 569,\n",
       "              '娅': 570,\n",
       "              '于': 571,\n",
       "              '直': 572,\n",
       "              '黛': 573,\n",
       "              '璋': 574,\n",
       "              '奎': 575,\n",
       "              '鉴': 576,\n",
       "              '士': 577,\n",
       "              '婉': 578,\n",
       "              '梦': 579,\n",
       "              '函': 580,\n",
       "              '勋': 581,\n",
       "              '西': 582,\n",
       "              '漫': 583,\n",
       "              '炎': 584,\n",
       "              '暖': 585,\n",
       "              '朝': 586,\n",
       "              '杲': 587,\n",
       "              '灌': 588,\n",
       "              '跏': 589,\n",
       "              '纭': 590,\n",
       "              '忆': 591,\n",
       "              '尘': 592,\n",
       "              '灯': 593,\n",
       "              '东': 594,\n",
       "              '炬': 595,\n",
       "              '桂': 596,\n",
       "              '祉': 597,\n",
       "              '前': 598,\n",
       "              '捧': 599,\n",
       "              '群': 600,\n",
       "              '京': 601,\n",
       "              '晶': 602,\n",
       "              '馥': 603,\n",
       "              '男': 604,\n",
       "              '州': 605,\n",
       "              '旻': 606,\n",
       "              '复': 607,\n",
       "              '进': 608,\n",
       "              '克': 609,\n",
       "              '吟': 610,\n",
       "              '贲': 611,\n",
       "              '系': 612,\n",
       "              '纤': 613,\n",
       "              '如': 614,\n",
       "              '恬': 615,\n",
       "              '紊': 616,\n",
       "              '姚': 617,\n",
       "              '展': 618,\n",
       "              '皈': 619,\n",
       "              '昀': 620,\n",
       "              '豆': 621,\n",
       "              '萱': 622,\n",
       "              '望': 623,\n",
       "              '姗': 624,\n",
       "              '沁': 625,\n",
       "              '霈': 626,\n",
       "              '兔': 627,\n",
       "              '声': 628,\n",
       "              '尤': 629,\n",
       "              '丰': 630,\n",
       "              '汶': 631,\n",
       "              '尧': 632,\n",
       "              '白': 633,\n",
       "              '登': 634,\n",
       "              '微': 635,\n",
       "              '寒': 636,\n",
       "              '师': 637,\n",
       "              '鹤': 638,\n",
       "              '初': 639,\n",
       "              '秩': 640,\n",
       "              '郁': 641,\n",
       "              '励': 642,\n",
       "              '量': 643,\n",
       "              '戎': 644,\n",
       "              '合': 645,\n",
       "              '乾': 646,\n",
       "              '溢': 647,\n",
       "              '申': 648,\n",
       "              '查': 649,\n",
       "              '麓': 650,\n",
       "              '蕴': 651,\n",
       "              '源': 652,\n",
       "              '宾': 653,\n",
       "              '愉': 654,\n",
       "              '皓': 655,\n",
       "              '方': 656,\n",
       "              '礼': 657,\n",
       "              '寿': 658,\n",
       "              '茜': 659,\n",
       "              '铂': 660,\n",
       "              '皑': 661,\n",
       "              '佰': 662,\n",
       "              '武': 663,\n",
       "              '高': 664,\n",
       "              '和': 665,\n",
       "              '烟': 666,\n",
       "              '善': 667,\n",
       "              '浠': 668,\n",
       "              '温': 669,\n",
       "              '标': 670,\n",
       "              '玢': 671,\n",
       "              '呐': 672,\n",
       "              '融': 673,\n",
       "              '威': 674,\n",
       "              '盟': 675,\n",
       "              '曾': 676,\n",
       "              '纳': 677,\n",
       "              '科': 678,\n",
       "              '爱': 679,\n",
       "              '娟': 680,\n",
       "              '怀': 681,\n",
       "              '浙': 682,\n",
       "              '苏': 683,\n",
       "              '祖': 684,\n",
       "              '隆': 685,\n",
       "              '弘': 686,\n",
       "              '之': 687,\n",
       "              '茵': 688,\n",
       "              '萃': 689,\n",
       "              '竹': 690,\n",
       "              '贵': 691,\n",
       "              '荚': 692,\n",
       "              '菲': 693,\n",
       "              '警': 694,\n",
       "              '洱': 695,\n",
       "              '焕': 696,\n",
       "              '蓉': 697,\n",
       "              '湘': 698,\n",
       "              '妹': 699,\n",
       "              '昌': 700,\n",
       "              '同': 701,\n",
       "              '少': 702,\n",
       "              '柱': 703,\n",
       "              '堂': 704,\n",
       "              '懋': 705,\n",
       "              '秦': 706,\n",
       "              '岳': 707,\n",
       "              '向': 708,\n",
       "              '菁': 709,\n",
       "              '夫': 710,\n",
       "              '采': 711,\n",
       "              '保': 712,\n",
       "              '菊': 713,\n",
       "              '乙': 714,\n",
       "              '度': 715,\n",
       "              '培': 716,\n",
       "              '铨': 717,\n",
       "              '日': 718,\n",
       "              '季': 719,\n",
       "              '跻': 720,\n",
       "              '剥': 721,\n",
       "              '卜': 722,\n",
       "              '渲': 723,\n",
       "              '祈': 724,\n",
       "              '亦': 725,\n",
       "              '宛': 726,\n",
       "              '洳': 727,\n",
       "              '扬': 728,\n",
       "              '皙': 729,\n",
       "              '骅': 730,\n",
       "              '烈': 731,\n",
       "              '宸': 732,\n",
       "              '淋': 733,\n",
       "              '薏': 734,\n",
       "              '章': 735,\n",
       "              '禄': 736,\n",
       "              '赛': 737,\n",
       "              '厚': 738,\n",
       "              '夕': 739,\n",
       "              '咛': 740,\n",
       "              '花': 741,\n",
       "              '斓': 742,\n",
       "              '施': 743,\n",
       "              '滢': 744,\n",
       "              '王': 745,\n",
       "              '钢': 746,\n",
       "              '添': 747,\n",
       "              '桥': 748,\n",
       "              '继': 749,\n",
       "              '咏': 750,\n",
       "              '宜': 751,\n",
       "              '虹': 752,\n",
       "              '绩': 753,\n",
       "              '三': 754,\n",
       "              '曼': 755,\n",
       "              '巍': 756,\n",
       "              '拯': 757,\n",
       "              '珠': 758,\n",
       "              '傅': 759,\n",
       "              '樱': 760,\n",
       "              '盛': 761,\n",
       "              '圣': 762,\n",
       "              '针': 763,\n",
       "              '霄': 764,\n",
       "              '晖': 765,\n",
       "              '栋': 766,\n",
       "              '十': 767,\n",
       "              '允': 768,\n",
       "              '锐': 769,\n",
       "              '津': 770,\n",
       "              '照': 771,\n",
       "              '柠': 772,\n",
       "              '廷': 773,\n",
       "              '洹': 774,\n",
       "              '涔': 775,\n",
       "              '河': 776,\n",
       "              '滨': 777,\n",
       "              '味': 778,\n",
       "              '芮': 779,\n",
       "              '訾': 780,\n",
       "              '炳': 781,\n",
       "              '真': 782,\n",
       "              '弟': 783,\n",
       "              '淏': 784,\n",
       "              '路': 785,\n",
       "              '阁': 786,\n",
       "              '演': 787,\n",
       "              '嵩': 788,\n",
       "              '墅': 789,\n",
       "              '鸶': 790,\n",
       "              '棂': 791,\n",
       "              '闱': 792,\n",
       "              '撑': 793,\n",
       "              '芩': 794,\n",
       "              '栏': 795,\n",
       "              '芊': 796,\n",
       "              '茂': 797,\n",
       "              '淅': 798,\n",
       "              '瀛': 799,\n",
       "              '葶': 800,\n",
       "              '周': 801,\n",
       "              '祝': 802,\n",
       "              '暄': 803,\n",
       "              '存': 804,\n",
       "              '琰': 805,\n",
       "              '律': 806,\n",
       "              '熠': 807,\n",
       "              '晰': 808,\n",
       "              '萧': 809,\n",
       "              '鸬': 810,\n",
       "              '珩': 811,\n",
       "              '辛': 812,\n",
       "              '好': 813,\n",
       "              '芯': 814,\n",
       "              '迅': 815,\n",
       "              '莞': 816,\n",
       "              '噇': 817,\n",
       "              '亮': 818,\n",
       "              '豫': 819,\n",
       "              '涿': 820,\n",
       "              '泳': 821,\n",
       "              '纹': 822,\n",
       "              '奇': 823,\n",
       "              '骉': 824,\n",
       "              '闳': 825,\n",
       "              '稳': 826,\n",
       "              '经': 827,\n",
       "              '牧': 828,\n",
       "              '仙': 829,\n",
       "              '翼': 830,\n",
       "              '哓': 831,\n",
       "              '农': 832,\n",
       "              '兵': 833,\n",
       "              '珲': 834,\n",
       "              '典': 835,\n",
       "              '定': 836,\n",
       "              '衍': 837,\n",
       "              '晞': 838,\n",
       "              '上': 839,\n",
       "              '缤': 840,\n",
       "              '翁': 841,\n",
       "              '泵': 842,\n",
       "              '烁': 843,\n",
       "              '慷': 844,\n",
       "              '卫': 845,\n",
       "              '幡': 846,\n",
       "              '淼': 847,\n",
       "              '瓒': 848,\n",
       "              '起': 849,\n",
       "              '岑': 850,\n",
       "              '野': 851,\n",
       "              '潮': 852,\n",
       "              '歌': 853,\n",
       "              '年': 854,\n",
       "              '掌': 855,\n",
       "              '以': 856,\n",
       "              '夏': 857,\n",
       "              '格': 858,\n",
       "              '加': 859,\n",
       "              '乔': 860,\n",
       "              '逊': 861,\n",
       "              '丝': 862,\n",
       "              '郦': 863,\n",
       "              '俏': 864,\n",
       "              '旁': 865,\n",
       "              '鼎': 866,\n",
       "              '帏': 867,\n",
       "              '改': 868,\n",
       "              '从': 869,\n",
       "              '郑': 870,\n",
       "              '粤': 871,\n",
       "              '榇': 872,\n",
       "              '拉': 873,\n",
       "              '程': 874,\n",
       "              '祗': 875,\n",
       "              '荀': 876,\n",
       "              '箔': 877,\n",
       "              '菖': 878,\n",
       "              '垚': 879,\n",
       "              '衷': 880,\n",
       "              '太': 881,\n",
       "              '琨': 882,\n",
       "              '闯': 883,\n",
       "              '意': 884,\n",
       "              '莘': 885,\n",
       "              '迈': 886,\n",
       "              '敬': 887,\n",
       "              '魏': 888,\n",
       "              '翠': 889,\n",
       "              '芒': 890,\n",
       "              '江': 891,\n",
       "              '玺': 892,\n",
       "              '非': 893,\n",
       "              '郅': 894,\n",
       "              '丁': 895,\n",
       "              '圆': 896,\n",
       "              '仔': 897,\n",
       "              '穗': 898,\n",
       "              '央': 899,\n",
       "              '钥': 900,\n",
       "              '译': 901,\n",
       "              '任': 902,\n",
       "              '汝': 903,\n",
       "              '慎': 904,\n",
       "              '米': 905,\n",
       "              '浈': 906,\n",
       "              '铖': 907,\n",
       "              '帧': 908,\n",
       "              '显': 909,\n",
       "              '钫': 910,\n",
       "              '钲': 911,\n",
       "              '颢': 912,\n",
       "              '旋': 913,\n",
       "              '镭': 914,\n",
       "              '寅': 915,\n",
       "              '硕': 916,\n",
       "              '法': 917,\n",
       "              '寰': 918,\n",
       "              '蓝': 919,\n",
       "              '昶': 920,\n",
       "              '材': 921,\n",
       "              '姜': 922,\n",
       "              '竣': 923,\n",
       "              '异': 924,\n",
       "              '偌': 925,\n",
       "              '艾': 926,\n",
       "              '桑': 927,\n",
       "              '岭': 928,\n",
       "              '叆': 929,\n",
       "              '嵋': 930,\n",
       "              '荔': 931,\n",
       "              '囡': 932,\n",
       "              '侯': 933,\n",
       "              '慕': 934,\n",
       "              '笛': 935,\n",
       "              '翌': 936,\n",
       "              '蠡': 937,\n",
       "              '韶': 938,\n",
       "              '弼': 939,\n",
       "              '赫': 940,\n",
       "              '绪': 941,\n",
       "              '轲': 942,\n",
       "              '五': 943,\n",
       "              '腾': 944,\n",
       "              '项': 945,\n",
       "              '媚': 946,\n",
       "              '绎': 947,\n",
       "              '凝': 948,\n",
       "              '炫': 949,\n",
       "              '殳': 950,\n",
       "              '易': 951,\n",
       "              '遥': 952,\n",
       "              '茔': 953,\n",
       "              '傲': 954,\n",
       "              '沄': 955,\n",
       "              '薛': 956,\n",
       "              '鲲': 957,\n",
       "              '巧': 958,\n",
       "              '臻': 959,\n",
       "              '湄': 960,\n",
       "              '呈': 961,\n",
       "              '均': 962,\n",
       "              '跃': 963,\n",
       "              '漱': 964,\n",
       "              '伯': 965,\n",
       "              '颐': 966,\n",
       "              '蓦': 967,\n",
       "              '唯': 968,\n",
       "              '足': 969,\n",
       "              '象': 970,\n",
       "              '栖': 971,\n",
       "              '韫': 972,\n",
       "              '郯': 973,\n",
       "              '隐': 974,\n",
       "              '丙': 975,\n",
       "              '予': 976,\n",
       "              '洗': 977,\n",
       "              '灿': 978,\n",
       "              '基': 979,\n",
       "              '骥': 980,\n",
       "              '檬': 981,\n",
       "              '宪': 982,\n",
       "              '杭': 983,\n",
       "              '筑': 984,\n",
       "              '廑': 985,\n",
       "              '衔': 986,\n",
       "              '疆': 987,\n",
       "              '焰': 988,\n",
       "              '誉': 989,\n",
       "              '历': 990,\n",
       "              '洵': 991,\n",
       "              '睦': 992,\n",
       "              '随': 993,\n",
       "              '璨': 994,\n",
       "              '萄': 995,\n",
       "              '交': 996,\n",
       "              '计': 997,\n",
       "              '划': 998,\n",
       "              '堰': 999,\n",
       "              '骋': 1000,\n",
       "              ...}))"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# checking the char dic\n",
    "dic = char_dict(train_ds)\n",
    "assert(dic['size'] == len(dic))\n",
    "dic['size'], dic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "a222363b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# save the char dic for ease of reusing them later on \n",
    "import json\n",
    "\n",
    "\n",
    "with open('data/char_dic.json', 'w') as f:\n",
    "    json.dump(dic, f)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a2f8e431",
   "metadata": {},
   "source": [
    "**Converting the names and genders as vectors.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "4ef31e28",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "\n",
    "def name2vec(name, char_dic, add_one=True):\n",
    "    '''convert a given name into vec with one-hot encoding. \n",
    "    \n",
    "    Params:\n",
    "        name: str\n",
    "        char_dic: dict\n",
    "            this dict contains the indices for chars in the selected train set that \n",
    "            can be used as an one-hot encoder\n",
    "    Returns:\n",
    "        name_vec: array-like (dim=(1, n+1)) where n = num of chars in the selected train set\n",
    "            please the first columm for this row vector is equal to 1 (i.e., x0=1)\n",
    "    '''\n",
    "    name_vec = np.zeros((1, char_dic['size']+1))\n",
    "    # x0 = 1\n",
    "    name_vec[0,0] = 1\n",
    "    for char in name:\n",
    "        name_vec[0, char_dic[char]] = 1\n",
    "    return name_vec\n",
    "\n",
    "\n",
    "def convert_example(ds, char_dic):\n",
    "    '''Converts the dataset and returns both names and gender as vectors.\n",
    "    '''\n",
    "    # m = num of examples, n = num of dimensions\n",
    "    m, n = len(ds), char_dic['size']\n",
    "    name_vec = np.zeros((m, n+1))\n",
    "    gender_vec = np.zeros((m, 1))\n",
    "    for i in range(m):\n",
    "        name_vec[i] = name2vec(ds[i][0], char_dic)\n",
    "        if ds[i][1] == 'F': gender_vec[i] = 1\n",
    "        elif ds[i][1] == 'U': gender_vec[i] = 2\n",
    "    \n",
    "    return name_vec, gender_vec"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "c0bef65e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training sample size: 100000\n"
     ]
    }
   ],
   "source": [
    "# Converting the dataset \n",
    "name_vec, gender_vec = convert_example(train_ds, dic)\n",
    "assert(len(name_vec) == len(gender_vec))\n",
    "print(f'Training sample size: {len(name_vec)}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f633de93",
   "metadata": {},
   "source": [
    "<a name='model'></a>\n",
    "## Model Training\n",
    "\n",
    "To train the logistic regression model, we first define the functions for the **sigmoid function** as follows:\n",
    "\n",
    "$$sigmoid(z) = \\frac{1}{1+e^{-z}} \\;\\;\\; where \\;\\;\\; z = scalar \\;\\;\\; or \\;\\;\\; z(x) = \\theta^T x$$\n",
    "\n",
    "where $\\theta^T x = \\theta_0 x_0 + \\theta_1 x_1 + ... + \\theta_n x_n = \\sum_{i=0}^{n}{\\theta_i x_i}$ and $x_0 = 1$. Since the $X$ in our model contains either &0& or &1&, separating $\\theta$ as $w$ and $b$ is less useful here.\n",
    "\n",
    "\n",
    "Then the **cost function** for the sigmoid is defined as follows (cross-entropy) without regularization:\n",
    "\n",
    "$$𝐽(𝜃)=−1𝑚∑𝑖=1𝑚[𝑦(𝑖)log(ℎ𝜃(𝑥(𝑖)))+(1−𝑦(𝑖))log(1−ℎ𝜃(𝑥(𝑖)))]$$\n",
    "\n",
    "And the **gradient** for each $\\theta$ (partial derivative) can be formularized as follows without regularization:\n",
    "\n",
    "$$ \\frac{\\partial J(\\theta)}{\\partial \\theta_j} = \\frac{1}{m} \\sum_{i=1}^m \\left( h_\\theta \\left( x^{(i)} \\right) - y^{(i)} \\right) x_j^{(i)} $$\n",
    "\n",
    "We do not add regularization terms for both cost and gradient functions here because the `scipy.optimize.fmin_tnc` will take care of all these. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "124d1300",
   "metadata": {},
   "outputs": [],
   "source": [
    "from scipy import optimize\n",
    "\n",
    "\n",
    "def sigmoid(z):\n",
    "    return 1/(1+np.exp(-z))\n",
    "\n",
    "\n",
    "def costFunc(theta, X, y):\n",
    "    '''\n",
    "    paras:\n",
    "        theta: array-like\n",
    "        X: array-like\n",
    "            the X should have an added first column filled with ones before inputted.\n",
    "        y: array-like\n",
    "    return:\n",
    "        float\n",
    "    '''\n",
    "    y_pred = sigmoid(X @ theta)\n",
    "    J = np.sum(np.multiply(y, np.log(y_pred)) + np.multiply((1-y), np.log(1 - y_pred)))\n",
    "    return -J/len(y)\n",
    "\n",
    "\n",
    "def gradient(theta, X, y):\n",
    "    '''\n",
    "    paras:\n",
    "        theta: array-like\n",
    "        X: array-like\n",
    "            X should have an added first column filled with ones before inputted.\n",
    "        y: array-like\n",
    "            y should be converted into a column vector before inputted. \n",
    "    return:\n",
    "        array-like\n",
    "    '''\n",
    "    return X.T @ (sigmoid(X @ theta) - y) / len(y)\n",
    "\n",
    "\n",
    "def optimized_theta(X, y, theta):\n",
    "    \n",
    "    opt_theta = optimize.fmin_tnc(func = costFunc,\n",
    "                                  x0 = theta, \n",
    "                                  fprime = gradient,\n",
    "                                  args = (X, y.flatten()))\n",
    "    return opt_theta[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "bd9180f2",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-8-916c5e24b73b>:19: RuntimeWarning: divide by zero encountered in log\n",
      "  J = np.sum(np.multiply(y, np.log(y_pred)) + np.multiply((1-y), np.log(1 - y_pred)))\n",
      "<ipython-input-8-916c5e24b73b>:19: RuntimeWarning: invalid value encountered in multiply\n",
      "  J = np.sum(np.multiply(y, np.log(y_pred)) + np.multiply((1-y), np.log(1 - y_pred)))\n"
     ]
    }
   ],
   "source": [
    "# the three sets of theta values that will be used to train three LR classifier  \n",
    "# To save your time, you can simply load the already trained paramters in the next two cell\n",
    "theta_all = np.zeros((dic['size']+1, 3))\n",
    "# train every classifer one at a time: one vs the rest of all\n",
    "for i in [0, 1, 2]:\n",
    "    theta_all[:, i]=optimized_theta(name_vec, gender_vec==i, theta_all[:, i])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "1249666f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# saving the trained model's parameters\n",
    "np.save('data/params.npy', theta_all)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5c775752",
   "metadata": {},
   "outputs": [],
   "source": [
    "# uncomment the below two lines to load the trained paramters directly\n",
    "# theta_all = np.load('data/params.npy')\n",
    "# theta_all"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "41c650ef",
   "metadata": {},
   "source": [
    "<a name='test'></a>\n",
    "## Testing\n",
    "\n",
    "- First, define the function to make prediction based on the model that has already been trained. \n",
    "- Second, define an accuracy function to see the accuracy score of the model as well as the mismatches cases.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "f455513e",
   "metadata": {},
   "outputs": [],
   "source": [
    "def predict(name, char_dic, theta=theta_all, show_all=True, full_name=False):\n",
    "    '''Predicting the gender of Chinese names based on the model trained. \n",
    "    \n",
    "    Params:\n",
    "        name: str, can be both only first name or full name\n",
    "        char_dic: dict that contain char-index pairs\n",
    "        theta: trained multi-class logistic regression model's parameters \n",
    "        show_all: bool, defaults to True. \n",
    "            if changed to False, the output will only show the optimal prediction.\n",
    "    '''\n",
    "    # getFirstName (from utils) returns the frist name of a given name\n",
    "    # no matter whether the last name is included or not\n",
    "    if not full_name:\n",
    "        X = name2vec(name, char_dic)\n",
    "    else:\n",
    "        fname = getFirstName(name)\n",
    "        X = name2vec(fname, char_dic)\n",
    "    prob = sigmoid(np.squeeze(X @ theta))\n",
    "    prob = prob/np.sum(prob)\n",
    "    if show_all:\n",
    "        return name, {'M': prob[0], 'F': prob[1], 'U': prob[2]}\n",
    "    else:\n",
    "        M, F, U = prob\n",
    "        if M==F and F==U: return name, 'M=F=U', M\n",
    "        elif M == np.max(prob): return name, 'M', M\n",
    "        elif F>U: return name, 'F', F\n",
    "        else: return name, 'U', U\n",
    "        \n",
    "        \n",
    "def accuracy(examples, char_dic, theta, exclude_U=False, full_name=False):\n",
    "    right = 0\n",
    "    mismatch = [['name', 'gender', 'pred', 'prob']]\n",
    "    smp_sz = len(examples)\n",
    "    if not exclude_U:\n",
    "        for example in examples:\n",
    "            name, gender = example\n",
    "            _, pred, prob = predict(name, char_dic, theta, show_all=False, full_name=full_name)\n",
    "            if gender == pred: right += 1\n",
    "            else: mismatch.append([name, gender, pred, prob])\n",
    "    else:\n",
    "        for example in examples:\n",
    "            name, gender = example\n",
    "            if gender != 'U':\n",
    "                _, pred, prob = predict(name, char_dic, theta, show_all=False, full_name=full_name)\n",
    "                if gender == pred: \n",
    "                    right += 1\n",
    "                else: mismatch.append([name, gender, pred, prob])\n",
    "            else:\n",
    "                smp_sz -= 1\n",
    "    return right/smp_sz, mismatch"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bdc1e8be",
   "metadata": {},
   "source": [
    "<a name='train_dev'></a>\n",
    "### Testing against the train/dev sets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "b5f56948",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('李柔落', 'F', 0.7406189716495426)\n",
      "('许健康', 'M', 0.9999990047182503)\n",
      "('黄恺之', 'M', 0.9985069564065047)\n",
      "('周牧', 'M', 0.9939343959114006)\n",
      "('梦娜', 'F', 0.9999985293819316)\n",
      "('爱富', 'M', 0.9655679649000578)\n"
     ]
    }
   ],
   "source": [
    "# first test the predict function \n",
    "names = ['李柔落', '许健康', '黄恺之', '周牧', '梦娜', '爱富']\n",
    "\n",
    "for name in names:\n",
    "    print(predict(name, dic, theta=theta_all, show_all=False))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "3dbd1665",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0.96199,\n",
       " [['name', 'gender', 'pred', 'prob'],\n",
       "  ['晔', 'U', 'M', 0.40640049019849533],\n",
       "  ['文冰', 'U', 'M', 0.6743175415253797],\n",
       "  ['文会', 'U', 'M', 0.6135669859259315],\n",
       "  ['乐懿', 'U', 'M', 0.8300395610175295],\n",
       "  ['水荣', 'U', 'M', 0.9270598936609558],\n",
       "  ['正漪', 'M', 'F', 0.7704180986894983],\n",
       "  ['晗', 'M', 'F', 0.6536717990218155],\n",
       "  ['逸', 'U', 'M', 0.7354662573578914],\n",
       "  ['晨云', 'M', 'F', 0.5007160117983905],\n",
       "  ['晓华', 'M', 'F', 0.5262898528073406],\n",
       "  ['宝懿', 'U', 'M', 0.939861336175106],\n",
       "  ['珣', 'U', 'M', 0.47723471469191386],\n",
       "  ['俊华', 'U', 'M', 0.8885340496765125],\n",
       "  ['郡', 'M', 'F', 0.4492423468024235],\n",
       "  ['游', 'U', 'M', 0.8091822301550559],\n",
       "  ['树玉', 'U', 'M', 0.7308590990958171],\n",
       "  ['常桃', 'M', 'F', 0.5039338213337725],\n",
       "  ['英', 'U', 'F', 0.8609835356439166],\n",
       "  ['杨华', 'U', 'M', 0.7864069411425665]])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# test the accuracy of the model on the selected train set\n",
    "# including those gender undefined cases\n",
    "accu, mismatch = accuracy(train_ds, dic, theta_all)\n",
    "accu, mismatch[:20]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "304aeaa5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0.9799027255207855,\n",
       " [['name', 'gender', 'pred', 'prob'],\n",
       "  ['正漪', 'M', 'F', 0.7704180986894983],\n",
       "  ['晗', 'M', 'F', 0.6536717990218155],\n",
       "  ['晨云', 'M', 'F', 0.5007160117983905],\n",
       "  ['晓华', 'M', 'F', 0.5262898528073406],\n",
       "  ['郡', 'M', 'F', 0.4492423468024235],\n",
       "  ['常桃', 'M', 'F', 0.5039338213337725],\n",
       "  ['旭彤', 'F', 'M', 0.7795938004611246],\n",
       "  ['海云', 'F', 'M', 0.6367215582947406],\n",
       "  ['松菱', 'M', 'F', 0.6828052234934916],\n",
       "  ['思嘉', 'M', 'F', 0.529137167730791],\n",
       "  ['华昕', 'M', 'F', 0.44526170742995663],\n",
       "  ['艳忠', 'M', 'F', 0.6786791188738392],\n",
       "  ['琦', 'M', 'F', 0.3581180542375616],\n",
       "  ['新捧', 'F', 'M', 0.5762718239593515],\n",
       "  ['寿丹', 'M', 'F', 0.8798694099727873],\n",
       "  ['云清', 'M', 'U', 0.3532734617770906],\n",
       "  ['桂阳', 'F', 'M', 0.5339663638265105],\n",
       "  ['祥英', 'F', 'M', 0.6135133617781545],\n",
       "  ['晶源', 'F', 'M', 0.5065397044341299]])"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# test the accuracy of the model on the selected train set\n",
    "# excluding those gender undefined cases\n",
    "accu, mismatch = accuracy(train_ds, dic, theta_all, exclude_U=True)\n",
    "accu, mismatch[:20]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "06ad1316",
   "metadata": {},
   "source": [
    "**For the rest of the original train set**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "f33e788d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0.9399856890247558,\n",
       " [['name', 'gender', 'pred', 'prob'],\n",
       "  ['喜凰', 'M', 'F', 0.8309708425166227],\n",
       "  ['贤云', 'U', 'M', 0.7071264765266768],\n",
       "  ['秋田', 'U', 'F', 0.6289107487859427],\n",
       "  ['痄闪', 'M', 'F', 0.9999877462419087],\n",
       "  ['晖敏', 'M', 'U', 0.45086867020702753],\n",
       "  ['启连', 'F', 'M', 0.9484412963561113],\n",
       "  ['海螺', 'F', 'M', 0.7080750455083059],\n",
       "  ['铃庚', 'M', 'F', 0.8383498813784845],\n",
       "  ['元琦', 'U', 'M', 0.9322678356667757],\n",
       "  ['满华', 'F', 'M', 0.7872297253423495],\n",
       "  ['裎鹭', 'M', 'F', 0.9460165883672208],\n",
       "  ['嵛涵', 'M', 'F', 0.5011353607034297],\n",
       "  ['颖辉', 'U', 'F', 0.6657750799137295],\n",
       "  ['向云', 'U', 'M', 0.8847812811524083],\n",
       "  ['云卓', 'M', 'F', 0.4217788799448591],\n",
       "  ['邦美', 'M', 'F', 0.7689001251900788],\n",
       "  ['芊江', 'M', 'F', 0.5320734732173031],\n",
       "  ['俊亚', 'U', 'M', 0.894946746291144],\n",
       "  ['任红', 'F', 'M', 0.6166244729718314]])"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# test the accuracy of the model on the rest of the original train set\n",
    "# including those gender undefined cases\n",
    "accu, mismatch = accuracy(train_ds_dev, dic, theta_all)\n",
    "accu, mismatch[:20]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "9ef44c19",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0.9518350451315245,\n",
       " [['name', 'gender', 'pred', 'prob'],\n",
       "  ['喜凰', 'M', 'F', 0.8309708425166227],\n",
       "  ['痄闪', 'M', 'F', 0.9999877462419087],\n",
       "  ['晖敏', 'M', 'U', 0.45086867020702753],\n",
       "  ['启连', 'F', 'M', 0.9484412963561113],\n",
       "  ['海螺', 'F', 'M', 0.7080750455083059],\n",
       "  ['铃庚', 'M', 'F', 0.8383498813784845],\n",
       "  ['满华', 'F', 'M', 0.7872297253423495],\n",
       "  ['裎鹭', 'M', 'F', 0.9460165883672208],\n",
       "  ['嵛涵', 'M', 'F', 0.5011353607034297],\n",
       "  ['云卓', 'M', 'F', 0.4217788799448591],\n",
       "  ['邦美', 'M', 'F', 0.7689001251900788],\n",
       "  ['芊江', 'M', 'F', 0.5320734732173031],\n",
       "  ['任红', 'F', 'M', 0.6166244729718314],\n",
       "  ['离', 'M', 'F', 0.6067424662448887],\n",
       "  ['条条', 'F', 'U', 0.554112070901956],\n",
       "  ['秩荧', 'M', 'F', 0.9769377797564126],\n",
       "  ['晓嬗', 'F', 'U', 0.47603372050175013],\n",
       "  ['楠希', 'M', 'F', 0.5002595657968659],\n",
       "  ['楫本', 'M', 'U', 0.5020391814162953]])"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# test the accuracy of the model on the rest of the original train set\n",
    "# excluding those gender undefined cases\n",
    "accu, mismatch = accuracy(train_ds_dev, dic, theta_all, exclude_U=True)\n",
    "accu, mismatch[:20]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e0c14aec",
   "metadata": {},
   "source": [
    "**For the original dev set**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "99a4a316",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(167528, [['瑞琳', 'F'], ['凯棋', 'M'], ['义祥', 'M'], ['识闻', 'M'], ['缤鲃', 'M']])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# loading the original dev set\n",
    "dev_ds = data_loader('data/dev_ds.txt')\n",
    "len(dev_ds), dev_ds[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "76b22174",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0.9483429635642997,\n",
       " [['name', 'gender', 'pred', 'prob'],\n",
       "  ['睿', 'U', 'M', 0.8311858450077128],\n",
       "  ['连', 'U', 'F', 0.6128853366948264],\n",
       "  ['丁', 'U', 'M', 0.7661898310914489],\n",
       "  ['乔', 'U', 'M', 0.46442977434012034],\n",
       "  ['韶华', 'U', 'M', 0.8835874436603273],\n",
       "  ['思宁', 'M', 'F', 0.5513951408603314],\n",
       "  ['詹晨', 'M', 'U', 0.4289339476065103],\n",
       "  ['萃卜', 'F', 'M', 0.5218822803705685],\n",
       "  ['畏', 'U', 'M', 0.5632632879903952],\n",
       "  ['云夕', 'M', 'F', 0.5139186479273077],\n",
       "  ['英', 'U', 'F', 0.8609835356439166],\n",
       "  ['亭', 'F', 'M', 0.5835804599462253],\n",
       "  ['云瑞', 'M', 'F', 0.5278466163026296],\n",
       "  ['庚', 'U', 'M', 0.7025191336050127],\n",
       "  ['润培', 'U', 'M', 0.948242222916413],\n",
       "  ['雪明', 'U', 'F', 0.5222983314582598],\n",
       "  ['九', 'U', 'M', 0.8759067846656167],\n",
       "  ['棣华', 'U', 'M', 0.7546019635927789],\n",
       "  ['文', 'U', 'M', 0.6451627573058977]])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# test the accuracy of the model on the original dev set\n",
    "# including those gender undefined cases\n",
    "accu, mismatch = accuracy(dev_ds, dic, theta_all)\n",
    "accu, mismatch[:20]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "71673c5a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0.9651273759520524,\n",
       " [['name', 'gender', 'pred', 'prob'],\n",
       "  ['思宁', 'M', 'F', 0.5513951408603314],\n",
       "  ['詹晨', 'M', 'U', 0.4289339476065103],\n",
       "  ['萃卜', 'F', 'M', 0.5218822803705685],\n",
       "  ['云夕', 'M', 'F', 0.5139186479273077],\n",
       "  ['亭', 'F', 'M', 0.5835804599462253],\n",
       "  ['云瑞', 'M', 'F', 0.5278466163026296],\n",
       "  ['坤玥', 'M', 'F', 0.5255140277152626],\n",
       "  ['乔楚', 'M', 'F', 0.5397862223520067],\n",
       "  ['继红', 'M', 'F', 0.5753134922560033],\n",
       "  ['会会', 'F', 'U', 0.5074093787912801],\n",
       "  ['琳伟', 'M', 'F', 0.7279461459535833],\n",
       "  ['军红', 'F', 'M', 0.9009563144219227],\n",
       "  ['文瑜', 'M', 'F', 0.6038500746856287],\n",
       "  ['佳桤', 'M', 'F', 0.7706763303877168],\n",
       "  ['滠芳', 'M', 'F', 0.9982434513411969],\n",
       "  ['银', 'F', 'M', 0.6033869728115671],\n",
       "  ['晗', 'M', 'F', 0.6536717990218155],\n",
       "  ['舍明', 'M', 'U', 0.5650848281413636],\n",
       "  ['砰', 'M', 'U', 0.554112070901956]])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# test the accuracy of the model on the original dev set\n",
    "# excluding those gender undefined cases\n",
    "accu, mismatch = accuracy(dev_ds, dic, theta_all, exclude_U=True)\n",
    "accu, mismatch[:20]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f965b1fb",
   "metadata": {},
   "source": [
    "<a name='test_set'></a>\n",
    "## Testing against the test set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "0b95af73",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(365810,\n",
       " [['邬爱清', 'F'], ['杜文吕', 'M'], ['任千焱', 'M'], ['鲍梦冉', 'F'], ['薛俊霖', 'M']])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# loading the original test set\n",
    "# please note that the test set contains full names\n",
    "test_ds = data_loader('data/test_ds.txt')\n",
    "len(test_ds), test_ds[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "5221cd2c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0.946272655203521,\n",
       " [['name', 'gender', 'pred', 'prob'],\n",
       "  ['顾仁疋', 'F', 'M', 0.951494650027016],\n",
       "  ['幸路', 'U', 'M', 0.7046140119549571],\n",
       "  ['商禹', 'U', 'M', 0.9143276590605068],\n",
       "  ['易宛其', 'M', 'F', 0.7085827978822331],\n",
       "  ['蔚韦君', 'M', 'F', 0.7219030270992229],\n",
       "  ['任爰好', 'M', 'F', 0.7521378056877494],\n",
       "  ['童仕君', 'U', 'M', 0.8478630748258673],\n",
       "  ['杨晓一', 'U', 'M', 0.7286339835935126],\n",
       "  ['舒海华', 'U', 'M', 0.7885843833905349],\n",
       "  ['卢少冰', 'U', 'M', 0.7701917415910329],\n",
       "  ['范文佳', 'M', 'F', 0.45505228616429405],\n",
       "  ['史梵', 'M', 'F', 0.39515143017123255],\n",
       "  ['吴昕阳', 'U', 'M', 0.9279654532438039],\n",
       "  ['王春清', 'F', 'M', 0.7151642962315684],\n",
       "  ['农月部', 'M', 'F', 0.9903755821554568],\n",
       "  ['淳于文灵', 'M', 'F', 0.5427887968864502],\n",
       "  ['井建华', 'U', 'M', 0.8129042376544283],\n",
       "  ['辜永红', 'F', 'M', 0.702495246245912],\n",
       "  ['安彬', 'U', 'M', 0.8379973144736247]])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# test the accuracy of the model on the original test set\n",
    "# including those gender undefined cases\n",
    "accu, mismatch = accuracy(test_ds, dic, theta_all, full_name=True)\n",
    "accu, mismatch[:20]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "8c91d7b5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0.9701746013923198,\n",
       " [['name', 'gender', 'pred', 'prob'],\n",
       "  ['顾仁疋', 'F', 'M', 0.951494650027016],\n",
       "  ['易宛其', 'M', 'F', 0.7085827978822331],\n",
       "  ['蔚韦君', 'M', 'F', 0.7219030270992229],\n",
       "  ['任爰好', 'M', 'F', 0.7521378056877494],\n",
       "  ['范文佳', 'M', 'F', 0.45505228616429405],\n",
       "  ['史梵', 'M', 'F', 0.39515143017123255],\n",
       "  ['王春清', 'F', 'M', 0.7151642962315684],\n",
       "  ['农月部', 'M', 'F', 0.9903755821554568],\n",
       "  ['淳于文灵', 'M', 'F', 0.5427887968864502],\n",
       "  ['辜永红', 'F', 'M', 0.702495246245912],\n",
       "  ['聂驯侘', 'M', 'U', 0.554112070901956],\n",
       "  ['李来月', 'F', 'M', 0.788230634374804],\n",
       "  ['宁文蔚', 'M', 'F', 0.5391734909647223],\n",
       "  ['常海仪', 'M', 'F', 0.7240201061900376],\n",
       "  ['蓝柳扬', 'F', 'M', 0.5368824575412814],\n",
       "  ['尹昕', 'F', 'M', 0.4691869057902933],\n",
       "  ['查群', 'F', 'U', 0.5271229497252596],\n",
       "  ['雷琦琦', 'M', 'F', 0.3581180542375616],\n",
       "  ['家雅凯', 'M', 'F', 0.6117890475114353]])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# test the accuracy of the model on the original test set\n",
    "# excluding those gender undefined cases\n",
    "accu, mismatch = accuracy(test_ds, dic, theta_all, exclude_U=True, full_name=True)\n",
    "accu, mismatch[:20]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9f522c80",
   "metadata": {},
   "source": [
    "<a name='problem'></a>\n",
    "## A problem\n",
    "\n",
    "When the first names are processed as full names, the accuracy will go down by around 7%. That is because the rule-based `getFirstName` function will take a first name as a full name if the first name starts with a char that can also be used as the last name. \n",
    "\n",
    "Things to do: either improve the accuracy of the `getFirstName` function or train the model with last names as well. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "0036c1d2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0.8710424526049377,\n",
       " [['name', 'gender', 'pred', 'prob'],\n",
       "  ['梅云', 'F', 'U', 0.5925564159943968],\n",
       "  ['穗君', 'F', 'U', 0.5206657693467208],\n",
       "  ['梦饶', 'F', 'M', 0.5512089884475259],\n",
       "  ['佳嘉', 'F', 'M', 0.6504799666787816],\n",
       "  ['玉墨', 'F', 'M', 0.6682442165247393],\n",
       "  ['承隍', 'M', 'U', 0.554112070901956],\n",
       "  ['艳砚', 'F', 'M', 0.5814055986164764],\n",
       "  ['天秘', 'M', 'F', 0.5832588205825965],\n",
       "  ['凤辰', 'F', 'M', 0.7723805377370389],\n",
       "  ['子懿', 'M', 'F', 0.5040915502527811],\n",
       "  ['宇烟', 'M', 'F', 0.9808327176375053],\n",
       "  ['奇', 'M', 'U', 0.554112070901956],\n",
       "  ['佳臻', 'F', 'M', 0.7452530112063541],\n",
       "  ['明华', 'M', 'U', 0.5883437014656338],\n",
       "  ['凤桐', 'F', 'M', 0.5139461732815358],\n",
       "  ['智', 'M', 'U', 0.554112070901956],\n",
       "  ['睿', 'U', 'M', 0.8311858450077128],\n",
       "  ['磊', 'M', 'U', 0.554112070901956],\n",
       "  ['文藻', 'M', 'U', 0.6607274090794647]])"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# originally = 0.9483429635642997\n",
    "# now = 0.8710424526049377\n",
    "accu, mismatch = accuracy(dev_ds, dic, theta_all, full_name=True)\n",
    "accu, mismatch[:20]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "b4b269b3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0.8831277771361534,\n",
       " [['name', 'gender', 'pred', 'prob'],\n",
       "  ['梅云', 'F', 'U', 0.5925564159943968],\n",
       "  ['穗君', 'F', 'U', 0.5206657693467208],\n",
       "  ['梦饶', 'F', 'M', 0.5512089884475259],\n",
       "  ['佳嘉', 'F', 'M', 0.6504799666787816],\n",
       "  ['玉墨', 'F', 'M', 0.6682442165247393],\n",
       "  ['承隍', 'M', 'U', 0.554112070901956],\n",
       "  ['艳砚', 'F', 'M', 0.5814055986164764],\n",
       "  ['天秘', 'M', 'F', 0.5832588205825965],\n",
       "  ['凤辰', 'F', 'M', 0.7723805377370389],\n",
       "  ['子懿', 'M', 'F', 0.5040915502527811],\n",
       "  ['宇烟', 'M', 'F', 0.9808327176375053],\n",
       "  ['奇', 'M', 'U', 0.554112070901956],\n",
       "  ['佳臻', 'F', 'M', 0.7452530112063541],\n",
       "  ['明华', 'M', 'U', 0.5883437014656338],\n",
       "  ['凤桐', 'F', 'M', 0.5139461732815358],\n",
       "  ['智', 'M', 'U', 0.554112070901956],\n",
       "  ['磊', 'M', 'U', 0.554112070901956],\n",
       "  ['文藻', 'M', 'U', 0.6607274090794647],\n",
       "  ['平', 'M', 'U', 0.554112070901956]])"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# originally = 0.9483429635642997\n",
    "# now = 0.8831277771361534\n",
    "accu, mismatch = accuracy(dev_ds, dic, theta_all, exclude_U=True, full_name=True)\n",
    "accu, mismatch[:20]"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}