Source: https://www.kaggle.com/c/quora-question-pairs
- Objective
- Data Description
- Metric
- Feature Extraction
- TFIDF Features
- Best Model
- Technologies Used
- Credits:
- Creator:
Quora is a platform to ask questions and connect with people who contribute unique insights and quality answers. Over 100 million people visit Quora every month, so it's no surprise that many people ask similarly worded questions. Multiple questions with the same intent can cause seekers to spend more time finding the best answer to their question, and make writers feel they need to answer multiple versions of the same question.
If we can identify which questions asked on Quora are duplicates of questions that have already been asked helps us to instantly provide answers to questions that have already been answered. This project focus is to classify whether question pairs are duplicates or not using advanced NLP techniques.
- The cost of a mis-classification can be very high.
- You would want a probability of a pair of questions to be duplicates so that you can choose any threshold of choice.
- No strict latency concerns.
- Interpretability is partially important.
- id: Looks like a simple rowID
- qid{1, 2}: The unique ID of each question in the pair
- question{1, 2}: The actual textual contents of the questions.
- is_duplicate: The label that we are trying to predict - whether the two questions are duplicates of each other.
- Number of rows in = 404,290
Since higher misclassification leads to Customer dissatisfaction (lower precision rate) and consumer disengagement (lower recall rate)
We have to be highly sure about our predictions So probailities of examples belonging to a class seems to be sound metric for our problem. LogLoss metric looks at the probabilities themselves.
- log-loss : LogarithmicLoss (metric we want to optimize for)
- Binary Confusion Matrix (more human intrepretable)
- cwc_min : Ratio of common_word_count to min lenghth of word count of Q1 and Q2
- cwc_max : Ratio of common_word_count to max lenghth of word count of Q1 and Q2
- csc_min : Ratio of common_stop_count to min lenghth of stop count of Q1 and Q2
- csc_max : Ratio of common_stop_count to max lenghth of stop count of Q1 and Q2
- ctc_min : Ratio of common_token_count to min lenghth of token count of Q1 and Q2
- ctc_max : Ratio of common_token_count to max lenghth of token count of Q1 and Q2
- last_word_eq : Check if First word of both questions is equal or not
- first_word_eq : Check if First word of both questions is equal or not
- abs_len_diff : Abs. length difference
- mean_len : Average Token Length of both Questions
- fuzz ratio
- fuzz partial ratio
- token sort ratio
- token set ratio
- Word mover distance
- Normalized word mover distance
- Cosine distance between vectors of question1 and question2
- Manhattan distance between vectors of question1 and question2
- Jaccard similarity between vectors of question1 and question2
- Canberra distance between vectors of question1 and question2
- Euclidean distance between vectors of question1 and question2
- Minkowski distance between vectors of question1 and question2
- Braycurtis distance between vectors of question1 and question2
- Skew of vector for question1
- Skew of vector for question2
- Kurtosis of vector for question1
- Kurtosis of vector for question2
- TF-IDF
- TF-IDF wegihted Word2Vec features
Trained various models like Logistic Regressin, SVM and XGBoost with hyperparameter tuning to select best model. XGBoost turns out be best model with highest accuracy of 84%.
clf = Pipeline([
('features',FeatureUnion([
('nlp_features',DF_NLP_extract()),
('nlp_stats_features',NLP_TextStats()),
('fuzzy_features',DF_fuzzySimExtract()),
('NLP_Dist_features',NLP_DistMetrics(gensim_model = gen_model,verbose = True)),
('TFIDF_vectorizer',Pipeline([
('combine',Converter()),
('TFIDF_wv_features',TfidfVectorizer(ngram_range=(1,2)))]))
]))])
param_grid = {
"n_estimators": [100, 150, 250, 500],
"learning_rate": [0.05, 0.10, 0.15, 0.20, 0.25, 0.30],
"max_depth": [3, 4, 5, 6, 8, 10, 12, 15],
"min_child_weight": [1, 3, 5, 7],
"gamma": [0.0, 0.1, 0.2, 0.3, 0.4],
"colsample_bytree": [0.3, 0.4, 0.5, 0.7],
'eta': [0.1, 0.2, 0.3]
}
{'n_estimators': 500,
'min_child_weight': 1,
'max_depth': 15,
'learning_rate': 0.25,
'gamma': 0.2,
'eta': 0.1,
'colsample_bytree': 0.4}
- FuzzyWuzzy - https://github.com/seatgeek/fuzzywuzzy#usage
- fuzzywuzzy features description - http://chairnerd.seatgeek.com/fuzzywuzzy-fuzzy-string-matching-in-python/
- Is That a Duplicate Quora Question? - https://www.linkedin.com/pulse/duplicate-quora-question-abhishek-thakur/