Regularized Drop with RoBERTa SEP Pooling

Dacon competitions

• 문장 유형 분류 AI 경진대회

How to Use

• Run train.py

Requirements

• transformers == 4.25.1
• pandas == 1.3.5
• numpy == 1.21.6
• torch == 1.13.1+cu116
• scikit-learn == 1.0.2
• tqdm == 4.64.1
• pyperclip == 1.8.2
• selenium == 4.7.2

Metric

• weighted F1 score

Score

• Public score : 75.771 (6/565) (ensemble seed = [3,4,5])
• Private score : 75.27 (18/565)

Workers

노영준

• Seed ensemble
• CV ensemble
• Exploratory Data Analysis
• Code refactoring
• Project Managing
• Regularized Dropout
• Undersampling

이가은

• AEDA
• Jensen-Shannon Divergence

이재윤

• [SEP] pooling

Citation

• R-Drop: Regularized Dropout for Neural Networks
• AEDA: An Easier Data Augmentation Technique for Text Classification
• RoBERTa: A Robustly Optimized BERT Pretraining Approach