Awesome Transfer Learning

In this markdown, there is a list of Transfer Learning material mostly for Natural Language Processing, divided into categories by the type of resource. Keep updating! Stay tuned!

Theory Papers:

How Transferable are Neural Networks in NLP Applications? (2016)

They tried to follow a similar setting with CNN for images while transfer to encoder-decoder frameworks. In the paper, they focus on two issues: (1)the specialization of higher layer neurons to their original task at the expense of performance on the target task, which was expected, and (2) optimization difficulties related to splitting networks between co-adapted neurons, which was not expected.

A Pilot Study of Domain Adaptation Effect for Neural Abstractive Summarization (2017, working on summarization)

This is a pilot work for domain adaptation+abstractive summarization. The model is simple but the analysis experiments are solid.

Recent Papers

DARLA: Improving Zero-Shot Transfer in Reinforcement Learning (ICML 17’)

This paper proposed a new multi-stage RL agent to zero-shot transfer learning tasks. The model significantly outperforms conventional baselines in computer vision.

Supervised and Unsupervised Transfer Learning for Question Answering (Naacl 18’) code

They proposed supervised and unsupervised methods for question answering based on three different datasets: TOEFL, MCTest and MovieQA. The model improves the performance on TOEFL dataset by 7%. They show that transfer learning is helpfull in an unsupervised learning setting.

NLP Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm (EMNLP 17’)

The paper shows that by extending the distant supervision to a more diverse set of noisy labels, the models can learn richer representations. The dataset contains 1246 million tweets containing one of 64 common emojis. They achieve sota results on 8 benchmark datasets.

EMNLP 17

Learning to select data for transfer learning with Bayesian Optimization

Cross-Lingual Induction and Transfer of Verb Classes Based on Word Vector Space Specialisation

Cross-Lingual Transfer Learning for POS Tagging without Cross-Lingual Resources

Two-Stage Synthesis Networks for Transfer Learning in Machine Comprehension

ACL 2018

Recursive Neural Structural Correspondence Network for Cross-domain Aspect and Opinion Co-Extraction

Strong Baselines for Neural Semi-supervised Learning under Domain Shift

Domain Adaptation with Adversarial Training and Graph Embeddings

Two Methods for Domain Adaptation of Bilingual Tasks: Delightfully Simple and Broadly Applicable

Domain Adapted Word Embeddings for Improved Sentiment Classification

Zero-Shot Transfer Learning for Event Extraction

Identifying Transferable Information Across Domains for Cross-domain Sentiment Classification

A Helping Hand: Transfer Learning for Deep Sentiment Analysis

Transfer Learning for Context-Aware Question Matching in Information-seeking Conversation Systems in E-commerce

Asymmetric Tri-training for Unsupervised Domain Adaptation

Codes & Papers:

Transfer Learning for Speech and Language Processing (2015)

Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation (2016)

Transfer Learning for Low-Resource Neural Machine Translation

TransNets: Learning to Transform for Recommendation (2017)

A Practitioners’ Guide to Transfer Learning for Text Classification using Convolutional Neural Networks (2018)

Presentation Slides and Talks

http://speech.ee.ntu.edu.tw/~tlkagk/courses/ML_2017/Lecture/transfer.pdf

This is a lecture slides for an introduction. Highly recommend. It has vivid examples for the concepts and few paper introductions. If you know Chinese, you can search for the YouTube video for the lecture.