13.
Transfer Learning for NLP
@seb_ruder |
Current status
• Not as straightforward as in CV
- No universal deep features
• However: “Simple” transfer through word
embeddings is pervasive
• History of research for task-speciﬁc transfer, e.g.
sentiment analysis, POS tagging leveraging NLP
phenomena such as structured features, sentiment
words, etc.
• Few research on transfer between tasks
• More recently: representation-based research
01.03.17 | LinkedIn Tech Talk

14.
Our research
@seb_ruder |
Research focus
Finding better ways to transfer knowledge to new
domains, tasks, and languages that
1. perform well in large-scale settings and real-
world applications;
2. are applicable to many tasks and models.
Current focus:
: Training and test distributions are
different.
P(XS) 6= P(XT )
01.03.17 | LinkedIn Tech Talk

15.
Our research
@seb_ruder |
Training and test distributions are different.
Different text types. Different accents/ages.
Different topics/categories.
Performance drop or even collapse is inevitable.
01.03.17 | LinkedIn Tech Talk