Virtual Adversarial Training for Semi-Supervised Text Classification

Venue

Publication Year

Authors

BibTeX

Abstract

Adversarial training provides a means of regularizing supervised learning
algorithms while virtual adversarial training is able to extend supervised learning
algorithms to the semi-supervised setting. However, both methods require making
small perturbations to numerous entries of the input vector, which is inappropriate
for sparse high-dimensional inputs such as one-hot word representations. We extend
adversarial and virtual adversarial training to the text domain by applying
perturbations to the word embeddings in a recurrent neural network rather than to
the original input itself. The proposed method achieves state of the art results on
multiple benchmark semi-supervised and purely supervised tasks. We provide
visualizations and analysis showing that the learned word embeddings have improved
in quality and that while training, the model is less prone to overfitting.