Proceedings of The 33rd International Conference on Machine Learning, PMLR 48:173-182, 2016.

Abstract

We show that an end-to-end deep learning approach can be used to recognize either English or Mandarin Chinese speech–two vastly different languages. Because it replaces entire pipelines of hand-engineered components with neural networks, end-to-end learning allows us to handle a diverse variety of speech including noisy environments, accents and different languages. Key to our approach is our application of HPC techniques, enabling experiments that previously took weeks to now run in days. This allows us to iterate more quickly to identify superior architectures and algorithms. As a result, in several cases, our system is competitive with the transcription of human workers when benchmarked on standard datasets. Finally, using a technique called Batch Dispatch with GPUs in the data center, we show that our system can be inexpensively deployed in an online setting, delivering low latency when serving users at scale.

Related Material

@InProceedings{pmlr-v48-amodei16,
title = {Deep Speech 2 : End-to-End Speech Recognition in English and Mandarin},
author = {Dario Amodei and Sundaram Ananthanarayanan and Rishita Anubhai and Jingliang Bai and Eric Battenberg and Carl Case and Jared Casper and Bryan Catanzaro and Qiang Cheng and Guoliang Chen and Jie Chen and Jingdong Chen and Zhijie Chen and Mike Chrzanowski and Adam Coates and Greg Diamos and Ke Ding and Niandong Du and Erich Elsen and Jesse Engel and Weiwei Fang and Linxi Fan and Christopher Fougner and Liang Gao and Caixia Gong and Awni Hannun and Tony Han and Lappi Johannes and Bing Jiang and Cai Ju and Billy Jun and Patrick LeGresley and Libby Lin and Junjie Liu and Yang Liu and Weigao Li and Xiangang Li and Dongpeng Ma and Sharan Narang and Andrew Ng and Sherjil Ozair and Yiping Peng and Ryan Prenger and Sheng Qian and Zongfeng Quan and Jonathan Raiman and Vinay Rao and Sanjeev Satheesh and David Seetapun and Shubho Sengupta and Kavya Srinet and Anuroop Sriram and Haiyuan Tang and Liliang Tang and Chong Wang and Jidong Wang and Kaifu Wang and Yi Wang and Zhijian Wang and Zhiqian Wang and Shuang Wu and Likai Wei and Bo Xiao and Wen Xie and Yan Xie and Dani Yogatama and Bin Yuan and Jun Zhan and Zhenyao Zhu},
booktitle = {Proceedings of The 33rd International Conference on Machine Learning},
pages = {173--182},
year = {2016},
editor = {Maria Florina Balcan and Kilian Q. Weinberger},
volume = {48},
series = {Proceedings of Machine Learning Research},
address = {New York, New York, USA},
month = {20--22 Jun},
publisher = {PMLR},
pdf = {http://proceedings.mlr.press/v48/amodei16.pdf},
url = {http://proceedings.mlr.press/v48/amodei16.html},
abstract = {We show that an end-to-end deep learning approach can be used to recognize either English or Mandarin Chinese speech–two vastly different languages. Because it replaces entire pipelines of hand-engineered components with neural networks, end-to-end learning allows us to handle a diverse variety of speech including noisy environments, accents and different languages. Key to our approach is our application of HPC techniques, enabling experiments that previously took weeks to now run in days. This allows us to iterate more quickly to identify superior architectures and algorithms. As a result, in several cases, our system is competitive with the transcription of human workers when benchmarked on standard datasets. Finally, using a technique called Batch Dispatch with GPUs in the data center, we show that our system can be inexpensively deployed in an online setting, delivering low latency when serving users at scale.}
}