LipNet Reads Lips - Until Disconnected, That Is

In 2001: A Space Odyssey, the 1968 film by Stanley Kubrick and Clarke, two astronauts discuss a matter that they believe is too sensitive for the HAL 9000 computer. So, they sit in a small capsule, insulated from HAL. But HAL can read their lips...

You'll need to see the film to find out how it turns out. But what are the odds that a computer could read lips, anyway?

(LipNet - Deep Mind at work)

The BBC supplied the Oxford researchers with clips from Breakfast, Newsnight, Question Time and other BBC news programmes, with subtitles aligned with the lip movements of the speakers. Then a neural network combining state-of-the-art image and speech recognition set to work to learn how to lip-read.

After examining 118,000 sentences in the clips, the system now has 17,500 words stored in its vocabulary. Because it has been trained on the language of news, it is now quite good at understanding that "Prime" will often be followed by "Minister" and "European" by "Union", but much less adept at recognising words not spoken by newsreaders.