End result: It’s now possible to pinpoint interesting tidbits embedded deep within shows, where you’d never otherwise be able to find them.

Searching caption information is far from a new idea (I think I first ran across it in software bundled with ATI’s All-in-Wonder TV tuner cards eons ago). And what you really want is a search index that uses not only captions but speech-to-text conversions of the soundtracks of shows, so even programs with no captions are fully incorporated. I suspect we’ll get that for virtually all video on the Web, and it won’t take too long. (If Google isn’t cooking up something along these lines even as we speak, I’d be startled.)

2 Comments For This Post

Harry,
You could be right about Google! They already did come up with automatic captions, which are not perfect but a start. Meanwhile, there is a bill in Congress, HR 3101, the 21st Century Communications and Video Accessibility Act of 2009, that will increase internet captioning.

You got it right about this not being new technology (searching captions.) However, as for your comment about soon expecting speech-to-text conversions of soundtracks so that shows without captions will be incorporated is way off base.

I do want to applaud Hulu for recognizing the value of searchable video. I am an advocate for persons who have hearing loss, a realtime captioner, and the technology officer of a company that has made it its mission to make online video fully searchable and accessible.

Hulu hasn’t cornered the market on searchable video, however. Our searchable video solution (Transendia) is customizable, and we also offer streaming realtime captioning and transcription. Our captioning is off screen, so it doesn’t cover up any of the video. Our searchable video includes meta data and “glossary” terms that the content providers can customize, in addition to the words spoken on the audio track, and videos using our technology can be “searched” from Google directly, from within the video itself, and as part of a collection of videos (like Hulu demonstrates.)

If you’d like to share any information with me to explain how you think the automated speech-to-text will be coming forth soon (with any meaningful level of accuracy), I’d love to hear about it.