February 20, 2012

Google Developing Siri-Like User Interface for Google TV

Just six days prior to Apple officially launching iCloud and Siri, Google was rushing their patent application to the US Patent and Trademark Office covering a new Google TV remote and/or application. The new remote will use voice controls associated with Google's own cloud services. The user will be able to use their Android Phone as the remote to make inquiries about TV shows and the TV will list what's available as illustrated in our cover graphic. Apple has had a similar feature under Remote for several years now, but it doesn't relate to live TV as Google's will. Google's real competitor on this particular front will come from Samsung who just announced their latest TV remote with voice controls and a touch pad. The race to bring the best next generation TV Remote to market is officially on.

Google's Invention is About Bringing a Siri-Like UI to Google TV

Google's engineers begin their overview by stating that people are interacting more and more with computers, and more specifically, more and more with different kinds of computers. While desktop and laptop computers may have been the most prevalent computers at one time, most people today are now likely interacting with smart phones, DVRs, televisions, and other consumer devices that include computers in them.

Certain computers are well-suited for entering and editing information, such as desktop and laptop computers. Other devices are better suited to delivering information but not receiving it, such as televisions that do not include keyboards, or that have keyboards of limited size. Also, some computers are best used in certain situations, and other computers in other situations. For example, a smartphone is typically best used on-the-go and at close quarters. In contrast, a television is better used while a user is stationary, and frequently from relatively long-distances.

In one such example, a viewer of a television program may be holding a smartphone and be using it as a remote control via an appropriate application or app installed on the smartphone. The smartphone may be programmed with APIs that accept voice input, that package audio of the voice input and send it to a speech-to-text server system via the internet, and that receive in response the text of what was spoken by the user.

The smartphone app may then forward the text to the television, a set-top box, a DVD player, or similar adjunct appliance that can be used with a television. The television may not have similar speech-to-text functionality, so that the use of the smartphone may effectively provide the television with that functionality.

The Ability to Direct Your Android Phone to Set-Up Your TV Show before you ever get Home

According to Google's invention, a user will be able to simply use voice commands on a future Android phone to remotely turn on your TV and have it set to the channel covering your show or sporting event of interest.

The patent specifically states that "the provision of the query to the television may occur when the user is within a set distance of his home also (e.g., by determining with GPS functionality on the smartphone that he is within 1/4 mile of the home), and the television may be turned on automatically as he approaches the home, with the television tuned to a channel that is determined to be most relevant to the query…"

The invention also states that this equally applies to music too. So if you want to have a certain playlist set up and ready to play on your TV when you and your date get home – Mr. or Miss "smoothy" will be able to set the mood as quickly as possible (nudge-nudge, wink-wink).

The Seinfeld TV Show Example

In certain examples, the smartphone may display results that are good for a small-screen display whereas the television may display results that are more appropriate for large-screen displays. For example, the smartphone may display a vertical list of programs that are responsive to a query (e.g., all upcoming episodes of Seinfeld if the user spoke the question "When is Seinfeld?"), while the television may show the same results but in the context of a two-dimensional program guide grid.

The user may step through the results in the list on the smartphone, and the grid may automatically jump, in a synchronized manner, to the corresponding episode on the grid on the television. If the user selects one of the episodes on the list, the television may immediately tune to the episode if it is currently being shown, or may jump to it later when it starts and/or set it for recording on a personal video recorder (PVR).

Google's patent FIG. 1A illustrates an example by which data may be submitted by a first computer (such as a future Android phone), and reviewed and handled at a second computer (such as a Google TV). In this example, the user is shown speaking into the smartphone 104, and asking the query "when is Seinfeld on?" This query, of course, indicates that the user would like to find out when the next episode or episodes of the television situation comedy Seinfeld is being shown by his television provider. The smartphone may be equipped with voice search capabilities, by which certain requests spoken into the smartphone are provided as sound files to a remote server system that may convert the sound files to text and then create search results that are responsive to the request.

The smartphone may initially communicate with a remote server system, receive results back from the remote server system, forward the results or generate information that is derivative of the results, and then send it directly to the television.

The communications with the servers may be over the internet while the communications between the smartphone and the television may be over a WiFi or similar network.

Google's Voice Command System will be Cloud Based

Google's patent FIG. 2A is a schematic diagram of a cloud based system for sharing information between computers. In general, the system (200) is established to allow a user who owns multiple computer devices, to share certain data between devices, including by passing one form of data to the central server system, and having the central server system obtain other data in response to the submissions and provide that other data to a separate target computer that is associated with the user, which association may be identified by determining that the two devices are logged into the same user account.

In one example, Google states that the search system may be implemented so that a user wanting to submit a voice query is required to use a trigger word before the query, either to start the device listening for the query, or to define a context for the query (e.g., "television"). The voice interface (210) may be programmed to extract the trigger word from the text after the speech-to-text conversion occurs, because the trigger word is not truly part of the user's intended query.

Google's patent FIG. 4A illustrates a swim lane diagram for coordinating information submission and information provision between various computers and a central server system. Very much like iCloud services from Apple, Google describes how all of user's devices will be synced together via a cloud system.

Google's patent application was officially filed on September 29, 2011and only recently made public. That's an incredibly fast pace from application to its public publishing date – if not the fastest that I've ever seen.

Samsung's Newest TV Remote with Voice Controls & Touch Pad

Samsung has developed a remote featuring a voice recognition function and a touch pad. It will be offered as an accessory for a smart TV that the company will release in 2012. By speaking to the microphone embedded in the remote, it is possible to change channels or sound volume and input characters.

Although Samsung introduced some of the features of their new remote at CES in January, they've since improved transmission and reception between the remote and the TV by using Bluetooth and infrared communication at the same time.

The race to deliver the best next generation TV Remote with voice and touchpad controls is officially on. Then again, Samsung's upcoming remote may actually be using Google's TV Remote technology to begin with. And if the voice controls work as planned, others will be lining up to adopt it as well – like LG. Perhaps this race will be a rather short one.

Yes, it's true, rumors abound that Apple may be thinking of surprising the market with a true HDTV of their own that will offer a Siri based TV Remote – but that's all they are for now: rumors. So for 2012 – it appears that Google and their ecosystem may have already won Round One with a TKO. But remember, Apple has a track record of delivering explosive results when introducing a new or updated product as we all witnessed with the release of the iPhone 4S with Siri and iCloud in Q4 2011. So I would never, ever count Apple out of the HDTV race just yet. But for now, Google TV is real, vastly improving - and has a rather cool smart TV Remote on the way. It's really that simple for now.

Notice:Patently Apple presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. Revelations found in patent applications shouldn't be interpreted as rumor or fast-tracked according to rumor timetables. About Comments: Patently Apple reserves the right to post, dismiss or edit comments.

Other IP News of Interest:A newly proposed bill going through the Canadain Parliment will give law enforcement agencies the power to snoop through a citizens computer online without a warrant. While it's not law yet, the fact that this is even being considered in a country such as Canada is disturbing. It's a Big Brother type of move that no citizen in any civilized country should tollerate. Read.

Note:The sites that we link to above offer you an avenue to make your comments about this report in other languages. These great community sites also provide our guests with varying takes on Apple's latest invention. Whether they're pro or con, you may find them to be interesting, fun or feisty. If you have the time, join in!

Do people think Google is copying Apple with this? I mean it's not like Google has been working on voice command for several years. It's not like Android had voice command before iPhone. Nah, let's just pretend.

3rd Dude...
Totally agree... Majel has been in the lab 5 years... yet is only emerged in fragmented implementation after Siri was acquired, integrated into iOS and then then rolled out a year later as a core controller for iPhone and soon to be iPad and MBP, iMac etc,.
It's good to be first, doesn't mean they get it right..
And even side by side voice command tests with the Google and Windows versions, only Siri gets it right. Not to mention the hardware fragmentation of where Majel may or may not work on... Happy days!

Guess what.. Google had voice control, voice to text long before apple bought the tech from another company. Even the Woz says Siri cant compete with Googles voice recognition. Siri often gets him wrong fro the same things he says to his Galaxy Nexus and it gets it right on the first try. Then there is MS. They have some sick voice control and the kinect is a prime example. Apple was late to the party and is still behind.

This is not surprising seeing as Android already has several siri competitors, SpeakToIt Assistant is at least as good if not better. Not to mention Google's own project Majel is supposed to blow everything else away.