Mobile

Windows Phone 8 App Development: Using Voice Commands

The query string parameters are very useful when you work with more complex voice commands. I'll dive deeper on how to take advantage of additional query string parameters in more complex scenarios in the next article. It is still necessary to review a few additional things related to the VCD files.

When you executed the app for the first time, the code in the Application_Launching event handler called the Windows.Phone.Speech.VoiceCommands.VoiceCommandService.InstallCommandSetsFromFileAsync method with an asynchronous execution:

This method receives a URI of the XML VCD file to register with the voice commands. In this case, the "ms-appx:///" prefix URI scheme indicates that the file is located in the app's install folder within the file structure of the final XAP package. The method validates all the CommandSet elements included in the specified XML VCD file. However, the method installs and activates only the CommandSet whose xml:lang attribute value matches the specified language for the GSE.

Working with Multiple Languages

In this example, I don't want the Recipes app to be limited to English (United States) voice commands, so I need to expand the XML VCD file to include two additional languages. The following lines for the VCD file define the previously explained voice commands that allow the user to create a new recipe in three languages:

English (United States). The command set is defined with xml:lang="en-US"

Italian (Italy), shown as Italiano in the phone UI. The command set is defined with xml:lang="it-IT"

Spanish (Spain), shown as Español (España) in the phone UI. The command set is defined with xml:lang="es-ES"

Build and start debugging the app. After the emulator launches the Recipes app, go to Settings | Speech | Speech language and select the desired active speech language for the GSE in the emulator. For example, if you select Italiano (Italia), you will be able to use GSE and say any of the following phrases to create a new recipe with your voice. I leave it to you to research the accurate pronunciation of the next phrases in Italian:

Ricette aggiungere

Ricette aggiungere una nuova ricetta

Ricette aggiungere nuova

Ricette aggiungere nuova ricetta

Ricette creare

Ricette creare una nuova ricotta

Ricette creare nuova

Ricette (Recipes in Italian) is the Italian name for the app, defined as the CommandPrefix for xml:lang="it-IT." Any of the phrases will match one of the two ListenFor elements for the voice command defined for the Italian language:

creare [una] [nuova] [ricetta]

aggiungere [una] [nuova] [ricetta]

When the GSE matches the phrase said by the user with the registered CreateNewRecipe command for xml:lang="it-IT," Windows Phone will display the app icon, its name, and the feedback text defined for the voice command in Italian. In this case, the feedback is "Creando una nuova ricetta…" and you will hear the phone's voice reading back the text in Italian (see Figure 16). Then, the debugger will show the same exception that appeared when you tested with English (United States) because there is no AddNewRecipePage yet.

Figure 16: Windows Phone starting the Recipes app and navigating to the AddNewRecipePage as a result of an Italian voice command.

Of course, there is more work to be done to make an app that uses voice commands support multiple languages and cultures with all the necessary adjustments. However, with simple changes to the XML VCD file, you can provide basic support for multiple languages for the voice commands. Then, you have to make the additional changes to the contents of the app to support each different culture.

Working with Phrase Lists

So far, the Recipes app has been using a very simple voice command to create a new recipe. However, I want the Recipes app to allow the users to use voice commands to search for a particular type of cooking recipe with a specific ingredient. I'm going to go back to US English as the single target language in order to keep things simple.

VCD files allow you to define collections of words or phrases with the PhraseList element. Each PhraseList element requires a value for the Label attribute to provide it an identifying name. Then, you can reference the PhraseList in both the ListenFor and Feedback elements enclosing the value of the Label attribute in curly brackets ({}). For example, "search {recipeTypes}" will match both "search" followed by any value specified in the PhraseList whose Label attribute is equal to recipeTypes. The following lines for the VCD file define three PhraseList elements with many words:

recipeTypes

Salads

Sandwiches

Desserts

recipeConnectors

of

with

for

about

recipeElements

tomatoes

potatoes

chicken

cheese

In addition, the VCD file defines the new SearchRecipe command that uses the three PhraseList elements:

Build and start debugging the app. After the emulator launches the Recipes app, go to Settings | Speech | Speech language and select English (United States) as the desired active speech language for the GSE in the emulator. Now, you can activate GSE and say any of the following phrases. Remember that "Recipes" is the CommandPrefix:

Recipes salads with tomatoes

Recipes sandwiches with cheese

Recipes search sandwiches with chicken

Recipes search sandwiches with tomatoes

Any of these phrases will match the ListenFor element for the new voice command, "[search] {recipeTypes} {recipeConnectors} {recipeElements}." The recipeConnectors list makes sure the voice command works when the user uses other words instead of "with" and the phone still understands the search. As a general rule, users don't have perfect grammar when they speak to the phone, and therefore, you should take this into account when you design your voice commands.

When the GSE matches the phrase said by the user with the registered SearchRecipe command, Windows Phone will display the app icon, its name, and the feedback text defined for the voice command. In this case, the feedback is dynamic because it uses the three PhraseList elements as fields. The feedback will use the recognized words to replace the PhraseList elements. For example, if you say, "Recipes search sandwiches with tomatoes," the feedback text will be "Searching sandwiches with tomatoes," and you will hear the phone's voice reading back the text (see Figure 17). Then, the debugger will show an Exception because there is no SearchRecipesPage yet.

Figure 17: Windows Phone starting the Recipes app and navigating to the SearchRecipesPage as a result of the "Recipes search sandwiches with tomatoes" voice command.

The URI that the GSE generates includes additional query string parameters that allow you to determine which values of the PhraseList have been recognized in the voice command. I'll dive deeper on this in the next article and explain how to update the PhraseList elements to dynamically add elements to them within the app. This way, the user will be able to search recipes with more elements than the initial elements included in the VCD file.

In this article, I provided a detailed overview of the new speech experience and started developing a sample app that takes advantage of the new extensible voice commands in Windows Phone 8. In the next article, I'll explain how to retrieve parameters when it is necessary to execute the voice commands, and show how to update phrase lists within the app. In addition, I'll explain how to use the additional Windows Phone 8 speech features to make the app interact with the user.

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task.
However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Video

This month's Dr. Dobb's Journal

This month,
Dr. Dobb's Journal is devoted to mobile programming. We introduce you to Apple's new Swift programming language, discuss the perils of being the third-most-popular mobile platform, revisit SQLite on Android
, and much more!