Using Siri, Google Now or Cortana to talk with your smartphone is pretty normal these days. But have you ever considered creating your own Voice Dialog? And would you think this is possible within a hybrid app using Cordova and the Ionic Framework? Let me prove it to you.

In this article I will show you how to build text to speech voice output and speech recognition input using Cordova plugins with the Ionic Framework.

Setting up the App

We start with a blank Ionic app and install our two Cordova plugins, so go ahead and run:

This creates a new folder cordova-speech with a clean Ionic Framework app and adds two Cordova plugins we need for our example:

The first plugin is a cordova wrapper for the native text to speech output functions. This plugin uses the AVSpeechSynthesizer on iOS and the android.speech.tts.TextToSpeech package on Android, so it’s really a simple mapper of the best native functions for giving out text. This plugin also seems to work for Windows phone, but I had no chance to test this part. If one of you has a Windows Phone why not give it a try and leave a comment with your findings below?

The second plugin is a cordova wrapper for speech recognition. On Android this plugin uses the native android.speech.SpeechRecognizer package, on iOS it’s a bit harder to achieve this task so the author used the iSpeech SDK which you can test for free for now. If you plan to use this SDK in production additional costs may arise.

Besides those two plugins we don’t need any more tools or fancy components inside our app!

Crafting our Speech View

The view of our app will be quite simple. First of all we need an input field for some text which our app can speak later on.

Below this field we need two buttons, the first is to call our text to speech action so the text we have inserted get’s read to our user. The second button triggers our speech recognition dialog and listens for whatever we say.

The recorded text (or what the app thinks we have said) will be printed below our other components, just to see how precise the recognition is. This can vary quite a bit, depending on surrounding sounds, how clear you speak, how difficult the words and sentences are and so on. You can play around with the results later and leave your opinions about this plugin below!

For now go ahead and open your www/index.html and replace the current body with:

As you can see there is not that much code needed for quite cool features! Let’s take a closer look at our controller implementation.

We create a data object which will is connected to our input field. We need to make it an object so the value is updated correctly using ng-model.

If we want our inserted text to be spoken, the speakText() function get’s called. We use the Cordova plugin by calling TTS.speak() and also pass an object with some properties. In this example we pass the text (which you should always do) and also a language and a rate which defines how fast the text is spoken. You can create quite funny outputs playing around with these values, trust me I had my fun creating this article!
Finally we can also specify a success and error callback, which I left empty for now because handling those errors is very custom for every app.

Our voice recognition starts on calling record(). Here we create a new instance of a SpeechRecognition object. After creating it, we assign a function to the onresult property of the object. This function will be called once the recognizer thinks he is done listening to you talking.

We check if we actually have any result, and if that’s the case we apply the text our recognizer understood to our scope variable which appears below our other view elements.

After setting the result block we only have to start our recognition and the app starts listening, that’s all!

Conclusion

In this step-by-step tutorial I explained how to use two Cordova plugins for text to speech and voice recognition inside an Ionic app. You might think that these two features are hard to implement, but given the great wrappers they are super cheap and easy to include in your hybrid app.

Holly,
You may want to add this to your blog. I have recently compiled a version history for Phonegap and Cordova. I noted a similar blog from Simon Macdonald. The story was posted on the ‘Phonegap Network’

Anna Has Been Pulled From the Play Storehttp://simonmacdonald.blogspot.com/2014/02/anna-has-been-pulled-from-play-store.html
“””
Back in September of 2012 I posted up my sample app called Anna which
highlighted the power of doing speech recognition on mobile devices
using PhoneGap. Today I’ve had to pull the app from the Play Store
because of this infringement notice from Wolfram Alpha.
“””
The source code is still available online.

I found a little bug and don’t know how to fix it. I have the code snippet above now working in my app. Well, sort of. I can run the app and it will speak text that I enter correctly. But, when I record, the very first time through it does not capture and report any text. Only on the second time hitting record will it process correctly. It does this in my app and in the app outlined above when I create it.

Ex. Start app. Hit record – it beeps, records, and beeps complete. No text is written to screen. Hit record again immediately. It beeps – records, and beeps complete. Then this text will show up on screen and continue to function properly. It is only the first time recording where it is not working.

Sounds pretty weird! Have you tried to debug/log what is coming back in the completion blog when you record for the first time? Would be interesting to see if maybe no text comes back, than this might be some problem of the plugin.

Hmm. Well, I’m a newb to ionic and cordova, but have had some success using other plug-ins. However, can’t seem to get TTS or SpeechRecognition to be recognized as variables in the JS (Error: Can’t find variable: TTS and Error: Can’t find variable: SpeechRecognition) when serving up with phonegap. Any hints? Using Windows and VS for development but haven’t had any issues with other plug-ins. Thanks. Dave.