Add Voice Commands to Your Next Project with Bluetooth LE

This project uses the Alorium Technology XLR8, a Bluetooth LE HID Implementation, and the EasyVR voice recognition module to add voice-activated movements to your next project. Or, you can build on the ideas to create a voice-activated remote control.

About the Project

This project uses an Alorium Technology XLR8 to integrate a Voice Recognition Module with a Bluetooth HID device to create a remote voice-activated controller. There are many mills, lathes, plasma tables, and printers on the market that are controlled by computer keyboards. Often, operators find that a part of the machine needs to be moved during loading and unloading of the machine when the operator is handling materials and away from the computer.

This project detects verbal commands and transmits keypresses via Bluetooth LE when a machine operator's hands are too full, too dirty, or too far away to safely operate the controls. It also recreates the HID Keyboard reports for "Page Up", "Page Down", and the keyboard "Arrow Keys", which correspond to directions of movement for the popular CNC controller Mach3.

Alternatively, use the ideas, hardware, and code provided here to build a different project—a voice-activated television remote control, a voice-activated mouse, or a voice-activated robot.

Alorium Technology XLR8

The Alorium Technology XLR8 is a drop-in replacement for the Arduino Uno R3. It was designed to allow Arduino Uno R3 users to have increased performance and capabilities without completely redesigning their projects for a more advanced 16- or 32-bit microcontroller. The XLR8 uses an Altera MAX 10 FPGA to emulate the Arduino's functionality (ATmega328 instruction set). The board is built to the dimensions of the UNO R3 and physically modeled off of the SparkFun RedBoard. For this project, AloriumTech provided the XLR8 board.

Not all of the 50,000 logic elements on the FPGA are needed to recreate the Arduino, which leaves plenty of room for configurable "Xcelerator Blocks" that provide improved performance and response over the ATMega328 in several areas.

The XLR8 blocks are enabled and loaded as libraries through the Arduino IDE, and the end-user never needs to know that they are using an FPGA instead of an 8-bit microcontroller. Current XLR8 blocks include NeoPixel Control, Servo Control, Floating Point Math, and Enhanced Analog-to-Digital (ADC) converter (12-bit resolution at 154k samples/second). Additional improvements include additional Serial Peripheral Interface (SPI) ports.

The XLR8 can also be used for FPGA development, visit this page for more information.

BlueFruit LE Shield

This shield communicates with the host controller through the SPI. This module can be configured for a variety of uses that range from transparent UART mode to HID keyboard emulation and responds to a full AT command set implemented by Adafruit. For this project, AAC purchased the Bluefruit LE Shield.

To use this shield with the XLR8, connect the SPI CLK (pin 13) to GND with a 100Ω - 200Ω resistor.

EasyVR

The EasyVR Shield and Module is a multi-language voice recognition module that can recognize pre-programmed and user-programmed voice commands and execute arbitrary code on an Arduino. It is programmed through a 6-pin UART connection (J7) or with an Arduino by plugging the module into a shield. For this project, AAC purchased the shield and module.

The front and reverse of the easy VR 3.0 module.

The product webpage indicates that this module comes with 26 speaker-independent commands (in English, Italian, Japanese, German, Spanish, and French) and can be programmed with additional speaker dependent commands. It can also output 21 minutes of pre-recorded sounds or speech and can record and play back up to 120 seconds of live messages. The module can be programmed with or without a computer. I recommend using the computer and the included software EasyVR Commander.

Project Overview

For this project, an EasyVR 3.0 Module will be attached to an EasyVR Shield, its purpose it to recognize keywords and alert the Alorium Technology XLR8. The Alorium Technology XLR8 will then send corresponding commands to the Bluefruit LE Shield. The Bluefruit LE Shield will connect to a Bluetooth USB dongle plugged into a computer.

Project connection overview

This board arrangement uses the Adafruit Bluefruit SPI shield and the EasyVR3.0 shield stacked on top of the Alorium Technology XLR8.

Clockwise from top left: Top and bottom sides of the Bluefruit LE shield, the complete project, top and bottom of the EasyVR 3.0 shield.

EasyVR Shield Setup

You can solder the module directly to the shield with the included male headers, but it will be very difficult to remove later. If this is a concern, solder female headers to the shield and male headers to the module.

You must move the SMD resistors on the bottom of the board marked "101." The product ships connecting Tx/Rx to pins 12/13, but to avoid conflict with the other shield, Tx/Rx need to connect to pins 8/9.

Bluefruit Shield Setup

Solder stackable headers on the Bluefruit shield. Use a sharp knife to break the trace that connects CS to pin 8 and reattach CS to pin 10. Place a 100 Ω - 200 Ω resistor on the breadboard area of the shield and use it to connect the SPI CLK pin (13) to GND.

Bluefruit Bluetooth Background Information

Bluetooth Low Energy (BLE)

Bluetooth allows for short-range data transmission between compatible devices. Fortunately for our project, Adafruit did the hard work of implementing the protocol stack and left us with devices that can be relatively quickly and easily programmed for our desired purpose. All of their products are thoroughly documented with one or many sample projects on Adafruit's website and all software & firmware can be found here.

About Bluetooth Connections

Two Bluetooth devices in radio range of one another begin an inquiry process. A device sends an inquiry and the other device responds to the inquiry with its MAC address, name, and other descriptors. The devices page(connect) with one another at the address sent during the inquiry. While connected, the devices pair, and share security information with one another, often with some type of authentication. While paired, the devices bond and save the authentication information so that they can automatically reconnect whenever they are within range of one another.

During prototyping, it is often necessary to return the Bluefruit device to a factory-default state, erasing its stored bonding information. Unfortunately, the Bluetooth USB dongle attached to the computer will not know that your device has reset.

The Bluefruit device begins the process at the inquiry stage again, while the device at the computer has already completed the bonding stage and the devices will never connect. To correct this, delete the Bluefruit device information from the computer and go through the process again. Once you are done prototyping, remove the software flag that allows for a factory reset to eliminate this issue.

Upgrading Bluefruit

Both Bluefruit devices can be connected and upgraded wirelessly using Adafruits "Bluefruit" app and the instructions on their page—the app is available in the Apple App Store and on Google Play.

Bluefruit Configuration

The Bluefruit SPI needs to be configured to be a keyboard—this is accomplished through a series of AT commands. First, change the name to something meaningful.

Sending Printable and Non-Printable Characters and Key Combinations

The Bluefruit device can now send printable and non-printable characters to a linked device. Many characters are easy to send—most ASCII character codes sent through the UART Rx line will be interpreted and transmitted to a remote Bluetooth connected device without any additional work.

A microcontroller that transmits to the Bluefruits' UART Rx at 115200 Baud (8, N, 1) will be able to print or write characters to the serial line and have them appear on the computer screen. The Bluefruits are programmed to take the ASCII code and convert it to a Bluetooth HID keycode, and that keycode is sent to the remote computer.

Both Bluefruit devices can send printable and non-printable characters through the Bluetooth HID reports. To send a non-printable character (or characters) requires a string of hex characters followed by a release code.

**Note: This is equivalent to pressing and releasing [Shift]+[a] on a keyboard and immediately releasing it.

HID codes are not the same as American Standard Code for Information Exchange (ASCII) codes. ASCII codes have a unique number assigned to every character, in ASCII, an "A" is 65, and an "a" is 97. HID keycodes use modifier keys to expand a limited number of base codes. The HID Keycode sequence for an "a" is "00-00-04" and the sequence for "A" is "02-00-04." The first byte modifies the characters that follow.

The second byte that follows the modifier is reserved and is always 0x00. The third byte is the key that you want to press, (again, using the HID Key codes, not ASCII). Up to four additional key codes can be sent in the string to allow for complex multi-key macros. Next, ensure that the keys are released by sending "00-00". Using this method, you can duplicate any single key-press, any key-press combination, and any combination sequence to control almost any computer program or Bluetooth connected device—whether that is adding voice commands that allow hands-free fast forward in Adobe Premiere, or creating custom game controller combinations for a game console.

Quick Synthesis (optional—adds more sounds to your module)

Quick Synthesis allows you to prepare custom sounds for your module. The EasyVR module has one default beep built in, but by installing additional sounds you can programmatically link sounds to events to provide audible feedback for voice-recognition successes and errors.

Select several WAV sounds to use. Use a website such as http://www.bfxr.net/ to create and save three or more custom WAV sounds to a folder on your computer.

Open Sensory QuickSynthesis:

Select File » New
At the prompt select "RSC4 Family"

Type a name for your project and save as type ".qxp"

Select Edit » Add WAV file
Browse to the folder on the computer where you saved your WAV files, and select all that you wish to use
Click Open

Easy VR Commander (Required)

Easy VR Commander allows you to record Speaker Dependent words in 15 different groups, and also allows you to record a custom "Trigger" word in addition to the predefined Trigger word "Robot." Default Speaker Independent words built into the module include "Robot", "Action", "Move", "Turn", "Run", "Look", "Attack", "Stop", "Hello", "Left", "Right", "Up", "Down", "Forward", "Backward", "Zero", "One", "Two", "Three", "Four", "Five", "Six", "Seven", "Eight", "Nine", and "Ten." All words (Speaker Dependent and Speaker Independent), when detected, can be used to execute arbitrary code on the Arduino.

Begin by plugging your EasyVR module into the Easy VR shield, and the Easy VR shield into your XLR8. Set the jumper on the shield to the update (UP) position.

Navigate to the firmware in the install directory "C:\Program Files (x86)\VeeaR\EasyVR Commander\EasyVR3_VW_REV4.EVRFW"

Select Open

☑ Slow Transfer

Select Download

Once the firmware update is complete, disconnect the host microcontroller board from power and USB, and then move the jumper on the shield to "PC." Reconnect the host microcontroller to power and USB.

In Easy VR Commander, select File » Connect

Select "Trigger" Group.

Select Edit » Add Command

Type a name for your Trigger command (e.g. "Computer")

Select Edit » Train Command:

Select Phase 1.

Speak your word into the microphone.

Select Phase 2.

Speak your word into the microphone.

QuickT2SI Software

The QuickT2SI allows for even further customization of the module by adding additional speaker independent recognition keywords. However, it is not a free product, and not necessary for this project, so nothing needs to be done. To read more about all of the products (and watch YouTube videos of how to go through the previous steps) visithttp://www.veear.eu/demos/.

Preparing the Bluefruit

Go to the Apple App Store or Google Play store and download the free app called "Bluefruit" by Adafruit Industries.

Plug the Bluefruit into the XLR8 board and connect to power.

Use the Bluefruit app on your phone to connect to the module and allow it to update.

Putting It All Together

Plug the Bluefruit shield into the XLR8, and plug the EasyVR shield into the Bluefruit Shield. Ensure that the mode jumper is in the "SW" position. Load the code below into the Arduino IDE and upload it to the XLR8 board.

About the Code

This sketch makes extensive use of switch cases to display errors returned from the Easy VR to a variable named "Err" and to change the values of two volatile variables called "Direction" and "Repetitions". If the Speaker Dependent keyword "Computer" is detected, the sketch expects to next hear a direction, and then a number of times to repeat the keypress.

Different circumstances might require that a key is pressed for a longer length of time instead of pressed quickly multiple times. This can be accomplished by inserting a delay before sending the "00-00" command that releases all depressed keys. The sketch makes extensive (although still not complete) efforts to handle errors and returns useful debug information to users through the serial port.

If the Speaker Independent keyword "Robot" is detected, the code will echo any characters typed into the serial monitor until a CR\NL is detected, at which point it starts listening for Trigger Words again.

Going Further

If you are going to create this project, consider adding an operator passcode and additional Speaker Dependent words in the EasyVR Commander software. Also, consider moving the #defines, variables, and unchanging error codes out of the main sketch—I only included them in a single file to prevent information loss as the code makes its rounds on the web.

Conclusion

This project demonstrates a way to add a voice activated keyboard and mouse to a machine, but it can be readily modified to add voice commanded key presses and macros to any device that can interpret Bluetooth LE—from a CNC plasma table to an XBOX One Controller.

And the next time that you create a project with an Arduino Uno R3, and the demands of the project exceed the technical capabilities of the Arduino Uno R3, consider Alorium Tech's XLR8 Board.

Hi @Suraj khande,
The required components are posted at the top of the page—an EasyVR shield that plugs into a BluefruitSPI shield that plugs into the Alorium. You do have to move a few pins around as described int he article.