TJBot - Build a Talking Robot With Watson Conversation

Introduction: TJBot - Build a Talking Robot With Watson Conversation

This instructable guides you through connecting a Raspberry Pi to Watson conversation services and making a talking robot. You will use (1) Watson Speech to Text to convert your voice to text, (2) Watson Conversation to process the text and calculate a response, and (3) Watson Text to Speech to talk the response back.

Step 1: Parts

Step 2: Prepare Your Pi

If you have used Raspberry Pi before, install Node.js and go to the next step. Otherwise, follow the instructions below to set up your Pi:

Getting Started with Your Pi

Raspberry Pi is similar to a full computer, which means you need a monitor, mouse, and keyboard for it. If you have a TV around, you can connect your Pi to your TV via a HDMI cable. In most of the Pi kits, the SD card is already preloaded with an image of the Raspberry Pi Operating System. You need to put the SD card in the Pi, turn the Pi ON and follow the instructions on screen to complete the installation of the operating system. If you have problems setting up your Pi, you can troubleshoot here.

Step 3: Assemble TJBot

Once your TJBot is ready, plug in your USB microphone and the speaker.

Depending on which audio output source you are using with your Pi (HDMI, 3.5mm audio jack, Bluetooth, USB speaker), you may need to set the audio config.

HDMI/ 3.5mm Audio Jack

If you use HDMI or 3.5mm audio jack, you may need to set the audio config. To do this, go to the terminal and open up raspi-config.

sudo raspi-config

This will open up the Raspberry Pi configuration screen:

Select "Advanced Options" and press Enter, then select "Audio" and press Enter. Choose the correct channel for the output audio. If you have connected an external speaker to the audio jack, you should select 3.5mm jack.

USB Speaker

If you have a USB audio, you need to update your /usr/share/alsa/alsa.config to set the USB audio as the default device. Begin with running the following command to make sure your USB is connected and listed there.

lsusb

Next is to detect the card number of your USB audio.

aplay -l

Take a note of the card number associated with your USB Audio. Then go to the alsa.config file to set it as default.

sudo nano /usr/share/alsa/alsa.conf

Look for

defaults.ctl.card 0
defaults.pcm.card 0

and update the card number (0 here) to the card number of your USB audio.

Different versions of Raspberry Pi OS may need a different setup. If you have problem with your USB setup, check out this guide to troubleshoot.

Now we need to copy your credentials from Bluemix. Click on 'Sevice Credentials' on the left menu and copy your credentials into clipboard.

Now make a copy of the default config file.

cp config.default.js config.js

Open config.js file using your favorite text editor and update it with your username and password.

The Conversation service needs one more ID to configure. It's called the Workspace ID (exports.conversationWorkspaceId). Don't worry about it for now. In the next step, we will walk you through creating a conversation workspace.

You are done with the Conversation credentials! Next stop is Watson Speech to Text. Follow the same process to create a service and copy your credentials.

You may leave the default values for the service & credential names and select 'Create'.

Now go to 'Sevice Credentials' on the left menu and copy your credentials into clipboard.

You need to update config.js with your Speech to Text credentials.

The last stop is Watson Text to Speech. You need to do the exact same steps you did with the other two services. You may leave the default values for service and credential name and select 'create'.

Step 6: Create a Conversation Flow

Log in, and in the the top left menu, select the name of your conversation service. Click on 'Import' icon to import the sample conversation.

In the dialog that opens, click 'Choose a file' and select 'workspace-sample.json' from the 'tjbot/recipes/conversation' folder. Make sure 'Everything (Intents, Entities, and Dialog)' is selected.

After the workspace has been imported, click the dotted menu and select 'View details'. Copy your Workspace ID as shown.

Next, update your config.js file with the workspace ID.

Congratulations, your conversation is ready!

Step 7: Run the Code

Now, you are ready to talk to your TJBot!

Open a terminal and execute the following command:

sudo node conversation.js

Watson conversation uses intents to label the purpose of a sentence. For example when you ask TJBot "Please introduce yourself", the intent is to make an introduction. You can add your own new intents in the Conversation editor, but for now, we have started you off with two intents:

Introduction. You can say phrases such as "Watson, please introduce yourself", "Watson, who are you", and "Watson, can you introduce yourself"

Joke. You can ask "Watson, please tell me a joke" or "Watson, I would like to hear a joke".

If you would like to create your own intents, we recommend following this guide.

An attention word is used so TJBot knows you are talking to him. The default attention word is 'Watson', but you can change it in conversation.js as follows. Update conversation.js and change tjConfig to add the 'robot' section:

If you see TJBot's response on the terminal but don't hear TJBot talking, there is a good chance that one of these two things has happened: (1) The audio output is directed to a wrong channel (you can fix it from raspi-config), (2) your sound modules are blocked. In that case, go to /etc/modprobe.d/ and remove blacklist-rgb-led.conf

Then run the following command:

sudo update-initramfs -u

Reboot and confirm the "snd" modules are running by executing the command "lsmod". This should solve the problem.

I am having the same problem. I have reinstalled and still I can't get it to work. I have configured the config.js and removed the default, but still no joy. I had this working in a previous installation but this one is not working. What does it mean by "Cannot find module 'tjbot'?"

I think this must have something to do with the directory that tjbot is installed. I have tried moving it around, but still not working.

I have some prior experience training the Watson Text to Speech and Speech to text systems, but looking through the code in github, I was unable to find if there was some where to specify a customization_id for either? And is it possible to specific the exact voice (so I could use the British voice for example) rather than the default?

I set everything up as you described and my new buddy is working beautifully. Now I want to go a bit further and have him tell me the time whenever I ask him. Since I have conversation.js running most (read: all) of the time, I'd like to expand this function so that it can do this as well. Any tips on how to do this efficiently for an absolute beginner?

Hello everyone, thankyou for reading my question. After setting up tjbot and running the code, everything boots up fine. However, when I give it a command like "watson introduce yourself" it recognises the command and posts it in the console, but then gives an error

The error above indicates you may be posting audio to the speech to text services that is above the normal payload. Normally this should not occur after a single short sentence (which is well below the limit). I just tested the STT service locally and it does work fine at my location.A few things I would suggest ...- If possible, please do try out a new set of STT credentials (it is possibly that your current credentials are in being used in a different application hence the limits)- Also try the to run the STT sample and let me know if you still have the same issue.- Please open an issue on Github - its easier to follow up there and track our discussion.

It appears you have created your dialog to have two consecutive turns ( user, TJ, TJ, user). Does the user say anything between these turns? Usually each party has a SINGLE turn - user, TJ, user, TJ, user, TJ etc. To maintain this pattern ... will the following work?

Got a problem, the conversation service returns an error. Can't find the problem. I have other recipes working (Speech to Text, Sentiment Analysis) and I have tested all tjbot-devices (ran all tests in jbot/bootstrap/tests). The error that I got is:

info: TJBot heard: Watson how are you error: the conversation service returned an error. Error: Resource not found at Request._callback (/home/pi/tjbot/recipes/conversation/node_modules/watson-developer-cloud/lib/requestwrapper.js:87:15) at Request.self.callback (/home/pi/tjbot/recipes/conversation/node_modules/request/request.js:188:22) at emitTwo (events.js:106:13) at Request.emit (events.js:191:7) at Request.<anonymous> (/home/pi/tjbot/recipes/conversation/node_modules/request/request.js:1171:10) at emitOne (events.js:96:13) at Request.emit (events.js:188:7) at Gunzip.<anonymous> (/home/pi/tjbot/recipes/conversation/node_modules/request/request.js:1091:12) at Gunzip.g (events.js:292:16) at emitNone (events.js:91:20) at Gunzip.emit (events.js:185:7) at endReadableNT (_stream_readable.js:974:12) at _combinedTickCallback (internal/process/next_tick.js:80:11) at process._tickCallback (internal/process/next_tick.js:104:9)

and in another repo by the same guy i found "Any small speaker with either a 3.5mm audio jack or Bluetooth will work. Note that if you are using the 3.5mm audio jack, you may wish to add a USB Audio Adapter to avoid audio interference with the LED."

Thank you, TJBot. I tried to order at amazon.com, but many of the components won't ship outside of US. I then tried at amazon.ca, can't find exactly the same brand for some components. I s there anyone in Toronto that bought all the required materials?

We've only seen that error come up when putting in the credentials incorrectly. The username is the very long string with dashes and the password is the shorter string. Make sure you are not copying with extra spaces at the beginning or at the end.

I deleted all Watson services changed the region from 'Germany' to 'United States (South)' created a division (no sure if it is division only have the german interface). After that I created all services anew, also noticed that there are more services available.

Works almost perfect only one thing: If I just say 'watson' and nothing else it crashes with some 'no text' error.

I get error:SyntaxError: Unexpected token = at exports.runInThisContext (vm.js:53:16) at Module._compile (module.js:373:25) at Object.Module._extensions..js (module.js:416:10) at Module.load (module.js:343:32) at Function.Module._load (module.js:300:12) at Module.require (module.js:353:17) at require (internal/module.js:12:17) at Object.<anonymous> (/home/pi/tjbot/recipes/conversation/node_modules/tjbot/index.js:1:80) at Module._compile (module.js:409:26) at Object.Module._extensions..js (module.js:416:10)

When running npm install I had 1 warning:

npm WARN deprecated node-uuid@1.4.8: Use uuid module instead

Username and password x 3 double checked as was exports.conversationWorkspaceId and opening and closing quotes.

Note this is a working Jessie from a Pi 1 B+ imaged to a new Pi 3. I ran steps 1 to 6 OK (well took a day to get the BT speaker working!). A USB microphone is installed but not tested. LEDs and servo not yet connected.

Seems to me this code is trying to control the LED but this Instructable does not include the LED in the build whereas another Instructable "Build TJBot Out of Cardboard" does have the LED. I did not fit the LED so why does it appear the conversation.js code is trying to control the LED?

Hi Alan,The above error is helpful in troubleshooting. The way the TJBot library is designed, there should be no call to "pulse" or LED since "led" is not specified as a config parameter in the conversation.js recipe.The correct way to disable LED is to not have it in the config parameter.

var hardware = ['microphone', 'speaker'];

One thing we can try is to uninstall and reinstall your node_modules.sudo rm -rf node_modulesnpm install.

I don't really need the LED and the parts list for this instructables doesn't list the LED but the code seems to crash trying to use an LED. Is there any way to disable the LED function so I can continue with the Conversation functions? Help much appreciated as I was hoping to show TJBot at meeting in 10 days time.