My motivation

I was so curious about excellence of the image recognition with TensorFlow on Raspberry Pi. Also, the Jupyter notebook is very convenient to instantly code as a quick prototype. So, in terms of error rate of the image classification, Inception V3(3.46%) is more excellent than human(5.1%) whereas raspberry pi’s processing speed is very slow compare to my laptop.

YOLO-Powered_Robot_Vision

Introduction

This is a Pi-based robot to implement visual recognition(by YOLO). The YOLO-Powered vision can recognize many objects such as people, car, bus, fruits, and so on.

Hardware: Raspberry-Pi2, Sony PS3 Eye Camera

(Available to use Logitech C270 USB camera with Raspberry Pi)

Software: YOLO(v2), Jupyter-Notebook

My motivation

I was so interested in performance of the image recognition with YOLO-2 on Raspberry Pi. In addition, the Jupyter notebook is really convenient to instantly code as a quick prototype. According to paper, I realised that YOLO is a fast, accurate visual detector, making it ideal for computer vision system. We connect YOLO to a webcam and verify that it maintains real-time performance. So, the Raspberry pi’s processing speed is very slow compare to my laptop.

I work as a robotics teacher in Sydney. I want to introduce my AI robot to my students in my class next month. In addition, I’m joining NASA Open Innovation Initiative (also known as NASA Space Apps Challenge) with my AI robot to measure the space environment such as temperature, humidity, and pressure. So, I’m so excited!!

Introduction

The IBM Watson Cloud Robot can recognize a human face, voice, and text like a human. The robot clearly recognized the celebrity (Elon Musk) and who he was. Also, it recognized my voice & any text. (YouTube)

This instructable will cover the basic steps that you need to follow to get started with open sources such as Watson nodes (Visual Recognition V3, Speech To Text, Text To Speech) for IBM Bluemix, Node-RED, MQTT v3.1. MQTT(Message Queueing Telemetry Transport) is a Machine-To-Machine(M2M) or Internet of Things (IoT) connectivity protocol that was designed to be extremely lightweight and useful when low battery power consumption and low network bandwidth is at a premium. It was invented in 1999 by Dr. Andy Stanford-Clark and Arlen Nipper and is now an Oasis Standard .

(7) Try dragging & dropping any node from the left-hand side to right-hand side. It’s really easy to code. ( You can conveniently use the visual editor offline as well as online. ) Download all files at Download list. (1) Click the number (1) at the right-hand side corner shown in NodeRED on web-browser. (2) Click the Import button on the drop down menu. (3) Open the Clipboard shown in the above 1st picture. (4) Lastly, paste the given JSON format text of ‘____ver0.1.txt’ (Download List) in Import nodes editor.

Step 5: Setting up MQTT v3.1 on Raspberry Pi2

There are two options such as using eclipse paho, installing a mosquitto sever. Also, you can use (1) option instead of (2) opption.

(1) Using “iot.eclipse.org”.

Click each MQTT node and Type it.

iot.eclipse.org

(2) Setting up MQTT v3.1 on Raspberry Pi2

This message broker(Mosquitto) is supported by MQTT v3.1 and it is easily installed on the Raspberry Pi and somewhat less easy to configure. Next we step through installing and configuring the Mosquitto broker. We are going to install & test the MQTT “mosquitto” on terminal window. Click that.

Step 6: Checking your NodeRED codes with MQTT on Raspberry Pi2

When you will use the JSON format of the ‘NodeRED_Text_files_ver0.1.txt’ (Download List) on Node-RED, it’s automatically set up & coded each data. I have already set up the each data in each node.

(1) Click each node.

(2) Check information inside each node has been prefilled.

(3) Please don’t change the set data. (The above can be customized for more advanced users.)

Step 7: Adding & Setting up PID node, Dashboard on Raspberry Pi2

Searching the Nodes

Node-RED comes with a core set of useful nodes, but there are a growing number of additional nodes available for installing from both the Node-RED project as well as the wider community. You can search for available nodes in the Node-RED library or on the npm repository .

For example, we are going to search ‘node-red-node-pidcontrol’ at the npm web. Click here .

Then, we are going to install npm package, node-red-node-pidcontrol, node-red-dashboard on Raspberry Pi.

To add additional nodes you must first install the npm tool, as it is not included in the default installation. The following commands install npm and then upgrade it to the latest 2.x version.

sudo apt-get update

sudo apt-get install npm

sudo npm install -g npm@2.x

hash -r

cd /home/pi/.node-red

For example, ‘npm install node-red-{example node name}’

Copy the ‘npm install node-red-node-pidcontrol’ from the npm web. Paste it on a terminal window.

Step 8: Configuring the PS3 EYE camera with microphone

This Sony PS3 eye USB camera that can achieve up to 187 frames per second can be found for under $8 on Amazon.com that should make it quite a bargain for those wishing to experiment with CV projects. The PlayStation Eye camera for the PS3 is similar to a web camera but can also be used for computer vision and gesture recognition tasks. The PlayStation Eye has been supported by the Linux kernel since the late Linux 2.6 days but with a future update (Linux 3.20 or later given that the 3.19 merge window is closed) will support higher modes.

(1) Install a USB driver on Raspberry Pi.

sudo apt-get install fswebcam

(2) Take a picture and then check the ‘visionImage.jpg’ file in the /home/pi

(3) Don’t forget to put the Bluemix service credentials for Watson Services such as Visual recognition, Speech to Text, and, Text to Speech. ( How to use the IBM Bluemix platform: https://console.ng.bluemix.net/docs/ )

(4) Make an image file (jpg) server for every boot.

<p>cd /etc/xdg/autostart/</p>

<p>sudo nano imageFileServer.desktop</p>

Type the description below or put the ‘imageFileServer.desktop’ file into /etc/xdg/autostart/ folder.

You need to explicitly enable the serial port on the GPIO pins. The reason for this is a change with the Pi 3 to use the hardware serial port for Bluetooth and instead use a slightly different software serial port for the GPIO pins. A side effect of this change is that the serial port will actually change speed as the Pi CPU clock throttles up and down–this will unfortunately cause problems for most serial devices like GPS receivers!

Luckily there’s an easy fix detailed in this excellet blog post to force the Pi CPU into a fixed frequency which prevents speed changes on the serial port. The Pi might not perform as well but it will have a stable serial port speed.

Introduction

This instructable will cover the basic steps that you need to follow to get started with open sources such as Watson nodes(Visual Recognition V3, Text To Speech) for IBM Bluemix, Node-RED, OpenCV, MQTT v3.1. MQTT(Message Queueing Telemetry Transport) is a Machine-To-Machine(M2M) or Internet of Things (IoT) connectivity protocol that was designed to be extremely lightweight and useful when low battery power consumption and low network bandwidth is at a premium. It was invented in 1999 by Dr. Andy Stanford-Clark and Arlen Nipper and is now an Oasis Standard.

I’ve already published an instructable of the Smart Gas Valve For Safety. In addition, I’m going to communicate between A Smart JPEG Camera and A Smart Gas Valve for M2M Communication by MQTT. Specifically, this instructable will cover how to code the Node-RED on Raspberry Pi2 as a MQTT client by connecting to your home wireless network and how to send sensor data. I will be using A Smart Gas Valve for M2M communication by MQTT.

(7) Try dragging & dropping any node from the left-hand side to right-hand side. It’s really easy to code. ( You can conveniently use the visual editor offline as well as online. ) Download the ‘SmartGasValve_NodeRED.txt’ file. (1) Click the number (1) at the right-hand side corner shown in NodeRED on the web browser.

Step 5: Setting up MQTT v3.1 on Raspberry Pi2

Setting up MQTT v3.1 on Raspberry Pi2

This message broker(Mosquitto) is supported by MQTT v3.1 and it is easily installed on the Raspberry Pi and somewhat less easy to configure. Next, we step through installing and configuring the Mosquitto broker. We are going to install & test the MQTT “mosquitto” on the terminal window.

curl -O http://repo.mosquitto.org/debian/mosquitto-repo.gpg.key

sudo apt-key add mosquitto-repo.gpg.key

rm mosquitto-repo.gpg.key

cd /etc/apt/sources.list.d/

sudo curl -O http://repo.mosquitto.org/debian/mosquitto-jessie.list

sudo apt-get update

Next install the broker and command line clients:

mosquitto – the MQTT broker (or in other words, a server)

mosquitto-clients – command line clients, very useful in debugging

python-mosquitto – the Python language bindings

sudo apt-get install mosquitto mosquitto-clients python-mosquitto

As is the case with most packages from Debian, the broker is immediately started. Since we have to configure it first, stop it.

sudo /etc/init.d/mosquitto stop

Now that the MQTT broker is installed on the Pi we will add some basic security.
Create a config file:

cd /etc/mosquitto/conf.d/
sudo nano mosquitto.conf

Let’s stop anonymous clients connecting to our broker by adding a few lines to your config file. To control client access to the broker we also need to define valid client names and passwords. Add the lines:

Step 8: Adding IBM Watson, IBM NoSQL DB, Play-Audio, and Twilio

Searching the Nodes

Node-RED comes with a core set of useful nodes, but there are a growing number of additional nodes available for installing from both the Node-RED project as well as the wider community. You can search for available nodes in the Node-RED library or on the npm repository.

For example, we are going to search Twilio at the npm web. Click here.

Then, we are going to install Twilio on Raspberry pi.

Installing npm packaged node

To add additional nodes you must first install the npm tool, as it is not included in the default installation. The following commands install npm and then upgrade it to the latest 2.x version.

sudo apt-get update

sudo apt-get install npm

sudo npm install -g npm@2.x

hash -r

cd /home/pi/.node-red

For example, ‘npm install node-red-{example node name}’

Copy the ‘npm install node-red-node-twilio’ from the npm web. Paste it on a terminal window.

Step 10: Testing M2M Communication.

Importing the enclosed files in each NodeRED.

Import the ‘M2M_SmartGasValve.txt‘ into the NodeRED of the smart gas valve.

(3) Check an IP address of the smart gas valve in the Raspberry Pi2.

Type ‘ifconfig’ on a terminal window as shown below.

ifconfig

When you see the IP address, copy the IP address in a terminal window.

(4) Put the IP address into the MQTT node in other Raspberry Pi2.

Click the MQTT node.

Put the IP address into Server.

Step 11: (Optional) Using OpenCV

Installing & Using OpenCV on Raspberry Pi2

We have already used the IBM Watson Visual Recognition. Watson Visual Recognition is very excellent whereas we can’t use it without connecting wifi. OpenCV is possible to use without internet connection but It’s not very easy for a beginner to install & code into OpenCV. So, I’m going to install the OpenCV.

It’s tough for an erstwhile Iron Man to work on creating their personal AI assistant on the weekends. Like any other time-pressured inventor without a PhD in computer science and linguistics, I decided to use a library for speech recognition and synthesis. Fortunately, Python offers several choices. Unfortunately, many of simply them don’t work any more. I will discuss the ones that are still functional and can be used with Python 2.7 and Python 3 (up to Python 3.5 at the time of writing).

I believe that there exist classic deep learning papers which are worth reading regardless of their applications. Rather than providing overwhelming amount of papers, I would like to provide a curated list of the classic deep learning papers which can be considered as must-reads in some area.