Category: Computer Vision

Introduction

This instructable will cover the basic steps that you need to follow to get started with open sources such as Watson nodes(Visual Recognition V3, Text To Speech) for IBM Bluemix, Node-RED, OpenCV, MQTT v3.1. MQTT(Message Queueing Telemetry Transport) is a Machine-To-Machine(M2M) or Internet of Things (IoT) connectivity protocol that was designed to be extremely lightweight and useful when low battery power consumption and low network bandwidth is at a premium. It was invented in 1999 by Dr. Andy Stanford-Clark and Arlen Nipper and is now an Oasis Standard.

I’ve already published an instructable of the Smart Gas Valve For Safety. In addition, I’m going to communicate between A Smart JPEG Camera and A Smart Gas Valve for M2M Communication by MQTT. Specifically, this instructable will cover how to code the Node-RED on Raspberry Pi2 as a MQTT client by connecting to your home wireless network and how to send sensor data. I will be using A Smart Gas Valve for M2M communication by MQTT.

(7) Try dragging & dropping any node from the left-hand side to right-hand side. It’s really easy to code. ( You can conveniently use the visual editor offline as well as online. ) Download the ‘SmartGasValve_NodeRED.txt’ file. (1) Click the number (1) at the right-hand side corner shown in NodeRED on the web browser.

Step 5: Setting up MQTT v3.1 on Raspberry Pi2

Setting up MQTT v3.1 on Raspberry Pi2

This message broker(Mosquitto) is supported by MQTT v3.1 and it is easily installed on the Raspberry Pi and somewhat less easy to configure. Next, we step through installing and configuring the Mosquitto broker. We are going to install & test the MQTT “mosquitto” on the terminal window.

curl -O http://repo.mosquitto.org/debian/mosquitto-repo.gpg.key

sudo apt-key add mosquitto-repo.gpg.key

rm mosquitto-repo.gpg.key

cd /etc/apt/sources.list.d/

sudo curl -O http://repo.mosquitto.org/debian/mosquitto-jessie.list

sudo apt-get update

Next install the broker and command line clients:

mosquitto – the MQTT broker (or in other words, a server)

mosquitto-clients – command line clients, very useful in debugging

python-mosquitto – the Python language bindings

sudo apt-get install mosquitto mosquitto-clients python-mosquitto

As is the case with most packages from Debian, the broker is immediately started. Since we have to configure it first, stop it.

sudo /etc/init.d/mosquitto stop

Now that the MQTT broker is installed on the Pi we will add some basic security.
Create a config file:

cd /etc/mosquitto/conf.d/
sudo nano mosquitto.conf

Let’s stop anonymous clients connecting to our broker by adding a few lines to your config file. To control client access to the broker we also need to define valid client names and passwords. Add the lines:

Step 8: Adding IBM Watson, IBM NoSQL DB, Play-Audio, and Twilio

Searching the Nodes

Node-RED comes with a core set of useful nodes, but there are a growing number of additional nodes available for installing from both the Node-RED project as well as the wider community. You can search for available nodes in the Node-RED library or on the npm repository.

For example, we are going to search Twilio at the npm web. Click here.

Then, we are going to install Twilio on Raspberry pi.

Installing npm packaged node

To add additional nodes you must first install the npm tool, as it is not included in the default installation. The following commands install npm and then upgrade it to the latest 2.x version.

sudo apt-get update

sudo apt-get install npm

sudo npm install -g npm@2.x

hash -r

cd /home/pi/.node-red

For example, ‘npm install node-red-{example node name}’

Copy the ‘npm install node-red-node-twilio’ from the npm web. Paste it on a terminal window.

Step 10: Testing M2M Communication.

Importing the enclosed files in each NodeRED.

Import the ‘M2M_SmartGasValve.txt‘ into the NodeRED of the smart gas valve.

(3) Check an IP address of the smart gas valve in the Raspberry Pi2.

Type ‘ifconfig’ on a terminal window as shown below.

ifconfig

When you see the IP address, copy the IP address in a terminal window.

(4) Put the IP address into the MQTT node in other Raspberry Pi2.

Click the MQTT node.

Put the IP address into Server.

Step 11: (Optional) Using OpenCV

Installing & Using OpenCV on Raspberry Pi2

We have already used the IBM Watson Visual Recognition. Watson Visual Recognition is very excellent whereas we can’t use it without connecting wifi. OpenCV is possible to use without internet connection but It’s not very easy for a beginner to install & code into OpenCV. So, I’m going to install the OpenCV.

Motivation
Modern GPU accelerators has become powerful and featured enough to be capable to perform general purpose computations (GPGPU). It is a very fast growing area that generates a lot of interest from scientists, researchers and engineers that develop computationally intensive applications. Despite of difficulties reimplementing algorithms on GPU, many people are doing it to check on how fast they could be. To support such efforts, a lot of advanced languages and tool have been available such as CUDA, OpenCL, C++ AMP, debuggers, profilers and so on.

Significant part of Computer Vision is image processing, the area that graphics accelerators were originally designed for. Other parts also suppose massive parallel computations and often naturally map to GPU architectures. So it’s challenging but very rewarding to implement all these advantages and accelerate OpenCV on graphics processors.

History
OpenCV includes GPU module that contains all GPU accelerated stuff. Supported by NVIDIA the work on the module, started in 2010 prior to the first release in Spring of 2011. It includes accelerated code for siginifcant part of the library, still keeps growing and is being adapted for the new computing technologies and GPU architectures.

Goals
1. Provide developers with a convenient computer vision framework on the GPU, maintain conceptual consistency with the current CPU functionality.
2. Achieve the best performance with GPUs (efficient kernels tuned for modern architectures, optimized dataflow like async. execution, copy overlaps, zero-copy)
3. Completeness (implement as much as possible, even if speed-up is not fantastic; such allows to run an algorithm entirely on GPU and save on coping overheads)

Performance

Design considerations
OpenCV GPU module is written using CUDA, therefore it benefits from the CUDA ecosystem. There is a large community, conferences, publications, many tools and libraries developed such as NVIDIA NPP, CUFFT, Thrust.
The GPU module is designed as host API extension. This design provides the user an explicit control on how data is moved between CPU and GPU memory. Although the user has to write some additional code to start using the GPU, this approach is both flexible and allows more efficient computations.
GPU modules includes class cv::gpu::GpuMat which is a primary container for data kept in GPU memory. It’s interface is very similar with cv::Mat, its CPU counterpart. All GPU functions receive GpuMat as input and output arguments. This allows to invoke several GPU algorithms without downloading data. GPU module API interface is also kept similar with CPU interface where possible. So developers who are familiar with Opencv on CPU could start using GPU straightaway.

Short sample
In the sample below an image is loaded from png0file, next it is uploaded to GPU, thresholded, downloaded and displayed.

Installing
This is a port of the Python bindings for OpenCV, and as such lives in the OpenCV source tree. When configured properly, the tree will produce an Octave binary package.

You must have a recent version of SWIG in your PATH (like from SVN, such that it supports Octave) and configure the tree with –with-octave and –with-swig. The build will produce an Octave package at $prefix/share/opencv/opencv-1.0.tar.gz. For example (assuming the loader sees /usr/local/lib),

Using the library
There is extensive documention on the library available here, and online help is available using the help command in octave. The online help gives only the function prototypes, but when used in conjunction with the OpenCV manual, it should be sufficient to use the library.

Interoperability with Octave matrices and images is provided via the cv2mat, mat2cv, cv2im, and im2cv functions. There is online help available for these.

Releases
3/27/08: These bindings have been integrated into OpenCV. Latest sources are now available only from OpenCV CVS.

[ Configuring Qt for OpenCV on OSX ]

In this tutorial we will learn how to configure Qt to use OpenCV. Although the tutorial is targeted for OSX users, you can modify my suggestions for use in Linux and Windows. I assume you have a working knowledge of Qt and you have at least built a “hello world” application using it.

Of the few different ways of configuring Qt for OpenCV, we will use the one that involves pkg-config.

Install OpenCV

For this tutorial I am assuming you have installed OpenCV 2 or OpenCV 3. If you have not, you can install them using Homebrew. The basic installation commands are shown below.

Install OpenCV 2.4.x on OSX using Homebrew

brew tap homebrew/science
brew install opencv

Install OpenCV 3 on OSX using Homebrew

brew tap homebrew/science
brew install opencv3

You can find detailed instructions for installing OpenCV using Homebrew by clicking here. Using OpenCV 3 in a Qt application is a bit tricky for OSX because opencv and opencv3 packages contain the same libraries and so opencv3 is not deployed to /usr/local/lib like other packages. I will help you navigate through these complexities.

Install Qt Creator

You can download Qt Creator and follow the onscreen instructions for installing it. There are some restrictions on using Qt in a commercial application and you should make sure you know about licensing issues. You may find this discussion helpful.

Build settings for Qt / OpenCV project

The very first thing to do is instruct Qt where to find pkg-config. The default location for pkg-config is /usr/local/bin.

which pkg-config
# returns /usr/local/bin/pkg-config

We need to add /usr/local/bin to PATH. Go to Project, expand Build Environment and add /usr/local/bin to PATH. Don’t forget to add a colon (:) before appending /usr/local/bin! See the screenshot below.

Qt build settings for OpenCV

If you are using OpenCV 3 you also need to add a new variable called PKG_CONFIG_PATH and set it to the directory that contains opencv.pc for your OpenCV 3 installation. You can find it using the following command

Caution : When both opencv and opencv3 packages are installed

When both opencv and opencv3 packages are installed, and you are using opencv3 in your Qt application, adding /usr/local/lib to your path ( as shown in the previous section ) can lead to linking errors. The best thing to do in such a case is unlink opencv.

brew unlink opencv
I have spent an embarrassing amount of time debugging the above problem in El Capitan. It does not show up on Yosemite. That’s all you need to know to build a Qt application with OpenCV, but let me throw in another goodie.

How to convert OpenCV Mat to QImage

The following piece of code converts OpenCV Mat to QImage. This will come in handy when you build your first Qt based OpenCV application.

Subscribe & Download Code

If you liked this article and would like to download code, please subscribe to our newsletter. You will also receive a free Computer Vision Resource guide. In our newsletter we share OpenCV tutorials and examples written in C++/Python, and Computer Vision and Machine Learning algorithms and news.