Voice Driven Camera Using Raspberry Pi

Develop a camera that can run on voice over commands, this is mainly designed for the people of all kinds, especially to the people who seek for the fun time photography.

Teacher Notes

Teachers! Did you use this instructable in your classroom? Add a Teacher Note to share how you incorporated it into your lesson.

Step 1: Hardware Guide

VDC is designed on the Raspberry Pi (Model B) and requires some additional hardware like a wifi-adapter(optional) and USB microphone. The suggested hardware is indicated below with links for further details. You may try slightly different brands/specifications of hardware. VDC is not affiliated with any of the linked hardware vendors.

The Raspberry Pi Verified Peripherals List may be helpful for finding substitutes for the products recommended above.

Assembly

Assembly of the required components is straightforward. Insert the microphone, SD card, wireless adapter (if you have one), micro-USB cable, ethernet cable, and speakers into the Raspberry Pi. The USB wall charging adapter is recommended to power, as a standalone device.

The ethernet cable will be used to log in to the pi from a computer during the software installation step. After installation, if you prefer to use a wireless connection, this cable can be removed.

Internet Connection

As mentioned above, the wireless adapter is optional. It runs just fine on a wired connection (via ethernet), so you can choose between the two setups depending on what works best for you.

Step 2: OS Installation on Raspberry Pi

Introduction

Raspberry Pi is a credit card sized micro processor available in different models with different processing speed starting from 700 MHz. Whether you have a model B or model B+, or the very old version, the installation process remains the same. People who have checked out the official Raspberry Pi website, might have seen them recommending the "NOOBS" or "NOOBS LITE" Operating System (aka "OS") for beginners. But using the Pi is very easy and from being a beginner, one will turn pro in no time. So, it's better to go with the more powerful and more efficient OS, the Raspbian. The main reason why Raspbian is extremely popular is that it has thousands of pre built libraries to perform many tasks and optimize the OS. This forms a huge advantage while building applications.

Downloading Raspbian and Image writer

Download the latest version of Raspbian from here. You can download it directly or via the torrents.

You will be needing an image writer to write the downloaded OS into the SD card (micro SD card in case of Raspberry Pi B+ model). So download the "win32 disk imager" from here.

Writing the image

Insert the SD card into the laptop/pc and run the image writer. Once open, browse and select the downloaded Raspbian image file. Select the correct device, that is the drive representing the SD card. If the drive (or device) selected is different from the SD card then the other selected drive will become corrupted. SO be careful.

After that, click on the "Write" button in the bottom. As an example, see the image below, where the SD card (or micro SD) drive is represented by the letter "G:\"

Once the write is complete, eject the SD card and insert it into the Raspberry Pi and turn it on. It should start booting up.

Setting up the Pi

Please remember that after booting the Pi, there might be situations when the user credentials like the "username" and password will be asked. Raspberry Pi comes with a default user name and password and so always use it whenever it is being asked. The credentials are:

login: pi
password: raspberry

When the Pi has been booted for the first time, a configuration screen called the "Setup Options" should appear and it will look like the image below.

If you have missed the "Setup Options" screen, its not a problem, you can always get it by typing the following command in the terminal.

sudo raspi-config

Once you execute this command the "Setup Options" screen will come up as shown in the image above.

Now that the Setup Options window is up, we will have to set a few things. After completing each of the steps below, if it asks to reboot the Pi, please do so. After the reboot, if you don't get the "Setup Options" screen, then follow the command given above to get the screen/window.

The first thing to do:

select the first option in the list of the setup options window, that is select the

Expand Filesystem

option and hit the enter key. We do this to make use of all the space present on the SD card as a full partition. All this does is, expand the OS to fit the whole space on the SD card which can then be used as the storage memory for the Pi. The second thing to do:

select the third option in the list of the setup options window, that is select the "Enable Boot To Desktop/Scratch" option and hit the enter key. It will take you to another window called the "choose boot option" window that looks like the image below.

In the "choose boot option window" , select the second option, that is, "Desktop Log in as user 'pi' at the graphical desktop" and hit the enter button. Once done you will be taken back to the "Setup Options" page, if not select the "OK" button at the bottom of this window and you will be taken back to the previous window. We do this because we wan to boot into the desktop environment which we are familiar with. If we don't do this step then the Raspberry Pi boots into a terminal each time with no GUI options. Once, both the steps are done, select the "finish" button at the bottom of the page and it should reboot automatically. If it doesn't, then use the following command in the terminal to reboot.

sudo reboot

Updating the firmware

After the reboot from the previous step, if everything went right, then you will end up on the desktop which looks like the image below.

Once you are on the desktop, open a terminal and enter the following command to update the firmware of the Pi.

sudo rpi-update

Updating the firmware is necessary because certain models of the Pi might not have all the required dependencies to run smoothly or it may have some bug. The latest firmware might have the fix to those bugs, thus its very important to update it in the beginning itself.

Step 3: Setup VNC on Raspberry Pi to Remotely Control

VNC (Virtual Network Computing)

Sometimes it is not convenient to work directly on the Raspberry Pi. Maybe you would like to work on it from another device by remote control.

VNC is a graphical desktop sharing system that allows you to remotely control the desktop interface of one computer (running VNC Server) from another computer or mobile device (running VNC Viewer). VNC Viewer transmits the keyboard and either mouse or touch events to VNC Server, and receives updates to the screen in return.

You will see the desktop of the Raspberry Pi inside a window on your computer or mobile device. You'll be able to control it as though you were working on the Raspberry Pi itself.

VNC Connect from RealVNC is included with Raspbian. It consists of both VNC Server, which allows you to control your Raspberry Pi remotely, and VNC Viewer, which allows you to control desktop computers remotely from your Raspberry Pi should you want to.

You must enable VNC Server before you can use it: instructions for this are given below. By default, VNC Server gives you remote access to the graphical desktop that is running on your Raspberry Pi, as though you were sitting in front of it.

Enabling VNC Server

On your Raspberry Pi, run the following commands to make sure you have the latest version of VNC Connect:

sudo apt-get update

sudo apt-get install realvnc-vnc-server realvnc-vnc-viewer

Now enable VNC Server. You can do this graphically or at the command line.

There are two ways to connect to your Raspberry Pi. You can use either or both, depending on what works best for you.

Establishing a direct connection

Direct connections are quick and simple providing you're joined to the same private local network as your Raspberry Pi. For example, this might be a wired or wireless network at home, at school, or in the office).

On your Raspberry Pi (using a terminal window or via SSH) use these instructions or run ifconfig to discover your private IP address.

ifconfig

On the device you'll use to take control, download VNC Viewer. For best results, use the compatible app from RealVNC.

Enter your Raspberry Pi's private IP address into VNC Viewer:

Establishing a cloud connection

You are entitled to use RealVNC's cloud service for free, provided that remote access is for educational or non-commercial purposes only.

Cloud connections are convenient and encrypted end-to-end. They are highly recommended for connecting to your Raspberry Pi over the internet. There's no firewall or router reconfiguration, and you don't need to know the IP address of your Raspberry Pi, or provide a static one.

Sign up for a RealVNC account here: it's free and it only takes a few seconds.

On your Raspberry Pi, sign in to VNC Server using your new RealVNC account credentials:

On the device you'll use to take control, download VNC Viewer. You must use the compatible app from RealVNC.

Sign in to VNC Viewer using the same RealVNC account credentials, and then either tap or click to connect to your Raspberry Pi:

Authenticating to VNC Server

To complete either a direct or cloud connection, you must authenticate to VNC Server.

If you're connecting from the compatible VNC Viewer app from RealVNC, enter the user name and password you normally use to log in to your user account on the Raspberry Pi. By default, these credentials are pi and raspberry.

If you're connecting from a non-RealVNC Viewer app, you'll first need to downgrade VNC Server's authentication scheme, specify a password unique to VNC Server, and then enter that instead. To do this, open the VNC Server dialog on your Raspberry Pi,

If your Raspberry Pi is headless (i.e. not plugged into a monitor) or controlling a robot, it is unlikely to be running a graphical desktop.

VNC Server can create a virtual desktop for you, giving you graphical remote access on demand. This virtual desktop exists only in your Raspberry Pi's memory:

To create and connect to a virtual desktop:

On your Raspberry Pi (using Terminal or via SSH), run vnc server. Make note of the IP address/display number that VNC Server will print to your Terminal (e.g. 192.167.**.**).

On the device you'll use to take control, enter this information into VNC Viewer. To destroy a virtual desktop, run the following command:

vncserver -kill :<display-number>

This will also stop any existing connections to this virtual desktop.

Step 4: Camera Configuration

Setting up the camera hardware

Warning: Cameras are sensitive to static. Earth yourself prior to handling the PCB. A sink tap or similar should suffice if you don’t have an earthing strap.

The camera board attaches to the Raspberry Pi via a 15-way ribbon cable. There are only two connections to make: the ribbon cable needs to be attached to the camera PCB, and to the Raspberry Pi itself. You need to get the cable the right way round, or the camera will not work. On the camera PCB, the blue backing on the cable should face away from the PCB, and on the Raspberry Pi it should face towards the Ethernet connection (or where the Ethernet connector would be if you're using a model A).

Although the connectors on the PCB and the Pi are different, they work in a similar way. On the Raspberry Pi itself, pull up the tabs on each end of the connector. It should slide up easily, and be able to pivot around slightly. Fully insert the ribbon cable into the slot, ensuring it is set straight, then gently press down the tabs to clip it into place. The camera PCB connector also requires you to pull the tabs away from the board, gently insert the cable, then push the tabs back. The PCB connector can be a little more awkward than the one on the Pi itself.

Setting up the camera software

Execute the following instructions on the command line to download and install the latest kernel, GPU firmware, and applications. You'll need an internet connection for this to work correctly.

sudo apt-get update

sudo apt-get upgrade

Now you need to enable camera support using the

raspi-config

program you will have used when you first set up your Raspberry Pi.

sudo raspi-config

Use the cursor keys to move to the camera option, and select 'enable'. On exiting raspi-config, it will ask to reboot. The enable option will ensure that on reboot the correct GPU firmware will be running with the camera driver and tuning, and the GPU memory split is sufficient to allow the camera to acquire enough memory to run correctly.

If it's not enabled, enable it and reboot your Pi to begin.

To test that the system is installed and working, try the following command:

raspistill -v -o test.jpg

The display should show a five-second preview from the camera and then take a picture, saved to the file test.jpg, whilst displaying various informational messages.

RASPIVID

Raspivid is the command line tool for capturing video with the camera module.

With the camera module connected and enabled, record a video using the following command:

raspivid -o vid.h264

Remember to use

-hf

and

-vf

to flip the image if required, like with

raspistill

This will save a 5 second video file to the path given here as vid.h264 (default length of time).

Specify length of video

To specify the length of the video taken, pass in the -t flag with a number of milliseconds. For example:

raspivid -o video.h264 -t 10000

This will record 10 seconds of video.

MP4 Video Format

The Pi captures video as a raw H264 video stream. Many media players will refuse to play it, or play it at an incorrect speed, unless it is "wrapped" in a suitable container format like MP4. The easiest way to obtain an MP4 file from the

raspivid

command is using MP4Box.

Install MP4Box with this command:

sudo apt-get install -y gpac

Capture your raw video with raspivid and wrap it in an MP4 container like this: