CS 363: Assignment #5

Interactive Robot

The purpose of this project is to add more perception to your
robot. You will use both lasers and vision to identify attributes of
the environment and respond to them appropriately.

The vision module, SVM (Social Vision Module), is a general purpose
vision system designed for interactive applications. It runs as a
separate program and broadcasts messages via a system called
IPC (Inter-Process Communication), developed at Carnegie Mellon
University. Programs can connect to an IPC central server and
subscribe to messages sent by other programs or send their own
messages. A program can tell SVM to turn on various algorithms, and
SVM will broadcast information about what it detects.

Setup

On both robots there is now an executable of the SVM module in
/usr/local/bin. To run SVM, you need to start two programs. Then you
can use a third program to control SVM and also view what it detects
while debugging.

To run SVM and view what it is detecting, create three terminals. In
one terminal, start the central server, if one is not already
running.

central

There is, in fact, very little point in quitting the central server,
so leaving one running at all times is fine. You can check if a
central server is running by looking at all active processes and
searching for central.

ps aux | grep central

In the second terminal, first copy the files colorCalibration,
camera0.conf and camera1.conf from /usr/local/etc to the directory
where you will run svm. Then use the following command.

svm-v4l2

That will run svm using the built-in camera on the laptop. If you
want to use a different camera (good idea) then you'll need to give it
the command line argument of the video device to use, such as
/dev/video1 and you'll need to modify the camera1.conf file so it is
active and camera0 is not.

In the third terminal, type the following to run dispipc, which
creates a window in which you can see what SVM is doing.

dispipc localhost

The program dispipc lets you see what the robot sees and control what
operators are active in SVM. An operator is a vision algorithm that
detects or recognizes something and is able to broadcast that via IPC.
You can active several operators at once. By typing keys or clicking
in the video window, you can do a number of things.

Key/Mouse

Action

click

Save current video frame to a file

1

Toggle greyscale video

2

Toggle color video

3

Set video size to small

4

Set video size to medium

5

Set video size to large

6

Set video quality to low

7

Set video quality to medum

8

Set video quality to high

r

toggle face detection

m

toggle motion detection

p

toggle pink blob detection

i

toggle italian flag detection

L

toggle soccer marker detection

q

quit dispipc, but leave SVM running

x

quit dispipc and terminate SVM

Note that disipc does not have to run on the same computer as SVM. I
hope to have local disipic versions on the dwarves, so that you can
run it locally and connect to either hasuful or arod by giving telling
it the hostname, as below.

dispipc hasufel.cs

Tasks

Download the two
files makefile.listener
and listener.c. Take a look at listener.c,
which is an example of how to turn on an SVM operator, listen for
responses, and turn off an operator. You can use the following command
to compile it.

make -f makefile.listener listener

If SVM is running, you can use the listener program to turn on an
operator and subscribe to messages. To run listener and turn on face
detection, make sure SVM is running and then use the following
command.

./listener -o 24

That will start the face detector and print out any detections.
Typing 'q' will terminate listener. The pink badge operator is
operator 0. You can figure out what the other operator ids are by
looking at the file SVM_VisionModule.h which is located in
/usr/local/include.

Write a stand-alone program that connects to central, tells the vision
module to turn on either the pink badge or face detector operator, and
then prints out any detections. Your program will look similar to the
listener operator, but you don't need to do any of the special stuff
with the terminal settings. You will need to link the ipc and gcm
libraries with your code during compilation, as in the
makefile.listener file.

In your navigation main program, have your program try to connect to
IPC when it starts up. Then put a call to IPC_listenClear() inside
your main loop. note that if you set the argument to IPC_listenClear
to 100, then you do not need a usleep in your main loop any more, as
the function will listen for 100ms before returning. Then terminate
the IPC connection when your program quits.

Add a new top-level state to your system PINK_TRACK. When entering
that state, the robot should turn on the pink badge detector. It
should then try to approach anything pink it sees in the environment
and stop a certain distance away from it, using sonar/lasers to
determine the distance. If it sees nothing, it should wander. The
robot should not run into anything. Note that you can substitute any
of the other vision operators for the pink badge detector, if you
wish.

Note that the operators in SVM should be returning the bounding box
and location of the detection. They should also be returning pan and
tilt angles, relative to the forward direction of the robot. These
may not be quite accurate, but they do tell you if you are left or
right of the target.

Extensions

Do something more intentional than a simple wander while searching.

Set it up to enter a follow mode once it has detected a pink badge.

Make the interaction more interesting.

Use another capability of the vision module to interact with the
environment.

Make connecting to IPC optional, so that all of your old code still
works the same.

Writeup

The report for each project should be a brief summary of what you did
along with some code examples, terminal output, or screen shots,
depending upon the assignment. Please organize the writeup as follows.

Title of the project and your name(s)

An abstract describing what you did in 200 words or less.

A brief description of code you wrote and experiments you ran.

Results.

A brief description of what you learned.

Handin

Make your writeup for the project a wiki page in your personal
space.

Once you have written up your assignment, give the page the label:

cs363s17project5

Do not put code on your writeup page or anywhere it can be publicly
accessed. To hand in code, put it into a project4 directory in the
Private folder of your handin folder on Courses.