Trail: Vision

Overview

It provides motion detection, color detection, light detection and interfacing
with robots that use the Lego Mindstorms RCX brick, to allow Mindstorms robots
to respond to what they see.

It supports most of the functions of the Lego Vision Command software, but in
an open extensible way that will allow much more sophisticated functionality
to be added.

Although it is designed for use for the Lego Cam that comes with the Lego
Mindstorms Vision Command product, it will work with most PC Web cameras.
It has been with the X10 XCam2 wireless camera. This camera can be mounted
on a Lego Robot to avoid the trailing USB wire, which is a problem with the
Lego Cam.

Acknowledgements

Some of the ideas come from the Robotics SDK produced by the UK Sun Technology
Evangelist, Simon Ritter - see

Remote Command Execution

Sending remote commands to the PC is done by the josx.rcxcomm remote method
execution classes. Client stubs and a server implementation of functions
needed for camera and robot movement are provided as part of the vision API.

Example

An example program that detects motion, sounds an alarm and takes a snapshot of
an Intruder:

Linux

This is for the Quickcam which is similar to the Lego Camera. Note that you need to get the source
from CVS and checkout the qc-usb module.

The JMF from Sun does not pick up the camera correctly and so you need to use the JMF from
Blackdown.org. However, at the time of this writing, there was an issue with licensing and Blackdown
had stopped the downloads but you can still get it from the mirror below:
You can get this from:

You may have problems with the XLibRenderer plugin. If so, you can use the AWTRenderer
instead.

Building on Linux and Cygwin

Make sure that JMFHOME is defined and pointing at your JMF installation directory.

Configuration

Ensure that JMF is on the CLASSPATH by following the JMF installation
instructions.
On Linux, make sure that the $JMFHOME/lib directory is added to LD_LIBRARY_PATH using
the command:

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$JMFHOME/lib

Run the JMStudio application to determine what video and sound capture
devices are available on your system.
Look at properties to determine the video and sound capture devices.

If the camera was not on when JMF was installed try doing "Detect capture devices".
If this does not work, try calling jmfinit from the JMF bin directory.

Edit the video.properties files in \examples\vision\motion.
There is currently a version in each example directory and you need to edit
each one to run the example. You need to set video-device-name and
sound-device-name to the exact text in the JMF registry. If Direct Sound
capture is available on your windows machine, use that, as Java Sound capture
does not seem to work on some PCs.

For some of the examples, you need to copy the Effects directory from the
Lego Vision Command VCSData and Piano and any other instrument
directory you want from the VCSData\Instruments directories.
For a default installation of the Lego Vision Command software
the full path for VCSData is
c:\Program Files\LEGO MINDSTORMS\Vision Command\data\hdData\VCSData.
They should be copied to subdirectories of \vision, e.g. c:\vision\Efects
and c:\vision\Piano.

Running the Examples

Motion example

Change directory to \examples\vision\motion and do:

javac Motion.java
java Motion

An alarm is sounded and a file Intruder[n].jpg is produced whenever motion
is detected.

Alarm example

Change directory to \examples\vision\alarm and do:

javac Alarm.java
java Alarm

This is a security alarm that can be turned off by the correct color being
detected in region 3. You will probably need to edit the source to change
the color to one that is detected by your camera. The current average RGB
value is what is being detected in region 3 is displayed on System.out.

Music example

Change directory to \examples\vision\music and do:

javac Music.java
java Music Piano

You can use other instruments that are available with the Lego Vision Command
software, e.g. "Guitar2"

Light Seeking example

This example uses the RCX remote control class to respond to what the camera
sees.

It is a very simple light seeking example. When it sees light in the central
region it moves forward. When it sees light in the left or right regions, it
turns towards the light.

It needs the Rover example program running on the RCX.

Change directory to \examples\vision\light\rcx and do:

lejosc Rover.java
lejos Rover

and press the Run button on the RCX.

You then need to compile and run the PC program.
To do this, change directory to \examples\vision\light\pc and do:

javac Light.java
java Light

Shine a bright light at the camera to move the robot about.

Remote control example

This example uses the RCX remote control class to control the robot and let
the robot respond to what it sees.

It needs the Rover example program running on the RCX.

Change directory to \examples\vision\light\rcx and do:

lejosc Rover.java
lejos Rover

and press the Run button on the RCX.

You then need to compile and run the PC program.
To do this, change directory to \examples\vision\remote\pc and do:

javac *.java
java RemoteControl

You should see a display with the image viewer in the center, remote controls
for the robot on the left, and camera and color controls of the right.

Move the robot about with the remote controls. Flip the camera up and down,
as necessary, to see what you need to see. When the robot gets near a red
object, it will move towards it and attempt to push it forwards. Click on
the little Media Properties control at the bottom right of the ImageViewer
and choose the Plug-In Settings, to control color sensitivity and the
proportion of the region that needs to be filled with the specified color.
Edit the RemoteControl.java program and change the color if you need to.