Section 3 - Video tracking and host computer control

With the microcontroller done, we then moved to controlling it via a video-tracking computer. At this point, we had created a simple program for testing out parallel port control of the light's three axes using the keyboard (tilt, pan, and shutter). This program was split into two files, a main, and a set of subroutines that could then be called and compiled into the full video tracking program. It uses libparapin to abstract interaction with the parallel port in userspace.

Second, the parallel port controller. This uses a thread and circular queue to provide asynchronous, non-blocking control of the communications channel. Non-blocking is critical, as the program waits a small period between low-high transition of the interrupt pin (pin 4) for the MSP430 to read the status of the three lines.

Then, we worked on the video tracking program. I will omit the some of the noncritical sections involving the firewire (ieee1394), Xv (video display) and X11 (window manager) calls, as the whole program is long, ~1000 lines. Below is 'main' -- see the comments for a detailed description.

Our tracking algorithm periodically opens and closes the shutter on the light. It is impossible to track a target based on brightness or even pattern detection, since the light is so bright it is impossible to image what it hits and what it does not with our cameras of limited dynamic range. (The human eye, of course, has far better dynamic range.) During the period when the light is off, we wait for the camera shutter speed to stabilize, then average the brightest spot over 10 consecutive frames to obtain a target position. Then, the shutter is opened, and visual feedback is used with a simple PD controller to guide the light to the target. When the device is deployed, we will make the update non-periodic and purely contingent on the detection of motion or of decreased solar cell output. See below for the thread that implements this logic, as well as blits the image onto the screen.

The PD controller uses very pessimistic values for the coefficients, as we discovered that the timing resolution on out older linux computer is low - about 5ms. This means that if too many velocity step commands are sent to the parallel port thread at one time, it will get backlogged, which will induce a phase-shift between control and actuation of velocity. Hence, the light must move rather slowly, on the order of one velocity step on each axis per frame. Again, below.

Our video tracking algorithm first uses a tree-like algorithm to quickly and robustly search for the brightest region in the scene; we presume, somewhat simplistically, that this will be the target. When the device is put into use with an actual monkey cage, we'll surround the camera with high-intensity infrared LEDs to effectively illuminate a retroreflector placed on the monkey's head. Below is the code which performs this computation.

The blocking (or blobbing) and search algorithm yields the estimated location of the brightest pixel in the image. This is passed to a specialized array-growth region growing algorithm which dynamically expands a region around the suggested brightest pixel to include all pixels that are within a threshold of brightness to the brightest. The region growing algorithm then computes the center of mass from the list of pixel coordinates, which are then passed to the PD and target location routines.

And that is, roughly, the entirety of the video tracking program! (Most of the rest of the code deals with the firewire bus and other less interesting details.) We conclude with a picture of the whole setup in the office.