Marker Tracking (FAQ)

Many of these questions deal with options to the Xtra marker and video functions. Most of this functionality has been integrated into the behaviors and is available via options in the property pages. If you search for the relevant commands, you can find (and change) their parameters.

1. What is the difference between ARToolkit & OpenCV pose estimation?
The last parameter to the videoMarkerTrackingInit command is a boolean saying if "opencv" pose estimation should be used. Currently, this should always be true. Originally, we would either use the ARToolkit ARMulti pose estimation (see the ARToolkit distribution for more on this, if you'd like), or use the OpenCV estimator. OpenCV is an open-source computer vision library, and we use a function in it to do a single, global pose estimation (e.g., take all the marker corners and their 3D positions on a panel, and use that to compute the pose once). ARMulti takes a different approach: it takes the pose of each individual fiducial square, and tries to unify them into one global pose. The OpenCV approach should be more robust and stable, especially when there are more than one fiducial seen in a panel and they are spread out on the screen.

2. How does marker history work?
Marker history (which can be toggled on or off with useMarkerTrackingLite) simply chooses between using the ARToolkit's arDetectMarker and arDetectMarkerLite. (Roughly) arDetectMarkerLite tries to find the fiducial ID's from scratch in each frame, while arDetectMarker will assume a marker is the same ID as in the previous frame if it is in approximately the same place. As above, look at the ARToolkit for more information on this.

3. Can sub-corner pixel refinement make a big difference? if so, what are
some guidelines for using it?
Yes, it can make a huge difference. The ARToolkit corner finder is quite poor, and can result in the estimated pixel locations being off by 2 or more pixels depending on lighting and threshold settings. The videoMarkerCornerRefinement command enables (or disables) a second step that looks for "better" corners in the image near where the ARToolkit reported corners. The parameters are the size of the region (radius of a square around the initial pixel, set to 0 to disable subpixel refinement), the maximum number of iterations to search, and the epsilon (if the the pixel estimate moves less than this during one iteration, the search stops). The best values are a function of the space around your markers, and how much possible speed hit you can tolerate. We often use something like (4, 10, 0.1). The subpixel refinement searches for the "best corner" (defined as a intensity gradient that has a cusp on it) in that region, so if the region is white with the ARToolkit corner, it will find it. If it ends up having more than one corner (such as when the markers are far away and the number of pixels between the markers becomes small), it may choose the wrong one, or a point between the two visible corners!)

4. The object actor sometimes jumps upside-down, is there a way to avoid
this?
This is generally caused by poor quality corners. At certain viewpoints, the poor corner estimation causes the most "likely" pose of the camera (given the global error) to be somewhere else. Adjusting the threshold value so it matches available lighting helps, as the corner estimates end up being better. Adding corner refinement will also make this less likely to happen. Finally, adjusting the focus and the parameters of your camera (using the software that came with it) so that you have a smaller shutter/esposure time (and thus less bluring in the images) will make the corner estimation better. Combining all three will make the marker tracker work MUCH better, giving better poses and less dropouts due to blurry images.

5. Can I see visually how well the various parameters are set and working?
Turn on debugging using videoDebugMarkerTracking, which will draw small colored dots on the image showing the corner locations and (if subpixel refinement is on) the refined corner locations. If you then call videoReplaceWithThresholded AFTER capturing a frame and AFTER calling videoTrackMakers, the thresholded image will be displayed instead of the original captured image. This functionality is in LiveVideo.