Fun with glyph recognition

As mentioned in a recent post, the molecular glyph recognition algorithm used by the Living Molecules app is going to have a few more iterations before it’s in peak condition. The current public version (1.0.1) does not behave too well in low lighting conditions, which is not so bad for using with an iPhone, because it comes with its own flashlight, but that’s not the case for iPads. The last couple of days has been spent pulling everything apart and putting it all back together again, because it turns out that distinguishing between white and black is slightly more complicated than one might think.

After a few boundary cases of pointing the camera at a glyph and being completely certain that it’s lined up flawlessly, and getting nothing, I decided that it was time to implement a permanently accessible “debug” feature. If you aim the target hex at your glyph, and nothing is happening, you can tap on the target once, and the app will take a freeze frame snapshot and display something similar to what you see in the screenshot on the right.

This is the analysis panel: instead of waving the phone around trying to get a lock, the picture stays right where it is, and the user interface allows the orientation to be rotated, expanded/shrunk and panned, so you can line everything up using the touchscreen interface. On the bottom are a couple of feedback dials: on the left, what the glyph looks like when it’s been prefiltered and oriented into the appropriate frame of reference; and on the right, the determined intensity of each of the “bits” within the main payload area. A red dot indicates whether or not it passed the intensity theshold. As soon as the lineup is good enough for a successful lock, the analysis panel disappears, and the content is displayed, just like it would be normally.

Sometimes the analysis panel makes it obvious why no lock could be achieve, e.g. when the photo is just to blurry.

The glyph recognition is already a lot better in dim light, so this additional panel is the kind of feature that should ideally obsolete itself: an always-available debugging tool means I can track down a lot more boundary cases and fix them.