Reverse reading program shows text in the eye of a computer

Optical character recognition (OCR), or the process of turning images of text into machine-readable text you can copy, paste, and search, is a big deal. OCR software is complex and difficult to streamline, requiring some of the most modern theories about coding and neural networks to get them working quickly and efficiently. It might sound easy, tracing the outline of a letter and recognizing it, but once you step outside of a single possible input font (let alone outside typed text), the difficultly skyrockets.

So now, as we can finally start to expect OCR tech to be quick, accurate, and versatile as a matter of course, it’s time to start working on alternate versions that turn the original goal on its head. Software developer Darius Kazemi has created a program that can do reverse OCR — and the results are surprisingly insightful.

What is reverse OCR? Well, OCR turns pictures of (often hand-written) words into machine-searchable text, so reverse OCR turns machine text into seemingly hand-written words. So if it set out to reverse OCR the word “elephant” it would begin by drawing an approximated “hand drawn” with a semi-random drawing algorithm. This obeys a very few laws about how long a letter segment (line) may be and how the next letter segment may be drawn from there, and when to move on to the next letter. A conventional OCR algorithm goes over this output and says, “Nope, no idea what that is,” until it finally recognizes the ever-more-complicated scribble as a word. Then a second check: does the word read match the target word, in this case elephant? If no, then delete the scribble and start again.

A computer can transcribe this coptic script just as easily as the Latin characters in English.

You’ll notice that this is not an evolutionary algorithm; there is no acceptance of failure as a path to later success, and based on the small amount of info Kazemi has released about its code, it doesn’t seem that the results of one run in any way influence the drawing algorithm. This means that the output is extremely rough and does not converge on some human- or machine-selected ideal; the ten thousandth accepted word will look just as different from legible English as the very first. That allows us a very interesting sort of insight: the extent of a computer’s tolerance for nonsense. Many of the scribbles recognized as words are virtually incomprehensible to a human eye, but you can (almost) always see where what the computer was keying on.

The vatican’s got some big plans for OCR.

When you type out a word for a CAPTCHA test, you are essentially doing the opposite comparison, putting your human OCR algorithm against a computer’s. Right now, it’s much harder for a computer to make sense of a warped version of a previously recognizable letter, because today’s computers don’t have the robust pattern-recognition hardware we do. We have built in circuitry for recognizing certain abstract concepts like faces, and the so-called pareidolia effect is so strong we can see faces in things like clouds, moon rocks, and tea leaves. Combined with the robust higher brain functions of an evolved simian, we can recognize a pretty amazing set of pattern-identifying features. A dog of a breed you’ve never seen before is still recognizably a dog, in the platonic sense.

OCR is being used by the Vatican to bring ancient texts to the public, and demonstrations like this show how an OCR algorithm might be able to make sense of words scrawled thousands of years ago in long-dead languages. When looking at the outline of a letter with calligraphy, or just an unsteady hand, the algorithm has to have a high tolerance for deviation — but not too high. This project could actually be useful in providing some intuitive visual feedback about just how divergent a particular OCR algorithm will allow a letter to be.

OCR will become a bigger part of our lives over time, as both the algorithms and the hardware running them become better. The cloud note-taking service Evernote already uses OCR heavily to help you quickly turn static images or physical documents into searchable text files. Look for wide-ranging image formats that embed OCR content automatically — search the text of scanned document attachments, or find that photo you took last year in front of the Hollywood sign. If rolled out in a big and user-friendly way, OCR has the potential to dramatically change our relationship with text, and the things we can do with it.