Explaining The Fundamental Accessibility Challenge of CAPTCHA

by Sina on December 6, 2014

There has been a lot of discussion of Google’s new approach to CAPTCHA called “No CAPTCHA reCAPTCHA.” Over the next nine minutes, I’d like to share my thoughts on why Google’s new approach is not a good idea for users with disabilities. I’ve included a transcript of the audio below as well. It’s a pure raw transcript of my spoken words generated by incredibly talented and amazing friend, Mirabai Knight. The text below is just a transcript, not a formatted blog post.

The Audio

The Transcript

Hi, everybody. We’ve been talking on Twitter about this Google CAPTCHA issue, so I feel it’s important to shed some light on the accessibility of CAPTCHA and how this Google CAPTCHA may or may not work, et cetera, et cetera. First we should define a few terms just really quick. For those who don’t know, a CAPTCHA is a way of doing an automated Turing test. It’s very simple. You pick something that computers find exceptionally difficult or straight up cannot do right now. And then it has to also be something that humans can do. In this way, you’ve got a test that can differentiate between humans and computers. You ask the humans to do this thing. Currently it’s identifying hard-to-read text. Which is very hard for a computer to do very well all the time. Even though there’s been massive strides in that domain in computer vision literature. But let’s not go there for now.

And then you ask the human to type in the stuff into an edit field, and in this way, you know it’s not a bot from some spam organization trying to register an account. So that’s how CAPTCHAs currently work. They’re made accessible oftentimes by doing an audio equivalent or by doing a textual equivalent. The textual equivalent is type in 5 + 7, or what is the answer of 5 + 7, what is the answer to 9 – 2. It’s usually single digit mathematics. And the person types it in. The idea here being that text processing is very hard. The reality being that a four line Perl script most likely — or most of the time — can solve about 90% of those. Happy to elaborate on that, if anybody wants to know.

The audio equivalent to these things is usually done via mixing in audio, like… Background noise. That kind of stuff. People talking in the background. And then a synthesized or human voice saying the digits. 6, 7, 4. And then you type those in. And there’s a variety of other means, but we don’t need to do a history of CAPTCHA. That’s just what’s going on right now. There was a Wired article a couple days ago or whenever then they talk about Google’s new thing. All Google’s done is realize that all of these systems are pretty much flawed, and also that ordinary users find that typing in CAPTCHA — even sighted ones — very difficult. Because it is fuzzy. They have to make it so difficult for computers not to be able to do it now that it’s making it exceptionally difficult for humans to do it. Which, by the way, is an insane approach from a usability point of view. Because you’re saying — I’m going to make my users’ lives more difficult so as to make my life easier as a business. And that’s the inverse of what you should be doing. That was a great point made, I believe, in that same Wired article. So what Google has done is the following. They’ve come up with a heuristic that they’ve said — okay, we know about things like Fitts’s law from HCI and other very simplistic approach vectors, of a mouse reaching a check box, along with a couple other signals like IP address and so forth, that we associate with spamming, and so we’re going to take all of these things, just like they do with everything, as signals into a very simple classifier, and we’re going to allow a box to be checked that says I’m not a bot. If the way that this box is checked looks fishy — in other words, it looks like a computer is doing it, or like somebody has scripted it to be done, then we pop up the old style CAPTCHA. We fall back. Okay?

So everybody thinks — hey, this is awesome. Right? If you’re blind, you just check the box, and you’re good to go. Right? Because then it’ll recognize you as a human, and you then don’t have to deal with the audio CAPTCHA and all of the difficulties that we associate with that. The fallacy here is very straightforward. Which is that this doesn’t work. If you’re going to make a CAPTCHA that is not going to be scriptable against, in this way, then by definition a screen reader will not be able to perform the same actions. And the reason for that is that screen readers work by programmatically accessing different elements on the page and toggling thing. The same way a script would work. So by blocking all of the possibilities that a script can do, you have to block the way a screen reader would do it.

This is the inherent problem at the kernel of most CAPTCHA accessibility issues, and why visual-based CAPTCHA just as a whole is an extremely bad idea. So the approach, just to recap, is not going to always fail. It will sometimes be appropriate, because they’ll use things like IP address or whether you’re logged into Google or whatever the idea is. And they will then be able to let you go through. So some people are saying — well, it works with NVDA, or it works with VoiceOver. Or it doesn’t work over here, or sometimes it works on my computer and sometimes it doesn’t. And that’s entirely the point. The point is that this is a flawed approach. It doesn’t matter if it worked for you. I don’t care.

Logically speaking, mathematically speaking, this approach will not also be accessible and prevent what CAPTCHA is designed to prevent. Those are antithetical goals to one another. At least in the current implementations of CAPTCHA. Those are logically opposite goals, unfortunately. So as a disabled user, you lack the sensory output, or, rather, input and the ability to output the correct information, just like a computer does. And so if you happen to have the same disability as a computer — in this case, a proper functioning vision system — then designing a CAPTCHA that excludes said computer by definition excludes you.

I do not care if occasionally on a Sunday afternoon, while Jupiter is in alignment, NVDA and Firefox worked for you. And really honestly, by the way, the reason that is probably because you logged into Gmail recently or your IP address was whitelisted for a while because of certain neighborhoods having less spam, et cetera, et cetera, et cetera. There’s all these other signals that come into play.

So then you might be asking: so what’s the big deal? The big deal is that this is yet another step. If the failure condition is so high so as to most of the time fail or even 50% of the time fail, all you’ve done is put one more step, one more failure condition, in front of users with disabilities, as well as other users. Mobile comes into mind, where you don’t use a mouse. And other things as well. Other access methodologies as well. You’ve put just one more failure possibility in front of said users to access your site. Whether it’s to sign up your for blog or get email notifications or whatever you’re using a CAPTCHA on. Whatever form — to do a financial transaction. Whatever it is. That’s part of the problem.

The second part of the problem — hopefully you’re now on board with the idea that this kind of CAPTCHA, if it works all the time, by definition, is inaccessible. This kind of CAPTCHA. I’m not saying that all CAPTCHA necessarily would be, but that would be an entire field of research. The second problem in this space is the idea of CAPTCHA in general. There’s only four possibilities. Either it doesn’t work and is not accessible, it doesn’t work and it is accessible, it does work and it is not accessible, or it does work and it is accessible. Those are the four permutations of working and accessibility. Right? Where I define accessible as what I think we all associate with accessibility, for persons with different functional abilities, as well as different varieties of access, as well as where I define working as it prevents spammers.

So putting the logical conclusions of the previous few minutes aside, if it doesn’t work and it’s not accessible, that’s useless. We don’t have to talk about that case. And if it doesn’t work… Excuse me, and if it does work and it’s not accessible, that’s inexcusable. So we don’t really have to talk about that particular case. If it doesn’t work and it is accessible, we really don’t have to talk about that case either, because, frankly, if it doesn’t work, it doesn’t matter if it’s accessible or not. It’s not protecting your blog or whatever. So the only case is really — if it does work and it is accessible.

And hopefully I’ve shown before that if it does work, by definition, it cannot be accessible, as currently implemented. So that is my reason for thinking and for firmly believing that based off of just pure science and, you know, logic, that this method of doing CAPTCHA is not an appropriate methodology for either protecting against spam or increasing accessibility. Those are my thoughts. If you want to follow me on Audioboom, I’m Sina Bahram, and it’s the same handle on Twitter. @sinabahram. Thanks for listening, folks.

Your Thoughts?

I’d love to hear what you think about this important issue in the comments below.