WARNING: CAPTCHA protection is an ineffective security mechanism and should be perceived as a "rate limiting" protection only!

Most current used CAPTCHA images can be easily cracked in a fully automated way using many commercial or opensource services. Commercial services are usually very cheap and provide a simple API for most programming languages.
It is not recommended to use CAPTCHA protection for security-critical applications, in this case it is more suitable to use SMS authentication or OTP tokens instead.

Most CAPTCHA images can be cracked in 1-15 seconds, therefore CAPTCHA should be perceived as a rate limiting protection only which stops the attacker for a limited amount of time.

Brief Summary

CAPTCHA ("Completely Automated Public Turing test to tell Computers and Humans Apart") is a type of challenge-response test used by many web applications to ensure that the response is not generated by a computer.

Description of the Issue

Despite the above-described CAPTCHA weakness, it can be still used against:

automated sending of many GET/POST requests in a short time where it is undesirable (e.g., SMS/MMS/email flooding), CAPTCHA provides a rate limiting function

enumeration attacks (login, registration or password reset forms are often vulnerable to enumeration attacks - without CAPTCHA the attacker can gain valid usernames, phone numbers or any other sensitive information in a short time)

automated creation/using of the account that should be used only by humans (e.g., creating webmail accounts, stop spamming)

automated posting to blogs, forums and wikis, whether as a result of commercial promotion, or harassment and vandalism

any automated attacks that massively gain or misuse sensitive information from the application

In security critical applications it is more suitable to use alternative verification channels (SMS authentication, OTP etc).

Using CAPTCHAs as a CSRF protection is not recommended (because there are stronger CSRF countermeasures).

Common CAPTCHA vulnerabilities and attacks

Client-side storage and hidden fields

the value of decoded CAPTCHA is sent by the client (as a GET parameter or as a hidden field of POST form). This value is often:

encrypted by simple algorithm and can be easily decrypted by observing of multiple decoded CAPTCHA values

hashed by a weak hash function (e.g., MD5) that can be easily broken

Chosen CAPTCHA text attack

rarely CAPTCHA is verified on the client side or verified on the server side, but generated on the client side (in javascript)

Arithmetic CAPTCHAs

if arithmetic questions are displayed in cleartext, it is trivial to bypass this CAPTCHA, just parse the HTML content, extract the arithmetic question and solve it

Limited set CAPTCHAs

if generated CAPTCHA questions have a very limited set of possible answers, it is trivial to gain all of them, solve and use them subsequently (CAPTCHA Rainbow Tables)

generated image CAPTCHA is weak (be aware that most current CAPTCHAs can be considered to be weak and easily crackable using existing CAPTCHA cracking services)

generated CAPTCHA questions have a very limited set of possible answers

possibility of replay attacks:

the application does not keep track of what ID of CAPTCHA image is sent to the user. Therefore, the attacker can simply obtain an appropriate CAPTCHA image and its ID, solve it, and send the value of the decoded CAPTCHA with its corresponding ID (the ID of a CAPTCHA could be a hash of the decoded CAPTCHA or any unique identifier)

the application does not destroy the session when the correct phrase is entered - by reusing the session ID of a known CAPTCHA it is possible to bypass CAPTCHA protected page

Solution

Because the CAPTCHA cracking attacks are still improving (and will improve in the future), CAPTCHA should be perceived as a rate-limiting protection only.
If it is implemented, the following considerations should be taken into account:

no CAPTCHA information (except the image itself) should be stored on the client side

the client should have no "control" over the CAPTCHA content

CAPTCHA images should be always randomly generated without possibility to perform image preprocessing, segmentation and classification

CAPTCHA images should not be reused.

Black Box testing and example

identify all parameters which are sent in addition to the decoded CAPTCHA value from the client to the server in order to check if these parameters contain encrypted or hashed values of decoded CAPTCHA and CAPTCHA ID number

try to send an old decoded CAPTCHA value with an old CAPTCHA ID (if the application accepts them, it is vulnerable to replay attacks)

try to send an old decoded CAPTCHA value with an old session ID (if the application accepts them, it is vulnerable to replay attacks)

Find out if similar CAPTCHAs have already been broken.

Verify if the set of possible answers for a CAPTCHA is limited and can be easily determined.