A well-established psychological test is coming under the gun as a new research effort reviews the test’s reliability and validity.

The Rorschach Inkblot Test involves a viewer looking at ten inkblots, one at a time, and describing what they see.

The rationale behind this test is that certain aspects of the subject’s personality will be exposed as they are interpreting the images, allowing for the possible diagnosis of various psychological disorders.

The new review, published in Psychological Science in the Public Interest, a journal of the Association for Psychological Science, determined that despite its popularity, the Rorschach may not be the best diagnostic tool and that practitioners need to be cautious in how they use this technique and interpret their results.

The Rorschach Inkblot Test was developed in the 1920s, but was already mired in controversy within 30 years. Critics argued that it was not always administered in a standardized way and evidence for its reliability was lacking.

However, the Rorschach was revived in the 1970s with the publication of John Exner’s Comprehensive System (CS), which detailed standards and norms for analyzing results. The CS was credited with providing a concrete, scientific basis for the Rorschach test and it became widely used in clinical and forensic settings.

Proponents of the CS claimed that it also provided a wealth of information for non-patient adults and children.

However, critics of this system argue that the norms established by CS are outdated and based on small sample sizes. Furthermore, the CS norms are not representative of the population and actually classify a portion of normal subjects as having pathological tendencies.

Many studies have also called into question the scoring reliability of the CS; that is, a number of experiments have shown that two practitioners will score one subject very differently using the CS method.

The authors observe that “disagreements can have particularly serious implications if the test results are used to reach important clinical or legal recommendations.”

In addition, some studies suggest that there may be a cultural bias associated with the CS. Research has shown that Blacks, Hispanics, and Native Americans score differently on a number of variables in the CS compared to Caucasians.

The authors note that “similar discrepancies have been reported for CS scores in Central and South American countries as well as in several European countries.” These findings suggest that any CS data acquired from various racial and cultural groups should be interpreted with extreme caution.

The authors acknowledge that not all the news concerning the Rorschach Inkblot Test is bad: There is evidence that this tool may be useful in identifying patients with schizophrenia, bipolar disorder, and borderline personality disorder.

They note, however, that the Rorschach has not been shown to be related to Major Depressive Disorder, Antisocial Personality Disorders, or Post-traumatic Stress Disorder.

Overall, the authors suggest that due to the inconsistent literature on the Rorschach Inkblot Test and other related psychological tools, practitioners should be very selective when they use these assessments and use them in ways which have strong empirical support.

“Whenever possible,” the authors conclude, “forensic and clinical evaluations should be based on more dependable assessment techniques, such as structured psychiatric interviews and well-validated self-report indexes.”