A follow-up to the book "Unnatural Acts: Critical Thinking, Skepticism and Science Exposed!" by Robert Todd Carroll, creator of The Skeptic's Dictionary. The blog will offer irregular postings about cognitive biases, logical fallacies, and illusions.

Site Map

Monday, January 7, 2013

anecdotal evidence (testimonials)

Testimonials and anecdotes are used to support claims in many fields. Advertisers often rely on testimonials to persuade consumers of the effectiveness or value of their products or services. Others use anecdotes to drive home the horror of some alleged activity or the danger of widely-used electronic devices like cell phones. In the mid-90s, there were many people, some in law enforcement, claiming that Satanists were abducting and abusing children on a massive scale. The anecdotes involved vivid descriptions of horrible sexual abuse, even murder of innocent children. The anecdotes were quite convincing, especially when they were repeated on nationally televised programs with popular hosts like Geraldo Rivera. A four-year study in the early 1990s found the allegations of satanic ritual abuse to be without merit. Researchers investigated more than 12,000 accusations and surveyed more
than 11,000 psychiatric, social service, and law enforcement personnel.
The researchers could find no unequivocal evidence for a single case of satanic cult ritual abuse.

There have also been scares fueled by anecdotes regarding such disparate items as silicone breast implants, cell phones, and vaccinations. In the 1990s many women blamed their cancers and other diseases on breast implants. Talk show hosts like Oprah Winfrey and Jenny Jones
presented groups of women who were suffering from cancer or some
other serious disease and who had been diagnosed after they'd had breast
implants. The stories tugged at the heartstrings and brought tears to many sensitive eyes, but the scientific evidence did not exist that there was a causal connection between the implants and any disease. That fact did not prevent lawyers from extorting $4.25 billion from implant
manufacturers. Marcia Angell, former executive
editor of the New England Journal of Medicine, brought the wrath of feminist hell upon herself in 1992
when she wrote an editorial challenging the Food and Drug
Administration's decision to ban the manufacture of silicone breast
implants. The scientific evidence wasn't there to justify the ban. She
eventually wrote a book describing the fiasco: Science on Trial: The Clash of Medical Evidence and the Law in the Breast Implant Case. The scientific evidence is now in. The implants don't cause cancer or other diseases, and the FDA has lifted its ban. When the data were collected, they showed that women with silicone breast implants did not suffer cancer or any other disease at a significantly higher rate than women who had not had implants.

The public fear that cellphones might be causing brain tumors was first aroused not by scientists but by a talk show host. On January 23, 1993, Larry King's guest was David Reynard, who announced that he and his wife Susan had sued NEC and GTE on the grounds that the cellphone David gave Susan caused his wife's brain tumor. There was nothing but junk science to back up her claim, plus the fact that the tumor appeared near where she held the phone to her ear. She was diagnosed seven months after receiving her phone and died a few months after filing the suit. The suit was dismissed in 1995. A dozen similar lawsuits followed; all were dismissed (1,2,3). For those who think scientists and industries don't take anecdotes seriously, consider this: soon after Susan's lawsuit was dismissed the cellphone industry committed $25 million for safety studies. Many studies have been conducted over the past fifteen years and so far no evidence of a causal link between cellphones and brain cancer has been found.

The fear of vaccines has led to outbreaks of measles and deaths to infants from whooping cough, events that should not be happening in this day and age. In Japan, when the vaccination rate for pertussis (whooping cough) dropped 70% from 1974 to 1976, the number of cases of pertussis went from 393 to more than 13,000 and the number of deaths from pertussis went from 0 to 41.*

Testimonials and vivid anecdotes are unreliable for various reasons. Stories are prone to contamination by beliefs, later experiences, feedback, selective attention to details, and so on. Most stories get distorted in the telling and the retelling. Events get exaggerated. Time sequences get confused. Details get muddled. Memories are imperfect and selective; they are often filled in after the fact. People misinterpret their experiences and are biased and selective in what interpretations they include and exclude from consideration. Experiences are conditioned by biases, memories, and beliefs, so people's perceptions might not be accurate. Most people aren't expecting to be deceived, so they may not be aware of deceptions that others might engage in. Some people make up stories. Some stories are delusions. Sometimes events are inappropriately deemed improbable when they might not be that improbable after all. In short, anecdotes are inherently problematic and are usually impossible to test for accuracy.

Some fields rely almost exclusively on anecdotal evidence and testimonials, e.g., alternative medicine, the paranormal, the supernatural, and the pseudoscientific. Stories of personal experience with acupuncture, mediums, ghosts of relatives, or free energy machines have little scientific value. Sincere and vivid accounts of one’s encounter with an angel or the Virgin Mary, an alien, a Bigfoot, a child claiming to have lived before, purple auras around dying patients, a miraculous dowser, a levitating guru, or a psychic surgeon are of little value in establishing the reasonableness of believing in such matters. If others cannot experience the same thing under the same conditions, then there will be no way to verify the experience. If there is no way to test the claim made, then there will be no way to tell if the experience was interpreted correctly. If others can experience the same thing, then it is possible to make a test of the testimonial and determine whether the claim based on it is worthy of belief. As parapsychologist Charles Tart once said after reporting an anecdote of a possibly paranormal event: “Let’s take this into the laboratory, where we can know exactly what conditions were. We don’t have to hear a story told years later and hope that it was accurate.” Another parapsychologist, Dean Radin, also noted that anecdotes aren't good proof of the paranormal because memory “is much more fallible than most people think” and eyewitness testimony “is easily distorted” (Radin 1997: 32).

Testimonials are of little use to science because selective thinking and self-deception can't be controlled for or mitigated in randomly experienced events as they must be in scientific observations and experiments. Most psychics and dowsers, for example, do not even realize that they need to do controlled tests of their powers to rule out the possibility that they are deceiving themselves. They are satisfied that their experiences provide them with enough positive feedback to justify the belief in their paranormal abilities. It is common for psychics, dowsers, and their admirers to remember their apparent successes and ignore or underplay their failures. Controlled tests can also determine whether other factors such as cheating might be involved.

If testimonials are scientifically worthless, why are they so popular and why are they so convincing? There are several reasons. Testimonials are often vivid and detailed, making the coincidental seem meaningful and giving a causal interpretation more credibility than it deserves. They are often made by enthusiastic people who seem trustworthy and honest and who seem to lack any reason to deceive us. Sometimes a testimonial is given soon after an experience while one’s mood is
still elevated from the desire for a positive outcome. The experience
and the testimonial it elicits are given more significance than they
deserve and are of little
value in establishing the probability of the claims they are put forth
to support. Testimonials are often made by people with a semblance of authority, such as those who wear a uniform or hold a Ph.D. or M.D. degree. Testimonials are often made by popular figures given a bully pulpit on widely viewed television programs. To some extent, testimonials are believable because people want to believe them. Testimonials accompanied by claims of government or Big Pharma conspiracies to stifle a new cancer cure or free energy device are popular among a certain class of people.

Testimonials and anecdotes are used to support claims in many fields, including medical science. Giving due
consideration to such testimonials is considered wise, not foolish. A
physician will use the testimonies of his or her patients to draw
conclusions about certain medications or procedures. For example, a
physician will take anecdotal evidence from a patient about a reaction
to a new medication and use that information in deciding to adjust the
prescribed dosage or to change the medication. This is quite reasonable.
But the physician cannot be selective in listening to testimony,
listening only to those claims that fit his or her own prejudices. To do
so is to risk harming one’s patients. Nor should the rest of us be
selective when listening to testimonials regarding some experience.

A testimonial match or testimonial game, often referred to simply as a testimonial, is a practice in some sports, notably football and especially in the United Kingdom, where a club puts on a match in honour of a player for service to the club.Clubs typically grant testimonials to players upon reaching ten years of service with a club, although they sometimes are given to players with less service as they approach retirement. These matches are always non-competitive. Typically, former greats for the club are invited to participate, either in the match itself or the surrounding festivities. For especially prominent players, some all-time greats of the game may be invited, or the opposition can be a national team. Alternatively, if a player is strongly identified with more than one club team, the testimonial may involve the player's current and past clubs.

The expression anecdotal evidence refers to evidence from anecdotes. Because of the small sample, there is a larger chance that it may be unreliable due to cherry-picked or otherwise non-representative samples of typical cases. Anecdotal evidence is considered dubious support of a claim; it is accepted only in lieu of more solid evidence. This is true regardless of the veracity of individual claims

his really is a really valuable slant on items. I'm a weblog beginner. I'm also thinking about such type data. You supply me an excellent example. Thanks really a lot. I will preserve on reading your blogs.

Anecdotal evidence has several definitions, which usually relate to how certain types of evidence cannot be used to logically conclude something. We see examples of this type of evidence all the time in commercials. A person tells us how their breath feels fresher after using a certain brand of toothpaste, or people testify to the clearing of their acne as a result of special products. Anecdotal evidence is often used in place of clinical or scientific evidence, and may completely ignore research or harder evidence that points to an opposite conclusion.

The expression anecdotal evidence refers to the use of particular instances or concrete examples to support a general claim. Such information (sometimes referred to pejoratively as "hearsay") may be compelling but does not, in itself, provide proof.