The Baloney Detection Kit

Carl Sagan was a pioneering astronomer at Cornell University and a uniquely talented science communicator, probably best known for the original Cosmos television series which he co-wrote and first presented in 1980.

He was also a wonderful writer who shared with readers the joys of discovering and understanding how the world and universe around us works through scientific inquiry.

Photo credit: Wikipedia

As a public educator Sagan wrote about the need for better science literacy.

“We live in a society exquisitely dependent on science and technology, in which hardly anyone knows anything about science and technology.”

Chapter 12 is the ‘The fine art of baloney detection’ where Sagan describes the tools of science that make up a Baloney Detection Kit, and how to use them to help us determine which assertions have merit and which do not, regardless of who is making them.

“What’s in the kit? Tools for skeptical thinking. What skeptical thinking boils down to is the means to construct, and to understand, a reasoned argument and – especially important – to recognize a fallacious or fraudulent argument.”

A condensed version of the kit is given below; the full version is here, and a videoversion by Michael Shermer (of Skeptic Magazine) includes updated examples of the kit’s use.

Tools in the kit:

Wherever possible there must be independent confirmation of the “facts.”

Encourage substantive debate on the evidence by knowledgeable proponents of all points of view.

Arguments from authority carry little weight – have the claims been verified by others?

Spin more than one hypothesis. If there’s something to be explained, think of all the different ways in which it could be explained.

Try not to get overly attached to a hypothesis just because it’s yours. Compare it fairly with the alternatives. See if you can find reasons for rejecting it. If you don’t, others will.

Quantify. If whatever it is you’re explaining has some measure, some numerical quantity attached to it, you’ll be much better able to discriminate among competing hypotheses. What is vague and qualitative is open to many explanations.

If there’s a chain of argument, every link in the chain must work (including the premise)—not just most of them. Logic, reason and empirical evidence are required.

Occam’s Razor. This convenient rule-of-thumb urges us when faced with two hypotheses that explain the data equally well to choose the simpler.

Always ask whether the hypothesis can be, at least in principle, falsified. Propositions that are untestable, unfalsifiable, are not worth much. You must be able to check assertions out.

The Baloney Detection Kit also serves as a filter for catching the most common fallacies of logic and rhetoric:

ad hominem, attacking the arguer and not the argument

argument from authority

argument from adverse consequences

appeal to ignorance – the claim that whatever has not been proved false must be true, and vice versa

special pleading, often to rescue a proposition in deep rhetorical trouble

begging the question, also called assuming the answer

observational selection – counting the hits and forgetting the misses

statistics of small numbers – where a small increase gives a large, and probably most impressive, percent change

misunderstanding of the nature of statistics (e.g., fully half of all Americans have below average intelligence)

false dichotomy—considering only the two extremes in a continuum of intermediate possibilities

short-term vs. long-term

slippery slope

confusion of correlation and causation

straw man – caricaturing a position to make it easier to attack

suppressed evidence, or half-truths

weasel words

In his last interview, Sagan noted that “Science is more than a body of knowledge. It is a way of thinking; a way of skeptically interrogating the universe with a fine understanding of human fallibility. If we are not able to ask skeptical questions, to interrogate those who tell us that something is true, to be skeptical of those in authority, then, we are up for grabs for the next charlatan (political or religious) who comes rambling along.”

The kit is of course also useful for detecting baloney in dubious claims made in the fields of medicine and health, the environment, agriculture and food.

When a woo train pulls into town with a load of pseudoscience, be sure to have your Baloney Detection Kit handy. It can be used to evaluate claims such as:

Vaccines cause autism, there is no global warming, the moon landings were faked, farmers using Bt cotton in India have increased rates of suicide, Kauai is drenched in pesticides and it is making residents sick, organic food is more nutritious and produced without pesticides, there is a new mysterious organism in Roundup Ready soybean and corn impairing the health of plants and animals, beer contains a complicated sounding chemical used in antifreeze, the Bt gene inserted into GE crops contains a toxin that will poke holes in your stomach, the viral promoter in GE crops could turn on silenced cancer genes in our cells, and on and on.