Skeptophilia (skep-to-fil-i-a) (n.) - the love of logical thought, skepticism, and thinking critically. Being an exploration of the applications of skeptical thinking to the world at large, with periodic excursions into linguistics, music, politics, cryptozoology, and why people keep seeing the face of Jesus on grilled cheese sandwiches.

Monday, June 19, 2017

Dowsing and statistical significance

A while back, my wife and I and two friends were in a gift shop, and on a rack of books for sale I saw one called Dowsing for Beginners by Richard Craxe. I picked it up, and flipped through a bit of it. I was a little surprised -- I never thought of dowsing as something anyone would write a how-to manual about.

Dowsing, for those of you (probably few) who don't know about this practice, is the use of a forked stick (or in some cases) a pair of bent wires to locate everything from sources of water to lost objects. The claim is that the dowsing rod exerts a pull on the dowser's hands, or actually turns and points toward the desired goal. As strange as this idea is, I find that of all the odd practices I hear about from students of mine, this one is the one that they will argue the most vociferously for. This, surprisingly, includes students whom I would normally think of as rationalistic skeptics -- students who scoff at other forms of woo-wooism.

You might wonder how this practice is supposed to work. Explanations, of course, vary. The use of willow branches for dowsing for underground water is sometimes explained, in all seriousness, as working because willow trees like growing near water, so the wood is magically attracted to sources of it. Other people believe that dowsers themselves are "sensitive," so that the dowsing rod itself is only acting as a tool to focus their mysterious ability. Dowsing for Beginners goes through some nonsense about there being a "universal mind" that everyone has access to, and it knows everything, and therefore when you practice dowsing, you're tapping into a source of knowledge that can provide you with information about where to drill for water or where you accidentally dropped your car keys.

[image courtesy of the Wikimedia Commons]

The next question is, does it work? The simple answer, of course, is no. Controlled studies have shown no results whatsoever, an outcome discussed at length in James Randi's wonderful book Flim-Flam! The fact is, my students who know "an uncle of a friend" who successfully dowsed for water are being suckered in by the fact that there's hardly anywhere in upstate New York that you won't hit water if you dig deeply enough.

A subtler problem with practices like dowsing is that a lot of people don't understand the concept of statistical significance. A fine example of this, apropos of dowsing, is a study done in 1988 by Hans-Dieter Betz, in which six dowsers were said to "[show] an extraordinarily high rate of success, which can scarcely if at all be explained as due to chance." However, the Betz experimental protocol was highly suspect from the beginning -- Betz and his group evaluated 500 dowsers in a preliminary test, eliminated all but the most successful fifty, and then found that of those, six of them scored much better than you would expect from chance alone.

The flaw is not that Betz hand-picked the subjects -- if dowsing works, presumably some individuals would be better at it than others, being more in touch with the "universal mind," or whatever. The problem is that even if success at dowsing is pure probability -- i.e., it doesn't work at all except by chance -- some people will do astonishingly well, and that fact means exactly nothing at all. To explain this a little more simply, let's suppose that we had a thousand people take a random, hundred-question test consisting of four-choice multiple-choice items. There is no skill involved; you just fill in a paper with a hundred random A's, B's, C's, and D's, and it's graded against an equally random key. What's your likely score?

Well, 25%, of course, would be the likeliest outcome. You have a 1/4 chance of getting each question "right," so you would be expected to score somewhere around a 25%. The problem is, that's just the most likely score; that's not necessarily your score. In fact, you might score far better than that, or far worse; 25% is just the average score. In a large enough sample of test-takers, some people would seem to score amazingly well, and it's not that they're psychic, or brought their dowsing rods along to point at the correct answers, or anything; it's just a probabilistic effect.

Same with the dowsers. Jim Enright, a prominent skeptic, criticized the Betz study, and showed statistically that in a group of 500 dowsers, you'd expect that six or so of them would be high scorers, just by random chance. Enright described the Betz study not as proving that dowsing is a real phenomenon, but as "the most convincing disproof imaginable that dowsers can do what they claim." Six above-average scorers out of a sample of 500 is simply not a statistically significant finding.

So, what we have here is a phenomenon that has (1) no empirical evidence in its favor, (2) no scientifically reasonable explanation about how it could work, and (3) a cogent argument that explains away cases where it has seemed to be successful. However, as usual, people are more convinced by a flashy practitioner of mystical arts than they are by talk of probability and scientifically controlled studies, so I've no real hope that dowsing will become any less popular. On the other hand, I suppose if it resulted in your finding a good site for your well, or locating your car keys, then who am I to argue? As Alexandre Dumas famously quipped, "Nothing succeeds like success."