Comments on: The Context of Anecdotes and Anomalieshttp://theness.com/neurologicablog/index.php/the-context-of-anecdotes-and-anomalies/
Your Daily Fix of Neuroscience, Skepticism, and Critical ThinkingFri, 09 Dec 2016 15:07:46 +0000hourly1https://wordpress.org/?v=4.6.1By: zoe237http://theness.com/neurologicablog/index.php/the-context-of-anecdotes-and-anomalies/#comment-28382
Mon, 13 Dec 2010 04:54:20 +0000http://theness.com/neurologicablog/?p=2560#comment-28382I agree with most of what you’ve written, that anecdotes and anomalies are considered in generating hypotheses. I don’t think, however, that you’ve proven that they aren’t used at all in the later justification. Many many research studies are nothing but a collection of anecdotes, particularly non rcts. We *attempt* to collect these observations methodically, but science is biased, there is no doubt in my mind about that. Confirmation bias is common every step of the way, not just the first, and subject to file drawer effect. I do agree that it is the most unbiased way of thinking that we currently have, but to ignore the human aspect of science is folly. Scientists must always be aware that they are human and subject to fallacy. Anomalies are usually found and measured via standard deviation and we attempt to explain them away. Sometimes anomalies accumulate until we DO need a new paradigm. So I’m unconvinced that anomalies and anecdotes are never a part of the later steps of the scientific method. Or maybe I’m missing the point?
]]>By: BillyJoe7http://theness.com/neurologicablog/index.php/the-context-of-anecdotes-and-anomalies/#comment-28376
Sat, 11 Dec 2010 20:33:37 +0000http://theness.com/neurologicablog/?p=2560#comment-28376

“For every anecdote, there is an equal and opposite anecdote”
“The plural of anectode is not data”

Anecdotes are useful for generating hypotheses but useless for testing hypotheses.

And Elmer,
Perhaps those weren’t the best examples, but they were the first things that came to mind. I was just using them to try and gain some tools for rational discourse in the future with people who advocate for whatever unconfirmed remedy/theory/what have you with a stockpile of anecdotes.

Seems to me that acupuncture and homeopathy are not the most relevant examples, since, beyond having mere anecdotal evidence to back them up, there is good evidence showing that they don’t work (I still think some kinds of dry needling that haven’t been studied adequately could possibly have non-placebo effects, but presumably we’ll see, eventually). Of course, many treatmnets commonly practiced by doctors have only anecdotal evidence to back them up.

I like that. It’s an easy, concise concept to remember. And I now feel better equipped to recognize confirmation bias.
Thank you both for the clarification.

]]>By: Steven Novellahttp://theness.com/neurologicablog/index.php/the-context-of-anecdotes-and-anomalies/#comment-28306
Wed, 08 Dec 2010 13:15:06 +0000http://theness.com/neurologicablog/?p=2560#comment-28306As BillyJoe said – the plural of anecdote is anecdotes, not data. And you already hit upon the reason – systematic and reinforcing biases. Anecdotes generate beliefs which generate more anecdotes in a process of confirmation bias that can seem compelling but is not predictive.

In other words – millions and even billions of people can be systematically wrong.

RedQueen – I would say no, but the separation of discovery and confirmation is simply just another demarcation problem. Most such dichotomies are, in fact, continua – but that does not mean they are not real. There is a difference between science and pseudoscience, even though there is not bright line between the two but rather a continuum.

So while many scientific activities combine or blur the lines between discovery and validation, that does not mean these are not two distinct concepts that need to be understood.