Tag Archives: Scientific method

When debating some controversial science claim, I’ve often heard people argue that “scientists are always wrong.” Usually it’s from those arguing for some “thing,” medicinal or otherwise, that’s supposed to make your life better, but seems to fly in the face of science, or at least isn’t backed by any reputable study.

For example: people arguing marijuana (or at least some of its chemical constituents) kills cancer, but “western medicine” wants to keep you sick with things like chemotherapy, so they’re suppressing the evidence. Something I largely debunked here with just a little critical thinking. So I don’t need to rehash that specific point again.

But what I do want to cover, is the notion that scientists are often wrong. If you were to ask this question, and require a simple “yes” or “no” answer to whether scientists are often wrong, the answer I suppose is “yes, yes they are.” But that’s partly by design, and this is an important part few seem to understand.

If you were to ask most people outside the science community what science is, they’d probably conjure up people in a lab with beakers mixing chemicals together, and hoping that by combining bleach, marijuana, gluten-free wheat, and organic apple seeds, somehow, you’ll have a cure for any particular rare condition that ails you.

But what is science really? It’s a method—thus the moniker “the scientific method.” It’s a means by which you can most likely find the truth about something.

This is WAY oversimplified, but it basically goes like this:

You observe something in the world, and go “Hmm?” Emphasis on the question mark.

This is how science starts—people have questions.

Non-scientists will often answer them with something complicated and/or supernatural like gods or aliens, if they’re struggling to find a more natural answer to their question. Others just make a random guess based on what they think is most likely the best answer, and go with it, evidence-be-damned.

Why?

Because science is hard work, and moving on past this phase requires far more than just imagination.

Scientists however will assume nothing until there is evidence of something. So if they’re compelled to answer the question, they’ll move to phase 2.

2. You gather as much evidence as you can on the thing you saw.

From this point forward, we separate the scientists (or skeptics like myself, since I’m not a professional scientist) from the non-scientists, because non-skeptics/scientists stopped after phase 1 when they opted for a guess.

If there’s no evidence to gather, sadly your work here is done, and you must accept that you don’t know. Think of cryptozoology, like Bigfoot ‘experts” or ghost hunters and such. They have no evidence to test (like an actual bigfoot to observe and test—alive or dead), yet they make claims anyway which are always pure speculation.

So whatever they’re doing, trust me, it isn’t science. Using scientific words, and scientific equipment doesn’t make one a scientist, following the rules of the scientific method does.

3. If you are able to gather evidence, you form a hypothesis, what a layperson might call an “educated guess,” based on the evidence you’ve gathered.

This forming of a hypothesis is different from a guess, in that it is based on the evidence you’ve gathered so far, and none of the evidence gathered should be contrary to your hypothesis.

A guess is often just what you think is most likely, but isn’t always weighed against the evidence you have. You see this often in political or religious debates, where people have an ideology, and any evidence they’ve gathered so far, if it doesn’t support their ideology, is thrown out as if the evidence must somehow be flawed. It’s a process called confirmation bias, and sadly we all do it. Especially if we’re not even aware it’s a thing, and that we should avoid it.

4. Here’s where those beakers might come in. Time to do some testing.

Now here’s the interesting part. If you’re a scientist, you try to prove yourself wrong. Yeah, I said it—WRONG. It’s a principle called falsification.

If you can’t disprove (falsify) your hypothesis, then you assume you have a potentially true hypothesis. Professionals will try to get such findings published in a peer-reviewed and reputable journal, then hope other scientists in their field will test it.

Know what those others will do?

You guessed it, try to prove the hypothesis wrong as well. Not because they want the first scientist to be wrong, or are their competition, but because that’s just how it works.

So why try to prove it wrong, versus prove it right? Derek Muller from the highly-respected YouTube channel Veritasium made an excellent video explaining why, in a pretty unique presentation. I encourage you to watch it. It will make you think differently, if you don’t already think this way.

It’s not all bad, though. One of these drugs for instance, was sildenafil, the active ingredient in Viagra. It was initially meant for the treatment of blood pressure, and through clinical testing proved ineffective for that purpose, but highly effective at “pitching tents.” Serendipity at its finest, since Viagra has proven to be far more profitable for its founder, than the blood pressure medicine would likely have been.

But such serendipity is simply an added benefit of rigorous testing, and the proper documentation of all findings. Science is technically always about the unknown. You can ignore things that don’t fit into your desired outcome, or you can follow the data wherever it takes you and learn from it.

But with medicine, obviously lives are at stake in a pretty profound way, so the level of scrutiny there is rightfully going to be higher than any other field of science.

To a layperson, this might seem like the argument is that scientists are wrong 4,999 times out of 5,000, and this is where the “scientists are always wrong” myth starts to germinate. Not because they are wrong, but because of how science is often reported.

You see, technically, they weren’t wrong. They never made the claim you often heard. They formed a plausible idea, and then tested it rigorously to see if it stood up to the scientific method. With medicine, the number of phases a drug goes through is staggering.

Again, very oversimplified, but it’s something like this:

Test it in a lab (say in a petri dish), basically taking some live diseased cells, put them in a dish, and see if the chemical in question kills them, or otherwise does what you’re hoping it does.

Test them in animals, like rats

Test them on an animal that may be closer to humans genetically, like a apes

Test them on a few healthy humans to make sure they don’t get sick

Test them on a very small amount of humans to see if it helps

Test them on a medium-sized group of humans to see if you can show a statistically significant result

Test them on a large group of humans so you have a result you can argue is most certainly one certain assumptions could be made about.

Now you can start to understand why it can take 12 years for a drug can get to market. But here’s where the “scientists are always wrong” argument often comes into play. Because after phase 1, the findings are published. After phase 2, the findings are published. After phase three, again, the findings are published. This will be true for all phases.

Abraham Lincoln Weighing In On the Internet

Now, a reporter, website, or any other type of media who knows nothing about science, picks up the published study from phase 1, and writes a big, attention-grabbing headline that reads “Scientists discover cure for cancer,” and a straw man of the finest quality is born.

Because they don’t understand these results are merely a step along the road of a cure, and with respect to cancer, each one is different anyway. The tests would surely be against one type of cancer, such as lung, breast, or prostate cancer, for instance. Not just cancer as a whole.

A year later when this substance fails phase 2, another reporter reports that scientists show the same substance now is not effective at curing cancer. And the public is left thinking scientists screwed up—they didn’t.

People who know nothing about science irresponsibly misrepresented the phase 1 story, the populace which aren’t largely scientists didn’t know how to decipher the misleading clickbaity headline, and voila, “Scientists are always wrong.”

You can also find this notion with people who are skeptical of larger theories, like the big bang theory, or evolution. They’ll point out that “evolution doesn’t explain how life started” or other things we don’t know yet.

But what such people seem to not understand, is that large theories have a couple important facets they aren’t considering.

First, think of a particular scientific theory as a puzzle depicting Albert Einstein standing in his study. Your puzzle has a thousand pieces, and you’ve so far rightly inserted 950 of them. You can clearly see it’s Einstein in his study at this point, but there’s a few small details (missing pieces), maybe a few books on the shelf in the background, that you can’t yet identify. You’re still not sure about such facts, and that may change the picture significantly, but it is much more likely it will not, instead just filling in those small blanks.

For evolution for instance, this might be the fact it isn’t understood how non-living organics (carbon-based substances) became living organisms (carbon-based life forms). Just because we don’t understand that facet, doesn’t mean the other “950 pieces” we do understand aren’t true, or are suspect.

The other important part to understand about a theory, is that it’s a theory instead of a law, because it isn’t entirely observable. We can see the effect of gravity on something and measure it accordingly, so that’s a law.

Charles Darwin: Author of The Origin of Species and impetus for the Theory of Evolution

But with evolution or the big bang, we can’t go back in time and watch it happen. So all we can do is theorize based on data we have, and try to recreate the event in some small way so we can observe it. From there, we can make a fair assumption the theory holds true if replicated.

Since skeptics are often religious in nature, they’ll refute science with the Bible, Quran, or other religious works, as if we should assume such works are true. But almost all claims made by modern-day scientists which contradict religion, have a mountain of evidence supporting them, to the point that people like the pope himself, have acquiesced to, as reported here. And it’s important to understand that such religious works aren’t supported by evidence either, as far as we know. We can’t go back in time and observe them being written, nor do we have any supporting documents to back up their claims. It could literally have been written by one delusional person thousands of years ago, sold to a larger group of people as truth, a religion was born thereafter, and we’d have no way of knowing. So assuming such religious texts must be right on the subject of gaps in scientific knowledge does not follow any reasonable logic.

So are scientists always wrong? Of course not. Through the course of their methods, they form hypothesis which they often prove wrong, but by the time they get to a point where they make a claim, they are demonstrably far more correct than any other group of people on the planet. Be a skeptic and question everything, including science. But proper skepticism should lead you to find that the scientists did their part correctly; the errors came in how that information made its way to you along the way.

There can be no doubt, many people suffer from iatrophobia ​—a fear of doctors. Being poked with needles, recommended for surgeries, or placed on never-ending drug regiments can make people want to curl up in a ball somewhere and hide forever.

As a result of this somewhat understandable fear, people often look to alternative medicine for the answers to their problems. Whether it be practices like chiropractic, homeopathic, holistic, acupuncture—the list is mind numbing.

So first, let’s look at the definition of the word medicine.

med·i·cine

ˈmedəsən/

noun

noun: medicine; plural noun: medicines

the science or practice of the diagnosis, treatment, and prevention of disease (in technical use often taken to exclude surgery).

a compound or preparation used for the treatment or prevention of disease, especially a drug or drugs taken by mouth.

When discussing the idea of alternative medicine, it should first be understood that there is essentially no such thing as alternative medicine.

There is medicine: practices or compounds that treat a medical condition or disease.

There is non-medicine: practices or compounds that do not treat a disease or medical condition.

There is no middle ground here, the item in question either works, or it doesn’t work. The level of effectiveness may very, but there has to be a noted effect, above the margin of error or the placebo effect, or it cannot be considered medicine.

In a fictional example, I will create a drug called Libertol (I had to throw a little politics in this thing), and I will have invented it to treat a disease called oppressionitis.

Assuming I’m a drug company, I’m going to be biased to confirm that the millions of dollars I spent developing Libertol actually yielded a functional, and thus marketable product. Since that bias could either unintentionally skew my results, or raise concerns I might purposefully have skewed the results, I would find an independent party to test Libertol in order to rule out any actual or perceived biases I might have.

The independent testing facility would start by looking for a number of people with oppresionitis, and ask them if they’d be willing to submit to a study on a new drug to treat this horrible disease. For the sake of argument, let’s say we get 500 people to be our guinea pigs.

Clinical Study Map. Click for more info

Why 500? Because “anecdotal evidence isn’t evidence.” (Great science maxim #1, there will be more) But what do scientists mean by that?

Imagine you flip a coin once, and it lands heads up. Would you then assume that every time you flip a coin, it will land heads up because of that one flip? Of course you wouldn’t. That is essentially anecdotal evidence. One, or barely more than one, instance is almost never to be treated as if its results are indicative of what should be expected on a consistent basis. This is also why you are wise to often get a 2nd or even 3rd opinion when seeing doctors.

So maybe you flip the coin ten times? It lands heads seven out of ten due to random chance, which is not implausible. If I’ve done my math right, is likely to happen about 1 out of 8 times (15:125 to be exact). So does that mean a coin is prone to land heads up 70% of the time? Again, of course not. The more times you flip it, the closer it will get to its actual probability of 50:50 as the odds start to balance out.

So when doing a clinical trial, the more people you can test, the more accurate your results will be, and this is why we want 500 people in our above fictional example.

These trials are actually done in phases, with only about 20-80 people at first, if the drug ends up having detrimental side effects, you don’t want it to affect a large number of people, after all. But by the time the study hits phase 3, there will likely be thousands of participants being evaluated. But I’m simplifying the three phases down to one, and the process in general, for the purposes of this article.

Once volunteers are gathered up, half of them will be given the actual drug, the other half will be given a placebo (a non-drug). This placebo is designed to control for the placebo effect, a condition whereby a person will convince themselves a drug works, even if it doesn’t.

The placebo effect will generally not alter things that are purely quantitative, like blood pressure, heart rate, or other measurable conditions, but it can have quite the profound effect on subjective data, such as pain level and other issues the user merely reports on versus being tested with equipment.

The results of the group who took Libertol would then be tested against the people who took the placebo they thought was Libertol, and Bob’s your uncle—you have a result.

If Libertol actually worked, it would show as such by being more effective than the placebo was at treating oppressionitis. Otherwise, the results will come back as “no more effective than a placebo,” and Libertol would be sent to the ineffective drug graveyard in the sky, barring any noted side effects that may be beneficial elsewhere.

So the moral here, is that you shouldn’t take medical advice from someone who tried something once and it worked. There are infinite other possibilities to explain why it seemed effective, rather than it actually being effective. Instead, ask your doctor and if you’re a true skeptic, research yourself for clinical trial results.

People who had actual acupuncture reported similar results, within the margin of error, to people who were treated with fake acupuncture where the needles were purposely misplaced. It has never been shown to be more effective despite its hundreds of years of history. The only “ancient Chinese secret” here, is that it doesn’t work.

Acupuncture

While we place a lot of trust in doctors, not all of them adhere to the scientific method exclusively, some are flat-out quacks. But if you want medicine versus non-medicine, you should demand as such. So lesson learned, I won’t be seeing that neurologist again.

There are two types of alternative medicine practitioners:
Fraudsters who know they’re taking advantage of you, or
ignorant people who simply don’t know better. But why give your money to either one?

If someone is purposefully deceiving you for a fee, that is a horribly immoral practice where you take someone’s hard-earned money that could be used for something that would help them, and instead sell them something that won’t. If I lived in an anarchistic country, I would want to destroy every one of these sociopaths. They are valueless human beings, in my opinion.

But even if they’re just ignorant and think these practices actually work, they’re still taking your hard-earned money for something that doesn’t. Assuming you don’t have money to burn, why do exactly that?

It is true, an argument can be made that if the placebo effect does work for things like pain, giving someone a placebo might make them feel better without introducing foreign chemicals into their body. But it’s still inherently dishonest, and I would hope none of you would willingly pay someone to lie to you.

All health claims are scientific in nature, meaning there are biological processes that are either going to get better, stay the same, or get worse in your body after treatment. So trust in things that have passed the test of the scientific method, and understand that “that which can be destroyed by the evidence, should be.” (Great science maxim #2)

So if I am a libertarian, why do I care? Shouldn’t people be free to put into their body whatever they want? Absolutely!

While I would never prosecute a non-medicine consumer (I refuse to call it alternative medicine), fraud is a violation of your right to property, specifically, your money, and should be prosecuted to the full extent of the law.

I’d like to quickly dispel a couple of these myths by showing you the purpose, methodology and power of skepticism, or critical though, hopefully encouraging you to do your own.

Things like acupuncture, for instance, are said to be practices that are hundreds of years old (as if that is somehow evidence), and that doctors won’t often recommend them because there’s no money to be made. Might I point out that acupuncturists do in fact work for money? Therefore there is actually money to be made—they’re making it. In order to believe this argument, you must do no critical thinking whatsoever. I just dispelled it in one sentence.

People argue we have a cure for cancer, but drug companies are suppressing it to make more money on drugs that only treat cancer, not cure it. This one requires a little more skepticism, but let’s bring up some valuable points to debunk this.

Drug companies do make drugs that cure cancer. Chemotherapy and radioactive seeds are but a couple. We simply don’t have one drug that cures all cancer all the time, nor one drug that doesn’t have harmful and potentially fatal side effects as chemotherapy does.

This assumption also means that a drug company would have to invest the typically millions of dollars required to develop a drug that works, then shelve it without recouping any of that money as a return on their investment. Why would they develop it in the first place if they don’t want such a return? These conspiracy theorists are arguing how greedy drug companies are, then asserting a claim that they are purposefully throwing money away—an overwhelming contradiction.

Then we must also assume that the scientists who went to medical school, usually with the intent of saving lives, many of them specifically dreaming of being “the one” who cures cancer, spent years developing a drug that works. However, once realizing their dream of curing cancer, completely eschewed their ideals and agreed to suppress the drug for money. A theory that’s insulting to every moral medical student who ever slaved away for eight years in college to save your life.

Lastly, the above two bullets would both involve more than one person. The drug company has a board of directors, and the labs often have tens or hundreds of people on staff, yet not one of them headed over to CNN or Fox News to blow the whistle?

I could go on and on pointing out the logical fallacies and ignorance of thinking these ways. But hopefully my two examples of how to be a proper skeptic will inspire you to do more critical thinking of your own, instead of buying into these radical conspiracies. The life and the pocketbook you save could be your own.

P.S. I linked to an article debunking acupuncture above. Here is another debunking homeopathy with what is effectively simple, yet astounding math. I promise, it is worth the read. (Click the picture)

I don’t often weigh in on the global warming debate as I’m not a climatologist. I have made it clear however, that I believe in maintaining proper scientific skepticism in life.

First and foremost, I wish to say that I believe climatologists on both sides of the aisle have done good science. Mankind certainly produces a lot of CO2 which will no doubt have an effect on the environment, and these effects are worth investigating.

That being said, I wish to consider a few points.

Earth is rather large. Every organism living on this planet, along with objects in our solar system, are all variables that affect our climate. Accounting for all of them is nearly impossible.

The Blue Marble

Therefore, not accounting for all of them while making claims about how they will react to increased CO2 production, will always be educated guesswork.

Also, when climatologists make predictive models, as near as I can tell, they often make these models while assuming all other variables will either remain constant, or will not counteract the change, but instead merely succumb to it.

For example, imagine one were to observe two birds in their back yard; they look every day for a month, but on average, they always see about two birds. Now imagine this person throws a bag of bird seed in their back yard each day. Considering no other variable, one would assume the result would be an ever-growing pile of bird seed in their yard. In reality, their bird population of two would elevate to fifty or more birds, which wouldn’t result in a pile of bird seed as predicted, but a pile of bird poop instead.

With Earth being an ecosystem, as we animals (yes, humans are animals) produce more CO2, I’ve yet to hear anyone rule out that the plant kingdom, which would thrive in a CO2 rich environment, would not simply grow in numbers, evolve plants which consume more CO2, and/or spawn a new mechanism for filtering or consuming CO2 that we haven’t even imagined; in doing so, counteracting the increased greenhouse gasses produced by the increasing animal population. Just as the deer population, if left unchecked, will die of disease and famine, nature always seems to randomly, and quite unpredictably at times, find a way to maintain balance through evolution.

I’m not making this case mind you; again, I’m not a climatologist—please no hate mail. But one thing I do know is that predictive climate models have often been wrong. Eschewing climate science would be a terrible mistake, but let’s continue to compare actual results with predicted ones; leaving politics out of it for now, until we can accurately predict the effects, and effectively devise mechanisms to deal with the issues that we determine nature cannot naturally resolve for us.

There’s a saying I once heard that in science, most great discoveries are not followed with an exclamation of “Eureka, I’ve found it!” but instead, a far less exciting, “Hmm, that’s odd.”

Viagra was supposed to be a heart medication; it failed miserably. But oddly enough, it turns out Viagra can pitch a tent like a scout troop leader. Microwave ovens came about after Percy Spencer’s chocolate bar melted when placed near a magnetron and he wondered why. Post-it notes were a failed attempt at making a strong adhesive, which it clearly wasn’t. Instead of scrapping a million dollar project, 3M made lemonade out of lemons.

The list of happy accidents like these goes on forever. Science isn’t just about resolving a given issue, it’s also about investigating random discoveries that were often diversions along the way.

Sometimes however, good intentions can go seriously wrong. For instance, I took a tour of Mammoth Cave in Kentucky; I highly recommend it. In Mammoth Cave, as with all caves, the temperature and the humidity are basically constants. Mammoth Cave is a cool and damp ≈54°F 24/7/365. In 1839, Dr. John Croghan, a sufferer of tuberculosis, observed that the cave’s cold and damp air made him feel refreshed and well.

He bought the cave and opened up a tuberculosis clinic inside it. Today, we know that warm and dry air is best for people suffering from TB, but Dr. Croghan found out the hard way that cold and damp air is bad for it—all 15 patients died. Dr. Croghan didn’t follow the scientific method, he proceeded based on anecdotal evidence and a hunch. Such science, when lives are at stake, while often informative by virtue of observed results, is dangerous and irresponsible.

Let’s look at the scientific method in a nutshell.

A person has a question that needs answered.

This person then runs tests and collects evidence.

Based on the evidence, a hypothesis is formed in an attempt to explain the question.

A person then tries to disprove their hypothesis, a process known as falsification. The purpose? If you have a valid hypothesis, it should be true under any tests one subjects it to.

If the hypothesis passes these tests, one then publishes it for peer review. They explain their method for coming to such conclusions, their methodology at attempting falsification, and then allow others to review it, debate it, attempt to falsify it, and/or attempt to replicate it with total consistency.

Once the hypothesis has passed all these steps, only then does it become accepted wisdom, or even accepted natural law, such as Isaac’s laws of motion.

Recently, NASA’s Gavin Schmidt went on FNC’s Stossel and made brilliant points about the research NASA has done on climate change . He laid out the testing they’ve done, explained how they eliminated other variables, and thus concluded that mankind is increasing the CO2 in the air, and that this ever-increasing CO2 production will cause detrimental climate change. Hearing him speak alone, you could be easily convinced he had done his homework and was spot on in his hypotheses. From there however, it all went wrong.

Dr. Gavin Schmidt

Good science, by definition, allows for more than one opinion, otherwise you merely have the will of one man—which is the basis of cult. ~ Quote from The Master (a movie loosely based on Scientology)

Gavin Schmidt refused to sit next to Dr. Roy Spencer, a climate change scientist himself, with proper credentials, who happens to be skeptical of the climate doomsday scenarios often portrayed by others. In doing so, violating the process of peer review and meaningful discussion. His reason? He said he wasn’t interested in being part of a political debate.

Dr. Roy Spencer

The discussion however was not about politics, it was about the science of climate change. If Gavin Schmidt is unwilling to have his science debated, he has zero business doing scientific research at all, especially on the taxpayer’s dime.

If his science is correct, there should be no fear in defending against a skeptic. Every objection the skeptic might raise should be easily explained and dismissed if Gavin has done a thorough job and come to proper conclusions. If he cannot overcome a skeptic’s objections, then guess what? That means it isn’t settled science and his work is incomplete or even possibly false.

Convincing people the Earth is round and that the sun doesn’t revolve around it took time. But barring the most ignorant of idiots, we all agree that these statements are true now.

Al Gore

Those purporting climate change need to stop sensationalizing like Al Gore, debate educated climate skeptics intelligently, and stop acting like we’re all idiots for not buying what they’re selling.

As for the politics of all this? I believe we should not bankrupt the nation based on phenomena that is still not fully understood, and legislators must recuse themselves from the debate until it is. Because much like me, they aren’t climatologists either.

Like this:

log·i·cal: capable of reasoning or of using reason in an orderly cogent fashion lib·er·tar·i·an: an advocate of the doctrine of free will; a person who upholds the principles of individual liberty especially of thought and action