Really Simple Statistics: p values #MRX

Welcome to Really Simple Statistics (RSS). There are lots of places online where you can ponder over the minute details of complicated equations but very few places that make statistics understandable to everyone. I won’t explain exceptions to the rule or special cases here. Let’s just get comfortable with the fundamentals.

What is a p value?

P value is a short form for probability value and another way of saying significance value. It refers to the chance that you are willing to take in being wrong. (I know, once in your life is too many times to be wrong.)

No matter how careful you are, random chance plays a part in everything. If you try to guess whether you’ll get heads or tails when you flip a coin, your chance of guessing correctly is only 50%. Half the time, you’ll flip tails even if you wanted to flip heads.

In research, we don’t like 50/50 odds. We instead only want to risk that 5% or 1% of our predictions are wrong. And, if you just picked 1% or 5%, you’ve just picked a peck of picked peppers. Whoops, I mean you’ve just picked a p value.

P values are almost always expressed out of 1. For example, a p value of 0.05 means you are willing to let 5% of your predictions be wrong. A p value of 0.1 means you are willing to let 10% of them be wrong. Don’t let that pesky decimal place fool you. A p value of 0.01 means 1% and a p value of 0.1 means 10%.

When you do a statistical test in software like SPSS or Systat, it will tell you the exact p value associated with your specific set data. For instance, it might indicate that the p value of your result is 0.035, or “Men are significantly taller than women, p=0.035.” That means there is a 3.5% chance that men are NOT actually taller than women and this result happened only because of random chance.

Post navigation

10 responses

I’m still confused by what the p value is supposed to represent. With a coin flip, shouldn’t the p value be 0.5, not 0.05?

“In research, we don’t like 50/50 odds. We instead only want to risk that 5% or 1% of our predictions are wrong.” – THAT’S the leap that all the text books and websites make that I don’t get! With a coin toss that 50/50 should stay at 50%, not 5%.

Because really, if a p value of 0.05 is the standard, does that mean to say that researchers will reject some result (of any particular study) if it it turns up as much as, say, 93% of the time? Like, if a particular medication works on anything less than 95% of the participants then it would be rejected? Sounds pretty extreme to me.

PLEASE clear this up for me as I have an assignment and exam in a few weeks!!!
Cheers

Tossing a coin one single time isn’t fair. It won’t tell you if you’ve got a bum coin. BUT,toss a coin 20 times and you’ll know ‘for sure’ if you’ve got a bum coin. A fair coin will probably come up 10 heads and 10 tails. BUT, if you get 1 head and 19 tails, you’re going to be suspicious that something is weird about that coin. That’s 1 in 20 or 5% or p=0.05. That is what we mean.

As for 93% probability, researchers aren’t THAT strict. It’s the technical rule, the letter of the law. But researchers know to use the spirit of the law. If they see 93%, they’d probably follow up with a second study to see if they got the same result again. If they got p=0.5, they’d probably ignore the result as something that happened just due to chance. BUT, if they got p=0.07 or p=0.03, they’d think they found something.

Great post. For technicians, I suppose the corrections made by Mans and Kevin McConway are correct, but this post is perfectly in line with the goal communicated at the top of the page: getting comfortable with the fundamentals.

Hi there. Correlations and p-values are two different things that go together. A correlation is a test of relationships between variables. (Other tests of relationships would be t-tests, chi-squares, anovas, etc.) A p-value tells you if the relationship is strong enough to pay attention to. Does that help? Good luck my dear stats friend :)

“That means there is a 3.5% chance that men are NOT actually taller than women and this result happened only because of random chance.”

No. There’s always a great need to be careful when you discuss the meaning of p-values. The p-value is not the probability that the hypothesis is true!

The p-value is the probability to obtain a result as extreme (in the direction of the alternative) as the observed outcome if the null hypothesis is true. “If men are not taller than women then we would only see a difference that is at least this large 3.5 % of the times we repeated this experiment with the same number of men and women.” That’s unfortunately far more difficult to interpret than what you wrote above, but it is the correct interpretation.

I really appreciate what you are doing here and the RSS posts are a great idea. Just trying to make sure that you don’t go from “simple statistics” to “wrong”. Keep up the good work! :)

You say: ‘For instance, it might indicate that the p value of your result is 0.035, or “Men are significantly taller than women, p=0.035.” That means there is a 3.5% chance that men are NOT actually taller than women[…]’

For those who want to get into the nitty gritty, this comment is for you. :)

Hi and welcome!

I'm a market research methodologist who blogs about sampling, surveys, statistics, charts, and more. My goal is to keep research real, current, and fun. I approve and post every single comment that makes it through my spam filter, usually within 24 hours.

I am the Editor-In-Chief of MRIA's Vue magazine, Canada's market research magazine. If you'd like to be an author, I'd love to publish your article. Release your inner expert!