Watch on YouTube

8:10

What a p-value doesn’t tell you

A p-value answers a narrower question than most readers think. It is not the probability the hypothesis is true, not the size of the effect, and not whether the effect matters to anyone. A trivial difference becomes significant in a large enough sample, and a large difference stays non-significant in a small one. That is why published p-values pile up just below 0.05 — a threshold shapes what gets written up. An effect size says how much, and a confidence interval says how precisely you know it. Report those and the yes-or-no verdict matters less.

Also covered: statistical vs clinical significance · effect size · confidence intervals · publication bias