A p-value tells us how likely we would be to see a difference this large or larger between the experimental and control groups just by chance, assuming the treatment has no real effect.
A p-value of .05 or less (1 in 20) is rare enough that psychologists stop blaming luck and call the result statistically significant.
A very large sample can make even a tiny difference statistically significant, so researchers also measure effect size to see whether the effect is big enough to matter.
The p-value and effect size answer different questions: only the p-value decides statistical significance, while effect size tells how big the effect is.
Descriptive statistics describe the people actually tested, while inferential statistics, like the p-value, help researchers infer whether results apply to the larger population.
Key terms
p-value
The probability that luck alone would produce a gap between groups as big as the one found.
statistically significant
A result with a p-value of .05 or less, meaning luck alone would produce it only 1 time in 20 or less.
effect size
A measure of how big an effect actually is: 0.2 or below is small, 0.3-0.7 medium, 0.8 or greater large.
descriptive statistic
A statistic that only describes the people who were tested, such as the gap between two groups.
inferential statistic
A statistic, such as a p-value, that helps researchers infer whether a result applies to the larger population.
Watch out
Statistically significant does not mean large or important: a low p-value says the result probably isn't luck; effect size says how big it is.
AP exam: you won't calculate p-values or effect sizes, but you must know what they mean and the effect size benchmarks.