# Some myths concerning statistical hypothesis testing

Sturla Molden sturla at molden_dot_net.invalid
Sat Nov 9 12:12:01 EST 2002

On Fri, 08 Nov 2002 20:04:27 +0100, Glen M. Sizemore wrote:

> Sorry, but the p-value IS the conditional probability of observing
> the data given that the null hypothesis

The p-value is not the conditional probability of getting the observed
data under the null hypothesis. Let me provide a simple proof for this:

Assume that you want to test if a dice is fair. Then you have:

H0:  Prob of getting 6 is 1/6
HA:  Prob of getting 6 is not 1/6

Now you throw the dice 10 times and observe a 6 on five of the trials.
Then the conditional probability of getting the observed result under
given that the nullhypotheis is true is:

Prob(data | H0) = Likelihood of H0
= Prob(x = 5) with x ~ binom(10, 1/6)
= 0.0130

However, the p-value is:

p-value = Prob(x = 5) + Prob(x = 6) + Prob(x = 7) + Prob(x = 8)
+ Prob(x = 9) + Prob(x = 9) + Prob(x = 10) with x ~ binom(10, 1/6)
= 0.0155

Thus in this case

p-value is not equal to Prob(data | H0),

which proves you wrong.

You can present this to any statistican you choose for verification.

Sturla Molden

(crossposted to sci.stat.math)