# Some myths concerning statistical hypothesis testing

Ellen Hertz ehertz at patriot.net
Sun Nov 10 22:37:37 EST 2002

```The p-value is the probability under the null hypothesis of getting the
observed  data or data more "extreme" than the observed data.

This is a little broad and reminiscent of Potter Stewart's famous definition
of pornography. Undoubtedly, it can be made  more precise.  However, the
example with the dice makes the point. If the observed proportion of sixes
is p_hat and  HA is being handled symmetrically, a reasonable  p value would
be the probability that, with a fair die, the absolute difference between
1/6 and the proportion of sixes is *at least* abs(p_hat-1/6).

Ellen Hertz

"Sturla Molden" <sturla at molden_dot_net.invalid> wrote in message
news:pan.2002.11.09.18.11.57.699444.1187 at molden_dot_net.invalid...
> On Fri, 08 Nov 2002 20:04:27 +0100, Glen M. Sizemore wrote:
>
> > Sorry, but the p-value IS the conditional probability of observing
> > the data given that the null hypothesis
>
> The p-value is not the conditional probability of getting the observed
> data under the null hypothesis. Let me provide a simple proof for this:
>
> Assume that you want to test if a dice is fair. Then you have:
>
>    H0:  Prob of getting 6 is 1/6
>    HA:  Prob of getting 6 is not 1/6
>
> Now you throw the dice 10 times and observe a 6 on five of the trials.
> Then the conditional probability of getting the observed result under
> given that the nullhypotheis is true is:
>
>    Prob(data | H0) = Likelihood of H0
>                    = Prob(x = 5) with x ~ binom(10, 1/6)
>                    = 0.0130
>
> However, the p-value is:
>
>    p-value = Prob(x = 5) + Prob(x = 6) + Prob(x = 7) + Prob(x = 8)
>              + Prob(x = 9) + Prob(x = 9) + Prob(x = 10) with x ~ binom(10,
1/6)
>            = 0.0155
>
> Thus in this case
>
>    p-value is not equal to Prob(data | H0),
>
> which proves you wrong.
>
> You can present this to any statistican you choose for verification.
>
>
>
> Sturla Molden
>
> (crossposted to sci.stat.math)

```