# Some myths concerning statistical hypothesis testing

Glen M. Sizemore gmsizemore2 at yahoo.com
Fri Nov 8 13:22:05 EST 2002

```And from a statistics textbook I used as an undergraduate decades ago:
"...we define the region of rejection as being those sample values of a
statistic which comprise the most unexpected 5 percent of the values,
assuming the [the null hypothesis; the author, of course, writes it
Hsubscript0] to be true."

"Sturla Molden" <sturla at molden_dot_net.invalid> wrote in message
news:pan.2002.11.08.13.18.06.391156.1184 at molden_dot_net.invalid...
> On Thu, 07 Nov 2002 00:22:18 +0100, Glen M. Sizemore wrote:
>
>
> > GS: You really should learn to be patient. The first is not an
> > assumption, it is a fact. A p-value expresses a conditional probability.
> > That is, a p-value expresses the probability of obtaining the
> > observation in question GIVEN THAT THE NULL HYPOTHESIS IS TRUE.
>
> No it does not. This would be the likelihood of the null hypothesis.
> The p-value is the probability of getting at least as impressive
> data by fluke. The p-valueis not the probability of getting equally
> impressive data by fluke.
>
> I.e. a p-value DOES NOT express "the probability of obtaining the
> observation in question GIVEN THAT THE NULL HYPOTHESIS IS TRUE."
>
> This means that the p-value is not the "statistcial likelihood" of
> the null hypothesis. A fundamental statistical theorem is the likelihood
> principle that can be interpreted as "evidence is proportional to
likelihood".
> As the p-value is not proportional to likelihood, it cannot be
proportional
> to evidence. And this is where classical statistics fail.
>
> A number of problems raise from this. A well-known problem is that of
> stopping rules in clinical trials: A severe ethical dilemma is introduced
> simply by using a flawed measure of evidence.
>
>
>
> > GS: Wrong. Remember that a p-value represents the probability that one
> > will observe certain data given that the null hypothesis is true. If one
> > asserts that the p-value is really the probability that the null
> > hypothesis is true given the data (which is the same thing as saying it
> > represents the probability that the observed data are "due to chance")
> > is to "reverse the conditionality."
>
> Wrong again. Bayes theorem is used to reverse conditionality in
probability
> theory (and this is what "Bayesian" tests does). However as the p-value
> is not equal to the likelihood (as you claimed it to be), it cannot be
> used to reverse the conditionality by inserting it into Bayes formula.
>
>
>
> Sturla Molden

```