Some myths concerning statistical hypothesis testing

mat mats_trash at
Wed Nov 6 14:02:59 EST 2002

"Glen M. Sizemore" <gmsizemore2 at> wrote in message news:<3dc887cd$1_4 at>...
> My apologies if this gets double posted....I gave my lame server two days to
> post it, but they loose about 30% of what I try to post. Anyway I thought
> that this would interest/anger many of you.
> Some myths concerning statistical hypothesis testing (from a recent paper
> published by Marc Branch).
> 1.) Tests of statistical significance do not provide a quantitative estimate
> of the reliability of the result.
> 2.) Tests of statistical significance do not estimate the probability that
> the results were due to chance.
> 3.) Tests of statistical significance usually do not answer a question to
> which the answer is unknown.

was there any reasoning behind these statements?  any mathematics to
back it up or did he just 'say' and 'argue' the above?  Unless he gave
a rigourous mathematical proof that the above are correct then it is
pointless arguing about them as statistics lies in the realm of
mathematics, obviously.  You don't 'discuss' mathematics.

The first assumption is incorrect even before we begin to debate. 
Statistics churns out numbers, so it is by definition quantitative. 
What those numbers actually mean is another matter.

The second point - is the argument that the procedures are incorrect
(i.e. the algorithm) or that the underlying basic assumptions are
incorrect (e.g. normal distribution).  If it is the former, then again
its rubbish, if its the latter then this argument is well known and he
presents nothing new.

What does he mean by 'answer'?  no, stats rarely gives categorical yes
or no (which is in a sense a qualitative answer, which he's previously
argued stats does give) but thats not what people expect.  Stats is
used to get a better understanding by measuring data pertinent to a
particular question with a host of well known caveats as to how the
answer can be interpreted.  Stats is very cautious about what
conclusions it allows to be drawn and never produces a 'certain'
answer either way.  Everyone knows this.

More information about the Neur-sci mailing list