# Testing Models against a tree (help)

Mary K. Kuhner mkkuhner at phylo.genetics.washington.edu
Tue Jul 25 22:02:26 EST 1995

```In article <1995Jul25.213229.40081 at ac.dal.ca> aroger at ac.dal.ca writes:

>Under model A I could construct a tree from the data and measure the
>tree to tree distance from the "known" tree of the organisms.  I could
>then construct a tree under model B from the data and derive the
>tree to tree distance of this tree compared to the known tree.  If
>model B yields a tree closer to the true tree (smaller tree to tree
>distance) then I suggest that this model is more realistic.  The
>only problem I can see is whether the tree under model B is
>significantly closer to the true tree than the tree under model A--
>ie I need a statistic attached to the tree to tree distance measures.

>Andrew Roger
>aroger at ac.dal.ca

A cautionary tale:  One group at the Evolution Society
meetings had a puzzling result from a simulation--a method usually
thought to be unreliable was getting the right tree every single
time.  On investigation, they found that their true tree was
perfectly adapted to this method's prejudices--was, in fact, the tree
that this method would have gotten even if it hadn't been true....
So there are cases in which a wrong model will give the true tree more
often than the right model.  (Hopefully such cases are not common.)

This doesn't mean it's not worth trying:  just risky.

You will have to pick a measure of tree distance carefully.  The
symmetrical distance (number of groups appearing in one tree but not the
other) is very sensitive to *which* taxon is misplaced.  Two methods
might each misplace one taxon, compared to the true tree, but have very
different distances.

You might be able to come up with a statistic using bootstrapping.  How
often does model B produce a tree at least as bad as model A's tree?
If it produces a tree that bad less than five percent of bootstrap
replicates, perhaps that is "significantly worse".

Mary Kuhner mkkuhner at genetics.washington.edu

```