open-source software for bioinformatics (was Re: Unix vs Linux - the movie.)
Richard P. Grant
rgrant at netscape.net
Tue Aug 8 03:29:55 EST 2000
In article <398ef9cd$1 at news.ucsc.edu>, karplus at cse.ucsc.edu (Kevin
> No one is going around checking that the lab protocols described in
> papers are precisely executed---it is assumed that the experimenters
> have normal competence.
I humbly submit that the analogy is inadequate. The protocols we use
are generally well-used, well-documented, and if the person using them
has made an error then it should become clear by inconsistencies in the
results ( hah, can we do relevant controls, then? ).
OTOH, if someone invents a *new* method, upon which they are basing a
piece of experimental work, then the new method should be vigorously and
thoroughly reviewed, and replicated in other labs. And, where possible,
the data needs to be verified using established methods.
> Nor is it required that someone publishing a
> sequence submit their sequencer hardware and software to peer review.
> Why are people advocating an impossibly higher standard of peer review
> for bioinformatics publications?
Maybe they should. Has anyone checked the algorithms and the code used
in the (e.g.) ABI software? Why not? - there are countless mistakes in
the published sequence databases.
If it really is unfeasible for a group of reviewers to take apart the
code, then at the very least, during the review process, source code
(and binaries, maybe) should be made available to the reviewers so that
they can test the new application(s)/library/whatever under 'laboratory
conditions' - e.g. using their own datasets which should give a known
answer. At the very least.
Remember, by writing a program you are creating a reagent that - in most
cases - is liable to be distributed within the scientific community
(licence/$$$ notwithstanding), and as such there needs to be some kind
of 'guarantee' that it damn' well does what it says on the box!
> -I would
> certainly refuse a 60-100-hour comittment to read a program.
I think that the editor of the medium where the program was to be
reported/published should have a better model than that of peer-reviewed
journals. (Let's not get into the argument of how many papers are
actually reviewed thoroughly, hey?)
Anyway, I'm an amateur, so go gently with me :-)
Richard P. Grant MAD Phil http://www2.mrc-lmb.cam.ac.uk/personal/rpg/
Structural Studies http://www.scienceboard.net/
MRC-LMB Please reply to rpg 'at' mrc-lmb.cam.ac.uk
More information about the Bio-soft