open-source software for bioinformatics (was Re: Unix vs Linux - the movie.)

Richard P. Grant rgrant at netscape.net
Tue Aug 8 03:29:55 EST 2000


In article <398ef9cd$1 at news.ucsc.edu>, karplus at cse.ucsc.edu (Kevin 
Karplus) wrote:

> No one is going around checking that the lab protocols described in
> papers are precisely executed---it is assumed that the experimenters
> have normal competence.  

I humbly submit that the analogy is inadequate.  The protocols we use 
are generally well-used, well-documented, and if the person using them 
has made an error then it should become clear by inconsistencies in the 
results ( hah, can we do relevant controls, then? ).  

OTOH, if someone invents a *new* method, upon which they are basing a 
piece of experimental work, then the new method should be vigorously and 
thoroughly reviewed, and replicated in other labs.  And, where possible, 
the data needs to be verified using established methods.

> Nor is it required that someone publishing a
> sequence submit their sequencer hardware and software to peer review.
> Why are people advocating an impossibly higher standard of peer review
> for bioinformatics publications?

Maybe they should.  Has anyone checked the algorithms and the code used 
in the (e.g.) ABI software?  Why not? -  there are countless mistakes in 
the published sequence databases.

If it really is unfeasible for a group of reviewers to take apart the 
code, then at the very least, during the review process, source code 
(and binaries, maybe) should be made available to the reviewers so that 
they can test the new application(s)/library/whatever under 'laboratory 
conditions' - e.g. using their own datasets which should give a known 
answer.  At the very least.

Remember, by writing a program you are creating a reagent that - in most 
cases - is liable to be distributed within the scientific community 
(licence/$$$ notwithstanding), and as such there needs to be some kind 
of 'guarantee' that it damn' well does what it says on the box!

Earlier:

> -I would
> certainly refuse a 60-100-hour comittment to read a program.

I think that the editor of the medium where the program was to be 
reported/published should have a better model than that of peer-reviewed 
journals.  (Let's not get into the argument of how many papers are 
actually reviewed thoroughly, hey?) 

Anyway,  I'm an amateur, so go gently with me :-)

Best,

Richard

-- 
Richard P. Grant MAD Phil       http://www2.mrc-lmb.cam.ac.uk/personal/rpg/
Structural Studies              http://www.scienceboard.net/
MRC-LMB                         Please reply to rpg 'at' mrc-lmb.cam.ac.uk
                                







More information about the Bio-soft mailing list