Case AGAINST peer review

Alexander Berezin berezin at MCMAIL.CIS.MCMASTER.CA
Sat Jun 22 16:12:35 EST 1996



On Fri, 21 Jun 1996, W. R. Gibbons wrote:

> On Sat, 15 Jun 1996, Bert Gold wrote:
> OK, I guess he forwarded: 
> > 
> > PAPER in FASEB Journal (1993) volume 7  pages 619-621
> > 
> > On giraffes and peer review
> > 
> > D. R. FORSDYKE

> <more deletia, then the stuff I disagreed with:>

(FORSDYKE): 
> > designers of the peer-review system failed to do. Two principles
> > of decision-making in uncertain environments are, (i) place most
> > weight on parameters which can be assessed objectively, and (ii)
> > hedge your bets. A design based on these principles, named
> > bicameral review, has been presented elsewhere 18,19. Grant
> > applications are divided into a major retrospective part and a
> > minor prospective part, which are routed separately. The
> > retrospective part (track record), is subjected to peer review.
> > The prospective part (proposed work) is subjected to in house
> > review by the agency, solely with respect to budget
> > justification. 

(GIBBONS): 
> There's a serious problem in this.  One thing the grantsmen have figured 
> out very, very well is that you must not only get the grant, but publish 
> papers.  Lots of papers.  The more papers the better, because if you 
> publish enough you will overwhelm the reviewers who have no time to read 
> all that you have written, to see if those papers are in fact any good.

BEREZIN:
In a way, yes, but here we came to the crux of the problem.
Why 'many papers' have a lot of weight ? 
Because they are 'validated' by the peer review aura.
(despite that by 'Sturgeon law' 90 % of everything 
published is garbage anyway).

Now drop ALL peer review. Publish everything
(EVERY thing). No problem, present technology 
allows it easily at a very reasonable cost
(like $ 200 - 300 per paper).

 Even more garbage ? No !!! The major initiative 
to publish as much as possible disappears, now
100 papers in 5 years is actually likely more 
suspicious than 10. And if track reviewers are
asked to evaluate what the researcher _really_
discovered (NOT how many papers and were
published, and 'how presegious' journals are), 
than the bicameral review works is a very 
sensible alternative.
(Less garbage, more science)

Non-peer reviewing of papers:

(1) remove their [ ficional ] value as a career
currency

(2) will improve speed of resaerch communications

(3) save a lot of time for reviewers AND readers 

> 
> I've argued on a study section that an applicant whose work I knew all too
> well had contributed very little to his field.  Others pointed to an
> impressive stack of papers and manuscripts--100 papers over 5 years--and
> said, "Look! How can you say that?"  I explained I had read, reviewed, and
> managed the review of many of those papers, and they were not significant
> works.  The author wrote reams of uncritical rubbish, and just kept
> submitting each paper to lesser and lesser journals until one accepted it.
> I had seen some papers once as an editor, and two or three times as a
> potential reviewer.  The only response I could get was, "He is *very*
> productive; he deserves to be funded." 

Dr. Gibbons confirms what I said above. 

> 
> So to some extent the retrospective analysis operates now, but badly. 
> Output is weighed, or counted (and counted in a silly way in which each
> author of an 8-author paper is credited with a full publication).  Output
> is rarely critically assessed.  If the proposed system were inagurated, it
> would in my opinion exacerbate problems if it did not address the question
> of what it means to be productive. 

Disengage paper counting from the track evaluation. 
That's all you need for bicamearl scheme to work. 
(# of papers and research significance are more
often than not in an INVERSE relationship anyway). 

 
>    Ray Gibbons  Dept. of Molecular Physiology & Biophysics
>                 Univ. of Vermont College of Medicine, Burlington, VT
>                 gibbons at northpole.med.uvm.edu  (802) 656-8910
> 
> 
> 



More information about the Bioforum mailing list