On Fri, 21 Jun 1996, W. R. Gibbons wrote:
> On Sat, 15 Jun 1996, Bert Gold wrote:
> OK, I guess he forwarded:
> >
> > PAPER in FASEB Journal (1993) volume 7 pages 619-621
> >
> > On giraffes and peer review
> >
> > D. R. FORSDYKE
> <more deletia, then the stuff I disagreed with:>
(FORSDYKE):
> > designers of the peer-review system failed to do. Two principles
> > of decision-making in uncertain environments are, (i) place most
> > weight on parameters which can be assessed objectively, and (ii)
> > hedge your bets. A design based on these principles, named
> > bicameral review, has been presented elsewhere 18,19. Grant
> > applications are divided into a major retrospective part and a
> > minor prospective part, which are routed separately. The
> > retrospective part (track record), is subjected to peer review.
> > The prospective part (proposed work) is subjected to in house
> > review by the agency, solely with respect to budget
> > justification.
(GIBBONS):
> There's a serious problem in this. One thing the grantsmen have figured
> out very, very well is that you must not only get the grant, but publish
> papers. Lots of papers. The more papers the better, because if you
> publish enough you will overwhelm the reviewers who have no time to read
> all that you have written, to see if those papers are in fact any good.
BEREZIN:
In a way, yes, but here we came to the crux of the problem.
Why 'many papers' have a lot of weight ?
Because they are 'validated' by the peer review aura.
(despite that by 'Sturgeon law' 90 % of everything
published is garbage anyway).
Now drop ALL peer review. Publish everything
(EVERY thing). No problem, present technology
allows it easily at a very reasonable cost
(like $ 200 - 300 per paper).
Even more garbage ? No !!! The major initiative
to publish as much as possible disappears, now
100 papers in 5 years is actually likely more
suspicious than 10. And if track reviewers are
asked to evaluate what the researcher _really_
discovered (NOT how many papers and were
published, and 'how presegious' journals are),
than the bicameral review works is a very
sensible alternative.
(Less garbage, more science)
Non-peer reviewing of papers:
(1) remove their [ ficional ] value as a career
currency
(2) will improve speed of resaerch communications
(3) save a lot of time for reviewers AND readers
>> I've argued on a study section that an applicant whose work I knew all too
> well had contributed very little to his field. Others pointed to an
> impressive stack of papers and manuscripts--100 papers over 5 years--and
> said, "Look! How can you say that?" I explained I had read, reviewed, and
> managed the review of many of those papers, and they were not significant
> works. The author wrote reams of uncritical rubbish, and just kept
> submitting each paper to lesser and lesser journals until one accepted it.
> I had seen some papers once as an editor, and two or three times as a
> potential reviewer. The only response I could get was, "He is *very*
> productive; he deserves to be funded."
Dr. Gibbons confirms what I said above.
>> So to some extent the retrospective analysis operates now, but badly.
> Output is weighed, or counted (and counted in a silly way in which each
> author of an 8-author paper is credited with a full publication). Output
> is rarely critically assessed. If the proposed system were inagurated, it
> would in my opinion exacerbate problems if it did not address the question
> of what it means to be productive.
Disengage paper counting from the track evaluation.
That's all you need for bicamearl scheme to work.
(# of papers and research significance are more
often than not in an INVERSE relationship anyway).
> Ray Gibbons Dept. of Molecular Physiology & Biophysics
> Univ. of Vermont College of Medicine, Burlington, VT
>gibbons at northpole.med.uvm.edu (802) 656-8910
>>>
