AN HONEST APPRAISAL OF PEER REVIEW

Arthur E. Sowers arthures at access.digex.net
Mon Jun 17 15:15:42 EST 1996



On Mon, 17 Jun 1996, Phil Koopman wrote:

> 
> Bert Gold <bgold at itsa.ucsf.edu> wrote:
> >PAPER in FASEB Journal (1993) volume 7  pages 619-621
> >On giraffes and peer review
> >D. R. FORSDYKE
> >
> >Department of Biochemistry, Queen's University, Kingston,
> >Ontario, Canada. K7L3N6
> 
> Peer review could also happen because the people in charge of the
> money aren't able to make the decisions themselves.  For example, with
> increased specialization it is hard to be knowledgeable enough to make
> decisions across a large field of competing requests.  In my mind this
> is a significant driving force that has to be dealt with (bicameral
> review might work -- I don't know much about it).
> 
> I recently came from an industrial lab where the funding allocation
> changed from "benign" dictatorship to a group-grope model.  IMHO
> productivity plunged as people spent most of their time chasing a pool
> of money that diminished with each downsizing (sound familiar?).

Ah, does that sound familiar! And, I do presume you spelled "group-grope"
correctly? It seems like a variation on the "musical chairs" game. Less
chairs than people. And for every person that gets knocked out, they take
away two, not one, chair.
 
> There are three problems with any such system:
> 
> - How do you find and fund the important radical new idea (especially
> in the face having to sift through the chaff of poorly researched new
> ideas that are simply recastings of old tired ideas).  The best ideas
> tend to upset the entrenched Establishment.

So, you've noticed that the usual "committee" behavior has trouble
recognizing something unless its seen it before (eg. the old
Japan-copy-the-USA effect in industry). And, conversely, if it has not
seen it before, then its from mars and should be destroyed instead of
nurtured (eg. I remember the historical stories about inventors of the
steamboat, electric motors who could not get financing from bankers)

> - How do you break the feedback of a dead-end research topic that has
> gained momentum (and thus is supported by researchers all praising the
> others' work)?

It litterally has to be starved out of existence by brute force
competition AND the appearance of new ideas that are lucky enough to just
get instant recognition and popularity.

> - *ANY* system is capable of being "gamed", and will be so in the face
> of inadequate resources.

And the game can be played with proper packaging (i.e. propaganda),
bribes, contacts & familiarity with the "right people", and other curried
favors.

>  How do you minimize the resources spent
> gaming the system?

Change your project. 

> IMHO a system that tends to bring dirty laundry
> out into the open is the best way to go, although no system is
> perfect.

This has some merit. 

> -- Phil
> 
> 
> Phil Koopman -- koopman at cs.cmu.edu -- http://www.cs.cmu.edu/~koopman
> 
> 

Art Sowers





More information about the Bioforum mailing list