AN HONEST APPRAISAL OF PEER REVIEW
Arthur E. Sowers
arthures at access.digex.net
Mon Jun 17 15:15:42 EST 1996
On Mon, 17 Jun 1996, Phil Koopman wrote:
> Bert Gold <bgold at itsa.ucsf.edu> wrote:
> >PAPER in FASEB Journal (1993) volume 7 pages 619-621
> >On giraffes and peer review
> >D. R. FORSDYKE
> >Department of Biochemistry, Queen's University, Kingston,
> >Ontario, Canada. K7L3N6
> Peer review could also happen because the people in charge of the
> money aren't able to make the decisions themselves. For example, with
> increased specialization it is hard to be knowledgeable enough to make
> decisions across a large field of competing requests. In my mind this
> is a significant driving force that has to be dealt with (bicameral
> review might work -- I don't know much about it).
> I recently came from an industrial lab where the funding allocation
> changed from "benign" dictatorship to a group-grope model. IMHO
> productivity plunged as people spent most of their time chasing a pool
> of money that diminished with each downsizing (sound familiar?).
Ah, does that sound familiar! And, I do presume you spelled "group-grope"
correctly? It seems like a variation on the "musical chairs" game. Less
chairs than people. And for every person that gets knocked out, they take
away two, not one, chair.
> There are three problems with any such system:
> - How do you find and fund the important radical new idea (especially
> in the face having to sift through the chaff of poorly researched new
> ideas that are simply recastings of old tired ideas). The best ideas
> tend to upset the entrenched Establishment.
So, you've noticed that the usual "committee" behavior has trouble
recognizing something unless its seen it before (eg. the old
Japan-copy-the-USA effect in industry). And, conversely, if it has not
seen it before, then its from mars and should be destroyed instead of
nurtured (eg. I remember the historical stories about inventors of the
steamboat, electric motors who could not get financing from bankers)
> - How do you break the feedback of a dead-end research topic that has
> gained momentum (and thus is supported by researchers all praising the
> others' work)?
It litterally has to be starved out of existence by brute force
competition AND the appearance of new ideas that are lucky enough to just
get instant recognition and popularity.
> - *ANY* system is capable of being "gamed", and will be so in the face
> of inadequate resources.
And the game can be played with proper packaging (i.e. propaganda),
bribes, contacts & familiarity with the "right people", and other curried
> How do you minimize the resources spent
> gaming the system?
Change your project.
> IMHO a system that tends to bring dirty laundry
> out into the open is the best way to go, although no system is
This has some merit.
> -- Phil
> Phil Koopman -- koopman at cs.cmu.edu -- http://www.cs.cmu.edu/~koopman
More information about the Bioforum