Peer review & grants

Alexander Berezin berezin at MCMAIL.CIS.MCMASTER.CA
Wed Jun 7 12:15:35 EST 1995

the following are some quotes from Rustum Roy
(Penn State), 
"Alternative to Peer Review" (reply to critics),
Science, 212: 1336-1338 (19 June 1981).

...  Two egregious but very common errors ... need to
be laid to rest:

     First is the notion that the peer review system is 
in some mysterious way linked with the progress of science
and is responsible for the "strongest research system 
in the world" [american system].

     Second, that peer review is able to find or 
define "quality".  

     Since most of the fundamentals of quantum mechanics,
organic synthesis, and DNA structure managed to be
disscovered without the blessing of peer review, the
basis claim is without foundation ...

... [ some critics ] make unsupported claims that "peer
review" is actually able to predict the quality
of _RESERACH NOT YET DONE_. To the contary, as every
journal editor knows, we cannot even judge the quality
of _COMPLETED WORK_ by peer review. Three sets of three 
reviewers can give mutually conflicting results. Physical
Review Letters (45, 1605, 1980) has explicitly acknowledged
these difficulties in its historic about-face on peer
review of papers.    

... [ despite (unsupported) claims there are no strong 
arguments that ] ... peer review process does better
in producing [good] research than even, say, a lottery
selection among qualified applicants.

[ Supporters of the "peer review grant selectivity" often
refer to ] ... statistical studies carried out on large
number of proposals to establish that _ON THE AVERAGE_ the
system is fair. One wonders if a plaintiff in a robbery
case would be satisfed by a similar statistical argument
that, _on the average_ , no one was robbed in New York.


More information about the Bioforum mailing list