Peer Review (Jan 30/96)

[Alexander Berezin]

A colleague provided some comments on PR (Peer Review),
combined from few other responses. I post my summarily
reply below. Alex Berezin  

> Alexander,
> 

> > 
> > >There has been some talk on discussions about the pro's and con's of
>  PR (Peer Review)

> [snip]
> > >far as I know.
> > >
> > >Have fun with it, and feel free to blast me directly by E-mail.
> > 
> > This is important enough that you should be blasted on net rather than
> > just by direct e-mail. The temptation is great to shortcut the
> > slow publishing process of the regular journals, particularly the
> > paper-based ones (still the majority and the only ones usually recognized
> > by Academia). But we shouldn't yield to it, at least not in this way!
> > 
> > The promise and threat of Internet and the Web is that anyone can put
> > anything on it instantly. A lot of people do, as we know (sigh!). So why
> > shouldn't we put all our scientific manuscripts there as soon as they are
> > in a readable state rather than wait nine months to three years before we
> > can get it published? I feel base to bring up the old
> > "what-if-everyone-did-the-same" argument, but this is exactly the problem:
> > if individual dumping of manuscripts became the common means of spreading
> > scientific results, then premature, erroneous, or simply falsified results
> > would be an immeasurably more serious problem than it is today. There are a
> > lot of things to say about the present peer-review system, but its
> > function, to keep the noise level bearably low, is important. As long as we
> > have not come up with something better, we should stay with this system and
> > try to improve rather than bypass it.
> 
> [snip]
> > 
> > (1) Direct submission of research results to the net without peer review
> > or similar quality control will continue to be suspect and, as a 
>> practice,
> > will generate more noise and disinformation than anything else.
> > 
> There seem to be some folks that are very seriously suggesting that the
> peer review system especially in an anonymous form does more harm than
> good.  One articulate and noisy chap is Alexander Berezin who has been
> posting to bionet.general and elsewhere.  I tend to think that in spite
> of its flaws it is better than the suggested alternatives.  Yet
> Alexander's and other argument is powerful enough that it makes one stop
> and think.  I suspect the problems may also be worse in area where a lot
> of grant money and competing labs allow folks to nip the competition. 
> 
> Are there some folks out there who have followed this topic more than
> me?  Can Alexander's arguments be dismissed?  
> 
> One thing is for sure articles on the internet make it easier for prof's
> at smaller liberal art colleges to get the information.  You don't have
> to  have your institution pay thousands for journals in your area or
> take off weeks in the summer to go viist a major library. But that is a
> totally different topic. I appreciate being able to download the
> manuscript. 
>  

BEREZIN:

The argument that the prime function of PR is a noise filter
USED to be of some (perhaps even strong) merit in earlier decades. 

However, it is no longer so, and PR (mostly as APR, anonymous PR)
is now mostly held by inertia by the extremely cumbersome,
unflexible and conservative reward system in academia and 
coupled with huge (multi-billion dollar) interests of 
academic publishers worldwide. 

With a growth of e-means and (equally importantly !)
much greater recent sophistication of abstracting services, 
the above role of APR (as an alleged 'quality filter') is 
getting more and more irrelevant and (simultaneously) its 
function as a censhorship/supression factor is
increasing. There are some good (long) texts on this
phenomenon and I am abstaining from more arguing on this
here (list of references on peer review, inevitably 
incomplet is attached in the end of this message).

In my opinion, the point of equilibrium (when the noise 
filtering and innovation-suppressive functions
of APR more-or-less equated each other), was passed 
a few years ago. Now we are QUICKLY moving into the
realm when APR beeing largley regressive phenomenon. The
main function of it now is to maintain (to validate)
the power, authority and prestige structures in
science, to serve as a frame of reference for the
establishement. 

This, I believe, is the sufficient reason to mount 
a massive opposition to APR and move towards what Darrel 
Chubin in his book (1992) has called PEERLESS SCIENCE. Of 
course, well established members of grantsmanship elite 
are bound to fiercely oppose dismantling of APR, but because 
the prime driver of science comes from (mostly young and 
unestablished) talented risk takers, I hope that the fat 
cats are goining to loose the battle and in the next 
few years we are going to see collapse of APR and its
(gradual) replacement by the system of interactive 
and comments from the reader (true or OPEN peer 
review).

Those who do genuimnly care about the progress of science
and intellectual discourse, should not be afraid of this
process and should be encouraged to contribute their efforts to 
dismantling of the obsolete, irrelevant and anti-intellectul 
system of APR. Science has nothing to loose from scrapping 
of APR and likely someting tangible to gain (easier
criticism of established dogms, for example).

So, the long kept secret is now coming to light: there is 
nothing (I repeat, NOTHING) really valuable behind APR, 
quoting old good H.-Ch. Andersen THE EMPEROR HAS NO CLOTHS.         
 
SOME REFERENCES ON PEER REVIEW AND FUNDING MODELS

Berezin, A. A. (1993). The SSC and peer review. Physics World 
(Dec.), 19. 

Berezin, A. A., R. Gordon & G. Hunter (1995). Anonymous peer   
     review and the QWERTY effect. Amer. Physics Soc. News,  
     March 1995. 

Berezin, A. A. & G. Hunter (1994). Myth of competition and NSERC
     policy of selectivity. Canadian Chemical News  46(3), 4-5. 

Forsdyke, D. R. (1983). Canadian medical research strategy for    
     the Eighties I. Damage-limitation or superelitism? Med.      
     Hypotheses  11, 141-145. 

Forsdyke, D. R. (1983). Canadian medical research strategy for    
     the Eighties II. Promise or performance as the basis for the 
     distribution of research funds? Med. Hypotheses  11,         
     147-156. 

Forsdyke, D. R. (1989). Sudden-death funding system. FASEB J.     
     3(10), 2221. 

Forsdyke, D. R. (1989). A systems analyst asks about AIDS         
     research funding. Lancet  2(December 9), 1382-1384. 

Forsdyke, D. R. (1991). Bicameral grant review: an alternative to 
     conventional peer review. FASEB J.  5, 2312-2314. 

Forsdyke, D. R. (1992). Bicameral grant review: how a systems     
     analyst with AIDS would reform research funding.
     Accountability in Research  3, 1-5. 

Forsdyke, D. R. (1993). On giraffes and peer review. FASEB J.  7, 
     619-621. 

Forsdyke, D. R.(1994). Authorship and misconduct. Nature 370, 91. 

Forsdyke, D. R. (1994). A theoretical basis for accepting         
     undergraduate academic record as a predictor of subsequent   
     success in a research career. Implications for peer review.  
     Accountability in Research  3, 269-274. 

Gordon, R. (1993). Grant agencies versus the search for truth. 
     Accountability in Research: Policies and Quality Assurance   
     2, 1-5. 

Gordon, R. (1993). Alternative reviews. University Affairs        
     (Assoc.of Universities and Colleges of Canada) 34(6), 26. 

Horrobin, D. (1981/1982). Peer review: Is the good the enemy of   
     the best?  J. Res. Communic. Stud.  3, 327-334. 

Horrobin, D. F. (1990). The philosophical basis of peer review    
     and the suppression of innovation. J. Amer. Med. Assoc.      
     263(10), 1438-1441. 

Kenward, Michael. (1984). Peer review and the axe murderers",
     New Scientist, 102 (1412), p. 13 (31 May, 1984). 

McCutchen, Charles W. (1991). Peer Review: Treacherous Servant,
     Disastrous Master. Technology Review, vol. 94, #7,  
     (October 1991), 28-40.

Osmond, D. H. (1983). Malice's Wonderland: research funding and   
     peer review. J. Neurobiol.  14(2), 95-112.

Savan, Beth. (1990). Science Under Siege (The Myth of             
     Objectivity in Scientific Research, CBC Enterprises,         
     Toronto, 1988. 

Szent-Gyorgyi, Albert. (1972). Dionysians and                     
      Apollonians, Science, 176, 966 (1972).
               
----------------------------------------------------------