Peer Review (fwd)

[Alexander Berezin]

---------- Forwarded message ----------
Date: Mon, 30 Oct 1995 10:33:00 -0500 (EST)
From: Alexander Berezin <berezin at mcmail.CIS.McMaster.CA>
To: Ashok Grover <groverak at fhs.csu.mcmaster.ca>
Cc: biocan at net.bio.net
Subject: Re: Peer Review


On 30 Oct 1995, Ashok Grover wrote:

> Three points:
> 
> (a)	The rejection of the manuscript of Gregor Mendel's paper by 
> Naggli did not occur by an anonymous peer review but in a signed letter.  

Fine. I never claim that abolishing of anonymity in PR will 
solve all the problems and bring a paradise. But it will
(in my opinion) make things significantly better. And as 
you know, Mendel (in a long run) have made it.  

> (b)	The journal Cell Calcium uses signed reviews.  However, I do not 
> see very many papers with greater insight in this journal than in those 
> which use anonymous peer review.

Glad to hear that the precedent exists. Unfortunately, your
observation (quality is about the same) is very difficult to
support or refute - not because I deny your insight (I don't),
but simply because an isolated opinion of this kind does not 
provide a sufficient data base.

As I said earlier: my critical experiment will be that 
(whatever) major journal run 2 sections in parallel for
several yeas (as somebody pointed out, 1 years in not 
enough).
   Section A: business as usual (anonymous PR)
   Section B: open revewies.

The submitting aothors are free to chose the option they
want their paper to be processed. This statistics (along
with a long range quality assessment differences: citation
analysis, etc) shoud be reported, so the community can
assess the results.   

> (c) 	If you feel that you do not believe in anonymous peer review, no one 
> stops you from signing your name.  I know a colleague who has a 
> stamping stating that he does not believe in anonymity of reviewers, uses 
> this stamp and signs his name on every thing he reviews.  The colleague 
> has remained successful in obtaining peer reviewed grants despite signing 
> his naem. If in deed such a populous feeling exists and more than half the
> scientists use this approach, they can effectively change the system.
> 

I have answeed the above in detail in a parallel posting to bioforum.
I do not relate the success in obtaing grants or getting papers
published with the fact that you (as a revieer) sign or not.

Alex Berezin

> 
> 
> 
>  On 28 Oct 1995, Alexander Berezin 
> wrote:
> 
> > 
> > 
> > ---------- Forwarded message ----------
> > Date: Sat, 28 Oct 1995 21:34:10 -0400 (EDT)
> > From: Alexander Berezin <berezin at mcmail.CIS.McMaster.CA>
> > To: Allan Frey <afrey at uunet.uu.net>
> > Cc: bioforum at net.bio.net
> > Subject: Re: Peer review-Faseb J. editorial
> > 
> > 
> > Dear Dr. Frey,
> > 
> > On behalf on very many (likely MANY THOUSANDS) of colleauges
> > in science who do not buy the myth that "peer review is the
> > best when it is secret" (even if many of us are FORCED, by 
> > the circumstances, to participate in this ugly ritual) let 
> > me thank you for posting your Editorial from the Faseb 
> > Journal. (and congratulations to the editors of Faseb J. 
> > for publishing it).
> > 
> > The issue which still waiting for the proper (perhaps,
> > detailed) answer is this:
> > 
> > PARADOX OF HISTORY:
> > 
> > How did it happen that the world community of scientists, 
> > if you wish an INTELLECTUAL BROTHERHOOD (and, of course, 
> > SISTERHOOD, according to polit.correct standards of today's
> > language) which undoubtedly includes many brilliant the 
> > most insightful minds this world has produced ("IF NOT 
> > THE SCIENTISTS, THAN WHO ?") can, nonetheless, build,
> > perfect (and subjugate itself to) such an utterly 
> > ANTI-intellectual and secretive, system as "Anonymous 
> > Peer Review" (APR). Why scientists THEMSELVES (no-one 
> > invented APR for them) have collectively chosen to
> > leave by gestappian standards ?
> > 
> > What do we respect more:
> > 
> >  - normally signed letters we recieve (from our friends,
> > colleagues, opponents, or even open enemies) OR:
> > 
> >  - dirty, unsigned, libelous notes squeezed under the door ?
> > (e.g. "your wife/husband was seen with..." kind of stuff)
> > 
> > Unlikely there will be too many votes in favour 
> > of the second option.
> > 
> > And yet, in science we do precisely this. By keeping 
> > its insistence to choose THE SECOND option (anonymous 
> > peer review) we declare our allegence to the squeezed
> > under the door option of which anonymous peer review
> > (APR) is a moral equivalent. 
> > 
> > Will Henkes eventually win the case in which they
> > seek to declare the anonimity in peer review illegal
> > (which it undoubtedly is from the point of view of
> > a normal sense, but not necessarilly the judicial
> > machinery) remain to be seen. But even if they fail,
> > some new (perhaps collective) legal battle(s) on this 
> > issue are well overdue.
> > 
> > Greatest scientists of the world (Euclid, Newton,
> > Descartes, Faraday, and scores of others) did not
> > see any need for APR. They would undoubtedly laugh 
> > at us if they could look at the end of 20th century, 
> > especially if they hear about such brilliant ideas 
> > as ANONYMOUS (!) peer review for ELECTRONIC (!) 
> > publications. Johnathan Swift (in Gulliver's Travel
> > to Laputa) was perhaps the very close to the correct
> > depicting of this nonsenical vanity fair.
> > 
> > Alex Berezin  
> > 
> > **********************************
> > Alexander A. Berezin, PhD
> > Department of Engineering Physics
> > McMaster University, Hamilton,
> > Ontario, Canada, L8S 4L7
> > tel. (905) 525-9140 ext. 24546
> > e-mail: BEREZIN at MCMASTER.CA
> > **********************************
> > 
> >      
> > On 28 Oct 1995, Allan Frey wrote:
> > 
> > > 
> > > In view of the continuing discussion of peer review, I thought the view
> > > of an editor of the Journal of the Federation of American Societies for
> > > Experimental Biology (FASEB J.) might be of interest.  The following is
> > > an editorial that I wrote and which recently appeared in the FASEB J.
> > > 
> > > Allan
> > > 
> > > Allan H. Frey                           email afrey at uunet.uu.net
> > > 11049 Seven Hill Lane                   voice 301.299.5181
> > > Potomac, MD 20854, USA
> > > 
> > > 
> > > The courts are considering whether the identity of peer reviewers should
> > > be secret.  Should we consider it?
> > > 
> > > Allan H. Frey
> > > 11049 Seven Hill Ln.
> > > Potomac, MD 20854  USA
> > > afrey at uunet.uu.net
> > > 
> > > It has been a fundamental tenet of science that the names of grant
> > > application and manuscript reviewers must be kept secret in order for the
> > > system to work.  But is that true?  The courts may soon force the
> > > scientific community to find out.  The first round concerning the question
> > > of secrecy has been fought in Federal District court; the second round is
> > > about to begin in a Federal Appeals Court (1).
> > > 
> > > Suit was brought in the US District Court for the District of Columbia to
> > > force the NSF to reveal the names of the reviewers who wrote the
> > > evaluations of a rejected grant application (Henke vs US Dept of Commerce
> > > and NSF).  The plaintiffs believe that some of the reviewers may have had
> > > a conflict-of-interest.  In the first round, the District Court issued a
> > > summary dismissal order based on the technical point that the reviewers
> > > were promised secrecy; consequently, the names are exempted from
> > > disclosure under the Privacy Act.  Thus, the real issue for science was not
> > > addressed.  The plaintiffs are appealing to a Federal Appeals Court.
> > > 
> > > Given the social changes that have been occurring in recent decades, it
> > > would not be surprising if a higher court found that the names should not
> > > be secret; if not in this case, in a subsequent case that will surely follow.
> > > Some people believe that the recent US Supreme Court ruling on Daubert vs
> > > Merrill Dow Pharmaceuticals will be used to subpoena secret records of
> > > granting agencies and journals.  Thus, a full debate on this secrecy issue
> > > within the scientific community at this time would be timely.  It would be
> > > much better for the scientific community to actively debate the issue, and
> > > explore and test alternatives in an orderly fashion, than to wait and find
> > > itself suddenly forced to make a precipitous change.
> > > 
> > > Through such an open debate and exploration, the scientific community
> > > may actually come up with something better.  The present system of
> > > secrecy has certainly not been without its problems. With reason, it has
> > > been argued that secrecy in peer review shields unqualified reviewers and
> > > biased editors; and invites abuse by reviewers who may have conflicts of
> > > interest, who may steal ideas and who may deliberately delay publication
> > > (2-4).
> > > 
> > > On the other hand, an apparently reasonable argument can be made that the
> > > identity of people who do peer reviews should not be revealed.  But
> > > implicit in this latter argument are assumptions that are not true in real
> > > life.  One implicit assumption is  that scientists  (reviewers) are as the
> > > public often images them,  objective, dispassionate, all-knowing beings.
> > > We know they are not.  A "peer" is  often a direct competitor for fame and
> > > funds and this influences his  actions in science as much as if he were in
> > > business or sports.  Scientists (reviewers) are  human beings with all the
> > > faults and motivations that you find in every  other field.  Once upon a
> > > time, when the body of knowledge was small, an editor could  filter out
> > > such abuses or could recognize the validity of an argument in  rebuttal.
> > > But now, the body of knowledge is so large and science is  divided into so
> > > many sub-specialities, that no editor or program manager  can prevent
> > > these abuses.
> > > 
> > > Further as Judson has pointed out "...Although peer review and refereeing
> > > seem rational, indispensable, and immutable, the histories demonstrate
> > > that they are social constructs of recent date.  They are not laws of
> > > nature, nor of epistemology."  He also points out that "... as over the years
> > > an ever-smaller proportion of grants gets funded and as the applications
> > > themselves ... are more difficult to put in any reliable rank order, politics
> > > become overt in the review process.  Rivalries between scientists,
> > > laboratories, and schools of thought emerge as palpable factors." (5).
> > > Given these facts, is secrecy wise?  Is it necessary for peer review to
> > > work?
> > > 
> > > It can, and has been argued, that a primary reason secrecy is required is
> > > that scientists will be afraid to review if their identity were revealed.
> > > There is something to this argument; but they may also feel constrained to
> > > give a fair review.  And consider the fact that the reviewer's name is
> > > always published with his review of a scientific book. Consider, also, that
> > > we already have other indications that secrecy is not necessary for peer
> > > review to work.  Scientists post mss on Internet newsgroups now and
> > > receive lively criticism from readers (reviewers) whose identity is not
> > > secret.   I put on the Internet for criticism sections of a book ms which
> > > was just published (6).  I didn't perceive anyone pulling punches by any
> > > means, and they knew I would know their identity.  Maybe the expanding
> > > use of the Internet will resolve the question for us.
> > > 
> > > As a scientist who has worked full-time in  research,  I have come to the
> > > conclusion that the identity of reviewers  should not be kept secret.  I
> > > personally tell editors and granting agencies  that they may reveal my
> > > identity to authors.  I think science and scientists are ill-served by the
> > > practice of secrecy and, ultimately, Society which benefits from and pays
> > > for scientific  research is harmed.
> > > 
> > > Consider also that this matter of secrecy is not a new problem for
> > > Society.  Society has struggled with it for a millennium in other forms and
> > > in other contexts.  This  question in science is merely its latest
> > > incarnation.  Its a matter of  balancing the pros and cons, and the balance
> > > that our Society has arrived at is incorporated in our Constitution as the
> > > clause concerned with due process.  In a criminal trial, witnesses are
> > > sometimes in fear for their life, but our Society through long experience
> > > has decided that, on balance, it best serves Society to allow the defendant
> > > to know the identity of and to cross-examine the accuser.  Why should we
> > > reviewers for journals and granting agencies be an exception to these
> > > rules that were incorporated in the Constitution after a millennium of
> > > effort to find the right balance.  In my judgment as a scientist, science
> > > and Society would be better off without the veil of secrecy.  Certainly
> > > there will be problems, just as there are now with the veil of secrecy.
> > > But on balance, with due process, science will find ways to manage as the
> > > courts have and will be better off.
> > > 
> > > References
> > > 1.  Henke vs US Dept of Commerce and the National Science Foundation.  US
> > > District Court for the District of Columbia, Civil action 94-0189
> > > 2.  Rennie, D., (1993) More peering into peer review.  JAMA  270,  2856-
> > > 2858
> > > 3.  Osmond, D. H., (1983) Malice's Wonderland: research funding and peer
> > > review.   J. Neurobiol. 14, 95-112.
> > > 4.  Broad, W. J., (1980) Imbroglio at Yale, I and II,  Science. 210, 38-41,
> > > 171-173.
> > > 5.  Judson, H. F., (1994) Structural transformations of the Sciences and the
> > > end of peer review.  JAMA. 272, 92-94.
> > > 6.  Frey, A. H., ed (1994) In "On the Nature of Electromagnetic Field
> > > Interactions with Biological Systems". R. G. Landes Co. Austin.
> > > 
> > > 
> > 
> > 
> > 
> 
>