Peer Review (fwd)

Alexander Berezin berezin at MCMAIL.CIS.MCMASTER.CA
Mon Oct 30 17:00:56 EST 1995



---------- Forwarded message ----------
Date: Mon, 30 Oct 1995 15:39:27 -0500 (EST)
From: Alexander Berezin <berezin at mcmail.CIS.McMaster.CA>
To: Ashok Grover <groverak at fhs.csu.McMaster.CA>
Cc: biocan at net.bio.net
Subject: Re: Peer Review


On Mon, 30 Oct 1995, Ashok Grover wrote:

> 
> 
> On Mon, 30 Oct 1995, Alexander Berezin wrote:
> 
> > 
> > On 30 Oct 1995, Ashok Grover wrote:
> > 
> > > Three points:

(GROVER): 
> > > (a)	The rejection of the manuscript of Gregor Mendel's paper by 
> > > Naggli did not occur by an anonymous peer review but in a signed letter.  

(BEREZIN): 
> > Fine. I never claim that abolishing of anonymity in PR will 
> > solve all the problems and bring a paradise. But it will
> > (in my opinion) make things significantly better. And as 
> > you know, Mendel (in a long run) have made it.  

(GROVER): 
> No.  Mendel did not make it.  The poor man died without his work being 
> recognized by a reviewer who was not anonymous.   
> 

BEREZIN:
I ment he made it is the same sense as Giordano Bruno
made it despite beeing burnt at stake. It is nice to
that in Mendel's (as well as in Bruno's and Galileos') 
the history at least least has the names of the retrogrades.
In thoudsands less monumental cases (anonymous peer review
of the present) the bastards get away. Lack of any real
responsibility in APR is the crux of the issue.  

(GROVER):
 > (b)	The journal Cell Calcium uses signed reviews.  However, I 
>>> do not see very many papers with greater insight in this 
>> journal than in those 
> > > which use anonymous peer review.

(BEREZIN): 
> > Glad to hear that the precedent exists. Unfortunately, your
> > observation (quality is about the same) is very difficult to
> > support or refute - not because I deny your insight (I don't),
> > but simply because an isolated opinion of this kind does not 
> > provide a sufficient data base.

(GROVER): 
> I have 11 papers in that journal and I know.

BEREZIN:
So, you don't see a good insight in the papers published
in this journal (Cell Calcium) and have yourself 
published 11 papers in it. Congratutations for such an 
honest admission on your part. Wish others follow your lead.
You are in a very big company.
  
> 
(BEREZIN): 
> > As I said earlier: my critical experiment will be that 
> > (whatever) major journal run 2 sections in parallel for
> > several yeas (as somebody pointed out, 1 years in not 
> > enough).
> >    Section A: business as usual (anonymous PR)
> >    Section B: open revewies.
> > 
> > The submitting aothors are free to chose the option they
> > want their paper to be processed. This statistics (along
> > with a long range quality assessment differences: citation
> > analysis, etc) shoud be reported, so the community can
> > assess the results.   

(GROVER): 
> > > (c) 	If you feel that you do not believe in anonymous peer review, no one 
> > > stops you from signing your name.  I know a colleague who has a 
> > > stamping stating that he does not believe in anonymity of reviewers, uses 
> > > this stamp and signs his name on every thing he reviews.  The colleague 
> > > has remained successful in obtaining peer reviewed grants despite signing 
> > > his naem. If in deed such a populous feeling exists and more than half the
> > > scientists use this approach, they can effectively change the system.
> > > 
> > 
> > I have answeed the above in detail in a parallel posting to bioforum.
> > I do not relate the success in obtaing grants or getting papers
> > published with the fact that you (as a revieer) sign or not.

(GROVER): 
> But if the desire was as populous as you make it out to be, all of these 
> guys would be signing the reviews.  It is not happening.  So where is 
> your data that most of the scientific commnity is against anonymous peer 
> review.

BEREZIN:
I don't say that the MAJORITY of the scientific community is
against APR (Anonynous peer review). Majority does not care.
Most issues which were ever brought a changes came from the
minority side. In middle ages majority new that sky is a 
crysatal sphere and only small minority guessed that stars
may be other suns in (infinite) space. Numbers on the above
issues mean very little. 

(GROVER): 
> By the way, you can extend the concept of reviewer anonymity to voter 
> confidentiality.  Each voter condemns or choses a person to be elected 
> anonymously.  When it does not happen, e.g., previously Soviet 
> nations, I am not sure that the selection is any better.  So think of 
> peer review anonymity the same way as voter confidentiality.

BEREZIN:
I have answed this in another (parallel) poster.

> 
> 
> > Alex Berezin
> > 
> > > 
> > > 
> > > 
> > >  On 28 Oct 1995, Alexander Berezin 
> > > wrote:
> > > 
> > > > 
> > > > 
> > > > ---------- Forwarded message ----------
> > > > Date: Sat, 28 Oct 1995 21:34:10 -0400 (EDT)
> > > > From: Alexander Berezin <berezin at mcmail.CIS.McMaster.CA>
> > > > To: Allan Frey <afrey at uunet.uu.net>
> > > > Cc: bioforum at net.bio.net
> > > > Subject: Re: Peer review-Faseb J. editorial
> > > > 
> > > > 
> > > > Dear Dr. Frey,
> > > > 
> > > > On behalf on very many (likely MANY THOUSANDS) of colleauges
> > > > in science who do not buy the myth that "peer review is the
> > > > best when it is secret" (even if many of us are FORCED, by 
> > > > the circumstances, to participate in this ugly ritual) let 
> > > > me thank you for posting your Editorial from the Faseb 
> > > > Journal. (and congratulations to the editors of Faseb J. 
> > > > for publishing it).
> > > > 
> > > > The issue which still waiting for the proper (perhaps,
> > > > detailed) answer is this:
> > > > 
> > > > PARADOX OF HISTORY:
> > > > 
> > > > How did it happen that the world community of scientists, 
> > > > if you wish an INTELLECTUAL BROTHERHOOD (and, of course, 
> > > > SISTERHOOD, according to polit.correct standards of today's
> > > > language) which undoubtedly includes many brilliant the 
> > > > most insightful minds this world has produced ("IF NOT 
> > > > THE SCIENTISTS, THAN WHO ?") can, nonetheless, build,
> > > > perfect (and subjugate itself to) such an utterly 
> > > > ANTI-intellectual and secretive, system as "Anonymous 
> > > > Peer Review" (APR). Why scientists THEMSELVES (no-one 
> > > > invented APR for them) have collectively chosen to
> > > > leave by gestappian standards ?
> > > > 
> > > > What do we respect more:
> > > > 
> > > >  - normally signed letters we recieve (from our friends,
> > > > colleagues, opponents, or even open enemies) OR:
> > > > 
> > > >  - dirty, unsigned, libelous notes squeezed under the door ?
> > > > (e.g. "your wife/husband was seen with..." kind of stuff)
> > > > 
> > > > Unlikely there will be too many votes in favour 
> > > > of the second option.
> > > > 
> > > > And yet, in science we do precisely this. By keeping 
> > > > its insistence to choose THE SECOND option (anonymous 
> > > > peer review) we declare our allegence to the squeezed
> > > > under the door option of which anonymous peer review
> > > > (APR) is a moral equivalent. 
> > > > 
> > > > Will Henkes eventually win the case in which they
> > > > seek to declare the anonimity in peer review illegal
> > > > (which it undoubtedly is from the point of view of
> > > > a normal sense, but not necessarilly the judicial
> > > > machinery) remain to be seen. But even if they fail,
> > > > some new (perhaps collective) legal battle(s) on this 
> > > > issue are well overdue.
> > > > 
> > > > Greatest scientists of the world (Euclid, Newton,
> > > > Descartes, Faraday, and scores of others) did not
> > > > see any need for APR. They would undoubtedly laugh 
> > > > at us if they could look at the end of 20th century, 
> > > > especially if they hear about such brilliant ideas 
> > > > as ANONYMOUS (!) peer review for ELECTRONIC (!) 
> > > > publications. Johnathan Swift (in Gulliver's Travel
> > > > to Laputa) was perhaps the very close to the correct
> > > > depicting of this nonsenical vanity fair.
> > > > 
> > > > Alex Berezin  
> > > > 
> > > > **********************************
> > > > Alexander A. Berezin, PhD
> > > > Department of Engineering Physics
> > > > McMaster University, Hamilton,
> > > > Ontario, Canada, L8S 4L7
> > > > tel. (905) 525-9140 ext. 24546
> > > > e-mail: BEREZIN at MCMASTER.CA
> > > > **********************************
> > > > 
> > > >      
> > > > On 28 Oct 1995, Allan Frey wrote:
> > > > 
> > > > > 
> > > > > In view of the continuing discussion of peer review, I thought the view
> > > > > of an editor of the Journal of the Federation of American Societies for
> > > > > Experimental Biology (FASEB J.) might be of interest.  The following is
> > > > > an editorial that I wrote and which recently appeared in the FASEB J.
> > > > > 
> > > > > Allan
> > > > > 
> > > > > Allan H. Frey                           email afrey at uunet.uu.net
> > > > > 11049 Seven Hill Lane                   voice 301.299.5181
> > > > > Potomac, MD 20854, USA
> > > > > 
> > > > > 
> > > > > The courts are considering whether the identity of peer reviewers should
> > > > > be secret.  Should we consider it?
> > > > > 
> > > > > Allan H. Frey
> > > > > 11049 Seven Hill Ln.
> > > > > Potomac, MD 20854  USA
> > > > > afrey at uunet.uu.net
> > > > > 
> > > > > It has been a fundamental tenet of science that the names of grant
> > > > > application and manuscript reviewers must be kept secret in order for the
> > > > > system to work.  But is that true?  The courts may soon force the
> > > > > scientific community to find out.  The first round concerning the question
> > > > > of secrecy has been fought in Federal District court; the second round is
> > > > > about to begin in a Federal Appeals Court (1).
> > > > > 
> > > > > Suit was brought in the US District Court for the District of Columbia to
> > > > > force the NSF to reveal the names of the reviewers who wrote the
> > > > > evaluations of a rejected grant application (Henke vs US Dept of Commerce
> > > > > and NSF).  The plaintiffs believe that some of the reviewers may have had
> > > > > a conflict-of-interest.  In the first round, the District Court issued a
> > > > > summary dismissal order based on the technical point that the reviewers
> > > > > were promised secrecy; consequently, the names are exempted from
> > > > > disclosure under the Privacy Act.  Thus, the real issue for science was not
> > > > > addressed.  The plaintiffs are appealing to a Federal Appeals Court.
> > > > > 
> > > > > Given the social changes that have been occurring in recent decades, it
> > > > > would not be surprising if a higher court found that the names should not
> > > > > be secret; if not in this case, in a subsequent case that will surely follow.
> > > > > Some people believe that the recent US Supreme Court ruling on Daubert vs
> > > > > Merrill Dow Pharmaceuticals will be used to subpoena secret records of
> > > > > granting agencies and journals.  Thus, a full debate on this secrecy issue
> > > > > within the scientific community at this time would be timely.  It would be
> > > > > much better for the scientific community to actively debate the issue, and
> > > > > explore and test alternatives in an orderly fashion, than to wait and find
> > > > > itself suddenly forced to make a precipitous change.
> > > > > 
> > > > > Through such an open debate and exploration, the scientific community
> > > > > may actually come up with something better.  The present system of
> > > > > secrecy has certainly not been without its problems. With reason, it has
> > > > > been argued that secrecy in peer review shields unqualified reviewers and
> > > > > biased editors; and invites abuse by reviewers who may have conflicts of
> > > > > interest, who may steal ideas and who may deliberately delay publication
> > > > > (2-4).
> > > > > 
> > > > > On the other hand, an apparently reasonable argument can be made that the
> > > > > identity of people who do peer reviews should not be revealed.  But
> > > > > implicit in this latter argument are assumptions that are not true in real
> > > > > life.  One implicit assumption is  that scientists  (reviewers) are as the
> > > > > public often images them,  objective, dispassionate, all-knowing beings.
> > > > > We know they are not.  A "peer" is  often a direct competitor for fame and
> > > > > funds and this influences his  actions in science as much as if he were in
> > > > > business or sports.  Scientists (reviewers) are  human beings with all the
> > > > > faults and motivations that you find in every  other field.  Once upon a
> > > > > time, when the body of knowledge was small, an editor could  filter out
> > > > > such abuses or could recognize the validity of an argument in  rebuttal.
> > > > > But now, the body of knowledge is so large and science is  divided into so
> > > > > many sub-specialities, that no editor or program manager  can prevent
> > > > > these abuses.
> > > > > 
> > > > > Further as Judson has pointed out "...Although peer review and refereeing
> > > > > seem rational, indispensable, and immutable, the histories demonstrate
> > > > > that they are social constructs of recent date.  They are not laws of
> > > > > nature, nor of epistemology."  He also points out that "... as over the years
> > > > > an ever-smaller proportion of grants gets funded and as the applications
> > > > > themselves ... are more difficult to put in any reliable rank order, politics
> > > > > become overt in the review process.  Rivalries between scientists,
> > > > > laboratories, and schools of thought emerge as palpable factors." (5).
> > > > > Given these facts, is secrecy wise?  Is it necessary for peer review to
> > > > > work?
> > > > > 
> > > > > It can, and has been argued, that a primary reason secrecy is required is
> > > > > that scientists will be afraid to review if their identity were revealed.
> > > > > There is something to this argument; but they may also feel constrained to
> > > > > give a fair review.  And consider the fact that the reviewer's name is
> > > > > always published with his review of a scientific book. Consider, also, that
> > > > > we already have other indications that secrecy is not necessary for peer
> > > > > review to work.  Scientists post mss on Internet newsgroups now and
> > > > > receive lively criticism from readers (reviewers) whose identity is not
> > > > > secret.   I put on the Internet for criticism sections of a book ms which
> > > > > was just published (6).  I didn't perceive anyone pulling punches by any
> > > > > means, and they knew I would know their identity.  Maybe the expanding
> > > > > use of the Internet will resolve the question for us.
> > > > > 
> > > > > As a scientist who has worked full-time in  research,  I have come to the
> > > > > conclusion that the identity of reviewers  should not be kept secret.  I
> > > > > personally tell editors and granting agencies  that they may reveal my
> > > > > identity to authors.  I think science and scientists are ill-served by the
> > > > > practice of secrecy and, ultimately, Society which benefits from and pays
> > > > > for scientific  research is harmed.
> > > > > 
> > > > > Consider also that this matter of secrecy is not a new problem for
> > > > > Society.  Society has struggled with it for a millennium in other forms and
> > > > > in other contexts.  This  question in science is merely its latest
> > > > > incarnation.  Its a matter of  balancing the pros and cons, and the balance
> > > > > that our Society has arrived at is incorporated in our Constitution as the
> > > > > clause concerned with due process.  In a criminal trial, witnesses are
> > > > > sometimes in fear for their life, but our Society through long experience
> > > > > has decided that, on balance, it best serves Society to allow the defendant
> > > > > to know the identity of and to cross-examine the accuser.  Why should we
> > > > > reviewers for journals and granting agencies be an exception to these
> > > > > rules that were incorporated in the Constitution after a millennium of
> > > > > effort to find the right balance.  In my judgment as a scientist, science
> > > > > and Society would be better off without the veil of secrecy.  Certainly
> > > > > there will be problems, just as there are now with the veil of secrecy.
> > > > > But on balance, with due process, science will find ways to manage as the
> > > > > courts have and will be better off.
> > > > > 
> > > > > References
> > > > > 1.  Henke vs US Dept of Commerce and the National Science Foundation.  US
> > > > > District Court for the District of Columbia, Civil action 94-0189
> > > > > 2.  Rennie, D., (1993) More peering into peer review.  JAMA  270,  2856-
> > > > > 2858
> > > > > 3.  Osmond, D. H., (1983) Malice's Wonderland: research funding and peer
> > > > > review.   J. Neurobiol. 14, 95-112.
> > > > > 4.  Broad, W. J., (1980) Imbroglio at Yale, I and II,  Science. 210, 38-41,
> > > > > 171-173.
> > > > > 5.  Judson, H. F., (1994) Structural transformations of the Sciences and the
> > > > > end of peer review.  JAMA. 272, 92-94.
> > > > > 6.  Frey, A. H., ed (1994) In "On the Nature of Electromagnetic Field
> > > > > Interactions with Biological Systems". R. G. Landes Co. Austin.
> > > > > 
> > > > > 
> > > > 
> > > > 
> > > > 
> > > 
> > > 
> > 
> 




More information about the Bioforum mailing list