Peer Review (fwd)
berezin at MCMAIL.CIS.MCMASTER.CA
Mon Oct 30 10:36:37 EST 1995
---------- Forwarded message ----------
Date: Mon, 30 Oct 1995 10:33:00 -0500 (EST)
From: Alexander Berezin <berezin at mcmail.CIS.McMaster.CA>
To: Ashok Grover <groverak at fhs.csu.mcmaster.ca>
Cc: biocan at net.bio.net
Subject: Re: Peer Review
On 30 Oct 1995, Ashok Grover wrote:
> Three points:
> (a) The rejection of the manuscript of Gregor Mendel's paper by
> Naggli did not occur by an anonymous peer review but in a signed letter.
Fine. I never claim that abolishing of anonymity in PR will
solve all the problems and bring a paradise. But it will
(in my opinion) make things significantly better. And as
you know, Mendel (in a long run) have made it.
> (b) The journal Cell Calcium uses signed reviews. However, I do not
> see very many papers with greater insight in this journal than in those
> which use anonymous peer review.
Glad to hear that the precedent exists. Unfortunately, your
observation (quality is about the same) is very difficult to
support or refute - not because I deny your insight (I don't),
but simply because an isolated opinion of this kind does not
provide a sufficient data base.
As I said earlier: my critical experiment will be that
(whatever) major journal run 2 sections in parallel for
several yeas (as somebody pointed out, 1 years in not
Section A: business as usual (anonymous PR)
Section B: open revewies.
The submitting aothors are free to chose the option they
want their paper to be processed. This statistics (along
with a long range quality assessment differences: citation
analysis, etc) shoud be reported, so the community can
assess the results.
> (c) If you feel that you do not believe in anonymous peer review, no one
> stops you from signing your name. I know a colleague who has a
> stamping stating that he does not believe in anonymity of reviewers, uses
> this stamp and signs his name on every thing he reviews. The colleague
> has remained successful in obtaining peer reviewed grants despite signing
> his naem. If in deed such a populous feeling exists and more than half the
> scientists use this approach, they can effectively change the system.
I have answeed the above in detail in a parallel posting to bioforum.
I do not relate the success in obtaing grants or getting papers
published with the fact that you (as a revieer) sign or not.
> On 28 Oct 1995, Alexander Berezin
> > ---------- Forwarded message ----------
> > Date: Sat, 28 Oct 1995 21:34:10 -0400 (EDT)
> > From: Alexander Berezin <berezin at mcmail.CIS.McMaster.CA>
> > To: Allan Frey <afrey at uunet.uu.net>
> > Cc: bioforum at net.bio.net
> > Subject: Re: Peer review-Faseb J. editorial
> > Dear Dr. Frey,
> > On behalf on very many (likely MANY THOUSANDS) of colleauges
> > in science who do not buy the myth that "peer review is the
> > best when it is secret" (even if many of us are FORCED, by
> > the circumstances, to participate in this ugly ritual) let
> > me thank you for posting your Editorial from the Faseb
> > Journal. (and congratulations to the editors of Faseb J.
> > for publishing it).
> > The issue which still waiting for the proper (perhaps,
> > detailed) answer is this:
> > PARADOX OF HISTORY:
> > How did it happen that the world community of scientists,
> > if you wish an INTELLECTUAL BROTHERHOOD (and, of course,
> > SISTERHOOD, according to polit.correct standards of today's
> > language) which undoubtedly includes many brilliant the
> > most insightful minds this world has produced ("IF NOT
> > THE SCIENTISTS, THAN WHO ?") can, nonetheless, build,
> > perfect (and subjugate itself to) such an utterly
> > ANTI-intellectual and secretive, system as "Anonymous
> > Peer Review" (APR). Why scientists THEMSELVES (no-one
> > invented APR for them) have collectively chosen to
> > leave by gestappian standards ?
> > What do we respect more:
> > - normally signed letters we recieve (from our friends,
> > colleagues, opponents, or even open enemies) OR:
> > - dirty, unsigned, libelous notes squeezed under the door ?
> > (e.g. "your wife/husband was seen with..." kind of stuff)
> > Unlikely there will be too many votes in favour
> > of the second option.
> > And yet, in science we do precisely this. By keeping
> > its insistence to choose THE SECOND option (anonymous
> > peer review) we declare our allegence to the squeezed
> > under the door option of which anonymous peer review
> > (APR) is a moral equivalent.
> > Will Henkes eventually win the case in which they
> > seek to declare the anonimity in peer review illegal
> > (which it undoubtedly is from the point of view of
> > a normal sense, but not necessarilly the judicial
> > machinery) remain to be seen. But even if they fail,
> > some new (perhaps collective) legal battle(s) on this
> > issue are well overdue.
> > Greatest scientists of the world (Euclid, Newton,
> > Descartes, Faraday, and scores of others) did not
> > see any need for APR. They would undoubtedly laugh
> > at us if they could look at the end of 20th century,
> > especially if they hear about such brilliant ideas
> > as ANONYMOUS (!) peer review for ELECTRONIC (!)
> > publications. Johnathan Swift (in Gulliver's Travel
> > to Laputa) was perhaps the very close to the correct
> > depicting of this nonsenical vanity fair.
> > Alex Berezin
> > **********************************
> > Alexander A. Berezin, PhD
> > Department of Engineering Physics
> > McMaster University, Hamilton,
> > Ontario, Canada, L8S 4L7
> > tel. (905) 525-9140 ext. 24546
> > e-mail: BEREZIN at MCMASTER.CA
> > **********************************
> > On 28 Oct 1995, Allan Frey wrote:
> > >
> > > In view of the continuing discussion of peer review, I thought the view
> > > of an editor of the Journal of the Federation of American Societies for
> > > Experimental Biology (FASEB J.) might be of interest. The following is
> > > an editorial that I wrote and which recently appeared in the FASEB J.
> > >
> > > Allan
> > >
> > > Allan H. Frey email afrey at uunet.uu.net
> > > 11049 Seven Hill Lane voice 301.299.5181
> > > Potomac, MD 20854, USA
> > >
> > >
> > > The courts are considering whether the identity of peer reviewers should
> > > be secret. Should we consider it?
> > >
> > > Allan H. Frey
> > > 11049 Seven Hill Ln.
> > > Potomac, MD 20854 USA
> > > afrey at uunet.uu.net
> > >
> > > It has been a fundamental tenet of science that the names of grant
> > > application and manuscript reviewers must be kept secret in order for the
> > > system to work. But is that true? The courts may soon force the
> > > scientific community to find out. The first round concerning the question
> > > of secrecy has been fought in Federal District court; the second round is
> > > about to begin in a Federal Appeals Court (1).
> > >
> > > Suit was brought in the US District Court for the District of Columbia to
> > > force the NSF to reveal the names of the reviewers who wrote the
> > > evaluations of a rejected grant application (Henke vs US Dept of Commerce
> > > and NSF). The plaintiffs believe that some of the reviewers may have had
> > > a conflict-of-interest. In the first round, the District Court issued a
> > > summary dismissal order based on the technical point that the reviewers
> > > were promised secrecy; consequently, the names are exempted from
> > > disclosure under the Privacy Act. Thus, the real issue for science was not
> > > addressed. The plaintiffs are appealing to a Federal Appeals Court.
> > >
> > > Given the social changes that have been occurring in recent decades, it
> > > would not be surprising if a higher court found that the names should not
> > > be secret; if not in this case, in a subsequent case that will surely follow.
> > > Some people believe that the recent US Supreme Court ruling on Daubert vs
> > > Merrill Dow Pharmaceuticals will be used to subpoena secret records of
> > > granting agencies and journals. Thus, a full debate on this secrecy issue
> > > within the scientific community at this time would be timely. It would be
> > > much better for the scientific community to actively debate the issue, and
> > > explore and test alternatives in an orderly fashion, than to wait and find
> > > itself suddenly forced to make a precipitous change.
> > >
> > > Through such an open debate and exploration, the scientific community
> > > may actually come up with something better. The present system of
> > > secrecy has certainly not been without its problems. With reason, it has
> > > been argued that secrecy in peer review shields unqualified reviewers and
> > > biased editors; and invites abuse by reviewers who may have conflicts of
> > > interest, who may steal ideas and who may deliberately delay publication
> > > (2-4).
> > >
> > > On the other hand, an apparently reasonable argument can be made that the
> > > identity of people who do peer reviews should not be revealed. But
> > > implicit in this latter argument are assumptions that are not true in real
> > > life. One implicit assumption is that scientists (reviewers) are as the
> > > public often images them, objective, dispassionate, all-knowing beings.
> > > We know they are not. A "peer" is often a direct competitor for fame and
> > > funds and this influences his actions in science as much as if he were in
> > > business or sports. Scientists (reviewers) are human beings with all the
> > > faults and motivations that you find in every other field. Once upon a
> > > time, when the body of knowledge was small, an editor could filter out
> > > such abuses or could recognize the validity of an argument in rebuttal.
> > > But now, the body of knowledge is so large and science is divided into so
> > > many sub-specialities, that no editor or program manager can prevent
> > > these abuses.
> > >
> > > Further as Judson has pointed out "...Although peer review and refereeing
> > > seem rational, indispensable, and immutable, the histories demonstrate
> > > that they are social constructs of recent date. They are not laws of
> > > nature, nor of epistemology." He also points out that "... as over the years
> > > an ever-smaller proportion of grants gets funded and as the applications
> > > themselves ... are more difficult to put in any reliable rank order, politics
> > > become overt in the review process. Rivalries between scientists,
> > > laboratories, and schools of thought emerge as palpable factors." (5).
> > > Given these facts, is secrecy wise? Is it necessary for peer review to
> > > work?
> > >
> > > It can, and has been argued, that a primary reason secrecy is required is
> > > that scientists will be afraid to review if their identity were revealed.
> > > There is something to this argument; but they may also feel constrained to
> > > give a fair review. And consider the fact that the reviewer's name is
> > > always published with his review of a scientific book. Consider, also, that
> > > we already have other indications that secrecy is not necessary for peer
> > > review to work. Scientists post mss on Internet newsgroups now and
> > > receive lively criticism from readers (reviewers) whose identity is not
> > > secret. I put on the Internet for criticism sections of a book ms which
> > > was just published (6). I didn't perceive anyone pulling punches by any
> > > means, and they knew I would know their identity. Maybe the expanding
> > > use of the Internet will resolve the question for us.
> > >
> > > As a scientist who has worked full-time in research, I have come to the
> > > conclusion that the identity of reviewers should not be kept secret. I
> > > personally tell editors and granting agencies that they may reveal my
> > > identity to authors. I think science and scientists are ill-served by the
> > > practice of secrecy and, ultimately, Society which benefits from and pays
> > > for scientific research is harmed.
> > >
> > > Consider also that this matter of secrecy is not a new problem for
> > > Society. Society has struggled with it for a millennium in other forms and
> > > in other contexts. This question in science is merely its latest
> > > incarnation. Its a matter of balancing the pros and cons, and the balance
> > > that our Society has arrived at is incorporated in our Constitution as the
> > > clause concerned with due process. In a criminal trial, witnesses are
> > > sometimes in fear for their life, but our Society through long experience
> > > has decided that, on balance, it best serves Society to allow the defendant
> > > to know the identity of and to cross-examine the accuser. Why should we
> > > reviewers for journals and granting agencies be an exception to these
> > > rules that were incorporated in the Constitution after a millennium of
> > > effort to find the right balance. In my judgment as a scientist, science
> > > and Society would be better off without the veil of secrecy. Certainly
> > > there will be problems, just as there are now with the veil of secrecy.
> > > But on balance, with due process, science will find ways to manage as the
> > > courts have and will be better off.
> > >
> > > References
> > > 1. Henke vs US Dept of Commerce and the National Science Foundation. US
> > > District Court for the District of Columbia, Civil action 94-0189
> > > 2. Rennie, D., (1993) More peering into peer review. JAMA 270, 2856-
> > > 2858
> > > 3. Osmond, D. H., (1983) Malice's Wonderland: research funding and peer
> > > review. J. Neurobiol. 14, 95-112.
> > > 4. Broad, W. J., (1980) Imbroglio at Yale, I and II, Science. 210, 38-41,
> > > 171-173.
> > > 5. Judson, H. F., (1994) Structural transformations of the Sciences and the
> > > end of peer review. JAMA. 272, 92-94.
> > > 6. Frey, A. H., ed (1994) In "On the Nature of Electromagnetic Field
> > > Interactions with Biological Systems". R. G. Landes Co. Austin.
> > >
> > >
More information about the Bioforum