Peer Review Anonymity
berezin at MCMAIL.CIS.MCMASTER.CA
Wed Nov 8 10:18:59 EST 1995
On 8 Nov 1995, Una Smith wrote:
> Alexander Berezin <berezin at mcmail.cis.mcmaster.ca> wrote:
> >-You may say, how about all the crackpot crap, etc ?
> >-This problem has to be dealt with at the editorial
> >-level, prior to the revieweing.
> Oh really? How is the end result different, if an editor
> rejects you or a reviewer does it? Maybe the editor has a
> lot of experience and good judgement. Maybe not. Maybe
> the reviewer is the most generous scientist in the world,
> maybe not.
This is why we need a system providing a genuine compromise,
not pretending to do it while in fact it does not.
This means that in case of a controvercy the ) author(s)
are allowed to publish a shortened version of an article
[ by how much - decided by the editor ] AND refereree
comments (SIGNED). It does not mean this is fully ideal
system, but many times better than present "yes-no" alternative
as it provides "hedging of the bets" strategy (the present
system does not).
> We all want to avoid reading incompetent publications by
This does not present any major problem (or rather
no problem at all). If you believe you are (even
moderately) competent in your area you should be
able to see the crap in 2 seconds. Then just then
don't read it. So, what's your problem here ?
> but at the same time we all want to get our own
> work published ASAP and as painlessly as possible.
No, not quite. What people are really struggling for
under the present system is to get their work
published in the highly acclaimed, prestigious journals,
so they will get credits for their careeer advancement,
grantsmanship, etc. This is the prime reason why peer
review flourishes, NOT "quality of science". We publish
LARGELY (though not exclusively) for this (career) purpose.
Example: In one area known to me all papers presented
at the annual conference are published (essentially,
un-peer reviewed) in the "Conference Record" (so, the
paper is PUBLISHED in a technical sense as the Conference
Record is, of course, a public domain information).
AND YET, afterwords you can submit THE SAME paper to
the (prestigious) peer reviewed journal and get it
published (if approved by peer review) in this journal.
And (most) people do exactly this (taking time from
reviewres, etc) with a SOLE purpose of getting career scores.
> In the
> absence of an ideal (and perhaps impossible) information
> system, these two goals are mutually exclusive. The peer
> review system tries to achieve a compromise between these
> two goals. I'd be happy to hear practical ideas for how
> to do better.
We are talking perhaps the same in diffent words.
Yes, any scheme involves some kind of a compromise.
What I am saying is that the present "classical"
anonymous peer review (APR) is far below the level
of the possible compromise attainable with the present
publication and information technology. The reasons,
as I said earlier, are the sociology of science
(present reward system), NOT technology OR
the "science" per se.
This is why, to account for the inherent uncertainty
of ANY peer review (and you can't do anything about
this uncertainty, it is as fundamental as Heisenberg
uncertainty principle) we suggest that:
(1) as the first step transformation to an open
(signed) reviewing be adopted in publishing
[ i.e. "rejected" authors be allowed to publishe
brief versions AND signed reviewers reports ]
(2) in funding the sliding scale be used [ robust
ranking instead of yes-no decisions ] and
principle "fund researchers, not proposals" be
gradually (re)introduced, as much more responsible
socially and ecomomical financially (at least for
what is called university-type academic research).
More information about the Bioforum