No Peer Review: LESS Garbage

Keith Robison robison at lipid.harvard.edu
Fri Jan 19 09:20:59 EST 1996


Alexander Berezin (berezin at MCMAIL.CIS.MCMASTER.CA) wrote:

: On 18 Jan 1996, Gregory R. Harriman wrote:

: > berezin at MCMAIL.CIS.MCMASTER.CA (Alexander Berezin) wrote:
: > 
: > (BEREZIN)
: > > Hogwash or not, but you may bother to look through
: > > citations studies which say that about HALF of all
: > > published papers (peer reviewed, of course) are NEVER
: > > cited even a single time (and half of the remaining
: > > are cited just once - often as self-citation).
: > >   Are this papers silly or not is not critically
: > > important - most imopratnt is that they are useless
: > > for all practical purposes (again, some exceptions
: > > of course may happen, but these are rarities). 

: (HARRIMAN): 
: >      Even if one agreed with this assertion, it is inconceivable that
: > doing away with all types of review (anonymous or otherwise) would
: > decrease, rather than increase, the number of useless papers that are
: > published.  

The key piece of data which is lacking here is where these
"orphan" papers appear.  If Berezin is right, and the current
system of peer review & journals of varying prestige drives
junk publishing, then the orphan papers should be uniformly
distributed across journals.  On the other hand, if 
the selectivity of Science, Nature, Cell, JBC, JMB, etc really does
reduce the number of junk papers, then there should be 
a non-random distribution of the orphans (very few in the
top journals; many in the bottom-of-the-barrel).  Or, to
look at it another way, a problem with the citation figures
for journals (and individuals) is that they always cite some
figure, but do not state the variance.


: BEREZIN:
: On the contrary, it is very much conceivable. In peer review
: environment publication of paper (esp. in 'highly prestigeous'
: journal) brings you Currency Unit of Recognition (CUR).
: In non-PR environment (or in non-PR section of PR journal)
: publication of paper as such brings no CUR (status of self-
: circulated manuscript). The journal pages is simply a vechicle
: to spead paper. The value (importance) of paper will come
: from comments. No comments - no CUR. Bad comments - USUALLY
: negative CUR, however often chance for positive CUR later
: (breakthrough reserach is often not reconnized as such
: instatntly).    

: The best (as I suggest) to start mixed system (non PR
: sections in PR journals) and see where it will go. We,
: scientists, should (I hope) ALL agree that the only
: (or at least the best) way to solve the problem is to
: run the experiments. This is precisely what I am suggesting,
: though I personally don't have the resources to do it
: on my own. Should there will be a group (critical mass)
: of people to run Non-PR experiment, I will be interested
: to join at some capacity. 
:   
: (HARRIMAN):
: > Without some type of mechanism for ascertaining the quality of
: > a submitted paper, the inevitable result will be more crap getting
: > published.

: BEREZIN
: I am not sure. Say, Physical Review will run the followin experiment
: for (say) 2 years: 10 % space goes in Non-PR section and another 5 % 
: on brief but OPEN (non-PR) comments on ANY article published recently
: (time frame may apply) in this journal (i.e. people are equally
: free to comment on PR and NPR). In 2 years it will enough material
: to establish the presence (if any) and sign of correlation between
: the relative quality and impact of PR and NPR papers as reflected 
: in the number and the content of comments. I personally expect
: much more hogwash on PR side, but won't insist untill the 
: experiment is actually run. 

You obviously don't read sci.physics.new-theories or sci.bio.misc.
As soon as you open a section to no peer-review (like Usenet),
there will be a horde of cranks just itching for the opportunity
to get published in a scientific journal.  This is a serious
consideration.

: > (ROBISON) 
: > > > PRACTICAL SUGGESTION FOR APR OPPONENTS:  Start your own journal.
: > > > Set up an E-journal. Set up a site which is a central repository
: > > > or listing of NPR E-papers.  Become a journal editor and institute
: > > > NPR.  Publish your own papers by the mechanisms listed above.
: > 
: > (BEREZIN)
: > > Why do you think that I want/need to do any of the above ?
: > > My reasons for the criticsm of the APR system (and PR in
: > > general) may be quite different from what you apparently
: > > believe they are. 

I was under the impression you actually wanted the system changed
-- apologies if that is not the case.  Constructive demonstrations
of new ideas are generally more convincing than rhetoric.

: (HARRIMAN): 
: >      I have always assumed your reasons were that you had a sincere
: > interest in improving the scientific process.  Given the severe criticisms
: > you have expressed previously in regard to peer review, I'd certainly be
: > interested in knowing what are your true motivations.

: BEREZIN:
: OK, OK, charge it on me this time. Although, to express 
: your 'true motivations' is a damn difficult business. Most 
: people fail, even if they believe their efforts (to explain
: themselves) are sincere. (to save the space on this, I refer
: you to a longer poster from anonymous colleague which 
: I re-posted earlier today). 

Keith Robison
Harvard University
Department of Cellular and Developmental Biology
Department of Genetics / HHMI

robison at mito.harvard.edu 





More information about the Bioforum mailing list