Peer Review: Reply to Robison

[Alexander Berezin]

Dear Dr. Robison:

Some readers may likely attribute the fact that 
your response below (no deletions this time) contains 
quite a number of personal remarks about my (alleged) 
abilities, etc indicates that your argumentation
IN ESSENSE is somewhat circular.

I provide few more comments to your point 
(not all points are discussed - have not much time 
today).

Alex Berezin 


On 18 Jan 1996, Keith Robison wrote:

> Alexander Berezin (berezin at MCMAIL.CIS.MCMASTER.CA) wrote:
> 
> : (KATHY):
> 
> : > Don't get me wrong... what I am talking about here is how 
> : > much slop which does get through peer-review, accepted into 
> : > the community .....
> 
> : BEREZIN:
> 
> : Nice point. (how much crap comes through peer review).
> : But let's make logical steps in attempt to see how 
> : the whole process likely works:
> 
> : (1) if you publish crap which is non-peer reviewed 'every 
> : one instantly knows that this a crap'.
> 
> : (2) but if same crap gets published through PR (most gets
> : through, don't worry), especially in 'prestigious' journal, 
> : it's no longer a crap ... at least who dares to question ?
> 
> UTTER HOGWASH!!!!  You've apparently been doing so little
> science that you've forgotten how it works.  If you publish
> in a prominent journal and say something silly, then other
> people will _line up_ with their take-downs.  

BEREZIN:
Hogwash or not, but you may bother to look through
citations studies which say that about HALF of all
published papers (peer reviewed, of course) are NEVER
cited even a single time (and half of the remaining
are cited just once - often as self-citation).
  Are this papers silly or not is not critically
important - most imopratnt is that they are useless
for all practical purposes (again, some exceptions
of course may happen, but these are rarities). 

> If you actually
> _read_ Science, Nature, or Cell you'd notice that -- 
> sections titled "Scientific Correspondence", "Technical Comment",
> and "Matters Arising".  If you bothered to read the
> "Minireviews" and "News & Views", you'd notice that these are
> not always glowing summaries, but frequently raise concerns
> about newly published work or older work that has been 
> called into question.
> 
> : (3) Hence the rat race to get publsihed through PR 
> : INCREASES (not decreases) a drive for more and more 
> : crap to print (about 90 % of all published).
>                        ^^^^
> Even more evidence to suggest that 90% of all statistics
> are invented on the spot.  
> 
> : (4) It's not that most people deliberately want to publish 
> : crap, but since the mechanism is that EVERYTHING which gets 
> : published through PR automatically gets some kind of
> : a validation sticker (somewhat similar to a money laundring 
> : process), the pressure to publish ANY-THING (and eventually 
> : almost anything slips trough PR), inevitably results in a
> : very large proporton of garbage.  The paradox of PR is 
> : that, in average, it leads to the results EXACTLY OPPOSITE
> : to what is the alleged itention of PR (to be quality
> : control cleansing mechanism).  
> 
> : In short, PEER REVIEW SYSTEM IS THE MAIN GARBAGE GENERATOR 
> : IN SCIENE, not the other way around.   
> 
> : (5) PR is essentially risk insurance. Paper in a PR journal
> : essentially re-deposits the risk from the author on-to the 
> : system (special bonus - system is anonymous, so almost NO 
> : attributed personal responsibility for the risk). 

(ROBISON) 
> Again, hogwash.  Ever notice that when a paper is published
> there is a list of names over it?  They're called authors.
> They are claiming they wrote the paper.  They are stating
> it publically.  They are taking responsibility.  This has
> been abused in the past, but the general trend has been
> to be much more careful in this regard (look what happened
> to D. Baltimore; name on one questionable paper led to
> losing a plum job).  If you put your name on a paper,
> you are hitching your wagon to that paper's star -- should
> it be a bad paper, and revealed as such, your reputation
> will suffer accordingly. 

BEREZIN:
You are confusing relatively small fraction of papers
which went to some stage of misconduct allegations with
the solid majority of correct (ethically clean) papers
which nonetheless useless in a short AND long run.

I have not seen a single case when a 'reputation'
was spoiled by publishing numerous 'correct' but
useless papers. If you play safe you are safe.

 
> : (6) In the peer-less environment (no peer review, or at 
> : least no in-depth PR, just relevance check and common sense), 

(ROBISON)
> Pardon, but what does Tom Paine's writings have to do with
> science?  Seriously, how would you define common sense?
> Isn't common sense our reflex rejection of that which
> does not conform to our previous experience, the exact
> phenomenon you claim to be trying to eliminate?

BEREZIN:
I am ashamed not to know who is Tom Paine and what
his writing have to do with science. What I am saying
about common sense is that I don't expect Cell to 
publish paper on astrophusics. As simple as this.
There are many good books on common sense and how to
use it.   
> 
> : the publication will be at the SOLE risk of the author, hence
> : people will be thinking twice (or many more times) before
> : publishing.
> 
> Most people do.  And if they didn't think enough times, then
> often the reviewers force them to think again.  I know -- from
> experience.  I also note that the Acknowledgements in many
> papers thank the anonymous reviewers.  The real world is
> much more complex than your caricature of it.

BEREZIN:
The progress in the world depend on the ability
of people to caricature it. That's what I am
trying to do - and that't what I found occasionally
working.

> 
> : (7) as a result, de-peer reviewisation of science will 
> : REDUCE (not increase) the amount of sloppy papers and very 
> : likely proportion of garbage will sharply FALL (not all 
> : people will be happy with the last aspect, though).             

> Experience with non-peer reviewed scientific media suggests
> how blinded you have become by zealotry.  The public DNA
> sequence databanks are not peer-reviewed; they shouldn't be,
> because they are a data repository.  These databanks are 
> notorious for their highly uneven quality and lack of
> detail/accuracy.  The Usenet is not peer-reviewed; the
> result is a very high noise-to-signal ratio.  PNAS is
> pseudo peer-reviewed, and it shows; lots of weak papers
> contributed by society members.
> 
> 
> : PRACTICAL SUGGESTION FOR JOURNAL EDITORS:
> 
> : As experiment, start running non-peerreviewd
> : sewctions of the journals. Papers published
> : there will be labled clearly "NOT PEER REVIWED".
> : See how many authors will opt for this
> : category as opposed to standard peer-review (APR)
> : process. Perhaps, to run a controlled experiment,
> : (for the beginning) allocate 10 % to 20 % of
> : all space for NPR section. Impose length limitations,
> : if you deem necessary, but accept EVERYTHING 
> : relevant to your area on a line-up basis (first
> : come, first published - the length of line will
> : likely be eventually deternined by the waiting
> : period). Naturally, those submitting to NPR
> : can NOT submit the same to PR process.
> 
> And of course, you will quickly have a long line for the NPR
> section.  Why?  Because almost everyone wants to be out quickly,
> and there would be the perception that avoiding review 
> expedites publishing.  Perhaps careful study of the
> time from submit-to-publish would give people reasonable
> predictions of the speed tradeoff (in an ideal "market",
> of course, the ratio of PR to NPR would be close to whatever
> gives both the same lag time).
> 
> PRACTICAL SUGGESTION FOR APR OPPONENTS:  Start your own journal.
> Set up an E-journal.  Set up a site which is a central repository
> or listing of NPR E-papers.  Become a journal editor and institute
> NPR.  Publish your own papers by the mechanisms listed above.

BEREZIN:
Why do you think that I want/need to do any of the above ?
My reasons for the criticsm of the APR system (and PR in
general) may be quite different from what you apparently
believe they are. 

> _Demonstrate_ the feasibility and side-effects of your proposals,
> rather than just making basis-free assertions.  You'd be amazed
> how many people you'd convince if you actually could back it
> up with experience.  

BEREZIN:
Again, not my problem. Have no need to volunteer for
the above assignment.

> 
> Keith Robison
> Harvard University
> Department of Cellular and Developmental Biology
> Department of Genetics / HHMI
> 
> robison at mito.harvard.edu 
> 
> 
> 
> 
> 
> 
> 
> 
> 
>