FX/JAMA/Peer Review

U27111 at uicvm.uic.edu U27111 at uicvm.uic.edu
Tue Dec 12 04:23:11 EST 1995




Alexander A. Berezin, PhD wrote:

>> ckrepel at post.its.mcw.edu (Candace Krepel) wrote on 8 Dec 1995
>> 08:41:52 -0600:
>
>> >As I am sure you know, the scientific method requires that the
>> >investigator control for all factors other than the one being
>> >tested.

>(KATHY):
>> And I'm sure you know just how much garbage does get published
>> under the guise that the scientific method was followed!

>BEREZIN:
>Garbage in science (all disciplines) gets published in
>astronomical quantities BECAUSE of peer review and
>resulting from it paper counting mentality (it is
>"prestigeous" to publish "peer reviewed" paper in
>"prestigeous" journal - no matter the content or "who
>reads it ?").

I disagree...

I think a lot of garbage gets published because:

1.   The lack of peer review... how many of the articles in Medline
     (for example) are peer-reviewed in the first place?  I seemed
     to recall reading somewhere that only about half (or a little
     more than half) of them are... ???

2.   Or because when there is peer review...

-    It's either not reviewed well

-    or the authors lied so well it fooled the reviewers (but in
     may cases that's not very hard to do?)

-    or the reviewers themselves are biased and only care to allow
     data/materials to get published which only agrees with their
     own ideas or models of what is supposed to be going on (thus,
     not really caring how credible it may really be... just that
     it is more ammunition to support their own claims or political
     objectives);

-    or, if someone is already 'established',... does the reviewers
     actually bother to question the validity/credibility of this
     'latest' paper presented for peer-review? ie., resting on
     one's laurels.

-    And finally, what of conflict of interest problems?  How do we
     know the reviewers for a paper on the effectiveness of a new
     drug\treatment don't have personal interest at stake (stock in
     the company, receives grants from that company, or in the case
     of that one journal editor - he received monies from the
     company involved in a lawsuit for being an expert witness in
     the court proceedings for them... he rejected two articles
     which supported the claims made by those who initiated the
     lawsuit while published an article he wrote himself which
     supported the company's position on the matter)

>"The more the better" is the prime principle in modern science
>(after it became an industry), and I've heard things like
>"... he must be a dead wood because he publishers ONLY two
>papers per year ..." (Copernicus had one in a whole life).

Quantity over quality is a problem due to the competition for grant
monies... ie. the 'publish or perish' syndrome.  And *that* is the
reason for the present paper counting mentality we see.

I don't think this is necessarily a direct problem with peer-review
in it self?

It does result in too may papers/too little time problems which
probably effects the caliber of the peer-view process in general.

But I really think it's really more of a combination of problems...
with the major one being politics!

>Only the scrapping of anonymous peer review system and its
>replacemet by a system of open commenting to all published
>(electronically and/or paper-bound) papers will relieve
>science from publish-perish trap and release the creative
>energy of scientists towards solving REAL problems instead
>spending their lives to fight pseudo-problems of prestige,
>grantsmanship, "competition" and other crap.

Now here I disagree... instead of scrapping the system entirely...
I think we just need to make the process better.  It could be
improved by creating a more strict conflict of interest definition
and enforcing it (possibly by legal measures?); by making the
authors of the papers presented anonymous to those reviewing it;
and by cutting down on the amount of politics involved.

To cut down on the politics... we need to create a more ethical
environment (where ethics is put back on the top of the list!).  To
do that, we need to standardized the field... certified researchers
in accredited laboratories.  People may still lie about their work
presented (but no matter what system you create, some will always
try to find a way around it)... but it would definitely cut down on
the slop being done in research which gets presented as 'good
work'.  For a sloppy lab would have a very hard time getting
accredited in the first place... let alone maintaining one.

As for 'open' comments system you purpose... I personally believe
people, in general, are just too apathetic to care either way.  And
it would most likely result with those with the loudest voices as
the only ones being heard in the end.  Not much different then we
have today actually?

-Kathy



More information about the Bioforum mailing list