Peer Review Anonymity on Trial

U27111 at uicvm.uic.edu U27111 at uicvm.uic.edu
Wed Jan 17 13:48:31 EST 1996



robison at lipid.harvard.edu (Keith Robison) wrote on 16 Jan 1996
07:13:32 GMT:

>Look at what is already regulated in research labs: nuclear
>and toxic waste handling, biohazards, etc.

That's another topic.

Which, BTW, clinical labs also do a hell of a lot better then
academic ones.

In academia (some of the places I have worked at least)...
radioactive waste is more of a number game... in that you just make
sure the amount which came in adds up to the amount which goes
out... and that's pretty much it.

And I've seen places were hot materials are dumped down regular
sinks and biohazardous materials are just tossed in the regular
garbage.

I wonder if we are just fighting a poor attitude here in acidemia?

>Look how many hours go into complying with those regulations.

??? Not from where I've been.  Many people tend to ignore all that.

Again, because of a self-interest lifestyle (and thus laziness
attitude)... it's just too much work to bother with.

>Look how many people are employed _solely_ to monitor these
>regulations.

??? You mean a lab safety office?  I don't know.  I think they are
pretty important to the scheme of things.

I know a lot of people in research think they are a big waste of
time (like you seem to??)

But I think they are there exactly because people don't care to do
things properly in the first place and, thus have to be monitored.

Not that lab safety, in general, is really capable of it.  A lot is
still left up to the PIs... who, generally ignores them.

Thus, this isn't really a guaranteed working system either... but
it helps - and I really can't imagine what it would be without
them?  Even worst!

No system is perfect... but this system is still better then no
system - which is why I think standardizing the field would help.

>Now multiply this by electrophoresis, dialysis, chromatography,
>PCR, electrophoresis, immunochemistry, etc., etc.

Were are talking about two different things here.

Take a look into a clinical lab and see how professionally those
are ran.

Regulations on how biohazardous and radioactive waste are handled
is completely different from the type of regulations I am talking
about here in running proper standardized assays and the such.

>Now how are you going to enforce it?  How do you guarantee
>that an "uncertified" person doesn't run a gel?

There is no guarantee.  And actually, you have a very valid point
here.  If we have academics who just make sure things look good on
paper in the handling of radioactive waste (as well as the
presentation of data?)... they could sure do the same thing here.
Just say they have a certified person.

But when it comes to processing a grant and putting the name of the
lab tech on it... they will need to include a copy of that person's
certification certificate!

Of course there is no guarantee that same person will be doing the
work once the monies come in.

But then again, when it comes to maintaining that lab
accreditation... again, they will have to prove the people working
in there are certified.  Except for grad students... who wouldn't
be certified yet.

>Hell, I'd _love_ it if nobody could run BLAST without a
>license -- not only would it guarantee me work, but it would
>prevent a lot of silly statements in the literature (whatever
>horrors you have seen -- remember, in my field I can find
>them in the _published_ data).

Actually... we are in the same field.  And I know exactly what you
mean.... I've seen some pretty silly stuff published as well (as
well as work generated published where they change the
materials/methods section).

On of my personal favorite comments (story?) comes from this MD-PhD
post-doc (no names, I've already had a few personal emails
concerning supposedly libelous statements I've made).... anyway, he
came back to the lab to do a poster presentation of his thesis for
an upcoming meeting.  And since he left, we have been trying to
repeat his work - to no avail (he even came in once previously to
re-run it and it still didn't work).

Well, I'm reading his poster as he is putting it together and there
are some changes in the materials/methods section from the known
protocol.  Perplexed, I turned to him and said that this isn't how
we have been doing it?  COuld this be why we couldn't repeat his
work?

And his reply was, "No, I don't think so.  But don't you think we
*should* be doing it this way anyway"

And it did certainly read better... he was right in that. And I
have to give him credit... he was truthful in his reply to me at
least?

And I think his reply gives an awful lot of incite about what it is
I have been talking about here.

For these are the type of attitudes which need to be changed.
Instead of tossing a project together... it should be well thought
out and well executed.  Instead of months later trying to figure ut
exactly how you did it and thinking up better ways of doing it well
after the fact?

We don't really teach this to students.  It's all quantity of work
over quality.  And to get it done fast.

And don't get me wrong... I understand the process of tweeking a
procedure and fixing it as we go along - only too well.  But this
here is different... this is presenting something which was only
half-thought out at the time and trying to tweek it well after the
fact.

It's unacceptable.  We are not going to progress as fast as we
could otherwise.  Too much wasted time and money.

>Trying to certify research would impose a huge new bureaucracy,
>with an attached cost.  Worse, it would probably have the
>effect of Prohibition -- raise the level of general disregard
>for regulations.

??? I'm not sure about that ???

I think we already have a pretty general disregard for regulations
as it is.

I just can't help to feel that if we can install some of the
professionalism as seen in clinical labs and forensic labs into
research labs.... it can only help.

How did the entire forensic field change itself into a standardized
one these past several years?  I don't know the details... but they
did it.  Actually they were forced to do it from a great amount of
criticisms in finding poorly ran RFLP test resulted in innocent
people going to jail (this is years before OJ, BTW... before they
standardized the field!).

So why is it not ok for innocent people to go to jail... but
suffering dying people can be given drugs in studies which came
from poorly ran animal drug studies?

>Remember -- one of the most talked-about papers in the
>last year involved PCR performed by a _mathematician_.
>Raising new barriers of entry to a field is not a sure-fire
>way to improve it.

I'm sorry... last year I was home taking care of my mother
(seriously) and missed out on a lot of things which went on in the
field.  I've been out of the loop.  Thus, I missed this.

Could you please ex[plain it a bit more and help me to understand
this comment you make?  Thanks.

>: In all you wrote about this Imanishi-Kari's case... after
>: several years now... did her original data prove to be true?

>Imanishi-Kari's data was based on unique reagents which
>have long been consumed -- in a certain sense, her
>experiments can never quite be replicated.
>
>Independent replication of the conclusions of her work
>(last I heard) remained a controversial subject.

Actually... I went to the library and just got the book "Science on
Trial: The Whistle-blower, the Accused, and the Nobel Laureate" by
Sarasohn.  Looks interesting... I hope to start reading it tonight.

But now that I know you are talking about the Baltimore affair...
I now know what you are talking about.  I just didn't recognize
Imanishi-Kari's name when it first came up in this thread... it's
been a while since I followed this case (1988-89?)... I haven't
kept up with it since.

>If you want a case which illustrates how messy it can be to deal
>with borderline science, her case would be a good place to start.

I don't know... just from a few of the articles I looked up and the
front and back cover of this book... it sounds an awful lot like
the Gallo case in a lot of respects, and now the Fisher case as
well?

If anything... it just sounds like we are going in circles here and
we don't really get anywhere when such cases presents itself.

A very good reason to standardized the field, IMHO.  More and more
such cases pop up and 7-10 years go by before issues are settled
(if ever?).

If we had accredited labs... you could go back and look at a labs
proficiency records to see exactly what level of caliber they've
maintained over the years.

More data to add to the pool when trying to decipher who's telling
the truth?

I don't know.


Something needs to be done.  This was the best idea I could come up
with.

What else can we do to try and raise the caliber of labs and the
quality of work products.

One of the things I read in a more recent Science article on this
Imanishi-Kari case... that she published a type of repeat of her
original Cell article work - and people in Immunology praised it
for how detailed it was, well written, good presentation of the
work etc.

I guess her original Cell article was not so well done in the first
place (besides the results being questioned?)?

It just struck me why did it take several investigations, two
congressional hearings, an ORI investigation and such a big
controversy over this entire situation for her to finally write a
more detailed paper (and obviously, to do such detail work)?

Don't get me wrong... what I am talking about here is how much slop
which does get through peer-review, accepted into the community -
when the person is indeed capable of much more - but they just
don't bother to take the time to do it?  And only when their work
is called into question... then this all kicks in.

This should be the standard in the first place.  No?

-Kathy



More information about the Bioforum mailing list