Peer Review Anonymity on Trial

U27111 at uicvm.uic.edu U27111 at uicvm.uic.edu
Mon Jan 15 16:30:46 EST 1996



On Mon, 15 Jan 1996 01:40:26 -0500 (EST) "Ferland Louis H."
<ferlandl at ERE.UMontreal.CA> wrote:

>(FERLAND)
>
>But, who do you suggest decides who's the best and who's to be
>phased out? (see also my next comments, below).

The individual societies/organizations who go around inspecting the
labs (for their specific specialties)... if a lab doesn't pass
accreditation - they aren't allowed to apply for funding.

Similar to the clinical labs... if a clinical lab doesn't pass...
it's shut down.

I know it's not a perfect system... and I imagine a lot of bad labs
will still get through (which also sometimes happens in clinical
labs) - but it's a start in raising the consciousness of the field
to meet these higher standards criteria.

>I don't think bad apples CAN be identified except in the most
>obvious cases of DEMONSTRATED fraud, etc.

Personally... I can walk into a cell culture lab and tell you if it
is a bad one or not just from how it looks.  It's not that
difficult.

As for other types of labs... it doesn't take long (2-3 days) and
I know.

If I was an inspector... I could do it in less time... look through
notebooks (assuming they even keep ones?), look into refrigerators,
etc.

And if we can somehow figure out a way in which proficiency testing
can be done in research labs - even easier.  If the lab doesn't
give back the right results - they loose accreditation.

>And I don't think the *best* ones can be identified at all
>(except, perhaps, by a utopian peer-review process). (Please note
>the word "utopian" and do not consider I am suggesting the present
>peer-review system qualifies.)

I know nothing will ever be perfect or 'utopian' (except on Star
Trek)... but here, I'm just talking about raising the awareness of
ethics and morals and rasing the standards of the community (which,
presently, has no standards).

Some of this conflicting studies problems we come across has a lot
to do with the fact that the use of lab equipment is not
standardized.  Someone runs an HPLC one way and someone else runs
it another way.  And how do we know who ran proper standard
controls between the two... if papers are written to best represent
each work as valid.

Proficiency testing of the labs would help tell us who is doing it
properly and who isn't (or, change the ones who are not doing it
right to do it more correctly?).

And this isn't something which one can do for just a one shot
(test)  sample.  If you have to do internal testing once a month
(like a clinical lab) and keep records of such testing... and then
get an outside unknown every 2-3 months... you ALWAYS have to be
running your samples correctly to pass all these tests.

And that's the point of maintaining an accreditation.

And by tying accreditation to receiving monies... you give this
whole process teeth.

>PhDs ARE certified! This is what it means to have acquired the
>right to write those letters after your name.

Writing those letters after your name doesn't give you the
knowledge of ALL lab equipment and testing.

For example... a PhD in immunology who does PCR.  This PhD is NOT
certified to do PCR testing.

And being smart enough to make it through a PhD program doesn't
mean you can just read a book on PCR and just do it!  And publish
papers on it... presenting it as valid work!

And as for new PhDs... many of them very little about how to do
proper bench work.  This is not what they are being taught.

They are being taught how to *best* present their work (whether
it's correct or not... how to make it look it's very best)!

And how to write grants and survive in research in general.

Technical skills are down at the bottom of the list of what they
are taught... or, correct laboratory procedures for that matter.
Many of these new PhDs I've meet have a very poor understanding of
the scientific method.

And I will agree with Trivol at this point... that better teaching
of these students are needed.  And more classes in ethics is a
start.

But from what I have seen... we have poorly trained PhDs
(technically speaking) who are training newer PhDs - and this just
helps propagate mis-information.

My most basic example... the proper way to do a cell count.

In the one lab I worked in for a week... not only did the PhD do it
incorrectly (and insisted it was valid!)... she was training her
grad students to do it exactly the same way as her.

I was out numbered... 6 to one... and nobody would bother to go
over to the hospital across the street... walk into a clinical alb
and ask someone over there what was the correct way of doing this
(as I suggested)... I was just plain wrong.

If this lab had to have certified techs (and PhDs) and meet certain
accreditation standards... these grad students would then know how
to do a proper cell count (as well as how to properly run their
hoods, work with their cell lines, dispose of their biohazardous
waste [which all went into the regular garbage and down the sink
unbleached, BTW])... or, they would loose funding.

It's just that simple.

>Same thing with techs: you go to school, you SUCCEED and get the
>degree, the university/college/whatever certifies it is so by
>issuing a diploma.

This is not certification.  Not even close.  Go talk to a med tech
and then come back and talk to me.

>You suggest some OTHER body be endowed the power to rescind the
>validity of this assessment of qualification?

Yes... the societies and organizations which offers meeting and
seminars (and even short classes sometimes)... you know, the type
of organizations just about everybody in their own specialty
belongs to... but never bother going to these meetings have the
time?

And maybe have the NSF as the over-all head of such a system?

I don't know.  More thought has to go into this idea (for such
details) as well as evaluation studies... to take a good look at
the clinical and forensics lab systems and how to best apply this
to research labs.

I don't have all the answers... just ideas on how to make the steps
into the right directions.

>(FERLAND)

>Two quick questions:

>1) How many people do you know show up drunk for their driving
>test exam?
>2) How many drunk drivers are there on our roads?

But we are talking about details and knowledge that you can't just
'wing it'... and I am serious here... most people (at least in cell
culture) REALLY DON'T KNOW THE DIFFERENCE.  And some of the people
I've seen in molecular don't know as well.  They think they are
doing it ok or good enough.  After all... it's only research.

I've seen hundreds of thousands of dollars wasted in the design of
brand new labs... because the MD/PhDs doing the designing didn't
bother to call up specialist within some of these special
organizations/societies and inquire how to best do this.

Thus, we ended up with two floors of brand new labs with a chemical
fume hood in each room (including cell culture and molecular rooms
which are suppose to have a 'clean-room' type mentality in it's
design)... and yet, only buy a milli q water system... instead of
bothering to spend the money on a double-distilled, nanopure
system.

I could go on and on here.  Intelligent people doing some pretty
stupid things because they were too egotistical to call up some
people who knew better... and there was NOBODY out there to require
that they design it more properly.

Now... I'm not saying that after accreditation becomes the norm
that ALL labs which doesn't have a good lab design not become
accredited (just labs, for example, which works with human cells
and tissue and only has a class I hood).

But, if we require certain lab design standards as a norm in the
future... then newly re-modeled labs would have a blueprint to copy
off of.

Do you see what I am talking about here?

We need to set up standards for current work.. as well as standards
for future labs.

>Now, do you still think additional certification will solve ANY
>part of the issue at hand?

YES!... Because I am not talking about additional certification.
I'm talking about certification period!

For every single society/organization each PhD belongs to (and
submit grant proposals which requires these individual special
techniques)... they should have a certification in that
technique/specialty.  And have a lab which is accredited in those
techniques as well.  And if they aren't... then they HAVE to
collaborate with someone who does.  For example, an accredited
immunology lab which has a grant proposal that requires PCR work.
They either also become accredited in PCR/molecular, or,
collaborate with an accredited PCR/molecular lab.

It's really that simple.

>Also, I'm NOT SURE AT ALL that the cost of implementing such
>certification for every individual AND every research group (in
>the world?)

These societies/organizations which already requires a membership
rate would just have to charge more.

As for overall accreditation... that's would cost.  And we would
have to see how much this costs in a clinical setting to compare it
to the costs here.  But I think if they could do it... so can we.

>(which Berezin wants to be limited in size to three people (!),
>see earlier postings, though he later conceded he would accept 5
>or 6 people under exceptional circumstances if I recall) would be
>anything but an extremely heavy and costly process.

???  I don't know.

>FERLAND
>
>So, again, who do you suggest carry out the "certification
>process" you suggest? How *can* it be implemented other than by
>people themselves qualified? Would this not be a case of
>self-policing? And who qualifies the examinators? They would be
>the result of self-policing, too, because no others than peers can
>judge scientific or technical quality.

Again... look at the clinical/forensic models and go from there.

If they can do it... why not us?

Why are *we* above this?

-Kathy



More information about the Bioforum mailing list