Peer Review: 2nd Reply to Berezin
Gregory R. Harriman
gregoryh at bcm.tmc.edu
Sat Nov 4 16:13:45 EST 1995
In article <Pine.3.89.9511031850.A15341-0100000 at mcmail.CIS.McMaster.CA>,
berezin at MCMAIL.CIS.MCMASTER.CA (Alexander Berezin) wrote:
> > HARRIMAN:
> > I guess I have trouble understanding English. But what you wrote in a
> > previous post was: "anyone who wants to do an experiment should be
> > funded".
> Assuming, of course, he/she is:
> (1) qualified to do the proposed type of research
> (2) do have reasonable conditions to do this, including
> being employed at teaching position in university/college,
> etc, having access to the equipment, etc
> (3) having track record to warrant the funding.
Thanks for clarifying your position. You certainly seem to be more
restrictive now in who should get funding than your previous posts
suggested. I am still not persuaded that your criteria of determining
whether someone is "qualified to do the proposed research" and 'having
track record to warrant the funding" represents a significant improvement
over the current system.
> You may say that the above conditions are tough. Yes,
> they are. But paradoxically under the present system
> (so called "proposal competition"):
> 1) many people who DO satisfy all the above
> conditions are nonetheless unfunded
> 2) many who do NOT satisfy [ mostly point 3 ] are
> nonetheless funded
> [ how to check for point 3 see below ]
Are you saying in 2) that people who do NOT satifsy point 3 (ie. don't
have a tract record) should not receive funding? So, regardless of what
they want to do with the money (no matter how good or how bad their plans
are), no funding if they don't have a tract record, but funding if they do
have a track record? Seems to me this presents an opportunity for abuse
and/or misuse of research funding which is every bit as bad as the current
> > HARRIMAN:
> > Where I am having difficulty is in understanding how you propose to ASSESS
> > whether researchers have "clearly demonstarted (sic) that they are able to
> > do research". It seems obvious that someone (a peer perhaps?) would have
> > to determine this. How does this differ in a meaningful way from peer
> > review?
> Very much differ. Peer review (in a regular meaning) deals with
> a single item (manuscript or a grant proposal). Under the present
> system the item can be rejected ALTOGETHER (paper denied publication,
> grant application not funded).
> When you assess LIFE-TIME record this much less likely to happen.
> The only time it CAN happen is when the assessors (call them peer
> reviewers or not) are ALL IN UNISON saying:
> "Soandso in a totally useless researcher, discovered
> nothing, all his/her work worth nothing at all".
So, unless there is total unanimity of the "assessors" that Dr. Soandso is
a totally useless researcher, he should continue to get funding.
> The difference from peer review that the committee
> assessing track record does NOT need to be a very narrow
> specialized in the applicant's area (hence arm-length
> rule applies and very low, if any, possibility of
> conflict of interest).
Maybe so, but I still don't accept your premise that scientists from
entirely different disciplines will necessarily be qualified and able to
assess the accomplishments of someone from outside their area of
> The proposal you are talking about (anonymous submission of
> manuscripts and/or grant proposals) is often discussed, however
> there are fundamenat flaws in it:
> (1) the author almost invariably can be identified
> through the bibliography
> (2) the MAIN point that the anonymity of the reviewer
> essentially realeases him from ANY responsibility for
> the process is not addressed.
These are valid points.
> It is virtually impossible that someone with a weak track
> record came up with really strong proposals. The only possible
> exceptions are some "junior geniuses" who can come with indeed
> something promising.
I find it quite disturbing that someone who appears to be so concerned
with the fairness and objectivity of the peer review process can make an
unequivocal, a priori judgement about the ability of someone to write a
good research proposal without ever seeing such a proposal and with no
statistics to back up the statement. It certainly seems that the only
thing you are trying to advance is your own preconceived ideas of how the
peer review process should be controlled. I'll take a study section that
is willing to objectively evaluate my research proposal on its merits over
your panel of closed minded, prejudging "assessors" any day.
More information about the Bioforum