Peer Review: 3rd Reply to Harriman

Alexander Berezin berezin at MCMAIL.CIS.MCMASTER.CA
Sat Nov 4 18:12:31 EST 1995

More clarifications on some points.
(a lot deleted) - Alex Berezin

On 4 Nov 1995, Gregory R. Harriman wrote:

> In article <Pine.3.89.9511031850.A15341-0100000 at mcmail.CIS.McMaster.CA>,
> berezin at MCMAIL.CIS.MCMASTER.CA (Alexander Berezin) wrote:
> Are you saying in 2) that people who do NOT satifsy point 3 (ie. don't
> have a tract record) should not receive funding?  So, regardless of what
> they want to do with the money (no matter how good or how bad their plans
> are), no funding if they don't have a tract record, but funding if they do
> have a track record? Seems to me this presents an opportunity for abuse
> and/or misuse of research funding which is every bit as bad as the current
> system.

You forget very important point, that is that I (along with
Donald Forsdyke) strongly advocate sliding funding scale (SFS).
Instead of sharp cut-offs SFS provides funding on the basis
of sigmoid (continuous) curve. This allows people to move
up and down (depending on the performance) and as such will
represent a genuine (interactive, and much more fair) 
competition than the (largely false) Olympic type competition 
(yes or no) which present NIH/NSERC/NSF system uses so

Having bracketted ranking (instead of yes or no decisions)
greatly reduce the level of abuse you are concerned about.

(I suggest very much that you read Forsdyke's papers on
sliding scale as they explain more details).

> > When you assess LIFE-TIME record this much less likely to happen.
> > The only time it (zero funding) CAN happen is when the 
> > assessors (call them peer reviewers or not) are ALL IN UNISON 
> > saying:
> > 
> >   "Soandso in a totally useless researcher, discovered
> >    nothing, all his/her work worth nothing at all".
> So, unless there is total unanimity of the "assessors" that 
> Dr. Soandso is a totally useless researcher, he should continue 
> to get funding.

Yes. On a SFS Soandso will be awarded very small grant,
perhaps just $1,000 per year to cover networking expenses
to seek avenues for coopearation, etc. But certainly
nowhere near the average grant amount.   

> ... but I still don't accept your premise that scientists from
> entirely different disciplines will necessarily be qualified and able to
> assess the accomplishments of someone from outside their area of
> expertise.

No, not "entirely different". I don't expect linguists be 
on panels to assess biochemists. But within broadly defined 
disciplines this is not only entirely possible but much 
preferable to a close-range assessmen (narrow specialists, 
i.e. direct competitors), as it virtually eliminates conflict 
of interest problems. This is why I believe that peer review
(regardless of anonymity issue) should be at a much greater
"arm-length" than it is now.

> Stuff deleted.
> > The proposal you are talking about (anonymous submission of 
> > manuscripts and/or grant proposals) is often discussed, however 
> > there are fundamenat flaws in it:
> > 
> > (1) the author almost invariably can be identified
> > through the bibliography
> > 
> > (2) the MAIN point that the anonymity of the reviewer
> > essentially realeases him from ANY responsibility for
> > the process is not addressed. 
> These are valid points.
> > It is virtually impossible that someone with a weak track
> > record came up with really strong proposals. The only possible
> > exceptions are some "junior geniuses" who can come with indeed 
> > something promising.
> I find it quite disturbing that someone who appears to be so concerned
> with the fairness and objectivity of the peer review process can make an
> unequivocal, a priori judgement about the ability of someone to write a
> good research proposal without ever seeing such a proposal and with no
> statistics to back up the statement.  (.........) 

Again, in SFS (sliding funding scale) this is largely a 
pseudo-problem. You don't expect anyone entirely without 
track record applying for independent research funding. At 
least PhD thesis and a couple of papers should be there (very
unlikely anyone will be hired to teach at university
without having those items). So, start from this (small
grant). Why do you need proposals ? Live it up to the
risk, if no advancements (results) for 2 or 3 years than
bye bye. (for filing purposes, you, of course, can request
1-page proposal, but I see little point in evaluating it
as it is largely a futurology anyway).

> Greg Harriman

More information about the Bioforum mailing list