View from the Trenches

Alexander Berezin berezin at MCMAIL.CIS.MCMASTER.CA
Fri Aug 4 13:59:45 EST 1995

Dear Bill, Kathy and all others:

It looks (at least to me) that this conversation
is getting more and more sense as we are indeed 
discussing (let controversial) but IMPORTANT
issues. As often happens is dialogues (actually,
polylogues) like this one, a lot of disagreement is
actually misunderstanding, deficeincy of expressing
points, etc. Some of my "comments-on-comments-
on-comments... " are below.

Yours - Alex Berezin 

On Fri, 4 Aug 1995, William Tivol wrote:

> Dear Alexander,
> > 
> > Why PEOPLE (and not the "ideas") should be funded:
> > 
> > I can't quite identify who said what in the attached
> > exchange, so I give a summaraly reply to this
> > (attached) poster. 
> > 
> 	All my contributions are indented.  The others are from Kathy,
> with a single ">" and another contributer, with ">>", etc.
> > Except of the Platonic world, the existence of "ideas"
> > per se has no (or very little bearing) on what we are
> > actually doing.
> 	I disagree.  What we are doing--at least in the context of
> what the grant funds are to be used for--is based on an idea and the
> details of how it is to be worked out.  For example, I applied for a
> small grant to develop an enhanced non-laser light source.  This kind
> of light source would be ideal for certain projects in confocal micro-
> scopy.  My idea was to modify a mercury-vapor plasma using magnetic
> fields, so as to produce a higher emission density (photons per mm**3).
> This idea was precisely what I would do if the grant was funded.

The above may be true for some highly specific projects with
well spelled goals. The work you decribe above sounds more
like R&D company development, rather than open-ended research.
Much (I believe, MOST) university research does not fit the 
above. My topic, for example, is "to explore randomness and
self-organization due to isotopic diversity". This is not
just "an idea" but the whole bunch of interrelated issues,
much of which can only be properly formulated in the
course of a study. 

> > Ideas, like AIDS (sorry for comparison)
> > exist only in PEOPLE. So, any sensible (funding) system 
> > should primarily fund PEOPLE (with demostrated capacity 
> > to generate ideas), so they can do precisely this - generate
> > and TEST the ideas. 
> > 
> 	But if the idea is good, shouldn't the person be funded re-
> gardless of whether (s)he has ten publications or 1000?  

The whole point that it is much more often than
not that there is NO WAY to say beforehands if the
idea is "good" or not. Was the idea of Pons and 
Fleishmann to search for nuclear reactions in 
deuterated palladium electrodes "good" or "bad" ?

Despite all the b/s thrown on them during 1989-1995
we still don't know the answer. What we have are
at best opinions, highly biased for the most part.  

>Do you
> believe that anyone with 1000 publications should get
> funding, even
> though the ideas (s)he proposes make no sense?  

Of course not. I never was for any publication
scoring. Furthermore, many of the holder of
these magnificient CVs with 300-400-plus
papers are nothing but high class copycats for
thier entire careers. This is why the
"track record evaluation" as described in
more detailes by Donald Forsdyke asks referees
(composed not only from seniors, but with a
good share of juniors) to state:

"What such and such ACTUALLY accomplished in
science".  And you may end up with somebody 
with 20 papers coming out better than 300+

This is why many fat (copy)cats will resist
this kind of REAL evaluation, as opposed to
fictionality of page counting of present
evaluations, etc. [ of course, none of
them admit page counting explicitly , but
they all do it ].  

> If you are arguing
> for a granting system like the McArthur Foundation grants, which
> are unrestricted money, I agree that there is a place for them, and
> perhaps all research funding should be in the form of unrestricted
> grants plus equipment grants, etc., so that competant researchers
> can be assured of funding and they can then generate any ideas
> they want.

I have never said such nonsense. My point is how to
the get the best mileage from the (very) lean budgets.
Sorry, but the true science is NEVER done by
(materially) happy scientists. 

> > But at least as far as fundamental core sciences
> > like physics [my PhD is in physics, BT] or mathematics are
> >concerned, the notion "fund ideas, not people" is comple-
> >tely bankrupt by history, it NEVER worked and NEVER WILL.
> 	The construction of the Bevatron at LBL was proposed specif-
> ically to produce and detect the antiproton.  This project would have
> made sense even if Segre and Chamberlain were not the ones who proposeed
> it, and, in fact, if it were not for Clyde Wiegand's expertise in the
> design of the electronics, the antiproton might not have been found.
> Clyde was not a household name, and if he proposed the Bevatron, it
> might not have been funded.

The above is not convincing example for me. High energy
physics (contary to most non-nuclear physics) is for a
long time done by very large groups of people [ papers
with several HUNDERD co-authors are not that rare).
In my observation as a physicist (not nuclear, though),
the personality role in nuclear physics is relatively
unimporatnt (at least, presently). Well, of course,
you have Leon Lederman, Carlo Rubia, etc but even they
did not manage to save Supercollider [ I personally,
believe - for good ].

> > All history of peer review demostates that it has no capa-
> >city  whatsoever in forecasting the "best" ideas better
> > than random. Pre-approval of "good ideas" by peer
> > review is a PROVEN nonsense. 
> > 
> 	But study sections routinely comment on the ideas presented
> in the proposals. 

As Don Forsdyke wrote in one of his papers:

"to put your best ideas in the propoasls is
to committ professional suicide. And this
is not because referees can pirate your ideas
(although this may happen), but because best 
ideas [ really innovative ] are almost certainly
will not be understood by peers"

(his recent paper in "Accountability in

Those who are norally  successful in getting 
funded write proposals EXCLUSIVELY as students
often write their term essays: 
  "what 'they' [ reviwers ] want to see in my
proposals which will likely satisfy them the most".  

(appologies before STUDENTS for the above
comparision - actually, term essays are normally
a much cleaner deal, while grantsmanship is
almost invariably asslicking). 

> My quarrel is with a study section which says,
> "The ideas in this proposal have merit, but we don't think your lab
> will be able to do the work."  While on the other hand, a famous lab
> is not questioned. 

Again, to say that "proposal has merit" 
is a vicious circle strategy. If we so
much disagree on the (value of) research
say anything reliable about the merit of
work YET TO BE DONE. This is simply
a nonsense.

(except, perhaps highly specific R&D

> Other posts in this thread pertain to the relation
> between "famous" and "competent" (or the lack thereof).  Especially
> since the famous scientists often do little of the work themselves,
> this kind of evaluation does not seem to be justified IMHO.

See above on copycats and track evaluation.

> >  However, peer review IS CAPABLE to assess (better
> > than random) the ability of specific people generate
> > those ideas which actually can (and were) IMPLEMENTED.
> 	I thought your observation of the Canadian system was that
> many competent researchers were not assessed properly. 

In our system (Canadian NSERC) it is largely
irrelevant if you were or not assessed properly.
As I have explained ours in a SINGLE GRANT system
working on a SELECTIVITY principle. Every year
it leave about 2,500 professors COMPLETLY unfunded.

THis is not far from Stalin's purges when the norms
on how many per-cents of the population ARE the
"enemies of the people" where assingend from the
the Center to regional NKVD sondercommands 
(usually 10 to 15 %).

Likewise, NSERC GSC's (Grant Selection Committee),
have a norms of ca. 30 % to cut from the ESTABLISHED 
researchers (ca. 50 % for the first time applicants).

Some decent NSERC Committee members have protested
to the bureaucracy on these quatas but of no avail.   

> Of course,
> if researchers are not funded, their ideas will not be implemented
> (at least by them).

FORTUNATELY, NOT SO. People doing mostly theoretical,
or computer work can get by paying for most critical
expenses (hardware, newtworking, publishing, etc)
from their personal salaries [ not tax-deductable,
though ]. I do know a case of a biochemist who 
re-mortgaged his hourse to continue on-going research,

> > Contary to futuroloical crystal balling over the 
> > UNDONE work ("proposals evaluation"), the assessment of
> > TRACK RECORD is a fully feasible task 
> 	Agreed; the record is there and can be evaluated.
> >  
> > (and give me a break with people with "good ideas,
> > but without track record").
> > 
> 	That's who the "young investigator" grants are for.  But
> allowances should be made for those who have been unable to get
> grant funding, and have relatively few--but significant--publications.
> These people do not look as good on paper, but deserve an opportunity.
> Perhaps some measure of track record (papers-per-grant-dollar) can
> suitably account for them.  If they have had grant after grant and
> produced nothing, then either their ideas are not good, or they can't
> implement them.

There are no really unsolvable problems in the
implementing all the above (e.g. young scientists, etc)
within the sliding scheme(s), track record review, etc.
Even quantifyable schemes were suggested (e.g. Rostom Roy), 
to minimize subjectivity/bias factors.

> 	BTW, my use of the word "idea" means not just the abstract core
> ("It would be good to build an accelerator to find the antiproton."),
> but the detailed idea ("To find the antiproton, we would need to acce-
> lerate protons to >6.2 GeV and detect them in a proton-proton scattering
> experiment with detectors capable of discriminating an antiproton-
> producing event from the background counts.").  This complete kind of
> idea can be evaluated on the basis that the proposed reaction (p+p ->
> p+p+p+anti-p) makes sense, the calculations are correct, the accelerator
> design is soundly based, the detector array will do the job, etc.  

> My
> contention is that the detailed plans will still work, regardless of who
> proposes them.

I am sorry to disagree - my experience that it is very
much depends WHO handles the idea. Again, the above
example from high energy phisics may not be the 
best here (due to the diminished human factor in very
large groups) but for the rest (of science) I certainly
would object your "regardless". 

In general terms, it is a rather philosophical
discussion: ("are DIS-covering the laws of nature
or CREATE them" ?).

I think, the modern philosophy of science splits
on this issue in approximately 50-50 ratio.

> > Anyone can generate plenty of "good ideas". 
> > In 1726 (!) Johnathan Swift in "Gulliver Travels"
> > described an "idea generation machine" used by
> > the Laputa scientists (randomly rotating cubes
> > put random key words together - and it 
> > works pretty much the same way as many actually
> > do now).
> > 
> 	But to make the ideas complete, one needs more than dice,
> unless the idea is for a Monte Carlo calculation.  The "idea generation
> machine" only generates the abstract core--and it may be perfectly good
> for that--but to flesh out the idea requires calculations, some connec-
> tion to previous knowledge, and an experimental plan.

Yes, of course. But in many cases it takes not
too much. Plus there is a lot of cooking books 

> > Once again - if we (you) want to do something
> > seriuos about the present NIH/NSF/NSERC/MRC mess,
> > the only way to do it is to fund MORE people
> > on (often) significantly LESSER levels than
> > at present. Of course, grantsmanship elite
> > will fiercely resist, but you have no choice
> > but to fight for this cause.     
> > 
> 	I agree 100%.
> > Ny estimate is that unless we will do the
> > above (reform the system from competition to
> > coopreation and mutual support rather than
> > internal grantsmanship warfare),
> 	Your very best point!  Cooperation produces much more than
> competition as far as science is concerned.  A system which encourages
> scientists to work together openly is one which will lead to the best
> use of resources.  The J-psi particle story, where one researcher lied
> to another so that he might get priority for the discovery was a low
> point for physics IMHO.  Although it is good to have two labs using
> different techniques both discovering the same thing, there is no need
> for these labs to be in competition.  In fact, more confirmation and
> independent duplication of results would uncover sloppiness and fraud,
> but this kind of work is not funded, because it's not original.  The
> competative system has many flaws, which you have pointed out.

Yes, these are the pipe dreams we all seem to share.
Fine. But there is absolutely no way to move anywhere
from "competition to cooperation" (with all 
qualifications on these words) unless the system 
recognizes some kind of basic funding for ALL active
researches, at least those who are in the univeristy
system (yes, I am sorry, I am leaving off board
many diserving outsiders of this system, but we can't
even start moving unless we fix this first. In
air planes "put your oxygen mask first and then
you will be able to help neighbour").

Yes, you can question what means "active 
reasearcher", criteria, etc. Let's set
first ground proposal for NSF/NSERC:

anyone having 3 peer reviewd papers
for the last 5 years is entitled on
basic grant of $ 3,000 per year (US)
or $ 4,000 (Can).

Highly primitive; LOOKS like
welfare (but not, it is NOT welfare),
but at least give us a point to start.

NSERC/NSF guys, do you hear us ?        

Alex Berezin

> 				Yours,
> 				Bill Tivol	

More information about the Bioforum mailing list