View from the Trenches
tivol at WADSWORTH.ORG
Fri Aug 4 17:28:38 EST 1995
> The above may be true for some highly specific projects with
> well spelled goals. The work you decribe above sounds more
> like R&D company development, rather than open-ended research.
True, but appropriate for the small grant applied for.
> Much (I believe, MOST) university research does not fit the
Possibly a more typical example is a grant (not yet
applied for) to develop electron crystallography on a high
voltage electron microscope. Like yours, the core is a set
of interrelated ideas; however, they *can* be properly form-
ulated at this time. There has been enough done to know how
to go about getting and analysing the data. That is not to
say that surprizes will not arise during the course of the
> The whole point that it is much more often than
> not that there is NO WAY to say beforehands if the
> idea is "good" or not. Was the idea of Pons and
> Fleishmann to search for nuclear reactions in
> deuterated palladium electrodes "good" or "bad" ?
An easy one. A few calculations (done subsequently
by others) show that there is no reason to expect the ex-
perimental set-up of Pons & Fleishmann to produce nuclear
reactions. Any competant study section would call this a
bad idea *unless* there was convincing evidence from pre-
liminary experiments that the conventional wisdom was in-
correct. If P & F had applied to NSF for a grant for cold
fusion with no calculations showing the feasibility of pro-
ducing nuclear reactions at room temperature and no results
suggesting that they would be successful, they would not
get a grant for cold fusion, again *unless* the study sec-
tion decided that they knew P & F were so competant that
the grant should be awarded. My point is that the bad
idea should overrule the decision based on who P & F are,
or who they know.
> Despite all the b/s thrown on them during 1989-1995
> we still don't know the answer. What we have are
> at best opinions, highly biased for the most part.
I believe the results from Caltech do provide the
answer. Those experiments are more than opinions.
> This is why the
> "track record evaluation" as described in
> more detailes by Donald Forsdyke asks referees
> (composed not only from seniors, but with a
> good share of juniors) to state:
> "What such and such ACTUALLY accomplished in
> science". And you may end up with somebody
> with 20 papers coming out better than 300+
I should read Donald Forsdyke's scheme for evaluation.
It is more difficult to evaluate track records this way, but
isn't doing things the correct way often more difficult? Try-
ing to evaluate how much of the work the PI did and how much
should be credited to (say) a postdoc is not going to be easy.
If a PI is good at gathering and directing good people, so the
lab is productive, is it a good idea to fund the lab just on
that basis? I'd guess "yes".
> > If you are arguing
> > for a granting system like the McArthur Foundation grants, which
> > are unrestricted money, I agree that there is a place for them, and
> > perhaps all research funding should be in the form of unrestricted
> > grants plus equipment grants, etc., so that competant researchers
> > can be assured of funding and they can then generate any ideas
> > they want.
> I have never said such nonsense. My point is how to
> the get the best mileage from the (very) lean budgets.
> Sorry, but the true science is NEVER done by
> (materially) happy scientists.
"Unrestricted" need not mean "big". Small grants to provide
supplies for research on any topic the recipient chooses, with sup-
plementary funds for expensive equipment was what I have in mind.
I don't think it's nonsense.
> The above is not convincing example for me. High energy
> physics (contary to most non-nuclear physics) is for a
> long time done by very large groups of people [ papers
> with several HUNDERD co-authors are not that rare).
The Segre-Chamberlain group consisted of about ten people.
That was about the right number for the 50's to 60's. My own re-
search at LBL was in few-nucleon experiments, with about six people.
This provided enough for two or three people for each 12 hour shift,
needed for the 24 hour-a-day operation of the cyclotron. Other
branches of physics probably lead to different group sizes, but I'm
not familiar with them.
>Well, of course,
> you have Leon Lederman, Carlo Rubia, etc but even they
> did not manage to save Supercollider [ I personally,
> believe - for good ].
The only bad part of the supercollider is the cost. The buil-
ding of the supercollider is, in fact, amply justified on scientific
grounds, and it would already be built if it could be done cheaply.
As it is, I agree that that much money can be used more effectively.
> As Don Forsdyke wrote in one of his papers:
> "to put your best ideas in the propoasls is
> to committ professional suicide. And this
> is not because referees can pirate your ideas
> (although this may happen), but because best
> ideas [ really innovative ] are almost certainly
> will not be understood by peers"
Yes, I've seen this.
> (his recent paper in "Accountability in
> Those who are norally successful in getting
> funded write proposals EXCLUSIVELY as students
> often write their term essays:
> "what 'they' [ reviwers ] want to see in my
> proposals which will likely satisfy them the most".
> (appologies before STUDENTS for the above
> comparision - actually, term essays are normally
> a much cleaner deal, while grantsmanship is
> almost invariably asslicking).
Here we agree on the problem.
> Again, to say that "proposal has merit"
> is a vicious circle strategy. If we so
> much disagree on the (value of) research
> ALREADY DONE AND PUBLISHED, how we can
> say anything reliable about the merit of
> work YET TO BE DONE. This is simply
> a nonsense.
I think you're wrong here. To use your example of the
supercollider, its scientific merits can be reasonably evaluated.
A study section can determine both the merit (will it work) and
the value (what good is it when it works) of a proposal, compare
these to the cost, and come to a reasonable decision. BTW, if
P & F wanted a small grant to explore the possibility of cold fu-
sion, were able to provide calculations of D-atom density in Pd
and the expected D-D reaction rates, and stated that the idea,
while perhaps improbable, held out the possibility of great
reward, I'd be inclined to fund the work.
It is no more difficult to evaluate the merits of re-
search already done and published, than it is to evaluate the
track record of a researcher based on what is already done &
published--the latter is, after all, only a subset of the former.
> In our system (Canadian NSERC) it is largely
> irrelevant if you were or not assessed properly.
> As I have explained ours in a SINGLE GRANT system
> working on a SELECTIVITY principle. Every year
> it leave about 2,500 professors COMPLETLY unfunded.
Although the numbers of unfunded professors and the
nature of the all-or-nothing process leave a lot to be desired,
If you were over-assessed, you will be funded, and if under-as-
sessed, you will not. IMHO, the accuracy of assessment is very
important. It is also true that the system you describe has
fundamental flaws unrelated to assessment.
> > Of course,
> > if researchers are not funded, their ideas will not be implemented
> > (at least by them).
> FORTUNATELY, NOT SO. People doing mostly theoretical,
> or computer work can get by paying for most critical
> expenses (hardware, newtworking, publishing, etc)
> from their personal salaries [ not tax-deductable,
> though ]. I do know a case of a biochemist who
> re-mortgaged his hourse to continue on-going research,
Not necessarily true here. Often, promotion and tenure decisions
are based on successful grant funding. Our institution recently put in
a requirement that all PI's must have grant funding for their research,
otherwise their PI status is removed and their research is directed by a
supervisor, or they can be removed from research altogether. This holds
true for theoretical work as well. Even re-mortgaging one's house (I as-
sume not horse :-)) will not be sufficient unless one were to work on one's
own time using one's own equipment.
> There are no really unsolvable problems in the
> implementing all the above (e.g. young scientists, etc)
> within the sliding scheme(s), track record review, etc.
> Even quantifyable schemes were suggested (e.g. Rostom Roy),
> to minimize subjectivity/bias factors.
Yes, but these schemes are not used. Instead, it's who-you-know.
If the track record review was to be done correctly--that is, the highest-
rated would be the most productive--then funding those with the best track
records would make good sense. However, this always seems to decend into
> > My
> > contention is that the detailed plans will still work, regardless of who
> > proposes them.
> I am sorry to disagree - my experience that it is very
> much depends WHO handles the idea. Again, the above
> example from high energy phisics may not be the
> best here (due to the diminished human factor in very
> large groups) but for the rest (of science) I certainly
> would object your "regardless".
The nub of science is that the outcome of an experiment is independent
of the observer. I know some people who would not be able to implement an
idea, but they would not be able to write up the detailed plans either. My
contention is that anyone who can set out an experimental protocol which is
convincing to a study section should be allowed the opportunity to pursue it.
> In general terms, it is a rather philosophical
> discussion: ("are DIS-covering the laws of nature
> or CREATE them" ?).
I'm on the side of the existance of objective reality; thus, nature
is what it is, and we are discovering what already exists.
> Yes, of course. But in many cases it takes not
> too much. Plus there is a lot of cooking books
If I can set out a protocol, and if I use a cookbook (or send out
the samples to have gels run) won't I get results which can be duplicated
by anyone else following the same procedures? Isn't that what makes for
good science? If the ideas have both merit and value, who cares which
person did the work? If I am confident that any lab can reproduce the
same results, how does it matter that the researcher has a Nobel Prize?
> Yes, these are the pipe dreams we all seem to share.
> Fine. But there is absolutely no way to move anywhere
> from "competition to cooperation" (with all
> qualifications on these words) unless the system
> recognizes some kind of basic funding for ALL active
Or, at least, many.
>at least those who are in the univeristy
Salaries and supply $ used to be provided
by universities, and it would be a good idea if this were done again.
If necessary, give block grants to each university to accomplish this.
> anyone having 3 peer reviewd papers
> for the last 5 years is entitled on
> basic grant of $ 3,000 per year (US)
> or $ 4,000 (Can).
Not too tough for PI's, especially if account is taken of other
measures of productivity (for example, no papers written because of a
published text, not pre-peer-reviewed, but generally accepted as good).
I also assume that straight-forward arrangements can be made for cases
where productivity is not in doubt, but the numbers aren't there.
> Highly primitive; LOOKS like
> welfare (but not, it is NOT welfare),
> but at least give us a point to start.
It is payment for services rendered; i.e. money to do research
based on research done over the past 5 years. Neither primitive nor
More information about the Bioforum