berezin at MCMAIL.CIS.MCMASTER.CA
Fri Dec 8 15:54:25 EST 1995
On Fri, 8 Dec 1995, William Tivol wrote:
> > BEREZIN:
> > 1) The scheme of blind proposals (reviewers don't know who is
> > the applicant), even if it is used somewhere now, is a very
> > weird ideas for [ at least ] two reasons:
> > A. Any serious proposals must be supported the explicit
> > credentials of the proposer.
> Here is where I disagree. Especially in a bicameral
> review, the proposal can and should stand or fall on its own.
> The "nature of space & time thought experiments" proposal, for
> example should be shot down if only that one sentence were
> included. Of course I would expect either Hawking or the
> unknown patent clerk would have much more to say, such as that
> considerations of momentum and energy in both moving and sta-
> tionary coordinates must give the same results, and that, since
> it is an experimental fact that no differences can be observed
> in the speed of light in a vacuum, a reasonable hypothesis is
> that the speed is constant in any frame. Thus, the thought
> experiments (or, to update a little, computer simulations) will
> tell us the consequences of these assumptions. I see no reason
> why such a well-thought-out proposal must have the PI's identi-
> fication attached.
In short the message is that good, well done proposals
showed be given preferential funding. No, I don't believe
it is generally working any better than predicting that
the "nice looking girl will be the most happy one".
My next passage expalins more why not.
> I have just finished reviewing some proposals
> which were in two cases very clearly thought out and in the other
> three were equally clearly fuzzy. The former cases gave detailed
> calculations in areas where problems might arise; whereas the
> others did not seem even to appreciate that there were potential
BEREZIN: Here the silent assumption that "clearly thought out
proposals" is somehow inherently better and/or have more
chance to bring the (promised) result than the "fuzzy"
proposals. I don't believe that the history of science
give any whatsoever confirmation to this. Proposal game
rewards good writers, and this is not at all the same as
good dicsoverers. The "fuzzy" proposal (end on 15 century)
was "Sail to the West to find perhaps some land". It
worked pretty well althogh peer review would undoubtedly
> It is just as bad (or worse) to give money to a
> competant researcher who will not give sufficient thought to an
> idea than to an unknown.
In some very specific areas of commercially-oriented
research proposals perhaps may have some sense.
But certainly not in basic sciences. Here, unless
there are some warning signes (like misconduct, etc)
a competent and active researcher should be given some
funds to carry whatever he/she believes is important. Period.
No proposals are necessary at all (except of 1 page
for filing purposes, NOT for its "evaluation"). Evaluate
the RESULTS this researches has ACTUALLY deliverd in the
prior funding term, not the PROMISES s/he makes. You are not
requesting a surgeon to apply every time for the funding to
get his/her tools to perform an emegency sugrery. But this
is what proposal-based funding scheme requires from scientists.
With the present NSF/NSERC/NIH, etc scheme things will just
keep going progressively worse unless the fundamental principle
"Fund Researchers, Not Proposals" (FRNP) will be reinstated.
> Perhaps some of your misgivings can
> be addressed by appropriate procedures in the "summation" phase
> of the review process.
Summation process can be worked out sensibly only AFTER
the need to return to the FRNP principle is passed.
Then several schemes can be explored for the optimization.
> > B. Any sensible person just be offended to review
> > anonymous proposals as it implies that you (a reviewer)
> > is mistrusted. Issue of the reviewer anonymity is much
> > more involved and there are arguments and both side.
> > But authorless proposals is a clearly no-goer.
> I would not be offended, and I don't think anyone
> should be.
I am sorry, I may be hypersensitive. But I unlikely
will come again for a party to a house where they have a
habit of checking pockets of the departing guests for
the silverware. No, thanks, not for me.
> This is the same kind of thing as double-blind
> controls, and it assures that the analog of the placebo
> effect does not affect the results. I would guess that there
> would be few surprizes when the authorless proposals were
> matched with the authors during the summation phase, but every
> once in a while, there will appear an unknown with a good
> proposal score or an established researcher who has clearly
> made a mistake (or who really doesn't understand what (s)he
> is proposing to do).
Regardless of our views on placebos (which well may be
very different), the above does not sound for me as
I strongly believe that the personality of the researcher
makes the difference. I am quite sceptical about the
idea of "objective truth" and more on a side of "interactive
Universe" (John Wheeler, Henry Stapp and others). The
only way to reasonably gap philosophical (inevitable)
differences is to have a robust enough funding scheme.
And we back again to FRNP and sliding scale.
> > 2) Sliding scale ... assures
> > at least some funding level is awarded for as far
> > as there is a passing score on either of the two.
> Yes, and this is very good. Researchers with
> good track records will be able to continue working,
> and unknowns with good ideas will be sufficiently
> funded to be able to develop a good track record.
If we truly agree on this all the above might be
just periferal misundersatnding on the verbalistics.
If so, we almost have a deal.
> > However, such odditties, as mathematicians say, have
> > almost zero measure (i.e. extremely rare).
> I'm not at all sure that the measure is small.
> I've had proposals shot down because the reviewers felt
> that although the ideas were good, they were not sure I
> had the expertise to carry them out. Perhaps everyone
> has had a similar experience at one time.
Ok, on this I can agree.
> > We have to postpone the issue "it it enough money" till
> > the distribution problem is addressed, i.e. sliding scale
> > is introdueced as a ground-level approximation. ONLY THEN
> > we will be able to responsibly assess are "they" give
> > enough money and if not how much more we need.
> True, only after the distribution process is in place
> for several years (maybe decades) will it be possible to assess
> how much has been produced per grant dollar and compare that
> with the productivity under the present system.
I don't think it will take that long. Most research operates on
2 to 3 year cycles. Track appointments are reevaluted generally
in two years and by that time people usually already published
the results. So, I beleieve, reliable concusions on FRNP can
be made in 2 - 3 years, not decades.
> > I have yet to
> > hear of a single case when the grant recipient refused to
> > accept the award because the award was "too low".
> I *have* heard of cases where the award was not suf-
> ficient to perform the research proposed.
"Not enough money" is the #1 complain for the last 6000 years.
Of course, THEY (researchers) can say this. But resaerch is
not a bridge construction where no contructor will take to job
if the estimated cost is 5 millions and he is given
only 2 millions. In research you have options and priorities.
"Less money than I need" means that you have to cut down
on lesser high priorities and the OVERALL quality is likely
to increase, not decrease (as all will be doing just the
best priority). Due to this effect ("funding paradox")
LESS money will means BETTER quality science. (Hope, this
is undersatndabel argument for the politicians).
> > So, at this point I see reletively little chance that this
> > issue (grad student & post doc funding) can be
> > meaninggfully addressed BEFORE the major curbing of
> > grantsmanship mentality is achieved through the
> > introduction of the sliding scale/bicameral review principle.
> Yes, this too must await a shift in political power.
"Improving science quality by cutting the budget" is
an excellent recommendation to sell to the politicians.
They should like it.
> Bill Tivol
More information about the Bioforum