DISCLAIMERTHIS IS NOT AN OFFICIAL DOCUMENT OF THE UNIVERSITY OF BIRMINGHAM OR THE SCHOOL OF COMPUTER SCIENCE. NEITHER THE UNIVERSITY NOR THE SCHOOL HAS ENDORSED THE OPINIONS EXPRESSED HERE. NEITHER HAVE THEY BEEN INVITED TO DO SO.
The Engineering and Physical Sciences Research Council, which claims
to be "The UK Government's leading funding agency for research and training
in engineering and the physical sciences" recently announced changes
in Funding Application procedures and review procedures. See:
http://www.epsrc.ac.uk/ResearchFunding/Changes/default.htm
One of the changes that has provoked much discussion is "Including
Economic Impact in Funding Applications" as described here.
There has been a lot of discussion of this on the both the CPHC
(Council of Professors and Heads of Computing) and the UKCRC
(UK Computing Research Committee) mailing lists, especially concerns
about the apparent intention to devalue long term (so-called "blue
skies") research, which does not have any short term economic
benefits, although the EPSRC web site states
"Quality of research remains the key criterion of funding but
these changes will give applicants a further opportunity to
demonstrate the importance of their work and to ask for
resources to support impact and dissemination routes."
I felt there were some points wrongly taken for granted in the
discussions and other points that were ignored altogether (e.g.
about the relations between research and teaching at University
level) and circulated the following missive to CPHC on 19th March
2009.
It will be modified from time to time in the light of comments and
criticisms. (In particular I need to add something about academic
and research staff recruitment procedures in UK universities, which
I think are seriously broken, partly because they generally require
an appointment to be made by deadline instead of requiring an
appointment to be made when a good enough person is available, and
because academics are not able to devote enough time and energy to
selection processes, and don't always give enough priority to
investigating quality, as opposed to very crude indicators of
quality.)
1. Much has been said about pressures towards research with short
term benefits and away from research whose benefits will occur only
in the long term. But there are also obvious benefits and unobvious
benefits, and in the latter case it may be impossible to tell in
advance whether solving some hard problem will ever produce
benefits, nor how long it will take. We all know stories about
erroneous predictions.
For several years I had the privilege of being mentored by Gerry Martin,
one of the founders of Eurotherm (heating controls) and a number of
other engineering companies.
For more on his life and work see
http://www.alanmacfarlane.com/FILES/gerry.html
One of his observations was that governments typically did not
understand the variety of types of feedback loop, in which effects
could have many different time scales and and whose effects could
vary in scope/impact/uncertainty.
As a proposal reviewer I know that people who predict the benefits
of their research are either foolish or deceitful, except in rare
cases.
An implication of the unpredictability is that research funding
*has* to be, to a large extent, a gamble. So what the government
should be doing is working out a good way to spend money gambling,
since they already approve of lotteries.
2. The current government policies for using reward-based management
of public services, including research activities, is analogous to a
policy of regularly monitoring the underwater supports of a major
off-shore platform or bridge, and redirecting resources to the
maintenance of the pillars that are in good condition.
[You've heard that before.]
3. The whole discussion on this list so far starts from the
assumption that most research should be funded from competitive
research council grants, as opposed to being funded by direct
allocations to universities to be used according to criteria decided
by the universities (including local competitions where
appropriate).
It would be far less wasteful for nation to try to work out how many
research universities it needs (or can afford or wants to afford --
along with various other post school educational institutions that
are not necessarily involved in research) and provide them with
sufficient funding to ensure that at least about 75% of academic
staff can get on with most of the research they need to do without
having to waste huge amounts of time and emotional energy every year
competing for grossly inadequate amounts of money, with the dreadful
consequence that local evaluations of individuals (e.g. in promotion
panels) give more weight to financial success than to doing good
research on hard problems.
A further 20% (or more, or less) of academic staff will focus mainly
on teaching and admin: Every great university needs to be a great
teaching institution, and that requires a subset of individuals
willing to put heart and soul into ensuring that that things like
examining, time-tabling, teaching-allocations, monitoring, student
selection, personal-tutoring, etc. etc. are done well.
And about 5% (or more, or less) should perhaps be doing research on
'big' projects that need to be funded and monitored at national
level. There could also be funding from industry and other
organisations for projects closely related to their needs.
(Of course, this will be strongly opposed by people in departments
that are most successful at getting research council grants. That's
the self-interested response, not a response based on trying to
design a good national system.)
4. To ensure that the directly allocated tax-payers' money is not
wasted, all research active departments should have internal and
external research reviews with constructive feedback from the
reviewers, along with proper mentoring of young researchers and
mutual mentoring of older researchers.
This micro-management with constructive and critical feedback should
produce far better results than the occasional shallow and expensive
national review that leads to decisions about allocations of large
sums of money for the next 5 years.
No sensible engineer would design a control system like that. It's
like trying to drive a large vehicle by opening your eyes and
holding the steering wheel for about twelve minutes in every hour.
It may also be better for most research money to be spent in
departments that have the power to redirect resources if things are
not going well, instead of being allocated in large three year (or
longer) chunks where it is only after the three years that any
evaluation of the project is done.
5. One of the most important functions of a university is to attract
and *educate* students who are going to be leaders, ground-breakers,
revolutionaries, and solvers of large and important problems.
The teaching of such people must involve an apprenticeship role,
working closely with people who are tackling hard problems, and
who can look at the learner's efforts in detail and provide
constructive criticisms and suggestions, while allowing the learners
to do the same in reverse. (I've just had some interesting critical
comments on one of my research papers from a final year
undergraduate).
That apprenticeship role cannot be replaced by giving students
second-hand reports of the results of research done by others,
anymore than you can teach someone to be a great violin player, or
athlete simply by presenting recordings, videos, lectures and notes
on what others have done.
This means that one of the most important effects of doing hard
research, independently of the importance of the *content* of the
research is the development of research skills and techniques and
being in a position to help the next generation acquire them.
A related result, which I am sure you have all observed, is that
someone not working on problem X can hear presentations given by
people who are working on problem X and make very useful critical
comments based on experience of working on other hard problems.
That can lead to improvements in research on X.
I suggest that for the *majority* of research done in universities
the side effect of providing an environment for apprenticeship-based
intellectual training and development of the ability to make useful
critical comments on research done or planned by others is FAR more
important than production of direct economic or social benefits of
*content* of the research, however much academics may like to think
they can justify public funds being used to produce the results they
report in workshops, journals, seminars, etc.
[Just read some of them!]
I wonder if anyone has tried to evaluate the impact of having had a
period of close contact with teachers who are active researchers
(and who care about teaching) on the quality of thinking of
ex-students who work in commerce, industry, government service,
school-teaching, and bringing up children.
6. Of course there is another kind of benefit of research, which is
extending knowledge and understanding of some aspect of reality, or
of some class of problems (as opposed to making yet another widget).
But that is comparable to production of great works of art:
including being hard to evaluate at first, and having its own worth
independent of any measurable economic or social benefit.
As we know well, the ability of large numbers of people to
appreciate the worth of advances in knowledge and to enjoy learning
about them and talking about them, can be diminished by a crappy
educational system, religious indoctrination, etc.
Summary:
1. The whole discussion assumes a research funding mechanism that is
badly broken as a control mechanism.
2. There are aspects of doing research that make it a worthwhile
thing to keep alive in universities that are independent of the
importance of the research results. For the other results to be
achieved it is more important that people be working on *hard*
problems requiring major intellectual advances of some kind, than
that they be working on problems of economic or social importance.
3. Offering young lecturers jobs involving teaching and research,
then expecting them to waste their precious formative years chasing
money instead of chasing solutions to hard problems is highly
immoral, and counter-productive both for research and for university
teaching.
Unfortunately these points are unlikely to be understood by
politicians and civil servants who are themselves products of an
educational system that is badly broken, especially people who have
never had any personal experience of trying to specify, design,
test, debug, analyse, extend and document a complex *working*,
formally describable, system of any kind, and who consequently think
that if you describe some desirable goal at a high level of
abstraction, and pay lots of people to achieve it, then it will be
achieved.
(It is appalling how often government ministers answer questions
about how some complex goal is going to be achieved by saying that
their policy is to achieve it and they are legislating for it and
putting money into it. The really frightening thing is that they
believe what they are saying.)
I previously produced a document arguing that insofar as there is a research funding mechanism that cannot possibly fund all worthwhile research, it should use a dynamically weighted lottery mechanism, described here: http:/www.cs.bham.ac.uk/~axs/lottery.html
Maintained by
Aaron Sloman
School of Computer Science
The University of Birmingham