Architecture-based vs reward-based motivation in biological evolution and development, and future robotics

( Human Computer Interaction (old) Series )

Aaron Sloman, School of Computer Science

Date and time: Tuesday 5th March 2013 at 16:00
Location: Strathcona LT7
Host: Andrew Howes

Many scientists, philosophers, and AI modellers/designers assume that all motivational mechanisms must make use of expected or estimated rewards in order to create, or choose between, motives. I'll argue that that assumption is based on (a) a failure to imagine alternative designs, (b) a failure to attend to empirical evidence, (c) a failure to appreciate how architecture-based motivational mechanisms can be superior to reward-based motivational mechanisms in some contexts.

As David Hume pointed out long ago, control mechanisms are required in addition to factual information and reasoning mechanisms if an agent A, however knowledgable and competent, is to do anything. This talk is about what forms of control are required. The simplest cases are reflexes. In at least some cases motives are superior to reflexes, and the control arises out of selection of a motive for action. That raises the question where motives come from. They can be generated and selected in different ways, but one way is not itself motivated: it merely involves the operation of mechanisms in the architecture of A that generate motives and select some of them for action -- cognitive reflexes. Those mechanisms may have been selected by evolution because of their consequences (e.g. because they generally produce powerful learning in the environments in which the species evolved), but A knows, and expects, nothing about that.

The view I wish to oppose is that all motives must be explicitly chosen by A because they somehow serve A's desires, or are likely to produce rewards that A wants: this is the assumption that all motivation must be reward-based -- known as "Hedonism" in the history of philosophy. In contrast, I claim that at least some motivation is architecture-based: i.e. motive generation and selection is in some cases a kind of internal reflex, not a rational process, and this has biological benefits, which robot designers ignore at their peril.

The talk will expand on this discussion note: http://www.cs.bham.ac.uk/research/projects/cogaff/09.html#907