Posted Tue Jul 30 12:20:21 BST 1996
Newsgroups: comp.ai.philosophy

Posted Tue Jul 30 02:20:09 BST 1996
Newsgroups: sci.psychology.consciousness
References: <L6i5DHAGbh+xEAPT@imprint.zynet.co.uk>
Subject: Are we always in control? (Was Re: I can't see -- all is dark inside)

Although I don't understand much of what Keith Sutherland
<keith@imprint.co.uk> writes, there was a recent comment that
resonated with work I've been doing for several years (partly
inspired by an important paper written by Herb Simon in 1967,
'Motivational and Emotional Controls of Cognition', reprinted in his
{\em Models of Thought,} Yale University Press, 29--38, 1979).

Keith wrote:

> ...But at the same
> time, as Jim Newman pointed out so eloquently, all of us (well at least
> most of us, most of the time), have the experience that "we" are in
> control and taking our own decisions.  The depth and ubiquity of this
> certainty cannot just written off as some sort of pre-scientific myth or
> a "transient computational illusion" produced by the opacity of the
> system.

I've been trying to understand this notion of "being in control" for
some time. I'll describe some cases and sketch a first draft
(simplified) explanatory architecture, which may link up with work being
done by Jim, Bernie and others.

There are uninteresting cases of not being in control and a host of
diverse more interesting cases.

The uninteresting cases involve involuntary movements and reflexes,
e.g. the knee jerk, heart pumping, muscular contractions to maintain
posture, saccades, etc.

There are more interesting cases where a highly trained athlete or
musical sight reader does something automatically. There is no
consideration of options and explicit selection: the action just
happens.

Yet you don't feel out of control: you have deliberately put yourself
into the state in which you react automatically to various stimuli, and
you can turn off that state if you wish, and prevent the reactions:
staring at the music without playing.

Then there are cases where the expertise cannot be turned off, e.g.
automatic and simultaneous recognition of a phrase and its meaning, e.g.
"Paris in the spring". Can anyone turn off the processes of recognition
and understanding while looking at the printed text?

Of course, that can be achieved by brain damage, but being able to
shoot yourself in the brain is probably not the form of control
Keith was talking about.

A still more interesting class of cases of loss of control involves
being emotional.

Alas, the word "emotion" is nearly as bad as "consciousness" in the
variety of different senses with which it is used and the extent to
which theories proliferate and people talk past each other.

There are many states and processes that are at least sometimes
labelled emotions, which are found in other animals (e.g. rats),
and which, or so I am told, involve activities in the limbic system
which is a very old part of the brain.

These states include being startled, terrified, distressed by extreme
pain, disgusted by horrible tasting food, perhaps being sexually
aroused, and no doubt many more.

These are also cases where we may temporarily lose control, and I
presume the mechanisms involved are largely the same as those found in
rats, monkeys, etc. though of course the repercussions in different
species may be very different.

These states can include uncontrolled physiological changes of various
sorts, which may be sensed by the individual and may also sometimes be
visible to others. Some also involve internal (cognitive) arousal,
whatever that means.

Some people (well trained athletes and spies?) can prevent these
reactions under certain conditions, but for most of us it is not
easy, and it always requires special training (though no doubt there
are brain abnormalities that prevent some of them.)

Anyhow those are not the sorts of emotions to which I wanted to draw
attention.

The sorts of case I have in mind are cases where you lose control of
your *thought* processes: i.e. what you think about or attend to. I
doubt that a rat ever is in control of its thought processes, and if not
it cannot lose control. (Correct me if I am wrong).

Most people will recognize examples of such states, such as: feeling
humiliated by a rival, being ashamed of what one has done to a friend,
being awe-struck by a magnificent building, being thrilled at winning a
gold medal, being obsessed with a desire for revenge or promotion. All
these sometimes produce partial loss of control of thought processes.

You decide to think about or attend to something else because it is
important to you and the work needs to be done, and yet you find your
thoughts being drawn back to the pleasant or unpleasant episode (if it's
past) or prospect (if it's future). In extreme cases, such as some forms
of grief this can be quite debilitating.

I once heard a woman talking about her grief a year after the death
of her teen-aged son in a road accident. She still found it
difficult to prevent her thoughts being dominated by the child,
including thoughts about what might have been done to avoid the
accident, what her son would have been doing now, what should be
done to the driver of the car, and so on, even though she wished not
to wallow in these thoughts, and even though these thoughts
interfered with other activities that she valued very highly and
wished to concentrate on (e.g. doing her job and looking after her
other children).

Note that this ability to lose control of thought processes presupposes
(a) the ability to monitor one's thought processes (b) the ability to
have preferences, desires, and intentions regarding what ones thought
processes are to be. E.g. I intend to get back to thinking about that
overdue paper after posting this.

It's partly because of their powerful effects on our mental lives (as
opposed to our bodies) that the characteristically human emotional
states (humiliation, grief, jealousy, infatuation, etc) are of interest
to poets, playwrights, novelists and gossips. (Whereas emotions we share
with rats appear to have received far more study from psychologists and
brain scientists.)

One feature of these human types of emotional states is that they
typically involve sophisticated conceptual capabilities enabled by
our culture and language.

But that is not the feature I am drawing attention to, which is a
*control* phenomenon rather than a matter of the *information*content*
of the states. (I guess Herb Simon saw all this 30 years ago?)

I call these states "perturbant" and the processes "perturbances"
(with deliberate echoes of "perturbation" and "disturbance").

In such states there typically is a partial loss of control of
thought processes. This makes sense only in the context where one
can sometimes be in control. In other words, to explain such
emotional states we have to explain how it is possible ever to be in
control.

One interesting feature of these states is that they can endure as
dispositions that are temporarily suppressed.

The grieving mother faced with a very distressed younger child may be
able to put thoughts of the dead child out of her mind while an urgent
and important task dominates her attention. But the grief is still
there, and when she later tries to concentrate on another task the grief
regains control.

The woman I heard found this so debilitating that she said she did not
wish to go on living. (That may have been self deception.)

The main question that interests me about this (there are many
questions) is what it is to be in control: what is it that we have
that we sometimes lose, and which I suspect rats cannot lose because
they don't have it? Likewise very young children, before their
control architecture has developed fully.

The answer must lie in the information processing architecture that
is characteristic of the human mind.

Over the last 16 years (since before my 1981 IJCAI paper: Why robots
will have emotions) I have been developing (with the help of research
students and colleagues) the notion that in human beings there are three
layers of architecture.

One layer, evolutionarily very old, and shared with many other
animals, is what would nowadays be called a reactive architecture.

In reactive architectures information is acquired through external
sensors and internal monitors and propagates through and around the
system (with many transformations), and out to effectors of various
kinds.

Everything happens automatically and in parallel because there are
dedicated, coexisting circuits, some continuous and some digital
implementing condition-action rules, with various kinds of feedback.

Such a system can react quickly because all processing is done in
parallel. Competitions between sub-systems can be resolved either by
weighted additive mechanisms where the appropriate behaviour is a
"resultant", or by winner-take-all circuits where
sub-systems tend to produce incompatible behaviours.

This sort of architecture can work very well provided that the
environment for which it has been developed (by evolution, or by an
explicit designer) does not change much. If there is a lot of
variability in the environment the survival rate for individuals
could be low, but that may not matter for the gene pool if very
large numbers of individuals can be produced cheaply. (Insects?
Plankton? etc.?)

I suspect that some animals are {\em entirely} reactive, some are
{\em largely} reactive (e.g. other animals whose behaviour is mostly
genetically determined), while chimps and humans are {\em partly}
reactive, with additional architectural layers providing additional
flexibility.

The second layer of architecture is deliberative, and has features
that relate to the global workspace that Bernie and others have
discussed, though I focus on particular features relating to the
information processing function of a planner or problem solver.

Unlike a reactive architecture where everything happens
automatically, a deliberative architecture allows the explicit
construction of structures representing alternative possible future
actions, including novel actions, along with processes of evaluation
and comparison, leading to the selection of one option, as a plan
(possibly a partial plan). This requires use of stored knowledge
about the environment, about the agent's capabilities, about
preconditions and effects of actions, and knowledge about how to build
plans.

In general the process of finding a suitable plan, or solution to a
problem requires a search in a space of possible NEW structures. This
can include symbolic trial and error searching, which may be cheaper,
quicker, and safer than trying out options in the real world.

(When the constructive abilities are iterative or recursive this may
be one root of mathematical capabilities.)

This sort of architecture requires a re-usable workspace within which
the temporary structures representing possible actions can be built and
evaluated, until a selection is made. The same workspace can then be
used for another problem.  (Commonplace in AI.) The use of such a
workspace is therefore inherently serial and slow, even if it is
implemented using parallel mechanisms, such as neural nets.

(Damage to the mechanisms involved in this workspace could explain
phenomena such as "left neglect", but leave that for now.)

The global store of knowledge may also be implemented using a highly
parallel mechanism, but that does not mean that it can answer lots of
questions in parallel: it too may be restricted to a serial mode of
functioning, to prevent distorting cross-talk.

For these and other reasons that are too complicated to go into
here, the overall functioning of a deliberative architecture in
which novel temporary structures are built and selected will be
inherently resource-limited and serial.

The deliberative architecture cannot (normally) be made faster by
multiplying workspaces, whereas in a reactive system additional parallel
circuits dedicated to performing specific tasks (e.g. visual sensing,
auditory sensing, touch sensing, posture control, control of digestion,
etc.) can be added to the architecture. (In humans this can happen
through training.)

I suspect that insects do not include a deliberative mechanism (though I
am not an expert on insects). I am sure that some other animals do (e.g.
chimpanzees and bonobos) though I don't know how far they compare with
human deliberative capabilities.

There's lot more to say about deliberative mechanisms (investigated
in some depth in symbolic AI over the last 30 years or so).

The third layer in the architecture, which my students (e.g. Luc
Beaudoin, Ian Wright) and I have been calling the "meta-management"
layer is needed because the deliberative mechanisms are resource
limited (as any re-usable workspace must be) and subject to various
kinds of interrupts (e.g. as new important and urgent goals arise
from processes within the reactive sub-architecture or new
percepts).

This message is already too long, so I'll just summarise by saying that
meta-management mechanisms are a subset of deliberative mechanisms that
can to some extent monitor internal processes including themselves (like
a computer operating system), evaluate options, select strategies, and
make plans for the deliberative system.

Conjecture: being in control of thoughts, or attention, involves being
in a state where the deliberative processes and other internally
monitored processes conform to, or do not contradict, goals and plans of
the meta-management system.

Then why should thoughts and attention ever NOT conform?

Answer: because the deliberative architecture *must* be capable of being
interrupted and diverted e.g. by detection of a rapidly approaching
danger or opportunity, etc.

Conjecture: being in some emotional perturbant state involves the
deliberative architecture being dominated by interrupts and diversions
that do not conform to high level goals and preferences of the
meta-management system. In other cases they do conform, but it's still
the case that even IF they didn't they could not be stopped or modified.

In some cases this may be a direct consequence of normal functioning of
the interrupts and filters, e.g. providing signals due to physiological
damage or stress. In other cases there may be pathological cases, e.g.
where concerns that have been rejected continue to surface.

It may be that one reason we are like that is that some of the things we
do are not in our interest but only in the interest of the genes (e.g.
having children can be very costly). So genetically determined
mechanisms may sometimes dominate the meta-management preferences
of the individual.

Of course, in some some sense it's all *you*, i.e. *you* are still
controlling the process. But there can be parts of you whose
behaviour does not conform to high level meta-management decisions, and
which cannot be suppressed even though the non-conforming processes are
monitored and evaluated negatively.

(St Paul: It is not I but sin that dwelleth in me? Robots will feel like
that too.)

Well there's a lot more to be said about this, not least because the
notion of a workspace is just a metaphor, and it needs to be filled out
by giving detailed specifications for its function and for possible ways
of designing it. (I need to compare with Bernie's specifications.)

For example one of the details of meta-management will involve ability
to inspect various kinds of intermediate information structures in low
level reactive perceptual mechanisms as well as the deliberative
processes.

This is one source of philosophical theories about sensory qualia, but
that's another long story. (I.e. qualia are not a myth: they are
INEVITABLE features of a robot with the sort of information processing
architecture sketched here, no matter how much people SAY that they can
imagine this architecture working without any qualia. They are just
kidding themselves. Sorry.)

> ...

Keith continues

> OK, so this is all folk psychology.  But the phenomena that we are
> trying to explain (consciousness, the self, intentionality) are all the
> product of normal human experience and these things have to be
> explained, rather than just dismissed.

I agree. But we have to be careful about what needs explaining, and any
claim that we are totally in control needs to be qualified, with
different cases carefully distinguished, so that we can check whether
proposed explanatory mechanisms actually have the required capabilities.

> ...The claim that the evidence from
> neuropsychology is sufficient to refute these intuitions...

I would never claim that.

On the contrary, I would expect neuroscience eventually to EXPLAIN how
the sort of architecture that makes possible both control and various
kinds of loss of control (normal and pathological) is implemented.

It's an open question whether the same architecture admits totally
different implementations, e.g. using computers. So far I see not the
slightest reason to believe that this sort of three layered architecture
needs novel quantum gravity engines. (They may be needed for other
reasons.)

> ...

Keith also wrote:

> To get back to John's opening remarks, it strikes me that the problem is
> not dualism per se, but rather the particular form that is associated
> with the names of Descartes and Eccles, rooted firmly in the traditions
> of western christianity.  Other thinkers (Einstein, Bohr, Shroedinger,
> Pauli) who also took these things seriously chose to turn to the more
> sophisticated (and ultimately unitary) dualistic traditions of the east.

>...

Whereas I think carefully cataloguing things we all know, using language
that is as precise as possible, and avoiding unjustified
over-generalisations, is a good start towards producing a detailed set
of requirements specifications that could drive an engineering design
process aimed at replicating, and thereby explaining the possibility of,
the phenomena.

If only I had a team of engineers, programmers, brain scientists,
psychologists, ethologists, philosophers ....

Aaron