Artificial Intelligence and Cognitive Systems


Aaron Sloman

Last updated: 18 Nov 2003

What is this?

This document is one of the documents I am assembling as part of the process of trying to clarify objectives and strategies for the Architecture of Brain and Mind Grand Challenge proposal

It is also relevant to other activities in which research goals and strategies are being discussed.


There are two main ways of trying to make progress in AI, Cognitive Science and related disciplines (or in science in general, but I am focusing more narrowly):

(a) Forward chaining

We can start from what we currently know, what we find interesting and challenging to do, what others have achieved, what tools are available, and propose a project that combines and builds on these resources.
That is how most research progresses.

(b) Backward chaining

We can try to investigate and analyse what we do NOT YET understand, what we CANNOT now model or explain, what our tools CANNOT do, and if we find such things lying beyond our current understanding and capabilities, but which need to be understood and modelled in order to advance our science (or, in some cases our engineering capabilities) then we can work backwards trying to develop concepts, theories, tools, techniques that have some chance of achieving that goal.

Any research project inevitably is partly of type (a). We have to start from somewhere and build on what we have achieved, though there are different ways of doing that, based on different long term visions.

(b) Is more difficult and painful, and involves more steps into the unknown. However it is the mode of working I have been trying to develop over the years and I think we really can make significant progress if we at least try to do (b) some of the time!

(a) tends to identify the strengths in what we have achieved,

(b) requires us to put significant effort into identifying current weaknesses.

Work in AI and Cognitive Science has made many advances over the last fifty years. Yet there are enormous gaps, which make me say that our research is still in its infancy.

These gaps may not be of interest to people with specific engineering goals: they have a design task defined by those goals. But in the context of trying to understand, model, or even go beyond natural intelligence there are many cases were we have gaps relating to our understanding of what organisms (including both humans and other animals) can do, let alone how they do it.

By identifying things people can do, for instance, we define new requirements for explanatory theories, for simulation models, and for tools and techniques.

However, identifying what people can do is not easy: simply observing them does not reveal much of what is going on. Sometimes you learn what people can do only when a proposed working model of how they do something clearly fails to replicate human capabilities. This can draw attention to gaps in the requirements that drove the development of the model.

I have tried below to assemble a list of such gaps in our understanding of what people can do. There is no claim that the items have never been noticed before, merely that, as far as I know, we do not have precise characterisations of any of them (precise enough to serve as requirements specifications to feed into a model building project), and we also do not have explanatory models for them.

Thus an important research goal is to fill the gaps by attempting to specify far more precisely than ever before what the capabilities are, so as to define new requirements for architectures, mechanisms, forms of representation, knowledge stores, tools, etc.

An additional research goal is to add to the list of gaps, or subdivide the categories where appropriate.

If something should be in the list because someone has already done the work (e.g. specified the type of competence in great precision), please let me know.

Examples of gaps

Here is a collection of biologically inspired task requirements not addressed by existing models, tools and architectures.
  1. Representational change

    Humans invent powerful new forms of representation and find useful ways of linking them. No AI system comes close. We know of many examples, but cannot yet produce a complete list, including the forms of representational change involved in child development. (Karmiloff-Smith's work is relevant.)

    Examples of representational change
    One of the most famous examples in our history was the invention of the Arabic notation for numerals (which is not as good as Roman numerals for adding small numbers with total less than 4, but for almost everything else is far superior). That included the invention of a representation for zero (extending the ontology of numbers). Later extensions added negative numbers, fractions, decimal numbers, exponents, vectors, matrices, etc.

    There are many other examples in the history of science, mathematics and art: notations for differentiation and integration, tensor calculus, musical notation, map notations (metrical, topological and mixed), chemical formulae, grammars, programming languages, dance notation, types of gestures, and most recently forms of representation that we don't manipulate directly but create inside computers using the computer as cognitive extensions of ourselves: e.g. list structures, artificial neural nets, procedure activation threads, etc.

    There have been many attempts to produce various AI systems to handle special cases of these, but they don't come near what humans can do. Has any AI system invented a new programming construct?

    I don't think anyone knows the architectural requirements for a system that can invent new forms of representation and use them, as we do.

    (How often do humans use that ability? Even if it is rare for most individuals, it has transformed all our lives.)

    Margaret Boden's book on Creativity is relevant to this. Also chapter 2 of The Computer Revolution in Philosophy (1978).

  2. Visual reasoning and inference

    One of the reasons for developing new forms of representation is that this can facilitate powerful new forms of reasoning. It has been widely acknowledged for many years that humans use visual/spatial reasoning in many tasks including planning of movements, predictions of what will happen in the environment, and many more abstract tasks such as mathematical reasoning.

    However it has proved very difficult to characterise the nature of this sort of capability. We need deeper analysis of what it is we grasp about spatial structures and spatial changes that enables us (and perhaps some other animals) to use our spatial understanding.

    It seems to be connected with the claim that the main function of perception (including visual perception) is not, as many believe, to identify objects, properties, relationships and changes in these, that exist or are occurring in the environment, but rather to identify affordances (e.g. what might or cannot occur if various actions are attempted).

    Thus a major function of vision is not the perception of which things exist in the environment but the perception of things (including actions and processes) that do not exist but could exist and the constraints that prevent existence of some of the possibilities.

    An incomplete draft paper on this is here http://www.cs.bham.ac.uk/research/cogaff/sloman-diag03.pdf

    This point is related to the next one.

  3. Development of perceptual ontologies

    Humans (e.g. children learning to do various tasks) seem to be able to detect inadequacies in their current perceptual ontology and remedy them, thereby learning to perceive new affordances, and acquiring new problem-solving capabilities.

    Some examples are discussed in this document, including a video of a young child unable to see why he cannot join two trucks by attaching two rings. The solution required one of the trucks to be rotated so that its hook could go into the other's ring. But apparently he did not understand the affordances. However, within about two weeks he had taught himself how to solve the problem. What is involved in that sort of transition? Follow the link above for more on this.

  4. Varieties of affective states and processes: aesthetic states

    Humans and other animals can, besides having goals and achieving them, also enjoy or dislike states, activities, experiences. Some things can be found funny also.

    Many of these involve aesthetic appreciation of something. I don't think there is any plausible general account of what these aesthetic states are, what functional roles they have, if any, what their architectural underpinnings are, how they evolved, how they change in an individual.

    (Though there are some heroic efforts that perhaps go part way.)

    Computers can make good pictures (Harold Cohen's Aaron) or compose good music (David Cope's system), or generate puns (Kim Binsted's PhD). I don't know of any AI system that can enjoy or dislike anything, or find a joke funny.

    A closely related point is that there is a very large variety of types of affective states and processes, which are often lumped together under one heading, e.g. "emotions" in a thoroughly confused way. We need to develop a useful taxonomy including such things as desires, dislikes, preferences, values, ideals, standards, attitudes, moods, interests, personality traits, etc.

    These operate on different time scales, have different types of semantic content (or none at all in some cases), require different sorts of architectural underpinning, relate to different biological mechanisms, evolved at different times, develop at different times in individuals, etc.

    The subject has recently become popular among AI and Cognitive Science researchers but much confusion and ambiguity abounds because so many researchers have a very narrow viewpoint.

    We need much more fine-grained specification of what we are talking about when we use affective labels. The we shall be in a better position to understand what needs to be explained.

    It may be that understanding what affective states and processes are, requires us also to understand how and why they develop: our next point.

  5. Affective development

    Humans do not have fixed affective bases, but change their preferencs, values, standards, ideals, attitudes, tastes, likes, dislikes, sense of humour over time. Probably other animals too.

    This is not just a matter of having a single high level goal that is fixed ("optimise utility") and then developing new sub-goals for different contexts.

    Ceasing to find a childhood joke funny is not just a matter of having different goals: there are differences in some of the uncontrollable reactions, not differences in the deliberative processes.

    You can see the biological need for mechanisms producing affective development in organisms whose "niche" changes in various ways from infancy to adulthood.

    E.g. the needs of a newborn baby are different from those of a toddler, a teenager, and an aged professor, etc. (Likewise for chimps, lions, eagles, etc. at different stages.)

    So evolution seems to have produced a delayed-action multi-stage affective bootstrapper.

    AI systems can learn new sub-goals but I don't know of any that acquire new top level goals in a biologically plausible way. (How did I acquire this driving obsession with trying to understand how I work? It may once have been a sub-goal, but it isn't now.)

    This is a case where it's easier to understand biological evolution than individual development.

    This may be a special case of the next pont.

  6. Architectural development

    Humans do not have the same information processing architecture at all stages of development. This is most obvious if foetal stages are considered. But even after birth there seem to be developments in the varieties of types of processes that can occur, e.g. changes in the kinds of self-awareness and self-control.

    We do not know what these architectural changes are, nor how to build systems that can develop their own architectures.

    The neuropsychiatrist Russell Barkley discusses some aspects of this in his 1997 book: ADHD and the Nature of Self-control.

  7. Extrapolative abilities.

    Humans have an enormously powerful ability to envisage indefinite extrapolation of many kinds, for instance:
    • counting: 1, 2, 3, 4, ...,
    • lines getting thinner,
    • lines getting straighter,
    • lines getting longer,
    • looking ever more closely at a complex shape (zooming in),
    • zooming out,
    • thinking about 2 dimensions, 3 dimensions, 4 dimensions ... infinitely many dimensions (Hilbert spaces),
    • thinking about finite ordinals, different classes of transfinite ordinals,

      and many more

    I don't know if these are many different abilities or different instantiations of some single powerful generic ability.

    I suspect the latter. I suspect it is profoundly linked to meta-management.

    I suspect some other animals have a restricted version of this: extrapolation but not indefinite extrapolation.

    What architectural or representational change made either possible, and how has that affected human and social evolution/development?

    Think of human language as a special case of an indefinitely extendable system.

  8. Varieties of SELF representation

    Humans (and maybe a few other species?) have a powerful self-representation capability with several different aspects. E.g.
    • Where am I in the room?
    • where will I be tomorrow if I travel that way?
    • Can X see me from there?
    • What does X think of me?,
    • This tasty looking piece of meat is my leg: I must not eat it.
    • What are my current thoughts?
    • What is it that I am now experiencing?
    • How does this experience relate to that one?
    • Is this the same feeling I had when I ate the blue stuff?
    • Why did I fail to see how to do X this time when I succeeded previously?
    • Why does moving left stop me seeing X?
    • Why does half-closing my eyes make things look fuzzy?
    • Why does eating X taste bad after eating Y, but not the other way round? etc.

    This is not just the ability to use a personal pronoun but seems to depend on architectural features and representational capabilities that are not well understood. In particular animals that do not have languages with personal or any other pronouns must be able to use some of this functionality. How?

    One of the consequences of our having this ability is that we easily get sucked into very confused philosophical discussions about consciousness (e.g. discussions of qualia, and 'what is it like to be a bat?'). But that is a FACT about human beings that needs to be explained. A sketch of part of an explanation is here: http://www.cs.bham.ac.uk/research/cogaff/talks/#talk25

    The ability that generates the philosophical puzzles is also a biologically important ability. You use it when you go to an optician or a dentist, and I suspect that close investigation will show that it is crucial to some of the ways we teach others, especially children, some important things. (E.g. how not to be selfish, how to do mathematics, how to throw accurately.)

    It may also be crucial for many social relationships (e.g. where empathy is important: know thyself in order to know others).

This is not intended to be a complete list. I shall insert additional items from time to time. The study of other animals, e.g. Betty the hook-making crow (see the video here http://news.bbc.co.uk/1/hi/sci/tech/2178920.stm) can help us identify many other gaps in our knowledge. E.g. how many forms of representation are used by bees in their search for nutrients and their communication about where they are?

Those (and other) biological competences are at present neither properly characterised (partly because people who study them professionally lack suitable conceptual tools), nor explained or replicated.

There are various reasons, but in part the problem is that these have not been properly characterised as research goals.

However, they may be pre-requisites for some of the breakthroughs that several current research programs are hoping for, e.g. the DARPA cognitive systems project in the USA, the Cognitive Systems Foresight project in the UK, and the Cognitive Systems initiative in the European Union.

Some are present in a typical 5 year old child. Others grow out of the five year old's capabilities. How?

Mozart may have had a super-charged boot-strapper. Can we understand it, and use that understanding to help others?

Conceptual confusions

We have many conceptual confusions making it hard to agree even on what the questions are. For some first draft attempts to clarify some of the most basic issues see:

More information

The topics above have been the subject matter of several slide presentations here: http://www.cs.bham.ac.uk/research/cogaff/talks/
And also many of the papers on the Cognition and Affect web site; http://www.cs.bham.ac.uk/research/cogaff/

Comments, suggestions, etc. to A.Sloman AT cs.bham.ac.uk please.