Those intermediate concepts are concerned with processes, including both processes that are occurring and processes that can occur, and the causal relationships between physical structures/forms/configurations and the possibilities for and constraints on such processes, independently of whether they are processes involving anyone's actions or goals. These intermediate concepts relate motions and constraints on motion to both geometric and topological structures in the environment and the kinds of 'stuff' of which things are composed, since, for example, rigid, flexible, and fluid stuffs support and constrain different sorts of motions.
They underlie affordance concepts. Attempts to study affordances without taking account of the intermediate concepts are bound to prove shallow and inadequate.
Our hypothesis is that the form and shape of objects is a key factor
deciding upon actions that can be performed with an object.
My hypothesis
Between (a) the form and shape of objects and (b) their functions and
associated affordances there is (c) a rich variety of phenomena involving
processes and possible processes that need to be perceived,
represented and reasoned about.
Type 1 research on vision: objects
There is much research, both in psychology and AI/Robotics/Machine Vision
that focuses on learning about object categories and recognition (and/or
tracking) of instances of those categories.
Type 2 research on vision: scene structures
There is also much research (going back to the 1960s and earlier) on
perception of object and scene structure, including locations,
orientations, curvature and other properties of objects, object parts and
surfaces, and their relationships. (That includes the "syntactic" approach
to vision, inspired by work in linguistics on parsing, etc.)
Type 3 research on vision: affordances
More recently, many people follow J.J.Gibson in emphasising the role of
perception of affordances in vision and other perceptual modalities
(e.g. haptic perception of affordances).
For an individual X to perceive affordances is normally construed as
involving
(1) X gaining information about which actions are possible for X (positive
affordances) or hindered (negative affordances), where
(2) X being capable of performing those actions in some situations, and
(3) performing those actions being able to contribute to achievement of X's
goals or prevention of states or occurrences that are undesirable for X.
Type 4 research on vision: processes and process possibilities (proto-affordances)
For some time I have been claiming that all three sorts of research (on objects,
structures and affordances) ignore visual competences that lie between
perception of structure and perception of affordances (between perception of
form and perception of function), and are more general and more fundamental than
either, namely:
Processes
(a.1) perception of processes in the environment (whether produced by the
perceiver, or another agent, or simply by physical forces, such as wind,
gravity, water pressure, changing weather, or plant behaviours, etc.)
(a.2) where the processes need not be relevant to the perceiver's goals or
preferences and need not be ones that the perceiver is capable of producing
or preventing, and
Possibilities of and constraints on processes
(b.1) perception of possibilities for and constraints on processes
of those kinds, independently of whether they are relevant to the
perceiver's possible actions, goals, or preferences (i.e. perception
of possibilities and constraints that are not affordances), and
(b.2) independently of whether the perceiver is capable of producing,
preventing, or constraining those possible processes.
I am not claiming that any individual can perceive ALL such possibilities and
constraints in the environment. A major form of learning is learning to see NEW
types of possible process and NEW types of constraint on processes.
Some of the concepts involved in perceiving, characterising, and reasoning about
actually occurring and possible processes make use of concepts of 'kinds of
stuff', i.e. different sorts of matter with different dispositional properties,
including different sorts of rigidity, breakability, flexibility, elasticity,
fluidity, stickiness, etc.
(See http://www.cs.bham.ac.uk/research/projects/cogaff/talks/#babystuff)
Proto-affordances
I call the possibilities and constraints in the environment that are not
affordances 'proto-affordances' since they are the kinds of primitive modal
facts about the environment that underlie affordances.
Two senses of "modal":
1. Facts are "modal" in the logician's sense (contrasted with "categorical") if
they are not about what is or is not the case, but about what can or cannot
or must or may be the case.
2. Facts can be logically modal yet "a-modal" in the psychologists' sense, i.e.
not concerned with any particular sensory or motor modality, though
possibly relevant to several.
Amodal concepts in the second sense can be either "somatic", i.e. concerned
with what's going on inside the agent, or "exosomatic", concerned with what
is happening in the environment (e.g. 3-D surfaces changing their
relationships).
(Theories about mirror neurons seem to me to fail to take account of the requirement
for this kind of abstraction.)
Vicarious affordances
In some cases where affordances are perceived they are not affordances for the
perceiver, but for some other individual, e.g. a child, or prey, or predator, or
collaborator. I call those 'vicarious affordances'.
Epistemic affordances
Besides processes in which physical structures, properties, and relationships
change there are also processes in which information is acquired, e.g. through
vision, touch, hearing, or other senses. Any particular configuration of
a portion of the world supports and hinders the acquisition of particular sorts
of information. The possibilities and constraints on information acquisition are
labelled epistemic affordances. There are many complex relationships
between proto-affordances, action-affordances and epistemic affordances.
In particular, intelligent agents need to work out which epistemic affordances
are relevant to their goals in order not to waste effort attempting to process
information of the wrong kind, e.g. wasting effort trying to cope with noise and
uncertainty where both can be ignored because there is enough unambiguous
information at a higher level of abstraction [8].
NOTE (added 21:44 BST 10 Sep 2009):
The above characterisation of X being able to perceive affordances can be
generalised to include seeing or learning what can and cannot be done by
individuals similar to X (i.e. with similar action capabilities and similar
kinds of goals, needs, interests, etc.) who are not necessarily present
at the time of perception.
This is very close to perception of 'vicarious affordances', except that the
relevant other other agent need not be present. It can also be related to
perceiving affordances that X would have if located in a different place
relative to the objects with those affordances. That can generate an
explosive set of possibilities, which can be avoided by abstracting away
from many of the details of particular actions, e.g. representing a wide
variety of types of grasping simply in terms of two opposing surfaces
coming together with an object between them.
A further generalisation would include abstracting away from various details
of X, e.g. seeing graspability in terms of a wide range of possible grasping
devices, not just devices similar to the perceivers' hands, claws etc.
This shades into perception of proto-affordances (See Type 4 research)
Relevant work
The above ideas were summarised, along with some implications for information
processing architectures, in a paper arising out of a previous Dagstuhl
seminar in Feb 2008[1]. See also [2], [3].
More recent research has been attempting to relate development of the ability to
perceive and reason about possible processes in the environment to the
development of an exo-somatic ontology that includes not only perceivable
processes, structures, surfaces, and their changing relationships in the
environment, but also unperceived but inferred kinds of 'stuff' (kinds of
matter) with unperceived but inferred properties that produce dispositions
influencing proto-affordances, e.g. properties such as rigidity, plasticity,
elasticity of various kinds, and various kinds of flexibility [4].
All of these competences need to be integrated within an information processing
architecture that is able to represent and reason about processes involving
continuously changing spatial relationships, including multi-strand processes.
One of the implications of these ideas is that much discussion of so called
'mirror neurones' is seriously confused (as some others have also begun to
realise, e.g. [5], though the arguments are not yet presented very clearly
there.)
My presentation will attempt to show that a failure to take account of all of
these logically modal, but psychologically/neurally amodal phenomena between
form and function, between structures and affordances, in an integrated theory,
will cause cognitive science, neuroscience, cognitive robotics, machine vision,
and developmental psychology to flounder about, studying only simplified
fragments of deep and difficult problems.
All this is relevant not only to animal perception and action, but also to
understanding the development of human mathematical competences [6]
and also problem solving in other animals.
REFERENCES
[1]
@InProceedings
{slomanTr0801a,
author = {Aaron Sloman},
title = {Architectural and Representational Requirements for Seeing Processes, Proto-affordances and Affordances},
booktitle = {Logic and Probability for Scene Interpretation },
year = {2008},
editor = {Anthony G. Cohn and David C. Hogg and Ralf M{\"o}ller and Bernd Neumann},
number = {08091},
series = {Dagstuhl Seminar Proceedings},
ISSN = {1862-4405},
publisher = {Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, Germany},
address = {Dagstuhl, Germany},
URL = {http://drops.dagstuhl.de/opus/volltexte/2008/1656},
annote = {Keywords: Vision, affordances, architectures, development, design space}
}
[2]
@InCollection
{slomanTr0801,
author = {Aaron Sloman},
year = {2009},
title = {{Architectural and representational requirements for seeing processes and affordances}},
editor = {Heinke, D. and Mavritsaki, E.},
booktitle = {{Computational Modelling in Behavioural Neuroscience: Closing the gap between neurophysiology and behaviour.}},
publisher = {Psychology Press},
address = {London},
pages = {303--331},
note = {http://www.cs.bham.ac.uk/research/projects/cosy/papers#tr0801},
}
[3]
@InCollection
{slomanTr0804,
author = {Aaron Sloman},
year = {2009},
title = {{Some Requirements for Human-like Robots:
Why the recent over-emphasis on embodiment has held up progress}},
editor = {B. Sendhoff and E. Koerner and O. Sporns and H. Ritter and K. Doya},
booktitle = {{Creating Brain-like Intelligence}},
publisher = {Springer-Verlag},
pages = {248--277},
address = {Berlin},
note = {http://www.cs.bham.ac.uk/research/projects/cosy/papers/#tr0804},
}
% whole book
% http://rapidshare.com/files/209786694/Creating_Brain-Like_Intelligence.zip
[4] http://www.cs.bham.ac.uk/research/projects/cogaff/talks/#babystuff
Talk 68: Ontologies for baby animals and robots
From "baby stuff" to the world of adult science: Developmental AI
from a Kantian viewpoint. (PDF)
(Last presented at Brown University in June 2009)
[5] Pierre Jacob
A Philosopher's Reflections on the Discovery of Mirror Neurons
Topics in Cognitive Science
Vol 1, No 3, July 1009
pp 570--595
http://www.philosophie.uni-hd.de/imperia/md/content/fakultaeten/phil/philosophischesseminar2/jacobv.pdf
[6] http://www.cs.bham.ac.uk/research/projects/cogaff/talks/#toddler
Talk 67: Why (and how) did biological evolution produce mathematicians?
Alternative title: A New Approach to Philosophy of Mathematics:
Design a young explorer, able to discover "toddler theorems"
(Or: "The Naive Mathematics Manifesto").
[7]
@InProceedings {Sloman96d,
author = {A. Sloman},
title = {Actual Possibilities},
editor = {L.C. Aiello and S.C. Shapiro},
booktitle = {Principles of Knowledge Representation and Reasoning:
Proceedings of the Fifth International Conference (KR `96)},
publisher = {Morgan Kaufmann Publishers},
address = {Boston, MA},
year = {1996},
pages = {627--638},
isbn = {1-55860-421-9},
note = {http://www.cs.bham.ac.uk/research/cogaff/96-99.html#15},
}
[8]
@TechReport
{slomanDp0702,
author = {Aaron Sloman},
title = {{Predicting Affordance Changes (Alternatives ways to deal with uncertainty)}},
month = {Nov},
year = {2007},
note = {Unpublished discussion paper
http://www.cs.bham.ac.uk/research/projects/cosy/papers/#dp0702 (HTML)},
number = {COSY-DP-0702},
institution = {School of Computer Science, University of Birmingham},
}
Maintained by
Aaron Sloman
School of Computer Science
The University of Birmingham