School of Computer Science THE UNIVERSITY OF BIRMINGHAM

Some Illustrative Videos Used in Several of my Presentations
(Offers of similar videos welcome!)
Aaron Sloman
School of Computer Science,
University of Birmingham


This web site is:
http://www.cs.bham.ac.uk/research/projects/cogaff/movies/vid
It is part of the Birmingham CogAff (Cognition and Affect) web site.


These videos are used in several of the presentations listed here.
and also in my talk given at this workshop in Paris Sept 2007:
COSY Meeting-Of-Minds Workshop Paris 15-18 Sept 2007

The videos show different levels of competence in young children, and in some cases,
surprising incompetence, which was later overcome, though I don't think anyone knows how
the changes occur.

There are some conjectures in
the presentations
, especially the presentations concerned
with mathematical development in young children, and its evolution.


NOTE: Added 6 Jul 2012

The idea of architecture-based motivation (contrasted with reward-based motivation) is
important for understanding what's going on in some of these videos:
http://www.cs.bham.ac.uk/research/projects/cogaff/misc/architecture-based-motivation.html


NOTE: Added 2 Apr 2011

My ideas have recently been reorganised and extended by reading Piaget's last two books,
and an important book by Karmiloff-Smith

The videos discussed here

Notes added below on the individual videos, point out how they illustrate interleaved
exploration of different "exploration domains" that can be "extracted" from a continuous
world, by using different sorts of actions on items in the environment: a rug, a piano, a
tub of yogurt and spoon, a toy train, etc.

These exploration processes must be partly genetically driven in some of their abstract
properties, while details of both the exploration processes and the discoveries they lead
to are largely driven by details of the learner's environment.

    Compare the notion of "parametric polymorphism" in Object Oriented Programming,
    where a generic method performs actions and builds structures in ways that
    depend on the types of entities provided as parameters when the method runs.

The notion of an exploration domain is closely related to the old AI notion of a
"micro-world", to Karmiloff-Smith's notion of a "microdomain", and to the notion of a
class (or related set of classes linked by methods) in Object Oriented Programming.

N.B.

It is often assumed that what an intelligent animal or robot can learn must be closely
constrained by its sensory-motor morphology and functionality.

This viewpoint ignores the role of the environment in presenting opportunities to learn
and providing examples and evidence regarding what can happen in the environment,
including processes initiated not by the learner but by wind, rain, gravity, other
animals, etc. Learning about your environment is not the same thing as learning about
your sensory motor signal patterns.

The sensory-motor analysis also ignores the similarities between types of process that can
be performed using different morphologies: e.g. carnivores use jaws to carry, manoeuvre
and disassemble objects whereas primates (and some birds) use hands and feet.

The animals that can produce a-modal information structures using exosomatic ontologies
(referring to things in the environment, not to sensory-motor signals) can learn things
that have nothing to do with their own sensory or motor mechanisms, e.g. about effects of
forces applied by one object to another, and effects of objects in constraining motions of
other objects.
They should also be far more flexible about transferring what they have learnt to new
configurations where the sensory motor signals are very different, but what happens in the
environment is very similar (e.g. a door is opened).

The comments on the videos relate to the notion of a "exploration domain" discussed in
some of my talks, e.g. in

    http://www.cs.bham.ac.uk/research/projects/cogaff/talks/#talk90
    Talk 90: Piaget (and collaborators) on Possibility and Necessity
    And the relevance of/to AI/Robotics

    (A related talk presented at Schloss Dagstuhl on 28th March 2011 is
    here.
    )


  1. Noticing and grasping edge of rug (Age about 6 months)
    Noticing the edge of the rug seems to trigger a very deliberate and controlled, but
    inexpert, attempt to grasp the edge, with clear success eventually, after rolling over
    onto his side.
    It looks as if this baby is exploring a number of different domains, initially
    Why did he use his right hand, not his left hand to grasp the rug?

  2. Playing piano and piano parts (Age about 9 months)
    No sound in this video but it is clear that the child alternates between fairly random
    thumping and controlled exploration of effects of pressing individual notes. What are the
    mechanisms that produce the motivation to do that? What cognitive processes are involved
    in the selection between possible actions, and their execution? What exploration domains
    are being extracted from the interaction with the environment. Compare the later piano
    video below.

  3. Feeding brain and stomach with yogurt
    (Age about 10 months)
    The child manipulates a spoon in various ways, including getting yogurt out of a tub held
    in front of him, transferring some to his mouth, and some to carpet and his thigh.
    He seems to do a number of experiments of different kinds with spoon and yogurt, including
    at least two failed attempts to transfer yogurt to a flat surface, on carpet and on his thigh.

    The video ends with a successful complex process of transferring the end of the spoon from
    right hand to left hand, which requires coping with the fact that one hand is an obstacle
    to grasping by the other.
    (This is well beyond the capabilities of any robot I know about.)

  4. Playing piano and piano parts, recorded with sound. (Age about 11 months)
    A sequence of experiments concerned with piano keys, music holder, balancing on chair.
    The child discovers and explores separate domains of exploration, involving
    The video ends just as he seems to be discovering a repeatable musical theme, or
    possibility a motor theme for his fingers, or perhaps a mapping between the two?

  5. Exploring materials: cloth (sweater) and mouse cable
    (Age about 11 months)
    (Doesn't quite strangle himself.)

  6. Pushing a broom (Age: about 15 months)
    A just about stable (well, upright anyway) dynamical system driven by an opportunistic
    cognitive explorer?
    Notice the various ways in which an action prepares for what is going to come later.
    This shows an understanding of constraints in the environment detected before the
    constraints operate.
    This cannot be achieved by purely reactive sensory-motor mechanisms where all internal
    processes are concerned with relating current inputs to current outputs.

  7. Failing to understand hook-and-eye mechanism (Age about 18 months)
    This child clearly has an impressive collection of motor, perceptual, and cognitive
    skills, and has mastery of a variety of exploration domains including switching between
    crouching and sitting positions, stacking objects, poking objects into holes, noticing a
    gap in the distance and walking to it in order to insert an object.

    He achieves all his goals apart from joining two train components -- trying to join two
    rings together instead of a hook and a ring. His understanding of possible ways of linking
    two objects is clearly still under developed. (I was told a few weeks later that he had
    mastered the problems, though nobody had observed steps in the transition.)

    Compare this with the intelligence of Betty the hook-making and hook-using crow.

  8. Several videos available at the Behavioural Ecology Research Group, Oxford
    These include the very famous 'Trial 7' video, which shows Betty, the New Caledonian Crow,
    very expertly making a hook from a straight piece of wire, and then very expertly using
    the hook to lift a bucket of food out of a vertical tube.
    She had no opportunity to learn anything about this from other crows or humans.
    Betty made headlines world-wide in June 2002. Give to google:
        betty crow hook
    
    What the researchers did not point out, and can easily be confirmed by looking at the
    other videos on the web site, is that Betty spontaneously made hooks from straight pieces
    of wire in at least five different ways. Even after successfully making a hook one way she
    notices and uses other possible strategies with no flailing about using trial and error.
    In each case she seems to know as soon as the action begins what she is going to do.

    I have argued elsewhere that such competences (and others) imply that many non-human
    species, and also pre-verbal children, must be able to use internal languages
    (forms of representation) that support structural variability, variations in complexity,
    compositional semantics, and use in making inferences.

    http://www.cs.bham.ac.uk/research/projects/cogaff/81-95.html#43
    The primacy of non-communicative language
    
    http://www.cs.bham.ac.uk/research/projects/cogaff/talks/#glang
    Evolution of minds and languages. What evolved first and develops first in children:
    Languages for communicating, or languages for thinking (Generalised Languages: GLs)?
    

  9. Warneken and Tomasello's videos of children and chimps being helpful
    See especially the cabinet task.
    Those researchers are mostly interested in the fact that very young children and some
    other animals spontaneously try to help someone else.

    A different sort of question that I am interested in concerns competences that must be
    prior to those motivations: what sort of cognitive mechanisms make it possible for a
    pre-verbal child to tell what a person is trying and failing to do, and to work out what
    could be done to provide assistance?

    The child in the video seems to be able to work out that the man carrying books needs to
    get the the door of the cabinet open, and he then adopts that motive for himself (the
    altruistic step) and works out what to do to achieve the open door, then executes the plan.

    Here the child seems to make use of a domain of possibilities that is not concerned with
    what he, the observer, can do, but with possibilities available to another agent.
    (I have called those "vicarious affordances" elsewhere: very important both for parents
    and for young learners.)
    What the child can do becomes relevant only after he has worked out what the adult wants
    to do, or needs to do.
    What motivates the child to use those cognitive abilities to provide help is an additional
    question. Being able to think about motivation requires a domain of meta-semantic
    competence: being able to represent and reason about things that can represent and reason.
    That clearly starts to develop before a child can talk.

    For both questions the answers are likely to involve a mixture of genetic mechanisms and
    competences learnt partly under the influence of the environment and partly under the
    influence of the genetic mechanisms.

    The learning mechanisms probably change during learning and development, since what is
    learnable changes. But this is not merely a matter of what prior content is available for
    each type of learning. Rather new ontologies, new forms of representation, new algorithms
    and new information-processing architectures seem to be involved.

  10. Video of parrot scratching neck with feather.
    What sort of ontology does the parrot need in order to be able to select and control this
    action, including combining action with beak and foot in order to change grip location?
    Do you think it knows what it is doing? What does that mean?

    Compare videos in which a parrot perches on one leg holding a walnut in the other foot,
    and alternating between holding it with foot and with beak rotates the walnut until it is
    in a good position for use of beak to crack it open.

    (Compare: you are holding a cold saucepan by its handle using one hand. You wish to get
    into a position in which you are holding it by the rim of the pan using two hands, and
    with the handle pointing away from you. What actions do you need to perform to achieve
    that end state -- with or without being able to rest the saucepan on a flat table top
    during the process?
    What enables you, or the parrot, to work out a sequence of actions to achieve a new state?)

  11. Boston Dynamics Big Dog (March 2008)
    Go to the Boston Dynamics web site to get more information.
    Do you think this robot knows what it is doing?
    Does it know anything about what it could do in the future?
    Does it know what it has done in the past?
    Does it know why it did that and not something else?
    Do the baby on the rug, or the baby playing with yogurt, or the toddler pushing a broom
    know what they are doing or not doing, or what they can do before they actually do it?
    What sorts of questions should biologists ask about robots?
    What sorts of questions should roboticists ask about animals?
See also

For further discussion of the issues raised by these videos see:
My slides for the 'Meeting of Minds' workshop in 2007, and the post-workshop notes on model-based semantics


Installed: 10 Nov 2010
Last Updated: 10 Nov 2010; 7 Jul 2012; 9 Jan 2013

Maintained by: Aaron Sloman