School of Computer Science THE UNIVERSITY OF BIRMINGHAM

Some Illustrative Videos Used in Several of my Presentations
(Offers of similar videos welcome!)
Aaron Sloman
School of Computer Science,
University of Birmingham

This web site is:
http://www.cs.bham.ac.uk/research/projects/cogaff/movies/vid
It is part of the Birmingham CogAff (Cognition and Affect) web site.

These videos are used in several of the presentations listed here.
and also in my talk given at this workshop in Paris Sept 2007:
COSY Meeting-Of-Minds Workshop Paris 15-18 Sept 2007

The videos show different levels of competence in young children, and in some cases, surprising incompetence,
which was later overcome, though I don't think anyone knows how the changes occur.

There are some conjectures in the presentations, especially the presentations concerned with mathematical development in young children, and its evolution.

NOTE: Added 2 Apr 2011

My ideas have recently been reorganised and extended by reading Piaget's last two books, and an important book by Karmiloff-Smith

Notes added below on the individual videos, point out how they illustrate interleaved exploration of different "exploration domains" that can be "extracted" from a continuous world, by using different sorts of actions on items in the environment: a rug, a piano, a tub of yogurt and spoon, a toy train, etc.

These exploration processes must be partly genetically driven in some of their abstract properties, while details of both the exploration processes and the discoveries they lead to are largely driven by details of the learner's environment. 

    Compare the notion of "parametric polymorphism" in Object Oriented Programming,
    where a generic method performs actions and builds structures in ways that
    depend on the types of entities provided as parameters when the method runs.

The notion of an exploration domain is closely related to the old AI notion of a "micro-world", to Karmiloff-Smith's notion of a "microdomain", and to the notion of a class (or related set of classes linked by methods) in Object Oriented Programming.

N.B.

It is often assumed that what an intelligent animal or robot can learn must be closely constrained by its sensory-motor morophology and functionality.

This viewpoint ignores the role of the environment in presenting opportunities to learn and providing examples and evidence regarding what can happen in the environment, including processes initiated not by the learner but by wind, rain, gravity, other animals, etc.

It also ignores the similarities between types of process that can be performed using different morphologies: e.g. carnivores use jaws to carry, manoeuvre and disassemble objects whereas primates (and some birds) use hands and feet.

The animals that can produce a-modal information structures using exosomatic ontologies (referring to things in the environment, not to sensory-motor signals) can learn things that have nothing to do with their own sensory or motor mechanisms, e.g. about effects of forces applied by one object to another, and effects of objects in constraining motions of other objects.

The comments on the videos relate to the notion of a "exploration domain" discussed in some of my talks, e.g. in 

    http://www.cs.bham.ac.uk/research/projects/cogaff/talks/#talk90
    Talk 90: Piaget (and collaborators) on Possibility and Necessity
    And the relevance of/to AI/Robotics

    (A related talk presented at Schloss Dagstuhl on 28th March 2011 will
    shortly be added.)

  1. Noticing and grasping edge of rug (Age about 6 months)
    Noticing the edge of the rug seems to trigger a very deliberate and controlled, but inexpert, attempt to grasp the edge, with clear success eventually, after rolling over onto his side.
    It looks as if this baby is exploring a number of different domains, initially
    Why did he use his right hand, not his left hand to grasp the rug?

  2. Playing piano and piano parts (Age about 9 months)
    No sound in this video but it is clear that the child alternates between fairly random thumping and controlled exploration of effects of pressing individual notes.
    What are the mechanisms that produce the motivation to do that?
    What cognitive processes are involved in the selection between possible actions, and their execution?
    What exploration domains are being extracted from the interaction with the enviroonment. Compare the later piano video below.

  3. Feeding brain and stomach with yogurt (Age about 10 months)
    The child manipulates a spoon in various ways, including getting yogurt out of a tub held in front of him, transferring some to his mouth, and some to carpet and his thigh.
    He seems to do a number of experiments of different kinds with spoon and yogurt, including at least two failed attempts to transfer yogurt to a flat surface, on carpet and on his thigh.
    The video ends with a successful complex process of transferring the end of the spoon from right hand to left hand, which requires coping with the fact that one hand is an obstacle to grasping by the other.
    (This is well beyond the capabilities of any robot I know about.)

  4. Playing piano and piano parts, recorded with sound. (Age about 11 months)
    A sequence of experiments concerned with piano keys, music holder, balancing on chair.
    The child discovers and explores separate domains of exploration, involving
    The video ends just as he seems to be discovering a repeatable musical theme, or possiblity a motor theme for his fingers, or perhaps a mapping between the two?

  5. Exploring materials: cloth (sweater) and mouse cable (Age about 11 months)
    (Doesn't quite strangle himself.)

  6. Pushing a broom (Age: about 15 months)
    A just about stable (well, upright anyway) dynamical system driven by an opportunistic cognitive explorer?
    Notice the various ways in which an action prepares for what is going to come later. This shows an understanding of constraints in the environment detected before the constraints operate. This cannot be achieved by purely reactive sensory-motor mechanisms where all internal processes are concerned with relating current inputs to current outputs.

  7. Failing to understand hook-and-eye mechanism (Age about 18 months)
    This child clearly has an impressive collection of motor, perceptual, and cognitive skills, and has mastery of a variety of exploration domains including switching between crouching and sitting positions, stacking objects, poking objets into holes, noticing a gap in the distance and walking to it in order to insert an object.

    He achieves all his goals apart from joining two train components -- trying to join two rings together instead of a hook and a ring. His understanding of possible ways of linking two objects is cearly still under developed. (I was told a few weeks later that he had mastered the problems, though nobody had observed steps in the transition.)

    Compare this with the intelligence of Betty the hook-making and hook-using crow.

  8. Several videos avaialable at the Behavioural Ecology Research Group, Oxford
    These include the very famous 'Trial 7' video, which shows Betty, the New Caledonian Crow, very expertly making a hook from a straight piece of wire, and then very expertly using the hook to lift a bucket of food out of a vertical tube.
    She had no opportunity to learn anything about this from other crows or humans.
    Betty made headlines world-wide in June 2002. Give to google: 
        betty crow hook

  9. Warneken and Tomasello's videos of children and chimps being helpful
    See especially the cabinet task.
    Those researchers are more interested in the fact that very young children and some other animals spontaneously try to help someone else.

    A different sort of question that I am interested in concerns competences that must be prior to those motivations: what sort of cognitive mechanisms make it possible for a pre-verbal child to tell what a person is trying and failing to do, and to work out what could be done to provide assistance. Here we seem to have a domain that is not concerned with what the observer can do but with possibilities available for another agent. (I have called those "vicarious affordances" elsewhere: very important both for parents and for young learners.)
    What motivates the child to use those cognitive abilities to provide help is an additional question. Being able to think about motivation requires a domain of meta-semantic competence: being able to represent and reason about things that can represent and reason. That clearly starts to develop before a child can talk.

    For both questions the answers are likely to involve a mixture of genetic mechanisms and competences learnt partly under the influence of the evironment and partly under the influence of the genetic mechanisms.

    The learning mechanisms probably change during learning and development, since what is learnable changes. But this is not merely a matter of what prior content is available for each type of learning. Rather new ontologies, new forms of representation, new algorithms and new information-processing architectures seem to be involved.

  10. Video of parrot scratching neck with feather.
    What sort of ontology does the parrot need in order to be able to select and control this action, including combining action with beak and foot in order to change grip location?
    Do you think it knows what it is doing? What does that mean?

  11. Boston Dynamics Big Dog (March 2008)
    Go to the Boston Dynamics web site to get more information.
    Do you think this robot knows what it is doing?
    Does it know anything about what it could do in the future?
    Does it know what it has done in the past?
    Does it know why it did that and not something else?
    Do the baby on the rug, or the baby playing with yogurt, or the toddler pushing a broom know what they are doing or not doing, or what they can do before they actually do it?
    What sorts of questions should biologists ask about robots?
    What sorts of questions should roboticists ask about animals?
See also

For further discussion of the issues raised by these videos see:
My slides for the 'Meeting of Minds' workshop in 2007, and the post-workshop notes on model-based semantics

Installed: 10 Nov 2010
Last Updated: 10 Nov 2010

Maintained by: Aaron Sloman