CoSy project

Cosy Meeting of Minds Workshop

List of Posters to be presented by members of the CoSy Project

For main schedule see

Hendrik Zender, Patric Jensfelt, Oscar Martinez Mozos, Geert-Jan M. Kruijff, and Wolfram Burgard (2007)
"An Integrated Robotic System for Spatial Understanding and Situated Interaction in Indoor Environments"

A major challenge in robotics and AI lies in creating robots that are to cooperate with people in human-populated environments, e.g. for domestic assistance or elderly care. Such robots need skills that allow them to interact with the world and the humans living and working there. In this work we investigate the question of spatial understanding of human-made environments. The functionalities of our system comprise perception of the world, natural language, learning, and reasoning. For this purpose we integrate state-of-the-art components from different disciplines in AI, robotics, and cognitive systems into a mobile robot system.
Here we describe the principles that were used for the integration, including cross-modal ontology- based mediation, and processing of perception on multiple levels of abstraction. Finally, we present experiments with the integrated "CoSy Explorer" system and list some major lessons that were learned from its design, implementation, and evaluation.

Danijel Skocaj, Alen Vrecko, Matej Kristan, Barry Ridge, Gregor Berginc, and Ales Leonardis
"A System for Continuous Learning of Visual Concepts"

We present an artificial system for learning simple visual concepts. It comprises of vision, communication and manipulation sub- systems, which provide visual input, enable verbal and non-verbal communication with a tutor and allow interaction with a given scene. The main goal is to learn associations between automatically extracted visual features and words that describe the scene in an open-ended, continuous manner. In particular, we address the problem of cross-modal learning of visual properties and spatial relations. We introduce and analyse several learning modes requiring different levels of tutor supervision.

Sanja Fidler and Ales Leonardis
"Towards Scalable Representations of Object Categories: Learning a Hierarchy of Parts"

We propose a novel approach to constructing a hierarchical representation of visual input that aims to enable recognition and detection of a large number of object categories. Inspired by the principles of efficient indexing, robust matching, and ideas of compositionality, our approach learns a hierarchy of spatially flexible compositions, i.e. parts, in an unsupervised, statistics-driven manner. Starting with simple, frequent features, we learn the statistically most significant compositions (parts composed of parts), which consequently define the next layer. Parts are learned sequentially, layer after layer, optimally adjusting to the visual data. The lower layers are learned in a category-independent way to obtain complex, yet sharable visual building blocks, which is a crucial step towards a scalable representation. Higher layers of the hierarchy, on the other hand, are constructed by using specific categories, achieving a category representation with a small number of highly generalizable parts that gained their structural flexibility through composition within the hierarchy. Built in this way, new categories can be efficiently and continuously added to the system by adding a small number of parts only in the higher layers. The approach is demonstrated on a large collection of images and a variety of object categories. Detection results confirm the effectiveness and robustness of the learned parts.

Nick Hawes, Aaron Sloman, Jeremy Wyatt, Henrik Jacobsson, Geert-Jan M. Kruijff, Michael Brenner, Gregor Berginc and Danijel Skocaj
"Towards an Integrated Robot with Multiple Cognitive Functions"

We present integration mechanisms for combining heterogeneous components in a situated information processing system, illustrated by a cognitive robot able to collaborate with a human and display some understanding of its surroundings. These mechanisms include an architectural schema that encourages parallel and incremental information processing, and a method for binding information from distinct representations that when faced with rapid change in the world can maintain a coherent, though distributed, view of it. Provisional results are demonstrated in a robot combining vision, manipulation, language, planning and reasoning capabilities interacting with a human and manipulable objects.

Henrik Jacobsson, Geert-Jan Kruijff and Maria Staudte
"Language Acquisition from Neural and Sensorimotor Systems"

A fundamental learning problem in adaptive, embodied cognitive systems is how to learn discrete models of situated experience which can mediate between sensorimotoric expe- rience and high-level cognitive processes (such as language and planning). Recent approaches include learning of discrete Hidden Markov Models for behaviour description and active (curiosity driven) learning of perception prediction. Through curiosity driven learning, also communication itself can be "discovered" among other actions as a source of rich information. It can also be argued that in order to learn a situated form of language that takes into account the embodiment of the robot, there needs to be a close correspon- dence between the sensorimotor and linguistic capabilities. The robot should communicate about what it knows, and through active learning it could know how its actions will affect the environment.
Continued here (PDF)

Michael Brenner, Nick Hawes, John Kelleher and Jeremy Wyatt
          Poster presented at IJCAI 2007:
        "Mediating Between Qualitative and Quantitative Representations for Task-Orientated Human-Robot Interaction"

A robot can only display long-term intelligent behaviour, in particular when interacting with humans, if it can reason qualitatively about current and future states of the world. On the other hand, being an embodied cognitive system a robot must also interact with the real world and therefore must link its qualitative representations to perceptions and actions in continuous space. For this kind of mediation between representations our paper contributes:

Somboon Hongeng and ...
Something on action perception


Juergen Sturm
Unsupervised Body Scheme Learning through Self-Perception

In this paper, we present an approach allowing a robot to learn a generative model of its own physical body from scratch using self-perception with a single monocular camera. Our approach yields a compact Bayesian network for the robot's kinematic structure including the forward and inverse models relating action commands and body pose. We propose to simultaneously learn local action models for all pairs of perceivable body parts from data generated through random "motor babbling." From this repertoire of local models, we construct a Bayesian network for the full system using the pose prediction accuracy on a separate cross validation data set as the criterion for model selection. The resulting model can be used to predict the body pose when no perception is available and allows for gradient-based posture control. In experiments with real and simulated manipulator arms, we show that our system is able to quickly learn compact and accurate models and to robustly deal with noisy observations.

Aaron Sloman [Did not find time to do this poster]
A poster on the problem of identifying requirements for a human-like robot, and some unobvious requirements.

Identifying what humans, at various stages of development, and for that matter other animals, can and cannot do is not an easy task. The obvious means, namely observing their actions, is fraught with problems because it is impossible to tell from any observed behaviour what competence underlies that behaviour. Paradoxically, it is possible to gain insight into possible explanations of the competence of a human or other animal by attempting to design robots capable of performing a similar variety of tasks provided that one studies a broad enough variety in enough detail, since a working artificial model that appears to explain some performance may fail to "scale out" to produce other performances in other contexts as natural competences do. Some of the requirements for performing those tasks become evident only as a result of developing and testing designs and investigating reasons why they do not work. It is also possible to do this retrospectively for the field as a whole: namely developing requirements by analysing limitations of work done so far and trying to understand what is needed for further progress. Theoretical work of that sort has played a significant role in the CoSy project, and some of the results in the first two years are in the papers on 'Requirements' and 'Architectures' in Work Package 1:
WP1: Architecture and Representation.  Other relevant work includes the euCognition Network's  Research roadmap  project and this proposal for de-fragmenting AI.

Maintained by Aaron Sloman
Last updated 3 Oct 2007