School of Computer Science

GeRT: Generalizing Robot Manipulation Tasks

Justin working with tea

DLR's robot Justin performing one of the GeRT example tasks.


In order to work naturally in human environments such as offices and homes, robots will need to be much more flexible and robust in the face of novelty. In this project we are working to develop new methods to cope with novelty in manipulation tasks. Our approach is to enable the robot to autonomously generalize its manipulation skills to new objects. The basic idea is that some successful implementations of a certain robot manipulation task, such as serving a drink, are given as input. These programs then constitute a database of prototypes representing that class of task.

When confronted with a novel instance of the same task the robot needs to generalize from the prototypes, establishing appropriate correspondences between objects and actions in the prototypes and their counterparts in the novel scenario. In this way, prototypical task plans may be mapped to a new plan that is suitable for handling different geometric, kinematic, and dynamic task settings, hence solving a task that is physically substantially different but similar at an abstract level. This kind of knowledge transfer or generalization is not restricted just to the most abstract layer. Rather, low-level primitives as well as high-level logical robot actions and operators will be adapted.


The overall aim of the GeRT project is to enable a robot to autonomously generalise its manipulation skills from a set of known objects to previously un-manipulated objects in order to achieve an everyday manipulation task.

Specifically, the project is working towards enabling a robot to generalize from known object instances to previously unseen objects drawn from the same class. In addition we are trying to marry low level robotic control with high level AI planning approaches to enable the robot to reason about how the overall task should influence manipulations of individual objects. The result will be a robot system able to manipulate previously unseen objects to achieve an everyday task, such as making a drink.

For a more detailed description of the project at Birmingham, see our GeRT Website.