Birmingham's Role in Planning Research in GeRT

The Planning work package aims at developing and evaluating techniques for generalising the high-level behaviour in the robot programs and then generating new high-level behaviours for novel but analog tasks using planning techniques. Thus, we will learn action models in some suitable hybrid symbolic-geometric representation based on sets of example programs. This involves learning preconditions and effects, and in particular the parts of those preconditions and effects that concern the kinematics of the robot and its relation to the geometry of the scene. The example programs can also provide us with useful information about how to actually execute the modelled action. Birmingham's involvement in the work package is mostly in learning the action models from the example programs, although we will also contribute to defining the hybrid representation language.

We will use the action models to automatically generate plans for solving novel tasks, by which we mean tasks involving novel configurations of novel objects belonging to familiar classes, and involving familiar types of actions. For that purpose, we will develop a hybrid geometric-symbolic planner, which can perform reasoning both on the symbolic level and on the geometrical/kinematic level. The major effort here will be on handling the hybrid states and transitions between them. Finally, the actions in these plans should be turned into robot programs that are appropriate for the current situation and executed by the robot. This work is primarily taking place at Örebro University, although Birmingham will also be involved.

Action Learning

Given that the problem is to learn actions from example programs and then build plans with those actions, a very important question is what the most appropriate level of abstraction is for the learned actions. There are three dimensions to the problem: Firstly, to determine the best trade-off between the learned representation and the difficulty of planning. More generalized action models (e.g. grasp-object(x)) make planning easier, as the action space is smaller. On the other hand, the learning task is easier if the action models concern more specific actions (e.g. grasp-bottle-on-left-side(x)).

Secondly, we need to choose how much of the geometric state to abstract away. A purely symbolic planner risks either over- or under-estimating the situations in which actions are applicable, and might make it impossible to express e.g. why a certain action failed in a given situation. However, including all the geometric information in the planner’s representation makes the planning task much more computationally demanding and makes domain modelling harder.

Finally, generalizing actions from examples is much easier if the example programs are annotated with information about for instance what the operators achieve. However, annotating the programs is a significant extra demand to place on program designers and in an ideal situation would be minimized or avoided. These three dimensions will be explored both theoretically and experimentally in the parts of the work package which address learning of action models and hybrid planning.

In addition to all of these issues, the major challenge in action learning is that the learning must be accomplished from a very small number of examples. Annotation of the examples will certainly help with this, as will the availablility of a kinetic simulation that will allow potentially useful actions to be evaluated in different conditions.

Research Tasks

The three specific tasks that Birmingham are involved with are:

Task 1.3: Defining an action language

This task consists of defining a language for hybrid action models and situations. The language should be able to express preconditions and effects both in symbolic and geometric terms. The language should also allow certain elements of the symbolic and geometric components to be connected, along the lines of how symbolic places are connected to configurations in the work of Cambon et al [2009]. For instance, it is important to be able to connect the clause right-hand-grasping(o1) to the geometric/kinematic configurations where Justin’s right hand is grasping the object designated by the symbol o1

Task 1.4: Generalizing programs to action models

The problem here is to go from a set of programs and runs to action models. Action models are parameterised, e.g. grasp(x), meaning that they can be applied to a range of objects x. They include preconditions and effects, which may involve geometric information. In particular the geometric aspects of the actions need to be learned. The trade-off between manual annotation and autonomous learning of different aspects of the model will be studied here, as will the trade-off between generality/specificity of the action models (e.g. grasp-object(x) vs. grasp-bottle-from-left-side(x)) and its impact on the learning task.

Task 1.5. Hybrid planning

The action models are used to generate plans, consisting of sequences of actions. While searching for a plan, the action models tell us when a certain action is applicable (preconditions), and what effects it would have. Besides the action models, the planner would take as input the present situation and a goal. The planner must work in this space of hybrid symbolic-geometric situations. Hence the major scientific challenges are how to check whether actions are applicable and how to compute their effects (transitions between hybrid situations). The trade-offs regarding how general/specific actions are and how this affects the planning task, and how much geometric information to use in action models, will be investigated. The potential role of the example programs in the search in the configuration space is also an interesting question that will be investigated.