School of Computer Science
Personal Web Page

Research Interests

Research

Overview

Autosub 6000
Autosub 6000 being recovered.

I do research in a number of different fields within artificial intelligence (AI), but the unifying theme is reasoning under uncertainty. I believe consideration of uncertainty is the area of AI where the most significant recent progress has been made, and will prove to be an extremely important tool as AI techniques are applied in more and more real-world situations. My research can be broadly categorized into work on autonomy, planning, scheduling, diagnosis, and machine learning.

Autonomous Robotics

I am interested in the general problem of how to build autonomous systems. This includes problems such as how to interleave planning and execution, architectures for autonomy, state estimation in real-time, optimising resource usage in autonomous vehicles and execution monitoring.

Planning

My interest in planning is mostly in using Markov decision problems (MDPs) to represent planning tasks in which there are actions with uncertain effects, multiple goals or rewards, and potentially uncertainty about the state of the system. I think one of the major contributions of AI has been to identify subclasses of general problems for which efficient algorithms exist. That is, to identify and exploit structure where it is present in problems. I am interested in finding efficient solution techniques for MDPs that have compact representations and hopefully also compact solutions. More recently, I have been using partially observable MDPs to plan information gathering actions in applications such as searching for hydrothermal vents on the sea floor, or exploring a new environment.

I also have an interest in more classical approaches to planning, although still with uncertainty. In particular, I have worked on contingent planning problems,  using techniques from graph-plan to generate heuristic estimates of the value of adding a branch at a particular point in a plan.

Machine Learning

Justin
The Justin robot used in the GeRT project.

My interest in machine learning is quite broad, but the majority of my published work has been in reinforcement learning, in particular in optimal exploration and structured representations. The exploration problem involves deciding between exploiting knowledge you already or exploring in the hopes of learning something that will let you do better in the future. The optimal exploration policy of a reinforcement learning problem can be defined as an MDP over the space of possible experiences. However, this MDP cannot be solved in practice. The work I have done in this area has been in trying to find heuristics that approximate a local solution to this  exploration MDP .

My work in diagnosis of hybrid systems (discussed below) has left me with the strong impression that building models of these systems is a very hard problem. With this in mind, I have recently been looking at approaches to learn these models directly from data, perhaps with some input from the user to indicate which discrete mode of the system is generating a particular part of the data.

This interest in learning models from data is also reflected in the GeRT project, where we are interested in learning models of actions from data in the form of hand-built programs.

Scheduling

Uncertainty about resource usage is as much an issue in scheduling problems as it is in planning. An example of the kind of problem I am interested in is a space telescope that only transmits data to the ground at certain intervals, and must optimize its use of onboard storage for images, without knowing in advance how big a particular image will be after compression. Scheduling problems in which there are explicit models of uncertainty are a relatively new area in which there are many interesting problems to explore.

Diagnosis

The K9 rover
Fault detection on K9.

From 2000-2004 I worked in the Model-Based Diagnosis and Recovery group at NASA Ames Research Center. In that time I have seen a number of interesting diagnosis problems for which new techniques need to be developed. The area that interests me most is diagnosis of hybrid systems. The problem is to track the state of a system described in terms of both continuous and discrete variables. In principle, diagnosis on these models is simply Bayesian belief updating as observations arrive. However, this is rarely computationally feasible in practice. Instead my research has concentrated on doing Monte Carlo approximate belief update using particle filters.  Diagnosis problems present some unique challenges for sample-based approximations such as particle filters, and most of my research in this area has been on overcoming those obstacles.