TITLE: Concurrent Discovery of Task Hierarchies

PRESENTER: Duncan Potts, Tel: +61 2 9385 6917

AFFILIATION: School of Computer Science & Engineering, UNSW and NATIONAL ICT AUSTRALIA

DATE: Friday 12th March 2004

TIME: 12:00 noon - 1:00pm

PLACE: Seminar Room K17

ABSTRACT:

Task hierarchies can be used to decompose an intractable problem into smaller more manageable tasks. This paper explores how task hierarchies can model a domain for control purposes, and examines an existing algorithm (HEXQ) that automatically discovers a task hierarchy through interaction with the environment. The initial performance of the algorithm can belimited because it must adequately explore each level of the hierarchy before starting construction of the next, and it cannot adapt to a dynamicenvironment. The aim of this talk is to present an algorithm that avoids any protracted period of initial exploration by discovering multiple levels ofthe hierarchy simultaneously. This can significantly improve initial performance as the agent takes advantage of all hierarchical levels early onin its development. Robustness is also improved because undiscovered features and environment changes can be incorporated later into the hierarchy. Empirical results show the new algorithm to significantlyoutperform HEXQ. This talk is based on a paper accepted for the 2004 AAAI Spring Symposium on Knowledge Representation and Ontology for Autonomous System.

BIOGRAPHY OF SPEAKER:

Duncan Potts has been a PhD student for 2 years here at UNSW. His research started in reinforcement learning, and specifically looking at ways of building hierarchies to make the systems more scalable. However recent research is extending these hierarchical methods to more general forms ofincremental learning, with a focus on controlling dynamic systems.

Seminar Convenor:

Bernhard Hengst
Tel: +61-2-9385-3988,
E-mail: bernhardh@cse.unsw.edu.au
School of Computer Science & Engineering, UNSW.