Dates for Semester 2, 2010.

See also the course calendar.

Week 1 - 19/7

Introduction

Course Overview
Agent/Planning systems

Introduction to Bayesian statistics

BayesianPrediction

What is a (Bayesian) probability?
What is Bayes' Rule?
How can you use Bayes' Rule to do sequence prediction?

Week 2 - 26/7

Utility Theory

?Bayesian Agents

Representation
  Simple probability distributions
  Observability assumptions

Lab (Level 3)

Intro to rUNSWift agent
  Subversion
  Compiling
  Running
  Base Station

AIBO-Overview

rUNSWift overview
  Aperios
  Our Objects/systems
    Low/High level vision
    Tracking
    Behaviours
    Wireless
    ActuatorControl
  Flow Control between objects
  Interfaces between objects

RobotVision

Robot Vision
  Colour segmentation
  Blobbing
  High level concepts

Kinematics

Motion (low level)
  Head Control
  Forward and Inverse Kinematics  (Head and Legs)
  Point Projection
  Non-Commutable Rotations
  Recorded motions

Week 3 - 2/8

BayesianLocalisationI

Tracking
  Bayesian Localization I
    Tabular representation

BayesianLocalisationII

Tracking
  Bayesian Localization II
    Particle Filters
    Kalman-Bucy Filters

Lab (Level 3)

Ball tracking
Assignment 1 out at end of lab (Tracking & Life Saver)
Assignment 3 out at end of lab (Grab Ball - get to other half of field)

Week 4 - 9/8

BayesianLocalisationIII

Finish Tracking
Information form Kalman Filters
SLAM
  Expectation Maximisation if time

Walking

Motion (Walking)
  Walking
    Normal Walk
    Odomentry Calibration
    Parameter Optimisation

Lab (Level 3)

Go To Point
  Experiments with tracking (odom/obs, bad mean/var)

Week 5 - 16/8

AIBO Behaviour and Comms

Wireless
  UDP vs TCP
Behaviours
  Hysteresis
  Hierarchies
  Software Engineering and State

Agent Architectures

Agent Architectures
  Engineered (subsumption)
  Logic based (Three layer, planning, golog)
  Cognitive (ACT-R, not Golog in this sense)
  Reinforcement Learning

Lab (Level 3)

Work on Assignment 1

Week 6 - 23/8

SearchI

Introduction to Search
  BFS/DFS
  A*
  Islands

SearchII

Further search - heuristics
  Mixing Search and Heuristics (Korf)
  Heuristics as simplifications/relaxations

Lab (Level 3)

Assignment 1 graded in lab

Week 7 - 30/8

Overview and Models

Reinforcemnt Learning
  Style of problem
  Optimality criteria
  Exploration/Exploitation trade-off

?PlaningI

Planning
  State Space search

Lab (Computer lab)

A*/Islands/Heuristics

Mid Semester Break - 6/9

Week 8 - 13/9

MDP

Markov Decision Processes
  Formal definition
  Model based updates
  Certainty equivalence
  Prioritized Sweeping
  Model free updates
    Q-learning
    Adaptive Heuristic Critic

PlanningII

Planning
  Plan space planning

Lab (Level 3)

Work on Assignment 3

Week 9 20/9

Discretization

Applying discrete search to continuous problems
  Obstacle expansion
  Different representations

OptimizationI

Rapidly-Exploring Random Trees
Introduction to Optimization
  1D Golden Section Search

Lab (Computer lab)

Planning

Week 10 - 27/9

OptimizationII

Gradient descent
  Simplex search
  Powell's Method

Examples

Gradient Descent
  Conjugate Gradient
  Quasi-Newton methods
  Gradient estimation

  Handling noise
    Oversampling
    Pegasus and pseudo-randomness
    Function fitting
    Meta-search

  Examples
    Walk learning as application of optimization
    Gradient descent over a stochastic policy in gridworld

Lab (Computer Lab)

Optimization

Week 11 - 4/10

POMDPs

Partially Observable Markov Decision Processes
  Belief state Tracking based methods
  History methods
  Explicit memory methods

  Piecewise linear value function

Genetics

Genetic Algorithms
  Shock

Lab

No Lab - use time for review if needed.

Week 12 - 11/10

Adversarial

Assignment 2 Due at start of lecture.

Adversarial Agents
  Minimax search
  Stochastic Policies
  Best Response Learning

Function Approximation

Review

Week 13 - 18/10

Lab (Level 3)

Assignment 3 graded in lab