Research Seminar - October 15, 2004

Applications of visual tracking

Abstract: Applications of visual tracking

The talk will comprise two parts discussing different applications of visual tracking: (i) human motion capture; (ii) visual localisation and mapping.

PART I: Human Motion Capture aims to determine the 3D pose and joint parameters of a moving person for subsequent use in a variety of applications ranging from gait analysis for orthopeadic rehabilition to reanimation for computer games. A standard approach fixes markers to a subject then tracks these in multiple, hardware synchronised and accurately calibrated cameras.

In the first part of this seminar I will discuss a system able to perform robust, markerless, visual tracking with an articulated body model. The approach to searching through the high-dimensional model configuration space is an algorithm called annealed particle filtering which finds the best fit to image data via multi-layer propagation of a stochastic particle set. This algorithm efficiently searches the configuration space without the need for strong dynamical models. I will describe the algorithm, and present results for a variety of agile motions.

PART II: As a video camera moves through the world, the image trajectories of static scene features provide constraints on both their locations in the world and on the motion of the camera itself. Thus, a video camera module connected to a computer can theoretically serve as a localisation device by identifying and mapping previously-unknown natural visual landmarks in the surroundings.

Though the theory of structure-from-motion is well understood, there remain various problems to overcome in any practical implementation. In the second part of the seminar I will describe a system that automatically acquires, tracks and maps visual features, and uses these to localise the camera sequentially in real time (i.e. 30Hz). The system is characterised by (i) a sparse, high-quality map of visual features; (ii) an Extended Kalman Filter whose state represents the instantaneous estimates of 3D feature and camera locations; (iii) uncertainty-guided active feature search for robust tracking.