Research Seminar - July 02, 1999

Abstract

The CMU Video Surveillance and Monitoring (VSAM) project is developing a surveillance system that will enable a single human operator to monitor activities over a complex area using a distributed network of active video sensors. Automated video understanding algorithms have been developed to automatically detect and track multiple people and vehicles within cluttered scenes, and monitor their activities over long periods of time. Human and vehicle targets are seamlessly tracked through the environment using a network of active sensors to cooperatively track targets over areas that cannot be viewed continuously by a single sensor alone. The idea is to allow a human operator to tap into a network of sensors deployed throughout the environment to get a broad overview of dynamic events.

Keeping track of people, vehicles, and their interactions, within a chaotic urban environment is a difficult task. The role of VSAM video understanding technology in achieving this goal is to automatically ``parse'' people and vehicles from raw video, determine their geolocations, and automatically insert them into a dynamic scene visualization. We have developed robust routines for detecting and tracking moving objects. Detected objects are classified into semantic categories such as human, human group, car, and truck using appearance-based and geometric properties, and these labels are used to improve tracking using temporal salience constraints. Further classification of human activity, such as walking and running, has also been achieved. Geolocations of labeled entities are determined from overlapping camera views via wide-baseline stereo, or from monocular camera views by intersecting viewing rays with a terrain model. Resulting object hypothesis information from all sensor processing units (SPUs), including object type and trajectory, are transmitted as symbolic data packets back to a central operator control unit (OCU), where they are displayed on a graphical user interface to give a broad overview of scene activities.

About the speaker

Dr. Alan Lipton received his PhD in Electrical and Computer Systems Engineering from Monash University, Melbourne, Australia in 1996. For his thesis, he studied the problem of mobile robot navigation by natural landmark recognition using on-board vision sensing. Since early 1997 he has been on the faculty at the Robotics Institute of Carnegie Mellon University and has been a project manager of DARPA's Video Surveillance and Monitoring (VSAM) effort. Under this program, he performs research in the areas of real-time object detection, tracking, and recognition from video.

An overview of the project can be found athttp://www-2.cs.cmu.edu/~vsam/