Research Seminar - April 08, 2003

Recognising Events in Video Streams

Abstract:

The recognition of events within multi-modal data is a challenging problem. I have been investigating the use of Hidden Markov Models (HMM) to model audio and video data sequences and also data fusion techniques in order to recognise these sequences. Specifically I have been looking at the recognition of play and break sequences in football and the segmentation of recordings of football games based on these two events. Play is when the ball is in normal play and break is when the game has stopped for some reason, such as a penalty, freekick or goal. Recognising relatively simple semantic events such as these is an important step towards full automatic indexing of such video material. I propose that modelling the audio and video streams separately for each sequence and fusing the decisions from each stream should yield an accurate and robust method of segmenting multi-modal data. The results presented here show that this technique improves segmentation over feature vector concatenation techniques.