News video story segmentation using fusion of multi-level multi-modal features in TRECVID 2003.
Journal
2004 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2004, Montreal, Quebec, Canada, May 17-21, 2004
Pages
645-648
Date Issued
2004
Author(s)
Hsu, Winston H.
Kennedy, Lyndon S.
Huang, Chih-Wei
Chang, Shih-Fu
Lin, Ching-Yung
Iyengar, Giridharan
Abstract
We present our new results in news video story segmentation and classification in the context of the TRECVID video retrieval benchmarking event 2003. We applied and extended the maximum entropy statistical model to fuse diverse features effectively from multiple levels and modalities, including visual, audio, and text. We have included various features such as motion, face, music/speech types, prosody, and high-level text segmentation information. The statistical fusion model is used to discover automatically relevant features contributing to the detection of story boundaries. One novel aspect of our method is the use of a feature wrapper to address different types of features - asynchronous, discrete, continuous and delta ones. We also developed several novel features related to prosody. Using the large news video set from the TRECVID 2003 benchmark, we demonstrate satisfactory performance (F1 measure up to 0.76) and, more importantly, observe an interesting opportunity for further improvement.
SDGs
Type
conference paper