video modeling is a fundamental
an important role in many content-based video analysis
applications. Recent research shows
Gaussian mixture models (GMMs) offer good capability
and flexibility of joint spatio-temporal video modeling.
However, the major bottleneck of this kind of approaches is
the high computational complexity and possible over-fitting
during model learning. In this work, we have improved the efficiency and
robustness of GMM training by incorporating key-frame extraction
model learning. Specifically, key-frame extraction is formulated as a
process for video segmentation. Hence we have proposed a new video generative model
that embeds frame saliency in GMM-based video modeling. The proposed
video model is able to speed up the GMM training significantly by optimally selecting a set of video frames with strong saliency.
interestingly, all video frames can be characterized by a saliency
indicator that exhibit specific joint spatio-temporal characteristics of
visual features. This new video model may also lead to new
functionalities for content-based video analysis.
X. Song and G. Fan, "Selecting
Salient Frames for Spatio-temporal Video Segmentation", IEEE Trans. on
Image Processing, Vol. 16, No. 12, Dec. 2007.
L. Liu and G. Fan, “Combined Key-frame and
Object-based Video Segmentation”, IEEE Trans. Circuits and System for
Video Technology, Volume 15, No. 7, pp869 – 884, July, 2005.
X. Song and G. Fan, "Key-frame Extraction for
Object-based Video Segmentation", in Proc. of the IEEE International
Conference on Acoustics, Speech and Signal Processing (ICASSP2005),
Philadelphia, PA, March 2005.
X. Song and G. Fan, “Joint Key-frame Extraction and
Object Segmentation for Content-based Video Analysis”, IEEE Trans. on
Circuits and Systems for Video Technology, Vol. 16, No. 7, pp904- 914,
X. Song and G. Fan, "Joint Key-frame
Extraction and Object-based Video Segmentation", in Proc. of IEEE
Workshop on Motion and Video Computing (MOTION 2005), Breckenridge,
Colorado, Jan. 5-6, 2005.
X. Song and G. Fan, “A New Video Analysis Approach
for Coherent Key-frame Extraction and Object Segmentation”, in Proc.
of IEEE International Workshop on Multimedia Signal Processing,
Shanghai, China, Oct-Nov. 2005.