Semantics-based Content Analysis and Organization in Movies and Sports Videos
Date Issued
2006
Date
2006
Author(s)
Chu, Wei-Ta
DOI
en-US
Abstract
Conducting content analysis approaching semantics level is an emerging trend in multimedia researches. Such kind of analysis matches users’ needs and facilitates content management and utilization in a more effective and reasonable way. Unlike conventional content-based retrieval or indexing, works on semantics analysis integrate techniques of statistical pattern recognition and machine learning with specific production rules or domain knowledge to bridge the semantic gap between low-level features and high-level semantics.
On the basis of machine learning and pattern recognition technologies, systems that combine analytical results from different classifiers, different features, or different modalities are developed. In this dissertation, we propose a general framework that introduces the idea of mid-level representation between audiovisual features and semantic concepts. Two types of techniques, i.e. statistical pattern recognition and rule-based decision, are combined to facilitate narrowing the semantic gap.
We develop three systems that respectively conduct semantic concept detection in action movies, in broadcasting baseball games, and in sports videos. In action movies, we detect semantic concepts, such as gunplay and car-chasing scenes, through analyzing aural information. Statistical approaches are exploited to characterize concept modeling and to facilitate mapping between different semantic granularities. In baseball games, visual and speech information are combined, and a hybrid method that includes rule-based and statistical techniques is designed for semantic concept detection. Thirteen semantic concepts, such as single, double, homerun, and strikeout, are explicitly detected, and several realistic applications can therefore be built. In general sports videos, we extract the ball trajectory to be a new type of metadata for describing content characteristics. Some novel semantic concepts, such as pitch types in baseball games, can therefore be modeled and detected. These studies are the instances of the proposed general framework and demonstrate the realization of automatic semantic concept detection.
Subjects
語意分析
影片分析與組織
事件與概念偵測
視訊檢索
semantic analysis
video analysis and organization
event and concept detection
video indexing
Type
thesis
File(s)![Thumbnail Image]()
Loading...
Name
ntu-95-D91922016-1.pdf
Size
23.31 KB
Format
Adobe PDF
Checksum
(MD5):ae11c7de818476bca672f6673ef4ee8d