吳家麟臺灣大學：資訊工程學研究所鄭文皇Cheng, Wen-HuangWen-HuangCheng2007-11-262018-07-052007-11-262018-07-052004http://ntur.lib.ntu.edu.tw//handle/246246/53649隨著多媒體文件在數量上的急遽增加，人們對於如何簡明地表現該些文件的精華變得更加熱切。其中一個重要的技術即為興趣區 (region-of-interest, ROI) 決定。傳統的興趣區分析主要著重於兩種多媒體文件型式：影像 (image) 與視訊 (video)。然而，對於視訊方面的研究成果卻遠落後於影像的相關研究。這種情形肇因於沒有適當地考量影像及視訊兩者間在本質上的差異，同時更忽略了視訊獨有的部份特性。 面對如此一個具挑戰性的研究課題，我們提出了一個以使用者注意模型 (user attention model) 為基礎的自動視訊興趣區決定架構。在這個研究中，視訊的注意特徵值 (attention features) 及應用媒體美學 (applied media aesthetics) 的知識都被同時考慮且利用。我們將視覺注意特徵值區分為三個基本種類：亮度 (intensity) 、顏色 (color) 及運動 (motion)。參考美學的原則，這些特徵值以一個新提出之稱為訊框切片 (Frame-segment) 的視訊分析單位為基礎，同時依據攝影機運鏡 (camera motion) 的種類而加以整合。在實驗中，對於數種不同的視訊資料進行了興趣區分析及使用者相關研究並證明了所提架構的有效性。我們視本研究為達成更高階具意義性視訊分析的一個重要基礎。With the amazing growth in the amount of multimedia documents, people have become enthusiastic to acquire a more concise and informative representation of these documents. One of the desired technologies is the region-of-interest (ROI) determination. Conventional ROI analysis concentrates on two fundamental types of multimedia documents: image and video. However, the research performance of videos is far behind that of images. The phenomena are arisen from unsuitably considering the essential differences between image and video, and some video’s specific characteristics are ignored. Facing such a challenging issue, we propose a framework for automatic ROI determination in videos based on user attention model. In this work, a set of attempts on using video attention features and knowledge of applied media aesthetics are made. We classify visual attention features into three fundamental categories: intensity, color, and motion. Referring to aesthetic principles, these features are combined according to the camera motion types on the basis of a proposed video analysis unit, the frame-segment. We conducted lots of experiments on several kinds of video data and demonstrated the effectiveness of the proposed framework. This work is viewed as a preliminary step towards the solution of high-level semantic video analysis.CHAPTER 1 INTRODUCTION 1.1 REGION-OF-INTEREST (ROI) 1.2 WHY IS THIS PROBLEM INTERESTING? 1.3 RELATED WORK 1.3.1 Prior Works on User Attention Model 1.4 PROBLEMS ADDRESSED 1.5 CONTRIBUTIONS 1.6 THESIS ORGANIZATION CHAPTER 2 SYSTEM FRAMEWORK CHAPTER 3 SHOT BOUNDARY DETECTION 3.1 INTRODUCTION 3.2 A UNIFIED SHOT BOUNDARY DETECTION SCHEME 3.2.1 Spatial Color Descriptor 3.2.2 Flashlight and Abrupt Cut Detection 3.2.3 Gradual Transition Detection 3.3 EXPERIMENTS 3.4 SUMMARY CHAPTER 4 VIDEO USER ATTENTION REPRESENTATION 4.1 INTRODUCTION 4.2 FRAME-SEGMENT 4.3 USER ATTENTION MODEL 4.3.1 Contrast Based Intensity and Color Feature Model 4.3.2 Motion Feature Model 4.3.3 Feature Map and Filtered Feature Map 4.4 RELATIONSHIP BETWEEN CAMERA MOTION AND USER ATTENTION 4.4.1 Camera Motion Registration 4.5 SALIENCY MAP GENERATION 4.6 SUMMARY CHAPTER 5 VIDEO ROI DETERMINATION 5.1 INTRODUCTION 5.2 SALIENCY WEIGHTED REGULAR MOMENT 5.3 DYNAMIC ROI NUMBER DETERMINATION 5.4 SUMMARY CHAPTER 6 EXPERIMENTAL RESULTS 6.1 SAMPLE RESULTS 6.2 THE USER STUDY CHAPTER 7 CONCLUSIONS AND FUTURE WORK 7.1 CONCLUSIONS 7.2 FUTURE WORK BIBLIOGRAPHY APPENDIX A2966403 bytesapplication/pdfen-US使用者注意模型興趣區訊框切片攝影機運鏡應用媒體美學user attention modelregion-of-interest (ROI)frame-segmentapplied media aestheticscamera motionthesishttp://ntur.lib.ntu.edu.tw/bitstream/246246/53649/1/ntu-93-R91922002-1.pdf