指導教授：簡韶逸臺灣大學：電子工程學研究所歐順興Ou, Shun-HsingShun-HsingOu2014-11-302018-07-102014-11-302018-07-102014http://ntur.lib.ntu.edu.tw//handle/246246/263920隨著通訊、計算能力、電腦視覺等技術的蓬勃發展，視訊感測器網路已被大量運用於日常生活中，例如安全監控、交通控制、環境監測等等。而在前景一片看好的物聯網時代，視訊感測器網路也將扮演重要角色，提供物聯網路豐富的資訊來源。然而，隨著大量增加的視訊感測器，將所有的影片串流至雲端或中央伺服器將會造成通訊頻寬以及儲存空間的不敷使用，如何有效管理大量的視訊資料成為相當重要的課題。另一方面，為了增加視訊感測器的靈活性，許多研究者投入無線視訊感測器網路的研究，由於不需要任何線路，無線視訊感測器可以提供更多元的應用。然而，電力的問題是無線視訊感測器網路的關鍵，由於影片資料相當龐大，透過無線通訊傳輸影片需要耗費大量的電力能源。因此，有效地在無線感測器上去除沒有實質資訊的資料將會是為無線視訊感測器的關鍵。 因應大量產生的影片資料，視訊摘要技術在近年來趨近成熟。視訊摘要技術的目的在於將原始影片「去蕪存菁」，以簡短的表示方式保留原本影片中所有的資訊。雖然大部分影片能夠提供大量資訊，但通常也有相當多沒有意義或是重複的資料。在視訊感測器網路中，重複的資料可能來自於重複發生的事物，也有可能來自於多隻畫面重疊或相關的感測器。 本論文提出將視訊摘要技術應用於視訊感測器網路，藉由視訊摘要技術將影片中重複、沒有資訊的部分去除，進而省下傳輸頻寬、儲存空間、以及無線視訊感測器的電力資源。然而，大部分現有的視訊摘要技術必須先將所有的影片集中儲存，才能進行摘要的運算，而且其運算通常需要大量的計算資源，因此不適合用於視訊感測器網路。在本篇論文中，我們提出兩種分散式線上多視角視訊摘要技術，此兩架構均不需要大量運算資源，視訊摘要的過程可以即時完成。實驗證明所提出的系統架構能夠有效的去除大量資料（60%–90%），只留下重要的資訊，節省傳輸頻寬、儲存空間、以及相當的電力資源（50%–80%）。 而本篇論文也利用嵌入式系統實作了所提出的無線視訊感測器網路，在無線視訊感測器上加上了視訊摘要原件，驗證整體系統的可行性。該實作證明所提出的多視角視訊摘要技術能被有效地運用在無線視訊感測器上。With the rapid development in video processing and sensing technologies, and the revolution of Internet-of-things (IoT) or machine-to-machine (M2M), the applications of video sensor networks become wider. However, due to the high data rate of video data, it is becoming impossible for all sensors to stream video to the server. Furthermore, in wireless video sensor networks, precise video data filtering is also the key to enable low-power wireless video sensors. Video summarization, or video abstraction, which aims to generate a short representation of the original videos, provides an excellent solution for video management. There are usually a lot of useless and redundant data in the video taken by video sensor networks, such as the redundancy in the temporal domain of the single camera, or the redundancy in spatial domain across a group of related cameras. Streaming all those video data wastes transmission bandwidth, storage spaces, and the precise energy on wireless video sensors. In this thesis, we proposed to apply multi-view video summarization in video sensor networks to help reducing transmission bandwidth, storage spaces, and the power consumption on wireless video sensors. The video data can be reduced while keeping important information. However, most existing summarization methods are off-line and centralized, which means all videos are still required to be streamed back the server before performing summarization. Furthermore, the computation complexity is usually high, making them not suitable to operate on video sensors. To address the above issues, two distributed on-line multi-view video summarization methods are proposed. Both methods can operate under low computational resources. Experiments show that although processing video under much more difficult constraints, the proposed distributed on-line summarization methods still generate comparable results compared with other centralized or off-line methods. Important events can be kept while reducing large amount of video data. Power analysis of the system also shows that a significant amount of energy can be saved. Besides simulation, we also implement our own wireless video sensor networks with the summarization system. The implementation further validates our ideas of the proposed summarization methods.List of Figures v List of Tables vii Abstract ix 1 Introduction 1 1.1 Video Sensor Networks....................... 1 1.2 Video Summarization........................ 4 1.3 The Proposed System........................ 4 1.4 Thesis Organization......................... 6 2 Background Knowledge and Overview 7 2.1 Related Works............................ 7 2.1.1 On-line Summarization................... 9 2.1.2 Multi-video Summarization ................ 11 2.2 Overview .............................. 12 2.2.1 Distributed On-line Multi-view Video Skimming . . . . . 12 2.2.2 Distributed On-line Multi-view Keyframe Extraction . . . 13 3 Distributed On-line Multi-View Video Skimming 15 3.1 Intra-view Stage: On-line Single-view Video Skimming . . . . . . 17 3.1.1 Feature Extraction ..................... 17 3.1.2 On-line Clustering ..................... 17 3.1.3 Single-view Summary Generation . . . . . . . . . . . . . 20 3.2 Inter-view Stage: Distributed View Selection . . . . . . . . . . . 22 3.2.1 Feature Extraction and Score Estimation . . . . . . . . . . 23 3.2.2 Communication Issues ................... 24 3.3 Summary Visualization....................... 27 3.3.1 Segment Linking...................... 28 3.3.2 Group Concatenating.................... 28 4 Distributed On-line Multi-View Keyframe Extraction 29 4.1 Video Maximal Marginal Relevance ................ 29 4.1.1 Feature Extraction ..................... 30 4.2 On-line and Distributed Keyframe Selection . . . . . . . . . . . . 31 4.3 MSWAVE .............................. 32 5 Experiments 35 5.1 Dataset ............................... 35 5.2 Distributed On-line Multi-view Video Skimming . . . . . . . . . 39 5.2.1 Intra-view Stage: Single-View Video Skimming . . . . . . 39 5.2.2 Entire System: Multi-view Video Skimming . . . . . . . 47 5.2.3 Computational Complexity................. 47 5.2.4 Packet Loss......................... 50 5.3 Distributed On-line Multi-view Keyframe Extraction . . . . . . . 51 5.4 Power Analysis ........................... 54 6 Implementation 59 6.1 Hardware Platform: Raspberry Pi ................. 59 6.2 System Design ........................... 60 6.2.1 Wireless Video Sensors................... 60 6.2.2 Video Sensor Networks................... 61 6.3 Implementation Detail ....................... 61 6.3.1 Video Acquisition and Encoding .............. 61 6.3.2 Wireless Communication.................. 64 6.4 Result................................ 64 7 Conclusion 67 8 Reference 6916827500 bytesapplication/pdf論文公開時間：2014/07/29論文使用權限：同意有償授權(權利金給回饋學校)視訊摘要視訊感測器網路thesishttp://ntur.lib.ntu.edu.tw/bitstream/246246/263920/1/ntu-103-R01943028-1.pdf