吳家麟臺灣大學：資訊工程學研究所楊恕先Yang, Shu-SianShu-SianYang2007-11-262018-07-052007-11-262018-07-052007http://ntur.lib.ntu.edu.tw//handle/246246/53768可調式視訊編碼 (Scalable Video Coding, SVC) 是針對需要傳輸或儲存多樣空間與時間解析度之影片的多媒體系統所設計，能藉由簡單地截去當前時間與空間解析度所不需要的位元流達成目標解析度影片之重建，同時並利用精細可調式編碼 (Fine Grain Scalability, FGS) 提供品質調整以適應頻道頻寬之變化。 在H.264/MPEG4-AVC 視訊編碼標準完成後，可調式視訊編碼成為 ISO/IEC MPEG 與 ITU-T VCEG 合組之聯合視訊小組 (Joint Video Team, JVT) 進行標準化之主要目標。在目前的設計中，多數H.264/MPEG4-AVC 所採用之先進編碼工具將被保留以達成基本層 (Base Layer) 的向後相容，此外，可調性與編碼效率間的取捨亦達到平衡，其 PSNR 值僅較傳統H.264/MPEG4-AVC 下降約 1 dB。 然而隨著可調式視訊編碼的高編碼效率及高彈性而來的是極為龐大的運算複雜度，如此高的複雜度限制實際應用的範圍，因此，在本論文中，我們探討如何善加利用可調式視訊編碼之分層結構以簡化其複雜度，並減少編碼所需之運算時間。我們提出的混合式移動估測 (Motion Estimation) 演算法能有效分配運算資源至最為需要的資料單元，能夠在大幅降低複雜度的同時，維持非常相近地編碼效率。此外，已運算資料之重複利用亦能大幅減少整體編碼所需時間。藉由上述方法，我們使整體編碼速度加速了 9 倍，而畫質或壓縮比僅因此些微衰退。 隨著多處理器系統的逐漸普及與多執行緒函示庫的成熟，平行處理被視為下個世代視訊編碼器的主流。然而傳統的單執行緒視訊編碼器並無法利用多處理器系統的優勢。我們基於可調式視訊編碼中的階層式 B-picture 結構所設計的平行處理演算法，可在輸出相同之編碼位元流下，提供較傳統 wave-front 平行演算法更高的運算速度。此外，我們亦提出一個高效率的H.264/MPEG4-AVC 去區塊效應濾波器 (Deblocking filter) 平行演算法，能提供較傳統方法高出 40% 之處理速度。Scalable Video Coding (SVC) is designed for multimedia systems which require video of various spatial and temporal resolutions to be transmitted or stored. In addition to the Fine Grain Scalability (FGS), SVC provides the capability of reconstructing reduced spatial and/or temporal resolution video by simply discarding unnecessary part of the bit-stream from the complete one. After the debut of the latest single layer video coding standard, H.264/MPEG4-AVC, the Joint Video Team of the ISO/IEC Moving Picture Experts Group and the ITU-T Video Coding Experts Group invests a great deal of effort in the Scalable Extension of H.264/MPEG4-AVC. Most components of H.264/MPEG4-AVC are adopted in current SVC design to allow the base layer of an SVC bit-stream can be decoded by any H.264/MPEG4-AVC conforming decoder. The tradeoff between scalability and coding efficiency is also well-compromised. The PSNR degradation is controlled within 1 dB in average. Although SVC provides good coding performance and scalabilities, the additional flexibility also brings high computation complexity which limits application scenarios. To accelerate the encoder, we exploit the characteristics of the layered coding structure of SVC. We propose a hybrid motion estimation algorithm which puts computation resources to the most needed parts. Besides, a data re-use design is adopted to further reduce the complexity. As compared to the original SVC reference software, our encoder achieves about 9X speed-up without inducing serious quality degradation and compression ratio drop. With the increasing popularity of multi-processor systems and the maturity of thread libraries, parallel video encoding becomes a promising way to significantly speed-up the encoding process. However, conventional video encoders including the SVC reference software is single-threaded and can not take advantage of the multiprocessor systems. We propose a picture decomposition method based on the hierarchical B-picture structure of SVC. The proposed method outperforms traditional macroblock decomposition by about 10.1% in speed while producing exactly the same result. In addition, an efficient parallel algorithm for H.264/MPEG4-AVC deblocking filter is proposed. The content adaptive in-loop deblocking filter is complicated since every 4×4 block has to be examined by series of conditional checks. About one-third of the computation time is consumed in this module in an optimized H.264/MPEG4-AVC decoder. The high data dependency among filtering operations restricts efficient parallel algorithms which can fully utilize the multi-processor systems. The proposed parallel algorithm based on an essential observation named “limited error propagation effect” outperforms traditional wave-front method by about 40% in speed.口試委員會審定書 ........................................................................................................... i 致謝 ................................................................................................................................ iii 摘要 ................................................................................................................................ iv Abstract ........................................................................................................................... v Chapter 1 Introduction ................................................................................................................ 1 1.1 Background ............................................................................................................ 2 1.2 H.264/MPEG4-AVC Scalable Extension .............................................................. 2 1.3 Encoder Optimization ............................................................................................ 3 1.4 Parallel Video Encoding ........................................................................................ 4 1.5 Experimental Parameters ....................................................................................... 5 1.6 Thesis Organization ............................................................................................... 6 2 Overview of H.264/MPEG4-AVC Scalable Extension ............................................ 7 2.1 Overview of H.264/MPEG4-AVC Coding Tools .................................................. 7 2.1.1 Temporal Domain Prediction ....................................................................... 7 2.1.1.1 Multiple Reference Frames ............................................................. 9 2.1.1.2 Variable Block Size ......................................................................... 9 2.1.1.3 Quarter Pixel Motion Estimation................................................... 10 2.1.2 Spatial Domain Prediction ......................................................................... 10 2.1.3 Adaptive Deblocking Filter ........................................................................ 11 2.1.4 Network Abstraction Layer (NAL) ............................................................ 12 2.2 Overview of H.264/MPEG4-AVC Scalable Extension ....................................... 13 2.3 Spatial Scalability ................................................................................................ 14 2.3.1 Inter-Layer Intra prediction ........................................................................ 15 2.3.2 Inter-Layer Motion Prediction ................................................................... 15 2.3.3 Inter-Layer Residue Prediction .................................................................. 16 2.4 Temporal Scalability ............................................................................................ 16 2.5 Quality Scalability ............................................................................................... 18 3 Fast Motion Estimation ............................................................................................ 19 3.1 Temporal Scalability ............................................................................................ 19 3.1.1 Characteristics of hierarchical B-picture structure ..................................... 19 3.1.2 Conventional Fast Motion Estimation Algorithms on SVC ....................... 23 3.1.2.1 Diamond Search ............................................................................ 24 3.1.2.2 Three Step Search .......................................................................... 24 3.1.2.3 Built-In Algorithm TZ ................................................................... 25 3.1.2.4 Performance Comparison .............................................................. 26 3.1.3 The Proposed Hybrid Algorithm ................................................................ 29 3.1.4 Summary .................................................................................................... 34 3.2 Spatial Scalability ................................................................................................ 34 3.2.1 Inter-Layer MV Prediction ......................................................................... 35 3.2.2 Inter-Layer Residue Prediction .................................................................. 36 3.2.3 Experiments ................................................................................................ 37 3.3 Complexity Reduction ......................................................................................... 40 3.4 Summary .............................................................................................................. 44 4 Parallel Video Encoding ........................................................................................... 45 4.1 Background .......................................................................................................... 45 4.2 Data Decomposition ............................................................................................ 46 4.2.1 GOP Decomposition .................................................................................. 47 4.2.2 Slice Decomposition .................................................................................. 47 4.2.3 Picture Decomposition in H.264/MPEG4-AVC ........................................ 48 4.2.4 Macroblock Decomposition ....................................................................... 48 4.3 Proposed Picture Decomposition Based on Hierarchical B-Picture Structure .... 49 4.4 Implementation .................................................................................................... 51 4.5 Experiments ......................................................................................................... 52 5 An Efficient Parallel Algorithm for H.264/MPEG4-AVC Deblocking Filter ..... 55 5.1 Background .......................................................................................................... 55 5.2 Previous Work ..................................................................................................... 57 5.3 H.264/MPEG4-AVC Deblocking Algorithm ...................................................... 57 5.4 Data Dependency ................................................................................................. 59 5.5 Proposed Algorithm ............................................................................................. 60 5.5.1 The Limited Error Propagation Effect........................................................ 60 5.5.2 The Proposed Algorithm ............................................................................ 63 5.6 Experimental Results ........................................................................................... 64 5.7 Summary .............................................................................................................. 67 6 Conclusions and Future Work ................................................................................ 68 6.1 Conclusions .......................................................................................................... 68 6.2 Future Work ......................................................................................................... 69 Bibliography .................................................................................................................. 70 Vita ................................................................................................................................ 764668046 bytesapplication/pdfen-US視訊編碼先進視訊編碼可調式視訊編碼跨層估測快速移動估測平行演算法去區塊效應濾波器Video codingH.264/MPEG4-AVCScalable Video Coding (SVC)inter-layer predictionfast motion estimationparallel algorithmdeblocking filterthesishttp://ntur.lib.ntu.edu.tw/bitstream/246246/53768/1/ntu-96-R94922012-1.pdf