傅立成臺灣大學：資訊工程學研究所黃世勳Huang, Shih-ShinhShih-ShinhHuang2007-11-262018-07-052007-11-262018-07-052007http://ntur.lib.ntu.edu.tw//handle/246246/53646人體動態分析在人機介面互動研究應用中，例如:虛擬實境、智慧型監控以及智慧型使用者介面，扮演著重要且不可或缺的角色，目前、以電腦視覺為基礎之人體動態分析在學術界上也引起許多廣泛的討論。一般而言，其主要包含四個部分:前景切割、人行偵測、姿態估測以及姿態追蹤，於本論文中，主要針對前景偵測以及姿態估測這兩個主題，分析當前文獻並分別提出一套有效的解決方法。 於前景切割部分，我們提出一個基於機率的方法，能將前景部份自動從影片中分割出來。為節省計算複雜度以及克服雜訊的干擾，我們提出一個以邊緣為基礎之變化偵測演算法去識別影像中可能為前景的區域。接著透過貝式網路，我們整合運動資訊將所偵測區域進一步分類為前景或背景，以有效過濾陰影效應、雜訊以及未遮蓋背影。而前景切割的問題可描述為:給定連續兩張影像以及前一個時間所獲得之前景切割結果，透過幾何運動限制以及背景觀察模型，我們可以定義運動位移場以及前景切割結果之共同條件機率。利用最佳化演算法，我們可以同時找出運動位移場以及前景切割的解。 人體姿態能提供有效之資訊作為感測與分析人類行為之重要依據。因此、我們提出一個機率架構去估測影像中人體的姿態。在此研究中，我們用以衡量觀測的主要線索為人形影像輪廓。首先、透過前景輪廓與所預估人體模型的差異，定義出所謂可能機率；透過引入身體各部位之幾何限制，定義出事前機率，最後利用RJMCMC近似演算法，在姿態參數空間中，找尋出可能的解。為了提升收斂速度，我們利用資料驅動的策略設計出有效之提案函數。Human dynamics analysis is currently one of the most active researches in computer vision because it is an important and fundamental component in many applications in the areas of human-computer interaction, such as virtual reality, smart surveillance, and intelligent user interface, etc. In the thesis, two issues which we take into considerations for human dynamics analysis are foreground segmentation and pose estimation. We present a probabilistic approach for automatically segmenting foreground objects from a video sequence. In order to perform foreground segmentation in a more semantic region level, we propose an edge-based change detection algorithm to automatically identify the regions with potential appearance variation due to the motion of objects. Then, we incorporate the motion information to perform foreground segmentation under a Bayesian framework. Given two consecutive images, the joint probability density function of the motion vector field and foreground segmentation mask is defined based on the constraints including observation likelihood and spatiotemporal constraint and thus is maximized to simultaneously achieve the foreground segmentation and the motion estimation in a mutually beneficial manner. Human pose is a natural way for a computer system to understand the intention of humans. Here, we want to propose a new statistical framework for estimating human pose by use of a reversible jump Markov Chain Monte Carlo (RJMCMC) approach, which tries to recovering the human body configuration based on its silhouette. Such problem is formulated as that of computing the maximum a posterior (MAP) of the probability density function of pose configuration given currently observed image. Equivalently, pose inference can be considered as traversing over the difference subspaces. Using of the data-driven mechanism, the mentioned reversible jump Markov chain Monte Carlo (RJMCMC) can explore such solution space much more efficiently.Contents iv List of Figures vi List of Tables ix 1 Introduction 1 1.1 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.1.1 Properties of Vision-Based Systems . . . . . . . . . . . . . . . . 2 1.1.2 Main Applications . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.2 Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.2.1 Environment Variation . . . . . . . . . . . . . . . . . . . . . . . 6 1.2.2 Appearance Variation . . . . . . . . . . . . . . . . . . . . . . . . 8 1.3 System Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.4 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.4.1 Algorithms for Foreground Segmentation . . . . . . . . . . . . . 11 1.4.2 Framework for Human Pose Estimation . . . . . . . . . . . . . . 12 1.5 Thesis Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2 State of the Art 15 2.1 Foreground Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.1.1 Background Subtraction . . . . . . . . . . . . . . . . . . . . . . 19 2.1.2 Motion-Based Segmentation . . . . . . . . . . . . . . . . . . . . 22 2.2 Pose Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 2.2.1 Component-Based Approach . . . . . . . . . . . . . . . . . . . . 29 2.2.2 Template-Based Approach . . . . . . . . . . . . . . . . . . . . . 30 2.2.3 Parameterization-Based Approach . . . . . . . . . . . . . . . . . 31 3 Region-Level Foreground Segmentation Based on Graphical Models 33 3.1 Application Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 3.2 Changed Region Detection . . . . . . . . . . . . . . . . . . . . . . . . . 36 3.2.1 CDM Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 3.2.2 Region Generation . . . . . . . . . . . . . . . . . . . . . . . . . 42 3.3 Motion Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 3.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 3.3.2 Non-parametric Approaches . . . . . . . . . . . . . . . . . . . . 45 3.3.3 Parametric Approaches . . . . . . . . . . . . . . . . . . . . . . . 47 3.4 Bayesian Foreground Segmentation . . . . . . . . . . . . . . . . . . . . 48 3.4.1 Bayesian Network Introduction . . . . . . . . . . . . . . . . . . 48 3.4.2 Bayesian Network Formulation . . . . . . . . . . . . . . . . . . 51 3.4.3 MAP Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . 53 3.5 Probability Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 3.5.1 Likelihood Model . . . . . . . . . . . . . . . . . . . . . . . . . . 54 3.5.2 Temporal Constraint . . . . . . . . . . . . . . . . . . . . . . . . 58 3.5.3 Spatial Constraint . . . . . . . . . . . . . . . . . . . . . . . . . 60 3.6 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 3.7 Experiment and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 64 3.7.1 Subjective Evaluation . . . . . . . . . . . . . . . . . . . . . . . 64 3.7.2 Objective Evaluation . . . . . . . . . . . . . . . . . . . . . . . . 70 4 Silhouette-Based Pose Estimation Using Reversible-Jump MCMC 74 4.1 Estimation Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 4.2 Pictorial Structure Human Model . . . . . . . . . . . . . . . . . . . . . 76 4.2.1 Part Description . . . . . . . . . . . . . . . . . . . . . . . . . . 77 4.2.2 Prior Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . 78 4.2.3 Likelihood Distribution . . . . . . . . . . . . . . . . . . . . . . . 79 4.3 Inference Using Reversible Jump Markov Chain Monte Carlo . . . . . . 80 4.3.1 Structure of Solution Space . . . . . . . . . . . . . . . . . . . . 81 4.3.2 Solution Exploration . . . . . . . . . . . . . . . . . . . . . . . . 82 4.4 Data-Driven Proposal Maps Generation . . . . . . . . . . . . . . . . . . 89 4.4.1 Body Part Extraction . . . . . . . . . . . . . . . . . . . . . . . 90 4.4.2 Face Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 4.5 Experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 5 Conclusion 102 6 Future Works 104 Bibliography 1054164954 bytesapplication/pdfen-US人體姿態分析前景切割貝式網路姿態估測資料驅動RJMCMC近似演算法Foreground SegmentationPose EstimationBayesian NetworkReversible Jump Markov Chain Monte CarloData-Driven Strategythesishttp://ntur.lib.ntu.edu.tw/bitstream/246246/53646/1/ntu-96-D89922013-1.pdf