莊永裕臺灣大學:資訊工程學研究所鄭逸廷Cheng, Yi-TingYi-TingCheng2010-06-092018-07-052010-06-092018-07-052009U0001-2707200914203300http://ntur.lib.ntu.edu.tw//handle/246246/185408本論文提出了一個以三維形變模型(3D morphable model)為基礎視訊中人臉置換(face replacement)系統。此系統將來源人物(sourceubject)在與目標人物(target subject)相似之角度、表情以及光照條件下換入目標影片中。視訊中人臉置換系統在特效產業以及娛樂產業中為有用，過去雖然有許多影像中人臉置換演算法被提出， 也有許多有軟體存在，然而應用到影片中仍然需要耗費大量的人力和時間。論文提出之人臉置換系統可大幅減少所需人力介入的需求，並且將源人物所需擷取資料量減少至只需一至兩張照片。此系統共有八個要模組，包括：人臉校準、方位估計、人臉擷取、三維人臉模型生、表情估計及生成、打光估計及人臉重打光、人臉影像生成、視訊影像合成。在這篇論文中，我們主要探討的主題是三維人臉模型生、表情估計及生成以及視訊中影像合成技術。為證實我們演算法的效性，我們將對電影中擷取之影像以及我們實際拍攝的影像進行測， 並且將結果完整呈現在論文之中。This thesis presents a system that replaces the target subject face in thearget video with the source subject face, under similar pose, expression, andllumination. Face replacement in video is a useful application in the entertainmentnd special effect industries. However, the frame-by-frame manipulationrocess using the software is often time-consuming and labor-intensive.ur approach is based on 3D morphable model and a expression database,nd the input information of the source subject face is reduced to one to twomages, while the face replacement procedure requires little user interference.ur system consists of several modules: face alignment, pose estimation, expressionstimation, face extraction, morphable model fitting, face synthesis,ace relighting, and composition. We mainly focus on morphable model fitting,xpression estimation, and composition in this thesis. To demonstratehe effectiveness of our system, we test our implementation on both movielips and video we captured.致謝i文摘要iiibstract v Introduction 1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Thesis Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Related Work 5.1 Image-Based Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Expression Manipulation . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3 Face Replacement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4 Seamless Image Composition . . . . . . . . . . . . . . . . . . . . . . . . 8 System Overview 9 3D Face Reconstruction 13.1 Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2 Morphable 3D Face Model . . . . . . . . . . . . . . . . . . . . . . . . . 17.3 Matching A Morphable Model to Images . . . . . . . . . . . . . . . . . 18.3.1 Image Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . 20.3.2 Fitting the Model to Images . . . . . . . . . . . . . . . . . . . . 20.4 Model Reconstruction Result . . . . . . . . . . . . . . . . . . . . . . . . 25 Matching Expression and Composition 29.1 Expression Cloning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29.1.1 3D Model Expression Database . . . . . . . . . . . . . . . . . . 30.1.2 Matching Correspondence . . . . . . . . . . . . . . . . . . . . . 31.1.3 Cloning Expression . . . . . . . . . . . . . . . . . . . . . . . . . 32.2 Matching Expressions to Target Video . . . . . . . . . . . . . . . . . . . 34.2.1 Matching the Mouth . . . . . . . . . . . . . . . . . . . . . . . . 36.2.2 Matching the Blinking . . . . . . . . . . . . . . . . . . . . . . . 39.3 Composition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.3.1 Poisson Image Editing . . . . . . . . . . . . . . . . . . . . . . . 42.3.2 Face Concealment . . . . . . . . . . . . . . . . . . . . . . . . . 43.3.3 Blending . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 Experiments and Results 45.1 Face Replacement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 Conclusion and Discussion 51ibliography 5328203241 bytesapplication/pdfen-US人臉置換計算攝影學Face ReplacementComputational Photographythesishttp://ntur.lib.ntu.edu.tw/bitstream/246246/185408/1/ntu-98-R96922015-1.pdf