指導教授：傅立成臺灣大學：資訊工程學研究所朱庭升Chu, Ting-ShengTing-ShengChu2014-11-262018-07-052014-11-262018-07-052014http://ntur.lib.ntu.edu.tw//handle/246246/261471社交互動是維持人們社交關係的一種重要方式，而人們的社交關係對於人與人間的人際關係也有著直接的影響，並且對於人們的心理狀態以及生理狀態都扮演著一個重要的影響因素，尤其是對於年紀較大的年長者們。由於近年來機器人領域的快速發展，透過機器人的輔助來加強人們的社交互動已經是可以被社會大眾所期待的。因此我們希望藉由推薦合適的社交活動並且提供相對應的輔助來賦與機器人增進人們社交互動的能力。 以此方向為目標，在本碩士論文之中我們開發一創新的活動推薦系統能夠適用於此種社交輔助機器人，且透過自我社交網路的分析可推論出適當的社交活動並應用於多人的環境之下。 在此系統之中我們首先提出一新穎的概念，即為結合機器人的視角及第一人稱視角來建立自我社交網路。根據我們所進行的案例研究，本論文提出四類社交互動的特徵用來感知人與人之間的互動親密程度以做為社交網路中的資訊。接著我們以先前所建立的自我社交網路做為基礎，創建一社交活動推薦模型用以推薦合適之社交活動。最後，透過在本論文中針對於各個部分進行詳細的實驗，本系統被證明為有能力可以推論以及推薦適當的社交活動並且提供合適的服務來輔助人們的社交互動。Social interaction is an important means for maintaining our social relationship. It directly affects humans'' interpersonal relationship, acting as an important factors which influence humans'' mental status as well as physiological condition especially for elders. Owing to vast developments in the field of robotics in recent years, robotic assistance to enhance social interactions among humans is now a general expectation. For this reason, we hope to endow robots with an ability to help humans promote social interactions through recommending of appropriate social activities and providing of corresponding assistance. With this as our aim, in this thesis we develop an innovative activity recommendation system for such social assistive robot based on the ego social network analysis in multi-human environment. At first, a novel idea to combine the first-person camera and the robot camera to construct an ego social network is introduced. Four types of social interaction features for perceiving the intimacy level are proposed subsequently based on a user study we have conducted. Afterwards, a social activity recommendation model is presented in order to recommend appropriate activities cooperating with the former ego social network analysis. Finally, through the evaluation by several conducted experiments, we demonstrate that our system have the ability to reason and offer the pertinent assistance for humans'' social interactions.Contents 口試委員會審定書 i 誌謝 ii 摘要 iv Abstract v Contents vi List of Figures ix List of Tables xi 1 Introduction 1 1.1 Motivation 1 1.2 Objective and Contributions 3 1.3 Related Work 4 1.3.1 Perceiving Intimacy 5 1.3.2 Ego Social Network 6 1.3.3 Multi-human Social Activity Recommendation 7 1.4 System Overview 7 1.5 Thesis Organization 9 2 Preliminaries 10 2.1 Robot Perception System 10 2.1.1 Human Detection and Tracking 11 2.1.2 Face Direction Estimation 13 2.2 Support Vector Machine 15 2.2.1 Linear Support Vector Machine 15 2.2.2 Kernel Support Vector Machine 18 2.3 Conditional Random Field 20 2.3.1 Chain Structure Conditional Random Field 21 2.3.2 Network Structure Conditional Random Field 22 3 Ego Social Network Construction 24 3.1 User Study of Social Interaction 24 3.1.1 Proxemics Social Signal 25 3.1.2 Non-verbal Social Signal 25 3.1.3 Verbal Social Signal 26 3.1.4 Temporal Social Signal 27 3.2 First-Person Camera 27 3.3 Robots View and First-Person View Cooperation 28 3.3.1 Global Information 28 3.3.2 Local Information 29 3.3.3 Information Fusion 30 3.4 Social Interaction Feature 32 3.4.1 Proxemics Interaction Feature 32 3.4.2 Non-verbal Interaction Feature 33 3.4.3 Verbal Interaction Feature 36 3.4.4 Temporal Interaction Feature 37 3.5 Intimacy Perceiving 37 3.6 Ego Social Network Construction 38 4 Activity Recommendation Model 41 4.1 Multi-human Social Activity 41 4.2 Ego Social Network Analysis 42 4.3 Hidden Conditional Random Field 43 4.3.1 General Hidden Conditional Random Field 43 4.3.2 Activity Recommendation Model using Hidden Conditional Random Field 45 4.4 Learning and Inference Method 48 4.5 Activity Recommendation List 49 5 Evaluation 51 5.1 Reliability of Sensing Component 51 5.1.1 Body Direction 52 5.1.2 Face Direction 53 5.1.3 Social Flags 54 5.2 Perceiving Intimacy for Network Construction 55 5.2.1 Subjects and Environment 56 5.2.2 Comparison of Different Types of Social Interaction Features 58 5.2.3 Result of Selected Features 59 5.3 Evaluation of Activity Recommendation Model 60 5.3.1 Social Interaction Data Collection 60 5.3.2 Accuracy of Inference for Individual Task 62 5.3.3 Mean Average Precision of Recommendation List 63 5.4 Overall Scenario Testing 66 5.4.1 First Scenario 66 5.4.2 Second Scenario 67 5.4.3 Third Scenario 69 6 Conclusion and Future Works 71 6.1 Conclusion 71 6.2 Future Works 73 References 7424846395 bytesapplication/pdf論文公開時間：2017/08/21論文使用權限：同意有償授權(權利金給回饋學校)社交機器人社交輔助自我社交網路活動推薦第一人稱視角攝影機thesishttp://ntur.lib.ntu.edu.tw/bitstream/246246/261471/1/ntu-103-R01922048-1.pdf