指導教授：江茂雄臺灣大學：工程科學及海洋工程學研究所龔聖翔KUNG, SHENG-HSIANGSHENG-HSIANGKUNG2014-11-252018-06-282014-11-252018-06-282014http://ntur.lib.ntu.edu.tw//handle/246246/260929本文旨在以SURF與HRI影像演算法整合三軸角錐形氣壓並聯式機構機械臂，使機械臂可藉由影像辨識物件形狀及位置，自動完成取放作業之工作。本文採用加速強健特徵點演算法 (Speed up robust feature, SURF)來定義目標物體的特徵點，並且藉由比對當前畫面中的特徵點來判斷目標物是否存在於影像中。為了強化特徵比對結果並算出抓物控制所需的參考點，本論文採用隨機取樣篩選演算法 (RANdom Sample Consensus, RANSAC)來估測平面轉換矩陣 (Homography matrix)以準確的標出目標物的中心點。並利用最新發展出的遊戲設備，ASUS Xtion Pro Live深度攝影機能夠直接擷取目標點3-D空間座標。此外，本文並發展出一套座標估測的校正法，提高對於目標物座標估測之準確度。 人機互動是提供非受訓練的使用者能夠更容易、更有效率與機器互動的一種方法。本研究提出一套利用手勢辨識來操控機械臂的方法，使用網路攝影機取得影像後經過多個處理程序，達到辨識之功能。其中的影像處理包含膚色偵測、雜訊消除和形態學以計算手指數量，並將資訊傳給控制器，利用以上所提出的理論規劃機械臂端點之軌跡。最後，經過實驗驗證本文提出之方法的可行性，證實此系統可成功導引三軸式機械臂之端點順利的移動並拿取所設定的目標物。The objective of this study is to develop the SURF and HRI image algorithm integrated with a three-axial pneumatic parallel manipulator. The manipulator can pick and place objects automatically by the feature information of the image through the SURF algorithm with scale- and rotation-invariants. The speed up robust feature (SURF) algorithm is used to define the feature of a target object and to match features between the current image and the object database for confirming the target. To strengthen the feature matching results and calculate the necessary reference control point, we adopt the RANSAC(RANdom Sample And Consensus) algorithm to estimate the planar transformation matrix (homography matrix) in order to accurately mark the center of target. The ASUS Xtion Pro Live depth camera which can directly estimate the 3-D location of target point is used in this study. A set of coordinate estimation calibration method is developed to improve the accuracy of target location estimation. Human-Robot-Interaction (HRI) offers a way to enable untrained users to interact with robots more easily and efficiently. This study also presents a method for hand gesture recognition to command the manipulator. The stages include skin detection to effectively capture only the skin region of the hand, noise elimination, and applications of the morphology to determine the active finger count. Once the finger count is determined, the information is transmitted to the manipulator controller. The end-effector of the manipulator can then move to the desired location according to the finger count. Finally, the experiments of the three-axial manipulator end-effector integrated with the feature recognition algorithm demonstrate that the proposed methods can achieve the feature recognition and pick and place of the target object successfully.誌謝…………………………………………. I 中文摘要………………………………… II ABSTRACT…………………………….. III LIST OF FIGURE……………………….. VII LIST OF TABLES………………………… XI Chapter 1 Introduction…………………. 1 1.1 Background 1 1.2 Literature Review 4 1.3 Motivation 7 1.4 Thesis Outline 9 Chapter 2 System Overview…………… 10 2.1 Mechanism Description 10 2.2 Test Rig Layout 14 2.2.1 Overall Manipulator System 14 2.2.2 Camera System 16 Chapter 3 Object Recognition……….. 19 3.1 Interest Point Detection 20 3.1.1 Integral Image 20 3.1.2 Hessian Matrix Based Interest Points 22 3.1.3 Scale Space Representation 25 3.1.4 Interest Point Localization 29 3.2 Interest Point Description 32 3.2.1 Orientation Assignment 33 3.3.2 Sum of Haar Wavelet Responses Descriptor 34 3.3 Feature Points Matching 38 3.3.1 Image Transformation 39 3.3.2 Exact Solution 40 3.3.3 Over-determined Solution 42 3.3.4 Normalized DLT 45 3.3.5 Robust Estimation using RANSAC 47 Chapter 4 Gesture Recognition……….. 51 4.1 Skin Color Classification 52 4.2 Noise Rejection 56 4.3 Distance Transform 57 4.4 Morphology 60 4.4.1 Structural Element 60 4.4.2 Erosion 61 4.4.3 Dilation 63 Chapter 5 3D Object Localization……. 67 5.1 Camera Model 67 5.2 Calibration of Depth Camera 71 5.3 Obtain 3D Location via Depth Camera 75 5.4 Hand-eye Coordinates Calibration 77 Chapter 6 Analysis of kinematics……… 81 6.1 Geometry of the Manipulator 81 6.2 Inverse Kinematic Analysis 84 6.3 Forward Kinematic Analysis 85 Chapter 7 Experiment…………………. 86 7.1 SURF Feature Detection 89 7.2 Feature Matching 92 7.3 Finger Count Appointment 97 7.4 Path Planning 100 7.5 Pick and Place Experiment 105 Chapter 8 Conclusions………………….. 111 REFERENCE…………………………… 1135112994 bytesapplication/pdf論文公開時間：2017/08/17論文使用權限：同意有償授權(權利金給回饋本人)影像辨識並聯式機構機械臂加速強健特徵點演算法隨機取樣篩選演算法手勢辨識thesishttp://ntur.lib.ntu.edu.tw/bitstream/246246/260929/1/ntu-103-R01525023-1.pdf