Multi-Sensor Based Hand Gesture Recognition for Human-Robot Interaction and the application of Intelligent Service Robotics

With advances in technology, robots play an important role in our lives. Nowadays, we have more chance to see robots service in our society such as intelligent robot for rescue and for service. Therefore, Human-Robot interaction becomes an essential issue for research. In this thesis, we propose a direct method for Human-Robot interaction on service robot. Thesis can be broadly divided into two parts; in the first part, we introduce a combining method for hand gesture recognition which is using computer vision and image processing to achieve the goal and second the part, we show several methods for Human-Robot interaction.
In the first part, we focus on our hand gesture recognition method. Hand sign recognition is an essential and fast way for Human-Robot Interaction (HRI). Sign language is the most intuitive and direct way to communication for impaired or disabled people. Through the hand or body gestures, the disabled can efficiently let caregiver or robot know what message and order they want to convey. In this thesis, we propose a combinatorial hands gesture recognition algorithm which combines two distinct recognizers. These two recognizers collectively determine the hand’s gesture via a process called Combinatorial Approach Recognizer (CAR) equation. These two recognizers are aimed to complement the ability of discrimination. To achieve this goal, one recognizer recognizes hand gesture by hand skeleton recognizer (HSR), and the other recognizer is based on Support Vector Machines (SVM). In addition, the corresponding classifiers of SVM are trained using variety features such as Gabor feature, local binary pattern and raw data. Furthermore, the trained images are using Bosphorus Hand Database [15][20][43] , in addition to the images taken by us. A set of rules including recognizer switching and combinatorial approach recognizer CAR equation is devised to synthesize the distinctive methods. We have successfully demonstrated gesture recognition experimentally with successful proof of concept. Our experiment can be use in self-developed intelligent robots and to achieve Human-Robot interaction.
In the second, we introduce several ways of Human-Robot interactions and demonstrate the application of intelligent service robots, such as hand gesture recognition which is proposed in first part implements on service robot, RGB-D sensor application and multi-sensor based person identification. We associate different kind of sensor to achieve innovative human-computer interaction methods.