Robot Facial Expressions with Lip Synchronization and Visual Attention for Human-Robot Interactions
Date Issued
2011
Date
2011
Author(s)
Chang, Shu-Ruei
Abstract
The growing process of the elderly population produces the demands of social welfare, medical care, education and various services. The topic of the auxiliary by the intelligent robots is the important subject of the world. In 21st century, the field of intelligent robotics is a high-priority development industry.
The main topic of this thesis is the human-robot interaction through robot facial expression with lip synchronization and visual attention. In the field of humanoid robotics the most important thing is to simulate human behavior and appearance. The robot face we design is based on the 27-year-old Albert Einstein. The great physical characteristics of the artificial skin “Frubber” contribute to mimic facial expressions. There are totally 30 degrees of freedom in the head, and the configuration is based on the anatomy. Therefore, the robot face can simulate so many action units of human and performs facial expressions such as happiness, surprised, fear, disgust, anger, hurt and so on.
In the robot speech system, 16 lip shapes are proposed to match the visemes. The real-time motor control synchronizes the lip shapes and synthesis speech. On the other hand speech recognition lets the robot to know what the interactor said, therefore it achieves the natural and simple way to have an interaction. We use Microsoft SAPI (Speech Application programming Interface) to build the speech system.
There are two cameras in the robot’s eyes with three degrees of freedom to control the gaze action. The vision system combines with face detection and pointing gesture recognition. We use the Adaboost as the algorithm of face detection, and combines robot neck control to achieve face tracking. The motion of human hands can provide abundant information about human intention and implicit meaning to the robots. Therefore through the pointing gesture recognition system the robot can receive more information and gives some responses.
All the systems, user interface, software and applications proposed in this thesis are implemented with C++ and C# programming language and Microsoft SAPI. The whole system and experiments are conducted on the “Einstein”, which is a humanoid robot developed by the Intelligent Robotics and Automation (IRA) Laboratory at National Taiwan University.
Subjects
humanoid robot
human-robot interaction
lip synchronization
facial expression
Type
thesis
File(s)![Thumbnail Image]()
Loading...
Name
ntu-100-R98522833-1.pdf
Size
23.54 KB
Format
Adobe PDF
Checksum
(MD5):f6d0d34311ae04d60b1cd883d70118bf