Vision Based Hand Gesture Recognition in Cluttered Backgrounds

A robust algorithm capable of segmenting specified hand gestures in cluttered image sequences is proposed. Typically, vision-based gesture recognition systems suffer from the difficulty of hand region segmentation, which includes the change of lighting conditions, the presence of other skin color objects and the movement of background objects. The main concern of this paper is to design an algorithm for RGB camera that locates hand region correctly even under complex backgrounds. The proposed segmentation algorithm, thumb-cover detection algorithm, restricts itself to support only three hand shapes, i.e. fist, pointing and victory. The thumb cover is the common part appearing in above mentioned hand shapes. It is an ideal target to detect for its distinctness from other background objects (especially facial organs). Once the thumb cover is detected, one can further apply other techniques on its neighboring region to recognize what gesture is posed. To train a detector of thumb cover, we collected 210 thumb-cover images from 10 people and 2521 random images as training samples. These samples, resized to 32 by 40 pixels, are combined to train an SVM with LBP feature applied. A virtual mouse human computer interaction (HCI) program basing on thumb-cover detection is also implemented. Users can manipulate the mouse cursor by moving his/her thumb cover and clicking his/her index finger in front of a camera. The αβγ filter is adopted to track and smooth the trajectory of thumb cover and optical flow is used to detect the clicking of finger. The HCI program runs at the speed of 25 frames per second (fps), which might be suitable for real time interaction.