Algorithm and Implementation of Vision-Based 3D Mouse System

The fast growths of 3D digital content and 3D display device are driving the development of 3D industry; since traditional 2D interface will gradually be difficult to meet consumer demand, the markets for 3D applications are promising. For example, the Vista which applies 3D-style window and the Wii which provides semi-3D motion sensing function have been successfully accepted by today's consumers. In addition to enjoying the 3D content on the 3D display, users may want to perform 3D interactions with the entertaining systems. Assume in the near future, the 3D computers will enter our life and drive the demand for 3D mouse, which users use to interact with computer system or other applications, providing interaction with one more dimension than the conventional 2D mouse.
Existing 3D mouse devices could be broadly devided into the following categories: hand-held type, desktop type and mechanical-arm type. There are some problems in these existing solutions, such as inconvenience to use and limited active sensing range.
In our proposed 3D mouse system, the most important concept is to provide our 3D mouse an unlimited active sensing range. The important features would allow user to perform interaction with host system at any places and toward any directions, enhancing the operation conveniences and allow users to perform interaction with unlimited large movement, which can not be done by a 3D mouse with limited sensing range, such as the Wii remote.
In order to achieve the unlimited active sensing range, we embedded the sensing device, the webcam, in our mouse, and use the captured video frames as clues to detect the translational displacement of the mouse.
Camera motion estimation has been studied for decades. Some prior art use the correspondences between image features to recover the camera motion while the other class use the optical flow to solve the problems. Both of them encountered the problems of global feature matching, which is usually hard to achieve high accuracy rate. In our proposed algorithm, we first perform the global motion estimation and compensation between two captured frames and the perform the local feature matching on the mismatching regions to detect the epipolar lines and then derive the epipole, which could be used to derive the translational motion parameters. Since only local feature matching is required, our proposed algorithm could achieve a very high accuracy rate (98%~99%) when there is a dominating plane (for GME) and few foreground objects(for feature matching) existing. The future work would include moving objects removing, 3D rotational parameters calculation and virtual plane generation (when there is no dominating plane existing).