DPDM: Feature-Based Pose Refinement with Deep Pose and Deep Match for Monocular Visual Odometry

In recent years, the metaverse has been a popular topic, and it drives many consumer electronics like AR/VR HMDs (Head Mounted Displays) and smart glasses. In these mobile devices, a critical technology is visual odometry (VO), which provides on-device motion tracking so that the user can interact with and move freely in the virtual information. In this work, we propose a novel hybrid monocular visual odometry framework named DPDM (Deep Pose and Deep Match), which properly integrates deep learning into geometry-based methods. We revisit the traditional feature-based optimization and improve it by replacing its crucial components with deep prediction. With the powerful high-level information extraction ability of deep neural networks, DPDM can obtain robust and accurate results through a simple frame-to-frame sparse feature-based pose refinement module. Experiments show that DPDM can outperform traditional VO and pure learning-based VO. Compared to state-of-the-art hybrid VO, DPDM can achieve competitive performance and higher FPS (Frames Per Second).