Extending Structured-light RGB-D Cameras in Outdoor Scenes via Cross-spectral Stereo

Structured-light based 3D cameras such as Microsoft''s Kinect or Asus''s Xtion are popular low-cost RGB-D sensors recent years. However, most of these sensors are assumed to be used in indoor environments with moderate ambient light. Once these devices are taken to outdoor scenes, the bright sunlight makes the projected pattern obscure to be seen and causes the dramatic reduction of working range. While IR pattern disappears in sunlight, the background becomes bright and clear in IR image. This brings the opportunity to use stereo algorithms on RGB and IR image to recover the unmeasured depth. In this work, we investigate the possibility of recovering the unmeasured depth information of the structured light device via stereo matching in outdoor scenes. Densely matching RGB and IR images is a challenging task since they represent the information in two almost non-overlapped spectrums. Different from other edge-based cross spectral stereo approaches, we analyze the camera imaging model and found the hidden relation between the RGB and IR spectrum in material level. Based on this relation, we propose a material-based color conversion method to make the cross-spectral problem become a general stereo problem. In addition, we also introduce a way to utilize depth information in the stereo disparity optimization stage. To evaluate our method, an outdoor dataset is collected via Xtion. The experiment results show that the proposed method works well the estimating of depth for the regions Xtion failed to compute.