A Depth Estimation Algorithm and Hardware Design for Light Field Cameras under Low Light Conditions

In computational photography, light field is an approach to describe light rays. In this research, we place a pin-hole array mask in front of the camera sensor to obtain positions and directions of lights. With the four-dimensional light field data, we can apply depth estimation technique to get depth information which can’t be obtained from traditional digital cameras.
However, it’s difficult to avoid noise impacts when taking photos, especially at night. To increase brightness, people usually increase the ISO and exposure time. Both approaches will induce more noise. Nowadays, there are two kinds of methods to reduce the influences of noise on depth maps. One is to apply denoising techniques to the original image before doing depth estimation. The other one is implementing denoising algorithms directly to depth maps. But there are problems with these methods. For the former method, we don’t know which denoising method is appropriate because we don’t have the specific information of the noise. For the latter method, if there are too many computational errors in the original depth map, it’s difficult for us to correct these errors. In this thesis, we propose an algorithm that combines denoising and depth estimation, which means we conduct denoising and depth estimation procedures at the same time. Therefore, we can solve the problems mentioned above and automatically generate depth maps with less noise impacts.
Depth estimation is a time-consuming algorithm. Since we also need to implement a denoising algorithm, the whole complexity is really high. In order to meet the real-time requirement, we propose a hardware processor to improve the computation speed. The chip and core sizes are 1.294 mm2 and 0.486 mm2 respectively. The power consumption is 54.81 mW when running at 110 MHz.