https://scholars.lib.ntu.edu.tw/handle/123456789/635603
標題: | Acquiring 360° Light Field by a Moving Dual-Fisheye Camera | 作者: | Lo, I. Chan HOMER H. CHEN |
關鍵字: | 360° cameras | 360° light field | convolutional neural network | depth estimation | light field generation | 公開日期: | 1-一月-2023 | 卷: | 32 | 來源出版物: | IEEE Transactions on Image Processing | 摘要: | In this paper, we propose an efficient deep learning pipeline for light field acquisition using a back-to-back dual-fisheye camera. The proposed pipeline generates a light field from a sequence of 360° raw images captured by the dual-fisheye camera. It has three main components: a convolutional network (CNN) that enforces a spatiotemporal consistency constraint on the subviews of the 360° light field, an equirectangular matching cost that aims at increasing the accuracy of disparity estimation, and a light field resampling subnet that produces the 360° light field based on the disparity information. Ablation tests are conducted to analyze the performance of the proposed pipeline using the HCI light field datasets with five objective assessment metrics (MSE, MAE, PSNR, SSIM, and GMSD). We also use real data obtained from a commercially available dual-fisheye camera to quantitatively and qualitatively test the effectiveness, robustness, and quality of the proposed pipeline. Our contributions include: 1) a novel spatiotemporal consistency loss that enforces the subviews of the 360° light field to be consistent, 2) an equirectangular matching cost that combats severe projection distortion of fisheye images, and 3) a light field resampling subnet that retains the geometric structure of spherical subviews while enhancing the angular resolution of the light field. |
URI: | https://scholars.lib.ntu.edu.tw/handle/123456789/635603 | ISSN: | 10577149 | DOI: | 10.1109/TIP.2023.3299444 |
顯示於: | 電機工程學系 |
在 IR 系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。