Light-field Processing System for Depth-based Real-time Video Applications
Date Issued
2015
Date
2015
Author(s)
Chen, Hong-Hui
Abstract
This thesis discusses the signal processing of multi-view light field. First we introduce several depth-based image processing and application systems. Among them, the light-field based systems are the main topic of this thesis. Applying depth search algorithm to the input light field, the depth information is extracted for the following different applications, for example, image refocus, skeleton analysis of human body, and tracking of moving objects. With the depth information, many algorithms and applications can achieve better performance and accuracy. We modify a digital single-lens reflex (DSLR) camera and put a five by five lens array on it. After capturing the raw light field, we apply suitable calibration process on the light field. We implement software version of the adopted depth searching algorithm to prove that the acquired light field is applicable to generate an adequate depth map with satisfying quality. We have confirmed that the resulting depth map quality from our captured light field outperforms the depth map from stereo depth searching algorithm (by using only two cameras). Then we set up the goal to apply the algorithm on a system capable of dealing with 1080p high definition images at video rate. Plenty of the previous works focus their effort on accelerating the processing on single still image. Several works targeting the light-field processing at video rate deliver their specification only up to 720p/30fps when generating the depth map. This specification cannot fulfill the need for future application. Our chip implementation supports processing video data from five cameras up to 1080p resolution at 30fps. We have proved that the proposed architecture is practical under current CMOS fabrication technology. The chip is realized under constrained power and area budget. We adopt the techniques of hardware processing parallelism and made tradeoff between the depth map quality from the searching algorithm and the hardware complexity. By overcoming the difficulties on system memory bandwidth, computational load, and power scalability, we have converged to an effective chip design. With this result, we expect that the adopted depth searching algorithm can be used in future light-field processing occasions that require video rate capability. A broader adoption of the light-field processing in different future applications is foreseeable.
Subjects
multi-view
light field
lens array
camera array
refocus
depth search
chip
Type
thesis
File(s)![Thumbnail Image]()
Loading...
Name
ntu-104-D96943001-1.pdf
Size
23.32 KB
Format
Adobe PDF
Checksum
(MD5):ddf6a922f32cda431c78a07e382833bf