Fast Algorithm and Hardware Architecture for Stereo Image Synthesis Systems
Date Issued
2006
Date
2006
Author(s)
Chen, Wan-Yu
DOI
en-US
Abstract
Stereo images can make users sense depth perception by showing two images to each
eye simultaneously. It gives users a vivid information about the scene structure and can
be used for 3D-TV, telepresence, and immersive communication. With the technologies
of 3D video capture device and 3D-TV display device getting more and more mature,
the importance of the stereo content will rise in the near future. Under this trend, stereo
image synthesis draws more and more attention. Stereo image synthesis is the essential
part in 3D image system. It is also the main part which is focused on in this thesis.
However, to build up a stereo image synthesis system, many design challenges, such as
bad synthesis quality, high computational complexity, and hardware architecture implementation,
etc., must be overcome.
In this thesis,a novel stereo image synthesis system from single lens auto focus camera
is proposed to overcome the overheads of traditional two lens stereo camera. First
we proposed a novel object based depth estimation algorithm to estimate depth map
from multi-focus images. Besides, we adopted depth-image-based-rendering with edge
dependent gaussian filter and interpolation to render left and right image with high quality.
Finally, we proposed a new depth background mode to enrich the background depth.
The proposed single camera stereo image synthesis system with efficient object based
depth from focus and depth image based rendering core design is proposed and discussed
in algorithm level, architecture level, and system level. In algorithm level, first
we focus our attention on the first stage: Object-Based-Depth-From-Focus(OBDFF).
We proposed a new object based DFF to increase the noise resistibility and discriminability
of depth map with multi-objects. Besides, the proposed algorithm only require
less than ten multi-focus images to accomplish stereo image generation. The simulation
results show the proposed object based depth from focus method can reach more
than 95% match rate compared with ground truth depth map. Furthermore, the novel
color segmentation based depth interpolation reduced the run-time of depth interpolation
in depth from focus by 99.6%. Then we focus our attention on the second stage:
Efficient Depth Image Based Rendering(DIBR) with Edge Dependent Gaussian Filter
and Interpolation The proposed DIBR algorithm outperforms previous work by 6–7
dB in PSNR performance. In addition to that, the number of instruction cycles is 3.7
percent compared with the previous work. Besides, in architecture level, a hardwired
DIBR core architecture design is proposed to provide a cost-effective solution to DIBR
core implementation in not only stereo image synthesis system but also advanced threedimensional
television system. We propose a novel depth image based rendering algorithm
with edge-dependent gaussian filter and interpolation to improve the rendered
stereo image quality. Based on our proposed algorithm, a fully-pipelined depth image
based rendering hardware accelerator is proposed to support real-time rendering. The
proposed hardware accelerator is optimized in three steps. First, we analyze the effect
of fixed point operation and choose the optimal wordlength to keep the stereo image
quality. Second, a three-parallel edge-dependent gaussian filter architecture is proposed
to solve the critical problem of memory bandwidth. Finally, we optimize the hardware
cost by the proposed hardware architecture. Only 1/21 amounts of vertical PEs and
1/11 amounts of horizontal PEs is needed by the proposed folded edge-dependent gaussian
filter architecture. Futhermore, by the proposed check mode, the whole Z-buffer
can be eliminated during 3D image warping. In additions, the on-chip SRAMs can be
reduced to 66.7 percent compared with direct implementation by global and local disparity
separation scheme. A prototype chip can achieve real-time requirement under the
operating frequency of 80 MHz for 25 SDTV frames per second (fps) in left and right
channel simultaneously. The simulation result also shows the hardware cost is quite
small compared with the conventional rendering architecture.
Moreover, in system integration level, a prototype of real-time stereo image synthesis
system is successfully built up. The system integrates a single camera, FPGA
accelerator, software optimization, and autostereoscopic 3D display LCD. The demonstration
results proves proposed algorithm and architecture of the DIBR core indeed
improve the performance in the stereo image synthesis system.
Subjects
三維電視
立體影像合成
立體影像
3D TV
Stereo Image Synthesis
Stereo Image
Type
thesis
File(s)![Thumbnail Image]()
Loading...
Name
ntu-95-R93943044-1.pdf
Size
23.31 KB
Format
Adobe PDF
Checksum
(MD5):855d0be1dc0d6fddaacaf156ee11fadb