Image Inpainting Based Algorithm and Architecture Design for Error Concealment

Error concealment(EC) technique can improve subjective video quality in decoder when the video bitstream is corrupted during transmission. In Chapter 2, a new error concealment algorithm for I-frames is proposed. To achieve perceptually comfortable results, image inpainting algorithm is adopted with structure information generated from edge information. In addition, mixed spatial-temporal inpainting scheme is also proposed for I-frames when the previous frame is available. Moreover, a block-based scheduling is also developed for the proposed algorithm to make it simpler for hardware implementation. Experimental results show that the proposed algorithm outperforms the official H.264 decoder and can deal with various error patterns. It also shows that it is suitable for hardware implementation in terms of on-chip memory requirement and predictable processing cycles. The reduction of on-chip memory is 90%, and the iteration number of each MB is limited to 25 at most. n Chapter 3, for low hardware cost design 1920x1080 real-time error concealment engine, the complexity of proposed algorithm is further reduced. The basic element becomes 4x4 block instead of pixel in spatial error concealment. And the BMA algorithm with four MV candidates and small search range are adopted in temporal error concealment. Scene change detection engine is added to select the concealed mode. With this algorithm, the reduction of on-chip memory is 95% comparing to original algorithm, and the iteration number of each MB is limited to 4 at most. he integrated system of video decoder and proposed EC engine is introduced in Chapter 4. Some engines in proposed EC algorithm share the hardware of video decoder, or add to the pipelining architecture of video decoder in [17]. In the end of chapter, hardware/software partition is considered. The low complexity functions which require flexibility are processed by ARM processor. On the other hand, the fixed or complex functions will be implemented as the dedicated hardware. Based on this principle, the integrated system is proposed. In Chapter 5, we analysis the work of spatial image inpainting and temporal error concealment. The new MB pipelining architecture is proposed to balance the required cycles of two concealed mode. Then the hardware architecture is proposed to support real-time decoding based on proposed MB pipelining architecture. This hardware engine is built on AHB-AMBA environment. Its characteristics are, two concealed mode share the same computational core to reduce hardware cost. And the cache based memory architecture is adopted to fulfill data reuse strategies between two processing MBs. The statistics of hardware implement can be seen in Chapter 6. A prototype chip is implemented using UMC 90NM Logic & Mixed-Mode 1P9M Process Low-K technology. The total gate count is about 121K synthesized at 200 MHz. The maximum processing capability is 244.8K macroblocks per second or 1920?1080 4:2:0 30Hz video. Totally about 5.6 Kbytes on-chip memory is required. The core size is 1.3?1.279 mm2. The average power dissipation is 131.4 mW when operating at 200 MHz with 1.1 V power supply. Compared to other error concealment works, the proposed design can achieve better perceptual quality. Therefore it is a good choice to be integrated into the video decoding applications which transmission error occurs frequently and require good quality. Furthermore, system level simulation environment is built in SoC designer 7.0. It proves that our hardware IP can run on kind of system successfully can co-work with other IP without violation. In addition, required bandwidth can be support by 32 bit AHB-AMBA bus. In the end of this paper, conclusion and future work are given.