Architecture Design of High Frame Rate Optical Flow Engine

Computer vision has been developed for decades, and has totally changed our lives. Thanks to the progress of technologies, we have entered the era of big data and smart devices. There are lots of new technologies, like 3D printers, wearable devices, light field refocus cameras and so on, been invented in recent years. We introduce the technology trends for past five years in Chapter. 1, we think the ""Action Prediction"" might be one of computing cores in many applications. Like home care robots, automatic cars, auto surveillance system... The action prediction is irreplaceable that can help robots assist people avoid accident happen. Predicting human action can dramatically reduce the accident rate, like the self-driving cars can promptly stop when detecting human is going to fall in the street. This is the motivation of our thesis, we hope our work can help the current technology move further. Then we survey the researches about action prediction and also pro filing the full system in Chapter. 1. We find that both of accuracy and computing speed are the critical parts to bring this technology into world. However, the full action prediction system may take more 2 minutes for processing. The most time consuming stage is the ""optical flow computing"", it takes about at least 1 minute for an VGA image, more than fifty percent of full system. The calculation speed is far from early determining the action as action prediction. So our work is providing a ""High Frame Rate Optical Flow Engine Chip"" to accelerate this basic but complicate computation. We introduce the difficulties of optical flow and related works of both of algorithms and architectures. The specification is explored in Chapter. 4, and it is the highest one comparing with other works in recent years. In Chapter. 3, we show the core idea that modifies the original optical flow algorithm to hardware friendly without losing accuracy. We test lots of conditions to ensure the accuracy of modified one is kept as original one. In short, we take the simple filter in the most complicate stage, but use complex filter in other stage to supplement the results. We show the ideas and the details of mapping algorithm to architecture step by step in Chapter. 4, then we optimize the architecture by using the ideas of computing reuse, weight quantization, pipeline structure and bit truncation. The computation reuse is the most influential optimized strategy of all, it reduces the area to 15 percent of original. It is based on the algorithm modification shown in Chapter. 3. The final result is also shown in the thesis; the goal of optimization is to build an area efficient chip for this highly parallel architecture. To sum up, an area efficient high frame rate optical flow engine is designed. It can be used in lots of wearable devices which is low power and low cost requirement. It is also a critical and necessary core for achieving action prediction.