https://scholars.lib.ntu.edu.tw/handle/123456789/633077
標題: | Accelerating Video Captioning on Heterogeneous System Architectures | 作者: | Huang, Horng Ruey Hong, Ding Yong Wu, Jan Jan Chen, Kung Fu PANGFENG LIU WEI-CHUNG HSU |
關鍵字: | dynamic programming | heterogeneous system architectures | model scheduling | pipelining | Video captioning | 公開日期: | 25-五月-2022 | 出版社: | ASSOC COMPUTING MACHINERY | 卷: | 19 | 期: | 3 | 來源出版物: | ACM Transactions on Architecture and Code Optimization | 摘要: | Video captioning is a core technology to many important applications, such as AI-assisted medical diagnosis, video question answering, storytelling through videos, and lip-reading. Video captioning employs a hybrid CNN + RNN model. Accelerating such a hybrid model on a heterogeneous system is challenging for two reasons. First, CNN and RNN exhibit very different computing behaviors, making the mapping between computation and heterogeneous devices difficult. Second, data dependency exists between the CNN and RNN within a video frame and between adjacent RNNs across video frames. These data dependencies prohibit the full parallelization of the hybrid model. The issues also include the utilization of accelerator resources, which is critical to maximizing the performance. In this work, we propose a fine-grained scheduling scheme for mapping computation and devices within a video frame, and a pipeline scheduling scheme for exploiting maximum parallelism between the execution of the video frames. In addition, we propose two capacity-guided scheduling methods. On the server, the concurrent kernel execution mechanism is exploited for improving GPU utilization. On the edge platform, we rearrange CNN computation among the CPU and EdgeTPUs guided by the EdgeTPU's SRAM capacity so that balanced computation is achieved and off-chip memory overhead is minimized. Experimental results show that our scheduling scheme improves video captioning performance by up to 3.24 with CPU + GPU collaboration over the GPU-only execution. On an edge platform with an ARM CPU and two EdgeTPUs, our CPU + EdgeTPU scheduling exhibits outstanding performance, which achieves up to 54.9 speedup compared to using ARM CPU only and can perform video captioning of 59 frames per second. |
URI: | https://scholars.lib.ntu.edu.tw/handle/123456789/633077 | ISSN: | 15443566 | DOI: | 10.1145/3527609 |
顯示於: | 資訊工程學系 |
在 IR 系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。