https://scholars.lib.ntu.edu.tw/handle/123456789/581454
標題: | Spatiotemporal super-resolution with cross-task consistency and its semi-supervised extension | 作者: | Lin H.-Y Hsiu P.-C Lin Y.-Y. TEI-WEI KUO |
公開日期: | 2020 | 卷: | 2021-January | 起(迄)頁: | 615-622 | 來源出版物: | IJCAI International Joint Conference on Artificial Intelligence | 摘要: | Spatiotemporal super-resolution (SR) aims to upscale both the spatial and temporal dimensions of input videos, and produces videos with higher frame resolutions and rates. It involves two essential sub-tasks: spatial SR and temporal SR. We design a two-stream network for spatiotemporal SR in this work. One stream contains a temporal SR module followed by a spatial SR module, while the other stream has the same two modules in the reverse order. Based on the interchangeability of performing the two sub-tasks, the two network streams are supposed to produce consistent spatiotemporal SR results. Thus, we present a cross-stream consistency to enforce the similarity between the outputs of the two streams. In this way, the training of the two streams is correlated, which allows the two SR modules to share their supervisory signals and improve each other. In addition, the proposed cross-stream consistency does not consume labeled training data and can guide network training in an unsupervised manner. We leverage this property to carry out semi-supervised spatiotemporal SR. It turns out that our method makes the most of training data, and can derive an effective model with few high-resolution and high-frame-rate videos, achieving the state-of-the-art performance. The source code of this work is available at https://hankweb.github.io/STSRwithCrossTask/. ? 2020 Inst. Sci. inf., Univ. Defence in Belgrade. All rights reserved. |
URI: | https://www.scopus.com/inward/record.uri?eid=2-s2.0-85097355045&partnerID=40&md5=3abde44133435ce3843b217d2ce446cd https://scholars.lib.ntu.edu.tw/handle/123456789/581454 |
ISSN: | 10450823 | SDG/關鍵字: | Artificial intelligence; Optical resolving power; High frame rate; High resolution; Labeled training data; Network streams; Network training; State-of-the-art performance; Super resolution; Temporal dimensions; Data streams |
顯示於: | 資訊工程學系 |
在 IR 系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。