ResFlow: Multi-tasking of Sequentially Pooling Spatiotemporal Features for Action Recognition and Optical Flow Estimation
Journal
IEEE International Conference on Intelligent Robots and Systems
Pages
2835-2840
Date Issued
2019
Author(s)
Abstract
Since deep-learning-based method has been widely-used and is capable of generating generic model, most existing methods about action recognition use either two-stream structure, considering spatial and temporal features separately, or C3D, costing lots of prices in memory and time. We aim to design a robust system to extract spatiotemporal features with aggregation mechanism to integrate local features in temporal order. In light of this, we propose ResFlow to estimate optical flow and predict action recognition simultaneously. Leveraging the characteristic of optical flow estimation, we extract spatiotemporal feature via an autoencoder. Via a novel Sequentially Pooling Mechanism which literally pool global spatiotemporal feature sequentially, we extract spatiotemporal feature at each time and aggregate these local features into global feature. This design use only RGB images as input with temporal information encoded, pre-trained by optical flow, and sequentially aggregate spatiotemporal features in high efficiency. We evaluate our ability of estimating optical flow on FlyingChairs dataset and show the promising results of action recognition on UCF-101 dataset through a series of experiments. ? 2019 IEEE.
Subjects
Aggregates; Costs; Deep learning; Intelligent robots; Learning systems; Action recognition; Aggregation mechanism; Generic modeling; Learning-based methods; Optical flow estimation; Spatio temporal features; Temporal features; Temporal information; Optical flows
Type
conference paper