Fully nested neural network for adaptive compression and quantization
Journal
IJCAI International Joint Conference on Artificial Intelligence
Journal Volume
2021-January
Pages
2080-2087
Date Issued
2020
Author(s)
Abstract
Neural network compression and quantization are important tasks for fitting state-of-the-art models into the computational, memory and power constraints of mobile devices and embedded hardware. Recent approaches to model compression/quantization are based on reinforcement learning or search methods to compress/quantize the neural network for a specific hardware platform. However, these methods require multiple runs to compress/quantize the same base neural network to different hardware setups. In this work, we propose a fully nested neural network (FN3) that runs only once to build a nested set of compressed/quantized models, which is optimal for different resource constraints. Specifically, we exploit the additive characteristic in different levels of building blocks in neural network and propose an ordered dropout (ODO) operation that ranks the building blocks. Given a trained FN3, a fast heuristic search algorithm is run offline to find the optimal removal of components to maximize the accuracy under different constraints. Compared with the related works on adaptive neural network designed only for channels or bits, the proposed approach is unified for different levels of building blocks (bits, neurons, channels, residual paths and layers). Empirical results validate strong practical performance of the proposed approach. ? 2020 Inst. Sci. inf., Univ. Defence in Belgrade. All rights reserved.
Other Subjects
Heuristic algorithms; Optimization; Reinforcement learning; Adaptive compression; Adaptive neural networks; Heuristic search algorithms; Model compression; Network compression; Power constraints; Resource Constraint; Specific hardware; Multilayer neural networks
Type
conference paper