Unifying and merging well-trained deep neural networks for inference stage
Journal
IJCAI International Joint Conference on Artificial Intelligence
Journal Volume
2018-July
Pages
2049-2056
Date Issued
2018
Author(s)
Abstract
We propose a novel method to merge convolutional neural-nets for the inference stage. Given two well-trained networks that may have different architectures that handle different tasks, our method aligns the layers of the original networks and merges them into a unified model by sharing the representative codes of weights. The shared weights are further re-trained to fine-tune the performance of the merged model. The proposed method effectively produces a compact model that may run original tasks simultaneously on resource-limited devices. As it preserves the general architectures and leverages the co-used weights of well-trained networks, a substantial training overhead can be reduced to shorten the system development time. Experimental results demonstrate a satisfactory performance and validate the effectiveness of the method. ? 2018 International Joint Conferences on Artificial Intelligence. All right reserved.
Subjects
Artificial intelligence; Network architecture; Compact model; General architectures; Inference stages; Resource-limited devices; System development; Training overhead; Unified Modeling; Deep neural networks
Type
conference paper