EECS: Efficient End-Cloud Collaborative System With On-Demand Offloading Mechanism for Convolutional Neural Network
Journal
IEEE Internet of Things Journal
Journal Volume
12
Journal Issue
8
Start Page
11081
End Page
11095
ISSN
2327-4662
Date Issued
2025-04-15
Author(s)
Abstract
The end-cloud synergy with neural networks offers significant promise for enhancing AI-IoT systems. Numerous studies have been conducted to effectively address the dynamic and resource-constrained nature of IoT; however, challenges persist in adaptability to constraints, horizontal integration between technologies, and system-level optimization. In this article, we propose an end-cloud collaborative system (EECS) with an on-demand offloading mechanism for AI-IoT. We enhance three critical areas previously narrowly explored: 1) model pruning to fit into the end device’s constraints; 2) dynamic data transmission between end and cloud; and 3) an effective offloading policy. We pay particular attention to the constraints imposed by multiply-and-add (MAC) operations and bandwidth (BW) at the end, with a focus on image classification tasks. Our contributions are threefold. First, we introduce a two-stage trainable pruning method that can automatically adjust the end model to end constraints and optimize from a system-wide perspective. Second, we propose an adaptive mechanism that accommodates fluctuating BW conditions based on our trainable pruning, integrating static pruning with dynamic inference. Third, we develop a smart offloading policy that enhances decision-making, thereby elevating system-level efficiency. Finally, our EECS shows substantial improvements, achieving 2.6×–4× reductions in MAC costs and 3.8×–10.4× reductions in BW costs compared to existing approaches, based on evaluations using the CIFAR-100 and Tiny-ImageNet-200 datasets.
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Type
journal article