A Robust Distributed Hierarchical Online Learning Approach for Dynamic MEC Networks
Journal
IEEE Journal on Selected Areas in Communications
Journal Volume
40
Journal Issue
2
Pages
641-656
Date Issued
2022
Author(s)
Abstract
We consider a resource allocation and offloading decision-making problem in a mobile edge computing (MEC) network. Since the locations of user equipments (UEs) vary over time in practice, we consider a dynamic network, where the UEs could leave or join the network coverage at any location. Since the joint offloading decision that minimizes the network cost also varies with the topology, the expected best offloading decision for the previous topology would not match the new topology. Consequently, the system suffers from recurring cost peaks due to the topology change. Thus, we propose a robust distributed hierarchical online learning approach to enhance the algorithm's robustness and reduce the cost peaks. Specifically, the UEs learn the utility of each offloading decision via deep Q networks (DQNs) from their interaction with the MEC network. Meanwhile, the computational access points (CAPs) train their deep neural networks (DNNs) online with the real-time data collected from the UEs to predict their corresponding Q-value vectors. Therefore, the UEs and CAPs form a hierarchical collaborative-learning structure. When the topology changes, each UE downloads its Q-value vector as the Q-bias vector and learns its difference from the actual Q-value vector via its DQN. With different agents learning distributedly, both the peak and sum costs are reduced as the joint offloading decision could start from a near-local-optimal point. In simulations, our robust approach successfully reduces the peak cost and sum cost by up to 50% and 30%, respectively. This demonstrates the need for a robust learning algorithm design in a practical dynamic MEC network. ? 1983-2012 IEEE.
Subjects
deep Q learning
hierarchical learning
latency
mobile edge computing
network topology
offloading
Resource allocation
robustness
Cost reduction
Decision making
Deep neural networks
E-learning
Learning algorithms
Reinforcement learning
Topology
Deep Q learning
Hierarchical learning
Latency
Network topology
Offloading
Q-learning
Q-values
Resources allocation
Robustness
User equipments
Mobile edge computing
Type
journal article