关键词： Mobile edge computing   Unmanned aerial vehicle   Task offloading   Federated reinforcement learning   Split learning  

摘要： The convergence of mobile edge computing (MEC) network with unmanned aerial vehicles (UAVs) presents an auspicious opportunity to revolutionize wireless communication and facilitate high-speed internet access in remote regions for mobile devices (MDs) as well as large scale artificial intelligence (AI) models. However, the substantial amount of data produced by the UAVs-assisted MEC network necessitates the integration of efficient distributed learning techniques in AI models. In recent times, distributed learning algorithms, including federated reinforcement learning (FRL) and split learning (SL), have been explored for the purpose of learning machine learning (ML) models that are distributed by sharing model parameters, as opposed to large raw data-sets as seen in traditional centralized learning algorithms. To implement the hybrid method, the model is first trained locally on each UAV-assisted MEC network using SL. Subsequently, the model parameters that have been encrypted are sent to a central server for federated averaging. Finally, after the model has been updated, it is distributed to each UAV-assisted MEC network for local fine-tuning. Our simulations indicate that the proposed split and federated reinforcement learning (SFRL) framework yields comparable high-test accuracy performance while consuming less energy compared to extant distributed learning algorithms. Furthermore, the SFRL algorithm efficiently realizes energy-efficient selection between the SL and FRL methods under different distributions. Numerical results shows that the proposed scheme improves the accuracy by 29.31% and reduced the energy consumption by around 67.34% and time delay by about 7.37%. as compared to the existing baseline schemes.

关键词： Unmanned aerial vehicle   mobile edge computing   trajectory planning   offloading scheduling   multi-objective reinforcement learning  

摘要： Utilizing unmanned aerial vehicles (UAVs) with edge server to assist terrestrial mobile edge computing (MEC) has attracted tremendous attention. Nevertheless, state-of-the-art schemes based on deterministic optimizations or single-objective reinforcement learning (RL) cannot reduce the backlog of task bits and simultaneously improve energy efficiency in highly dynamic network environments, where the design problem amounts to a sequential decision-making problem. In order to address the aforementioned problems, as well as the curses of dimensionality introduced by the growing number of terrestrial terrestrial users, this paper proposes a distributed multi-objective (MO) dynamic trajectory planning and offloading scheduling scheme, integrated with MORL and the kernel method. The design of n-step return is also applied to average fluctuations in the backlog. Numerical results reveal that the n-step return can benefit the proposed kernel-based approach, achieving significant improvement in the long-term average backlog performance, compared to the conventional 1-step return design. Due to such design and the kernel-based neural network, to which decision-making features can be continuously added, the kernel-based approach can outperform the approach based on fully-connected deep neural network, yielding improvement in energy consumption and the backlog performance, as well as a significant reduction in decision-making and online learning time.

关键词： Reinforcement learning   collaborative caching   content popularity   mobile edge computing  

摘要： With the enormous growth in mobile data traffic over the 5G environment, Adaptive BitRate (ABR) video streaming has become a challenging problem. Recent advances in Mobile Edge Computing (MEC) technology make it feasible to use Base Stations (BSs) intelligently by network caching, popularity-based video streaming, and more. Additional computing resources on the edge node offer an opportunity to reduce network traffic on the backhaul links during peak traffic hours. More recently, it has been found in the literature that collaborative caching strategies between neighbouring BSs (i.e., MEC servers) make it more efficient to reduce backhaul traffic and network congestion and thus improve the viewer experience substantially. In this work, we propose a Reinforcement Learning (RL)-based collaborative caching mechanism in which the edge servers cooperate to serve the requested content from the end-users. Specifically, this research aims to improve the overall cache hit rate at the MEC, where the edge servers are clustered based on their geographic locations. This task is modelled as a multi-objective optimization problem and solved using an RL framework. In addition, a novel cache admission and eviction policy is defined by calculating the priority score of video segments in the clustered MEC mesh network.

关键词： task offloading   deep reinforcement learning   mobile edge computing  

摘要： With the proliferation of intelligent applications, mobile devices are increasingly handling computation-intensive tasks but often struggle with limited computing power and energy resources. Mobile Edge Computing (MEC) offers a solution by enabling these devices to offload computation-intensive tasks to resource-rich edge servers, thus reducing processing latency and energy consumption. However, existing task-offloading strategies often neglect critical security concerns. In this paper, we propose a security-aware task-offloading framework that utilizes Deep Reinforcement Learning (DRL) to solve these challenges. Our framework is designed to minimize the latency of task accomplishment and energy consumption while ensuring data security. We model system utility as a Markov Decision Process (MDP) and design a Proximal Policy Optimization (PPO)-based algorithm to derive optimal offloading strategies. Experimental results demonstrate that the proposed algorithm outperforms traditional methods regarding task execution latency and energy consumption.

关键词： Internet of Things   Digital twins   Delays   Approximation algorithms   Sensors   Heuristic algorithms   Data models   Digital twin   mobile edge computing   query services   age of information   IoT applications   service delays   approximation algorithms   resource allocation   optimization  

摘要： The Mobile Edge Computing (MEC) paradigm gives impetus to the vigorous advancement of the Internet of Things (IoT), through provisioning low-latency computing services at network edges. The emerging digital twin technique has been explosively growing in the IoT community, which bridges the gap between physical objects and their digital representations in an MEC network, enabling real-time monitoring and analysis, simulations on the dynamics of systems, accurate predictions on behaviours of objects, and optimization on network resource allocation. In this paper, we consider AoI-aware query services in an MEC network empowered by digital twin technology for diverse IoT applications. We aim to maximize the weighted sum of the accumulative freshness of query results measured by the Age of Information (AoI) and the total query service delay of admitted requests. To this end, we first formulate a novel minimization problem that explores nontrivial trade-offs between the two conflicting optimization objectives: the freshness of query results and service delays, and we show the NP-hardness of the problem. Then, we propose an approximation algorithm with a provable approximation ratio for the problem, at the expense of bounded computing capacity violations. We also develop a heuristic for the problem without any capacity violations. We finally evaluate the performance of the proposed algorithms via simulations. The simulation results demonstrate that the proposed algorithms are promising, and outperform the comparison benchmarks.

关键词： federated learning   reverse auction   mobile edge computing  

摘要： Federated learning (FL) can break the problem of data silos and allow multiple data owners to collaboratively train shared machine learning models without disclosing local data in mobile edge computing. However, how to incentivize these clients to actively participate in training and ensure efficient convergence and high test accuracy of the model has become an important issue. Traditional methods often use a reverse auction framework but ignore the consideration of client volatility. This paper proposes a multi-dimensional reverse auction mechanism (MRATR) that considers the uncertainty of client training time and reputation. First, we introduce reputation to objectively reflect the data quality and training stability of the client. Then, we transform the goal of maximizing social welfare into an optimization problem, which is proven to be NP-hard. Then, we propose a multi-dimensional auction mechanism MRATR that can find the optimal client selection and task allocation strategy considering clients' volatility and data quality differences. The computational complexity of this mechanism is polynomial, which can promote the rapid convergence of FL task models while ensuring near-optimal social welfare maximization and achieving high test accuracy. Finally, the effectiveness of this mechanism is verified through simulation experiments. Compared with a series of other mechanisms, the MRATR mechanism has faster convergence speed and higher testing accuracy on both the CIFAR-10 and IMAGE-100 datasets.

关键词： Task analysis   Autonomous aerial vehicles   Data collection   Heuristic algorithms   Training   Information sharing   Crowdsensing   Decentralized and autonomous data collection   mobile crowdsensing   mobile edge computing   multi-agent deep reinforcement learning   task state information sharing  

摘要： With the rapid development of edge computing technology, edge-assisted unmanned aerial vehicle (UAV) networks have become popular, helping with fast and cost-effective data collection in mobile crowdsensing (MCS) environments. This paper investigates the online data collection problem for MCS over an edge-assisted UAV network architecture, where UAVs work to collect the data required by tasks at different on-ground point-of-interests (PoIs) in an autonomous and cooperative manner. Different from conventional edge-assisted UAV networks, edge nodes in our paper help distribute, aggregate, share, and update the task state information (TSI, e.g., if a task has been completed, or if it still requires more data), to support efficient and cost-effective data collection, e.g., avoiding repetitive and ineffective task execution. In particular, a UAV can exchange TSI with an edge node when it is within the signal coverage of the edge node, while making decisions on PoI selection and path planning in an online and decentralized manner. To address the issue of edge-assisted UAV data collection, we propose a multi-agent deep reinforcement learning-based algorithm using personalized training with decentralized executing (PTDE) architecture. Different from the traditional centralized training with decentralized executing (CTDE) architecture, our considered architecture adopts the agent-specific state for critic networks instead of the joint observation, thus achieving effective utilization of environmental information. Furthermore, we propose an observation enhancement algorithm based on artificial potential field (APF). Extensive simulation results demonstrate that our proposed algorithm greatly outperforms baseline algorithms in terms of total profit, data collection ratio, geographical fairness, and energy efficiency.

关键词： Edge user allocation   fairness   game theory   integer programming   mobile edge computing   optimization   potential game  

摘要： Mobile edge computing (MEC) offers a new computing paradigm that turns computing and storage resources to the network edge to provide minimal service latency compared to cloud computing. Many research works have attempted to help app vendors allocate users to appropriate edge servers for high-performance service provisioning. However, existing edge user allocation (EUA) approaches have ignored fairness in users' data rates caused by interference, which is crucial in service provisioning in the MEC environment. To pursue fairness in EUA, edge users need to be assigned to edge servers so their quality of experience can be ensured at minimum costs without significant service performance differences among them. In this paper, we make the first attempt to address this fair edge user allocation (FEUA) problem. Specifically, we formulate the FEUA problem, prove its $\mathcal {NP}$NP-hardness, and propose an optimal approach to solve small-scale FEUA problems. To accommodate large-scale FEUA scenarios, we propose a game-theoretic approach called FEUAGame that transforms the FEUA problem into a potential game that admits a Nash equilibrium. FEUA employs a decentralized algorithm to find the Nash equilibrium in the potential game as the solution to the FEUA problem. A widely-used real-world data set is utilised to experimentally compare the performance of FEUAGame to four representative approaches. The numerical outcomes show the effectiveness and efficiency of the proposed approaches in solving the FEUA problem.

关键词： Training   Servers   Optimization   Deep learning   Computational modeling   Reinforcement learning   Data models   Deep reinforcement learning   edge learning   incentive mechanism   mobile edge computing  

摘要： Over the past few years, edge learning has achieved significant success in mobile edge networks. Few works have designed incentive mechanism that motivates edge nodes to participate in edge learning. However, most existing works only consider myopic optimization and assume that all edge nodes are honest, which lacks long-term sustainability and the final performance assurance. In this paper, we propose Chiron, an incentive-driven Byzantine-resistant long-term mechanism based on hierarchical reinforcement learning (HRL). First, our optimization goal includes both learning-algorithm performance criteria (i.e., global accuracy) and systematical criteria (i.e., resource consumption), which aim to improve the edge learning performance under a given resource budget. Second, we propose a three-layer HRL architecture to handle long-term optimization, short-term optimization, and byzantine resistance, respectively. Finally, we conduct experiments on various edge learning tasks to demonstrate the superiority of the proposed approach. Specifically, our system can successfully exclude malicious nodes and lazy nodes out of the edge learning participation and achieves 14.96% higher accuracy and 12.66% higher total utility than the state-of-the-art methods under the same budget limit.

关键词： UAV routing   Mobile Edge Computing   heterogeneous multiple UAVs   IoT ground devices   heuristic  

摘要： The rapid proliferation of Internet of Things (IoT) ground devices (GDs) has created an unprecedented demand for computing resources and real-time data-processing capabilities. Integrating unmanned aerial vehicles (UAVs) into Mobile Edge Computing (MEC) emerges as a promising solution to bring computation and storage closer to the data sources. However, UAV heterogeneity and the time window constraints for task execution pose a significant challenge. This paper addresses the multiple heterogeneity UAV routing problem in MEC environments, modeling it as a multi-traveling salesman problem (MTSP) with soft time constraints. We propose a two-stage heuristic algorithm, heterogeneous multiple UAV routing (HMUR). The approach first identifies task areas (TAs) and optimal hovering positions for the UAVs and defines an effective fitness measurement to handle UAV heterogeneity. A novel scoring function further refines the path determination, prioritizing real-time task compliance to enhance Quality of Service (QoS). The simulation results demonstrate that our proposed HMUR method surpasses the existing baseline algorithms on multiple metrics, validating its effectiveness in optimizing resource scheduling in MEC environments.