关键词： Autonomous aerial vehicles   Quality of service   Optimization   Resource management   Costs   Batteries   Computational modeling   Throughput   Transformers   Trajectory   Autonomous aerial vehicle   mobile edge computing   decision transformer   deep reinforcement learning  

摘要： Deep reinforcement learning (DRL) has been widely applied to dynamic resource allocation for the autonomous aerial vehicle (AAV)-assisted mobile edge computing (MEC) networks. However, it is often challenging to generalize a well-trained DRL model to new MEC scenarios once the system constraints change, since retraining DRL models from scratch is time and energy consuming. In this letter, we jointly optimize the AAV's trajectory and computing resource allocation for maximizing the fairness-based throughput under the battery capacity and quality of service (QoS) constraints. The sequential optimization problem is formulated as a constrained Markov decision process (CMDP) and solved via the constrained DRL algorithms. To generalize the optimized resource allocation policies across various energy and QoS constraints, we propose an offline pre-training and online fine-tuning based constrained Decision Transformer (CDT) framework. In particular, the CDT is first pre-trained on the training samples collected by the constrained DRL algorithm offline, and then fine-tuned online for rapid adaptation to the unseen constraint thresholds. Simulation results show that compared with the benchmark DRL algorithms, the CDT is capable of effectively improving the fairness-based throughput under the battery capacity and QoS constraints, and demonstrates rapid convergence when constraints change.

关键词： Computational efficiency   Resource management   Delays   Servers   Edge computing   Minimization   Wireless communication   Time-frequency analysis   RF signals   Radio spectrum management   Mobile edge computing   wireless power transfer   multi-agent deep reinforcement learning   energy provision minimization  

摘要： This paper investigates a wireless powered mobile edge computing (WP-MEC) network with multiple hybrid access points (HAPs) in a dynamic environment, where wireless devices (WDs) harvest energy from radio frequency (RF) signals of HAPs, and then compute their computation data locally (i.e., local computing mode) or offload it to the chosen HAPs (i.e., edge computing mode). In order to pursue a green computing design, we formulate an optimization problem that minimizes the long-term energy provision of the WP-MEC network subject to the energy, computing delay and computation data demand constraints. The transmit power of HAPs, the duration of the wireless power transfer (WPT) phase, the offloading decisions of WDs, the time allocation for offloading and the CPU frequency for local computing are jointly optimized adapting to the time-varying generated computation data and wireless channels of WDs. To efficiently address the formulated non-convex mixed integer programming (MIP) problem in a distributed manner, we propose a Two-stage Multi-Agent deep reinforcement learning-based Distributed computation Offloading (TMADO) framework, which consists of a high-level agent and multiple low-level agents. The high-level agent residing in all HAPs optimizes the transmit power of HAPs and the duration of the WPT phase, while each low-level agent residing in each WD optimizes its offloading decision, time allocation for offloading and CPU frequency for local computing. Simulation results show the superiority of the proposed TMADO framework in terms of the energy provision minimization.

关键词： Hybrid quantum model   mobile edge computing   quantum reinforcement learning   quantum reinforcement learning   resource allocation   resource allocation   task offloading   task offloading   task offloading  

摘要： Mobile edge computing (MEC) has revolutionized the way computational tasks are offloaded and latency is reduced by leveraging edge servers close to end devices. Efficient resource allocation and task offloading are crucial for enhancing system performance in MEC environments. Traditional reinforcement learning (RL) approaches have shown promise in optimizing resource allocation and task offloading problems. However, they often face challenges such as high computational complexity and the need for extensive training data. Quantum reinforcement learning (QRL) emerges as a promising solution to overcome these limitations by leveraging quantum computing principles to enhance efficiency and scalability. In this paper, we propose a hybrid quantum-classical non-sequential model for joint resource allocation and task offloading in MEC systems. Our model combines the advantages of RL in handling environmental dynamics and quantum computing in reducing adjustable parameters and accelerating the training process. Extensive experiments demonstrate that our proposed algorithm can achieve higher training and inference performance under various parameter settings compared to traditional RL models and previous QRL models.

关键词： Mobile edge computing   UAV   radar sensing   radar sensing   data compression   data compression   multi-agent deep reinforcement learning   multi-agent deep reinforcement learning   multi-agent deep reinforcement learning  

摘要： In this paper, we propose a multi-unmanned aerial vehicle (UAV)-assisted integrated sensing, communication, and computation network. Specifically, the treble-functional UAVs are capable of offering communication and edge computing services to mobile users (MUs) in proximity, alongside their target sensing capabilities by using multi-input multi-output arrays. For the purpose of enhance the computation efficiency, we consider task compression, where each MU can partially compress their offloaded data prior to transmission to trim its size. The objective is to minimize the weighted energy consumption by jointly optimizing the transmit beamforming, the UAVs' trajectories, the compression and offloading partition, the computation resource allocation, while fulfilling the causal-effect correlation between communication and computation as well as adhering to the constraints on sensing quality. To tackle it, we first reformulate the original problem as a multi-agent Markov decision process (MDP), which involves heterogeneous agents to decompose the large state spaces and action spaces of MDP. Then, we propose a multi-agent proximal policy optimization algorithm with attention mechanism to handle the decision-making problem. Simulation results validate the significant effectiveness of the proposed method in reducing energy consumption. Moreover, it demonstrates superior performance compared to the baselines in relation to resource utilization and convergence speed.

关键词： Mobile edge computing   trajectory planning   task offoading   MIMO AAV   Mobile edge computing   trajectory planning   task offoading   MIMO AAV  

摘要： Edge computing is conducive to reducing service response time and improving service quality by pushing cloud functions to a network's edges. Most existing works in edge computing focus on utility maximization of task offloading on static edges with a single antenna. Besides, trajectory planning of mobile edges, e.g., autonomous aerial vehicles (AAVs) is also rarely discussed. In this paper, we are the first to jointly discuss the deadline-ware task offloading and AAV trajectory planning problem in a multi-input multi-output (MIMO) AAV-aided mobile edge computing system. Due to discrete variables and highly coupling nonconvex constraints, we equivalently convert the original problem into a more solvable form by introducing auxiliary variables. Next, a penalty dual decomposition-based algorithm is developed to achieve a global optimal solution to the problem. Besides, we proposed a profit-based fireworks algorithm in a relatively lower time to reduce the execution time for large-scale networks. Extensive evaluation results reveal that our proposed optimal algorithms could significantly outperform static offloading algorithms and other algorithms by 25% on average.

关键词： Mobile edge computing   blockchain   task- offloading   task- offloading   multi-armed bandits   multi-armed bandits  

摘要： As mobile devices proliferate and mobile applications diversify, Mobile Edge Computing (MEC) has become widely adopted to efficiently allocate computing resources at the network edge and alleviate network congestion. In the MEC initial phase, the absence of vital information presents challenges in devising task-offloading policies, and identifying malicious devices responsible for providing inaccurate feedback is complex. To fill in such gaps, we introduce a consortium blockchain-enabled Committee Voting based Task Offloading Model (CVTOM) to collaboratively formulate resource allocation policies and establish deterrence against malicious servers producing erroneous results intentionally. Different voting principle mechanisms of each committee member are first designed in a Blockchain-enabled system which helps to represent the system's resource status. Additionally, we propose a Multi-armed Bandits related Thompson Sampling based Adaptive Preference Optimization (TSAPO) algorithm for task-offloading policy, enhancing the timely identification of potent edge servers to improve computing resource utilization which first considers dynamic edge server space and parallel computing scenarios. The solid proof process greatly contributes to the theoretical analysis of the TSAPO. The simulation experiments demonstrate the delay and budget can be reduced by around 25% and 10% respectively, showcasing the superior performance of our approach.

关键词： Autonomous aerial vehicles   Trajectory   Trajectory planning   Games   Costs   Computational modeling   Resource management   Planning   Benchmark testing   Time-frequency analysis   Mobile edge computing   autonomous aerial vehicle   Stackelberg game   deep reinforcement learning  

摘要： Nowadays, Mobile Edge Computing (MEC) has been widely deployed to enhance the computational capabilities of mobile devices. However, the geographic location of MEC servers is usually fixed. In order to provide flexible edge computing services, some works have considered integrating Autonomous aerial vehicles (AAVs) into MEC networks. In the context of AAV-assisted edge computing, there usually exist multiple AAVs and users, and each AAV may aim to maximize its profit by providing computing services, while users will decide which AAVs to utilize based on their preferences. In this context, how AAVs and users effectively plan their trajectories becomes particularly important as it will affect the profitability of AAVs and the user experience. Since the trajectories of AAVs and users are affected by each other, we model the trajectories of AAVs and users as a Stackelberg game, and then design trajectory planning strategies for users and AAVs based on Independent Proximal Policy Optimization (IPPO) and Proximal Policy Optimization (PPO) respectively, aiming to maximize AAVs' profits while ensuring user acceptance of AAV services. Finally, we evaluate the proposed trajectory planning strategy against three typical benchmark strategies using synthetic and realistic datasets. The experimental results demonstrate that our strategy can outperform benchmark strategies in terms of AAV profit while guaranteeing users' service experience.

关键词： Servers   Streaming media   Data models   Training   Federated learning   Backhaul networks   Adaptation models   Mobile edge computing   federated learning   reinforcement learning   adaptive bitrate streaming   content caching  

摘要： The explosive growth of mobile data traffic generated primarily from resource-hungry video streaming sessions challenges the service providers to deliver a better Quality of Service to the heterogeneous end-users. Content caching in Edge Computing is a promising solution to cope with this exponential rise in video traffic. The most popular videos are typically stored in the local caches of edge servers to provide fast and continuous access to videos. However, various edge caching strategies fail to cope with the dynamic request patterns of the users. Most learning-based caching models are generally trained in a centralized way, which overconsumes the network resources during training and transmission of video requests. Therefore, we propose a Federated Learning-based Reinforcement Learning caching framework called FedCache in this work. In FedCache, the training is decentralised on the end-user devices with its local data. The trained parameters from the end users are aggregated at the central server. A comprehensive set of experiments were performed, and FedCache outperformed existing state-of-the-art caching strategies concerning cache hit rate, access delay and backhaul traffic significantly.

关键词： Ultra reliable low latency communication   Optimization   Minimization   Array signal processing   Reliability   Reconfigurable intelligent surfaces   Real-time systems   Metaverse   Wireless communication   Industries   Alternating optimization   digital twin   imperfect CSI   mobile edge computing   simultaneously transmitting and reflecting   ultra-reliable low-latency communication  

摘要： In the context of ultra-reliable low-latency communication (URLLC) in Internet-of-Things (IoT) networks, conventional half-space coverage limits the flexibility of reconfigurable intelligent surface (RIS) deployment. To overcome these constraints, this paper makes use of active simultaneously transmitting and reflecting RIS (STAR-RIS), which is seamlessly integrated into digital twin (DT) and mobile edge computing (MEC) frameworks. Our primary research objective is to achieve full-space coverage by enabling simultaneous transmission and reflection of the signals while improving uplink data transmission from IoT URLLC user nodes (UNs) to the base station (BS) with the assistance of active STAR-RIS, even in the presence of imperfect channel state information (CSI). We formulate the problem of minimizing total end-to-end (e2e) latency, computed using the alternating optimization (AO) algorithm. Subsequently, we have evaluated the performance of the AO algorithm against the stochastic gradient descent (SGD) algorithm, which serves as the benchmark solution. The simulation outcomes delineate a performance evaluation under perfect and imperfect CSI scenarios. The AO algorithm outperforms SGD with latency reductions of 19.7% at N=32 and 20.4% at N=64 . Increasing N from 32 to 64 results in a 39.3% latency reduction for AO, surpassing SGD's 38.8%. However, the SGD algorithm consistently exhibits lower computational complexity compared to the AO algorithm. Additionally, the energy splitting mode achieves the system's total e2e latency reductions of 28.4% over the mode switching mode and 11.04% over time switching mode. Furthermore, active STAR-RIS optimal beamforming (ARO) achieves approximate to 10 % latency reduction over the predictive optimal beamforming (PRO), which itself surpasses active STAR-RIS with random beamforming (ARR) by approximate to 9 %. This comparison considers key factors such as the power budget, the number of RIS elements, the caching capacity of the e

关键词： Optimization   Servers   Resource management   Measurement   Genetic algorithms   5G mobile communication   Particle swarm optimization   Internet of Things   Heuristic algorithms   Delays   5G   mobile edge computing   task offloading   mixed integer (MILP) linear optimization   latency   dropped task ratio   task prioritization  

摘要： Multi-Access Edge Computing (MEC) is widely recognized as an essential enabler for applications that necessitate minimal latency. However, the dropped task ratio metric has not been studied thoroughly in literature. Neglecting this metric can potentially reduce the system's capability to effectively manage tasks, leading to an increase in the number of eliminated or unprocessed tasks. This paper presents a 5G-MEC task offloading scenario with a focus on minimizing the dropped task ratio, computational latency, and communication latency. We employ Mixed Integer Linear Programming (MILP), Particle Swarm Optimization (PSO), and Genetic Algorithm (GA) to optimize the latency and dropped task ratio. We conduct an analysis on how the quantity of tasks and User Equipment (UE) impacts the ratio of dropped tasks and the latency. The tasks that are generated by UEs are classified into two categories: urgent tasks and non-urgent tasks. The UEs with urgent tasks are prioritized in processing to ensure a zero-dropped task ratio. Our proposed method improves the performance of the baseline methods, First Come First Serve (FCFS) and Shortest Task First (STF), in the context of 5G-MEC task offloading. Under the MILP-based approach, the latency is reduced by approximately 55% compared to GA and 35% compared to PSO. The dropped task ratio under the MILP-based approach is reduced by approximately 70% compared to GA and by 40% compared to PSO.