动态车联网中的任务卸载策略

Task Offloading Strategy in Dynamic Internet of Vehicles

摘要

为解决动态车联网(Internet of Vehicles, IoV)环境中的任务卸载决策问题,首先,提出一个由车辆、多个边缘服务器和云服务器组成的三层任务卸载网络架构;其次,提出以最小化任务卸载时延和能耗、提高服务质量为目标的任务卸载决策问题(task offloading decisions problem with the aim of minimizing latency, energy consumption, and enhancing service quality, TOD-LEQ);再次,将TOD-LEQ建模为Markov决策过程(Markov decision process, MDP)模型;最后,为应对动态变化环境对车联网中任务卸载的影响,提出一种基于元学习的分布式强化学习任务卸载(meta learning-based distributed reinforcement learning task offloading, ME-DRO)算法,该算法可在多线程中执行,以寻求最优的任务卸载决策.仿真实验结果表明,ME-DRO算法在时延、服务质量和收敛速度方面明显优于其他基线算法.

Abstract

To address the task offloading decision problem in a dynamic Internet of Vehicles (IoV) network environment, a three-tier task offloading network architecture consisting of vehicles, multiple edge servers, and cloud servers is proposed. Secondly, the task offloading decisions problem with the aim of minimizing latency, energy consumption, and enhancing service quality (TOD-LEQ) in this network is defined. Thirdly, the TOD-LEQ is modeled as a Markov decision process (MDP) model. Finally, in order to cope with the impact of dynamically changing environments on task offloading in telematics, a meta learning-based distributed reinforcement learning task offloading (ME-DRO) algorithm is proposed, which can be executed in multiple threads to seek the optimal task offloading decision. Simulation results show that the ME-DRO algorithm significantly outperforms other baseline algorithms in terms of latency, quality of service and convergence speed.

关键词

车联网 / 任务卸载 / Markov决策过程 / 元学习

Key words

Internet of Vehicles / task offloading / Markov decision process / meta learning

参考文献

[1] Wang Z G, Zhan J, Duan C G, et al. A Review of Vehicle Detection Techniques for Intelligent Vehicles[J]. IEEE Transactions on Neural Networks and Learning Systems, 2023, 34(8): 3811-3831.

[2] 王亚丽, 娄世豪. 基于多智能体深度强化学习的车联网可信任务卸载策略[J]. 计算机应用研究, 2024, 41(7):1971-1976.

[3] (Wang Y L, Lou S H. Strategy on Trusted Task Offloading for Internet of Vehicles Based on Multi-agent Deep Reinforcement Learning[J]. Computer Application Research, 2024, 41(7): 1971-1976.)

[4] Ahmed M, Raza S, Mirza M A, et al. A Survey on Vehicular Task Offloading: Classification, Issues, and Challenges[J]. Journal of King Saud University-Computer and Information Sciences, 2022, 34(7): 4135-4162.

[5] Akyildiz O, Okay F Y, Kök̇ I, et al. Road to Efficiency: Mobility-Driven Joint Task Offloading and Resource Utilization Protocol for Connected Vehicle Networks[J]. Future Generation Computer Systems, 2024, 156: 157-167.

[6] 朱思峰, 王钰, 陈昊, 等. 车联网边缘计算场景下基于改进型NSGA-Ⅱ算法的边缘服务器部署决策[J]. 物联网学报, 2024, 8(1):84-97.

[7] (Zhu S F, Wang Y, Chen H, et al. Edge Server Deployment Decision Based on Improved NSGA-Ⅱ in the Internet of Vehicles Edge Computing Scenario[J]. Journal of the Internet of Things, 2024, 8(1): 84-97.)

[8] Dai X X, Xiao Z, Jiang H B, et al. UAV-Assisted Task Offloading in Vehicular Edge Computing Networks[J]. IEEE Transactions on Mobile Computing, 2024, 23(4): 2520-2534.

[9] Fan X G, Gu W T, Long C Q, et al. Optimizing Task Offloading and Resource Allocation in Vehicular Edge Computing Based on Heterogeneous Cellular Networks[J]. IEEE Transactions on Vehicular Technology, 2024, 73(5): 7175-7187.

[10] Tang H J, Wu H M, Qu G J, et al. Double Deep Q-Network Based Dynamic Framing Offloading in Vehicular Edge Computing[J]. IEEE Transactions on Network Science and Engineering, 2023, 10(3): 1297-1310.

[11] Ke H C, Wang J, Deng L Y, et al. Deep Reinforcement Learning-Based Adaptive Computation Offloading for MEC in Heterogeneous Vehicular Networks[J]. IEEE Transactions on Vehicular Technology, 2020, 69(7): 7916-7929.

[12] Cheng W J, Liu X S, Wang X T, et al. Task Offloading and Resource Allocation for Industrial Internet of Things: A Double-Dueling Deep Q-Network Approach[J]. IEEE Access, 2022, 10: 103111-103120.

[13] Lu H D, He X M, Du M, et al. Edge QoE: Computation Offloading with Deep Reinforcement Learning for Internet of Things[J]. IEEE Internet of Things Journal, 2020, 7(10): 9255-9265.

基金资助

国家自然科学基金(62402298)

山西省基础研究计划资助项目(202403021212167)

山西省高等学校科技创新项目(Z48019)