面向 SAGIN 场景的无人机缓存决策

Research on UAV cache decision in SAGIN scenario

摘要

针对移动边缘计算场景下数据流量高速增长带来的任务缓存时延过长的问题，提出基于深度强化学习算法的缓存优化方案。首先，在使用5G技术的空天地一体化网络场景架构下，以最小化任务缓存时延为目标，对缓存和通信模型进行建模；其次，使用SAC（soft actor-critic）算法对局部最小缓存时延进行干扰，并根据一定概率接受新方案，从而获得全局最大缓存命中率；最后，对上述过程重复迭代，得到目标问题的一个最优解，确保了任务文件被预缓存在最优位置。仿真结果表明，在空天地一体化网络协同架构下，与PPO方案相比，该方案能够减少优化传输效率，降低缓存时延5.30%，提高缓存命中率3.90%。

Abstract

In the mobile edge computing scenario, the task cache delay becomes excessively long due to the rapid growth of data traffic. To address this issue, a cache scheme based on deep reinforcement learning algorithm in mobile edge computing scenarios is proposed. Firstly, under a 5G-based space-air-ground integrated network (SAGIN) architecture, a cache and communication model is established to minimize the content cache delay. Secondly, the SAC (soft actor critical) algorithm is used to interfere with the local minimum cache delay, and a new scheme is accepted with a certain probability, thereby achieving the global maximum cache hit rate. Finally, the above process is iterated repeatedly to obtain an optimal solution for the target problem, ensuring that the task files are pre-delayed in the optimal location. The simulation results show that under the SAGIN cooperation architecture, compared with the PPO (proximal policy optimization) scheme, the cache scheme can reduce the optimization transmission efficiency, reducing the cache delay by 5.30%, and improving the cache hit rate by 3.90%.

关键词

移动边缘计算 / 深度强化学习 / 无人机 / 空天地一体化网络 / 缓存决策

Key words

mobile edge computing / deep reinforcement learning / unmanned aerial vehicle / air-space-ground integration network / caching strategy

参考文献

[1] MACH P , BECVAR Z . Mobile edge computing: A survey on architecture and computation offloading[J]. IEEE Communications Surveys & Tutorials, 2017, 19(3): 1628-1656.

[2] SONG Z Y , HAO Y Y , SUN X . Computation offloading and resource allocation algorithm for collaborative LEO satellite multi—access edge computing[J]. Acta Electronica Sinica, 2022, 50(3): 567-573.

[3] LI Z Y , WANG Q , CHEN Y F , et al. A survey on task offloading research in vehicular edge computing[J]. Chinese Journal of Computers, 2021, 44(5): 963-982.

[4] LIU Y , NI L Q , PENG M G . A secure and efficient authentication protocol for satellite—terrestrial networks[J]. IEEE Internet of Things Journal, 2022, 10(7): 5810-5822.

[5] NIU Z S , SHEN X M , ZHANG Q Y , et al. Space—air—ground integrated vehicular network for connected and automated vehicles: Challenges and solutions[J]. Intelligent and Converged Networks, 2020, 1(2): 142-169.

[6] JI B F , WANG Y A , SONG K , et al. A survey of computational intelligence for 6G: Key technologies, applications and trends[J]. IEEE Transactions on Industrial Informatics, 2021, 17(10): 7145-7154.

[7] YANG L , PAN C S , KONG X G , et al. Review on 5G—satellite integrated network[J]. Journal on Communications, 2022, 43(4): 202-215.

[8] CUI H X , ZHANG J , GENG Y H , et al. Space—air—ground integrated network (SAGIN) for 6G: Requirements, architecture and challenges[J]. China Communications, 2022, 19(2): 90-108.

[9] TANG Q Q , LI B . Overview of mobile edge computing in space—air—ground integrated network[J]. Radio Communications Technology, 2021, 47(1): 25-35.

[10] ZHAO J , LI Q , GONG Y , et al. Computation offloading and resource allocation for cloud assisted mobile edge computing in vehicular networks[J]. IEEE Trans on Vehicular Technology, 2019, 68(8): 7944-7956.

[11] ZHOU F , HU R Q . Computation efficiency maximization in wirelesspowered mobile edge computing networks[J]. IEEE Trans on Wireless Communications, 2020, 19(5): 3170-3184.

[12] WANG Y , LANG P , TIAN D , et al. A game—based computation offloading method in vehicular multiaccess edge computing networks[J]. IEEE Internet of Things Journal, 2020, 7(6): 4987-4996.

[13] NING Z , DONG P , KONG X , et al. A cooperative partial computation offloading scheme for mobile edge computing enabled internet of things[J]. IEEE Internet of Things Journal, 2019, 6(3): 4804-4814.

[14] ZHOU S , JADOON W . The partial computation offloading strategy based on game theory for multi—user in mobile edge computing environment[J]. Computer Networks, 2020, 178: 107334.

[15] BAHREINI T , BADRI H , GROSU D . Mechanisms for resource allocation and pricing in mobile edge computing systems[J]. IEEE Trans on Parallel and Distributed Systems, 2022, 33(3): 667-682.

[16] HU H , SONG W , WANG Q , et al. Energy efficiency and delay tradeoff in an MEC—enabled mobile IoT network[J]. IEEE Internet of Things Journal, 2022, 9(17): 15942-15956.

[17] SABUJ S R , ASIEDU D K P , LEE K J , et al. Delay optimization in mobile edge computing: Cognitive UAV—assisted eMBB and mMTC services[J]. IEEE Trans on Cognitive Communications and Networking, 2022, 8(2): 1019-1033.

[18] FENG J , RICHARD Y F , PEI Q , et al. Cooperative computation offloading and resource allocation for blockchain—enabled mobile—Edge computing: A deep reinforcement learning approach[J]. IEEE Internet of Things Journal, 2020, 7(7): 6214-6228.

[19] LIU F , ZHANG Z , WANG Z , et al. ECC: Edge collaborative caching strategy for differentiated services load—balancing[J]. Computers, Materials and Continua, 2021, 69(2): 2045-2060.

[20] 刘亮, 荆腾祥, 段洁, 等. 空天地一体化网络中基于联邦深度强化学习的边缘协作缓存策[J]. 通信学报, 2025, 46(1): 93-107.

[21] LIU L , JING T X , DUAN J , et al. Federated deep reinforcement learning—based edge collaborativecaching strategy in space—air—ground integrated network[J]. Journal on Communications, 2025, 46(1): 93-107.

[22] PENG K , NIE J , KUMAR N , et al. Joint optimization of service chain caching and task offloading in mobile edge computing[J]. Applied Soft Computing, 2021, 103: 107142.

基金资助

国家自然科学基金(62172457)

天津市自然科学基金重点项目(22JCZDJC00600)

河南省高校科技创新人才支持计划(23HASTIT029)

河南省科技攻关项目(242102210027)