To enhance the operational efficiency and intelligent decision making capability of automated container terminals, this study addresses the collision free path planning problem for multiple AGVs operating bidirectionally in the horizontal transport area. Considering the layout characteristics of the terminal’s horizontal transport zone, a grid-based map is constructed, and the AGV collision-free path planning problem is formulated as a mathematical programming model with the objective of minimizing the makespan (i.e., the maximum completion time) of all tasks. A PPO algorithm is then designed. A simulation environment tailored for bidirectional guideway systems is developed, incorporating specifically designed action and state spaces for multi-AGV operations. An anti-detour heuristic is introduced to improve the search efficiency of the algorithm. The proposed algorithm is benchmarked against the commercial solver Gurobi, the A* algorithm, and a genetic algorithm. Simulation results demonstrate that the proposed algorithm exhibits superior solution stability and stronger convergence capability, particularly for large-scale problem instances.
ROYD, de KOSTERR, BEKKERR. Modeling and Design of Container Terminal Operations[J]. Operations Research, 2020, 68(3): 686-715.
[2]
YUANRuiping, DONGTingting, LIJuntao. Research on the Collision-free Path Planning of Multi-AGVs System Based on Improved A* Algorithm[J]. American Journal of Operations Research, 2016, 6(6): 442-449.
[3]
YANGYongsheng, ZHONGMeisu, DESSOUKYY, et al. An Integrated Scheduling Method for AGV Routing in Automated Container Terminals[J]. Computers & Industrial Engineering, 2018, 126: 482-493.
[4]
MinLÜ, GAOTong, ZHANGNian. Research of AGV Scheduling and Path Planning of Automatic Transport System[J]. International Journal of Control and Automation, 2016, 9(4): 1-12.
[5]
KIMC W, TANCHOCOJJ M A. Operational Control of a Bidirectional Automated Guided Vehicle System[J]. International Journal of Production Research, 1993, 31(9): 2123-2138.
[6]
WANGZehao, ZENGQingcheng. A Branch-and-bound Approach for AGV Dispatching and Routing Problems in Automated Container Terminals[J]. Computers & Industrial Engineering, 2022, 166: 107968.
GAOYilu, HUZhihua. Path Planning for Automated Guided Vehicles Based on Tempo-spatial Network at Automated Container Terminal[J]. Journal of Computer Applications, 2020, 40(7): 2155-2163.
[9]
SHENLixin, WANGYaodong, LIUKunpeng, et al. Synergistic Path Planning of Multi-UAVs for Air Pollution Detection of Ships in Ports[J]. Transportation Research Part E: Logistics and Transportation Review, 2020, 144: 102128.
LIANGChengji, SHENShanshan, HUWenhui. AGV Path Planning Considering Alternative Paths Based on Time Window of Road Section[J]. Chinese Journal of Engineering Design, 2018, 25(2): 200-208.
[12]
HUYejun, DONGLiangcai, XULei. Multi-AGV Dispatching and Routing Problem Based on a Three-stage Decomposition Method[J]. Mathematical Biosciences and Engineering, 2020, 17(5): 5150-5172.
[13]
XUYixiang, QILiang, LUANWenjing, et al. Load-in-load-out AGV Route Planning in Automatic Container Terminal[J]. IEEE Access, 2020, 8: 157081-157088.
JIANGChenkai, LIZhi, PANShubao, et al. Collision-free Path Planning of AGVs Based on Improved Dijkstra Algorithm[J]. Computer Science, 2020, 47(8): 272-277.
[16]
TANGGang, TANGCongqiang, CLARAMUNTC, et al. Geometric A-star Algorithm: an Improved A-star Algorithm for AGV Path Planning in a Port Environment[J]. IEEE Access, 2021, 9: 59196-59210.
DINGYi, YUANHao, FANGHuaijin, et al. An Optimal Scheduling Method of AGVs at Automated Container Terminal Considering Conflict Avoidance[J]. Journal of Transport Information and Safety, 2022, 40(3): 96-107.
[19]
ZHONGMeisu, YANGYongsheng, DESSOUKYY, et al. Multi-AGV Scheduling for Conflict-free Path Planning in Automated Container Terminals[J]. Computers & Industrial Engineering, 2020, 142: 106371.
[20]
WUMaopu, GAOJian, LILe, et al. Control Optimisation of Automated Guided Vehicles in Container Terminal Based on Petri Network and Dynamic Path Planning[J]. Computers and Electrical Engineering, 2022, 104: 108471.
[21]
CHENTianjian, SUNYuan, DAIWei, et al. On the Shortest and Conflict-free Path Planning of Multi-AGV System Based on Dijkstra Algorithm and the Dynamic Time-window Method[J]. Advanced Materials Research, 2013, 645: 267-271.
[22]
LIUWenqian, ZHUXiaoning, WANGLi, et al. Multiple Equipment Scheduling and AGV Trajectory Generation in U-shaped Sea-rail Intermodal Automated Container Terminal[J]. Measurement, 2023, 206: 112262.
[23]
CAOMenglong, PENGZhang. Research on Loading and Unloading Path Optimization for AGV at Automatic Container Terminal Based on Improved Particle Swarm Algorithm[C]∥2020 4th Annual International Conference on Data Science and Business Analytics (ICDSBA). IEEE, 2020: 1-5.
[24]
LUTong, SUNZhaohui, QIUSiqi, et al. Time Window Based Genetic Algorithm for Multi-AGVs Conflict-free Path Planning in Automated Container Terminals[C]∥2021 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM). IEEE, 2021: 603-607.
YUEChunlei, HUANGJun, DENGLele. Research on Improved Ant Colony Algorithm in AGV Path Planning[J]. Computer Engineering and Design, 2022, 43(9): 2533-2541.
[27]
荀燕琴. 基于群体智能优化的AGV路径规划算法研究[D]. 长春: 吉林大学, 2017.
[28]
XUNYanqin. Research on AGV Path Planning Algorithm Based on Group Intelligence Optimization[D]. Changchun: Jilin University, 2017.
[29]
CRUZD L, YUWen. Path Planning of Multi-agent Systems in Unknown Environment with Neural Kernel Smoothing and Reinforcement Learning[J]. Neurocomputing, 2017, 233: 34-42.
[30]
POPPERJ, YFANTISV, RUSKOWSKIM. Simultaneous Production and AGV Scheduling Using Multi-agent Deep Reinforcement Learning[J]. Procedia CIRP, 2021, 104: 1523-1528.
[31]
ÇETINKAYAM. Multi-agent Path Planning Using Deep Reinforcement Learning[J]. ArXiv Preprint ArXiv: 2021.
[32]
CHENXinqiang, LIUShuhao, ZHAOJiansen, et al. Autonomous Port Management Based AGV Path Planning and Optimization via an Ensemble Reinforcement Learning Framework[J]. Ocean & Coastal Management, 2024, 251: 107087.
[33]
WANGTingzhong, ZHANGBinbin, ZHANGMengyan, et al. Multi-UAV Collaborative Path Planning Method Based on Attention Mechanism[J]. Mathematical Problems in Engineering, 2021, 2021: 6964875.
[34]
RIEDMILLERM, HAFNERR, LAMPET, et al. Learning by Playing—Solving Sparse Reward Tasks from Scratch[C]∥Proceedings of the 35th International Conference on Machine Learning (ICML). Stockholmsmässan, 2018: 4344-4353.
CAIZe, HUYaoguang, WENJingqian, et al. Collision Avoidance for AGV Based on Deep Reinforcement Learning in Complex Dynamic Environment[J]. Computer Integrated Manufacturing Systems, 2023(1): 236-245.
[37]
SARTORETTIG, KERRJ, SHIYunfei, et al. PRIMAL: Pathfinding via Reinforcement and Imitation Multi-agent Learning[J]. IEEE Robotics and Automation Letters, 2019, 4(3): 2378-2385.
[38]
SCHULMANJ, WOLSKIF, DHARIWALP, et al. Proximal Policy Optimization Algorithms[PP/OL].V2.arXiv(2017-08-28).
[39]
CAOYu, YANGAng, LIUYang, et al. AGV Dispatching and Bidirectional Conflict-free Routing Problem in Automated Container Terminal[J]. Computers & Industrial Engineering, 2023, 184: 109611.