港口多模态知识图谱与大模型融合的驱动机制与路径设计

Driving mechanism and path design of integrating port multimodal knowledge graphs with large models

摘要

在港口智能化升级背景下，业务场景的动态复杂化与数据的多源异构化，暴露出当前港口智能系统在“感知－认知－决策”链路存在断点：知识图谱虽擅长符号推理，却难以处理多模态语义与动态知识融合；大模型虽具备强大的自然语言能力，却因可解释性不足与领域知识缺失，难以实现精准可靠的决策推理。这一“感知与认知分离、生成与推理割裂”的现状，使单一技术无法满足港口对高可靠、可解释、自适应智能决策的需求。融合多模态知识图谱与大模型，已成为推动港口智能系统迭代升级的关键路径。本文系统梳理了人工智能、知识图谱与大模型在港口智能化领域的研究现状，从产业需求与技术融合双重视角，剖析了多模态知识图谱与大模型融合的内在驱动机制，并搭建了层次化技术架构与功能体系。同时，结合港口工程实践与行业发展现状，梳理并探讨了二者实现深度融合面临的关键技术瓶颈，以期为相关理论研究与工程落地应用提供参考依据和实践借鉴。

Abstract

Under the background of port intelligence upgrades, the dynamic complexity of business scenarios and the multi-source heterogeneity of data have exposed gaps in the “perception-cognition-decision” chain of current port intelligence systems. Knowledge graphs excel at symbolic reasoning but struggle with handling multi-modal semantics and dynamic knowledge fusion. Large models possess powerful natural language capabilities but face challenges in achieving accurate and reliable decision-making reasoning due to insufficient interpretability and lack of domain knowledge. This situation of “separation of perception and cognition, and disconnection of generation and reasoning” makes it impossible for a single technology to meet the port’s demands for highly reliable, interpretable, and adaptive intelligent decision-making. Therefore, integrating multi-modal knowledge graphs and large models has become a key path for driving the evolution of port intelligence systems. This paper systematically reviewed the related work on artificial intelligence, knowledge graphs, and large models in port intelligence, analyzed the driving mechanism of their integration under the dual perspective of industrial demand and technology integration, and constructed hierarchical technical and functional frameworks. Meanwhile, based on port engineering practice and development status, this paper identified and discussed key technical challenges faced in achieving deep integration, aiming to provide theoretical basis and practical guidance for related research and engineering applications.

关键词

港口智能化 / 多模态知识图谱 / 大模型

Key words

port intelligentization / multimodal knowledge graphs / large models

参考文献

[1] WANG P, HU Q Y, MEI Q, et al. Intelligent port logistics: a spatio-temporal knowledge graph and AI-agent framework for berth allocation[J]. Advanced Engineering Informatics, 2025, 68: 103633.

[2] 薛桂香, 王辉, 周卫峰, 等. 基于知识图谱和时空扩散图卷积网络的港口交通流量预测[J]. 计算机应用, 2024, 44(9): 2952-2957.

[3] XUE G X, WANG H, ZHOU W F, et al. Port traffic flow prediction based on knowledge graph and spatio-temporal diffusion graph convolutional network[J]. Journal of Computer Applications, 2024, 44(9): 2952-2957. (in Chinese)

[4] ZHANG H, GUO J N, GUO S S, et al. An optimization study on dynamic berth allocation based on vessel arrival time prediction[J]. Ocean Engineering, 2025, 332: 121321.

[5] CHU Z, YAN R, WANG S A. Vessel turnaround time prediction: a machine learning approach[J]. Ocean & Coastal Management, 2024, 249: 107021.

[6] ZHANG Y Z, SU Z Q, LI J K, et al. Doing shipping well with predictions: machine learning-based port congestion analysis[J]. Maritime Policy & Management, 2026, 53(2): 263-284.

[7] KUO S Y, LIN P C, HUANG X R, et al. Cad-transformer: a CNN-transformer hybrid framework for automatic appearance defect classification of shipping containers[J]. IEEE Transactions on Instrumentation and Measurement, 2025, 74: 1-21.

[8] WANG H Q, YAN R, AU M H, et al. Federated learning for green shipping optimization and management[J]. Advanced Engineering Informatics, 2023, 56: 101994.

[9] JIN J H, MA M Y, JIN H, et al. Container terminal daily gate in and gate out forecasting using machine learning methods[J]. Transport Policy, 2023, 132: 163-174.

[10] LIU S L, HUANG L. Application of machine learning in port throughput prediction[C]// Fourth International Conference on Computer Vision, Application, and Algorithm (CVAA2024). [S.l.: s.n.], 2025, 13486: 691-696.

[11] HUANG H X, YAN Q L, YANG Y, et al. Spatial classification model of port facilities and energy reserve prediction based on deep learning for port management ― a case study of Ningbo[J]. Ocean & Coastal Management, 2024, 258: 107413.

[12] CUONG T N, KIM H S, LONG L N B, et al. Deep learning-enhanced quantum optimization for integrated job scheduling in container terminals[J]. Engineering Applications of Artificial Intelligence, 2025, 148: 110431.

[13] LV Y Q, ZOU M K, LI J, et al. Dynamic berth allocation under uncertainties based on deep reinforcement learning towards resilient ports[J]. Ocean & Coastal Management, 2024, 252: 107113.

[14] LI R F, ZHANG X Y, JIANG L L, et al. An adaptive heuristic algorithm based on reinforcement learning for ship scheduling optimization problem[J]. Ocean & Coastal Management, 2022, 230: 106375.

[15] LEE W H, CHO S W. Reinforcement learning approach for outbound container stacking in container terminals[J]. Computers & Industrial Engineering, 2025, 204: 111069.

[16] RUBÍ B, CACACE J, RODRIGUEZ J, et al. VESSELimg: a large UAV-based vessel image dataset for port surveillance[C]// 2024 International Conference on Unmanned Aircraft Systems (ICUAS). Chania: IEEE, 2024: 76-83.

[17] TIEN I N B W, CUI G, NANAYAT, et al. FedRSnet: a federated learning approach for collaborative multi-modal maritime analytics[C]// 2024 27th International Conference on Information Fusion (FUSION). Venice: IEEE, 2024. doi: 10.23919/FUSION59988.2024.10706321.

[18] MORANDO E, DAFFINÀ F C, STAHL T, et al. Multi-sensor data fusion to enhance maritime situational awareness[C]// 2023 IEEE International Geoscience and Remote Sensing Symposium. Pasadena: IEEE, 2023: 6829-6831.

[19] GUPTA S, SAINI N, KUNDU S, et al. Synergizing vision and language in remote sensing: a multimodal approach for enhanced disaster classification in emergency response systems[C]// 2024 IEEE International Geoscience and Remote Sensing Symposium. Athens: IEEE, 2024: 3278-3281.

[20] YANG W Q, BAO X Y, ZHENG Y, et al. A digital twin framework for large comprehensive ports and a case study of Qingdao Port[J]. The International Journal of Advanced Manufacturing Technology, 2024, 131(11): 5571-5588.

[21] KLAR R, FREDRIKSSON A, ANGELAXIS V. Digital twins for ports: derived from smart city and supply chain twinning experience[J]. IEEE Access, 2023, 11: 71777-71799.

[22] ZHANG X Y, LIU C Y, XU Y, et al. A knowledge graph-based inspection items recommendation method for port state control inspection of LNG carriers[J]. Ocean Engineering, 2024, 313: 119434.

[23] ZHU R C, HU X F, BAI Y P, et al. Risk analysis of terrorist attacks on LNG storage tanks at ports[J]. Safety Science, 2021, 137: 105192.

[24] LIU S, WANG W Y, ZHONG S P, et al. A graph-based approach for integrating massive data in container terminals with application to scheduling problem[J]. International Journal of Production Research, 2024, 62(16): 5945-5965.

[25] 尹秀腾, 李响, 刘国辉, 等. 面向港口突发事件的多层网络知识图谱构建[J]. 信息工程大学学报, 2025, 26(2): 189-195.

[26] YIN X T, LI X, LIU G H, et al. Construction of a multi-layer network knowledge graph for port emergency events[J]. Journal of Information Engineering University, 2025, 26(2): 189-195. (in Chinese)

[27] 崔明月, 白晓勇, 王清哲. 基于知识图谱的航海信息个性化推荐应用研究[J]. 舰船科学技术, 2024, 46(4): 152-157.

[28] CUI M Y, BAI X Y, WANG Q Z. Research on personalized recommendation of navigation information based on knowledge graph[J]. Ship Science and Technology, 2024, 46(4): 152-157. (in Chinese)

[29] CHEN N Y, YANG A R, WU H, et al. SEMINT: an LLM-empowered long-term vessel trajectory prediction framework[J]. International Journal of Geographical Information Science, 2025, 39(9): 1938-1972.

[30] DIN M U, AKRAM W, BAKHTA B, et al. Maritime mission planning for unmanned surface vessel using large language model[C]// 2025 IEEE International Conference on Simulation, Modeling, and Programming for Autonomous Robots (SIMPAR). Palermo: IEEE, 2025: 1-6.

[31] 娄云洁, 艾明飞, 庄术洁, 等. DK-Port: 基于大语言模型和强化学习的港口自动驾驶仿真环境构建与验证[J]. 智能科学与技术学报, 2025, 7(1): 98-113.

[32] LOU Y J, AI M F, ZHUANG S J, et al. DK-Port: construction and validation of port autonomous driving simulation environment based on large language models and reinforcement learning[J]. Chinese Journal of Intelligent Science and Technology, 2025, 7(1): 98-113. (in Chinese)

[33] LV C C, SONG N, NIE J, et al. MS-LIP: Multiscale semantic information integration with large language models for marine prediction[J]. IEEE Transactions on Geoscience and Remote Sensing, 2025, 63: 1-15.

[34] LAN L, WANG F X, ZHENG X T, et al. Efficient prompt tuning of large vision-language model for fine-grained ship classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2025, 63: 1-10.

[35] 郭漩, 李博浩, 虞鹃秀, 等. 大语言模型增强的海运网络社团发现可视分析方法[J/OL]. 计算机辅助设计与图形学学报, 2025. (2025-06-20)[2025-08-19]. https://link.cnki.net/urlid/11.2925.TP.20250620.0902.002.

[36] GUO X, LI B H, YU J X, et al. Large language model enhanced visual analysis method for community detection in maritime networks[J/OL]. Journal of Computer-Aided Design & Computer Graphics. 2025. (2025-06-20)[2025-08-19]. https://link.cnki.net/urlid/11.2925.TP.20250620.0902.002. (in Chinese)

[37] KIM Y, LEE K, HAN Y. Exploring large language models (LLMs)-based chatbots for pump maintenance in ships[J]. Ships and Offshore Structures, 2025: 1-19.

[38] 杨玉林, 胡伟, 贾晓, 等. 基于大模型技术的港口设备运维助手智能体系统[J]. 港口科技, 2025(2): 1-5.

[39] YANG Y L, HU W, JIA X, et al. An intelligent-agent system for port equipment O&M based on large-model technology[J]. Port Science & Technology, 2025(2): 1-5. (in Chinese)

基金资助

国家自然科学基金航运创新联合基金项目(U2572201)