For the problem that emerging entities appear in the testing phase, and tail entity structural neighborhoods are missing in inductive multimodal knowledge graph completion, an inductive multimodal knowledge graph completion method based on knowledge graph enhancement and retrieval re-ranking was proposed. First, a vision-language model was utilized to combine tail entity text descriptions and image information to generate entity types, descriptive mentions, and pseudo triples with the tail entity as the head entity, achieving tail entity knowledge enhancement. Second, a query encoder and an entity encoder were designed, and text-visual semantic alignment was achieved through unified cross-modal contrastive learning. Third, a knowledge store and an entity store were established, and dual-path retrieval reasoning was completed by combining global entity distribution and semantic neighbor distribution. Finally, a large language model was introduced to filter candidate entity-related triples and make re-ranking decisions. The constructed framework balances multimodal representation alignment, prior knowledge memory, and evidence-driven re-ranking under unseen entity scenarios and provides a feasible inductive reasoning path for multimodal knowledge graph completion.
近年来,大型语言模型(large language model,LLM)和视觉语言模型(vision-language model,VLM)展现出较强的语义理解能力、世界知识掌握能力和生成推理能力,为KG构建与补全提供了新思路[10-11].相关方法可通过提示工程(Prompting)或微调来生成候选答案或三元组[12].同时,对比学习能够提高表示空间的判别性[13];检索增强生成(retrieval-augmented generation,RAG)和开放域问答研究表明,外部知识检索有助于缓解幻觉并提升推理质量[14-15].
在此基础上,KGC研究逐渐由“直接生成答案”转向“结构增强-上下文增强-检索推理-候选重排”的协同范式.CATS利用类型约束、子图路径和邻域事实辅助LLM判断候选三元组合理性[16];KGR3通过检索、推理和重排提升长尾实体与复杂关系预测效果[17];ReranKGC采用粗粒度检索与细粒度重排,挖掘文本和视觉属性中的多模态语义[18].此外,FtG通过“先过滤、再生成”降低候选搜索难度和幻觉风险[19];SAT缓解自然语言空间与图结构空间的不一致[20];MusKGC利用多源知识增强和事实一致性验证提升开放世界KGC的可靠性[21].MgHiSal和Make your choice for MKGC则分别关注层次化语义对齐与模态选择融合问题[22-23].这些研究表明,归纳、多模态和开放世界KGC正逐步走向“知识增强-候选检索-语义推理-精细重排”的协同框架.
JiS X, PanS R, CambriaE, et al. A survey on knowledge graphs: representation, acquisition, and applications[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33(2): 494-514.
[2]
PanS R, LuoL H, WangY F, et al. Unifying large language models and knowledge graphs: a roadmap[J]. IEEE Transactions on Knowledge and Data Engineering, 2024, 36(7): 3580-3599.
[3]
BordesA, UsunierN, Garcia-DuránA, et al. Translating embeddings for modeling multi-relational data[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems (NIPS).South Lake Tahoe:Curran Associates Inc.,2013: 2787-2795.
[4]
SunZ Q, DengZ H, NieJ Y, et al. RotatE: knowledge graph embedding by relational rotation in complex space[EB/OL]. (2019-02-19)[2026-01-05].
[5]
TeruK K, DenisE G, HamiltonW L. Inductive relation prediction by subgraph reasoning[C]//Proceedings of the 37th International Conference on Machine Learning. Online: PMLR,2020: 9448-9457.
[6]
SchlichtkrullM, KipfT N, BloemP, et al. Modeling relational data with graph convolutional networks[C]//The Semantic Web. Cham: Springer, 2018: 593-607.
ZhuX R, LiZ X, WangX D, et al. Multi-modal knowledge graph construction and application: a survey[J]. IEEE Transactions on Knowledge and Data Engineering, 2024, 36(2): 715-735.
WangX Z, GaoT Y, ZhuZ C, et al. KEPLER: a unified model for knowledge embedding and pre-trained language representation[J]. Transactions of the Association for Computational Linguistics, 2021,9: 176-194.
[12]
JiangZ B, XuF F, ArakiJ, et al. How can we know what language models know?[J]. Transactions of the Association for Computational Linguistics, 2020, 8: 423-438.
[13]
GaoT Y, YaoX C, ChenD Q. SimCSE: simple contrastive learning of sentence embeddings[C]//Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Online: Association for Computational Linguistics,2021: 6894-6910.
[14]
LewisP, PerezE, PiktusA, et al. Retrieval-augmented generation for knowledge-intensive NLP tasks[C]//Proceedings of the 34th International Conference on Neural Information Processing Systems. Online: Curran Associates Inc.,2020: 9459-9474.
[15]
IzacardG, GraveE. Leveraging passage retrieval with generative models for open domain question answering[C]//Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics. Online: Association for Computational Linguistics, 2021: 874-880.
[16]
LiM Z, YangC H, XuC J, et al. Context-aware inductive knowledge graph completion with latent type constraints and subgraph reasoning[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2025, 39(11): 12102-12111.
[17]
LiM Z, YangC H, XuC J, et al. Retrieval, reasoning, re-ranking: a context-enriched framework for knowledge graph completion[C]//Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). Albuquerque, 2025: 4349-4363.
[18]
GaoM, XieY T, ChenW, et al. ReranKGC: a cooperative retrieve-and-rerank framework for multi-modal knowledge graph completion[J]. Neural Networks, 2025, 188: 107467.
[19]
LiuB, ZhangJ H, LinF Q, et al. Filter-then-generate: large language models with structure-text adapter for knowledge graph completion[C]//Proceedings of the 31st International Conference on Computational Linguistics. Abu Dhabi,2025: 11181-11195.
[20]
LiuY, CaoY N, LinX X, et al. Enhancing large language model for knowledge graph completion via structure-aware alignment-tuning[C]//Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. Suzhou, 2025: 20970-20984.
[21]
SongX, LiuH Y, WangH Y, et al. MusKGC: a flexible multi-source knowledge enhancement framework for open-world knowledge graph completion[C]//Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. Suzhou, 2025: 10031-10049.
XueY F, RenS Y, CaiW Q, et al. Make your choice for multimodal knowledge graph completion[J]. Knowledge-Based Systems, 2025, 323: 113750.
[24]
WangB, ShenT, LongG D, et al. Structure-augmented text representation learning for efficient knowledge graph completion[C]//Proceedings of the Web Conference 2021. Ljubljana, 2021: 1737-1748.
[25]
WangP, XieX, WangX H, et al. Reasoning through memorization: nearest neighbor knowledge graph embeddings[C]//Natural Language Processing and Chinese Computing. Cham: Springer, 2023: 111-122.
[26]
MarkowitzE, BalasubramanianK, MirtaheriM, et al. StATIK: structure and text for inductive knowledge graph completion[C]//Findings of the Association for Computational Linguistics: NAACL 2022. Seattle, 2022: 604-615.
[27]
ZhaoY, ZhangY, ZhouB H, et al. Contrast then memorize: semantic neighbor retrieval-enhanced inductive multimodal knowledge graph completion[C]//Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. Washington DC,2024: 102-111.