基于语义-结构信息自适应融合的知识图谱补全模型

Knowledge Graph Completion Model Based on Semantic-structure Adaptive Fusion

摘要

现有的知识图谱补全方法依赖单一语义或结构信息,而融合模型又难平衡语义与结构信息,导致泛化能力和可解释性有限.本文旨在提升知识图谱补全的准确性与泛化能力,针对融合模型存在的问题,提出一种语义-结构自适应融合的知识图谱补全模型SSA-KGC(Semantic-Structure Adaptive Fusion for Knowledge Graph Completion).该模型通过预训练语言模型生成实体与关系的语义嵌入,结合拓扑结构提取实体邻居与路径特征,再利用动态权重融合策略实现两类信息的自适应整合;引入关系感知机制为不同关系分配专属权重矩阵,配合注意力机制聚焦关键特征,利用负采样策略构造多样化对比样本强化模型对相似实体的判别能力.在WN18RR、FB15k-237和UMLS数据集上的实验表明,SSA-KGC的各项评估指标均优于目前语义结构信息融合主流模型,在语义关联密集场景、通用领域及生物医学等专业领域中均表现优异,验证了其在平衡语义与结构信息、适配复杂关系模式上的有效性,为知识图谱补全提供了可靠解决方案.

Abstract

Existing knowledge graph completion methods rely on single semantic or structural information,while fusion models struggle to balance semantic and structural information,resulting in limited generalization ability and interpretability.This paper aims to improve the accuracy and generalization ability of knowledge graph completion.To address the issues existing in fusion models,it proposes a knowledge graph completion model with semantic-structure adaptive fusion,namely SSA-KGC (Semantic-Structure Adaptive Fusion for Knowledge Graph Completion).The model generates semantic embeddings of entities and relations using a pre-trained language model,extracts entity neighbor and path features by combining topological structures,and then realizes the adaptive integration of the two types of information through a dynamic weight fusion strategy.It introduces a relation-aware mechanism to assign exclusive weight matrices to different relations,collaborates with an attention mechanism to focus on key features,and employs a negative sampling strategy to construct diverse contrastive samples,thereby enhancing the model′s ability to distinguish similar entities.Experiments on the WN18RR,FB15k-237,and UMLS datasets demonstrate that SSA-KGC outperforms current mainstream models for semantic-structural information fusion in all evaluation metrics.It exhibits excellent performance in scenarios with dense semantic associations,general domains,and professional fields such as biomedicine.This verifies its effectiveness in balancing semantic and structural information and adapting to complex relationship patterns,providing a reliable solution for knowledge graph completion.

关键词

语义-结构融合 / 关系感知 / 注意力机制 / 负采样策略

Key words

semantic-structural fusion / relational perception / attention mechanism / negative sampling strategy

[1] Bordes A,Chopra S,Weston J.Question answering with subgraph embeddings[J].arxiv preprint arxiv:1406.3676,2014.
[2] Ma Y,Crook P A,Sarikaya R,et al.Knowledge graph inference for spoken dialog systems[C]//IEEE International Conference on Acoustics,Speech and Signal Processing,2015:5346-5350.
[3] Zhang F,Yuan N J,Lian D,et al.Collaborative knowledge base embedding for recommender systems[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining,2016:353-362.
[4] Li S,Huang Z,Cheng G,et al.Enriching documents with compact,representative,relevant knowledge graphs[C]//Proceedings of the 29th International Conference on International Joint Conferences on Artificial Intelligence,2021:1748-1754.
[5] Bollacker K,Evans C,Paritosh P,et al.Freebase:a collaboratively created graph database for structuring human knowledge[C]//Proceedings of the ACM SIGMOD International Conference on Management of Data,2008:1247-1250.
[6] Miller G A.WordNet:a lexical database for English[J].Communications of the ACM,1995,38(11):39-41.
[7] Mitchell T,Cohen W,Hruschka E,et al.Never-ending learning[J].Communications of the ACM,2018,61(5):103-115.
[8] Shen J,Wang C,Gong L,et al.Joint language semantic and structure embedding for knowledge graph completion[J].arxiv preprint arxiv:2209.08721,2022.
[9] Qiu C,Qian P,Wang C,et al.Joint pre-encoding representation and structure embedding for efficient and low-resource knowledge graph completion[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing,2024:15257-15269.
[10] Bordes A,Usunier N,Garcia Duran A,et al.Translating embeddings for modeling multi-relational data[C]//Advances in Neural Information Processing Systems,2013:2787-2795.
[11] Wang Z,Zhang J,Feng J,et al.Knowledge graph embedding by translating on hyperplanes[C]//Proceedings of the AAAI Conference on Artificial Intelligence,2014,doi:10.1609/aaai.v28i1.8870.
[12] Lin Y,Liu Z,Sun M,et al.Learning entity and relation embeddings for knowledge graph completion[C]//Proceedings of the AAAI Conference on Artificial Intelligence,2015:2181-2187,doi:10.1609/aaai.v29i1.9491.
[13] Ji G,He S,Xu L,et al.Knowledge graph embedding via dynamic map** matrix[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1:Long Papers),2015:687-696.
[14] Sun Z,Deng Z H,Nie J Y,et al.Rotate:knowledge graph embedding by relational rotation in complex space[J].arxiv preprint arxiv:1902.10197,2019.
[15] Nickel M,Tresp V,Kriegel H P.A three-way model for collective learning on multi-relational data[C]//International Conference on Machine Learning,2011:809-816.
[16] Yang B,Yih W,He X,et al.Embedding entities and relations for learning and inference in knowledge bases[J].arxiv preprint arxiv:1412.6575,2014.
[17] Nickel M,Rosasco L,Poggio T.Holographic embeddings of knowledge graphs[C]//Proceedings of the AAAI Conference on Artificial Intelligence,2016:1955-1961.
[18] Trouillon T,Welbl J,Riedel S,et al.Complex embeddings for simple link prediction[C]//International Conference on Machine Learning,2016:2071-2080.
[19] Dettmers T,Minervini P,Stenetorp P,et al.Convolutional 2D Knowledge Graph Embeddings[C]//Proceedings of the AAAI Conference on Artificial Intelligence,2018:1811-1818.
[20] Jiang X,Wang Q,Wang B.Adaptive convolution for multi-relational learning[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,Volume 1 (Long and Short Papers),2019:978-987.
[21] Vashishth S,Sanyal S,Nitin V,et al.Interacte:improving convolution-based knowledge graph embeddings by increasing feature interactions[C]//Proceedings of the AAAI Conference on Artificial Intelligence,2020:3009-3016.
[22] Liu X,Tan H,Chen Q,et al.Ragat:relation aware graph attention network for knowledge graph completion[J].IEEE Access,2021,9:20840-20849.
[23] Schlichtkrull M,Kipf T N,Bloem P,et al.Modeling relational data with graph convolutional networks[C]//European Semantic Web Conference,2018:593-607.
[24] Kipf T N.Semi-supervised classification with graph convolutional networks[J].arxiv preprint arxiv:1609.02907,2016.
[25] Zhu D.PRGNN:modeling high-order proximity with relational graph neural network for knowledge graph completion[J].Neurocomputing,2024,594:127857,doi:10.1016/j.neucom.2024.127857.
[26] Wang J,Zhang Z,He Q,et al.MMPGCN:multi-hop message passing graph convolutional network for knowledge graph completion[C]//IEEE International Conference on Systems,Man,and Cybernetics (SMC),2024:5379-5384.
[27] Lu S,Xu G,Sun S.SRE-KGC:a knowledge graph completion model based on hidden graph structure and relational semantic enhancement[C]//IEEE International Conference on Systems,Man,and Cybernetics,2024:553-558.
[28] Yao L,Mao C,Luo Y.KG-BERT:BERT for knowledge graph completion[J].arxiv preprint arxiv:1909.03193,2019.
[29] Kim B,Hong T,Ko Y,et al.Multi-task learning for knowledge graph completion with pre-trained language models[C]//Proceedings of the 28th International Conference on Computational Linguistics,2020:1737-1743.
[30] Li D,Zhu B,Yang S,et al.Multi-task pre-training language model for semantic network completion[J].ACM Transactions on Asian and Low-Resource Language Information Processing,2023,22(11):1-20.
[31] Wang L,Zhao W,Wei Z,et al.Simkgc:simple contrastive knowledge graph completion with pre-trained language models[J].arXiv preprint arXiv:2203.02167,2022.
[32] Clouatre L,Trempe P,Zouaq A,et al.MLMLM:link prediction with mean likelihood masked language model[J].arxiv preprint arxiv:2009.07058,2020.
[33] Wang X,Gao T,Zhu Z,et al.KEPLER:a unified model for knowledge embedding and pre-trained language representation[J].Transactions of the Association for Computational Linguistics,2021,9:176-194.
[34] Saxena A,Kochsiek A,Gemulla R.Sequence-to-sequence knowledge graph completion and question answering[J].arxiv preprint arxiv:2203.10321,2022.
[35] Qiao Z,Ye W,Yu D,et al.Improving knowledge graph completion with generative hard negative mining[C]//Findings of the Association for Computational Linguistics,2023:5866-5878.
[36] Zhang Z,Liu X,Zhang Y,et al.Pretrain-KGE:learning knowledge representation from pretrained language models[C]//Findings of the Association for Computational Linguistics,2020:259-266.
[37] Daza D,Cochez M,Groth P.Inductive entity representations from text via link prediction[C]//Proceedings of the Web Conference,2021:798-808.
[38] Youn J,Tagkopoulos I.Kglm:integrating knowledge graph structure in language models for link prediction[J].arxiv preprint arxiv:2211.02744,2022.
[39] Wei X,Zhang Y,Wang H.Joint semantic embedding with structural knowledge and entity description for knowledge representation learning[J].Neural Computing and Applications,2023,35(5):3883-3902.
[40] Wang J,Zhang Z,He Q,et al.MMPGCN:multi-hop message passing graph convolutional network for knowledge graph completion[C]//IEEE International Conference on Systems,Man,and Cybernetics (SMC),2024:5379-5384.
[41] Sabet M,Pajoohan M R,Moosavi M R.Representation learning of knowledge graphs with correlation-based methods[J].Information Sciences,2023,641:119043,doi:10.1016/j.ins.2023.119043.
[42] MA Z Y,GAO Y,ZHANG Q,et al.Research on knowledge graph completion model integrating semantic and structural information[J].Data Analysis and Knowledge Discovery,2024,8(4):39-49.
附中文参考文献:
[42] 马志远,高 颖,张 强,等.融合语义与结构信息的知识图谱补全模型研究[J].数据分析与知识发现,2024,8(4):39-49.

基金资助

上海交通大学医工交叉项目(YG2024QNB05)资助.