基于图注意力神经网络的实体消歧方法

Entity disambiguation method based on graph attention networks

摘要

针对链接对象为存在半结构化数据的知识库,提出了一种基于图注意力神经网络的短文本实体指称消歧方法。通过信息抽取与融入关键词,将含有半结构化数据的知识库构建为全局知识图谱;同时基于Bert预训练模型对短文本中的实体指称项进行嵌入融合;使用图注意力神经网络对全局知识图谱中候选实体节点进行加权聚合表征,并计算实体指称项与各候选实体之间的相似度得分,实现实体消歧。在CCKS2019数据集上的实验结果表明,基于图注意力神网络的实体消歧模型有效提高了实体消歧效果。

Abstract

We propose an entity disambiguation method based on graph attention networks for semi-structured knowledge base data. First, a global knowledge graph is constructed from the semi-structured knowledge base, and the entity reference items are embedded by Bert pre-trained model meanwhile. Next, graph attention networks which leverages masked self-attention layers is applyed on candidate entity nodes of global knowledge graph to fetch a vector of node level. Furtherly, we compute similarity scores rank between the entity reference items and the candidate entity to complete the task of entity disambiguation. The experimental results on CCKS2019 dataset achieves state-of-the-art.

关键词

实体消歧 / 知识图谱 / 关键词提取 / 图注意力神经网络 / 自然语言处理

Key words

entity disambiguation / knowledge graph / keyword extraction / graph attention networks / natural language processing

参考文献

[1] 李天然, 刘明童, 张玉洁, 等 . 基于深度学习的实体链接研究综述[J]. 北京大学学报(自然科学版), 2021, 57(1): 91-98.

[2] LI Ziran, LIU Mingtong, ZHANG Yujie, et al. Review of entity linking research based on deep learning[J]. Journal of Peking University(Natural Science Edition), 2021, 57(1): 91-98.

[3] 刘峤, 李杨, 段宏, 等 . 知识图谱构建技术综述[J]. 计算机研究与发展, 2016, 53(3): 582-600.

[4] LIU Qiao, LI Yang, DUAN Hong, et al. Overview of knowledge graph construction technology[J]. Computer Research and Development, 2016, 53(3): 582-600.

[5] 段宗涛, 李菲, 陈柘. 实体消歧综述[J]. 控制与决策, 2021, 36(5): 1025-1039.

[6] DUAN Zongtao, LI Fei, CHEN Zhe. Overview of entity disambiguation[J]. Control and Decision, 2021, 36(5): 1025-1039.

[7] BOIŃSKIT, SZYMANŃSKIJ, DUDEKB, et al. NLP question answering using DBpedia and YAGO[J]. Vietnam Journal of Computer Science, 2020(3): 1-16.

[8] SINGHK, LYTRAI, RADHAKRISHNASA, et al. No one is perfect: analysing the performance of question answering components over the DBpedia knowledge graph[J]. Journal of Web Semantics, 2020, 65(1): 100594.

[9] HUANG Zhipeng, BOGDANC, REYNOLDC, et al. Entity—based query recommendation for long—tail queries[J]. ACM Transactions on Knowledge Discovery from Data, 2018, 12(6): 1-24.

[10] 张丹阳, 李楠, 陈翀. 实体链接技术研究述评[J]. 情报工程, 2020, 6(6): 45-55.

[11] ZHANG Danyang, LI Nan, CHEN Chong. Review of research on entity link technology[J]. Information Engineering, 2020, 6(6): 45-55.

[12] HUANGD, WANGJ. An approach on Chinese microblog entity linking combining baidu encyclopedia and word2vec[J]. Procedia Computer Science, 2017, 111: 37-45.

[13] GUO Zhaochen, BARBOSA Denilson. Robust named entity disambiguation with random walks[J]. Semantic Web, 2017, 9(11): 1-21.

[14] 武川, 陆伟. 基于上下文特征的短文本实体链接研究[J]. 情报科学, 2016(2): 144-147.

[15] WU Chuan, LU Wei. Research on short text entity linking based on context features[J]. Intelligence Science, 2016(2): 144-147.

[16] 谭咏梅, 王睿, 李茂林. 基于上下文信息和排序学习的实体链接方法[J]. 北京邮电大学学报, 2015(5): 33-36.

[17] TAN Yongmei, WANG Rui, LI Maolin. Entity linking method based on context information and ranking learning[J]. Journal of Beijing University of Posts and Telecommunications, 2015(5): 33-36.

[18] 昝红英, 吴泳钢, 贾玉祥, 等 . 基于多源知识的中文微博命名实体链接[J]. 山东大学学报(理学版), 2015(7): 9-16.

[19] ZAN Hongying, WU Yonggang, JIA Yuxiang, et al. Chinese Weibo named entity link based on multi—source knowledge[J]. Journal of Shandong University(Natural Science), 2015(7): 9-16.

[20] 周鹏程, 武川, 陆伟. 基于多知识库的短文本实体链接方法研究: 以Wikipedia和Freebase为例[J]. 现代图书情报技术, 2016(6): 1-11.

[21] ZHOU Pengcheng, WU Chuan, LU Wei. Research on short text entity linking method based on multiple knowledge bases: taking Wikipedia and Freebase as examples[J]. Modern Library and Information Technology, 2016(6): 1-11.

[22] SUN Chenchen, SHEN Derong, KOU Yue, et al. Topological features based entity disambiguation[J]. Journal of Computer Science and Technology, 2016, 31(5): 1053-1068.

[23] FRANCIS—LANDAUM, DURRETTG, KLEIND. Capturing semantic similarity for entity linking with convolutional neural networks[J/OL]. arXiv, 2016. https://arxiv.org/pdf/1604.00734.pdf.

[24] HUANGH, HECK L , JIH. Leveraging deep neural networks and knowledge graphs for entity disambiguation[J]. CoRR, 2015, abs/1504.07678.

[25] 张涛, 刘康, 赵军. 一种基于图模型的维基概念相似度计算方法及其在实体链接系统中的应用[J]. 中文信息学报, 2015, 29(2): 58-67.

[26] ZHANG Tao, LIU Kang, ZHAO Jun. A graph model based Wiki concept similarity calculation method and its application in entity link systems[J]. Chinese Journal of Information Technology, 2015, 29(2): 58-67.

[27] 周金, 朱永华, 张铁男, 等 . 基于图的联合特征实体链接方法[J]. 上海大学学报(自然科学版), 2020, 26(5): 747-755.

[28] ZHOU Jin, ZHU Yonghua, ZHANG Tienan, et al. Graph based joint feature entity linking method[J]. Journal of Shanghai University(Natural Science), 2020, 26(5): 747-755.

[29] 郭宇航, 秦兵, 刘挺, 等 . 实体链指技术研究进展[J]. 智能计算机与应用, 2014, 4(5): 9-13.

[30] GUO Yuhang, QIN Bing, LIU Ting, et al. Research progress in physical chain finger technology[J]. Intelligent Computers and Applications, 2014, 4(5): 9-13.

[31] MULANǴ IO, SINGHK, PRABHUC, et al. Evaluating the impact of knowledge graph context on entity disambiguation models[C]// Proceedings of the 29th ACM International Conference on Information & Knowledge Management. New York: ACM, 2020: 2157-2160.

[32] CETOLI A, BRAGAGLIA S, Ó HÁRNEAD Y, et al. A neural approach to entity linking on wikidata[C]// European Conference on Information Retrieval. Berlin: Springer, 2019: 78-86.

[33] 祁志卫, 王笳辉, 岳昆, 等 . 图嵌入方法与应用:研究综述[J]. 电子学报, 2020, 48(4): 808-818.

[34] QI Zhiwei, WANG Jiahui, YUE Kun, et al. Graph embedding methods and applications: research review[J] Journal of Electronics, 2020, 48(4): 808-818.

[35] WANGD, CUIP, ZHUW. Structural deep network embedding[C]// Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2016: 1225-1234.

[36] CAOS, LUW, XUQ. Deep neural networks for learning graph representations[C]// Proceedings of the AAAI Conference on Artificial Intelligence.[S.l.]: AAAI, 2016, 30(1).

[37] HAMILTONWL, YINGR, LESKOVECJ. Inductive representation learning on large graphs[J/OL]. arXiv, 2017. https://arxiv.org/pdf/1706.02216.pdf.

[38] KIPFTN, WELLINGM. Semi—supervised classification with graph convolutional networks[J/OL]. arXiv, 2016. https://arxiv.org/abs/1609.02907.

[39] VELIČKOVIĆ P, CUCURULL G, CASANOVA A, et al. Graph attention networks[J]. arXiv, 2017. https://arxiv.org/abs/1710.10903.

[40] 张玉帅, 赵欢, 李博. 基于BERT和BiLSTM的语义槽填充[J]. 计算机科学, 2021, 48(1): 247-252.

[41] ZHANG Yushuai, ZHAO Huan, LI Bo. Semantic slot filling based on BERT and BiLSTM[J]. Computer Science, 2021, 48(1): 247-252.

[42] DEVLINJ, CHANGMW, LEEK, et al. Bert: pre—training of deep bidirectional transformers for language understanding[J/OL]. arXiv, 2018. https://arxiv.org/abs/1810.04805v2.

[43] VASWANIA, SHAZEERN, PARMARN, et al. Attention is all you need[J/OL]. arXiv, 2017. https://arxiv.org/abs/1706.03762v3.

[44] 金燕, 黄杰. 基于信息熵与词长信息改进的TFIDF算法[J]. 浙江工业大学学报, 2021, 49(2): 203-209.

[45] JIN Yan, HUANG Jie. Improved TFIDF algorithm based on information entropy and word length information[J]. Journal of Zhejiang University of Technology, 2021, 49(2): 203-209.

[46] 张晟旗, 王元龙, 李茹, 等 . 基于局部注意力机制的中文短文本实体链接[J]. 计算机工程, 2021, 47(11): 77-83.

[47] ZHANG Shengqi, WANG Yuanlong, LI Ru, et al. Chinese short text entity link based on local attention mechanism[J]. Computer Engineering, 2021, 47(11): 77-83.

[48] 姚源林, 王树伟, 徐睿峰, 等 . 面向微博文本的情绪标注语料库构建[J]. 中文信息学报, 2014, 28(5): 83-91.

[49] YAO Yuanlin, WANG Shuwei, XU Ruifeng, et al. The construction of an emotion annotated corpus on microblog text[J]. Journal of Chinese Information Processing, 2014, 28(5): 83-91.

[50] LIS, ZHAOZ, HUR, et al. Analogical reasoning on Chinese morphological and semantic relations[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2018: 138-143.

[51] DEMSZKYD, MOVSHOVITZ—ATTIASD, KOJ, et al. GoEmotions: a dataset of fine—grained emotions[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2020: 4040-4054.

[52] KIMY. Convolutional neural networks for sentence classification[C]// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: ACL, 2014: 1746-1751.

[53] SCHUSTERM, PALIWALKK. Bidirectional recurrent neural networks[J]. IEEE Transactions on Signal Processing, 1997, 45(11): 2673-2681.

[54] VAN DER MAATENL, HINTONG. Visualizing data using t—SNE[J]. Journal of Machine Learning Research, 2008, 9(11): 2579-2605.

[55] SONGY, SHIS, LIJ, et al. Directional skip—gram: explicitly distinguishing left and right context for word embeddings[C]// Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics. Stroudsburg: ACL, 2018: 175-180.

[56] JOULINA, GRAVEE, BOJANOWSKIP, et al. Bag of tricks for efficient text classification[C]// Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics. Stroudsburg: ACL, 2017: 427-431.

[57] TANGD, WEIF, YANGN, et al. Learning sentiment—specific word embedding for Twitter sentiment classification[C]// Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2014: 1555-1565.

基金资助

国家重点研发计划资助项目(2018YFB1402905)

陕西省重点研发计划资助项目(2020GY-227)

陕西省重点研发计划资助项目(2020ZDLGY09-05)

陕西省技术创新引导专项基金(2022PT-49)