基于社团优化的动态网络 Transformer 表示学习

李天鹏 ,  郭翾 ,  张旺 ,  陶润国 ,  廖浩然 ,  王文俊

天津大学学报(自然科学与工程技术版) ›› 2026, Vol. 59 ›› Issue (11) : 1219 -1230.

PDF (1947KB)
天津大学学报(自然科学与工程技术版) ›› 2026, Vol. 59 ›› Issue (11) : 1219 -1230. DOI: 10.11784/tdxbz202507027

基于社团优化的动态网络 Transformer 表示学习

作者信息 +

Transformer-Based Dynamic Network Embedding via Community Optimization

Author information +
文章历史 +
PDF (1992K)

摘要

现有动态网络表示学习方法对动态网络时序演化的建模大多服从网络平滑性假设与隐马尔可夫性假设,然而现实世界的数据往往并不严格遵循上述假设,部分动态网络在时序演化过程中可能出现周期性依赖或长程依赖,进而降低了现有动态网络表示学习方法的有效性和泛用能力.为解决这一问题,本文提出了基于社团优化的Transformer 动态网络表示学习模型 CO-TDNE,该方法利用 Transformer 堆叠的注意力机制结合时间位置编码策略刻画高度可变的图结构和动态网络复杂的时间依赖关系,更加适应真实世界动态网络在时序维度的突变与噪声影响.另外为了缓解 Transformer 在大规模图数据上的高计算成本和有限可扩展性问题,通过在全局视角下识别稳定社团,并在社团内进行参数共享来提升效率. 这一策略在不影响模型效果的同时将训练速度提高了 30%~35%. 本文在 6 组真实世界数据对动态网络的隐马尔可夫性假设的有效性进行了实证验证,证明了现实世界的数据并不严格符合该假设. 在现实世界数据集上的新边链接预测与缺失边链接预测实验表明,CO-TDNE 对动态网络时序演化模式的建模以及拓扑结构的刻画优于对比方法. 所构建的大量消融实验也证明了本文所提出的全局稳定社团优化策略能够有效降低模型参数规模,并证明了基于网络层次结构优化策略的有效性.

Abstract

Existing methods for dynamic network representation learning typically rely on assumptions of network smoothness and first-order Markovian dynamics to model temporal evolution. However,real-world dynamic networks often violate these assumptions,exhibiting periodic or long-range dependencies that undermine the effectiveness and generalizability of current techniques. To overcome this limitation,we introduce a community-optimized Transformer-based dynamic network embedding(CO-TDNE) model. The CO-TDNE model employs stacked Transformer attention mechanisms combined with a temporal positional encoding strategy to effectively capture diverse graph structures and complex temporal dependencies,thereby accommodating abrupt changes and noise inherent in real-world dynamic networks. To address the high computational cost and scalability challenges of Transformer on large-scale graphs,we propose a novel optimization strategy that identifies stable communities from a global perspective and implements parameter sharing within these communities. This strategy improves training efficiency by 30%—35% while maintaining model performance. Empirical validation on six real-world datasets confirms that the first-order Markovian assumption does not strictly hold in practice. In addition,experiments on new link and missing link prediction tasks demonstrate that CO-TDNE outperforms baseline methods in modeling temporal evolution patterns and topological structures. Comprehensive ablation studies further demonstrate the efficiency of the proposed global stable community optimization strategy in reducing model parameter complexity and highlight the effectiveness of the community-based hierarchical network optimization approach.

关键词

动态网络 / 动态图 / 网络表示学习 / 动态链接预测

Key words

dynamic network / dynamic graph / network representation learning / dynamic link prediction

引用本文

引用格式 ▾
李天鹏,郭翾,张旺,陶润国,廖浩然,王文俊. 基于社团优化的动态网络 Transformer 表示学习[J]. 天津大学学报(自然科学与工程技术版), 2026, 59(11): 1219-1230 DOI:10.11784/tdxbz202507027

登录浏览全文

4963

注册一个新账户 忘记密码

参考文献

[1]

Guo X, Li J, Jiao P F, et al. Counterfactual learning for higher—order relation prediction in heterogeneous information networks[J]. Neural Networks, 2025, 183: 107024.

[2]

Zhang Q R, Yang P, Yu J L, et al. A survey on point—of—interest recommendation:Models,architectures,and security[J]. IEEE Transactions on Knowledge and Data Engineering, 2025, 37(6): 3153-3172.

[3]

Goyal P, Chhetri S R, Canedo A. Dyngraph2vec:Capturing network dynamics using dynamic graph representation learning[J]. Knowledge—Based Systems, 2020, 187(1): 104816.

[4]

Hajiramezanali E, Hasanzadeh A, Narayanan K, et al. Variational graph recurrent neural networks[C]// Proceedings of the Advances in Neural Information Processing Systems. Vancouver,Canada, 2019, 32: 1-11.

[5]

Karita S, Chen N X, Hayashi T, et al. A comparative study on transformer vs RNN in speech applications [C]// Proceedings of the 2019 IEEE Automatic Speech Recognition and Understanding Workshop. Singapore, 2019: 449-456.

[6]

Sankar A, Wu Y H, Gou L, et al. Dysat:Deep neural representation learning on dynamic graphs via self—attention networks[C]// Proceedings of the 13th International Conference on Web Search and Data Mining. Houston,USA, 2020: 519-527.

[7]

Cong W L, Wu Y H, Tian Y D, et al. DyFormer:A scalable dynamic graph transformer with provable benefits on generalization ability[C]// Proceedings of the 2023 SIAM International Conference on Data Mining. Paul Twin Cities,USA, 2023: 442-450.

[8]

Zhou H Y, Zhang S H, Peng J Q, et al. Informer:Beyond efficient transformer for long sequence time—series forecasting[C]// Proceedings of the AAAI Conference on Artificial Intelligence. Austin,USA, 2021, 35(12): 11106-11115.

[9]

Goyal P, Nitin N, He X R, et al. DynGEM:Deep embedding method for dynamic graphs[EB/OL]. https://arxiv.org/abs/1805.11273, 2018—05—29.

[10]

Kumar M, Mishra S, Singh S S, et al. Community—enhanced link prediction in dynamic networks[J]. ACM Transactions on the Web, 2024, 18(2): 1-32.

[11]

He X, Ghasemian A, Lee E, et al. Sequential stacking link prediction algorithms for temporal networks[J]. Nature Communications, 2024, 15: 1364.

[12]

Gao C, Zhu J Y, Zhang F, et al. A novel representation learning for dynamic graphs based on graph convolutional networks[J]. IEEE Transactions on Cybernetics, 2023, 53(6): 3599-3612.

[13]

Bai Q J, Nie C L, Zhang H W, et al. HGWave Net:A hyperbolic graph neural network for temporal link prediction[C]// Proceedings of the ACM Web Conference. Austin,USA, 2023: 523-532.

[14]

Sun L, Ye J D, Peng H, et al. A self—supervised Riemannian GNN with time varying curvature for temporal graph learning[C]//Proceedings of the 31st ACM International Conference on Information & Knowledge Management. Atlanta,USA, 2022: 1827-1836.

[15]

Bei Y C, Xu H, Zhou S, et al. CPDG:A contrastive pre—training method for dynamic graph neural networks [C]// Proceedings of the IEEE 40th International Conference on Data Engineering. Utrecht , Netherlands , 2024: 1199-1212.

[16]

Benson A R, Abebe R, Schaub M T, et al. Simplicial closure and higher—order link prediction[J]. Proceedings of the National Academy of Sciences, 2018, 115(48): 11221-11230.

[17]

Rossi R. Ahmed N. The network data repository with interactive graph analytics and visualization[C]// Proceedings of the AAAI Conference on Artificial Intelligence. Austin,USA, 2015: 4292-4293.

基金资助

国家自然科学基金资助项目(6247072183)

AI Summary AI Mindmap
PDF (1947KB)

0

访问

0

被引

详细

导航
相关文章

AI思维导图

/

〈 〉