基于解耦自训练的不平衡节点分类

Decoupling Self-training for Imbalanced Node Classification

摘要

为了解决图神经网络(graph neural network,GNN)在不平衡数据集上对少数类样本分类精度低的问题,将解耦训练和自训练的思想相结合并引入 GNN 中,实现编码器和分类器的学习解耦。 在分类器的学习阶段,加入自适应标签软化和渐进式平衡采样学习策略,使少数类节点在前期也能加入网络训练的同时缓解训练带来的过拟合问题。 在编码器的学习阶段,通过自训练的思想,将分类器中较高置信度的少数类样本扩充到训练集中以提高少数类样本的多样性。 同时,采用最坏情况估计的策略来反向学习更具区分性的节点表示。 在 11 个数据集上的实验结果表明,该方法优于目前处理不平衡节点分类问题的方法,并与其他的编码器有很好的适配性。

Abstract

To solve the problem of low accuracy of graph neural network (GNN) classification of minority samples on imbalanced datasets, the combination of decoupling training and self-training is introduced into GNN, decoupling the learning of encoder and classifier. In the learning stage of the classifier, adaptive label softening and progressive balanced sampling learning strategies were added, so that minority nodes could be added to the network training in the early stage and alleviated the overfitting problem. In the learning stage of the encoder, the minority samples with higher confidence in the classifier were expanded into the training set to improve the sample diversity of the minority through the idea of self-training. At the same time, a worst-case estimation strategy was adopted to inversely learn more discriminative node representations. Experimental results on 11 datasets show that the proposed method outperforms the existing methods for dealing with the problem of imbalanced node classification, and has good adaptability with other encoders.

关键词

图节点分类 / 类不平衡 / 图神经网络(graph neural network,GNN) / 过采样 / 解耦训练 / 自训练

Key words

graph node classification / class imbalance / graph neural network (GNN) / oversampling / decoupling training / self-training

参考文献

[1] ZHANG Y F, GAO H C, PEI J, et al. Robust self—supervised structural graph neural network for social network prediction[C]// Proceedings of the ACM Web Conference 2022. New York: ACM, 2022: 1352-1361.

[2] LU M L, DAI Y L, ZHANG Z Q . Social network alignment: a bi—layer graph attention neural networks based method[J]. Applied Intelligence, 2022, 52(14): 16310-16333.

[3] YANG H R, ZHAO X Y, LI Y C, et al. An empirical study towards prompt—tuning for graph contrastive pre—training in recommendations[C]// 37th Conference on Neural Information Processing Systems. Red Hook, NY: Curran Associates Inc., 2023: 1-16.

[4] WU S W, SUN F, ZHANG W T, et al. Graph neural networks in recommender systems: a survey[J]. ACM Computing Surveys, 2022, 55(5): 1-37.

[5] VATANI N, RAHMANI A M, JAVADI H H S . Personality—based and trust—aware products recommendation in social networks[J]. Applied Intelligence, 2023, 53(1): 879-903.

[6] YANG S, ZHANG Z Q, ZHOU J, et al. Financial risk analysis for SMEs with graph—based supply chain mining[C]// Proceedings of the Twenty—Ninth International Joint Conference on Artificial Intelligence. Menlo Park, CA: AAAI Press, 2021: 4661-4667.

[7] ZHANG S G, MA X, FANG Z, et al. Financial time series forecasting based on momentum—driven graph signal processing[J]. Applied Intelligence, 2023, 53(18): 20950-20966.

[8] LAZCANO A, HERRERA P J, MONGE M . A combined model based on recurrent neural networks and graph convolutional networks for financial time series forecasting[J]. Mathematics, 2023, 11(1): 224.

[9] CHEN W Q, CHEN L, XIE Y, et al. Multi—range attentive bicomponent graph convolutional network for traffic forecasting[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(4): 3529-3536.

[10] WAIKHOM L, PATGIRI R, SINGH L D . Dynamic temporal position observant graph neural network for traffic forecasting[J]. Applied Intelligence, 2023, 53(20): 23166-23178.

[11] LIU Y, AO X, QIN Z D, et al. Pick and choose: a GNN—based imbalanced learning approach for fraud detection[C]// Proceedings of the Web Conference 2021. New York: ACM, 2021: 3168-3177.

[12] WANG J H, GUO Y, WEN X X, et al. Improving graph—based label propagation algorithm with group partition for fraud detection[J]. Applied Intelligence, 2020, 50: 3291-3300.

[13] KIPF T N, WELLING M . Semi—supervised classification with graph convolutional networks[EB/OL]. [2023—12—08]. https://arxiv.org/abs/1609.02907v4.

[14] PARK J, SONG J, YANG E . GraphENS: neighbor—aware ego network synthesis for class—imbalanced node classification[C]// The Tenth International Conference on Learning Representations. Washington, D.C.: OpenReview, 2022: 1-20.

[15] HONG Y, HAN S, CHOI K, et al. Disentangling label distribution for long—tailed visual recognition[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2021: 6622-6632.

[16] JAPKOWICZ N, STEPHEN S . The class imbalance problem: a systematic study[J]. Intelligent Data Analysis, 2002, 6(5): 429-449.

[17] SHI M, TANG Y F, ZHU X Q, et al. Multi—class imbalanced graph convolutional network learning[C]// Proceedings of the Twenty—Ninth International Joint Conference on Artificial Intelligence. Menlo Park, CA: AAAI Press, 2020: 2879-2885.

[18] VERMA V, QU M, KAWAGUCHI K, et al. GraphMix: improved training of GNNs for semi—supervised learning[C]// The Thirty—Fifth AAAI Conference on Artificial Intelligence. Menlo Park, CA: AAAI Press, 2021: 10024-10032.

[19] KANG B Y, XIE S N, ROHRBACH M, et al. Decoupling representation and classifier for long—tailed recognition[EB/OL]. [2023—12—08]. https://arxiv.org/abs/1910.09217.

[20] ZHAO T X, ZHANG X, WANG S H . GraphSMOTE: imbalanced node classification on graphs with graph neural networks[C]// Proceedings of the 14th ACM International Conference on Web Search and Data Mining. New York: ACM, 2021: 833-841.

[21] QU L, ZHU H S, ZHENG R Q, et al. ImGAGN: imbalanced network embedding via generative adversarial graph networks[C]// Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. New York: ACM, 2021: 1390-1398.

[22] GILMER J, SCHOENHOLZ S S, RILEY P F, et al. Neural message passing for quantum chemistry[C]// International Conference on Machine Learning. New York: ACM, 2017: 1263-1272.

[23] SNELL J, SWERSKY K, ZEMEL R S . Prototypical networks for few—shot learning[C]. Neural Information Processing Systems. Red Hook, NY: Curran Associates Inc., 2017: 4080-4090.

[24] GUERRIERO S, CAPUTO B, MENSINK T . DeepNCM: deep nearest class mean classifiers[C]// The sixth International Conference on Learning Representations. Washington, D.C.: OpenReview, 2018: 1-4.

[25] ZHANG Y B, TOKMAKOV P, HEBERT M, et al. A study on action detection in the wild[EB/OL]. [2023—12—08]. https://arxiv.org/abs/1904.12993.

[26] REBUFFI S A, KOLESNIKOV A, SPERL G, et al. iCaRL: incremental classifier and representation learning[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2017: 5533-5542.

[27] HUANG C, LI Y N, LOY C C, et al. Learning deep representation for imbalanced classification[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2016: 5375-5384.

[28] WANG Y X, RAMANAN D, HEBERT M . Learning to model the tail[C]// Neural Information Processing Systems. Red Hook, NY: Curran Associates Inc., 2017: 7032-7042.

[29] CUI Y, SONG Y, SUN C, et al. Large scale fine—grained categorization and domain—specific transfer learning[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2018: 4109-4118.

[30] CAO K D, WEI C L, CAIDON A, et al. Learning imbalanced datasets with label—distribution—aware margin loss[C]// Neural Information Processing Systems. Red Hook, NY: Curran Associates Inc., 2019: 1567-1578.

[31] MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality[C]// Neural Information Processing Systems. Red Hook, NY: Curran Associates Inc., 2013: 3111-3119.

[32] MAHAJAN D, GIRSHICK R, RAMANATHAN V, et al. Exploring the limits of weakly supervised pretraining[C]// European Conference on Computer Vision. Cham: Springer, 2018: 185-201.

[33] DU L, WANG Y, SONG G J, et al. Dynamic network embedding: an extended approach for skip—gram based network embedding[C]// Proceedings of the Twenty—Seventh International Joint Conference on Artificial Intelligence. Menlo Park, CA: AAAI Press, 2018: 2086-2092.

[34] CHEN B X, JIANG J G, WANG X M, et al. Debiased self—training for semi—supervised learning[J]. Advances in Neural Information Processing Systems, 2022, 35: 32424-32437.

[35] YANG Z L, COHEN W W, SALAKHUTDINOV R . Revisiting semi—supervised learning with graph embeddings[C]// International Conference on Machine Learning. New York: ACM, 2016: 1-9.

[36] SHCHUR O, MUMME M, BOJCHEVSKI A, et al. Pitfalls of graph neural network evaluation[EB/OL]. [2023—12—08]. https://arxiv.org/abs/1811.05868.

[37] PEI H B, WEI B Z, CHANG K C, et al. Geom—GCN: geometric graph convolutional networks[EB/OL]. [2023—12—08]. https://arxiv.org/abs/2002.05287.

[38] LI W Z, WANG C D, XIONG H, et al. GraphSHA: synthesizing harder samples for class—imbalanced node classification[EB/OL]. [2023—07—14]. https://arxiv.org/abs/2306.09612.

[39] ZHOU M T, GONG Z G . GraphSR: a data augmentation algorithm for imbalanced node classification[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2023, 37(4): 4954-4962.

[40] VAN DER MAATEN L, HINTON G . Visualizing data using t—SNE[J]. Journal of Machine Learning Research, 2008, 9(86): 2579-2605.

基金资助

国家自然科学基金资助项目(62172023)