基于噪声标签先验挖掘的弱监督图像分类方法

Weakly Supervised Image Classification Method Based on Prior Mining of Noisy Labels

摘要

标签中含有噪声是弱监督图像分类中的典型情况之一. 传统图像分类模型通常易于过拟合噪声标签而损害性能，即使是噪声鲁棒方法，也常因其依赖于对噪声分布的特定先验知识而存在缺陷. 针对该问题，提出一种基于噪声标签先验挖掘的弱监督图像分类方法. 具体来说，为在未知噪声模式下挖掘先验知识，首先设计了一种基于双路变分自编码器与期望最大化算法的噪声先验挖掘模块，该模块通过一路变分自编码器压缩风格特征，解耦图像内容与噪声标签的强关联性；另一路结合期望最大化算法建模训练数据的噪声分布特性. 其次，基于挖掘到的噪声先验知识，设计一种自适应动态类平衡的样本选择方法，该方法通过构建动态阈值准确划分样本为干净样本和噪声样本. 此外，设计一种一致性正则化框架加权融合训练损失，该框架结合样本选择的结果，通过标签修正和权重评估等方法，有效协调干净样本和噪声样本对模型性能的贡献，优化模型的泛化能力. 在2个合成噪声数据集和3个真实世界数据集上进行实验对比与分析. 最终结果表明所提方法在不同噪声场景下均取得最优性能，尤其在合成噪声数据集CIFAR10N和CIFAR100N的80%对称噪声的极端噪声场景中，相较于DISC、SED等当前先进噪声鲁棒方法，分别实现了3.08%和6.17%的分类准确率提升，充分验证了方法的有效性与先进性.

Abstract

Noisy labels are prevalent in weakly supervised image classification，where they cause overfitting in conventional image classification models. Even noise-robust methods remain limited because of their reliance on rigid，pre-defined prior knowledge of noise distributions. To address this issue，a weakly supervised image classification method based on noisy label prior mining is proposed. Specifically，to uncover prior knowledge under unknown noise patterns，a noise prior mining module is developed using a dual-path variational autoencoder combined with the expectation-maximization(EM)algorithm. One path compresses style features to decouple the strong correlation between image content and noisy labels，whereas the other path，together with the EM algorithm，models and extracts the characteristics of the noise distribution in the training data. Subsequently，an adaptive dynamic class-balanced sample selection method is developed based on the mined noise prior. This method constructs dynamic thresholds to precisely partition samples into clean and noisy subsets. In addition，a consistency regularization framework is designed for weighted loss fusion. This framework integrates the sample selection results with techniques such as label correction and reweighting to effectively coordinate the contributions of clean and noisy samples to model performance and to enhance model generalization. Comparative experiments and analysis are conducted on two synthetically corrupted datasets(CIFAR10N and CIFAR100N)and three real-world datasets(Web-Aircraft，Web-Car and Web-Bird). The results demonstrate that the proposed method achieves optimal performance under various noise settings. In particular，under the extreme 80% symmetric noise scenario on the synthetically corrupted datasets CIFAR10N and CIFAR100N，compared with state-of-the-art noise-robust methods such as DISC and SED，the proposed method improves classification accuracy by 3.08% and 6.17%，respectively，fully validating its effectiveness and superiority.

关键词

图像分类 / 双路变分自编码器 / 噪声估计 / 噪声标签 / 弱监督学习

Key words

image classification / dual-path variational autoencoder / noise estimation / noisy label / weakly supervised learning

参考文献

[1] Englesson E, Azizpour H. Robust classification via regression for learning with noisy labels[C]// International Conference on Learning Representations. Vienna，Austria， 2024:56956-56976.

[2] 冀中, 吴伊兵, 王轩. 双路特征提取与度量的少样本细粒度图像分类方法[J]. 天津大学学报(自然科学与工程技术版), 2024, 57(2):137-146.

[3] Ji Zhong, Wu Yibing, Wang Xuan. Dual—path feature extraction and metrics for few—shot fine—grained image classification[J]. Journal of Tianjin University(Science and Technology), 2024, 57(2):137-146(in Chinese).

[4] Kirillov A, Mintun E, Ravi N, et al. Segment anything[C]// Proceedings of the IEEE International Conference on Computer Vision. Paris，France， 2023:4015-4026.

[5] Li W, Wang L M, Li W, et al. WebVision Database：Visual Learning and Understanding from Web Data[EB/OL]. https://arxiv.org/abs/1708.02862, 2017—08—09.

[6] Li S Y, Zheng Y X, Shi Y, et al. KD—crowd：A knowledge distillation framework for learning from crowds[J]. Frontiers of Computer Science, 2025, 19(1):191302.

[7] Zhang Y, Niu G, Sugiyama M. Learning noise transition matrix from only noisy labels via total variation regularization[C]// International Conference on Machine Learning. Jeju Island，Republic of Korea， 2021:12501-12511.

[8] Yao Y, Liu T L, Han B, et al. Dual T：Reducing estimation error for transition matrix in label—noise learning[J]. Advances in Neural Information Processing Systems, 2020, 33:7260-7271.

[9] Xia X B, Liu T L, Wang N N, et al. Are anchor points really indispensable in label—noise learning?[J]. Advances in Neural Information Processing Systems, 2019, 32:6835-6846.

[10] Li X F, Liu T L, Han B, et al. Provably end—to—end label—noise learning without anchor points[C]// International Conference on Machine Learning. Jeju Island，Republic of Korea， 2021:6403-6413.

[11] Que X F, Yu Q. Dual—level curriculum meta—learning for noisy few—shot learning tasks[C]// Proceedings of the AAAI Conference on Artificial Intelligence. Vancouver，Canada， 2024, 38(13):14740-14748.

[12] Zhang D, Hu R, Rundensteiner E. CoLafier：Collaborative noisy label purifier with local intrinsic dimensionality guidance[C]// Proceedings of the SIAM International Conference on Data Mining. Houston，USA， 2024:82-90.

[13] Han B, Yao Q M, Yu X R, et al. Co—teaching：Robust training of deep neural networks with extremely noisy labels[J]. Advances in Neural Information Processing Systems, 2018, 31:8536-8546.

[14] Yu X R, Han B, Yao J C, et al. How does disagreement help generalization against label corruption?[C]// International Conference on Machine Learning. Long Beach，USA， 2019:7164-7173.

[15] Sun Z R, Shen F M, Huang D, et al. PNP：Robust learning from noisy labels by probabilistic noise prediction[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. New Orleans，USA， 2022:5311-5320.

[16] 魏琦, 孙皓亮, 马玉玲, 等. 面向标签噪声的联合训练框架[J]. 中国科学：信息科学, 2024, 54(1):144-158.

[17] Wei Qi, Sun Haoliang, Ma Yuling, et al. A joint training framework for learning with noisy labels[J]. Scientia Sinica(Informationis), 2024, 54(1):144-158(in Chinese).

[18] Li J N, Socher R, Hoi S C H. Dividemix：Learning with noisy labels as semi—supervised learning[C]// International Conference on Learning Representations. Addis Ababa，Ethiopia， 2020:8923-8936.

[19] Arazo E, Ortego D, Albert P, et al. Pseudo—labeling and confirmation bias in deep semi—supervised learning[C]// International Joint Conference on Neural Networks. Glasgow，UK， 2020:1-8.

[20] Yang F, Wu K, Zhang S Y, et al. Class—aware contrastive semi—supervised learning[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. New Orleans，USA， 2022:14421-14430.

[21] Fang T T, Lu N, Niu G, et al. Rethinking importance weighting for deep learning under distribution shift[J]. Advances in Neural Information Processing Systems, 2020, 33:11996-12007.

[22] Wang S S, Wang B L, Zhang Z, et al. Class—aware sample reweighting optimal transport for multi—source domain adaptation[J]. Neurocomputing, 2023, 523:213-223.

[23] Tu Y P, Zhang B S, Li Y X, et al. Learning from noisy labels with decoupled meta label purifier[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Vancouver，Canada， 2023:19934-19943.

[24] Zhang H Y, Cisse M, Dauphin Y N, et al. Mixup：Beyond empirical risk minimization[C]// International Conference on Learning Representations. Vancouver，Canada， 2018:2866-2878.

[25] Wei J H, Liu H Y, Liu T L, et al. To smooth or not? when label smoothing meets noisy labels[C]// International Conference on Machine Learning. Baltimore，USA， 2022:23589-23614.

[26] Bai Y B, Yang E K, Han B, et al. Understanding and improving early stopping for learning with noisy labels[J]. Advances in Neural Information Processing Systems, 2021, 34:24392-24403.

[27] Yang H S, Yao Q M, Han B, et al. Searching to exploit memorization effect in deep learning with noisy labels[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024, 46(12):7833-7849.

[28] Yao Y Z, Sun Z R, Zhang C Y, et al. Jo—SRC：A contrastive approach for combating noisy labels[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Kuala Lumpur，Malaysia， 2021:5192-5201.

[29] Li Y F, Han H, Shan S G, et al. DISC：Learning from noisy labels via dynamic instance—specific selection and correction[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Vancouver，Canada， 2023:24070-24079.

[30] Sheng M M, Sun Z R, Chen T, et al. Foster adaptivity and balance in learning with noisy labels[C]// European Conference on Computer Vision. Milan，Italy， 2024:217-235.

[31] Ghosh A, Kumar H, Sastry P S. Robust loss functions under label noise for deep neural networks[C]// Proceedings of the AAAI Conference on Artificial Intelligence. San Francisco，USA， 2017:1919-1925.

[32] Zhang Z L, Sabuncu M R. Generalized cross entropy loss for training deep neural networks with noisy labels[J]. Advances in Neural Information Processing Systems, 2018, 31:8792-8802.

[33] Wang Y S, Ma X J, Chen Z Y, et al. Symmetric cross entropy for robust learning with noisy labels[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach，USA， 2019:322-330.

[34] Ma X J, Huang H X, Wang Y S, et al. Normalized loss functions for deep learning with noisy labels[C]// International Conference on Machine Learning. Vienna，Austria， 2020:6543-6553.

[35] Liu S, Niles—Weed J, Razavian N, et al. Early—learning regularization prevents memorization of noisy labels[J]. Advances in Neural Information Processing Systems, 2020, 33:20331-20342.

[36] Iscen A, Valmadre J, Arnab A, et al. Learning with neighbor consistency for noisy labels[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. New Orleans，USA， 2022:4672-4681.

[37] Zhang S, Li Y W, Wang Z Y, et al. Learning with noisy labels using hyperspherical margin weighting[C]// Proceedings of the AAAI Conference on Artificial Intelligence. Vancouver，Canada， 2024:16848-16856.

[38] Zhang H Y, Xing X M, Liu L. DualGraph：A graph—based method for reasoning about label noise[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Kuala Lumpur，Malaysia， 2021:9654-9663.

[39] Jang E, Gu S X, Poole B. Categorical reparameterization with gumbel—softmax[C]// International Conference on Learning Representations. Toulon，France， 2017:1920-1931.

[40] Liu Y, Cheng H, Zhang K. Identifiability of label noise transition matrix[C]// International Conference on Machine Learning. Hawaii，USA， 2023:21475-21496.

[41] Wei H X, Feng L, Chen X Y, et al. Combating noisy labels by agreement：A joint training method with co—regularization[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Seattle，USA， 2020:13726-13735.

[42] Grandvalet Y, Bengio Y. Semi—supervised learning by entropy minimization[J]. Advances in Neural Information Processing Systems, 2005, 17:529-536.

[43] Krizhevsky A, Hinton G. Learning multiple layers of features from tiny images[R]. Toronto：University of Toronto， 2009.

[44] Sun Z R, Yao Y Z, Wei X S, et al. Webly supervised fine—grained recognition：Benchmark datasets and an approach[C]// Proceedings of the IEEE International Conference on Computer Vision. Montreal，Canada， 2021:10602-10611.

[45] Sun Z R, Liu H F, Wang Q, et al. Co—LDL：A co—training—based label distribution learning method for tackling label noise[J]. IEEE Transactions on Multimedia, 2021, 24:1093-1104.

[46] Karim N, Rizve M N, Rahnavard N, et al. UNICON：combating label noise through uniform selection and contrastive learning[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. New Orleans，USA， 2022:9666-9676.

[47] Liu S, Zhu Z H, Qu Q, et al. Robust training under label noise by overparameterization[C]// International Conference on Machine Learning. Baltimore，USA， 2022:14153-14172.

[48] Shi X S, Guo Z H, Li K, et al. Self—paced resistance learning against overfitting on noisy labels[J]. Pattern Recognition, 2023, 134:109080-109094.

[49] Zhou X, Liu X M, Zhai D M, et al. Asymmetric loss functions for noise—tolerant learning：Theory and applications[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(7):8094-8109.

基金资助

国家自然科学基金资助项目(62441235)

国家自然科学基金资助项目(62176178)