结合标签语义和对比学习的小样本文本分类

Few-shot Text Classification with Label Semantics and Contrastive Learning

摘要

元学习在小样本文本分类中表现出良好效果,其中的原型网络通过计算查询样本与类别原型之间的距离进行分类.然而,由于较大的类内差异和较小的类间差异,现有方法在构建准确的类别原型时常常受限.针对上述问题,本文提出一种小样本文本分类框架,该框架通过标签语义融合增强特征表示,并引入基于三元组损失的对比学习机制来优化特征分布.首先,通过注意力机制将标签信息与样本特征融合,提升类内一致性;其次,利用查询样本和标签信息校准类别原型,缓解由随机采样支持集样本导致的原型估计偏差;最后,引入三元组损失优化策略压缩类内特征分布并增强类间判别性.在8个基准数据集上的实验表明,本文方法优于基线模型.

Abstract

The meta-learning has demonstrated significant effectiveness in few-shot text classification,among which Prototypical Networks achieves classification by computing the distance between prototypes and query samples.However,due to large intra-class variance and small inter-class differences,existing methods often struggle to construct accurate class prototypes.To address this issue,this paper proposes a novel few-shot text classification framework that enhances feature representations through label semantics fusion and optimizes feature distribution via a triplet loss-based contrastive learning mechanism.Specifically,an attention mechanism is employed to integrate label information with sample features,improving intra-class consistency.Then,query samples and label semantics are utilized to calibrate class prototypes,mitigating estimation bias caused by randomly sampled support sets.Finally,a triplet loss optimization strategy is introduced to compress intra-class feature distribution and enhance inter-class discriminability.Experiments on eight benchmark datasets demonstrate that the proposed method outperforms baseline models.

关键词

小样本学习 / 元学习 / 原型网络 / 文本分类 / 自然语言处理

Key words

few-shot learning / meta-learning / prototypical networks / text classification / natural language processing

[1] Han C,Fan Z,Zhang D,et al.Meta-learning adversarial domain adaptation network for few-shot text classification[C]//Findings of the Association for Computational Linguistics:ACL-IJCNLP,2021:1664-1673.
[2] Dixit M,Kwitt R,Niethammer M,et al.Aga:attribute-guided augmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2017:7455-7463.
[3] Gao T,Fisch A,Chen D.Making pre-trained language models better few-shot learners[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing,2021:3816-3830.
[4] Dong H,Zhang W,Che W.Metricprompt:prompting model as a relevance metric for few-shot text classification[C]//Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining,2023:426-436.
[5] Hu S,Ding N,Wang H,et al.Knowledgeable prompt-tuning:incorporating knowledge into prompt verbalizer for text classification[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics,2022:2225-2240.
[6] Finn C,Abbeel P,Levine S.Model-agnostic meta-learning for fast adaptation of deep networks[C]//International Conference on Machine Learning,PMLR,2017:1126-1135.
[7] Snell J,Swersky K,Zemel R.Prototypical networks for few-shot learning[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems,2017:4080-4090.
[8] Lei S,Zhang X,He J,et al.TART:improved few-shot text classification using task-adaptive reference transformation[C]//Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics,2023:11014-11026.
[9] Chen J,Zhang R,Mao Y,et al.Contrastnet:a contrastive learning framework for few-shot text classification[C]//Proceedings of the AAAI Conference on Artificial Intelligence,2022:10492-10500.
[10] Han C,Wang Y,Fu Y,et al.Meta-learning siamese network for few-shot text classification[C]//International Conference on Database Systems for Advanced Applications,2023:737-752.
[11] Liang W,Zhang T,Liu H,et al.SELP:a semantically-driven approach for separated and accurate class prototypes in few-shot text classification[C]//Findings of the Association for Computational Linguistics,2024:9732-9741.
[12] Liu X,Gao Y,Zong L,et al.Improve meta-learning for few-shot text classification with all you can acquire from the tasks[C]//Findings of the Association for Computational Linguistics:EMNLP,2024:223-235.
[13] Rusu A A,Rao D,Sygnowski J,et al.Meta-learning with latent embedding optimization[C]//International Conference on Learning Representations,2019.
[14] Lei T,Hu H,Luo Q,et al.Adaptive meta-learner via gradient similarity for few-shot text classification[C]//Proceedings of the 29th International Conference on Computational Linguistics,2022:4873-4882.
[15] Koch G,Zemel R,Salakhutdinov R.Siamese neural networks for one-shot image recognition[C]//ICML Deep Learning Workshop,2015:1-30.
[16] Vinyals O,Blundell C,Lillicrap T,et al.Matching networks for one shot learning[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems,2016:3637-3645.
[17] Liu H,Zhang F,Zhang X,et al.Boosting few-shot text classification via distribution estimation[C]//Proceedings of the AAAI Conference on Artificial Intelligence,2023:13219-13227.
[18] Lee J H,Hahn J,Seo H T,et al.SuperST:superficial self-training for few-shot text classification[C]//Proceedings of the Joint International Conference on Computational Linguistics,Language Resources and Evaluation(LREC-COLING),2024:15436-15447.
[19] Wang Y,Li C.Inter-class distance enhanced prototypical network for few-shot text classification[J].Multimedia Systems,2025,31(3):185,doi:10.1007/s00530-025-01770-0.
[20] Schroff F,Kalenichenko D,Philbin J.Facenet:a unified embedding for face recognition and clustering[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2015:815-823.
[21] Dong K,Jiang B,Li H,et al.Meta-learning triplet contrast network for few-shot text classification[J].Knowledge-Based Systems,2024,303:112440,doi:10.1016/j.knosys.2024.112440.
[22] Kendall A,Gal Y,Cipolla R.Multi-task learning using uncertainty to weigh losses for scene geometry and semantics[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2018:7482-7491.
[23] Dopierre T,Gravier C,Logerais W.ProtAugment:intent detection meta-learning through unsupervised diverse paraphrasing[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing,Association for Computational Linguistics,2021:2454-2466.
[24] Cui G,Hu S,Ding N,et al.Prototypical verbalizer for prompt-based few-shot tuning[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics,2022:7014-7024.

基金资助

安徽省重点研究与开发计划项目(202004d07020009)资助.