自适应降维引导下的多度量学习方法

Multi-metric Learning Method Guided by Adaptive Dimensionality Reduction

摘要

传统度量学习方法往往面临高维数据下的计算瓶颈,其常见的解决思路是引入主成分分析等降维方法作为独立的前置步骤.然而,这种两阶段设计导致降维过程与下游任务的度量目标相互脱节,成为制约模型泛化性能的关键因素.针对该问题,提出了一种自适应降维引导的多度量学习方法(MLM-ADR:A Multi-Metric Learning Method Guided by Adaptive Dimensionality Reduction).该方法在降维学习后的嵌入空间中学习多种局部度量,并通过保持其一致性来捕捉数据内在的复杂结构与多样性.由于判别特征在不同局部环境中往往保持稳定,该方法能够更精准地学习关键区分信息,从而实现降维与多度量学习的协同优化.此外,构建了一个简单有效的优化模型,其求解过程简单且高效.在多个公开数据集上的实验结果表明,所提出方法在多种指标下的性能均优于现有主流度量学习方法.

Abstract

Traditional metric learning methods often facecomputational bottlenecks when dealing with high-dimensional data.A common solution is to introducedimensionality reduction methods such as Principal Component Analysis(PCA) as an independent preprocessing step.However,this two-stagedesign disconnects thedimensionality reduction process from themetric objectives of downstream tasks,which becomes a key factor restricting thegeneralization performanceof themodel.To address this issue,a multi-metric learning method guided by adaptivedimensionality reduction(MLM-ADR:A Multi-Metric Learning Method Guided by Adaptive Dimensionality Reduction) is proposed.This method learns multiplelocal metrics in theembedded spaceafter dimensionality reduction learning,and captures theintrinsic complex structureand diversity of data by maintaining theconsistency of theselocal metrics.Sincediscriminativefeatures tend to remain stablein different local environments,themethod can learn key distinguishing information moreaccurately,thereby realizing thecollaborativeoptimization of dimensionality reduction and multi-metric learning.In addition,a simpleand effectiveoptimization model is constructed,and its solution process is simpleand efficient.Experimental results on multiplepublic datasets show that theperformanceof theproposed method outperforms existing mainstream metric learning methods under various evaluation metrics.

关键词

度量学习 / 多度量学习 / 局部度量一致性 / 自适应降维 / 协同优化

Key words

metric learning / multiplemetric learning / local metric consistency / adaptive dimensionality reduction / collaborative optimization

[1] Bengio Y,CourvilleA,Vincent P.Representation learning:a review and new perspectives[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2013,35(8):1798-1828.
[2] Xiao X,Cao S,Wang L,et al.Deep hashing imageretrieval based on hybrid neural network and optimized metric learning[J].Knowledge Based Systems,2024,284:111336-111347,doi:10.1016/j.knosys.2023.111336.
[3] Warburg F,Miani M,Brack S,et al.Bayesian metric learning for uncertainty quantification in imageretrieval[C]//Advances in Neural Information Processing Systems,2024:69178-69190.
[4] Zhang L,Li H,Liu R,et al.Quality guided metric learning for domain adaptation person re-identification[J].IEEE Transactions on Consumer Electronics,2024,70(3):6023- 6030.
[5] Xiong J,Lai J.Similarity metric learning for RGB-infrared group reidentification[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition,2023:13662-13671.
[6] Wang S.Self-supervised metric learning in multi-view data:a downstream task perspective[J].Journal of the American Statistical Association,2023,118(544):2454-2467.
[7] Mao Q,Ma S.Enhancing style-guided image-to-image translation via self-supervised metric learning[J].IEEE Transactions on Multimedia,2023,25(1):8511-8526.
[8] Avdiukhin D,Chatziafratis V,Makarychev K,et al.Approximation schemefor weighted metric clustering via Sherali-Adams[C]//AAAI Conference on Artificial Intelligence,2024:7926-7934.
[9] WU J G,WEI W,GUO X Y,et al.Metric learning guided weighted clustering ensemble[J].Journal of Chinese Computer Systems,2023,44(8):1607-1615.
[10] Ye H J,Zhan D C,Jiang Y,et al.What makes objects similar:a unified multi-metric learning approach[J].Advances in Neural Information Processing Systems,2019,41(5):1257-1270.
[11] Sanakoyeu A,Tschernezki V,Buchler U,et al.Divideand conquer the embedding space for metric learning[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition,2019:471-480.
[12] Ye H J,Zhan D C,Li N,et al.Learning multiplelocal metrics:Global consideration helps[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2019,42(7):1698-1712.
[13] Sanakoyeu A,MA P,Tschernezki V,et al.Improving deep metric learning by divideand conquer[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2021,44(11):8306-8320.
[14] Hu J,Lu J,Tan Y P.Sharableand individual multi-view metric learning[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,40(9):2281-2288.
[15] Guo X Y,Li L,Dang C Y,et al.Multiplemetric learning via local metric fusion[J].Information Sciences,2023,40(9):2281-2288.
[16] Bellet A,Amaury H,Marc S.Metric learning[M].Switzerland:Springer Nature,2015.
[17] Kaya M,BilgeH S.Deep metric learning:a survey[J].Symmetry,2019,11(9):1066,doi:10.3390/sym11091066.
[18] Weinberger K Q,Saul L K.Distancemetric learning for largemargin nearest neighbor classification[J].Journal of Machine Learning Research,2009,10(2):207-244.
[19] WANG X,GUO X Y,WEI W,et al.Metric learning algorithm with adversarial sample triples constraints[J].CAAI Transactions on Intelligent Systems,2021,16(1):30-37.
[20] Zuo W M,Wang F Q,Zhang D,et al.Distancemetric learning via iterated support vector machines[J].IEEE Transactions on Image Processing,2017,26(10):4937-4950.
[21] Bucher M,Herbin S,Jurie F.Hard negative mining for metric learning based zero-shot classification[C]//IEEE Conference on European Conference on Computer Vision,2016:524-531.
[22] Schroff F,Kalenichenko D,Philbin J.Facenet:a unified embedding for facerecognition and clustering[C]//IEEE Conference on Computer Vision and Pattern Recognition,2015:815-823.
[23] Xuan H,Stylianou A,Pless R.Improved embeddings with easy positive triplet mining[C]//IEEE Winter Conference on Applications of Computer Vision,2020:2474-2482.
[24] Cheng S,Gong C,Yang J,et al.Adversarial metric learning[C]//International Joint Conference on Artificial Intelligence,2018:2021-2027.
[25] Duan Y Q,Zheng W Z,Lin X D,et al.Deep adversarial metric learning[C]//IEEE Conference on Computer Vision and Pattern Recognition,2018:2780-2789.
[26] Lin X D,Duan Y Q,Dong Q Y,et al.Deep variational metric learning[C]//IEEE Conference on European Conference on Computer Vision,2018:689-704.
[27] Liu C,Yu H,Li B Y,et al.Noise-resistant deep metric learning with ranking-based instances election[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition,2021:6811-6820.
[28] Ruan Y B,Xiao Y S,Hao Z F,A nearest-neighbor search model for distance metric learning[J].Information Sciences,2021,552(4):261-277.
[29] Guo X Y,Wei W,Liang J Q,et al.Metric learning via perturbing hard-to-classify instances[J].Pattern Recognition,2022,132(12):108928-108938.
[30] Li D W,Tian Y J.Global and local metric learning via eigen vectors[J].Knowledge-Based Systems,2017,116(1):152-162.
[31] Harandi M,Salzmann M,Hartley R.Joint dimensionality reduction and metric learning:a geometric take[C]//International Conference on Machine Learning,2017:1404-1413.
[32] Li B,Fan Z T,Zhang X L,et al.Robust dimensionality reduction via featurespaceto feature space distance metric learning[J].Neural Networks,2019,112(4):1-14.
[33] Perrot M,Habrard A.Regressi vevirtual metric learning[C]//Advances in Neural Information Processing Systems,2015:1810-1818.
[34] Goldberger J,Hinton G E,Roweis S,et al.Neighbourhood components analysis[C]//Advances in Neural Information Processing Systems,2004:513-520.
[35] Shi Y,Bellet A,Sha F.Sparsecompositional metric learning[C]//AAAI Conference on Artificial Intelligence,2014:2078-2084.
[36] Zhan D C,Li M,Li Y F,et al.Learning instancespecific distances using metric propagation[C]//26th Annual International Conference on MachineLearning,2009:1225-1232.
[37] Fetaya E,Ullman S.Learning local invariant Mahalanobis distances[C]//International Conference on Machine Learning,2015:162-168.
[38] YeJ P,Zhao Z,Liu H.Adaptive distance metric learning for clustering[C]//IEEE Conference on Computer Vision and Pattern Recognition,2007:1-7.
[39] Noh Y K,Zhang B T,Lee D.Generative local metric learning for nearest neighbor classification[J].Advances in Neural Information Processing Systems,2018,40(1):106-118.
[40] Changpinyo S,Liu K,Sha F.Similarity component analysis[C]//Advances in Neural Information Processing Systems,2013:1511-1519.
[41] Chopra S,Hadsell R,LeCun Y.Learning a similarity metric discriminatively with application to faceverification[C]//IEEE Conference on Computer Vision and Pattern Recognition,2005:539-546.
[42] Hoffer E,Ailon N.Deep metric learning using triplet network[C]//Similarity-Based Pattern Recognition:3rd International Workshop,2015:84-92.
[43] Sohn K.Improved deep metric learning with multi-class n-pair loss objective[C]//Advances in Neural Information Processing Systems,2016:1857-1865.
[44] Oh H,Xiang Y,Jegelka S,et al.Deep metric learning via lifted structured feature embedding[C]//IEEE Conference on Computer Vision and Pattern Recognition,2016:4004-4012.
[45] Wang Y F,Liu P P,Lang Y J,et al.Learnable dynamic margin in deep metric learning[J].Pattern Recognition,2022,132(12):108961-108978.
[46] Qian Q,Shang L,Sun B G,et al.Soft-tripleloss:deep metric learning without triplet sampling[C]//IEEE Conference on International Conference on Computer Vision,2019:6450-6458.
[47] Movshovitz Y,Toshev A,Leung T K,et al.No fuss distancemetric learning using proxies[C]//IEEE Conference on International Conference on Computer Vision,2017:360-368.
[48] Wang X,Han X T,Huang W L,et al.Multi-similarity loss with general pair weighting for deep metric learning[C]//IEEE Conference on Computer Vision and Pattern Recognition,2019:5022-5030.
[49] Song K,Han J W,Cheng G,et al.Adaptive neighborhood metric learning[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2021,44(9):4591-4604.
[50] Nguyen B,Ferri F J,Morell C,et al.An efficient method for clustered multi-metric learning[J].Information Sciences,2019,471(1):149-163.
[51] Harandi M,Salzmann M,Hartley R.Joint dimensionality reduction and metric learning:a geometric take[C]//International Conference on Machine Learning,2017:1404-1413.
[52] Li B,Fan Z T,Zhang X L,et al.Robust dimensionality reduction via feature space to feature space distance metric learning[J].Neural Networks,2019,112(4):1-14.
[53] Van L,Hinton G.Visualizing data using t-SNE[J].Journal of Machine Learning Research,2008,9(11):2579-2605.
[54] Van L.Accelerating t-SNeusing tree-based algorithms[J].The Journal of Machine Learning Research,2014,15(1):3221-3245.
[55] Barshan E,Ghodsi A,Azimifar Z,et al.Supervised principal component analysis:visualization,classification and regression on subspaces and submanifolds[J].Pattern Recognition,2011,44(7):1357-1371.
[56] Pang Y W,Zhou B,NieF P.Simultaneously learning neighborship and projection matrix for supervised dimensionality reduction[J].IEEE Transactions on Neural Networks and Learning Systems,2019,30(9):2779-2793.
附中文参考文献:
[9] 吴建国,魏 巍,郭鑫垚,等.度量学习引导的加权聚类集成算法[J].小型微型计算机系统,2023,44(8):1607-1615.
[19] 王 鑫,郭鑫垚,魏 巍,等.对抗样本三元组约束的度量学习算法[J].智能系统学报,2021,16(1):30-37.

基金资助

国家自然科学基金青年基金项目(62406297)资助;山西省自然科学基金项目(202303021222083,202203021222048)资助.