面向资源受限环境的渐进式多保真度神经架构搜索

Progressive Multi Fidelity Neural Architecture Search for Resource Constrained Environments

摘要

神经架构搜索(NAS)是深度学习自动化的关键技术,但其高昂的计算成本严重限制了实际应用.传统方法需要对每个候选架构进行完整训练,导致搜索过程耗时且资源密集.本文提出渐进式多保真度神经架构搜索方法(PMF-NAS),通过三阶段渐进策略实现高效架构搜索.PMF-NAS在全局探索阶段使用低保真度快速评估识别高潜力区域,在区域搜索阶段采用中等保真度在缩小的空间内细化搜索,在精细优化阶段对最优候选进行高保真度验证.该方法的核心是基于早期训练特征的性能预测器,能够准确预测架构最终性能,避免大量无效计算.同时引入自适应资源分配机制,根据架构潜力和不确定性动态调整评估投入.实验表明,PMF-NAS在单GPU环境下可在8~9小时内完成搜索,同时在多个数据集上达到最优或接近最优的准确率.文本为资源受限环境下的神经架构搜索提供了实用解决方案,降低了NAS技术的应用门槛,有望推动其在更广泛领域的应用.

Abstract

Neural Architecture Search(NAS)is a key technology for deep learning automation,but its high computational cost severely limits its practical applications.Traditional methods require complete training for each candidate architecture,resulting in a time-consuming and resource intensive search process.This article proposes a progressive multi fidelity neural architecture search method(PMF-NAS),which achieves efficient architecture search through a three-stage progressive strategy.PMF-NAS uses low fidelity to quickly evaluate and identify high potential areas during the global exploration phase,uses medium fidelity to refine the search within a reduced space during the area search phase,and performs high fidelity validation on the optimal candidate during the fine optimization phase.The core of this method is a performance predictor based on early training features,which can accurately predict the final performance of the architecture and avoid a large amount of ineffective computation.At the same time,an adaptive resource allocation mechanism is introduced to dynamically adjust the evaluation investment based on the potential and uncertainty of the architecture.Experiments have shown that PMF-NAS can complete searches in 8~9 hours in a single GPU environment,while achieving optimal or near optimal accuracy on multiple datasets.Text provides a practical solution for neural architecture search in resource constrained environments,reducing the application threshold of NAS technology and potentially promoting its application in a wider range of fields.

关键词

神经架构搜索 / 多保真度评估 / 渐进式搜索 / 性能预测 / 资源优化

Key words

neural architecture search / multi fidelity evaluation / progressive search / performance prediction / resource optimization

[1] Lecun Y,Bengio Y,Hinton G.Deep learning[J].Nature,2015,521(7553):436-444.
[2] He K,Zhang X,Ren S,et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2016:770-778.
[3] WU J H,LI K Y,CHEN L X,et al.A survey on neural architecture search techniques[J].Application Research of Computers,2025,42(1):11-18.
[4] Zoph B,Le Q V.Neural architecture search with reinforcement learning[J]arXiv preprint arXiv:1611.01578,2016.
[5] Salmani Pour Avval S,Eskue N D,Groves R M,et al.Systematic review on neural architecture search[J].Artificial Intelligence Review,2025,58(3):73,doi:10.1007/s10462-024-11058-w.
[6] Real E,Aggarwal A,Huang Y,et al.Regularized evolution for image classifier architecture search[C]//Proceedings of the AAAI Conference on Artificial Intelligence,2019:4780-4789.
[7] Poyser M,Breckon T P.Neural architecture search:a contemporary literature review for computer vision applications[J].Pattern Recognition,2024,147:110052.
[8] Salehin I,Islam M S,Saha P,et al.AutoML:a systematic review on automated machine learning with neural architecture search[J].Journal of Information and Intelligence,2024,2(1):52-81.
[9] Pham H,Guan M,Zoph B,et al.Efficient neural architecture search via parameters sharing[C]//International Conference on Machine Learning,2018:4095-4104.
[10] Guo Z,Zhang X,Mu H,et al.Single path one-shot neural architecture search with uniform sampling[C]//European Conference on Computer Vision,2020:544-560.
[11] Chu X,Zhang B,Xu R.FairNAS:rethinking evaluation fairness of weight sharing neural architecture search[C]//Proceedings of the IEEE International Conference on Computer Vision,2021:12239-12248.
[12] Liu H,Simonyan K,Yang Y.DARTS:differentiable architecture search[J].arXiv preprint arXiv:1806.09055,2018.
[13] Xu Y,Xie L,Zhang X,et al.PC-DARTS:partial channel connections for memory-efficient architecture search[J].arXiv preprint arXiv:1907.05737,2019.
[14] Dong X,Yang Y.Searching for a robust neural architecture in four gpu hours[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2019:1761-1770.
[15] YANG J,LIN Y J,ZHOU P.Differentiable graph neural architecture search based on relational features[J].Journal of Beijing University of Aeronautics and Astronautics,2025:1-17,doi:10.13700/j.bh.1001-5965-2024-0794.
[16] JIANG P C,XUE Y.Evolutionary neural architecture search method based on ranking score prediction[J].Chinese Journal of Computers,2024,47(11):2522-2535.
[17] Luo R,Tian F,Qin T,et al.Neural architecture optimization[C]//Advances in Neural Information Processing Systems,2018:7816-7827.
[18] Li G,Hoang D,Bhardwaj K,et al.Zero-shot neural architecture search:challenges,solutions,and opportunities[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2024,46(12):7618-7635.
[19] Han X,Xue Y,Wang Z,et al.SaDENAS:a self-adaptive differential evolution algorithm for neural architecture search[J].Swarm and Evolutionary Computation,2024,91:101736.
[20] Song X,Lv Z,Fan J,et al.Evolutionary multi-objective spiking neural architecture search for image classification[J].IEEE Transactions on Evolutionary Computation,2025,doi:10.1109/TEVC.2025.3528471.
[21] Cai H,Zhu L,Han S.ProxylessNAS:direct neural architecture search on target task and hardware[J].arXiv preprint arXiv:1812.00332,2018.
[22] Tan M,Chen B,Pang R,et al.MnasNet:platform-aware neural architecture search for mobile[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2019:2820-2828.
[23] Lupión M,Cruz N C,Ortigosa E M,et al.A holistic approach for resource-constrained neural network architecture search[J].Applied Soft Computing,2025:112832,doi:10.1016/j.asoc.2025.112832.
[24] Cai H,Gan C,Wang T,et al.Once-for-all:train one network and specialize it for efficient deployment[C]//International Conference on Learning Representations,2020,doi:10.48550/arXiv.1908.09791.
[25] Xu Z,Wu J.Contrastive meta-reinforcement learning for heterogeneous graph neural architecture search[J].Expert Systems with Applications,2025,260:125433.
[26] REN P Z,LIANG X D,CHANG X J,et al.Neural architecture search on temporal convolutions for complex action recognition[J].Journal of Computer Research and Development,2025,62(8):1862-1874.
[27] Salehin I,Islam M S,Saha P,et al.AutoML:a systematic review on automated machine learning with neural architecture search[J].Journal of Information and Intelligence,2024,2(1):52-81.
[28] Yu K,Sciuto C,Jaggi M,et al.Evaluating the search phase of neural architecture search[J].arXiv preprint arXiv:1902.08142,2019.
[29] Zela A,Siems J,Zimmer L,et al.Understanding and robustifying differentiable architecture search[J].arXiv preprint arXiv:1909.09656,2019.
[30] Ning X,Zheng Y,Zhao T,et al.A generic graph-based neural architecture encoding scheme for predictor-based NAS[C]//European Conference on Computer Vision,2021:189-204.
附中文参考文献:
[3] 武家辉,李科研,陈丽新,等.神经架构搜索技术研究综述[J].计算机应用研究,2025,42(1):11-18.
[15] 杨 军,蔺颖婕,周 鹏.基于关系特征的可微图神经架构搜索[J].北京航空航天大学学报,2025:1-17,doi:10.13700/j.bh.1001-5965.2024.0794.
[16] 蒋鹏程,薛 羽.基于排序得分预测的演化神经架构搜索方法[J].计算机学报,2024,47(11):2522-2535.
[26] 任鹏真,梁小丹,常晓军,等.基于时间卷积神经架构搜索的复杂动作识别[J].计算机研究与发展,2025,62(8):1862-1874.

基金资助

国家自然科学基金项目(61972208,62272239)资助;江苏省农业科技创新基金项目(JASTIF)CX(22)1007资助.