面向资源受限环境的渐进式多保真度神经架构搜索

Progressive Multi Fidelity Neural Architecture Search for Resource Constrained Environments

摘要

神经架构搜索（NAS）是深度学习自动化的关键技术，但其高昂的计算成本严重限制了实际应用.传统方法需要对每个候选架构进行完整训练，导致搜索过程耗时且资源密集。本文提出渐进式多保真度神经架构搜索方法（PMF-NAS），通过三阶段渐进策略实现高效架构搜索 PMF-NAS 在全局探索阶段使用低保真度快速评估识别高潜力区域，在区域搜索阶段采用中等保真度在缩小的空间内细化搜索，在精细优化阶段对最优侯选进行高保真度验证。孩方法的核心是基于早期训练特征的性能预测器，能够准确预测架构最终性能，避免大量无效计算。同时引入自适应资源分配机制，根据架构潜力和不确定性动态调整评估投入.实验表明，PMF-NAS 在单 GPU 环境下可在 8∼9 小时内完成搜索，同时在多个数据集上达到最优或接近最优的准确率。文本为资源受限环境下的神经架构搜索提供了实用解决方案，降低了 NAS 技术的应用门槛，有望推动其在更广泛领域的应用。

Abstract

Neural Architecture Search（NAS）is a key technology for deep learning automation，but its high computational cost severely limits its practical applications.Traditional methods require complete training for each candidate architecture，resulting in a time-consu- ming and resource intensive search process.This article proposes a progressive multi fidelity neural architecture search method（PMF- NAS），which achieves efficient architecture search through a three-stage progressive strategy.PMF-NAS uses low fidelity to quickly e- valuate and identify high potential areas during the global exploration phase，uses medium fidelity to refine the search within a reduced space during the area search phase，and performs high fidelity validation on the optimal candidate during the fine optimization phase. The core of this method is a performance predictor based on early training features，which can accurately predict the final performance of the architecture and avoid a large amount of ineffective computation.At the same time，an adaptive resource allocation mechanism is introduced to dynamically adjust the evaluation investment based on the potential and uncertainty of the architecture.Experiments have shown that PMF-NAS can complete searches in 8∼9 hours in a single GPU environment，while achieving optimal or near optimal ac- curacy on multiple datasets.Text provides a practical solution for neural architecture search in resource constrained environments，re- ducing the application threshold of NAS technology and potentially promoting its application in a wider range of fields.

关键词

神经架怐捜索 / 多保真度评估 / 渐进式捜索 / 性能预测 / 资源优化

Key words

neural architecture search / multi fidelity evaluation / progressive search / performance prediction / resource optimization

参考文献

[1] Lecun Y, Bengio Y, Hinton G. Deep learning[J]. Nature, 2015, 521 (7553):436-444.

[2] He K, Zhang X, Ren S, et al. Deep residual learning for image rec- ognition[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016:770-778.

[3] WU J H, LI K Y, CHEN L X, et al. A survey on neural architecture search techniques[J]. Application Research of Computers, 2025, 42 (1):11-18.

[4] Zoph B, Le Q V. Neural architecture search with reinforcement learning[J]. arXiv preprint arXiv:1611.01578,2016.

[5] Salmani Pour Avval S, Eskue N D, Groves R M, et al. Systematic review on neural architecture search[J]. Artificial Intelligence Re- vicw, 2025, 58(3):73,doi:10.1007/s10462-024-11058-w.

[6] Real E, Aggarwal A, Huang Y, et al. Regularized evolution for im- age classifier architecture search[C]// Proceedings of the AAAI Conference on Artificial Intelligence, 2019:4780-4789.

[7] Poyser M, Breckon T P. Neural architecture search:a contemporary literature review for computer vision applications[J]. Pattern Rec- ognition, 2024, 147:110052.

[8] Salehin I, Islam M S, Saha P, et al. AutoML:a systematic review on automated machine learning with neural architecture search[J]. Journal of Information and Intelligence, 2024, 2(1):52-81.

[9] Pham H, Guan M, Zoph B, et al. Efficient neural architecture search via parameters sharing[C]// International Conference on Machine Learning, 2018:4095-4104.

[10] Guo Z, Zhang X, Mu H, et al. Single path one-shot neural architec- ture search with uniform sampling[C]// European Conference on Computer Vision, 2020:544-560.

[11] Chu X, Zhang B, Xu R. FairNAS:rethinking evaluation fairness of weight sharing neural architecture search[C]// Proceedings of the IEEE International Conference on Computer Vision, 2021:12239-12248.

[12] Liu H, Simonyan K, Yang Y. DARTS:differentiable architecture search[J]. arXiv preprint arXiv:1806.09055,2018.

[13] Xu Y, Xie L, Zhang X, et al. PC-DARTS :partial channel connec- tions for memory-efficient architecture search[J]. arXiv preprint arXiv:1907.05737,2019.

[14] Dong X, Yang Y. Searching for a robust neural architecture in four gpu hours[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019:1761-1770.

[15] YANG J, LIN Y J, ZHOU P. Differentiable graph neural architec- ture search based on relational features[J]. Journal of Beijing Uni- versity of Aeronautics and Astronautics, 2025:1-17,doi:10.13700/j.bh.1001-5965-2024-0794.

[16] JIANG P C, XUE Y. Evolutionary neural architecture search meth- od based on ranking score prediction[J]. Chinese Journal of Com- puters, 2024, 47(11):2522-2535.

[17] Luo R, Tian F, Qin T, et al. Neural architecture optimization[C]// Advances in Neural Information Processing Systems, 2018:7816-7827.

[18] Li G, Hoang D, Bhardwaj K, et al. Zero-shot neural architecture search: challenges,solutions,and opportunities[J]. IEEE Transactions on Pat- tern Analysis and Machine Intelligence, 2024, 46(12):7618-7635.

[19] Han X, Xue Y, Wang Z, et al. SaDENAS:a self-adaptive differenti- al evolution algorithm for neural architecture search[J]. Swarm and Evolutionary Computation, 2024, 91:101736.

[20] Song X, Lv Z, Fan J, et al. Evolutionary multi-objective spiking neural architecture search for image classification[J]. IEEE Trans- actions on Evolutionary Computation, 2025,doi:10.1109/TEVC.2025.3528471.

[21] Cai H, Zhu L, Han S. ProxylessNAS:direct neural architecture search on target task and hardware[J]. arXiv preprint arXiv:1812.00332,2018.

[22] Tan M, Chen B, Pang R, et al. MnasNet:platform-aware neural ar- chitecture search for mobile[C]// Proceedings of the IEEE Confer- ence on Computer Vision and Pattern Recognition, 2019:2820-2828.

[23] Lupión M, Cruz N C, Ortigosa E M, et al. A holistic approach for resource-constrained neural network architecture search[I]. Ap- plied Soft Computing, 2025:112832,doi:10.1016/j.asoc.2025.112832.

[24] Cai H, Gan C, Wang T, et al. Once-for-all: train one network and spe- cialize it for efficient deployment[C]// International Conference on Learning Representations, 2020,doi:10.48550/arXiv.1908.09791.

[25] Xu Z, Wu J. Contrastive meta-reinforcement learning for heteroge- neous graph neural architecture search[J]. Expert Systems with Applications, 2025, 260:125433.

[26] REN P Z, LIANG X D, CHANG X J, et al. Neural architecture search on temporal convolutions for complex action recognition[J]. Journal of Computer Research and Development, 2025, 62 (8):1862-1874.

[27] Salehin I, Islam M S, Saha P, et al. AutoML:a systematic review on automated machine learning with neural architecture search[J]. Journal of Information and Intelligence, 2024, 2(1):52-81.

[28] Yu K, Sciuto C, Jaggi M, et al. Evaluating the search phase of neu- ral architecture search[J]. arXiv preprint arXiv: 1902. 08142,2019.

[29] Zela A, Siems J, Zimmer L, et al. Understanding and robustifying differentiable architecture search[ J]. arXiv preprint arXiv: 1909. 09656,2019.

[30] Ning X, Zheng Y, Zhao T, et al. A generic graph-based neural ar- chitecture encoding scheme for predictor-based NAS[C]// Europe- an Conference on Computer Vision, 2021:189-204.

[31] 武家辉, 李科研, 陈丽新, 等. 神经架构搜索技术研究综述[J]. 计算机应用研究, 2025, 42(1):11-18.

[32] 杨军, 繭颖婕, 周鹏. 基于火系特征的可微图神经架构搜索[J]. 北京航空航大大学学报, 2025:1-17,doi:10.13700/j.bh.1001-5965.2024.0794.

[33] 将鹏程, 苩羽. 基于排序得分预测的演化神经架构搜索方法[J]. 计算机学报, 2024, 47(11):2522-2535.

[34] 仕鹏真, 梁小丹, 常晓军, 等. 基于时间卷积神经架构搜索的复杂动作识别[J]. 计算机研究与发展, 2025, 62(8):1862-1874.

基金资助

国家白然科学基金项目(61972208)

国家白然科学基金项目(62272239)