个性化联邦学习综述

Survey of Personalized Federated Learning

摘要

联邦学习（Federated Learning，FL）是一种典型的分布式机器学习方法。由于在隐私保护方面有着独特的优势，联邦学习在近年来受到了广泛的关注和研究。然而，传统的联邦学习存在着两大亟待解决的核心难题：数据异构性困境、模型泛化与个性化之间的冲突。为了解决这些问题，个性化联邦学习（Personalized Federated Learning，PFL）的概念被引入，它能够针对每个联邦学习客户端的本地数据特点进行个性化的模型调整，允许联邦学习客户端在保护自身敏感数据隐私的同时根据自己的需求构建个性化模型。本文概述了个性化联邦学习的概念以及目前所面临的关键问题，分类综述了个性化联邦学习各类方法的发展现状，同时介绍了个性化联邦学习的一些新兴研究方向和领域应用情况。

Abstract

Federated Learning（FL）is a typical distributed machine learning approach.Due to its unique advantages in privacy preser vation，FL has garnered widespread attention and research in recent years.However，traditional federated learning faces two core chal- lenges that urgently need to be addressed；the issue of data heterogeneity and the conflict between model generalization and personali- zation.To tackle these problems，the concept of Personalized Federated Learning（PFL）has been introduced.PFL enables personalized model adjustments based on the local data characteristics of each federated learning client，allowing clients to build customized models according to their specific needs while protecting the privacy of their sensitive data.This paper provides an overview of the concept of personalized federated learning and the key challenges it currently faces，categorizes and reviews the developmental progress of various PFL methods，and introduces some emerging research directions and practical applications of personalized federated learning.

关键词

联邦学习 / 个性化联邦学习 / 知识蒸馏 / 预训练-微调 / 个性化特征聚合

Key words

federated learning / personalized federated learning / knowledge distillation / pre-training and fine-tuning / personalized feature aggregation

参考文献

[1] Jin H, Bai D, Yao D, et al. Personalized edge intelligence via federa- ted self-knowledge distillation[J]. IEEE Transactions on Parallel and Distributed Systems, 2022, 34(2):567-580.

[2] Wu J, Bao W, Ainsworth E, et al. Personalized federated learning with parameter propagation[C]//Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2023:2594-2605.

[3] Wang P, Liu B, Zeng D, et al. Personalized federated learning via backbone self-distillation[C]// Proceedings of the 5th ACM Inter- national Conference on Multimedia in Asia(MMAsia), 2023:1-7.

[4] Yang F E, Wang C Y, Wang Y C F. Efficient model personalization in federated learning via client-specific prompt generation[C]// Proceedings of the IEEE/CVF International Conference on Comput- er Vision(ICCV), 2023:19159-19168.

[5] Sun G, Mendieta M, Luo J, et al. Fedpertix:towards partial model personalization of vision transformers in federated learning[C]// Proceedings of the IEEE/CVF International Conference on Comput- er Vision(ICCV), 2023:4988-4998.

[6] Mclaughlin C, Su L. Personalized federated learning via feature dis- tribution adaptation[C]// Advances in Neural Information Process- ing Systems( NeurIPS), 2024:77038-77059.

[7] Liu Y, Wu X,Liu Chengkun.Research on personalized federated learning algorithm in industrial internet of things[J]. Journal of Chinese Computer Systems, 2025, 46(1):209-216.

[8] Huang Yuchen, Zhao Yanchao, Hao Jiangshan, et al. Research on performance optimization of federated learning for data heterogenei- ty[J]. Journal of Chinese Computer Systems, 2024, 45(4):777-783.

[9] Ye M, Fang X, Du B, et al. Heterogeneous federated learning:state- of-the-art and research challenges[J]. ACM Computing Surveys, 2023, 56(3):1-44.

[10] Guo Guijuan, Tian Hui, Pi Huijuan, et al. Advances in federated learning for non-independent identically distributed data[J]. Jour- nal of Chinese Computer Systems, 2023, 44(11):2442-2449.

[11] Feng C M, Yu K, Liu N, et al. Towards instance-adaptive inference for federated learning[C]// Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision(ICCV), 2023:23287-23296.

[12] Zhang J, Hua Y, Wang H, et al. Fedala: adaptive local aggregation for personalized federated learning[C]// Proceedings of the AAAI Conference on Artificial Intelligence( AAAI), 2023:11237-11244.

[13] Liang H, Zhan Z, Liu W, et al. FedReMa:improving personalized federated learning via leveraging the most relevant clients[M]. Netherlands: IOS Press, 2024.

[14] Mei H, Cai D, Zhou A, et al. FedMoE:personalized federated learn- ing via hetcrogencous mixturc of experts[J]. arXiv preprint arXiv: 2408.11304, 2024.

[15] Xie C, Huang D A, Chu W, et al. Perada:parameter-efficient feder- ated learning personalization with generalization guarantees[C]// Proceedings of the IEFE/CVF Conference on Computer Vision and Pattern Recognition(CVPR), 2024:23838-23848.

[16] Chen Cong, Li Jing. Knowledge distillation for federated learning with Non-IID data[J]. Journal of Chinese Computer Systems, 2025, 46(6):1289-1297.

[17] Peng Y, Jiang F, Dong L, et al. Personalized federated learning for generative AI-assisted semantic communications[J]. arXiv preprint arXiv:2410.02450,2024.

[18] Chen Z, Yang H, Quek T, et al. Spectral co-distillation for personal- ized federated learning[C]// Advances in Neural Information Pro- cessing Systems(NeurTPS), 2023:8757-8773.

[19] Xie L, Lin M, Luan T, et al. MH-pFLID:model heterogeneous per- sonalized federated learning via injection and distillation for medi- cal data analysis[C]// Proceedings of the 41st International Con- ference on Machine Learning(ICML), 2024:54561-54575.

[20] Yi L, Yu H, Wang G, et al. pFedLoRA:model-heterogeneous per- sonalized federated learning with LoRA tuning[J]. arXiv preprint arXiv:2310.13283,2023.

[21] Du Y, Zhang Z, Yue L, et al. Communication-efficient personalized federated learning for speech-to-text tasks[C]// IEEE International Conference on Acoustics,Speech and Signal Processing(IC- ASSP),2024:10001-10005.

[22] M Ghari P, Shen Y. Personalized federated learning with mixture of models for adaptive prediction and model fine-tuning[C]// Ad-vances in Neural Information Processing Systems, 2024:92155-92183.

[23] Zhang M, Sapra K, Fidler S, et al. Personalized federated learning with first order model optimization[C]// International Conference on Learning Representations(ICLR), 2021:10622-10638.

[24] Yin K, Mao J. Personalized sederated learning with adaptive feature aggregation and knowledge transfer[J]. arXiv preprint arXiv: 2410.15073, 2024.

[25] Lai J, Li J, Xu J, et al. pFedGPA:diffusion-based generative param- eter aggregation for personalized federated learning[C]// Proceed- ings of the AAAI Conference on Artificial Intelligence( AAAI), 2025:17999-18007.

[26] Tan Y, Long G, Jiang J, et al. Influence-oriented personalized feder- ated learning「J].arXiv preprint arXiv:2410.03315,2024.

[27] Tamirisa R, Xie C, Bao W, et al. Fedselect:personalized federated learning with customized selection of parameters for fine-tuning[C]// Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition( CVPR), 2024:23985-23994.

[28] Zhang J, Hua Y, Wang H, et al. Fedcp:separating feature informa- tion for personalized federated learning via conditional policy[C]// Proceedings of the 29 th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2023:3249-3261.

[29] Zhang J, Hua Y, Wang H, et al. Gpfl:simultaneously learning glob- al and personalized feature information for personalized federated learning[C]// Proceedings of the IEEE/CVF International Confer- ence on Computer Vision(ICCV), 2023:5041-5051.

[30] Huang Y, Chu L, Zhou Z, et al. Personalized cross-silo federated learning on non-IID data[ ]]//Proceedings of the AAAI Confer- ence on Artificial Intelligence( AAAI),2021:7865-7873.

[31] Wu X, Liu X, Niu J, et al. Bold but cautious:unlocking the poten- tial of personalized federated learning through cautiously aggressive collaboration[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision(ICCV), 2023:19375-19384.

[32] Wu X, Liu X, Niu J, et al. The diversity bonus:learning from dis- similar distributed clients in personalized federated learning[J]. arXiv preprint arXiv:2407.15464,2024.

[33] Li T, Hu S, Beirami A, et al. Ditto:fair and robust federated learn- ing through personalization[C]// International Conference on Ma- chine Learning( ICML), 2021:6357-6368.

[34] T Dinh C, Tran N, Nguyen J. Personalized federated learning with moreau envelopes[C]// Advances in Neural Information Process- ing Systems(NeurIPS), 2020:21394-21405.

[35] Chen D, Yao L, Gao D, et al. Efficient personalized federated learn- ing via sparse model-adaptation[C]// International Conference on Machine Learning(ICML), 2023:5234-5256.

[36] Fallah A, Mokhtari A, Ozdaglar A. Personalized federated learning: a meta-learning approach[C]// Proceedings of the 34th Conference on Neural Information Processing Systems(NeurIPS), 2020,doi: 10.48550/arXiv.2002.07948.

[37] Acar D A E, Zhao Y, Zhu R, et al. Debiasing model updates for im- proving personalized federated training[C]// International Confer- ence on Machine Learning(ICML), 2021:21-31.

[38] Reisser M, Louizos C, Gavves E, et al. Federated mixture of experts[J]. arXiv preprint arXiv:2107.06724,2021.

[39] Zadouri T, Ustün A, Ahmadian A, et al. Pushing mixture of experts to the limit:extremely parameter efficient MoE for instruction tun- ing[C]// 12th International Conference on Learning Representa- tions(ICLR), 2024:22214-22233.

[40] Yi L, Yu H, Ren C, et al. pFedMoE:data-level personalization with mixture of experts for model-heterogeneous personalized federated learning[J]. arXiv preprint arXiv:2402.01350,2024.

[41] Zhang W, Zhou Z, Wang Y, et al. Dm-pfl:hitchhiking generic fed- erated learning for efficient shift-robust personalization[C]// Pro- ceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2023:3396-3408.

[42] Qin Z, Yao L, Chen D, et al. Revisiting personalized federated learning:robustness against backdoor attacks[C]// Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2023:4743-4755.

[43] Qin Z, Deng S, Zhao M, et al. Fedapen:personalized cross-silo fed- erated learning with adaptability to statistical heterogeneity[C]// Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2023:1954-1964.

[44] Xia H, Li K, Ding Z. Personalized semantics excitation for federa- ted image classification[C]// Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision(ICCV), 2023:19301-19310.

[45] An Z, Johnson T T, Ma M. Formal logic enabled personalized federated learning through property inference[C]// Proceedings of the AAAI Conference on Artificial Intelligence( AAAI),2024:10882-10890.

[46] Xiao Z, Chen Z, Liu L, et al. FedLoGe:joint local and generic fed- erated learning under long-tailed data[C]// 12th International Con- ference on Learning Representations(ICLR), 2024:32383-32402.

[47] Baek J, Jeong W, Jin J, et al. Personalized subgraph federated learn- ing[C]// Proceedings of the 40th International Conference on Ma- chine Learning( ICML ), 2023:1396-1415.

[48] Liang W, Zhao Y, She R, et al. FedSheafHN:personalized federated learning on graph-structured data[ J]. arXiv preprint arXiv: 2405. 16056,2021.

[49] Li Y, Xu W, Wang H, et al. Personalized federated domain-incre- mental learning based on adaptive knowledge matching[C]// Eu- ropean Conference on Computer Vision(ECCV), 2024:127-144.

[50] Liu Q, Sun S, Liang Y, et al. Personalized federated learning for spatio-temporal forecasting:a dual semantic alignment-based cont- rastive approach[C]// Proceedings of the AAAI Conference on Artificial Intelligence, Philadelphia( AAAI),2025:12192 12200.

[51] Yu H, Yang X, Gao X, et al. Personalized federated continual learn- ing via multi-granularity prompt[C]// Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2024:4023-4034.

[52] Yin B, Chen Z, Tao M. Aggregation design for personalized federa- ted multi-modal learning over wireless networks[J]. IEEE Com- munications Letters, 2024, 28(8):1850-1854.

[53] Shi J, Chen T, Zhang S, et al. Personalized quantum federated learning for privacy image classification[J]. arXiv preprint arXiv: 2410.02547,2024.

[54] Dave D, Vyas K, Jayagopal J K, et al. FedGlu:a personalized feder- ated learning-based glucose forecasting algorithm for improved per- formance in glycemic excursion regions[J]. arXiv preprint arXiv: 2408.13926, 2024.

[55] Yu P L, Kundu A, Wynter L, et al. Fed＋:a unified approach to ro- bust personalized federated learning[J]. arXiv preprint arXiv: 2009.06303,2021.

[56] Sáinz Pardo Díaz J, Castrillo M, Bartok J, et al. Personalized federated learning for improving radar based precipitation nowcasting on hetero- geneous areas[J]. Earth Science Informatics, 2024, 17(6):5561-5584.

基金资助

浙江工业大学科技项月(KYY-HX-20220288)

浙江工业大学科技项月(KYY-HX-20180649)