个性化联邦学习综述

Survey of Personalized Federated Learning

摘要

联邦学习(Federated Learning,FL)是一种典型的分布式机器学习方法.由于在隐私保护方面有着独特的优势,联邦学习在近年来受到了广泛的关注和研究.然而,传统的联邦学习存在着两大亟待解决的核心难题:数据异构性困境、模型泛化与个性化之间的冲突.为了解决这些问题,个性化联邦学习(Personalized Federated Learning,PFL)的概念被引入,它能够针对每个联邦学习客户端的本地数据特点进行个性化的模型调整,允许联邦学习客户端在保护自身敏感数据隐私的同时根据自己的需求构建个性化模型.本文概述了个性化联邦学习的概念以及目前所面临的关键问题,分类综述了个性化联邦学习各类方法的发展现状,同时介绍了个性化联邦学习的一些新兴研究方向和领域应用情况.

Abstract

Federated Learning (FL) is a typical distributed machine learning approach.Due to its unique advantages in privacy preservation,FL has garnered widespread attention and research in recent years.However,traditional federated learning faces two core challenges that urgently need to be addressed:the issue of data heterogeneity and the conflict between model generalization and personalization.To tackle these problems,the concept of Personalized Federated Learning (PFL) has been introduced.PFL enables personalized model adjustments based on the local data characteristics of each federated learning client,allowing clients to build customized models according to their specific needs while protecting the privacy of their sensitive data.This paper provides an overview of the concept of personalized federated learning and the key challenges it currently faces,categorizes and reviews the developmental progress of various PFL methods,and introduces some emerging research directions and practical applications of personalized federated learning.

关键词

联邦学习 / 个性化联邦学习 / 知识蒸馏 / 预训练-微调 / 个性化特征聚合

Key words

federated learning / personalized federated learning / knowledge distillation / pre-training and fine-tuning / personalized feature aggregation

[1] Jin H,Bai D,Yao D,et al.Personalized edge intelligence via federated self-knowledge distillation[J].IEEE Transactions on Parallel and Distributed Systems,2022,34(2):567-580.
[2] Wu J,Bao W,Ainsworth E,et al.Personalized federated learning with parameter propagation[C]//Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining,2023:2594-2605.
[3] Wang P,Liu B,Zeng D,et al.Personalized federated learning via backbone self-distillation[C]//Proceedings of the 5th ACM International Conference on Multimedia in Asia(MMAsia),2023:1-7.
[4] Yang F E,Wang C Y,Wang Y C F.Efficient model personalization in federated learning via client-specific prompt generation[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision(ICCV),2023:19159-19168.
[5] Sun G,Mendieta M,Luo J,et al.Fedperfix:towards partial model personalization of vision transformers in federated learning[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision(ICCV),2023:4988-4998.
[6] Mclaughlin C,Su L.Personalized federated learning via feature distribution adaptation[C]//Advances in Neural Information Processing Systems(NeurIPS),2024:77038-77059.
[7] Liu Y,Wu X,Liu Chengkun.Research on personalized federated learning algorithm in industrial internet of things[J].Journal of Chinese Computer Systems,2025,46(1):209-216.
[8] Huang Yuchen,Zhao Yanchao,Hao Jiangshan,et al.Research on performance optimization of federated learning for data heterogeneity[J].Journal of Chinese Computer Systems,2024,45(4):777-783.
[9] Ye M,Fang X,Du B,et al.Heterogeneous federated learning:state-of-the-art and research challenges[J].ACM Computing Surveys,2023,56(3):1-44.
[10] Guo Guijuan,Tian Hui,Pi Huijuan,et al.Advances in federated learning for non-independent identically distributed data[J].Journal of Chinese Computer Systems,2023,44(11):2442-2449.
[11] Feng C M,Yu K,Liu N,et al.Towards instance-adaptive inference for federated learning[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision(ICCV),2023:23287-23296.
[12] Zhang J,Hua Y,Wang H,et al.Fedala:adaptive local aggregation for personalized federated learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence(AAAI),2023:11237-11244.
[13] Liang H,Zhan Z,Liu W,et al.FedReMa:improving personalized federated learning via leveraging the most relevant clients[M].Netherlands:IOS Press,2024.
[14] Mei H,Cai D,Zhou A,et al.FedMoE:personalized federated learning via heterogeneous mixture of experts[J].arXiv preprint arXiv:2408.11304,2024.
[15] Xie C,Huang D A,Chu W,et al.Perada:parameter-efficient federated learning personalization with generalization guarantees[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR),2024:23838-23848.
[16] Chen Cong,Li Jing.Knowledge distillation for federated learning with Non-IID data[J].Journal of Chinese Computer Systems,2025,46(6):1289-1297.
[17] Peng Y,Jiang F,Dong L,et al.Personalized federated learning for generative AI-assisted semantic communications[J].arXiv preprint arXiv:2410.02450,2024.
[18] Chen Z,Yang H,Quek T,et al.Spectral co-distillation for personalized federated learning[C]//Advances in Neural Information Processing Systems(NeurIPS),2023:8757-8773.
[19] Xie L,Lin M,Luan T,et al.MH-pFLID:model heterogeneous personalized federated learning via injection and distillation for medical data analysis[C]//Proceedings of the 41st International Conference on Machine Learning(ICML),2024:54561-54575.
[20] Yi L,Yu H,Wang G,et al.pFedLoRA:model-heterogeneous personalized federated learning with LoRA tuning[J].arXiv preprint arXiv:2310.13283,2023.
[21] Du Y,Zhang Z,Yue L,et al.Communication-efficient personalized federated learning for speech-to-text tasks[C]//IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP),2024:10001-10005.
[22] M Ghari P,Shen Y.Personalized federated learning with mixture of models for adaptive prediction and model fine-tuning[C]//Advances in Neural Information Processing Systems,2024:92155-92183.
[23] Zhang M,Sapra K,Fidler S,et al.Personalized federated learning with first order model optimization[C]//International Conference on Learning Representations(ICLR),2021:10622-10638.
[24] Yin K,Mao J.Personalized federated learning with adaptive feature aggregation and knowledge transfer[J].arXiv preprint arXiv:2410.15073,2024.
[25] Lai J,Li J,Xu J,et al.pFedGPA:diffusion-based generative parameter aggregation for personalized federated learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence(AAAI),2025:17999-18007.
[26] Tan Y,Long G,Jiang J,et al.Influence-oriented personalized federated learning[J].arXiv preprint arXiv:2410.03315,2024.
[27] Tamirisa R,Xie C,Bao W,et al.Fedselect:personalized federated learning with customized selection of parameters for fine-tuning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR),2024:23985-23994.
[28] Zhang J,Hua Y,Wang H,et al.Fedcp:separating feature information for personalized federated learning via conditional policy[C]//Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining,2023:3249-3261.
[29] Zhang J,Hua Y,Wang H,et al.Gpfl:simultaneously learning global and personalized feature information for personalized federated learning[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision(ICCV),2023:5041-5051.
[30] Huang Y,Chu L,Zhou Z,et al.Personalized cross-silo federated learning on non-IID data[C]//Proceedings of the AAAI Conference on Artificial Intelligence(AAAI),2021:7865-7873.
[31] Wu X,Liu X,Niu J,et al.Bold but cautious:unlocking the potential of personalized federated learning through cautiously aggressive collaboration[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision(ICCV),2023:19375-19384.
[32] Wu X,Liu X,Niu J,et al.The diversity bonus:learning from dissimilar distributed clients in personalized federated learning[J].arXiv preprint arXiv:2407.15464,2024.
[33] Li T,Hu S,Beirami A,et al.Ditto:fair and robust federated learning through personalization[C]//International Conference on Machine Learning(ICML),2021:6357-6368.
[34] T Dinh C,Tran N,Nguyen J.Personalized federated learning with moreau envelopes[C]//Advances in Neural Information Processing Systems(NeurIPS),2020:21394-21405.
[35] Chen D,Yao L,Gao D,et al.Efficient personalized federated learning via sparse model-adaptation[C]//International Conference on Machine Learning(ICML),2023:5234-5256.
[36] Fallah A,Mokhtari A,Ozdaglar A.Personalized federated learning:a meta-learning approach[C]//Proceedings of the 34th Conference on Neural Information Processing Systems(NeurIPS),2020,doi:10.48550/arXiv.2002.07948.
[37] Acar D A E,Zhao Y,Zhu R,et al.Debiasing model updates for improving personalized federated training[C]//International Conference on Machine Learning(ICML),2021:21-31.
[38] Reisser M,Louizos C,Gavves E,et al.Federated mixture of experts[J].arXiv preprint arXiv:2107.06724,2021.
[39] Zadouri T,Üstün A,Ahmadian A,et al.Pushing mixture of experts to the limit:extremely parameter efficient MoE for instruction tuning[C]//12th International Conference on Learning Representations(ICLR),2024:22214-22233.
[40] Yi L,Yu H,Ren C,et al.pFedMoE:data-level personalization with mixture of experts for model-heterogeneous personalized federated learning[J].arXiv preprint arXiv:2402.01350,2024.
[41] Zhang W,Zhou Z,Wang Y,et al.Dm-pfl:hitchhiking generic federated learning for efficient shift-robust personalization[C]//Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining,2023:3396-3408.
[42] Qin Z,Yao L,Chen D,et al.Revisiting personalized federated learning:robustness against backdoor attacks[C]//Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining,2023:4743-4755.
[43] Qin Z,Deng S,Zhao M,et al.Fedapen:personalized cross-silo federated learning with adaptability to statistical heterogeneity[C]//Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining,2023:1954-1964.
[44] Xia H,Li K,Ding Z.Personalized semantics excitation for federated image classification[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision(ICCV),2023:19301-19310.
[45] An Z,Johnson T T,Ma M.Formal logic enabled personalized federated learning through property inference[C]//Proceedings of the AAAI Conference on Artificial Intelligence(AAAI),2024:10882-10890.
[46] Xiao Z,Chen Z,Liu L,et al.FedLoGe:joint local and generic federated learning under long-tailed data[C]//12th International Conference on Learning Representations(ICLR),2024:32383-32402.
[47] Baek J,Jeong W,Jin J,et al.Personalized subgraph federated learning[C]//Proceedings of the 40th International Conference on Machine Learning(ICML),2023:1396-1415.
[48] Liang W,Zhao Y,She R,et al.FedSheafHN:personalized federated learning on graph-structured data[J].arXiv preprint arXiv:2405.16056,2024.
[49] Li Y,Xu W,Wang H,et al.Personalized federated domain-incremental learning based on adaptive knowledge matching[C]//European Conference on Computer Vision(ECCV),2024:127-144.
[50] Liu Q,Sun S,Liang Y,et al.Personalized federated learning for spatio-temporal forecasting:a dual semantic alignment-based contrastive approach[C]//Proceedings of the AAAI Conference on Artificial Intelligence,Philadelphia(AAAI),2025:12192-12200.
[51] Yu H,Yang X,Gao X,et al.Personalized federated continual learning via multi-granularity prompt[C]//Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining,2024:4023-4034.
[52] Yin B,Chen Z,Tao M.Aggregation design for personalized federated multi-modal learning over wireless networks[J].IEEE Communications Letters,2024,28(8):1850-1854.
[53] Shi J,Chen T,Zhang S,et al.Personalized quantum federated learning for privacy image classification[J].arXiv preprint arXiv:2410.02547,2024.
[54] Dave D,Vyas K,Jayagopal J K,et al.FedGlu:a personalized federated learning-based glucose forecasting algorithm for improved performance in glycemic excursion regions[J].arXiv preprint arXiv:2408.13926,2024.
[55] Yu P L,Kundu A,Wynter L,et al.Fed+:a unified approach to robust personalized federated learning[J].arXiv preprint arXiv:2009.06303,2021.
[56] Sáinz Pardo Díaz J,Castrillo M,Bartok J,et al.Personalized federated learning for improving radar based precipitation nowcasting on heterogeneous areas[J].Earth Science Informatics,2024,17(6):5561-5584.

基金资助

浙江工业大学科技项目(KYY-HX-20220288,KYY-HX-20180649)资助.