恶意被动方场景下的纵向联邦学习安全加权聚合

Secure weighted aggregation for VFL with malicious passive parties

摘要

针对纵向联邦学习中的不可信参与方发动数据投毒攻击阻碍模型训练,以及半诚实参与方发动隐私推理攻击窃取其他参与方私有数据的问题,提出了一种恶意被动方场景下的纵向联邦学习安全加权聚合方案。首先,设计效用评估算法抵御数据投毒攻击,通过计算最大容忍距离过滤有毒样本所对应的嵌入向量。然后,提出自适应权重计算算法,确保在长尾数据场景下依然能够有效抵御数据投毒攻击并保持模型的高收敛率和准确率。最后,利用掩蔽机制和对称同态加密算法保护嵌入向量隐私,抵御隐私推理攻击。理论分析和仿真结果表明本方案具有较好的计算效率和模型性能,能有效抵御隐私推理攻击和数据投毒攻击,与最新相关工作相比模型准确率提高约5%~10%。

Abstract

Considering the problem that untrustworthy participants in vertical federated learning launch data poisoning attacks to hinder model training, and that semi-honest participants launch inference attacks to steal privacy information of other participants, a securely weighted aggregation scheme for vertical federated learning with malicious passive parties is proposed. First, a utility evaluation algorithm is combined to defend against data poisoning attacks, and the maximum tolerance distance is designed to filter the poisoned embedding vectors; Second, an adaptive weight calculation algorithm is designed to ensure that the model can still effectively resist data poisoning attacks and maintain high convergence rate and accuracy in long-tailed data scenarios. Finally, the masking mechanism and symmetric homomorphic encryption algorithm are utilized to protect the privacy of embedding vectors against privacy inference attacks. Theoretical analysis and simulation results show that the proposed protocol has better computational efficiency and model performance, can effectively resist privacy inference attacks and data poisoning attacks, and improves the model accuracy by about 5%-10% compared with the latest related work.

关键词

纵向联邦学习 / 数据投毒攻击 / 隐私推理攻击 / 长尾数据

Key words

vertical federated learning / data poisoning attacks / privacy inference attack / long-tail data

参考文献

[1] MCMAHAN B, MOORE E, RAMAGE D, et al. Communication—efficient learning of deep networks from decentralized data[C]// Proceedings of Artificial Intelligence and Statistics. Cambridge: PMLR, 2017: 1273-1282.

[2] ROMANINI D, HALLAJ, PAPADOPOULOS P, et al. Pyvertical: a vertical federated learning framework for multi—headed split nn[EB/OL]. arXiv: https://arxiv.org/abs/2104.00489.

[3] LUO X J, WU Y C, XIAO X K, et al. Feature inference attack on model predictions in vertical federated learning[C]// 2021 IEEE 37th International Conference on Data Engineering (ICDE). Chania: IEEE, 2021: 181-192.

[4] ERDOĞAN E, KÜPÇÜ A, ÇİÇEK A E. UnSplit: data—oblivious model inversion, model stealing, and label inference attacks against split learning[C]// Proceedings of the 21st Workshop on Privacy in the Electronic Society. Los Angeles: ACM, 2022: 115-124.

[5] LIU Z L, CHEN Y Y, YU H, et al. GTG—shapley: efficient and accurate participant contribution evaluation in federated learning[J]. ACM Transactions on Intelligent Systems and Technology, 2022, 13(4): 1-21.

[6] 王勇, 李国良, 李开宇. 联邦学习贡献评估综述[J]. 软件学报, 2023, 34(3): 1168-1192.

[7] WANG Yong, LI Guoliang, LI Kaiyu. Survey on contribution evaluation for federated learning [J]. Journal of Software, 2023, 34(3): 1168-1192.

[8] BONAWITZ K, IVANOV V, KREUTER B, et al. Practical secure aggregation for privacy—preserving machine learning[C]// Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. Dallas: ACM, 2017: 1175-1191.

[9] MAHDIKHANI H, LU R X, ZHENG Y D, et al. Achieving O(log ³n) communication—efficient privacy—preserving range query in fog—based IoT [J]. IEEE Internet of Things Journal, 2020, 7(6): 5220-5232.

[10] SHEN S Q, TOPLE S, SAXENA P. Auror: defending against poisoning attacks in collaborative deep learning systems[C]// Proceedings of the 32nd Annual Conference on Computer Security Applications. Los Angeles California: ACM, 2016: 508-519.

[11] FUNG C, YOON C J M , BESCHASTNIKHI. The limitations of federated learning in sybil settings [C]// Proceedings of the 23rd International Symposium on Research in Attacks, Intrusions and Defenses (RAID 2020). 2020: 301-316.

[12] ANDREINA S, MARSON G A, MOLLERING H, et al. BaFFLe: backdoor detection via feedback—based federated learning[C]// 2021 IEEE 41st International Conference on Distributed Computing Systems (ICDCS). Washitom: IEEE, 2021: 852-863.

[13] ZHAO L C, WANG Q, ZOU Q, et al. Privacy—preserving collaborative deep learning with unreliable participants[J]. IEEE Transactions on Information Forensics and Security, 2019, 15: 1486-1500.

[14] QIU P, ZHANG X, JI S, et al. Hijack vertical federated learning models with adversarial embedding[EB/OL]. arXiv: https://arxiv.org/abs/2212.00322.

[15] HE Y, SHEN Z L, HUA J Y, et al. Backdoor attack against split neural network—based vertical federated learning[J]. IEEE Transactions on Information Forensics and Security, 2023, 19: 748-763.

[16] WANG S, GAI K K, YU J, et al. BDVFL: blockchain—based decentralized vertical federated learning[C]// 2023 IEEE International Conference on Data Mining (ICDM). Shanghai: IEEE, 2023: 628-637.

[17] GAO X, ZHANG L. PCAT: functionality and data stealing from split learning by pseudo—client attack[C]// Proceedings of the 32nd USENIX Security Symposium, 2023: 5271-5288.

[18] SATHYAS S , VEPAKOMMA P , RASKAR R , et al. A review of homomorphic encryption libraries for secure computation[EB/OL]. arXiv: https://arxiv.org/abs/1812.02428.

[19] BOYLE E, GILBOA N, ISHAI Y. Function secret sharing: improvements and extensions[C]// Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. Vienna: ACM, 2016: 1292-1303.

[20] HUANG Y M, WANG W W, ZHAO X Y, et al. EFMVFL: an efficient and flexible multi—party vertical federated learning without a third party[J]. ACM Transactions on Knowledge Discovery from Data, 2024, 18(3): 1-20.

[21] CAI S W, CHAI D, YANG L, et al. Secure forward aggregation for vertical federated neural networks[M]// Trustworthy Federated Learning. Cham: Springer, 2023: 115-129.

[22] FU F C, XUE H R, CHENG Y, et al. BlindFL: vertical federated machine learning without peeking into your data[C]// Proceedings of the 2022 International Conference on Management of Data. Philadelphia: ACM, 2022: 1316-1330.

[23] SUN H, ZHANG Y, LIM X, et al. FLFHNN: an efficient and flexible vertical federated learning framework for heterogeneous neural network[M]// Wireless Algorithms, Systems, and Applications. Cham: Springer Nature, 2022: 338-350.

[24] CHEN T Y, JIN X, SUN Y J, et al. Vertical asynchronous federated learning: algorithms and theoretic guarantees[M]// Federated Learning. Amsterdam: Elsevier, 2024: 199-217.

[25] THAPA C, MAHAWAGA ARACHCHIGE P C, CAMTEPE S, et al. SplitFed: when federated learning meets split learning[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2022, 36(8): 8485-8493.

[26] XU D P, YUAN S H, WU X T. Achieving differential privacy in vertically partitioned multiparty learning[C]// 2021 IEEE International Conference on Big Data (BigData). Orlando: IEEE, 2021: 5474-5483.

[27] SHI H R, XU Y H, JIANG Y L, et al. Efficient asynchronous multi—participant vertical federated learning[J]. IEEE Transactions on Big Data, 2024, 10(6): 940-952.

[28] LI S, YAO D, LIU J. FedVS: straggler—resilient and privacy—preserving vertical federated learning for split models[C]// Proceedings of the International Conference on Machine Learning. Cambridge: PMLR, 2023: 20296-20311.

[29] WANG S, GAI K, YU J, et al. VFedMH: vertical federated learning for training multi—party heterogeneous models[J]. [2024—10—15] https://arxiv.org/abs/2310.13367.

[30] MISHRA P, LEHMKUHL R, SRINIVASAN A, et al. Delphi: a cryptographic inference service for neural networks[C]// Proceedings of the 29th USENIX Security Symposium. Los Alamitos: IEEE, 2020: 2505-2522.

[31] XIA W S, LI Y, ZHANG L, et al. Cascade vertical federated learning towards straggler mitigation and label privacy over distributed labels[J]. IEEE Transactions on Big Data, 2024, 10(6): 926-939.

基金资助

国家自然科学基金资助项目(61802217)

山东省自然科学基金资助项目(ZR2023MF082)

青岛科技计划重点研发项目(22-3-4-xxgg-10-gx)

青岛市自然科学基金原创探索项目(23-2-1-164-zyyd-jch)