基于联邦学习与知识蒸馏的轻量化负荷分解方法

A Lightweight Load Monitoring Method Based on Federated Learning and Knowledge Distillation

摘要

针对深度学习模型在非侵入式负荷分解中面临的数据隐私保护和边缘部署两个问题，提出了一种基于联邦学习与知识蒸馏的轻量化负荷分解框架与方法．首先，设计了一种结合卷积神经网络(CNN)和Transformer(CNN-Transformer)的混合架构，通过CNN模块高效提取负荷序列的局部时序特征，利用改进的Transformer结构增强对长期时序依赖关系的建模能力，提高了模型的整体辨识性能；其次，提出基于知识蒸馏的模型轻量化策略，通过设计知识迁移机制，将大参量教师模型的决策能力有效压缩至轻量级学生模型，实现模型的高效轻量化；最后，构建了基于联邦学习-知识蒸馏的云边协同训练架构，采用联邦平均算法实现模型参数的高效聚合，使边缘节点在不共享原始数据的情况下参与模型训练，同时引入轻量化模型作为全局模型显著降低了通信开销．实验结果表明：所提模型在REDD和UK-DALE数据集上的辨识性能优于现有方法；轻量化策略在将模型参数降低90%的同时保持了较好的精度；所提框架较传统联邦学习降低了约85%的通信量，为非侵入式负荷分解在边缘计算场景中的部署提供了有效方案.

Abstract

To address the issues of data privacy protection and edge deployment in deep learning models for non-intrusive load monitoring(NILM)，this paper proposes a lightweight load decomposition framework and method based on federated learning and knowledge distillation. First，a hybrid convolutional neural network and Transformer(CNN-Transformer) architecture was designed. The CNN module efficiently extracted local temporal features from load sequences，while the improved Transformer structure enhanced the modeling capability of long-term temporal dependencies，thus improving the overall identification performance. Second，a model lightweight strategy based on knowledge distillation was proposed. Through a designed knowledge transfer mechanism，the decision-making capability of the large-parameter teacher model was effectively compressed into a lightweight student model，achieving efficient model compression. Finally，a cloud-edge collaborative training architecture based on federated learning and knowledge distillation was constructed. The federated averaging algorithm enabled efficient aggregation of model parameters，allowing edge nodes to participate in model training without sharing raw data. The introduction of the lightweight model as the global model significantly reduced communication overhead. Experimental results show that the proposed model outperforms existing methods on REDD and UK-DALE datasets. The lightweight strategy maintains good accuracy while reducing model parameters by 90%. Compared to traditional federated learning，the proposed framework reduces communication volume by approximately 85%，providing an effective solution for NILM deployment in edge computing scenarios.

关键词

非侵入式负荷分解 / 联邦学习 / 知识蒸馏 / 轻量化 / 隐私保护

Key words

non-invasive load monitoring(NILM) / federated learning / knowledge distillation / lightweight / privacy protection

参考文献

[1] 于浩, 杨家辉, 习伟, 等. 智能配用电数据压缩方法研究进展及展望[J]. 供用电, 2024, 41(8):2-14.

[2] Yu Hao, Yang Jiahui, Xi Wei, et al. Research progress and prospects on compression methods of smart power distribution and consumption data[J]. Power Supply and Consumption, 2024, 41(8):2-14(in Chinese).

[3] Tanoni G, Stankovic L, Stankovic V, et al. Knowledge distillation for scalable nonintrusive load monitoring[J]. IEEE Transactions on Industrial Informatics, 2024, 20(3):4710-4721.

[4] 郭陆阳, 王守相, 陈海文, 等. 基于双向GRNN与时间序列翻译模型的非侵入式负荷分解算法[J]. 供用电, 2019, 36(10):9-15，86.

[5] Guo Luyang, Wang Shouxiang, Chen Haiwen, et al. Non—invasive load decomposition algorithm based on bidirectional GRNN and time series translation model[J]. Power Supply and Consumption, 2019, 36(10):9-15，86(in Chinese).

[6] 冷喜武, 陈国平, 蒋宇, 等. 智能电网监控运行大数据分析系统的数据规范和数据处理[J]. 电力系统自动化, 2018, 42(19):169-176.

[7] Leng Xiwu, Chen Guoping, Jiang Yu, et al. Data specification and processing in big—data analysis system for monitoring and operation of smart grid[J]. Automation of Electric Power Systems, 2018, 42(19):169-176(in Chinese).

[8] Hart G W. Nonintrusive appliance load monitoring[J]. Proceedings of the IEEE, 1992, 80(12):1870-1891.

[9] 仝杰, 齐子豪, 蒲天骄, 等. 电力物联网边缘智能：概念、架构、技术及应用[J]. 中国电机工程学报, 2024, 44(14):5473-5496.

[10] Tong Jie, Qi Zihao, Pu Tianjiao, et al. Edge intelligence to power internet of things：Concept，architecture，technology and application[J]. Proceedings of the CSEE, 2024, 44(14):5473-5496(in Chinese).

[11] Schirmer P A, Mporas I. Non—intrusive load monitoring：A review[J]. IEEE Transactions on Smart Grid, 2022, 14(1):769-784.

[12] Schirmer P A, Mporas I, Paraskevas M. Energy disaggregation using elastic matching algorithms[J]. Entropy, 2020, 22(1):71.

[13] Zhong M, Goddard N, Sutton C. Interleaved factorial non—homogeneous hidden Markov models for energy disaggregation[EB/OL]. https://arxiv.org/abs/1406.7665， 2014—06—30.

[14] Kelly J, Knottenbelt W. Neural NILM：Deep neural networks applied to energy disaggregation[C]// Proceedings of the 2nd ACM International Conference on Embedded Systems for Energy—Efficient Built Environments. Seoul，Republic of Korea， 2015:55-64.

[15] Yue Z, Witzig C R, Jorde D, et al. BERT4NILM：A bidirectional transformer model for non—intrusive load monitoring[C]// Proceedings of the 5th International Workshop on Non—Intrusive Load Monitoring. Japan， 2020:89-93.

[16] Bao K, Ibrahimov K, Wagner M, et al. Enhancing neural non—intrusive load monitoring with generative adversarial networks[J]. Energy Informatics, 2018, 1:295-302.

[17] 鲍光海, 黄逸欣. 基于ResNeXt网络和迁移学习的非侵入式负荷监测[J]. 电力系统自动化, 2023, 47(13):110-120.

[18] Bao Guanghai, Huang Yixin. Non—intrusive load monitoring based on ResNeXt network and transfer learning[J]. Automation of Electric Power Systems, 2023, 47(13):110-120(in Chinese).

[19] Pan Y, Liu K, Shen Z, et al. Sequence—to—subsequence learning with conditional gan for power disaggregation[C]// 2020 IEEE International Conference on Acoustics，Speech and Signal Processing(ICASSP). Barcelona，Spain， 2020:3202-3206.

[20] 何耿生, 黄宇, 曾金灿, 等. 基于多通道融合机制的非侵入式负荷监测方法[J]. 计算机应用, 2025, 45(增1):317-322.

[21] He Gengsheng, Huang Yu, Zeng Jincan, et al. Non—intrusive load monitoring method based on multi—channel fusion mechanism[J]. Computer Applications, 2025, 45(Suppl 1):317-322(in Chinese).

[22] 王守相, 郭陆阳, 陈海文, 等. 基于特征融合与深度学习的非侵入式负荷辨识算法[J]. 电力系统自动化, 2020, 44(9):103-110.

[23] Wang Shouxiang, Guo Luyang, Chen Haiwen, et al. Non—intrusive load identification algorithm based on feature fusion and deep learning[J]. Automation of Electric Power Systems, 2020, 44(9):103-110(in Chinese).

[24] 凡航, 徐葳, 范晓昱, 等. 隐私计算在新型电力系统中的应用分析与展望[J]. 电力系统自动化, 2023, 47(19):187-199.

[25] Fan Hang, Xu Wei, Fan Xiaoyu, et al. Application analysis and prospect of privacy—preserving computation in new power system[J]. Automation of Electric Power Systems, 2023, 47(19):187-199(in Chinese).

[26] 宋祺鹏, 王继东, 张丽伟, 等. 本地化差分隐私下的电力物联网终端数据隐私保护方法[J]. 重庆邮电大学学报(自然科学版), 2023, 35(6):1001-1010.

[27] Song Qipeng, Wang Jidong, Zhang Liwei, et al. Terminal data privacy protection method for power Internet of Things based on localized differential privacy[J]. Journal of Chongqing University of Posts and Telecommunications(Natural Science Edition), 2023, 35(6):1001-1010(in Chinese).

[28] Kukunuri R, Aglawe A, Chauhan J, et al. EdgeNILM：Towards NILM on edge devices[C]// Proceedings of the 7th ACM International Conference on Systems for Energy—Efficient Buildings，Cities，and Transportation. Japan， 2020:90-99.

[29] McMahan B, Moore E, Ramage D, et al. Communication—efficient learning of deep networks from decentralized data[C]// Proceedings of the 20th International Conference on Artificial Intelligence and Statistics. Florida，USA， 2017:1273-1282.

[30] Dai S, Meng F, Wang Q, et al. DP2—NILM：A distributed and privacy—preserving framework for non—intrusive load monitoring[J]. Renewable and Sustainable Energy Reviews, 2024, 191:114091.

[31] Lin Y H, Ciou J C. A privacy—preserving distributed energy management framework based on vertical federated learning—based smart data cleaning for smart home electricity data[J]. Internet of Things, 2024, 26:101222.

[32] Hinton G. Distilling the knowledge in a neural network[EB/OL]. https://arxiv.org/pdf/1503.02531， 2015—03—09.

[33] Rafiq H, Zhang H X, Li H M, et al. Regularized LSTM based deep learning model：First step towards real—time non—intrusive load monitoring[C]// 2018 IEEE International Conference on Smart Energy Grid Engineering(SEGE). Oshawa，Canada， 2018:234-239.

[34] Zhang C Y, Zhong M J, Wang Z Z, et al. Sequence—to—point learning with neural networks for non—intrusive load monitoring[C]// Proceedings of the AAAI Conference on Artificial Intelligence. New Orleans，USA， 2018:2604-2611.

基金资助

国家电网公司总部科技资助项目(SGHEJY00PSJS2310023)