基于深度学习的可见光与红外图像融合方法研究

Research on Visible-Infrared Image Fusion Method Based on Deep Learning

摘要

为提高可见光与红外图像融合的性能，以 SwinFusion 模型作为主干网络进行改进研究。首先，将一种表示学习引导的红外与可见光图像融合网络（LRRNet）中基于低秩和稀疏表示的模块引入模型的浅层与深层特征提取之间，以增强多尺度特征的表达能力；其次，将 SwinFuse 中基于 L1 范数的容和策略引入主干网络的特征融合部分，以优化多模态信息的整合。仿真实验结果表明改进后的模型能准确捕捉各自模态的特征分布，验证了所提方法的有效性。

Abstract

To improve the performance of infrared and visible image fusion，this study conducts an enhancement of the SwinFusion model as the backbone network.Firstly，a module based on low-rank and sparse representation from the novel representation learning guided fusion network（LRRNet）is introduced between the shallow and deep feature extraction stages，aiming to enhance the expression capability of multiscale features.Secondly，the L1-norm-based fusion strategy from SwinFuse is incorporated into the feature fusion part of the backbone network to optimize the integration of multimodal information.Simulation results demonstrate that the improved model can accurately capture the feature distributions of each modality，validating the effectiveness of the proposed method.

关键词

可见光与红外图像融合 / 深度学习 / 特征融合

Key words

visible-infrared fusion / deep learning / feature fusion

参考文献

[1] Li H, WuX J, Kittler J. RFN-Ncst:an end-to-cnd residual fusion network for infrared and visible images[J]. Information Fusion, 2021, 73:72-86.

[2] 孙学蕾, 高宏伟. 改进小波变换的红外与可见光融合方法研究[J]. 沈阳理工大学学报, 2021, 40(3):19-23,28.

[3] Sun X L, Gao H W. Study on infrared and visible light fusion methods to improve wavelet transformation[J]. Journal of Shenyang Ligong University, 2021, 40 (3):19-23,28. (in Chinese)

[4] 刘砚菊, 崔洁, 宋建辉, 等. GAN 网络的红外与可见光图像融合方法研究[J]. 沈阳理工大学学报, 2022, 41(3):14-19,26.

[5] Liu Y J, Cui J, Song J H, et al. Research on infrared and visi- ble images fusion method of GAN network[J]. Journal of Shenyang Ligong University, 2022, 41(3):14-19,26. (in Chinese)

[6] 沈瑜, 梁丽, 王海龙, 等. 基于N-RGAN模型的红外与可见光图像融合[J]. 红外技术, 2023, 45(9):897-906.

[7] Shen Y, Liang L, Wang H L, et al. Infrared and visible image fusion based on N-RGAN model[J]. Infrared Technology, 2023, 45(9):897-906. (in Chinese)

[8] Liang Z Y, Yu M X, Sun Y C, et al. PEFusion:an infrared and visible light fusion method for enhanced representation of prominent targets and exposure errors[J]. Infrared Physics & Technology, 2024, 141:105496.

[9] Liu Q, Pi J T, Gao P, et al. STFNet:self-supervised transform- er for infrared and visible image fusion[J]. IEEE Transactions on Emerging Topics in Computational Intelligence, 2024, 8 (2):1513-1526.

[10] Yuan B, Sun H Y, Guo Y J, et al. Explainable analysis of infrared and visible light image fusion based on deep learning[J]. Scientific Reports, 2025, 15(1):2223.

[11] Lu Q, Han Z D, Hu L K, et al. An infrared and visible image fusion algorithm method based on a dual bilateral least squares hybrid filter[J]. Electronics, 2023, 12(10):2292.

[12] Gong G, Wang X P, Zhang J H, et al. MSFF:a multi-scale fea- ture fusion convolutional neural network for hyperspectral image classification[J]. Electronics, 2025, 14(4):797.

[13] Ma J Y, Tang L F, Fan F, et al. SwinFusion:cross-domain long-range learning for general image fusion via swin transformer[J]. IEEE/CAA Journal of Automatica Sinica, 2022, 9 (7):1200-1217.

[14] Li H, Xu T Y, Wu X J, et al. LRRNet:a novel representation learning guided fusion network for infrared and visible images[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(9):11040-11052.

[15] Wang Z S, Chen Y L, Shao W Y, et al. SwinFuse:a residual Swin Transformer fusion network for infrared and visible images[J]. IEEE Transactions on Instrumentation and Measurement, 2022, 71:1-12.

[16] Sun Y M, Cao B, Zhu P F, et al. Drone-based RGB-infrared cross-modality vehicle detection via uncertainty-aware learning[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32(10):6700-6713.

[17] Zhao Z X, Xu S, Zhang J S, et al. Efficient and model-based infrared and visible image fusion via algorithm unrolling[J]. IEEE Transactions on Circuits and Systems for Video Tech- nology, 2022, 32(3):1186-1196.

[18] Fu Y, Wu X J. A Dual-branch network for infrared and visible image fusion[C]//2020 25th International Conference on Pattern Recognition(ICPR).Milan,1taly:IEEE,2021:10675-10680.

[19] Rao D Y, Xu T Y, Wu X J. TGFuse:An infrared and visible image fusion approach based on transformer and generative adversarial network[J]. IEEE Transactions on Image Processing, 2023, PP(99):1.

[20] Zhao Z X, Xu S, Zhang C X, et al. DIDFuse:deep image decomposition for infrared and visible image fusion[C]//2020 29th International Joint Conference on Artificial Intelligence (IJCAI).Yokohama,Japan:IJCAI Organization, 2020:970-976.

[21] Li H, Wu X J. DenseFuse:a fusion approach to infrared and visible images[J]. IEEE Transactions on Image Processing, 2019, 28(5):2614-2623.

[22] Li H, Wu X J, Durrani T. NestFuse:an infrared and visible image fusion architecture based on nest connection and spatial/channel attention models[J]. IEEE Transactions on Instrumentation and Measurement, 2020, 69(12):9645-9656.

基金资助

辽宁省教育厅高等学校基本科研项目(LJ232410144070)