频率空间协作的红外可见光图像融合网络

Frequency-spatial Collaboration Network for Infrared and Visible Image Fusion

摘要

红外可见光图像融合旨在生成既突出显著目标又包含丰富纹理的图像。现有的融合方法主要关注空间域特征，忽略了频率域信息。因此，本文提出一种频率空间协作的红外可见光图像融合网络。首先通过频率分解模块将源图像分解为高频部分 （模态特有特征）和低频部分（模态共有特征）。同时，粗略提取源图像的空间域特征，以保留良好的空间结构。最后，利用跨域自适应融合模块学习自适应权重，动态调整频率域和空间域特征，以缓解域间差异并生成高质量融合图像。在 TNO 和VOT2020- RGBT 数据集上的定量和定性实验结果表明，本文方法在 6 项评价指标上表现优异，且相比 7 种先进的融合方法，能更有效地融合多模态互补信息，生成显著性目标突出、细节丰富的融合图像。

Abstract

Infrared and visible image fusion aims to generate a single image highlighting salient objects and rich textures.Existing fu- sion methods predominantly focus on spatial characteristics while ignoring valuable frequency information.Therefore，this paper propo- ses a frequency-spatial collaboration network for infrared and visible image fusion.Firstly，the frequency decomposition module de- composes the source image into high-frequency（ modality-specific features ）and low-frequency components（ modality-shared fea- tures）.Simultaneously，the spatial characteristics are roughly extracted to preserve the spatial structure of the fused image.Finally，the cross-domain adaptive fusion module learns adaptive weights to dynamically adjust features in both frequency and spatial domains， thereby mitigating inter-domain differences and generating high-quality fused images.Quantitative and qualitative experimental results on the TNO and VOT2020-RGBT datasets demonstrate that the proposed method performs excellently across six evaluation metrics. Compared with seven state-of-the-art methods，it effectively integrates multi-modal complementary information and generates fused im- ages with prominent salient targets and fine-grained texture information.

关键词

红外图像 / 可见光图像 / 图像融合 / 频率分解 / 跨域自适应融合

Key words

infrared image / visible image / image fusion / frequency decomposition / cross-domain adaptive fusion

参考文献

[1] Liu J, Fan X, Huang Z, et al. Target-aware dual adversarial learning and a multi-scenario multi-modality benchmark to fuse infrared and visible for object detection[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 5792-5801.

[2] YU M Y I. IU X Y. Ship detection method based on multimodal visible and infrared image fusion[J/OL]. Computer Engineering, 2025,1-10,https://doi.org/10.19678/j.issn.1000-3428.0070436,2025-01-15.

[3] WANG Y T, LIU Z M, WAN Y P, et al. Target detection under low light conditions based on visible and infrared images[J]. Com- puter Engineering, 2024, 50(8):270-281.

[4] CHENG Q H, JIAN H F, ZHENG S K, et al. Illumination-aware in- frared/visible fusion for object detection[J]. Computer Science, 2025, 52(2):173-182.

[5] Tang Z, Xu T, Li H, et al. Exploring fusion strategies for accurate RGBT visual object tracking[J]. Information Fusion, 2023, 99: 101-881,doi:10.48550/arXiv.2201.08673.

[6] Wang D, Liu J, Liu R, et al. An interactivcly rcinforced paradigm for joint infrared-visible image fusion and saliency object detection[J]. Information Fusion, 2023, 98:101-828,doi:10.1016/j.inf-fus.2023.101828.

[7] LIU D, ZHANG G Y, SHI Y Q, et al. Zero-shot infrared and visible image fusion based on fusion curve[J]. Pattern Recognition and Artificial Intelligence, 2025, 38(3):268-279.

[8] Li H, Wu X J. DenseFuse: a fusion approach to infrared and visible images[J]. IEEE Transactions on Image Processing, 2019, 28 (5): 2614-2623.

[9] YANG Y, LIU J X, HUANG S Y, et al. Convolutional auto-enco- ding fusion network for infrared and visible image fusion[J]. Jour- nal of Chinese Computer Systems, 2019, 40(12):2673-2680.

[10] QIU D F, HU X Y, LIANG P W, et al. A deep progressive infrared and visible image fusion network[J]. Journal of Image and Graph- ics, 2023, 28(1):156-165.

[11] Tang L, Yuan J, Ma J. Image fusion in the loop of high-level vision tasks:a semantic-aware real-time infrared and visible image fusion network[J]. Information Fusion, 2022, 82:28-42,doi:10.1016/j.inffus.2021.12.004.

[12] Li H, Xu T, Wu X, et al. LRRNet:a novel representation learning guided fusion network for infrared and visible images[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(9):11040-11052.

[13] Ma J, Yu W, Liang P, et al. FusionGAN:a generative adversarial network for infrared and visible image fusion[J]. Information Fu-sion, 2019, 48:11-26,doi:10.1016/j.inffus.2018.09.004.

[14] Zhou H, Wu W, Zhang Y, et al. Semantic-supervised infrared and visible image fusion via a dual-discriminator generative adversarial network[J]. IEEE Transactions on Multimedia, 2021, 25:635-648, doi:10.1109/TMM.2021.3129609.

[15] Tang W, He F, Lin Y. YDTR:infrared and visible image fusion via Y-shape dynamic transformer[J]. IEEE Transactions on Multime- dia, 2022, 25:5413-5428,doi:10.1109/TMM.2022.3192661.

[16] SUN X H, GUAN Z, WANG X. Vision transformer for fusing in- frared and visible images in groups[J]. Journal of Image and Graphics, 2023, 28(1):166-178.

[17] Chen Y, Fan H, Xu B, et al. Drop an octave:reducing spatial redun- dancy in convolutional neural networks with octave convolution[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019:3434-3443.

[18] Tang L, Yuan J, Zhang H, et al. PIAFusion:a progressive infrared and visible image fusion network based on illumination aware[J]. Information Fusion, 2022,83-84:79-92,doi:10.1016/j.inffus.2022.03.007.

[19] Li H, Wu X. CrossFuse:a novel cross attention mechanism based infrared and visible image fusion approach[J]. Information Fu sion, 2024,103:102147,doi:10.1016/j.inffus.2023.102147.

[20] Alexander T. TNO image fusion dataset[J]. Figshare Dataset, 2014,doi:10.6084/m9.figshare.1008029.v2.

[21] Kristan M, Leonardis A, Matas J, et al. The eighth visual object tracking VOT2020 challenge results[C]// Computer Vision-ECCV Workshops, European Conferenceon Computer Vision,2020:1-56.

[22] Liu J, Fan X, Jiang J, et al. Learning a deep multi-scale feature en- semble and an edge-attention guidance for image fusion[J]. IFEF Transactions on Circuits and Systems for Video Technology, 2022, 32(1):105-119.

[23] Liang P, Jiang J, Liu X, et al. Fusion from decomposition:a self-su- pervised decomposition approach for image fusion[C]// European Conference on Computer Vision,2022,doi: 10.1007/978-3-031-19797-0_41.

[24] Zhao Z, Bai H, Zhu Y, et al. DDFM: denoising diffusion model for multi-modality image fusion[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023:8082-8093.

[25] Wang X, Guan Z, Qian W, et al. CS2Fusion:contrastive learning for self-supervised infrared and visible image fusion by estimating feature compensation map[J]. Information Fusion, 2024, 102:1-15,doi:10.1016/j.inffus.2023.102039.

[26] Roberts J W, Van Aardt J A, Ahmed F B. Assessment of image fu- sion procedures using entropy,image quality,and multispectral classification[J]. Journal of Applied Remote Sensing, 2008, 2(1): 1-28.

[27] MA Jiayi, MA Yong, LI Chang. Infrared and visible image fusion methods and applications;a survey[J]. Information Fusion, 2019, 45:153-178,doi:10.1016/j.inffus.2018.02.004.

[28] Wang Z, Bovik A C, Sheikh H R, et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4):600-612.

[29] Yu H, Cai Y, Cao Y, et al. A new image fusion performance metric based on visual information fidelity[J]. Information Fusion, 2013, 14(2):127-135.

[30] Cui G, Feng H, Xu Z, et al. Detail preserved fusion of visible and infrared images using regional saliency extraction and multi-scale image decomposition[J]. Optics Communications, 2015, 341:199-209,doi :10.1016/j.optcom.2014.12.032.

[31] Haghighat M, Razian M A. Fast-FMI: non-reference image fusion metric[C]// IEEE 8th International Conference on Application of Information and Communication Technologies, 2014:1-3.

[32] 于梦源, 刘向阳. 基于多模态可见光和红外图像融合的船舶检测方法[J/OL]. 计算机上程, 2025,1-10,https://doi.org/10.19678/j.issn.1000-3428.0070436,2025-01-15.

[33] 王昱婷, 刘志明, 万业平, 等. 基于可见光与红外图像的弱光条件下甘标检测[J]. 计算机上程, 2024, 50(8):270-281.

[34] 程清华, 鉴海防, 郑帅康, 等. 基于光照感垁的红外/可见光融合目标检测[J]. 计算机科学, 2025, 52(2):173-182.

[35] 刘铎, 张国印, 史一岐, 等. 基于融合山线的零样本红外与可见光图像融合方法[J]. 模式识别与人工智能, 2025, 38 (3): 268-279.

[36] 杨勇, 刘家样, 黄淑英, 等. 卷积白编码融合网络的红外可见光图像融合[J]. 小型微型计算机系统, 2019, 40(12):2673-2680.

[37] 邱德粉, 胡星宇, 梁鹏伟, 等. 红外与可见光图像渐进融合深度网络[J]. 中国图象图形学报, 2023, 28(1):156-165.

[38] 孙処辉, 官铮, 王学. 红外与可见光图像分组融合的视觉 Transformer[J]. 中国图象图形学报, 2023, 28(1):166-178.

基金资助

国家白然科学基金项目(U21A20487)