一种用于无配对医学超声图像稳定增强的CycleGAN网络

A CycleGAN Network for Stable Enhancement of Unpaired Medical Ultrasound Images

摘要

针对便携式超声设备受硬件限制成像质量低,且高低质量超声图像难以配对、传统增强方法细节保留不足的问题,提出一种无需配准的生成对抗网络,用于无配对医学超声图像稳定增强.该网络以循环生成对抗网络为基础,设计最大感知特征提取增强模块提取图像全局和局部特征,并保留空间结构,构建聚类特征增强学习网络强化对图像的复杂细节提取能力,同时优化损失函数,融入伪标签分类损失和多层感知器特征损失.在超声图像增强挑战赛数据集USenhance2023上的实验结果表明,该网络的三项核心评价指标均显著优于主流及最新超声图像增强方法,增强后图像噪声更低,器官的结构、纹理和轮廓细节更清晰.

Abstract

Aiming at the problems that portable ultrasound devices suffered from low imaging quality due to hardware constraints, high- and low-quality ultrasound images were difficult to pair, and traditional enhancement methods failed to sufficiently preserve fine details, we proposed a registration-free generative adversarial network for the stable enhancement of unpaired medical ultrasound images. The network was based on the cycle-consistent generative adversarial network, we designed a maximum perception feature extraction enhancement module to extract global and local features of images while preserving their spatial structures. We constructed a clustering feature enhancement learning network to strengthen the ability of extracting complex details from the image. Meanwhile, the loss function was optimized by integrating pseudo-label classification loss and multilayer perceptron feature loss. Experimental results on the USenhance2023 ultrasound image enhancement challenge dataset show that the three core evaluation metrics of the proposed network are significantly superior to those of mainstream and state-of-the-art ultrasound image enhancement methods. The enhanced images exhibit lower noise, with clearer structural, textural, and contour details of organs.

关键词

医学超声图像增强 / 深度学习 / 生成对抗网络

Key words

medical ultrasound image enhancement / deep learning / generative adversarial network

参考文献

[1] Jiang Z, Salcudean S E, Navab N. Robotic Ultrasound Imaging: State-of-the-Art and Future Perspectives[J]. Medical Image Analysis, 2023, 89: 102878-1-102878-21.

[2] Zhou Z X, Wang Y Y, Guo Y, et al. Image Quality Improvement of Hand-Held Ultrasound Devices with a Two-Stage Generative Adversarial Network[J]. IEEE Transactions Biomedical Engineering, 2020, 67(1): 298-311.

[3] 唐志翔, 禹龙, 田生伟. 基于多尺度特征信息增强的息肉分割网络[J]. 东北师大学报(自然科学版), 2025, 57(3):95-102.

[4] (Tang Z X, Yu L, Tian S W. Polyp Segmentation Network Based on Multi-scale Feature Information Enhancement[J]. Journal of Northeast Normal University (Natural Science Edition), 2025, 57(3): 95-102.)

[5] 陈应显, 朱喆, 富颉鹏, 等. 基于条件生成对抗网络的深部水文地质剖面生成方法[J]. 吉林大学学报(地球科学版), 2026, 56(3):975-985.

[6] (Chen Y X, Zhu Z, Fu J P, et al. Deep Hydrogeological Profile Generation Method Based on Conditional Generative Adversarial Network[J]. Journal of Jilin University (Earth Science Edition), 2026, 56(3): 975-985.)

[7] 詹永照, 孙慧敏, 夏惠芬, 等. 基于线性注意和类别关联特征学习的在线动作检测[J]. 江苏大学学报(自然科学版), 2026, 47(1):39-47.

[8] (Zhan Y Z, Sun H M, Xia H F, et al. Online Action Detection Based on Linear Attention and Category-Associated Feature Learning[J]. Journal of Jiangsu University (Natural Science Edition), 2026, 47(1): 39-47.)

[9] Guo W, Bian Q, Yang S, et al. Comparative Analysis of Clinical Diagnosis and Ultrasonic Diagnosis between Primary Biliary Cirrhosis and Viral Hepatitis Cirrhosis[J]. Minerva Medica, 2022, 113(2): 361-362.

[10] Gao Y, Liu Y, Wang Y, et al. A Universal Intensity Standardization Method Based on a Many-to-One Weak-Paired Cycle Generative Adversarial Network for Magnetic Resonance Images[J]. IEEE Transactions on Medical Imaging, 2019, 38(9): 2059-2069.

[11] Holla M R, Pais A R. High-Performance Medical Image Secret Sharing Using Super-resolution for CAD Systems[J]. Applied Intelligence, 2022, 52(14): 16852-16868.

[12] Bashir S M A, Wang Y, Khan M, et al. A Comprehensive Review of Deep Learning-Based Single Image Super-resolution[J]. Peer J Computer Science, 2022, 7: 621-677.

[13] Sawatn A, Kulkarni S. Ultrasound Image Enhancement Using Super Resolution[J]. Biomedical Engineering Advances, 2022, 3: 1-9.

[14] Wang X T, Xie L B, Dong C, et al. Real-ESRGAN: Training Real-World Blind Super-resolution with Pure Synthetic Data[C]// 2021 IEEE/CVF International Conference on Computer Vision Workshops. Piscataway, NJ: IEEE, 2021: 1905-1914.

[15] Gao G W, Li W J, Li J C, et al. Feature Distillation Interaction Weighting Network for Lightweight Image Super-resolution[C]// Proceedings of the 36th AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2022: 661-669.

[16] Ma J Y, Tang L F, Xu M L, et al. STDFusionNet: An Infrared and Visible Image Fusion Network Based on Salient Target Detection[J]. IEEE Transactions Instrumencation Measurement, 2021, 70: 1-13.

[17] Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative Adversarial Networks[J]. Communication of the ACM, 2020, 63(11): 139-144.

[18] Liu H, Sigona M K, Manuel T J, et al. Synthetic CT Skull Generation for Transcranial MR Imaging-Guided Focused Ultrasound Interventions with Conditional Adversarial Networks[EB/OL]. (2022-02-21)[2025-10-01]. https://arxiv.org/abs/2202.10136.

[19] Tie X, Lam S, Zhang Y, et al. Pseudo-CT Generation from Multi-parametric MRI Using a Novel Multi-channel Multi-path Conditional Generative Adversarial Network for Nasopharyngeal Carcinoma Patients[J]. Medical Physics, 2020, 47(4): 1750-1762.

[20] Liu Y, Meng L, Zhong J P. MAGAN: Mask Attention Generative Adversarial Network for Liver Tumor CT Image Synthesis[J]. Journal of Healthcare Engineering, 2021, 2021(1): 1-11.

[21] Zhu J Y, Park T, Isola P, et al. Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks[C]// 2017 IEEE International Conference on Computer Vision. Piscataway, NJ: IEEE, 2017: 2242-2251.

[22] Wolterink J M, Dinkla A M, Savenije M H F, et al. Deep MR to CT Synthesis Using Unpaired Data [EB/OL]. (2017-08-03)[2025-11-01]. https://arxiv.org/abs/1708.01115.

[23] Liu Y X, Chen A N, Shi H Y, et al. CT Synthesis from MRI Using Multi-cycle GAN for Head-and-Neck Radiation Therapy[J]. Computerized Medical Imaging and Graphics, 2021, 91: 101953-101961.

[24] Sun B, Jia S F, Jiang X L, et al. Double U-Net CycleGAN for 3D MR to CT Image Synthesis[J]. International Journal of Comput Assisted Radiology and Surgery, 2022, 18(1): 149-156.

[25] Shi H C, Hayat M, Cai J F, et al. Unpaired Referring Expression Grounding via Bidirectional Cross-Modal Matching[J]. Neurocomputing, 2023, 518: 39-49.

[26] Zhou Z D, Guo Y, Wang Y Y. Handheld Ultrasound Video High-Quality Reconstruction Using a Low-Rank Representation Multipathway Generative Adversarial Network[J]. IEEE Transactions on Neural Networks and Learning Systems, 2021, 32(2): 575-588.

[27] Xia M H, Yang H B, Qu Y N, et al. Multilevel Structure-Preserved GAN for Domain Adaptation in Intravascular Ultrasound Analysis[J]. Medical Image Analysis, 2022, 82: 102614-102626.

[28] Huang L H, Zhou Z X, Guo Y, et al. A Stability-Enhanced CycleGAN for Effective Domain Transformation of Unpaired Ultrasound Images[J]. Biomedical Signal Processing and Control, 2022, 77: 103831-103843.

[29] Zhan F N, Yu Y C, Wu R L, et al. Multimodal Image Synthesis and Editing: The Generative AI Era[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(12): 15098-15119.

[30] Wang J Y, Wu Q M J, Pourpanah F. DC-CycleGAN: Bidirectional CT-to-MR Synthesis from Unpaired Data[J]. Computerized Medical Imaging and Graphics, 2023, 108: 102249-102261.

[31] Lei H J, Tian Z H, Xie H, et al. LAC-GAN: Lesion Attention Conditional GAN for Ultra-widefield Image Synthesis[J]. Neural Networks, 2023, 158: 89-98.

[32] Abu-Srhan A, Almallahi I, Abushariah M A M, et al. Paired-Unpaired Unsupervised Attention Guided GAN with Transfer Learning for Bidirectional Brain MR-CT Synthesis[J]. Computers in Biology and Medicine, 2021, 136: 104763-104773.

[33] Sun H F, Xi Q Y, Fan R B, et al. Synthesis of Pseudo-CT Images from Pelvic MRI Images Based on an MD-CycleGAN Model for Radiotherapy[J]. Physics in Medicine and Biology, 2022, 67(3): 41-52.

[34] Iskandar M, Mannering H, Sun Z, et al. Towards Realistic Ultrasound Fetal Brain Imaging Synthesis[J]. IEEE Transactions on Medical Imaging, 2024, 39(12): 4413-4424.

基金资助

吉林省教育厅科技攻关计划项目(JJKH20240951KJ)