基于改进Real-ESRGAN的超广角眼底图像超分辨率重建

Super-resolution Reconstruction of Ultra-widefield Fundus Images Using an Enhanced Real-ESRGAN Framework

摘要

超广角眼底成像（ultra-widefield fundus imaging，UWF）技术在视网膜病变的早期筛查与诊断中具有重要应用价值，但受制于成像设备性能及光学条件，UWF 图像存在分辨率不足、伪影明显等问题，影响诊断的准确性。为此，本文在真实场景增强超分辨率生成对抗网络（Real-ESRGAN）框架基础上融入注意力机制和多尺度特征提取模块，优化生成器网络结构，并改进判别器损失函数，提出一种改进 Real-ESRGAN 的超广角眼底图像超分辨率重建方法，强化对关键细节与纹理信息的重建能力。基于UWF数据集进行眼底图像的 2 倍与 4 倍超分辨率重建实验，并将本文方法与 ESRGAN、CATANet、MSRGAN、SwinIR、RCAN-it 及原始 Real-ESR-GAN 等主流方法进行对比，结果表明，本文改进方法在峰值信噪比（PSNR）和结构相似性指数 （SSIM）等客观指标上均优于其他方法，主观评价结果进一步验证了其在图像病灶区域清晰度提升与伪影抑制方面的优势，本文方法可为临床诊断提供更为可靠的图像支持。

Abstract

Ultra-widefield fundus imaging（UWF）is an advanced ophthalmic imaging technology that holds significant value in the early screening and diagnosis of diseases such as diabetic retinopathy and retinal detachment.However，due to limitations in imaging device performance and optical conditions，UWF images often suffer from issues like low resolution and prominent artifacts，which severely hinder diagnostic accuracy.To address these challenges，in this study，attention mechanisms and multi-scale feature extraction module are integrated into the Real-ESRGAN framework，to optimize the generator network structure，and improve the discriminator and loss function，which helps thereby enhancing the ability to reconstruct critical details and texture information.Super-resolution reconstruction experiments were conducted on the UWF dataset at magnification of 2 and 4 times， and a comprehensive comparison was made among prominent methods，including ESRGAN，CATANet，MSRGAN，SwinIR，and RCAN-it.Experimental results demonstrate that the proposed method outperforms the others in terms of objective metrics such as peak signal-to-noise ratio（PSNR） and structural similarity index（SSIM）.Subjective assessments further validate its superiority in improving lesion region clarity and suppressing artifacts，being able to provide more reliable imaging support for clinical diagnosis.

关键词

超广角眼底成像 / Real-ESRGAN / 多尺度特征提取 / 注意力机制

Key words

ultra-widefield fundus imaging / Real-ESRGAN / multi-scale feature extraction / attention mechanism

参考文献

[1] Yoo T K, Ryu I H, Kim J K, et al. Deep learning can generate traditional retinal fundus photographs using ultra-widefield images via generative adversarial networks[J]. Computer Methods and Programs in Biomedicine, 2020, 197:105761.

[2] 邓轩, 张静琳. 超广角眼底成像技术的发展及应用[J]. 眼科学报, 2022, 37(7):585-594.

[3] Deng X, Zhang J L. Development and application of ultrawide-field fundus imaging[J]. Eye Science, 2022. 37 (7): 585-594. (in Chinese)

[4] Lee K G, Song S J, Lee S, et al. FQ-UWF:unpaired generative image enhancement for fundus quality ultra-widefield retinal images[J]. Bioengineering, 2024, 11(6):568.

[5] Wei Q L, Gu Z Y, Tan W M, et al. Development and validation of an automatic ultrawide-field fundus imaging enhancement system for facilitating clinical diagnosis :a cross-sectional multicenter study[J]. Engineering, 2024, 41:179-188.

[6] Nandal P, Pahal S, Khanna A, et al. Super-resolution of medical images using real ESRGAN[J]. IEEE Access, 2024, 12: 176155-176170.

[7] Joshi S, Forjaz A, Han K S, et al. InterpolAI:deep learningbased optical flow interpolation and restoration of biomedical images for improved 3D tissue mapping[J]. Nature Methods, 2025, 22(7):1556-1567.

[8] Wu J H, Wyman O, Tang Y D, et al. Multi-view 3D reconstruction based on deep learning:a survey and comparison of methods[J]. Neurocomputing, 2024, 582:127553.

[9] Muhammad Y, Feng J C, Kaleem A, et al. Deep learning- based image reconstruction for different medical imaging modalities[J]. Computational and Mathematical Methods in Medicine, 2022, 2022:8750648.

[10] El-Shafai W, Mahmoud Mohamed E, Zeghid M, et al. Hybrid single image super-resolution algorithm for medical images[J]. Computers,Materials ＆ Continua, 2022, 72 (3): 4879-4896.

[11] Sun N, Li H N. Super resolution reconstruction of images based on interpolation and full convolutional neural network and application in medical fields[J]. IEEE Access, 2019, 7: 186470-186479.

[12] Ahmad W, Ali H, Shah Z, et al. A new generative adversarial network for medical images super resolution[J]. Scientific Reports, 2022, 12(1):9533.

[13] Gu Y C, Zeng Z T, Chen H B, et al. MedSRGAN:medical images super-resolution using generative adversarial networks[J]. Multimedia Tools and Applications, 2020, 79 (29): 21815-21840.

[14] Mukadam S B, Patil H Y. Fusion of ESRGAN,adaptive NSCT,and multi-attention CNN with wavelet transform for histopathological image classification[J]. IEEE Access, 2024, 12:129977-129993.

[15] Lu W J, Jiang J H, Tian H, et al. Asymmetric convolution Swin Transformer for medical image super-resolution[J]. Alexandria Engineering Journal, 2023, 85:177-184.

[16] Wang S P, Liu P Y, Chen S J, et al. A method for image super-resolution reconstruction with attention mechanism in generative adversarial networks[C]//Proceedings of the 2024 13th International Conference on Computing and Pattern Recognition.Tianjin,China:ACM,2024:112-119.

[17] Wang X T, Xie L B, Dong C, et al. Real-ESRGAN:training real-world blind super-resolution with pure synthetic data[C]//2021 IEEE/CVF International Conference on Computer Vision Workshops(ICCVW).Montreal,BC,Canada: IEEE,2021:1905-1914.

[18] Zhu Z W, Lei Y S, Qin Y L, et al. IRE:improved image super-resolution based on real-ESRGAN[J]. IEEE Access, 2023, 11:45334-45348.

[19] Selvaraju R R, Cogswell M, Das A, et al. Grad-CAM:visual explanations from deep networks via gradient-based localization[J]. International Journal of Computer Vision, 2020, 128 (2):336-359.

[20] Wang X T, Yu K, Wu S X, et al. ESRGAN:enhanced super- resolution generative adversarial networks[C]//Computer Vision European Conference on Computer Vision(ECCV) 2018 Workshops.Munich,Germany:Springer,2018:63-79.

[21] Liu X, Liu J, Tang J, et al. CATANet:efficient content-aware token aggregation for lightweight image super-resolution[C]//2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).Nashville,TN,USA:IEEE, 2025:17902-17912.

[22] Liang J Y, Cao J Z, Sun G L, et al. SwinIR:image restoration using Swin Transformer[C]//2021 IEEE/CVF International Conference on Computer Vision Workshops(ICCVW). Montreal,QC,Canada:IEEE,2021:1833-1844.

[23] Lin Z, Garg P, Banerjee A, et al. Revisiting RCAN:improved training for image super-resolution[PP/OL]. arXiv(2022-01-27)[2025-05-17]. https://doi.org/10.48550/arXiv.2201.11279.

基金资助

辽宁省教育厅高等学校基本科研项目(LJ212410144052)