腔内镜与MR影像的配准融合算法在胃肠道疾病中的应用

Application of endoscopic and MRI image registration and fusion algorithm in gastrointestinal diseases

摘要

目的：开发一种鲁棒、精准的腔内镜-MR影像配准融合算法，并系统验证其在胃肠道肿瘤精确定位与边界界定、炎症性肠病评估等临床场景中的应用价值，以推动多模态影像融合在消化内镜领域的实用化进程。方法：提出一种基于混合特征与深度强化学习的非刚性配准框架（Hybrid-DRL）。采用包含120例患者的配对腔内镜（白光、窄带成像、蓝光成像模式）与三维T₂加权MR影像数据集进行训练与测试。算法首先提取基于深度学习的稠密特征图与改进的加速稳健特征关键点作为混合特征，随后通过近端策略优化深度强化学习代理优化非刚性形变场参数。结果：该算法的平均靶点配准误差相比传统基于互信息的算法和单纯基于深度学习的VoxelMorph算法显著降低（P<0.01）。在临床应用中，融合影像使早期胃癌边界的观察者间一致性从0.72提高至0.91，并辅助发现12%的病例存在传统内镜难以识别的黏膜下浸润迹象。结论：提出的Hybrid-DRL算法能有效实现腔内镜与MR影像的高精度配准，并在提升胃肠道疾病诊疗精准性方面具有明确价值，为胃肠道疾病的精准诊疗提供了可靠的技术框架与实证依据，为多模态智能融合从算法创新向临床实用转化提供关键支持。

Abstract

Objective To develop a robust and precise registration and fusion algorithm for endoscopic and MRI images, and systematically validate its application potential in clinical scenarios including precise localization and boundary definition of gastrointestinal tumors and the evaluation of inflammatory bowel diseases, thereby promoting the practical application of multimodal image fusion in digestive endoscopy. Methods A non-rigid registration framework (Hybrid-DRL) was established based on hybrid features and deep reinforcement learning. A paired dataset consisting of endoscopic images (white light, narrow band imaging, and blue light imaging modes) and three-dimensional T₂-weighted MR images from 120 patients was used for model training and testing. The algorithm first extracted dense feature maps based on deep learning and improved speeded up robust features key points to construct hybrid features, and then optimized non-rigid deformation field parameters through the proximal policy optimization-based deep reinforcement learning agent. Results The average target registration error of the proposed algorithm was significantly lower than that of the traditional mutual information-based algorithm and the deep learning-based VoxelMorph algorithm (P<0.01). In clinical applications, the fusion images improved the inter-observer consistency of boundary observation for early gastric cancer from 0.72 to 0.91, and assisted in detecting 12% of cases with submucosal infiltration signs that were difficult to identify by traditional endoscopy. Conclusion The proposed Hybrid-DRL algorithm can effectively achieve high-precision registration of endoscopic and MRI images, and possesses promising value in improving the diagnostic and therapeutic accuracy of gastrointestinal diseases. These findings provide a reliable technical framework and empirical basis for the precise diagnosis and treatment of gastrointestinal diseases, as well as key support for the transformation of multimodal intelligent fusion from algorithm innovation to clinical application.

关键词

医学影像配准 / 多模态融合 / 腔内镜 / 磁共振成像 / 胃肠道疾病 / 深度学习 / 计算机辅助诊断

Key words

medical image registration / multimodal fusion / endoscopy / magnetic resonance imaging / gastrointestinal disease / deep learning / computer-aided diagnosis

参考文献

[1] 向廷坚 . 消化内镜下胃肠道早期肿瘤的诊断与治疗分析[J]. 世界最新医学信息文摘, 2017, 17(89): 178.

[2] Xiang TJ . Diagnosis and treatment analysis of digestive endoscopic early gastrointestinal tumor[J]. World Latest Medicine Information, 2017, 17(89): 178.

[3] 中华医学会消化内镜学分会外科学组, 中国医师协会内镜医师分会消化内镜专业委员会, 中华医学会外科学分会胃肠外科学组. 中国消化道黏膜下肿瘤内镜诊治专家共识(2018版)[J]. 中华消化内镜杂志, 2018, 35(8): 536—546.

[4] Surgical Section of Chinese Society of Digestive Endoscopology, Chinese Medical Doctor Association Endoscope Branch Digestive Endoscopy Professional Committee, Gastrointestinal Surgery Section of Chinese Society of Surgery. The Chinese consensus on endoscopic diagnosis and management of gastrointestinal submucosal tumor (version 2018)[J]. Chinese Journal of Digestive Endoscopy, 2018, 35(8): 536—546.

[5] 张联合 . 磁共振小肠造影(MRE)在小肠疾病诊断中的应用进展[J]. 中华消化杂志, 2024, 44(3): 161-165.

[6] Zhang LH . Advances in the application of magnetic resonance enterography (MRE) in the diagnosis of small intestinal diseases[J]. Chinese Journal of Digestion, 2024, 44(3): 161-165.

[7] 马露凡, 罗凤, 严江鹏, 等 . 深度医学图像配准研究进展: 迈向无监督学习[J]. 中国图象图形学报, 2021, 26(9): 2037-2057.

[8] Ma LF, Luo F, Yan JP, et al. Deep—learning based medical image registration pathway: towards unsupervised learning[J]. Journal of Image and Graphics, 2021, 26(9): 2037-2057.

[9] 邹茂扬, 杨昊, 潘光晖, 等 . 深度学习在医学图像配准上的研究进展与挑战[J]. 生物医学工程学杂志, 2019, 36(4): 677-683.

[10] Zou MY, Yang H, Pan GH, et al. Research progress and challenges of deep learning in medical image registration[J]. Journal of Biomedical Engineering, 2019, 36(4): 677-683.

[11] Han R, Jones CK, Lee J, et al. Deformable MR—CT image registration using an unsupervised, dual—channel network for neurosurgical guidance[J]. Med Image Anal, 2022, 75: 102292.

[12] 陈逸致, 潘骏, 廖专, 等 . 胶囊内镜新兴技术研究与应用进展[J]. 中华消化内镜杂志, 2020, 37(5): 372-376.

[13] Chen YZ, Pan J, Liao Z, et al. Research and application progress of emerging technologies in capsule endoscopy[J]. Chinese Journal of Digestive Endoscopy, 2020, 37(5): 372-376.

[14] 熊欢, 蒋丹丹, 陈涛, 等 . 窄带成像技术联合放大内镜在胃上皮内瘤变及早期胃癌中的应用价值[J]. 陕西医学杂志, 2024, 53(11): 1517-1520.

[15] Xiong H, Jiang DD, Chen T, et al. Application value of narrow band imaging combined with magnifying endoscopy in gastric intraepithelial neoplasia and early gastric cancer[J]. Shaanxi Medical Journal, 2024, 53(11): 1517-1520.

[16]

[17]

[18] 殷琳琳, 郑文凤, 刘学进 . 消化内镜技术对消化道肿瘤患者早期诊断价值的分析[J]. 实用癌症杂志, 2024, 39(6): 1008-1011.

[19] Yin LL, Zheng WF, Liu XJ . Analysis of the value of digestive endoscopy in the early diagnosis of patients with gastrointestinal tumors[J]. The Practical Journal of Cancer, 2024, 39(6): 1008-1011.

[20] Ariyurek C, Lu LN, Sideris GA, et al. Improved IVIM imaging in adolescent Crohn's disease using dual—echo EPI distortion and motion correction[J]. NMR Biomed, 2025, 38(9): e70117.

[21] Amjad A, Xu JF, Thill D, et al. Deep learning auto—segmentation on multi—sequence magnetic resonance images for upper abdominal organs[J]. Front Oncol, 2023, 13: 1209558.

[22] 杜欣, 蔡莉莉, 周酥 . 基于深度学习的医学图像配准研究综述[J]. 电脑知识与技术, 2025, 21(17): 19-21.

[23] Du X, Cai LL, Zhou S . A review of medical image registration based on deep learning[J]. Computer Knowledge and Technology, 2025, 21(17): 19-21.

[24] Liang X, Lin S, Liu F, et al. ORRN: an ODE—based recursive registration network for deformable respiratory motion estimation with lung 4DCT images[J]. IEEE Trans Biomed Eng, 2023, 70(12): 3265-3276.

[25] Li TT, Zhang MX, Qi WY, et al. Motion correction of respiratory—gated PET images using deep learning based image registration framework[J]. Phys Med Biol, 2020, 65(15): 155003.

[26] Lu F, Zhou D, Chen HY, et al. S2P—matching: self—supervised patch—based matching using transformer for capsule endoscopic images stitching[J]. IEEE Trans Biomed Eng, 2025, 72(2): 540-551.

[27] Wang Y, Zhang H, Liu S, et al. Sliding window—based non—rigid registration for gastrointestinal endoscopic video stabilization[J]. IEEE J Biomed Health Inform, 2018, 22(3): 780-790.

[28] 王建锋, 郭晓强, 段大兵, 等 . MSCT血管成像技术对部分型肺静脉异位引流的应用价值[J]. 国际医药卫生导报, 2020, 26(17): 2559-2561.

[29] Wang JF, Guo XQ, Duan DB, et al. Application value of MSCT angiography in partial anomalous pulmonary venous connection[J]. International Medicine and Health Guidance News, 2020, 26(17): 2559-2561.

[30] 李欢, 韩晓磊, 赵星 . 超声内镜联合CT门静脉成像技术对肝硬化GOV程度及疗效的评价[J]. 影像科学与光化学, 2022, 40(1): 83-88.

[31] Li H, Han XL, Zhao X . The evaluation value of ultrasound endoscopy combined with CT portal vein imaging technology on GOV degree and therapeutic effect of liver cirrhosis[J]. Imaging Science and Photochemistry, 2022, 40(1): 83-88.

[32] 肖帆 . 基于深度剂量模型的磁共振加速器计划核查与验证研究[D]. 广州: 南方医科大学, 2023.

[33] Xiao F . Deep dose model—based plan check and verification study for MR—LINACs[D]. Guangzhou: Southern Medical University, 2023.

[34] 曹泽余, 宁波 . 基于人工智能的超声内镜在胃肠道间质瘤中的应用[J]. 临床医学进展, 2025, 15(5): 987-996.

[35] Cao ZY, Ning B . Application of AI—Based endoscopic ultrasound in gastrointestinal stromal tumors[J]. Advances in Clinical Medicine, 2025, 15(5): 987-996.

[36] 李楠楠, 赵芹, 杨华, 等 . 内镜黏膜下剥离术后患者中重度疼痛影响因素分析及预测模型的构建[J]. 现代医学, 2025, 53(12): 1851-1858.

[37] Li NN, Zhao Q, Yang H, et al. Analysis of influencing factors and construction of a predictive model for moderate to severe postoperative pain in patients undergoing endoscopic submucosal dissection[J]. Modern Medical Journal, 2025, 53(12): 1851-1858.

[38] Liu J, Shen N, Wang WY, et al. Lightweight cross—resolution coarse—to—fine network for efficient deformable medical image registration[J]. Med Phys, 2025, 52(7): e17827.

[39] Vagdargi P, Uneri A, Liu SZ, et al. Self—supervised feature detection and 3D reconstruction for real—time neuroendoscopic guidance[J]. IEEE Trans Biomed Eng, 2025, 72(7): 2249-2260.

[40] 邹茂扬, 杨昊, 潘光晖, 等 . 深度学习在医学图像配准上的研究进展与挑战[J]. 生物医学工程学杂志, 2019, 36(4): 677-683.

[41] Zou MY, Yang H, Pan GH, et al. Research progress and challenges of deep learning in medical image registration[J]. Journal of Biomedical Engineering, 2019, 36(4): 677-683.

基金资助

河北省医学科学研究课题(20241389)