采用分割一切模型的弱监督互学习三维医学图像配准方法

Segment Anything Model Guided Weakly-Supervised Mutual Learning for 3D Medical Image Registration

摘要

针对三维医学图像配准中无监督方法配准精度不足、弱监督方法依赖高质量分割标签且分割基础模型生成标签难以直接稳定指导配准训练的问题，提出了一种采用分割一切模型的弱监督互学习三维医学图像配准方法。首先，利用分割一切模型（SAM）从多视角提取主要解剖区域掩码，并通过融合构建交集掩码与并集掩码，以表征不同粒度的结构先验信息；其次，在弱监督配准框架中引入互学习方法，通过结合可信度与位置信息模块，促进不同掩码监督下配准模型之间的知识蒸馏与协同优化，从而实现互补信息的有效利用并提升训练稳定性。在3组脑图像数据集上的实验结果表明，所提方法能够在充分利用标签信息的同时保持稳定训练，与现有三维图像配准方法相比，在区域重叠度等指标上表现更优，其中重叠系数提升约5%，体现出更高的配准精度与可靠性。

Abstract

To address the problems of insufficient registration accuracy of unsupervised methods, heavy reliance of weakly supervised methods on high-quality segmentation labels, and inability of labels generated by foundation segmentation models to directly and stably guide registration training in 3D medical image registration, a segment anything model (SAM)-guided weakly supervised mutual learning method for 3D medical image registration was proposed. First, masks of major anatomical regions were extracted from different views using the SAM, and intersection and union masks were constructed through mask fusion to represent structural prior information at different granularities. Second, a mutual-learning mechanism was introduced into the weakly-supervised registration framework, and a module integrating confidence and position information was embedded to promote knowledge distillation and collaborative optimization between registration models under different mask supervision, thereby enabling effective utilization of complementary information and improving training stability. Finally, experimental results on three brain image datasets demonstrate that the proposed method can fully utilize label information while maintaining stable training, and outperforms existing 3D image registration methods in terms of regional overlap, with the Dice coefficient improved by approximately 5%, indicating higher registration accuracy and reliability.

关键词

医学图像配准 / 分割一切模型 / 弱监督方法 / 多视角 / 互学习方法

Key words

medical image registration / segment anything model / weakly-supervised method / multi-view / mutual learning method

参考文献

[1] Wang Quanxin, Ding Songlin, Li Yang, et al. The Allen mouse brain common coordinate framework: a 3D reference atlas[J]. Cell, 2020, 181(4): 936-953.

[2] Peng Hanchuan, Xie Peng, Liu Lijuan, et al. Morphological diversity of single neurons in molecularly defined cell types[J]. Nature, 2021, 598(7879): 174-181.

[3] Li Zhongyu, Shang Zengyi, Liu Jingyi, et al. D—LMBmap: a fully automated deep—learning pipeline for whole—brain profiling of neural circuitry[J]. Nature Methods, 2023, 20(10): 1593-1604.

[4] Balakrishnan G, Zhao A, Sabuncu M R, et al. VoxelMorph: a learning framework for deformable medical image registration[J]. IEEE Transactions on Medical Imaging, 2019, 38(8): 1788-1800.

[5] 余升林, 吴彤, 葛明锋, 等. 循环一致性的无监督可变形图像配准方法[J]. 计算机辅助设计与图形学学报, 2023, 35(4): 516-524.

[6] Yu Shenglin, Wu Tong, Ge Mingfeng, et al. Unsupervised deformable image registration method with cyclic consistency[J]. Journal of Computer—Aided Design & Computer Graphics, 2023, 35(4): 516-524.

[7] Chen Junyu, Frey E C, He Yufan, et al. TransMorph: transformer for unsupervised medical image registration[J]. Medical Image Analysis, 2022, 82: 102615.

[8] Ghahremani M, Khateri M, Jian Bailiang, et al. H—ViT: a hierarchical vision transformer for deformable image registration[C]//2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2024: 11513-11523.

[9] Jia Xi, Lu Wenqi, Cheng Xinxing, et al. Decoder—only image registration[J]. IEEE Transactions on Medical Imaging, 2025, 44(8): 3356-3369.

[10] Kim B, Kim D H, Park S H, et al. CycleMorph: cycle consistent unsupervised deformable image registration[J]. Medical Image Analysis, 2021, 71: 102036.

[11] Zhu Yongpei, Lu Shi. Swin—VoxelMorph: a symmetric unsupervised learning model for deformable medical image registration using Swin transformer[C]//Medical Image Computing and Computer Assisted Intervention—MICCAI 2022. Cham: Springer Nature Switzerland, 2022: 78-87.

[12] 赵海峰, 姚丽莎, 罗斌. 改进的人工鱼群算法和Powell法结合的医学图像配准[J]. 西安交通大学学报, 2011, 45(4): 46-52.

[13] Zhao Haifeng, Yao Lisha, Luo Bin. Registration of multi—resolution medical images using a modified artificial fish—swarm algorithm combined with Powell’s method[J]. Journal of Xi’an Jiaotong University, 2011, 45(4): 46-52.

[14] 高军, 李学伟, 张建, 等. 基于模板匹配的图像配准算法[J]. 西安交通大学学报, 2007, 41(3): 307-311.

[15] Gao Jun, Li Xuewei, Zhang Jian, et al. Image registration algorithm based on template matching[J]. Journal of Xi’an Jiaotong University, 2007, 41(3): 307-311.

[16] 吴攀超, 王宗义, 林欣堂. 采用局部差分模型描述的彩色图像配准技术[J]. 西安交通大学学报, 2011, 45(10): 30-37.

[17] Wu Panchao, Wang Zongyi, Lin Xintang. Color image registration algorithm using local derivative pattern approach[J]. Journal of Xi’an Jiaotong University, 2011, 45(10): 30-37.

[18] Yew Z J, Lee G H. 3DFeat—Net: weakly supervised local 3D features for point cloud registration[C]//Computer Vision—ECCV 2018. Cham, Germany: Springer International Publishing, 2018: 630-646.

[19] 顾峰. 基于分割辅助和级联的医学图像配准[D].西安: 西安电子科技大学, 2021.

[20] Hu Yipeng, Modat M, Gibson E, et al. Label—driven weakly—supervised learning for multimodal deformable image registration[C]//2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018). Piscataway, NJ, USA: IEEE, 2018: 1070-1074.

[21] Wu Menglin, He Xuchen, Li Fan, et al. Weakly supervised volumetric prostate registration for MRI—TRUS image driven by signed distance map[J]. Computers in Biology and Medicine, 2023, 163: 107150.

[22] Xu Zhenlin, Niethammer M. DeepAtlas: joint semi—supervised learning of image registration and segmentation[C]//Medical Image Computing and Computer Assisted Intervention—MICCAI 2019. Cham, Germany: Springer International Publishing, 2019: 420-429.

[23] Kirillov A, Mintun E, Ravi N, et al. Segment anything[C]//2023 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway, NJ, USA: IEEE, 2023: 3992-4003.

[24] Huang Shiqi, Xu Tingfa, Shen Ziyi, et al. One registration is worth two segmentations[C]//Medical Image Computing and Computer Assisted Intervention—MICCAI 2024. Cham, Germany: Springer Nature Switzerland, 2024: 665-675.

[25] Xu Hao, Xue Tengfei, Liu Dongnan, et al. SAT—morph: unsupervised deformable medical image registration using vision foundation models with anatomically aware text prompt[C]//International Workshop on Foundation Models for General Medical AI. Cham, Germany: Springer Nature Switzerland, 2025: 71-80.

[26] 冯林, 管慧娟, 孙焘, 等. 医学图像非刚性配准研究进展[J]. 北京生物医学工程, 2006, 25(4): 437-440.

[27] Feng Lin, Guan Huijuan, Sun Tao, et al. Research progress of non—rigid registration in medical images[J]. Beijing Biomedical Engineering, 2006, 25(4): 437-440.

[28] 孟勋. 典型医学图像配准方法及优化策略的研究进展[J]. 中国医疗设备, 2015, 30(8): 64-69.

[29] Meng Xun. Research on the typical medical image registration method and its optimization strategies[J]. China Medical Devices, 2015, 30(8): 64-69.

[30] Rohr K, Stiehl H S, Sprengel R, et al. Landmark—based elastic registration using approximating thin—plate splines[J]. IEEE Transactions on Medical Imaging, 2001, 20(6): 526-534.

[31] 靳峰, 冯大政. 利用空间序列描述子的快速准确的图像配准算法[J]. 西安交通大学学报, 2014, 48(6): 19-24.

[32] Jin Feng, Feng Dazheng. A fast and accurate image registration algorithm using space order descriptor[J]. Journal of Xi’an Jiaotong University, 2014, 48(6): 19-24.

[33] DeTone D, Malisiewicz T, Rabinovich A. Deep image homography estimation[EB/OL].arXiv (2016—06—13) [2025—10—27].https://arxiv.org/abs/1606.03798.

[34] Rohé M M, Datar M, Heimann T, et al. SVF—Net: learning deformable image registration using shape matching[C]//Medical Image Computing and Computer Assisted Intervention—MICCAI 2017. Cham, Germany: Springer International Publishing, 2017: 266-274.

[35] 马露凡, 罗凤, 严江鹏, 等. 深度医学图像配准研究进展: 迈向无监督学习[J]. 中国图象图形学报, 2021, 26(9): 2037-2057.

[36] Ma Lufan, Luo Feng, Yan Jiangpeng, et al. Deep—learning based medical image registration pathway: towards unsupervised learning[J]. Journal of Image and Graphics, 2021, 26(9): 2037-2057.

[37] Ronneberger O, Fischer P, Brox T. U—Net: convolutional networks for biomedical image segmentation[C]//Medical Image Computing and Computer—Assisted Intervention—MICCAI 2015. Cham, German: Springer International Publishing, 2015: 234-241.

[38] Jaderberg M, Simonyan K, Zisserman A. Spatial transformer networks[C]//Proceedings of the 29th International Conference on Neural Information Processing Systems. Cambridge, MA, USA: MIT Press, 2015: 2017-2025.

[39] Blendowski M, Hansen L, Heinrich M P. Weakly—supervised learning of multi—modal features for regularised iterative descent in 3D image registration[J]. Medical Image Analysis, 2021, 67: 101822.

[40] Li Yiwen, Fu Yunguan, Gayo I J M B, et al. Semi—weakly—supervised neural network training for medical image registration[J]. arXiv (2024—02—16) [2015—09—12].https://arxiv.org/abs/2402.10728.

[41] Zhao Ziheng, Zhang Yao, Wu Chaoyi, et al. One model to rule them all: towards universal segmentation for medical images with text prompts[J]. arXiv (2024—07—11) [2025—09—12].https://arxiv.org/abs/2312.17183v3.

[42] Ma Jun, He Yuting, Li Feifei, et al. Segment anything in medical images[J]. Nature Communications, 2024, 15(1): 654.

[43] Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network[J]. arXiv (2015—03—09) [2025—09—12].https://arxiv.org/abs/1503.02531.

[44] Zhang Ying, Xiang Tao, Hospedales T M, et al. Deep mutual learning[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, USA: IEEE, 2018: 4320-4328.

[45] Pang Youwei, Zhao Xiaoqi, Zhang Lihe, et al. Multi—scale interactive network for salient object detection[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2020: 9410-9419.

[46] You Chenyu, Zhou Yuan, Zhao Ruihan, et al. SimCVD: simple contrastive voxel—wise representation distillation for semi—supervised medical image segmentation[J]. IEEE Transactions on Medical Imaging, 2022, 41(9): 2228-2237.

[47] Kong Lingtong, Yang Jie. MDFlow: unsupervised optical flow learning by reliable mutual knowledge distillation[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 33(2): 677-688.

[48] Liu Jun, Wang Wenyi, Shen Nuo, et al. Mutualreg: mutual learning for unsupervised medical image registration[C]//ICASSP 2024—2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, NJ, USA: IEEE, 2024: 1561-1565.

[49] Avants B B, Tustison N J, Song Gang, et al. A reproducible evaluation of ANTs similarity metric performance in brain image registration[J]. NeuroImage, 2011, 54(3): 2033-2044.

[50] Klein S, Staring M, Murphy K, et al. Elastix: a toolbox for intensity—based medical image registration[J]. IEEE Transactions on Medical Imaging, 2010, 29(1): 196-205.

基金资助

国家自然科学基金资助项目(62202367)

国家自然科学基金资助项目(62577042)