基于模态可靠性与可形变跨尺度共注意力的多模态脑肿瘤MRI分割

Multi-modal brain tumor MRI segmentation based on modality reliability and deformable cross-scale co-attention

摘要

目的：在Medical Imaging Segmentation Toolkit集成的U-Net骨干基础上提出一种基于模态可靠性与可形变跨尺度共注意力的三维（3D）多模态脑肿瘤分割网络。方法：设计3项轻量级改进：在输入端引入模态可靠性感知融合模块，自适应抑制低质量模态噪声；在解码阶段构建可形变跨尺度共注意力模块，增强肿瘤边界及形变区域的表达；在深监督分支中加入不确定性引导深监督模块，提升对小病灶和模糊区域的优化能力。结果：在BraTS2021数据集上进行验证，与多种3D分割基线模型进行比较。结果发现本文模型在完整肿瘤、肿瘤核心和增强肿瘤区域均取得性能领先的效果：完整肿瘤区的Dice相似性系数（DSC）为0.921，肿瘤核心区的DSC为0.895，增强肿瘤区的DSC为0.891，平均95% Hausdorff距离为4.122 mm，优于其他模型。结论：本研究提出的3D多模态脑肿瘤分割网络在保持网络结构轻量化的前提下，有效提升多模态信息融合质量、跨尺度特征对齐能力及深监督优化效率，为脑肿瘤MRI自动分割提供一种具有鲁棒性和精细度的方案。

Abstract

Objective To propose a three-dimensional (3D) multi-modal brain tumor segmentation network built upon the U-Net backbone implemented in the Medical Imaging Segmentation Toolkit, which integrates modality reliability and deformable cross-scale co-attention. Methods Three lightweight improvements were designed for the network. Specifically, a modality reliability-aware fusion module was embedded at the input stage to adaptively suppress low-quality modality noise. A deformable cross-scale co-attention module was introduced within the decoder to enhance the representation of tumor boundaries and structurally deformed regions. An uncertainty-guided deep supervision module was used in the deep supervision branch to improve the optimization performance for small lesions and ambiguous regions. Results The proposed method was validated on the BraTS2021 dataset and compared with several 3D segmentation baseline models. Results showed that the proposed model achieved optimal performance across whole tumor, tumor core, and enhancing tumor subregions, with Dice similarity coefficients of 0.921, 0.895, and 0.891, respectively, and a mean 95% Hausdorff distance reaching 4.122 mm, which outperformed the other models. Conclusion The proposed 3D multi-modal brain tumor segmentation network effectively enhances multi-modal feature fusion, cross-scale feature alignment, and deep supervision optimization, while maintaining a lightweight architecture. This study provides a robust and fine-grained solution for automated brain tumor segmentation from MRI images.

关键词

脑肿瘤 / 图像分割 / 模态可靠性感知融合 / 跨尺度可形变共注意力 / 不确定性引导深监督

Key words

brain tumor / image segmentation / modality reliability-aware fusion / deformable cross-scale co-attention / uncertainty-guided deep supervision

参考文献

[1] Louis DN, Perry A, Reifenberger G, et al. The 2016 World Health Organization classification of tumors of the central nervous system: a summary[J]. Acta Neuropathol, 2016, 131(6): 803-820.

[2] van den Bent MJ, Geurts M, French PJ, et al. Primary brain tumours in adults[J]. Lancet, 2023, 402(10412): 1564-1579.

[3] Hesamian MH, Jia WJ, He XJ, et al. Deep learning techniques for medical image segmentation: achievements and challenges[J]. J Digit Imaging, 2019, 32(4): 582-596.

[4] Litjens G, Kooi T, Bejnordi BE, et al. A survey on deep learning in medical image analysis[J]. Med Image Anal, 2017, 42: 60-88.

[5] 刘海超, 宋丽娟 . 多模态MRI脑肿瘤分割方法的特征融合技术综述[J]. 计算机工程与应用, 2024, 60(23): 28-48.

[6] Liu HC, Song LJ . Review of feature fusion techniques for multimodal MRI brain tumor segmentation methods[J]. Computer Engineering and Applications, 2024, 60(23): 28-48.

[7] Verma A, Yadav AK . Brain tumor segmentation with deep learning: current approaches and future perspectives[J]. J Neurosci Methods, 2025, 418: 110424.

[8] Zaitoon R, Syed H . A comprehensive review: tumour recognition, segmentation and classification in brain MRI[C]// 2024 International Conference on Expert Clouds and Applications (ICOECA). Piscataway, NJ, USA: IEEE, 2024: 865-870.

[9] 王楠, 王华, 魏德健, 等 . U—Net及其变体在脑肿瘤图像分割中的应用进展[J]. 激光与光电子学进展, 2025, 62(16): 14-34.

[10] Wang N, Wang H, Wei DJ, et al. Applications and advancements of U—Net and its variants in brain tumor image segmentation[J]. Laser & Optoelectronics Progress, 2025, 62(16): 14-34.

[11] Brosch T, Tang LY, Yoo Y, et al. Deep 3D convolutional encoder networks with shortcuts for multiscale feature integration applied to multiple sclerosis lesion segmentation[J]. IEEE Trans Med Imaging, 2016, 35(5): 1229-1239.

[12] Çiçek Ö, Abdulkadir A, Lienkamp SS, et al. 3D U—Net: learning dense volumetric segmentation from sparse annotation[C]// Medical Image Computing and Computer—Assisted Intervention—MICCAI 2016. Cham: Springer International Publishing, 2016: 424-432.

[13] Milletari F, Navab N, Ahmadi SA . V—Net: fully convolutional neural networks for volumetric medical image segmentation[C]// 2016 Fourth International Conference on 3D Vision (3DV). Piscataway, NJ, USA: IEEE, 2016: 565-571.

[14] Kamnitsas K, Ledig C, Newcombe VF, et al. Efficient multi—scale 3D CNN with fully connected CRF for accurate brain lesion segmentation[J]. Med Image Anal, 2017, 36: 61-78.

[15] Isensee F, Jaeger PF, Kohl SA, et al. nnU—Net: a self—configuring method for deep learning—based biomedical image segmentation[J]. Nat Methods, 2021, 18(2): 203-211.

[16] Sun JD, Peng YJ, Guo YF, et al. Segmentation of the multimodal brain tumor image used the multi—pathway architecture method based on 3D FCN[J]. Neurocomputing, 2021, 423: 34-45.

[17] Zhuang YZ, Liu H, Song EM, et al. A 3D cross—modality feature interaction network with volumetric feature alignment for brain tumor and tissue segmentation[J]. IEEE J Biomed Health Inform, 2023, 27(1): 75-86.

[18] Dolz J, Gopinath K, Yuan J, et al. HyperDense—Net: a hyper—densely connected CNN for multi—modal image segmentation[J]. IEEE Trans Med Imaging, 2019, 38(5): 1116-1126.

[19] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook, NY, USA: Curran Associates Inc., 2017: 6000-6010.

[20] Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16x16 words: transformers for image recognition at scale[EB/OL]. (2021—06—03)[2025—08—20]. https://arxiv.org/abs/2010.11929.

[21] Liu Z, Lin YT, Cao Y, et al. Swin transformer: hierarchical vision transformer using shifted windows[EB/OL]. (2021—08—17)[2025—09—15]. https://arxiv.org/abs/2103.14030.

[22] Wang WX, Chen C, Ding M, et al. TransBTS: multimodal brain tumor segmentation using transformer[C]// Medical Image Computing and Computer Assisted Intervention—MICCAI 2021. Cham: Springer International Publishing, 2021: 109-119.

[23] Hatamizadeh A, Nath V, Tang YC, et al. Swin UNETR: Swin transformers for semantic segmentation of brain tumors in MRI images[EB/OL]. (2022—01—4)[2025—09—13]. https://arxiv.org/abs/2201.01266.

[24] Menze BH, Jakab A, Bauer S, et al. The multimodal brain tumor image segmentation benchmark (BRATS)[J]. IEEE Trans Med Imaging, 2015, 34(10): 1993-2024.

[25] Bakas S, Akbari H, Sotiras A, et al. Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features[J]. Sci Data, 2017, 4: 170117.

[26] Shaker A, Maaz M, Rasheed H, et al. UNETR++: delving into efficient and accurate 3D medical image segmentation[J]. IEEE Trans Med Imaging, 2024, 43(9): 3377-3390.

[27] Zhou HY, Guo JS, Zhang YH, et al. nnFormer: volumetric medical image segmentation via a 3D transformer [J]. IEEE Trans Image Process, 2023, 32: 4036-4045.

[28] Dai JF, Qi HZ, Xiong YW, et al. Deformable convolutional networks[C]// 2017 IEEE International Conference on Computer Vision (ICCV). Piscataway, NJ, USA: IEEE, 2017: 764-773.

[29] Zhu XZ, Su WJ, Lu LW, et al. Deformable DETR: deformable transformers for end—to—end object detection[EB/OL]. (2021—03—18)[2025—10—20]. https://arxiv.org/abs/2010.04159.

[30] Chen JN, Lu YY, Yu QH, et al. TransUNet: transformers make strong encoders for medical image segmentation[EB/OL]. (2021—02—8)[2025—09—21]. https://arxiv.org/abs/2102.04306.

[31] Roy S, Koehler G, Ulrich C, et al. MedNeXt: transformer—driven scaling of ConvNets for medical image segmentation[EB/OL]. (2024—06—02)[2025—09—13]. https://arxiv.org/abs/2303.09975.

[32] Baid U, Ghodasara S, Mohan S, et al. The RSNA—ASNR—MICCAI BraTS 2021 benchmark on brain tumor segmentation and radiogenomic classification[EB/OL]. (2021—09—12)[2026—01—10]. https://arxiv.org/abs/2107.02314.

[33] Celaya A, Actor JA, Muthusivarajan R, et al. PocketNet: a smaller neural network for medical image analysis[J]. IEEE Trans Med Imaging, 2023, 42(4): 1172-1184.

[34] Celaya A, Lim E, Glenn R, et al. MIST: a simple and scalable end—to—end 3D medical imaging segmentation framework[EB/OL]. (2024—11—18)[2025—09—11]. https://arxiv.org/abs/2407.21343.

基金资助

泉州市科技计划项目(2026QZNY048)