联合全局建模与局部补偿的高分辨率遥感影像语义分割

Semantic segmentation for remote sensing images based on joint global feature modeling and local feature compensation

摘要

为解决高分辨率遥感影像语义分割中普遍存在的局部细节刻画不足以及全局建模效率较低的问题，本文提出了一种结合卷积神经网络与状态空间模型的分割网络模型—残差曼巴 U 型网络模型。该模型采用轻量级的 ResNet18 作为编码器，并引入视觉状态空间模块构建解码器，实现高效的全局上下文建模。同时，为增强对细粒度语义信息的感知能力，设计了局部特征补偿模块。针对深浅层特征融合过程中易出现的语义偏差问题，进一步提出跨层级融合注意力模块，以实现空间与语义信息的有效协同。结果表明，该模型在 Vaihingen、Potsdam 和 LoveDA 三大遥感数据集上的平均交并比分别达到 83.78%、87.09% 和 52.85%，均优于现有的主流模型。该模型在维持较低计算复杂度的同时，显著提升了模型的特征表达能力与分割精度。研究结果为高分辨率遥感影像的高效智能解译提供了一种兼顾性能与效率的可行方案。

Abstract

To address the common challenges of insufficient local detail representation and low global modeling efficiency in high-resolution remote sensing image semantic segmentation, this paper proposes a segmentation network that integrates convolutional neural networks and state-space models, named RMUNet. The proposed model employs a lightweight ResNet18 as the encoder and introduces a visual state space block (VSSBlock) in the decoder to achieve efficient global context modeling. Meanwhile, a local feature compensation module (LFCM) is designed to enhance the perception of fine-grained semantic information. To mitigate the semantic bias that may arise during the fusion of shallow and deep features, a cross-level fusion attention module (CFAM) is further proposed to enable effective collaboration between spatial and semantic representations. Experimental results demonstrate that RMUNet achieves mean intersection-over-union (mIoU) scores of 83.78%, 87.09%, and 52.85% on the Vaihingen, Potsdam, and LoveDA datasets, respectively, outperforming existing mainstream methods. While maintaining low computational complexity, RMUNet significantly enhances feature representation and segmentation accuracy, providing an efficient and effective solution for high-resolution remote sensing image interpretation.

关键词

高分辨率遥感影像 / 语义分割 / Mamba / 卷积神经网络

Key words

high-resolution remote sensing images / semantic segmentation / Mamba / convolutional neural networks

作者贡献声明

张建昌负责论文的整体构思与实验方案设计，撰写论文初稿。徐伟铭提供研究经费。余大文负责全过程的论文指导。梁昌钰负责实验数据的分析与解释。陈枭负责实验数据的收集与处理。

AI使用声明

本文的英文题名、摘要和关键词采用腾讯元宝翻译生成，并进行了部分修改。

利益冲突声明

作者声明，不存在已知的可能影响本论文所报告工作的竞争性经济利益或个人关系。

伦理声明

本研究不涉及人类或动物的生物学研究。

数据可用性

支持论文结论所需的所有数据均为公开数据集。

参考文献

[1] Liu Y C， Fan B， Wang L F， et al. Semantic labeling in very high resolution imagesvia a self—cascaded convolutional neural network [J]. ISPRS Journal of Photogrammetry and Remote Sensing， 2018， 145： 78-95.

[2] Schumann G J P， Brakenridge G R， Kettner A J， et al. Assisting flood disaster response with earth observation data and products：a critical assessment[J]. Remote Sensing， 2018， 10(8)： 1230.

[3] Chen S W . SAR image speckle filtering with context covariance matrix formulation and similarity test[J]. IEEE Transactions on Image Processing， 2020， 29： 6641-6654.

[4] Zhao T J， Wang S， Ouyang C J， et al. Artificial intelligence for geoscience：progress，challenges，and perspectives[J]. The Innovation， 2024， 5(5)： 100691.

[5] Voulodimos A， Doulamis N， Doulamis A， et al. Deep learning for computer vision：a brief review[J]. Computational Intelligence and Neuroscience， 2018， 2018(1)： 7068349.

[6] Vaswani A， Shazeer N， Parmar N， et al. Attention is all you need[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach：Curran Associates Inc. ， 2017： 6000-6010.

[7] Ronneberger O， Fischer P， Brox T . U—Net：convolutional networks for biomedical image segmentation[C]// Proceedings of the 18th International Conference on Medical Image Computing and Computer—Assisted Intervention. Munich：Springer， 2015： 234-241.

[8] Chen L C， Zhu Y K， Papandreou G， et al. Encoder—decoder with atrous separable convolution for semantic image segmentation[C]// Proceedings of the 15th European Conference on Computer Vision （ ECCV） . Munich： Springer， 2018： 833-851.

[9] Zhao H S， Shi J P， Qi X J， et al. Pyramid scene parsing network[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu：IEEE， 2017： 6230-6239.

[10] Strudel R， Garcia R， Laptev I， et al. Segmenter：transformer for semantic segmentation[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Montreal：IEEE， 2021： 7242-7252.

[11] Xie E Z， Wang W H， Yu Z D， et al. SegFormer：simple and efficient design for semantic segmentation with transformers[C]// Advances in Neural Information Processing Systems 34 (NeurIPS 2021). New York: Curran Associates, 2021: 12077-12090.

[12] Cao H， Wang Y Y， Chen J， et al. Swin—Unet：Unet—like pure transformer for medical image segmentation[C]// Proceedings of the European Conference on Computer Vision. Tel Aviv：Springer， 2022： 205-218.

[13] Liu Z， Lin Y T， Cao Y， et al. Swin transformer：hierarchical vision transformer using shifted windows[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Montreal：IEEE， 2021： 9992-10002.

[14] Chen J N， Lu Y Y， Yu Q H， et al. TransUNet：transformers make strong encoders for medical image segmentation[C]// Medical Image Computing and Computer Assisted Intervention—MICCAI 2021: 24th International Conference, Strasbourg, France, September 27—October 1, 2021, Proceedings, Part II. Cham: Springer International Publishing, 2021: 234-243.

[15] Wang L B， Li R， Duan C X， et al. A novel transformer based semantic segmentation scheme for fine—resolution remote sensing images[J]. IEEE Geoscience and Remote Sensing Letters， 2022， 19： 6506105.

[16] Wang L B， Li R， Wang D Z， et al. Transformer meets convolution：a bilateral awareness network for semantic segmentation of very fine resolution urban scene images[J]. Remote Sensing， 2021， 13(16)： 3065.

[17] Wang L B， Li R， Zhang C， et al. UNetFormer：a UNet—like transformer for efficient semantic segmentation of remote sensing urban scene imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing， 2022， 190： 196-214.

[18] Zhang H W， Zhu Y， Wang D， et al. A survey on visual Mamba[J]. Applied Sciences， 2024， 14(13)： 5683.

[19] Gu A， Dao T . Mamba：linear—time sequence modeling with selective state spaces[C]// Proceedings of the 41st International Conference on Machine Learning (ICML 2024). Vienna, Austria: PMLR, 2024, 235: 16237-16280.

[20] Gu A， Goel K， Ré C . Efficiently modeling long sequences with structured state spaces[C]// Proceedings of the 10th International Conference on Learning Representations (ICLR 2022). Virtual Conference: OpenReview.net, 2022: uYLFoz1vlAC.

[21] Liu Y， Tian Y J， Zhao Y Z， et al. VMamba：visual state space model[C]// Proceedings of the 38th International Conference on Neural Information Processing Systems. Vancouver：Curran Associates Inc. ， 2024： 3273.

[22] Zhao S J， Chen H， Zhang X L， et al. RS—Mamba for large remote sensing image dense prediction[J]. IEEE Transactions on Geoscience and Remote Sensing， 2024， 62： 5633314.

[23] Zhu Q F， Cai Y Z， Fang Y， et al. Samba：semantic segmentation of remotely sensed images with state space model[J]. Heliyon， 2024， 10(19)： e38495.

[24] Zhu E Z， Chen Z， Wang D K， et al. UNetMamba：an efficient UNet—like Mamba for semantic segmentation of high—resolution remote sensing images[J]. IEEE Geoscience and Remote Sensing Letters， 2025， 22： 6001205.

[25] Ma X P， Zhang X K， Pun M O . RS³Mamba：visual state space model for remote sensing image semantic segmentation [J]. IEEE Geoscience and Remote Sensing Letters， 2024， 21： 6011405.

[26] Wang L B， Li D X， Dong S J， et al. PyramidMamba：rethinking pyramid feature fusion with selective space state model for semantic segmentation of remote sensing imagery[J]. International Journal of Applied Earth Observation and Geoinformation， 2024， 144： 104884.

[27] Liu M S， Dan J， Lu Z Q， et al. CM—UNet：hybrid CNN—Mamba UNet for remote sensing image semantic segmentation[PP/OL]. arXiv (2024—05—16)[2025—07—01]. https://arxiv.org/abs/2405.10530.

[28] Milletari F， Navab N， Ahmadi S A . V—Net：fully convolutional neural networks for volumetric medical image segmentation[C]// Proceedings of the 4th International Conference on 3D Vision （3DV）. Stanford：IEEE， 2016： 565-571.

[29] Zhu Z， Xu M D， Bai S， et al. Asymmetric non—local neural networks for semantic segmentation[C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul：IEEE， 2019： 593-602.