1.School of Resources & Civil Engineering,Northeastern University,Shenyang 110819,China
2.Shenyang Branch of Liaoning Institute of Water Resources and Hydropower Research Co. ,Ltd. ,Shenyang 110003,China. Corresponding author: WANG Zhi,E-mail: wangzhi@mail. neu. edu. cn
In the Liaohe conservation areas, the boundaries of land features are irregular and difficult to discern. To monitor encroachments into arable and forest lands within these areas, a multi-scale layered fusion Swin Transformer module method for land cover classification is proposed. This method includes a hierarchical feature extraction module with stacked Swin Transformer modules to capture both global spatial and local contextual information; a multi-scale feature decoding module for efficient fusion of high-level and low-level features; additional auxiliary decoding heads for more accurate classification; and comprehensive fusion of features from two decoding branches. Validation using a land cover classification dataset created from GF-2 satellite imagery and the publicly available GID-5 dataset shows that the method achieves average intersection-over-union ratio of 82.83% and 69.05%, demonstrating the best classification performance compared to other land cover classification mainstream methods.
每个Swin Transformer模块都由基于窗口的多头自注意力(windows multi-head self-attention,W_MSA)模块和基于移动窗口的多头自注意力(shifted windows multi-head self-attention,SW_MSA)模块串联组成,具体结构如图2所示.
引入LN(LayerNorm)层对输入特征进行归一化处理,将特征输入到1个通过高斯误差线性单元(gaussian error linear units,GELU)非线性激活函数连接的2个全连接层的多层感知机(multilayer perceptron,MLP),对输入特征进行非线性变换和重新编码,更好地捕捉图像中的长距离关系,每个模块采用残差连接,计算过程如式(1)所示.
LiR, ZhengS Y, DuanC X, et al. Land cover classification from remote sensing images based on multi-scale fully convolutional network[J]. Geo-spatial Information Science, 2022, 25(2): 278-294.
[2]
YinH, PflugmacherD, LiA, et al. Land use and land cover change in Inner Mongolia-understanding the effects of China’s re-vegetation programs[J]. Remote Sensing of Environment, 2018, 204: 918-930.
[3]
TuY, ChenB, ZhangT, et al. Regional mapping of essential urban land use categories in China: a segmentation-based approach[J]. Remote Sensing, 2020, 12(7): 1058.
[4]
CaoY X, HuangX. A coarse-to-fine weakly supervised learning method for green plastic cover segmentation using high-resolution remote sensing images[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2022, 188: 157-176.
[5]
JohnsonB A, MaL. Image segmentation and object-based image analysis for environmental monitoring: recent areas of interest, researchers’ views on the future priorities[J]. Remote Sensing, 2020, 12(11): 1772.
[6]
EnoguanbhorE C, GollnowF, NielsenJ O, et al. Land cover change in the Abuja city-region, Nigeria: integrating GIS and remotely sensed data to support land use planning[J]. Sustainability, 2019, 11(5): 1313.
[7]
ShiH, ChenL, BiF K, et al. Accurate urban area detection in remote sensing images[J]. IEEE Geoscience and Remote Sensing Letters, 2015, 12(9): 1948-1952.
[8]
SassuA, GambellaF, GhianiL, et al. Advances in unmanned aerial system remote sensing for precision viticulture[J]. Sensors, 2021, 21(3): 956.
[9]
LuT, LiS T, FangL Y, et al. From subpixel to superpixel: a novel fusion framework for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(8): 4398-4411.
[10]
SunB, KangX D, LiS T, et al. Random-walker-based collaborative learning for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 55(1): 212-222.
[11]
SargentI, ZhangC, AtkinsonP M. Joint deep learning for land cover and land use classification: US10984532 [P]. 2021-04-20.
[12]
PapoutsisI, BountosN I, ZavrasA, et al. Benchmarking and scaling of deep learning models for land cover image classification[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2023, 195: 250-268.
[13]
ManzanarezS, ManianV, SantosM, et al. Land use land cover labeling of GLOBE images using a deep learning fusion model[J]. Sensors, 2022, 22(18): 6895.
[14]
KroupiE, KesaM, Navarro-SánchezV D, et al. Deep convolutional neural networks for land-cover classification with Sentinel-2 images[J]. Journal of Applied Remote Sensing, 2019, 13(2): 024525.
[15]
HuangC Q, HeC, WuQ, et al. Classification of the land cover of a megacity in ASEAN using two band combinations and three machine learning algorithms: a case study in Ho Chi Minh City[J]. Sustainability, 2023, 15(8): 6798.
[16]
ShelhamerE, LongJ, DarrellT. Fully convolutional networks for semantic segmentation[J] IEEE Transactions on Pattern Analysis and Machine Intelligence. 2017,39(4): 640-651.
[17]
RonnebergerO, FischerP, BroxT. U-Net: convolutional networks for biomedical image segmentation[C]//Medical Image Computing and Computer-Assisted Intervention 2015. Cham: Springer, 2015: 234-241.
[18]
ChenL C, PapandreouG, SchroffF, et al. Rethinking atrous convolution for semantic image segmentation[EB/OL]. (2017-12-05)[2024-07-23].
[19]
BadrinarayananV, KendallA, CipollaR. SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495.
[20]
ZhaoH S, ShiJ P, QiX J, et al. Pyramid scene parsing network[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE, 2017: 6230-6239.
[21]
FuJ, LiuJ, TianH J, et al. Dual attention network for scene segmentation[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach: IEEE, 2020: 3141-3149.
[22]
AlhichriH, AlswayedA S, BaziY, et al. Classification of remote sensing images using EfficientNet-B3 CNN model with attention[J]. IEEE Access, 2021, 9: 14078-14094.
[23]
HeC, LiuY L, WangD C, et al. Automatic extraction of bare soil land from high-resolution remote sensing images based on semantic segmentation with deep learning[J]. Remote Sensing, 2023, 15(6): 1646.
[24]
MaX Y, XuJ D, ChongQ P, et al. FCUnet: refined remote sensing image segmentation method based on a fuzzy deep learning conditional random field network[J]. IET Image Processing, 2023, 17(12): 3616-3629.
[25]
KhanS, NaseerM, HayatM, et al. Transformers in vision: a survey[EB/OL]. (2022-01-19)[2024-07-23].
[26]
AshishV. Attention is all you need[EB/OL]. (2023-08-02)[2024-07-23].
[27]
XieE Z, WangW H, YuZ D, et al. SegFormer: simple and efficient design for semantic segmentation with transformers [EB/OL].(2021-10-28)[2024-05-21].
[28]
LiuZ, LinY T, CaoY, et al. Swin transformer: hierarchical vision transformer using shifted windows[C]//2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal: IEEE. 2022: 9992-10002.
[29]
WangL, LiR B, ZhangC, et al. UNetFormer: a UNet-like transformer for efficient semantic segmentation of remote sensing urban scene imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2022, 190: 196-214.
[30]
MaJ G, ShenH R, CaiY X, et al. UCTNet with dual-flow architecture: snow coverage mapping with Sentinel-2 satellite imagery[J]. Remote Sensing, 2023, 15(17): 4213.
[31]
HuZ H, QianY R, XiaoZ Q, et al. SABNet: self-attention bilateral network for land cover classification[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2024, 17: 8559-8569.
[32]
TongX Y, XiaG S, LuQ K, et al. Land-cover classification with high-resolution remote sensing images using transferable deep models[J]. Remote Sensing of Environment, 2020, 237: 111322.