基于多尺度特征融合与改进注意力的锈蚀螺栓螺帽检测

Rusty bolt and nut detection based on multi-scale feature fusion and improved attention mechanism

摘要

针对输电线路巡检中螺栓螺帽目标尺寸小、数量多、且存在复杂背景遮挡的检测问题,提出具有多尺度特征融合机制和改进注意力机制的螺栓-螺母检测Transformer(bolt-nut detection Transformer, BN-DETR)算法。构建跨阶段部分连接网络(cross stage partial darknet, CSPDarknet)为主干网络的特征提取模块,通过集成局部感知、全局注意力机制和多层感知机实现多尺度特征的高效聚合,设计基于改进注意力的尺度内特征交互模块,通过动态选择关键点降低计算复杂度,同时保持全局信息交互能力,提出多层次注意力融合机制(全局、局部、像素级)提升特征表达能力。实验结果表明, BN-DETR算法的平均精度均值@50较基线算法提高3%,为电力设施微小缺陷检测提供有效的技术参考。

Abstract

To address the detection challenges associated with bolts and nuts in transmission line inspections—such as small object sizes, large quantities, and complex background occlusions, the method of bolt-nut detection Transformer (BN-DETR) incorporating a multi-scale feature fusion mechanism and enhanced attention mechanisms is proposed. A feature extraction module is constructed utilizing the cross-stage partial darknet (CSPDarknet) as the backbone, which efficiently aggregates multi-scale features through the integration of local perception, global attention mechanisms, and multi-layer perceptron. Scale-wise feature interaction module based on improved attention is designed to dynamically select key sampling points, thereby reducing computational complexity while preserving global information exchange. A multi-level attention fusion mechanism encompassing global, local, and pixel-level attention are introduced to augment feature representation. Experimental results demonstrate that the proposed BN-DETR achieves 3% improvement in mean average precision at IoU threshold 0.5 (MAP@50) compared with the baseline method. The proposed method offers an effective technical reference for the detection of small defects in power infrastructure.

关键词

螺栓螺帽 / 小目标检测 / 锈蚀 / Transformer

Key words

bolt-nut / small object detection / rust / Transformer

参考文献

[1] ALHASSAN A B, ZHANG X D, SHEN H M, et al. Power transmission line inspection robots: a review, trends and challenges for future research[J]. International Journal of Electrical Power & Energy Systems, 2020, 118: 105862.

[2] EISENBEISS H. A mini unmanned aerial vehicle (UAV): system overview and image acquisition[J]. International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, 2004, 36(5/W1): 1-7.

[3] 周庆泽, 郭擎, 王海荣, 等. 双判别器深度残差GAN高光谱图像融合[J]. 中国图象图形学报, 2024, 29(7): 2046-2062.

[4] ZHOU Qingze, GUO Qing, WANG Hairong, et al. Dual—discriminator deep residual GAN hyperspectral image fusion[J]. Journal of Image and Graphics, 2024, 29(7): 2046-2062.

[5] 孙彬, 高云翔, 诸葛吴为, 等. 可见光与红外图像融合质量评价指标分析[J]. 中国图象图形学报, 2023, 28(1): 144-155.

[6] SUN Bin, GAO Yunxiang, ZHUGE Wuwei, et al. Analysis of quality evaluation metrics for visible and infrared image fusion[J]. Journal of Image and Graphics, 2023, 28(1): 144-155.

[7] 韩鲁光, 陈纯毅, 申忠业, 等. 视觉显著性驱动的全景渲染图非局部降噪[J]. 中国图象图形学报, 2024, 29(4): 939-0952.

[8] HAN Luguang, CHEN Chunyi, SHEN Zhongye, et al. Panoramic rendering non—local denoising driven by visual saliency[J]. Journal of Image and Graphics, 2024, 29(4): 939-952.

[9] 张珂, 周睿恒, 石超君, 等. 自监督E—Swin的输电线路金具检测[J]. 中国图象图形学报, 2023, 28(10): 3064-3076.

[10] ZHANG Ke, ZHOU Ruiheng, SHI Chaojun, et al. Self—supervised E—Swin transmission line hardware detection[J]. Journal of Image and Graphics, 2023, 28(10): 3064-3076.

[11] 赵振兵, 蒋志钢, 李延旭, 等. 输电线路部件视觉缺陷检测综述[J]. 中国图象图形学报, 2021, 26(11): 2545-2560.

[12] ZHAO Zhenbing, JIANG Zhigang, LI Yanxu, et al. A review of visual defect detection in transmission line components[J]. Journal of Image and Graphics, 2021, 26(11): 2545-2560.

[13] AKBARI Y, ALMAADEED N, AL—MAADEED S, et al. Applications, databases and open computer vision research from drone videos and images: a survey[J]. Artificial Intelligence Review, 2021, 54: 3887-3938.

[14] DILSHAD N, HWANG J Y, SONG J S, et al. Applications and challenges in video surveillance via drone: a brief survey[C]// 2020 International Conference on Information and Communication Technology Convergence (ICTC). Jeju: IEEE, 2020: 728-732.

[15] SAMPEDRO C, MARTINEZ C, CHAUHAN A, et al. A supervised approach to electric tower detection and classification for powerline inspection[C]// 2014 International Joint Conference on Neural Networks (IJCNN). Beijing: IEEE, 2014: 1970-1977.

[16] MENG Xiaobo, CHEN Wencan, MEI Hongwei, et al. Corrosion mechanism of UHV transmission line tower foot in southern china[J]. IEEE Transactions on Power Delivery, 2023, 39(1): 210-219.

[17] ZHANG Heng, YANG Wen, YU Huai, et al. Detecting powerlines in UAV images with convolutional features and structured constraints[J]. Remote Sensing, 2019, 11(11): 1342.

[18] ZHENG Jianfeng, WU Hang, ZHANG Han. Insulator—defect detection algorithm based on improved YOLOv7[J]. Sensors, 2022, 22(22): 8801.

[19] ZHAO Zhenbing, WU Xueliang, QI Yincheng, et al. A knowledge—guided model of fitting detection in aerial transmission line images[J]. Energy Reports, 2020, 6: 1071-1078.

[20] LUO Peng, WANG Bo, WANG Hongxia, et al. An ultrasmall bolt defect detection method for transmission line inspection[J]. IEEE Transactions on Instrumentation and Measurement, 2023, 72: 1-12.

[21] GIRSHICK R. Fast R—CNN[C]// 2015 IEEE International Conference on Computer Vision (ICCV). Santiago, Chile: IEEE, 2015: 1440-1448.

[22] REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real—time object detection[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 779-788.

[23] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//In Proceedings of the 31st International Conference on Neural Information Processing Systems. California: Curran Associates Incorporation, 2017: 6000-6010.

[24] CARION N, MASSA F, SYNNAEVE G, et al. End—to—end object detection with transformers[C]// Computer Vision—ECCV 2020. Glasgow: Springer, 2020: 213-229.

[25] WANG Yingming, ZHANG Xiangyu, YANG Tong, et al. Anchor DETR: query design for transformer—based object detection[C]// Proceedings of the AAAI Conference on Artificial Intelligence. Vancouver: AAAI, 2022: 2567-2575.

[26] MENG Depu, CHEN Xiaokang, FAN Zejia, et al. Conditional DETR for fast training convergence[C]// 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal: IEEE, 2021: 3631-3640.

[27] ZHU Xizhou, SU Weijie, LU Lewei, et al. Deformable DETR: deformable transformers for end—to—end object detection[J/OL]. arXiv preprint. (2020—10—15)[2025—10—08]. https://arxiv.org/abs/2010.04159.

[28] LIU Shilong, LI Feng, ZHANG Hao, et al. DAB—DETR: dynamic anchor boxes are better queries for detr[J/OL]. arXiv preprint. (2022—03—07)[2025—10—08]. https://arxiv.org/abs/2203.03605.

[29] ZHAO Yian, LÜ Wenyu, XU Shangliang, et al. Detrs beat YOLO on real—time object detection[J/OL]. arXiv preprint. (2023—04—17)[2025—10—08]. https://arxiv.org/abs/2304.08069.

[30] XIA Zhuofan, PAN Xuren, SONG Shiji, et al. Vision transformer with deformable attention[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans: IEEE, 2022: 4784-4793.

[31] GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]// 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Columbus: IEEE, 2014: 580-587.

[32] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis Machine Intelligence, 2014, 37(9): 1904-1916.

[33] REN Shaoqing, HE Kaiming, GIRSHICK R, et al. Faster R—CNN: towards real—time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis Machine Intelligence, 2017, 39(6): 1137-1149.

[34] KIRILLOV A, GIRSHICK R, HE K, et al. Panoptic feature pyramid networks[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach: IEEE, 2019: 6399-6408.

[35] LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[J]. IEEE Transactions on Pattern Analysis Machine Intelligence, 2017, 99: 2999-3007.

[36] DUAN Kaiwen, BAI Song, XIE Lingxi, et al. Centernet++ for object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024, 46(5): 3509-3521.

[37] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: transformers for image recognition at scale[J/OL]. arXiv preprint. (2020—10—22)[2025—10—08]. https://arxiv.org/abs/2010.11929.

[38] WANG Tao, YUAN Li, CHEN Yunpeng, et al. PNP—DETR: towards efficient visual analysis with transformers[C]// 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal: IEEE, 2021: 4641-4650.

[39] ROH B, SHIN J W, SHIN W, et al. Sparse DETR: efficient end—to—end object detection with learnable sparsity[J/OL]. arXiv preprint. (2021—11—29)[2025—10—08]. https://arxiv.org/abs/2111.14330.

[40] DAI Xiyang, CHEN Yinpeng, YANG Jianwei, et al. Dynamic DETR: end—to—end object detection with dynamic attention[C]// 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal: IEEE, 2021: 462-471.

[41] LUO Peng, WANG Bo, WANG Hongxia, et al. An ultrasmall bolt defect detection method for transmission line inspection[J]. IEEE Transactions on Instrumentation and Measurement, 2023, 72: 5006512.

[42] SONG Zhiwei, HUANG Xinbo, JI Chao, et al. Deformable YOLOs: detection and rust warning method of transmission line connection fittings based on image processing technology[J]. IEEE Transactions on Instrumentation and Measurement, 2023, 72: 2504321.

[43] HUA Guoxiang, ZHANG Huai, HUANG Chen, et al. An enhanced YOLOv8—based bolt detection algorithm for transmission line[J]. IET Generation, Transmission & Distribution, 2024, 18(24): 4065-4077.

[44] QI Yincheng, HUO Yalin, LIU Shaohang, et al. Optimized YOLOs based transmission line bolt cascade detection[C]// 2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC). Prague: IEEE, 2022: 2099-2102.

[45] ZHANG Ke, LOU Wenshuo, WANG Jiacun, et al. PA—DETR: end—to—end visually indistinguishable bolt defects detection method based on transmission line knowledge reasoning[J]. IEEE Transactions on Instrumentation and Measurement, 2023, 72: 1-14.

[46] ZHU Pengfei, WEN Longyin, DU Dawei, et al. Detection and tracking meet drones challenge[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 44(11): 7380-7399.

[47] SUO Jiashun, WANG Tianyi, ZHANG Xingzhou, et al. HIT—UAV: a high—altitude infrared thermal dataset for unmanned aerial vehicle—based object detection[J]. Scientific Data, 2023, 10: 227.