Objective On the transport belt used for normal coal flow, large coal gangue, anchor rods, and other foreign objects can be present. When large coal gangue or other foreign objects accumulate at the coal drop port, issues such as coal stacking and coal blockage occur. Anchor rods and other foreign objects can become entangled with transport belt components, causing surface scratches or even severe belt tearing, which seriously affects the normal coal flow transport. Deep learning methods previously applied demonstrate inferior baseline network performance compared to the YOLOv8 (You Only Look Once) model and fail to incorporate targeted lightweight optimization for edge deployment scenarios. Currently, computer vision-based detection methods do not achieve performance improvements over the YOLOv8 model in coal mine target detection tasks. Therefore, this study proposes a foreign object detection method for coal mine conveyor belts, YOLOv8‒SPCD, which is developed based on an improved YOLOv8 framework. Methods The YOLOv8‒SPCD model introduced several key improvements to enhance the detection performance of the original YOLOv8 model. First, the coal belt foreign body dataset was constructed based on existing mine images. The labelme tool was utilized to annotate the image data, and the images were divided into the training set (train), validation set (val), and test set (test) based on a ratio of 8:1:1. Then, SPD‒Conv was utilized to replace the convolutional component in the Backbone, and the spatial blocks of the input feature map were rearranged into the channel dimension to increase the number of channels, reduce the spatial resolution, and retain richer information during the feature extraction stage. Next, partial convolution was introduced to improve the C2f structure in the original network. The computation of redundant feature maps was reduced, while the spatial features of the input images were still effectively extracted by applying convolution only to part of the input channels. Then, a lightweight cross-scale feature fusion module (CCFM), was utilized to improve the Neck component and enhance the detection capability of the model for objects at different scales. Finally, to eliminate the adverse effect of the penalty term in the original loss function on convergence speed and to obtain faster and more effective regression results, the improved Inner‒DIoU function was introduced to optimize the bounding box regression loss of the network, enabling faster convergence and more accurate localization of belt foreign bodies during training. Results and Discussions Groups 1 to 4 experiments were independent experiments in which the improved modules were modified separately on the baseline network, allowing the impact of each individual module on the baseline network to be clearly observed. In the third group of experiments, the CF‒Neck structure was utilized to replace the original Neck component, and the mAP value remained unchanged even though the number of model parameters was reduced by 37%, indicating that CF‒Neck enhanced the detection capability of the model for objects at different scales. In the fourth group of experiments, Inner‒DIoU was utilized to replace the CIoU loss function, and the experimental indicators, such as mAP@0.5 and FPS, were improved, indicating that Inner‒DIoU effectively enhanced the fitting performance of the model. The ninth group of experiments corresponded to the YOLOv8‒SPCD model proposed in this study. The model weight was reduced to 43% of the baseline network, GFLOPs was reduced to 59% of the original value, mAP@0.5 was increased by 4.3 percentage points, mAP@0.5:0.95 was increased by 4.1 percentage points, and FPS was slightly improved. The effectiveness of the proposed method for detecting foreign objects on coal mine belts was thus verified. The training loss curves of the YOLOv8‒SPCD model with Inner‒DIoU and without Inner‒DIoU were compared in this study, and the results showed that the convergence speed of the YOLOv8‒SPCD model with Inner‒DIoU was significantly faster than that of the model without Inner‒DIoU. The Box Loss, which measured the discrepancy between the actual boundary box and the predicted boundary box of the target object, and the Classification Loss, which measured the accuracy of the model in predicting each target category, were both significantly reduced. The distribution focal loss (DFL), which was utilized to correct errors in predicting object boundary frames, remained similar to that before modification during training, indicating that the fitting performance of the proposed model on the mine image dataset was superior to that of the original model. The proposed model was also compared to mainstream target detection models such as YOLOv3-tiny, YOLOv5n, YOLOv6n, SSD, and Faster R‒CNN. The comparison results showed that the proposed model exhibited clear advantages. Conclusions The YOLOv8 model provides a feasible technical solution for detecting the presence of coal gangue, bolts, and other foreign matter during the coal conveying process on conveyor belts. The improved model integrates a series of enhancement strategies, including SPD‒Conv, PConv, the CCFM, and the Inner idea, demonstrating the broad application potential of the YOLOv8 model in coal mine target detection. This work provides a prerequisite for deployment at the mine edge. Then, the research objective is to deploy the improved model on embedded equipment at the mine edge end, realize practical algorithm application, and further optimize the model during the deployment process.
其中, X [·]为以指定参数执行Slice操作后的特征图。以为例,为坐标(1,0)为起点的子特征图,1:S:L表示水平方向以特征图的横坐标1为开始、S为结束、L为步进进行下采样,0:S:L表示垂直方向以特征图的纵坐标0为开始、S为结束、L为步进进行下采样。
一般来说,给定任意 X,子特征图(、y分别为横、纵向起始坐标)由满足条件的特征点对应的特征图组成, X (·)为以括号内坐标为起点且保持不变的特征图,和分别为分割前任一点子特征图的横、纵坐标,和都可以被L整除。因此,每个子特征图就是将 X 下采样一个比例因子得到的。图3、4均以L=2为例,得到4个大小为的子特征图。
式(4)~(6)中:U为交并比(intersection over union,IoU),表示预测框(predicted box)与真实框(ground truth box)的交集和并集面积的比值;函数用于计算两点间的欧氏距离;和分别为预测框和真实框的中心点坐标和;为能够同时包含预测框和真实框的最小边界框对角线长度,如图8所示;为权衡参数;v为衡量预测框与真实框长宽比一致性的惩罚项;和分别为宽度和高度;wpd和hpd分别为预测框的宽度和高度。
均值平均精度(mean average precision,mAP,记为)是所有类别的平均精度,mAP值越高,表示模型在不同阈值下的表现越好,计算方法如下:
式中,M为类别总数,下标i为的类别序号。
每秒10亿次的浮点运算数(giga floating-point operations per second,GFLOPs)用于表示模型的计算量,是衡量模型计算复杂度的指标。每秒帧数(frames per second,FPS)表示每秒内可以检测的图片数量,是衡量模型检测速度的指标。设备和数据相同的情况下,FPS值越大代表检测算法运行速度越快,也意味着目标检测模型算法复杂度越低。
XuPeng.Study on the key technology of foreign object detection of coal mine belt based on edge computing[D].Xuzhou:China University of Mining and Technology,2021.
[2]
许鹏.基于边缘计算的煤矿井下皮带异物检测关键技术研究[D].徐州:中国矿业大学,2021.
[3]
AlfarzaeaiM S, HuEryi, PengWang,et al.Coal gangue classification based on the feature extraction of the volume visual perception ExM-SVM[J].Energies,2023,16(4):2064. doi:10.3390/en16042064
[4]
ShuklaR K, TiwariA K, JhaA K.An efficient approach of face detection and prediction of drowsiness using SVM[J].Mathematical Problems in Engineering,2023,2023:2168361. doi:10.1155/2023/2168361
[5]
GirshickR, DonahueJ, DarrellT,et al.Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition.Columbus:IEEE,2014:580‒587. doi:10.1109/cvpr.2014.81
[6]
HeKaiming, ZhangXiangyu, RenShaoqing,et al.Spatial pyramid pooling in deep convolutional networks for visual recognition[C]//Computer Vision‒ECCV 2014.Cham:Springer,2014:346‒361. doi:10.1007/978-3-319-10578-9_23
[7]
GirshickR.Fast R-CNN[C]//2015 IEEE International Conference on Computer Vision(ICCV).Santiago:IEEE,2015:1440‒1448. doi:10.1109/iccv.2015.169
[8]
RenShaoqing, HeKaiming, GirshickR,et al.Faster R-CNN:Towards real-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(6):1137‒1149. doi:10.1109/tpami.2016.2577031
[9]
RedmonJ, DivvalaS, GirshickR,et al.You only look once:Unified,real-time object detection[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Las Vegas:IEEE,2016:779‒788. doi:10.1109/cvpr.2016.91
WangHao, XiaoNanfeng.Underwater object detection method based on improved faster RCNN[J].Applied Sciences,2023,13(4):2746. doi:10.3390/app13042746
[12]
WangYuanbin, WangYujing, DangLangfei.Video detection of foreign objects on the surface of belt conveyor underground coal mine based on improved SSD[J].Journal of Ambient Intelligence and Humanized Computing,2023,14(5):5507‒5516. doi:10.1007/s12652-020-02495-w
[13]
ZhengJunhui, WangDeyong, GengZexun.Real-time detection of safety hazards in coal mines utilizing an enhanced YOLOv3 algorithm[J].Traitement Du Signal,2023,40(4):1565‒1572. doi:10.18280/ts.400424
[14]
ChenYiming, SunXu, XuLiang,et al.Application of YOLOv4 algorithm for foreign object detection on a belt conveyor in a low-illumination environment[J].Sensors,2022,22(18):6851. doi:10.3390/s22186851
[15]
LiDeyong, WangGuofa, GuoYongcun,et al.An identification and positioning method for coal gangue based on lightweight mixed domain attention[J].International Journal of Coal Preparation and Utilization,2023,43(9):1542‒1560. doi:10.1080/19392699.2022.2119561
[16]
ZhangLei, WangHaosheng, LeiWeiqiang,et al.Coal gangue target detection of belt conveyor based on YOLOv5s‒SDE[J].Journal of Mine Automation,2023,49(4):106‒112.
MaoQinghua, LiShikun, HuXin,et al.Foreign object recognition of belt conveyor in coal mine based on improved YOLOv7[J].Journal of Mine Automation,2022,48(12):26‒32. doi:10.13272/j.issn.1671-251x.2022100011
LuoBingxin, KouZiming, HanCong,et al.A "hardware-friendly" foreign object identification method for belt conveyors based on improved YOLOv8[J].Applied Sciences,2023,13(20):11464. doi:10.3390/app132011464
[21]
SunkaraR, LuoTie.No more strided convolutions or pooling:A new CNN building block for low-resolution images and small objects[M]//Machine Learning and Knowledge Discovery in Databases.Cham:Springer Nature Switzerland,2023:443‒459. doi:10.1007/978-3-031-26409-2_27
[22]
PengYanfei, JiYue.Road crack detection algorithm based on improved YOLOv8[C]//Proceedings of the 2023 5th International Conference on Artificial Intelligence and Computer Applications(ICAICA).Dalian:IEEE,2024:28‒32. doi:10.1109/icaica58456.2023.10405428
[23]
WangYuanyuan, JiangFeilong, LiYazhou,et al.Safety helmet detection algorithm for complex scenarios based on PConv-YOLOv8[C]//Proceedings of the 2023 International Conference on the Cognitive Computing and Complex Data(ICCD).Huaian:IEEE,2024:90‒94. doi:10.1109/iccd59681.2023.10420675
LiMao, XiaoYangyi, ZongWangyuan,et al.Detecting chestnuts using improved lightweight YOLOv8[J].Transactions of the Chinese Society of Agricultural Engineering,2024,40(1):201‒209.
ZhaoYian, LvWenyu, XuShangliang,et al.DETRs beat YOLOs on real-time object detection[EB/OL].(2023‒04‒17)[2024‒04‒08].doi:10.1109/cvpr52733.2024.01605
[28]
ZhengZhaohui, WangPing, LiuWei,et al.Distance‒IoU loss:Faster and better learning for bounding box regression[J].Proceedings of the AAAI Conference on Artificial Intelligence,2020,34(7):12993‒13000. doi:10.1609/aaai.v34i07.6999
[29]
ZhangYifan, RenWeiqiang, ZhangZhang,et al.Focal and efficient IOU loss for accurate bounding box regression[J].Neurocomputing,2022,506:146‒157. doi:10.1016/j.neucom.2022.07.042
[30]
ZhangHao, XuCong, ZhangShuaijie.Inner-IoU:More effective intersection over union loss with auxiliary bounding box[EB/OL].(2023‒11‒06)[2024‒04‒08].
[31]
PanWei, WeiChao, QianChunyu,et al.Improved YOLOv8s model for small object detection from perspective of drones[J].Computer Engineering and Applications,2024,60(9):142‒150. doi:10.3778/j.issn.1002-8331.2312-0043
ChengDeqiang, XuJinyang, KouQiqi,et al.Lightweight network based on residual information for foreign body classification on coal conveyor belt[J].Journal of China Coal Society,2022,47(3):1361‒1369. doi:10.13225/j.cnki.jccs.XR21.1736
LiuQinghua, YangXinyi, HaoJie,et al.Rice grain detection based on YOLO v7 fusing of GhostNetV2[J].Transactions of the Chinese Society for Agricultural Machinery,2023,54(12):253‒260.