Research on an Autonomous UGV Inspection System Integrating Trajectory-prediction-based Dynamic Obstacle Avoidance and Intelligent Rock-mass Grade Classification for Deep Underground Engineering
1.State Key Laboratory of Intelligent Construction and Healthy Operation and Maintenance of Deep Underground Engineering, Sichuan University, Chengdu 610065, China
2.School of Water Resources and Hydropower, Sichuan University, Chengdu 610065, China
3.College of Electrical Engineering, Sichuan University, Chengdu 610065, China
Objective Rock-mass grade assessment at the tunnel face provides an essential basis for excavation-scheme optimization, support-parameter adjustment, and construction-risk control in deep underground engineering. Manual assessment based on close-range geological observation, field sketching, empirical judgment, and standardized evaluation is affected by operational risks, low efficiency, subjectivity, and inconsistent judgment. Autonomous unmanned ground vehicle (UGV) inspection reduces personnel exposure and supports close-range tunnel-face information acquisition. However, deep underground tunnel environments involve Global Navigation Satellite System (GNSS) denial, narrow spaces, dust interference, low illumination, and dynamic obstacles such as workers and construction equipment. These conditions require the UGV to predict the future motion of dynamic obstacles, avoid potential collisions, acquire tunnel-face images, and provide reliable visual inputs for rock-mass grade classification. This study develops an autonomous UGV inspection framework combining multimodal trajectory prediction, prediction-based dynamic obstacle avoidance, tunnel-face image acquisition, intelligent rock-mass grade classification, and explainable result visualization. Methods A multimodal trajectory prediction network, named the Endpoint-Prototype-Guided Trend-State Hybrid Network (EP-TSHNet), was constructed to predict dynamic-obstacle motion. Historical trajectories were transformed into relative-displacement sequences. For the ETH-UCY and SDD benchmarks, the prediction setting adopted 8 observed frames, 12 future frames, and 20 candidate trajectories. EP-TSHNet included a trend-state temporal encoder, a hybrid graph interaction encoder, and an endpoint-prototype-guided hierarchical multimodal decoder. The trend-state temporal encoder extracted local displacement changes, velocity changes, acceleration changes, and terminal motion trends from historical trajectories. The hybrid graph interaction encoder modeled spatial-proximity relationships and motion-feature similarity relationships among dynamic targets. The endpoint-prototype-guided hierarchical multimodal decoder generated representative future endpoint prototypes, branch-level motion variations, anchor trajectories, residual refinements, and modal probabilities. The training objective combined multimodal trajectory-distribution loss, endpoint auxiliary supervision, mean-displacement auxiliary supervision, endpoint-prototype matching, and prototype-separation constraints. Based on EP-TSHNet, a trajectory-prediction-based dynamic obstacle-avoidance method, EP-TSHNet-PRA, was designed. Dynamic targets were detected and tracked from laser-sensing data, and their historical trajectories were used for online trajectory prediction. The predicted trajectories were transformed into look-ahead risk information and incorporated into local cost-map updating, collision-risk corridor assessment, and safe-velocity constraints. Dynamic obstacle-avoidance experiments were conducted on a Robot Operating System (ROS)/Gazebo tunnel-simulation platform using a Husky A200 UGV. Low-, medium-, and high-density dynamic-obstacle scenarios were constructed, and a closed-loop inspection task was designed, including start-point departure, tunnel dynamic obstacle avoidance, tunnel-face target-region arrival, image acquisition, return-trip obstacle avoidance, and final arrival at the starting point. For tunnel-face rock-mass grade classification, a confusion-aware RepViT-based rock classifier, CA-RepViT-Rock, was developed. The network used RepViT-Rock as the feature-extraction backbone and incorporated multi-scale depthwise convolution, the Convolutional Block Attention Module (CBAM), a multi-pooling cosine classification head, and confusion-category refinement branches. A tunnel-face image dataset from an under-construction tunnel project in northwest China was used. The dataset contained 1 800 high-resolution original tunnel-face images of four rock-mass grades, including Ⅲ-1, Ⅲ-2, Ⅳ, and Ⅴ. The original images were first grouped according to their source tunnel faces and then divided into training, validation, and test sets at a ratio of 70%, 20%, and 10%, corresponding to 1 260, 360, and 180 original images, respectively. Local image patches were subsequently extracted separately within each subset from the effective rock-mass regions and subjected to unified size preprocessing before being fed into the network. Each local image patch inherited the expert-reviewed rock-mass grade label of its source original image. The 180 original test images yielded 678 test patches. Quantitative classification metrics and the confusion matrix were calculated on the 678 test patches. Separately, representative original tunnel-face images from the test set were preprocessed and fed into the trained model for qualitative classification and interpretability analysis. Intermediate-layer feature responses, Gradient-weighted Class Activation Mapping (Grad-CAM), and Grad-CAM++ visualizations were generated from these original images. The original-image-level results were used only for qualitative analysis and were not included in the patch-level performance statistics. Results and Discussions EP-TSHNet achieved minADE20/minFDE20 values of 0.17/0.30 on ETH-UCY and 7.66/13.20 on the Stanford Drone Dataset (SDD), demonstrating its applicability to pedestrian-interaction and aerial-view complex-motion scenarios. On SDD, compared with TDAGCN, EP-TSHNet reduced minADE20 and minFDE20 by approximately 12.06% and 13.39%, respectively. The ablation experiment on SDD showed that removing any key module or constraint degraded prediction performance. Removing the endpoint-prototype-guided hierarchical multimodal decoder increased minADE20/minFDE20 from 7.66/13.20 to 8.76/15.34, demonstrating the importance of endpoint prototypes and branch expansion for multimodal-intention coverage. Removing the hybrid graph interaction encoder increased the errors to 8.21/14.44, indicating the contribution of target-interaction modeling. Removing the trend-state temporal encoder, endpoint auxiliary supervision, mean-displacement auxiliary supervision, or the prototype-separation constraint also increased the prediction errors. In the dynamic obstacle-avoidance experiments, the conventional method without look-ahead prediction achieved success rates of 80.0%, 65.0%, and 35.0% in low-, medium-, and high-density scenarios, respectively, with an average success rate of 60.0%. EP-TSHNet-PRA achieved success rates of 100.0%, 95.0%, and 75.0% under the same conditions, increasing the average success rate to 90.0%. The results showed that the predicted-trajectory cost regions and safe-velocity constraints enabled the local planner to avoid future occupied areas before dynamic obstacles entered the immediate collision region. In the closed-loop inspection task, the UGV completed autonomous navigation, dynamic obstacle avoidance, tunnel-face image acquisition, and return navigation under varying dynamic-obstacle densities and dust interference. For patch-level rock-mass grade classification, the 678 test patches comprised 77, 139, 385, and 77 patches of grades Ⅲ-1, Ⅲ-2, Ⅳ, and Ⅴ, respectively. The corresponding numbers of correctly classified patches were 75, 119, 365, and 70, yielding 629 correctly classified patches and an overall patch-level Accuracy of 0.927 7. CA-RepViT-Rock achieved a Macro-F1 of 0.921 1 and a Weighted-F1 of 0.927 5. The patch-level F1-scores for grades Ⅲ-1, Ⅲ-2, Ⅳ, and Ⅴ were 0.967 7, 0.875 0, 0.944 4, and 0.897 4, respectively. The normalized patch-level confusion matrix showed correct recognition rates of 97%, 86%, 95%, and 91% for grades Ⅲ-1, Ⅲ-2, Ⅳ, and Ⅴ, respectively. Misclassifications mainly occurred between adjacent grades, reflecting the continuous transition and indistinct boundaries of local rock-mass visual features. Qualitative classification examples on representative original tunnel-face images showed that the selected images were correctly classified into their expert-reviewed grades. Grad-CAM and Grad-CAM++ visualizations generated from these original images showed that the model primarily focused on rock texture, fractures, broken zones, structural planes, and texture-transition regions. These original-image-level classification and visualization results were analyzed separately from the patch-level quantitative evaluation. Conclusions The results demonstrate that EP-TSHNet provides multimodal trajectory prediction for dynamic obstacles by integrating short-term motion-trend extraction, target-interaction modeling, and endpoint-prototype-guided multimodal decoding. EP-TSHNet-PRA transforms predicted trajectories into look-ahead risk information for local cost-map updating, collision-risk assessment, and safe-velocity control, improving UGV dynamic obstacle avoidance in tunnel-simulation scenarios. CA-RepViT-Rock achieved effective patch-level tunnel-face rock-mass grade classification, while separate qualitative analysis on representative original tunnel-face images illustrated its classification responses and attention regions through intermediate-layer feature responses and Grad-CAM++. The developed study framework combines autonomous UGV navigation, dynamic obstacle avoidance, and tunnel-face image acquisition with separately validated patch-level rock-mass grade classification and original-image-level interpretability analysis, providing a basis for future on-site closed-loop integration. The results provide a methodological foundation for autonomous UGV inspection, safe tunnel navigation, close-range tunnel-face information acquisition, and intelligent rock-mass grade assessment in deep underground engineering.
式中: H 和 C 分别表示sLSTM输出的最终隐藏状态和最终记忆状态,用于混合图交互编码器中的目标状态交互增强;表示由隐藏状态 H 经多层感知机残差映射得到的最终时间编码特征,用于后续终点辅助监督、平均位移辅助监督和分层多模态解码。通过上述设计,趋势状态时间编码器能够同时表征局部位移变化、近期运动趋势和时间记忆依赖。
XieHeping, ZhangRu, DengJianhui,et al.A preliminary study on the technical system of deep earth science and geo disaster prevention-control based on the"deep earth‒surface"linkage strategy[J].Advanced Engineering Sciences,2021,53(4):1‒12.
XieHeping, ZhangRu, ZhangZetian,et al.Reflections and explorations on deep earth science and deep earth engineering technology[J].Journal of China Coal Society,2023,48(11):3959‒3978.
ZhangShishu, ZhaoXiaoping.Concept and key technical challenges of deep underground hydraulic and hydropower engineering[J].Advanced Engineering Sciences,2026,58(1):1‒17.
ChenWeidong, LiTianbin, HuangYinhao,et al.Intelligent identification of classification features of tunnel surrounding rock and visualization[J].Journal of Railway Science and Engineering,2024,21(1):406‒421.
ShenYanjun, YouLyu, LiShuguang,et al.Research progress of tunnel surrounding rock classification method and application trend of "artificial intelligence+"[J].Tunnel Construction,2023,43(4):563‒582.
ZhangRu, YouLyu, ZhangZetian,et al.Intelligent UAV inspection for the full lifecycle of deep underground engineering:State-of-the-art,challenges, and future directions[J].Journal of China University of Mining & Technology,2025,54(6)1326‒1356.
McClymondsA D, StephansT, AsadiS,et al.Evaluation of parameters that impact data collection for robot progress detection of a masonry wall[J].Journal of Computing in Civil Engineering,2026,40(2):04025150. doi:10.1061/jccee5.cpeng-6815
[14]
QinChengshuai, MaChuanyi, LiLiping,et al.Development and application of an intelligent robot for rock mass structure detection:A case study of Letuan tunnel in Shandong,China[J].International Journal of Rock Mechanics and Mining Sciences,2023,169:105419. doi:10.1016/j.ijrmms.2023.105419
[15]
ZapataS, UrvinaR, AroK,et al.Una estrategia híbrida de aprendizaje por refuerzo informada por RRT* para la planificación de caminos de robots móviles en minería a cielo abierto[J].Revista Iberoamericana de Automática e Informática Industrial,2024,22(1):57‒68. doi:10.4995/riai.2024.21581
[16]
ZhangZe, HajieghraryH, DeanE,et al.Prescient collision-free navigation of mobile robots with iterative multimodal motion prediction of dynamic obstacles[J].IEEE Robotics and Automation Letters,2023,8(9):5488‒5495. doi:10.1109/lra.2023.3296333
[17]
ZhangYue, WangJing, WangDingheng,et al.Dynamic obstacle avoidance method for unmanned vehicle based on pedestrian trajectory prediction[J].Systems Engineering and Electronics,2025,47(7):2371‒2382. doi:10.12305/j.issn.1001-506X.2025.07.29
QianZenghui, ChenRuoyang, YiChangyan,et al.Collision avoidance control for autonomous driving with multiple dynamic obstacles in IoV:A prediction-enhanced APF-based approach[J].IEEE Internet of Things Journal,2025,12(13):24968‒24984. doi:10.1109/jiot.2025.3556450
[20]
ZhaoMinghui, GaoPan, ZhangLipeng,et al.Emergency obstacle avoidance trajectory planning under dynamics characteristics constraints of corner module architecture intelligent electric vehicle[J].Robotics and Autonomous Systems,2026,196:105238. doi:10.1016/j.robot.2025.105238
[21]
ZhangKai, GaoRuizhen, ZhangJingjun.Research on trajectory tracking and obstacle avoidance of nonholonomic mobile robots in a dynamic environment[J].Robotics,2020,9(3):74. doi:10.3390/robotics9030074
[22]
MaJunjie, LiTianbin, YangGang,et al.A real-time intelligent classification model using machine learning for tunnel surrounding rock and its application[J].Georisk:Ass-essment and Management of Risk for Engineered Systems and Geohazards,2023,17(1):148‒168. doi:10.1080/17499518.2023.2182891
[23]
SunHuaiyuan, ZhuMengqi, DaiYiming,et al.Digital twin of advanced geological conditions in rock tunnel construction[J].Automation in Construction,2025,178:106407. doi:10.1016/j.autcon.2025.106407
[24]
WangMingnian, ZhaoSiguang, TongJianjun,et al.Intelligent classification model of surrounding rock of tunnel using drilling and blasting method[J].Underground Space,2021,6(5):539‒550. doi:10.1016/j.undsp.2020.10.001
[25]
HuangHongwei, ChenJiayao.Machine vision-based study on intelligent rating and excavation safety risk assessment of rock tunnel[J].Journal of Basic Science and Engineering,2023,31(6):1382‒1409.
WangMingnian, XiaQinyong, LinPeng,et al.An intelligent classification model of surrounding rock for large cross-section tunnel based on the information fusion of drilling parameters and tunnel face images[J].China Railway Science,2025,46(3):103‒115.
NieYaoqi, ZhangQian, HouLili,et al.TBM rock mass classification using XGBoost and Interpretable Machine learning[J].Advanced Engineering Informatics,2025,66:103459. doi:10.1016/j.aei.2025.103459
MangalamK, GiraseH, AgarwalS,et al.It is not the journey but the destination:Endpoint conditioned trajectory pr-ediction[C]//Computer Vision-ECCV 2020.Cham:Springer,2020:759‒776. doi:10.1007/978-3-030-58536-5_45
[33]
ShiLiushuai, WangLe, LongChengjiang,et al.SGCN:Sparse graph convolution network for pedestrian trajectory prediction[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).Nashville:IEEE,2021:8990‒8999. doi:10.1109/cvpr46437.2021.00888
[34]
PangBo, ZhaoTianyang, XieXu,et al.Trajectory prediction with latent belief energy-based model[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).Nashville:IEEE,2021:11809‒11819. doi:10.1109/cvpr46437.2021.01164
[35]
XuChenxin, MaoWeibo, ZhangWenjun,et al.Remember intentions:Retrospective-memory-based trajectory prediction[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).New Orleans:IEEE,2022:6478‒6487. doi:10.1109/cvpr52688.2022.00638
[36]
BaeI, ParkJ H, JeonH G.Non-probability sampling network for stochastic human trajectory prediction[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).New Orleans:IE-EE,2022:6467‒6477. doi:10.1109/cvpr52688.2022.00637
[37]
GuTianpei, ChenGuangyi, LiJunlong,et al.Stochastic trajectory prediction via motion indeterminacy diffusion[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).New Orleans:IEEE,2022:17092‒17101. doi:10.1109/cvpr52688.2022.01660
[38]
ChenYuxiao, IvanovicB, PavoneM.ScePT:Scene-consistent,policy-based trajectory predictions for planning[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).New Orleans:IEEE,2022:17082‒17091. doi:10.1109/cvpr52688.2022.01659
[39]
MaoWeibo, XuChenxin, ZhuQi,et al.Leapfrog diffusion model for stochastic trajectory prediction[C]//Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).Vancouver:IEEE,2023:5517‒5526. doi:10.1109/cvpr52729.2023.00534
[40]
LinXuanqi, ZhangYong, WangShun,et al.OST-HGCN:Optimized spatial-temporal hypergraph convolution network for trajectory prediction[J].IEEE Transactions on Intelligent Transportation Systems,2025,26(3):3056‒3070. doi:10.1109/tits.2025.3529666
[41]
SadeghianA, KosarajuV, SadeghianA,et al.SoPhie:An attentive GAN for predicting paths compliant to social and physical constraints[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).Long Beach:IEEE,2020:1349‒1358. doi:10.1109/cvpr.2019.00144
[42]
LiRuochen, KatsigiannisS, ShumH P H.Multiclass-SGCN:Sparse graph-based trajectory prediction with agent class embedding[C]//Proceedings of the 2022 IEEE International Conference on Image Processing(ICIP).Bordeaux:IEEE,2022:2346‒2350. doi:10.1109/icip46576.2022.9897644
[43]
DendorferP, OšepA, Leal-TaixéL.Goal-GAN:Multimodal trajectory prediction based on goal position estimation[C]//Computer Vision-ACCV 2020.Cham:Springer,2021:405‒420. doi:10.1007/978-3-030-69532-3_25
[44]
DeoN, TrivediM M.Trajectory forecasts in unknown environments conditioned on grid-based plans[EB/OL].[2026-05-12].doi:10.1109/ivs.2018.8500493
[45]
LiGuanlue, LuoGuiyang, YuanQuan,et al.Trajectory prediction with heterogeneous graph neural network[M]//PRICAI 2022:Trends in Artificial Intelligence.Cham:Springer Nature Switzerland,2022:375‒387. doi:10.1007/978-3-031-20865-2_28
ShiMengnan, ChenYun, GaoTianyu,et al.Accurate and reliable vision-based tunnel face rock mass classification by explainable deep learning[J].Tunnelling and Underground Space Technology,2026,168:107233. doi:10.1016/j.tust.2025.107233
[48]
ZhangRu, YouLyu, ZhangZetian,et al.Development and prospect of multidimensional information perception and intelligent construction in deep earth engineering[J].Journal of China Coal Society,2024,49(3):1259‒1290.
TranzattoM, DharmadhikariM, BernreiterL,et al.CER-BERUS in the DARPA Subterranean Challenge[J].Science Robotics,2022,7(66):eabp9742. doi:10.1126/scirobotics.abp9742
[51]
XieJ, ChenB, JiangS H,et al.Enhancing data reuse in tunnelling site investigation through transfer learning-based historical data mining[J].Underground Space,2025,23:161‒174. doi:10.1016/j.undsp.2025.02.003