面向移动机器人的多传感器紧耦合导航定位方法

A tightly coupled multi-sensor fusion navigation and localization method for mobile robots

摘要

移动机器人依赖单一传感器往往难以克服光照变化、外部干扰、反射表面影响以及累积误差等问题，限制了环境感知能力和自身位姿测量的精度与可靠性。该文采用一种非线性优化的方法，实现（IMU、红外相机、RGB相机、激光雷达）数据层面紧耦合组合定位建图系统IIVL-LM。提出一种基于RGB图像信息的实时照度值转换模型，系统根据不同照度值通过非线性插值法输入视觉SLAM模型中进行实时建图，然后通过动态加权法对红外相机与RGB相机的关键帧特征提取融合。在模拟的室内救援场景数据集下，与多种主流融合定位方法相比，IIVL-LM在照度变化的苛刻条件下尤其是在低照度下性能提升明显，平均RMSE ATE提升了23%～39%（0.006～0.013）。IIVL-LM保证了系统始终会在不少于3个传感器有效的状态下进行，在确保精度的同时对未知开放场景有更强的鲁棒性，尤其对于室内救援这种复杂场景的应用具有一定的价值。

Abstract

Mobile robots relying solely on a single sensor often struggle to overcome challenges such as illumination change, external disturbances, effects of reflective surfaces, and cumulative errors, which limit their environmental perception capabilities as well as the accuracy and reliability of pose estimation. This article adopts a nonlinear optimization method to achieve a tightly coupled integrated localization and mapping system, IIVL-LM, at the data level (IMU, infrared camera, RGB camera, LiDAR). A real-time luminance conversion model based on RGB image information is proposed. The system incorporates varying luminance values into the visual SLAM model through nonlinear interpolation for real-time mapping, then fuses the feature extraction of key frames from the infrared camera and the RGB camera through dynamic weighting. In a simulated indoor rescue scenario dataset, compared to various mainstream fusion positioning methods, the IIVL-LM system exhibits a notable performance improvement under challenging luminance conditions, especially in low-light environments. The average Root Mean Square Error (RMSE) of the Absolute Trajectory Error (ATE) improved by 23% to 39% (0.006 to 0.013). The IIVL-LM system ensures that it operates with at least three active sensors at all times, thereby enhancing its robustness in unknown and open environments while maintaining precision. This capability is particularly valuable for applications in complex settings such as indoor rescue scenarios.

关键词

移动机器人 / 多传感器融合 / 照度转换 / 非线性紧耦合 / SLAM

Key words

mobile robots / multi-sensor fusion / illuminance conversion / nonlinear tight coupling / SLAM

参考文献

[1] AOYAMA Y, SARAVANOS A D, THEODOROU E A. Receding horizon differential dynamic programming under parametric uncertainty[EB/OL]. [2023—10—10].https://arxiv.org/pdf/2104.10836.

[2] SHI Y L, ZHANG W M, YAO Z, et al. Design of a hybrid indoor location system based on multi—sensor fusion for robot navigation[J].Sensors, 2018, 18(10): 3581.

[3] SHEN S J, MULGAONKAR Y, MICHAEL N, et al. Multi—sensor fusion for robust autonomous flight in indoor and outdoor environments with a rotorcraft MAV[C]//Proceedings of the IEEE International Conference on Robotics and Automation. New York: IEEE, 2014: 4974-4981.

[4] CHEN L, LI G, ZHAO K Q, et al. A perceptually adaptive long—term tracking method for the complete occlusion and disappearance of a target[J].Cognitive Computation, 2023, 15(6): 2120-2131.

[5] CHEN L, LI G, XIE W S, et al. A survey of computer vision detection, visual SLAM algorithms, and their applications in energy—efficient autonomous systems[J].Energies, 2024, 17(20): 5177.

[6] CHEN C, ZHU H, LI M G, et al. A review of visual—inertial simultaneous localization and mapping from filtering—based and optimization—based perspectives[J].Robotics, 2018, 7(3): 45.

[7] LE GENTIL C, VIDAL—CALLEJA T, HUANG S D. IN2LAMA: INertial lidar localisation and MApping[C]//Proceedings of the International Conference on Robotics and Automation. Montreal, QC: IEEE, 2019: 6388-6394.

[8] ZHANG J, SINGH S. Low—drift and real—time lidar odometry and mapping[J].Autonomous Robots, 2017, 41(2): 401-416.

[9] LYNEN S, ACHTELIK M W, WEISS S, et al. A robust and modular multi—sensor fusion approach applied to MAV navigation[C]//Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems. New York: IEEE, 2013: 3923-3929.

[10] HENING S, IPPOLITO C A, KRISHNAKUMAR K S, et al. 3D LiDAR SLAM integration with GPS/INS for UAVs in urban GPS—degraded environments[C]//Proceedings of the AIAA Information Systems—AIAA Infotech @ Aerospace. Reston, Virginia: AIAA, 2017: 0448.

[11] DAVISON A J. Real—time simultaneous localisation and mapping with a single camera[C]//Proceedings of the Proceedings 9th IEEE International Conference on Computer Vision. New York: IEEE, 2003: 1403-1410.

[12] CIVERA J, DAVISON A J, MARTÍNEZ MONTIEL J M. Inverse depth parametrization for monocular SLAM[J].IEEE Transactions on Robotics, 2008, 24(5): 932-945.

[13] KLEIN G, MURRAY D. Parallel tracking and mapping for small AR workspaces[C]//Proceedings of the 6th IEEE and ACM International Symposium on Mixed and Augmented Reality. New York: IEEE, 2007: 225-234.

[14] CAMPOS C, ELVIRA R, RODRÍGUEZ J J G, et al. ORB—SLAM3: An accurate open—source library for visual, visual—inertial, and multimap SLAM[J].IEEE Transactions on Robotics, 2021, 37(6): 1874-1890.

[15] AGARWAL S, KEIR M. Ceres solver: Tutorial & reference[EB/OL]. [2023—10—11].https://www.helloandroid.cn/android/4.3_r1/download/external/ceres—solver/docs/ceres—solver.pdf.

[16] KASAR A. Benchmarking and comparing popular visual SLAM algorithms[EB/OL]. [2024—11—24].https://arxiv.org/abs/1811.09895.

[17] MUR—ARTAL R, TARDÓS J D. Visual—inertial monocular SLAM with map reuse[J].IEEE Robotics and Automation Letters, 2017, 2(2): 796-803.

[18] QIN T, LI P L, SHEN S J. VINS—mono: A robust and versatile monocular visual—inertial state estimator[J].IEEE Transactions on Robotics, 2018, 34(4): 1004-1020.

[19] LIN J R, ZHANG F. R3LIVE++: A Robust, real—time, radiance reconstruction package with a tightly—coupled LiDAR—Inertial—Visual state estimator[EB/OL]. [2024—11—08].https://arxiv.org/abs/2209.03666.

[20] ENGEL J, KOLTUN V, CREMERS D. Direct sparse odometry[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(3): 611-625.

[21] FORSTER C, ZHANG Z C, GASSNER M, et al. SVO: Semidirect visual odometry for monocular and multicamera systems[J].IEEE Transactions on Robotics, 2017, 33(2): 249-265.

基金资助

人才启航计划(2025QHXM24)