To address the challenges faced by cameras and LiDAR sensors in low-light, rainy, and snowy conditions, such as difficulties in perception, low target detection accuracy, and missed detections, a target detection scheme that fuses images and LiDAR point clouds was adopted. A dynamic weight sensor fusion method based on fuzzy theory was introduced, which dynamically allocates fusion weights by recognizing different weather and lighting conditions to achieve optimal detection performance. For rainy, snowy, and low-light environments, four experimental scenarios were designed and conducted: rain, snow, low light, and oncoming vehicles. The results show that the LiDAR and camera fusion algorithm effectively reduces the number of missed detections in unfavorable weather, improving the vehicle’s perception capability under rain, snow, nighttime conditions, as well as in environments with oncoming vehicles and frequent obstacles. The results also validate the effectiveness of the proposed fuzzy theory based dynamic weight sensor fusion method.
XiangC, FengC, XieX P, et al. Multi-sensor fusion and cooperative perception for autonomous driving: a review[J]. IEEE Intelligent Transportation Systems Magazine, 2023, 15(5): 36-58.
[2]
LiuZ, CaiY F, WangH, et al. Robust target recognition and tracking of self-driving cars with radar and camera information fusion under severe weather conditions[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(7): 6640-6653.
[3]
BijelicM, GruberT, MannanF, et al. Seeing through fog without seeing fog: deep multimodal sensor fusion in unseen adverse weather[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, 2020: 11679-11689.
[4]
MendezJ, MolinaM, RodriguezN, et al. Camera-LiDAR multi-level sensor fusion for target detection at the network edge[J]. Sensors, 2021, 21(12): 3992.
[5]
SongY Y, XieZ Y, WangX W, et al. MS-YOLO: object detection based on YOLOv5 optimized fusion millimeter-wave radar and machine vision[J]. IEEE Sensors Journal, 2022, 22(15): 15435-15447.
[6]
日本工业标准调查会. 照度基准总则: JIS Z 9110 [S]. 东京: 日本工业标准调查会, 2024.
[7]
Japanese Industrial standards Commitee. General rules for illuminance standards: JIS Z 9110 [S]. Tokyo: Japanese Industrial standards Committee. 2024.
[8]
WangC Y, BochkovskiyA, LiaoH M. YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver, 2023: 7464-7475.