This article initially provides an overview of the processes and standard regulations involved in the safety verification and validation of autonomous vehicles. Building upon the human-vehicle-road system theory, the article further introduces a novel classification approach, categorizing and summarizing the current technologies and assessment standards for safety verification and validation in autonomous vehicles. It also consolidates and comparatively analyzes three major categories of methods: those based on real-world scenarios, virtual scenarios, and a combination of both. The article conducts a comparative evaluation of the limitations, advantages, and disadvantages of 16 different verification and validation methods across eight characteristic dimensions. Finally, it briefly extrapolates on the challenges and future prospects in the research of safety verification and validation schemes for autonomous vehicles.
自动驾驶技术的应用能够避免人为错误,增加交通安全性、方便特殊交通参与者出行并缓解交通拥堵状况,极大地提高未来交通系统的智能化水平[1]。然而,当前自动驾驶技术仍然未具备全天候全场景下的完全安全驾驶能力。在探索自动驾驶技术的边界时,面临一系列复杂且多元的安全挑战,比如,由于海量复杂交通场景与极端天气环境的未知性、自动驾驶系统感知认知能力与决策执行功能的局限性、自动驾驶系统数据隐私性、合理可预见的人为误用[2]和伦理法律的合理性等潜在危险因素。这些因素共同促生了一系列紧迫的安全问题,如预期功能安全问题[3](Safety of the intended functionality,SOTIF)、电子系统软硬件功能失效产生的功能安全问题[4](Functional safety,FuSa)、自动驾驶系统网络数据泄露产生的信息安全问题[5]、基于社会属性的交互安全问题以及关于法律责任界定的法律和道德伦理问题[6]等。自动驾驶车辆(Autonomous vehicle,AV)面临的这些多维度安全问题相互交织,增加了行驶风险,导致交通事故频率升高,严重威胁到驾驶者及相关人员的生命和财产安全。这些问题还引发了公众对自动驾驶技术的信任危机,进而在一定程度上阻碍了该技术的普及和推广。
自动驾驶车辆安全性提升的措施当前主要集中在技术开发层面,如感知融合、冗余决策与控制以及安全人机交互[3]等,对自动驾驶系统安全性验证和确认(Verification and validation,V&V)的方法论研究关注较少[4]。随着更高度自动驾驶系统在现实世界中的部署,面向现实交通环境以高覆盖度验证和确认自动驾驶车辆安全性至关重要。验证技术指在开发的各个阶段,从技术人员的角度,测试当前的开发成果主观上是否符合设计的规范[7]。确认技术是从用户的角度,测试当前的开发成果,客观上是否符合用户的真正需求。自动驾驶通过驾驶系统安全性的验证与确认,能够有效减少由于自动驾驶系统软硬件功能失效以及感知、决策局限或人机交互技术本身而导致的自动驾驶事故,达到已知不安全与未知不安全降到可容忍的FuSa和SOTIF驾驶决策目标[8-10],保障自动驾驶车辆在现实世界中拥有应对任意复杂的交通场景的能力。因此,自动驾驶车辆安全性的验证和确认也被定义为自动驾驶安全框架的基础。
(5)仅驾驶员要素为真实数据的验证和确认方法:该类方法采用真实驾驶员测试要素,结合虚拟试驾,驾驶员通过模拟器进行验证。如驾驶员在环(Driver in the loop,DIL)[46]、虚拟重构与模型孪生验证,是一种通过虚拟要素的手段,利用动态驾驶员模拟器(驾舱)、环境视听模拟设备及相关人车检测设备重现“人-车-环境”在实际车辆驾驶中的相互作用的新型测试系统。
(6)基于故障注入的仿真场景验证和确认方法:故障注入是一种针对计算和网络物理系统在故障情况下的弹性和错误处理能力[131]的测试方法,已被广泛应用于自动驾驶汽车的验证和验证过程[132]。这种方法通过故意引入可能在常规测试中难以触发的故障,有助于检验系统级的稳健性[133]。故障注入可以从软件和硬件两个方面进行,并且按照故障类型进一步细分为硬件故障和软件故障。例如,Juez等[134]提出了一种基于模拟的故障注入方法,将其应用于动态运动规划系统的一个基本子功能,即用于危害分析和风险评估的行为模型。这一方法展示了故障注入在提高系统健壮性和发现潜在漏洞方面的重要作用,对确保自动驾驶汽车的运行安全具有实际意义。一方面,Tian等[65]提出了一种基于故障注入和基于模型的系统工程的SoS(Systems of systems,SoS)自动测试生成方法。另一方面,Karunakaran[135]提出了一种基于强化学习的场景误证方法,旨在行人交通情境中发现高风险场景。这对自动驾驶车辆的安全性验证至关重要,尤其在高度自动化水平下,安全责任从驾驶员转移至系统。该方法采用基于英特尔责任敏感安全(RSS)、欧几里得距离和潜在碰撞距离的奖励函数,有助于在车辆上路前发现系统缺陷。
LiuXin. Ministry of Industry and Information Technology: regulating road testing and demonstration of intelligent connected vehicles[J]. China Equipment Engineering, 2021(16): 1.
[3]
HoS S. Complementary and competitive framing of driverless cars: framing effects, attitude volatility, or attitude resistance?[J]. International Journal of Public Opinion Research, 2021, 33(3): 512-531.
[4]
YurtseverE, LambertJ, CarballoA, et al. A survey of autonomous driving: common practices and emerging technologies[J]. IEEE Access, 2020, 8: 58443-58469.
[5]
KoopmanP, WagnerM. Autonomous vehicle safety: an interdisciplinary challenge[J]. IEEE Intelligent Transportation Systems Magazine, 2017, 9(1): 90-96.
BaoShuang, LiLi-xiang, PengHai-peng. Research on information security in intelligent internet of vehicles [J]. Information Security and Communications Privacy, 2023(3): 10-20.
ISO/PAS 21448—2019 .International Standard Organization Road Vehicles-Safety of the Intended Functionality [S].
[10]
ChenL, JiaoJ, ZhaoT. A novel hazard analysis and risk assessment approach for road vehicle functional safety through integrating STPA with FMEA[J]. Applied Sciences, 2020, 10: No. 7400.
[11]
WachenfeldW, WinnerH. The release of autonomous vehicles[J]. Autonomous Driving: Technical, Legal and Social Aspects, 2016(5): 425-449.
[12]
ISO/TR 4804—2020,Road vehicles—Safety and cybersecurity for automated driving systems—Design, verification and validation [S].
[13]
张鑫. 基于卡车行驶数据的驾驶行为分析评价系统研究[D].天津:天津大学机械工程学院, 2021.
[14]
ZhangXin. Research on driving behavior analysis and evaluation system based on truck driving data[D].Tianjin: School of Mechanical Engineering, Tianjin University, 2021.
FangYi-ge. Lightweight design of battery pack based on driving range and safety performance[D]. Changsha: School of Mechanical and Vehicle Engineering, Hunan University, 2021.
WangHui-ran. Research on key technologies of lane change control of autonomous vehicles based on expected functional safety[D]. Hefei: School of Automotive and Traffic Engineering, Hefei University of Technology, 2021.
XiaChun-yan, HuangSong, ZhengChang-you, et al. Modeling and verification methods of autonomous driving intersection test scenarios [J]. Journal of Software, 2023, 34(7): 3002-3021.
[23]
王殿林. 自动在线规约挖掘技术研究[D]. 长沙:国防科学技术大学研究生院, 2011.
[24]
WangDian-lin. Research on automatic online protocol mining technology[D]. Changsha: Graduate School of National University of Defense Technology, 2011.
[25]
PekC, ManzingerS, KoschiM, et al. Using online verification to prevent autonomous vehicles from causing accidents[J]. Nature Machine Intelligence, 2020(9): 518-528.
[26]
中华人民共和国国家标准. 质量管理体系基础和术语[J]. 质量春秋, 2009(6): 12.
[27]
Standardization Administration of China. Fundamentals and vocabulary of quality management systems [J]. Quality Spring and Autumn, 2009(6): 12.
LiuJia-xi, GuoHui, LiJun. Functional safety standard ISO 26262 for automotive electrical and electronic systems[J]. Shanghai Automobile, 2011(10): 57-61.
[32]
赵晨熙. 《道路交通安全法》大修[J]. 人民交通, 2021(10): 54-56.
[33]
ZhaoChen-xi. Major revision of the Road Traffic Safety Law[J]. People's Transportation, 2021(10): 54-56.
[34]
LeungK, SchmerlingE, ZhangM, et al. On infusing reachability-based safety assurance within planning frameworks for human-robot vehicle interactions[J]. The International Journal of Robotics Research, 2020, 39(10/11): 1326-1345.
[35]
叶强. 德国自动驾驶立法评析[J]. 国外社会科学, 2022(2): 73-86, 197.
[36]
YeQiang. Review of legislation on autonomous driving in Germany [J]. Foreign Social Sciences, 2022(2): 73-86, 197.
[37]
SmallM M, JohnstonI. Lane keeping systems for light vehicles: draft ADR 107/00[R]. Canberra: Australasian College of Road Safety, 2022.
[38]
TenbrockA, KönigA, KeutgensT, et al. The conscend dataset: concrete scenarios from the highd dataset according to alks regulation unece r157 in openx[C]∥IEEE Intelligent Vehicles Symposium Workshops, Beijing, 2021: 174-181.
LiPei-ying. Interpretation and response to the opinions on strengthening the access management of intelligent connected vehicle manufacturers and products[J]. Cybersecurity and Informatization, 2021(9): 47.
SunHang, ZhangLu, JiGuo-tian. Research and prospects of the standard system and key standards for intelligent connected vehicles[J]. Journal of Automotive Safety and Energy, 2024, 15(6): 795-812.
[47]
SAE J3016_201401, Taxonomy and definitions for terms related to on-road motor vehicle automated driving systems [S].
[48]
RiedmaierS, PonnT, LudwigD, et al. Survey on scenario-based safety assessment of automated vehicles[J]. IEEE Access, 2020, 8: 87456-87477.
[49]
KoopmanP. UL 4600: What to include in an autonomous vehicle safety case[J]. Computer, 2023, 56(5): 101-104.
[50]
InternationalSAE. SAE international releases updated J3018™ Standard for On-Road Testing of SAE Level 3, 4 and 5 Prototype Automated Driving System (ADS)[R]. Warrendale: Newswire P, 2019.
[51]
ForestThomas M. Cybersecurity Guidebook for Cyber-Physical Vehicle Systems[M]. New York:SAE International, 2016.
[52]
van Den BergJ, FergusonD, KuffnerJ. Anytime path planning and replanning in dynamic environments[C]∥Proceedings of IEEE International Conference on Robotics and Automation, Shanghai, China, 2006: 2366-2371.
[53]
SchmidtC, OechsleF, BranzW. Research on trajectory planning in emergency situations with multiple objects[C]∥IEEE Intelligent Transportation Systems Conference, Toranto, Canada, 2006: 988-992.
[54]
BMVI. Germany will be the World Leader in Autonomous Driving[R]. Federal Ministry of Transport and Digital Infrastructure, Germany, 2021.
[55]
Shalev-ShwartzS, ShammahS, ShashuaA. On a formal model of safe and scalable self-driving cars[J/OL].[2023-08-22].
[56]
WangX, LiuX, ZhongF, et al. A scenario generation method for typical operations of power systems with pv integration considering weather factors[J]. Sustainability, 2023, 15(20): No.15007.
[57]
Shalev-ShwartzS, ShammahS, ShashuaA. On a formal model of safe and scalable self-driving cars[J/OL].[2023-08-22].
[58]
DavidN, Hon-LeungL, NgJulia, et al. An Introduction to the Safety Force Field[M]. City of Santa Clara: NVIDIA Corporation, 2019.
ZhuBing, ZhangPei-xing, ZhaoJian, et al. Research progress of scenario-based virtual testing for autonomous vehicles[J]. China Journal of Highway and Transport, 2019,32(6): 1-19.
[67]
XuX, WangX, WuX, et al. Calibration and evaluation of the responsibility-sensitive safety model of autonomous car-following maneuvers using naturalistic driving study data[J]. Transportation Research Part C: Emerging Technologies, 2021, 123:No.102988.
[68]
StraussM, MitschS. Slow down, move over: a case study in formal verification, refinement, and testing of the responsibility-sensitive safety model for self-driving cars[C]∥The 17th International Conference on Tests and Proofs,Leicester, UK, 2023: 149-167.
[69]
WangX, YeC, QuddusM, et al. Pedestrian safety in an automated driving environment: calibrating and evaluating the responsibility-sensitive safety model[J]. Accident Analysis and Prevention, 2023, 192: No.107265.
[70]
SukH, KimT, ParkH, et al. Rationale-aware autonomous driving policy utilizing safety force field implemented on CARLA simulator[J/OL].[2023-08-22].
[71]
JiX, CuiX, LiZ, et al. Research and evaluation of the allosteric protein-specific force field based on a pre-training deep learning model[J]. Journal of Chemical Information and Modeling, 2023, 63(8):2456-2468.
LiuXiu-fang, CaoKai, YangXu, et al. Safety verification method for autonomous vehicles based on reachable sets[J]. Journal of Guangxi University (Natural Science Edition), 2018, 43(2): 488-497.
[74]
AlthoffM, FrehseG, GirardA. Set propagation techniques for reachability analysis[J]. Annual Review of Control, Robotics, and Autonomous Systems, 2021, 4(1): 369-395.
[75]
WetzlingerM, KulmburgA, AlthoffM. Adaptive parameter tuning for reachability analysis of nonlinear systems [C]∥Proceedings of the 24th International Conference on Hybrid Systems, Nashville Tennessee, USA, 2021: 1-11.
[76]
WetzlingerM, KulmburgA, Le PenvenA, et al. Adaptive reachability algorithms for nonlinear systems using abstraction error analysis[J]. Nonlinear Analysis: Hybrid Systems, 2022, 46: No.101252.
[77]
邱锡鹏. 神经网络与深度学习[M]. 北京: 电子工业出版社, 2020: 448.
[78]
LeCunY. 1.1 Deep learning hardware: past, present, and future[C]∥IEEE International Solid-State Circuits Conference-(ISSCC),San Francisco, USA, 2019: 12-19.
[79]
RibeiroC, PintoT, SilvaM, et al. Data mining approach for decision support in real data based smart grid scenario[C]∥The 26th International Workshop on Database and Expert Systems Applications (DEXA), Valencia, Spain, 2015: 73-77.
[80]
YangY, YuanZ, MengR. Exploring traffic crash occurrence mechanism toward cross-area freeways via an improved data mining approach[J]. Journal of Transportation Engineering, Part A: Systems, 2022, 148(9): No.04022052.
[81]
YichaoZ, ZhendaH, LinyaoZ, et al. Study on scenario generation based on deep clustering and its application in supply-demand imbalance assessment[C]∥ The 8th Asia Conference on Power and Electrical Engineering(ACPEE), Tianjin, China, 2023: 1903-1910.
[82]
DuanJ, GaoF, HeY. Test scenario generation and optimization technology for intelligent driving systems[J]. IEEE Intelligent Transportation Systems Magazine, 2020, 14(1): 115-127.
[83]
GuoP, GaoF. Automated scenario generation and evaluation strategy for automatic driving system[C]∥ The 7th International Conference on Information Science and Control Engineering(ICISCE),2020: 1722-1733.
[84]
TianY C, JingD L. Towards an automatic test generation method for systems of systems based on fault injection and model-based systems engineering[J]. Applied Sciences, 2022,12(22): No.11863.
ChenQi. The Guidelines for access management of enterprises and products of intelligent connected vehicles: a clarion call for industrial reform[J]. Automobile and Parts, 2021(11): 28-29.
[87]
GassmannB, OborilF, BuerkleC, et al. Towards standardization of AV safety: C++ library for responsibility sensitive safety[C]∥IEEE Intelligent Vehicles Symposium (IV), Paris, France, 2019:2265-2271.
[88]
HekmatnejadM, YaghoubiS, DokhanchiA, et al. Encoding and monitoring responsibility sensitive safety rules for automated vehicles in signal temporal logic[C]∥Proceedings of the 17th ACM-IEEE International Conference on Formal Methods and Models for System Design, 2019: 1-11.
[89]
SalayR, CzarneckiK, ElliM S, et al. PURSS: towards perceptual uncertainty aware responsibility sensitive safety with ML[C]∥Workshop on Artificial Intelligence Safety, New York,USA, 2020:91-95.
[90]
ChaiC, ZengX, WuX, et al. Evaluation and optimization of responsibility-sensitive safety models on autonomous car-following maneuvers[J]. Transportation Research Record, 2020, 2674(11): 662-673.
[91]
LiuS, WangX, HassaninO, et al. Calibration and evaluation of responsibility-sensitive safety (RSS) in automated vehicle performance during cut-in scenarios[J]. Transportation Research Part C: Emerging Technologies, 2021, 125: No.103037.
[92]
TranD Q, BaeS H. Improved responsibility-sensitive safety algorithm through a partially observable Markov decision process framework for automated driving behavior at non-signalized intersection[J]. International Journal of Automotive Technology, 2021, 22(2): 301-314.
LuoSi-qing, ChenHui. Image scene transformation based on generative adversarial networks[J]. Computer Engineering, 2023, 49(4): 217-225.
[95]
JunietzP, BonakdarF, KlamannB, et al. Criticality metric for the safety validation of automated driving using model predictive trajectory optimization[C]∥2018 21st International Conference on Intelligent Transportation Systems (ITSC), 2018: 60-65.
[96]
Kress-GazitH, EderK, HoffmanG, et al. Formalizing and guaranteeing human-robot interaction[J]. Communications of the ACM, 2021, 64(9): 78-84.
PfefferR, LeichsenringT. Continuous development of highly automated driving functions with vehicle-in-the-loop using the example of Euro NCAP scenarios[C]∥The 7th Conference on Simulation and Testing for Vehicle Technology, Berlin, Germany, 2016: 33-42.
[101]
TettamantiT, SzalaiM, VassS, et al. Vehicle-in-the-loop test environment for autonomous driving with microscopic traffic simulation[C]∥IEEE International Conference on Vehicular Electronics and Safety (ICVES), Madrid, Spain,2018: 1-6.
[102]
WengB, CapitoL, OzgunerU, et al. Towards guaranteed safety assurance of automated driving systems with scenario sampling: an invariant set perspective[J]. IEEE Transactions on Intelligent Vehicles,2021, 7(3): 638-651.
WuXiao-qian, LiGuo-xuan, LengLu-lu. Research on autonomous vehicle technology[J]. Automobile Testing Report, 2023(5): 40-42.
[105]
KoopmanP, WagnerM. Challenges in autonomous vehicle testing and validation[J]. SAE International Journal of Transportation Safety, 2016, 4(1): 15-24.
[106]
TumovaJ, HallG C, KaramanS, et al. Least-violating control strategy synthesis with safety rules[C]∥Proceedings of The 16th International Conference on Hybrid Systems: Computation and Control, Marrakech, Morocco, 2013: 1-10.
[107]
Kress-GazitH, FainekosG E, PappasG J. Temporal-logic-based reactive mission and motion planning[J]. IEEE Transactions on Robotics, 2009, 25(6): 1370-1381.
ChanN, KuffnerJ, ZuckerM. Improved motion planning speed and safety using regions of inevitable collision[C]∥The 17th CISM-IFToMM Symposium on Robot Design, Dynamics, and Control, Tokyo, Japan, 2008 :103-114.
[110]
DahlJ, de CamposG R, OlssonC, et al. Collision avoidance: a literature review on threat-assessment techniques[J]. IEEE Transactions on Intelligent Vehicles, 2018, 4(1): 101-113.
NieYin-yu. Research on data-driven vehicle dynamics modeling and simulation[D]. Chengdu: State Key Laboratory of Traction Power,Southwest Jiaotong University, 2016.
[113]
徐学进. 基于驾驶模拟器的车辆动力学建模研究[D].武汉: 武汉理工大学汽车工程学院, 2007.
[114]
XuXue-jin. Research on vehicle dynamics modeling based on driving simulator[D]. Wuhan: School of Automotive Engineering, Wuhan University of Technology, 2007.
[115]
HuangH M, PavekK, AlbusJ, et al. Autonomy levels for unmanned systems(ALFUS) framework: an update[J].Unmanned Ground Vehicle Technology VII,2005, 5804: 439-448.
ZhaoWen-bo. Research on camera-in-the-loop test method of pedestrian collision avoidance system for intelligent vehicles[D].Changchun: College of Automotive Engineering, Jilin University, 2021.
peng-chengXia-Song, PeiLing, ZhuYi-fan, et al. High-fidelity simulation method for multi-source fusion positioning based on GNSS hardware-in-the-loop [J]. Journal of China Inertial Technology, 2020, 28(2): 8-12.
HuXiao. Research on embedded software testing of EPS system using ECU-in-the-loop simulation[D].Changchun: College of Automotive Engineering, Jilin University, 2012.
[126]
DroliaU, WangZ, VemuriS, et al. Demo abstract: AutoPlug—an automotive test-bed for ECU testing, validation and verification[C]∥Proceedings of the 10th ACM/IEEE International Conference on Information Processing in Sensor Networks, Chicago, USA, 2011: 131-132.
[127]
ZhaoT, YurtseverE, PaulsonJ A, et al. Formal certification methods for automated vehicle safety assessment[J]. IEEE Transactions on Intelligent Vehicles, 2022,8(1): 232-249.
[128]
卜磊, 解定宝. 混成系统形式化验证[J]. 软件学报, 2014, 25(2): 15-25.
[129]
BuLei, XieDing-bao. Formal verification of hybrid systems[J]. Journal of Software, 2014, 25(2): 15-25.
[130]
DomeniciA, FagioliniA, PalmieriM. Integrated simulation and formal verification of a simple autonomous vehicle[C]∥Software Engineering and Formal Methods: SEFM 2017 Collocated Workshops: DataMod, FAACS, MSE, CoSim-CPS, and FOCLASA, Trento, Italy, 2017: 300-314.
LiuBing-zheng, CaoKai, MaJian-jun. Safety verification method for behavior decision-making of autonomous vehicles [J]. Journal of Shandong University of Technology(Natural Science Edition), 2011, 25(6): 7-10.
ZhangYa-shun. Formal verification method of hybrid system and its application[D].Hefei: School of Electrical Engineering and Automation, Hefei University of Technology, 2006.
[135]
MitchellI M. A toolbox of level set methods version 1.0[J]. Journal of Molecular Graphics & Modelling, 2004, 25(4): 442-454.
[136]
KiranB R, SobhI, TalpaertV, et al. Deep reinforcement learning for autonomous driving: a survey[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 23(6): 4909-4926.
[137]
LiL, LinY L, CaoD P, et al. Parallel learning-a new framework for machine learning[J]. Acta Automatica Sinica, 2017, 43(1): 1-8.
[138]
LyA O, AkhloufiM. Learning to drive by imitation: an overview of deep behavior cloning methods[J]. IEEE Transactions on Intelligent Vehicles, 2020, 6(2): 195-209.
[139]
MaY, WangZ, YangH, et al. Artificial intelligence applications in the development of autonomous vehicles: a survey[J]. IEEE/CAA Journal of Automatica Sinica, 2020, 7(2): 315-329.
[140]
ChauhanN K, SinghK. A review on conventional machine learning vs deep learning[C]∥International Conference on Computing, Power and Communication Technologies(GUCON), Beijing, China, 2018: 347-352.
[141]
SunC, SuL, GuS, et al. Cross validation for CNN based affordance learning and control for autonomous driving[C]∥IEEE Intelligent Transportation Systems Conference (ITSC), Auckland, New Zealand, 2019: 1519-1524.
[142]
Al-SharmanM, MurdochD, CaoD, et al. A sensorless state estimation for a safety-oriented cyber-physical system in urban driving: deep learning approach[J]. IEEE/CAA Journal of Automatica Sinica, 2020, 8(1): 169-178.
[143]
DingW, WangW, ZhaoD. A multi-vehicle trajectories generator to simulate vehicle-to-vehicle encountering scenarios[C]∥International Conference on Robotics and Automation (ICRA), Montreal,Canada, 2019: 4255-4261.
[144]
WangW, RameshA, ZhuJ, et al. Clustering of driving encounter scenarios using connected vehicle trajectories[J]. IEEE Transactions on Intelligent Vehicles, 2020, 5(3): 485-496.
[145]
KrajewskiR, MoersT, NergerD, et al. Data-driven maneuver modeling using generative adversarial networks and variational autoencoders for safety validation of highly automated vehicles[C]∥The 21st International Conference on Intelligent Transportation Systems(ITSC), Maui, USA, 2018: 2383-2390.
[146]
RoweJ, SmithA, PokornyB, et al. Toward automated scenario generation with deep reinforcement learning in GIFT[C]∥Proceedings of the Sixth Annual GIFT User Symposium,Orlando,USA, 2018: 65-74.
[147]
LittmanM L. Reinforcement learning improves behaviour from evaluative feedback[J]. Nature, 2015, 521(7553): 445-451.
[148]
LiN, OylerD W, ZhangM, et al. Game theoretic modeling of driver and vehicle interactions for verification and validation of autonomous vehicle control systems[J]. IEEE Transactions on Control Systems Technology, 2017, 26(5): 1782-1797.
[149]
OylerD W, YildizY, GirardA R, et al. A game theoretical model of traffic with multiple interacting drivers for use in autonomous vehicle development[C]∥American Control Conference (ACC), Boston, USA, 2016: 1705-1710.
[150]
AlbabaB M, YildizY. Driver modeling through deep reinforcement learning and behavioral game theory[J]. IEEE Transactions on Control Systems Technology, 2021, 30(2): 885-892.
[151]
FengS, FengY, SunH, et al. Testing scenario library generation for connected and automated vehicles, part Ⅱ: case studies[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 22(9):5635-5647.
[152]
KangS, GuoH, SuP, et al. Ecsas: exploring critical scenarios from action sequence in autonomous driving[C]∥2023 IEEE 32nd Asian Test Symposium (ATS), Beijing, China, 2023: 1-6.
[153]
HumeniukD, KhomhF, AntoniolG. Reinforcement learning informed evolutionary search for autonomous systems testing[J]. ACM Transactions on Software Engineering and Methodology, 2024, 33(8): 1-45.
[154]
WangX, ZhangS, PengH. Comprehensive safety evaluation of highly automated vehicles at the roundabout scenario[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(11): 20873-20888.
[155]
LuC. Test scenario generation for autonomous driving systems with reinforcement learning[C]∥IEEE/ACM 45th International Conference on Software Engineering: Companion Proceedings(ICSE-Companion), Melbourne,Australia, 2023: 317-319.
[156]
GaoJ, LiS, TaoC, et al. An approach to GUI test scenario generation using machine learning[C]∥ IEEE International Conference on Artificial Intelligence Testing(AITest), Newark, USA, 2022: 79-86.
[157]
WachiA. Failure-scenario maker for rule-based agent using multi-agent adversarial reinforcement learning and its application to autonomous driving[J/OL].[2023-08-22].
[158]
MaY, SunC, ChenJ, et al. Verification and validation methods for decision-making and planning of automated vehicles: a review[J]. IEEE Transactions on Intelligent Vehicles, 2022, 7(3): 480-498.
[159]
NiuH, RenK, XuY, et al. (Re) 2H2O: Autonomous driving scenario generation via reversely regularized hybrid offline-and-online reinforcement learning[C]∥IEEE Intelligent Vehicles Symposium(IV), Anchorage, Alaska, USA, 2023 :1-8.
[160]
BehzadanV, MunirA. Adversarial reinforcement learning framework for benchmarking collision avoidance mechanisms in autonomous vehicles[J]. IEEE Intelligent Transportation Systems Magazine, 2019,13(2): 236-241.
[161]
ChenB, ChenX, WuQ, et al. Adversarial evaluation of autonomous vehicles in lane-change scenarios[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 23(8): 10333-10342.
[162]
ErdoganA, KaplanE, LeitnerA, et al. Parametrized end-to-end scenario generation architecture for autonomous vehicles[C]∥The 6th International Conference on Control Engineering & Information Technology(CEIT),Hong Kong, China, 2018:1-6.
[163]
PützA, ZlockiA, KüfenJ, et al. Database approach for the sign-off process of highly automated vehicles[C]∥25th International Technical Conference on the Enhanced Safety of Vehicles (ESV) National Highway Traffic Safety Administration, Detroit, Michigan, USA, 2017: 1-8.
[164]
HsuehM C, TsaiT K, IyerR K. Fault injection techniques and tools[J]. Computer, 1997,30(4): 75-82.
[165]
JuezG, AmparanE, LattaruloR, et al. Early safety assessment of automotive systems using sabotage simulation-based fault injection framework[C]∥International Conference on Computer Safety, Reliability, and Security,Vienna, Austria, 2017: 255-269.
[166]
FuY, TerechkoA, BijlsmaT, et al. A Retargetable Fault Injection Framework for Safety Validation of Autonomous Vehicles[M]. Hamburg:ICSA-C,2019:69-76.
[167]
JuezG, AmparanE, LattaruloR, et al. Early safety assessment of automotive systems using sabotage simulation-based fault injection framework[C]∥International Conference on Computer Safety, Reliability, and Security, Vienna, Austria, 2017: 255-269
[168]
KarunakaranD, BerrioJS, WorrallS, et al. Critical concrete scenario generation using scenario-based falsification[C]∥IEEE International Conference on Recent Advances in Systems Science and Engineering (RASSE), Tainan, China, 2022: 1-8.
[169]
LeungK, SchmerlingE, ZhangM, et al. On infusing reachability-based safety assurance within planning frameworks for human-robot vehicle interactions[J]. The International Journal of Robotics Research, 2020, 39(10-11): 1326-1345.
[170]
ChenM, TamQ, LivingstonS C, et al. Signal temporal logic meets reachability: connections and applications[C]∥International Workshop on the Algorithmic Foundations of Robotics, Shenzhen, China, 2018:581-601.