Advancing autonomous driving system testing: Demands, challenges, and future directions

Context: Autonomous driving systems (ADSs) promise enhanced transportation efficiency but face critical challenges in ensuring reliability across complex driving environments. Effective testing is essential to validate ADS performance and mitigate real-world risks. Objective: This study investigates current ADS testing practices for both modular and end-to-end systems, identifies key demands (needs required by practitioners and researchers), and examines gaps between research and real-world demands. We review critical testing techniques and extend to involve Vehicle-to-Everything (V2X) communication and Foundation Models (FMs), including large language models and vision foundation models, in enhancing ADS testing performance. We provide literature reviews and outline future directions for each demand of industry practitioners and academic researchers. Methods: A large-scale survey was conducted with 100 participants, including industry practitioners and academic researchers. We first discuss survey questions with professionals, distribute them to industry practitioners and academic researchers, and conduct follow-ups. Quantitative and qualitative analyses uncover key trends and challenges. Results: Findings reveal that existing ADS testing techniques struggle to evaluate real-world performance comprehensively, including the diversity of corner cases, the gap between simulation and real-world testing, the lack of comprehensive testing criteria, potential attacks, practical deployment for V2X, and the computational costs for FMs. By analyzing participants’ responses and 105 relevant papers, we summarize the future research directions. Conclusion: Our study highlights critical research gaps in ADS testing and underscores the demands of industry practitioners and academic researchers. We provide future directions for ADS: comprehensive testing criteria, cross-model collaboration in V2X, enhancing cross-modality (e.g., text and image) adaptation in FM testing, and scalable ADS validation frameworks. These insights contribute to advancing software engineering practices for ADS development, ensuring safer and more reliable autonomous systems.