SYNERGIES

Forecasting model and maintenance

Client

Partners

The problem

Safety of CCAM (Connected, Cooperative and Automated Mobility) systems needs to be ensured.

The main challenge for safety validation is that depending on the ODD (Operational Design Domain) many different driving situations and complex scenarios need to be tested and validated. Additionally, the European automotive sector is guided by strict testing and validation rules, which mandatorily impose a thorough evaluation of the possible situations a CCAM system will face (including multiactor complex scenarios, hazards, unusual situations, and challenging conditions).

However, the complexity and uncertainty of the driving environment, and the complexity of the driving task itself, imply that the number of possible scenarios that an Automated Driving System (ADS) may encounter is virtually infinite.

Other studies suggest that to test a CCAM system to assess its safety and prove that it is 20% better than human driven vehicle, it needs to be driven for over 11 billion miles. Additionally, Hazard Based Testing (HBT) and sociotechnical systems advocate that the number of miles driven alone is not sufficient to judge confidence in CCAM systems.

Instead, the crucial aspect is the range and variety of scenarios encountered during testing. The must be on understanding and identifying ‘how a system can fail or misbehave’ and subsequently ensuring it does so in a safe and trustworthy manner. The nature of scenarios is fundamental to an assessment of safety.
Key challenges we will tackle from scenario identification are:

1. 1. Identifying the “right” and “representative” scenarios for the relevant ODD.
   2. Understanding if “enough” scenarios have been identified to ensure safety of the system.
   3. Considering the probability of occurrence of the scenarios.
   4. Defining the safety risk associated with the scenario and the vehicle response to the scenario. Key aspect of this is the definition of “good behaviour” or pass/fail criteria for the system for a given scenario.

The solution

By using big data, together with more than 30 partners and leveraging insights from previous Risk Analysis and Field Operational Tests (FOTs), we will enable swift identification and implementation in the market to enhance CCAM systems’ safety.

Moreover, our use of artificial intelligence (AI) based tools will help expand the system requirements space, making the process more effective and efficient. This will enable safe CCAM systems from testing to deployment.

Our SYNERGIES Platform provides the European stakeholders the needed scenarios and tools for accelerated and accepted development, training, virtual testing, and validation of CCAM systems, reducing development time and costs, increasing safety and reliability, and supporting wider adoption.

Our contribution focuses on the data trustworthiness requirements and the definition of the trust metrics of scenarios data. We also research to detect and forecast the new technologies, new scenarios, new actors, market trends, etc.

We will define high level road maps per cases and analyse how the requirements evolve over time per road map (including impact and probability analysis to prioritise requirement changes). We will then assess the applicability of the future requirements considering new technologies, automotive innovations, data & hardware capabilities.

In data quality assessment, we will build a methodology to analyse both available data and synthetically generated data in terms of quality and relevance to its potential to generate scenarios, we will define the proper metrics for each quality descriptor along with acceptable thresholds, which will be defined via statistical research and domain experts, and we will examinate the gaps and limitations of the datasets and provide feedback to the data owner/provider in order to increase the utility/accuracy/correctness/completeness of the data. We will develop an AI-driven data inspector for data quality to evaluate the correctness, completeness, accuracy and consistency of the datasets and timeliness, validate the analysis and evaluate the feature importance. MOSAIC will also create of a methodology and/or tool that assesses the representativeness of a scenario applied in other areas or countries.

Regarding the platform, we will contribute in the definition of the strategy to integrate the necessary components within the platform, will participate in the testing activities of the individual components of the platform and will contribute to the implementation of improvements after the testing phase.

Data

Results

The project will result in a European platform aimed at improving the development, training, virtual testing, and validation of CCAM systems.

The SYNERGIES Platform consists of a
(i) Scenario Dataspace, aligned with the European approach on data sharing and competitiveness, and (ii) Marketplace, to ensure continuous updates and scalability of the Dataspace.