Hybrid intelligence for advanced collective perception and decision making in complex urban environments
Client
Partners
The problem
Urban mobility presents a range of challenges for Connected, Cooperative, and Automated Mobility (CCAM) systems. Among the most critical is the timely and reliable detection of occluded objects, especially vulnerable road users (VRUs).
The solution
To address this, the HIDDEN project is pioneering advanced collective awareness (CA) and decision-making algorithms, designed to function both with and without support from road infrastructure. The CA system is specifically focused on detecting occluded objects, including VRUs, and predicting their short-term trajectories using sophisticated behavioural models.
At Mosaic Factor, we will lead the development of groundbreaking tools that enable autonomous systems to see beyond obstacles, detecting hidden road users and making more ethical, human-centered decisions.
Our Human-Feedback Reinforcement Learning toolkit will promote transparency, nurture trust, and enhance safety across cities throughout Europe.
In parallel, HIDDEN is developing an innovative driver gaze tracking and status monitoring system. The data generated from both the CA module and the gaze tracking system feed into real-time decision-making algorithms that are built to be explainable, ethically sound, and closely aligned with human driving behaviours.
HIDDEN employs Hybrid Intelligence across the entire perception-to-decision chain, leveraging the combined strengths of human and machine intelligence. Ethical and legal considerations related to AI are carefully integrated into the process through the creation of a dedicated framework.
The project’s developments will undergo both real-world and simulated evaluations. Field tests will be performed using the consortium’s eight autonomous vehicles, deployed across various testing facilities and public roads. For virtual validation, novel co-simulation environments will be introduced.
Four main use cases have been pre-selected to guide the technical direction of the project. Through these efforts, HIDDEN aims to create CCAM systems that are not only technologically cutting-edge but also deeply rooted in ethical principles, human-centric design, and regulatory compliance, setting a new standard for autonomous vehicle technology.
Data
HIDDEN develops a hybrid intelligence framework that fuses data from Vehicle‑to‑Everything (V2X) communications, infrastructure sensors, other vehicles and driver gaze monitoring to build advanced collective awareness. The system detects occluded objects and VRUs, predicts their short‑term trajectories using behavioural models and feeds this information to real‑time decision‑making algorithms that are explainable and aligned with human driving styles and ethical principles.
- Sensor and perception data from connected and automated vehicles (including cameras, lidar, radar and V2X communications).
- Roadside infrastructure and vulnerable road user data shared via V2X.
- Driver gaze tracking and status monitoring data used to model human behaviour.
- Behavioural models and simulation data from co‑simulation environments for trajectory prediction and decision‑making.
Results
HIDDEN started on 1 July 2025 and will run for 36 months. Results will be reported during the project.
To amplify impact, the consortium is actively engaging with CCAM stakeholders across Europe and beyond, maintaining continuous dialogue with EU type-approval authorities and UNECE working groups, and promoting mature solutions for standardisation.
Expected outcomes include:
- improved ability to detect occluded VRUs and objects,
- ethically aligned decision‑making that enhances safety and trust,
- and proof‑of‑concept demonstrations in four challenging urban scenarios.