Cabin Monitoring

CARISSMA's passive safety research group shows expertise in the field of human-centred artificial intelligence by combining machine learning expertise with knowledge of human-machine interaction in road safety. Its advanced know-how in the fields of computer vision systems, application of artificial intelligence methods for image processing, driver behaviour and behavioural changes from the interaction with automated functions, as well as anthropometry and biometrics, has enabled the group to collaborate with the industry and other research institutes in the passt years.

Work conducted sofar focused on the areas of pose detection and passenger state assessment.

Pose detection

In-Vehicle occupant detection includes not only the determination of the number of passengers, but also their exact positioning and the postures made. This information is important for the recognition of Out-of-Position (OoP) situations and the targeted activation of new, adapted restraint systems, which can protect passengers dependent on their position. It is well known that drivers perform additional tasks (so-called secondary tasks or non-driver tasks) when driving manually. Apart from the fact that these tasks may pose a safety risk (the driver has to divide his attention between driving and other activities), to perform them, drivers move their head, arms and upper body, sometimes leaving the ideal upright position required for optimal protection in the event of an airbag deployment.

With increasing automation of vehicles, drivers will no longer need to continuously monitor the environment. They will have more freedom to interact with other passengers and perform activities. As a result, the number of Out-of-Position situations will increase exponentially, posing a challenge for new adaptive restraint systems. Our work on the detection of passengers, objects and animals inside vehicles (passenger cars and public transport) therefore aims at testing and optimising deep learning convolutional neural network architectures for pose recognition and estimation. The focus lies on the creation of a human body segment graph for the exact positioning of passengers. Especially the distance of the different body segments in relation to a certain point or region in the cabin is of great importance. The research group is also working on accomplishing accurate and robust real-time positioning in different scenarios (different number of passengers, clothing characteristics and movement amplitude).

Passenger State Assessment

The work, which focus on passenger state assessment and fitness to drive, is based on the development and improvement of (mainly non-invasive) techniques for measuring physiological and behavioural parameters, that can help to infer conditions such as drowsiness, distraction or stress. These conditions may prevent the driver from performing the driving task manually or, in the case of a higher degree of automation, may impede adequate vehicle monitoring or take-over control actions. Modern methods of artificial intelligence are applied to sensors data, such as like cameras or radar. These results are then compared with classical evaluation methods (e.g. ECG), which are usually considered gold standards.

Recently, the group created a private database (100 driving hours, 130 TB) which contains naturalistic data from daily commuters and long-distance drivers. This database comprises videos of drivers' faces (from RGB and IR cameras) and the road environment ahead (Dashboard Camera), physiological data (Heart Rate and Respiration Frequency from Smartwatch and ECG) and subjective assessments of stress and drowsiness along all trips. Current work on image processing using this database aims to develop algorithms to assess driver's fitness and alertness state. The realistic nature of the data, resulting of the fact that the journeys have taken place under different external environmental conditions (different light, road layout and traffic changes), allows to test the robustness of the algorithms used.

Contact: Prof. Dr. Alessandro Zimmer; Dr. Marta Pereira Cocron

Contact: Prof. Dr. Alessandro Zimmer