The increasing use of artificial intelligence in safety-critical application areas creates new challenges for IT security. In particular, in medical technology and medical decision-making processes, attacks on AI systems can have significant implications for diagnosis, treatment, and patient safety.

The Medical and AI Security research focus takes a holistic view of the security of AI applications throughout the entire AI lifecycle:  from data collection and modelling to training and deployment in real-world systems. The aim is to systematically analyse threats and attacks on AI-based applications, evaluate the robustness of neural networks, and identify security risks associated with the integration of AI into existing processes at an early stage. This includes attacks that deliberately manipulate classification decisions, for example in tumour detection, as well as attacks targeting data, models, and the underlying infrastructure.

 

The current focus areas of the “Medical and AI Security” research group include:

• Attacks on AI applications throughout the AI lifecycle Manipulation of classification decisions, particularly in medical application scenarios
• Evaluation of the robustness of neural networks against targeted attacks and perturbations
• Analysis of security risks and opportunities arising from the integration of AI into existing medical and technical processes
• Hardware-related attacks as well as data and model exfiltration in AI-integrated medical technology, for example in laboratory environments involving sonography or endoscopy systems
   

Theses (Bachelor’s/Master’s) and student placements are available at any time. Please feel free to get in touch.

Publications: see https://www.thi.de/personen/prof-patrizia-heinl/