A Reference Model of Explainable AI for the Medical Domain

The progress of statistical machine learning methods has made Artificial Intelligence (AI) increasingly successful. Deep learning exceed human performance even in the medical domain. However, their full potential is limited by the difficulty to generate the underlying explanatory structures. The central problem is that they are regarded as black-boxes and even if we understand the mathematical principles, they lack an explicit declarative knowledge representation. This calls for tools enabling to make decisions transparent, understandable and explainable. A huge motivation for this project are rising legal and privacy issues, which make black-boxes difficult to use. This does not imply a ban on automatic learning approaches or an obligation to explain everything all the time, however, there must be a possibility to make the results re-traceable on demand. The topic of this project is explainability and our goal is to make existing AI/ML algorithms transparent, re- traceable, thus understandable to a medical professional and outline a possible new approach towards explainable AI. We learned of a variety of technical solutions which are currently in development, which could help explain AI/ML systems and their decisions. Transparent algorithms could appropriately enhance trust of medical professionals, thereby raising acceptance of machine learning solutions specifically and of AI generally. We will combine ML with a continuous input stream of sensoric data and (implicit) knowledge capturing thereby making use of human cognitive abilities. All outcomes of this project, algorithms and tools, will be made openly available to the international research community.

Artificial Intelligence (AI) has achieved remarkable progress through advances in statistical machine learning, with deep learning models now exceeding human performance in several areas, including medicine. Yet, these systems often remain opaque, offering little insight into how decisions are made. The project "explainable Artificial Intelligence" (FWF P-32554) addresses this critical challenge by developing transparent and trustworthy AI methods that enable humans to understand and retrace machine decision processes. The project aims to enhance trust and acceptance of AI in sensitive domains such as healthcare by creating a library of explanatory patterns and a formal grammar for combining them into meaningful explanations. It further establishes criteria and benchmarks to answer the question "What is a good explanation?" and introduces principles for measuring the effectiveness of explainability. By mapping human understanding to machine-generated explanations, the project provides an open explanatory framework, complete with benchmarks and datasets, to inspire further international research. All results are openly shared with the scientific community, contributing to global efforts toward transparent, causal, and human-centered AI.