Image based Data Evaluation Analyses and Medical Application

Automated assessment of processed data is indispensable for testing, improving and monitoring preceding methods and to ensure a certain quality for subsequent use. The choice of the quality measure directly affects the outcome of the evaluation or even guides the development of a novel algorithm. In general, suitable measures for data assessment are strongly needed to ensure meaningful conclusions and to avoid data-based problems in subsequent tasks. More specifically, measuring digital image quality is of importance in numerous applied research fields, such as image acquisition, processing or reconstruction. Particularly medical tasks in clinical settings require image quality assessment (IQA) to ensure real-time treatment. Subjective evaluation, i.e. assessment by human beings, is often too time-consuming and expensive. Moreover, manual annotations in complicated medical problems often suffer from inconsistencies. Therefore, automated IQA that yields fast and consistent evaluation schemes is needed. Objective IQA, i.e. assessment without human evaluation, is very important in many applications and is based on available prior data, knowledge or assumptions. It is a difficult task, where successful measures are often tailored for very specific tasks and properties. Unfortunately, more broad methods for natural images are not very well suited for medical image assessment, e.g. patient data after acquisition or reconstruction, because these image modalities rely on important features that are not prevalent in natural images. Some of the most common standard IQA measures that are used to assess novel machine learning methods, are known for their drawbacks when used for medical image data, but are broadly used nonetheless. An universally suitable IQA, tailored to capture diverse medical imaging modalities, has not been introduced yet. Therefore, we aim to design a meaningful, adaptable image quality measure for diverse medical imaging modalities by merging new results on orthogonal projections for data representations, as well as analyses of medical image quality measures. The core of the project is the interdisciplinarity, merging research from collaborations with experts on medical imaging, IQA as well as new mathematical data representations.

Medical imaging plays a crucial role in modern healthcare, supporting diagnoses, treatment planning, and clinical decision-making. As imaging technologies and image-processing algorithms continue to advance and produce huge amounts of data, there is a growing need to reliably assess the quality of medical images in an automated way in the development stage. The project addressed a fundamental challenge in this area: how to measure image quality in the algorithm's development stage beyond general image properties in a way that also reflects what medical experts need. We demonstrated the urgent need for medically informed image quality measures and provided new methods and open data that together pave the way toward more reliable, transparent, and clinically meaningful evaluation of medical imaging algorithms. As an initial step, we published a comprehensive review highlighting limitations and pitfalls of widely used image quality measures when applied to medical imaging data. This work emerged from an international collaboration involving experts across diverse medical imaging fields. We found that many established quality measures, originally developed for natural images such as photographs, frequently fail to accurately assess image quality in medical contexts. The scarcity of publicly available medical image datasets with expert annotations remains a major obstacle to the systematic evaluation and development of image quality measures tailored to medical applications. Building on these insights, we conducted a series of experimental studies to investigate how well different common image quality measures align with expert opinion across various medical imaging modalities. Using clinical chest X-ray images rated by multiple radiologists, as well as photoacoustic image data assessed by domain experts, we evaluated how closely different measures reflected human judgment. The results clearly demonstrated that many state-of-the-art quality measures perform poorly for medical images. In particular, measures optimized for natural images fail to capture clinically relevant information and may favor visual properties that are irrelevant or even misleading in a medical context. This has important implications for the development and evaluation of image-processing algorithms, including machine-learning-based reconstruction methods used in modalities such as MRI. If inappropriate quality measures are used, algorithms may be optimized to produce visually appealing images that overlook critical diagnostic content. Next, to improve applicability to medical imaging data, we refined an existing image quality measure that had shown the most robust performance across the tested image modalities. Our findings indicate that the adapted measure exhibits a substantially stronger correlation with medical expert assessments than commonly used standard measures. To support further research in this area, we publicly released the photoacoustic image dataset via the open repository Zenodo as well as accompanying code on GitHub. This allows other researchers to benchmark their methods and further explore image quality assessment techniques tailored to medical imaging data.