Deep Active Learning for Medical Image Segmentation

Medical image analysis can be used to automatically find anatomical structures like organs or brain cells as well as pathologies like tumors in data from imaging modalities such as Magnetic Resonance Imaging, Computed Tomography, or Electron Microscopy. Modern machine learning methods, more specifically deep neural networks deliver excellent performance in finding these structures, in a prediction process called image segmentation. However, these methods require large datasets of labelled (annotated) examples to appropriately instruct the neural networks during their training phase. Since this labelling has to be done by human domain experts and in a pixel-by-pixel manner, this is a costly and time-consuming effort. Active learning is an important area of research in this context, where images from a large pool of unlabelled data are cleverly selected to minimize the number of manual annotations by a domain expert, while still promising competitive segmentation performance. Current methods for active learning based on neural networks for segmentation have their limitations, since they are often very specific to an application area. Furthermore, it is a challenge to constantly update the trained prediction models for segmentation in such a way that models do not forget what they have already learned in previous versions. In this project we will investigate how the current generation of foundation models, which are widely used for example in large language models like ChatGPT, or in image generation tools like Midjourney, can be used to improve active learning in the context of deep neural network-based medical image segmentation. Therefore, we aim to develop: (1) novel methods to most effectively select images from the pool of unlabeled data, (2) improved techniques to re-train neural networks such that already learned knowledge is not forgotten, (3) innovative combinations of general purpose segmentation tools with methods rooted in medical imaging to enable efficient interactive labeling, and (4) a proof of concept active learning software that will be tested on a real-world preclinical electron microscopy dataset of insect brain neurons from our collaboration partner. The project will be made possible by a collaboration hosted at the Medical University of Graz, consisting of the group of Ass. Prof. Martin Urschler (Institute for Medical Informatics, Statistics and Documentation), an expert in medical image analysis and machine learning applied to medical imaging, and Assoc. Prof. Gerd Leitinger (Electron Microscopy Techniques Research Unit at the Gottfried Schatz Research Centers Division of Cell Biology, Histology, and Embryology), an expert on reconstruction of 3D stacks of neuronal cells and sub-cellular structures.