Modelling Vasculature and Neurons from 3D scans

High resolution Computed Tomography (CT) and Magnetic Resonance (MR) Angiography scans reveal blood vessels smaller than one millimeter in the diameter, letting doctors inspect veins and arteries in human bodies in unprecedented detail. These imaging capacities hold the promise of studying and diagnosing diseases that provoke vascular degeneration in a new way, by comparing the vascular structure of selected organs in healthy and diseased people. But, to capture structural properties of vasculature, like the number of bifurcations, or the curvature of blood vessels, the shape and topology of the vascular network needs to be extracted from the scans and encoded in the form of a vasculature model. Unfortunately, reconstructing a sufficiently large number of such models to produce statistically valid arguments is currently very costly, because it requires hours of interactive tracing of blood vessels in the scans by means of dedicated software. A similar roadblock hampers the progress towards reconstructing the wiring pattern of the mouse brain. Recent light microscopes produce 3D scans that reveal neuronal arborizations at the scale of the entire brain. But raw scans are not enough to understand the brain, for example, to simulate neuronal activity. This latter goal requires reconstructing a very large number of neuron models from the microscopy scans. Just like reconstructing vasculature models from CT scans, reconstructing neuron models from microscopy scans is currently prohibitively costly due to the required manual labor. Our goal is to help neuroscientists and medical researchers overcome these limitations. To that end, we will propose new algorithms for reconstructing vasculature and neuron models from 3D images with minimum human intervention. These two tasks can be conveniently performed with the same methods, because both vasculature and neuronal arborizations can be represented as networks of thin, curvy tubes. Our approach is based on deep learning, the machine learning technique that revolutionized the field of computer vision, bringing a large boost of accuracy in image classification and image segmentation, which consists in assigning a class to each image pixel. By contrast to these tasks, focused on classifying either whole images or individual pixels, our goal is to produce models of vasculature or neurons that represent their structure, that is, the presence or absence of connections between individual blood vessels and neurites, in addition to their trajectories. There is currently not enough research on using deep learning to reconstruct such models of objects featuring complex and variable structure from images, and the existing methods fail to produce results that could be used by neuroscientists and medical researchers. Our main challenge will therefore be to develop new deep learning algorithms, optimized for structural correctness of the resulting models, as opposed to the usual classification accuracy.

For millennia, human understanding of the human body was limited by our capacity to observe anatomic structures, and its development often resulted from the progress in techniques to surpass the limitations of the naked eye. Today, as microscopes and CT scanners produce ever more detailed 3D images, the frontiers of human knowledge are no longer shaped by our ability to image structures inside the human body, but by our capacity to analyse the overwhelming amounts of data the imaging devices can produce. Moving these frontiers hinges on techniques of automated image analysis. In this project, we harnessed deep learning for two kinds of such analysis. First, we studied constructing models of blood vessels and neurites visible in 3D scans. These models are pivotal for the analysis of neuronal and vascular networks, for example, for simulating blood flow or neuronal signal propagation. But constructing them manually at the scale of entire organs is prohibitively time-consuming. Such large-scale reconstruction is a perfect use-case for deep learning. Unfortunately, current deep learning algorithms are not well suited for this task, because they produce models marred by connectivity errors --- erroneous interruptions or false connections between vessels or neurons. To address this problem, we developed new methods of training deep networks to correctly represent the connectivity of thin structures. These methods not only improve the resulting neurite and vasculature models, but also enable using coarse training annotations, easier and faster to produce than precise ones. The second type of analysis that we studied consists in detecting the Bronchiolitis Obliterans Syndrome (BOS) from CT scans. BOS is a progressive, fibrotic airway disease that affects patients after lung transplantation. Early diagnosis is pivotal for effective management of BOS, but remains challenging. Diagnostic use of CT imaging is not fully established, because specific changes are often seen only when the disease is advanced. Deep networks can capture subtle manifestations of diseases in CT scans, that might be difficult to observe for humans, but training them to distinguish signs of the disease from anatomic characteristics of individual patients requires large volumes of training data, while the cohort of patients in our study was modest. We addressed this problem by simultaneously training the network to detect BOS and to differentiate between late and early stages of the disease in the same patient. This helped the network home in on the symptoms of the disease while ignoring patient-specific anatomic features. Our method attained high performance in detecting BOS from CT scans and showed promise in identifying patients at risk of developing the disease in the future. However, it still needs to be tested in clinical conditions before it may be introduced to routine diagnostic practice.