KCL logo

Motion Modelling and Analysis Group

School of Biomedical Engineering and Imaging Sciences,

King's College London

Segmentation

Incorporating Topological Knowledge Into Deep Learning Based Segmentation

(Funded by EPSRC grant EP/P001009/1, NIHR Doctoral Research Fellowship for Nick Byrne)

There are many cases in which the topology of a structure to be segmented is known a priori. For example, when segmenting the left ventricle of the heart from short-axis cine MR images, it is known that the correct segmentation will be a topological closed loop in two dimensions. However, the current state-of-the-art in deep learning based segmentation makes no use of this knowledge. In this work, we explicitly incorporate topological priors into a deep learning segmentation model for the first time, based on the concept of persistent homology. These priors require no additional ground truth segmentations and so the model is suitable for use in a semi-supervised setting. We found that by including the topological priors the number of topological errors was reduced whilst maintaining high overlap with the ground truth. James Clough demonstrated this principle for the first time in a single-class setting, whilst Nick Byrne subsequently extended it to the multi-class case.

Examples of topology in segmentation
Three example MRI images of the short-axis view of the heart. The estimated segmentations produced by a U-net model (a, c, e) show topological errors. The segmentations for the same model trained with our topological prior (b, d, f) have the correct topology.

Publications:

In related work, topological considerations were introduced into deep learning based 3D cardiac segmentation. In this research, it was shown that conventional data augmentation schemes that are widely employed to boost the performance of deep learning segmentation models result in violations of ground truth topology. Furthermore, conventional metrics of segmentation success such as the Dice coefficient are not sensitive to these violations. A straightforward topological correction scheme was proposed, based on the concept of cardiac contiguous topology, to correct the topology of ground truth augmentations. This resulted in fewer topological errors in the segmentations produced by the resulting model.

Examples of topological errors in 3D cardiac segmentation
Examples of topological errors in 3D cardiac segmentation: (a) ground truth cardiac segmentation overlaid onto MR image; (b) predicted segmentation with topological error indicated by arrow; (c) 3D rendering of predicted segmentation including topological errors.

Publications:

Machine Learning for Automated Interpretation of Ultrasound Images

(Funded by King's China Scholarship for Zhen Yuan)

Patients suffering from Sickle Cell Disease can suffer from a range of complications including abnormal enlargement of the spleen. Ultrasound forms part of the clinical pipeline for spleen assessment, but measuring its size in ultrasound images is a subjective process. Furthermore, in many parts of the developing world there is a lack of skills for performing reliable spleen assessments. We investigated the use of deep learning to automate this process, reaching almost human-level performance.

Automated segmentation and measurement of the spleen in ultrasound images
Automated segmentation and measurement of the spleen in ultrasound images.

Publications:

Deep Learning-Based Segmentation of the Vocal Tract From Speech MRI

(Funded by NIHR Career Development Award for Matthieu Ruthven)

MR is increasingly playing a role in speech analyis - dynamic MR images of patients acquired during speech can be used for planning and monitoring of treatment for speech problems such as velopharyngeal insufficiency. In this project our ultimate aim is to construct dynamic 3D speech models of patients for informing treatment decisions. Initial work has investigated the use of deep learning for full segmentation of the articulators in the vocal tract from 2D MR, resulting in the first fully-automatic model demonstrated for this task.

Automated segmentation of the vocal tract articulators from dynamic MR images: left - ground truth, middle - deep learning based segmentation, right - after post-processing.

Publications: