Incorporating Topological Knowledge Into Deep Learning Based Segmentation
(Funded by EPSRC grant EP/P001009/1, NIHR Doctoral Research Fellowship for Nick Byrne)
There are many cases in which the topology of a structure to be segmented is known a priori. For example, when segmenting the left ventricle of the heart from short-axis cine MR images, it is known that the correct segmentation will be a topological closed loop in two dimensions. However, the current state-of-the-art in deep learning based segmentation makes no use of this knowledge. In this work, we explicitly incorporate topological priors into a deep learning segmentation model for the first time, based on the concept of persistent homology. These priors require no additional ground truth segmentations and so the model is suitable for use in a semi-supervised setting. We found that by including the topological priors the number of topological errors was reduced whilst maintaining high overlap with the ground truth. James Clough demonstrated this principle for the first time in a single-class setting, whilst Nick Byrne subsequently extended it to the multi-class case.
- J. R. Clough, I. Oksuz, N. Byrne, J. A. Schnabel, A. P. King, "Explicit Topological Priors for Deep-Learning Based Image Segmentation Using Persistent Homology", Proceedings IPMI, 2019. (Arxiv paper)
- J. Clough, N. Byrne, I. Oksuz, V. A. Zimmer, J. A. Schnabel, A. P. King, "A Topological Loss Function for Deep-Learning based Image Segmentation using Persistent Homology", IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020. (paper)
- N. Byrne, J. R. Clough, G. Montana, A. P. King, "A Persistent Homology-based Topological Loss Function for Multi-class CNN Segmentation of Cardiac MRI", Proceedings MICCAI STACOM, 2020. (Arxiv paper)
In related work, topological considerations were introduced into deep learning based 3D cardiac segmentation. In this research, it was shown that conventional data augmentation schemes that are widely employed to boost the performance of deep learning segmentation models result in violations of ground truth topology. Furthermore, conventional metrics of segmentation success such as the Dice coefficient are not sensitive to these violations. A straightforward topological correction scheme was proposed, based on the concept of cardiac contiguous topology, to correct the topology of ground truth augmentations. This resulted in fewer topological errors in the segmentations produced by the resulting model.
- N. Byrne, J. R. Clough, I. Valverde, G. Montana, A. P. King, "Topology-preserving Augmentation for CNN-based Segmentation of Congenital Heart Defects from 3D Paediatric CMR", Proceedings MICCAI PIPPI, 2019. (Arxiv paper)
Machine Learning for Automated Interpretation of Ultrasound Images
(Funded by King's China Scholarship for Zhen Yuan)
Patients suffering from Sickle Cell Disease can suffer from a range of complications including abnormal enlargement of the spleen. Ultrasound forms part of the clinical pipeline for spleen assessment, but measuring its size in ultrasound images is a subjective process. Furthermore, in many parts of the developing world there is a lack of skills for performing reliable spleen assessments. We investigated the use of deep learning to automate this process, reaching almost human-level performance.
- Z. Yuan, E. Puyol-Antón, H. Jogeesvaran, C. Reid, B. Inusa, A. P. King, "Deep Learning for Automatic Spleen Length Measurement in Sickle Cell Disease Patients", Proceedings MICCAI ASMUS, 2020. (Arxiv paper)
Deep Learning-Based Segmentation of the Vocal Tract From Speech MRI
MR is increasingly playing a role in speech analyis - dynamic MR images of patients acquired during speech can be used for planning and monitoring of treatment for speech problems such as velopharyngeal insufficiency. In this project our ultimate aim is to construct dynamic 3D speech models of patients for informing treatment decisions. Initial work has investigated the use of deep learning for full segmentation of the articulators in the vocal tract from 2D MR, resulting in the first fully-automatic model demonstrated for this task.
- M. Ruthven, M. E. Miquel, A. P. King, "Deep-learning-based segmentation of the vocal tract and articulators in real-time magnetic resonance images of speech", Computer Methods and Programs in Biomedicine, 2020. (paper)