Deep learning-based unlearning of dataset bias for MRI harmonisation and confound removal

Increasingly large MRI neuroimaging datasets are becoming available, including many highly multi-site multi-scanner datasets. Combining the data from the different scanners is vital for increased statistical power; however, this leads to an increase in variance due to nonbiological factors such as the differences in acquisition protocols and hardware, which can mask signals of interest. We propose a deep learning based training scheme, inspired by domain adaptation techniques, which uses an iterative update approach to aim to create scanner-invariant features while simultaneously maintaining performance on the main task of interest, thus reducing the influence of scanner on network predictions. We demonstrate the framework for regression, classification and segmentation tasks with two different network architectures. We show that not only can the framework harmonise many-site datasets but it can also adapt to many data scenarios, including biased datasets and limited training labels. Finally, we show that the framework can be extended for the removal of other known confounds in addition to scanner. The overall framework is therefore flexible and should be applicable to a wide range of neuroimaging studies.

Learning-based prediction of gestational age from ultrasound images of the fetal brain

We propose an automated framework for predicting gestational age (GA) and neurodevelopmental maturation of a fetus based on 3D ultrasound (US) brain image appearance. Our method capitalizes on age-related sonographic image patterns in conjunction with clinical measurements to develop, for the first time, a predictive age model which improves on the GA-prediction potential of US images. The framework benefits from a manifold surface representation of the fetal head which delineates the inner skull boundary and serves as a common coordinate system based on cranial position. This allows for fast and efficient sampling of anatomically-corresponding brain regions to achieve like-for-like structural comparison of different developmental stages. We develop bespoke features which capture neurosonographic patterns in 3D images, and using a regression forest classifier, we characterize structural brain development both spatially and temporally to capture the natural variation existing in a healthy population (N=447) over an age range of active brain maturation (18-34weeks). On a routine clinical dataset (N=187) our age prediction results strongly correlate with true GA (r=0.98,accurate within±6.10days), confirming the link between maturational progression and neurosonographic activity observable across gestation. Our model also outperforms current clinical methods by ±4.57 days in the third trimester-a period complicated by biological variations in the fetal population. Through feature selection, the model successfully identified the most age-discriminating anatomies over this age range as being the Sylvian fissure, cingulate, and callosal sulci.

Learning patterns of the ageing brain in MRI using deep convolutional networks

Both normal ageing and neurodegenerative diseases cause morphological changes to the brain. Age-related brain changes are subtle, nonlinear, and spatially and temporally heterogenous, both within a subject and across a population. Machine learning models are particularly suited to capture these patterns and can produce a model that is sensitive to changes of interest, despite the large variety in healthy brain appearance. In this paper, the power of convolutional neural networks (CNNs) and the rich UK Biobank dataset, the largest database currently available, are harnessed to address the problem of predicting brain age. We developed a 3D CNN architecture to predict chronological age, using a training dataset of 12,802 T1-weighted MRI images and a further 6,885 images for testing. The proposed method shows competitive performance on age prediction, but, most importantly, the CNN prediction errors Δ_BrainAge=Age_Predicted-Age_True correlated significantly with many clinical measurements from the UK Biobank in the female and male groups. In addition, having used images from only one imaging modality in this experiment, we examined the relationship between Δ_BrainAge and the image-derived phenotypes (IDPs) from all other imaging modalities in the UK Biobank, showing correlations consistent with known patterns of ageing. Furthermore, we show that the use of nonlinearly registered images to train CNNs can lead to the network being driven by artefacts of the registration process and missing subtle indicators of ageing, limiting the clinical relevance. Due to the longitudinal aspect of the UK Biobank study, in the future it will be possible to explore whether the Δ_BrainAge from models such as this network were predictive of any health outcomes.

Computational methods for quantifying in vivo muscle fascicle curvature from ultrasound images

Muscle fascicles curve during contraction, and this has been seen using B-mode ultrasound. Curvature can vary along a fascicle, and amongst the fascicles within a muscle. The purpose of this study was to develop an automated method for quantifying curvature across the entirety of an imaged muscle, to test the accuracy of the method against synthetic images of known curvature and noise, and to test the sensitivity of the method to ultrasound probe placement. Both synthetic and ultrasound images were processed using multiscale vessel enhancement filtering to accentuate the muscle fascicles, wavelet-based methods were used to quantify fascicle orientations and curvature distribution grids were produced by quantifying local curvatures for each point within the image. Ultrasound images of ramped isometric contractions of the human medial gastrocnemius were acquired in a test-retest study. The methods enabled distinct curvatures to be determined in different regions of the muscle. The methods were sensitive to kernel sizes during image processing, noise within the image and the variability of probe placements during retesting. Across the physiological range of curvatures and noise, curvatures calculated from validation grids were quantified with a typical standard error of less than 0.026 m⁻¹, and this is about 1% of the maximum curvatures observed in fascicles of contracting muscle.

Learning to map 2D ultrasound images into 3D space with minimal human annotation

In fetal neurosonography, aligning two-dimensional (2D) ultrasound scans to their corresponding plane in the three-dimensional (3D) space remains a challenging task. In this paper, we propose a convolutional neural network that predicts the position of 2D ultrasound fetal brain scans in 3D atlas space. Instead of purely supervised learning that requires heavy annotations for each 2D scan, we train the model by sampling 2D slices from 3D fetal brain volumes, and target the model to predict the inverse of the sampling process, resembling the idea of self-supervised learning. We propose a model that takes a set of images as input, and learns to compare them in pairs. The pairwise comparison is weighted by the attention module based on its contribution to the prediction, which is learnt implicitly during training. The feature representation for each image is thus computed by incorporating the relative position information to all the other images in the set, and is later used for the final prediction. We benchmark our model on 2D slices sampled from 3D fetal brain volumes at 18-22 weeks' gestational age. Using three evaluation metrics, namely, Euclidean distance, plane angles and normalized cross correlation, which account for both the geometric and appearance discrepancy between the ground-truth and prediction, in all these metrics, our model outperforms a baseline model by as much as 23%, when the number of input images increases. We further demonstrate that our model generalizes to (i) real 2D standard transthalamic plane images, achieving comparable performance as human annotations, as well as (ii) video sequences of 2D freehand fetal brain scans.

Subcortical Segmentation of the Fetal Brain in 3D Ultrasound using Deep Learning

The quantification of subcortical volume development from 3D fetal ultrasound can provide important diagnostic information during pregnancy monitoring. However, manual segmentation of subcortical structures in ultrasound volumes is time-consuming and challenging due to low soft tissue contrast, speckle and shadowing artifacts. For this reason, we developed a convolutional neural network (CNN) for the automated segmentation of the choroid plexus (CP), lateral posterior ventricle horns (LPVH), cavum septum pellucidum et vergae (CSPV), and cerebellum (CB) from 3D ultrasound. As ground-truth labels are scarce and expensive to obtain, we applied few-shot learning, in which only a small number of manual annotations (n = 9) are used to train a CNN. We compared training a CNN with only a few individually annotated volumes versus many weakly labelled volumes obtained from atlas-based segmentations. This showed that segmentation performance close to intra-observer variability can be obtained with only a handful of manual annotations. Finally, the trained models were applied to a large number (n = 278) of ultrasound image volumes of a diverse, healthy population, obtaining novel US-specific growth curves of the respective structures during the second trimester of gestation.

Challenges for machine learning in clinical translation of big data imaging studies

Combining deep learning image analysis methods and large-scale imaging datasets offers many opportunities to neuroscience imaging and epidemiology. However, despite these opportunities and the success of deep learning when applied to a range of neuroimaging tasks and domains, significant barriers continue to limit the impact of large-scale datasets and analysis tools. Here, we examine the main challenges and the approaches that have been explored to overcome them. We focus on issues relating to data availability, interpretability, evaluation, and logistical challenges and discuss the problems that still need to be tackled to enable the success of "big data" deep learning approaches beyond research.

Low-Memory CNNs Enabling Real-Time Ultrasound Segmentation Towards Mobile Deployment

Convolutional Neural Networks (CNNs), which are currently state-of-the-art for most image analysis tasks, are ill suited to leveraging the key benefits of ultrasound imaging - specifically, ultrasound's portability and real-time capabilities. CNNs have large memory footprints, which obstructs their implementation on mobile devices, and require numerous floating point operations, which results in slow CPU inference times. In this paper, we propose three approaches to training efficient CNNs that can operate in real-time on a CPU (catering to the clinical setting), with a low memory footprint, for minimal compromise in accuracy. We first demonstrate the power of 'thin' CNNs, with very few feature channels, for fast medical image segmentation. We then leverage separable convolutions to further speed up inference, reduce parameter count and facilitate mobile deployment. Lastly, we propose a novel knowledge distillation technique to boost the accuracy of light-weight models, while maintaining inference speed-up. For a negligible sacrifice in test set Dice performance on the challenging ultrasound analysis task of nerve segmentation, our final proposed model processes images at 30fps on a CPU, which is 9× faster than the standard U-Net, while requiring 420× less space in memory.

Self-Supervised Ultrasound to MRI Fetal Brain Image Synthesis

Fetal brain magnetic resonance imaging (MRI) offers exquisite images of the developing brain but is not suitable for second-trimester anomaly screening, for which ultrasound (US) is employed. Although expert sonographers are adept at reading US images, MR images which closely resemble anatomical images are much easier for non-experts to interpret. Thus in this article we propose to generate MR-like images directly from clinical US images. In medical image analysis such a capability is potentially useful as well, for instance for automatic US-MRI registration and fusion. The proposed model is end-to-end trainable and self-supervised without any external annotations. Specifically, based on an assumption that the US and MRI data share a similar anatomical latent space, we first utilise a network to extract the shared latent features, which are then used for MRI synthesis. Since paired data is unavailable for our study (and rare in practice), pixel-level constraints are infeasible to apply. We instead propose to enforce the distributions to be statistically indistinguishable, by adversarial learning in both the image domain and feature space. To regularise the anatomical structures between US and MRI during synthesis, we further propose an adversarial structural constraint. A new cross-modal attention technique is proposed to utilise non-local spatial information, by encouraging multi-modal knowledge fusion and propagation. We extend the approach to consider the case where 3D auxiliary information (e.g., 3D neighbours and a 3D location index) from volumetric data is also available, and show that this improves image synthesis. The proposed approach is evaluated quantitatively and qualitatively with comparison to real fetal MR images and other approaches to synthesis, demonstrating its feasibility of synthesising realistic MR images.

BEAN: Brain Extraction and Alignment Network for 3D Fetal Neurosonography

Brain extraction (masking of extra-cerebral tissues) and alignment are fundamental first steps of most neuroimage analysis pipelines. The lack of automated solutions for 3D ultrasound (US) has therefore limited its potential as a neuroimaging modality for studying fetal brain development using routinely acquired scans. In this work, we propose a convolutional neural network (CNN) that accurately and consistently aligns and extracts the fetal brain from minimally pre-processed 3D US scans. Our multi-task CNN, Brain Extraction and Alignment Network (BEAN), consists of two independent branches: 1) a fully-convolutional encoder-decoder branch for brain extraction of unaligned scans, and 2) a two-step regression-based branch for similarity alignment of the brain to a common coordinate space. BEAN was tested on 356 fetal head 3D scans spanning the gestational range of 14 to 30 weeks, significantly outperforming all current alternatives for fetal brain extraction and alignment. BEAN achieved state-of-the-art performance for both tasks, with a mean Dice Similarity Coefficient (DSC) of 0.94 for the brain extraction masks, and a mean DSC of 0.93 for the alignment of the target brain masks. The presented experimental results show that brain structures such as the thalamus, choroid plexus, cavum septum pellucidum, and Sylvian fissure, are consistently aligned throughout the dataset and remain clearly visible when the scans are averaged together. The BEAN implementation and related code can be found under www.github.com/felipemoser/kelluwen.

The association between flow and oxygenation and cortical development in fetuses with congenital heart defects using a brain-age prediction algorithm

Objectives

Presumably, changes in fetal circulation contribute to the delay in maturation of the cortex in fetuses with congenital heart defect (CHD). The aim of the current study is to analyze fetal brain development based on hemodynamic differences, using novel brain‐age prediction software.

Methods

We have performed detailed neurosonography, including acquiring 3D volumes, prospectively in cases with isolated CHD from 20 weeks onwards. An algorithm that assesses the degree of fetal brain‐age automatically was used to compare CHD cases to controls. We stratified CHD cases according to flow and oxygenation profiles by lesion physiology and performed subgroup analyses.

Results

A total of 616 ultrasound volumes of 162 CHD cases and 75 controls were analyzed. Significant differences in maturation of the cortex were observed in cases with normal blood flow toward the brain (−3.8 days, 95%CI [−5.5; −2.0], P = <.001) and low (−4.0 days, 95% CI [−6.7; −1.2] P = <.05; hypoplastic left heart syndrome[HLHS]) and mixed (−4.4 days, 95%CI [−6.4; −2.5] p = <.001) oxygen saturation in the ascending aorta (TGA) and in cardiac mixing (eg, Fallot) cases.

Conclusion

The current study shows significant delay in brain‐age in TGA and Fallot cases as compared to control cases. However, the small differences found in this study questions the clinical relevance.

Learning to segment key clinical anatomical structures in fetal neurosonography informed by a region-based descriptor

We present a general framework for automatic segmentation of fetal brain structures in ultrasound images inspired by recent advances in machine learning. The approach is based on a region descriptor that characterizes the shape and local intensity context of different neurological structures without explicit models. To validate our framework, we present experiments to segment two fetal brain structures of clinical importance that have quite different ultrasonic appearances-the corpus callosum (CC) and the choroid plexus (CP). Results demonstrate that our approach achieves high region segmentation accuracy (dice coefficient: [Formula: see text] CC, [Formula: see text] CP) relative to human delineation, whereas the derived automated biometry measurement deviations are within human intra/interobserver variations. The use of our proposed method may help to standardize intracranial anatomy measurements for both the routine examination and the detection of congenital conditions in the future.

Regional variations in fascicle curvatures within a muscle belly change during contraction

During muscle contraction, the fascicles curve in response to changes in internal pressures within the muscle. Muscle modelling studies have predicted that fascicles curve to different extents in different regions of the muscle and, as such, curvature is expected to vary along and across the muscle belly. In the present study, the local variations in fascicle curvature within the muscle belly were investigated for a range of contractile conditions. B-mode ultrasound scans of the medial and lateral gastrocnemii muscles were collected at five ankle positions-ranging from dorsiflexion to plantarflexion. An automated algorithm was applied to the images in order to extract the local curvatures from the muscle belly regions. Significant variations in fascicle curvature were seen in the superficial-to-deep direction. Curvatures were positive in the superficial layer, negative in the deep layer, and had intermediate values close to zero in the central muscle region. This is indicative of the fascicles following an S-shaped trajectory across the muscle image. The relation between external pressure and curvature regionalization was also investigated by applying elastic compression bandages on the calf. The application of pressure was associated with greater negative curvatures in the distal and central regions of the middle layer, but appeared to have little effect on the superficial and deep layers. The results from this study showed that (1) fascicle curvature increases with contraction level, (2) there is curvature regionalization within the muscle belly, (3) curvature increases with pressure, and (4) fascicles follow an S-shaped trajectory across the muscle images.

The impact of transfer learning on 3D deep learning convolutional neural network segmentation of the hippocampus in mild cognitive impairment and Alzheimer disease subjects

Research on segmentation of the hippocampus in magnetic resonance images through deep learning convolutional neural networks (CNNs) shows promising results, suggesting that these methods can identify small structural abnormalities of the hippocampus, which are among the earliest and most frequent brain changes associated with Alzheimer disease (AD). However, CNNs typically achieve the highest accuracy on datasets acquired from the same domain as the training dataset. Transfer learning allows domain adaptation through further training on a limited dataset. In this study, we applied transfer learning on a network called spatial warping network segmentation (SWANS), developed and trained in a previous study. We used MR images of patients with clinical diagnoses of mild cognitive impairment (MCI) and AD, segmented by two different raters. By using transfer learning techniques, we developed four new models, using different training methods. Testing was performed using 26% of the original dataset, which was excluded from training as a hold‐out test set. In addition, 10% of the overall training dataset was used as a hold‐out validation set. Results showed that all the new models achieved better hippocampal segmentation quality than the baseline SWANS model (p_s < .001), with high similarity to the manual segmentations (mean dice [best model] = 0.878 ± 0.003). The best model was chosen based on visual assessment and volume percentage error (VPE). The increased precision in estimating hippocampal volumes allows the detection of small hippocampal abnormalities already present in the MCI phase (SD = [3.9 ± 0.6]%), which may be crucial for early diagnosis.

Cortical development in fetuses with congenital heart defects using an automated brain-age prediction algorithm

INTRODUCTION

Congenital heart defects are associated with neurodevelopmental delay. It is hypothesized that fetuses affected by congenital heart defect have altered cerebral oxygen perfusion and are therefore prone to delay in cortical maturation. The aim of this study was to determine the difference in fetal brain age between consecutive congenital heart defect cases and controls in the second and third trimester using ultrasound.

MATERIAL AND METHODS

Since 2014, we have included 90 isolated severe congenital heart defect cases in the Heart And Neurodevelopment (HAND)-study. Every 4 weeks, detailed neurosonography was performed in these fetuses, including the recording of a 3D volume of the fetal brain, from 20 weeks onwards. In all, 75 healthy fetuses underwent the same protocol to serve as a control group. The volumes were analyzed by automated age prediction software which determines gestational age by the assessment of cortical maturation.

RESULTS

In total, 477 volumes were analyzed using the age prediction software (199 volumes of 90 congenital heart defect cases; 278 volumes of 75 controls). Of these, 16 (3.2%) volume recordings were excluded because of imaging quality. The age distribution was 19-33 weeks. Mixed model analysis showed that the age predicted by brain maturation was 3 days delayed compared with the control group (P = .002).

CONCLUSIONS

This study shows that fetuses with isolated cases of congenital heart defects show some delay in cortical maturation as compared with healthy control cases. The clinical relevance of this small difference is debatable. This finding was consistent throughout pregnancy and did not progress during the third trimester.

Multimodal Deep Learning for Predicting Adverse Birth Outcomes Based on Early Labour Data

Cardiotocography (CTG) is a widely used technique to monitor fetal heart rate (FHR) during labour and assess the health of the baby. However, visual interpretation of CTG signals is subjective and prone to error. Automated methods that mimic clinical guidelines have been developed, but they failed to improve detection of abnormal traces. This study aims to classify CTGs with and without severe compromise at birth using routinely collected CTGs from 51,449 births at term from the first 20 min of FHR recordings. Three 1D-CNN and LSTM based architectures are compared. We also transform the FHR signal into 2D images using time-frequency representation with a spectrogram and scalogram analysis, and subsequently, the 2D images are analysed using a 2D-CNNs. In the proposed multi-modal architecture, the 2D-CNN and the 1D-CNN-LSTM are connected in parallel. The models are evaluated in terms of partial area under the curve (PAUC) between 0-10% false-positive rate; and sensitivity at 95% specificity. The 1D-CNN-LSTM parallel architecture outperformed the other models, achieving a PAUC of 0.20 and sensitivity of 20% at 95% specificity. Our future work will focus on improving the classification performance by employing a larger dataset, analysing longer FHR traces, and incorporating clinical risk factors.

Predicting fetal neurodevelopmental age from ultrasound images

We propose an automated framework for predicting age and neurodevelopmental maturation of a fetus based on 3D ultrasound (US) brain image appearance. A topology-preserving manifold representation of the fetal skull enabled design of bespoke scale-invariant image features. Our regression forest model used these features to learn a mapping from age-related sonographic image patterns to fetal age and development. The Sylvian Fissure was identified as a critical region for accurate age estimation, and restricting the search space to this anatomy improved prediction accuracy on a set of 130 healthy fetuses (error ± 3.8 days; r = 0.98 performing the best current clinical method. Our framework remained robust when applied to a routine clinical population.

STAMP: Simultaneous Training and Model Pruning for low data regimes in medical image segmentation

Acquisition of high quality manual annotations is vital for the development of segmentation algorithms. However, to create them we require a substantial amount of expert time and knowledge. Large numbers of labels are required to train convolutional neural networks due to the vast number of parameters that must be learned in the optimisation process. Here, we develop the STAMP algorithm to allow the simultaneous training and pruning of a UNet architecture for medical image segmentation with targeted channelwise dropout to make the network robust to the pruning. We demonstrate the technique across segmentation tasks and imaging modalities. It is then shown that, through online pruning, we are able to train networks to have much higher performance than the equivalent standard UNet models while reducing their size by more than 85% in terms of parameters. This has the potential to allow networks to be directly trained on datasets where very low numbers of labels are available.

Normative spatiotemporal fetal brain maturation with satisfactory development at 2 years

Maturation of the human fetal brain should follow precisely scheduled structural growth and folding of the cerebral cortex for optimal postnatal function1. We present a normative digital atlas of fetal brain maturation based on a prospective international cohort of healthy pregnant women2, selected using World Health Organization recommendations for growth standards3. Their fetuses were accurately dated in the first trimester, with satisfactory growth and neurodevelopment from early pregnancy to 2 years of age4,5. The atlas was produced using 1,059 optimal quality, three-dimensional ultrasound brain volumes from 899 of the fetuses and an automated analysis pipeline6-8. The atlas corresponds structurally to published magnetic resonance images9, but with finer anatomical details in deep grey matter. The between-study site variability represented less than 8.0% of the total variance of all brain measures, supporting pooling data from the eight study sites to produce patterns of normative maturation. We have thereby generated an average representation of each cerebral hemisphere between 14 and 31 weeks' gestation with quantification of intracranial volume variability and growth patterns. Emergent asymmetries were detectable from as early as 14 weeks, with peak asymmetries in regions associated with language development and functional lateralization between 20 and 26 weeks' gestation. These patterns were validated in 1,487 three-dimensional brain volumes from 1,295 different fetuses in the same cohort. We provide a unique spatiotemporal benchmark of fetal brain maturation from a large cohort with normative postnatal growth and neurodevelopment.

An automated method for tendon image segmentation on ultrasound using grey-level co-occurrence matrix features and hidden Gaussian Markov random fields

BACKGROUND

Despite knowledge of qualitative changes that occur on ultrasound in tendinopathy, there is currently no objective and reliable means to quantify the severity or prognosis of tendinopathy on ultrasound.

OBJECTIVE

The primary objective of this study is to produce a quantitative and automated means of inferring potential structural changes in tendinopathy by developing and implementing an algorithm which performs a texture based segmentation of tendon ultrasound (US) images.

METHOD

A model-based segmentation approach is used which combines Gaussian mixture models, Markov random field theory and grey-level co-occurrence (GLCM) features. The algorithm is trained and tested on 49 longitudinal B-mode ultrasound images of the Achilles tendons which are labelled as tendinopathic (24) or healthy (25). Hyperparameters are tuned, using a training set of 25 images, to optimise a decision tree based classification of the images from texture class proportions. We segment and classify the remaining test images using the decision tree.

RESULTS

Our approach successfully detects a difference in the texture profiles of tendinopathic and healthy tendons, with 22/24 of the test images accurately classified based on a simple texture proportion cut-off threshold. Results for the tendinopathic images are also collated to gain insight into the topology of structural changes that occur with tendinopathy. It is evident that distinct textures, which are predominantly present in tendinopathic tendons, appear most commonly near the transverse boundary of the tendon, though there was a large variability among diseased tendons.

CONCLUSION

The GLCM based segmentation of tendons under ultrasound resulted in distinct segmentations between healthy and tendinopathic tendons and provides a potential tool to objectively quantify damage in tendinopathy.

Anatomically plausible segmentations: Explicitly preserving topology through prior deformations

Since the rise of deep learning, new medical segmentation methods have rapidly been proposed with extremely promising results, often reporting marginal improvements on the previous state-of-the-art (SOTA) method. However, on visual inspection errors are often revealed, such as topological mistakes (e.g. holes or folds), that are not detected using traditional evaluation metrics. Incorrect topology can often lead to errors in clinically required downstream image processing tasks. Therefore, there is a need for new methods to focus on ensuring segmentations are topologically correct. In this work, we present TEDS-Net: a segmentation network that preserves anatomical topology whilst maintaining segmentation performance that is competitive with SOTA baselines. Further, we show how current SOTA segmentation methods can introduce problematic topological errors. TEDS-Net achieves anatomically plausible segmentation by using learnt topology-preserving fields to deform a prior. Traditionally, topology-preserving fields are described in the continuous domain and begin to break down when working in the discrete domain. Here, we introduce additional modifications that more strictly enforce topology preservation. We illustrate our method on an open-source medical heart dataset, performing both single and multi-structure segmentation, and show that the generated fields contain no folding voxels, which corresponds to full topology preservation on individual structures whilst vastly outperforming the other baselines on overall scene topology. The code is available at: https://github.com/mwyburd/TEDS-Net.

Sensorless volumetric reconstruction of fetal brain freehand ultrasound scans with deep implicit representation

Three-dimensional (3D) ultrasound imaging has contributed to our understanding of fetal developmental processes by providing rich contextual information of the inherently 3D anatomies. However, its use is limited in clinical settings, due to the high purchasing costs and limited diagnostic practicality. Freehand 2D ultrasound imaging, in contrast, is routinely used in standard obstetric exams, but inherently lacks a 3D representation of the anatomies, which limits its potential for more advanced assessment. Such full representations are challenging to recover even with external tracking devices due to internal fetal movement which is independent from the operator-led trajectory of the probe. Capitalizing on the flexibility offered by freehand 2D ultrasound acquisition, we propose ImplicitVol to reconstruct 3D volumes from non-sensor-tracked 2D ultrasound sweeps. Conventionally, reconstructions are performed on a discrete voxel grid. We, however, employ a deep neural network to represent, for the first time, the reconstructed volume as an implicit function. Specifically, ImplicitVol takes a set of 2D images as input, predicts their locations in 3D space, jointly refines the inferred locations, and learns a full volumetric reconstruction. When testing natively-acquired and volume-sampled 2D ultrasound video sequences collected from different manufacturers, the 3D volumes reconstructed by ImplicitVol show significantly better visual and semantic quality than the existing interpolation-based reconstruction approaches. The inherent continuity of implicit representation also enables ImplicitVol to reconstruct the volume to arbitrarily high resolutions. As formulated, ImplicitVol has the potential to integrate seamlessly into the clinical workflow, while providing richer information for diagnosis and evaluation of the developing brain.