Combining Citizen Science and Deep Learning to Amplify Expertise in Neuroimaging

Quantifying brain development in the HEALthy Brain and Child Development (HBCD) Study: The magnetic resonance imaging and spectroscopy protocol

The HEALthy Brain and Child Development (HBCD) Study, a multi-site prospective longitudinal cohort study, will examine human brain, cognitive, behavioral, social, and emotional development beginning prenatally and planned through early childhood. The acquisition of multimodal magnetic resonance-based brain development data is central to the study's core protocol. However, application of Magnetic Resonance Imaging (MRI) methods in this population is complicated by technical challenges and difficulties of imaging in early life. Overcoming these challenges requires an innovative and harmonized approach, combining age-appropriate acquisition protocols together with specialized pediatric neuroimaging strategies. The HBCD MRI Working Group aimed to establish a core acquisition protocol for all 27 HBCD Study recruitment sites to measure brain structure, function, microstructure, and metabolites. Acquisition parameters of individual modalities have been matched across MRI scanner platforms for harmonized acquisitions and state-of-the-art technologies are employed to enable faster and motion-robust imaging. Here, we provide an overview of the HBCD MRI protocol, including decisions of individual modalities and preliminary data. The result will be an unparalleled resource for examining early neurodevelopment which enables the larger scientific community to assess normative trajectories from birth through childhood and to examine the genetic, biological, and environmental factors that help shape the developing brain.

Motion Artifact Detection for T1-Weighted Brain MR Images Using Convolutional Neural Networks

Quality assessment (QA) of magnetic resonance imaging (MRI) encompasses several factors such as noise, contrast, homogeneity, and imaging artifacts. Quality evaluation is often not standardized and relies on the expertise, and vigilance of the personnel, posing limitations especially with large datasets. Machine learning based on convolutional neural networks (CNNs) is a promising approach to address these challenges by performing automated inspection of MR images. In this study, a CNN for the detection of random head motion artifacts (RHM) in T1-weighted MRI as one aspect of image quality is proposed. A two-step approach aimed to first identify images exhibiting pronounced motion artifacts, and second to evaluate the feasibility of a more detailed three-class classification. The utilized dataset consisted of 420 T1-weighted whole-brain image volumes with isotropic resolution. Human experts assigned each volume to one of three classes of artifact prominence. Results demonstrate an accuracy of 95% for the identification of images with pronounced artifact load. The addition of an intermediate class retained an accuracy of 76%. The findings highlight the potential of CNN-based approaches to increase the efficiency of post-hoc QAs in large datasets by flagging images with potentially relevant artifact loads for closer inspection.

Functional connectivity development along the sensorimotor-association axis enhances the cortical hierarchy

Human cortical maturation has been posited to be organized along the sensorimotor-association axis, a hierarchical axis of brain organization that spans from unimodal sensorimotor cortices to transmodal association cortices. Here, we investigate the hypothesis that the development of functional connectivity during childhood through adolescence conforms to the cortical hierarchy defined by the sensorimotor-association axis. We tested this pre-registered hypothesis in four large-scale, independent datasets (total n = 3355; ages 5-23 years): the Philadelphia Neurodevelopmental Cohort (n = 1207), Nathan Kline Institute-Rockland Sample (n = 397), Human Connectome Project: Development (n = 625), and Healthy Brain Network (n = 1126). Across datasets, the development of functional connectivity systematically varied along the sensorimotor-association axis. Connectivity in sensorimotor regions increased, whereas connectivity in association cortices declined, refining and reinforcing the cortical hierarchy. These consistent and generalizable results establish that the sensorimotor-association axis of cortical organization encodes the dominant pattern of functional connectivity development.

Modelling heterogeneity in the classification process in multi-species distribution models can improve predictive performance

Species distribution models and maps from large-scale biodiversity data are necessary for conservation management. One current issue is that biodiversity data are prone to taxonomic misclassifications. Methods to account for these misclassifications in multi-species distribution models have assumed that the classification probabilities are constant throughout the study. In reality, classification probabilities are likely to vary with several covariates. Failure to account for such heterogeneity can lead to biased prediction of species distributions. Here, we present a general multi-species distribution model that accounts for heterogeneity in the classification process. The proposed model assumes a multinomial generalised linear model for the classification confusion matrix. We compare the performance of the heterogeneous classification model to that of the homogeneous classification model by assessing how well they estimate the parameters in the model and their predictive performance on hold-out samples. We applied the model to gull data from Norway, Denmark and Finland, obtained from the Global Biodiversity Information Facility. Our simulation study showed that accounting for heterogeneity in the classification process increased the precision of true species' identity predictions by 30% and accuracy and recall by 6%. Since all the models in this study accounted for misclassification of some sort, there was no significant effect of accounting for heterogeneity in the classification process on the inference about the ecological process. Applying the model framework to the gull dataset did not improve the predictive performance between the homogeneous and heterogeneous models (with parametric distributions) due to the smaller misclassified sample sizes. However, when machine learning predictive scores were used as weights to inform the species distribution models about the classification process, the precision increased by 70%. We recommend multiple multinomial regression to be used to model the variation in the classification process when the data contains relatively larger misclassified samples. Machine learning prediction scores should be used when the data contains relatively smaller misclassified samples.

A systematic review of (semi-)automatic quality control of T1-weighted MRI scans

PURPOSE

Artifacts in magnetic resonance imaging (MRI) scans degrade image quality and thus negatively affect the outcome measures of clinical and research scanning. Considering the time-consuming and subjective nature of visual quality control (QC), multiple (semi-)automatic QC algorithms have been developed. This systematic review presents an overview of the available (semi-)automatic QC algorithms and software packages designed for raw, structural T1-weighted (T1w) MRI datasets. The objective of this review was to identify the differences among these algorithms in terms of their features of interest, performance, and benchmarks.

METHODS

We queried PubMed, EMBASE (Ovid), and Web of Science databases on the fifth of January 2023, and cross-checked reference lists of retrieved papers. Bias assessment was performed using PROBAST (Prediction model Risk Of Bias ASsessment Tool).

RESULTS

A total of 18 distinct algorithms were identified, demonstrating significant variations in methods, features, datasets, and benchmarks. The algorithms were categorized into rule-based, classical machine learning-based, and deep learning-based approaches. Numerous unique features were defined, which can be roughly divided into features capturing entropy, contrast, and normative measures.

CONCLUSION

Due to dataset-specific optimization, it is challenging to draw broad conclusions about comparative performance. Additionally, large variations exist in the used datasets and benchmarks, further hindering direct algorithm comparison. The findings emphasize the need for standardization and comparative studies for advancing QC in MR imaging. Efforts should focus on identifying a dataset-independent measure as well as algorithm-independent methods for assessing the relative performance of different approaches.

The impact of quality control on cortical morphometry comparisons in autism

Structural magnetic resonance imaging (MRI) quality is known to impact and bias neuroanatomical estimates and downstream analysis, including case-control comparisons, and a growing body of work has demonstrated the importance of careful quality control (QC) and evaluated the impact of image and image-processing quality. However, the growing size of typical neuroimaging datasets presents an additional challenge to QC, which is typically extremely time and labour intensive. One of the most important aspects of MRI quality is the accuracy of processed outputs, which have been shown to impact estimated neurodevelopmental trajectories. Here, we evaluate whether the quality of surface reconstructions by FreeSurfer (one of the most widely used MRI processing pipelines) interacts with clinical and demographic factors. We present a tool, FSQC, that enables quick and efficient yet thorough assessment of outputs of the FreeSurfer processing pipeline. We validate our method against other existing QC metrics, including the automated FreeSurfer Euler number, two other manual ratings of raw image quality, and two popular automated QC methods. We show strikingly similar spatial patterns in the relationship between each QC measure and cortical thickness; relationships for cortical volume and surface area are largely consistent across metrics, though with some notable differences. We next demonstrate that thresholding by QC score attenuates but does not eliminate the impact of quality on cortical estimates. Finally, we explore different ways of controlling for quality when examining differences between autistic individuals and neurotypical controls in the Autism Brain Imaging Data Exchange (ABIDE) dataset, demonstrating that inadequate control for quality can alter results of case-control comparisons.

Quality control in functional MRI studies with MRIQC and fMRIPrep

The implementation of adequate quality assessment (QA) and quality control (QC) protocols within the magnetic resonance imaging (MRI) research workflow is resource- and time-consuming and even more so is their execution. As a result, QA/QC practices highly vary across laboratories and "MRI schools", ranging from highly specialized knowledge spots to environments where QA/QC is considered overly onerous and costly despite evidence showing that below-standard data increase the false positive and false negative rates of the final results. Here, we demonstrate a protocol based on the visual assessment of images one-by-one with reports generated by MRIQC and fMRIPrep, for the QC of data in functional (blood-oxygen dependent-level; BOLD) MRI analyses. We particularize the proposed, open-ended scope of application to whole-brain voxel-wise analyses of BOLD to correspondingly enumerate and define the exclusion criteria applied at the QC checkpoints. We apply our protocol on a composite dataset (n = 181 subjects) drawn from open fMRI studies, resulting in the exclusion of 97% of the data (176 subjects). This high exclusion rate was expected because subjects were selected to showcase artifacts. We describe the artifacts and defects more commonly found in the dataset that justified exclusion. We moreover release all the materials we generated in this assessment and document all the QC decisions with the expectation of contributing to the standardization of these procedures and engaging in the discussion of QA/QC by the community.

Application of a convolutional neural network to the quality control of MRI defacing

Large-scale neuroimaging datasets present unique challenges for automated processing pipelines. Motivated by a large clinical trials dataset with over 235,000 MRI scans, we consider the challenge of defacing - anonymisation to remove identifying facial features. The defacing process must undergo quality control (QC) checks to ensure that the facial features have been removed and that the brain tissue is left intact. Visual QC checks are time-consuming and can cause delays in preparing data. We have developed a convolutional neural network (CNN) that can assist with the QC of the application of MRI defacing; our CNN is able to distinguish between scans that are correctly defaced and can classify defacing failures into three sub-types to facilitate parameter tuning during remedial re-defacing. Since integrating the CNN into our anonymisation pipeline, over 75,000 scans have been processed. Strict thresholds have been applied so that ambiguous classifications are referred for visual QC checks, however all scans still undergo an efficient verification check before being marked as passed. After applying the thresholds, our network is 92% accurate and can classify nearly half of the scans without the need for protracted manual checks. Our model can generalise across MRI modalities and has comparable performance when tested on an independent dataset. Even with the introduction of the verification checks, incorporation of the CNN has reduced the time spent undertaking QC checks by 42% during initial defacing, and by 35% overall. With the help of the CNN, we have been able to successfully deface 96% of the scans in the project whilst maintaining high QC standards. In a similarly sized new project, we would expect the model to reduce the time spent on manual QC checks by 125 h. Our approach is applicable to other projects with the potential to greatly improve the efficiency of imaging anonymisation pipelines.

An analysis-ready and quality controlled resource for pediatric brain white-matter research

We created a set of resources to enable research based on openly-available diffusion MRI (dMRI) data from the Healthy Brain Network (HBN) study. First, we curated the HBN dMRI data (N = 2747) into the Brain Imaging Data Structure and preprocessed it according to best-practices, including denoising and correcting for motion effects, susceptibility-related distortions, and eddy currents. Preprocessed, analysis-ready data was made openly available. Data quality plays a key role in the analysis of dMRI. To optimize QC and scale it to this large dataset, we trained a neural network through the combination of a small data subset scored by experts and a larger set scored by community scientists. The network performs QC highly concordant with that of experts on a held out set (ROC-AUC = 0.947). A further analysis of the neural network demonstrates that it relies on image features with relevance to QC. Altogether, this work both delivers resources to advance transdiagnostic research in brain connectivity and pediatric mental health, and establishes a novel paradigm for automated QC of large datasets.

How Machine Learning is Powering Neuroimaging to Improve Brain Health

This report presents an overview of how machine learning is rapidly advancing clinical translational imaging in ways that will aid in the early detection, prediction, and treatment of diseases that threaten brain health. Towards this goal, we aresharing the information presented at a symposium, "Neuroimaging Indicators of Brain Structure and Function - Closing the Gap Between Research and Clinical Application", co-hosted by the McCance Center for Brain Health at Mass General Hospital and the MIT HST Neuroimaging Training Program on February 12, 2021. The symposium focused on the potential for machine learning approaches, applied to increasingly large-scale neuroimaging datasets, to transform healthcare delivery and change the trajectory of brain health by addressing brain care earlier in the lifespan. While not exhaustive, this overview uniquely addresses many of the technical challenges from image formation, to analysis and visualization, to synthesis and incorporation into the clinical workflow. Some of the ethical challenges inherent to this work are also explored, as are some of the regulatory requirements for implementation. We seek to educate, motivate, and inspire graduate students, postdoctoral fellows, and early career investigators to contribute to a future where neuroimaging meaningfully contributes to the maintenance of brain health.

A longitudinal resource for studying connectome development and its psychiatric associations during childhood

Most psychiatric disorders are chronic, associated with high levels of disability and distress, and present during pediatric development. Scientific innovation increasingly allows researchers to probe brain-behavior relationships in the developing human. As a result, ambitions to (1) establish normative pediatric brain development trajectories akin to growth curves, (2) characterize reliable metrics for distinguishing illness, and (3) develop clinically useful tools to assist in the diagnosis and management of mental health and learning disorders have gained significant momentum. To this end, the NKI-Rockland Sample initiative was created to probe lifespan development as a large-scale multimodal dataset. The NKI-Rockland Sample Longitudinal Discovery of Brain Development Trajectories substudy (N = 369) is a 24- to 30-month multi-cohort longitudinal pediatric investigation (ages 6.0-17.0 at enrollment) carried out in a community-ascertained sample. Data include psychiatric diagnostic, medical, behavioral, and cognitive phenotyping, as well as multimodal brain imaging (resting fMRI, diffusion MRI, morphometric MRI, arterial spin labeling), genetics, and actigraphy. Herein, we present the rationale, design, and implementation of the Longitudinal Discovery of Brain Development Trajectories protocol.

Combining collective and artificial intelligence for global health diseases diagnosis using crowdsourced annotated medical images

Visual inspection of microscopic samples is still the gold standard diagnostic methodology for many global health diseases. Soil-transmitted helminth infection affects 1.5 billion people worldwide, and is the most prevalent disease among the Neglected Tropical Diseases. It is diagnosed by manual examination of stool samples by microscopy, which is a time-consuming task and requires trained personnel and high specialization. Artificial intelligence could automate this task making the diagnosis more accessible. Still, it needs a large amount of annotated training data coming from experts.In this work, we proposed the use of crowdsourced annotated medical images to train AI models (neural networks) for the detection of soil-transmitted helminthiasis in microscopy images from stool samples leveraging non-expert knowledge collected through playing a video game. We collected annotations made by both school-age children and adults, and we showed that, although the quality of crowdsourced annotations made by school-age children are sightly inferior than the ones made by adults, AI models trained on these crowdsourced annotations perform similarly (AUC of 0.928 and 0.939 respectively), and reach similar performance to the AI model trained on expert annotations (AUC of 0.932). We also showed the impact of the training sample size and continuous training on the performance of the AI models.In conclusion, the workflow proposed in this work combined collective and artificial intelligence for detecting soil-transmitted helminthiasis. Embedded within a digital health platform can be applied to any other medical image analysis task and contribute to reduce the burden of disease.

The Partnership of Citizen Science and Machine Learning: Benefits, Risks, and Future Challenges for Engagement, Data Collection, and Data Quality

Advances in artificial intelligence (AI) and the extension of citizen science to various scientific areas, as well as the generation of big citizen science data, are resulting in AI and citizen science being good partners, and their combination benefits both fields. The integration of AI and citizen science has mostly been used in biodiversity projects, with the primary focus on using citizen science data to train machine learning (ML) algorithms for automatic species identification. In this article, we will look at how ML techniques can be used in citizen science and how they can influence volunteer engagement, data collection, and data validation. We reviewed several use cases from various domains and categorized them according to the ML technique used and the impact of ML on citizen science in each project. Furthermore, the benefits and risks of integrating ML in citizen science are explored, and some recommendations are provided on how to enhance the benefits while mitigating the risks of this integration. Finally, because this integration is still in its early phases, we have proposed some potential ideas and challenges that can be implemented in the future to leverage the power of the combination of citizen science and AI, with the key emphasis being on citizen science in this article.

Brainhack: Developing a culture of open, inclusive, community-driven neuroscience

Brainhack is an innovative meeting format that promotes scientific collaboration and education in an open, inclusive environment. This NeuroView describes the myriad benefits for participants and the research community and how Brainhacks complement conventional formats to augment scientific progress.

Management and Quality Control of Large Neuroimaging Datasets: Developments From the Barcelonaβeta Brain Research Center

Recent decades have witnessed an increasing number of large to very large imaging studies, prominently in the field of neurodegenerative diseases. The datasets collected during these studies form essential resources for the research aiming at new biomarkers. Collecting, hosting, managing, processing, or reviewing those datasets is typically achieved through a local neuroinformatics infrastructure. In particular for organizations with their own imaging equipment, setting up such a system is still a hard task, and relying on cloud-based solutions, albeit promising, is not always possible. This paper proposes a practical model guided by core principles including user involvement, lightweight footprint, modularity, reusability, and facilitated data sharing. This model is based on the experience from an 8-year-old research center managing cohort research programs on Alzheimer’s disease. Such a model gave rise to an ecosystem of tools aiming at improved quality control through seamless automatic processes combined with a variety of code libraries, command line tools, graphical user interfaces, and instant messaging applets. The present ecosystem was shaped around XNAT and is composed of independently reusable modules that are freely available on GitLab/GitHub. This paradigm is scalable to the general community of researchers working with large neuroimaging datasets.

Variations in structural MRI quality significantly impact commonly used measures of brain anatomy

Subject motion can introduce noise into neuroimaging data and result in biased estimations of brain structure. In-scanner motion can compromise data quality in a number of ways and varies widely across developmental and clinical populations. However, quantification of structural image quality is often limited to proxy or indirect measures gathered from functional scans; this may be missing true differences related to these potential artifacts. In this study, we take advantage of novel informatic tools, the CAT12 toolbox, to more directly measure image quality from T1-weighted images to understand if these measures of image quality: (1) relate to rigorous quality-control checks visually completed by human raters; (2) are associated with sociodemographic variables of interest; (3) influence regional estimates of cortical surface area, cortical thickness, and subcortical volumes from the commonly used Freesurfer tool suite. We leverage public-access data that includes a community-based sample of children and adolescents, spanning a large age-range (N = 388; ages 5-21). Interestingly, even after visually inspecting our data, we find image quality significantly impacts derived cortical surface area, cortical thickness, and subcortical volumes from multiple regions across the brain (~ 23.4% of all areas investigated). We believe these results are important for research groups completing structural MRI studies using Freesurfer or other morphometric tools. As such, future studies should consider using measures of image quality to minimize the influence of this potential confound in group comparisons or studies focused on individual differences.

Is it time to switch your T1W sequence? Assessing the impact of prospective motion correction on the reliability and quality of structural imaging

New large neuroimaging studies, such as the Adolescent Brain Cognitive Development study (ABCD) and Human Connectome Project (HCP) Development studies are adopting a new T1-weighted imaging sequence with prospective motion correction (PMC) in favor of the more traditional 3-Dimensional Magnetization-Prepared Rapid Gradient-Echo Imaging (MPRAGE) sequence. Here, we used a developmental dataset (ages 5-21, N = 348) from the Healthy Brain Network (HBN) Initiative to directly compare two widely used MRI structural sequences: one based on the Human Connectome Project (MPRAGE) and another based on the ABCD study (MPRAGE+PMC). We aimed to determine if the morphometric measurements obtained from both protocols are equivalent or if one sequence has a clear advantage over the other. The sequences were also compared through quality control measurements. Inter- and intra-sequence reliability were assessed with another set of participants (N = 71) from HBN that performed two MPRAGE and two MPRAGE+PMC sequences within the same imaging session, with one MPRAGE (MPRAGE1) and MPRAGE+PMC (MPRAGE+PMC1) pair at the beginning of the session and another pair (MPRAGE2 and MPRAGE+PMC2) at the end of the session. Intraclass correlation coefficients (ICC) scores for morphometric measurements such as volume and cortical thickness showed that intra-sequence reliability is the highest with the two MPRAGE+PMC sequences and lowest with the two MPRAGE sequences. Regarding inter-sequence reliability, ICC scores were higher for the MPRAGE1 - MPRAGE+PMC1 pair at the beginning of the session than the MPRAGE1 - MPRAGE2 pair, possibly due to the higher motion artifacts in the MPRAGE2 run. Results also indicated that the MPRAGE+PMC sequence is robust, but not impervious, to high head motion. For quality control metrics, the traditional MPRAGE yielded better results than MPRAGE+PMC in 5 of the 8 measurements. In conclusion, morphometric measurements evaluated here showed high inter-sequence reliability between the MPRAGE and MPRAGE+PMC sequences, especially in images with low head motion. We suggest that studies targeting hyperkinetic populations use the MPRAGE+PMC sequence, given its robustness to head motion and higher reliability scores. However, neuroimaging researchers studying non-hyperkinetic participants can choose either MPRAGE or MPRAGE+PMC sequences, but should carefully consider the apparent tradeoff between relatively increased reliability, but reduced quality control metrics when using the MPRAGE+PMC sequence.

Comparison of Objective Image Quality Metrics to Expert Radiologists' Scoring of Diagnostic Quality of MR Images

Image quality metrics (IQMs) such as root mean square error (RMSE) and structural similarity index (SSIM) are commonly used in the evaluation and optimization of accelerated magnetic resonance imaging (MRI) acquisition and reconstruction strategies. However, it is unknown how well these indices relate to a radiologist's perception of diagnostic image quality. In this study, we compare the image quality scores of five radiologists with the RMSE, SSIM, and other potentially useful IQMs: peak signal to noise ratio (PSNR) multi-scale SSIM (MSSSIM), information-weighted SSIM (IWSSIM), gradient magnitude similarity deviation (GMSD), feature similarity index (FSIM), high dynamic range visible difference predictor (HDRVDP), noise quality metric (NQM), and visual information fidelity (VIF). The comparison uses a database of MR images of the brain and abdomen that have been retrospectively degraded by noise, blurring, undersampling, motion, and wavelet compression for a total of 414 degraded images. A total of 1017 subjective scores were assigned by five radiologists. IQM performance was measured via the Spearman rank order correlation coefficient (SROCC) and statistically significant differences in the residuals of the IQM scores and radiologists' scores were tested. When considering SROCC calculated from combining scores from all radiologists across all image types, RMSE and SSIM had lower SROCC than six of the other IQMs included in the study (VIF, FSIM, NQM, GMSD, IWSSIM, and HDRVDP). In no case did SSIM have a higher SROCC or significantly smaller residuals than RMSE. These results should be considered when choosing an IQM in future imaging studies.

A Standardized Protocol for Efficient and Reliable Quality Control of Brain Registration in Functional MRI Studies

Automatic alignment of brain anatomy in a standard space is a key step when processing magnetic resonance imaging for group analyses. Such brain registration is prone to failure, and the results are therefore typically reviewed visually to ensure quality. There is however no standard, validated protocol available to perform this visual quality control (QC). We propose here a standardized QC protocol for brain registration, with minimal training overhead and no required knowledge of brain anatomy. We validated the reliability of three-level QC ratings (OK, Maybe, Fail) across different raters. Nine experts each rated N = 100 validation images, and reached moderate to good agreement (kappa from 0.4 to 0.68, average of 0.54 ± 0.08), with the highest agreement for "Fail" images (Dice from 0.67 to 0.93, average of 0.8 ± 0.06). We then recruited volunteers through the Zooniverse crowdsourcing platform, and extracted a consensus panel rating for both the Zooniverse raters (N = 41) and the expert raters. The agreement between expert and Zooniverse panels was high (kappa = 0.76). Overall, our protocol achieved a good reliability when performing a two level assessment (Fail vs. OK/Maybe) by an individual rater, or aggregating multiple three-level ratings (OK, Maybe, Fail) from a panel of experts (3 minimum) or non-experts (15 minimum). Our brain registration QC protocol will help standardize QC practices across laboratories, improve the consistency of reporting of QC in publications, and will open the way for QC assessment of large datasets which could be used to train automated QC systems.

Knowing What You Know in Brain Segmentation Using Bayesian Deep Neural Networks

In this paper, we describe a Bayesian deep neural network (DNN) for predicting FreeSurfer segmentations of structural MRI volumes, in minutes rather than hours. The network was trained and evaluated on a large dataset (n = 11,480), obtained by combining data from more than a hundred different sites, and also evaluated on another completely held-out dataset (n = 418). The network was trained using a novel spike-and-slab dropout-based variational inference approach. We show that, on these datasets, the proposed Bayesian DNN outperforms previously proposed methods, in terms of the similarity between the segmentation predictions and the FreeSurfer labels, and the usefulness of the estimate uncertainty of these predictions. In particular, we demonstrated that the prediction uncertainty of this network at each voxel is a good indicator of whether the network has made an error and that the uncertainty across the whole brain can predict the manual quality control ratings of a scan. The proposed Bayesian DNN method should be applicable to any new network architecture for addressing the segmentation problem.

Supervised machine learning quality control for magnetic resonance artifacts in neonatal data sets

Quality control (QC) of brain magnetic resonance images (MRI) is an important process requiring a significant amount of manual inspection. Major artifacts, such as severe subject motion, are easy to identify to naïve observers but lack automated identification tools. Clinical trials involving motion-prone neonates typically pool data to obtain sufficient power, and automated quality control protocols are especially important to safeguard data quality. Current study tested an open source method to detect major artifacts among 2D neonatal MRI via supervised machine learning. A total of 1,020 two-dimensional transverse T2-weighted MRI images of preterm newborns were examined and classified as either QC Pass or QC Fail. Then 70 features across focus, texture, noise, and natural scene statistics categories were extracted from each image. Several different classifiers were trained and their performance was compared with subjective rating as the gold standard. We repeated the rating process again to examine the stability of the rating and classification. When tested via 10-fold cross validation, the random undersampling and adaboost ensemble (RUSBoost) method achieved the best overall performance for QC Fail images with 85% positive predictive value along with 75% sensitivity. Similar classification performance was observed in the analyses of the repeated subjective rating. Current results served as a proof of concept for predicting images that fail quality control using no-reference objective image features. We also highlighted the importance of evaluating results beyond mere accuracy as a performance measure for machine learning in imbalanced group settings due to larger proportion of QC Pass quality images.