A multi-task, multi-stage deep transfer learning model for early prediction of neurodevelopment in very preterm infants

Machine Learning: A Potential Therapeutic Tool to Facilitate Neonatal Therapeutic Decision Making

Bacterial infection is one of the major causes of neonatal morbidity and mortality worldwide. Finding rapid and reliable methods for early recognition and diagnosis of bacterial infections and early individualization of antibacterial drug administration are essential to eradicate these infections and prevent serious complications. However, this is often difficult to perform due to non-specific clinical presentations, low accuracy of current diagnostic methods, and limited knowledge of neonatal pharmacokinetics. Although neonatal medicine has been relatively late to embrace the benefits of machine learning (ML), there have been some initial applications of ML for the early prediction of neonatal sepsis and individualization of antibiotics. This article provides a brief introduction to ML and discusses the current state of the art in diagnosing and treating neonatal bacterial infections, gaps, potential uses of ML, and future directions to address the limitations of current studies. Neonatal bacterial infections involve a combination of physiologic development, disease expression, and treatment response outcomes. To address this complex relationship, future models could consider appropriate ML algorithms to capture time series features while integrating influences from the host, microbes, and drugs to optimize antimicrobial drug use in neonates. All models require prospective clinical trials to validate their clinical utility before clinical use.

Computational Approaches for Connecting Maternal Stress to Preterm Birth

Multiple studies have hinted at a complex connection between maternal stress and preterm birth (PTB). This article describes the potential of computational methods to provide new insights into this relationship. For this, we outline existing approaches for stress assessments and various data modalities available for profiling stress responses, and review studies that sought either to establish a connection between stress and PTB or to predict PTB based on stress-related factors. Finally, we summarize the challenges of computational methods, highlighting potential future research directions within this field.

Prediction of treatment response in major depressive disorder using a hybrid of convolutional recurrent deep neural networks and effective connectivity based on EEG signal

In this study, we have developed a novel method based on deep learning and brain effective connectivity to classify responders and non-responders to selective serotonin reuptake inhibitors (SSRIs) antidepressants in major depressive disorder (MDD) patients prior to the treatment using EEG signal. The effective connectivity of 30 MDD patients was determined by analyzing their pretreatment EEG signals, which were then concatenated into delta, theta, alpha, and beta bands and transformed into images. Using these images, we then fine tuned a hybrid Convolutional Neural Network that is enhanced with bidirectional Long Short-Term Memory cells based on transfer learning. The Inception-v3, ResNet18, DenseNet121, and EfficientNet-B0 models are implemented as base models. Finally, the models are followed by BiLSTM and dense layers in order to classify responders and non-responders to SSRI treatment. Results showed that the EfficiencyNet-B0 has the highest accuracy of 98.33, followed by DensNet121, ResNet18 and Inception-v3. Therefore, a new method was proposed in this study that uses deep learning models to extract both spatial and temporal features automatically, which will improve classification results. The proposed method provides accurate identification of MDD patients who are responding, thereby reducing the cost of medical facilities and patient care.

Progress and trends in neurological disorders research based on deep learning

In recent years, deep learning (DL) has emerged as a powerful tool in clinical imaging, offering unprecedented opportunities for the diagnosis and treatment of neurological disorders (NDs). This comprehensive review explores the multifaceted role of DL techniques in leveraging vast datasets to advance our understanding of NDs and improve clinical outcomes. Beginning with a systematic literature review, we delve into the utilization of DL, particularly focusing on multimodal neuroimaging data analysis-a domain that has witnessed rapid progress and garnered significant scientific interest. Our study categorizes and critically analyses numerous DL models, including Convolutional Neural Networks (CNNs), LSTM-CNN, GAN, and VGG, to understand their performance across different types of Neurology Diseases. Through particular analysis, we identify key benchmarks and datasets utilized in training and testing DL models, shedding light on the challenges and opportunities in clinical neuroimaging research. Moreover, we discuss the effectiveness of DL in real-world clinical scenarios, emphasizing its potential to revolutionize ND diagnosis and therapy. By synthesizing existing literature and describing future directions, this review not only provides insights into the current state of DL applications in ND analysis but also covers the way for the development of more efficient and accessible DL techniques. Finally, our findings underscore the transformative impact of DL in reshaping the landscape of clinical neuroimaging, offering hope for enhanced patient care and groundbreaking discoveries in the field of neurology. This review paper is beneficial for neuropathologists and new researchers in this field.

Supervised contrastive learning enhances graph convolutional networks for predicting neurodevelopmental deficits in very preterm infants using brain structural connectome

Very preterm (VPT) infants (born at less than 32 weeks gestational age) are at high risk for various adverse neurodevelopmental deficits. Unfortunately, most of these deficits cannot be accurately diagnosed until the age of 2-5 years old. Given the benefits of early interventions, accurate diagnosis and prediction soon after birth are urgently needed for VPT infants. Previous studies have applied deep learning models to learn the brain structural connectome (SC) to predict neurodevelopmental deficits in the preterm population. However, none of these models are specifically designed for graph-structured data, and thus may potentially miss certain topological information conveyed in the brain SC. In this study, we aim to develop deep learning models to learn the SC acquired at term-equivalent age for early prediction of neurodevelopmental deficits at 2 years corrected age in VPT infants. We directly treated the brain SC as a graph, and applied graph convolutional network (GCN) models to capture complex topological information of the SC. In addition, we applied the supervised contrastive learning (SCL) technique to mitigate the effects of the data scarcity problem, and enable robust training of GCN models. We hypothesize that SCL will enhance GCN models for early prediction of neurodevelopmental deficits in VPT infants using the SC. We used a regional prospective cohort of ∼280 VPT infants who underwent MRI examinations at term-equivalent age from the Cincinnati Infant Neurodevelopment Early Prediction Study (CINEPS). These VPT infants completed neurodevelopmental assessment at 2 years corrected age to evaluate cognition, language, and motor skills. Using the SCL technique, the GCN model achieved mean areas under the receiver operating characteristic curve (AUCs) in the range of 0.72∼0.75 for predicting three neurodevelopmental deficits, outperforming several competing models. Our results support our hypothesis that the SCL technique is able to enhance the GCN model in our prediction tasks.

Applied Artificial Intelligence in Healthcare: A Review of Computer Vision Technology Application in Hospital Settings

Computer vision (CV), a type of artificial intelligence (AI) that uses digital videos or a sequence of images to recognize content, has been used extensively across industries in recent years. However, in the healthcare industry, its applications are limited by factors like privacy, safety, and ethical concerns. Despite this, CV has the potential to improve patient monitoring, and system efficiencies, while reducing workload. In contrast to previous reviews, we focus on the end-user applications of CV. First, we briefly review and categorize CV applications in other industries (job enhancement, surveillance and monitoring, automation, and augmented reality). We then review the developments of CV in the hospital setting, outpatient, and community settings. The recent advances in monitoring delirium, pain and sedation, patient deterioration, mechanical ventilation, mobility, patient safety, surgical applications, quantification of workload in the hospital, and monitoring for patient events outside the hospital are highlighted. To identify opportunities for future applications, we also completed journey mapping at different system levels. Lastly, we discuss the privacy, safety, and ethical considerations associated with CV and outline processes in algorithm development and testing that limit CV expansion in healthcare. This comprehensive review highlights CV applications and ideas for its expanded use in healthcare.

Do it the transformer way: A comprehensive review of brain and vision transformers for autism spectrum disorder diagnosis and classification

Autism spectrum disorder (ASD) is a condition observed in children who display abnormal patterns of interaction, behavior, and communication with others. Despite extensive research efforts, the underlying causes of this neurodevelopmental disorder and its biomarkers remain unknown. However, advancements in artificial intelligence and machine learning have improved clinicians' ability to diagnose ASD. This review paper investigates various MRI modalities to identify distinct features that characterize individuals with ASD compared to typical control subjects. The review then moves on to explore deep learning models for ASD diagnosis, including convolutional neural networks (CNNs), autoencoders, graph convolutions, attention networks, and other models. CNNs and their variations are particularly effective due to their capacity to learn structured image representations and identify reliable biomarkers for brain disorders. Computer vision transformers often employ CNN architectures with transfer learning techniques like fine-tuning and layer freezing to enhance image classification performance, surpassing traditional machine learning models. This review paper contributes in three main ways. Firstly, it provides a comprehensive overview of a recommended architecture for using vision transformers in the systematic ASD diagnostic process. To this end, the paper investigates various pre-trained vision architectures such as VGG, ResNet, Inception, InceptionResNet, DenseNet, and Swin models that were fine-tuned for ASD diagnosis and classification. Secondly, it discusses the vision transformers of 2020th like BiT, ViT, MobileViT, and ConvNeXt, and applying transfer learning methods in relation to their prospective practicality in ASD classification. Thirdly, it explores brain transformers that are pre-trained on medically rich data and MRI neuroimaging datasets. The paper recommends a systematic architecture for ASD diagnosis using brain transformers. It also reviews recently developed brain transformer-based models, such as METAFormer, Com-BrainTF, Brain Network, ST-Transformer, STCAL, BolT, and BrainFormer, discussing their deep transfer learning architectures and results in ASD detection. Additionally, the paper summarizes and discusses brain-related transformers for various brain disorders, such as MSGTN, STAGIN, and MedTransformer, in relation to their potential usefulness in ASD. The study suggests that developing specialized transformer-based models, following the success of natural language processing (NLP), can offer new directions for image classification problems in ASD brain biomarkers learning and classification. By incorporating the attention mechanism, treating MRI modalities as sequence prediction tasks trained on brain disorder classification problems, and fine-tuned on ASD datasets, brain transformers can show a great promise in ASD diagnosis.

The past, current, and future of neonatal intensive care units with artificial intelligence: a systematic review

Machine learning and deep learning are two subsets of artificial intelligence that involve teaching computers to learn and make decisions from any sort of data. Most recent developments in artificial intelligence are coming from deep learning, which has proven revolutionary in almost all fields, from computer vision to health sciences. The effects of deep learning in medicine have changed the conventional ways of clinical application significantly. Although some sub-fields of medicine, such as pediatrics, have been relatively slow in receiving the critical benefits of deep learning, related research in pediatrics has started to accumulate to a significant level, too. Hence, in this paper, we review recently developed machine learning and deep learning-based solutions for neonatology applications. We systematically evaluate the roles of both classical machine learning and deep learning in neonatology applications, define the methodologies, including algorithmic developments, and describe the remaining challenges in the assessment of neonatal diseases by using PRISMA 2020 guidelines. To date, the primary areas of focus in neonatology regarding AI applications have included survival analysis, neuroimaging, analysis of vital parameters and biosignals, and retinopathy of prematurity diagnosis. We have categorically summarized 106 research articles from 1996 to 2022 and discussed their pros and cons, respectively. In this systematic review, we aimed to further enhance the comprehensiveness of the study. We also discuss possible directions for new AI models and the future of neonatology with the rising power of AI, suggesting roadmaps for the integration of AI into neonatal intensive care units.

Machine learning and disease prediction in obstetrics

Machine learning technologies and translation of artificial intelligence tools to enhance the patient experience are changing obstetric and maternity care. An increasing number of predictive tools have been developed with data sourced from electronic health records, diagnostic imaging and digital devices. In this review, we explore the latest tools of machine learning, the algorithms to establish prediction models and the challenges to assess fetal well-being, predict and diagnose obstetric diseases such as gestational diabetes, pre-eclampsia, preterm birth and fetal growth restriction. We discuss the rapid growth of machine learning approaches and intelligent tools for automated diagnostic imaging of fetal anomalies and to asses fetoplacental and cervix function using ultrasound and magnetic resonance imaging. In prenatal diagnosis, we discuss intelligent tools for magnetic resonance imaging sequencing of the fetus, placenta and cervix to reduce the risk of preterm birth. Finally, the use of machine learning to improve safety standards in intrapartum care and early detection of complications will be discussed. The demand for technologies to enhance diagnosis and treatment in obstetrics and maternity should improve frameworks for patient safety and enhance clinical practice.

A Semi-Supervised Graph Convolutional Network for Early Prediction of Motor Abnormalities in Very Preterm Infants

Approximately 32-42% of very preterm infants develop minor motor abnormalities. Earlier diagnosis soon after birth is urgently needed because the first two years of life represent a critical window of opportunity for early neuroplasticity in infants. In this study, we developed a semi-supervised graph convolutional network (GCN) model that is able to simultaneously learn the neuroimaging features of subjects and consider the pairwise similarity between them. The semi-supervised GCN model also allows us to combine labeled data with additional unlabeled data to facilitate model training. We conducted our experiments on a multisite regional cohort of 224 preterm infants (119 labeled subjects and 105 unlabeled subjects) who were born at 32 weeks or earlier from the Cincinnati Infant Neurodevelopment Early Prediction Study. A weighted loss function was applied to mitigate the impact of an imbalanced positive:negative (~1:2) subject ratio in our cohort. With only labeled data, our GCN model achieved an accuracy of 66.4% and an AUC of 0.67 in the early prediction of motor abnormalities, outperforming prior supervised learning models. By taking advantage of additional unlabeled data, the GCN model had significantly better accuracy (68.0%, p = 0.016) and a higher AUC (0.69, p = 0.029). This pilot work suggests that the semi-supervised GCN model can be utilized to aid early prediction of neurodevelopmental deficits in preterm infants.

Application of artificial intelligence for automatic cataract staging based on anterior segment images: comparing automatic segmentation approaches to manual segmentation

Purpose

Cataract is one of the leading causes of blindness worldwide, accounting for >50% of cases of blindness in low- and middle-income countries. In this study, two artificial intelligence (AI) diagnosis platforms are proposed for cortical cataract staging to achieve a precise diagnosis.

Methods

A total of 647 high quality anterior segment images, which included the four stages of cataracts, were collected into the dataset. They were divided randomly into a training set and a test set using a stratified random-allocation technique at a ratio of 8:2. Then, after automatic or manual segmentation of the lens area of the cataract, the deep transform-learning (DTL) features extraction, PCA dimensionality reduction, multi-features fusion, fusion features selection, and classification models establishment, the automatic and manual segmentation DTL platforms were developed. Finally, the accuracy, confusion matrix, and area under the receiver operating characteristic (ROC) curve (AUC) were used to evaluate the performance of the two platforms.

Results

In the automatic segmentation DTL platform, the accuracy of the model in the training and test sets was 94.59 and 84.50%, respectively. In the manual segmentation DTL platform, the accuracy of the model in the training and test sets was 97.48 and 90.00%, respectively. In the test set, the micro and macro average AUCs of the two platforms reached >95% and the AUC for each classification was >90%. The results of a confusion matrix showed that all stages, except for mature, had a high recognition rate.

Conclusion

Two AI diagnosis platforms were proposed for cortical cataract staging. The resulting automatic segmentation platform can stage cataracts more quickly, whereas the resulting manual segmentation platform can stage cataracts more accurately.

Machine learning for understanding and predicting neurodevelopmental outcomes in premature infants: a systematic review

BACKGROUND

Machine learning has been attracting increasing attention for use in healthcare applications, including neonatal medicine. One application for this tool is in understanding and predicting neurodevelopmental outcomes in preterm infants. In this study, we have carried out a systematic review to identify findings and challenges to date.

METHODS

This systematic review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analysis guidelines. Four databases were searched in February 2022, with articles then screened in a non-blinded manner by two authors.

RESULTS

The literature search returned 278 studies, with 11 meeting the eligibility criteria for inclusion. Convolutional neural networks were the most common machine learning approach, with most studies seeking to predict neurodevelopmental outcomes from images and connectomes describing brain structure and function. Studies to date also sought to identify features predictive of outcomes; however, results varied greatly.

CONCLUSIONS

Initial studies in this field have achieved promising results; however, many machine learning techniques remain to be explored, and the consensus is yet to be reached on which clinical and brain features are most predictive of neurodevelopmental outcomes.

IMPACT

This systematic review looks at the question of whether machine learning can be used to predict and understand neurodevelopmental outcomes in preterm infants. Our review finds that promising initial works have been conducted in this field, but many challenges and opportunities remain. Quality assessment of relevant articles is conducted using the Newcastle-Ottawa Scale. This work identifies challenges that remain and suggests several key directions for future research. To the best of the authors' knowledge, this is the first systematic review to explore this topic.

Current and emerging artificial intelligence applications for pediatric abdominal imaging

Artificial intelligence (AI) uses computers to mimic cognitive functions of the human brain, allowing inferences to be made from generally large datasets. Traditional machine learning (e.g., decision tree analysis, support vector machines) and deep learning (e.g., convolutional neural networks) are two commonly employed AI approaches both outside and within the field of medicine. Such techniques can be used to evaluate medical images for the purposes of automated detection and segmentation, classification tasks (including diagnosis, lesion or tissue characterization, and prediction), and image reconstruction. In this review article we highlight recent literature describing current and emerging AI methods applied to abdominal imaging (e.g., CT, MRI and US) and suggest potential future applications of AI in the pediatric population.

Biomarker und Neuromonitoring zur Entwicklungsprognose nach perinataler Hirnschädigung

Das sich entwickelnde Gehirn ist in der Perinatalperiode besonders empfindlich für eine Vielzahl von Insulten, wie z. B. Extremfrühgeburtlichkeit und perinatale Asphyxie. Ihre Komplikationen können zu lebenslangen neurokognitiven, sensorischen und psychosozialen Einschränkungen führen; deren Vorhersage bleibt eine Herausforderung. Eine Schlüsselfunktion kommt der möglichst exakten Identifikation von Hirnläsionen und funktionellen Störungen zu. Die Prädiktion stützt sich auf frühe diagnostische Verfahren und die klinische Erfassung der Meilensteine der Entwicklung. Zur klinischen Diagnostik und zum Neuromonitoring in der Neonatal- und frühen Säuglingsperiode stehen bildgebende Verfahren zur Verfügung. Hierzu zählen zerebrale Sonographie, MRT am errechneten Termin, amplitudenintegriertes (a)EEG und/oder klassisches EEG, Nah-Infrarot-Spektroskopie, General Movements Assessment und die frühe klinische Nachuntersuchung z. B. mithilfe der Hammersmith Neonatal/Infant Neurological Examination. Innovative Biomarker und -muster (Omics) sowie (epi)genetische Prädispositionen sind Gegenstand wissenschaftlicher Untersuchungen. Neben der Erfassung klinischer Risiken kommt psychosozialen Faktoren im Umfeld des Kindes eine entscheidende Rolle zu. Eine möglichst akkurate Prognose ist mit hohem Aufwand verbunden, jedoch zur gezielten Beratung der Familien und der Einleitung von frühen Interventionen, insbesondere vor dem Hintergrund der hohen Plastizität des sich entwickelnden Gehirns, von großer Bedeutung. Diese Übersichtsarbeit fokussiert die Charakterisierung der oben genannten Verfahren und ihrer Kombinationsmöglichkeiten. Zudem wird ein Ausblick gegeben, wie innovative Techniken in Zukunft die Prädiktion der Entwicklung und Nachsorge dieser Kinder vereinfachen können.

Machine Learning Prediction Models for Neurodevelopmental Outcome After Preterm Birth: A Scoping Review and New Machine Learning Evaluation Framework

BACKGROUND AND OBJECTIVES

Outcome prediction of preterm birth is important for neonatal care, yet prediction performance using conventional statistical models remains insufficient. Machine learning has a high potential for complex outcome prediction. In this scoping review, we provide an overview of the current applications of machine learning models in the prediction of neurodevelopmental outcomes in preterm infants, assess the quality of the developed models, and provide guidance for future application of machine learning models to predict neurodevelopmental outcomes of preterm infants.

METHODS

A systematic search was performed using PubMed. Studies were included if they reported on neurodevelopmental outcome prediction in preterm infants using predictors from the neonatal period and applying machine learning techniques. Data extraction and quality assessment were independently performed by 2 reviewers.

RESULTS

Fourteen studies were included, focusing mainly on very or extreme preterm infants, predicting neurodevelopmental outcome before age 3 years, and mostly assessing outcomes using the Bayley Scales of Infant Development. Predictors were most often based on MRI. The most prevalent machine learning techniques included linear regression and neural networks. None of the studies met all newly developed quality assessment criteria. Studies least prone to inflated performance showed promising results, with areas under the curve up to 0.86 for classification and R2 values up to 91% in continuous prediction. A limitation was that only 1 data source was used for the literature search.

CONCLUSIONS

Studies least prone to inflated prediction results are the most promising. The provided evaluation framework may contribute to improved quality of future machine learning models.

Current Status and Future Directions of Neuromonitoring With Emerging Technologies in Neonatal Care

Neonatology has experienced a significant reduction in mortality rates of the preterm population and critically ill infants over the last few decades. Now, the emphasis is directed toward improving long-term neurodevelopmental outcomes and quality of life. Brain-focused care has emerged as a necessity. The creation of neonatal neurocritical care units, or Neuro-NICUs, provides strategies to reduce brain injury using standardized clinical protocols, methodologies, and provider education and training. Bedside neuromonitoring has dramatically improved our ability to provide assessment of newborns at high risk. Non-invasive tools, such as continuous electroencephalography (cEEG), amplitude-integrated electroencephalography (aEEG), and near-infrared spectroscopy (NIRS), allow screening for seizures and continuous evaluation of brain function and cerebral oxygenation at the bedside. Extended and combined uses of these techniques, also described as multimodal monitoring, may allow practitioners to better understand the physiology of critically ill neonates. Furthermore, the rapid growth of technology in the Neuro-NICU, along with the increasing use of telemedicine and artificial intelligence with improved data mining techniques and machine learning (ML), has the potential to vastly improve decision-making processes and positively impact outcomes. This article will cover the current applications of neuromonitoring in the Neuro-NICU, recent advances, potential pitfalls, and future perspectives in this field.

Multivariable Predictive Models of Death or Neurodevelopmental Impairment Among Extremely Low Birth Weight Infants Using Heart Rate Characteristics

OBJECTIVE

We hypothesized that a cumulative heart rate characteristics (HRC) index in real-time throughout the neonatal intensive care unit (NICU) hospitalization, alone or combined with birth demographics and clinical characteristics, can predict a composite outcome of death or neurodevelopmental impairment (NDI).

STUDY DESIGN

We performed a retrospective analysis using data from extremely low birth weight infants who were monitored for HRC during neonatal intensive care. Surviving infants were assessed for NDI at 18-22 months of age. Multivariable predictive modeling of subsequent death or NDI using logistic regression, cross-validation with repeats, and step-wise feature elimination was performed each postnatal day through day 60.

RESULTS

Among the 598 study participants, infants with the composite outcome of death or moderate-to-severe NDI had higher mean HRC scores during their stay in the NICU (3.1 ± 1.8 vs 1.3 ± 0.8; P < .001). Predictive models for subsequent death or NDI were consistently higher when the cumulative mean HRC score was included as a predictor variable. A parsimonious model including birth weight, sex, ventilatory status, and cumulative mean HRC score had a cross-validated receiver-operator characteristic curve as high as 0.84 on days 4, 5, 6, and 8 and as low as 0.78 on days 50-52 and 56-58 to predict subsequent death or NDI.

CONCLUSIONS

In extremely low birth weight infants, higher mean HRC scores throughout their stay in the NICU were associated with a higher risk of the composite outcome of death or NDI.

TRIAL REGISTRATION

ClinicalTrials.gov: NCT00307333.

Patchless Multi-Stage Transfer Learning for Improved Mammographic Breast Mass Classification

Simple Summary

In this study, we propose a novel deep-learning method based on multi-stage transfer learning (MSTL) from ImageNet and cancer cell line image pre-trained models to classify mammographic masses as either benign or malignant. The proposed method alleviates the challenge of obtaining large amounts of labeled mammogram training data by utilizing a large number of cancer cell line microscopic images as an intermediate domain of learning between the natural domain (ImageNet) and medical domain (mammography). Moreover, our method does not utilize patch separation (to segment the region of interest before classification), which renders it computationally simple and fast compared to previous studies. The findings of this study are of crucial importance in the early diagnosis of breast cancer in young women with dense breasts because mammography does not provide reliable diagnosis in such cases.

Abstract

Despite great achievements in classifying mammographic breast-mass images via deep-learning (DL), obtaining large amounts of training data and ensuring generalizations across different datasets with robust and well-optimized algorithms remain a challenge. ImageNet-based transfer learning (TL) and patch classifiers have been utilized to address these challenges. However, researchers have been unable to achieve the desired performance for DL to be used as a standalone tool. In this study, we propose a novel multi-stage TL from ImageNet and cancer cell line image pre-trained models to classify mammographic breast masses as either benign or malignant. We trained our model on three public datasets: Digital Database for Screening Mammography (DDSM), INbreast, and Mammographic Image Analysis Society (MIAS). In addition, a mixed dataset of the images from these three datasets was used to train the model. We obtained an average five-fold cross validation AUC of 1, 0.9994, 0.9993, and 0.9998 for DDSM, INbreast, MIAS, and mixed datasets, respectively. Moreover, the observed performance improvement using our method against the patch-based method was statistically significant, with a p-value of 0.0029. Furthermore, our patchless approach performed better than patch- and whole image-based methods, improving test accuracy by 8% (91.41% vs. 99.34%), tested on the INbreast dataset. The proposed method is of significant importance in solving the need for a large training dataset as well as reducing the computational burden in training and implementing the mammography-based deep-learning models for early diagnosis of breast cancer.

Transfer learning for non-image data in clinical research: A scoping review

BACKGROUND

Transfer learning is a form of machine learning where a pre-trained model trained on a specific task is reused as a starting point and tailored to another task in a different dataset. While transfer learning has garnered considerable attention in medical image analysis, its use for clinical non-image data is not well studied. Therefore, the objective of this scoping review was to explore the use of transfer learning for non-image data in the clinical literature.

METHODS AND FINDINGS

We systematically searched medical databases (PubMed, EMBASE, CINAHL) for peer-reviewed clinical studies that used transfer learning on human non-image data. We included 83 studies in the review. More than half of the studies (63%) were published within 12 months of the search. Transfer learning was most often applied to time series data (61%), followed by tabular data (18%), audio (12%) and text (8%). Thirty-three (40%) studies applied an image-based model to non-image data after transforming data into images (e.g. spectrograms). Twenty-nine (35%) studies did not have any authors with a health-related affiliation. Many studies used publicly available datasets (66%) and models (49%), but fewer shared their code (27%).

CONCLUSIONS

In this scoping review, we have described current trends in the use of transfer learning for non-image data in the clinical literature. We found that the use of transfer learning has grown rapidly within the last few years. We have identified studies and demonstrated the potential of transfer learning in clinical research in a wide range of medical specialties. More interdisciplinary collaborations and the wider adaption of reproducible research principles are needed to increase the impact of transfer learning in clinical research.

A Novel Multistage Transfer Learning for Ultrasound Breast Cancer Image Classification

Breast cancer diagnosis is one of the many areas that has taken advantage of artificial intelligence to achieve better performance, despite the fact that the availability of a large medical image dataset remains a challenge. Transfer learning (TL) is a phenomenon that enables deep learning algorithms to overcome the issue of shortage of training data in constructing an efficient model by transferring knowledge from a given source task to a target task. However, in most cases, ImageNet (natural images) pre-trained models that do not include medical images, are utilized for transfer learning to medical images. Considering the utilization of microscopic cancer cell line images that can be acquired in large amount, we argue that learning from both natural and medical datasets improves performance in ultrasound breast cancer image classification. The proposed multistage transfer learning (MSTL) algorithm was implemented using three pre-trained models: EfficientNetB2, InceptionV3, and ResNet50 with three optimizers: Adam, Adagrad, and stochastic gradient de-scent (SGD). Dataset sizes of 20,400 cancer cell images, 200 ultrasound images from Mendeley and 400 ultrasound images from the MT-Small-Dataset were used. ResNet50-Adagrad-based MSTL achieved a test accuracy of 99 ± 0.612% on the Mendeley dataset and 98.7 ± 1.1% on the MT-Small-Dataset, averaging over 5-fold cross validation. A p-value of 0.01191 was achieved when comparing MSTL against ImageNet based TL for the Mendeley dataset. The result is a significant improvement in the performance of artificial intelligence methods for ultrasound breast cancer classification compared to state-of-the-art methods and could remarkably improve the early diagnosis of breast cancer in young women.

Artificial intelligence, machine learning, and deep learning for clinical outcome prediction

AI is a broad concept, grouping initiatives that use a computer to perform tasks that would usually require a human to complete. AI methods are well suited to predict clinical outcomes. In practice, AI methods can be thought of as functions that learn the outcomes accompanying standardized input data to produce accurate outcome predictions when trialed with new data. Current methods for cleaning, creating, accessing, extracting, augmenting, and representing data for training AI clinical prediction models are well defined. The use of AI to predict clinical outcomes is a dynamic and rapidly evolving arena, with new methods and applications emerging. Extraction or accession of electronic health care records and combining these with patient genetic data is an area of present attention, with tremendous potential for future growth. Machine learning approaches, including decision tree methods of Random Forest and XGBoost, and deep learning techniques including deep multi-layer and recurrent neural networks, afford unique capabilities to accurately create predictions from high dimensional, multimodal data. Furthermore, AI methods are increasing our ability to accurately predict clinical outcomes that previously were difficult to model, including time-dependent and multi-class outcomes. Barriers to robust AI-based clinical outcome model deployment include changing AI product development interfaces, the specificity of regulation requirements, and limitations in ensuring model interpretability, generalizability, and adaptability over time.

Deep Multimodal Learning From MRI and Clinical Data for Early Prediction of Neurodevelopmental Deficits in Very Preterm Infants

The prevalence of disabled survivors of prematurity has increased dramatically in the past 3 decades. These survivors, especially, very preterm infants (VPIs), born ≤ 32 weeks gestational age, are at high risk for neurodevelopmental impairments. Early and clinically effective personalized prediction of outcomes, which forms the basis for early treatment decisions, is urgently needed during the peak neuroplasticity window—the first couple of years after birth—for at-risk infants, when intervention is likely to be most effective. Advances in MRI enable the noninvasive visualization of infants' brains through acquired multimodal images, which are more informative than unimodal MRI data by providing complementary/supplementary depicting of brain tissue characteristics and pathology. Thus, analyzing quantitative multimodal MRI features affords unique opportunities to study early postnatal brain development and neurodevelopmental outcome prediction in VPIs. In this study, we investigated the predictive power of multimodal MRI data, including T2-weighted anatomical MRI, diffusion tensor imaging, resting-state functional MRI, and clinical data for the prediction of neurodevelopmental deficits. We hypothesize that integrating multimodal MRI and clinical data improves the prediction over using each individual data modality. Employing the aforementioned multimodal data, we proposed novel end-to-end deep multimodal models to predict neurodevelopmental (i.e., cognitive, language, and motor) deficits independently at 2 years corrected age. We found that the proposed models can predict cognitive, language, and motor deficits at 2 years corrected age with an accuracy of 88.4, 87.2, and 86.7%, respectively, significantly better than using individual data modalities. This current study can be considered as proof-of-concept. A larger study with external validation is important to validate our approach to further assess its clinical utility and overall generalizability.

Neural Speech Encoding in Infancy Predicts Future Language and Communication Difficulties

Purpose This study aimed to construct an objective and cost-effective prognostic tool to forecast the future language and communication abilities of individual infants. Method Speech-evoked electroencephalography (EEG) data were collected from 118 infants during the first year of life during the exposure to speech stimuli that differed principally in fundamental frequency. Language and communication outcomes, namely four subtests of the MacArthur-Bates Communicative Development Inventories (MCDI)-Chinese version, were collected between 3 and 16 months after initial EEG testing. In the two-way classification, children were classified into those with future MCDI scores below the 25th percentile for their age group and those above the same percentile, while the three-way classification classified them into < 25th, 25th-75th, and > 75th percentile groups. Machine learning (support vector machine classification) with cross validation was used for model construction. Statistical significance was assessed. Results Across the four MCDI measures of early gestures, later gestures, vocabulary comprehension, and vocabulary production, the areas under the receiver-operating characteristic curve of the predictive models were respectively .92 ± .031, .91 ± .028, .90 ± .035, and .89 ± .039 for the two-way classification, and .88 ± .041, .89 ± .033, .85 ± .047, and .85 ± .050 for the three-way classification (p < .01 for all models). Conclusions Future language and communication variability can be predicted by an objective EEG method that indicates the function of the auditory neural pathway foundational to spoken language development, with precision sufficient for individual predictions. Longer-term research is needed to assess predictability of categorical diagnostic status. Supplemental Material https://doi.org/10.23641/asha.15138546.