Transfer learning for non-image data in clinical research: A scoping review

Opportunities and Challenges in Soft Tissue Sarcoma Risk Stratification in the Era of Personalised Medicine

OPINION STATEMENT

Soft tissue sarcomas (STS) are a rare and heterogeneous group of cancers. Treatment options have changed little in the past thirty years, and the role of neoadjuvant chemotherapy is controversial. Accurate risk stratification is crucial in STS in order to facilitate clinical discussions around peri-operative treatment. Current risk stratification tools used in clinic, such as Sarculator, use clinicopathological characteristics and may be specific to anatomical site or to histology. More recently, risk stratification tools have been developed using molecular or immunological data. Combining Sarculator with other risk stratification tools may identify novel patient groups with differential clinical outcomes. There are several considerations when translating risk stratification tools into widespread clinical use, including establishing clinical utility, health economic value, being applicable to existing clinical pathways, having strong real-world performance, and being supported by investment into infrastructure. Future work may include incorporation of novel modalities and data integration techniques.

Community-engaged artificial intelligence research: A scoping review

The degree to which artificial intelligence healthcare research is informed by data and stakeholders from community settings has not been previously described. As communities are the principal location of healthcare delivery, engaging them could represent an important opportunity to improve scientific quality. This scoping review systematically maps what is known and unknown about community-engaged artificial intelligence research and identifies opportunities to optimize the generalizability of these applications through involvement of community stakeholders and data throughout model development, validation, and implementation. Embase, PubMed, and MEDLINE databases were searched for articles describing artificial intelligence or machine learning healthcare applications with community involvement in model development, validation, or implementation. Model architecture and performance, the nature of community engagement, and barriers or facilitators to community engagement were reported according to PRISMA extension for Scoping Reviews guidelines. Of approximately 10,880 articles describing artificial intelligence healthcare applications, 21 (0.2%) described community involvement. All articles derived data from community settings, most commonly by leveraging existing datasets and sources that included community subjects, and often bolstered by internet-based data acquisition and subject recruitment. Only one article described inclusion of community stakeholders in designing an application-a natural language processing model that detected cases of likely child abuse with 90% accuracy using harmonized electronic health record notes from both hospital and community practice settings. The primary barrier to including community-derived data was small sample sizes, which may have affected 11 of the 21 studies (53%), introducing substantial risk for overfitting that threatens generalizability. Community engagement in artificial intelligence healthcare application development, validation, or implementation is rare. As healthcare delivery occurs primarily in community settings, investigators should consider engaging community stakeholders in user-centered design, usability, and clinical implementation studies to optimize generalizability.

Optimizing postprandial glucose prediction through integration of diet and exercise: Leveraging transfer learning with imbalanced patient data

BACKGROUND

In recent years, numerous methods have been introduced to predict glucose levels using machine-learning techniques on patients' daily behavioral and continuous glucose data. Nevertheless, a definitive consensus remains elusive regarding modeling the combined effects of diet and exercise for optimal glucose prediction. A notable challenge is the propensity for observational patient datasets from uncontrolled environments to overfit due to skewed feature distributions of target behaviors; for instance, diabetic patients seldom engage in high-intensity exercise post-meal.

METHODS

In this study, we introduce a unique application of Bayesian transfer learning for postprandial glucose prediction using randomized controlled trial (RCT) data. The data comprises a time series of three key variables: continuous glucose levels, exercise expenditure, and carbohydrate intake. For building the optimal model to predict postprandial glucose levels we initially gathered balanced training data from RCTs on healthy participants by randomizing behavioral conditions. Subsequently, we pretrained the model's parameter distribution using RCT data from the healthy cohort. This pretrained distribution was then adjusted, transferred, and utilized to determine the model parameters for each patient.

RESULTS

The efficacy of the proposed method was appraised using data from 68 gestational diabetes mellitus (GDM) patients in uncontrolled settings. The evaluation underscored the enhanced performance attained through our method. Furthermore, when modeling the joint impact of diet and exercise, the synergetic model proved more precise than its additive counterpart.

CONCLUSION

An innovative application of the transfer-learning utilizing randomized controlled trial data can improve the challenging modeling task of postprandial glucose prediction for GDM patients, integrating both dietary and exercise behaviors. For more accurate prediction, future research should focus on incorporating the long-term effects of exercise and other glycemic-related factors such as stress, sleep.

Optimizing clinico-genomic disease prediction across ancestries: a machine learning strategy with Pareto improvement

BACKGROUND

Accurate prediction of an individual's predisposition to diseases is vital for preventive medicine and early intervention. Various statistical and machine learning models have been developed for disease prediction using clinico-genomic data. However, the accuracy of clinico-genomic prediction of diseases may vary significantly across ancestry groups due to their unequal representation in clinical genomic datasets.

METHODS

We introduced a deep transfer learning approach to improve the performance of clinico-genomic prediction models for data-disadvantaged ancestry groups. We conducted machine learning experiments on multi-ancestral genomic datasets of lung cancer, prostate cancer, and Alzheimer's disease, as well as on synthetic datasets with built-in data inequality and distribution shifts across ancestry groups.

RESULTS

Deep transfer learning significantly improved disease prediction accuracy for data-disadvantaged populations in our multi-ancestral machine learning experiments. In contrast, transfer learning based on linear frameworks did not achieve comparable improvements for these data-disadvantaged populations.

CONCLUSIONS

This study shows that deep transfer learning can enhance fairness in multi-ancestral machine learning by improving prediction accuracy for data-disadvantaged populations without compromising prediction accuracy for other populations, thus providing a Pareto improvement towards equitable clinico-genomic prediction of diseases.

Revolutionizing Postoperative Ileus Monitoring: Exploring GRU-D's Real-Time Capabilities and Cross-Hospital Transferability

Background

Postoperative ileus (POI) after colorectal surgery leads to increased morbidity, costs, and hospital stays. Identifying POI risk for early intervention is important for improving surgical outcomes especially given the increasing trend towards early discharge after surgery. While existing studies have assessed POI risk with regression models, the role of deep learning's remains unexplored.

Methods

We assessed the performance and transferability (brutal force/instance/parameter transfer) of Gated Recurrent Unit with Decay (GRU-D), a longitudinal deep learning architecture, for real-time risk assessment of POI among 7,349 colorectal surgeries performed across three hospital sites operated by Mayo Clinic with two electronic health records (EHR) systems. The results were compared with atemporal models on a panel of benchmark metrics.

Results

GRU-D exhibits robust transferability across different EHR systems and hospital sites, showing enhanced performance by integrating new measurements, even amid the extreme sparsity of real-world longitudinal data. On average, for labs, vitals, and assisted living status, 72.2%, 26.9%, and 49.3% respectively lack measurements within 24 hours after surgery. Over the follow-up period with 4-hour intervals, 98.7%, 84%, and 95.8% of data points are missing, respectively. A maximum of 5% decrease in AUROC was observed in brutal-force transfer between different EHR systems with non-overlapping surgery date frames. Multi-source instance transfer witnessed the best performance, with a maximum of 2.6% improvement in AUROC over local learning. The significant benefit, however, lies in the reduction of variance (a maximum of 86% decrease). The GRU-D model's performance mainly depends on the prediction task's difficulty, especially the case prevalence rate. Whereas the impact of training data and transfer strategy is less crucial, underscoring the challenge of effectively leveraging transfer learning for rare outcomes. While atemporal Logit models show notably superior performance at certain pre-surgical points, their performance fluctuate significantly and generally underperform GRU-D in post-surgical hours.

Conclusion

GRU-D demonstrated robust transferability across EHR systems and hospital sites with highly sparse real-world EHR data. Further research on built-in explainability for meaningful intervention would be highly valuable for its integration into clinical practice.

MSDeepAMR: antimicrobial resistance prediction based on deep neural networks and transfer learning

Introduction

Antimicrobial resistance (AMR) is a global health problem that requires early and effective treatments to prevent the indiscriminate use of antimicrobial drugs and the outcome of infections. Mass Spectrometry (MS), and more particularly MALDI-TOF, have been widely adopted by routine clinical microbiology laboratories to identify bacterial species and detect AMR. The analysis of AMR with deep learning is still recent, and most models depend on filters and preprocessing techniques manually applied on spectra.

Methods

This study propose a deep neural network, MSDeepAMR, to learn from raw mass spectra to predict AMR. MSDeepAMR model was implemented for Escherichia coli, Klebsiella pneumoniae, and Staphylococcus aureus under different antibiotic resistance profiles. Additionally, a transfer learning test was performed to study the benefits of adapting the previously trained models to external data.

Results

MSDeepAMR models showed a good classification performance to detect antibiotic resistance. The AUROC of the model was above 0.83 in most cases studied, improving the results of previous investigations by over 10%. The adapted models improved the AUROC by up to 20% when compared to a model trained only with external data.

Discussion

This study demonstrate the potential of the MSDeepAMR model to predict antibiotic resistance and their use on external MS data. This allow the extrapolation of the MSDeepAMR model to de used in different laboratories that need to study AMR and do not have the capacity for an extensive sample collection.

An Interpretable Machine Learning Framework for Rare Disease: A Case Study to Stratify Infection Risk in Pediatric Leukemia

Background: Datasets on rare diseases, like pediatric acute myeloid leukemia (AML) and acute lymphoblastic leukemia (ALL), have small sample sizes that hinder machine learning (ML). The objective was to develop an interpretable ML framework to elucidate actionable insights from small tabular rare disease datasets. Methods: The comprehensive framework employed optimized data imputation and sampling, supervised and unsupervised learning, and literature-based discovery (LBD). The framework was deployed to assess treatment-related infection in pediatric AML and ALL. Results: An interpretable decision tree classified the risk of infection as either "high risk" or "low risk" in pediatric ALL (n = 580) and AML (n = 132) with accuracy of ∼79%. Interpretable regression models predicted the discrete number of developed infections with a mean absolute error (MAE) of 2.26 for bacterial infections and an MAE of 1.29 for viral infections. Features that best explained the development of infection were the chemotherapy regimen, cancer cells in the central nervous system at initial diagnosis, chemotherapy course, leukemia type, Down syndrome, race, and National Cancer Institute risk classification. Finally, SemNet 2.0, an open-source LBD software that links relationships from 33+ million PubMed articles, identified additional features for the prediction of infection, like glucose, iron, neutropenia-reducing growth factors, and systemic lupus erythematosus (SLE). Conclusions: The developed ML framework enabled state-of-the-art, interpretable predictions using rare disease tabular datasets. ML model performance baselines were successfully produced to predict infection in pediatric AML and ALL.

Prediction of stroke patients' bedroom-stay duration: machine-learning approach using wearable sensor data

Background: The importance of being physically active and avoiding staying in bed has been recognized in stroke rehabilitation. However, studies have pointed out that stroke patients admitted to rehabilitation units often spend most of their day immobile and inactive, with limited opportunities for activity outside their bedrooms. To address this issue, it is necessary to record the duration of stroke patients staying in their bedrooms, but it is impractical for medical providers to do this manually during their daily work of providing care. Although an automated approach using wearable devices and access points is more practical, implementing these access points into medical facilities is costly. However, when combined with machine learning, predicting the duration of stroke patients staying in their bedrooms is possible with reduced cost. We assessed using machine learning to estimate bedroom-stay duration using activity data recorded with wearable devices. Method: We recruited 99 stroke hemiparesis inpatients and conducted 343 measurements. Data on electrocardiograms and chest acceleration were measured using a wearable device, and the location name of the access point that detected the signal of the device was recorded. We first investigated the correlation between bedroom-stay duration measured from the access point as the objective variable and activity data measured with a wearable device and demographic information as explanatory variables. To evaluate the duration predictability, we then compared machine-learning models commonly used in medical studies. Results: We conducted 228 measurements that surpassed a 90% data-acquisition rate using Bluetooth Low Energy. Among the explanatory variables, the period spent reclining and sitting/standing were correlated with bedroom-stay duration (Spearman's rank correlation coefficient (R) of 0.56 and -0.52, p < 0.001). Interestingly, the sum of the motor and cognitive categories of the functional independence measure, clinical indicators of the abilities of stroke patients, lacked correlation. The correlation between the actual bedroom-stay duration and predicted one using machine-learning models resulted in an R of 0.72 and p < 0.001, suggesting the possibility of predicting bedroom-stay duration from activity data and demographics. Conclusion: Wearable devices, coupled with machine learning, can predict the duration of patients staying in their bedrooms. Once trained, the machine-learning model can predict without continuously tracking the actual location, enabling more cost-effective and privacy-centric future measurements.

Reducing Ophthalmic Health Disparities Through Transfer Learning: A Novel Application to Overcome Data Inequality

Purpose

Race disparities in the healthcare system and the resulting inequality in clinical data among different races hinder the ability to generate equitable prediction results. This study aims to reduce healthcare disparities arising from data imbalance by leveraging advanced transfer learning (TL) methods.

Method

We examined the ophthalmic healthcare disparities at a population level using electronic medical records data from a study cohort (N = 785) receiving care at an academic institute. Regression-based TL models were usesd, transferring valuable information from the dominant racial group (White) to improve visual field mean deviation (MD) rate of change prediction particularly for data-disadvantaged African American (AA) and Asian racial groups. Prediction results of TL models were compared with two conventional approaches.

Results

Disparities in socioeconomic status and baseline disease severity were observed among the AA and Asian racial groups. The TL approach achieved marked to comparable improvement in prediction accuracy compared to the two conventional approaches as evident by smaller mean absolute errors or mean square errors. TL identified distinct key features of visual field MD rate of change for each racial group.

Conclusions

The study introduces a novel application of TL that improved reliability of the analysis in comparison with conventional methods, especially in small sample size groups. This can improve assessment of healthcare disparity and subsequent remedy approach.

Translational Relevance

TL offers an equitable and efficient approach to mitigate healthcare disparities analysis by enhancing prediction performance for data-disadvantaged group.

Machine Learning Models for ASCVD Risk Prediction in an Asian Population - How to Validate the Model is Important

Introduction

Atherosclerotic cardiovascular disease (ASCVD) is prevalent worldwide including Taiwan, however widely accepted tools to assess the risk of ASCVD are lacking in Taiwan. Machine learning models are potentially useful for risk evaluation. In this study we used two cohorts to test the feasibility of machine learning with transfer learning for developing an ASCVD risk prediction model in Taiwan.

Methods

Two multi-center observational registry cohorts, T-SPARCLE and T-PPARCLE were used in this study. The variables selected were based on European, U.S. and Asian guidelines. Both registries recorded the ASCVD outcomes of the patients. Ten-fold validation and temporal validation methods were used to evaluate the performance of the binary classification analysis [prediction of major adverse cardiovascular (CV) events in one year]. Time-to-event analyses were also performed.

Results

In the binary classification analysis, eXtreme Gradient Boosting (XGBoost) and random forest had the best performance, with areas under the receiver operating characteristic curve (AUC-ROC) of 0.72 (0.68-0.76) and 0.73 (0.69-0.77), respectively, although it was not significantly better than other models. Temporal validation was also performed, and the data showed significant differences in the distribution of various features and event rate. The AUC-ROC of XGBoost dropped to 0.66 (0.59-0.73), while that of random forest dropped to 0.69 (0.62-0.76) in the temporal validation method, and the performance also became numerically worse than that of the logistic regression model. In the time-to-event analysis, most models had a concordance index of around 0.70.

Conclusions

Machine learning models with appropriate transfer learning may be a useful tool for the development of CV risk prediction models and may help improve patient care in the future.

Large Language Models for Epidemiological Research via Automated Machine Learning: Case Study Using Data From the British National Child Development Study

Background

Large language models have had a huge impact on natural language processing (NLP) in recent years. However, their application in epidemiological research is still limited to the analysis of electronic health records and social media data.

objectives

To demonstrate the potential of NLP beyond these domains, we aimed to develop prediction models based on texts collected from an epidemiological cohort and compare their performance to classical regression methods.

Methods

We used data from the British National Child Development Study, where 10,567 children aged 11 years wrote essays about how they imagined themselves as 25-year-olds. Overall, 15% of the data set was set aside as a test set for performance evaluation. Pretrained language models were fine-tuned using AutoTrain (Hugging Face) to predict current reading comprehension score (range: 0-35) and future BMI and physical activity (active vs inactive) at the age of 33 years. We then compared their predictive performance (accuracy or discrimination) with linear and logistic regression models, including demographic and lifestyle factors of the parents and children from birth to the age of 11 years as predictors.

Results

NLP clearly outperformed linear regression when predicting reading comprehension scores (root mean square error: 3.89, 95% CI 3.74-4.05 for NLP vs 4.14, 95% CI 3.98-4.30 and 5.41, 95% CI 5.23-5.58 for regression models with and without general ability score as a predictor, respectively). Predictive performance for physical activity was similarly poor for the 2 methods (area under the receiver operating characteristic curve: 0.55, 95% CI 0.52-0.60 for both) but was slightly better than random assignment, whereas linear regression clearly outperformed the NLP approach when predicting BMI (root mean square error: 4.38, 95% CI 4.02-4.74 for NLP vs 3.85, 95% CI 3.54-4.16 for regression). The NLP approach did not perform better than simply assigning the mean BMI from the training set as a predictor.

Conclusions

Our study demonstrated the potential of using large language models on text collected from epidemiological studies. The performance of the approach appeared to depend on how directly the topic of the text was related to the outcome. Open-ended questions specifically designed to capture certain health concepts and lived experiences in combination with NLP methods should receive more attention in future epidemiological studies.

Addressing the Challenge of Biomedical Data Inequality: An Artificial Intelligence Perspective

Artificial intelligence (AI) and other data-driven technologies hold great promise to transform healthcare and confer the predictive power essential to precision medicine. However, the existing biomedical data, which are a vital resource and foundation for developing medical AI models, do not reflect the diversity of the human population. The low representation in biomedical data has become a significant health risk for non-European populations, and the growing application of AI opens a new pathway for this health risk to manifest and amplify. Here we review the current status of biomedical data inequality and present a conceptual framework for understanding its impacts on machine learning. We also discuss the recent advances in algorithmic interventions for mitigating health disparities arising from biomedical data inequality. Finally, we briefly discuss the newly identified disparity in data quality among ethnic groups and its potential impacts on machine learning.

Bottom-up and top-down paradigms of artificial intelligence research approaches to healthcare data science using growing real-world big data

OBJECTIVES

As the real-world electronic health record (EHR) data continue to grow exponentially, novel methodologies involving artificial intelligence (AI) are becoming increasingly applied to enable efficient data-driven learning and, ultimately, to advance healthcare. Our objective is to provide readers with an understanding of evolving computational methods and help in deciding on methods to pursue.

TARGET AUDIENCE

The sheer diversity of existing methods presents a challenge for health scientists who are beginning to apply computational methods to their research. Therefore, this tutorial is aimed at scientists working with EHR data who are early entrants into the field of applying AI methodologies.

SCOPE

This manuscript describes the diverse and growing AI research approaches in healthcare data science and categorizes them into 2 distinct paradigms, the bottom-up and top-down paradigms to provide health scientists venturing into artificial intelligent research with an understanding of the evolving computational methods and help in deciding on methods to pursue through the lens of real-world healthcare data.

Pre-training in Medical Data: A Survey

Medical data refers to health-related information associated with regular patient care or as part of a clinical trial program. There are many categories of such data, such as clinical imaging data, bio-signal data, electronic health records (EHR), and multi-modality medical data. With the development of deep neural networks in the last decade, the emerging pre-training paradigm has become dominant in that it has significantly improved machine learning methods’ performance in a data-limited scenario. In recent years, studies of pre-training in the medical domain have achieved significant progress. To summarize these technology advancements, this work provides a comprehensive survey of recent advances for pre-training on several major types of medical data. In this survey, we summarize a large number of related publications and the existing benchmarking in the medical domain. Especially, the survey briefly describes how some pre-training methods are applied to or developed for medical data. From a data-driven perspective, we examine the extensive use of pre-training in many medical scenarios. Moreover, based on the summary of recent pre-training studies, we identify several challenges in this field to provide insights for future studies.

Neurosurgery inpatient outcome prediction for discharge planning with deep learning and transfer learning

INTRODUCTION

Deep learning may be able to assist with the prediction of neurosurgical inpatient outcomes. The aims of this study were to investigate deep learning and transfer learning in the prediction of several inpatient outcomes including timing of discharge and discharge destination.

METHOD

Data were collected on consecutive neurosurgical admissions from existing databases over a 15-month period. Following pre-processing artificial neural networks were applied to admission notes and ward round notes to predict four inpatient outcomes. Models were developed on the training dataset, before being tested on a hold-out test dataset and a validation dataset.

RESULTS

1341 individual admissions were included in the study. Using transfer learning and an artificial neural network an area under the receiver operator curve (AUC) of 0.81 and 0.80 on the derivation and validation datasets was able to be achieved for the prediction of discharge within the next 48 hours using daily ward round notes. This result is in comparison to an AUC of 0.71 and 0.68 using an artificial neural network without transfer learning for the same outcome. When the artificial neural network with transfer learning was applied to the other outcomes AUC of 0.72, 0.93 and 0.83 was achieved on the validation datasets for predicting discharge within the next 7 days, survival to discharge and discharge to home as a destination.

CONCLUSIONS

Deep learning may predict inpatient neurosurgery outcomes from free-text medical data. Recurrent predictions with ward round notes enable the use of information obtained throughout hospital admissions in these estimates.

Deep Transfer Learning Enables Robust Prediction of Antimicrobial Resistance for Novel Antibiotics

Antimicrobial resistance (AMR) has become one of the serious global health problems, threatening the effective treatment of a growing number of infections. Machine learning and deep learning show great potential in rapid and accurate AMR predictions. However, a large number of samples for the training of these models is essential. In particular, for novel antibiotics, limited training samples and data imbalance hinder the models’ generalization performance and overall accuracy. We propose a deep transfer learning model that can improve model performance for AMR prediction on small, imbalanced datasets. As our approach relies on transfer learning and secondary mutations, it is also applicable to novel antibiotics and emerging resistances in the future and enables quick diagnostics and personalized treatments.