A clinician's guide to understanding and critically appraising machine learning studies: a checklist for Ruling Out Bias Using Standard Tools in Machine Learning (ROBUST-ML)

Development and validation of AI-assisted transcriptomic signatures to personalize adjuvant chemotherapy in patients with pancreatic ductal adenocarcinoma

BACKGROUND

After surgical resection of pancreatic ductal adenocarcinoma (PDAC), patients are predominantly treated with adjuvant chemotherapy, commonly consisting of gemcitabine-based regimens or the modified FOLFIRINOX regimen (mFFX). While mFFX has been shown to be more effective than gemcitabine-based regimens, it is also associated with higher toxicity. Current treatment decisions are based on patient performance status rather than on the molecular characteristics of the tumor. To address this gap, the goal of this study was to develop drug-specific transcriptomic signatures for personalized chemotherapy treatment.

PATIENTS AND METHODS

We used PDAC datasets from preclinical models, encompassing chemotherapy response profiles for the mFFX-regimen components. From them we identified specific gene transcripts associated with chemotherapy response. Three transcriptomic AI-signatures were obtained by combining Independent Component Analysis, Least Absolute Shrinkage and the Selection Operator-Random Forest approach. We integrated a previously developed gemcitabine signature with three newly developed ones. The machine learning strategy employed to enhance these signatures incorporates transcriptomic features from the tumor microenvironment, leading to the development of the Pancreas-View tool ultimately clinically validated in a cohort of 343 patients from the PRODIGE-24/CCTG PA6 trial.

RESULTS

Patients who were predicted to be sensitive to the administered drugs (n=164; 47.8%) had longer disease-free survival (DFS) than the other patients. The median DFS in the mFFX sensitive group treated with mFFX was 50.0 months (stratified HR: 0.31; 95% CI, 0.21-0.44; p<0.001) and 33.7 months (stratified HR: 0.40; 95% CI, 0.17-0.59; p<0.001) in the gemcitabine sensitive group when treated with gemcitabine. Comparatively patients with signature predictions unmatched with the treatments (n=86; 25.1%) or those resistant to all drugs (n=93; 27.1%) had shorter DFS (10.6 and 10.8 months, respectively).

CONCLUSIONS

This study presents a transcriptome-based tool that was developed using preclinical models and machine learning to accurately predict sensitivity to mFFX and gemcitabine.

Use of Artificial Intelligence in Improving Outcomes in Heart Disease: A Scientific Statement From the American Heart Association

A major focus of academia, industry, and global governmental agencies is to develop and apply artificial intelligence and other advanced analytical tools to transform health care delivery. The American Heart Association supports the creation of tools and services that would further the science and practice of precision medicine by enabling more precise approaches to cardiovascular and stroke research, prevention, and care of individuals and populations. Nevertheless, several challenges exist, and few artificial intelligence tools have been shown to improve cardiovascular and stroke care sufficiently to be widely adopted. This scientific statement outlines the current state of the art on the use of artificial intelligence algorithms and data science in the diagnosis, classification, and treatment of cardiovascular disease. It also sets out to advance this mission, focusing on how digital tools and, in particular, artificial intelligence may provide clinical and mechanistic insights, address bias in clinical studies, and facilitate education and implementation science to improve cardiovascular and stroke outcomes. Last, a key objective of this scientific statement is to further the field by identifying best practices, gaps, and challenges for interested stakeholders.

A critical moment in machine learning in medicine: on reproducible and interpretable learning

Over the past two decades, advances in computational power and data availability combined with increased accessibility to pre-trained models have led to an exponential rise in machine learning (ML) publications. While ML may have the potential to transform healthcare, this sharp increase in ML research output without focus on methodological rigor and standard reporting guidelines has fueled a reproducibility crisis. In addition, the rapidly growing complexity of these models compromises their interpretability, which currently impedes their successful and widespread clinical adoption. In medicine, where failure of such models may have severe implications for patients' health, the high requirements for accuracy, robustness, and interpretability confront ML researchers with a unique set of challenges. In this review, we discuss the semantics of reproducibility and interpretability, as well as related issues and challenges, and outline possible solutions to counteracting the "black box". To foster reproducibility, standard reporting guidelines need to be further developed and data or code sharing encouraged. Editors and reviewers may equally play a critical role by establishing high methodological standards and thus preventing the dissemination of low-quality ML publications. To foster interpretable learning, the use of simpler models more suitable for medical data can inform the clinician how results are generated based on input data. Model-agnostic explanation tools, sensitivity analysis, and hidden layer representations constitute further promising approaches to increase interpretability. Balancing model performance and interpretability are important to ensure clinical applicability. We have now reached a critical moment for ML in medicine, where addressing these issues and implementing appropriate solutions will be vital for the future evolution of the field.

Improving cardiovascular risk prediction through machine learning modelling of irregularly repeated electronic health records

Aims

Existing electronic health records (EHRs) often consist of abundant but irregular longitudinal measurements of risk factors. In this study, we aim to leverage such data to improve the risk prediction of atherosclerotic cardiovascular disease (ASCVD) by applying machine learning (ML) algorithms, which can allow automatic screening of the population.

Methods and results

A total of 215 744 Chinese adults aged between 40 and 79 without a history of cardiovascular disease were included (6081 cases) from an EHR-based longitudinal cohort study. To allow interpretability of the model, the predictors of demographic characteristics, medication treatment, and repeatedly measured records of lipids, glycaemia, obesity, blood pressure, and renal function were used. The primary outcome was ASCVD, defined as non-fatal acute myocardial infarction, coronary heart disease death, or fatal and non-fatal stroke. The eXtreme Gradient boosting (XGBoost) algorithm and Least Absolute Shrinkage and Selection Operator (LASSO) regression models were derived to predict the 5-year ASCVD risk. In the validation set, compared with the refitted Chinese guideline-recommended Cox model (i.e. the China-PAR), the XGBoost model had a significantly higher C-statistic of 0.792, (the differences in the C-statistics: 0.011, 0.006-0.017, P < 0.001), with similar results reported for LASSO regression (the differences in the C-statistics: 0.008, 0.005-0.011, P < 0.001). The XGBoost model demonstrated the best calibration performance (men: Dx = 0.598, P = 0.75; women: Dx = 1.867, P = 0.08). Moreover, the risk distribution of the ML algorithms differed from that of the conventional model. The net reclassification improvement rates of XGBoost and LASSO over the Cox model were 3.9% (1.4-6.4%) and 2.8% (0.7-4.9%), respectively.

Conclusion

Machine learning algorithms with irregular, repeated real-world data could improve cardiovascular risk prediction. They demonstrated significantly better performance for reclassification to identify the high-risk population correctly.

Toward fairness in artificial intelligence for medical image analysis: identification and mitigation of potential biases in the roadmap from data collection to model deployment

Purpose

To recognize and address various sources of bias essential for algorithmic fairness and trustworthiness and to contribute to a just and equitable deployment of AI in medical imaging, there is an increasing interest in developing medical imaging-based machine learning methods, also known as medical imaging artificial intelligence (AI), for the detection, diagnosis, prognosis, and risk assessment of disease with the goal of clinical implementation. These tools are intended to help improve traditional human decision-making in medical imaging. However, biases introduced in the steps toward clinical deployment may impede their intended function, potentially exacerbating inequities. Specifically, medical imaging AI can propagate or amplify biases introduced in the many steps from model inception to deployment, resulting in a systematic difference in the treatment of different groups.

Approach

Our multi-institutional team included medical physicists, medical imaging artificial intelligence/machine learning (AI/ML) researchers, experts in AI/ML bias, statisticians, physicians, and scientists from regulatory bodies. We identified sources of bias in AI/ML, mitigation strategies for these biases, and developed recommendations for best practices in medical imaging AI/ML development.

Results

Five main steps along the roadmap of medical imaging AI/ML were identified: (1) data collection, (2) data preparation and annotation, (3) model development, (4) model evaluation, and (5) model deployment. Within these steps, or bias categories, we identified 29 sources of potential bias, many of which can impact multiple steps, as well as mitigation strategies.

Conclusions

Our findings provide a valuable resource to researchers, clinicians, and the public at large.

Artificial intelligence in detecting dentomaxillofacial fractures in diagnostic imaging: a scoping review protocol

INTRODUCTION

The dentomaxillofacial (DMF) area, which includes the teeth, maxilla, mandible, zygomaticum, orbits and midface, plays a crucial role in the maintenance of the physiological functions despite its susceptibility to fractures, which are mostly caused by mechanical trauma. As a diagnostic tool, radiographic imaging helps clinicians establish a diagnosis and determine a treatment plan; however, the presence of human factors in image interpretation can result in missed detection of fractures. Therefore, an artificial intelligence (AI) computing system with the potential to help detect abnormalities on radiographic images is currently being developed. This scoping review summarises the literature and assesses the current status of AI in DMF fracture detection in diagnostic imaging.

METHODS AND ANALYSIS

This proposed scoping review will be conducted using the framework of Arksey and O'Malley, with each step incorporating the recommendations of Levac et al. By using relevant keywords based on the research questions. PubMed, Science Direct, Scopus, Cochrane Library, Springerlink, Institute of Electrical and Electronics Engineers, and ProQuest will be the databases used in this study. The included studies are published in English between 1 January 2000 and 30 June 2023. Two independent reviewers will screen titles and abstracts, followed by full-text screening and data extraction, which will comprise three components: research study characteristics, comparator and AI characteristics.

ETHICS AND DISSEMINATION

This study does not require ethical approval because it analyses primary research articles. The research findings will be distributed through international conferences and peer-reviewed publications.

Machine learning for ECG diagnosis and risk stratification of occlusion myocardial infarction

Patients with occlusion myocardial infarction (OMI) and no ST-elevation on presenting electrocardiogram (ECG) are increasing in numbers. These patients have a poor prognosis and would benefit from immediate reperfusion therapy, but, currently, there are no accurate tools to identify them during initial triage. Here we report, to our knowledge, the first observational cohort study to develop machine learning models for the ECG diagnosis of OMI. Using 7,313 consecutive patients from multiple clinical sites, we derived and externally validated an intelligent model that outperformed practicing clinicians and other widely used commercial interpretation systems, substantially boosting both precision and sensitivity. Our derived OMI risk score provided enhanced rule-in and rule-out accuracy relevant to routine care, and, when combined with the clinical judgment of trained emergency personnel, it helped correctly reclassify one in three patients with chest pain. ECG features driving our models were validated by clinical experts, providing plausible mechanistic links to myocardial injury.

Big Data in Oncology Nursing Research: State of the Science

OBJECTIVE

To review the state of oncology nursing science as it pertains to big data. The authors aim to define and characterize big data, describe key considerations for accessing and analyzing big data, provide examples of analyses of big data in oncology nursing science, and highlight ethical considerations related to the collection and analysis of big data.

DATA SOURCES

Peer-reviewed articles published by investigators specializing in oncology, nursing, and related disciplines.

CONCLUSION

Big data is defined as data that are high in volume, velocity, and variety. To date, oncology nurse scientists have used big data to predict patient outcomes from clinician notes, identify distinct symptom phenotypes, and identify predictors of chemotherapy toxicity, among other applications. Although the emergence of big data and advances in computational methods provide new and exciting opportunities to advance oncology nursing science, several challenges are associated with accessing and using big data. Data security, research participant privacy, and the underrepresentation of minoritized individuals in big data are important concerns.

IMPLICATIONS FOR NURSING PRACTICE

With their unique focus on the interplay between the whole person, the environment, and health, nurses bring an indispensable perspective to the interpretation and application of big data research findings. Given the increasing ubiquity of passive data collection, all nurses should be taught the definition, characteristics, applications, and limitations of big data. Nurses who are trained in big data and advanced computational methods will be poised to contribute to guidelines and policies that preserve the rights of human research participants.

Machine Learning for the ECG Diagnosis and Risk Stratification of Occlusion Myocardial Infarction at First Medical Contact

Patients with occlusion myocardial infarction (OMI) and no ST-elevation on presenting ECG are increasing in numbers. These patients have a poor prognosis and would benefit from immediate reperfusion therapy, but we currently have no accurate tools to identify them during initial triage. Herein, we report the first observational cohort study to develop machine learning models for the ECG diagnosis of OMI. Using 7,313 consecutive patients from multiple clinical sites, we derived and externally validated an intelligent model that outperformed practicing clinicians and other widely used commercial interpretation systems, significantly boosting both precision and sensitivity. Our derived OMI risk score provided superior rule-in and rule-out accuracy compared to routine care, and when combined with the clinical judgment of trained emergency personnel, this score helped correctly reclassify one in three patients with chest pain. ECG features driving our models were validated by clinical experts, providing plausible mechanistic links to myocardial injury.

Continuous ECG monitoring should be the heart of bedside AI-based predictive analytics monitoring for early detection of clinical deterioration

The idea that we can detect subacute potentially catastrophic illness earlier by using statistical models trained on clinical data is now well-established. We review evidence that supports the role of continuous cardiorespiratory monitoring in these predictive analytics monitoring tools. In particular, we review how continuous ECG monitoring reflects the patient and not the clinician, is less likely to be biased, is unaffected by changes in practice patterns, captures signatures of illnesses that are interpretable by clinicians, and is an underappreciated and underutilized source of detailed information for new mathematical methods to reveal.

Emerging ECG methods for acute coronary syndrome detection: Recommendations & future opportunities

Despite being the mainstay for the initial noninvasive assessment of patients with symptomatic coronary artery disease, the 12‑lead ECG remains a suboptimal diagnostic tool for myocardial ischemia detection with only acceptable sensitivity and specificity scores. Although myocardial ischemia affects the configuration of the QRS complex and the STT waveform, current guidelines primarily focus on ST segment amplitude, which constitutes a missed opportunity and may explain the suboptimal diagnostic performance of the ECG. This possible opportunity and the low cost and ease of use of the ECG provide compelling motivation to enhance the diagnostic accuracy of the ECG to ischemia detection. This paper describes numerous computational ECG methods and approaches that have been shown to dramatically increase ECG sensitivity to ischemia detection. Briefly, these emerging approaches can be conceptually grouped into one of the following four approaches: (1) leveraging novel ECG waveform features and signatures indicative of ischemic injury other than the classical ST-T amplitude measures; (2) applying body surface potentials mapping (BSPM)-based approaches to enhance the spatial coverage of the surface ECG to detecting ischemia; (3) developing an inverse ECG solution to reconstruct anatomical models of activation and recovery pathways to detect and localize injury currents; and (4) exploring artificial intelligence (AI)-based techniques to harvest ECG waveform signatures of ischemia. We present recent advances, shortcomings, and future opportunities for each of these emerging ECG methods. Future research should focus on the prospective clinical testing of these approaches to establish clinical utility and to expedite potential translation into clinical practice.