Disadvantages of using the area under the receiver operating characteristic curve to assess imaging tests: a discussion and proposal for an alternative approach

Clinical Breast MRI-based Radiomics for Distinguishing Benign and Malignant Lesions: An Analysis of Sequences and Enhanced Phases

BACKGROUND

Previous studies have used different imaging sequences and different enhanced phases for breast lesion calsification in radiomics. The optimal sequence and contrast enhanced phase is unclear.

PURPOSE

To identify the optimal magnetic resonance imaging (MRI) radiomics model for lesion clarification, and to simulate its incremental value for multiparametric MRI (mpMRI)-guided biopsy.

STUDY TYPE

Retrospective.

POPULATION

329 female patients (138 malignant, 191 benign), divided into a training set (first site, n = 192) and an independent test set (second site, n = 137).

FIELD STRENGTH/SEQUENCE

3.0-T, fast spoiled gradient-echo and fast spin-echo T1-weighted imaging (T1WI), fast spin-echo T2-weighted imaging (T2WI), echo-planar diffusion-weighted imaging (DWI), and fast spoiled gradient-echo contrast-enhanced MRI (CE-MRI).

ASSESSMENT

Two breast radiologists with 3 and 10 years' experience developed radiomics model on CE-MRI, CE-MRI + DWI, CE-MRI + DWI + T2WI, CE-MRI + DWI + T2WI + T1WI at each individual phase (P) and for multiple combinations of phases. The optimal radiomics model (Rad-score) was identified as having the highest area under the receiver operating characteristic curve (AUC) in the test set. Specificity was compared between a traditional mpMRI model and an integrated model (mpMRI + Rad-score) at sensitivity >98%.

STATISTICAL TESTS

Wilcoxon paired-samples signed rank test, Delong test, McNemar test. Significance level was 0.05 and Bonferroni method was used for multiple comparisons (P = 0.007, 0.05/7).

RESULTS

For radiomics models, CE-MRI/P3 + DWI + T2WI achieved the highest performance in the test set (AUC = 0.888, 95% confidence interval: 0.833-0.944). The integrated model had significantly higher specificity (55.3%) than the mpMRI model (31.6%) in the test set with a sensitivity of 98.4%.

DATA CONCLUSION

The CE-MRI/P3 + DWI + T2WI model is the optimized choice for breast lesion classification in radiomics, and has potential to reduce benign biopsies (100%-specificity) from 68.4% to 44.7% while retaining sensitivity >98%.

LEVEL OF EVIDENCE

3 TECHNICAL EFFICACY: Stage 2.

A novel framework for enhancing transparency in credit scoring: Leveraging Shapley values for interpretable credit scorecards

Credit scorecards are essential tools for banks to assess the creditworthiness of loan applicants. While advanced machine learning models like XGBoost and random forest often outperform traditional logistic regression in predictive accuracy, their lack of interpretability hinders their adoption in practice. This study bridges the gap between research and practice by developing a novel framework for constructing interpretable credit scorecards using Shapley values. We apply this framework to two credit datasets, discretizing numerical variables and utilizing one-hot encoding to facilitate model development. Shapley values are then employed to derive credit scores for each predictor variable group in XGBoost, random forest, LightGBM, and CatBoost models. Our results demonstrate that this approach yields credit scorecards with interpretability comparable to logistic regression while maintaining superior predictive accuracy. This framework offers a practical and effective solution for credit practitioners seeking to leverage the power of advanced models without sacrificing transparency and regulatory compliance.

Age-appropriate BMI cut-offs for malnutrition among older adults in India

With the increasing prevalence of obesity in India, body mass index (BMI) has garnered importance as a disease predictor. The current World Health Organization (WHO) body mass index (BMI) cut-offs may not accurately portray these health risks in older adults aged 60 years and above. This study aims to define age-appropriate cut-offs for older adults (60-74 years and 75 years and above) and compare the performance of these cut-offs with the WHO BMI cut-offs using cardio-metabolic conditions as outcomes. Using baseline data from the Longitudinal Ageing Study in India (LASI), classification and regression tree (CART) cross-sectional analysis was conducted to obtain age-appropriate BMI cut-offs based on cardio-metabolic conditions as outcomes. Logistic regression models were estimated to compare the association of the two sets of cut-offs with cardio-metabolic outcomes. The area under the receiver operating characteristic curve (AUC), sensitivity and specificity were estimated. Agreement with waist circumference, an alternate measure of adiposity, was conducted. For older adults aged 60-74 years and 75 years and above, the cut-off for underweight reduced from < 18.5 to < 17.4 and < 13.3 respectively. The thresholds for overweight and obese increased for older adults aged 60-74 years old from >  = 25 to > 28.8 and >  = 30 to > 33.7 respectively. For older adults aged 75 years and above, the thresholds decreased for both categories. The largest improvement in AUC was observed in older adults aged 75 years and above. The newly derived cut-offs also demonstrated higher sensitivity and specificity among all age-sex stratifications. There is a need to adopt greater rigidity in defining overweight/obesity among older adults aged 75 years and above, as opposed to older adults aged 60-74 years old among whom the thresholds need to be less conservative. Further stratification in the low risk category could also improve BMI classification among older adults. These age-specific thresholds may act as improved alternatives of the current WHO BMI thresholds and improve classification among older adults in India.

Defining the Minimal Important Change and Meaningful Change Value of the Disease Activity Index for Psoriatic Arthritis: A Chinese Longitudinal Study

OBJECTIVE

To determine the minimal important change (MIC) and meaningful change value (MCV) of the Disease Activity Index for Psoriatic Arthritis (DAPSA) and the effect size (ES) of DAPSA.

METHODS

This was a retrospective cohort study, recruiting 106 patients who agreed to participate in the research from the Department of Dermatology, Xiangya Hospital, between November 1, 2019, and April 1, 2023. An anchor-based method using linear regression analyses was used to determine the MICs and MCVs of the DAPSA. The anchor question assessed whether the patient's well-being had changed since their previous visit, employing a 5-point Likert scale that ranged from "much improved" to "much deteriorated."

RESULTS

The overall MIC value was 8.4 (95% CI 0.01-16.75). The MIC improvement was 9.5 (95% CI 0.89-18.14) and MIC deterioration was 1.1 (95% CI -9.81 to 12.05). The overall MCV was 10.5 (95% CI 4.34-16.72). MCV improvement was 11.4 (95% CI 5.95-16.95) and MCV deterioration was 1.1 (95% CI -9.81 to 12.05). The ES was 0.6.

CONCLUSION

A change in DAPSA of 8.4 is indicative of an MIC, offering physicians an additional means to contextualize the patient's perception of disease activity during treatment, and a change in DAPSA of 10.5 is likely to be regarded as MCV. These values can enhance the utility of DAPSA in psoriatic arthritis clinical trials.

Accuracy of frailty instruments in predicting outcomes following perioperative cardiac arrest

BACKGROUND

Frailty is associated with increased 30-day mortality and non-home discharge following perioperative cardiac arrest. We estimated the predictive accuracy of frailty when added to baseline risk prediction models.

METHODS

In this retrospective cohort study using 2015-2020 NSQIP data for 3048 patients aged 50+ undergoing non-cardiac surgery and resuscitation on post-operative day 0 (i.e., intraoperatively or postoperatively on the day of surgery), baseline models including age, sex, ASA physical status, preoperative sepsis or septic shock, and emergent surgery were compared to models that added frailty indices, either RAI or mFI-5, to predict 30-day mortality and non-home discharge. Predictive accuracy was characterized by area under the receiver operating characteristic curve (AUC-ROC), integrated calibration index (ICI), and continuous net reclassification index (NRI).

RESULTS

1786 patients (58.6%) died in the study cohort within 30 days, and 38.6% of eligible patients experienced non-home discharge. The baseline model showed good discrimination (AUC-ROC 0.77 for 30-day mortality and 0.74 for non-home discharge). AUC-ROC and ICI did not significantly change after adding frailty for 30-day mortality or non-home discharge. Adding RAI significantly improved NRI for 30-day mortality and non-home discharge; however, the magnitude was small and difficult to interpret, given other results including false positive and negative rates showing no difference in predictive accuracy.

CONCLUSIONS

Incorporating frailty did not significantly improve predictive accuracy of models for 30-day mortality and non-home discharge following perioperative resuscitation. Thus, demonstrated associations between frailty and outcomes of perioperative resuscitation may not translate into improved predictive accuracy. When engaging patients in shared decision-making regarding do-not-resuscitate orders perioperatively, providers should acknowledge uncertainty in anticipating resuscitation outcomes.

Doug Altman, medical statistician par excellence: What can radiologists learn from his legacy?

This narrative review describes our experience of working with Doug Altman, the most highly cited medical statistician in the world. Doug was particularly interested in diagnostics, and imaging studies in particular. We describe how his insights helped improve our own radiological research studies and we provide advice for other researchers hoping to improve their own research practice.

[Artificial intelligence-enhanced electrocardiography : Will it revolutionize diagnosis and management of our patients?]

The use of artificial intelligence (AI) in healthcare has made significant progress in the last 10 years. Many experts believe that utilization of AI technologies, especially deep learning, will bring about drastic changes in how physicians understand, diagnose, and treat diseases. One aspect of this development is AI-enhanced electrocardiography (ECG) analysis. It involves not only optimizing the traditional ECG analysis by the physician and improving the accuracy of automatic interpretation by the ECG device but also introducing entirely new diagnostic options enabled by AI. Examples include assessing left ventricular function, predicting atrial fibrillation, and diagnosing both cardiac and noncardiac conditions. Through AI, the ECG becomes a comprehensive tool for screening, diagnosis, and patient management, potentially revolutionizing clinical practices. This paper provides an overview of the current state of this development, discusses existing limitations, and explores the challenges that may arise for healthcare professionals in this context.

Enhancing credit scoring accuracy with a comprehensive evaluation of alternative data

This study explores the potential of utilizing alternative data sources to enhance the accuracy of credit scoring models, compared to relying solely on traditional data sources, such as credit bureau data. A comprehensive dataset from the Home Credit Group's home loan portfolio is analysed. The research examines the impact of incorporating alternative predictors that are typically overlooked, such as an applicant's social network default status, regional economic ratings, and local population characteristics. The modelling approach applies the model-X knockoffs framework for systematic variable selection. By including these alternative data sources, the credit scoring models demonstrate improved predictive performance, achieving an area under the curve metric of 0.79360 on the Kaggle Home Credit default risk competition dataset, outperforming models that relied solely on traditional data sources, such as credit bureau data. The findings highlight the significance of leveraging diverse, non-traditional data sources to augment credit risk assessment capabilities and overall model accuracy.

Current perspectives on the use of artificial intelligence in critical patient safety

Intensive Care Units (ICUs) have undergone enhancements in patient safety, and artificial intelligence (AI) emerges as a disruptive technology offering novel opportunities. While the published evidence is limited and presents methodological issues, certain areas show promise, such as decision support systems, detection of adverse events, and prescription error identification. The application of AI in safety may pursue predictive or diagnostic objectives. Implementing AI-based systems necessitates procedures to ensure secure assistance, addressing challenges including trust in such systems, biases, data quality, scalability, and ethical and confidentiality considerations. The development and application of AI demand thorough testing, encompassing retrospective data assessments, real-time validation with prospective cohorts, and efficacy demonstration in clinical trials. Algorithmic transparency and explainability are essential, with active involvement of clinical professionals being crucial in the implementation process.

Machine learning on cardiotocography data to classify fetal outcomes: A scoping review

INTRODUCTION

Uterine contractions during labour constrict maternal blood flow and oxygen delivery to the developing baby, causing transient hypoxia. While most babies are physiologically adapted to withstand such intrapartum hypoxia, those exposed to severe hypoxia or with poor physiological reserves may experience neurological injury or death during labour. Cardiotocography (CTG) monitoring was developed to identify babies at risk of hypoxia by detecting changes in fetal heart rate (FHR) patterns. CTG monitoring is in widespread use in intrapartum care for the detection of fetal hypoxia, but the clinical utility is limited by a relatively poor positive predictive value (PPV) of an abnormal CTG and significant inter and intra observer variability in CTG interpretation. Clinical risk and human factors may impact the quality of CTG interpretation. Misclassification of CTG traces may lead to both under-treatment (with the risk of fetal injury or death) or over-treatment (which may include unnecessary operative interventions that put both mother and baby at risk of complications). Machine learning (ML) has been applied to this problem since early 2000 and has shown potential to predict fetal hypoxia more accurately than visual interpretation of CTG alone. To consider how these tools might be translated for clinical practice, we conducted a review of ML techniques already applied to CTG classification and identified research gaps requiring investigation in order to progress towards clinical implementation.

MATERIALS AND METHOD

We used identified keywords to search databases for relevant publications on PubMed, EMBASE and IEEE Xplore. We used Preferred Reporting Items for Systematic Review and Meta-Analysis for Scoping Reviews (PRISMA-ScR). Title, abstract and full text were screened according to the inclusion criteria.

RESULTS

We included 36 studies that used signal processing and ML techniques to classify CTG. Most studies used an open-access CTG database and predominantly used fetal metabolic acidosis as the benchmark for hypoxia with varying pH levels. Various methods were used to process and extract CTG signals and several ML algorithms were used to classify CTG. We identified significant concerns over the practicality of using varying pH levels as the CTG classification benchmark. Furthermore, studies needed to be more generalised as most used the same database with a low number of subjects for an ML study.

CONCLUSION

ML studies demonstrate potential in predicting fetal hypoxia from CTG. However, more diverse datasets, standardisation of hypoxia benchmarks and enhancement of algorithms and features are needed for future clinical implementation.

Medical Relevance, State-of-the-Art and Perspectives of "Sweet Metacode" in Liquid Biopsy Approaches

This review briefly introduces readers to an area where glycomics meets modern oncodiagnostics with a focus on the analysis of sialic acid (Neu5Ac)-terminated structures. We present the biochemical perspective of aberrant sialylation during tumourigenesis and its significance, as well as an analytical perspective on the detection of these structures using different approaches for diagnostic and therapeutic purposes. We also provide a comparison to other established liquid biopsy approaches, and we mathematically define an early-stage cancer based on the overall prognosis and effect of these approaches on the patient's quality of life. Finally, some barriers including regulations and quality of clinical validations data are discussed, and a perspective and major challenges in this area are summarised.

Artificial intelligence applied to magnetic resonance imaging reliably detects the presence, but not the location, of meniscus tears: a systematic review and meta-analysis

OBJECTIVES

To review and compare the accuracy of convolutional neural networks (CNN) for the diagnosis of meniscal tears in the current literature and analyze the decision-making processes utilized by these CNN algorithms.

MATERIALS AND METHODS

PubMed, MEDLINE, EMBASE, and Cochrane databases up to December 2022 were searched in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) statement. Risk of analysis was used for all identified articles. Predictive performance values, including sensitivity and specificity, were extracted for quantitative analysis. The meta-analysis was divided between AI prediction models identifying the presence of meniscus tears and the location of meniscus tears.

RESULTS

Eleven articles were included in the final review, with a total of 13,467 patients and 57,551 images. Heterogeneity was statistically significantly large for the sensitivity of the tear identification analysis (I2 = 79%). A higher level of accuracy was observed in identifying the presence of a meniscal tear over locating tears in specific regions of the meniscus (AUC, 0.939 vs 0.905). Pooled sensitivity and specificity were 0.87 (95% confidence interval (CI) 0.80-0.91) and 0.89 (95% CI 0.83-0.93) for meniscus tear identification and 0.88 (95% CI 0.82-0.91) and 0.84 (95% CI 0.81-0.85) for locating the tears.

CONCLUSIONS

AI prediction models achieved favorable performance in the diagnosis, but not location, of meniscus tears. Further studies on the clinical utilities of deep learning should include standardized reporting, external validation, and full reports of the predictive performances of these models, with a view to localizing tears more accurately.

CLINICAL RELEVANCE STATEMENT

Meniscus tears are hard to diagnose in the knee magnetic resonance images. AI prediction models may play an important role in improving the diagnostic accuracy of clinicians and radiologists.

KEY POINTS

• Artificial intelligence (AI) provides great potential in improving the diagnosis of meniscus tears. • The pooled diagnostic performance for artificial intelligence (AI) in identifying meniscus tears was better (sensitivity 87%, specificity 89%) than locating the tears (sensitivity 88%, specificity 84%). • AI is good at confirming the diagnosis of meniscus tears, but future work is required to guide the management of the disease.

Artificial Intelligence and Panendoscopy-Automatic Detection of Clinically Relevant Lesions in Multibrand Device-Assisted Enteroscopy

Device-assisted enteroscopy (DAE) is capable of evaluating the entire gastrointestinal tract, identifying multiple lesions. Nevertheless, DAE's diagnostic yield is suboptimal. Convolutional neural networks (CNN) are multi-layer architecture artificial intelligence models suitable for image analysis, but there is a lack of studies about their application in DAE. Our group aimed to develop a multidevice CNN for panendoscopic detection of clinically relevant lesions during DAE. In total, 338 exams performed in two specialized centers were retrospectively evaluated, with 152 single-balloon enteroscopies (Fujifilm®, Porto, Portugal), 172 double-balloon enteroscopies (Olympus®, Porto, Portugal) and 14 motorized spiral enteroscopies (Olympus®, Porto, Portugal); then, 40,655 images were divided in a training dataset (90% of the images, n = 36,599) and testing dataset (10% of the images, n = 4066) used to evaluate the model. The CNN's output was compared to an expert consensus classification. The model was evaluated by its sensitivity, specificity, positive (PPV) and negative predictive values (NPV), accuracy and area under the precision recall curve (AUC-PR). The CNN had an 88.9% sensitivity, 98.9% specificity, 95.8% PPV, 97.1% NPV, 96.8% accuracy and an AUC-PR of 0.97. Our group developed the first multidevice CNN for panendoscopic detection of clinically relevant lesions during DAE. The development of accurate deep learning models is of utmost importance for increasing the diagnostic yield of DAE-based panendoscopy.

The clinical performance and population health impact of birthweight-for-gestational age indices at term gestation

BACKGROUND

The assessment of birthweight for gestational age and the identification of small- and large-for-gestational age (SGA and LGA) infants remain contentious, despite the recent creation of the Intergrowth 21st Project and World Health Organisation (WHO) birthweight-for-gestational age standards.

OBJECTIVE

We carried out a study to identify birthweight-for-gestational age cut-offs, and corresponding population-based, Intergrowth 21st and WHO centiles associated with higher risks of adverse neonatal outcomes, and to evaluate their ability to predict serious neonatal morbidity and neonatal mortality (SNMM) at term gestation.

METHODS

The study population was based on non-anomalous, singleton live births between 37 and 41 weeks' gestation in the United States from 2003 to 2017. SNMM included 5-min Apgar score <4, neonatal seizures, need for assisted ventilation, and neonatal death. Birthweight-specific SNMM was modelled by gestational week using penalised B-splines. The birthweights at which SNMM odds were minimised (and higher by 10%, 50% and 100%) were estimated, and the corresponding population, Intergrowth 21st, and WHO centiles were identified. The clinical performance and population impact of these cut-offs for predicting SNMM were evaluated.

RESULTS

The study included 40,179,663 live births and 991,486 SNMM cases. Among female singletons at 39 weeks' gestation, SNMM odds was lowest at 3203 g birthweight, and 10% higher at 2835 g and 3685 g (population centiles 11th and 82nd, Intergrowth centiles 17th and 88th and WHO centiles 15th and 85th). Birthweight cut-offs were poor predictors of SNMM, for example, the cut-offs associated with 10% and 50% higher odds of SNMM among female singletons at 39 weeks' gestation resulted in a sensitivity, specificity, and population attributable fraction of 12.5%, 89.4%, and 2.1%, and 2.9%, 98.4% and 1.3%, respectively.

CONCLUSIONS

Reference- and standard-based birthweight-for-gestational age indices and centiles perform poorly for predicting adverse neonatal outcomes in individual infants, and their associated population impact is also small.

A real-world evaluation of the diagnostic accuracy of radiologists using positive predictive values verified from deep learning and natural language processing chest algorithms deployed retrospectively

Objectives

This diagnostic study assessed the accuracy of radiologists retrospectively, using the deep learning and natural language processing chest algorithms implemented in Clinical Review version 3.2 for: pneumothorax, rib fractures in digital chest X-ray radiographs (CXR); aortic aneurysm, pulmonary nodules, emphysema, and pulmonary embolism in CT images.

Methods

The study design was double-blind (artificial intelligence [AI] algorithms and humans), retrospective, non-interventional, and at a single NHS Trust. Adult patients (≥18 years old) scheduled for CXR and CT were invited to enroll as participants through an opt-out process. Reports and images were de-identified, processed retrospectively, and AI-flagged discrepant findings were assigned to two lead radiologists, each blinded to patient identifiers and original radiologist. The radiologist's findings for each clinical condition were tallied as a verified discrepancy (true positive) or not (false positive).

Results

The missed findings were: 0.02% rib fractures, 0.51% aortic aneurysm, 0.32% pulmonary nodules, 0.92% emphysema, and 0.28% pulmonary embolism. The positive predictive values (PPVs) were: pneumothorax (0%), rib fractures (5.6%), aortic dilatation (43.2%), pulmonary emphysema (46.0%), pulmonary embolus (11.5%), and pulmonary nodules (9.2%). The PPV for pneumothorax was nil owing to lack of available studies that were analysed for outpatient activity.

Conclusions

The number of missed findings was far less than generally predicted. The chest algorithms deployed retrospectively were a useful quality tool and AI augmented the radiologists' workflow.

Advances in knowledge

The diagnostic accuracy of our radiologists generated missed findings of 0.02% for rib fractures CXR, 0.51% for aortic dilatation, 0.32% for pulmonary nodule, 0.92% for pulmonary emphysema, and 0.28% for pulmonary embolism for CT studies, all retrospectively evaluated with AI used as a quality tool to flag potential missed findings. It is important to account for prevalence of these chest conditions in clinical context and use appropriate clinical thresholds for decision-making, not relying solely on AI.

Ethical Considerations for Artificial Intelligence in Medical Imaging: Deployment and Governance

The deployment of artificial intelligence (AI) has the potential to make nuclear medicine and medical imaging faster, cheaper, and both more effective and more accessible. This is possible, however, only if clinicians and patients feel that these AI medical devices (AIMDs) are trustworthy. Highlighting the need to ensure health justice by fairly distributing benefits and burdens while respecting individual patients' rights, the AI Task Force of the Society of Nuclear Medicine and Molecular Imaging has identified 4 major ethical risks that arise during the deployment of AIMD: autonomy of patients and clinicians, transparency of clinical performance and limitations, fairness toward marginalized populations, and accountability of physicians and developers. We provide preliminary recommendations for governing these ethical risks to realize the promise of AIMD for patients and populations.

Optimizing Clinical Decision Making with Decision Curve Analysis: Insights for Clinical Investigators

A large number of prediction models are published with the objective of allowing personalized decision making for diagnostic or prognostic purposes. Conventional statistical measures of discrimination, calibration, or other measures of model performance are not well-suited for directly and clearly assessing the clinical value of scores or biomarkers. Decision curve analysis is an increasingly popular technique used to assess the clinical utility of a prognostic or diagnostic score/rule, or even of a biomarker. Clinical utility is expressed as the net benefit, which represents the net balance of patients' benefits and harms and considers, implicitly, the consequences of clinical actions taken in response to a certain prediction score, rule, or biomarker. The net benefit is plotted against a range of possible exchange rates, representing the spectrum of possible patients' and clinicians' preferences. Decision curve analysis is a powerful tool for judging whether newly published or existing scores may truly benefit patients, and represents a significant advancement in improving transparent clinical decision making. This paper is meant to be an introduction to decision curve analysis and its interpretation for clinical investigators. Given the extensive advantages, we advocate applying decision curve analysis to all models intended for use in clinical practice.

Radiomics-based machine learning models for prediction of medulloblastoma subgroups: a systematic review and meta-analysis of the diagnostic test performance

BACKGROUND

Medulloblastomas are a major cause of cancer-related mortality in the pediatric population. Four molecular groups have been identified, and these molecular groups drive risk stratification, prognostic modeling, and the development of novel treatment modalities. It has been demonstrated that radiomics-based machine learning (ML) models are effective at predicting the diagnosis, molecular class, and grades of CNS tumors.

PURPOSE

To assess radiomics-based ML models' diagnostic performance in predicting medulloblastoma subgroups and the methodological quality of the studies.

MATERIAL AND METHODS

A comprehensive literature search was performed on PubMed; the last search was conducted on 1 May 2022. Studies that predicted all four medulloblastoma subgroups in patients with histopathologically confirmed medulloblastoma and reporting area under the curve (AUC) values were included in the study. The quality assessments were conducted according to the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) and Checklist for Artificial Intelligence in Medical Imaging (CLAIM). A meta-analysis of radiomics-based ML studies' diagnostic performance for the preoperative evaluation of medulloblastoma subgrouping was performed.

RESULTS

Five studies were included in this meta-analysis. Regarding patient selection, two studies indicated an unclear risk of bias according to the QUADAS-2. The five studies had an average CLAIM score and compliance score of 23.2 and 0.57, respectively. The meta-analysis showed pooled AUCs of 0.88, 0.82, 0.83, and 0.88 for WNT, SHH, group 3, and group 4 for classification, respectively.

CONCLUSION

Radiomics-based ML studies have good classification performance in predicting medulloblastoma subgroups, with AUCs >0.80 in every subgroup. To be applied to clinical practice, they need methodological quality improvement and stability.

Machine learning based prediction for oncologic outcomes of renal cell carcinoma after surgery using Korean Renal Cell Carcinoma (KORCC) database

We developed a novel prediction model for recurrence and survival in patients with localized renal cell carcinoma (RCC) after surgery and a novel statistical method of machine learning (ML) to improve accuracy in predicting outcomes using a large Asian nationwide dataset, updated KOrean Renal Cell Carcinoma (KORCC) database that covered data for a total of 10,068 patients who had received surgery for RCC. After data pre-processing, feature selection was performed with an elastic net. Nine variables for recurrence and 13 variables for survival were extracted from 206 variables. Synthetic minority oversampling technique (SMOTE) was used for the training data set to solve the imbalance problem. We applied the most of existing ML algorithms introduced so far to evaluate the performance. We also performed subgroup analysis according to the histologic type. Diagnostic performances of all prediction models achieved high accuracy (range, 0.77-0.94) and F1-score (range, 0.77-0.97) in all tested metrics. In an external validation set, high accuracy and F1-score were well maintained in both recurrence and survival. In subgroup analysis of both clear and non-clear cell type RCC group, we also found a good prediction performance.

Decision curve analysis confirms higher clinical utility of multi-domain versus single-domain prediction models in patients with open abdomen treatment for peritonitis

BACKGROUND

Prediction modelling increasingly becomes an important risk assessment tool in perioperative systems approaches, e.g. in complex patients with open abdomen treatment for peritonitis. In this population, combining predictors from multiple medical domains (i.e. demographical, physiological and surgical variables) outperforms the prediction capabilities of single-domain prediction models. However, the benefit of these prediction models for clinical decision-making remains to be investigated. We therefore examined the clinical utility of mortality prediction models in patients suffering from peritonitis with a decision curve analysis.

METHODS

In this secondary analysis of a large dataset, a traditional logistic regression approach, three machine learning methods and a stacked ensemble were employed to examine the predictive capability of demographic, physiological and surgical variables in predicting mortality under open abdomen treatment for peritonitis. Calibration was examined with calibration belts and predictive performance was assessed with the area both under the receiver operating characteristic curve (AUROC) and under the precision recall curve (AUPRC) and with the Brier Score. Clinical utility of the prediction models was examined by means of a decision curve analysis (DCA) within a treatment threshold range of interest of 0-30%, where threshold probabilities are traditionally defined as the minimum probability of disease at which further intervention would be warranted.

RESULTS

Machine learning methods supported available evidence of a higher prediction performance of a multi- versus single-domain prediction models. Interestingly, their prediction performance was similar to a logistic regression model. The DCA demonstrated that the overall net benefit is largest for a multi-domain prediction model and that this benefit is larger compared to the default "treat all" strategy only for treatment threshold probabilities above about 10%. Importantly, the net benefit for low threshold probabilities is dominated by physiological predictors: surgical and demographics predictors provide only secondary decision-analytic benefit.

CONCLUSIONS

DCA provides a valuable tool to compare single-domain and multi-domain prediction models and demonstrates overall higher decision-analytic value of the latter. Importantly, DCA provides a means to clinically differentiate the risks associated with each of these domains in more depth than with traditional performance metrics and highlighted the importance of physiological predictors for conservative intervention strategies for low treatment thresholds. Further, machine learning methods did not add significant benefit either in prediction performance or decision-analytic utility compared to logistic regression in these data.

The Matthews correlation coefficient (MCC) should replace the ROC AUC as the standard metric for assessing binary classification

Binary classification is a common task for which machine learning and computational statistics are used, and the area under the receiver operating characteristic curve (ROC AUC) has become the common standard metric to evaluate binary classifications in most scientific fields. The ROC curve has true positive rate (also called sensitivity or recall) on the y axis and false positive rate on the x axis, and the ROC AUC can range from 0 (worst result) to 1 (perfect result). The ROC AUC, however, has several flaws and drawbacks. This score is generated including predictions that obtained insufficient sensitivity and specificity, and moreover it does not say anything about positive predictive value (also known as precision) nor negative predictive value (NPV) obtained by the classifier, therefore potentially generating inflated overoptimistic results. Since it is common to include ROC AUC alone without precision and negative predictive value, a researcher might erroneously conclude that their classification was successful. Furthermore, a given point in the ROC space does not identify a single confusion matrix nor a group of matrices sharing the same MCC value. Indeed, a given (sensitivity, specificity) pair can cover a broad MCC range, which casts doubts on the reliability of ROC AUC as a performance measure. In contrast, the Matthews correlation coefficient (MCC) generates a high score in its [Formula: see text] interval only if the classifier scored a high value for all the four basic rates of the confusion matrix: sensitivity, specificity, precision, and negative predictive value. A high MCC (for example, MCC [Formula: see text] 0.9), moreover, always corresponds to a high ROC AUC, and not vice versa. In this short study, we explain why the Matthews correlation coefficient should replace the ROC AUC as standard statistic in all the scientific studies involving a binary classification, in all scientific fields.

Diagnostic Performance of Various Ultrasound Risk Stratification Systems for Benign and Malignant Thyroid Nodules: A Meta-Analysis

BACKGROUND

To evaluate the diagnostic performance of ultrasound risk-stratification systems for the discrimination of benign and malignant thyroid nodules and to determine the optimal cutoff values of individual risk-stratification systems.

METHODS

PubMed, Embase, SCOPUS, Web of Science, and Cochrane library databases were searched up to August 2022. Sensitivity and specificity data were collected along with the characteristics of each study related to ultrasound risk stratification systems.

RESULTS

Sixty-seven studies involving 76,512 thyroid nodules were included in this research. The sensitivity, specificity, diagnostic odds ratios, and area under the curves by K-TIRADS (4), ACR-TIRADS (TR5), ATA (high suspicion), EU-TIRADS (5), and Kwak-TIRADS (4b) for malignancy risk stratification of thyroid nodules were 92.5%, 63.5%, 69.8%, 70.6%, and 95.8%, respectively; 62.8%, 89.6%, 87.2%, 83.9%, and 63.8%, respectively; 20.7111, 16.8442, 15.7398, 12.2986, and 38.0578, respectively; and 0.792, 0.882, 0.859, 0.843, and 0.929, respectively.

CONCLUSION

All ultrasound-based risk-stratification systems had good diagnostic performance. Although this study determined the best cutoff values in individual risk-stratification systems based on statistical assessment, clinicians could adjust or alter cutoff values based on the clinical purpose of the ultrasound and the reciprocal changes in sensitivity and specificity.

Prognostic Value of CRASH and IMPACT Models for Predicting Mortality and Unfavorable Outcome in Traumatic Brain Injury; a Systematic Review and Meta-Analysis

Introduction

The Corticosteroid Randomization After Significant Head injury (CRASH) and the International Mission for Prognosis and Analysis of Clinical Trials (IMPACT) are two prognostic models frequently used in predicting the outcome of patients with traumatic brain injury. There are ongoing debates about which of the two models has a better prognostic value. This study aims to compare the CRASH and IMPACT in predicting mortality and unfavorable outcome of patients with traumatic brain injury.

Method

We performed a literature search using Medline (via PubMed), Embase, Scopus, and Web of Science databases until August 17, 2022. After two independent researchers screened the articles, we included all the original articles comparing the prognostic value of IMPACT and CRASH models in patients with traumatic brain injury. The outcomes evaluated were mortality and unfavorable outcome. The data of the included articles were analyzed using STATA 17.0 statistical program, and we reported an odds ratio (OR) with a 95% confidence interval (95% CI) for comparison.

Results

We included the data from 16 studies. The analysis showed that the areas under the curve of the IMPACT core model and CRASH basic model do not differ in predicting the mortality of patients (OR=0.99; p=0.905) and their six-month unfavorable outcome (OR=1.01; p=0.719). Additionally, the CRASH CT model showed no difference from the IMPACT extended (OR=0.98; p=0.507) and IMPACT Lab (OR=1.00; p=0.298) models in predicting the mortality of patients with traumatic brain injury. We also observed similar findings in the six-month unfavorable outcome, showing that the CRASH CT model does not differ from the IMPACT extended (OR=1.00; p=0.990) and IMPACT Lab (OR=1.00; p=0.570) in predicting the unfavorable outcome in head trauma patients.

Conclusion

Low to very low level of evidence shows that IMPACT and CRASH models have similar values in predicting mortality and unfavorable outcome in patients with traumatic brain injury. Since the discriminative power of the IMPACT Core and CRASH basic models is not different from the IMPACT extended, IMPACT Lab, and CRASH CT models, it may be possible to only use the core and basic models in examining the prognosis of patients with traumatic injuries to the brain.

Maternal characteristics associated with referral to obstetrician-led care in low-risk pregnant women in the Netherlands: A retrospective cohort study

BACKGROUND

In the Netherlands, maternity care is divided into midwife-led care (for low-risk women) and obstetrician-led care (for high-risk women). Referrals from midwife-led to obstetrician-led care have increased over the past decade. The majority of women are referred during their pregnancy or labour. Referrals are based on a continuous risk assessment of the health and characteristics of mother and child, yet referral for non-medical factors and characteristics remain unclear. This study investigated which maternal characteristics are associated with women's referral from midwife-led to obstetrician-led care.

MATERIALS AND METHODS

A retrospective cohort study in one midwife-led care practice in the Netherlands included 1096 low-risk women during January 2015-17. The primary outcomes were referral from midwife-led to obstetrician-led care in (1) the antepartum period and (2) the intrapartum period. In total, 11 maternal characteristics were identified. Logistic regression models of referral in each period were fitted and stratified by parity.

RESULTS

In the antepartum period, referral among nulliparous women was associated with an older maternal age (aOR, 1.07; 95%CI, 1.05-1.09), being underweight (0.45; 0.31-0.64), overweight (2.29; 1.91-2.74), or obese (2.65; 2.06-3.42), a preconception period >1 year (1.34; 1.07-1.66), medium education level (0.76; 0.58-1.00), deprivation (1.87; 1.54-2.26), and sexual abuse (1.44; 1.14-1.82). Among multiparous women, a referral was associated with being underweight (0.40; 0.26-0.60), obese (1.61; 1.30-1.98), a preconception period >1 year (1.71; 1.27-2.28), employment (1.38; 1.19-1.61), deprivation (1.23; 1.03-1.46), highest education level (0.63; 0.51-0.80), psychological problems (1.24; 1.06-1.44), and one or multiple consultations with an obstetrician (0.68; 0.58-0.80 and 0.64; 0.54-0.76, respectively). In the intrapartum period, referral among nulliparous women was associated with an older maternal age (1.02; 1.00-1.05), being underweight (1.67; 1.15-2.42), a preconception period >1 year (0.42; 0.31-0.57), medium or high level of education (2.09; 1.49-2.91 or 1.56; 1.10-2.22, respectively), sexual abuse (0.46; 0.33-0.63), and multiple consultations with an obstetrician (1.49; 1.15-1.94). Among multiparous women, referral was associated with an older maternal age (1.02; 1.00-1.04), being overweight (0.65; 0.51-0.83), a preconception period >1 year (0.33; 0.17-0.65), non-Dutch ethnicity (1.98; 1.61-2.45), smoking (0.75; 0.57-0.97), sexual abuse (1.49; 1.09-2.02), and one or multiple consultations with an obstetrician (1.34; 1.06-1.70 and 2.09; 1.63-2.69, respectively).

CONCLUSIONS

This exploratory study showed that several non-medical maternal characteristics of low-risk pregnant women are associated with referral from midwife-led to obstetrician-led care, and how these differ by parity and partum period.

Ten quick tips for computational analysis of medical images

Medical imaging is a great asset for modern medicine, since it allows physicians to spatially interrogate a disease site, resulting in precise intervention for diagnosis and treatment, and to observe particular aspect of patients' conditions that otherwise would not be noticeable. Computational analysis of medical images, moreover, can allow the discovery of disease patterns and correlations among cohorts of patients with the same disease, thus suggesting common causes or providing useful information for better therapies and cures. Machine learning and deep learning applied to medical images, in particular, have produced new, unprecedented results that can pave the way to advanced frontiers of medical discoveries. While computational analysis of medical images has become easier, however, the possibility to make mistakes or generate inflated or misleading results has become easier, too, hindering reproducibility and deployment. In this article, we provide ten quick tips to perform computational analysis of medical images avoiding common mistakes and pitfalls that we noticed in multiple studies in the past. We believe our ten guidelines, if taken into practice, can help the computational-medical imaging community to perform better scientific research that eventually can have a positive impact on the lives of patients worldwide.

Assessing metabolic syndrome prediction quality using seven anthropometric indices among Jordanian adults: a cross-sectional study

Metabolic syndrome (MSyn) is a considerable health concern in developing and developed countries, and it is a critical predictor of all-cause mortality. Obesity, specifically central obesity, is highly associated with MSyn incidence and development. In this study, seven anthropometric indices (Body Mass Index (BMI), Waist circumference (WC), Waist-to-Height Ratio (WHtR), A Body Shape Index (ABSI), Body Roundness Index (BRI), conicity index (CI), and the Visceral Adiposity Index (VAI)) were used to identify individuals with MSyn among the Jordanian population. These indices were assessed to identify their superiority in predicting the risk of MSyn. A total of 756 subjects (410 were male and 346 were female) were met between May 2018 and September 2019 and enrolled in this study. Height, weight, and waist circumferences were measured and BMI, WHtR, ABSI, BRI, CI, and VAI were calculated. Fasting plasma glucose level, lipid profile, and blood pressure were measured. Receiver-operating characteristic (ROC) curve was used to determine the discriminatory power of the anthropometric indices as classifiers for MSyn presence using the Third Adult Treatment Panel III (ATP III) definition. MSyn prevalence was 42.5%, and obese women and men have a significantly higher prevalence. BRI and WHtR showed the highest ability to predict MSyn (AUC = 0.83 for both indices). The optimal cutoff point for an early diagnosis of MSyn was > 28.4 kg/m² for BMI, > 98.5 cm for WC, > 5.13 for BRI, > 0.09 m^11/6 kg^-2/3 for ABSI, > 5.55 cm² for AVI, > 1.33 m^3/2 kg^-1/2 for CI, and > 0.59 for WHtR with males having higher cutoff points for MSyn early detection than females. In conclusion, we found that WHtR and BRI may be the best-suggested indices for MSyn prediction among Jordanian adults. These indices are affordable and might result in better early detection for MSyn and thereby may be helpful in the prevention of MSyn and its complications.

Diagnostic Performance of ACR and Kwak TI-RADS for Benign and Malignant Thyroid Nodules: An Update Systematic Review and Meta-Analysis

(1) Background: To determine the optimal cut-off values of two risk stratification systems to discriminate malignant thyroid nodules and to compare the diagnostic performance; (2) Methods: True and false positive and negative data were collected, and methodological quality was assessed for forty-six studies involving 39,085 patients; (3) Results: The highest area under the receiver operating characteristic (ROC) curve (AUC) of ACR and Kwak TI-RADS were 0.875 and 0.884. Based on the optimal sensitivity and specificity, the highest accuracy values of ROC curves or diagnostic odds ratios (DOR) were taken as the cut-off values for TR4 (moderate suspicious) and 4B. The sensitivity, specificity, DOR, and AUC by ACR (TR4) and Kwak TI-RADS (4B) for malignancy risk stratification of thyroid nodules were 94.3% and 96.4%; 52.2% and 53.7%; 17.5185 and 31.8051; 0.786 and 0.884, respectively. There were no significant differences in diagnostic accuracy in any of the direction comparisons of the two systems; (4) Conclusions: ACR and Kwak TI-RADS had good diagnostic performances (AUCs > 85%). Although we determined the best cut-off values in individual risk stratification systems based on statistical assessment, clinicians can adjust the optimal cut-off value according to the clinical purpose of the ultrasonography because raising or lowering cut-points leads to reciprocal changes in sensitivity and specificity.

Predicting Health Risks of Adult Asthmatics Susceptible to Indoor Air Quality Using Improved Logistic and Quantile Regression Models

The increasing global patterns for asthma disease and its associated fiscal burden to healthcare systems demand a change to healthcare processes and the way asthma risks are managed. Patient-centered health care systems equipped with advanced sensing technologies can empower patients to participate actively in their health risk control, which results in improving health outcomes. Despite having data analytics gradually emerging in health care, the path to well established and successful data driven health care services exhibit some limitations. Low accuracy of existing predictive models causes misclassification and needs improvement. In addition, lack of guidance and explanation of the reasons of a prediction leads to unsuccessful interventions. This paper proposes a modeling framework for an asthma risk management system in which the contributions are three fold: First, the framework uses a deep learning technique to improve the performance of logistic regression classification models. Second, it implements a variable sliding window method considering spatio-temporal properties of the data, which improves the quality of quantile regression models. Lastly, it provides a guidance on how to use the outcomes of the two predictive models in practice. To promote the application of predictive modeling, we present a use case that illustrates the life cycle of the proposed framework. The performance of our proposed framework was extensively evaluated using real datasets in which results showed improvement in the model classification accuracy, approximately 11.5–18.4% in the improved logistic regression classification model and confirmed low relative errors ranging from 0.018 to 0.160 in quantile regression model.

The False Dawn of Polygenic Risk Scores for Human Disease Prediction

Polygenic risk scores (PRSs) are being constructed for many diseases and are presented today as a promising avenue in the field of human genetics. These scores aim at predicting the risk of developing a disease by leveraging the many genome-wide association studies (GWAS) conducted during the two last decades. Important investments are being made to improve score estimates by increasing GWAS sample sizes, by developing more sophisticated methods, and by proposing different corrections for potential biases. PRSs have entered the market with direct-to-consumer companies proposing to compute them from saliva samples and even recently to help parents select the healthiest embryos. In this paper, we recall how PRSs arose and question the credit they are given by revisiting underlying assumptions in light of the history of human genetics and by comparing them with estimated breeding values (EBVs) used for selection in livestock.

UK National Screening Committee's approach to reviewing evidence on artificial intelligence in breast cancer screening

Artificial intelligence (AI) could have the potential to accurately classify mammograms according to the presence or absence of radiological signs of breast cancer, replacing or supplementing human readers (radiologists). The UK National Screening Committee's assessments of the use of AI systems to examine screening mammograms continues to focus on maximising benefits and minimising harms to women screened, when deciding whether to recommend the implementation of AI into the Breast Screening Programme in the UK. Maintaining or improving programme specificity is important to minimise anxiety from false positive results. When considering cancer detection, AI test sensitivity alone is not sufficiently informative, and additional information on the spectrum of disease detected and interval cancers is crucial to better understand the benefits and harms of screening. Although large retrospective studies might provide useful evidence by directly comparing test accuracy and spectrum of disease detected between different AI systems and by population subgroup, most retrospective studies are biased due to differential verification (ie, the use of different reference standards to verify the target condition among study participants). Enriched, multiple-reader, multiple-case, test set laboratory studies are also biased due to the laboratory effect (ie, radiologists' performance in retrospective, laboratory, observer studies is substantially different to their performance in a clinical environment). Therefore, assessment of the effect of incorporating any AI system into the breast screening pathway in prospective studies is required as it will provide key evidence for the effect of the interaction of medical staff with AI, and the impact on women's outcomes.

Pyroptosis is related to immune infiltration and predictive for survival of colon adenocarcinoma patients

Pyroptosis is a novel type of programmed cell death, initiated by inflammasome. Pyroptosis inhibits the development and metastasis of colon cancer and is associated with patients’ prognosis. However, how the pyroptosis-related genes predict the survival of patients is still unclear. In the study, colon adenocarcinoma (COAD) patients were divided into two groups according to the expression of pyroptosis-related regulators through consensus clustering. DEGs between two clusters were analyzed by using COX and Lasso regression. Then, regression coefficients in Lasso were used to calculate the risk score for every patient. Patients were classified into two types: low- and high-risk group according to their risk score. The difference of immune microenvironment infiltration and clinicopathological characteristics between subgroups was performed. Moreover, the nomogram model was built on the bases of risk model and clinicopathological factors. The TCGA-COAD cohort and GEO cohort were used as training and validating set respectively. 398 COAD patients in TCGA training set were identified as two regulation patterns via unsupervised clustering method. Patients in cluster 2 showed better prognosis (P = 0.002). Through differentiated expression analysis, COX and Lasso regression, a 5-gene prognostic risk model was constructed. This risk model was significantly associated with OS (HR: 2.088, 95% CI: 1.183–3.688, P = 0.011), validated in GEO set (HR:1.344, 95%CI: 1.061–1.704, P = 0.014), and patients with low risk had better prognosis (P < 0.001 in TCGA; P = 0.038 in GEO). Through ROC analysis, it can be found that this model presented better predictive accuracy for long-term survival. Clinical analyses demonstrated that high-risk group had more advanced N stage, higher risk of metastasis and later pathological stage. Immune-related analysis illustrated that low-risk group had more immune cell infiltration and more activated immune pathways. The pyroptosis-related risk model can be predictive for the survival of COAD patients. That patients with higher risk had poorer prognosis was associated with more advanced tumor stage and higher risk of metastasis, and resulted from highly activated pro-tumor pathways and inhibited immune system and poorer integrity of intestinal epithelial. This study proved the relationship between pyroptosis and immune, which offered basis for future studies.

Decisions are not all equal-Introducing a utility metric based on case-wise raters' perceptions

Background and Objective Evaluation of AI-based decision support systems (AI-DSS) is of critical importance in practical applications, nonetheless common evaluation metrics fail to properly consider relevant and contextual information. In this article we discuss a novel utility metric, the weighted Utility (wU), for the evaluation of AI-DSS, which is based on the raters' perceptions of their annotation hesitation and of the relevance of the training cases. Methods We discuss the relationship between the proposed metric and other previous proposals; and we describe the application of the proposed metric for both model evaluation and optimization, through three realistic case studies. Results We show that our metric generalizes the well-known Net Benefit, as well as other common error-based and utility-based metrics. Through the empirical studies, we show that our metric can provide a more flexible tool for the evaluation of AI models. We also show that, compared to other optimization metrics, model optimization based on the wU can provide significantly better performance (AUC 0.862 vs 0.895, p-value <0.05), especially on cases judged to be more complex by the human annotators (AUC 0.85 vs 0.92, p-value <0.05). Conclusions We make the point for having utility as a primary concern in the evaluation and optimization of machine learning models in critical domains, like the medical one; and for the importance of a human-centred approach to assess the potential impact of AI models on human decision making also on the basis of further information that can be collected during the ground-truthing process.

Multi-Class CNN for Classification of Multispectral and Autofluorescence Skin Lesion Clinical Images

In this work, we propose to use an artificial neural network to classify limited data of clinical multispectral and autofluorescence images of skin lesions. Although the amount of data is limited, the deep convolutional neural network classification of skin lesions using a multi-modal image set is studied and proposed for the first time. The unique dataset consists of spectral reflectance images acquired under 526 nm, 663 nm, 964 nm, and autofluorescence images under 405 nm LED excitation. The augmentation algorithm was applied for multi-modal clinical images of different skin lesion groups to expand the training datasets. It was concluded from saliency maps that the classification performed by the convolutional neural network is based on the distribution of the major skin chromophores and endogenous fluorophores. The resulting classification confusion matrices, as well as the performance of trained neural networks, have been investigated and discussed.

Utility of pre-operative haemoglobin concentration to guide peri-operative blood tests for hip and knee arthroplasty: A decision curve analysis

Objective

Assess the prognostic value of pre‐operative haemoglobin concentration (Hb) for identifying patients who develop severe post‐operative anaemia or require blood transfusion following primary total hip or knee, or unicompartmental knee arthroplasty (THA, TKA, UKA).

Background

Pre‐operative group and save (G&S), and post‐operative Hb measurement may be unnecessary for many patients undergoing hip and knee arthroplasty provided individuals at greatest risk of severe post‐operative anaemia can be identified.

Methods and Materials

Patients undergoing THA, TKA, or UKA between 2011 and 2018 were included. Outcomes were post‐operative Hb below 70 and 80 g/L, and peri‐operative blood transfusion. Logistic regression assessed the association between pre‐operative Hb and each outcome. Decision curve analysis compared strategies for selecting patients for G&S and post‐operative Hb measurement.

Results

10 015 THA, TKA and UKA procedures were performed in 8582 patients. The incidence of blood transfusion (4.5%) decreased during the study. Using procedure specific Hb thresholds to select patients for pre‐operative G&S and post‐operative Hb testing had a greater net benefit than selecting all patients, no patients, or patients with pre‐operative anaemia.

Conclusions

Pre‐operative G&S and post‐operative Hb measurement may not be indicated for UKA or TKA when adopting restrictive transfusion thresholds, provided clinicians accept a 0.1% risk of patients developing severe undiagnosed post‐operative anaemia (Hb < 70 g/L). The decision to perform these blood tests for THA patients should be based on local institutional data and selection of acceptable risk thresholds.

The Tibial Tubercle-Trochlear Groove Distance/Trochlear Dysplasia Index Quotient Is the Most Accurate Indicator for Determining Patellofemoral Instability Risk

PURPOSE

The primary aim of our study was to evaluate diagnostic accuracy of the tibial tubercle-trochlear groove (TT-TG) distance relative to associated quotients produced from trochlear width (TT-TG distance/TW) and trochlear dysplasia index (TT-TG distance/TDI) for detecting patellofemoral instability. Secondary aims included identifying thresholds for risk and comparing differences between cases and controls.

METHODS

Consecutive sampling of electronic medical records produced 48 (21 males, 27 females) patellofemoral instability cases (19 ± 7 years old) and 79 (61 males, 18 females) controls (23 ± 4 years old) who had a history of isolated meniscal lesion, as evaluated by magnetic resonance imaging. Standardized methods were employed with measurements executed in a blinded and randomized manner. A receiver operating characteristic curve assessed accuracy by area under the curve (AUC). The index of union (IU) was employed to identify a threshold for risk. Two-sample t-tests examined group differences. P < .05 denoted statistical significance.

RESULTS

The AUC values were .69 (.60, .79) for TT-TG distance, .81 (.73, .88) for TT-TG distance/TW, and .85 (.78, .91) for TT-TG distance/TDI. Thresholds were 14.7 mm for TT-TG distance, .36 for TT-TG distance/TW, and 1.88 for TT-TG distance/TDI. Cases demonstrated statistically significant (P < .001) greater values for each measure compared with controls: TT-TG distance (15.8 ± 4.2 mm vs 12.9 ± 3.6 mm, [1.4, 4.3]); TT-TG distance/TW (.51 ± .24 vs .31 ± .09, [.13, .27]); TT-TG distance/TDI (3.07 ± 1.55 vs 1.7 ± .7, [.9, 1.84]).

CONCLUSION

The TT-TG distance, TT-TG distance/TW, and TT-TG distance/TDI measures were 69%, 81%, and 85%, respectively, accurate for determining patellofemoral instability risk. Thresholds for risk were 14.7 mm for TT-TG distance, .36 for TT-TG distance/TW, and 1.88 for TT-TG distance/TDI. The thresholds reported in this study may help in advancing clinical decision-making.

LEVEL OF EVIDENCE

Level III, diagnostic retrospective comparative observatory trial.

Platelet reactivity testing in peripheral artery disease

DISCLAIMER

In an effort to expedite the publication of articles related to the COVID-19 pandemic, AJHP is posting these manuscripts online as soon as possible after acceptance. Accepted manuscripts have been peer-reviewed and copyedited, but are posted online before technical formatting and author proofing. These manuscripts are not the final version of record and will be replaced with the final article (formatted per AJHP style and proofed by the authors) at a later time.

PURPOSE

Oral antiplatelet therapy is routinely used to prevent adverse cardiovascular events in patients with peripheral artery disease (PAD). Several laboratory tests are available to quantify the degree of platelet inhibition following antiplatelet therapy. This article aims to provide a review of the literature surrounding platelet functional testing in patients with PAD receiving oral P2Y12 inhibitors and to offer guidance to clinicians for the use and interpretation of these tests.

SUMMARY

A literature search of PubMed and the Web of Science Core Collection database was conducted. All studies that performed platelet function testing and reported clinical outcomes in patients with PAD were included. Evaluation of the data suggests that, among the available testing strategies, the VerifyNow platelet reactivity unit (PRU) test is the most widely used. Despite numerous investigations attempting to define a laboratory threshold indicating suboptimal response to antiplatelet therapy, controversy exists about which PRU value best correlates with cardiovascular outcomes (ie, mortality, stent thrombosis, etc). In the PAD literature, the most commonly used PRU thresholds are 208 or higher and 235 or higher. Nonetheless, adjusting antiplatelet regimens based on suboptimal P2Y12 reactivity values has yet to be proven useful in reducing the incidence of adverse cardiovascular outcomes. This review examines platelet function testing in patients with PAD and discusses the interpretation and application of these tests when monitoring the safety and efficacy of P2Y12 inhibitors.

CONCLUSION

Although platelet functional tests may be simple to use, clinical trials thus far have failed to show benefit from therapy adjustments based on test results. Clinicians should be cautioned against relying on this test result alone and should instead consider a combination of laboratory, clinical, and patient-specific factors when adjusting P2Y12 inhibitor therapy in clinical practice.

Artificial intelligence to identify genetic alterations in conventional histopathology

Precision oncology relies on the identification of targetable molecular alterations in tumor tissues. In many tumor types, a limited set of molecular tests is currently part of standard diagnostic workflows. However, universal testing for all targetable alterations, especially rare ones, is limited by the cost and availability of molecular assays. From 2017 to 2021, multiple studies have shown that artificial intelligence (AI) methods can predict the probability of specific genetic alterations directly from conventional hematoxylin and eosin (H&E) tissue slides. Although these methods are currently less accurate than gold-standard testing (e.g. immunohistochemistry, polymerase chain reaction or next-generation sequencing), they could be used as pre-screening tools to reduce the workload of genetic analyses. In this systematic literature review, we summarize the state of the art in predicting molecular alterations from H&E using AI. We found that AI methods perform reasonably well across multiple tumor types, although few algorithms have been broadly validated. In addition, we found that genetic alterations in FGFR, IDH, PIK3CA, BRAF, TP53 and DNA repair pathways are predictable from H&E in multiple tumor types, while many other genetic alterations have rarely been investigated or were only poorly predictable. Finally, we discuss the next steps for the implementation of AI-based surrogate tests in diagnostic workflows. This article is protected by copyright. All rights reserved.

Advancements in Oncology with Artificial Intelligence—A Review Article

Simple Summary

With the advancement of artificial intelligence, including machine learning, the field of oncology has seen promising results in cancer detection and classification, epigenetics, drug discovery, and prognostication. In this review, we describe what artificial intelligence is and its function, as well as comprehensively summarize its evolution and role in breast, colorectal, and central nervous system cancers. Understanding the origin and current accomplishments might be essential to improve the quality, accuracy, generalizability, cost-effectiveness, and reliability of artificial intelligence models that can be used in worldwide clinical practice. Students and researchers in the medical field will benefit from a deeper understanding of how to use integrative AI in oncology for innovation and research.

Abstract

Well-trained machine learning (ML) and artificial intelligence (AI) systems can provide clinicians with therapeutic assistance, potentially increasing efficiency and improving efficacy. ML has demonstrated high accuracy in oncology-related diagnostic imaging, including screening mammography interpretation, colon polyp detection, glioma classification, and grading. By utilizing ML techniques, the manual steps of detecting and segmenting lesions are greatly reduced. ML-based tumor imaging analysis is independent of the experience level of evaluating physicians, and the results are expected to be more standardized and accurate. One of the biggest challenges is its generalizability worldwide. The current detection and screening methods for colon polyps and breast cancer have a vast amount of data, so they are ideal areas for studying the global standardization of artificial intelligence. Central nervous system cancers are rare and have poor prognoses based on current management standards. ML offers the prospect of unraveling undiscovered features from routinely acquired neuroimaging for improving treatment planning, prognostication, monitoring, and response assessment of CNS tumors such as gliomas. By studying AI in such rare cancer types, standard management methods may be improved by augmenting personalized/precision medicine. This review aims to provide clinicians and medical researchers with a basic understanding of how ML works and its role in oncology, especially in breast cancer, colorectal cancer, and primary and metastatic brain cancer. Understanding AI basics, current achievements, and future challenges are crucial in advancing the use of AI in oncology.

Biomarkers of Vascular Inflammation for Cardiovascular Risk Prognostication: A Meta-Analysis

OBJECTIVES

The purpose of this study was to systematically explore the added value of biomarkers of vascular inflammation for cardiovascular prognostication on top of clinical risk factors.

BACKGROUND

Measurement of biomarkers of vascular inflammation is advocated for the risk stratification for coronary heart disease (CHD).

METHODS

We systematically explored published reports in MEDLINE for cohort studies on the prognostic value of common biomarkers of vascular inflammation in stable patients without known CHD. These included common circulating inflammatory biomarkers (ie, C-reactive protein, interleukin-6 and tumor necrosis factor-a, arterial positron emission tomography/computed tomography and coronary computed tomography angiography-derived biomarkers of vascular inflammation, including anatomical high-risk plaque features and perivascular fat imaging. The main endpoint was the difference in c-index (Δ[c-index]) with the use of inflammatory biomarkers for major adverse cardiovascular events (MACEs) and mortality. We calculated I2 to test heterogeneity. This study is registered with PROSPERO (CRD42020181158).

RESULTS

A total of 104,826 relevant studies were screened and a final of 39 independent studies (175,778 individuals) were included in the quantitative synthesis. Biomarkers of vascular inflammation provided added prognostic value for the composite endpoint and for MACEs only (pooled estimate for Δ[c-index]% 2.9, 95% CI: 1.7-4.1 and 3.1, 95% CI: 1.8-4.5, respectively). Coronary computed tomography angiography-related biomarkers were associated with the highest added prognostic value for MACEs: high-risk plaques 5.8%, 95% CI: 0.6 to 11.0, and perivascular adipose tissue (on top of coronary atherosclerosis extent and high-risk plaques): 8.2%, 95% CI: 4.0 to 12.5). In meta-regression analysis, the prognostic value of inflammatory biomarkers was independent of other confounders including study size, length of follow-up, population event incidence, the performance of the baseline model, and the level of statistical adjustment. Limitations in the published literature include the lack of reporting of other metrics of improvement of risk stratification, the net clinical benefit, or the cost-effectiveness of such biomarkers in clinical practice.

CONCLUSIONS

The use of biomarkers of vascular inflammation enhances risk discrimination for cardiovascular events.

A hierarchical opportunistic screening model for osteoporosis using machine learning applied to clinical data and CT images

Background

Osteoporosis is a common metabolic skeletal disease and usually lacks obvious symptoms. Many individuals are not diagnosed until osteoporotic fractures occur. Bone mineral density (BMD) measured by dual-energy X-ray absorptiometry (DXA) is the gold standard for osteoporosis detection. However, only a limited percentage of people with osteoporosis risks undergo the DXA test. As a result, it is vital to develop methods to identify individuals at-risk based on methods other than DXA.

Results

We proposed a hierarchical model with three layers to detect osteoporosis using clinical data (including demographic characteristics and routine laboratory tests data) and CT images covering lumbar vertebral bodies rather than DXA data via machine learning. 2210 individuals over age 40 were collected retrospectively, among which 246 individuals’ clinical data and CT images are both available. Irrelevant and redundant features were removed via statistical analysis. Consequently, 28 features, including 16 clinical data and 12 texture features demonstrated statistically significant differences (p < 0.05) between osteoporosis and normal groups. Six machine learning algorithms including logistic regression (LR), support vector machine with radial-basis function kernel, artificial neural network, random forests, eXtreme Gradient Boosting and Stacking that combined the above five classifiers were employed as classifiers to assess the performances of the model. Furthermore, to diminish the influence of data partitioning, the dataset was randomly split into training and test set with stratified sampling repeated five times. The results demonstrated that the hierarchical model based on LR showed better performances with an area under the receiver operating characteristic curve of 0.818, 0.838, and 0.962 for three layers, respectively in distinguishing individuals with osteoporosis and normal BMD.

Conclusions

The proposed model showed great potential in opportunistic screening for osteoporosis without additional expense. It is hoped that this model could serve to detect osteoporosis as early as possible and thereby prevent serious complications of osteoporosis, such as osteoporosis fractures.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12859-022-04596-z.

Assessment of Executive Function in Everyday Life—Psychometric Properties of the Norwegian Adaptation of the Children’s Cooking Task

Background: There are few standardized measures available to assess executive function (EF) in a naturalistic setting for children. The Children’s Cooking Task (CCT) is a complex test that has been specifically developed to assess EF in a standardized open-ended environment (cooking). The aim of the present study was to evaluate the internal consistency, inter-rater reliability, sensitivity and specificity, and also convergent and divergent validity of the Norwegian version of CCT among children with pediatric Acquired Brain Injury (pABI) and healthy controls (HCs).

Methods: The present study has a cross-sectional design, based on baseline data derived from a multicenter RCT. Seventy-five children with pABI from two university hospitals with parent-reported executive dysfunction and minimum of 12 months since injury/completed cancer therapy, as well as 59 HCs aged 10–17 years, were assessed with CCT using total errors as the main outcome measure. The pABI group completed tests assessing EF (i.e., inhibition, cognitive flexibility, working memory, and planning) on the impairment level within the ICF framework (performance-based neuropsychological tests and the Behavioral Assessment of the Dysexecutive Syndrome for Children), and on the participation level (questionnaires). In addition, they completed tests of intellectual ability, processing speed, attention, learning, and memory. Finally, overall functional outcome (pediatric Glasgow Outcome Scale-Extended) was evaluated for the children with pABI.

Results: Acceptable internal consistency and good inter-rater reliability were found for the CCT. Children with pABI performed significantly worse on the CCT than the HCs. The CCT identified group membership, but the sensitivity and specificity were overall classified as poor. Convergent validity was demonstrated by associations between the CCT and performance-based tests assessing inhibition, cognitive flexibility, and working memory, as well as teacher-reported executive dysfunction (questionnaires). Divergent validity was supported by the lack of association with performance-based measures of learning and memory, attention, and verbal intellectual ability. However, there was a moderate association between the CCT and performance-based tests of processing speed. Lastly, better performance on the CCT was associated with a better functional outcome.

Conclusion: Our study with a relatively large sample of children with pABI and HC’s demonstrated good psychometric properties of the CCT. CCT performance was associated with the overall level of disability and function, suggesting that CCT is related to the level of activity in everyday life and participation in society. Hence, our study suggests that the CCT has the potential to advance the assessment of EF by providing a valid analysis of real-world performance. Nevertheless, further research is needed on larger samples, focusing on predictors of task performance, and evaluating the ability of CCT to detect improvement in EF over time. The patterns of error and problem-solving strategies evaluated by the CCT could be used to inform neuropsychological rehabilitation treatmentand represent a more valid outcome measure of rehabilitation interventions.

Artificial Intelligence Applications in Urology: Reporting Standards to Achieve Fluency for Urologists

The growth and adoption of artificial intelligence has led to impressive results in urology. As artificial intelligence grows more ubiquitous, it is important to establish artificial intelligence literacy in the workforce. To this end, we present a narrative review of the literature of artificial intelligence and machine learning in urology and propose a checklist of reporting standards to improve readability and evaluate the current state of the literature. The listed article demonstrated heterogeneous reporting of methodologies and outcomes, limiting generalizability of research. We hope that this review serves as a foundation for future evaluation of medical research in artificial intelligence.

General Performance Score for classification problems

Several performance metrics are currently available to evaluate the performance of Machine Learning (ML) models in classification problems. ML models are usually assessed using a single measure because it facilitates the comparison between several models. However, there is no silver bullet since each performance metric emphasizes a different aspect of the classification. Thus, the choice depends on the particular requirements and characteristics of the problem. An additional problem arises in multi-class classification problems, since most of the well-known metrics are only directly applicable to binary classification problems. In this paper, we propose the General Performance Score (GPS), a methodological approach to build performance metrics for binary and multi-class classification problems. The basic idea behind GPS is to combine a set of individual metrics, penalising low values in any of them. Thus, users can combine several performance metrics that are relevant in the particular problem based on their preferences obtaining a conservative combination. Different GPS-based performance metrics are compared with alternatives in classification problems using real and simulated datasets. The metrics built using the proposed method improve the stability and explainability of the usual performance metrics. Finally, the GPS brings benefits in both new research lines and practical usage, where performance metrics tailored for each particular problem are considered.

Recalibration and validation of the Charlson Comorbidity Index in acute kidney injury patients underwent continuous renal replacement therapy

Background

Comorbid conditions impact the survival of patients with severe acute kidney injury (AKI) who require continuous renal replacement therapy (CRRT). The weights assigned to comorbidities in predicting survival vary based on type of index, disease, and advances in management of comorbidities. We developed a modified Charlson Comorbidity Index (CCI) for use in patients with AKI requiring CRRT (mCCI-CRRT) and improved the accuracy of risk stratification for mortality.

Methods

A total of 828 patients who received CRRT between 2008 and 2013, from three university hospital cohorts was included to develop the comorbidity score. The weights of the comorbidities were recalibrated using a Cox proportional hazards model adjusted for demographic and clinical information. The modified index was validated in a university hospital cohort (n = 919) using the data of patients treated from 2009 to 2015.

Results

Weights for dementia, peptic ulcer disease, any tumor, and metastatic solid tumor were used to recalibrate the mCCI-CRRT. Use of these calibrated weights achieved a 35.4% (95% confidence interval [CI], 22.1%–48.1%) higher performance than unadjusted CCI in reclassification based on continuous net reclassification improvement in logistic regression adjusted for age and sex. After additionally adjusting for hemoglobin and albumin, consistent results were found in risk reclassification, which improved by 35.9% (95% CI, 23.3%–48.5%).

Conclusion

The mCCI-CRRT stratifies risk of mortality in AKI patients who require CRRT more accurately than does the original CCI, suggesting that it could serve as a preferred index for use in clinical practice.

Can minimal clinically important differences in patient reported outcome measures be predicted by machine learning in patients with total knee or hip arthroplasty? A systematic review

OBJECTIVES

To systematically review studies using machine learning (ML) algorithms to predict whether patients undergoing total knee or total hip arthroplasty achieve an improvement as high or higher than the minimal clinically important differences (MCID) in patient reported outcome measures (PROMs) (classification problem).

METHODS

Studies were eligible to be included in the review if they collected PROMs both pre- and postintervention, reported the method of MCID calculation and applied ML. ML was defined as a family of models which automatically learn from data when selecting features, identifying nonlinear relations or interactions. Predictive performance must have been assessed using common metrics. Studies were searched on MEDLINE, PubMed Central, Web of Science Core Collection, Google Scholar and Cochrane Library. Study selection and risk of bias assessment (ROB) was conducted by two independent researchers.

RESULTS

517 studies were eligible for title and abstract screening. After screening title and abstract, 18 studies qualified for full-text screening. Finally, six studies were included. The most commonly applied ML algorithms were random forest and gradient boosting. Overall, eleven different ML algorithms have been applied in all papers. All studies reported at least fair predictive performance, with two reporting excellent performance. Sample size varied widely across studies, with 587 to 34,110 individuals observed. PROMs also varied widely across studies, with sixteen applied to TKA and six applied to THA. There was no single PROM utilized commonly in all studies. All studies calculated MCIDs for PROMs based on anchor-based or distribution-based methods or referred to literature which did so. Five studies reported variable importance for their models. Two studies were at high risk of bias.

DISCUSSION

No ML model was identified to perform best at the problem stated, nor can any PROM said to be best predictable. Reporting standards must be improved to reduce risk of bias and improve comparability to other studies.

A deep learning model for early risk prediction of heart failure with preserved ejection fraction by DNA methylation profiles combined with clinical features

Background

Heart failure with preserved ejection fraction (HFpEF), affected collectively by genetic and environmental factors, is the common subtype of chronic heart failure. Although the available risk assessment methods for HFpEF have achieved some progress, they were based on clinical or genetic features alone. Here, we have developed a deep learning framework, HFmeRisk, using both 5 clinical features and 25 DNA methylation loci to predict the early risk of HFpEF in the Framingham Heart Study Cohort.

Results

The framework incorporates Least Absolute Shrinkage and Selection Operator and Extreme Gradient Boosting-based feature selection, as well as a Factorization-Machine based neural network-based recommender system. Model discrimination and calibration were assessed using the AUC and Hosmer–Lemeshow test. HFmeRisk, including 25 CpGs and 5 clinical features, have achieved the AUC of 0.90 (95% confidence interval 0.88–0.92) and Hosmer–Lemeshow statistic was 6.17 (P = 0.632), which outperformed models with clinical characteristics or DNA methylation levels alone, published chronic heart failure risk prediction models and other benchmark machine learning models. Out of them, the DNA methylation levels of two CpGs were significantly correlated with the paired transcriptome levels (R < −0.3, P < 0.05). Besides, DNA methylation locus in HFmeRisk were associated with intercellular signaling and interaction, amino acid metabolism, transport and activation and the clinical variables were all related with the mechanism of occurrence of HFpEF. Together, these findings give new evidence into the HFmeRisk model.

Conclusion

Our study proposes an early risk assessment framework for HFpEF integrating both clinical and epigenetic features, providing a promising path for clinical decision making.