Caveats and pitfalls of ROC analysis in clinical microarray research (and how to avoid them)

LOCC: a novel visualization and scoring of cutoffs for continuous variables with hepatocellular carcinoma prognosis as an example

BACKGROUND

The interpretation of large datasets, such as The Cancer Genome Atlas (TCGA), for scientific and research purposes, remains challenging despite their public availability. In this study, we focused on identifying gene expression profiles most relevant to patient prognosis and aimed to develop a method and database to address this issue. To achieve this, we introduced Luo's Optimization Categorization Curve (LOCC), an innovative tool for visualizing and scoring continuous variables against dichotomous outcomes. To demonstrate the efficacy of LOCC using real-world data, we analyzed gene expression profiles and patient data from TCGA hepatocellular carcinoma samples.

RESULTS

To showcase LOCC, we demonstrate an optimal cutoff for E2F1 expression in hepatocellular carcinoma, which was subsequently validated in an independent cohort. Compared to ROC curves and their AUC, LOCC offered a superior description of the predictive value of E2F1 expression across various cancer types. The LOCC score, comprised of factors representing significance, range, and impact of the biomarker, facilitated the ranking of all gene expression profiles in hepatocellular carcinoma, aiding in the evaluation and understanding of previously published prognostic gene signatures. We also demonstrate that LOCC does not have the same assumptions required of Cox proportional hazards modeling for accurate analysis. Repeated sampling demonstrated that LOCC scores outperformed ROC's AUC in discriminating predictors from non-predictors. Additionally, gene set enrichment analysis revealed significant associations between certain genes and prognosis, such as E2F target genes and G2M checkpoint with poor prognosis, and bile acid metabolism and oxidative phosphorylation with good prognosis.

CONCLUSION

In summary, we present LOCC as a novel visualization tool for the analysis of gene expression in cancer, particularly for understanding and selecting cutoffs. Our findings suggest that LOCC scores, which effectively rank genes based on their prognostic potential, represent a more suitable approach than ROC curves and Cox proportional hazard for prognostic modeling and understanding in cancer gene expression analysis. LOCC holds promise as an invaluable tool for advancing precision medicine and furthering biomarker research. Further research regarding multivariable integration and validation will help LOCC reach its full potential and establish its utility across diverse cancer types and clinical settings.

Machine learning model with output correction: Towards reliable bradycardia detection in neonates

Bradycardia is a commonly occurring condition in premature infants, often causing serious consequences and cardiovascular complications. Reliable and accurate detection of bradycardia events is pivotal for timely intervention and effective treatment. Excessive false alarms pose a critical problem in bradycardia event detection, eroding trust in machine learning (ML)-based clinical decision support tools designed for such detection. This could result in disregarding the algorithm's accurate recommendations and disrupting workflows, potentially compromising the quality of patient care. This article introduces an ML-based approach incorporating an output correction element, designed to minimise false alarms. The approach has been applied to bradycardia detection in preterm infants. We applied five ML-based autoencoder techniques, using recurrent neural network (RNN), long-short-term memory (LSTM), gated recurrent unit (GRU), 1D convolutional neural network (1D CNN), and a combination of 1D CNN and LSTM. The analysis is performed on ∼440 hours of real-time preterm infant data. The proposed approach achieved 0.978, 0.73, 0.992, 0.671 and 0.007 in AUC-ROC, AUC-PRC, recall, F1 score, and false positive rate (FPR) respectively and a false alarms reduction of 36% when compared with methods without the correction approach. This study underscores the imperative of cultivating solutions that alleviate alarm fatigue and encourage active engagement among healthcare professionals.

Improving the Prediction of 1-Year Right Ventricular Failure After Left Ventricular Assist Device Implantation

Previous predictive models for postimplant right heart failure (RHF) following left ventricular assist device (LVAD) implantation have demonstrated limited performance on validation datasets and are susceptible to overfitting. Thus, the objective of this study was to develop an improved predictive model with reduced overfitting and improved accuracy in predicting RHF in LVAD recipients. The study involved 11,967 patients who underwent continuous-flow LVAD implantation between 2008 and 2016, with an RHF incidence of 9% at 1 year. Using an eXtreme Gradient Boosting (XGBoost) algorithm, the training data were used to predict RHF at 1 year postimplantation, resulting in promising area under the curve (AUC)-receiver operating characteristic (ROC) of 0.8 and AUC-precision recall curve (PRC) of 0.24. The calibration plot showed that the predicted risk closely corresponded with the actual observed risk. However, the model based on data collected 48 hours before LVAD implantation exhibited high sensitivity but low precision, making it an excellent screening tool but not a diagnostic tool.

Non-obese lipedema patients show a distinctly altered quantitative sensory testing profile with high diagnostic potential

Introduction and Objectives

Lipedema is a widespread severe chronic disease affecting mostly women. Characterized by painful bilateral fat accumulation in extremities sparing hands and feet, objective measurement-based diagnosis is currently missing. We tested for characteristic psychometric and/or sensory alterations including pain and for their potential for medical routine diagnosis.

Methods

Pain psychometry was assessed using the German Pain Questionnaire. Sensory sensitivity toward painful and nonpainful stimuli was characterized in non-obese lipedema patients and matched controls using the validated quantitative sensory testing (QST) protocol of the German Research Network on Neuropathic Pain.

Results

Lipedema patients showed no overt psychometric abnormalities. Pain was reported as somatic rather than psychosomatic aversive. All QST measurements were normal, but the z-score of pressure pain thresholds (PPT) was twofold reduced and the z-score of vibration detection thresholds (VDT) was two and a half times increased. Both thresholds were selectively altered at the affected thigh but not the unaffected hand. Receiver operating characteristic analysis of the combination of PPT and VDT of thigh vs hand into a PVTH score (PPT, VDT, thigh, hand-score) shows high sensitivity and specificity, categorizing correctly 95.8% of the participants as lipedema patients or healthy controls. Bayesian inference analysis corroborated the diagnostic potential of such a combined PVTH score.

Conclusion

We propose to assess PPT and VDT at the painful thigh and the pain-free hand. Combination in a PVTH score may allow a convenient lipedema diagnosis early during disease development.

External validation of the ARCH score in patients undergoing aortic arch reconstruction under circulatory arrest

BACKGROUND

Aortic arch surgery with hypothermic circulatory arrest (HCA) carries a higher risk of morbidity and mortality compared to routine cardiac surgical procedures. The newly developed ARCH (arch reconstruction under circulatory arrest with hypothermia) score has not been externally validated. We sought to externally validate this score in our local population.

METHODS

All consecutive open aortic arch surgeries with HCA performed between 2014 and 2023 were included. Univariable and multivariable analyses were performed. Model discrimination was assessed by the C-statistic with 95% confidence intervals as part of the receiver operating characteristic (ROC) curve analysis. Model performance was visualized by a calibration plot and quantified by the Brier score.

RESULTS

A total of 760 patients (38.3% females) were included. The mean age was 61 (±13.6) years, with 56.4% of patients' age >60 years. The procedures were carried out mostly emergently or urgently (59.6%). Total arch replacement was performed in 32.5% of the patients, and aortic root procedures were carried out in 74.6%. In-hospital death occurred in 64 patients (8.4%), and stroke occurred in 5.4%. The C-statistic revealed a low discriminatory ability for predicting in-hospital mortality (area under the ROC curve, 0.62; 95% confidence interval, 0.54-0.69; P = .002); however, model calibration was found to be excellent (Brier score of 0.07).

CONCLUSIONS

The ARCH score for in-hospital mortality showed low discriminatory ability in our local population, although with excellent ability for prediction of mortality.

Towards optimal model evaluation: enhancing active testing with actively improved estimators

With rapid advancements in machine learning and statistical models, ensuring the reliability of these models through accurate evaluation has become imperative. Traditional evaluation methods often rely on fully labeled test data, a requirement that is becoming increasingly impractical due to the growing size of datasets. In this work, we address this issue by extending existing work on active testing (AT) methods which are designed to sequentially sample and label data for evaluating pre-trained models. We propose two novel estimators: the Actively Improved Levelled Unbiased Risk (AILUR) and the Actively Improved Inverse Probability Weighting (AIIPW) estimators which are derived from nonparametric smoothing estimation. In addition, a model recalibration process is designed for the AIIPW estimator to optimize the sampling probability within the AT framework. We evaluate the proposed estimators on four real-world datasets and demonstrate that they consistently outperform existing AT methods. Our study also shows that the proposed methods are robust to changes in subsample sizes, and effective at reducing labeling costs.

Multiplex Serology for Sensitive and Specific Flavivirus IgG Detection: Addition of Envelope Protein Domain III to NS1 Increases Sensitivity for Tick-Borne Encephalitis Virus IgG Detection

Tick-borne encephalitis is a vaccine-preventable disease of concern for public health in large parts of Europe, with EU notification rates increasing since 2018. It is caused by the orthoflavivirus tick-borne encephalitis virus (TBEV) and a diagnosis of infection is mainly based on serology due to its short viremic phase, often before symptom onset. The interpretation of TBEV serology is hampered by a history of orthoflavivirus vaccination and by previous infections with related orthoflaviviruses. Here, we sought to improve TBEV sero-diagnostics using an antigen combination of in-house expressed NS1 and EDIII in a multiplex, low-specimen-volume set-up for the detection of immune responses to TBEV and other clinically important orthoflaviviruses (i.e., West Nile virus, dengue virus, Japanese encephalitis virus, Usutu virus and Zika virus). We show that the combined use of NS1 and EDIII results in both a specific and sensitive test for the detection of TBEV IgG for patient diagnostics, vaccination responses and in seroprevalence studies. This novel approach potentially allows for a low volume-based, simultaneous analysis of IgG responses to a range of orthoflaviviruses with overlapping geographic circulations and clinical manifestations.

Direct Detection of 4-Dimensions of SARS-CoV-2: Infection (vRNA), Infectivity (Antigen), Binding Antibody, and Functional Neutralizing Antibody in Saliva

We developed a 4-parameter clinical assay using Electric Field Induced Release and Measurement (EFIRM) technology to simultaneously assess SARS-CoV-2 RNA (vRNA), nucleocapsid antigen, host binding (BAb) and neutralizing antibody (NAb) levels from a drop of saliva with performance that equals or surpasses current EUA-approved tests. The vRNA and antigen assays achieved lower limit of detection (LOD) of 100 copies/reaction and 3.5 TCID₅₀/mL, respectively. The vRNA assay differentiated between acutely infected (n=10) and infection-naïve patients (n=33) with an AUC of 0.9818, sensitivity of 90%, and specificity of 100%. The antigen assay similarly differentiated these patient populations with an AUC of 1.000. The BAb assay detected BAbs with an LOD of 39 pg/mL and distinguished acutely infected (n=35), vaccinated with prior infection (n=13), and vaccinated infection-naïve patients (n=13) from control (n=81) with AUC of 0.9481, 1.000, and 0.9962, respectively. The NAb assay detected NAbs with an LOD of 31.6 Unit/mL and differentiated between COVID-19 recovered or vaccinated patients (n=31) and pre-pandemic controls (n=60) with an AUC 0.923, sensitivity of 87.10%, and specificity of 86.67%. Our multiparameter assay represents a significant technological advancement to simultaneously address SARS-CoV-2 infection and immunity, and it lays the foundation for tackling potential future pandemics.

Novel nomogram for predicting paradoxical chest wall movement in patients with flail segment of traumatic rib fracture: a retrospective cohort study

Flail chest is a severe injury to the chest wall and is related to adverse outcomes. A flail chest is classified as the physiologic, paradoxical motion of a chest wall or flail segment of rib fracture (RFX). We hypothesized that patients with paradoxical chest wall movement would present different clinical features from patients with a flail segment. This retrospective observational study included patients with blunt chest trauma who visited our level 1 trauma center between January 2019 and October 2022 and were diagnosed with one or more flail segments by computed tomography. The primary outcome of our study was a clinically diagnosed visible, paradoxical chest wall motion. We used the least absolute shrinkage and selection operator (LASSO) logistic regression model to minimize overfitting. After a feature selection using the LASSO regression model, we constructed a multivariable logistic regression (MLR) model and nomogram. A total of five risk factors were selected in the LASSO model and applied to the multivariable logistic regression model. Of these, four risk factors were statistically significant: the total number of RFX (adjusted OR [aOR], 1.28; 95% confidence interval [CI], 1.09-1.49; p = 0.002), number of segmental RFX including Grade III fractures (aOR, 1.78; 95% CI, 1.14-2.79; p = 0.012), laterally located primary fracture lines (aOR, 4.00; 95% CI, 1.69-9.43; p = 0.002), and anterior-lateral flail segments (aOR, 4.20; 95% CI, 1.60-10.99; p = 0.004). We constructed a nomogram to predict the personalized probability of the flail motion. A novel nomogram was developed in patients with flail segments of traumatic RFX to predict paradoxical chest wall motion. The number of RFX, Grade III segmental RFX, and the location of the RFX were significant risk factors.

DeepGAMI: deep biologically guided auxiliary learning for multimodal integration and imputation to improve genotype-phenotype prediction

BACKGROUND

Genotypes are strongly associated with disease phenotypes, particularly in brain disorders. However, the molecular and cellular mechanisms behind this association remain elusive. With emerging multimodal data for these mechanisms, machine learning methods can be applied for phenotype prediction at different scales, but due to the black-box nature of machine learning, integrating these modalities and interpreting biological mechanisms can be challenging. Additionally, the partial availability of these multimodal data presents a challenge in developing these predictive models.

METHOD

To address these challenges, we developed DeepGAMI, an interpretable neural network model to improve genotype-phenotype prediction from multimodal data. DeepGAMI leverages functional genomic information, such as eQTLs and gene regulation, to guide neural network connections. Additionally, it includes an auxiliary learning layer for cross-modal imputation allowing the imputation of latent features of missing modalities and thus predicting phenotypes from a single modality. Finally, DeepGAMI uses integrated gradient to prioritize multimodal features for various phenotypes.

RESULTS

We applied DeepGAMI to several multimodal datasets including genotype and bulk and cell-type gene expression data in brain diseases, and gene expression and electrophysiology data of mouse neuronal cells. Using cross-validation and independent validation, DeepGAMI outperformed existing methods for classifying disease types, and cellular and clinical phenotypes, even using single modalities (e.g., AUC score of 0.79 for Schizophrenia and 0.73 for cognitive impairment in Alzheimer's disease).

CONCLUSION

We demonstrated that DeepGAMI improves phenotype prediction and prioritizes phenotypic features and networks in multiple multimodal datasets in complex brains and brain diseases. Also, it prioritized disease-associated variants, genes, and regulatory networks linked to different phenotypes, providing novel insights into the interpretation of gene regulatory mechanisms. DeepGAMI is open-source and available for general use.

The Role of the Multi-Inflammatory Index as a Novel Predictor of Hospital Mortality in Acute Ischemic Stroke

Background and objective Ischemic strokes account for the majority of all strokes. The severity of an acute ischemic stroke (AIS) can be estimated with the help of a number of different scoring systems. However, there is a need for bedside tests that will support the clinical diagnosis and thus help predict the severity of stroke. The research on the multi-inflammatory index (MII), which is calculated using hemogram parameters, has shown immense promise. In light of this, the aim of this study was to establish the association between MII and the severity of AIS. Methods The study included 452 ischemic stroke patients over the age of 18 years who presented to the hospital within 72 hours of the onset of symptoms. Demographic information such as patient age and gender, hemogram parameters, ratios, indices, hospitalization, and mortality status were all recorded. The demographic data, hemogram parameters, neutrophil/lymphocyte ratio (NLR), platelet/lymphocyte ratio (PLR), and C-reactive protein (CRP)/lymphocyte ratio (CLR), and MII 1, 2, and 3 were compared between surviving and deceased patients. Results The MII-1, MII-2, and MII-3 index values were determined to be significantly low in the patients with Glasgow Coma Scale (GCS) scores of 13-15 compared to those with GCS scores ≤8, and in patients with National Institutes of Health Stroke Scale (NIHSS) score of 1-4 compared to those with scores of 5-14, 15-20, and ≥21. The NLR, CLR, PLR, MII-1, MII-2, and MII-3 index values were significantly higher in the non-survivors (PLR: p=0.004, all other values: p<0.001). The performances of multiple models developed for the mortality cut-off points were evaluated. Together with other factors, Model 1 included the MII-1, Model 2 the MII-2, and Model 3 the MII-3. Although there was no significant difference between the AUC values of the models, the highest sensitivity rate was reached with Model 2 (74.48%), and the highest specificity rate with Model 3 (90.62%). Conclusion Based on our findings, MII is a simple and practical biomarker that can be easily obtained from NLR, PLR, and CRP, and can help in the early detection of poor prognosis in AIS. NLR was found to be superior to PLR and CLR in distinguishing fatal AIS cases.

LOCC: a novel visualization and scoring of cutoffs for continuous variables

Objective

There is a need for new methods to select and analyze cutoffs employed to define genes that are most prognostic significant and impactful. We designed LOCC (Luo's Optimization Categorization Curve), a novel tool to visualize and score continuous variables for a dichotomous outcome.

Methods

To demonstrate LOCC with real world data, we analyzed TCGA hepatocellular carcinoma gene expression and patient data using LOCC. We compared LOCC visualization to receiver operating characteristic (ROC) curve for prognostic modeling to showcase its utility in understanding predictors in various TCGA datasets.

Results

Analysis of E2F1 expression in hepatocellular carcinoma using LOCC demonstrated appropriate cutoff selection and validation. In addition, we compared LOCC visualization and scoring to ROC curves and c-statistics, demonstrating that LOCC better described predictors. Analysis of a previously published gene signature showed large differences in LOCC scoring, and removing the lowest scoring genes did not affect prognostic modeling of the gene signature demonstrating LOCC scoring could distinguish which predictors were most critical.

Conclusion

Overall, LOCC is a novel visualization tool for understanding and selecting cutoffs, particularly for gene expression analysis in cancer. The LOCC score can be used to rank genes for prognostic potential and is more suitable than ROC curves for prognostic modeling.

Limitations of receiver operating characteristic curve on imbalanced data: Assist device mortality risk scores

OBJECTIVE

In the left ventricular assist device domain, the receiver operating characteristic is a commonly applied metric of performance of classifiers. However, the receiver operating characteristic can provide a distorted view of classifiers' ability to predict short-term mortality due to the overwhelmingly greater proportion of patients who survive, that is, imbalanced data. This study illustrates the ambiguity of the receiver operating characteristic in evaluating 2 classifiers of 90-day left ventricular assist device mortality and introduces the precision recall curve as a supplemental metric that is more representative of left ventricular assist device classifiers in predicting the minority class.

METHODS

This study compared the receiver operating characteristic and precision recall curve for 2 classifiers for 90-day left ventricular assist device mortality, HeartMate Risk Score and Random Forest for 800 patients (test group) recorded in the Interagency Registry for Mechanically Assisted Circulatory Support who received a continuous-flow left ventricular assist device between 2006 and 2016 (mean age, 59 years; 146 female vs 654 male patients), in whom 90-day mortality rate is only 8%.

RESULTS

The receiver operating characteristic indicates similar performance of Random Forest and HeartMate Risk Score classifiers with respect to area under the curve of 0.77 and Random Forest 0.63, respectively. This is in contrast to their precision recall curve with area under the curve of 0.43 versus 0.16 for Random Forest and HeartMate Risk Score, respectively. The precision recall curve for HeartMate Risk Score showed the precision rapidly decreased to only 10% with slightly increasing sensitivity.

CONCLUSIONS

The receiver operating characteristic can portray an overly optimistic performance of a classifier or risk score when applied to imbalanced data. The precision recall curve provides better insight about the performance of a classifier by focusing on the minority class.

The Matthews correlation coefficient (MCC) should replace the ROC AUC as the standard metric for assessing binary classification

Binary classification is a common task for which machine learning and computational statistics are used, and the area under the receiver operating characteristic curve (ROC AUC) has become the common standard metric to evaluate binary classifications in most scientific fields. The ROC curve has true positive rate (also called sensitivity or recall) on the y axis and false positive rate on the x axis, and the ROC AUC can range from 0 (worst result) to 1 (perfect result). The ROC AUC, however, has several flaws and drawbacks. This score is generated including predictions that obtained insufficient sensitivity and specificity, and moreover it does not say anything about positive predictive value (also known as precision) nor negative predictive value (NPV) obtained by the classifier, therefore potentially generating inflated overoptimistic results. Since it is common to include ROC AUC alone without precision and negative predictive value, a researcher might erroneously conclude that their classification was successful. Furthermore, a given point in the ROC space does not identify a single confusion matrix nor a group of matrices sharing the same MCC value. Indeed, a given (sensitivity, specificity) pair can cover a broad MCC range, which casts doubts on the reliability of ROC AUC as a performance measure. In contrast, the Matthews correlation coefficient (MCC) generates a high score in its [Formula: see text] interval only if the classifier scored a high value for all the four basic rates of the confusion matrix: sensitivity, specificity, precision, and negative predictive value. A high MCC (for example, MCC [Formula: see text] 0.9), moreover, always corresponds to a high ROC AUC, and not vice versa. In this short study, we explain why the Matthews correlation coefficient should replace the ROC AUC as standard statistic in all the scientific studies involving a binary classification, in all scientific fields.

Construct validity of the Single Arm Military Press (SAMP) test for upper limb function in patients with neck pain

BACKGROUND

Neck pain (NP) is often associated with upper limb disability (ULD). A clinically feasible measure to evaluate ULD in NP patients is necessary. The Single Arm Military Press (SAMP) is a ULD performance-based measure, specifically for NP patients. The validity of the SAMP in patients is still unknown.

OBJECTIVE

To explore the construct validity (hypotheses testing) of the SAMP in NP patients.

METHODS

A total of 210 NP patients and 81 controls were recruited. The SAMP; Disability of the Arm, Shoulder, and Hand (DASH); and Neck Disability Index (NDI) were completed at baseline and 4-7 days later. The Visual Analogue Scale (VAS) was used to measure NP and ULD severity pre-testing in both occasions. Patients were divided into 4-subgroups based on their NDI score. Convergent validity was examined using Pearson correlation. The t-test and analysis of variance (ANOVA) were used to evaluate discriminant and known groups' validity. To determine SAMP cut-off scores, the sensitivity and specificity were explored.

RESULTS

A negative correlation between the SAMP and DASH/NDI scores was found in the patient group (r = -0.91 and -0.87, p < 0.01). The t-test revealed substantial differences (t = -23.96) in score between patients and controls. Also, ANOVA revealed substantial differences (f = 20.86) between the patients' subgroups. The area under the curve (AUC) for patients and controls exceeded 0.90 when sensitivity and specificity were at equal importance.

CONCLUSION

The SAMP can distinguish between NP patients and controls, and between different NP disability levels. The responsiveness of the SAMP needs to be investigated.

PToPI: A Comprehensive Review, Analysis, and Knowledge Representation of Binary Classification Performance Measures/Metrics

Although few performance evaluation instruments have been used conventionally in different machine learning-based classification problem domains, there are numerous ones defined in the literature. This study reviews and describes performance instruments via formally defined novel concepts and clarifies the terminology. The study first highlights the issues in performance evaluation via a survey of 78 mobile-malware classification studies and reviews terminology. Based on three research questions, it proposes novel concepts to identify characteristics, similarities, and differences of instruments that are categorized into 'performance measures' and 'performance metrics' in the classification context for the first time. The concepts reflecting the intrinsic properties of instruments such as canonical form, geometry, duality, complementation, dependency, and leveling, aim to reveal similarities and differences of numerous instruments, such as redundancy and ground-truth versus prediction focuses. As an application of knowledge representation, we introduced a new exploratory table called PToPI (Periodic Table of Performance Instruments) for 29 measures and 28 metrics (69 instruments including variant and parametric ones). Visualizing proposed concepts, PToPI provides a new relational structure for the instruments including graphical, probabilistic, and entropic ones to see their properties and dependencies all in one place. Applications of the exploratory table in six examples from different domains in the literature have shown that PToPI aids overall instrument analysis and selection of the proper performance metrics according to the specific requirements of a classification problem. We expect that the proposed concepts and PToPI will help researchers comprehend and use the instruments and follow a systematic approach to classification performance evaluation and publication.

Chronic stress causes striatal disinhibition mediated by SOM-interneurons in male mice

Chronic stress (CS) is associated with a number of neuropsychiatric disorders, and it may also contribute to or exacerbate motor function. However, the mechanisms by which stress triggers motor symptoms are not fully understood. Here, we report that CS functionally alters dorsomedial striatum (DMS) circuits in male mice, by affecting GABAergic interneuron populations and somatostatin positive (SOM) interneurons in particular. Specifically, we show that CS impairs communication between SOM interneurons and medium spiny neurons, promoting striatal overactivation/disinhibition and increased motor output. Using probabilistic machine learning to analyze animal behavior, we demonstrate that in vivo chemogenetic manipulation of SOM interneurons in DMS modulates motor phenotypes in stressed mice. Altogether, we propose a causal link between dysfunction of striatal SOM interneurons and motor symptoms in models of chronic stress.

Identifying patients at risk of unplanned re-hospitalisation using statewide electronic health records

Preventing unplanned hospitalisations, including readmissions and re-presentations to the emergency department, is an important strategy for addressing the growing demand for hospital care. Significant successes have been reported from interventions put in place by hospitals to reduce their incidence. However, there is limited use of data-driven algorithms in hospital services to identify patients for enrolment into these intervention programs. Here we present the results of a study aiming to develop algorithms deployable at scale as part of a state government’s initiative to address rehospitalizations and which fills several gaps identified in the state-of-the-art literature. To the best of our knowledge, our study involves the largest-ever sample size for developing risk models. Logistic regression, random forests and gradient boosted techniques were explored as model candidates and validated retrospectively on five years of data from 27 hospitals in Queensland, Australia. The models used a range of predictor variables sourced from state-wide Emergency Department(ED), inpatient, hospital-dispensed medications and hospital-requested pathology databases. The investigation leads to several findings: (i) the advantage of looking at a longer patient data history, (ii) ED and inpatient datasets alone can provide useful information for predicting hospitalisation risk and the addition of medications and pathology test results leads to trivial performance improvements, (iii) predicting readmissions to the hospital was slightly easier than predicting re-presentations to ED after an inpatient stay, which was slightly easier again than predicting re-presentations to ED after an EDstay, (iv) a gradient boosted approach (XGBoost) was systematically the most powerful modelling approach across various tests.

Explainable machine learning for real-time deterioration alert prediction to guide pre-emptive treatment

The Electronic Medical Record (EMR) provides an opportunity to manage patient care efficiently and accurately. This includes clinical decision support tools for the timely identification of adverse events or acute illnesses preceded by deterioration. This paper presents a machine learning-driven tool developed using real-time EMR data for identifying patients at high risk of reaching critical conditions that may demand immediate interventions. This tool provides a pre-emptive solution that can help busy clinicians to prioritize their efforts while evaluating the individual patient risk of deterioration. The tool also provides visualized explanation of the main contributing factors to its decisions, which can guide the choice of intervention. When applied to a test cohort of 18,648 patient records, the tool achieved 100% sensitivity for prediction windows 2–8 h in advance for patients that were identified at 95%, 85% and 70% risk of deterioration.

N6-Methyladenosine-Related Long Non-Coding RNAs Are Identified as a Potential Prognostic Biomarker for Lung Squamous Cell Carcinoma and Validated by Real-Time PCR

Currently, the precise mechanism by which N6-methyladenosine (m6A) modification of long non-coding RNAs (lncRNAs) promotes the occurrence and development of lung squamous cell carcinoma (LUSC) and influences tumor microenvironment (TME) remains unclear. Therefore, we studied the prognostic value of m6A-related lncRNAs and their relationship with TME in 495 LUSC samples from The Cancer Genome Atlas (TCGA) database. Pearson’s correlation and univariate Cox regression analysis identified 6 m6A-related lncRNAs with prognostic values for LUSC patients. LUSC patients were divided into two subgroups (clusters 1 and 2) using principal component analysis. The expression of PD-L1 was lower in tumor tissues and cluster 2 of LUSC patients. Cluster 2 of LUSC patients had a high immune score, stromal score, and unique immune cell infiltration. The focal adhesion kinase (FAK) pathway and cytokine receptor pathways are enriched in cluster 1. The m6A-related lncRNA prognostic markers (m6A-LPMs) were established using the least absolute shrinkage and selection operator (LASSO) Cox regression analysis. The risk score was calculated by 4 m6A-LPMs and associated with OS, TME, clinicopathological characteristics of LUSC patients. After adjusting for age, gender, and stage, the risk score was also an independent prognostic factor for LUSC patients. Real-time PCR results showed that the expression of 4 m6A-LPMs was consistent with our prediction results. Our study found that 4 m6A-LPMs (AC138035.1, AC243919.2, HORMAD2-AS1, and AL122125.1) are closely associated with LUSC prognosis, in future, they may as novel diagnostic biomarkers for LUSC and provide new immunotherapy targets for LUSC patients.

Performance metric curve analysis framework to assess impact of the decision variable threshold, disease prevalence, and dataset variability in two-class classification

Purpose: The aim of this study is to (1) demonstrate a graphical method and interpretation framework to extend performance evaluation beyond receiver operating characteristic curve analysis and (2) assess the impact of disease prevalence and variability in training and testing sets, particularly when a specific operating point is used. Approach: The proposed performance metric curves (PMCs) simultaneously assess sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV), and the 95% confidence intervals thereof, as a function of the threshold for the decision variable. We investigated the utility of PMCs using six example operating points associated with commonly used methods to select operating points (including the Youden index and maximum mutual information). As an example, we applied PMCs to the task of distinguishing between malignant and benign breast lesions using human-engineered radiomic features extracted from dynamic contrast-enhanced magnetic resonance images. The dataset had 1885 lesions, with the images acquired in 2015 and 2016 serving as the training set (1450 lesions) and those acquired in 2017 as the test set (435 lesions). Our study used this dataset in two ways: (1) the clinical dataset itself and (2) simulated datasets with features based on the clinical set but with five different disease prevalences. The median and 95% CI of the number of type I (false positive) and type II (false negative) errors were determined for each operating point of interest. Results: PMCs from both the clinical and simulated datasets demonstrated that PMCs could support interpretation of the impact of decision threshold choice on type I and type II errors of classification, particularly relevant to prevalence. Conclusion: PMCs allow simultaneous evaluation of the four performance metrics of sensitivity, specificity, PPV, and NPV as a function of the decision threshold. This may create a better understanding of two-class classifier performance in machine learning.

A real use case of semi-supervised learning for mammogram classification in a local clinic of Costa Rica

The implementation of deep learning-based computer-aided diagnosis systems for the classification of mammogram images can help in improving the accuracy, reliability, and cost of diagnosing patients. However, training a deep learning model requires a considerable amount of labelled images, which can be expensive to obtain as time and effort from clinical practitioners are required. To address this, a number of publicly available datasets have been built with data from different hospitals and clinics, which can be used to pre-train the model. However, using models trained on these datasets for later transfer learning and model fine-tuning with images sampled from a different hospital or clinic might result in lower performance. This is due to the distribution mismatch of the datasets, which include different patient populations and image acquisition protocols. In this work, a real-world scenario is evaluated where a novel target dataset sampled from a private Costa Rican clinic is used, with few labels and heavily imbalanced data. The use of two popular and publicly available datasets (INbreast and CBIS-DDSM) as source data, to train and test the models on the novel target dataset, is evaluated. A common approach to further improve the model’s performance under such small labelled target dataset setting is data augmentation. However, often cheaper unlabelled data is available from the target clinic. Therefore, semi-supervised deep learning, which leverages both labelled and unlabelled data, can be used in such conditions. In this work, we evaluate the semi-supervised deep learning approach known as MixMatch, to take advantage of unlabelled data from the target dataset, for whole mammogram image classification. We compare the usage of semi-supervised learning on its own, and combined with transfer learning (from a source mammogram dataset) with data augmentation, as also against regular supervised learning with transfer learning and data augmentation from source datasets. It is shown that the use of a semi-supervised deep learning combined with transfer learning and data augmentation can provide a meaningful advantage when using scarce labelled observations. Also, we found a strong influence of the source dataset, which suggests a more data-centric approach needed to tackle the challenge of scarcely labelled data. We used several different metrics to assess the performance gain of using semi-supervised learning, when dealing with very imbalanced test datasets (such as the G-mean and the F2-score), as mammogram datasets are often very imbalanced.

Graphical Abstract

Causal reasoning over knowledge graphs leveraging drug-perturbed and disease-specific transcriptomic signatures for drug discovery

Network-based approaches are becoming increasingly popular for drug discovery as they provide a systems-level overview of the mechanisms underlying disease pathophysiology. They have demonstrated significant early promise over other methods of biological data representation, such as in target discovery, side effect prediction and drug repurposing. In parallel, an explosion of -omics data for the deep characterization of biological systems routinely uncovers molecular signatures of disease for similar applications. Here, we present RPath, a novel algorithm that prioritizes drugs for a given disease by reasoning over causal paths in a knowledge graph (KG), guided by both drug-perturbed as well as disease-specific transcriptomic signatures. First, our approach identifies the causal paths that connect a drug to a particular disease. Next, it reasons over these paths to identify those that correlate with the transcriptional signatures observed in a drug-perturbation experiment, and anti-correlate to signatures observed in the disease of interest. The paths which match this signature profile are then proposed to represent the mechanism of action of the drug. We demonstrate how RPath consistently prioritizes clinically investigated drug-disease pairs on multiple datasets and KGs, achieving better performance over other similar methodologies. Furthermore, we present two case studies showing how one can deconvolute the predictions made by RPath as well as predict novel targets.

A Comparative Analysis of Novel Biomarkers in Sepsis and Cardiovascular Disease

(1) Background: Sepsis still represents a major health care challenge, with mortality rates exceeding 25% in the western world. To further improve outcomes in this patient collective, new cardiovascular biomarkers present a promising opportunity as they target the paramount prognostic processes in sepsis: inflammation and ischemia. However, in contrast to cardiovascular diseases, a detailed analysis of novel biomarkers in sepsis is still lacking. (2) Objective: In this project, we aimed to perform a comparative analysis of biomarker levels in ischemic cardiovascular disease and sepsis. Analyzed markers comprised soluble suppression of tumorigenicity 2 (sST2; hemodynamics and inflammation), growth-differentiation factor 15 (GDF-15; injury, remodelling), soluble urokinase-type plasminogen activator receptor (suPAR; inflammation and remodeling) and heart-type fatty acid binding protein (H-FABP; myocardial ischemia). (3) Methods: In total, 311 patients were included in the study: 123 heart-failure (HF) patients, 60 patients with ST-segment elevation myocardial infarction (STEMI) and 53 sepsis patients. A total of 75 patients without coronary artery disease or signs of heart failure served as a control group. Plasma samples were analyzed by use of ELISA after informed consent. (4) Results: Patients with sepsis showed significantly increased plasma levels in all tested biomarkers compared to cardiovascular disease entities (sST2, suPAR, GDF-15: p < 0.001; H-FABP: compared to HF p < 0.001) and controls (sST2: 7.4-fold, suPAR: 3.4-fold, GDF-15: 6.5-fold and H-FABP: 15.3-fold increased plasma levels, p < 0.001). Moreover, in patients with sepsis, serum concentrations of sST2 and suPAR were significantly elevated in patients with HF and patients with STEMI (sST2: HF: 1.6-fold increase and STEMI: 2.5-fold increase, p < 0.001; suPAR: HF: 1.4-fold increase, p < 0.001 and STEMI: 1.4-fold increase, p < 0.01), whereas plasma levels of GDF-15 and H-FABP were markedly elevated in patients with STEMI only (GDF-15: 1.6-fold increase, H-FABP: 6.4-fold increase, p < 0.001). (5) Conclusions: All tested novel cardiac biomarkers showed significantly elevated levels in sepsis patients. Interestingly, a secretion pattern similar to STEMI was observed with regards to sST2 and HFABP. Thus, by providing an assessment tool especially covering the cardiovascular component of the disease, novel biomarkers offer a promising tool in sepsis patients.

Structure-based discovery and bio-evaluation of a cyclopenta[4,5]thieno[2,3-d]pyrimidin-4-one as a phosphodiesterase 10A inhibitor

Phosphodiesterase10A (PDE10A) is a potential therapeutic target for the treatment of several neurodegenerative disorders. Thus, extensive efforts of medicinal chemists have been directed toward developing potent PDE10A inhibitors with minimal side effects. However, PDE10A inhibitors are not approved as a treatment for neurodegenerative disorders, possibly due to the lack of research in this area. Therefore, the discovery of novel and diverse scaffolds targeting PDE10A is required. In this study, we described the identification of a new PDE10A inhibitor by structure-based virtual screening combining pharmacophore modelling, molecular docking, molecular dynamics simulations, and biological evaluation. Zinc42657360 with a cyclopenta[4,5]thieno[2,3-d]pyrimidin-4-one scaffold from the zinc database exhibited a significant inhibitory activity of 1.60 μM against PDE10A. The modelling studies demonstrated that Zinc42657360 is involved in three hydrogen bonds with ASN226, THR187 and ASP228, and two aromatic interactions with TYR78 and PHE283, besides the common interactions with the P-clamp residues PHE283 and ILE246. The novel scaffold of Zinc42657360 can be used for the rational design of PDE10A inhibitors with improved affinity.

Phosphodiesterase10A (PDE10A) is a potential therapeutic target for the treatment of several neurodegenerative disorders.

Identifying diverse metal oxide nanomaterials with lethal effects on embryonic zebrafish using machine learning

Manufacturers of nanomaterial-enabled products need models of endpoints that are relevant to human safety to support the "safe by design" paradigm and avoid late-stage attrition. Increasingly, embryonic zebrafish (Danio Rerio) are recognised as a key human safety relevant in vivo test system. Hence, machine learning models were developed for identifying metal oxide nanomaterials causing lethality to embryonic zebrafish up to 24 hours post-fertilisation, or excess lethality in the period of 24-120 hours post-fertilisation, at concentrations of 250 ppm or less. Models were developed using data from the Nanomaterial Biological-Interactions Knowledgebase for a dataset of 44 diverse, coated and uncoated metal or, in one case, metalloid oxide nanomaterials. Different modelling approaches were evaluated using nested cross-validation on this dataset. Models were initially developed for both lethality endpoints using multiple descriptors representing the composition of the core, shell and surface functional groups, as well as particle characteristics. However, interestingly, the 24 hours post-fertilisation data were found to be harder to predict, which could reflect different exposure routes. Hence, subsequent analysis focused on the prediction of excess lethality at 120 hours-post fertilisation. The use of two data augmentation approaches, applied for the first time in nano-QSAR research, was explored, yet both failed to boost predictive performance. Interestingly, it was found that comparable results to those originally obtained using multiple descriptors could be obtained using a model based upon a single, simple descriptor: the Pauling electronegativity of the metal atom. Since it is widely recognised that a variety of intrinsic and extrinsic nanomaterial characteristics contribute to their toxicological effects, this is a surprising finding. This may partly reflect the need to investigate more sophisticated descriptors in future studies. Future studies are also required to examine how robust these modelling results are on truly external data, which were not used to select the single descriptor model. This will require further laboratory work to generate comparable data to those studied herein.

Development and Characterization of a Multiplex Assay to Quantify Complement-Fixing Antibodies against Dengue Virus

Antibodies capable of activating the complement system (CS) when bound with antigen are referred to as "complement-fixing antibodies" and are involved in protection against Flaviviruses. A complement-fixing antibody test has been used in the past to measure the ability of dengue virus (DENV)-specific serum antibodies to activate the CS. As originally developed, the test is time-consuming, cumbersome, and has limited sensitivity for DENV diagnosis. Here, we developed and characterized a novel multiplex anti-DENV complement-fixing assay based on the Luminex platform to quantitate serum antibodies against all four serotypes (DENV1-4) that activate the CS based on their ability to fix the complement component 1q (C1q). The assay demonstrated good reproducibility and showed equivalent performance to a DENV microneutralization assay that has been used to determine DENV serostatus. In non-human primates, antibodies produced in response to primary DENV1-4 infection induced C1q fixation on homologous and heterologous serotypes. Inter-serotype cross-reactivity was associated with homology of the envelope protein. Interestingly, the antibodies produced following vaccination against Zika virus fixed C1q on DENV. The anti-DENV complement fixing antibody assay represents an alternative approach to determine the quality of functional antibodies produced following DENV natural infection or vaccination and a biomarker for dengue serostatus, while providing insights about immunological cross-reactivity among different Flaviviruses.

Using predictive machine learning models for drug response simulation by calibrating patient-specific pathway signatures

The utility of pathway signatures lies in their capability to determine whether a specific pathway or biological process is dysregulated in a given patient. These signatures have been widely used in machine learning (ML) methods for a variety of applications including precision medicine, drug repurposing, and drug discovery. In this work, we leverage highly predictive ML models for drug response simulation in individual patients by calibrating the pathway activity scores of disease samples. Using these ML models and an intuitive scoring algorithm to modify the signatures of patients, we evaluate whether a given sample that was formerly classified as diseased, could be predicted as normal following drug treatment simulation. We then use this technique as a proxy for the identification of potential drug candidates. Furthermore, we demonstrate the ability of our methodology to successfully identify approved and clinically investigated drugs for four different cancers, outperforming six comparable state-of-the-art methods. We also show how this approach can deconvolute a drugs' mechanism of action and propose combination therapies. Taken together, our methodology could be promising to support clinical decision-making in personalized medicine by simulating a drugs' effect on a given patient.

Acute myocardial injury secondary to severe acute liver failure: A retrospective analysis supported by animal data

To investigate whether acute liver failure (ALF) leads to secondary acute myocardial injury, 100 ALF patients that were retrospectively identified in a single center based on ICD 10 codes and 8 rats from an experimental study that died early after bile duct ligation (BDL) were examined. Creatine kinase (CK), creatine kinase-MB isoenzyme (CKMB) and cardiac troponin-I (cTnI) were analyzed as markers of myocardial injury. For histological analysis, hematoxylin-eosin (HE), elastic Van Gieson (EVG), CD41 and myeloperoxidase were used to stain rat hearts. Major adverse cardiac events (MACEs) were a critical factor for mortality (p = 0.037) in human ALF. Deceased patients exhibited higher levels of CKMB than survivors (p = 0.023). CKMB was a predictor of mortality in ALF (p = 0.013). Animals that died early after BDL exhibited increased cTnI, CKMB, tumor necrosis factor α (TNFα) and interleukin-6 (IL-6) levels compared to controls (cTnI: p = 0.011, CKMB: p = 0.008, TNFα: p = 0.003, IL-6: p = 0.006). These animals showed perivascular lesions and wavy fibers, microthrombi and neutrophilic infiltration in the heart. MACEs are decisive for mortality in human ALF, and elevated CKMB values indicate that this might be due to structural myocardial damage. Accordingly, CKMB was found to have predictive value for mortality in ALF. The results are substantiated by data from a rat BDL model demonstrating diffuse myocardial injury.

Pathogenic potential assessment of the Shiga toxin-producing Escherichia coli by a source attribution-considered machine learning model

Instead of conventional serotyping and virulence gene combination methods, methods have been developed to evaluate the pathogenic potential of newly emerging pathogens. Among them, the machine learning (ML)-based method using whole-genome sequencing (WGS) data are getting attention because of the recent advances in ML algorithms and sequencing technologies. Here, we developed various ML models to predict the pathogenicity of Shiga toxin-producing Escherichia coli (STEC) isolates using their WGS data. The input dataset for the ML models was generated using distinct gene repertoires from positive (pathogenic) and negative (nonpathogenic) control groups in which each STEC isolate was designated based on the source attribution, the relative risk potential of the isolation sources. Among the various ML models examined, a model using the support vector machine (SVM) algorithm, the SVM model, discriminated between the two control groups most accurately. The SVM model successfully predicted the pathogenicity of the isolates from the major sources of STEC outbreaks, the isolates with the history of outbreaks, and the isolates that cannot be assessed by conventional methods. Furthermore, the SVM model effectively differentiated the pathogenic potentials of the isolates at a finer resolution. Permutation importance analyses of the input dataset further revealed the genes important for the estimation, proposing the genes potentially essential for the pathogenicity of STEC. Altogether, these results suggest that the SVM model is a more reliable and broadly applicable method to evaluate the pathogenic potential of STEC isolates compared with conventional methods.

Usefulness of postural sway spectral analysis in the diagnostic route and clinical integration of cervicogenic and vestibular sources of dizziness: A cross-sectional preliminary study

BACKGROUND

Posturography power spectra (PS) implementation has been proven to discriminate between sensory inputs detriment of vestibular and proprioceptive origin.

OBJECTIVE

To deepen the role of posturography testing in the diagnostic route of dizzy conditions, by comparing two groups of patients -93 affected by cervicogenic dizziness (CGD) and 72 by unilateral vestibular hypofunction (UVH) -with a group of 98 age- and gender-matched healthy subjects, serving as control group (CON).

METHODS

All participants underwent otoneurological testing including video head impulse test (vHIT) and posturography testing with PS analysis. They also filled in Dizziness Handicap Inventory (DHI), Tampa Scale for Kinesiophobia and Hospital Anxiety and Depression Scale questionnaires.

RESULTS

UVH and CGD patients were found to have significant increase in vestibular- and proprioceptive-related PS values when compared with CON. Receiver operating characteristic curves found PS values to reliably discriminate both groups from CON. Positive and negative correlations were respectively found between vestibular-/proprioceptive-related PS domain and DHI in both groups and between PS and vHIT scores in UVH patients.

CONCLUSIONS

PS analysis demonstrated to be useful in differentiating CGD and UVH patients each other and when compared to CON, to objectively represent perceived symptoms filled along the DHI scale and to corroborate the rate of vestibular deficit in UVH patients.

Identification of 14-3-3 epsilon as a regulator of the neural apoptotic pathway for chronic-stress-induced depression

Major depression is a prevalent and long-lasting psychiatric illness with severe functional impairment and high suicide rate. We have previously shown that the ventrolateral orbital cortex (VLO) plays a key role in the stress responses in mice, but the underlying mechanisms remains unclear. Here, we used proteomic method to identify differentially expressed proteins in VLO of chronic unpredictable mild stress (CUMS) mice. Of 4,953 quantified proteins, 45 proteins were differentially expressed following CUMS. The integrated pathway analyses identified 14-3-3ε and TrkB signaling as differentially downregulated in association with stress-induced depressive-like behaviors. 14-3-3ε overexpression in VLO relieved the depressive-like behaviors by rescue of Bad-mediated apoptosis. Moreover, treatment with the 14-3-3ε stabilizer FC-A precluded neuronal apoptotic signaling in VLO of depressed mice. Because 14-3-3ε provides significant protection against chronic stress, boosting 14-3-3ε expression, pharmacological stabilization of 14-3-3s (e.g. with FC-A) is identified as an exciting therapeutic target for major depression.

•

Novel screening of chronic mild stress-induced depression phenotypes in mice

•

Proteomics identify 14-3-3ε as a key modulator of depressive behaviors in VLO

•

14-3-3ε partially reversed depressive behaviors through neural apoptotic pathway

•

14-3-3ε stabilizer FC-A ameliorates depression phenotypes after chronic mild stress

Immune variations throughout the course of tuberculosis treatment and its relationship with adrenal hormone changes in HIV-1 patients co-infected with Mycobacterium tuberculosis

HIV infection is a major risk factor predisposing for Mycobacterium tuberculosis infection and progression to active tuberculosis (TB). As host immune response defines the course of infection, we aimed to identify immuno-endocrine changes over six-months of anti-TB chemotherapy in HIV+ people. Plasma levels of cortisol, DHEA and DHEA-S, percentages of CD4⁺ regulatory T cell subsets and number of IFN-γ-secreting cells were determined. Several cytokines, chemokines and C-reactive protein levels were measured. Results were correlated with clinical parameters as predictors of infection resolution and compared to similar data from HIV+ individuals, HIV-infected persons with latent TB infection and healthy donors. Throughout the course of anti-TB/HIV treatment, DHEA and DHEA-S plasma levels raised while cortisol diminished, which correlated to predictive factors of infection resolution. Furthermore, the balance between cortisol and DHEA, together with clinical assessment, may be considered as an indicator of clinical outcome after anti-TB treatment in HIV+ individuals. Clinical improvement was associated with reduced frequency of unconventional Tregs, increment in IFN-γ-secreting cells, diminution of systemic inflammation and changes of circulating cytokines and chemokines. This study suggests that the combined anti-HIV/TB therapies result in partial restoration of both, immune function and adrenal hormone plasma levels.

Drug2ways: Reasoning over causal paths in biological networks for drug discovery

Elucidating the causal mechanisms responsible for disease can reveal potential therapeutic targets for pharmacological intervention and, accordingly, guide drug repositioning and discovery. In essence, the topology of a network can reveal the impact a drug candidate may have on a given biological state, leading the way for enhanced disease characterization and the design of advanced therapies. Network-based approaches, in particular, are highly suited for these purposes as they hold the capacity to identify the molecular mechanisms underlying disease. Here, we present drug2ways, a novel methodology that leverages multimodal causal networks for predicting drug candidates. Drug2ways implements an efficient algorithm which reasons over causal paths in large-scale biological networks to propose drug candidates for a given disease. We validate our approach using clinical trial information and demonstrate how drug2ways can be used for multiple applications to identify: i) single-target drug candidates, ii) candidates with polypharmacological properties that can optimize multiple targets, and iii) candidates for combination therapy. Finally, we make drug2ways available to the scientific community as a Python package that enables conducting these applications on multiple standard network formats.

A Comparison of T2 Relaxation-Based MRI Stroke Timing Methods in Hyperacute Ischemic Stroke Patients: A Pilot Study

Background:

T₂ relaxation-based magnetic resonance imaging (MRI) signals may provide onset time for acute ischemic strokes with an unknown onset. The ability of visual and quantitative MRI-based methods in a cohort of hyperacute ischemic stroke patients was studied.

Methods:

A total of 35 patients underwent 3T (3 Tesla) MRI (<9-hour symptom onset). Diffusion-weighted (DWI), apparent diffusion coefficient (ADC), T₁-weighted (T₁w), T₂-weighted (T₂w), and T₂ relaxation time (T₂) images were acquired. T₂-weighted fluid attenuation inversion recovery (FLAIR) images were acquired for 17 of these patients. Image intensity ratios of the average intensities in ischemic and non-ischemic reference regions were calculated for ADC, DWI, T₂w, T₂ relaxation, and FLAIR images, and optimal image intensity ratio cut-offs were determined. DWI and FLAIR images were assessed visually for DWI/FLAIR mismatch.

Results:

The T₂ relaxation time image intensity ratio was the only parameter with significant correlation with stroke duration (r = 0.49, P = .003), an area under the receiver operating characteristic curve (AUC = 0.77, P < .0001), and an optimal cut-off (T₂ ratio = 1.072) that accurately identified patients within the 4.5-hour thrombolysis treatment window with sensitivity of 0.74 and specificity of 0.74. In the patients with the additional FLAIR, areas under the precision-recall-gain curve (AUPRG) and F₁ scores showed that the T₂ relaxation time ratio (AUPRG = 0.60, F₁ = 0.73) performed considerably better than the FLAIR ratio (AUPRG = 0.39, F₁ = 0.57) and the visual DWI/FLAIR mismatch (F₁ = 0.25).

Conclusions:

Quantitative T₂ relaxation time is the preferred MRI parameter in the assessment of patients with unknown onset for treatment stratification.

Pediatric contact allergy: A comparative study with adults

BACKGROUND

Pediatric allergic contact dermatitis is increasing. The patch test allergens included in pediatric baseline series vary globally. The worldwide prevalence of pediatric reactions to allergens needs clarification.

OBJECTIVES

Identify the prevalence, associated factors, and culprit allergens for contact allergy among patch-tested Thai children, and compare with those for adults.

METHODS

Baseline series patch test results from 2010-2019 were collected for patients younger than 18 years of age. As a control group, sex-matched adult patients were randomly selected. The results and characteristics of the two groups were compared.

RESULTS

The median age of 112 patch tested pediatric patients was 16 (range 2-17) years. Of the children, 35.5% had at least one positive reaction, significantly less than the 56.6% for adults. The five most common pediatric allergens were nickel sulfate (12.1%), potassium dichromate (8.0%), methylisothiazolinone (7.1%), fragrance mix II (6.0%), and carba mix (5.4%). Although similar, the 10 most common allergens of the groups differed in order. Positive reactions to cosmetic allergens were significantly less frequent among the children. Many allergens remained entirely negative.

CONCLUSIONS

The prevalence of positive reactions was lower in children, varying by population and region. The top-10 pediatric and adult causative allergens were almost identical. We recommend using the same baseline patch test series for children and adults in our region.

A Somatic Coliphage Threshold Approach To Improve the Management of Activated Sludge Wastewater Treatment Plant Effluents in Resource-Limited Regions

Effective wastewater management is crucial to ensure the safety of water reuse projects and effluent discharge into surface waters. Multiple studies have demonstrated that municipal wastewater treatment with conventional activated sludge processes is inefficient for the removal of a wide spectrum of viruses in sewage. In this study, a well-accepted statistical approach was used to investigate the relationship between viral indicators and human enteric viruses during wastewater treatment in a resource-limited region. Influent and effluent samples from five urban wastewater treatment plants (WWTPs) in Costa Rica were analyzed for somatic coliphage and human enterovirus, hepatitis A virus, norovirus genotypes I and II, and rotavirus. All WWTPs provide primary treatment followed by conventional activated sludge treatment prior to discharge into surface waters that are indirectly used for agricultural irrigation. The results revealed a statistically significant relationship between the detection of at least one of the five human enteric viruses and somatic coliphage. Multiple logistic regression and receiver operating characteristic curve analysis identified a threshold of 3.0 × 103 (3.5 log10) somatic coliphage PFU per 100 ml, which corresponded to an increased likelihood of encountering enteric viruses above the limit of detection (>1.83 × 102 virus targets/100 ml). Additionally, quantitative microbial risk assessment was executed for farmers indirectly reusing WWTP effluent that met the proposed threshold. The resulting estimated median cumulative annual disease burden complied with World Health Organization recommendations. Future studies are needed to validate the proposed threshold for use in Costa Rica and other regions.IMPORTANCE Effective wastewater management is crucial to ensure safe direct and indirect water reuse; nevertheless, few countries have adopted the virus log reduction value management approach established by the World Health Organization. In this study, we investigated an alternative and/or complementary approach to the virus log reduction value framework for the indirect reuse of activated sludge-treated wastewater effluent. Specifically, we employed a well-accepted statistical approach to identify a statistically sound somatic coliphage threshold value which corresponded to an increased likelihood of human enteric virus detection. This study demonstrates an alternative approach to the virus log reduction value framework which can be applied to improve wastewater reuse practices and effluent management.

Evaluation of an optimal cutoff of parathyroid venous sampling gradient for localizing primary hyperparathyroidism

INTRODUCTION

Parathyroid venous sampling (PVS) has been reported to be a useful adjunctive test in localizing lesions in elusive cases of primary hyperparathyroidism (PHPT). Conventional cutoff (twofold) is now widely being used, but optimal cutoff threshold for PVS gradient based on discriminatory performance remains unclear.

MATERIALS AND METHODS

Among a total of 197 consecutive patients (mean age 58.2 years, female 74.6%) with PHPT who underwent parathyroidectomy at a tertiary center between 2012 and 2018, we retrospectively analyzed 59 subjects who underwent PVS for persistent or recurrent disease after previous parathyroidectomy, or for equivocal or negative results from conventional imaging modalities including ultrasonography (US) and Tc^99m-Sestamibi SPECT-CT (MIBI). True parathyroid lesions were confirmed by combination of surgical, pathological findings, and intraoperative parathyroid hormone (PTH) changes. Optimal PVS cutoff were determined by receiver-operating characteristics (ROC) analysis with Youden and Liu method.

RESULTS

Compared to subjects who did not require PVS, PVS group tends to have lower PTH (119.8 pg/mL vs 133.7 pg/mL, p = 0.075). A total of 79 culprit parathyroid lesions (left 40; right 39) from 59 patients (left 24; right 26; bilateral 9) were confirmed by surgery. The optimal cutoff for PVS gradient was estimated as 1.5-fold gradient (1.5 ×) with sensitivity of 61.8% and specificity of 84%. When 1.5 × cutoff was applied, PVS improved the discrimination for true parathyroid lesions substantially based on area under ROC (0.892 to 0.942, p < 0.001) when added to US and MIBI.

CONCLUSION

Our findings suggest that PVS with cutoff threshold 1.5 × can provide useful complementary information for pre-operative localization in selected cases.

Effects of Study Population, Labeling and Training on Glaucoma Detection Using Deep Learning Algorithms

Purpose

To compare performance of independently developed deep learning algorithms for detecting glaucoma from fundus photographs and to evaluate strategies for incorporating new data into models.

Methods

Two fundus photograph datasets from the Diagnostic Innovations in Glaucoma Study/African Descent and Glaucoma Evaluation Study and Matsue Red Cross Hospital were used to independently develop deep learning algorithms for detection of glaucoma at the University of California, San Diego, and the University of Tokyo. We compared three versions of the University of California, San Diego, and University of Tokyo models: original (no retraining), sequential (retraining only on new data), and combined (training on combined data). Independent datasets were used to test the algorithms.

Results

The original University of California, San Diego and University of Tokyo models performed similarly (area under the receiver operating characteristic curve = 0.96 and 0.97, respectively) for detection of glaucoma in the Matsue Red Cross Hospital dataset, but not the Diagnostic Innovations in Glaucoma Study/African Descent and Glaucoma Evaluation Study data (0.79 and 0.92; P < .001), respectively. Model performance was higher when classifying moderate-to-severe compared with mild disease (area under the receiver operating characteristic curve = 0.98 and 0.91; P < .001), respectively. Models trained with the combined strategy generally had better performance across all datasets than the original strategy.

Conclusions

Deep learning glaucoma detection can achieve high accuracy across diverse datasets with appropriate training strategies. Because model performance was influenced by the severity of disease, labeling, training strategies, and population characteristics, reporting accuracy stratified by relevant covariates is important for cross study comparisons.

Translational Relevance

High sensitivity and specificity of deep learning algorithms for moderate-to-severe glaucoma across diverse populations suggest a role for artificial intelligence in the detection of glaucoma in primary care.

Sighting acute myocardial infarction through platelet gene expression

Acute myocardial infarction is primarily due to coronary atherosclerotic plaque rupture and subsequent thrombus formation. Platelets play a key role in the genesis and progression of both atherosclerosis and thrombosis. Since platelets are anuclear cells that inherit their mRNA from megakaryocyte precursors and maintain it unchanged during their life span, gene expression profiling at the time of an acute myocardial infarction provides information concerning the platelet gene expression preceding the coronary event. In ST-segment elevation myocardial infarction (STEMI), a gene-by-gene analysis of the platelet gene expression identified five differentially expressed genes: FKBP5, S100P, SAMSN1, CLEC4E and S100A12. The logistic regression model used to combine the gene expression in a STEMI vs healthy donors score showed an AUC of 0.95. The same five differentially expressed genes were externally validated using platelet gene expression data from patients with coronary atherosclerosis but without thrombosis. Platelet gene expression profile highlights five genes able to identify STEMI patients and to discriminate them in the background of atherosclerosis. Consequently, early signals of an imminent acute myocardial infarction are likely to be found by platelet gene expression profiling before the infarction occurs.

Chronic Stress Induces Activity, Synaptic, and Transcriptional Remodeling of the Lateral Habenula Associated with Deficits in Motivated Behaviors

Chronic stress (CS) is a major risk factor for the development of depression. Here, we demonstrate that CS-induced hyperactivity in ventral tegmental area (VTA)-projecting lateral habenula (LHb) neurons is associated with increased passive coping (PC), but not anxiety or anhedonia. LHb→VTA neurons in mice with increased PC show increased burst and tonic firing as well as synaptic adaptations in excitatory inputs from the entopeduncular nucleus (EP). In vivo manipulations of EP→LHb or LHb→VTA neurons selectively alter PC and effort-related motivation. Conversely, dorsal raphe (DR)-projecting LHb neurons do not show CS-induced hyperactivity and are targeted indirectly by the EP. Using single-cell transcriptomics, we reveal a set of genes that can collectively serve as biomarkers to identify mice with increased PC and differentiate LHb→VTA from LHb→DR neurons. Together, we provide a set of biological markers at the level of genes, synapses, cells, and circuits that define a distinctive CS-induced behavioral phenotype.

Rethinking Drug Repositioning and Development with Artificial Intelligence, Machine Learning, and Omics

Pharmaceutical industry and the art and science of drug development are sorely in need of novel transformative technologies in the current age of digital health and artificial intelligence (AI). Often described as game-changing technologies, AI and machine learning algorithms have slowly but surely begun to revolutionize pharmaceutical industry and drug development over the past 5 years. In this expert review, we describe the most frequently used machine learning algorithms in drug development pipelines and the -omics databases well poised to support machine learning and drug discovery. Subsequently, we analyze the emerging new computational approaches to drug discovery and the in silico pipelines for drug repositioning and the synergies among -omics system sciences, AI and machine learning. As with system sciences, AI and machine learning embody a system scale and Big Data driven vision for drug discovery and development. We conclude with a future outlook on the ways in which machine learning approaches can be implemented to buttress and expedite drug discovery and precision medicine. As AI and machine learning are rapidly entering pharmaceutical industry and the art and science of drug development, we need to critically examine the attendant prospects and challenges to benefit patients and public health.

Patient clustering improves efficiency of federated machine learning to predict mortality and hospital stay time using distributed electronic medical records

Electronic medical records (EMRs) support the development of machine learning algorithms for predicting disease incidence, patient response to treatment, and other healthcare events. But so far most algorithms have been centralized, taking little account of the decentralized, non-identically independently distributed (non-IID), and privacy-sensitive characteristics of EMRs that can complicate data collection, sharing and learning. To address this challenge, we introduced a community-based federated machine learning (CBFL) algorithm and evaluated it on non-IID ICU EMRs. Our algorithm clustered the distributed data into clinically meaningful communities that captured similar diagnoses and geographical locations, and learnt one model for each community. Throughout the learning process, the data was kept local at hospitals, while locally-computed results were aggregated on a server. Evaluation results show that CBFL outperformed the baseline federated machine learning (FL) algorithm in terms of Area Under the Receiver Operating Characteristic Curve (ROC AUC), Area Under the Precision-Recall Curve (PR AUC), and communication cost between hospitals and the server. Furthermore, communities' performance difference could be explained by how dissimilar one community was to others.

MicroRNAs in porcine uterus and serum are affected by zearalenone and represent a new target for mycotoxin biomarker discovery

The mycotoxin zearalenone (ZEN) poses a risk to animal health because of its estrogenic effects. Diagnosis of ZEN-induced disorders remains challenging due to the lack of appropriate biomarkers. In this regard, circulating microRNAs (small non-coding RNAs) have remarkable potential, as they can serve as indicators for pathological processes in tissue. Thus, we combined untargeted and targeted transcriptomics approaches to investigate the effects of ZEN on the microRNA expression in porcine uterus, jejunum and serum, respectively. To this end, twenty-four piglets received uncontaminated feed (Control) or feed containing 0.17 mg/kg ZEN (ZEN low), 1.46 mg/kg ZEN (ZEN medium) and 4.58 mg/kg ZEN (ZEN high). After 28 days, the microRNA expression in the jejunum remained unaffected, while significant changes in the uterine microRNA profile were observed. Importantly, 14 microRNAs were commonly and dose-dependently affected in both the ZEN medium and ZEN high group, including microRNAs from the miR-503 cluster (i.e. ssc-miR-424-5p, ssc-miR-450a, ssc-miR-450b-5p, ssc-miR-450c-5p, ssc-miR-503 and ssc-miR-542-3p). Predicted target genes for those microRNAs are associated with regulation of gene expression and signal transduction (e.g. cell cycle). Although the effects in serum were less pronounced, receiver operating characteristic analysis revealed that several microRNA ratios were able to discriminate properly between non-exposed and ZEN-exposed pigs (e.g. ssc-miR-135a-5p/ssc-miR-432-5p, ssc-miR-542-3p/ssc-miR-493-3p). This work sheds new light on the molecular mechanisms of ZEN, and fosters biomarker discovery.

A urinary microRNA panel that is an early predictive biomarker of delayed graft function following kidney transplantation

Predicting immediate and subsequent graft function is important in clinical decision-making around kidney transplantation, but is difficult using available approaches. Here we have evaluated urinary microRNAs as biomarkers in this context. Profiling of 377 microRNAs in the first urine passed post-transplantation identified 6 microRNAs, confirmed to be upregulated by RT-qPCR in an expanded cohort (miR-9, -10a, -21, -29a, -221, and -429, n = 33, P < 0.05 for each). Receiver operating characteristic analysis showed Area Under the Curve 0.94 for this panel. To establish whether this early signal was sustained, miR-21 was measured daily for 5 days post-transplant, and was consistently elevated in those developing Delayed Graft Function (n = 165 samples from 33 patients, p < 0.05). The biomarker panel was then evaluated in an independent cohort, sampled at varying times in the first week post-transplantation in a separate transplant center. When considered individually, all miRs in the panel showed a trend to increase or a significant increase in those developing delayed Graft Function (miR-9: P = 0.068, mIR-10a: P = 0.397, miR-21: P = 0.003, miR-29a: P = 0.019, miR-221: P = 0.1, and miR-429: P = 0.013, n = 47) with Area Under the Curve 0.75 for the panel. In conclusion, combined measurement of six microRNAs had predictive value for delayed graft function following kidney transplantation.