The central role of receiver operating characteristic (ROC) curves in evaluating tests for the early detection of cancer

Identification of Gαi3 as a promising molecular oncotarget of pancreatic cancer

The increasing mortality rate of pancreatic cancer globally necessitates the urgent identification for novel therapeutic targets. This study investigated the expression, functions, and mechanistic insight of G protein inhibitory subunit 3 (Gαi3) in pancreatic cancer. Bioinformatics analyses reveal that Gαi3 is overexpressed in human pancreatic cancer, correlating with poor prognosis, higher tumor grade, and advanced classification. Elevated Gαi3 levels are also confirmed in human pancreatic cancer tissues and primary/immortalized cancer cells. Gαi3 shRNA or knockout (KO) significantly reduced cell viability, proliferation, cell cycle progression, and mobility in primary/immortalized pancreatic cancer cells. Conversely, Gαi3 overexpression enhanced pancreatic cancer cell growth. RNA-sequencing and bioinformatics analyses of Gαi3-depleted cells indicated Gαi3's role in modulating the Akt-mTOR and PKA-Hippo-YAP pathways. Akt-S6 phosphorylation was decreased in Gαi3-depleted cells, but was increased with Gαi3 overexpression. Additionally, Gαi3 depletion elevated PKA activity and activated the Hippo pathway kinase LATS1/2, leading to YAP/TAZ inactivation, while Gαi3 overexpression exerted the opposite effects. There is an increased binding between Gαi3 promoter and the transcription factor TCF7L2 in pancreatic cancer tissues and cells. Gαi3 expression was significantly decreased following TCF7L2 silencing, but increased with TCF7L2 overexpression. In vivo, intratumoral injection of Gαi3 shRNA-expressing adeno-associated virus significantly inhibited subcutaneous pancreatic cancer xenografts growth in nude mice. A significant growth reduction was also observed in xenografts from Gαi3 knockout pancreatic cancer cells. Akt-mTOR inactivation and increased PKA activity coupled with YAP/TAZ inactivation were also detected in xenograft tumors upon Gαi3 depletion. Furthermore, bioinformatic analysis and multiplex immunohistochemistry (mIHC) staining on pancreatic cancer tissue microarrays showed a reduced proportion of M1-type macrophages and an increase in PD-L1 positive cells in Gαi3-high pancreatic cancer tissues. Collectively, these findings highlight Gαi3's critical role in promoting pancreatic cancer cell growth, potentially through the modulation of the Akt-mTOR and PKA-Hippo-YAP pathways and its influence on the immune landscape.

Logistic regression and other statistical tools in diagnostic biomarker studies

A biomarker is a measured indicator of a variety of processes, and is often used as a clinical tool for the diagnosis of diseases. While the developmental process of biomarkers from lab to clinic is complex, initial exploratory stages often focus on characterizing the potential of biomarkers through utilizing various statistical methods that can be used to assess their discriminatory performance, establish an appropriate cut-off that transforms continuous data to apt binary responses of confirming or excluding a diagnosis, or establish a robust association when tested against confounders. This review aims to provide a gentle introduction to the most common tools found in diagnostic biomarker studies used to assess the performance of biomarkers with an emphasis on logistic regression.

Development of a Multilevel Model to Identify Patients at Risk for Delay in Starting Cancer Treatment

Importance

Delays in starting cancer treatment disproportionately affect vulnerable populations and can influence patients' experience and outcomes. Machine learning algorithms incorporating electronic health record (EHR) data and neighborhood-level social determinants of health (SDOH) measures may identify at-risk patients.

Objective

To develop and validate a machine learning model for estimating the probability of a treatment delay using multilevel data sources.

Design, Setting, and Participants

This cohort study evaluated 4 different machine learning approaches for estimating the likelihood of a treatment delay greater than 60 days (group least absolute shrinkage and selection operator [LASSO], bayesian additive regression tree, gradient boosting, and random forest). Criteria for selecting between approaches were discrimination, calibration, and interpretability/simplicity. The multilevel data set included clinical, demographic, and neighborhood-level census data derived from the EHR, cancer registry, and American Community Survey. Patients with invasive breast, lung, colorectal, bladder, or kidney cancer diagnosed from 2013 to 2019 and treated at a comprehensive cancer center were included. Data analysis was performed from January 2022 to June 2023.

Exposures

Variables included demographics, cancer characteristics, comorbidities, laboratory values, imaging orders, and neighborhood variables.

Main Outcomes and Measures

The outcome estimated by machine learning models was likelihood of a delay greater than 60 days between cancer diagnosis and treatment initiation. The primary metric used to evaluate model performance was area under the receiver operating characteristic curve (AUC-ROC).

Results

A total of 6409 patients were included (mean [SD] age, 62.8 [12.5] years; 4321 [67.4%] female; 2576 [40.2%] with breast cancer, 1738 [27.1%] with lung cancer, and 1059 [16.5%] with kidney cancer). A total of 1621 (25.3%) experienced a delay greater than 60 days. The selected group LASSO model had an AUC-ROC of 0.713 (95% CI, 0.679-0.745). Lower likelihood of delay was seen with diagnosis at the treating institution; first malignant neoplasm; Asian or Pacific Islander or White race; private insurance; and lacking comorbidities. Greater likelihood of delay was seen at the extremes of neighborhood deprivation. Model performance (AUC-ROC) was lower in Black patients, patients with race and ethnicity other than non-Hispanic White, and those living in the most disadvantaged neighborhoods. Though the model selected neighborhood SDOH variables as contributing variables, performance was similar when fit with and without these variables.

Conclusions and Relevance

In this cohort study, a machine learning model incorporating EHR and SDOH data was able to estimate the likelihood of delays in starting cancer therapy. Future work should focus on additional ways to incorporate SDOH data to improve model performance, particularly in vulnerable populations.

Exact Probability Distribution for the ROC Area under Curve

The Receiver Operating Characteristic (ROC) is a de facto standard for determining the accuracy of in vitro diagnostic (IVD) medical devices, and thus the exactness in its probability distribution is crucial toward accurate statistical inference. We show the exact probability distribution of the ROC AUC-value, hence exact critical values and p-values are readily obtained. Because the exact calculations are computationally intense, we demonstrate a method of geometric interpolation, which is exact in a special case but generally an approximation, vastly increasing computational speeds. The method is illustrated through open access data, demonstrating superiority of 26 composite biomarkers relative to a predicate device. Especially under correction for testing of multiple hypotheses, traditional asymptotic approximations are encumbered by considerable imprecision, adversely affecting IVD device development. The ability to obtain exact p-values will allow more efficient IVD device development.

The Matthews correlation coefficient (MCC) should replace the ROC AUC as the standard metric for assessing binary classification

Binary classification is a common task for which machine learning and computational statistics are used, and the area under the receiver operating characteristic curve (ROC AUC) has become the common standard metric to evaluate binary classifications in most scientific fields. The ROC curve has true positive rate (also called sensitivity or recall) on the y axis and false positive rate on the x axis, and the ROC AUC can range from 0 (worst result) to 1 (perfect result). The ROC AUC, however, has several flaws and drawbacks. This score is generated including predictions that obtained insufficient sensitivity and specificity, and moreover it does not say anything about positive predictive value (also known as precision) nor negative predictive value (NPV) obtained by the classifier, therefore potentially generating inflated overoptimistic results. Since it is common to include ROC AUC alone without precision and negative predictive value, a researcher might erroneously conclude that their classification was successful. Furthermore, a given point in the ROC space does not identify a single confusion matrix nor a group of matrices sharing the same MCC value. Indeed, a given (sensitivity, specificity) pair can cover a broad MCC range, which casts doubts on the reliability of ROC AUC as a performance measure. In contrast, the Matthews correlation coefficient (MCC) generates a high score in its [Formula: see text] interval only if the classifier scored a high value for all the four basic rates of the confusion matrix: sensitivity, specificity, precision, and negative predictive value. A high MCC (for example, MCC [Formula: see text] 0.9), moreover, always corresponds to a high ROC AUC, and not vice versa. In this short study, we explain why the Matthews correlation coefficient should replace the ROC AUC as standard statistic in all the scientific studies involving a binary classification, in all scientific fields.

The Role of procollagen type 1 amino-terminal propertied (P1NP) Cytochrome P450 (CYPs) and Osteoprotegerin (OPG) as Potential Bone function markers in Prostate Cancer Bone Metastasis

Background: Procollagen type I amino-terminal propeptide (PINP) is often present during osteoblast development and could be a biomarker of early bone development. Osteoprotegerin (OPG) may protect tumor cells from apoptosis. Cytochrome P450 enzymes help tumor development and treatment (CYPs). Cytochrome P450 activates and deactivates anticancer drugs and procarcinogens.

Objective: The study examined the amounts of a diagnostic marker of bone formation, the amino terminal propeptide of type I procollagen (PINP), Osteoprotegerin (OPG), and P450, in prostate cancer patients at different stages and its ability to detect osteoblastic metastases.

Methods: ELISA was used to measure PINP, OPG, and P450 levels in 30 prostate cancer patients. (n = 32) and healthy men’s serum (n = 36).

Results: Prostate cancer patients had higher blood levels of PINP, OPG, and P450 than healthy persons (301.3±134.9, 980±467.2, and 84.2±28.4 pg/mL, respectively). Compared to I+II prostate cancer patients, III+IV patients showed higher serum PINP, OPG, and P450 levels (P 0.001). OPG, P450, and PINP had statistically significant Area under the ROC curve (0.9467, P= 0.0001, 0.91, P= 0.0001, and 0.6977, P= 0.4035) in prostate cancer patients.

Conclusions: Metastatic prostate cancer patients had greater PINP, OPG, and P450 levels, according to our findings. PINP, OPG, and P450 levels may affect prostate cancer progression. These findings imply that serum PINP, OPG, and P450 levels may predict and diagnose prostate cancer.

Language Semantics Interpretation with an Interaction-Based Recurrent Neural Network

Text classification is a fundamental language task in Natural Language Processing. A variety of sequential models are capable of making good predictions, yet there is a lack of connection between language semantics and prediction results. This paper proposes a novel influence score (I-score), a greedy search algorithm, called Backward Dropping Algorithm (BDA), and a novel feature engineering technique called the “dagger technique”. First, the paper proposes to use the novel influence score (I-score) to detect and search for the important language semantics in text documents that are useful for making good predictions in text classification tasks. Next, a greedy search algorithm, called the Backward Dropping Algorithm, is proposed to handle long-term dependencies in the dataset. Moreover, the paper proposes a novel engineering technique called the “dagger technique” that fully preserves the relationship between the explanatory variable and the response variable. The proposed techniques can be further generalized into any feed-forward Artificial Neural Networks (ANNs) and Convolutional Neural Networks (CNNs), and any neural network. A real-world application on the Internet Movie Database (IMDB) is used and the proposed methods are applied to improve prediction performance with an 81% error reduction compared to other popular peers if I-score and “dagger technique” are not implemented.

Fundamental Statistical Concepts in Clinical Trials and Diagnostic Testing

This article explores basic statistical concepts of clinical trial design and diagnostic testing, or how one starts with a question, formulates it into a hypothesis on which a clinical trial is then built, and integrates it with statistics and probability, such as determining the probability of rejecting the null hypothesis when it is actually true (type I error) and the probability of failing to reject the null hypothesis when it is false (type II error). There are a variety of tests for different types of data, and the appropriate test must be chosen for which the sample data meet the assumptions. Correcting type I error in the presence of multiple testing is needed to control the error's inflation. Within diagnostic testing, identifying false-positive and false-negative results is critical to understanding the performance of a test. These are used to determine the sensitivity and specificity of a test along with the test's negative predictive value and positive predictive value. These quantities, specifically sensitivity and specificity, are used to determine the accuracy of a diagnostic test using receiver-operating-characteristic curves. These concepts are briefly introduced to provide a basic understanding of clinical trial design and analysis, with references to allow the reader to explore various concepts at a more detailed level if desired.

Simple Methods for Evaluating 4 Types of Biomarkers: Surrogate Endpoint, Prognostic, Predictive, and Cancer Screening

We review simple methods for evaluating 4 types of biomarkers. First, we discuss the evaluation of surrogate endpoint biomarkers (to shorten a randomized trial) using 2 statistical and 3 biological criteria. Second, we discuss the evaluation of prognostic biomarkers (to predict the risk of disease) by comparing data collection costs with the anticipated net benefit of risk prediction. Third, we discuss the evaluation of predictive markers (to search for a promising subgroup in a randomized trial) using a multivariate subpopulation treatment effect pattern plot involving a risk difference or responders-only benefit function. Fourth, we discuss the evaluation of cancer screening biomarkers (to predict cancer in asymptomatic persons) using methodology to substantially reduce the sample size with stored specimens.

Risk Prediction Models for Head and Neck Cancer in the US Population From the INHANCE Consortium

Head and neck cancer (HNC) risk prediction models based on risk factor profiles have not yet been developed. We took advantage of the large database of the International Head and Neck Cancer Epidemiology (INHANCE) Consortium, including 14 US studies from 1981-2010, to develop HNC risk prediction models. Seventy percent of the data were used to develop the risk prediction models; the remaining 30% were used to validate the models. We used competing-risk models to calculate absolute risks. The predictors included age, sex, education, race/ethnicity, alcohol drinking intensity, cigarette smoking duration and intensity, and/or family history of HNC. The 20-year absolute risk of HNC was 7.61% for a 60-year-old woman who smoked more than 20 cigarettes per day for over 20 years, consumed 3 or more alcoholic drinks per day, was a high school graduate, had a family history of HNC, and was non-Hispanic white. The 20-year risk for men with a similar profile was 6.85%. The absolute risks of oropharyngeal and hypopharyngeal cancers were generally lower than those of oral cavity and laryngeal cancers. Statistics for the area under the receiver operating characteristic curve (AUC) were 0.70 or higher, except for oropharyngeal cancer in men. This HNC risk prediction model may be useful in promoting healthier behaviors such as smoking cessation or in aiding persons with a family history of HNC to evaluate their risks.

A Variance-reduction Approach to Detection of the Thyroid-nodule Boundary on Ultrasound Images

To perform computer-aided diagnosis of the thyroid nodules on ultrasound images, the location and boundary of nodules should be clearly defined. However, the identification of thyroid nodule boundary is a difficult issue due to the biological characteristics of the nodules, the physics and quality of ultrasound imaging, and the subjective factors and operating conditions of the operator. In this study, we propose a novel and semiautomatic method for detecting the boundary of thyroid nodule based on the Variance-Reduction (V-R) statistics without image preprocessing. The region of interest (ROI) is first automatically generated according to the initial inputs of the nodule's major and minor axes. The boundary candidate pixel points are then extracted by using the V-R statistics from the grayscale values of all pixel points in the ROI. Three filtering methods are further applied to eliminate the outlier pixel points to ensure that the remaining candidate pixel points are located on the nodule boundary. Finally, the remaining pixel points are smoothened and linked together to form the final boundary. The proposed method is validated with ultrasound images of 538 thyroid nodules, with manual delineation by experienced radiologist as gold standard. The effectiveness is evaluated and compared with previous publications using boundary error metrics and overlapping area metrics with the same data set. The results show that the normalized average mean boundary error is 1.02%, the true positive overlapping area ratio achieves 93.66% and false positive overlapping area ratio is limited to 7.68%. In conclusion, our proposed method is reliable and effective in detecting thyroid nodule boundary on ultrasound images.

Volatomic pattern of breast cancer and cancer-free tissues as a powerful strategy to identify potential biomarkers

Breast cancer (BC), ranked as the fifth amongst all cancers, remains at the top of women's cancers worldwide followed by colorectal, lung, cervix, and stomach cancers. The main handicap of most of the screening/diagnostic methods is based on their low sensitivity and specificity and the invasive behavior of most sampling procedures. The aim of this study was to establish the volatomic pattern of BC and cancer-free (CF) tissues (n = 30) from the same patients, as a powerful tool to identify a set of volatile organic metabolite (VOM) potential BC biomarkers which might be used together or complement with the traditional BC diagnostics strategies, through the integration of chromatographic data, obtained by solid-phase microextraction followed by gas chromatography-mass spectrometry (SPME/GC-qMS), with chemometric tools. A total of four metabolites: limonene, decanoic acid, acetic acid and furfural presented the highest contribution towards discrimination of BC and CF tissues (VIP > 1, p < 0.05). The discrimination efficiency and accuracy of BC tissue metabolites was ascertained by ROC curve analysis that allowed the identification of some metabolites with high sensitivity and specificity. The results obtained with this approach suggest the possibility of identifying endogenous metabolites as a platform to find potential BC biomarkers and pave the way to investigate the related metabolomic pathways in order to improve BC diagnostic tools. Moreover, deeper investigations could unravel novel mechanistic insights into the disease pathophysiology.

Understanding the predictive value of continuous markers for censored survival data using a likelihood ratio approach

BACKGROUND

The likelihood ratio function (LR), the ratio of conditional probabilities of obtaining a specific marker value among those with the event of interest over those without, provides an easily interpretable way to quantify the update of the risk prediction due to the knowledge of the marker value. The LR has been explored for both binary and continuous markers for binary events (e.g., diseased or not), however the use of the LR in censored data has not been fully explored.

METHODS

We extend the concept of LR to a time-dependent LR (TD-LR) for survival outcomes that are subject to censoring. Estimation for the TD-LR is done using Kaplan-Meier estimation and a univariate Cox proportional hazards (PH) model. A "scale invariant" approach based on marker quantiles is provided to allow comparison of predictive values between markers with different scales. Relationships to time-dependent receiver-operator characteristic (ROC) curves, area under the curve (AUC), and optimal cut-off values are considered.

RESULTS

The proposed methods were applied to data from a bladder cancer clinical trial to determine whether the neutrophil-to-lymphocyte ratio (NLR) is a valuable biomarker for predicting overall survival following surgery or combined chemotherapy and surgery. The TD-LR method yielded results consistent with the original findings while providing an easily interpretable three-dimensional surface display of how NLR related to the likelihood of event in the trial data.

CONCLUSIONS

The TD-LR provides a more nuanced understanding of the relationship between continuous markers and the likelihood of events in censored survival data. This method also allows more straightforward communication with a clinical audience through graphical presentation.

Implementing a central composite design for the optimization of solid phase microextraction to establish the urinary volatomic expression: a first approach for breast cancer

INTRODUCTION

Breast cancer (BC) is positioned as the second among all cancers remaining at the top of women´s diseases worldwide followed by colorectum, lung, cervix, and thyroid cancers. The main drawback of most the screening/diagnostic methods is their low sensitivity/specificity and in some cases the invasive procedure required to obtain the samples.

OBJECTIVES

On the present investigation, we report a statistical design was to evaluate by central composite design the influence towards the optimization of the most significant variables of solid-phase microextraction (SPME) procedure for the isolation of volatile organic metabolites (VOMs) from urine of BC patients (N = 31) and healthy individuals (CTL; N = 40). The establishment of the urinary volatomic composition, through gas chromatography-mass spectrometry (GC-MS) analysis, can boost the identification of volatile organic metabolites (VOMs) potential BC biomarkers useful to be used together or to complement the current BC diagnostics tools. Better early detection methods are needed to improve the outcomes of patients with BC.

METHODS

Several combinations of experiments were considered with a central composite design (CCD) of response surface methodology (RSM) for the urinary volatomic pattern. Three-level three-factor CCD was employed assessing the most important extraction-influencing variables-fiber coating, NaCl amount, extraction time and temperature. The optimal conditions were achieved using a carboxen/polydimethylsiloxane fiber with 15% (w/v) NaCl during 75 min at 50 °C.

RESULTS

A total of ten VOMs belonging to sulfur compounds, terpenoids and carbonyl compounds presented the highest contribution towards discrimination of BC patients from CTL (variable importance in projection (VIP) > 1, p < 0.05). The discrimination efficiency and accuracy of urinary metabolites was ascertained by receiver operating characteristic (ROC) curve analysis that allowed the identification of some metabolites with highest sensitivity and specificity to discriminate the groups.

CONCLUSIONS

The results obtained with this approach suggest the possibility to identify endogenous metabolites as a platform to discovery potential BC biomarkers and paves a way to explore the related metabolomic pathways in order to improve BC diagnostic tools.

Clinical Proteomics for Post-Hematopoeitic Stem Cell Transplantation Outcomes

Allogeneic hematopoietic stem cell transplantation (HSCT) is the most effective form of tumor immunotherapy available to date. However, while HSCT can induce beneficial graft-versus-leukemia (GVL) effect, the adverse effect of graft-versus-host disease (GVHD), which is closely linked to GVL, is the major source of morbidity and mortality following HSCT. Until recently, available diagnostic and staging tools frequently fail to identify those at higher risk of disease progression or death. Furthermore, there are shortcomings in the prediction of the need for therapeutic interventions or the response rates to different forms of therapy. The past decade has been characterized by an explosive evolution of proteomics technologies, largely due to important advances in high-throughput MS instruments and bioinformatics. Building on these opportunities, blood biomarkers have been identified and validated both as promising diagnostic tools, prognostic tools that risk-stratify patients before future occurrence of GVHD and as predictive tools for responsiveness to GVHD therapy and non-relapse mortality. These biomarkers might facilitate timely and selective therapeutic intervention. This review summarizes current information on clinical proteomics for GVHD as well as other complications following HSCT. Finally, it proposes future directions for the translation of clinical proteomics to discovery of new potential therapeutic targets to the development of drugs.

ROC Curves for the Statistical Analysis of Microarray Data

A receiver operating characteristic (ROC) curve is a graphical plot that illustrates the diagnostic ability of a binary classifier as a function of its discrimination threshold. This chapter is an overview on the use of ROC curves for microarray data. The notion of ROC curve and its motivation is introduced in Subheading 1. Relevant scientific contributions concerning the use of ROC curves for microarray data are briefly reviewed in Subheading 2. The special case with covariates is considered in Subheading 3. Two relevant aspects are reviewed in this section: the use of LASSO techniques for selecting and combining relevant markers and how to correct for multiple testing when a large number of markers are available. Finally, some conclusions are included.

Plasma N-glycans in colorectal cancer risk

Aberrant glycosylation has been associated with a number of diseases including cancer. Our aim was to elucidate changes in whole plasma N-glycosylation between colorectal cancer (CRC) cases and controls in one of the largest cohorts of its kind. A set of 633 CRC patients and 478 age and gender matched controls was analysed. Additionally, patients were stratified into four CRC stages. Moreover, N-glycan analysis was carried out in plasma of 40 patients collected prior to the initial diagnosis of CRC. Statistically significant differences were observed in the plasma N-glycome at all stages of CRC, this included a highly significant decrease in relation to the core fucosylated bi-antennary glycans F(6)A2G2 and F(6)A2G2S(6)1 (P < 0.0009). Stage 1 showed a unique biomarker signature compared to stages 2, 3 and 4. There were indications that at risk groups could be identified from the glycome (retrospective AUC = 0.77 and prospective AUC = 0.65). N-glycome biomarkers related to the pathogenic progress of the disease would be a considerable asset in a clinical setting and it could enable novel therapeutics to be developed to target the disease in patients at risk of progression.

GeneGini: Assessment via the Gini Coefficient of Reference "Housekeeping" Genes and Diverse Human Transporter Expression Profiles

The expression levels of SLC or ABC membrane transporter transcripts typically differ 100- to 10,000-fold between different tissues. The Gini coefficient characterizes such inequalities and here is used to describe the distribution of the expression of each transporter among different human tissues and cell lines. Many transporters exhibit extremely high Gini coefficients even for common substrates, indicating considerable specialization consistent with divergent evolution. The expression profiles of SLC transporters in different cell lines behave similarly, although Gini coefficients for ABC transporters tend to be larger in cell lines than in tissues, implying selection. Transporter genes are significantly more heterogeneously expressed than the members of most non-transporter gene classes. Transcripts with the stablest expression have a low Gini index and often differ significantly from the "housekeeping" genes commonly used for normalization in transcriptomics/qPCR studies. PCBP1 has a low Gini coefficient, is reasonably expressed, and is an excellent novel reference gene. The approach, referred to as GeneGini, provides rapid and simple characterization of expression-profile distributions and improved normalization of genome-wide expression-profiling data.

Identification of risk factors in epidemiologic study based on ROC curve and network

This article proposes a new non-parametric approach for identification of risk factors and their correlations in epidemiologic study, in which investigation data may have high variations because of individual differences or correlated risk factors. First, based on classification information of high or low disease incidence, we estimate Receptor Operating Characteristic (ROC) curve of each risk factor. Then, through the difference between ROC curve of each factor and diagonal, we evaluate and screen for the important risk factors. In addition, based on the difference of ROC curves corresponding to any pair of factors, we define a new type of correlation matrix to measure their correlations with disease, and then use this matrix as adjacency matrix to construct a network as a visualization tool for exploring the structure among factors, which can be used to direct further studies. Finally, these methods are applied to analysis on water pollutants and gastrointestinal tumor, and analysis on gene expression data in tumor and normal colon tissue samples.

Combining biomarkers for classification with covariate adjustment

Combining multiple markers can improve classification accuracy compared with using a single marker. In practice, covariates associated with markers or disease outcome can affect the performance of a biomarker or biomarker combination in the population. The covariate-adjusted receiver operating characteristic (ROC) curve has been proposed as a tool to tease out the covariate effect in the evaluation of a single marker; this curve characterizes the classification accuracy solely because of the marker of interest. However, research on the effect of covariates on the performance of marker combinations and on how to adjust for the covariate effect when combining markers is still lacking. In this article, we examine the effect of covariates on classification performance of linear marker combinations and propose to adjust for covariates in combining markers by maximizing the nonparametric estimate of the area under the covariate-adjusted ROC curve. The proposed method provides a way to estimate the best linear biomarker combination that is robust to risk model assumptions underlying alternative regression-model-based methods. The proposed estimator is shown to be consistent and asymptotically normally distributed. We conduct simulations to evaluate the performance of our estimator in cohort and case/control designs and compare several different weighting strategies during estimation with respect to efficiency. Our estimator is also compared with alternative regression-model-based estimators or estimators that maximize the empirical area under the ROC curve, with respect to bias and efficiency. We apply the proposed method to a biomarker study from an human immunodeficiency virus vaccine trial. Copyright © 2017 John Wiley & Sons, Ltd.

Neutrophil gelatinase-associated lipocalin as a screening test in prostate cancer

OBJECTIVE

Prostate specific antigen (PSA) with digital rectal examination is used for diagnosis of prostate cancer (PCa), where definite diagnosis can only be made by prostate biopsy. Recently neutrophil gelatinase-associated lipocalin (NGAL), a lipocalin family member glycoprotein, come into prominence as a cancer biomarker. This study is aimed to test serum NGAL as a diagnostic biomarker for PCa and discriminate PCa from benign prostatic hyperplasia (BPH).

MATERIAL AND METHODS

In this prospective study, 90 patients who underwent transrectal ultrasound-guided 12-core prostate biopsy between May 2015 and September 2015, were evaluated. Histopathologically diagnosed 45 PCa and 45 BPH patients were enrolled in this study. Serum NGAL and PSA levels of all participants were measured, then these data were evaluated by statistical programs.

RESULTS

When sensitivity fixed to 80% specificity of NGAL was better than PSA (49%, 31% respectively). Receiver operating characteristic (ROC) curve analysis showed that NGAL alone or its combined use with PSA have better area under curve (AUC) results than PSA alone (0.662, 0.693, and 0.623 respectively).

CONCLUSION

In conclusion NGAL gave promising results such as increased sensitivity and a better AUC values in order to distinguish PCa from BPH. NGAL showed a potential to be a non-invasive biomarker which may decrease the number of unnecessary biopsies.

Biomarkers in cardiovascular disease: Statistical assessment and section on key novel heart failure biomarkers

Cardiovascular disease (CVD) is a leading cause of death worldwide and continues to increase in prevalence compared to previous decades, in part because of the aging of the world population. Atherosclerotic CVD starts at a very young age and progresses over time allowing sufficient time for screening and early detection of the condition. Advances in biomarker research and developments related to CVD over the past 30 years have led to more sensitive screening methods, a greater emphasis on its early detection and diagnosis, and improved treatments resulting in more favorable clinical outcomes in the community. However, the use of biomarkers for different purposes in CVD remains an important area of research that has been explored by scientists over the years and many new developments are still underway. Therefore, a detailed description of all CVD biomarkers that are currently been used or investigated for future use in the field of cardiovascular medicine is out of scope for any review article. In the present review, we do not intend to replicate the information from previous exhaustive review on biomarkers, but highlight key statistical and clinical issues with an emphasis on methods to evaluate the incremental yield of biomarkers, including their clinical utility, a prerequisite before any putative novel biomarker is utilized in clinical practice. In addition, we will summarize information regarding recent novel heart failure biomarkers in current practice, which are undergoing scrutiny before they can be available for clinical use, and their impact on clinical outcomes.

Canine olfaction as an alternative to analytical instruments for disease diagnosis: understanding 'dog personality' to achieve reproducible results

Recent literature has touted the use of canine olfaction as a diagnostic tool for identifying pre-clinical disease status, especially cancer and infection from biological media samples. Studies have shown a wide range of outcomes, ranging from almost perfect discrimination, all the way to essentially random results. This disparity is not likely to be a detection issue; dogs have been shown to have extremely sensitive noses as proven by their use for tracking, bomb detection and search and rescue. However, in contrast to analytical instruments, dogs are subject to boredom, fatigue, hunger and external distractions. These challenges are of particular importance in a clinical environment where task repetition is prized, but not as entertaining for a dog as chasing odours outdoors. The question addressed here is how to exploit the intrinsic sensitivity and simplicity of having a dog simply sniff out disease, in the face of variability in behavior and response.

The Brier score does not evaluate the clinical utility of diagnostic tests or prediction models

BACKGROUND

A variety of statistics have been proposed as tools to help investigators assess the value of diagnostic tests or prediction models. The Brier score has been recommended on the grounds that it is a proper scoring rule that is affected by both discrimination and calibration. However, the Brier score is prevalence dependent in such a way that the rank ordering of tests or models may inappropriately vary by prevalence.

METHODS

We explored four common clinical scenarios: comparison of a highly accurate binary test with a continuous prediction model of moderate predictiveness; comparison of two binary tests where the importance of sensitivity versus specificity is inversely associated with prevalence; comparison of models and tests to default strategies of assuming that all or no patients are positive; and comparison of two models with miscalibration in opposite directions.

RESULTS

In each case, we found that the Brier score gave an inappropriate rank ordering of the tests and models. Conversely, net benefit, a decision-analytic measure, gave results that always favored the preferable test or model.

CONCLUSIONS

Brier score does not evaluate clinical value of diagnostic tests or prediction models. We advocate, as an alternative, the use of decision-analytic measures such as net benefit.

TRIAL REGISTRATION

Not applicable.

Maximizing the usefulness of statistical classifiers for two populations with illustrative applications

The usefulness of two-class statistical classifiers is limited when one or both of the conditional misclassification rates is unacceptably high. Incorporating a neutral zone region into the classifier provides a mechanism to refer ambiguous cases to follow-up where additional information might be obtained to clarify the classification decision. Through the use of the neutral zone region, the conditional misclassification rates can be controlled and the classifier becomes useful. Three real-life examples, including applications to prostate cancer and kidney dysfunction following heart surgery, are used to illustrate how neutral zone regions can extract utility from disappointing classifiers that might otherwise be abandoned.

Fully non-parametric receiver operating characteristic curve estimation for random-effects meta-analysis

Meta-analyses, broadly defined as the quantitative review and synthesis of the results of related but independent comparable studies, allow to know the state of the art of one considered topic. Since the amount of available bibliography has enhanced in almost all fields and, specifically, in biomedical research, its popularity has drastically increased during the last decades. In particular, different methodologies have been developed in order to perform meta-analytic studies of diagnostic tests for both fixed- and random-effects models. From a parametric point of view, these techniques often compute a bivariate estimation for the sensitivity and the specificity by using only one threshold per included study. Frequently, an overall receiver operating characteristic curve based on a bivariate normal distribution is also provided. In this work, the author deals with the problem of estimating an overall receiver operating characteristic curve from a fully non-parametric approach when the data come from a meta-analysis study i.e. only certain information about the diagnostic capacity is available. Both fixed- and random-effects models are considered. In addition, the proposed methodology lets to use the information of all cut-off points available (not only one of them) in the selected original studies. The performance of the method is explored through Monte Carlo simulations. The observed results suggest that the proposed estimator is better than the reference one when the reported information is related to a threshold based on the Youden index and when information for two or more points are provided. Real data illustrations are included.

Vesicoureteral refux detection in children: a comparison of the midline-to-orifice distance measurement by ultrasound and voiding urosonography

BACKGROUND

Due to the questionable clinical role of vesicoureteral reflux (VUR) and the search for noninvasive, radiation-free procedures sufficiently reliable to detect VUR, we compared the correlation between the midline-to-orifice distance (MOD) measured by ultrasonography (US) and echo-enhanced voiding urosonography (VUS) for detecting VUR in children. The aim of the study was to determine whether measuring MOD by US could be a reliable predictor of VUR in children.

METHODS

A total of 116 children, aged 0.25-84 months, with 232 potentially refluxing units were investigated simultaneously by measuring the MOD and performing VUS. Indications for cystography were urinary tract infection and follow-up of a previously detected VUR. VUS was performed after the MOD measurement. The results were analyzed with VUS as the reference method.

RESULTS

The MOD was significantly larger in VUR grade III (10.7 mm; p = 0.003) and VUR grade II (9.9 mm; p = 0.001) refluxing units than in non-refluxing units (7.8 mm), even when controlling for the estimated volume/expected maximal capacity (Vest/Vmax) ratio. A MOD cutoff value of 7.4 mm was chosen as a predictor of either the presence or absence of VUR; the sensitivity and specificity of this cutoff measurement for VUR detection were found to be 89 and 24%, respectively.

CONCLUSIONS

Despite the statistically significant difference between the MOD of refluxing versus non-refluxing units identified in our study, the MOD measurement needs further evaluation to determine its potential value as a diagnostic tool for the detection of VUR.

A Review of Cutoffs for Nutritional Biomarkers

The nutritional status of an individual or population needs to be assessed through valid and reliable biomarkers. Cutoffs generally have an underlying relation to health status and are one of the important quantitative criteria against which biomarker outputs are compared. For this reason, cutoffs are integral for surveys, surveillance, screening, interventions, monitoring, and evaluation. Despite their importance, nutritional biomarker cutoffs have not been adequately addressed in the literature. Furthermore, the field has not reached a consensus on which cutoff to use for each biomarker, and different cutoffs are often used for the same biomarkers in published studies. This review provides a comprehensive overview of cutoffs related to nutritional biomarkers and highlights some of the high-priority research gaps and challenges of using micronutrient case studies.

Lymph node retrieval for colorectal cancer: Estimation of the minimum resection length to achieve at least 12 lymph nodes for the pathological analysis

INTRODUCTION

Adequate lymph node retrieval is important in colorectal cancer staging for the selection of patients that necessitate adjuvant treatments. The minimum number of 12 lymph nodes is one of the premises and is dependent, among the other factors, from the length of bowel resected. We have reviewed our specimens to identify the high-risk operations for inadequate nodal sampling and estimate the minimum length of bowel needed to resect to achieve this purpose.

MATERIALS AND METHODS

A retrospective review of colorectal specimens over 10 years of activity looking at data including location of the tumor, type of operation performed, length of bowel resected and number of lymph nodes retrieved.

RESULTS

Abdominoperineal and Hartmann's resections produced significant lower adequate retrievals compared to other colorectal operations, corresponding to 45.4% and 59.1% of cases respectively. The measured average length of bowel was 30 cm and 25 cm respectively, increasing the length to 36 cm and 42 cm would increase the adequacy rate to 90%.

CONCLUSIONS

Abdominoperineal and Hartmann's resections are, in our series, high-risk operations that frequently do not produce the minimum number of lymph nodes necessary. These operations may require additional maneuvers such as mobilization of the splenic flexure to achieve the minimum length of bowel to resect.

Development of a High-Throughput Functional Screen Using Nanowell-Assisted Cell Patterning

Living-cell-based screens can facilitate lead discovery of functional therapeutics of interest. A versatile and scalable method is reported that uses dense arrays of nanowells for imparting defined patterns on monolayers of cells. It is shown that this approach can coordinate a multi-component biological assay by designing and implementing a high-throughput, functional nanoliter-scale neutralization assay to identify neutralizing antibodies against HIV.

Targeted salivary biomarkers for discrimination of periodontal health and disease(s)

Generally, clinical parameters are used in dental practice for periodontal disease, yet several drawbacks exist with the clinical standards for addressing the needs of the public at large in determining the current status/progression of the disease, and requiring a significant amount of damage before these parameters can document disease. Therefore, a quick, easy and reliable method of assessing and monitoring periodontal disease should provide important diagnostic information that improves and speeds treatment decisions and moves the field closer to individualized point-of-care diagnostics.

OBJECTIVE

This report provides results for a saliva-based diagnostic approach for periodontal health and disease based upon the abundance of salivary analytes coincident with disease, and the significant progress already made in the identification of discriminatory salivary biomarkers of periodontitis.

METHODS

We evaluated biomarkers representing various phases of periodontitis initiation and progression (IL-1ß, IL-6, MMP-8, MIP-1α) in whole saliva from 209 subjects categorized with periodontal health, gingivitis, and periodontitis.

RESULTS

Evaluation of the salivary analytes demonstrated utility for individual biomarkers to differentiate periodontitis from health. Inclusion of gingivitis patients into the analyses provided a more robust basis to estimate the value of each of these analytes. Various clinical and statistical approaches showed that pairs or panels of the analytes were able to increase the sensitivity and specificity for the identification of disease.

CONCLUSIONS

Salivary concentrations of IL-1ß, IL-6, MMP-8, MIP-1α alone and in combination are able to distinguish health from gingivitis and periodontitis. The data clearly demonstrated a heterogeneity in response profiles of these analytes that supports the need for refinement of the standard clinical classifications if we are to move toward precision/personalized dentistry for the twenty-first century.

Elevated IgG4 in patient circulation is associated with the risk of disease progression in melanoma

Emerging evidence suggests pathological and immunoregulatory functions for IgG4 antibodies and IgG4⁺ B cells in inflammatory diseases and malignancies. We previously reported that IgG4 antibodies restrict activation of immune effector cell functions and impair humoral responses in melanoma. Here, we investigate IgG4 as a predictor of risk for disease progression in a study of human sera (n = 271: 167 melanoma patients; 104 healthy volunteers) and peripheral blood B cells (n = 71: 47 melanoma patients; 24 healthy volunteers). IgG4 (IgG4/IgG_total) serum levels were elevated in melanoma. High relative IgG4 levels negatively correlated with progression-free survival (PFS) and overall survival. In early stage (I-II) disease, serum IgG4 was independently negatively prognostic for progression-free survival, as was elevation of IgG4⁺ circulating B cells (CD45⁺CD22⁺CD19⁺CD3^-CD14^-). In human tissues (n = 256; 108 cutaneous melanomas; 56 involved lymph nodes; 60 distant metastases; 32 normal skin samples) IgG4⁺ cell infiltrates were found in 42.6% of melanomas, 21.4% of involved lymph nodes and 30% of metastases, suggesting inflammatory conditions that favor IgG4 at the peripheral and local levels. Consistent with emerging evidence for an immunosuppressive role for IgG4, these findings indicate association of elevated IgG4 with disease progression and less favorable clinical outcomes. Characterizing immunoglobulin and other humoral immune profiles in melanoma might identify valuable prognostic tools for patient stratification and in the future lead to more effective treatments less prone to tumor-induced blockade mechanisms.

Predictive Levels of CD24 in Peripheral Blood Leukocytes for the Early Detection of Colorectal Adenomas and Adenocarcinomas

CD24 is expressed in 90% of colorectal adenomas and adenocarcinomas. Colorectal cancer (CRC) can be mostly prevented but average risk population screening by stool testing or colonoscopy faces many hurdles. Blood testing is clinically needed. We aimed to evaluate the utility of CD24 expression in peripheral blood leukocytes (PBLs). Two independent case studies were conducted in eligible individuals undergoing colonoscopy. Protein extracted from PBLs was subjected to immunoblotting using anti-CD24 monoclonal antibodies. CD24 sensitivity and specificity were determined using receiver operating characteristic (ROC) analysis. Initially, 150 subjects were examined: 63 had CRC, 19 had adenomas, and 68 had normal colonoscopies. The sensitivity and specificity of CD24 for distinguishing CRC from normal subjects were 70.5% (95% CI, 54.8–83.2%) and 83.8% (95% CI, 74.6–92.7%) and for adenomas 84.2% (95% CI, 60.4–96.4%) and 73.5% (95% CI, 61.4–83.5%), respectively. In the second trial (n = 149), a similar specificity but higher sensitivity was achieved: 80.0% (95% CI, 63.1–91.6%) for CRC and 89.2% (95% CI, 74.6–97%) for adenomas. A simple noninvasive blood test evaluating CD24 levels has high sensitivity and specificity for detecting colorectal adenomas and cancer in patients undergoing colonoscopy at an urban medical center. Larger multicenter studies are warranted to establish the potential of this promising test.

Disadvantages of using the area under the receiver operating characteristic curve to assess imaging tests: a discussion and proposal for an alternative approach

OBJECTIVES

The objectives are to describe the disadvantages of the area under the receiver operating characteristic curve (ROC AUC) to measure diagnostic test performance and to propose an alternative based on net benefit.

METHODS

We use a narrative review supplemented by data from a study of computer-assisted detection for CT colonography.

RESULTS

We identified problems with ROC AUC. Confidence scoring by readers was highly non-normal, and score distribution was bimodal. Consequently, ROC curves were highly extrapolated with AUC mostly dependent on areas without patient data. AUC depended on the method used for curve fitting. ROC AUC does not account for prevalence or different misclassification costs arising from false-negative and false-positive diagnoses. Change in ROC AUC has little direct clinical meaning for clinicians. An alternative analysis based on net benefit is proposed, based on the change in sensitivity and specificity at clinically relevant thresholds. Net benefit incorporates estimates of prevalence and misclassification costs, and it is clinically interpretable since it reflects changes in correct and incorrect diagnoses when a new diagnostic test is introduced.

CONCLUSIONS

ROC AUC is most useful in the early stages of test assessment whereas methods based on net benefit are more useful to assess radiological tests where the clinical context is known. Net benefit is more useful for assessing clinical impact.

KEY POINTS

• The area under the receiver operating characteristic curve (ROC AUC) measures diagnostic accuracy. • Confidence scores used to build ROC curves may be difficult to assign. • False-positive and false-negative diagnoses have different misclassification costs. • Excessive ROC curve extrapolation is undesirable. • Net benefit methods may provide more meaningful and clinically interpretable results than ROC AUC.

Opportunities and challenges of proteomics in pediatric patients: circulating biomarkers after hematopoietic stem cell transplantation as a successful example

Biomarkers have the potential to improve diagnosis and prognosis, facilitate-targeted treatment, and reduce health care costs. Thus, there is great hope that biomarkers will be integrated in all clinical decisions in the near future. A decade ago, the biomarker field was launched with great enthusiasm because MS revealed that blood contains a rich library of candidate biomarkers. However, biomarker research has not yet delivered on its promise due to several limitations: (i) improper sample handling and tracking as well as limited sample availability in the pediatric population, (ii) omission of appropriate controls in original study designs, (iii) lability and low abundance of interesting biomarkers in blood, and (iv) the inability to mechanistically tie biomarker presence to disease biology. These limitations as well as successful strategies to overcome them are discussed in this review. Several advances in biomarker discovery and validation have been made in hematopoietic stem cell transplantation, the current most effective tumor immunotherapy, and these could serve as examples for other conditions. This review provides fresh optimism that biomarkers clinically relevant in pediatrics are closer to being realized based on: (i) a uniform protocol for low-volume blood collection and preservation, (ii) inclusion of well-controlled independent cohorts, (iii) novel technologies and instrumentation with low analytical sensitivity, and (iv) integrated animal models for exploring potential biomarkers and targeted therapies.

Clinical proteomic biomarkers: relevant issues on study design & technical considerations in biomarker development

Biomarker research is continuously expanding in the field of clinical proteomics. A combination of different proteomic-based methodologies can be applied depending on the specific clinical context of use. Moreover, current advancements in proteomic analytical platforms are leading to an expansion of biomarker candidates that can be identified. Specifically, mass spectrometric techniques could provide highly valuable tools for biomarker research. Ideally, these advances could provide with biomarkers that are clinically applicable for disease diagnosis and/ or prognosis. Unfortunately, in general the biomarker candidates fail to be implemented in clinical decision making. To improve on this current situation, a well-defined study design has to be established driven by a clear clinical need, while several checkpoints between the different phases of discovery, verification and validation have to be passed in order to increase the probability of establishing valid biomarkers. In this review, we summarize the technical proteomic platforms that are available along the different stages in the biomarker discovery pipeline, exemplified by clinical applications in the field of bladder cancer biomarker research.

Predictability, efficacy and safety of radiosensitization of glioblastoma-initiating cells by the ATM inhibitor KU-60019

We have previously shown that pharmacological inhibition of ataxia telangiectasia mutated (ATM) protein sensitizes glioblastoma-initiating cells (GICs) to ionizing radiation (IR). Herein, we report the experimental conditions to overcome GIC radioresistance in vitro using the specific ATM inhibitor KU-60019, two major determinants of the tumor response to this drug and the absence of toxicity of this treatment in vitro and in vivo. Repeated treatments with KU-60019 followed by IR substantially delayed GIC proliferation in vitro and even eradicated radioresistant cells, whereas GIC treated with vehicle plus radiation recovered early and expanded. The tumor response to the drug occurred under a cutoff level of expression of TP53 and over a cutoff level of expression of phosphatidylinositol 3-kinase (PI3K). No increased clastogenicity or point mutagenicity was induced by KU-60019 plus radiation when compared to vehicle plus radiation. No significant histological changes to the brain or other organs were observed after prolonged infusion into the brain of KU-60019 at millimolar concentrations. Taken together, these findings suggest that GIC-driven tumors with low expression of TP53 and high expression of PI3K might be effectively and safely radiosensitized by KU-60019.

Feature Screening for Ultrahigh Dimensional Categorical Data with Applications

Ultrahigh dimensional data with both categorical responses and categorical covariates are frequently encountered in the analysis of big data, for which feature screening has become an indispensable statistical tool. We propose a Pearson chi-square based feature screening procedure for categorical response with ultrahigh dimensional categorical covariates. The proposed procedure can be directly applied for detection of important interaction effects. We further show that the proposed procedure possesses screening consistency property in the terminology of Fan and Lv (2008). We investigate the finite sample performance of the proposed procedure by Monte Carlo simulation studies, and illustrate the proposed method by two empirical datasets.

Limitations of early serum creatinine variations for the assessment of kidney injury in neonates and infants with cardiac surgery

Background

Changes in kidney function, as assessed by early and even small variations in serum creatinine (ΔsCr), affect survival in adults following cardiac surgery but such associations have not been reported in infants. This raises the question of the adequate assessment of kidney function by early ΔsCr in infants undergoing cardiac surgery.

Methodology

The ability of ΔsCr within 2 days of surgery to assess the severity of kidney injury, accounted for by the risk of 30-day mortality, was explored retrospectively in 1019 consecutive neonates and infants. Patients aged ≤ 10 days were analyzed separately because of the physiological improvement in glomerular filtration early after birth. The Kml algorithm, an implementation of k-means for longitudinal data, was used to describe creatinine kinetics, and the receiver operating characteristic and the reclassification methodology to assess discrimination and the predictive ability of the risk of death.

Results

Three clusters of ΔsCr were identified: in 50% of all patients creatinine decreased, in 41.4% it increased slightly, and in 8.6% it rose abruptly. Mortality rates were not significantly different between the first and second clusters, 1.6% [0.0–4.1] vs 5.9% [1.9–10.9], respectively, in patients aged ≤ 10 days, and 1.6% [0.5–3.0] vs 3.8% [1.9–6.0] in older ones. Mortality rates were significantly higher when creatinine rose abruptly, 30.3% [15.1–46.2] in patients aged ≤ 10 days, and 15.1% [5.9–25.5] in older ones. However, only 41.3% of all patients who died had an abrupt increase in creatinine. ΔsCr improved prediction in survivors, but not in patients who died, and did not improve discrimination over a clinical mortality model.

Conclusions

The present results suggest that a postoperative decrease in creatinine represents the normal course in neonates and infants with cardiac surgery, and that early creatinine variations lack sensitivity for the assessment of the severity of kidney injury.

HI-bone: a scoring system for identifying phenylisothiocyanate-derivatized peptides based on precursor mass and high intensity fragment ions

Peptide sequence matching algorithms used for peptide identification by tandem mass spectrometry (MS/MS) enumerate theoretical peptides from the database, predict their fragment ions, and match them to the experimental MS/MS spectra. Here, we present an approach for scoring MS/MS identifications based on the high mass accuracy matching of precursor ions, the identification of a high intensity b1 fragment ion, and partial sequence tags from phenylthiocarbamoyl-derivatized peptides. This derivatization process boosts the b1 fragment ion signal, which turns it into a powerful feature for peptide identification. We demonstrate the effectiveness of our scoring system by implementing it on a computational tool called "HI-bone" and by identifying mass spectra of an Escherichia coli sample acquired on an Orbitrap Velos instrument using Higher-energy C-trap dissociation. Following this strategy, we identified 1614 peptide spectrum matches with a peptide false discovery rate (FDR) below 1%. These results were significantly higher than those from Mascot and SEQUEST using a similar FDR.