Lehmann family of ROC curves

Estimating transformations for evaluating diagnostic tests with covariate adjustment

Receiver operating characteristic analysis is one of the most popular approaches for evaluating and comparing the accuracy of medical diagnostic tests. Although various methodologies have been developed for estimating receiver operating characteristic curves and their associated summary indices, there is no consensus on a single framework that can provide consistent statistical inference while handling the complexities associated with medical data. Such complexities might include non-normal data, covariates that influence the diagnostic potential of a test, ordinal biomarkers or censored data due to instrument detection limits. We propose a regression model for the transformed test results which exploits the invariance of receiver operating characteristic curves to monotonic transformations and accommodates these features. Simulation studies show that the estimates based on transformation models are unbiased and yield coverage at nominal levels. The methodology is applied to a cross-sectional study of metabolic syndrome where we investigate the covariate-specific performance of weight-to-height ratio as a non-invasive diagnostic test. Software implementations for all the methods described in the article are provided in the tram add-on package to the R system for statistical computing and graphics.

Lockdown Without Loss? A Natural Experiment of Net Payoffs from COVID-19 Lockdowns

Lacking a federal policy to control the spread of COVID-19, state governors ordered lockdowns and mask mandates, at different times, generating a massive natural experiment. The authors exploit this natural experiment to address four issues: (1) Were lockdowns effective in reducing infections? (2) What were the costs to consumers? (3) Did lockdowns increase (signaling effect) or reduce (substitution effect) consumers' mask adoption? (4) Did governors' decisions depend on medical science or nonmedical drivers? Analyses via difference-in-differences and generalized synthetic control methods indicate that lockdowns causally reduced infections. Although lockdowns reduced infections by 480 per million consumers per day (equivalent to a reduction of 56%), they reduced customer satisfaction by 2.2%, consumer spending by 7.5%, and gross domestic product by 5.4% and significantly increased unemployment by 2% per average state by the end of the observation period. A counterfactual analysis shows that a nationwide lockdown on March 15, 2020, would have reduced total cases by 60%, whereas the absence of any state lockdowns would have resulted in five times more cases by April 30. The average cost of reducing the number of cases by one new infection was about $28,000 in lower gross domestic product.

A robust hazard ratio for general modeling of survival-times

Hazard ratios (HR) associated with the well-known proportional hazard Cox regression models are routinely used for measuring the impact of one factor of interest on a time-to-event outcome. However, if the underlying real model does not fit with the theoretical requirements, the interpretation of those HRs is not clear. We propose a new index, gHR, which generalizes the HR beyond the underlying survival model. We consider the case in which the study factor is a binary variable and we are interested in both the unadjusted and adjusted effect of this factor on a time-to-event variable, potentially, observed in a right-censored scenario. We propose non-parametric estimations for unadjusted gHR and semi-parametric regression-induced techniques for the adjusted case. The behavior of those estimators is studied in both large and finite sample situations. Monte Carlo simulations reveal that both estimators provide good approximations of their respective inferential targets. Data from the Health and Lifestyle Study are used for studying the relationship of the tobacco use and the age of death and illustrate the practical application of the proposed technique. gHR is a promising index which can help facilitate better understanding of the association of one study factor on a time-dependent outcome.

Pre-HAART CD4+ T-lymphocytes as biomarkers of post-HAART immune recovery in HIV-infected children with or without TB co-infection

BACKGROUND

Infection with the Human Immunodeficiency Virus (HIV) dramatically increases the risk of developing active tuberculosis (TB). Several studies have indicated that co-infection with TB increases the risk of HIV progression and death. Sub-Saharan Africa bears the brunt of these dual epidemics, with about 2.4 million HIV-infected people living with TB. The main objective of our study was to assess whether the pre-HAART CD4+ T-lymphocyte counts and percentages could serve as biomarkers for post-HAART treatment immune-recovery in HIV-positive children with and without TB co-infection.

METHODS

The data analyzed in this retrospective study were collected from a cohort of 305 HIV-infected children being treated with HAART. A Lehmann family of ROC curves were used to assess the diagnostic performance of pre- HAART treatment CD4+ T-lymphocyte count and percentage as biomarkers for post-HAART immune recovery. The Kaplan-Meier estimator was used to compare differences in post-HAART recovery times between patients with and without TB co-infection.

RESULTS

We found that the diagnostic performance of both pre-HARRT treatment CD4+ T-lymphocyte count and percentage was comparable and achieved accuracies as high as 74%. Furthermore, the predictive capability of pre-HAART CD4+ T-lymphocyte count and percentage were slightly better in TB-negative patients. Our analyses also indicate that TB-negative patients have a shorter recovery time compared to the TB-positive patients.

CONCLUSIONS

Pre-HAART CD4+ T-lymphocyte count and percentage are stronger predictors of immune recovery in TB-negative pediatric patients, suggesting that TB co-infection complicates the treatment of HIV in this cohort. These findings suggest that the detection and treatment of TB is essential for the effectiveness of HAART in HIV-infected pediatric patients.

Comparing the Diagnostics Accuracy of CD4+ T-Lymphocyte Count and Percent as a Surrogate Markers of Pediatric HIV Disease

The percentage CD4+ T-lymphocytes is used to monitor pediatric HIV disease. However, in resource-limited settings, enumerating the percentage of CD4+ T-lymphocytes is hampered by the lack of laboratory infrastructure and trained technicians. In this paper, we investigated the performances of the percentage and absolute CD4+ T-lymphocytes as markers of pediatric HIV disease progression using data from HIV-infected children enrolled through the Yale Prospective Longitudinal Pediatric Cohort study. A Lehmann family of Receiver Operating Characteristic (ROC) curves were used to estimate and compare the performance of the two biomarkers in monitoring pediatric HIV disease progression. The area under the ROC (AUC) curve and its empirical estimator have previously been used to assess the performance of biomarkers for a cross-sectional data. However, there is a paucity of literature on the AUC for correlated longitudinal biomarkers. Previous works on the estimation and inference of the AUC for longitudinal biomarkers have largely focused on independent biomarkers or failed to consider the effect of covariates. The Lehmann approach allowed us to estimate the AUC of the aforementioned correlated longitudinal biomarkers as functions of explanatory variables. We found that the overall performance of the two biomarkers was comparable. The area under the ROC curves for CD4+ T cell count and percentage were 0.681 [SE = 0.029; 95% CI: 0.624-0.737] and 0.678 [SE = 0.024; 95% CI:0.630-0.725], respectively. Our results suggest that absolute CD4+ T-lymphocyte counts could be used as a proxy for percentage of CD4+ T-lymphocytes in monitoring pediatric HIV in resource-limited settings.

ROC Estimation from Clustered Data with an Application to Liver Cancer Data

In this article, we propose a regression model to compare the performances of different diagnostic methods having clustered ordinal test outcomes. The proposed model treats ordinal test outcomes (an ordinal categorical variable) as grouped-survival time data and uses random effects to explain correlation among outcomes from the same cluster. To compare different diagnostic methods, we introduce a set of covariates indicating diagnostic methods and compare their coefficients. We find that the proposed model defines a Lehmann family and can also introduce a location-scale family of a receiver operating characteristic (ROC) curve. The proposed model can easily be estimated using standard statistical software such as SAS and SPSS. We illustrate its practical usefulness by applying it to testing different magnetic resonance imaging (MRI) methods to detect abnormal lesions in a liver.

Using Dual Beta Distributions to Create "Proper" ROC Curves Based on Rating Category Data

BACKGROUND

Receiver operating characteristic (ROC) analysis helps investigators quantify and describe how well a diagnostic system discriminates between 2 mutually exclusive conditions. The conventional binormal (CvB) curve-fitting model usually produces ROCs that are improper in that they do not have the ever-decreasing slope required by signal detection theory. When data sets evaluated under the CvB model have hooks, the resulting ROCs can contain misleading information about the diagnostic performance of the method at low and high false positive rates.

OBJECTIVE

To present and evaluate a dual beta (DB) ROC model that assumes diagnostic data arise from 2 β distributions. The DB model's parameter constraints assure that the resulting ROC curve has a positive, monotonically decreasing slope.

DESIGN/METHOD

Computer simulation study comparing results from CvB, DB, and weighted power function (WPF) models.

RESULTS

The DB model produces results that are as good as or better than those from the WPF model, and less biased and closer to the true values than curves obtained using the CvB model.

CONCLUSIONS

The DB ROC model expresses the relationship between the false positive rate and true positive rate in closed form and allows for quick ROC area calculations using spreadsheet functions. Because it posits simple relationships among the decision axis, operating points, and model parameters, the DB model offers investigators a flexible, easy-to-grasp ROC form that is simpler to implement than other proper ROC models.

Non-small cell lung cancer is characterized by dramatic changes in phospholipid profiles

Non-small cell lung cancer (NSCLC) is the leading cause of cancer death globally. To develop better diagnostics and more effective treatments, research in the past decades has focused on identification of molecular changes in the genome, transcriptome, proteome, and more recently also the metabolome. Phospholipids, which nevertheless play a central role in cell functioning, remain poorly explored. Here, using a mass spectrometry (MS)-based phospholipidomics approach, we profiled 179 phospholipid species in malignant and matched non-malignant lung tissue of 162 NSCLC patients (73 in a discovery cohort and 89 in a validation cohort). We identified 91 phospholipid species that were differentially expressed in cancer versus non-malignant tissues. Most prominent changes included a decrease in sphingomyelins (SMs) and an increase in specific phosphatidylinositols (PIs). Also a decrease in multiple phosphatidylserines (PSs) was observed, along with an increase in several phosphatidylethanolamine (PE) and phosphatidylcholine (PC) species, particularly those with 40 or 42 carbon atoms in both fatty acyl chains together. 2D-imaging MS of the most differentially expressed phospholipids confirmed their differential abundance in cancer cells. We identified lipid markers that can discriminate tumor versus normal tissue and different NSCLC subtypes with an AUC (area under the ROC curve) of 0.999 and 0.885, respectively. In conclusion, using both shotgun and 2D-imaging lipidomics analysis, we uncovered a hitherto unrecognized alteration in phospholipid profiles in NSCLC. These changes may have important biological implications and may have significant potential for biomarker development.

Meta-analysis and meta-modelling for diagnostic problems

BACKGROUND

A proportional hazards measure is suggested in the context of analyzing SROC curves that arise in the meta-analysis of diagnostic studies. The measure can be motivated as a special model: the Lehmann model for ROC curves. The Lehmann model involves study-specific sensitivities and specificities and a diagnostic accuracy parameter which connects the two.

METHODS

A study-specific model is estimated for each study, and the resulting study-specific estimate of diagnostic accuracy is taken as an outcome measure for a mixed model with a random study effect and other study-level covariates as fixed effects. The variance component model becomes estimable by deriving within-study variances, depending on the outcome measure of choice. In contrast to existing approaches - usually of bivariate nature for the outcome measures - the suggested approach is univariate and, hence, allows easily the application of conventional mixed modelling.

RESULTS

Some simple modifications in the SAS procedure proc mixed allow the fitting of mixed models for meta-analytic data from diagnostic studies. The methodology is illustrated with several meta-analytic diagnostic data sets, including a meta-analysis of the Mini-Mental State Examination as a diagnostic device for dementia and mild cognitive impairment.

CONCLUSIONS

The proposed methodology allows us to embed the meta-analysis of diagnostic studies into the well-developed area of mixed modelling. Different outcome measures, specifically from the perspective of whether a local or a global measure of diagnostic accuracy should be applied, are discussed as well. In particular, variation in cut-off value is discussed together with recommendations on choosing the best cut-off value. We also show how this problem can be addressed with the proposed methodology.

Constructing “Proper” ROCs from Ordinal Response Data Using Weighted Power Functions

Background. Receiver operating characteristic (ROC) analysis is the standard method for describing the accuracy of diagnostic systems where the decision task involves distinguishing between 2 mutually exclusive possibilities. The popular binormal curve-fitting model usually produces ROCs that are improper in that they do not have the ever-decreasing slope required by signal detection theory. Not infrequently, binormal ROCs have visible hooks that falsely imply worse-than-chance diagnostic differentiation where the curve lies below the no-information diagonal. In this article, we present and evaluate a 2-parameter, weighted power function (WPF) model that always results in a proper ROC curve with a positive, monotonically decreasing slope. Methods. We used a computer simulation study to compare results from binormal and WPF models. Results. The WPF model produces ROC curves that are less biased and closer to the true values than are curves obtained using the binormal model. The better performance of the WPF model follows from its design constraint as a necessarily proper ROC. Conclusions. The WPF model fits a broader variety of data sets than previously published power function models while maintaining straightforward relationships among the original decision variable, specific operating points, ROC curve contours, and model parameters. Compared with other proper ROC models, the WPF model is distinctive in its simplicity, and it avoids the flaws of the conventional binormal ROC model.

Smooth ROC curves and surfaces for markers subject to a limit of detection using monotone natural cubic splines

The use of ROC curves in evaluating a continuous or ordinal biomarker for the discrimination of two populations is commonplace. However, in many settings, marker measurements above or below a certain value cannot be obtained. In this paper, we study the construction of a smooth ROC curve (or surface in the case of three populations) when there is a lower or upper limit of detection. We propose the use of spline models that incorporate monotonicity constraints for the cumulative hazard function of the marker distribution. The proposed technique is computationally stable and simulation results showed a satisfactory performance. Other observed covariates can be also accommodated by this spline-based approach.

Post-transplantation lymphoproliferative disorder after kidney transplantation: report of a nationwide French registry and the development of a new prognostic score

PURPOSE

Post-transplantation lymphoproliferative disorder (PTLD) is associated with significant mortality in kidney transplant recipients. We conducted a prospective survey of the occurrence of PTLD in a French nationwide population of adult kidney recipients over 10 years.

PATIENTS AND METHODS

A French registry was established to cover a nationwide population of transplant recipients and prospectively enroll all adult kidney recipients who developed PTLD between January 1, 1998, and December 31, 2007. Five hundred patient cases of PTLD were referred to the French registry. The prognostic factors for PTLD were investigated using Kaplan-Meier and Cox analyses.

RESULTS

Patients with PTLD had a 5-year survival rate of 53% and 10-year survival rate of 45%. Multivariable analyses revealed that age > 55 years, serum creatinine level > 133 μmol/L, elevated lactate dehydrogenase levels, disseminated lymphoma, brain localization, invasion of serous membranes, monomorphic PTLD, and T-cell PTLD were independent prognostic indicators of poor survival. Considering five variables at diagnosis (age, serum creatinine, lactate dehydrogenase, PTLD localization, and histology), we constructed a prognostic score that classified patients with PTLD as being at low, moderate, high, or very high risk for death. The 10-year survival rate was 85% for low-, 80% for moderate-, 56% for high-, and 0% for very high-risk recipients.

CONCLUSION

This nationwide study highlights the prognostic factors for PTLD and enables the development of a new prognostic score. After validation in an independent cohort, the use of this score should allow treatment strategies to be better tailored to individual patients in the future.

Meta-analysis of diagnostic studies based upon SROC-curves: a mixed model approach using the Lehmann family

Meta-analysis of diagnostic studies experiences the common problem that different studies might not be comparable since they have been using a different cut-off value for the continuous or ordered categorical diagnostic test value defining different regions for which the diagnostic test is defined to be positive. Hence specificities and sensitivities arising from different studies might vary just because the underlying cut-off value had been different. To cope with the cut-off value problem, interest is usually directed towards the receiver operating characteristic (ROC) curve which consists of pairs of sensitivities and false positive rate (1–specificity). In the context of meta-analysis, one pair represents one study and the associated diagram is called SROC curve where the S stands for ‘summary’. The paper will consider—as a novel approach—modelling SROC curves with the Lehmann family that assumes log-sensitivity is proportional to the log-false positive rate across studies. The approach allows for study-specific false positive rates which are treated as (infinitely many) nuisance parameters and eliminated by means of the profile likelihood. The adjusted profile likelihood turns out to have a simple univariate Gaussian structure which is ultimately used for building inference for the parameter of the Lehmann family. The Lehmann model is further extended by allowing the constant of proportionality to vary across studies to cope with unobserved heterogeneity. The simple Gaussian form of the adjusted profile likelihood allows this extension easily as a form of a mixed model in which unobserved heterogeneity is incorporated by means of a normal random effect. Some meta-analytic applications on diagnostic studies including brain natriuretic peptides for heart failure, alcohol use disorder identification test (AUDIT) and the consumption part of AUDIT for detection of unhealthy alcohol use as well as the mini-mental state examination for cognitive disorders are discussed to illustrate the methodology.