Optimal Allocation of Gold Standard Testing under Constrained Availability: Application to Assessment of HIV Treatment Failure

Estimation of causal effects of multiple treatments in observational studies with a binary outcome

There is a dearth of robust methods to estimate the causal effects of multiple treatments when the outcome is binary. This paper uses two unique sets of simulations to propose and evaluate the use of Bayesian additive regression trees in such settings. First, we compare Bayesian additive regression trees to several approaches that have been proposed for continuous outcomes, including inverse probability of treatment weighting, targeted maximum likelihood estimator, vector matching, and regression adjustment. Results suggest that under conditions of non-linearity and non-additivity of both the treatment assignment and outcome generating mechanisms, Bayesian additive regression trees, targeted maximum likelihood estimator, and inverse probability of treatment weighting using generalized boosted models provide better bias reduction and smaller root mean squared error. Bayesian additive regression trees and targeted maximum likelihood estimator provide more consistent 95% confidence interval coverage and better large-sample convergence property. Second, we supply Bayesian additive regression trees with a strategy to identify a common support region for retaining inferential units and for avoiding extrapolating over areas of the covariate space where common support does not exist. Bayesian additive regression trees retain more inferential units than the generalized propensity score-based strategy, and shows lower bias, compared to targeted maximum likelihood estimator or generalized boosted model, in a variety of scenarios differing by the degree of covariate overlap. A case study examining the effects of three surgical approaches for non-small cell lung cancer demonstrates the methods.

Constrained binary classification using ensemble learning: an application to cost-efficient targeted PrEP strategies

Binary classification problems are ubiquitous in health and social sciences. In many cases, one wishes to balance two competing optimality considerations for a binary classifier. For instance, in resource-limited settings, an human immunodeficiency virus prevention program based on offering pre-exposure prophylaxis (PrEP) to select high-risk individuals must balance the sensitivity of the binary classifier in detecting future seroconverters (and hence offering them PrEP regimens) with the total number of PrEP regimens that is financially and logistically feasible for the program. In this article, we consider a general class of constrained binary classification problems wherein the objective function and the constraint are both monotonic with respect to a threshold. These include the minimization of the rate of positive predictions subject to a minimum sensitivity, the maximization of sensitivity subject to a maximum rate of positive predictions, and the Neyman-Pearson paradigm, which minimizes the type II error subject to an upper bound on the type I error. We propose an ensemble approach to these binary classification problems based on the Super Learner methodology. This approach linearly combines a user-supplied library of scoring algorithms, with combination weights and a discriminating threshold chosen to minimize the constrained optimality criterion. We then illustrate the application of the proposed classifier to develop an individualized PrEP targeting strategy in a resource-limited setting, with the goal of minimizing the number of PrEP offerings while achieving a minimum required sensitivity. This proof of concept data analysis uses baseline data from the ongoing Sustainable East Africa Research in Community Health study. Copyright © 2017 John Wiley & Sons, Ltd.

Improved HIV-1 Viral Load Monitoring Capacity Using Pooled Testing With Marker-Assisted Deconvolution

OBJECTIVE

Improve pooled viral load (VL) testing to increase HIV treatment monitoring capacity, particularly relevant for resource-limited settings.

DESIGN

We developed marker-assisted mini-pooling with algorithm (mMPA), a new VL pooling deconvolution strategy that uses information from low-cost, routinely collected clinical markers to determine an efficient order of sequential individual VL testing and dictates when the sequential testing can be stopped.

METHODS

We simulated the use of pooled testing to ascertain virological failure status on 918 participants from 3 studies conducted at the Academic Model Providing Access to Healthcare in Eldoret, Kenya, and estimated the number of assays needed when using mMPA and other pooling methods. We also evaluated the impact of practical factors, such as specific markers used, prevalence of virological failure, pool size, VL measurement error, and assay detection cutoffs on mMPA, other pooling methods, and single testing.

RESULTS

Using CD4 count as a marker to assist deconvolution, mMPA significantly reduces the number of VL assays by 52% [confidence interval (CI): 48% to 57%], 40% (CI: 38% to 42%), and 19% (CI: 15% to 22%) compared with individual testing, simple mini-pooling, and mini-pooling with algorithm, respectively. mMPA has higher sensitivity and negative/positive predictive values than mini-pooling with algorithm, and comparable high specificity. Further improvement is achieved with additional clinical markers, such as age and time on therapy, with or without CD4 values. mMPA performance depends on prevalence of virological failure and pool size but is insensitive to VL measurement error and VL assay detection cutoffs.

CONCLUSIONS

mMPA can substantially increase the capacity of VL monitoring.

Implementation and Operational Research: Risk Charts to Guide Targeted HIV-1 Viral Load Monitoring of ART: Development and Validation in Patients From Resource-Limited Settings

BACKGROUND

HIV-1 RNA viral load (VL) testing is recommended to monitor antiretroviral therapy (ART) but not available in many resource-limited settings. We developed and validated CD4-based risk charts to guide targeted VL testing.

METHODS

We modeled the probability of virologic failure up to 5 years of ART based on current and baseline CD4 counts, developed decision rules for targeted VL testing of 10%, 20%, or 40% of patients in 7 cohorts of patients starting ART in South Africa, and plotted cutoffs for VL testing on colour-coded risk charts. We assessed the accuracy of risk chart-guided VL testing to detect virologic failure in validation cohorts from South Africa, Zambia, and the Asia-Pacific.

RESULTS

In total, 31,450 adult patients were included in the derivation and 25,294 patients in the validation cohorts. Positive predictive values increased with the percentage of patients tested: from 79% (10% tested) to 98% (40% tested) in the South African cohort, from 64% to 93% in the Zambian cohort, and from 73% to 96% in the Asia-Pacific cohort. Corresponding increases in sensitivity were from 35% to 68% in South Africa, from 55% to 82% in Zambia, and from 37% to 71% in Asia-Pacific. The area under the receiver operating curve increased from 0.75 to 0.91 in South Africa, from 0.76 to 0.91 in Zambia, and from 0.77 to 0.92 in Asia-Pacific.

CONCLUSIONS

CD4-based risk charts with optimal cutoffs for targeted VL testing maybe useful to monitor ART in settings where VL capacity is limited.

Classification in two-stage screening

Decision theory is applied to the problem of setting thresholds in medical screening when it is organised in two stages. In the first stage that involves a less expensive procedure that can be applied on a mass scale, an individual is classified as a negative or a likely positive. In the second stage, the likely positives are subjected to another test that classifies them as (definite) positives or negatives. The second-stage test is more accurate, but also more expensive and more involved, and so there are incentives to restrict its application. Robustness of the method with respect to the parameters, some of which have to be set by elicitation, is assessed by sensitivity analysis.

Super Learner Analysis of Electronic Adherence Data Improves Viral Prediction and May Provide Strategies for Selective HIV RNA Monitoring

OBJECTIVE

Regular HIV RNA testing for all HIV-positive patients on antiretroviral therapy (ART) is expensive and has low yield since most tests are undetectable. Selective testing of those at higher risk of failure may improve efficiency. We investigated whether a novel analysis of adherence data could correctly classify virological failure and potentially inform a selective testing strategy.

DESIGN

Multisite prospective cohort consortium.

METHODS

We evaluated longitudinal data on 1478 adult patients treated with ART and monitored using the Medication Event Monitoring System (MEMS) in 16 US cohorts contributing to the MACH14 consortium. Because the relationship between adherence and virological failure is complex and heterogeneous, we applied a machine-learning algorithm (Super Learner) to build a model for classifying failure and evaluated its performance using cross-validation.

RESULTS

Application of the Super Learner algorithm to MEMS data, combined with data on CD4 T-cell counts and ART regimen, significantly improved classification of virological failure over a single MEMS adherence measure. Area under the receiver operating characteristic curve, evaluated on data not used in model fitting, was 0.78 (95% confidence interval: 0.75 to 0.80) and 0.79 (95% confidence interval: 0.76 to 0.81) for failure defined as single HIV RNA level >1000 copies per milliliter or >400 copies per milliliter, respectively. Our results suggest that 25%-31% of viral load tests could be avoided while maintaining sensitivity for failure detection at or above 95%, for a cost savings of $16-$29 per person-month.

CONCLUSIONS

Our findings provide initial proof of concept for the potential use of electronic medication adherence data to reduce costs through behavior-driven HIV RNA testing.