Developing a cancer-specific trigger tool to identify adverse events using administrative data

238

Background: “Trigger tools” identify complications of care and potential patient safety hazards. However, attempts to create triggers that flag treatment-related complications in oncology have been largely unsuccessful. To address this problem, the authors built a set of claims-based oncology-specific triggers based on a promising pilot study conducted at Memorial Sloan-Kettering Cancer Center. Methods: We selected subjects from the OptumLabs data warehouse, a repository of > 160 million de-identified patients drawn from commercial claims. The cohort included patients with breast, colorectal, lung, and prostate cancer undergoing an initial course of cancer-directed therapy from 2008-14. Using ICD and CPT codes, we defined 16 oncology-specific triggers drawn from the pilot study, all with PPVs ≥50%. Triggers included events such as neutropenic fever, abnormal serum potassium or bicarbonate, and initiation of therapeutic anticoagulation. To distinguish treatment-related complications from other comorbidities, we required a logical and temporal relationship between a treatment and the associated trigger. We tabulated the prevalence of cancer triggers by cancer type and metastatic status during a one-year follow up period and created multivariate logistic regression models to examine the association of triggered cases with one-year mortality. Results: The cohort comprised 369,354 unique subjects including 29% with metastatic disease. The prevalence of triggered events was greatest among non-metastatic patients with lung (33%) and colorectal (21%) cancers, and among those with metastatic disease. The most common triggers included abnormal chemistry tests, blood transfusions, hypoxemia, and chest CT following radiation therapy. The mortality rate was substantially higher among patients with at least one trigger compared to patients with none. Experiencing at least one cancer-specific trigger increased the one-year risk of death by 1.69 (95% CI 1.28-2.24). Conclusions: Oncology-specific triggers provide researchers a promising method for studying patient safety in cancer care.

Targeted Incentive Programs For Lung Cancer Screening Can Improve Population Health And Economic Efficiency

Because an intervention's clinical benefit depends on who receives it, a key to improving the efficiency of lung cancer screening with low-dose computed tomography (LDCT) is to incentivize its use among the current or former smokers who are most likely to benefit from it. Despite its clinical advantages and cost-effectiveness, only 3.9 percent of the eligible population underwent LDCT screening in 2015. Using individual lung cancer mortality risk, we developed a policy simulation model to explore the potential impact of implementing risk-targeted incentive programs, compared to either implementing untargeted incentive programs or doing nothing. We found that compared to the status quo, an untargeted incentive program that increased overall LDCT screening from 3,900 (baseline) to 10,000 per 100,000 eligible people would save 12,300 life-years and accrue a net monetary benefit (NMB) of $771 million over a lifetime horizon. Increasing screening by the same amount but targeting higher-risk people would yield an additional 2,470-6,600 life-years and an additional $210-$560 million NMB, depending on the extent of the risk-targeting. Risk-targeted incentive programs could include provider-level bonuses, health plan premium subsidies, and smoking cessation programs to maximize their impact. As clinical medicine becomes more personalized, targeting and incentivizing higher-risk people will help enhance population health and economic efficiency.

Models with interactions overestimated heterogeneity of treatment effects and were prone to treatment mistargeting

OBJECTIVES

We aimed to compare the performance of different regression modeling approaches for the prediction of heterogeneous treatment effects.

STUDY DESIGN AND SETTING

We simulated trial samples (n = 3,600; 80% power for a treatment odds ratio of 0.8) from a superpopulation (N = 1,000,000) with 12 binary risk predictors, both without and with six true treatment interactions. We assessed predictions of treatment benefit for four regression models: a "risk model" (with a constant effect of treatment assignment) and three "effect models" (including interactions of risk predictors with treatment assignment). Three novel performance measures were evaluated: calibration for benefit (i.e., observed vs. predicted risk difference in treated vs. untreated), discrimination for benefit, and prediction error for benefit.

RESULTS

The risk modeling approach was well-calibrated for benefit, whereas effect models were consistently overfit, even with doubled sample sizes. Penalized regression reduced miscalibration of the effect models considerably. In terms of discrimination and prediction error, the risk modeling approach was superior in the absence of true treatment effect interactions, whereas penalized regression was optimal in the presence of true treatment interactions.

CONCLUSION

A risk modeling approach yields models consistently well calibrated for benefit. Effect modeling may improve discrimination for benefit in the presence of true interactions but is prone to overfitting. Hence, effect models-including only plausible interactions-should be fitted using penalized regression.

Personalized Decision Making in Early Stage Breast Cancer: Applying Clinical Prediction Models for Anthracycline Cardiotoxicity and Breast Cancer Mortality Demonstrates Substantial Heterogeneity of Benefit-Harm Trade-off

BACKGROUND

Anthracycline agents can cause cardiotoxicity. We used multivariable risk prediction models to identify a subset of patients with breast cancer at high risk of cardiotoxicity, for whom the harms of anthracycline chemotherapy may balance or exceed the benefits.

PATIENTS AND METHODS

A clinical prediction model for anthracycline cardiotoxicity was created in 967 patients with human epidermal growth factor receptor-negative breast cancer treated with doxorubicin in the ECOG-ACRIN study E5103. Cardiotoxicity was defined as left ventricular ejection fraction (LVEF) decline of ≥ 10% to < 50% and/or a centrally adjudicated clinical heart failure diagnosis. Patient-specific incremental absolute benefit of anthracyclines (compared with non-anthracycline taxane chemotherapy) was estimated using the PREDICT model to assess breast cancer mortality risk.

RESULTS

Of the 967 women who initiated therapy, 51 (5.3%) developed cardiotoxicity (12 with clinical heart failure). In a multivariate model, increasing age (odds ratio [OR], 1.04; 95% confidence interval [CI], 1.01-1.08), higher body mass index (OR, 1.06; 95% CI, 1.02-1.10), and lower baseline LVEF (OR, 0.93; 95% CI, 0.89-0.98) at baseline were significantly associated with cardiotoxicity. The concordance statistic of the risk model was 0.70 (95% CI, 0.63-0.77). In patients with low anticipated treatment benefit (n = 176) from the addition of anthracycline (< 2% absolute risk difference of breast cancer mortality at 10 years), 16 (9%) of 176 had a > 10% risk of cardiotoxicity and 61 (35%) of 176 had a 5% to 10% risk of cardiotoxicity at 1 year.

CONCLUSION

Older age, higher body mass index, and lower baseline LVEF were associated with increased risk of cardiotoxicity. We identified a subgroup with low predicted absolute benefit of anthracyclines but with high predicted risk of cardiotoxicity. Additional studies are needed incorporating long-term cardiac outcomes and cardiotoxicity model external validation prior to implementation in routine clinical practice.

Natural Language Processing for the Identification of Silent Brain Infarcts From Neuroimaging Reports

Background

Silent brain infarction (SBI) is defined as the presence of 1 or more brain lesions, presumed to be because of vascular occlusion, found by neuroimaging (magnetic resonance imaging or computed tomography) in patients without clinical manifestations of stroke. It is more common than stroke and can be detected in 20% of healthy elderly people. Early detection of SBI may mitigate the risk of stroke by offering preventative treatment plans. Natural language processing (NLP) techniques offer an opportunity to systematically identify SBI cases from electronic health records (EHRs) by extracting, normalizing, and classifying SBI-related incidental findings interpreted by radiologists from neuroimaging reports.

Objective

This study aimed to develop NLP systems to determine individuals with incidentally discovered SBIs from neuroimaging reports at 2 sites: Mayo Clinic and Tufts Medical Center.

Methods

Both rule-based and machine learning approaches were adopted in developing the NLP system. The rule-based system was implemented using the open source NLP pipeline MedTagger, developed by Mayo Clinic. Features for rule-based systems, including significant words and patterns related to SBI, were generated using pointwise mutual information. The machine learning models adopted convolutional neural network (CNN), random forest, support vector machine, and logistic regression. The performance of the NLP algorithm was compared with a manually created gold standard. The gold standard dataset includes 1000 radiology reports randomly retrieved from the 2 study sites (Mayo and Tufts) corresponding to patients with no prior or current diagnosis of stroke or dementia. 400 out of the 1000 reports were randomly sampled and double read to determine interannotator agreements. The gold standard dataset was equally split to 3 subsets for training, developing, and testing.

Results

Among the 400 reports selected to determine interannotator agreement, 5 reports were removed due to invalid scan types. The interannotator agreements across Mayo and Tufts neuroimaging reports were 0.87 and 0.91, respectively. The rule-based system yielded the best performance of predicting SBI with an accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of 0.991, 0.925, 1.000, 1.000, and 0.990, respectively. The CNN achieved the best score on predicting white matter disease (WMD) with an accuracy, sensitivity, specificity, PPV, and NPV of 0.994, 0.994, 0.994, 0.994, and 0.994, respectively.

Conclusions

We adopted a standardized data abstraction and modeling process to developed NLP techniques (rule-based and machine learning) to detect incidental SBIs and WMDs from annotated neuroimaging reports. Validation statistics suggested a high feasibility of detecting SBIs and WMDs from EHRs using NLP.

Patients' responses to incidentally discovered silent brain infarcts - a qualitative study

Background

Incidentally discovered silent brain infarcts (id-SBIs) are an understudied condition with probable clinical significance, but it is not known how patients respond to or prioritize this condition. We sought to assess reporting of id-SBIs and how patients approach their diagnosis.

Methods

Patients with id-SBIs were identified from sequential scans between 12/2015–5/2016, were referred by treating clinicians, or self-referred for the study. This study used qualitative semi-structured interviews. Purposeful sampling was used to achieve diversity in acuity, setting, and recruitment strategy. Interviews were audio-recorded and transcribed. A constant comparative method was used to develop a coding schema, find consensus, and iteratively explore emergent themes until thematic saturation was achieved.

Results

Only 10 of 102 patients prospectively identified by neuroimaging were informed of the imaging findings. Twelve participants in total were interviewed. Among the study participants, the primary themes were cognitive, emotional, and behavioral responses to diagnostic, prognostic, and therapeutic uncertainty regarding id-SBIs. Clinicians described id-SBIs to participants as an ambiguous condition. Participants feared potential consequences of id-SBIs, including symptomatic stroke, dementia, and disability. Participants attempted to reduce uncertainty with strategies including equating id-SBIs with symptomatic stroke, self-education about stroke, and seeking second opinions.

Conclusion

Participants considered id-SBIs to be a serious medical condition. Ambiguous counseling by clinicians on id-SBIs provoked or failed to attenuate fear, leading to participants adopting strategies aimed at reducing uncertainty.

Electronic supplementary material

The online version of this article (10.1186/s41687-019-0112-7) contains supplementary material, which is available to authorized users.

Abstract 111: External Validations of Clinical Predictive Models for Patients with Sudden Cardiac Arrest

Background: Sudden cardiac arrest is associated with high morbidity and mortality. A number of clinical predictive models (CPMs) have been introduced to help with patient-specific prognostication of survival and neurologic outcomes. Here, we evaluate the performance of these CPMs with attention to key variables and models with rigorous validation.

Methods: We performed a systematic review and citation search of cardiac arrest CPMs in the Tufts Predictive Analytics and Comparative Effectiveness (PACE) CPM Registry and identified external validations of these models through September 2018. We extracted information on CPM performance from both original reports and external validations. For external validations, we calculated the percent change in discrimination.

Results: We identified 65 unique cardiac arrest CPMs (median n=611, IQR=781) published between 1981 and 2018. Thirty-eight of the 65 models (58%) reported a c-statistic (ROC AUC) (median=0.82, IQR=0.09). The median number of predictive variables was 4 (IQR=3), and the three most common variables were 1) initial cardiac rhythm (n=41 of 65; 63%), 2) age (n=33 of 65; 51%), and 3) witnessed arrest (n=24 of 65; 37%). We identified external validations for 26 of 65 (40%) CPMs. All validations (n=44) reported discrimination, but only 21 of 44 (48%) validations reported information regarding calibration. Of the CPMs that reported discrimination and were externally validated at least once (n=15), we noted a median percent change in discrimination of -1.9% (IQR = 11.3%). The three most rigorously validated cardiac arrest CPMs were 1) the OHCA score (n=5, median AUC=0.79), 2) the CAHP score (n=3, median AUC=0.85), and 3) the GO-FAR score (n=3, median AUC=0.82).

Conclusions: While few cardiac arrest CPMs have been externally validated, those that have demonstrate stable discriminatory power.

Early withdrawal of life support after resuscitation from cardiac arrest is common and may result in additional deaths

AIM

"Early" withdrawal of life support therapies (eWLST) within the first 3 calendar days after resuscitation from cardiac arrest (CA) is discouraged. We evaluated a prospective multicenter registry of patients admitted to hospitals after resuscitation from CA to determine predictors of eWLST and estimate its impact on outcomes.

METHODS

CA survivors enrolled from 2012-2017 in the International Cardiac Arrest Registry (INTCAR) were included. We developed a propensity score for eWLST and matched a cohort with similar probabilities of eWLST who received ongoing care. The incidence of good outcome (Cerebral Performance Category of 1 or 2) was measured across deciles of eWLST in the matched cohort.

RESULTS

2688 patients from 24 hospitals were included. Median ischemic time was 20 (IQR 11, 30) minutes, and 1148 (43%) had an initial shockable rhythm. Withdrawal of life support occurred in 1162 (43%) cases, with 459 (17%) classified as eWLST. Older age, initial non-shockable rhythm, increased ischemic time, shock on admission, out-of-hospital arrest, and admission in the United States were each independently associated with eWLST. All patients with eWLST died, while the matched cohort, good outcome occurred in 21% of patients. 19% of patients within the eWLST group were predicted to have a good outcome, had eWLST not occurred.

CONCLUSIONS

Early withdrawal of life support occurs frequently after cardiac arrest. Although the mortality of patients matched to those with eWLST was high, these data showed excess mortality with eWLST.

Scoring System to Optimize Pioglitazone Therapy After Stroke Based on Fracture Risk

Background and Purpose- The insulin sensitizer, pioglitazone, reduces cardiovascular risk in patients after an ischemic stroke or transient ischemic attack but increases bone fracture risk. We conducted a secondary analysis of the IRIS trial (Insulin Resistance Intervention After Stroke) to assess the effect of pretreatment risk for fracture on the net benefits of pioglitazone therapy. Methods- IRIS was a randomized placebo-controlled trial of pioglitazone that enrolled patients with insulin resistance but without diabetes mellitus within 180 days of an ischemic stroke or transient ischemic attack. Participants were recruited at 179 international centers from February 2005 to January 2013 and followed for a median of 4.8 years. Fracture risk models were developed from patient characteristics at entry. Within fracture risk strata, we quantified the effects of pioglitazone compared with placebo by estimating the relative risks and absolute 5-year risk differences for fracture and stroke or myocardial infarction. Results- The fracture risk model included points for age, race-ethnicity, sex, body mass index, disability, and medications. The relative increment in fracture risk with pioglitazone was similar in the lower ( Collapse

Key Words

• fractures, bone
• insulin resistance
• myocardial infarction
• pioglitazone
• risk
• stroke

Collapse

MESH Headings Collapse

Grants

• U01 NS044876 NINDS NIH HHS

Collapse

Personalized evidence based medicine: predictive approaches to heterogeneous treatment effects

The use of evidence from clinical trials to support decisions for individual patients is a form of "reference class forecasting": implicit predictions for an individual are made on the basis of outcomes in a reference class of "similar" patients treated with alternative therapies. Evidence based medicine has generally emphasized the broad reference class of patients qualifying for a trial. Yet patients in a trial (and in clinical practice) differ from one another in many ways that can affect the outcome of interest and the potential for benefit. The central goal of personalized medicine, in its various forms, is to narrow the reference class to yield more patient specific effect estimates to support more individualized clinical decision making. This article will review fundamental conceptual problems with the prediction of outcome risk and heterogeneity of treatment effect (HTE), as well as the limitations of conventional (one-variable-at-a-time) subgroup analysis. It will also discuss several regression based approaches to "predictive" heterogeneity of treatment effect analysis, including analyses based on "risk modeling" (such as stratifying trial populations by their risk of the primary outcome or their risk of serious treatment-related harms) and analysis based on "effect modeling" (which incorporates modifiers of relative effect). It will illustrate these approaches with clinical examples and discuss their respective strengths and vulnerabilities.

Population-Based Validation of a Clinical Prediction Model for Congenital Diaphragmatic Hernias

OBJECTIVE

To examine the external validity of a well-known congenital diaphragmatic hernia (CDH) clinical prediction model using a population-based cohort.

STUDY DESIGN

Newborns with CDH born in California between 2007 and 2012 were extracted from the Vital Statistics and Patient Discharge Data Linked Files. The total CDH risk score was calculated according to the Congenital Diaphragmatic Hernia Study Group (CDHSG) model using 5 independent predictors: birth weight, 5-minute Apgar, pulmonary hypertension, major cardiac defects, and chromosomal anomalies. CDHSG model performance on our cohort was validated for discrimination and calibration.

RESULTS

A total of 705 newborns with CDH were extracted from 3 213 822 live births. Newborns with CDH were delivered in 150 different hospitals, whereas only 28 hospitals performed CDH repairs (1-85 repairs per hospital). The observed mortality for low-, intermediate-, and high-risk groups were 7.7%, 34.3%, and 54.7%, and predicted mortality for these groups were 4.0%, 23.2%, and 58.5%. The CDHSG model performed well within our cohort with a c-statistic of 0.741 and good calibration.

CONCLUSIONS

We successfully validated the CDHSG prediction model using an external population-based cohort of newborns with CDH in California. This cohort may be used to investigate hospital volume-outcome relationships and guide policy development.

Role of Secondary Prophylaxis With Valganciclovir in the Prevention of Recurrent Cytomegalovirus Disease in Solid Organ Transplant Recipients

Background

Cytomegalovirus (CMV) is a major contributor to morbidity and mortality in solid organ transplant recipients (SOTRs). Ganciclovir and valganciclovir are highly effective antiviral drugs with a well-established role in primary prophylaxis and treatment of CMV disease. Our objective in this study was to examine the effect of secondary prophylaxis (SP) on the risk of relapse in SOTRs following an episode of CMV disease.

Methods

We performed a retrospective cohort study of SOTRs from 1995 to 2015 and used propensity score-based inverse probability of treatment weighting methodology to control for confounding by indication. A weighted Cox model was created to determine the effect of SP on time to relapse within 1 year of treatment completion.

Results

Fifty-two heart, 34 liver, 79 kidney, and 5 liver-kidney transplant recipients who completed treatment for an episode of CMV infection/disease were included. A total of 120 (70.6%) received SP (median duration, 61 days; range, 5-365) and 39 (23%) relapsed. SP was protective against relapse from 0 to 6 weeks following treatment completion (hazard ratio [HR], 0.19; 95% confidence interval [CI], 0.05-0.69). However, after 6 weeks, risk of relapse did not significantly differ between the 2 groups (HR, 1.18; 95% CI, 0.46-2.99).

Conclusions

Our findings demonstrate that use of SP following treatment of CMV disease did not confer long-term protection against relapse, although it did delay relapse while patients were receiving antivirals. This suggests that SP has limited clinical utility in the overall prevention of recurrent CMV disease.

Digoxin Benefit Varies by Risk of Heart Failure Hospitalization: Applying the Tufts MC HF Risk Model

BACKGROUND

Digoxin has been shown to reduce heart failure hospitalizations with a neutral effect on mortality. It is unknown whether there is heterogeneity of treatment effect for digitalis therapy according to predicted risk of heart failure hospitalization.

METHODS AND RESULTS

We conducted a post hoc analysis of the Digitalis Investigator Group (DIG) studies, randomized controlled trials of digoxin vs placebo in participants with heart failure and left ventricular ejection fraction ≤45% (main DIG study, n = 6800) or >45% (ancillary DIG study, n = 988). Using a previously derived multistate model to risk-stratify DIG study participants, we determined the differential treatment effect on hospitalization and mortality outcomes. There was a 13% absolute reduction in the risk of any heart failure hospitalizations (39% vs 52%; odds ratio 0.58; 95% confidence interval 0.47-0.71) in the digoxin vs placebo arms in the highest-risk quartile, compared with a 3% absolute risk reduction for any heart failure hospitalization (17% vs 20%; odds ratio 0.84; 95% confidence interval, 0.66-1.08) in the lowest-risk quartile. There were 12 fewer total all-cause hospitalizations per 100 person-years in the highest-risk quartile compared with an increase of 8 hospitalizations per 100 person-years in the lowest-risk quartile. There was neutral effect of digoxin on mortality in all risk quartiles and no interaction between baseline risk and the effect of digoxin on mortality (P = .94).

CONCLUSIONS

Participants in the DIG study at higher risk of hospitalization as identified by a multistate model were considerably more likely to benefit from digoxin therapy to reduce heart failure hospitalization.

Evaluation of person-level heterogeneity of treatment effects in published multiperson N-of-1 studies: systematic review and reanalysis

OBJECTIVE

Individual patients with the same condition may respond differently to similar treatments. Our aim is to summarise the reporting of person-level heterogeneity of treatment effects (HTE) in multiperson N-of-1 studies and to examine the evidence for person-level HTE through reanalysis.

STUDY DESIGN

Systematic review and reanalysis of multiperson N-of-1 studies.

DATA SOURCES

Medline, Cochrane Controlled Trials, EMBASE, Web of Science and review of references through August 2017 for N-of-1 studies published in English.

STUDY SELECTION

N-of-1 studies of pharmacological interventions with at least two subjects.

DATA SYNTHESIS

Citation screening and data extractions were performed in duplicate. We performed statistical reanalysis testing for person-level HTE on all studies presenting person-level data.

RESULTS

We identified 62 multiperson N-of-1 studies with at least two subjects. Statistical tests examining HTE were described in only 13 (21%), of which only two (3%) tested person-level HTE. Only 25 studies (40%) provided person-level data sufficient to reanalyse person-level HTE. Reanalysis using a fixed effect linear model identified statistically significant person-level HTE in 8 of the 13 studies (62%) reporting person-level treatment effects and in 8 of the 14 studies (57%) reporting person-level outcomes.

CONCLUSIONS

Our analysis suggests that person-level HTE is common and often substantial. Reviewed studies had incomplete information on person-level treatment effects and their variation. Improved assessment and reporting of person-level treatment effects in multiperson N-of-1 studies are needed.

Effect of Dysphagia Screening Strategies on Clinical Outcomes After Stroke: A Systematic Review for the 2018 Guidelines for the Early Management of Patients With Acute Ischemic Stroke

Introduction—

Dysphagia screening protocols have been recommended to identify patients at risk for aspiration. The American Heart Association convened an evidence review committee to systematically review evidence for the effectiveness of dysphagia screening protocols to reduce the risk of pneumonia, death, or dependency after stroke.

Methods—

The Medline, Embase, and Cochrane databases were searched on November 1, 2016, to identify randomized controlled trials (RCTs) comparing dysphagia screening protocols or quality interventions with increased dysphagia screening rates and reporting outcomes of pneumonia, death, or dependency.

Results—

Three RCTs were identified. One RCT found that a combined nursing quality improvement intervention targeting fever and glucose management and dysphagia screening reduced death and dependency but without reducing the pneumonia rate. Another RCT failed to find evidence that pneumonia rates were reduced by adding the cough reflex to routine dysphagia screening. A smaller RCT randomly assigned 2 hospital wards to a stroke care pathway including dysphagia screening or regular care and found that patients on the stroke care pathway were less likely to require intubation and mechanical ventilation; however, the study was small and at risk for bias.

Conclusions—

There were insufficient RCT data to determine the effect of dysphagia screening protocols on reducing the rates of pneumonia, death, or dependency after stroke. Additional trials are needed to compare the validity, feasibility, and clinical effectiveness of different screening methods for dysphagia.

Using group data to treat individuals: understanding heterogeneous treatment effects in the age of precision medicine and patient-centred evidence

Although often conflated, determining the best treatment for an individual (the task of a doctor) is fundamentally different from determining the average effect of treatment in a population (the purpose of a trial). In this paper, we review concepts of heterogeneity of treatment effects (HTE) essential in providing the evidence base for precision medicine and patient-centred care, and explore some inherent limitations of using group data (e.g. from a randomized trial) to guide treatment decisions for individuals. We distinguish between person-level HTE (i.e. that individuals experience different effects from a treatment) and group-level HTE (i.e. that subgroups have different average treatment effects), and discuss the reference class problem, engendered by the large number of potentially informative subgroupings of a study population (each of which may lead to applying a different estimated effect to the same patient), and the scale dependence of group-level HTE. We also review the limitations of conventional 'one-variable-at-a-time' subgroup analyses and discuss the potential benefits of using more comprehensive subgrouping schemes that incorporate information on multiple variables, such as those based on predicted outcome risk. Understanding the conceptual underpinnings of HTE is critical for understanding how studies can be designed, analysed, and interpreted to better inform individualized clinical decisions.

Risk-Targeted Lung Cancer Screening: A Cost-Effectiveness Analysis

BACKGROUND

Targeting low-dose computed tomography (LDCT) for lung cancer screening to persons at highest risk for lung cancer mortality has been suggested to improve screening efficiency.

OBJECTIVE

To quantify the value of risk-targeted selection for lung cancer screening compared with National Lung Screening Trial (NLST) eligibility criteria.

DESIGN

Cost-effectiveness analysis using a multistate prediction model.

DATA SOURCES

NLST.

TARGET POPULATION

Current and former smokers eligible for lung cancer screening.

TIME HORIZON

Lifetime.

PERSPECTIVE

Health care sector.

INTERVENTION

Risk-targeted versus NLST-based screening.

OUTCOME MEASURES

Incremental 7-year mortality, life expectancy, quality-adjusted life-years (QALYs), costs, and cost-effectiveness of screening with LDCT versus chest radiography at each decile of lung cancer mortality risk.

RESULTS OF BASE-CASE ANALYSIS

Participants at greater risk for lung cancer mortality were older and had more comorbid conditions and higher screening-related costs. The incremental lung cancer mortality benefits during the first 7 years ranged from 1.2 to 9.5 lung cancer deaths prevented per 10 000 person-years for the lowest to highest risk deciles, respectively (extreme decile ratio, 7.9). The gradient of benefits across risk groups, however, was attenuated in terms of life-years (extreme decile ratio, 3.6) and QALYs (extreme decile ratio, 2.4). The incremental cost-effectiveness ratios (ICERs) were similar across risk deciles ($75 000 per QALY in the lowest risk decile to $53 000 per QALY in the highest risk decile). Payers willing to pay $100 000 per QALY would pay for LDCT screening for all decile groups.

RESULTS OF SENSITIVITY ANALYSIS

Alternative assumptions did not substantially alter our findings.

LIMITATION

Our model did not account for all correlated differences between lung cancer mortality risk and quality of life.

CONCLUSIONS

Although risk targeting may improve screening efficiency in terms of early lung cancer mortality per person screened, the gains in efficiency are attenuated and modest in terms of life-years, QALYs, and cost-effectiveness.

PRIMARY FUNDING SOURCE

National Institutes of Health (U01NS086294).

Abstract 97: Accuracy of Prediction Instruments for Diagnosing Large Vessel Occlusion in Persons With Suspected Stroke: A Systematic Review for the 2018 AHA/ASA Guidelines for the Early Management of Patients With Acute Ischemic Stroke

Introduction: Prediction instruments for large vessel occlusion (LVO) have been proposed to identify patients for rapid transport to endovascular thrombectomy (EVT) capable hospitals. This Evidence Review Committee was commissioned by the AHA/ASA to systematically review evidence for the accuracy of LVO prediction instruments.

Methods: Medline, Embase, and Cochrane databases were searched on October 27, 2016. Study quality was assessed using the Quality Assessment of Diagnostic Accuracy (QUADAS)-2 tool.

Results: Thirty-six relevant studies were identified. Most (21/36) recruited patients with confirmed ischemic stroke, with few studies in the pre-hospital setting (4/36) and in populations that included hemorrhagic stroke or stroke mimics (12/36). Most studies had either some risk of bias or unclear risk of bias. Discrimination of LVO, as measured by the c-statistic, mostly ranged from 0.70-0.85. In meta-analysis, no threshold on any instrument predicted LVO with both high sensitivity and specificity (Table). With a positive LVO prediction test, the probability of LVO could be 50% or greater (depending on the LVO prevalence in the population), but the probability of LVO with a negative test could still be 10% or more.

Conclusions: No scale predicted LVO with both high sensitivity and specificity. Systems that use LVO prediction instruments for triage will miss some patients with LVO and milder stroke. More prospective studies are needed in the pre-hospital setting in all patients with suspected stroke, including hemorrhagic stroke and stroke mimics.

Abstract WP173: A Systematic Review of Clinical Prediction Models for Patients With Stroke

Introduction: Clinical prediction models (CPMs) estimate the probability of clinical outcomes and hold the potential to improve decision-making and individualize care. We aim to conduct a systematic study of published CPMs predicting mortality, functional outcome or stroke recurrence for patients with stroke.

Methods: The Tufts PACE CPM Registry is based on a systematic review of cerebrovascular and cardiovascular CPMs published in English-language articles from 1/1990-3/2015, and includes 1084 unique CPMs extracted from 747 articles. CPMs predicting outcomes for patients with stroke were characterized based on index condition (hemorrhagic, ischemic or all stroke) and outcome (mortality, functional outcome or stroke recurrence). We identified the most commonly occurring covariates in models grouped by index condition-outcome pair (I-O pair).

Results: Among 1084 total models in the registry, 116 (11%) predicted mortality, functional outcomes or stroke recurrence among patients with stroke. The top three most frequent models predicted functional outcomes among ischemic stroke patients (n=23), mortality among all stroke patients (n=19), and mortality among patients with hemorrhagic stroke (n=18). The median reported C statistic was 0.84 (among n=78 models reporting this measure). About half (45%) of models reported internal validations, with only 25% reporting external validations. The most commonly occurring covariates in the models were age (77%), stroke severity (51%), and functional status (26%) (see Figure). Neuroimaging findings were included relatively infrequently (21%), but were included in all 9 models predicting functional outcome among hemorrhagic stroke patients.

Conclusions: There is an abundance of CPMs to predict clinically important outcomes in stroke populations. More work is needed to understand how this prognostic information might be used to improve decision making and outcomes for stroke patients.

Patient Variability Seldom Assessed in Cost-effectiveness Studies

BACKGROUND

Cost-effectiveness analysis (CEA) estimates can vary substantially across patient subgroups when patient characteristics influence preferences, outcome risks, treatment effectiveness, life expectancy, or associated costs. However, no systematic review has reported the frequency of subgroup analysis in CEA, what type of heterogeneity they address, and how often heterogeneity influences whether cost-effectiveness ratios exceed or fall below conventional thresholds.

METHODS

We reviewed the CEA literature cataloged in the Tufts Medical Center CEA Registry, a repository describing cost-utility analyses published through 2016. After randomly selecting 200 of 642 articles published in 2014, we ascertained whether each study reported subgroup results and collected data on the defining characteristics of these subgroups. We identified whether any of the CEA subgroup results crossed conventional cost-effectiveness benchmarks (e.g., $100,000 per QALY) and compared characteristics of studies with and without subgroup-specific findings.

RESULTS

Thirty-eight studies (19%) reported patient subgroup results. Articles reporting subgroup analyses were more likely to be US-based, government funded (v. drug industry- or nonprofit foundation-funded) studies, with a focus on primary or secondary (v. tertiary) prevention (P < 0.05 for comparisons). One or more patient characteristics were used to stratify CEA results 68 times within the 38 studies, with most stratifications using one characteristic (n = 47), most commonly age (n = 35). Among the 23 stratifications reported alongside average ratios in US studies, 13 produced subgroup ratios that crossed a conventional CEA ratio benchmark.

CONCLUSIONS

Most CEAs do not report any subgroup results, and those that do most often stratify only by patient age. Over half of the subgroup analyses reported could lead to different value-based decision making for at least some patients.