Abstract P604: Incidentally-Discovered Asymptomatic Ischemic Lesions Identified Using Natural Language Processing Are Strong Risk Markers for Future Ischemic Stroke

Background: White matter disease (WMD) and silent brain infarction (SBI) are known to be risk markers for stroke. Nevertheless, the predictive value of these changes when seen incidentally on routinely-obtained neuroimages is unknown.

Methods: In this retrospective cohort study, Kaiser Permanente-Southern California health plan enrollees aged ≥ 50 years old with a brain CT or MRI scan between 2009-2019 and without a prior history of ischemic stroke, transient ischemic attack, or dementia were identified. Natural language processing (NLP) was used to identify patients with SBI and WMD on the index neuroimaging report. We used Cox proportional hazards to estimate the risk of future ischemic stroke associated with the presence of SBI and of WMD, controlling for major stroke risk factors.

Results: Among 262,875 individuals receiving brain neuroimaging, 13,154 (5.0%) and 78,330 (29.8%) had SBI and WMD, respectively. The Table below summarizes the crude stroke incidence rates. The crude hazard ratio (HR) was 3.40 (95% CI 3.25-3.56) for SBI and 2.63 (95% CI 2.54-2.71) for WMD. In the multivariable model controlling for all major stroke risk factors, the effect of SBI was found to be stronger in younger versus older patients and for MRI- versus CT-discovered lesions. With MRI, the average adjusted HR over time was 2.95 (95% CI 2.53-3.44) for those < age 65 and 2.15 (95% CI 1.91-2.41) for those ≥ age 65. With CT scan, the average adjusted HR over time was 2.48 (95%CI 2.19-2.81) for those < age 65 and 1.81 (95% CI 1.71-1.91) for those ≥ age 65. The adjusted HR associated with a finding of WMD was 1.76 (95% CI 1.69-1.82) and was not modified by age or imaging modality. The effect of SBI decreased gradually over time, while the effect of WMD remained constant.

Conclusion: Incidentally-discovered SBI and WMD are common in patients ≥ age 50 and are associated with substantial increases in the risk of subsequent symptomatic stroke. The findings may represent an opportunity for stroke prevention.

Proposal for Updated Nomenclature and Classification of Potential Causative Mechanism in Patent Foramen Ovale-Associated Stroke

Importance

Recent epidemiologic and therapeutic advances have transformed understanding of the role of and therapeutic approach to patent foramen ovale (PFO) in ischemic stroke. Patent foramen ovale is likely responsible for approximately 5% of all ischemic strokes and 10% of those occurring in young and middle-aged adults.

Observations

Randomized clinical trials have demonstrated that, to prevent recurrent ischemic stroke in patients with PFO and an otherwise-cryptogenic index ischemic stroke, PFO closure is superior to antiplatelet medical therapy alone; these trials have provided some evidence that, among medical therapy options, anticoagulants may be more effective than antiplatelet agents.

Conclusions and Relevance

These new data indicate a need to update classification schemes of causative mechanisms in stroke, developed in an era in which an association between PFO and stroke was viewed as uncertain. We propose a revised general nomenclature and classification framework for PFO-associated stroke and detailed revisions for the 3 major stroke subtyping algorithms in wide use.

Precision Health Analytics With Predictive Analytics and Implementation Research: JACC State-of-the-Art Review

Emerging data science techniques of predictive analytics expand the quality and quantity of complex data relevant to human health and provide opportunities for understanding and control of conditions such as heart, lung, blood, and sleep disorders. To realize these opportunities, the information sources, the data science tools that use the information, and the application of resulting analytics to health and health care issues will require implementation research methods to define benefits, harms, reach, and sustainability; and to understand related resource utilization implications to inform policymakers. This JACC State-of-the-Art Review is based on a workshop convened by the National Heart, Lung, and Blood Institute to explore predictive analytics in the context of implementation science. It highlights precision medicine and precision public health as complementary and compelling applications of predictive analytics, and addresses future research and training endeavors that might further foster the application of predictive analytics in clinical medicine and public health.

Redevelopment and validation of the SYNTAX score II to individualise decision making between percutaneous and surgical revascularisation in patients with complex coronary artery disease: secondary analysis of the multicentre randomised controlled SYNTAXES trial with external cohort validation

BACKGROUND

Randomised controlled trials are considered the gold standard for testing the efficacy of novel therapeutic interventions, and typically report the average treatment effect as a summary result. As the result of treatment can vary between patients, basing treatment decisions for individual patients on the overall average treatment effect could be suboptimal. We aimed to develop an individualised decision making tool to select an optimal revascularisation strategy in patients with complex coronary artery disease.

METHODS

The SYNTAX Extended Survival (SYNTAXES) study is an investigator-driven extension follow-up of a multicentre, randomised controlled trial done in 85 hospitals across 18 North American and European countries between March, 2005, and April, 2007. Patients with de-novo three-vessel and left main coronary artery disease were randomly assigned (1:1) to either the percutaneous coronary intervention (PCI) group or coronary artery bypass grafting (CABG) group. The SYNTAXES study ascertained 10-year all-cause deaths. We used Cox regression to develop a clinical prognostic index for predicting death over a 10-year period, which was combined, in a second stage, with assigned treatment (PCI or CABG) and two prespecified effect-modifiers, which were selected on the basis of previous evidence: disease type (three-vessel disease or left main coronary artery disease) and anatomical SYNTAX score. We used similar techniques to develop a model to predict the 5-year risk of major adverse cardiovascular events (defined as a composite of all-cause death, non-fatal stroke, or non-fatal myocardial infarction) in patients receiving PCI or CABG. We then assessed the ability of these models to predict the risk of death or a major adverse cardiovascular event, and their differences (ie, the estimated benefit of CABG versus PCI by calculating the absolute risk difference between the two strategies) by cross-validation with the SYNTAX trial (n=1800 participants) and external validation in the pooled population (n=3380 participants) of the FREEDOM, BEST, and PRECOMBAT trials. The concordance (C)-index was used to measure discriminative ability, and calibration plots were used to assess the degree of agreement between predictions and observations.

FINDINGS

At cross-validation, the newly developed SYNTAX score II, termed SYNTAX score II 2020, showed a helpful discriminative ability in both treatment groups for predicting 10-year all-cause deaths (C-index=0·73 [95% CI 0·69-0·76] for PCI and 0·73 [0·69-0·76] for CABG) and 5-year major adverse cardiovascular events (C-index=0·65 [0·61-0·69] for PCI and C-index=0·71 [0·67-0·75] for CABG). At external validation, the SYNTAX score II 2020 showed helpful discrimination (C-index=0·67 [0·63-0·70] for PCI and C-index=0·62 [0·58-0·66] for CABG) and good calibration for predicting 5-year major adverse cardiovascular events. The estimated treatment benefit of CABG over PCI varied substantially among patients in the trial population, and the benefit predictions were well calibrated.

INTERPRETATION

The SYNTAX score II 2020 for predicting 10-year deaths and 5-year major adverse cardiovascular events can help to identify individuals who will benefit from either CABG or PCI, thereby supporting heart teams, patients, and their families to select optimal revascularisation strategies.

FUNDING

The German Heart Research Foundation and the Patient-Centered Outcomes Research Institute.

Predictive approaches to heterogeneous treatment effects: a scoping review

BACKGROUND

Recent evidence suggests that there is often substantial variation in the benefits and harms across a trial population. We aimed to identify regression modeling approaches that assess heterogeneity of treatment effect within a randomized clinical trial.

METHODS

We performed a literature review using a broad search strategy, complemented by suggestions of a technical expert panel.

RESULTS

The approaches are classified into 3 categories: 1) Risk-based methods (11 papers) use only prognostic factors to define patient subgroups, relying on the mathematical dependency of the absolute risk difference on baseline risk; 2) Treatment effect modeling methods (9 papers) use both prognostic factors and treatment effect modifiers to explore characteristics that interact with the effects of therapy on a relative scale. These methods couple data-driven subgroup identification with approaches to prevent overfitting, such as penalization or use of separate data sets for subgroup identification and effect estimation. 3) Optimal treatment regime methods (12 papers) focus primarily on treatment effect modifiers to classify the trial population into those who benefit from treatment and those who do not. Finally, we also identified papers which describe model evaluation methods (4 papers).

CONCLUSIONS

Three classes of approaches were identified to assess heterogeneity of treatment effect. Methodological research, including both simulations and empirical evaluations, is required to compare the available methods in different settings and to derive well-informed guidance for their application in RCT analysis.

Prehospital Triage Strategies for the Transportation of Suspected Stroke Patients in the United States

Background and Purpose:

Ischemic stroke patients with large vessel occlusion (LVO) could benefit from direct transportation to an intervention center for endovascular treatment, but non-LVO patients need rapid IV thrombolysis in the nearest center. Our aim was to evaluate prehospital triage strategies for suspected stroke patients in the United States.

Methods:

We used a decision tree model and geographic information system to estimate outcome of suspected stroke patients transported by ambulance within 4.5 hours after symptom onset. We compared the following strategies: (1) Always to nearest center, (2) American Heart Association algorithm (ie, directly to intervention center if a prehospital stroke scale suggests LVO and total driving time from scene to intervention center is <30 minutes, provided that the delay would not exclude from thrombolysis), (3) modified algorithms with a maximum additional driving time to the intervention center of <30 minutes, <60 minutes, or without time limit, and (4) always to intervention center. Primary outcome was the annual number of good outcomes, defined as modified Rankin Scale score of 0–2. The preferred strategy was the one that resulted in the best outcomes with an incremental number needed to transport to intervention center (NNTI) <100 to prevent one death or severe disability (modified Rankin Scale score of >2).

Results:

Nationwide implementation of the American Heart Association algorithm increased the number of good outcomes by 594 (+1.0%) compared with transportation to the nearest center. The associated number of non-LVO patients transported to the intervention center was 16 714 (NNTI 28). The modified algorithms yielded an increase of 1013 (+1.8%) to 1369 (+2.4%) good outcomes, with a NNTI varying between 28 and 32. The algorithm without time limit was preferred in the majority of states (n=32 [65%]), followed by the algorithm with <60 minutes delay (n=10 [20%]). Tailoring policies at county-level slightly reduced the total number of transportations to the intervention center (NNTI 31).

Conclusions:

Prehospital triage strategies can greatly improve outcomes of the ischemic stroke population in the United States, but increase the number of non-LVO stroke patients transported to an intervention center. The current American Heart Association algorithm is suboptimal as a nationwide policy and should be modified to allow more delay when directly transporting LVO-suspected patients to an intervention center.

Risk of Paradoxical Embolism (RoPE)-Estimated Attributable Fraction Correlates With the Benefit of Patent Foramen Ovale Closure: An Analysis of 3 Trials

BACKGROUND AND PURPOSE

In patients with cryptogenic stroke and patent foramen ovale (PFO), the Risk of Paradoxical Embolism (RoPE) Score has been proposed as a method to estimate a patient-specific "PFO-attributable fraction"-the probability that a documented PFO is causally-related to the stroke, rather than an incidental finding. The objective of this research is to examine the relationship between this RoPE-estimated PFO-attributable fraction and the effect of closure in 3 randomized trials.

METHODS

We pooled data from the CLOSURE-I (Evaluation of the STARFlex Septal Closure System in Patients With a Stroke and/or Transient Ischemic Attack due to Presumed Paradoxical Embolism through a Patent Foramen Ovale), RESPECT (Randomized Evaluation of Recurrent Stroke Comparing PFO Closure to Established Current Standard of Care Treatment), and PC (Clinical Trial Comparing Percutaneous Closure of Patent Foramen Ovale [PFO] Using the Amplatzer PFO Occluder With Medical Treatment in Patients With Cryptogenic Embolism) trials. We examine the treatment effect of closure in high RoPE score (≥7) versus low RoPE score (<7) patients. We also estimated the relative risk reduction associated with PFO closure across each level of the RoPE score using Cox proportional hazard analysis. We estimated a patient-specific attributable fraction using a PC trial-compatible (9-point) RoPE equation (omitting the neuroradiology variable), as well as a 2-trial analysis using the original (10-point) RoPE equation. We examined the Pearson correlation between the estimated attributable fraction and the relative risk reduction across RoPE strata.

RESULTS

In the low RoPE score group (<7, n=912), the rate of recurrent strokes per 100 person-years was 1.37 in the device arm versus 1.68 in the medical arm (hazard ratio, 0.82 [0.42-1.59] P=0.56) compared with 0.30 versus 1.03 (hazard ratio, 0.31 [0.11-0.85] P=0.02) in the high RoPE score group (≥7, n=1221); treatment-by-RoPE score group interaction, P=0.12. The RoPE score estimated attributable fraction anticipated the relative risk reduction across all levels of the RoPE score, in both the 3-trial (r=0.95, P<0.001) and 2-trial (r=0.92, P<0.001) analyses.

CONCLUSIONS

The RoPE score estimated attributable fraction is highly correlated to the relative risk reduction of device versus medical therapy. This observation suggests the RoPE score identifies patients with cryptogenic stroke who are likely to have a PFO that is pathogenic rather than incidental.

Atrial fibrillation ablation in practice: assessing CABANA generalizability

AIMS

The Catheter Ablation vs. Antiarrhythmic Drug Therapy for Atrial Fibrillation (CABANA) trial aimed to assess the impact of ablation on morbidity and mortality. This observational study was conducted in parallel to CABANA to assess trial generalizability.

METHODS AND RESULTS

Using a large US administrative database, we identified 183 760 patients with atrial fibrillation (AF) treated with ablation or medical therapy (antiarrhythmic or rate control drugs) between 1 August 2009 and 30 April 2016 (CABANA enrolment period). Propensity score weighting was used to balance patients treated with ablation (N = 12 032) or medical therapy alone (N = 171 728) on 90 dimensions. Ablation was associated with a reduction in the composite endpoint of all-cause mortality, stroke, major bleeding, and cardiac arrest [hazard ratio (HR) 0.75, 95% confidence interval (CI) 0.70-0.81; P < 0.001]. The majority of patients (73.8%) were potentially trial eligible; among whom the risk reduction associated with ablation was greatest (HR 0.70, 95% CI 0.63-0.77; P < 0.001). Among the 3.8% of patients who failed to meet the inclusion criterion, i.e. patients under 65 years without stroke risk factors, the event rates were low and there was no significant relationship with ablation (HR 0.67, 95% CI 0.29-1.56; P = 0.35). Among the 22.4% patients who met at least one of the trial exclusion criteria, there was a lesser but statistically significant reduction associated with ablation (HR 0.85, 95% CI 0.75-0.95; P = 0.01).

CONCLUSION

In routine clinical care, ablation was associated with a reduction in the primary CABANA composite endpoint of all-cause mortality, stroke, major bleeding, and cardiac arrest, particularly in patients who were eligible for the trial.

Clinical Predictive Models of Sudden Cardiac Arrest: A Survey of the Current Science and Analysis of Model Performances

Background

More than 500 000 sudden cardiac arrests (SCAs) occur annually in the United States. Clinical predictive models (CPMs) may be helpful tools to differentiate between patients who are likely to survive or have good neurologic recovery and those who are not. However, which CPMs are most reliable for discriminating between outcomes in SCA is not known.

Methods and Results

We performed a systematic review of the literature using the Tufts PACE (Predictive Analytics and Comparative Effectiveness) CPM Registry through February 1, 2020, and identified 81 unique CPMs of SCA and 62 subsequent external validation studies. Initial cardiac rhythm, age, and duration of cardiopulmonary resuscitation were the 3 most commonly used predictive variables. Only 33 of the 81 novel SCA CPMs (41%) were validated at least once. Of 81 novel SCA CPMs, 56 (69%) and 61 of 62 validation studies (98%) reported discrimination, with median c‐statistics of 0.84 and 0.81, respectively. Calibration was reported in only 29 of 62 validation studies (41.9%). For those novel models that both reported discrimination and were validated (26 models), the median percentage change in discrimination was −1.6%. We identified 3 CPMs that had undergone at least 3 external validation studies: the out‐of‐hospital cardiac arrest score (9 validations; median c‐statistic, 0.79), the cardiac arrest hospital prognosis score (6 validations; median c‐statistic, 0.83), and the good outcome following attempted resuscitation score (6 validations; median c‐statistic, 0.76).

Conclusions

Although only a small number of SCA CPMs have been rigorously validated, the ones that have been demonstrate good discrimination.

Predictably unequal: understanding and addressing concerns that algorithmic clinical prediction may increase health disparities

The machine learning community has become alert to the ways that predictive algorithms can inadvertently introduce unfairness in decision-making. Herein, we discuss how concepts of algorithmic fairness might apply in healthcare, where predictive algorithms are being increasingly used to support decision-making. Central to our discussion is the distinction between algorithmic fairness and algorithmic bias. Fairness concerns apply specifically when algorithms are used to support polar decisions (i.e., where one pole of prediction leads to decisions that are generally more desired than the other), such as when predictions are used to allocate scarce health care resources to a group of patients that could all benefit. We review different fairness criteria and demonstrate their mutual incompatibility. Even when models are used to balance benefits-harms to make optimal decisions for individuals (i.e., for non-polar decisions)-and fairness concerns are not germane-model, data or sampling issues can lead to biased predictions that support decisions that are differentially harmful/beneficial across groups. We review these potential sources of bias, and also discuss ways to diagnose and remedy algorithmic bias. We note that remedies for algorithmic fairness may be more problematic, since we lack agreed upon definitions of fairness. Finally, we propose a provisional framework for the evaluation of clinical prediction models offered for further elaboration and refinement. Given the proliferation of prediction models used to guide clinical decisions, developing consensus for how these concerns can be addressed should be prioritized.

Can Clinical Predictive Models Identify Patients Who Should Not Receive TAVR? A Systematic Review

Background

One third of high- and prohibitive-risk TAVR patients remain severely symptomatic or die 1 year after treatment. There is interest in identifying individuals for whom this procedure is futile and should not be offered.

Methods

We performed a systematic review of the highest reported stratum of risk in TAVR clinical predictive models (CPMs). We explore whether currently available predictive models can identify patients for whom TAVR is futile, based on a quantitative futility definition and the observed and predicted outcomes for patients in the highest stratum of risk.

Results

17 TAVR CPMs representing 69,191 treated patients were published from 2013 to 2018. When reported, the median number of patients in the highest stratum of risk was 569 (range 1 to 1759). Observed mortality for this risk stratum ranged from 9% at 30 days to 59% at 1 year after TAVR. Statistical confidence in these observed event rates was low. The highest predicted event rates ranged from 11.0% for in-hospital mortality to 75.1% for the composite of mortality or high symptom burden 1 year after TAVR.

Conclusion

No high-risk TAVR group in currently available TAVR CPMs had an appropriate event rate and adequate statistical power to meet a quantitative definition of futility.

Why clinical trials may not help patients make treatment decisions: results from focus group discussions with 22 patients

Aim: Despite broad interest in advancing personalized medicine, most evidence is currently derived from average results of clinical trials that may obscure heterogeneity of trial participants. Little is known currently about how patients view heterogeneity in trials and whether they can participate in methodological discussions about this concept. Materials & methods: In structured discussions with three focus groups involving 22 participants, we assessed how representatives of patient communities have used research to guide individual treatment decisions. Discussion themes were organized into a framework describing patient decision-making in four steps: decisions patients make in the course of care; information used to make decisions; sources for information; and quality of information. Results/conclusion: Patients prioritize information that reflects their own characteristics, preferences and values. They struggle applying clinical research to their own case.

Prior Stroke in PFO Patients Is Associated With Both PFO-Related and -Unrelated Factors

Background and Purpose: To identify factors associated with prior stroke at presentation in patients with cryptogenic stroke (CS) and patent foramen ovale (PFO).

Methods: We studied cross-sectional data from the International PFO Consortium Study (NCT00859885). Patients with first-ever stroke and those with prior stroke at baseline were analyzed for an association with PFO-related (right-to-left shunt at rest, atrial septal aneurysm, deep venous thrombosis, pulmonary embolism, and Valsalva maneuver) and PFO-unrelated factors (age, gender, BMI, hypertension, diabetes mellitus, hypercholesterolemia, smoking, migraine, coronary artery disease, aortic plaque). A multivariable analysis was used to adjust effect estimation for confounding, e.g., owing to the age-dependent definition of study groups in this cross-sectional study design.

Results: We identified 635 patients with first-ever and 53 patients with prior stroke. Age, BMI, hypertension, diabetes mellitus, hypercholesterolemia, coronary artery disease, and right-to-left shunt (RLS) at rest were significantly associated with prior stroke. Using a pre-specified multivariable logistic regression model, age (Odds Ratio 1.06), BMI (OR 1.06), hypercholesterolemia (OR 1.90) and RLS at rest (OR 1.88) were strongly associated with prior stroke.Based on these factors, we developed a nomogram to illustrate the strength of the relation of individual factors to prior stroke.

Conclusion: In patients with CS and PFO, the likelihood of prior stroke is associated with both, PFO-related and PFO-unrelated factors.

When predictions are used to allocate scarce health care resources: three considerations for models in the era of Covid-19

BACKGROUND

The need for life-saving interventions such as mechanical ventilation may threaten to outstrip resources during the Covid-19 pandemic. Allocation of these resources to those most likely to benefit can be supported by clinical prediction models. The ethical and practical considerations relevant to predictions supporting decisions about microallocation are distinct from those that inform shared decision-making in ways important for model design.

MAIN BODY

We review three issues of importance for microallocation: (1) Prediction of benefit (or of medical futility) may be technically very challenging; (2) When resources are scarce, calibration is less important for microallocation than is ranking to prioritize patients, since capacity determines thresholds for resource utilization; (3) The concept of group fairness, which is not germane in shared decision-making, is of central importance in microallocation. Therefore, model transparency is important.

CONCLUSION

Prediction supporting allocation of life-saving interventions should be explicit, data-driven, frequently updated and open to public scrutiny. This implies a preference for simple, easily understood and easily applied prognostic models.

Practice advisory update summary: Patent foramen ovale and secondary stroke prevention: Report of the Guideline Subcommittee of the American Academy of Neurology

OBJECTIVE

To update the 2016 American Academy of Neurology (AAN) practice advisory for patients with stroke and patent foramen ovale (PFO).

METHODS

The guideline panel followed the AAN 2017 guideline development process to systematically review studies published through December 2017 and formulate recommendations.

MAJOR RECOMMENDATIONS

In patients being considered for PFO closure, clinicians should ensure that an appropriately thorough evaluation has been performed to rule out alternative mechanisms of stroke (level B). In patients with a higher risk alternative mechanism of stroke identified, clinicians should not routinely recommend PFO closure (level B). Clinicians should counsel patients that having a PFO is common; that it occurs in about 1 in 4 adults in the general population; that it is difficult to determine with certainty whether their PFO caused their stroke; and that PFO closure probably reduces recurrent stroke risk in select patients (level B). In patients younger than 60 years with a PFO and embolic-appearing infarct and no other mechanism of stroke identified, clinicians may recommend closure following a discussion of potential benefits (absolute recurrent stroke risk reduction of 3.4% at 5 years) and risks (periprocedural complication rate of 3.9% and increased absolute rate of non-periprocedural atrial fibrillation of 0.33% per year) (level C). In patients who opt to receive medical therapy alone without PFO closure, clinicians may recommend an antiplatelet medication such as aspirin or anticoagulation (level C).

Assessment of the impact of EHR heterogeneity for clinical research through a case study of silent brain infarction

Background

The rapid adoption of electronic health records (EHRs) holds great promise for advancing medicine through practice-based knowledge discovery. However, the validity of EHR-based clinical research is questionable due to poor research reproducibility caused by the heterogeneity and complexity of healthcare institutions and EHR systems, the cross-disciplinary nature of the research team, and the lack of standard processes and best practices for conducting EHR-based clinical research.

Method

We developed a data abstraction framework to standardize the process for multi-site EHR-based clinical studies aiming to enhance research reproducibility. The framework was implemented for a multi-site EHR-based research project, the ESPRESSO project, with the goal to identify individuals with silent brain infarctions (SBI) at Tufts Medical Center (TMC) and Mayo Clinic. The heterogeneity of healthcare institutions, EHR systems, documentation, and process variation in case identification was assessed quantitatively and qualitatively.

Result

We discovered a significant variation in the patient populations, neuroimaging reporting, EHR systems, and abstraction processes across the two sites. The prevalence of SBI for patients over age 50 for TMC and Mayo is 7.4 and 12.5% respectively. There is a variation regarding neuroimaging reporting where TMC are lengthy, standardized and descriptive while Mayo’s reports are short and definitive with more textual variations. Furthermore, differences in the EHR system, technology infrastructure, and data collection process were identified.

Conclusion

The implementation of the framework identified the institutional and process variations and the heterogeneity of EHRs across the sites participating in the case study. The experiment demonstrates the necessity to have a standardized process for data abstraction when conducting EHR-based clinical studies.

Abstract 135: Examining the Information Loss Between Neuroimages and Neuroimaging Reports for Detection of Silent Brain Infarcts and White Matter Disease Using Artificial Intelligence Technologies

Background: Real world studies of silent brain infarction (SBI) and white matter disease (WMD) are impeded by challenges in cohort identification. Natural language processing (NLP) from imaging reports may facilitate future studies. However, electronic health records can be heterogeneous and the process of interpreting neuroimages and generating reports can vary. Understanding knowledge representation and relationships between neuroimages and imaging reports is crucial for using NLP to facilitate disease management and cohort identification.

Methods: A balanced sample of head neuroimages (CT, MRI) of patients >50 years without clinical histories of symptomatic stroke, TIA, or dementia were obtained at Mayo Clinic and Tufts Medical Center. A team of 4 radiology residents performed report interpretation (RI) on 1000 reports according to a standardized protocol for the presence of SBIs, the presence of WMD, and WMD grade. A random subsample of 400 was doubly read for interrater reliability. For benchmarking, a team of 4 neuroradiologists directly reviewed and described findings on a subsample of 182 images, each doubly read. We assessed interrater reliability for direct review (DR) and RI, and agreement between these 2 information sources. An NLP algorithm was developed to review and extract findings from 1000 imaging reports.

Results: For DR, interrater reliability was moderate for SBIs and WMD (k = 0.53, 95% CI 0.43-0.64 and k = 0.47, 95% CI 0.33-0.61) and good for WMD grade (Spearman 0.71, p<0.001). For RI, interrater reliability for SBIs, WMD and WMD grade was good (k = 0.88, 95% CI 0.80-0.97; k = 0.98, 95% CI 0.97-1.00; and Spearman = 0.985, p<0.001, respectively). Agreement between DR and RI was good for SBIs (k = 0.77, 95% CI 0.67-0.86) and WMD (k = 0.65, 95% CI 0.54-0.77). Spearman rank correlations comparing WMD grade interpretation DR to RI was 0.60 (p<0.001). In identifying the presence of SBIs and WMDs, the accuracy of the NLP algorithm was 0.991 and 0.994, respectively.

Conclusion: For the presence of SBI and WMD, and WMD grade, agreement between RI and DR was similar to agreement between 2 neuroradiologists directly reviewing neuroimages. It is highly feasible to use NLP to identify patients with SBIs and WMDs for clinical studies.

The Predictive Approaches to Treatment effect Heterogeneity (PATH) Statement

Heterogeneity of treatment effect (HTE) refers to the nonrandom variation in the magnitude or direction of a treatment effect across levels of a covariate, as measured on a selected scale, against a clinical outcome. In randomized controlled trials (RCTs), HTE is typically examined through a subgroup analysis that contrasts effects in groups of patients defined "1 variable at a time" (for example, male vs. female or old vs. young). The authors of this statement present guidance on an alternative approach to HTE analysis, "predictive HTE analysis." The goal of predictive HTE analysis is to provide patient-centered estimates of outcome risks with versus without the intervention, taking into account all relevant patient attributes simultaneously. The PATH (Predictive Approaches to Treatment effect Heterogeneity) Statement was developed using a multidisciplinary technical expert panel, targeted literature reviews, simulations to characterize potential problems with predictive approaches, and a deliberative process engaging the expert panel. The authors distinguish 2 categories of predictive HTE approaches: a "risk-modeling" approach, wherein a multivariable model predicts the risk for an outcome and is applied to disaggregate patients within RCTs to define risk-based variation in benefit, and an "effect-modeling" approach, wherein a model is developed on RCT data by incorporating a term for treatment assignment and interactions between treatment and baseline covariates. Both approaches can be used to predict differential absolute treatment effects, the most relevant scale for clinical decision making. The authors developed 4 sets of guidance: criteria to determine when risk-modeling approaches are likely to identify clinically important HTE, methodological aspects of risk-modeling methods, considerations for translation to clinical practice, and considerations and caveats in the use of effect-modeling approaches. The PATH Statement, together with its explanation and elaboration document, may guide future analyses and reporting of RCTs.

The Predictive Approaches to Treatment effect Heterogeneity (PATH) Statement: Explanation and Elaboration

The PATH (Predictive Approaches to Treatment effect Heterogeneity) Statement was developed to promote the conduct of, and provide guidance for, predictive analyses of heterogeneity of treatment effects (HTE) in clinical trials. The goal of predictive HTE analysis is to provide patient-centered estimates of outcome risk with versus without the intervention, taking into account all relevant patient attributes simultaneously, to support more personalized clinical decision making than can be made on the basis of only an overall average treatment effect. The authors distinguished 2 categories of predictive HTE approaches (a "risk-modeling" and an "effect-modeling" approach) and developed 4 sets of guidance statements: criteria to determine when risk-modeling approaches are likely to identify clinically meaningful HTE, methodological aspects of risk-modeling methods, considerations for translation to clinical practice, and considerations and caveats in the use of effect-modeling approaches. They discuss limitations of these methods and enumerate research priorities for advancing methods designed to generate more personalized evidence. This explanation and elaboration document describes the intent and rationale of each recommendation and discusses related analytic considerations, caveats, and reservations.

Developing a cancer-specific trigger tool to identify treatment-related adverse events using administrative data

Background

As there are few validated tools to identify treatment‐related adverse events across cancer care settings, we sought to develop oncology‐specific “triggers” to flag potential adverse events among cancer patients using claims data.

Methods

322 887 adult patients undergoing an initial course of cancer‐directed therapy for breast, colorectal, lung, or prostate cancer from 2008 to 2014 were drawn from a large commercial claims database. We defined 16 oncology‐specific triggers using diagnosis and procedure codes. To distinguish treatment‐related complications from comorbidities, we required a logical and temporal relationship between a treatment and the associated trigger. We tabulated the prevalence of triggers by cancer type and metastatic status during 1‐year of follow‐up, and examined cancer trigger risk factors.

Results

Cancer‐specific trigger events affected 19% of patients over the initial treatment year. The trigger burden varied by disease and metastatic status, from 6% of patients with nonmetastatic prostate cancer to 41% and 50% of those with metastatic colorectal and lung cancers, respectively. The most prevalent triggers were abnormal serum bicarbonate, blood transfusion, non‐contrast chest CT scan following radiation therapy, and hypoxemia. Among patients with metastatic disease, 10% had one trigger event and 29% had two or more. Triggers were more common among older patients, women, non‐whites, patients with low family incomes, and those without a college education.

Conclusions

Oncology‐specific triggers offer a promising method for identifying potential patient safety events among patients across cancer care settings.

Absolute Lymphocyte Count: A Predictor of Recurrent Cytomegalovirus Disease in Solid Organ Transplant Recipients

Background

Recurrent cytomegalovirus (CMV) disease in solid organ transplant recipients frequently occurs despite effective antiviral therapy. We previously demonstrated that patients with lymphopenia before liver transplantation are more likely to develop posttransplant infectious complications including CMV. The aim of this study was to explore absolute lymphocyte count (ALC) as a predictor of relapse following treatment for CMV disease.

Methods

We performed a retrospective cohort study of heart, liver, and kidney transplant recipients treated for an episode of CMV disease. Our primary outcome was time to relapse of CMV within 6 months. Data on potential predictors of relapse including ALC were collected at the time of CMV treatment completion. Univariate and multivariate hazard ratios (HRs) were calculated with a Cox model. Multiple imputation was used to complete the data.

Results

Relapse occurred in 33 of 170 participants (19.4%). Mean ALC in relapse-free patients was 1.08 ± 0.69 vs 0.73 ± 0.42 × 103 cells/μL in those who relapsed, corresponding to an unadjusted hazard ratio of 1.11 (95% confidence interval, 1.03-1.21; P = .009, n = 133) for every decrease of 100 cells/μL. After adjusting for potential confounders, the association between ALC and relapse remained significant (HR, 1.11 [1.03-1.20]; P = .009).

Conclusions

Low ALC at the time of CMV treatment completion was a strong independent predictor for recurrent CMV disease. This finding is biologically plausible given the known importance of T-cell immunity in maintaining CMV latency. Future studies should consider this inexpensive, readily available marker of host immunity.

Clinical Prediction Models for Valvular Heart Disease

Background

While many clinical prediction models (CPMs) exist to guide valvular heart disease treatment decisions, the relative performance of these CPMs is largely unknown. We systematically describe the CPMs available for patients with valvular heart disease with specific attention to performance in external validations.

Methods and Results

A systematic review identified 49 CPMs for patients with valvular heart disease treated with surgery (n=34), percutaneous interventions (n=12), or no intervention (n=3). There were 204 external validations of these CPMs. Only 35 (71%) CPMs have been externally validated. Sixty‐five percent (n=133) of the external validations were performed on distantly related populations. There was substantial heterogeneity in model performance and a median percentage change in discrimination of −27.1% (interquartile range, −49.4%–−5.7%). Nearly two‐thirds of validations (n=129) demonstrate at least a 10% relative decline in discrimination. Discriminatory performance of EuroSCORE II and Society of Thoracic Surgeons (2009) models (accounting for 73% of external validations) varied widely: EuroSCORE II validation c‐statistic range 0.50 to 0.95; Society of Thoracic Surgeons (2009) Models validation c‐statistic range 0.50 to 0.86. These models performed well when tested on related populations (median related validation c‐statistics: EuroSCORE II, 0.82 [0.76, 0.85]; Society of Thoracic Surgeons [2009], 0.72 [0.67, 0.79]). There remain few (n=9) external validations of transcatheter aortic valve replacement CPMs.

Conclusions

Many CPMs for patients with valvular heart disease have never been externally validated and isolated external validations appear insufficient to assess the trustworthiness of predictions. For surgical valve interventions, there are existing predictive models that perform reasonably well on related populations. For transcatheter aortic valve replacement (CPMs additional external validations are needed to broadly understand the trustworthiness of predictions.