The Predictive Approaches to Treatment effect Heterogeneity (PATH) Statement: Explanation and Elaboration

Estimating the Effect of Radical Prostatectomy: Combining Data From the SPCG4 and PIVOT Randomized Trials With Contemporary Cohorts

PURPOSE

Two randomized trials (SPCG4 and PIVOT) have compared surgery to conservative management for localized prostate cancer. The applicability of these trials to contemporary practice remains uncertain. We aimed to develop an individualized prediction model for prostate cancer mortality comparing immediate surgery at a high-volume center to active surveillance.

MATERIALS AND METHODS

We determined whether the relative risk of prostate cancer mortality with surgery vs observation varied by baseline risk. We then used various estimates of relative risk to estimate 15-year mortality with and without surgery using, as a predictor, risk of biochemical recurrence calculated from a model.

RESULTS

We saw no evidence that relative risk varied by baseline risk, supporting the use of a constant relative risk. Compared with observation, surgery was associated with negligible benefit for patients with Grade Group (GG) 1 disease (0.2% mortality reduction at 15 years) and small benefit for patients with GG2 with lower PSA and stage (≤5% mortality reduction). Benefit was greater (6%-9%) for patients with GG3 or GG4 though still modest, but effect estimates varied widely depending on choice of hazard ratio for surgery (6%-36% absolute risk reduction).

CONCLUSIONS

Surgery should be avoided for men with low-risk (GG1) prostate cancer and for many men with GG2 disease. Surgical benefits are greater in men with higher-risk disease. Integration of findings with a life expectancy model will allow patients to make informed treatment decisions given their oncologic risk, risk of death from other causes, and estimated effects of surgery.

Transforming Cardiovascular Care With Artificial Intelligence: From Discovery to Practice: JACC State-of-the-Art Review

Artificial intelligence (AI) has the potential to transform every facet of cardiovascular practice and research. The exponential rise in technology powered by AI is defining new frontiers in cardiovascular care, with innovations that span novel diagnostic modalities, new digital native biomarkers of disease, and high-performing tools evaluating care quality and prognosticating clinical outcomes. These digital innovations promise expanded access to cardiovascular screening and monitoring, especially among those without access to high-quality, specialized care historically. Moreover, AI is propelling biological and clinical discoveries that will make future cardiovascular care more personalized, precise, and effective. The review brings together these diverse AI innovations, highlighting developments in multimodal cardiovascular AI across clinical practice and biomedical discovery, and envisioning this new future backed by contemporary science and emerging discoveries. Finally, we define the critical path and the safeguards essential to realizing this AI-enabled future that helps achieve optimal cardiovascular health and outcomes for all.

Gut microbiota, inflammatory factors, and scoliosis: A Mendelian randomization study

Several studies have reported a potential association between the gut microbiota (GM) and scoliosis. However, the causal relationship between GM and scoliosis and the role of inflammatory factors (IFs) as mediators remain unclear. This study aimed to analyze the relationship between GM, IFs, and scoliosis. We investigated whether IFs act as mediators in pathways from the GM to scoliosis. Additionally, using reverse Mendelian randomization (MR) analysis, we further investigated the potential impact of genetic predisposition to scoliosis on the GM and IFs. In this study, we searched for publicly available genome-wide association study aggregate data and utilized the MR method to establish bidirectional causal relationships among 211 GM taxa, 91 IFs, and scoliosis. To ensure the reliability of our research findings, we employed 5 MR methods, with the inverse variance weighting approach serving as the primary statistical method, and assessed the robustness of the results through various sensitivity analyses. Additionally, we investigated whether IFs mediate pathways from GM to scoliosis. Three negative causal correlations were observed between the genetic predisposition to GM and scoliosis. Additionally, both positive and negative correlations were found between IFs and scoliosis, with 3 positive and 3 negative correlations observed. IFs do not appear to act as mediators in the pathway from GM to scoliosis. In conclusion, this study demonstrated a causal association between the GM, IFs, and scoliosis, indicating that IFs are not mediators in the pathway from the GM to scoliosis. These findings offer new insights into prevention and treatment strategies for scoliosis.

Human-centered Design of a Health Recommender System for Orthopaedic Shoulder Treatment

Background

Rich data on diverse patients and their treatments and outcomes within Electronic Health Record (EHR) systems can be used to generate real world evidence. A health recommender system (HRS) framework can be applied to a decision support system application to generate data summaries for similar patients during the clinical encounter to assist physicians and patients in making evidence-based shared treatment decisions.

Objective

A human-centered design (HCD) process was used to develop a HRS for treatment decision support in orthopaedic medicine, the Informatics Consult for Individualized Treatment (I-C-IT). We also evaluate the usability and utility of the system from the physician's perspective, focusing on elements of utility and shared decision-making in orthopaedic medicine.

Methods

The HCD process for I-C-IT included 6 steps across three phases of analysis, design, and evaluation. A team of informaticians and comparative effectiveness researchers directly engaged with orthopaedic surgeon subject matter experts in a collaborative I-C-IT prototype design process. Ten orthopaedic surgeons participated in a mixed methods evaluation of the I-C-IT prototype that was produced.

Results

The HCD process resulted in a prototype system, I-C-IT, with 14 data visualization elements and a set of design principles crucial for HRS for decision support. The overall standard system usability scale (SUS) score for the I-C-IT Webapp prototype was 88.75 indicating high usability. In addition, utility questions addressing shared decision-making found that 90% of orthopaedic surgeon respondents either strongly agreed or agreed that I-C-IT would help them make data informed decisions with their patients.

Conclusion

The HCD process produced an HRS prototype that is capable of supporting orthopaedic surgeons and patients in their information needs during clinical encounters. Future research should focus on refining I-C-IT by incorporating patient feedback in future iterative cycles of system design and evaluation.

Impact of the PATH Statement on Analysis and Reporting of Heterogeneity of Treatment Effect in Clinical Trials: A Scoping Review

Background

The Predictive Approaches to Treatment Effect Heterogeneity (PATH) Statement provides guidance for using predictive modeling to identify differences (i.e., heterogeneity) in treatment effects (benefits and harms) among participants in randomized clinical trials (RCTs). It distinguished risk modeling, which uses a multivariable model to predict risk of trial outcome(s) and then examines treatment effects within strata of predicted risk, from effect modeling, which predicts trial outcomes using models that include treatment, individual participant characteristics and interactions of treatment with selected characteristics.

Purpose

To describe studies of heterogeneous treatment effects (HTE) that use predictive modeling in RCT data and cite the PATH Statement.

Data Sources

The Cited By functions in PubMed, Google Scholar, Web of Science and SCOPUS databases (Jan 7, 2020 - June 5, 2023).

Study Selection

42 reports presenting 45 predictive models.

Data Extraction

Double review with adjudication to identify risk and effect modeling and examine consistency with Statement consensus statements. Credibility of HTE findings was assessed using criteria adapted from the Instrument to assess Credibility of Effect Modification Analyses (ICEMAN). Clinical importance of credible HTE findings was also assessed.

Data Synthesis

The numbers of reports, especially risk modeling reports, increased year-on-year. Consistency with consensus statements was high, except for two: only 15 of 32 studies with positive overall findings included a risk model; and most effect models explored many candidate covariates with little prior evidence for effect modification. Risk modeling was more likely than effect modeling to identify both credible HTE (14/19 vs 5/26) and clinically important HTE (10/19 vs 4/26).

Limitations

Risk of reviewer bias: reviewers assessing credibility and clinical importance were not blinded to adherence to PATH recommendations.

Conclusions

The PATH Statement appears to be influencing research practice. Risk modeling often uncovered clinically important HTE; effect modeling was more often exploratory.

Variation in Health Status With Invasive vs Conservative Management of Chronic Coronary Disease

BACKGROUND

The ISCHEMIA trial found that patients with chronic coronary disease randomized to invasive strategy had better health status than those randomized to conservative strategy. It is unclear how best to translate these population-level results to individual patients.

OBJECTIVES

The authors sought to identify patient characteristics associated with health status from invasive and conservative strategies, and develop a prediction algorithm for shared decision-making.

METHODS

One-year disease-specific health status was assessed in ISCHEMIA with the Seattle Angina Questionnaire (SAQ) Summary Score (SAQ SS) and Angina Frequency, Physical Limitations (PL), and Quality of Life (QL) domains (range 0-100, higher = less angina/better health status).

RESULTS

Among 4,617 patients from 320 sites in 37 countries, mean SAQ SS was 74.1 ± 18.9 at baseline and 85.7 ± 15.6 at 1 year. Lower baseline SAQ SS and younger age were associated with better 1-year health status with invasive strategy (P interaction = 0.009 and P interaction = 0.004, respectively). For the individual domains, there were significant treatment interactions for baseline SAQ score (Angina Frequency, PL), age (PL, QL), anterior ischemia (PL), and number of baseline antianginal medications (QL), with more benefit of invasive in patients with worse baseline health status, younger age, anterior ischemia, and on more antianginal medications. Parsimonious prediction models were developed for 1-year SAQ domains with invasive or conservative strategies to support shared decision-making.

CONCLUSIONS

In the management of chronic coronary disease, individual patient characteristics are associated with 1-year health status, with younger age and poorer angina-related health status showing greater benefit from invasive management. This prediction algorithm can support the translation of the ISCHEMIA trial results to individual patients. (International Study of Comparative Health Effectiveness With Medical and Invasive Approaches [ISCHEMIA]; NCT01471522).

Making predictions under interventions: a case study from the PREDICT-CVD cohort in New Zealand primary care

Background

Most existing clinical prediction models do not allow predictions under interventions. Such predictions allow predicted risk under different proposed strategies to be compared and are therefore useful to support clinical decision making. We aimed to compare methodological approaches for predicting individual level cardiovascular risk under three interventions: smoking cessation, reducing blood pressure, and reducing cholesterol.

Methods

We used data from the PREDICT prospective cohort study in New Zealand to calculate cardiovascular risk in a primary care setting. We compared three strategies to estimate absolute risk under intervention: (a) conditioning on hypothetical interventions in non-causal models; (b) combining existing prediction models with causal effects estimated using observational causal inference methods; and (c) combining existing prediction models with causal effects reported in published literature.

Results

The median absolute cardiovascular risk among smokers was 3.9%; our approaches predicted that smoking cessation reduced this to a median between a non-causal estimate of 2.5% and a causal estimate of 2.8%, depending on estimation methods. For reducing blood pressure, the proposed approaches estimated a reduction of absolute risk from a median of 4.9% to a median between 3.2% and 4.5% (both derived from causal estimation). Reducing cholesterol was estimated to reduce median absolute risk from 3.1% to between 2.2% (non-causal estimate) and 2.8% (causal estimate).

Conclusions

Estimated absolute risk reductions based on non-causal methods were different to those based on causal methods, and there was substantial variation in estimates within the causal methods. Researchers wishing to estimate risk under intervention should be explicit about their causal modelling assumptions and conduct sensitivity analysis by considering a range of possible approaches.

Improving methods of clinical practice guidelines: From guidelines to pathways to fast-and-frugal trees and decision analysis to develop individualised patient care

BACKGROUND

Current methods for developing clinical practice guidelines have several limitations: they are characterised by the "black box" operation-a process with defined inputs and outputs but an incomplete understanding of its internal workings; they have "the integration problem"-a lack of framework for explicitly integrating factors such as patient preferences and trade-offs between benefits and harms; they generate one recommendation at a time that typically are not connected in a coherent analytical framework; and they apply to "average" patients, while clinicians and their patients seek advice tailored to individual circumstances.

METHODS

We propose augmenting the current guideline development method by converting evidence-based pathways into fast-and-frugal decision trees (FFTs) and integrating them with generalised decision curve analysis to formulate clear, individualised management recommendations.

RESULTS

We illustrate the process by developing recommendations for the management of heparin-induced thrombocytopenia (HIT). We converted evidence-based pathways for HIT, developed by the American Society of Hematology, into an FFT. Here, we consider only thrombotic complications and major bleeding. We leveraged the predictive potential of FFTs to compare the effects of argatroban, bivalirudin, fondaparinux, and direct oral anticoagulants (DOACs) using generalised decision curve analysis. We found that DOACs were superior to other treatments if the FFT-predicted probability of HIT exceeded 3%.

CONCLUSIONS

The proposed analytical framework connects guidelines, pathways, FFTs, and decision analysis, offering risk-tailored personalised recommendations and addressing current guideline development critiques.

Assessing the properties of patient-specific treatment effect estimates from causal forest algorithms under essential heterogeneity

BACKGROUND

Treatment variation from observational data has been used to estimate patient-specific treatment effects. Causal Forest Algorithms (CFAs) developed for this task have unknown properties when treatment effect heterogeneity from unmeasured patient factors influences treatment choice - essential heterogeneity.

METHODS

We simulated eleven populations with identical treatment effect distributions based on patient factors. The populations varied in the extent that treatment effect heterogeneity influenced treatment choice. We used the generalized random forest application (CFA-GRF) to estimate patient-specific treatment effects for each population. Average differences between true and estimated effects for patient subsets were evaluated.

RESULTS

CFA-GRF performed well across the population when treatment effect heterogeneity did not influence treatment choice. Under essential heterogeneity, however, CFA-GRF yielded treatment effect estimates that reflected true treatment effects only for treated patients and were on average greater than true treatment effects for untreated patients.

CONCLUSIONS

Patient-specific estimates produced by CFAs are sensitive to why patients in real-world practice make different treatment choices. Researchers using CFAs should develop conceptual frameworks of treatment choice prior to estimation to guide estimate interpretation ex post.

Decision curve analysis based on summary data

BACKGROUND

To realize the potential of precision medicine, predictive models should be integrated within the framework of decision analysis, such as the decision curve analysis (DCA). To date, its application has required individual patient data (IPD) that are often unavailable. Performing DCA using aggregate data without requiring IPD may advance the goals of precision medicine.

METHODS

We present a statistical framework demonstrating that DCA can be conducted by using only the mean and standard deviation (SD) from the raw probabilities of the predictive model. We tested our theoretical framework by performing extensive simulations and comparing the aggregate-based DCA with IPD DCA. The latter was conducted using IPD from four predictive models that employed logistic regression, Cox or competing risk time-to-event modeling including (a) statins for primary prevention of cardiovascular disease (n = 4859), (b) hospice referral for terminally ill patients (n = 9104), (c) use of thromboprophylaxis for preventing venous thromboembolism in patients with cancer (n = 425) and (d) prevention of sinusoidal obstruction syndrome after hematopoietic cell transplantation (SCT) (n = 80).

RESULTS

Simulations assuming perfect calibration showed that regardless of which probability distributions informed the predictive models, the differences in DCA were negligible. Similarly, for the adequately powered models, the results of DCA based on the summary data were similar to IPD-derived DCA. The inherent instability of the predictive models, based on the smaller sample sizes, resulted in a somewhat larger discrepancy between aggregate and IPD-based DCA.

CONCLUSIONS

DCA informed by adequately powered and well-calibrated models using only summary statistical estimates (mean and SD) approximates well models using IPD. Use of aggregate data will facilitate broader integration of predictive with decision modeling toward the goals of individualized decision-making.

A Risk Profile Using Simple Hematologic Parameters to Assess Benefits From Baricitinib in Patients Hospitalized With COVID-19: A Post Hoc Analysis of the Adaptive COVID-19 Treatment Trial-2

BACKGROUND

The ACTT risk profile, which was developed from ACTT-1 (Adaptive COVID-19 Treatment Trial-1), demonstrated that hospitalized patients with COVID-19 in the high-risk quartile (characterized by low absolute lymphocyte count [ALC], high absolute neutrophil count [ANC], and low platelet count at baseline) benefited most from treatment with the antiviral remdesivir. It is unknown which patient characteristics are associated with benefit from treatment with the immunomodulator baricitinib.

OBJECTIVE

To apply the ACTT risk profile to the ACTT-2 cohort to investigate potential baricitinib-related treatment effects by risk quartile.

DESIGN

Post hoc analysis of ACTT-2, a randomized, double-blind, placebo-controlled trial. (ClinicalTrials.gov: NCT04401579).

SETTING

Sixty-seven trial sites in 8 countries.

PARTICIPANTS

Adults hospitalized with COVID-19 (n = 999; 85% U.S. participants).

INTERVENTION

Baricitinib+remdesivir versus placebo+remdesivir.

MEASUREMENTS

Mortality, progression to invasive mechanical ventilation (IMV) or death, and recovery, all within 28 days; ALC, ANC, and platelet count trajectories.

RESULTS

In the high-risk quartile, baricitinib+remdesivir was associated with reduced risk for death (hazard ratio [HR], 0.38 [95% CI, 0.16 to 0.86]; P = 0.020), decreased progression to IMV or death (HR, 0.57 [CI, 0.35 to 0.93]; P = 0.024), and improved recovery rate (HR, 1.53 [CI, 1.16 to 2.02]; P = 0.002) compared with placebo+remdesivir. After 5 days, participants receiving baricitinib+remdesivir had significantly larger increases in ALC and significantly larger decreases in ANC compared with control participants, with the largest effects observed in the high-risk quartile.

LIMITATION

Secondary analysis of data collected before circulation of current SARS-CoV-2 variants.

CONCLUSION

The ACTT risk profile identifies a subgroup of hospitalized patients who benefit most from baricitinib treatment and captures a patient phenotype of treatment response to an immunomodulator and an antiviral. Changes in ALC and ANC trajectory suggest a mechanism whereby an immunomodulator limits severe COVID-19.

PRIMARY FUNDING SOURCE

National Institute of Allergy and Infectious Diseases.

Heterogeneous Treatment Effects of Intensive Glycemic Control on Kidney Microvascular Outcomes and Mortality in ACCORD

SIGNIFICANCE STATEMENT

Identifying and quantifying treatment effect variation across patients is the fundamental challenge of precision medicine. Here we quantify heterogeneous treatment effects of intensive glycemic control in the Action to Control Cardiovascular Risk in Diabetes (ACCORD) trial, considering three outcomes of interest-a composite kidney outcome (driven by macroalbuminuria), all-cause mortality, and first assisted hypoglycemic event. We demonstrate that the effects of intensive glycemic control vary with risk of kidney failure, as predicted by the kidney failure risk equation (KFRE). Participants at highest risk of kidney failure gain the largest absolute kidney benefit of intensive glycemic control but also experience the largest absolute risk of death and hypoglycemic events. Our findings illustrate the value of identifying clinically meaningful treatment heterogeneity, particularly when treatments have different effects on multiple end points.

OBJECTIVE

Clear criteria to individualize glycemic targets in patients with type II diabetes are lacking. In this post hoc analysis of the ACCORD, we evaluate whether the KFRE can identify patients for whom intensive glycemic control confers more benefit in preventing kidney microvascular outcomes.

RESEARCH DESIGN AND METHODS

We divided the ACCORD trial population into quartiles on the basis of 5-year kidney failure risk using the KFRE. We estimated conditional treatment effects within each quartile and compared them with the average treatment effect in the trial. The treatment effects of interest were the 7-year restricted mean survival time (RMST) differences between intensive and standard glycemic control arms on ( 1 ) time-to-first development of severely elevated albuminuria or kidney failure and ( 2 ) all-cause mortality.

RESULTS

We found evidence that the effect of intensive glycemic control on kidney microvascular outcomes and all-cause mortality varies with baseline risk of kidney failure. Patients with elevated baseline risk of kidney failure derived the most from intensive glycemic control in reducing kidney microvascular outcomes (7-year RMST difference of 114.8 [95% confidence interval 58.1 to 176.4] versus 48.4 [25.3 to 69.6] days in the entire trial population) However, this same patient group also experienced a shorter time to death (7-year RMST difference of -56.7 [-100.2 to -17.5] v. -23.6 [-42.2 to -6.6] days).

CONCLUSIONS

We found evidence of heterogenous treatment effects of intensive glycemic control on kidney microvascular outcomes in ACCORD as a function of predicted baseline risk of kidney failure. Patients with higher kidney failure risk experienced the most pronounced reduction in kidney microvascular outcomes but also experienced the highest risk of all-cause mortality.

Development and validation of a prognostic score to identify the optimal candidate for preemptive TIPS in patients with cirrhosis and acute variceal bleeding

BACKGROUND AND AIM

Baveno VII workshop recommends the use of preemptive TIPS (p-TIPS) in patients with cirrhosis and acute variceal bleeding (AVB) at high- risk of treatment failure. However, the criteria defining "high-risk" have low clinical accessibility or include subjective variables. We aimed to develop and externally validate a model for better identification of p-TIPS candidates.

APPROACH AND RESULTS

The derivation cohort included 1554 patients with cirrhosis and AVB who were treated with endoscopy plus drug (n = 1264) or p-TIPS (n = 290) from 12 hospitals in China between 2010 and 2017. We first used competing risk regression to develop a score for predicting 6-week and 1-year mortality in patients treated with endoscopy plus drugs, which included age, albumin, bilirubin, international normalized ratio, white blood cell, creatinine, and sodium. The score was internally validated with the bootstrap method, which showed good discrimination (6 wk/1 y concordance-index: 0.766/0.740) and calibration, and outperformed other currently available models. In the second stage, the developed score was combined with treatment and their interaction term to predicate the treatment effect of p-TIPS (mortality risk difference between treatment groups) in the whole derivation cohort. The estimated treatment effect of p-TIPS varied substantially among patients. The prediction model had good discriminative ability (6 wk/1 y c -for-benefit: 0.696/0.665) and was well calibrated. These results were confirmed in the validation dataset of 445 patients with cirrhosis with AVB from 6 hospitals in China between 2017 and 2019 (6-wk/1-y c-for-benefit: 0.675/0.672).

CONCLUSIONS

We developed and validated a clinical prediction model that can help to identify individuals who will benefit from p-TIPS, which may guide clinical decision-making.

Generalised decision curve analysis for explicit comparison of treatment effects

RATIONALE

Decision curve analysis (DCA) helps integrate prediction models with treatment assessments to guide personalised therapeutic choices among multiple treatment options. However, the current versions of DCA do not explicitly model treatment effects in the analysis but implicitly or holistically assess therapeutic benefits and harms. In addition, the existing DCA cannot allow the comparison of multiple treatments using a standard metric.

AIMS AND OBJECTIVES

To develop a generalised version of DCA (gDCA) by decomposing holistically assessed net benefits and harms into patient preferences versus empirical evidence (as obtained in the trials, meta-analyses of clinical studies, etc.) to allow individualised comparison of single or multiple treatments using a common metric.

METHODS

We reformulated DCA by (1) decomposing holistic, implicit utilities into specific utilities related to treatment effects and patient's relative values (RV) about disease outcomes versus treatment harms, (2) explicitly modelling each treatment effect at the level of probabilities and/or utilities (outcomes) in a decision tree, and (3) avoiding scaling effects employed in the original DCA to enable comparison of treatment effects against the common metrics. We used data from a published network meta-analysis of randomised trials to inform the use of statin treatment according to Framingham Risk Model.

RESULTS

We illustrate the analysis by modelling the effects of three statins in the primary prevention of cardiovascular disease. We performed simultaneous comparisons against standard metrics (RV) for all treatments. We examined for which RV values, a predictive model for guiding personalised treatment, outperformed the strategies of treating everyone or treating no one. We found that the magnitude of benefits (efficacy) seems more important than the simple ratio of efficacy/harms.

CONCLUSION

We describe gDCA for evaluating single or multiple treatments to help tailor therapy toward individual risk characteristics. gDCA further helps integrate the principles of evidence-based medicine with decision analysis.

Antiplatelet Agent Use After Stroke due to Intracerebral Hemorrhage

This focused update about antiplatelet agents to reduce the high risk of major adverse cardiovascular events after stroke due to spontaneous (nontraumatic) intracerebral hemorrhage (ICH) complements earlier updates about blood pressure-lowering, lipid-lowering, and oral anticoagulation or left atrial appendage occlusion for atrial fibrillation after ICH. When used for secondary prevention in people without ICH, antiplatelet agents reduce the risk of major adverse cardiovascular event (rate ratio, 0.81 [95% CI, 0.75-0.87]) and might increase the risk of ICH (rate ratio, 1.67 [95% CI, 0.97-2.90]). Before 2019, guidance for clinical decisions about antiplatelet agent use after ICH has focused on estimating patients' predicted absolute risks and severities of ischemic and hemorrhagic major adverse cardiovascular event and applying the known effects of these drugs in people without ICH to estimate whether individual ICH survivors in clinical practice might be helped or harmed by antiplatelet agents. In 2019, the main results of the RESTART (Restart or Stop Antithrombotics Randomized Trial) randomized controlled trial including 537 survivors of ICH associated with antithrombotic drug use showed, counterintuitively, that antiplatelet agents might not increase the risk of recurrent ICH compared to antiplatelet agent avoidance over 2 years of follow-up (12/268 [4%] versus 23/268 [9%]; adjusted hazard ratio, 0.51 [95% CI, 0.25-1.03]; P=0.060). Guidelines in the United States, Canada, China, and the United Kingdom and Ireland have classified the level of evidence as B and indicated that antiplatelet agents may be considered/reasonable after ICH associated with antithrombotic agent use. Three subsequent clinical trials have recruited another 174 participants with ICH, but they will not be sufficient to determine the effects of antiplatelet therapy on all major adverse cardiovascular events reliably when pooled with RESTART. Therefore, ASPIRING (Antiplatelet Secondary Prevention International Randomized Study After Intracerebral Hemorrhage) aims to recruit 4148 ICH survivors to determine the effects of antiplatelet agents after ICH definitively overall and in subgroups.

Can Machine Learning Aid the Selection of Percutaneous vs Surgical Revascularization?

BACKGROUND

In patients with 3-vessel coronary artery disease (CAD) and/or left main CAD, individual risk prediction plays a key role in deciding between percutaneous coronary intervention (PCI) and coronary artery bypass grafting (CABG).

OBJECTIVES

The aim of this study was to assess whether these individualized revascularization decisions can be improved by applying machine learning (ML) algorithms and integrating clinical, biological, and anatomical factors.

METHODS

In the SYNTAX (Synergy between PCI with Taxus and Cardiac Surgery) study, ML algorithms (Lasso regression, gradient boosting) were used to develop a prognostic index for 5-year death, which was combined, in the second stage, with assigned treatment (PCI or CABG) and prespecified effect-modifiers: disease type (3-vessel or left main CAD) and anatomical SYNTAX score. The model's discriminative ability to predict the risk of 5-year death and treatment benefit between PCI and CABG was cross-validated in the SYNTAX trial (n = 1,800) and externally validated in the CREDO-Kyoto (Coronary REvascularization Demonstrating Outcome Study in Kyoto) registry (n = 7,362), and then compared with the original SYNTAX score II 2020 (SSII-2020).

RESULTS

The hybrid gradient boosting model performed best for predicting 5-year all-cause death with C-indexes of 0.78 (95% CI: 0.75-0.81) in cross-validation and 0.77 (95% CI: 0.76-0.79) in external validation. The ML models discriminated 5-year mortality better than the SSII-2020 in the external validation cohort and identified heterogeneity in the treatment benefit of CABG vs PCI.

CONCLUSIONS

An ML-based approach for identifying individuals who benefit from CABG or PCI is feasible and effective. Implementation of this model in health care systems-trained to collect large numbers of parameters-may harmonize decision making globally. (Synergy Between PCI With TAXUS and Cardiac Surgery: SYNTAX Extended Survival [SYNTAXES]; NCT03417050; SYNTAX Study: TAXUS Drug-Eluting Stent Versus Coronary Artery Bypass Surgery for the Treatment of Narrowed Arteries; NCT00114972).

Heterogeneous treatment effects of coronary artery bypass grafting in ischemic cardiomyopathy: A machine learning causal forest analysis

OBJECTIVES

We aim to evaluate the heterogeneous treatment effects of coronary artery bypass grafting in patients with ischemic cardiomyopathy and to identify a group of patients to have greater benefits from coronary artery bypass grafting compared with medical therapy alone.

METHODS

Machine learning causal forest modeling was performed to identify the heterogeneous treatment effects of coronary artery bypass grafting in patients with ischemic cardiomyopathy from the Surgical Treatment for Ischemic Heart Failure trial. The risks of death from any cause and death from cardiovascular causes between coronary artery bypass grafting and medical therapy alone were assessed in the identified subgroups.

RESULTS

Among 1212 patients enrolled in the Surgical Treatment for Ischemic Heart Failure trial, left ventricular end-systolic volume index, serum creatinine, and age were identified by the machine learning algorithm to distinguish patients with heterogeneous treatment effects. Among patients with left ventricular end-systolic volume index greater than 84 mL/m2 and age 60.27 years or less, coronary artery bypass grafting was associated with a significantly lower risk of death from any cause (adjusted hazard ratio, 0.61; 95% CI, 0.45-0.84) and death from cardiovascular causes (adjusted hazard ratio, 0.63; 95% CI, 0.45-0.89). By contrast, the survival benefits of coronary artery bypass grafting no longer exist in patients with left ventricular end-systolic volume index 84 mL/m2 or less and serum creatinine 1.04 mg/dL or less, or patients with left ventricular end-systolic volume index greater than 84 mL/m2 and age more than 60.27 years.

CONCLUSIONS

The current post hoc analysis of the Surgical Treatment for Ischemic Heart Failure trial identified heterogeneous treatment effects of coronary artery bypass grafting in patients with ischemic cardiomyopathy. Younger patients with severe left ventricular enlargement were more likely to derive greater survival benefits from coronary artery bypass grafting.

Prognosis and prediction of antibiotic benefit in adults with clinically diagnosed acute rhinosinusitis: an individual participant data meta-analysis

BACKGROUND

A previous individual participant data meta-analysis (IPD-MA) of antibiotics for adults with clinically diagnosed acute rhinosinusitis (ARS) showed a marginal overall effect of antibiotics, but was unable to identify patients that are most likely to benefit from antibiotics when applying conventional (i.e. univariable or one-variable-at-a-time) subgroup analysis. We updated the systematic review and investigated whether multivariable prediction of patient-level prognosis and antibiotic treatment effect may lead to more tailored treatment assignment in adults presenting to primary care with ARS.

METHODS

An IPD-MA of nine double-blind placebo-controlled trials of antibiotic treatment (n=2539) was conducted, with the probability of being cured at 8-15 days as the primary outcome. A logistic mixed effects model was developed to predict the probability of being cured based on demographic characteristics, signs and symptoms, and antibiotic treatment assignment. Predictive performance was quantified based on internal-external cross-validation in terms of calibration and discrimination performance, overall model fit, and the accuracy of individual predictions.

RESULTS

Results indicate that the prognosis with respect to risk of cure could not be reliably predicted (c-statistic 0.58 and Brier score 0.24). Similarly, patient-level treatment effect predictions did not reliably distinguish between those that did and did not benefit from antibiotics (c-for-benefit 0.50).

CONCLUSIONS

In conclusion, multivariable prediction based on patient demographics and common signs and symptoms did not reliably predict the patient-level probability of cure and antibiotic effect in this IPD-MA. Therefore, these characteristics cannot be expected to reliably distinguish those that do and do not benefit from antibiotics in adults presenting to primary care with ARS.

Evaluation of randomized controlled trials: a primer and tutorial for mental health researchers

BACKGROUND

Considered one of the highest levels of evidence, results of randomized controlled trials (RCTs) remain an essential building block in mental health research. They are frequently used to confirm that an intervention "works" and to guide treatment decisions. Given their importance in the field, it is concerning that the quality of many RCT evaluations in mental health research remains poor. Common errors range from inadequate missing data handling and inappropriate analyses (e.g., baseline randomization tests or analyses of within-group changes) to unduly interpretations of trial results and insufficient reporting. These deficiencies pose a threat to the robustness of mental health research and its impact on patient care. Many of these issues may be avoided in the future if mental health researchers are provided with a better understanding of what constitutes a high-quality RCT evaluation.

METHODS

In this primer article, we give an introduction to core concepts and caveats of clinical trial evaluations in mental health research. We also show how to implement current best practices using open-source statistical software.

RESULTS

Drawing on Rubin's potential outcome framework, we describe that RCTs put us in a privileged position to study causality by ensuring that the potential outcomes of the randomized groups become exchangeable. We discuss how missing data can threaten the validity of our results if dropouts systematically differ from non-dropouts, introduce trial estimands as a way to co-align analyses with the goals of the evaluation, and explain how to set up an appropriate analysis model to test the treatment effect at one or several assessment points. A novice-friendly tutorial is provided alongside this primer. It lays out concepts in greater detail and showcases how to implement techniques using the statistical software R, based on a real-world RCT dataset.

DISCUSSION

Many problems of RCTs already arise at the design stage, and we examine some avoidable and unavoidable "weak spots" of this design in mental health research. For instance, we discuss how lack of prospective registration can give way to issues like outcome switching and selective reporting, how allegiance biases can inflate effect estimates, review recommendations and challenges in blinding patients in mental health RCTs, and describe problems arising from underpowered trials. Lastly, we discuss why not all randomized trials necessarily have a limited external validity and examine how RCTs relate to ongoing efforts to personalize mental health care.

A scoping review of statistical methods in studies of biomarker-related treatment heterogeneity for breast cancer

BACKGROUND

Many scientific papers are published each year and substantial resources are spent to develop biomarker-based tests for precision oncology. However, only a handful of tests is currently used in daily clinical practice, since development is challenging. In this situation, the application of adequate statistical methods is essential, but little is known about the scope of methods used.

METHODS

A PubMed search identified clinical studies among women with breast cancer comparing at least two different treatment groups, one of which chemotherapy or endocrine treatment, by levels of at least one biomarker. Studies presenting original data published in 2019 in one of 15 selected journals were eligible for this review. Clinical and statistical characteristics were extracted by three reviewers and a selection of characteristics for each study was reported.

RESULTS

Of 164 studies identified by the query, 31 were eligible. Over 70 different biomarkers were evaluated. Twenty-two studies (71%) evaluated multiplicative interaction between treatment and biomarker. Twenty-eight studies (90%) evaluated either the treatment effect in biomarker subgroups or the biomarker effect in treatment subgroups. Eight studies (26%) reported results for one predictive biomarker analysis, while the majority performed multiple evaluations, either for several biomarkers, outcomes and/or subpopulations. Twenty-one studies (68%) claimed to have found significant differences in treatment effects by biomarker level. Fourteen studies (45%) mentioned that the study was not designed to evaluate treatment effect heterogeneity.

CONCLUSIONS

Most studies evaluated treatment heterogeneity via separate analyses of biomarker-specific treatment effects and/or multiplicative interaction analysis. There is a need for the application of more efficient statistical methods to evaluate treatment heterogeneity in clinical studies.

Heterogeneous treatment effects of intensive glycemic control on kidney microvascular outcomes in ACCORD

Objective

Clear criteria to individualize glycemic targets are lacking. In this post-hoc analysis of the Action to Control Cardiovascular Risk in Diabetes trial (ACCORD), we evaluate whether the kidney failure risk equation (KFRE) can identify patients who disproportionately benefit from intensive glycemic control on kidney microvascular outcomes.

Research design and methods

We divided the ACCORD trial population in quartiles based on 5-year kidney failure risk using the KFRE. We estimated conditional treatment effects within each quartile and compared them to the average treatment effect in the trial. The treatment effects of interest were the 7-year restricted-mean-survival-time (RMST) differences between intensive and standard glycemic control arms on (1) time-to-first development of severely elevated albuminuria or kidney failure and (2) all-cause mortality.

Results

We found evidence that the effect of intensive glycemic control on kidney microvascular outcomes and all-cause mortality varies with baseline risk of kidney failure. Patients with elevated baseline risk of kidney failure benefitted the most from intensive glycemic control on kidney microvascular outcomes (7-year RMST difference of 115 v. 48 days in the entire trial population) However, this same patient group also experienced shorter times to death (7-year RMST difference of -57 v. -24 days).

Conclusions

We found evidence of heterogenous treatment effects of intensive glycemic control on kidney microvascular outcomes in ACCORD as a function of predicted baseline risk of kidney failure. Patients with higher kidney failure risk experienced the most pronounced benefits of treatment on kidney microvascular outcomes but also experienced the highest risk of all-cause mortality.

Use of the Shizuoka Hip Fracture Prognostic Score (SHiPS) to Predict Long-Term Mortality in Patients With Hip Fracture in Japan: A Cohort Study Using the Shizuoka Kokuho Database

Hip fractures are common in patients of advanced age and are associated with excess mortality. Rapid and accurate prediction of the prognosis using information that can be easily obtained before surgery would be advantageous to clinical management. We performed a population-based retrospective cohort study using an 8.5-year Japanese claims database (April 2012-September 2020) to develop and validate a predictive model for long-term mortality after hip fracture. The study included 43,529 patients (34,499 [79.3%] women) aged ≥65 years with first-onset hip fracture. During the observation period, 43% of the patients died. Cox regression analysis identified the following prognostic predictors: sex, age, fracture site, nursing care certification, and several comorbidities (any malignancy, renal disease, congestive heart failure, chronic pulmonary disease, liver disease, metastatic solid tumor, and deficiency anemia). We then developed a scoring system called the Shizuoka Hip Fracture Prognostic Score (SHiPS); this system was established by scoring based on each hazard ratio and classifying the degree of mortality risk into four categories based on decision tree analysis. The area under the receiver operating characteristic (ROC) curve (AUC) (95% confidence interval [CI]) of 1-year, 3-year, and 5-year mortality based on the SHiPS was 0.718 (95% CI, 0.706-0.729), 0.736 (95% CI, 0.728-0.745), and 0.758 (95% CI, 0.747-0.769), respectively, indicating good predictive performance of the SHiPS for as long as 5 years after fracture onset. Even when the SHiPS was individually applied to patients with or without surgery after fracture, the prediction performance by the AUC was >0.7. These results indicate that the SHiPS can predict long-term mortality using preoperative information regardless of whether surgery is performed after hip fracture.

Overall average treatment effects from clinical trials, one-variable-at-a-time subgroup analyses and predictive approaches to heterogeneous treatment effects: Toward a more patient-centered evidence-based medicine

Despite the predominance of the evidence-based medicine paradigm, a fundamental incongruity remains: Evidence is derived from groups of people, yet medical decisions are made by and for individuals. Randomization ensures the comparability of treatment groups within a clinical trial, which allows for unbiased estimation of average treatment effects. If we treated groups of patients instead of individuals, or if patients with the same disease were identical to one another in all factors that determined the harms and the benefits of therapy, then these group-level averages would make a perfectly sound foundation for medical decision-making. But patients differ from one another in many ways that determine the likelihood of an outcome, both with and without a treatment. Nevertheless, popular approaches to evidence-based medicine have encouraged a reliance on the average treatment effects estimated from clinical trials and meta-analysis as guides to decision-making for individuals. Here, we discuss the limitations of this approach as well as limitations of conventional, one-variable-at-a-time subgroup analysis; finally, we discuss the rationale for "predictive" approaches to heterogeneous treatment effects. Predictive approaches to heterogeneous treatment effects combine methods for causal inference (e.g. randomization) with methods for prediction that permit inferences about which patients are likely to benefit and which are not, taking into account multiple relevant variables simultaneously to yield "personalized" estimates of benefit-harm trade-offs. We focus on risk modeling approaches, which rely on the mathematical dependence of the absolute treatment effect with the baseline risk, which varies substantially "across patients" in most trials. While there are a number of examples of risk modeling approaches that have been practice-changing, risk modeling does not provide ideal estimates of individual treatment effects, since risk modeling does not account for how individual variables might modify the effects of therapy. In "effect modeling," prediction models are developed directly on clinical trial data, including terms for treatment and treatment effect interactions. These more flexible approaches may better uncover individualized treatment effects, but are also prone to overfitting when dimensionality is high, power is low, and there is limited prior knowledge about effect modifiers.

Heterogeneity of Treatment Effects of Hydrocortisone by Risk of Bronchopulmonary Dysplasia or Death Among Extremely Preterm Infants in the National Institute of Child Health and Human Development Neonatal Research Network Trial: A Secondary Analysis of a Randomized Clinical Trial

Importance

Extremely preterm infants who develop bronchopulmonary dysplasia (BPD) are at a higher risk for adverse pulmonary and neurodevelopmental outcomes. In the National Institute of Child Health and Human Development Neonatal Research Network (NICHD NRN) Hydrocortisone Trial, hydrocortisone neither reduced rates of BPD or death nor increased rates of neurodevelopmental impairment (NDI) or death.

Objective

To determine whether estimated risk for grades 2 to 3 BPD or death is associated with the effect of hydrocortisone on the composite outcomes of (1) grades 2 to 3 BPD or death and (2) moderate or severe NDI or death.

Design, Setting, and Participants

This secondary post hoc analysis used data from the NICHD NRN Hydrocortisone Trial, which was a double-masked, placebo-controlled, randomized clinical trial conducted in 19 US academic centers. The NICHD HRN Hydrocortisone Trial enrolled infants born at a gestational age of less than 30 weeks who received mechanical ventilation for at least 7 days, including at the time of enrollment, and who were aged 14 to 28 postnatal days. Infants were enrolled between August 22, 2011, and February 4, 2018, with follow-up between 22 and 26 months of corrected age completed on March 29, 2020. Data were analyzed from September 13, 2021, to March 25, 2023.

Intervention

Infants were randomized to 10 days of hydrocortisone or placebo treatment.

Main Outcomes and Measures

Infants' baseline risk of grades 2 to 3 BPD or death was estimated using the NICHD Neonatal BPD Outcome Estimator. Differences in absolute and relative treatment effects by baseline risk were evaluated using interaction terms in models fitted to the efficacy outcome of grades 2 to 3 BPD or death and the safety outcome of moderate or severe NDI or death by follow-up.

Results

Among the 799 infants included in the analysis (421 boys [52.7%]), the mean (SD) gestational age was 24.9 (1.5) weeks, and the mean (SD) birth weight was 715 (167) g. The mean estimated baseline risk for grades 2 to 3 BPD or death was 54% (range, 18%-84%) in the study population. The interaction between treatment group and baseline risk was not statistically significant on a relative or absolute scale for grades 2 to 3 BPD or death; the size of the effect ranged from a relative risk of 1.13 (95% CI, 0.82-1.55) in quartile 1 to 0.94 (95% CI, 0.81-1.09) in quartile 4. Similarly, the interaction between treatment group and baseline risk was not significant on a relative or absolute scale for moderate or severe NDI or death; the size of the effect ranged from a relative risk of 1.04 (95% CI, 0.80-1.36) in quartile 1 to 0.99 (95% CI, 0.80-1.22) in quartile 4.

Conclusions and Relevance

In this secondary analysis of a randomized clinical trial, the effect of hydrocortisone vs placebo was not appreciably modified by baseline risk for grades 2 to 3 BPD or death.

Trial Registration

ClinicalTrials.gov Identifier: NCT01353313.

Identifying Patients for Intensive Blood Pressure Treatment Based on Cognitive Benefit: A Secondary Analysis of the SPRINT Randomized Clinical Trial

Importance

Intensive vs standard treatment to lower systolic blood pressure (SBP) reduces risk of mild cognitive impairment (MCI) or dementia; however, the magnitude of cognitive benefit likely varies among patients.

Objective

To estimate the magnitude of cognitive benefit of intensive vs standard systolic BP (SBP) treatment.

Design, Setting, and Participants

In this ad hoc secondary analysis of the Systolic Blood Pressure Intervention Trial (SPRINT), 9361 randomized clinical trial participants 50 years or older with high cardiovascular risk but without a history of diabetes, stroke, or dementia were followed up. The SPRINT trial was conducted between November 1, 2010, and August 31, 2016, and the present analysis was completed on October 31, 2022.

Intervention

Systolic blood pressure treatment to an intensive (<120 mm Hg) vs standard (<140 mm Hg) target.

Main Outcomes and Measures

The primary outcome was a composite of adjudicated probable dementia or amnestic MCI.

Results

A total of 7918 SPRINT participants were included in the analysis; 3989 were in the intensive treatment group (mean [SD] age, 67.9 [9.2] years; 2570 [64.4%] men; 1212 [30.4%] non-Hispanic Black) and 3929 were in the standard treatment group (mean [SD] age, 67.9 [9.4] years; 2570 [65.4%] men; 1249 [31.8%] non-Hispanic Black). Over a median follow-up of 4.13 (IQR, 3.50-5.88) years, there were 765 and 828 primary outcome events in the intensive treatment group and standard treatment group, respectively. Older age (hazard ratio [HR] per 1 SD, 1.87 [95% CI, 1.78-1.96]), Medicare enrollment (HR per 1 SD, 1.42 [95% CI, 1.35-1.49]), and higher baseline serum creatinine level (HR per 1 SD, 1.24 [95% CI, 1.19-1.29]) were associated with higher risk of the primary outcome, while better baseline cognitive functioning (HR per 1 SD, 0.43 [95% CI, 0.41-0.44]) and active employment status (HR per 1 SD, 0.44 [95% CI, 0.42-0.46]) were associated with lower risk of the primary outcome. Risk of the primary outcome by treatment goal was estimated accurately based on similar projected and observed absolute risk differences (C statistic = 0.79). Higher baseline risk for the primary outcome was associated with greater benefit (ie, larger absolute reduction of probable dementia or amnestic MCI) of intensive vs standard treatment across the full range of estimated baseline risk.

Conclusions and Relevance

In this secondary analysis of the SPRINT trial, participants with higher baseline projected risk of probable dementia or amnestic MCI gained greater absolute cognitive benefit from intensive vs standard SBP treatment in a monotonic fashion.

Trial Registration

ClinicalTrials.gov Identifier: NCT01206062.

Measuring the performance of prediction models to personalize treatment choice

When data are available from individual patients receiving either a treatment or a control intervention in a randomized trial, various statistical and machine learning methods can be used to develop models for predicting future outcomes under the two conditions, and thus to predict treatment effect at the patient level. These predictions can subsequently guide personalized treatment choices. Although several methods for validating prediction models are available, little attention has been given to measuring the performance of predictions of personalized treatment effect. In this article, we propose a range of measures that can be used to this end. We start by defining two dimensions of model accuracy for treatment effects, for a single outcome: discrimination for benefit and calibration for benefit. We then amalgamate these two dimensions into an additional concept, decision accuracy, which quantifies the model's ability to identify patients for whom the benefit from treatment exceeds a given threshold. Subsequently, we propose a series of performance measures related to these dimensions and discuss estimating procedures, focusing on randomized data. Our methods are applicable for continuous or binary outcomes, for any type of prediction model, as long as it uses baseline covariates to predict outcomes under treatment and control. We illustrate all methods using two simulated datasets and a real dataset from a trial in depression. We implement all methods in the R package predieval. Results suggest that the proposed measures can be useful in evaluating and comparing the performance of competing models in predicting individualized treatment effect.

Estimating individualized treatment effects using a risk-modeling approach: an application to epidural steroid injections for lumbar spinal stenosis

ABSTRACT

Conventional "1-variable-at-a-time" analyses to identify treatment effect modifiers are often underpowered and prone to false-positive results. This study used a "risk-modeling" approach guided by the Predictive Approaches to Treatment effect Heterogeneity (PATH) Statement framework: (1) developing and validating a multivariable model to estimate predicted future back-related functional limitations as measured by the Roland-Morris Disability Questionnaire (RMDQ) and (2) stratifying patients from a randomized controlled trial (RCT) of lumbar epidural steroid injections (LESIs) for the treatment of lumbar spinal stenosis into subgroups with different individualized treatment effects on RMDQ scores at the 3-week follow-up. Model development and validation were conducted in a cohort (n = 3259) randomly split into training and testing sets in a 4:1 ratio. The model was developed in the testing set using linear regression with least absolute shrinkage and selection regularization and 5-fold cross-validation. The model was then applied in the testing set and subsequently in patients receiving the control treatment in the RCT of LESI. R2 values in the training set, testing set, and RCT were 0.38, 0.32, and 0.34, respectively. There was statistically significant modification ( P = 0.03) of the LESI treatment effect according to predicted risk quartile, with clinically relevant LESI treatment effect point estimates in the 2 quartiles with greatest predicted risk (-3.7 and -3.3 RMDQ points) and no effect in the lowest 2 quartiles. A multivariable risk-modeling approach identified subgroups of patients with lumbar spinal stenosis with a clinically relevant treatment effect of LESI on back-related functional limitations.

Admission systolic blood pressure and effect of endovascular treatment in patients with ischaemic stroke: an individual patient data meta-analysis

BACKGROUND

Current guidelines for ischaemic stroke treatment recommend a strict, but arbitrary, upper threshold of 185/110 mm Hg for blood pressure before endovascular thrombectomy. Nevertheless, whether admission blood pressure influences the effect of endovascular thrombectomy on outcome remains unknown. Our aim was to study the influence of admission systolic blood pressure (SBP) on functional outcome and on the effect of endovascular thrombectomy.

METHODS

We used individual patient data from seven randomised controlled trials (MR CLEAN, ESCAPE, EXTEND-IA, SWIFT PRIME, REVASCAT, PISTE, and THRACE) that randomly assigned patients with anterior circulation ischaemic stroke to endovascular thrombectomy (predominantly using stent retrievers) or standard medical therapy (control) between June 1, 2010, and April 30, 2015. We included all patients for whom SBP data were available at hospital admission. The primary outcome was functional outcome (modified Rankin Scale) at 90 days. We assessed the association of SBP with outcome in both the endovascular thrombectomy group and the control group using multilevel regression analysis and tested for non-linearity and for interaction between SBP and effect of endovascular thrombectomy, taking into account treatment with intravenous thrombolysis.

FINDINGS

We included 1753 patients (867 assigned to endovascular thrombectomy, 886 assigned to control) after excluding 11 patients for whom SBP data were missing. We found a non-linear association between SBP and functional outcome with an inflection point at 140 mm Hg (732 [42%] of 1753 patients had SBP <140 mm Hg and 1021 [58%] had SBP ≥140 mm Hg). Among patients with SBP of 140 mm Hg or higher, admission SBP was associated with worse functional outcome (adjusted common odds ratio [acOR] 0·86 per 10 mm Hg SBP increase; 95% CI 0·81-0·91). We found no association between SBP and functional outcome in patients with SBP less than 140 mm Hg (acOR 0·97 per 10 mm Hg SBP decrease, 95% CI 0·88-1·05). There was no significant interaction between SBP and effect of endovascular thrombectomy on functional outcome (p=0·96).

INTERPRETATION

In our meta-analysis, high admission SBP was associated with worse functional outcome after stroke, but SBP did not seem to negate the effect of endovascular thrombectomy. This finding suggests that admission SBP should not form the basis for decisions to withhold or delay endovascular thrombectomy for ischaemic stroke, but randomised trials are needed to further investigate this possibility.

FUNDING

Medtronic.

A standardized framework for risk-based assessment of treatment effect heterogeneity in observational healthcare databases

Treatment effects are often anticipated to vary across groups of patients with different baseline risk. The Predictive Approaches to Treatment Effect Heterogeneity (PATH) statement focused on baseline risk as a robust predictor of treatment effect and provided guidance on risk-based assessment of treatment effect heterogeneity in a randomized controlled trial. The aim of this study is to extend this approach to the observational setting using a standardized scalable framework. The proposed framework consists of five steps: (1) definition of the research aim, i.e., the population, the treatment, the comparator and the outcome(s) of interest; (2) identification of relevant databases; (3) development of a prediction model for the outcome(s) of interest; (4) estimation of relative and absolute treatment effect within strata of predicted risk, after adjusting for observed confounding; (5) presentation of the results. We demonstrate our framework by evaluating heterogeneity of the effect of thiazide or thiazide-like diuretics versus angiotensin-converting enzyme inhibitors on three efficacy and nine safety outcomes across three observational databases. We provide a publicly available R software package for applying this framework to any database mapped to the Observational Medical Outcomes Partnership Common Data Model. In our demonstration, patients at low risk of acute myocardial infarction receive negligible absolute benefits for all three efficacy outcomes, though they are more pronounced in the highest risk group, especially for acute myocardial infarction. Our framework allows for the evaluation of differential treatment effects across risk strata, which offers the opportunity to consider the benefit-harm trade-off between alternative treatments.

Estimating individualized treatment effects from randomized controlled trials: a simulation study to compare risk-based approaches

BACKGROUND

Baseline outcome risk can be an important determinant of absolute treatment benefit and has been used in guidelines for "personalizing" medical decisions. We compared easily applicable risk-based methods for optimal prediction of individualized treatment effects.

METHODS

We simulated RCT data using diverse assumptions for the average treatment effect, a baseline prognostic index of risk, the shape of its interaction with treatment (none, linear, quadratic or non-monotonic), and the magnitude of treatment-related harms (none or constant independent of the prognostic index). We predicted absolute benefit using: models with a constant relative treatment effect; stratification in quarters of the prognostic index; models including a linear interaction of treatment with the prognostic index; models including an interaction of treatment with a restricted cubic spline transformation of the prognostic index; an adaptive approach using Akaike's Information Criterion. We evaluated predictive performance using root mean squared error and measures of discrimination and calibration for benefit.

RESULTS

The linear-interaction model displayed optimal or close-to-optimal performance across many simulation scenarios with moderate sample size (N = 4,250; ~ 785 events). The restricted cubic splines model was optimal for strong non-linear deviations from a constant treatment effect, particularly when sample size was larger (N = 17,000). The adaptive approach also required larger sample sizes. These findings were illustrated in the GUSTO-I trial.

CONCLUSIONS

An interaction between baseline risk and treatment assignment should be considered to improve treatment effect predictions.

The impact of heterotopic ossification prophylaxis after surgical fixation of acetabular fractures: national treatment patterns and related outcomes

BACKGROUND

Heterotopic ossification (HO) is a common complication after surgical fixation of acetabular fractures. Numerous strategies have been employed to prevent HO formation, but results are mixed and optimal treatment strategy remains controversial. The purpose of the study was to describe current national heterotopic ossification (HO) prophylaxis patterns among academic trauma centers, determine the association between prophylaxis type and radiographic HO, and identify if heterogeneity in treatment effects exist based on outcome risk strata.

METHODS

We used data from a subset of participants enrolled in the Pragmatic Randomized Trial Evaluating Pre-Operative Alcohol Skin Solutions in Fractured Extremities (PREPARE) trial. We included only patients with closed AO-type 62 acetabular fractures that were surgically treated via a posterior (Kocher-Langenbeck), combined anterior and posterior, or extensile exposure. PREPARE Clinical Trial Registration Number: NCT03523962 Patient population This cohort study was nested within the Pragmatic Randomized Trial Evaluating Pre-Operative Alcohol Skin Solutions in Fractured Extremities (PREPARE) trial. The PREPARE trial is a multicenter cluster-randomized crossover trial evaluating the effectiveness of two alcohol-based pre-operative antiseptic skin solutions. All PREPARE trial clinical centers that enrolled at least one patient with a closed AO-type 62 acetabular fracture were invited to participate in the nested study.

RESULTS

277 patients from 20 level 1 and level 2 trauma centers in the U.S. and Canada were included in this study. 32 patients (12%) received indomethacin prophylaxis, 100 patients (36%) received XRT prophylaxis, and 145 patients (52%) received no prophylaxis. Administration of XRT was associated with a 68% reduction in the adjusted odds of overall HO (OR 0.32, 95% CI, 0.14 - 0.69, p = 0.005). The overall severe HO (Brooker classes III or IV) rate was 8% for the entire cohort; XRT reduced the rate of severe HO in high-risk patients only (p=0.03).

CONCLUSION

HO prophylaxis patterns after surgical fixation of acetabular fractures have changed dramatically over the last two decades. Most centers included in this study did not administer HO prophylaxis. XRT was associated with a marked reduction in the rate of overall HO and the rate of severe HO in high-risk patients. Randomized trials are needed to fully elucidate the potential benefit of XRT. PREPARE Clinical Trial Registration Number: NCT03523962.

Heterogeneous treatment effects of dexamethasone 12 mg versus 6 mg in patients with COVID-19 and severe hypoxaemia-Post hoc exploratory analyses of the COVID STEROID 2 trial

BACKGROUND

Corticosteroids improve outcomes in patients with severe COVID-19. In the COVID STEROID 2 randomised clinical trial, we found high probabilities of benefit with dexamethasone 12 versus 6 mg daily. While no statistically significant heterogeneity in treatment effects (HTE) was found in the conventional, dichotomous subgroup analyses, these analyses have limitations, and HTE could still exist.

METHODS

We assessed whether HTE was present for days alive without life support and mortality at Day 90 in the trial according to baseline age, weight, number of comorbidities, category of respiratory failure (type of respiratory support system and oxygen requirements) and predicted risk of mortality using an internal prediction model. We used flexible models for continuous variables and logistic regressions for categorical variables without dichotomisation of the baseline variables of interest. HTE was assessed both visually and with p and S values from likelihood ratio tests.

RESULTS

There was no strong evidence for substantial HTE on either outcome according to any of the baseline variables assessed with all p values >.37 (and all S values <1.43) in the planned analyses and no convincingly strong visual indications of HTE.

CONCLUSIONS

We found no strong evidence for HTE with 12 versus 6 mg dexamethasone daily on days alive without life support or mortality at Day 90 in patients with COVID-19 and severe hypoxaemia, although these results cannot rule out HTE either.

Risk Model Development and Validation in Clinical Oncology: Lessons Learned

Reliable risk models can greatly facilitate patient-centered inferences and decisions. Herein we summarize key considerations related to risk modeling in clinical oncology. Often overlooked challenges include data quality, missing data, effective sample size estimation, and selecting the variables to be included in the risk model. The stability and quality of the model should be carefully interrogated with particular emphasis on rigorous internal validation.

Endovascular versus neurosurgical aneurysm treatment: study protocol for the development and validation of a clinical prediction tool for individualised decision making

INTRODUCTION

Treatment decisions for aneurysmal subarachnoid haemorrhage patients should be supported by individualised predictions of the effects of aneurysm treatment. We present a study protocol and analysis plan for the development and external validation of models to predict benefit of neurosurgical versus endovascular aneurysm treatment on functional outcome and durability of treatment.

METHODS AND ANALYSIS

We will use data from the International Subarachnoid Aneurysm Trial for model development. The outcomes are functional outcome, measured with modified Rankin Scale at 12 months, and any retreatment or rebleed of the target aneurysm during follow-up. We will develop an ordinal logistic regression model and Cox regression model, considering age, World Federation of Neurological Surgeons grade, Fisher grade, vasospasm at presentation, aneurysm lumen size, aneurysm neck size, aneurysm location and time-to-aneurysm-treatment as predictors. We will test for interactions with treatment and with baseline risk and derive individualised predicted probabilities of treatment benefit. A benefit of ≥5% will be considered clinically relevant. Discriminative performance of the outcome predictions will be assessed with the c-statistic. Calibration will be assessed with calibration plots. Discriminative performance of the benefit predictions will be assessed with the c-for benefit. We will assess internal validity with bootstrapping and external validity with leave-one-out internal-external cross-validation.

ETHICS AND DISSEMINATION

The medical ethical research committee of the Erasmus MC University Medical Center Rotterdam approved the study protocol under the exemption category and waived the need for written informed consent (MEC-2020-0810). We will disseminate our results through an open-access peer-reviewed scientific publication and with a web-based clinical prediction tool.

Evaluation and validation of a prediction model for extubation success in very preterm infants

OBJECTIVE

To perform an external validation of a publicly available model predicting extubation success in very preterm infants.

STUDY DESIGN

Retrospective study of infants born <1250 g at a single center. Model performance evaluated using the area under the receiver operating characteristic curve (AUROC) and comparing observed and expected probabilities of extubation success, defined as survival ≥5 d without an endotracheal tube.

RESULTS

Of 177 infants, 120 (68%) were extubated successfully. The median (IQR) gestational age was 27 weeks (25-28) and weight at extubation was 915 g (755-1050). The model had acceptable discrimination (AUROC 0.72 [95% CI 0.65-0.80]) and adequate calibration (calibration slope 0.96, intercept -0.06, mean observed-to-expected difference in probability of extubation success -0.08 [95% CI -0.01, -0.15]).

CONCLUSIONS

The extubation success prediction model has acceptable performance in an external cohort. Additional prospective studies are needed to determine if the model can be improved or how it can be used for clinical benefit.

Optimizing Use of Large Databases in Joint Arthroplasty and Orthopaedics

The potential users of "big data" need to consider many factors when choosing whether to use a large observational database for their research question and, if so, which database is the best fit for the scientific question. The first section of this paper, written by Dr. James A. Browne, provides a framework (who, what, where, when, and why?) to assess the critical elements that are included in a large database, which allows the user to determine if interrogation of the data is likely to answer the research question. The next section of this paper, written by Dr. Bryan Springer, focuses on the importance of having an a priori research question before deciding the best data source to answer the question; it also elaborates on the differences between administrative databases and clinical databases. The final section of the paper, written by Dr. Kurt P. Spindler, reviews the concepts of hypothesis-generating and hypothesis-testing studies and discusses in detail the differences, strengths, limitations, and appropriate uses of observational data versus randomized controlled trials.

Personalized Data Science and Personalized (N-of-1) Trials: Promising Paradigms for Individualized Health Care

The term 'data science' usually refers to the process of extracting value from big data obtained from a large group of individuals. An alternative rendition, which we call personalized data science (Per-DS), aims to collect, analyze, and interpret personal data to inform personal decisions. This article describes the main features of Per-DS, and reviews its current state and future outlook. A Per-DS investigation is of, by, and for an individual, the Per-DS investigator, acting simultaneously as her own investigator, study participant, and beneficiary, and making personalized decisions for study design and implementation. The scope of Per-DS studies may include systematic monitoring of physiological or behavioral patterns, case-crossover studies for symptom triggers, pre-post trials for exposure-outcome relationships, and personalized (N-of-1) trials for effectiveness. Per-DS studies produce personal knowledge generalizable to the individual's future self (thus benefiting herself) rather than knowledge generalizable to an external population (thus benefiting others). This endeavor requires a pivot from data mining or extraction to data gardening, analogous to home gardeners producing food for home consumption-the Per-DS investigator needs to 'cultivate the field' by setting goals, specifying study design, identifying necessary data elements, and assembling instruments and tools for data collection. Then, she can implement the study protocol, harvest her personal data, and mine the data to extract personal knowledge. To facilitate Per-DS studies, Per-DS investigators need support from community-based, scientific, philanthropic, business, and government entities, to develop and deploy resources such as peer forums, mobile apps, 'virtual field guides,' and scientific and regulatory guidance.

A Causal Framework for Making Individualized Treatment Decisions in Oncology

Simple Summary

Physicians routinely make individualized treatment decisions by accounting for the joint effects of patient prognostic covariates and treatments on clinical outcomes. Ideally, this is performed using historical randomized clinical trial (RCT) data. Randomization ensures that unbiased estimates of causal treatment effect parameters can be obtained from the historical RCT data and used to predict each new patient’s outcome based on the joint effect of their baseline covariates and each treatment being considered. However, this process becomes problematic if a patient seen in the clinic is very different from the patients who were enrolled in the RCT. That is, if a new patient does not satisfy the entry criteria of the RCT, then the patient does not belong to the population represented by the patients who were studied in the RCT. In such settings, it still may be possible to utilize the RCT data to help choose a new patient’s treatment. This may be achieved by combining the RCT data with data from other clinical trials, or possibly preclinical experiments, and using the combined dataset to predict the patient’s expected outcome for each treatment being considered. In such settings, combining data from multiple sources in a way that is statistically reliable is not entirely straightforward, and correctly identifying and estimating the effects of treatments and patient covariates on clinical outcomes can be complex. Causal diagrams provide a rational basis to guide this process. The first step is to construct a causal diagram that reflects the plausible relationships between treatment variables, patient covariates, and clinical outcomes. If the diagram is correct, it can be used to determine what additional data may be needed, how to combine data from multiple sources, how to formulate a statistical model for clinical outcomes as a function of treatment and covariates, and how to compute an unbiased treatment effect estimate for each new patient. We use adjuvant therapy of renal cell carcinoma to illustrate how causal diagrams may be used to guide these steps.

Abstract

We discuss how causal diagrams can be used by clinicians to make better individualized treatment decisions. Causal diagrams can distinguish between settings where clinical decisions can rely on a conventional additive regression model fit to data from a historical randomized clinical trial (RCT) to estimate treatment effects and settings where a different approach is needed. This may be because a new patient does not meet the RCT’s entry criteria, or a treatment’s effect is modified by biomarkers or other variables that act as mediators between treatment and outcome. In some settings, the problem can be addressed simply by including treatment–covariate interaction terms in the statistical regression model used to analyze the RCT dataset. However, if the RCT entry criteria exclude a new patient seen in the clinic, it may be necessary to combine the RCT data with external data from other RCTs, single-arm trials, or preclinical experiments evaluating biological treatment effects. For example, external data may show that treatment effects differ between histological subgroups not recorded in an RCT. A causal diagram may be used to decide whether external observational or experimental data should be obtained and combined with RCT data to compute statistical estimates for making individualized treatment decisions. We use adjuvant treatment of renal cell carcinoma as our motivating example to illustrate how to construct causal diagrams and apply them to guide clinical decisions.

Assessing the ability of an instrumental variable causal forest algorithm to personalize treatment evidence using observational data: the case of early surgery for shoulder fracture

Background

Comparative effectiveness research (CER) using observational databases has been suggested to obtain personalized evidence of treatment effectiveness. Inferential difficulties remain using traditional CER approaches especially related to designating patients to reference classes a priori. A novel Instrumental Variable Causal Forest Algorithm (IV-CFA) has the potential to provide personalized evidence using observational data without designating reference classes a priori, but the consistency of the evidence when varying key algorithm parameters remains unclear. We investigated the consistency of IV-CFA estimates through application to a database of Medicare beneficiaries with proximal humerus fractures (PHFs) that previously revealed heterogeneity in the effects of early surgery using instrumental variable estimators.

Methods

IV-CFA was used to estimate patient-specific early surgery effects on both beneficial and detrimental outcomes using different combinations of algorithm parameters and estimate variation was assessed for a population of 72,751 fee-for-service Medicare beneficiaries with PHFs in 2011. Classification and regression trees (CART) were applied to these estimates to create ex-post reference classes and the consistency of these classes were assessed. Two-stage least squares (2SLS) estimators were applied to representative ex-post reference classes to scrutinize the estimates relative to known 2SLS properties.

Results

IV-CFA uncovered substantial early surgery effect heterogeneity across PHF patients, but estimates for individual patients varied with algorithm parameters. CART applied to these estimates revealed ex-post reference classes consistent across algorithm parameters. 2SLS estimates showed that ex-post reference classes containing older, frailer patients with more comorbidities, and lower utilizers of healthcare were less likely to benefit and more likely to have detriments from higher rates of early surgery.

Conclusions

IV-CFA provides an illuminating method to uncover ex-post reference classes of patients based on treatment effects using observational data with a strong instrumental variable. Interpretation of treatment effect estimates within each ex-post reference class using traditional CER methods remains conditional on the extent of measured information in the data.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01663-0.

Designing clinically useful psychopharmacological trials: challenges and ways forward

The clinical guidelines that underpin the use of drugs for mental disorders are informed by evidence from randomised controlled trials (RCTs). RCTs are performed to obtain marketing authorisation from regulators. The methods used in these RCTs could be appropriate for early phases of drug development because they identify drugs with important harms and drugs that are efficacious for specific health problems and populations. RCTs done before marketing authorisation do not tend to address clinical questions that concern the effectiveness of a drug in heterogeneous and comorbid populations, the optimisation of drug sequencing and discontinuation, or the comparative benefits and harms of different drugs that could be used for the same health problem. This Review proposes an overview of some shortcomings of RCTs, at an individual level and at the whole portfolio level, and identifies some methods in planning, conducting, and carrying out analyses in RCTs that could enhance their ability to support therapeutic decisions. These suggestions include: identifying patient-important questions to be investigated by psychopharmacological RCTs; embedding pragmatic RCTs within clinical practice to improve generalisability to target populations; collecting evidence about drugs in overlooked populations; developing methods to facilitate the recruitment of patients with mental disorders and to reduce the number of patients who drop out, using specific methods; using core outcome sets to standardise the assessment of benefits and harms; and recording systematically serious objective outcomes, such as suicide or hospitalisation, to be evaluated in meta-analyses. This work is a call to address questions relevant to patients using diverse design of RCTs, thus contributing to the development of a patient-centred, evidence-based psychiatry.

Early predictors of functional outcome in poor-grade aneurysmal subarachnoid hemorrhage: a systematic review and meta-analysis

BACKGROUND

Patients with poor-grade aneurysmal subarachnoid hemorrhage (aSAH) often receive delayed or no aneurysm treatment, although recent studies suggest that functional outcome following early aneurysm treatment has improved. We aimed to systematically review and meta-analyze early predictors of functional outcome in poor-grade aSAH patients.  METHODS: We included studies investigating the association of early predictors and functional outcome in adult patients with confirmed poor-grade aSAH, defined as World Federation of Neurological Surgeons (WFNS) grade or Hunt and Hess (H-H) grade IV-V. Studies had to use multivariable regression analysis to estimate independent predictor effects of favorable functional outcome measured with the Glasgow Outcome Scale or modified Rankin Scale. We calculated pooled adjusted odds ratios (aOR) and 95% confidence intervals (CI) with random effects models.  RESULTS: We included 27 studies with 3287 patients. The likelihood of favorable outcome increased with WFNS grade or H-H grade IV versus V (aOR 2.9, 95% CI 1.9-4.3), presence of clinical improvement before aneurysm treatment (aOR 3.3, 95% CI 2.0-5.3), and intact pupillary light reflex (aOR 2.9, 95% CI 1.6-5.1), and decreased with older age (aOR 0.7, 95% CI 0.5-1.0, per decade), increasing modified Fisher grade (aOR 0.4, 95% CI 0.3-0.5, per grade), and presence of intracerebral hematoma on admission imaging (aOR 0.4, 95% CI 0.2-0.8).

CONCLUSIONS

We present a summary of early predictors of functional outcome in poor-grade aSAH patients that can help to discriminate between patients with favorable and with unfavorable prognosis and may aid in selecting patients for early aneurysm treatment.

A Machine-Learning Approach for Estimating Subgroup- and Individual-Level Treatment Effects: An Illustration Using the 65 Trial

Personalizing treatment recommendations or guidelines requires evidence about the heterogeneity of treatment effects (HTE). Machine-learning (ML) approaches can explore HTE by considering many covariates, including complex interactions between them. Causal ML approaches can avoid overfitting, which arises when the same dataset is used to select covariate by treatment interaction terms as to make inferences and reduce reliance on the correct specification of fixed parametric models. We investigate causal forests (CF), a ML method based on modified decision trees that can estimate subgroup- and individual-level treatment effects, without requiring correct prespecification of the effect model. We consider CF alongside parametric approaches for estimating HTE, within the 65 Trial, which evaluates the effect of a permissive hypotension strategy versus usual care on 90-d mortality for critically ill patients aged 65 y or older with vasodilatory hypotension. Here, the CF approach provides similar estimates of treatment effectiveness for prespecified and post hoc subgroups to the parametric approach, and the results of a test for overall HTE show weak evidence of heterogeneity. The CF estimates of individual-level treatment effects, the expected effects of treatment for individuals in subpopulations defined by their covariates, suggest that the permissive hypotension strategy is expected to reduce 90-d mortality for 98.7% of patients but with 95% confidence intervals that include zero for 71.6% of patients. A ML approach is then used to assess the patient characteristics associated with these individual-level effects, and to help target future research that can identify those patient subgroups for whom the intervention is most effective.

External Validation of the FREEDOM Score for Individualized Decision Making Between CABG and PCI

BACKGROUND

Although randomized trials have established that coronary artery bypass grafting (CABG) is, on average, the most effective revascularization strategy compared with percutaneous coronary intervention (PCI) in patients with diabetes and multivessel disease (MVD), individual patients differ in many characteristics that can affect the benefits and harms of treatment. The FREEDOM (Future Revascularization Evaluation in Patients with Diabetes Mellitus) score was developed to predict different outcomes with CABG vs PCI on the basis of 8 patient characteristics and the smoking-treatment interaction.

OBJECTIVES

This study aimed to assess the ability of the 5-year major adverse cardiovascular event (MACE) model to predict treatment benefit of CABG vs PCI in the SYNTAX (Synergy Between Percutaneous Coronary Intervention With Taxus and Cardiac Surgery) and BEST (Bypass Surgery and Everolimus-Eluting Stent Implantation in the Treatment of Patients with Multivessel Coronary Artery Disease) trials.

METHODS

This study identified 702 patients with diabetes and MVD to mirror the FREEDOM participants. Discrimination was assessed by C-index, and calibration was assessed by calibration plots in the PCI and CABG arms, respectively. The ability of the FREEDOM score to predict treatment benefit of CABG vs PCI was assessed.

RESULTS

Overall, CABG was associated with a lower rate of 5-year MACE compared with PCI (12.4% vs 20.3%; log-rank P = 0.021) irrespective of a history of smoking (P_interaction = 0.975). Both discrimination and calibration were helpful in the PCI arm (C-index: 0.69; slope: 0.96, intercept: -0.24), but moderate in the CABG arm (C-index: 0.61; slope: 0.61; intercept: -0.53). The FREEDOM score showed some heterogeneity of treatment benefit.

CONCLUSIONS

The FREEDOM score could identify some heterogeneity of treatment benefit of CABG vs PCI for 5-year MACE. Until further prospective validations are performed, these results should be taken into consideration when using the FREEDOM score in patients with diabetes and MVD. (Synergy Between Percutaneous Coronary Intervention With Taxus and Cardiac Surgery [SYNTAX]; NCT00114972) (Bypass Surgery and Everolimus-Eluting Stent Implantation in the Treatment of Patients with Multivessel Coronary Artery Disease [BEST]; NCT00997828) (Future Revascularization Evaluation in Patients with Diabetes Mellitus [FREEDOM]; NCT00086450).

An Electronic Health Record-Compatible Model to Predict Personalized Treatment Effects From the Diabetes Prevention Program: A Cross-Evidence Synthesis Approach Using Clinical Trial and Real-World Data

OBJECTIVE

To develop an electronic health record (EHR)-based risk tool that provides point-of-care estimates of diabetes risk to support targeting interventions to patients most likely to benefit.

PATIENTS AND METHODS

A risk prediction model was developed and validated in a large observational database of patients with an index visit date between January 1, 2012, and December 31, 2016, with treatment effect estimates from risk-based reanalysis of clinical trial data. The risk model development cohort included 1.1 million patients with prediabetes from the OptumLabs Data Warehouse (OLDW); the validation cohort included a distinct sample of 1.1 million patients in OLDW. The randomly assigned clinical trial cohort included 3081 people from the Diabetes Prevention Program (DPP) study.

RESULTS

Eleven variables reliably obtainable from the EHR were used to predict diabetes risk. This model validated well in the OLDW (C statistic = 0.76; observed 3-year diabetes rate was 1.8% (95% confidence interval [CI], 1.7 to 1.9) in the lowest-risk quarter and 19.6% (19.4 to 19.8) in the highest-risk quarter). In the DPP, the hazard ratio (HR) for lifestyle modification was constant across all levels of risk (HR, 0.43; 95% CI, 0.35 to 0.53), whereas the HR for metformin was highly risk dependent (HR, 1.1; 95% CI, 0.61 to 2.0 in the lowest-risk quarter vs HR, 0.45; 95% CI, 0.35 to 0.59 in the highest-risk quarter). Fifty-three percent of the benefits of population-wide dissemination of the DPP lifestyle modification and 73% of the benefits of population-wide metformin therapy can be obtained by targeting the highest-risk quarter of patients.

CONCLUSION

The Tufts-Predictive Analytics and Comparative Effectiveness DPP Risk model is an EHR-compatible tool that might support targeted diabetes prevention to more efficiently realize the benefits of the DPP interventions.