Recommendations for the Conduct and Reporting of Research Involving Flexible Electronic Health Record-Based Interventions

Conceptualizing clinical decision support as complex interventions: a meta-analysis of comparative effectiveness trials

OBJECTIVES

Complex interventions with multiple components and behavior change strategies are increasingly implemented as a form of clinical decision support (CDS) using native electronic health record functionality. Objectives of this study were, therefore, to (1) identify the proportion of randomized controlled trials with CDS interventions that were complex, (2) describe common gaps in the reporting of complexity in CDS research, and (3) determine the impact of increased complexity on CDS effectiveness.

MATERIALS AND METHODS

To assess CDS complexity and identify reporting gaps for characterizing CDS interventions, we used the Preferred Reporting Items for Systematic Reviews and Meta-Analyses reporting tool for complex interventions. We evaluated the effect of increased complexity using random-effects meta-analysis.

RESULTS

Most included studies evaluated a complex CDS intervention (76%). No studies described use of analytical frameworks or causal pathways. Two studies discussed use of theory but only one fully described the rationale and put it in context of a behavior change. A small but positive effect (standardized mean difference, 0.147; 95% CI, 0.039-0.255; P < .01) in favor of increasing intervention complexity was observed.

DISCUSSION

While most CDS studies should classify interventions as complex, opportunities persist for documenting and providing resources in a manner that would enable CDS interventions to be replicated and adapted. Unless reporting of the design, implementation, and evaluation of CDS interventions improves, only slight benefits can be expected.

CONCLUSION

Conceptualizing CDS as complex interventions may help convey the careful attention that is needed to ensure these interventions are contextually and theoretically informed.

OUP accepted manuscript

The increasing volume and richness of healthcare data collected during routine clinical practice have not yet translated into significant numbers of actionable insights that have systematically improved patient outcomes. An evidence-practice gap continues to exist in healthcare. We contest that this gap can be reduced by assessing the use of nudge theory as part of clinical decision support systems (CDSS). Deploying nudges to modify clinician behaviour and improve adherence to guideline-directed therapy represents an underused tool in bridging the evidence-practice gap. In conjunction with electronic health records (EHRs) and newer devices including artificial intelligence algorithms that are increasingly integrated within learning health systems, nudges such as CDSS alerts should be iteratively tested for all stakeholders involved in health decision-making: clinicians, researchers, and patients alike. Not only could they improve the implementation of known evidence, but the true value of nudging could lie in areas where traditional randomized controlled trials are lacking, and where clinical equipoise and variation dominate. The opportunity to test CDSS nudge alerts and their ability to standardize behaviour in the face of uncertainty may generate novel insights and improve patient outcomes in areas of clinical practice currently without a robust evidence base.

Clinician Versus Nomogram Predicted Estimates of Kidney Stone Recurrence Risk

Purpose: Kidney stone recurrence rates vary between patients. A patient's risk informs the frequency and intensity of preventative interventions. Clinicians routinely use clinical experience to estimate risk. We sought to compare clinician estimated recurrence risk with the recurrence of kidney stones (ROKS) nomogram. Materials and Methods: We surveyed members of the Endourological Society with clinical expertise in kidney stones. Respondents estimated the risk of recurrence for patients in three clinical vignettes corresponding to low, intermediate, and high recurrence risk from the nomogram. Clinician estimates were compared with ROKS estimates. Results: The majority of the 318 respondents were from North America (n = 127, 40%). The most commonly estimated recurrence was 50% at 5 years. The respondents' estimates were significantly different from the ROKS predicted recurrence rate for all cases (Case 1, 50% vs 93% p < 0.0001; Case 2, 50% vs 60% p < 0.0001; Case 3, 60% vs 22% p < 0.0001). The ROKS predicted estimates ranged from 22% to 93%, whereas the median urologist-derived 5-year risk estimates for each case ranged from 50% to 60%. The median range of estimates by respondents across cases was 20%, narrower than the 71% for the ROKS nomogram. The majority of respondents (95%) do not use nomograms in practice, mostly because of lack of awareness of useful nomograms (59%). Conclusions: This study suggests that clinicians may not be able to distinguish those with high and low recurrence risk when compared with peers and when compared with a nomogram. Clinical decision support tools are needed to enable clinicians to better estimate stone recurrence risk.