Designing, Conducting, and Reporting Clinical Decision Support Studies: Recommendations and Call to Action

Moonshot. Long shot. Or sure shot. What needs to happen to realize the full potential of AI in the fertility sector?

Quality healthcare requires two critical components: patients' best interests and best decisions to achieve that goal. The first goal is the lodestar, unchanged and unchanging over time. The second component is a more dynamic and rapidly changing paradigm in healthcare. Clinical decision-making has transitioned from an opinion-based paradigm to an evidence-based and data-driven process. A realization that technology and artificial intelligence can bring value adds a third component to the decision process. And the fertility sector is not exempt. The debate about AI is front and centre in reproductive technologies. Launching the transition from a conventional provider-driven decision paradigm to a software-enhanced system requires a roadmap to enable effective and safe implementation. A key nodal point in the ascending arc of AI in the fertility sector is how and when to bring these innovations into the ART routine to improve workflow, outcomes, and bottom-line performance. The evolution of AI in other segments of clinical care would suggest that caution is needed as widespread adoption is urged from several fronts. But the lure and magnitude for the change that these tech tools hold for fertility care remain deeply engaging. Exploring factors that could enhance thoughtful implementation and progress towards a tipping point (or perhaps not) should be at the forefront of any 'next steps' strategy. The objective of this Opinion is to discuss four critical areas (among many) considered essential to successful uptake of any new technology. These four areas include value proposition, innovative disruption, clinical agency, and responsible computing.

Digital Interventions to Support Lung Cancer Screening: A Systematic Review

INTRODUCTION

Lung cancer remains a leading cause of cancer-related deaths globally. Lung cancer screening (LCS) with low-dose computed tomography (LDCT) can reduce lung cancer mortality, but its adoption in the U.S. has been limited. Digital interventions have the potential to improve uptake of LCS. This systematic review aims to summarize the evidence for the effectiveness of digital interventions in promoting LCS.

METHODS

A systematic search of three electronic databases (PubMed, Embase, and Medline) was conducted to identify studies published between January 2014 and May 2023. Studies were reviewed and abstracted between February 2023 and July 2023. Outcomes related to knowledge, decision-making and screening were measured. Study quality was assessed using the Joanna Briggs Institute (JBI) critical appraisal tools.

RESULTS

Of 1,979 screened articles, 30 studies were included in this review. Digital interventions evaluated included decision aids (n=20), electronic health record (EHR)-based interventions (n=7), social media campaigns and mobile applications (n=3). Decision aids were the most commonly studied digital interventions, with most studies showing improved knowledge (13/13) and reduced decisional conflict (7/9) but most did not show a substantial change in screening use. Fewer studies tested clinician-facing or multi-level interventions.

DISCUSSION

Digital interventions, particularly decision aids, have shown promise in improving knowledge and the quality of decision-making around LCS. However, few interventions have been shown to substantially alter screening behavior and few clinician-facing or multi-level interventions have been rigorously tested. Further research is needed to develop effective tools for engaging patients in LCS, to compare the efficacy of different interventions, and evaluate implementation strategies in diverse healthcare settings.

Effect of clinical decision support for severe hypercholesterolemia on low-density lipoprotein cholesterol levels

Severe hypercholesterolemia/possible familial hypercholesterolemia (FH) is relatively common but underdiagnosed and undertreated. We investigated whether implementing clinical decision support (CDS) was associated with lower low-density lipoprotein cholesterol (LDL-C) in patients with severe hypercholesterolemia/possible FH (LDL-C ≥ 190 mg/dL). As part of a pre-post implementation study, a CDS alert was deployed in the electronic health record (EHR) in a large health system comprising 3 main sites, 16 hospitals and 53 clinics. Data were collected for 3 months before ('silent mode') and after ('active mode') its implementation. Clinicians were only able to view the alert in the EHR during active mode. We matched individuals 1:1 in both modes, based on age, sex, and baseline lipid lowering therapy (LLT). The primary outcome was difference in LDL-C between the two groups and the secondary outcome was initiation/intensification of LLT after alert trigger. We identified 800 matched patients in each mode (mean ± SD age 56.1 ± 11.8 y vs. 55.9 ± 11.8 y; 36.0% male in both groups; mean ± SD initial LDL-C 211.3 ± 27.4 mg/dL vs. 209.8 ± 23.9 mg/dL; 11.2% on LLT at baseline in each group). LDL-C levels were 6.6 mg/dL lower (95% CI, -10.7 to -2.5; P = 0.002) in active vs. silent mode. The odds of high-intensity statin use (OR, 1.78; 95% CI, 1.41-2.23; P < 0.001) and LLT initiation/intensification (OR, 1.30, 95% CI, 1.06-1.58, P = 0.01) were higher in active vs. silent mode. Implementation of a CDS was associated with lowering of LDL-C levels in patients with severe hypercholesterolemia/possible FH, likely due to higher rates of clinician led LLT initiation/intensification.

Electronic health record intervention to increase use of NSAIDs as analgesia for hospitalised patients: a cluster randomised controlled study

BACKGROUND

Prescribing non-opioid pain medications, such as non-steroidal anti-inflammatory (NSAIDs) medications, has been shown to reduce pain and decrease opioid use, but it is unclear how to effectively encourage multimodal pain medication prescribing for hospitalised patients. Therefore, the aim of this study is to evaluate the effect of prechecking non-opioid pain medication orders on clinician prescribing of NSAIDs among hospitalised adults.

METHODS

This was a cluster randomised controlled trial of adult (≥18 years) hospitalised patients admitted to three hospital sites under one quaternary hospital system in the USA from 2 March 2022 to 3 March 2023. A multimodal pain order panel was embedded in the admission order set, with NSAIDs prechecked in the intervention group. The intervention group could uncheck the NSAID order. The control group had access to the same NSAID order. The primary outcome was an increase in NSAID ordering. Secondary outcomes include NSAID administration, inpatient pain scores and opioid use and prescribing and relevant clinical harms including acute kidney injury, new gastrointestinal bleed and in-hospital death.

RESULTS

Overall, 1049 clinicians were randomised. The study included 6239 patients for a total of 9595 encounters. Both NSAID ordering (36 vs 43%, p<0.001) and administering (30 vs 34%, p=0.001) by the end of the first full hospital day were higher in the intervention (prechecked) group. There was no statistically significant difference in opioid outcomes during the hospitalisation and at discharge. There was a statistically but perhaps not clinically significant difference in pain scores during both the first and last full hospital day.

CONCLUSIONS

This cluster randomised controlled trial showed that prechecking an order for NSAIDs to promote multimodal pain management in the admission order set increased NSAID ordering and administration, although there were no changes to pain scores or opioid use. While prechecking orders is an important way to increase adoption, safety checks should be in place.

Information displays for automated surveillance algorithms of in-hospital patient deterioration: a scoping review

OBJECTIVE

Surveillance algorithms that predict patient decompensation are increasingly integrated with clinical workflows to help identify patients at risk of in-hospital deterioration. This scoping review aimed to identify the design features of the information displays, the types of algorithm that drive the display, and the effect of these displays on process and patient outcomes.

MATERIALS AND METHODS

The scoping review followed Arksey and O'Malley's framework. Five databases were searched with dates between January 1, 2009 and January 26, 2022. Inclusion criteria were: participants-clinicians in inpatient settings; concepts-intervention as deterioration information displays that leveraged automated AI algorithms; comparison as usual care or alternative displays; outcomes as clinical, workflow process, and usability outcomes; and context as simulated or real-world in-hospital settings in any country. Screening, full-text review, and data extraction were reviewed independently by 2 researchers in each step. Display categories were identified inductively through consensus.

RESULTS

Of 14 575 articles, 64 were included in the review, describing 61 unique displays. Forty-one displays were designed for specific deteriorations (eg, sepsis), 24 provided simple alerts (ie, text-based prompts without relevant patient data), 48 leveraged well-accepted score-based algorithms, and 47 included nurses as the target users. Only 1 out of the 10 randomized controlled trials reported a significant effect on the primary outcome.

CONCLUSIONS

Despite significant advancements in surveillance algorithms, most information displays continue to leverage well-understood, well-accepted score-based algorithms. Users' trust, algorithmic transparency, and workflow integration are significant hurdles to adopting new algorithms into effective decision support tools.

Evaluation of an adaptive, rule-based dosing algorithm to maintain therapeutic anticoagulation during atrial fibrillation ablation

Background

Cerebral thromboembolism during atrial fibrillation (AF) ablation is an infrequent (0.17%) complication in part owing to strict adherence to intraprocedural anticoagulation. Failure to maintain therapeutic anticoagulation can lead to an increase in events, including silent cerebral ischemia.

Objective

To evaluate a computerized, clinical decision support system (CDSS) to dose intraprocedural anticoagulation and determine if it leads to improved intraprocedural anticoagulation outcomes during AF ablation.

Methods

The Digital Intern dosing algorithm is an adaptive, rule-based CDSS for heparin dosing. The initial dose is calculated from the patient's weight, baseline activated clotting time (ACT), and outpatient anticoagulant. Subsequent recommendations adapt based on individual patient ACT changes. Outcomes from 50 cases prior to algorithm introduction were compared to 139 cases using the algorithm.

Results

Procedures using the dosing algorithm reached goal ACT (over 300 seconds) faster (17.6 ± 11.1 minutes vs 33.3 ± 23.6 minutes pre-algorithm, P < .001). ACTs fell below goal while in the LA (odds ratio 0.20 [0.10-0.39], P < .001) and rose above 400 seconds less frequently (odds ratio 0.21 [0.07-0.59], P = .003). System Usability Scale scores were excellent (96 ± 5, n = 7, score >80.3 excellent). Preprocedure anticoagulant, weight, baseline ACT, age, sex, and renal function were potential predictors of heparin dose to achieve ACT >300 seconds and final infusion rate.

Conclusion

A heparin dosing CDSS based on rules and adaptation to individual patient response improved maintenance of therapeutic ACT during AF ablation and was rated highly by nurses for usability.

Examining primary care provider experiences with using a clinical decision support tool for pain management

Objective

To evaluate primary care provider (PCP) experiences using a clinical decision support (CDS) tool over 16 months following a user-centered design process and implementation.

Materials and Methods

We conducted a qualitative evaluation of the Chronic Pain OneSheet (OneSheet), a chronic pain CDS tool. OneSheet provides pain- and opioid-related risks, benefits, and treatment information for patients with chronic pain to PCPs. Using the 5 Rights of CDS framework, we conducted and analyzed semi-structured interviews with 19 PCPs across 2 academic health systems.

Results

PCPs stated that OneSheet mostly contained the right information required to treat patients with chronic pain and was correctly located in the electronic health record. PCPs used OneSheet for distinct subgroups of patients with chronic pain, including patients prescribed opioids, with poorly controlled pain, or new to a provider or clinic. PCPs reported variable workflow integration and selective use of certain OneSheet features driven by their preferences and patient population. PCPs recommended broadening OneSheet access to clinical staff and patients for data entry to address clinician time constraints.

Discussion

Differences in patient subpopulations and workflow preferences had an outsized effect on CDS tool use even when the CDS contained the right information identified in a user-centered design process.

Conclusions

To increase adoption and use, CDS design and implementation processes may benefit from increased tailoring that accommodates variation and dynamics among patients, visits, and providers.

A behavioral intervention to promote use of multimodal pain medication for hospitalized patients: A randomized controlled trial

BACKGROUND

The use of nonsteroidal anti-inflammatory drugs (NSAIDs) can reduce pain and has become a core strategy to decrease opioid use, but there is a lack of data to describe encouraging use when admitting patients using electronic health record systems.

OBJECTIVE

Assess an electronic health record system to increase ordering of NSAIDs for hospitalized adults.

DESIGNS, SETTINGS AND PARTICIPANTS

We performed a cluster randomized controlled trial of clinicians admitting adult patients to a health system over a 9-month period. Clinicians were randomized to use a standard admission order set.

INTERVENTION

Clinicians in the intervention arm were required to actively order or decline NSAIDs; the control arm was shown the same order but without a required response.

MAIN OUTCOME AND MEASURES

The primary outcome was NSAIDs ordered and administered by the first full hospital day. Secondary outcomes included pain scores and opioid prescribing.

RESULTS

A total of 20,085 hospitalizations were included. Among these hospitalizations, patients had a mean age of 58 years, and a Charlson comorbidity score of 2.97, while 50% and 56% were female and White, respectively. Overall, 52% were admitted by a clinician randomized to the intervention arm. NSAIDs were ordered in 2267 (22%) interventions and 2093 (22%) control admissions (p = .10). Similarly, there were no statistical differences in NSAID administration, pain scores, or opioid prescribing. Average pain scores (0-5 scale) were 3.36 in the control group and 3.39 in the intervention group (p = .46). There were no differences in clinical harms.

CONCLUSIONS AND RELEVANCE

Requiring an active decision to order an NSAID at admission had no demonstrable impact on NSAID ordering. Multicomponent interventions, perhaps with stronger decision support, may be necessary to encourage NSAID ordering.

The effects of computerised decision support systems on nursing and allied health professional performance and patient outcomes: a systematic review and user contextualisation

Background

Computerised decision support systems (CDSS) are widely used by nurses and allied health professionals but their effect on clinical performance and patient outcomes is uncertain.

Objectives

Evaluate the effects of clinical decision support systems use on nurses', midwives' and allied health professionals' performance and patient outcomes and sense-check the results with developers and users.

Eligibility criteria

Comparative studies (randomised controlled trials (RCTs), non-randomised trials, controlled before-and-after (CBA) studies, interrupted time series (ITS) and repeated measures studies comparing) of CDSS versus usual care from nurses, midwives or other allied health professionals.

Information sources

Nineteen bibliographic databases searched October 2019 and February 2021.

Risk of bias

Assessed using structured risk of bias guidelines; almost all included studies were at high risk of bias.

Synthesis of results

Heterogeneity between interventions and outcomes necessitated narrative synthesis and grouping by: similarity in focus or CDSS-type, targeted health professionals, patient group, outcomes reported and study design.

Included studies

Of 36,106 initial records, 262 studies were assessed for eligibility, with 35 included: 28 RCTs (80%), 3 CBA studies (8.6%), 3 ITS (8.6%) and 1 non-randomised trial, a total of 1318 health professionals and 67,595 patient participants. Few studies were multi-site and most focused on decision-making by nurses (71%) or paramedics (5.7%). Standalone, computer-based CDSS featured in 88.7% of the studies; only 8.6% of the studies involved 'smart' mobile or handheld technology. Care processes - including adherence to guidance - were positively influenced in 47% of the measures adopted. For example, nurses' adherence to hand disinfection guidance, insulin dosing, on-time blood sampling, and documenting care were improved if they used CDSS. Patient care outcomes were statistically - if not always clinically - significantly improved in 40.7% of indicators. For example, lower numbers of falls and pressure ulcers, better glycaemic control, screening of malnutrition and obesity, and accurate triaging were features of professionals using CDSS compared to those who were not.

Evidence limitations

Allied health professionals (AHPs) were underrepresented compared to nurses; systems, studies and outcomes were heterogeneous, preventing statistical aggregation; very wide confidence intervals around effects meant clinical significance was questionable; decision and implementation theory that would have helped interpret effects - including null effects - was largely absent; economic data were scant and diverse, preventing estimation of overall cost-effectiveness.

Interpretation

CDSS can positively influence selected aspects of nurses', midwives' and AHPs' performance and care outcomes. Comparative research is generally of low quality and outcomes wide ranging and heterogeneous. After more than a decade of synthesised research into CDSS in healthcare professions other than medicine, the effect on processes and outcomes remains uncertain. Higher-quality, theoretically informed, evaluative research that addresses the economics of CDSS development and implementation is still required.

Future work

Developing nursing CDSS and primary research evaluation.

Funding

This project was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme and will be published in Health and Social Care Delivery Research; 2023. See the NIHR Journals Library website for further project information.

Registration

PROSPERO [number: CRD42019147773].

Where Do Real-Time Prescription Benefit Tools Fit in the Landscape of High US Prescription Medication Costs? A Narrative Review

The problem of unaffordable prescription medications in the United States is complex and can result in poor patient adherence to therapy, worse clinical outcomes, and high costs to the healthcare system. While providers are aware of the financial burden of healthcare for patients, there is a lack of actionable price transparency at the point of prescribing. Real-time prescription benefit (RTPB) tools are new electronic clinical decision support tools that retrieve patient- and medication-specific out-of-pocket cost information and display it to clinicians at the point of prescribing. The rise in US healthcare costs has been a major driver for efforts to increase medication price transparency, and mandates from the Centers for Medicare & Medicaid Services for Medicare Part D sponsors to adopt RTPB tools may spur integration of such tools into electronic health records. Although multiple factors affect the implementation of RTPB tools, there is limited evidence on outcomes. Further research will be needed to understand the impact of RTPB tools on end results such as prescribing behavior, out-of-pocket medication costs for patients, and adherence to pharmacologic treatment. We review the terminology and concepts essential in understanding the landscape of RTPB tools, implementation considerations, barriers to adoption, and directions for future research that will be important to patients, prescribers, health systems, and insurers.

Operating Room Performance Optimization Metrics: a Systematic Review

Literature proposes numerous initiatives for optimization of the Operating Room (OR). Despite multiple suggested strategies for the optimization of workflow on the OR, its patients and (medical) staff, no uniform description of 'optimization' has been adopted. This makes it difficult to evaluate the proposed optimization strategies. In particular, the metrics used to quantify OR performance are diverse so that assessing the impact of suggested approaches is complex or even impossible. To secure a higher implementation success rate of optimisation strategies in practice we believe OR optimisation and its quantification should be further investigated. We aim to provide an inventory of the metrics and methods used to optimise the OR by the means of a structured literature study. We observe that several aspects of OR performance are unaddressed in literature, and no studies account for possible interactions between metrics of quality and efficiency. We conclude that a systems approach is needed to align metrics across different elements of OR performance, and that the wellbeing of healthcare professionals is underrepresented in current optimisation approaches.

Perspective: leveraging the electronic health record to improve sleep in the hospital

Inpatient sleep loss can worsen health outcomes, including delirium and falls. Sleep disruptions in the hospital often originate from provider-patient interactions ordered electronically through computerized provider order entry. These orders contain clinical decision support systems with default schedules. These defaults are often around-the-clock, may not align with patients' needs, and cause iatrogenic sleep loss. Optimizing clinical decision support in the electronic health record can decrease unnecessary sleep disruptions and influence sleep-friendly decision-making.

CITATION

Kadura S, Siala T, Arora VM. Perspective: Leveraging the electronic health record to improve sleep in the hospital. J Clin Sleep Med. 2023;19(2):421-423.

Patient-Facing Clinical Decision Support for High Blood Pressure Control: Patient Survey

BACKGROUND

High blood pressure (HBP) affects nearly half of adults in the United States and is a major factor in heart attacks, strokes, kidney disease, and other morbidities. To reduce risk, guidelines for HBP contain more than 70 recommendations, including many related to patient behaviors, such as home monitoring and lifestyle changes. Thus, the patient's role in controlling HBP is crucial. Patient-facing clinical decision support (CDS) tools may help patients adhere to evidence-based care, but customization is required.

OBJECTIVE

Our objective was to understand how to adapt CDS to best engage patients in controlling HBP.

METHODS

We conducted a mixed methods study with two phases: (1) survey-guided interviews with a limited cohort and (2) a nationwide web-based survey. Participation in each phase was limited to adults aged between 18 and 85 years who had been diagnosed with hypertension. The survey included general questions that assessed goal setting, treatment priorities, medication load, comorbid conditions, satisfaction with blood pressure (BP) management, and attitudes toward CDS, and also a series of questions regarding A/B preferences using paired information displays to assess perceived trustworthiness of potential CDS user interface options.

RESULTS

We conducted 17 survey-guided interviews to gather patient needs from CDS, then analyzed results and created a second survey of 519 adults with clinically diagnosed HBP. A large majority of participants reported that BP control was a high priority (83%), had monitored BP at home (82%), and felt comfortable using technology (88%). Survey respondents found displays with more detailed recommendations more trustworthy (56%-77% of them preferred simpler displays), especially when incorporating social trust and priorities from providers and patients like them, but had no differences in action taken.

CONCLUSIONS

Respondents to the survey felt that CDS capabilities could help them with HBP control. The more detailed design options for BP display and recommendations messaging were considered the most trustworthy yet did not differentiate perceived actions.

Barriers and enablers to implementing and using clinical decision support systems for chronic diseases: a qualitative systematic review and meta-aggregation

Background

Clinical decision support (CDS) is increasingly used to facilitate chronic disease care. Despite increased availability of electronic health records and the ongoing development of new CDS technologies, uptake of CDS into routine clinical settings is inconsistent. This qualitative systematic review seeks to synthesise healthcare provider experiences of CDS—exploring the barriers and enablers to implementing, using, evaluating, and sustaining chronic disease CDS systems.

Methods

A search was conducted in Medline, CINAHL, APA PsychInfo, EconLit, and Web of Science from 2011 to 2021. Primary research studies incorporating qualitative findings were included if they targeted healthcare providers and studied a relevant chronic disease CDS intervention. Relevant CDS interventions were electronic health record-based and addressed one or more of the following chronic diseases: cardiovascular disease, diabetes, chronic kidney disease, hypertension, and hypercholesterolaemia. Qualitative findings were synthesised using a meta-aggregative approach.

Results

Thirty-three primary research articles were included in this qualitative systematic review. Meta-aggregation of qualitative data revealed 177 findings and 29 categories, which were aggregated into 8 synthesised findings. The synthesised findings related to clinical context, user, external context, and technical factors affecting CDS uptake. Key barriers to uptake included CDS systems that were simplistic, had limited clinical applicability in multimorbidity, and integrated poorly into existing workflows. Enablers to successful CDS interventions included perceived usefulness in providing relevant clinical knowledge and structured chronic disease care; user confidence gained through training and post training follow-up; external contexts comprised of strong clinical champions, allocated personnel, and technical support; and CDS technical features that are both highly functional, and attractive.

Conclusion

This systematic review explored healthcare provider experiences, focussing on barriers and enablers to CDS use for chronic diseases. The results provide an evidence-base for designing, implementing, and sustaining future CDS systems. Based on the findings from this review, we highlight actionable steps for practice and future research.

Trial registration

PROSPERO CRD42020203716

Supplementary Information

The online version contains supplementary material available at 10.1186/s43058-022-00326-x.

Clinical Decision Support in the PICU: Implications for Design and Evaluation

OBJECTIVES

To assess the current landscape of clinical decision support (CDS) tools in PICUs in order to identify priority areas of focus in this field.

DESIGN

International, quantitative, cross-sectional survey.

SETTING

Role-specific, web-based survey administered in November and December 2020.

SUBJECTS

Medical directors, bedside nurses, attending physicians, and residents/advanced practice providers at Pediatric Acute Lung Injury and Sepsis Network-affiliated PICUs.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

The survey was completed by 109 respondents from 45 institutions, primarily attending physicians from university-affiliated PICUs in the United States. The most commonly used CDS tools were people-based resources (93% used always or most of the time) and laboratory result highlighting (86%), with order sets, order-based alerts, and other electronic CDS tools also used frequently. The most important goal providers endorsed for CDS tools were a proven impact on patient safety and an evidence base for their use. Negative perceptions of CDS included concerns about diminished critical thinking and the burden of intrusive processes on providers. Routine assessment of existing CDS was rare, with infrequent reported use of observation to assess CDS impact on workflows or measures of individual alert burden.

CONCLUSIONS

Although providers share some consensus over CDS utility, we identified specific priority areas of research focus. Consensus across practitioners exists around the importance of evidence-based CDS tools having a proven impact on patient safety. Despite broad presence of CDS tools in PICUs, practitioners continue to view them as intrusive and with concern for diminished critical thinking. Deimplementing ineffective CDS may mitigate this burden, though postimplementation evaluation of CDS is rare.

Design, effectiveness, and economic outcomes of contemporary chronic disease clinical decision support systems: a systematic review and meta-analysis

Objectives

Electronic health record-based clinical decision support (CDS) has the potential to improve health outcomes. This systematic review investigates the design, effectiveness, and economic outcomes of CDS targeting several common chronic diseases.

Material and Methods

We conducted a search in PubMed (Medline), EBSCOHOST (CINAHL, APA PsychInfo, EconLit), and Web of Science. We limited the search to studies from 2011 to 2021. Studies were included if the CDS was electronic health record-based and targeted one or more of the following chronic diseases: cardiovascular disease, diabetes, chronic kidney disease, hypertension, and hypercholesterolemia. Studies with effectiveness or economic outcomes were considered for inclusion, and a meta-analysis was conducted.

Results

The review included 76 studies with effectiveness outcomes and 9 with economic outcomes. Of the effectiveness studies, 63% described a positive outcome that favored the CDS intervention group. However, meta-analysis demonstrated that effect sizes were heterogenous and small, with limited clinical and statistical significance. Of the economic studies, most full economic evaluations (n = 5) used a modeled analysis approach. Cost-effectiveness of CDS varied widely between studies, with an estimated incremental cost-effectiveness ratio ranging between USD$2192 to USD$151 955 per QALY.

Conclusion

We summarize contemporary chronic disease CDS designs and evaluation results. The effectiveness and cost-effectiveness results for CDS interventions are highly heterogeneous, likely due to differences in implementation context and evaluation methodology. Improved quality of reporting, particularly from modeled economic evaluations, would assist decision makers to better interpret and utilize results from these primary research studies.

Registration

PROSPERO (CRD42020203716)

Clinical decision support in cardiovascular medicine

Despite considerable progress in tackling cardiovascular disease over the past 50 years, many gaps in the quality of care for cardiovascular disease remain. Multiple missed opportunities have been identified at every step in the prevention and treatment of cardiovascular disease, such as failure to make risk factor modifications, failure to diagnose cardiovascular disease, and failure to use proper evidence based treatments. With the digital transformation of medicine and advances in health information technology, clinical decision support (CDS) tools offer promise to enhance the efficiency and effectiveness of delivery of cardiovascular care. However, to date, the promise of CDS delivering scalable and sustained value for patient care in clinical practice has not been realized. This article reviews the evidence on key emerging questions around the development, implementation, and regulation of CDS with a focus on cardiovascular disease. It first reviews evidence on the effectiveness of CDS on healthcare process and clinical outcomes related to cardiovascular disease and design features associated with CDS effectiveness. It then reviews the barriers encountered during implementation of CDS in cardiovascular care, with a focus on unintended consequences and strategies to promote successful implementation. Finally, it reviews the legal and regulatory environment of CDS with specific examples for cardiovascular disease.

Clinical Decision Support for Fall Prevention: Defining End-User Needs

BACKGROUND AND SIGNIFICANCE

Falls in community-dwelling older adults are common, and there is a lack of clinical decision support (CDS) to provide health care providers with effective, individualized fall prevention recommendations.

OBJECTIVES

The goal of this research is to identify end-user (primary care staff and patients) needs through a human-centered design process for a tool that will generate CDS to protect older adults from falls and injuries.

METHODS

Primary care staff (primary care providers, care coordinator nurses, licensed practical nurses, and medical assistants) and community-dwelling patients aged 60 years or older associated with Brigham & Women's Hospital-affiliated primary care clinics and the University of Florida Health Archer Family Health Care primary care clinic were eligible to participate in this study. Through semi-structured and exploratory interviews with participants, our team identified end-user needs through content analysis.

RESULTS

User needs for primary care staff (n = 24) and patients (n = 18) were categorized under the following themes: workload burden; systematic communication; in-person assessment of patient condition; personal support networks; motivational tools; patient understanding of fall risk; individualized resources; and evidence-based safe exercises and expert guidance. While some of these themes are specific to either primary care staff or patients, several address needs expressed by both groups of end-users.

CONCLUSION

Our findings suggest that there are many care gaps in fall prevention management in primary care and that personalized, actionable, and evidence-based CDS has the potential to address some of these gaps.

Evaluation in Life Cycle of Information Technology (ELICIT) framework: Supporting the innovation life cycle from business case assessment to summative evaluation

OBJECTIVE

Our objective was to develop an evaluation framework for electronic health record (EHR)-integrated innovations to support evaluation activities at each of four information technology (IT) life cycle phases: planning, development, implementation, and operation.

METHODS

The evaluation framework was developed based on a review of existing evaluation frameworks from health informatics and other domains (human factors engineering, software engineering, and social sciences); expert consensus; and real-world testing in multiple EHR-integrated innovation studies.

RESULTS

The resulting Evaluation in Life Cycle of IT (ELICIT) framework covers four IT life cycle phases and three measure levels (society, user, and IT). The ELICIT framework recommends 12 evaluation steps: (1) business case assessment; (2) stakeholder requirements gathering; (3) technical requirements gathering; (4) technical acceptability assessment; (5) user acceptability assessment; (6) social acceptability assessment; (7) social implementation assessment; (8) initial user satisfaction assessment; (9) technical implementation assessment; (10) technical portability assessment; (11) long-term user satisfaction assessment; and (12) social outcomes assessment.

DISCUSSION

Effective evaluation requires a shared understanding and collaboration across disciplines throughout the entire IT life cycle. In contrast with previous evaluation frameworks, the ELICIT framework focuses on all phases of the IT life cycle across the society, user, and IT levels. Institutions seeking to establish evaluation programs for EHR-integrated innovations could use our framework to create such shared understanding and justify the need to invest in evaluation.

CONCLUSION

As health care undergoes a digital transformation, it will be critical for EHR-integrated innovations to be systematically evaluated. The ELICIT framework can facilitate these evaluations.

A randomized controlled trial for evaluating the impact of integrating a computerized clinical decision support system and a socially assistive humanoid robot into grand rounds during pre/post-operative care

Although clinical decision support systems (CDSSs) are increasingly emphasized as one of the possible levers for improving care, they are still not widely used due to different barriers, such as doubts about systems’ performance, their complexity and poor design, practitioners’ lack of time to use them, poor computer skills, reluctance to use them in front of patients, and deficient integration into existing workflows. While several studies on CDSS exist, there is a need for additional high-quality studies using large samples and examining the differences between outcomes following a decision based on CDSS support and those following decisions without this kind of information. Even less is known about the effectiveness of a CDSS that is delivered during a grand round routine and with the help of socially assistive humanoid robots (SAHRs). In this study, 200 patients will be randomized into a Control Group (i.e. standard care) and an Intervention Group (i.e. standard care and novel CDSS delivered via a SAHR). Health care quality and Quality of Life measures will be compared between the two groups. Additionally, approximately 22 clinicians, who are also active researchers at the University Clinical Center Maribor, will evaluate the acceptability and clinical usability of the system. The results of the proposed study will provide high-quality evidence on the effectiveness of CDSS systems and SAHR in the grand round routine.

Internal Validation of a Risk Scoring System for Venous Thromboembolism After Total hip or Knee Arthroplasty

We developed a computerized clinical decision support system (CCDSS) for venous thromboembolism (VTE) risk assessment. We aimed to demonstrate its relevance and evaluate associations between risk level and VTE incidence in patients undergoing total hip/knee arthroplasty. In this case-control study, VTE was confirmed using ultrasonography/computed tomography angiography in 1098 adults at a tertiary care hospital over five years (2013-2018). Postoperative VTE incidence was classified into three risk levels (moderate, high, and highest). The overall VTE incidence was 11.7%, which increased with a risk level of 0%, 5.8%, and 12.8% in moderate-risk, high-risk, and highest-risk patients, respectively. Highest-risk patients were significantly more likely to develop VTE than high-risk patients (odds ratio [OR] 2.4; 95% confidence interval [CI] 1.2-5.5; p = 0.01). VTE development was more likely in patients with risk scores ≥4 relative to those with risk scores of 2-3 (OR 1.8; 95% CI 1.2-2.7; p = 0.003) and -1 to 1 (OR 3.3; 95% CI 1.6-7.7; p < 0.001). This study indicates that risk level and VTE incidence are associated; our scoring system appears useful for patients undergoing total hip/knee arthroplasty.

Systematic review of prediction models for postacute care destination decision-making

OBJECTIVE

This article reports a systematic review of studies containing development and validation of models predicting postacute care destination after adult inpatient hospitalization, summarizes clinical populations and variables, evaluates model performance, assesses risk of bias and applicability, and makes recommendations to reduce bias in future models.

MATERIALS AND METHODS

A systematic literature review was conducted following PRISMA guidelines and the Cochrane Prognosis Methods Group criteria. Online databases were searched in June 2020 to identify all published studies in this area. Data were extracted based on the CHARMS checklist, and studies were evaluated based on predictor variables, validation, performance in validation, risk of bias, and applicability using the Prediction Model Risk of Bias Assessment Tool (PROBAST) tool.

RESULTS

The final sample contained 28 articles with 35 models for evaluation. Models focused on surgical (22), medical (5), or both (8) populations. Eighteen models were internally validated, 10 were externally validated, and 7 models underwent both types. Model performance varied within and across populations. Most models used retrospective data, the median number of predictors was 8.5, and most models demonstrated risk of bias.

DISCUSSION AND CONCLUSION

Prediction modeling studies for postacute care destinations are becoming more prolific in the literature, but model development and validation strategies are inconsistent, and performance is variable. Most models are developed using regression, but machine learning methods are increasing in frequency. Future studies should ensure the rigorous variable selection and follow TRIPOD guidelines. Only 14% of the models have been tested or implemented beyond original studies, so translation into practice requires further investigation.

Multicomponent intervention to improve blood pressure management in chronic kidney disease: a protocol for a pragmatic clinical trial

INTRODUCTION

The purpose of this study is to incorporate behavioural economic principles and user-centred design principles into a multicomponent intervention for the management of uncontrolled hypertension (HTN) in chronic kidney disease (CKD) in primary care.

METHODS AND ANALYSIS

This is a multicentre, pragmatic, controlled trial cluster-randomised at the clinician level at The Brigham and Women's Practice -Based Research Network of 15 practices. Of 220 total clinicians, 184 were eligible to be enrolled, and the remainder were excluded (residents and clinicians who see urgent care or walk-in patients); no clinicians opted out. The intervention consists of a clinical decision support system based in behavioural economic and user-centred design principles that will: (1) synthesise existing laboratory tests, medication orders and vital sign data; (2) increase recognition of CKD, (3) increase recognition of uncontrolled HTN in CKD patients and (4) deliver evidence-based CKD and HTN management recommendations. The primary endpoint is the change in mean systolic blood pressure between baseline and 6 months compared across arms. We will use the Reach Effectiveness Adoption Implementation Maintenance framework. At the conclusion of this study, we will have: (1) validated an intervention that combines laboratory tests, medication records and clinical information collected by electronic health records to recognise uncontrolled HTN in CKD patients and recommend a course of care, (2) tested the effectiveness of said intervention and (3) collected information about the implementation of the intervention that will aid in dissemination of the intervention to other practice settings.

ETHICS AND DISSEMINATION

The Human Subjects Institutional Review Board at Brigham and Women's Hospital provided an expedited review and approval for this study protocol, and a Data Safety Monitoring Board will ensure the ongoing safety of the trial.

TRIAL REGISTRATION NUMBER

NCT03679247.

Effects of computerised clinical decision support systems (CDSS) on nursing and allied health professional performance and patient outcomes: a systematic review of experimental and observational studies

OBJECTIVE

Computerised clinical decision support systems (CDSS) are an increasingly important part of nurse and allied health professional (AHP) roles in delivering healthcare. The impact of these technologies on these health professionals' performance and patient outcomes has not been systematically reviewed. We aimed to conduct a systematic review to investigate this.

MATERIALS AND METHODS

The following bibliographic databases and grey literature sources were searched by an experienced Information Professional for published and unpublished research from inception to February 2021 without language restrictions: MEDLINE (Ovid), Embase Classic+Embase (Ovid), PsycINFO (Ovid), HMIC (Ovid), AMED (Allied and Complementary Medicine) (Ovid), CINAHL (EBSCO), Cochrane Central Register of Controlled Trials (Wiley), Cochrane Database of Systematic Reviews (Wiley), Social Sciences Citation Index Expanded (Clarivate), ProQuest Dissertations & Theses Abstracts & Index, ProQuest ASSIA (Applied Social Science Index and Abstract), Clinical Trials.gov, WHO International Clinical Trials Registry (ICTRP), Health Services Research Projects in Progress (HSRProj), OpenClinical(www.OpenClinical.org), OpenGrey (www.opengrey.eu), Health.IT.gov, Agency for Healthcare Research and Quality (www.ahrq.gov). Any comparative research studies comparing CDSS with usual care were eligible for inclusion.

RESULTS

A total of 36 106 non-duplicate records were identified. Of 35 included studies: 28 were randomised trials, three controlled-before-and-after studies, three interrupted-time-series and one non-randomised trial. There were ~1318 health professionals and ~67 595 patient participants in the studies. Most studies focused on nurse decision-makers (71%) or paramedics (5.7%). CDSS as a standalone Personal Computer/LAPTOP-technology was a feature of 88.7% of the studies; only 8.6% of the studies involved 'smart' mobile/handheld-technology.

DISCUSSION

CDSS impacted 38% of the outcome measures used positively. Care processes were better in 47% of the measures adopted; examples included, nurses' adherence to hand disinfection guidance, insulin dosing, on-time blood sampling and documenting care. Patient care outcomes in 40.7% of indicators were better; examples included, lower numbers of falls and pressure ulcers, better glycaemic control, screening of malnutrition and obesity and triaging appropriateness.

CONCLUSION

CDSS may have a positive impact on selected aspects of nurses' and AHPs' performance and care outcomes. However, comparative research is generally low quality, with a wide range of heterogeneous outcomes. After more than 13 years of synthesised research into CDSS in healthcare professions other than medicine, the need for better quality evaluative research remains as pressing.

Implementation of App-Based Diabetes Medication Management: Outpatient and Perioperative Clinical Decision Support

PURPOSE OF REVIEW

Outpatient and perioperative therapeutic decision making for patients with diabetes involves increasingly complex medical-decision making due to rapid advances in knowledge and treatment modalities. We sought to review mobile decision support tools available to clinicians for this essential and increasingly difficult task, and to highlight the development and implementation of novel mobile applications for these purposes.

RECENT FINDINGS

We found 211 mobile applications related to diabetes from the search, but only five were found to provide clinical decision support for outpatient diabetes management and none for perioperative decision support. We found a dearth of tools for clinicians to navigate these tasks. We highlight key aspects for effective development of future diabetes decision support. These include just-in-time availability, respect for the five rights of clinical decision support, and integration with clinical workflows including the electronic medical record.

Establishing a multidisciplinary initiative for interoperable electronic health record innovations at an academic medical center

Objective

To establish an enterprise initiative for improving health and health care through interoperable electronic health record (EHR) innovations.

Materials and Methods

We developed a unifying mission and vision, established multidisciplinary governance, and formulated a strategic plan. Key elements of our strategy include establishing a world-class team; creating shared infrastructure to support individual innovations; developing and implementing innovations with high anticipated impact and a clear path to adoption; incorporating best practices such as the use of Fast Healthcare Interoperability Resources (FHIR) and related interoperability standards; and maximizing synergies across research and operations and with partner organizations.

Results

University of Utah Health launched the ReImagine EHR initiative in 2016. Supportive infrastructure developed by the initiative include various FHIR-related tooling and a systematic evaluation framework. More than 10 EHR-integrated digital innovations have been implemented to support preventive care, shared decision-making, chronic disease management, and acute clinical care. Initial evaluations of these innovations have demonstrated positive impact on user satisfaction, provider efficiency, and compliance with evidence-based guidelines. Return on investment has included improvements in care; over $35 million in external grant funding; commercial opportunities; and increased ability to adapt to a changing healthcare landscape.

Discussion

Key lessons learned include the value of investing in digital innovation initiatives leveraging FHIR; the importance of supportive infrastructure for accelerating innovation; and the critical role of user-centered design, implementation science, and evaluation.

Conclusion

EHR-integrated digital innovation initiatives can be key assets for enhancing the EHR user experience, improving patient care, and reducing provider burnout.

Effect of a smartphone-based intervention on secondary prevention medication prescriptions after coronary artery bypass graft surgery: The MISSION-1 randomized controlled trial

BACKGROUND

Studies found that patients who underwent coronary artery bypass grafting (CABG) often fail to receive optimal evidence-based secondary prevention medications. We evaluated the effectiveness of a smartphone-based quality improvement effort on improving the prescription of medical therapies.

METHODS

In this cluster-randomized controlled trial, 60 hospitals were randomized to a control arm (n = 30) or to an intervention arm using smartphone-based multifaceted quality improvement interventions (n = 30). The primary outcome was the prescription of statin. The secondary outcomes were prescription of beta-blocker, angiotensin-converting enzyme inhibitor, or angiotensin receptor blocker (ACE inhibitor or ARB), and optimal medical therapy for eligible patients.

RESULTS

Between June 1, 2015 and September 15, 2016, a total of 10,006 CABG patients were enrolled (5,653 in 26 intervention and 4,353 in 29 control hospitals, 5 hospitals withdrew). Statin prescribing rate was 87.8% in the intervention arm and 84.4% in the control arm. We saw no evidence of an effect of intervention on statin prescribing in the intention-to-treat analysis (odds ratio [OR], 1.31; 95% confidence interval (CI), 0.68-2.54; P = .43) or in key patient subsets. The prescription rates of ACE inhibitor or ARB and optimal medical therapy were comparable between study groups, while beta-blocker was more often prescribed in the intervention arm. Post hoc analysis demonstrated a greater increase in statin prescribing rate over time in the intervention arm.

CONCLUSIONS

A smartphone-based quality improvement intervention compared with usual care did not increase statin prescribing for patients who received CABG. New studies focusing on the best practice of this technique may be warranted.

Developing an Ophthalmology Clinical Decision Support System to Identify Patients for Low Vision Rehabilitation

Purpose

The purpose of this study was to develop and evaluate an electronic health record (EHR) clinical decision support system to identify patients meeting criteria for low vision rehabilitation (LVR) referral.

Methods

In this quality improvement project, we applied a user-centered design approach to develop an interactive electronic alert for LVR referral within the Johns Hopkins Wilmer Eye Institute. We invited 15 ophthalmology physicians from 8 subspecialties to participate in the design and implementation, and to provide user experience feedback. The three project phases incorporated development evaluation, feedback analysis, and system refinement. We report on the final alert design, firing accuracy, and user experiences.

Results

The alert was designed as physician-centered and patient-specific. Alert firing relied on visual acuity and International Classification of Diseases (ICD)-10 diagnosis (hemianopia/quadrantanopia) criteria. The alert suppression considerations included age < 5 years, recent surgeries, prior LVR visit, and related alert actions. False positive rate (firing when alert should have been suppressed or when firing criteria not met) was 0.2%. The overall false negative rate (alert not firing when visual acuity or encounter diagnosis criteria met) was 5.6%. Of the 13 physicians who completed the survey, 8 agreed that the alert is easy to use, and 12 would consider ongoing usage.

Conclusions

This EHR-based clinical decision support system shows reliable firing metrics in identifying patients with vision impairment and promising acceptance by ophthalmologist users to facilitate care and LVR referral.

Translational Relevance

The use of real-time data offers an opportunity to translate ophthalmic guidelines and best practices into systematic action for clinical care and research purposes across subspecialties.

Impact of a C. difficile infection (CDI) reduction bundle and its components on CDI diagnosis and prevention

BACKGROUND

Published bundles to reduce Clostridioides difficile Infection (CDI) frequently lack information on compliance with individual elements. We piloted a computerized clinical decision support-based intervention bundle and conducted detailed evaluation of several intervention-related measures.

METHODS

A quasi-experimental study of a bundled intervention was performed at 2 acute care community hospitals in Maryland. The bundle had five components: (1) timely placement in enteric precautions, (2) appropriate CDI testing, (3) reducing proton-pump inhibitor (PPI) use, (4) reducing high-CDI risk antibiotic use, and (5) optimizing use of a sporicidal agent for environmental cleaning. Chi-square and Kruskal-Wallis tests were used to compare measure differences. An interrupted time series analysis was used to evaluate impact on hospital-onset (HO)-CDI.

RESULTS

Placement of CDI suspects in enteric precautions before test results did not change. Only hospital B decreased the frequency of CDI testing and reduced inappropriate testing related to laxative use. Both hospitals reduced the use of PPI and high-risk antibiotics. A 75% decrease in HO-CDI immediately postimplementation was observed for hospital B only.

CONCLUSION

A CDI reduction bundle showed variable impact on relevant measures. Hospital-specific differential uptake of bundle elements may explain differences in effectiveness, and emphasizes the importance of measuring processes and intermediate outcomes.

Special Commentary: Using Clinical Decision Support Systems to Bring Predictive Models to the Glaucoma Clinic

Advances in the field of predictive modeling using artificial intelligence and machine learning have the potential to improve clinical care and outcomes, but only if the results of these models are presented appropriately to clinicians at the time they make decisions for individual patients. Clinical decision support (CDS) systems could be used to accomplish this. Modern CDS systems are computer-based tools designed to improve clinician decision making for individual patients. However, not all CDS systems are effective. Four principles that have been shown in other medical fields to be important for successful CDS system implementation are (1) integration into clinician workflow, (2) user-centered interface design, (3) evaluation of CDS systems and rules, and (4) standards-based development so the tools can be deployed across health systems.