An electronic chart prompt to decrease proprietary antibiotic prescription to self-pay patients

The role of information systems in emergency department decision-making-a literature review

OBJECTIVES

Healthcare providers employ heuristic and analytical decision-making to navigate the high-stakes environment of the emergency department (ED). Despite the increasing integration of information systems (ISs), research on their efficacy is conflicting. Drawing on related fields, we investigate how timing and mode of delivery influence IS effectiveness. Our objective is to reconcile previous contradictory findings, shedding light on optimal IS design in the ED.

MATERIALS AND METHODS

We conducted a systematic review following PRISMA across PubMed, Scopus, and Web of Science. We coded the ISs' timing as heuristic or analytical, their mode of delivery as active for automatic alerts and passive when requiring user-initiated information retrieval, and their effect on process, economic, and clinical outcomes.

RESULTS

Our analysis included 83 studies. During early heuristic decision-making, most active interventions were ineffective, while passive interventions generally improved outcomes. In the analytical phase, the effects were reversed. Passive interventions that facilitate information extraction consistently improved outcomes.

DISCUSSION

Our findings suggest that the effectiveness of active interventions negatively correlates with the amount of information received during delivery. During early heuristic decision-making, when information overload is high, physicians are unresponsive to alerts and proactively consult passive resources. In the later analytical phases, physicians show increased receptivity to alerts due to decreased diagnostic uncertainty and information quantity. Interventions that limit information lead to positive outcomes, supporting our interpretation.

CONCLUSION

We synthesize our findings into an integrated model that reveals the underlying reasons for conflicting findings from previous reviews and can guide practitioners in designing ISs in the ED.

The influence of electronic reminders on recording diagnoses in a primary health care emergency department: a register-based study in a Finnish town

OBJECTIVE

This study examines whether implementation of electronic reminders is associated with a change in the amount and content of diagnostic data recorded in primary health care emergency departments (ED).

DESIGN

A register-based 12-year follow-up study with a before-and-after design.

SETTING

This study was performed in a primary health care ED in Finland. An electronic reminder was installed in the health record system to remind physicians to include the diagnosis code of the visit to the health record.

SUBJECTS AND MAIN OUTCOME MEASURES

The report generator of the electronic health record-system provided monthly figures for the number of different recorded diagnoses by using the International Classification of Diagnoses (ICD-10th edition) and the total number of ED physician visits, thus allowing the calculation of the recording rate of diagnoses on a monthly basis and the comparison of diagnoses before and after implementing electronic reminders.

RESULTS

The most commonly recorded diagnoses in the ED were acute upper respiratory infections of various and unspecified sites (5.8%), abdominal and pelvic pain (4.8%), suppurative and unspecified otitis media (4.5%) and dorsalgia (4.0%). The diagnosis recording rate in the ED doubled from 41.2 to 86.3% (p < 0.001) after the application of electronic reminders. The intervention especially enhanced the recording rate of symptomatic diagnoses (ICD-10 group-R) and alcohol abuse-related diagnoses (ICD-10 code F10). Mental and behavioural disorders (group F) and injuries (groups S-Y) were also better recorded after this intervention.

CONCLUSION

Electronic reminders may alter the documentation habits of physicians and recording of clinical data, such as diagnoses, in the EDs. This may be of use when planning resource managing in EDs and planning their actions.KEY POINTSElectronic reminders enhance recording of diagnoses in primary care but what happens in emergency departments (EDs) is not known.Electronic reminders enhance recording of diagnoses in primary care ED.Especially recording of symptomatic diagnoses and alcohol abuse-related diagnoses increased.

Applying A/B Testing to Clinical Decision Support: Rapid Randomized Controlled Trials

BACKGROUND

Clinical decision support (CDS) is a valuable feature of electronic health records (EHRs) designed to improve quality and safety. However, due to the complexities of system design and inconsistent results, CDS tools may inadvertently increase alert fatigue and contribute to physician burnout. A/B testing, or rapid-cycle randomized tests, is a useful method that can be applied to the EHR in order to rapidly understand and iteratively improve design choices embedded within CDS tools.

OBJECTIVE

This paper describes how rapid randomized controlled trials (RCTs) embedded within EHRs can be used to quickly ascertain the superiority of potential CDS design changes to improve their usability, reduce alert fatigue, and promote quality of care.

METHODS

A multistep process combining tools from user-centered design, A/B testing, and implementation science was used to understand, ideate, prototype, test, analyze, and improve each candidate CDS. CDS engagement metrics (alert views, acceptance rates) were used to evaluate which CDS version is superior.

RESULTS

To demonstrate the impact of the process, 2 experiments are highlighted. First, after multiple rounds of usability testing, a revised CDS influenza alert was tested against usual care CDS in a rapid (~6 weeks) RCT. The new alert text resulted in minimal impact on reducing firings per patients per day, but this failure triggered another round of review that identified key technical improvements (ie, removal of dismissal button and firings in procedural areas) that led to a dramatic decrease in firings per patient per day (23.1 to 7.3). In the second experiment, the process was used to test 3 versions (financial, quality, regulatory) of text supporting tobacco cessation alerts as well as 3 supporting images. Based on 3 rounds of RCTs, there was no significant difference in acceptance rates based on the framing of the messages or addition of images.

CONCLUSIONS

These experiments support the potential for this new process to rapidly develop, deploy, and rigorously evaluate CDS within an EHR. We also identified important considerations in applying these methods. This approach may be an important tool for improving the impact of and experience with CDS.

TRIAL REGISTRATION

Flu alert trial: ClinicalTrials.gov NCT03415425; https://clinicaltrials.gov/ct2/show/NCT03415425. Tobacco alert trial: ClinicalTrials.gov NCT03714191; https://clinicaltrials.gov/ct2/show/NCT03714191.

Development, Implementation, and Evaluation of a Personalized Machine Learning Algorithm for Clinical Decision Support: Case Study With Shingles Vaccination

BACKGROUND

Although clinical decision support (CDS) alerts are effective reminders of best practices, their effectiveness is blunted by clinicians who fail to respond to an overabundance of inappropriate alerts. An electronic health record (EHR)-integrated machine learning (ML) algorithm is a potentially powerful tool to increase the signal-to-noise ratio of CDS alerts and positively impact the clinician's interaction with these alerts in general.

OBJECTIVE

This study aimed to describe the development and implementation of an ML-based signal-to-noise optimization system (SmartCDS) to increase the signal of alerts by decreasing the volume of low-value herpes zoster (shingles) vaccination alerts.

METHODS

We built and deployed SmartCDS, which builds personalized user activity profiles to suppress shingles vaccination alerts unlikely to yield a clinician's interaction. We extracted all records of shingles alerts from January 2017 to March 2019 from our EHR system, including 327,737 encounters, 780 providers, and 144,438 patients.

RESULTS

During the 6 weeks of pilot deployment, the SmartCDS system suppressed an average of 43.67% (15,425/35,315) potential shingles alerts (appointments) and maintained stable counts of weekly shingles vaccination orders (326.3 with system active vs 331.3 in the control group; P=.38) and weekly user-alert interactions (1118.3 with system active vs 1166.3 in the control group; P=.20).

CONCLUSIONS

All key statistics remained stable while the system was turned on. Although the results are promising, the characteristics of the system can be subject to future data shifts, which require automated logging and monitoring. We demonstrated that an automated, ML-based method and data architecture to suppress alerts are feasible without detriment to overall order rates. This work is the first alert suppression ML-based model deployed in practice and serves as foundational work in encounter-level customization of alert display to maximize effectiveness.

Live Usability Testing of Two Complex Clinical Decision Support Tools: Observational Study

BACKGROUND

Potential of the electronic health records (EHR) and clinical decision support (CDS) systems to improve the practice of medicine has been tempered by poor design and the resulting burden they place on providers. CDS is rarely tested in the real clinical environment. As a result, many tools are hard to use, placing strain on providers and resulting in low adoption rates. The existing CDS usability literature relies primarily on expert opinion and provider feedback via survey. This is the first study to evaluate CDS usability and the provider-computer-patient interaction with complex CDS in the real clinical environment.

OBJECTIVE

This study aimed to further understand the barriers and facilitators of meaningful CDS usage within a real clinical context.

METHODS

This qualitative observational study was conducted with 3 primary care providers during 6 patient care sessions. In patients with the chief complaint of sore throat, a CDS tool built with the Centor Score was used to stratify the risk of group A Streptococcus pharyngitis. In patients with a chief complaint of cough or upper respiratory tract infection, a CDS tool built with the Heckerling Rule was used to stratify the risk of pneumonia. During usability testing, all human-computer interactions, including audio and continuous screen capture, were recorded using the Camtasia software. Participants' comments and interactions with the tool during clinical sessions and participant comments during a postsession brief interview were placed into coding categories and analyzed for generalizable themes.

RESULTS

In the 6 encounters observed, primary care providers toggled between addressing either the computer or the patient during the visit. Minimal time was spent listening to the patient without engaging the EHR. Participants mostly used the CDS tool with the patient, asking questions to populate the calculator and discussing the results of the risk assessment; they reported the ability to do this as the major benefit of the tool. All providers were interrupted during their use of the CDS tool by the need to refer to other sections of the chart. In half of the visits, patients' clinical symptoms challenged the applicability of the tool to calculate the risk of bacterial infection. Primary care providers rarely used the incorporated incentives for CDS usage, including progress notes and patient instructions.

CONCLUSIONS

Live usability testing of these CDS tools generated insights about their role in the patient-provider interaction. CDS may contribute to the interaction by being simultaneously viewed by the provider and patient. CDS can improve usability and lessen the strain it places on providers by being short, flexible, and customizable to unique provider workflow. A useful component of CDS is being as widely applicable as possible and ensuring that its functions represent the fastest way to perform a particular task.

Improving Provider Adoption With Adaptive Clinical Decision Support Surveillance: An Observational Study

Background

Successful clinical decision support (CDS) tools can help use evidence-based medicine to effectively improve patient outcomes. However, the impact of these tools has been limited by low provider adoption due to overtriggering, leading to alert fatigue. We developed a tracking mechanism for monitoring trigger (percent of total visits for which the tool triggers) and adoption (percent of completed tools) rates of a complex CDS tool based on the Wells criteria for pulmonary embolism (PE).

Objective

We aimed to monitor and evaluate the adoption and trigger rates of the tool and assess whether ongoing tool modifications would improve adoption rates.

Methods

As part of a larger clinical trial, a CDS tool was developed using the Wells criteria to calculate pretest probability for PE at 2 tertiary centers’ emergency departments (EDs). The tool had multiple triggers: any order for D-dimer, computed tomography (CT) of the chest with intravenous contrast, CT pulmonary angiography (CTPA), ventilation-perfusion scan, or lower extremity Doppler ultrasound. A tracking dashboard was developed using Tableau to monitor real-time trigger and adoption rates. Based on initial low provider adoption rates of the tool, we conducted small focus groups with key ED providers to elicit barriers to tool use. We identified overtriggering of the tool for non-PE-related evaluations and inability to order CT testing for intermediate-risk patients. Thus, the tool was modified to allow CT testing for the intermediate-risk group and not to trigger for CT chest with intravenous contrast orders. A dialogue box, “Are you considering PE for this patient?” was added before the tool triggered to account for CTPAs ordered for aortic dissection evaluation.

Results

In the ED of tertiary center 1, 95,295 patients visited during the academic year. The tool triggered for an average of 509 patients per month (average trigger rate 2036/30,234, 6.73%) before the modifications, reducing to 423 patients per month (average trigger rate 1629/31,361, 5.22%). In the ED of tertiary center 2, 88,956 patients visited during the academic year, with the tool triggering for about 473 patients per month (average trigger rate 1892/29,706, 6.37%) before the modifications and for about 400 per month (average trigger rate 1534/30,006, 5.12%) afterward. The modifications resulted in a significant 4.5- and 3-fold increase in provider adoption rates in tertiary centers 1 and 2, respectively. The modifications increased the average monthly adoption rate from 23.20/360 (6.5%) tools to 81.60/280.20 (29.3%) tools and 46.60/318.80 (14.7%) tools to 111.20/263.40 (42.6%) tools in centers 1 and 2, respectively.

Conclusions

Close postimplementation monitoring of CDS tools may help improve provider adoption. Adaptive modifications based on user feedback may increase targeted CDS with lower trigger rates, reducing alert fatigue and increasing provider adoption. Iterative improvements and a postimplementation monitoring dashboard can significantly improve adoption rates.

Scope and Influence of Electronic Health Record-Integrated Clinical Decision Support in the Emergency Department: A Systematic Review

STUDY OBJECTIVE

As electronic health records evolve, integration of computerized clinical decision support offers the promise of sorting, collecting, and presenting this information to improve patient care. We conducted a systematic review to examine the scope and influence of electronic health record-integrated clinical decision support technologies implemented in the emergency department (ED).

METHODS

A literature search was conducted in 4 databases from their inception through January 18, 2018: PubMed, Scopus, the Cumulative Index of Nursing and Allied Health, and Cochrane Central. Studies were included if they examined the effect of a decision support intervention that was implemented in a comprehensive electronic health record in the ED setting. Standardized data collection forms were developed and used to abstract study information and assess risk of bias.

RESULTS

A total of 2,558 potential studies were identified after removal of duplicates. Of these, 42 met inclusion criteria. Common targets for clinical decision support intervention included medication and radiology ordering practices, as well as more comprehensive systems supporting diagnosis and treatment for specific disease entities. The majority of studies (83%) reported positive effects on outcomes studied. Most studies (76%) used a pre-post experimental design, with only 3 (7%) randomized controlled trials.

CONCLUSION

Numerous studies suggest that clinical decision support interventions are effective in changing physician practice with respect to process outcomes such as guideline adherence; however, many studies are small and poorly controlled. Future studies should consider the inclusion of more specific information in regard to design choices, attempt to improve on uncontrolled before-after designs, and focus on clinically relevant outcomes wherever possible.

A systematic review of clinical decision support systems for antimicrobial management: are we failing to investigate these interventions appropriately?

OBJECTIVES

Clinical decision support systems (CDSS) for antimicrobial management can support clinicians to optimize antimicrobial therapy. We reviewed all original literature (qualitative and quantitative) to understand the current scope of CDSS for antimicrobial management and analyse existing methods used to evaluate and report such systems.

METHOD

PRISMA guidelines were followed. Medline, EMBASE, HMIC Health and Management and Global Health databases were searched from 1 January 1980 to 31 October 2015. All primary research studies describing CDSS for antimicrobial management in adults in primary or secondary care were included. For qualitative studies, thematic synthesis was performed. Quality was assessed using Integrated quality Criteria for the Review Of Multiple Study designs (ICROMS) criteria. CDSS reporting was assessed against a reporting framework for behaviour change intervention implementation.

RESULTS

Fifty-eight original articles were included describing 38 independent CDSS. The majority of systems target antimicrobial prescribing (29/38;76%), are platforms integrated with electronic medical records (28/38;74%), and have a rules-based infrastructure providing decision support (29/38;76%). On evaluation against the intervention reporting framework, CDSS studies fail to report consideration of the non-expert, end-user workflow. They have narrow focus, such as antimicrobial selection, and use proxy outcome measures. Engagement with CDSS by clinicians was poor.

CONCLUSION

Greater consideration of the factors that drive non-expert decision making must be considered when designing CDSS interventions. Future work must aim to expand CDSS beyond simply selecting appropriate antimicrobials with clear and systematic reporting frameworks for CDSS interventions developed to address current gaps identified in the reporting of evidence.

User centered clinical decision support tools: adoption across clinician training level

BACKGROUND

Dissemination and adoption of clinical decision support (CDS) tools is a major initiative of the Affordable Care Act's Meaningful Use program. Adoption of CDS tools is multipronged with personal, organizational, and clinical settings factoring into the successful utilization rates. Specifically, the diffusion of innovation theory implies that 'early adopters' are more inclined to use CDS tools and younger physicians tend to be ranked in this category.

OBJECTIVE

This study examined the differences in adoption of CDS tools across providers' training level.

PARTICIPANTS

From November 2010 to 2011, 168 residents and attendings from an academic medical institution were enrolled into a randomized controlled trial.

INTERVENTION

The intervention arm had access to the CDS tool through the electronic health record (EHR) system during strep and pneumonia patient visits.

MAIN MEASURES

The EHR system recorded details on how intervention arm interacted with the CDS tool including acceptance of the initial CDS alert, completion of risk-score calculators and the signing of medication order sets. Using the EHR data, the study performed bivariate tests and general estimating equation (GEE) modeling to examine the differences in adoption of the CDS tool across residents and attendings.

KEY RESULTS

The completion rates of the CDS calculator and medication order sets were higher amongst first year residents compared to all other training levels. Attendings were the less likely to accept the initial step of the CDS tool (29.3%) or complete the medication order sets (22.4%) that guided their prescription decisions, resulting in attendings ordering more antibiotics (37.1%) during an CDS encounter compared to residents.

CONCLUSION

There is variation in adoption of CDS tools across training levels. Attendings tended to accept the tool less but ordered more medications. CDS tools should be tailored to clinicians' training levels.

Physician awareness of drug cost: a systematic review

BACKGROUND

Pharmaceutical costs are the fastest-growing health-care expense in most developed countries. Higher drug costs have been shown to negatively impact patient outcomes. Studies suggest that doctors have a poor understanding of pharmaceutical costs, but the data are variable and there is no consistent pattern in awareness. We designed this systematic review to investigate doctors' knowledge of the relative and absolute costs of medications and to determine the factors that influence awareness.

METHODS AND FINDINGS

Our search strategy included The Cochrane Library, EconoLit, EMBASE, and MEDLINE as well as reference lists and contact with authors who had published two or more articles on the topic or who had published within 10 y of the commencement of our review. Studies were included if: either doctors, trainees (interns or residents), or medical students were surveyed; there were more than ten survey respondents; cost of pharmaceuticals was estimated; results were expressed quantitatively; there was a clear description of how authors defined "accurate estimates"; and there was a description of how the true cost was determined. Two authors reviewed each article for eligibility and extracted data independently. Cost accuracy outcomes were summarized, but data were not combined in meta-analysis because of extensive heterogeneity. Qualitative data related to physicians and drug costs were also extracted. The final analysis included 24 articles. Cost accuracy was low; 31% of estimates were within 20% or 25% of the true cost, and fewer than 50% were accurate by any definition of cost accuracy. Methodological weaknesses were common, and studies of low methodological quality showed better cost awareness. The most important factor influencing the pattern and accuracy of estimation was the true cost of therapy. High-cost drugs were estimated more accurately than inexpensive ones (74% versus 31%, Chi-square p < 0.001). Doctors consistently overestimated the cost of inexpensive products and underestimated the cost of expensive ones (binomial test, 89/101, p < 0.001). When asked, doctors indicated that they want cost information and feel it would improve their prescribing but that it is not accessible.

CONCLUSIONS

Doctors' ignorance of costs, combined with their tendency to underestimate the price of expensive drugs and overestimate the price of inexpensive ones, demonstrate a lack of appreciation of the large difference in cost between inexpensive and expensive drugs. This discrepancy in turn could have profound implications for overall drug expenditures. Much more focus is required in the education of physicians about costs and the access to cost information. Future research should focus on the accessibility and reliability of medical cost information and whether the provision of this information is used by doctors and makes a difference to physician prescribing. Additionally, future work should strive for higher methodological standards to avoid the biases we found in the current literature, including attention to the method of assessing accuracy that allows larger absolute estimation ranges for expensive drugs.

Evaluation of outpatient computerized physician medication order entry systems: a systematic review

This paper provides a systematic literature review of CPOE evaluation studies in the outpatient setting on: safety; cost and efficiency; adherence to guideline; alerts; time; and satisfaction, usage, and usability. Thirty articles with original data (randomized clinical trial, non-randomized clinical trial, or observational study designs) met the inclusion criteria. Only four studies assessed the effect of CPOE on safety. The effect was not significant on the number of adverse drug events. Only one study showed a significant reduction of the number of medication errors. Three studies showed significant reductions in medication costs; five other studies could not support this. Most studies on adherence to guidelines showed a significant positive effect. The relatively small number of evaluation studies published to date do not provide adequate evidence that CPOE systems enhance safety and reduce cost in the outpatient settings. There is however evidence for (a) increasing adherence to guidelines, (b) increasing total prescribing time, and (c) high frequency of ignored alerts.