Reduction in Clinical Variance Using Targeted Design Changes in Computerized Provider Order Entry (CPOE) Order Sets: Impact on Hospitalized Children with Acute Asthma Exacerbation

Effectiveness of non-interruptive nudge interventions in electronic health records to improve the delivery of care in hospitals: a systematic review

OBJECTIVES

To describe the application of nudges within electronic health records (EHRs) and their effects on inpatient care delivery, and identify design features that support effective decision-making without the use of interruptive alerts.

MATERIALS AND METHODS

We searched Medline, Embase, and PsychInfo (in January 2022) for randomized controlled trials, interrupted time-series and before-after studies reporting effects of nudge interventions embedded in hospital EHRs to improve care. Nudge interventions were identified at full-text review, using a pre-existing classification. Interventions using interruptive alerts were excluded. Risk of bias was assessed using the ROBINS-I tool (Risk of Bias in Non-randomized Studies of Interventions) for non-randomized studies or the Cochrane Effective Practice and Organization of Care Group methodology for randomized trials. Study results were summarized narratively.

RESULTS

We included 18 studies evaluating 24 EHR nudges. An improvement in care delivery was reported for 79.2% (n = 19; 95% CI, 59.5-90.8) of nudges. Nudges applied were from 5 of 9 possible nudge categories: change choice defaults (n = 9), make information visible (n = 6), change range or composition of options (n = 5), provide reminders (n = 2), and change option-related effort (n = 2). Only one study had a low risk of bias. Nudges targeted ordering of medications, laboratory tests, imaging, and appropriateness of care. Few studies evaluated long-term effects.

DISCUSSION

Nudges in EHRs can improve care delivery. Future work could explore a wider range of nudges and evaluate long-term effects.

CONCLUSION

Nudges can be implemented in EHRs to improve care delivery within current system capabilities; however, as with all digital interventions, careful consideration of the sociotechnical system is crucial to enhance their effectiveness.

De-escalation of High-flow Respiratory Support for Children Admitted with Bronchiolitis: A Quality Improvement Initiative

Bronchiolitis is the most common cause for hospitalization in the first year of life, with hypoxemia and acute respiratory failure as major determinants leading to hospitalization. In addition, the lack of existing guidelines for weaning and discontinuing supplemental oxygen, including high-flow nasal cannula, may contribute to prolonged hospitalization and increased resource utilization.

The Role of Electronic Medical Records in Reducing Unwarranted Clinical Variation in Acute Health Care: Systematic Review

BACKGROUND

The use of electronic medical records (EMRs)/electronic health records (EHRs) provides potential to reduce unwarranted clinical variation and thereby improve patient health care outcomes. Minimization of unwarranted clinical variation may raise and refine the standard of patient care provided and satisfy the quadruple aim of health care.

OBJECTIVE

A systematic review of the impact of EMRs and specific subcomponents (PowerPlans/SmartSets) on variation in clinical care processes in hospital settings was undertaken to summarize the existing literature on the effects of EMRs on clinical variation and patient outcomes.

METHODS

Articles from January 2000 to November 2020 were identified through a comprehensive search that examined EMRs/EHRs and clinical variation or PowerPlans/SmartSets. Thirty-six articles met the inclusion criteria. Articles were examined for evidence for EMR-induced changes in variation and effects on health care outcomes and mapped to the quadruple aim of health care.

RESULTS

Most of the studies reported positive effects of EMR-related interventions (30/36, 83%). All of the 36 included studies discussed clinical variation, but only half measured it (18/36, 50%). Those studies that measured variation generally examined how changes to variation affected individual patient care (11/36, 31%) or costs (9/36, 25%), while other outcomes (population health and clinician experience) were seldom studied. High-quality study designs were rare.

CONCLUSIONS

The literature provides some evidence that EMRs can help reduce unwarranted clinical variation and thereby improve health care outcomes. However, the evidence is surprisingly thin because of insufficient attention to the measurement of clinical variation, and to the chain of evidence from EMRs to variation in clinical practices to health care outcomes.

Development and implementation of a clinical decision support-based initiative to drive intravenous fluid prescribing

OBJECTIVE

Studies suggest superior outcomes with use of intravenous (IV) balanced fluids compared to normal saline (NS). However, significant fluid prescribing variability persists, highlighting the knowledge-to-practice gap. We sought to identify contributors to prescribing variation and utilize a clinical decision support system (CDSS) to increase institutional balanced fluid prescribing.

MATERIALS AND METHODS

This single-center informatics-enabled quality improvement initiative for patients hospitalized or treated in the emergency department included stepwise interventions of 1) identification of design factors within the computerized provider order entry (CPOE) of our electronic health record (EHR) that contribute to preferential NS ordering, 2) clinician education, 3) fluid stocking modifications, 4) re-design and implementation of a CDSS-integrated IV fluid ordering panel, and 5) comparison of fluid prescribing before and after the intervention. EHR-derived prescribing data was analyzed via single interrupted time series.

RESULTS

Pre-intervention (3/2019-9/2019), balanced fluids comprised 33% of isotonic fluid orders, with gradual uptake (1.4%/month) of balanced fluid prescribing. Clinician education (10/2019-2/2020) yielded a modest (4.4%/month, 95% CI 1.6-7.2, p = 0.01) proportional increase in balanced fluid prescribing, while CPOE redesign (3/2020) yielded an immediate (20.7%, 95% CI 17.7-23.6, p < 0.0001) and sustained increase (72% of fluid orders in 12/2020). The intervention proved most effective among those with lower baseline balanced fluids utilization, including emergency medicine (57% increase, 95% CI 0.7-1.8, p < 0.0001) and internal medicine/subspecialties (18% increase, 95% CI 14.4-21.3, p < 0.0001) clinicians and substantially reduced institutional prescribing variation.

CONCLUSION

Integration of CDSS into an EHR yielded a robust and sustained increase in balanced fluid prescribing. This impact far exceeded that of clinician education highlighting the importance of CDSS.

Best practices in pediatric sepsis: building and sustaining an evidence-based pediatric sepsis quality improvement program

Pediatric sepsis is a common problem worldwide and is associated with significant morbidity and mortality. Best practice recommendations have been published by both the American College of Critical Care Medicine and the Surviving Sepsis Campaign to guide the recognition and treatment of pediatric sepsis. However, implementation of these recommendations can be challenging due to the complexity of the care required and intensity of resources needed to successfully implement programs. This paper outlines the experience with implementation of a pediatric sepsis quality improvement program at Primary Children's Hospital, a free-standing, quaternary care children's hospital in Salt Lake City. The hospital has implemented sepsis projects across multiple care settings. Challenges, lessons learned, and suggestions for implementation are described.PLAIN LANGUAGE SUMMARYSepsis is a life-threatening condition that results from an inappropriate response to an infection by the body's immune system. All children are potentially susceptible to sepsis, with nearly 8,000 children dying from the disease in the US each year. Sepsis is a complicated disease, and several international groups have published guidelines to help hospital teams treat children with sepsis appropriately. However, because recognizing and treating sepsis in children is challenging and takes a coordinated effort from many different types of healthcare team members, following the international sepsis guidelines effectively can be difficult and resource intensive. This paper describes how one children's hospital (Primary Children's Hospital in Salt Lake City, Utah) approached the challenge of implementing pediatric sepsis guidelines, some lessons learned from their experience, and suggestions for others interested in implementing sepsis guidelines for children.

Effect of default order set settings on telemetry ordering

OBJECTIVE

To investigate the effects of adjusting the default order set settings on telemetry usage.

MATERIALS AND METHODS

We performed a retrospective, controlled, before-after study of patients admitted to a house staff medicine service at an academic medical center examining the effect of changing whether the admission telemetry order was pre-selected or not. Telemetry orders on admission and subsequent orders for telemetry were monitored pre- and post-change. Two other order sets that had no change in their default settings were used as controls.

RESULTS

Between January 1, 2017 and May 1, 2018, there were 1, 163 patients admitted using the residency-customized version of the admission order set which initially had telemetry pre-selected. In this group of patients, there was a significant decrease in telemetry ordering in the post-intervention period: from 79.1% of patients in the 8.5 months prior ordered to have telemetry to 21.3% of patients ordered in the 7.5 months after (χ2 = 382; P < .001). There was no significant change in telemetry usage among patients admitted using the two control order sets.

DISCUSSION

Default settings have been shown to affect clinician ordering behavior in multiple domains. Consistent with prior findings, our study shows that changing the order set settings can significantly affect ordering practices. Our study was limited in that we were unable to determine if the change in ordering behavior had significant impact on patient care or safety.

CONCLUSION

Decisions about default selections in electronic health record order sets can have significant consequences on ordering behavior.

Effect of Combined Clinical Practice Guideline and Electronic Order Set Implementation on Febrile Infant Evaluation and Management

OBJECTIVE

Management of febrile infants 60 days and younger for suspected serious infection varies widely. Clinical practice guidelines (CPGs) are intended to improve clinician adherence to evidence-based practices. In 2011, a CPG for managing febrile infants was implemented in an urban children's hospital with simultaneous release of an electronic order set and algorithm to guide clinician decisions for managing infants for suspected serious bacterial infection. The objective of the present study was to determine the association of CPG implementation with order set use, clinical practices, and clinical outcomes.

METHODS

Records of febrile infants 60 days and younger from February 1, 2009, to January 31, 2013, were retrospectively reviewed. Clinical documentation, order set use, clinical management practices, and outcomes were compared pre-CPG and post-CPG release.

RESULTS

In total, 1037 infants pre-CPG and 930 infants post-CPG implementation were identified. After CPG release, more infants 29 to 60 days old underwent lumbar puncture (56% vs 62%, P = 0.02). Overall antibiotic use and duration of antibiotic use decreased for infants 29 to 60 days (57% vs 51%, P = 0.02). Blood culture and urine culture obtainment remained unchanged for older infants. Diagnosed infections, hospital readmissions, and length of stay were unchanged. Electronic order sets were used in 80% of patient encounters.

CONCLUSIONS

Antibiotic use and lumbar puncture performance modestly changed in accordance with CPG recommendations provided in the electronic order set and algorithm, suggesting that the presence of embedded prompts may affect clinician decision-making. Our results highlight the potential usefulness of these decision aids to improve adherence to CPG recommendations.

The Impact of Changes to an Electronic Admission Order Set on Prescribing and Clinical Outcomes in the Intensive Care Unit

BACKGROUND

Implementation of disease-specific order sets has improved compliance with standards of care for a variety of diseases. Evidence of the impact admission order sets can have on care is limited.

OBJECTIVE

The main purpose of this article is to evaluate the impact of changes made to an electronic critical care admission order set on provider prescribing patterns and clinical outcomes.

METHODS

A retrospective, observational before-and-after exploratory study was performed on adult patients admitted to the medical intensive care unit using the Inpatient Critical Care Admission Order Set. The primary outcome measure was the percentage change in the number of orders for scheduled acetaminophen, a histamine-2 receptor antagonist (H2RA), and lactated ringers at admission before implementation of the revised order set compared with after implementation. Secondary outcomes assessed clinical impact of changes made to the order set.

RESULTS

The addition of a different dosing strategy for a medication already available on the order set (scheduled acetaminophen vs. as needed acetaminophen) had no impact on physician prescribing (0 vs. 0%, p = 1.000). The addition of a new medication class (an H2RA) to the order set significantly increased the number of patients prescribed an H2RA for stress ulcer prophylaxis (0 vs. 20%, p < 0.001). Rearranging the list of maintenance intravenous fluids to make lactated ringers the first fluid option in place of normal saline significantly decreased the number of orders for lactated ringers (17 vs. 4%, p = 0.005). The order set changes had no significant impact on clinical outcomes such as incidence of transaminitis, gastrointestinal bleed, and acute kidney injury.

CONCLUSION

Making changes to an admission order set can impact provider prescribing patterns. The type of change made to the order set, in addition to the specific medication changed, may have an effect on how influential the changes are on prescribing patterns.

Exploring Different Approaches in Measuring EHR-based Adherence to Best Practice - A Case Study with Order Sets and Associated Outcomes

In connection with a recent enterprise-wide rollout of a new electronic health record, Intermountain Healthcare is investing significant effort in building a central library of best-practice order sets. These order sets represent best practice guidelines for specific clinical scenarios and are deployed with the intent of standardizing care, reducing variation, and consistently delivering good clinical outcomes to the populations we serve. The importance of measuring their use and the level to which caregivers adhere to these standards becomes an important factor in understanding and characterizing the impact that they deliver. Notwithstanding the importance of these metrics, well- defined methods for measuring adherence to a given clinical guideline as delivered through an order set are not fully characterized in the medical literature. In this paper, we describe initial efforts at measuring compliance to a defined 'best practice' standard by means of content utilization analysis, a calculated adherence model, and relevant clinical key performance indicators. The degree to which specified clinical outcomes vary across these measurement models are compared for a group of order sets tied to treating coronary artery bypass graft patients and heart failure patients. While the patterns derived from this analysis show some uncertainty, more granular methods that look at line-item, or 'order level' detail reveal more significant differences in the corresponding set of outcomes than higher-level adherence surrogates.

Sustained Improvement in Inflammatory Bowel Disease Quality Measures Using an Electronic Health Record Intervention

BACKGROUND

Inflammatory bowel disease (IBD) is a chronic condition with wide variation in treatment and resource utilization because of many different disease presentations and treatment options. In an effort to standardize care and improve health outcomes, several organizations have created performance measures to monitor various aspects of IBD care.

OBJECTIVES

We aimed to assess longitudinal documentation adherence with physician quality reporting system's (PQRS) IBD performance measures before, immediately after, and 1 year following the implementation of a comprehensive electronic health record (EHR) IBD clinical documentation support tool intervention.

METHODS

We reviewed 50 patient charts that were randomly selected from consecutive outpatient IBD visits at our tertiary care center from September 1, 2015 to June 30, 2016, prior to implementation of an IBD-specific note template, order set, and patient education handout on September 1, 2016. Two additional cohorts of 50 patient charts were randomly selected from September 1, 2016 to June 30, 2017 and September 1, 2017 to June 30, 2018. These charts were reviewed to assess adherence of pertinent PQRS performance measures for outpatient IBD care. The project was deemed not human subjects research and received exempt approval by the Institutional Review Board (IRB#: IRB00040399).

RESULTS

The cohort immediately after the intervention showed significant increases in documentation rates of influenza immunization (19-59%, p < 0.001), pneumococcal immunizations (2-38%, p < 0.001), tobacco cessation (28.6-77.8%, p = 0.049), and proportion of all eligible measures (40.6-62.2%, p < 0.001) when compared with the preintervention group. Moreover, documentation rates were sustained in the 1-year follow-up group when compared with the postintervention group.

CONCLUSION

A multifaceted, EHR focused approach can significantly and sustainably improve documentation of outpatient IBD quality measures.

When order sets do not align with clinician workflow: assessing practice patterns in the electronic health record

BACKGROUND

Order sets are widely used tools in the electronic health record (EHR) for improving healthcare quality. However, there is limited insight into how well they facilitate clinician workflow. We assessed four indicators based on order set usage patterns in the EHR that reflect potential misalignment between order set design and clinician workflow needs.

METHODS

We used data from the EHR on all orders of medication, laboratory, imaging and blood product items at an academic hospital and an itemset mining approach to extract orders that frequently co-occurred with order set use. We identified the following four indicators: infrequent ordering of order set items, rapid retraction of medication orders from order sets, additional a la carte ordering of items not included in order sets and a la carte ordering of items despite being listed in the order set.

RESULTS

There was significant variability in workflow alignment across the 11 762 order set items used in the 77 421 inpatient encounters from 2014 to 2017. The median ordering rate was 4.1% (IQR 0.6%-18%) and median medication retraction rate was 4% (IQR 2%-10%). 143 (5%) medications were significantly less likely while 68 (3%) were significantly more likely to be retracted than if the same medication was ordered a la carte. 214 (39%) order sets were associated with least one additional item frequently ordered a la carte and 243 (45%) order sets contained at least one item that was instead more often ordered a la carte.

CONCLUSION

Order sets often do not align with what clinicians need at the point of care. Quantitative insights from EHRs may inform how order sets can be optimised to facilitate clinician workflow.

A Quality Improvement Initiative to Decrease Platelet Ordering Errors and a Proposed Model for Evaluating Clinical Decision Support Effectiveness

BACKGROUND AND OBJECTIVES

Clinical decision support (CDS) and computerized provider order entry have been shown to improve health care quality and safety, but may also generate previously unanticipated errors. We identified multiple CDS tools for platelet transfusion orders. In this study, we sought to evaluate and improve the effectiveness of those CDS tools while creating and testing a framework for future evaluation of other CDS tools.

METHODS

Using a query of an enterprise data warehouse at a tertiary care pediatric hospital, we conducted a retrospective analysis to assess baseline use and performance of existing CDS for platelet transfusion orders. Our outcome measure was the percentage of platelet undertransfusion ordering errors. Errors were defined as platelet transfusion volumes ordered which were less than the amount recommended by the order set used. We then redesigned our CDS and measured the impact of our intervention prospectively using statistical process control methodology.

RESULTS

We identified that 62% of all platelet transfusion orders were placed with one of two order sets (Inpatient Service 1 and Inpatient Service 2). The Inpatient Service 1 order set had a significantly higher occurrence of ordering errors (3.10% compared with 1.20%). After our interventions, platelet transfusion order error occurrence on Inpatient Service 1 decreased from 3.10 to 0.33%.

CONCLUSION

We successfully reduced platelet transfusion ordering errors by redesigning our CDS tools. We suggest that the use of collections of clinical data may help identify patterns in erroneous ordering, which could otherwise go undetected. We have created a framework which can be used to evaluate the effectiveness of other similar CDS tools.

Successful Use of Quality Improvement Methodology to Reduce Inpatient Length of Stay in Bronchiolitis Through Judicious Use of Intermittent Pulse Oximetry

BACKGROUND AND OBJECTIVES

The American Academy of Pediatrics 2014 bronchiolitis guidelines recommend against the routine use of continuous pulse oximetry (CPO) because it has been implicated in prolonging the length of stay (LOS). At our institution, infants admitted with bronchiolitis were monitored by using CPO during the entire hospital stay and intermittent desaturations <90% appeared to delay discharge. This quality improvement initiative was designed to reduce the LOS by decreasing the use of CPO in stable infants with nonsevere bronchiolitis.

METHODS

The quality improvement project was implemented on the inpatient units of 2 community hospitals during the 2016 and 2017 bronchiolitis seasons. In cycle 1 (January 2016 to April 2016), the bronchiolitis pathway from the associated quaternary children's hospital was used to (1) limit the use of CPO to patients with severe bronchiolitis and those at high risk for apnea or severe disease, (2) discontinue CPO as patients improved and stabilized, and (3) standardize discharge criteria. In cycle 2 (November 2016 to April 2017), the clinical pathway was adopted. The main outcome measure was LOS, measured from the time of the admission order to the time of the discharge order. Process measures included compliance with the interventions.

RESULTS

The project included 373 patients, 180 preintervention and 193 postintervention. The average LOS decreased by 20 hours, from 53 hours at baseline to 33 hours in cycle 2. No adverse events were noted, and there was no significant change in the number of emergency department revisits and readmissions within 7 days.

CONCLUSIONS

In our study, LOS was successfully reduced in bronchiolitis patients by using a clinical pathway that limited CPO to patients with severe bronchiolitis and those at risk for severe disease or apnea.

A Quality Improvement Bundle Including Pay for Performance for the Standardization of Order Set Use in Moderate Asthma

OBJECTIVE

In order to standardize use of our hospital's computerized asthma order set, which was developed based on an asthma clinical practice guideline, for moderately ill children presenting for care of asthma, we developed a quality improvement bundle, including a time-limited pay-for-performance component, for pediatric emergency department and pediatric urgent care faculty members.

METHODS

Following baseline measurement, we used a run-in period for education, feedback, and improvement of the asthma order set. Then, faculty members earned 0.1% of salary during each of 10 successive months (evaluation period) in which the asthma order set was used in managing 90% or more of eligible patients.

RESULTS

At baseline, the asthma order set was used in managing 60.5% of eligible patients. Order set use rose sharply during the run-in period. During the 10-month evaluation period, use of the asthma order set was significantly above baseline, with a mean of 91.6%; faculty earned pay-for-performance bonuses during 8 of 10 possible months. Following completion of the evaluation period, asthma order set use remained high.

CONCLUSIONS

A quality improvement bundle, including a time-limited pay-for-performance component, was associated with a sustained increase in the use of a computerized asthma order set for managing moderately ill asthmatic children.

Extracting Actionable Recommendations for Modifying Enterprise Order Set Templates from CPOE Utilization Patterns

As part of an enterprise-wide rollout of a new EHR, Intermountain Healthcare is investing significant effort in building a central library of best-practice order sets. As part of this effort, we have built analytics tools that can capture and determine actionable opportunities for change to order set templates, as reflected by aggregate user data. In order to determine the acceptability of this system and set meaningful thresholds for actual use, we extracted recommendations for additions, removals, and change in initial order selection status for a series of thirteen order sets. We asked local clinical experts to review the changes and classify them as acceptable or not. In total, the system identified 362 potential changes in the order set templates and 186 were deemed acceptable. While further enhancement will co sharpen the efficacy of the intervention, we expect that this type of utility will provide useful insight for content owners.

Personalization and Patient Involvement in Decision Support Systems: Current Trends

OBJECTIVES

This survey aims at highlighting the latest trends (2012-2014) on the development, use, and evaluation of Information and Communication Technologies (ICT) based decision support systems (DSSs) in medicine, with a particular focus on patient-centered and personalized care.

METHODS

We considered papers published on scientific journals, by querying PubMed and Web of ScienceTM. Included studies focused on the implementation or evaluation of ICT-based tools used in clinical practice. A separate search was performed on computerized physician order entry systems (CPOEs), since they are increasingly embedding patient-tailored decision support.

RESULTS

We found 73 papers on DSSs (53 on specific ICT tools) and 72 papers on CPOEs. Although decision support through the delivery of recommendations is frequent (28/53 papers), our review highlighted also DSSs only based on efficient information presentation (25/53). Patient participation in making decisions is still limited (9/53), and mostly focused on risk communication. The most represented medical area is cancer (12%). Policy makers are beginning to be included among stakeholders (6/73), but integration with hospital information systems is still low. Concerning knowledge representation/management issues, we identified a trend towards building inference engines on top of standard data models. Most of the tools (57%) underwent a formal assessment study, even if half of them aimed at evaluating usability and not effectiveness.

CONCLUSIONS

Overall, we have noticed interesting evolutions of medical DSSs to improve communication with the patient, consider the economic and organizational impact, and use standard models for knowledge representation. However, systems focusing on patient-centered care still do not seem to be available at large.

Predicting inpatient clinical order patterns with probabilistic topic models vs conventional order sets

OBJECTIVE

Build probabilistic topic model representations of hospital admissions processes and compare the ability of such models to predict clinical order patterns as compared to preconstructed order sets.

MATERIALS AND METHODS

The authors evaluated the first 24 hours of structured electronic health record data for > 10 K inpatients. Drawing an analogy between structured items (e.g., clinical orders) to words in a text document, the authors performed latent Dirichlet allocation probabilistic topic modeling. These topic models use initial clinical information to predict clinical orders for a separate validation set of > 4 K patients. The authors evaluated these topic model-based predictions vs existing human-authored order sets by area under the receiver operating characteristic curve, precision, and recall for subsequent clinical orders.

RESULTS

Existing order sets predict clinical orders used within 24 hours with area under the receiver operating characteristic curve 0.81, precision 16%, and recall 35%. This can be improved to 0.90, 24%, and 47% ( P  < 10 -20 ) by using probabilistic topic models to summarize clinical data into up to 32 topics. Many of these latent topics yield natural clinical interpretations (e.g., "critical care," "pneumonia," "neurologic evaluation").

DISCUSSION

Existing order sets tend to provide nonspecific, process-oriented aid, with usability limitations impairing more precise, patient-focused support. Algorithmic summarization has the potential to breach this usability barrier by automatically inferring patient context, but with potential tradeoffs in interpretability.

CONCLUSION

Probabilistic topic modeling provides an automated approach to detect thematic trends in patient care and generate decision support content. A potential use case finds related clinical orders for decision support.

Decaying relevance of clinical data towards future decisions in data-driven inpatient clinical order sets

OBJECTIVE

Determine how varying longitudinal historical training data can impact prediction of future clinical decisions. Estimate the "decay rate" of clinical data source relevance.

MATERIALS AND METHODS

We trained a clinical order recommender system, analogous to Netflix or Amazon's "Customers who bought A also bought B..." product recommenders, based on a tertiary academic hospital's structured electronic health record data. We used this system to predict future (2013) admission orders based on different subsets of historical training data (2009 through 2012), relative to existing human-authored order sets.

RESULTS

Predicting future (2013) inpatient orders is more accurate with models trained on just one month of recent (2012) data than with 12 months of older (2009) data (ROC AUC 0.91 vs. 0.88, precision 27% vs. 22%, recall 52% vs. 43%, all P<10^-10). Algorithmically learned models from even the older (2009) data was still more effective than existing human-authored order sets (ROC AUC 0.81, precision 16% recall 35%). Training with more longitudinal data (2009-2012) was no better than using only the most recent (2012) data, unless applying a decaying weighting scheme with a "half-life" of data relevance about 4 months.

DISCUSSION

Clinical practice patterns (automatically) learned from electronic health record data can vary substantially across years. Gold standards for clinical decision support are elusive moving targets, reinforcing the need for automated methods that can adapt to evolving information.

CONCLUSIONS AND RELEVANCM

Prioritizing small amounts of recent data is more effective than using larger amounts of older data towards future clinical predictions.

Visualization of Order Set Creation and Usage Patterns in Early Implementation Phases of an Electronic Health Record

Robust order set catalogs are considered to be a vital part of a computerized physician order entry (CPOE) implementation. Tools and processes for building, localizing, and maintaining these content sets in a centralized repository are important in facilitating the knowledge management lifecycle. Collectively, these order sets represent a significant investment of effort and expertise in capturing and distributing best clinical practice throughout an enterprise. In order to address an important gap of understanding how order sets are both created and used in practice in a current EHR installation, we have developed tools to analyze how order sets are used and customized in clinical practice. In this paper, we present the capabilities of these tools. We further characterize early development patterns in our enterprise order set catalog in early phases of a system-wide vendor EHR rollout. We present data that show how personalized order sets (favorites) are authored and then used in clinical practice. We anticipate that this type of utility will provide useful insight and feedback for those tasked with content governance and maintenance in CPOE systems.

Impact of Increased Early Statin Administration on Ischemic Stroke Outcomes: A Multicenter Electronic Medical Record Intervention

BACKGROUND

Statin administration early in ischemic stroke may influence outcomes. Our aim was to determine the clinical impact of increasing statin administration early in ischemic stroke hospitalization.

METHODS AND RESULTS

This is a retrospective analysis of a multicenter electronic medical record (EMR) intervention to increase early statin administration in ischemic stroke across all 20 hospitals of an integrated healthcare delivery system. A stroke EMR order set was modified from an "opt-in" to "opt-out" mode of statin ordering. Outcomes were mortality by 90 days, discharge disposition, and increase in stroke severity. We examined the relationship between intervention and outcome using autoregressive integrated moving average (ARIMA) time-series modeling. The EMR intervention increased both overall in-hospital statin administration (from 87.2% to 90.7%, P<0.001) and early statin administration (from 16.9% to 26.3%, P<0.001). ARIMA models showed a small increase in the rate of survival (difference in probability [Pdiff]=0.02, P=0.016) and discharge to home or rehabilitation facility (Pdiff=0.04, P=0.034) associated with the intervention. The increase in statin administration <8 hours was associated with much larger increases in survival (Pdiff=0.17, P=0.033) and rate of discharge to home or rehabilitation (Pdiff=0.29, P=0.011), as well as a decreased rate of neurological deterioration in-hospital (Pdiff=-0.14, P=0.026).

CONCLUSIONS

A simple EMR change increased early statin administration in ischemic stroke and was associated with improved clinical outcomes. This is, to our knowledge, the first EMR intervention study to show that a modification of an electronic order set resulted in improved clinical outcomes.

Increasing compliance with low tidal volume ventilation in the ICU with two nudge-based interventions: evaluation through intervention time-series analyses

OBJECTIVES

Low tidal volume (TVe) ventilation improves outcomes for ventilated patients, and the majority of clinicians state they implement it. Unfortunately, most patients never receive low TVes. 'Nudges' influence decision-making with subtle cognitive mechanisms and are effective in many contexts. There have been few studies examining their impact on clinical decision-making. We investigated the impact of 2 interventions designed using principles from behavioural science on the deployment of low TVe ventilation in the intensive care unit (ICU).

SETTING

University Hospitals Bristol, a tertiary, mixed medical and surgical ICU with 20 beds, admitting over 1300 patients per year.

PARTICIPANTS

Data were collected from 2144 consecutive patients receiving controlled mechanical ventilation for more than 1 hour between October 2010 and September 2014. Patients on controlled mechanical ventilation for more than 20 hours were included in the final analysis.

INTERVENTIONS

(1) Default ventilator settings were adjusted to comply with low TVe targets from the initiation of ventilation unless actively changed by a clinician. (2) A large dashboard was deployed displaying TVes in the format mL/kg ideal body weight (IBW) with alerts when TVes were excessive.

PRIMARY OUTCOME MEASURE

TVe in mL/kg IBW.

FINDINGS

TVe was significantly lower in the defaults group. In the dashboard intervention, TVe fell more quickly and by a greater amount after a TVe of 8 mL/kg IBW was breached when compared with controls. This effect improved in each subsequent year for 3 years.

CONCLUSIONS

This study has demonstrated that adjustment of default ventilator settings and a dashboard with alerts for excessive TVe can significantly influence clinical decision-making. This offers a promising strategy to improve compliance with low TVe ventilation, and suggests that using insights from behavioural science has potential to improve the translation of evidence into practice.

Default settings of computerized physician order entry system order sets drive ordering habits

BACKGROUND

Computerized physician order entry (CPOE) systems are quickly becoming ubiquitous, and groups of orders ("order sets") to allow for easy order input are a common feature. This provides a streamlined mechanism to view, modify, and place groups of related orders. This often serves as an electronic equivalent of a specialty requisition. A characteristic, of these order sets is that specific orders can be predetermined to be "preselected" or "defaulted-on" whenever the order set is used while others are "optional" or "defaulted-off" (though there is typically the option is to "deselect" defaulted-on tests in a given situation). While it seems intuitive that the defaults in an order set are often accepted, additional study is required to understand the impact of these "default" settings in an order set on ordering habits. This study set out to quantify the effect of changing the default settings of an order set.

METHODS

For quality improvement purposes, order sets dealing with transfusions were recently reviewed and modified to improve monitoring of outcome. Initially, the order for posttransfusion hematocrits and platelet count had the default setting changed from "optional" to "preselected." The default settings for platelet count was later changed back to "optional," allowing for a natural experiment to study the effect of the default selections of an order set on clinician ordering habits.

RESULTS

Posttransfusion hematocrit values were ordered for 8.3% of red cell transfusions when the default order set selection was "off" and for 57.4% of transfusions when the default selection was "preselected" (P < 0.0001). Posttransfusion platelet counts were ordered for 7.0% of platelet transfusions when the initial default order set selection was "optional," increased to 59.4% when the default was changed to "preselected" (P < 0.0001), and then decreased to 7.5% when the default selection was returned to "optional." The posttransfusion platelet count rates during the two "optional" periods: 7.0% versus 7.5% - were not statistically different (P = 0.620).

DISCUSSION

Default settings in CPOE order sets can significantly influence physician selection of laboratory tests. Careful consideration by all stakeholders, including clinicians and pathologists, should be obtained when establishing default settings in order sets.

From bench to bed: bridging from informatics theory to practice. An exploratory analysis

BACKGROUND

In 2009, the journal Applied Clinical Informatics (ACI) commenced publication. Focused on applications in clinical informatics, ACI was intended to be a companion journal to METHODS of Information in Medicine (MIM). Both journals are official journals of IMIA, the International Medical Informatics Association.

OBJECTIVES

To explore, after five years, which congruencies and interdependencies exist in publications of these journals and to determine if gaps exist. To achieve this goal, major topics discussed in ACI and in MIM had to be analysed. Finally, we wanted to explore, whether the intention of publishing these companion journals to provide an information bridge from informatics theory to informatics practice and from practice to theory could be supported by this model. In this manuscript we will report on congruencies and interdependencies from practise to theory and on major topis in ACI. Further results will be reported in a second paper.

METHODS

Retrospective, prolective observational study on recent publications of ACI and MIM. All publications of the years 2012 and 2013 from these journals were indexed and analysed.

RESULTS

Hundred and ninety-six publications have been analysed (87 ACI, 109 MIM). In ACI publications addressed care coordination, shared decision support, and provider communication in its importance for complex patient care and safety and quality. Other major themes included improving clinical documentation quality and efficiency, effectiveness of clinical decision support and alerts, implementation of health information technology systems including discussion of failures and succeses. An emerging topic in the years analyzed was a focus on health information technology to predict and prevent hospital admissions and managing population health including the application of mobile health technology. Congruencies between journals could be found in themes, but with different focus in its contents. Interdependencies from practise to theory found in these publications, were only limited.

CONCLUSIONS

Bridging from informatics theory to practise and vice versa remains a major component of successful research and practise as well as a major challenge.

Paving the COWpath: data-driven design of pediatric order sets

OBJECTIVE

Evidence indicates that users incur significant physical and cognitive costs in the use of order sets, a core feature of computerized provider order entry systems. This paper develops data-driven approaches for automating the construction of order sets that match closely with user preferences and workflow while minimizing physical and cognitive workload.

MATERIALS AND METHODS

We developed and tested optimization-based models embedded with clustering techniques using physical and cognitive click cost criteria. By judiciously learning from users' actual actions, our methods identify items for constituting order sets that are relevant according to historical ordering data and grouped on the basis of order similarity and ordering time. We evaluated performance of the methods using 47,099 orders from the year 2011 for asthma, appendectomy and pneumonia management in a pediatric inpatient setting.

RESULTS

In comparison with existing order sets, those developed using the new approach significantly reduce the physical and cognitive workload associated with usage by 14-52%. This approach is also capable of accommodating variations in clinical conditions that affect order set usage and development.

DISCUSSION

There is a critical need to investigate the cognitive complexity imposed on users by complex clinical information systems, and to design their features according to 'human factors' best practices. Optimizing order set generation using cognitive cost criteria introduces a new approach that can potentially improve ordering efficiency, reduce unintended variations in order placement, and enhance patient safety.

CONCLUSIONS

We demonstrate that data-driven methods offer a promising approach for designing order sets that are generalizable, data-driven, condition-based, and up to date with current best practices.

Using 'nudge' principles for order set design: a before and after evaluation of an electronic prescribing template in critical care

OBJECTIVE

Computerised order sets have the potential to reduce clinical variation and improve patient safety but the effect is variable. We sought to evaluate the impact of changes to the design of an order set on the delivery of chlorhexidine mouthwash and hydroxyethyl starch (HES) to patients in the intensive care unit.

METHODS

The study was conducted at University Hospitals Bristol NHS Foundation Trust, UK. Our intensive care unit uses a clinical information system (CIS). All drugs and fluids are prescribed with the CIS and drug and fluid charts are stored within a database. Chlorhexidine mouthwash was added as a default prescription to the prescribing template in January 2010. HES was removed from the prescribing template in April 2009. Both interventions were available to prescribe manually throughout the study period. We conducted a database review of all patients eligible for each intervention before and after changes to the configuration of choices within the prescribing system.

RESULTS

2231 ventilated patients were identified as appropriate for treatment with chlorhexidine, 591 before the intervention and 1640 after. 55.3% were prescribed chlorhexidine before the change and 90.4% after (p<0.001). 6199 patients were considered in the HES intervention, 2177 before the intervention and 4022 after. The mean volume of HES infused per patient fell from 630 mL to 20 mL after the change (p<0.001) and the percentage of patients receiving HES fell from 54.1% to 3.1% (p<0.001). These results were well sustained with time.

CONCLUSIONS

The presentation of choices within an electronic prescribing system influenced the delivery of evidence-based interventions in a predictable way and the effect was well sustained. This approach has the potential to enhance the effectiveness of computerised order sets.