Interface, information, interaction: a narrative review of design and functional requirements for clinical decision support

A Discount Approach to Reducing Nursing Alert Burden

BACKGROUND

Numerous programs have arisen to address interruptive clinical decision support (CDS) with the goals of reducing alert burden and alert fatigue. These programs often have standing committees with broad stakeholder representation, significant governance efforts, and substantial analyst hours to achieve reductions in alert burden which can be difficult for hospital systems to replicate.

OBJECTIVE

 This study aimed to reduce nursing alert burden with a primary nurse informaticist and small support team through a quality-improvement approach focusing on high-volume alerts.

METHODS

 Target alerts were identified from the period of January 2022 to April 2022 and four of the highest firing alerts were chosen initially, which accounted for 43% of all interruptive nursing alerts and an estimated 86 hours per month of time across all nurses occupied resolving these alerts per month. Work was done concurrently for each alert with design changes based on the Five Rights of CDS and following a quality-improvement framework. Priority for work was based on operational engagement for design review and approval. Once initial design changes were approved, alerts were taken for in situ usability testing and additional changes were made as needed. Final designs were presented to stakeholders for approval prior to implementation.

RESULTS

 The total number of interruptive nursing alert firings decreased by 58% from preintervention period (1 January 2022-30 June 2022) to postintervention period (July 1, 2022-December 31, 2022). Action taken on alerts increased from 8.1 to 17.3%. The estimated time spent resolving interruptive alerts summed across all nurses in the system decreased from 197 hours/month to 114 hours/month.

CONCLUSION

 While CDS may improve use of evidence-based practices, implementation without a clear framework for evaluation and monitoring often results in alert burden and fatigue without clear benefits. An alert burden reduction effort spearheaded by a single empowered nurse informaticist efficiently reduced nursing alert burden substantially.

An Infrastructure Framework for Remote Patient Monitoring Interventions and Research

Remote patient monitoring (RPM) enables clinicians to maintain and adjust their patients' plan of care by using remotely gathered data, such as vital signs, to proactively make medical decisions about a patient's care. RPM interventions have been touted as a means to improve patient care and well-being while reducing costs and resource needs within the health care ecosystem. However, multiple interworking components must be successfully implemented for an RPM intervention to yield the desired outcomes, and the design and key driver of each component can vary depending on the medical context. This viewpoint and perspective paper presents a 4-component RPM infrastructure framework based on a synthesis of existing literature and practice related to RPM. Specifically, these components are identified and considered: (1) data collection, (2) data transmission and storage, (3) data analysis, and (4) information presentation. Interaction points to consider between components include transmission, interoperability, accessibility, workflow integration, and transparency. Within each of the 4 components, questions affecting research and practice emerge that can affect the outcomes of RPM interventions. This framework provides a holistic perspective of the technologies involved in RPM interventions and how these core elements interact to provide an appropriate infrastructure for deploying RPM in health systems. Further, it provides a common vocabulary to compare and contrast RPM solutions across health contexts and may stimulate new research and intervention opportunities.

Designing the User Interface of a Nitroglycerin Dose Titration Decision Support System: User-Centered Design Study

BACKGROUND

 Nurses adjust intravenous nitroglycerin infusions to provide acute relief for angina by manually increasing or decreasing the dosage. However, titration can pose challenges, as excessively high doses can lead to hypotension, and low doses may result in inadequate pain relief. Clinical decision support systems (CDSSs) that predict changes in blood pressure for nitroglycerin dose adjustments may assist nurses with titration.

OBJECTIVE

 This study aimed to design a user interface for a CDSS for nitroglycerin dose titration (Nitroglycerin Dose Titration Decision Support System [nitro DSS]).

METHODS

 A user-centered design (UCD) approach, consisting of an initial qualitative study with semistructured interviews to identify design specifications for prototype development, was used. This was followed by three iterative rounds of usability testing. Nurses with experience titrating nitroglycerin infusions in coronary care units participated.

RESULTS

 A total of 20 nurses participated, including 7 during the qualitative study and 15 during usability testing (2 nurses participated in both phases). Analysis of the qualitative data revealed four themes for the interface design to be (1) clear and consistent, (2) vigilant, (3) interoperable, and (4) reliable. The major elements of the final prototype included a feature for viewing the predicted and actual blood pressure over time to determine the reliability of the predictions, a drop-down option to report patient side effects, a feature to report reasons for not accepting the prediction, and a visual alert indicating any systolic blood pressure predictions below 90 mm Hg. Nurses' ratings on the questionnaires indicated excellent usability and acceptability of the final nitro DSS prototype.

CONCLUSION

 This study successfully applied a UCD approach to collaborate with nurses in developing a user interface for the nitro DSS that supports the clinical decision-making of nurses titrating nitroglycerin.

Clinical Decision Support Requirements for Ventricular Tachycardia Diagnosis Within the Frameworks of Knowledge and Practice: Survey Study

BACKGROUND

Ventricular tachycardia (VT) diagnosis is challenging due to the similarity between VT and some forms of supraventricular tachycardia, complexity of clinical manifestations, heterogeneity of underlying diseases, and potential for life-threatening hemodynamic instability. Clinical decision support systems (CDSSs) have emerged as promising tools to augment the diagnostic capabilities of cardiologists. However, a requirements analysis is acknowledged to be vital for the success of a CDSS, especially for complex clinical tasks such as VT diagnosis.

OBJECTIVE

The aims of this study were to analyze the requirements for a VT diagnosis CDSS within the frameworks of knowledge and practice and to determine the clinical decision support (CDS) needs.

METHODS

Our multidisciplinary team first conducted semistructured interviews with seven cardiologists related to the clinical challenges of VT and expected decision support. A questionnaire was designed by the multidisciplinary team based on the results of interviews. The questionnaire was divided into four sections: demographic information, knowledge assessment, practice assessment, and CDS needs. The practice section consisted of two simulated cases for a total score of 10 marks. Online questionnaires were disseminated to registered cardiologists across China from December 2022 to February 2023. The scores for the practice section were summarized as continuous variables, using the mean, median, and range. The knowledge and CDS needs sections were assessed using a 4-point Likert scale without a neutral option. Kruskal-Wallis tests were performed to investigate the relationship between scores and practice years or specialty.

RESULTS

Of the 687 cardiologists who completed the questionnaire, 567 responses were eligible for further analysis. The results of the knowledge assessment showed that 383 cardiologists (68%) lacked knowledge in diagnostic evaluation. The overall average score of the practice assessment was 6.11 (SD 0.55); the etiological diagnosis section had the highest overall scores (mean 6.74, SD 1.75), whereas the diagnostic evaluation section had the lowest scores (mean 5.78, SD 1.19). A majority of cardiologists (344/567, 60.7%) reported the need for a CDSS. There was a significant difference in practice competency scores between general cardiologists and arrhythmia specialists (P=.02).

CONCLUSIONS

There was a notable deficiency in the knowledge and practice of VT among Chinese cardiologists. Specific knowledge and practice support requirements were identified, which provide a foundation for further development and optimization of a CDSS. Moreover, it is important to consider clinicians' specialization levels and years of practice for effective and personalized support.

Complex decision making in an intensive care environment: Perceived practice versus observed reality

RATIONALE

Advancing our understanding of how decisions are made in cognitively, socially and technologically complex hospital environments may reveal opportunities to improve healthcare delivery, medical education and the experience of patients, families and clinicians.

AIMS AND OBJECTIVES

Explore factors impacting clinician decision making in the Boston Children's Hospital Cardiac Intensive Care Unit.

METHODS

A convergent mixed methods design was used. Quantitative and qualitative data sources consisted of a faculty survey, direct observations of clinical rounds in a specific patient population identified by a clinical decision support system (CDSS) and semistructured interviews (SSIs). Deductive and inductive coding was used for qualitative data. Qualitative data were translated into images using social network analysis which illustrate the frequency and connectivity of the codes in each data set.

RESULTS

A total of 25 observations of eight faculty-led interprofessional teams were performed between 12 February and 31 March 2021. Individual patient characteristics were noted by faculty in SSIs to be the most important factor in their decision making, yet ethnographic observations suggested faculty cognitive traits, team expertise and value-based decisions were more heavily weighted. The development of expertise was impacted by role modeling. Decisions were perceived to be influenced by the system and environment.

CONCLUSIONS

Clinician perception of decision making was not congruent with the observed behaviours in a complicated and dynamic system. This study identifies important considerations in clinical curricula as well as the design and implementation of CDSS. Our method of using social network analysis to visualize components of decision making could be adopted to explore other complex environments.

Inpatient nurses' preferences and decisions with risk information visualization

OBJECTIVE

We examined the influence of 4 different risk information formats on inpatient nurses' preferences and decisions with an acute clinical deterioration decision-support system.

MATERIALS AND METHODS

We conducted a comparative usability evaluation in which participants provided responses to multiple user interface options in a simulated setting. We collected qualitative data using think aloud methods. We collected quantitative data by asking participants which action they would perform after each time point in 3 different patient scenarios.

RESULTS

More participants (n = 6) preferred the probability format over relative risk ratios (n = 2), absolute differences (n = 2), and number of persons out of 100 (n = 0). Participants liked average lines, having a trend graph to supplement the risk estimate, and consistent colors between trend graphs and possible actions. Participants did not like too much text information or the presence of confidence intervals. From a decision-making perspective, use of the probability format was associated with greater concordance in actions taken by participants compared to the other 3 risk information formats.

DISCUSSION

By focusing on nurses' preferences and decisions with several risk information display formats and collecting both qualitative and quantitative data, we have provided meaningful insights for the design of clinical decision-support systems containing complex quantitative information.

CONCLUSION

This study adds to our knowledge of presenting risk information to nurses within clinical decision-support systems. We encourage those developing risk-based systems for inpatient nurses to consider expressing risk in a probability format and include a graph (with average line) to display the patient's recent trends.

Effect of knowledgebase transition of a clinical decision support system on medication order and alert patterns in an emergency department

A knowledgebase (KB) transition of a clinical decision support (CDS) system occurred at the study site. The transition was made from one commercial database to another, provided by a different vendor. The change was applied to all medications in the institute. The aim of this study was to analyze the effect of KB transition on medication-related orders and alert patterns in an emergency department (ED). Data of patients, medication-related orders and alerts, and physicians in the ED from January 2018 to December 2020 were analyzed in this study. A set of definitions was set to define orders, alerts, and alert overrides. Changes in order and alert patterns before and after the conversion, which took place in May 2019, were assessed. Overall, 101,450 patients visited the ED, and 1325 physicians made 829,474 prescription orders to patients during visit and at discharge. Alert rates (alert count divided by order count) for periods A and B were 12.6% and 14.1%, and override rates (alert override count divided by alert count) were 60.8% and 67.4%, respectively. Of the 296 drugs that were used more than 100 times during each period, 64.5% of the drugs had an increase in alert rate after the transition. Changes in alert rates were tested using chi-squared test and Fisher's exact test. We found that the CDS system knowledgebase transition was associated with a significant change in alert patterns at the medication level in the ED. Careful consideration is advised when such a transition is performed.

Harnessing the power of clinical decision support systems: challenges and opportunities

Clinical decision support systems (CDSSs) are increasingly integrated into healthcare settings to improve patient outcomes, reduce medical errors and enhance clinical efficiency by providing clinicians with evidence-based recommendations at the point of care. However, the adoption and optimisation of these systems remain a challenge. This review aims to provide an overview of the current state of CDSS, discussing their development, implementation, benefits, limitations and future directions. We also explore the potential for enhancing their effectiveness and provide an outlook for future developments in this field. There are several challenges in CDSS implementation, including data privacy concerns, system integration and clinician acceptance. While CDSS have demonstrated significant potential, their adoption and optimisation remain a challenge.

Identifying the data elements and functionalities of clinical decision support systems to administer medication for neonates and pediatrics: a systematic literature review

BACKGROUND

Patient safety is a central healthcare policy worldwide. Adverse drug events (ADE) are among the main threats to patient safety. Children are at a higher risk of ADE in each stage of medication management process. ADE rate is high in the administration stage, as the final stage of preventing medication errors in pediatrics and neonates. The most effective way to reduce ADE rate is using medication administration clinical decision support systems (MACDSSs). The present study reviewed the literature on MACDSS for neonates and pediatrics. It identified and classified the data elements that mapped onto the Fast Healthcare Interoperability Resources (FHIR) standard and the functionalities of these systems to guide future research.

METHODS

PubMed/ MEDLINE, Embase, CINAHL, and ProQuest databases were searched from 1995 to June 31, 2021. Studies that addressed developing or applying medication administration software for neonates and pediatrics were included. Two authors reviewed the titles, abstracts, and full texts. The quality of eligible studies was assessed based on the level of evidence. The extracted data elements were mapped onto the FHIR standard.

RESULTS

In the initial search, 4,856 papers were identified. After removing duplicates, 3,761 titles, and abstracts were screened. Finally, 56 full-text papers remained for evaluation. The full-text review of papers led to the retention of 10 papers which met the eligibility criteria. In addition, two papers from the reference lists were included. A total number of 12 papers were included for analysis. Six papers were categorized as high-level evidence. Only three papers evaluated their systems in a real environment. A variety of data elements and functionalities could be observed. Overall, 84 unique data elements were extracted from the included papers. The analysis of reported functionalities showed that 18 functionalities were implemented in these systems.

CONCLUSION

Identifying the data elements and functionalities as a roadmap by developers can significantly improve MACDSS performance. Though many CDSSs have been developed for different medication processes in neonates and pediatrics, few have actually evaluated MACDSSs in reality. Therefore, further research is needed on the application and evaluation of MACDSSs in the real environment.

PROTOCOL REGISTRATION

(dx.doi.org/10.17504/protocols.io.bwbwpape).

Impact of model-informed precision dosing on achievement of vancomycin exposure targets in pediatric patients with cystic fibrosis

BACKGROUND

Vancomycin is commonly used to treat acute pulmonary exacerbations in pediatric patients with cystic fibrosis (CF) and a history of methicillin-resistant Staphylococcus aureus. Optimizing vancomycin exposure during therapy is essential and area under-the-curve (AUC)-guided dosing is now recommended. Model-informed precision dosing (MIPD) utilizing Bayesian forecasting is a powerful approach that can support AUC-guided dose individualization. The objective of the current study was to examine the impact of implementing an AUC-guided dose individualization approach supported via a MIPD clinical decision support (CDS) tool on vancomycin exposure, target attainment rate, and safety in pediatric patients with CF treated with vancomycin during clinical care.

METHODS

A retrospective chart review was performed in patients with CF at a single children's hospital comparing pre- and post-implementation of a MIPD approach for vancomycin supported by a cloud-based, CDS tool integrated into the electronic health record (EHR). In the pre-MIPD period, vancomycin starting doses of 60 mg/kg/day (<13 years) or 45 mg/kg/day (≥13 years) were used. Dose adjustment was guided by therapeutic drug monitoring (TDM) with a target trough 10-20 mg/L. In the post-MIPD period, starting dose and dose adjustment were based on the MIPD CDS tool predictions with a target 24 h AUC (AUC24 ) 400-600 mg*h/L. Exposure and target achievement rates were retrospectively calculated and compared. Rates of acute kidney injury (AKI) were also compared.

RESULTS

Overall, 23 patient courses were included in the pre-MIPD period and 21 patient courses in the post-MIPD period. In the post-MIPD period, an individualized MIPD starting dose resulted in 71% of patients achieving target AUC24 compared to 39% in the pre-MIPD period (p < 0.05). After the first TDM and dose adjustment, target AUC24 achievement was also higher post-MIPD versus pre-MIPD (86% vs. 57%; p < 0.05). AKI rates were low and similar between periods (pre-MIPD 8.7% vs. post-MIPD 9.5%; p = 0.9).

CONCLUSION

An MIPD approach implemented within a cloud-based, EHR-integrated CDS tool safely supported vancomycin AUC-guided dosing and resulted in high rates of target achievement.

Current evidence for designing self-management support for underserved populations: an integrative review using the example of diabetes

AIMS

With numerous and continuing attempts at adapting diabetes self-management support programmes to better account for underserved populations, its important that the lessons being learned are understood and shared. The work we present here reviews the latest evidence and best practice in designing and embedding culturally and socially sensitive, self-management support programmes.

METHODS

We explored the literature with regard to four key design considerations of diabetes self-management support programmes: Composition - the design and content of written materials and digital tools and interfaces; Structure - the combination of individual and group sessions, their frequency, and the overall duration of programmes; Facilitators - the combination of individuals used to deliver the programme; and Context - the influence and mitigation of a range of individual, socio-cultural, and environmental factors.

RESULTS

We found useful and recent examples of design innovation within a variety of countries and models of health care delivery including Brazil, Mexico, Netherlands, Spain, United Kingdom, and United States of America. Within Composition we confirmed the importance of retaining best practice in creating readily understood written information and intuitive digital interfaces; Structure the need to offer group, individual, and remote learning options in programmes of flexible duration and frequency; Facilitators where the benefits of using culturally concordant peers and community-based providers were described; and finally in Context the need to integrate self-management support programmes within existing health systems, and tailor their various constituent elements according to the language, resources, and beliefs of individuals and their communities.

CONCLUSIONS

A number of design principles across the four design considerations were identified that together offer a promising means of creating the next generation of self-management support programme more readily accessible for underserved communities. Ultimately, we recommend that the precise configuration should be co-produced by all relevant service and patient stakeholders and its delivery embedded in local health systems.

An Ontology-Based Approach to Improving Medication Appropriateness in Older Patients: Algorithm Development and Validation Study

Background: Inappropriate medication in older patients with multimorbidity results in a greater risk of adverse drug events. Clinical decision support systems (CDSSs) are intended to improve medication appropriateness. One approach to improving CDSSs is to use ontologies instead of relational databases. Previously, we developed OntoPharma-an ontology-based CDSS for reducing medication prescribing errors. Objective: The primary aim was to model a domain for improving medication appropriateness in older patients (chronic patient domain). The secondary aim was to implement the version of OntoPharma containing the chronic patient domain in a hospital setting. Methods: A 4-step process was proposed. The first step was defining the domain scope. The chronic patient domain focused on improving medication appropriateness in older patients. A group of experts selected the following three use cases: medication regimen complexity, anticholinergic and sedative drug burden, and the presence of triggers for identifying possible adverse events. The second step was domain model representation. The implementation was conducted by medical informatics specialists and clinical pharmacists using Protégé-OWL (Stanford Center for Biomedical Informatics Research). The third step was OntoPharma-driven alert module adaptation. We reused the existing framework based on SPARQL to query ontologies. The fourth step was implementing the version of OntoPharma containing the chronic patient domain in a hospital setting. Alerts generated from July to September 2022 were analyzed. Results: We proposed 6 new classes and 5 new properties, introducing the necessary changes in the ontologies previously created. An alert is shown if the Medication Regimen Complexity Index is ≥40, if the Drug Burden Index is ≥1, or if there is a trigger based on an abnormal laboratory value. A total of 364 alerts were generated for 107 patients; 154 (42.3%) alerts were accepted. Conclusions: We proposed an ontology-based approach to provide support for improving medication appropriateness in older patients with multimorbidity in a scalable, sustainable, and reusable way. The chronic patient domain was built based on our previous research, reusing the existing framework. OntoPharma has been implemented in clinical practice and generates alerts, considering the following use cases: medication regimen complexity, anticholinergic and sedative drug burden, and the presence of triggers for identifying possible adverse events.

Clinician acceptability of an antibiotic prescribing knowledge support system for primary care: a mixed-method evaluation of features and context

BACKGROUND

Overprescribing of antibiotics is a major concern as it contributes to antimicrobial resistance. Research has found highly variable antibiotic prescribing in (UK) primary care, and to support more effective stewardship, the BRIT Project (Building Rapid Interventions to optimise prescribing) is implementing an eHealth Knowledge Support System. This will provide unique individualised analytics information to clinicians and patients at the point of care. The objective of the current study was to gauge the acceptability of the system to prescribing healthcare professionals and highlight factors to maximise intervention uptake.

METHODS

Two mixed-method co-design workshops were held online with primary care prescribing healthcare professionals (n = 16). Usefulness ratings of example features were collected using online polls and online whiteboards. Verbal discussion and textual comments were analysed thematically using inductive (participant-centred) and deductive perspectives (using the Theoretical Framework of Acceptability).

RESULTS

Hierarchical thematic coding generated three overarching themes relevant to intervention use and development. Clinician concerns (focal issues) were safe prescribing, accessible information, autonomy, avoiding duplication, technical issues and time. Requirements were ease and efficiency of use, integration of systems, patient-centeredness, personalisation, and training. Important features of the system included extraction of pertinent information from patient records (such as antibiotic prescribing history), recommended actions, personalised treatment, risk indicators and electronic patient communication leaflets. Anticipated acceptability and intention to use the knowledge support system was moderate to high. Time was identified as a focal cost/ burden, but this would be outweighed if the system improved patient outcomes and increased prescribing confidence.

CONCLUSION

Clinicians anticipate that an eHealth knowledge support system will be a useful and acceptable way to optimise antibiotic prescribing at the point of care. The mixed method workshop highlighted issues to assist person-centred eHealth intervention development, such as the value of communicating patient outcomes. Important features were identified including the ability to efficiently extract and summarise pertinent information from the patient records, provide explainable and transparent risk information, and personalised information to support patient communication. The Theoretical Framework of Acceptability enabled structured, theoretically sound feedback and creation of a profile to benchmark future evaluations. This may encourage a consistent user-focused approach to guide future eHealth intervention development.

Adoption of shared decision-making and clinical decision support for reducing cardiovascular disease risk in community health centers

Objective

Electronic health record (EHR)-based shared decision-making (SDM) and clinical decision support (CDS) systems can improve cardiovascular disease (CVD) care quality and risk factor management. Use of the CV Wizard system showed a beneficial effect on high-risk community health center (CHC) patients' CVD risk within an effectiveness trial, but system adoption was low overall. We assessed which multi-level characteristics were associated with system use.

Materials and Methods

Analyses included 80 195 encounters with 17 931 patients with high CVD risk and/or uncontrolled risk factors at 42 clinics in September 2018-March 2020. Data came from the CV Wizard repository and EHR data, and a survey of 44 clinic providers. Adjusted, mixed-effects multivariate Poisson regression analyses assessed factors associated with system use. We included clinic- and provider-level clustering as random effects to account for nested data.

Results

Likelihood of system use was significantly higher in encounters with patients with higher CVD risk and at longer encounters, and lower when providers were >10 minutes behind schedule, among other factors. Survey participants reported generally high satisfaction with the system but were less likely to use it when there were time constraints or when rooming staff did not print the system output for the provider.

Discussion

CHC providers prioritize using this system for patients with the greatest CVD risk, when time permits, and when rooming staff make the information readily available. CHCs' financial constraints create substantial challenges to addressing barriers to improved system use, with health equity implications.

Conclusion

Research is needed on improving SDM and CDS adoption in CHCs.

Trial Registration

ClinicalTrials.gov, NCT03001713, https://clinicaltrials.gov/.

Examining clinician choice to follow-up (or not) on automated notifications of medication non-adherence by clinical decision support systems

BACKGROUND

Maintaining medication adherence can be challenging for people living with mental ill-health. Clinical decision support systems (CDSS) based on automated detection of problematic patterns in Electronic Health Records (EHRs) have the potential to enable early intervention into non-adherence events ("flags") through suggesting evidence-based courses of action. However, extant literature shows multiple barriers-perceived lack of benefit in following up low-risk cases, veracity of data, human-centric design concerns, etc.-to clinician follow-up in real-world settings. This study examined patterns in clinician decision making behaviour related to follow-up of non-adherence prompts within a community mental health clinic.

METHODS

The prompts for follow-up, and the recording of clinician responses, were enabled by CDSS software (AI²). De-identified clinician notes recorded after reviewing a prompt were analysed using a thematic synthesis approach-starting with descriptions of clinician comments, then sorting into analytical themes related to design and, in parallel, a priori categories describing follow-up behaviours. Hypotheses derived from the literature about the follow-up categories' relationships with client and medication-subtype characteristics were tested.

RESULTS

The majority of clients were Not Followed-up (n = 260; 78%; Followed-up: n = 71; 22%). The analytical themes emerging from the decision notes suggested contextual factors-the clients' environment, their clinical relationships, and medical needs-mediated how clinicians interacted with the CDSS flags. Significant differences were found between medication subtypes and follow-up, with Anti-depressants less likely to be followed up than Anti-Psychotics and Anxiolytics (χ² = 35.196, 44.825; p < 0.001; v = 0.389, 0.499); and between the time taken to action Followed-up₀ and Not-followed up₁ flags (M₀ = 31.78; M₁ = 45.55; U = 12,119; p < 0.001; η² = .05).

CONCLUSION

These analyses encourage actively incorporating the input of consumers and carers, non-EHR data streams, and better incorporation of data from parallel health systems and other clinicians into CDSS designs to encourage follow-up.

An Accessible Clinical Decision Support System to Curtail Anesthetic Greenhouse Gases in a Large Health Network: Implementation Study

BACKGROUND

Inhaled anesthetics in the operating room are potent greenhouse gases and are a key contributor to carbon emissions from health care facilities. Real-time clinical decision support (CDS) systems lower anesthetic gas waste by prompting anesthesia professionals to reduce fresh gas flow (FGF) when a set threshold is exceeded. However, previous CDS systems have relied on proprietary or highly customized anesthesia information management systems, significantly reducing other institutions' accessibility to the technology and thus limiting overall environmental benefit.

OBJECTIVE

In 2018, a CDS system that lowers anesthetic gas waste using methods that can be easily adopted by other institutions was developed at the University of California San Francisco (UCSF). This study aims to facilitate wider uptake of our CDS system and further reduce gas waste by describing the implementation of the FGF CDS toolkit at UCSF and the subsequent implementation at other medical campuses within the University of California Health network.

METHODS

We developed a noninterruptive active CDS system to alert anesthesia professionals when FGF rates exceeded 0.7 L per minute for common volatile anesthetics. The implementation process at UCSF was documented and assembled into an informational toolkit to aid in the integration of the CDS system at other health care institutions. Before implementation, presentation-based education initiatives were used to disseminate information regarding the safety of low FGF use and its relationship to environmental sustainability. Our FGF CDS toolkit consisted of 4 main components for implementation: sustainability-focused education of anesthesia professionals, hardware integration of the CDS technology, software build of the CDS system, and data reporting of measured outcomes.

RESULTS

The FGF CDS system was successfully deployed at 5 University of California Health network campuses. Four of the institutions are independent from the institution that created the CDS system. The CDS system was deployed at each facility using the FGF CDS toolkit, which describes the main components of the technology and implementation. Each campus made modifications to the CDS tool to best suit their institution, emphasizing the versatility and adoptability of the technology and implementation framework.

CONCLUSIONS

It has previously been shown that the FGF CDS system reduces anesthetic gas waste, leading to environmental and fiscal benefits. Here, we demonstrate that the CDS system can be transferred to other medical facilities using our toolkit for implementation, making the technology and associated benefits globally accessible to advance mitigation of health care-related emissions.

Risk assessment tools to predict postpartum hemorrhage

Postpartum hemorrhage (PPH) is a leading cause of maternal morbidity and mortality, and accurate risk assessments may allow providers to anticipate and prevent serious hemorrhage-related adverse events. Multiple category-based tools have been developed by national societies through expert consensus, and these tools assign low, medium, or high risk of hemorrhage based on a review of each patient's risk factors. Validation studies of these tools show varying performance, with a wide range of positive and negative predictive values. Risk prediction models for PPH have been developed and studied, and these models offer the advantage of more nuanced and individualized prediction. However, there are no published studies demonstrating external validation or successful clinical use of such models. Future work should include refinement of these models, study of best practices for implementation, and ultimately linkage of prediction to improved patient outcomes.

OntoPharma: ontology based clinical decision support system to reduce medication prescribing errors

Background

Clinical decision support systems (CDSS) have been shown to reduce medication errors. However, they are underused because of different challenges. One approach to improve CDSS is to use ontologies instead of relational databases. The primary aim was to design and develop OntoPharma, an ontology based CDSS to reduce medication prescribing errors. Secondary aim was to implement OntoPharma in a hospital setting.

Methods

A four-step process was proposed. (1) Defining the ontology domain. The ontology scope was the medication domain. An advisory board selected four use cases: maximum dosage alert, drug-drug interaction checker, renal failure adjustment, and drug allergy checker. (2) Implementing the ontology in a formal representation. The implementation was conducted by Medical Informatics specialists and Clinical Pharmacists using Protégé-OWL. (3) Developing an ontology-driven alert module. Computerised Physician Order Entry (CPOE) integration was performed through a REST API. SPARQL was used to query ontologies. (4) Implementing OntoPharma in a hospital setting. Alerts generated between July 2020/ November 2021 were analysed.

Results

The three ontologies developed included 34,938 classes, 16,672 individuals and 82 properties. The domains addressed by ontologies were identification data of medicinal products, appropriateness drug data, and local concepts from CPOE. When a medication prescribing error is identified an alert is shown. OntoPharma generated 823 alerts in 1046 patients. 401 (48.7%) of them were accepted.

Conclusions

OntoPharma is an ontology based CDSS implemented in clinical practice which generates alerts when a prescribing medication error is identified. To gain user acceptance OntoPharma has been designed and developed by a multidisciplinary team. Compared to CDSS based on relational databases, OntoPharma represents medication knowledge in a more intuitive, extensible and maintainable manner.

Barriers and enablers to implementing and using clinical decision support systems for chronic diseases: a qualitative systematic review and meta-aggregation

BACKGROUND

Clinical decision support (CDS) is increasingly used to facilitate chronic disease care. Despite increased availability of electronic health records and the ongoing development of new CDS technologies, uptake of CDS into routine clinical settings is inconsistent. This qualitative systematic review seeks to synthesise healthcare provider experiences of CDS-exploring the barriers and enablers to implementing, using, evaluating, and sustaining chronic disease CDS systems.

METHODS

A search was conducted in Medline, CINAHL, APA PsychInfo, EconLit, and Web of Science from 2011 to 2021. Primary research studies incorporating qualitative findings were included if they targeted healthcare providers and studied a relevant chronic disease CDS intervention. Relevant CDS interventions were electronic health record-based and addressed one or more of the following chronic diseases: cardiovascular disease, diabetes, chronic kidney disease, hypertension, and hypercholesterolaemia. Qualitative findings were synthesised using a meta-aggregative approach.

RESULTS

Thirty-three primary research articles were included in this qualitative systematic review. Meta-aggregation of qualitative data revealed 177 findings and 29 categories, which were aggregated into 8 synthesised findings. The synthesised findings related to clinical context, user, external context, and technical factors affecting CDS uptake. Key barriers to uptake included CDS systems that were simplistic, had limited clinical applicability in multimorbidity, and integrated poorly into existing workflows. Enablers to successful CDS interventions included perceived usefulness in providing relevant clinical knowledge and structured chronic disease care; user confidence gained through training and post training follow-up; external contexts comprised of strong clinical champions, allocated personnel, and technical support; and CDS technical features that are both highly functional, and attractive.

CONCLUSION

This systematic review explored healthcare provider experiences, focussing on barriers and enablers to CDS use for chronic diseases. The results provide an evidence-base for designing, implementing, and sustaining future CDS systems. Based on the findings from this review, we highlight actionable steps for practice and future research.

TRIAL REGISTRATION

PROSPERO CRD42020203716.

Evaluating a Prototype Clinical Decision Support Tool for Chronic Pain Treatment in Primary Care

OBJECTIVES

 The Chronic Pain Treatment Tracker (Tx Tracker) is a prototype decision support tool to aid primary care clinicians when caring for patients with chronic noncancer pain. This study evaluated clinicians' perceived utility of Tx Tracker in meeting information needs and identifying treatment options, and preferences for visual design.

METHODS

 We conducted 12 semi-structured interviews with primary care clinicians from four health systems in Indiana. The interviews were conducted in two waves, with prototype and interview guide revisions after the first six interviews. The interviews included exploration of Tx Tracker using a think-aloud approach and a clinical scenario. Clinicians were presented with a patient scenario and asked to use Tx Tracker to make a treatment recommendation. Last, participants answered several evaluation questions. Detailed field notes were collected, coded, and thematically analyzed by four analysts.

RESULTS

 We identified several themes: the need for clinicians to be presented with a comprehensive patient history, the usefulness of Tx Tracker in patient discussions about treatment planning, potential usefulness of Tx Tracker for patients with high uncertainty or risk, potential usefulness of Tx Tracker in aggregating scattered information, variability in expectations about workflows, skepticism about underlying electronic health record data quality, interest in using Tx Tracker to annotate or update information, interest in using Tx Tracker to translate information to clinical action, desire for interface with visual cues for risks, warnings, or treatment options, and desire for interactive functionality.

CONCLUSION

 Tools like Tx Tracker, by aggregating key information about past, current, and potential future treatments, may help clinicians collaborate with their patients in choosing the best pain treatments. Still, the use and usefulness of Tx Tracker likely relies on continued improvement of its functionality, accurate and complete underlying data, and tailored integration with varying workflows, care team roles, and user preferences.

Development and assessment of PharmaCheck: an electronic screening tool for the prevention of twenty major adverse drug events

BACKGROUND

Adverse drug events (ADEs) can be prevented by deploying clinical decision support systems (CDSS) that directly assist physicians, via computerized order entry systems, and clinical pharmacists performing medication reviews as part of medical rounds. However, physicians using CDSS are known to be exposed to the alert-fatigue phenomenon. Our study aimed to assess the performance of PharmaCheck-a CDSS to help clinical pharmacists detect high-risk situations with the potential to lead to ADEs-and its impact on clinical pharmacists' activities.

METHODS

Twenty clinical rules, divided into four risk classes, were set for the daily screening of high-risk situations in the electronic health records of patients admitted to our General Internal Medicine Department. Alerts to clinical pharmacists encouraged them to telephone prescribers and suggest any necessary treatment adjustments. PharmaCheck's performance was assessed using the intervention's positive predictive value (PPV), which characterizes the proportion of interventions for each alert triggered. PharmaCheck's impact was assessed by considering clinical pharmacists as a filter for ruling out futile alerts and by comparing the final clinical PPV with a pharmacist (the proportion of interventions that led to a change in the medical regimen) to the final clinical PPV without a pharmacist.

RESULTS

Over 132 days, 447 alerts were triggered for 383 patients, leading to 90 interventions (overall intervention PPV = 20.1%). By risk class, intervention PPVs made up 26.9% (n = 65/242) of abnormal laboratory value alerts, 3.1% (4/127) of alerts for contraindicated medications or medications to be used with caution, 28.2% (20/71) of drug-drug interaction alerts, and 14.3% (1/7) of inadequate mode of administration alerts. Clinical PPVs reached 71.0% (64/90) when pharmacists filtered alerts and 14% (64/242) if they were not doing it.

CONCLUSION

PharmaCheck enabled clinical pharmacists to improve their traditional processes and broaden their coverage by focusing on 20 high-risk situations. Alert management by pharmacists seemed to be a more effective way of preventing risky situations and alert-fatigue than a model addressing alerts to physicians exclusively. Some fine-tuning could enhance PharmaCheck's performance by considering the information quality of triggers, the variability of clinical settings, and the fact that some prescription processes are already highly secured.

Computerized Clinical Decision Support Systems for the Early Detection of Sepsis Among Pediatric, Neonatal, and Maternal Inpatients: Scoping Review

BACKGROUND

Sepsis is a severe condition associated with extensive morbidity and mortality worldwide. Pediatric, neonatal, and maternal patients represent a considerable proportion of the sepsis burden. Identifying sepsis cases as early as possible is a key pillar of sepsis management and has prompted the development of sepsis identification rules and algorithms that are embedded in computerized clinical decision support (CCDS) systems.

OBJECTIVE

This scoping review aimed to systematically describe studies reporting on the use and evaluation of CCDS systems for the early detection of pediatric, neonatal, and maternal inpatients at risk of sepsis.

METHODS

MEDLINE, Embase, CINAHL, Cochrane, Latin American and Caribbean Health Sciences Literature (LILACS), Scopus, Web of Science, OpenGrey, ClinicalTrials.gov, and ProQuest Dissertations and Theses Global (PQDT) were searched by using a search strategy that incorporated terms for sepsis, clinical decision support, and early detection. Title, abstract, and full-text screening was performed by 2 independent reviewers, who consulted a third reviewer as needed. One reviewer performed data charting with a sample of data. This was checked by a second reviewer and via discussions with the review team, as necessary.

RESULTS

A total of 33 studies were included in this review-13 (39%) pediatric studies, 18 (55%) neonatal studies, and 2 (6%) maternal studies. All studies were published after 2011, and 27 (82%) were published from 2017 onward. The most common outcome investigated in pediatric studies was the accuracy of sepsis identification (9/13, 69%). Pediatric CCDS systems used different combinations of 18 diverse clinical criteria to detect sepsis across the 13 identified studies. In neonatal studies, 78% (14/18) of the studies investigated the Kaiser Permanente early-onset sepsis risk calculator. All studies investigated sepsis treatment and management outcomes, with 83% (15/18) reporting on antibiotics-related outcomes. Usability and cost-related outcomes were each reported in only 2 (6%) of the 31 pediatric or neonatal studies. Both studies on maternal populations were short abstracts.

CONCLUSIONS

This review found limited research investigating CCDS systems to support the early detection of sepsis among pediatric, neonatal, and maternal patients, despite the high burden of sepsis in these vulnerable populations. We have highlighted the need for a consensus definition for pediatric and neonatal sepsis and the study of usability and cost-related outcomes as critical areas for future research.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/24899.

Modulators Influencing Medication Alert Acceptance: An Explorative Review

OBJECTIVES

Clinical decision support systems (CDSSs) use alerts to enhance medication safety and reduce medication error rates. A major challenge of medication alerts is their low acceptance rate, limiting their potential benefit. A structured overview about modulators influencing alert acceptance is lacking. Therefore, we aimed to review and compile qualitative and quantitative modulators of alert acceptance and organize them in a comprehensive model.

METHODS

In accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guideline, a literature search in PubMed was started in February 2018 and continued until October 2021. From all included articles, qualitative and quantitative parameters and their impact on alert acceptance were extracted. Related parameters were then grouped into factors, allocated to superordinate determinants, and subsequently further allocated into five categories that were already known to influence alert acceptance.

RESULTS

Out of 539 articles, 60 were included. A total of 391 single parameters were extracted (e.g., patients' comorbidity) and grouped into 75 factors (e.g., comorbidity), and 25 determinants (e.g., complexity) were consequently assigned to the predefined five categories, i.e., CDSS, care provider, patient, setting, and involved drug. More than half of all factors were qualitatively assessed (n = 21) or quantitatively inconclusive (n = 19). Furthermore, 33 quantitative factors clearly influenced alert acceptance (positive correlation: e.g., alert type, patients' comorbidity; negative correlation: e.g., number of alerts per care provider, moment of alert display in the workflow). Two factors (alert frequency, laboratory value) showed contradictory effects, meaning that acceptance was significantly influenced both positively and negatively by these factors, depending on the study. Interventional studies have been performed for only 12 factors while all other factors were evaluated descriptively.

CONCLUSION

This review compiles modulators of alert acceptance distinguished by being studied quantitatively or qualitatively and indicates their effect magnitude whenever possible. Additionally, it describes how further research should be designed to comprehensively quantify the effect of alert modulators.

Computerized Clinical Decision Support Systems for the Early Detection of Sepsis Among Adult Inpatients: Scoping Review

Background

Sepsis is a significant cause of morbidity and mortality worldwide. Early detection of sepsis followed promptly by treatment initiation improves patient outcomes and saves lives. Hospitals are increasingly using computerized clinical decision support (CCDS) systems for the rapid identification of adult patients with sepsis.

Objective

This scoping review aims to systematically describe studies reporting on the use and evaluation of CCDS systems for the early detection of adult inpatients with sepsis.

Methods

The protocol for this scoping review was previously published. A total of 10 electronic databases (MEDLINE, Embase, CINAHL, the Cochrane database, LILACS [Latin American and Caribbean Health Sciences Literature], Scopus, Web of Science, OpenGrey, ClinicalTrials.gov, and PQDT [ProQuest Dissertations and Theses]) were comprehensively searched using terms for sepsis, CCDS, and detection to identify relevant studies. Title, abstract, and full-text screening were performed by 2 independent reviewers using predefined eligibility criteria. Data charting was performed by 1 reviewer with a second reviewer checking a random sample of studies. Any disagreements were discussed with input from a third reviewer. In this review, we present the results for adult inpatients, including studies that do not specify patient age.

Results

A search of the electronic databases retrieved 12,139 studies following duplicate removal. We identified 124 studies for inclusion after title, abstract, full-text screening, and hand searching were complete. Nearly all studies (121/124, 97.6%) were published after 2009. Half of the studies were journal articles (65/124, 52.4%), and the remainder were conference abstracts (54/124, 43.5%) and theses (5/124, 4%). Most studies used a single cohort (54/124, 43.5%) or before-after (42/124, 33.9%) approach. Across all 124 included studies, patient outcomes were the most frequently reported outcomes (107/124, 86.3%), followed by sepsis treatment and management (75/124, 60.5%), CCDS usability (14/124, 11.3%), and cost outcomes (9/124, 7.3%). For sepsis identification, the systemic inflammatory response syndrome criteria were the most commonly used, alone (50/124, 40.3%), combined with organ dysfunction (28/124, 22.6%), or combined with other criteria (23/124, 18.5%). Over half of the CCDS systems (68/124, 54.8%) were implemented alongside other sepsis-related interventions.

Conclusions

The current body of literature investigating the implementation of CCDS systems for the early detection of adult inpatients with sepsis is extremely diverse. There is substantial variability in study design, CCDS criteria and characteristics, and outcomes measured across the identified literature. Future research on CCDS system usability, cost, and impact on sepsis morbidity is needed.

International Registered Report Identifier (IRRID)

RR2-10.2196/24899

Encouraging Behavior Changes and Preventing Cardiovascular Diseases Using the Prevent Connect Mobile Health App: Conception and Evaluation of App Quality

Background

Cardiovascular diseases are a major cause of death worldwide. Mobile health apps could help in preventing cardiovascular diseases by improving modifiable risk factors such as eating habits, physical activity levels, and alcohol or tobacco consumption.

Objective

The aim of this study was to design a mobile health app, Prevent Connect, and to assess its quality for (1) assessing patient behavior for 4 cardiovascular risk factors (unhealthy eating, sedentary lifestyle, alcohol, and tobacco consumption) and (2) suggesting personalized recommendations and mobile health interventions for risky behaviors.

Methods

The knowledge base of the app is based on French national recommendations for healthy eating, physical activity, and limiting alcohol and tobacco consumption. It contains a list of patient behaviors and related personalized recommendations and digital health interventions. The interface was designed according to usability principles. Its quality was assessed by a panel of 52 users in a 5-step process: completion of the demographic form, visualization of a short presentation of the app, testing of the app, completion of the user version of the Mobile App Rating Scale (uMARS), and an open group discussion.

Results

This app assesses patient behaviors through specific questionnaires about 4 risk factors (unhealthy eating, sedentary lifestyle, alcohol, and tobacco consumption) and suggests personalized recommendations and digital health interventions for improving behavior. The app was deemed to be of good quality, with a mean uMARS quality score of 4 on a 5-point Likert scale. The functionality and information content of the app were particularly appreciated, with a mean uMARS score above 4. Almost all the study participants appreciated the navigation system and found the app easy to use. More than three-quarters of the study participants found the app content relevant, concise, and comprehensive. The aesthetics and the engagement of the app were also appreciated (uMARS score, 3.7). Overall, 80% (42/52) of the study participants declared that the app helped them to become aware of the importance of addressing health behavior, and 65% (34/52) said that the app helped motivate them to change lifestyle habits.

Conclusions

The app assessed the risky behaviors of the patients and delivered personalized recommendations and digital health interventions for multiple risk factors. The quality of the app was considered to be good, but the impact of the app on behavior changes is yet to be demonstrated and will be assessed in further studies.

Clinical decision support for intervention reduction in neonatal patients: A usability assessment

Objective

This study investigated how effectively simplified cognitive walkthroughs, performed independently by four nonclinical researchers, can be used to assess the usability of clinical decision support software. It also helped illuminate the types of usability issues in clinical decision support software tools that cognitive walkthroughs can identify.

Method

A human factors professor and three research assistants each conducted an independent cognitive walkthrough of a web-based demonstration version of T3, a physiologic monitoring system featuring a new clinical decision support software tool called MAnagement Application (MAP). They accessed the demo on personal computers in their homes and used it to walk through several pre-specified tasks, answering three standard questions at each step. Then they met to review and prioritize the findings.

Results

Evaluators acknowledged several positive features including concise, helpful tooltips and an informative column in the patient overview which allows users direct (one-click) access to protocol eligibility and compliance criteria. Recommendations to improve usability include: modify the language to clarify what user actions are possible; visually indicate when eligibility flags are snoozed; and specify which protocol's data is currently being shown.

Conclusion

Independent, simplified cognitive walkthroughs can help ensure that clinical decision support software tools will appropriately support clinicians. Four researchers used this technique to quickly, inexpensively, and effectively assess T3's new MAP tool, which suggests positive actions, such as removing a patient from a ventilator. Results indicate that, while there is room for usability improvements, the MAP tool may help reduce clinician's cognitive load, facilitating improved care. The study also confirmed that cognitive walkthroughs identify issues that make clinical decision support software hard to learn or remember to use.

Development of a Perioperative Medication-Related Clinical Decision Support Tool to Prevent Medication Errors: An Analysis of User Feedback

OBJECTIVES

 Medication use in the perioperative setting presents many patient safety challenges that may be improved with electronic clinical decision support (CDS). The objective of this paper is to describe the development and analysis of user feedback for a robust, real-time medication-related CDS application designed to provide patient-specific dosing information and alerts to warn of medication errors in the operating room (OR).

METHODS

 We designed a novel perioperative medication-related CDS application in four phases: (1) identification of need, (2) alert algorithm development, (3) system design, and (4) user interface design. We conducted group and individual design feedback sessions with front-line clinician leaders and subject matter experts to gather feedback about user requirements for alert content and system usability. Participants were clinicians who provide anesthesia (attending anesthesiologists, nurse anesthetists, and house staff), OR pharmacists, and nurses.

RESULTS

 We performed two group and eight individual design feedback sessions, with a total of 35 participants. We identified 20 feedback themes, corresponding to 19 system changes. Key requirements for user acceptance were: Use hard stops only when necessary; provide as much information as feasible about the rationale behind alerts and patient/clinical context; and allow users to edit fields such as units, time, and baseline values (e.g., baseline blood pressure).

CONCLUSION

 We incorporated user-centered design principles to build a perioperative medication-related CDS application that uses real-time patient data to provide patient-specific dosing information and alerts. Emphasis on early user involvement to elicit user requirements, workflow considerations, and preferences during application development can result in time and money efficiencies and a safer and more usable system.

Introduction of Systematized Nomenclature of Medicine-Clinical Terms Coding Into an Electronic Health Record and Evaluation of its Impact: Qualitative and Quantitative Study

BACKGROUND

This study describes the conversion within an existing electronic health record (EHR) from the International Classification of Diseases, Tenth Revision coding system to the SNOMED-CT (Systematized Nomenclature of Medicine-Clinical Terms) for the collection of patient histories and diagnoses. The setting is a large acute hospital that is designing and building its own EHR. Well-designed EHRs create opportunities for continuous data collection, which can be used in clinical decision support rules to drive patient safety. Collected data can be exchanged across health care systems to support patients in all health care settings. Data can be used for research to prevent diseases and protect future populations.

OBJECTIVE

The aim of this study was to migrate a current EHR, with all relevant patient data, to the SNOMED-CT coding system to optimize clinical use and clinical decision support, facilitate data sharing across organizational boundaries for national programs, and enable remodeling of medical pathways.

METHODS

The study used qualitative and quantitative data to understand the successes and gaps in the project, clinician attitudes toward the new tool, and the future use of the tool.

RESULTS

The new coding system (tool) was well received and immediately widely used in all specialties. This resulted in increased, accurate, and clinically relevant data collection. Clinicians appreciated the increased depth and detail of the new coding, welcomed the potential for both data sharing and research, and provided extensive feedback for further development.

CONCLUSIONS

Successful implementation of the new system aligned the University Hospitals Birmingham NHS Foundation Trust with national strategy and can be used as a blueprint for similar projects in other health care settings.

Machine learning in medicine: It has arrived, let's embrace it

Machine learning and artificial intelligence (AI) have arrived in medicine and the healthcare community is experiencing significant growth in their adoption across numerous patient care settings. There are countless applications for machine learning and AI in medicine ranging from patient outcome prediction, to clinical decision support, to predicting future patient therapeutic setpoints. This commentary discusses a recent application leveraging machine learning to predict one-year patient survival following orthotopic heart transplantation. This modeling approach has significant implications in terms of improving clinical decision-making, patient counseling, and ultimately organ allocation and has been shown to significantly outperform pre-existing algorithms. This commentary also discusses how adoption and advancement of this modeling approach in the future can provide increased personalization of patient care. The continued expansion of information systems and growth of electronic patient data sources in health care will continue to pave the way for increased use and adoption of data science in medicine. Personalized medicine has been a long-standing goal of the healthcare community and with machine learning and AI now being continually incorporated into clinical settings and practice, this technology is well on the pathway to make a considerable impact to greatly improve patient care in the near future.

Pharmacogenomic Clinical Decision Support: A Review, How-to Guide, and Future Vision

Clinical decision support (CDS) is an essential part of any pharmacogenomics (PGx) implementation. Increasingly, institutions have implemented CDS tools in the clinical setting to bring PGx data into patient care, and several have published their experiences with these implementations. However, barriers remain that limit the ability of some programs to create CDS tools to fit their PGx needs. Therefore, the purpose of this review is to summarize the types, functions, and limitations of PGx CDS currently in practice. Then, we provide an approachable step‐by‐step how‐to guide with a case example to help implementers bring PGx to the front lines of care regardless of their setting. Particular focus is paid to the five “rights” of CDS as a core around designing PGx CDS tools. Finally, we conclude with a discussion of opportunities and areas of growth for PGx CDS.

Candidemia Risk Prediction (CanDETEC) Model for Patients With Malignancy: Model Development and Validation in a Single-Center Retrospective Study

BACKGROUND

Appropriate empirical treatment for candidemia is associated with reduced mortality; however, the timely diagnosis of candidemia in patients with sepsis remains poor.

OBJECTIVE

We aimed to use machine learning algorithms to develop and validate a candidemia prediction model for patients with cancer.

METHODS

We conducted a single-center retrospective study using the cancer registry of a tertiary academic hospital. Adult patients diagnosed with malignancies between January 2010 and December 2018 were included. Our study outcome was the prediction of candidemia events. A stratified undersampling method was used to extract control data for algorithm learning. Multiple models were developed-a combination of 4 variable groups and 5 algorithms (auto-machine learning, deep neural network, gradient boosting, logistic regression, and random forest). The model with the largest area under the receiver operating characteristic curve (AUROC) was selected as the Candida detection (CanDETEC) model after comparing its performance indexes with those of the Candida Score Model.

RESULTS

From a total of 273,380 blood cultures from 186,404 registered patients with cancer, we extracted 501 records of candidemia events and 2000 records as control data. Performance among the different models varied (AUROC 0.771- 0.889), with all models demonstrating superior performance to that of the Candida Score (AUROC 0.677). The random forest model performed the best (AUROC 0.889, 95% CI 0.888-0.889); therefore, it was selected as the CanDETEC model.

CONCLUSIONS

The CanDETEC model predicted candidemia in patients with cancer with high discriminative power. This algorithm could be used for the timely diagnosis and appropriate empirical treatment of candidemia.

Understanding User Acceptance of Clinical Decision Support Systems to Promote Increased Cancer Screening Rates in a Primary Care Practice

OBJECTIVE

Clinical decision support systems (CDDSs) in the electronic medical record (EMR) have been implemented in primary care settings to identify patients due for cancer screening tests, while functioning as a real time reminder system. There is little known about primary care providers (PCPs) perspective or user acceptance of CDSS. The purpose of this study was to investigate primary care provider perceptions of utilizing CDSS alerts in the EMR to promote increased screening rates for breast cancer, cervical cancer, and colorectal cancer.

METHODS

An electronic survey was administered to PCPs in a Midwest Health Institution community internal medicine practice from September 25, 2019 through November 27, 2019.

RESULTS

Among 37 participants (9 NP/Pas and 28 MD/DOs), the NP/PA group was more likely to agree that alerts were helpful (50%; P-value = .0335) and the number of alerts (89%; P = .0227) in the EMR was appropriate. The NP/PA group also was more likely to find alerts straightforward to use (78%, P = .0239). Both groups agreed about feeling comfortable using the health maintenance alerts (MD/DO = 79%; NP/PA = 100%).

CONCLUSION

CDSSs can promote and facilitate ordering of cancer screening tests. The use of technology can promptly identify patients due for a test and act as a reminder to the PCP. PCPs identify these alerts to be a beneficial tool in the EMR when they do not interrupt workflow and provide value to patient care. More work is needed to identify factors that could optimize alerts to be even more helpful, particularly to MD/DO groups.

A Web Interface for Antibiotic Prescription Recommendations in Primary Care: User-Centered Design Approach

BACKGROUND

Antibiotic misuse is a serious public health problem worldwide. National health authorities release clinical practice guidelines (CPGs) to guide general practitioners (GPs) in their choice of antibiotics. However, despite the large-scale dissemination of CPGs, GPs continue to prescribe antibiotics that are not recommended as first-line treatments. This nonadherence to recommendations may be due to GPs misunderstanding the CPGs. A web interface displaying antibiotic prescription recommendations and their justifications could help to improve the comprehensibility and readability of CPGs, thereby increasing the adoption of recommendations regarding antibiotic treatment.

OBJECTIVE

This study aims to design and evaluate a web interface for antibiotic prescription displaying both the recommended antibiotics and their justifications in the form of antibiotic properties.

METHODS

A web interface was designed according to the same principles as e-commerce interfaces and was assessed by 117 GPs. These GPs were asked to answer 17 questions relating to the usefulness, user-friendliness, and comprehensibility and readability of the interface, and their satisfaction with it. Responses were recorded on a 4-point Likert scale (ranging from "absolutely disagree" to "absolutely agree"). At the end of the evaluation, the GPs were allowed to provide optional, additional free comments.

RESULTS

The antibiotic prescription web interface consists of three main sections: a clinical summary section, a filter section, and a recommended antibiotics section. The majority of GPs appreciated the clinical summary (90/117, 76.9%) and filter (98/117, 83.8%) sections, whereas 48.7% (57/117) of them reported difficulty reading some of the icons in the recommended antibiotics section. Overall, 82.9% (97/117) of GPs found the display of drug properties useful, and 65.8% (77/117) reported that the web interface improved their understanding of CPG recommendations.

CONCLUSIONS

The web interface displaying antibiotic recommendations and their properties can help doctors understand the rationale underlying CPG recommendations regarding antibiotic treatment, but further improvements are required before its implementation into a clinical decision support system.

User testing of a diagnostic decision support system with machine-assisted chart review to facilitate clinical genomic diagnosis

Objectives

There is a need in clinical genomics for systems that assist in clinical diagnosis, analysis of genomic information and periodic reanalysis of results, and can use information from the electronic health record to do so. Such systems should be built using the concepts of human-centred design, fit within clinical workflows and provide solutions to priority problems.

Methods

We adapted a commercially available diagnostic decision support system (DDSS) to use extracted findings from a patient record and combine them with genomic variant information in the DDSS interface. Three representative patient cases were created in a simulated clinical environment for user testing. A semistructured interview guide was created to illuminate factors relevant to human factors in CDS design and organisational implementation.

Results

Six individuals completed the user testing process. Tester responses were positive and noted good fit with real-world clinical genetics workflow. Technical issues related to interface, interaction and design were minor and fixable. Testers suggested solving issues related to terminology and usability through training and infobuttons. Time savings was estimated at 30%–50% and additional uses such as in-house clinical variant analysis were suggested for increase fit with workflow and to further address priority problems.

Conclusion

This study provides preliminary evidence for usability, workflow fit, acceptability and implementation potential of a modified DDSS that includes machine-assisted chart review. Continued development and testing using principles from human-centred design and implementation science are necessary to improve technical functionality and acceptability for multiple stakeholders and organisational implementation potential to improve the genomic diagnosis process.

Improving the care of term babies at risk of hypoglycaemia: A microsystem approach

AIM

Neonatal hypoglycaemia is a common problem, often requiring admission to the neonatal intensive care unit (NICU). Our aim was to reduce term admissions to NICU for hypoglycaemia by 50% over 4 years.

METHODS

Inborn term babies from 1 January 2015 to 31 December 2018 were included. Using quality-improvement methodology, we designed interventions based on human factors to incorporate best practice recommendations for babies at-risk of hypoglycaemia. This included standardisation of local guidelines, introduction of educational programmes to reiterate changes to practice and a multidisciplinary steering group to review term admissions to better understand the cause of failure of the maternal-neonatal pathway. The outcome measures were the number of term babies admitted to NICU for hypoglycaemia and the proportion of these babies not requiring intravenous (IV) dextrose. Run charts were used to monitor hypoglycaemia admissions and the impact of each intervention.

RESULTS

There was an overall reduction in the number of term babies admitted to NICU for hypoglycaemia from 36 babies in 2014 (baseline) to 5 babies in 2018. The percentage of babies admitted to the neonatal unit who did not require IV dextrose decreased from 22/36 (61%) in 2014 to 0/5 (0%) in 2018. Admissions from the delivery suite decreased from 21/36 (58%) to 1/5 (20%). There were no adverse outcomes observed in the period before or after the intervention.

CONCLUSIONS

We demonstrate a simple, cost-effective quality improvement project using fundamental human factors principles. This initiative successfully reduced the number of term admissions for hypoglycaemia over 4 years.

Medication safety alert fatigue may be reduced via interaction design and clinical role tailoring: a systematic review

OBJECTIVE

Alert fatigue limits the effectiveness of medication safety alerts, a type of computerized clinical decision support (CDS). Researchers have suggested alternative interactive designs, as well as tailoring alerts to clinical roles. As examples, alerts may be tiered to convey risk, and certain alerts may be sent to pharmacists. We aimed to evaluate which variants elicit less alert fatigue.

MATERIALS AND METHODS

We searched for articles published between 2007 and 2017 using the PubMed, Embase, CINAHL, and Cochrane databases. We included articles documenting peer-reviewed empirical research that described the interactive design of a CDS system, to which clinical role it was presented, and how often prescribers accepted the resultant advice. Next, we compared the acceptance rates of conventional CDS-presenting prescribers with interruptive modal dialogs (ie, "pop-ups")-with alternative designs, such as role-tailored alerts.

RESULTS

Of 1011 articles returned by the search, we included 39. We found different methods for measuring acceptance rates; these produced incomparable results. The most common type of CDS-in which modals interrupted prescribers-was accepted the least often. Tiering by risk, providing shortcuts for common corrections, requiring a reason to override, and tailoring CDS to match the roles of pharmacists and prescribers were the most common alternatives. Only 1 alternative appeared to increase prescriber acceptance: role tailoring. Possible reasons include the importance of etiquette in delivering advice, the cognitive benefits of delegation, and the difficulties of computing "relevance."

CONCLUSIONS

Alert fatigue may be mitigated by redesigning the interactive behavior of CDS and tailoring CDS to clinical roles. Further research is needed to develop alternative designs, and to standardize measurement methods to enable meta-analyses.

User stories as lightweight requirements for agile clinical decision support development

Objective

We sought to demonstrate applicability of user stories, progressively elaborated by testable acceptance criteria, as lightweight requirements for agile development of clinical decision support (CDS).

Materials and Methods

User stories employed the template: As a [type of user], I want [some goal] so that [some reason]. From the “so that” section, CDS benefit measures were derived. Detailed acceptance criteria were elaborated through ensuing conversations. We estimated user story size with “story points,” and depicted multiple user stories with a use case diagram or feature breakdown structure. Large user stories were split to fit into 2-week iterations.

Results

One example user story was: As a rheumatologist, I want to be advised if my patient with rheumatoid arthritis is not on a disease-modifying anti-rheumatic drug (DMARD), so that they receive optimal therapy and can experience symptom improvement. This yielded a process measure (DMARD use), and an outcome measure (Clinical Disease Activity Index). Following implementation, the DMARD nonuse rate decreased from 3.7% to 1.4%. Patients with a high Clinical Disease Activity Index improved from 13.7% to 7%. For a thromboembolism prevention CDS project, diagrams organized multiple user stories.

Discussion

User stories written in the clinician’s voice aid CDS governance and lead naturally to measures of CDS effectiveness. Estimation of relative story size helps plan CDS delivery dates. User stories prove to be practical even on larger projects.

Conclusions

User stories concisely communicate the who, what, and why of a CDS request, and serve as lightweight requirements for agile development to meet the demand for increasingly diverse CDS.

Interests and needs of eye care providers in clinical decision support for glaucoma

Objective

To study whether clinicians who treat glaucoma are interested in using clinical decision support (CDS) tools for glaucoma, what glaucoma clinical decisions they feel would benefit from CDS, and what characteristics of CDS design they feel would be important in glaucoma clinical practice.

Methods and analysis

Working with the American Glaucoma Society, the Utah Ophthalmology Society and the Utah Optometric Association, we identified a group of clinicians who care for patients with glaucoma. We asked these clinicians about interest in CDS, what glaucoma clinical decisions would benefit from CDS, and what characteristics of CDS tool design would be important in glaucoma clinical practice.

Results

Of the 105 clinicians (31 optometrists, 10 general ophthalmologists and 64 glaucoma specialists), 93 (88.6%) were either ‘definitely’ or ‘probably’ interested in using CDS for glaucoma. There were no statistically significant differences in interest between clinical specialties (p=0.12), years in practice (p=0.85) or numbers of patients seen daily (p=0.99). Identifying progression of glaucoma was the clinical decision the largest number of clinicians felt would benefit from CDS (104/105, 99.1%). An easy to use interface was the CDS characteristic the largest number of clinicians felt would be ‘very important’ (93/105, 88.6%).

Conclusion

Of this group of clinicians who treat glaucoma, 88.6% were interested in using CDS for glaucoma and 99.1% felt that identification of glaucomatous progression could benefit from CDS. This level of interest supports future work to develop CDS for glaucoma.

Evaluation of a Pilot Vancomycin Precision Dosing Advisory Service on Target Exposure Attainment Using an Interrupted Time Series Analysis

This study evaluated the ability of a pilot therapeutic drug monitoring (TDM) Advisory Service to facilitate vancomycin therapeutic target attainment within a real-world clinical setting. The Service provided area under the concentration-time curve (AUC)-guided vancomycin dose recommendations, using Bayesian forecasting software and clinical expertise, to prescribers at an Australian hospital. A retrospective audit of intravenous vancomycin therapy (> 48 hours) in adults (≥ 18 years old) was undertaken over a 54-month period to evaluate attainment of established vancomycin pharmacokinetic/pharmacodynamic targets (AUC over 24 hours / minimum inhibitory concentration: 400-600) before (36-month period) and after (18-month period) Service implementation. Interrupted time series analysis was employed to evaluate monthly measures of the median proportion of therapy spent within the target range. Indices of time to target attainment were also assessed before and after Service implementation. The final cohort comprised 1,142 courses of vancomycin (816 patients); 835 courses (596 patients) and 307 courses (220 patients) administered before and after Service implementation, respectively. Prior to piloting the Service, the median proportion of time in the target range was 40.1% (95% CI, 34.3-46.0%); this increased by 10.4% (95% CI, 1.2-19.6%, P = 0.03) after the Service, and was sustained throughout the post-Service evaluation period. Post-Service target attainment at 48-72 hours after initiation of therapy was increased (7.8%, 95% CI, 1.3-14.3%, P = 0.02). The findings of this study provide evidence that a consultative TDM Service can facilitate attainment of vancomycin therapeutic targets; however, optimization of the Service may further improve the use of vancomycin.

Unique Needs for the Implementation of Emergency Department Human Immunodeficiency Virus Screening in Adolescents

BACKGROUND

The Centers for Disease Control and Prevention (CDC) recommend universal human immunodeficiency virus (HIV) screening starting at 13 years, which has been implemented in many general U.S. emergency departments (EDs) but infrequently in pediatric EDs. We aimed to 1) implement a pilot of routine adolescent HIV screening in a pediatric ED and 2) determine the unique barriers to CDC-recommended screening in this region of high HIV prevalence.

METHODS

This was a prospective 4-month implementation of a routine HIV screening pilot in a convenience sample of adolescents 13 to 18 years at a single pediatric ED, based on study personnel availability. Serum-based fourth-generation HIV testing was run through a central laboratory. Parents were allowed to remain in the room for HIV counseling and testing. Data were collected regarding patient characteristics and HIV testing quality metrics. Comparisons were made using chi-square and Fisher's exact tests. Regression analysis was performed to assess for an association between parent presence at the time of enrollment and adolescent decision to participate in HIV screening.

RESULTS

Over 4 months, 344 of 806 adolescents approached consented to HIV screening (57% female, mean ± SD = 15.1 ± 1.6 years). Adolescents with HIV screening were more likely to be older than those who declined (p = 0.025). Other blood tests were collected with the HIV sample for 21% of adolescents; mean time to result was 105 minutes (interquartile range = 69 to 123) and 79% were discharged before the result was available. Having a parent present for enrollment was not associated with adolescent participation (adjusted odds ratio = 1.07, 95% CI = 0.67 to 1.70). Barriers to testing included: fear of needlestick, time to results, cost, and staff availability. One of 344 tests was positive in a young adolescent with Stage 1 HIV.

CONCLUSIONS

Routine HIV screening in adolescents was able to be implemented in this pediatric ED and led to the identification of early infection in a young adolescent who would have otherwise been undetected at this stage of disease. Addressing the unique barriers to adolescent HIV screening is critical in high-prevalence regions and may lead to earlier diagnosis and treatment in this vulnerable population.

A Conceptual Framework to Study the Implementation of Clinical Decision Support Systems (BEAR): Literature Review and Concept Mapping

Background

The implementation of clinical decision support systems (CDSSs) as an intervention to foster clinical practice change is affected by many factors. Key factors include those associated with behavioral change and those associated with technology acceptance. However, the literature regarding these subjects is fragmented and originates from two traditionally separate disciplines: implementation science and technology acceptance.

Objective

Our objective is to propose an integrated framework that bridges the gap between the behavioral change and technology acceptance aspects of the implementation of CDSSs.

Methods

We employed an iterative process to map constructs from four contributing frameworks—the Theoretical Domains Framework (TDF); the Consolidated Framework for Implementation Research (CFIR); the Human, Organization, and Technology-fit framework (HOT-fit); and the Unified Theory of Acceptance and Use of Technology (UTAUT)—and the findings of 10 literature reviews, identified through a systematic review of reviews approach.

Results

The resulting framework comprises 22 domains: agreement with the decision algorithm; attitudes; behavioral regulation; beliefs about capabilities; beliefs about consequences; contingencies; demographic characteristics; effort expectancy; emotions; environmental context and resources; goals; intentions; intervention characteristics; knowledge; memory, attention, and decision processes; patient–health professional relationship; patient’s preferences; performance expectancy; role and identity; skills, ability, and competence; social influences; and system quality. We demonstrate the use of the framework providing examples from two research projects.

Conclusions

We proposed BEAR (BEhavior and Acceptance fRamework), an integrated framework that bridges the gap between behavioral change and technology acceptance, thereby widening the view established by current models.

Electronic clinical decision support to facilitate a change in clinical practice: Small details can make or break success

BACKGROUND

The use of electronic clinical decision support (CDS) is becoming common to change historically common clinical practices considered outdated by current guidelines. Preimplementation design of CDS tools is key to their success in changing clinical behaviors. Unfortunately, there are no established protocols for CDS tool development, and CDS failure can result from even small design flaws. This paper describes an example of a design oversight and how correction resulted in CDS success.

STUDY DESIGN AND METHODS

We performed a retrospective review of compliance with a CDS tool to encourage the use of prothrombin complex concentrate over plasma transfusion for the emergent reversal of warfarin. We identified a potential design flaw, made the necessary modifications, and repeated the compliance review.

RESULTS

After CDS, plasma orders declined by 150 units/mo; however, 48% of orders placed for non-warfarin coagulopathy were still for warfarin reversal. Hospital-wide, this noncompliance was 36% and was 80% in the emergency department. By simply relocating the qualifier "NOT on warfarin" from the end to the beginning of the order, noncompliance for warfarin reversal was reduced to 5% (P < .0001 by chi-square).

CONCLUSIONS

The successful use of electronic clinical decision support in the electronic medical record can depend on optimal design. Missing even small design elements such as the positioning of key terms within the tool can result in an ineffective CDS. Important design strategies to avoid poor performance are discussed as they relate to the CDS tool we describe.

Clinician Perceptions of Timing and Presentation of Drug-Drug Interaction Alerts

OBJECTIVE

Alert presentation of clinical decision support recommendations is a common method for providing information; however, many alerts are overridden suggesting presentation design improvements can be made. This study attempts to assess pediatric prescriber information needs for drug-drug interactions (DDIs) alerts and to evaluate the optimal presentation timing and presentation in the medication ordering process.

METHODS

Six case scenarios presented interactions between medications used in pediatric specialties of general medicine, infectious disease, cardiology, and neurology. Timing varied to include alert interruption at medication selection versus order submission; or was noninterruptive. Interviews were audiotaped, transcribed, and independently analyzed to derive central themes.

RESULTS

Fourteen trainee and attending clinicians trained in pediatrics, cardiology, and neurology participated. Coders derived 8 central themes from 929 quotes. Discordance exists between medication prescribing frequency and DDI knowledge; providers may commonly prescribe medications for which they do not recognize DDIs. Providers wanted alerts at medication selection rather than at order signature. Alert presentation themes included standardizing text, providing interaction-specific incidence/risk information, DDI rating scales, consolidating alerts, and providing alternative therapies. Providers want alerts to be actionable, for example, allowing medication discontinuation and color visual cues for essential information. Despite alert volume, participants did not "mind being reminded because there is always the chance that at that particular moment (they) do not remember it" and acknowledged the importance of alerts as "essential in terms of patient safety."

CONCLUSION

Clinicians unanimously agreed on the importance of receiving DDI alerts to improve patient safety. The perceived alert value can be improved by incorporating clinician preferences for timing and presentation.

Pragmatic cluster randomized trial to evaluate effectiveness and implementation of enhanced EHR-facilitated cancer symptom control (E2C2)

Background

The prevalence of inadequate symptom control among cancer patients is quite high despite the availability of definitive care guidelines and accurate and efficient assessment tools.

Methods

We will conduct a hybrid type 2 stepped wedge pragmatic cluster randomized clinical trial to evaluate a guideline-informed enhanced, electronic health record (EHR)-facilitated cancer symptom control (E2C2) care model. Teams of clinicians at five hospitals that care for patients with various cancers will be randomly assigned in steps to the E2C2 intervention. The E2C2 intervention will have two levels of care: level 1 will offer low-touch, automated self-management support for patients reporting moderate sleep disturbance, pain, anxiety, depression, and energy deficit symptoms or limitations in physical function (or both). Level 2 will offer nurse-managed collaborative care for patients reporting more intense (severe) symptoms or functional limitations (or both). By surveying and interviewing clinical staff, we will also evaluate whether the use of a multifaceted, evidence-based implementation strategy to support adoption and use of the E2C2 technologies improves patient and clinical outcomes. Finally, we will conduct a mixed methods evaluation to identify disparities in the adoption and implementation of the E2C2 intervention among elderly and rural-dwelling patients with cancer.

Discussion

The E2C2 intervention offers a pragmatic, scalable approach to delivering guideline-based symptom and function management for cancer patients. Since discrete EHR-imbedded algorithms drive defining aspects of the intervention, the approach can be efficiently disseminated and updated by specifying and modifying these centralized EHR algorithms.

Trial registration

ClinicalTrials.gov, NCT03892967. Registered on 25 March 2019.

Designing, Conducting, and Reporting Clinical Decision Support Studies: Recommendations and Call to Action

By enabling more efficient and effective medical decision making, computer-based clinical decision support (CDS) could unlock widespread benefits from the significant investment in electronic health record (EHR) systems in the United States. Evidence from high-quality CDS studies is needed to enable and support this vision of CDS-facilitated care optimization, but limited guidance is available in the literature for designing and reporting CDS studies. To address this research gap, this article provides recommendations for designing, conducting, and reporting CDS studies to: 1) ensure that EHR data to inform the CDS are available; 2) choose decision rules that are consistent with local care processes; 3) target the right users and workflows; 4) make the CDS easy to access and use; 5) minimize the burden placed on users; 6) incorporate CDS success factors identified in the literature, in particular the automatic provision of CDS as a part of clinician workflow; 7) ensure that the CDS rules are adequately tested; 8) select meaningful evaluation measures; 9) use as rigorous a study design as is feasible; 10) think about how to deploy the CDS beyond the original host organization; 11) report the study in context; 12) help the audience understand why the intervention succeeded or failed; and 13) consider the financial implications. If adopted, these recommendations should help advance the vision of more efficient, effective care facilitated by useful and widely available CDS.

Model-Informed Precision Dosing of Vancomycin in Hospitalized Children: Implementation and Adoption at an Academic Children's Hospital

Background

Model-informed precision dosing (MIPD) can serve as a powerful tool during therapeutic drug monitoring (TDM) to help individualize dosing in populations with large pharmacokinetic variation. Yet, adoption of MIPD in the clinical setting has been limited. Overcoming technologic hurdles that allow access to MIPD at the point-of-care and placing it in the hands of clinical specialists focused on medication dosing may encourage adoption.

Objective

To describe the hospital implementation and usage of a MIPD clinical decision support (CDS) tool for vancomycin in a pediatric population.

Methods

Within an academic children’s hospital, MIPD for vancomycin was implemented via a commercial cloud-based CDS tool that utilized Bayesian forecasting. Clinical pharmacists were recognized as local champions to facilitate adoption of the tool and operated as end-users. Integration within the electronic health record (EHR) and automatic transmission of patient data to the tool were identified as important requirements. A web-link icon was developed within the EHR which when clicked sends users and needed patient-level clinical data to the CDS platform. Individualized pharmacokinetic predictions and exposure metrics for vancomycin are then presented in the form of a web-based dashboard. Use of the CDS tool as part of TDM was tracked and users were surveyed on their experience.

Results

After a successful pilot phase in the neonatal intensive care unit, implementation of MIPD was expanded to the pediatric intensive care unit, followed by availability to the entire hospital. During the first 2+ years since implementation, a total of 853 patient-courses (n = 96 neonates, n = 757 children) and 2,148 TDM levels were evaluated using the CDS tool. For the most recent 6 months, the CDS tool was utilized to support 79% (181/230) of patient-courses in which TDM was performed. Of 26 users surveyed, > 96% agreed or strongly agreed that automatic transmission of patient data to the tool was a feature that helped them complete tasks more efficiently; 81% agreed or strongly agreed that they were satisfied with the CDS tool.

Conclusions

Integration of a vancomycin CDS tool within the EHR, along with leveraging the expertise of clinical pharmacists, allowed for successful adoption of MIPD in clinical care.

Application of human factors to improve usability of clinical decision support for diagnostic decision-making: a scenario-based simulation study

OBJECTIVE

In this study, we used human factors (HF) methods and principles to design a clinical decision support (CDS) that provides cognitive support to the pulmonary embolism (PE) diagnostic decision-making process in the emergency department. We hypothesised that the application of HF methods and principles will produce a more usable CDS that improves PE diagnostic decision-making, in particular decision about appropriate clinical pathway.

MATERIALS AND METHODS

We conducted a scenario-based simulation study to compare a HF-based CDS (the so-called CDS for PE diagnosis (PE-Dx CDS)) with a web-based CDS (MDCalc); 32 emergency physicians performed various tasks using both CDS. PE-Dx integrated HF design principles such as automating information acquisition and analysis, and minimising workload. We assessed all three dimensions of usability using both objective and subjective measures: effectiveness (eg, appropriate decision regarding the PE diagnostic pathway), efficiency (eg, time spent, perceived workload) and satisfaction (perceived usability of CDS).

RESULTS

Emergency physicians made more appropriate diagnostic decisions (94% with PE-Dx; 84% with web-based CDS; p<0.01) and performed experimental tasks faster with the PE-Dx CDS (on average 96 s per scenario with PE-Dx; 117 s with web-based CDS; p<0.001). They also reported lower workload (p<0.001) and higher satisfaction (p<0.001) with PE-Dx.

CONCLUSIONS

This simulation study shows that HF methods and principles can improve usability of CDS and diagnostic decision-making. Aspects of the HF-based CDS that provided cognitive support to emergency physicians and improved diagnostic performance included automation of information acquisition (eg, auto-populating risk scoring algorithms), minimisation of workload and support of decision selection (eg, recommending a clinical pathway). These HF design principles can be applied to the design of other CDS technologies to improve diagnostic safety.