Systems engineering and human factors support of a system of novel EHR-integrated tools to prevent harm in the hospital

Effect of digital tools to promote hospital quality and safety on adverse events after discharge

OBJECTIVES

Post-discharge adverse events (AEs) are common and heralded by new and worsening symptoms (NWS). We evaluated the effect of electronic health record (EHR)-integrated digital tools designed to promote quality and safety in hospitalized patients on NWS and AEs after discharge.

MATERIALS AND METHODS

Adult general medicine patients at a community hospital were enrolled. We implemented a dashboard which clinicians used to assess safety risks during interdisciplinary rounds. Post-implementation patients were randomized to complete a discharge checklist whose responses were incorporated into the dashboard. Outcomes were assessed using EHR review and 30-day call data adjudicated by 2 clinicians and analyzed using Poisson regression. We conducted comparisons of each exposure on post-discharge outcomes and used selected variables and NWS as independent predictors to model post-discharge AEs using multivariable logistic regression.

RESULTS

A total of 260 patients (122 pre, 71 post [dashboard], 67 post [dashboard plus discharge checklist]) enrolled. The adjusted incidence rate ratios (aIRR) for NWS and AEs were unchanged in the post- compared to pre-implementation period. For patient-reported NWS, aIRR was non-significantly higher for dashboard plus discharge checklist compared to dashboard participants (1.23 [0.97,1.56], P = .08). For post-implementation patients with an AE, aIRR for duration of injury (>1 week) was significantly lower for dashboard plus discharge checklist compared to dashboard participants (0 [0,0.53], P < .01). In multivariable models, certain patient-reported NWS were associated with AEs (3.76 [1.89,7.82], P < .01).

DISCUSSION

While significant reductions in post-discharge AEs were not observed, checklist participants experiencing a post-discharge AE were more likely to report NWS and had a shorter duration of injury.

CONCLUSION

Interventions designed to prompt patients to report NWS may facilitate earlier detection of AEs after discharge.

CLINICALTRIALS.GOV

NCT05232656.

Artificial Intelligence Can Be Regulated Using Current Patient Safety Procedures and Infrastructure in Hospitals

This Viewpoint describes the potential benefits and harms of using artificial intelligence (AI) in health care decision-making processes.

Integrating human factors engineering into your pediatric radiology practice

Human factors engineering involves the study and development of methods aimed at enhancing performance, improving safety, and optimizing user satisfaction. The focus of human factors engineering encompasses the design of work environments and an understanding of human mental processes to prevent errors. In this review, we summarize the history, applications, and impacts of human factors engineering on the healthcare field. To illustrate these applications and impacts, we provide several examples of how successful integration of a human factors engineer in our pediatric radiology department has positively impacted various projects. The successful integration of human factors engineering expertise has contributed to projects including improving response times for portable radiography requests, deploying COVID-19 response resources, informing the redesign of scheduling workflows, and implementation of a virtual ergonomics program for remote workers. In sum, the integration of human factors engineering insight into our department has resulted in tangible benefits and has also positioned us as proactive contributors to broader hospital-wide improvements.

Design and Implementation of an Inpatient Fall Risk Management Information System

BACKGROUND

Falls had been identified as one of the nursing-sensitive indicators for nursing care in hospitals. With technological progress, health information systems make it possible for health care professionals to manage patient care better. However, there is a dearth of research on health information systems used to manage inpatient falls.

OBJECTIVE

This study aimed to design and implement a novel hospital-based fall risk management information system (FRMIS) to prevent inpatient falls and improve nursing quality.

METHODS

This implementation was conducted at a large academic medical center in central China. We established a nurse-led multidisciplinary fall prevention team in January 2016. The hospital's fall risk management problems were summarized by interviewing fall-related stakeholders, observing fall prevention workflow and post-fall care process, and investigating patients' satisfaction. The FRMIS was developed using an iterative design process, involving collaboration among health care professionals, software developers, and system architects. We used process indicators and outcome indicators to evaluate the implementation effect.

RESULTS

The FRMIS includes a fall risk assessment platform, a fall risk warning platform, a fall preventive strategies platform, fall incident reporting, and a tracking improvement platform. Since the implementation of the FRMIS, the inpatient fall rate was significantly lower than that before implementation (P<.05). In addition, the percentage of major fall-related injuries was significantly lower than that before implementation. The implementation rate of fall-related process indicators and the reporting rate of high risk of falls were significantly different before and after system implementation (P<.05).

CONCLUSIONS

The FRMIS provides support to nursing staff in preventing falls among hospitalized patients while facilitating process control for nursing managers.

Adopting and expanding ethical principles for generative artificial intelligence from military to healthcare

In 2020, the U.S. Department of Defense officially disclosed a set of ethical principles to guide the use of Artificial Intelligence (AI) technologies on future battlefields. Despite stark differences, there are core similarities between the military and medical service. Warriors on battlefields often face life-altering circumstances that require quick decision-making. Medical providers experience similar challenges in a rapidly changing healthcare environment, such as in the emergency department or during surgery treating a life-threatening condition. Generative AI, an emerging technology designed to efficiently generate valuable information, holds great promise. As computing power becomes more accessible and the abundance of health data, such as electronic health records, electrocardiograms, and medical images, increases, it is inevitable that healthcare will be revolutionized by this technology. Recently, generative AI has garnered a lot of attention in the medical research community, leading to debates about its application in the healthcare sector, mainly due to concerns about transparency and related issues. Meanwhile, questions around the potential exacerbation of health disparities due to modeling biases have raised notable ethical concerns regarding the use of this technology in healthcare. However, the ethical principles for generative AI in healthcare have been understudied. As a result, there are no clear solutions to address ethical concerns, and decision-makers often neglect to consider the significance of ethical principles before implementing generative AI in clinical practice. In an attempt to address these issues, we explore ethical principles from the military perspective and propose the "GREAT PLEA" ethical principles, namely Governability, Reliability, Equity, Accountability, Traceability, Privacy, Lawfulness, Empathy, and Eutonomy, for generative AI in healthcare. Furthermore, we introduce a framework for adopting and expanding these ethical principles in a practical way that has been useful in the military and can be applied to healthcare for generative AI, based on contrasting their ethical concerns and risks. Ultimately, we aim to proactively address the ethical dilemmas and challenges posed by the integration of generative AI into healthcare practice.

Machine learning models to detect and predict patient safety events using electronic health records: A systematic review

INTRODUCTION

Identifying patient safety events using electronic health records (EHRs) and automated machine learning-based detection methods can help improve the efficiency and quality of healthcare service provision.

OBJECTIVE

This study aimed to systematically review machine learning-based methods and techniques, as well as their results for patient safety event management using EHRs.

METHODS

We reviewed the studies that focused on machine learning techniques, including automatic prediction and detection of patient safety events and medical errors through EHR analysis to manage patient safety events. The data were collected by searching Scopus, PubMed (Medline), Web of Science, EMBASE, and IEEE Xplore databases.

RESULTS

After screening, 41 papers were reviewed. Support vector machine (SVM), random forest, conditional random field (CRF), and bidirectional long short-term memory with conditional random field (BiLSTM-CRF) algorithms were mostly applied to predict, identify, and classify patient safety events using EHRs; however, they had different performances. BiLSTM-CRF was employed in most of the studies to extract and identify concepts, e.g., adverse drug events (ADEs) and adverse drug reactions (ADRs), as well as relationships between drug and severity, drug and ADEs, drug and ADRs. Recurrent neural networks (RNN) and BiLSTM-CRF had the best results in detecting ADEs compared to other patient safety events. Linear classifiers and Naive Bayes (NB) had the highest performance for ADR detection. Logistic regression had the best results in detecting surgical site infections. According to the findings, the quality of articles has non-significantly improved in recent years, but they had low average scores.

CONCLUSIONS

Machine learning can be useful in automatic detection and prediction of patient safety events. However, most of these algorithms have not yet been externally validated or prospectively tested. Therefore, further studies are required to improve the performance of these automated systems.

Advancing health information during interhospital transfer: An interrupted time series

INTRODUCTION

Although the transfer of patients between acute care hospitals (interhospital transfer, IHT) is common, health information exchange (HIE) during IHT remains inadequate, with fragmented communication and unreliable access to clinical information. This study aims to design, implement, and rigorously evaluate the implementation of a HIE platform to improve data access during IHT.

METHODS AND ANALYSIS

Study subjects include patients aged >18 transferred to the medical, cardiology, oncology, or intensive care unit (ICU) services at an 800-bed quaternary care hospital; and healthcare workers involved in their care. The first aim of this study is to optimize clinician workflow, data visualization, and interoperability through user-centered design sessions for HIE platform development. The second aim is to evaluate the impact of the intervention on clinician-reported medical errors among 500 pre- and 500 postintervention IHT patients using interrupted time series methodology, adjusting for confounding variables and temporal trends. The third aim is to evaluate intervention fidelity, use and perceived usability of the platform, and barriers and facilitators of implementation from interprofessional stakeholder input, using mixed-methods evaluation. The fourth aim is to consolidate key findings to create a toolkit for spread and sustainability.

ETHICS AND DISSEMINATION

We will track patient safety endpoints and clinician workflow burdens and ensure the protection of patient data throughout the study. We will disseminate our findings via the creation of a toolkit for spread and sustainability, partnering with our funder (AHRQ) for dissemination, and communicating our results via abstracts and publications.

Artificial intelligence in the field of pharmacy practice: A literature review

Artificial intelligence (AI) is a transformative technology used in various industrial sectors including healthcare. In pharmacy practice, AI has the potential to significantly improve medication management and patient care. This review explores various AI applications in the field of pharmacy practice. The incorporation of AI technologies provides pharmacists with tools and systems that help them make accurate and evidence-based clinical decisions. By using AI algorithms and Machine Learning, pharmacists can analyze a large volume of patient data, including medical records, laboratory results, and medication profiles, aiding them in identifying potential drug-drug interactions, assessing the safety and efficacy of medicines, and making informed recommendations tailored to individual patient requirements. Various AI models have been developed to predict and detect adverse drug events, assist clinical decision support systems with medication-related decisions, automate dispensing processes in community pharmacies, optimize medication dosages, detect drug-drug interactions, improve adherence through smart technologies, detect and prevent medication errors, provide medication therapy management services, and support telemedicine initiatives. By incorporating AI into clinical practice, health care professionals can augment their decision-making processes and provide patients with personalized care. AI allows for greater collaboration between different healthcare services provided to a single patient. For patients, AI may be a useful tool for providing guidance on how and when to take a medication, aiding in patient education, and promoting medication adherence and AI may be used to know how and where to obtain the most cost-effective healthcare and how best to communicate with healthcare professionals, optimize the health monitoring using wearables devices, provide everyday lifestyle and health guidance, and integrate diet and exercise.

Developing, pilot testing, and refining requirements for 3 EHR-integrated interventions to improve diagnostic safety in acute care: a user-centered approach

Objective

To describe a user-centered approach to develop, pilot test, and refine requirements for 3 electronic health record (EHR)-integrated interventions that target key diagnostic process failures in hospitalized patients.

Materials and Methods

Three interventions were prioritized for development: a Diagnostic Safety Column (DSC) within an EHR-integrated dashboard to identify at-risk patients; a Diagnostic Time-Out (DTO) for clinicians to reassess the working diagnosis; and a Patient Diagnosis Questionnaire (PDQ) to gather patient concerns about the diagnostic process. Initial requirements were refined from analysis of test cases with elevated risk predicted by DSC logic compared to risk perceived by a clinician working group; DTO testing sessions with clinicians; PDQ responses from patients; and focus groups with clinicians and patient advisors using storyboarding to model the integrated interventions. Mixed methods analysis of participant responses was used to identify final requirements and potential implementation barriers.

Results

Final requirements from analysis of 10 test cases predicted by the DSC, 18 clinician DTO participants, and 39 PDQ responses included the following: DSC configurable parameters (variables, weights) to adjust baseline risk estimates in real-time based on new clinical data collected during hospitalization; more concise DTO wording and flexibility for clinicians to conduct the DTO with or without the patient present; and integration of PDQ responses into the DSC to ensure closed-looped communication with clinicians. Analysis of focus groups confirmed that tight integration of the interventions with the EHR would be necessary to prompt clinicians to reconsider the working diagnosis in cases with elevated diagnostic error (DE) risk or uncertainty. Potential implementation barriers included alert fatigue and distrust of the risk algorithm (DSC); time constraints, redundancies, and concerns about disclosing uncertainty to patients (DTO); and patient disagreement with the care team's diagnosis (PDQ).

Discussion

A user-centered approach led to evolution of requirements for 3 interventions targeting key diagnostic process failures in hospitalized patients at risk for DE.

Conclusions

We identify challenges and offer lessons from our user-centered design process.

Registered nurses' approach to pressure injury prevention: A descriptive qualitative study

AIMS

To explore Registered Nurses' approaches to pressure injury prevention, including how they perceive their roles, how they prioritize pressure injury prevention and factors influencing prevention in the Chinese context.

DESIGN

A qualitative descriptive study.

METHODS

Audio-recorded, face-to-face, semi-structured individual interviews were conducted with Registered Nurses in a large tertiary hospital in China from August to December 2020. Using the System Engineering Initiative for Patient Safety Model, the interview guide was developed to describe the work system, processes and outcomes (three domains) associated with nurses' pressure injury prevention practices. Deductive and inductive content analyses were used.

FINDINGS

Twenty-seven nurses participated in the interviews. Four themes related to two domains of the model emerged: Work system: (i) Nurses lead and coordinate pressure injury prevention; Work processes: (ii) Individualized pressure injury prevention is founded on comprehensive patient assessment; (iii) Collaborating ensures patients receive appropriate pressure injury prevention; and (iv) Competing factors influence the delivery of appropriate pressure injury prevention. One category emerged about work outcome: Nurses strive to do their best in pressure injury prevention but hold major concerns when pressure injuries occur.

CONCLUSIONS

Nurses play a leading role in pressure injury prevention delivery but require appropriate resources and assistance and support from other healthcare personnel, patients and carers. Understaffing, lack of resources, complex reporting and poor patient compliance challenge nurses in their delivery of pressure injury prevention.

IMPACT

Pressure injury prevention is primarily a nursing responsibility therefore nurses' approaches to prevention were explored. Nurses rely on collaboration with others and access to various resources to provide pressure injury prevention. They recognize the patients' and carers' roles and acknowledge the importance of accessing guidance and support from nursing leaders and wound experts. Acknowledging nurses leading role in prevention and ensuring they have adequate resources are important for quality care.

Toward an Ecologically Valid Conceptual Framework for the Use of Artificial Intelligence in Clinical Settings: Need for Systems Thinking, Accountability, Decision-making, Trust, and Patient Safety Considerations in Safeguarding the Technology and Clinicians

The health care management and the medical practitioner literature lack a descriptive conceptual framework for understanding the dynamic and complex interactions between clinicians and artificial intelligence (AI) systems. As most of the existing literature has been investigating AI's performance and effectiveness from a statistical (analytical) standpoint, there is a lack of studies ensuring AI's ecological validity. In this study, we derived a framework that focuses explicitly on the interaction between AI and clinicians. The proposed framework builds upon well-established human factors models such as the technology acceptance model and expectancy theory. The framework can be used to perform quantitative and qualitative analyses (mixed methods) to capture how clinician-AI interactions may vary based on human factors such as expectancy, workload, trust, cognitive variables related to absorptive capacity and bounded rationality, and concerns for patient safety. If leveraged, the proposed framework can help to identify factors influencing clinicians' intention to use AI and, consequently, improve AI acceptance and address the lack of AI accountability while safeguarding the patients, clinicians, and AI technology. Overall, this paper discusses the concepts, propositions, and assumptions of the multidisciplinary decision-making literature, constituting a sociocognitive approach that extends the theories of distributed cognition and, thus, will account for the ecological validity of AI.

Evaluation of electronic health record-integrated digital health tools to engage hospitalized patients in discharge preparation

OBJECTIVE

To evaluate the effect of electronic health record (EHR)-integrated digital health tools comprised of a checklist and video on transitions-of-care outcomes for patients preparing for discharge.

MATERIALS AND METHODS

English-speaking, general medicine patients (>18 years) hospitalized at least 24 hours at an academic medical center in Boston, MA were enrolled before and after implementation. A structured checklist and video were administered on a mobile device via a patient portal or web-based survey at least 24 hours prior to anticipated discharge. Checklist responses were available for clinicians to review in real time via an EHR-integrated safety dashboard. The primary outcome was patient activation at discharge assessed by patient activation (PAM)-13. Secondary outcomes included postdischarge patient activation, hospital operational metrics, healthcare resource utilization assessed by 30-day follow-up calls and administrative data and change in patient activation from discharge to 30 days postdischarge.

RESULTS

Of 673 patients approached, 484 (71.9%) enrolled. The proportion of activated patients (PAM level 3 or 4) at discharge was nonsignificantly higher for the 234 postimplementation compared with the 245 preimplementation participants (59.8% vs 56.7%, adjusted OR 1.23 [0.38, 3.96], P = .73). Postimplementation participants reported 3.75 (3.02) concerns via the checklist. Mean length of stay was significantly higher for postimplementation compared with preimplementation participants (10.13 vs 6.21, P < .01). While there was no effect on postdischarge outcomes, there was a nonsignificant decrease in change in patient activation within participants from pre- to postimplementation (adjusted difference-in-difference of -16.1% (9.6), P = .09).

CONCLUSIONS

EHR-integrated digital health tools to prepare patients for discharge did not significantly increase patient activation and was associated with a longer length of stay. While issues uncovered by the checklist may have encouraged patients to inquire about their discharge preparedness, other factors associated with patient activation and length of stay may explain our observations. We offer insights for using PAM-13 in context of real-world health-IT implementations.

TRIAL REGISTRATION

NIH US National Library of Medicine, NCT03116074, clinicaltrials.gov.

Use of machine learning in geriatric clinical care for chronic diseases: a systematic literature review

Objectives

Geriatric clinical care is a multidisciplinary assessment designed to evaluate older patients’ (age 65 years and above) functional ability, physical health, and cognitive well-being. The majority of these patients suffer from multiple chronic conditions and require special attention. Recently, hospitals utilize various artificial intelligence (AI) systems to improve care for elderly patients. The purpose of this systematic literature review is to understand the current use of AI systems, particularly machine learning (ML), in geriatric clinical care for chronic diseases.

Materials and Methods

We restricted our search to eight databases, namely PubMed, WorldCat, MEDLINE, ProQuest, ScienceDirect, SpringerLink, Wiley, and ERIC, to analyze research articles published in English between January 2010 and June 2019. We focused on studies that used ML algorithms in the care of geriatrics patients with chronic conditions.

Results

We identified 35 eligible studies and classified in three groups: psychological disorder (n = 22), eye diseases (n = 6), and others (n = 7). This review identified the lack of standardized ML evaluation metrics and the need for data governance specific to health care applications.

Conclusion

More studies and ML standardization tailored to health care applications are required to confirm whether ML could aid in improving geriatric clinical care.

Role of Artificial Intelligence in Patient Safety Outcomes: Systematic Literature Review

BACKGROUND

Artificial intelligence (AI) provides opportunities to identify the health risks of patients and thus influence patient safety outcomes.

OBJECTIVE

The purpose of this systematic literature review was to identify and analyze quantitative studies utilizing or integrating AI to address and report clinical-level patient safety outcomes.

METHODS

We restricted our search to the PubMed, PubMed Central, and Web of Science databases to retrieve research articles published in English between January 2009 and August 2019. We focused on quantitative studies that reported positive, negative, or intermediate changes in patient safety outcomes using AI apps, specifically those based on machine-learning algorithms and natural language processing. Quantitative studies reporting only AI performance but not its influence on patient safety outcomes were excluded from further review.

RESULTS

We identified 53 eligible studies, which were summarized concerning their patient safety subcategories, the most frequently used AI, and reported performance metrics. Recognized safety subcategories were clinical alarms (n=9; mainly based on decision tree models), clinical reports (n=21; based on support vector machine models), and drug safety (n=23; mainly based on decision tree models). Analysis of these 53 studies also identified two essential findings: (1) the lack of a standardized benchmark and (2) heterogeneity in AI reporting.

CONCLUSIONS

This systematic review indicates that AI-enabled decision support systems, when implemented correctly, can aid in enhancing patient safety by improving error detection, patient stratification, and drug management. Future work is still needed for robust validation of these systems in prospective and real-world clinical environments to understand how well AI can predict safety outcomes in health care settings.

Assessing the cognitive and work load of an inpatient safety dashboard in the context of opioid management

For health information technology to realize its potential to improve flow, care, and patient safety, applications should be intuitive to use and burden neutral for frontline clinicians. We assessed the impact of a patient safety dashboard on clinician cognitive and work load within a simulated information-seeking task for safe inpatient opioid medication management. Compared to use of an electronic health record for the same task, the dashboard was associated with significantly reduced time on task, mouse clicks, and mouse movement (each p < 0.001), with no significant increases in cognitive load nor task inaccuracy. Cognitive burden was higher for users with less experience, possibly partly attributable to usability issues identified during this study. Findings underscore the importance of assessing the usability, cognitive, and work load analysis during the design and implementation of health information technology applications.

Interactive Digital Health Tools to Engage Patients and Caregivers in Discharge Preparation: Implementation Study

BACKGROUND

Poor discharge preparation during hospitalization may lead to adverse events after discharge. Checklists and videos that systematically engage patients in preparing for discharge have the potential to improve safety, especially when integrated into clinician workflow via the electronic health record (EHR).

OBJECTIVE

This study aims to evaluate the implementation of a suite of digital health tools integrated with the EHR to engage hospitalized patients, caregivers, and their care team in preparing for discharge.

METHODS

We used the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework to identify pertinent research questions related to implementation. We iteratively refined patient and clinician-facing intervention components using a participatory process involving end users and institutional stakeholders. The intervention was implemented at a large academic medical center from December 2017 to July 2018. Patients who agreed to participate were coached to watch a discharge video, complete a checklist assessing discharge readiness, and request postdischarge text messaging with a physician 24 to 48 hours before their expected discharge date, which was displayed via a patient portal and bedside display. Clinicians could view concerns reported by patients based on their checklist responses in real time via a safety dashboard integrated with the EHR and choose to open a secure messaging thread with the patient for up to 7 days after discharge. We used mixed methods to evaluate our implementation experience.

RESULTS

Of 752 patient admissions, 510 (67.8%) patients or caregivers participated: 416 (55.3%) watched the video and completed the checklist, and 94 (12.5%) completed the checklist alone. On average, 4.24 concerns were reported per each of the 510 checklist submissions, most commonly about medications (664/2164, 30.7%) and follow-up (656/2164, 30.3%). Of the 510 completed checklists, a member of the care team accessed the safety dashboard to view 210 (41.2%) patient-reported concerns. For 422 patient admissions where postdischarge messaging was available, 141 (33.4%) patients requested this service; of these, a physician initiated secure messaging for 3 (2.1%) discharges. Most patient survey participants perceived that the intervention promoted self-management and communication with their care team. Patient interview participants endorsed gaps in communication with their care team and thought that the video and checklist would be useful closer toward discharge. Clinicians participating in focus groups perceived the value for patients but suggested that low awareness and variable workflow regarding the intervention, lack of technical optimization, and inconsistent clinician leadership limited the use of clinician-facing components.

CONCLUSIONS

A suite of EHR-integrated digital health tools to engage patients, caregivers, and clinicians in discharge preparation during hospitalization was feasible, acceptable, and valuable; however, important challenges were identified during implementation. We offer strategies to address implementation barriers and promote adoption of these tools.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03116074; https://clinicaltrials.gov/ct2/show/NCT03116074.

Application of human factors to improve usability of clinical decision support for diagnostic decision-making: a scenario-based simulation study

OBJECTIVE

In this study, we used human factors (HF) methods and principles to design a clinical decision support (CDS) that provides cognitive support to the pulmonary embolism (PE) diagnostic decision-making process in the emergency department. We hypothesised that the application of HF methods and principles will produce a more usable CDS that improves PE diagnostic decision-making, in particular decision about appropriate clinical pathway.

MATERIALS AND METHODS

We conducted a scenario-based simulation study to compare a HF-based CDS (the so-called CDS for PE diagnosis (PE-Dx CDS)) with a web-based CDS (MDCalc); 32 emergency physicians performed various tasks using both CDS. PE-Dx integrated HF design principles such as automating information acquisition and analysis, and minimising workload. We assessed all three dimensions of usability using both objective and subjective measures: effectiveness (eg, appropriate decision regarding the PE diagnostic pathway), efficiency (eg, time spent, perceived workload) and satisfaction (perceived usability of CDS).

RESULTS

Emergency physicians made more appropriate diagnostic decisions (94% with PE-Dx; 84% with web-based CDS; p<0.01) and performed experimental tasks faster with the PE-Dx CDS (on average 96 s per scenario with PE-Dx; 117 s with web-based CDS; p<0.001). They also reported lower workload (p<0.001) and higher satisfaction (p<0.001) with PE-Dx.

CONCLUSIONS

This simulation study shows that HF methods and principles can improve usability of CDS and diagnostic decision-making. Aspects of the HF-based CDS that provided cognitive support to emergency physicians and improved diagnostic performance included automation of information acquisition (eg, auto-populating risk scoring algorithms), minimisation of workload and support of decision selection (eg, recommending a clinical pathway). These HF design principles can be applied to the design of other CDS technologies to improve diagnostic safety.

Lessons learned implementing a complex and innovative patient safety learning laboratory project in a large academic medical center

OBJECTIVE

The objective of this paper is to share challenges, recommendations, and lessons learned regarding the development and implementation of a Patient Safety Learning Laboratory (PSLL) project, an innovative and complex intervention comprised of a suite of Health Information Technology (HIT) tools integrated with a newly implemented Electronic Health Record (EHR) vendor system in the acute care setting at a large academic center.

MATERIALS AND METHODS

The PSLL Administrative Core engaged stakeholders and study personnel throughout all phases of the project: problem analysis, design, development, implementation, and evaluation. Implementation challenges and recommendations were derived from direct observations and the collective experience of PSLL study personnel.

RESULTS

The PSLL intervention was implemented on 12 inpatient units during the 18-month study period, potentially impacting 12,628 patient admissions. Challenges to implementation included stakeholder engagement, project scope/complexity, technology/governance, and team structure. Recommendations to address each of these challenges were generated, some enacted during the trial, others as lessons learned for future iterative refinements of the intervention and its implementation.

CONCLUSION

Designing, implementing, and evaluating a suite of tools integrated within a vendor EHR to improve patient safety has a variety of challenges. Keys to success include continuous stakeholder engagement, involvement of systems and human factors engineers within a multidisciplinary team, an iterative approach to user-centered design, and a willingness to think outside of current workflows and processes to change health system culture around adverse event prevention.