Assessing the cognitive and work load of an inpatient safety dashboard in the context of opioid management

Developing, pilot testing, and refining requirements for 3 EHR-integrated interventions to improve diagnostic safety in acute care: a user-centered approach

Objective

To describe a user-centered approach to develop, pilot test, and refine requirements for 3 electronic health record (EHR)-integrated interventions that target key diagnostic process failures in hospitalized patients.

Materials and Methods

Three interventions were prioritized for development: a Diagnostic Safety Column (DSC) within an EHR-integrated dashboard to identify at-risk patients; a Diagnostic Time-Out (DTO) for clinicians to reassess the working diagnosis; and a Patient Diagnosis Questionnaire (PDQ) to gather patient concerns about the diagnostic process. Initial requirements were refined from analysis of test cases with elevated risk predicted by DSC logic compared to risk perceived by a clinician working group; DTO testing sessions with clinicians; PDQ responses from patients; and focus groups with clinicians and patient advisors using storyboarding to model the integrated interventions. Mixed methods analysis of participant responses was used to identify final requirements and potential implementation barriers.

Results

Final requirements from analysis of 10 test cases predicted by the DSC, 18 clinician DTO participants, and 39 PDQ responses included the following: DSC configurable parameters (variables, weights) to adjust baseline risk estimates in real-time based on new clinical data collected during hospitalization; more concise DTO wording and flexibility for clinicians to conduct the DTO with or without the patient present; and integration of PDQ responses into the DSC to ensure closed-looped communication with clinicians. Analysis of focus groups confirmed that tight integration of the interventions with the EHR would be necessary to prompt clinicians to reconsider the working diagnosis in cases with elevated diagnostic error (DE) risk or uncertainty. Potential implementation barriers included alert fatigue and distrust of the risk algorithm (DSC); time constraints, redundancies, and concerns about disclosing uncertainty to patients (DTO); and patient disagreement with the care team's diagnosis (PDQ).

Discussion

A user-centered approach led to evolution of requirements for 3 interventions targeting key diagnostic process failures in hospitalized patients at risk for DE.

Conclusions

We identify challenges and offer lessons from our user-centered design process.

Requirements and challenges of hospital dashboards: a systematic literature review

Background

Today, the use of data in administrative and clinical processes is quite challenging due to the large volume of data, data collection from various sources, and lack of data structure. As a data management tool, dashboards play an important role in timely visual display of critical information on key performances.

Objectives

This systematic review aimed to identify functional and non-functional requirements, as well as challenges of using dashboards in hospitals.

Methods

In this systematic review, four databases, including the Web of Science, PubMed, EMBASE, and Scopus, were searched to find relevant articles from 2000 until May 30, 2020. The final search was conducted on May 30, 2020. Data collection was performed using a data extraction form and reviewing the content of relevant studies on the potentials and challenges of dashboard implementation.

Results

Fifty-four out of 1254 retrieved articles were selected for this study based on the inclusion and exclusion criteria. The functional requirements for dashboards included reporting, reminders, customization, tracking, alert creation, and assessment of performance indicators. On the other hand, the non-functional requirements included the dashboard speed, security, ease of use, installation on different devices (e.g., PCs and laptops), integration with other systems, web-based design, inclusion of a data warehouse, being up-to-data, and use of data visualization elements based on the user’s needs. Moreover, the identified challenges were categorized into four groups: data sources, dashboard content, dashboard design, implementation, and integration in other systems at the hospital level.

Conclusion

Dashboards, by providing information in an appropriate manner, can lead to the proper use of information by users. In order for a dashboard to be effective in clinical and managerial processes, particular attention must be paid to its capabilities, and the challenges of its implementation need to be addressed.

Concepts and metrics of clinician attention: a scoping review protocol

INTRODUCTION

There is growing emphasis on the importance of both the cognitive and behavioural phenomenon of attention for clinicians engaged in patient care. Aspects of attention such as cognitive load, distraction and task switching have been studied in various settings with different methodologies. Using the protocol described here, we aim to systematically review the medical literature in order to map the concept of attention and to synthesise diverse concepts and methods under the broader category of research focused on 'attention'.

METHODS AND ANALYSIS

Following the methodology described by the Joanna Briggs Institute and Arksey and O'Malley, our scoping review conducts an iterative search of Cumulative Index of Nursing and Allied Health Literature (CINAHL), Medline (PubMed) and EMBASE (Ovid). An initial limited search based on key concepts and terminology will generate relevant articles which in turn will be mined for additional keywords and index terms to guide a formal literature search. Our multidisciplinary team will extract data into a matrix, including a small random sample of the same studies (to ensure concordance), and present the results in a descriptive narrative format.

ETHICS AND DISSEMINATION

As a secondary analysis, our study does not require ethics approval, and we will ensure that included studies have appropriate approval. We anticipate results will identify diverse ways of conceptualising clinician attention and will provide a foundation for developing additional metrics and study methods to optimise attention in the clinical environment. We will disseminate results through journals and conferences and coordinate with colleagues doing work in adjacent fields.

Toward a Learning Health Care System: A Systematic Review and Evidence-Based Conceptual Framework for Implementation of Clinical Analytics in a Digital Hospital

Objective  A learning health care system (LHS) uses routinely collected data to continuously monitor and improve health care outcomes. Little is reported on the challenges and methods used to implement the analytics underpinning an LHS. Our aim was to systematically review the literature for reports of real-time clinical analytics implementation in digital hospitals and to use these findings to synthesize a conceptual framework for LHS implementation.

Methods  Embase, PubMed, and Web of Science databases were searched for clinical analytics derived from electronic health records in adult inpatient and emergency department settings between 2015 and 2021. Evidence was coded from the final study selection that related to (1) dashboard implementation challenges, (2) methods to overcome implementation challenges, and (3) dashboard assessment and impact. The evidences obtained, together with evidence extracted from relevant prior reviews, were mapped to an existing digital health transformation model to derive a conceptual framework for LHS analytics implementation.

Results  A total of 238 candidate articles were reviewed and 14 met inclusion criteria. From the selected studies, we extracted 37 implementation challenges and 64 methods employed to overcome such challenges. We identified common approaches for evaluating the implementation of clinical dashboards. Six studies assessed clinical process outcomes and only four studies evaluated patient health outcomes. A conceptual framework for implementing the analytics of an LHS was developed.

Conclusion  Health care organizations face diverse challenges when trying to implement real-time data analytics. These challenges have shifted over the past decade. While prior reviews identified fundamental information problems, such as data size and complexity, our review uncovered more postpilot challenges, such as supporting diverse users, workflows, and user-interface screens. Our review identified practical methods to overcome these challenges which have been incorporated into a conceptual framework. It is hoped this framework will support health care organizations deploying near-real-time clinical dashboards and progress toward an LHS.

An analysis of the structure and content of dashboards used to monitor patient safety in the inpatient setting

The objective of this study is to review and compare patient safety dashboards used by hospitals and identify similarities and differences in their design, format, and scope. We reviewed design features of electronic copies of patient safety dashboards from a representative sample of 10 hospitals. The results show great heterogeneity in the format, presentation, and scope of patient safety dashboards. Hospitals varied in their use of performance indicators (targets, trends, and benchmarks), style of color coding, and timeframe for the displayed metrics. The average number of metrics per dashboard display was 28, with a wide range from 7 to 84. Given the large variation in dashboard design, there is a need for future work to assess which approaches are associated with the best outcomes, and how specific elements contribute to usability, to help customize dashboards to meet the needs of different clinical, and operational stakeholders.

Human Factors and Organizational Issues in Health Informatics: Innovations and Opportunities

OBJECTIVE

Human factors and ergonomics (HF/E) frameworks and methods are becoming embedded in the health informatics community. There is now broad recognition that health informatics tools must account for the diverse needs, characteristics, and abilities of end users, as well as their context of use. The objective of this review is to synthesize the current nature and scope of HF/E integration into the health informatics community.

METHODS

Because the focus of this synthesis is on understanding the current integration of the HF/E and health informatics research communities, we manually reviewed all manuscripts published in primary HF/E and health informatics journals during 2020.

RESULTS

HF/E-focused health informatics studies included in this synthesis focused heavily on EHR customizations, specifically clinical decision support customizations and customized data displays, and on mobile health innovations. While HF/E methods aimed to jointly improve end user safety, performance, and satisfaction, most HF/E-focused health informatics studies measured only end user satisfaction.

CONCLUSION

HF/E-focused health informatics researchers need to identify and communicate methodological standards specific to health informatics, to better synthesize findings across resource intensive HF/E-focused health informatics studies. Important gaps in the HF/E design and evaluation process should be addressed in future work, including support for technology development platforms and training programs so that health informatics designers are as diverse as end users.

Electronic health record usability and workload changes over time for provider and nursing staff following transition to new EHR

The ubiquity of EHRs in healthcare means that small EHR inefficiencies can have a major impact on clinician workload. We conducted a sequential explanatory mixed methods study to: 1) identify EHR-associated workload and usability effects for clinicians following an EHR change over time, 2) determine workload and usability differences for providers (MD and Advance Practice Nurses) versus nurses (RNs and MAs), 3) determine if usability predicts workload, 4) identify potential sources of EHR design flaws. Workload (NASA-Task Load Index) and usability (System Usability Scale) measures were administered pre, 6-8 month and 30-32 months post-implementation. We found significant increase in perceived workload post-implementation that persisted for 2.5 years (p < .001). The workload increase was associated with usability ratings, which in turn may relate to EHR interface design violations identified by a heuristic evaluation. Our findings suggest further innovation and attention to interface design flaws are needed to improve EHR usability and reduce clinician workload.

Remembering Ben-Tzion Karsh's scholarship, impact, and legacy

Dr. Ben-Tzion (Bentzi) Karsh was a mentor, collaborator, colleague, and friend who profoundly impacted the fields of human factors and ergonomics (HFE), medical informatics, patient safety, and primary care, among others. In this paper we honor his contributions by reflecting on his scholarship, impact, and legacy in three ways: first, through an updated simplified bibliometric analysis in 2020, highlighting the breadth of his scholarly impact from the perspective of the number and types of communities and collaborators with which and whom he engaged; second, through targeted reflections on the history and impact of Dr. Karsh's most cited works, commenting on the particular ways they impacted our academic community; and lastly, through quotes from collaborators and mentees, illustrating Dr. Karsh's long-lasting impact on his contemporaries and students.