Evaluation of an integrated intensive care unit monitoring display by critical care fellow physicians

Centralized Multipatient Dashboards' Impact on Intensive Care Unit Clinician Performance and Satisfaction: A Systematic Review

BACKGROUND

 Intensive care unit (ICU) clinicians encounter frequent challenges with managing vast amounts of fragmented data while caring for multiple critically ill patients simultaneously. This may lead to increased provider cognitive load that may jeopardize patient safety.

OBJECTIVES

 This systematic review assesses the impact of centralized multipatient dashboards on ICU clinician performance, perceptions regarding the use of these tools, and patient outcomes.

METHODS

 A literature search was conducted on February 9, 2023, using the EBSCO CINAHL, Cochrane Central Register of Controlled Trials, Embase, IEEE Xplore, MEDLINE, Scopus, and Web of Science Core Collection databases. Eligible studies that included ICU clinicians as participants and tested the effect of dashboards designed for use by multiple users to manage multiple patients on user performance and/or satisfaction compared with the standard practice. We narratively synthesized eligible studies following the SWiM (Synthesis Without Meta-analysis) guidelines. Studies were grouped based on dashboard type and outcomes assessed.

RESULTS

 The search yielded a total of 2,407 studies. Five studies met inclusion criteria and were included. Among these, three studies evaluated interactive displays in the ICU, one study assessed two dashboards in the pediatric ICU (PICU), and one study examined centralized monitor in the PICU. Most studies reported several positive outcomes, including reductions in data gathering time before rounds, a decrease in misrepresentations during multidisciplinary rounds, improved daily documentation compliance, faster decision-making, and user satisfaction. One study did not report any significant association.

CONCLUSION

 The multipatient dashboards were associated with improved ICU clinician performance and were positively perceived in most of the included studies. The risk of bias was high, and the certainty of evidence was very low, due to inconsistencies, imprecision, indirectness in the outcome measure, and methodological limitations. Designing and evaluating multipatient tools using robust research methodologies is an important focus for future research.

Avatar-based patient monitoring improves information transfer, diagnostic confidence and reduces perceived workload in intensive care units: computer-based, multicentre comparison study

Patient monitoring is the foundation of intensive care medicine. High workload and information overload can impair situation awareness of staff, thus leading to loss of important information about patients' conditions. To facilitate mental processing of patient monitoring data, we developed the Visual-Patient-avatar Intensive Care Unit (ICU), a virtual patient model animated from vital signs and patient installation data. It incorporates user-centred design principles to foster situation awareness. This study investigated the avatar's effects on information transfer measured by performance, diagnostic confidence and perceived workload. This computer-based study compared Visual-Patient-avatar ICU and conventional monitor modality for the first time. We recruited 25 nurses and 25 physicians from five centres. The participants completed an equal number of scenarios in both modalities. Information transfer, as the primary outcome, was defined as correctly assessing vital signs and installations. Secondary outcomes included diagnostic confidence and perceived workload. For analysis, we used mixed models and matched odds ratios. Comparing 250 within-subject cases revealed that Visual-Patient-avatar ICU led to a higher rate of correctly assessed vital signs and installations [rate ratio (RR) 1.25; 95% CI 1.19-1.31; P < 0.001], strengthened diagnostic confidence [odds ratio (OR) 3.32; 95% CI 2.15-5.11, P < 0.001] and lowered perceived workload (coefficient - 7.62; 95% CI - 9.17 to - 6.07; P < 0.001) than conventional modality. Using Visual-Patient-avatar ICU, participants retrieved more information with higher diagnostic confidence and lower perceived workload compared to the current industry standard monitor.

Novel Interface Designs for Patient Monitoring Applications in Critical Care Medicine: Human Factors Review

Background

The patient monitor (PM) is one of the most commonly used medical devices in hospitals worldwide. PMs are used to monitor patients’ vital signs in a wide variety of patient care settings, especially in critical care settings, such as intensive care units. An interesting observation is that the design of PMs has not significantly changed over the past 2 decades, with the layout and structure of PMs more or less unchanged, with incremental changes in design being made rather than transformational changes. Thus, we believe it well-timed to review the design of novel PM interfaces, with particular reference to usability and human factors.

Objective

This paper aims to review innovations in PM design proposed by researchers and explore how clinicians responded to these design changes.

Methods

A literature search of relevant databases, following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, identified 16 related studies. A detailed description of the interface design and an analysis of each novel PM were carried out, including a detailed analysis of the structure of the different user interfaces, to inform future PM design. The test methodologies used to evaluate the different designs are also presented.

Results

Most of the studies included in this review identified some level of improvement in the clinician’s performance when using a novel display in comparison with the traditional PM. For instance, from the 16 reviewed studies, 12 studies identified an improvement in the detection and response times, and 10 studies identified an improvement in the accuracy or treatment efficiency. This indicates that novel displays have the potential to improve the clinical performance of nurses and doctors. However, the outcomes of some of these studies are weakened because of methodological deficiencies. These deficiencies are discussed in detail in this study.

Conclusions

More careful study design is warranted to investigate the user experience and usability of future novel PMs for real time vital sign monitoring, to establish whether or not they could be used successfully in critical care. A series of recommendations on how future novel PM designs and evaluations can be enhanced are provided.

Patient information organization in the intensive care setting: expert knowledge elicitation with card sorting methods

Introduction

Many electronic health records fail to support information uptake because they impose low-level information organization tasks on users. Clinical concept-oriented views have shown information processing improvements, but the specifics of this organization for critical care are unclear.

Objective

To determine high-level cognitive processes and patient information organization schema in critical care.

Methods

We conducted an open card sort of 29 patient data elements and a modified Delphi card sort of 65 patient data elements. Study participants were 39 clinicians with varied critical care training and experience. We analyzed the open sort with a hierarchical cluster analysis (HCA) and factor analysis (FA). The Delphi sort was split into three initiating groups that resulted in three unique solutions. We compared results between open sort analyses (HCA and FA), between card sorting exercises (open and Delphi), and across the Delphi solutions.

Results

Between the HCA and FA, we observed common constructs including cardiovascular and hemodynamics, infectious disease, medications, neurology, patient overview, respiratory, and vital signs. The more comprehensive Delphi sort solutions also included gastrointestinal, renal, and imaging constructs.

Conclusions

We identified primarily system-based groupings (e.g., cardiovascular, respiratory). Source-based (e.g., medications, laboratory) groups became apparent when participants were asked to sort a longer list of concepts. These results suggest a hybrid approach to information organization, which may combine systems, source, or problem-based groupings, best supports clinicians' mental models. These results can contribute to the design of information displays to better support clinicians' access and interpretation of information for critical care decisions.

Critical care information display approaches and design frameworks: A systematic review and meta-analysis

OBJECTIVE

To systematically review original user evaluations of patient information displays relevant to critical care and understand the impact of design frameworks and information presentation approaches on decision-making, efficiency, workload, and preferences of clinicians.

METHODS

We included studies that evaluated information displays designed to support real-time care decisions in critical care or anesthesiology using simulated tasks. We searched PubMed and IEEExplore from 1/1/1990 to 6/30/2018. The search strategy was developed iteratively with calibration against known references. Inclusion screening was completed independently by two authors. Extraction of display features, design processes, and evaluation method was completed by one and verified by a second author.

RESULTS

Fifty-six manuscripts evaluating 32 critical care and 22 anesthesia displays were included. Primary outcome metrics included clinician accuracy and efficiency in recognizing, diagnosing, and treating problems. Implementing user-centered design (UCD) processes, especially iterative evaluation and redesign, resulted in positive impact in outcomes such as accuracy and efficiency. Innovative display approaches that led to improved human-system performance in critical care included: (1) improving the integration and organization of information, (2) improving the representation of trend information, and (3) implementing graphical approaches to make relationships between data visible.

CONCLUSION

Our review affirms the value of key principles of UCD. Improved information presentation can facilitate faster information interpretation and more accurate diagnoses and treatment. Improvements to information organization and support for rapid interpretation of time-based relationships between related quantitative data is warranted. Designers and developers are encouraged to involve users in formal iterative design and evaluation activities in the design of electronic health records (EHRs), clinical informatics applications, and clinical devices.

Association of Data Integration Technologies With Intensive Care Clinician Performance: A Systematic Review and Meta-analysis

IMPORTANCE

Sources of data in the intensive care setting are increasing exponentially, but the benefits of displaying multiparametric, high-frequency data are unknown. Decision making may not benefit from this technology if clinicians remain cognitively overburdened by poorly designed data integration and visualization technologies (DIVTs).

OBJECTIVE

To systematically review and summarize the published evidence on the association of user-centered DIVTs with intensive care clinician performance.

DATA SOURCES

MEDLINE, Embase, Cochrane Central Register of Controlled Trials, PsycINFO, and Web of Science were searched in May 2014 and January 2018.

STUDY SELECTION

Studies had 3 requirements: (1) the study tested a viable DIVT, (2) participants involved were intensive care clinicians, and (3) the study reported quantitative results associated with decision making in an intensive care setting.

DATA EXTRACTION AND SYNTHESIS

Of 252 records screened, 20 studies, published from 2004 to 2016, were included. The human factors framework to assess health technologies was applied to measure study completeness, and the Quality Assessment Instrument was used to assess the quality of the studies. PRISMA guidelines were adapted to conduct the systematic review and meta-analysis.

MAIN OUTCOMES AND MEASURES

Study completeness and quality; clinician performance; physical, mental, and temporal demand; effort; frustration; time to decision; and decision accuracy.

RESULTS

Of the 20 included studies, 16 were experimental studies with 410 intensive care clinician participants and 4 were survey-based studies with 1511 respondents. Scores for study completeness ranged from 27 to 43, with a maximum score of 47, and scores for study quality ranged from 46 to 79, with a maximum score of 90. Of 20 studies, DIVTs were evaluated in clinical settings in 2 studies (10%); time to decision was measured in 14 studies (70%); and decision accuracy was measured in 11 studies (55%). Measures of cognitive workload pooled in the meta-analysis suggested that any DIVT was an improvement over paper-based data in terms of self-reported performance, mental and temporal demand, and effort. With a maximum score of 22, median (IQR) mental demand scores for electronic display were 10 (7-13), tabular display scores were 8 (6.0-11.5), and novel visualization scores were 8 (6-12), compared with 17 (14-19) for paper. The median (IQR) temporal demand scores were also lower for all electronic visualizations compared with paper, with scores of 8 (6-11) for electronic display, 7 (6-11) for tabular and bar displays, 7 (5-11) for novel visualizations, and 16 (14.3-19.0) for paper. The median (IQR) performance scores improved for all electronic visualizations compared with paper (lower score indicates better self-reported performance), with scores of 6 (3-11) for electronic displays, 6 (4-11) for tabular and bar displays, 6 (4-11) for novel visualizations, and 14 (11-16) for paper. Frustration and physical demand domains of cognitive workload did not change, and differences between electronic displays were not significant.

CONCLUSIONS AND RELEVANCE

This review suggests that DIVTs are associated with increased integration and consistency of data. Much work remains to identify which visualizations effectively reduce cognitive workload to enhance decision making based on intensive care data. Standardizing human factors testing by developing a repository of open access benchmarked test protocols, using a set of outcome measures, scenarios, and data sets, may accelerate the design and selection of the most appropriate DIVT.

An exploratory clinical evaluation of a head-worn display based multiple-patient monitoring application: impact on supervising anesthesiologists' situation awareness

PURPOSE

Supervising anesthesiologists overseeing several operating rooms must be aware of the status of multiple patients, so they can consult with the anesthetist in single operating rooms or respond quickly to critical events. However, maintaining good situation awareness can be challenging when away from patient bedsides or a central monitoring station. In this proof-of-concept study, we evaluated the potential of a head-worn display that showed multiple patients' vital signs and alarms to improve supervising anesthesiologists' situation awareness.

METHODS

Eight supervising anesthesiologists each monitored the vital signs of patients in six operating rooms for 3 h with the head-worn display, and for another 3 h without the head-worn display. In interviews with each anesthesiologist, we assessed in which situations the head-worn display was used and whether the continuous availability of the vital signs improved situation awareness. We also measured situation awareness quantitatively from six of the eight anesthesiologists, by instructing them to press a button whenever they noticed a patient alarm.

RESULTS

The median number of patient alarms occurring was similar when the anesthesiologists monitored with the head-worn display (42.0) and without the head-worn display (40.5). However, the anesthesiologists noticed significantly more patient alarms with the head-worn display (66.7%) than without (7.1%), P = 0.028, and they reported improved situation awareness with the head-worn display. The head-worn display helped the anesthesiologists to perceive and comprehend patients' current status and to anticipate future developments. A negative effect of the head-worn display was its tendency to distract during demanding procedures.

CONCLUSIONS

Head-worn displays can improve supervising anesthesiologists' situation awareness in multiple-patient monitoring situations. The anesthesiologists who participated in the study expressed enthusiasm about monitoring patients with a head-worn display and wished to use and evaluate it further.

Informatics for the Modern Intensive Care Unit

Advanced informatics systems can help improve health care delivery and the environment of care for critically ill patients. However, identifying, testing, and deploying advanced informatics systems can be quite challenging. These processes often require involvement from a collaborative group of health care professionals of varied disciplines with knowledge of the complexities related to designing the modern and "smart" intensive care unit (ICU). In this article, we explore the connectivity environment within the ICU, middleware technologies to address a host of patient care initiatives, and the core informatics concepts necessary for both the design and implementation of advanced informatics systems.

"Usability of data integration and visualization software for multidisciplinary pediatric intensive care: a human factors approach to assessing technology"

Background

Intensive care clinicians use several sources of data in order to inform decision-making. We set out to evaluate a new interactive data integration platform called T3™ made available for pediatric intensive care. Three primary functions are supported: tracking of physiologic signals, displaying trajectory, and triggering decisions, by highlighting data or estimating risk of patient instability. We designed a human factors study to identify interface usability issues, to measure ease of use, and to describe interface features that may enable or hinder clinical tasks.

Methods

Twenty-two participants, consisting of bedside intensive care physicians, nurses, and respiratory therapists, tested the T3™ interface in a simulation laboratory setting. Twenty tasks were performed with a true-to-setting, fully functional, prototype, populated with physiological and therapeutic intervention patient data. Primary data visualization was time series and secondary visualizations were: 1) shading out-of-target values, 2) mini-trends with exaggerated maxima and minima (sparklines), and 3) bar graph of a 16-parameter indicator. Task completion was video recorded and assessed using a use error rating scale. Usability issues were classified in the context of task and type of clinician. A severity rating scale was used to rate potential clinical impact of usability issues.

Results

Time series supported tracking a single parameter but partially supported determining patient trajectory using multiple parameters. Visual pattern overload was observed with multiple parameter data streams. Automated data processing using shading and sparklines was often ignored but the 16-parameter data reduction algorithm, displayed as a persistent bar graph, was visually intuitive. However, by selecting or automatically processing data, triggering aids distorted the raw data that clinicians use regularly. Consequently, clinicians could not rely on new data representations because they did not know how they were established or derived.

Conclusions

Usability issues, observed through contextual use, provided directions for tangible design improvements of data integration software that may lessen use errors and promote safe use. Data-driven decision making can benefit from iterative interface redesign involving clinician-users in simulated environments. This study is a first step in understanding how software can support clinicians’ decision making with integrated continuous monitoring data. Importantly, testing of similar platforms by all the different disciplines who may become clinician users is a fundamental step necessary to understand the impact on clinical outcomes of decision aids.

Electronic supplementary material

The online version of this article (doi:10.1186/s12911-017-0520-7) contains supplementary material, which is available to authorized users.

Driven to distraction: The nature and apparent purpose of interruptions in critical care and implications for HIT

OBJECTIVES

To examine the apparent purpose of interruptions in a Pediatric Intensive Care Unit and opportunities to reduce their burden with informatics solutions.

MATERIALS AND METHODS

In this prospective observational study, researchers shadowed clinicians in the unit for one hour at a time, recording all interruptions participating clinicians experienced or initiated, their starting time, duration, and a short description that could help to infer their apparent purpose. All captured interruptions were classified inductively on their source and apparent purpose and on the optimal representational media for fulfilling their apparent purpose.

RESULTS

The researchers observed thirty-four one-hour sessions with clinicians in the unit, including 21 nurses and 13 residents and house physicians. The physicians were interrupted on average 11.9 times per hour and interrupted others 8.8 times per hour. Nurses were interrupted 8.6 times per hour and interrupted others 5.1 times per hour. The apparent purpose of interruptions included Information Seeking and Sharing (n=259, 46.3%), Directives and Requests (n=70, 12%), Shared Decision-Making (n=49, 8.8%), Direct Patient Care (n=36, 6.4%), Social (n=71, 12.7%), Device Alarms (n=28, 5%), and Non-Clinical (n=10, 1.8%); 6.6% were not classified due to insufficient description. Of all captured interruptions, 29.5% were classified as being better served with informational displays or computer-mediated communication.

CONCLUSIONS

Deeper understanding of the purpose of interruptions in critical care can help to distinguish between interruptions that require face-to-face conversation and those that can be eliminated with informatics solutions. The proposed taxonomy of interruptions and representational analysis can be used to further advance the science of interruptions in clinical care.

A Review of Visual Representations of Physiologic Data

Background

Physiological data is derived from electrodes attached directly to patients. Modern patient monitors are capable of sampling data at frequencies in the range of several million bits every hour. Hence the potential for cognitive threat arising from information overload and diminished situational awareness becomes increasingly relevant. A systematic review was conducted to identify novel visual representations of physiologic data that address cognitive, analytic, and monitoring requirements in critical care environments.

Objective

The aims of this review were to identify knowledge pertaining to (1) support for conveying event information via tri-event parameters; (2) identification of the use of visual variables across all physiologic representations; (3) aspects of effective design principles and methodology; (4) frequency of expert consultations; (5) support for user engagement and identifying heuristics for future developments.

Methods

A review was completed of papers published as of August 2016. Titles were first collected and analyzed using an inclusion criteria. Abstracts resulting from the first pass were then analyzed to produce a final set of full papers. Each full paper was passed through a data extraction form eliciting data for comparative analysis.

Results

In total, 39 full papers met all criteria and were selected for full review. Results revealed great diversity in visual representations of physiological data. Visual representations spanned 4 groups including tabular, graph-based, object-based, and metaphoric displays. The metaphoric display was the most popular (n=19), followed by waveform displays typical to the single-sensor-single-indicator paradigm (n=18), and finally object displays (n=9) that utilized spatiotemporal elements to highlight changes in physiologic status. Results obtained from experiments and evaluations suggest specifics related to the optimal use of visual variables, such as color, shape, size, and texture have not been fully understood. Relationships between outcomes and the users’ involvement in the design process also require further investigation. A very limited subset of visual representations (n=3) support interactive functionality for basic analysis, while only one display allows the user to perform analysis including more than one patient.

Conclusions

Results from the review suggest positive outcomes when visual representations extend beyond the typical waveform displays; however, there remain numerous challenges. In particular, the challenge of extensibility limits their applicability to certain subsets or locations, challenge of interoperability limits its expressiveness beyond physiologic data, and finally the challenge of instantaneity limits the extent of interactive user engagement.

Full of sound and fury, signifying nothing: burden of transient noncritical monitor alarms in a trauma resuscitation unit

We examined the types of patient monitor alarms encountered in the trauma resuscitation unit of a major level 1 trauma center. Over a 1-year period, 316688 alarms were recorded for 6701 trauma patients (47 alarms/patient). Alarms were more frequent among patients with a Glasgow Coma Scale of 8 or less. Only 2.4% of all alarms were classified as "patient crisis," with the rest in the presumably less critical categories "patient advisory," "patient warning," and "system warning." Nearly half of alarms were ≤5 seconds in duration. In this patient population, a 2-second delay would reduce alarms by 25%, and a delay of 5 seconds would reduce all alarms by 49%.

Innovative designs for the smart ICU: Part 3: Advanced ICU informatics

This third and final installment of this series on innovative designs for the smart ICU addresses the steps involved in conceptualizing, actualizing, using, and maintaining the advanced ICU informatics infrastructure and systems. The smart ICU comprehensively and electronically integrates the patient in the ICU with all aspects of care, displays data in a variety of formats, converts data to actionable information, uses data proactively to enhance patient safety, and monitors the ICU environment to facilitate patient care and ICU management. The keys to success in this complex informatics design process include an understanding of advanced informatics concepts, sophisticated planning, installation of a robust infrastructure capable of both connectivity and interoperability, and implementation of middleware solutions that provide value. Although new technologies commonly appear compelling, they are also complicated and challenging to incorporate within existing or evolving hospital informatics systems. Therefore, careful analysis, deliberate testing, and a phased approach to the implementation of innovative technologies are necessary to achieve the multilevel solutions of the smart ICU.

Human Factors Issues in the Intensive Care Unit

In the context of an aging population, more critically ill patients, and a change in intensive care unit (ICU) services stemming from advances in technology, prevalent medical errors and staff burnout in the ICU are not surprising. The ICU provides ample opportunity for human factors experts to apply their knowledge about the strengths and weaknesses of human capabilities to design more effective care delivery. Human factors experts design work processes, technology, and environmental factors to effectively and constructively channel the attention and behavior of those providing care; a few areas of focus can have marked impacts on care delivery and patient outcomes. In this review, we focus on these 3 areas and investigate the solutions and problems addressed by previous research.

The proportion of clinically relevant alarms decreases as patient clinical severity decreases in intensive care units: a pilot study

OBJECTIVES

To determine (1) the proportion and number of clinically relevant alarms based on the type of monitoring device; (2) whether patient clinical severity, based on the sequential organ failure assessment (SOFA) score, affects the proportion of clinically relevant alarms and to suggest; (3) methods for reducing clinically irrelevant alarms in an intensive care unit (ICU).

DESIGN

A prospective, observational clinical study.

SETTING

A medical ICU at the University of Tokyo Hospital in Tokyo, Japan.

PARTICIPANTS

All patients who were admitted directly to the ICU, aged ≥18 years, and not refused active treatment were registered between January and February 2012.

METHODS

The alarms, alarm settings, alarm messages, waveforms and video recordings were acquired in real time and saved continuously. All alarms were annotated with respect to technical and clinical validity.

RESULTS

18 ICU patients were monitored. During 2697 patient-monitored hours, 11 591 alarms were annotated. Only 740 (6.4%) alarms were considered to be clinically relevant. The monitoring devices that triggered alarms the most often were the direct measurement of arterial pressure (33.5%), oxygen saturation (24.2%), and electrocardiogram (22.9%). The numbers of relevant alarms were 12.4% (direct measurement of arterial pressure), 2.4% (oxygen saturation) and 5.3% (electrocardiogram). Positive correlations were established between patient clinical severities and the proportion of relevant alarms. The total number of irrelevant alarms could be reduced by 21.4% by evaluating their technical relevance.

CONCLUSIONS

We demonstrated that (1) the types of devices that alarm the most frequently were direct measurements of arterial pressure, oxygen saturation and ECG, and most of those alarms were not clinically relevant; (2) the proportion of clinically relevant alarms decreased as the patients' status improved and (3) the irrelevance alarms can be considerably reduced by evaluating their technical relevance.