Novel displays of patient information in critical care settings: a systematic review

The role of information systems in emergency department decision-making-a literature review

OBJECTIVES

Healthcare providers employ heuristic and analytical decision-making to navigate the high-stakes environment of the emergency department (ED). Despite the increasing integration of information systems (ISs), research on their efficacy is conflicting. Drawing on related fields, we investigate how timing and mode of delivery influence IS effectiveness. Our objective is to reconcile previous contradictory findings, shedding light on optimal IS design in the ED.

MATERIALS AND METHODS

We conducted a systematic review following PRISMA across PubMed, Scopus, and Web of Science. We coded the ISs' timing as heuristic or analytical, their mode of delivery as active for automatic alerts and passive when requiring user-initiated information retrieval, and their effect on process, economic, and clinical outcomes.

RESULTS

Our analysis included 83 studies. During early heuristic decision-making, most active interventions were ineffective, while passive interventions generally improved outcomes. In the analytical phase, the effects were reversed. Passive interventions that facilitate information extraction consistently improved outcomes.

DISCUSSION

Our findings suggest that the effectiveness of active interventions negatively correlates with the amount of information received during delivery. During early heuristic decision-making, when information overload is high, physicians are unresponsive to alerts and proactively consult passive resources. In the later analytical phases, physicians show increased receptivity to alerts due to decreased diagnostic uncertainty and information quantity. Interventions that limit information lead to positive outcomes, supporting our interpretation.

CONCLUSION

We synthesize our findings into an integrated model that reveals the underlying reasons for conflicting findings from previous reviews and can guide practitioners in designing ISs in the ED.

Centralized Multipatient Dashboards' Impact on Intensive Care Unit Clinician Performance and Satisfaction: A Systematic Review

BACKGROUND

 Intensive care unit (ICU) clinicians encounter frequent challenges with managing vast amounts of fragmented data while caring for multiple critically ill patients simultaneously. This may lead to increased provider cognitive load that may jeopardize patient safety.

OBJECTIVES

 This systematic review assesses the impact of centralized multipatient dashboards on ICU clinician performance, perceptions regarding the use of these tools, and patient outcomes.

METHODS

 A literature search was conducted on February 9, 2023, using the EBSCO CINAHL, Cochrane Central Register of Controlled Trials, Embase, IEEE Xplore, MEDLINE, Scopus, and Web of Science Core Collection databases. Eligible studies that included ICU clinicians as participants and tested the effect of dashboards designed for use by multiple users to manage multiple patients on user performance and/or satisfaction compared with the standard practice. We narratively synthesized eligible studies following the SWiM (Synthesis Without Meta-analysis) guidelines. Studies were grouped based on dashboard type and outcomes assessed.

RESULTS

 The search yielded a total of 2,407 studies. Five studies met inclusion criteria and were included. Among these, three studies evaluated interactive displays in the ICU, one study assessed two dashboards in the pediatric ICU (PICU), and one study examined centralized monitor in the PICU. Most studies reported several positive outcomes, including reductions in data gathering time before rounds, a decrease in misrepresentations during multidisciplinary rounds, improved daily documentation compliance, faster decision-making, and user satisfaction. One study did not report any significant association.

CONCLUSION

 The multipatient dashboards were associated with improved ICU clinician performance and were positively perceived in most of the included studies. The risk of bias was high, and the certainty of evidence was very low, due to inconsistencies, imprecision, indirectness in the outcome measure, and methodological limitations. Designing and evaluating multipatient tools using robust research methodologies is an important focus for future research.

A Novel Continuous Real-Time Vital Signs Viewer for Intensive Care Units: Design and Evaluation Study

BACKGROUND

Clinicians working in intensive care units (ICUs) are immersed in a cacophony of alarms and a relentless onslaught of data. Within this frenetic environment, clinicians make high-stakes decisions using many data sources and are often oversaturated with information of varying quality. Traditional bedside monitors only depict static vital signs data, and these data are not easily viewable remotely. Clinicians must rely on separate nursing charts-handwritten or electric-to review physiological patterns, including signs of potential clinical deterioration. An automated physiological data viewer has been developed to provide at-a-glance summaries and to assist with prioritizing care for multiple patients who are critically ill.

OBJECTIVE

This study aims to evaluate a novel vital signs viewer system in a level 1 trauma center by subjectively assessing the viewer's utility in a high-volume ICU setting.

METHODS

ICU attendings were surveyed during morning rounds. Physicians were asked to conduct rounds normally, using data reported from nurse charts and briefs from fellows to inform their clinical decisions. After the physician finished their assessment and plan for the patient, they were asked to complete a questionnaire. Following completion of the questionnaire, the viewer was presented to ICU physicians on a tablet personal computer that displayed the patient's physiologic data (ie, shock index, blood pressure, heart rate, temperature, respiratory rate, and pulse oximetry), summarized for up to 72 hours. After examining the viewer, ICU physicians completed a postview questionnaire. In both questionnaires, the physicians were asked questions regarding the patient's stability, status, and need for a higher or lower level of care. A hierarchical clustering analysis was used to group participating ICU physicians and assess their general reception of the viewer.

RESULTS

A total of 908 anonymous surveys were collected from 28 ICU physicians from February 2015 to June 2017. Regarding physicians' perception of whether the viewer enhanced the ability to assess multiple patients in the ICU, 5% (45/908) strongly agreed, 56.6% (514/908) agreed, 35.3% (321/908) were neutral, 2.9% (26/908) disagreed, and 0.2% (2/908) strongly disagreed.

CONCLUSIONS

Morning rounds in a trauma center ICU are conducted in a busy environment with many data sources. This study demonstrates that organized physiologic data and visual assessment can improve situation awareness, assist clinicians with recognizing changes in patient status, and prioritize care.

Information displays for automated surveillance algorithms of in-hospital patient deterioration: a scoping review

OBJECTIVE

Surveillance algorithms that predict patient decompensation are increasingly integrated with clinical workflows to help identify patients at risk of in-hospital deterioration. This scoping review aimed to identify the design features of the information displays, the types of algorithm that drive the display, and the effect of these displays on process and patient outcomes.

MATERIALS AND METHODS

The scoping review followed Arksey and O'Malley's framework. Five databases were searched with dates between January 1, 2009 and January 26, 2022. Inclusion criteria were: participants-clinicians in inpatient settings; concepts-intervention as deterioration information displays that leveraged automated AI algorithms; comparison as usual care or alternative displays; outcomes as clinical, workflow process, and usability outcomes; and context as simulated or real-world in-hospital settings in any country. Screening, full-text review, and data extraction were reviewed independently by 2 researchers in each step. Display categories were identified inductively through consensus.

RESULTS

Of 14 575 articles, 64 were included in the review, describing 61 unique displays. Forty-one displays were designed for specific deteriorations (eg, sepsis), 24 provided simple alerts (ie, text-based prompts without relevant patient data), 48 leveraged well-accepted score-based algorithms, and 47 included nurses as the target users. Only 1 out of the 10 randomized controlled trials reported a significant effect on the primary outcome.

CONCLUSIONS

Despite significant advancements in surveillance algorithms, most information displays continue to leverage well-understood, well-accepted score-based algorithms. Users' trust, algorithmic transparency, and workflow integration are significant hurdles to adopting new algorithms into effective decision support tools.

The Intensive Care Unit Bundle Board: A Novel Real-Time Data Visualization Tool to Improve Maintenance Care for Invasive Catheters

BACKGROUND

Critically ill patients are at greater risk of healthcare-associated infections (HAIs). The use of maintenance bundles helps to reduce this risk but also generates a rapid accumulation of complex data that is difficult to aggregate and subsequently act upon.

OBJECTIVES

We hypothesized that a digital display summarizing nursing documentation of invasive catheters (including central venous access devices, arterial catheters, and urinary catheters) would improve invasive device maintenance care and documentation. Our secondary objectives were to see if this summary would reduce the duration of problematic conditions, that is, characteristics associated with increased risk of infection.

METHODS

We developed and implemented a data visualization tool called the "Bundle Board" to display nursing observations on invasive devices. The intervention was studied in a 28-bed medical intensive care unit (MICU). The Bundle Board was piloted for 6 weeks in June 2022 and followed by a comparison phase, where one MICU had Bundle Board access and another MICU at the same center did not. We retrospectively applied tile color coding logic to prior nursing documentation from 2021 until the pilot phase to facilitate comparison pre- and post-Bundle Board release.

RESULTS

After adjusting for time, other quality improvement efforts, and nursing shift, multiple linear regression demonstrated a statistically significant improvement in the completion of catheter care and documentation during the pilot phase (p < 0.0001) and comparison phase (p = 0.002). The median duration of documented problematic conditions was significantly reduced during the pilot phase (p < 0.0001) and in the MICU with the Bundle Board (comparison phase, p = 0.027).

CONCLUSION

We successfully developed a data visualization tool that changed ICU provider behavior, resulting in increased completion and documentation of maintenance care and reduced duration of problematic conditions for invasive catheters in MICU patients.

User Perceptions of Visual Blood: An International Mixed Methods Study on Novel Blood Gas Analysis Visualization

Blood gas analysis plays a central role in modern medicine. Advances in technology have expanded the range of available parameters and increased the complexity of their interpretation. By applying user-centered design principles, it is possible to reduce the cognitive load associated with interpreting blood gas analysis. In this international, multicenter study, we explored anesthesiologists' perspectives on Visual Blood, a novel visualization technique for presenting blood gas analysis results. We conducted interviews with participants following two computer-based simulation studies, the first utilizing virtual reality (VR) (50 participants) and the second without VR (70 participants). Employing the template approach, we identified key themes in the interview responses and formulated six statements, which were rated using Likert scales from 1 (strongly disagree) to 5 (strongly agree) in an online questionnaire. The most frequently mentioned theme was the positive usability features of Visual Blood. The online survey revealed that participants found Visual Blood to be an intuitive method for interpreting blood gas analysis (median 4, interquartile range (IQR) 4-4, p < 0.001). Participants noted that minimal training was required to effectively learn how to interpret Visual Blood (median 4, IQR 4-4, p < 0.001). However, adjustments are necessary to reduce visual overload (median 4, IQR 2-4, p < 0.001). Overall, Visual Blood received a favorable response. The strengths and weaknesses derived from these data will help optimize future versions of Visual Blood to improve the presentation of blood gas analysis results.

Dealing with information overload: a comprehensive review

Information overload is a problem that is being exacerbated by the ongoing digitalization of the world of work and the growing use of information and communication technologies. Therefore, the aim of this systematic literature review is to provide an insight into existing measures for prevention and intervention related to information overload. The methodological approach of the systematic review is based on the PRISMA standards. A keyword search in three interdisciplinary scientific databases and other more practice-oriented databases resulted in the identification of 87 studies, field reports, and conceptual papers that were included in the review. The results show that a considerable number of papers have been published on interventions on the behavioral prevention level. At the level of structural prevention, there are also many proposals on how to design work to reduce information overload. A further distinction can be made between work design approaches at the level of information and communication technology and at the level of teamwork and organizational regulations. Although the identified studies cover a wide range of possible interventions and design approaches to address information overload, the strength of the evidence from these studies is mixed.

Avatar-based patient monitoring improves information transfer, diagnostic confidence and reduces perceived workload in intensive care units: computer-based, multicentre comparison study

Patient monitoring is the foundation of intensive care medicine. High workload and information overload can impair situation awareness of staff, thus leading to loss of important information about patients' conditions. To facilitate mental processing of patient monitoring data, we developed the Visual-Patient-avatar Intensive Care Unit (ICU), a virtual patient model animated from vital signs and patient installation data. It incorporates user-centred design principles to foster situation awareness. This study investigated the avatar's effects on information transfer measured by performance, diagnostic confidence and perceived workload. This computer-based study compared Visual-Patient-avatar ICU and conventional monitor modality for the first time. We recruited 25 nurses and 25 physicians from five centres. The participants completed an equal number of scenarios in both modalities. Information transfer, as the primary outcome, was defined as correctly assessing vital signs and installations. Secondary outcomes included diagnostic confidence and perceived workload. For analysis, we used mixed models and matched odds ratios. Comparing 250 within-subject cases revealed that Visual-Patient-avatar ICU led to a higher rate of correctly assessed vital signs and installations [rate ratio (RR) 1.25; 95% CI 1.19-1.31; P < 0.001], strengthened diagnostic confidence [odds ratio (OR) 3.32; 95% CI 2.15-5.11, P < 0.001] and lowered perceived workload (coefficient - 7.62; 95% CI - 9.17 to - 6.07; P < 0.001) than conventional modality. Using Visual-Patient-avatar ICU, participants retrieved more information with higher diagnostic confidence and lower perceived workload compared to the current industry standard monitor.

Effectiveness of clinical dashboards as audit and feedback or clinical decision support tools on medication use and test ordering: a systematic review of randomized controlled trials

BACKGROUND

Clinical dashboards used as audit and feedback (A&F) or clinical decision support systems (CDSS) are increasingly adopted in healthcare. However, their effectiveness in changing the behavior of clinicians or patients is still unclear. This systematic review aims to investigate the effectiveness of clinical dashboards used as CDSS or A&F tools (as a standalone intervention or part of a multifaceted intervention) in primary care or hospital settings on medication prescription/adherence and test ordering.

METHODS

Seven major databases were searched for relevant studies, from inception to August 2021. Two authors independently extracted data, assessed the risk of bias using the Cochrane RoB II scale, and evaluated the certainty of evidence using GRADE. Data on trial characteristics and intervention effect sizes were extracted. A narrative synthesis was performed to summarize the findings of the included trials.

RESULTS

Eleven randomized trials were included. Eight trials evaluated clinical dashboards as standalone interventions and provided conflicting evidence on changes in antibiotic prescribing and no effects on statin prescribing compared to usual care. Dashboards increased medication adherence in patients with inflammatory arthritis but not in kidney transplant recipients. Three trials investigated dashboards as part of multicomponent interventions revealing decreased use of opioids for low back pain, increased proportion of patients receiving cardiovascular risk screening, and reduced antibiotic prescribing for upper respiratory tract infections.

CONCLUSION

There is limited evidence that dashboards integrated into electronic medical record systems and used as feedback or decision support tools may be associated with improvements in medication use and test ordering.

A Framework for Evaluating Dashboards in Healthcare

In the era of 'information overload', effective information provision is essential for enabling rapid response and critical decision making. In making sense of diverse information sources, dashboards have become an indispensable tool, providing fast, effective, adaptable, and personalized access to information for professionals and the general public alike. However, these objectives place heavy requirements on dashboards as information systems in usability and effective design. Understanding these issues is challenging given the absence of consistent and comprehensive approaches to dashboard evaluation. In this article we systematically review literature on dashboard implementation in healthcare, where dashboards have been employed widely, and where there is widespread interest for improving the current state of the art, and subsequently analyse approaches taken towards evaluation. We draw upon consolidated dashboard literature and our own observations to introduce a general definition of dashboards which is more relevant to current trends, together with seven evaluation scenarios - task performance, behaviour change, interaction workflow, perceived engagement, potential utility, algorithm performance and system implementation. These scenarios distinguish different evaluation purposes which we illustrate through measurements, example studies, and common challenges in evaluation study design. We provide a breakdown of each evaluation scenario, and highlight some of the more subtle questions. We demonstrate the use of the proposed framework by a design study guided by this framework. We conclude by comparing this framework with existing literature, outlining a number of active discussion points and a set of dashboard evaluation best practices for the academic, clinical and software development communities alike.

Anesthesia providers' visual attention in simulated anesthesia emergencies using conventional number-based and avatar-based patient monitoring: a prospective, eye-tracking study

BACKGROUND

Inadequate situational awareness accounts for two-thirds of preventable complications in anesthesia. An essential tool for situational awareness in the perioperative setting is the patient monitor. However, the conventional monitor has several weaknesses. Avatar-based patient monitoring may address these shortcomings and promote situation awareness, a prerequisite for good decision making.

OBJECTIVE

The spatial distribution of visual attention is a fundamental process for achieving adequate situation awareness and thus a potential quantifiable surrogate for situation awareness. Moreover, measuring visual attention with a head-mounted eye-tracker may provide insights into usage and acceptance of the new avatar-based patient monitoring modality.

METHODS

This prospective eye-tracking study compared anesthesia providers' visual attention on conventional and avatar-based patient monitors during simulated critical anesthesia events. We defined visual attention, measured as fixation count and dwell time, as our primary outcome. We correlated visual attention with the potential confounders: performance in managing simulated critical anesthesia events (task performance), work experience, and profession. We used mixed linear models to analyze the results.

RESULTS

Fifty-two teams performed 156 simulations. After a manual quality check of the eye-tracking footage, we excluded 57 simulations due to technical problems and quality issues. Participants had a median of 198 (IQR 92.5 - 317.5) fixations on the patient monitor with a median dwell time of 30.2 (IQR 14.9 - 51.3) seconds. We found no significant difference in participants' visual attention when using avatar-based patient monitoring or conventional patient monitoring. However, we found that with each percentage point of better task performance, the number of fixations decreased by about 1.39 (coefficient -1.39; 95%CI: -2.44 to -0.34; P=.02), and the dwell time diminished by 0.23 seconds (coefficient -0.23; 95%CI: -0.4 to -0.06; P=.01).

CONCLUSIONS

Using eye-tracking, we found no significant difference in visual attention when anesthesia providers used avatar-based monitoring or conventional patient monitoring in simulated critical anesthesia events. However, we identified visual attention in conjunction with task performance as a surrogate for situational awareness.

CLINICALTRIAL

Business Management System for Ethics Committees Number Req-2020-00059.

Integration of an ICU Visualization Dashboard (i-Dashboard) as a Platform to Facilitate Multidisciplinary Rounds: A Cluster Randomized Controlled Trial (Preprint)

Background

 Multidisciplinary rounds (MDRs) are scheduled, patient-focused communication mechanisms among multidisciplinary providers in the intensive care unit (ICU).

Objective

i-Dashboard is a custom-developed visualization dashboard that supports (1) key information retrieval and reorganization, (2) time-series data, and (3) display on large touch screens during MDRs. This study aimed to evaluate the performance, including the efficiency of prerounding data gathering, communication accuracy, and information exchange, and clinical satisfaction of integrating i-Dashboard as a platform to facilitate MDRs.

Methods

A cluster-randomized controlled trial was performed in 2 surgical ICUs at a university hospital. Study participants included all multidisciplinary care team members. The performance and clinical satisfaction of i-Dashboard during MDRs were compared with those of the established electronic medical record (EMR) through direct observation and questionnaire surveys.

Results

Between April 26 and July 18, 2021, a total of 78 and 91 MDRs were performed with the established EMR and i-Dashboard, respectively. For prerounding data gathering, the median time was 10.4 (IQR 9.1-11.8) and 4.6 (IQR 3.5-5.8) minutes using the established EMR and i-Dashboard (P<.001), respectively. During MDRs, data misrepresentations were significantly less frequent with i-Dashboard (median 0, IQR 0-0) than with the established EMR (4, IQR 3-5; P<.001). Further, effective recommendations were significantly more frequent with i-Dashboard than with the established EMR (P<.001). The questionnaire results revealed that participants favored using i-Dashboard in association with the enhancement of care plan development and team participation during MDRs.

Conclusions

 i-Dashboard increases efficiency in data gathering. Displaying i-Dashboard on large touch screens in MDRs may enhance communication accuracy, information exchange, and clinical satisfaction. The design concepts of i-Dashboard may help develop visualization dashboards that are more applicable for ICU MDRs.

Trial Registration

ClinicalTrials.gov NCT04845698; https://clinicaltrials.gov/ct2/show/NCT04845698

Enhancing Diagnosis Through Technology: Decision Support, Artificial Intelligence, and Beyond

Patient care in intensive care environments is complex, time-sensitive, and data-rich, factors that make these settings particularly well-suited to clinical decision support (CDS). A wide range of CDS interventions have been used in intensive care unit environments. The field needs well-designed studies to identify the most effective CDS approaches. Evolving artificial intelligence and machine learning models may reduce information-overload and enable teams to take better advantage of the large volume of patient data available to them. It is vital to effectively integrate new CDS into clinical workflows and to align closely with the cognitive processes of frontline clinicians.

Iterative heuristic design of temporal graphic displays with clinical domain experts

Conventional electronic health record information displays are not optimized for efficient information processing. Graphical displays that integrate patient information can improve information processing, especially in data-rich environments such as critical care. We propose an adaptable and reusable approach to patient information display with modular graphical components (widgets). We had two study objectives. First, reduce numerous widget prototype alternatives to preferred designs. Second, derive widget design feature recommendations. Using iterative human-centered design methods, we interviewed experts to hone design features of widgets displaying frequently measured data elements, e.g., heart rate, for acute care patient monitoring and real-time clinical decision-making. Participant responses to design queries were coded to calculate feature-set agreement, average prototype score, and prototype agreement. Two iterative interview cycles covering 64 design queries and 86 prototypes were needed to reach consensus on six feature sets. Interviewers agreed that line graphs with a smoothed or averaged trendline, 24-h timeframe, and gradient coloring for urgency were useful and informative features. Moreover, users agreed that widgets should include key functions: (1) adjustable reference ranges, (2) expandable timeframes, and (3) access to details on demand. Participants stated graphical widgets would be used to identify correlating patterns and compare abnormal measures across related data elements at a specific time. Combining theoretical principles and validated design methods was an effective and reproducible approach to designing widgets for healthcare displays. The findings suggest our widget design features and recommendations match critical care clinician expectations for graphical information display of continuous and frequently updated patient data.

Impact of integrated graphical display on expert and novice diagnostic performance in critical care

OBJECTIVE

To determine the impact of a graphical information display on diagnosing circulatory shock.

MATERIALS AND METHODS

This was an experimental study comparing integrated and conventional information displays. Participants were intensivists or critical care fellows (experts) and first-year medical residents (novices).

RESULTS

The integrated display was associated with higher performance (87% vs 82%; P < .001), less time (2.9 vs 3.5 min; P = .008), and more accurate etiology (67% vs 54%; P = .048) compared to the conventional display. When stratified by experience, novice physicians using the integrated display had higher performance (86% vs 69%; P < .001), less time (2.9 vs 3.7 min; P = .03), and more accurate etiology (65% vs 42%; P = .02); expert physicians using the integrated display had nonsignificantly improved performance (87% vs 82%; P = .09), time (2.9 vs 3.3; P = .28), and etiology (69% vs 67%; P = .81).

DISCUSSION

The integrated display appeared to support efficient information processing, which resulted in more rapid and accurate circulatory shock diagnosis. Evidence more strongly supported a difference for novices, suggesting that graphical displays may help reduce expert-novice performance gaps.

A Perioperative Care Display for Understanding High Acuity Patients

BACKGROUND

The data visualization literature asserts that the details of the optimal data display must be tailored to the specific task, the background of the user, and the characteristics of the data. The general organizing principle of a concept-oriented display is known to be useful for many tasks and data types.

OBJECTIVES

In this project, we used general principles of data visualization and a co-design process to produce a clinical display tailored to a specific cognitive task, chosen from the anesthesia domain, but with clear generalizability to other clinical tasks. To support the work of the anesthesia-in-charge (AIC) our task was, for a given day, to depict the acuity level and complexity of each patient in the collection of those that will be operated on the following day. The AIC uses this information to optimally allocate anesthesia staff and providers across operating rooms.

METHODS

We used a co-design process to collaborate with participants who work in the AIC role. We conducted two in-depth interviews with AICs and engaged them in subsequent input on iterative design solutions.

RESULTS

Through a co-design process, we found (1) the need to carefully match the level of detail in the display to the level required by the clinical task, (2) the impedance caused by irrelevant information on the screen such as icons relevant only to other tasks, and (3) the desire for a specific but optional trajectory of increasingly detailed textual summaries.

CONCLUSION

This study reports a real-world clinical informatics development project that engaged users as co-designers. Our process led to the user-preferred design of a single binary flag to identify the subset of patients needing further investigation, and then a trajectory of increasingly detailed, text-based abstractions for each patient that can be displayed when more information is needed.

Mixed methods evaluation of a computerised audit and feedback dashboard to improve patient safety through targeting acute kidney injury (AKI) in primary care

BACKGROUND

Reducing the harms associated with acute kidney injury (AKI) requires addressing a wide range of patient safety issues, including polypharmacy and transitions of care, particularly for vulnerable patient groups. Computerised audit and feedback can transform the way healthcare organisations measure, analyse and learn from quality and safety data across different care settings, potentially improving patient safety.

OBJECTIVE

To implement and evaluate an audit and feedback dashboard targeting AKI to improve patient safety, focusing on factors affecting a range of user characteristics in primary care.

METHODS

We performed a mixed methods study in three stages. Semi-structured interviews were initially performed with both primary (n = 10) and secondary care (n = 5) staff to gather user requirements for six quality indicators extracted from national guidance on post-discharge AKI care. Modified indicators were implemented in the Performance Improvement plaN GeneratoR (PINGR) audit and feedback dashboard for six months, across 45 general practices in Salford. Primary care professionals were then interviewed again (n = 7) and completed usability questionnaires. This was triangulated with an interrupted time series analysis on indicator performance, alongside software usage statistics.

RESULTS

Improvements were observed for the indicators for medication review (+9.01 %; 95 % Confidence Interval (CI), +6.95 % to +11.06 %) and blood pressure measurement (+5.20 %; 95 % CI + 3.61 % to +6.78 %). Variable performance and engagement were observed for other indicators including AKI coding (+0.39 %; 95 % CI -1.88 % to +2.65 %), serum creatinine (-3.40 %; 95 % CI -7.66 % to +0.85 %), proteinuria (-1.08 %; 95 % CI -1.47 % to +0.32 %) and providing patient information (+0.16 %; 95 % CI -0.41 % to +0.73 %). A key facilitator to engagement was the development of 'champions of change', achieved through a raised awareness of high-risk patients, guidelines, inconsistencies in coding practice and evidence for quality and safety performance. Barriers related to the specificity and perceived achievability of indicators, and limitations in resources.

CONCLUSION

In a six-month, quasi-experimental evaluation of an electronic audit and feedback dashboard targeting AKI, we found improvements for two out of six quality indicators. While information technology can facilitate improvements in patient safety, further allocation of protected staff time and investment into shared learning are needed to realise those improvements in practice.

User-Centered Clinical Display Design Issues for Inpatient Providers

BACKGROUND

Suboptimal information display in electronic health records (EHRs) is a notorious pain point for users. Designing an effective display is difficult, due in part to the complex and varied nature of clinical practice.

OBJECTIVE

This article aims to understand the goals, constraints, frustrations, and mental models of inpatient medical providers when accessing EHR data, to better inform the display of clinical information.

METHODS

A multidisciplinary ethnographic study of inpatient medical providers.

RESULTS

Our participants' primary goal was usually to assemble a clinical picture around a given question, under the constraints of time pressure and incomplete information. To do so, they tend to use a mental model of multiple layers of abstraction when thinking of patients and disease; they prefer immediate pattern recognition strategies for answering clinical questions, with breadth-first or depth-first search strategies used subsequently if needed; and they are sensitive to data relevance, completeness, and reliability when reading a record.

CONCLUSION

These results conflict with the ubiquitous display design practice of separating data by type (test results, medications, notes, etc.), a mismatch that is known to encumber efficient mental processing by increasing both navigation burden and memory demands on users. A popular and obvious solution is to select or filter the data to display exactly what is presumed to be relevant to the clinical question, but this solution is both brittle and mistrusted by users. A less brittle approach that is more aligned with our users' mental model could use abstraction to summarize details instead of filtering to hide data. An abstraction-based approach could allow clinicians to more easily assemble a clinical picture, to use immediate pattern recognition strategies, and to adjust the level of displayed detail to their particular needs. It could also help the user notice unanticipated patterns and to fluidly shift attention as understanding evolves.

Association of an Electronic Health Record Add-on App for Neonatal Bilirubin Management With Physician Efficiency and Care Quality

IMPORTANCE

The usefulness of electronic health record (EHR) systems could be significantly enhanced by innovative, third-party EHR add-on apps.

OBJECTIVE

To evaluate whether an EHR add-on app for neonatal bilirubin management can save clinicians time and improve patient care.

DESIGN, SETTING, AND PARTICIPANTS

This quality improvement study was conducted at the University of Utah Health Well Baby nursery and outpatient clinics and consisted of 4 substudies: (1) time savings were estimated in an experimental task-timing study comparing the time required for physicians to manage newborns' bilirubin levels with and without the add-on app, (2) app use was estimated from app logs, (3) health care use measures and guideline compliance were compared retrospectively before and after the intervention, and (4) clinician-perceived usability was measured through System Usability Scale surveys. The study took place between April 1, 2016, and September 3, 2019. Data analyses were conducted from October 30, 2018, to September 23, 2019.

INTERVENTIONS

At baseline, clinicians used a manual approach to ensure compliance with an evidence-based clinical guideline for neonatal bilirubin management. To facilitate guideline compliance, an EHR add-on app that automatically retrieves, organizes, and visualizes relevant patient data was developed. The app provides patient-specific assessments and recommendations, including the risk of rebound hyperbilirubinemia following phototherapy based on a predictive model. The add-on app was integrated with the University of Utah Health EHR on April 12, 2017.

MAIN OUTCOMES AND MEASURES

Clinician time savings, app use, health care use measures, guideline-compliant phototherapy ordering, and perceived usability as measured by the System Usability Scale survey. The survey is composed of 10 statements with responses ranging from 1 (strongly disagree) to 5 (strongly agree). The survey results in a single score ranging from 0 to 100, with ratings described as worst imaginable (mean System Usability Scale score, 12.5), awful (20.3), poor (35.7), okay (50.9), good (71.4), excellent (85.5), and best imaginable (90.9).

RESULTS

In 2018, the application was used 20 516 times by clinicians for 91.84% of eligible newborns. Use of the app saved 66 seconds for bilirubin management tasks compared with a commonly used tool (95% CI, 53-79 seconds; P < .001). Following the intervention, health care use rates remained stable, while orders for clinically appropriate phototherapy during hospitalization increased for newborns with bilirubin levels above the guideline-recommended threshold (odds ratio, 1.84; 95% CI, 1.16-2.90; P = .009). Surveys indicated excellent usability (System Usability Scale score, 83.90; 95% CI, 81.49-86.31).

CONCLUSIONS AND RELEVANCE

Well-designed EHR add-on apps may save clinicians time and improve patient care. If time-saving apps, such as the bilirubin app, were implemented widely across institutions and care domains, the potential association with improved patient care and clinician efficiency could be significant. The University of Utah Health bilirubin app is being prepared for release into EHR app stores as free-to-use software.

Critical care information display approaches and design frameworks: A systematic review and meta-analysis

OBJECTIVE

To systematically review original user evaluations of patient information displays relevant to critical care and understand the impact of design frameworks and information presentation approaches on decision-making, efficiency, workload, and preferences of clinicians.

METHODS

We included studies that evaluated information displays designed to support real-time care decisions in critical care or anesthesiology using simulated tasks. We searched PubMed and IEEExplore from 1/1/1990 to 6/30/2018. The search strategy was developed iteratively with calibration against known references. Inclusion screening was completed independently by two authors. Extraction of display features, design processes, and evaluation method was completed by one and verified by a second author.

RESULTS

Fifty-six manuscripts evaluating 32 critical care and 22 anesthesia displays were included. Primary outcome metrics included clinician accuracy and efficiency in recognizing, diagnosing, and treating problems. Implementing user-centered design (UCD) processes, especially iterative evaluation and redesign, resulted in positive impact in outcomes such as accuracy and efficiency. Innovative display approaches that led to improved human-system performance in critical care included: (1) improving the integration and organization of information, (2) improving the representation of trend information, and (3) implementing graphical approaches to make relationships between data visible.

CONCLUSION

Our review affirms the value of key principles of UCD. Improved information presentation can facilitate faster information interpretation and more accurate diagnoses and treatment. Improvements to information organization and support for rapid interpretation of time-based relationships between related quantitative data is warranted. Designers and developers are encouraged to involve users in formal iterative design and evaluation activities in the design of electronic health records (EHRs), clinical informatics applications, and clinical devices.