Hospital performance dashboards: a literature review

A Geospatial Drug Abuse Risk Assessment and Monitoring Dashboard Tailored for School Students: Development Study With Requirement Analysis and Acceptance Evaluation

BACKGROUND

The enormous consequences of drugs include suicides, traffic accidents, and violence, affecting the individual, family, society, and country. Therefore, it is necessary to constantly identify and monitor the drug abuse rate among school-going youth. A geospatial dashboard is vital for the monitoring of drug abuse and related crime incidence in a decision support system.

OBJECTIVE

This paper mainly focuses on developing MyAsriGeo, a geospatial drug abuse risk assessment and monitoring dashboard tailored for school students. It introduces innovative functionality, seamlessly orchestrating the assessment of drug abuse usage patterns and risks using multivariate student data.

METHODS

A geospatial drug abuse dashboard for monitoring and analysis was designed and developed in this study based on agile methodology and prototyping. Using focus group and interviews, we first examined and gathered the requirements, feedback, and user approval of the MyAsriGeo dashboard. Experts and stakeholders such as the National Anti-Drugs Agency, police, the Federal Department of Town and Country Planning, school instructors, students, and researchers were among those who responded. A total of 20 specialists were involved in the requirement analysis and acceptance evaluation of the pilot and final version of the dashboard. The evaluation sought to identify various user acceptance aspects, such as ease of use and usefulness, for both the pilot and final versions, and 2 additional factors based on the Post-Study System Usability Questionnaire and Task-Technology Fit models were enlisted to assess the interface quality and dashboard sufficiency for the final version.

RESULTS

The MyAsriGeo geospatial dashboard was designed to meet the needs of all user types, as identified through a requirement gathering process. It includes several key functions, such as a geospatial map that shows the locations of high-risk areas for drug abuse, data on drug abuse among students, tools for assessing the risk of drug abuse in different areas, demographic information, and a self-problem test. It also includes the Alcohol, Smoking, and Substance Involvement Screening Test and its risk assessment to help users understand and interpret the results of student risk. The initial prototype and final version of the dashboard were evaluated by 20 experts, which revealed a significant improvement in the ease of use (P=.047) and usefulness (P=.02) factors and showed a high acceptance mean scores for ease of use (4.2), usefulness (4.46), interface quality (4.29), and sufficiency (4.13).

CONCLUSIONS

The MyAsriGeo geospatial dashboard is useful for monitoring and analyzing drug abuse among school-going youth in Malaysia. It was developed based on the needs of various stakeholders and includes a range of functions. The dashboard was evaluated by a group of experts. Overall, the MyAsriGeo geospatial dashboard is a valuable resource for helping stakeholders understand and respond to the issue of drug abuse among youth.

Business intelligence and the leverage of information in healthcare organizations from a managerial perspective: a systematic literature review and research agenda

PURPOSE

Business intelligence (BI) systems and tools are deemed to be a transformative source with the potential to contribute to reshaping the way different healthcare organizations' (HCOs) services are offered and managed. However, this emerging field of research still appears underdeveloped and fragmented. Hence, this paper aims to reconciling, analyzing and synthesizing different strands of managerial-oriented literature on BI in HCOs and to enhance both theoretical and applied future contributions.

DESIGN/METHODOLOGY/APPROACH

A literature-based framework was developed to establish and guide a three-stage state-of-the-art systematic literature review (SLR). The SLR was undertaken adopting a hybrid methodology that combines a bibliometric and a content analysis.

FINDINGS

In total, 34 peer-review articles were included. Results revealed significant heterogeneity in theoretical basis and methodological strategies. Nonetheless, the knowledge structure of this research's stream seems to be primarily composed of five clusters of interconnected topics: (1) decision-making, relevant capabilities and value creation; (2) user satisfaction and quality; (3) process management, organizational change and financial effectiveness; (4) decision-support information, dashboard and key performance indicators; and (5) performance management and organizational effectiveness.

ORIGINALITY/VALUE

To the authors' knowledge, this is the first SLR providing a business and management-related state-of-the-art on the topic. Besides, the paper offers an original framework disentangling future research directions from each emerged cluster into issues pertaining to BI implementation, utilization and impact in HCOs. The paper also discusses the need of future contributions to explore possible integrations of BI with emerging data-driven technologies (e.g. artificial intelligence) in HCOs, as the role of BI in addressing sustainability challenges.

A mixed methods evaluation assessing the feasibility of implementing a PrEP data dashboard in the Southeastern United States

BACKGROUND

Alabama is one of seven priority states for the National Ending the HIV Epidemic Initiative due to a disproportionate burden of rural infections. To reverse growing infection rates, the state must increase its focus on prevention efforts, including novel strategies. One such approach is to utilize dashboards that visualize real-time data on the pre-exposure prophylaxis (PrEP) care continuum to assist in prioritizing evidence-based preventative care for those most vulnerable for HIV infection.

METHODS

We conducted a mixed methods evaluation to ascertain stakeholders' perceptions on the acceptability, feasibility, appropriateness, and usability of a PrEP care continuum dashboard, as well as gain insight on ways to improve the activities necessary to sustain it. Clinicians, administrators, and data personnel from participating sites in Alabama completed surveys (n = 9) and participated in key informant interviews (n = 10) to better understand their experiences with the prototype data dashboard and to share feedback on how it can be modified to best fit their needs.

RESULTS

Surveys and interviews revealed that all participants find the pilot data dashboard to be an acceptable, feasible, and appropriate intervention for clinic use. Overall, stakeholders find the pilot dashboard to be usable and helpful in administrative efforts, such as report and grant writing; however, additional refining is needed in order to reduce burden and optimize usefulness. Participants voiced concerns about their site's abilities to sustain the dashboard, including the lack of systematized PrEP protocols and limited funds and staff time dedicated to PrEP data collection, cleaning, and upload.

CONCLUSION

Study participants from clinics providing HIV prevention services, including PrEP, in Alabama voiced interest in sustaining and refining a data dashboard that tracks clients across the PrEP care continuum. Despite viewing the platform itself as an acceptable, feasible, and appropriate intervention, participants agreed that efforts need to be focused on standardizing PrEP data collection protocols in order to ensure consistent, accurate data capture and that limited funds and staff time are barriers to the sustained implementation of the dashboard in practice.

Comprehensive Assessment of Clinical Learning Environments to Drive Improvement: Lessons Learned from a Pilot Program

Problem: Recognition of the importance of clinical learning environments (CLEs) in health professions education has led to calls to evaluate and improve the quality of such learning environments. As CLEs sit at the crossroads of education and healthcare delivery, leadership from both entities should share the responsibility and accountability for this work. Current data collection about the experience and outcomes for learners, faculty, staff, and patients tends to occur in fragmented and siloed ways, and available tools to assess clinical learning environments are limited in scope. In addition, from an organizational perspective oversight of education and patient care is often done by separate entities, and not infrequently is there a sense of competing interests. Intervention: We aimed to design and pilot a holistic approach to assessment and review of CLEs and establish whether such a formative assessment process could be used to engage stakeholders from education, departmental, and health systems leadership in improvement of CLEs. Utilizing concepts of implementation science, we planned and executed a holistic assessment process for CLEs, monitored the impact of the assessment, and reflected on the process. We focused the assessment on four pillars characterizing exemplary learning environments: 1) Environment is inclusive, promotes diversity and collaboration; 2) Focus on continuous quality improvement; 3) Alignment between work and learning; and 4) Integration of education and healthcare mission. Context: At our institution, medical trainees rotate through several different health systems, but clinical and educational leadership converge at the departmental level. We therefore focused this proof-of-concept project on two large clinical departments at our institution, centering on medical learners from undergraduate and graduate medical education. For each department, a small team of champions helped create an assessment grid based on the four pillars and identified existing quantitative evaluation data sources. Champions subsequently collected qualitative data through observations, focus groups, and interviews to fill any gaps in available quantitative data. Impact: The project teams shared reports summarizing findings and recommendations with departmental, clinical, and educational leadership. Subsequent meetings with these stakeholders led to actionable plans for improvement as well as sustained structures for collaborative work between the different stakeholder groups. Lessons Learned: This project demonstrated the feasibility and effectiveness of collating, analyzing, and sharing data from various sources in engaging different stakeholder groups to initiate actionable improvement plans. Collating quantitative data from existing resources was a powerful way to demonstrate common issues in CLEs, and qualitative data provided further detail to inform improvement initiatives. Other institutions can adapt this approach to guide assessment and quality improvement of CLEs. As a next step, we are creating a comprehensive learning environment scorecard to allow for comparison of clinical learning environment quality across institutions and over time.

Graphical User Interface Development for a Hospital-Based Predictive Risk Tool: Protocol for a Co-Design Study

BACKGROUND

This co-design research method details the iterative process developed to identify health professional recommendations for the graphical user interface (GUI) of an artificial intelligence (AI)-enabled risk prediction tool. Driving the decision to include a co-design process is the belief that choices regarding the aesthetic and functionality of an intervention are best made by its intended users and that engaging these users in its design will promote the tool's adoption and use.

OBJECTIVE

The aim of this research is to identify health professional design and uptake recommendations for the GUI of an AI-enabled predictive risk tool.

METHODS

We will hold 3 research phases, each consisting of 2 workshops with health professionals, between mid-2023 and mid-2024. A total of 6 health professionals will be sought per workshop, resulting in a total enrollment of 36 health professionals at the conclusion of the research. A total of 7 workshop activities have been scheduled across the 3 workshops; these include context of use, notifiers, format, AI survey-Likert, prototype, AI survey-written, and testing. The first 6 of these activities will be repeated in each workshop to enable the iterative development and refinement of GUI. The last activity (testing) will be performed in the final workshop to examine health professionals' thoughts on the final GUI iteration. Qualitative and quantitative results data will be produced from tasks in each research activity. Qualitative data will be examined through inductive thematic analysis or deductive thematic analysis in accordance with the Nonadoption, Abandonment, and Challenges to the Scale-up, Spread, and Sustainability (NASSS) framework; visual data will be examined in accordance with "framework of interactivity;" and quantitative data will be examined using descriptive statistics.

RESULTS

Project registration with the Australia and New Zealand Clinical Trial Registry has been requested (#384098). Finalized design recommendations are expected in early to mid-2024, with a results manuscript to be submitted in mid-2024. This research method has human research ethics approval from the South Australian Department of Health and Wellbeing (#2022/HRE00131) as well as from the Human Research Ethics Committee of the University of South Australia (application ID#204143).

CONCLUSIONS

Understanding whether an intervention is needed in a particular situation is just the start; designing an intervention so that it is used within that situation is paramount. This co-design process engages end users to create a GUI that includes the aesthetic and functional details they need in a manner that aligns with their existing work practices. Indeed, interventions that fail to do this may be disliked, and at worst, they may be dangerous.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/47717.

Assessing and Improving Provider Knowledge for a Cardiothoracic Intensive Care Unit Electronic Dashboard Initiative

BACKGROUND

Electronic dashboards measure intensive care unit (ICU) performance by tracking quality indicators, especially pinpointing sub-standard metrics. This helps ICUs scrutinize and change current practices in an effort to improve failing metrics. However, its technological value is lost if end users are unaware of its importance. This results in decreased staff participation, leading to unsuccessful initiation of the dashboard. Therefore, the purpose of this project was to improve cardiothoracic ICU providers' understanding of electronic dashboards by providing an educational training bundle in preparation for an electronic dashboard initiation.

METHODS

A Likert survey assessing providers' knowledge, attitudes, skills, and application of electronic dashboards was conducted. Subsequently, an educational training bundle, consisting of a digital flier and laminated pamphlets, was made available to providers for four months. After bundle review, providers were assessed using the same pre-bundle Likert survey.

RESULTS

A comparison of summated scores from pre-bundle (mean = 38.75) and post-bundle surveys (mean = 46.13) yielded an increased summated score overall (mean = 7.38, p ≤ 0.001).

CONCLUSION

An educational bundle improved providers' understanding and increased their likelihood of using electronic dashboards upon its initiation. Further studies are needed to continue increasing staff participation such as providing specific education to navigate the interface for data retrieval and interpretation.

Developing a Nursing Dashboard to Align Nursing Care Delivery With Patient and Family Needs

Clinical units and their nursing staffs are heterogeneous. Advances in data and analytics provide opportunities to better match patient needs with nurse competencies. Building upon a previous publication on a unit profile dashboard, team members now describe development of a nursing dashboard aggregating characteristics of staff on each clinical unit of the hospital. This article describes methods, challenges, and future directions for nurse leaders to use the dashboards to optimize care delivery and patient and nurse outcomes.

Usability Evaluation of Dashboards: A Systematic Literature Review of Tools

Introduction

In recent years, the use of dashboards in healthcare has been considered an effective approach for the visual presentation of information to support clinical and administrative decisions. Effective and efficient use of dashboards in clinical and managerial processes requires a framework for the design and development of tools based on usability principles.

Objectives

The present study is aimed at investigating the existing questionnaires used for the usability evaluation framework of dashboards and at presenting more specific usability criteria for evaluating dashboards.

Methods

This systematic review was conducted using PubMed, Web of Science, and Scopus, without any time restrictions. The final search of articles was performed on September 2, 2022. Data collection was performed using a data extraction form, and the content of selected studies was analyzed based on the dashboard usability criteria.

Results

After reviewing the full text of relevant articles, a total of 29 studies were selected according to the inclusion criteria. Regarding the questionnaires used in the selected studies, researcher-made questionnaires were used in five studies, while 25 studies applied previously used questionnaires. The most widely used questionnaires were the System Usability Scale (SUS), Technology Acceptance Model (TAM), Situation Awareness Rating Technique (SART), Questionnaire for User Interaction Satisfaction (QUIS), Unified Theory of Acceptance and Use of Technology (UTAUT), and Health Information Technology Usability Evaluation Scale (Health-ITUES), respectively. Finally, dashboard evaluation criteria, including usefulness, operability, learnability, ease of use, suitability for tasks, improvement of situational awareness, satisfaction, user interface, content, and system capabilities, were suggested.

Conclusion

General questionnaires that were not specifically designed for dashboard evaluation were mainly used in reviewed studies. The current study suggested specific criteria for measuring the usability of dashboards. When selecting the usability evaluation criteria for dashboards, it is important to pay attention to the evaluation objectives, dashboard features and capabilities, and context of use.

Market Analysis with Business Intelligence System for Marketing Planning

The automotive and auto parts industries are important economic sectors in Thailand. With rapidly changing technology, every organization should understand what needs to be improved clearly, and shift their strategies to meet evolving consumer demands. The purpose of this research is to develop a Business Intelligence system for a brake pad manufacturing company in Thailand. By analyzing the relationship between market demand and supply components of the company through regression analysis and the principles of the marketing mix, we develop a product lifecycle curve for forecasting product sales. The developed system increases the workflow efficiency of the case study company, being able to simplify the traditional data preparation process that requires employees to collect and summarize data every time a request is made. An intelligence dashboard is subsequently created to help support decision-making, facilitate communication within the company, and eventually improve team efficiency and productivity.

Situation Awareness-Oriented Dashboard in ICUs in Support of Resource Management in Time of Pandemics

In a pediatric intensive care unit (PICU) of 32 beds, clinicians manage resources 24 hours a day, 7 days a week, from a large-screen dashboard implemented in 2017. This resource management dashboard efficiently replaces the handwriting information displayed on a whiteboard, offering a synthetic view of the bed's layout and specific information on staff and equipment at bedside. However, in 2020 when COVID-19 hit, the resource management dashboard showed several limitations. Mainly, its visualization offered to the clinicians limited situation awareness (SA) to perceive, understand and predict the impacts on resource management and decision-making of an unusual flow of patients affected by the most severe form of coronavirus. To identify the SA requirements during a pandemic, we conducted goal-oriented interviews with 11 clinicians working in ICUs. The result is the design of an SA-oriented dashboard with 22 key indicators (KIs): 1 on the admission capacity, 15 at bedside and 6 displayed as statistics in the central area. We conducted a usability evaluation of the SA-oriented dashboard compared to the resource management dashboard with 6 clinicians. The results showed five usability improvements of the SA-oriented dashboard and five limitations. Our work contributes to new knowledge on the clinicians' SA requirements to support resource management and decision-making in ICUs in times of pandemics.

Development and validation of automated electronic health record data reuse for a multidisciplinary quality dashboard

Objective

To describe the development and validation of automated electronic health record data reuse for a multidisciplinary quality dashboard.

Materials and methods

Comparative study analyzing a manually extracted and an automatically extracted dataset with 262 patients treated for HNC cancer in a tertiary oncology center in the Netherlands in 2020. The primary outcome measures were the percentage of agreement on data elements required for calculating quality indicators and the difference between indicators results calculated using manually collected and indicators that used automatically extracted data.

Results

The results of this study demonstrate high agreement between manual and automatically collected variables, reaching up to 99.0% agreement. However, some variables demonstrate lower levels of agreement, with one variable showing only a 20.0% agreement rate. The indicator results obtained through manual collection and automatic extraction show high agreement in most cases, with discrepancy rates ranging from 0.3% to 3.5%. One indicator is identified as a negative outlier, with a discrepancy rate of nearly 25%.

Conclusions

This study shows that it is possible to use routinely collected structured data to reliably measure the quality of care in real-time, which could render manual data collection for quality measurement obsolete. To achieve reliable data reuse, it is important that relevant data is recorded as structured data during the care process. Furthermore, the results also imply that data validation is conditional to development of a reliable dashboard.

Requirements and challenges of hospital dashboards: a systematic literature review

Background

Today, the use of data in administrative and clinical processes is quite challenging due to the large volume of data, data collection from various sources, and lack of data structure. As a data management tool, dashboards play an important role in timely visual display of critical information on key performances.

Objectives

This systematic review aimed to identify functional and non-functional requirements, as well as challenges of using dashboards in hospitals.

Methods

In this systematic review, four databases, including the Web of Science, PubMed, EMBASE, and Scopus, were searched to find relevant articles from 2000 until May 30, 2020. The final search was conducted on May 30, 2020. Data collection was performed using a data extraction form and reviewing the content of relevant studies on the potentials and challenges of dashboard implementation.

Results

Fifty-four out of 1254 retrieved articles were selected for this study based on the inclusion and exclusion criteria. The functional requirements for dashboards included reporting, reminders, customization, tracking, alert creation, and assessment of performance indicators. On the other hand, the non-functional requirements included the dashboard speed, security, ease of use, installation on different devices (e.g., PCs and laptops), integration with other systems, web-based design, inclusion of a data warehouse, being up-to-data, and use of data visualization elements based on the user’s needs. Moreover, the identified challenges were categorized into four groups: data sources, dashboard content, dashboard design, implementation, and integration in other systems at the hospital level.

Conclusion

Dashboards, by providing information in an appropriate manner, can lead to the proper use of information by users. In order for a dashboard to be effective in clinical and managerial processes, particular attention must be paid to its capabilities, and the challenges of its implementation need to be addressed.

Data-driven clinical improvement: Oncology nurse leaders' perceptions and experiences of organisational data reports

AIM

To elicit oncology nurse leaders' perceptions and experiences of accessing, using and interpreting report data at the unit level, and their suggestions for future reports.

BACKGROUND

Nurse leaders are expected to use data reports for decisions about unit-level operations, yet data may be inaccessible, unavailable and lack relevance for improving patient care and unit-level outcomes.

METHODS

A purposeful sampling was used to recruit 12 unit-level nurse leaders. Qualitative data were collected through semi-structured interviews and analysed using thematic content analysis.

RESULTS

Consistent themes included the lack of accurate, useful and meaningful data specifically related to patient care. Accessibility Challenges, Limits to Applicability and Suggestions for Improvement were the main themes.

CONCLUSION

Nurse leaders require real-time data to effectively implement clinical interventions and practice changes for improving unit-level patient care.

IMPLICATIONS FOR NURSING MANAGEMENT

Nurse leaders emphasized that their insight into the development of customizable reports is crucial for obtaining meaningful data relevant to the varied unit-level health care setting. Reports targeting unit-level outcomes would provide meaningful data to facilitate clinical improvement where patient care is provided. Improved reports increase the likelihood of their use and the potential for enhancing the quality and safe care outcomes.

Conducting research by mixed methods: an autoethnography account of a PhD research in facilities management from a developing country

Purpose

Despite the increasing discourse on mixed methods (MM) in academic literature, less attention has been focused on its methodological development in the built environment in developing countries. This paper aims to examine the basis and challenges of the application of MM for health-care facilities management (FM) research in Ghana.

Design/methodology/approach

This paper adopts autoethnography as its research method, coupled with a scholarly review of methodological literature to position a technical view on the application of mixed methods for health-care FM research in a developing country.

Findings

This paper contributes and lends support to the methodological strength of MM as epistemologically coherent and useful for understanding the intricacies of health-care FM. This paper advances the proposition that the nascent stage of FM development in Ghana, paucity of literature, epistemological and axiological considerations underscore the choice of a sequential MM. The attitude of research participants, ethical challenges and time-lapse for data analysis were observed as practical challenges.

Research limitations/implications

Discussion of data integration is excluded.

Practical implications

This paper provides a nuanced understanding of the concept of MM in health-care FM and set forth practical recommendations worthy to enhance the application of MM research.

Originality/value

This paper is among the few focusing on methodological discussion of health-care FM. This paper proposes a framework to guide researchers in the application of mixed methods.

Integrating training, practice, and reflection within a new model for Canadian medical licensure: a concept paper prepared for the Medical Council of Canada

In 2020 the Medical Council of Canada created a task force to make recommendations on the modernization of its practices for granting licensure to medical trainees. This task force solicited papers on this topic from subject matter experts. As outlined within this Concept Paper, our proposal would shift licensure away from the traditional focus on high-stakes summative exams in a way that integrates training, clinical practice, and reflection. Specifically, we propose a model of graduated licensure that would have three stages including: a trainee license for trainees that have demonstrated adequate medical knowledge to begin training as a closely supervised resident, a transition to practice license for trainees that have compiled a reflective educational portfolio demonstrating the clinical competence required to begin independent practice with limitations and support, and a fully independent license for unsupervised practice for attendings that have demonstrated competence through a reflective portfolio of clinical analytics. This proposal was reviewed by a diverse group of 30 trainees, practitioners, and administrators in medical education. Their feedback was analyzed and summarized to provide an overview of the likely reception that this proposal would receive from the medical education community.

Developing a Unit Profile Dashboard to Inform Nursing Care Delivery Based on Unique Needs of Patients

Nurse leaders need real-time, accurate data to design care delivery models and make decisions that reflect the patient populations they serve. To support nurse leader practice and optimize patient care, we developed a unit profile dashboard to describe the unique characteristics of patient populations cared for on each clinical unit at our hospital. In this article, we describe dashboard development, challenges, use cases, and plans for data analytics to further advance nursing practice.

Maintaining Implementation through Dynamic Adaptations (MIDAS): protocol for a cluster-randomized trial of implementation strategies to optimize and sustain use of evidence-based practices in Veteran Health Administration (VHA) patients

Background

The adoption and sustainment of evidence-based practices (EBPs) is a challenge within many healthcare systems, especially in settings that have already strived but failed to achieve longer-term goals. The Veterans Affairs (VA) Maintaining Implementation through Dynamic Adaptations (MIDAS) Quality Enhancement Research Initiative (QUERI) program was funded as a series of trials to test multi-component implementation strategies to sustain optimal use of three EBPs: (1) a deprescribing approach intended to reduce potentially inappropriate polypharmacy; (2) appropriate dosing and drug selection of direct oral anticoagulants (DOACs); and (3) use of cognitive behavioral therapy as first-line treatment for insomnia before pharmacologic treatment. We describe the design and methods for a harmonized series of cluster-randomized control trials comparing two implementation strategies.

Methods

For each trial, we will recruit 8–12 clinics (24–36 total). All will have access to relevant clinical data to identify patients who may benefit from the target EBP at that clinic and provider. For each trial, clinics will be randomized to one of two implementation strategies to improve the use of the EBPs: (1) individual-level academic detailing (AD) or (2) AD plus the team-based Learn. Engage. Act. Process. (LEAP) quality improvement (QI) learning program. The primary outcomes will be operationalized across the three trials as a patient-level dichotomous response (yes/no) indicating patients with potentially inappropriate medications (PIMs) among those who may benefit from the EBP. This outcome will be computed using month-by-month administrative data. Primary comparison between the two implementation strategies will be analyzed using generalized estimating equations (GEE) with clinic-level monthly (13 to 36 months) percent of PIMs as the dependent variable. Primary comparative endpoint will be at 18 months post-baseline. Each trial will also be analyzed independently.

Discussion

MIDAS QUERI trials will focus on fostering sustained use of EBPs that previously had targeted but incomplete implementation. Our implementation approaches are designed to engage frontline clinicians in a dynamic optimization process that integrates the use of actional clinical data and making incremental changes, designed to be feasible within busy clinical settings.

Trial registration

ClinicalTrials.gov: NCT05065502. Registered October 4, 2021—retrospectively registered.

Supplementary Information

The online version contains supplementary material available at 10.1186/s43058-022-00297-z.

Toward a Learning Health Care System: A Systematic Review and Evidence-Based Conceptual Framework for Implementation of Clinical Analytics in a Digital Hospital

Objective  A learning health care system (LHS) uses routinely collected data to continuously monitor and improve health care outcomes. Little is reported on the challenges and methods used to implement the analytics underpinning an LHS. Our aim was to systematically review the literature for reports of real-time clinical analytics implementation in digital hospitals and to use these findings to synthesize a conceptual framework for LHS implementation.

Methods  Embase, PubMed, and Web of Science databases were searched for clinical analytics derived from electronic health records in adult inpatient and emergency department settings between 2015 and 2021. Evidence was coded from the final study selection that related to (1) dashboard implementation challenges, (2) methods to overcome implementation challenges, and (3) dashboard assessment and impact. The evidences obtained, together with evidence extracted from relevant prior reviews, were mapped to an existing digital health transformation model to derive a conceptual framework for LHS analytics implementation.

Results  A total of 238 candidate articles were reviewed and 14 met inclusion criteria. From the selected studies, we extracted 37 implementation challenges and 64 methods employed to overcome such challenges. We identified common approaches for evaluating the implementation of clinical dashboards. Six studies assessed clinical process outcomes and only four studies evaluated patient health outcomes. A conceptual framework for implementing the analytics of an LHS was developed.

Conclusion  Health care organizations face diverse challenges when trying to implement real-time data analytics. These challenges have shifted over the past decade. While prior reviews identified fundamental information problems, such as data size and complexity, our review uncovered more postpilot challenges, such as supporting diverse users, workflows, and user-interface screens. Our review identified practical methods to overcome these challenges which have been incorporated into a conceptual framework. It is hoped this framework will support health care organizations deploying near-real-time clinical dashboards and progress toward an LHS.

Dashboards in Health Care Settings: Protocol for a Scoping Review

Background

Health care organizations increasingly depend on business intelligence tools, including “dashboards,” to capture, analyze, and present data on performance metrics. Ideally, dashboards allow users to quickly visualize actionable data to inform and optimize clinical and organizational performance. In reality, dashboards are typically embedded in complex health care organizations with massive data streams and end users with distinct needs. Thus, designing effective dashboards is a challenging task and theoretical underpinnings of health care dashboards are poorly characterized; even the concept of the dashboard remains ill-defined. Researchers, informaticists, clinical managers, and health care administrators will benefit from a clearer understanding of how dashboards have been developed, implemented, and evaluated, and how the design, end user, and context influence their uptake and effectiveness.

Objective

This scoping review first aims to survey the vast published literature of “dashboards” to describe where, why, and for whom they are used in health care settings, as well as how they are developed, implemented, and evaluated. Further, we will examine how dashboard design and content is informed by intended purpose and end users.

Methods

In July 2020, we searched MEDLINE, Embase, Web of Science, and the Cochrane Library for peer-reviewed literature using a targeted strategy developed with a research librarian and retrieved 5188 results. Following deduplication, 3306 studies were screened in duplicate for title and abstract. Any abstracts mentioning a health care dashboard were retrieved in full text and are undergoing duplicate review for eligibility. Articles will be included for data extraction and analysis if they describe the development, implementation, or evaluation of a dashboard that was successfully used in routine workflow. Articles will be excluded if they were published before 2015, the full text is unavailable, they are in a non-English language, or they describe dashboards used for public health tracking, in settings where direct patient care is not provided, or in undergraduate medical education. Any discrepancies in eligibility determination will be adjudicated by a third reviewer. We chose to focus on articles published after 2015 and those that describe dashboards that were successfully used in routine practice to identify the most recent and relevant literature to support future dashboard development in the rapidly evolving field of health care informatics.

Results

All articles have undergone dual review for title and abstract, with a total of 2019 articles mentioning use of a health care dashboard retrieved in full text for further review. We are currently reviewing all full-text articles in duplicate. We aim to publish findings by mid-2022. Findings will be reported following guidance from the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist.

Conclusions

This scoping review will provide stakeholders with an overview of existing dashboard tools, highlighting the ways in which dashboards have been developed, implemented, and evaluated in different settings and for different end user groups, and identify potential research gaps. Findings will guide efforts to design and use dashboards in the health care sector more effectively.

International Registered Report Identifier (IRRID)

DERR1-10.2196/34894

The voluntary medical male circumcision Site Capacity and Productivity Assessment Tool (SCPT): An innovative visual management tool to optimize site service delivery

Given constrained funding for HIV, achieving global goals on VMMC scale-up requires that providers improve service delivery operations and use labor and capital inputs as efficiently as possible to produce as many quality VMMCs as feasible. The Voluntary Medical Male Circumcision Site Capacity and Productivity Assessment Tool (SCPT) is an electronic visual management tool developed to help VMMC service providers to understand and improve their site's performance. The SCPT allows VMMC providers to: 1) track the most important human resources and capital inputs to VMMC service delivery, 2) strategically plan site capacity and targets, and 3) monitor key site-level VMMC service delivery performance indicators. To illustrate a real-world application of the SCPT, we present selected data from two provinces in Mozambique-Manica and Tete, where the SCPT was piloted We looked at the data prior to the introduction of SCPT (October 2014 to August 2016), and during the period when the tool began to be utilized (September 2016 to September 2017). The tool was implemented as part of a broader VMMC site optimization strategy that VMMC implementers in Mozambique put in place to maximize programmatic impact. Routine program data for Manica and Tete from October 2014 to September 2017 showcase the turnaround of the VMMC program that accompanied the implementation of the SCPT together with the other components of the VMMC site optimizatio strategy. From October 2016, there was a dramatic increase in the number of VMMCs performed. The number of fixed service delivery sites providing VMMC services was expanded, and each fixed site extended service delivery by performing VMMCs in outreach sites. Alignment between site targets and the number of VMMCs performed improved from October 2016. Utilization rates stabilized between October 2016 and September 2017, with VMMCs performed closely tracking VMMC site capacity in most sites. The SCPT is designed to address the need for site level data for programmatic decision-making during site planning, implementation, monitoring and evaluation. Deployment of the SCPT can help VMMC providers monitor the performance of VMMC service delivery sites and improve their performance. We recommend use of the customized version of this tool and model to the need of other programs.

Development, Implementation, and User Evaluation of COVID-19 Dashboard in a Third-Level Hospital in Iran

INTRODUCTION

The implementation of a dashboard enables managers to make informed and evidence-based decisions through data visualization and graphical presentation of information. This study aimed to design and implement a COVID-19 management dashboard in a third-level hospital in Mashhad, Iran.

MATERIALS AND METHODS

This descriptive developmental applied study was conducted in the second half of 2020 in three stages, using user-centered design methodology in four phases: (1) specification of the application context, (2) specification of requirements, (3) creation of design solutions, and (4) evaluation of designs. Data collection in each phase was performed through holding group discussions with the main users, nominal group techniques, interviews, and questioners. The dashboard prototype for the data display was designed using the Power BI Desktop software. Subsequently, users' comments were obtained using the focus group method and included in the dashboard.

RESULTS

In total, 25 indicators related to input, process, and output areas were identified based on the findings of the first stage. Moreover, eight items were introduced by participants as dashboard requirements. The dashboard was developed based on users' feedback and suggestions, such as the use of colors, reception of periodic and specific reports based on key performance indicators, and rearrangement of the components visible on the page. The result of the user satisfaction survey indicated their satisfaction with the developed dashboard.

CONCLUSION

The selection of proper criteria for the implementation of an effective dashboard is critical for the health care organization since they are designed with a high-tech and content-based environment. The dashboard in the present study was a successful combination of clinical and managerial indicators. Future studies should focus on the design and development of dashboards, as well as benchmarking by using data from several hospitals.

A practical framework for performance measurement of facilities management services in developing countries’ public hospitals

Purpose

Facilities management (FM) plays a vital role toward creating the hospitable environment that supports the core clinical business of rendering quality health care. To ensure optimal performance of FM, public hospitals require a performance measurement (PM) framework that could guide and assist facilities managers in their operations. This paper aims to respond to this need by developing a performance measurement framework useful to improve FM performance within the context of developing countries hospital FM services delivery.

Design/methodology/approach

The paper synthesizes knowledge from a comprehensive review of FM performance measurement literature and the results of an earlier PhD research on FM performance measurement in Ghana’s public hospitals.

Findings

The paper presents a simplified performance measurement framework which demonstrates four key performance measurement tasks: establishing performance goals; measuring the FM service delivery; evaluating FM outcomes; and performance review and reporting. The study suggests the establishment of a central coordination point to enhance performance measurement activities of public hospitals’ FM practices.

Research limitations/implications

Although reference is made to research on key performance indicators (KPIs), future study is recommended to develop appropriate methodology for selecting relevant KPIs. The author also intends to pilot the framework to a health facility in Ghana to leverage its usefulness, given the fact that it has not been tested empirically.

Practical implications

The application of the framework is expected to lead to the identification of performance gaps and weakness which when acted upon could lead to operational effectiveness and efficiency of the FM process, ultimately leading to improved FM performance.

Originality/value

The paper develops a novel performance measurement framework useful within the context of the nascent FM industry in the developing world.

Co-designing a dashboard of predictive analytics and decision support to drive care quality and client outcomes in aged care: a mixed-method study protocol

INTRODUCTION

There is a clear need for improved care quality and quality monitoring in aged care. Aged care providers collect an abundance of data, yet rarely are these data integrated and transformed in real-time into actionable information to support evidence-based care, nor are they shared with older people and informal caregivers. This protocol describes the co-design and testing of a dashboard in residential aged care facilities (nursing or care homes) and community-based aged care settings (formal care provided at home or in the community). The dashboard will comprise integrated data to provide an 'at-a-glance' overview of aged care clients, indicators to identify clients at risk of fall-related hospitalisations and poor quality of life, and evidence-based decision support to minimise these risks. Longer term plans for dashboard implementation and evaluation are also outlined.

METHODS

This mixed-method study will involve (1) co-designing dashboard features with aged care staff, clients, informal caregivers and general practitioners (GPs), (2) integrating aged care data silos and developing risk models, and (3) testing dashboard prototypes with users. The dashboard features will be informed by direct observations of routine work, interviews, focus groups and co-design groups with users, and a community forum. Multivariable discrete time survival models will be used to develop risk indicators, using predictors from linked historical aged care and hospital data. Dashboard prototype testing will comprise interviews, focus groups and walk-through scenarios using a think-aloud approach with staff members, clients and informal caregivers, and a GP workshop.

ETHICS AND DISSEMINATION

This study has received ethical approval from the New South Wales (NSW) Population & Health Services Research Ethics Committee and Macquarie University's Human Research Ethics Committee. The research findings will be presented to the aged care provider who will share results with staff members, clients, residents and informal caregivers. Findings will be disseminated as peer-reviewed journal articles, policy briefs and conference presentations.

A performance measurement model for the housekeeping services in healthcare facilities

Purpose

The purpose of this study is to develop a customised model to measure the performance of housekeeping (HK) services in Sri Lankan health-care facilities.

Design/methodology/approach

An exploratory sequential mixed approach was adopted in the study to collect and analyse data. A case study strategy was adopted in the initial phase to explore the current HK practices, and to determine the applicable key performance indicators (KPIs) found through literature. Semi-structured interviews were used as the data collection method in the selected case studies. A questionnaire survey was conducted in the next phase to verify the validity of case study findings. Quantitative data were analysed using descriptive statistics. One sample t-test was used to identify significant KPIs and to formulate the customised performance measurement (PM) model.

Findings

The study identified and ranked 46 KPIs, which can be used to measure the performance of HK divisions of health-care facilities, in balanced scorecard perspectives. The findings also revealed that the KPIs determined using the model have a significant impact on implementing effective health-care HK services and could be used to measure both real and subjective cleanliness.

Practical implications

The developed model can be used for numerous PM requirements in health-care setups in both developing and developed countries. The KPIs determined using this model can be presented in quality audits and government inspections as proof of effective HK management. The HK managers of the health-care sector can effectively monitor the performance of their divisions using the proposed PM model and this model can be customised to match the other facilities management service divisions.

Originality/value

Many studies focus on overall health-care PM. This study expands the knowledge on HK PM in the health-care sector by presenting a collection of performance indicators centred on HK function.

A Dashboard for Tracking Mortality After Cardiac Surgery Using a National Administrative Database

BACKGROUND

Mortality after cardiac surgery is publicly reportable and used as a quality metric by national organizations. However, detailed institutional comparisons are often limited in publicly reported ratings, while publicly reported mortality data are generally limited to 30-day outcomes. Dashboards represent a useful method for aggregating data to identify areas for quality improvement.

METHODS

We present the development of a dashboard of cardiac surgery performance using cardiac surgery admissions in a national administrative dataset, allowing institutions to better analyze their clinical outcomes. We identified cardiac surgery admissions in the Medicare Limited Data Sets from April 2016 to March 2017 using diagnosis-related group (DRG) codes for cardiac valve and coronary bypass surgeries.

RESULTS

Using these data, we created a dashboard prototype to enable hospitals to compare their individual performance against state and national benchmarks, by all cardiac surgeries, specific cardiac surgery DRGs and by specific surgeons. Mortality rates are provided at 30, 60 and 90 days post-operatively as well as 1 year. Users can filter results by state, hospital and surgeon, and visualize summary data comparing these filtered results to national metrics. Examples of using the dashboard to examine hospital and individual surgeon mortality are provided.

CONCLUSIONS

We demonstrate how this database can be used to compare data between comparator hospitals on local, state and national levels to identify trends in mortality and areas for quality improvement.

Lessons from mandated implementation of a performance management system

PURPOSE

Lean-inspired approaches and performance management systems are being implemented in public healthcare organisations internationally. However, the literature is inconclusive regarding the benefits of these management tools and there is a lack of knowledge regarding processes for large-scale implementation of these tools. This article aims to describe the implementation process and to better understand how this process influences the mandated performance management system.

DESIGN/METHODOLOGY/APPROACH

This research is based on a comparative case study of three healthcare organisations in Canada. Data consist documents, non-participant observation and semi-structured interviews with key actors (n = 30). Analysis is based on a sociotechnical approach to management tools that considers organisational context, and the tool's technical substrate, theory of action and managerial philosophy.

FINDINGS

Results show that despite a standardised national mandate, the tool as implemented varied between organisations in terms of technical substrate and managerial philosophy. These variations are explained by the flexibility of the technical substrate, the lack of clarity of the managerial philosophy, and some contextual elements. Successful implementation may rest upon high hybridization of the tool on these different dimensions. A precise and prescribed technical substrate is not sufficient to guarantee implementation of a managerial philosophy.

PRACTICAL IMPLICATIONS

Mandated implementation of management tools may be more successful if it is explicit on the managerial philosophy, the technical substrate and the link between the two, and if it provides some leeway to adapt both to the organisational context.

ORIGINALITY/VALUE

This is one of the few studies to describe and analyse the process involved in mandated large-scale implementation of performance management systems in public healthcare organisations.

Evaluation of a Clinical Platform to Promote Chronic Disease Management

Self-reported data suggest American adults with multiple chronic conditions account for 25.7% to 60% of the population. Despite emphasis on information technology to improve quality in health care, data addressing outcomes of clinically focused, provider-oriented dashboards are limited. To explore integrating performance platforms into clinical care, the authors designed a platform-based intervention to address 2 prevalent chronic conditions with significant long-term burden. This study used a performance platform to enhance clinicians' management of patients with diabetes and osteoporosis. Descriptive statistics were used to summarize patients' surveys and quality metrics, and to analyze clinicians' knowledge, attitudes, and beliefs in the pre and post time frames. The frequency of screening for osteoporosis in women improved post intervention (40% vs. 44%, P < 0.0001), whereas other quality metrics did not. Clinician respondents were primarily physicians (82%), white (73%), internal medicine specialists (58%), with an average of 18 years' experience, and nearly equally male and female. Their percent of correctly answered knowledge questions increased slightly in the postintervention phase for osteoporosis and hypoglycemia (0.53 and 1.74, respectively); however, results were not statistically significant (P > 0.4). Post intervention, clinicians reported that their attitudes and beliefs regarding disease management had changed in the past 6 months in a positive direction. Although few outcomes studied changed over time, results suggest that performance platforms may have a role to play in managing chronic conditions. However, their efficacy must continue to be evaluated in order to improve understanding of optimal approaches to integrating technology into patient care.

A visual management tool for program planning, project management and evaluation in paediatric health care

This article describes the development and implementation of a custom-designed Excel-based visual management tool. The tool's purpose was to support program planning and evaluation by our resource support team within a paediatric health care setting. Our aims in developing it were to 1) establish a streamlined process and supporting tools to efficiently plan and prioritize program directions and activities; 2) track progress; and 3) evaluate and report on our performance, outputs and outcomes. A collaborative approach based on the ADKAR (Awareness, Desire, Knowledge, Ability, Reinforcement) change management model and the LEADS (Lead self, Engage others, Achieve results, Develop coalitions, Systems transformation) leadership framework was used to guide the design and implementation processes. Team members reported high perceived effectiveness and efficiency with respect to the tool's utility in supporting its proposed aims. A graded approach to building knowledge and skills in using the tool, to individual responsibility for data entry, and to accountability by team members facilitated its successful implementation. Administrative support is important for sustainability and continual improvement of the tool to address changing team needs over time.

The relative citation ratio: what is it and why should medical librarians care?

Bibliometrics is becoming increasingly prominent in the world of medical libraries. The number of presentations related to research impact at the Medical Library Association (MLA) annual meeting has been increasing in past years. Medical centers have been using institutional dashboards to track clinical performance for over a decade, and more recently, these institutional dashboards have included measures of academic performance. This commentary reviews current practices and considers the role for a newer metric, the relative citation ratio.

A Dashboard for Monitoring Opioid-Related Adverse Drug Events Following Surgery Using a National Administrative Database

Opioid-related adverse drug events (ORADEs) include a range of complications, from respiratory arrest to ileus and urinary retention. ORADEs correlate to morbidity, mortality, and increased costs. The Centers for Medicare & Medicaid Services database, which represents approximately 35% of hospital discharges. The authors searched for previously published ICD-9 codes that defined ORADEs. A group of surgical diagnosis-related groups (DRGs) were selected. Recurring queries were programmed using these ICD codes and DRGs and used to update an online dashboard. The dashboard presents an estimate of the burden of ORADEs to frontline clinicians and hospital leadership and allows users to compare local data on ORADEs rates to other hospitals. Users are able to refine their search by surgery type or ORADE type. An interface was created, using national administrative claims data, to allow hospitals to access their ORADEs and benchmark local data against national trends.

Quality Improvement in Anesthesiology - Leveraging Data and Analytics to Optimize Outcomes

Quality improvement is at the heart of practice of anesthesiology. Objective data are critical for any quality improvement initiative; when possible, a combination of process, outcome, and balancing metrics should be evaluated to gauge the value of an intervention. Quality improvement is an ongoing process; iterative reevaluation of data is required to maintain interventions, ensure continued effectiveness, and continually improve. Dashboards can facilitate rapid analysis of data and drive decision making. Large data sets can be useful to establish benchmarks and compare performance against other providers, practices, or institutions. Audit and feedback strategies are effective in facilitating positive change.