Developing a hospital-wide quality and safety dashboard: a qualitative research study

TeamSTEPPS improves patient safety

BACKGROUND

Examine how Team Strategies and Tools to Enhance Performance and Patient Safety (TeamSTEPPS) can be used to manage patient safety and improve the standard of care for patients.

METHODS

In order to improve key medical training in areas like surgical safety management, blood transfusion closed-loop management, drug safety management and identity recognition, we apply the TeamSTEPPS teaching methodology. We then examine the effects of this implementation on changes in pertinent indicators.

RESULTS

Our hospital's perioperative death rate dropped to 0.019%, unscheduled reoperations dropped to 0.11%, and defined daily doses fell to 24.85. Antibiotic usage among hospitalised patients declined to 40.59%, while the percentage of antibacterial medicine prescriptions for outpatient patients decreased to 13.26%. Identity recognition requirements were implemented at a rate of 94.5%, and the low-risk group's death rate dropped to 0.01%. Critical transfusion episodes were less common, with an incidence of 0.01%. The physician's TeamSTEPPS Teamwork Perceptions Questionnaire and Teamwork Attitudes Questionnaire scores dramatically improved following the TeamSTEPPS team instruction course.

CONCLUSION

An evidence-based team collaboration training programme called TeamSTEPPS combines clinical practice with team collaboration skills to enhance team performance in the healthcare industry and raise standards for medical quality, safety, and effectiveness.

Quality Dashboards in Hospital Settings: A Systematic Review With Implications for Nurses

BACKGROUND

Dashboards visually display quality and safety data to aid nurses in making informed decisions.

PURPOSE

This systematic review evaluated quality improvement (QI) dashboard characteristics associated with interventions to improve patient outcomes and positive end-user evaluation.

METHODS

Literature was searched from 2012 to 2022 in PubMed, CINAHL, Scopus, MEDLINE, and Google Scholar.

RESULTS

Sixteen articles were included. Varied dashboard characteristics were noted, with mixed patient outcomes and end-user responses. Graphs and tabular presentations were associated with improved patient outcomes, whereas graphs were associated with end-user satisfaction. Benchmarks were noted with improved patient outcomes but not end-user satisfaction. Interactive dashboards were important for end users and improved patient outcomes.

CONCLUSION

Nurses can find dashboards helpful in guiding QI projects. Dashboards may include graphs and/or tables, benchmarks, and interactivity but should be useful, usable, and aligned to unit needs. Future research should focus on the use of quality dashboards in nursing practice.

Building a NICU quality & safety infrastructure

The American Academy of Pediatrics (AAP) Standards for Levels of Neonatal Care, published in 2023, highlights key components of a Neonatal Patient Safety and Quality Improvement Program (NPSQIP). A comprehensive Neonatal Intensive Care Unit (NICU) quality and safety infrastructure (QSI) is based on four foundational domains: quality improvement, quality assurance, safety culture, and clinical guidelines. This paper serves as an operational guide for NICU clinical leaders and quality champions to navigate these domains and develop their local QSI to include the AAP NPSQIP standards.

Identifying and prioritizing the key performance indicators for hospital management dashboard at a national level: Viewpoint of hospital managers

Participation of main users in identifying key performance indicators (KPIs) for management dashboards contributes to their success. The aim of this study was to identify and prioritize the KPIs of hospital management dashboards from the viewpoint of hospital managers. This study was conducted on managers of public hospitals at a national level in Iran in 2020. Data were collected using a self-administrated questionnaire. The KPIs were classified into five categories, namely financial, operational, human resources, safety and quality of care, services provided to patients. A total of 234 hospital managers participated in this study. Totally, 25 KPIs were determined for the hospital management dashboard, including the patient falls rate, waiting time for patients in the emergency department, patient satisfaction, total hospital revenue, financial balance, bed occupancy rate, patients' discharge with own agreement, average length of stay, and personnel satisfaction. For designing hospital management dashboards, the domains of services provided to patients, safety and quality of care, financial resources, human resources, and operational are important from the hospital managers' viewpoint, respectively. The results of this study can be helpful for developers of business intelligence tools, such as hospital management dashboards, to visualize the most important indicators for managers.

Triptychon: Usability evaluation and implementation of a web-based application for patients' lab and vital parameters

Background

A major challenge in healthcare is the interpretation of the constantly increasing amount of clinical data of interest to inpatients for diagnosis and therapy. It is vital to accurately structure and represent data from different sources to help clinicians make informed decisions.

Objective

We evaluated the usability of our tool 'Triptychon' - a three-part visualisation dashboard of essential patients' medical data provided by a direct overview of their hospitalisation information, laboratory, and vital parameters over time.

Methods

The study followed a cohort of 20 participants using the mixed-methods approach, including interviews and the usability questionnaires, Health Information Technology Usability Evaluation Scale (Health-ITUES), and User Experience Questionnaire (UEQ). The participant's interactions with the dashboard were also observed. A thematic analysis approach was applied to analyse qualitative data and the quantitative data's task completion time and success rates.

Results

The usability evaluation of the visualisation dashboard revealed issues relating to the terminology used in the user interface and colour coding in its left and middle panels. The Health-ITUES score was 3.72 (standard deviation (SD) = 1.0), and the UEQ score was 1.6 (SD = 0.74). The study demonstrated improvements in intuitive dashboard use and overall satisfaction with using the dashboard daily.

Conclusion

The Triptychon dashboard is a promising new tool for medical data presentation. We identified design and layout issues of the dashboard for improving its usability in routine clinical practice. According to users' feedback, the three panels on the dashboard provided a holistic view of a patient's hospital stay.

How do we measure organisational wellness? Development of a comprehensive patient-centred and employee-centred visual analytical solution

BACKGROUND

Dashboards are visual information systems frequently employed by healthcare organisations to track key quality improvement and patient safety performance metrics. The typical healthcare dashboard focuses on specific metrics, disease processes or units within a larger healthcare organisation. Here, we describe the development of a visual analytical solution (keystone dashboard) for monitoring an entire healthcare organisation.

METHODS

The improvement team reviewed and assessed various data sources across the organisation and selected a group of patient and employee related metrics that afforded a broad overview of the organisation's well-being. Metrics spanned the organisation and included data from patient safety, quality improvement, human resources, risk management and medical staff affairs. Each metric was assigned a numeric weight that correlated with its impact. A visual model incorporating the various data fields was then constructed.

RESULTS

The keystone dashboard incorporates a data heatmap and density visualisation to emphasis areas of higher density and/or weighted values. The heatmap is used to indicate the weight/magnitude of each metric within a data range in two dimensions: location and time. The visualisation 'heats up' depending on the combination of counts events and their assigned impact for the reporting month. Most data sources update in near real time.

SUMMARY

The keystone dashboard serves as a comprehensive and collaborative integration of data from patient safety, quality improvement, human resources, risk management and medical staff affairs. This visual analytical solution incorporates and analyses metrics into a single view with the intent of providing valuable insight into the health of an entire organisation. This dashboard is unique as it provides a broad overview of a healthcare organisation by incorporating key metrics that span the organisation.

Requirements and challenges of hospital dashboards: a systematic literature review

Background

Today, the use of data in administrative and clinical processes is quite challenging due to the large volume of data, data collection from various sources, and lack of data structure. As a data management tool, dashboards play an important role in timely visual display of critical information on key performances.

Objectives

This systematic review aimed to identify functional and non-functional requirements, as well as challenges of using dashboards in hospitals.

Methods

In this systematic review, four databases, including the Web of Science, PubMed, EMBASE, and Scopus, were searched to find relevant articles from 2000 until May 30, 2020. The final search was conducted on May 30, 2020. Data collection was performed using a data extraction form and reviewing the content of relevant studies on the potentials and challenges of dashboard implementation.

Results

Fifty-four out of 1254 retrieved articles were selected for this study based on the inclusion and exclusion criteria. The functional requirements for dashboards included reporting, reminders, customization, tracking, alert creation, and assessment of performance indicators. On the other hand, the non-functional requirements included the dashboard speed, security, ease of use, installation on different devices (e.g., PCs and laptops), integration with other systems, web-based design, inclusion of a data warehouse, being up-to-data, and use of data visualization elements based on the user’s needs. Moreover, the identified challenges were categorized into four groups: data sources, dashboard content, dashboard design, implementation, and integration in other systems at the hospital level.

Conclusion

Dashboards, by providing information in an appropriate manner, can lead to the proper use of information by users. In order for a dashboard to be effective in clinical and managerial processes, particular attention must be paid to its capabilities, and the challenges of its implementation need to be addressed.

Quality improvements of healthcare trajectories by learning from aggregated patient-reported outcomes: a mixed-methods systematic literature review

Background

In healthcare, analysing patient-reported outcome measures (PROMs) on an aggregated level can improve and regulate healthcare for specific patient populations (meso level). This mixed-methods systematic review aimed to summarize and describe the effectiveness of quality improvement methods based on aggregated PROMs. Additionally, it aimed to describe barriers, facilitators and lessons learned when using these quality improvement methods.

Methods

A mixed-methods systematic review was conducted. Embase, MEDLINE, CINAHL and the Cochrane Library were searched for studies that described, implemented or evaluated a quality improvement method based on aggregated PROMs in the curative hospital setting. Quality assessment was conducted via the Mixed Methods Appraisal Tool. Quantitative data were synthesized into a narrative summary of the characteristics and findings. For the qualitative analysis, a thematic synthesis was conducted.

Results

From 2360 unique search records, 13 quantitative and three qualitative studies were included. Four quality improvement methods were identified: benchmarking, plan-do-study-act cycle, dashboards and internal statistical analysis. Five studies reported on the effectiveness of the use of aggregated PROMs, of which four identified no effect and one a positive effect. The qualitative analysis identified the following themes for facilitators and barriers: (1) conceptual (i.e. stakeholders, subjectivity of PROMs, aligning PROMs with clinical data, PROMs versus patient-reported experience measures [PREMs]); (2a) methodological—data collection (i.e. choice, timing, response rate and focus); (2b) methodological—data processing (i.e. representativeness, responsibility, case-mix control, interpretation); (3) practical (i.e. resources).

Conclusion

The results showed little to no effect of quality improvement methods based on aggregated PROMs, but more empirical research is needed to investigate different quality improvement methods. A shared stakeholder vision, selection of PROMs, timing of measurement and feedback, information on interpretation of data, reduction of missing data, and resources for data collection and feedback infrastructure are important to consider when implementing and evaluating quality improvement methods in future research.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12961-022-00893-4.

What is already known on this topic

The aggregated patient-reported outcome measures (PROMs) can be used for analytical and organizational aspects of improving and regulating healthcare, but there is little empirical evidence regarding the effectiveness of aggregated PROMS.

What this study adds

This study adds a detailed overview of the types of quality improvement methods and recommendations for implementation in practice.

How this study might affect research, practice or policy

Researchers and policy-makers should consider the barriers, facilitators and lessons learned for future implementation and evaluation of quality improvement methods, as presented in this manuscript, to further advance this field.

Utility of the finance-electronic medical record digital dashboard in pediatric otolaryngology

BACKGROUND

The time and cost of data collection via chart review of the electronic medical record (EMR) is a research barrier. This study describes the development of a digital dashboard conjoining EMR and finance data and its application in a pediatric otolaryngology practice.

METHODS

The dashboard creates a common language crosswalk between surgeries via the EMR, financial data, and national Vizient database. First, all Otolaryngology procedures billed via ICD-10 or CPT codes were categorized into Procedure Groups, which constitute the common language that links all data sources. The joined dataset was inputted into a Tableau workbook supporting dynamic filtering and custom real-time analysis.

RESULTS

The dashboard includes 84 Procedure Groups within Otolaryngology. Examples for pediatrics include Sistrunk procedure and supraglottoplasty. User-friendly dynamic filtering by Procedure Group, surgery date range, age, insurance, hospital, surgeon, and discharge status were developed. Outcomes include length of stay, telephone callbacks, postoperative hemorrhage, reoperations, return to Emergency Department, readmissions, and mortality. National comparisons can be analyzed via embedded Vizient data. The usability of the dashboard was tested by evaluating pediatric tonsillectomy outcomes, which revealed a significantly higher rate of postoperative hemorrhages and reoperations during the COVID-19 pandemic.

CONCLUSION

The hybrid finance/EMR dashboard creates a crosswalk between data sources and shows utility for use in evaluating patient outcomes via real-time data analysis and dynamic filtering. This innovative dashboard expedites data extraction, promoting efficient implementation of quality improvement initiatives and surgical outcomes research.

Real-time analysis and display of quantitative measures to track and improve clinical workflow

Purpose

Radiotherapy treatment planning is a complex process with multiple, dependent steps involving an interdisciplinary patient care team. Effective communication and real‐time tracking of resources and care path activities are key for clinical efficiency and patient safety.

Materials and Methods

We designed and implemented a secure, interactive web‐based dashboard for patient care path, clinical workflow, and resource utilization management. The dashboard enables visualization of resource utilization and tracks progress in a patient's care path from the time of acquisition of the planning CT to the time of treatment in real‐time. It integrates with the departmental electronic medical records (EMR) system without the creation and maintenance of a separate database or duplication of work by clinical staff. Performance measures of workflow were calculated.

Results

The dashboard implements a standardized clinical workflow and dynamically consolidates real‐time information queried from multiple tables in the EMR database over the following views: (1) CT Sims summarizes patient appointment information on the CT simulator and patient load; (2) Linac Sims summarizes patient appointment times, setup history, and notes, and patient load; (3) Task Status lists the clinical tasks associated with a treatment plan, their due date, status and ownership, and patient appointment details; (4) Documents provides the status of all documents in the patients' charts; and (5) Diagnoses and Interventions summarizes prescription information, imaging instructions and whether the plan was approved for treatment. Real‐time assessment and quantification of progress and delays in a patient's treatment start were achieved.

Conclusions

This study indicates it is feasible to develop and implement a dashboard, tailored to the needs of an interdisciplinary team, which derives and integrates information from the EMR database for real‐time analysis and display of resource utilization and clinical workflow in radiation oncology. The framework developed facilitates informed, data‐driven decisions on clinical workflow management as we seek to optimize clinical efficiency and patient safety.

Health Care Quality and Safety in a Correctional System: Creating Goals and Performance Measures for Improvement

This article summarizes a collaboration between the Illinois Department of Corrections and an academic partner, University of Illinois at Chicago College of Nursing (UIC Nursing), to deliver a quality improvement (QI) and patient safety plan to the state. Included is a description of the need for meaningful data-driven QI as a method to deliver health care to the incarcerated population. An overview of the assessment, gap analysis, and collaboration between partners to develop a mission, vision, and performance dashboard with a beginning set of measures is presented. The dashboard with performance measures was developed to highlight the priorities of the organization and to focus improvement work to meet those goals. Next step recommendations for UIC Nursing to consult on implementation priorities and training are described.

Big Data Health Care Innovations: Performance Dashboarding as a Process of Collective Sensemaking

Big data is poised to revolutionize health care, and performance dashboards can be an important tool to manage big data innovations. Dashboards show the progress being made and provide critical management information about effectiveness and efficiency. However, performance dashboards are more than just a clear and straightforward representation of performance in the health care context. Instead, the development and maintenance of informative dashboards can be more productively viewed as an interactive and iterative process involving all stakeholders. We refer to this process as dashboarding and reflect on our learnings within a large European Union–funded project. Within this project, multiple big data applications in health care are being developed, piloted, and scaled up. In this paper, we discuss the ways in which we cope with the inherent sensitivities and tensions surrounding dashboarding in such a dynamic environment.

Enriching the Value of Patient Experience Feedback: Web-Based Dashboard Development Using Co-design and Heuristic Evaluation

Background

There is an abundance of patient experience data held within health care organizations, but stakeholders and staff are often unable to use the output in a meaningful and timely way to improve care delivery. Dashboards, which use visualized data to summarize key patient experience feedback, have the potential to address these issues.

Objective

The aim of this study is to develop a patient experience dashboard with an emphasis on Friends and Family Test (FFT) reporting, as per the national policy drive.

Methods

A 2-stage approach was used—participatory co-design involving 20 co-designers to develop a dashboard prototype, followed by iterative dashboard testing. Language analysis was performed on free-text patient experience data from the FFT, and the themes and sentiments generated were used to populate the dashboard with associated FFT metrics. Heuristic evaluation and usability testing were conducted to refine the dashboard and assess user satisfaction using the system usability score.

Results

The qualitative analysis from the co-design process informed the development of the dashboard prototype with key dashboard requirements and a significant preference for bubble chart display. The heuristic evaluation revealed that most cumulative scores had no usability problems (18/20, 90%), had cosmetic problems only (7/20, 35%), or had minor usability problems (5/20, 25%). The mean System Usability Scale score was 89.7 (SD 7.9), suggesting an excellent rating.

Conclusions

The growing capacity to collect and process patient experience data suggests that data visualization will be increasingly important in turning feedback into improvements to care. Through heuristic usability, we demonstrated that very large FFT data can be presented in a thematically driven, simple visual display without the loss of the nuances and still allow for the exploration of the original free-text comments. This study establishes guidance for optimizing the design of patient experience dashboards that health care providers find meaningful, which in turn drives patient-centered care.

The experiences of 33 national COVID-19 dashboard teams during the first year of the pandemic in the World Health Organization European Region: A qualitative study

Background

Governments across the World Health Organization (WHO) European Region have prioritised dashboards for reporting COVID-19 data. The ubiquitous use of dashboards for public reporting is a novel phenomenon.

Objective

This study explores the development of COVID-19 dashboards during the first year of the pandemic and identifies common barriers, enablers and lessons from the experiences of teams responsible for their development.

Methods

We applied multiple methods to identify and recruit COVID-19 dashboard teams, using a purposive, quota sampling approach. Semi-structured group interviews were conducted from April to June 2021. Using elaborative coding and thematic analysis, we derived descriptive and explanatory themes from the interview data. A validation workshop was held with study participants in June 2021.

Results

Eighty informants participated, representing 33 national COVID-19 dashboard teams across the WHO European Region. Most dashboards were launched swiftly during the first months of the pandemic, February to May 2020. The urgency, intense workload, limited human resources, data and privacy constraints and public scrutiny were common challenges in the initial development stage. Themes related to barriers or enablers were identified, pertaining to the pre-pandemic context, pandemic itself, people and processes and software, data and users. Lessons emerged around the themes of simplicity, trust, partnership, software and data and change.

Conclusions

COVID-19 dashboards were developed in a learning-by-doing approach. The experiences of teams reveal that initial underpreparedness was offset by high-level political endorsement, the professionalism of teams, accelerated data improvements and immediate support with commercial software solutions. To leverage the full potential of dashboards for health data reporting, investments are needed at the team, national and pan-European levels.

Development, Implementation, and User Evaluation of COVID-19 Dashboard in a Third-Level Hospital in Iran

INTRODUCTION

The implementation of a dashboard enables managers to make informed and evidence-based decisions through data visualization and graphical presentation of information. This study aimed to design and implement a COVID-19 management dashboard in a third-level hospital in Mashhad, Iran.

MATERIALS AND METHODS

This descriptive developmental applied study was conducted in the second half of 2020 in three stages, using user-centered design methodology in four phases: (1) specification of the application context, (2) specification of requirements, (3) creation of design solutions, and (4) evaluation of designs. Data collection in each phase was performed through holding group discussions with the main users, nominal group techniques, interviews, and questioners. The dashboard prototype for the data display was designed using the Power BI Desktop software. Subsequently, users' comments were obtained using the focus group method and included in the dashboard.

RESULTS

In total, 25 indicators related to input, process, and output areas were identified based on the findings of the first stage. Moreover, eight items were introduced by participants as dashboard requirements. The dashboard was developed based on users' feedback and suggestions, such as the use of colors, reception of periodic and specific reports based on key performance indicators, and rearrangement of the components visible on the page. The result of the user satisfaction survey indicated their satisfaction with the developed dashboard.

CONCLUSION

The selection of proper criteria for the implementation of an effective dashboard is critical for the health care organization since they are designed with a high-tech and content-based environment. The dashboard in the present study was a successful combination of clinical and managerial indicators. Future studies should focus on the design and development of dashboards, as well as benchmarking by using data from several hospitals.

Exploring Changes to the Actionability of COVID-19 Dashboards Over the Course of 2020 in the Canadian Context: Descriptive Assessment and Expert Appraisal Study

BACKGROUND

Public web-based COVID-19 dashboards are in use worldwide to communicate pandemic-related information. Actionability of dashboards, as a predictor of their potential use for data-driven decision-making, was assessed in a global study during the early stages of the pandemic. It revealed a widespread lack of features needed to support actionability. In view of the inherently dynamic nature of dashboards and their unprecedented speed of creation, the evolution of dashboards and changes to their actionability merit exploration.

OBJECTIVE

We aimed to explore how COVID-19 dashboards evolved in the Canadian context during 2020 and whether the presence of actionability features changed over time.

METHODS

We conducted a descriptive assessment of a pan-Canadian sample of COVID-19 dashboards (N=26), followed by an appraisal of changes to their actionability by a panel of expert scorers (N=8). Scorers assessed the dashboards at two points in time, July and November 2020, using an assessment tool informed by communication theory and health care performance intelligence. Applying the nominal group technique, scorers were grouped in panels of three, and evaluated the presence of the seven defined features of highly actionable dashboards at each time point.

RESULTS

Improvements had been made to the dashboards over time. These predominantly involved data provision (specificity of geographic breakdowns, range of indicators reported, and explanations of data sources or calculations) and advancements enabled by the technologies employed (customization of time trends and interactive or visual chart elements). Further improvements in actionability were noted especially in features involving local-level data provision, time-trend reporting, and indicator management. No improvements were found in communicative elements (clarity of purpose and audience), while the use of storytelling techniques to narrate trends remained largely absent from the dashboards.

CONCLUSIONS

Improvements to COVID-19 dashboards in the Canadian context during 2020 were seen mostly in data availability and dashboard technology. Further improving the actionability of dashboards for public reporting will require attention to both technical and organizational aspects of dashboard development. Such efforts would include better skill-mixing across disciplines, continued investment in data standards, and clearer mandates for their developers to ensure accountability and the development of purpose-driven dashboards.

Improving the Standard of Care for All-A Practical Guide to Developing a Center of Excellence

Pancreatic surgery is one of the more challenging procedures performed by surgeons. The operations are technically complex and have historically been accompanied by a substantial risk for mortality and postoperative complications. Other pancreatic pathologies require advanced therapeutic procedures that are highly endoscopist-dependent, requiring specific, knowledge-based training for optimal outcomes. An increase in diagnosed pancreatic pathologies every year reinforces a critical need for experienced surgeons, gastroenterologists/endoscopists, hospitals, and support personnel in the management of complex pancreatic cases and thus, well-designed Centers of Excellence (CoE). In this paper, we outline the framework for a Pancreas CoE across three developmental domains: (1) establishing the foundation; (2) formalizing the program; (3) solidifying the CoE status. This framework can likely be translated to any disease or procedure-specific service-line and facilitate the development of a successful CoE.

Requirements for a Dashboard to Support Quality Improvement Teams in Pain Management

Pain management is often considered lower priority than many other aspects of health management in hospitals. However, there is potential for Quality Improvement (QI) teams to improve pain management by visualising and exploring pain data sets. Although dashboards are already used by QI teams in hospitals, there is limited evidence of teams accessing visualisations to support their decision making. This study aims to identify the needs of the QI team in a UK Critical Care Unit (CCU) and develop dashboards that visualise longitudinal data on the efficacy of patient pain management to assist the team in making informed decisions to improve pain management within the CCU. This research is based on an analysis of transcripts of interviews with healthcare professionals with a variety of roles in the CCU and their evaluation of probes. We identified two key uses of pain data: direct patient care (focusing on individual patient data) and QI (aggregating data across the CCU and over time); in this paper, we focus on the QI role. We have identified how CCU staff currently interpret information and determine what supplementary information can better inform their decision making and support sensemaking. From these, a set of data visualisations has been proposed, for integration with the hospital electronic health record. These visualisations are being iteratively refined in collaboration with CCU staff and technical staff responsible for maintaining the electronic health record. The paper presents user requirements for QI in pain management and a set of visualisations, including the design rationale behind the various methods proposed for visualising and exploring pain data using dashboards.

Features Constituting Actionable COVID-19 Dashboards: Descriptive Assessment and Expert Appraisal of 158 Public Web-Based COVID-19 Dashboards

Background

Since the outbreak of COVID-19, the development of dashboards as dynamic, visual tools for communicating COVID-19 data has surged worldwide. Dashboards can inform decision-making and support behavior change. To do so, they must be actionable. The features that constitute an actionable dashboard in the context of the COVID-19 pandemic have not been rigorously assessed.

Objective

The aim of this study is to explore the characteristics of public web-based COVID-19 dashboards by assessing their purpose and users (“why”), content and data (“what”), and analyses and displays (“how” they communicate COVID-19 data), and ultimately to appraise the common features of highly actionable dashboards.

Methods

We conducted a descriptive assessment and scoring using nominal group technique with an international panel of experts (n=17) on a global sample of COVID-19 dashboards in July 2020. The sequence of steps included multimethod sampling of dashboards; development and piloting of an assessment tool; data extraction and an initial round of actionability scoring; a workshop based on a preliminary analysis of the results; and reconsideration of actionability scores followed by joint determination of common features of highly actionable dashboards. We used descriptive statistics and thematic analysis to explore the findings by research question.

Results

A total of 158 dashboards from 53 countries were assessed. Dashboards were predominately developed by government authorities (100/158, 63.0%) and were national (93/158, 58.9%) in scope. We found that only 20 of the 158 dashboards (12.7%) stated both their primary purpose and intended audience. Nearly all dashboards reported epidemiological indicators (155/158, 98.1%), followed by health system management indicators (85/158, 53.8%), whereas indicators on social and economic impact and behavioral insights were the least reported (7/158, 4.4% and 2/158, 1.3%, respectively). Approximately a quarter of the dashboards (39/158, 24.7%) did not report their data sources. The dashboards predominately reported time trends and disaggregated data by two geographic levels and by age and sex. The dashboards used an average of 2.2 types of displays (SD 0.86); these were mostly graphs and maps, followed by tables. To support data interpretation, color-coding was common (93/158, 89.4%), although only one-fifth of the dashboards (31/158, 19.6%) included text explaining the quality and meaning of the data. In total, 20/158 dashboards (12.7%) were appraised as highly actionable, and seven common features were identified between them. Actionable COVID-19 dashboards (1) know their audience and information needs; (2) manage the type, volume, and flow of displayed information; (3) report data sources and methods clearly; (4) link time trends to policy decisions; (5) provide data that are “close to home”; (6) break down the population into relevant subgroups; and (7) use storytelling and visual cues.

Conclusions

COVID-19 dashboards are diverse in the why, what, and how by which they communicate insights on the pandemic and support data-driven decision-making. To leverage their full potential, dashboard developers should consider adopting the seven actionability features identified.

Measurement and monitoring patient safety in prehospital care: a systematic review

BACKGROUND

Prehospital care is potentially hazardous with the possibility for patients to experience an adverse event. However, as compared to secondary care, little is known about how patient safety is managed in prehospital care settings.

OBJECTIVES

The objectives of this systematic review were to identify and classify the methods of measuring and monitoring patient safety that have been used in prehospital care using the five dimensions of the Measuring and Monitoring Safety (MMS) framework and use this classification to identify where there are safety 'blind spots' and make recommendations for how these deficits could be addressed.

METHODS

Searches were conducted in January 2020, with no limit on publication year, using Medline, PsycInfo, CINAHL, Web of Science and Academic Search. Reference lists of included studies and existing related reviews were also screened. English-language, peer-reviewed studies concerned with measuring and monitoring safety in prehospital care were included. Two researchers independently extracted data from studies and applied a quality appraisal tool (the Quality Assessment Tool for Studies with Diverse Designs).

RESULTS

A total of 5301 studies were screened, with 52 included in the review. A total of 73% (38/52) of the studies assessed past harm, 25% (13/52) the reliability of safety critical processes, 1.9% (1/52) sensitivity to operations, 38.5% (20/52) anticipation and preparedness and 5.8% (3/52) integration and learning. A total of 67 methods for measuring and monitoring safety were used across the included studies. Of these methods, 38.8% (26/67) were surveys, 29.9% (20/67) were patient records reviews, 14.9% (10/67) were incident reporting systems, 11.9% (8/67) were interviews or focus groups and 4.5% (3/67) were checklists.

CONCLUSIONS

There is no single method of measuring and monitoring safety in prehospital care. Arguably, most safety monitoring systems have evolved, rather than been designed. This leads to safety blind spots in which information is lacking, as well as to redundancy and duplication of effort. It is suggested that the findings from this systematic review, informed by the MMS framework, can provide a structure for critically thinking about how safety is being measured and monitored in prehospital care. This will support the design of a safety surveillance system that provides a comprehensive understanding of what is being done well, where improvements should be made and whether safety interventions have had the desired effect.

Real-Time Monitoring System to Manage Mental Healthcare Emergency Unit

Objectives

Real-time relevant information helps guide the healthcare decision-making process in daily clinical practice as well as the management and optimization of healthcare processes. However, proprietary business intelligence suite solutions supporting the production of decision-making information requires investment that is out of reach of small and medium-sized healthcare facilities or those with limited resources, particularly in developing countries. This paper describes our experience in designing and implementing a real-time healthcare monitoring system solution to manage healthcare emergency units.

Methods

Through the use of free Business Intelligence tools and Python data science language we designed a real-time monitoring system, which was implemented to explore the Electronic Medical Records system of a university mental health emergency unit and render an electronic dashboard to support health professional daily practice.

Results

Three main dashboards were created to monitor patient waiting time, to access the clinical notes summary for the next waiting patient, and to obtain insights into activity during the last 24 hours.

Conclusions

The designed system could serve as a monitoring support model using free and user-friendly data science tools, which are good alternatives to proprietary business intelligence solutions and drastically reduce cost. Still, the key to success in decision-making systems is based on investment in human resources, business intelligence skills training, the organizational aspect of the decision-making process, and data production quality insurance.

Case-mix adjusted postanaesthesia care unit length of stay and business intelligence dashboards for feedback to anaesthetists

BACKGROUND

Despite advances in business intelligence software and evidence that feedback to doctors can improve outcomes, objective feedback regarding patient outcomes for individual anaesthetists is hampered by lack of useful benchmarks. We aimed to address this issue by producing case-mix and risk-adjusted postanaesthesia care unit (PACU) length of stay (LOS) benchmarks for integration into modern reporting tools.

METHODS

We extended existing hospital information systems to calculate predicted PACU LOS using a neural network trained on patient age, surgery duration, sex, operating specialty, urgency, weekday, and insurance status (n=100 511). We then calculated the difference between observed mean and predicted PACU LOS for individual doctors, and compared the results with and without case-mix adjustment. We report practical implications of using visual analytics dashboards displaying the difference between observed and predicted PACU LOS to provide feedback to anaesthetic doctors.

RESULTS

The neural network accounted for over half of observed variation in individual doctors' mean PACU LOS (mean predicted and mean actual LOS Spearman's r²=0.57). Account for case-mix reduced apparent spread, with 80% of individual doctors falling in a band of 4.3 min after case-mix adjusting, compared with a range of 24 min without adjustment. Case-mix adjusting also identified different individual doctors as outliers (Weighted Cohen's kappa [κ]=0.27). Finally, we demonstrated that we were able to integrate the adjusted metrics into routine reporting tools.

CONCLUSION

With caution, case-mix adjustment of anaesthetic outcome measures such as PACU LOS potentially provides a useful continuous quality improvement tool. Unadjusted outcome measures are imprecise at best and misleading at worst.

Creation of a chemotherapy-induced nausea/vomiting dashboard to improve outcomes for pediatric cancer patients

PURPOSE

Two of the most common acute side effects of chemotherapy are nausea and vomiting. Nausea and vomiting impact quality of life, nutritional status, and ability to tolerate further chemotherapy. Parents of pediatric oncology patients rank nausea as one of the most bothersome treatment-related symptoms.

METHODS

Utilizing Quality Improvement methodology, we developed a dashboard interface to facilitate extraction of data from the electronic medical record (EMR), which is presented via a visual display that summarizes the type of chemotherapy and antiemetic medications, use of as needed medications, and number of episodes of emesis.

RESULTS

This dashboard interface allows for rapid and efficient identification of patients whose antiemetic regimen is mismatched for the emetogenicity of ordered chemotherapy, thus providing a timely opportunity to modify the antiemetic regimen based on published guidelines before administration of chemotherapy drugs. It also allows measurement of the effectiveness of the antiemetic regimen in terms of the number of break through emesis and the need for as needed medications.

CONCLUSIONS

A novel CINV dashboard was created, which visually conveys complex information about antiemetics, chemotherapy emetogenicity, as needed medications, and breakthrough vomiting for inpatient pediatric oncology patients.

Requirements for a quality dashboard: Lessons from National Clinical Audits

Healthcare organizations worldwide use quality dashboards to provide feedback to clinical teams and managers, in order to monitor care quality and stimulate quality improvement. However, there is limited evidence regarding the impact of quality dashboards and audit and feedback research focuses on feedback to individual clinicians, rather than to clinical and managerial teams. Consequently, we know little about what features a quality dashboard needs in order to provide benefit. We conducted 54 interviews across five healthcare organizations in the National Health Service in England, interviewing personnel at different levels of the organization, to understand how national (UK) clinical audit data are used for quality improvement and factors that support or constrain use of these data. The findings, organized around the themes of choosing performance indicators, assessing performance, identifying causes, communicating from ward to board, and data quality, have implications for the design of quality dashboards, which we have translated into a series of requirements.

How, in what contexts, and why do quality dashboards lead to improvements in care quality in acute hospitals? Protocol for a realist feasibility evaluation

INTRODUCTION

National audits are used to monitor care quality and safety and are anticipated to reduce unexplained variations in quality by stimulating quality improvement (QI). However, variation within and between providers in the extent of engagement with national audits means that the potential for national audit data to inform QI is not being realised. This study will undertake a feasibility evaluation of QualDash, a quality dashboard designed to support clinical teams and managers to explore data from two national audits, the Myocardial Ischaemia National Audit Project (MINAP) and the Paediatric Intensive Care Audit Network (PICANet).

METHODS AND ANALYSIS

Realist evaluation, which involves building, testing and refining theories of how an intervention works, provides an overall framework for this feasibility study. Realist hypotheses that describe how, in what contexts, and why QualDash is expected to provide benefit will be tested across five hospitals. A controlled interrupted time series analysis, using key MINAP and PICANet measures, will provide preliminary evidence of the impact of QualDash, while ethnographic observations and interviews over 12 months will provide initial insight into contexts and mechanisms that lead to those impacts. Feasibility outcomes include the extent to which MINAP and PICANet data are used, data completeness in the audits, and the extent to which participants perceive QualDash to be useful and express the intention to continue using it after the study period.

ETHICS AND DISSEMINATION

The study has been approved by the University of Leeds School of Healthcare Research Ethics Committee. Study results will provide an initial understanding of how, in what contexts, and why quality dashboards lead to improvements in care quality. These will be disseminated to academic audiences, study participants, hospital IT departments and national audits. If the results show a trial is feasible, we will disseminate the QualDash software through a stepped wedge cluster randomised trial.

Walking the tightrope: how rebels “do” quality of care in healthcare organizations

Purpose

The purpose of this paper is to empirically explore and conceptualize how healthcare professionals and managers give shape to the increasing call for compassionate care as an alternative for system-based quality management systems. The research demonstrates how quality rebels craft deviant practices of good care and how they account for them.

Design/methodology/approach

Ethnographic research was conducted in three Dutch hospitals, studying clinical groups that were identified as deviant: a nursing ward for infectious diseases, a mother–child department and a dialysis department. The research includes over 120 h of observation, 41 semi-structured interviews and 2 focus groups.

Findings

The research shows that rebels’ quality practices are an emerging set of collaborative activities to improving healthcare and meeting (individual) patient needs. They conduct “contexting work” to achieve their quality aims by expanding their normative work to outside domains. As rebels deviate from hospital policies, they are sometimes forced to act “under the radar” causing the risk of groupthink and may undermine the aim of public accounting.

Practical implications

The research shows that in order to come to more compassionate forms of care, organizations should allow for more heterogeneity accompanied with ongoing dialogue(s) on what good care yields as this may differ between specific fields or locations.

Originality/value

This is the first study introducing quality rebels as a concept to understanding social deviance in the everyday practices of doing compassionate and good care.

Understanding corporate governance of healthcare quality: a comparative case study of eight Australian public hospitals

Background

Patients are sometimes harmed in the course of receiving hospital care. Existing research has highlighted a positive association between board engagement in healthcare quality activities and healthcare outcomes. However, most research has been undertaken through surveys examining board engagement in a limited number of governance processes. This paper presents evidence of a comprehensive range of processes related to governing healthcare quality undertaken at the corporate governance level. This provides a more detailed picture than previously described of how corporate governance of healthcare quality is enacted by boards and management.

Methods

A comparative case study of eight Australian public hospitals was undertaken. Case studies varying is size and location were selected from two Australian states. Data collection included a review of key governance documentation, semi structured interviews with board members and senior management and an observation of a board quality committee meeting. Thematic analysis was undertaken to identify processes related to key tasks in governing healthcare quality.

Results

Two key tasks in the corporate governance of healthcare quality, evaluating healthcare quality and overseeing quality priorities, were examined. Numerous processes related to these two tasks were found. Case studies, while found to be similar in engagement on previously identified processes, were found to differ in engagement in these additional processes. While generally low levels of engagement in processes of overseeing quality priorities were found, cases differed markedly in their engagement in evaluating healthcare quality processes. Additional processes undertaken at some case studies represent innovative and mature responses to the need for effective corporate governance of healthcare quality. In addition, a group of processes, related to broader governance taskwork, were found to be important in enabling effective corporate governance of healthcare quality.

Conclusion

The work of governing healthcare quality, undertaken at the corporate governance level, is redefined in terms of these more detailed processes. This paper highlights that it is how well these key tasks are undertaken that is important in effective governance. When processes related to key tasks are omitted, the rituals of governance may appear to be satisfied but the responsibility may not be met. Boards and managers need to differentiate between common approaches to governance and practices that enable the fulfilment of governance responsibilities. This study provides practical guidance in outlining processes for effective corporate governance of healthcare quality and highlights areas for further examination.