Benchmarking in laboratory medicine: are we measuring the right outcomes?

Benchmarking medical laboratory performance: survey validation and results for Europe, Middle East, and Africa

OBJECTIVES

Medical laboratory performance is a relative concept, as are quality and safety in medicine. Therefore, repetitive benchmarking appears to be essential for sustainable improvement in health care. The general idea in this approach is to establish a reference level, upon which improvement may be strived for and quantified. While the laboratory community traditionally is highly aware of the need for laboratory performance and public scrutiny is more intense than ever due to the SARS-CoV-2 pandemic, few initiatives span the globe. The aim of this study was to establish a good practice approach towards benchmarking on a high abstraction level for three key dimensions of medical laboratory performance, generate a tentative snapshot of the current state of the art in the region of Europe, Middle East, and Africa (EMEA), and thus set the stage for global follow-up studies.

METHODS

The questionnaire used and previously published in this initiative consisted of 50 items, roughly half relating to laboratory operations in general with the other half addressing more specific topics. An international sample of laboratories from EMEA was approached to elicit high fidelity responses with the help of trained professionals. Individual item results were analyzed using standard descriptive statistics. Dimensional reduction of specific items was performed using exploratory factor analysis and assessed with confirmatory factor analysis, resulting in individual laboratory scores for the three subscales of "Operational performance", "Integrated clinical care performance", and "Financial sustainability".

RESULTS

Altogether, 773 laboratories participated in the survey, of which 484 were government hospital laboratories, 129 private hospital laboratories, 146 commercial laboratories, and 14 were other types of laboratories (e.g. research laboratories). Respondents indicated the need for digitalization (e.g. use of IT for order management, auto-validation), automation (e.g. pre-analytics, automated sample transportation), and establishment of formal quality management systems (e.g. ISO 15189, ISO 9001) as well as sustainably embedding them in the fabric of laboratory operations. Considerable room for growth also exists for services provided to physicians, such as "Diagnostic pathways guidance", "Proactive consultation on complex cases", and "Real time decision support" which were provided by less than two thirds of laboratories. Concordantly, the most important kind of turn-around time (TAT) for clinicians, sample-to-result TAT, was monitored by only 40% of respondents.

CONCLUSIONS

Altogether, the need for stronger integration of laboratories into the clinical care process became apparent and should be a main trajectory of future laboratory management. Factor analysis confirmed the theoretical constructs of the questionnaire design phase, resulting in a reasonably valid tool for further benchmarking activities on the three aimed-for key dimensions.

Benchmarking diagnostic laboratory performance: Survey results for Germany, Austria, and Switzerland

BACKGROUND AND AIMS

The need for patient safety through consistent diagnostic performance has increasingly been brought into focus during the last two decades. Around the globe operational efficiency of diagnostic laboratories plays a key role in satisfying this need, which has impressively been shown during the recent months of the SARS-CoV‑2 pandemic. On a global level, however, there has been a lack to collate and benchmark data for diagnostic laboratories. The goals of this study were to design and pilot a questionnaire addressing key aspects of diagnostic laboratory management.

METHODS

The questionnaire was designed using an iterative process and taking into consideration information that could be extracted from the literature, author experience and feedback from informal focus groups of laboratory professionals. The resulting tool consisted of 50 items, either relating to general information or more specifically addressing the topics of "operational performance", "integrated clinical care performance", and "financial sustainability". A limited number of laboratories were surveyed to be able to further improve the newly developed tool and motivate the global laboratory community to participate in further benchmarking activity.

RESULTS AND CONCLUSION

Altogether, 65 laboratories participated in the survey, 42 were hospital laboratories and 23 were commercial laboratories. Potential for further improvement and standardization became apparent across the board, e.g. use of IT for order management, auto-validation, or turn-around time (TAT) monitoring. Notably, a gap was identified regarding services provided to physicians, in particular "reflexive test suggestions", "proactive consultation on complex cases", and "diagnostic pathways guidance", which were only provided by about two thirds of laboratories. Concordantly, within-laboratory TAT (Lab TAT) was monitored by about 80% of respondents, while sample-to-result TAT, which is arguably the TAT most relevant to clinicians, was only monitored by 32% of respondents. Altogether, the need for stronger integration of the laboratory into the clinical care process became apparent and should be a main trajectory of future laboratory management.

Implementation of medical tests in a Value-Based healthcare environment: A framework for delivering value

Significant variation in the utilisation of medical tests is known to have an adverse impact on health outcomes and analysis of this variation is an important tool for quality assurance in healthcare. The introduction of a new medical test into a care pathway requires two distinct processes, termed adoption and implementation. One cause of the unwarranted variation in the use of medical tests is poor adoption and implementation. Adoption is the decision to acquire a technology and make it available to the users and is supported with evidence of clinical and cost effectiveness. Implementation is delivering the benefits promised in the business case, based on evidence of the impact of a test on each stakeholder involved in delivering the care pathway. The business case will have identified the benefits delivered to all stakeholders, as set out in a value proposition, and according to the quality domains typically addressed in quality improvement, namely clinical, process and structure (resource utilisation) outcomes. The outcome measures extend beyond those of clinical and cost effectiveness required for adoption. We describe an implementation framework which is designed to document the changes to the care pathway, the resource inputs and the expected outcomes with associated quality metrics.

DB4US: A Decision Support System for Laboratory Information Management

Background

Until recently, laboratory automation has focused primarily on improving hardware. Future advances are concentrated on intelligent software since laboratories performing clinical diagnostic testing require improved information systems to address their data processing needs. In this paper, we propose DB4US, an application that automates information related to laboratory quality indicators information. Currently, there is a lack of ready-to-use management quality measures. This application addresses this deficiency through the extraction, consolidation, statistical analysis, and visualization of data related to the use of demographics, reagents, and turn-around times. The design and implementation issues, as well as the technologies used for the implementation of this system, are discussed in this paper.

Objective

To develop a general methodology that integrates the computation of ready-to-use management quality measures and a dashboard to easily analyze the overall performance of a laboratory, as well as automatically detect anomalies or errors. The novelty of our approach lies in the application of integrated web-based dashboards as an information management system in hospital laboratories.

Methods

We propose a new methodology for laboratory information management based on the extraction, consolidation, statistical analysis, and visualization of data related to demographics, reagents, and turn-around times, offering a dashboard-like user web interface to the laboratory manager. The methodology comprises a unified data warehouse that stores and consolidates multidimensional data from different data sources. The methodology is illustrated through the implementation and validation of DB4US, a novel web application based on this methodology that constructs an interface to obtain ready-to-use indicators, and offers the possibility to drill down from high-level metrics to more detailed summaries. The offered indicators are calculated beforehand so that they are ready to use when the user needs them. The design is based on a set of different parallel processes to precalculate indicators. The application displays information related to tests, requests, samples, and turn-around times. The dashboard is designed to show the set of indicators on a single screen.

Results

DB4US was deployed for the first time in the Hospital Costa del Sol in 2008. In our evaluation we show the positive impact of this methodology for laboratory professionals, since the use of our application has reduced the time needed for the elaboration of the different statistical indicators and has also provided information that has been used to optimize the usage of laboratory resources by the discovery of anomalies in the indicators. DB4US users benefit from Internet-based communication of results, since this information is available from any computer without having to install any additional software.

Conclusions

The proposed methodology and the accompanying web application, DB4US, automates the processing of information related to laboratory quality indicators and offers a novel approach for managing laboratory-related information, benefiting from an Internet-based communication mechanism. The application of this methodology has been shown to improve the usage of time, as well as other laboratory resources.

Applicability and potential benefits of benchmarking in Brazilian clinical laboratory services

Purpose

The purpose of this paper is to test the applicability and benefits of benchmarking as a tool for quality analysis in Brazilian laboratory medical services.

Design/methodology/approach

A primary observational study is performed in eight hospital laboratories by tracking the receipt, analysis and return to participants of monitoring reports relating to several quality indicators for the years 2005 and 2006. Whenever possible, the paper applies 6σ criteria as an independent assessment of process quality.

Findings

Data obtained for the eight laboratories showed a monthly average (±SD) of 178,579 (±153,670) tests performed per laboratory, with 40,256 (±44,858) requisitions and 4.77 (±1.33) tests per requisition. Overall, productivity was 7.35 (±2.46) tests per man‐hour of work (MHW), increasing to 15.36 (±6.00) when considering only the analytical sector staff. An average of 1.63 (±1.14) lost hours per hundred MHW were reported (level 3.6σ), with 3.86 (±5.10) accidents at work reported (AWR) per hundred thousand MHW (level 5.5σ) and 4.22 (±2.61) redraws per thousand requisitions attended (level 4.1σ). The turn‐around‐times were 2.25 (±0.98), 3.29 (±2.12) and 8.54 (±3.25) hours for glucose level, haemogram and human immunodeficiency virus serology, respectively.

Practical implications

Benchmarking proved to be a useful and feasible tool for quality management in Brazilian clinical laboratories, particularly when associated with independent tools for evaluating the quality of laboratorial processes.

Originality/value

This is the first Brazilian study reporting that benchmarking provides useful information on the performance of different clinical laboratory processes and, therefore, could become an important tool for laboratory management.

Benchmarking: A Process for Learning or Simply Raising the Bar?

Benchmarking has been a credible approach to quality improvement and program evaluation for more than 30 years. However, the term has been used and abused so much since the 1970s that now an author might be describing anything from the establishment of performance indicators, to competitor ranking, or gap analysis, through to a continuous quality improvement process. Equally confusing is the plethora of different benchmarking models published. A literature review was conducted in which 65 publications from 1998-2008 were analysed and synthesised in order to untangle the confusion surrounding the contemporary theory and practice of benchmarking. This article clarifies the terminology and concepts of benchmarking, attempts a concise practical definition, examines various approaches in the current theory and application of benchmarking, and explicates the features most consistent with the goal of continuous quality improvement and organisational learning.