Framework for evaluating performance and quality improvement in hospitals

The development of the concept of return-on-investment from large-scale quality improvement programmes in healthcare: an integrative systematic literature review

BACKGROUND

Return on Investment (ROI) is increasingly being used to evaluate financial benefits from healthcare Quality Improvement (QI). ROI is traditionally used to evaluate investment performance in the commercial field. Little is known about ROI in healthcare. The aim of this systematic review was to analyse and develop ROI as a concept and develop a ROI conceptual framework for large-scale healthcare QI programmes.

METHODS

We searched Medline, Embase, Global health, PsycInfo, EconLit, NHS EED, Web of Science, Google Scholar using ROI or returns-on-investment concepts (e.g., cost-benefit, cost-effectiveness, value). We combined this terms with healthcare and QI. Included articles discussed at least three organisational QI benefits, including financial or patient benefits. We synthesised the different ways in which ROI or return-on-investment concepts were used and discussed by the QI literature; first the economically focused, then the non-economically focused QI literature. We then integrated these literatures to summarise their combined views.

RESULTS

We retrieved 10 428 articles. One hundred and two (102) articles were selected for full text screening. Of these 34 were excluded and 68 included. The included articles were QI economic, effectiveness, process, and impact evaluations as well as reports and conceptual literature. Fifteen of 68 articles were directly focused on QI programme economic outcomes. Of these, only four focused on ROI. ROI related concepts in this group included cost-effectiveness, cost-benefit, ROI, cost-saving, cost-reduction, and cost-avoidance. The remaining articles mainly mentioned efficiency, productivity, value, or benefits. Financial outcomes were not the main goal of QI programmes. We found that the ROI concept in healthcare QI aligned with the concepts of value and benefit, both monetary and non-monetary.

CONCLUSION

Our analysis of the reviewed literature indicates that ROI in QI is conceptualised as value or benefit as demonstrated through a combination of significant outcomes for one or more stakeholders in healthcare organisations. As such, organisations at different developmental stages can deduce benefits that are relevant and legitimate as per their contextual needs.

TRIAL REGISTRATION

Review registration: PROSPERO; CRD42021236948.

Identifying and understanding benefits associated with return-on-investment from large-scale healthcare Quality Improvement programmes: an integrative systematic literature review

BACKGROUND

We previously developed a Quality Improvement (QI) Return-on-Investment (ROI) conceptual framework for large-scale healthcare QI programmes. We defined ROI as any monetary or non-monetary value or benefit derived from QI. We called the framework the QI-ROI conceptual framework. The current study describes the different categories of benefits covered by this framework and explores the relationships between these benefits.

METHODS

We searched Medline, Embase, Global health, PsycInfo, EconLit, NHS EED, Web of Science, Google Scholar, organisational journals, and citations, using ROI or returns-on-investment concepts (e.g., cost-benefit, cost-effectiveness, value) combined with healthcare and QI. Our analysis was informed by Complexity Theory in view of the complexity of large QI programmes. We used Framework analysis to analyse the data using a preliminary ROI conceptual framework that was based on organisational obligations towards its stakeholders. Included articles discussed at least three organisational benefits towards these obligations, with at least one financial or patient benefit. We synthesized the different QI benefits discussed.

RESULTS

We retrieved 10 428 articles. One hundred and two (102) articles were selected for full text screening. Of these 34 were excluded and 68 included. Included articles were QI economic, effectiveness, process, and impact evaluations as well as conceptual literature. Based on these literatures, we reviewed and updated our QI-ROI conceptual framework from our first study. Our QI-ROI conceptual framework consists of four categories: 1) organisational performance, 2) organisational development, 3) external outcomes, and 4) unintended outcomes (positive and negative). We found that QI benefits are interlinked, and that ROI in large-scale QI is not merely an end-outcome; there are earlier benefits that matter to organisations that contribute to overall ROI. Organisations also found positive aspects of negative unintended consequences, such as learning from failed QI.

DISCUSSION AND CONCLUSION

Our analysis indicated that the QI-ROI conceptual framework is made-up of multi-faceted and interconnected benefits from large-scale QI programmes. One or more of these may be desirable depending on each organisation's goals and objectives, as well as stage of development. As such, it is possible for organisations to deduce incremental benefits or returns-on-investments throughout a programme lifecycle that are relevant and legitimate.

The dynamic use of a balanced scorecard in an Italian public hospital

Purpose

This paper aims to analyse the dynamic use of the balanced scorecard (BSC) in an Italian public hospital.

Design/Methodology/Approach

A longitudinal case study was conducted at an Italian public teaching hospital over a period of 5 years. The emergence of dynamic use of BSC was traced over a different combination of social, political, economic and organizational realities. A deeper understanding of these realities requires the adoption of a holistic approach to BSC use. Henri's types of system use (i.e., monitoring, attention focussing, strategic decision‐making and legitimizing) frame this approach in a more concrete manner.

Findings

This study adds to the debate on whether BSC is used for aspects other than monitoring in public contexts. The case study offers the first example of a legitimizing use of the system and a first longitudinal case study that traces a dynamic use of BSC: the use evolves from monitoring and attention focussing to monitoring and legitimization. Norms, political parties and top managers play a determining role in this process.

Originality/Value

Through a longitudinal approach, this study presents how BSC can be a dynamic tool steered by legitimacy pressures. The longitudinal study explores how social, political, economic and organizational context shape the implementation and the revision of BSC affecting the use of the tool by top managers. The browse of this dynamism is supported by Henri's type of use along with an in‐depth analysis of the BSC literature evolution in terms of its ‘static, dynamic and expected’ use.

Performance management systems, such as balanced scorecard (BSC), should be understood in a holistic view, namely taking into account the interdependency between the design, the implementation, and the use of the systems and the interdependency between different systems operating at the same time in the same organization.

How managers use BSC in hospitals changes over time.

At an organizational level, a BSC could provide justification of actions and decisions as well as monitoring results, supporting strategic decision‐making processes and focussing attention on priorities.

BSC is a pivotal lever that can be governed by hospital managers as a strategic management tool for facing legitimacy pressures.

Performance evaluation of medical service for breast cancer patients based on diagnosis related groups

Background

To evaluate the performance of medical service for patients with breast cancer in Henan Province, China, using diagnosis related groups (DRGs) indicators and to provide data to inform practices and policies for the prevention and control of breast cancer.

Methods

The data were collected from the front pages of medical records (FPMR) of all hospitals above class II that admitted breast cancer patients in Henan Province between 2016 and 2019. Breast cancer patients were the subjects in our study. China DRGs (CN-DRGs) was used as a risk adjustment tool. Three indicators, including the case mix index (CMI), number of DRGs, and total weight, were used to evaluate the range of available services for patients with breast cancer, while indicators including the charge efficiency index (CEI), time efficiency index (TEI) and inpatient mortality of low-risk group cases (IMLRG) were used to evaluate medical service efficiency and medical safety.

Results

Between 2016 and 2019, there were 103,760 patients with breast cancer. The total weight increased over the study period at an average annual rate of 21.71%. The TEI decreased over the study period by 15.60%. The CEI exhibited an increasing trend, but the average annual rate of increase was small (2.94%). The IMLRP was 0.02, 0, 0 and 0.01% in 2016, 2017, 2018 and 2019, respectively.

Conclusion

The performance of medical service improved between 2016 and 2019 for breast cancer patients discharged from study hospitals in Henan Province. The main area of improvement was in the range of available services, but medical institutions must still make efforts to improve the efficiency of medical services and ensure medical safety. DRGs is an effective evaluation tool.

Performance evaluation: a comparative study between public and private sectors

Purpose

This study analysed international publications related to the performance evaluation (PE) theme to compare the public and private sectors with a focus on the metrics and a reflection on the consideration of each sector's characteristics.

Design/methodology/approach

In total, two bibliographic portfolios (BPs) were selected with the aid of the Knowledge Development Process-Constructivist (ProKnow-C) instrument, and the literature was systematically and critically reviewed to construct a theme literature map; the metrics analysed the performance evaluation systems (PESs) used in the studies and identified each sector's characteristics and PES behaviour based on these characteristics.

Findings

The two sectors stand out because of the low incidence of articles with cardinal scales, which enable the measurement and identification of the consequences of performance that are below or above the target. The behaviour of the sector characteristics was unusual, especially regarding the user and the organisational structure; it was found that the presence or absence of critical factors, such as communication, may prove to be a differential of success or failure. An absence of the organisation's particularities when designing and using a PES was noted.

Originality/value

The contributions relate to (1) identification of the public and private sector characteristics based on the selected literature, which helps in understanding the critical success and failure factors of an organisation's PES, especially with regard to metrics adequacy for each context; and (2) verification of the paths taken by the literature in both sectors.

A multicriteria model on calculating the Sustainable Business Excellence Index of a firm with fuzzy AHP and TOPSIS

Purpose

– The purpose of this paper is to provide an integrated methodology for benchmarking the sustainability of organizations. The fuzzy analytical hierarchy process (FAHP) and technique for order of preference by similarity to ideal solution (TOPSIS) methods have been used for this purpose. The FAHP is used to determine the weights of the criteria by decision makers, and the rankings of the alternatives are determined by TOPSIS. The proposed instrument is used to calculate the Sustainable Business Excellence Index (SBEI) and its potential impact on the formulation of firm strategy. To demonstrate the applicability of the model, illustrative examples are presented.

Design/methodology/approach

– After a careful literature review, a sustainable business excellence framework is created and a fuzzy system is developed to assess firms’ sustainability. Finally, the SBEI is computed.

Findings

– The results indicate that the suggested fuzzy approach is feasible for benchmarking the sustainability of organizations. It allows the decision makers to express their opinion regarding the importance of criteria and evaluate each alternative and then have this input coordinated in a quantitative fashion.

Research limitations/implications

– Practitioners and consultants can use the instrument for conducting quality management benchmarking within and across organizations. Researchers can use the instrument in future studies for further theory development in this area.

Originality/value

– As far as the authors are aware, no previous study research has assessed the SBEI of an organization with fuzzy sets. As such, it responds to a number of contemporary challenges in the business excellence theory, most importantly the broad need to identify agile organizations.

Measuring patient-perceived hospital service quality: a conceptual framework

Purpose

Although measuring healthcare service quality is not a new phenomenon, the instruments used to measure are timeworn. With the shift in focus to patient centric processes in hospitals and recognising healthcare to be different compared to other services, service quality measurement needs to be tuned specifically to healthcare. The study’s purpose is to design a conceptual framework for measuring patient perceived hospital service quality, based on existing service quality literature

Design/methodology/approach

Using hospital service quality theories, expanding existing healthcare service models and literature, a conceptual framework is proposed to measure hospital service quality. The article outlines inpatient perceived service quality dimensions

Findings

An instrument for measuring hospital service quality dimensions is developed and compared with other service quality measuring instruments. The latest dimensions are in line with previous studies, but a relationship dimension is added.

Practical implications

The framework empowers managers to assess healthcare quality in corporate, public and teaching hospitals.

Originality/value

The article helps academics and practitioners to assess hospital service quality from a patient perspective.

Performance Measurement and Performance Indicators

Performance measurement systems (PMSs) are developed and implemented to support the accomplishment of objectives of an organization or organizational initiative. PMSs are structured to enable monitoring of performance. This monitoring is accomplished through the identification of areas of concern and success, with the ultimate goals of developing organizational capabilities and enhancing organizational learning. This article reviews the literature on performance measurement and its implementation, and proposes a model to guide the development and implementation of PMSs. The article concludes with a consideration of the broader implications of PMSs and provides recommendations for future research in PMSs with regard to human resource development.

Benchmarking Six Sigma implementation in services companies operating in an emerging economy

Purpose

– The paper reports case studies conducted in services organizations that apply Six Sigma in an emerging economy. Aspects of human resources and full-time specialist, infra-structure, top management leadership, customer requirements, focus on financial and non-financial results, structured method and strategic project selection are investigated. The paper aims to discuss these issues.

Design/methodology/approach

– Case-based research is employed as methodological approach. Data were mostly gathered through semi-structured interviews with managers involved with Six Sigma implementation. When available, archival data were also used to achieve more evidence towards triangulation. Then, the aggregation of the results was done a posteriori.

Findings

– It was verified that Six Sigma was successfully implemented in those companies, however with some drawbacks. The studied companies adopt Six Sigma in association with other quality management approaches, mainly ISO 9000: 2008 and lean production.

Research limitations/implications

– The study was limited to three service companies. Therefore, the study findings are not subject to generalisation for other similar firms.

Practical implications

– The research shows the importance of Six Sigma implementation in service companies in addition to its contribution to practical application by identifying the relevant aspects of its introduction. It enables practitioners to use the findings on factors that should be taken into consideration when applying Six Sigma.

Originality/value

– It is one of the few studies on Six Sigma analysis in emerging economies. In addition, the empirical results found support in existing theory to some extent.

Obstacles to TQM success in health care systems

PURPOSE

Many healthcare organisations have found it difficult to implement total quality management (TQM) successfully. The aim of this paper is to explore the barriers to TQM successful implementation in the healthcare sector.

DESIGN/METHODOLOGY/APPROACH

This paper reports a literature review exploring the major reasons for the failure of TQM programmes in healthcare organisations.

FINDINGS

TQM implementation and its impact depend heavily on the ability of managers to adopt and adapt its values and concepts in professional healthcare organisations. Unsuccessful TQM efforts in healthcare organisations can be attributed to the strongly departmentalised, bureaucratic and hierarchical structure, professional autonomy, tensions between managers and professionals and the difficulties involved in evaluating healthcare processes and outcomes. Other obstacles to TQM success include lack of consistent managers' and employees' commitment to and involvement in TQM implementation, poor leadership and management, lack of a quality-oriented culture, insufficient training, and inadequate resources. The review was limited to empirical articles written in the English language during the past 30 years (1980-2010).

PRACTICAL IMPLICATIONS

The findings of this article provide policy makers and managers with a practical understanding of the factors that are likely to obstruct TQM implementation in the healthcare sector.

ORIGINALITY/VALUE

Understanding the factors that obstruct TQM implementation would enable managers to develop more effective strategies for implementing TQM successfully in healthcare organisations.

Cooperation and competition: balanced scorecard and hospital privatization

PURPOSE

In 2000 the Skåne Region (a public authority) and a private contractor made a five-year agreement for the provision of both in-patient care and out-patient medical services to about 30,000 inhabitants in the south-east part of the region. The Skåne Region is the main provider of health care to about one million inhabitants in the south of Sweden and is responsible for all health care (private and public), including ten hospitals. This paper seeks to answer the question of how the Skåne Region can control and cooperate with a private contractor, entering into competition with the public health care providers in the region.

DESIGN/METHODOLOGY/APPROACH

This is a longitudinal study conducted between 2001-2006. It is based on 28 taped interviews with employees responsible for the contracting process, participating observations and comprehensive secondary material. The study presents experiences made by the contractor and the public authority on how to work out and follow-up assignments within the health care sector regarding patient interest, public interest and professional medical interest.

FINDINGS

Measurement within the frames of the balanced scorecard (BSC) made it possible to control both volumes and health care quality delivered by the private competing contractor. The political purchaser claims that the Skåne Region has established a cost-effective and successful control system based on trust and measurement.

ORIGINALITY/VALUE

This paper reports on a control system, between public purchaser and a private provider within health care, that focuses on and follow-up not only health care production but also health care quality.

Healthcare quality and moderators of patient satisfaction: testing for causality

PURPOSE

The purpose of this article is to present a comprehensive structural equation based service quality and patient satisfaction model taking into account the patient's condition before and after discharge. The authors aim to test for causality in a sample of patients from United Arab Emirates public hospitals.

DESIGN/METHODOLOGY/APPROACH

Data were collected using questionnaires completed by adults discharged (n = 244) from UAE public hospitals. The proposed model consists of five main constructs. Three represent service quality: quality of care (four variables); process and administration (four variables) and information (four variables). There is also one construct that represents patient's status (two variables--health status before admission and after discharge). Finally, there is one construct that represents patient's satisfaction with care (two variables--general and relative satisfaction). Structural equation modeling and LISREL using maximum likelihood estimation was used to test hypothesized model(s)/parameters(s) derived deductively from the literature.

FINDINGS

The structural equation modeling representation provides a comprehensive picture that allows healthcare constructs and patient satisfaction causality to be tested. The goodness-of-fit statistics supported the healthcare quality-patient status-satisfaction model.

ORIGINALITY/VALUE

The model has been found to capture attributes that characterize healthcare quality in a developing country and could represent other modern healthcare systems. Also, it can be used to evaluate other healthcare practices from patients' viewpoints. The study highlights the importance of healthcare quality as patient satisfaction predictors by capturing other effects such as patient status.

Testing not-so-obvious models of healthcare quality

PURPOSE

The purpose of this paper is to show that, although there has been some research to identify the dimensions on which healthcare quality and in-patient satisfaction should be measured, the confirmation of constructs and indicators that constitute an overall care quality and satisfaction remains unclear. The objective is to present several models of service quality and satisfaction in healthcare for discharged patients; and to test those models in a sample of discharged patients in public hospitals in the United Arab Emirates.

DESIGN/METHODOLOGY/APPROACH

A detailed in-patient survey (using interviews) was used. Data were collected with questionnaires from adult discharges (n = 244) in public hospitals in the UAE. Several structures are proposed and tested. Confirmatory Factor Analysis (CFA) and LISREL SIMPLIS using maximum likelihood estimation were used to estimate and test the parameters of the hypothesized models derived deductively from the previous literature.

FINDINGS

Several models (with one, two, three and four constructs) with different structures were tested using CFA. The final recommended model is based on three constructs--quality of care, process and administration, and information. The goodness-of-fit statistics supported the basic solution of the healthcare quality-satisfaction model.

ORIGINALITY/VALUE

The model has been found to capture attributes that characterize healthcare quality in a developing country such as the UAE and could represent other modern healthcare systems. It can be used as a basis for evaluation in healthcare practices from discharges (in-patients) point of view. The study highlights the importance of patients' satisfaction with care as predictors of quality of care. The results also confirm the construct validity of the previously discussed healthcare quality scales.

Gestão Hospitalar: identificação das práticas de aprendizagem existentes em hospitais

A profissionalização da gestão em hospitais, a partir de diferentes práticas da administração, tem sido amplamente discutida, à luz de conceitos como gestão baseada em evidência, gestão do conhecimento e aprendizagem organizacional. Entretanto, são poucos os artigos identificando e discutindo essas práticas nas organizações de saúde. Este é o objetivo do presente artigo. Buscou-se realizar uma análise histórica da gestão em saúde no Brasil, considerando uma mudança a partir do processo de descentralização. Foi realizado o mapeamento das práticas de gestão enfatizando-se as metodologias de aprendizagem existentes no setor, a partir de pesquisas documentais. Pôde-se perceber uma mudança de modelo mental na área, passando de um enfoque linear para um enfoque sistêmico. Da mesma forma, identificou-se o processo de construção do conhecimento nessas organizações, especialmente a partir da prática médica. Constatou-se ainda que o processo de aprendizagem enfatiza a aprendizagem individual e não a aprendizagem organizacional.

Reviewing the application of the balanced scorecard with implications for low-income health settings

High-income countries (HICs) are increasingly making use of the balanced scorecard (BSC) in healthcare. Evidence about BSC usage in low-income countries (LICs) is deficient. This study assessed feasibility of BSC use in LICs. Systematic review of electronic databases shows that the BSC improved patient, staff, clinical, and financial outcomes in HICs. To translate the experience of BSC use in HICs to their use in LICs, the applicability parameters of the National Committee for Quality Assurance were applied. Despite contextual challenges, pilot testing of BSC use can be undertaken in selected LICs. Committed leadership, cultural readiness, quality information systems, viable strategic plans, and optimum resources are required.