Reviewing the application of the balanced scorecard with implications for low-income health settings

Assessing Patient Experience and Attitude: BSC-PATIENT Development, Translation, and Psychometric Evaluation-A Cross-Sectional Study

Health care organizations (HCO) did not consider engaging patients in balanced scorecard (BSC) implementations to evaluate their performance. This paper aims to develop an instrument to engage patients in assessing BSC perspectives (BSC-PATIENT) and customize it for Palestinian hospitals. Two panels of experts participated in the item generation of BSC-PATIENT. Translation was performed based on guidelines. Pretesting was performed for 30 patients at one hospital. Then, 1000 patients were recruited at 14 hospitals between January and October 2021. Construct validity was tested through exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). Additionally, the composite reliability (CR), interitem correlation (IIC), and corrected item total correlation (CITC) were assessed to find redundant and low correlated items. As a result, the scales had a highly adequate model fit in the EFA and CFA. The final best fit model in CFA comprised ten constructs with 36 items. In conclusion, BSC-PATIENT is the first self-administered questionnaire specifically developed to engage patients in BSC and will allow future researchers to evaluate the impact of patient experience on attitudes toward BSC perspectives, as well as to compare the differences based on patient and hospital characteristics.

A systematic review: the dimensions to evaluate health care performance and an implication during the pandemic

BACKGROUND

The balanced scorecard (BSC) has been implemented to evaluate the performance of health care organizations (HCOs). BSC proved to be effective in improving financial performance and patient satisfaction.

AIM

This systematic review aims to identify all the perspectives, dimensions, and KPIs that are vital and most frequently used by health care managers in BSC implementations.

METHODS

This systematic review adheres to PRISMA guidelines. The PubMed, Embase, Cochrane, and Google Scholar databases and Google search engine were inspected to find all implementations of BSC at HCO. The risk of bias was assessed using the nonrandomized intervention studies (ROBINS-I) tool to evaluate the quality of observational and quasi-experimental studies and the Cochrane (RoB 2) tool for randomized controlled trials (RCTs).

RESULTS

There were 33 eligible studies, of which we identified 36 BSC implementations. The categorization and regrouping of the 797 KPIs resulted in 45 subdimensions. The reassembly of these subdimensions resulted in 13 major dimensions: financial, efficiency and effectiveness, availability and quality of supplies and services, managerial tasks, health care workers' (HCWs) scientific development error-free and safety, time, HCW-centeredness, patient-centeredness, technology, and information systems, community care and reputation, HCO building, and communication. On the other hand, this review detected that BSC design modification to include external and managerial perspectives was necessary for many BSC implementations.

CONCLUSION

This review solves the KPI categorization dilemma. It also guides researchers and health care managers in choosing dimensions for future BSC implementations and performance evaluations in general. Consequently, dimension uniformity will improve the data sharing and comparability among studies. Additionally, despite the pandemic negatively influencing many dimensions, the researchers observed a lack of comprehensive HCO performance evaluations. In the same vein, although some resulting dimensions were assessed separately during the pandemic, other dimensions still lack investigation. Last, BSC dimensions may play an essential role in tackling the COVID-19 pandemic. However, further research is required to investigate the BSC implementation effect in mitigating the pandemic consequences on HCO.

The deployment of balanced scorecard in health care organizations: is it beneficial? A systematic review

BACKGROUND

Balanced Scorecard (BSC) has been implemented for three decades to evaluate and improve the performance of organizations. To the best of the researchers' knowledge, no previous systematic review has performed a comprehensive and rigorous methodological approach to figure out the impact of BSC implementation in Health Care Organizations (HCO).

AIMS

The current work was intended to assess the impact of implementing the BSC on Health Care Workers' (HCW) satisfaction, patient satisfaction, and financial performance.

METHODS

The authors prepared the present systematic review according to PRISMA guidelines. Further, the authors customized the search strategy for PubMed, Embase, Cochrane, Google Scholar databases, and Google's search engine. The obtained studies were screened to isolate those measuring scores related to HCW satisfaction, patient satisfaction, and financial performance. The Risk of Bias (RoB) in the non-Randomized Intervention Studies (ROBINS-I) tool was used to assess the quality of observational and quasi-experimental studies. On the other hand, for the Randomized Controlled Trials (RCTs), the Cochrane (RoB 2) tool was used.

RESULTS

Out of 4031 studies, the researchers included 20 studies that measured the impact of BSC on one or more of the three entities (HCW satisfaction, patient satisfaction, and financial performance). Throughout these 20 studies, it was found that 17 studies measured the impact of the BSC on patient satisfaction, seven studies measured the impact on HCW satisfaction, and 12 studies measured the impact on financial performance.

CONCLUSION

This systematic review provides managers and policymakers with evidence to support utilizing BSC in the health care sector. BSC implementation demonstrated positive outcomes for patient satisfaction and the financial performance of HCOs. However, only a mild impact was demonstrated for effects related to HCW satisfaction. However, it is worth noting that many of the studies reflected a high RoB, which may have affected the impacts on the three primary outcomes measured. As such, this systematic review reflects the necessity for further focus on this area in the future. Moreover, future research is encouraged to measure the real and current impact of implementing BSC in HCO during the pandemic since we did not find any.

Scoping review of balanced scorecards for use in healthcare settings: development and implementation

OBJECTIVE

Balanced scorecards (BSCs) were developed in the early 1990s in corporate settings as a strategic performance management tool that emphasised measurement from multiple perspectives. Since their introduction, BSCs have been adapted for a variety of industries, including to healthcare settings. The aim of this scoping review was to describe the application of BSCs in healthcare.

METHODS

Medline, Embase and CINHAL databases were searched using keywords and medical subject headings for 'balanced scorecard' and related terms from 1992 to 17/04/2020. Title and abstract screening and full text review were conducted in duplicate by two reviewers. Studies describing the development and/or implementation of a BSC in a healthcare setting were included. Data were abstracted using pilot-tested forms and reviewed for key themes and findings.

RESULTS

8129 records were identified and 841 underwent a full text review. 87 articles were included. Over 26 countries were represented and the majority of BSCs were applied at a local level (54%) in hospital settings (41%). While almost all discussed Kaplan and Norton's original BSC (97%), only 69% described alignment with a strategic plan. Patients/family members were rarely involved in development teams (3%) which typically were comprised of senior healthcare leaders/administrators. Only 21% of BSCs included perspectives using identical formatting to the original BSC description. Lessons learnt during development addressed three main themes: scorecard design, stakeholder engagement and feasibility.

CONCLUSIONS

BSC frameworks have been used in various healthcare settings but frequently undergo adaptation from the original description in order to suit a specific healthcare context. Future BSCs should aim to include patients/families to promote patient-centred healthcare systems. Considering the heterogeneity evident in development approaches, methodological guidance in this area is warranted.

Developing citizen report cards for primary health care in low and middle-income countries: Results from cognitive interviews in rural Tajikistan

Citizen report cards on health care providers have been identified as a potential means to increase citizen engagement, provider accountability and health systems performance. Research in high-income settings indicates that the wording, presentation and display of performance information are critical to achieve these goals. However, there are limited insights on developing effective report card designs for middle- and low-income settings. We conducted cognitive interviews to assess consumers’ understanding, interpretation of and preferences for displaying information for a health care report card in rural Tajikistan. We recruited a convenience sample of 40 citizens (20 women and 20 men aged 18–45) from rural areas of two provinces of Tajikistan (Soghd and Khatlon oblasts). The interview protocol was adapted from the model of cognitive interviews used in social science research to improve survey questionnaires. We used multivariate regression to assess understanding and interpretation of the report card; chi2 tests to assess differences in preferences for displaying information; and tests of proportions to assess the preferred comparison group. Respondents understood the main idea of the report card and are not confused by the indicators or display. However, many respondents had difficulties making comparisons, and when asked to identify worst-performing services. Respondents preferred detailed rankings using school grades, comparisons of their local clinic with the regional or national average performance, and the use of color in the report card. We found some heterogeneity across the two provinces. Overall, our findings are promising regarding the citizens’ comprehension of health care report cards in rural Tajikistan, while underscoring the challenges of effectively providing health care performance information to communities. Cognitive interviews and iterative testing can support an effective implementation of reporting initiatives.

Developing an evaluation framework for clinical redesign programs: lessons learnt

Purpose The purpose of this paper is to present lessons learnt through the development of an evaluation framework for a clinical redesign programme - the aim of which was to improve the patient journey through improved discharge practices within an Australian public hospital. Design/methodology/approach The development of the evaluation framework involved three stages - namely, the analysis of secondary data relating to the discharge planning pathway; the analysis of primary data including field-notes and interview transcripts on hospital processes; and the triangulation of these data sets to devise the framework. The evaluation framework ensured that resource use, process management, patient satisfaction, and staff well-being and productivity were each connected with measures, targets, and the aim of clinical redesign programme. Findings The application of business process management and a balanced scorecard enabled a different way of framing the evaluation, ensuring measurable outcomes were connected to inputs and outputs. Lessons learnt include: first, the importance of mixed-methods research to devise the framework and evaluate the redesigned processes; second, the need for appropriate tools and resources to adequately capture change across the different domains of the redesign programme; and third, the value of developing and applying an evaluative framework progressively. Research limitations/implications The evaluation framework is limited by its retrospective application to a clinical process redesign programme. Originality/value This research supports benchmarking with national and international practices in relation to best practice healthcare redesign processes. Additionally, it provides a theoretical contribution on evaluating health services improvement and redesign initiatives.

Use of Balanced Scorecard Methodology for Performance Measurement of the Health Extension Program in Ethiopia

In 2004, Ethiopia introduced a community-based Health Extension Program to deliver basic and essential health services. We developed a comprehensive performance scoring methodology to assess the performance of the program. A balanced scorecard with six domains and 32 indicators was developed. Data collected from 1,014 service providers, 433 health facilities, and 10,068 community members sampled from 298 villages were used to generate weighted national, regional, and agroecological zone scores for each indicator. The national median indicator scores ranged from 37% to 98% with poor performance in commodity availability, workforce motivation, referral linkage, infection prevention, and quality of care. Indicator scores showed significant difference by region (P < 0.001). Regional performance varied across indicators suggesting that each region had specific areas of strength and deficiency, with Tigray and the Southern Nations, Nationalities and Peoples Region being the best performers while the mainly pastoral regions of Gambela, Afar, and Benishangul-Gumuz were the worst. The findings of this study suggest the need for strategies aimed at improving specific elements of the program and its performance in specific regions to achieve quality and equitable health services.

Configuring balanced scorecards for measuring health system performance: evidence from 5 years' evaluation in Afghanistan

BACKGROUND

In 2004, Afghanistan pioneered a balanced scorecard (BSC) performance system to manage the delivery of primary health care services. This study examines the trends of 29 key performance indicators over a 5-year period between 2004 and 2008.

METHODS AND FINDINGS

Independent evaluations of performance in six domains were conducted annually through 5,500 patient observations and exit interviews and 1,500 provider interviews in >600 facilities selected by stratified random sampling in each province. Generalized estimating equation (GEE) models were used to assess trends in BSC parameters. There was a progressive improvement in the national median scores scaled from 0-100 between 2004 and 2008 in all six domains: patient and community satisfaction of services (65.3-84.5, p<0.0001); provider satisfaction (65.4-79.2, p<0.01); capacity for service provision (47.4-76.4, p<0.0001); quality of services (40.5-67.4, p<0.0001); and overall vision for pro-poor and pro-female health services (52.0-52.6). The financial domain also showed improvement until 2007 (84.4-95.7, p<0.01), after which user fees were eliminated. By 2008, all provinces achieved the upper benchmark of national median set in 2004.

CONCLUSIONS

The BSC has been successfully employed to assess and improve health service capacity and service delivery using performance benchmarking during the 5-year period. However, scorecard reconfigurations are needed to integrate effectiveness and efficiency measures and accommodate changes in health systems policy and strategy architecture to ensure its continued relevance and effectiveness as a comprehensive health system performance measure. The process of BSC design and implementation can serve as a valuable prototype for health policy planners managing performance in similar health care contexts. Please see later in the article for the Editors' Summary.

Design, implementation and scaling up of the balanced scorecard for hospitals in Lebanon: policy coherence and application lessons for low and middle income countries

OBJECTIVES

This paper describes the development and implementation of the first national hospital performance indicators in Lebanon including its institutionalization within existing policy framework and the initiation of independent governance structure for sustainability.

METHODS

Guided by the Ontario Acute Care Balanced Scorecard framework, a step-wise approach was used. Guiding principles were non-punitive reporting, anonymity, voluntary participation, stakeholder involvement, consensus and feasibility. Modified Delphi technique was used, readiness assessment surveys in 52 hospitals were conducted, pilot testing and evaluation were completed in 14 hospitals.

RESULTS

Initial balanced set of 21 indicators were selected. Findings showed wide variations in indicators' measurement in hospitals including formulas and tools. Barriers to measurement included lack of quality culture, physician resistance and resources. A gradual implementation strategy was developed and selected indicators were divided into two levels. Most piloted indicators proved to be valid, feasible and reliable. The initiative was linked to the national hospital accreditation system resulting in a balanced set of 40 indicators. An independent, not-for-profit, arm's-length organization was established.

CONCLUSIONS

This is among the first attempts made in the East Mediterranean Region to adapt the BSC approach and translate the experience of its development to addresses local needs and contextual reality.

Understanding the context of balanced scorecard implementation: a hospital-based case study in Pakistan

Background

As a response to a changing operating environment, healthcare administrators are implementing modern management tools in their organizations. The balanced scorecard (BSC) is considered a viable tool in high-income countries to improve hospital performance. The BSC has not been applied to hospital settings in low-income countries nor has the context for implementation been examined. This study explored contextual perspectives in relation to BSC implementation in a Pakistani hospital.

Methods

Four clinical units of this hospital were involved in the BSC implementation based on their willingness to participate. Implementation included sensitization of units towards the BSC, developing specialty specific BSCs and reporting of performance based on the BSC during administrative meetings. Pettigrew and Whipp's context (why), process (how) and content (what) framework of strategic change was used to guide data collection and analysis. Data collection methods included quantitative tools (a validated culture assessment questionnaire) and qualitative approaches including key informant interviews and participant observation.

Results

Method triangulation provided common and contrasting results between the four units. A participatory culture, supportive leadership, financial and non-financial incentives, the presentation of clear direction by integrating support for the BSC in policies, resources, and routine activities emerged as desirable attributes for BSC implementation. The two units that lagged behind were more involved in direct inpatient care and carried a considerable clinical workload. Role clarification and consensus about the purpose and benefits of the BSC were noted as key strategies for overcoming implementation challenges in two clinical units that were relatively ahead in BSC implementation. It was noted that, rather than seeking to replace existing information systems, initiatives such as the BSC could be readily adopted if they are built on existing infrastructures and data networks.

Conclusion

Variable levels of the BSC implementation were observed in this study. Those intending to apply the BSC in other hospital settings need to ensure a participatory culture, clear institutional mandate, appropriate leadership support, proper reward and recognition system, and sensitization to BSC benefits.

Culture and quality care perceptions in a Pakistani hospital

PURPOSE

Organizational culture is a determinant for quality improvement. This paper aims to assess organizational culture in a hospital setting, understand its relationship with perceptions about quality of care and identify areas for improvement.

DESIGN/METHODOLOGY/APPROACH

The paper is based on a cross-sectional survey in a large clinical department that used two validated questionnaires. The first contained 20 items addressing perceptions of cultural typology (64 respondents). The second one assessed staff views on quality improvement implementation (48 faculty) in three domains: leadership, information and analysis and human resource utilization (employee satisfaction).

FINDINGS

All four cultural types received scoring, from a mean of 17.5 (group), 13.7 (developmental), 31.2 (rational) to 37.2 (hierarchical). The latter was the dominant cultural type. Group (participatory) and developmental (open) culture types had significant positive correlation with optimistic perceptions about leadership (r = 0.48 and 0.55 respectively, p < 0.00). Hierarchical (bureaucratic) culture was significantly negatively correlated with domains; leadership (r = -0.61,p < 0.00), information and analysis (-0.50, p < 0.00) and employee satisfaction (r = -0.55, p < 0.00). Responses reveal a need for leadership to better utilize suggestions for improving quality of care, strengthening the process of information analysis and encouraging reward and recognition for employees.

RESEARCH LIMITATIONS/IMPLICATIONS

It is likely that, by adopting a participatory and open culture, staff views about organizational leadership will improve and employee satisfaction will be enhanced. This finding has implications for quality care implementation in other hospital settings.

ORIGINALITY/VALUE

The paper bridges an important gap in the literature by addressing the relationship between culture and quality care perceptions in a Pakistani hospital. As such a new and informative perspective is added.

Benchmarking for small hospitals: size didn't matter!

Benchmarking is an indispensable tool as hospital leaders face challenges to balance efficiency with safe and effective care. Selection of appropriate "like" hospitals is critical to the benchmarking aim of understanding comparative performance. Based on 10 years of observed outcome differences between small and large hospitals, the Collaborative Alliance for Nursing Outcomes (CALNOC) sought to empirically define small hospitals, and to determine if there were statistical differences between small and large hospitals for selected nursing sensitive outcome indicators. This article reports the examination of hospital size as a proxy characteristic to define "like" hospitals for the purpose of benchmarking outcomes. Findings suggest that optimal classifications into small and large hospital size based on the outcome indicators of falls, falls with injury, and hospital-acquired pressure ulcers stage 2 or worse (HAPU 2+) were not consistent with historical administrative categories based on average daily census and not consistent by outcome. Statistical differences were only found with HAPU 2+ in critical care units, with no differences in the fall outcomes. These data did not support the use of size-based categories to define like hospitals for benchmark comparisons.

Designing a balanced scorecard for a tertiary care hospital in Pakistan: a modified Delphi group exercise

Balanced Scorecards (BSC) are being implemented in high income health settings linking organizational strategies with performance data. At this private university hospital in Pakistan an elaborate information system exists. This study aimed to make best use of available data for better performance management. Applying the modified Delphi technique an expert panel of clinicians and hospital managers reduced a long list of indicators to a manageable size. Indicators from existing documents were evaluated for their importance, scientific soundness, appropriateness to hospital's strategic plan, feasibility and modifiability. Panel members individually rated each indicator on a scale of 1-9 for the above criteria. Median scores were assigned. Of an initial set of 50 indicators, 20 were finally selected to be assigned to the four BSC quadrants. These were financial (n = 4), customer or patient (n = 4), internal business or quality of care (n = 7) and innovation/learning or employee perspectives (n = 5). A need for stringent definitions, international benchmarking and standardized measurement methods was identified. BSC compels individual clinicians and managers to jointly work towards improving performance. This scorecard is now ready to be implemented by this hospital as a performance management tool for monitoring indicators, addressing measurement issues and enabling comparisons with hospitals in other settings.