"What is meant by public?": Stakeholder views on strengthening impacts of public reporting of hospital performance data

Patient Perspectives of Center-Specific Reporting in Kidney Failure Care: An Australian Qualitative Study

Introduction

Public reporting of quality of care indicators in healthcare is intended to inform consumer decision-making; however, people may be unaware that such information exists, or it may not capture their priorities. The aim of this study was to understand the views of people with kidney disease about public reporting of dialysis and transplant center outcomes.

Methods

This qualitative study involved 27 patients with lived experience of kidney disease in Australia who participated in 11 online focus groups between August and December 2022. Transcripts were analyzed thematically.

Results

Patients from all Australian states and territories participated, with 22 (81%) having a functioning kidney transplant and 22 (81%) having current or previous experience of dialysis. Five themes were identified as follows: (i) surrendering to the health system, (ii) the complexity of quality, (iii) benefits for patient care and experience, (iv) concerned about risks and unintended consequences, and (v) optimizing the impact of data.

Conclusion

Patients desire choice among kidney services but perceive this as rarely possible in the Australian context. Health professionals are trusted to make decisions about appropriate centers. Public reporting of center outcomes may induce fear and a loss of balanced perspective; however, it was supported by all participants and represents an opportunity for self-advocacy and informed decision-making. Strategies to mitigate potential risks include availability of trusted clinicians and community members to aid in data interpretation, providing context about centers and patients, and framing statistics to promote positivity and hope.

Mechanisms and impact of public reporting on physicians and hospitals' performance: A systematic review (2000-2020)

BACKGROUND

Public performance reporting (PPR) of physician and hospital data aims to improve health outcomes by promoting quality improvement and informing consumer choice. However, previous studies have demonstrated inconsistent effects of PPR, potentially due to the various PPR characteristics examined. The aim of this study was to undertake a systematic review of the impact and mechanisms (selection and change), by which PPR exerts its influence.

METHODS

Studies published between 2000 and 2020 were retrieved from five databases and eight reviews. Data extraction, quality assessment and synthesis were conducted. Studies were categorised into: user and provider responses to PPR and impact of PPR on quality of care.

RESULTS

Forty-five studies were identified: 24 on user and provider responses to PPR, 14 on impact of PPR on quality of care, and seven on both. Most of the studies reported positive effects of PPR on the selection of providers by patients, purchasers and providers, quality improvement activities in primary care clinics and hospitals, clinical outcomes and patient experiences.

CONCLUSIONS

The findings provide moderate level of evidence to support the role of PPR in stimulating quality improvement activities, informing consumer choice and improving clinical outcomes. There was some evidence to demonstrate a relationship between PPR and patient experience. The effects of PPR varied across clinical areas which may be related to the type of indicators, level of data reported and the mode of dissemination. It is important to ensure that the design and implementation of PPR considered the perspectives of different users and the health system in which PPR operates in. There is a need to account for factors such as the structural characteristics and culture of the hospitals that could influence the uptake of PPR.

The impact of public performance reporting on cancer elective surgery waiting times: a data linkage study

Background

Excessive waiting times for cancer elective surgery are a concern in publicly funded healthcare systems. Several countries including Australia have introduced healthcare reforms involving time-based targets and public performance reporting (PPR) of hospital data. However, there is mixed evidence of their benefits. We sought to examine the impact of targets and PPR of cancer elective surgery waiting times on access to breast, bowel and lung cancer elective surgery.

Methods

We analysed routinely-collected linked data on admissions and waiting times for patients aged 15 years or over (n = 199,885) who underwent cancer surgery in a public hospital in Victoria, Australia over a 10-year period. We conducted difference-in-differences analyses to compare waiting times before (2006–07 to 2011–12) and after (2012–13 to 2015–16) the introduction of PPR in meeting these targets.

Results

Across all cancer types, urgent patients were all treated within 30 days before and after PPR. Following PPR, there was a slight increase in the mean waiting times across all cancer types and urgency categories. Patients with lung cancer waited on average two and half days longer for treatment and patients with breast cancer waited on average half-a-day less. There was no effect of PPR on waiting times for patients with bowel cancer across urgency categories.

Conclusions

Our findings suggest that time-based targets and PPR had minimal impact on surgical waiting times. This may be due to reasonable waiting times prior to PPR, improved efficiency being masked by 20% growth in the population, lack of public knowledge that waiting times are publicly reported, or lack of real-time reporting to drive behavioural change. The use of generic elective surgery recommended waiting time measures for cancer is discussed.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-06132-w.

Public disclosure of hospital clinicians' performance data: insights from medical directors

Objective This study gathered information from public hospital chief medical officers to better understand underlying mechanisms through which public reporting affects institutional behavioural change and decision making towards quality improvement. Methods This qualitative study used thematic analysis of 17 semistructured, in-depth interviews among a peak group of medical directors representing 26 health services in Victoria, Australia. Results The medical directors indicated a high level of in-principle support for public reporting of identifiable, individual clinician-level data. However, they also described varying conceptual understanding of what public reporting of performance data is. Overall, they considered public reporting of individual clinicians' performance data a means to improve health care quality, increase transparency and inform consumer healthcare decision making. Most identified caveats that would need to be met before such data should be publicly released, in particular the need to resolve issues around data quality and timeliness, context and interpretation and ethics. Acknowledgement of the public's right to access individual clinician-level data was at odds with some medical directors' belief that such reporting may diminish trust between clinicians and their employers, thus eroding rather than motivating quality improvement. Conclusions Public reporting of identifiable individual healthcare clinicians' performance data is an issue that merits robust research and debate given the effects such reporting may have on doctors and on hospital quality and safety. What is known about the topic? The public reporting of individual clinician-level data is a mechanism used in some countries, but not in Australia, for increasing health care transparency and quality. Clinician-level public reporting of doctors' performance attracts contention and debate in Australia. What does this paper add? This paper informs debate around the public reporting of individual clinician-level performance data. Among a discrete cohort of senior hospital administrators in Victoria, Australia, there was strong in-principle support for such public reporting as a means to improve hospital quality and safety. What are the implications for practitioners? Before public reporting of individual clinician performance data could occur in Australia, resolution of issues would be required relating to legality and ethics, data context and interpretation, data quality and timeliness.

Can publicly reported data be used to understand performance in an Australian rural hospital?

BACKGROUND

Despite agreement among policymakers, funders, consumers and researchers about the value of public reporting of health information, limited attention has been paid to how it can be used to understand the performance of rural hospitals.

OBJECTIVE

To determine whether publicly available information can be used to measure health service performance in a rural hospital.

METHOD

The study used performance data routinely reported for public consumption in Australia. Data across four domains, multiple measures and time periods were collected to examine access and equity; efficiency and sustainability; quality, safety and patient orientation; and employee engagement. Performance of the rural hospital was examined using a visualisation tool.

RESULTS

Visualisation of multiple measures of performance over time was achievable but required a high degree of health information management skills.

CONCLUSION AND IMPLICATIONS

Publicly reported data can be used to represent performance for a rural hospital. Timeliness, level of detail available and peer groupings of data limits optimal utility. Consumers, clinicians and health service managers wanting to understand the performance of rural hospitals will need to use significant health information management skills to gain a picture of performance. Further research in the applied use of publicly available performance data and relevant dashboards for rural hospitals is suggested.

Impact of public release of performance data on the behaviour of healthcare consumers and providers

BACKGROUND

It is becoming increasingly common to publish information about the quality and performance of healthcare organisations and individual professionals. However, we do not know how this information is used, or the extent to which such reporting leads to quality improvement by changing the behaviour of healthcare consumers, providers, and purchasers.

OBJECTIVES

To estimate the effects of public release of performance data, from any source, on changing the healthcare utilisation behaviour of healthcare consumers, providers (professionals and organisations), and purchasers of care. In addition, we sought to estimate the effects on healthcare provider performance, patient outcomes, and staff morale.

SEARCH METHODS

We searched CENTRAL, MEDLINE, Embase, and two trials registers on 26 June 2017. We checked reference lists of all included studies to identify additional studies.

SELECTION CRITERIA

We searched for randomised or non-randomised trials, interrupted time series, and controlled before-after studies of the effects of publicly releasing data regarding any aspect of the performance of healthcare organisations or professionals. Each study had to report at least one main outcome related to selecting or changing care.

DATA COLLECTION AND ANALYSIS

Two review authors independently screened studies for eligibility and extracted data. For each study, we extracted data about the target groups (healthcare consumers, healthcare providers, and healthcare purchasers), performance data, main outcomes (choice of healthcare provider, and improvement by means of changes in care), and other outcomes (awareness, attitude, knowledge of performance data, and costs). Given the substantial degree of clinical and methodological heterogeneity between the studies, we presented the findings for each policy in a structured format, but did not undertake a meta-analysis.

MAIN RESULTS

We included 12 studies that analysed data from more than 7570 providers (e.g. professionals and organisations), and a further 3,333,386 clinical encounters (e.g. patient referrals, prescriptions). We included four cluster-randomised trials, one cluster-non-randomised trial, six interrupted time series studies, and one controlled before-after study. Eight studies were undertaken in the USA, and one each in Canada, Korea, China, and The Netherlands. Four studies examined the effect of public release of performance data on consumer healthcare choices, and four on improving quality.There was low-certainty evidence that public release of performance data may make little or no difference to long-term healthcare utilisation by healthcare consumers (3 studies; 18,294 insurance plan beneficiaries), or providers (4 studies; 3,000,000 births, and 67 healthcare providers), or to provider performance (1 study; 82 providers). However, there was also low-certainty evidence to suggest that public release of performance data may slightly improve some patient outcomes (5 studies, 315,092 hospitalisations, and 7502 providers). There was low-certainty evidence from a single study to suggest that public release of performance data may have differential effects on disadvantaged populations. There was no evidence about effects on healthcare utilisation decisions by purchasers, or adverse effects.

AUTHORS' CONCLUSIONS

The existing evidence base is inadequate to directly inform policy and practice. Further studies should consider whether public release of performance data can improve patient outcomes, as well as healthcare processes.