Employers' efforts to measure and improve hospital quality: determinants of success

Exploring the factors impacting choice and quality of overnight private hospital stays and consumer perspectives on patient reported experience measures (PREMs) in Australia: a qualitative interview study

OBJECTIVES

Patient reported experience measures (PREMs) are tools often utilised in hospitals to support quality improvements and to provide objective feedback on care experiences. Less commonly PREMs can be used to support consumers choices in their hospital care. Little is known about the experience and views of the Australian consumer regarding PREMs nor the considerations these consumers have when they need to make decisions about attending hospital. This study aimed to explore consumer awareness of PREMs, consumer attitudes towards PREMs and the utility of PREMs as a decision-making tool in accessing hospital care.

METHODS

Qualitative study involving semi-structured interviews conducted over the phone. Participants (n = 40) were recruited from across Australia and purposively sampled according to key characteristics: holding private health insurance, > 30-years of age, may have accessed private hospital care in the past year, variety of educational and cultural backgrounds, and if urban or rural residing. Interviews were audio-recorded, transcribed, and analysed thematically.

RESULTS

Four overarching themes and six subthemes were identified from the data. Major findings were that prior awareness of PREMs was limited; however, many had filled in a PREM either for themselves or for someone they cared for following a hospital stay. Most respondents preferred to listen to experience of self or family/friends or the recommendation of their physician when choosing a hospital to attend. Participants appeared to be more interested in the treating clinician than the hospital with this clinician often dictating the hospital or hospital options. If provided choice in hospital, issues of additional costs, timeliness of treatment and location were important factors.

CONCLUSION

While PREMs were considered a possible tool to assist in hospital decision-making process, previous hospital experiences, the doctor and knowing up-front cost are an overriding consideration for consumers when choosing their hospital. Consideration to format and presentation of PREMs data is needed to facilitate understanding and allow meaningful comparisons. Future research could examine the considerations of those consumers who primarily access public healthcare facilities and how to improve the utility of PREMs.

How do aggregated patient-reported outcome measures data stimulate health care improvement? A realist synthesis

Objectives Internationally, there has been considerable debate about the role of data in supporting quality improvement in health care. Our objective was to understand how, why and in what circumstances the feedback of aggregated patient-reported outcome measures data improved patient care. Methods We conducted a realist synthesis. We identified three main programme theories underlying the use of patient-reported outcome measures as a quality improvement strategy and expressed them as nine 'if then' propositions. We identified international evidence to test these propositions through searches of electronic databases and citation tracking, and supplemented our synthesis with evidence from similar forms of performance data. We synthesized this evidence through comparing the mechanisms and impact of patient-reported outcome measures and other performance data on quality improvement in different contexts. Results Three programme theories were identified: supporting patient choice, improving accountability and enabling providers to compare their performance with others. Relevant contextual factors were extent of public disclosure, use of financial incentives, perceived credibility of the data and the practicality of the results. Available evidence suggests that patients or their agents rarely use any published performance data when selecting a provider. The perceived motivation behind public reporting is an important determinant of how providers respond. When clinicians perceived that performance indicators were not credible but were incentivized to collect them, gaming or manipulation of data occurred. Outcome data do not provide information on the cause of poor care: providers needed to integrate and interpret patient-reported outcome measures and other outcome data in the context of other data. Lack of timeliness of performance data constrains their impact. Conclusions Although there is only limited research evidence to support some widely held theories of how aggregated patient-reported outcome measures data stimulate quality improvement, several lessons emerge from interventions sharing the same programme theories to help guide the increasing use of these measures.

Functionality and feedback: a realist synthesis of the collation, interpretation and utilisation of patient-reported outcome measures data to improve patient care

Background

The feedback of patient-reported outcome measures (PROMs) data is intended to support the care of individual patients and to act as a quality improvement (QI) strategy.

Objectives

To (1) identify the ideas and assumptions underlying how individual and aggregated PROMs data are intended to improve patient care, and (2) review the evidence to examine the circumstances in which and processes through which PROMs feedback improves patient care.

Design

Two separate but related realist syntheses: (1) feedback of aggregate PROMs and performance data to improve patient care, and (2) feedback of individual PROMs data to improve patient care.

Interventions

Aggregate – feedback and public reporting of PROMs, patient experience data and performance data to hospital providers and primary care organisations. Individual – feedback of PROMs in oncology, palliative care and the care of people with mental health problems in primary and secondary care settings.

Main outcome measures

Aggregate – providers’ responses, attitudes and experiences of using PROMs and performance data to improve patient care. Individual – providers’ and patients’ experiences of using PROMs data to raise issues with clinicians, change clinicians’ communication practices, change patient management and improve patient well-being.

Data sources

Searches of electronic databases and forwards and backwards citation tracking.

Review methods

Realist synthesis to identify, test and refine programme theories about when, how and why PROMs feedback leads to improvements in patient care.

Results

Providers were more likely to take steps to improve patient care in response to the feedback and public reporting of aggregate PROMs and performance data if they perceived that these data were credible, were aimed at improving patient care, and were timely and provided a clear indication of the source of the problem. However, implementing substantial and sustainable improvement to patient care required system-wide approaches. In the care of individual patients, PROMs function more as a tool to support patients in raising issues with clinicians than they do in substantially changing clinicians’ communication practices with patients. Patients valued both standardised and individualised PROMs as a tool to raise issues, but thought is required as to which patients may benefit and which may not. In settings such as palliative care and psychotherapy, clinicians viewed individualised PROMs as useful to build rapport and support the therapeutic process. PROMs feedback did not substantially shift clinicians’ communication practices or focus discussion on psychosocial issues; this required a shift in clinicians’ perceptions of their remit.

Strengths and limitations

There was a paucity of research examining the feedback of aggregate PROMs data to providers, and we drew on evidence from interventions with similar programme theories (other forms of performance data) to test our theories.

Conclusions

PROMs data act as ‘tin openers’ rather than ‘dials’. Providers need more support and guidance on how to collect their own internal data, how to rule out alternative explanations for their outlier status and how to explore the possible causes of their outlier status. There is also tension between PROMs as a QI strategy versus their use in the care of individual patients; PROMs that clinicians find useful in assessing patients, such as individualised measures, are not useful as indicators of service quality.

Future work

Future research should (1) explore how differently performing providers have responded to aggregate PROMs feedback, and how organisations have collected PROMs data both for individual patient care and to improve service quality; and (2) explore whether or not and how incorporating PROMs into patients’ electronic records allows multiple different clinicians to receive PROMs feedback, discuss it with patients and act on the data to improve patient care.

Study registration

This study is registered as PROSPERO CRD42013005938.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.

Hospital Report Cards: Intent, Impact, and Illusion

The public is currently being bombarded with a growing number of hospital report cards revealing hospital-specific outcome data. The original intent of these report cards was to provide information to the public to enable them to make a more educated choice in selecting health care services and to stimulate provider activities to improve outcomes of care. Serious issues have been raised about the accuracy, validity, meaningfulness, and interpretability of these reports, which has affected both provider and consumer response. With public image and financial implications on the line, hospitals need to be aware of these reports and take the necessary steps to improve outcomes of care.

A qualitative and quantitative evaluation of the Advancing Quality pay-for-performance programme in the NHS North West

Background

Advancing Quality (AQ) is a voluntary programme providing financial incentives for improvement in the quality of care provided to NHS patients in the north-west of England.

Objectives

(1) To identify the impact of AQ on key stakeholders and clinical practice; (2) to assess its cost-effectiveness; (3) to identify key factors that assist or impede its successful implementation; and (4) to provide lessons for the wider implementation of pay-for-performance schemes across the NHS.

Design

We tested whether or not the financial incentives of AQ had an impact on mortality using two methods: a between-region difference-in-differences analysis comparing the North West region and the rest of England for the incentivised and non-incentivised conditions and a triple-difference analysis comparing performance on the incentivised conditions, as well as the non-incentivised conditions, in the North West region and the rest of England. A cost-effectiveness analysis of AQ based on the first 18 months of the programme was also undertaken. We used interviews and observation to explore how and why changes occurred.

Results

Risk-adjusted mortality rates for all three of the conditions we studied (pneumonia, heart failure and myocardial infarction) decreased in both the North West region and the rest of England during the first 18 months of the scheme. The reduction in mortality for incentivised conditions was greater in the North West region than in the rest of England. Compared with non-incentivised conditions within the North West region, there was a significant reduction in overall mortality for incentivised conditions, comprising a statistically significant reduction in pneumonia and non-significant reductions in the other two conditions. Comparing mortality for the incentivised conditions with mortality for these conditions in other regions, there was a significant reduction in overall mortality in the North West region, again made up of individually significant reductions in pneumonia and non-significant reductions in the other two conditions. The reduction in mortality over the 18-month period studied for non-incentivised conditions was not significantly different between the North West region and the rest of England. The between-region difference-in-differences analysis after 42 months showed that risk-adjusted mortality for the incentivised conditions fell in the rest of England and the North West region. This reduction in the rest of England was significantly larger than in the North West region and was concentrated in pneumonia. However, the reductions in mortality were larger for the non-incentivised conditions in the North West region than in the rest of England between these periods. For incentivised conditions, the triple-difference analysis shows a larger reduction in mortality for the rest of England than in the North West region between the short- and long-term periods.

Conclusions

Based on the first 18 months, the AQ programme was a relatively effective and cost-effective intervention. However, findings at 42 months are open to interpretation. One interpretation is that the short-term improvements were not sustained and that the observed improvements in mortality in the non-incentivised conditions within hospitals participating in AQ were unrelated to the programme. An alternative interpretation is that these improvements are related to the positive spillover effect of AQ. Further research should be undertaken to determine the explanation for the findings.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.

Dissemination of performance information and continuous improvement: A narrative systematic review

PURPOSE

Developing a performance measure and reporting the results to support decision making at an individual level has yielded poor results in many health systems. The purpose of this paper is to highlight the factors associated with the dissemination of performance information that generate and support continuous improvement in health organizations.

DESIGN/METHODOLOGY/APPROACH

A systematic data collection strategy that includes empirical and theoretical research published from 1980 to 2010, both qualitative and quantitative, was performed on Web of Science, Current Contents, EMBASE and MEDLINE. A narrative synthesis method was used to iteratively detail explicative processes that underlie the intervention. A classification and synthesis framework was developed, drawing on knowledge transfer and exchange (KTE) literature. The sample consisted of 114 articles, including seven systematic or exhaustive reviews.

FINDINGS

Results showed that dissemination in itself is not enough to produce improvement initiatives. Successful dissemination depends on various factors, which influence the way collective actors react to performance information such as the clarity of objectives, the relationships between stakeholders, the system's governance and the available incentives.

RESEARCH LIMITATIONS/IMPLICATIONS

This review was limited to the process of knowledge dissemination in health systems and its utilization by users at the health organization level. Issues related to improvement initiatives deserve more attention.

PRACTICAL IMPLICATIONS

Knowledge dissemination goes beyond better communication and should be considered as carefully as the measurement of performance. Choices pertaining to intervention should be continuously prompted by the concern to support organizational action.

ORIGINALITY/VALUE

While considerable attention was paid to the public reporting of performance information, this review sheds some light on a more promising avenue for changes and improvements, notably in public health systems.

Mandatory quality reports in Germany from the hospitals' point of view: a cross-sectional observational study

Background

Public reporting of hospital quality is to enable providers, patients and the public to make comparisons regarding the quality of care and thus contribute to informed decisions. It stimulates quality improvement activities in hospitals and thus positively impacts treatment results. Hospitals often use publicly reported data for further internal or external purposes.

As of 2005, German hospitals are obliged to publish structured quality reports (QR) every two years. This gives them the opportunity to demonstrate their performance by number, type and quality in a transparent way. However, it constitutes a major burden to hospitals to generate and publish data required, and it is yet unknown if hospitals feel adequately represented and at the same time consider the effort appropriate.

This study assesses hospital leaders’ judgement about the capability of QR to put legally defined aims effectively and efficiently into practice. It also explores the additional purposes hospitals use their QR for.

Methods

In a cross-sectional observational study, a representative random sample out of 2,064 German hospitals (N=748) was invited to assess QR via questionnaire; 333 hospitals participated. We recorded the suitability of QR for representing number, type and quality of services, the adequacy of cost and benefits (6-level Likert scales) and additional purposes QR are used for (free text question). For representation purposes, the net sample was weighted for hospital size and hospital ownership (direct standardization). Data was analyzed descriptively and using inferential statistics (chi-2 test) or for the purpose of generating hypotheses.

Results

German hospitals rated the QR as suitable to represent the number of services but less so for the type and quality of services. The cost-benefit ratio was seen as inadequate. There were no significant differences between hospitals of different size or ownership.

Public hospitals additionally used their reports for mostly internal purposes (e.g. comparison with competitors, quality management) whereas private ones used them externally (e.g. communication, marketing) (p=0.024, chi-2 test, hypotheses-generating level).

Conclusions

German hospitals consider the mandatory QR as only partially capable to put the legally defined aims effectively and efficiently into practice. In order for public reporting to achieve its potentially positive effects, the QR must be more closely aligned to the needs of hospitals.

Intraclass reliability for assessing how well Taiwan constrained hospital-provided medical services using statistical process control chart techniques

BACKGROUND

Few studies discuss the indicators used to assess the effect on cost containment in healthcare across hospitals in a single-payer national healthcare system with constrained medical resources. We present the intraclass correlation coefficient (ICC) to assess how well Taiwan constrained hospital-provided medical services in such a system.

METHODS

A custom Excel-VBA routine to record the distances of standard deviations (SDs) from the central line (the mean over the previous 12 months) of a control chart was used to construct and scale annual medical expenditures sequentially from 2000 to 2009 for 421 hospitals in Taiwan to generate the ICC. The ICC was then used to evaluate Taiwan's year-based convergent power to remain unchanged in hospital-provided constrained medical services. A bubble chart of SDs for a specific month was generated to present the effects of using control charts in a national healthcare system.

RESULTS

ICCs were generated for Taiwan's year-based convergent power to constrain its medical services from 2000 to 2009. All hospital groups showed a gradually well-controlled supply of services that decreased from 0.772 to 0.415. The bubble chart identified outlier hospitals that required investigation of possible excessive reimbursements in a specific time period.

CONCLUSION

We recommend using the ICC to annually assess a nation's year-based convergent power to constrain medical services across hospitals. Using sequential control charts to regularly monitor hospital reimbursements is required to achieve financial control in a single-payer nationwide healthcare system.

Effective population management practices in diabetes care - an observational study

BACKGROUND

Ensuring that evidence based medicine reaches patients with diabetes in the US and internationally is challenging. The chronic care model includes evidence based management practices which support evidence based care. However, despite numerous studies, it is unclear which practices are most effective. Few studies assess the effect of simultaneous practices implemented to varying degrees. The present study evaluates the effect of fifteen practices applied concurrently and takes variation in implementation levels into account while assessing the impact of diabetes care management practices on glycemic and lipid monitoring.

METHODS

Fifteen management practices were identified. Implementation levels of the practices in 41 medical centres caring for 553,556 adults with diabetes were assessed from structured interviews with key informants. Stepwise logistic regression models with management practices as explanatory variables and glycemic and lipid monitoring as outcome variables were used to identify the diabetes care practices most associated with high performance.

RESULTS

Of the 15 practices studied, only provider alerts were significantly associated with higher glycemic and lipid monitoring rates. The odds ratio for glycemic monitoring was 4.07 (p < 0.00001); the odds ratio for lipid monitoring was 1.63 (p < 0.006). Weaker associations were found between action plans and glycemic monitoring (odds ratio = 1.44; p < 0.03) and between guideline distribution and training and lipid monitoring (odds ratio = 1.46; p < 0.03). The covariates of gender, age, cardiac disease and depression significantly affected monitoring rates.

CONCLUSIONS

Of fifteen diabetes care management practices, our data indicate that high performance is most associated with provider alerts and more weakly associated with action plans and with guideline distribution and training. Lack of convergence in the literature on effective care management practices suggests that factors contributing to high performance may be highly context-dependent or that the factors involved may be too numerous or their implementation too nuanced to be reliably identified in observational studies.

The Economics of Specialty Hospitals

Specialty hospitals, particularly those specializing in surgery and owned by physicians, have generated a relatively high degree of policy attention over the past several years. The main focus of policy debates has been in two areas: the extent to which specialty hospitals might compete unfairly with incumbent general hospitals and the extent to which physician ownership might be associated with higher usage. Largely absent from the debates, however, has been a discussion of the basic economic model of specialty hospitals. This article reviews existing literature, reports, and findings from site visits to explore the economic rationale for specialty hospitals. The discussion focuses on six factors associated with specialization: consumer demand, procedural operating margins, clinical efficiencies, procedural economies of scale, economies (and diseconomies) of scope, and competencies and learning. A better understanding of the economics of specialization will help policy makers evaluate the full spectrum of advantages and disadvantages of specialty hospitals.

Value-driven health care: the purchasers' perspective

Value-driven health care seeks to directly link quality health care to reimbursement. The 4 cornerstones of this movement are health information technology standards, quality standards, price standards, and incentives. These standards and how purchasers plan to implement them are summarized in this article.

Reviewing the application of the balanced scorecard with implications for low-income health settings

High-income countries (HICs) are increasingly making use of the balanced scorecard (BSC) in healthcare. Evidence about BSC usage in low-income countries (LICs) is deficient. This study assessed feasibility of BSC use in LICs. Systematic review of electronic databases shows that the BSC improved patient, staff, clinical, and financial outcomes in HICs. To translate the experience of BSC use in HICs to their use in LICs, the applicability parameters of the National Committee for Quality Assurance were applied. Despite contextual challenges, pilot testing of BSC use can be undertaken in selected LICs. Committed leadership, cultural readiness, quality information systems, viable strategic plans, and optimum resources are required.

Public reporting and pay-for-performance: safety-net hospital executives' concerns and policy suggestions

Safety-net hospitals (SNHs) may gain little financial benefit from the rapidly spreading adoption of public reporting and pay-for-performance, but may feel compelled to participate (and bear the costs of data collection) to meet public expectations of transparency and accountability. To better understand the concerns that SNH administrators have regarding public reporting and pay-for-performance, we interviewed 37 executives at randomly selected California SNHs. The main concerns noted by SNH executives were that human and financial resource constraints made it difficult for SNHs to accurately measure their performance. Additionally, some executives felt that market-driven public reporting and pay-for-performance may focus on clinical areas and incentive structures that may not be high-priority clinical areas for SNHs. Executives at SNHs suggested several policy responses to these concerns-such as offering training programs for SNH data collectors-that could be relatively inexpensive and might improve the cost-benefit ratio of public reporting and pay-for-performance programs.

A behavioral model of clinician responses to incentives to improve quality

The use of pay for performance (P4P) and public reporting of performance (PR) in health care is increasing rapidly worldwide. The rationale for P4P and PR comes from experience in other industries and from theories about incentive use from psychology, economics, and organizational behavior. This paper reviews the major themes from this prior research and considers how they might be applied to health care. The resulting conceptual model addresses the dual nature (combining direct financial and reputational incentives) of the initiatives many policymakers are pursuing. It also includes explicit recognition of the key contextual factors (at the levels of the markets and the provider organization) and provider and patient characteristics that can enhance or mitigate response to incentives. Evaluation of the existing literature (through June 2005) about incentive use in health care in light of the conceptual model highlights important weaknesses in the way that trials have been reported to date and suggests future research topics.

Using public reports of patient satisfaction for hospital quality improvement

OBJECTIVE

To explore the impact of statewide public reporting of hospital patient satisfaction on hospital quality improvement (QI), using Rhode Island (RI) as a case example.

DATA SOURCE

Primary data collected through semi-structured interviews between September 2002 and January 2003.

STUDY DESIGN

The design is a retrospective study of hospital executives at all 11 general and two specialty hospitals in RI. Respondents were asked about hospital QI activities at several points throughout the public reporting process, as well as about hospital structure and processes to accomplish QI. Qualitative analysis of the interview data proceeded through an iterative process to identify themes and categories in the data.

PRINCIPAL FINDINGS

Data from the standardized statewide patient satisfaction survey process were used by hospitals to identify and target new QI initiatives, evaluate performance, and monitor progress. While all hospitals fully participated in the public reporting process, they varied in the stage of development of their QI activities and adoption of the statewide standardized survey for ongoing monitoring of their QI programs. Most hospitals placed responsibility for QI within each department, with results reported to top management, who were perceived as giving strong support for QI. The external environment facilitated QI efforts.

CONCLUSION

Public reporting of comparative data on patient views can enhance and reinforce QI efforts in hospitals. The participation of key stakeholders facilitated successful implementation of statewide public reporting. This experience in RI offers lessons for other states or regions as they move to public reporting of hospital quality data.

PUBLIC REPORTING OF PROVIDER PERFORMANCE: Can Its Impact Be Made Greater?

Public reporting of provider performance is becoming increasingly commonplace. In this chapter, we first review studies of prior public reports (or report cards) that show real but small impact on provider attempts to improve quality, on consumers' impressions of providers, and even on consumer selection of providers. Among other factors, two potential explanations for the low level of impact are that, in most early reports, the large majority of providers have been labeled "average" and consumers may have had difficulty understanding the statistical assessments. In response, some current report card producers are using or considering a variety of methods to increase the number of distinctions among providers and the ease of comprehension of the labels used. Therefore, we also consider the advantages and disadvantages of several novel approaches to analyzing and reporting provider performance.

Publicly Reporting Comprehensive Quality and Cost Data: A Health Care System’s Transparency Initiative

BACKGROUND

Transparency in health care, including the public reporting of health care results, is an expanding and unstoppable phenomenon. Health care systems have an opportunity to: (1) be proactive and accountable for the care they provide, (2) help patients learn more about their condition as a supplement to understanding the performance measures, and (3) use public reporting to foster process of care and outcome improvement initiatives. An overview is provided of the first 22 months of a transparency initiative at Dartmouth-Hitchcock Medical Center (DHMC).

LAUNCHING THE TRANSPARENCY INITIATIVE

An interdisciplinary operations group works with the various clinical programs--both providers and patients--to identify what quality and cost measures are most desired by patients and what measures are the focus of the clinical program's internal measurement and reporting processes. The measures are presented on the DHMC Web site, with access to additional resources, such as clinical decision aids.

DISCUSSION

A variety of factors are important to the transparency initiative--senior leaders' perceptions, risk management issues, resources required for the design and maintenance of the initiative, and developing both methodological protocols and technical systems.

Methodological issues in public reporting of patient perspectives on hospital quality

BACKGROUND

Increasing attention is being focused on public reporting of patient satisfaction and experience with hospital care, both nationally and at the state level. Comparative reports on hospital patient satisfaction use a standard survey, but little is known about underlying methodological approaches for reporting these quality measures.

METHODS

Literature, Web sites, and key informants were used to identify nine public reports. In-depth reviews were conducted to determine approaches to collecting, analyzing, and publicly reporting comparative data. Data were grouped into four analytic categories: survey, sampling, computation of scores, and reporting of scores.

RESULTS

The reports were similar in response rates and sampling procedures but differed in the number of hospitals included, the survey instrument, and survey procedure. The reports varied considerably in the techniques for computing hospital scores and decisions about reporting scores.

CONCLUSIONS

Reports from nine locales illustrate the decision making necessary to produce comparative reports on hospital patient satisfaction. Differences stem from decisions about the survey instrument and statistical decisions about how to interpret and report data. These issues should be clearly delineated as part of any public reporting process.

What is driving hospitals' patient-safety efforts?

The Institute of Medicine's report To Err Is Human described the alarming prevalence of medical errors and recommended a range of activities to improve patient safety. Three general mechanisms for stimulating hospitals to reduce medical errors are professionalism, regulation, and market forces. Although some believe that market forces are becoming more important, we found that a quasi-regulatory organization (the Joint Commission on Accreditation of Healthcare Organizations) has been the primary driver of hospitals' patient-safety initiatives. Professional and market initiatives have also facilitated improvement, but hospitals report that these have had less impact to date.