Public disclosure of performance data: learning from the US experience

Web-Based Public Reporting as a Decision-Making Tool for Consumers of Long-Term Care in the United States and the United Kingdom: Systematic Analysis of Report Cards

BACKGROUND

Report cards can help consumers make an informed decision when searching for a long-term care facility.

OBJECTIVE

This study aims to examine the current state of web-based public reporting on long-term care facilities in the United States and the United Kingdom.

METHODS

We conducted an internet search for report cards, which allowed for a nationwide search for long-term care facilities and provided freely accessible quality information. On the included report cards, we drew a sample of 1320 facility profiles by searching for long-term care facilities in 4 US and 2 UK cities. Based on those profiles, we analyzed the information provided by the included report cards descriptively.

RESULTS

We found 40 report cards (26 in the United States and 14 in the United Kingdom). In total, 11 of them did not state the source of information. Additionally, 7 report cards had an advanced search field, 24 provided simplification tools, and only 3 had a comparison function. Structural quality information was always provided, followed by consumer feedback on 27 websites, process quality on 15 websites, prices on 12 websites, and outcome quality on 8 websites. Inspection results were always displayed as composite measures.

CONCLUSIONS

Apparently, the identified report cards have deficits. To make them more helpful for users and to bring public reporting a bit closer to its goal of improving the quality of health care services, both countries are advised to concentrate on optimizing the existing report cards. Those should become more transparent and improve the reporting of prices and consumer feedback. Advanced search, simplification tools, and comparison functions should be integrated more widely.

Beyond Targets: Measuring Better and Rebuilding Trust Comment on "Gaming New Zealand's Emergency Department Target: How and Why Did It Vary Over Time and Between Organisations?"

Tenbensel and colleagues identify that a target for emergency department (ED) stays in New Zealand met with gaming in response from local hospitals. The result is in line with studies in other jurisdictions. The enthusiasm for targets and tight performance measurement in some health systems reflects a lack of trust in professionals to do the right thing for altruistic reasons. However such measurement systems have failed to address this loss of trust and may, ironically, have worsened the situation. A more promising approach for both improving performance and restoring trust may depend upon collaboration and partnership between consumers, local providers, and central agencies in agreeing and tracking appropriate local responses to high level national goals rather than imposing tight, and potentially misleading measures from the centre.

Citizen perspectives on the use of publicly reported primary care performance information: Results from citizen-patient dialogues in three Canadian provinces

OBJECTIVE

Performance measurement and reporting is proliferating in all sectors of the healthcare system, including primary care, despite a dearth of evidence on how the public uses reports on primary care performance. We explored how the public might use this information, to guide the development of effective reporting systems for primary care.

METHODS

We conducted six full-day deliberative dialogue sessions with a purposive sample of 56 citizen-patients across three Canadian provinces (British Columbia, Ontario and Nova Scotia). Participants identified how they would use publicly reported performance data. We conducted a thematic analysis of the data by region.

RESULTS

Common uses for primary care performance information emerged across all sessions. Participants most often discussed the utility of this information for community advocacy and participation in health system decision making. Similar barriers for using performance information to choose a primary care provider were identified in each region including the perceived lack of choice of providers and the high value placed on relationships with current providers. Finally, the value of public performance reporting in enhancing trust that people would receive good care was also a common theme.

CONCLUSIONS

Citizen-patient perspectives highlight that public reporting on primary care performance could promote the health system's responsiveness by enabling public engagement in decision making at the community level. The role of public reporting in promoting trust rather than empowering patient choice may reflect unique elements of the Canadian health system's context.

Functionality and feedback: a realist synthesis of the collation, interpretation and utilisation of patient-reported outcome measures data to improve patient care

Background

The feedback of patient-reported outcome measures (PROMs) data is intended to support the care of individual patients and to act as a quality improvement (QI) strategy.

Objectives

To (1) identify the ideas and assumptions underlying how individual and aggregated PROMs data are intended to improve patient care, and (2) review the evidence to examine the circumstances in which and processes through which PROMs feedback improves patient care.

Design

Two separate but related realist syntheses: (1) feedback of aggregate PROMs and performance data to improve patient care, and (2) feedback of individual PROMs data to improve patient care.

Interventions

Aggregate – feedback and public reporting of PROMs, patient experience data and performance data to hospital providers and primary care organisations. Individual – feedback of PROMs in oncology, palliative care and the care of people with mental health problems in primary and secondary care settings.

Main outcome measures

Aggregate – providers’ responses, attitudes and experiences of using PROMs and performance data to improve patient care. Individual – providers’ and patients’ experiences of using PROMs data to raise issues with clinicians, change clinicians’ communication practices, change patient management and improve patient well-being.

Data sources

Searches of electronic databases and forwards and backwards citation tracking.

Review methods

Realist synthesis to identify, test and refine programme theories about when, how and why PROMs feedback leads to improvements in patient care.

Results

Providers were more likely to take steps to improve patient care in response to the feedback and public reporting of aggregate PROMs and performance data if they perceived that these data were credible, were aimed at improving patient care, and were timely and provided a clear indication of the source of the problem. However, implementing substantial and sustainable improvement to patient care required system-wide approaches. In the care of individual patients, PROMs function more as a tool to support patients in raising issues with clinicians than they do in substantially changing clinicians’ communication practices with patients. Patients valued both standardised and individualised PROMs as a tool to raise issues, but thought is required as to which patients may benefit and which may not. In settings such as palliative care and psychotherapy, clinicians viewed individualised PROMs as useful to build rapport and support the therapeutic process. PROMs feedback did not substantially shift clinicians’ communication practices or focus discussion on psychosocial issues; this required a shift in clinicians’ perceptions of their remit.

Strengths and limitations

There was a paucity of research examining the feedback of aggregate PROMs data to providers, and we drew on evidence from interventions with similar programme theories (other forms of performance data) to test our theories.

Conclusions

PROMs data act as ‘tin openers’ rather than ‘dials’. Providers need more support and guidance on how to collect their own internal data, how to rule out alternative explanations for their outlier status and how to explore the possible causes of their outlier status. There is also tension between PROMs as a QI strategy versus their use in the care of individual patients; PROMs that clinicians find useful in assessing patients, such as individualised measures, are not useful as indicators of service quality.

Future work

Future research should (1) explore how differently performing providers have responded to aggregate PROMs feedback, and how organisations have collected PROMs data both for individual patient care and to improve service quality; and (2) explore whether or not and how incorporating PROMs into patients’ electronic records allows multiple different clinicians to receive PROMs feedback, discuss it with patients and act on the data to improve patient care.

Study registration

This study is registered as PROSPERO CRD42013005938.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.

Bridging the Gap between Theory and Practice in Integrated Care: The Case of the Diabetic Foot Pathway in Tuscany

INTRODUCTION AND BACKGROUND

As diabetic foot (DF) care benefits from integration, monitoring geographic variations in lower limb Major Amputation rate enables to highlight potential lack of Integrated Care. In Tuscany (Italy), these DF outcomes were good on average but they varied within the region. In order to stimulate an improvement process towards integration, the project aimed to shift health professionals' focus on the geographic variation issue, promote the Population Medicine approach, and engage professionals in a community of practice.

METHOD

Three strategies were thus carried out: the use of a transparent performance evaluation system based on benchmarking; the use of patient stories and benchmarking analyses on outcomes, service utilization and costs that cross-checked delivery- and population-based perspectives; the establishment of a stable community of professionals to discuss data and practices.

RESULTS

The project enabled professionals to shift their focus on geographic variation and to a joint accountability on outcomes and costs for the entire patient pathways. Organizational best practices and gaps in integration were identified and improvement actions towards Integrated Care were implemented.

CONCLUSION AND DISCUSSION

For the specific category of care pathways whose geographic variation is related to a lack of Integrated Care, a comprehensive strategy to improve outcomes and reduce equity gaps by diffusing integration should be carried out.

Accounting for quality: on the relationship between accounting and quality improvement in healthcare

BACKGROUND

Accounting-that is, standardized measurement, public reporting, performance evaluation and managerial control-is commonly seen to provide the core infrastructure for quality improvement in healthcare. Yet, accounting successfully for quality has been a problematic endeavor, often producing dysfunctional effects. This has raised questions about the appropriate role for accounting in achieving quality improvement. This paper contributes to this debate by contrasting the specific way in which accounting is understood and operationalized for quality improvement in the UK National Health Service (NHS) with findings from the broadly defined 'social studies of accounting' literature and illustrative examples.

DISCUSSION

This paper highlights three significant differences between the way that accounting is understood to operate in the dominant health policy discourse and recent healthcare reforms, and in the social studies of accounting literature. It shows that accounting does not just find things out, but makes them up. It shows that accounting is not simply a matter of substance, but of style. And it shows that accounting does not just facilitate, but displaces, control. The illumination of these differences in the way that accounting is conceptualized helps to diagnose why accounting interventions often fail to produce the quality improvements that were envisioned. This paper concludes that accounting is not necessarily incompatible with the ambition of quality improvement, but that it would need to be understood and operationalized in new ways in order to contribute to this end. Proposals for this new way of advancing accounting are discussed. They include the cultivation of overlapping and even conflicting measures of quality, the evaluation of accounting regimes in terms of what they do to practice, and the development of distinctively skeptical calculative cultures.

Dissemination of performance information and continuous improvement: A narrative systematic review

PURPOSE

Developing a performance measure and reporting the results to support decision making at an individual level has yielded poor results in many health systems. The purpose of this paper is to highlight the factors associated with the dissemination of performance information that generate and support continuous improvement in health organizations.

DESIGN/METHODOLOGY/APPROACH

A systematic data collection strategy that includes empirical and theoretical research published from 1980 to 2010, both qualitative and quantitative, was performed on Web of Science, Current Contents, EMBASE and MEDLINE. A narrative synthesis method was used to iteratively detail explicative processes that underlie the intervention. A classification and synthesis framework was developed, drawing on knowledge transfer and exchange (KTE) literature. The sample consisted of 114 articles, including seven systematic or exhaustive reviews.

FINDINGS

Results showed that dissemination in itself is not enough to produce improvement initiatives. Successful dissemination depends on various factors, which influence the way collective actors react to performance information such as the clarity of objectives, the relationships between stakeholders, the system's governance and the available incentives.

RESEARCH LIMITATIONS/IMPLICATIONS

This review was limited to the process of knowledge dissemination in health systems and its utilization by users at the health organization level. Issues related to improvement initiatives deserve more attention.

PRACTICAL IMPLICATIONS

Knowledge dissemination goes beyond better communication and should be considered as carefully as the measurement of performance. Choices pertaining to intervention should be continuously prompted by the concern to support organizational action.

ORIGINALITY/VALUE

While considerable attention was paid to the public reporting of performance information, this review sheds some light on a more promising avenue for changes and improvements, notably in public health systems.

Analytical perspectives on performance-based management: an outline of theoretical assumptions in the existing literature

Performance-based management (PBM) has become a dominant form of governance in health care and there is a need for careful assessment of its function and effects. This article contains a cross-disciplinary literature synthesis of current studies of PBM. Literature was retrieved by database searches and categorized according to analytical differences and similarities concerning (1) purpose and (2) governance mechanism of PBM. The literature could be grouped into three approaches to the study of PBM, which we termed: the ‘functionalist’, the ‘interpretive’ and the ‘post-modern’ perspective. In the functionalist perspective, PBM is perceived as a management tool aimed at improving health care services by means of market-based mechanisms. In the interpretive perspective, the adoption of PBM is understood as consequence of institutional and individual agents striving for public legitimacy. In the post-modern perspective, PBM is analysed as a form of governance, which has become so ingrained in Western culture that health care professionals internalize and understand their own behaviour and goals according to the values expressed in these governance systems. The recognition of differences in analytical perspectives allows appreciation of otherwise implicit assumptions and potential implications of PBM. Reflections on such differences are important to ensure vigilant appropriation of shifting management tools in health quality governance.

Patient safety as perceived by Swedish leaders

PURPOSE

The purpose of this paper is to survey Swedish healthcare leaders' patient safety awareness, the priority they give to safety issues and their views on suitable safety management strategies.

DESIGN/METHODOLOGY/APPROACH

A total 623 leaders of a sample of 1,129 responded to a mail questionnaire (55 percent response rate). Descriptive statistics of the responses are presented as frequency distributions across respondent subgroups. Means were tested for similarity by a repetitive one-way ANOVA procedure. Homogeneous response groups were sought by hierarchical cluster analysis.

FINDINGS

Swedish healthcare leaders show relatively high safety awareness and how their organizations prioritize safety management. There is a marked polarization between leaders; half feel that the system works reasonably well, and that adequate funds are available to improve or maintain services. The other half thinks the system needs major change and calls for additional funding. A majority sees system errors as the main cause for adverse events; a substantial minority find human errors to be more important. Two-thirds were willing to make safety performance information on organizations and specialties public, one third was restrictive.

RESEARCH LIMITATIONS/IMPLICATIONS

Survey instruments used to explore leaders' patient safety views have not yet been rigorously tested against psychometric criteria. One hospital type was slightly over-represented and three regions somewhat under-represented in the respondent groups.

ORIGINALITY/VALUE

This is the first systematic attempt to explore the views of Swedish healthcare leaders on patient safety. It provides input to a national strategy to improve patient safety.

Evidence of methodological bias in hospital standardised mortality ratios: retrospective database study of English hospitals

OBJECTIVE

To assess the validity of case mix adjustment methods used to derive standardised mortality ratios for hospitals, by examining the consistency of relations between risk factors and mortality across hospitals.

DESIGN

Retrospective analysis of routinely collected hospital data comparing observed deaths with deaths predicted by the Dr Foster Unit case mix method.

SETTING

Four acute National Health Service hospitals in the West Midlands (England) with case mix adjusted standardised mortality ratios ranging from 88 to 140.

PARTICIPANTS

96 948 (April 2005 to March 2006), 126 695 (April 2006 to March 2007), and 62 639 (April to October 2007) admissions to the four hospitals.

MAIN OUTCOME MEASURES

Presence of large interaction effects between case mix variable and hospital in a logistic regression model indicating non-constant risk relations, and plausible mechanisms that could give rise to these effects.

RESULTS

Large significant (P

CONCLUSIONS

The Dr Foster Unit hospital standardised mortality ratio is derived from an internationally adopted/adapted method, which uses at least two variables (the Charlson comorbidity index and emergency admission) that are unsafe for case mix adjustment because their inclusion may actually increase the very bias that case mix adjustment is intended to reduce. Claims that variations in hospital standardised mortality ratios from Dr Foster Unit reflect differences in quality of care are less than credible.

Collapse

Key Words Collapse

MESH Headings

• Bias
• Emergencies/epidemiology
• England/epidemiology
• Hospital Mortality
• Length of Stay/statistics & numerical data
• Patient Admission/statistics & numerical data
• Regression Analysis
• Retrospective Studies
• Risk Adjustment/statistics & numerical data
• Risk Factors

Collapse

Grants

• G0800808 Medical Research Council

Collapse

Affiliation(s)

Mohammed A Mohammed Unit of Public Health, Epidemiology and Biostatistics, University of Birmingham, Birmingham B15 2TT.
Jonathan J Deeks
Alan Girling
Gavin Rudge
Martin Carmalt
Andrew J Stevens
Richard J Lilford

Collapse

Are high-quality cardiac surgeons less likely to operate on high-risk patients compared to low-quality surgeons? Evidence from New York State

CONTEXT

It is unknown whether high-risk cardiac surgical patients have less access to high-quality surgeons compared with lower-risk patients.

OBJECTIVE

To determine whether high-quality surgeons are less likely to perform coronary artery bypass graft (CABG) surgery on high-risk patients compared with low-quality surgeons.

DESIGN, SETTING, AND PATIENTS

Retrospective cohort study using the New York State (NYS) CABG Surgery Reporting System (CSRS) of all patients undergoing CABG surgery in NYS who were discharged between 1997 and 1999 (51,750 patients; 2.20 percent mortality). Regression modeling was used to estimate the association between surgeon quality and patient risk of death. Surgeon quality was quantified using the observed-to-expected mortality ratio (O-to-E ratio).

RESULTS

Higher-risk patients are more likely to receive CABG surgery from higher-quality surgeons. For every 10 percentage point increase in patient risk of death (e.g., from 5 to 15 percent), there is an absolute reduction of 0.034 in the surgeon O-to-E ratio (p < .001).

CONCLUSION

This study suggests that high-risk CABG patients are significantly more likely to receive care from high-quality surgeons compared with lower risk patients.

Pay for performance, public reporting, and racial disparities in health care: how are programs being designed?

Pay-for-performance and public reporting programs may have a neutral, narrowing, or widening effect on racial disparities in health care. The authors begin this article by suggesting that certain characteristics of these programs may affect disparities. They then present results from a systematic review of the literature on the effects of performance incentive programs on racial disparities in health care. The review revealed that only one empirical study provided data on this issue: It showed that a major public reporting program increased disparities in coronary artery bypass graft rates. The authors then present the results of interviews with leaders of 15 major performance incentive programs in the United States. The interviews indicated that current programs are not designed to reduce disparities and often lack characteristics that may be important in reducing disparities. The article concludes with program leaders' recommendations on how performance incentive programs could be designed to reduce disparities.

Evaluating quality of primary care using the electronic medical record

Healthcare organizations are under increasing pressure to evaluate and report the level of quality in their health services. The electronic medical record (EMR) has been used in acute care settings to provide clinical data for quality evaluations. The implementation of the EMR in primary care settings is a more recent development, and as a result, the EMR has not been widely used to evaluate quality in primary care. Little research exists that uses the primary care medical record as a source of data. What remains to be seen is the extent to which EMRs contain the variables needed to address quality of primary care. This article describes a study that investigated the viability of the EMR as a database for evaluating quality in a women's primary health clinic.

Indicators of anesthesia safety and quality

In 1999, the Institute of Medicine shocked the world by claiming that medical error was among the leading causes of death in the United States. In contrast, anesthesiology was cited as an area in which there have been impressive gains in safety and quality. The mechanisms to which these impressive gains have been attributed include practice guidelines, anesthesia simulators, and benchmarking. Unfortunately, the current literature offers little evidence that these systematic approaches to patient safety have resulted in measurable improvements in quality, but efforts continue in the development of traditional and non-traditional quality indicators.

Publishing outcome data: is it an effective approach?

RATIONALE, AIMS AND OBJECTIVES

The publication of health outcome data--rather than merely the measurement and collection--is being given increasing consideration. Publication reflects society's increasing emphasis on a general 'right to know', as well as being a means of informing consumer choice. In theory, publication may help to promote public trust, support patient choice, and stimulate action to improve the quality of care whilst controlling costs.

METHODS

Drawing on a literature review, this paper overviews the strategies employed in the UK and US to publish outcome data. The focus is on outcomes, and certain related process measures, that measure the performance of hospitals or surgeons.

RESULTS AND CONCLUSIONS

Presenting the limited evidence that exists, we review the potential beneficial and harmful effects of publishing hospital outcome data. We also consider the risks of making incorrect inferences based on these data and the potential for dysfunctional consequences. Recognizing that the public largely mistrusts currently published health outcome data, we offer some recommendations for the future direction of strategies for publication.

Feasibility first: Developing public performance indicators on patient safety and clinical effectiveness for Dutch hospitals

This paper describes the development and implementation of the first national, public and obligatory set of hospital performance indicators in the Netherlands. Focusing on effectiveness and safety, the set was developed by the Dutch Health Care Inspectorate to improve the effectiveness and efficiency of their task: monitoring the quality of the care delivered by providers. In addition, the set would enhance the transparency of the hospital sector, and stimulate individual hospitals to improve their scores. Bridging some of the classic distinctions between 'internal' and 'external' indicators, the Inspectorate's vision was to rapidly produce a feasible set of indicators that would fulfill these aims, while maximally preventing 'side effects' such as misinterpretations, defensive or perverse reactions. Explicitly avoiding the trap of searching for exhaustive validity of the indicators, the inspectorate's motto was 'feasability first'. This paper describes how this simultaneously philosophical, political and pragmatic strategy played out successfully, and how the indicator set was ultimately embraced by all parties involved.

Publicly Reporting Comprehensive Quality and Cost Data: A Health Care System’s Transparency Initiative

BACKGROUND

Transparency in health care, including the public reporting of health care results, is an expanding and unstoppable phenomenon. Health care systems have an opportunity to: (1) be proactive and accountable for the care they provide, (2) help patients learn more about their condition as a supplement to understanding the performance measures, and (3) use public reporting to foster process of care and outcome improvement initiatives. An overview is provided of the first 22 months of a transparency initiative at Dartmouth-Hitchcock Medical Center (DHMC).

LAUNCHING THE TRANSPARENCY INITIATIVE

An interdisciplinary operations group works with the various clinical programs--both providers and patients--to identify what quality and cost measures are most desired by patients and what measures are the focus of the clinical program's internal measurement and reporting processes. The measures are presented on the DHMC Web site, with access to additional resources, such as clinical decision aids.

DISCUSSION

A variety of factors are important to the transparency initiative--senior leaders' perceptions, risk management issues, resources required for the design and maintenance of the initiative, and developing both methodological protocols and technical systems.

Data feedback efforts in quality improvement: lessons learned from US hospitals

BACKGROUND

Data feedback is a fundamental component of quality improvement efforts, but previous studies provide mixed results on its effectiveness. This study illustrates the diversity of hospital based efforts at data feedback and highlights successful strategies and common pitfalls in designing and implementing data feedback to support performance improvement.

METHODS

Open ended interviews with 45 clinical and administrative staff in eight US hospitals in 2000 concerning their perceptions about the effectiveness of data feedback in supporting performance improvement efforts were analysed. The hospitals were chosen to represent a range of sizes, geographical regions, and beta blocker improvement rates over a 3 year period. Data were organized and analyzed in NUD-IST 4 using the constant comparative method of qualitative data analysis.

RESULTS

Although the data feedback efforts at the hospitals were diverse, the interviews suggested that seven key themes may be important: (1) data must be perceived by physicians as valid to motivate change; (2) it takes time to develop the credibility of data within a hospital; (3) the source and timeliness of data are critical to perceived validity; (4) benchmarking improves the meaningfulness of data feedback; (5) physician leaders can enhance the effectiveness of data feedback; (6) data feedback that profiles an individual physician's practices can be effective but may be perceived as punitive; (7) data feedback must persist to sustain improved performance. Embedded in several themes was the view that the effectiveness of data feedback depends not only on the quality and timeliness of the data, but also on the organizational context in which such efforts are implemented.

CONCLUSIONS

Data feedback is a complex and textured concept. Data feedback strategies that might be most effective are suggested, as well as potential pitfalls in using data to promote performance improvement.

Public reporting on quality in the United States and the United Kingdom

The public reporting of comparative information about health care quality is becoming an accepted way of improving accountability and quality. Quality report cards have been prominent in the United States for more than a decade and are a central feature of British health system reform. In this paper we examine the common challenges and differences in implementation of the policy in the two countries. We use this information to explore some key questions relating to the content, target audience, and use of published information. We end by making specific recommendations for maximizing the effectiveness of public reporting.

Internet infrastructures and health care systems: a qualitative comparative analysis on networks and markets in the British National Health Service and Kaiser Permanente

BACKGROUND

The Internet and emergent telecommunications infrastructures are transforming the future of health care management. The costs of health care delivery systems, products, and services continue to rise everywhere, but performance of health care delivery is associated with institutional and ideological considerations as well as availability of financial and technological resources.

OBJECTIVE

to identify the effects of ideological differences on health care market infrastructures including the Internet and telecommunications technologies by a comparative case analysis of two large health care organizations: the British National Health Service and the California-based Kaiser Permanente health maintenance organization.

METHODS

A qualitative comparative analysis focusing on the British National Health Service and the Kaiser Permanente health maintenance organization to show how system infrastructures vary according to market dynamics dominated by health care institutions ("push") or by consumer demand ("pull"). System control mechanisms may be technologically embedded, institutional, or behavioral.

RESULTS

The analysis suggests that telecommunications technologies and the Internet may contribute significantly to health care system performance in a context of ideological diversity.

CONCLUSIONS

The study offers evidence to validate alternative models of health care governance: the national constitution model, and the enterprise business contract model. This evidence also suggests important questions for health care policy makers as well as researchers in telecommunications, organizational theory, and health care management.

Research on patients' views in the evaluation and improvement of quality of care

The identification of methods for assessing the views of patients on health care has only developed over the last decade or so. The use of patients' views to improve healthcare delivery requires valid and reliable measurement methods. Four approaches are recognised: inclusion of patients' views in the information to those seeking health care, identification of patient preferences in episodes of care, patient feedback on delivery of health care, and patients' views in decision making on healthcare systems. Outcome measures for the evaluation of the use of patients' views should reflect the aims in terms of processes or outcomes of care, including possible negative consequences. Rigorous methodologies for the evaluation of methods have yet to be implemented.

Reporting health care performance: learning from the past, prospects for the future

In the USA, where public reporting of data on clinical performance is most advanced, comparative performance information, in the form of 'report cards', 'provider profiles' and 'physician profiling', has been published for over a decade. Many other countries are now following a similar route and are seeking to develop comparative data on health care performance. Notwithstanding the idiosyncratic nature of US health care, and the implications this has for the generalizability of findings from the USA to other countries, it is pertinent to ask what other countries can learn from the US experience. Based on a series of structured interviews with leading experts on the US health system, this article draws out the key lessons for other countries as they develop similar policies in this area. This paper highlights three concerns that have dominated the development of adequate measures in the USA, and that require consideration when developing similar schemes elsewhere. Firstly, the need to develop indicators with sound metric properties - high in validity and meaningfulness, and appropriately risk-adjusted. Secondly, the need to involve all stakeholders in the design of indicators, and a requirement that those measures be adapted to different audiences. Thirdly, a need to understand the needs of end users and to engage with them in partnerships to increase the attention paid to measurement. This study concludes that the greatest challenge is posed by the desire to make comparative performance data more influential in leveraging performance improvement. Simply collecting, processing, analysing and disseminating comparative data is an enormous logistical and resource-intensive task, yet it is insufficient. Any national strategy emphasizing comparative data must grapple with how to engage the serious attention of those individuals and organizations to whom change is to be delivered.

Using the Internet to teach consumers about quality care

BACKGROUND

The Internet is an important source of health information for consumers. Patients can learn about their diagnoses, review treatments and medications, and locate other health information for themselves and their families. Information about quality care can also be found on the Internet. Few consumers, though, use these Web sites for learning about quality care.

SEARCH FOR WEB SITES ON QUALITY CARE

In 2000 the investigators searched the Internet and generated a list of approximately 90 relevant Internet documents under the broad heading of quality health care. They then pared the list to 34, by using the Health Information Technology Institute (HITI) criteria.

TESTING OF INTERNET DOCUMENTS BY CONSUMERS

In the second phase of the project, 5 of the 34 Internet documents were tested by a convenience sample of 32 consumers. Most of the participants had experience in using the Internet, although generally not in the area of quality care. They found the Web sites easy to use and indicated that the Internet resources would help them assess the quality of care they receive from physicians, nurses, and others.

DISCUSSION

Web sites need to be evaluated to ensure that the information they provide is accurate and current, among other criteria. All patients should understand their health benefits and the importance of making informed decisions about their health care, as well as how quality care is measured, how to use quality reports, how to choose providers and hospitals, how to assess the quality of their own care and be more involved in it, and what they should do when faced with new diagnoses.

Systematic review of studies of quality of clinical care in general practice in the UK, Australia and New Zealand

OBJECTIVES

Little is known about the quality of clinical care provided outside the hospital sector, despite the increasingly important role of clinical generalists working in primary care. In this study we aimed to summarise published evaluations of the quality of clinical care provided in general practice in the UK, Australia, and New Zealand.

DESIGN

A systematic review of published studies assessing the quality of clinical care in general practice for the period 1995-9.

SETTING

General practice based care in the UK, Australia, and New Zealand. Main outcome measures-Study design, sampling strategy and size, clinical conditions studied, quality of care attained for each condition (compared with explicit or implicit standards for the process of care), and country of origin for each study.

RESULTS

Ninety papers fulfilled the entry criteria for the review, 80 from the UK, six from Australia, and four from New Zealand. Two thirds of the studies assessed care in self-selected practices and 20% of the studies were based in single practices. The majority (85.5%) examined the quality of care provided for chronic conditions including cardiovascular disease (22%), hypertension (14%), diabetes (14%), and asthma (13%). A further 12% and 2% examined preventive care and acute conditions, respectively. In almost all studies the processes of care did not attain the standards set out in national guidelines or those set by the researchers themselves. For example, in the highest achieving practices 49% of diabetic patients had had their fundii examined in the previous year and 47% of eligible patients had been prescribed beta blockers after an acute myocardial infarction.

CONCLUSIONS

This study adopts an overview of the magnitude and the nature of clinical quality problems in general practice in three countries. Most of the studies in the systematic review come from the UK and the small number of papers from Australia and New Zealand make it more difficult to draw conclusions about the quality of care in these two countries. The review helps to identify deficiencies in the research, clinical and policy agendas in a part of the health care system where quality of care has been largely ignored to date. Further work is required to evaluate the quality of clinical care in a representative sample of the population, to identify the reasons for substandard care, and to test strategies to improve the clinical care provided in general practice.

Systematic review of studies of quality of clinical care in general practice in the UK, Australia and New Zealand

OBJECTIVES

Little is known about the quality of clinical care provided outside the hospital sector, despite the increasingly important role of clinical generalists working in primary care. In this study we aimed to summarise published evaluations of the quality of clinical care provided in general practice in the UK, Australia, and New Zealand.

DESIGN

A systematic review of published studies assessing the quality of clinical care in general practice for the period 1995-9.

SETTING

General practice based care in the UK, Australia, and New Zealand. Main outcome measures-Study design, sampling strategy and size, clinical conditions studied, quality of care attained for each condition (compared with explicit or implicit standards for the process of care), and country of origin for each study.

RESULTS

Ninety papers fulfilled the entry criteria for the review, 80 from the UK, six from Australia, and four from New Zealand. Two thirds of the studies assessed care in self-selected practices and 20% of the studies were based in single practices. The majority (85.5%) examined the quality of care provided for chronic conditions including cardiovascular disease (22%), hypertension (14%), diabetes (14%), and asthma (13%). A further 12% and 2% examined preventive care and acute conditions, respectively. In almost all studies the processes of care did not attain the standards set out in national guidelines or those set by the researchers themselves. For example, in the highest achieving practices 49% of diabetic patients had had their fundii examined in the previous year and 47% of eligible patients had been prescribed beta blockers after an acute myocardial infarction.

CONCLUSIONS

This study adopts an overview of the magnitude and the nature of clinical quality problems in general practice in three countries. Most of the studies in the systematic review come from the UK and the small number of papers from Australia and New Zealand make it more difficult to draw conclusions about the quality of care in these two countries. The review helps to identify deficiencies in the research, clinical and policy agendas in a part of the health care system where quality of care has been largely ignored to date. Further work is required to evaluate the quality of clinical care in a representative sample of the population, to identify the reasons for substandard care, and to test strategies to improve the clinical care provided in general practice.

Impact of published clinical outcomes data: case study in NHS hospital trusts

OBJECTIVE

To examine the impact of the publication of clinical outcomes data on NHS Trusts in Scotland to inform the development of similar schemes elsewhere.

DESIGN

Case studies including semistructured interviews and a review of background statistics.

SETTING

Eight Scottish NHS acute trusts.

PARTICIPANTS

48 trust staff comprising chief executives, medical directors, stroke consultants, breast cancer consultants, nurse managers, and junior doctors.

MAIN OUTCOME MEASURES

Staff views on the benefits and drawbacks of clinical outcome indicators provided by the clinical resource and audit group (CRAG) and perceptions of the impact of these data on clinical practice and continuous improvement of quality.

RESULTS

The CRAG indicators had a low profile in the trusts and were rarely cited as informing internal quality improvement or used externally to identify best practice. The indicators were mainly used to support applications for further funding and service development. The poor effect was attributable to a lack of professional belief in the indicators, arising from perceived problems around quality of data and time lag between collection and presentation of data; limited dissemination; weak incentives to take action; a predilection for process rather than outcome indicators; and a belief that informal information is often more useful than quantitative data in the assessment of clinical performance.

CONCLUSIONS

Those responsible for developing clinical indicator programmes should develop robust datasets. They should also encourage a working environment and incentives such that these data are used to improve continuously.

Publicly disclosed information about the quality of health care: response of the US public

Public disclosure of information about the quality of health plans, hospitals, and doctors continues to be controversial. The US experience of the past decade suggests that sophisticated quality measures and reporting systems that disclose information on quality have improved the process and outcomes of care in limited ways in some settings, but these efforts have not led to the "consumer choice" market envisaged. Important reasons for this failure include limited salience of objective measures to consumers, the complexity of the task of interpretation, and insufficient use of quality results by organised purchasers and insurers to inform contracting and pricing decisions. Nevertheless, public disclosure may motivate quality managers and providers to undertake changes that improve the delivery of care. Efforts to measure and report information about quality should remain public, but may be most effective if they are targeted to the needs of institutional and individual providers of care.