Public reporting of hospital quality: recommendations to benefit patients and hospitals

Tailoring an educational program on the AHRQ Patient Safety Indicators to meet stakeholder needs: lessons learned in the VA

BACKGROUND

Given that patient safety measures are increasingly used for public reporting and pay-for performance, it is important for stakeholders to understand how to use these measures for improvement. The Agency for Healthcare Research and Quality (AHRQ) Patient Safety Indicators (PSIs) are one particularly visible set of measures that are now used primarily for public reporting and pay-for-performance among both private sector and Veterans Health Administration (VA) hospitals. This trend generates a strong need for stakeholders to understand how to interpret and use the PSIs for quality improvement (QI). The goal of this study was to develop an educational program and tailor it to stakeholders' needs. In this paper, we share what we learned from this program development process.

METHODS

Our study population included key VA stakeholders involved in reviewing performance reports and prioritizing and initiating quality/safety initiatives. A pre-program formative evaluation through telephone interviews and web-based surveys assessed stakeholders' educational needs/interests. Findings from the formative evaluation led to development and implementation of a cyberseminar-based program, which we tailored to stakeholders' needs/interests. A post-program survey evaluated program participants' perceptions about the PSI educational program.

RESULTS

Interview data confirmed that the concepts we had developed for the interviews could be used for the survey. Survey results informed us on what program delivery mode and content topics were of high interest. Six cyberseminars were developed-three of which focused on two content areas that were noted of greatest interest: learning how to use PSIs for monitoring trends and understanding how to interpret PSIs. We also used snapshots of VA PSI reports so that participants could directly apply learnings. Although initial interest in the program was high, actual attendance was low. However, post-program survey results indicated that perceptions about the program were positive.

CONCLUSIONS

Conducting a formative evaluation was a highly important process in program development. The useful information that we collected through the interviews and surveys allowed us to tailor the program to stakeholders' needs and interests. Our experiences, particularly with the formative evaluation process, yielded valuable lessons that can guide others when developing and implementing similar educational programs.

The Nursing Home Compare Report Card: Perceptions of Residents and Caregivers Regarding Quality Ratings and Nursing Home Choice

OBJECTIVE

To evaluate the perceived usefulness of publicly reported nursing home quality indicators.

STUDY SETTING

Primary data were collected from October 2013 to August 2014 among a convenience sample of persons (or family member) recently admitted or anticipating admission to a nursing home within 75 miles of the city of Philadelphia.

STUDY DESIGN

Structured interviews were conducted to assess the salience of data on the Medicare Nursing Home Compare website, including star ratings, clinical quality measures, and benchmarking of individual nursing home quality with state and national data.

DATA COLLECTION

Interviews were transcribed verbatim, independently coded by two reviewers, and agreement determined. A thematic analysis of transcripts was undertaken.

PRINCIPAL FINDINGS

Thirty-five interviews were completed. Eighty-three percent (n = 29) were caregivers and 17 percent (n = 6) were residents. Star ratings, clinical quality measures, and benchmarking information were salient to decision making, with preferred formats varying across participants. Participants desired additional information on the source of quality data. Confusion was evident regarding the relationship between domain-specific and overall star quality ratings.

CONCLUSIONS

The Nursing Home Compare website provides salient content and formats for consumers. Increased awareness of this resource and clarity regarding the definition of measures could further support informed decision making regarding nursing home choice.

Information presentation features and comprehensibility of hospital report cards: design analysis and online survey among users

Background

Improving the transparency of information about the quality of health care providers is one way to improve health care quality. It is assumed that Internet information steers patients toward better-performing health care providers and will motivate providers to improve quality. However, the effect of public reporting on hospital quality is still small. One of the reasons is that users find it difficult to understand the formats in which information is presented.

Objective

We analyzed the presentation of risk-adjusted mortality rate (RAMR) for coronary angiography in the 10 most commonly used German public report cards to analyze the impact of information presentation features on their comprehensibility. We wanted to determine which information presentation features were utilized, were preferred by users, led to better comprehension, and had similar effects to those reported in evidence-based recommendations described in the literature.

Methods

The study consisted of 5 steps: (1) identification of best-practice evidence about the presentation of information on hospital report cards; (2) selection of a single risk-adjusted quality indicator; (3) selection of a sample of designs adopted by German public report cards; (4) identification of the information presentation elements used in public reporting initiatives in Germany; and (5) an online panel completed an online questionnaire that was conducted to determine if respondents were able to identify the hospital with the lowest RAMR and if respondents’ hospital choices were associated with particular information design elements.

Results

Evidence-based recommendations were made relating to the following information presentation features relevant to report cards: evaluative table with symbols, tables without symbols, bar charts, bar charts without symbols, bar charts with symbols, symbols, evaluative word labels, highlighting, order of providers, high values to indicate good performance, explicit statements of whether high or low values indicate good performance, and incomplete data (“N/A” as a value). When investigating the RAMR in a sample of 10 hospitals’ report cards, 7 of these information presentation features were identified. Of these, 5 information presentation features improved comprehensibility in a manner reported previously in literature.

Conclusions

To our knowledge, this is the first study to systematically analyze the most commonly used public reporting card designs used in Germany. Best-practice evidence identified in international literature was in agreement with 5 findings about German report card designs: (1) avoid tables without symbols, (2) include bar charts with symbols, (3) state explicitly whether high or low values indicate good performance or provide a “good quality” range, (4) avoid incomplete data (N/A given as a value), and (5) rank hospitals by performance. However, these findings are preliminary and should be subject of further evaluation. The implementation of 4 of these recommendations should not present insurmountable obstacles. However, ranking hospitals by performance may present substantial difficulties.

Comparative performance information plays no role in the referral behaviour of GPs

Background

Comparative performance information (CPI) about the quality of hospital care is information used to identify high-quality hospitals and providers. As the gatekeeper to secondary care, the general practitioner (GP) can use CPI to reflect on the pros and cons of the available options with the patient and choose a provider best fitted to the patient’s needs. We investigated how GPs view their role in using CPI to choose providers and support patients.

Method

We used a mixed-method, sequential, exploratory design to conduct explorative interviews with 15 GPs about their referral routines, methods of referral consideration, patient involvement, and the role of CPI. Then we quantified the qualitative results by sending a survey questionnaire to 81 GPs affiliated with a representative national research network.

Results

Seventy GPs (86% response rate) filled out the questionnaire. Most GPs did not know where to find CPI (87%) and had never searched for it (94%). The GPs reported that they were not motivated to use CPI due to doubts about its role as support information, uncertainty about the effect of using CPI, lack of faith in better outcomes, and uncertainty about CPI content and validity. Nonetheless, most GPs believed that patients would like to be informed about quality-of-care differences (62%), and about half the GPs discussed quality-of-care differences with their patients (46%), though these discussions were not based on CPI.

Conclusion

Decisions about referrals to hospital care are not based on CPI exchanges during GP consultations. As a gatekeeper, the GP is in a good position to guide patients through the enormous amount of quality information that is available. Nevertheless, it is unclear how and whether the GP’s role in using information about quality of care in the referral process can grow, as patients hardly ever initiate a discussion based on CPI, though they seem to be increasingly more critical about differences in quality of care. Future research should address the conditions needed to support GPs’ ability and willingness to use CPI to guide their patients in the referral process.

Understanding and using comparative healthcare information; the effect of the amount of information and consumer characteristics and skills

Background

Consumers are increasingly exposed to comparative healthcare information (information about the quality of different healthcare providers). Partly because of its complexity, the use of this information has been limited. The objective of this study was to examine how the amount of presented information influences the comprehension and use of comparative healthcare information when important consumer characteristics and skills are taken into account.

Methods

In this randomized controlled experiment, comparative information on total hip or knee surgery was used as a test case. An online survey was distributed among 800 members of the NIVEL Insurants Panel and 76 hip- or knee surgery patients. Participants were assigned to one of four subgroups, who were shown 3, 7, 11 or 15 quality aspects of three hospitals. We conducted Kruskall-Wallis tests, Chi-square tests and hierarchical multiple linear regression analyses to examine relationships between the amount of information and consumer characteristics and skills (literacy, numeracy, active choice behaviour) on one hand, and outcome measures related to effectively using information (comprehension, perceived usefulness of information, hospital choice, ease of making a choice) on the other hand.

Results

414 people (47%) participated. Regression analysis showed that the amount of information slightly influenced the comprehension and the perceived usefulness of comparative healthcare information. It did not affect consumers’ hospital choice and ease of making this choice. Consumer characteristics (especially age) and skills (especially literacy) were the most important factors affecting the comprehension of information and the ease of making a hospital choice. For the perceived usefulness of comparative information, active choice behaviour was the most influencing factor.

Conclusion

The effects of the amount of information were not unambiguous. It remains unclear what the ideal amount of quality information to be presented would be. Reducing the amount of information will probably not automatically result in more effective use of comparative healthcare information by consumers. More important, consumer characteristics and skills appeared to be more influential factors contributing to information comprehension and use. Consequently, we would suggest that more emphasis on improving consumers’ skills is needed to enhance the use of comparative healthcare information.

Primary care physicians' use of publicly reported quality data in hospital referral decisions

BACKGROUND

Despite government investment in public reporting of hospital quality data, patients still rely on recommendations from their primary care physicians (PCPs). Little is known about how physicians make hospital referrals.

OBJECTIVE

To characterize factors that influence PCPs' hospital referral choices.

DESIGN

Web-based physician survey.

PARTICIPANTS

PCPs affiliated with 3 Massachusetts hospitals.

MAIN MEASURES

Physician demographics, familiarity with public reporting, and opinions about which factors would influence hospital referral decisions for an elderly patient with pneumonia.

RESULTS

Of 194 PCPs who received invitations, 92 (47%) responded. Although 93% maintained admitting privileges, only 20% admitted patients. The following were considered "very" important in referral decisions: "familiarity with the hospital" (70%), "patient preference" (62%), and "admitting arrangements with a hospitalist group" (62%). "Publicly available quality measures" were "not at all" important to 42%. Only 61% were aware of hospital quality reporting; 16% were familiar with Hospital Compare, a Centers for Medicare and Medicaid Services (CMS) Web site. No physicians reported ever using quality information to make a referral decision or discussing it with patients. No physician factors were associated with awareness of publicly reported data. PCPs identified the following factors as being "very" important in determining the quality of pneumonia care: antibiotics within 6 hours of arrival (66%), appropriate initial antibiotic (63%), and blood cultures performed prior to the administration of antibiotics (51%).

CONCLUSIONS

Although PCPs most valued the information available through Hospital Compare, only 16% were aware of it, and none used publicly reported quality data in referral decisions. Medicare and high-performing hospitals should consider marketing Hospital Compare to PCPs.

Different patient subgroup, different ranking? Which quality indicators do patients find important when choosing a hospital for hip- or knee arthroplasty?

BACKGROUND

Patients are increasingly expected to become active, critical consumers in healthcare. They can use comparative healthcare information presented on websites to make informed choices for healthcare providers. However, the use of this information has been limited so far. An obstacle can be that the information is not perceived as relevant by patients. Presenting only the most important quality indicators might improve the usefulness of this information. The aim of this study was to explore which quality indicators different subgroups of patients find important when choosing a hospital for total hip arthroplasty (THA) or total knee arthroplasty (TKA).

METHODS

In this explorative, cross-sectional study, questionnaires were distributed to 265 patients who underwent or had to undergo THA/TKA. Participants were asked to rank the importance of three types of quality indicators: patient experience indicators, clinical performance indicators, and indicators about hospital services. We used random effects regression analyses to assess the relative importance of the indicators in different subgroups of patients.

RESULTS

110 patients (response rate 41.5%) who underwent or had to undergo THA/TKA participated. Conduct of doctors, the presence of procedures to prevent adverse effects of thrombosis and information about the specialist area of orthopaedists were the most important patient experience indicator, clinical performance indicator and indicator about hospital services, respectively. We found a few differences between patient subgroups in the importance attached to the quality indicators.

CONCLUSIONS

This study provides a first insight into which quality indicators patients find important when choosing a hospital for THA/TKA, and shows that subgroups of patients differ in the value they attach to these indicators. More extended research is needed to establish the indicators that should at least be presented in succinct overviews of comparative healthcare information for patients choosing a hospital for THA/TKA.