England's Experience incorporating "anecdotal" reports from consumers into their national reporting system: lessons for the United States of what to do or not to do?

Referring physicians' intention to use hospital report cards for hospital referral purposes in the presence or absence of patient-reported outcomes: a randomized trial

PURPOSE

This study aims to determine the intention to use hospital report cards (HRCs) for hospital referral purposes in the presence or absence of patient-reported outcomes (PROs) as well as to explore the relevance of publicly available hospital performance information from the perspective of referring physicians.

METHODS

We identified the most relevant information for hospital referral purposes based on a literature review and qualitative research. Primary survey data were collected (May-June 2021) on a sample of 591 referring orthopedists in Germany and analyzed using structural equation modeling. Participating orthopedists were recruited using a sequential mixed-mode strategy and randomly allocated to work with HRCs in the presence (intervention) or absence (control) of PROs.

RESULTS

Overall, 420 orthopedists (mean age 53.48, SD 8.04) were included in the analysis. The presence of PROs on HRCs was not associated with an increased intention to use HRCs (p = 0.316). Performance expectancy was shown to be the most important determinant for using HRCs (path coefficient: 0.387, p < .001). However, referring physicians have doubts as to whether HRCs can help them. We identified "complication rate" and "the number of cases treated" as most important for the hospital referral decision making; PROs were rated slightly less important.

CONCLUSIONS

This study underpins the purpose of HRCs, namely to support referring physicians in searching for a hospital. Nevertheless, only a minority would support the use of HRCs for the next hospital search in its current form. We showed that presenting relevant information on HRCs did not increase their use intention.

A participatory action research study to inform combined type 2 diabetes and chronic kidney disease care provided in the context of advanced practice nursing

AIM

To outline the development, implementation and evaluation of an alternative combined type 2 diabetes and chronic kidney disease (CKD) care delivery strategy provided in the context of advanced practice nursing informed by co-created knowledge, between patients and healthcare teams to address identified gaps in care provision.

DESIGN

This study was informed by the theoretical underpinnings of critical theory and operationalized using a participatory action research (PAR) design. This approach enabled alignment between the theoretical underpinnings and research methodology to respond to practice-level problems and address issues related to healthcare delivery.

METHODS

The study was undertaken in a series of three cycles of enquiry over a 20 month period (July 2018-February 2020) in five geographically diverse specialist diabetes and nephrology centres. Data collection methods included PAR group minute consensus, retrospective chart review, discovery and focus group interviews. Thematic analysis and descriptive statistics were used to analyse the data.

RESULTS

Compared with international standard care outcomes, a higher proportion of patients attending combined care achieved best practice recommendations for blood pressure, Hba1c and lipid management (64.3% vs. <40%). Patients attending combined care experienced improved detection and management of CKD-related anaemia. Additionally, the combined care service experienced significantly lower levels of patient's failure to attend scheduled reviews compared with participating standard care services (0% vs. 7%-23.3%).

CONCLUSIONS

Best practice clinical care outcomes and patient attendance improved when a combined type 2 diabetes and CKD care delivery strategy provided in the context of advanced practice nursing was adopted.

IMPACT

This alternative care strategy represented a departure from standard care delivery approaches, in which care for type 2 diabetes and CKD is provided separately and led by physicians. Outcomes demonstrate improved clinical and attendance outcomes which may inform the development of future services led by advanced nurse practitioners.

Testing the effects on information use by older versus younger women of modality and narration style in a hospital report card

Background

Hospital report cards (HRCs) are usually presented in a textual and factual format, likely hampering information processing.

Objective

This study aimed to investigate the effects of audiovisual and narrative information in HRCs on user responses, and to test differences between older and younger women.

Design

A 2 (modality [textual vs. audiovisual]) × 3 (narration style [factual vs. process narrative vs. experience narrative]) online experiment was conducted. Information about breast cancer care was used as a case example. Age (younger [<65] vs. older [≥65]) was included as a potential effect modifier.

Setting and Participants

A total of 631 disease‐naïve women (M_age = 56.06) completed an online survey. The outcomes were perceived cognitive load, satisfaction, comprehension, information recall and decisional conflict. Data were analysed using AN(C)OVAs.

Results

Audiovisual (vs. textual) information resulted in higher information satisfaction across age groups, but was associated with lower comprehension in older women. An experience narrative (vs. factual information) increased satisfaction with attractiveness and emotional support of the information only in older women. A three‐way interaction effect was found, suggesting that older women were most satisfied with the comprehensibility of audiovisual factual or textual process narrative information. Younger women were most satisfied with the comprehensibility of audiovisual process narrative or textual factual information.

Discussion and Conclusion

Audiovisual and narrative information in an HRC showed beneficial effects on satisfaction measures. In particular, audiovisual information could be incorporated into HRCs to increase satisfaction with information.

Public Contribution

Lay persons helped in optimizing the visuals used in the stimulus materials by checking for clarity.

Characteristics and decision making of hospital report card consumers: Lessons from an onsite-based cross-sectional study

BACKGROUND

Hospitals report cards (HRCs) have had little impact on the hospital choice of patients. Thus, health policy makers should learn more about HRC consumers to better understand how to present and target hospital-related quality information.

OBJECTIVE

We sought to learn more about consumers of HRCs and determine the impact of the complexity and tailoring of HRCs on the hospital choice.

METHODS

We used primary data drawn from an onsite-based survey, conducted in 2017 at Germany's premier portal, Weisse Liste (N = 635). We performed hierarchical multivariate logistic regression models to identify main predictors associated with hospital choices.

RESULTS

HRC consumers differ from the national online population and the national population in general. Eighty percent of those patients or family members, who have used a HRC before, confirmed an impact on the hospital choice. The quality of hospital choices decreased with an increasing level of complexity (p < .001); the latter was identified as a significant predictor for making good choices. However, tailoring HRCs did not have an impact on the quality of the hospital choice (p > .05).

CONCLUSIONS

HRCs have a significant impact on the hospital choice among report card consumers. Health policy makers might focus on decreasing the level of complexity; this, more than tailoring report cards, may help consumers make good hospital choices.

What Do Patients Complain About Online: A Systematic Review and Taxonomy Framework Based on Patient Centeredness

Background

Complaints made online by patients about their health care experiences are becoming prevalent because of widespread worldwide internet connectivity. An a priori framework, based on patient centeredness, may be useful in identifying the types of issues patients complain about online across multiple settings. It may also assist in examining whether the determinants of patient-centered care (PCC) mirror the determinants of patient experiences.

Objective

The objective of our study was to develop a taxonomy framework for patient complaints online based on patient centeredness and to examine whether the determinants of PCC mirror the determinants of patient experiences.

Methods

First, the best fit framework synthesis technique was applied to develop the proposed a priori framework. Second, electronic databases, including Web of Science, Scopus, and PubMed, were searched for articles published between 2000 and June 2018. Studies were only included if they collected primary quantitative data on patients’ online complaints. Third, a deductive and inductive thematic analysis approach was adopted to code the themes of recognized complaints into the framework.

Results

In total, 17 studies from 5 countries were included in this study. Patient complaint online taxonomies and theme terms varied. According to our framework, patients expressed most dissatisfaction with patient-centered processes (101,586/204,363, 49.71%), followed by prerequisites (appropriate skills and knowledge of physicians; 50,563, 24.74%) and the care environment (48,563/204,363, 23.76%). The least dissatisfied theme was expected outcomes (3651/204,363, 1.79%). People expressed little dissatisfaction with expanded PCC dimensions, such as involvement of family and friends (591/204,363, 0.29%). Variation in the concerns across different countries’ patients were also observed.

Conclusions

Online complaints made by patients are of major value to health care providers, regulatory bodies, and patients themselves. Our PCC framework can be applied to analyze them under a wide range of conditions, treatments, and countries. This review has shown significant heterogeneity of patients’ online complaints across different countries.

Patterns of performance and improvement in US Medicare's Hospital Star Ratings, 2016-2017

BACKGROUND

Publicly reported quality data can help consumers make informed choices about where to seek medical care. The Centers for Medicare and Medicaid Services developed a composite Hospital Compare Overall Star Rating for US acute-care hospitals in 2016. However, patterns of performance and improvement have not been previously described.

OBJECTIVE

To characterise high-quality and low-quality hospitals as assessed by Star Ratings.

DESIGN

We performed a retrospective cross-sectional study of 3429 US acute-care hospitals assigned Overall Star Ratings in both 2016 and 2017. We used multivariable logistic regression models to identify characteristics associated with receiving 4 or 5 stars.

RESULTS

Small hospitals were more likely to receive 4 or 5 stars in 2016 (33% of small hospitals, 26% of medium hospitals and 21% of large hospitals, OR for medium 0.78, p=0.02, and for large, 0.61, p=0.003). Non-profit status (OR 1.37, p=0.01), midwest region (OR=2.30, p<0.001), west region (OR 1.30 in 2016, p=0.06) and system membership (OR 1.33, p=0.003) were associated with higher odds of achieving a higher Star Rating. Hospitals with the most Medicaid patients were markedly less likely to receive 4 or 5 stars (OR for highest quartile=0.32, p<0.001), and hospitals with the highest proportion of Medicare patients were somewhat less likely to do so (OR for highest quartile=0.68, p=0.01). These associations remained largely consistent over the first two years of reporting and were also associated with the highest likelihood of improvement.

CONCLUSIONS

Small hospitals with fewer Medicaid patients had the highest odds of performing well on Star Ratings. Further monitoring of these trends is needed as patients, clinicians and policymakers strive to use this information to promote high-quality care.

A Rigorous Approach to Large-Scale Elicitation and Analysis of Patient Narratives

Patient narratives have emerged as promising vehicles for making health care more responsive by helping clinicians to better understand their patients' expectations, perceptions, or concerns and encouraging consumers to engage with information about quality. A growing number of websites incorporate patients' comments. But existing comments have fragmentary content, fail to represent less vocal patients, and can be manipulated to "manage" providers' reputations. In this article, we offer the first empirical test of the proposition that patient narratives can be elicited rigorously and reliably using a five-question protocol that can be incorporated into large-scale patient experience surveys. We tested whether elicited narratives about outpatient care are complete (report all facets of patient experience), balanced (convey an accurate mix of positive and negative events), meaningful (have a coherent storyline), and representative (draw fulsome narratives from all relevant subsets of patients). The tested protocol is strong on balance and representativeness, more mixed on completeness and meaningfulness.

Public Awareness, Usage, and Predictors for the Use of Doctor Rating Websites: Cross-Sectional Study in England

BACKGROUND

With the advent and popularity of social media and consumer rating websites, as well as the emergence of the digitally engaged patient, there has been an increased interest in doctor rating websites or online patient feedback websites, both inside and outside academia. However, there is very little known about how the public across England views such rating websites as a mode to give patient experience feedback.

OBJECTIVE

The aim of the overall study was to measure and understand public awareness, usage, and attitudes towards doctor rating websites as a mode to give experiential feedback about GPs in general practice in England. This paper reports on the findings of one of the aims of the study, which was to measure public awareness, current usage and future consideration of usage of online patient feedback websites, within the context of other feedback methods, This could allow the value of online patient feedback websites to be determined from the patients' perspective.

METHODS

A mixed methods population questionnaire was designed, validated and implemented face-to-face using a cross-sectional design with a representative sample of the public (n=844) in England. The results of the questionnaire were analyzed using chi-square tests, binomial logistic regressions, and content analysis. The qualitative results will be reported elsewhere.

RESULTS

Public awareness of online patient feedback websites as a channel to leave experiential feedback about GPs was found to be low at 15.2% (128/844). However, usage and future consideration to use online patient feedback websites were found to be extremely low, with current patient usage at just 0.4% (3/844), and patient intention to use online patient feedback in the future at 17.8% (150/844). Furthermore, only 4.0-5.0% of those who would consider leaving feedback about a GP in the future selected doctor rating websites as their most preferred method; more than half of patients said they would consider leaving feedback about GPs using another method, but not using an online patient feedback website.

CONCLUSIONS

The findings suggest that online patient feedback websites may not be an effective channel for collecting feedback on patient experience in general practice. Feedback on online patient feedback websites is not likely to be representative of the patient experience in the near future, challenging the use of online patient feedback not just as a mode for collecting patient experience data, but for patient choice and monitoring too. We recommend the National Health Service channels its investment and resources towards providing more direct and private feedback methods in general practice (such as opportunities for face-to-face feedback, email-based feedback, and web-based private feedback forms), as these are currently much more likely to be used by the majority of patients in England.

A cross-sectional study assessing the association between online ratings and clinical quality of care measures for US hospitals: results from an observational study

Background

Little is known about the usefulness of online ratings when searching for a hospital. We therefore assess the association between quantitative and qualitative online ratings for US hospitals and clinical quality of care measures.

Methods

First, we collected a stratified random sample of 1000 quantitative and qualitative online ratings for hospitals from the website RateMDs. We used an integrated iterative approach to develop a categorization scheme to capture both the topics and sentiment in the narrative comments. Next, we matched the online ratings with hospital-level quality measures published by the Centers for Medicare and Medicaid Services. Regarding nominally scaled measures, we checked for differences in the distribution among the online rating categories. For metrically scaled measures, we applied the Spearman rank coefficient of correlation.

Results

Thirteen of the twenty-nine quality of care measures were significantly associated with the quantitative online ratings (Spearman p = ±0.143, p < 0.05 for all). Thereof, eight associations indicated better clinical outcomes for better online ratings. Seven of the twenty-nine clinical measures were significantly associated with the sentiment of patient narratives (p = ±0.114, p < 0.05 for all), whereof four associations indicated worse clinical outcomes in more favorable narrative comments.

Conclusions

There seems to be some association between quantitative online ratings and clinical performance measures. However, the relatively weak strength and inconsistency of the direction of the association as well as the lack of association with several other clinical measures may not enable the drawing of strong conclusions. Narrative comments also seem to have limited potential to reflect the clinical quality of care in its current form. Thus, online ratings are of limited usefulness in guiding patients towards high-performing hospitals from a clinical point of view. Nevertheless, patients might prefer different aspects of care when choosing a hospital.

Electronic supplementary material

The online version of this article (10.1186/s12913-018-2886-3) contains supplementary material, which is available to authorized users.

Are Facebook user ratings associated with hospital cost, quality and patient satisfaction? A cross-sectional analysis of hospitals in New York State

BACKGROUND

Hospital care costs are high while quality varies across hospitals. Patient satisfaction may be associated with better clinical quality, and social media ratings may offer another opportunity to measure patient satisfaction with care.

OBJECTIVES

To test if Facebook user ratings of hospitals are associated with existing measures of patient satisfaction, cost and quality.

RESEARCH DESIGN

Data were obtained from Centers for Medicare and Medicaid Services Hospital Compare, the Hospital Inpatient Prospective Payment System impact files and the Area Health Resource File for 2015. Information from hospitals' Facebook pages was collected in July 2016. Multivariate linear regression was used to test if there is an association between Facebook user ratings (star rating and adjusted number of 'likes') and Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) patient satisfaction measures, the 30-day all-cause readmission rate, and the Medicare spending per beneficiary (MSPB) ratio.

SUBJECTS

One hundred and thirty-six acute care hospitals in New York State in 2015.

RESULTS

An increase in the Facebook star rating is associated with significant increases in 21/23 HCAHPS measures (p≤0.003). An increase in the adjusted number of 'likes' is associated with very small increases in 3/23 HCAHPS measures (p<0.05). Facebook user ratings are not associated with the 30-day all-cause readmission rate or the Medicare spending per beneficiary ratio.

CONCLUSIONS

Results demonstrate an association between HCAHPS patient satisfaction measures and Facebook star ratings. Adjusted number of 'likes' may not be a useful measure of patient satisfaction.

Perceived barriers to effective implementation of public reporting of hospital performance data in Australia: a qualitative study

Background

Public reporting of government funded (public) hospital performance data was mandated in Australia in 2011. Studies suggest some benefit associated with such public reporting, but also considerable scope to improve reporting systems.

Methods

In 2015, a purposive sample of 41 expert informants were interviewed, representing consumer, provider and purchasers perspectives across Australia’s public and private health sectors, to ascertain expert opinion on the utility and impact of public reporting of health service performance. Qualitative data was thematically analysed with a focus on reporting perceived strengths and barriers to public reporting of hospital performance data (PR).

Results

Many more weaknesses and barriers to PR were identified than strengths. Barriers were: conceptual (unclear objective, audience and reporting framework); systems-level (including lack of consumer choice, lack of consumer and clinician involvement, jurisdictional barriers, lack of mandate for private sector reporting); technical and resource related (including data complexity, lack of data relevance consistency, rigour); and socio-cultural (including provider resistance to public reporting, poor consumer health literacy, lack of consumer empowerment).

Conclusions

Perceptions of the Australian experience of PR highlight important issues in its implementation that can provide lessons for Australia and elsewhere. A considerable weakness of PR in Australia is that the public are often not considered its major audience, resulting in information ineffectually framed to meet the objective of PR informing consumer decision-making about treatment options. Greater alignment is needed between the primary objective of PR, its audience and audience needs; more than one system of PR might be necessary to meet different audience needs and objectives. Further research is required to assess objectively the potency of the barriers to PR suggested by our panel of informants.

Electronic supplementary material

The online version of this article (doi:10.1186/s12913-017-2336-7) contains supplementary material, which is available to authorized users.

Hospital Quality Reporting in the United States: Does Report Card Design and Incorporation of Patient Narrative Comments Affect Hospital Choice?

OBJECTIVE

To explore the impact of hospital report card design and incorporation of patient narrative comments on consumers' choices of hospitals.

DATA SOURCES

Primary data collected from an online survey with 1,350 respondents in February, 2015.

STUDY DESIGN

A randomized 2 (narrative comments: yes, no) × 3 (design: representation of clinical performance in textual, star, numerical formats) between-subject online-based cross-sectional experiment.

PRINCIPAL FINDINGS

In 51 percent of all cases, respondents selected the hospital with the best clinical results. Report cards with a numerical design induced choices more focused on clinical ratings (56.0 percent chose the highest rated hospital) than those with textual information (48.1 percent) or star ratings (47.3 percent) (p < .001). Report cards without narrative comments (49.7 percent) and with narratives (51.4 percent) were not associated with significant difference in selecting top-rated clinical hospitals (p = .376). But there were significant interactions affecting choice of hospitals among exposure to narratives, formatting of clinical performance, and respondents' education.

CONCLUSIONS

Consumers have a difficult time synthesizing quality data in various formats. Hospital report cards continue to pose challenging choices, especially for those with limited education. Narrative comments in their earliest emerging forms do not seem to be altering hospital choice as much as the literature has suggested for other providers, but they may have consequential impact on the choices of certain subsets of consumers.

Improving patient experience in primary care: a multimethod programme of research on the measurement and improvement of patient experience

Background

There has been an increased focus towards improving quality of care within the NHS in the last 15 years; as part of this, there has been an emphasis on the importance of patient feedback within policy, through National Service Frameworks and the Quality and Outcomes Framework. The development and administration of large-scale national patient surveys to gather representative data on patient experience, such as the national GP Patient Survey in primary care, has been one such initiative. However, it remains unclear how the survey is used by patients and what impact the data may have on practice.

Objectives

Our research aimed to gain insight into how different patients use surveys to record experiences of general practice; how primary care staff respond to feedback; and how to engage primary care staff in responding to feedback.

Methods

We used methods including quantitative survey analyses, focus groups, interviews, an exploratory trial and an experimental vignette study.

Results

(1)Understanding patient experience data. Patients readily criticised their care when reviewing consultations on video, although they were reluctant to be critical when completing questionnaires. When trained raters judged communication during a consultation to be poor, a substantial proportion of patients rated the doctor as ‘good’ or ‘very good’. Absolute scores on questionnaire surveys should be treated with caution; they may present an overoptimistic view of general practitioner (GP) care. However, relative rankings to identify GPs who are better or poorer at communicating may be acceptable, as long as statistically reliable figures are obtained. Most patients have a particular GP whom they prefer to see; however, up to 40% of people who have such a preference are unable regularly to see the doctor of their choice. Users of out-of-hours care reported worse experiences when the service was run by a commercial provider than when it was run by a not-for profit or NHS provider. (2)Understanding patient experience in minority ethnic groups. Asian respondents to the GP Patient Survey tend to be registered with practices with generally low scores, explaining about half of the difference in the poorer reported experiences of South Asian patients than white British patients. We found no evidence that South Asian patients used response scales differently. When viewing the same consultation in an experimental vignette study, South Asian respondents gave higher scores than white British respondents. This suggests that the low scores given by South Asian respondents in patient experience surveys reflect care that is genuinely worse than that experienced by their white British counterparts. We also found that service users of mixed or Asian ethnicity reported lower scores than white respondents when rating out-of-hours services. (3)Using patient experience data. We found that measuring GP–patient communication at practice level masks variation between how good individual doctors are within a practice. In general practices and in out-of-hours centres, staff were sceptical about the value of patient surveys and their ability to support service reconfiguration and quality improvement. In both settings, surveys were deemed necessary but not sufficient. Staff expressed a preference for free-text comments, as these provided more tangible, actionable data. An exploratory trial of real-time feedback (RTF) found that only 2.5% of consulting patients left feedback using touch screens in the waiting room, although more did so when reminded by staff. The representativeness of responding patients remains to be evaluated. Staff were broadly positive about using RTF, and practices valued the ability to include their own questions. Staff benefited from having a facilitated session and protected time to discuss patient feedback.

Conclusions

Our findings demonstrate the importance of patient experience feedback as a means of informing NHS care, and confirm that surveys are a valuable resource for monitoring national trends in quality of care. However, surveys may be insufficient in themselves to fully capture patient feedback, and in practice GPs rarely used the results of surveys for quality improvement. The impact of patient surveys appears to be limited and effort should be invested in making the results of surveys more meaningful to practice staff. There were several limitations of this programme of research. Practice recruitment for our in-hours studies took place in two broad geographical areas, which may not be fully representative of practices nationally. Our focus was on patient experience in primary care; secondary care settings may face different challenges in implementing quality improvement initiatives driven by patient feedback. Recommendations for future research include consideration of alternative feedback methods to better support patients to identify poor care; investigation into the factors driving poorer experiences of communication in South Asian patient groups; further investigation of how best to deliver patient feedback to clinicians to engage them and to foster quality improvement; and further research to support the development and implementation of interventions aiming to improve care when deficiencies in patient experience of care are identified.

Funding

The National Institute for Health Research Programme Grants for Applied Research programme.

Do Health Care Providers Use Online Patient Ratings to Improve the Quality of Care? Results From an Online-Based Cross-Sectional Study

Background

Physician-rating websites have become a popular tool to create more transparency about the quality of health care providers. So far, it remains unknown whether online-based rating websites have the potential to contribute to a better standard of care.

Objective

Our goal was to examine which health care providers use online rating websites and for what purposes, and whether health care providers use online patient ratings to improve patient care.

Methods

We conducted an online-based cross-sectional study by surveying 2360 physicians and other health care providers (September 2015). In addition to descriptive statistics, we performed multilevel logistic regression models to ascertain the effects of providers’ demographics as well as report card-related variables on the likelihood that providers implement measures to improve patient care.

Results

Overall, more than half of the responding providers surveyed (54.66%, 1290/2360) used online ratings to derive measures to improve patient care (implemented measures: mean 3.06, SD 2.29). Ophthalmologists (68%, 40/59) and gynecologists (65.4%, 123/188) were most likely to implement any measures. The most widely implemented quality measures were related to communication with patients (28.77%, 679/2360), the appointment scheduling process (23.60%, 557/2360), and office workflow (21.23%, 501/2360). Scaled-survey results had a greater impact on deriving measures than narrative comments. Multilevel logistic regression models revealed medical specialty, the frequency of report card use, and the appraisal of the trustworthiness of scaled-survey ratings to be significantly associated predictors for implementing measures to improve patient care because of online ratings.

Conclusions

Our results suggest that online ratings displayed on physician-rating websites have an impact on patient care. Despite the limitations of our study and unintended consequences of physician-rating websites, they still may have the potential to improve patient care.

Exploring Patients' Views Toward Giving Web-Based Feedback and Ratings to General Practitioners in England: A Qualitative Descriptive Study

Background

Patient feedback websites or doctor rating websites are increasingly being used by patients to give feedback about their health care experiences. There is little known about why patients in England may give Web-based feedback and what may motivate or dissuade them from giving Web-based feedback.

Objective

The aim of this study was to explore patients’ views toward giving Web-based feedback and ratings to general practitioners (GPs), within the context of other feedback methods available in primary care in England, and in particular, paper-based feedback cards.

Methods

A descriptive exploratory qualitative approach using face-to-face semistructured interviews was used in this study. Purposive sampling was used to recruit 18 participants from different age groups in London and Coventry. Interviews were transcribed verbatim and analyzed using applied thematic analysis.

Results

Half of the participants in this study were not aware of the opportunity to leave feedback for GPs, and there was limited awareness about the methods available to leave feedback for a GP. The majority of participants were not convinced that formal patient feedback was needed by GPs or would be used by GPs for improvement, regardless of whether they gave it via a website or on paper. Some participants said or suggested that they may leave feedback on a website rather than on a paper-based feedback card for several reasons: because of the ability and ease of giving it remotely; because it would be shared with the public; and because it would be taken more seriously by GPs. Others, however, suggested that they would not use a website to leave feedback for the opposite reasons: because of accessibility issues; privacy and security concerns; and because they felt feedback left on a website may be ignored.

Conclusions

Patient feedback and rating websites as they currently are will not replace other mechanisms for patients in England to leave feedback for a GP. Rather, they may motivate a small number of patients who have more altruistic motives or wish to place collective pressure on a GP to give Web-based feedback. If the National Health Service or GP practices want more patients to leave Web-based feedback, we suggest they first make patients aware that they can leave anonymous feedback securely on a website for a GP. They can then convince them that their feedback is needed and wanted by GPs for improvement, and that the reviews they leave on the website will be of benefit to other patients to decide which GP to see or which GP practice to join.

Breaking Narrative Ground: Innovative Methods for Rigorously Eliciting and Assessing Patient Narratives

OBJECTIVE

To design a methodology for rigorously eliciting narratives about patients' experiences with clinical care that is potentially useful for public reporting and quality improvement.

DATA SOURCES/STUDY SETTING

Two rounds of experimental data (N = 48 each) collected in 2013-2014, using a nationally representative Internet panel.

STUDY DESIGN

Our study (1) articulates and operationalizes criteria for assessing narrative elicitation protocols; (2) establishes a "gold standard" for assessment of such protocols; and (3) creates and tests a protocol for narratives about outpatient treatment experiences.

DATA COLLECTION/EXTRACTION METHODS

We randomized participants between telephone and web-based modalities and between protocols placed before and after a closed-ended survey.

PRINCIPAL FINDINGS

Elicited narratives can be assessed relative to a gold standard using four criteria: (1) meaningfulness, (2) completeness, (3) whether the narrative accurately reflects the balance of positive and negative events, and (4) representativeness, which reflects the protocol's performance across respondent subgroups. We demonstrate that a five-question protocol that has been tested and refined yields three- to sixfold increases in completeness and four- to tenfold increases in meaningfulness, compared to a single open-ended question. It performs equally well for healthy and sick patients.

CONCLUSIONS

Narrative elicitation protocols suitable for inclusion in extant patient experience surveys can be designed and tested against objective performance criteria, thus advancing the science of public reporting.

From the closest observers of patient care: a thematic analysis of online narrative reviews of hospitals

OBJECTIVE

Patient-centred care has become a priority in many countries. It is unknown whether current tools capture aspects of care patients and their surrogates consider important. We investigated whether online narrative reviews from patients and surrogates reflect domains in the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) and we described additional potential domains.

DESIGN

We used thematic analysis to assess online narrative reviews for reference to HCAHPS domains and salient non-HCAHPS domains and compared results by reviewer type (patient vs surrogate).

SETTING

We identified hospitals for review from the American Hospital Association database using a stratified random sampling approach. This approach ensured inclusion of reviews of a diverse set of hospitals. We searched online in February 2013 for narrative reviews from any source for each hospital.

PARTICIPANTS

We included up to two narrative reviews for each hospital.

EXCLUSIONS

Outpatient or emergency department reviews, reviews from self-identified hospital employees, or reviews of <10 words.

RESULTS

50.0% (n=122) of reviews (N=244) were from patients and 38.1% (n=93) from friends or family members. Only 57.0% (n=139) of reviews mentioned any HCAHPS domain. Additional salient domains were: Financing, including unexpected out-of-pocket costs and difficult interactions with billing departments; system-centred care; and perceptions of safety. These domains were mentioned in 51.2% (n=125) of reviews. Friends and family members commented on perceptions of safety more frequently than patients.

CONCLUSIONS

A substantial proportion of consumer reviews do not mention HCAHPS domains. Surrogates appear to observe care differently than patients, particularly around safety.

General Practitioners' Concerns About Online Patient Feedback: Findings From a Descriptive Exploratory Qualitative Study in England

Background

The growth in the volume of online patient feedback, including online patient ratings and comments, suggests that patients are embracing the opportunity to review online their experience of receiving health care. Very little is known about health care professionals’ attitudes toward online patient feedback and whether health care professionals are comfortable with the public nature of the feedback.

Objective

The aim of the overall study was to explore and describe general practitioners’ attitudes toward online patient feedback. This paper reports on the findings of one of the aims of the study, which was to explore and understand the concerns that general practitioners (GPs) in England have about online patient feedback. This could then be used to improve online patient feedback platforms and help to increase usage of online patient feedback by GPs and, by extension, their patients.

Methods

A descriptive qualitative approach using face-to-face semistructured interviews was used in this study. A topic guide was developed following a literature review and discussions with key stakeholders. GPs (N=20) were recruited from Cambridgeshire, London, and Northwest England through probability and snowball sampling. Interviews were transcribed verbatim and analyzed in NVivo using the framework method, a form of thematic analysis.

Results

Most participants in this study had concerns about online patient feedback. They questioned the validity of online patient feedback because of data and user biases and lack of representativeness, the usability of online patient feedback due to the feedback being anonymous, the transparency of online patient feedback because of the risk of false allegations and breaching confidentiality, and the resulting impact of all those factors on them, their professional practice, and their relationship with their patients.

Conclusions

The majority of GPs interviewed had reservations and concerns about online patient feedback and questioned its validity and usefulness among other things. Based on the findings from the study, recommendations for online patient feedback website providers in England are given. These include suggestions to make some specific changes to the platform and the need to promote online patient feedback more among both GPs and health care users, which may help to reduce some of the concerns raised by GPs about online patient feedback in this study.

Using Patient-Reported Information to Improve Clinical Practice

OBJECTIVE

To assess what is known about the relationship between patient experience measures and incentives designed to improve care, and to identify how public policy and medical practices can promote patient-valued outcomes in health systems with strong financial incentives.

DATA SOURCES/STUDY SETTING

Existing literature (gray and peer-reviewed) on measuring patient experience and patient-reported outcomes, identified from Medline and Cochrane databases; evaluations of pay-for-performance programs in the United States, Europe, and the Commonwealth countries.

STUDY DESIGN/DATA COLLECTION

We analyzed (1) studies of pay-for-performance, to identify those including metrics for patient experience, and (2) studies of patient experience and of patient-reported outcomes to identify evidence of influence on clinical practice, whether through public reporting or private reporting to clinicians.

PRINCIPAL FINDINGS

First, we identify four forms of "patient-reported information" (PRI), each with distinctive roles shaping clinical practice: (1) patient-reported outcomes measuring self-assessed physical and mental well-being, (2) surveys of patient experience with clinicians and staff, (3) narrative accounts describing encounters with clinicians in patients' own words, and (4) complaints/grievances signaling patients' distress when treatment or outcomes fall short of expectations. Because these forms vary in crucial ways, each must be distinctively measured, deployed, and linked with financial incentives. Second, although the literature linking incentives to patients experience is limited, implementing pay-for-performance systems appears to threaten certain patient-valued aspects of health care. But incentives can be made compatible with the outcomes patients value if: (a) a sufficient portion of incentives is tied to patient-reported outcomes and experiences, (b) incentivized forms of PRI are complemented by other forms of patient feedback, and (c) health care organizations assist clinicians to interpret and respond to PRI. Finally, we identify roles for the public and private sectors in financing PRI and orchestrating an appropriate balance among its four forms.

CONCLUSIONS

Unless public policies are attentive to patients' perspectives, stronger financial incentives for clinicians can threaten aspects of care that patients most value. Certain policy parameters are already clear, but additional research is required to clarify how best to collect patient narratives in varied settings, how to report narratives to consumers in conjunction with quantified metrics, and how to promote a "culture of learning" at the practice level that incorporates patient feedback.

How Financial and Reputational Incentives Can Be Used to Improve Medical Care

OBJECTIVES

Narrative review of the impact of pay-for-performance (P4P) and public reporting (PR) on health care outcomes, including spillover effects and impact on disparities.

PRINCIPAL FINDINGS

The impact of P4P and PR is dependent on the underlying payment system (fee-for-service, salary, capitation) into which these schemes are introduced. Both have the potential to improve care, but they can also have substantial unintended consequences. Evidence from the behavioral economics literature suggests that individual physicians will vary in how they respond to incentives. We also discuss issues to be considered when including patient-reported outcome measures (PROMs) or patient-reported experience measures into P4P and PR schemes.

CONCLUSION

We provide guidance to payers and policy makers on the design of P4P and PR programs so as to maximize their benefits and minimize their unintended consequences. These include involving clinicians in the design of the program, taking into account the payment system into which new incentives are introduced, designing the structure of reward programs to maximize the likelihood of intended outcomes and minimize the likelihood of unintended consequences, designing schemes that minimize the risk of increasing disparities, providing stability of incentives over some years, and including outcomes that are relevant to patients' priorities. In addition, because of the limitations of PR and P4P as effective interventions in their own right, it is important that they are combined with other policies and interventions intended to improve quality to maximize their likely impact.

A cross-sectional study assessing the association between online ratings and structural and quality of care measures: results from two German physician rating websites

Background

Even though physician rating websites (PRWs) have been gaining in importance in both practice and research, little evidence is available on the association of patients’ online ratings with the quality of care of physicians. It thus remains unclear whether patients should rely on these ratings when selecting a physician. The objective of this study was to measure the association between online ratings and structural and quality of care measures for 65 physician practices from the German Integrated Health Care Network “Quality and Efficiency” (QuE).

Methods

Online reviews from two German PRWs were included which covered a three-year period (2011 to 2013) and included 1179 and 991 ratings, respectively. Information for 65 QuE practices was obtained for the year 2012 and included 21 measures related to structural information (N = 6), process quality (N = 10), intermediate outcomes (N = 2), patient satisfaction (N = 1), and costs (N = 2). The Spearman rank coefficient of correlation was applied to measure the association between ratings and practice-related information.

Results

Patient satisfaction results from offline surveys and the patients per doctor ratio in a practice were shown to be significantly associated with online ratings on both PRWs. For one PRW, additional significant associations could be shown between online ratings and cost-related measures for medication, preventative examinations, and one diabetes type 2-related intermediate outcome measure. There again, results from the second PRW showed significant associations with the age of the physicians and the number of patients per practice, four process-related quality measures for diabetes type 2 and asthma, and one cost-related measure for medication.

Conclusions

Several significant associations were found which varied between the PRWs. Patients interested in the satisfaction of other patients with a physician might select a physician on the basis of online ratings. Even though our results indicate associations with some diabetes and asthma measures, but not with coronary heart disease measures, there is still insufficient evidence to draw strong conclusions. The limited number of practices in our study may have weakened our findings.

Electronic supplementary material

The online version of this article (doi:10.1186/s12913-015-1051-5) contains supplementary material, which is available to authorized users.

Sources of traffic and visitors' preferences regarding online public reports of quality: web analytics and online survey results

Background

In the context of the Affordable Care Act, there is extensive emphasis on making provider quality transparent and publicly available. Online public reports of quality exist, but little is known about how visitors find reports or about their purpose in visiting.

Objective

To address this gap, we gathered website analytics data from a national group of online public reports of hospital or physician quality and surveyed real-time visitors to those websites.

Methods

Websites were recruited from a national group of online public reports of hospital or physician quality. Analytics data were gathered from each website: number of unique visitors, method of arrival for each unique visitor, and search terms resulting in visits. Depending on the website, a survey invitation was launched for unique visitors on landing pages or on pages with quality information. Survey topics included type of respondent (eg, consumer, health care professional), purpose of visit, areas of interest, website experience, and demographics.

Results

There were 116,657 unique visitors to the 18 participating websites (1440 unique visitors/month per website), with most unique visitors arriving through search (63.95%, 74,606/116,657). Websites with a higher percent of traffic from search engines garnered more unique visitors (P=.001). The most common search terms were for individual hospitals (23.25%, 27,122/74,606) and website names (19.43%, 22,672/74,606); medical condition terms were uncommon (0.81%, 605/74,606). Survey view rate was 42.48% (49,560/116,657 invited) resulting in 1755 respondents (participation rate=3.6%). There were substantial proportions of consumer (48.43%, 850/1755) and health care professional respondents (31.39%, 551/1755). Across websites, proportions of consumer (21%-71%) and health care professional respondents (16%-48%) varied. Consumers were frequently interested in using the information to choose providers or assess the quality of their provider (52.7%, 225/427); the majority of those choosing a provider reported that they had used the information to do so (78%, 40/51). Health care professional (26.6%, 115/443) and consumer (20.8%, 92/442) respondents wanted cost information and consumers wanted patient narrative comments (31.5%, 139/442) on the public reports. Health care professional respondents rated the experience on the reports higher than consumers did (mean 7.2, SD 2.2 vs mean 6.2, SD 2.7; scale 0-10; P<.001).

Conclusions

Report sponsors interested in increasing the influence of their reports could consider using techniques to improve search engine traffic, providing cost information and patient comments, and improving the website experience for both consumers and health care professionals.

"I wish they could be in my shoes": patients' insights into tertiary health care for type 2 diabetes mellitus

BACKGROUND

Insightful accounts of patient experience within a health care system can be valuable for facilitating improvements in service delivery.

OBJECTIVE

The aim of this study was to explore patients' perceptions and experiences regarding a tertiary hospital Diabetes and Endocrinology outpatient service for the management of type 2 diabetes mellitus (T2DM).

METHOD

Nine patients participated in discovery interviews with an independent trained facilitator. Patients' stories were synthesized thematically using a constant comparative approach.

RESULTS

Three major themes were identified from the patients' stories: 1) understanding T2DM and diabetes management with subthemes highlighting that specialist care is highly valued by patients who experience a significant burden of diabetes on daily life and who may have low health literacy and low self confidence; 2) relationships with practitioners were viewed critical and perceived lack of empathy impacted the effectiveness of care; and 3) impact of health care systems on service delivery with lack of continuity of care relating to the tertiary hospital model and limitations with appointment bookings negatively impacting on patient experience.

DISCUSSION

The patients' stories suggest that the expectation of establishing a productive, ongoing relationship with practitioners is highly valued. Tertiary clinics for T2DM are well placed to incorporate novel technological approaches for monitoring and follow-up, which may overcome many of the perceived barriers of traditional service delivery.

CONCLUSION

Investing in strategies that promote patient-practitioner relationships may enhance effectiveness of treatment for T2DM by meeting patient expectations of personalized care. Future changes in service delivery would benefit from incorporating patients as key stakeholders in service evaluation.

Postscript: research agenda to guide the next generation of public reports for consumers

There is significant interest in building the next generation of public reporting tools that will more effectively engage consumers and better enable them to make use of comparative performance information when selecting a provider. Demand for public reporting tools that make health care cost and quality information transparent is fueled by a variety of market forces underway. A host of public reporting efforts and studies have identified a number of challenges, highlighting that we still do not understand how best to design public reports to meet the needs of the consumer. We identify five areas for additional research that, if addressed, could foster better design and delivery of quality and cost information to consumers.