General Practitioners' Concerns About Online Patient Feedback: Findings From a Descriptive Exploratory Qualitative Study in England

Characteristics of positive feedback provided by UK health service users: content analysis of examples from two databases

BACKGROUND

Most feedback received by health services is positive. Our systematic scoping review mapped all available empirical evidence for how positive patient feedback creates healthcare change. Most included papers did not provide specific details on positive feedback characteristics.

OBJECTIVES

Describe positive feedback characteristics by (1) developing heuristics for identifying positive feedback; (2) sharing annotated feedback examples; (3) describing their positive content.

METHODS

200 items were selected from two contrasting databases: (1) https://careopinion.org.uk/; (2) National Health Service (NHS) Friends and Family Test data collected by an NHS trust. Preliminary heuristics and positive feedback categories were developed from a small convenience sample, and iteratively refined.

RESULTS

Categories were identified: positive-only; mixed; narrative; factual; grateful. We propose a typology describing tone (positive-only, mixed), form (factual, narrative) and intent (grateful). Separating positive and negative elements in mixed feedback was sometimes impossible due to ambiguity. Narrative feedback often described the cumulative impact of interactions with healthcare providers, healthcare professionals, influential individuals and community organisations. Grateful feedback was targeted at individual staff or entire units, but the target was sometimes ambiguous.

CONCLUSION

People commissioning feedback collection systems should consider mechanisms to maximise utility by limiting ambiguity. Since being enabled to provide narrative feedback can allow contributors to make contextualised statements about what worked for them and why, then there may be trade-offs to negotiate between limiting ambiguity, and encouraging rich narratives. Groups tasked with using feedback should plan the human resources needed for careful inspection, and consider providing narrative analysis training.

Capturing patient feedback to improve healthcare services

The primary purpose of gathering patient feedback is to listen to, reflect on and act on the feedback to improve patients' experiences, interactions and health outcomes. Nurses use patient feedback to guide person-centred care or to inform healthcare decisions. However, when healthcare services attempt to improve the quality of care, there can be a lack of process clarity, clear measurement and evidence of improvements. This article provides an overview of the strategies used to capture patient feedback and offers guidance on how nurses can make use of such information to promote healthcare improvement.

Online patient feedback as a safety valve: An automated language analysis of unnoticed and unresolved safety incidents

Safety reporting systems are widely used in healthcare to identify risks to patient safety. But, their effectiveness is undermined if staff do not notice or report incidents. Patients, however, might observe and report these overlooked incidents because they experience the consequences, are highly motivated, and independent of the organization. Online patient feedback may be especially valuable because it is a channel of reporting that allows patients to report without fear of consequence (e.g., anonymously). Harnessing this potential is challenging because online feedback is unstructured and lacks demonstrable validity and added value. Accordingly, we developed an automated language analysis method for measuring the likelihood of patient-reported safety incidents in online patient feedback. Feedback from patients and families (n = 146,685, words = 22,191,427, years = 2013-2019) about acute NHS trusts (hospital conglomerates; n = 134) in England were analyzed. The automated measure had good precision (0.69) and excellent recall (0.98) in identifying incidents; was independent of staff-reported incidents (r = -0.04 to 0.19); and was associated with hospital-level mortality rates (z = 3.87; p < 0.001). The identified safety incidents were often reported as unnoticed (89%) or unresolved (21%), suggesting that patients use online platforms to give visibility to safety concerns they believe have been missed or ignored. Online stakeholder feedback is akin to a safety valve; being independent and unconstrained it provides an outlet for reporting safety issues that may have been unnoticed or unresolved within formal channels.

Web-Based Public Ratings of General Practitioners in Norway: Validation Study

BACKGROUND

Understanding the complex relationships among multiple strategies for gathering users' perspectives in the evaluation of the performance of services is crucial for the interpretation of user-reported measures.

OBJECTIVE

The main objectives were to (1) evaluate the psychometric performance of an 11-item web-based questionnaire of ratings of general practitioners (GPs) currently used in Norway (Legelisten.no) and (2) assess the association between web-based and survey-based patient experience indicators.

METHODS

We included all published ratings on GPs and practices on Legelisten.no in the period of May 5, 2012, to December 15, 2021 (N=76,521). The questionnaire consists of 1 mandatory item and 10 voluntary items with 5 response categories (1 to 5 stars), alongside an open-ended review question and background variables. Questionnaire dimensionality and internal consistency were assessed with Cronbach α, exploratory factor, and item response theory analyses, and a priori hypotheses were developed for assessing construct validity (chi-square analysis). We calculated Spearman correlations between web-based ratings and reference patient experience indicators based on survey data using the patient experiences with the GP questionnaire (n=5623 respondents for a random sample of 50 GPs).

RESULTS

Web-based raters were predominantly women (n=32,074, 64.0%), in the age range of 20-50 years (n=35,113, 74.6%), and reporting 5 or fewer consultations with the GP each year (n=28,798, 64.5%). Ratings were missing for 18.9% (n=14,500) to 27.4% (n=20,960) of nonmandatory items. A total of 4 of 11 rating items showed a U-shaped distribution, with >60% reporting 5 stars. Factor analysis and internal consistency testing identified 2 rating scales: "GP" (5 items; α=.98) and "practice" (6 items; α=.85). Some associations were not consistent with a priori hypotheses and allowed only partial confirmation of the construct validity of ratings. Item response theory analysis results were adequate for the "practice" scale but not for the "GP" scale, with items with inflated discrimination (>5) distributed over a narrow interval of the scale. The correlations between the web-based ratings GP scale and GP reference indicators ranged from 0.34 (P=.021) to 0.44 (P=.002), while the correlation between the web-based ratings practice scale and reference indicators ranged from 0.17 (not significant) to 0.49 (P<.001). The strongest correlations between web-based and survey scores were found for items measuring practice-related experiences: phone availability (ρ=0.51), waiting time in the office (ρ=0.62), other staff (ρ=0.54-0.58; P<.001).

CONCLUSIONS

The practice scale of the web-based ratings has adequate psychometric performance, while the GP suffers from important limitations. The associations with survey-based patient experience indicators were accordingly mostly weak to modest. Our study underlines the importance of interpreting web-based ratings with caution and the need to further develop rating sites.

Turning the gaze: Digital patient feedback and the silent pathology of the NHS

Online review and rating sites, where patients can leave feedback on their experience of the health-care encounter, are becoming an increasing feature of primary care in the NHS. Previous research has analysed how digital surveillance is re-shaping the clinical gaze, as health-care professionals are subject to increased public monitoring. Here, we draw on an empirical study of 41 GP practice staff to show how the gaze is turning, not simply from the patient to the health-care provider, but additionally to the body politic of the NHS. Drawing on focus group and interview data conducted in five UK practices, we show how discourses of online reviews and ratings are producing new professional subjectivities among health-care professionals and the extent to which the gaze extends not only to individual health-care interactions but to the health-care service writ large. We identify three counter-discourses characterising the evolving ways in which online reviews and ratings are creating new subjects in primary care practices: victimhood, prosumption versus traditional values and taking control. We show how the ways in which staff speak about online feedback are patterned by the social environment in which they work and the constraints of the NHS they encounter on a day-to-day basis.

Medical Students' Perceptions of a Blockchain-Based Decentralized Work History and Credentials Portfolio: Qualitative Feasibility Study

Background

Increased digitization of health care might challenge some of the trust functions that are established in a traditional health care system. We have, with the concept of VerifyMed, developed a decentralized service for work history and competence verification, as a means to increase trust in the virtual interaction between a patient and a caregiver, mitigate administrative burden, and provide patient-reported outcomes seamlessly for health professionals.

Objective

This research aimed to validate the use case of a decentralized credentials service for health care professionals in Norway. We also aimed to evaluate the proof-of-concept of VerifyMed, a blockchain-based credential service for health care professionals.

Methods

A qualitative approach was applied with data collection through 9 semistructured interviews and 2 focus groups (one with 4 participants and the other with 5 participants). The System Usability Scale (SUS) was used as a part of the interviews. Data were analyzed through the principles of systematic text condensation. The recruitment of participants ended when it was concluded that the data had reached saturation.

Results

The following 5 themes were identified from the interviews and focus groups: (1) the need for aggregated storage of work- and study-related verification, (2) trust in a virtual health care environment, (3) the potential use of patient feedback, (4) trust in blockchain technology, and (5) improvements of the VerifyMed concept. The SUS questionnaire gave a score of 69.7.

Conclusions

This study has validated the need for a decentralized system where health care professionals can control their credentials and, potentially, their reputation. Future work should update the VerifyMed system according to this input. We concluded that a decentralized system for the storage of work-related verifiable credentials could increase trust in a virtualized health care system.

Caring for care: Online feedback in the context of public healthcare services

People increasingly provide feedback about healthcare services online. These practices have been lauded for enhancing patient power, choice and control, encouraging greater transparency and accountability, and contributing to healthcare service improvement. Online feedback has also been critiqued for being unrepresentative, spreading inaccurate information, undermining care relations, and jeopardising professional autonomy. Through a thematic analysis of 37 qualitative interviews, this paper explores the relationship between online feedback and care improvement as articulated by healthcare service users (patients and family members) who provided feedback across different online platforms and social media in the UK. Online feedback was framed by interviewees as, ideally, a public and, in many cases, anonymous 'conversation' between service users and healthcare providers. These 'conversations' were thought of not merely as having the potential to bring about tangible improvements to healthcare, but as in themselves constituting an improvement in care. Vital to this was the premise that providing feedback was an enactment of care - care for other patients, certainly, but also care for healthcare as such and even for healthcare professionals. Ultimately, feedback was understood as an enactment of care for the National Health Service (NHS), as symbolically encompassing all of the above. Putting these findings in dialogue with STS scholarship on care, we argue that, in this context, the provision of online feedback can be understood as a form of care that is, simultaneously, both directed at healthcare (in the round, including patients, professionals, services, organisations, and, of course, health itself) and part of healthcare. We conceptualise this as 'caring for care'. This conceptualization moves beyond dominant framings of online feedback in terms of 'choice' and 'voice'. It embeds online feedback within pre-existing healthcare systems, relations and moral commitments, foregrounds the mutuality of care relations, and draws attention to the affective labour of feedback practices.

A good dentist or not: understanding the role of storytelling in online dentist reviews

Purpose

Narrative comments about dentists on physician review sites have been documented to increasingly influence people's selection of their dentists. From a communication standpoint, these comments are a type of narrative communication that people share their experiences with dentists by telling stories. Based on the frameworks of rhetoric structure theory and extended elaborated likelihood model, this study aimed to examine the effects of such storytelling from two perspectives including narrative structure and narrative focus.

Design/methodology/approach

A 4 (narrative structure) × 2 (narrative focus) between-subjects experiment was conducted to examine the proposed hypotheses and research questions

Findings

The results showed that a one-sided comprehensive comment focusing on technical competence generated the strongest persuasion effects measured by attitude and behavioral intention. These effects were mediated by perceived narrative credibility and enjoyment.

Originality/value

This study contributes to the extant literature in two ways. First, it extends previous studies of online narrative comments by showing which narrative structure and focus are deemed to be more persuasive when selecting a dentist. Second, it offers a test of two routes of information processing (i.e. cognitive and experiential) to understand the mechanism underlying the effects of narrative comments.

Peer review

The peer-review history for this article is available at: https://publons.com/publon/10.1108/OIR-08-2020-0359

Exploring patients' pharmacy stories: an analysis of online feedback

Background Studies have demonstrated the potential for patient feedback to inform quality care as well as a direct relationship between patient experience and clinical outcomes. Over recent years, there has been increasing use of online patient feedback platforms, however, there has been little study of the content of patient feedback relating to pharmacy and pharmacy services. Objective This study explores the content of online feedback provided by patients from across the UK in relation to their experiences of their interaction with pharmacy staff and pharmacy services. Main outcome measure Content of online patient feedback relating to pharmacy. Method Patient stories published on Care Opinion, a national online patient feedback platform, for a one-year period were searched for all content relating to patients’ pharmacy experiences. A thematic and sentiment analysis was conducted on 237 patient stories. Results Patient stories related to supply, staff attitudes, services, accessibility, systems, and errors. Patient sentiment depended on pharmacy setting, but staff attitudes, services, and accessibility were generally positive across all settings. Waiting time was the most common complaint in both hospital and community pharmacies with stories relaying experiences of slow discharge, stock shortages and poor communication and collaboration between pharmacies and GP surgeries. Conclusions Online patient feedback highlighted factors important to patients when interacting with pharmacies and their staff. Medication supply was the primary topic of patient stories with waiting times and stock shortages being clear areas for improvement; however, accessibility, pharmacy services and advice were key strengths of the profession. Further research is needed to understand how online patient feedback can be used effectively to inform improvements in pharmacy services.

Influence of Health Literacy on Effects of Patient Rating Websites: Survey Study Using a Hypothetical Situation and Fictitious Doctors

Background

Physician rating websites (PRWs) are a device people use actively and passively, although their objective capabilities are insufficient when it comes to judging the medical performance and qualification of physicians. PRWs are an innovation born of the potential of the Internet and boosted very much by the longstanding policy of improving and encouraging patient participation in medical decision-making. A mismatch is feared between patient motivations to participate and their capabilities of doing so well. Awareness of such a mismatch might contribute to some skepticism of patient-written physician reviews on PRWs.

Objective

We intend to test whether health literacy is able to dampen the effects that a patient-written review of a physician’s performance might have on physician choice.

Methods

An experiment was conducted within a survey interview. Participants were put into a fictitious decision situation in which they had to choose between two physicians on the basis of their profiles on a PRW. One of the physician profiles contained the experimental stimulus in the form of a friendly and a critical written review. The dependent variable was physician choice. An attitude differential, trust differential, and two measures of health literacy, the newest vital sign as an example of a performance-based measure and eHealth Literacy Scale as an example of a perception-based measure, were tested for roles as intermediary variables. Analysis traced the influence of the review tendency on the dependent variables and a possible moderating effect of health literacy on these influences.

Results

Reviews of a physician’s competence and medical skill affected participant choice of a physician. High health literacy dampened these effects only in the case of the perception-based measure and only for the negative review. Correspondingly, the effect of the review tendency appeared to be stronger for the positive review. Attitudes and trust only affected physician choice when included as covariants, considerably increasing the variance explained by regression models.

Conclusions

Findings sustain physician worries that even one negative PRW review can affect patient choice and damage doctors’ reputations. Hopes that health literacy might raise awareness of the poor basis of physician reviews and ratings given by patients have some foundation.

Exploring UK doctors' attitudes towards online patient feedback: Thematic analysis of survey data

Introduction

Patients are increasingly using online platforms to give feedback about their health-care experiences. Online feedback has been proposed as a way to drive transformative change in the health service through informing choice and improving quality. Attitudes held by health-care professionals influence the uptake of new technologies. Understanding these attitudes is essential in exploring the potential of online patient feedback as a standard feedback mechanism. This study explores the content of free-text comments left by doctors responding to a survey with the aim of understanding their attitudes towards online feedback.

Methods

A cross-sectional online questionnaire was completed by 1001 UK primary and secondary-care doctors. Doctors were given the opportunity to leave a free-text comment about online patient feedback. Doctors’ attitudes towards online patient feedback were identified and explored using thematic analysis. Descriptive statistics and chi-square tests were used to examine demographic differences between those doctors who left a comment and those who did not.

Results

Thematic analysis identified five key interrelated themes: anonymity, confidentiality, representativeness, moderation/regulation of online feedback and platform type. The characteristics of those leaving a comment very closely matched those of the entire survey sample.

Conclusion

Across the comments, the most prominent finding was a general scepticism and caution towards online feedback, with most of the key themes relating to the perceived limitations and challenges. Further work exploring ways of addressing and verifying online comments without breaching confidentiality could provide valuable information to health systems seeking to drive improvement through patient online feedback.

Online patient feedback as a measure of quality in primary care: a multimethod study using correlation and qualitative analysis

OBJECTIVES

To ascertain the relationship between online patient feedback and the General Practice Patient Survey (GPPS) and the Friends and Family Test (FFT). To consider the potential benefit it may add by describing the content of public reviews found on NHS Choices for all general practices in one Clinical Commissioning Group in England.

DESIGN

Multimethod study using correlation and thematic analysis.

SETTING

1396 public online reviews and ratings on NHS Choices for all General Practices (n=70) in Oxfordshire Clinical Commissioning Group in England.

RESULTS

Significant moderate correlations were found between the online patient feedback and the GPPS and the FFT. Three themes were developed through the qualitative analysis: (1) online feedback largely provides positive reinforcement for practice staff; (2) online feedback is used as a platform for suggesting service organisation and delivery improvements; (3) online feedback can be a source of insight into patients' expectations of care. These themes illustrate the wide range of topics commented on by patients, including their medical care, relationships with various members of staff, practice facilities, amenities and services in primary care settings.

CONCLUSIONS

This multimethod study demonstrates that online feedback found on NHS Choices is significantly correlated with established measures of quality in primary care. This suggests it has a potential use in understanding patient experience and satisfaction, and a potential use in quality improvement and patient safety. The qualitative analysis shows that this form of feedback contains helpful information about patients' experiences of general practice that provide insight into issues of quality and patient safety relevant to primary care. Health providers should offer patients multiple ways of offering feedback, including online, and should have systems in place to respond to and act on this feedback.

What Patients Value in Physicians: Analyzing Drivers of Patient Satisfaction Using Physician-Rating Website Data

BACKGROUND

Customer-oriented health care management and patient satisfaction have become important for physicians to attract patients in an increasingly competitive environment. Satisfaction influences patients' choice of physician and leads to higher patient retention and higher willingness to engage in positive word of mouth. In addition, higher satisfaction has positive effects on patients' willingness to follow the advice given by the physician. In recent years, physician-rating websites (PRWs) have emerged in the health care sector and are increasingly used by patients. Patients' usage includes either posting an evaluation to provide feedback to others about their own experience with a physician or reading evaluations of other patients before choosing a physician. The emergence of PRWs offers new avenues to analyze patient satisfaction and its key drivers. PRW data enable both satisfaction analyses and implications on the level of the individual physician as well as satisfaction analyses and implications on an overall level.

OBJECTIVE

This study aimed to identify linear and nonlinear effects of patients' perceived quality of physician appointment service attributes on the overall evaluation measures that are published on PRWs.

METHODS

We analyzed large-scale survey data from a German PRW containing 84,680 surveys of patients rating a total of 7038 physicians on 24 service attributes and 4 overall evaluation measures. Elasticities are estimated from regression models with perceived attribute quality as explanatory variables and overall evaluation measures as dependent variables. Depending on the magnitude of the elasticity, service attributes are classified into 3 categories: attributes with diminishing, constant, or increasing returns to overall evaluation.

RESULTS

The proposed approach revealed new insights into what patients value when visiting physicians and what they take for granted. Improvements in the physicians' pleasantness and friendliness have increasing returns to the publicly available overall evaluation (b=1.26). The practices' cleanliness (b=1.05) and the communication behavior of a physician during a visit (b level between .97 and 1.03) have constant returns. Indiscretion in the waiting rooms, extended waiting times, and a lack of modernity of the medical equipment (b level between .46 and .59) have the strongest diminishing returns to overall evaluation.

CONCLUSIONS

The categorization of the service attributes supports physicians in identifying potential for improvements and prioritizing resource allocation to improve the publicly available overall evaluation ratings on PRWs. Thus, the study contributes to patient-centered health care management and, furthermore, promotes the utility of PRWs through large-scale data analysis.

The Use of Recommendations on Physician Rating Websites: The Number of Raters Makes the Difference When Adjusting Decisions

Physician rating websites allow users to check physicians' profiles, write reviews, or rate their performance. The opinion of other users regarding a physician can affect our decision to visit her/him. To investigate the specific role of the number of users rating a physician when choosing a physician with support of these platforms, we used a Judge-Advisor System in which participants answered their likelihood to visit a physician before and after seeing the recommendations of others. Within the experiment, three conditions were presented: high and low number of reviewers recommending a physician, and no recommendations. We found that the participants' likelihood to visit a physician varied with respect to the displayed physician characteristics on the platform. Importantly, after the recommendation of others was presented, participants' likelihood to visit the physician changed significantly. The participants' adjusted response was significantly closer to the recommendation coming from a higher number of users, which indicate that this online, social media cue influences our decision to visit physicians. Comments and ratings on physician ratings are generally positive, but we show that negative ratings have a direct negative influence in the decision to visit a physician. We suggest administrators of these platforms to pay special attention to the content that users upload.

Using online patient feedback to improve NHS services: the INQUIRE multimethod study

Background

Online customer feedback has become routine in many industries, but it has yet to be harnessed for service improvement in health care.

Objectives

To identify the current evidence on online patient feedback; to identify public and health professional attitudes and behaviour in relation to online patient feedback; to explore the experiences of patients in providing online feedback to the NHS; and to examine the practices and processes of online patient feedback within NHS trusts.

Design

A multimethod programme of five studies: (1) evidence synthesis and stakeholder consultation; (2) questionnaire survey of the public; (3) qualitative study of patients’ and carers’ experiences of creating and using online comment; (4) questionnaire surveys and a focus group of health-care professionals; and (5) ethnographic organisational case studies with four NHS secondary care provider organisations.

Setting

The UK.

Methods

We searched bibliographic databases and conducted hand-searches to January 2018. Synthesis was guided by themes arising from consultation with 15 stakeholders. We conducted a face-to-face survey of a representative sample of the UK population (n = 2036) and 37 purposively sampled qualitative semistructured interviews with people with experience of online feedback. We conducted online surveys of 1001 quota-sampled doctors and 749 nurses or midwives, and a focus group with five allied health professionals. We conducted ethnographic case studies at four NHS trusts, with a researcher spending 6–10 weeks at each site.

Results

Many people (42% of internet users in the general population) read online feedback from other patients. Fewer people (8%) write online feedback, but when they do one of their main reasons is to give praise. Most online feedback is positive in its tone and people describe caring about the NHS and wanting to help it (‘caring for care’). They also want their feedback to elicit a response as part of a conversation. Many professionals, especially doctors, are cautious about online feedback, believing it to be mainly critical and unrepresentative, and rarely encourage it. From a NHS trust perspective, online patient feedback is creating new forms of response-ability (organisations needing the infrastructure to address multiple channels and increasing amounts of online feedback) and responsivity (ensuring responses are swift and publicly visible).

Limitations

This work provides only a cross-sectional snapshot of a fast-emerging phenomenon. Questionnaire surveys can be limited by response bias. The quota sample of doctors and volunteer sample of nurses may not be representative. The ethnographic work was limited in its interrogation of differences between sites.

Conclusions

Providing and using online feedback are becoming more common for patients who are often motivated to give praise and to help the NHS improve, but health organisations and professionals are cautious and not fully prepared to use online feedback for service improvement. We identified several disconnections between patient motivations and staff and organisational perspectives, which will need to be resolved if NHS services are to engage with this source of constructive criticism and commentary from patients.

Future work

Intervention studies could measure online feedback as an intervention for service improvement and longitudinal studies could examine use over time, including unanticipated consequences. Content analyses could look for new knowledge on specific tests or treatments. Methodological work is needed to identify the best approaches to analysing feedback.

Study registration

The ethnographic case study work was registered as Current Controlled Trials ISRCTN33095169.

Funding

This project was funded by the National institute for Health Research (NIHR) Health Services and Delivery Research programme and will be published in full in Health Services and Delivery Research; Vol. 7, No. 38. See the NIHR Journals Library website for further project information.

The Content and Nature of Narrative Comments on Swiss Physician Rating Websites: Analysis of 849 Comments

Background

The majority of physician rating websites (PRWs) provide users the option to leave narrative comments about their physicians. Narrative comments potentially provide richer insights into patients’ experiences and feelings that cannot be fully captured in predefined quantitative rating scales and are increasingly being examined. However, the content and nature of narrative comments on Swiss PRWs has not been examined to date.

Objective

This study aimed to examine (1) the types of issues raised in narrative comments on Swiss PRWs and (2) the evaluation tendencies of the narrative comments.

Methods

A random stratified sample of 966 physicians was generated from the regions of Zürich and Geneva. Every selected physician was searched for on 3 PRWs (OkDoc, DocApp, and Medicosearch) and Google, and narrative comments were collected. Narrative comments were analyzed and classified according to a theoretical categorization framework of physician-, staff-, and practice-related issues.

Results

The selected physicians had a total of 849 comments. In total, 43 subcategories addressing the physician (n=21), staff (n=8), and practice (n=14) were identified. None of the PRWs’ comments covered all 43 subcategories of the categorization framework; comments on Google covered 86% (37/43) of the subcategories, Medicosearch covered 72% (31/43), DocApp covered 60% (26/43), and OkDoc covered 56% (24/43). In total, 2441 distinct issues were identified within the 43 subcategories of the categorization framework; 83.65% (2042/2441) of the issues related to the physician, 6.63% (162/2441) related to the staff, and 9.70% (237/2441) related to the practice. Overall, 95% (41/43) of the subcategories of the categorization framework and 81.60% (1992/2441) of the distinct issues identified were concerning aspects of performance (interpersonal skills of the physician and staff, infrastructure, and organization and management of the practice) that are considered assessable by patients. Overall, 83.0% (705/849) of comments were classified as positive, 2.5% (21/849) as neutral, and 14.5% (123/849) as negative. However, there were significant differences between PRWs, regions, and specialty regarding negative comments: 90.2% (111/123) of negative comments were on Google, 74.7% (92/123) were regarding physicians in Zurich, and 73.2% (90/123) were from specialists.

Conclusions

From the narrative comments analyzed, it can be reported that interpersonal issues make up nearly half of all negative issues identified, and it is recommended that physicians should focus on improving these issues. The current suppression of negative comments by Swiss PRWs is concerning, and there is a need for a consensus-based criterion to be developed to determine which comments should be published publicly. Finally, it would be helpful if Swiss patients are made aware of the current large differences between Swiss PRWs regarding the frequency and nature of ratings to help them determine which PRW will provide them with the most useful information.

Online patient feedback: a scoping review and stakeholder consultation to guide health policy

Objective

To provide a synthesis of the current evidence base of online patient feedback using a scoping review and a consultation of stakeholders in England, UK.

Methods

We searched MEDLINE, EMBASE, PsycINFO, CINAHL and the Social Science Citation Index and conducted hand searches up to January 2018. We included primary studies of internet-based reviews and other online feedback (e.g. social media and blogs) from patients, carers or the public about health care providers (individuals, services or organizations). Key findings were extracted and tabulated for further synthesis guided by the themes arising from a stakeholder consultation.

Results

The review found that awareness and usage of online feedback is increasing. Most feedback is about physicians, and is typically positive. Online reviews and ratings are used by some service users to inform choice of provider or treatment while providers tend to be concerned about the validity and representativeness of feedback. Reviewed studies found that those who post feedback are generally not representative of the general population, tending to be younger and more educated, but online feedback does broadly correlate with some other measures of health care quality.

Conclusions

In an increasingly digital society, where citizens provide and use feedback for a range of goods and services, online patient feedback can offer a convenient, low cost and widely accessible mechanism to capture experiences of health care, while being mindful to avoid issues of digital exclusion. This review provides important insights to inform policy development seeking to harness the opportunities offered by online feedback.

Quantitative Ratings and Narrative Comments on Swiss Physician Rating Websites: Frequency Analysis

Background

Physician rating websites (PRWs) have been developed as part of a wider move toward transparency around health care quality, and these allow patients to anonymously rate, comment, and discuss physicians’ quality on the Web. The first Swiss PRWs were established in 2008, at the same time as many international PRWs. However, there has been limited research conducted on PRWs in Switzerland to date. International research has indicated that a key shortcoming of PRWs is that they have an insufficient number of ratings.

Objective

The aim of this study was to examine the frequency of quantitative ratings and narrative comments on the Swiss PRWs.

Methods

In November 2017, a random stratified sample of 966 physicians was generated from the regions of Zürich and Geneva. Every selected physician was searched for on 4 rating websites (OkDoc, DocApp, Medicosearch, and Google) between November 2017 and July 2018. It was recorded whether the physician could be identified, what the physician’s quantitative rating was, and whether the physician had received narrative comments. In addition, Alexa Internet was used to examine the number of visitors to the PRWs, compared with other websites.

Results

Overall, the portion of physicians able to be identified on the PRWs ranged from 42.4% (410/966) on OkDoc to 87.3% (843/966) on DocApp. Of the identifiable physicians, only a few of the selected physicians had been rated quantitatively (4.5% [38/843] on DocApp to 49.8% [273/548] on Google) or received narrative comments (4.5% [38/843] on DocApp to 31.2% [171/548] on Google) at least once. Rated physicians also had, on average, a low number of quantitative ratings (1.47 ratings on OkDoc to 3.74 rating on Google) and narrative comments (1.23 comment on OkDoc to 3.03 comments on Google). All 3 websites allowing ratings used the same rating scale (1-5 stars) and had a very positive average rating: DocApp (4.71), Medicosearch (4.69), and Google (4.41). There were significant differences among the PRWs (with the majority of ratings being posted on Google in past 2 years) and regions (with physicians in Zurich more likely to have been rated and have more ratings on average). Only Google (position 1) and Medicosearch (position 8358) are placed among the top 10,000 visited websites in Switzerland.

Conclusions

It appears that this is the first time Google has been included in a study examining physician ratings internationally and it is noticeable how Google has had substantially more ratings than the 3 dedicated PRWs in Switzerland over the past 2 and a half years. Overall, this study indicates that Swiss PRWs are not yet a reliable source of unbiased information regarding patient experiences and satisfaction with Swiss physicians; many selected physicians were unable to be identified, only a few physicians had been rated, and the ratings posted were overwhelmingly positive.

Online patient feedback: a cross-sectional survey of the attitudes and experiences of United Kingdom health care professionals

Objectives

Online patient feedback is a growing phenomenon but little is known about health professional attitudes and behaviours in relation to it. We aimed to identify the characteristics, attitudes and self-reported behaviours and experiences of doctors and nurses towards online feedback from their patients or their carers.

Methods

We conducted a cross-sectional self-completed online questionnaire of 1001 registered doctors and 749 nurses and midwives involved in direct patient care in the United Kingdom.

Results

Just over a quarter (27.7% or 277/1001) of doctors and 21% (157/749) of nurses were aware that patients/carers had provided online feedback about an episode of care in which they were involved, and 20.5% (205/1001) of doctors and 11.1% (83/749) of nurses had experienced online feedback about them as an individual practitioner. Feedback on reviews/ratings sites was seen as more useful than social media feedback to help improve services. Both types of feedback were more likely to be seen as useful by nurses compared with doctors and by hospital-based professionals compared with those based in community settings. Doctors were more likely than nurses to believe that online feedback is unrepresentative and generally negative in tone. The majority of respondents had never encouraged patients/carers to leave online feedback.

Conclusions

Despite enthusiasm from health policymakers, many health care professionals have little direct experience of online feedback, and rarely encourage it, as they view it as unrepresentative and with limited value for improving the quality of health services. The difference in opinion between doctors and nurses has the potential to disrupt any use of online patient feedback. The findings have implications for policy and practice in how online patient feedback is solicited and acted upon.

Content, Quality, and Assessment Tools of Physician-Rating Websites in 12 Countries: Quantitative Analysis

BACKGROUND

Websites on which users can rate their physician are becoming increasingly popular, but little is known about the website quality, the information content, and the tools they offer users to assess physicians. This study assesses these aspects on physician-rating websites in German- and English-speaking countries.

OBJECTIVE

The objective of this study was to collect information on websites with a physician rating or review tool in 12 countries in terms of metadata, website quality (transparency, privacy and freedom of speech of physicians and patients, check mechanisms for appropriateness and accuracy of reviews, and ease of page navigation), professional information about the physician, rating scales and tools, as well as traffic rank.

METHODS

A systematic Web search based on a set of predefined keywords was conducted on Google, Bing, and Yahoo in August 2016. A final sample of 143 physician-rating websites was analyzed and coded for metadata, quality, information content, and the physician-rating tools.

RESULTS

The majority of websites were registered in the United States (40/143) or Germany (25/143). The vast majority were commercially owned (120/143, 83.9%), and 69.9% (100/143) displayed some form of physician advertisement. Overall, information content (mean 9.95/25) as well as quality were low (mean 18.67/47). Websites registered in the United Kingdom obtained the highest quality scores (mean 26.50/47), followed by Australian websites (mean 21.50/47). In terms of rating tools, physician-rating websites were most frequently asking users to score overall performance, punctuality, or wait time in practice.

CONCLUSIONS

This study evidences that websites that provide physician rating should improve and communicate their quality standards, especially in terms of physician and user protection, as well as transparency. In addition, given that quality standards on physician-rating websites are low overall, the development of transparent guidelines is required. Furthermore, attention should be paid to the financial goals that the majority of physician-rating websites, especially the ones that are commercially owned, pursue.

Unhappy Patients Are Not Alike: Content Analysis of the Negative Comments from China's Good Doctor Website

BACKGROUND

With the rise in popularity of Web 2.0 technologies, the sharing of patient experiences about physicians on online forums and medical websites has become a common practice. However, negative comments posted by patients are considered to be more influential by other patients and physicians than those that are satisfactory.

OBJECTIVE

The aim of this study was to analyze negative comments posted online about physicians and to identify possible solutions to improve patient satisfaction, as well as their relationship with physicians.

METHODS

A Java-based program was developed to collect patient comments on the Good Doctor website, one of the most popular online health communities in China. A total of 3012 negative comments concerning 1029 physicians (mean 2.93 [SD 4.14]) from 5 highly ranked hospitals in Beijing were extracted for content analysis. An initial coding framework was constructed with 2 research assistants involved in the codification.

RESULTS

Analysis, based on the collected 3012 negative comments, revealed that unhappy patients are not alike and that their complaints cover a wide range of issues experienced throughout the whole process of medical consultation. Among them, physicians in Obstetrics and Gynecology (606/3012, 20.12%; P=.001) and Internal Medicine (487/3012, 16.17%; P=.80) received the most negative comments. For negative comments per physician, Dermatology and Sexually Transmitted Diseases (mean 5.72, P<.001) and Andrology (mean 5, P=.02) ranked the highest. Complaints relating to insufficient medical consultation duration (577/3012, 19.16%), physician impatience (527/3012, 17.50%), and perceived poor therapeutic effect (370/3012, 12.28%) received the highest number of negative comments. Specific groups of people, such as those accompanying older patients or children, traveling patients, or very important person registrants, were shown to demonstrate little tolerance for poor medical service.

CONCLUSIONS

Analysis of online patient complaints provides an innovative approach to understand factors associated with patient dissatisfaction. The outcomes of this study could be of benefit to hospitals or physicians seeking to improve their delivery of patient-centered services. Patients are expected to be more understanding of overloaded physicians' workloads, which are impacted by China's stretched medical resources, as efforts are made to build more harmonious physician-patient relationships.

Developments in the Frequency of Ratings and Evaluation Tendencies: A Review of German Physician Rating Websites

BACKGROUND

Physician rating websites (PRWs) have been developed to allow all patients to rate, comment, and discuss physicians' quality online as a source of information for others searching for a physician. At the beginning of 2010, a sample of 298 randomly selected physicians from the physician associations in Hamburg and Thuringia were searched for on 6 German PRWs to examine the frequency of ratings and evaluation tendencies.

OBJECTIVE

The objective of this study was to examine (1) the number of identifiable physicians on German PRWs; (2) the number of rated physicians on German PRWs; (3) the average and maximum number of ratings per physician on German PRWs; (4) the average rating on German PRWs; (5) the website visitor ranking positions of German PRWs; and (6) how these data compare with 2010 results.

METHODS

A random stratified sample of 298 selected physicians from the physician associations in Hamburg and Thuringia was generated. Every selected physician was searched for on the 6 PRWs (Jameda, Imedo, Docinsider, Esando, Topmedic, and Medführer) used in the 2010 study and a PRW, Arztnavigator, launched by Allgemeine Ortskrankenkasse (AOK).

RESULTS

The results were as follows: (1) Between 65.1% (194/298) on Imedo to 94.6% (282/298) on AOK-Arztnavigator of the physicians were identified on the selected PRWs. (2) Between 16.4% (49/298) on Esando to 83.2% (248/298) on Jameda of the sample had been rated at least once. (3) The average number of ratings per physician ranged from 1.2 (Esando) to 7.5 (AOK-Arztnavigator). The maximum number of ratings per physician ranged from 3 (Esando) to 115 (Docinsider), indicating an increase compared with the ratings of 2 to 27 in the 2010 study sample. (4) The average converted standardized rating (1=positive, 2=neutral, and 3=negative) ranged from 1.0 (Medführer) to 1.2 (Jameda and Topmedic). (5) Only Jameda (position 317) and Medführer (position 9796) were placed among the top 10,000 visited websites in Germany.

CONCLUSIONS

Whereas there has been an overall increase in the number of ratings when summing up ratings from all 7 analyzed German PRWs, this represents an average addition of only 4 new ratings per physician in a year. The increase has also not been even across the PRWs, and it would be advisable for the users of PRWs to utilize a number of PRWs to ascertain the rating of any given physician. Further research is needed to identify barriers for patients to rate their physicians and to assist efforts to increase the number of ratings on PRWs to consequently improve the fairness and practical importance of PRWs.

Web-Based Physician Ratings for California Physicians on Probation

Background

 Web-based physician ratings systems are a popular tool to help patients evaluate physicians. Websites help patients find information regarding physician licensure, office hours, and disciplinary records along with ratings and reviews. Whether higher patient ratings are associated with higher quality of care is unclear.

Objective

 The aim of this study was to characterize the impact of physician probation on consumer ratings by comparing website ratings between doctors on probation against matched controls.

Methods

 A retrospective review of data from the Medical Board of California for physicians placed on probation from December 1989 to September 2015 was performed. Violations were categorized into nine types. Nonprobation controls were matched by zip code and specialty with probation cases in a 2:1 ratio using the California Department of Consumer Affairs website. Web-based reviews were recorded from vitals.com, healthgrades.com, and ratemds.com (ratings range from 1-5).

Results

 A total of 410 physicians were placed on probation for 866 violations. The mean (standard deviation [SD]) number of ratings per doctor was 5.2 (7.8) for cases and 4 (6.3) for controls (P=.003). The mean rating for physicians on probation was 3.7 (1.6) compared with 4.0 (1.0) for controls when all three rating websites were pooled (P<.001). Violations for medical documentation, incompetence, prescription negligence, and fraud were found to have statistically significant lower rating scores. Conversely, scores for professionalism, drugs or alcohol, crime, sexual misconduct, and personal illness were similar between cases and controls. In a univariate analysis, probation was found to be associated with lower rating, odds ratio=1.5 (95% CI 1.0-2.2). This association was not significant in a multivariate model when we included age and gender.

Conclusions

 Web-based physician ratings were lower for doctors on probation indicating that patients may perceive a difference. Despite these statistical findings, the absolute difference was quite small. Physician rating websites have utility but are imperfect proxies for competence. Further research on physician Web-based ratings is warranted to understand what they measure and how they are associated with quality.

Do Physicians Respond to Web-Based Patient Ratings? An Analysis of Physicians' Responses to More Than One Million Web-Based Ratings Over a Six-Year Period

Background

Physician-rating websites (PRWs) may lead to quality improvements in case they enable and establish a peer-to-peer communication between patients and physicians. Yet, we know little about whether and how physicians respond on the Web to patient ratings.

Objective

The objective of this study was to describe trends in physicians’ Web-based responses to patient ratings over time, to identify what physician characteristics influence Web-based responses, and to examine the topics physicians are likely to respond to.

Methods

We analyzed physician responses to more than 1 million patient ratings displayed on the German PRW, jameda, from 2010 to 2015. Quantitative analysis contained chi-square analyses and the Mann-Whitney U test. Quantitative content techniques were applied to determine the topics physicians respond to based on a randomly selected sample of 600 Web-based ratings and corresponding physician responses.

Results

Overall, physicians responded to 1.58% (16,640/1,052,347) of all Web-based ratings, with an increasing trend over time from 0.70% (157/22,355) in 2010 to 1.88% (6377/339,919) in 2015. Web-based ratings that were responded to had significantly worse rating results than ratings that were not responded to (2.15 vs 1.74, P<.001). Physicians who respond on the Web to patient ratings differ significantly from nonresponders regarding several characteristics such as gender and patient recommendation results (P<.001 each). Regarding scaled-survey rating elements, physicians were most likely to respond to the waiting time within the practice (19.4%, 99/509) and the time spent with the patient (18.3%, 110/600). Almost one-third of topics in narrative comments were answered by the physicians (30.66%, 382/1246).

Conclusions

So far, only a minority of physicians have taken the chance to respond on the Web to patient ratings. This is likely because of (1) the low awareness of PRWs among physicians, (2) the fact that only a few PRWs enable physicians to respond on the Web to patient ratings, and (3) the lack of an active moderator to establish peer-to-peer communication. PRW providers should foster more frequent communication between the patient and the physician and encourage physicians to respond on the Web to patient ratings. Further research is needed to learn more about the motivation of physicians to respond or not respond to Web-based patient ratings.

Physician and Patient Views on Public Physician Rating Websites: A Cross-Sectional Study

BACKGROUND

Numerical ratings and narrative comments about physicians are increasingly available online. These physician rating websites include independent websites reporting crowd-sourced data from online users and health systems reporting data from their internal patient experience surveys.

OBJECTIVE

To assess patient and physician views on physician rating websites.

DESIGN

Cross-sectional physician (electronic) and patient (paper) surveys conducted in August 2015.

PARTICIPANTS

Eight hundred twenty-eight physicians (response rate 43%) affiliated with one of four hospitals in a large accountable care organization in eastern Massachusetts; 494 adult patients (response rate 34%) who received care in this system in May 2015.

MAIN MEASURES

Use and perceptions of physician rating websites.

KEY RESULTS

Fifty-three percent of physicians and 39% of patients reported visiting a physician rating website at least once. Physicians reported higher levels of agreement with the accuracy of numerical data (53%) and narrative comments (62%) from health system patient experience surveys compared to numerical data (36%) and narrative comments (36%) on independent websites. Patients reported higher levels of agreement with trusting the accuracy of data obtained from independent websites (57%) compared to health system patient experience surveys (45%). Twenty-one percent of physicians and 51% of patients supported posting narrative comments online for all consumers. The majority (78%) of physicians believed that posting narrative comments online would increase physician job stress; smaller proportions perceived a negative effect on the physician-patient relationship (46%), health care overuse (34%), and patient-reported experiences of care (33%). Over one-fourth of patients (29%) believed that posting narrative comments would cause them to be less open.

CONCLUSIONS

Physicians and patients have different views on whether independent or health system physician rating websites are the more reliable source of information. Their views on whether such data should be shared on public websites are also discordant.

Exploring Patients' Views Toward Giving Web-Based Feedback and Ratings to General Practitioners in England: A Qualitative Descriptive Study

Background

Patient feedback websites or doctor rating websites are increasingly being used by patients to give feedback about their health care experiences. There is little known about why patients in England may give Web-based feedback and what may motivate or dissuade them from giving Web-based feedback.

Objective

The aim of this study was to explore patients’ views toward giving Web-based feedback and ratings to general practitioners (GPs), within the context of other feedback methods available in primary care in England, and in particular, paper-based feedback cards.

Methods

A descriptive exploratory qualitative approach using face-to-face semistructured interviews was used in this study. Purposive sampling was used to recruit 18 participants from different age groups in London and Coventry. Interviews were transcribed verbatim and analyzed using applied thematic analysis.

Results

Half of the participants in this study were not aware of the opportunity to leave feedback for GPs, and there was limited awareness about the methods available to leave feedback for a GP. The majority of participants were not convinced that formal patient feedback was needed by GPs or would be used by GPs for improvement, regardless of whether they gave it via a website or on paper. Some participants said or suggested that they may leave feedback on a website rather than on a paper-based feedback card for several reasons: because of the ability and ease of giving it remotely; because it would be shared with the public; and because it would be taken more seriously by GPs. Others, however, suggested that they would not use a website to leave feedback for the opposite reasons: because of accessibility issues; privacy and security concerns; and because they felt feedback left on a website may be ignored.

Conclusions

Patient feedback and rating websites as they currently are will not replace other mechanisms for patients in England to leave feedback for a GP. Rather, they may motivate a small number of patients who have more altruistic motives or wish to place collective pressure on a GP to give Web-based feedback. If the National Health Service or GP practices want more patients to leave Web-based feedback, we suggest they first make patients aware that they can leave anonymous feedback securely on a website for a GP. They can then convince them that their feedback is needed and wanted by GPs for improvement, and that the reviews they leave on the website will be of benefit to other patients to decide which GP to see or which GP practice to join.

The anonymity paradox in patient engagement: reputation, risk and web-based public feedback

The UK National Health Service (NHS) has long espoused patient and public engagement. Recent years have seen increasing use of internet-based methods of collecting feedback about patient experience and public and staff views about NHS services and priorities. Often hailed as a means of facilitating participative democratic patient engagement, these processes raise a number of complex issues. A key aspect of it is the opportunity for comment to be made anonymously. Our research reveals an anonymity paradox whereby patients clearly demonstrate a perception that anonymity is a prerequisite for effective use of these feedback processes, whereas professionals demonstrate a perception that patient anonymity is a barrier to effective use. The risks of anonymity are constructed very differently by patients and professionals. Patient concerns around anonymity were not motivated by a general concern about a loss of privacy, but more that a positive identification might compromise future care. For professionals, concerns were voiced more around risks of reputational damage for specific practitioners or practices (in that anyone could say anything) and also that this anonymous feedback was available publicly and that it might go against the medical opinion of the professional. These concerns pointed to important differences in perceptions of patient and professional vulnerability. In the qualitative analysis that follows the key finding was that while anonymity makes service users feel less vulnerable, it can have the opposite effect on managers and clinical staff. This raises important implications for the use and utility of internet-based methods of collecting patient feedback.