The Content and Nature of Narrative Comments on Swiss Physician Rating Websites: Analysis of 849 Comments

The Credibility of Physician Rating Websites: A Systematic Literature Review

OBJECTIVES

Increasingly, the credibility of online reviews is drawing critical attention due to the lack of control mechanisms, the constant debate about fake reviews and, last but not least, current developments in the field of artificial intelligence. For this reason, the aim of this study was to examine the extent to which assessments recorded on physician rating websites (PRWs) are credible, based on a comparison to other evaluation criteria.

METHODS

Referring to the PRISMA guidelines, a comprehensive literature search was conducted across different scientific databases. Data were synthesized by comparing individual statistical outcomes, objectives and conclusions.

RESULTS

The chosen search strategy led to a database of 36,755 studies of which 28 were ultimately included in the systematic review. The literature review yielded mixed results regarding the credibility of PRWs. While seven publications supported the credibility of PRWs, six publications found no correlation between PRWs and alternative datasets. 15 studies reported mixed results.

CONCLUSIONS

This study has shown that ratings on PRWs seem to be credible when relying primarily on patients' perception. However, these portals seem inadequate to represent alternative comparative values such as the medical quality of physicians. For health policy makers our results show that decisions based on patients' perceptions may be well supported by data from PRWs. For all other decisions, however, PRWs do not seem to contain sufficiently useful data.

Factors Associated with the Patient/Client Use of Report Cards, Physician Rating Websites, Social Media, and Google for Hospital and Physician Selection: A Nationwide Survey

Objective: To explore the factors associated with the different uses of report cards, physician rating websites, social media, and Google, including awareness, physician finding, and decision-making based on reviews from the patient/client perspective. Methods: We used computer-assisted telephone interviews to conduct a nationwide representative survey in Taiwan. Results: The urbanization level of the area, income, and long-term health conditions were not associated with the three kinds of usage of the websites studied. Seeking health information was an important factor in the three kinds of website use. The employment industry was associated with awareness, and education level was associated with physician seeking and actions based on reviews. Conclusions: Different factors influenced the three kinds of usage: awareness, actual use (i.e., finding an appropriate physician), and decision-making based on reviews. Seeking health information is of primary importance regardless of how the websites are used. Practical implications: Policy-makers should focus on educating individuals working outside the health care sector to increase awareness of these websites and to assist individuals with low levels of education in increasing their use of these websites.

Practices and attitudes of Bavarian stakeholders regarding the secondary-use of health data for research purposes during the COVID-19 pandemic: a qualitative interview study (Preprint)

Background

The COVID-19 pandemic is a threat to global health and requires collaborative health research efforts across organizations and countries to address it. Although routinely collected digital health data are a valuable source of information for researchers, benefiting from these data requires accessing and sharing the data. Health care organizations focusing on individual risk minimization threaten to undermine COVID-19 research efforts, and it has been argued that there is an ethical obligation to use the European Union’s General Data Protection Regulation (GDPR) scientific research exemption during the COVID-19 pandemic to support collaborative health research.

Objective

This study aims to explore the practices and attitudes of stakeholders in the German federal state of Bavaria regarding the secondary use of health data for research purposes during the COVID-19 pandemic, with a specific focus on the GDPR scientific research exemption.

Methods

Individual semistructured qualitative interviews were conducted between December 2020 and January 2021 with a purposive sample of 17 stakeholders from 3 different groups in Bavaria: researchers involved in COVID-19 research (n=5, 29%), data protection officers (n=6, 35%), and research ethics committee representatives (n=6, 35%). The transcripts were analyzed using conventional content analysis.

Results

Participants identified systemic challenges in conducting collaborative secondary-use health data research in Bavaria; secondary health data research generally only happens when patient consent has been obtained, or the data have been fully anonymized. The GDPR research exemption has not played a significant role during the pandemic and is currently seldom and restrictively used. Participants identified 3 key groups of barriers that led to difficulties: the wider ecosystem at many Bavarian health care organizations, legal uncertainty that leads to risk-adverse approaches, and ethical positions that patient consent ought to be obtained whenever possible to respect patient autonomy. To improve health data research in Bavaria and across Germany, participants wanted greater legal certainty regarding the use of pseudonymized data for research purposes without the patient’s consent.

Conclusions

The current balance between enabling the positive goals of health data research and avoiding associated data protection risks is heavily skewed toward avoiding risks; so much so that it makes reaching the goals of health data research extremely difficult. This is important, as it is widely recognized that there is an ethical imperative to use health data to improve care. The current approach also creates a problematic conflict with the ambitions of Germany, and the federal state of Bavaria, to be a leader in artificial intelligence. A recent development in the field of German public administration known as norm screening (Normenscreening) could potentially provide a systematic approach to minimize legal barriers. This approach would likely be beneficial to other countries.

Barriers and Facilitating Factors for Conducting Systematic Evidence Assessments in Academic Clinical Trials

IMPORTANCE

A systematic assessment of existing research should justify the conduct and inform the design of new clinical research but is often lacking. There is little research on the barriers to and factors facilitating systematic evidence assessments.

OBJECTIVE

To examine the practices and attitudes of Swiss stakeholders and international funders regarding conducting systematic evidence assessments in academic clinical trials.

DESIGN, SETTING, AND PARTICIPANTS

In this qualitative study, individual semistructured qualitative interviews were conducted between February and August 2020 with 48 Swiss stakeholder groups (27 primary investigators, 9 funders and sponsors, 6 clinical trial support organizations, and 6 ethics committee members) and between January and March 2021 with 9 international funders of clinical trials from North America and Europe with a reputation for requiring systematic evidence synthesis in applications for academic clinical trials.

MAIN OUTCOMES AND MEASURES

The main outcomes were practices and attitudes of Swiss stakeholders and international funders regarding conducting systematic evidence assessments in academic clinical trials. Interviews were analyzed using conventional content analysis.

RESULTS

Of the 57 participants, 40 (70.2%) were male. Participants universally acknowledged that a comprehensive understanding of the previous evidence is important but reported wide variation regarding how this should be achieved. Participants reported that the conduct of formal systematic reviews was currently not expected before most clinical trials, but most international funders reported expecting a systematic search for the existing evidence. Whereas time and resources were reported by all participants as barriers to conducting systematic reviews, the Swiss research ecosystem was reported not to be as supportive of a systematic approach compared with international settings.

CONCLUSIONS AND RELEVANCE

In this qualitative study, Swiss stakeholders and international funders generally agreed that new clinical trials should be justified by a systematic evidence assessment but that barriers on individual, organizational, and political levels kept them from implementing it. More explicit requirements from funders appear to be needed to clarify the required level of comprehensiveness in summarizing existing evidence for different types of clinical trials.

One Decade of Online Patient Feedback: Longitudinal Analysis of Data From a German Physician Rating Website

Background

Feedback from patients is an essential element of a patient-oriented health care system. Physician rating websites (PRWs) are a key way patients can provide feedback online. This study analyzes an entire decade of online ratings for all medical specialties on a German PRW.

Objective

The aim of this study was to examine how ratings posted on a German PRW have developed over the past decade. In particular, it aimed to explore (1) the distribution of ratings according to time-related aspects (year, month, day of the week, and hour of the day) between 2010 and 2019, (2) the number of physicians with ratings, (3) the average number of ratings per physician, (4) the average rating, (5) whether differences exist between medical specialties, and (6) the characteristics of the patients rating physicians.

Methods

All scaled-survey online ratings that were posted on the German PRW jameda between 2010 and 2019 were obtained.

Results

In total, 1,906,146 ratings were posted on jameda between 2010 and 2019 for 127,921 physicians. The number of rated physicians increased constantly from 19,305 in 2010 to 82,511 in 2018. The average number of ratings per rated physicians increased from 1.65 (SD 1.56) in 2010 to 3.19 (SD 4.69) in 2019. Overall, 75.2% (1,432,624/1,906,146) of all ratings were in the best rating category of “very good,” and 5.7% (107,912/1,906,146) of the ratings were in the lowest category of “insufficient.” However, the mean of all ratings was 1.76 (SD 1.53) on the German school grade 6-point rating scale (1 being the best) with a relatively constant distribution over time. General practitioners, internists, and gynecologists received the highest number of ratings (343,242, 266,899, and 232,914, respectively). Male patients, those of higher age, and those covered by private health insurance gave significantly (P<.001) more favorable evaluations compared to their counterparts. Physicians with a lower number of ratings tended to receive ratings across the rating scale, while physicians with a higher number of ratings tended to have better ratings. Physicians with between 21 and 50 online ratings received the lowest ratings (mean 1.95, SD 0.84), while physicians with >100 ratings received the best ratings (mean 1.34, SD 0.47).

Conclusions

This study is one of the most comprehensive analyses of PRW ratings to date. More than half of all German physicians have been rated on jameda each year since 2016, and the overall average number of ratings per rated physicians nearly doubled over the decade. Nevertheless, we could also observe a decline in the number of ratings over the last 2 years. Future studies should investigate the most recent development in the number of ratings on both other German and international PRWs as well as reasons for the heterogeneity in online ratings by medical specialty.

Practices and Attitudes of Swiss Stakeholders Regarding Investigator-Initiated Clinical Trial Funding Acquisition and Cost Management

IMPORTANCE

Randomized clinical trials (RCTs) are an essential method of evaluating health care interventions and a cornerstone for evidence-based health care. However, RCTs have become increasingly complex and costly, which is particularly challenging for independent investigator-initiated clinical trials (IICTs). IICTs have an essential role in clinical research, and it is important that efforts are made to ensure IICTs are adequately funded and are conducted cost-effectively.

OBJECTIVE

To examine the practices and attitudes of Swiss stakeholders regarding IICT funding acquisition and cost management.

DESIGN, SETTING, AND PARTICIPANTS

For this qualitative study, interviews were conducted in Switzerland between February and August 2020. The purposive sample comprised 48 stakeholders from 4 different groups: primary investigators (n = 27), funders and sponsors (n = 9), clinical trial support organizations (n = 6), and ethics committee members (n = 6).

MAIN OUTCOMES AND MEASURES

Practices and attitudes of stakeholders regarding IICT funding acquisition and cost management were assessed using individual semistructured qualitative interviews. Interviews were analyzed using conventional content analysis.

RESULTS

After interviews with 48 IICT stakeholders (75% male presenting), these participants identified a systemic problem of IICTs being underfunded, which can lead to compromises being made regarding the quality and conduct of IICTs. Participants identified 2 overarching and interconnected groups of reasons why IICTs in Switzerland are regularly underfunded. First, it was reported that IICT budget estimations are often inaccurate because of poor planning and preparation, unforeseeable events, investigators intentionally underestimating budgets, and limited budget assessment and oversight. Second, with the exception of a specific IICT funding program by the Swiss National Science Foundation, it was reported that limited funding sources and unrealistic expectation of funders led to underlying challenges in getting IICTs fully funded. A number of measures that could help reduce the underfunding of IICTs were identified, including improving the support of investigators and IICTs, strengthening networking and guidance, harmonizing and simplifying bureaucracy, and increasing public funding of IICTs.

CONCLUSIONS AND RELEVANCE

This study highlights the inadequate expertise of Swiss stakeholders to correctly, systematically, and reproducibly calculate RCT budgets and the need for transparency on trial costs as well as training in budgeting practices. Limited financial resources for academic clinical research and issues regarding the professional planning and conduct of IICTs are persistent issues that many other countries also face.

Correlation of Online Physician Rating Subscores and Association With Overall Satisfaction: Observational Study of 212,933 Providers

BACKGROUND

Online physician rating websites commonly ask consumers to rate providers across multiple physician-based (eg, spending sufficient time, listening) and office-based (eg, appointment scheduling, friendliness) subdimensions of care in addition to overall satisfaction. However, it is unclear if consumers can differentiate between the various rated subdimensions of physicians. It is also unclear how each subdimension is related to overall satisfaction.

OBJECTIVE

The objectives of our study were to determine the correlation of physician-based and office-based subdimensions of care and the association of each with overall satisfaction.

METHODS

We sampled 212,933 providers from the Healthgrades website and calculated average provider metrics for overall satisfaction (likelihood to recommend doctor), physician-based subdimensions (trust in physician, ability to explain, ability to listen and answer questions, and spending adequate time), and office-based subdimensions (ease of scheduling, office environment, staff friendliness, and wait time). We used Spearman rank correlation to assess correlation between subdimension ratings. Factor analysis was used to identify potential latent factors predicting overall satisfaction. Univariate and multivariable linear regression were performed to assess the effect of physician and office-based factors on overall satisfaction.

RESULTS

Physician-based metrics were highly correlated with each other (r=.95 to .98, P<.001), as were office-based metrics (r=.84 to .88, P<.001). Correlations between physician-based and office-based ratings were less robust (r=.79 to .81, P<.001). Factor analysis identified two factors, clearly distinguishing between physician-based metrics (factor loading = 0.84 to 0.88) and office-based metrics (factor loading = 0.76 to 0.84). In multivariable linear regression analysis, the composite factor representing physician-based metrics (0.65, 95% CI 0.65 to 0.65) was more strongly associated with overall satisfaction than the factor representing office-based metrics (0.42, 95% CI 0.42 to 0.42). These factors eclipsed other demographic variables in predicting overall satisfaction.

CONCLUSIONS

Consumers do not differentiate between commonly assessed subdimensions of physician-based care or subdimensions of office-based care, but composite factors representing these broader categories are associated with overall satisfaction. These findings argue for a simpler ratings system based on two metrics: one addressing physician-based aspects of care and another addressing office-based aspects of care.

Rejected Online Feedback From a Swiss Physician Rating Website Between 2008 and 2017: Analysis of 2352 Ratings

Background

Previous research internationally has only analyzed publicly available feedback on physician rating websites (PRWs). However, it appears that many PRWs are not publishing all the feedback they receive. Analysis of this rejected feedback could provide a better understanding of the types of feedback that are currently not published and whether this is appropriate.

Objective

The aim of this study was to examine (1) the number of patient feedback rejected from the Swiss PRW Medicosearch, (2) the evaluation tendencies of the rejected patient feedback, and (3) the types of issues raised in the rejected narrative comments.

Methods

The Swiss PRW Medicosearch provided all the feedback that had been rejected between September 16, 2008, and September 22, 2017. The feedback were analyzed and classified according to a theoretical categorization framework of physician-, staff-, and practice-related issues.

Results

Between September 16, 2008, and September 22, 2017, Medicosearch rejected a total of 2352 patient feedback. The majority of feedback rejected (1754/2352, 74.6%) had narrative comments in the German language. However, 11.9% (279/2352) of the rejected feedback only provided a quantitative rating with no narrative comment. Overall, 25% (588/2352) of the rejected feedback were positive, 18.7% (440/2352) were neutral, and 56% (1316/2352) were negative. The average rating of the rejected feedback was 2.8 (SD 1.4). In total, 44 subcategories addressing the physician (n=20), staff (n=9), and practice (n=15) were identified. In total, 3804 distinct issues were identified within the 44 subcategories of the categorization framework; 75% (2854/3804) of the issues were related to the physician, 6.4% (242/3804) were related to the staff, and 18.6% (708/3804) were related to the practice. Frequently mentioned issues identified from the rejected feedback included (1) satisfaction with treatment (533/1903, 28%); (2) the overall assessment of the physician (392/1903, 20.6%); (3) recommending the physician (345/1903, 18.1%); (4) the physician’s communication (261/1903, 13.7%); (5) the physician’s caring attitude (220/1903, 11.6%); and (6) the physician’s friendliness (203/1903, 10.6%).

Conclusions

It is unclear why the majority of the feedback were rejected. This is problematic and raises concerns that online patient feedback are being inappropriately manipulated. If online patient feedback is going to be collected, there needs to be clear policies and practices about how this is handled. It cannot be left to the whims of PRWs, who may have financial incentives to suppress negative feedback, to decide which feedback is or is not published online. Further research is needed to examine how many PRWs are using criteria for determining which feedback is published or not, what those criteria are, and what measures PRWs are using to address the manipulation of online patient feedback.