Online physician ratings fail to predict actual performance on measures of quality, value, and peer review

Gender and personalized profile information influence online ratings of Canadian academic ophthalmologists

OBJECTIVE

To determine the characteristics associated with higher online ratings of academic ophthalmologists in Canada.

DESIGN

Retrospective cross-sectional study.

METHODS

All ophthalmologists affiliated with Canadian ophthalmology departments were queried in March 2023 using WebMDs. Online ratings and physician profile details were extracted and descriptively analyzed using nonparametric tests with significance at p < 0.05. Subgroup analysis was conducted using ≥ 4-star rated profiles.

RESULTS

Eight hundred and ninety-nine department faculty from 15 institutions were considered, and 660 ophthalmologists with active, rated profiles were included. A total of 27,823 online ratings with a median of 4.14 stars (out of 5) were observed. Most profiles were of men (74.1%). Women received lower overall ratings compared to men (median = 4.08 vs. 4.20; p = 0.021), and lower number of reviews (median = 23 vs. 34; p < 0.001). Most profiles included office addresses (87.9%), private practice affiliation (79.8%), and contact information (51.1%). There was a positive correlations between higher ratings and profiles that included biographies (rho = 0.13; p = 0.001), languages spoken (rho = 0.15; p < 0.001), educational background (rho = 0.13; p < 0.001), areas of expertise (rho = 0.10; p = 0.010), awards (rho = 0.12; p = 0.002), and among physicians indicating they accept new patients (rho = 0.15; p < 0.001) and accommodate virtual visits (rho = 0.09; p = 0.020).

CONCLUSIONS

Canadian ophthalmologists having certain personal information on their online profiles tended to have higher ratings, despite weak associations, possibly due to wider public outreach. Women had fewer and lower overall ratings compared to men. Further research about online ratings' influence on physician selection and physician career satisfaction is needed.

Decision Making of Healthcare Consumers Based on Factors Associated with Online Review and Ratings

The study deals with understanding of the factors associated with online ratings and reviews that help the healthcare consumers in better decision making. Data was collected from 303 participants using mixed methods. The results indicate the factors that help in decision making which includes use of ratings as a reference, impact of positive and negative ratings, whether these ratings depict the actual worth of healthcare service provider and the relevance of these websites in decision making. This study concludes that there is significant relevance of online rating and review websites on healthcare consumers in selecting a healthcare provider.

Online Medical Consultation Service-Oriented Recommendations: Systematic Review

BACKGROUND

Online health communities have given rise to a new e-service known as online medical consultations (OMCs), enabling remote interactions between physicians and patients. To address challenges, such as patient information overload and uneven distribution of physician visits, online health communities should develop OMC-oriented recommenders.

OBJECTIVE

We aimed to comprehensively investigate what paradigms lead to the success of OMC-oriented recommendations.

METHODS

A literature search was conducted through e-databases, including PubMed, ACM Digital Library, Springer, and ScienceDirect, from January 2011 to December 2023. This review included all papers directly and indirectly related to the topic of health care-related recommendations for online services.

RESULTS

The search identified 611 articles, of which 26 (4.3%) met the inclusion criteria. Despite the growing academic interest in OMC recommendations, there remains a lack of consensus among researchers on the definition of e-service-oriented recommenders. The discussion highlighted 3 key factors influencing recommender success: features, algorithms, and metrics. It advocated for moving beyond traditional e-commerce-oriented recommenders to establish an innovative theoretical framework for e-service-oriented recommenders and addresses critical technical issues regarding 2-sided personalized recommendations.

CONCLUSIONS

This review underscores the essence of e-services, particularly in knowledge- and labor-intensive domains such as OMCs, where patients seek interpretable recommendations due to their lack of domain knowledge and physicians must balance their energy levels to avoid overworking. Our study's findings shed light on the importance of customizing e-service-oriented personalized recommendations to meet the distinct expectations of 2-sided users considering their cognitive abilities, decision-making perspectives, and preferences. To achieve this, a paradigm shift is essential to develop unique attributes and explore distinct content tailored for both parties involved.

Examining the Role of Physician Characteristics in Web-Based Verified Primary Care Physician Reviews: Observational Study

BACKGROUND

Doctor review websites have become increasingly popular as a source of information for patients looking to select a primary care provider. Zocdoc is one such platform that allows patients to not only rate and review their experiences with doctors but also directly schedule appointments. This study examines how several physician characteristics including gender, age, race, languages spoken in a physician's office, education, and facial attractiveness impact the average numerical rating of primary care doctors on Zocdoc.

OBJECTIVE

The aim of this study was to investigate the association between physician characteristics and patient satisfaction ratings on Zocdoc.

METHODS

A data set of 1455 primary care doctor profiles across 30 cities was scraped from Zocdoc. The profiles contained information on the physician's gender, education, and languages spoken in their office. Age, facial attractiveness, and race were imputed from profile pictures using commercial facial analysis software. Each doctor profile listed an average overall satisfaction rating, bedside manner rating, and wait time rating from verified patients. Descriptive statistics, the Wilcoxon rank sum test, and multivariate logistic regression were used to analyze the data.

RESULTS

The average overall rating on Zocdoc was highly positive, with older age, lower facial attractiveness, foreign degrees, allopathic degrees, and speaking more languages negatively associated with the average rating. However, the effect sizes of these factors were relatively small. For example, graduates of Latin American medical schools had a mean overall rating of 4.63 compared to a 4.77 rating for US graduates (P<.001), a difference roughly equivalent to a 2.8% decrease in appointments. On multivariate analysis, being Asian and having a doctor of osteopathic medicine degree were positively associated with higher overall ratings, while attending a South Asian medical school and speaking more European and Middle Eastern languages in the office were negatively associated with higher overall ratings.

CONCLUSIONS

Overall, the findings suggest that age, facial attractiveness, education, and multilingualism do have some impact on web-based doctor reviews, but the numerical effect is small. Notably, bias may play out in many forms. For example, a physician's appearance or accent may impact a patient's trust, confidence, or satisfaction with their physician, which could in turn influence their take-up of preventative services and lead to either better or worse health outcomes. The study highlights the need for further research in how physician characteristics influence patient ratings of care.

Assessing extremely negative online patient reviews and complaints of musculoskeletal oncology surgeons in the United States: a retrospective analysis

INTRODUCTION

Physician-review websites (PRWs) are commonly used by patients while searching for a surgeon. There is no current literature investigating the factors that contribute to online one-star reviews of musculoskeletal oncology surgeons. This retrospective study aims to identify these factors to determine areas of care affecting patient's subjective reviews.

METHODS

Patient ratings and comments regarding musculoskeletal oncology surgeons from the Musculoskeletal Tumor Society (MSTS) were collected from Vitals.com. One-star reviews with comments were then classified as either operative or nonoperative. These complaints were then further classified based on content including wait time, uncontrolled pain, time spent with the physician, surgical outcomes, medical staff/institutional complaints, and bedside manner.

RESULTS

A total of 169 reviews (375 complaints) from 181 physicians were included. Of these complaints, 198 were from patients in the operative category while 177 were from patients in the nonoperative category. Bedside manner was the most common complaint. Operative patients reported higher instances of uncontrolled pain in their reviews, whereas nonoperative patients more frequently cited wait time. No significant difference in the complaints that mentioned the amount of time spent with the physician, bedside manner, a disagreement with the plan, or the medical staff or institution was found.

CONCLUSION

Online one-star reviews of musculoskeletal oncology surgeons on Vitals.com referenced both surgical and non-surgical aspects of patient encounters, with bedside manner being the most popular complaint overall. Surgical patients were more likely to complain of uncontrolled pain whereas non-operative patients were more likely to complain of wait time.

TYPE OF STUDY

Outcomes 2c.

A typology of evaluative health platforms: Commercial interests and their implications for patient voice

Interactions in the healthcare system today involve an important new set of actors: evaluative health platforms (EHPs). These platforms are not neutral intermediaries, but active moderators of how patients express opinions, choose providers, and consume health-related information. This paper adds to our understanding of the varied and evolving commercial interests of EHPs and the implications these have for patient voice. We analyze 71 platforms in the USA, UK, and Germany and identify five ideal types: subscribers, analyzers, advertisers, regulators, and scammers. Each platform type enacts a unique competitive strategy through an evaluative infrastructure which constrains but also generates possibilities for patient voice. Based on our typology, we develop three contributions. First, we nuance universalizing claims about the consequences of platform capitalism by specifying the diverse strategies underpinning competition between EHPs in different countries, and showing how each strategy leads evaluative infrastructures to develop in ways that impact patient voice. Second, we show how patients can navigate the challenges of a complex EHP space by exercising their ability to choose between platforms. Finally, we outline the conditions platforms need to fulfil to become empowering. Overall, this study highlights the varied and complex relationship between platform business models and user voice, which exists not only in healthcare, but also in many other fields.

Application of Patient Sentiment Analysis to Evaluate Glaucoma Care

PURPOSE

Patients utilize online physician reviews to decide between and rate ophthalmologists. Sentiment analysis allows for better understanding of patient experiences. In this study, Valence Aware Dictionary sEntiment Reasoner (VADER) and word frequency analysis of glaucoma specialist Healthgrades reviews were used to determine factors prioritized by patients.

DESIGN

Retrospective cross-sectional analysis.

PARTICIPANTS

N/A.

METHODS

Written reviews and Star ratings of glaucoma specialists listed under the Physicians Payments Sunshine Acts were obtained, and demographic information was collected. Valence Aware Dictionary sEntiment Reasoner produced Negative, Neutral, Positive, and Compound scores of reviews, and these were stratified by demographic variables. Word frequency review was applied to determine popular words and phrases.

MAIN OUTCOME MEASURES

Star ratings, VADER Compound score of written reviews, and highest word frequencies.

RESULTS

A total of 203 glaucoma specialists and 3531 written reviews were assessed. Glaucoma specialists had an average of 4.26/5 stars, with a mean of 30 ratings per physician on Healthgrades. Most physicians (86%) had overall Positive written reviews (VADER = 0.74), indicating high patient satisfaction. Specialists who were women or had fewer years of practice had higher Compound and Star scores than their respective male and senior counterparts, with statistical significance observed between junior and senior physician Stars (P < 0.001). Repeated words pertaining to the surgery, staff, wait times, and questions were common overall and among the most positive and most negative reviews.

CONCLUSIONS

Glaucoma specialist patients value nonclinical factors, such as appointment setting and nonphysician health-care staff members, in their written reviews. Thus, factors beyond clinical outcomes are influential in the overall patient experience and should be considered to improve health-care delivery. These results can also advise ophthalmologists on factors that patients prioritize when evaluating physicians, which influences the decisions of other patients seeking glaucoma care.

FINANCIAL DISCLOSURE(S)

The author(s) have no proprietary or commercial interest in any materials discussed in this article.

Online ratings and narrative comments of American Head and Neck Society surgeons

BACKGROUND

We analyzed online rating scores and comments of head and neck surgeons to understand factors that contribute to higher ratings.

METHODS

Numerical ratings and comments for American Head and Neck Society physicians were extracted from Healthgrades, Vitals, RateMDs, and Yelp, with narrative comments categorized based on content. Physician practice location, education, and residency training were also compiled.

RESULTS

Patient ratings were significantly higher with supportive staff and affable physician demeanor but showed significant drops with longer wait times and difficulties scheduling appointments or follow-ups. Physician education and postgraduate training did not significantly affect ratings.

CONCLUSION

Online ratings and comments correlated to modifiable factors in clinical practice and may be informative in understanding patient needs.

Validity of patients' online reviews at direct-to-consumer teleconsultation platforms: a protocol for a cross-sectional study using unannounced standardised patients

INTRODUCTION

As direct-to-consumer teleconsultation (hereafter referred to as 'teleconsultation') has gained popularity, an increasing number of patients have been leaving online reviews of their teleconsultation experiences. These reviews can help guide patients in identifying doctors for teleconsultation. However, few studies have examined the validity of online reviews in assessing the quality of teleconsultation against a gold standard. Therefore, we aim to use unannounced standardised patients (USPs) to validate online reviews in assessing both the technical and patient-centred quality of teleconsultations. We hypothesise that online review results will be more consistent with the patient-centred quality, rather than the technical quality, as assessed by the USPs.

METHODS AND ANALYSIS

In this cross-sectional study, USPs representing 11 common primary care conditions will randomly visit 253 physicians via the three largest teleconsultation platforms in China. Each physician will receive a text-based and a voice/video-based USP visit, resulting in a total of 506 USP visits. The USP will complete a quality checklist to assess the proportion of clinical practice guideline-recommended items during teleconsultation. After each visit, the USP will also complete the Patient Perception of Patient-Centeredness Rating. The USP-assessed results will be compared with online review results using the intraclass correlation coefficient (ICC). If ICC >0.4 (p<0.05), we will assume reasonable concordance between the USP-assessed quality and online reviews. Furthermore, we will use correlation analysis, Lin's Coordinated Correlation Coefficient and Kappa as supplementary analyses.

ETHICS AND DISSEMINATION

This study has received approval from the Institutional Review Board of Southern Medical University (#Southern Medical Audit (2022) No. 013). Results will be actively disseminated through print and social media, and USP tools will be made available for other researchers.

TRIAL REGISTRATION

The study has been registered at the China Clinical Trials Registry (ChiCTR2200062975).

The Measurement Reliability of Complications and Patient Satisfaction After Common Surgical Procedures

OBJECTIVE

To evaluate the reliability of surgeon outcomes.

BACKGROUND

Surgeons' outcomes are now widely used in public reporting and value-based reimbursement, but the reliability of these measures continues to raise concerns.

METHODS

We performed a retrospective study of surgeons performing cholecystectomy, colectomy, and hernia repair on adult patients between January 1, 2017, and December 31, 2020. Outcomes were risk-adjusted rates of complications and high patient satisfaction. We estimated the reliability of each outcome, its relationship with case volume, and the number of surgeons who reached an acceptable level of reliability (≥0.70).

RESULTS

A total of 23,533 patients with a mean age of 56.8 (16.2) years and 10,191 (43.3%) females underwent operations by 333 surgeons. Risk-adjusted complication rate was 2.5% [95% confidence interval (CI): 2.2%-2.8%] and risk-adjusted high satisfaction rate was 79.9% (95% CI: 78.7%-81.0%). The reliability of the complication rate was 0.27 (95% CI: 0.25-0.29) and the reliability of the high satisfaction rate was 0.53 (95% CI: 0.50-0.55). Reliability increased with case volume; however, only 5 (1.5%) surgeons performed enough cases to reach acceptable reliability for their complication rate, while 86 (25.8%) surgeons reached acceptable reliability for their patient satisfaction rate. After adjustment for reliability, the range of complication rates decreased 29-fold from 0% to 14.3% to 2.4% to 2.9%, and the range of patient satisfaction decreased 2.6-fold from 25.3% to 100.0% to 64.9% to 92.4%.

CONCLUSIONS

Among surgeons performing common operations, complications and patient satisfaction had relatively low reliability. Although reliability increased with volume, most surgeons had insufficient case volume to achieve acceptable reliability of their outcomes. As such, these measures likely offer little to no meaningful information to inform decision-making.

The Credibility of Physician Rating Websites: A Systematic Literature Review

OBJECTIVES

Increasingly, the credibility of online reviews is drawing critical attention due to the lack of control mechanisms, the constant debate about fake reviews and, last but not least, current developments in the field of artificial intelligence. For this reason, the aim of this study was to examine the extent to which assessments recorded on physician rating websites (PRWs) are credible, based on a comparison to other evaluation criteria.

METHODS

Referring to the PRISMA guidelines, a comprehensive literature search was conducted across different scientific databases. Data were synthesized by comparing individual statistical outcomes, objectives and conclusions.

RESULTS

The chosen search strategy led to a database of 36,755 studies of which 28 were ultimately included in the systematic review. The literature review yielded mixed results regarding the credibility of PRWs. While seven publications supported the credibility of PRWs, six publications found no correlation between PRWs and alternative datasets. 15 studies reported mixed results.

CONCLUSIONS

This study has shown that ratings on PRWs seem to be credible when relying primarily on patients' perception. However, these portals seem inadequate to represent alternative comparative values such as the medical quality of physicians. For health policy makers our results show that decisions based on patients' perceptions may be well supported by data from PRWs. For all other decisions, however, PRWs do not seem to contain sufficiently useful data.

Building better pediatric surgeons: A sentiment analysis of online physician review websites

PURPOSE

Physician review websites are a heavily utilized patient tool for finding, rating, and reviewing surgeons. Natural language processing such as sentiment analysis provides a comprehensive approach to better understand the nuances of patient perception. This study utilizes sentiment analysis to examine how specific patient sentiments correspond to positive and negative experiences in online reviews of pediatric orthopedic surgeons.

METHODS

The online written reviews and star ratings of pediatric surgeons belonging to the Pediatric Orthopaedic Society of North America were obtained from healthgrades.com. A sentiment analysis package obtained compound scores of each surgeon's reviews. Inferential statistics analyzed relationships between demographic variables and star/sentiment scores. Word frequency analyses and multiple logistic regression analyses were performed on key terms.

RESULTS

A total of 749 pediatric surgeons (3830 total online reviews) were included. 80.8% were males and 33.8% were below 50 years of age. Male surgeons and younger surgeons had higher mean star ratings. Surgeon attributes including "confident" (p < 0.01) and "comfortable" (p < 0.01) improved the odds of positive reviews, while "rude" (p < 0.01) and "unprofessional" (p < 0.01) decreased these odds. Comments regarding "pain" lowered the odds of positive reviews (p < 0.01), whereas "pain-free" increased these odds (p < 0.01).

CONCLUSION

Pediatric surgeons who were younger, communicated effectively, eased pain, and curated a welcoming office setting were more likely to receive positively written online reviews. This suggests that a spectrum of interpersonal and ancillary factors impact patient experience and perceptions beyond surgical skill. These outcomes can advise pediatric surgeons on behavioral and office qualities that patients and families prioritize when rating/recommending surgeons online.

LEVEL OF EVIDENCE

IV.

Rating Spine Surgeons: Physician Review Websites Versus a Patient-reported Outcomes-derived Ranking

STUDY DESIGN

This was an observational study.

OBJECTIVES

This study aims to determine the correlation between patient-reported outcomes (PROs) pulled from a national spine registry and physician ratings from physician review websites (PRWs).

SUMMARY OF BACKGROUND DATA

PRWs are frequently utilized by patients to make health care decisions; however, many PRWs appear to incorporate subjective experiences unrelated to a surgeon's clinical performance into ratings. As such, their utility as a health care decision-making tool remains unclear.

MATERIALS AND METHODS

This study evaluated 8834 patients from the Quality Outcomes Database (QOD) who underwent 1-level elective lumbar spine surgery. The lumbar module of QOD was queried to rank 124 surgeons using PROs (Oswestry Disability Index, EuroQOL, Numerical Rating Scale-back/leg pain, and patient satisfaction). The QOD PRO-ranking system was compared against PRWs including Healthgrades, Vitals, WebMD, and Google. The Spearman correlation coefficients, Kruskal-Wallis tests, and multiple linear regression models were used for statistical comparison. The primary outcome was the correlation between PRW scores and PROs.

RESULTS

Surgeon PRO-derived ranking showed high intercorrelational congruence with coefficients between the 3 PROs (Oswestry Disability Index, EuroQOL, Numerical Rating Scale back/leg) ranging between 0.70 and 0.88. Low correlations were observed between PRO-derived rankings and PRWs, ranging from 0.23 to 0.37. Healthgrades performed most similarly to PRO-derived rankings, correlating best with patient satisfaction, though the correlation was low (ρ=0.37).

CONCLUSIONS

While PRWs are often used to evaluate surgeon competency, these results demonstrate they poorly correlate with a surgeon's clinical ability measured by PROs. PRWs should be used with caution when making health care decisions by patients, payers, and administrators.

LEVEL OF EVIDENCE

Level III.

How Referring Providers Choose Specialists for Their Patients: a Systematic Review

BACKGROUND

Physician referrals are a critical step in directing patients to high-quality specialists. Despite efforts to encourage referrals to high-volume hospitals, many patients receive treatment at low-volume centers with worse outcomes. We aimed to determine the most important factors considered by referring providers when selecting specialists for their patients through a systematic review of medical and surgical literature.

METHODS

PubMed and Embase were searched from January 2000 to July 2021 using terms related to referrals, specialty, surgery, primary care, and decision-making. We included survey and interview studies reporting the factors considered by healthcare providers as they refer patients to specialists in the USA. Studies were screened by two independent reviewers. Quality was assessed using the CASP Checklist. A qualitative thematic analysis was performed to synthesize common decision factors across studies.

RESULTS

We screened 1,972 abstracts and identified 7 studies for inclusion, reporting on 1,575 providers. Thematic analysis showed that referring providers consider factors related to the specialist's clinical expertise (skill, training, outcomes, and assessments), interactions between the patient and specialist (prior experience, rapport, location, scheduling, preference, and insurance), and interactions between the referring physician and specialist (personal relationships, communication, reputation, reciprocity, and practice or system affiliation). Notably, studies did not describe how providers assess clinical or technical skills.

CONCLUSIONS

Referring providers rely on subjective factors and assessments to evaluate quality when selecting a specialist. There may be a role for guidelines and objective measures of quality to inform the choice of specialist by referring providers.

Use of Online Patient Reviews to Assess Medical Oncologist Competency: Mixed-Method Sequential Explanatory Study (Preprint)

BACKGROUND

Patients increasingly use web-based evaluation tools to assess their physicians, health care teams, and overall medical experience.

OBJECTIVE

This study aimed to evaluate the extent to which the standardized physician competencies of the CanMEDS Framework are present in web-based patient reviews (WPRs) and to identify patients' perception of important physician qualities in the context of quality cancer care.

METHODS

The WPRs of all university-affiliated medical oncologists in midsized cities with medical schools in the province of Ontario (Canada) were collected. Two reviewers (1 communication studies researcher and 1 health care professional) independently assessed the WPRs according to the CanMEDS Framework and identified common themes. Comment scores were then evaluated to identify κ agreement rates between the reviewers, and a descriptive quantitative analysis of the cohort was completed. Following the quantitative analysis, an inductive thematic analysis was performed.

RESULTS

This study identified 49 actively practicing university-affiliated medical oncologists in midsized urban areas in Ontario. A total of 473 WPRs reviewing these 49 physicians were identified. Among the CanMEDS competencies, those defining the roles of medical experts, communicators, and professionals were the most prevalent (303/473, 64%; 182/473, 38%; and 129/473, 27%, respectively). Common themes in WPRs include medical skill and knowledge, interpersonal skills, and answering questions (from the patient to the physician). Detailed WPRs tend to include the following elements: experience and connection; discussion and evaluation of the physician's knowledge, professionalism, interpersonal skills, and punctuality; in positive reviews, the expression of feelings of gratitude and a recommendation; and in negative reviews, discouragement from seeking the physician's care. Patients' perception of medical skills is less specific than their perception of interpersonal qualities, although medical skills are the most commented-on element of care in WPRs. Patients' perception of interpersonal skills (listening, compassion, and overall caring demeanor) and other experiential phenomena, such as feeling rushed during appointments, is often specific and detailed. Details about a physician's interpersonal skills or "bedside manner" are highly perceived, valued, and shareable in an WPR context. A small number of WPRs reflected a distinction between the value of medical skills and that of interpersonal skills. The authors of these WPRs claimed that for them, a physician's medical skills and competence are more important than their interpersonal skills.

CONCLUSIONS

CanMEDS roles and competencies that are explicitly patient facing (ie, those directly experienced by patients in their interactions with physicians and through the care that physicians provide) are the most likely to be present and reported on in WPRs. The findings demonstrate the opportunity to learn from WPRs, not simply to discern physicians' popularity but to grasp what patients may expect from their physicians. In this context, WPRs can represent a method for the measurement and assessment of patient-facing physician competency.

Patient Satisfaction Scores Are a Poor Metric of Orthopedic Care

Patient satisfaction scores are a popular metric used to evaluate orthopedic care. There is little consistency with how satisfaction is described in the orthopedic literature. Online physician reviews are a growing trend that directly and indirectly affect a surgeon's reputation. There is little correlation of higher satisfaction with improved surgical outcomes, so rating surgical care may be misguided and possibly dangerous. Patient satisfaction is an important part of the patient-centered care model, so rating systems should directly reflect quality. More research is needed to determine the relationship between patient satisfaction and the delivery of quality care. [Orthopedics. 2022;45(3):e127-e133.].

Comparing the Impact of Online Ratings and Report Cards on Patient Choice of Cardiac Surgeon: Large Observational Study

Background

Patients may use two information sources about a health care provider’s quality: online physician reviews, which are written by patients to reflect their subjective experience, and report cards, which are based on objective health outcomes.

Objective

The aim of this study was to examine the impact of online ratings on patient choice of cardiac surgeon compared to that of report cards.

Methods

We obtained ratings from a leading physician review platform, Vitals; report card scores from Pennsylvania Cardiac Surgery Reports; and information about patients’ choices of surgeons from inpatient records on coronary artery bypass graft (CABG) surgeries done in Pennsylvania from 2008 to 2017. We scraped all reviews posted on Vitals for surgeons who performed CABG surgeries in Pennsylvania during our study period. We linked the average overall rating and the most recent report card score at the time of a patient’s surgery to the patient’s record based on the surgeon’s name, focusing on fee-for-service patients to avoid impacts of insurance networks on patient choices. We used random coefficient logit models with surgeon fixed effects to examine the impact of receiving a high online rating and a high report card score on patient choice of surgeon for CABG surgeries.

Results

We found that a high online rating had positive and significant effects on patient utility, with limited variation in preferences across individuals, while the impact of a high report card score on patient choice was trivial and insignificant. About 70.13% of patients considered no information on Vitals better than a low rating; the corresponding figure was 26.66% for report card scores. The findings were robust to alternative choice set definitions and were not explained by surgeon attrition, referral effect, or admission status. Our results also show that the interaction effect of rating information and a time trend was positive and significant for online ratings, but small and insignificant for report cards.

Conclusions

A patient’s choice of surgeon is affected by both types of rating information; however, over the past decade, online ratings have become more influential, while the effect of report cards has remained trivial. Our findings call for information provision strategies that incorporate the advantages of both online ratings and report cards.

Correlation between social media utilization by academic neurosurgery departments and higher online patient ratings

OBJECTIVE

Patients increasingly utilize online physician review websites (PRWs) and social media to inform healthcare-related decisions. This provides neurosurgeons with opportunities for increased patient engagement. And despite the growing use of social media among neurosurgeons, the relationship between social media utilization and online reviews remains unknown. The goal of this study was to characterize the relationship between social media utilization and PRW ratings across academic neurosurgery departments.

METHODS

Social media accounts (Twitter, Facebook, YouTube, Instagram) of academic neurosurgery departments were identified. Online reviews for individual faculty were obtained from Healthgrades, Vitals, WebMD, and Google. Reviews were aggregated to identify the total number of reviews per department, to generate a composite departmental rating, and to calculate a summed departmental score. US News & World Report (USNWR) and Doximity rankings were recorded for each department. Social media utilization by individual neurosurgeons and associated ratings were investigated within the departments with the highest social media utilization.

RESULTS

Seventy-eight percent of academic neurosurgery departments utilized social media. The most prevalent platform was YouTube (49.1%), followed by Twitter (46.5%), Facebook (38.6%), and Instagram (16.7%). Higher patient ratings on PRWs were associated with the utilization of YouTube (p = 0.048) or Twitter (p = 0.02). The number of social media platforms utilized demonstrated a significant, positive correlation with patient ratings (p = 0.006) and summed patient ratings (p = 0.048). Although USNWR (p = 0.02) and Doximity (p = 0.0008) rankings correlated with patient ratings, only the number of social media platforms utilized remained a significant predictor of patient ratings on multivariate analysis (p = 0.0001). Thirty-one percent of academic neurosurgeons from departments with high social media utilization were active on social media. The most prevalent social media platform among individual neurosurgeons was Twitter (27.4%), followed by Instagram (8.4%), Facebook (4.9%), and YouTube (2.2%). Higher summed patient scores were associated with individual neurosurgeon utilization of YouTube (p = 0.04), Facebook (p < 0.0001), and Instagram (p = 0.01). Increased social media utilization among neurosurgeons was correlated with a greater number of patient reviews (p = 0.006) and higher summed patient scores (p = 0.003). On multivariate analysis, only Facebook use remained a significant predictor of the number of patient reviews received (p = 0.002) and summed patient satisfaction scores (p < 0.001).

CONCLUSIONS

An increased social media presence is associated with higher ratings on PRWs. As neurosurgeons continue to expand their online presence, they should be aware of the possible impact of social media on online patient reviews.

Questioning the Yelp Effect: Mixed Methods Analysis of Web-Based Reviews of Urgent Cares

Background

Providers of on-demand care, such as those in urgent care centers, may prescribe antibiotics unnecessarily because they fear receiving negative reviews on web-based platforms from unsatisfied patients—the so-called Yelp effect. This effect is hypothesized to be a significant driver of inappropriate antibiotic prescribing, which exacerbates antibiotic resistance.

Objective

In this study, we aimed to determine the frequency with which patients left negative reviews on web-based platforms after they expected to receive antibiotics in an urgent care setting but did not.

Methods

We obtained a list of 8662 urgent care facilities from the Yelp application programming interface. By using this list, we automatically collected 481,825 web-based reviews from Google Maps between January 21 and February 10, 2019. We used machine learning algorithms to summarize the contents of these reviews. Additionally, 200 randomly sampled reviews were analyzed by 4 annotators to verify the types of messages present and whether they were consistent with the Yelp effect.

Results

We collected 481,825 reviews, of which 1696 (95% CI 1240-2152) exhibited the Yelp effect. Negative reviews primarily identified operations issues regarding wait times, rude staff, billing, and communication.

Conclusions

Urgent care patients rarely express expectations for antibiotics in negative web-based reviews. Thus, our findings do not support an association between a lack of antibiotic prescriptions and negative web-based reviews. Rather, patients’ dissatisfaction with urgent care was most strongly linked to operations issues that were not related to the clinical management plan.

Patient Experience Surveys Reveal Gender-Biased Descriptions of Their Care Providers

Patient experience surveys (PES) are collected by healthcare systems as a surrogate marker of quality and published unedited online for the purpose of transparency, but these surveys may reflect gender biases directed toward healthcare providers. This retrospective study evaluated PES at a single university hospital between July 2016 and June 2018. Surveys were stratified by overall provider rating and self-identified provider gender. Adjectives from free-text survey comments were extracted using natural language processing techniques and applied to a statistical machine learning model to identify descriptors predictive of provider gender. 109,994 surveys were collected, 17,395 contained free-text comments describing 687 unique providers. The mean overall rating between male (8.84, n = 8558) and female (8.80, n = 8837) providers did not differ (p = 0.149). However, highly-rated male providers were more often described for their agentic qualities using adjectives such as "informative," "forthright," "superior," and "utmost" (OR 1.48, p < 0.01)-whereas highly-rated female providers were more often described by their communal qualities through adjectives such as "empathetic," "sweet," "warm," "attentive," and "approachable" (OR 2.11, p < 0.0001). PES may contain gender stereotypes, raising questions about their impact on physicians and their validity as a quality metric which must be balanced with the need for unedited transparency. Future prospective studies are needed to further characterize this trend across geographically and racially diverse healthcare providers.

What Do Patients Care About? Mining Fine-grained Patient Concerns from Online Physician Reviews Through Computer-Assisted Multi-level Qualitative Analysis

Online physician review (OPR) websites have been increasingly used by healthcare consumers to make informed decisions in selecting healthcare providers. However, consumer-generated online reviews are often unstructured and contain plural topics with varying degrees of granularity, making it challenging to analyze using conventional topic modeling techniques. In this paper, we designed a novel natural language processing pipeline incorporating qualitative coding and supervised and unsupervised machine learning. Using this method, we were able to identify not only coarse-grained topics (e.g., relationship, clinic management), but also fine-grained details such as diagnosis, timing and access, and financial concerns. We discuss how healthcare providers could improve their ratings based on consumer feedback. We also reflect on the inherent challenges of analyzing user-generated online data, and how our novel pipeline may inform future work on mining consumer-generated online data.

Are Online Reviews of Physicians Reliable Indicators of Clinical Outcomes? A Focus on Chronic Disease Management

Are Online Reviews of Physicians Reliable Indicators of Clinical Outcomes? A Focus on Chronic Disease Management

Comparing Precision Machine Learning With Consumer, Quality, and Volume Metrics for Ranking Orthopedic Surgery Hospitals: Retrospective Study

Background

Patients’ choices of providers when undergoing elective surgeries significantly impact both perioperative outcomes and costs. There exist a variety of approaches that are available to patients for evaluating between different hospital choices.

Objective

This paper aims to compare differences in outcomes and costs between hospitals ranked using popular internet-based consumer ratings, quality stars, reputation rankings, average volumes, average outcomes, and precision machine learning–based rankings for hospital settings performing hip replacements in a large metropolitan area.

Methods

Retrospective data from 4192 hip replacement surgeries among Medicare beneficiaries in 2018 in a the Chicago metropolitan area were analyzed for variations in outcomes (90-day postprocedure hospitalizations and emergency department visits) and costs (90-day total cost of care) between hospitals ranked through multiple approaches: internet-based consumer ratings, quality stars, reputation rankings, average yearly surgical volume, average outcome rates, and machine learning–based rankings. The average rates of outcomes and costs were compared between the patients who underwent surgery at a hospital using each ranking approach in unadjusted and propensity-based adjusted comparisons.

Results

Only a minority of patients (1159/4192, 27.6% to 2078/4192, 49.6%) were found to be matched to higher-ranked hospitals for each of the different approaches. Of the approaches considered, hip replacements at hospitals that were more highly ranked by consumer ratings, quality stars, and machine learning were all consistently associated with improvements in outcomes and costs in both adjusted and unadjusted analyses. The improvement was greatest across all metrics and analyses for machine learning–based rankings.

Conclusions

There may be a substantive opportunity to increase the number of patients matched to appropriate hospitals across a broad variety of ranking approaches. Elective hip replacement surgeries performed at hospitals where patients were matched based on patient-specific machine learning were associated with better outcomes and lower total costs of care.

Negative Patient Reviews and Online Defamation: A Guide for the Obstetrician-Gynecologist

Physician review websites can serve as a means for patients to address physician actions, both good and bad. At times, patient reviews may be negative and perceived as defamatory. Yet there are specific legal requirements that must be met for an online review to be considered defamatory. Prior cases indicate that pursuing a defamation lawsuit, although appropriate in some circumstances, is oftentimes a difficult and futile endeavor. To avoid litigation, physicians can take proactive approaches to address questionable reviews and to bolster their online reputations.

Correlation of Online Physician Rating Subscores and Association With Overall Satisfaction: Observational Study of 212,933 Providers

BACKGROUND

Online physician rating websites commonly ask consumers to rate providers across multiple physician-based (eg, spending sufficient time, listening) and office-based (eg, appointment scheduling, friendliness) subdimensions of care in addition to overall satisfaction. However, it is unclear if consumers can differentiate between the various rated subdimensions of physicians. It is also unclear how each subdimension is related to overall satisfaction.

OBJECTIVE

The objectives of our study were to determine the correlation of physician-based and office-based subdimensions of care and the association of each with overall satisfaction.

METHODS

We sampled 212,933 providers from the Healthgrades website and calculated average provider metrics for overall satisfaction (likelihood to recommend doctor), physician-based subdimensions (trust in physician, ability to explain, ability to listen and answer questions, and spending adequate time), and office-based subdimensions (ease of scheduling, office environment, staff friendliness, and wait time). We used Spearman rank correlation to assess correlation between subdimension ratings. Factor analysis was used to identify potential latent factors predicting overall satisfaction. Univariate and multivariable linear regression were performed to assess the effect of physician and office-based factors on overall satisfaction.

RESULTS

Physician-based metrics were highly correlated with each other (r=.95 to .98, P<.001), as were office-based metrics (r=.84 to .88, P<.001). Correlations between physician-based and office-based ratings were less robust (r=.79 to .81, P<.001). Factor analysis identified two factors, clearly distinguishing between physician-based metrics (factor loading = 0.84 to 0.88) and office-based metrics (factor loading = 0.76 to 0.84). In multivariable linear regression analysis, the composite factor representing physician-based metrics (0.65, 95% CI 0.65 to 0.65) was more strongly associated with overall satisfaction than the factor representing office-based metrics (0.42, 95% CI 0.42 to 0.42). These factors eclipsed other demographic variables in predicting overall satisfaction.

CONCLUSIONS

Consumers do not differentiate between commonly assessed subdimensions of physician-based care or subdimensions of office-based care, but composite factors representing these broader categories are associated with overall satisfaction. These findings argue for a simpler ratings system based on two metrics: one addressing physician-based aspects of care and another addressing office-based aspects of care.

Data Quality Issues With Physician-Rating Websites: Systematic Review

BACKGROUND

In recent years, online physician-rating websites have become prominent and exert considerable influence on patients' decisions. However, the quality of these decisions depends on the quality of data that these systems collect. Thus, there is a need to examine the various data quality issues with physician-rating websites.

OBJECTIVE

This study's objective was to identify and categorize the data quality issues afflicting physician-rating websites by reviewing the literature on online patient-reported physician ratings and reviews.

METHODS

We performed a systematic literature search in ACM Digital Library, EBSCO, Springer, PubMed, and Google Scholar. The search was limited to quantitative, qualitative, and mixed-method papers published in the English language from 2001 to 2020.

RESULTS

A total of 423 articles were screened. From these, 49 papers describing 18 unique data quality issues afflicting physician-rating websites were included. Using a data quality framework, we classified these issues into the following four categories: intrinsic, contextual, representational, and accessible. Among the papers, 53% (26/49) reported intrinsic data quality errors, 61% (30/49) highlighted contextual data quality issues, 8% (4/49) discussed representational data quality issues, and 27% (13/49) emphasized accessibility data quality. More than half the papers discussed multiple categories of data quality issues.

CONCLUSIONS

The results from this review demonstrate the presence of a range of data quality issues. While intrinsic and contextual factors have been well-researched, accessibility and representational issues warrant more attention from researchers, as well as practitioners. In particular, representational factors, such as the impact of inline advertisements and the positioning of positive reviews on the first few pages, are usually deliberate and result from the business model of physician-rating websites. The impact of these factors on data quality has not been addressed adequately and requires further investigation.

Online Ratings for Vascular Interventional Proceduralists Vary by Physician Specialty

BACKGROUND

Multiple specialties offer vascular interventional care, creating potential competition for referrals and procedures. At the same time, patient/consumer ratings have become more impactful for physicians who perform vascular procedures. We hypothesized that there are differences in online ratings based on specialty.

METHODS

We used official program lists from the Association for Graduate Medical Education to identify institutions with training programs in integrated vascular surgery (VS), integrated interventional radiology (IR), and interventional cardiology (IC). Faculty providers were identified in each specialty at these institutions. A standardized search was performed to collect online ratings from Vitals.com, Healthgrades.com, and Google.com as well as from online demographics. Between specialty differences were analyzed using chi-squared and analysis of variance tests as appropriate. Multivariable linear regression was used to identify factors associated with review volume and star rating.

RESULTS

A total of 1,330 providers (n = 454 VS, n = 451 IR, n = 425 IC) were identified across 47 institutions in 27 states. VS (55.5%-69.4%) and IC (63.8%-71.1%) providers were significantly more likely to have reviews than IR (28.6%-48.8%) providers across all online platforms (P < 0.001 for all websites). Across all platforms, IC providers were rated significantly higher than VS and IR providers. Multivariable regression showed that provider specialty and additional time in practice were associated with higher review volume. In addition to specialty, review volume was associated with star rating as those physicians with more reviews tended to have a higher rating.

CONCLUSIONS

On average, vascular surgeons have more reviews and are more highly rated than interventional radiologists but tend to have fewer reviews and lower ratings than interventional cardiologists. VS providers may benefit from encouraging patients to file online reviews, especially in competitive markets.

Association Between Web-Based Physician Ratings and Physician Disciplinary Convictions: Retrospective Observational Study

BACKGROUND

Physician rating websites are commonly used by the public, yet the relationship between web-based physician ratings and health care quality is not well understood.

OBJECTIVE

The objective of our study was to use physician disciplinary convictions as an extreme marker for poor physician quality and to investigate whether disciplined physicians have lower ratings than nondisciplined matched controls.

METHODS

This was a retrospective national observational study of all disciplined physicians in Canada (751 physicians, 2000 to 2013). We searched ratings (2005-2015) from the country's leading online physician rating website for this group, and for 751 matched controls according to gender, specialty, practice years, and location. We compared overall ratings (out of a score of 5) as well as mean ratings by the type of misconduct. We also compared ratings for each type of misconduct and punishment.

RESULTS

There were 62.7% (471/751) of convicted and disciplined physicians (cases) with web-based ratings and 64.6% (485/751) of nondisciplined physicians (controls) with ratings. Of 312 matched case-control pairs, disciplined physicians were rated lower than controls overall (3.62 vs 4.00; P<.001). Disciplined physicians had lower ratings for all types of misconduct and punishment-except for physicians disciplined for sexual offenses (n=90 pairs; 3.83 vs 3.86; P=.81). Sexual misconduct was the only category in which mean ratings for physicians were higher than those for other disciplined physicians (3.63 vs 3.35; P=.003).

CONCLUSIONS

Physicians convicted for disciplinary misconduct generally had lower web-based ratings. Physicians convicted of sexual misconduct did not have lower ratings and were rated higher than other disciplined physicians. These findings may have future implications for the identification of physicians providing poor-quality care.

Patient satisfaction survey scores are not an appropriate metric to differentiate performance among anesthesiologists

STUDY OBJECTIVE

With the focus of patient-centered care in healthcare organizations, patient satisfaction plays an increasingly important role in healthcare quality measurement. We sought to determine whether an automated patient satisfaction survey could be effectively used to identify outlying anesthesiologists.

DESIGN

Retrospective Observational Study.

SETTING

Vanderbilt University Medical Center (VUMC).

MEASUREMENTS

Patient satisfaction data were obtained between October 24, 2016 and November 1, 2017. A multivariable ordered probit regression was conducted to evaluate the relationship between the mean scores of responses to Likert-scale questions on SurveyVitals' Anesthesia Patient Satisfaction Questionnaire 2. Fixed effects included demographics, clinical variables, providers and surgeons. Hypothesis tests to compare each individual anesthesiologist with the median-performing anesthesiologist were conducted.

MAIN RESULTS

We analyzed 10,528 surveys, with a 49.5% overall response rate. Younger patient (odds ratio (OR) 1.011 [per year of age]; 95% confidence interval (CI) 1.008 to 1.014; p < 0.001), regional anesthesia (versus general anesthesia) (OR 1.695; 95% CI 1.186 to 2.422; p = 0.004) and daytime surgery (versus nighttime surgery) (OR 1.795; 95% CI 1.091 to 2.959; p = 0.035) were associated with higher satisfaction scores. Compared with the median-ranked anesthesiologist, we found the adjusted odds ratio for an increase in satisfaction score ranged from 0.346 (95% CI 0.158 to 0.762) to 1.649 (95% CI 0.687 to 3.956) for the lowest and highest scoring providers, respectively. Only 10.10% of anesthesiologists at our institution had an odds ratio for satisfaction with a 95% CI not inclusive of 1.

CONCLUSIONS

Patient satisfaction is impacted by multiple factors. There was very little information in patient satisfaction scores to discriminate the providers, after adjusting for confounding. While patient satisfaction scores may facilitate identification of extreme outliers among anesthesiologists, there is no evidence that this metric is useful for the routine evaluation of individual provider performance.

Paediatric clinicians' experiences of parental online health information seeking: A qualitative study

AIM

The aim of this research was to explore clinicians' experiences of parents' online health information seeking (OHIS) behaviour about selective dorsal rhizotomy for the management of cerebral palsy.

METHODS

Using qualitative methodology, clinicians likely to have had experience with parents requesting selective dorsal rhizotomy were invited to participate in semi-structured interviews. Interviews with 13 clinicians were recorded, transcribed and inductive content analysis was used to identify, code and organise the data into themes.

RESULTS

Participants highlighted how parental OHIS was changing clinical communication. Negative effects included a shift in clinicians' attention from giving advice and guidance to spending time discussing online findings, justifying how this information applies to a particular child and managing parents' judgments about clinical views. Positive effects included more collaboration and sharing of ideas. These results are presented in three main themes: (i) the informed parent; (ii) the clinicians' role; and (iii) a new clinical dynamic.

CONCLUSION

This research reinforces the notion that OHIS is changing the communication dynamic and clinicians' and parents' roles within the clinical encounter. Of significance was the number of challenges clinicians are facing as a result of online information, including managing parental understanding of non-evidenced information and responding to negative feedback about their practice. This research suggests a need for educational support and ongoing professional development for clinicians to assist them to adjust to new goals and expectations of clinical interactions with 'informed' parents.

Making surgery safer in an increasingly digital world: the internet-friend or foe?

PURPOSE

The internet has resulted in huge efficiency gains in health care, the ability to deal with massive data accumulation and better manage patient data. However, potential and real pitfalls exist, including breeches in security of data and patient confidentiality, data storage issues, errors, and user interface issues.

METHODS

A MEDLINE review was performed using MeSH terms "health care" and "information technology." Cross-referencing was used to explore the different opportunities and challenges the internet has to offer.

RESULTS

As health professionals, we are fast adopting technologies at our fingertips, such as WhatsApp and video capabilities, into our clinical practice to increase productivity and improve patient care. However, the potential security breaches are significant for the health professional and health service. Further, electronic medical records have theoretical advantages to improve patient care, reduce medication errors, and expedite referrals. The downside is a less personalized approach to patient care, as well as the potential for these systems to be even more cumbersome. In regard to the acquisition of knowledge, there is no doubt the internet is our friend. Health care professionals as well as patients have unlimited resources for learning, including podcasts videos, apps, simulators, and wearable devices. Unfortunately, this comes with a risk of misinformation and poorly referenced data with little to no regulation of content.

CONCLUSION

In this increasing digital world, it is our task as health care providers to embrace these new technologies but develop guidelines and control systems to minimize the pitfalls.

The Use of Recommendations on Physician Rating Websites: The Number of Raters Makes the Difference When Adjusting Decisions

Physician rating websites allow users to check physicians' profiles, write reviews, or rate their performance. The opinion of other users regarding a physician can affect our decision to visit her/him. To investigate the specific role of the number of users rating a physician when choosing a physician with support of these platforms, we used a Judge-Advisor System in which participants answered their likelihood to visit a physician before and after seeing the recommendations of others. Within the experiment, three conditions were presented: high and low number of reviewers recommending a physician, and no recommendations. We found that the participants' likelihood to visit a physician varied with respect to the displayed physician characteristics on the platform. Importantly, after the recommendation of others was presented, participants' likelihood to visit the physician changed significantly. The participants' adjusted response was significantly closer to the recommendation coming from a higher number of users, which indicate that this online, social media cue influences our decision to visit physicians. Comments and ratings on physician ratings are generally positive, but we show that negative ratings have a direct negative influence in the decision to visit a physician. We suggest administrators of these platforms to pay special attention to the content that users upload.

A Comparison of Online Physician Ratings and Internal Patient-Submitted Ratings from a Large Healthcare System

BACKGROUND

Physician online ratings are ubiquitous and influential, but they also have their detractors. Given the lack of scientific survey methodology used in online ratings, some health systems have begun to publish their own internal patient-submitted ratings of physicians.

OBJECTIVE

The purpose of this study was to compare online physician ratings with internal ratings from a large healthcare system.

DESIGN

Retrospective cohort study comparing online ratings with internal ratings from a large healthcare system.

SETTING

Kaiser Permanente, a large integrated healthcare delivery system.

PARTICIPANTS

Physicians in the Southern California region of Kaiser Permanente, including all specialties with ambulatory clinic visits.

MAIN MEASURES

The primary outcome measure was correlation between online physician ratings and internal ratings from the integrated healthcare delivery system.

RESULTS

Of 5438 physicians who met inclusion and exclusion criteria, 4191 (77.1%) were rated both online and internally. The online ratings were based on a mean of 3.5 patient reviews, while the internal ratings were based on a mean of 119 survey returns. The overall correlation between the online and internal ratings was weak (Spearman's rho .23), but increased with the number of reviews used to formulate each online rating.

CONCLUSIONS

Physician online ratings did not correlate well with internal ratings from a large integrated healthcare delivery system, although the correlation increased with the number of reviews used to formulate each online rating. Given that many consumers are not aware of the statistical issues associated with small sample sizes, we would recommend that online rating websites refrain from displaying a physician's rating until the sample size is sufficiently large (for example, at least 15 patient reviews). However, hospitals and health systems may be able to provide better information for patients by publishing the internal ratings of their physicians.

Association of Social Media Presence with Online Physician Ratings and Surgical Volume Among California Urologists: Observational Study

BACKGROUND

Urologists are increasingly using various forms of social media to promote their professional practice and attract patients. Currently, the association of social media on a urologists' practice is unknown.

OBJECTIVES

We aimed to determine whether social media presence is associated with higher online physician ratings and surgical volume among California urologists.

METHODS

We sampled 195 California urologists who were rated on the ProPublica Surgeon Scorecard website. We obtained information on professional use of online social media (Facebook, Instagram, Twitter, blog, and YouTube) in 2014 and defined social media presence as a binary variable (yes/no) for use of an individual platform or any platform. We collected data on online physician ratings across websites (Yelp, Healthgrades, Vitals, RateMD, and UCompareHealthcare) and calculated the mean physician ratings across all websites as an average weighted by the number of reviews. We then collected data on surgical volume for radical prostatectomy from the ProPublica Surgeon Scorecard website. We used multivariable linear regression to determine the association of social media presence with physician ratings and surgical volume.

RESULTS

Among our sample of 195 urologists, 62 (32%) were active on some form of social media. Social media presence on any platform was associated with a slightly higher mean physician rating (β coefficient: .3; 95% CI 0.03-0.5; P=.05). However, only YouTube was associated with higher physician ratings (β coefficient: .3; 95% CI 0.2-0.5; P=.04). Social media presence on YouTube was strongly associated with increased radical prostatectomy volume (β coefficient: 7.4; 95% CI 0.3-14.5; P=.04). Social media presence on any platform was associated with increased radical prostatectomy volume (β coefficient: 7.1; 95% CI -0.7 to 14.2; P=.05).

CONCLUSIONS

Urologists' use of social media, especially YouTube, is associated with a modest increase in physician ratings and prostatectomy volume. Although a majority of urologists are not currently active on social media, patients may be more inclined to endorse and choose subspecialist urologists who post videos of their surgical technique.

Is There an Association Between Physician Review Websites and Press Ganey Survey Results in a Neurosurgical Outpatient Clinic?

OBJECTIVE

Recent studies suggest a poor association between physician review websites and the validated metrics used by the Centers for Medicare and Medicaid Services. The purpose of this study was to evaluate the association between online and outpatient Press Ganey (PG) measures of patient satisfaction in a neurosurgical department.

METHODS

We obtained PG survey results from one large academic institution's outpatient neurosurgery clinic. Popular physician review websites were searched for each of the faculty captured in the PG data. Average physician rating and percent Top Box scores were calculated for each physician. PG data were separated into new and established clinic visits for subset analysis. Spearman's rank correlation coefficients were calculated to determine associations.

RESULTS

Twelve neurosurgeons were included. Established patients demonstrated greater PG scores as compared with new patients, with an average physician rating increase of 0.55 and an average Top Box increase of 12.5%. Online physician ratings were found to demonstrate strong agreement with PG scores for the entire PG population, new patient subset, and established patient subset (ρ = 0.77-0.79, P < 0.05). Online Top Box scores demonstrated moderate agreement with overall PG Top Box scores (ρ = 0.59, P = 0.042), moderate agreement with the new patient population Top Box scores (ρ = 0.56, P = 0.059), and weak agreement with established patient population Top Box scores (ρ = 0.38, P = 0.217).

CONCLUSIONS

Our findings demonstrated a strong agreement between PG ratings and online physician ratings and a poorer correlation when comparing PG Top Box scores with online physician Top Box scores, particularly in the established patient population.

What Do Patients Say About Doctors Online? A Systematic Review of Studies on Patient Online Reviews

Background

The number of patient online reviews (PORs) has grown significantly, and PORs have played an increasingly important role in patients’ choice of health care providers.

Objective

The objective of our study was to systematically review studies on PORs, summarize the major findings and study characteristics, identify literature gaps, and make recommendations for future research.

Methods

A major database search was completed in January 2019. Studies were included if they (1) focused on PORs of physicians and hospitals, (2) reported qualitative or quantitative results from analysis of PORs, and (3) peer-reviewed empirical studies. Study characteristics and major findings were synthesized using predesigned tables.

Results

A total of 63 studies (69 articles) that met the above criteria were included in the review. Most studies (n=48) were conducted in the United States, including Puerto Rico, and the remaining were from Europe, Australia, and China. Earlier studies (published before 2010) used content analysis with small sample sizes; more recent studies retrieved and analyzed larger datasets using machine learning technologies. The number of PORs ranged from fewer than 200 to over 700,000. About 90% of the studies were focused on clinicians, typically specialists such as surgeons; 27% covered health care organizations, typically hospitals; and some studied both. A majority of PORs were positive and patients’ comments on their providers were favorable. Although most studies were descriptive, some compared PORs with traditional surveys of patient experience and found a high degree of correlation and some compared PORs with clinical outcomes but found a low level of correlation.

Conclusions

PORs contain valuable information that can generate insights into quality of care and patient-provider relationship, but it has not been systematically used for studies of health care quality. With the advancement of machine learning and data analysis tools, we anticipate more research on PORs based on testable hypotheses and rigorous analytic methods.

Trial Registration

International Prospective Register of Systematic Reviews (PROSPERO) CRD42018085057; https://www.crd.york.ac.uk/PROSPERO/display_record.php?RecordID=85057 (Archived by WebCite at http://www.webcitation.org/76ddvTZ1C)

Developing Embedded Taxonomy and Mining Patients' Interests From Web-Based Physician Reviews: Mixed-Methods Approach

BACKGROUND

Web-based physician reviews are invaluable gold mines that merit further investigation. Although many studies have explored the text information of physician reviews, very few have focused on developing a systematic topic taxonomy embedded in physician reviews. The first step toward mining physician reviews is to determine how the natural structure or dimensions is embedded in reviews. Therefore, it is relevant to develop the topic taxonomy rigorously and systematically.

OBJECTIVE

This study aims to develop a hierarchical topic taxonomy to uncover the latent structure of physician reviews and illustrate its application for mining patients' interests based on the proposed taxonomy and algorithm.

METHODS

Data comprised 122,716 physician reviews, including reviews of 8501 doctors from a leading physician review website in China (haodf.com), collected between 2007 and 2015. Mixed methods, including a literature review, data-driven-based topic discovery, and human annotation were used to develop the physician review topic taxonomy.

RESULTS

The identified taxonomy included 3 domains or high-level categories and 9 subtopics or low-level categories. The physician-related domain included the categories of medical ethics, medical competence, communication skills, medical advice, and prescriptions. The patient-related domain included the categories of the patient profile, symptoms, diagnosis, and pathogenesis. The system-related domain included the categories of financing and operation process. The F-measure of the proposed classification algorithm reached 0.816 on average. Symptoms (Cohen d=1.58, Δu=0.216, t=229.75, and P<.001) are more often mentioned by patients with acute diseases, whereas communication skills (Cohen d=-0.29, Δu=-0.038, t=-42.01, and P<.001), financing (Cohen d=-0.68, Δu=-0.098, t=-99.26, and P<.001), and diagnosis and pathogenesis (Cohen d=-0.55, Δu=-0.078, t=-80.09, and P<.001) are more often mentioned by patients with chronic diseases. Patients with mild diseases were more interested in medical ethics (Cohen d=0.25, Δu 0.039, t=8.33, and P<.001), operation process (Cohen d=0.57, Δu 0.060, t=18.75, and P<.001), patient profile (Cohen d=1.19, Δu 0.132, t=39.33, and P<.001), and symptoms (Cohen d=1.91, Δu=0.274, t=62.82, and P<.001). Meanwhile, patients with serious diseases were more interested in medical competence (Cohen d=-0.99, Δu=-0.165, t=-32.58, and P<.001), medical advice and prescription (Cohen d=-0.65, Δu=-0.082, t=-21.45, and P<.001), financing (Cohen d=-0.26, Δu=-0.018, t=-8.45, and P<.001), and diagnosis and pathogenesis (Cohen d=-1.55, Δu=-0.229, t=-50.93, and P<.001).

CONCLUSIONS

This mixed-methods approach, integrating literature reviews, data-driven topic discovery, and human annotation, is an effective and rigorous way to develop a physician review topic taxonomy. The proposed algorithm based on Labeled-Latent Dirichlet Allocation can achieve impressive classification results for mining patients' interests. Furthermore, the mining results reveal marked differences in patients' interests across different disease types, socioeconomic development levels, and hospital levels.

Differences in Online Consumer Ratings of Health Care Providers Across Medical, Surgical, and Allied Health Specialties: Observational Study of 212,933 Providers

Background

Health care consumers are increasingly using online ratings to select providers, but differences in the distribution of scores across specialties and skew of the data have the potential to mislead consumers about the interpretation of ratings.

Objective

The objective of our study was to determine whether distributions of consumer ratings differ across specialties and to provide specialty-specific data to assist consumers and clinicians in interpreting ratings.

Methods

We sampled 212,933 health care providers rated on the Healthgrades consumer ratings website, representing 29 medical specialties (n=128,678), 15 surgical specialties (n=72,531), and 6 allied health (nonmedical, nonnursing) professions (n=11,724) in the United States. We created boxplots depicting distributions and tested the normality of overall patient satisfaction scores. We then determined the specialty-specific percentile rank for scores across groupings of specialties and individual specialties.

Results

Allied health providers had higher median overall satisfaction scores (4.5, interquartile range [IQR] 4.0-5.0) than physicians in medical specialties (4.0, IQR 3.3-4.5) and surgical specialties (4.2, IQR 3.6-4.6, P<.001). Overall satisfaction scores were highly left skewed (normal between –0.5 and 0.5) for all specialties, but skewness was greatest among allied health providers (–1.23, 95% CI –1.280 to –1.181), followed by surgical (–0.77, 95% CI –0.787 to –0.755) and medical specialties (–0.64, 95% CI –0.648 to –0.628). As a result of the skewness, the percentages of overall satisfaction scores less than 4 were only 23% for allied health, 37% for surgical specialties, and 50% for medical specialties. Percentile ranks for overall satisfaction scores varied across specialties; percentile ranks for scores of 2 (0.7%, 2.9%, 0.8%), 3 (5.8%, 16.6%, 8.1%), 4 (23.0%, 50.3%, 37.3%), and 5 (63.9%, 89.5%, 86.8%) differed for allied health, medical specialties, and surgical specialties, respectively.

Conclusions

Online consumer ratings of health care providers are highly left skewed, fall within narrow ranges, and differ by specialty, which precludes meaningful interpretation by health care consumers. Specialty-specific percentile ranks may help consumers to more meaningfully assess online physician ratings.

Scope, Breadth, and Differences in Online Physician Ratings Related to Geography, Specialty, and Year: Observational Retrospective Study

Background

Physician ratings websites have emerged as a novel forum for consumers to comment on their health care experiences. Little is known about such ratings in Canada.

Objective

We investigated the scope and trends for specialty, geographic region, and time for online physician ratings in Canada using a national data source from the country’s leading physician-rating website.

Methods

This observational retrospective study used online ratings data from Canadian physicians (January 2005-September 2013; N=640,603). For specialty, province, and year of rating, we assessed whether physicians were likely to be rated favorably by using the proportion of ratings greater than the overall median rating.

Results

In total, 57,412 unique physicians had 640,603 individual ratings. Overall, ratings were positive (mean 3.9, SD 1.3). On average, each physician had 11.2 (SD 10.1) ratings. By comparing specialties with Canadian Institute of Health Information physician population numbers over our study period, we inferred that certain specialties (obstetrics and gynecology, family practice, surgery, and dermatology) were more commonly rated, whereas others (pathology, radiology, genetics, and anesthesia) were less represented. Ratings varied by specialty; cardiac surgery, nephrology, genetics, and radiology were more likely to be rated in the top 50th percentile, whereas addiction medicine, dermatology, neurology, and psychiatry were more often rated in the lower 50th percentile of ratings. Regarding geographic practice location, ratings were more likely to be favorable for physicians practicing in eastern provinces compared with western and central Canada. Regarding year, the absolute number of ratings peaked in 2007 before stabilizing and decreasing by 2013. Moreover, ratings were most likely to be positive in 2007 and again in 2013.

Conclusions

Physician-rating websites are a relatively novel source of provider-level patient satisfaction and are a valuable source of the patient experience. It is important to understand the breadth and scope of such ratings, particularly regarding specialty, geographic practice location, and changes over time.

AN OUTCOMES-BASED APPROACH ACROSS THE MEDICAL EDUCATION CONTINUUM

The medical education continuum spans undergraduate medical education (i.e., medical school), graduate medical education (i.e., residency and fellowship), and continuing medical education. This article provides three examples across the medical education and practice continuum where an outcomes-based approach is being used. By focusing on outcomes instead of process, a more predictable product of undergraduate medical education will be a medical student capable on day 1 of performing the work required of residency. Assessing the quality of medical education by the quality of care a graduate delivers once they enter practice will allow more effective design of medical education to improve care. A more comprehensive assessment of lifelong physician competence will help ensure the health of patients, their health care experience, and the value of care are of the highest possible standard and are continually improving.