Patients' evaluations of health care providers in the era of social networking: an analysis of physician-rating websites

The Influence of Physician-Rating Websites on Patient Physician Preference

Backgroud

Physician-rating websites (PRWs) are designed to publicly report physician quality information while bringing forth a sense of transparency. This study looks to identify the influence PRWs have on a patient's choice of orthopedic hand surgeon while stratifying patient physician preference by various demographic characteristics.

Methods

This survey-based study was conducted in a suburban outpatient orthopedic hand practice. All patients between 18 to 89 years of age who presented for an appointment were asked to participate. Survey questions aimed to identify patient demographics and the sources patients used to choose their hand surgeon.

Results

Overall, 104 patients completed our survey. Our study population was predominantly between 51 and 70 years of age (50.0%), women (60.6%), and Caucasian (84.6%), received a general education degree or high school diploma (36.5%), was employed (49.0%), and owned private health insurance (59.6%). One hundred and two patients (98.1%) answered that their physician's reputation is important. Seventy-five patients (72.1%) reported that they heard about their surgeon by physician referral, while only two (1.9%) used online search engines. Sixty-six patients (63.5%) noted that physician referrals were most trustworthy. Only 10 patients (9.6%) consulted PRWs to choose their surgeon, most of whom were younger than 50 years (n = 6), Caucasian (n = 8), and employed (n = 7) and had schooling after high school (n = 8).

Conclusions

Despite increases in digital information exchange platforms, PRWs are not commonly used by suburban orthopedic hand patients to exchange information about or choose their hand surgeon. Patients still primarily rely on physician referrals and word of mouth from family and friends to choose their surgeon.

Patient-Recorded Physician Ratings: What Can We Learn From 11,527 Online Reviews of Orthopedic Surgeons?

BACKGROUND

Physician rating websites have become an increasingly popular medium for patients to give feedback and to obtain information about physicians. Each physician rating website uses different criteria to evaluate physicians, with the option for patients to provide written reviews. Our goal is to identify factors that patients value when seeking an orthopedic surgeon.

METHODS

The study design was observational. We analyzed 7 common physician rating websites (RateMDs.com, HealthGrades.com, Vitals.com, WebMD.com, CareDash.com, Wellness.com, and ZocDoc.com) to evaluate orthopedic surgeon ratings and examine the variables that influence these ratings. Numeric ratings were standardized on a scale from 0-100 (higher number = positive rating). Multilevel mixed-effect Poisson regression models were used where appropriate.

RESULTS

A total of 11,527 online reviews of orthopedic surgeons in a single metropolitan area were evaluated in April 2019. The average overall rating was positive at 83.66 (±12.33) on our scale. The majority of surgeons amassed ≤20 reviews on each website. Higher ratings were correlated with staff friendliness (P = .010), punctuality (P = .009), and knowledge/expertise (P = .031). Analysis of written reviews showed that resolution of original patient complaints was associated with a high-scoring review.

CONCLUSION

This study may impact the ability of orthopedic surgeons to improve patient satisfaction and experience. The online image of orthopedic surgeons is positive in general, with weight placed on timeliness, knowledge, and staff friendliness. Further study is needed to examine whether differences exist in patient-centric domains reviewing orthopedic subspecialties. Future multiple geographic area studies will help elucidate any regional variation in patient-recorded data, as well as the role of any participation bias.

Online Reviews of Specialized Drug Treatment Facilities-Identifying Potential Drivers of High and Low Patient Satisfaction

BACKGROUND

Despite the importance of high-quality and patient-centered substance use disorder treatment, there are no standardized ratings of specialized drug treatment facilities and their services. Online platforms offer insights into potential drivers of high and low patient experience.

OBJECTIVE

We sought to analyze publicly available online review content of specialized drug treatment facilities and identify themes within high and low ratings.

DESIGN

This was a retrospective analysis of online ratings and reviews of specialized drug treatment facilities in Pennsylvania listed within the 2016 National Directory of Drug and Alcohol Abuse Treatment Facilities. Latent Dirichlet Allocation, a machine learning approach to narrative text, was used to identify themes within reviews. Differential Language Analysis was then used to measure correlations between themes and star ratings.

SETTING

Online reviews of Pennsylvania's specialized drug treatment facilities posted to Google and Yelp (July 2010-August 2018).

RESULTS

A total of 7823 online ratings were posted over 8 years. The distribution was bimodal (43% 5-star and 34% 1-star). The average weighted rating of a facility was 3.3 stars. Online themes correlated with 5-star ratings were the following: focus on recovery (r = 0.53), helpfulness of staff (r = 0.43), compassionate care (r = 0.37), experienced a life-changing moment (r = 0.32), and staff professionalism (r = 0.29). Themes correlated with a 1-star rating were waiting time (r = 0.41), poor accommodations (0.26), poor phone communication (r = 0.24), medications given (0.24), and appointment availability (r = 0.23). Themes derived from review content were similar to 9 of the 14 facility-level services highlighted by the Substance Abuse and Mental Health Services Administration's National Survey of Substance Abuse Treatment Services.

CONCLUSIONS

Individuals are sharing their ratings and reviews of specialized drug treatment facilities on online platforms. Organically derived reviews of the patient experience, captured by online platforms, reveal potential drivers of high and low ratings. These represent additional areas of focus which can inform patient-centered quality metrics for specialized drug treatment facilities.

Respected as a Client, Cared for as a Patient: Evidence of Heuristic Decision-Making from Yelp Reviews of Obstetrician-Gynecologists

Previous literature has demonstrated a gap between what health care consumers say they want to know about their physicians and the publicly available information the health care industry provides. This systematic analysis of Yelp reviews from the 25 most populous U.S. metropolitan areas is predicated on the assumptions that patients who post online physician reviews include information they would find useful when choosing a physician, and that this information represents an ecologically valid sample for making inferences regarding patients' decision making process. Obstetrician gynecologists are twice as likely to be reviewed as other physicians, but this is, to our knowledge, the first study examining online reviews of obstetrician-gynecologists specifically. This study contributes to the literature on medical decision making, demonstrating that the physician choice decision is made using a heuristic tallying model, in which only two, nearly equally weighted parameters are meaningful: patient (which, within the reviews, incorporates physician interpersonal manner and physician knowledge and skills) and office management. Yelp reviews of obstetrician-gynecologists follow patterns previously established in scholarship: ratings are bimodal, and approximately two-thirds are positive. In the absence of objective, user-friendly physician-level information, patients will turn to review sites like Yelp when choosing physicians. Rather than resisting this trend, physicians and hospital systems would be better served by working with review sites, or creating their own, to emphasize aspects of the experience patients are qualified to evaluate: the physician's interpersonal manner and office management.

Association Between Web-Based Physician Ratings and Physician Disciplinary Convictions: Retrospective Observational Study

BACKGROUND

Physician rating websites are commonly used by the public, yet the relationship between web-based physician ratings and health care quality is not well understood.

OBJECTIVE

The objective of our study was to use physician disciplinary convictions as an extreme marker for poor physician quality and to investigate whether disciplined physicians have lower ratings than nondisciplined matched controls.

METHODS

This was a retrospective national observational study of all disciplined physicians in Canada (751 physicians, 2000 to 2013). We searched ratings (2005-2015) from the country's leading online physician rating website for this group, and for 751 matched controls according to gender, specialty, practice years, and location. We compared overall ratings (out of a score of 5) as well as mean ratings by the type of misconduct. We also compared ratings for each type of misconduct and punishment.

RESULTS

There were 62.7% (471/751) of convicted and disciplined physicians (cases) with web-based ratings and 64.6% (485/751) of nondisciplined physicians (controls) with ratings. Of 312 matched case-control pairs, disciplined physicians were rated lower than controls overall (3.62 vs 4.00; P<.001). Disciplined physicians had lower ratings for all types of misconduct and punishment-except for physicians disciplined for sexual offenses (n=90 pairs; 3.83 vs 3.86; P=.81). Sexual misconduct was the only category in which mean ratings for physicians were higher than those for other disciplined physicians (3.63 vs 3.35; P=.003).

CONCLUSIONS

Physicians convicted for disciplinary misconduct generally had lower web-based ratings. Physicians convicted of sexual misconduct did not have lower ratings and were rated higher than other disciplined physicians. These findings may have future implications for the identification of physicians providing poor-quality care.

Social Media Presence Correlated with Improved Online Review Scores for Spine Surgeons

BACKGROUND

In the next decade, health care reimbursement will be more aligned to patient clinical outcomes. These outcomes are influenced by the patient's perceived opinion of his or her care. An evaluation into the role of surgeon demographics, social media (SM) accessibility, and office wait times was conducted to identify correlations with these among 3 online review platforms.

METHODS

A total of 206 (148 orthopedic, 58 neurosurgery trained) spine surgeons were included. Spine surgeon ratings and demographics data from 3 physician rating websites (Healthgrades.com [HG], Vitals.com, Google.com [G]) were collected in November 2019. Using the first 10 search results from G we then identified if the surgeons had publicly accessible Facebook, Twitter, or Instagram (IG) accounts.

RESULTS

The mean age of the cohort was 54.3 years (±9.40 years), and 28.2% had one form of publicly accessible SM. Having any SM was significantly correlated with higher scores on HG and G. An IG account was associated with significantly higher scores on all 3 platforms, and having a Facebook account correlated with significantly higher scores on HG in multivariate analysis. An office wait time between 16 and 30 minutes and >30 minutes was associated with worse scores on all 3 platforms (all P < 0.05). An academic practice was associated with higher scores on all 3 platforms (P < 0.05).

CONCLUSIONS

A shorter office wait time and an academic setting practice are associated with higher patient satisfaction scores on all 3 physician review websites. Accessible SM accounts are also associated with higher ratings on physician review websites, particularly IG.

The Buffering Effect of Health Care Provider Video Biographies When Viewed in Combination With Negative Reviews: "You Can't Fake Nice"

Background

Patients seek information from numerous sources before choosing a primary care provider; two of the most popular sources are providers’ own online biographies and patient rating websites. However, prior research has generally only examined how these sources influence patients’ decisions in isolation.

Objective

This study aimed to determine how primary care providers’ online biographies and online patient ratings interact to affect patients’ decision making, especially in the face of negative reviews.

Methods

An 8-condition online experiment (n=866) was conducted, manipulating patient ratings and the timing of viewing a provider’s online biographical video (pre- or postrating viewing).

Results

When participants were shown a short video introduction of a provider after reading predominantly negative reviews a positive expectancy violation occurred, which was also related to more positive perceptions of the provider. When exposed to all negative reviews, 43% of participants indicated they would still choose to make an appointment with the provider, with many indicating that the video provided the evidence needed to help make up their own minds.

Conclusions

These findings are especially relevant to health care organizations seeking to combat a recent rise in fake patient reviews. Providing patients with realistic expectations of the care that clinicians can offer via their own online biographical videos can help counteract negative patient comments online.

Influence of Health Literacy on Effects of Patient Rating Websites: Survey Study Using a Hypothetical Situation and Fictitious Doctors

Background

Physician rating websites (PRWs) are a device people use actively and passively, although their objective capabilities are insufficient when it comes to judging the medical performance and qualification of physicians. PRWs are an innovation born of the potential of the Internet and boosted very much by the longstanding policy of improving and encouraging patient participation in medical decision-making. A mismatch is feared between patient motivations to participate and their capabilities of doing so well. Awareness of such a mismatch might contribute to some skepticism of patient-written physician reviews on PRWs.

Objective

We intend to test whether health literacy is able to dampen the effects that a patient-written review of a physician’s performance might have on physician choice.

Methods

An experiment was conducted within a survey interview. Participants were put into a fictitious decision situation in which they had to choose between two physicians on the basis of their profiles on a PRW. One of the physician profiles contained the experimental stimulus in the form of a friendly and a critical written review. The dependent variable was physician choice. An attitude differential, trust differential, and two measures of health literacy, the newest vital sign as an example of a performance-based measure and eHealth Literacy Scale as an example of a perception-based measure, were tested for roles as intermediary variables. Analysis traced the influence of the review tendency on the dependent variables and a possible moderating effect of health literacy on these influences.

Results

Reviews of a physician’s competence and medical skill affected participant choice of a physician. High health literacy dampened these effects only in the case of the perception-based measure and only for the negative review. Correspondingly, the effect of the review tendency appeared to be stronger for the positive review. Attitudes and trust only affected physician choice when included as covariants, considerably increasing the variance explained by regression models.

Conclusions

Findings sustain physician worries that even one negative PRW review can affect patient choice and damage doctors’ reputations. Hopes that health literacy might raise awareness of the poor basis of physician reviews and ratings given by patients have some foundation.

What Patients Value in Physicians: Analyzing Drivers of Patient Satisfaction Using Physician-Rating Website Data

BACKGROUND

Customer-oriented health care management and patient satisfaction have become important for physicians to attract patients in an increasingly competitive environment. Satisfaction influences patients' choice of physician and leads to higher patient retention and higher willingness to engage in positive word of mouth. In addition, higher satisfaction has positive effects on patients' willingness to follow the advice given by the physician. In recent years, physician-rating websites (PRWs) have emerged in the health care sector and are increasingly used by patients. Patients' usage includes either posting an evaluation to provide feedback to others about their own experience with a physician or reading evaluations of other patients before choosing a physician. The emergence of PRWs offers new avenues to analyze patient satisfaction and its key drivers. PRW data enable both satisfaction analyses and implications on the level of the individual physician as well as satisfaction analyses and implications on an overall level.

OBJECTIVE

This study aimed to identify linear and nonlinear effects of patients' perceived quality of physician appointment service attributes on the overall evaluation measures that are published on PRWs.

METHODS

We analyzed large-scale survey data from a German PRW containing 84,680 surveys of patients rating a total of 7038 physicians on 24 service attributes and 4 overall evaluation measures. Elasticities are estimated from regression models with perceived attribute quality as explanatory variables and overall evaluation measures as dependent variables. Depending on the magnitude of the elasticity, service attributes are classified into 3 categories: attributes with diminishing, constant, or increasing returns to overall evaluation.

RESULTS

The proposed approach revealed new insights into what patients value when visiting physicians and what they take for granted. Improvements in the physicians' pleasantness and friendliness have increasing returns to the publicly available overall evaluation (b=1.26). The practices' cleanliness (b=1.05) and the communication behavior of a physician during a visit (b level between .97 and 1.03) have constant returns. Indiscretion in the waiting rooms, extended waiting times, and a lack of modernity of the medical equipment (b level between .46 and .59) have the strongest diminishing returns to overall evaluation.

CONCLUSIONS

The categorization of the service attributes supports physicians in identifying potential for improvements and prioritizing resource allocation to improve the publicly available overall evaluation ratings on PRWs. Thus, the study contributes to patient-centered health care management and, furthermore, promotes the utility of PRWs through large-scale data analysis.

Zip Codes May Not Be an Adequate Method to Risk Adjust for Socioeconomic Status Following Total Joint Arthroplasty at the Individual Surgeon Level

BACKGROUND

Increasing consumerism in healthcare has included a push toward the ranking of individual surgeons. These rankings rely on the adjustment of patient outcomes based on individual patient risk. Socioeconomic status (SES) has been identified as an important variable impacting patient outcomes following total joint arthroplasty, and patient zip code has been proposed as a proxy. Our study attempts to determine if zip code is an acceptable proxy for SES within a single surgeon's practice.

METHODS

Using public zip code and Geographic Information Systems (GIS) tax map data, we compared the real estate holdings of 244 patients undergoing total joint arthroplasty from an individual hip and knee arthroplasty surgeon's practice within an academic medical center over a 14-month period. An independent t-test was used to compare GIS data with the average home value within a given zip code. A Pearson correlation coefficient was calculated between GIS values and average home value per zip code.

RESULTS

In a sample of 244 patients, mean home value calculated from GIS data was $335,993 (standard deviation [SD] $246,549), and $243,663 with zip code data (SD $84,731). The Pearson correlation coefficient was 0.411 (P < .001). There was a significant difference between mean home values calculated from zip code data and GIS data (P < .001). Using zip code estimates would have mischaracterized home value, as defined as greater than or less than 1 SD, in 15% of patients.

CONCLUSION

Although there was some relationship between zip code and real estate holdings, the correlation is only moderate in strength and a substantial number of outliers were present. Given the sample size at the individual surgeon level, we question whether zip code can be used as a proxy for SES risk adjustment for the purposes of surgeon ranking.

Online Ratings and Reviews of American Orthopaedic Foot and Ankle Surgeons

Background. Utilization of physician rating websites continues to expand. There is limited information on how these websites function and influence patient perception and physicians' practices. No study to our knowledge has investigated online ratings and comments of orthopaedic foot and ankle surgeons. We identified factors impacting online ratings and comments for this subset of surgeons. Methods. 210 orthopaedic foot and ankle surgeons were selected from the American Orthopaedic Foot and Ankle Society (AOFAS) website. Demographic information, ratings, and comments were reviewed on the 3 most visited public domain physician ratings websites: HealthGrades.com , Vitals.com, and Ratemds.com. Content differences between positive and negative comments were evaluated. Results. The mean review rating of 4.03 ± 0.57 out of 5 stars. 84% of the total number of ratings were either 1 star (17%) or 5 stars (67%). Most positive comments related to outcome, physician personality, and communication, whereas most negative comments related to outcome, bedside manner, and waiting time. χ² Analyses revealed statistically significant proportions of positive comments pertaining to surgeon-dependent factors (eg, physician personality, knowledge, skills) and negative comments concerning surgeon-independent factors (eg, waiting time, logistics). Conclusion. This study examined online ratings and written comments of orthopaedic foot and ankle surgeons. Surgeons had a generally favorable rating and were likely to have positive comments. Patients were likely to write positive comments about surgeon personality and communication, and negative comments pertaining to bedside manner and waiting time. Knowledge and management of online content may allow surgeons to improve patient satisfaction and online ratings. Level of Evidence: Level IV.

The Use of Recommendations on Physician Rating Websites: The Number of Raters Makes the Difference When Adjusting Decisions

Physician rating websites allow users to check physicians' profiles, write reviews, or rate their performance. The opinion of other users regarding a physician can affect our decision to visit her/him. To investigate the specific role of the number of users rating a physician when choosing a physician with support of these platforms, we used a Judge-Advisor System in which participants answered their likelihood to visit a physician before and after seeing the recommendations of others. Within the experiment, three conditions were presented: high and low number of reviewers recommending a physician, and no recommendations. We found that the participants' likelihood to visit a physician varied with respect to the displayed physician characteristics on the platform. Importantly, after the recommendation of others was presented, participants' likelihood to visit the physician changed significantly. The participants' adjusted response was significantly closer to the recommendation coming from a higher number of users, which indicate that this online, social media cue influences our decision to visit physicians. Comments and ratings on physician ratings are generally positive, but we show that negative ratings have a direct negative influence in the decision to visit a physician. We suggest administrators of these platforms to pay special attention to the content that users upload.

Physician Beliefs About Online Reporting of Quality and Experience Data

IMPORTANCE

Physician attitudes about websites that publicly report health care quality and experience data have not been recently described.

OBJECTIVES

To examine physician attitudes about the accuracy of websites that report information about quality of care and patient experience and to describe physician beliefs about the helpfulness of these data for patients choosing a physician.

DESIGN, PARTICIPANTS, AND MEASURES

The Rhode Island Department of Health (RIDOH) and a multi-stakeholder group developed and piloted two questions that were added to RIDOH's biennial physician survey of all 4197 practicing physicians in Rhode Island: (1) "How accurate of a picture do you feel that the following types of online resources give about the quality of care that physicians provide?" (with choices) and (2) "Which types of physician-specific information (i.e., not about the practice overall) would be helpful to include in online resources for patients to help them choose a new physician? (Select all that apply)." Responses were stratified by primary care vs. subspecialty clinicians. Summary statistics and chi-squared tests were used to analyze the results.

RESULTS

Among 1792 respondents (response rate 43%), 45% were unaware of RIDOH's site and 54% were unaware of the Centers for Medicare & Medicaid Services (CMS)' quality reporting sites. Only 2% felt that Medicare sites were "very accurate" in depicting physician quality. Most physicians supported public reporting of general information about physicians (e.g., board certification), but just over one-third of physicians felt that performance-based quality measures are "helpful" (and a similar percentage reported that patient reviews felt are "helpful") for patients choosing a physician.

CONCLUSIONS

Physician-respondents were either uninformed or skeptical about public reporting websites. In contrast to prior reports that a majority of patients value some forms of publicly reported data, most physicians do not consider quality metrics and patient-generated reviews helpful for patients who are choosing a physician.

Using online patient feedback to improve NHS services: the INQUIRE multimethod study

Background

Online customer feedback has become routine in many industries, but it has yet to be harnessed for service improvement in health care.

Objectives

To identify the current evidence on online patient feedback; to identify public and health professional attitudes and behaviour in relation to online patient feedback; to explore the experiences of patients in providing online feedback to the NHS; and to examine the practices and processes of online patient feedback within NHS trusts.

Design

A multimethod programme of five studies: (1) evidence synthesis and stakeholder consultation; (2) questionnaire survey of the public; (3) qualitative study of patients’ and carers’ experiences of creating and using online comment; (4) questionnaire surveys and a focus group of health-care professionals; and (5) ethnographic organisational case studies with four NHS secondary care provider organisations.

Setting

The UK.

Methods

We searched bibliographic databases and conducted hand-searches to January 2018. Synthesis was guided by themes arising from consultation with 15 stakeholders. We conducted a face-to-face survey of a representative sample of the UK population (n = 2036) and 37 purposively sampled qualitative semistructured interviews with people with experience of online feedback. We conducted online surveys of 1001 quota-sampled doctors and 749 nurses or midwives, and a focus group with five allied health professionals. We conducted ethnographic case studies at four NHS trusts, with a researcher spending 6–10 weeks at each site.

Results

Many people (42% of internet users in the general population) read online feedback from other patients. Fewer people (8%) write online feedback, but when they do one of their main reasons is to give praise. Most online feedback is positive in its tone and people describe caring about the NHS and wanting to help it (‘caring for care’). They also want their feedback to elicit a response as part of a conversation. Many professionals, especially doctors, are cautious about online feedback, believing it to be mainly critical and unrepresentative, and rarely encourage it. From a NHS trust perspective, online patient feedback is creating new forms of response-ability (organisations needing the infrastructure to address multiple channels and increasing amounts of online feedback) and responsivity (ensuring responses are swift and publicly visible).

Limitations

This work provides only a cross-sectional snapshot of a fast-emerging phenomenon. Questionnaire surveys can be limited by response bias. The quota sample of doctors and volunteer sample of nurses may not be representative. The ethnographic work was limited in its interrogation of differences between sites.

Conclusions

Providing and using online feedback are becoming more common for patients who are often motivated to give praise and to help the NHS improve, but health organisations and professionals are cautious and not fully prepared to use online feedback for service improvement. We identified several disconnections between patient motivations and staff and organisational perspectives, which will need to be resolved if NHS services are to engage with this source of constructive criticism and commentary from patients.

Future work

Intervention studies could measure online feedback as an intervention for service improvement and longitudinal studies could examine use over time, including unanticipated consequences. Content analyses could look for new knowledge on specific tests or treatments. Methodological work is needed to identify the best approaches to analysing feedback.

Study registration

The ethnographic case study work was registered as Current Controlled Trials ISRCTN33095169.

Funding

This project was funded by the National institute for Health Research (NIHR) Health Services and Delivery Research programme and will be published in full in Health Services and Delivery Research; Vol. 7, No. 38. See the NIHR Journals Library website for further project information.

Effects of Online Physician Reviews and Physician Gender on Perceptions of Physician Skills and Primary Care Physician (PCP) Selection

Physician review websites have become more relevant and important in people's selection of physicians. The current study experimentally examined how online physician reviews endorsing a primary care physician's (PCP's) technical or interpersonal skills, along with a physician's gender, may influence people's perceptions of the physician's skills and their willingness to choose the physician. Participants were randomly assigned to view a mockup physician review web page and to imagine that they needed to find a new PCP in a new city. They were then asked to report their perceptions of the physician and willingness to choose the physician as their PCP. The results suggested that people's willingness to choose a PCP was affected by physician reviews through their influence on people's perceptions of the PCP's technical and interpersonal skills. More importantly, this study found that when physician reviews endorsed a PCP's technical skills people perceived a female PCP to be more interpersonally competent than a male PCP and thus were more likely to choose the female PCP. The gendered perception, however, was not extended to a PCP's technical skills. Practical implications for health providers and consumers are discussed.

The Content and Nature of Narrative Comments on Swiss Physician Rating Websites: Analysis of 849 Comments

Background

The majority of physician rating websites (PRWs) provide users the option to leave narrative comments about their physicians. Narrative comments potentially provide richer insights into patients’ experiences and feelings that cannot be fully captured in predefined quantitative rating scales and are increasingly being examined. However, the content and nature of narrative comments on Swiss PRWs has not been examined to date.

Objective

This study aimed to examine (1) the types of issues raised in narrative comments on Swiss PRWs and (2) the evaluation tendencies of the narrative comments.

Methods

A random stratified sample of 966 physicians was generated from the regions of Zürich and Geneva. Every selected physician was searched for on 3 PRWs (OkDoc, DocApp, and Medicosearch) and Google, and narrative comments were collected. Narrative comments were analyzed and classified according to a theoretical categorization framework of physician-, staff-, and practice-related issues.

Results

The selected physicians had a total of 849 comments. In total, 43 subcategories addressing the physician (n=21), staff (n=8), and practice (n=14) were identified. None of the PRWs’ comments covered all 43 subcategories of the categorization framework; comments on Google covered 86% (37/43) of the subcategories, Medicosearch covered 72% (31/43), DocApp covered 60% (26/43), and OkDoc covered 56% (24/43). In total, 2441 distinct issues were identified within the 43 subcategories of the categorization framework; 83.65% (2042/2441) of the issues related to the physician, 6.63% (162/2441) related to the staff, and 9.70% (237/2441) related to the practice. Overall, 95% (41/43) of the subcategories of the categorization framework and 81.60% (1992/2441) of the distinct issues identified were concerning aspects of performance (interpersonal skills of the physician and staff, infrastructure, and organization and management of the practice) that are considered assessable by patients. Overall, 83.0% (705/849) of comments were classified as positive, 2.5% (21/849) as neutral, and 14.5% (123/849) as negative. However, there were significant differences between PRWs, regions, and specialty regarding negative comments: 90.2% (111/123) of negative comments were on Google, 74.7% (92/123) were regarding physicians in Zurich, and 73.2% (90/123) were from specialists.

Conclusions

From the narrative comments analyzed, it can be reported that interpersonal issues make up nearly half of all negative issues identified, and it is recommended that physicians should focus on improving these issues. The current suppression of negative comments by Swiss PRWs is concerning, and there is a need for a consensus-based criterion to be developed to determine which comments should be published publicly. Finally, it would be helpful if Swiss patients are made aware of the current large differences between Swiss PRWs regarding the frequency and nature of ratings to help them determine which PRW will provide them with the most useful information.

Online Ratings of Neurosurgeons: An Examination of Web Data and its Implications

BACKGROUND

Patient satisfaction ratings are increasingly used for hospital rankings, referral base and physician reimbursement. As such, online physician rating websites (PRWs) are quickly becoming a topic of interest.

OBJECTIVE

To analyze the distribution of neurosurgeons' ratings on the 3 most widely used PRWs, and examine factors associated with positive and negative ratings.

METHODS

We used a key term search to identify board-certified neurosurgeons on 3 widely used PRWs: RateMD.com, Healthgrades.com, and Vitals.com. Data were collected on average rating and number of ratings. Demographic, training-related and practice-related data, as well as location of practice, and place of training were also collected.

RESULTS

Data was non-normally distributed (P < .001 for all 3). Having fewer reviews was associated with higher variance in ratings between PRWs for a given surgeon (odds ratio 0.99, P = .001). All surgeons below the 25th percentile with respect to the number of reviews that had been written about them were eliminated. Of the remaining surgeons (n = 3054), the median composite score was 4.11 out of 5, interquartile range (3.69, 4.44). Surgeons had higher median modified composite scores if they were fellowship-trained (P = .0001) or graduated from a top 25 medical school (P = .0117), but not if they graduated from a top 25 residency (P = .1056). Surgeons located in major cities had higher median composite scores (P = .0025).

CONCLUSION

Online ratings for neurosurgeons must be evaluated in context. Median ratings are generally high, but variable between websites. Median scores also vary among regions and practice settings. Higher scores were associated with ranking of medical school, recent graduation, and fellowship training completion.

Impact of Surgeon Gender on Online Physician Reviews

BACKGROUND

The increasing use of review websites by consumers has become a crucial first step in choosing a physician with more than half of Americans consulting review sites before physician selection. We sought to identify whether differences exist in the quality and content of online reviews for men versus women surgeons.

METHODS

Using a deliberate sampling algorithm of the two most populated physician review websites, RateMDs.com and Yelp.com, we purposefully sampled reviews for the top 20 surgeons per tercile from the four most populated urban areas in the United States: New York, Houston, Los Angeles, and Chicago. A grounded theory qualitative assessment was performed of major and minor thematic elements including global rating, communication, technical skills, and comments on ancillary elements.

RESULTS

Four-hundred and thirty-one online patient reviews of 238 surgeons were identified from RateMDs.com (51%) and Yelp.com (49%) with available information on gender for analysis. Seventy-six percent of reviews were of male surgeons. Reviewers were more likely to mention a global rating and technical skill for men compared with women surgeons. Most reviews were positive with no difference in global rating by gender (83.7% positive for men and 74.3% positive for women, P = 0.08). Women surgeons were more likely to have positive comments on social interactions as compared with men (94.7% versus 88.0%, P = 0.03); whereas men surgeons were more likely to have a positive rating on technical skill compared with women (88.2% versus 76.2%, P = 0.04).

CONCLUSIONS

The content and quality of online surgeon reviews differ by gender. There is no difference in global rating between men and women. Women are rated higher for social interaction domains and men are rated higher on technical skill domains.

What Do Patients Complain About Online: A Systematic Review and Taxonomy Framework Based on Patient Centeredness

Background

Complaints made online by patients about their health care experiences are becoming prevalent because of widespread worldwide internet connectivity. An a priori framework, based on patient centeredness, may be useful in identifying the types of issues patients complain about online across multiple settings. It may also assist in examining whether the determinants of patient-centered care (PCC) mirror the determinants of patient experiences.

Objective

The objective of our study was to develop a taxonomy framework for patient complaints online based on patient centeredness and to examine whether the determinants of PCC mirror the determinants of patient experiences.

Methods

First, the best fit framework synthesis technique was applied to develop the proposed a priori framework. Second, electronic databases, including Web of Science, Scopus, and PubMed, were searched for articles published between 2000 and June 2018. Studies were only included if they collected primary quantitative data on patients’ online complaints. Third, a deductive and inductive thematic analysis approach was adopted to code the themes of recognized complaints into the framework.

Results

In total, 17 studies from 5 countries were included in this study. Patient complaint online taxonomies and theme terms varied. According to our framework, patients expressed most dissatisfaction with patient-centered processes (101,586/204,363, 49.71%), followed by prerequisites (appropriate skills and knowledge of physicians; 50,563, 24.74%) and the care environment (48,563/204,363, 23.76%). The least dissatisfied theme was expected outcomes (3651/204,363, 1.79%). People expressed little dissatisfaction with expanded PCC dimensions, such as involvement of family and friends (591/204,363, 0.29%). Variation in the concerns across different countries’ patients were also observed.

Conclusions

Online complaints made by patients are of major value to health care providers, regulatory bodies, and patients themselves. Our PCC framework can be applied to analyze them under a wide range of conditions, treatments, and countries. This review has shown significant heterogeneity of patients’ online complaints across different countries.

Is There an Association Between Physician Review Websites and Press Ganey Survey Results in a Neurosurgical Outpatient Clinic?

OBJECTIVE

Recent studies suggest a poor association between physician review websites and the validated metrics used by the Centers for Medicare and Medicaid Services. The purpose of this study was to evaluate the association between online and outpatient Press Ganey (PG) measures of patient satisfaction in a neurosurgical department.

METHODS

We obtained PG survey results from one large academic institution's outpatient neurosurgery clinic. Popular physician review websites were searched for each of the faculty captured in the PG data. Average physician rating and percent Top Box scores were calculated for each physician. PG data were separated into new and established clinic visits for subset analysis. Spearman's rank correlation coefficients were calculated to determine associations.

RESULTS

Twelve neurosurgeons were included. Established patients demonstrated greater PG scores as compared with new patients, with an average physician rating increase of 0.55 and an average Top Box increase of 12.5%. Online physician ratings were found to demonstrate strong agreement with PG scores for the entire PG population, new patient subset, and established patient subset (ρ = 0.77-0.79, P < 0.05). Online Top Box scores demonstrated moderate agreement with overall PG Top Box scores (ρ = 0.59, P = 0.042), moderate agreement with the new patient population Top Box scores (ρ = 0.56, P = 0.059), and weak agreement with established patient population Top Box scores (ρ = 0.38, P = 0.217).

CONCLUSIONS

Our findings demonstrated a strong agreement between PG ratings and online physician ratings and a poorer correlation when comparing PG Top Box scores with online physician Top Box scores, particularly in the established patient population.

Quantitative Ratings and Narrative Comments on Swiss Physician Rating Websites: Frequency Analysis

Background

Physician rating websites (PRWs) have been developed as part of a wider move toward transparency around health care quality, and these allow patients to anonymously rate, comment, and discuss physicians’ quality on the Web. The first Swiss PRWs were established in 2008, at the same time as many international PRWs. However, there has been limited research conducted on PRWs in Switzerland to date. International research has indicated that a key shortcoming of PRWs is that they have an insufficient number of ratings.

Objective

The aim of this study was to examine the frequency of quantitative ratings and narrative comments on the Swiss PRWs.

Methods

In November 2017, a random stratified sample of 966 physicians was generated from the regions of Zürich and Geneva. Every selected physician was searched for on 4 rating websites (OkDoc, DocApp, Medicosearch, and Google) between November 2017 and July 2018. It was recorded whether the physician could be identified, what the physician’s quantitative rating was, and whether the physician had received narrative comments. In addition, Alexa Internet was used to examine the number of visitors to the PRWs, compared with other websites.

Results

Overall, the portion of physicians able to be identified on the PRWs ranged from 42.4% (410/966) on OkDoc to 87.3% (843/966) on DocApp. Of the identifiable physicians, only a few of the selected physicians had been rated quantitatively (4.5% [38/843] on DocApp to 49.8% [273/548] on Google) or received narrative comments (4.5% [38/843] on DocApp to 31.2% [171/548] on Google) at least once. Rated physicians also had, on average, a low number of quantitative ratings (1.47 ratings on OkDoc to 3.74 rating on Google) and narrative comments (1.23 comment on OkDoc to 3.03 comments on Google). All 3 websites allowing ratings used the same rating scale (1-5 stars) and had a very positive average rating: DocApp (4.71), Medicosearch (4.69), and Google (4.41). There were significant differences among the PRWs (with the majority of ratings being posted on Google in past 2 years) and regions (with physicians in Zurich more likely to have been rated and have more ratings on average). Only Google (position 1) and Medicosearch (position 8358) are placed among the top 10,000 visited websites in Switzerland.

Conclusions

It appears that this is the first time Google has been included in a study examining physician ratings internationally and it is noticeable how Google has had substantially more ratings than the 3 dedicated PRWs in Switzerland over the past 2 and a half years. Overall, this study indicates that Swiss PRWs are not yet a reliable source of unbiased information regarding patient experiences and satisfaction with Swiss physicians; many selected physicians were unable to be identified, only a few physicians had been rated, and the ratings posted were overwhelmingly positive.

There's a frog in my salad! A review of online media coverage for wild vertebrates found in prepackaged produce in the United States

Prepackaged leafy green vegetables represent one of the fastest growing segments of the fresh-produce industry in the United States. Several steps in the production process have been mechanized to meet the downstream demand for prebagged lettuces. The growth in this market, however, has come with drawbacks, and chief among them are consumers finding wild animals in prepackaged crops. These incidents may signal an overburdened produce supply chain, but we currently lack the information needed to determine if this is a food-safety problem or food-quality concern. Here, we address this gap by reviewing online media coverage of wild vertebrates found in prepackaged produce items by customers in the United States. We discovered 40 independent incidents since 2003 with 95% having occurred during 2008-2018, suggesting that the frequency of incidents may have increased during the last decade. The minority of incidents included wild animals found in organic produce (27.5%), whereas the majority involved conventionally grown crops (72.5%). Most incidents involved amphibians (52.5%) and reptiles (22.5%), while fewer contained mammals (17.5%) and birds (7.5%). Frogs and toads made up all of the amphibian-related incidents, with more than 60% comprising small-bodied treefrogs found in various types of fresh leafy greens. At least seven incidents involved Pacific Treefrogs (Hyliola regilla) and three comprised Green Anoles (Anolis carolinensis). One lizard and nine frogs were found alive, and at least two frogs were released into non-native areas. This is the first review quantifying incidents of vertebrates found by customers in prepackaged produce, yet it remains unclear whether these occurrences indicate a food-safety crisis or a complaint against food quality. Nevertheless, wild animals can spread diseases to humans via contaminated produce, therefore we contend that industry professionals can reduce the potential health risk to their consumers and negative economic consequences to themselves through increased attention to this matter.

Online Ratings of Urologists: Comprehensive Analysis

Background

Physician-rating websites are being increasingly used by patients to help guide physician choice. As such, an understanding of these websites and factors that influence ratings is valuable to physicians.

Objective

We sought to perform a comprehensive analysis of online urology ratings information, with a specific focus on the relationship between number of ratings or comments and overall physician rating.

Methods

We analyzed urologist ratings on the Healthgrades website. The data retrieval focused on physician and staff ratings information. Our analysis included descriptive statistics of physician and staff ratings and correlation analysis between physician or staff performance and overall physician rating. Finally, we performed a best-fit analysis to assess for an association between number of physician ratings and overall rating.

Results

From a total of 9921 urology profiles analyzed, there were 99,959 ratings and 23,492 comments. Most ratings were either 5 (“excellent”) (67.53%, 67,505/99,959) or 1 (“poor”) (24.22%, 24,218/99,959). All physician and staff performance ratings demonstrated a positive and statistically significant correlation with overall physician rating (P<.001 for all analyses). Best-fit analysis demonstrated a negative relationship between number of ratings or comments and overall rating until physicians achieved 21 ratings or 6 comments. Thereafter, a positive relationship was seen.

Conclusions

In our study, a dichotomous rating distribution was seen with more than 90% of ratings being either excellent or poor. A negative relationship between number of ratings or comments and overall rating was initially seen, after which a positive relationship was demonstrated. Combined, these data suggest that physicians can benefit from understanding online ratings and that proactive steps to encourage patient rating submissions may help optimize overall rating.

The Impact of Web-Based Ratings on Patient Choice of a Primary Care Physician Versus a Specialist: Randomized Controlled Experiment

Background

Physician review websites have empowered prospective patients to acquire information about physicians. However, little is known about how Web-based ratings on different aspects of a physician may affect patients’ selection of physicians differently.

Objective

The objectives of this study were to examine (1) how patients weigh ratings on a physician’s technical skills and interpersonal skills in their selection of physicians and (2) whether and how people’s choice of a primary care physician versus a specialist is affected differently by Web-based ratings.

Methods

A 2×2×2×2 between-subjects experiment was conducted. Over 600 participants were recruited through a crowdsourcing website and randomly assigned to view a mockup physician review Web page that contained information on a physician’s basic information and patients’ ratings. After reviewing the Web page, participants were asked to complete a survey on their perceptions of the physician and willingness to seek health care from the physician.

Results

The results showed that participants were more willing to choose a physician with higher ratings on technical skills than on interpersonal skills compared with a physician with higher ratings on interpersonal skills than on technical skills, t_369.96=22.36, P<.001, Cohen d=1.22. In the selection of different types of physicians, patients were more likely to choose a specialist with higher ratings on technical skills than on interpersonal skills, compared with a primary care physician with the same ratings, F_1,521=5.34, P=.021.

Conclusions

The findings suggest that people place more weight on technical skills than interpersonal skills in their selection of a physician based on their ratings on the Web. Specifically, people are more likely to make a compromise on interpersonal skills in their choice of a specialist compared with a primary care physician. This study emphasizes the importance of examining Web-based physician ratings in a more nuanced way in relation to the selection of different types of physicians.

Trial Registration

ISRCTN Registry ISRCTN91316463; http://www.isrctn.com/ISRCTN91316463

When Similarity Beats Expertise-Differential Effects of Patient and Expert Ratings on Physician Choice: Field and Experimental Study

Background

Increasing numbers of patients consult Web-based rating platforms before making health care decisions. These platforms often provide ratings from other patients, reflecting their subjective experience. However, patients often lack the knowledge to be able to judge the objective quality of health services. To account for this potential bias, many rating platforms complement patient ratings with more objective expert ratings, which can lead to conflicting signals as these different types of evaluations are not always aligned.

Objective

This study aimed to fill the gap on how consumers combine information from 2 different sources—patients or experts—to form opinions and make purchase decisions in a health care context. More specifically, we assessed prospective patients’ decision making when considering both types of ratings simultaneously on a Web-based rating platform. In addition, we examined how the influence of patient and expert ratings is conditional upon rating volume (ie, the number of patient opinions).

Methods

In a field study, we analyzed a dataset from a Web-based physician rating platform containing clickstream data for more than 5000 US doctors. We complemented this with an experimental lab study consisting of a sample of 112 students from a Dutch university. The average age was 23.1 years, and 60.7% (68/112) of the respondents were female.

Results

The field data illustrated the moderating effect of rating volume. If the patient advice was based on small numbers, prospective patients tended to base their selection of a physician on expert rather than patient advice (profile clicks beta=.14, P<.001; call clicks beta=.28, P=.03). However, when the group of patients substantially grew in size, prospective patients started to rely on patients rather than the expert (profile clicks beta=.23, SE=0.07, P=.004; call clicks beta=.43, SE=0.32, P=.10). The experimental study replicated and validated these findings for conflicting patient versus expert advice in a controlled setting. When patient ratings were aggregated from a high number of opinions, prospective patients’ evaluations were affected more strongly by patient than expert advice (mean_{patient positive/expert negative}=3.06, SD=0.94; mean_{expert positive/patient negative}=2.55, SD=0.89; F_1,108=4.93, P=.03). Conversely, when patient ratings were aggregated from a low volume, participants were affected more strongly by expert compared with patient advice (mean_{patient positive/expert negative}=2.36, SD=0.76; mean_{expert positive/patient negative}=3.01, SD=0.81; F_1,108=8.42, P=.004). This effect occurred despite the fact that they considered the patients to be less knowledgeable than experts.

Conclusions

When confronted with information from both sources simultaneously, prospective patients are influenced more strongly by other patients. This effect reverses when the patient rating has been aggregated from a (very) small number of individual opinions. This has important implications for how to present health care provider ratings to prospective patients to aid their decision-making process.

Online physician reviews: is there a place for them?

Web-based physician ratings are increasingly popular but imperfect proxies for clinical competence. Yet they provide valuable information to patients and providers when taken in proper context. Providers need to embrace the reviews and use them to enact positive change in order to improve the quality of our patients’ experience. Patients need to realize the limitations of online ratings, particularly with smaller sample size and be discerning about the reasons behind the review.

Validation of the Primary Care Patient Measure of Safety (PC PMOS) questionnaire

BACKGROUND

The Primary Care Patient Measure of Safety (PC PMOS) is designed to capture patient feedback about the contributing factors to patient safety incidents in primary care. It required further reliability and validity testing to produce a robust tool intended to improve safety in practice.

METHOD

490 adult patients in nine primary care practices in Greater Manchester, UK, completed the PC PMOS. Practice staff (n = 81) completed a survey on patient safety culture to assess convergent validity. Confirmatory factor analysis (CFA) assessed the construct validity and internal reliability of the PC PMOS domains and items. A multivariate analysis of variance was conducted to assess discriminant validity, and Spearman correlation was conducted to establish test-retest reliability.

RESULTS

Initial CFA results showed data did not fit the model well (a chi-square to df ratio (CMIN/DF) = 5.68; goodness-of-fit index (GFI) = 0.61, CFI = 0.57, SRMR = 0.13 and root mean square error of approximation (RMSEA) = 0.10). On the basis of large modification indices (>10), standardised residuals >± 2.58 and assessment of item content; 22 items were removed. This revised nine-factor model (28 items) was found to fit the data satisfactorily (CMIN/DF = 2.51; GFI = 0.87, CFI = 0.91, SRMR = 0.04 and RMSEA = 0.05). New factors demonstrated good internal reliability with average inter-item correlations ranging from 0.20 to 0.70. The PC PMOS demonstrated good discriminant validity between primary care practices (F = 2.64, df = 72, p < 0.001) and showed some association with practice staff safety score (convergent validity) but failed to reach statistical significance (r = -0.64, k = 9, p = 0.06).

CONCLUSION

This study led to a reliable and valid 28-item PC PMOS. It could enhance or complement current data collection methods used in primary care to identify and prevent error.

"But His Yelp Reviews Are Awful!": Analysis of General Surgeons' Yelp Reviews

Background

Patients use Web-based platforms to review general surgeons. However, little is known about the free-form text and structured content of the reviews or how they relate to the physicians’ characteristics or their practices.

Objective

This observational study aimed to analyze the Web-based reviews of general surgeons on the west side of Los Angeles.

Methods

Demographics, practice characteristics, and Web-based presence were recorded. We evaluated frequency and types of Yelp reviews and assigned negative remarks to 5 categories. Tabulated results were evaluated using independent t test, one-way analysis of variance, and Pearson correlation analysis to determine associations between the number of total and negative reviews with respect to practice structure and physician characteristics.

Results

Of the 146 general surgeons, 51 (35%) had at least 1 review and 29 (20%) had at least 1 negative review. There were 806 total reviews, 679 (84.2%) positive reviews and 127 (15.8%) negative reviews. The negative reviews contained a total of 376 negative remarks, categorized into physician demeanor (124/376, 32.9%), clinical outcomes (81/376, 22%), office or staff (83/376, 22%), scheduling (44/376, 12%), and billing (44/376, 12%). Surgeons with a professional website had significantly more reviews than those without (P=.003). Surgeons in private practice had significantly more reviews (P=.002) and more negative reviews (P=.03) than surgeons who were institution employed. A strong and direct correlation was found between a surgeon’s number of reviews and number of negative reviews (P<.001).

Conclusions

As the most common category of complaints was about physician demeanor, surgeons may optimize their Web-based reputation by improving their bedside manner. A surgeon’s Web presence, private practice, and the total number of reviews are significantly associated with both positive and negative reviews.

Predicting HCAHPS scores from hospitals' social media pages: A sentiment analysis

BACKGROUND

Social media is an important communication channel that can help hospitals and consumers obtain feedback about quality of care. However, despite the potential value of insight from consumers who post comments about hospital care on social media, there has been little empirical research on the relationship between patients' anecdotal feedback and formal measures of patient experience.

PURPOSE

The aim of the study was to test the association between informal feedback posted in the Reviews section of hospitals' Facebook pages and scores on two global items from the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey, Overall Hospital Rating and Willingness to Recommend the Hospital.

METHODOLOGY/APPROACH

We retrieved star ratings and anecdotal comments posted in Reviews sections of 131 hospitals' Facebook pages. Using a machine learning algorithm, we analyzed 57,985 comments to measure consumers' sentiment about the hospitals. We used regression analysis to determine whether consumers' quantitative and qualitative postings would predict global measures from the HCAHPS survey.

RESULTS

Both number of stars and the number of positive comments posted on hospitals' Facebook Reviews sections were associated with higher overall ratings and willingness to recommend the hospital. The findings suggest that patients' informal comments help predict a hospital's formal measures of patient experience.

CONCLUSION

Consistent with crowd wisdom, ordinary consumers may have valid insights that can help others to assess patient experience at a hospital. Given that some people will judge hospital quality based on opinions voiced in social media, further research should continue to explore associations between anecdotal commentary and a variety of quality indicators.

PRACTICE IMPLICATIONS

Administrators can tap into the wealth of commentary on social media as the forum continues to expand its influence in health care. Comments on social media may also serve as an early snapshot of patient-reported experiences, alerting administrators to problems that may appear in subsequent HCAHPS survey results.

What Do Patients Say About Doctors Online? A Systematic Review of Studies on Patient Online Reviews

Background

The number of patient online reviews (PORs) has grown significantly, and PORs have played an increasingly important role in patients’ choice of health care providers.

Objective

The objective of our study was to systematically review studies on PORs, summarize the major findings and study characteristics, identify literature gaps, and make recommendations for future research.

Methods

A major database search was completed in January 2019. Studies were included if they (1) focused on PORs of physicians and hospitals, (2) reported qualitative or quantitative results from analysis of PORs, and (3) peer-reviewed empirical studies. Study characteristics and major findings were synthesized using predesigned tables.

Results

A total of 63 studies (69 articles) that met the above criteria were included in the review. Most studies (n=48) were conducted in the United States, including Puerto Rico, and the remaining were from Europe, Australia, and China. Earlier studies (published before 2010) used content analysis with small sample sizes; more recent studies retrieved and analyzed larger datasets using machine learning technologies. The number of PORs ranged from fewer than 200 to over 700,000. About 90% of the studies were focused on clinicians, typically specialists such as surgeons; 27% covered health care organizations, typically hospitals; and some studied both. A majority of PORs were positive and patients’ comments on their providers were favorable. Although most studies were descriptive, some compared PORs with traditional surveys of patient experience and found a high degree of correlation and some compared PORs with clinical outcomes but found a low level of correlation.

Conclusions

PORs contain valuable information that can generate insights into quality of care and patient-provider relationship, but it has not been systematically used for studies of health care quality. With the advancement of machine learning and data analysis tools, we anticipate more research on PORs based on testable hypotheses and rigorous analytic methods.

Trial Registration

International Prospective Register of Systematic Reviews (PROSPERO) CRD42018085057; https://www.crd.york.ac.uk/PROSPERO/display_record.php?RecordID=85057 (Archived by WebCite at http://www.webcitation.org/76ddvTZ1C)

Reporting of Patient Experience Data on Health Systems' Websites and Commercial Physician-Rating Websites: Mixed-Methods Analysis

BACKGROUND

Some hospitals' and health systems' websites report physician-level ratings and comments drawn from the Consumer Assessment of Healthcare Providers and Systems surveys.

OBJECTIVE

The aim was to examine the prevalence and content of health system websites reporting these data and compare narratives from these sites to narratives from commercial physician-rating sites.

METHODS

We identified health system websites active between June 1 and 30, 2016, that posted clinician reviews. For 140 randomly selected clinicians, we extracted the number of star ratings and narrative comments. We conducted a qualitative analysis of a random sample of these physicians' narrative reviews and compared these to a random sample of reviews from commercial physician-rating websites. We described composite quantitative scores for sampled physicians and compared the frequency of themes between reviews drawn from health systems' and commercial physician-rating websites.

RESULTS

We identified 42 health systems that published composite star ratings (42/42, 100%) or narratives (33/42, 79%). Most (27/42, 64%) stated that they excluded narratives deemed offensive. Of 140 clinicians, the majority had composite scores listed (star ratings: 122/140, 87.1%; narrative reviews: 114/140, 81.4%), with medians of 110 star ratings (IQR 42-175) and 25.5 (IQR 13-48) narratives. The rating median was 4.8 (IQR 4.7-4.9) out of five stars, and no clinician had a score less than 4.2. Compared to commercial physician-rating websites, we found significantly fewer negative comments on health system websites (35.5%, 76/214 vs 12.8%, 72/561, respectively; P<.001).

CONCLUSIONS

The lack of variation in star ratings on health system sites may make it difficult to differentiate between clinicians. Most health systems report that they remove offensive comments, and we notably found fewer negative comments on health system websites compared to commercial physician-rating sites.

The Effect of Online Effort and Reputation of Physicians on Patients' Choice: 3-Wave Data Analysis of China's Good Doctor Website

Background

Nowadays, patients are seeking physician information more frequently via the internet. Physician-rating websites (PRWs) have been recognized as the most convenient way to gain insight and detailed information about specific physicians before receiving consultation. However, little is known about how the information provided on PRWs may affect patients’ decisions to seek medical advice.

Objective

This study aimed to examine whether the physicians’ online efforts and their reputation have a relationship with patients’ choice of physician on PRWs.

Methods

A model, based on social exchange theory, was developed to analyze the factors associated with the number of online patients. A 3-wave data collection exercise, covering 4037 physicians on China’s Good Doctor website, was conducted during the months of February, April, and June 2017. Increases in consultation in a 60-day period were used as the dependent variable, whereas 2 series of data were analyzed using linear regression modeling. The fixed-effect model was used to analyze the 3-wave data.

Results

The adjusted R² value in the linear regression models were 0.28 and 0.27, whereas in the fixed-effect model, it was .30. Both the linear regression and fixed-effect models yielded a good fit. A positive effect of physicians’ effort on the aggregated number of online patients was identified in all models (R²=0.30 and R²=0.37 in 2 regression models; R²=0.23 in fixed effect model; P<.001). The proxies of physicians’ reputations indicated different results, with total number of page views of physicians’ homepages (R²=0.43 and R²=0.46; R²=0.16; P<.001) and number of votes received (R²=0.33 and R²=0.27; R²=0.43; P<.001) being seen as positive. Virtual gifts were not significant in all models, whereas thank-you messages were only significant in the fixed-effect model (R²=0.11; P=.02). The effort made by physicians online is positively associated with their aggregated number of patients consulted, whereas the effect of a physician’s reputation remains uncertain. The control effect of a physician’s title and hospital’s level was not significant in all linear regressions.

Conclusions

Both the effort and reputation of physicians online contribute to the increased number of online patients’ consultation; however, the influence of a physician’s reputation varies. This may imply that physicians’ online effort and reputation are critical in attracting patients and that strategic manipulation of physician profiles is worthy of study. Practical insights are also discussed.

Physician Rating Websites: an Analysis of Physician Evaluation and Physician Perception

BACKGROUND

The goal of this study was to evaluate current physician ratings websites (PRWs) to determine which factors correlated to higher physician scores and evaluate physician perspective of PRWs.

METHODS

This study evaluated two popular websites, Healthgrades.com and Vitals.com, to gather information on practicing physician members of the American Shoulder and Elbow Society database. A survey was conducted of the American Shoulder and Elbow Society (ASES) membership to gather data on the perception held by individual physicians regarding PRWs.

RESULTS

We found that patients were more likely to give physicians positive reviews and the average overall score was 8.35 (3.75-10). Patient wait time (P=0.052) trended toward significance as a major factor in determining the overall scores, while ratings in both physician bedside manner (P=0.001) and physician/staff courtesy (P=0.002) were significant in reflecting the overall score given to the physician. According to our survey, a majority of the respondents were indifferent to highly unfavorable to PRWs (88%) and the validity of their ratings (78%).

CONCLUSION

As PRWs become increasingly popular amongst patients in this digital age, it is critical to understand that the scores are not reflective of a significant proportion of the physicians' patient population. Physicians can use this study to determine what affects a patient's experience and focus efforts on improving patients' perception of quality, overall satisfaction, and overall care. Consumers may use this study to increase their awareness of the potential for significant sampling error inherent in PRWs when making decisions about their care.

Analysis of Internet Review Site Comments for Spine Surgeons: How Office Staff, Physician Likeability, and Patient Outcome Are Associated With Online Evaluations

STUDY DESIGN

Observational study.

OBJECTIVE

To evaluate how online patient comments will affect website ratings for spine surgeons.

SUMMARY OF BACKGROUND DATA

With the ever-growing utilization of physician review websites, healthcare consumers are assuming more control over whom they choose for care. We evaluated patient feedback and satisfaction scores of spine surgeons using comments from three leading physician rating websites: Healthgrades.com, Vitals.com, Google.com. This is the largest review of online comments and the largest review of spine surgeon comments.

METHODS

From the North American Spine Society (NASS) membership directory, 210 spine surgeons practicing in Florida (133 orthopedic trained; 77 neurosurgery trained) with online comments available for review were identified, yielding 4701 patient comments. These were categorized according to subject: (1) surgeon competence, (2) surgeon likeability/character, (3) office staff, ease of scheduling, office environment. Type 1 and 2 comments were surgeon-dependent factors whereas type 3 comments were surgeon-independent factors. Patient comments also reported a score (1-5), 5 being the most favorable and 1 being the least favorable.

RESULTS

There were 1214 (25.8%) comments from Healthgrades, 2839 (60.4%) from Vitals, and 648 (13.8%) from Google. 89.9% (4225) of comments pertained to surgeon outcomes and likeability (comment type 1 and 2), compared with 10.1% (476) surgeon-independent comments (comment type 3) (P < 0.0001). There was a significantly higher number of favorable ratings associated with surgeon-dependent comments (types 1 and 2) compared with surgeon-independent comments (type 3). Surgeon-independent comments were associated with significantly lower scores compared with comments regarding surgeon-dependent factors on all review sites.

CONCLUSION

Spine surgeons are more likely to receive favorable reviews for factors pertaining to outcomes, likeability/character, and negative reviews based on ancillary staff interactions, billing, and office environment. Surgeons should continue to take an active role in modifying factors patients perceive as negative, even if not directly related to the physician.

LEVEL OF EVIDENCE

3.

Online communities as a new source of exploring patient dissatisfaction

PURPOSE

The purpose of this paper is to analyse whether internet forums are an appropriate source for identification of causes of dissatisfaction of patients with non-medical aspects of healthcare services.

DESIGN/METHODOLOGY/APPROACH

Based on the guidelines of netnography qualitative research the authors identify relevant posts or comments on selected online forums in which web users show their dissatisfaction with healthcare services. Five popular Slovenian forums representing different interest communities have been chosen and 42 forums' topics have been reviewed.

FINDINGS

Online communities have an important role in exploring patient dissatisfaction. Through content analysis comments were coded into meaningful categories and subcategories.

RESEARCH LIMITATIONS/IMPLICATIONS

Some comments were more explicit, while others have provided general and looser reasons for dissatisfaction, and in such cases coding and content analysis of comments was more difficult.

PRACTICAL IMPLICATIONS

Contents expressed within online communities are helpful in designing improvement activities since they enable determination of concrete relevant measures aiming at eliminating and preventing the established causes of discontent, such as instituting new policies, introducing training programs, determining desired changes in culture.

ORIGINALITY/VALUE

Usefulness of the netnography as a qualitative method of research is confirmed through confirmation that causes of dissatisfaction of Slovenian patients, which have been identified in the authors research are similar to those identified in previous research in the field of patient satisfaction conducted in Slovenia. Results constitute a new form of researching patient dissatisfaction and expose the specific causes of patient dissatisfaction with healthcare services in Slovenia.

Do German public reporting websites provide information that office-based physicians consider before referring patients to hospital? A four-step analysis

BACKGROUND

In recent years, many different performance frameworks and quality assurance systems have been developed to measure health care quality. In Germany, an external quality assurance system for hospitals was introduced in 2005. The data of these systems are often reported by public reporting websites (PRWs) to inform patients and other stakeholders interested in health care systems about health care providers' quality. However, publication is obligatory (at least in Germany) for most of the existing quality assurance measures; some may be reported voluntarily. An important target group for this information is the group of all office-based physicians as they are crucial for patients' hospital choice. However, public reporting initiatives in Germany and other countries have not increased the use of quality reports for hospital choice.

OBJECTIVES

(1) To summarize the criteria that office-based physicians consider to be of high, medium, and low importance for hospital selection when referring patients and (2) to examine whether German public reporting websites (PRWs) provide these hospital-related criteria.

METHODS

The analysis comprised four steps: 1) Five databases were systematically searched for peer-reviewed English- and German-language literature. 2) The selection of articles was based on compliance with inclusion criteria, and all the criteria relevant to the referral of patients to hospital were extracted. 3) The criteria were then divided into five main categories: structural quality, process quality, outcome quality, patient experience, and referring physicians' experience. In addition, the criteria were classified into three importance categories (high-, medium-, and low-priority criteria) according to their relevance to the referral decision. 4) We investigated whether German PRWs publicly report high-priority criteria.

RESULTS

A total of N=11 articles published in peer-reviewed journals met our inclusion criteria. The studies were published in Germany (n=4), the Netherlands (n=3), Denmark, France, Norway, and the USA (n=1 each). In total, N=86 criteria were identified, most of them relating to structural quality (n=43) and process quality (n=26). We found just n=3 outcome quality criteria, only one of which fell in the high-priority category (breast cancer indicators with clinically relevant differences). In total, n=25 low-, n=40 medium-, and n=34 high-priority criteria could be established, which is due to the fact that some criteria had been investigated in several studies evaluating the importance of some criteria differently. Most of the high-priority criteria were related to process quality. All the high-priority structural quality criteria and high-priority outcome quality criteria were available on German PRWs, whereas just 38.5 % of those relating to process quality could be identified on these portals. We also identified 66.7 % of the high-priority criteria regarding patient experience and 50.0 % concerning the referring physicians' experience. Overall, a larger amount of low- and medium-priority criteria are available on German PRWs than high-priority criteria.

DISCUSSION

A substantial amount of hospital information regarding structural quality and outcome quality is available on German PRWs. However, the development of further process quality criteria (which are currently underrepresented) should be considered, for example whether hospital physicians continue the medication initiated by office-based doctors. Also, hospital quality reports should be tailored for specific user groups, for instance for referring gynecologists or referring general practitioners (GPs).

Influence of physician evaluation portals on orthodontist selection by patients

OBJECTIVE

This survey aimed to determine the influence of physician evaluation portals (PEP) on a patient's choice of physicians, particularly orthodontists.

MATERIALS AND METHODS

Questionnaires were used to collect sociodemographic data, reasons for orthodontist selection, type of Internet use, as well as information on the knowledge, use and evaluation of 14 popular PEPs. A total of 506 questionnaires were evaluated, and a descriptive statistical evaluation was conducted using the χ² test.

RESULTS

The majority of the respondents selected orthodontists on the basis of personal recommendations by other physicians (35%), family/friends (33%) or patient referral (14%). Currently, the most popular portals in Germany, which are mostly found through Internet searches, are jameda.de (36%) and arztauskunft.de (19%). A total of 5% of the respondents have already used a PEP to evaluate a physician. Moreover, 70% of the respondents described PEPs as helpful, 28% as recommendable and 2% use PEPs regularly. Knowledge of PEPs is correlated with the level of educational attainment (p = 0.024) and the frequency of Internet use (p < 0.001).

CONCLUSION

On the selection of healthcare providers, particularly orthodontists, PEPs have little influence. Patients select physicians on the basis of personal recommendations. Physicians' concerns about negative evaluations on PEPs are unfounded given the low level of awareness of PEPs by the general populace.

Why are so few patients rating their physicians on German physician rating websites? A qualitative study

Background

Physician rating websites (PRWs) allow patients to rate, comment and discuss physicians’ quality online as a source of information for others searching for a physician. It is generally assumed that PRWs will only be helpful for users, and fair for the rated, if there are a high number of ratings. However, the number of ratings on PRWs remains low internationally and there is currently a lack of research examining the reasons why patients are not rating their physicians. The aim of this study is to therefore identify the spectrum of factors influencing people’s willingness to rate their physician on PRWs.

Methods

A mailed cross-sectional survey sent to a random sample from 4 North German cities between April and July 2016. Fifty participants who had previously used PRWs but not rated a physician provided reasons for why that had not rated a physician in a free text response. Semi-structured qualitative telephone interviews were then conducted with 22 interested participants to explore factors influencing their willingness to rate their physician on PRWs in more detail.

Results

Participants identified a total of 21 distinct incentives and disincentives for rating physicians on PRWs, which could be further categorised under the headings: user-specific, PRW-specific and physician-specific. Two key overarching groups of factors emerged: (1) factors concerning the physician-patient relationship, and (2) factors issues regarding technical aspects of PRWs.

Conclusion

These findings will be helpful in guiding future research and health policy initiatives to increase the usefulness and fairness of PRWs.

Prevention of and dealing with poor performance: an interview study about how professional associations aim to support healthcare professionals

Objective

To explore how professional associations of nine healthcare professions aim to support professionals to prevent and deal with poor performance.

Design

Qualitative interview study.

Setting

The Netherlands.

Participants

Representatives of professional associations for dentists, general practitioners, medical specialists, midwives, nurses, pharmacists, physiotherapists, psychologists and psychotherapists.

Interventions

During nine face-to-face semi-structured interviews we asked how associations aim to support professionals in prevention of and dealing with poor performance. Following the first interview, we monitored new initiatives in support over a 2.5-year period, after which we conducted a second interview. Interviews were analysed using thematic analysis.

Main outcome measures

Available policy and support regarding poor performance.

Results

Three themes emerged from our data (i.e. elaborating on professional performance, performance insight and dealing with poor performance) for which we identified a total of 10 categories of support. Support concerned professional codes, guidelines and codes of conduct, quality registers, individual performance assessment, peer consultation, practice evaluation, helpdesk and expert counselling, a protocol for dealing with poor performance, a place for support and to report poor performance, and internal disciplinary procedures.

Conclusions

This study provides an overview of support given to nine healthcare professions by their associations regarding poor performance, and identifies gaps that associations could follow up on, such as clarifying what to do when confronted with a poorly performing colleague, supporting professionals that poorly perform, and developing methods for individual performance assessment to gain performance insight. A next step would be to evaluate the use and effect of different types of support.

Physician Rating Scales Do Not Accurately Rate Physicians

The purpose of this study was to determine the proportion of questions used by online physician rating scales to directly rate physicians themselves. A systematic review was performed of online, patient-reported physician rating scales. Fourteen websites were identified containing patient-reported physician rating scales, with the most common questions pertaining to office staff courtesy, wait time, overall rating (entered, not calculated), trust/confidence in physician, and time spent with patient. Overall, 28% directly rated the physician, 48% rated both the physician and the office, and 24% rated the office alone. There is great variation in the questions used, and most fail to directly rate physicians themselves. [Orthopedics. 2018; 41(4):e445-e456.].

Use of Obstetric Practice Web Sites to Distribute Zika Virus Information to Pregnant Women During a Zika Virus Outbreak

Supplemental Digital Content is Available in the Text.

Objective:

To describe the current use of obstetric practice Web sites to disseminate Zika virus information to patients.

Design:

Review of 913 randomly selected practice Web sites and associated social media accounts in January and August 2016.

Setting:

Obstetric practice Web sites and associated social media accounts, United States of America.

Participants:

N/A.

Main Outcome Measures:

Proportion of obstetric practice Web sites and linked social media accounts providing Zika virus information.

Results:

Twenty-five percent and 35% of obstetric practice Web sites had information posted about Zika virus in January 2016 and August 2016, respectively. Between the 2 time points, the proportion of practices posting Zika virus content on Facebook and Twitter declined (Facebook: 15% in January, 9% in August; Twitter: 12% in January, 8% in August). In August, the most frequently observed Zika virus–related content themes were the use of insect repellent (14%) and travel advisories (14%). At both time points, practices affiliated with large university hospitals were more likely to have posted information on Zika virus than independent OB/GYN-only practices: January: odds ratio (OR) (95% confidence interval [CI]) = 5.68 (3.50-9.20); August: OR (95% CI) = 8.37 (5.31-13.17). Similarly, practices associated with nonuniversity hospitals were more likely to have posted information than independent OB/GYN-only practices: January: OR (95% CI) = 2.71 (1.88-3.92); August: OR (95% CI) = 6.75 (4.75-9.60).

Conclusion:

Obstetric care practices are not fully utilizing their practice Web sites to relay Zika virus information to their patients. Since practitioner-sponsored Web sites have the capacity to directly reach the populations at greatest risk for Zika virus complications, public health professionals should consider adapting their materials and provider outreach campaigns to more easily accommodate Web site–based information dissemination during this type of public health emergency. There must be greater recognition of the value information gains in the eyes of the patient when it is validated by their own provider, especially when that patient is part of the highest-risk population for a given emergency. Public health organizations should strive to minimize the burden it takes for providers to relay useful resources to patients in order to maximize the impact that those resources can have.

The art of building initial trust in medical services

Purpose

The purpose of this study is to investigate the antecedents of the patient’s initial trust in the doctor. In this sense, it is proposed that trust in the hospital plays a mediating role in the relationship between the physical evidence in the service environment and the patient’s initial trust in the doctor.

Design/methodology/approach

Two experimental studies with factorial between-subjects design with random assignment were used. The data were analyzed through an analysis of variance.

Findings

The results show that design and social factors affect the patient’s initial trust in the doctor through his trust in the hospital. The results also showed that reputation and recommendation affect the initial trust.

Originality/value

This is the first study to consider antecedents of patient’s initial trust in the doctor. Most of the studies about trust focuses on investigating trust in situations where there is a prior relationship; however, this study arises some evidences that trust starts to be constructing even before the patient properly meets the doctor. These findings are valuable because they highlight the importance of physical evidences, reputation and positive word-of-mouth for building initial trust.

Factors Impacting Online Ratings for Otolaryngologists

OBJECTIVE

To identify factors associated with online patient ratings and comments for a nationwide sample of otolaryngologists.

METHODS

Ratings, demographic information, and written comments were obtained for a random sample of otolaryngologists from HealthGrades.com and Vitals.com . Online Presence Score (OPS) was based on 10 criteria, including professional website and social media profiles. Regression analyses identified factors associated with increased rating. We evaluated for correlations between OPS and other attributes with star rating and used chi-square tests to evaluate content differences between positive and negative comments.

RESULTS

On linear regression, increased OPS was associated with higher ratings on HealthGrades and Vitals; higher ratings were also associated with younger age on Vitals and less experience on HealthGrades. However, detailed correlation studies showed weak correlation between OPS and rating; age and graduation year also showed low correlation with ratings. Negative comments more likely focused on surgeon-independent factors or poor bedside manner.

CONCLUSION

Though younger otolaryngologists with greater online presence tend to have higher ratings, weak correlations suggest that age and online presence have only a small impact on the content found on ratings websites. While most written comments are positive, deficiencies in bedside manner or other physician-independent factors tend to elicit negative comments.

Physicians' attitudes towards the media and peer-review selection of the 'best cancer doctor': comparison of two different selection methods

OBJECTIVES

The choice of doctor is an important issue for patients with cancer, and the reputation of the doctor is the single most important factor for patients to choose a doctor. Media are providing information about the 'best cancer doctor', but they vary widely in their selection methodology. We investigated cancer physicians' attitudes towards the selection of the 'best cancer doctor' by the media, by comparing two different selection methodologies: selection by media personnel or selection through peer-review system.

DESIGN

Nationwide, cross-sectional survey.

SETTING

National Cancer Center and 12 Regional Cancer Centers across Korea.

PARTICIPANTS

A total of 680 cancer care physicians participated in the survey (75.5% participation rate), and two were excluded due to incomplete response.

MAIN OUTCOME MEASURES

Physicians' opinions on the credibility, fairness, validity, helpfulness to patients, their intention to use the information and helpfulness to improve the quality of cancer care of the two different methods.

RESULTS

Only a few physicians believed that the selection method of the 'best cancer doctor' by the media personnel was credible (9.1%), fair (6.1%) or valid (10.0%). In contrast, the majority agreed that the peer-selection method of the 'best doctor' is credible (74.7%), fair (64.7%) and valid (67.4%). More physicians believed the latter methods would be useful for patients when selecting their doctor (38.5% vs 82.2%) and may lead to improvement of the quality of cancer care from the perspective of the healthcare system (12.6% vs 59.8%). The need for ensuring objectiveness and transparency was also raised.

CONCLUSION

Physicians showed different attitudes towards two different selection methods. Regulations or guidelines for selecting the 'best cancer doctor' and for disclosing the information should be considered in order to control the quality of the information and to protect the customers.

Anonymity-preserving Reputation Management System for health sector

In health sector, trust is considered important because it indirectly influences the quality of health care through patient satisfaction, adherence and the continuity of its relationship with health care professionals and the promotion of accurate and timely diagnoses. One of the important requirements of TRSs in the health sector is rating secrecy, which mandates that the identification information about the service consumer should be kept secret to prevent any privacy violation. Anonymity and trust are two imperative objectives, and no significant explicit efforts have been made to achieve both of them at the same time. In this paper, we present a framework for solving the problem of reconciling trust with anonymity in the health sector. Our solution comprises Anonymous Reputation Management (ARM) protocol and Context-aware Trustworthiness Assessment (CTA) protocol. ARM protocol ensures that only those service consumers who received a service from a specific service provider provide a recommendation score anonymously with in the specified time limit. The CTA protocol computes the reputation of a user as a service provider and as a recommender. To determine the correctness of the proposed ARM protocol, formal modelling and verification are performed using High Level Petri Nets (HLPN) and Z3 Solver. Our simulation results verify the accuracy of the proposed context-aware trust assessment scheme.

Does experience matter? A meta-analysis of physician rating websites of Orthopaedic Surgeons

PURPOSE

To perform a systematic review evaluating online ratings of Orthopaedic Surgeons to determine: (1) the number of reviews per surgeon by website, (2) whether the number of reviews and rate of review acquisition correlated with years in practice, and (3) whether the use of ratings websites varied based on the surgeons' geographic region of practice.

METHODS

The USA was divided into nine geographic regions, and the most populous city in each region was selected. HealthGrades and the American Board of Orthopaedic Surgery (ABOS) database were used to identify and screen (respectively) all Orthopaedic Surgeons within each of these nine cities. These surgeons were divided into three "age" groups by years since board certification (0-10, 10-20, and 20-30 years were assigned as Groups 1, 2, and 3, respectively). An equal number of surgeons were randomly selected from each region for final analysis. The online profiles for each surgeon were reviewed on four online physician rating websites (PRW, i.e. HealthGrades, Vitals, RateMDs, Yelp) for the number of available patient reviews. Descriptive statistics, analysis of variance (ANOVA), and Pearson correlations were used.

RESULTS

Using HealthGrades, 2802 "Orthopaedic Surgeons" were identified in nine cities. However, 1271 (45%) of these were not found in the ABOS board certification database. After randomization, a total of 351 surgeons were included in the final analysis. For these 351 surgeons, the mean number of reviews per surgeon found on all four websites was 9.0 ± 14.8 (range 0-184). The mean number of reviews did not differ between the three age groups (p > 0.05) with 8.7 ± 14.4, (2) 10.3 ± 18.3, and (3) 8.0 ± 10.8 for Groups 1, 2, and 3, respectively. However, the rate that reviews were obtained (i.e. reviews per surgeon per year) was significantly higher (p < 0.001) in Group 1 (2.6 ± 7.7 reviews per year) compared to Group 2 (1.4 ± 2.4) and Group 3 (1.1 ± 1.4). There was no correlation between the number of reviews and years in practice (R < 0.001), and there was a poor correlation between number of reviews and regional population (R = 0.199).

CONCLUSIONS

The number of reviews per surgeon did not differ significantly between the three defined age groups based on years in practice. However, surgeons with less than 10 years in practice were accumulating reviews at a significantly higher rate. Interestingly nearly half of "Orthopaedic Surgeons" listed were not found to be ABOS-certified Orthopaedic Surgeons.