What are patients saying about you online? A sentiment analysis of online written reviews on Scoliosis Research Society surgeons

AI-Assisted Sentiment Analysis of ACFAS Fellowship-Trained Foot and Ankle Surgeon Online Reviews

Online physician ratings and reviews prove useful among patients when selecting a provider. Analyzing such reviews across medical and surgical specialties to determine their emotional tone through sentiment analysis yielded varying levels of positivity, negativity, and neutrality. To provide insight into what patients are saying, this study similarly analyzes the sentiment of physician ratings and reviews among foot and ankle surgeons. Healthgrades ratings and reviews, entered as of February 2024, were collected among the American College of Foot and Ankle Surgeons (ACFAS) fellows along with each surgeon's demographic information. ChatGPT was used to perform a sentiment analysis to describe the positivity, negativity, and neutrality of online physician reviews. Ratings and review sentiment were described among the sample and between sexes. Among 268 fellows, men received higher average rating scores than women (p = .02), From the 2339 reviews, women received a greater proportion of negative reviews compared to men (p < .001). The overall sentiment scores among men were higher than women (p < .001). There existed a very weak inverse relationship between ratings and years in practice (R = -0.16; p = .01). Fellowship-trained foot and ankle surgeons received predominantly positive reviews. When comparing sexes, males received higher ratings with higher sentiment scores. As patients place high credence in online reviews when selecting their provider, surgeons should remain mindful of and monitor or manage their online reputation.

Extracting big data from the internet to support the development of a new patient-reported outcome measure for breast implant illness: a proof of concept study

PURPOSE

Individuals with health conditions often use online patient forums to share their experiences. These patient data are freely available and have rarely been used in patient-reported outcomes (PRO) research. Web scraping, the automated identification and coding of webpage data, can be employed to collect patient experiences for PRO research. The objective of this study was to assess the feasibility of using web scraping to support the development of a new PRO measure for breast implant illness (BII).

METHODS

Nine publicly available BII-specific web forums were chosen post-consultation with two prominent BII advocacy leaders. The Python Selenium and Pandas packages were used to automate extraction of de-identified text from the individual posts/comments into a spreadsheet. Data were coded using a line-by-line approach and constant comparison was used to create top-level domains and sub-domains.

RESULTS

6362 unique codes were identified and organized into four top-level domains of information needs, symptom experiences, life impact of BII, and care experiences. Information needs of women included seeking/sharing information pre-breast implant surgery, post-breast implant surgery, while contemplating explant surgery, and post-explant surgery. Symptoms commonly described by women included fatigue, brain fog, and musculoskeletal symptoms. Many comments described BII's impact on daily activities and psychosocial wellbeing. Lastly, some comments described negative care experiences and experiences related to advocating for themselves to providers.

CONCLUSION

This proof-of-concept study demonstrated the feasibility of employing web scraping as a cost-effective, efficient method to understand the experiences of women with BII. These data will be used to inform the development of a BII-specific PROM.

Characterizing Negative Online Reviews of Spine Surgeons

DESIGN

Retrospective review.

OBJECTIVE

Characterize negative reviews of spine surgeons in the United States.

SUMMARY

Physician rating websites significantly influence the selection of doctors by other patients. Negative experiences are impacted by various factors, both clinical and nonclinical, geography, and practice structure. The purpose of this study was to evaluate and categorize negative reviews of spine surgeons in the United States, with a focus on surgical versus nonsurgical reviewers.

METHODS

Spine surgeons were selected from available online professional society membership directories. A search for reviews was performed on Healthgrades.com, Vitals.com, and RateMDs.com for the past 10 years. Free response reviews were coded by complaint, and qualitative analysis was performed. χ 2 and Fisher exact tests were used to compare categorical variables, and multiple comparisons were adjusted with Benjamini-Hochberg correction. A binary logistic regression model was performed for the top three most mentioned nonclinical and clinical complaint labels. A P -value <0.05 was considered statistically significant.

RESULTS

A total of 16,695 online reviews were evaluated, including 1690 one-star reviews (10.1%). Among one-star reviews, 64.7% were written by nonsurgical patients and 35.3% by surgical patients. Nonclinical and clinical comments constituted 54.9% and 45.1% of reviews, respectively. Surgeons in the South had more "bedside manner" comments (43.3%, P <0.0001), while Northeast surgeons had more "poor surgical outcome" remarks compared with all other geographic regions (14.4%, P <0.001). Practicing in the South and Northeast were independent predictors of having complaints about "bedside manner" and "poor surgical outcome," respectively.

CONCLUSION

Most one-star reviews of spine surgeons were attributed to nonsurgical patients, who tended to be unsatisfied with nonclinical factors, especially "bedside manner." However, there was substantial geographic variation. These results suggest that spine surgeons could benefit from focusing on nonclinical factors (bedside manner), especially among nonoperative patients, and that regional nuances should be considered in delivering spine care.

LEVEL OF EVIDENCE

Level- 5.

Sentiment Analysis of Online Patient-Written Reviews of Vascular Surgeons

BACKGROUND

Online patient reviews influence a patient's choice of a vascular surgeon. The aim of this study is to examine underlying factors that contribute to positive and negative patient reviews by leveraging sentiment analysis and machine learning methods.

METHODS

The Society of Vascular Surgeons publicly accessible member directory was queried and cross-referenced with a popular patient-maintained physician review website, healthgrades.com. Sentiment analysis and machine learning methods were used to analyze several parameters. Demographics (gender, age, and state of practice), star rating (of 5 stars), and written reviews were obtained for corresponding vascular surgeons. A sentiment analysis model was applied to patient-written reviews and validated against the star ratings. Student's t-test or one-way analysis of variance assessed demographic relationships with reviews. Word frequency assessments and multivariable logistic regression analyses were conducted to identify common and determinative components of written reviews.

RESULTS

A total of 1,799 vascular surgeons had public profiles with reviews. Female gender of surgeon was associated with lower star ratings (male = 4.19, female = 3.95, P < 0.01) and average sentiment score (male = 0.50, female = 0.40, P < 0.01). Younger physician age was associated with higher star rating (P = 0.02) but not average sentiment score (P = 0.12). In the Best reviews, the most commonly used one-words were Care (N = 999), Caring (N = 767), and Kind (N = 479), while the most commonly used two-word pairs were Saved/Life (N = 189), Feel/Comfortable (N = 106), and Kind/Caring (N = 104). For the Worst reviews, the most commonly used one-words were Pain (N = 254) and Rude (N = 148), while the most commonly used two-word pairs were No/One (N = 27), Waste/Time (N = 25), and Severe/Pain (N = 18). In a multiple logistic regression, satisfactory reviews were associated with words such as Confident (odds ratio [OR] = 8.93), Pain-free (OR = 4.72), Listens (OR = 2.55), and Bedside Manner (OR = 1.70), while unsatisfactory reviews were associated with words such as Rude (OR = 0.01), Arrogant (OR = 0.09), Infection (OR = 0.20), and Wait (OR = 0.48).

CONCLUSIONS

Female surgeons received significantly worse reviews and younger surgeons tended to receive better reviews. The positivity and negativity of reviews were largely related to words associated with the patient-doctor experience and pain. Vascular surgeons should focus on these 2 areas to improve patient experiences and their own reviews.

Building better pediatric surgeons: A sentiment analysis of online physician review websites

PURPOSE

Physician review websites are a heavily utilized patient tool for finding, rating, and reviewing surgeons. Natural language processing such as sentiment analysis provides a comprehensive approach to better understand the nuances of patient perception. This study utilizes sentiment analysis to examine how specific patient sentiments correspond to positive and negative experiences in online reviews of pediatric orthopedic surgeons.

METHODS

The online written reviews and star ratings of pediatric surgeons belonging to the Pediatric Orthopaedic Society of North America were obtained from healthgrades.com. A sentiment analysis package obtained compound scores of each surgeon's reviews. Inferential statistics analyzed relationships between demographic variables and star/sentiment scores. Word frequency analyses and multiple logistic regression analyses were performed on key terms.

RESULTS

A total of 749 pediatric surgeons (3830 total online reviews) were included. 80.8% were males and 33.8% were below 50 years of age. Male surgeons and younger surgeons had higher mean star ratings. Surgeon attributes including "confident" (p < 0.01) and "comfortable" (p < 0.01) improved the odds of positive reviews, while "rude" (p < 0.01) and "unprofessional" (p < 0.01) decreased these odds. Comments regarding "pain" lowered the odds of positive reviews (p < 0.01), whereas "pain-free" increased these odds (p < 0.01).

CONCLUSION

Pediatric surgeons who were younger, communicated effectively, eased pain, and curated a welcoming office setting were more likely to receive positively written online reviews. This suggests that a spectrum of interpersonal and ancillary factors impact patient experience and perceptions beyond surgical skill. These outcomes can advise pediatric surgeons on behavioral and office qualities that patients and families prioritize when rating/recommending surgeons online.

LEVEL OF EVIDENCE

IV.

Natural Language Processing in Spine Surgery: A Systematic Review of Applications, Bias, and Reporting Transparency

BACKGROUND

Natural language processing (NLP) is a discipline of machine learning concerned with the analysis of language and text. Although NLP has been applied to various forms of clinical text, the applications and utility of NLP in spine surgery remain poorly characterized. Here, we systematically reviewed studies that use NLP for spine surgery applications, and analyzed applications, bias, and reporting transparency of the studies.

METHODS

We performed a literature search using the PubMed, Scopus, and Embase databases. Data extraction was performed after appropriate screening. The risk of bias and reporting quality were assessed using the PROBAST and TRIPOD tools.

RESULTS

A total of 12 full-text articles were included. The most common diseases represented include spondylolisthesis (25%), scoliosis (17%), and lumbar disk herniation (17%). The most common procedures included spinal fusion (42%), imaging (e.g. magnetic resonance, X-ray) (25%), and scoliosis correction (17%). Reported outcomes were diverse and included incidental durotomy, venous thromboembolism, and the tone of social media posts regarding scoliosis surgery. Common sources of bias identified included the use of older methods that do not capture the nuance of a text, and not using a prespecified or standard outcome measure when evaluating NLP methods.

CONCLUSIONS

Although the application of NLP to spine surgery is expanding, current studies face limitations and none are indicated as ready for clinical use. Thus, for future studies we recommend an emphasis on transparent reporting and collaboration with NLP experts to incorporate the latest developments to improve models and contribute to further innovation.