Analysis of Online Reviews of Orthopaedic Surgeons and Orthopaedic Practices Using Natural Language Processing

Clinical effect and contributing factors of acupuncture for knee osteoarthritis: a systematic review and pairwise and exploratory network meta-analysis

OBJECTIVES

This study aims to evaluate (1) the effect and safety of acupuncture in patients with knee osteoarthritis (KOA) and explore (2) whether the effect of acupuncture differed according to acupuncture type, acupuncture dose and follow-up time.

DESIGN

Systematic review and pairwise and exploratory network meta-analysis.

SETTING

PubMed, Embase, Cochrane Central Register of Controlled Trials, Web of Science, China National Knowledge Infrastructure, Chinese Biomedical Literature Database, VIP Database for Chinese Technical Periodicals and Wanfang from inception to 13 November 2023.

PARTICIPANTS

Randomised controlled trials comparing acupuncture with sham acupuncture, non-steroidal anti-inflammatory drugs (NSAIDs), usual care or waiting list groups, intra-articular (IA) injection and blank groups in patients with KOA.

INTERVENTIONS

Eligible interventions included manual acupuncture (MA) and electroacupuncture (EA).

MAIN OUTCOMES MEASURES

The primary outcome was pain intensity at the end of treatment.

RESULTS

80 trials (9933 participants) were included. Very low certainty evidence suggested that acupuncture may reduce pain intensity compared with sham acupuncture (standardised mean difference, SMD -0.74, 95% CI -1.08 to -0.39, corresponded to a difference in Visual Analogue Scale of -18.50 mm, -27.00 to -9.75), NSAIDs (SMD -0.86 -1.26 to -0.46, corresponded to -21.50 mm, -31.50 to -11.50), usual care or waiting list groups (SMD -1.01, -1.47 to -0.54, corresponded to -25.25 mm, -36.75 to -13.50) and blank groups (SMD -1.65, -1.99 to -1.32, corresponded to -41.25 mm, -49.75 to -33.00), but not IA injection. Similar results were also found in other outcomes. For most of the subgroup analyses, acupuncture type, acupuncture dose and follow-up time did not show a significant relative effect. Only when compared with NSAIDs, a higher dose of acupuncture may provide greater pain relief (interaction p<0.001). The network meta-analysis revealed that electroacupuncture (SMD -0.75, 95% CI -1.34 to -0.17) had a greater effect on pain relief in patients with KOA compared with manual acupuncture.

CONCLUSIONS

The findings suggest that acupuncture may provide clinically important effects in reducing pain and improving physical function in patients with KOA, but the certainty of evidence was very low. Electroacupuncture and higher dose of acupuncture probably are two potential contributing factors.

PROSPERO REGISTRATION NUMBER

CRD42021232177.

Assessing extremely negative online patient reviews and complaints of musculoskeletal oncology surgeons in the United States: a retrospective analysis

INTRODUCTION

Physician-review websites (PRWs) are commonly used by patients while searching for a surgeon. There is no current literature investigating the factors that contribute to online one-star reviews of musculoskeletal oncology surgeons. This retrospective study aims to identify these factors to determine areas of care affecting patient's subjective reviews.

METHODS

Patient ratings and comments regarding musculoskeletal oncology surgeons from the Musculoskeletal Tumor Society (MSTS) were collected from Vitals.com. One-star reviews with comments were then classified as either operative or nonoperative. These complaints were then further classified based on content including wait time, uncontrolled pain, time spent with the physician, surgical outcomes, medical staff/institutional complaints, and bedside manner.

RESULTS

A total of 169 reviews (375 complaints) from 181 physicians were included. Of these complaints, 198 were from patients in the operative category while 177 were from patients in the nonoperative category. Bedside manner was the most common complaint. Operative patients reported higher instances of uncontrolled pain in their reviews, whereas nonoperative patients more frequently cited wait time. No significant difference in the complaints that mentioned the amount of time spent with the physician, bedside manner, a disagreement with the plan, or the medical staff or institution was found.

CONCLUSION

Online one-star reviews of musculoskeletal oncology surgeons on Vitals.com referenced both surgical and non-surgical aspects of patient encounters, with bedside manner being the most popular complaint overall. Surgical patients were more likely to complain of uncontrolled pain whereas non-operative patients were more likely to complain of wait time.

TYPE OF STUDY

Outcomes 2c.

An Analysis of Negative One-star Patient Reviews and Complaints for Pediatric Orthopaedic Surgeons throughout the United States: A Retrospective Study

OBJECTIVE

Patients commonly use physician review websites when choosing a surgeon for an elective procedure. Although data exist regarding other orthopaedic specialties, no study has investigated one-star reviews for pediatric orthopaedic surgeons. The goal of this retrospective study was to classify the factors contributing to one-star reviews of pediatric orthopaedic surgeons to identify which areas contribute to lower patient satisfaction.

METHODS

Patient ratings on a 5-star system and comments about pediatric orthopaedic surgeons were collected from the state with the most physicians registered in the "Pediatric Orthopaedic Society of North American" database for each of the 9 geographical regions of the United States as defined by the Association of American Medical Colleges. One-star reviews that included comments were classified as either surgical or nonsurgical. These comments were then further classified based on their content.

RESULTS

Three hundred fifty-four one-star reviews with 700 complaints were included in this study. Of these complaints, 481 (68.7%) were from nonsurgical patients and 219 (31.3%) were from surgical patients. Nonsurgical patients were significantly more likely to reference the amount of time spent with the physician (12.9% to 6.6%, P = 0.026), wait time (11.9% to 0.0%, P < 0.001), and bedside manner (41.2% to 22.8%, P < 0.001). Patients who said they had undergone a surgical procedure in their one-star review were significantly more likely to reference a disagreement with the physician's decision or plan (35.3% to 17.5%, P < 0.001), and uncontrolled pain (21.6% to 5.2%, P < 0.001). There was no significant difference in the comments that referenced medical staff or institutional complaints between surgical and nonsurgical patients (13.8% to 11.4%, P = 0.424).

CONCLUSION

Most one-star reviews of pediatric orthopaedic surgeons referenced interpersonal skills and other nonclinical aspects of a clinical encounter, with bedside manner being the most frequent complaint. Patients who had undergone surgical procedures were less likely to leave a one-star review, but if they did, their comment was usually about a disagreement with the physician's plan.

CLINICAL RELEVANCE

Prognostic studies III.

Identification of Emotional Spectrums of Patients Taking an Erectile Dysfunction Medication: Ontology-Based Emotion Analysis of Patient Medication Reviews on Social Media

BACKGROUND

Patient medication reviews on social networking sites provide valuable insights into the experiences and sentiments of individuals taking specific medications. Understanding the emotional spectrum expressed by patients can shed light on their overall satisfaction with medication treatment. This study aims to explore the emotions expressed by patients taking phosphodiesterase type 5 (PDE5) inhibitors and their impact on sentiment.

OBJECTIVE

This study aimed to (1) identify the distribution of 6 Parrot emotions in patient medication reviews across different patient characteristics and PDE5 inhibitors, (2) determine the relative impact of each emotion on the overall sentiment derived from the language expressed in each patient medication review while controlling for different patient characteristics and PDE5 inhibitors, and (3) assess the predictive power of the overall sentiment in explaining patient satisfaction with medication treatment.

METHODS

A data set of patient medication reviews for sildenafil, vardenafil, and tadalafil was collected from 3 popular social networking sites such as WebMD, Ask-a-Patient, and Drugs.com. The Parrot emotion model, which categorizes emotions into 6 primary classes (surprise, anger, love, joy, sadness, and fear), was used to analyze the emotional content of the reviews. Logistic regression and sentiment analysis techniques were used to examine the distribution of emotions across different patient characteristics and PDE5 inhibitors and to quantify their contribution to sentiment.

RESULTS

The analysis included 3070 patient medication reviews. The most prevalent emotions expressed were joy and sadness, with joy being the most prevalent among positive emotions and sadness being the most prevalent among negative emotions. Emotion distributions varied across patient characteristics and PDE5 inhibitors. Regression analysis revealed that joy had the strongest positive impact on sentiment, while sadness had the most negative impact. The sentiment score derived from patient reviews significantly predicted patient satisfaction with medication treatment, explaining 19% of the variance (increase in R2) when controlling for patient characteristics and PDE5 inhibitors.

CONCLUSIONS

This study provides valuable insights into the emotional experiences of patients taking PDE5 inhibitors. The findings highlight the importance of emotions in shaping patient sentiment and satisfaction with medication treatment. Understanding these emotional dynamics can aid health care providers in better addressing patient needs and improving overall patient care.

Gender and culture bias in letters of recommendation for computer science and data science masters programs

Letters of Recommendation (LORs) are widely utilized for admission to both undergraduate and graduate programs, and are becoming even more important with the decreasing role that standardized tests play in the admissions process. However, LORs are highly subjective and thus can inject recommender bias into the process, leading to an inequitable evaluation of the candidates' competitiveness and competence. Our study utilizes natural language processing methods and manually determined ratings to investigate gender and cultural differences and biases in LORs written for STEM Master's program applicants. We generate features to measure important characteristics of the LORs and then compare these characteristics across groups based on recommender gender, applicant gender, and applicant country of origin. One set of features, which measure the underlying sentiment, tone, and emotions associated with each LOR, is automatically generated using IBM Watson's Natural Language Understanding (NLU) service. The second set of features is measured manually by our research team and quantifies the relevance, specificity, and positivity of each LOR. We identify and discuss features that exhibit statistically significant differences across gender and culture study groups. Our analysis is based on approximately 4000 applications for the MS in Data Science and MS in Computer Science programs at Fordham University. To our knowledge, no similar study has been performed on these graduate programs.

What Makes a 5-Star Orthopaedic Sports Medicine Surgeon? An Analysis of Positive Online Patient Reviews

Background

The increased focus on patient satisfaction has led to growth in the use of physician rating websites.

Purpose

To analyze the factors associated with online 5-star patient reviews for orthopaedic sports medicine surgeons.

Study Design

Cross-sectional study.

Methods

A total of 70 orthopaedic sports medicine surgeons were randomly selected from the AOSSM website. A search was performed for these surgeons on Yelp.com. All reviews other than 5 stars (of a possible 5 stars) were excluded from the study. Each review was categorized as referring to a surgical or nonsurgical aspect of care, and each comment within the review was categorized as being clinically or nonclinically related. Comments were further subcategorized by specific features such as bedside manner, clinical outcomes, and patient education. Categorical variables were analyzed using the chi-square test.

Results

Overall, 400 five-star reviews were included in the study, comprising 1225 total positive comments. Of the 400 five-star reviews, 200 (50%) were from surgically treated patients, and 200 (50%) were from nonsurgically treated patients. Of the 1225 positive comments, 505 (41%) were clinically related, and 720 (59%) were nonclinical. The most common positive clinical comments were for clear treatment plans (191 reviews [48%]), good outcomes (173 reviews [43%]), and providing alternative treatment plans (55 reviews [14%]). The most common positive nonclinical comments were for good physician bedside manner (287 reviews [72%]), friendly/professional staff (194 reviews [49%]), and ease of scheduling (68 reviews [17%]).

Conclusion

The majority of 5-star patient reviews left positive comments regarding nonclinical aspects of care such as physician bedside manner and friendly staff. The most common positive comments regarding clinical aspects concerned good outcomes and clear treatment plans. The overall most common positive comment, in both surgically and nonsurgically treated patients, referred to good bedside manner.

The Relationship between Internet Patient Satisfaction Ratings and COVID-19 Outcomes

Our prior research showed that patient experience-as reported by Google, Yelp, and the Hospital Consumer Assessment of Healthcare Providers and Systems survey-is associated with health outcomes. Upon learning that COVID-19 mortality rates differed among U.S. geographic areas, we sought to determine if COVID-19 outcomes were associated with patient experience. We reviewed daily, U.S.-county-level-accrued COVID-19 infections and deaths during the first year of the pandemic using each locality's mean online patient review rating, correcting for county-level demographic factors. We found doctor star ratings were significantly associated with COVID-19 outcomes. We estimated the absolute risk reduction (ARR) and relative risk reduction (RRR) for each outcome by comparing the real-world-observed outcomes, observed with the mean star rating, to the outcomes predicted by our model with a 0.3 unit higher average star rating. Geographic areas with higher patient satisfaction online review ratings in our models had substantially better COVID-19 outcomes. Our models predict that, had medical practices nationwide maintained a 4-star average online review rating-a 0.3-star increase above the current national average-the U.S may have experienced a nearly 11% lower COVID-19 infection rate and a nearly 17% lower death rate among those infected.

Characterizing Single-star Negative Online Reviews of Orthopaedic Trauma Association Members

INTRODUCTION

The purpose of this study was to characterize factors that contribute to 1-star negative reviews regarding orthopaedic trauma surgeons.

METHODS

A search was done for Orthopaedic Trauma Association members on Yelp.com, Healthgrade.com, and Vitals.com in New York, Boston, San Francisco, Los Angeles, Dallas, Phoenix, Seattle, Baltimore, Denver, Houston, Philadelphia, and Washington, DC. All single-star reviews (out of a possible 5 stars) were included in this study. Reviews were categorized as either clinical or nonclinical and then further subcategorized. Categorical variables were analyzed using a chi-square test. The rate ratio (the ratio of the rate for nonsurgical divided by surgical reviews) was determined for each category.

RESULTS

Two hundred eighty-eight single-star reviews were included in the study, comprising 655 total complaints. Of all complaints, 274 (41.8%) were clinically related and 381 (58.2%) were nonclinical. Of the 288 single-star reviews, 96 (33.3%) were from surgically treated patients and 192 (66.7%) were from nonsurgical patients. Most complaints were in reference to nonclinical aspects of care such as physician bedside manner (173 reviews, 60%), not enough time spent with provider (58 reviews, 20%), and wait time (42 complaints, 15%). The most common clinical complaints were for complication (61 reviews, 21%), disagree with decision/plan (49 reviews, 17%), and uncontrolled pain (45 reviews, 16%). Surgical patients had a significantly higher rate of clinical complaints than nonsurgical patients (1.57 vs. 0.64 clinical complaints per review, P < 0.001). Nonsurgical patients had a significantly higher rate of nonclinical complaints than surgical patients (1.43 vs. 1.10 nonclinical complaints per review, P < 0.001).

DISCUSSION

Most 1-star reviews referenced a nonclinical aspect of care with a physician's bedside manner being the most common complaint. Surgical patients were markedly more likely to reference a clinical aspect of care, such as complications or misdiagnosis compared with nonsurgical patients, who more commonly referenced nonclinical aspects of care.

Building better pediatric surgeons: A sentiment analysis of online physician review websites

PURPOSE

Physician review websites are a heavily utilized patient tool for finding, rating, and reviewing surgeons. Natural language processing such as sentiment analysis provides a comprehensive approach to better understand the nuances of patient perception. This study utilizes sentiment analysis to examine how specific patient sentiments correspond to positive and negative experiences in online reviews of pediatric orthopedic surgeons.

METHODS

The online written reviews and star ratings of pediatric surgeons belonging to the Pediatric Orthopaedic Society of North America were obtained from healthgrades.com. A sentiment analysis package obtained compound scores of each surgeon's reviews. Inferential statistics analyzed relationships between demographic variables and star/sentiment scores. Word frequency analyses and multiple logistic regression analyses were performed on key terms.

RESULTS

A total of 749 pediatric surgeons (3830 total online reviews) were included. 80.8% were males and 33.8% were below 50 years of age. Male surgeons and younger surgeons had higher mean star ratings. Surgeon attributes including "confident" (p < 0.01) and "comfortable" (p < 0.01) improved the odds of positive reviews, while "rude" (p < 0.01) and "unprofessional" (p < 0.01) decreased these odds. Comments regarding "pain" lowered the odds of positive reviews (p < 0.01), whereas "pain-free" increased these odds (p < 0.01).

CONCLUSION

Pediatric surgeons who were younger, communicated effectively, eased pain, and curated a welcoming office setting were more likely to receive positively written online reviews. This suggests that a spectrum of interpersonal and ancillary factors impact patient experience and perceptions beyond surgical skill. These outcomes can advise pediatric surgeons on behavioral and office qualities that patients and families prioritize when rating/recommending surgeons online.

LEVEL OF EVIDENCE

IV.

Quantitative Patient-Reported Experience Measures Derived From Natural Language Processing Have a Normal Distribution and No Ceiling Effect

BACKGROUND AND OBJECTIVES

Patient-reported experience measures have the potential to guide improvement in health care delivery. Many patient-reported experience measures are limited by the presence of strong ceiling effects that limit their analytical utility.

METHODS

We used natural language processing to develop 2 new methods of evaluating patient experience using text comments and associated ordinal and categorical ratings of willingness to recommend from 1390 patients receiving specialty or nonspecialty care at our offices. One method used multivariable analysis based on linguistic factors to derive a formula to estimate the ordinal likelihood to recommend. The other method used the meaning extraction method of thematic analysis to identify words associated with categorical ratings of likelihood to recommend with which we created an equation to compute an experience score. We measured normality of the 2 score distributions and ceiling effects.

RESULTS

Spearman rank-order correlation analysis identified 36 emotional and linguistic constructs associated with ordinal rating of likelihood to recommend, 9 of which were independently associated in multivariable analysis. The calculation derived from this model corresponded with the original ordinal rating with an accuracy within 0.06 units on a 0 to 10 scale. This score and the score developed from thematic analysis both had a relatively normal distribution and limited or no ceiling effect.

CONCLUSIONS

Quantitative ratings of patient experience developed using natural language processing of text comments can have relatively normal distributions and no ceiling effect.

Predicting review helpfulness in the omnichannel retailing context: An elaboration likelihood model perspective

As increasingly retail enterprises have adopted the omnichannel retailing strategy, both online-generated and offline-generated reviews should be considered to better understand the helpfulness of online reviews in the omnichannel retailing context. Drawing on the Elaboration Likelihood Model, the present study attempts to examine the impacts of review label volume, review content length, and review label-content relevance on review helpfulness in the omnichannel retailing context. The empirical data of 2,822 product reviews were collected from Suning.com. The results of Negative Binomial Regression showed that both central cue (review label-content relevance) and peripheral cue (review content length) positively affect review helpfulness. Specifically, the positive effect of review content length on review helpfulness will be stronger when the online review is submitted from an omnichannel retailer's online store. On the contrary, the positive effect of review label-content relevance on review helpfulness will be weaker when the online review is generated from an omnichannel retailer's online channel.

Characterizing Negative Online Reviews of Pediatric Orthopaedic Surgeons

BACKGROUND

The growing focus on subjective patient experiences has created an increase in popularity for physician rating websites. The purpose of this study was to characterize extremely negative reviews of pediatric orthopaedic surgeons.

METHODS

Pediatric orthopaedic surgeons were randomly selected using the Pediatric Orthopaedic Society of North America comprehensive list of surgeons. A search was then performed on Healthgrades.com, Vitals.com, and Yelp.com for 1-star reviews. Reviews were classified into clinical and nonclinical categories. Statistical analyses were performed regarding the frequency of reviews and complaints for each respective category.

RESULTS

Of the 279 one-star reviews categorized, 248 reviews (88.9% of reviews) included nonclinical complaints, and 182 reviews (65.2% of reviews) included clinical complaints. Nonsurgical patients were associated with 255 reviews, and the remaining 24 were related to surgical patients. Of the 430 comments within reviews, 248 referenced nonclinical aspects of care, and 182 referenced clinical care. Clinical factors most frequently noted included clinical disagreement (37%), unclear treatment plan (25%), complication (17%), misdiagnosis (15%), uncontrolled pain (13%), and delay in care (8%). The most addressed nonclinical factors included physician bedside manner (68%), time spent with provider (21%), wait time (18%), unprofessional staff (17%), scheduling issues (9%), cost (8%), and billing (8%). Compared with surgical reviews, nonsurgical reviews were more likely to contain nonclinical complaints (rate ratio: 1.5; P<0.05) and less likely to contain clinical complaints (rate ratio: 0.7; P<0.05). The most common complaint by surgical patients was complications (91.7%).

CONCLUSIONS

To our knowledge, this is the first study to examine the factors associated with negative reviews of pediatric orthopaedic surgeons. The majority of reviews of pediatric orthopaedic surgeons were left by nonsurgical patients and were related to nonclinical aspects of care. We also found surgeon-dependent factors such as poor physician bedside manner, unclear treatment plan, or parents' disagreement with treatment plan were the most common reasons for negative reviews.

LEVEL OF EVIDENCE

Level IV.

Natural Language Processing and Its Use in Orthopaedic Research

PURPOSE OF REVIEW

This review aims to demonstrate how natural language processing is used in orthopaedic research.

RECENT FINDINGS

Natural language processing is a form of artificial intelligence that involves encoding human-generated text or speech into a form which can be interpreted by computers to perform a variety of tasks. Natural language processing gathers, processes, and organizes large amounts of free-text data more efficiently than humans. In orthopaedics, it has been utilized for retrospective chart review, automated reporting of electronic health record data, analyzing operative notes and radiology reports, and patient reviews of physicians and practices. Although still in its infancy, natural language processing promises to be a valuable tool in the future of orthopaedic research. It will not eliminate the need for the essential human component of questioning involved in research, but natural language processing can improve the quality, efficiency, and thoroughness of research, thus improving patient care.

Emotional tones in scientific writing: comparison of commercially funded studies and non-commercially funded orthopedic studies

Background and purpose - There is ongoing debate as to whether commercial funding influences reporting of medical studies. We asked: Is there a difference in reported tones between abstracts, introductions, and discussions of orthopedic journal studies that were commercially funded and those that were not commercially funded?Methods - We conducted a systematic PubMed search to identify commercially funded studies published in 20 orthopedic journals between January 1, 2000 and December 1, 2019. We identified commercial funding of studies by including in our search the names of 10 medical device companies with the largest revenue in 2019. Commercial funding was designated when either the study or 1 or more of the authors received funding from a medical device company directly related to the content of the study. We matched 138 commercially funded articles 1 to 1 with 138 non-commercially funded articles with the same study design, published in the same journal, within a time range of 5 years. The IBM Watson Tone Analyzer was used to determine emotional tones (anger, fear, joy, and sadness) and language style (analytical, confident, and tentative).Results - For abstract and introduction sections, we found no differences in reported tones between commercially funded and non-commercially funded studies. Fear tones (non-commercially funded studies 5.1%, commercially funded studies 0.7%, p = 0.04), and analytical tones (non-commercially funded studies 95%, commercially funded studies 88%, p = 0.03) were more common in discussions of studies that were not commercially funded.Interpretation - Commercially funded studies have comparable tones to non-commercially funded studies in the abstract and introduction. In contrast, the discussion of non-commercially funded studies demonstrated more fear and analytical tones, suggesting them to be more tentative, accepting of uncertainty, and dispassionate. As text analysis tools become more sophisticated and mainstream, it might help to discern commercial bias in scientific reports.