1
|
Casciato DJ, Mateen S. AI-Assisted Sentiment Analysis of ACFAS Fellowship-Trained Foot and Ankle Surgeon Online Reviews. J Foot Ankle Surg 2024; 63:577-579. [PMID: 38879146 DOI: 10.1053/j.jfas.2024.05.014] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/05/2024] [Revised: 05/12/2024] [Accepted: 05/25/2024] [Indexed: 07/10/2024]
Abstract
Online physician ratings and reviews prove useful among patients when selecting a provider. Analyzing such reviews across medical and surgical specialties to determine their emotional tone through sentiment analysis yielded varying levels of positivity, negativity, and neutrality. To provide insight into what patients are saying, this study similarly analyzes the sentiment of physician ratings and reviews among foot and ankle surgeons. Healthgrades ratings and reviews, entered as of February 2024, were collected among the American College of Foot and Ankle Surgeons (ACFAS) fellows along with each surgeon's demographic information. ChatGPT was used to perform a sentiment analysis to describe the positivity, negativity, and neutrality of online physician reviews. Ratings and review sentiment were described among the sample and between sexes. Among 268 fellows, men received higher average rating scores than women (p = .02), From the 2339 reviews, women received a greater proportion of negative reviews compared to men (p < .001). The overall sentiment scores among men were higher than women (p < .001). There existed a very weak inverse relationship between ratings and years in practice (R = -0.16; p = .01). Fellowship-trained foot and ankle surgeons received predominantly positive reviews. When comparing sexes, males received higher ratings with higher sentiment scores. As patients place high credence in online reviews when selecting their provider, surgeons should remain mindful of and monitor or manage their online reputation.
Collapse
Affiliation(s)
| | - Sara Mateen
- Hackensack University Medical Center, Hackensack, NJ
| |
Collapse
|
2
|
Hu S, Liu J, Cornacchi SD, Klassen AF, Pusic AL, Kaur MN. Extracting big data from the internet to support the development of a new patient-reported outcome measure for breast implant illness: a proof of concept study. Qual Life Res 2024; 33:1975-1983. [PMID: 38771557 DOI: 10.1007/s11136-024-03672-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 04/23/2024] [Indexed: 05/22/2024]
Abstract
PURPOSE Individuals with health conditions often use online patient forums to share their experiences. These patient data are freely available and have rarely been used in patient-reported outcomes (PRO) research. Web scraping, the automated identification and coding of webpage data, can be employed to collect patient experiences for PRO research. The objective of this study was to assess the feasibility of using web scraping to support the development of a new PRO measure for breast implant illness (BII). METHODS Nine publicly available BII-specific web forums were chosen post-consultation with two prominent BII advocacy leaders. The Python Selenium and Pandas packages were used to automate extraction of de-identified text from the individual posts/comments into a spreadsheet. Data were coded using a line-by-line approach and constant comparison was used to create top-level domains and sub-domains. RESULTS 6362 unique codes were identified and organized into four top-level domains of information needs, symptom experiences, life impact of BII, and care experiences. Information needs of women included seeking/sharing information pre-breast implant surgery, post-breast implant surgery, while contemplating explant surgery, and post-explant surgery. Symptoms commonly described by women included fatigue, brain fog, and musculoskeletal symptoms. Many comments described BII's impact on daily activities and psychosocial wellbeing. Lastly, some comments described negative care experiences and experiences related to advocating for themselves to providers. CONCLUSION This proof-of-concept study demonstrated the feasibility of employing web scraping as a cost-effective, efficient method to understand the experiences of women with BII. These data will be used to inform the development of a BII-specific PROM.
Collapse
Affiliation(s)
- Sophia Hu
- Department of Surgery, Patient-Reported Outcomes, Value and Experience (PROVE) Center, Brigham and Women's Hospital, Harvard Medical School, 75 Francis Street, Boston, MA, 02115, USA
| | - Jinjie Liu
- Institute for Quantitative Social Science, Harvard University, 1737 Cambridge Street, Cambridge, MA, 02138, USA
| | - Sylvie D Cornacchi
- Faculty of Health Sciences, McMaster University, 1280 Main Street W, Hamilton, ON, L8S 1C7, Canada
| | - Anne F Klassen
- Faculty of Health Sciences, McMaster University, 1280 Main Street W, Hamilton, ON, L8S 1C7, Canada
| | - Andrea L Pusic
- Department of Surgery, Patient-Reported Outcomes, Value and Experience (PROVE) Center, Brigham and Women's Hospital, Harvard Medical School, 75 Francis Street, Boston, MA, 02115, USA
| | - Manraj N Kaur
- Department of Surgery, Patient-Reported Outcomes, Value and Experience (PROVE) Center, Brigham and Women's Hospital, Harvard Medical School, 75 Francis Street, Boston, MA, 02115, USA.
| |
Collapse
|
3
|
Artz N, Dalton J, Ramanathan R, Lin RT, Sadhwani S, Como CJ, Oyekan A, Tang YM, Li V, Nwankwo J, Lee JY, Shaw JD. Characterizing Negative Online Reviews of Spine Surgeons. Spine (Phila Pa 1976) 2024; 49:E154-E163. [PMID: 38351707 DOI: 10.1097/brs.0000000000004962] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/24/2023] [Accepted: 02/05/2024] [Indexed: 05/09/2024]
Abstract
DESIGN Retrospective review. OBJECTIVE Characterize negative reviews of spine surgeons in the United States. SUMMARY Physician rating websites significantly influence the selection of doctors by other patients. Negative experiences are impacted by various factors, both clinical and nonclinical, geography, and practice structure. The purpose of this study was to evaluate and categorize negative reviews of spine surgeons in the United States, with a focus on surgical versus nonsurgical reviewers. METHODS Spine surgeons were selected from available online professional society membership directories. A search for reviews was performed on Healthgrades.com, Vitals.com, and RateMDs.com for the past 10 years. Free response reviews were coded by complaint, and qualitative analysis was performed. χ 2 and Fisher exact tests were used to compare categorical variables, and multiple comparisons were adjusted with Benjamini-Hochberg correction. A binary logistic regression model was performed for the top three most mentioned nonclinical and clinical complaint labels. A P -value <0.05 was considered statistically significant. RESULTS A total of 16,695 online reviews were evaluated, including 1690 one-star reviews (10.1%). Among one-star reviews, 64.7% were written by nonsurgical patients and 35.3% by surgical patients. Nonclinical and clinical comments constituted 54.9% and 45.1% of reviews, respectively. Surgeons in the South had more "bedside manner" comments (43.3%, P <0.0001), while Northeast surgeons had more "poor surgical outcome" remarks compared with all other geographic regions (14.4%, P <0.001). Practicing in the South and Northeast were independent predictors of having complaints about "bedside manner" and "poor surgical outcome," respectively. CONCLUSION Most one-star reviews of spine surgeons were attributed to nonsurgical patients, who tended to be unsatisfied with nonclinical factors, especially "bedside manner." However, there was substantial geographic variation. These results suggest that spine surgeons could benefit from focusing on nonclinical factors (bedside manner), especially among nonoperative patients, and that regional nuances should be considered in delivering spine care. LEVEL OF EVIDENCE Level- 5.
Collapse
Affiliation(s)
- Nicolas Artz
- Department of Orthopaedic Surgery, University of Pittsburgh Medical Center, Pittsburgh, PA
- Pittsburgh Orthopaedic Spine Research (POSR) Group, Department of Orthopaedic Surgery, University of Pittsburgh, Pittsburgh, PA
| | - Jonathan Dalton
- Department of Orthopaedic Surgery, University of Pittsburgh Medical Center, Pittsburgh, PA
- Pittsburgh Orthopaedic Spine Research (POSR) Group, Department of Orthopaedic Surgery, University of Pittsburgh, Pittsburgh, PA
- Orland Bethel Family Musculoskeletal Research Center (BMRC), Pittsburgh, PA
| | - Rahul Ramanathan
- Department of Orthopaedic Surgery, University of Pittsburgh Medical Center, Pittsburgh, PA
- Pittsburgh Orthopaedic Spine Research (POSR) Group, Department of Orthopaedic Surgery, University of Pittsburgh, Pittsburgh, PA
- Orland Bethel Family Musculoskeletal Research Center (BMRC), Pittsburgh, PA
| | - Ryan T Lin
- Department of Orthopaedic Surgery, University of Pittsburgh Medical Center, Pittsburgh, PA
- Pittsburgh Orthopaedic Spine Research (POSR) Group, Department of Orthopaedic Surgery, University of Pittsburgh, Pittsburgh, PA
| | - Shaan Sadhwani
- Department of Orthopaedic Surgery, University of Pittsburgh Medical Center, Pittsburgh, PA
- Pittsburgh Orthopaedic Spine Research (POSR) Group, Department of Orthopaedic Surgery, University of Pittsburgh, Pittsburgh, PA
| | - Christopher J Como
- Department of Orthopaedic Surgery, University of Pittsburgh Medical Center, Pittsburgh, PA
- Pittsburgh Orthopaedic Spine Research (POSR) Group, Department of Orthopaedic Surgery, University of Pittsburgh, Pittsburgh, PA
- Orland Bethel Family Musculoskeletal Research Center (BMRC), Pittsburgh, PA
| | - Anthony Oyekan
- Department of Orthopaedic Surgery, University of Pittsburgh Medical Center, Pittsburgh, PA
- Pittsburgh Orthopaedic Spine Research (POSR) Group, Department of Orthopaedic Surgery, University of Pittsburgh, Pittsburgh, PA
- Orland Bethel Family Musculoskeletal Research Center (BMRC), Pittsburgh, PA
| | - Yunting Melissa Tang
- Department of Orthopaedic Surgery, University of Pittsburgh Medical Center, Pittsburgh, PA
- Pittsburgh Orthopaedic Spine Research (POSR) Group, Department of Orthopaedic Surgery, University of Pittsburgh, Pittsburgh, PA
- Orland Bethel Family Musculoskeletal Research Center (BMRC), Pittsburgh, PA
| | - Vivian Li
- Department of Orthopaedic Surgery, University of Pittsburgh Medical Center, Pittsburgh, PA
- Pittsburgh Orthopaedic Spine Research (POSR) Group, Department of Orthopaedic Surgery, University of Pittsburgh, Pittsburgh, PA
| | - Josephine Nwankwo
- Department of Orthopaedic Surgery, University of Pittsburgh Medical Center, Pittsburgh, PA
- Pittsburgh Orthopaedic Spine Research (POSR) Group, Department of Orthopaedic Surgery, University of Pittsburgh, Pittsburgh, PA
| | - Joon Y Lee
- Department of Orthopaedic Surgery, University of Pittsburgh Medical Center, Pittsburgh, PA
- Pittsburgh Orthopaedic Spine Research (POSR) Group, Department of Orthopaedic Surgery, University of Pittsburgh, Pittsburgh, PA
- Orland Bethel Family Musculoskeletal Research Center (BMRC), Pittsburgh, PA
| | - Jeremy D Shaw
- Department of Orthopaedic Surgery, University of Pittsburgh Medical Center, Pittsburgh, PA
- Pittsburgh Orthopaedic Spine Research (POSR) Group, Department of Orthopaedic Surgery, University of Pittsburgh, Pittsburgh, PA
- Orland Bethel Family Musculoskeletal Research Center (BMRC), Pittsburgh, PA
| |
Collapse
|
4
|
Cho LD, Tang JE, Pitaro N, Bai H, Cooke PV, Arvind V, Kim J, Ting W. Sentiment Analysis of Online Patient-Written Reviews of Vascular Surgeons. Ann Vasc Surg 2023; 88:249-255. [PMID: 36028181 DOI: 10.1016/j.avsg.2022.07.016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2022] [Revised: 06/29/2022] [Accepted: 07/08/2022] [Indexed: 01/05/2023]
Abstract
BACKGROUND Online patient reviews influence a patient's choice of a vascular surgeon. The aim of this study is to examine underlying factors that contribute to positive and negative patient reviews by leveraging sentiment analysis and machine learning methods. METHODS The Society of Vascular Surgeons publicly accessible member directory was queried and cross-referenced with a popular patient-maintained physician review website, healthgrades.com. Sentiment analysis and machine learning methods were used to analyze several parameters. Demographics (gender, age, and state of practice), star rating (of 5 stars), and written reviews were obtained for corresponding vascular surgeons. A sentiment analysis model was applied to patient-written reviews and validated against the star ratings. Student's t-test or one-way analysis of variance assessed demographic relationships with reviews. Word frequency assessments and multivariable logistic regression analyses were conducted to identify common and determinative components of written reviews. RESULTS A total of 1,799 vascular surgeons had public profiles with reviews. Female gender of surgeon was associated with lower star ratings (male = 4.19, female = 3.95, P < 0.01) and average sentiment score (male = 0.50, female = 0.40, P < 0.01). Younger physician age was associated with higher star rating (P = 0.02) but not average sentiment score (P = 0.12). In the Best reviews, the most commonly used one-words were Care (N = 999), Caring (N = 767), and Kind (N = 479), while the most commonly used two-word pairs were Saved/Life (N = 189), Feel/Comfortable (N = 106), and Kind/Caring (N = 104). For the Worst reviews, the most commonly used one-words were Pain (N = 254) and Rude (N = 148), while the most commonly used two-word pairs were No/One (N = 27), Waste/Time (N = 25), and Severe/Pain (N = 18). In a multiple logistic regression, satisfactory reviews were associated with words such as Confident (odds ratio [OR] = 8.93), Pain-free (OR = 4.72), Listens (OR = 2.55), and Bedside Manner (OR = 1.70), while unsatisfactory reviews were associated with words such as Rude (OR = 0.01), Arrogant (OR = 0.09), Infection (OR = 0.20), and Wait (OR = 0.48). CONCLUSIONS Female surgeons received significantly worse reviews and younger surgeons tended to receive better reviews. The positivity and negativity of reviews were largely related to words associated with the patient-doctor experience and pain. Vascular surgeons should focus on these 2 areas to improve patient experiences and their own reviews.
Collapse
Affiliation(s)
- Logan D Cho
- Icahn School of Medicine at Mount Sinai, New York, NY.
| | - Justin E Tang
- Icahn School of Medicine at Mount Sinai, New York, NY
| | | | - Halbert Bai
- Icahn School of Medicine at Mount Sinai, New York, NY
| | - Peter V Cooke
- Icahn School of Medicine at Mount Sinai, New York, NY
| | - Varun Arvind
- Department of Orthopaedics, Icahn School of Medicine at Mount Sinai, New York, NY
| | - Jun Kim
- Department of Orthopaedics, Icahn School of Medicine at Mount Sinai, New York, NY
| | - Windsor Ting
- Division of Vascular Surgery, Department of Surgery, Icahn School of Medicine at Mount Sinai, New York, NY
| |
Collapse
|
5
|
Butler LR, Tang JE, Hess SM, White CA, Arvind V, Kim JS, Allen AK, Ranade SC. Building better pediatric surgeons: A sentiment analysis of online physician review websites. J Child Orthop 2022; 16:498-504. [PMID: 36483646 PMCID: PMC9723867 DOI: 10.1177/18632521221133812] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/18/2022] [Accepted: 10/03/2022] [Indexed: 02/03/2023] Open
Abstract
PURPOSE Physician review websites are a heavily utilized patient tool for finding, rating, and reviewing surgeons. Natural language processing such as sentiment analysis provides a comprehensive approach to better understand the nuances of patient perception. This study utilizes sentiment analysis to examine how specific patient sentiments correspond to positive and negative experiences in online reviews of pediatric orthopedic surgeons. METHODS The online written reviews and star ratings of pediatric surgeons belonging to the Pediatric Orthopaedic Society of North America were obtained from healthgrades.com. A sentiment analysis package obtained compound scores of each surgeon's reviews. Inferential statistics analyzed relationships between demographic variables and star/sentiment scores. Word frequency analyses and multiple logistic regression analyses were performed on key terms. RESULTS A total of 749 pediatric surgeons (3830 total online reviews) were included. 80.8% were males and 33.8% were below 50 years of age. Male surgeons and younger surgeons had higher mean star ratings. Surgeon attributes including "confident" (p < 0.01) and "comfortable" (p < 0.01) improved the odds of positive reviews, while "rude" (p < 0.01) and "unprofessional" (p < 0.01) decreased these odds. Comments regarding "pain" lowered the odds of positive reviews (p < 0.01), whereas "pain-free" increased these odds (p < 0.01). CONCLUSION Pediatric surgeons who were younger, communicated effectively, eased pain, and curated a welcoming office setting were more likely to receive positively written online reviews. This suggests that a spectrum of interpersonal and ancillary factors impact patient experience and perceptions beyond surgical skill. These outcomes can advise pediatric surgeons on behavioral and office qualities that patients and families prioritize when rating/recommending surgeons online. LEVEL OF EVIDENCE IV.
Collapse
Affiliation(s)
| | | | | | | | | | | | | | - Sheena C Ranade
- Sheena C Ranade, Department of Orthopaedic Surgery, Icahn School of Medicine at Mount Sinai, 5 East 98th Street, New York, NY 10029, USA.
| |
Collapse
|
6
|
Quinones A, Tang JE, Vasan V, Li T, Schupper AJ, Ali M, White CA, Hannah TC, Asfaw Z, Li AY, Durbin J, Arvind V, Kim JS, Choudhri TF. Trends in Online Patient Perspectives of Neurosurgeons: A Sentiment Analysis. NEUROSURGERY OPEN 2022. [DOI: 10.1227/neuopn.0000000000000023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/05/2023] Open
|
7
|
Huang BB, Huang J, Swong KN. Natural Language Processing in Spine Surgery: A Systematic Review of Applications, Bias, and Reporting Transparency. World Neurosurg 2022; 167:156-164.e6. [PMID: 36049723 DOI: 10.1016/j.wneu.2022.08.109] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2022] [Revised: 08/22/2022] [Accepted: 08/23/2022] [Indexed: 10/31/2022]
Abstract
BACKGROUND Natural language processing (NLP) is a discipline of machine learning concerned with the analysis of language and text. Although NLP has been applied to various forms of clinical text, the applications and utility of NLP in spine surgery remain poorly characterized. Here, we systematically reviewed studies that use NLP for spine surgery applications, and analyzed applications, bias, and reporting transparency of the studies. METHODS We performed a literature search using the PubMed, Scopus, and Embase databases. Data extraction was performed after appropriate screening. The risk of bias and reporting quality were assessed using the PROBAST and TRIPOD tools. RESULTS A total of 12 full-text articles were included. The most common diseases represented include spondylolisthesis (25%), scoliosis (17%), and lumbar disk herniation (17%). The most common procedures included spinal fusion (42%), imaging (e.g. magnetic resonance, X-ray) (25%), and scoliosis correction (17%). Reported outcomes were diverse and included incidental durotomy, venous thromboembolism, and the tone of social media posts regarding scoliosis surgery. Common sources of bias identified included the use of older methods that do not capture the nuance of a text, and not using a prespecified or standard outcome measure when evaluating NLP methods. CONCLUSIONS Although the application of NLP to spine surgery is expanding, current studies face limitations and none are indicated as ready for clinical use. Thus, for future studies we recommend an emphasis on transparent reporting and collaboration with NLP experts to incorporate the latest developments to improve models and contribute to further innovation.
Collapse
Affiliation(s)
- Bonnie B Huang
- Department of Neurological Surgery, Northwestern University Feinberg School of Medicine, Chicago, Illinois, USA
| | - Jonathan Huang
- Department of Neurological Surgery, Northwestern University Feinberg School of Medicine, Chicago, Illinois, USA
| | - Kevin N Swong
- Department of Neurological Surgery, Northwestern University Feinberg School of Medicine, Chicago, Illinois, USA.
| |
Collapse
|