Online Ratings of the Patient Experience: Emergency Departments Versus Urgent Care Centers

Association Between Online Reviews of Substance Use Disorder Treatment Facilities and Drug-Induced Mortality Rates: Cross-Sectional Analysis

BACKGROUND

Drug-induced mortality across the United States has continued to rise. To date, there are limited measures to evaluate patient preferences and priorities regarding substance use disorder (SUD) treatment, and many patients do not have access to evidence-based treatment options. Patients and their families seeking SUD treatment may begin their search for an SUD treatment facility online, where they can find information about individual facilities, as well as a summary of patient-generated web-based reviews via popular platforms such as Google or Yelp. Web-based reviews of health care facilities may reflect information about factors associated with positive or negative patient satisfaction. The association between patient satisfaction with SUD treatment and drug-induced mortality is not well understood.

OBJECTIVE

The objective of this study was to examine the association between online review content of SUD treatment facilities and drug-induced state mortality.

METHODS

A cross-sectional analysis of online reviews and ratings of Substance Abuse and Mental Health Services Administration (SAMHSA)-designated SUD treatment facilities listed between September 2005 and October 2021 was conducted. The primary outcomes were (1) mean online rating of SUD treatment facilities from 1 star (worst) to 5 stars (best) and (2) average drug-induced mortality rates from the Centers for Disease Control and Prevention (CDC) WONDER Database (2006-2019). Clusters of words with differential frequencies within reviews were identified. A 3-level linear model was used to estimate the association between online review ratings and drug-induced mortality.

RESULTS

A total of 589 SAMHSA-designated facilities (n=9597 reviews) were included in this study. Drug-induced mortality was compared with the average. Approximately half (24/47, 51%) of states had below average ("low") mortality rates (mean 13.40, SD 2.45 deaths per 100,000 people), and half (23/47, 49%) had above average ("high") drug-induced mortality rates (mean 21.92, SD 3.69 deaths per 100,000 people). The top 5 themes associated with low drug-induced mortality included detoxification and addiction rehabilitation services (r=0.26), gratitude for recovery (r=-0.25), thankful for treatment (r=-0.32), caring staff and amazing experience (r=-0.23), and individualized recovery programs (r=-0.20). The top 5 themes associated with high mortality were care from doctors or providers (r=0.24), rude and insensitive care (r=0.23), medication and prescriptions (r=0.22), front desk and reception experience (r=0.22), and dissatisfaction with communication (r=0.21). In the multilevel linear model, a state with a 10 deaths per 100,000 people increase in mortality was associated with a 0.30 lower average Yelp rating (P=.005).

CONCLUSIONS

Lower online ratings of SUD treatment facilities were associated with higher drug-induced mortality at the state level. Elements of patient experience may be associated with state-level mortality. Identified themes from online, organically derived patient content can inform efforts to improve high-quality and patient-centered SUD care.

How Are Patients Reviewing Spine Surgeons Online? A Sentiment Analysis of Physician Review Website Written Comments

STUDY DESIGN

A Sentiment Analysis of online reviews of spine surgeons.

OBJECTIVES

Physician review websites have significant impact on a patient's provider selection. Written reviews are subjective, but sentiment analysis through machine learning can quantitatively analyze these reviews. This study analyzes online written reviews of spine surgeons and reports biases associated with demographic factors and trends in words utilized.

METHODS

Online written and star-reviews of spine surgeons were obtained from healthgrades.com. A sentiment analysis package was used to analyze the written reviews. The relationship of demographic variables to these scores was analyzed with t-tests and word and bigram frequency analyses were performed. Additionally, a multiple regression analysis was performed on key terms.

RESULTS

8357 reviews of 480 surgeons were analyzed. There was a significant difference between the means of sentiment analysis scores and star scores for both gender and age. Younger, male surgeons were rated more highly on average (P < .01). Word frequency analysis indicated that behavioral factors and pain were the main contributing factors to both the best and worst reviewed surgeons. Additionally, several clinically relevant words, when included in a review, affected the odds of a positive review.

CONCLUSIONS

The best reviews laud surgeons for their ability to manage pain and for exhibiting positive bedside manner. However, the worst reviews primarily focus on pain and its management, as exhibited by the frequency and multivariate analysis. Pain is a clear contributing factor to reviews, thus emphasizing the importance of establishing proper pain expectations prior to any intervention.

The Credibility of Physician Rating Websites: A Systematic Literature Review

OBJECTIVES

Increasingly, the credibility of online reviews is drawing critical attention due to the lack of control mechanisms, the constant debate about fake reviews and, last but not least, current developments in the field of artificial intelligence. For this reason, the aim of this study was to examine the extent to which assessments recorded on physician rating websites (PRWs) are credible, based on a comparison to other evaluation criteria.

METHODS

Referring to the PRISMA guidelines, a comprehensive literature search was conducted across different scientific databases. Data were synthesized by comparing individual statistical outcomes, objectives and conclusions.

RESULTS

The chosen search strategy led to a database of 36,755 studies of which 28 were ultimately included in the systematic review. The literature review yielded mixed results regarding the credibility of PRWs. While seven publications supported the credibility of PRWs, six publications found no correlation between PRWs and alternative datasets. 15 studies reported mixed results.

CONCLUSIONS

This study has shown that ratings on PRWs seem to be credible when relying primarily on patients' perception. However, these portals seem inadequate to represent alternative comparative values such as the medical quality of physicians. For health policy makers our results show that decisions based on patients' perceptions may be well supported by data from PRWs. For all other decisions, however, PRWs do not seem to contain sufficiently useful data.

Antibiotic and antitussive prescribing among urgent care and emergency department visits for respiratory diagnoses in a large health system

Objective

Urgent care centers (UCs) commonly evaluate patients with respiratory infections, and patients increasingly prefer UCs to emergency departments (EDs) because of their customer‐centric approach. The aim of this study is to describe antibiotic and opioid prescribing among UC and ED visits with respiratory diagnoses.

Methods

This is a cross‐sectional study of visits to 7 EDs and 6 UCs in the greater Chicago area. We included visits from July 1, 2017, to June 30, 2019, with a primary diagnosis of upper or lower respiratory infection. We describe the proportion of visits resulting in an antibiotic or antitussive prescription as well as the most frequently prescribed medications in these categories. We also describe the demographic and clinical characteristics of visits.

Results

Of 9134 ED visits, 32.9% were prescribed an antibiotic and 14.4% an antitussive (6.6% opioid). Of 41,380 UC visits for respiratory diagnoses, 57.9% were prescribed an antibiotic and 25.0% an antitussive (9.3% opioid). The most frequently prescribed antibiotics among ED and UC visits were penicillins (36.6% and 44.5%, respectively) and macrolides (44.1% and 35.3%, respectively). The most commonly prescribed opioid was codeine (55.6% and 91.0%, respectively). Median waiting room time was 16 and 5 minutes for ED and UC visits, respectively; median length of stay was 178 and 41 minutes, respectively.

Conclusions

Antibiotics and antitussives, including opioids, are frequently prescribed for ED and UC visits with non‐bacterial respiratory diagnoses. These findings suggest greater attention to the appropriateness of antibiotic prescribing in both settings and the incorporation of specific guidance on codeine products in opioid‐prescribing guidelines.

Patient Experience and Satisfaction in Online Reviews of Obstetric Care: Observational Study

Background

The quality of care in labor and delivery is traditionally measured through the Hospital Consumer Assessment of Healthcare Providers and Systems but less is known about the experiences of care reported by patients and caregivers on online sites that are more easily accessed by the public.

Objective

The aim of this study was to generate insight into the labor and delivery experience using hospital reviews on Yelp.

Methods

We identified all Yelp reviews of US hospitals posted online from May 2005 to March 2017. We used a machine learning tool, latent Dirichlet allocation, to identify 100 topics or themes within these reviews and used Pearson r to identify statistically significant correlations between topics and high (5-star) and low (1-star) ratings.

Results

A total of 1569 hospitals listed in the American Hospital Association directory had at least one Yelp posting, contributing a total of 41,095 Yelp reviews. Among those hospitals, 919 (59%) had at least one Yelp rating for labor and delivery services (median of 9 reviews), contributing a total of 6523 labor and delivery reviews. Reviews concentrated among 5-star (n=2643, 41%) and 1-star reviews (n=1934, 30%). Themes strongly associated with favorable ratings included the following: top-notch care (r=0.45, P<.001), describing staff as comforting (r=0.52, P<.001), the delivery experience (r=0.46, P<.001), modern and clean facilities (r=0.44, P<.001), and hospital food (r=0.38, P<.001). Themes strongly correlated with 1-star labor and delivery reviews included complaints to management (r=0.30, P<.001), a lack of agency among patients (r=0.47, P<.001), and issues with discharging from the hospital (r=0.32, P<.001).

Conclusions

Online review content about labor and delivery can provide meaningful information about patient satisfaction and experiences. Narratives from these reviews that are not otherwise captured in traditional surveys can direct efforts to improve the experience of obstetrical care.

Association Between Crowdsourced Health Care Facility Ratings and Mortality in US Counties

IMPORTANCE

Mortality across US counties varies considerably, from 252 to 1847 deaths per 100 000 people in 2018. Although patient satisfaction with health care is associated with patient- and facility-level health outcomes, the association between health care satisfaction and community-level health outcomes is not known.

OBJECTIVE

To examine the association between online ratings of health care facilities and mortality across US counties and to identify language specific to 1-star (lowest rating) and 5-star (highest rating) reviews in counties with high vs low mortality.

DESIGN, SETTING, AND PARTICIPANTS

This retrospective population-based cross-sectional study examined reviews and ratings of 95 120 essential health care facilities across 1301 US counties. Counties that had at least 1 essential health care facility with reviews available on Yelp, an online review platform, were included. Essential health care was defined according to the 10 essential health benefits covered by Affordable Care Act insurance plans.

MAIN OUTCOMES AND MEASURES

The mean rating of essential health care facilities was calculated by county from January 1, 2015, to December 31, 2019. Ratings were on a scale of 1 to 5 stars, with 1 being the worst rating and 5 the best. County-level composite measures of health behaviors, clinical care, social and economic factors, and physical environment were obtained from the University of Wisconsin School of Medicine and Public Health County Health Rankings database. The 2018 age-adjusted mortality by county was obtained from the Centers for Disease Control and Prevention Wide-ranging Online Data for Epidemiological Research database. Multiple linear regression analysis was used to estimate the association between mean facility rating and mortality, adjusting for county health ranking variables. Words with frequencies of use that were significantly different across 1-star and 5-star reviews in counties with high vs low mortality were identified.

RESULTS

The 95 120 facilities meeting inclusion criteria were distributed across 1301 of 3142 US counties (41.4%). At the county level, a 1-point increase in mean rating was associated with a mean (SE) age-adjusted decrease of 18.05 (3.68) deaths per 100 000 people (P < .001). Words specific to 1-star reviews in high-mortality counties included told, rude, and wait, and words specific to 5-star reviews in low-mortality counties included Dr, pain, and professional.

CONCLUSIONS AND RELEVANCE

This study found that, at the county level, higher online ratings of essential health care facilities were associated with lower mortality. Equivalent online ratings did not necessarily reflect equivalent experiences of care across counties with different mortality levels, as evidenced by variations in the frequency of use of key words in reviews. These findings suggest that online ratings and reviews may provide insight into unequal experiences of essential health care.

Online Reviews of Mental Health Treatment Facilities: Narrative Themes Associated With Positive and Negative Ratings

OBJECTIVE

Previous studies indicate that patients' satisfaction with mental health care is correlated with both treatment outcomes and quality of life. The aims of this study were to describe online reviews of mental health treatment facilities, including key themes in review content, and to evaluate the correlation between narrative review themes, facility characteristics, and review ratings.

METHODS

United States National Mental Health Services Survey (N-MHSS) facilities were linked to corresponding Yelp pages, created between March 2007 and September 2019. Correlations between review ratings and both machine learning-generated latent Dirichlet allocation topics and N-MHSS-reported facility characteristics were measured by using Spearman's rank-order correlation coefficient. Significance was defined by a Bonferroni-adjusted p<0.001.

RESULTS

Of 10,191 unique mental health treatment facilities, 1,383 (13.6%) had relevant Yelp pages with 8,133 corresponding reviews. The number of newly reviewed facilities and the number of new reviews increased throughout the study period. Narrative topics positively correlated with review ratings included caring staff (Spearman's ρ=0.39) and nonpharmacologic treatment (ρ=0.16). Topics negatively correlated with review ratings included rude staff (ρ=-0.14) and safety and abuse (ρ=-0.14). Of 126 N-MHSS survey items, 11 were positively correlated with review rating, including "outpatient mental health facility" (ρ=0.13), and 33 were negatively correlated with review rating, including accepting Medicare (ρ=-0.21).

CONCLUSIONS

Narrative topics provide information beyond what is currently collected through the N-MHSS. Topics associated with positive and negative reviews, such as staff attitude toward patients, can guide improvement in patients' satisfaction and engagement with mental health care.

Access to Urologic Care at Urgent Care Centers

OBJECTIVE

To evaluate Medicaid insurance access disparities for urologic care at urgent care centers (UCCs) in the United States.

MATERIALS AND METHODS

We conducted a cross-sectional study using a "secret shopper" methodology. We sampled 240 UCCs across 8 states. Using a standardized script, researchers posed as a patient with either Medicaid or commercial insurance in the clinical setting of obstructing nephrolithiasis. The primary study endpoint was whether a patient's insurance (Medicaid vs commercial) was accepted. We assessed factors associated with Medicaid acceptance using logistic regression models adjusted for state-level and facility-level characteristics. Additionally, we calculated triage rates, emergency department referral rates, and the ability of a UCC to refer the patient to a specialist.

RESULTS

Of 240 UCCs contacted, 239 (99.6%) accepted commercial insurance and 159 (66.2%) accepted Medicaid. UCCs in Medicaid expansion states more frequently accepted patients with Medicaid insurance (74.2% vs 58.3%, respectively, P < .01). On multivariable logistic regression analysis, state Medicaid expansion (OR 1.84, 95% CI 1.04-3.26, P = .04) and affiliation with an institution (OR 2.97, 95% CI 1.59-5.57, P < .01) were independently associated with greater odds of accepting Medicaid. Medicaid-insured patients were significantly less likely to be triaged or referred to the emergency department compared to commercial patients.

CONCLUSION

We identified significant disparities in access to UCCs for Medicaid patients presenting with a urologic condition. Given the expanding national role of UCCs, these findings highlight potential sources of insurance disparity in the context of a urologic emergency.

A Retrospective Cohort Study of Urgent Care Visits and Revisits for Headache/Migraine

OBJECTIVE

Increasingly, patients are seeking same-day care at urgent care (UC) facilities. Little is known about the health care utilization patterns of patients who visit UC facilities for headache and migraine. We examined the frequency of headache and migraine visits and revisits at UC facilities.

METHODS

We conducted a retrospective cohort study of headache not otherwise specified (NOS) and migraine visits from 67 NYC UC facilities over an eight-month period. We report descriptive analyses, the frequency of headache NOS revisits, and the elapsed time to revisits.

RESULTS

There were 10,240 patients who visited UC facilities for headache NOS or migraine within the eight-month period. The majority of patients, 6,994 (68.3%), were female, and the mean age (SD) was 35.1 (15.0) years. Most (93.9%) patients (N = 9,613) lived within 60 miles of NYC; 5.5% (N = 564) had at least one revisit, and among re-visitors, there was an average (SD) of 2.2 (0.7) visits to UC facilities during the study period and an average time to revisit (SD) of 61.3 (55.2) days.

CONCLUSIONS

In just eight months, there were >10,000 headache NOS and migraine visits to UC facilities in NYC, with half of revisits occurring within 90 days. Future work should examine headache management in UC facilities.

Assessing Patient Experience and Healthcare Quality of Dental Care Using Patient Online Reviews in the United States: Mixed Methods Study

Background

Over the last two decades, patient review websites have emerged as an essential online platform for doctor ratings and reviews. Recent studies suggested the significance of such websites as a data source for patients to choose doctors for healthcare providers to learn and improve from patient feedback and to foster a culture of trust and transparency between patients and healthcare providers. However, as compared to other medical specialties, studies of online patient reviews that focus on dentists in the United States remain absent.

Objective

This study sought to understand to what extent online patient reviews can provide performance feedbacks that reflect dental care quality and patient experience.

Methods

Using mixed informatics methods incorporating statistics, natural language processing, and domain expert evaluation, we analyzed the online patient reviews of 204,751 dentists extracted from HealthGrades with two specific aims. First, we examined the associations between patient ratings and a variety of dentist characteristics. Second, we identified topics from patient reviews that can be mapped to the national assessment of dental patient experience measured by the Patient Experience Measures from the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Dental Plan Survey.

Results

Higher ratings were associated with female dentists (t₇₁₈₈₁=2.45, P<.01, g=0.01), dentists at a younger age (F_{7, 107128}=246.97, P<.001, g=0.11), and those whose patients experienced a short wait time (F_{4, 150055}=10417.77, P<0.001, g=0.18). We also identified several topics that corresponded to CAHPS measures, including discomfort (eg, painful/painless root canal or deep cleaning), and ethics (eg, high-pressure sales, and unnecessary dental work).

Conclusions

These findings suggest that online patient reviews could be used as a data source for understanding the patient experience and healthcare quality in dentistry.

Patient and Caregiver Perceptions of Nursing Home Physicians: Insight from Yelp Reviews, 2009-2018

OBJECTIVES

To describe the content of reviews submitted on Yelp that pertain to nursing home (NH) physicians.

DESIGN

Retrospective qualitative study.

SETTING

NHs in the United States reviewed on Yelp from 2009 to 2018 with reviews that discussed NH physicians.

PARTICIPANTS

Physicians in 375 NHs in 31 states.

MEASUREMENTS

Content analysis was performed to detect recurrent themes and divergent ideas about NH physicians perceived by reviewers.

RESULTS

Average rating among NHs with physician reviews was 2.0 (standard deviation = 1.5; range = 1-5). The major themes from content analysis of reviews that discussed physicians were perceived attitudes of clinicians toward patients and caregivers, physician communication with patients or caregivers, and perceived clinical expertise of the physicians.

CONCLUSION

Unsolicited online reviews of NHs that comment on patient and caregiver perceptions of physician care represent a small but growing number of online comments about NHs. These reviews contain important information about aspects of physician care valued by NH patients and caregivers. However, the sample of comments about physicians represents a small proportion of online reviews and is not representative of all NHs. As online reviews grow in number and become more representative, these data could be used by physicians to improve perceptions of their care by NH patients and their families, and by NHs to inform expectations of their medical director and attending physicians.

Online Reviews of Specialized Drug Treatment Facilities-Identifying Potential Drivers of High and Low Patient Satisfaction

BACKGROUND

Despite the importance of high-quality and patient-centered substance use disorder treatment, there are no standardized ratings of specialized drug treatment facilities and their services. Online platforms offer insights into potential drivers of high and low patient experience.

OBJECTIVE

We sought to analyze publicly available online review content of specialized drug treatment facilities and identify themes within high and low ratings.

DESIGN

This was a retrospective analysis of online ratings and reviews of specialized drug treatment facilities in Pennsylvania listed within the 2016 National Directory of Drug and Alcohol Abuse Treatment Facilities. Latent Dirichlet Allocation, a machine learning approach to narrative text, was used to identify themes within reviews. Differential Language Analysis was then used to measure correlations between themes and star ratings.

SETTING

Online reviews of Pennsylvania's specialized drug treatment facilities posted to Google and Yelp (July 2010-August 2018).

RESULTS

A total of 7823 online ratings were posted over 8 years. The distribution was bimodal (43% 5-star and 34% 1-star). The average weighted rating of a facility was 3.3 stars. Online themes correlated with 5-star ratings were the following: focus on recovery (r = 0.53), helpfulness of staff (r = 0.43), compassionate care (r = 0.37), experienced a life-changing moment (r = 0.32), and staff professionalism (r = 0.29). Themes correlated with a 1-star rating were waiting time (r = 0.41), poor accommodations (0.26), poor phone communication (r = 0.24), medications given (0.24), and appointment availability (r = 0.23). Themes derived from review content were similar to 9 of the 14 facility-level services highlighted by the Substance Abuse and Mental Health Services Administration's National Survey of Substance Abuse Treatment Services.

CONCLUSIONS

Individuals are sharing their ratings and reviews of specialized drug treatment facilities on online platforms. Organically derived reviews of the patient experience, captured by online platforms, reveal potential drivers of high and low ratings. These represent additional areas of focus which can inform patient-centered quality metrics for specialized drug treatment facilities.

Inappropriate Antibiotic Prescribing for Respiratory Conditions Does Not Improve Press Ganey Patient Satisfaction Scores in the Emergency Department

BACKGROUND

The literature has mixed results regarding the relationship between antibiotic prescribing and patient satisfaction in the emergency department (ED) for antibiotic-inappropriate respiratory diagnoses. The objective of the study was to determine if ED patients who receive nonindicated antibiotic prescriptions for respiratory tract conditions have increased Press Ganey patient satisfaction scores compared with those who do not receive antibiotics.

METHODS

This was a retrospective observational study. Using an administrative electronic health record data set from 2 EDs in the Midwest, we identified 619 ED encounters resulting in discharge for antibiotic-inappropriate respiratory diagnoses with a corresponding Press Ganey patient satisfaction survey. We compared sociodemographics, encounter variables, and overall Press Ganey patient satisfaction scores between those who did and did not receive antibiotics. We analyzed Press Ganey scores by categorical score distribution and as a dichotomized scale of top box (5) vs other scores. A logistic regression estimated the odds of a top box Press Ganey patient satisfaction score based on antibiotic prescribing while controlling for other covariates.

RESULTS

In the final sample, 158 (26%) encounters involving antibiotic-inappropriate respiratory diagnoses involved an antibiotic prescription. There were no differences in sociodemographic, encounter or categorical, or top box Press Ganey overall patient satisfaction scores between the groups that did and did not receive inappropriate antibiotics. In the fully adjusted regression model, antibiotic prescriptions were not associated with increased odds of top box Press Ganey patient satisfaction score (odds ratio, 0.78; 95% CI, 0.53-1.14).

CONCLUSIONS

Our findings suggest that nonindicated antibiotic prescribing for respiratory tract conditions is not a primary driver of overall Press Ganey scores in the ED.

What Do Patients Say About Doctors Online? A Systematic Review of Studies on Patient Online Reviews

Background

The number of patient online reviews (PORs) has grown significantly, and PORs have played an increasingly important role in patients’ choice of health care providers.

Objective

The objective of our study was to systematically review studies on PORs, summarize the major findings and study characteristics, identify literature gaps, and make recommendations for future research.

Methods

A major database search was completed in January 2019. Studies were included if they (1) focused on PORs of physicians and hospitals, (2) reported qualitative or quantitative results from analysis of PORs, and (3) peer-reviewed empirical studies. Study characteristics and major findings were synthesized using predesigned tables.

Results

A total of 63 studies (69 articles) that met the above criteria were included in the review. Most studies (n=48) were conducted in the United States, including Puerto Rico, and the remaining were from Europe, Australia, and China. Earlier studies (published before 2010) used content analysis with small sample sizes; more recent studies retrieved and analyzed larger datasets using machine learning technologies. The number of PORs ranged from fewer than 200 to over 700,000. About 90% of the studies were focused on clinicians, typically specialists such as surgeons; 27% covered health care organizations, typically hospitals; and some studied both. A majority of PORs were positive and patients’ comments on their providers were favorable. Although most studies were descriptive, some compared PORs with traditional surveys of patient experience and found a high degree of correlation and some compared PORs with clinical outcomes but found a low level of correlation.

Conclusions

PORs contain valuable information that can generate insights into quality of care and patient-provider relationship, but it has not been systematically used for studies of health care quality. With the advancement of machine learning and data analysis tools, we anticipate more research on PORs based on testable hypotheses and rigorous analytic methods.

Trial Registration

International Prospective Register of Systematic Reviews (PROSPERO) CRD42018085057; https://www.crd.york.ac.uk/PROSPERO/display_record.php?RecordID=85057 (Archived by WebCite at http://www.webcitation.org/76ddvTZ1C)