Crowd-sourced hospital ratings are correlated with patient satisfaction but not surgical safety

Google star ratings of Canadian hospitals: a nationwide cross-sectional analysis

BACKGROUND

Data on patients' self-reported hospital experience can help guide quality improvement. Traditional patient survey programmes are resource intensive, and results are not always publicly accessible. Unsolicited online hospital reviews are an alternative data source; however, the nature of online reviews for Canadian hospitals is unknown.

METHODS

We conducted a nationwide cross-sectional study of Canadian acute care hospitals with more than 10 Google Reviews during the 2018-2019 fiscal year. We characterised the volume and distribution of Google Reviews of Canadian hospitals, and assessed their correlation with hospital characteristics (teaching status, size, occupancy rate, length of stay, resource utilisation) and Canadian Patient Experience Survey on Inpatient Care (CPES-IC) scores.

RESULTS

167 out of 523 (31.9%) acute care hospitals in Canada met the inclusion criteria. Among included hospitals, there was a total of 10 395 Google Reviews and a median of 35 reviews per hospital. The mean Google Star Rating for included hospitals was 2.85 out of 5, with a range of 1.36-4.57. Teaching hospitals had significantly higher mean Google Star Ratings compared with non-teaching hospitals (3.16 vs 2.81, p <0.01). There was a weak, positive correlation between hospitals' Google Star Ratings and CPES-IC 'Overall Hospital Experience' scores (p =0.04), but no significant correlation between Google Star Ratings and other hospital characteristics or subcategories of CPES-IC scores.

INTERPRETATION

There is significant interhospital variation in patients' self-reported care experiences at Canadian acute care hospitals. Online reviews can serve as a readily accessible source of real-time data for hospitals to monitor and improve the patient experience.

Influence of external assessment on quality and safety in surgery: a qualitative study of surgeons' perspectives

INTRODUCTION

Transparency about the occurrence of adverse events has been a decades-long governmental priority, defining external feedback to healthcare providers as a key measure to improve the services and reduce the number of adverse events. This study aimed to explore surgeons' experiences of assessment by external bodies, with a focus on its impact on transparency, reporting and learning from serious adverse events. External bodies were defined as external inspection, police internal investigation, systems of patient injury compensation and media.

METHODS

Based on a qualitative study design, 15 surgeons were recruited from four Norwegian university hospitals and examined with individual semi-structured interviews. Data were analysed by deductive content analysis.

RESULTS

Four overarching themes were identified, related to influence of external inspection, police investigation, patient injury compensation and media publicity, (re)presented by three categories: (1) sense of criminalisation and reinforcement of guilt, being treated as suspects, (2) lack of knowledge and competence among external bodies causing and reinforcing a sense of clashing cultures between the 'medical and the outside world' with minor influence on quality improvement and (3) involving external bodies could stimulate awareness about internal issues of quality and safety, depending on relevant competence, knowledge and communication skills.

CONCLUSIONS AND IMPLICATIONS

This study found that external assessment might generate criminalisation and scapegoating, reinforcing the sense of having medical perspectives on one hand and external regulatory perspectives on the other, which might hinder efforts to improve quality and safety. External bodies could, however, inspire useful adjustment of internal routines and procedures. The study implies that the variety and interconnections between external bodies may expose the surgeons to challenging pressure. Further studies are required to investigate these challenges to quality and safety in surgery.

The Credibility of Physician Rating Websites: A Systematic Literature Review

OBJECTIVES

Increasingly, the credibility of online reviews is drawing critical attention due to the lack of control mechanisms, the constant debate about fake reviews and, last but not least, current developments in the field of artificial intelligence. For this reason, the aim of this study was to examine the extent to which assessments recorded on physician rating websites (PRWs) are credible, based on a comparison to other evaluation criteria.

METHODS

Referring to the PRISMA guidelines, a comprehensive literature search was conducted across different scientific databases. Data were synthesized by comparing individual statistical outcomes, objectives and conclusions.

RESULTS

The chosen search strategy led to a database of 36,755 studies of which 28 were ultimately included in the systematic review. The literature review yielded mixed results regarding the credibility of PRWs. While seven publications supported the credibility of PRWs, six publications found no correlation between PRWs and alternative datasets. 15 studies reported mixed results.

CONCLUSIONS

This study has shown that ratings on PRWs seem to be credible when relying primarily on patients' perception. However, these portals seem inadequate to represent alternative comparative values such as the medical quality of physicians. For health policy makers our results show that decisions based on patients' perceptions may be well supported by data from PRWs. For all other decisions, however, PRWs do not seem to contain sufficiently useful data.

A cross-sectional analysis of Yelp and Google reviews of hospitals in the United States

Objective

Patient satisfaction is now an important metric in emergency medicine, but the means by which satisfaction is assessed is evolving. We sought to examine hospital ratings on Google and Yelp as compared to those on Medicare's Care Compare (CC) and to determine if certain hospital characteristics are associated with crowdsourced ratings.

Methods

We performed a cross-sectional analysis of hospital ratings on Google and Yelp as compared to those on CC using data collected between July 8 and August 2, 2021. For each hospital, we recorded the CC ratings, Yelp ratings, Google ratings, and each hospital's characteristics. Using multivariable linear regression, we assessed for associations between hospital characteristics and crowdsourced ratings. We calculated Spearman's correlation coefficients for CC ratings versus crowdsourced ratings.

Results

Among 3000 analyzed hospitals, the median hospital ratings on Yelp and Google were 2.5 stars (interquartile ratio [IQR], 2-3) and 3 stars (IQR, 2.7-3.5), respectively. The median number of Yelp and Google reviews per hospital was 13 and 150, respectively. The correlation coefficients for Yelp and Google ratings with CC's overall star ratings were 0.19 and 0.20, respectively. For Yelp and Google ratings with CC's patient survey ratings, correlation coefficients were 0.26 and 0.22, respectively. On multivariable analysis, critical access hospitals had 0.22 (95% confidence interval [CI], 0.14-0.30) more Google stars and hospitals in the West had 0.12 (95% CI, 0.05-0.18) more Google stars than references standard hospitals.

Conclusion

Patients use Google more frequently than Yelp to review hospitals. Median UnS hospital ratings on Yelp and Google are 2.5 and 3 stars, respectively. Crowdsourced reviews weakly correlate with CC ratings. Critical access hospitals and hospitals in the West have higher crowdsourced ratings.

Characterizing the Relationship Between Hospital Google Star Ratings, Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) Scores, and Quality

Google searches for hospitals typically yield a Google star rating (GSR). These ratings are an important source of information for consumers. The degree to which GSRs are associated with traditional quality measures has not been evaluated recently. We sought to characterize the relationship between a hospital’s GSR, its Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) scores, and Centers for Medicare and Medicaid Services (CMS) quality measures. We found a moderate association between a hospital’s GSR and its HCAHPS score. The relationship between a hospital’s GSR and CMS quality measures was statistically significant, but the magnitude was quite low. Our findings suggest that consumers should not use GSRs as a hospital quality proxy.

Hospital Facebook Reviews Analysis Using a Machine Learning Sentiment Analyzer and Quality Classifier

While experts have recognised the significance and necessity of social media integration in healthcare, no systematic method has been devised in Malaysia or Southeast Asia to include social media input into the hospital quality improvement process. The goal of this work is to explain how to develop a machine learning system for classifying Facebook reviews of public hospitals in Malaysia by using service quality (SERVQUAL) dimensions and sentiment analysis. We developed a Machine Learning Quality Classifier (MLQC) based on the SERVQUAL model and a Machine Learning Sentiment Analyzer (MLSA) by manually annotated multiple batches of randomly chosen reviews. Logistic regression (LR), naive Bayes (NB), support vector machine (SVM), and other methods were used to train the classifiers. The performance of each classifier was tested using 5-fold cross validation. For topic classification, the average F1-score was between 0.687 and 0.757 for all models. In a 5-fold cross validation of each SERVQUAL dimension and in sentiment analysis, SVM consistently outperformed other methods. The study demonstrates how to use supervised learning to automatically identify SERVQUAL domains and sentiments from patient experiences on a hospital's Facebook page. Malaysian healthcare providers can gather and assess data on patient care via the use of these content analysis technology to improve hospital quality of care.

Patient Satisfaction and Hospital Quality of Care Evaluation in Malaysia Using SERVQUAL and Facebook

Social media sites, dubbed patient online reviews (POR), have been proposed as new methods for assessing patient satisfaction and monitoring quality of care. However, the unstructured nature of POR data derived from social media creates a number of challenges. The objectives of this research were to identify service quality (SERVQUAL) dimensions automatically from hospital Facebook reviews using a machine learning classifier, and to examine their associations with patient dissatisfaction. From January 2017 to December 2019, empirical research was conducted in which POR were gathered from the official Facebook page of Malaysian public hospitals. To find SERVQUAL dimensions in POR, a machine learning topic classification utilising supervised learning was developed, and this study's objective was established using logistic regression analysis. It was discovered that 73.5% of patients were satisfied with the public hospital service, whereas 26.5% were dissatisfied. SERVQUAL dimensions identified were 13.2% reviews of tangible, 68.9% of reliability, 6.8% of responsiveness, 19.5% of assurance, and 64.3% of empathy. After controlling for hospital variables, all SERVQUAL dimensions except tangible and assurance were shown to be significantly related with patient dissatisfaction (reliability, p < 0.001; responsiveness, p = 0.016; and empathy, p < 0.001). Rural hospitals had a higher probability of patient dissatisfaction (p < 0.001). Therefore, POR, assisted by machine learning technologies, provided a pragmatic and feasible way for capturing patient perceptions of care quality and supplementing conventional patient satisfaction surveys. The findings offer critical information that will assist healthcare authorities in capitalising on POR by monitoring and evaluating the quality of services in real time.

Assessing Patient-Perceived Hospital Service Quality and Sentiment in Malaysian Public Hospitals Using Machine Learning and Facebook Reviews

Social media is emerging as a new avenue for hospitals and patients to solicit input on the quality of care. However, social media data is unstructured and enormous in volume. Moreover, no empirical research on the use of social media data and perceived hospital quality of care based on patient online reviews has been performed in Malaysia. The purpose of this study was to investigate the determinants of positive sentiment expressed in hospital Facebook reviews in Malaysia, as well as the association between hospital accreditation and sentiments expressed in Facebook reviews. From 2017 to 2019, we retrieved comments from 48 official public hospitals' Facebook pages. We used machine learning to build a sentiment analyzer and service quality (SERVQUAL) classifier that automatically classifies the sentiment and SERVQUAL dimensions. We utilized logistic regression analysis to determine our goals. We evaluated a total of 1852 reviews and our machine learning sentiment analyzer detected 72.1% of positive reviews and 27.9% of negative reviews. We classified 240 reviews as tangible, 1257 reviews as trustworthy, 125 reviews as responsive, 356 reviews as assurance, and 1174 reviews as empathy using our machine learning SERVQUAL classifier. After adjusting for hospital characteristics, all SERVQUAL dimensions except Tangible were associated with positive sentiment. However, no significant relationship between hospital accreditation and online sentiment was discovered. Facebook reviews powered by machine learning algorithms provide valuable, real-time data that may be missed by traditional hospital quality assessments. Additionally, online patient reviews offer a hitherto untapped indication of quality that may benefit all healthcare stakeholders. Our results confirm prior studies and support the use of Facebook reviews as an adjunct method for assessing the quality of hospital services in Malaysia.