1
|
Tse MP, Dhalla I, Nayyar D. Google star ratings of Canadian hospitals: a nationwide cross-sectional analysis. BMJ Open Qual 2024; 13:e002713. [PMID: 39038856 DOI: 10.1136/bmjoq-2023-002713] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2023] [Accepted: 06/29/2024] [Indexed: 07/24/2024] Open
Abstract
BACKGROUND Data on patients' self-reported hospital experience can help guide quality improvement. Traditional patient survey programmes are resource intensive, and results are not always publicly accessible. Unsolicited online hospital reviews are an alternative data source; however, the nature of online reviews for Canadian hospitals is unknown. METHODS We conducted a nationwide cross-sectional study of Canadian acute care hospitals with more than 10 Google Reviews during the 2018-2019 fiscal year. We characterised the volume and distribution of Google Reviews of Canadian hospitals, and assessed their correlation with hospital characteristics (teaching status, size, occupancy rate, length of stay, resource utilisation) and Canadian Patient Experience Survey on Inpatient Care (CPES-IC) scores. RESULTS 167 out of 523 (31.9%) acute care hospitals in Canada met the inclusion criteria. Among included hospitals, there was a total of 10 395 Google Reviews and a median of 35 reviews per hospital. The mean Google Star Rating for included hospitals was 2.85 out of 5, with a range of 1.36-4.57. Teaching hospitals had significantly higher mean Google Star Ratings compared with non-teaching hospitals (3.16 vs 2.81, p <0.01). There was a weak, positive correlation between hospitals' Google Star Ratings and CPES-IC 'Overall Hospital Experience' scores (p =0.04), but no significant correlation between Google Star Ratings and other hospital characteristics or subcategories of CPES-IC scores. INTERPRETATION There is significant interhospital variation in patients' self-reported care experiences at Canadian acute care hospitals. Online reviews can serve as a readily accessible source of real-time data for hospitals to monitor and improve the patient experience.
Collapse
Affiliation(s)
| | - Irfan Dhalla
- Unity Health Toronto, Toronto, Ontario, Canada
- Department of Medicine, University of Toronto, Toronto, Ontario, Canada
| | - Dhruv Nayyar
- Unity Health Toronto, Toronto, Ontario, Canada
- Department of Medicine, University of Toronto, Toronto, Ontario, Canada
| |
Collapse
|
2
|
Øyri SF, Wiig S, Tjomsland O. Influence of external assessment on quality and safety in surgery: a qualitative study of surgeons' perspectives. BMJ Open Qual 2024; 13:e002672. [PMID: 38724111 PMCID: PMC11086481 DOI: 10.1136/bmjoq-2023-002672] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2023] [Accepted: 04/17/2024] [Indexed: 05/12/2024] Open
Abstract
INTRODUCTION Transparency about the occurrence of adverse events has been a decades-long governmental priority, defining external feedback to healthcare providers as a key measure to improve the services and reduce the number of adverse events. This study aimed to explore surgeons' experiences of assessment by external bodies, with a focus on its impact on transparency, reporting and learning from serious adverse events. External bodies were defined as external inspection, police internal investigation, systems of patient injury compensation and media. METHODS Based on a qualitative study design, 15 surgeons were recruited from four Norwegian university hospitals and examined with individual semi-structured interviews. Data were analysed by deductive content analysis. RESULTS Four overarching themes were identified, related to influence of external inspection, police investigation, patient injury compensation and media publicity, (re)presented by three categories: (1) sense of criminalisation and reinforcement of guilt, being treated as suspects, (2) lack of knowledge and competence among external bodies causing and reinforcing a sense of clashing cultures between the 'medical and the outside world' with minor influence on quality improvement and (3) involving external bodies could stimulate awareness about internal issues of quality and safety, depending on relevant competence, knowledge and communication skills. CONCLUSIONS AND IMPLICATIONS This study found that external assessment might generate criminalisation and scapegoating, reinforcing the sense of having medical perspectives on one hand and external regulatory perspectives on the other, which might hinder efforts to improve quality and safety. External bodies could, however, inspire useful adjustment of internal routines and procedures. The study implies that the variety and interconnections between external bodies may expose the surgeons to challenging pressure. Further studies are required to investigate these challenges to quality and safety in surgery.
Collapse
Affiliation(s)
- Sina Furnes Øyri
- Faculty of Health Sciences, University of Stavanger, Stavanger, Norway
- Stavanger University Hospital, Stavanger, Norway
| | - Siri Wiig
- Faculty of Health Sciences, University of Stavanger, Stavanger, Norway
| | - Ole Tjomsland
- Faculty of Health Sciences, University of Stavanger, Stavanger, Norway
- Division of Quality and Specialist Areas, South-Eastern Norway Regional Health Authority, Hamar, Norway
| |
Collapse
|
3
|
Guetz B, Bidmon S. The Credibility of Physician Rating Websites: A Systematic Literature Review. Health Policy 2023; 132:104821. [PMID: 37084700 DOI: 10.1016/j.healthpol.2023.104821] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2022] [Revised: 04/05/2023] [Accepted: 04/11/2023] [Indexed: 04/23/2023]
Abstract
OBJECTIVES Increasingly, the credibility of online reviews is drawing critical attention due to the lack of control mechanisms, the constant debate about fake reviews and, last but not least, current developments in the field of artificial intelligence. For this reason, the aim of this study was to examine the extent to which assessments recorded on physician rating websites (PRWs) are credible, based on a comparison to other evaluation criteria. METHODS Referring to the PRISMA guidelines, a comprehensive literature search was conducted across different scientific databases. Data were synthesized by comparing individual statistical outcomes, objectives and conclusions. RESULTS The chosen search strategy led to a database of 36,755 studies of which 28 were ultimately included in the systematic review. The literature review yielded mixed results regarding the credibility of PRWs. While seven publications supported the credibility of PRWs, six publications found no correlation between PRWs and alternative datasets. 15 studies reported mixed results. CONCLUSIONS This study has shown that ratings on PRWs seem to be credible when relying primarily on patients' perception. However, these portals seem inadequate to represent alternative comparative values such as the medical quality of physicians. For health policy makers our results show that decisions based on patients' perceptions may be well supported by data from PRWs. For all other decisions, however, PRWs do not seem to contain sufficiently useful data.
Collapse
Affiliation(s)
- Bernhard Guetz
- Department of Marketing and International Management, Alpen-Adria- Universitaet Klagenfurt, Universitaetsstrasse 65-67, Klagenfurt am Woerthersee, 9020, Austria.
| | - Sonja Bidmon
- Department of Marketing and International Management, Alpen-Adria- Universitaet Klagenfurt, Universitaetsstrasse 65-67, Klagenfurt am Woerthersee, 9020, Austria
| |
Collapse
|
4
|
Zitek T, Bui J, Day C, Ecoff S, Patel B. A cross-sectional analysis of Yelp and Google reviews of hospitals in the United States. J Am Coll Emerg Physicians Open 2023; 4:e12913. [PMID: 36852191 PMCID: PMC9960977 DOI: 10.1002/emp2.12913] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/27/2023] Open
Abstract
Objective Patient satisfaction is now an important metric in emergency medicine, but the means by which satisfaction is assessed is evolving. We sought to examine hospital ratings on Google and Yelp as compared to those on Medicare's Care Compare (CC) and to determine if certain hospital characteristics are associated with crowdsourced ratings. Methods We performed a cross-sectional analysis of hospital ratings on Google and Yelp as compared to those on CC using data collected between July 8 and August 2, 2021. For each hospital, we recorded the CC ratings, Yelp ratings, Google ratings, and each hospital's characteristics. Using multivariable linear regression, we assessed for associations between hospital characteristics and crowdsourced ratings. We calculated Spearman's correlation coefficients for CC ratings versus crowdsourced ratings. Results Among 3000 analyzed hospitals, the median hospital ratings on Yelp and Google were 2.5 stars (interquartile ratio [IQR], 2-3) and 3 stars (IQR, 2.7-3.5), respectively. The median number of Yelp and Google reviews per hospital was 13 and 150, respectively. The correlation coefficients for Yelp and Google ratings with CC's overall star ratings were 0.19 and 0.20, respectively. For Yelp and Google ratings with CC's patient survey ratings, correlation coefficients were 0.26 and 0.22, respectively. On multivariable analysis, critical access hospitals had 0.22 (95% confidence interval [CI], 0.14-0.30) more Google stars and hospitals in the West had 0.12 (95% CI, 0.05-0.18) more Google stars than references standard hospitals. Conclusion Patients use Google more frequently than Yelp to review hospitals. Median UnS hospital ratings on Yelp and Google are 2.5 and 3 stars, respectively. Crowdsourced reviews weakly correlate with CC ratings. Critical access hospitals and hospitals in the West have higher crowdsourced ratings.
Collapse
Affiliation(s)
- Tony Zitek
- Department of Emergency MedicineMount Sinai Medical CenterMiami BeachFloridaUSA,Herbert Wertheim College of Medicine at Florida International UniversityMiamiFloridaUSA
| | - Joseph Bui
- Herbert Wertheim College of Medicine at Florida International UniversityMiamiFloridaUSA
| | - Christopher Day
- Herbert Wertheim College of Medicine at Florida International UniversityMiamiFloridaUSA
| | - Sara Ecoff
- Nova Southeastern University Dr. Kiran C. Patel College of Osteopathic MedicineFort LauderdaleFloridaUSA
| | - Brijesh Patel
- Department of Emergency MedicineMount Sinai Medical CenterMiami BeachFloridaUSA
| |
Collapse
|
5
|
Ellenbogen MI, Ellenbogen PM, Rim N, Brotman DJ. Characterizing the Relationship Between Hospital Google Star Ratings, Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) Scores, and Quality. J Patient Exp 2022; 9:23743735221092604. [PMID: 35425850 PMCID: PMC9003640 DOI: 10.1177/23743735221092604] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/04/2022] Open
Abstract
Google searches for hospitals typically yield a Google star rating (GSR). These ratings are an important source of information for consumers. The degree to which GSRs are associated with traditional quality measures has not been evaluated recently. We sought to characterize the relationship between a hospital’s GSR, its Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) scores, and Centers for Medicare and Medicaid Services (CMS) quality measures. We found a moderate association between a hospital’s GSR and its HCAHPS score. The relationship between a hospital’s GSR and CMS quality measures was statistically significant, but the magnitude was quite low. Our findings suggest that consumers should not use GSRs as a hospital quality proxy.
Collapse
Affiliation(s)
- Michael I. Ellenbogen
- Department of Medicine, Johns Hopkins University School of Medicine, Baltimore, MD, USA
- Hopkins Business of Health Initiative, Johns Hopkins University, Baltimore, MD, USA
| | - Paul M. Ellenbogen
- Department of Computer Science, Princeton University, Princeton, NJ, USA
| | - Nayoung Rim
- Department of Economics, United State Naval Academy, Annapolis, MD, USA
| | - Daniel J. Brotman
- Department of Medicine, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| |
Collapse
|
6
|
Rahim AIA, Ibrahim MI, Chua SL, Musa KI. Hospital Facebook Reviews Analysis Using a Machine Learning Sentiment Analyzer and Quality Classifier. Healthcare (Basel) 2021; 9:1679. [PMID: 34946405 PMCID: PMC8701188 DOI: 10.3390/healthcare9121679] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2021] [Revised: 11/30/2021] [Accepted: 12/02/2021] [Indexed: 02/05/2023] Open
Abstract
While experts have recognised the significance and necessity of social media integration in healthcare, no systematic method has been devised in Malaysia or Southeast Asia to include social media input into the hospital quality improvement process. The goal of this work is to explain how to develop a machine learning system for classifying Facebook reviews of public hospitals in Malaysia by using service quality (SERVQUAL) dimensions and sentiment analysis. We developed a Machine Learning Quality Classifier (MLQC) based on the SERVQUAL model and a Machine Learning Sentiment Analyzer (MLSA) by manually annotated multiple batches of randomly chosen reviews. Logistic regression (LR), naive Bayes (NB), support vector machine (SVM), and other methods were used to train the classifiers. The performance of each classifier was tested using 5-fold cross validation. For topic classification, the average F1-score was between 0.687 and 0.757 for all models. In a 5-fold cross validation of each SERVQUAL dimension and in sentiment analysis, SVM consistently outperformed other methods. The study demonstrates how to use supervised learning to automatically identify SERVQUAL domains and sentiments from patient experiences on a hospital's Facebook page. Malaysian healthcare providers can gather and assess data on patient care via the use of these content analysis technology to improve hospital quality of care.
Collapse
Affiliation(s)
- Afiq Izzudin A. Rahim
- Department of Community Medicine, School of Medical Science, Universiti Sains Malaysia, Kubang Kerian, Kota Bharu 16150, Kelantan, Malaysia; (A.I.A.R.); (K.I.M.)
| | - Mohd Ismail Ibrahim
- Department of Community Medicine, School of Medical Science, Universiti Sains Malaysia, Kubang Kerian, Kota Bharu 16150, Kelantan, Malaysia; (A.I.A.R.); (K.I.M.)
| | - Sook-Ling Chua
- Faculty of Computing and Informatics, Multimedia University, Persiaran Multimedia, Cyberjaya 63100, Selangor, Malaysia
| | - Kamarul Imran Musa
- Department of Community Medicine, School of Medical Science, Universiti Sains Malaysia, Kubang Kerian, Kota Bharu 16150, Kelantan, Malaysia; (A.I.A.R.); (K.I.M.)
| |
Collapse
|
7
|
Rahim AIA, Ibrahim MI, Musa KI, Chua SL, Yaacob NM. Patient Satisfaction and Hospital Quality of Care Evaluation in Malaysia Using SERVQUAL and Facebook. Healthcare (Basel) 2021; 9:1369. [PMID: 34683050 PMCID: PMC8544585 DOI: 10.3390/healthcare9101369] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2021] [Revised: 09/27/2021] [Accepted: 10/12/2021] [Indexed: 02/05/2023] Open
Abstract
Social media sites, dubbed patient online reviews (POR), have been proposed as new methods for assessing patient satisfaction and monitoring quality of care. However, the unstructured nature of POR data derived from social media creates a number of challenges. The objectives of this research were to identify service quality (SERVQUAL) dimensions automatically from hospital Facebook reviews using a machine learning classifier, and to examine their associations with patient dissatisfaction. From January 2017 to December 2019, empirical research was conducted in which POR were gathered from the official Facebook page of Malaysian public hospitals. To find SERVQUAL dimensions in POR, a machine learning topic classification utilising supervised learning was developed, and this study's objective was established using logistic regression analysis. It was discovered that 73.5% of patients were satisfied with the public hospital service, whereas 26.5% were dissatisfied. SERVQUAL dimensions identified were 13.2% reviews of tangible, 68.9% of reliability, 6.8% of responsiveness, 19.5% of assurance, and 64.3% of empathy. After controlling for hospital variables, all SERVQUAL dimensions except tangible and assurance were shown to be significantly related with patient dissatisfaction (reliability, p < 0.001; responsiveness, p = 0.016; and empathy, p < 0.001). Rural hospitals had a higher probability of patient dissatisfaction (p < 0.001). Therefore, POR, assisted by machine learning technologies, provided a pragmatic and feasible way for capturing patient perceptions of care quality and supplementing conventional patient satisfaction surveys. The findings offer critical information that will assist healthcare authorities in capitalising on POR by monitoring and evaluating the quality of services in real time.
Collapse
Affiliation(s)
- Afiq Izzudin A. Rahim
- Department of Community Medicine, School of Medical Science, Universiti Sains Malaysia, Kubang Kerian, Kota Bharu 16150, Kelantan, Malaysia; (A.I.A.R.); (K.I.M.)
| | - Mohd Ismail Ibrahim
- Department of Community Medicine, School of Medical Science, Universiti Sains Malaysia, Kubang Kerian, Kota Bharu 16150, Kelantan, Malaysia; (A.I.A.R.); (K.I.M.)
| | - Kamarul Imran Musa
- Department of Community Medicine, School of Medical Science, Universiti Sains Malaysia, Kubang Kerian, Kota Bharu 16150, Kelantan, Malaysia; (A.I.A.R.); (K.I.M.)
| | - Sook-Ling Chua
- Faculty of Computing and Informatics, Multimedia University, Persiaran Multimedia, Cyberjaya 63100, Selangor, Malaysia;
| | - Najib Majdi Yaacob
- Unit of Biostatistics and Research Methodology, Health Campus, School of Medical Sciences, Universiti Sains Malaysia, Kubang Kerian, Kota Bharu 16150, Kelantan, Malaysia;
| |
Collapse
|
8
|
A. Rahim AI, Ibrahim MI, Musa KI, Chua SL, Yaacob NM. Assessing Patient-Perceived Hospital Service Quality and Sentiment in Malaysian Public Hospitals Using Machine Learning and Facebook Reviews. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2021; 18:9912. [PMID: 34574835 PMCID: PMC8466628 DOI: 10.3390/ijerph18189912] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/02/2021] [Revised: 09/17/2021] [Accepted: 09/18/2021] [Indexed: 02/05/2023]
Abstract
Social media is emerging as a new avenue for hospitals and patients to solicit input on the quality of care. However, social media data is unstructured and enormous in volume. Moreover, no empirical research on the use of social media data and perceived hospital quality of care based on patient online reviews has been performed in Malaysia. The purpose of this study was to investigate the determinants of positive sentiment expressed in hospital Facebook reviews in Malaysia, as well as the association between hospital accreditation and sentiments expressed in Facebook reviews. From 2017 to 2019, we retrieved comments from 48 official public hospitals' Facebook pages. We used machine learning to build a sentiment analyzer and service quality (SERVQUAL) classifier that automatically classifies the sentiment and SERVQUAL dimensions. We utilized logistic regression analysis to determine our goals. We evaluated a total of 1852 reviews and our machine learning sentiment analyzer detected 72.1% of positive reviews and 27.9% of negative reviews. We classified 240 reviews as tangible, 1257 reviews as trustworthy, 125 reviews as responsive, 356 reviews as assurance, and 1174 reviews as empathy using our machine learning SERVQUAL classifier. After adjusting for hospital characteristics, all SERVQUAL dimensions except Tangible were associated with positive sentiment. However, no significant relationship between hospital accreditation and online sentiment was discovered. Facebook reviews powered by machine learning algorithms provide valuable, real-time data that may be missed by traditional hospital quality assessments. Additionally, online patient reviews offer a hitherto untapped indication of quality that may benefit all healthcare stakeholders. Our results confirm prior studies and support the use of Facebook reviews as an adjunct method for assessing the quality of hospital services in Malaysia.
Collapse
Affiliation(s)
- Afiq Izzudin A. Rahim
- Department of Community Medicine, School of Medical Science, Universiti Sains Malaysia, Kubang Kerian, Kota Bharu 16150, Kelantan, Malaysia; (A.I.A.R.); (K.I.M.)
| | - Mohd Ismail Ibrahim
- Department of Community Medicine, School of Medical Science, Universiti Sains Malaysia, Kubang Kerian, Kota Bharu 16150, Kelantan, Malaysia; (A.I.A.R.); (K.I.M.)
| | - Kamarul Imran Musa
- Department of Community Medicine, School of Medical Science, Universiti Sains Malaysia, Kubang Kerian, Kota Bharu 16150, Kelantan, Malaysia; (A.I.A.R.); (K.I.M.)
| | - Sook-Ling Chua
- Faculty of Computing and Informatics, Multimedia University, Persiaran Multimedia, Cyberjaya 63100, Selangor, Malaysia;
| | - Najib Majdi Yaacob
- Units of Biostatistics and Research Methodology, School of Medical Sciences, Health Campus, Universiti Sains Malaysia, Kubang Kerian, Kota Bharu 16150, Kelantan, Malaysia;
| |
Collapse
|