Data Quality Issues With Physician-Rating Websites: Systematic Review

Investigating Patient Satisfaction Through Web-Based Reviews of Norwegian Dentists: Quantitative Study Using the Meaning Extraction Method

BACKGROUND

Challenging encounters in health care professions, including in dentistry, are relatively common. Challenging encounters can be defined as stressful or emotional situations involving patients that could impact both treatment outcomes and patients' experiences. Through written web-based reviews, patients can share their experiences with health care providers, and these posts can be a useful source for investigating patient satisfaction and their experiences of challenging encounters.

OBJECTIVE

This study aims to identify dominant themes from patient-written, web-based reviews of dentists and investigate how these themes are related to patient satisfaction with dental treatment.

METHODS

The study data consisted of 11,764 reviews written by dental patients, which included 1- to 5-star ratings on overall satisfaction and free-text comments. The free-text comments were analyzed using Linguistic Inquiry and Word Count software, and the meaning extraction method was used to group words into thematic categories. These themes were used as variables in a multilevel logistic regression analysis to predict patient satisfaction.

RESULTS

Eight themes emerged from the analyses, of which 6 (75%)-explanation (odds ratio [OR] 2.56, 95% CI 2.16-3.04; P<.001), assurance (OR 3.61, 95% CI 2.57-5.06; P<.001), performance assessment (OR 2.17, 95% CI 1.84-2.55; P<.001), professional advice (OR 1.81, 95% CI 1.55-2.13; P<.001), facilities (OR 1.78, 95% CI 1.08-2.91; P=.02), and recommendation (OR 1.31, 95% CI 1.12-1.53; P<.001)-increased the odds of high patient satisfaction. The remaining themes (2/8, 25%)-consequences of treatment need (OR 0.24, 95% CI 0.20-0.29; P<.001) and patient-centered care (OR 0.62, 95% CI 0.52-0.74; P<.001)-reduced the odds of high patient satisfaction.

CONCLUSIONS

The meaning extraction method is an interesting approach to explore patients' written accounts of encounters with dental health professionals. The experiences described by patients provide insight into key elements related to patient satisfaction that can be used in the education of dental health professionals and to improve the provision of dental health services.

Data Quality in Health Research: Integrative Literature Review

BACKGROUND

Decision-making and strategies to improve service delivery must be supported by reliable health data to generate consistent evidence on health status. The data quality management process must ensure the reliability of collected data. Consequently, various methodologies to improve the quality of services are applied in the health field. At the same time, scientific research is constantly evolving to improve data quality through better reproducibility and empowerment of researchers and offers patient groups tools for secured data sharing and privacy compliance.

OBJECTIVE

Through an integrative literature review, the aim of this work was to identify and evaluate digital health technology interventions designed to support the conducting of health research based on data quality.

METHODS

A search was conducted in 6 electronic scientific databases in January 2022: PubMed, SCOPUS, Web of Science, Institute of Electrical and Electronics Engineers Digital Library, Cumulative Index of Nursing and Allied Health Literature, and Latin American and Caribbean Health Sciences Literature. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses checklist and flowchart were used to visualize the search strategy results in the databases.

RESULTS

After analyzing and extracting the outcomes of interest, 33 papers were included in the review. The studies covered the period of 2017-2021 and were conducted in 22 countries. Key findings revealed variability and a lack of consensus in assessing data quality domains and metrics. Data quality factors included the research environment, application time, and development steps. Strategies for improving data quality involved using business intelligence models, statistical analyses, data mining techniques, and qualitative approaches.

CONCLUSIONS

The main barriers to health data quality are technical, motivational, economical, political, legal, ethical, organizational, human resources, and methodological. The data quality process and techniques, from precollection to gathering, postcollection, and analysis, are critical for the final result of a study or the quality of processes and decision-making in a health care organization. The findings highlight the need for standardized practices and collaborative efforts to enhance data quality in health research. Finally, context guides decisions regarding data quality strategies and techniques.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1101/2022.05.31.22275804.

The Credibility of Physician Rating Websites: A Systematic Literature Review

OBJECTIVES

Increasingly, the credibility of online reviews is drawing critical attention due to the lack of control mechanisms, the constant debate about fake reviews and, last but not least, current developments in the field of artificial intelligence. For this reason, the aim of this study was to examine the extent to which assessments recorded on physician rating websites (PRWs) are credible, based on a comparison to other evaluation criteria.

METHODS

Referring to the PRISMA guidelines, a comprehensive literature search was conducted across different scientific databases. Data were synthesized by comparing individual statistical outcomes, objectives and conclusions.

RESULTS

The chosen search strategy led to a database of 36,755 studies of which 28 were ultimately included in the systematic review. The literature review yielded mixed results regarding the credibility of PRWs. While seven publications supported the credibility of PRWs, six publications found no correlation between PRWs and alternative datasets. 15 studies reported mixed results.

CONCLUSIONS

This study has shown that ratings on PRWs seem to be credible when relying primarily on patients' perception. However, these portals seem inadequate to represent alternative comparative values such as the medical quality of physicians. For health policy makers our results show that decisions based on patients' perceptions may be well supported by data from PRWs. For all other decisions, however, PRWs do not seem to contain sufficiently useful data.

Awareness of and interaction with physician rating websites: A cross-sectional study in Austria

To date, the digital assessment of service experiences represents a decisive process step of a feedback culture in numerous economic areas. In view of this digitalization of service assessments, the importance of Physician Rating Websites (PRWs) has also increased steadily in recent years. Even though these websites could be perceived as a powerful communication tool for the exchange of health specific information, the knowledge about whether and how different population segments use these portals has been limited so far. For this reason, our aim was to investigate the level of awareness regarding PRWs among the study population and to discover how users interact with this specific type of online portals. We performed an online survey including 558 participants. To ensure the attention and integrity of participants, attention checks were included in the questionnaire. Study participants who did not exceed the mentioned security levels were excluded from the study. Statistical analyses were carried out, using IBM SPSS Statistics 27. To illustrate the relationship between demographic variables and dependent variables, two tailed chi square tests were performed. Comparison of means and t-testing was used to investigate the relationship between psychographic variables and the dependent variables. In addition to that, the awareness levels regarding different rating portals were evaluated using descriptive methods. Our results suggest that the general awareness regarding PRWs is relatively high (75.6%, 423/558), especially among female (x21 = 9.880, P = .002), middle-aged (x29 = 26.810, P = .002), more highly educated (x24 = 19.038, P = .001), urban (x21 = 6.274, P = .012), digitally literate (t203 = 2.63, P = .009) individuals and particularly among respondents with a higher eHealth literacy (t203 = 2.37, P = .019). Even though more than three quarters of the respondents know that PRWs exist, compared to other rating platforms, they are only in the lower midfield. The upper ranks are taken by websites on which restaurant visits (98.9%, 552/558), hotel stays (97.7%, 545/558) or movies (95.5%, 533/558) can be rated. The most popular PRWs in Austria include Docfinder.at (31.3%, 175/558; 77.8%, 434/558) followed by the evaluation tools provided by Google.at (8.24%, 46/558; 70.3%, 392/558) and Herold.at (1.61%, 9/558; 44.8%, 250/558). In Austria, PRWs seem to be characterized by a high degree of interaction (89.2%, 498/558) with a wide variety of different types of interactions. While many respondents use PRWs to retrieve general information (83.2%, 464/558), there are significantly fewer who read physicians' reviews (60.9%, 340/558) and use this portal to select a physician (60.6%, 338/558). Respondents who have already rated a doctor themselves belong to the smallest group accounting for just 14.7% (82/558). Significant effects regarding the interaction with PRWs exist between different genders, ages, education levels, marital statuses, occupations and areas of living. In addition to that, respondents with better feelings towards the internet, greater digital literacy as well as a higher eHealth literacy were also characterized with a higher interaction rate regarding PRWs. According to the high level of awareness of and interaction with PRWs within our study population, PRWs appear to be a successful medium for health-related communication. Especially for female, middle-aged, more highly educated, urban and more technology savvy population segments, PRWs seem to represent an effective tool to support the health-specific decision-making process.

Physician-Rating Websites and Social Media Usage: A Global Survey of Academic Orthopaedic Surgeons: AOA Critical Issues

The relationship between orthopaedic surgeons and the internet is complicated. Social media allows surgeons to educate their patients while marketing to them at the same time. Conversely, patients are able to better communicate with their surgeons while anonymously rating their service and expertise. This study aims to look at the complex relationship between surgeons and social media use.

A New Remote Guided Method for Supervised Web-Based Cognitive Testing to Ensure High-Quality Data: Development and Usability Study

BACKGROUND

The global COVID-19 pandemic has triggered a fundamental reexamination of how human psychological research can be conducted safely and robustly in a new era of digital working and physical distancing. Online web-based testing has risen to the forefront as a promising solution for the rapid mass collection of cognitive data without requiring human contact. However, a long-standing debate exists over the data quality and validity of web-based studies. This study examines the opportunities and challenges afforded by the societal shift toward web-based testing and highlights an urgent need to establish a standard data quality assurance framework for online studies.

OBJECTIVE

This study aims to develop and validate a new supervised online testing methodology, remote guided testing (RGT).

METHODS

A total of 85 healthy young adults were tested on 10 cognitive tasks assessing executive functioning (flexibility, memory, and inhibition) and learning. Tasks were administered either face-to-face in the laboratory (n=41) or online using remote guided testing (n=44) and delivered using identical web-based platforms (Cambridge Neuropsychological Test Automated Battery, Inquisit, and i-ABC). Data quality was assessed using detailed trial-level measures (missed trials, outlying and excluded responses, and response times) and overall task performance measures.

RESULTS

The results indicated that, across all data quality and performance measures, RGT data was statistically-equivalent to in-person data collected in the lab (P>.40 for all comparisons). Moreover, RGT participants out-performed the lab group on measured verbal intelligence (P<.001), which could reflect test environment differences, including possible effects of mask-wearing on communication.

CONCLUSIONS

These data suggest that the RGT methodology could help ameliorate concerns regarding online data quality-particularly for studies involving high-risk or rare cohorts-and offer an alternative for collecting high-quality human cognitive data without requiring in-person physical attendance.

Social media use by shoulder and elbow surgeons increases the number of ratings on physician review websites

BACKGROUND

The use of physician review websites by patients is increasingly frequent. One potential way for shoulder and elbow surgeons to enhance their online reputation and attract patients is via social media, yet its impact is unknown. This study sought to analyze the effect of social media use on the number of online ratings and overall rating of shoulder and elbow surgeons. We secondarily studied variables affecting social media use.

METHODS

The American Shoulder and Elbow Surgeons directory was probed to identify practicing surgeons. Surgeon evaluation data, including ratings, comments, and reviews, were compiled from 3 physician review websites (Google, Healthgrades, and Vitals). Google was queried to assess for a professional Facebook, Twitter, or Instagram account, as well as obtain information on surgeon training, practice location, and other demographic variables.

RESULTS

A total of 646 surgeons met the inclusion criteria (93.8% male and 6.2% female surgeons). Overall, 37% had a professional social media account (Facebook, 23.1%; Twitter, 24.1%; and Instagram, 9.4%). Linear regression analysis showed that Facebook use correlated with an average increase of 48.6 in the number of ratings. No social media platform correlated with physician rating. The surgeons more likely to use social media were those who graduated residency in 2000 or later (40.8% vs. 29.2%), those who practiced in cities with higher populations (mean, 1188.9 vs. 708.4 [per 1000]), and those with more surgeons practicing in the same city (mean, 7.0 vs. 5.0).

CONCLUSION

The majority of shoulder and elbow surgeons do not have a professional social media account. Those using this platform are younger and located in more populous cities with more competition. Having a professional social media profile was not correlated with ratings, but there was a positive association with the number of online ratings, and Facebook had the strongest association. Surgeon ratings are overwhelmingly positive with minimal variability; therefore, a high number of ratings confirms a surgeon's exceptional reputation. The residency graduation year, city population, and number of nearby surgeons affected ratings, although the absolute differences were minimal. For shoulder and elbow surgeons, a professional social media account correlates with an increase in the number of online physician ratings. Recent graduates practicing in competitive locations may feel increased pressure to leverage this in an attempt to build their practices.

Psychiatry in the Digital Age: A Blessing or a Curse?

Social distancing and the shortage of healthcare professionals during the COVID-19 pandemic, the impact of population aging on the healthcare system, as well as the rapid pace of digital innovation are catalyzing the development and implementation of new technologies and digital services in psychiatry. Is this transformation a blessing or a curse for psychiatry? To answer this question, we conducted a literature review covering a broad range of new technologies and eHealth services, including telepsychiatry; computer-, internet-, and app-based cognitive behavioral therapy; virtual reality; digital applied games; a digital medicine system; omics; neuroimaging; machine learning; precision psychiatry; clinical decision support; electronic health records; physician charting; digital language translators; and online mental health resources for patients. We found that eHealth services provide effective, scalable, and cost-efficient options for the treatment of people with limited or no access to mental health care. This review highlights innovative technologies spearheading the way to more effective and safer treatments. We identified artificially intelligent tools that relieve physicians from routine tasks, allowing them to focus on collaborative doctor-patient relationships. The transformation of traditional clinics into digital ones is outlined, and the challenges associated with the successful deployment of digitalization in psychiatry are highlighted.

Perceptual Gaps Between Clinicians and Technologists on Health Information Technology-Related Errors in Hospitals: Observational Study

BACKGROUND

Health information technology (HIT) has been widely adopted in hospital settings, contributing to improved patient safety. However, many types of medical errors attributable to information technology (IT) have negatively impacted patient safety. The continued occurrence of many errors is a reminder that HIT software testing and validation is not adequate in ensuring errorless software functioning within the health care organization.

OBJECTIVE

This pilot study aims to classify technology-related medical errors in a hospital setting using an expanded version of the sociotechnical framework to understand the significant differences in the perceptions of clinical and technology stakeholders regarding the potential causes of these errors. The paper also provides some recommendations to prevent future errors.

METHODS

Medical errors were collected from previous studies identified in leading health databases. From the main list, we selected errors that occurred in hospital settings. Semistructured interviews with 5 medical and 6 IT professionals were conducted to map the events on different dimensions of the expanded sociotechnical framework.

RESULTS

Of the 2319 identified publications, 36 were included in the review. Of the 67 errors collected, 12 occurred in hospital settings. The classification showed the "gulf" that exists between IT and medical professionals in their perspectives on the underlying causes of medical errors. IT experts consider technology as the source of most errors and suggest solutions that are mostly technical. However, clinicians assigned the source of errors within the people, process, and contextual dimensions. For example, for the error "Copied and pasted charting in the wrong window: Before, you could not easily get into someone else's chart accidentally...because you would have to pull the chart and open it," medical experts highlighted contextual issues, including the number of patients a health care provider sees in a short time frame, unfamiliarity with a new electronic medical record system, nurse transitions around the time of error, and confusion due to patients having the same name. They emphasized process controls, including failure modes, as a potential fix. Technology experts, in contrast, discussed the lack of notification, poor user interface, and lack of end-user training as critical factors for this error.

CONCLUSIONS

Knowledge of the dimensions of the sociotechnical framework and their interplay with other dimensions can guide the choice of ways to address medical errors. These findings lead us to conclude that designers need not only a high degree of HIT know-how but also a strong understanding of the medical processes and contextual factors. Although software development teams have historically included clinicians as business analysts or subject matter experts to bridge the gap, development teams will be better served by more immersive exposure to clinical environments, leading to better software design and implementation, and ultimately to enhanced patient safety.

Standards, Processes and Tools Used to Evaluate Health Information Systems Quality: A Systematic Literature Review (Preprint)

Background

Evaluating health information system (HIS) quality is strategically advantageous for improving the quality of patient care. Nevertheless, few systematic studies have reported what methods, such as standards, processes, and tools, were proposed to evaluate HIS quality.

Objective

This study aimed to identify and discuss the existing literature that describes standards, processes, and tools used to evaluate HIS quality.

Methods

We conducted a systematic literature review using review guidelines focused on software and systems. We examined seven electronic databases—Scopus, ACM (Association for Computing Machinery), ScienceDirect, Google Scholar, IEEE Xplore, Web of Science, and PubMed—to search for and select primary studies.

Results

Out of 782 papers, we identified 17 (2.2%) primary studies. We found that most of the primary studies addressed quality evaluation from a management perspective. On the other hand, there was little explicit and pragmatic evidence on the processes and tools that allowed for the evaluation of HIS quality.

Conclusions

To promote quality evaluation of HISs, it is necessary to define mechanisms and methods that operationalize the standards in HISs. Additionally, it is necessary to create metrics that measure the quality of the most critical components and processes of HISs.