Adding Dimensions to the Analysis of the Quality of Health Information of Websites Returned by Google: Cluster Analysis Identifies Patterns of Websites According to their Classification and the Type of Intervention Described

Evaluating online health information quality using machine learning and deep learning: A systematic literature review

Background

Due to the large volume of online health information, while quality remains dubious, understanding the usage of artificial intelligence to evaluate health information and surpass human-level performance is crucial. However, the existing studies still need a comprehensive review highlighting the vital machine, and Deep learning techniques for the automatic health information evaluation process.

Objective

Therefore, this study outlines the most recent developments and the current state of the art regarding evaluating the quality of online health information on web pages and specifies the direction of future research.

Methods

In this article, a systematic literature is conducted according to the PRISMA statement in eight online databases PubMed, Science Direct, Scopus, ACM, Springer Link, Wiley Online Library, Emerald Insight, and Web of Science to identify all empirical studies that use machine and deep learning models for evaluating the online health information quality. Furthermore, the selected techniques are compared based on their characteristics, such as health quality criteria, quality measurement tools, algorithm type, and achieved performance.

Results

The included papers evaluate health information on web pages using over 100 quality criteria. The results show no universal quality dimensions used by health professionals and machine or deep learning practitioners while evaluating health information quality. In addition, the metrics used to assess the model performance are not the same as those used to evaluate human performance.

Conclusions

This systemic review offers a novel perspective in approaching the health information quality in web pages that can be used by machine and deep learning practitioners to tackle the problem more effectively.

Clinical relevance of the cagA and vacA s1m1 status and antibiotic resistance in Helicobacter pylori: a systematic review and meta-analysis

BACKGROUND

The role of Helicobacter pylori (H. pylori) virulence factors of such as vacA s1m1 and cagA in designating clinical outcomes and eradication rate has been deeply challenged in the last decade. The goal of this analysis was to identify the potential relevance between cagA and vacA genotypes with reported antibiotic resistance observed in clinical H. pylori isolates.

METHODS

This literature search was conducted in databases such as Clarivate analytics, PubMed, Scopus, EMBASE, DOAJ, and Google Scholar by April 2022, regardless of language restrictions and publication date. Quality of the included studies was assessed by the Newcastle-Ottawa scale. Statistical analysis of retrieved studies was fulfilled using Comprehensive Meta-Analysis software version 2.2. Following quality appraisal of eligible studies, potential association between the status of cagA and vacA genes with resistance to clarithromycin, metronidazole, amoxicillin, tetracycline, and levofloxacin was measured using odds ratio with 95% confidence interval. We also used sensitivity analyses and meta-regression to eliminate the source of heterogeneity from the overall estimates. Publication bias was assessed using funnel plot, Egger's test, Begg's test with the trim and fill procedure to assess the presence and magnitude of publication bias in the included studies.

RESULTS

Our findings suggested that a significant relationship between cagA status ‎and increase resistance ‎to metronidazole (OR: 2.69; 95% CI: 1.24-5.83‎‏‎). In subgroup analysis, we ‎found that in the Western ‎population, infection with cagA-positive strains could be led to increase in ‎the resistance to ‎metronidazole (OR: 1.59; 95% CI: ‎0.78-3.21‎‏‎), ‎amoxicillin (OR: ‎19.68‎; 95% CI: 2.74-‎‎141.18), ‎and ‎levofloxacin (OR: ‎11.33; 95% CI: ‎1.39-‎‎91.85). After implementation of trim and fill method, the adjusted OR was not significantly differed from original estimates which in turn represented our subgroup analysis was statistically robust. On the other hand, vacA ‎genotypes usually ‎reduce the antibiotic resistance of this bacterium, so that vacA s1m1 significantly reduces the ‎resistance to ‎metronidazole (OR: 0.41; 95% CI: 0.20-0.86‎‏‎). Surprisingly, resistance of vacA s2m2 strains to antibiotics was low, the reason may be due ‎to the non-inflammatory properties of strains containing vacA s2m2. The meta-regression and sensitivity analyses successfully reduced the effect of heterogeneity from the overall estimates. In addition, although the pooled OR is reduced after trim and fill adjustment but results do not change the conclusion regarding vacA genotypes and antibiotic resistance.

CONCLUSIONS

According to our findings, it was clearly demonstrated that cagA-positive strains are resistance to metronidazole, especially in Western countries. In Western countries, vacA s1m1 increases resistance to amoxicillin and levofloxacin. Based on the present findings, the vacA s1m1 genotype significantly increases resistance to metronidazole, while the vacA s1m2 decreases resistance to clarithromycin and metronidazole. Resistance to antibiotics in less virulent (vacA s2m2) strains is statistically significant lower than others.

Evaluating the reliability and readability of online information on osteoporosis

OBJECTIVE

Internet usage for obtaining health-related information is widely popular among patients. However, there are still concerns about the reliability and comprehensibility of online information. The purpose of this study is to investigate the reliability and readability of osteoporosis-related websites.

METHODS

On April 2, 2020, we searched the term "osteoporosis" on Google (https://www.google.com). We evaluated the first 200 uniform resource locators (URLs) in the query results regarding typology, the Journal of the American Medical Association (JAMA) scores, Health on the Net Foundation Code of conduct (HONcode) certification, Flesch-Kincaid Grade (FKG), and Simple Measure of Gobbledygook (SMOG) scores. The JAMA scoring system and HONcode stamp were used for assessing the reliability, whereas FKG and SMOG scores were used to assess the readability of online information.

RESULTS

Of the 151 analyzed websites, 57 (37.7%) were classified as highly reliable, and 19 (12.6%) were assigned with HONcode certification. The average FKG scores (8.81 ± 2.21) and SMOG scores (7.63 ± 1.81) were below the recommended grade, which is considered as easily readable. High reliable information was found to have higher readability scores, thereby representing the difficulty of readability. We observed a weak correlation between the increased reliability of information and decreased readability.

CONCLUSION

Osteoporosis-related content on the internet generally has low reliability. High-reliable information is available online in scientific published materials, health portals, and news. Although the readability of the overall material is acceptable, the high-reliable websites still require high literacy and comprehension skills.

Evaluation of information quality on the internet for periodontal disease patients

OBJECTIVE

To evaluate the quality of accessible information on periodontal diseases on the Internet using different scales.

MATERIALS AND METHODS

A search was performed using the Google search engine with questions about periodontal disease symptoms. The first 30 web sites obtained after searching for each question were evaluated. Duplicate web sites, advertisements, discussion groups, links to research articles, videos, and images were excluded. A total of 90 web sites were included and evaluated with Health on the Net Code of Conduct Certification (HONCode) presence, Journal of American Medical Association (JAMA) benchmarks, and the quality criteria for consumer health information (DISCERN) toolkit.

RESULTS

Only 27.8% of the analyzed web sites contain HONCode certificates. No webpages fulfilled all JAMA criteria, whereas 32.2% of the web sites did not provide any of them. Majority of the web sites' (44.4%) overall rating score was 2 with the DISCERN instrument. In the DISCERN Section Scores comparison between various types of web sites, information web site scores were higher than Dental Health Center Web sites in Section 1 and Section 3 scores (p = .000 and p = .001, respectively).

CONCLUSION

Overall quality of periodontal information based on patients' questions on the Internet has serious shortcomings especially in terms of attribution and the quality of information on treatment choices.

Quality and readability of online information on ankylosing spondylitis

INTRODUCTION/OBJECTIVE

Obtaining online health-related information is becoming increasingly popular among patients. The attainment of information through websites is easy and practical, but there is no mechanism to check the accuracy and quality of this information. This leads to concerns about information from websites. Therefore, we aimed to evaluate the quality and readability of ankylosing spondylitis-related websites in this study.

METHODS

This is a descriptive study. Websites were searched on a popular search engine with the search term ankylosing spondylitis on March 2, 2019. We recorded the URLs of the first 200 websites listed in the query results. Typologies, quality, and readability were evaluated on these websites. Websites were divided into eight categories (commercial, government, health portal, news, non-profit, professional, scientific journal, and others) according to typology. The JAMA scoring system and the presence of HONcode certification were used to assess the quality. The Flesch-Kincaid grade and the Simple Measure of Gobbledygook were used to evaluate the readability.

RESULTS

Of the websites analyzed, 46% were in the high-quality group. We found that scientific journals and news were of higher quality, and commercial and other websites were of poorer quality. The average readability grades of the websites were 8.59 ± 2.42 and 7.33 ± 1.54, which were slightly worse than the recommended value. Additionally, the readability grades were significantly higher on high-quality websites (p < 0.05).

CONCLUSION

The quality of information on websites is variable. High-quality information about ankylosing spondylitis is available online, particularly from scientific journals and news. The poor readability of websites that provide high-quality information is a problem for patients with low health literacy. Editors should take into account readability while aiming to present high-quality information on websites. Key Points • Websites have become an important source of health-related information in parallel with the increase in internet use. • Less than half of the ankylosing spondylitis-related websites (46%) were of high quality according to JAMA scores. • The average readability grades of the ankylosing spondylitis-related websites were slightly worse than the recommended values. • High-quality websites had higher readability grades. Therefore, high-quality websites may not be understood by patients with low literacy levels. • No significant difference was found between the websites on the first page (n = 10) and remaining websites (n = 102) in terms of quality and readability.

What Is Health Information Quality? Ethical Dimension and Perception by Users

Introduction: The popularity of seeking health information online makes information quality (IQ) a public health issue. The present study aims at building a theoretical framework of health information quality (HIQ) that can be applied to websites and defines which IQ criteria are important for a website to be trustworthy and meet users' expectations.

Methods: We have identified a list of HIQ criteria from existing tools and assessment criteria and elaborated them into a questionnaire that was promoted via social media and mainly the University. Responses (329) were used to rank the different criteria for their importance in trusting a website and to identify patterns of criteria using hierarchical cluster analysis.

Results: HIQ criteria were organized in five dimensions based on previous theoretical frameworks as well as on how they cluster together in the questionnaire response. We could identify a top-ranking dimension (scientific completeness) that describes what the user is expecting to know from the websites (in particular: description of symptoms, treatments, side effects). Cluster analysis also identified a number of criteria borrowed from existing tools for assessing HIQ that could be subsumed to a broad “ethical” dimension (such as conflict of interests, privacy, advertising policies) that were, in general, ranked of low importance by the participants. Subgroup analysis revealed significant differences in the importance assigned to the various criteria based on gender, language and whether or not of biomedical educational background.

Conclusions: We identified criteria of HIQ and organized them in dimensions. We observed that ethical criteria, while regarded highly in the academic and medical environment, are not considered highly by the public.

Fake News or Weak Science? Visibility and Characterization of Antivaccine Webpages Returned by Google in Different Languages and Countries

The 1998 Lancet paper by Wakefield et al., despite subsequent retraction and evidence indicating no causal link between vaccinations and autism, triggered significant parental concern. The aim of this study was to analyze the online information available on this topic. Using localized versions of Google, we searched “autism vaccine” in English, French, Italian, Portuguese, Mandarin, and Arabic and analyzed 200 websites for each search engine result page (SERP). A common feature was the newsworthiness of the topic, with news outlets representing 25–50% of the SERP, followed by unaffiliated websites (blogs, social media) that represented 27–41% and included most of the vaccine-negative websites. Between 12 and 24% of websites had a negative stance on vaccines, while most websites were pro-vaccine (43–70%). However, their ranking by Google varied. While in Google.com, the first vaccine-negative website was the 43rd in the SERP, there was one vaccine-negative webpage in the top 10 websites in both the British and Australian localized versions and in French and two in Italian, Portuguese, and Mandarin, suggesting that the information quality algorithm used by Google may work better in English. Many webpages mentioned celebrities in the context of the link between vaccines and autism, with Donald Trump most frequently. Few websites (1–5%) promoted complementary and alternative medicine (CAM) but 50–100% of these were also vaccine-negative suggesting that CAM users are more exposed to vaccine-negative information. This analysis highlights the need for monitoring the web for information impacting on vaccine uptake.

Online Information on Antioxidants: Information Quality Indicators, Commercial Interests, and Ranking by Google

The idea that antioxidant supplements can prevent or cure many diseases is extremely popular. To study the public understanding of antioxidants on the Web, we searched the term “antioxidants” in http://Google.com and analyzed 200 websites in terms of typology (news, commercial, professional, health portal, no-profit or government organization, scientific journals), disease or biological process mentioned (aging, immunity, neurological disease, diabetes, arthritis, etc.), and stance toward antioxidants, whether neutral, positive, or negative. Commercial and news websites were prevalent (over half of the total) but not in the top 10 returned by Google, where the most frequent were health portals, government, and professional websites. Among the diseases mentioned, cancer was the first, followed by vascular and eye diseases. A negative stance toward supplements was prevalent in the whole search, and this was even more evident for cancer. Information on aging or immunity had the largest proportion of pro-supplement and commercial websites. This study shows that some diseases are highly associated with antioxidants on the Internet and that information on antioxidants in aging and immunity is more likely to describe the positive effects of antioxidant supplements.

Bad News: Analysis of the Quality of Information on Influenza Prevention Returned by Google in English and Italian

Information available to the public influences the approach of the population toward vaccination against influenza compared with other preventative approaches. In this study, we have analyzed the first 200 websites returned by searching Google on two topics (prevention of influenza and influenza vaccine), in English and Italian. For all the four searches above, websites were classified according to their typology (government, commercial, professional, portals, etc.) and for their trustworthiness as defined by the Journal of the American Medical Association (JAMA) score, which assesses whether they provide some basic elements of information quality (IQ): authorship, currency, disclosure, and references. The type of information described was also assessed to add another dimension of IQ. Websites on influenza prevention were classified according to the type of preventative approach mentioned (vaccine, lifestyle, hygiene, complementary medicine, etc.), whether the approaches were in agreement with evidence-based medicine (EBM) or not. Websites on influenza vaccination were classified as pro- or anti-vaccine, or neutral. The great majority of websites described EBM approaches to influenza prevention and had a pro-vaccine orientation. Government websites mainly pointed at EBM preventative approaches and had a pro-vaccine orientation, while there was a higher proportion of commercial websites among those which promote non-EBM approaches. Although the JAMA score was lower in commercial websites, it did not correlate with the preventative approaches suggested or the orientation toward vaccines. For each of the four search engine result pages (SERP), only one website displayed the health-of-the-net (HON) seal. In the SERP on vaccines, journalistic websites were the most abundant category and ranked higher than average in both languages. Analysis using natural language processing showed that journalistic websites were mostly reporting news about two specific topics (different in the two languages). While the ranking by Google favors EBM approaches and, in English, does not promote commercial websites, in both languages it gives a great advantage to news. Thus, the type of news published during the influenza season probably has a key importance in orienting the public opinion due to its high visibility. This raises important questions on the relationships between health IQ, trustworthiness, and newsworthiness.