Reliable or not? An automated classification of webpages about early childhood vaccination using supervised machine learning

Detecting health misinformation: A comparative analysis of machine learning and graph convolutional networks in classification tasks

In the digital age, the proliferation of health-related information online has heightened the risk of misinformation, posing substantial threats to public well-being. This research conducts a meticulous comparative analysis of classification models, focusing on detecting health misinformation. The study evaluates the performance of traditional machine learning models and advanced graph convolutional networks (GCN) across critical algorithmic metrics. The results comprehensively understand each algorithm's effectiveness in identifying health misinformation and provide valuable insights for combating the pervasive spread of false health information in the digital landscape. GCN with TF-IDF gives the best result, as shown in the result section. •The research method involves a comparative analysis of classification algorithms to detect health misinformation, exploring traditional machine learning models and graph convolutional networks.•This research used algorithms such as Passive Aggressive Classifier, Random Forest, Decision Tree, Logistic Regression, Light GBM, GCN, GCN with BERT, GCN with TF-IDF, and GCN with Word2Vec were employed. Performance Metrics: Accuracy: for Passive Aggressive Classifier: 85.75 %, Random Forest: 86 %, Decision Tree: 81.30 %, Light BGM: 83.29 %, normal GCN: 84.53 %, GCN with BERT: 85.00 %, GCN with TR-IDF: 93.86 % and GCN with word2Vec: 81.00 %•Algorithmic performance metrics, including accuracy, precision, recall, and F1-score, were systematically evaluated to assess the efficacy of each model in detecting health misinformation, focusing on understanding the strengths and limitations of different approaches. The superior performance of Graph Convolutional Networks (GCNs) with TF-IDF embedding, achieving an accuracy of 93.86.

Automatic detection of health misinformation: a systematic review

The spread of health misinformation has the potential to cause serious harm to public health, from leading to vaccine hesitancy to adoption of unproven disease treatments. In addition, it could have other effects on society such as an increase in hate speech towards ethnic groups or medical experts. To counteract the sheer amount of misinformation, there is a need to use automatic detection methods. In this paper we conduct a systematic review of the computer science literature exploring text mining techniques and machine learning methods to detect health misinformation. To organize the reviewed papers, we propose a taxonomy, examine publicly available datasets, and conduct a content-based analysis to investigate analogies and differences among Covid-19 datasets and datasets related to other health domains. Finally, we describe open challenges and conclude with future directions.

Evaluating online health information quality using machine learning and deep learning: A systematic literature review

Background

Due to the large volume of online health information, while quality remains dubious, understanding the usage of artificial intelligence to evaluate health information and surpass human-level performance is crucial. However, the existing studies still need a comprehensive review highlighting the vital machine, and Deep learning techniques for the automatic health information evaluation process.

Objective

Therefore, this study outlines the most recent developments and the current state of the art regarding evaluating the quality of online health information on web pages and specifies the direction of future research.

Methods

In this article, a systematic literature is conducted according to the PRISMA statement in eight online databases PubMed, Science Direct, Scopus, ACM, Springer Link, Wiley Online Library, Emerald Insight, and Web of Science to identify all empirical studies that use machine and deep learning models for evaluating the online health information quality. Furthermore, the selected techniques are compared based on their characteristics, such as health quality criteria, quality measurement tools, algorithm type, and achieved performance.

Results

The included papers evaluate health information on web pages using over 100 quality criteria. The results show no universal quality dimensions used by health professionals and machine or deep learning practitioners while evaluating health information quality. In addition, the metrics used to assess the model performance are not the same as those used to evaluate human performance.

Conclusions

This systemic review offers a novel perspective in approaching the health information quality in web pages that can be used by machine and deep learning practitioners to tackle the problem more effectively.

Visualizing the Interpretation of a Criteria-Driven System That Automatically Evaluates the Quality of Health News: Exploratory Study of 2 Approaches

BACKGROUND

Machine learning techniques have been shown to be efficient in identifying health misinformation, but the results may not be trusted unless they can be justified in a way that is understandable.

OBJECTIVE

This study aimed to provide a new criteria-based system to assess and justify health news quality. Using a subset of an existing set of criteria, this study compared the feasibility of 2 alternative methods for adding interpretability. Both methods used classification and highlighting to visualize sentence-level evidence.

METHODS

A total of 3 out of 10 well-established criteria were chosen for experimentation, namely whether the health news discussed the costs of the intervention (the cost criterion), explained or quantified the harms of the intervention (the harm criterion), and identified the conflicts of interest (the conflict criterion). The first step of the experiment was to automate the evaluation of the 3 criteria by developing a sentence-level classifier. We tested Logistic Regression, Naive Bayes, Support Vector Machine, and Random Forest algorithms. Next, we compared the 2 visualization approaches. For the first approach, we calculated word feature weights, which explained how classification models distill keywords that contribute to the prediction; then, using the local interpretable model-agnostic explanation framework, we selected keywords associated with the classified criterion at the document level; and finally, the system selected and highlighted sentences with keywords. For the second approach, we extracted sentences that provided evidence to support the evaluation result from 100 health news articles; based on these results, we trained a typology classification model at the sentence level; and then, the system highlighted a positive sentence instance for the result justification. The number of sentences to highlight was determined by a preset threshold empirically determined using the average accuracy.

RESULTS

The automatic evaluation of health news on the cost, harm, and conflict criteria achieved average area under the curve scores of 0.88, 0.76, and 0.73, respectively, after 50 repetitions of 10-fold cross-validation. We found that both approaches could successfully visualize the interpretation of the system but that the performance of the 2 approaches varied by criterion and highlighting the accuracy decreased as the number of highlighted sentences increased. When the threshold accuracy was ≥75%, this resulted in a visualization with a variable length ranging from 1 to 6 sentences.

CONCLUSIONS

We provided 2 approaches to interpret criteria-based health news evaluation models tested on 3 criteria. This method incorporated rule-based and statistical machine learning approaches. The results suggested that one might visually interpret an automatic criterion-based health news quality evaluation successfully using either approach; however, larger differences may arise when multiple quality-related criteria are considered. This study can increase public trust in computerized health information evaluation.

Infodemics and health misinformation: a systematic review of reviews

Objective

To compare and summarize the literature regarding infodemics and health misinformation, and to identify challenges and opportunities for addressing the issues of infodemics.

Methods

We searched MEDLINE®, Embase®, Cochrane Library of Systematic Reviews, Scopus and Epistemonikos on 6 May 2022 for systematic reviews analysing infodemics, misinformation, disinformation and fake news related to health. We grouped studies based on similarity and retrieved evidence on challenges and opportunities. We used the AMSTAR 2 approach to assess the reviews' methodological quality. To evaluate the quality of the evidence, we used the Grading of Recommendations Assessment, Development and Evaluation guidelines.

Findings

Our search identified 31 systematic reviews, of which 17 were published. The proportion of health-related misinformation on social media ranged from 0.2% to 28.8%. Twitter, Facebook, YouTube and Instagram are critical in disseminating the rapid and far-reaching information. The most negative consequences of health misinformation are the increase of misleading or incorrect interpretations of available evidence, impact on mental health, misallocation of health resources and an increase in vaccination hesitancy. The increase of unreliable health information delays care provision and increases the occurrence of hateful and divisive rhetoric. Social media could also be a useful tool to combat misinformation during crises. Included reviews highlight the poor quality of published studies during health crises.

Conclusion

Available evidence suggests that infodemics during health emergencies have an adverse effect on society. Multisectoral actions to counteract infodemics and health misinformation are needed, including developing legal policies, creating and promoting awareness campaigns, improving health-related content in mass media and increasing people's digital and health literacy.

Vec4Cred: a model for health misinformation detection in web pages

Research aimed at finding solutions to the problem of the diffusion of distinct forms of non-genuine information online across multiple domains has attracted growing interest in recent years, from opinion spam to fake news detection. Currently, partly due to the COVID-19 virus outbreak and the subsequent proliferation of unfounded claims and highly biased content, attention has focused on developing solutions that can automatically assess the genuineness of health information. Most of these approaches, applied both to Web pages and social media content, rely primarily on the use of handcrafted features in conjunction with Machine Learning. In this article, instead, we propose a health misinformation detection model that exploits as features the embedded representations of some structural and content characteristics of Web pages, which are obtained using an embedding model pre-trained on medical data. Such features are employed within a deep learning classification model, which categorizes genuine health information versus health misinformation. The purpose of this article is therefore to evaluate the effectiveness of the proposed model, namely Vec4Cred, with respect to the problem considered. This model represents an evolution of a previous one, with respect to which new features and architectural choices have been considered and illustrated in this work.

Facts Tell, Stories Sell? Assessing the Availability Heuristic and Resistance as Cognitive Mechanisms Underlying the Persuasive Effects of Vaccination Narratives

Online vaccine-critical sentiments are often expressed in appealing personal narratives, whereas vaccine-supporting information is often presented in a non-narrative, expository mode describing scientific facts. In two experiments, we empirically test whether and how these different formats impact the way in which readers process and retrieve information about childhood vaccination, and how this may impact their perceptions regarding vaccination. We assess two psychological mechanisms that are hypothesized to underlie the persuasive nature of vaccination narratives: the availability heuristic (experiment 1, N = 418) and cognitive resistance (experiment 2, N = 403). The results of experiment 1 showed no empirical evidence for the availability heuristic, but exploratory analyses did indicate that an anti-vaccination narrative (vs. expository) might reduce cognitive resistance, decrease vaccination attitudes and reduce attitude certainty in a generally pro-vaccination sample, especially for those who were more vaccine hesitant. Preregistered experiment 2 formally tested this and showed that not narrative format, but prior vaccine hesitancy predicts cognitive resistance and post-reading attitudes. Hesitant participants showed less resistance toward an anti-vaccine text than vaccine-supporting participants, as well as less positive post-reading attitudes and attitude certainty. These findings demonstrate belief consistency effects rather than narrative persuasion, which has implications for scientific research as well as public health policy.