Using Machine Learning to Compare Provaccine and Antivaccine Discourse Among the Public on Social Media: Algorithm Development Study

COVID-19 pediatric vaccine Hesitancy: Themes and interactions with verified twitter accounts

BACKGROUND

Though vaccine hesitancy and misinformation has been pervasive online, via platforms such as Twitter, little is known about the characteristics of pediatric-specific vaccine hesitancy and how online users interact with verified user accounts that may hold larger influence. Identifying specific COVID-19 pediatric vaccine hesitancy themes and online user interaction and sentiment may help inform health promotion that addresses vaccine hesitancy more effectively among parents and caregivers of pediatric populations.

METHODS

Keywords were used to query the public streaming twitter application programming interface to collect tweets associated with COVID-19 pediatric vaccines. From this corpus of tweets, we used topic modeling to output 20 topic clusters of tweet content and examined the 10 most retweeted tweets from each cluster to classify for relevance to pediatric COVID-19 vaccine hesitancy topics. Tweets were inductively coded to identify specific themes. Publicly available user metadata were assessed to identify verified accounts and self-reporting of racial or ethnic identity, and parental status. Replies to tweets were coded for user sentiment. A chi-squared test was used to determine the proportion of users agreeing with misinformation tweets RESULTS: 863,007 tweets were collected between October 2020-October 2021. The 230 top tweets reviewed after outputting topic clusters accounted for 236,121 tweets and retweets. 84 unique tweets were identified as related to pediatric COVID-19 vaccine topics by verified users. Twenty three tweets (generating 44,509 retweets) contained misinformation-related themes. Seventy-one percent (n = 742) of user replies agreed with misinformation sentiment of the parent tweet. Main themes identified included vaccine development conspiracy, vaccine is experimental, and vaccine as a control tactic discussions. This study found that users who interacted with misinformation posted by verified accounts were more likely to agree than disagree with misinformation sentiment.

Investigating the Classification of Living Kidney Donation Experiences on Reddit and Understanding the Sensitivity of ChatGPT to Prompt Engineering: Content Analysis

BACKGROUND

Living kidney donation (LKD), where individuals donate one kidney while alive, plays a critical role in increasing the number of kidneys available for those experiencing kidney failure. Previous studies show that many generous people are interested in becoming living donors; however, a huge gap exists between the number of patients on the waiting list and the number of living donors yearly.

OBJECTIVE

To bridge this gap, we aimed to investigate how to identify potential living donors from discussions on public social media forums so that educational interventions could later be directed to them.

METHODS

Using Reddit forums as an example, this study described the classification of Reddit content shared about LKD into three classes: (1) present (presently dealing with LKD personally), (2) past (dealt with LKD personally in the past), and (3) other (LKD general comments). An evaluation was conducted comparing a fine-tuned distilled version of the Bidirectional Encoder Representations from Transformers (BERT) model with inference using GPT-3.5 (ChatGPT). To systematically evaluate ChatGPT's sensitivity to distinguishing between the 3 prompt categories, we used a comprehensive prompt engineering strategy encompassing a full factorial analysis in 48 runs. A novel prompt engineering approach, dialogue until classification consensus, was introduced to simulate a deliberation between 2 domain experts until a consensus on classification was achieved.

RESULTS

BERT and GPT-3.5 exhibited classification accuracies of approximately 75% and 78%, respectively. Recognizing the inherent ambiguity between classes, a post hoc analysis of incorrect predictions revealed sensible reasoning and acceptable errors in the predictive models. Considering these acceptable mismatched predictions, the accuracy improved to 89.3% for BERT and 90.7% for GPT-3.5.

CONCLUSIONS

Large language models, such as GPT-3.5, are highly capable of detecting and categorizing LKD-targeted content on social media forums. They are sensitive to instructions, and the introduced dialogue until classification consensus method exhibited superior performance over stand-alone reasoning, highlighting the merit in advancing prompt engineering methodologies. The models can produce appropriate contextual reasoning, even when final conclusions differ from their human counterparts.

Vaccine rhetoric on social media and COVID-19 vaccine uptake rates: A triangulation using self-reported vaccine acceptance

The primary goal of this study is to examine the association between vaccine rhetoric on Twitter and the public's uptake rates of COVID-19 vaccines in the United States, compared to the extent of an association between self-reported vaccine acceptance and the CDC's uptake rates. We downloaded vaccine-related posts on Twitter in real-time daily for 13 months, from October 2021 to September 2022, collecting over half a billion tweets. A previously validated deep-learning algorithm was then applied to (1) filter out irrelevant tweets and (2) group the remaining relevant tweets into pro-, anti-, and neutral vaccine sentiments. Our results indicate that the tweet counts (combining all three sentiments) were significantly correlated with the uptake rates of all stages of COVID-19 shots (p < 0.01). The self-reported level of vaccine acceptance was not correlated with any of the stages of COVID-19 shots (p > 0.05) but with the daily new infection counts. These results suggest that although social media posts on vaccines may not represent the public's opinions, they are aligned with the public's behaviors of accepting vaccines, which is an essential step for developing interventions to increase the uptake rates. In contrast, self-reported vaccine acceptance represents the public's opinions, but these were not correlated with the behaviors of accepting vaccines. These outcomes provide empirical support for the validity of social media analytics for gauging the public's vaccination behaviors and understanding a nuanced perspective of the public's vaccine sentiment for health emergencies.

A taxonomy of anti-vaccination arguments from a systematic literature review and text modelling

The proliferation of anti-vaccination arguments is a threat to the success of many immunization programmes. Effective rebuttal of contrarian arguments requires an approach that goes beyond addressing flaws in the arguments, by also considering the attitude roots-that is, the underlying psychological attributes driving a person's belief-of opposition to vaccines. Here, through a pre-registered systematic literature review of 152 scientific articles and thematic analysis of anti-vaccination arguments, we developed a hierarchical taxonomy that relates common arguments and themes to 11 attitude roots that explain why an individual might express opposition to vaccination. We further validated our taxonomy on coronavirus disease 2019 anti-vaccination misinformation, through a combination of human coding and machine learning using natural language processing algorithms. Overall, the taxonomy serves as a theoretical framework to link expressed opposition of vaccines to their underlying psychological processes. This enables future work to develop targeted rebuttals and other interventions that address the underlying motives of anti-vaccination arguments.

Childhood Vaccine Attitude and Refusal among Turkish Parents

We aimed to understand and resolve anti-vaccine attitudes by examining the factors associated with vaccine attitudes and exploring potential strategies to improve childhood vaccination rates. Between 2014 and 2021, a total of 628 families refused vaccination in Adiyaman. A total of 300 families accepted visits and were visited. During the visits, the families were administered a questionnaire to determine the reasons for vaccine rejection and their opinions on the matter. While providing general information about the vaccine, parents were encouraged to reconsider their decision, and at the end, parents completed the questionnaire. The questionnaire included sociodemographic questions, reasons for vaccine refusal, and a vaccine attitude scale. Among the participants in the study, 9.3% were convinced about the vaccine. The mean vaccine attitude scale score was calculated as 23.6 ± 2.5 (min = 15-max = 29). Significantly higher rates of persuasion were observed among fathers (17.3%) compared to mothers (7.7%) (p = 0.038). Participants who had received some vaccinations had a higher rate of persuasion (11.6%) compared to those who had not received any vaccinations (2.6%) (p = 0.02). Childhood vaccine refusal is a complex issue that has been the subject of numerous studies. Studies on this subject will increase awareness of vaccines.

Automatic detection of health misinformation: a systematic review

The spread of health misinformation has the potential to cause serious harm to public health, from leading to vaccine hesitancy to adoption of unproven disease treatments. In addition, it could have other effects on society such as an increase in hate speech towards ethnic groups or medical experts. To counteract the sheer amount of misinformation, there is a need to use automatic detection methods. In this paper we conduct a systematic review of the computer science literature exploring text mining techniques and machine learning methods to detect health misinformation. To organize the reviewed papers, we propose a taxonomy, examine publicly available datasets, and conduct a content-based analysis to investigate analogies and differences among Covid-19 datasets and datasets related to other health domains. Finally, we describe open challenges and conclude with future directions.

Media Data and Vaccine Hesitancy: Scoping Review

Background

Media studies are important for vaccine hesitancy research, as they analyze how the media shapes risk perceptions and vaccine uptake. Despite the growth in studies in this field owing to advances in computing and language processing and an expanding social media landscape, no study has consolidated the methodological approaches used to study vaccine hesitancy. Synthesizing this information can better structure and set a precedent for this growing subfield of digital epidemiology.

Objective

This review aimed to identify and illustrate the media platforms and methods used to study vaccine hesitancy and how they build or contribute to the study of the media's influence on vaccine hesitancy and public health.

Methods

This study followed the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines. A search was conducted on PubMed and Scopus for any studies that used media data (social media or traditional media), had an outcome related to vaccine sentiment (opinion, uptake, hesitancy, acceptance, or stance), were written in English, and were published after 2010. Studies were screened by only 1 reviewer and extracted for media platform, analysis method, the theoretical models used, and outcomes.

Results

In total, 125 studies were included, of which 71 (56.8%) used traditional research methods and 54 (43.2%) used computational methods. Of the traditional methods, most used content analysis (43/71, 61%) and sentiment analysis (21/71, 30%) to analyze the texts. The most common platforms were newspapers, print media, and web-based news. The computational methods mostly used sentiment analysis (31/54, 57%), topic modeling (18/54, 33%), and network analysis (17/54, 31%). Fewer studies used projections (2/54, 4%) and feature extraction (1/54, 2%). The most common platforms were Twitter and Facebook. Theoretically, most studies were weak. The following five major categories of studies arose: antivaccination themes centered on the distrust of institutions, civil liberties, misinformation, conspiracy theories, and vaccine-specific concerns; provaccination themes centered on ensuring vaccine safety using scientific literature; framing being important and health professionals and personal stories having the largest impact on shaping vaccine opinion; the coverage of vaccination-related data mostly identifying negative vaccine content and revealing deeply fractured vaccine communities and echo chambers; and the public reacting to and focusing on certain signals-in particular cases, deaths, and scandals-which suggests a more volatile period for the spread of information.

Conclusions

The heterogeneity in the use of media to study vaccines can be better consolidated through theoretical grounding. Areas of suggested research include understanding how trust in institutions is associated with vaccine uptake, how misinformation and information signaling influence vaccine uptake, and the evaluation of government communications on vaccine rollouts and vaccine-related events. The review ends with a statement that media data analyses, though groundbreaking in approach, should supplement-not supplant-current practices in public health research.

Using Deep Learning to Identify Linguistic Features that Facilitate or Inhibit the Propagation of Anti- and Pro-Vaccine Content on Social Media

Anti-vaccine content is rapidly propagated via social media, fostering vaccine hesitancy, while pro-vaccine content has not replicated the opponent's successes. Despite this disparity in the dissemination of anti- and pro-vaccine posts, linguistic features that facilitate or inhibit the propagation of vaccine-related content remain less known. Moreover, most prior machine-learning algorithms classified social-media posts into binary categories (e.g., misinformation or not) and have rarely tackled a higher-order classification task based on divergent perspectives about vaccines (e.g., anti-vaccine, pro-vaccine, and neutral). Our objectives are (1) to identify sets of linguistic features that facilitate and inhibit the propagation of vaccine-related content and (2) to compare whether anti-vaccine, provaccine, and neutral tweets contain either set more frequently than the others. To achieve these goals, we collected a large set of social media posts (over 120 million tweets) between Nov. 15 and Dec. 15, 2021, coinciding with the Omicron variant surge. A two-stage framework was developed using a fine-tuned BERT classifier, demonstrating over 99 and 80 percent accuracy for binary and ternary classification. Finally, the Linguistic Inquiry Word Count text analysis tool was used to count linguistic features in each classified tweet. Our regression results show that anti-vaccine tweets are propagated (i.e., retweeted), while pro-vaccine tweets garner passive endorsements (i.e., favorited). Our results also yielded the two sets of linguistic features as facilitators and inhibitors of the propagation of vaccine-related tweets. Finally, our regression results show that anti-vaccine tweets tend to use the facilitators, while pro-vaccine counterparts employ the inhibitors. These findings and algorithms from this study will aid public health officials' efforts to counteract vaccine misinformation, thereby facilitating the delivery of preventive measures during pandemics and epidemics.

The Role of Influential Actors in Fostering the Polarized COVID-19 Vaccine Discourse on Twitter: Mixed Methods of Machine Learning and Inductive Coding

Background

Since COVID-19 vaccines became broadly available to the adult population, sharp divergences in uptake have emerged along partisan lines. Researchers have indicated a polarized social media presence contributing to the spread of mis- or disinformation as being responsible for these growing partisan gaps in uptake.

Objective

The major aim of this study was to investigate the role of influential actors in the context of the community structures and discourse related to COVID-19 vaccine conversations on Twitter that emerged prior to the vaccine rollout to the general population and discuss implications for vaccine promotion and policy.

Methods

We collected tweets on COVID-19 between July 1, 2020, and July 31, 2020, a time when attitudes toward the vaccines were forming but before the vaccines were widely available to the public. Using network analysis, we identified different naturally emerging Twitter communities based on their internal information sharing. A PageRank algorithm was used to quantitively measure the level of “influentialness” of Twitter accounts and identifying the “influencers,” followed by coding them into different actor categories. Inductive coding was conducted to describe discourses shared in each of the 7 communities.

Results

Twitter vaccine conversations were highly polarized, with different actors occupying separate “clusters.” The antivaccine cluster was the most densely connected group. Among the 100 most influential actors, medical experts were outnumbered both by partisan actors and by activist vaccine skeptics or conspiracy theorists. Scientists and medical actors were largely absent from the conservative network, and antivaccine sentiment was especially salient among actors on the political right. Conversations related to COVID-19 vaccines were highly polarized along partisan lines, with “trust” in vaccines being manipulated to the political advantage of partisan actors.

Conclusions

These findings are informative for designing improved vaccine information communication strategies to be delivered on social media especially by incorporating influential actors. Although polarization and echo chamber effect are not new in political conversations in social media, it was concerning to observe these in health conversations on COVID-19 vaccines during the vaccine development process.

COVID-19 vaccine hesitancy: a social media analysis using deep learning

Hesitant attitudes have been a significant issue since the development of the first vaccines-the WHO sees them as one of the most critical global health threats. The increasing use of social media to spread questionable information about vaccination strongly impacts the population's decision to get vaccinated. Developing text classification methods that can identify hesitant messages on social media could be useful for health campaigns in their efforts to address negative influences from social media platforms and provide reliable information to support their strategies against hesitant-vaccination sentiments. This study aims to evaluate the performance of different machine learning models and deep learning methods in identifying vaccine-hesitant tweets that are being published during the COVID-19 pandemic. Our concluding remarks are that Long Short-Term Memory and Recurrent Neural Network models have outperformed traditional machine learning models on detecting vaccine-hesitant messages in social media, with an accuracy rate of 86% against 83%.