Applying Machine Learning to Identify Anti-Vaccination Tweets during the COVID-19 Pandemic

Mixed methods examination of risk perception on vaccination intentions: The perspective of doctor-patient communication

AIM

From the perspective of doctor-patient communication, this research used multiple methods combined natural language processing (NLP), a cross-sectional survey and an online experiment to investigated how risk perception influenced people's vaccination intention.

METHODS

In Study 1, we used Python to crawl 335,045 comments about COVID-19 vaccine published in a social media platform Sina Weibo (equivalent of Twitter in China) from 31 December 2020 to 31 December 2021. Text analysis and sentiment analysis was used to examine how vaccination intention, as measured by linguistic features from the LIWC dictionary, changed with individuals' perceptions of pandemic risk. In Study 2, we adopted a cross-sectional questionnaire survey to further test the relation of risk perception, vaccination intention, trust in physicians, and perceived medical recommendations in a Chinese sample (n = 386). In Study 3, we conducted an online experiment where we recruited 127 participants with high trust in physicians and 127 participants with low trust, and subsequently randomly allocated them into one of three conditions: control, rational recommendation, or perceptual recommendation.

RESULTS

Text and sentiment analysis revealed that the use of negative words towards COVID-19 vaccine had a significant decrease at high (vs. low) risk perception level time (Study 1). Trust in physicians mediated the effect of risk perception on vaccination intention and this effect was reinforced for participants with low (vs. high) level of perceived medical recommendation (Study 2), especially for the rational (vs. perceptual) recommendation condition (Study 3).

CONCLUSION

Risk perception increased vaccination intention through the mediating effect of trust in physicians and the moderating effect of perceived medical recommendations. Rational (vs. perceptual) recommendation is more effective in increasing intention to get vaccinated in people with low trust in physicians.

Pre-trained language models in medicine: A survey

With the rapid progress in Natural Language Processing (NLP), Pre-trained Language Models (PLM) such as BERT, BioBERT, and ChatGPT have shown great potential in various medical NLP tasks. This paper surveys the cutting-edge achievements in applying PLMs to various medical NLP tasks. Specifically, we first brief PLMS and outline the research of PLMs in medicine. Next, we categorise and discuss the types of tasks in medical NLP, covering text summarisation, question-answering, machine translation, sentiment analysis, named entity recognition, information extraction, medical education, relation extraction, and text mining. For each type of task, we first provide an overview of the basic concepts, the main methodologies, the advantages of applying PLMs, the basic steps of applying PLMs application, the datasets for training and testing, and the metrics for task evaluation. Subsequently, a summary of recent important research findings is presented, analysing their motivations, strengths vs weaknesses, similarities vs differences, and discussing potential limitations. Also, we assess the quality and influence of the research reviewed in this paper by comparing the citation count of the papers reviewed and the reputation and impact of the conferences and journals where they are published. Through these indicators, we further identify the most concerned research topics currently. Finally, we look forward to future research directions, including enhancing models' reliability, explainability, and fairness, to promote the application of PLMs in clinical practice. In addition, this survey also collect some download links of some model codes and the relevant datasets, which are valuable references for researchers applying NLP techniques in medicine and medical professionals seeking to enhance their expertise and healthcare service through AI technology.

Artificial intelligence in the healthcare sector: comparison of deep learning networks using chest X-ray images

Purpose

Artificial intelligence has led to significant developments in the healthcare sector, as in other sectors and fields. In light of its significance, the present study delves into exploring deep learning, a branch of artificial intelligence.

Methods

In the study, deep learning networks ResNet101, AlexNet, GoogLeNet, and Xception were considered, and it was aimed to determine the success of these networks in disease diagnosis. For this purpose, a dataset of 1,680 chest X-ray images was utilized, consisting of cases of COVID-19, viral pneumonia, and individuals without these diseases. These images were obtained by employing a rotation method to generate replicated data, wherein a split of 70 and 30% was adopted for training and validation, respectively.

Results

The analysis findings revealed that the deep learning networks were successful in classifying COVID-19, Viral Pneumonia, and Normal (disease-free) images. Moreover, an examination of the success levels revealed that the ResNet101 deep learning network was more successful than the others with a 96.32% success rate.

Conclusion

In the study, it was seen that deep learning can be used in disease diagnosis and can help experts in the relevant field, ultimately contributing to healthcare organizations and the practices of country managers.

Identification, analysis and prediction of valid and false information related to vaccines from Romanian tweets

Introduction

The online misinformation might undermine the vaccination efforts. Therefore, given the fact that no study specifically analyzed online vaccine related content written in Romanian, the main objective of the study was to detect and evaluate tweets related to vaccines and written in Romanian language.

Methods

1,400 Romanian vaccine related tweets were manually classified in true, neutral and fake information and analyzed based on wordcloud representations, a correlation analysis between the three classes and specific tweet characteristics and the validation of several predictive machine learning algorithms.

Results and discussion

The tweets annotated as misinformation showed specific word patterns and were liked and reshared more often as compared to the true and neutral ones. The validation of the machine learning algorithms yielded enhanced results in terms of Area Under the Receiver Operating Characteristic Curve Score (0.744-0.843) when evaluating the Support Vector Classifier. The predictive model estimates in a well calibrated manner the probability that a specific Twitter post is true, neutral or fake. The current study offers important insights regarding vaccine related online content written in an Eastern European language. Future studies must aim at building an online platform for rapid identification of vaccine misinformation and raising awareness for the general population.

COVID-19 Vaccine Hesitancy: A Global Public Health and Risk Modelling Framework Using an Environmental Deep Neural Network, Sentiment Classification with Text Mining and Emotional Reactions from COVID-19 Vaccination Tweets

Popular social media platforms, such as Twitter, have become an excellent source of information with their swift information dissemination. Individuals with different backgrounds convey their opinions through social media platforms. Consequently, these platforms have become a profound instrument for collecting enormous datasets. We believe that compiling, organizing, exploring, and analyzing data from social media platforms, such as Twitter, can offer various perspectives to public health organizations and decision makers in identifying factors that contribute to vaccine hesitancy. In this study, public tweets were downloaded daily from Tweeter using the Tweeter API. Before performing computation, the tweets were preprocessed and labeled. Vocabulary normalization was based on stemming and lemmatization. The NRCLexicon technique was deployed to convert the tweets into ten classes: positive sentiment, negative sentiment, and eight basic emotions (joy, trust, fear, surprise, anticipation, anger, disgust, and sadness). t-test was used to check the statistical significance of the relationships among the basic emotions. Our analysis shows that the p-values of joy-sadness, trust-disgust, fear-anger, surprise-anticipation, and negative-positive relations are close to zero. Finally, neural network architectures, including 1DCNN, LSTM, Multiple-Layer Perceptron, and BERT, were trained and tested in a COVID-19 multi-classification of sentiments and emotions (positive, negative, joy, sadness, trust, disgust, fear, anger, surprise, and anticipation). Our experiment attained an accuracy of 88.6% for 1DCNN at 1744 s, 89.93% accuracy for LSTM at 27,597 s, while MLP achieved an accuracy of 84.78% at 203 s. The study results show that the BERT model performed the best, with an accuracy of 96.71% at 8429 s.

Anti-vaccination attitude trends during the COVID-19 pandemic: A machine learning-based analysis of tweets

Objective

Vaccine hesitancy has been ranked by the World Health Organization among the top 10 threats to global health. With a surge in misinformation and conspiracy theories against vaccination observed during the COVID-19 pandemic, attitudes toward vaccination may be worsening. This study investigates trends in anti-vaccination attitudes during the COVID-19 pandemic and within the United States, Canada, the United Kingdom, and Australia.

Methods

Vaccine-related English tweets published between 1 January 2020 and 27 June 2021 were used. A deep learning model using a dynamic word embedding method, Bidirectional Encoder Representations from Transformers (BERTs), was developed to identify anti-vaccination tweets. The classifier achieved a micro F1 score of 0.92. Time series plots and country maps were used to examine vaccination attitudes globally and within countries.

Results

Among 9,352,509 tweets, 232,975 (2.49%) were identified as anti-vaccination tweets. The overall number of vaccine-related tweets increased sharply after the implementation of the first vaccination round since November 2020 (daily average of 6967 before vs. 31,757 tweets after 9/11/2020). The number of anti-vaccination tweets increased after conspiracy theories spread on social media. Percentages of anti-vaccination tweets were 3.45%, 2.74%, 2.46%, and 1.86% for the United States, the United Kingdom, Australia, and Canada, respectively.

Conclusions

Strategies and information campaigns targeting vaccination misinformation may need to be specifically designed for regions with the highest anti-vaccination Twitter activity and when new vaccination campaigns are initiated.

Cross-platform spread: vaccine-related content, sources, and conspiracy theories in YouTube videos shared in early Twitter COVID-19 conversations

High uptake of vaccinations is essential in fighting infectious diseases, including severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) that causes the ongoing coronavirus disease 2019 (COVID-19) pandemic. Social media play a crucial role in propagating misinformation about vaccination, including through conspiracy theories and can negatively impact trust in vaccination. Users typically engage with multiple social media platforms; however, little is known about the role and content of cross-platform use in spreading vaccination-related information. This study examined the content and dynamics of YouTube videos shared in vaccine-related tweets posted to COVID-19 conversations before the COVID-19 vaccine rollout. We screened approximately 144 million tweets posted to COVID-19 conversations and identified 930,539 unique tweets in English that discussed vaccinations posted between 1 February and 23 June 2020. We then identified links to 2,097 unique YouTube videos that were tweeted. Analysis of the video transcripts using Latent Dirichlet Allocation topic modeling and independent coders indicate the dominance of conspiracy theories. Following the World Health Organization's declaration of the COVID-19 outbreak as a public health emergency of international concern, anti-vaccination frames rapidly transitioned from claiming that vaccines cause autism to pandemic conspiracy theories, often featuring Bill Gates. Content analysis of the 20 most tweeted videos revealed that the majority (n = 15) opposed vaccination and included conspiracy theories. Their spread on Twitter was consistent with spamming and coordinated efforts. These findings show the role of cross-platform sharing of YouTube videos over Twitter as a strategy to propagate primarily anti-vaccination messages. Future policies and interventions should consider how to counteract misinformation spread via such cross-platform activities.

Machine learning for optimizing daily COVID-19 vaccine dissemination to combat the pandemic

Introduction

Vaccines are the most important instrument for bringing the pandemic to a close and saving lives and helping to reduce the risks of infection. It is important that everyone has equal access to immunizations that are both safe and effective. There is no one who is safe until everyone gets vaccinated. COVID-19 vaccinations are a game-changer in the fight against diseases. In addition to examining attitudes toward these vaccines in Africa, Asia, Oceania, Europe, North America, and South America, the purpose of this paper is to predict the acceptability of COVID-19 vaccines and study their predictors.

Materials and methods

Kaggle datasets are used to estimate the prediction outcomes of the daily COVID-19 vaccination to prevent a pandemic. The Kaggle data sets are classified into training and testing datasets. The training dataset is comprised of COVID-19 daily data from the 13th of December 2020 to the 13th of June 2021, while the testing dataset is comprised of COVID-19 daily data from the 14th of June 2021 to the 14th of October 2021. For the prediction of daily COVID-19 vaccination, four well-known machine learning algorithms were described and used in this study: CUBIST, Gaussian Process (GAUSS), Elastic Net (ENET), Spikes, and Slab (SPIKES).

Results

Among the models considered in this paper, CUBIST has the best prediction accuracy in terms of Mean Absolute Scaled Error (MASE) of 9.7368 for Asia, 2.8901 for America, 13.2169 for Oceania, and 3.9510 for South America respectively.

Conclusion

This research shows that machine learning can be of great benefit for optimizing daily immunization of citizens across the globe. And if used properly, it can help decision makers and health administrators to comprehend immunization rates and create strategies to enhance them.

A review on Natural Language Processing Models for COVID-19 research

This survey paper reviews Natural Language Processing Models and their use in COVID-19 research in two main areas. Firstly, a range of transformer-based biomedical pretrained language models are evaluated using the BLURB benchmark. Secondly, models used in sentiment analysis surrounding COVID-19 vaccination are evaluated. We filtered literature curated from various repositories such as PubMed and Scopus and reviewed 27 papers. When evaluated using the BLURB benchmark, the novel T-BPLM BioLinkBERT gives groundbreaking results by incorporating document link knowledge and hyperlinking into its pretraining. Sentiment analysis of COVID-19 vaccination through various Twitter API tools has shown the public’s sentiment towards vaccination to be mostly positive. Finally, we outline some limitations and potential solutions to drive the research community to improve the models used for NLP tasks.

Discussions About COVID-19 Vaccination on Twitter in Turkey: Sentiment Analysis

OBJECTIVES

The present study aims to examine coronavirus disease 2019 (COVID-19) vaccination discussions on Twitter in Turkey and conduct sentiment analysis.

METHODS

The current study performed sentiment analysis of Twitter data with the artificial intelligence (AI) Natural Language Processing (NLP) method. The tweets were retrieved retrospectively from March 10, 2020, when the first COVID-19 case was seen in Turkey, to April 18, 2022. A total of 10,308 tweets accessed. The data were filtered before analysis due to excessive noise. First, the text is tokenized. Many steps were applied in normalizing texts. Tweets about the COVID-19 vaccines were classified according to basic emotion categories using sentiment analysis. The resulting dataset was used for training and testing ML (ML) classifiers.

RESULTS

It was determined that 7.50% of the tweeters had positive, 0.59% negative, and 91.91% neutral opinions about the COVID-19 vaccination. When the accuracy values of the ML algorithms used in this study were examined, it was seen that the XGBoost (XGB) algorithm had higher scores.

CONCLUSIONS

Three of 4 tweets consist of negative and neutral emotions. The responsibility of professional chambers and the public is essential in transforming these neutral and negative feelings into positive ones.

COVID-19 vaccine hesitancy: a social media analysis using deep learning

Hesitant attitudes have been a significant issue since the development of the first vaccines-the WHO sees them as one of the most critical global health threats. The increasing use of social media to spread questionable information about vaccination strongly impacts the population's decision to get vaccinated. Developing text classification methods that can identify hesitant messages on social media could be useful for health campaigns in their efforts to address negative influences from social media platforms and provide reliable information to support their strategies against hesitant-vaccination sentiments. This study aims to evaluate the performance of different machine learning models and deep learning methods in identifying vaccine-hesitant tweets that are being published during the COVID-19 pandemic. Our concluding remarks are that Long Short-Term Memory and Recurrent Neural Network models have outperformed traditional machine learning models on detecting vaccine-hesitant messages in social media, with an accuracy rate of 86% against 83%.

Spread of COVID-19 Vaccine Misinformation in the Ninth Inning: Retrospective Observational Infodemic Study

Background

Shortly after Pfizer and Moderna received emergency use authorizations from the Food and Drug Administration, there were increased reports of COVID-19 vaccine-related deaths in the Vaccine Adverse Event Reporting System (VAERS). In January 2021, Major League Baseball legend and Hall of Famer, Hank Aaron, passed away at the age of 86 years from natural causes, just 2 weeks after he received the COVID-19 vaccine. Antivaccination groups attempted to link his death to the Moderna vaccine, similar to other attempts misrepresenting data from the VAERS to spread COVID-19 misinformation.

Objective

This study assessed the spread of misinformation linked to erroneous claims about Hank Aaron’s death on Twitter and then characterized different vaccine misinformation and hesitancy themes generated from users who interacted with this misinformation discourse.

Methods

An initial sample of tweets from January 31, 2021, to February 6, 2021, was queried from the Twitter Search Application Programming Interface using the keywords “Hank Aaron” and “vaccine.” The sample was manually annotated for misinformation, reporting or news media, and public reaction. Nonmedia user accounts were also classified if they were verified by Twitter. A second sample of tweets, representing direct comments or retweets to misinformation-labeled content, was also collected. User sentiment toward misinformation, positive (agree) or negative (disagree), was recorded. The Strategic Advisory Group of Experts Vaccine Hesitancy Matrix from the World Health Organization was used to code the second sample of tweets for factors influencing vaccine confidence.

Results

A total of 436 tweets were initially sampled from the Twitter Search Application Programming Interface. Misinformation was the most prominent content type (n=244, 56%) detected, followed by public reaction (n=122, 28%) and media reporting (n=69, 16%). No misinformation-related content reviewed was labeled as misleading by Twitter at the time of the study. An additional 1243 comments on misinformation-labeled tweets from 973 unique users were also collected, with 779 comments deemed relevant to study aims. Most of these comments expressed positive sentiment (n=612, 78.6%) to misinformation and did not refute it. Based on the World Health Organization Strategic Advisory Group of Experts framework, the most common vaccine hesitancy theme was individual or group influences (n=508, 65%), followed by vaccine or vaccination-specific influences (n=110, 14%) and contextual influences (n=93, 12%). Common misinformation themes observed included linking the death of Hank Aaron to “suspicious” elderly deaths following vaccination, claims about vaccines being used for depopulation, death panels, federal officials targeting Black Americans, and misinterpretation of VAERS reports. Four users engaging with or posting misinformation were verified on Twitter at the time of data collection.

Conclusions

Our study found that the death of a high-profile ethnic minority celebrity led to the spread of misinformation on Twitter. This misinformation directly challenged the safety and effectiveness of COVID-19 vaccines at a time when ensuring vaccine coverage among minority populations was paramount. Misinformation targeted at minority groups and echoed by other verified Twitter users has the potential to generate unwarranted vaccine hesitancy at the expense of people such as Hank Aaron who sought to promote public health and community immunity.

Multi-perspectives systematic review on the applications of sentiment analysis for vaccine hesitancy

A substantial impediment to widespread Coronavirus disease (COVID-19) vaccination is vaccine hesitancy. Many researchers across scientific disciplines have presented countless studies in favor of COVID-19 vaccination, but misinformation on social media could hinder vaccination efforts and increase vaccine hesitancy. Nevertheless, studying people's perceptions on social media to understand their sentiment presents a powerful medium for researchers to identify the causes of vaccine hesitancy and therefore develop appropriate public health messages and interventions. To the best of the authors' knowledge, previous studies have presented vaccine hesitancy in specific cases or within one scientific discipline (i.e., social, medical, and technological). No previous study has presented findings via sentiment analysis for multiple scientific disciplines as follows: (1) social, (2) medical, public health, and (3) technology sciences. Therefore, this research aimed to review and analyze articles related to different vaccine hesitancy cases in the last 11 years and understand the application of sentiment analysis on the most important literature findings. Articles were systematically searched in Web of Science, Scopus, PubMed, IEEEXplore, ScienceDirect, and Ovid from January 1, 2010, to July 2021. A total of 30 articles were selected on the basis of inclusion and exclusion criteria. These articles were formed into a taxonomy of literature, along with challenges, motivations, and recommendations for social, medical, and public health and technology sciences. Significant patterns were identified, and opportunities were promoted towards the understanding of this phenomenon.

Development and multicenter validation of chest X-ray radiography interpretations based on natural language processing

Background

Artificial intelligence can assist in interpreting chest X-ray radiography (CXR) data, but large datasets require efficient image annotation. The purpose of this study is to extract CXR labels from diagnostic reports based on natural language processing, train convolutional neural networks (CNNs), and evaluate the classification performance of CNN using CXR data from multiple centers

Methods

We collected the CXR images and corresponding radiology reports of 74,082 subjects as the training dataset. The linguistic entities and relationships from unstructured radiology reports were extracted by the bidirectional encoder representations from transformers (BERT) model, and a knowledge graph was constructed to represent the association between image labels of abnormal signs and the report text of CXR. Then, a 25-label classification system were built to train and test the CNN models with weakly supervised labeling.

Results

In three external test cohorts of 5,996 symptomatic patients, 2,130 screening examinees, and 1,804 community clinic patients, the mean AUC of identifying 25 abnormal signs by CNN reaches 0.866 ± 0.110, 0.891 ± 0.147, and 0.796 ± 0.157, respectively. In symptomatic patients, CNN shows no significant difference with local radiologists in identifying 21 signs (p > 0.05), but is poorer for 4 signs (p < 0.05). In screening examinees, CNN shows no significant difference for 17 signs (p > 0.05), but is poorer at classifying nodules (p = 0.013). In community clinic patients, CNN shows no significant difference for 12 signs (p > 0.05), but performs better for 6 signs (p < 0.001).

Conclusion

We construct and validate an effective CXR interpretation system based on natural language processing.

Chest X-rays are accompanied by a report from the radiologist, which contains valuable diagnostic information in text format. Extracting and interpreting information from these reports, such as keywords, is time-consuming, but artificial intelligence (AI) can help with this. Here, we use a type of AI known as natural language processing to extract information about abnormal signs seen on chest X-rays from the corresponding report. We develop and test natural language processing models using data from multiple hospitals and clinics, and show that our models achieve similar performance to interpretation from the radiologists themselves. Our findings suggest that AI might help radiologists to speed up interpretation of chest X-ray reports, which could be useful not only in patient triage and diagnosis but also cataloguing and searching of radiology datasets.

Zhang et al. develop a natural language processing approach, based on the BERT model, to extract linguistic information from chest X-ray radiography reports. The authors establish a 25-label classification system for abnormal findings described in the reports and validate their model using data from multiple sites.

Public perception of COVID-19 vaccines from the digital footprints left on Twitter: analyzing positive, neutral and negative sentiments of Twitterati

Purpose

Twitter is gaining popularity as a microblogging and social networking service to discuss various social issues. Coronavirus disease 2019 (COVID-19) has become a global pandemic and is discussed worldwide. Social media is an instant platform to deliberate various dimensions of COVID-19. The purpose of the study is to explore and analyze the public sentiments related to COVID-19 vaccines across the Twitter messages (positive, neutral, and negative) and the impact tweets make across digital social circles.

Design/methodology/approach

To fetch the vaccine-related posts, a manual examination of randomly selected 500 tweets was carried out to identify the popular hashtags relevant to the vaccine conversation. It was found that the hashtags “covid19vaccine” and “coronavirusvaccine” were the two popular hashtags used to discuss the communications related to COVID-19 vaccines. 23,575 global tweets available in public domain were retrieved through “Twitter Application Programming Interface” (API), using “Orange Software”, an open-source machine learning, data visualization and data mining toolkit. The study was confined to the tweets posted in English language only. The default data cleaning and preprocessing techniques available in the “Orange Software” were applied to the dataset, which include “transformation”, “tokenization” and “filtering”. The “Valence Aware Dictionary for sEntiment Reasoning” (VADER) tool was used for classification of tweets to determine the tweet sentiments (positive, neutral and negative) as well as the degree of sentiments (compound score also known as sentiment score). To assess the influence/impact of tweets account wise (verified and unverified) and sentiment wise (positive, neutral, and negative), the retweets and likes, which offer a sort of reward or acknowledgment of tweets, were used.

Findings

A gradual decline in the number of tweets over the time is observed. Majority (11,205; 47.52%) of tweets express positive sentiments, followed by neutral (7,948; 33.71%) and negative sentiments (4,422; 18.75%), respectively. The study also signifies a substantial difference between the impact of tweets tweeted by verified and unverified users. The tweets related to verified users have a higher impact both in terms of retweets (65.91%) and likes (84.62%) compared to the tweets tweeted by unverified users. Tweets expressing positive sentiments have the highest impact both in terms of likes (mean = 10.48) and retweets (mean = 3.07) compared to those that express neutral or negative sentiments.

Research limitations/implications

The main limitation of the study is that the sentiments of the people expressed over one single social platform, that is, Twitter have been studied which cannot generalize the global public perceptions. There can be a variation in the results when the datasets from other social media platforms will be studied.

Practical implications

The study will help to know the people's sentiments and beliefs toward the COVID-19 vaccines. Sentiments that people hold about the COVID-19 vaccines are studied, which will help health policymakers understand the polarity (positive, negative, and neutral) of the tweets and thus see the public reaction and reflect the types of information people are exposed to about vaccines. The study can aid the health sectors to intensify positive messages and eliminate negative messages for an enhanced vaccination uptake. The research can also help design more operative vaccine-advocating communication by customizing messages using the obtained knowledge from the sentiments and opinions about the vaccines.

Originality/value

The paper focuses on an essential aspect of COVID-19 vaccines and how people express themselves (positively, neutrally and negatively) on Twitter.

COVID-19 Vaccine and Social Media in the U.S.: Exploring Emotions and Discussions on Twitter

The understanding of the public response to COVID-19 vaccines is the key success factor to control the COVID-19 pandemic. To understand the public response, there is a need to explore public opinion. Traditional surveys are expensive and time-consuming, address limited health topics, and obtain small-scale data. Twitter can provide a great opportunity to understand public opinion regarding COVID-19 vaccines. The current study proposes an approach using computational and human coding methods to collect and analyze a large number of tweets to provide a wider perspective on the COVID-19 vaccine. This study identifies the sentiment of tweets using a machine learning rule-based approach, discovers major topics, explores temporal trend and compares topics of negative and non-negative tweets using statistical tests, and discloses top topics of tweets having negative and non-negative sentiment. Our findings show that the negative sentiment regarding the COVID-19 vaccine had a decreasing trend between November 2020 and February 2021. We found Twitter users have discussed a wide range of topics from vaccination sites to the 2020 U.S. election between November 2020 and February 2021. The findings show that there was a significant difference between tweets having negative and non-negative sentiment regarding the weight of most topics. Our results also indicate that the negative and non-negative tweets had different topic priorities and focuses. This research illustrates that Twitter data can be used to explore public opinion regarding the COVID-19 vaccine.