Botometer 101: social bot practicum for computational social scientists

Improving Social Bot Detection Through Aid and Training

OBJECTIVE

We test the effects of three aids on individuals' ability to detect social bots among Twitter personas: a bot indicator score, a training video, and a warning.

BACKGROUND

Detecting social bots can prevent online deception. We use a simulated social media task to evaluate three aids.

METHOD

Lay participants judged whether each of 60 Twitter personas was a human or social bot in a simulated online environment, using agreement between three machine learning algorithms to estimate the probability of each persona being a bot. Experiment 1 compared a control group and two intervention groups, one provided a bot indicator score for each tweet; the other provided a warning about social bots. Experiment 2 compared a control group and two intervention groups, one receiving the bot indicator scores and the other a training video, focused on heuristics for identifying social bots.

RESULTS

The bot indicator score intervention improved predictive performance and reduced overconfidence in both experiments. The training video was also effective, although somewhat less so. The warning had no effect. Participants rarely reported willingness to share content for a persona that they labeled as a bot, even when they agreed with it.

CONCLUSIONS

Informative interventions improved social bot detection; warning alone did not.

APPLICATION

We offer an experimental testbed and methodology that can be used to evaluate and refine interventions designed to reduce vulnerability to social bots. We show the value of two interventions that could be applied in many settings.

From Tweets to Streets: Observational Study on the Association Between Twitter Sentiment and Anti-Asian Hate Crimes in New York City from 2019 to 2022

BACKGROUND

Anti-Asian hate crimes escalated during the COVID-19 pandemic; however, limited research has explored the association between social media sentiment and hate crimes toward Asian communities.

OBJECTIVE

This study aims to investigate the relationship between Twitter (rebranded as X) sentiment data and the occurrence of anti-Asian hate crimes in New York City from 2019 to 2022, a period encompassing both before and during COVID-19 pandemic conditions.

METHODS

We used a hate crime dataset from the New York City Police Department. This dataset included detailed information on the occurrence of anti-Asian hate crimes at the police precinct level from 2019 to 2022. We used Twitter's application programming interface for Academic Research to collect a random 1% sample of publicly available Twitter data in New York State, including New York City, that included 1 or more of the selected Asian-related keywords and applied support vector machine to classify sentiment. We measured sentiment toward the Asian community using the rates of negative and positive sentiment expressed in tweets at the monthly level (N=48). We used negative binomial models to explore the associations between sentiment levels and the number of anti-Asian hate crimes in the same month. We further adjusted our models for confounders such as the unemployment rate and the emergence of the COVID-19 pandemic. As sensitivity analyses, we used distributed lag models to capture 1- to 2-month lag times.

RESULTS

A point increase of 1% in negative sentiment rate toward the Asian community in the same month was associated with a 24% increase (incidence rate ratio [IRR] 1.24; 95% CI 1.07-1.44; P=.005) in the number of anti-Asian hate crimes. The association was slightly attenuated after adjusting for unemployment and COVID-19 emergence (ie, after March 2020; P=.008). The positive sentiment toward Asian tweets with a 0-month lag was associated with a 12% decrease (IRR 0.88; 95% CI 0.79-0.97; P=.002) in expected anti-Asian hate crimes in the same month, but the relationship was no longer significant after adjusting for the unemployment rate and the emergence of COVID-19 pandemic (P=.11).

CONCLUSIONS

A higher negative sentiment level was associated with more hate crimes specifically targeting the Asian community in the same month. The findings highlight the importance of monitoring public sentiment to predict and potentially mitigate hate crimes against Asian individuals.

SEGCN: a subgraph encoding based graph convolutional network model for social bot detection

Message passing neural networks such as graph convolutional networks (GCN) can jointly consider various types of features for social bot detection. However, the expressive power of GCN is upper-bounded by the 1st-order Weisfeiler-Leman isomorphism test, which limits the detection performance for the social bots. In this paper, we propose a subgraph encoding based GCN model, SEGCN, with stronger expressive power for social bot detection. Each node representation of this model is computed as the encoding of a surrounding induced subgraph rather than encoding of immediate neighbors only. Extensive experimental results on two publicly available datasets, Twibot-20 and Twibot-22, showed that the proposed model improves the accuracy of the state-of-the-art social bot detection models by around 2.4%, 3.1%, respectively.

How does Twitter account moderation work? Dynamics of account creation and suspension on Twitter during major geopolitical events

Social media moderation policies are often at the center of public debate, and their implementation and enactment are sometimes surrounded by a veil of mystery. Unsurprisingly, due to limited platform transparency and data access, relatively little research has been devoted to characterizing moderation dynamics, especially in the context of controversial events and the platform activity associated with them. Here, we study the dynamics of account creation and suspension on Twitter during two global political events: Russia's invasion of Ukraine and the 2022 French Presidential election. Leveraging a large-scale dataset of 270M tweets shared by 16M users in multiple languages over several months, we identify peaks of suspicious account creation and suspension, and we characterize behaviors that more frequently lead to account suspension. We show how large numbers of accounts get suspended within days of their creation. Suspended accounts tend to mostly interact with legitimate users, as opposed to other suspicious accounts, making unwarranted and excessive use of reply and mention features, and sharing large amounts of spam and harmful content. While we are only able to speculate about the specific causes leading to a given account suspension, our findings contribute to shedding light on patterns of platform abuse and subsequent moderation during major events.

Evaluative altmetrics: is there evidence for its application to research evaluation?

Introduction

Altmetrics have been demonstrated as a promising tool for analyzing scientific communication on social media. Nevertheless, its application for research evaluation remains underdeveloped, despite the advancement of research in the study of diverse scientific interactions.

Methods

This paper develops a method for applying altmetrics in the evaluation of researchers, focusing on a case study of the Environment/Ecology ESI field publications by researchers at the University of Granada. We considered Twitter as a mirror of social attention, news outlets as media, and Wikipedia as educational, exploring mentions from these three sources and the associated actors in their respective media, contextualizing them using various metrics.

Results

Our analysis evaluated different dimensions such as the type of audience, local attention, engagement generated around the mention, and the profile of the actor. Our methodology effectively provided dashboards that gave a comprehensive view of the different instances of social attention at the author level.

Discussion

The use of altmetrics for research evaluation presents significant potential, as shown by our case study. While this is a novel method, our results suggest that altmetrics could provide valuable insights into the social attention that researchers garner. This can be an important tool for research evaluation, expanding our understanding beyond traditional metrics.

Analysis and mining of an election-based network using large-scale twitter data: a retrospective study

The user-generated Twitter data are a rich source of study and research that reflects the various social, economic, political, and other issues affecting people across the world. Analysis of the social interactions among users, who express themselves online, reveals different internal dynamics and provides detailed insights into real-world phenomena. In this paper, the structure and dynamics of the state assembly election-based tweet-reply network have been studied, as generated by Twitter users across the country of India for a period of 6-weeks. We study the flow of Twitter activity pertaining to the West Bengal assembly elections, along with the identification of the hashtags used by the three main political contenders. This information is used to identify the cluster-level dominance in the Twitter network over the 6-weeks of study. It is observed that this cluster dominance information is representative of the actual outcome of the elections, and can be effectively used as a forecasting tool. The collected tweets are used for lexicon-based emotion detection and further analysis. This highlights the reaction of the social media users in response to the events related to the election. It is observed that fear is the dominant emotion, while happiness is scarce in the opinions expressed during the studied duration. Next, the study and analysis of the complete reply-based social networks during weeks 1, 4, and 6 are undertaken. Important political and media actors are identified with standard network-level measures toward determining the efforts put in by the different clusters and individual actors involved in the election to control the network dominance.

Social media engagement and cryptocurrency performance

Cryptocurrencies are highly speculative assets with large price volatility. If one could forecast their behavior, this would make them more attractive to investors. In this work we study the problem of predicting the future performance of cryptocurrencies using social media data. We propose a new model to measure the engagement of users with topics discussed on social media based on interactions with social media posts. This model overcomes the limitations of previous volume and sentiment based approaches. We use this model to estimate engagement coefficients for 48 cryptocurrencies created between 2019 and 2021 using data from Twitter from the first month of the cryptocurrencies' existence. We find that the future returns of the cryptocurrencies are dependent on the engagement coefficients. Cryptocurrencies whose engagement coefficients have extreme values have lower returns. Low engagement coefficients signal a lack of interest, while high engagement coefficients signal artificial activity which is likely from automated accounts known as bots. We measure the amount of bot posts for the cryptocurrencies and find that generally, cryptocurrencies with more bot posts have lower future returns. While future returns are dependent on both the bot activity and engagement coefficient, the dependence is strongest for the engagement coefficient, especially for short-term returns. We show that simple investment strategies which select cryptocurrencies with engagement coefficients exceeding a fixed threshold perform well for holding times of a few months.

Public Opinion Manipulation on Social Media: Social Network Analysis of Twitter Bots during the COVID-19 Pandemic

Social media is not only an essential platform for the dissemination of public health-related information, but also an important channel for people to communicate during the COVID-19 pandemic. However, social bots can interfere with the social media topics that humans follow. We analyzed and visualized Twitter data during the prevalence of the Wuhan lab leak theory and discovered that 29% of the accounts participating in the discussion were social bots. We found evidence that social bots play an essential mediating role in communication networks. Although human accounts have a more direct influence on the information diffusion network, social bots have a more indirect influence. Unverified social bot accounts retweet more, and through multiple levels of diffusion, humans are vulnerable to messages manipulated by bots, driving the spread of unverified messages across social media. These findings show that limiting the use of social bots might be an effective method to minimize the spread of conspiracy theories and hate speech online.

Political polarization on Twitter during the COVID-19 pandemic: a case study in Brazil

The debate over the COVID-19 pandemic is constantly trending at online conversations since its beginning in 2019. The discussions in many social media platforms is related not only to health aspects of the disease, but also public policies and non-pharmacological measures to mitigate the spreading of the virus and propose alternative treatments. Divergent opinions regarding these measures are leading to heated discussions and polarization. Particularly in highly politically polarized countries, users tend to be divided in those in-favor or against government policies. In this work we present a computational method to analyze Twitter data and: (i) identify users with a high probability of being bots using only COVID-19 related messages; (ii) quantify the political polarization of the Brazilian general public in the context of the COVID-19 pandemic; (iii) analyze how bots tweet and affect political polarization. We collected over 100 million tweets from 26 April 2020 to 3 January 2021, and observed in general a highly polarized population (with polarization index varying from 0.57 to 0.86), which focuses on very different topics of discussions over the most polarized weeks–but all related to government and health-related events.