Combating hate speech using an adaptive ensemble learning model with a case study on COVID-19

Mapping the scientific knowledge and approaches to defining and measuring hate crime, hate speech, and hate incidents: A systematic review

Background

The difficulties in defining hate crime, hate incidents and hate speech, and in finding a common conceptual basis constitute a key barrier toward operationalisation in research, policy and programming. Definitions disagree about issues such as the identities that should be protected, the types of behaviours that should be referred to as hateful, and how the 'hate element' should be assessed. The lack of solid conceptual foundations is reflected in the absence of sound data. These issues have been raised since the early 1990s (Berk, 1990; Byers & Venturelli, 1994) but they proved to be an intractable problem that continues to affect this research and policy domain.

Objectives

Our systematic review has two objectives that are fundamentally connected: mapping (1) original definitions and (2) original measurement tools of hate crime, hate speech, hate incidents and surrogate terms, that is, alternative terms used for these concepts (e.g., prejudice-motivated crime, bias crime, among many others).

Search Methods

We systematically searched over 19 databases to retrieve academic and grey literature, as well as legislation. In addition, we contacted 26 country experts and searched 211 websites, as well as bibliographies of published reviews of related literature, and scrutiny of annotated bibliographies of related literature.

Inclusion Criteria

This review included documents published after 1990 found in academic literature, grey literature and legislation. We included academic empirical articles with any study design, as well as theoretical articles that focused specifically on defining hate crime, hate speech, hate incidents or surrogate terms. We also reviewed current criminal or civil legislation that is intended to regulate forms of hate speech, hate incidents and hate crimes. Eligible countries included Canada, USA, UK, Ireland, Germany, France, Italy, Spain, Australia and New Zealand. For documents to be included in relation to research objective (1), they had to contain at least one original definition of hate speech, hate incidents or hate crimes, or any surrogate term. For documents to be included in relation to research objective (2), they had to contain at least one original measurement tool of hate speech, hate incidents or hate crimes, or any surrogate term. Documents could be included in relation to both research objectives.

Data Collection and Analysis

The systematic search covered 1 January 1990 to 31 December 2021, with searches of academic databases conducted between 8th March and 12th April 2022 yielding 35,191 references. We carried out country-specific searches for grey literature published in the same time period between 27th August and 2nd December 2021. These searches yielded a total of 2748 results. We coded characteristics of the definitions and measurement tools, including the protected characteristics, the approaches to categorise the 'hate element' and other variables. We used univariate and bivariate statistical methods for data analysis. We also carried out a social network analysis.

Main Results

We provide as annex complete lists of the original definitions and measurement tools that met our inclusion criteria, for the use of researchers and policy makers worldwide. We included 423 definitions and 168 measurement tools in academic and grey literature, and 83 definitions found in legislation. To support future research and policy work in this area, we included a synthetic assessment of the (1) the operationalisability of each definition and (2) the theoretical robustness and transparency of each measurement tool. Our mapping of the definitions and measurement tools revealed numerous significant trends, clusters and differences between and within definitions and measurement tools focusing on hate crime, hate speech and hate incidents. For example, definitions and measurement tools tend to focus more on ethnic and religious identities (e.g., racism, antisemitism, Islamophobia) compared to sexual, gender and disability-related identities. This gap is greater in the definitions and measurement tools of hate speech than hate crime. Our analysis showed geographical patterns: hate crime definitions and measurement tools are more likely to originate from Anglophonic countries, especially the USA, but hate speech definitions and measurement tools are more likely to originate from continental Europe. In terms of disciplinary fragmentation, our social network analysis revealed that the collaboration and exchange of conceptual frameworks and methodological tools between social sciences and computer science is limited, with most definitions and measurement tools clustering along disciplinary lines. More detailed findings are presented in the results section of the report.

Authors' Conclusions

There is an urgent need to close the research and policy gap between the protections of 'ethnic and religious identities' and other (less) protected characteristics such as gender and sexual identities, age and disability. There is also an urgent need to improve the quality of methodological and reporting standards in research examining hate behaviours, including transparency in methodology and data reporting, and discussion of limitations (e.g., bias in data). Many of the measurement tools found in the academic literature were excluded because they did not report transparently how they collected and analysed the data. Further, 41% of documents presenting research on hate behaviours did not provide a definition of what they were looking at. Given the importance of this policy domain, it is vital to raise the quality and trustworthiness of research in this area. This review found that researchers in different disciplinary areas (e.g., social sciences and computer science) rarely collaborate. Future research should attempt to build on existing definitions and measurement tools (instead of duplicating efforts), and engage in more interdisciplinary collaborations. It is our hope that that this review can provide a solid foundation for researchers, government, and other bodies to build cumulative knowledge and collaboration in this important field.

Perception of Hate Speech by the Public and Experts: Insights into Predictors of the Perceived Hate Speech Towards Migrants

Online hate speech is a matter of concern for social media platforms, regulators, researchers, and the public. Despite its widespread prevalence and contentious nature, little research has been done on the perception of hate speech and its psychosocial predictors. To address this gap, we conducted a study on the perception of hate speech toward migrants in online comments, analyzing the differences between a public group (NPublic = 649) and an expert group (NExperts = 27) and exploring the correlation between the proposed hate speech indicators and perceived hate speech in both groups. Additionally, we explored various predictors of hate speech perception, including demographic and psychological variables such as human values, prejudice, aggression, impulsiveness, social media behavior, attitudes toward migrants and migration, and trust in institutions. Our results show that the public and experts have differing sensitivities toward hate speech, with the expert group perceiving comments as more hateful and emotionally harmful compared with the general population, who tend to agree more with antimigrant hateful comments. The proposed hate speech indicators and especially their total scores have a strong correlation with both groups' perceptions of hate speech. Psychological predictors, such as the human values of universalism, tradition, security, and subjective social distance, were significant predictors of online hate speech sensitivity. Our findings emphasize the need for public and scholarly discussions, more robust educational policies, and intervention programs with specific measures to counter hate speech online.

Novel Hate Speech Detection Using Word Cloud Visualization and Ensemble Learning Coupled with Count Vectorizer

A plethora of negative behavioural activities have recently been found in social media. Incidents such as trolling and hate speech on social media, especially on Twitter, have grown considerably. Therefore, detection of hate speech on Twitter has become an area of interest among many researchers. In this paper, we present a computational framework to (1) examine out the computational challenges behind hate speech detection and (2) generate high performance results. First, we extract features from Twitter data by utilizing a count vectorizer technique. Then, we provide the labeled dataset of constructed features to adopted ensemble methods, including Bagging, AdaBoost, and Random Forest. After training, we classify new tweet examples into one of the two categories, hate speech or non-hate speech. Experimental results show (1) that Random Forest has surpassed other methods by generating 95% using accuracy performance results and (2) word cloud displays the most prominent tweets that are responsible for hateful sentiments.

A Literature Review of Textual Hate Speech Detection Methods and Datasets

Online toxic discourses could result in conflicts between groups or harm to online communities. Hate speech is complex and multifaceted harmful or offensive content targeting individuals or groups. Existing literature reviews have generally focused on a particular category of hate speech, and to the best of our knowledge, no review has been dedicated to hate speech datasets. This paper systematically reviews textual hate speech detection systems and highlights their primary datasets, textual features, and machine learning models. The results of this literature review are integrated with content analysis, resulting in several themes for 138 relevant papers. This study shows several approaches that do not provide consistent results in various hate speech categories. The most dominant sets of methods combine more than one deep learning model. Moreover, the analysis of several hate speech datasets shows that many datasets are small in size and are not reliable for various tasks of hate speech detection. Therefore, this study provides the research community with insights and empirical evidence on the intrinsic properties of hate speech and helps communities identify topics for future work.