Measuring political radicalism and extremism in surveys: Three new scales

This paper introduces three new scales to measure left- and right-wing radical as well as general extremist attitudes that can be applied across Western European countries. We therefore propose a thorough conceptualization of extremist attitudes that consists of two dimensions: general extremism, by which we understand attitudes that oppose the constitutional democratic state, and another dimension that differentiates between right- and left-wing radicalism by which we understand people who take far-reaching but often one-sided positions on political issues (e.g., on nationalism or anti-imperialism) by advocating fundamental socio-political change. Based on data from Germany, Great Britain, and the Netherlands (n = 6,201) we created short indices for general extremism and left- and right-wing radicalism. We check for convergence validity by assessing the psychometric properties of the extracted indices, i.e. their internal coherence and the degree to which a scale is able to distinguish strongly extremist and non-extremist individuals. Finally, we correlate the scales with various constructs that are likely related to extremist attitudes in order to assure external or construct validity. The results indicate that the three scales are highly valid and applicable across three Western European countries. Overall, we find that about two to four percent of citizens in each country hold left-wing or right-wing extremist attitudes.

Observing many researchers using the same data and hypothesis reveals a hidden universe of uncertainty

This study explores how researchers' analytical choices affect the reliability of scientific findings. Most discussions of reliability problems in science focus on systematic biases. We broaden the lens to emphasize the idiosyncrasy of conscious and unconscious decisions that researchers make during data analysis. We coordinated 161 researchers in 73 research teams and observed their research decisions as they used the same data to independently test the same prominent social science hypothesis: that greater immigration reduces support for social policies among the public. In this typical case of social science research, research teams reported both widely diverging numerical findings and substantive conclusions despite identical start conditions. Researchers' expertise, prior beliefs, and expectations barely predict the wide variation in research outcomes. More than 95% of the total variance in numerical results remains unexplained even after qualitative coding of all identifiable decisions in each team's workflow. This reveals a universe of uncertainty that remains hidden when considering a single study in isolation. The idiosyncratic nature of how researchers' results and conclusions varied is a previously underappreciated explanation for why many scientific hypotheses remain contested. These results call for greater epistemic humility and clarity in reporting scientific findings.

How populist attitudes scales fail to capture support for populists in power

Populist attitudes are generally measured in surveys through three necessary and non-compensatory elements of populism, namely anti-elitism, people-centrism, and Manicheanism. Using Comparative Study of Electoral Systems Module 5 (2016-2020) data for 30 countries, we evaluate whether this approach explains voting for populist parties across countries in Asia, Europe and the Americas. We show that the existing scales of populist attitudes effectively explain voting for populists in countries where populist leaders and parties are in opposition but fail to explain voting for populist parties in countries where they are in power. We argue that current approaches assume "the elite" to mean "politicians", thus failing to capture attitudes towards "non-political elites" often targeted by populists in office-in particular, journalists, academics/experts, bureaucrats, and corporate business leaders. The results reveal limits to the usefulness of existing survey batteries in cross-national studies of populism and emphasize the need to develop approaches that are more generalizable across political and national contexts.