1
|
Li X, Liu J, Gao W, Cohen GL. Challenging the N-Heuristic: Effect size, not sample size, predicts the replicability of psychological science. PLoS One 2024; 19:e0306911. [PMID: 39178270 PMCID: PMC11343368 DOI: 10.1371/journal.pone.0306911] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/27/2023] [Accepted: 06/25/2024] [Indexed: 08/25/2024] Open
Abstract
Large sample size (N) is seen as a key criterion in judging the replicability of psychological research, a phenomenon we refer to as the N-Heuristic. This heuristic has led to the incentivization of fast, online, non-behavioral studies-to the potential detriment of psychological science. While large N should in principle increase statistical power and thus the replicability of effects, in practice it may not. Large-N studies may have other attributes that undercut their power or validity. Consolidating data from all systematic, large-scale attempts at replication (N = 307 original-replication study pairs), we find that the original study's sample size did not predict its likelihood of being replicated (rs = -0.02, p = 0.741), even with study design and research area controlled. By contrast, effect size emerged as a substantial predictor (rs = 0.21, p < 0.001), which held regardless of the study's sample size. N may be a poor predictor of replicability because studies with larger N investigated smaller effects (rs = -0.49, p < 0.001). Contrary to these results, a survey of 215 professional psychologists, presenting them with a comprehensive list of methodological criteria, found sample size to be rated as the most important criterion in judging a study's replicability. Our findings strike a cautionary note with respect to the prioritization of large N in judging the replicability of psychological science.
Collapse
Affiliation(s)
- Xingyu Li
- Graduate School of Education, Stanford University, Stanford, California, United States of America
| | - Jiting Liu
- School of Humanities and Social Science, The Chinese University of Hong Kong, Shenzhen, Shenzhen, China
| | - Weijia Gao
- Teachers College, Columbia University, New York, New York, United States of America
| | - Geoffrey L. Cohen
- Department of Psychology, Stanford University, Stanford, California, United States of America
| |
Collapse
|
2
|
Giner-Sorolla R, Montoya AK, Reifman A, Carpenter T, Lewis NA, Aberson CL, Bostyn DH, Conrique BG, Ng BW, Schoemann AM, Soderberg C. Power to Detect What? Considerations for Planning and Evaluating Sample Size. PERSONALITY AND SOCIAL PSYCHOLOGY REVIEW 2024; 28:276-301. [PMID: 38345247 PMCID: PMC11193916 DOI: 10.1177/10888683241228328] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/23/2024]
Abstract
ACADEMIC ABSTRACT In the wake of the replication crisis, social and personality psychologists have increased attention to power analysis and the adequacy of sample sizes. In this article, we analyze current controversies in this area, including choosing effect sizes, why and whether power analyses should be conducted on already-collected data, how to mitigate the negative effects of sample size criteria on specific kinds of research, and which power criterion to use. For novel research questions, we advocate that researchers base sample sizes on effects that are likely to be cost-effective for other people to implement (in applied settings) or to study (in basic research settings), given the limitations of interest-based minimums or field-wide effect sizes. We discuss two alternatives to power analysis, precision analysis and sequential analysis, and end with recommendations for improving the practices of researchers, reviewers, and journal editors in social-personality psychology. PUBLIC ABSTRACT Recently, social-personality psychology has been criticized for basing some of its conclusions on studies with low numbers of participants. As a result, power analysis, a mathematical way to ensure that a study has enough participants to reliably "detect" a given size of psychological effect, has become popular. This article describes power analysis and discusses some controversies about it, including how researchers should derive assumptions about effect size, and how the requirements of power analysis can be applied without harming research on hard-to-reach and marginalized communities. For novel research questions, we advocate that researchers base sample sizes on effects that are likely to be cost-effective for other people to implement (in applied settings) or to study (in basic research settings). We discuss two alternatives to power analysis, precision analysis and sequential analysis, and end with recommendations for improving the practices of researchers, reviewers, and journal editors in social-personality psychology.
Collapse
Affiliation(s)
| | | | | | | | - Neil A. Lewis
- Cornell University & Weill Cornell Medical College, Ithaca, NY, USA
| | | | | | | | | | | | | |
Collapse
|
3
|
Chong JY, Anderson GM, Fraley RC. Natural clues to danger: attachment behavior in threatening situations. Attach Hum Dev 2024; 26:366-382. [PMID: 38995104 DOI: 10.1080/14616734.2024.2376764] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2024] [Accepted: 07/02/2024] [Indexed: 07/13/2024]
Abstract
A fundamental principle of attachment theory is that threatening situations give rise to individual differences in the extent to which people seek proximity to close others. The current research examines the way in which attachment styles predict individual differences in attachment-relevant behavior during threatening events. We tested alternative theoretical perspectives concerning the association between adult attachment (specifically, attachment avoidance) and attachment behavior in the presence of natural clues to danger by observing couples (N = 204) when they were watching horror vs. control film excerpts. Results suggest that highly avoidant people engaged in less attachment behavior across both threatening and non-threatening situations. These findings have implications for the understanding of attachment-related processes and how working models of the self and others facilitate (or inhibit) the expression of attachment behavior.
Collapse
Affiliation(s)
- Jia Y Chong
- Department of Psychology, University of Illinois at Urbana-Champaign, Champaign, USA
| | - Gabrielle M Anderson
- Department of Psychology, University of Illinois at Urbana-Champaign, Champaign, USA
| | - R Chris Fraley
- Department of Psychology, University of Illinois at Urbana-Champaign, Champaign, USA
| |
Collapse
|
4
|
Cuskley C, Sulik J. The Burden for High-Quality Online Data Collection Lies With Researchers, Not Recruitment Platforms. PERSPECTIVES ON PSYCHOLOGICAL SCIENCE 2024:17456916241242734. [PMID: 38648556 DOI: 10.1177/17456916241242734] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/25/2024]
Abstract
A recent article in Perspectives on Psychological Science (Webb & Tangney, 2022) reported a study in which just 2.6% of participants recruited on Amazon's Mechanical Turk (MTurk) were deemed "valid." The authors highlighted some well-established limitations of MTurk, but their central claims-that MTurk is "too good to be true" and that it captured "only 14 human beings . . . [out of] N = 529"-are radically misleading, yet have been repeated widely. This commentary aims to (a) correct the record (i.e., by showing that Webb and Tangney's approach to data collection led to unusually low data quality) and (b) offer a shift in perspective for running high-quality studies online. Negative attitudes toward MTurk sometimes reflect a fundamental misunderstanding of what the platform offers and how it should be used in research. Beyond pointing to research that details strategies for effective design and recruitment on MTurk, we stress that MTurk is not suitable for every study. Effective use requires specific expertise and design considerations. Like all tools used in research-from advanced hardware to specialist software-the tool itself places constraints on what one should use it for. Ultimately, high-quality data is the responsibility of the researcher, not the crowdsourcing platform.
Collapse
Affiliation(s)
- Christine Cuskley
- Language Evolution, Acquisition, and Development Group, Newcastle University
| | | |
Collapse
|
5
|
Keith MG, McKay AS. Too Anecdotal to Be True? Mechanical Turk Is Not All Bots and Bad Data: Response to Webb and Tangney (2022). PERSPECTIVES ON PSYCHOLOGICAL SCIENCE 2024:17456916241234328. [PMID: 38451252 DOI: 10.1177/17456916241234328] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/08/2024]
Abstract
In response to Webb and Tangney (2022) we call into question the conclusion that data collected on Amazon's Mechanical Turk (MTurk) was "at best-only 2.6% valid" (p. 1). We suggest that Webb and Tangney made certain choices during the study-design and data-collection process that adversely affected the quality of the data collected. As a result, the anecdotal experience of these authors provides weak evidence that MTurk provides low-quality data as implied. In our commentary we highlight best practice recommendations and make suggestions for more effectively collecting and screening online panel data.
Collapse
Affiliation(s)
| | - Alexander S McKay
- Department of Management and Entrepreneurship, School of Business, Virginia Commonwealth University
| |
Collapse
|
6
|
Messeri L, Crockett MJ. Artificial intelligence and illusions of understanding in scientific research. Nature 2024; 627:49-58. [PMID: 38448693 DOI: 10.1038/s41586-024-07146-0] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2023] [Accepted: 01/31/2024] [Indexed: 03/08/2024]
Abstract
Scientists are enthusiastically imagining ways in which artificial intelligence (AI) tools might improve research. Why are AI tools so attractive and what are the risks of implementing them across the research pipeline? Here we develop a taxonomy of scientists' visions for AI, observing that their appeal comes from promises to improve productivity and objectivity by overcoming human shortcomings. But proposed AI solutions can also exploit our cognitive limitations, making us vulnerable to illusions of understanding in which we believe we understand more about the world than we actually do. Such illusions obscure the scientific community's ability to see the formation of scientific monocultures, in which some types of methods, questions and viewpoints come to dominate alternative approaches, making science less innovative and more vulnerable to errors. The proliferation of AI tools in science risks introducing a phase of scientific enquiry in which we produce more but understand less. By analysing the appeal of these tools, we provide a framework for advancing discussions of responsible knowledge production in the age of AI.
Collapse
Affiliation(s)
- Lisa Messeri
- Department of Anthropology, Yale University, New Haven, CT, USA.
| | - M J Crockett
- Department of Psychology, Princeton University, Princeton, NJ, USA.
- University Center for Human Values, Princeton University, Princeton, NJ, USA.
| |
Collapse
|
7
|
Krendl AC, Hugenberg K, Kennedy DP. Comparing data quality from an online and in-person lab sample on dynamic theory of mind tasks. Behav Res Methods 2024; 56:2353-2375. [PMID: 37322311 PMCID: PMC11061881 DOI: 10.3758/s13428-023-02152-y] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/24/2023] [Indexed: 06/17/2023]
Abstract
Nearly half the published research in psychology is conducted with online samples, but the preponderance of these studies rely primarily on self-report measures. The current study validated data quality from an online sample on a novel, dynamic task by comparing performance between an in-lab and online sample on two dynamic measures of theory of mind-the ability to infer others' mental states. Theory of mind is a cognitively complex construct that has been widely studied across multiple domains of psychology. One task was based on the show The Office®, and has been previously validated by the authors with in-lab samples. The second was a novel task based on the show Nathan for You®, which was selected to account for familiarity effects associated with The Office. Both tasks measured various dimensions of theory of mind (inferring beliefs, understanding motivations, detecting deception, identifying faux pas, and understanding emotions). The in-person lab samples (N = 144 and 177, respectively) completed the tasks between-subject, whereas the online sample (N = 347 from Prolific Academic) completed them within-subject, with order counterbalanced. The online sample's performance across both tasks was reliable (Cronbach's α = .66). For The Office, the in-person sample outperformed the online sample on some types of theory of mind, but this was driven by their greater familiarity with the show. Indeed, for the relatively unfamiliar show Nathan for You, performance did not differ between the two samples. Together, these results suggest that crowdsourcing platforms elicit reliable performance on novel, dynamic, complex tasks.
Collapse
Affiliation(s)
- Anne C. Krendl
- Department of Psychological & Brain Sciences, Indiana University, Bloomington, 1101 E. 10th St., Bloomington, IN 47405, USA
| | - Kurt Hugenberg
- Department of Psychological & Brain Sciences, Indiana University, Bloomington, 1101 E. 10th St., Bloomington, IN 47405, USA
| | - Daniel P. Kennedy
- Department of Psychological & Brain Sciences, Indiana University, Bloomington, 1101 E. 10th St., Bloomington, IN 47405, USA
| |
Collapse
|
8
|
Bauer PJ. Attention to Authenticity: An Essential Analogue to Focus on Rigor and Replicability. Psychol Sci 2024; 35:3-6. [PMID: 38096250 DOI: 10.1177/09567976231220895] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/23/2024] Open
|
9
|
Moss AJ, Rosenzweig C, Robinson J, Jaffe SN, Litman L. Is it ethical to use Mechanical Turk for behavioral research? Relevant data from a representative survey of MTurk participants and wages. Behav Res Methods 2023; 55:4048-4067. [PMID: 37217711 DOI: 10.3758/s13428-022-02005-0] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/07/2022] [Indexed: 05/24/2023]
Abstract
To understand human behavior, social scientists need people and data. In the last decade, Amazon's Mechanical Turk (MTurk) emerged as a flexible, affordable, and reliable source of human participants and was widely adopted by academics. Yet despite MTurk's utility, some have questioned whether researchers should continue using the platform on ethical grounds. The brunt of their concern is that people on MTurk are financially insecure, subject to abuse, and earn inhumane wages. We investigated these issues with two representative probability surveys of the U.S. MTurk population (N = 4094). The surveys revealed: (1) the financial situation of people on MTurk mirrors the general population, (2) most participants do not find MTurk stressful or requesters abusive, and (3) MTurk offers flexibility and benefits that most people value above other options for work. People reported it is possible to earn more than $10 per hour and said they would not trade the flexibility of MTurk for less than $25 per hour. Altogether, our data are important for assessing whether MTurk is an ethical place for research.
Collapse
Affiliation(s)
| | - Cheskie Rosenzweig
- CloudResearch, Queens, NY, USA
- Department of Clinical Psychology, Columbia University, New York, NY, USA
| | - Jonathan Robinson
- CloudResearch, Queens, NY, USA
- Department of Computer Science, Lander College, Flushing, NY, USA
| | - Shalom N Jaffe
- CloudResearch, Queens, NY, USA
- Department of Psychology, Lander College, Flushing, NY, USA
| | - Leib Litman
- CloudResearch, Queens, NY, USA.
- Department of Psychology, Lander College, Flushing, NY, USA.
| |
Collapse
|
10
|
Uher J. What's wrong with rating scales? Psychology's replication and confidence crisis cannot be solved without transparency in data generation. SOCIAL AND PERSONALITY PSYCHOLOGY COMPASS 2023. [DOI: 10.1111/spc3.12740] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/12/2023]
Affiliation(s)
- Jana Uher
- School of Human Sciences University of Greenwich London UK
- London School of Economics and Political Science London UK
| |
Collapse
|
11
|
Treger S, Benau EM, Timko CA. Not so terrifying after all? A set of failed replications of the mortality salience effects of Terror Management Theory. PLoS One 2023; 18:e0285267. [PMID: 37159447 PMCID: PMC10168577 DOI: 10.1371/journal.pone.0285267] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2022] [Accepted: 04/18/2023] [Indexed: 05/11/2023] Open
Abstract
Terror Management Theory (TMT) postulates that humans, in response to awareness of their death, developed complex defenses to remove the salience and discomfort stemming from those thoughts. In a standard paradigm to test this theory, an individual is presented with a death-related prime (Mortality Salience; MS), such as writing the details of their own death, or something neutral, such as watching television. After a distractor task (for delay), participants complete the dependent variable, such as rating how much they like or agree with a pro- or anti-national essay and its author. Individuals in the MS condition typically exhibit greater worldview defense than control conditions by rating the pro-national essay more positively and the anti-national essay more negatively. We completed five separate studies across five unique samples with the goal of replicating and extending this well-established pattern to provide further understanding of the phenomena that underlie the effects of MS. However, despite using standard procedures, we were unable to replicate basic patterns of the dependent variable in the MS conditions. We also pooled all responses into two meta-analyses, one examining all dependent variables and one focusing on the anti-national essay; yet the effect sizes in these analyses did not significantly differ from zero. We discuss the methodological and theoretical implications of these (unintended) failures to replicate. It is not clear if these null findings were due to methodological limitations, restraints of online/crowd-sourced recruitment, or ever-evolving sociocultural factors.
Collapse
Affiliation(s)
- Stanislav Treger
- Northern Trust Corporation, Chicago, IL United States of America
| | - Erik M Benau
- Department of Psychology, SUNY Old Westbury, Old Westbury, NY, United States of America
| | - C Alix Timko
- Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States of America
- Department of Psychology, Towson University, Towson, MD, United States of America
| |
Collapse
|
12
|
Uher J. Rating scales institutionalise a network of logical errors and conceptual problems in research practices: A rigorous analysis showing ways to tackle psychology's crises. Front Psychol 2022; 13:1009893. [PMID: 36643697 PMCID: PMC9833395 DOI: 10.3389/fpsyg.2022.1009893] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2022] [Accepted: 10/14/2022] [Indexed: 12/29/2022] Open
Abstract
This article explores in-depth the metatheoretical and methodological foundations on which rating scales-by their very conception, design and application-are built and traces their historical origins. It brings together independent lines of critique from different scholars and disciplines to map out the problem landscape, which centres on the failed distinction between psychology's study phenomena (e.g., experiences, everyday constructs) and the means of their exploration (e.g., terms, data, scientific constructs)-psychologists' cardinal error. Rigorous analyses reveal a dense network of 12 complexes of problematic concepts, misconceived assumptions and fallacies that support each other, making it difficult to be identified and recognised by those (unwittingly) relying on them (e.g., various forms of reductionism, logical errors of operationalism, constructification, naïve use of language, quantificationism, statisticism, result-based data generation, misconceived nomotheticism). Through the popularity of rating scales for efficient quantitative data generation, uncritically interpreted as psychological measurement, these problems have become institutionalised in a wide range of research practices and perpetuate psychology's crises (e.g., replication, confidence, validation, generalizability). The article provides an in-depth understanding that is needed to get to the root of these problems, which preclude not just measurement but also the scientific exploration of psychology's study phenomena and thus its development as a science. From each of the 12 problem complexes; specific theoretical concepts, methodologies and methods are derived as well as key directions of development. The analyses-based on three central axioms for transdisciplinary research on individuals, (1) complexity, (2) complementarity and (3) anthropogenicity-highlight that psychologists must (further) develop an explicit metatheory and unambiguous terminology as well as concepts and theories that conceive individuals as living beings, open self-organising systems with complementary phenomena and dynamic interrelations across their multi-layered systemic contexts-thus, theories not simply of elemental properties and structures but of processes, relations, dynamicity, subjectivity, emergence, catalysis and transformation. Philosophical and theoretical foundations of approaches suited for exploring these phenomena must be developed together with methods of data generation and methods of data analysis that are appropriately adapted to the peculiarities of psychologists' study phenomena (e.g., intra-individual variation, momentariness, contextuality). Psychology can profit greatly from its unique position at the intersection of many other disciplines and can learn from their advancements to develop research practices that are suited to tackle its crises holistically.
Collapse
Affiliation(s)
- Jana Uher
- School of Human Sciences, University of Greenwich, London, United Kingdom
- London School of Economics, London, United Kingdom
| |
Collapse
|
13
|
Torrez B, Hudson STJ, Dupree CH. Racial equity in social psychological science: A guide for scholars, institutions, and the field. SOCIAL AND PERSONALITY PSYCHOLOGY COMPASS 2022. [DOI: 10.1111/spc3.12720] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
Affiliation(s)
- Brittany Torrez
- School of Management Yale University New Haven Connecticut USA
| | | | | |
Collapse
|
14
|
Hajek A, König H. Climate anxiety in Germany. Public Health 2022; 212:89-94. [DOI: 10.1016/j.puhe.2022.09.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2022] [Revised: 09/09/2022] [Accepted: 09/09/2022] [Indexed: 11/06/2022]
|
15
|
Simon AF, Wilder D. Methods and measures in social and personality psychology: a comparison of JPSP publications in 1982 and 2016. THE JOURNAL OF SOCIAL PSYCHOLOGY 2022:1-17. [PMID: 36259541 DOI: 10.1080/00224545.2022.2135088] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2021] [Accepted: 10/09/2022] [Indexed: 11/06/2022]
Abstract
Reviews of articles from social and personality psychology journals have been largely limited to subsets of publications from particular years and to a focus on descriptive qualities of articles. This paper compares the methods and measures employed in all empirical articles published in 1982 to those that appeared in 2016 from Journal of Personality and Social Psychology. There has been an increase in self-report data, a decline in behavioral observations, and an increase in detailed reports of procedures and analyses. A heavy reliance on laboratory experimentation involving college students has been supplanted by online participant pools and data collection procedures. Compared to 1982, articles in 2016 were fewer in number but longer in length, included more studies per article, and had a greater number and diversity of authors. Explanations are offered for these findings along with implications for social and personality psychology.
Collapse
|
16
|
Keith MG, Stevenor BA, McAbee ST. Scale Mean and Variance Differences in MTurk and Non-MTurk Samples. JOURNAL OF PERSONNEL PSYCHOLOGY 2022. [DOI: 10.1027/1866-5888/a000309] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022]
Abstract
Abstract. We meta-analytically examined mean and variance differences between MTurk and non-MTurk samples for a variety of scales used in the organizational sciences. The influence of moderators (i.e., construct domain and valence, sample type, use of qualifications, and data cleaning procedures) was also examined. Across all scales (120 scales, N = 110,090), we found that, overall, MTurk and non-MTurk samples do not have significantly different scale means or variances. Our moderator analyses, however, indicated that MTurk samples may have larger variances than non-MTurk samples under certain conditions. Additionally, our results highlight the importance of data screening and cleaning procedures such that MTurk samples had larger variances than non-MTurk samples when MTurk samples were subjected to more stringent data screening and cleaning procedures.
Collapse
Affiliation(s)
- Melissa G. Keith
- Department of Psychology, Bowling Green State University, Bowling Green, OH, USA
| | - Brent A. Stevenor
- Department of Psychology, Bowling Green State University, Bowling Green, OH, USA
| | - Samuel T. McAbee
- Department of Psychology, Bowling Green State University, Bowling Green, OH, USA
| |
Collapse
|
17
|
Bo O’Connor B, Lee K, Campbell D, Young L. Moral psychology from the lab to the wild: Relief registries as a paradigm for studying real-world altruism. PLoS One 2022; 17:e0269469. [PMID: 35696389 PMCID: PMC9191725 DOI: 10.1371/journal.pone.0269469] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2021] [Accepted: 05/20/2022] [Indexed: 11/19/2022] Open
Abstract
Experimental psychology's recent shift toward low-effort, high-volume methods (e.g., self-reports, online studies) and away from the more effortful study of naturalistic behavior raises concerns about the ecological validity of findings from these fields, concerns that have become particularly apparent in the field of moral psychology. To help address these concerns, we introduce a method allowing researchers to investigate an important, widespread form of altruistic behavior-charitable donations-in a manner balancing competing concerns about internal validity, ecological validity, and ease of implementation: relief registries, which leverage existing online gift registry platforms to allow research subjects to choose among highly needed donation items to ship directly to charitable organizations. Here, we demonstrate the use of relief registries in two experiments exploring the ecological validity of the finding from our own research that people are more willing to help others after having imagined themselves doing so. In this way, we sought to provide a blueprint for researchers seeking to enhance the ecological validity of their own research in a narrow sense (i.e., by using the relief registry method we introduce) and in broader terms by adapting methods that take advantage of modern technology to directly impact others' lives outside the lab.
Collapse
Affiliation(s)
- Brendan Bo O’Connor
- Department of Psychology, University at Albany, State University of New York, Albany, New York, United States of America
| | - Karen Lee
- Department of Psychology, Boston College, Chestnut Hill, Massachusetts, United States of America
| | - Dylan Campbell
- Department of Psychology, University at Albany, State University of New York, Albany, New York, United States of America
| | - Liane Young
- Department of Psychology, Boston College, Chestnut Hill, Massachusetts, United States of America
| |
Collapse
|
18
|
Ledgerwood A, Hudson SKTJ, Lewis NA, Maddox KB, Pickett CL, Remedios JD, Cheryan S, Diekman AB, Dutra NB, Goh JX, Goodwin SA, Munakata Y, Navarro DJ, Onyeador IN, Srivastava S, Wilkins CL. The Pandemic as a Portal: Reimagining Psychological Science as Truly Open and Inclusive. PERSPECTIVES ON PSYCHOLOGICAL SCIENCE 2022; 17:937-959. [PMID: 35235485 DOI: 10.1177/17456916211036654] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/20/2022]
Abstract
Psychological science is at an inflection point: The COVID-19 pandemic has exacerbated inequalities that stem from our historically closed and exclusive culture. Meanwhile, reform efforts to change the future of our science are too narrow in focus to fully succeed. In this article, we call on psychological scientists-focusing specifically on those who use quantitative methods in the United States as one context for such conversations-to begin reimagining our discipline as fundamentally open and inclusive. First, we discuss whom our discipline was designed to serve and how this history produced the inequitable reward and support systems we see today. Second, we highlight how current institutional responses to address worsening inequalities are inadequate, as well as how our disciplinary perspective may both help and hinder our ability to craft effective solutions. Third, we take a hard look in the mirror at the disconnect between what we ostensibly value as a field and what we actually practice. Fourth and finally, we lead readers through a roadmap for reimagining psychological science in whatever roles and spaces they occupy, from an informal discussion group in a department to a formal strategic planning retreat at a scientific society.
Collapse
Affiliation(s)
| | | | | | | | | | | | | | - Amanda B Diekman
- Department of Psychological and Brain Sciences, Indiana University
| | - Natalia B Dutra
- Laboratory of Evolution of Human Behavior, Department of Physiology and Behavior, Federal University of Rio Grande do Norte
| | - Jin X Goh
- Department of Psychology, Colby College
| | - Stephanie A Goodwin
- Department of Psychology, Wright State University.,Department of Social Sciences, Stevens Institute of Technology
| | - Yuko Munakata
- Department of Psychology, University of California, Davis
| | | | | | | | | |
Collapse
|
19
|
How Cross-Discipline Understanding and Communication Can Improve Research on Multiracial Populations. SOCIAL SCIENCES-BASEL 2022. [DOI: 10.3390/socsci11030090] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/04/2023]
Abstract
One of the strengths of Critical Mixed Race Studies is that it represents research methodologies and frameworks from multiple disciplines across the social sciences and humanities. However, if these disciplines are not in dialogue with each other, that benefit may be lost. Here, we use psychological and sociological research on Multiracial populations as examples to argue how strict disciplinarity and methodological trends may limit scientific production. We propose that reading and citing work across disciplines, expanding methodological training, and rejecting hegemonic “white logic” assumptions about what is “publishable” can enhance Multiracial research. First, the ability to cite effectively across disciplines will shorten the time it takes for new theories to be developed that focus on empirically underrepresented populations. Secondly, increasing understanding of both quantitative and qualitative methods will allow more effective reading between disciplines while also creating opportunities to engage with both causality and the richness of experiences that comprise being Multiracial. Finally, these changes would then situate scholars to be more effective reviewers, thereby enhancing the peer-reviewed publication process to one that routinely rejects color evasive racist practices that privilege work on majority populations.
Collapse
|
20
|
Ma X, Liao J. Buying to Cope With Scarcity During Public Emergencies: A Serial Mediation Model Based on Cognition-Affect Theory. Front Psychol 2022; 12:791850. [PMID: 35153914 PMCID: PMC8828481 DOI: 10.3389/fpsyg.2021.791850] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2021] [Accepted: 12/07/2021] [Indexed: 11/17/2022] Open
Abstract
Panic buying is a common phenomenon that occurs during public emergencies and has a significant undesirable impact on society. This research explored the effect of scarcity on panic buying and the role of perceived control and panic in this effect through big data, an online survey and behavior experiments in a real public emergency (i.e., COVID-19) and simulative public emergencies. The findings showed that scarcity aggravates panic buying (Studies 1–3), and this aggravation effect is serially mediated by perceived control and panic (Studies 2–3). Moreover, this serial mediation model is more suitable for public health emergencies (Study 3). These findings enrich the understanding of panic buying and provide important enlightenment for guiding rational public behavior and managing public opinion during public emergencies.
Collapse
Affiliation(s)
- Xinran Ma
- Department of Psychology, Tsinghua University, Beijing, China
| | - Jiangqun Liao
- Department of Psychology, Tsinghua University, Beijing, China
| |
Collapse
|
21
|
Campbell DS, Reiman AK. Has social psychology lost touch with reality? Exploring public perceptions of the realism and consequentiality of social psychological research. JOURNAL OF EXPERIMENTAL SOCIAL PSYCHOLOGY 2022. [DOI: 10.1016/j.jesp.2021.104255] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
|
22
|
Obschonka M, Cai Q, Chan ACY, Marsalis S, Basha SAJ, Lee SK, Gewirtz AH. International psychological research addressing the early phase of the COVID-19 pandemic: A rapid scoping review and implications for global psychology. INTERNATIONAL JOURNAL OF PSYCHOLOGY 2021; 57:1-19. [PMID: 34904220 DOI: 10.1002/ijop.12823] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/07/2021] [Accepted: 11/13/2021] [Indexed: 12/21/2022]
Abstract
In March 2020, the World Health Organization (WHO) declared COVID-19 a global pandemic. Given that keeping abreast of international perspectives and research results is of particular importance for such massive global emergencies, we employed a scoping review methodology to rapidly map the field of international psychological research addressing this important early phase of the pandemic. We included a total of 79 studies, with data mostly collected between March and June 2020. This review aimed to systematically identify and map the nature and scope of international studies examining psychological aspects of the unfolding COVID-19 pandemic. We mapped key research themes, subfields of psychology, the nature and extent of international research collaboration, data methods employed, and challenges and enablers faced by psychological researchers in the early stages of the pandemic. Among the wide range of themes covered, mental health and social behaviours were the key themes. Most studies were in clinical/health psychology and social psychology. Network analyses revealed how authors collaborated and to what extent the studies were international. Europe and the United States were often at the centre of international collaboration. The predominant study design was cross-sectional and online with quantitative analyses. We also summarised author reported critical challenges and enablers for international psychological research during the COVID pandemic, and conclude with implications for the field of psychology.
Collapse
Affiliation(s)
- Martin Obschonka
- School of Management, Queensland University of Technology, Brisbane, QLD, Australia
| | - Qiyue Cai
- Department of Psychology, Arizona State University, Tempe, AZ, USA
| | - Athena C Y Chan
- Department of Family Social Science, University of Minnesota Twin Cities, Minneapolis, MN, USA
| | - Scott Marsalis
- University Libraries, University of Minnesota Twin Cities, Minneapolis, MN, USA
| | - Sydni A J Basha
- Department of Psychology, Arizona State University, Tempe, AZ, USA
| | - Sun-Kyung Lee
- Department of Psychology, Arizona State University, Tempe, AZ, USA
| | - Abigail H Gewirtz
- Department of Psychology, Arizona State University, Tempe, AZ, USA.,Department of Family Social Science, University of Minnesota Twin Cities, Minneapolis, MN, USA
| |
Collapse
|
23
|
Abstract
Abstract. The present article reviews web-based research in psychology. It captures principles, learnings, and trends in several types of web-based research that show similar developments related to web technology and its major shifts (e.g., appearance of search engines, browser wars, deep web, commercialization, web services, HTML5…) as well as distinct challenges. The types of web-based research discussed are web surveys and questionnaire research, web-based tests, web experiments, Mobile Experience Sampling, and non-reactive web research, including big data. A number of web-based methods are presented and discussed that turned out to become important in research methodology. These are one-item-one-screen design, seriousness check, instruction manipulation and other attention checks, multiple site entry technique, subsampling technique, warm-up technique, and web-based measurement. Pitfalls and best practices are described then, especially regarding dropout and other non-response, recruitment of participants, and interaction between technology and psychological factors. The review concludes with a discussion of important concepts that have developed over 25 years and an outlook on future developments in web-based research.
Collapse
Affiliation(s)
- Ulf-Dietrich Reips
- Psychological Methods and Assessment / Experimental Psychology and Internet Science, Department of Psychology, University of Konstanz, Germany
| |
Collapse
|
24
|
Reips UD, Buchanan T. From Modems to Mobile Apps. ZEITSCHRIFT FUR PSYCHOLOGIE-JOURNAL OF PSYCHOLOGY 2021. [DOI: 10.1027/2151-2604/a000464] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022]
Affiliation(s)
- Ulf-Dietrich Reips
- Psychological Methods and Assessment / Experimental Psychology and Internet Science, Department of Psychology, University of Konstanz, Germany
| | - Tom Buchanan
- School of Social Sciences, University of Westminster, London, UK
| |
Collapse
|
25
|
Hilbig BE, Thielmann I. On the (Mis)Use of Deception in Web-Based Research. ZEITSCHRIFT FUR PSYCHOLOGIE-JOURNAL OF PSYCHOLOGY 2021. [DOI: 10.1027/2151-2604/a000466] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022]
Abstract
Abstract. The deception of research participants remains a controversial issue in the behavioral sciences. Current ethics codes consistently limit the use of deception to cases in which non-deceptive alternatives are unfeasible and, crucially, require that participants subjected to deception be debriefed correspondingly along with an option to withdraw their data after learning about the deception. These conditions pose a particular challenge in the context of web-based research because participants can typically discontinue a study unilaterally (i.e., dropout by simply closing the browser window) in which case full debriefing and an option to withdraw one’s data are no longer available. As a consequence, the study would no longer be compatible with ethical standards. Based on recent meta-analytical data, we provide an existence proof of this problem, showing that deception is used in web-based research with little to no indication of safeguards ensuring full debriefing and subsequent data withdrawal options. We close by revisiting recommendations for the (non-)use of deception in web-based research and offer solutions to implement such safeguards in case deception is truly unavoidable.
Collapse
Affiliation(s)
| | - Isabel Thielmann
- Cognitive Psychology, University of Koblenz-Landau, Landau, Germany
| |
Collapse
|
26
|
Chuey A, Asaba M, Bridgers S, Carrillo B, Dietz G, Garcia T, Leonard JA, Liu S, Merrick M, Radwan S, Stegall J, Velez N, Woo B, Wu Y, Zhou XJ, Frank MC, Gweon H. Moderated Online Data-Collection for Developmental Research: Methods and Replications. Front Psychol 2021; 12:734398. [PMID: 34803813 PMCID: PMC8595939 DOI: 10.3389/fpsyg.2021.734398] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2021] [Accepted: 10/06/2021] [Indexed: 11/24/2022] Open
Abstract
Online data collection methods are expanding the ease and access of developmental research for researchers and participants alike. While its popularity among developmental scientists has soared during the COVID-19 pandemic, its potential goes beyond just a means for safe, socially distanced data collection. In particular, advances in video conferencing software has enabled researchers to engage in face-to-face interactions with participants from nearly any location at any time. Due to the novelty of these methods, however, many researchers still remain uncertain about the differences in available approaches as well as the validity of online methods more broadly. In this article, we aim to address both issues with a focus on moderated (synchronous) data collected using video-conferencing software (e.g., Zoom). First, we review existing approaches for designing and executing moderated online studies with young children. We also present concrete examples of studies that implemented choice and verbal measures (Studies 1 and 2) and looking time (Studies 3 and 4) across both in-person and online moderated data collection methods. Direct comparison of the two methods within each study as well as a meta-analysis of all studies suggest that the results from the two methods are comparable, providing empirical support for the validity of moderated online data collection. Finally, we discuss current limitations of online data collection and possible solutions, as well as its potential to increase the accessibility, diversity, and replicability of developmental science.
Collapse
Affiliation(s)
- Aaron Chuey
- Department of Psychology, Stanford University, Palo Alto, CA, United States
| | - Mika Asaba
- Department of Psychology, Stanford University, Palo Alto, CA, United States
| | - Sophie Bridgers
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, United States
| | - Brandon Carrillo
- Department of Psychology, Stanford University, Palo Alto, CA, United States
| | - Griffin Dietz
- Department of Psychology, Stanford University, Palo Alto, CA, United States
| | - Teresa Garcia
- Department of Psychology, Stanford University, Palo Alto, CA, United States
| | - Julia A Leonard
- Department of Psychology, Yale University, New Haven, CT, United States
| | - Shari Liu
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, United States
| | - Megan Merrick
- Department of Psychological and Brain Sciences, Indiana University, Bloomington, IN, United States
| | - Samaher Radwan
- Department of Psychology, Stanford University, Palo Alto, CA, United States
| | - Jessa Stegall
- Department of Psychology, Stanford University, Palo Alto, CA, United States
| | - Natalia Velez
- Department of Psychology, Harvard University, Cambridge, MA, United States
| | - Brandon Woo
- Department of Psychology, Harvard University, Cambridge, MA, United States
| | - Yang Wu
- Department of Psychology, Stanford University, Palo Alto, CA, United States
| | - Xi J Zhou
- Department of Psychology, Stanford University, Palo Alto, CA, United States
| | - Michael C Frank
- Department of Psychology, Stanford University, Palo Alto, CA, United States
| | - Hyowon Gweon
- Department of Psychology, Stanford University, Palo Alto, CA, United States
| |
Collapse
|
27
|
Lawlor J, Thomas C, Guhin AT, Kenyon K, Lerner MD, Drahota A. Suspicious and fraudulent online survey participation: Introducing the REAL framework. METHODOLOGICAL INNOVATIONS 2021. [DOI: 10.1177/20597991211050467] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Online survey research has significantly increased in popularity in recent years. With its use, researchers have a new set of concerns about data collection and analysis to consider, including the possibility of fraudulent survey submissions. The purpose of this article is to demonstrate to survey researchers an innovative and systematized process for addressing online survey fraud over the course of collecting survey data, especially when respondents collect incentives for participation. We provide the Reflect, Expect, Analyze, Label Framework, which includes four sets of guiding questions for use by online survey researchers to plan for addressing survey fraud and making determinations about the inclusion or exclusion of participant submissions from the dataset based on level of suspicion. We also provide a full case example utilizing the Reflect, Expect, Analyze, Label Framework as an appendix. Those wanting to apply the Reflect, Expect, Analyze, Label Framework should keep in mind several considerations as they apply it, including determining logistical needs ahead of survey implementation, considering the ethical issues related to including or excluding data in a study, and considering the issues related to providing incentives for participating in research. Future research should assess the frequency of survey fraud, investigate the reasons for its occurrence and explore the role social networks may play in fraudulent participants sharing information. We suggest that researchers consider online survey fraud as an issue over the lifespan of their survey and apply the guiding questions we present to address the issue throughout.
Collapse
Affiliation(s)
- Jennifer Lawlor
- School of Information, University of Michigan, Ann Arbor, MI, USA
| | - Carl Thomas
- Department of Psychology, Michigan State University, East Lansing, MI, USA
| | - Andrew T Guhin
- Department of Psychology, Michigan State University, East Lansing, MI, USA
| | - Kendra Kenyon
- Department of Psychology, Michigan State University, East Lansing, MI, USA
| | - Matthew D Lerner
- Department of Psychology, Stony Brook University, Stony Brook, NY, USA
| | - Amy Drahota
- Department of Psychology, Michigan State University, East Lansing, MI, USA
| | | |
Collapse
|
28
|
Searching for meaning is associated with costly prosociality. PLoS One 2021; 16:e0258769. [PMID: 34695151 PMCID: PMC8544877 DOI: 10.1371/journal.pone.0258769] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2021] [Accepted: 10/05/2021] [Indexed: 01/07/2023] Open
Abstract
The study of meaning in life has largely centered on its relationship with personal well-being, while a focus on how meaning is related to enhancing the well-being of others has received less research attention. Although searching for meaning may imply lower personal well-being, we find that meaning-seekers are more motivated to perform costly prosocial actions for the sake of others’ well-being, given the perceived meaningfulness of these behaviors. Studies 1–4 (N = 780) show that meaning-seeking correlates with the motivation to engage in a range of costly prosocial behaviors. Meaning-seeking is further shown to be distinct from pursuing happiness in its relationship with costly prosociality (Study 2 & 3) and to share a stronger association with high-cost than low-cost prosociality (Study 3 & 4). Study 5 (N = 370; pre-registered) further shows that the search for meaning is related to costly prosocial behavior in the recent past. While our studies are cross-sectional, the pattern of findings suggests that seeking meaning (rather than happiness) may play an important role in motivating altruistic tendencies.
Collapse
|
29
|
Kovera MB, Evelo AJ. Eyewitness identification in its social context. JOURNAL OF APPLIED RESEARCH IN MEMORY AND COGNITION 2021. [DOI: 10.1016/j.jarmac.2021.04.003] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
30
|
Booker JA, Fivush R, Graci ME. Narrative identity informs psychological adjustment: Considering three themes captured across five time points and two event valences. J Pers 2021; 90:324-342. [PMID: 34411304 DOI: 10.1111/jopy.12668] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2021] [Revised: 07/26/2021] [Accepted: 08/02/2021] [Indexed: 11/26/2022]
Abstract
OBJECTIVES How narrative identity and well-being are intertwined as emerging adults process their lived experiences remains a critical theoretical and empirical question. We studied narrative identity among US emerging adults in a multiphase study. We aimed to test (1) if and how narrative identity themes (i.e., coherence, agency, growth) change rapidly across repeated narrations; (2) are related to reports of psychological adjustment (i.e., well-being, recent stress) over time; and (3) whether the valence of the autobiographical event nuanced the ways narrative identity and adjustment co-evolve. METHODS In a mini-longitudinal study conducted over three months, 300 adults aged 18-to-29-years (M age = 24.39 years; 60% women) provided autobiographical narratives about high-point and low-point (LP) life events at five time points, as well as repeated reports on well-being and recent stress. RESULTS Overall, coherence showed (1) the most consistency across time and valence than other narrative themes and (2) the most consistent associations with adjustment. In multilevel models, LP coherence and LP growth coincided with higher levels of adjustment. CONCLUSIONS Findings reinforce the ways narrative identity reflects dynamic processes of understanding the events of one's life, and the ways individual differences in framing and reasoning about life are important for psychological adjustment.
Collapse
Affiliation(s)
- Jordan A Booker
- Department of Psychological Sciences, University of Missouri, Columbia, Missouri, USA
| | - Robyn Fivush
- Institute for the Liberal Arts, Emory University, Atlanta, Georgia, USA
| | | |
Collapse
|
31
|
Wegener DT, Fabrigar LR, Pek J, Hoisington-Shaw K. Evaluating Research in Personality and Social Psychology: Considerations of Statistical Power and Concerns About False Findings. PERSONALITY AND SOCIAL PSYCHOLOGY BULLETIN 2021; 48:1105-1117. [PMID: 34308722 DOI: 10.1177/01461672211030811] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/20/2023]
Abstract
Traditionally, statistical power was viewed as relevant to research planning but not evaluation of completed research. However, following discussions of high false finding rates (FFRs) associated with low statistical power, the assumed level of statistical power has become a key criterion for research acceptability. Yet, the links between power and false findings are not as straightforward as described. Assumptions underlying FFR calculations do not reflect research realities in personality and social psychology. Even granting the assumptions, the FFR calculations identify important limitations to any general influences of statistical power. Limits for statistical power in inflating false findings can also be illustrated through the use of FFR calculations to (a) update beliefs about the null or alternative hypothesis and (b) assess the relative support for the null versus alternative hypothesis when evaluating a set of studies. Taken together, statistical power should be de-emphasized in comparison to current uses in research evaluation.
Collapse
Affiliation(s)
| | | | - Jolynn Pek
- The Ohio State University, Columbus, USA
| | | |
Collapse
|
32
|
Pink AE, Cheon BK. Development of a Simplified Portion Size Selection Task. Foods 2021; 10:foods10051121. [PMID: 34070072 PMCID: PMC8158092 DOI: 10.3390/foods10051121] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2021] [Revised: 05/11/2021] [Accepted: 05/12/2021] [Indexed: 12/19/2022] Open
Abstract
Portion size is an important determinant of energy intake and the development of easy to use and valid tools for measuring portion size are required. Standard measures, such as ad libitum designs and currently available computerized portion selection tasks (PSTs), have several limitations including only being able to capture responses to a limited number of foods, requiring participants’ physical presence and logistical/technical demands. The objective of the current study was to develop and test robust and valid measures of portion size that can be readily prepared by researchers and be reliably utilized for remote online data collection. We developed and tested two simplified PSTs that could be utilized online: (1) portion size images presented simultaneously along a horizontal continuum slider and (2) multiple-choice images presented vertically. One hundred and fifty participants (M = 21.35 years old) completed both simplified PSTs, a standard computerized PST and a series of questionnaires of variables associated with portion size (e.g., hunger, food item characteristics, Three Factor Eating Questionnaire). We found average liking of foods was a significant predictor of all three tasks and cognitive restraint also predicted the two simplified PSTs. We also found significant agreement between the standard PST and estimated portion sizes derived from the simplified PSTs when accounting for average liking. Overall, we show that simplified versions of the standard PST can be used online as an analogue of estimating ideal portion size.
Collapse
Affiliation(s)
- Aimee E. Pink
- School of Social Sciences, Nanyang Technological University, Singapore 639818, Singapore;
- Department of Psychology, Swansea University, Swansea SA2 8PP, UK
| | - Bobby K. Cheon
- School of Social Sciences, Nanyang Technological University, Singapore 639818, Singapore;
- Singapore Institute for Clinical Sciences, A*STAR, Singapore 117599, Singapore
- Correspondence: ; Tel.: +65-6592-1570
| |
Collapse
|
33
|
Bezouw MJ, Toorn J, Becker JC. Social creativity: Reviving a social identity approach to social stability. EUROPEAN JOURNAL OF SOCIAL PSYCHOLOGY 2021. [DOI: 10.1002/ejsp.2732] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Affiliation(s)
| | - Jojanneke Toorn
- Utrecht University Utrecht the Netherlands
- Leiden University Leiden the Netherlands
| | | |
Collapse
|
34
|
Barnett MA, Jones TL, Schmitt KA, Cordas KC, Harrod S. Adults' attitudes toward children, adolescents, and adults who are obese and tax-funded programs to combat obesity. The Journal of Social Psychology 2021; 161:379-393. [PMID: 33198607 DOI: 10.1080/00224545.2020.1845114] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
Abstract
The goals of the present study were to assess (1) adults' patterns of beliefs about and attitudes toward children, adolescents, and adults who are obese and (2) their attitudes toward hypothetical tax-funded programs designed to combat obesity in children, adolescents, and adults. A total of 267 participants, ranging in age from 19 to 88 years old, were recruited through Amazon's Mechanical Turk (MTurk) to participate online in the present study. The participants rated adults who are obese less favorably, and as less malleable, than children or adolescents who are obese. Furthermore, they were less supportive of tax-funded programs designed to combat obesity in adults than children or adolescents. In general, the participants' relatively unfavorable response to adults who are obese appears to be associated with the beliefs that older individuals who are obese are relatively unchangeable and have heightened personal fault for their plight.
Collapse
Affiliation(s)
- Mark A Barnett
- Department of Psychological Sciences, Kansas State University, Manhattan, KS, USA
| | - Tucker L Jones
- Department of Psychological Sciences, Kansas State University, Manhattan, KS, USA
| | - Kennedy A Schmitt
- Department of Psychological Sciences, Kansas State University, Manhattan, KS, USA
| | - Kyla C Cordas
- Department of Psychological Sciences, Kansas State University, Manhattan, KS, USA
| | - Sarah Harrod
- Department of Psychological Sciences, Kansas State University, Manhattan, KS, USA
| |
Collapse
|
35
|
Tam K, Leung AK, Clayton S. Research on climate change in social psychology publications: A systematic review. ASIAN JOURNAL OF SOCIAL PSYCHOLOGY 2021. [DOI: 10.1111/ajsp.12477] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/28/2022]
Affiliation(s)
- Kim‐Pong Tam
- The Hong Kong University of Science and Technology Hong Kong China
| | | | | |
Collapse
|
36
|
Entringer TM, Gebauer JE, Paulhus DL. Extracting Agency and Communion From the Big Five: A Four-Way Competition. Assessment 2021; 29:1216-1235. [PMID: 33813905 PMCID: PMC9301169 DOI: 10.1177/10731911211003978] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Agency and communion are the two fundamental content dimensions in psychology. The two dimensions figure prominently in many psychological realms (personality, social, self, motivational, cross-cultural, etc.). In contemporary research, however, personality is most commonly measured within the Big Five framework. We developed novel agency and communion scales based on the items from the most popular nonpropriety measure of the Big Five-the Big Five Inventory. We compared four alternative scale-construction methods: expert rating, target scale, ant colony, and brute force. Across three samples (Ntotal = 942), all methods yielded reliable and valid agency and communion scales. Our research provides two main contributions. For psychometric theory, it extends knowledge on the four scale-construction methods and their relative convergence. For psychometric practice, it enables researchers to examine agency and communion hypotheses with extant Big Five Inventory data sets, including those collected in their own labs as well as openly accessible, large-scale data sets.
Collapse
Affiliation(s)
- Theresa M Entringer
- German Institute for Economic Research, Berlin, Germany.,University of Mannheim, Mannheim, Baden-Württemberg, Germany
| | - Jochen E Gebauer
- University of Mannheim, Mannheim, Baden-Württemberg, Germany.,University of Copenhagen, Copenhagen, Denmark
| | - Delroy L Paulhus
- University of British Columbia, Vancouver, British Columbia, Canada
| |
Collapse
|
37
|
Fabrigar LR, Wegener DT, Petty RE. A Validity-Based Framework for Understanding Replication in Psychology. PERSONALITY AND SOCIAL PSYCHOLOGY REVIEW 2020; 24:316-344. [DOI: 10.1177/1088868320931366] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
In recent years, psychology has wrestled with the broader implications of disappointing rates of replication of previously demonstrated effects. This article proposes that many aspects of this pattern of results can be understood within the classic framework of four proposed forms of validity: statistical conclusion validity, internal validity, construct validity, and external validity. The article explains the conceptual logic for how differences in each type of validity across an original study and a subsequent replication attempt can lead to replication “failure.” Existing themes in the replication literature related to each type of validity are also highlighted. Furthermore, empirical evidence is considered for the role of each type of validity in non-replication. The article concludes with a discussion of broader implications of this classic validity framework for improving replication rates in psychological research.
Collapse
|
38
|
Field experiment in psychology. Is there still room for conducting research in natural conditions? HEALTH PSYCHOLOGY REPORT 2020. [DOI: 10.5114/hpr.2020.97159] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022] Open
Abstract
BackgroundThis paper reflects on the status of the field experiment in contemporary psychology (especially its social branch). The subject of analysis is how frequently this method is used, especially in the context of the growing popularity of on-line research.Participants and procedureThe empirical part of the article presents an experiment revealing the uniqueness of the field experiment as a method allowing for the identification of subtle variables that can have a real impact on the results obtained. In the whole study 128 people were examined (half of the group were women). The obtained results show that some phenomena cannot be examined in any other way than by means of the field experiment.ResultsThe process of moving away from the study of real behaviours in the real environment, observed over the last twenty years, has not stopped since the criticism of this phenomenon expressed about 20 years ago. On the contrary, it has intensified. The phenomenon described in the article is definitely harmful to the scientific disci-pline so strongly connected with the social context of research, namely social psychology.ConclusionsTime will show whether the increasingly loud voices criticising this trend will translate into an actual change of methods used by social psychologists.
Collapse
|
39
|
Krpan D. Unburdening the Shoulders of Giants: A Quest for Disconnected Academic Psychology. PERSPECTIVES ON PSYCHOLOGICAL SCIENCE 2020; 15:1042-1053. [PMID: 32369707 PMCID: PMC7370646 DOI: 10.1177/1745691620904775] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Abstract
In current academic psychology, scholars typically develop their research and ideas by drawing on the work of other contemporary and preceding psychological scientists and by following certain conventions of the field. I refer to this variant of psychology as connected because the emphasis is on connecting various research findings and ideas generated by different scholars (e.g., by showing how they are related to each other via referencing). In this article, I argue that, although connected psychology advances psychological knowledge, it restricts the total amount of knowledge that could eventually be produced and therefore limits the potential of the discipline to improve the understanding of psychological phenomena. As a solution, I propose that, alongside the currently existing connected psychology, disconnected psychology should be established. In disconnected psychology, researchers develop their ideas by following the main principles of psychological method, but they are disconnected from a “field” consisting of other psychologists and therefore do not follow the discipline’s norms and conventions. By drawing on one of the core constructs from information theory—information entropy—I argue that combining the two streams of psychology would result in the most significant advancement of psychological knowledge.
Collapse
Affiliation(s)
- Dario Krpan
- Department of Psychological and Behavioural Science, London School of Economics and Political Science
| |
Collapse
|
40
|
Plessen CY, Franken FR, Ster C, Schmid RR, Wolfmayr C, Mayer AM, Sobisch M, Kathofer M, Rattner K, Kotlyar E, Maierwieser RJ, Tran US. Humor styles and personality: A systematic review and meta-analysis on the relations between humor styles and the Big Five personality traits. PERSONALITY AND INDIVIDUAL DIFFERENCES 2020. [DOI: 10.1016/j.paid.2019.109676] [Citation(s) in RCA: 20] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/25/2022]
|
41
|
Gaither SE, Chen JM, Rule N. Perceptions and experiences of (people with) unconventional identities. SELF AND IDENTITY 2020. [DOI: 10.1080/15298868.2020.1711802] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
Affiliation(s)
- Sarah E. Gaither
- Department of Psychology and Neuroscience, Samuel DuBois Cook Center on Social Equity, Center on Health and Society, Duke University, Durham, USA
| | | | - Nicholas Rule
- Department of Psychology, University of Toronto, Toronto, Canada
| |
Collapse
|
42
|
Greitemeyer T. The contagious impact of playing violent video games on aggression: Longitudinal evidence. Aggress Behav 2019; 45:635-642. [PMID: 31410869 PMCID: PMC6790614 DOI: 10.1002/ab.21857] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2019] [Revised: 06/07/2019] [Accepted: 06/12/2019] [Indexed: 11/09/2022]
Abstract
Meta-analyses have shown that violent video game play increases aggression in the player. The present research suggests that violent video game play also affects individuals with whom the player is connected. A longitudinal study (N = 980) asked participants to report on their amount of violent video game play and level of aggression as well as how they perceive their friends and examined the association between the participant's aggression and their friends' amount of violent video game play. As hypothesized, friends' amount of violent video game play at Time 1 was associated with the participant's aggression at Time 2 even when controlling for the impact of the participant's aggression at Time 1. Mediation analyses showed that friends' aggression at Time 1 accounted for the impact of friends' amount of violent video game play at Time 1 on the participant's aggression at Time 2. These findings suggest that increased aggression in video game players has an impact on the player's social network.
Collapse
|
43
|
Sassenberg K, Ditrich L. Research in Social Psychology Changed Between 2011 and 2016: Larger Sample Sizes, More Self-Report Measures, and More Online Studies. ADVANCES IN METHODS AND PRACTICES IN PSYCHOLOGICAL SCIENCE 2019. [DOI: 10.1177/2515245919838781] [Citation(s) in RCA: 66] [Impact Index Per Article: 13.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/17/2022]
Abstract
The debate about false positives in psychological research has led to a demand for higher statistical power. To meet this demand, researchers need to collect data from larger samples—which is important to increase replicability, but can be costly in both time and money (i.e., remuneration of participants). Given that researchers might need to compensate for these higher costs, we hypothesized that larger sample sizes might have been accompanied by more frequent use of less costly research methods (i.e., online data collection and self-report measures). To test this idea, we analyzed social psychology studies published in 2009, 2011, 2016, and 2018. Indeed, research reported in 2016 and 2018 (vs. 2009 and 2011) had larger sample sizes and relied more on online data collection and self-report measures. Thus, over these years, research improved in its statistical power, but also changed with regard to the methods applied. Implications for social psychology as a discipline are discussed.
Collapse
Affiliation(s)
- Kai Sassenberg
- Leibniz-Institut für Wissensmedien (Knowledge Media Research Center), Tübingen, Germany
- School of Science, University of Tübingen
| | - Lara Ditrich
- Leibniz-Institut für Wissensmedien (Knowledge Media Research Center), Tübingen, Germany
| |
Collapse
|
44
|
MTurk Participants Have Substantially Lower Evaluative Subjective Well-Being Than Other Survey Participants. COMPUTERS IN HUMAN BEHAVIOR 2019; 94:1-8. [PMID: 30880871 DOI: 10.1016/j.chb.2018.12.042] [Citation(s) in RCA: 16] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022]
Abstract
Amazon's MTurk platform has become a popular site for obtaining relatively inexpensive and convenient adult samples for use in behavioral research. Concerns have been raised about selection issues, because MTurk workers chose to participate in the platform and select the tasks they perform (of many offered to them). Prior studies have documented demographic and psychological differences with national samples. In this paper we studied evaluative subjective well-being (the Cantril Ladder) in an MTurk sample, a national Internet panel sample, and a national telephone survey conducted by Gallup-Sharecare. A surprising finding was that MTurk participants' Ladder scores were substantial lower than the other two samples. Analyses controlling for six demographic differences among the samples only slightly reduced the mean differences. However, patterns of demographic-well-being associations were similar within the samples. To corroborate these results, we conducted a secondary analysis on another three samples, one MTurk sample and two Internet panel samples. The same group differences in Ladder scores were observed. These findings add to the growing literature documenting the characteristics of MTurk samples and we discuss the implications for future research with such samples.
Collapse
|