1
|
Axt J, Buttrick N, Feng RY. A Comparative Investigation of the Predictive Validity of Four Indirect Measures of Bias and Prejudice. PERSONALITY AND SOCIAL PSYCHOLOGY BULLETIN 2024; 50:871-888. [PMID: 36660861 PMCID: PMC11080383 DOI: 10.1177/01461672221150229] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2022] [Accepted: 12/20/2022] [Indexed: 01/21/2023]
Abstract
Although measures of implicit associations are influential in the prejudice literature, comparative tests of the predictive power of these measures are lacking. A large-scale (N > 100,000) analysis of four commonly used measures-the Implicit Association Test (IAT), Single-Category IAT (SC-IAT), evaluative priming task (EPT), and sorting paired features task (SPF)-across 10 intergroup domains and 250 outcomes found clear evidence for the superiority of the SC-IAT in predictive and incremental predictive validity. Follow-up analyses suggested that the SC-IAT benefited from an exclusive focus on associations toward stigmatized group members, as associations toward non-stigmatized group members diluted the predictive strength of relative measures like the IAT, SPF, and EPT. These results highlight how conclusions about predictive validity can vary drastically depending on the measure selected and reveal novel insights about the value of different measures when focusing on predictive than convergent validity.
Collapse
Affiliation(s)
- Jordan Axt
- McGill University, Montreal, Quebec, Canada
- Project Implicit, Seattle, Washington, USA
| | | | | |
Collapse
|
2
|
Kvam PD, Irving LH, Sokratous K, Smith CT. Improving the reliability and validity of the IAT with a dynamic model driven by similarity. Behav Res Methods 2024; 56:2158-2193. [PMID: 37450219 DOI: 10.3758/s13428-023-02141-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/02/2023] [Indexed: 07/18/2023]
Abstract
The Implicit Association Test (IAT), like many behavioral measures, seeks to quantify meaningful individual differences in cognitive processes that are difficult to assess with approaches like self-reports. However, much like other behavioral measures, many IATs appear to show low test-retest reliability and typical scoring methods fail to quantify all of the decision-making processes that generate the overt task performance. Here, we develop a new modeling approach for IATs based on the geometric similarity representation (GSR) model. This model leverages both response times and accuracy on IATs to make inferences about representational similarity between the stimuli and categories. The model disentangles processes related to response caution, stimulus encoding, similarities between concepts and categories, and response processes unrelated to the choice itself. This approach to analyzing IAT data illustrates that the unreliability in IATs is almost entirely attributable to the methods used to analyze data from the task: GSR model parameters show test-retest reliability around .80-.90, on par with reliable self-report measures. Furthermore, we demonstrate how model parameters result in greater validity compared to the IAT D-score, Quad model, and simple diffusion model contrasts, predicting outcomes related to intergroup contact and motivation. Finally, we present a simple point-and-click software tool for fitting the model, which uses a pre-trained neural network to estimate best-fit parameters of the GSR model. This approach allows easy and instantaneous fitting of IAT data with minimal demands on coding or technical expertise on the part of the user, making the new model accessible and effective.
Collapse
Affiliation(s)
- Peter D Kvam
- Department of Psychology, University of Florida, Florida, USA.
| | - Louis H Irving
- Department of Psychology, University of Florida, Florida, USA
| | | | | |
Collapse
|
3
|
Guglielmi V, Capoccia D, Russo B, Lubrano C, Mariani S, Poggiogalle E, Furia G, Alizadeh AH, Patrizi C, Sapienza M, Damiani G, Tarsitano MG, Conte C, Frontoni S. Knowledge, experiences, and perceptions relating to obesity management among primary care physicians in the Lazio Region, Italy. Front Endocrinol (Lausanne) 2023; 14:1249233. [PMID: 38027122 PMCID: PMC10668048 DOI: 10.3389/fendo.2023.1249233] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/28/2023] [Accepted: 09/25/2023] [Indexed: 12/01/2023] Open
Abstract
Background Primary care providers (PCPs) play an essential role in obesity care as they represent the first contact for patients seeking weight loss interventions. Objective This study explored the knowledge, experiences, and perceptions of PCPs in the Lazio Region of Italy in the management of obesity. Design and subjects We conducted an anonymous survey delivered from March to July 2022 via the newsletter of Rome Provincial Order of Physicians and Dentists and at the annual meeting of the regional section of the Italian Obesity Society. Approach The survey consisted of 24 closed-ended questions grouped into 5 sections: sociodemographic and work information; assessment of obesity; management of obesity; connections with regional Centres for Obesity Management; attitudes towards obesity. Key results A total of 92 PCPs accessed the survey. Of those, 2.2% were excluded because they did not see any patients with obesity. A total of 68 PCPs (75.6%) had complete questionnaires and were included in this analysis. All participants reported asking their patients about their eating habits, lifestyle, and clinical complications at the first assessment. Body weight and blood pressure were measured by 98.5% of participants and 82% calculate body mass index (BMI), while a small proportion of PCPs analysed body composition and fat distribution. Over 80% prescribed laboratory tests and ECG. Approximately 40% of PCPs did not refer patients for nutritional counselling, and most prescribed a low-calorie diet. Sixty-three percent referred patients to an endocrinologist, 48.5% to a psychotherapist, and a minority to specialists for obesity complications. Twenty-three percent prescribed anti-obesity medications and 46.5% referred patients for bariatric surgery only in severe cases. Ninety-one percent stated that obesity is "a complex and multifactorial disease" and 7.4% considered obesity to be secondary to other conditions. Conclusions Despite most PCPs adopt a correct approach to manage patients with obesity, many aspects could be improved to ensure optimal and multidisciplinary management.
Collapse
Affiliation(s)
- Valeria Guglielmi
- Department of Systems Medicine, University of Rome Tor Vergata, Rome, Italy
- Internal Medicine Unit - Obesity Center, University Hospital Policlinico Tor Vergata, Rome, Italy
- Italian Obesity Society (SIO), Pisa, Italy
| | - Danila Capoccia
- Italian Obesity Society (SIO), Pisa, Italy
- Department of Medical Surgical Sciences and Biotechnologies, Sapienza University of Rome, Santa Maria Goretti Hospital, Latina, Italy
| | - Benedetta Russo
- Department of Systems Medicine, University of Rome Tor Vergata, Rome, Italy
- Italian Obesity Society (SIO), Pisa, Italy
- Unit of Endocrinology, Diabetes and Metabolism, Fatebenefratelli Gemelli Isola Hospital, Rome, Italy
| | - Carla Lubrano
- Italian Obesity Society (SIO), Pisa, Italy
- Department of Experimental Medicine, Section of Medical Pathophysiology, Food Science and Endocrinology, Policlinico Umberto I, Sapienza University of Rome, Rome, Italy
| | - Stefania Mariani
- Italian Obesity Society (SIO), Pisa, Italy
- Department of Experimental Medicine, Section of Medical Pathophysiology, Food Science and Endocrinology, Policlinico Umberto I, Sapienza University of Rome, Rome, Italy
| | - Eleonora Poggiogalle
- Italian Obesity Society (SIO), Pisa, Italy
- Department of Experimental Medicine, Section of Medical Pathophysiology, Food Science and Endocrinology, Policlinico Umberto I, Sapienza University of Rome, Rome, Italy
| | - Giuseppe Furia
- Directive Council of Order of Physicians and Dentists of the Province of Rome, Rome, Italy
- Local Health Authority Roma 1, Hospital Management Area, Rome, Italy
| | - Aurora Heidar Alizadeh
- Directive Council of Order of Physicians and Dentists of the Province of Rome, Rome, Italy
- Department of Life Sciences and Public Health, Università Cattolica del Sacro Cuore, Rome, Italy
| | - Cristina Patrizi
- Directive Council of Order of Physicians and Dentists of the Province of Rome, Rome, Italy
| | - Martina Sapienza
- Directive Council of Order of Physicians and Dentists of the Province of Rome, Rome, Italy
- Department of Life Sciences and Public Health, Università Cattolica del Sacro Cuore, Rome, Italy
| | - Gianfranco Damiani
- Directive Council of Order of Physicians and Dentists of the Province of Rome, Rome, Italy
- Department of Life Sciences and Public Health, Università Cattolica del Sacro Cuore, Rome, Italy
- Department of Woman and Child Health and Public Health, Fondazione Policlinico Universitario “A. Gemelli” Istituto di Ricovero e Cura a Carattere Scientifico (IRCCS), Rome, Italy
| | - Maria Grazia Tarsitano
- Directive Council of Order of Physicians and Dentists of the Province of Rome, Rome, Italy
- Department of Medical and Surgical Science, University Magna Grecia, Catanzaro, Italy
| | - Caterina Conte
- Italian Obesity Society (SIO), Pisa, Italy
- Department of Human Sciences and Promotion of the Quality of Life, San Raffaele Roma Open University, Rome, Italy
- Department of Endocrinology, Nutrition and Metabolic Diseases, Istituto di Ricovero e Cura a Carattere Scientifico (IRCCS) MultiMedica, Sesto San Giovanni, Italy
| | - Simona Frontoni
- Department of Systems Medicine, University of Rome Tor Vergata, Rome, Italy
- Italian Obesity Society (SIO), Pisa, Italy
- Unit of Endocrinology, Diabetes and Metabolism, Fatebenefratelli Gemelli Isola Hospital, Rome, Italy
| |
Collapse
|
4
|
Carpenter TP, Goedderz A, Lai CK. Individual Differences in Implicit Bias Can Be Measured Reliably by Administering the Same Implicit Association Test Multiple Times. PERSONALITY AND SOCIAL PSYCHOLOGY BULLETIN 2023; 49:1363-1378. [PMID: 35751174 DOI: 10.1177/01461672221099372] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
The use of the Implicit Association Test (IAT) as a measure of individual differences is stymied by insufficient test-retest reliability for assessing trait-level constructs. We assess the degree to which the IAT measures individual differences and test a method to improve its validity as a "trait" measure: aggregating across IATs. Across three studies, participants (total n = 960) completed multiple IATs in the same session or across multiple sessions. Using latent-variable models, we found that half of the variance in IAT scores reflects individual differences. Aggregating across multiple IATs approximately doubled the variance explained with explicit measures compared with a single IAT D-score. These findings show that IAT scores contain considerable noise and that a single IAT is inadequate to estimate trait bias. However, aggregation across multiple administrations can correct this and better estimate individual differences in implicit attitudes.
Collapse
|
6
|
Vrantsidis TH, Lombrozo T. Simplicity as a Cue to Probability: Multiple Roles for Simplicity in Evaluating Explanations. Cogn Sci 2022; 46:e13169. [PMID: 35738485 DOI: 10.1111/cogs.13169] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2022] [Revised: 05/20/2022] [Accepted: 05/26/2022] [Indexed: 11/28/2022]
Abstract
People often face the challenge of evaluating competing explanations. One approach is to assess the explanations' relative probabilities-for example, applying Bayesian inference to compute their posterior probabilities. Another approach is to consider an explanation's qualities or "virtues," such as its relative simplicity (i.e., the number of unexplained causes it invokes). The current work investigates how these two approaches are related. Study 1 found that simplicity is used to infer the inputs to Bayesian inference (explanations' priors and likelihoods). Studies 1 and 2 found that simplicity is also used as a direct cue to the outputs of Bayesian inference (the posterior probability of an explanation), such that simplicity affects estimates of posterior probability even after controlling for elicited (Study 1) or provided (Study 2) priors and likelihoods, with simplicity having a larger effect in Study 1, where posteriors are more uncertain and difficult to compute. Comparing Studies 1 and 2 also suggested that simplicity plays additional roles unrelated to approximating probabilities, as reflected in simplicity's effect on how "satisfying" (vs. probable) an explanation is, which remained largely unaffected by the difficulty of computing posteriors. Together, these results suggest that the virtue of simplicity is used in multiple ways to approximate probabilities (i.e., serving as a cue to priors, likelihoods, and posteriors) when these probabilities are otherwise uncertain or difficult to compute, but that the influence of simplicity also goes beyond these roles.
Collapse
|
7
|
Science-utility and science-trust associations and how they relate to knowledge about how science works. PLoS One 2021; 16:e0260586. [PMID: 34914732 PMCID: PMC8675735 DOI: 10.1371/journal.pone.0260586] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2020] [Accepted: 11/12/2021] [Indexed: 11/22/2022] Open
Abstract
Knowledge about how science works, trust in scientists, and the perceived utility of science currently appear to be eroding in these times in which “alternative facts” or personal experiences and opinions are used as arguments. Yet, in many situations, it would be beneficial for the individual and all of society if scientific findings were considered in decision-making. For this to happen, people have to trust in scientists and perceive science as useful. Still, in university contexts, it might not be desirable to report negative beliefs about science. In addition, science-utility and science-trust associations may differ from explicit beliefs because associations were learned through the co-occurrence of stimuli rather than being based on propositional reasoning. We developed two IATs to measure science-utility and science-trust associations in university students and tested the psychometric properties and predictive potential of these measures. In a study of 261 university students, the IATs were found to have good psychometric properties and small correlations with their corresponding self-report scales. Science-utility and science-trust associations predicted knowledge about how science works over and above self-reported beliefs. The results suggest that indirect measures are useful for assessing beliefs about science and can be used to predict outcome measures.
Collapse
|
8
|
The good and the bad: Are some attribute words better than others in the Implicit Association Test? Behav Res Methods 2021; 53:2512-2527. [PMID: 33948922 DOI: 10.3758/s13428-021-01592-8] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/29/2021] [Indexed: 01/12/2023]
Abstract
The Implicit Association Test (IAT) is one of the most popular measures in psychological research. A lack of standardization across IATs has resulted in significant variability among stimuli used by researchers, including the positive and negative words used in evaluative IATs. Does the variability in attribute words in evaluative IATs produce unwanted variability in measurement quality across studies? The present work investigated the effect of evaluative stimuli across three studies using 13 IATs and over 60,000 participants. The 64 positive and negative words that we tested provided similar measurement quality. Further, measurement was satisfactory even in IATs that used only category labels as stimuli. These results suggest that common sense is probably a sufficient method for selection of evaluative stimuli in the IAT. For reasonable measurement quality, we recommend that researchers using evaluative IATs in English select words randomly from the set we tested in the present research.
Collapse
|
10
|
Wang X, Wang J. Information Matching: How Regulatory Focus Affects Information Preference and Information Choice. Front Psychol 2021; 12:618537. [PMID: 34122215 PMCID: PMC8192962 DOI: 10.3389/fpsyg.2021.618537] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2020] [Accepted: 04/30/2021] [Indexed: 11/24/2022] Open
Abstract
Individuals often prefer information that matches their needs. In this study, we aimed to explore the relationship between regulatory focus and information preference. Specifically, we investigated the effects of promotion-focused information and prevention-focused information on explicit and implicit information preferences and choice behavior, and examined the mediating roles of information preference. In Experiment 1, we found that prevention-focused individuals were more likely to choose functional information, whereas promotion-focused people were more likely to choose hedonic information. However, there was no significant relationship between regulatory focus and explicit preference and no mediating effect of explicit information preference. In Experiment 2, we found that promotion-focused individuals had a greater implicit preference for hedonic information than did prevention-focused individuals. Implicit information preference mediated the influence of regulatory focus on information choice. The findings of this study may help us understand the psychological mechanism underlying information preference and have important implications for information dissemination.
Collapse
Affiliation(s)
- Xiaomei Wang
- School of Media Studies and Humanities, Zhejiang University City College, Hangzhou, China
| | - Jia Wang
- School of Media Studies and Humanities, Zhejiang University City College, Hangzhou, China
| |
Collapse
|
11
|
Machery E. Anomalies in implicit attitudes research. WILEY INTERDISCIPLINARY REVIEWS. COGNITIVE SCIENCE 2021; 13:e1569. [PMID: 34130361 DOI: 10.1002/wcs.1569] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/10/2020] [Revised: 04/27/2021] [Accepted: 05/09/2021] [Indexed: 11/06/2022]
Abstract
In this review, I provide a pessimistic assessment of the indirect measurement of attitudes by highlighting the persisting anomalies in the science of implicit attitudes, focusing on their validity, reliability, predictive power, and causal efficiency, and I draw some conclusions concerning the validity of the implicit bias construct. This article is categorized under: Psychology > Reasoning and Decision Making.
Collapse
Affiliation(s)
- Edouard Machery
- Center for Philosophy of Science, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| |
Collapse
|
12
|
Scaife R, Stafford T, Bunge A, Holroyd J. To Blame? The Effects of Moralized Feedback on Implicit Racial Bias. COLLABRA: PSYCHOLOGY 2020. [DOI: 10.1525/collabra.251] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022] Open
Abstract
Implicit bias training (IBT) is now frequently provided by employers, in order to raise awareness of the problems related to implicit biases, and of how to safeguard against discrimination that may result. However, as Atewologun et al. (2018) have noted, there is very little systematicity in IBT, and there are many unknowns about what constitutes good IBT. One important issue concerns the tone of information provided regarding implicit bias. This paper engages this question, focusing in particular on the observation that much bias training is delivered in exculpatory tone, emphasising that individuals are not to blame for possessing implicit biases. Normative guidance around IBT exhorts practitioners to adopt this strategy (Moss-Racusin et al. 2014). However, existing evidence about the effects of moralized feedback about implicit bias is equivocal (Legault et al. 2011; Czopp et al. 2006). Through a series of studies, culminating in an experiment with a pre-registered analysis plan, we develop a paradigm for evaluating the impact of moralized feedback on participants’ implicit racial bias scores. We also conducted exploratory analyses of the impact on their moods, and behavioural intentions. Our results indicated that an exculpatory tone, rather than a blaming or neutral tone, did not make participants less resistant to changing their attitudes and behaviours. In fact, participants in the blame condition had significantly stronger explicit intentions to change future behaviour than those in the ‘no feedback’ condition (see experiment 3). These results indicate that considerations of efficacy do not support the need for implicit bias feedback to be exculpatory. We tease out the implications of these findings, and directions for future research.
Collapse
|