1
|
Tian K, Ren Y, Chang Y, Chen Z, Yang X. Influence of respondents' Differentiation of subjective response on water knowledge stock test scale: Evaluation based on two-parameter-multidimensional IRT model. ENVIRONMENTAL RESEARCH 2023; 238:117181. [PMID: 37742755 DOI: 10.1016/j.envres.2023.117181] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/31/2023] [Revised: 09/07/2023] [Accepted: 09/20/2023] [Indexed: 09/26/2023]
Abstract
Insufficient awareness of water issues is a crucial bottleneck restricting the sustainable utilization of water resources. To accurately measure citizens' water knowledge stock and overcome the differences between scales and respondents' characteristic levels on test results, the research focuses on developing and evaluating water knowledge stock test scales. The mechanism for identifying indicators is designed based on the grounded theory, and as a result, the water knowledge stock test indicator system is derived. The data was collected by the form of survey questionnaire developed with the test indicator system. A two-parameter multidimensional item response theoretical model is constructed based on item parameter estimation, data model fitting, and item information function. The survey data and optimization model are used to optimize the water knowledge stock test scale and verify the fitting degree with the characteristics of the respondents. The test information function and standard error function indicate that the scale is most informative for individuals with characteristic levels ranging from -2 to 3, resulting in a highly reliable test effect. The research has established a measurement indicator system, methodology, and presented results that serve as a foundation for measuring the stock of water knowledge.
Collapse
Affiliation(s)
- Kang Tian
- College of Information and Management Science, Henan Agricultural University, No. 218, Ping'an Avenue, Zhengzhou, 450046, PR China; Citizen's Water Literacy Research Center, North China University of Water Resources and Electric Power, No.136, Jinshui East Road, Zhengzhou, 450046, PR China.
| | - Yunlong Ren
- School of Engineering, University of Manchester, Oxford Road, Manchester. M13 9PL, UK
| | - Yuanbo Chang
- Trade Union Committee, Henan University of Economics and Law, No.180, Jinshui East Road, Zhengzhou, 450046, PR China
| | - Zhen Chen
- College of Information and Management Science, Henan Agricultural University, No. 218, Ping'an Avenue, Zhengzhou, 450046, PR China
| | - Xue Yang
- School of Management and Economics, North China University of Water Resources and Electric Power, No.136, Jinshui East Road, Zhengzhou, 450046, PR China.
| |
Collapse
|
2
|
Taylor JR, Ivanoff J. The Active Suppression of a Distractor's Location Can Be Elusive. Exp Psychol 2023; 70:119-134. [PMID: 37534439 DOI: 10.1027/1618-3169/a000583] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/04/2023]
Abstract
Our visual system is inundated with distracting objects that vie for our attention. While visual attention selects relevant information, inhibitory mechanisms might be useful to suppress the locations occupied by irrelevant distractors. Yet, there is a dearth of behavioral evidence for the active suppression of a distractor's location (ASDL) using central cues that provide preliminary information about a distractor's location. In the first two experiments, we attempt to conceptually replicate, using an online platform, experiments that provide evidence of the ASDL. We replicate the distractor cueing effect in a localization task (Experiment 1) wherein responses to targets were faster when a central arrow cued the location of an impending distractor than an empty location. This effect was larger in the first block of trials than it was in the second. In a discrimination task (Experiment 2), unlike previous studies, we found no evidence for an effect of distractor cueing. In Experiment 3, we replaced the central arrow cues with central number cues because arrow cues may elicit a symbolic shift of attention that might offset the ASDL. Once again, the best model was one in which the distractor cueing effect was absent. We replicate these failures to find evidence of the ASDL in two more experiments. The results suggest that the ASDL can be elusive and may be tied to the response system, not attention.
Collapse
Affiliation(s)
- Jillian R Taylor
- Department of Psychology, Saint Mary's University, Halifax, NS, Canada
| | - Jason Ivanoff
- Department of Psychology, Saint Mary's University, Halifax, NS, Canada
| |
Collapse
|
3
|
From Lab-Testing to Web-Testing in Cognitive Research: Who You Test is More Important than how You Test. J Cogn 2023; 6:13. [PMID: 36721797 PMCID: PMC9854315 DOI: 10.5334/joc.259] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2022] [Accepted: 12/28/2022] [Indexed: 01/20/2023] Open
Abstract
The transition to web-testing, although promising, entails many new concerns. Web-testing is harder to monitor, so researchers need to ensure that the quality of the data collected is comparable to the quality of data typically achieved by lab-testing. Our study yields a novel contribution to this issue, by being the first to distinguish between the impact of web-testing and the impact of sourcing individuals from different participant pools, including crowdsourcing platforms. We presented a fairly general working memory task to 196 MTurk participants, 300 Prolific participants, and 255 students from the University of Geneva, allowing for a comparison of data quality across different participant pools. Among university students, 215 were web-tested, and 40 were lab-tested, allowing for a comparison of testing modalities within the same participant pool. Data quality was measured by assessing multiple data characteristics (i.e., reaction time, accuracy, anomalous values) and the presence of two behavioral benchmark effects. Our results revealed that who you test (i.e., participant pool) is more important than how you test (i.e., testing modality). Concerning how you test, our results showed that web-testing incurs a small, yet acceptable loss of data quality compared to lab-testing. Concerning who you test, Prolific participants were almost indistinguishable from web-tested students, but MTurk participants differed drastically from the other pools. Our results therefore encourage the use of web-testing in the domain of cognitive psychology, even when using complex paradigms. Nevertheless, these results urge for caution regarding how researchers select web-based participant pools when conducting online research.
Collapse
|
4
|
Cheung HN, Williams JM, Chan YS, Chan SWY. Measurement invariance of the Multidimensional Depression Assessment Scale (MDAS) across gender and ethnic groups of Asian, Caucasian, Black, and Hispanic. J Affect Disord 2022; 308:221-228. [PMID: 35429539 DOI: 10.1016/j.jad.2022.04.035] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/24/2021] [Revised: 03/05/2022] [Accepted: 04/09/2022] [Indexed: 10/18/2022]
Abstract
BACKGROUND Major Depressive Disorder is a severe and highly disabling mental illness. Almost all self-reported questionnaires have overlooked the interpersonal symptoms of depression which are important across gender and culture. The Multidimensional Depression Assessment Scale (MDAS) developed by Cheung and Power (2012) entails comprehensive emotional, cognitive, somatic, and interpersonal subscales. It addresses the criticism that existing self-report depression scales might not cover sufficient phenomenological heterogeneity of depression. The current study aimed to evaluate the psychometric properties of the MDAS across gender and four major ethnic groups of Caucasian, Black, Asian and Hispanic, including reliability and concurrent validity against the Centre for Epidemiological Studies Depression Scale (CESD) and Patients Health Questionnaire (PHQ-9). It also aimed to establish a stable factor structure across gender and ethnic groups and test the measurement invariance to enhance its potential for clinical use. METHODS A community sample of 3499 participants from four ethnic groups were recruited via online crowdsourcing sites of Qualtrics and Amazon M Turk. Each individual completed a demographic questionnaire, the MDAS, CESD and PHQ-9. RESULTS There was good internal consistency (Cronbach's alpha >0.90) and concurrent reliability across gender and ethnic groups. Strict measurement invariance was established for MDAS over a four-factor factor structure corresponding to the four subscales. CONCLUSIONS The MDAS showed good psychometric properties and measurement invariance of a four-factor structure, suggesting its potential to be used in clinical settings across gender and ethnic groups. LIMITATIONS Participants all answered the questionnaires in English, which could hinder cultural variations in their expression of symptoms.
Collapse
Affiliation(s)
- H N Cheung
- School of Arts and Social Sciences, Hong Kong Metropolitan University, Hong Kong.
| | - Joanne M Williams
- Department of Clinical Psychology, University of Edinburgh, United Kingdom
| | - Y S Chan
- School of Applied Social Studies, Hong Kong Polytechnic University, Hong Kong
| | - Stella W Y Chan
- School of Psychology and Clinical Language Sciences, University of Reading, United Kingdom
| |
Collapse
|
5
|
Wistar A, Hall MG, Bercholz M, Taillie LS. Designing Environmental Messages to Discourage Red Meat Consumption: An Online Experiment. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2022; 19:2919. [PMID: 35270622 PMCID: PMC8910317 DOI: 10.3390/ijerph19052919] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/10/2022] [Revised: 02/27/2022] [Accepted: 02/28/2022] [Indexed: 02/06/2023]
Abstract
Reducing red meat consumption in high-consuming countries is critical for mitigating climate change and preventing chronic disease. This study tested the effectiveness of messages conveying the worsening or reduction of environmental harms at discouraging red meat consumption. 1078 U.S. adults viewed seven messages in an online survey highlighting the reduction or worsening of environmental harms associated with eating red meat (between-subjects factor) and rated the messages on how much they discouraged them from wanting to buy beef. Each message highlighted a different environmental harm: deforestation, climate change, water shortages, biodiversity loss, carbon footprint, greenhouse gas emissions, or environment (within-subjects factor). No statistically significant difference was found between the reduction and worsening of environmental harms conditions for most topics, though the worsening of harms frame slightly outperformed the reduction of harms frame for the 'environment' topic. 'Environment' was also the message topic that elicited the strongest response from participants overall. Latino participants, those with more than a high school degree, and those who consume beef once a week or less rated messages as more effective than non-Latino participants, those who completed high school or less, and those who consumed beef more than once a week. Future research should explore the effect of messages on behavioral outcomes.
Collapse
Affiliation(s)
- Alice Wistar
- Program in Global Health and Health Policy, Center for Health and Wellbeing, Princeton School of Public and International Affairs, Princeton University, Princeton, NJ 08544, USA;
| | - Marissa G. Hall
- Department of Health Behavior, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA;
- Lineberger Comprehensive Cancer Center, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
- Carolina Population Center, University of North Carolina at Chapel Hill, Chapel Hill, NC 27516, USA;
| | - Maxime Bercholz
- Carolina Population Center, University of North Carolina at Chapel Hill, Chapel Hill, NC 27516, USA;
| | - Lindsey Smith Taillie
- Carolina Population Center, University of North Carolina at Chapel Hill, Chapel Hill, NC 27516, USA;
- Department of Nutrition, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| |
Collapse
|
6
|
The misrepresentation of spatial uncertainty in visual search: Single- versus joint-distribution probability cues. Atten Percept Psychophys 2020; 83:603-623. [PMID: 33025465 DOI: 10.3758/s13414-020-02145-5] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/08/2020] [Indexed: 11/08/2022]
Abstract
The present study used information theory to quantify the extent to which different spatial cues conveyed the entropy associated with the identity and location of a visual search target. Single-distribution cues reflected the probability that the target would appear at one fixed location whereas joint-distribution cues reflected the probability that the target would appear at the location where another cue (arrow) pointed. The present study used a novel demand-selection paradigm to examine the extent to which individuals explicitly preferred one type of probability cue over the other. Although both cues conveyed equal entropy, the main results suggested representation of greater target entropy for joint- than for single-distribution cues based on a comparison between predicted and observed probability cue choices across four experiments. The present findings emphasize the importance of understanding how individuals represent basic information-theoretic quantities that underlie more complex decision-theoretic processes such as Bayesian and active inference.
Collapse
|
7
|
A new jsPsych plugin for psychophysics, providing accurate display duration and stimulus onset asynchrony. Behav Res Methods 2020; 53:301-310. [PMID: 32700239 DOI: 10.3758/s13428-020-01445-w] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
A JavaScript framework named 'jsPsych' developed by de Leeuw (2015) is widely used for conducting Web-based experiments, and its functionality can be enhanced by using plugins. This article introduces a new jsPsych plugin which enables experimenters to set different onset times for geometric figures, images, sounds, and moving objects, and present them synchronized with the refresh of the display. Moreover, this study evaluated the stimulus onset asynchronies (SOAs) using visual and audio stimuli. The results showed that: (i) the deviations from the intended SOAs between two visual stimuli were less than 10 ms, (ii) the variability across browser-computer combinations was reduced compared with the no-plugin condition, and (iii) the variability of the SOAs between visual and audio stimuli was relatively large (about 50 ms). This study concludes that although the use of audio stimuli is somewhat limited, the new plugin provides experimenters with useful and accurate methods for conducting psychophysical experiments online. The latest version of the plugin can be downloaded freely from https://jspsychophysics.hes.kyushu-u.ac.jp under the MIT license.
Collapse
|
8
|
Hong M, Steedle JT, Cheng Y. Methods of Detecting Insufficient Effort Responding: Comparisons and Practical Recommendations. EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 2020; 80:312-345. [PMID: 32158024 PMCID: PMC7047258 DOI: 10.1177/0013164419865316] [Citation(s) in RCA: 21] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/29/2023]
Abstract
Insufficient effort responding (IER) affects many forms of assessment in both educational and psychological contexts. Much research has examined different types of IER, IER's impact on the psychometric properties of test scores, and preprocessing procedures used to detect IER. However, there is a gap in the literature in terms of practical advice for applied researchers and psychometricians when evaluating multiple sources of IER evidence, including the best strategy or combination of strategies when preprocessing data. In this study, we demonstrate how the use of different IER detection methods may affect psychometric properties such as predictive validity and reliability. Moreover, we evaluate how different data cleansing procedures can detect different types of IER. We provide evidence via simulation studies and applied analysis using the ACT's Engage assessment as a motivating example. Based on the findings of the study, we provide recommendations and future research directions for those who suspect their data may contain responses reflecting careless, random, or biased responding.
Collapse
Affiliation(s)
| | | | - Ying Cheng
- University of Notre Dame, Notre Dame,
IN, USA
| |
Collapse
|
9
|
Abstract
Self-report data are common in psychological and survey research. Unfortunately, many of these samples are plagued with careless responses, due to unmotivated participants. The purpose of this study was to propose and evaluate a robust estimation method to detect careless or unmotivated responders, while leveraging item response theory (IRT) person-fit statistics. First, we outlined a general framework for robust estimation specific for IRT models. Subsequently, we conducted a simulation study covering multiple conditions in order to evaluate the performance of the proposed method. Ultimately, we showed that robust maximum marginal likelihood (RMML) estimation significantly improves detection rates for careless responders and reduces bias in item parameters across conditions. Furthermore, we applied our method to a real data set, to illustrate the utility of the proposed method. Our findings suggest that robust estimation coupled with person-fit statistics offers a powerful procedure to identify careless respondents for further review and to provide more accurate item parameter estimates in the presence of careless responses.
Collapse
|
10
|
Stiff C. The Dark Triad and Facebook surveillance: How Machiavellianism, psychopathy, but not narcissism predict using Facebook to spy on others. COMPUTERS IN HUMAN BEHAVIOR 2019. [DOI: 10.1016/j.chb.2018.12.044] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
11
|
Pauszek JR, Gibson BS. Dealing with distractors in the spatial cueing paradigm can reflect the strategic influence of cognitive effort minimization rather than a limit to selective attention. VISUAL COGNITION 2019. [DOI: 10.1080/13506285.2019.1590488] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
|
12
|
Dubey N, Witzel N, Witzel J. Script differences and masked translation priming: Evidence from Hindi-English bilinguals. Q J Exp Psychol (Hove) 2018; 71:2421-2438. [PMID: 30362410 DOI: 10.1177/1747021817743241] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
This study reports on two experiments investigating the effects of script differences on masked translation priming in highly proficient early Hindi-English bilinguals. In Experiment 1 (the cross-script experiment), L1 Hindi was presented in the standard Devanagari script, while L2 English was presented in the Roman alphabet. In Experiment 2 (the same-script experiment), both L1 Hindi and L2 English were presented in the Roman alphabet. Both experiments revealed translation priming in the L1-L2 direction. However, L2-L1 priming was obtained in the same-script experiment, but not in the cross-script experiment. These findings are discussed in relation to the orthographic cue hypothesis as well as hypotheses that hold that script differences influence the distance between the L1 and L2 in lexical space and/or cross-language lateral inhibition. We also provide alternative accounts for these results in terms of how orthographic cues provided by L1 targets might lead to the discontinuation or disruption of processing for L2 primes.
Collapse
Affiliation(s)
- Namrata Dubey
- Department of Linguistics & TESOL, University of Texas at Arlington, Arlington, TX, USA
| | - Naoko Witzel
- Department of Linguistics & TESOL, University of Texas at Arlington, Arlington, TX, USA
| | - Jeffrey Witzel
- Department of Linguistics & TESOL, University of Texas at Arlington, Arlington, TX, USA
| |
Collapse
|