Detecting careless respondents in web-based questionnaires: Which method to use?

On the Utility of Indirect Methods for Detecting Faking

Indirect indices for faking detection in questionnaires make use of a respondent's deviant or unlikely response pattern over the course of the questionnaire to identify them as a faker. Compared with established direct faking indices (i.e., lying and social desirability scales), indirect indices have at least two advantages: First, they cannot be detected by the test taker. Second, their usage does not require changes to the questionnaire. In the last decades, several such indirect indices have been proposed. However, at present, the researcher's choice between different indirect faking detection indices is guided by relatively little information, especially if conceptually different indices are to be used together. Thus, we examined and compared how well indices of a representative selection of 12 conceptionally different indirect indices perform and how well they perform individually and jointly compared with an established direct faking measure or validity scale. We found that, first, the score on the agreement factor of the Likert-type item response process tree model, the proportion of desirable scale endpoint responses, and the covariance index were the best-performing indirect indices. Second, using indirect indices in combination resulted in comparable and in some cases even better detection rates than when using direct faking measures. Third, some effective indirect indices were only minimally correlated with substantive scales and could therefore be used to partial faking variance from response sets without losing substance. We, therefore, encourage researchers to use indirect indices instead of direct faking measures when they aim to detect faking in their data.

The Mediating Role of Problematic Use of Loot Boxes Between Internet Gaming Disorder and Online Gambling Disorder: Cross-Sectional Analytical Study

Background

The video game industry has introduced a new form of monetization through microtransactions. A controversial example has been the so-called "loot boxes" (LBs) as virtual objects, which are randomized and bought with legal money. In recent years, LBs have come to connect 2 distinct problem behaviors, namely internet gaming disorder (IGD) and online gambling disorder (OGD). Many association studies have been conducted on the 3 constructs, but few have delved into the relationship of problematic use of LBs (PU-LB) with IGD and OGD.

Objective

This study aims to explore the mediating role of the PU-LB between IGD and OGD.

Methods

This cross-sectional and analytical study used incidental sampling in 24 Spanish schools. The final sample consisted of 542 participants (male: n=523, 96.5%; age: range 11-30 y) who played video games, bought LBs, and had gambled online in the last 12 months. Participants then completed the Spanish versions of the Internet Gaming Disorder Scale-Short Form, Online Gambling Disorder Questionnaire, and PU-LB scale.

Results

IGD scores were found to be significantly associated with both PU-LB (r=0.473, P<.001) and OGD (r=0.209, P<.001). Moreover, PU-LB was significantly associated with OGD (r=0.351, P<.001). The structural equation model results indicated that IGD had no significant direct effect on OGD (P=.903). However, the indirect effect of IGD on OGD through PU-LB was significant (P<.001). Therefore, PU-LB fully mediated the relationship between IGD and OGD. Furthermore, these results were found in the subsamples of both minors (<18 y) and young adults (≥18 y).

Conclusions

It is suggested that there is a mediation effect of problematic LB use between internet gambling and online gambling problems in both minors and young adults. This has potential practical implications by providing more evidence on how LBs have become a hinge feature between 2 clinically relevant and independent issues. In this regard, adequate industry self-regulation is needed, and effective legislation for the protection of minors is necessary.

Occupational self-efficacy scale: Validity in teachers

Occupational self-efficacy has gained attention because of its importance in understanding the effects of psychosocial factors at work, but because of its relevance, it is necessary to study it in the context of the COVID-19 pandemic. The Occupational Self-Efficacy Short Scale Form (OSS-SF) is a measure of individual variability in self-efficacy within the work context and has been used in some studies in Latin America. The aim of this study was to obtain evidence of the validity of the OSS-SF for 214 Peruvian teachers and to evaluate its internal structure and associations with other constructs. The sample, composed of 214 teachers (Mage = 44, SDage = 10), was selected using nonprobabilistic convenience sampling and evaluated via online forms. Nonparametric item response theory was used. Among the results, a unidimensional structure and high scalability at the item and scale levels were obtained (>0.70). The reliability was approximately 0.90. There was moderate convergence with job satisfaction (0.39) and slight convergence with the perception of the management of virtual tools (0.18). The OSS-SF is a scale with adequate evidence of validity and reliability for Peruvian teachers who work remotely. Therefore, it can be used as a diagnostic measure of intervention and training needs to benefit teachers and students.

Careless responding detection revisited: Accuracy of direct and indirect measures

To screen for careless responding, researchers have a choice between several direct measures (i.e., bogus items, requiring the respondent to choose a specific answer) and indirect measures (i.e., unobtrusive post hoc indices). Given the dearth of research in the area, we examined how well direct and indirect indices perform relative to each other. In five experimental studies, we investigated whether the detection rates of the measures are affected by contextual factors: severity of the careless response pattern, type of item keying, and type of item presentation. We fully controlled the information environment by experimentally inducing careless response sets under a variety of contextual conditions. In Studies 1 and 2, participants rated the personality of an actor that presented himself in a 5-min-long videotaped speech. In Studies 3, 4, and 5, participants had to rate their own personality across two measurements. With the exception of maximum longstring, intra-individual response variability, and individual contribution to model misfit, all examined indirect indices performed better than chance in most of the examined conditions. Moreover, indirect indices had detection rates as good as and, in many cases, better than the detection rates of direct measures. We therefore encourage researchers to use indirect indices, especially within-person consistency indices, instead of direct measures.

The role of late adolescents' emotion regulation in the experience of COVID-19 lockdown: A longitudinal study

The COVID-19 pandemic may be considered a unique mass-trauma experience. This study examined the relations between Italian late adolescents' emotion regulation strategies, their anxiety states, and their experience of the lockdown (in terms of discomfort related to restrictions, capacities to create new functional daily routines, and to find positive changes in one's own life) during the first wave of this pandemic. We analysed how participants' reports of cognitive reappraisal and expressive suppression were associated with anxiety states during the 2020 Italian COVID-19 lockdown (large scale physical distancing and movement restrictions) and one month after the lockdown restrictions had been removed. We also examined how cognitive reappraisal, expressive suppression, and anxiety states were linked to late adolescents' experience of lockdown. The participants were 497 Italian adolescents, aged from 17 to 24 years (Mage = 21.11, SD = 1.83). A longitudinal structural equation modelling showed that emotion regulation strategies and anxiety states were not associated across time. Cognitive reappraisal was positively associated with routine reorganization and positive changes. In contrast, participants' expressive suppression was negatively related to their discomfort related to restrictions, ability to functionally reorganise their daily routine, and ability to find positive changes related to the COVID-19 emergency. Anxiety was positively linked to discomfort related to restrictions. The findings are discussed in light of the current literature related to emotion regulation and anxiety. Limitations and implications for practice are presented.

Model-agnostic unsupervised detection of bots in a Likert-type questionnaire

To detect bots in online survey data, there is a wealth of literature on statistical detection using only responses to Likert-type items. There are two traditions in the literature. One tradition requires labeled data, forgoing strong model assumptions. The other tradition requires a measurement model, forgoing collection of labeled data. In the present article, we consider the problem where neither requirement is available, for an inventory that has the same number of Likert-type categories for all items. We propose a bot detection algorithm that is both model-agnostic and unsupervised. Our proposed algorithm involves a permutation test with leave-one-out calculations of outlier statistics. For each respondent, it outputs a p value for the null hypothesis that the respondent is a bot. Such an algorithm offers nominal sensitivity calibration that is robust to the bot response distribution. In a simulation study, we found our proposed algorithm to improve upon naive alternatives in terms of 95% sensitivity calibration and, in many scenarios, in terms of classification accuracy.

Peer cybervictimization and cyberaggression as a function of developmental stage during adolescence: A preliminary study

Peer cybervictimization and cyberaggression are educational and social concerns which have been extensively studied during adolescence but there is less research conducted specifically across all stages of adolescence (early, middle, late, and emerging adulthood). The objective was to analyse the prevalence of cybervictimization and cyberaggression, the roles, and the associated behaviors, depending on the stages of adolescence. The sample was composed of 7295 non-university Spanish adolescents, between 11 and 22 years old from 47 schools. The frequency of cybervictimization and cyberaggression was 22.5 % and 15 %, respectively. The highest frequencies are found in late adolescence and the lowest in emerging adulthood. Mainly, involvement increases from early to late adolescence and decreases in emerging adulthood. The magnitude of cybervictimization and cyberaggression behaviors for the roles of pure cybervictim and pure cyberaggressor is similar through all stages.

Detecting non-content-based response styles in survey data: An application of mixture factor analysis

It is common for some participants in self-report surveys to be careless, inattentive, or lacking in effort. Data quality can be severely compromised by responses that are not based on item content (non-content-based [nCB] responses), leading to strong biases in the results of data analysis and misinterpretation of individual scores. In this study, we propose a specification of factor mixture analysis (FMA) to detect nCB responses. We investigated the usefulness and effectiveness of the FMA model in detecting nCB responses using both simulated data (Study 1) and real data (Study 2). In the first study, FMA showed reasonably robust sensitivity (.60 to .86) and excellent specificity (.96 to .99) on mixed-worded scales, suggesting that FMA had superior properties as a screening tool under different sample conditions. However, FMA performance was poor on scales composed of only positive items because of the difficulty in distinguishing acquiescent patterns from valid responses representing high levels of the trait. In Study 2 (real data), FMA detected a minority of cases (6.5%) with highly anomalous response patterns. Removing these cases resulted in a large increase in the fit of the unidimensional model and a substantial reduction in spurious multidimensionality.

Q-Herilearn: Assessing heritage learning in digital environments. A mixed approach with factor and IRT models

The assessment of heritage learning in digital environments lacks instruments that measure it with sufficient guarantees of accuracy, validity, and reliability. This study attempts to fill this gap by developing an instrument that has shown solid metric qualities. The process of design and calibration of a scale applied to 1,454 participants between 19 and 63 years of age is presented in this article. Exploratory factor analysis (Exploratory Structural Equation Modeling ESEM) and Item Response Theory models (Graded Response Model GRM) were used. Sufficient evidence of both reliability and validity based on content and internal structure was obtained. Invariance of scores as a function of gender and age of participants has also been demonstrated. The discrimination parameters of the items have been found to be high, and the test information curves have shown that the subscales measure with sufficient precision wide ranges of the respective latent variables. The instrument presents wide possibilities of application to various areas of Heritage Education (e.g., design of programs in HE, definition and planning of teaching objectives, evaluation of programs, etc., in virtual environments).

Cross-cultural adaptation, reliability and validity of the Spanish version of the long-term quality of life questionnaire

Purpose

The aim of this study was to translate, culturally adapt, and evaluate the psychometric properties of the Spanish Long-Term Quality of Life (LTQL) questionnaire.

Methods

The LTQL was initially translated into Spanish and cross-culturally adapted based on established guidelines. The Spanish LTQL was administered to patients with breast cancer who had completed their initial treatment 5 years earlier, along with other self-report measures: Quality of Life in Adult Cancer Survivors (QLACS), Hospital Anxiety and Depression Scale (HADS) and EORT-QLQ-BR23. Reliability was evaluated using internal consistency and test-retest. Convergent and known-groups validity were examined. Structural validity as determined by confirmatory factor analysis (CFA) and Rasch analyses was used to assess the unidimensionality and item-functioning of the LTQL domains.

Results

Cronbach's alpha were above 0.7 in all domains. Test-retest coefficients were between 0.72 to 0.96 for LTQL domains. LTQL total score was correlated with others total scores of other measures: QLACS (r=-0.39), HADS depression (r=-0.57), HADS anxiety (-0.45) and EORTC-QLQ-BR23 (r=-0.50). CFA provided satisfactory fit indices, with RMSEA value of 0.077 and TLI and CFI values of 0.901 and 0.909, respectively. All factor loadings were higher than 0.40 and statistically significant (P<0.001). Rasch analysis showed that Somatic Concerns domain had 4 misfitting items, and Philosophical/Spiritual View of Life and social Support domains only 1 misfit item. However, unidimensionality was supported for the four domains.

Conclusion

The findings support the validity and reliability of the Spanish version of LTQL questionnaire to be used in long-term cancer female survivors.

Modeling insufficient effort responses in mixed-worded scales

Surveys often add reverse-coded questions to monitor respondents with insufficient effort responses (IERs) but often wrongly assume that all respondents consistently answer all questions with full effort. By contrast, this study expanded the mixture model for IERs and ran a simulation via LatentGOLD to show the harmful consequences of ignoring IERs to positively and negatively worded questions: less test reliability, bias and less accuracy in slope and intercept parameters. We showed its practical application to two public data sets: Machiavellianism (five-point scale) and self-reported depression (four-point scale).

A sequential Moken scaling approach to evaluate response quality in survey research

Careless responding, where participants do not fully engage with item content, is pervasive in survey research. Left undetected, carelessness can compromise the interpretation and use of survey results, including information about participant locations on the construct, item difficulty, and the psychometric quality of the instrument. We present and illustrate a sequential procedure for evaluating response quality in survey research using indicators from Mokken scale analysis (MSA). We use a real data illustration and a simulation study to compare a sequential procedure to a standalone procedure. We also consider how identifying and removing responses with evidence of poor measurement properties affects item quality indicators. Results suggest that the sequential procedure was effective in identifying potentially problematic response patterns that may not always be captured by traditional methods for identifying careless responders but was not always sensitive to specific carelessness patterns. We discuss implications for research and practice.

A sequential Bayesian changepoint detection procedure for aberrant behaviours in computerized testing

Changepoints are abrupt variations in a sequence of data in statistical inference. In educational and psychological assessments, it is essential to properly differentiate examinees' aberrant behaviours from solution behaviour to ensure test reliability and validity. In this paper, we propose a sequential Bayesian changepoint detection algorithm to monitor the locations of changepoints for response times in real time and, subsequently, further identify types of aberrant behaviours in conjunction with response patterns. Two simulation studies were conducted to investigate the efficiency and accuracy of the proposed detection procedure in terms of identifying one or multiple changepoints at different locations. In addition to manipulating the number and locations of changepoints, two types of aberrant behaviours were also considered: rapid guessing behaviour and cheating behaviour. Simulation results indicate that ability estimates could be improved after removing responses from aberrant behaviours identified by our approach. Two empirical examples were analysed to illustrate the application of the proposed sequential Bayesian changepoint detection procedure.

Accounting for careless and insufficient effort responding in large-scale survey data-development, evaluation, and application of a screen-time-based weighting procedure

Careless and insufficient effort responding (C/IER) poses a major threat to the quality of large-scale survey data. Traditional indicator-based procedures for its detection are limited in that they are only sensitive to specific types of C/IER behavior, such as straight lining or rapid responding, rely on arbitrary threshold settings, and do not allow taking the uncertainty of C/IER classification into account. Overcoming these limitations, we develop a two-step screen-time-based weighting procedure for computer-administered surveys. The procedure allows considering the uncertainty in C/IER identification, is agnostic towards the specific types of C/IE response patterns, and can feasibly be integrated with common analysis workflows for large-scale survey data. In Step 1, we draw on mixture modeling to identify subcomponents of log screen time distributions presumably stemming from C/IER. In Step 2, the analysis model of choice is applied to item response data, with respondents' posterior class probabilities being employed to downweigh response patterns according to their probability of stemming from C/IER. We illustrate the approach on a sample of more than 400,000 respondents being administered 48 scales of the PISA 2018 background questionnaire. We gather supporting validity evidence by investigating relationships between C/IER proportions and screen characteristics that entail higher cognitive burden, such as screen position and text length, relating identified C/IER proportions to other indicators of C/IER as well as by investigating rank-order consistency in C/IER behavior across screens. Finally, in a re-analysis of the PISA 2018 background questionnaire data, we investigate the impact of the C/IER adjustments on country-level comparisons.

Using Mokken scaling techniques to explore carelessness in survey research

Careless responding is a pervasive issue that impacts the interpretation and use of responses from survey instruments. Researchers have proposed numerous useful methods for detecting carelessness in survey research, including relatively simple summary statistics such as the frequency of adjacent responses in the same category (e.g., "long-string" analysis) and outlier statistics (e.g., Mahalanobis distance). Researchers have also used methods based on item response theory (IRT) models to identify examinees whose response patterns are unexpected given item parameters. However, researchers have not fully considered the use of nonparametric IRT methods based on Mokken scale analysis (MSA) to detect carelessness in survey research. MSA is a promising framework in which to consider participant carelessness because it is well suited to contexts in which parametric IRT models may not be appropriate, while still maintaining a focus on fundamental measurement requirements. We used a real data analysis and a simulation study to examine the sensitivity of MSA indicators of response quality to examinee carelessness and compared the results to those from standalone indicators. We also examined the impact of carelessness on the sensitivity of MSA item quality indicators. Numeric and graphical indicators of response quality from MSA indicators were sensitive to examinee carelessness. Graphical displays of nonparametric person response functions (PRFs) provided supplementary insight that can alert researchers to potentially problematic responses. Our results also indicated that MSA indicators of item quality are robust to the presence of participant carelessness. We consider the implications of our findings for research and practice.

Comparing Person-Fit and Traditional Indices Across Careless Response Patterns in Surveys

Methods to identify carelessness in survey research can be valuable tools in reducing bias during survey development, validation, and use. Because carelessness may take multiple forms, researchers typically use multiple indices when identifying carelessness. In the current study, we extend the literature on careless response identification by examining the usefulness of three item-response theory-based person-fit indices for both random and overconsistent careless response identification: infit MSE outfit MSE, and the polytomous lz statistic. We compared these statistics with traditional careless response indices using both empirical data and simulated data. The empirical data included 2,049 high school student surveys of teaching effectiveness from the Network for Educator Effectiveness. In the simulated data, we manipulated type of carelessness (random response or overconsistency) and percent of carelessness present (0%, 5%, 10%, 20%). Results suggest that infit and outfit MSE and the lz statistic may provide complementary information to traditional indices such as LongString, Mahalanobis Distance, Validity Items, and Completion Time. Receiver operating characteristic curves suggested that the person-fit indices showed good sensitivity and specificity for classifying both over-consistent and under-consistent careless patterns, thus functioning in a bidirectional manner. Carelessness classifications based on low fit values correlated with carelessness classifications from LongString and completion time, and classifications based on high fit values correlated with classifications from Mahalanobis Distance. We consider implications for research and practice.

Understanding professional development challenges of Chinese public health professionals: association and prediction analyses with data validity screening

Background

Little is known about the public health professionals engaged in educating and training new or future researchers in public health. Research in this direction identifies their issues, concerns, challenges, and needs. This study focused on the professional development challenges of Chinese public health professionals.

Methods

Snowball sampling was utilized. A total of 265 public health professionals participated. An instrument of 6 dimensions (burnout, sleep issue, mood issue, friends' support, exercise, and challenges) was developed, revised, and administered online. Two different approaches, the conventional and data screening approaches, were applied. The former started with item quality analyses, whereas the latter began with data quality checks. The chi-square tests of associations and logistic regressions were performed on both approaches.

Results and discussion

19.25% of the participants were detected and deleted as careless respondents. Using both approaches, six professional development challenges except one ("Multidisciplinary learning") were significantly associated with various demographic features. The two approaches produced different models though they converged sometimes. The latent variables of exercise predicted professional development challenges more frequently than other latent variables. Regarding correct classification rates, results from the data screening approach were comparable to those from the conventional approach.

Conclusion

The latent variables of exercise, such as "Exercise effects," "Expectations of exercise," and "Belief in exercise," might be understudied. More research is necessary for professional development challenges using exercise as a multidimensional construct. Based on the current study, screening and deleting careless responses in survey research is necessary.

Using heterogeneous sources of data and interpretability of prediction models to explain the characteristics of careless respondents in survey data

Prior to further processing, completed questionnaires must be screened for the presence of careless respondents. Different people will respond to surveys in different ways. Some take the easy path and fill out the survey carelessly. The proportion of careless respondents determines the survey's quality. As a result, identifying careless respondents is critical for the quality of obtained results. This study aims to explore the characteristics of careless respondents in survey data and evaluate the predictive power and interpretability of different types of data and indices of careless responding. The research question focuses on understanding the behavior of careless respondents and determining the effectiveness of various data sources in predicting their responses. Data from a three-month web-based survey on participants' personality traits such as honesty-humility, emotionality, extraversion, agreeableness, conscientiousness and openness to experience was used in this study. Data for this study was taken from Schroeders et al.. The gradient boosting machine-based prediction model uses data from the answers, time spent for answering, demographic information on the respondents as well as some indices of careless responding from all three types of data. Prediction models were evaluated with tenfold cross-validation repeated a hundred times. Prediction models were compared based on balanced accuracy. Models' explanations were provided with Shapley values. Compared with existing work, data fusion from multiple types of information had no noticeable effect on the performance of the gradient boosting machine model. Variables such as "I would never take a bribe, even if it was a lot", average longstring, and total intra-individual response variability were found to be useful in distinguishing careless respondents. However, variables like "I would be tempted to use counterfeit money if I could get away with it" and intra-individual response variability of the first section of a survey showed limited effectiveness. Additionally, this study indicated that, whereas the psychometric synonym score has an immediate effect and is designed with the goal of identifying careless respondents when combined with other variables, it is not necessarily the optimal choice for fitting a gradient boosting machine model.

Solicitation and Sexualized Interactions of Minors with Adults: Prevalence, Overlap with Other Forms of Cybervictimization, and Relationship with Quality of Life

Sexual solicitation and sexualized interaction with minors by adults constitute one of the most pernicious risks of the Internet. Little is known about the age range in which this phenomenon is most prevalent or the relationship and overlap of this problem with other risks, such as peer-to-peer cybervictimization and cyber dating abuse. Additionally, little empirical evidence exists on whether the overlap between these types of online victimization affects health-related quality of life (HRQoL). This study aimed to: 1) analyze the prevalence of sexual solicitation and interaction according to sex and stage of adolescence; 2) relate this problem to other forms of online victimization (cybervictimization and cyber dating abuse); 3) analyze the overlap between these forms of online victimization and differences in HRQoL scores. A cross-sectional and analytical study with 3,578 adolescents (52.7% girls) aged between 10-15 years was carried out. Of the adolescents in the study sample, 12.6% (n = 448) had received sexual requests and 6.4% (n = 230) had interacted sexually with adults. Sexual solicitation was most common among girls in middle adolescence. Of the participants, 33.9% (n = 1,216) had been involved in some form of online victimization. Peer cybervictimization and cyber dating abuse were positively and significantly correlated with sexualized solicitation/interactions with adults. Victims with the overlapping of all three types of online victimization (2.7%, n = 98) presented the lowest HQRoL scores (p < .001).

Dissociating cognitive and affective empathy across psychopathy dimensions: The role of interoception and alexithymia

This study examined the associations between psychopathy dimensions (triarchic phenotypes and classical factors), empathy domains (cognitive and affective), and interoception (interoceptive attention and accuracy) while accounting for the putative role of alexithymia. A community sample (n = 515) completed an online survey encompassing: Triarchic Psychopathy Measure (boldness, meanness, disinhibition); Levenson Self-Report Psychopathy Scale (primary and secondary psychopathy); Body Perception Questionnaire (interoceptive attention); Interoceptive Accuracy Scale; Toronto Alexithymia Scale. Hierarchical linear regression models were implemented for hypothesis-driven analyses examining the associations between psychopathy, empathy, and interoception while controlling for sex, age, and alexithymia. Exploratory path models were employed to investigate alexithymia and/or cognitive empathy as mediators between interoception and psychopathy. Our results largely confirmed the postulated empathy profiles across psychopathy dimensions, as meanness and primary psychopathy displayed a broad empathy impairment, while disinhibition and secondary psychopathy were only associated with diminished cognitive empathy. Importantly, boldness displayed a unique pattern (enhanced cognitive empathy and reduced affective empathy), further reinforcing its importance within the constellation of psychopathy traits. Contrary to our hypotheses, self-perceived interoceptive attention and accuracy were not associated with either psychopathy dimension after controlling for alexithymia. However, interoceptive accuracy and alexithymia were associated with cognitive empathy, while alexithymia was also positively related to all psychopathy dimensions (as expected), despite the unexpected strong and negative association with boldness. Exploratory analyses suggested significant indirect effects (mediation) between interoceptive accuracy and psychopathy via alexithymia and/or cognitive empathy. These mediating effects must be interpreted with caution and future studies should be designed to formally test this model.

Supervised Classes, Unsupervised Mixing Proportions: Detection of Bots in a Likert-Type Questionnaire

Administering Likert-type questionnaires to online samples risks contamination of the data by malicious computer-generated random responses, also known as bots. Although nonresponsivity indices (NRIs) such as person-total correlations or Mahalanobis distance have shown great promise to detect bots, universal cutoff values are elusive. An initial calibration sample constructed via stratified sampling of bots and humans-real or simulated under a measurement model-has been used to empirically choose cutoffs with a high nominal specificity. However, a high-specificity cutoff is less accurate when the target sample has a high contamination rate. In the present article, we propose the supervised classes, unsupervised mixing proportions (SCUMP) algorithm that chooses a cutoff to maximize accuracy. SCUMP uses a Gaussian mixture model to estimate, unsupervised, the contamination rate in the sample of interest. A simulation study found that, in the absence of model misspecification on the bots, our cutoffs maintained accuracy across varying contamination rates.

Measuring the suicidal mind: The 'open source' Suicidality Scale, for adolescents and adults

Clinicians are expected to provide accurate and useful mental health assessments, sometimes in emergency settings. The most urgent challenge may be in calculating suicide risk. Unfortunately, existing instruments often fail to meet requirements. To address this situation, we used a sustainable scale development approach to create a publicly available Suicidality Scale (SS). Following a critical review of current measures, community input, and panel discussions, an international item pool survey included 5,115 English-speaking participants aged 13-82 years. Revisions were tested with two follow-up cross-sectional surveys (Ns = 814 and 626). Pool items and SS versions were critically examined through item response theory, hierarchical cluster, factor and bifactor analyses, resulting in a unidimensional eight-item scale. Psychometric properties were high (loadings > .77; discrimination > 2.2; test-retest r = .87; internal consistency, ω = .96). Invariance checks were satisfied for age, gender, ethnicity, rural/urban residence, first language, self-reported psychiatric diagnosis and suicide attempt history. The SS showed stronger psychometric properties, and significant differences in bivariate associations with depressive symptoms, compared with included suicide measures. The 'open source' Suicidality Scale represents a significant step forward in accurate assessment for people aged 13+, and diverse populations. This study provides an example of sustainable scale development utilizing community input, emphasis on strong psychometric evidence from diverse samples, and a free-to-use license allowing instrument revisions. These methods can be used to develop a wide variety of psychosocial instruments that can benefit clinicians, researchers, and the public.

Internal Structure of the Work-Family Conflict Questionnaire (WFCQ) in Teacher Teleworking

The interference between family and work roles has led to the development of scales for their measurement. However, instrumental studies of work-family conflict have not been conducted in the context of teacher teleworking during the COVID-19 pandemic. For this reason, the objectives of this study were set to obtain evidence of the internal structure and fairness of the Blanch and Aluja Work-Family Conflict Questionnaire, as well as its association with job satisfaction and other sociodemographic variables. A total of 235 Peruvian school teachers between the ages of 24 and 72 years (M = 43.79 and SD = 9.67) responded to the scale using the online form. The analysis employed the non-parametric item response theory modeling (Mokken scaling analysis). The structure of two correlated factors was confirmed: work conflict in the family (WCF) and family conflict in the work (FCW). Both dimensions were invariant with respect to sex group and educational level. The association of both dimensions with job satisfaction was theoretically convergent, and the gender of the teachers slightly moderated this relationship. The reliability was adequate for group research. Finally, the instrument can be useful in the organizational context of teachers who telework.

Machine learning to detect invalid text responses: Validation and comparison to existing detection methods

A crucial step in analysing text data is the detection and removal of invalid texts (e.g., texts with meaningless or irrelevant content). To date, research topics that rely heavily on analysis of text data, such as autobiographical memory, have lacked methods of detecting invalid texts that are both effective and practical. Although researchers have suggested many data quality indicators that might identify invalid responses (e.g., response time, character/word count), few of these methods have been empirically validated with text responses. In the current study, we propose and implement a supervised machine learning approach that can mimic the accuracy of human coding, but without the need to hand-code entire text datasets. Our approach (a) trains, validates, and tests on a subset of texts manually labelled as valid or invalid, (b) calculates performance metrics to help select the best model, and (c) predicts whether unlabelled texts are valid or invalid based on the text alone. Model validation and evaluation using autobiographical memory texts indicated that machine learning accurately detected invalid texts with performance near human coding, significantly outperforming existing data quality indicators. Our openly available code and instructions enable new methods of improving data quality for researchers using text as data.

Predictors of sustainment of two distinct nutrition and physical activity programs in early care and education

Introduction

The goal of the present study was to investigate factors associated with sustainment of two evidence-based programs for nutrition promotion in early care and education (ECE) settings - Food Friends (FF) and Together, We Inspire Smart Eating (WISE).

Materials and methods

In a cross-sectional study design, ECE directors (N = 55) from centers that had previously been trained in WISE or FF completed a survey. Program-specific measures included Steckler's Perception of Innovations, the Program Sustainability Assessment Tool (PSAT), and the Organizational Readiness for Change Assessment (ORCA). For our primary outcomes, two measures of sustainment were examined: Nutrition Continued Practice (i.e., the use of or general focus on nutrition programs) and Program Fidelity (i.e., how well centers used specific evidence-based practices of WISE or FF). Multiple regression was used to determine the association of these outcomes with program, years since last implementation, and overall scores on predictors. Follow-up correlation analyses were used to investigate outcome relationships with context submeasures due to high intercorrelations between predictor submeasures.

Results

Nutrition Continued Practice was significantly predicted by program and overall PSAT score. WISE programs had significantly higher Nutrition Continued Practice scores than FF program (p = 0.03). All subscales of the PSAT (e.g., environmental support, funding stability, organizational capacity, program adaptation, communications, and strategic planning) were significantly correlated with Nutrition Continued Practice (all rs > 0.30, all ps < 0.03). Program Fidelity was significantly predicted by PSAT and Steckler Perception of Innovation scores. All subscales of the PSAT were strongly positively correlated with Program Fidelity (all rs > 0.48, all ps < 0.001); relative advantage (r = 0.54, p < 0.001) and level of institutionalization (r = 0.61, p < 0.001) were positively correlated with Program Fidelity.

Conclusion

This study suggests that factors associated with the continued practice of program principles are partially distinct from those that are associated with the sustainment of specific practices driving program fidelity. Results suggest capacity building strategies may be important for both continued attention to nutrition and physical activity as well as sustaining fidelity to specific evidence-based practices.

An explanatory mixture IRT model for careless and insufficient effort responding in self-report measures

Careless and insufficient effort responding (C/IER) on self-report measures results in responses that do not reflect the trait to be measured, thereby posing a major threat to the quality of survey data. Reliable approaches for detecting C/IER aid in increasing the validity of inferences being made from survey data. First, once detected, C/IER can be taken into account in data analysis. Second, approaches for detecting C/IER support a better understanding of its occurrence, which facilitates designing surveys that curb the prevalence of C/IER. Previous approaches for detecting C/IER are limited in that they identify C/IER at the aggregate respondent or scale level, thereby hindering investigations of item characteristics evoking C/IER. We propose an explanatory mixture item response theory model that supports identifying and modelling C/IER at the respondent-by-item level, can detect a wide array of C/IER patterns, and facilitates a deeper understanding of item characteristics associated with its occurrence. As the approach only requires raw response data, it is applicable to data from paper-and-pencil and online surveys. The model shows good parameter recovery and can well handle the simultaneous occurrence of multiple types of C/IER patterns in simulated data. The approach is illustrated on a publicly available Big Five inventory data set, where we found later item positions to be associated with higher C/IER probabilities. We gathered initial supporting validity evidence for the proposed approach by investigating agreement with multiple commonly employed indicators of C/IER.

Crafting work-nonwork balance involving life domain boundaries: Development and validation of a novel scale across five countries

Ongoing developments, such as digitalization, increased the interference of the work and nonwork life domains, urging many to continuously manage engagement in respective domains. The COVID-19 pandemic and subsequent home-office regulations further boosted the need for employees to find a good work-nonwork balance, thereby optimizing their health and well-being. Consequently, proactive individual-level crafting strategies for balancing work with other relevant life domains were becoming increasingly important. However, these strategies received insufficient attention in previous research despite their potential relevance for satisfying psychological needs, such as psychological detachment. We addressed this research gap by introducing a new scale measuring crafting for a work-nonwork balance and examining its relevance in job-and life satisfaction, work engagement, subjective vitality, family role and job performance, boundary management and self-rated work-nonwork balance. The Work-Nonwork Balance Crafting Scale was validated in five countries (Austria, Finland, Germany, Japan, and Switzerland), encompassing data from a heterogeneous sample of more than 4,200 employees. In study 1, exploratory factor analysis revealed a two-factorial scale structure. Confirmatory factor analysis, test for measurement invariance, and convergent validity were provided in study 2. Replication of confirmatory factor analysis, incremental and criterion validity of the Work-Nonwork Balance Crafting Scale for job and life satisfaction were assessed in study 3. Study 4 displayed criterion validity, test–retest reliability, testing measurement invariance, and applicability of the scale across work cultures. Finally, study 5 delivered evidence for the Work-Nonwork Balance Crafting Scale in predicting work-nonwork balance. The novel Work-Nonwork Balance Crafting Scale captured crafting for the challenging balance between work and nonwork and performed well across several different working cultures in increasingly digitalized societies. Both researchers and practitioners may use this tool to assess crafting efforts to balance both life domains and to study relationships with outcomes relevant to employee health and well-being.

A mixed-methods study of problematic social media use, attention dysregulation, and social media use motives

Problematic social media use (PSMU) refers to excessive uncontrolled use of social media which impacts upon daily functioning (Blackwell et al., 2017). Self-regulation is central to the development and experience of PSMU, and conceptually interrelates with individual usage motivations (Reinecke et al., 2022). While there is a growing body of research on social media use motivations, how usage motivations and self-regulation combined influence PSMU is not well understood. There are also persistent questions around the effectiveness of addiction-based measures of PSMU. The quantitative component of this nested mixed-methods study (N = 607) employed hierarchical regression and structural equation modelling, principally identifying that impulsive social media usage mediates the pathway between perceived executive/attentional functioning and the Bergen Social Media Addiction Scale (BSMAS, Andreassen et al., 2012, 2016), a popular tool used to measure PSMU. In contrast, social-engagement motivations had a negative influence on the BSMAS. The qualitative component, comprising interview/open-ended questionnaire, explored individual experiences self-regulating social media use. Participants (N = 24) were recruited from the survey study, based on meeting screening criteria for executive dysfunction (Adult Self-Report ADHD Scale, Kessler et al., 2005), with sub-groups defined by top and bottom quartile BSMAS scores (evenly grouped). Thematic analysis found that most individuals with attention dysregulation, regardless of their BSMAS category, perceive self-regulation of social media use as highly challenging and effortful, describing broadly problematic relationship with social media. They also described rich combination of motivations and context of using social media, and strategies for managing use. This research questions the effectiveness of the BSMAS as a measure of general PSMU (lacking a formed self-regulation component), especially in individuals with attentional dysregulation. Future research investigating self-regulation strategies and focusing on characteristics of positive social media use is needed.

How a Few Inconsistent Respondents Can Confound the Structure of Personality Survey Data

Abstract. In survey data, inconsistent responses due to careless/insufficient effort (C/IE) can lead to problems of replicability and validity. However, data cleaning prior to the main analyses is not yet a standard practice. We investigated the effect of C/IE responses on the structure of personality survey data. For this purpose, we analyzed the structure of the Core-Self Evaluations scale (CSE-S), including the detection of aberrant responses in the study design. While the original theoretical model of the CSE-S assumes that the construct is unidimensional ( Judge et al., 2003 ), recent studies have argued for a multidimensional solution (positive CSE and negative CSE). We hypothesized that this multidimensionality is not substantive but a result of the tendency of C/IE data to generate spurious dimensions. We estimated the confirmatory models before and after removing highly inconsistent response vectors in two independent samples (6% and 4.7%). The analysis of the raw samples clearly favored retaining the two-dimensional model. In contrast, the analysis of the clean datasets suggested the retention of a single factor. A mere 6% C/IE response rate showed enough power to confound the results of the factor analysis. This result suggests that the factor structure of positive and negative CSE factors is spurious, resulting from uncontrolled wording variance produced by a limited proportion of highly inconsistent response vectors. We encourage researchers to include screening for inconsistent responses in their research designs.

COVID-19 vaccine behaviors and intentions among a national sample of United States adults ages 18-45

BACKGROUND

Vaccination for SARS-CoV-2, the virus that causes COVID-19 illness, is an important public health tool to reduce hospitalizations and deaths.

PURPOSE

This report focuses on intentions and behaviors related to COVID-19 vaccination among United States (U.S.) adults ages 18-45.

METHODS

From February 25-March 24, 2021, we conducted an online survey assessing COVID-19 vaccine intentions and behaviors, health beliefs, vaccine attitudes, and sociodemographic characteristics. Participants were adults aged 18-45, living throughout the U.S. with oversampling in Florida, panelists of a research panel company directly or via verified partners, and able to read, write, and understand English. Associations between COVID-19 vaccination uptake, intentions, and other study variables were examined through multivariable logistic and proportional odds regression analyses.

RESULTS

Among participants in the final analytic sample (n = 2722), 18% reported having received at least one dose of a COVID-19 vaccine. Approximately 31% of unvaccinated participants reported strong intentions to receive a COVID-19 vaccine in the next year, whereas 35% reported strong intentions to receive a COVID-19 vaccine if it were strongly recommended by a healthcare provider. All COVID-19 vaccination outcomes were associated with male gender, sexual minority status, higher levels of education, and previous influenza vaccination. All vaccination intention outcomes were associated with vaccine attitudes and geographic region. Vaccination status and intentions were differentially associated with multiple additional sociodemographic, attitudinal, and/or healthcare experience variables.

CONCLUSIONS

Several demographic variables, vaccine attitudes, and healthcare experiences were found to contribute to COVID-19 vaccine receipt and intentions. Targeted efforts are necessary to increase uptake of the vaccine in the U.S.

Evaluating Links between Social Withdrawal Motivations and Indices of Psychosocial Adjustment among Norwegian Emerging Adults

Social withdrawal is the behavioral tendency to remove oneself from social situations - a tendency that often contributes to reductions in individuals' mental health. The current study evaluated the links between different motivations for social withdrawal (shyness, unsociability, social avoidance) and indices of psychosocial adjustment in a Norwegian sample of emerging adults. Participants were N = 194 Norwegian university students who completed self-report measures of life satisfaction, loneliness, and depressive symptoms, as well as withdrawal motivations. Among the results, a newly translated version of the Social Preference Scale-Revised (SPS-R) was validated for use in Norway. Findings showed that shyness was uniquely and positively associated with loneliness and depressive symptoms, as well as lower life satisfaction, whereas social avoidance was positively associated with depressive symptoms. Unsociability was uniquely linked to lower levels of loneliness and depressive symptoms. Findings provide novel information about the psychosocial correlates of social withdrawal motivations during emerging adulthood in the under-explored cultural context of Norway. Understanding nuances in the correlates of different motivations may aid in the development of culturally and developmentally sensitive interventions.

Strategies and Lessons Learned During Cleaning of Data From Research Panel Participants: Cross-sectional Web-Based Health Behavior Survey Study

Background

The use of web-based methods to collect population-based health behavior data has burgeoned over the past two decades. Researchers have used web-based platforms and research panels to study a myriad of topics. Data cleaning prior to statistical analysis of web-based survey data is an important step for data integrity. However, the data cleaning processes used by research teams are often not reported.

Objective

The objectives of this manuscript are to describe the use of a systematic approach to clean the data collected via a web-based platform from panelists and to share lessons learned with other research teams to promote high-quality data cleaning process improvements.

Methods

Data for this web-based survey study were collected from a research panel that is available for scientific and marketing research. Participants (N=4000) were panelists recruited either directly or through verified partners of the research panel, were aged 18 to 45 years, were living in the United States, had proficiency in the English language, and had access to the internet. Eligible participants completed a health behavior survey via Qualtrics. Informed by recommendations from the literature, our interdisciplinary research team developed and implemented a systematic and sequential plan to inform data cleaning processes. This included the following: (1) reviewing survey completion speed, (2) identifying consecutive responses, (3) identifying cases with contradictory responses, and (4) assessing the quality of open-ended responses. Implementation of these strategies is described in detail, and the Checklist for E-Survey Data Integrity is offered as a tool for other investigators.

Results

Data cleaning procedures resulted in the removal of 1278 out of 4000 (31.95%) response records, which failed one or more data quality checks. First, approximately one-sixth of records (n=648, 16.20%) were removed because respondents completed the survey unrealistically quickly (ie, <10 minutes). Next, 7.30% (n=292) of records were removed because they contained evidence of consecutive responses. A total of 4.68% (n=187) of records were subsequently removed due to instances of conflicting responses. Finally, a total of 3.78% (n=151) of records were removed due to poor-quality open-ended responses. Thus, after these data cleaning steps, the final sample contained 2722 responses, representing 68.05% of the original sample.

Conclusions

Examining data integrity and promoting transparency of data cleaning reporting is imperative for web-based survey research. Ensuring a high quality of data both prior to and following data collection is important. Our systematic approach helped eliminate records flagged as being of questionable quality. Data cleaning and management procedures should be reported more frequently, and systematic approaches should be adopted as standards of good practice in this type of research.

A Response-Time-Based Latent Response Mixture Model for Identifying and Modeling Careless and Insufficient Effort Responding in Survey Data

Careless and insufficient effort responding (C/IER) can pose a major threat to data quality and, as such, to validity of inferences drawn from questionnaire data. A rich body of methods aiming at its detection has been developed. Most of these methods can detect only specific types of C/IER patterns. However, typically different types of C/IER patterns occur within one data set and need to be accounted for. We present a model-based approach for detecting manifold manifestations of C/IER at once. This is achieved by leveraging response time (RT) information available from computer-administered questionnaires and integrating theoretical considerations on C/IER with recent psychometric modeling approaches. The approach a) takes the specifics of attentive response behavior on questionnaires into account by incorporating the distance-difficulty hypothesis, b) allows for attentiveness to vary on the screen-by-respondent level, c) allows for respondents with different trait and speed levels to differ in their attentiveness, and d) at once deals with various response patterns arising from C/IER. The approach makes use of item-level RTs. An adapted version for aggregated RTs is presented that supports screening for C/IER behavior on the respondent level. Parameter recovery is investigated in a simulation study. The approach is illustrated in an empirical example, comparing different RT measures and contrasting the proposed model-based procedure against indicator-based multiple-hurdle approaches.

Put your feet up: The impact of personality traits, job pressure, and social support on the need for recovery after work

The need for recovery after work (NFR) is an important warning of work-related fatigue. NFR is linked to prolonged work-related efforts and depletion of resources, creating a need for temporary respite from work demands. The aim of the current study was to investigate the relationships between NFR and the five-factor model (FFM), comprising the personality traits of emotional stability (ES), extraversion (E), agreeableness (A), conscientiousness (C), and openness to experience (O). Perceived job pressure and perceived social support were included as mediators. The study was conducted using structural equation modelling (SEM) on cross-sectional data from a sample of 681 participants from several work sectors (N females = 376, N males = 305; M age = 46.9 years; SD = 11.1). The results showed that NFR was affected both directly and indirectly by FFM traits. High ES and high O contributed directly to reduced and increased NFR, respectively. High perceived social support contributed to reduced NFR, while high perceived job pressure contributed to increased NFR. High ES contributed indirectly to reduced NFR through perceived job pressure and social support, high O contributed indirectly to increased NFR through perceived social support, and high E contributed indirectly to increased NFR through perceived job pressure. A and C were not related to NFR. The findings demonstrate that personality traits, especially ES, are firmly related to NFR and highlight the importance of incorporating personality factors into studies of work environmental factors on NFR.

Supplementary Information

The online version contains supplementary material available at 10.1007/s12144-022-02950-1.

Detecting Careless Responding in Survey Data Using Stochastic Gradient Boosting

Careless responding is a bias in survey responses that disregards the actual item content, constituting a threat to the factor structure, reliability, and validity of psychological measurements. Different approaches have been proposed to detect aberrant responses such as probing questions that directly assess test-taking behavior (e.g., bogus items), auxiliary or paradata (e.g., response times), or data-driven statistical techniques (e.g., Mahalanobis distance). In the present study, gradient boosted trees, a state-of-the-art machine learning technique, are introduced to identify careless respondents. The performance of the approach was compared with established techniques previously described in the literature (e.g., statistical outlier methods, consistency analyses, and response pattern functions) using simulated data and empirical data from a web-based study, in which diligent versus careless response behavior was experimentally induced. In the simulation study, gradient boosting machines outperformed traditional detection mechanisms in flagging aberrant responses. However, this advantage did not transfer to the empirical study. In terms of precision, the results of both traditional and the novel detection mechanisms were unsatisfactory, although the latter incorporated response times as additional information. The comparison between the results of the simulation and the online study showed that responses in real-world settings seem to be much more erratic than can be expected from the simulation studies. We critically discuss the generalizability of currently available detection methods and provide an outlook on future research on the detection of aberrant response patterns in survey research.

Is use of the general system justification scale across countries justified? Testing its measurement equivalence

System justification is a widely researched topic in social and political psychology. One major measurement instrument in system justification research is the General System Justification Scale (G-SJS). This scale has been used, among others, for comparisons across social groups in different countries. Such comparisons rely on the assumption that the scale is measurement equivalent. However, this assumption has never been comprehensively tested. Thus, the present two studies assessed the measurement equivalence of the G-SJS following classic measurement equivalence guidelines (i.e., multigroup confirmatory factor analyses) in Study 1 and using a new method for comparing larger numbers of groups in Study 2 (i.e., alignment optimization). In Study 1, we analysed the measurement equivalence in Great Britain (n = 444), Germany (n = 454), and France (n = 463). In Study 2, we used a publicly available dataset consisting of 66 samples from 30 countries (N = 13,495) to again assess the measurement equivalence of the scale. Results indicated (partial) metric equivalence, but not scalar equivalence in both studies. Overall, the studies indicate that mean comparisons across the examined countries are not warranted with the current form of the G-SJS. The scale needs to be revised for valid cross-country comparisons of means.

SENNA Inventory for the Assessment of Social and Emotional Skills in Public School Students in Brazil: Measuring Both Identity and Self-Efficacy

Responding to the need for school-based, broadly applicable, low-cost, and brief assessments of socio-emotional skills, we describe the conceptual background and empirical development of the SENNA inventory and provide new psychometric information on its internal structure. Data were obtained through a computerized survey from 50,000 Brazilian students enrolled in public school grades 6 to 12, spread across the entire State of São Paulo. The SENNA inventory was designed to assess 18 particular skills (e.g., empathy, responsibility, tolerance of frustration, and social initiative), each operationalized by nine items that represent three types of items: three positively keyed trait-identity items, three negatively keyed identity items, and three (always positively keyed) self-efficacy items, totaling a set of 162 items. Results show that the 18 skill constructs empirically defined a higher-order structure that we interpret as the social-emotional Big Five, labeled as Engaging with Others, Amity, Self-Management, Emotional Regulation, and Open-Mindedness. The same five factors emerged whether we assessed the 18 skills with items representing (a) a trait-identity approach that emphasizes lived skills (what do I typically do?) or (b) a self-efficacy approach that emphasizes capability (how well can I do that?). Given that its target youth group is as young as 11 years old (grade 6), a population particularly prone to the response bias of acquiescence, SENNA is also equipped to correct for individual differences in acquiescence, which are shown to systematically bias results when not corrected.

Respondent Burden Effects on Item Non-Response and Careless Response Rates: An Analysis of Two Types of Surveys

The respondent burden refers to the effort required by a respondent to answer a questionnaire. Although this concept was introduced decades ago, few studies have focused on the quantitative detection of such a burden. In this paper, a face-to-face survey and a telephone survey conducted in Valencia (Spain) are analyzed. The presence of burden is studied in terms of both item non-response rates and careless response rates. In particular, two moving-window statistics based on the coefficient of unalikeability and the average longstring index are proposed for characterizing careless responding. Item non-response and careless response rates are modeled for each survey by using mixed-effects models, including respondent-level and question-level covariates and also temporal random effects to assess the existence of respondent burden during the questionnaire. The results suggest that the sociodemographic characteristics of the respondents and the typology of the question impact item non-response and careless response rates. Moreover, the estimates of the temporal random effects indicate that item non-response and careless response rates are time-varying, suggesting the presence of respondent burden. In particular, an increasing trend in item non-response rates in the telephone survey has been found, which supports the hypothesis of the burden. Regarding careless responding, despite the presence of some temporal variation, no clear trend has been identified.

Mental health and its association with coping strategies and intolerance of uncertainty during the COVID-19 pandemic among the general population in Saudi Arabia: cross-sectional study

BACKGROUND

The COVID-19 pandemic has had a major impact on people's lives globally. The outbreak in Saudi Arabia worsened when the number of cases and deaths rose in March and April of 2020, leading to a national lockdown. This study aimed to assess the factors associated with mental health symptoms in a sample of people residing in Saudi Arabia during the COVID-19 pandemic.

METHODS

We conducted an observational cross-sectional study using an online survey distributed via social media, completed by 3032 respondents from all Saudi regions. We collected demographic data, illness history, and scores of validated self-report scales to assess mental health symptoms, intolerance of uncertainty, and coping strategies.

RESULTS

In total, respondents indicated moderate to very severe symptoms during the pandemic as follows: 20.9% for depression, 17.5% for anxiety, and 12.6% for stress. Younger age, female gender, and history of mental illness were associated with higher levels of depression, anxiety, stress, and insomnia. Intolerance of uncertainty and certain coping strategies (such as denial or self-blame) were associated with more severe symptoms.

CONCLUSIONS

Mental health is a key concern during the COVID-19 pandemic, especially for the identified vulnerable groups. Agencies concerned with mental health during crises may use the studied associated factors of mental health symptoms to generate targeted policies or interventions.

Alexithymia, Prior Trauma, Alcohol Use, and Sexual Aggression Perpetration: A Cross-Cultural Comparison of a Moderated Mediation Model

In this study, we examined the relationships between prior trauma, alexithymia, and sexual aggression perpetration among 610 U.S. college students and 107 college students from the Philippines utilizing a cross-sectional retrospective design. We tested a moderated mediation model with alexithymia as the mediator between prior trauma and sexual aggression perpetration, and alcohol use as a moderator of the alexithymia-sexual aggression link. Moreover, given that cultural norms may influence these relationships, we also examined the structural invariance of the proposed moderated mediation model. Path analyses and multiple group analysis were used to examine the moderated mediation model, and examine model differences between samples. We did not find evidence for alexithymia as a mediator, but there was a significant interaction between alcohol use severity and alexithymia on sexual aggression perpetration across both samples. Alexithymia is a key variable in understanding the alcohol use-sexual aggression perpetration relationship. Clinical implications and recommendations for future research are also discussed.

Asymptotically Corrected Person Fit Statistics for Multidimensional Constructs with Simple Structure and Mixed Item Types

Person fit statistics are frequently used to detect aberrant behavior when assuming an item response model generated the data. A common statistic, [Formula: see text], has been shown in previous studies to perform well under a myriad of conditions. However, it is well-known that [Formula: see text] does not follow a standard normal distribution when using an estimated latent trait. As a result, corrections of [Formula: see text], called [Formula: see text], have been proposed in the literature for specific item response models. We propose a more general correction that is applicable to many types of data, namely survey or tests with multiple item types and underlying latent constructs, which subsumes previous work done by others. In addition, we provide corrections for multiple estimators of [Formula: see text], the latent trait, including MLE, MAP and WLE. We provide analytical derivations that justifies our proposed correction, as well as simulation studies to examine the performance of the proposed correction with finite test lengths. An applied example is also provided to demonstrate proof of concept. We conclude with recommendations for practitioners when the asymptotic correction works well under different conditions and also future directions.

Can intelligent agents improve data quality in online questiosnnaires? A pilot study

We explored the utility of chatbots for improving data quality arising from collection via sonline surveys. Three-hundred Australian adults sampled via Prolific Academic were randomized across chatbot-supported or unassisted online questionnaire conditions. The questionnaire comprised validated measures, along with challenge items formulated to be confusing yet aligned with the validated targets. The chatbot condition provided optional assistance with item clarity via a virtual support agent. Chatbot use and user satisfaction were measured through session logs and user feedback. Data quality was operationalized as between-group differences in relationships among validated and challenge measures. Findings broadly supported chatbot utility for online surveys, showing that most participants with chatbot access utilized it, found it helpful, and demonstrated modestly improved data quality (vs. controls). Absence of confusion for one challenge item is believed to have contributed to an underestimated effect. Findings show that assistive chatbots can enhance data quality, will be utilized by many participants if available, and are perceived as beneficial by most users. Scope constraints for this pilot study are believed to have led to underestimated effects. Future testing with longer-form questionnaires incorporating expanded item difficulty may further understanding of chatbot utility for survey completion and data quality.

Is it better not to think about it? Effects of positive and negative perfectionistic cognitions when there is increased pressure to perform

With the present research, we aimed to extend prior knowledge on the associations between dispositional perfectionism, perfectionistic cognitions and indicators of distress by considering both the level of between-person differences and the level of within-person fluctuations. We conducted two studies in which we assessed dispositional perfectionism once, plus we assessed perfectionistic cognitions across a predefined period of time (Study 1) and on a daily level (Study 2) during a highly demanding period of time. We found evidence for differential bivariate and unique associations of perfectionistic strivings cognitions (PSC) and perfectionistic concerns cognitions (PCC) with dispositional perfectionistic strivings (DPS) and dispositional perfectionistic concerns (DPC), perfectionism-specific cues, and indicators of distress. Beyond this, PCC emerged as an important mediator of the effects of DPS and DPC on distress.

The Occupational Depression Inventory: A new tool for clinicians and epidemiologists

BACKGROUND

Depressive symptoms induced by insurmountable job stress and sick leave for mental health reasons have become a focal concern among occupational health specialists. The present study introduces the Occupational Depression Inventory (ODI), a measure designed to quantify the severity of work-attributed depressive symptoms and establish provisional diagnoses of job-ascribed depression. The ODI comprises nine symptom items and a subsidiary question assessing turnover intention.

METHODS

A total of 2254 employed individuals were recruited in the U.S., New Zealand, and France. We examined the psychometric and structural properties of the ODI as well as the nomological network of work-attributed depressive symptoms. We adopted an approach centered on exploratory structural equation modeling (ESEM) bifactor analysis. We developed a diagnostic algorithm for identifying likely cases of job-ascribed depression (SPSS syntax provided).

RESULTS

The ODI showed strong reliability and high factorial validity. ESEM bifactor analysis indicated that, as intended, the ODI can be used as a unidimensional measure (Explained Common Variance = 0.891). Work-attributed depressive symptoms correlated in the expected direction with our other variables of interest-e.g., job satisfaction, general health status-and were markedly associated with turnover intention. Of our 2254 participants, 7.6% (n = 172) met the criteria for a provisional diagnosis of job-ascribed depression.

CONCLUSIONS

This study suggests that the ODI constitutes a sound measure of work-attributed depressive symptoms. The ODI may help occupational health researchers and practitioners identify, track, and treat job-ascribed depression more effectively. ODI-based research may contribute to informing occupational health policies and regulations in the future.