Detecting Social Desirability Bias Using Factor Mixture Models

Identifying person misfit using the person backward stepwise reliability curve (PBRC)

The goal of the present study was to propose a visualization of aberrant response patterns based on the idea put forth by the Cronbach-Mesbach curve. First, an index of person reliability is developed using the K-R 20 formula followed by a backward stepwise procedure in which one person at a time is deleted from the model. Observations for which reliability is no longer monotonically increasing suggest that they are candidates for aberrant responding. Using data from the quantitative domain of a national aptitude test the proposed visualization technique was demonstrated. The external validity of the procedure was tested by contrasting the person fit reliability estimates with those derived from other indices of aberrant responding such as the Ht. Results indicated that individuals not covarying with other individuals concerning their response patterns and concordance to the measurement of a unified latent trait were identified by both the present procedure and Ht and U3 at a rate of 100%. By plotting those individuals using Person Response Curves (PRCs) results confirmed the lack of monotonicity in the relationship between item difficulty and person skill. Consequently, results confirm the usefulness of the present methodology as an index for identifying responders who manifest themselves with aberrant responses and who are not conducive to the measurement of the latent trait.

A Comparison of Person-Fit Indices to Detect Social Desirability Bias

Social desirability bias (SDB) has been a major concern in educational and psychological assessments when measuring latent variables because it has the potential to introduce measurement error and bias in assessments. Person-fit indices can detect bias in the form of misfitted response vectors. The objective of this study was to compare the performance of 14 person-fit indices to identify SDB in simulated responses. The area under the curve (AUC) of receiver operating characteristic (ROC) curve analysis was computed to evaluate the predictive power of these statistics. The findings showed that the agreement statistic ( A ) outperformed all other person-fit indices, while the disagreement statistic ( D ) , dependability statistic ( E ) , and the number of Guttman errors ( G ) also demonstrated high AUCs to detect SDB. Recommendations for practitioners to use these fit indices are provided.

Chinese nurses' preparedness and opinions in responding to intimate partner violence: An adaptation and validation study

AIM

To adapt and psychometrically test two scales from The Physician Readiness to Manage Intimate Partner Violence Survey (PREMIS) for measuring nurses' preparedness to manage intimate partner violence (IPV) and opinions on IPV in China.

DESIGN

A cross-sectional study for cross-cultural adaptation and validation of instruments.

METHODS

The study was conducted in two tertiary hospitals in northeastern and southwestern China in 2020: (Step 1) translation and back-translation by four translators; (Step 2) expert consultation to assess content validity by a five-person panel; (Step 3) cognitive debriefing to evaluate the online survey with six nurses; (Step 4) pilot testing to finalize the survey with 79 nurses; and (Step 5) psychometric evaluation of construct validity and internal consistency reliability including post hoc analysis with 1071 nurses.

RESULTS

The Chinese scales of Perceived Preparation and Opinions demonstrated generally acceptable content validity, construct validity and internal consistency reliability; some factor analysis results were uninterpretable, and socially desirable responses existed. A post hoc inspection of item and score distributions assisted the item classification into Confident, Cautious and Concerned regarding the authenticity of nurses' responses.

CONCLUSION

Reliability and construct validity of the Chinese versions of Perceived Preparation and Opinions were supported in this sample. The instrument measures nurses' preparedness and opinions towards IPV. Item classification into Confident, Cautious and Concerned categories is a pragmatic way to interpret the results.

IMPACT

The Chinese scales can be used to measure nurses' preparedness and opinions towards IPV in China. The study also provides an approach for interpretation of results and identification of socially desirable responses in validation and survey studies in other contexts. The adaptation and use of the Chinese scales serve as a primary step in responding to IPV in China's health sector and can be used as a template for adapting the instrument in other low- and middle-income countries.

The reading profiles of late elementary English learners with and without risk for dyslexia

Few studies have systematically investigated the reading skill profiles of English learners (ELs) in late elementary school, a critical developmental period for language and literacy and the most common grades for initial identification with specific learning disabilities (O'Connor et al., Learning Disabilities Research & Practice, 28(3), 98-112, 2013). We investigated the reading skill profiles of 331 ELs in 3rd and 4th grades, including ELs with and without risk for dyslexia due to significant deficits in word and pseudo-word reading accuracy and fluency. We utilized latent profile analysis and factor mixture modeling to investigate (1) the nature and distribution of reading skill profiles; (2) whether these profiles were associated with differences in reading comprehension growth across one academic year; and (3) the stability of reading profiles across an academic year. We selected a two-class solution (reading disabled and typically developing) based on model fit indices, theoretical considerations, pattern of results across profile-solutions and time-points, and parameterizations, making the approach stronger and more generalizable. These classes demonstrated clear, consistent differences in performance across reading component skills, with the RD class scoring consistently below the TD class across code-based and meaning-based domains of reading. Across the year, the TD class demonstrated significantly higher patterns of growth in reading comprehension (χ2 (1) = 206.21, p < 0.001). Class membership was largely stable (97% of participants maintain class membership). These results suggest that ELs with risk for dyslexia demonstrate multiple component skill deficits that may require long-term, comprehensive, intensive interventions to remediate.

How the characteristics of pediatric neurologists in Latin America influence the communication of sudden unexpected death in epilepsy to patients and caregivers

Objective

This study aimed to describe the characteristics of pediatric neurologists (PNs) in Latin America (LA) who attend to children and adolescents with epilepsy and convey to them the risk of sudden unexpected death in epilepsy (SUDEP).

Methods

Personal data and details of discussion of SUDEP with families, including relevance of SUDEP disclosure, frequency of such communication, perceived benefits and risks of disclosure, extent of training received on such disclosure, and professional experience with SUDEP, were collected through an online survey of PNs from LA. Their personal experience in carrying out this conversation was obtained through responses to an open question, further used to identify the main barriers.

Results

Of the 442 surveys received, 367 (83%) were analyzed. Most participants (73.8%) responded that the communication of SUDEP risk was relevant or very relevant; however, only 17.9% reported communicating it always or very frequently. Factors that increased the frequency of SUDEP communication included patients with higher levels of complexity (OR = 2.18, P = .003) and the physician's personal experience with SUDEP (OR = 2.305, P < .001). Direct questions from the family and avoiding scaring them about a rare outcome were the main motivations behind discussing and not discussing SUDEP, respectively. In the open question, respondents identified worries about the patient's ability to understand the information and cultural gaps as barriers. “Informing with the intention of improving adherence to treatment” and “establishing an empathic relationship” were significantly related. Further, the concept of “do not scare” was significantly related to “personal difficulties in discussing SUDEP.”

Significance

Although most PNs agree that communication about SUDEP is relevant, only a minority actually engages in it. Participants identified a lack of appropriate training in such communication as a barrier. A better understanding of communication expectations, education of health professionals, and communication techniques have a strong relevance in diminishing the gap between guidelines and practice.

Correlates of self-rated health in Southern Europe: evidences from national representative samples in Italy and Serbia

Background

Self-rated health (SRH) is a widely adopted tool to compare health across countries. Our aim was to examine SRH differences between Italy and Serbia and to observe the role of predictors of SRH referring to health behaviors within and between both countries.

Methods

We used cross-sectional population-based data from Italian and Serbian national health surveys carried out in 2013. Post hoc cross-standardization was undertaken to ensure that the information from both data sets was comparable.

Results

Univariate and multivariate multinomial logistic regressions showed that Serbians reported bad-SRH significantly more often than Italians. Moreover, consistently across national groups, younger participants, males, higher educated participants and participants with lower body mass index (BMI) had more chances than older, lower educated and higher BMI participants, respectively, to report better SRH. Finally, smoking and drinking behaviors did not correlate with SRH, while the frequency of fruits and vegetables intake was differently associated with SRH across countries.

Conclusion

Health assessments based on SRH in Italian and Serbian national surveys are directly comparable and show similar relationships with socio-demographic correlates and BMI. However, the effect of health behaviors on SRH may differ according to national and cultural contexts.

The Effect of Faking on the Correlation Between Two Ordinal Variables: Some Population and Monte Carlo Results

Correlational measures are probably the most spread statistical tools in psychological research. They are used by researchers to investigate, for example, relations between self-report measures usually collected using paper-pencil or online questionnaires. Like many other statistical analysis, also correlational measures can be seriously affected by specific sources of bias which constitute serious threats to the final observed results. In this contribution, we will focus on the impact of the fake data threat on the interpretation of statistical results for two well-know correlational measures (the Pearson product-moment correlation and the Spearman rank-order correlation). By using the Sample Generation by Replacement (SGR) approach, we analyze uncertainty in inferences based on possible fake data and evaluate the implications of fake data for correlational results. A population-level analysis and a Monte Carlo simulation are performed to study different modulations of faking on bivariate discrete variables with finite supports and varying sample sizes. We show that by using our paradigm it is always possible, under specific faking conditions, to increase (resp. decrease) the original correlation between two discrete variables in a predictable and systematic manner.

A (Latent) Class of Their Own: Response Patterns in Trading Off Quantity and Quality of Life in Time Trade-Off Exercises

BACKGROUND

Conflicting results regarding associations of time trade-off (TTO) valuations with respondent characteristics have been reported, mostly on the basis of regression analyses. Alternative approaches, such as the latent class analysis (LCA), may add to the further understanding of variations in TTO responses.

OBJECTIVES

To identify whether subgroups of respondents can be identified on the basis of their responses to TTO exercises and to investigate which respondent characteristics are associated with membership of the identified subgroups.

METHODS

Members of the Dutch general public, aged 18 to 65 years, completed a Web-based questionnaire concerning sociodemographic characteristics, three TTO exercises valuing health states described using the domains of the EuroQol five-dimensional questionnaire, and preference for quality versus quantity of life. LCA was used to identify patterns in the responses. Predictive variables were included in the final LCA model to identify the particular respondent characteristics that predict subgroup membership.

RESULTS

The sample consisted of 1067 respondents. Four latent classes were identified in the responses to TTO exercises. Two were high traders, focusing on quality of life and trading off a relatively high number of years. The other two were low traders, focusing on length of life. Predictive analyses revealed significant differences between subgroups in terms of age, sex, subjective life expectancy, and preference for quantity over quality of life.

CONCLUSIONS

We showed that distinct classes of respondents can be discerned in TTO responses from the general public, distinguishing subgroups of low and high traders. More research in this area should confirm our findings and investigate their implications for health state valuation exercises.

The use of latent variable mixture models to identify invariant items in test construction

Purpose

Patient-reported outcome measures (PROMs) are frequently used in heterogeneous patient populations. PROM scores may lead to biased inferences when sources of heterogeneity (e.g., gender, ethnicity, and social factors) are ignored. Latent variable mixture models (LVMMs) can be used to examine measurement invariance (MI) when sources of heterogeneity in the population are not known a priori. The goal of this article is to discuss the use of LVMMs to identify invariant items within the context of test construction.

Methods

The Draper-Lindely-de Finetti (DLD) framework for the measurement of latent variables provides a theoretical context for the use of LVMMs to identify the most invariant items in test construction. In an expository analysis using 39 items measuring daily activities, LVMMs were conducted to compare 1- and 2-class item response theory models (IRT). If the 2-class model had better fit, item-level logistic regression differential item functioning (DIF) analyses were conducted to identify items that were not invariant. These items were removed and LVMMs and DIF testing repeated until all remaining items showed MI.

Results

The 39 items had an essentially unidimensional measurement structure. However, a 1-class IRT model resulted in many statistically significant bivariate residuals, indicating suboptimal fit due to remaining local dependence. A 2-class LVMM had better fit. Through subsequent rounds of LVMMs and DIF testing, nine items were identified as being most invariant.

Conclusions

The DLD framework and the use of LVMMs have significant potential for advancing theoretical developments and research on item selection and the development of PROMs for heterogeneous populations.

Assessment of Differential Rater Functioning in Latent Classes with New Mixture Facets Models

Multifaceted data are very common in the human sciences. For example, test takers' responses to essay items are marked by raters. If multifaceted data are analyzed with standard facets models, it is assumed there is no interaction between facets. In reality, an interaction between facets can occur, referred to as differential facet functioning. A special case of differential facet functioning is the interaction between ratees and raters, referred to as differential rater functioning (DRF). In existing DRF studies, the group membership of ratees is known, such as gender or ethnicity. However, DRF may occur when the group membership is unknown (latent) and thus has to be estimated from data. To solve this problem, in this study, we developed a new mixture facets model to assess DRF when the group membership is latent and we provided two empirical examples to demonstrate its applications. A series of simulations were also conducted to evaluate the performance of the new model in the DRF assessment in the Bayesian framework. Results supported the use of the mixture facets model because all parameters were recovered fairly well, and the more data there were, the better the parameter recovery.

Empirical Scenarios of Fake Data Analysis: The Sample Generation by Replacement (SGR) Approach

Many self-report measures of attitudes, beliefs, personality, and pathology include items whose responses can be easily manipulated or distorted, as an example in order to give a positive impression to others, to obtain financial compensation, to avoid being charged with a crime, to get a job, or else. This fact confronts both researchers and practitioners with the crucial problem of biases yielded by the usage of standard statistical models. The current paper presents three empirical applications to the issue of faking of a recent probabilistic perturbation procedure called Sample Generation by Replacement (SGR; Lombardi and Pastore, 2012). With the intent to study the behavior of some statistics under fake perturbation and data reconstruction processes, ad-hoc faking scenarios were implemented and tested. Overall, results proved that SGR could be successfully applied both in the case of research designs traditionally proposed in order to deal with faking (e.g., use of fake-detecting scales, experimentally induced faking, or contrasting applicants vs. incumbents), and in the case of ecological research settings, where no information as regards faking could be collected by the researcher or the practitioner. Implications and limitations are presented and discussed.

Drugs As Instruments: Describing and Testing a Behavioral Approach to the Study of Neuroenhancement

Neuroenhancement (NE) is the non-medical use of psychoactive substances to produce a subjective enhancement in psychological functioning and experience. So far empirical investigations of individuals' motivation for NE however have been hampered by the lack of theoretical foundation. This study aimed to apply drug instrumentalization theory to user motivation for NE. We argue that NE should be defined and analyzed from a behavioral perspective rather than in terms of the characteristics of substances used for NE. In the empirical study we explored user behavior by analyzing relationships between drug options (use over-the-counter products, prescription drugs, illicit drugs) and postulated drug instrumentalization goals (e.g., improved cognitive performance, counteracting fatigue, improved social interaction). Questionnaire data from 1438 university students were subjected to exploratory and confirmatory factor analysis to address the question of whether analysis of drug instrumentalization should be based on the assumption that users are aiming to achieve a certain goal and choose their drug accordingly or whether NE behavior is more strongly rooted in a decision to try or use a certain drug option. We used factor mixture modeling to explore whether users could be separated into qualitatively different groups defined by a shared "goal × drug option" configuration. Our results indicate, first, that individuals' decisions about NE are eventually based on personal attitude to drug options (e.g., willingness to use an over-the-counter product but not to abuse prescription drugs) rather than motivated by desire to achieve a specific goal (e.g., fighting tiredness) for which different drug options might be tried. Second, data analyses suggested two qualitatively different classes of users. Both predominantly used over-the-counter products, but "neuroenhancers" might be characterized by a higher propensity to instrumentalize over-the-counter products for virtually all investigated goals whereas "fatigue-fighters" might be inclined to use over-the-counter products exclusively to fight fatigue. We believe that psychological investigations like these are essential, especially for designing programs to prevent risky behavior.

What Is the Nature of Faking? Modeling Distinct Response Patterns and Quantitative Differences in Faking at the Same Time

Research trying to uncover the true nature of faking is currently dominated by two competing modeling approaches. One approach views faking as the manifestation of distinct and qualitatively different response patterns. Typically, mixed Rasch models are used within this approach. The alternative approach views faking as a continuous and quantitative variable resulting from the interaction between test taker personality and situational demands. Modeling techniques for this approach range from regression analyses to structural equation modeling. So far, there has been no study in which both modeling approaches have been applied within one data set. More importantly, so far there has been no methodological model in which both views of faking could have been modeled simultaneously. Within the present article such a modeling approach is introduced and applied to a data set of N = 497 applicants. By combining factor mixture modeling with a latent change score model, it was possible to test both views of faking within the same model. Findings support the view of faking mainly as a continuous and quantitative variable. Theoretical implications are discussed.

A Note on Parameter Estimate Comparability: Across Latent Classes in Mixture IRT Modeling

The use of mixture item response theory modeling is exemplified typically by comparing item profiles across different latent groups. The comparisons of item profiles presuppose that all model parameter estimates across latent classes are on a common scale. This note discusses the conditions and the model constraint issues to establish a common scale across latent classes.

I-DAQ: a new test to assess direct and indirect aggression free of response bias

Self-reports of aggression are deeply impacted by response bias, especially by social desirability, but there are no specific methods for controlling this bias. Furthermore, despite the importance of the subject few instruments have been designed to assess both direct and indirect forms of aggression. The aim of the present research was to develop a brief measure that comprises both forms of aggression and which makes it possible to obtain scores free of social desirability and acquiescence effects. The scales were created using recently developed methods for controlling response bias effects in a sample of 750 participants over a wide age range. The items were chosen by a panel of judges from among the best of the existing aggression measures. Confirmatory factor analysis showed the expected three factor structure (CFI = .98; AGFI = .97 and RSMEA = .078, 90% C.I. = .074 - .083) and the scales showed good psychometric properties in that they had good reliability (ranging from θxx = .77 to θxx = .83), and convergent and criterion validity.

Feasibility of audit methods to study access to substance use treatment

Audit studies represent an emerging method for examining disparities in access to care, like substance use treatment, whereby fake patients (i.e., actors) attempt to procure a service with one or more characteristics isolated across condition. This allows for manipulation of variables, like insurance status, that are normally fixed or impossible to standardize with precision when studying actual patients. This pilot study explored whether these methods were feasible for the examination of community-based substance use treatment access. Masked telephone calls (n=48) were made to providers (k=8) in a single city seeking an appointment. A male and female "patient" made calls in three insurance status conditions: no insurance, state-funded insurance, and private insurance. All other subject characteristics were held constant. Results showed an audit design to be a feasible method for examining disparities in access and demonstrated substantial barriers to voluntary treatment. Implications and future directions are discussed.

Identifying differential responders and their characteristics in clinical trials: innovative methods for analyzing longitudinal data

OBJECTIVES

To present a step-by-step example of the examination of heterogeneity within clinical trial data by using a growth mixture modeling (GMM) approach.

METHODS

Secondary data from a longitudinal double-blind clinical drug study were used. Patients received enalapril or placebo and were followed for 2 years during the drug component, followed by a 3-year postdrug component. Primary variables of interest were creatinine levels during the drug component and number of hospitalizations in the postdrug component. Latent growth modeling (LGM) methods were used to examine the treatment response variability in the data. GMM methods were applied where substantial variability was found to identify latent (unobserved) subsets of differential responders, using treatment groups as known classes. Post hoc analyses were applied to characterize emergent subgroups.

RESULTS

LGM methods demonstrated a large variability in creatinine levels. GMM methods identified two subsets of patients for each treatment group. Placebo class 2 (7.0% of the total sample) and enalapril class 2 (8.5%) include individuals whose creatinine levels start at 1.114 mg/dl and 1.108 mg/dl, respectively, and show worsening (slopes: 0.023 and 0.017, respectively). Placebo class 1 (43.1%) and enalapril class 1 (41.4%) individuals start with lower creatinine levels (1.082 and 1.083 mg/dl, respectively) and show very minimal change (0.008 and 0.003, respectively). Post hoc analyses revealed significant differences between placebo/enalapril class 1 and placebo/enalapril class 2 in terms of New York Heart Association functional ability, depression, functional impairment, creatinine levels, mortality, and hospitalizations.

CONCLUSIONS

GMM methods can identify subsets of differential responders in clinical trial data. This can result in a more accurate understanding of treatment effects.

Vividness of visual imagery and social desirable responding: correlations of the vividness of visual imagery questionnaire with the balanced inventory of desirable responding and the Marlowe-Crowne scale

Correlational research investigating the relationship between scores on self-report imagery questionnaires and measures of social desirable responding has shown only a weak association. However, researchers have argued that this research may have underestimated the size of the relationship because it relied primarily on the Marlowe-Crowne scale (MC; Crowne & Marlowe, Journal of Consulting Psychology, 24, 349-354, 1960), which loads primarily on the least relevant form of social desirable responding for this particular context, the moralistic bias. Here we report the analysis of data correlating the Vividness of Visual Imagery Questionnaire (VVIQ; Marks, Journal of Mental Imagery, 19, 153-166, 1973) with the Balanced Inventory of Desirable Responding (BIDR; Paulhus, 2002) and the MC scale under anonymous testing conditions. The VVIQ correlated significantly with the Self-Deceptive Enhancement (SDE) and Agency Management (AM) BIDR subscales and with the MC. The largest correlation was with SDE. The ability of SDE to predict VVIQ scores was not significantly enhanced by adding either AM or MC. Correlations between the VVIQ and BIDR egoistic scales were larger when the BIDR was continuously rather than dichotomously scored. This analysis indicates that the relationship between self-reported imagery and social desirable responding is likely to be stronger than previously thought.

Application of latent growth and growth mixture modeling to identify and characterize differential responders to treatment for COPD

OBJECTIVE

To explore the utility of applying growth mixture models (GMMs) in secondary analyses of clinical trials to identify sources of variability in data reported by patients with COPD.

METHODS

Analyses were performed on data from two 6-month clinical trials comparing indacaterol and open-label tiotropium or blinded salmeterol and the first six months of a 12-month trial comparing indacaterol and blinded formoterol. Latent growth model (LGM) analyses were conducted to explore the response of the SGRQ Symptoms score from baseline to six months and GMM analyses were evaluated as a method to identify latent classes of differential responders.

RESULTS

Variability in SGRQ Symptom scores was found suggesting subsets of patients with differential response to treatment. GMM analyses found subsets of non-responders in all trials. When the responders were analyzed separately from non-responders, there were increased treatment effects (e.g., symptoms score improvement over six months for whole groups: indacaterol=8-12 units, tiotropium=7 units, salmeterol=9 units, formoterol=11 units. Responder subgroup improvement: indacaterol=9-21 units, tiotropium=7 units, salmeterol=10 units, formoterol=20 units). Responders had significantly different baseline SGRQ Symptom scores, smoking history, age, and mMRC dyspnea scores than non-responders.

CONCLUSIONS

Patients with COPD represent a heterogeneous population in terms of their reporting of symptoms and response to treatment. GMM analyses are able to identify sub-groups of responders and non-responders. Application of this methodology could be of value on other endpoints in COPD and in other disease areas.