Explanatory, Multilevel Person-Fit Analysis of Response Consistency on the Spielberger State-Trait Anxiety Inventory

Studies in the Mentality of Literates: 3. Conceptual Structure and Nonsense of Personality Testing

In this study, third in a series of studies of the relationships between the dominant type of the Word Meaning Structure (WMS) and various psychic processes, response patterns on personality questionnaires with Likert-type response format of individuals with different levels of education (including adult illiterates) in Brazil (N = 102) and in Estonia (N = 520) were assessed with person oriented methods of data analysis. We found that responses to two personality questionnaires (International Personality Item Pool Questionnaire, IPIP-Q60 and Estonian Collectivism Scale, ESTCOL) are inconsistent and do not correspond to theories that underlie construction and interpretation of such assessment tools. Two novel ways to assess inconsistent response patterns were developed. The Consistency Index (CI) characterizes between-item inconsistency and the Determinacy Index (DI) characterizes within-item inconsistency. The dominant type of the WMS and the level of education were related to both CI and DI. Higher level of between-item inconsistency characterizes everyday conceptual thinkers with lower levels of education and higher level of within-item inconsistency was observed among logical conceptual thinkers with higher levels of education. Systematic relationships between WMS and inconsistent patterns of responses indicate that responses on personality questionnaires cannot be interpreted in terms of personality characteristics. The results of our study also provide further support to the idea that dominant type of the WMS is a pervasive characteristic of the psyche and determines qualitatively possibilities and limits of the psychic processes. The results of this study are in agreement with the idea that WMS defines the "Great Divide."

Predicting Dropout from Inpatient Substance Use Disorder Treatment: A Prospective Validation Study of the OQ-Analyst

Background and Aims:

There is an urgent need for tools allowing therapists to identify patients at risk of dropout. The OQ-Analyst, an increasingly popular computer-based system, is used to track patient progress and predict dropout. However, we have been unable to find empirical documentation regarding the ability of OQ-Analyst to predict dropout. The aim of the present study was to perform the first direct test of the ability of the OQ-Analyst to predict dropout.

Design:

Patients were consecutively enlisted in a naturalistic, prospective, longitudinal clinical trial. As interventions based on feedback from the OQ-Analyst could alter the outcome and potentially render the prediction wrong, feedback was withheld from patients and therapists.

Setting:

The study was carried out during 2011–2013 in an inpatient substance use disorder clinic in Oslo, Norway.

Participants:

Patients aged 18 to 28 years who met criteria for a principal diagnosis of mental or behavioural disorder due to psychoactive substance use (ICD 10; F10.2–F19.2).

Measurements:

Red signal (predictions of high risk) from the Norwegian version of the OQ-Analyst were compared with dropouts identified using patient medical records as the standard for predictive accuracy.

Findings:

A total of 40 patients completed 647 OQ assessments resulting in 46 red signals. There were 27 observed dropouts, only one of which followed after a red signal. Patients indicated by the OQ-Analyst as being at high risk of dropping out were no more likely to do so than those indicated as being at low risk. Random intercept logistic regression predicting dropout from a red signal was statistically nonsignificant. Bayes factor supports no association.

Conclusions:

The study does not support the predictive ability of the OQ-Analyst for the present patient population. In the absence of empirical evidence of predictive ability, it may be better not to assume such ability.

Person misfit on the Inventory of Depressive Symptomatology: Low quality self-report or true atypical symptom profile?

Person misfit on a self-report measure refers to a response pattern that is unlikely given a theoretical measurement model. Person misfit may reflect low quality self-report data, for example due to random responding or misunderstanding of items. However, recent research in the context of psychopathology suggests that person misfit may reflect atypical symptom profiles that have implications for diagnosis or treatment. We followed-up on Wanders et al. (Journal of Affective Disorders, 180, 36-43, 2015) who investigated person misfit on the Inventory of Depressive Symptomatology (IDS) in the Netherlands Study of Depression and Anxiety (n = 2,981). Our goal was to investigate the extent to which misfit on the IDS reflects low-quality self-report patterns and the extent to which it reflects true atypical symptom profiles. Regression analysis showed that person misfit related more strongly to self-report quality indicators than to variables quantifying theoretically-derived atypical symptom profiles. A data-driven atypical symptom profile explained most variance in person misfit, suggesting that person misfit on the IDS mainly reflects a sample- and questionnaire-specific atypical symptom profile. We concluded that person-fit statistics are useful for detecting IDS scores that may not be valid. Further research is necessary to support the interpretation of person misfit as reflecting a meaningful atypical symptom combination.

Longitudinal measurement invariance of the Dutch Outcome Questionnaire-45 in a clinical sample

Purpose

In the absence of measurement invariance across measurement occasions, change scores based on pretest–posttest measurements may be inaccurate representations of real change on the latent variable. In this study, we examined whether measurement invariance held in the Dutch version of Outcome Questionnaire-45 (OQ-45).

Method

Using secondary data analysis of a sample of N = 540 Dutch outpatients, we tested the stability of the factorial structure (gamma change) and the metric and scalar invariance (beta change) across pretest and posttest measurements using a combination of factor analysis and item response theory methodology.

Results

Results revealed a stable factorial structure from pretest to posttest and minor violations of metric invariance for two items in the Dutch OQ-45.

Conclusion

Even though for two items the assumption of invariance was violated, results suggest that the effects of these violations on practical change assessment using the OQ-45 were negligible.

Identifying Person-Fit Latent Classes, and Explanation of Categorical and Continuous Person Misfit

Latent class (LC) cluster analysis of a set of subscale l_z person-fit statistics was proposed to explain person misfit on multiscale measures. The proposed explanatory LC person-fit analysis was used to analyze data of students (N = 91,648) on the nine-subscale School Attitude Questionnaire Internet (SAQI). Inspection of the class-specific l_z mean and variance structure combined with explanatory analysis of class membership showed that the data included a poor-fit class, a class showing good fit combined with social desirability bias, a good-fit class, and two classes that were more difficult to interpret. A comparison of multinomial logistic regression predicting class membership and multiple regression predicting continuous person fit showed that LC cluster analysis provided information about aberrant responding unattainable by means of linear multiple regression. It was concluded that LC person-fit analysis has added value to common approaches to explaining aberrant responding to multiscale measures.

Detecting and Explaining Aberrant Responding to the Outcome Questionnaire-45

We applied item response theory based person-fit analysis (PFA) to data of the Outcome Questionnaire-45 (OQ-45) to investigate the prevalence and causes of aberrant responding in a sample of Dutch clinical outpatients. The [Formula: see text] person-fit statistic was used to detect misfitting item-score patterns and the standardized residual statistic for identifying the source of the misfit in the item-score patterns identified as misfitting. Logistic regression analysis was used to predict person misfit from clinical diagnosis, OQ-45 total score, and Global Assessment of Functioning code. The [Formula: see text] statistic classified 12.6% of the item-score patterns as misfitting. Person misfit was positively related to the severity of psychological distress. Furthermore, patients with psychotic disorders, somatoform disorders, or substance-related disorders more likely showed misfit than the baseline group of patients with mood and anxiety disorders. The results suggest that general outcome measures such as the OQ-45 are not equally appropriate for patients with different disorders. Our study emphasizes the importance of person-misfit detection in clinical practice.