Initial Validation of Short Forms of the SIMS for Neuropsychological Evaluations

Machine learning item selection for short scale construction: A proof-of-concept using the SIMS

ObjectiveThis proof-of-concept paper provides evidence to support machine learning (ML) as a valid alternative to traditional psychometric techniques in the development of short forms of longer parent psychological tests. ML comprises a variety of feature selection techniques that can be efficiently applied to identify the set of items that best replicates the characteristics of the original test. MethodsIn the present study, we integrated a dataset of 329 participants from published and unpublished datasets used in previous research on the Structured Inventory of Malingered Symptomatology (SIMS) to develop a short version of the scale. The SIMS is a multi-axial self-report questionnaire and a highly efficient psychometric measure of symptom validity, which is frequently applied in forensic settings. Results State-of-the-art ML item selection techniques achieved a 72% reduction in length while capturing 92% of the variance of the original SIMS. The new SIMS short form now consists of 21 items. ConclusionsThe results suggest that the proposed ML-based item selection technique represents a promising alternative to standard psychometric correlation-based methods (i.e. item selection, item response theory), especially when selection techniques (e.g. wrapper) are employed that evaluate global, rather than local, item value.

Preliminary findings from reevaluating the MMPI Response Bias Scale items in veterans undergoing neuropsychological evaluation

The Response Bias Scale (RBS) was developed to predict non-credible cognitive presentations among disability claimants without head injury. Developers used empirical keying, which is independent of apparent content, to select items from the Minnesota Multiphasic Personality Inventory-2 (MMPI-2) item pool that distinguished between individuals passing or failing performance validity tests (PVTs). No study has examined which of these items would have psychometric value when used in clinical neuropsychological evaluations. This study reexamined items comprising RBS with reference to manifest item content, internal consistency, PVTs, and a symptom validity test (SVT) in a sample of 173 predominately White male veterans (MAGE = 50.70, MEDU = 13.73) in a VA outpatient neuropsychology clinic. Participants completed the MMPI-2 Restructured Form (MMPI-2-RF), PVTs, and an SVT. The 28-item RBS appears to contain three types of items: those that manifestly address cognitive functioning, those that are supported but do not appear to address cognitive functioning, and nine items that were unrelated to cognition and not statistically supported. The 19 empirically supported items, or RBS-19, predicted PVT and SVT failures marginally better than the RBS. Both the RBS and RBS-19 had stronger relationships with SVTs relative to PVTs. Although the removal of the nine problematic items improved the diagnostic accuracy of the scale, it still did not reach the level that is generally considered to be clinically optimal. The RBS-19 offers a measure with improved internal consistency and predictive validity compared to the RBS and warrants additional research.