The Development of a Short Version of the SIMS Using Machine Learning to Detect Feigning in Forensic Assessment

Machine learning item selection for short scale construction: A proof-of-concept using the SIMS

ObjectiveThis proof-of-concept paper provides evidence to support machine learning (ML) as a valid alternative to traditional psychometric techniques in the development of short forms of longer parent psychological tests. ML comprises a variety of feature selection techniques that can be efficiently applied to identify the set of items that best replicates the characteristics of the original test. MethodsIn the present study, we integrated a dataset of 329 participants from published and unpublished datasets used in previous research on the Structured Inventory of Malingered Symptomatology (SIMS) to develop a short version of the scale. The SIMS is a multi-axial self-report questionnaire and a highly efficient psychometric measure of symptom validity, which is frequently applied in forensic settings. Results State-of-the-art ML item selection techniques achieved a 72% reduction in length while capturing 92% of the variance of the original SIMS. The new SIMS short form now consists of 21 items. ConclusionsThe results suggest that the proposed ML-based item selection technique represents a promising alternative to standard psychometric correlation-based methods (i.e. item selection, item response theory), especially when selection techniques (e.g. wrapper) are employed that evaluate global, rather than local, item value.

Reconstructing individual responses to direct questions: a new method for reconstructing malingered responses

Introduction

The false consensus effect consists of an overestimation of how common a subject opinion is among other people. This research demonstrates that individual endorsement of questions may be predicted by estimating peers' responses to the same question. Moreover, we aim to demonstrate how this prediction can be used to reconstruct the individual's response to a single item as well as the overall response to all of the items, making the technique suitable and effective for malingering detection.

Method

We have validated the procedure of reconstructing individual responses from peers' estimation in two separate studies, one addressing anxiety-related questions and the other to the Dark Triad. The questionnaires, adapted to our scopes, were submitted to the groups of participants for a total of 187 subjects across both studies. Machine learning models were used to estimate the results.

Results

According to the results, individual responses to a single question requiring a "yes" or "no" response are predicted with 70-80% accuracy. The overall participant-predicted score on all questions (total test score) is predicted with a correlation of 0.7-0.77 with actual results.

Discussion

The application of the false consensus effect format is a promising procedure for reconstructing truthful responses in forensic settings when the respondent is highly likely to alter his true (genuine) response and true responses to the tests are missing.

A Survey of Practices and Beliefs of Italian Psychologists Regarding Malingering and Symptom Validity Assessment

A few years ago, an article describing the current status of Symptom Validity Assessment (SVA) practices and beliefs in European countries reported that there was little research activity in Italy (Merten et al., 2013). The same article also highlighted that Italian practitioners were less inclined to use Symptom Validity Tests (SVTs) and Performance Validity Tests (PVTs) in their assessments, compared with their colleagues from other major European countries. Considering that several articles on malingering and SVA have been published by Italian authors in recent years, we concluded that an update of the practices and beliefs of Italian professionals regarding malingering and SVA would be beneficial. Accordingly, from a larger survey that examined general psychological assessment practices and beliefs of Italian professionals, we extracted a subset of items specifically related to malingering and SVA and analyzed the responses of a sample of Italian psychologists who have some experience with malingering-related assessments. Taken together, the results of our analyses indicated that even though our respondents tend to use SVTs and PVTs relatively often in their evaluations, at this time, they likely trust more their own personal observations, impressions, and overall clinical judgment, in their SVA practice. Additionally, our results also indicated that Italian practitioners with some familiarity with malingering-related evaluations consider malingering to occur in about one-third of psychological assessments in which the evaluee might have an interest in overreporting.

Symptom and Performance Validity Assessment in European Countries: an Update

In 2013, a special issue of the Spanish journal Clínica y Salud published a review on symptom and performance validity assessment in European countries (Merten et al. in Clínica y Salud, 24(3), 129–138, 2013). At that time, developments were judged to be in their infancy in many countries, with major publication activities stemming from only four countries: Spain, The Netherlands, Great Britain, and Germany. As an introduction to a special issue of Psychological Injury and Law, this is an updated report of developments during the last 10 years. In that period of time, research activities have reached a level where it is difficult to follow all developments; some validity measures were newly developed, others were adapted for European languages, and validity assessment has found a much stronger place in real-world evaluation contexts. Next to an update from the four nations mentioned above, reports are now given from Austria, Italy, and Switzerland, too.

Screening for Depression in Mobile Devices Using Patient Health Questionnaire-9 (PHQ-9) Data: A Diagnostic Meta-Analysis via Machine Learning Methods

PURPOSE

Depression is a symptom commonly encountered in primary care; however, it is often not detected by doctors. Recently, disease diagnosis and treatment approaches have been attempted using smart devices. In this study, instrumental effectiveness was confirmed with the diagnostic meta-analysis of studies that demonstrated the diagnostic effectiveness of PHQ-9 for depression using mobile devices.

PATIENTS AND METHODS

We found all published and unpublished studies through EMBASE, MEDLINE, MEDLINE In-Process, and PsychINFO up to March 26, 2021. We performed a meta-analysis by including 1099 subjects in four studies. We performed a diagnostic meta-analysis according to the PHQ-9 cut-off score and machine learning algorithm techniques. Quality assessment was conducted using the QUADAS-2 tool. Data on the sensitivity and specificity of the studies included in the meta-analysis were extracted in a standardized format. Bivariate and summary receiver operating characteristic (SROC) curve were constructed using the metandi, midas, metabias, and metareg functions of the Stata algorithm meta-analysis words.

RESULTS

Using four studies out of the 5476 papers searched, a diagnostic meta-analysis of the PHQ-9 scores of 1099 people diagnosed with depression was performed. The pooled sensitivity and specificity were 0.797 (95% CI = 0.642-0.895) and 0.85 (95% CI = 0.780-0.900), respectively. The diagnostic odds ratio was 22.16 (95% CI = 7.273-67.499). Overall, a good balance was maintained, and no heterogeneity or publication bias was presented.

CONCLUSION

Through various machine learning algorithm techniques, it was possible to confirm that PHQ-9 depression screening in mobiles is an effective diagnostic tool when integrated into a diagnostic meta-analysis.

Predicting Perceived Stress Related to the Covid-19 Outbreak through Stable Psychological Traits and Machine Learning Models

The global SARS-CoV-2 outbreak and subsequent lockdown had a significant impact on people's daily lives, with strong implications for stress levels due to the threat of contagion and restrictions to freedom. Given the link between high stress levels and adverse physical and mental consequences, the COVID-19 pandemic is certainly a global public health issue. In the present study, we assessed the effect of the pandemic on stress levels in N = 2053 Italian adults, and characterized more vulnerable individuals on the basis of sociodemographic features and stable psychological traits. A set of 18 psycho-social variables, generalized regressions, and predictive machine learning approaches were leveraged. We identified higher levels of perceived stress in the study sample relative to Italian normative values. Higher levels of distress were found in women, participants with lower income, and participants living with others. Higher rates of emotional stability and self-control, as well as a positive coping style and internal locus of control, emerged as protective factors. Predictive learning models identified participants with high perceived stress, with a sensitivity greater than 76%. The results suggest a characterization of people who are more vulnerable to experiencing high levels of stress during the COVID-19 pandemic. This characterization may contribute to early and targeted intervention strategies.