Adaptive screening for depression--recalibration of an item bank for the assessment of depression in persons with mental and somatic diseases and evaluation in a simulated computer-adaptive test environment

Effectiveness and cost-effectiveness of a web-based routine assessment with integrated recommendations for action for depression and anxiety (RehaCAT+): protocol for a cluster randomised controlled trial for patients with elevated depressive symptoms in rehabilitation facilities

INTRODUCTION

The integration of a web-based computer-adaptive patient-reported outcome test (CAT) platform with persuasive design optimised features including recommendations for action into routine healthcare could provide a promising way to translate reliable diagnostic results into action. This study aims to evaluate the effectiveness and cost-effectiveness of such a platform for depression and anxiety (RehaCAT+) compared with the standard diagnostic system (RehaCAT) in cardiological and orthopaedic health clinics in routine care.

METHODS AND ANALYSIS

A two-arm, pragmatic, cluster-randomised controlled trial will be conducted. Twelve participating rehabilitation clinics in Germany will be randomly assigned to a control (RehaCAT) or experimental group (RehaCAT+) in a 1:1 design. A total sample of 1848 participants will be recruited across all clinics. The primary outcome, depression severity at 12 months follow-up (T3), will be assessed using the CAT Patient-Reported Outcome Measurement Information System Emotional Distress-Depression Item set. Secondary outcomes are depression at discharge (T1) and 6 months follow-up (T2) as well as anxiety, satisfaction with participation in social roles and activities, pain impairment, fatigue, sleep, health-related quality of life, self-efficacy, physical functioning, alcohol, personality and health economic-specific general quality of life and socioeconomic cost and benefits at T1-3. User behaviour, acceptance, facilitating and hindering factors will be assessed with semistructured qualitative interviews. Additionally, a smart sensing substudy will be conducted, with daily ecological momentary assessments and passive collection of smartphone usage variables. Data analysis will follow the intention-to-treat principle with additional per-protocol analyses. Cost-effectiveness analyses will be conducted from a societal perspective and the perspective of the statutory pension insurance.

ETHICS AND DISSEMINATION

The study will be conducted according to the Declaration of Helsinki. The Ethics Committee of Ulm University, has approved the study (on 24 February 2021 ref. 509/20). Written informed consent will be obtained for all participants. Results will be published via peer-reviewed journals.

TRIAL REGISTRATION NUMBER

DRKS00027447.

Procedures to Develop a Computerized Adaptive Testing to Advance the Measurement of Narcissistic Personality

Narcissistic personality (NP) has recently attracted a great deal of attention. In this study, we mainly investigated the feasibility and effectiveness of using computerized adaptive testing (CAT) to measure NP (CAT-NP). The CAT in this study was simulated by the responses of several NP questionnaires of 1,013 university students as if their responses were collected adaptively. The item bank (85 items) that met the requirements of the psychometric properties of Item Response Theory (IRT) was first established, and then the CAT dynamically selected items according to the estimates of current trait level until the prespecified measurement precision is achieved. Finally, the efficiency and validity of the CAT were verified. The results showed that the proposed CAT-NP had reasonable reliability, validity, predictive utility, and high correlation. In addition, the CAT-NP could significantly save item usage without losing measurement accuracy, which greatly improves the test efficiency. The advantages and limitations of CAT in measuring NP and other psychological tests are discussed in the final section.

A new approach to assessing shyness of college students using computerized adaptive testing: CAT-Shyness

Assessing shy symptoms via computerized adaptive testing (CAT) provides greater measurement precision coupled with a lower test burden compared to conventional tests. The computerized adaptive test for shyness (CAT-Shyness) was developed based on a large sample of 1400 participants from China. Item bank development included the investigation of unidimensionality, local independence, and exploration of differential item functioning (DIF). CAT simulations based on the real data were carried out to investigate the reliability, validity, and predicted utility (sensitivity and specificity) of the CAT-Shyness. The CAT-Shyness item bank was successfully built and proved to have excellent psychometric properties: high content validity, unidimensionality, local independence, and no DIF. The CAT simulations needed 14 items to achieve a high measurement precision with a reliability of .9. Moreover, the results revealed that the proposed CAT-Shyness had acceptable and reasonable marginal reliability, criterion-related validity, and sensitivity and specificity. It not only had acceptable psychometric properties, but also had a shorter but efficient assessment of shyness, which can save significant test time and reduce the test burden for individuals with less information loss.

Calibration of an item bank in 474 orthopedic patients using Rasch analysis for computer-adaptive assessment of anxiety

OBJECTIVE

To calibrate an item bank of anxiety-related questions for use in orthopedic patients within a computer-adaptive test.

DESIGN

This is a psychometric study.

SETTING

The sample of orthopedic patients was recruited in two orthopedic rehabilitation clinics in Germany.

SUBJECTS

A total of 474 orthopedic rehabilitation patients were recruited for this study.

INTERVENTIONS

Not applicable.

MAIN MEASURES

The main measure is an adapted version of an existing anxiety item pool for cardiovascular rehabilitation patients.

RESULTS

The results of the confirmatory factor analysis and Mokken analysis confirmed a one-factor structure and double monotonicity. An anxiety item bank (48 items) could be developed and calibrated using Rasch analysis. It fitted to the Rasch model with a non-significant item-trait interaction (χ²(203) = 172.59; P = .94) and was free of differential item functioning. Unidimensionality could be verified and the person separation reliability was .96. The category threshold parameters varied between 4.72 and 3.16 (7.88 logits).

CONCLUSION

The unidimensional anxiety item bank provides the basis for a computer-adaptive test to assess a wide range of anxiety in rehabilitation patients with orthopedic diseases with very good psychometric characteristics.

Development of Rasch-based short screenings for the assessment of treatment motivation in patients with cardiovascular diseases

Purpose: The aim of the study was to develop unidimensional test-fair and economic short screenings to assess treatment motivation in patients with cardiovascular diseases using the Rasch analysis.Materials and methods: After pretesting for relevance and comprehension, a pool of 132 items on treatment motivation was completed by a sample consisting of 1168 patients with cardiovascular diseases recruited in two German cardiological rehabilitation centers. Confirmatory factor analyses and the Rasch analyses were conducted.Results: The confirmatory factor analyses confirmed a three-factor structure of the treatment motivation construct with task self-efficacy, outcome expectancies and intention as factors. Using the Rasch analysis for each of the three factors and removing items with misfit, differential item functioning and local response dependency reduced the initial item pool to the three short screenings. The short screenings fit to the Rasch model with a root mean square error of approximation (RMSEA = 0.021 (task self-efficacy; seven items); RMSEA = 0.024 (outcome expectancies; 12 items), RMSEA = 0.027 (intention; nine items). Person-separation reliability was 0.81, 0.82, and 0.73. Unidimensionality could be verified.Conclusions: The calibrated, unidimensional short screenings provide a psychometrically sound option for an initial- and follow-up assessment of treatment motivation in rehabilitation patients with cardiovascular diseases. Further testing in other cardiovascular diseases populations is needed to increase generalizability.Implications for rehabilitationNew short screenings for the assessment of treatment motivation: task self-efficacy, outcome expectancies, intention in rehabilitation patients with cardiovascular diseases are available.Treatment motivation short screenings_{self-efficacy/outcome expectancies/intention} consist of seven items (treatment motivation short screening_{self-efficacy}), 12 items (treatment motivation short screening_{outcome expectancies}), nine items (treatment motivation short screening_intention) and are therefore especially timesaving.The short screenings demonstrate good psychometric properties, cover a wide spectrum of task self-efficacy, outcome expectancies and intention, and are free of local dependencies and of differential item functioning regarding to gender, age and cardiovascular diagnoses.Using a Rasch based unidimensional short screening is a test-fair and economic method to assess patients' treatment motivation, which might help to improve rehabilitation health care tailored to patients' needs.

Development and Validation of an Item Bank for Depression Screening in the Chinese Population Using Computer Adaptive Testing: A Simulation Study

With the increasing prevalence of depression, creating a simple and precise tool for measuring depression is becoming more important. This study developed a computer adaptive testing for depression (CAT-Depression) from a Chinese sample. The depression item bank was constructed from a sample of 1,135 participants with or without depression using the Graded Response Model (GRM; Samejima, 1969). The final depression item bank with strict unidimensionality comprised 68 items, which had local independence, good item-fit, high discrimination, no differential item functioning (DIF), and each item measured at least one symptom of diagnostic criteria for depression in ICD-10. In addition, the mean IRT discrimination of the item bank reached 1.784, which clearly showed that the item bank of CAT-Depression was high-quality. Moreover, a simulation CAT study with real response data was conducted to investigate the characteristics, marginal reliability, criterion-related validity, and predictive utility (sensitivity and specificity) of CAT-Depression. The results revealed that the proposed CAT-Depression had acceptable and reasonable marginal reliability, criterion-related validity, and sensitivity and specificity.

Defeat and entrapment: more than meets the eye? Applying network analysis to estimate dimensions of highly correlated constructs

BACKGROUND

Defeat and entrapment have been shown to be of central relevance to the development of different disorders. However, it remains unclear whether they represent two distinct constructs or one overall latent variable. One reason for the unclarity is that traditional factor analytic techniques have trouble estimating the right number of clusters in highly correlated data. In this study, we applied a novel approach based on network analysis that can deal with correlated data to establish whether defeat and entrapment are best thought of as one or multiple constructs.

METHODS

Explanatory graph analysis was used to estimate the number of dimensions within the 32 items that make up the defeat and entrapment scales in two samples: an online community sample of 480 participants, and a clinical sample of 147 inpatients admitted to a psychiatric hospital after a suicidal attempt or severe suicidal crisis. Confirmatory Factor analysis (CFA) was used to test whether the proposed structure fits the data.

RESULTS

In both samples, bootstrapped exploratory graph analysis suggested that the defeat and entrapment items belonged to different dimensions. Within the entrapment items, two separate dimensions were detected, labelled internal and external entrapment. Defeat appeared to be multifaceted only in the online sample. When comparing the CFA outcomes of the one, two, three and four factor models, the one factor model was preferred.

CONCLUSIONS

Defeat and entrapment can be viewed as distinct, yet, highly associated constructs. Thus, although replication is needed, results are in line with theories differentiating between these two constructs.

Parallel short forms for the assessment of activities of daily living in cardiovascular rehabilitation patients (PADL-cardio): development and validation

PURPOSE

To develop and validate parallel short forms for the assessment of activities of daily living in cardiac rehabilitation patients (PADL-cardio I & II).

METHOD

PADL-cardio I & II were developed based on a sample of 106 patients [mean age  =  57.6; standard deviation (SD) = 11.1; 72.6% males] using Rasch analysis and validated with a sample of 81 patients (mean age  =  59.1; SD  =  11.1; 88.9% males). All patients answered PADL-cardio and the Short Form 12 Health Survey.

RESULTS

Both versions of PADL-cardio are composed of 10 items. The fit to the Rasch model was given documented by a non-significant Item-trait interaction score (PADL-cardio I: χ² = 31.08, df  =  30, p  =  0.41; PADL-cardio II: χ² = 45.6, df  =  40, p  =  0.25). The two versions were free of differential item functioning. Person-separation reliability was 0.72/0.78 and unidimensionality was given. The two versions correlated with r = 0.98 and the correlation between PADL-cardio and the underlying item bank was 0.99 for both versions. Concurrent validity is indicated through correlations with the Short Form 12 Health Survey (r  = -0.37 to -0.40).

CONCLUSION

PADL-cardio provides a short and psychometrically sound option for the assessment of activities of daily living in cardiovascular rehabilitation patients. The two versions of PADL-cardio are equivalent. Hence, they can be used to reduce practice and retest effects in repeated measurement, facilitating the longitudinal assessment of activities of daily living. Implications for Rehabilitation New parallel test forms for the assessment of activities of daily living in cardiac rehabilitation (PADL-cardio I & PADL-cardio II) are available. PADL-cardio I & II consist of 10 items and are therefore especially timesaving. Concurrent validity is given through correlations with the Short Form Health Survey 12. Therapeutic success could be determined more precisely by the parallel forms reducing practice and retest effects.

[Internet- and mobile-based approaches : Psycho-social diagnostics and treatment in medical rehabilitation]

Technology-based approaches for psychosocial diagnostics and interventions provide an attractive opportunity to optimize medical rehabilitation. Based on an Internet- and mobile-based assessment of existing functional health impairments, appropriate planning, implementation of corresponding courses of action as well as outcome assessment can take place. This can be implemented in the form of Internet- and mobile-based interventions (IMI).The present article provides an overview of the basic knowledge of IMI and their evidence base both in general and in particular for their use in medical rehabilitation. Important aspects of internet and mobile-based psycho-social diagnostics are discussed subsequently. Finally, an outlook for the use of Internet- and mobile-based diagnostics and interventions in medical rehabilitation is given.

Development and validation of parallel short forms PaSA-cardio for the assessment of general anxiety in cardiovascular rehabilitation patients using Rasch analysis

OBJECTIVE

To develop and validate parallel short forms for the assessment of general anxiety in cardiovascular rehabilitation patients, that facilitate repeated measurement over time without contamination from residual practice effect variance.

DESIGN

Development of the parallel short forms using Rasch analysis. Validation study.

SETTING

Cardiac rehabilitation centres in Germany.

SUBJECTS

Cardiovascular rehabilitation patients.

INTERVENTIONS

Not applicable.

MAIN MEASURES

Parallel short forms PaSA-cardio, Hospital Anxiety and Depression Scale, Short Form Health Survey SF-12 and Structured Clinical Interview for DSM-IV Axis I Disorders.

RESULTS

Each version of the parallel short forms (PaSA-cardio-I and PaSA-cardio-II) comprises ten items. The two forms fitted to the Rasch model with a non-significant item-trait interaction (PaSA-cardio-I: chi-square = 39.49, degrees of freedom = 30, probability = 0.12; PaSA-cardio-II: chi-square = 26.56, degrees of freedom = 30, probability = 0.65). Person-separation reliability was 0.75/0.76. Unidimensionality could be verified. Correlation between the two models was 0.94 and 0.95, and correlations with the underlying item bank were 0.95 and 0.93. Validity could be confirmed. The area under the curve was between 0.88 and 0.97 for PaSA-cardio-I and between 0.92 and 0.95 for PaSA-cardio-II.

CONCLUSIONS

Assessment of general anxiety in cardiovascular rehabilitation patients with the PaSA-cardio was valid, economical and accurate. The two forms of the PaSA-cardio are equivalent and allow retest without contamination from residual practice effect variance.

Exploring the relationship of decentering to health related concepts and cognitive and metacognitive processes in a student sample

Background

Decentering, a central change strategy of Mindfulness-Based Cognitive Therapy, is a process of stepping outside of one’s own mental events leading to an objective and non-judging stance towards the self. The study aimed at investigating associated mechanisms of decentering.

Method

The present study investigated the relation of decentering, operationalized by means of the German Version of the Experiences Questionnaire, to severity of depressive symptoms, assessed by the adaptive Rasch-based depression screening, and self-focussed attention, assessed by the Questionnaire of Dysfunctional and Functional Self-Consciousness. Furthermore, the relationship between decentering and a) the ability to shift and allocate attention by means of the Stroop test, and b) metacognitive monitoring, i.e. the absolute difference between judged and real task performance, was investigated. These relationships were examined in 55 healthy students using Pearson’s correlations.

Results

In line with our assumptions, higher decentering scores were significantly associated with lower scores on severity of depressive symptoms, with higher functional- and lower dysfunctional self-focussed attention. Contrary to our expectations, results neither indicated a relationship between decentering and attention ability, nor between decentering and metacognitive monitoring.

Conclusions

The present results suggest that decentering is associated with concepts of mental health (i.e. less severity of depressive symptoms and higher functional self-focussed attention). Overall, the concept decentering seems to be mainly composed of self-focussed aspects when investigated in a healthy sample without intervention. Further investigations of associated concepts of decentering should consider aspects of self-relevance and emotional valence.

Psychometric Properties and Performance of the Patient Reported Outcomes Measurement Information System® (PROMIS®) Depression Short Forms in Ethnically Diverse Groups

Short form measures from the Patient Reported Outcomes Measurement Information System^® (PROMIS^®) are used widely. The present study was among the first to examine differential item functioning (DIF) in the PROMIS Depression short form scales in a sample of over 5000 racially/ethnically diverse patients with cancer. DIF analyses were conducted across different racial/ethnic, educational, age, gender and language groups.

METHODS

DIF hypotheses, generated by content experts, informed the evaluation of the DIF analyses. The graded item response theory (IRT) model was used to evaluate the five-level ordinal items. The primary tests of DIF were Wald tests; sensitivity analyses were conducted using the IRT ordinal logistic regression procedure. Magnitude was evaluated using expected item score functions, and the non-compensatory differential item functioning (NCDIF) and T1 indexes, both based on group differences in the item curves. Aggregate impact was evaluated with expected scale score (test) response functions; individual impact was assessed through examination of differences in DIF adjusted and unadjusted depression estimates.

RESULTS

Many items evidenced DIF; however, only a few had slightly elevated magnitude. No items evidenced salient DIF with respect to NCDIF and the scale-level impact was minimal for all group comparisons. The following short form items might be targeted for further study because they were also hypothesized to evidence DIF. One item showed slightly higher magnitude of DIF for age: nothing to look forward to; conditional on depression, this item was more likely to be endorsed in the depressed direction by individuals in older groups as contrasted with the cohort aged 21 to 49. This item was also hypothesized to show age DIF. Only one item (failure) showed DIF of slightly higher magnitude (just above threshold) for Whites vs. Asians/Pacific Islanders in the direction of higher likelihood of endorsement for Asians/Pacific Islanders. This item was also hypothesized to show DIF for minority groups. The impact of DIF was negligible. Conditional on depression, the items, worthless and hopeless were more likely to be endorsed in the depressed direction by respondents with less than high school education vs. those with a graduate degree; the magnitude of DIF was slightly above the T1 threshold, but not that of NCDIF. These items were also hypothesized to show DIF in the direction of more feelings of worthlessness by groups with lower education. While the magnitude and aggregate impact of DIF was small, in a few instances, individual impact was observed. Information provided was relatively high, particularly in the middle upper (depressed) tail of the distribution. Reliability estimates were high (> 0.90) across all studied groups, regardless of estimation method.

CONCLUSIONS

This was the first study to evaluate measurement equivalence of the PROMIS Depression short forms across large samples of ethnically diverse groups. There were few items with DIF, and none of high magnitude, thus supporting the use of PROMIS Depression short form measures across such groups. These results could be informative for those using the short forms in minority populations or clinicians evaluating individuals with the depression short forms.

Testing the PROMIS® Depression measures for monitoring depression in a clinical sample outside the US

The Patient Reported Outcomes Measurement Information System (PROMIS) was devised to facilitate assessment of patient self-reported health status, taking advantage of Item Response Theory. We aimed to assess measurement properties of the PROMIS Depression item bank and an 8-item static short form in a Spanish clinical sample. A three-month follow-up study of patients with active mood/anxiety symptoms (n = 218) was carried out. We assessed model unidimensionality (Confirmatory Item Factor Analysis), reliability (internal consistency and Item Information Curves), and validity (convergent-discriminant with correlations; known-groups with comparison of means and effect sizes; and criterion validity with Receiver operating Characteristics (ROC) analysis). We also assessed 3-month responsiveness to change (Cohen's effect sizes (d) in stable and recovered patients). The unidimensional model showed adequate fit (CFI = 0.97, RMSEA = 0.08). Information Curves had reliabilities over 0.90 throughout most of the score continuum. As expected, we observed high correlations with external self-reported depression, and moderate with self-reported anxiety and clinical measures. The item bank showed an increasing severity gradient from no disorder (mean = 48, SE = 0.6) to depression with comorbid anxiety (mean = 55.8, SE = 0.4). PROMIS detected depression disorder with great accuracy according to the area under the curve (AUC = 0.89). Both formats, item bank and short form, were highly responsive to change in recovered patients (d > 0.7) and had small changes in stable patients (d < 0.2). The good metric properties of the Spanish PROMIS Depression measures provide further evidence of their adequacy for monitoring depression levels of patients in clinical settings. This double check of quality (within countries and populations) supports the ability of PROMIS measures for guaranteeing fair comparisons across languages and countries in specific clinical populations.