Construct and criterion validity of patient-reported outcomes (PROs) for depression: A clinimetric comparison

A comparison of the original major depression inventory with a modified version: A Danish validation study

BACKGROUND

The Major Depression Inventory (MDI) is a patient-reported outcome measure used by general practitioners to assist with diagnosing and evaluation of the severity of a patient's depression. However, recent studies have questioned the structural validity of the MDI.

OBJECTIVES

We proposed a modified version (mMDI) of the MDI with fewer response categories and four rephrased items and aimed to compare the psychometric properties of the changes in a joint cohort of patients from general practice and mental health associations.

METHODS

We used Rasch analysis, confirmatory factor analysis, and the area under the receiver operating curve (AUROC) to assess the validity and reliability of the two versions. Equipercentile linking was used to compute cut-off points for the mMDI.

RESULTS

For both versions, local dependence was found between the three item pairs (loss of interest, lack of energy), (lack of self-confidence, feelings of guilt), and (concentration problems, feeling restless/slowed down). The mMDI displayed lower measurement error in the upper end of the scale and better item level fit for three of the four reformulated items compared to the MDI. For the MDI, 5.3% of the respondents gave improbable responses; the corresponding number was 3.4% for the mMDI. The mMDI displayed better fit to a one-factor model compared to the MDI. When comparing the correlation of the scales with the WHO-5 instrument, the corresponding AUROC estimates for the mMDI and MDI were found to be 0.93 (0.92; 0.96) and 0.91 (0.87; 0.94), respectively. The cut-off points for mild, moderate, and severe depression in the mMDI were found to be 17, 20, and 23, respectively.

CONCLUSION

The proposed changes of the MDI are psychometrically sound upgrades of the original.

Structural validity of the 5-item World Health Organization Well-being Index (WHO-5) in patients with schizophrenia spectrum disorders

BACKGROUND

When monitoring the severity and impact of schizophrenia spectrum disorders, a measure of subjective well-being should ideally accompany measures of symptom severity and medication side effects. The self-reported 5-item World Health Organization Well-being Index (WHO-5) is a brief, generic, and widely used measure of subjective well-being. However, the structural validity of the WHO-5, namely, whether the individual item scores can be combined to produce a meaningful total score, has not been examined among patients with schizophrenia spectrum disorders.

METHOD

Utilizing data from 399 Danish patients with schizophrenia spectrum disorders attending the Psychiatric Services of the Central Denmark Region, we employed Rasch analysis to examine the structural validity (i.e., unidimensionality, overall fit to the Rasch model, and differential item functioning) of the WHO-5.

RESULTS

The WHO-5 was found to be unidimensional with no differential item functioning for age, sex, or inpatient/outpatient status. However, in the initial analysis, some misfit to the Rasch model, partially caused by the disordering of response categories, was evident. In adjusted analyses in which the item response categories 2 (Less than half of the time) and 3 (More than half of the time) were merged, overall fit to the model was improved.

CONCLUSIONS

When two item response categories were merged, the Danish version of the WHO-5 was found to be structurally valid for patients with schizophrenia spectrum disorders. This suggests that the WHO-5 holds promise as a measure of subjective well-being in this patient population.

Validation of Chinese version of the 5-item WHO well-being index in type 2 diabetes mellitus patients

BACKGROUND

For better disease management and improved prognosis, early identification of co-morbid depression in diabetic patients is warranted. the WHO-5 well-being index (WHO-5) has been used to screen for depression in diabetic patients, and its Chinese version (WHO-5-C) has been validated. However, its psychometric properties remain to be further validated in the type 2 diabetes patient population. The aim of our study was to examine the reliability and validity of the WHO-5-C in patients with type 2 diabetes mellitus.

METHODS

The cross-sectional study was conducted on 200 patients from July 2014 to March 2015. All patients should complete the WHO-5-C, the Patient Health Questionnaire-9 (PHQ-9), the 20-item Problem Areas in Diabetes Scale (PAID-20), the Mini International Neuropsychiatric Interview (M.I.N.I), and Hamilton Rating Scale for Depression (HAM-D). Internal consistency of WHO-5 was revealed by Cronbach's alpha, and constructive validity by confirmatory factor analysis (CFA). Relationship with PHQ-9, HAM-D, and PAID-20 was examined for concurrent validity, and ROC analysis was performed for criterion validity.

RESULTS

The WHO-5-C presented satisfactory reliability (Cronbach's alpha = 0.88). CFA confirmed the unidimensional factor structure of WHO-5-C. The WHO-5-C had significant negative correlation with HAM-D (r = -0.610), PHQ-9 (r = -0.694) and PAID-20 (r = -0.466), confirming good concurrent validity. Using M.I.N.I as the gold standard, the cut-off value of WHO-5-C was 42, with a sensitivity of 0.83 and specificity of 0.75.

CONCLUSION

The WHO-5-C holds satisfactory reliability and validity that is suitable for depression screening in type 2 diabetes patients as a short and convenient instrument.

The Validity and Reliability of the SINBAD Classification System for Diabetic Foot Ulcers

ABSTRACT

Diabetic foot ulcers (DFUs) are a serious and costly complication of diabetes mellitus with a global prevalence of 6.3% and cost of £8,800 per unhealed DFU in the National Health Service. The three main types of DFU are neuropathic, ischemic, and neuroischemic, with an estimated prevalence of 35%, 15%, and 50%, respectively. Because 85% of lower-limb amputations in patients with diabetes are preceded by a DFU, the task of reducing the current and future burden of DFUs on an international level is of crucial importance. Classification of a DFU is an important and complex process with many independent variables that influence the wound severity. Correct classification of a DFU is important to prevent deterioration in the short term and lower-limb amputation in the long term. Both the accuracy of the clinician's interpretation of categorical data from a classification model and grasp of contextual risk factors can refine diagnoses. The term SINBAD is an acronym for six independent variables: site, ischemia, neuropathy, bacterial infection, area, and depth. This system uses comprehensive parameters with strict criteria to facilitate quick and accurate clinical decisions to prevent lower-limb amputation. In addition to providing quantitative measurement, SINBAD also spotlights the multifaceted characteristics of DFUs. By evaluating the validity and reliability of the SINBAD classification system, its applicability for the assessment of DFUs and prevention of lower-limb amputation can be better understood.

Pilot Trial of Workable: A Therapist-Supported Digital Program for Injured Workers

Workplace sickness absence is a major public health and economic problem, and common mental disorders (CMDs) such as anxiety and depression are associated with particularly high rates of long-term sickness absence. Effective return-to-work (RTW) interventions are required. This pilot study investigates the feasibility, acceptability, and potential effectiveness of a new therapist-assisted Web-based RTW intervention (Workable) for injured workers on sick leave for a psychological or physical injury. A single-group open pilot trial design was used, with assessments at pre-treatment and post-treatment. The intervention consisted of 6 weeks of online modules and 6 coaching calls from a psychologist. A total of 13 participants were recruited and 9 completed all questionnaires. Program adherence was high, with 92% of participants completing the 6-week intervention. Participants reported high levels of intervention satisfaction and ease of use. There were large and significant reductions between pre- and post-treatment on measures of depression, anxiety, stress, and workdays missed over the past four weeks, along with a significant increase in self-reported work ability. These results suggest that Workable is a feasible and acceptable intervention for injured workers, with the potential to improve mental health and RTW outcomes. A randomized controlled trial is required to determine the efficacy of the intervention.

Clinimetric properties of the diagnostic criteria for psychosomatic research among the elderly

INTRODUCTION

Among the elderly, the availability of tool assessing psychosomatic syndromes is limited. The present study aims at testing inter-rater reliability and concurrent validity of the semi-structured interview for the Diagnostic Criteria for Psychosomatic Research (DCPR-R-SSI) in the elderly of the general population.

METHOD

One hundred eight subjects were recruited. Participants received a clinical assessment which included the DCPR-R-SSI, the Illness Attitude Scale (IAS), the Geriatric Depression Scale (GDS), the Psychosocial Index (PSI), the Toronto Alexithymia Scale-20 (TAS-20). Analyses of inter-rater reliability of DCPR-R-SSI and concurrent validity between DCPR-R-SSI and self-administered questionnaires were conducted.

RESULTS

DCPR-R-SSI showed excellent inter-rater reliability with a percent of agreement of 90.7% (K Cohen: 0.856 [SE = 0.043], 95% CI: 0.77-0.94). DCPR-R demoralization showed fair concurrent validity with GDS; concurrent validity was also fair between DCPR-R Alexithymia and TAS-20, and between DCPR-R allostatic overload and PSI allostatic load, while the concurrent validity between DCPR-R Disease Phobia and IAS was moderate.

CONCLUSION

DCPR-R-SSI represents a reliable and valid tool to assess psychosomatic syndromes in the elderly. DCPR-R is in need of being implemented in the elderly clinical evaluation.

The Hopkins Symptom Checklist (SCL-90-R): A Patient-Reported Outcome Measure in Parkinson's Disease

Objective: This is the first study applying Clinimetric Patient-Reported Outcome Measures (CLIPROM) criteria to evaluate the construct validity, sensitivity, and clinical utility of the SCL-90-R in patients with Parkinson's disease (PD). Methods: A Rasch analysis was conducted using a sample of 488 PD outpatients. Results: Testing for dimensionality revealed that less than 5% of t-tests were significant, indicating that the SCL-90-R subscales entailed the property of construct validity. As to the total score, a Person Separation Reliability Index of .96 was found. Conclusions: The SCL-90-R total score is a sensitive screening measure that can be used not only to differentiate healthy stress reactions from symptoms of psychological distress but also to detect PD patients with an increased risk for psychiatric complications. As to the subscales, the brief versions that did not include misfitting items should be used to assess the severity of specific symptoms of psychological distress affecting PD patients.

Cross-cultural validity of the WHO-5 Well-Being Index and Euthymia Scale: A clinimetric analysis

BACKGROUND

The assessment of psychological well-being and euthymia represents an emerging issue in clinical psychology and psychiatry. Rating scales and indices such as the 5-item version of the World Health Organization Well-Being Index (WHO-5) and the Euthymia Scale (ES) were developed but insufficient attention has been devoted to the evaluation of their cross-cultural validity. This is the first study using Clinimetric Patient-Reported Outcome Measures (CLIPROM) criteria to assess cross-cultural validity and sensitivity of five different versions of the WHO-5 and ES.

METHODS

A multicenter cross-sectional study involving a total of 3762 adult participants from different European (i.e., Italy, Poland, Denmark) and non-European (i.e., China, Japan) countries was conducted. Item Response Theory models (Mokken and Rasch analyses) were applied.

RESULTS

Mokken coefficients of scalability were found to range from 0.42 to 0.84. The majority of the versions of the WHO-5 fitted the Rasch model expectations. Paired t-tests revealed that the Italian and Danish WHO-5 versions were unidimensional. Person Separation Reliability indices showed that the Polish, Danish, and Japanese ES versions could reliably discriminate between subjects with different levels of euthymia.

LIMITATIONS

A convenience sampling was used, thus limiting the generalizability of study findings. In addition, no measures of negative mental health were administered.

CONCLUSIONS

WHO-5 can be used in international studies for cross-cultural comparisons since it covers transcultural components of subjective well-being. Findings also suggest that the ES can be used as a cross-cultural screening tool since it entailed the clinimetric property of sensitivity.

Overestimation and underestimation of youths' health-related quality of life are associated with youth and caregiver positive screens for depression: results of a population-based study among youths with longstanding type 1 diabetes

BACKGROUND

This study aimed to analyze the extent and direction of disagreement between self- and proxy-reported quality of life (QoL) and the factors associated with QoL overestimation and underestimation by caregivers compared with self-reports.

METHODS

This study used data from population-based questionnaire surveys conducted in 2012-2013 and 2015-2016 with 11- to 17-year-olds with a duration of type 1 diabetes of 10 years or longer and their caregivers (n = 1058). QoL in youth was assessed via 10-item KIDSCREEN (KIDSCREEN-10) self- and proxy-reported questionnaires. The scores ranged from 0 to 100, with higher scores indicating better QoL. Depression screening was performed via the Center for Epidemiological Studies Depression Scale for Children for youths (CES-DC screen positive: score > 15) and WHO-5 Well-being Index for parents/caregivers (WHO-5 screen positive: score ≤ 50).

RESULTS

The mean self- and proxy-reported normalized KIDSCREEN-10 scores were 64.2 (standard deviation [SD] 11.4) and 66.1 (11.5), respectively. More caregivers overestimated (self-reported minus proxy-reported score < - 0.5*SD self-reported score) than underestimated (self-reported minus proxy-reported score > 0.5*SD self-reported score) youths' QoL (37% versus 23%, p < 0.001). Youths who screened positive for depression (18%) were at higher risk of their QoL being overestimated and lower risk of their QoL being underestimated by caregivers than youths who screened negative for depression (RR_{Overestimation} 1.30 [95% CI 1.10-1.52], RR_{Underestimation} 0.27 [0.15-0.50]). Caregivers who screened positive for depression (28%) overestimated the QoL of their children less often and underestimated the QoL of their children more often than caregivers who screened negative for depression (RR_{Overestimation} 0.73 [0.60-0.89], RR_{Underestimation} 1.41 [1.14-1.75]).

CONCLUSIONS

Caregivers often over- or underestimated their children's QoL. Positive screens for depression among both youths and caregivers contributed to the observed differences between self- and caregiver-reported QoL.

The Well-Being of the German Adult Population Measured with the WHO-5 over Different Phases of the COVID-19 Pandemic: An Analysis within the COVID-19 Snapshot Monitoring Study (COSMO)

The aim of this study is to evaluate factors associated with the subjective well-being (SWB) and suspected depression measured with WHO-5 among German adults during different phases of the COVID-19 pandemic. Survey data were analyzed from the COVID-19 Snapshot Monitoring (COSMO) study, which collected data from 972, 1013, and 973 participants in time point 1 (19–20 May 2020), time point 2 (15–16 September 2020), and time point 3 (21–22 December 2020), respectively. Descriptive analyses and logistic regression analyses to identify the factors associated with suspected depression (WHO-5 ≤ 50) were conducted. Data showed that the mean WHO-5 scores in three time points were 56.17, 57.27, and 53.93, respectively. The risk of suspected depression was increased by about 1.5 times for females, 2.5–3 times among 18–24 year-olds compared to ages above 65 years, 1.5 times for singles, 2 times for those with chronic illnesses, and 2–3 times for people living in poverty. The main study findings show that German adult SWB is lower than pre-pandemic reference values. Special focus should be placed on vulnerable groups, such as females, younger persons, and people living in poverty who are most prone to a reduction in SWB and therefore suspected depression.

Clinimetric properties of the Smoking Abstinence Expectancies Questionnaire

Smoking abstinence expectancies are beliefs about negative and positive short-term psychological and physiological consequences of not smoking. The Smoking Abstinence Expectancies Questionnaire (SAEQ) is a widely used Patient-Reported Outcome Measure (PROM) to assess smoking abstinence expectancies. It has four subscales: negative mood, somatic symptoms, harmful consequences, positive consequences. Although studied from a psychometric perspective, the SAEQ needs further evaluation. Clinimetrics, and its Clinimetric Criteria for Patient-Reported Outcome Measures (CLIPROM), offers a robust method to evaluate the SAEQ. We verified construct validity and sensitivity of the Italian version of the SAEQ applying CLIPROM criteria. A total of 293 adult Italian smokers were consecutively enrolled at two smoking cessation clinics and assessed via the SAEQ. Item Response Theory models (i.e., combining Rasch and Mokken analyses) were used to test construct validity and sensitivity. The total score of the SAEQ was not found to be unidimensional but each SAEQ subscale score was. PSI (0.90) indicated that the total score of the SAEQ could reliably discriminate between respondents with different levels of the trait under assessment, whereas SAEQ subscales on negative mood and harmful consequences could reliably distinguish between different groups but not between different subjects (PSI ranging from 0.77 and 0.78). Overall, the total score of the SAEQ is a sensitive screening PROM and can be used at smoking cessation clinics to discriminate between subjects with different levels of smoking abstinence expectancies. SAEQ subscales should be used to detect severity and subjective burden of a wide range of expected effects of nicotine abstinence.

A clinimetric analysis of the euthymia, resilience, and positive mental health scales

BACKGROUND

Euthymia has been described as a transdiagnostic construct characterized by the absence of mood disturbances and the presence of an integration of psychic forces, such as coping strategy and well-being. A multidimensional measure, the Euthymia Scale (ES), has been proposed to assess it. We investigated construct and concurrent validity of the ES.

METHODS

Construct validity was studied via Rasch and Mokken analyses and compared with that of the 11-item Resilience Scale (RS-11) and 9-item Positive Mental Health Scale (PMH-Scale). A total of 951 participants were recruited (77.5% women; 24.86 ± 5.62 years).

RESULTS

The ES, RS and PMH demonstrated similar sensitivity and construct validity. Findings indicate minor needs for adjustments only. As expected the ES demonstrated a strong negative correlation with neuroticism.

LIMITATIONS

The convenience sample of subjects recruited primarily from female Italian university students and a community-based data collection limit the generalizability of the present findings. The cross-sectional design precludes the assessment of test-retest reliability, predictive and incremental validity. Only self-report measures and a Likert version of the ES were used.

CONCLUSIONS

ES is the most comprehensive measure of euthymia. The RS-11 is a valid measure of a specific component of euthymia, namely subjective ability to cope with stress and empower well-being in face of life adversities. The PMH-Scale is a valid measure of overall positive mental health.

Clinimetric Criteria for Patient-Reported Outcome Measures

Patient-reported outcome measures (PROMs) are self-rated scales and indices developed to improve the detection of the patients' subjective experience. Given that a considerable number of PROMs are available, it is important to evaluate their validity and usefulness in a specific research or clinical setting. Published guidelines, based on psychometric criteria, do not fit in with the complexity of clinical challenges, because of their quest for homogeneity of components and inadequate attention to sensitivity. Psychometric theory has stifled the field and led to the routine use of scales widely accepted yet with a history of poor performance. Clinimetrics, the science of clinical measurements, may provide a more suitable conceptual and methodological framework. The aims of this paper are to outline the major limitations of the psychometric model and to provide criteria for clinimetric patient-reported outcome measures (CLIPROMs). The characteristics related to reliability, sensitivity, validity, and clinical utility of instruments are critically reviewed, with particular reference to the differences between clinimetric and psychometric approaches. Of note is the fact that PROMs, rating scales, and indices developed according to psychometric criteria may display relevant clinimetric properties. The present paper underpins the importance of the clini-metric methodology in choosing the appropriate PROMs. CLIPROM criteria may also guide the development of new indices and the validation of existing PROMs to be employed in clinical settings.

Sensitivity and concurrent validity of the Japanese version of the Euthymia scale: a clinimetric analysis

BACKGROUND

Euthymia is characterized by the lack of mood disorders, the presence of positive affects, psychological flexibility and well-being, a unifying outlook on life, and resistance to stress. The Euthymia Scale (ES) is a 10-item self-rating clinimetric index assessing euthymia.

OBJECTIVES

The present study was conducted to examine the clinimetric sensitivity and concurrent validity of the Japanese version of the Euthymia Scale (ES-J).

METHODS

A cross-sectional online survey was conducted. The Mini-International Neuropsychiatric Interview was used to determine the presence of past or current major depressive episodes (MDE). The clinimetric sensitivity was evaluated using the Analysis of Variance (ANOVA). Pearson's correlation coefficients were performed to examine the concurrent validity of the ES-J.

RESULTS

A total of 1030 eligible participants completed the survey. The ES-J differentiated healthy subjects from complete remission (i.e., those with a past history of MDE without current MDE) (p < 0.001), from those with past or current history of MDE (p < 0.001), subjects with current MDE from those with sub-threshold symptoms of depression (p < 0.001), and healthy participants from subjects with moderate to severe symptoms of psychological distress (p < 0.001). The associations between the ES-J and measures of psychological well-being, resilience, life satisfaction, and social support were significantly positive (0.353 < r < 0.666, p < 0.001). A negative relationship between the ES-J and measures of psychological distress was also found (r = - 0.595, p < 0.001).

CONCLUSIONS

The findings of the present study indicated that the ES-J is a valid and highly sensitive clinimetric index, which can be used as a screening measure in the clinical process of assessment of recovery, particularly when symptoms are expected to be mild and/or when dealing with subclinical symptoms of psychological distress and depression. The findings of this study also support the use of the ES-J to detect vulnerability to depression and to identify subjects at higher risk of relapse.