Pre-training inter-rater reliability of clinical instruments in an international psychosis research project

Validation of ratings on the six-item Positive and Negative Syndrome Scale obtained via the Simplified Negative and Positive Symptoms Interview among outpatients with schizophrenia

BACKGROUND

The six-item Positive and Negative Syndrome Scale (PANSS-6) is a measure of the severity of core symptoms of schizophrenia, which can be administered via the brief Simplified Negative and Positive Symptoms Interview (SNAPSI). A recent study has confirmed the validity of PANSS-6 ratings as derived by SNAPSI (PANSS-6_SNAPSI) among inpatients with schizophrenia.

AIMS

We aimed to test the validity of PANSS-6_SNAPSI among outpatients with schizophrenia using PANSS-6 ratings extracted from the 30-item PANSS-30 as derived by the Structured Clinical Interview for the Positive and Negative Syndrome Scale (PANSS-6_SCI-PANSS) as a gold standard reference.

METHODS

PANSS-6_SNAPSI and PANSS-6_SCI-PANSS ratings were obtained at two time points by independent raters with established inter-rater reliability. Agreement between PANSS-6_SNAPSI and PANSS-6_SCI-PANSS ratings was estimated via intra-class coefficients (ICCs) and responsiveness over time was quantified using Spearman's rank correlation coefficients. Post hoc "leave-one-out" analyses were carried out, in which each rater in turn was excluded from the ICC calculations.

RESULTS

Seventy-three outpatients with schizophrenia participated in the study (mean age: 38.3 years; 56% males). The ICC for PANSS-6_SNAPSI versus PANSS-6_SCI-PANSS was 0.67 [95%CI = 0.56-0.76] and the Spearman's rank correlation coefficient for responsiveness was 0.40 (p = 0.004). When data from a specific outlying rater were excluded, the ICC for PANSS-6_SNAPSI versus PANSS-6_SCI-PANSS was 0.75 [95% CI = 0.63-0.83] and the Spearman's rank correlation coefficient for responsiveness was 0.55 (p = 0.018).

CONCLUSIONS

We found PANSS-6_SNAPSI ratings to have acceptable clinical validity, suggesting that PANSS-6_SNAPSI can be used for both inpatients and outpatients with schizophrenia.

The Relationship Between Grey Matter Volume and Clinical and Functional Outcomes in People at Clinical High Risk for Psychosis

Objective

To examine the association between baseline alterations in grey matter volume (GMV) and clinical and functional outcomes in people at clinical high risk (CHR) for psychosis.

Methods

265 CHR individuals and 92 healthy controls were recruited as part of a prospective multi-center study. After a baseline assessment using magnetic resonance imaging (MRI), participants were followed for at least two years to determine clinical and functional outcomes, including transition to psychosis (according to the Comprehensive Assessment of an At Risk Mental State, CAARMS), level of functioning (according to the Global Assessment of Functioning), and symptomatic remission (according to the CAARMS). GMV was measured in selected cortical and subcortical regions of interest (ROI) based on previous studies (ie orbitofrontal gyrus, cingulate gyrus, gyrus rectus, inferior temporal gyrus, parahippocampal gyrus, striatum, and hippocampus). Using voxel-based morphometry, we analysed the relationship between GMV and clinical and functional outcomes.

Results

Within the CHR sample, a poor functional outcome (GAF < 65) was associated with relatively lower GMV in the right striatum at baseline (P < .047 after Family Wise Error correction). There were no significant associations between baseline GMV and either subsequent remission or transition to psychosis.

Conclusions

In CHR individuals, lower striatal GMV was associated with a poor level of overall functioning at follow-up. This finding was not related to effects of antipsychotic or antidepressant medication. The failure to replicate previous associations between GMV and later psychosis onset, despite studying a relatively large sample, is consistent with the findings of recent large-scale multi-center studies.

Impact of smoking Behavior on cognitive functioning in persons at risk for psychosis and healthy controls: A longitudinal study

Background

The high prevalence of smoking in individuals who are at ultra-high risk (UHR) for psychosis is well known and moderate cognitive deficits have also been found in UHR. However, the association between smoking and cognition in UHR is unknown and longitudinal studies are lacking.

Method

A cohort study with 330 UHR individuals and 66 controls was conducted, as part of the European network of national schizophrenia networks studying gene–environment interactions (EU-GEI). At baseline and after 6, 12, and 24 months, smoking behavior was assessed with the Composite International Diagnostic Interview and cognitive functioning with a comprehensive test battery. Linear mixed-effects analyses were used to examine the multicross-sectional and prospective associations between (change in) smoking behavior and cognitive functioning, accounting for confounding variables.

Results

At baseline, 53% of UHR and 27% of controls smoked tobacco. Smoking UHR and controls did not significantly differ from nonsmoking counterparts on the tested cognitive domains (speed of processing, attention/vigilance, working memory, verbal learning, or reasoning/problem solving) across different assessment times. Neither smoking cessation nor initiation was associated with a significant change in cognitive functioning in UHR.

Conclusions

No associations were found between smoking and cognitive impairment in UHR nor in controls. However, the fact that one in every two UHR individuals report daily use of tobacco is alarming. Our data suggest that UHR have fewer cognitive impairments and higher smoking cessation rates compared to patients with first-episode psychosis found in literature. Implications to promote smoking cessation in the UHR stage need further investigation.

The continuity of effect of schizophrenia polygenic risk score and patterns of cannabis use on transdiagnostic symptom dimensions at first-episode psychosis: findings from the EU-GEI study

Diagnostic categories do not completely reflect the heterogeneous expression of psychosis. Using data from the EU-GEI study, we evaluated the impact of schizophrenia polygenic risk score (SZ-PRS) and patterns of cannabis use on the transdiagnostic expression of psychosis. We analysed first-episode psychosis patients (FEP) and controls, generating transdiagnostic dimensions of psychotic symptoms and experiences using item response bi-factor modelling. Linear regression was used to test the associations between these dimensions and SZ-PRS, as well as the combined effect of SZ-PRS and cannabis use on the dimensions of positive psychotic symptoms and experiences. We found associations between SZ-PRS and (1) both negative (B = 0.18; 95%CI 0.03-0.33) and positive (B = 0.19; 95%CI 0.03-0.35) symptom dimensions in 617 FEP patients, regardless of their categorical diagnosis; and (2) all the psychotic experience dimensions in 979 controls. We did not observe associations between SZ-PRS and the general and affective dimensions in FEP. Daily and current cannabis use were associated with the positive dimensions in FEP (B = 0.31; 95%CI 0.11-0.52) and in controls (B = 0.26; 95%CI 0.06-0.46), over and above SZ-PRS. We provide evidence that genetic liability to schizophrenia and cannabis use map onto transdiagnostic symptom dimensions, supporting the validity and utility of the dimensional representation of psychosis. In our sample, genetic liability to schizophrenia correlated with more severe psychosis presentation, and cannabis use conferred risk to positive symptomatology beyond the genetic risk. Our findings support the hypothesis that psychotic experiences in the general population have similar genetic substrates as clinical disorders.

An fMRI study of cognitive remediation in drug-naïve subjects diagnosed with first episode schizophrenia

Objective

The purpose of our functional magnetic resonance imaging (fMRI) study was to examine brain activity using a “1-back” paradigm as working memory task in drug-naïve subjects with first episode schizophrenia before and after cognitive remediation training.

Methods

In this study 15 drug-naïve first episode subjects who met DSM-IV criteria for schizophrenia were randomized to receive either atypical antipsychotics (AP, n = 8) or atypical antipsychotics in combination with cognitive remediation therapy (AP + CR, n = 7), 11 subjects had a follow-up fMRI examination after therapy (AP, n = 5; AP + CR, n = 6).

Results

In 4 of the 6 AP + CR subjects the number of activation clusters increased, whereas in 4 out of the 5 AP subjects the number of clusters decreased (mean number of clusters: AP + CR = 5.53, SD 12.79, AP = −5.8, SD 6.9).

Conclusion

In this randomized study the number of activation clusters during a working memory task increased after cognitive remediation training. Our data show that neurobiological effects of cognitive remediation can be identified in the very early course of schizophrenia.

The bumpy road to achieve reliability of clinical profile characteristics in psychosis and related disorders

OBJECTIVES

Profile characteristics are factors that are relevant for diagnosis, prognosis or treatment. The present study aims to develop a set of clinically relevant profile characteristics. Moreover, our goal is to determine the inter-rater reliability (IRR) of the selected profile characteristics.

METHODS

Potential profile characteristics were determined by literature review. Assessment of IRR was done by comparing scores on profile characteristics determined by two researchers. We conducted three subsequent studies: (1) assessment of pre-training IRR, (2) IRR following implementation of an instruction manual, (3) IRR after optimizing scoring methods. IRR was measured with the Intraclass Correlation Coefficient (ICC).

RESULTS

IRR scores of profile characteristic Illegal activities were high across the three studies (ICC ≥ 0.75). Following training procedures in study 2 and 3, reliability estimates remained low to moderate (ICC < 0.75) for the profile characteristics Support of relatives, Aggression recent and lifetime, substance use and insight recent. IRR scores of the other eight profile characteristics varied from low, moderate to high across studies.

CONCLUSION

IRR scores of profile characteristics were highly variable, and mostly inadequate in all three studies. Consequently, further research should focus on specification of severity scores of profile characteristics, optimizing scoring methods and re-evaluation of IRR.