The Early Psychosis Screener for Internet (EPSI)-SR: Predicting 12 month psychotic conversion using machine learning

INTRODUCTION

A faster and more accurate self-report screener for early psychosis is needed to promote early identification and intervention.

METHODS

Self-report Likert-scale survey items were administered to individuals being screened with the Structured Interview for Psychosis-risk Syndromes (SIPS) and followed at eight early psychosis clinics. An a priori analytic plan included Spectral Clustering Analysis to reduce the item pool, followed by development of Support Vector Machine (SVM) classifiers.

RESULTS

The cross-validated positive predictive value (PPV) of the EPSI at the default cut-off (76.5%) exceeded that of the clinician-administered SIPS (68.5%) at separating individuals who would not convert to psychosis within 12 months from those who either would convert within 12 months or who had already experienced a first episode psychosis (FEP). When used in tandem with the SIPS on clinical high risk participants, the EPSI increased the combined PPV to 86.6%. The SVM classified as FEP/converters only 1% of individuals in non-clinical and 4% of clinical low risk populations. Sensitivity of the EPSI, however, was 51% at the default cut-off.

DISCUSSION

The EPSI identifies, comparably to the SIPS but in less time and with fewer resources, individuals who are either at very high risk to develop a psychotic disorder within 12 months or who are already psychotic. At its default cut-off, EPSI misses 49% of current or future psychotic cases. The cut-off can, however, be adjusted based on purpose. The EPSI is the first validated assessment to predict 12-month psychotic conversion. An online screening system, www.eps.telesage.org, is under development.

The Early Psychosis Screener (EPS): Item development and qualitative validation

A panel of experts assembled and analyzed a comprehensive item bank from which a highly sensitive and specific early psychosis screener could be developed. Twenty well-established assessments relating to the prodromal stage, early psychosis, and psychosis were identified. Using DSM-5 criteria, we identified the core concepts represented by each of the items in each of the assessments. These granular core concepts were converted into a uniform set of 490 self-report items using a Likert scale and a 'past 30days' time frame. Partial redundancy was allowed to assure adequate concept coverage. A panel of experts and TeleSage staff rated these items and eliminated 189 items, resulting in 301 items. The items were subjected to five rounds of cognitive interviewing with 16 individuals at clinically high risk for psychosis and 26 community mental health center patients. After each round, the expert panel iteratively reviewed, rated, revised, added, or deleted items to maximize clarity and centrality to the concept. As a result of the interviews, 36 items were revised, 52 items were added, and 205 items were deleted. By the last round of cognitive interviewing, all of the items were clearly understood by all participants. In future work, responses to the final set of 148 items and machine learning techniques will be used to quantitatively identify the subset of items that will best predict clinical high-risk status and conversion.

The Early Psychosis Screener (EPS): Quantitative validation against the SIPS using machine learning

Machine learning techniques were used to identify highly informative early psychosis self-report items and to validate an early psychosis screener (EPS) against the Structured Interview for Psychosis-risk Syndromes (SIPS). The Prodromal Questionnaire-Brief Version (PQ-B) and 148 additional items were administered to 229 individuals being screened with the SIPS at 7 North American Prodrome Longitudinal Study sites and at Columbia University. Fifty individuals were found to have SIPS scores of 0, 1, or 2, making them clinically low risk (CLR) controls; 144 were classified as clinically high risk (CHR) (SIPS 3-5) and 35 were found to have first episode psychosis (FEP) (SIPS 6). Spectral clustering analysis, performed on 124 of the items, yielded two cohesive item groups, the first mostly related to psychosis and mania, the second mostly related to depression, anxiety, and social and general work/school functioning. Items within each group were sorted according to their usefulness in distinguishing between CLR and CHR individuals using the Minimum Redundancy Maximum Relevance procedure. A receiver operating characteristic area under the curve (AUC) analysis indicated that maximal differentiation of CLR and CHR participants was achieved with a 26-item solution (AUC=0.899±0.001). The EPS-26 outperformed the PQ-B (AUC=0.834±0.001). For screening purposes, the self-report EPS-26 appeared to differentiate individuals who are either CLR or CHR approximately as well as the clinician-administered SIPS. The EPS-26 may prove useful as a self-report screener and may lead to a decrease in the duration of untreated psychosis. A validation of the EPS-26 against actual conversion is underway.

Satisfaction of forensic psychiatric patients with remote telepsychiatric evaluation

The level of satisfaction with telepsychiatry evaluations was determined in a sample of 43 forensic psychiatric patient inmates in a large urban jail. A forensic psychiatrist interviewed 20 patients in person, the other 23 remotely via interactive video. Demographic characteristics, physical health status, and psychiatric symptom severity on the Global Severity Index of the Brief Symptom Inventory were comparable in the two groups. Patient satisfaction with the evaluations was moderately high for patients in both groups, with no significant differences between them.

Satisfaction of forensic psychiatric patients with remote telepsychiatric evaluation

The level of satisfaction with telepsychiatry evaluations was determined in a sample of 43 forensic psychiatric patient inmates in a large urban jail. A forensic psychiatrist interviewed 20 patients in person, the other 23 remotely via interactive video. Demographic characteristics, physical health status, and psychiatric symptom severity on the Global Severity Index of the Brief Symptom Inventory were comparable in the two groups. Patient satisfaction with the evaluations was moderately high for patients in both groups, with no significant differences between them.