Classifying Schizophrenia Cases by Artificial Neural Network Using Japanese Web-Based Survey Data: Case-Control Study

External Validation of a Machine Learning Model for Schizophrenia Classification

Background and Objective: Excellent generalizability is the precondition for the widespread practical implementation of machine learning models. In our previous study, we developed the schizophrenia classification model (SZ classifier) to identify potential schizophrenia patients in the Japanese population. The SZ classifier has exhibited impressive performance during internal validation. However, ensuring the robustness and generalizability of the SZ classifier requires external validation across independent sample sets. In this study, we aimed to present an external validation of the SZ classifier using outpatient data. Methods: The SZ classifier was trained by using online survey data, which incorporate demographic, health-related, and social comorbidity features. External validation was conducted using an outpatient sample set which is independent from the sample set during the model development phase. The model performance was assessed based on the sensitivity and misclassification rates for schizophrenia, bipolar disorder, and major depression patients. Results: The SZ classifier demonstrated a sensitivity of 0.75 when applied to schizophrenia patients. The misclassification rates were 59% and 55% for bipolar disorder and major depression patients, respectively. Conclusions: The SZ classifier currently encounters challenges in accurately determining the presence or absence of schizophrenia at the individual level. Prior to widespread practical implementation, enhancements are necessary to bolster the accuracy and diminish the misclassification rates. Despite the current limitations of the model, such as poor specificity for certain psychiatric disorders, there is potential for improvement if including multiple types of psychiatric disorders during model development.

Can feature structure improve model's precision? A novel prediction method using artificial image and image identification

Objectives

This study aimed to develop an approach to enhance the model precision by artificial images.

Materials and Methods

Given an epidemiological study designed to predict 1 response using f features with M samples, each feature was converted into a pixel with certain value. Permutated these pixels into F orders, resulting in F distinct artificial image sample sets. Based on the experience of image recognition techniques, appropriate training images results in higher precision model. In the preliminary experiment, a binary response was predicted by 76 features, the sample set included 223 patients and 1776 healthy controls.

Results

We randomly selected 10 000 artificial sample sets to train the model. Models' performance (area under the receiver operating characteristic curve values) depicted a bell-shaped distribution.

Conclusion

The model construction strategy developed in the research has potential to capture feature order related information and enhance model predictability.