Comparing machine learning to a rule-based approach for predicting suicidal behavior among adolescents: Results from a longitudinal population-based survey

Age-stratified predictions of suicide attempts using machine learning in middle and late adolescence

BACKGROUND

Prevalence of suicidal behaviour increases rapidly in middle to late adolescence. Predicting suicide attempts across different ages would enhance our understanding of how suicidal behaviour manifests in this period of rapid development. This study aimed to develop separate models to predict suicide attempts within a cohort at middle and late adolescence. It also sought to examine differences between the models derived across both developmental stages.

METHODS

This study used data from the nationally representative Longitudinal Study of Australian Children (N = 2266). We selected over 700 potential suicide attempt predictors measured via self-report questionnaires, and linked healthcare and education administrative datasets. Logistic regression, random forests, and gradient boosting algorithms were developed to predict suicide attempts across two stages (mid-adolescence: 14-15 years; late adolescence: 18-19 years) using predictors sampled two years prior (mid-adolescence: 12-13 years; late adolescence: 16-17 years).

RESULTS

The late adolescence models (AUROC = 0.77-0.88, F1-score = 0.22-0.28, Sensitivity = 0.54-0.64) performed better than the mid-adolescence models (AUROC = 0.70-0.76, F1-score = 0.12-0.19, Sensitivity = 0.40-0.64). The most important features for predicting suicide attempts in mid-adolescence were mostly school-related, while the most important features in late adolescence included measures of prior suicidality, psychosocial health, and future plans.

CONCLUSIONS

To date, this is the first study to use machine learning models to predict suicide attempts at different ages. Our findings suggest that the optimal suicide risk prediction model differs by stage of adolescence. Future research and interventions should consider that risk presentations can change rapidly during adolescence.

Factors and Reasons Associated with Hesitating to Seek Care for Migraine: Results of the OVERCOME (US) Study

INTRODUCTION

Despite a variety of available treatment options for migraine, many people with migraine do not seek medical care, thereby reducing opportunities for diagnosis and effective treatment and potentially leading to missed opportunities to reduce the burden of disease. Understanding why people hesitate to seek care for migraine may help healthcare professionals and advocates address barriers and improve outcomes. The aim of this study, in a large adult population sample in the United States (US), was to identify factors associated with and reasons for hesitating to seek healthcare for migraine.

METHODS

The web-based OVERCOME (US) survey study identified adults with active migraine in a demographically representative US sample who answered questions about hesitating to seek care from a healthcare provider for migraine and reasons for hesitating. Supervised machine learning (random forest, least absolute shrinkage and selection operator) identified factors associated with hesitation; logistic regression models assessed association of factors on hesitation.

RESULTS

The study results show that of the 58,403 participants with active migraine who completed the OVERCOME (US) baseline survey and provided responses to the question on hesitating to seek care for migraine, 45.1% (n = 26,330/58,403) with migraine indicated that they had ever hesitated to seek care for migraine. Factors most associated with hesitating to seek care were hiding migraine (odds ratio [OR] = 2.69; 95% confidence interval [CI]: 2.50, 2.89), experiencing migraine-related stigma (OR = 2.13; 95% CI 1.95, 2.33), higher migraine-related disability (OR = 1.30; 95% CI 1.23, 1.38), and higher ictal cutaneous allodynia (OR = 1.26; 95% CI 1.19, 1.35). The most common reasons participants stated for hesitating included (1) 44.2% wanting to try and take care of migraine on their own, (2) 33.8% feeling that their migraine or headache would not be taken seriously, (3) 29.2% thinking that their migraine was not serious/painful enough, and (4) 27.4% not being able to afford it or not wanting to spend the money. The main limitation of the study includes the requirement for respondents to have internet, access which may have reflected cohort bias, and the quota sampling rather than random sampling to create a demographically representative sample.

CONCLUSIONS

Hesitating to seek migraine care is common and is most strongly associated with hiding the disease and migraine-related stigma. Those experiencing higher migraine-related burden are more hesitant to seek the care that might alleviate the burden. These findings suggest that migraine's social context (e.g., stigma) is a major determinant of hesitance to seek migraine care.

Examining Highly Novel Positive Future Thinking in Suicidal and Nonsuicidal Adolescents

OBJECTIVE

Although traditionally considered protective, certain forms of positive future thinking (PFT) may be associated with greater suicide risk. In this first a priori investigation of potential maladaptive forms of PFT, we tested whether novelty (i.e., dissimilarity to past experiences) and lack of attainment of the imagined positive future may explain counterintuitive associations between PFT and suicidal ideation (SI).

METHOD

At baseline, adolescents (N = 76, ages 12-19) completed a behavioral measure of PFT (i.e., Future Thinking Task) and rated the novelty of each positive future thought. At the 3-month follow-up, we measured attainment of the positive future events generated at baseline by asking adolescents whether the event happened and, if it did, if it was as positive as had been imagined at baseline. Past-month SI severity was assessed at baseline, 3 months, and 6 months.

RESULTS

PFT, only when highly novel, was associated with stronger recent SI severity at baseline, above and beyond depressive symptoms. It also significantly predicted recent SI severity 3 and 6 months later, although not after we controlled for baseline SI severity. Novelty of the imagined positive future was not related to whether the event happened. However, when those events did happen, adolescents who imagined more novel events tended to experience them less positively than imagined, which separately predicted stronger recent SI severity at the 6-month follow-up.

CONCLUSIONS

Results support that PFT is a heterogeneous construct that is not uniformly beneficial. Better understanding potential pitfalls of PFT may help us discern how to best incorporate PFT into clinical interventions.

Predicting inmate suicidal behavior with an interpretable ensemble machine learning approach in smart prisons

The convergence of smart technologies and predictive modelling in prisons presents an exciting opportunity to revolutionize the monitoring of inmate behaviour, allowing for the early detection of signs of distress and the effective mitigation of suicide risks. While machine learning algorithms have been extensively employed in predicting suicidal behaviour, a critical aspect that has often been overlooked is the interoperability of these models. Most of the work done on model interpretations for suicide predictions often limits itself to feature reduction and highlighting important contributing features only. To address this research gap, we used Anchor explanations for creating human-readable statements based on simple rules, which, to our knowledge, have never been used before for suicide prediction models. We also overcome the limitation of anchor explanations, which create weak rules on high-dimensionality datasets, by first reducing data features with the help of SHapley Additive exPlanations (SHAP). We further reduce data features through anchor interpretations for the final ensemble model of XGBoost and random forest. Our results indicate significant improvement when compared with state-of-the-art models, having an accuracy and precision of 98.6% and 98.9%, respectively. The F1-score for the best suicide ideation model appeared to be 96.7%.

Development of a Computable Phenotype for Prehospital Pediatric Asthma Encounters

INTRODUCTION

Asthma exacerbations are a common cause of pediatric Emergency Medical Services (EMS) encounters. Accordingly, prehospital management of pediatric asthma exacerbations has been designated an EMS research priority. However, accurate identification of pediatric asthma exacerbations from the prehospital record is nuanced and difficult due to the heterogeneity of asthma symptoms, especially in children. Therefore, this study's objective was to develop a prehospital-specific pediatric asthma computable phenotype (CP) that could accurately identify prehospital encounters for pediatric asthma exacerbations.

METHODS

This is a retrospective observational study of patient encounters for ages 2-18 years from the ESO Data Collaborative between 2018 and 2021. We modified two existing rule-based pediatric asthma CPs and created three new CPs (one rule-based and two machine learning-based). Two pediatric emergency medicine physicians independently reviewed encounters to assign labels of asthma exacerbation or not. Taking that labeled encounter data, a 50/50 train/test split was used to create training and test sets from the labeled data. A 90/10 split was used to create a small validation set from the training set. We used specificity, sensitivity, positive predictive value (PPV), negative predictive value (NPV) and macro F1 to compare performance across all CP models.

RESULTS

After applying the inclusion and exclusion criteria, 24,283 patient encounters remained. The machine-learning models exhibited the best performance for the identification of pediatric asthma exacerbations. A multi-layer perceptron-based model had the best performance in all metrics, with an F1 score of 0.95, specificity of 1.00, sensitivity of 0.91, negative predictive value of 0.98, and positive predictive value of 1.00.

CONCLUSION

We modified existing and developed new pediatric asthma CPs to retrospectively identify prehospital pediatric asthma exacerbation encounters. We found that machine learning-based models greatly outperformed rule-based models. Given the high performance of the machine-learning models, the development and application of machine learning-based CPs for other conditions and diseases could help accelerate EMS research and ultimately enhance clinical care by accurately identifying patients with conditions of interest.

Predicting Lifetime Suicide Attempts in a Community Sample of Adolescents Using Machine Learning Algorithms

Suicide is a major global health concern and a prominent cause of death in adolescents. Previous research on suicide prediction has mainly focused on clinical or adult samples. To prevent suicides at an early stage, however, it is important to screen for risk factors in a community sample of adolescents. We compared the accuracy of logistic regressions, elastic net regressions, and gradient boosting machines in predicting suicide attempts by 17-year-olds in the Millennium Cohort Study (N = 7,347), combining a large set of self- and other-reported variables from different categories. Both machine learning algorithms outperformed logistic regressions and achieved similar balanced accuracies (.76 when using data 3 years before the self-reported lifetime suicide attempts and .85 when using data from the same measurement wave). We identified essential variables that should be considered when screening for suicidal behavior. Finally, we discuss the usefulness of complex machine learning models in suicide prediction.

Predicting suicide attempts among Norwegian adolescents without using suicide-related items: a machine learning approach

Introduction

Research on the classification models of suicide attempts has predominantly depended on the collection of sensitive data related to suicide. Gathering this type of information at the population level can be challenging, especially when it pertains to adolescents. We addressed two main objectives: (1) the feasibility of classifying adolescents at high risk of attempting suicide without relying on specific suicide-related survey items such as history of suicide attempts, suicide plan, or suicide ideation, and (2) identifying the most important predictors of suicide attempts among adolescents.

Methods

Nationwide survey data from 173,664 Norwegian adolescents (ages 13-18) were utilized to train a binary classification model, using 169 questionnaire items. The Extreme Gradient Boosting (XGBoost) algorithm was fine-tuned to classify adolescent suicide attempts, and the most important predictors were identified.

Results

XGBoost achieved a sensitivity of 77% with a specificity of 90%, and an AUC of 92.1% and an AUPRC of 47.1%. A coherent set of predictors in the domains of internalizing problems, substance use, interpersonal relationships, and victimization were pinpointed as the most important items related to recent suicide attempts.

Conclusion

This study underscores the potential of machine learning for screening adolescent suicide attempts on a population scale without requiring sensitive suicide-related survey items. Future research investigating the etiology of suicidal behavior may direct particular attention to internalizing problems, interpersonal relationships, victimization, and substance use.

Are false positives in suicide classification models a risk group? Evidence for "true alarms" in a population-representative longitudinal study of Norwegian adolescents

Introduction

False positives in retrospective binary suicide attempt classification models are commonly attributed to sheer classification error. However, when machine learning suicide attempt classification models are trained with a multitude of psycho-socio-environmental factors and achieve high accuracy in suicide risk assessment, false positives may turn out to be at high risk of developing suicidal behavior or attempting suicide in the future. Thus, they may be better viewed as "true alarms," relevant for a suicide prevention program. In this study, using large population-based longitudinal dataset, we examine three hypotheses: (1) false positives, compared to the true negatives, are at higher risk of suicide attempt in future, (2) the suicide attempts risk for the false positives increase as a function of increase in specificity threshold; and (3) as specificity increases, the severity of risk factors between false positives and true positives becomes more similar.

Methods

Utilizing the Gradient Boosting algorithm, we used a sample of 11,369 Norwegian adolescents, assessed at two timepoints (1992 and 1994), to classify suicide attempters at the first time point. We then assessed the relative risk of suicide attempt at the second time point for false positives in comparison to true negatives, and in relation to the level of specificity.

Results

We found that false positives were at significantly higher risk of attempting suicide compared to true negatives. When selecting a higher classification risk threshold by gradually increasing the specificity cutoff from 60% to 97.5%, the relative suicide attempt risk of the false positive group increased, ranging from minimum of 2.96 to 7.22 times. As the risk threshold increased, the severity of various mental health indicators became significantly more comparable between false positives and true positives.

Conclusion

We argue that the performance evaluation of machine learning suicide classification models should take the clinical relevance into account, rather than focusing solely on classification error metrics. As shown here, the so-called false positives represent a truly at-risk group that should be included in suicide prevention programs. Hence, these findings should be taken into consideration when interpreting machine learning suicide classification models as well as planning future suicide prevention interventions for adolescents.

Demand prediction of medical services in home and community-based services for older adults in China using machine learning

Background

Home and community-based services are considered an appropriate and crucial caring method for older adults in China. However, the research examining demand for medical services in HCBS through machine learning techniques and national representative data has not yet been carried out. This study aimed to address the absence of a complete and unified demand assessment system for home and community-based services.

Methods

This was a cross-sectional study conducted on 15,312 older adults based on the Chinese Longitudinal Healthy Longevity Survey 2018. Models predicting demand were constructed using five machine-learning methods: Logistic regression, Logistic regression with LASSO regularization, Support Vector Machine, Random Forest, and Extreme Gradient Boosting (XGboost), and based on Andersen's behavioral model of health services use. Methods utilized 60% of older adults to develop the model, 20% of the samples to examine the performance of models, and the remaining 20% of cases to evaluate the robustness of the models. To investigate demand for medical services in HCBS, individual characteristics such as predisposing, enabling, need, and behavior factors constituted four combinations to determine the best model.

Results

Random Forest and XGboost models produced the best results, in which both models were over 80% at specificity and produced robust results in the validation set. Andersen's behavioral model allowed for combining odds ratio and estimating the contribution of each variable of Random Forest and XGboost models. The three most critical features that affected older adults required medical services in HCBS were self-rated health, exercise, and education.

Conclusion

Andersen's behavioral model combined with machine learning techniques successfully constructed a model with reasonable predictors to predict older adults who may have a higher demand for medical services in HCBS. Furthermore, the model captured their critical characteristics. This method predicting demands could be valuable for the community and managers in arranging limited primary medical resources to promote healthy aging.

Alpha-beta decoupling relevant to inhibition deficits leads to suicide attempt in major depressive disorder

BACKGROUND

One devastating outcome of major depressive disorder (MDD) is high suicidality, especially for patients with suicide attempt (SA). Evidence indicated that SA may be strongly associated with inhibitory control deficits. We hypothesized that the inhibition function deficits of patient with SA might be underpinned by abnormal neuronal oscillations.

METHODS

Our study recruited 111 subjects including 74 patients and 37 controls, who performed a GO/NOGO task during magnetoencephalography recording. Time-frequency-representations and phase-amplitude-coupling were measured for the brain circuits involved in the inhibitory function. Phase-slope-indexes were calculated between regions to determine the direction of power flow.

RESULTS

Significant increased reaction time and decreased judgment accuracy were observed in SA group. During the perception stage of GO task (approximately 125 ms), SA group manifested elevated alpha power in ventral prefrontal cortex (VPFC) and attenuated beta power in dorsal anterior cingulate (dACC) compared with other groups (p < 0.01). In the processing stage of NOGO task (approximately 300 ms), they showed decreased beta power in VPFC and increased alpha power in dACC (p < 0.01). Alpha-beta decoupling during both tasks was observed in SA group. Furthermore, the decoupling from VPFC to dACC under NOGO tasks was significantly correlated with suicide risk level.

LIMITATIONS

The number of participants was relatively small, and psychological elements were not involved in current study.

CONCLUSION

Dysregulated oscillatory activities of dACC and VPFC suggested deficits in execution and inhibition functions triggering high suicide risks. The alpha-beta decoupling from VPFC to dACC could be served as a neuro-electrophysiological biomarker for identifying potential suicide risk.