Illusory generalizability of clinical prediction models

Ethnokinesiology: towards a neuromechanical understanding of cultural differences in movement

Each individual's movements are sculpted by constant interactions between sensorimotor and sociocultural factors. A theoretical framework grounded in motor control mechanisms articulating how sociocultural and biological signals converge to shape movement is currently missing. Here, we propose a framework for the emerging field of ethnokinesiology aiming to provide a conceptual space and vocabulary to help bring together researchers at this intersection. We offer a first-level schema for generating and testing hypotheses about cultural differences in movement to bridge gaps between the rich observations of cross-cultural movement variations and neurophysiological and biomechanical accounts of movement. We explicitly dissociate two interacting feedback loops that determine culturally relevant movement: one governing sensorimotor tasks regulated by neural signals internal to the body, the other governing ecological tasks generated through actions in the environment producing ecological consequences. A key idea is the emergence of individual-specific and culturally influenced motor concepts in the nervous system, low-dimensional functional mappings between sensorimotor and ecological task spaces. Motor accents arise from perceived differences in motor concept topologies across cultural contexts. We apply the framework to three examples: speech, gait and grasp. Finally, we discuss how ethnokinesiological studies may inform personalized motor skill training and rehabilitation, and challenges moving forward.This article is part of the theme issue 'Minds in movement: embodied cognition in the age of artificial intelligence'.

Machine learning and Bayesian network analyses identifies associations with insomnia in a national sample of 31,285 treatment-seeking college students

BACKGROUND

A better understanding of the relationships between insomnia and anxiety, mood, eating, and alcohol-use disorders is needed given its prevalence among young adults. Supervised machine learning provides the ability to evaluate which mental disorder is most associated with heightened insomnia among U.S. college students. Combined with Bayesian network analysis, probable directional relationships between insomnia and interacting symptoms may be illuminated.

METHODS

The current exploratory analyses utilized a national sample of college students across 26 U.S. colleges and universities collected during population-level screening before entering a randomized controlled trial. We used a 4-step statistical approach: (1) at the disorder level, an elastic net regularization model examined the relative importance of the association between insomnia and 7 mental disorders (major depressive disorder, generalized anxiety disorder, social anxiety disorder, panic disorder, post-traumatic stress disorder, anorexia nervosa, and alcohol use disorder); (2) This model was evaluated within a hold-out sample. (3) at the symptom level, a completed partially directed acyclic graph (CPDAG) was computed via a Bayesian hill-climbing algorithm to estimate potential directionality among insomnia and its most associated disorder [based on SHAP (SHapley Additive exPlanations) values)]; (4) the CPDAG was then tested for generalizability by assessing (in)equality within a hold-out sample using structural hamming distance (SHD).

RESULTS

Of 31,285 participants, 20,597 were women (65.8%); mean (standard deviation) age was 22.96 (4.52) years. The elastic net model demonstrated clinical significance in predicting insomnia severity in the training sample [R2 = .44 (.01); RMSE = 5.00 (0.08)], with comparable performance in the hold-out sample (R2 = .33; RMSE = 5.47). SHAP values indicated that the presence of any mental disorder was associated with higher insomnia scores, with major depressive disorder as the most important disorder associated with heightened insomnia (mean |SHAP|= 3.18). The training CPDAG and hold-out CPDAG (SHD = 7) suggested depression symptoms presupposed insomnia with depressed mood, fatigue, and self-esteem as key parent nodes.

CONCLUSION

These findings provide insights into the associations between insomnia and mental disorders among college students and warrant further investigation into the potential direction of causality between insomnia and depression.

TRIAL REGISTRATION

Trial was registered on the National Institute of Health RePORTER website (R01MH115128 || 23/08/2018).

What matters beyond particle matter?: Examining air pollution's synergistic effects on bodies and health through Bio3Science in Medellin

Scientific literature on the health effects of air pollution is diverse, and broadly acknowledges the importance of human experience and social and economic precarity as modifying factors. Still, the inclusion of the embodied experience of air pollution has been limited. Also, the health effects of pollution are often studied at the group or population level, without adequately considering individual difference. This paper uses a Bio3Science framework, which integrates biology, biography, and biosphere, to explore how air pollution affects residents in Medellín, Colombia. By using qualitative research on individual experiences of air pollution (biography) to probe the intersection of individual health (biology) and environment (biosphere), we illustrate how pollution shapes lived rhythms at multiple scales. Our findings emphasize that air pollution's health impacts extend beyond measurable pollutants to include the complex synergies of smoke, noise, stress, and disruptions to daily life. This comprehensive approach provides a nuanced understanding of how air pollution materially shapes the lives of individuals and communities, advocating for research models that capture the subtle, everyday experiences often overlooked by traditional group or population-level analyses.

Initial evaluation of a personalized advantage index to determine which individuals may benefit from mindfulness-based cognitive therapy for suicide prevention

OBJECTIVE

Develop and evaluate a treatment matching algorithm to predict differential treatment response to Mindfulness-Based Cognitive Therapy for suicide prevention (MBCT-S) versus enhanced treatment-as-usual (eTAU).

METHODS

Analyses used data from Veterans at high-risk for suicide assigned to either MBCT-S (n = 71) or eTAU (n = 69) in a randomized clinical trial. Potential predictors (n = 55) included available demographic, clinical, and neurocognitive variables. Random forest models were used to predict risk of suicidal event (suicidal behaviors, or ideation resulting in hospitalization or emergency department visit) within 12 months following randomization, characterize the prediction, and develop a Personalized Advantage Index (PAI).

RESULTS

A slightly better prediction model emerged for MBCT-S (AUC = 0.70) than eTAU (AUC = 0.63). Important outcome predictors for participants in the MBCT-S arm included PTSD diagnosis, decisional efficiency on a neurocognitive task (Go/No-Go), prior-year mental health residential treatment, and non-suicidal self-injury. Significant predictors for participants in the eTAU arm included past-year acute psychiatric hospitalizations, past-year outpatient psychotherapy visits, past-year suicidal ideation severity, and attentional control (indexed by Stroop task). A moderation analysis showed that fewer suicidal events occurred among those randomized to their PAI-indicated optimal treatment.

CONCLUSIONS

PAI-guided treatment assignment may enhance suicide prevention outcomes. However, prior to real-world application, additional research is required to improve model accuracy and evaluate model generalization.

Are Psychiatric Nosologies Limiting the Success of Clinical Prediction Models?

This Viewpoint discusses the limitations of clinical prediction models in psychiatric research.

An updated AL-Base reveals ranked enrichment of immunoglobulin light chain variable genes in AL amyloidosis

Background

Each monoclonal antibody light chain associated with AL amyloidosis has a unique sequence. Defining how these sequences lead to amyloid deposition could facilitate faster diagnosis and lead to new treatments.

Methods

Light chain sequences are collected in the Boston University AL-Base repository. Monoclonal sequences from AL amyloidosis, multiple myeloma and the healthy polyclonal immune repertoire were compared to identify differences in precursor gene use, mutation frequency and physicochemical properties.

Results

AL-Base now contains 2,193 monoclonal light chain sequences from plasma cell dyscrasias. Sixteen germline precursor genes were enriched in AL amyloidosis, relative to multiple myeloma and the polyclonal repertoire. Two genes, IGKV1-16 and IGLV1-36, were infrequently observed but highly enriched in AL amyloidosis. The number of mutations varied widely between light chains. AL-associated κ light chains harbored significantly more mutations compared to multiple myeloma and polyclonal sequences, whereas AL-associated λ light chains had fewer mutations. Machine learning tools designed to predict amyloid propensity were less accurate for new sequences than their original training data.

Conclusions

Rarely-observed light chain variable genes may carry a high risk of AL amyloidosis. New approaches are needed to define sequence-associated risk factors for AL amyloidosis. AL-Base is a foundational resource for such studies.

A review of model evaluation metrics for machine learning in genetics and genomics

Machine learning (ML) has shown great promise in genetics and genomics where large and complex datasets have the potential to provide insight into many aspects of disease risk, pathogenesis of genetic disorders, and prediction of health and wellbeing. However, with this possibility there is a responsibility to exercise caution against biases and inflation of results that can have harmful unintended impacts. Therefore, researchers must understand the metrics used to evaluate ML models which can influence the critical interpretation of results. In this review we provide an overview of ML metrics for clustering, classification, and regression and highlight the advantages and disadvantages of each. We also detail common pitfalls that occur during model evaluation. Finally, we provide examples of how researchers can assess and utilise the results of ML models, specifically from a genomics perspective.

Transatlantic transferability and replicability of machine-learning algorithms to predict mental health crises

Transferring and replicating predictive algorithms across healthcare systems constitutes a unique yet crucial challenge that needs to be addressed to enable the widespread adoption of machine learning in healthcare. In this study, we explored the impact of important differences across healthcare systems and the associated Electronic Health Records (EHRs) on machine-learning algorithms to predict mental health crises, up to 28 days in advance. We evaluated both the transferability and replicability of such machine learning models, and for this purpose, we trained six models using features and methods developed on EHR data from the Birmingham and Solihull Mental Health NHS Foundation Trust in the UK. These machine learning models were then used to predict the mental health crises of 2907 patients seen at the Rush University System for Health in the US between 2018 and 2020. The best one was trained on a combination of US-specific structured features and frequency features from anonymized patient notes and achieved an AUROC of 0.837. A model with comparable performance, originally trained using UK structured data, was transferred and then tuned using US data, achieving an AUROC of 0.826. Our findings establish the feasibility of transferring and replicating machine learning models to predict mental health crises across diverse hospital systems.

Bringing Imaging Biomarkers Into Clinical Reality in Psychiatry

Importance

Advancing precision psychiatry, where treatments are based on an individual's biology rather than solely their clinical presentation, requires attention to several key attributes for any candidate biomarker. These include test-retest reliability, sensitivity to relevant neurophysiology, cost-effectiveness, and scalability. Unfortunately, these issues have not been systematically addressed by biomarker development efforts that use common neuroimaging tools like magnetic resonance imaging (MRI) and electroencephalography (EEG). Here, the critical barriers that neuroimaging methods will need to overcome to achieve clinical relevance in the near to intermediate term are examined.

Observations

Reliability is often overlooked, which together with sensitivity to key aspects of neurophysiology and replicated predictive utility, favors EEG-based methods. The principal barrier for EEG has been the lack of large-scale data collection among multisite psychiatric consortia. By contrast, despite its high reliability, structural MRI has not demonstrated clinical utility in psychiatry, which may be due to its limited sensitivity to psychiatry-relevant neurophysiology. Given the prevalence of structural MRIs, establishment of a compelling clinical use case remains its principal barrier. By contrast, low reliability and difficulty in standardizing collection are the principal barriers for functional MRI, along with the need for demonstration that its superior spatial resolution over EEG and ability to directly image subcortical regions in fact provide unique clinical value. Often missing, moreover, is consideration of how these various scientific issues can be balanced against practical economic realities of psychiatric health care delivery today, for which embedding economic modeling into biomarker development efforts may help direct research efforts.

Conclusions and Relevance

EEG seems most ripe for near- to intermediate-term clinical impact, especially considering its scalability and cost-effectiveness. Recent efforts to broaden its collection, as well as development of low-cost turnkey systems, suggest a promising pathway by which neuroimaging can impact clinical care. Continued MRI research focused on its key barriers may hold promise for longer-horizon utility.

Childhood Trauma Questionnaire-based child maltreatment profiles to predict efficacy of the Cognitive Behavioral Analysis System of Psychotherapy versus non-specific psychotherapy in adults with early-onset chronic depression: cluster analysis of data from a randomised controlled trial

BACKGROUND

Child maltreatment is a broadly confirmed risk factor for mental and physical illness. Some psychological treatments specifically target mental health conditions associated with child maltreatment. For example, the Cognitive Behavioral Analysis System of Psychotherapy (CBASP) focuses on maladaptive interpersonal behaviours in chronic depression. However, how the assessment of child maltreatment could inform personalised treatment is unclear. We used data from a previously published clinical trial to investigate whether a pre-established child maltreatment clustering approach predicts differential outcomes after CBASP versus non-specific supportive psychotherapy in patients with early-onset chronic depression.

METHODS

We did a cluster analysis of data from a previous randomised controlled trial of unmedicated adult outpatients with early-onset chronic depression who were treated at eight university clinics and psychological institutes in Germany with 32 sessions of CBASP or non-specific supportive psychotherapy. Participants were eligible for the original trial if they were aged 18-65 years; had major depressive disorder (MDD) with an early onset and duration of at least 2 years, current MDD superimposed on a pre-existing dysthymic disorder, or recurrent MDD with incomplete remission between episodes as defined by DSM-IV; and had a score of at least 20 points on the 24-item Hamilton Rating Scale for Depression (HRSD-24). Participants were included in the current study if they had completed the short form of the Childhood Trauma Questionnaire (CTQ) at trial baseline. We used an agglomerative hierarchical clustering approach to derive child maltreatment clusters from individual patterns across the five domains of the CTQ. We used linear mixed models to investigate whether clustering could predict differential clinical outcomes (change in symptom severity on the HRSD-24) up to 2 years after treatment onset. People with lived experience were involved in the current study.

FINDINGS

253 patients (129 [51%] treated with CBASP and 124 [49%] with supportive psychotherapy) had complete CTQ records and were included in the analysis. 169 (67%) participants were women, 84 (33%) were men, and the mean age was 45·9 years (SD 11·7). We identified seven child maltreatment clusters and found significant differences in treatment effects of CBASP and supportive psychotherapy between the clusters (F(6,948·76)=2·47; p=0·023); differences were maintained over the 2-year follow-up. CBASP was superior in distinct clusters of co-occurring child maltreatment: predominant emotional neglect (change in β -6·02 [95% CI -11·9 to -0·13]; Cohen's d=-0·98 [95% CI -1·94 to -0·02]; p=0·045), predominant emotional neglect and abuse (-6·39 [-10·22 to -2·56]; -1·04 [-1·67 to -0·42]; p=0·0011), and emotional neglect and emotional and physical abuse (-9·41 [-15·91 to -2·91]; -1·54 [-2·6 to -0·47]; p=0·0046).

INTERPRETATION

CTQ-based cluster analysis can facilitate identification of patients with early-onset chronic depression who would specifically benefit from CBASP. Child maltreatment clusters could be implemented in clinical assessments and serve to develop and personalise trauma-informed care in mental health.

FUNDING

The German Research Foundation and the German Federal Ministry of Education and Research.

Responsible development of clinical speech AI: Bridging the gap between clinical research and technology

This perspective article explores the challenges and potential of using speech as a biomarker in clinical settings, particularly when constrained by the small clinical datasets typically available in such contexts. We contend that by integrating insights from speech science and clinical research, we can reduce sample complexity in clinical speech AI models with the potential to decrease timelines to translation. Most existing models are based on high-dimensional feature representations trained with limited sample sizes and often do not leverage insights from speech science and clinical research. This approach can lead to overfitting, where the models perform exceptionally well on training data but fail to generalize to new, unseen data. Additionally, without incorporating theoretical knowledge, these models may lack interpretability and robustness, making them challenging to troubleshoot or improve post-deployment. We propose a framework for organizing health conditions based on their impact on speech and promote the use of speech analytics in diverse clinical contexts beyond cross-sectional classification. For high-stakes clinical use cases, we advocate for a focus on explainable and individually-validated measures and stress the importance of rigorous validation frameworks and ethical considerations for responsible deployment. Bridging the gap between AI research and clinical speech research presents new opportunities for more efficient translation of speech-based AI tools and advancement of scientific discoveries in this interdisciplinary space, particularly if limited to small or retrospective datasets.

Current best practices and future opportunities for reproducible findings using large-scale neuroimaging in psychiatry

Research into the brain basis of psychopathology is challenging due to the heterogeneity of psychiatric disorders, extensive comorbidities, underdiagnosis or overdiagnosis, multifaceted interactions with genetics and life experiences, and the highly multivariate nature of neural correlates. Therefore, increasingly larger datasets that measure more variables in larger cohorts are needed to gain insights. In this review, we present current "best practice" approaches for using existing databases, collecting and sharing new repositories for big data analyses, and future directions for big data in neuroimaging and psychiatry with an emphasis on contributing to collaborative efforts and the challenges of multi-study data analysis.

Psychiatry and psychotherapy: the science of sitting with uncertainty

OBJECTIVE

This perspective piece aims to delve into the challenges and possibilities arising from uncertainty in psychiatry and psychotherapy.

METHOD

This is a perspective piece.

RESULTS

Historical considerations, polarised conceptual frameworks, interacting systems, limited randomised controlled research, and varying practice approaches, coalesce to form an exoskeleton of ambiguity and uncertainty in psychiatry and psychotherapy that seems almost impenetrable. Yet it is these very things that allow psychiatry to challenge accepted norms, avoid complacency, and shed its skin.

CONCLUSION

Ambiguity and uncertainty in the setting of partial knowledge presents psychiatry with a challenge, yes; but it also enables psychiatry to be innovative in its approach, and to address the complexities of the human condition in a way that is unique and unparalleled within the field of medicine.

Accuracy and transportability of machine learning models for adolescent suicide prediction with longitudinal clinical records

Machine Learning models trained from real-world data have demonstrated promise in predicting suicide attempts in adolescents. However, their transportability, namely the performance of a model trained on one dataset and applied to different data, is largely unknown, hindering the clinical adoption of these models. Here we developed different machine learning-based suicide prediction models based on real-world data collected in different contexts (inpatient, outpatient, and all encounters) with varying purposes (administrative claims and electronic health records), and compared their cross-data performance. The three datasets used were the All-Payer Claims Database in Connecticut, the Hospital Inpatient Discharge Database in Connecticut, and the Electronic Health Records data provided by the Kansas Health Information Network. We included 285,320 patients among whom we identified 3389 (1.2%) suicide attempters and 66% of the suicide attempters were female. Different machine learning models were evaluated on source datasets where models were trained and then applied to target datasets. More complex models, particularly deep long short-term memory neural network models, did not outperform simpler regularized logistic regression models in terms of both local and transported performance. Transported models exhibited varying performance, showing drops or even improvements compared to their source performance. While they can achieve satisfactory transported performance, they are usually upper-bounded by the best performance of locally developed models, and they can identify additional new cases in target data. Our study uncovers complex transportability patterns and could facilitate the development of suicide prediction models with better performance and generalizability.

Neural activity during inhibitory control predicts suicidal ideation with machine learning

Suicide is a leading cause of death in the US and worldwide. Current strategies for preventing suicide are often focused on the identification and treatment of risk factors, especially suicidal ideation (SI). Hence, developing data-driven biomarkers of SI may be key for suicide prevention and intervention. Prior attempts at biomarker-based prediction models for SI have primarily used expensive neuroimaging technologies, yet clinically scalable and affordable biomarkers remain elusive. Here, we investigated the classification of SI using machine learning (ML) on a dataset of 76 subjects with and without SI(+/-) (n = 38 each), who completed a neuro-cognitive assessment session synchronized with electroencephalography (EEG). SI+/- groups were matched for age, sex, and mental health symptoms of depression and anxiety. EEG was recorded at rest and while subjects engaged in four cognitive tasks of inhibitory control, interference processing, working memory, and emotion bias. We parsed EEG signals in physiologically relevant theta (4-8 Hz), alpha (8-13 Hz), and beta (13-30 Hz) frequencies and performed cortical source imaging on the neural signals. These data served as SI predictors in ML models. The best ML model was obtained for beta band power during the inhibitory control (IC) task, demonstrating high sensitivity (89%), specificity (98%). Shapley explainer plots further showed top neural predictors as feedback-related power in the visual and posterior default mode networks and response-related power in the ventral attention, fronto-parietal, and sensory-motor networks. We further tested the external validity of the model in an independent clinically depressed sample (n = 35, 12 SI+) that engaged in an adaptive test version of the IC task, demonstrating 50% sensitivity and 61% specificity in this sample. Overall, the study suggests a promising, scalable EEG-based biomarker approach to predict SI that may serve as a target for risk identification and intervention.

A generalizable data-driven model of atrophy heterogeneity and progression in memory clinic settings

Memory clinic patients are a heterogeneous population representing various aetiologies of pathological ageing. It is not known whether divergent spatiotemporal progression patterns of brain atrophy, as previously described in Alzheimer's disease patients, are prevalent and clinically meaningful in this group of older adults. To uncover distinct atrophy subtypes, we applied the Subtype and Stage Inference (SuStaIn) algorithm to baseline structural MRI data from 813 participants enrolled in the DELCODE cohort (mean ± standard deviation, age = 70.67 ± 6.07 years, 52% females). Participants were cognitively unimpaired (n = 285) or fulfilled diagnostic criteria for subjective cognitive decline (n = 342), mild cognitive impairment (n = 118) or dementia of the Alzheimer's type (n = 68). Atrophy subtypes were compared in baseline demographics, fluid Alzheimer's disease biomarker levels, the Preclinical Alzheimer Cognitive Composite (PACC-5) as well as episodic memory and executive functioning. PACC-5 trajectories over up to 240 weeks were examined. To test whether baseline atrophy subtype and stage predicted clinical trajectories before manifest cognitive impairment, we analysed PACC-5 trajectories and mild cognitive impairment conversion rates of cognitively unimpaired participants and those with subjective cognitive decline. Limbic-predominant and hippocampal-sparing atrophy subtypes were identified. Limbic-predominant atrophy initially affected the medial temporal lobes, followed by further temporal regions and, finally, the remaining cortical regions. At baseline, this subtype was related to older age, more pathological Alzheimer's disease biomarker levels, APOE ε4 carriership and an amnestic cognitive impairment. Hippocampal-sparing atrophy initially occurred outside the temporal lobe, with the medial temporal lobe spared up to advanced atrophy stages. This atrophy pattern also affected individuals with positive Alzheimer's disease biomarkers and was associated with more generalized cognitive impairment. Limbic-predominant atrophy, in all participants and in only unimpaired participants, was linked to more negative longitudinal PACC-5 slopes than observed in participants without or with hippocampal-sparing atrophy and increased the risk of mild cognitive impairment conversion. SuStaIn modelling was repeated in a sample from the Swedish BioFINDER-2 cohort. Highly similar atrophy progression patterns and associated cognitive profiles were identified. Cross-cohort model generalizability, at both the subject and the group level, was excellent, indicating reliable performance in previously unseen data. The proposed model is a promising tool for capturing heterogeneity among older adults at early at-risk states for Alzheimer's disease in applied settings. The implementation of atrophy subtype- and stage-specific end points might increase the statistical power of pharmacological trials targeting early Alzheimer's disease.

Prospects for artificial intelligence-enhanced electrocardiogram as a unified screening tool for cardiac and non-cardiac conditions: an explorative study in emergency care

Aims

Current deep learning algorithms for automatic ECG analysis have shown notable accuracy but are typically narrowly focused on singular diagnostic conditions. This exploratory study aims to investigate the capability of a single deep learning model to predict a diverse range of both cardiac and non-cardiac discharge diagnoses based on a single ECG collected in the emergency department.

Methods and results

In this study, we assess the performance of a model trained to predict a broad spectrum of diagnoses. We find that the model can reliably predict 253 ICD codes (81 cardiac and 172 non-cardiac) in the sense of exceeding an AUROC score of 0.8 in a statistically significant manner.

Conclusion

The model demonstrates proficiency in handling a wide array of cardiac and non-cardiac diagnostic scenarios, indicating its potential as a comprehensive screening tool for diverse medical encounters.

Quality over quantity: powering neuroimaging samples in psychiatry

Neuroimaging has been widely adopted in psychiatric research, with hopes that these non-invasive methods will provide important clues to the underpinnings and prediction of various mental health symptoms and outcomes. However, the translational impact of neuroimaging has not yet reached its promise, despite the plethora of computational methods, tools, and datasets at our disposal. Some have lamented that too many psychiatric neuroimaging studies have been underpowered with respect to sample size. In this review, we encourage this discourse to shift from a focus on sheer increases in sample size to more thoughtful choices surrounding experimental study designs. We propose considerations at multiple decision points throughout the study design, data modeling and analysis process that may help researchers working in psychiatric neuroimaging boost power for their research questions of interest without necessarily increasing sample size. We also provide suggestions for leveraging multiple datasets to inform each other and strengthen our confidence in the generalization of findings to both population-level and clinical samples. Through a greater emphasis on improving the quality of brain-based and clinical measures rather than merely quantity, meaningful and potentially translational clinical associations with neuroimaging measures can be achieved with more modest sample sizes in psychiatry.

Removing the physician from the equation: Patient-controlled, home-based therapeutic drug self-monitoring of tacrolimus

The dosing of tacrolimus, which forms the backbone of immunosuppressive therapy after kidney transplantation, is complex. This is due to its variable pharmacokinetics (both between and within individual patients), narrow therapeutic index, and the severe consequences of over- and underexposure, which may cause toxicity and rejection, respectively. Tacrolimus is, therefore, routinely dosed by means of therapeutic drug monitoring (TDM). TDM is performed for as long as the transplant functions and frequent and often lifelong sampling is therefore the rule. This puts a significant burden on patients and transplant professionals and is associated with high healthcare-associated costs. Furthermore, by its very nature, TDM is reactive and has no predictive power. Finally, the current practice of TDM does not foresee in an active role for patients themselves. Rather, the physician or pharmacist prescribes the next tacrolimus dose after obtaining the concentration measurement test results. In this article, we propose a strategy of patient-controlled, home-based, self-TDM of the immunosuppressant tacrolimus after transplantation. We argue that with the combined use of population tacrolimus pharmacokinetic models, home-based sampling by means of dried blood spotting and implementation of telemedicine, this may become a feasible approach in the near future.

[Data-driven intensive care: a lack of comprehensive datasets]

Intensive care units provide a data-rich environment with the potential to generate datasets in the realm of big data, which could be utilized to train powerful machine learning (ML) models. However, the currently available datasets are too small and exhibit too little diversity due to their limitation to individual hospitals. This lack of extensive and varied datasets is a primary reason for the limited generalizability and resulting low clinical utility of current ML models. Often, these models are based on data from single centers and suffer from poor external validity. There is an urgent need for the development of large-scale, multicentric, and multinational datasets. Ensuring data protection and minimizing re-identification risks pose central challenges in this process. The "Amsterdam University Medical Center database (AmsterdamUMCdb)" and the "Salzburg Intensive Care database (SICdb)" demonstrate that open access datasets are possible in Europe while complying with the data protection regulations of the General Data Protection Regulation (GDPR). Another challenge in building intensive care datasets is the absence of semantic definitions in the source data and the heterogeneity of data formats. Establishing binding industry standards for the semantic definition is crucial to ensure seamless semantic interoperability between datasets.

Personalized Machine Learning-Based Prediction of Wellbeing and Empathy in Healthcare Professionals

Healthcare professionals are known to suffer from workplace stress and burnout, which can negatively affect their empathy for patients and quality of care. While existing research has identified factors associated with wellbeing and empathy in healthcare professionals, these efforts are typically focused on the group level, ignoring potentially important individual differences and implications for individualized intervention approaches. In the current study, we implemented N-of-1 personalized machine learning (PML) to predict wellbeing and empathy in healthcare professionals at the individual level, leveraging ecological momentary assessments (EMAs) and smartwatch wearable data. A total of 47 mood and lifestyle feature variables (relating to sleep, diet, exercise, and social connections) were collected daily for up to three months followed by applying eight supervised machine learning (ML) models in a PML pipeline to predict wellbeing and empathy separately. Predictive insight into the model architecture was obtained using Shapley statistics for each of the best-fit personalized models, ranking the importance of each feature for each participant. The best-fit model and top features varied across participants, with anxious mood (13/19) and depressed mood (10/19) being the top predictors in most models. Social connection was a top predictor for wellbeing in 9/12 participants but not for empathy models (1/7). Additionally, empathy and wellbeing were the top predictors of each other in 64% of cases. These findings highlight shared and individual features of wellbeing and empathy in healthcare professionals and suggest that a one-size-fits-all approach to addressing modifiable factors to improve wellbeing and empathy will likely be suboptimal. In the future, such personalized models may serve as actionable insights for healthcare professionals that lead to increased wellness and quality of patient care.

Automated mood disorder symptoms monitoring from multivariate time-series sensory data: getting the full picture beyond a single number

Mood disorders (MDs) are among the leading causes of disease burden worldwide. Limited specialized care availability remains a major bottleneck thus hindering pre-emptive interventions. MDs manifest with changes in mood, sleep, and motor activity, observable in ecological physiological recordings thanks to recent advances in wearable technology. Therefore, near-continuous and passive collection of physiological data from wearables in daily life, analyzable with machine learning (ML), could mitigate this problem, bringing MDs monitoring outside the clinician's office. Previous works predict a single label, either the disease state or a psychometric scale total score. However, clinical practice suggests that the same label may underlie different symptom profiles, requiring specific treatments. Here we bridge this gap by proposing a new task: inferring all items in HDRS and YMRS, the two most widely used standardized scales for assessing MDs symptoms, using physiological data from wearables. To that end, we develop a deep learning pipeline to score the symptoms of a large cohort of MD patients and show that agreement between predictions and assessments by an expert clinician is clinically significant (quadratic Cohen's κ and macro-average F1 score both of 0.609). While doing so, we investigate several solutions to the ML challenges associated with this task, including multi-task learning, class imbalance, ordinal target variables, and subject-invariant representations. Lastly, we illustrate the importance of testing on out-of-distribution samples.

Evaluating generalizability of artificial intelligence models for molecular datasets

Deep learning has made rapid advances in modeling molecular sequencing data. Despite achieving high performance on benchmarks, it remains unclear to what extent deep learning models learn general principles and generalize to previously unseen sequences. Benchmarks traditionally interrogate model generalizability by generating metadata based (MB) or sequence-similarity based (SB) train and test splits of input data before assessing model performance. Here, we show that this approach mischaracterizes model generalizability by failing to consider the full spectrum of cross-split overlap, i.e., similarity between train and test splits. We introduce Spectra, a spectral framework for comprehensive model evaluation. For a given model and input data, Spectra plots model performance as a function of decreasing cross-split overlap and reports the area under this curve as a measure of generalizability. We apply Spectra to 18 sequencing datasets with associated phenotypes ranging from antibiotic resistance in tuberculosis to protein-ligand binding to evaluate the generalizability of 19 state-of-the-art deep learning models, including large language models, graph neural networks, diffusion models, and convolutional neural networks. We show that SB and MB splits provide an incomplete assessment of model generalizability. With Spectra, we find as cross-split overlap decreases, deep learning models consistently exhibit a reduction in performance in a task- and model-dependent manner. Although no model consistently achieved the highest performance across all tasks, we show that deep learning models can generalize to previously unseen sequences on specific tasks. Spectra paves the way toward a better understanding of how foundation models generalize in biology.

Machine Learning and Bayesian Network Analyses Identifies Psychiatric Disorders and Symptom Associations with Insomnia in a national sample of 31,285 Treatment-Seeking College Students

Background

A better understanding of the structure of relations among insomnia and anxiety, mood, eating, and alcohol-use disorders is needed, given its prevalence among young adults. Supervised machine learning provides the ability to evaluate the discriminative accuracy of psychiatric disorders associated with insomnia. Combined with Bayesian network analysis, the directionality between symptoms and their associations may be illuminated.

Methods

The current exploratory analyses utilized a national sample of college students across 26 U.S. colleges and universities collected during population-level screening before entering a randomized controlled trial. Firstly, an elastic net regularization model was trained to predict, via repeated 10-fold cross-validation, which psychiatric disorders were associated with insomnia severity. Seven disorders were included: major depressive disorder, generalized anxiety disorder, social anxiety disorder, panic disorder, post-traumatic stress disorder, anorexia nervosa, and alcohol use disorder. Secondly, using a Bayesian network approach, completed partially directed acyclic graphs (CPDAG) built on training and holdout samples were computed via a Bayesian hill-climbing algorithm to determine symptom-level interactions of disorders most associated with insomnia [based on SHAP (SHapley Additive exPlanations) values)] and were evaluated for stability across networks.

Results

Of 31,285 participants, 20,597 were women (65.8%); mean (standard deviation) age was 22.96 (4.52) years. The elastic net model demonstrated clinical significance in predicting insomnia severity in the training sample [R2 = .449 (.016); RMSE = 5.00 [.081]), with comparable performance in accounting for variance explained in the holdout sample [R2 = .33; RMSE = 5.47). SHAP indicated the presence of any psychiatric disorder was associated with higher insomnia severity, with major depressive disorder demonstrated to be the most associated disorder. CPDAGs showed excellent fit in the holdout sample and suggested that depressed mood, fatigue, and self-esteem were the most important depression symptoms that presupposed insomnia.

Conclusion

These findings offer insights into associations between psychiatric disorders and insomnia among college students and encourage future investigation into the potential direction of causality between insomnia and major depressive disorder.

Trial registration

Trial may be found on the National Institute of Health RePORTER website: Project Number: R01MH115128-05.

Brain-phenotype predictions can survive across diverse real-world data

Recent work suggests that machine learning models predicting psychiatric treatment outcomes based on clinical data may fail when applied to unharmonized samples. Neuroimaging predictive models offer the opportunity to incorporate neurobiological information, which may be more robust to dataset shifts. Yet, among the minority of neuroimaging studies that undertake any form of external validation, there is a notable lack of attention to generalization across dataset-specific idiosyncrasies. Research settings, by design, remove the between-site variations that real-world and, eventually, clinical applications demand. Here, we rigorously test the ability of a range of predictive models to generalize across three diverse, unharmonized samples: the Philadelphia Neurodevelopmental Cohort (n=1291), the Healthy Brain Network (n=1110), and the Human Connectome Project in Development (n=428). These datasets have high inter-dataset heterogeneity, encompassing substantial variations in age distribution, sex, racial and ethnic minority representation, recruitment geography, clinical symptom burdens, fMRI tasks, sequences, and behavioral measures. We demonstrate that reproducible and generalizable brain-behavior associations can be realized across diverse dataset features with sample sizes in the hundreds. Results indicate the potential of functional connectivity-based predictive models to be robust despite substantial inter-dataset variability. Notably, for the HCPD and HBN datasets, the best predictions were not from training and testing in the same dataset (i.e., cross-validation) but across datasets. This result suggests that training on diverse data may improve prediction in specific cases. Overall, this work provides a critical foundation for future work evaluating the generalizability of neuroimaging predictive models in real-world scenarios and clinical settings.