Combining Clinical With Cognitive or Magnetic Resonance Imaging Data for Predicting Transition to Psychosis in Ultra High-Risk Patients: Data From the PACE 400 Cohort

BACKGROUND

Multimodal modeling that combines biological and clinical data shows promise in predicting transition to psychosis in individuals who are at ultra-high risk. Individuals who transition to psychosis are known to have deficits at baseline in cognitive function and reductions in gray matter volume in multiple brain regions identified by magnetic resonance imaging.

METHODS

In this study, we used Cox proportional hazards regression models to assess the additive predictive value of each modality-cognition, cortical structure information, and the neuroanatomical measure of brain age gap-to a previously developed clinical model using functioning and duration of symptoms prior to service entry as predictors in the Personal Assessment and Crisis Evaluation (PACE) 400 cohort. The PACE 400 study is a well-characterized cohort of Australian youths who were identified as ultra-high risk of transitioning to psychosis using the Comprehensive Assessment of At Risk Mental States (CAARMS) and followed for up to 18 years; it contains clinical data (from N = 416 participants), cognitive data (n = 213), and magnetic resonance imaging cortical parameters extracted using FreeSurfer (n = 231).

RESULTS

The results showed that neuroimaging, brain age gap, and cognition added marginal predictive information to the previously developed clinical model (fraction of new information: neuroimaging 0%-12%, brain age gap 7%, cognition 0%-16%).

CONCLUSIONS

In summary, adding a second modality to a clinical risk model predicting the onset of a psychotic disorder in the PACE 400 cohort showed little improvement in the fit of the model for long-term prediction of transition to psychosis.

Attention-based multimodal fusion with contrast for robust clinical prediction in the face of missing modalities

OBJECTIVE

With the increasing amount and growing variety of healthcare data, multimodal machine learning supporting integrated modeling of structured and unstructured data is an increasingly important tool for clinical machine learning tasks. However, it is non-trivial to manage the differences in dimensionality, volume, and temporal characteristics of data modalities in the context of a shared target task. Furthermore, patients can have substantial variations in the availability of data, while existing multimodal modeling methods typically assume data completeness and lack a mechanism to handle missing modalities.

METHODS

We propose a Transformer-based fusion model with modality-specific tokens that summarize the corresponding modalities to achieve effective cross-modal interaction accommodating missing modalities in the clinical context. The model is further refined by inter-modal, inter-sample contrastive learning to improve the representations for better predictive performance. We denote the model as Attention-based cRoss-MOdal fUsion with contRast (ARMOUR). We evaluate ARMOUR using two input modalities (structured measurements and unstructured text), six clinical prediction tasks, and two evaluation regimes, either including or excluding samples with missing modalities.

RESULTS

Our model shows improved performances over unimodal or multimodal baselines in both evaluation regimes, including or excluding patients with missing modalities in the input. The contrastive learning improves the representation power and is shown to be essential for better results. The simple setup of modality-specific tokens enables ARMOUR to handle patients with missing modalities and allows comparison with existing unimodal benchmark results.

CONCLUSION

We propose a multimodal model for robust clinical prediction to achieve improved performance while accommodating patients with missing modalities. This work could inspire future research to study the effective incorporation of multiple, more complex modalities of clinical data into a single model.

Revisiting correlation-based functional connectivity and its relationship with structural connectivity

Patterns of brain structural connectivity (SC) and functional connectivity (FC) are known to be related. In SC-FC comparisons, FC has classically been evaluated from correlations between functional time series, and more recently from partial correlations or their unnormalized version encoded in the precision matrix. The latter FC metrics yield more meaningful comparisons to SC because they capture ‘direct’ statistical dependencies, that is, discarding the effects of mediators, but their use has been limited because of estimation issues. With the rise of high-quality and large neuroimaging datasets, we revisit the relevance of different FC metrics in the context of SC-FC comparisons. Using data from 100 unrelated Human Connectome Project subjects, we first explore the amount of functional data required to reliably estimate various FC metrics. We find that precision-based FC yields a better match to SC than correlation-based FC when using 5 minutes of functional data or more. Finally, using a linear model linking SC and FC, we show that the SC-FC match can be used to further interrogate various aspects of brain structure and function such as the timescales of functional dynamics in different resting-state networks or the intensity of anatomical self-connections.