Identifying depression-related topics in smartphone-collected free-response speech recordings using an automatic speech recognition system and a deep learning topic model

BACKGROUND

Prior research has associated spoken language use with depression, yet studies often involve small or non-clinical samples and face challenges in the manual transcription of speech. This paper aimed to automatically identify depression-related topics in speech recordings collected from clinical samples.

METHODS

The data included 3919 English free-response speech recordings collected via smartphones from 265 participants with a depression history. We transcribed speech recordings via automatic speech recognition (Whisper tool, OpenAI) and identified principal topics from transcriptions using a deep learning topic model (BERTopic). To identify depression risk topics and understand the context, we compared participants' depression severity and behavioral (extracted from wearable devices) and linguistic (extracted from transcribed texts) characteristics across identified topics.

RESULTS

From the 29 topics identified, we identified 6 risk topics for depression: 'No Expectations', 'Sleep', 'Mental Therapy', 'Haircut', 'Studying', and 'Coursework'. Participants mentioning depression risk topics exhibited higher sleep variability, later sleep onset, and fewer daily steps and used fewer words, more negative language, and fewer leisure-related words in their speech recordings.

LIMITATIONS

Our findings were derived from a depressed cohort with a specific speech task, potentially limiting the generalizability to non-clinical populations or other speech tasks. Additionally, some topics had small sample sizes, necessitating further validation in larger datasets.

CONCLUSION

This study demonstrates that specific speech topics can indicate depression severity. The employed data-driven workflow provides a practical approach for analyzing large-scale speech data collected from real-world settings.

Towards a scalable approach to assess speech organization across the psychosis-spectrum -online assessment in conjunction with automated transcription and extraction of speech measures

Automatically extracted measures of speech constitute a promising marker of psychosis as disorganized speech is associated with psychotic symptoms and predictive of psychosis-onset. The potential of speech markers is, however, hampered by (i) lengthy assessments in laboratory settings and (ii) manual transcriptions. We investigated whether a short, scalable data collection (online) and processing (automated transcription) procedure would provide data of sufficient quality to extract previously validated speech measures. To evaluate the fit of our approach for purpose, we assessed speech in relation to psychotic-like experiences in the general population. Participants completed an 8-minute-long speech task online. Sample 1 included measures of psychometric schizotypy and delusional ideation (N = 446). Sample 2 included a low and high psychometric schizotypy group (N = 144). Recordings were transcribed both automatically and manually, and connectivity, semantic, and syntactic speech measures were extracted for both types of transcripts. 73%/86% participants in sample 1/2 completed the experiment. Nineteen out of 25 speech measures were strongly (r > 0.7) and significantly correlated between automated and manual transcripts in both samples. Amongst the 14 connectivity measures, 11 showed a significant relationship with delusional ideation. For the semantic and syntactic measures, On Topic score and the Frequency of personal pronouns were negatively correlated with both schizotypy and delusional ideation. Combined with demographic information, the speech markers could explain 11-14% of the variation of delusional ideation and schizotypy in Sample 1 and could discriminate between high-low schizotypy with high accuracy (0.72-0.70, AUC = 0.78-0.79) in Sample 2. The moderate to high retention rate, strong correlation of speech measures across manual and automated transcripts and sensitivity to psychotic-like experiences provides initial evidence that online collected speech in combination with automatic transcription is a feasible approach to increase accessibility and scalability of speech-based assessment of psychosis.

Automatic detection of expressed emotion from Five-Minute Speech Samples: Challenges and opportunities

Research into clinical applications of speech-based emotion recognition (SER) technologies has been steadily increasing over the past few years. One such potential application is the automatic recognition of expressed emotion (EE) components within family environments. The identification of EE is highly important as they have been linked with a range of adverse life events. Manual coding of these events requires time-consuming specialist training, amplifying the need for automated approaches. Herein we describe an automated machine learning approach for determining the degree of warmth, a key component of EE, from acoustic and text natural language features. Our dataset of 52 recorded interviews is taken from recordings, collected over 20 years ago, from a nationally representative birth cohort of British twin children, and was manually coded for EE by two researchers (inter-rater reliability 0.84-0.90). We demonstrate that the degree of warmth can be predicted with an F1-score of 64.7% despite working with audio recordings of highly variable quality. Our highly promising results suggest that machine learning may be able to assist in the coding of EE in the near future.

Automated analysis of speech as a marker of sub-clinical psychotic experiences

Automated speech analysis techniques, when combined with artificial intelligence and machine learning, show potential in capturing and predicting a wide range of psychosis symptoms, garnering attention from researchers. These techniques hold promise in predicting the transition to clinical psychosis from at-risk states, as well as relapse or treatment response in individuals with clinical-level psychosis. However, challenges in scientific validation hinder the translation of these techniques into practical applications. Although sub-clinical research could aid to tackle most of these challenges, there have been only few studies conducted in speech and psychosis research in non-clinical populations. This work aims to facilitate this work by summarizing automated speech analytical concepts and the intersection of this field with psychosis research. We review psychosis continuum and sub-clinical psychotic experiences, and the benefits of researching them. Then, we discuss the connection between speech and psychotic symptoms. Thirdly, we overview current and state-of-the art approaches to the automated analysis of speech both in terms of language use (text-based analysis) and vocal features (audio-based analysis). Then, we review techniques applied in subclinical population and findings in these samples. Finally, we discuss research challenges in the field, recommend future research endeavors and outline how research in subclinical populations can tackle the listed challenges.

A wearable device perspective on the standard definitions of disability progression in multiple sclerosis

INTRODUCTION

Multiple sclerosis (MS) is a leading cause of disability among young adults, but standard clinical scales may not accurately detect subtle changes in disability occurring between visits. This study aims to explore whether wearable device data provides more granular and objective measures of disability progression in MS.

METHODS

Remote Assessment of Disease and Relapse in Central Nervous System Disorders (RADAR-CNS) is a longitudinal multicenter observational study in which 400 MS patients have been recruited since June 2018 and prospectively followed up for 24 months. Monitoring of patients included standard clinical visits with assessment of disability through use of the Expanded Disability Status Scale (EDSS), 6-minute walking test (6MWT) and timed 25-foot walk (T25FW), as well as remote monitoring through the use of a Fitbit.

RESULTS

Among the 306 patients who completed the study (mean age, 45.6 years; females 67%), confirmed disability progression defined by the EDSS was observed in 74 patients, who had approximately 1392 fewer daily steps than patients without disability progression. However, the decrease in the number of steps experienced over time by patients with EDSS progression and stable patients was not significantly different. Similar results were obtained with disability progression defined by the 6MWT and the T25FW.

CONCLUSION

The use of continuous activity monitoring holds great promise as a sensitive and ecologically valid measure of disability progression in MS.

Multilingual markers of depression in remotely collected speech samples: A preliminary analysis

BACKGROUND

Speech contains neuromuscular, physiological and cognitive components, and so is a potential biomarker of mental disorders. Previous studies indicate that speaking rate and pausing are associated with major depressive disorder (MDD). However, results are inconclusive as many studies are small and underpowered and do not include clinical samples. These studies have also been unilingual and use speech collected in controlled settings. If speech markers are to help understand the onset and progress of MDD, we need to uncover markers that are robust to language and establish the strength of associations in real-world data.

METHODS

We collected speech data in 585 participants with a history of MDD in the United Kingdom, Spain, and Netherlands as part of the RADAR-MDD study. Participants recorded their speech via smartphones every two weeks for 18 months. Linear mixed models were used to estimate the strength of specific markers of depression from a set of 28 speech features.

RESULTS

Increased depressive symptoms were associated with speech rate, articulation rate and intensity of speech elicited from a scripted task. These features had consistently stronger effect sizes than pauses.

LIMITATIONS

Our findings are derived at the cohort level so may have limited impact on identifying intra-individual speech changes associated with changes in symptom severity. The analysis of features averaged over the entire recording may have underestimated the importance of some features.

CONCLUSIONS

Participants with more severe depressive symptoms spoke more slowly and quietly. Our findings are from a real-world, multilingual, clinical dataset so represent a step-change in the usefulness of speech as a digital phenotype of MDD.

Online speech assessment of the psychotic spectrum: Exploring the relationship between overlapping acoustic markers of schizotypy, depression and anxiety

BACKGROUND

Remote assessment of acoustic alterations in speech holds promise to increase scalability and validity in research across the psychosis spectrum. A feasible first step in establishing a procedure for online assessments is to assess acoustic alterations in psychometric schizotypy. However, to date, the complex relationship between alterations in speech related to schizotypy and those related to comorbid conditions such as symptoms of depression and anxiety has not been investigated. This study tested whether (1) depression, generalized anxiety and high psychometric schizotypy have similar voice characteristics, (2) which acoustic markers of online collected speech are the strongest predictors of psychometric schizotypy, (3) whether including generalized anxiety and depression symptoms in the model can improve the prediction of schizotypy.

METHODS

We collected cross-sectional, online-recorded speech data from 441 participants, assessing demographics, symptoms of depression, generalized anxiety and psychometric schizotypy.

RESULTS

Speech samples collected online could predict psychometric schizotypy, depression, and anxiety symptoms with weak to moderate predictive power, and with moderate and good predictive power when basic demographic variables were added to the models. Most influential features of these models largely overlapped. The predictive power of speech marker-based models of schizotypy significantly improved after including symptom scores of depression and generalized anxiety in the models (from R2 = 0.296 to R2 = 0. 436).

CONCLUSIONS

Acoustic features of online collected speech are predictive of psychometric schizotypy as well as generalized anxiety and depression symptoms. The acoustic characteristics of schizotypy, depression and anxiety symptoms significantly overlap. Speech models that are designed to predict schizotypy or symptoms of the schizophrenia spectrum might therefore benefit from controlling for symptoms of depression and anxiety.

Challenges in Using mHealth Data From Smartphones and Wearable Devices to Predict Depression Symptom Severity: Retrospective Analysis

BACKGROUND

Major depressive disorder (MDD) affects millions of people worldwide, but timely treatment is not often received owing in part to inaccurate subjective recall and variability in the symptom course. Objective and frequent MDD monitoring can improve subjective recall and help to guide treatment selection. Attempts have been made, with varying degrees of success, to explore the relationship between the measures of depression and passive digital phenotypes (features) extracted from smartphones and wearables devices to remotely and continuously monitor changes in symptomatology. However, a number of challenges exist for the analysis of these data. These include maintaining participant engagement over extended time periods and therefore understanding what constitutes an acceptable threshold of missing data; distinguishing between the cross-sectional and longitudinal relationships for different features to determine their utility in tracking within-individual longitudinal variation or screening individuals at high risk; and understanding the heterogeneity with which depression manifests itself in behavioral patterns quantified by the passive features.

OBJECTIVE

We aimed to address these 3 challenges to inform future work in stratified analyses.

METHODS

Using smartphone and wearable data collected from 479 participants with MDD, we extracted 21 features capturing mobility, sleep, and smartphone use. We investigated the impact of the number of days of available data on feature quality using the intraclass correlation coefficient and Bland-Altman analysis. We then examined the nature of the correlation between the 8-item Patient Health Questionnaire (PHQ-8) depression scale (measured every 14 days) and the features using the individual-mean correlation, repeated measures correlation, and linear mixed effects model. Furthermore, we stratified the participants based on their behavioral difference, quantified by the features, between periods of high (depression) and low (no depression) PHQ-8 scores using the Gaussian mixture model.

RESULTS

We demonstrated that at least 8 (range 2-12) days were needed for reliable calculation of most of the features in the 14-day time window. We observed that features such as sleep onset time correlated better with PHQ-8 scores cross-sectionally than longitudinally, whereas features such as wakefulness after sleep onset correlated well with PHQ-8 longitudinally but worse cross-sectionally. Finally, we found that participants could be separated into 3 distinct clusters according to their behavioral difference between periods of depression and periods of no depression.

CONCLUSIONS

This work contributes to our understanding of how these mobile health-derived features are associated with depression symptom severity to inform future work in stratified analyses.

Automatic Assessment of the 2-Minute Walk Distance for Remote Monitoring of People with Multiple Sclerosis

The aim of this study was to investigate the feasibility of automatically assessing the 2-Minute Walk Distance (2MWD) for monitoring people with multiple sclerosis (pwMS). For 154 pwMS, MS-related clinical outcomes as well as the 2MWDs as evaluated by clinicians and derived from accelerometer data were collected from a total of 323 periodic clinical visits. Accelerometer data from a wearable device during 100 home-based 2MWD assessments were also acquired. The error in estimating the 2MWD was validated for walk tests performed at hospital, and then the correlation (r) between clinical outcomes and home-based 2MWD assessments was evaluated. Robust performance in estimating the 2MWD from the wearable device was obtained, yielding an error of less than 10% in about two-thirds of clinical visits. Correlation analysis showed that there is a strong association between the actual and the estimated 2MWD obtained either at hospital (r = 0.71) or at home (r = 0.58). Furthermore, the estimated 2MWD exhibits moderate-to-strong correlation with various MS-related clinical outcomes, including disability and fatigue severity scores. Automatic assessment of the 2MWD in pwMS is feasible with the usage of a consumer-friendly wearable device in clinical and non-clinical settings. Wearable devices can also enhance the assessment of MS-related clinical outcomes.

Digital endpoints in clinical trials of Alzheimer's disease and other neurodegenerative diseases: challenges and opportunities

Alzheimer's disease (AD) and other neurodegenerative diseases such as Parkinson's disease (PD) and Huntington's disease (HD) are associated with progressive cognitive, motor, affective and consequently functional decline considerably affecting Activities of Daily Living (ADL) and quality of life. Standard assessments, such as questionnaires and interviews, cognitive testing, and mobility assessments, lack sensitivity, especially in early stages of neurodegenerative diseases and in the disease progression, and have therefore a limited utility as outcome measurements in clinical trials. Major advances in the last decade in digital technologies have opened a window of opportunity to introduce digital endpoints into clinical trials that can reform the assessment and tracking of neurodegenerative symptoms. The Innovative Health Initiative (IMI)-funded projects RADAR-AD (Remote assessment of disease and relapse-Alzheimer's disease), IDEA-FAST (Identifying digital endpoints to assess fatigue, sleep and ADL in neurodegenerative disorders and immune-mediated inflammatory diseases) and Mobilise-D (Connecting digital mobility assessment to clinical outcomes for regulatory and clinical endorsement) aim to identify digital endpoints relevant for neurodegenerative diseases that provide reliable, objective, and sensitive evaluation of disability and health-related quality of life. In this article, we will draw from the findings and experiences of the different IMI projects in discussing (1) the value of remote technologies to assess neurodegenerative diseases; (2) feasibility, acceptability and usability of digital assessments; (3) challenges related to the use of digital tools; (4) public involvement and the implementation of patient advisory boards; (5) regulatory learnings; and (6) the significance of inter-project exchange and data- and algorithm-sharing.

The usability of daytime and night-time heart rate dynamics as digital biomarkers of depression severity

BACKGROUND

Alterations in heart rate (HR) may provide new information about physiological signatures of depression severity. This 2-year study in individuals with a history of recurrent major depressive disorder (MDD) explored the intra-individual variations in HR parameters and their relationship with depression severity.

METHODS

Data from 510 participants (Number of observations of the HR parameters = 6666) were collected from three centres in the Netherlands, Spain, and the UK, as a part of the remote assessment of disease and relapse-MDD study. We analysed the relationship between depression severity, assessed every 2 weeks with the Patient Health Questionnaire-8, with HR parameters in the week before the assessment, such as HR features during all day, resting periods during the day and at night, and activity periods during the day evaluated with a wrist-worn Fitbit device. Linear mixed models were used with random intercepts for participants and countries. Covariates included in the models were age, sex, BMI, smoking and alcohol consumption, antidepressant use and co-morbidities with other medical health conditions.

RESULTS

Decreases in HR variation during resting periods during the day were related with an increased severity of depression both in univariate and multivariate analyses. Mean HR during resting at night was higher in participants with more severe depressive symptoms.

CONCLUSIONS

Our findings demonstrate that alterations in resting HR during all day and night are associated with depression severity. These findings may provide an early warning of worsening depression symptoms which could allow clinicians to take responsive treatment measures promptly.

Long-term participant retention and engagement patterns in an app and wearable-based multinational remote digital depression study

Recent growth in digital technologies has enabled the recruitment and monitoring of large and diverse populations in remote health studies. However, the generalizability of inference drawn from remotely collected health data could be severely impacted by uneven participant engagement and attrition over the course of the study. We report findings on long-term participant retention and engagement patterns in a large multinational observational digital study for depression containing active (surveys) and passive sensor data collected via Android smartphones, and Fitbit devices from 614 participants for up to 2 years. Majority of participants (67.6%) continued to remain engaged in the study after 43 weeks. Unsupervised clustering of participants' study apps and Fitbit usage data showed 3 distinct engagement subgroups for each data stream. We found: (i) the least engaged group had the highest depression severity (4 PHQ8 points higher) across all data streams; (ii) the least engaged group (completed 4 bi-weekly surveys) took significantly longer to respond to survey notifications (3.8 h more) and were 5 years younger compared to the most engaged group (completed 20 bi-weekly surveys); and (iii) a considerable proportion (44.6%) of the participants who stopped completing surveys after 8 weeks continued to share passive Fitbit data for significantly longer (average 42 weeks). Additionally, multivariate survival models showed participants' age, ownership and brand of smartphones, and recruitment sites to be associated with retention in the study. Together these findings could inform the design of future digital health studies to enable equitable and balanced data collection from diverse populations.

The Feasibility of Implementing Remote Measurement Technologies in Psychological Treatment for Depression: Mixed Methods Study on Engagement

BACKGROUND

Remote measurement technologies (RMTs) such as smartphones and wearables can help improve treatment for depression by providing objective, continuous, and ecologically valid insights into mood and behavior. Engagement with RMTs is varied and highly context dependent; however, few studies have investigated their feasibility in the context of treatment.

OBJECTIVE

A mixed methods design was used to evaluate engagement with active and passive data collection via RMT in people with depression undergoing psychotherapy. We evaluated the effects of treatment on 2 different types of engagement: study attrition (engagement with study protocol) and patterns of missing data (engagement with digital devices), which we termed data availability. Qualitative interviews were conducted to help interpret the differences in engagement.

METHODS

A total of 66 people undergoing psychological therapy for depression were followed up for 7 months. Active data were gathered from weekly questionnaires and speech and cognitive tasks, and passive data were gathered from smartphone sensors and a Fitbit (Fitbit Inc) wearable device.

RESULTS

The overall retention rate was 60%. Higher-intensity treatment (χ21=4.6; P=.03) and higher baseline anxiety (t56.28=-2.80, 2-tailed; P=.007) were associated with attrition, but depression severity was not (t50.4=-0.18; P=.86). A trend toward significance was found for the association between longer treatments and increased attrition (U=339.5; P=.05). Data availability was higher for active data than for passive data initially but declined at a sharper rate (90%-30% drop in 7 months). As for passive data, wearable data availability fell from a maximum of 80% to 45% at 7 months but showed higher overall data availability than smartphone-based data, which remained stable at the range of 20%-40% throughout. Missing data were more prevalent among GPS location data, followed by among Bluetooth data, then among accelerometry data. As for active data, speech and cognitive tasks had lower completion rates than clinical questionnaires. The participants in treatment provided less Fitbit data but more active data than those on the waiting list.

CONCLUSIONS

Different data streams showed varied patterns of missing data, despite being gathered from the same device. Longer and more complex treatments and clinical characteristics such as higher baseline anxiety may reduce long-term engagement with RMTs, and different devices may show opposite patterns of missingness during treatment. This has implications for the scalability and uptake of RMTs in health care settings, the generalizability and accuracy of the data collected by these methods, feature construction, and the appropriateness of RMT use in the long term.

The utility of wearable devices in assessing ambulatory impairments of people with multiple sclerosis in free-living conditions

BACKGROUND AND OBJECTIVES

Multiple sclerosis (MS) is a progressive inflammatory and neurodegenerative disease of the central nervous system affecting over 2.5 million people globally. In-clinic six-minute walk test (6MWT) is a widely used objective measure to evaluate the progression of MS. Yet, it has limitations such as the need for a clinical visit and a proper walkway. The widespread use of wearable devices capable of depicting patients' activity profiles has the potential to assess the level of MS-induced disability in free-living conditions.

METHODS

In this work, we extracted 96 features in different temporal granularities (from minute-level to day-level) from wearable data and explored their utility in estimating 6MWT scores in a European (Italy, Spain, and Denmark) MS cohort of 337 participants over an average of 10 months' duration. We combined these features with participants' demographics using three regression models including elastic net, gradient boosted trees and random forest. In addition, we quantified the individual feature's contribution using feature importance in these regression models, linear mixed-effects models, generalized estimating equations, and correlation-based feature selection (CFS).

RESULTS

The results showed promising estimation performance with R² of 0.30, which was derived using random forest after CFS. This model was able to distinguish the participants with low disability from those with high disability. Furthermore, we observed that the minute-level (≤ 8 minutes) step count, particularly those capturing the upper end of the step count distribution, had a stronger association with 6MWT. The use of a walking aid was indicative of ambulatory function measured through 6MWT.

CONCLUSIONS

This study demonstrates the utility of wearables devices in assessing ambulatory impairments in people with MS in free-living conditions and provides a basis for future investigation into the clinical relevance.

Biopsychosocial Response to the COVID-19 Lockdown in People with Major Depressive Disorder and Multiple Sclerosis

BACKGROUND

Changes in lifestyle, finances and work status during COVID-19 lockdowns may have led to biopsychosocial changes in people with pre-existing vulnerabilities such as Major Depressive Disorders (MDDs) and Multiple Sclerosis (MS).

METHODS

Data were collected as a part of the RADAR-CNS (Remote Assessment of Disease and Relapse-Central Nervous System) program. We analyzed the following data from long-term participants in a decentralized multinational study: symptoms of depression, heart rate (HR) during the day and night; social activity; sedentary state, steps and physical activity of varying intensity. Linear mixed-effects regression analyses with repeated measures were fitted to assess the changes among three time periods (pre, during and post-lockdown) across the groups, adjusting for depression severity before the pandemic and gender.

RESULTS

Participants with MDDs (N = 255) and MS (N = 214) were included in the analyses. Overall, depressive symptoms remained stable across the three periods in both groups. A lower mean HR and HR variation were observed between pre and during lockdown during the day for MDDs and during the night for MS. HR variation during rest periods also decreased between pre- and post-lockdown in both clinical conditions. We observed a reduction in physical activity for MDDs and MS upon the introduction of lockdowns. The group with MDDs exhibited a net increase in social interaction via social network apps over the three periods.

CONCLUSIONS

Behavioral responses to the lockdown measured by social activity, physical activity and HR may reflect changes in stress in people with MDDs and MS. Remote technology monitoring might promptly activate an early warning of physical and social alterations in these stressful situations. Future studies must explore how stress does or does not impact depression severity.

303 SCREENING INSTRUMENTS TO PREDICT ADVERSE OUTCOMES FOR UNDIFFERENTIATED OLDER ADULTS ATTENDING THE EMERGENCY DEPARTMENT: RESULTS OF SOAED PROSPECTIVE COHORT STUDY

Background

Frailty screening facilitates the stratification of older adults at most risk of adverse events for urgent assessment and subsequent intervention in the acute or community setting. We assessed the validity of the ISAR (Identification of Seniors at Risk), Rockwood Clinical Frailty Scale (CFS), PRISMA-7 and InterRAI-ED at predicting adverse outcomes at 30 days and six months among older adults presenting to the ED.

Methods

A prospective cohort study of consecutive older adults (≥65 years) who presented to the ED at a University Hospital was conducted. The ISAR, CFS, PRISMA-7 and InterRAI-ED were performed by an experienced ED research nurse. Blinded follow-up telephone interviews were completed at 30 days and six months to assess the incidence of mortality, ED re-attendance, hospital readmission, functional decline and nursing home admission. The sensitivity and specificity of the screening tools were calculated using 2×2 tables.

Results

419 patients were recruited with 49% female and a mean age of 76.9 years (SD 7.15). The prevalence of frailty varied across the screening tools (ISAR, 47% vs InterRAI-ED, 63%). At 30-days, mortality rate was 5.4%, ED re-attendance 16.9%, hospital readmission 13.6%, functional decline 47.1% and nursing home admission 7.3%. Older adults who screened positive for frailty demonstrated an increased risk of all adverse outcomes at 30 days and 6 months, regardless of frailty screening tool administered. All tools had a relatively high sensitivity but low specificity. The ISAR was the only tool which was statistically significant at predicting all outcomes at 30 days.

Conclusion

The ISAR, CFS, PRISMA-7 and InterRAI-ED demonstrated modest validity at predicting adverse outcomes at 30 days and 6 months. We would recommend the implementation of one of these frailty screening tools in Irish EDs to support clinicians in identifying older adults most likely to benefit from specialised geriatric assessment and intervention in the hospital or community setting.

Associations Between Depression Symptom Severity and Daily-Life Gait Characteristics Derived From Long-Term Acceleration Signals in Real-World Settings: Retrospective Analysis

BACKGROUND

Gait is an essential manifestation of depression. However, the gait characteristics of daily walking and their relationships with depression have yet to be fully explored.

OBJECTIVE

The aim of this study was to explore associations between depression symptom severity and daily-life gait characteristics derived from acceleration signals in real-world settings.

METHODS

We used two ambulatory data sets (N=71 and N=215) with acceleration signals collected by wearable devices and mobile phones, respectively. We extracted 12 daily-life gait features to describe the distribution and variance of gait cadence and force over a long-term period. Spearman coefficients and linear mixed-effects models were used to explore the associations between daily-life gait features and depression symptom severity measured by the 15-item Geriatric Depression Scale (GDS-15) and 8-item Patient Health Questionnaire (PHQ-8) self-reported questionnaires. The likelihood-ratio (LR) test was used to test whether daily-life gait features could provide additional information relative to the laboratory gait features.

RESULTS

Higher depression symptom severity was significantly associated with lower gait cadence of high-performance walking (segments with faster walking speed) over a long-term period in both data sets. The linear regression model with long-term daily-life gait features (R²=0.30) fitted depression scores significantly better (LR test P=.001) than the model with only laboratory gait features (R²=0.06).

CONCLUSIONS

This study indicated that the significant links between daily-life walking characteristics and depression symptom severity could be captured by both wearable devices and mobile phones. The daily-life gait patterns could provide additional information for predicting depression symptom severity relative to laboratory walking. These findings may contribute to developing clinical tools to remotely monitor mental health in real-world settings.

Predictors of engagement with remote sensing technologies for symptom measurement in Major Depressive Disorder

BACKGROUND

Remote sensing for the measurement and management of long-term conditions such as Major Depressive Disorder (MDD) is becoming more prevalent. User-engagement is essential to yield any benefits. We tested three hypotheses examining associations between clinical characteristics, perceptions of remote sensing, and objective user engagement metrics.

METHODS

The Remote Assessment of Disease and Relapse - Major Depressive Disorder (RADAR-MDD) study is a multicentre longitudinal observational cohort study in people with recurrent MDD. Participants wore a FitBit and completed app-based assessments every two weeks for a median of 18 months. Multivariable random effects regression models pooling data across timepoints were used to examine associations between variables.

RESULTS

A total of 547 participants (87.8% of the total sample) were included in the current analysis. Higher levels of anxiety were associated with lower levels of perceived technology ease of use; increased functional disability was associated with small differences in perceptions of technology usefulness and usability. Participants who reported higher system ease of use, usefulness, and acceptability subsequently completed more app-based questionnaires and tended to wear their FitBit activity tracker for longer. All effect sizes were small and unlikely to be of practical significance.

LIMITATIONS

Symptoms of depression, anxiety, functional disability, and perceptions of system usability are measured at the same time. These therefore represent cross-sectional associations rather than predictions of future perceptions.

CONCLUSIONS

These findings suggest that perceived usability and actual use of remote measurement technologies in people with MDD are robust across differences in severity of depression, anxiety, and functional impairment.

Codeveloping and Evaluating a Campaign to Reduce Dementia Misconceptions on Twitter: Machine Learning Study

Background

Dementia misconceptions on Twitter can have detrimental or harmful effects. Machine learning (ML) models codeveloped with carers provide a method to identify these and help in evaluating awareness campaigns.

Objective

This study aimed to develop an ML model to distinguish between misconceptions and neutral tweets and to develop, deploy, and evaluate an awareness campaign to tackle dementia misconceptions.

Methods

Taking 1414 tweets rated by carers from our previous work, we built 4 ML models. Using a 5-fold cross-validation, we evaluated them and performed a further blind validation with carers for the best 2 ML models; from this blind validation, we selected the best model overall. We codeveloped an awareness campaign and collected pre-post campaign tweets (N=4880), classifying them with our model as misconceptions or not. We analyzed dementia tweets from the United Kingdom across the campaign period (N=7124) to investigate how current events influenced misconception prevalence during this time.

Results

A random forest model best identified misconceptions with an accuracy of 82% from blind validation and found that 37% of the UK tweets (N=7124) about dementia across the campaign period were misconceptions. From this, we could track how the prevalence of misconceptions changed in response to top news stories in the United Kingdom. Misconceptions significantly rose around political topics and were highest (22/28, 79% of the dementia tweets) when there was controversy over the UK government allowing to continue hunting during the COVID-19 pandemic. After our campaign, there was no significant change in the prevalence of misconceptions.

Conclusions

Through codevelopment with carers, we developed an accurate ML model to predict misconceptions in dementia tweets. Our awareness campaign was ineffective, but similar campaigns could be enhanced through ML to respond to current events that affect misconceptions in real time.

1081 A COMPARISON OF THE ISAR TOOL AND THE CLINICAL FRAILTY SCALE TO PREDICT MORTALITY AND ED REATTENDANCE IN A COHORT OF ED ATTENDER

Background

Frailty Screening is one method by which we can risk stratify older adults to urgent assessment in the Emergency Department. The ISAR (Identification of Seniors at Risk) and Rockwood Clinical Frailty Scale are two frailty screening tools. We assessed the validity of these tools at predicting adverse outcomes for older adults presenting to the Emergency Department.

Method

This was a prospective cohort study. Patients over 65 were recruited, baseline.

demographics were obtained and a research nurse assessed them using both the Clinical Frailty Scale and ISAR. Patients were assessed by telephone interviews at one month and six months. The outcome measures assessed were mortality, ED re-attendance, hospital readmission, functional decline and institutionalisation.

Results

419 patients were recruited. 53.3% (223) were male with a median age of 76 (IQR = 10). The median ISAR and CFS score was 2,5 respectively at baseline. The mortality rate was 5.4% and rate of ED re-attendance was 16.9% at one month. The relative risk of ED re-admission with an ISAR score >/= 2 more was 1.84 (1.12, 3.02) and CFS > 4 was 1.85 (1.08, 3.16). The ISAR tool >/= 2, had a sensitivity of 74.29 (95% CI = 62.44, 83.99) and specificity of 41.18 (95% CI = 35.90, 46.61) when used as a diagnostic tool for ED re-admission at one month. The CFS > 4 had a sensitivity of 71.43 (95% CI = 57.79, 82.70) and specificity of 45.23 (95% CI = 39.33, 51.23) for the same outcome.

Conclusion

The ISAR tool >/= 2 was the more sensitive at predicting ED reattendance at one month in comparison to the Clinical Frailty Scale. We would advocate using this tool in the ED setting to highlight those at greatest risk of adverse outcomes and those most likely to benefit from Comprehensive Geriatric Assessment.

Remote Assessment of Disease and Relapse in Major Depressive Disorder (RADAR-MDD): Recruitment, retention, and data availability in a longitudinal remote measurement study

Introduction

Major Depressive Disorder (MDD) is prevalent, often chronic, and requires ongoing monitoring of symptoms to track response to treatment and identify early indicators of relapse. Remote Measurement Technologies (RMT) provide an exciting opportunity to transform the measurement and management of MDD, via data collected from inbuilt smartphone sensors and wearable devices alongside app-based questionnaires and tasks.

Objectives

To describe the amount of data collected during a multimodal longitudinal RMT study, in an MDD population.

Methods

RADAR-MDD is a multi-centre, prospective observational cohort study. People with a history of MDD were provided with a wrist-worn wearable, and several apps designed to: a) collect data from smartphone sensors; and b) deliver questionnaires, speech tasks and cognitive assessments and followed-up for a maximum of 2 years.

Results

A total of 623 individuals with a history of MDD were enrolled in the study with 80% completion rates for primary outcome assessments across all timepoints. 79.8% of people participated for the maximum amount of time available and 20.2% withdrew prematurely. Data availability across all RMT data types varied depending on the source of data and the participant-burden for each data type. We found no evidence of an association between the severity of depression symptoms at baseline and the availability of data. 110 participants had > 50% data available across all data types, and thus able to contribute to multiparametric analyses.

Conclusions

RADAR-MDD is the largest multimodal RMT study in the field of mental health. Here, we have shown that collecting RMT data from a clinical population is feasible.

Disclosure

No significant relationships.

Longitudinal Relationships Between Depressive Symptom Severity and Phone-Measured Mobility: Dynamic Structural Equation Modeling Study

BACKGROUND

The mobility of an individual measured by phone-collected location data has been found to be associated with depression; however, the longitudinal relationships (the temporal direction of relationships) between depressive symptom severity and phone-measured mobility have yet to be fully explored.

OBJECTIVE

We aimed to explore the relationships and the direction of the relationships between depressive symptom severity and phone-measured mobility over time.

METHODS

Data used in this paper came from a major EU program, called the Remote Assessment of Disease and Relapse-Major Depressive Disorder, which was conducted in 3 European countries. Depressive symptom severity was measured with the 8-item Patient Health Questionnaire (PHQ-8) through mobile phones every 2 weeks. Participants' location data were recorded by GPS and network sensors in mobile phones every 10 minutes, and 11 mobility features were extracted from location data for the 2 weeks prior to the PHQ-8 assessment. Dynamic structural equation modeling was used to explore the longitudinal relationships between depressive symptom severity and phone-measured mobility.

RESULTS

This study included 2341 PHQ-8 records and corresponding phone-collected location data from 290 participants (age: median 50.0 IQR 34.0, 59.0) years; of whom 215 (74.1%) were female, and 149 (51.4%) were employed. Significant negative correlations were found between depressive symptom severity and phone-measured mobility, and these correlations were more significant at the within-individual level than the between-individual level. For the direction of relationships over time, Homestay (time at home) (φ=0.09, P=.01), Location Entropy (time distribution on different locations) (φ=-0.04, P=.02), and Residential Location Count (reflecting traveling) (φ=0.05, P=.02) were significantly correlated with the subsequent changes in the PHQ-8 score, while changes in the PHQ-8 score significantly affected (φ=-0.07, P<.001) the subsequent periodicity of mobility.

CONCLUSIONS

Several phone-derived mobility features have the potential to predict future depression, which may provide support for future clinical applications, relapse prevention, and remote mental health monitoring practices in real-world settings.

Fitbeat: COVID-19 estimation based on wristband heart rate using a contrastive convolutional auto-encoder

This study proposes a contrastive convolutional auto-encoder (contrastive CAE), a combined architecture of an auto-encoder and contrastive loss, to identify individuals with suspected COVID-19 infection using heart-rate data from participants with multiple sclerosis (MS) in the ongoing RADAR-CNS mHealth research project. Heart-rate data was remotely collected using a Fitbit wristband. COVID-19 infection was either confirmed through a positive swab test, or inferred through a self-reported set of recognised symptoms of the virus. The contrastive CAE outperforms a conventional convolutional neural network (CNN), a long short-term memory (LSTM) model, and a convolutional auto-encoder without contrastive loss (CAE). On a test set of 19 participants with MS with reported symptoms of COVID-19, each one paired with a participant with MS with no COVID-19 symptoms, the contrastive CAE achieves an unweighted average recall of 95.3 % , a sensitivity of 100 % and a specificity of 90.6 % , an area under the receiver operating characteristic curve (AUC-ROC) of 0.944, indicating a maximum successful detection of symptoms in the given heart rate measurement period, whilst at the same time keeping a low false alarm rate.

Evaluating the Impact of Voice Activity Detection on Speech Emotion Recognition for Autistic Children

Individuals with autism are known to face challenges with emotion regulation, and express their affective states in a variety of ways. With this in mind, an increasing amount of research on automatic affect recognition from speech and other modalities has recently been presented to assist and provide support, as well as to improve understanding of autistic individuals' behaviours. As well as the emotion expressed from the voice, for autistic children the dynamics of verbal speech can be inconsistent and vary greatly amongst individuals. The current contribution outlines a voice activity detection (VAD) system specifically adapted to autistic children's vocalisations. The presented VAD system is a recurrent neural network (RNN) with long short-term memory (LSTM) cells. It is trained on 130 acoustic Low-Level Descriptors (LLDs) extracted from more than 17 h of audio recordings, which were richly annotated by experts in terms of perceived emotion as well as occurrence and type of vocalisations. The data consist of 25 English-speaking autistic children undertaking a structured, partly robot-assisted emotion-training activity and was collected as part of the DE-ENIGMA project. The VAD system is further utilised as a preprocessing step for a continuous speech emotion recognition (SER) task aiming to minimise the effects of potential confounding information, such as noise, silence, or non-child vocalisation. Its impact on the SER performance is compared to the impact of other VAD systems, including a general VAD system trained from the same data set, an out-of-the-box Web Real-Time Communication (WebRTC) VAD system, as well as the expert annotations. Our experiments show that the child VAD system achieves a lower performance than our general VAD system, trained under identical conditions, as we obtain receiver operating characteristic area under the curve (ROC-AUC) metrics of 0.662 and 0.850, respectively. The SER results show varying performances across valence and arousal depending on the utilised VAD system with a maximum concordance correlation coefficient (CCC) of 0.263 and a minimum root mean square error (RMSE) of 0.107. Although the performance of the SER models is generally low, the child VAD system can lead to slightly improved results compared to other VAD systems and in particular the VAD-less baseline, supporting the hypothesised importance of child VAD systems in the discussed context.

Identifying schizophrenia stigma on Twitter: a proof of principle model using service user supervised machine learning

Stigma has negative effects on people with mental health problems by making them less likely to seek help. We develop a proof of principle service user supervised machine learning pipeline to identify stigmatising tweets reliably and understand the prevalence of public schizophrenia stigma on Twitter. A service user group advised on the machine learning model evaluation metric (fewest false negatives) and features for machine learning. We collected 13,313 public tweets on schizophrenia between January and May 2018. Two service user researchers manually identified stigma in 746 English tweets; 80% were used to train eight models, and 20% for testing. The two models with fewest false negatives were compared in two service user validation exercises, and the best model used to classify all extracted public English tweets. Tweets classed as stigmatising by service users were more negative in sentiment (t (744) = 12.02, p < 0.001 [95% CI: 0.196–0.273]). Our linear Support Vector Machine was the best performing model with fewest false negatives and higher service user validation. This model identified public stigma in 47% of English tweets (n5,676) which were more negative in sentiment (t (12,143) = 64.38, p < 0.001 [95% CI: 0.29–0.31]). Machine learning can identify stigmatising tweets at large scale, with service user involvement. Given the prevalence of stigma, there is an urgent need for education and online campaigns to reduce it. Machine learning can provide a real time metric on their success.

The Association Between Home Stay and Symptom Severity in Major Depressive Disorder: Preliminary Findings From a Multicenter Observational Study Using Geolocation Data From Smartphones

BACKGROUND

Most smartphones and wearables are currently equipped with location sensing (using GPS and mobile network information), which enables continuous location tracking of their users. Several studies have reported that various mobility metrics, as well as home stay, that is, the amount of time an individual spends at home in a day, are associated with symptom severity in people with major depressive disorder (MDD). Owing to the use of small and homogeneous cohorts of participants, it is uncertain whether the findings reported in those studies generalize to a broader population of individuals with MDD symptoms.

OBJECTIVE

The objective of this study is to examine the relationship between the overall severity of depressive symptoms, as assessed by the 8-item Patient Health Questionnaire, and median daily home stay over the 2 weeks preceding the completion of a questionnaire in individuals with MDD.

METHODS

We used questionnaire and geolocation data of 164 participants with MDD collected in the observational Remote Assessment of Disease and Relapse-Major Depressive Disorder study. The participants were recruited from three study sites: King's College London in the United Kingdom (109/164, 66.5%); Vrije Universiteit Medisch Centrum in Amsterdam, the Netherlands (17/164, 10.4%); and Centro de Investigación Biomédica en Red in Barcelona, Spain (38/164, 23.2%). We used a linear regression model and a resampling technique (n=100 draws) to investigate the relationship between home stay and the overall severity of MDD symptoms. Participant age at enrollment, gender, occupational status, and geolocation data quality metrics were included in the model as additional explanatory variables. The 95% 2-sided CIs were used to evaluate the significance of model variables.

RESULTS

Participant age and severity of MDD symptoms were found to be significantly related to home stay, with older (95% CI 0.161-0.325) and more severely affected individuals (95% CI 0.015-0.184) spending more time at home. The association between home stay and symptoms severity appeared to be stronger on weekdays (95% CI 0.023-0.178, median 0.098; home stay: 25th-75th percentiles 17.8-22.8, median 20.9 hours a day) than on weekends (95% CI -0.079 to 0.149, median 0.052; home stay: 25th-75th percentiles 19.7-23.5, median 22.3 hours a day). Furthermore, we found a significant modulation of home stay by occupational status, with employment reducing home stay (employed participants: 25th-75th percentiles 16.1-22.1, median 19.7 hours a day; unemployed participants: 25th-75th percentiles 20.4-23.5, median 22.6 hours a day).

CONCLUSIONS

Our findings suggest that home stay is associated with symptom severity in MDD and demonstrate the importance of accounting for confounding factors in future studies. In addition, they illustrate that passive sensing of individuals with depression is feasible and could provide clinically relevant information to monitor the course of illness in patients with MDD.

240 A COMPARISON OF THE ISAR AND THE CLINICAL FRAILTY SCALE TO PREDICT MORTALITY AND ED RE-ATTENDANCE IN OLDER ED ATTENDERS

Background

Frailty Screening is one method by which we can risk stratify older adults to urgent assessment in the Emergency Department. The ISAR (Identification of Seniors at Risk) and Rockwood Clinical Frailty Scale (CFS) are two frailty screening tools. We assessed the validity of these tools at predicting adverse outcomes for older adults presenting to the Emergency Department.

Methods

This was a prospective cohort study. Patients over 65 were recruited, baseline demographics were obtained and a research nurse assessed them using both the CFS and ISAR. Patients were assessed by telephone interviews at one month and six months. The outcome measures assessed were mortality, ED re-attendance, hospital readmission, functional decline and institutionalisation.

Results

419 patients were recruited. 53.3% (223) were male with a median age of 76 (IQR = 10). At baseline, the median ISAR was 2 and CFS was 5. The mortality rate was 5.4% and the rate of ED re-attendance was 16.9% at one month. The relative risk of ED re-admission with an ISAR score ≥ 2 was 1.84 (1.12, 3.02) and CFS &gt; 4 was 1.85 (1.08, 3.16). The ISAR tool ≥2 had a sensitivity of 74% (95% CI = 62.44, 83.99) and specificity of 41% (95% CI = 35.90, 46.61) when used as a diagnostic tool for ED re-admission at one month. The CFS &gt; 4 had a sensitivity of 71% (95% CI = 57.79, 82.70) and specificity of 45% (95% CI = 39.33, 51.23) for ED re-attendance.

Conclusion

The ISAR tool ≥2 was more sensitive at predicting ED reattendance at one month in comparison to the CFS. We advocate to use this tool in the ED setting to highlight those at greatest risk of adverse outcomes and those most likely to benefit from Comprehensive Geriatric Assessment.

Investigating the impact of COVID-19 lockdown on adults with a recent history of recurrent major depressive disorder: a multi-Centre study using remote measurement technology

BACKGROUND

The outbreak of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), which causes a clinical illness Covid-19, has had a major impact on mental health globally. Those diagnosed with major depressive disorder (MDD) may be negatively impacted by the global pandemic due to social isolation, feelings of loneliness or lack of access to care. This study seeks to assess the impact of the 1st lockdown - pre-, during and post - in adults with a recent history of MDD across multiple centres.

METHODS

This study is a secondary analysis of an on-going cohort study, RADAR-MDD project, a multi-centre study examining the use of remote measurement technology (RMT) in monitoring MDD. Self-reported questionnaire and passive data streams were analysed from participants who had joined the project prior to 1st December 2019 and had completed Patient Health and Self-esteem Questionnaires during the pandemic (n = 252). We used mixed models for repeated measures to estimate trajectories of depressive symptoms, self-esteem, and sleep duration.

RESULTS

In our sample of 252 participants, 48% (n = 121) had clinically relevant depressive symptoms shortly before the pandemic. For the sample as a whole, we found no evidence that depressive symptoms or self-esteem changed between pre-, during- and post-lockdown. However, we found evidence that mean sleep duration (in minutes) decreased significantly between during- and post- lockdown (- 12.16; 95% CI - 18.39 to - 5.92; p <  0.001). We also found that those experiencing clinically relevant depressive symptoms shortly before the pandemic showed a decrease in depressive symptoms, self-esteem and sleep duration between pre- and during- lockdown (interaction p = 0.047, p = 0.045 and p <  0.001, respectively) as compared to those who were not.

CONCLUSIONS

We identified changes in depressive symptoms and sleep duration over the course of lockdown, some of which varied according to whether participants were experiencing clinically relevant depressive symptoms shortly prior to the pandemic. However, the results of this study suggest that those with MDD do not experience a significant worsening in symptoms during the first months of the Covid - 19 pandemic.

Predicting Depressive Symptom Severity Through Individuals' Nearby Bluetooth Device Count Data Collected by Mobile Phones: Preliminary Longitudinal Study

BACKGROUND

Research in mental health has found associations between depression and individuals' behaviors and statuses, such as social connections and interactions, working status, mobility, and social isolation and loneliness. These behaviors and statuses can be approximated by the nearby Bluetooth device count (NBDC) detected by Bluetooth sensors in mobile phones.

OBJECTIVE

This study aimed to explore the value of the NBDC data in predicting depressive symptom severity as measured via the 8-item Patient Health Questionnaire (PHQ-8).

METHODS

The data used in this paper included 2886 biweekly PHQ-8 records collected from 316 participants recruited from three study sites in the Netherlands, Spain, and the United Kingdom as part of the EU Remote Assessment of Disease and Relapse-Central Nervous System (RADAR-CNS) study. From the NBDC data 2 weeks prior to each PHQ-8 score, we extracted 49 Bluetooth features, including statistical features and nonlinear features for measuring the periodicity and regularity of individuals' life rhythms. Linear mixed-effect models were used to explore associations between Bluetooth features and the PHQ-8 score. We then applied hierarchical Bayesian linear regression models to predict the PHQ-8 score from the extracted Bluetooth features.

RESULTS

A number of significant associations were found between Bluetooth features and depressive symptom severity. Generally speaking, along with depressive symptom worsening, one or more of the following changes were found in the preceding 2 weeks of the NBDC data: (1) the amount decreased, (2) the variance decreased, (3) the periodicity (especially the circadian rhythm) decreased, and (4) the NBDC sequence became more irregular. Compared with commonly used machine learning models, the proposed hierarchical Bayesian linear regression model achieved the best prediction metrics (R2=0.526) and a root mean squared error (RMSE) of 3.891. Bluetooth features can explain an extra 18.8% of the variance in the PHQ-8 score relative to the baseline model without Bluetooth features (R2=0.338, RMSE=4.547).

CONCLUSIONS

Our statistical results indicate that the NBDC data have the potential to reflect changes in individuals' behaviors and statuses concurrent with the changes in the depressive state. The prediction results demonstrate that the NBDC data have a significant value in predicting depressive symptom severity. These findings may have utility for the mental health monitoring practice in real-world settings.

Relationship Between Major Depression Symptom Severity and Sleep Collected Using a Wristband Wearable Device: Multicenter Longitudinal Observational Study

Background

Sleep problems tend to vary according to the course of the disorder in individuals with mental health problems. Research in mental health has associated sleep pathologies with depression. However, the gold standard for sleep assessment, polysomnography (PSG), is not suitable for long-term, continuous monitoring of daily sleep, and methods such as sleep diaries rely on subjective recall, which is qualitative and inaccurate. Wearable devices, on the other hand, provide a low-cost and convenient means to monitor sleep in home settings.

Objective

The main aim of this study was to devise and extract sleep features from data collected using a wearable device and analyze their associations with depressive symptom severity and sleep quality as measured by the self-assessed Patient Health Questionnaire 8-item (PHQ-8).

Methods

Daily sleep data were collected passively by Fitbit wristband devices, and depressive symptom severity was self-reported every 2 weeks by the PHQ-8. The data used in this paper included 2812 PHQ-8 records from 368 participants recruited from 3 study sites in the Netherlands, Spain, and the United Kingdom. We extracted 18 sleep features from Fitbit data that describe participant sleep in the following 5 aspects: sleep architecture, sleep stability, sleep quality, insomnia, and hypersomnia. Linear mixed regression models were used to explore associations between sleep features and depressive symptom severity. The z score was used to evaluate the significance of the coefficient of each feature.

Results

We tested our models on the entire dataset and separately on the data of 3 different study sites. We identified 14 sleep features that were significantly (P<.05) associated with the PHQ-8 score on the entire dataset, among them awake time percentage (z=5.45, P<.001), awakening times (z=5.53, P<.001), insomnia (z=4.55, P<.001), mean sleep offset time (z=6.19, P<.001), and hypersomnia (z=5.30, P<.001) were the top 5 features ranked by z score statistics. Associations between sleep features and PHQ-8 scores varied across different sites, possibly due to differences in the populations. We observed that many of our findings were consistent with previous studies, which used other measurements to assess sleep, such as PSG and sleep questionnaires.

Conclusions

We demonstrated that several derived sleep features extracted from consumer wearable devices show potential for the remote measurement of sleep as biomarkers of depression in real-world settings. These findings may provide the basis for the development of clinical tools to passively monitor disease state and trajectory, with minimal burden on the participant.

New Avenues in Audio Intelligence: Towards Holistic Real-life Audio Understanding

Computer audition (i.e., intelligent audio) has made great strides in recent years; however, it is still far from achieving holistic hearing abilities, which more appropriately mimic human-like understanding. Within an audio scene, a human listener is quickly able to interpret layers of sound at a single time-point, with each layer varying in characteristics such as location, state, and trait. Currently, integrated machine listening approaches, on the other hand, will mainly recognise only single events. In this context, this contribution aims to provide key insights and approaches, which can be applied in computer audition to achieve the goal of a more holistic intelligent understanding system, as well as identifying challenges in reaching this goal. We firstly summarise the state-of-the-art in traditional signal-processing-based audio pre-processing and feature representation, as well as automated learning such as by deep neural networks. This concerns, in particular, audio interpretation, decomposition, understanding, as well as ontologisation. We then present an agent-based approach for integrating these concepts as a holistic audio understanding system. Based on this, concluding, avenues are given towards reaching the ambitious goal of 'holistic human-parity' machine listening abilities.

Using Smartphones and Wearable Devices to Monitor Behavioral Changes During COVID-19

BACKGROUND

In the absence of a vaccine or effective treatment for COVID-19, countries have adopted nonpharmaceutical interventions (NPIs) such as social distancing and full lockdown. An objective and quantitative means of passively monitoring the impact and response of these interventions at a local level is needed.

OBJECTIVE

We aim to explore the utility of the recently developed open-source mobile health platform Remote Assessment of Disease and Relapse (RADAR)-base as a toolbox to rapidly test the effect and response to NPIs intended to limit the spread of COVID-19.

METHODS

We analyzed data extracted from smartphone and wearable devices, and managed by the RADAR-base from 1062 participants recruited in Italy, Spain, Denmark, the United Kingdom, and the Netherlands. We derived nine features on a daily basis including time spent at home, maximum distance travelled from home, the maximum number of Bluetooth-enabled nearby devices (as a proxy for physical distancing), step count, average heart rate, sleep duration, bedtime, phone unlock duration, and social app use duration. We performed Kruskal-Wallis tests followed by post hoc Dunn tests to assess differences in these features among baseline, prelockdown, and during lockdown periods. We also studied behavioral differences by age, gender, BMI, and educational background.

RESULTS

We were able to quantify expected changes in time spent at home, distance travelled, and the number of nearby Bluetooth-enabled devices between prelockdown and during lockdown periods (P<.001 for all five countries). We saw reduced sociality as measured through mobility features and increased virtual sociality through phone use. People were more active on their phones (P<.001 for Italy, Spain, and the United Kingdom), spending more time using social media apps (P<.001 for Italy, Spain, the United Kingdom, and the Netherlands), particularly around major news events. Furthermore, participants had a lower heart rate (P<.001 for Italy and Spain; P=.02 for Denmark), went to bed later (P<.001 for Italy, Spain, the United Kingdom, and the Netherlands), and slept more (P<.001 for Italy, Spain, and the United Kingdom). We also found that young people had longer homestay than older people during the lockdown and fewer daily steps. Although there was no significant difference between the high and low BMI groups in time spent at home, the low BMI group walked more.

CONCLUSIONS

RADAR-base, a freely deployable data collection platform leveraging data from wearables and mobile technologies, can be used to rapidly quantify and provide a holistic view of behavioral changes in response to public health interventions as a result of infectious outbreaks such as COVID-19. RADAR-base may be a viable approach to implementing an early warning system for passively assessing the local compliance to interventions in epidemics and pandemics, and could help countries ease out of lockdown.

Real-time assessment of COVID-19 prevalence among multiple sclerosis patients: a multicenter European study

We assessed the prevalence and impact of COVID-19 among multiple sclerosis (MS) patients across Europe by leveraging participant data collected as part of the ongoing EU IMI2 RADAR-CNS major programme aimed at finding new ways of monitoring neurological disorders using wearable devices and smartphone technology. In the present study, 399 patients of RADAR-MS have been included (mean age 43.9 years, 60.7% females) with 87/399 patients (21.8%) reporting major symptoms suggestive of COVID-19. A trend for an increased risk of COVID-19 symptoms under alemtuzumab and cladribine treatments in comparison to injectables was observed. Remote monitoring technologies may support health authorities in monitoring and containing the ongoing pandemic.

Low Level Texture Features for Snore Sound Discrimination

Snoring is often associated with serious health risks such as obstructive sleep apnea and heart disease and may require targeted surgical interventions. In this regard, research into automatically and unobtrusively analysing the site of blockages that cause snore sounds is growing in popularity. Herein, we investigate the use of low level image texture features in classification of four specific types of snore sounds. Specifically, we explore histogram of local binary patterns (LBP) in dense grid of rectangular regions and histogram of oriented gradients (HOG) extracted from colour spectrograms for snore sound characterisation. Support vector machines with homogeneous mapping are used in the classification stage of the proposed method. Various experimental works are carried out with both LBP and HOG descriptors on the INTERSPEECH ComParE 2017 snoring sub-challenge dataset. Results presented indicate that LBP descriptors are better than the HOG descriptors in snore type detection and fusion of the LBP and HOG descriptors produces stronger results than either individual descriptor. Further, when compared to the challenge baseline and state-of-the-art deep spectrum features, our approach achieved relative percentage increases in unweighted average recall of 23.1% and 8.3% respectively.

Emotional expression in psychiatric conditions: New technology for clinicians

AIM

Emotional expressions are one of the most widely studied topics in neuroscience, from both clinical and non-clinical perspectives. Atypical emotional expressions are seen in various psychiatric conditions, including schizophrenia, depression, and autism spectrum conditions. Understanding the basics of emotional expressions and recognition can be crucial for diagnostic and therapeutic procedures. Emotions can be expressed in the face, gesture, posture, voice, and behavior and affect physiological parameters, such as the heart rate or body temperature. With modern technology, clinicians can use a variety of tools ranging from sophisticated laboratory equipment to smartphones and web cameras. The aim of this paper is to review the currently used tools using modern technology and discuss their usefulness as well as possible future directions in emotional expression research and treatment strategies.

METHODS

The authors conducted a literature review in the PubMed, EBSCO, and SCOPUS databases, using the following key words: 'emotions,' 'emotional expression,' 'affective computing,' and 'autism.' The most relevant and up-to-date publications were identified and discussed. Search results were supplemented by the authors' own research in the field of emotional expression.

RESULTS

We present a critical review of the currently available technical diagnostic and therapeutic methods. The most important studies are summarized in a table.

CONCLUSION

Most of the currently available methods have not been adequately validated in clinical settings. They may be a great help in everyday practice; however, they need further testing. Future directions in this field include more virtual-reality-based and interactive interventions, as well as development and improvement of humanoid robots.

Deep Unsupervised Representation Learning for Abnormal Heart Sound Classification

Given the world-wide prevalence of heart disease, the robust and automatic detection of abnormal heart sounds could have profound effects on patient care and outcomes. In this regard, a comparison of conventional and state-of-theart deep learning based computer audition paradigms for the audio classification task of normal, mild abnormalities, and moderate/severe abnormalities as present in phonocardiogram recordings, is presented herein. In particular, we explore the suitability of deep feature representations as learnt by sequence to sequence autoencoders based on the auDeep toolkit. Key results, gained on the new Heart Sounds Shenzhen corpus, indicate that a fused combination of deep unsupervised features is well suited to the three-way classification problem, achieving our highest unweighted average recall of 47.9% on the test partition.

Speech analysis for health: Current state-of-the-art and the increasing impact of deep learning

Due to the complex and intricate nature associated with their production, the acoustic-prosodic properties of a speech signal are modulated with a range of health related effects. There is an active and growing area of machine learning research in this speech and health domain, focusing on developing paradigms to objectively extract and measure such effects. Concurrently, deep learning is transforming intelligent signal analysis, such that machines are now reaching near human capabilities in a range of recognition and analysis tasks. Herein, we review current state-of-the-art approaches with speech-based health detection, placing a particular focus on the impact of deep learning within this domain. Based on this overview, it is evident while that deep learning based solutions be become more present in the literature, it has not had the same overall dominating effect seen in other related fields. In this regard, we suggest some possible research directions aimed at fully leveraging the advantages that deep learning can offer speech-based health detection.

Snore sound recognition: On wavelets and classifiers from deep nets to kernels

In this paper, we present a comprehensive comparison of wavelet features for the classification of snore sounds. Wavelet features have proven to be efficient in our previous work; however, the benefits of wavelet transform energy (WTE) and wavelet packet transform energy (WPTE) features were not clearly established. In this study, we firstly present our updated snore sounds database, expanded from 24 patients (collected by one medical centre) to 40 patients (collected by three medical centres). We then study the effects of varying frame sizes and overlaps for extraction of the wavelet low-level descriptors, the effect of which have yet to be fully established. We also compare the performance of the WTE and WPTE features when fed into multiple classifiers, namely, Support Vector Machines (SVM), K-Nearest Neighbours, Linear Discriminant Analysis, Random Forests, Extreme Learning Machines, Kernel Extreme Learning Machines, Multilayer Perceptron, and Deep Neural Networks. Key results presented indicate that, when fed into a SVM, WTE outperforms WPTE (one-tailed z-test, p<;0.002). Further, WPTE can achieve a significant improvement when trained by a k-nearest neighbours classifier (one-tailed z-test, p <; 0.001).

"You sound ill, take the day off": Automatic recognition of speech affected by upper respiratory tract infection

A combination of passive, non-invasive and nonintrusive smart monitoring technologies is currently transforming healthcare. These technologies will soon be able to provide immediate health related feedback for a range of illnesses and conditions. Such tools would be game changing for serious public health concerns, such as seasonal cold and flu, for which early diagnosis and social isolation play a key role in reducing the spread. In this regard, this paper explores, for the first times, the automated classification of individuals with Upper Respiratory Tract Infections (URTI) using recorded speech samples. Key results presented indicate that our classifiers can achieve similar results to those seen in related health-based detection tasks indicating the promise of using computational paralinguistic analysis for the detection of URTI related illnesses.

ViEWS from the prehospital perspective: a comparison with a prehospital score to triage categorisation in the emergency department

This aim of this observ estigate how the recently introduced National Early Warning Score (ViEWS) applied in the prehospital setting (Ph-ViEWS) compares with the Manchester Triage System (MTS) used in most Emergency Departments (ED) in Ireland. 386 patients fitted the inclusion criteria of which 272 (69%) had a complete set of values. Of 272 MTS 1 and 2 patients, only 114 (42%) had a Ph-ViEWS ⋝7. This study found that a substantial number of patients deemed urgent at the time of triage do not have elevated Ph-ViEWS.

The use of technology in Suicide Prevention

Suicide is one of the leading causes of death globally, and is notably a significant cause of death amongst young people. A suicide outcome is a complex combination of personal, social, and health factors, and therefore suicide prevention is a challenge, requiring a systems approach incorporating public health strategies, screening at-risk individuals, targeted interventions, and follow-up for suicide survivors and those bereaved by suicide. Engineering practice has been implicated in the hindrance of the adoption of suicide prevention strategies, such as installing safety barriers at the Golden Gate Bridge, however technological developments offer new opportunities in suicide prevention, and the potential to reduce the number of deaths by suicide. We present an overview of current technological developments which are facilitating research in the field of suicide prevention, including multiple modes of screening such as network analysis of mobile-phone collected connectivity data, automatic detection of suicidality from social media content, and crisis detection from acoustic variability in speech patterns. The current field of mhealth apps for suicide prevention is assessed, and an innovative app for an Indigenous population is presented. From this overview, future challenges - technical and ethical - are discussed.