Acceptability of wearable devices for measuring mobility remotely: Observations from the Mobilise-D technical validation study

Exploratory study to evaluate the acceptability of a wearable accelerometer to assess motor progression in motor neuron disease

Motor neuron disease (MND) is a rapidly progressive condition traditionally assessed using a questionnaire to evaluate physical function, the revised amyotrophic lateral sclerosis functional rating scale (ALSFRS-R). Its use can be associated with poor sensitivity in detecting subtle changes over time and there is an urgent need for more sensitive and specific outcome measures. The ActiGraph GT9X is a wearable device containing multiple sensors that can be used to provide metrics that represent physical activity. The primary aim of this study was to investigate the initial suitability and acceptability of limb-worn wearable devices to group of people with MND in Scotland. A secondary aim was to explore the preliminary associations between the accelerometer sensor data within the ActiGraph GT9X and established measures of physical function. 10 participants with MND completed a 12-week schedule of assessments including fortnightly study visits, both in-person and over videoconferencing software. Participants wore the device on their right wrist and right ankle for a series of movements, during a 6-min walking test and for a period of 24-h wear, including overnight. Participants also completed an ALSFRS-R and questionnaires on their experience with the devices. 80% of the participants found wearing these devices to be a positive experience and no one reported interference with daily living or added burden. However, 30% of the participants experienced technical issues with their devices. Data from the wearable devices correlated with established measures of physical function.

A wearable sensor and machine learning estimate step length in older adults and patients with neurological disorders

Step length is an important diagnostic and prognostic measure of health and disease. Wearable devices can estimate step length continuously (e.g., in clinic or real-world settings), however, the accuracy of current estimation methods is not yet optimal. We developed machine-learning models to estimate step length based on data derived from a single lower-back inertial measurement unit worn by 472 young and older adults with different neurological conditions, including Parkinson's disease and healthy controls. Studying more than 80,000 steps, the best model showed high accuracy for a single step (root mean square error, RMSE = 6.08 cm, ICC(2,1) = 0.89) and higher accuracy when averaged over ten consecutive steps (RMSE = 4.79 cm, ICC(2,1) = 0.93), successfully reaching the predefined goal of an RMSE below 5 cm (often considered the minimal-clinically-important-difference). Combining machine-learning with a single, wearable sensor generates accurate step length measures, even in patients with neurologic disease. Additional research may be needed to further reduce the errors in certain conditions.

Patient-centricity in digital measure development: co-evolution of best practice and regulatory guidance

Digital health technologies (DHTs) have the potential to modernize drug development and clinical trial operations by remotely, passively, and continuously collecting ecologically valid evidence that is meaningful to patients' lived experiences. Such evidence holds potential for all drug development stakeholders, including regulatory agencies, as it will help create a stronger evidentiary link between approval of new therapeutics and the ultimate aim of improving patient lives. However, only a very small number of novel digital measures have matured from exploratory usage into regulatory qualification or efficacy endpoints. This shows that despite the clear potential, actually gaining regulatory agreement that a new measure is both fit-for-purpose and delivers value remains a serious challenge. One of the key stumbling blocks for developers has been the requirement to demonstrate that a digital measure is meaningful to patients. This viewpoint aims to examine the co-evolution of regulatory guidance in the United States (U.S.) and best practice for integration of DHTs into the development of clinical outcome assessments. Contextualizing guidance on meaningfulness within the larger shift towards a patient-centric drug development approach, this paper reviews the U.S. Food and Drug Administration (FDA) guidance and existing literature surrounding the development of meaningful digital measures and patient engagement, including the recent examples of rejections by the FDA that further emphasize patient-centricity in digital measures. Finally, this paper highlights remaining hurdles and provides insights into the established frameworks for development and adoption of digital measures in clinical research.

Digital wearable insole-based identification of knee arthropathies and gait signatures using machine learning

Gait is impaired in musculoskeletal conditions, such as knee arthropathy. Gait analysis is used in clinical practice to inform diagnosis and monitor disease progression or intervention response. However, clinical gait analysis relies on subjective visual observation of walking as objective gait analysis has not been possible within clinical settings due to the expensive equipment, large-scale facilities, and highly trained staff required. Relatively low-cost wearable digital insoles may offer a solution to these challenges. In this work, we demonstrate how a digital insole measuring osteoarthritis-specific gait signatures yields similar results to the clinical gait-lab standard. To achieve this, we constructed a machine learning model, trained on force plate data collected in participants with knee arthropathy and controls. This model was highly predictive of force plate data from a validation set (area under the receiver operating characteristics curve [auROC] = 0.86; area under the precision-recall curve [auPR] = 0.90) and of a separate, independent digital insole dataset containing control and knee osteoarthritis subjects (auROC = 0.83; auPR = 0.86). After showing that digital insole-derived gait characteristics are comparable to traditional gait measurements, we next showed that a single stride of raw sensor time-series data could be accurately assigned to each subject, highlighting that individuals using digital insoles can be identified by their gait characteristics. This work provides a framework for a promising alternative to traditional clinical gait analysis methods, adds to the growing body of knowledge regarding wearable technology analytical pipelines, and supports clinical development of at-home gait assessments, with the potential to improve the ease, frequency, and depth of patient monitoring.

Contextualizing remote fall risk: Video data capture and implementing ethical AI

Wearable inertial measurement units (IMUs) are being used to quantify gait characteristics that are associated with increased fall risk, but the current limitation is the lack of contextual information that would clarify IMU data. Use of wearable video-based cameras would provide a comprehensive understanding of an individual's habitual fall risk, adding context to clarify abnormal IMU data. Generally, there is taboo when suggesting the use of wearable cameras to capture real-world video, clinical and patient apprehension due to ethical and privacy concerns. This perspective proposes that routine use of wearable cameras could be realized within digital medicine through AI-based computer vision models to obfuscate/blur/shade sensitive information while preserving helpful contextual information for a comprehensive patient assessment. Specifically, no person sees the raw video data to understand context, rather AI interprets the raw video data first to blur sensitive objects and uphold privacy. That may be more routinely achieved than one imagines as contemporary resources exist. Here, to showcase/display the potential an exemplar model is suggested via off-the-shelf methods to detect and blur sensitive objects (e.g., people) with an accuracy of 88%. Here, the benefit of the proposed approach includes a more comprehensive understanding of an individual's free-living fall risk (from free-living IMU-based gait) without compromising privacy. More generally, the video and AI approach could be used beyond fall risk to better inform habitual experiences and challenges across a range of clinical cohorts. Medicine is becoming more receptive to wearables as a helpful toolbox, camera-based devices should be plausible instruments.

Mobilise-D insights to estimate real-world walking speed in multiple conditions with a wearable device

This study aimed to validate a wearable device's walking speed estimation pipeline, considering complexity, speed, and walking bout duration. The goal was to provide recommendations on the use of wearable devices for real-world mobility analysis. Participants with Parkinson's Disease, Multiple Sclerosis, Proximal Femoral Fracture, Chronic Obstructive Pulmonary Disease, Congestive Heart Failure, and healthy older adults (n = 97) were monitored in the laboratory and the real-world (2.5 h), using a lower back wearable device. Two walking speed estimation pipelines were validated across 4408/1298 (2.5 h/laboratory) detected walking bouts, compared to 4620/1365 bouts detected by a multi-sensor reference system. In the laboratory, the mean absolute error (MAE) and mean relative error (MRE) for walking speed estimation ranged from 0.06 to 0.12 m/s and - 2.1 to 14.4%, with ICCs (Intraclass correlation coefficients) between good (0.79) and excellent (0.91). Real-world MAE ranged from 0.09 to 0.13, MARE from 1.3 to 22.7%, with ICCs indicating moderate (0.57) to good (0.88) agreement. Lower errors were observed for cohorts without major gait impairments, less complex tasks, and longer walking bouts. The analytical pipelines demonstrated moderate to good accuracy in estimating walking speed. Accuracy depended on confounding factors, emphasizing the need for robust technical validation before clinical application.Trial registration: ISRCTN - 12246987.

Overview on wearable sensors for the management of Parkinson's disease

Parkinson's disease (PD) is affecting about 1.2 million patients in Europe with a prevalence that is expected to have an exponential increment, in the next decades. This epidemiological evolution will be challenged by the low number of neurologists able to deliver expert care for PD. As PD is better recognized, there is an increasing demand from patients for rigorous control of their symptoms and for therapeutic education. In addition, the highly variable nature of symtoms between patients and the fluctuations within the same patient requires innovative tools to help doctors and patients monitor the disease in their usual living environment and adapt treatment in a more relevant way. Nowadays, there are various body-worn sensors (BWS) proposed to monitor parkinsonian clinical features, such as motor fluctuations, dyskinesia, tremor, bradykinesia, freezing of gait (FoG) or gait disturbances. BWS have been used as add-on tool for patients' management or research purpose. Here, we propose a practical anthology, summarizing the characteristics of the most used BWS for PD patients in Europe, focusing on their role as tools to improve treatment management. Consideration regarding the use of technology to monitor non-motor features is also included. BWS obviously offer new opportunities for improving management strategy in PD but their precise scope of use in daily routine care should be clarified.

Ambient assisted living systems for falls monitoring at home

INTRODUCTION

Monitoring systems at home are critical in the event of a fall, and can range from standalone fall detection devices to activity recognition devices that aim to identify behaviors in which the user may be at risk of falling, or to detect falls in real-time and alert emergency personnel.

AREAS COVERED

This review analyzes the current literature concerning the different devices available for home fall detection.

EXPERT OPINION

Included studies highlight how fall detection at home is an important challenge both from a clinical-assistance point of view and from a technical-bioengineering point of view. There are wearable, non-wearable and hybrid systems that aim to detect falls that occur in the patient's home. In the near future, a greater probability of predicting falls is expected thanks to an improvement in technologies together with the prediction ability of machine learning algorithms. Fall prevention must involve the clinician with a person-centered approach, low cost and minimally invasive technologies able to evaluate the movement of patients and machine learning algorithms able to make an accurate prediction of the fall event.

Digital endpoints in clinical trials of Alzheimer's disease and other neurodegenerative diseases: challenges and opportunities

Alzheimer's disease (AD) and other neurodegenerative diseases such as Parkinson's disease (PD) and Huntington's disease (HD) are associated with progressive cognitive, motor, affective and consequently functional decline considerably affecting Activities of Daily Living (ADL) and quality of life. Standard assessments, such as questionnaires and interviews, cognitive testing, and mobility assessments, lack sensitivity, especially in early stages of neurodegenerative diseases and in the disease progression, and have therefore a limited utility as outcome measurements in clinical trials. Major advances in the last decade in digital technologies have opened a window of opportunity to introduce digital endpoints into clinical trials that can reform the assessment and tracking of neurodegenerative symptoms. The Innovative Health Initiative (IMI)-funded projects RADAR-AD (Remote assessment of disease and relapse-Alzheimer's disease), IDEA-FAST (Identifying digital endpoints to assess fatigue, sleep and ADL in neurodegenerative disorders and immune-mediated inflammatory diseases) and Mobilise-D (Connecting digital mobility assessment to clinical outcomes for regulatory and clinical endorsement) aim to identify digital endpoints relevant for neurodegenerative diseases that provide reliable, objective, and sensitive evaluation of disability and health-related quality of life. In this article, we will draw from the findings and experiences of the different IMI projects in discussing (1) the value of remote technologies to assess neurodegenerative diseases; (2) feasibility, acceptability and usability of digital assessments; (3) challenges related to the use of digital tools; (4) public involvement and the implementation of patient advisory boards; (5) regulatory learnings; and (6) the significance of inter-project exchange and data- and algorithm-sharing.

Assessing real-world gait with digital technology? Validation, insights and recommendations from the Mobilise-D consortium

BACKGROUND

Although digital mobility outcomes (DMOs) can be readily calculated from real-world data collected with wearable devices and ad-hoc algorithms, technical validation is still required. The aim of this paper is to comparatively assess and validate DMOs estimated using real-world gait data from six different cohorts, focusing on gait sequence detection, foot initial contact detection (ICD), cadence (CAD) and stride length (SL) estimates.

METHODS

Twenty healthy older adults, 20 people with Parkinson's disease, 20 with multiple sclerosis, 19 with proximal femoral fracture, 17 with chronic obstructive pulmonary disease and 12 with congestive heart failure were monitored for 2.5 h in the real-world, using a single wearable device worn on the lower back. A reference system combining inertial modules with distance sensors and pressure insoles was used for comparison of DMOs from the single wearable device. We assessed and validated three algorithms for gait sequence detection, four for ICD, three for CAD and four for SL by concurrently comparing their performances (e.g., accuracy, specificity, sensitivity, absolute and relative errors). Additionally, the effects of walking bout (WB) speed and duration on algorithm performance were investigated.

RESULTS

We identified two cohort-specific top performing algorithms for gait sequence detection and CAD, and a single best for ICD and SL. Best gait sequence detection algorithms showed good performances (sensitivity > 0.73, positive predictive values > 0.75, specificity > 0.95, accuracy > 0.94). ICD and CAD algorithms presented excellent results, with sensitivity > 0.79, positive predictive values > 0.89 and relative errors < 11% for ICD and < 8.5% for CAD. The best identified SL algorithm showed lower performances than other DMOs (absolute error < 0.21 m). Lower performances across all DMOs were found for the cohort with most severe gait impairments (proximal femoral fracture). Algorithms' performances were lower for short walking bouts; slower gait speeds (< 0.5 m/s) resulted in reduced performance of the CAD and SL algorithms.

CONCLUSIONS

Overall, the identified algorithms enabled a robust estimation of key DMOs. Our findings showed that the choice of algorithm for estimation of gait sequence detection and CAD should be cohort-specific (e.g., slow walkers and with gait impairments). Short walking bout length and slow walking speed worsened algorithms' performances. Trial registration ISRCTN - 12246987.

Reverse Engineering of Digital Measures: Inviting Patients to the Conversation

Background

Digital measures offer an unparalleled opportunity to create a more holistic picture of how people who are patients behave in their real-world environments, thereby establishing a better connection between patients, caregivers, and the clinical evidence used to drive drug development and disease management. Reaching this vision will require achieving a new level of co-creation between the stakeholders who design, develop, use, and make decisions using evidence from digital measures.

Summary

In September 2022, the second in a series of meetings hosted by the Swiss Federal Institute of Technology in Zürich, the Foundation for the National Institutes of Health Biomarkers Consortium, and sponsored by Wellcome Trust, entitled "Reverse Engineering of Digital Measures," was held in Zurich, Switzerland, with a broad range of stakeholders sharing their experience across four case studies to examine how patient centricity is essential in shaping development and validation of digital evidence generation tools.

Key Messages

In this paper, we discuss progress and the remaining barriers to widespread use of digital measures for evidence generation in clinical development and care delivery. We also present key discussion points and takeaways in order to continue discourse and provide a basis for dissemination and outreach to the wider community and other stakeholders. The work presented here shows us a blueprint for how and why the patient voice can be thoughtfully integrated into digital measure development and that continued multistakeholder engagement is critical for further progress.