Digital Measures That Matter to Patients: A Framework to Guide the Selection and Development of Digital Measures of Health

Implementing sensor-based digital health technologies in clinical trials: Key considerations from the eCOA Consortium

The increased use of sensor-based digital health technologies (DHTs) in clinical trials brought to light concerns about implementation practices that might introduce burden on trial participants, resulting in suboptimal compliance and become an additional complicating factor in clinical trial conduct. These concerns may contribute to the lower-than-anticipated uptake of DHT deployment and data use for regulatory decision-making, despite well-articulated benefits. The Electronic Clinical Outcome Assessment (eCOA) Consortium gathered collective experience on deploying sensor-based DHTs and supplemented this with relevant literature focusing on mechanisms that may enhance participant compliance. The process for DHT implementation starts with identifying a clinical concept of interest followed by a digital measure selection, defining active or passive data capture and their sources, the number of sensors with respective body location, plus the duration and frequency of use in the context of perceived participant burden. Roundtable discussions among patient groups, physicians, and technology providers prior to protocol development can be very impactful for optimizing trial design. While diversity and inclusion are essential for any clinical trial, patient populations should be considered carefully in the context of trial-specific aims, requirements, and anticipated patient burden. Minimizing site burden includes assessment of training, research engagement, and logistical burden which needs to be triaged differently for early and late-stage clinical trials. Additional considerations include sharing trial results with study participants and leveraging publicly available data for compliance modeling. To the best of our knowledge, this report provides holistic considerations for sensor-based DHT implementation that may optimize participant compliance.

How can patients shape digital medicine? A rapid review of patient and public involvement and engagement in the development of digital health technologies for neurological conditions

INTRODUCTION

Patient and Public Involvement and Engagement (PPIE) involves working 'with' or 'by' patients and the public, rather than 'to,' 'about,' or 'for' them, and is integral to neurological and digital health research. This rapid review examined PPIE integration in the development and implementation of digital health technologies for neurological conditions.

METHODS

Key terms were input into six databases. Included articles were qualitative studies or PPIE activities involving patient perspectives in shaping digital health technologies for neurological conditions. Bias was evaluated using the NICE qualitative checklist, with reporting following PRISMA guidelines.

RESULTS

2,140 articles were identified, with 28 included. Of these, 25 were qualitative studies, and only three were focused PPIE activities. Patient involvement was mostly limited to one-off consultations during development.There was little evidence of PPIE during implementation, and minimal reporting on its impact.

CONCLUSIONS

PPIE has been inconsistently reported in this research area, highlighting the need for more guidance and best-practice examples This review used a UK-based definition of PPIE, which may have excluded relevant activities from other countries. Future reviews should broaden terminology to capture PPIE integration globally.

Quantitative Gait and Balance Outcomes for Ataxia Trials: Consensus Recommendations by the Ataxia Global Initiative Working Group on Digital-Motor Biomarkers

With disease-modifying drugs on the horizon for degenerative ataxias, ecologically valid, finely granulated, digital health measures are highly warranted to augment clinical and patient-reported outcome measures. Gait and balance disturbances most often present as the first signs of degenerative cerebellar ataxia and are the most reported disabling features in disease progression. Thus, digital gait and balance measures constitute promising and relevant performance outcomes for clinical trials.This narrative review with embedded consensus will describe evidence for the sensitivity of digital gait and balance measures for evaluating ataxia severity and progression, propose a consensus protocol for establishing gait and balance metrics in natural history studies and clinical trials, and discuss relevant issues for their use as performance outcomes.

Digital health technologies to strengthen patient-centred outcome assessment in clinical trials in inflammatory arthritis

Common to all inflammatory arthritides, namely rheumatoid arthritis, psoriatic arthritis, axial spondyloarthritis, and juvenile idiopathic arthritis, is a potential for reduced mobility that manifests through joint pain, swelling, stiffness, and ultimately joint damage. Across these conditions, consensus has been reached on the need to capture outcomes related to mobility, such as functional capacity and physical activity, as core domains in randomised controlled trials. Existing endpoints within these core domains rely wholly on self-reported questionnaires that capture patients' perceptions of their symptoms and activities. These questionnaires are subjective, inherently vulnerable to recall bias, and do not capture the granularity of fluctuations over time. Several early adopters have integrated sensor-based digital health technology (DHT)-derived endpoints to measure physical function and activity in randomised controlled trials for conditions including Parkinson's disease, Duchenne's muscular dystrophy, chronic obstructive pulmonary disease, and heart failure. Despite these applications, there have been no sensor-based DHT-derived endpoints in clinical trials recruiting patients with inflammatory arthritis. Borrowing from case studies across medicine, we outline the opportunities and challenges in developing novel sensor-based DHT-derived endpoints that capture the symptoms and disease manifestations most relevant to patients with inflammatory arthritis.

Do Measures of Real-World Physical Behavior Provide Insights Into the Well-Being and Physical Function of Cancer Survivors? Cross-Sectional Analysis

BACKGROUND

As the number of cancer survivors increases, maintaining health-related quality of life in cancer survivorship is a priority. This necessitates accurate and reliable methods to assess how cancer survivors are feeling and functioning. Real-world digital measures derived from wearable sensors offer potential for monitoring well-being and physical function in cancer survivorship, but questions surrounding the clinical utility of these measures remain to be answered.

OBJECTIVE

In this secondary analysis, we used 2 existing data sets to examine how measures of real-world physical behavior, captured with a wearable accelerometer, were related to aerobic fitness and self-reported well-being and physical function in a sample of individuals who had completed cancer treatment.

METHODS

Overall, 86 disease-free cancer survivors aged 21-85 years completed self-report assessments of well-being and physical function, as well as a submaximal exercise test that was used to estimate their aerobic fitness, quantified as predicted submaximal oxygen uptake (VO2). A thigh-worn accelerometer was used to monitor participants' real-world physical behavior for 7 days. Accelerometry data were used to calculate average values of the following measures of physical behavior: sedentary time, step counts, time in light and moderate to vigorous physical activity, time and weighted median cadence in stepping bouts over 1 minute, and peak 30-second cadence.

RESULTS

Spearman correlation analyses indicated that 6 (86%) of the 7 accelerometry-derived measures of real-world physical behavior were not significantly correlated with Functional Assessment of Cancer Therapy-General total well-being or linked Patient-Reported Outcomes Measurement Information System-Physical Function scores (Ps≥.08). In contrast, all but one of the physical behavior measures were significantly correlated with submaximal VO2 (Ps≤.03). Comparing these associations using likelihood ratio tests, we found that step counts, time in stepping bouts over 1 minute, and time in moderate to vigorous activity were more strongly associated with submaximal VO2 than with self-reported well-being or physical function (Ps≤.03). In contrast, cadence in stepping bouts over 1 minute and peak 30-second cadence were not more associated with submaximal VO2 than with the self-reported measures (Ps≥.08).

CONCLUSIONS

In a sample of disease-free cancer survivors, we found that several measures of real-world physical behavior were more associated with aerobic fitness than with self-reported well-being and physical function. These results highlight the possibility that in individuals who have completed cancer treatment, measures of real-world physical behavior may provide additional information compared with self-reported and performance measures. To advance the appropriate use of digital measures in oncology clinical research, further research evaluating the clinical utility of real-world physical behavior over time in large, representative samples of cancer survivors is warranted.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03781154; https://clinicaltrials.gov/ct2/show/NCT03781154.

Digital biomarkers for non-motor symptoms in Parkinson's disease: the state of the art

Digital biomarkers that remotely monitor symptoms have the potential to revolutionize outcome assessments in future disease-modifying trials in Parkinson's disease (PD), by allowing objective and recurrent measurement of symptoms and signs collected in the participant's own living environment. This biomarker field is developing rapidly for assessing the motor features of PD, but the non-motor domain lags behind. Here, we systematically review and assess digital biomarkers under development for measuring non-motor symptoms of PD. We also consider relevant developments outside the PD field. We focus on technological readiness level and evaluate whether the identified digital non-motor biomarkers have potential for measuring disease progression, covering the spectrum from prodromal to advanced disease stages. Furthermore, we provide perspectives for future deployment of these biomarkers in trials. We found that various wearables show high promise for measuring autonomic function, constipation and sleep characteristics, including REM sleep behavior disorder. Biomarkers for neuropsychiatric symptoms are less well-developed, but show increasing accuracy in non-PD populations. Most biomarkers have not been validated for specific use in PD, and their sensitivity to capture disease progression remains untested for prodromal PD where the need for digital progression biomarkers is greatest. External validation in real-world environments and large longitudinal cohorts remains necessary for integrating non-motor biomarkers into research, and ultimately also into daily clinical practice.

Development of data processing algorithm to calculate adherence for adults with cystic fibrosis using inhaled therapy - a multi-center observational study within the CFHealthHub learning health system

OBJECTIVES

To develop a robust algorithm to accurately calculate 'daily complete dose counts' for inhaled medicines, used in percent adherence calculations, from electronically-captured nebulizer data within the CFHealthHub Learning Health System.

METHODS

A multi-center, cross-sectional study involved participants and clinicians reviewing real-world inhaled medicine usage records and triangulating them with objective nebulizer data to establish a consensus on 'daily complete dose counts.' An algorithm, which used only objective nebulizer data, was then developed using a derivation dataset and evaluated using internal validation dataset. The agreement and accuracy between the algorithm-derived and consensus-derived 'daily complete dose counts' was examined, with the consensus-derived count as the reference standard.

RESULTS

Twelve people with CF participated. The algorithm derived a 'daily complete dose count' by screening out 'invalid' doses (those <60s in duration or run in cleaning mode), combining all doses starting within 120s of each other, and then screening out all doses with duration < 480s which were interrupted by power supply failure. The kappa co-efficient was 0.85 (0.71-0.91) in the derivation and 0.86 (0.77-0.94) in the validation dataset.

CONCLUSIONS

The algorithm demonstrated strong agreement with the participant-clinician consensus, enhancing confidence in CFHealthHub data. Publishingdata processing methods can encourage trust in digital endpoints and serve as an exemplar for other projects.

Unlocking the full potential of digital endpoints for decision making: a novel modular evidence concept enabling re-use and advancing collaboration

INTRODUCTION

Over the last decade increasing examples indicate opportunities to measure patient functioning and its relevance for clinical and regulatory decision making via endpoints collected through digital health technologies. More recently, we have seen such measures support primary study endpoints and enable smaller trials. The field is advancing fast: validation requirements have been proposed in the literature and regulators are releasing new guidances to review these endpoints. Pharmaceutical companies are embracing collaborations to develop them and working with academia and patient organizations in their development. However, the road to validation and regulatory acceptance is lengthy. The full value of digital endpoints cannot be unlocked until better collaboration and modular evidence frameworks are developed enabling re-use of evidence and repurposing of digital endpoints.

AREAS COVERED

This paper proposes a solution by presenting a novel modular evidence framework -the Digital Evidence Ecosystem and Protocols (DEEP)- enabling repurposing of measurement solutions, re-use of evidence, application of standards and also facilitates collaboration with health technology assessment bodies.

EXPERT OPINION

The integration of digital endpoints in healthcare, essential for personalized and remote care, requires harmonization and transparency. The proposed novel stack model offers a modular approach, fostering collaboration and expediting the adoption in patient care.

Video-based assessments of activities of daily living: generating real-world evidence in pediatric rare diseases

INTRODUCTION

Preserving function and independence to perform activities of daily living (ADL) is critical for patients and carers to manage the burden of care and improve quality of life. In children living with rare diseases, video recording ADLs offer the opportunity to collect the patients' experience in a real-life setting and accurately reflect treatment effectiveness on outcomes that matter to patients and families.

AREAS COVERED

We reviewed the measurement of ADL in pediatric rare diseases and the use of video to develop at-home electronic clinical outcome assessments (eCOA) by leveraging smartphone apps and artificial intelligence-based analysis. We broadly searched PubMed using Boolean combinations of the following MeSH terms 'Rare Diseases,' 'Quality of Life,' 'Activities of Daily Living,' 'Child,' 'Video Recording,' 'Outcome Assessment, Healthcare,' 'Intellectual disability,' and 'Genetic Diseases, Inborn.' Non-controlled vocabulary was used to include human pose estimation in movement analysis.

EXPERT OPINION

Broad uptake of video eCOA in drug development is linked to the generation of technical and clinical validation evidence to confidently assess a patient's functional abilities. Software platforms handling video data must align with quality regulations to ensure data integrity, security, and privacy. Regulatory flexibility and optimized validation processes should facilitate video eCOA to support benefit/risk drug assessment.

Leveraging sensor-based functional outcomes to enhance understanding of the patient experience: challenges and opportunities

INTRODUCTION

Sensor-based digital health technology (DHT) has emerged as a promising means to assess patient functioning within and outside clinical trials. Sensor-based functional outcomes (SBFOs) provide valuable insights that complement other measures of how a patient feels or functions to enhance understanding of the patient experience to inform medical product development.

AREAS COVERED

This perspective paper provides recommendations for defining SBFOs, discusses the core evidence required to support SBFOs to inform decision-making, and considers future directions for the field.

EXPERT COMMENTARY

The clinical outcome assessment (COA) development process provides an important starting point for developing patient-centered SBFOs; however, given the infancy of the field, SBFO development may benefit from a hybrid approach to evidence generation by merging exploratory data analysis with patient engagement in measure development. Effective SBFO development requires combining unique expertise in patient engagement, measurement and regulatory science, and digital health and analytics. Challenges specific to SBFO development include identifying concepts of interest, ensuring measurement of meaningful aspects of health, and identifying thresholds for meaningful change. SBFOs are complementary to other COAs and, as part of an integrated evidence strategy, offer great promise in fostering a holistic understanding of patient experience and treatment benefits, particularly in real-world settings.

From wearable sensor data to digital biomarker development: ten lessons learned and a framework proposal

Wearable sensor technologies are becoming increasingly relevant in health research, particularly in the context of chronic disease management. They generate real-time health data that can be translated into digital biomarkers, which can provide insights into our health and well-being. Scientific methods to collect, interpret, analyze, and translate health data from wearables to digital biomarkers vary, and systematic approaches to guide these processes are currently lacking. This paper is based on an observational, longitudinal cohort study, BarKA-MS, which collected wearable sensor data on the physical rehabilitation of people living with multiple sclerosis (MS). Based on our experience with BarKA-MS, we provide and discuss ten lessons we learned in relation to digital biomarker development across key study phases. We then summarize these lessons into a guiding framework (DACIA) that aims to informs the use of wearable sensor data for digital biomarker development and chronic disease management for future research and teaching.

Consensus Recommendations for Clinical Outcome Assessments and Registry Development in Ataxias: Ataxia Global Initiative (AGI) Working Group Expert Guidance

To accelerate and facilitate clinical trials, the Ataxia Global Initiative (AGI) was established as a worldwide research platform for trial readiness in ataxias. One of AGI's major goals is the harmonization and standardization of outcome assessments. Clinical outcome assessments (COAs) that describe or reflect how a patient feels or functions are indispensable for clinical trials, but similarly important for observational studies and in routine patient care. The AGI working group on COAs has defined a set of data including a graded catalog of COAs that are recommended as a standard for future assessment and sharing of clinical data and joint clinical studies. Two datasets were defined: a mandatory dataset (minimal dataset) that can ideally be obtained during a routine clinical consultation and a more demanding extended dataset that is useful for research purposes. In the future, the currently most widely used clinician-reported outcome measure (ClinRO) in ataxia, the scale for the assessment and rating of ataxia (SARA), should be developed into a generally accepted instrument that can be used in upcoming clinical trials. Furthermore, there is an urgent need (i) to obtain more data on ataxia-specific, patient-reported outcome measures (PROs), (ii) to demonstrate and optimize sensitivity to change of many COAs, and (iii) to establish methods and evidence of anchoring change in COAs in patient meaningfulness, e.g., by determining patient-derived minimally meaningful thresholds of change.

Methods for Neuroscience Drug Development: Guidance on Standardization of the Process for Defining Clinical Outcome Strategies in Clinical Trials

Neurosciences clinical trials continue to have notoriously high failure rates. Appropriate outcomes selection in early clinical trials is key to maximizing the likelihood of identifying new treatments in psychiatry and neurology. The field lacks good standards for designing outcome strategies, therefore The Outcomes Research Group was formed to develop and promote good practices in outcome selection. This article describes the first published guidance on the standardization of the process for clinical outcomes in neuroscience. A minimal step process is defined starting as early as possible, covering key activities for evidence generation in support of content validity, patient-centricity, validity requirements and considerations for regulatory acceptance. Feedback from expert members is provided, regarding the risks of shortening the process and examples supporting the recommended process are summarized. This methodology is now available to researchers in industry, academia or clinics aiming to implement consensus-based standard practices for clinical outcome selection, contributing to maximizing the efficiency of clinical research.

Patient-centricity in digital measure development: co-evolution of best practice and regulatory guidance

Digital health technologies (DHTs) have the potential to modernize drug development and clinical trial operations by remotely, passively, and continuously collecting ecologically valid evidence that is meaningful to patients' lived experiences. Such evidence holds potential for all drug development stakeholders, including regulatory agencies, as it will help create a stronger evidentiary link between approval of new therapeutics and the ultimate aim of improving patient lives. However, only a very small number of novel digital measures have matured from exploratory usage into regulatory qualification or efficacy endpoints. This shows that despite the clear potential, actually gaining regulatory agreement that a new measure is both fit-for-purpose and delivers value remains a serious challenge. One of the key stumbling blocks for developers has been the requirement to demonstrate that a digital measure is meaningful to patients. This viewpoint aims to examine the co-evolution of regulatory guidance in the United States (U.S.) and best practice for integration of DHTs into the development of clinical outcome assessments. Contextualizing guidance on meaningfulness within the larger shift towards a patient-centric drug development approach, this paper reviews the U.S. Food and Drug Administration (FDA) guidance and existing literature surrounding the development of meaningful digital measures and patient engagement, including the recent examples of rejections by the FDA that further emphasize patient-centricity in digital measures. Finally, this paper highlights remaining hurdles and provides insights into the established frameworks for development and adoption of digital measures in clinical research.

Patient Perspectives on AI-Driven Predictions of Schizophrenia Relapses: Understanding Concerns and Opportunities for Self-Care and Treatment

Early detection and intervention for relapse is important in the treatment of schizophrenia spectrum disorders. Researchers have developed AI models to predict relapse from patient-contributed data like social media. However, these models face challenges, including misalignment with practice and ethical issues related to transparency, accountability, and potential harm. Furthermore, how patients who have recovered from schizophrenia view these AI models has been underexplored. To address this gap, we first conducted semi-structured interviews with 28 patients and reflexive thematic analysis, which revealed a disconnect between AI predictions and patient experience, and the importance of the social aspect of relapse detection. In response, we developed a prototype that used patients' Facebook data to predict relapse. Feedback from seven patients highlighted the potential for AI to foster collaboration between patients and their support systems, and to encourage self-reflection. Our work provides insights into human-AI interaction and suggests ways to empower people with schizophrenia.

Tepid Uptake of Digital Health Technologies in Clinical Trials by Pharmaceutical and Medical Device Firms

Digital health technologies (DHTs) can enable more patient-centric therapeutic development by generating evidence that captures how patients feel and function, enabling decentralized trial designs that increase participant inclusivity and convenience, and collecting and structuring patient-generated data for regulators to use in approval decisions alongside traditional clinical outcomes. Although a growing body of evidence has documented increasing use of DHTs in clinical trials overall, the use of DHTs in clinical trials supporting medical product development is unclear; here, we quantify the use of DHTs in clinical trials sponsored by pharmaceutical and medical device firms. Despite interest from pharmaceutical and medical device manufacturers in DHTs, we find tepid uptake of DHTs in trials by these sponsor types over time. Further, to date, these sponsors have most frequently used conventional, hardware-based technologies that have been available for many years (e.g., Holter monitors and glucose meters) rather than newer activity monitors, mobile apps, and other online-based tools that are frequently used by non-industry sponsors. Considering the recent and evolving nature of regulatory guidance around DHT use in clinical trials, our findings suggest that organizations pursuing product development still appear hesitant to incorporate DHTs in trials that provide the most critical evidence for regulatory review and impact how new products are used. This suggests there are likely additional opportunities for sponsors of regulated trials to incorporate (more) DHTs and patient-centric endpoints into product development clinical trials. However, additional regulatory clarity and efforts to reduce operational barriers may be needed in order to more fully capture these opportunities.

Digital Gait Measures Capture 1-Year Progression in Early-Stage Spinocerebellar Ataxia Type 2

BACKGROUND

With disease-modifying drugs in reach for cerebellar ataxias, fine-grained digital health measures are highly warranted to complement clinical and patient-reported outcome measures in upcoming treatment trials and treatment monitoring. These measures need to demonstrate sensitivity to capture change, in particular in the early stages of the disease.

OBJECTIVE

Our aim is to unravel gait measures sensitive to longitudinal change in the-particularly trial-relevant-early stage of spinocerebellar ataxia type 2 (SCA2).

METHODS

We performed a multicenter longitudinal study with combined cross-sectional and 1-year interval longitudinal analysis in early-stage SCA2 participants (n = 23, including nine pre-ataxic expansion carriers; median, ATXN2 CAG repeat expansion 38 ± 2; median, Scale for the Assessment and Rating of Ataxia [SARA] score 4.8 ± 4.3). Gait was assessed using three wearable motion sensors during a 2-minute walk, with analyses focused on gait measures of spatio-temporal variability that have shown sensitivity to ataxia severity (eg, lateral step deviation).

RESULTS

We found significant changes for gait measures between baseline and 1-year follow-up with large effect sizes (lateral step deviation P = 0.0001, effect size rprb = 0.78), whereas the SARA score showed no change (P = 0.67). Sample size estimation indicates a required cohort size of n = 43 to detect a 50% reduction in natural progression. Test-retest reliability and minimal detectable change analysis confirm the accuracy of detecting 50% of the identified 1-year change.

CONCLUSIONS

Gait measures assessed by wearable sensors can capture natural progression in early-stage SCA2 within just 1 year-in contrast to a clinical ataxia outcome. Lateral step deviation represents a promising outcome measure for upcoming multicenter interventional trials, particularly in the early stages of cerebellar ataxia. © 2024 The Authors. Movement Disorders published by Wiley Periodicals LLC on behalf of International Parkinson and Movement Disorder Society.

An aligned framework of actively collected and passively monitored clinical outcome assessments (COAs) for measure selection

Regulators increasingly require clinical outcome assessment (COA) data for approval. COAs can be collected via questionnaires or digital health technologies (DHTs), yet no single resource provides a side-by-side comparison of tools that collect complementary or related COA measures. We propose how to align ontologies for actively collected and passively monitored COAs into a single framework to allow for rapid, evidence-based, and fit-for-purpose measure selection.

Smartphone applications for Movement Disorders: Towards collaboration and re-use

BACKGROUND

Numerous smartphone and tablet applications (apps) are available to monitor movement disorders, but an overview of their purpose and stage of development is missing.

OBJECTIVES

To systematically review published literature and classify smartphone and tablet apps with objective measurement capabilities for the diagnosis, monitoring, assessment, or treatment of movement disorders.

METHODS

We systematically searched for publications covering smartphone or tablet apps to monitor movement disorders until November 22nd, 2023. We reviewed the target population, measured domains, purpose, and technology readiness level (TRL) of the proposed app and checked their availability in common app stores.

RESULTS

We identified 113 apps. Most apps were developed for Parkinson's disease specifically (n = 82; 73%) or for movement disorders in general (n = 17; 15%). Apps were either designed to momentarily assess symptoms (n = 65; 58%), support treatment (n = 22; 19%), aid in diagnosis (n = 16; 14%), or passively track symptoms (n = 11; 10%). Commonly assessed domains across movement disorders included fine motor skills (n = 34; 30%), gait (n = 36; 32%), and tremor (n = 32; 28%) for the motor domain and cognition (n = 16; 14%) for the non-motor domain. Twenty-six (23%) apps were proof-of-concepts (TRL 1-3), while most apps were tested in a controlled setting (TRL 4-6; n = 63; 56%). Twenty-four apps were tested in their target setting (TRL 7-9) of which 10 were accessible in common app stores or as Android Package.

CONCLUSIONS

The development of apps strongly gravitates towards Parkinson's disease and a selection of motor symptoms. Collaboration, re-use and further development of existing apps is encouraged to avoid reinventions of the wheel.

Dementia Ideal Care: Ecosystem Map of Best Practices and Care Pathways Enhanced by Technology and Community

Background

Globally, much work has been done by nonprofit, private, and academic groups to develop best practices for the care of people living with dementia (PLWD), including Alzheimer's disease. However, these best practices reside in disparate repositories and tend to focus on one phase of the patient journey or one relevant group.

Objective

To fill this gap, we developed a Dementia Ideal Care Map that everyone in the dementia ecosystem can use as an actionable tool for awareness, policy development, funding, research, training, service delivery, and technology design. The intended audience includes (and not limited to) policymakers, academia, industry, technology developers, health system leaders, clinicians, social service providers, patient advocates, PLWD, their families, and communities at large.

Methods

A search was conducted for published dementia care best practices and quality measures, which were then summarized in a visual diagram. The draft diagram was analyzed to identify barriers to ideal care. Then, additional processes, services, technologies, and quality measures to overcome those challenges were brainstormed. Feedback was then obtained from experts.

Results

The Dementia Ideal Care Map summarizes the ecosystem of over 200 best practices, nearly 100 technology enablers, other infrastructure, and enhanced care pathways in one comprehensive diagram. It includes psychosocial interventions, care partner support, community-based organizations; awareness, risk reduction; initial detection, diagnosis, ongoing medical care; governments, payers, health systems, businesses, data, research, and training.

Conclusions

Dementia Ideal Care Map is a practical tool for planning and coordinating dementia care. This visualized ecosystem approach can be applied to other conditions.

Digital Health Technologies for Alzheimer's Disease and Related Dementias: Initial Results from a Landscape Analysis and Community Collaborative Effort

Digital health technologies offer valuable advantages to dementia researchers and clinicians as screening tools, diagnostic aids, and monitoring instruments. To support the use and advancement of these resources, a comprehensive overview of the current technological landscape is essential. A multi-stakeholder working group, convened by the Digital Medicine Society (DiMe), conducted a landscape review to identify digital health technologies for Alzheimer's disease and related dementia populations. We searched studies indexed in PubMed, Embase, and APA PsycInfo to identify manuscripts published between May 2003 to May 2023 reporting analytical validation, clinical validation, or usability/feasibility results for relevant digital health technologies. Additional technologies were identified through community outreach. We collated peer-reviewed manuscripts, poster presentations, or regulatory documents for 106 different technologies for Alzheimer's disease and related dementia assessment covering diverse populations such as Lewy Body, vascular dementias, frontotemporal dementias, and all severities of Alzheimer's disease. Wearable sensors represent 32% of included technologies, non-wearables 61%, and technologies with components of both account for the remaining 7%. Neurocognition is the most prevalent concept of interest, followed by physical activity and sleep. Clinical validation is reported in 69% of evidence, analytical validation in 34%, and usability/feasibility in 20% (not mutually exclusive). These findings provide clinicians and researchers a landscape overview describing the range of technologies for assessing Alzheimer's disease and related dementias. A living library of technologies is presented for the clinical and research communities which will keep findings up-to-date as the field develops.

Mobilizing Patient and Public Involvement in the Development of Real-World Digital Technology Solutions: Tutorial

Although the value of patient and public involvement and engagement (PPIE) activities in the development of new interventions and tools is well known, little guidance exists on how to perform these activities in a meaningful way. This is particularly true within large research consortia that target multiple objectives, include multiple patient groups, and work across many countries. Without clear guidance, there is a risk that PPIE may not capture patient opinions and needs correctly, thereby reducing the usefulness and effectiveness of new tools. Mobilise-D is an example of a large research consortium that aims to develop new digital outcome measures for real-world walking in 4 patient cohorts. Mobility is an important indicator of physical health. As such, there is potential clinical value in being able to accurately measure a person's mobility in their daily life environment to help researchers and clinicians better track changes and patterns in a person's daily life and activities. To achieve this, there is a need to create new ways of measuring walking. Recent advancements in digital technology help researchers meet this need. However, before any new measure can be used, researchers, health care professionals, and regulators need to know that the digital method is accurate and both accepted by and produces meaningful outcomes for patients and clinicians. Therefore, this paper outlines how PPIE structures were developed in the Mobilise-D consortium, providing details about the steps taken to implement PPIE, the experiences PPIE contributors had within this process, the lessons learned from the experiences, and recommendations for others who may want to do similar work in the future. The work outlined in this paper provided the Mobilise-D consortium with a foundation from which future PPIE tasks can be created and managed with clearly defined collaboration between researchers and patient representatives across Europe. This paper provides guidance on the work required to set up PPIE structures within a large consortium to promote and support the creation of meaningful and efficient PPIE related to the development of digital mobility outcomes.

Physical function endpoints in cancer cachexia clinical trials: Systematic Review 1 of the cachexia endpoints series

In cancer cachexia trials, measures of physical function are commonly used as endpoints. For drug trials to obtain regulatory approval, efficacy in physical function endpoints may be needed alongside other measures. However, it is not clear which physical function endpoints should be used. The aim of this systematic review was to assess the frequency and diversity of physical function endpoints in cancer cachexia trials. Following a comprehensive electronic literature search of MEDLINE, Embase and Cochrane (1990-2021), records were retrieved. Eligible trials met the following criteria: adults (≥18 years), controlled design, more than 40 participants, use of a cachexia intervention for more than 14 days and use of a physical function endpoint. Physical function measures were classified as an objective measure (hand grip strength [HGS], stair climb power [SCP], timed up and go [TUG] test, 6-min walking test [6MWT] and short physical performance battery [SPPB]), clinician assessment of function (Karnofsky Performance Status [KPS] or Eastern Cooperative Oncology Group-Performance Status [ECOG-PS]) or patient-reported outcomes (physical function subscale of the European Organisation for the Research and Treatment of Cancer Quality of Life Questionnaires [EORTC QLQ-C30 or C15]). Data extraction was performed using Covidence and followed PRISMA guidance (PROSPERO registration: CRD42022276710). A total of 5975 potential studies were examined and 71 were eligible. Pharmacological interventions were assessed in 38 trials (54%). Of these, 11 (29%, n = 1184) examined megestrol and 5 (13%, n = 1928) examined anamorelin; nutritional interventions were assessed in 21 trials (30%); and exercise-based interventions were assessed in 6 trials (8%). The remaining six trials (8%) assessed multimodal interventions. Among the objective measures of physical function (assessed as primary or secondary endpoints), HGS was most commonly examined (33 trials, n = 5081) and demonstrated a statistically significant finding in 12 (36%) trials (n = 2091). The 6MWT was assessed in 12 trials (n = 1074) and was statistically significant in 4 (33%) trials (n = 403), whereas SCP, TUG and SPPB were each assessed in 3 trials. KPS was more commonly assessed than the newer ECOG-PS (16 vs. 9 trials), and patient-reported EORTC QLQ-C30 physical function was reported in 25 trials. HGS is the most commonly used physical function endpoint in cancer cachexia clinical trials. However, heterogeneity in study design, populations, intervention and endpoint selection make it difficult to comment on the optimal endpoint and how to measure this. We offer several recommendations/considerations to improve the design of future clinical trials in cancer cachexia.

A patient-centred conceptual model of nocturnal scratch and its impact in atopic dermatitis: A mixed-methods study supporting the development of novel digital measurements

Background

Emerging digital measures and clinical outcome assessments (COAs) leveraging digital health technologies (DHTs) could address the need for objective, quantitative measures of symptoms of atopic dermatitis (AD), such as nocturnal scratching. Development of such measures needs to be supported by evidence reflecting meaningfulness to patients.

Objectives

To assess nocturnal scratching as a concept of interest associated with meaningful aspects of health of patients with AD (adults and children); and to explore patient-centred considerations for novel COAs measuring nocturnal scratch using DHTs.

Methods

Phase 1 evaluated disease impacts on everyday life and the lived experience with nocturnal scratching through qualitative interviews of AD patients and caregivers. Phase 2 deployed a quantitative survey to a sample of AD patients as well as caregivers.

Results

Four cohorts with various AD severity levels participated in Phase 1: (1) adults with AD (n = 15), (2) their caregivers/spouses/partners (n = 6), (3) children with AD (n = 14), and (4) their adult caregivers (n = 14). Findings were used to develop a conceptual model for nocturnal scratching as a potential concept of interest. The Phase 2 survey was completed by 1349 of 27640 invited adults with AD and caregivers of children with AD. The most burdensome aspects of AD reported were itchy skin and scratching. Overall, ∼65% of participants reported nocturnal scratching ≥1 day/week, resulting in ∼1-1.4 h of sleep lost per night. In all, 85%-91% of respondents considered it at least somewhat valuable that a treatment reduces night-time scratching. About 50% reported willingness to use technology to this end and ∼25% were unsure.

Conclusion

Our results represented by the conceptual model confirm that nocturnal scratch is a concept of interest related to meaningful aspects of health for patients with AD and therefore is worth being captured as a distinct outcome for clinical and research purposes. DHTs are suitable tools presenting an important measurement opportunity to assess and evaluate occurrence, frequency, and other parameters of nocturnal scratching as a disease biomarker or COA of treatment efficacy.

Mixed-methods study protocol to identify expectations of people with type 1 diabetes and their caregivers about voice-based digital health solutions to support the management of diabetes distress: the PsyVoice study

INTRODUCTION

Type 1 diabetes (T1D) requires continuous management to obtain a good metabolic control and prevent acute complications. This often affects psychological well-being. People with T1D frequently report diabetes distress (DD). Psychological issues can negatively affect metabolic control and well-being. New technologies can improve quality of life, reduce the treatment burden and improve glycaemic control. Voice technology may serve as an innovative and inexpensive remote monitoring device to evaluate psychological well-being. Tailoring digital health interventions according to the ability and interest of their intended 'end-users' increases the acceptability of the intervention itself. PsyVoice explores the perspectives and needs of people with T1D on voice-based digital health interventions to manage DD.

METHODS AND ANALYSIS

PsyVoice is a mixed-methods study with qualitative and quantitative data sources. For the qualitative part, the researchers will invite 20 people with a T1D or caregivers of children with T1D to participate in in-depth semi-structured interviews. They will be invited as well to answer three questionnaires to assess socio-demographics, diabetes management, e-Health literacy and diabetes distress. Information from questionnaires will be integrated with themes developed in the qualitative analysis of the interviews. People with T1D will be invited to participate in the protocol and give feedback on interview guides, questionnaires, information sheets and informed consent.

ETHICS AND DISSEMINATION

PsyVoice received ethical approval from Luxembourg's National Research Ethics Committee. Participants will receive information about the purpose, risks and strategies to ensure the confidentiality and anonymity of the study. The results of PsyVoice will guide the selection and development of voice-based technological interventions for managing DD. The outcome will be disseminated to academic and non-academic stakeholders through peer-reviewed open-access journals and a lay public report.

TRIAL REGISTRATION NUMBER

This study is registered on ClinicalTrials.gov with the number NCT05517772.

Measuring What Is Meaningful in Cancer Cachexia Clinical Trials: A Path Forward With Digital Measures of Real-World Physical Behavior

PURPOSE

The burden of cancer cachexia on patients' health-related quality of life, specifically their physical functioning, is well documented, but clinical trials thus far have failed to show meaningful improvement in physical functioning. The purpose of this review is to summarize existing methods of assessing physical function in cancer cachexia, outline a path forward for measuring what is meaningful to patients using digital measures derived from digital health technologies (DHTs), and discuss the current landscape of digital measures from the clinical and regulatory standpoint.

DESIGN

For this narrative review, peer-reviewed articles were searched on PubMed, clinical trials records were searched on clinicaltrials.gov, and records of digital measures submitted for regulatory qualification were searched on the US Food and Drug Administration's Drug Development Tool Qualification Program database.

RESULTS

There are gaps in assessing aspects of physical function that matter to patients. Existing assessment methods such as patient-reported outcomes and objective performance outcomes have limitations, including their episodic nature and burden to patients. DHTs such as wearable sensors can capture real-world physical behavior continuously, passively, and remotely, and may provide a more comprehensive picture of patients' everyday functioning. Recent regulatory submissions showcase potential clinical implementation of digital measures in various therapeutic areas.

CONCLUSION

Digital measures of real-world physical behavior present an opportunity to detect and demonstrate improvements in physical functioning in cancer cachexia, but evidence-based development is critical. For their use in clinical and regulatory decision making, studies demonstrating meaningfulness to patients as well as feasibility and validation are necessary.

Digital health technology derived measures: Biomarkers or clinical outcome assessments?

Digital health technologies (DHTs) present unique opportunities for clinical evidence generation but pose certain challenges. These challenges stem, in part, from existing definitions of drug development tools, which were not created with DHT-derived measures in mind. DHT-derived measures can be leveraged as either clinical outcome assessments (COAs) or as biomarkers since they share properties with both categories of drug development tools. Examples from the literature indicate a variety of applications for DHT-derived data, including capturing disease physiology, symptom tracking, or response to therapies. The distinction between the categorization of DHT-derived measures as COAs or as biomarkers can be very fine, with terminology variability among regulatory authorities. This has significant implications for integration of DHT-derived measures in clinical trials, leading to confusion regarding the evidence required to support these tools' use in drug development. There is a need to amend definitions and create clear evidentiary requirements to support broad adoption of these new and innovative tools. The biopharma industry, the technology sector, consulting businesses, academic researchers, and regulators need a dialogue via multi-stakeholder collaborations to clarify questions around DHT-derived measures, to unify definitions, and to create the foundations for evidentiary package requirements, providing a path forward to predictable results.

Digital Tools-Regulatory Considerations for Application in Clinical Trials

During the last few years, the pharmaceutical industry has adopted digital technologies/digital health technology (DHT) to improve the drug development process and the commercialization of new products. Technological advances are strongly supported by both the US-FDA and the EMA, but the regulatory landscape in the US seems to be more suitable to promote innovation in the digital health sector (e.g. Cures Act). In contrast, the new Medical Device Regulation sets high hurdles for Medical Device software to pass regulatory scrutiny.On both sides of the Atlantic, a digital tool must be fit-for-purpose for the intended use in the clinical drug trial. Irrespective of its status as a Medical Device, at least the basic safety and performance requirements according to local regulations should be met, quality system and surveillance requirements should be fulfilled, and the sponsor must ensure conformity with GxP and the local data privacy and cybersecurity legislations.There is an overlap in technical and clinical validation for drug development tool qualification in both regions to ensure that the digital tools deliver reliable data with tangible clinical benefits. Based on an analysis of the regulatory framework of the FDA and the EMA, this study proposes regulatory strategies for a global pharma company: It would be prudent for drug development companies to a) use approved solutions or b) consider qualification of drug development tools early and in parallel to clinical development. Early engagement with the FDA and the EMA/CA is recommended to define evidentiary standards and corresponding regulatory pathways for different contexts-of-use and to clarify regulator's expectations as to what extent data collected by digital tools are acceptable to support marketing authorization applications (MAA).Hence a harmonization of the partly disparate regulatory requirements in the US and the EU accompanied by further development of the regulatory landscape in the EU, could further foster the use of digital tools in drug clinical development. The outlook for the use of digital tools in clinical trials is hopeful.

From Meaningful Outcomes to Meaningful Change Thresholds: A Path to Progress for Establishing Digital Endpoints

This paper examines the use of digital endpoints (DEs) derived from digital health technologies (DHTs), focusing primarily on the specific considerations regarding the determination of meaningful change thresholds (MCT). Using DHTs in drug development is becoming more commonplace. There is general acceptance of the value of DHTs supporting patient-centric trial design, capturing data outside the traditional clinical trial setting, and generating DEs with the potential to be more sensitive to change than conventional assessments. However, the transition from exploratory endpoints to primary and secondary endpoints capable of supporting labeling claims requires these endpoints to be substantive with reproducible population-specific values. Meaningful change represents the amount of change in an endpoint measure perceived as important to patients and should be determined for each digital endpoint and given population under consideration. This paper examines existing approaches to determine meaningful change thresholds and explores examples of these methodologies and their use as part of DE development: emphasizing the importance of determining what aspects of health are important to patients and ensuring the DE captures these concepts of interest and aligns with the overarching endpoint strategy. Examples are drawn from published DE qualification documentation and responses to qualification submissions under review by the various regulatory authorities. It is the hope that these insights will inform and strengthen the development and validation of DEs as drug development tools, particularly for those new to the approaches to determine MCTs.

Co-Design of a Voice-Based Digital Health Solution to Monitor Persisting Symptoms Related to COVID-19 (UpcomingVoice Study): Protocol for a Mixed Methods Study

BACKGROUND

Between 10% and 20% of people with a COVID-19 infection will develop the so-called long COVID syndrome, which is characterized by fluctuating symptoms. Long COVID has a high impact on the quality of life of affected people, who often feel abandoned by the health care system and are demanding new tools to help them manage their symptoms. New digital monitoring solutions could allow them to visualize the evolution of their symptoms and could be tools to communicate with health care professionals (HCPs). The use of voice and vocal biomarkers could facilitate the accurate and objective monitoring of persisting and fluctuating symptoms. However, to assess the needs and ensure acceptance of this innovative approach by its potential users-people with persisting COVID-19-related symptoms, with or without a long COVID diagnosis, and HCPs involved in long COVID care-it is crucial to include them in the entire development process.

OBJECTIVE

In the UpcomingVoice study, we aimed to define the most relevant aspects of daily life that people with long COVID would like to be improved, assess how the use of voice and vocal biomarkers could be a potential solution to help them, and determine the general specifications and specific items of a digital health solution to monitor long COVID symptoms using vocal biomarkers with its end users.

METHODS

UpcomingVoice is a cross-sectional mixed methods study and consists of a quantitative web-based survey followed by a qualitative phase based on semistructured individual interviews and focus groups. People with long COVID and HCPs in charge of patients with long COVID will be invited to participate in this fully web-based study. The quantitative data collected from the survey will be analyzed using descriptive statistics. Qualitative data from the individual interviews and the focus groups will be transcribed and analyzed using a thematic analysis approach.

RESULTS

The study was approved by the National Research Ethics Committee of Luxembourg (number 202208/04) in August 2022 and started in October 2022 with the launch of the web-based survey. Data collection will be completed in September 2023, and the results will be published in 2024.

CONCLUSIONS

This mixed methods study will identify the needs of people affected by long COVID in their daily lives and describe the main symptoms or problems that would need to be monitored and improved. We will determine how using voice and vocal biomarkers could meet these needs and codevelop a tailored voice-based digital health solution with its future end users. This project will contribute to improving the quality of life and care of people with long COVID. The potential transferability to other diseases will be explored, which will contribute to the deployment of vocal biomarkers in general.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05546918; https://clinicaltrials.gov/ct2/show/NCT05546918.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/46103.

Commentary: Patient Perspectives on Artificial Intelligence; What have We Learned and How Should We Move Forward?

Artificial intelligence (AI) in healthcare has now begun to make its contributions to real-world patient care with varying degrees of both public and clinical acceptability around it. The heavy investment from governments, industry and academia needed to reach this point has helped to surface different perspectives on AI. As clinical AI applications become a reality, however, there is an increasing need to harness and integrate patient perspectives, which address the distinct needs of different populations, healthcare systems and clinical problems more closely. Despite this need, patient perspectives on AI implementation have little presence in academic literature and within implementation science and are not sufficiently considered throughout the MedTech and eHealthtech product development cycle, which brings its own challenges and opportunities. This joint patient expert/clinician commentary aims to briefly summarise views on AI. It reflects upon recommendations on how stakeholders such as clinicians and Health & MedTech small and medium-sized enterprises (SMEs) can make practical usage of these views. The recommendations of the authors centre around how to work better with patients to enable both product centric and patient centric innovation and person-centred care.

Automated detection of progressive speech changes in early Alzheimer's disease

Speech and language changes occur in Alzheimer's disease (AD), but few studies have characterized their longitudinal course. We analyzed open-ended speech samples from a prodromal-to-mild AD cohort to develop a novel composite score to characterize progressive speech changes. Participant speech from the Clinical Dementia Rating (CDR) interview was analyzed to compute metrics reflecting speech and language characteristics. We determined the aspects of speech and language that exhibited significant longitudinal change over 18 months. Nine acoustic and linguistic measures were combined to create a novel composite score. The speech composite exhibited significant correlations with primary and secondary clinical endpoints and a similar effect size for detecting longitudinal change. Our results demonstrate the feasibility of using automated speech processing to characterize longitudinal change in early AD. Speech-based composite scores could be used to monitor change and detect response to treatment in future research.

HIGHLIGHTS

Longitudinal speech samples were analyzed to characterize speech changes in early AD.Acoustic and linguistic measures showed significant change over 18 months.A novel speech composite score was computed to characterize longitudinal change.The speech composite correlated with primary and secondary trial endpoints.Automated speech analysis could facilitate remote, high frequency monitoring in AD.

Listening to the patients' voice: a conceptual framework of the walking experience

BACKGROUND

walking is crucial for an active and healthy ageing, but the perspectives of individuals living with walking impairment are still poorly understood.

OBJECTIVES

to identify and synthesise evidence describing walking as experienced by adults living with mobility-impairing health conditions and to propose an empirical conceptual framework of walking experience.

METHODS

we performed a systematic review and meta-ethnography of qualitative evidence, searching seven electronic databases for records that explored personal experiences of walking in individuals living with conditions of diverse aetiology. Conditions included Parkinson's disease, multiple sclerosis, chronic obstructive pulmonary disease, hip fracture, heart failure, frailty and sarcopenia. Data were extracted, critically appraised using the NICE quality checklist and synthesised using standardised best practices.

RESULTS

from 2,552 unique records, 117 were eligible. Walking experience was similar across conditions and described by seven themes: (i) becoming aware of the personal walking experience, (ii) the walking experience as a link between individuals' activities and sense of self, (iii) the physical walking experience, (iv) the mental and emotional walking experience, (v) the social walking experience, (vi) the context of the walking experience and (vii) behavioural and attitudinal adaptations resulting from the walking experience. We propose a novel conceptual framework that visually represents the walking experience, informed by the interplay between these themes.

CONCLUSION

a multi-faceted and dynamic experience of walking was common across health conditions. Our conceptual framework of the walking experience provides a novel theoretical structure for patient-centred clinical practice, research and public health.

Clinical Validation of Novel Digital Measures: Statistical Methods for Reliability Evaluation

Background

Assessment of reliability is one of the key components of the validation process designed to demonstrate that a novel clinical measure assessed by a digital health technology tool is fit-for-purpose in clinical research, care, and decision-making. Reliability assessment contributes to characterization of the signal-to-noise ratio and measurement error and is the first indicator of potential usefulness of the proposed clinical measure.

Summary

Methodologies for reliability analyses are scattered across literature on validation of PROs, wet biomarkers, etc., yet are equally useful for digital clinical measures. We review a general modeling framework and statistical metrics typically used for reliability assessments as part of the clinical validation. We also present methods for the assessment of agreement and measurement error, alongside modified approaches for categorical measures. We illustrate the discussed techniques using physical activity data from a wearable device with an accelerometer sensor collected in clinical trial participants.

Key Messages

This paper provides statisticians and data scientists, involved in development and validation of novel digital clinical measures, an overview of the statistical methodologies and analytical tools for reliability assessment.

Capturing Measures That Matter: The Potential Value of Digital Measures of Physical Behavior for Alzheimer's Disease Drug Development

Alzheimer's disease (AD) is a devastating neurodegenerative disease and the primary cause of dementia worldwide. Despite the magnitude of AD's impact on patients, caregivers, and society, nearly all AD clinical trials fail. A potential contributor to this high rate of failure is that established clinical outcome assessments fail to capture subtle clinical changes, entail high burden for patients and their caregivers, and ineffectively address the aspects of health deemed important by patients and their caregivers. AD progression is associated with widespread changes in physical behavior that have impacts on the ability to function independently, which is a meaningful aspect of health for patients with AD and important for diagnosis. However, established assessments of functional independence remain underutilized in AD clinical trials and are limited by subjective biases and ceiling effects. Digital measures of real-world physical behavior assessed passively, continuously, and remotely using digital health technologies have the potential to address some of these limitations and to capture aspects of functional independence in patients with AD. In particular, measures of real-world gait, physical activity, and life-space mobility captured with wearable sensors may offer value. Additional research is needed to understand the validity, feasibility, and acceptability of these measures in AD clinical research.

Patient Willingness to Use Digital Health Technologies: A Quantitative and Qualitative Survey in Patients with Cancer Cachexia

Purpose

The objective of this study was to gain insights into the patients' perspectives on the impact of cancer cachexia on physical activity and their willingness to wear digital health technology (DHT) devices in clinical trials.

Patients and Methods

We administered a quantitative 20-minute online survey on aspects of physical activity (on a 0-100 scale) to 50 patients with cancer cachexia recruited through Rare Patient Voice, LLC. A subset of 10 patients took part in qualitative 45-minute web-based interviews with a demonstration of DHT devices. Survey questions related to the impact of weight loss (a key characteristic in Fearon's cachexia definition) on physical activity, patients' expectations regarding desired improvements and their level of meaningful activities, as well as preferences for DHT.

Results

Seventy-eight percent of patients reported that their physical activity was impacted by cachexia, and for 77% of them, such impact was consistent over time. Patients perceived most impact of weight loss on walking distance, time and speed, and on level of activity during the day. Sleep, activity level, walking quality and distance were identified as the most meaningful activities to improve. Patients would like to see a moderate improvement of activity levels and consider it meaningful to perform physical activity of moderate intensity (eg, walk at normal pace) on a regular basis. The wrist was the preferred location for wearing a DHT device, followed by arm, ankle, and waist.

Conclusion

Most patients reported physical activity limitations since the occurrence of weight loss compatible with cancer-associated cachexia. Walking distance, sleep and quality of walk were the most meaningful activities to moderately improve, and patients consider moderate physical activity as meaningful. Finally, this study population found the proposed wear of DHT devices on the wrist and around the waist acceptable for the duration of clinical studies.

Reverse Engineering of Digital Measures: Inviting Patients to the Conversation

Background

Digital measures offer an unparalleled opportunity to create a more holistic picture of how people who are patients behave in their real-world environments, thereby establishing a better connection between patients, caregivers, and the clinical evidence used to drive drug development and disease management. Reaching this vision will require achieving a new level of co-creation between the stakeholders who design, develop, use, and make decisions using evidence from digital measures.

Summary

In September 2022, the second in a series of meetings hosted by the Swiss Federal Institute of Technology in Zürich, the Foundation for the National Institutes of Health Biomarkers Consortium, and sponsored by Wellcome Trust, entitled "Reverse Engineering of Digital Measures," was held in Zurich, Switzerland, with a broad range of stakeholders sharing their experience across four case studies to examine how patient centricity is essential in shaping development and validation of digital evidence generation tools.

Key Messages

In this paper, we discuss progress and the remaining barriers to widespread use of digital measures for evidence generation in clinical development and care delivery. We also present key discussion points and takeaways in order to continue discourse and provide a basis for dissemination and outreach to the wider community and other stakeholders. The work presented here shows us a blueprint for how and why the patient voice can be thoughtfully integrated into digital measure development and that continued multistakeholder engagement is critical for further progress.

The Case for the Patient-Centric Development of Novel Digital Sleep Assessment Tools in Major Depressive Disorder

Background

Depression imposes a major burden on public health as the leading cause of disability worldwide. Sleep disturbance is a core symptom of depression that affects the vast majority of patients. Nonetheless, it is frequently not resolved by depression treatment and may even be worsened through some pharmaceutical interventions. Disturbed sleep negatively impact patients' quality of life, and persistent sleep disturbance increases the risk of recurrence, relapse, and even suicide. However, the development of novel treatments that might improve sleep problems is hindered by the lack of reliable low-burden objective measures that can adequately assess disturbed sleep in this population.

Summary

Developing improved digital measurement tools that are fit for use in clinical trials for major depressive disorder could promote the inclusion of sleep as a focus for treatment, clinical drug development, and research. This perspective piece explores the path toward the development of novel digital measures, reviews the existing evidence on the meaningfulness of sleep in depression, and summarizes existing methods of sleep assessments, including the use of digital health technologies.

Key Messages

Our objective was to make a clear call to action and path forward for the qualification of new digital outcome measures which would enable assessment of sleep disturbance as an aspect of health that truly matters to patients, promoting sleep as an important outcome for clinical development, and ultimately ensure that disturbed sleep will not remain the forgotten symptom of depression.

Defining digital measurement of scratching during sleep, or “Nocturnal Scratch”: A review of the literature (Preprint)

BACKGROUND

Digital sensing solutions represent a convenient, objective, relatively inexpensive method that could be leveraged for assessing symptoms of various health conditions. Recent progress in the capabilities of digital sensing products has targeted the measurement of scratching during sleep, traditionally referred to as nocturnal scratching, in patients with atopic dermatitis or other skin conditions. Many solutions measuring nocturnal scratch have been developed; however, a lack of efforts toward standardization of the measure's definition and contextualization of scratching during sleep hampers the ability to compare different technologies for this purpose.

OBJECTIVE

We aimed to address this gap and bring forth unified measurement definitions for nocturnal scratch.

METHODS

We performed a narrative literature review of definitions of scratching in patients with skin inflammation and a targeted literature review of sleep in the context of the period during which such scratching occurred. Both searches were limited to English language studies in humans. The extracted data were synthesized into themes based on study characteristics: scratch as a behavior, other characterization of the scratching movement, and measurement parameters for both scratch and sleep. We then developed ontologies for the digital measurement of sleep scratching.

RESULTS

In all, 29 studies defined inflammation-related scratching between 1996 and 2021. When cross-referenced with the results of search terms describing the sleep period, only 2 of these scratch-related papers also described sleep-related variables. From these search results, we developed an evidence-based and patient-centric definition of nocturnal scratch: an action of rhythmic and repetitive skin contact movement performed during a delimited time period of intended and actual sleep that is not restricted to any specific time of the day or night. Based on the measurement properties identified in the searches, we developed ontologies of relevant concepts that can be used as a starting point to develop standardized outcome measures of scratching during sleep in patients with inflammatory skin conditions.

CONCLUSIONS

This work is intended to serve as a foundation for the future development of unified and well-described digital health technologies measuring nocturnal scratching and should enable better communication and sharing of results between various stakeholders taking part in research in atopic dermatitis and other inflammatory skin conditions.

On the use of wearable sensors as mobility biomarkers in the marketing authorization of new drugs: A regulatory perspective

The loss of mobility is a common trait in multiple health conditions (e.g., Parkinson's disease) and is associated with reduced quality of life. In this context, being able to monitor mobility in the real world, is important. Until recently, the technology was not mature enough for this; but today, miniaturized sensors and novel algorithms promise to monitor mobility accurately and continuously in the real world, also in pathological populations. However, before any such methodology can be employed to support the development and testing of new drugs in clinical trials, they need to be qualified by the competent regulatory agencies (e.g., European Medicines Agency). Nonetheless, to date, only very narrow scoped requests for regulatory qualification were successful. In this work, the Mobilise-D Consortium shares its positive experience with the European regulator, summarizing the two requests for Qualification Advice for the Mobilise-D methodologies submitted in October 2019 and June 2020, as well as the feedback received, which resulted in two Letters of Support publicly available for consultation on the website of the European Medicines Agency. Leveraging on this experience, we hereby propose a refined qualification strategy for the use of digital mobility outcome (DMO) measures as monitoring biomarkers for mobility in drug trials.

Considerations for Analyzing and Interpreting Data from Biometric Monitoring Technologies in Clinical Trials

BACKGROUND

The proliferation and increasing maturity of biometric monitoring technologies allow clinical investigators to measure the health status of trial participants in a more holistic manner, especially outside of traditional clinical settings. This includes capturing meaningful aspects of health in daily living and a more granular and objective manner compared to traditional tools in clinical settings.

SUMMARY

Within multidisciplinary teams, statisticians and data scientists are increasingly involved in clinical trials that incorporate digital clinical measures. They are called upon to provide input into trial planning, generation of evidence on the clinical validity of novel clinical measures, and evaluation of the adequacy of existing evidence. Analysis objectives related to demonstrating clinical validity of novel clinical measures differ from typical objectives related to demonstrating safety and efficacy of therapeutic interventions using established measures which statisticians are most familiar with.

KEY MESSAGES

This paper discusses key considerations for generating evidence for clinical validity through the lens of the type and intended use of a clinical measure. This paper also briefly discusses the regulatory pathways through which clinical validity evidence may be reviewed and highlights challenges that investigators may encounter while dealing with data from biometric monitoring technologies.

Digital Gait Biomarkers Allow to Capture 1-Year Longitudinal Change in Spinocerebellar Ataxia Type 3

Measures of step variability and body sway during gait have shown to correlate with clinical ataxia severity in several cross-sectional studies. However, to serve as a valid progression biomarker, these gait measures have to prove their sensitivity to robustly capture longitudinal change, ideally within short time frames (eg, 1 year). We present the first multicenter longitudinal gait analysis study in spinocerebellar ataxias. We performed a combined cross-sectional (n = 28) and longitudinal (1-year interval, n = 17) analysis in Spinocerebellar Ataxia type 3 subjects (including seven preataxic mutation carriers). Longitudinal analysis showed significant change in gait measures between baseline and 1-year follow-up, with high effect sizes (stride length variability: P = 0.01, effect size r_prb  = 0.66; lateral sway: P = 0.007, r_prb  = 0.73). Sample size estimation for lateral sway indicates a required cohort size of n = 43 for detecting a 50% reduction of natural progression, compared with n = 240 for the clinical ataxia score Scale for the Assessment and Rating of Ataxia (SARA). These measures thus present promising motor biomarkers for upcoming interventional studies. © 2022 The Authors. Movement Disorders published by Wiley Periodicals LLC on behalf of International Parkinson and Movement Disorder Society.

Considerations for Conducting Bring Your Own “Device” (BYOD) Clinical Studies

Background

Digital health technologies are attracting attention as novel tools for data collection in clinical research. They present alternative methods compared to in-clinic data collection, which often yields snapshots of the participants' physiology, behavior, and function that may be prone to biases and artifacts, e.g., white coat hypertension, and not representative of the data in free-living conditions. Modern digital health technologies equipped with multi-modal sensors combine different data streams to derive comprehensive endpoints that are important to study participants and are clinically meaningful. Used for data collection in clinical trials, they can be deployed as provisioned products where technology is given at study start or in a bring your own “device” (BYOD) manner where participants use their technologies to generate study data.

Summary

The BYOD option has the potential to be more user-friendly, allowing participants to use technologies that they are familiar with, ensuring better participant compliance, and potentially reducing the bias that comes with introducing new technologies. However, this approach presents different technical, operational, regulatory, and ethical challenges to study teams. For example, BYOD data can be more heterogeneous, and recruiting historically underrepresented populations with limited access to technology and the internet can be challenging. Despite the rapid increase in digital health technologies for clinical and healthcare research, BYOD use in clinical trials is limited, and regulatory guidance is still evolving.

Key Messages

We offer considerations for academic researchers, drug developers, and patient advocacy organizations on the design and deployment of BYOD models in clinical research. These considerations address: (1) early identification and engagement with internal and external stakeholders; (2) study design including informed consent and recruitment strategies; (3) outcome, endpoint, and technology selection; (4) data management including compliance and data monitoring; (5) statistical considerations to meet regulatory requirements. We believe that this article acts as a primer, providing insights into study design and operational requirements to ensure the successful implementation of BYOD clinical studies.

Quantifying the Benefits of Digital Biomarkers and Technology-Based Study Endpoints in Clinical Trials: Project Moneyball

Introduction

Digital biomarkers have significant potential to transform drug development, but only a few have contributed meaningfully to bring new treatments to market. There are uncertainties in how they will generate quantifiable benefits in clinical trial performance and ultimately to the chances of phase 3 success. Here we have proposed a statistical framework and ran a proof-of-concept model with hypothetical digital biomarkers and visualized them in a familiar manner to study power calculation.

Methods

A Monte Carlo simulation for Parkinson's disease (PD) was performed using the Captario SUM® platform and illustrative study technology impact calculations were generated. We took inspiration from the EMA-qualified wearable-derived digital endpoint stride velocity 95^th centile (SV95C) for Duchenne muscular dystrophy, and we imagined a similar measurement for PD would be available in the future. DaTscan enrichment and “SV95C-like” endpoint biomarkers were assumed on a hypothetical disease-modifying drug pivotal trial aiming for an 80% probability of achieving a study p value of less than 0.05.

Results

Four scenarios with different combinations of technologies were illustrated. The model illustrated a way to quantify the magnitude of the contributions that enrichment and endpoint technologies could make to drug development studies.

Discussion/Conclusion

Quantitative models could be valuable not only for the study sponsors but also as an interactive and collaborative engagement tool for technology players and multi-stakeholder consortia. Establishing values of digital biomarkers could also facilitate business cases and financial investments.

A Remote Digital Monitoring Platform to Assess Cognitive and Motor Symptoms in Huntington Disease: Cross-sectional Validation Study

Background

Remote monitoring of Huntington disease (HD) signs and symptoms using digital technologies may enhance early clinical diagnosis and tracking of disease progression, guide treatment decisions, and monitor response to disease-modifying agents. Several recent studies in neurodegenerative diseases have demonstrated the feasibility of digital symptom monitoring.

Objective

The aim of this study was to evaluate a novel smartwatch- and smartphone-based digital monitoring platform to remotely monitor signs and symptoms of HD.

Methods

This analysis aimed to determine the feasibility and reliability of the Roche HD Digital Monitoring Platform over a 4-week period and cross-sectional validity over a 2-week interval. Key criteria assessed were feasibility, evaluated by adherence and quality control failure rates; test-retest reliability; known-groups validity; and convergent validity of sensor-based measures with existing clinical measures. Data from 3 studies were used: the predrug screening phase of an open-label extension study evaluating tominersen (NCT03342053) and 2 untreated cohorts—the HD Natural History Study (NCT03664804) and the Digital-HD study. Across these studies, controls (n=20) and individuals with premanifest (n=20) or manifest (n=179) HD completed 6 motor and 2 cognitive tests at home and in the clinic.

Results

Participants in the open-label extension study, the HD Natural History Study, and the Digital-HD study completed 89.95% (1164/1294), 72.01% (2025/2812), and 68.98% (1454/2108) of the active tests, respectively. All sensor-based features showed good to excellent test-retest reliability (intraclass correlation coefficient 0.89-0.98) and generally low quality control failure rates. Good overall convergent validity of sensor-derived features to Unified HD Rating Scale outcomes and good overall known-groups validity among controls, premanifest, and manifest participants were observed. Among participants with manifest HD, the digital cognitive tests demonstrated the strongest correlations with analogous in-clinic tests (Pearson correlation coefficient 0.79-0.90).

Conclusions

These results show the potential of the HD Digital Monitoring Platform to provide reliable, valid, continuous remote monitoring of HD symptoms, facilitating the evaluation of novel treatments and enhanced clinical monitoring and care for individuals with HD.

Measuring Health-Related Quality of Life With Multimodal Data: Viewpoint

The ability to objectively measure aspects of performance and behavior is a fundamental pillar of digital health, enabling digital wellness products, decentralized trial concepts, evidence generation, digital therapeutics, and more. Emerging multimodal technologies capable of measuring several modalities simultaneously and efforts to integrate inputs across several sources are further expanding the limits of what digital measures can assess. Experts from the field of digital health were convened as part of a multi-stakeholder workshop to examine the progress of multimodal digital measures in two key areas: detection of disease and the measurement of meaningful aspects of health relevant to the quality of life. Here we present a meeting report, summarizing key discussion points, relevant literature, and finally a vision for the immediate future, including how multimodal measures can provide value to stakeholders across drug development and care delivery, as well as three key areas where headway will need to be made if we are to continue to build on the encouraging progress so far: collaboration and data sharing, removal of barriers to data integration, and alignment around robust modular evaluation of new measurement capabilities.

Virtual exam for Parkinson's disease enables frequent and reliable remote measurements of motor function

Sensor-based remote monitoring could help better track Parkinson's disease (PD) progression, and measure patients' response to putative disease-modifying therapeutic interventions. To be useful, the remotely-collected measurements should be valid, reliable, and sensitive to change, and people with PD must engage with the technology. We developed a smartwatch-based active assessment that enables unsupervised measurement of motor signs of PD. Participants with early-stage PD (N = 388, 64% men, average age 63) wore a smartwatch for a median of 390 days. Participants performed unsupervised motor tasks both in-clinic (once) and remotely (twice weekly for one year). Dropout rate was 5.4%. Median wear-time was 21.1 h/day, and 59% of per-protocol remote assessments were completed. Analytical validation was established for in-clinic measurements, which showed moderate-to-strong correlations with consensus MDS-UPDRS Part III ratings for rest tremor (⍴ = 0.70), bradykinesia (⍴ = -0.62), and gait (⍴ = -0.46). Test-retest reliability of remote measurements, aggregated monthly, was good-to-excellent (ICC = 0.75-0.96). Remote measurements were sensitive to the known effects of dopaminergic medication (on vs off Cohen's d = 0.19-0.54). Of note, in-clinic assessments often did not reflect the patients' typical status at home. This demonstrates the feasibility of smartwatch-based unsupervised active tests, and establishes the analytical validity of associated digital measurements. Weekly measurements provide a real-life distribution of disease severity, as it fluctuates longitudinally. Sensitivity to medication-induced change and improved reliability imply that these methods could help reduce sample sizes needed to demonstrate a response to therapeutic interventions or disease progression.