The Future of Clinical Trial Design: The Transition from Hard Endpoints to Value-Based Endpoints

Patient engagement as a core element of translating clinical evidence into practice- application of the COM-B model behaviour change model

BACKGROUND

The results of rehabilitation trials are often not fully attained when the intervention is implemented beyond the initial trial. One of the key reasons is that a patients' ability and/or capacity to take part in their own healthcare is not considered in the trial design yet has significant impact on the outcomes during the implementation phase.

BODY OF TEXT

We propose a shift from a therapist-focus to patient-focus in trial design, through addressing patient engagement as a core consideration in trials. We argue that engaging patients in any rehabilitation program is a process of behavioural change. Exercise prescription is used as an example to illustrate how the Behaviour Change Wheel can be applied to analyse barriers and facilitators associated with patients' capabilities, opportunities and motivations in integrating trial interventions into their daily life. We propose a framework to assist in this shift.

CONCLUSION

A core part of implementing rehabilitation interventions at the primary care level requires patient engagement. Related aspects of interventions should be identified and assessed using the COM-B model at the outset of trial design to ensure that the results are realistic, meaningful and transferable, so as to enable real impact.

Question-based development of high-risk medical devices: A proposal for a structured design and review process

INTRODUCTION

The recent introduction of the European Medical Device Regulation poses stricter legislation for manufacturers developing medical devices in the EU. Many devices have been placed into a higher risk category, thus requiring more data before market approval, and a much larger focus has been placed on safety. For implantable and Class III devices, the highest risk class, clinical evidence is a necessity. However, the requirements of clinical study design and developmental outcomes are only described in general terms due to the diversity of devices.

METHODS

A structured approach to determining the requirements for the clinical development of high-risk medical devices is introduced, utilizing the question-based development framework, which is already used for pharmaceutical drug development. An example of a novel implantable device for haemodialysis demonstrates how to set up a relevant target product profile defining the device requirements and criteria. The framework can be used in the medical device design phase to define specific questions to be answered during the ensuing clinical development, based upon five general questions, specified by the question-based framework.

RESULTS

The result is a clear and evaluable overview of requirements and methodologies to verify and track these requirements in the clinical development phase. Development organizations will be guided to the optimal route, also to abandon projects destined for failure early on to minimize development risks.

CONCLUSION

The framework could facilitate communication with funding agencies, regulators and clinicians, while highlighting remaining 'known unknowns' that require answering in the post-market phase after sufficient benefit is established relative to the risks.

Development and technical validation of a smartphone-based pediatric cough detection algorithm

INTRODUCTION

Coughing is a common symptom in pediatric lung disease and cough frequency has been shown to be correlated to disease activity in several conditions. Automated cough detection could provide a noninvasive digital biomarker for pediatric clinical trials or care. The aim of this study was to develop a smartphone-based algorithm that objectively and automatically counts cough sounds of children.

METHODS

The training set was composed of 3228 pediatric cough sounds and 480,780 noncough sounds from various publicly available sources and continuous sound recordings of 7 patients admitted due to respiratory disease. A Gradient Boost Classifier was fitted on the training data, which was subsequently validated on recordings from 14 additional patients aged 0-14 admitted to the pediatric ward due to respiratory disease. The robustness of the algorithm was investigated by repeatedly classifying a recording with the smartphone-based algorithm during various conditions.

RESULTS

The final algorithm obtained an accuracy of 99.7%, sensitivity of 47.6%, specificity of 99.96%, positive predictive value of 82.2% and negative predictive value 99.8% in the validation dataset. The correlation coefficient between manual- and automated cough counts in the validation dataset was 0.97 (p < .001). The intra- and interdevice reliability of the algorithm was adequate, and the algorithm performed best at an unobstructed distance of 0.5-1 m from the audio source.

CONCLUSION

This novel smartphone-based pediatric cough detection application can be used for longitudinal follow-up in clinical care or as digital endpoint in clinical trials.

Clinical validation of digital biomarkers for pediatric patients with asthma and cystic fibrosis - Potential for clinical trials and clinical care

BACKGROUND

Digital biomarkers are a promising novel method to capture clinical data in a home-setting. However, clinical validation prior to implementation is of vital importance. The aim of this study was to clinically validate physical activity, heart rate, sleep and FEV1 as digital biomarkers measured by a smartwatch and portable spirometer in children with asthma and cystic fibrosis (CF).

METHODS

This was a prospective cohort study including 60 children with asthma and 30 children with CF (age 6-16). Participants wore a smartwatch, performed daily spirometry at home and completed a daily symptom questionnaire for 28-days. Physical activity, heart rate, sleep and FEV1 were considered candidate digital endpoints. Data from 128 healthy children was used for comparison. Reported outcomes were compliance, difference between patients and controls, correlation with disease-activity and potential to detect clinical events. Analysis was performed with linear mixed effect models.

RESULTS

Median compliance was 88%. On average, patients exhibited lower physical activity and FEV1 compared to healthy children, whereas the heart rate of children with asthma was higher compared to healthy children. Days with a higher symptom score were associated with lower physical activity for children with uncontrolled asthma and CF. Furthermore, FEV1 was lower and (nocturnal) heart rate was higher for both patient groups on days with more symptoms. Candidate biomarkers and showed a distinct pattern before- and after a pulmonary exacerbation.

CONCLUSION

Portable spirometer- and smartwatch-derived digital biomarkers show promise as candidate endpoints for use in clinical trials or clinical care in pediatric lung disease.

Theoretical Performance of Nonlinear Mixed-Effect Models Incorporating Saliva as an Alternative Sampling Matrix for Therapeutic Drug Monitoring in Pediatrics: A Simulation Study

BACKGROUND

Historically, pharmacokinetic (PK) studies and therapeutic drug monitoring (TDM) have relied on plasma as a sampling matrix. Noninvasive sampling matrices, such as saliva, can reduce the burden on pediatric patients. The variable plasma-saliva relationship can be quantified using population PK models (nonlinear mixed-effect models). However, criteria regarding acceptable levels of variability in such models remain unclear. In this simulation study, the authors aimed to propose a saliva TDM evaluation framework and evaluate model requirements in the context of TDM, with gentamicin and lamotrigine as model compounds.

METHODS

Two population pharmacokinetic models for gentamicin in neonates and lamotrigine in pediatrics were extended with a saliva compartment including a delay constant (kSALIVA), a saliva:plasma ratio, and between-subject variability (BSV) on both parameters. Subjects were simulated using a realistic covariate distribution. Bayesian maximum a posteriori TDM was applied to assess the performance of an increasing number of TDM saliva samples and varying levels of BSV and residual variability. Saliva TDM performance was compared with plasma TDM performance. The framework was applied to a known voriconazole saliva model as a case study.

RESULTS

TDM performed using saliva resulted in higher target attainment than no TDM, and a residual proportional error <25% on saliva observations led to saliva TDM performance comparable with plasma TDM. BSV on kSALIVA did not affect performance, whereas increasing BSV on saliva:plasma ratios by >25% for gentamicin and >50% for lamotrigine reduced performance. The simulated target attainment for voriconazole saliva TDM was >90%.

CONCLUSIONS

Saliva as an alternative matrix for noninvasive TDM is possible using nonlinear mixed-effect models combined with Bayesian optimization. This article provides a workflow to explore TDM performance for compounds measured in saliva and can be used for evaluation during model building.

Accelerating vaccine trial conduct in a pandemic with a hot spot-based inclusion strategy using trial and epidemic simulation

Clinical development of vaccines in a pandemic situation should be rigorous but expedited to tackle the pandemic threat as fast as possible. We explored the effects of a novel vaccine trial strategy that actively identifies and enrolls subjects in local areas with high infection rates. In addition, we assessed the practical requirements needed for such a strategy. Clinical trial simulations were used to assess the effects of utilizing these so‐called “hot spot strategy” compared to a traditional vaccine field trial. We used preset parameters of a pandemic outbreak and incorporated realistic aspects of conducting a trial in a pandemic setting. Our simulations demonstrated that incorporating a hot spot strategy shortened the duration of the vaccine trial considerably, even if only one hot spot was identified during the clinical trial. The active hot spot strategy described in this paper has clear advantages compared to a “wait‐and‐see” approach that is used in traditional vaccine efficacy trials. Completion of a clinical trial can be expedited by adapting to resurgences and outbreaks that will occur in a population during a pandemic. However, this approach requires a speed of response that is unusual for a traditional phase III clinical trial. Therefore, several recommendations are made to help accomplish rapid clinical trial setup in areas identified as local outbreaks. The described model and hot spot vaccination strategy can be adjusted to disease‐specific transmission characteristics and could therefore be applied to any future pandemic threat.

Stakeholder Perspectives on Barriers and Facilitators for the Adoption of Virtual Clinical Trials: Qualitative Study

Background

Conventional clinical trials are essential for generating high-quality evidence by measuring the efficacy of interventions in rigorously controlled clinical environments. However, their execution can be expensive and time-consuming. In addition, clinical trials face several logistical challenges regarding the identification, recruitment, and retention of participants; consistent data collection during trials; and adequate patient follow-up. This might lead to inefficient resource utilization. In order to partially address the current problems with conventional clinical trials, there exists the need for innovations. One such innovation is the virtual clinical trial (VCT). VCTs allow for the collection and integration of diverse data from multiple information sources, such as electronic health records, clinical and demographic data, patient-reported outcomes, anthropometric and activity measurements, and data collected by digital biomarkers or (small) samples that participants can collect themselves. Although VCTs have the potential to provide substantial value to clinical research and patients because they can lower clinical trial costs, increase the volume of data collected from patients’ daily environment, and reduce the burden of patient participation, so far VCT adoption is not commonplace.

Objective

This paper aims to better understand the barriers and facilitators to VCT adoption by determining the factors that influence individuals’ considerations regarding VCTs from the perspective of various stakeholders.

Methods

Based on online semistructured interviews, a qualitative study was conducted with pharmaceutical companies, food and health organizations, and an applied research organization in Europe. Data were thematically analyzed using Rogers’ diffusion of innovation theory.

Results

A total of 16 individuals with interest and experience in VCTs were interviewed, including persons from pharmaceutical companies (n=6), food and health organizations (n=4), and a research organization (n=6). Key barriers included a potentially low degree of acceptance by regulatory authorities, technical issues (standardization, validation, and data storage), compliance and adherence, and lack of knowledge or comprehension regarding the opportunities VCTs have to offer. Involvement of regulators in development processes, stakeholder exposure to the results of pilot studies, and clear and simple instructions and assistance for patients were considered key facilitators.

Conclusions

Collaboration among all stakeholders in VCT development is crucial to increase knowledge and awareness. Organizations should invest in accurate data collection technologies, and compliance of patients in VCTs needs to be ensured. Multicriteria decision analysis can help determine if a VCT is a preferred option by stakeholders. The findings of this study can be a good starting point to accelerate the development and widespread implementation of VCTs.

Postdischarge Recovery after Acute Pediatric Lung Disease Can Be Quantified with Digital Biomarkers

BACKGROUND

Pediatric patients admitted for acute lung disease are treated and monitored in the hospital, after which full recovery is achieved at home. Many studies report in-hospital recovery, but little is known regarding the time to full recovery after hospital discharge. Technological innovations have led to increased interest in home-monitoring and digital biomarkers. The aim of this study was to describe at-home recovery of 3 common pediatric respiratory diseases using a questionnaire and wearable device.

METHODS

In this study, patients admitted due to pneumonia (n = 30), preschool wheezing (n = 30), and asthma exacerbation (AE; n = 11) were included. Patients were monitored with a smartwatch and a questionnaire during admission, with a 14-day recovery period and a 10-day "healthy" period. Median compliance was calculated, and a mixed-effects model was fitted for physical activity and heart rate (HR) to describe the recovery period, and the physical activity recovery trajectory was correlated to respiratory symptom scores.

RESULTS

Median compliance was 47% (interquartile range [IQR] 33-81%) during the entire study period, 68% (IQR 54-91%) during the recovery period, and 28% (IQR 0-74%) during the healthy period. Patients with pneumonia reached normal physical activity 12 days postdischarge, while subjects with wheezing and AE reached this level after 5 and 6 days, respectively. Estimated mean physical activity was closely correlated with the estimated mean symptom score. HR measured by the smartwatch showed a similar recovery trajectory for subjects with wheezing and asthma, but not for subjects with pneumonia.

CONCLUSIONS

The digital biomarkers, physical activity, and HR obtained via smartwatch show promise for quantifying postdischarge recovery in a noninvasive manner, which can be useful in pediatric clinical trials and clinical care.

Development of Novel, Value-Based, Digital Endpoints for Clinical Trials: A Structured Approach Toward Fit-for-Purpose Validation

Novel digital endpoints gathered via wearables, small devices, or algorithms hold great promise for clinical trials. However, implementation has been slow because of a lack of guidelines regarding the validation process of these new measurements. In this paper, we propose a pragmatic approach toward selection and fit-for-purpose validation of digital endpoints. Measurements should be value-based, meaning the measurements should directly measure or be associated with meaningful outcomes for patients. Devices should be assessed regarding technological validity. Most importantly, a rigorous clinical validation process should appraise the tolerability, difference between patients and controls, repeatability, detection of clinical events, and correlation with traditional endpoints. When technically and clinically fit-for-purpose, case building in interventional clinical trials starts to generate evidence regarding the response to new or existing health-care interventions. This process may lead to the digital endpoint replacing traditional endpoints, such as clinical rating scales or questionnaires in clinical trials. We recommend initiating more data-sharing collaborations to prevent unnecessary duplication of research and integration of value-based measurements in clinical care to enhance acceptance by health-care professionals. Finally, we invite researchers and regulators to adopt this approach to ensure a timely implementation of digital measurements and value-based thinking in clinical trial design and health care. SIGNIFICANCE STATEMENT: Novel digital endpoints are often cited as promising for the clinical trial of the future. However, clear validation guidelines are lacking in the literature. This paper contains pragmatic criteria for the selection, technical validation, and clinical validation of novel digital endpoints and provides recommendations for future work and collaboration.

Towards remote monitoring in pediatric care and clinical trials-Tolerability, repeatability and reference values of candidate digital endpoints derived from physical activity, heart rate and sleep in healthy children

Background

Digital devices and wearables allow for the measurement of a wide range of health-related parameters in a non-invasive manner, which may be particularly valuable in pediatrics. Incorporation of such parameters in clinical trials or care as digital endpoint could reduce the burden for children and their parents but requires clinical validation in the target population. This study aims to determine the tolerability, repeatability, and reference values of novel digital endpoints in healthy children.

Methods

Apparently healthy children (n = 175, 46% male) aged 2–16 were included. Subjects were monitored for 21 days using a home-monitoring platform with several devices (smartwatch, spirometer, thermometer, blood pressure monitor, scales). Endpoints were analyzed with a mixed effects model, assessing variables that explained within- and between-subject variability. Endpoints based on physical activity, heart rate, and sleep-related parameters were included in the analysis. For physical-activity-related endpoints, a sample size needed to detect a 15% increase was calculated.

Findings

Median compliance was 94%. Variability in each physical activity-related candidate endpoint was explained by age, sex, watch wear time, rain duration per day, average ambient temperature, and population density of the city of residence. Estimated sample sizes for candidate endpoints ranged from 33–110 per group. Daytime heart rate, nocturnal heart rate and sleep duration decreased as a function of age and were comparable to reference values published in the literature.

Conclusions

Wearable- and portable devices are tolerable for pediatric subjects. The raw data, models and reference values presented here can be used to guide further validation and, in the future, clinical trial designs involving the included measures.

Technical validity and usability of a novel smartphone-connected spirometry device for pediatric patients with asthma and cystic fibrosis

BACKGROUND

Diagnosis and follow-up of respiratory diseases traditionally rely on pulmonary function tests (PFTs), which are currently performed in hospitals and require trained personnel. Smartphone-connected spirometers, like the Air Next spirometer, have been developed to aid in the home monitoring of patients with pulmonary disease. The aim of this study was to investigate the technical validity and usability of the Air Next spirometer in pediatric patients.

METHODS

Device variability was tested with a calibrated syringe. About 90 subjects, aged 6 to 16, were included in a prospective cohort study. Fifty-eight subjects performed conventional spirometry and subsequent Air Next spirometry. The bias and the limits of agreement between the measurements were calculated. Furthermore, subjects used the device for 28 days at home and completed a subject-satisfaction questionnaire at the end of the study period.

RESULTS

Interdevice variability was 2.8% and intradevice variability was 0.9%. The average difference between the Air Next and conventional spirometry was 40 mL for forced expiratory volume in 1 second (FEV1) and 3 mL for forced vital capacity (FVC). The limits of agreement were -270 mL and +352 mL for FEV1 and -403 mL and +397 mL for FVC. About 45% of FEV1 measurements and 41% of FVC measurements at home were acceptable and reproducible according to American Thoracic Society/European Respiratory Society criteria. Parents scored difficulty, usefulness, and reliability of the device 1.9, 3.5, and 3.8 out of 5, respectively.

CONCLUSION

The Air Next device shows validity for the measurement of FEV1 and FVC in a pediatric patient population.

Why Are New Drugs Expensive and How Can They Stay Affordable?

Increasing life expectancy leading to a higher median age causes an increasing need for healthcare resources, which is aggravated by an increasing prevalence of preventable diseases such as type 2 diabetes. This includes increasing expenditures for medicines, although these increases when expressed as a share of overall societal wealth are more moderate than often claimed. An increasing use of generic medicines (currently about 90% of all prescriptions) means that costs for discovery and development of innovative drugs must be recovered on a shrinking percentage of prescriptions. However, the key challenge to affordable drugs is exponentially increasing costs to bring a new medicine to the market, which in turn are largely driven by an about 90% attrition rate after start of clinical development. While many factors will be required in concert to keep innovative medicines affordable, reducing attrition appears to be the factor with the greatest potential to contain escalating drug development costs and thereby medication expenditures.