Sample size calculations in pediatric clinical trials conducted in an ICU: a systematic review

A Decade of Efforts to Add Value to Child Health Research Practices

OBJECTIVE

To identify practices that add value to improve the design, conduct, and reporting of child health research and reduce research waste.

STUDY DESIGN

In order to categorize the contributions of members of Standards for Research (StaR) in Child Health network, we developed a novel Child Health Improving Research Practices (CHIRP) framework comprised of 5 domains meant to counteract avoidable child health research waste and improve quality: 1) address research questions relevant to children, their families, clinicians, and researchers; 2) apply appropriate research design, conduct and analysis; 3) ensure efficient research oversight and regulation; 4) Provide accessible research protocols and reports; and 5) develop unbiased and usable research reports, including 17 responsible research practice recommendations. All child health research relevant publications by the 48 original StaR standards' authors over the last decade were identified, and main topic areas were categorized using this framework.

RESULTS

A total of 247 publications were included in the final sample: 100 publications (41%) in domain 1 (3 recommendations), 77 publications (31%) in domain 2 (3), 35 publications (14%) in domain 3 (4), 20 publications (8%) in domain 4 (4), and 15 publications (6%) in domain 5 (3). We identified readily implementable "responsible" research practices to counter child health research waste and improve quality, especially in the areas of patients and families' engagement throughout the research process, developing Core Outcome Sets, and addressing ethics and regulatory oversight issues.

CONCLUSION

While most of the practices are readily implementable, increased awareness of methodological issues and wider guideline uptake is needed to improve child health research. The CHIRP Framework can be used to guide responsible research practices that add value to child health research.

Guidelines for reporting pediatric and child health clinical trial protocols and reports: study protocol for SPIRIT-Children and CONSORT-Children

BACKGROUND

Despite the critical importance of clinical trials to provide evidence about the effects of intervention for children and youth, a paucity of published high-quality pediatric clinical trials persists. Sub-optimal reporting of key trial elements necessary to critically appraise and synthesize findings is prevalent. To harmonize and provide guidance for reporting in pediatric controlled clinical trial protocols and reports, reporting guideline extensions to the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) and Consolidated Standards of Reporting Trials (CONSORT) guidelines specific to pediatrics are being developed: SPIRIT-Children (SPIRIT-C) and CONSORT-Children (CONSORT-C).

METHODS

The development of SPIRIT-C/CONSORT-C will be informed by the Enhancing the Quality and Transparency of Health Research Quality (EQUATOR) method for reporting guideline development in the following stages: (1) generation of a preliminary list of candidate items, informed by (a) items developed during initial development efforts and child relevant items from recent published SPIRIT and CONSORT extensions; (b) two systematic reviews and environmental scan of the literature; (c) workshops with young people; (2) an international Delphi study, where a wide range of panelists will vote on the inclusion or exclusion of candidate items on a nine-point Likert scale; (3) a consensus meeting to discuss items that have not reached consensus in the Delphi study and to "lock" the checklist items; (4) pilot testing of items and definitions to ensure that they are understandable, useful, and applicable; and (5) a final project meeting to discuss each item in the context of pilot test results. Key partners, including young people (ages 12-24 years) and family caregivers (e.g., parents) with lived experiences with pediatric clinical trials, and individuals with expertise and involvement in pediatric trials will be involved throughout the project. SPIRIT-C/CONSORT-C will be disseminated through publications, academic conferences, and endorsement by pediatric journals and relevant research networks and organizations.

DISCUSSION

SPIRIT/CONSORT-C may serve as resources to facilitate comprehensive reporting needed to understand pediatric clinical trial protocols and reports, which may improve transparency within pediatric clinical trials and reduce research waste.

TRIAL REGISTRATION

The development of these reporting guidelines is registered with the EQUATOR Network: SPIRIT-Children ( https://www.equator-network.org/library/reporting-guidelines-under-development/reporting-guidelines-under-development-for-clinical-trials-protocols/#35 ) and CONSORT-Children ( https://www.equator-network.org/library/reporting-guidelines-under-development/reporting-guidelines-under-development-for-clinical-trials/#CHILD ).

Systematic review of sample size calculations and reporting in randomized controlled trials in ophthalmology over a 20-year period

PURPOSE

Randomized Controlled Trials (RCTs) are considered the gold standard for the practice of evidence-based medicine. The purpose of this study is to systematically assess the reporting of sample size calculations in ophthalmology RCTs in 5 leading journals over a 20-year period. Reviewing sample size calculations in ophthalmology RCTs will shed light on the methodological quality of RCTs and, by extension, on the validity of published results.

METHODS

The MEDLINE database was searched to identify full reports of RCTs in the journals Ophthalmology, JAMA Ophthalmology, American Journal of Ophthalmology, Investigative Ophthalmology and Visual Science, and British Journal of Ophthalmology between January and December of the years 2000, 2010 and 2020. Screening identified 559 articles out of which 289 met the inclusion criteria for this systematic review. Data regarding sample size calculation reporting and trial characteristics was extracted for each trial by independent investigators.

RESULTS

In 2020, 77.9% of the RCTs reported sample size calculations as compared with 37% in 2000 (p < 0.001) and 60.7% in 2010 (p = 0.012). Studies reporting all necessary parameters for sample size recalculation increased significantly from 17.2% in 2000 to 39.3% in 2010 and 43.0% in 2020 (p < 0.001). Reporting of funding was greater in 2020 (98.8%) compared with 2010 (89.3%) and 2000 (53.1%). Registration in a clinical trials database occurred more frequently in 2020 (94.2%) compared to 2000 (1.2%; p < 0.001) and 2010 (68%; p < 0.001). In 2020, 38.4% of studies reported different sample sizes in the online registry from the published article. Overall, the most studied area in 2000 was glaucoma (29.6% of RCTs), whereas in 2010 and 2020, it was retina (40.2 and 37.2% of the RCTs, respectively). The number of patients enrolled in a study and the number of eyes studied was significantly greater in 2020 compared to 2000 and 2010 (p < 0.001).

CONCLUSION

Sample size calculation reporting in ophthalmology RCTs has improved significantly between the years 2000 and 2020 and is comparable to other fields in medicine. However, reporting of certain parameters remains inconsistent with current publication guidelines.

Remodeling an existing rare disease registry to be used in regulatory context: Lessons learned and recommendations

Disease registries have been used as an interesting source of real-world data for supporting regulatory decision-making. In fact, drug studies based on registries cover pre-approval investigation, registry randomized clinical trials, and post-authorization studies. This opportunity has been investigated particularly for rare diseases—conditions affecting a small number of individuals worldwide—that represent a peculiar scenario. Several guidelines, concepts, suggestions, and laws are already available to support the design or improvement of a rare disease registry, opening the way for implementation of a registry capable of managing regulatory purposes. The present study aims to highlight the key stages performed for remodeling the existing Registry of Multiple Osteochondromas—REM into a tool consistent with EMA observations and recommendations, as well as to lead the readers through the entire adapting, remodeling, and optimizing process. The process included a variety of procedures that can be summarized into three closely related categories: semantic interoperability, data quality, and governance. At first, we strengthened interoperability within the REM registry by integrating ontologies and standards for proper data collection, in accordance with FAIR principles. Second, to increase data quality, we added additional parameters and domains and double-checked to limit human error to a bare minimum. Finally, we established two-level governance that has increased the visibility for the scientific community and for patients and carers. In conclusion, our remodeled REM registry fits with most of the scientific community’s needs and indications, as well as the best techniques for providing real-world evidence for regulatory aspects.

Challenging the Established Order: Innovating Clinical Trials for Amyotrophic Lateral Sclerosis

Development of effective treatments for amyotrophic lateral sclerosis (ALS) has been hampered by disease heterogeneity, a limited understanding of underlying pathophysiology, and methodologic design challenges. We have evaluated 2 major themes in the design of pivotal, phase 3 clinical trials for ALS—(1) patient selection and (2) analytical strategy—and discussed potential solutions with the European Medicines Agency. Several design considerations were assessed using data from 5 placebo-controlled clinical trials (n = 988), 4 population-based cohorts (n = 5,100), and 2,436 placebo-allocated patients from the Pooled Resource Open-Access ALS Clinical Trials (PRO-ACT) database. The validity of each proposed design modification was confirmed by means of simulation and illustrated for a hypothetical setting. Compared to classical trial design, the proposed design modifications reduce the sample size by 30.5% and placebo exposure time by 35.4%. By making use of prognostic survival models, one creates a potential to include a larger proportion of the population and maximize generalizability. We propose a flexible design framework that naturally adapts the trial duration when inaccurate assumptions are made at the design stage, such as enrollment or survival rate. In case of futility, the follow-up time is shortened and patient exposure to ineffective treatments or placebo is minimized. For diseases such as ALS, optimizing the use of resources, widening eligibility criteria, and minimizing exposure to futile treatments and placebo is critical to the development of effective treatments. Our proposed design modifications could circumvent important pitfalls and may serve as a blueprint for future clinical trials in this population.

Use of Composite Outcomes in Neonatal Trials: An Analysis of the Cochrane Reviews

INTRODUCTION

Composite outcomes are used to increase the power of a study by combining event rates. Many composite outcomes in adult clinical trials have components that differ substantially in patient importance, event rate, and effect size, making interpretation challenging. Little is known about the use of composite outcomes in neonatal randomized controlled trials (RCTs).

METHODS

We assessed the use of composite outcomes in neonatal RCTs included in Cochrane Neonatal reviews published till November 2017. Two authors reviewed the components of the composite outcomes to compare their patient importance and computed the ratios of effect sizes and event rates between the components, with an a priori threshold of 1.5, indicating a substantial difference. Descriptive statistics were presented.

RESULTS

We extracted 7,766 outcomes in 2,134 RCTs in 312 systematic reviews. Among them, 55 composite outcomes (0.7%) were identified in 46 RCTs. The vast majority (92.7%) of composite outcomes had 2 components, with death being the most common component (included 51 times [92.7%]). The components in nearly three-quarters of the composite outcomes (n = 40 [72.7%]) had different patient importance, while the effect sizes and event rates differed substantially between the components in 27 (49.1%) and 35 (63.6%) outcomes, respectively, with up to 43-fold difference in the event rates observed.

CONCLUSIONS

The majority of composite outcomes in neonatal RCTs had different patient importance with contrasting effect sizes and event rates between the components. In patient communication, clinicians should highlight individual components, rather than the composites, with explanation on the relationship between the components, to avoid misleading impression on the effect of the intervention. Future trials should report the estimates of all individual components alongside the composite outcomes presented.

Critical design considerations for time-to-event endpoints in amyotrophic lateral sclerosis clinical trials

BACKGROUND

Funding and resources for low prevalent neurodegenerative disorders such as amyotrophic lateral sclerosis (ALS) are limited, and optimising their use is vital for efficient drug development. In this study, we review the design assumptions for pivotal ALS clinical trials with time-to-event endpoints and provide optimised settings for future trials.

METHODS

We extracted design settings from 13 completed placebo-controlled trials. Optimal assumptions were estimated using parametric survival models in individual participant data (n=4991). Designs were compared in terms of sample size, trial duration, drug use and costs.

RESULTS

Previous trials overestimated the hazard rate by 18.9% (95% CI 3.4% to 34.5%, p=0.021). The median expected HR was 0.56 (range 0.33-0.66). Additionally, we found evidence for an increasing mean hazard rate over time (Weibull shape parameter of 2.03, 95% CI 1.93 to 2.15, p<0.001), which affects the design and planning of future clinical trials. Incorporating accrual time and assuming an increasing hazard rate at the design stage reduced sample size by 33.2% (95% CI 27.9 to 39.4), trial duration by 17.4% (95% CI 11.6 to 23.3), drug use by 14.3% (95% CI 9.6 to 19.0) and follow-up costs by 21.2% (95% CI 15.6 to 26.8).

CONCLUSIONS

Implementing distributional knowledge and incorporating accrual at the design stage could achieve large gains in the efficiency of ALS clinical trials with time-to-event endpoints. We provide an open-source platform that helps investigators to make more accurate sample size calculations and optimise the use of their available resources.

Rare disease registries: potential applications towards impact on development of new drug treatments

BACKGROUND

Low prevalence, lack of knowledge about the disease course, and phenotype heterogeneity hamper the development of drugs for rare diseases. Rare disease registries (RDRs) can be helpful by playing a role in understanding the course of the disease, and providing information necessary for clinical trial design, if designed and maintained properly. We describe the potential applications of a RDR and what type of information should be incorporated to support the design of clinical trials in the process of drug development, based on a broad inventory of registry experience. We evaluated two existing RDRs in more detail to check the completeness of these RDRs for trial design.

RESULTS

Before and during the application for regulatory approval a RDR can improve the efficiency and quality in clinical trial design by informing the sample size calculation and expected disease course. In exceptional circumstances information from RDRs has been used as historical controls for a one-armed clinical trial, and high quality RDRs may be used for registry-based randomized controlled trials. In the post marketing phase of (conditional) drug approval a disease-specific RDR is likely to provide more relevant information than a product-specific registry.

CONCLUSIONS

A RDR can be very helpful to improve the efficiency and quality of clinical trial design in several ways. To enable the applicability and optimal use of a RDR longitudinal data collection is indispensable, and specific data collection, prepared for repeated measurement, is needed. The developed checklist can help to define the appropriate variables to include. Attention should be paid to the inclusion of patient-relevant outcome measures in the RDR from the start. More research and experience is needed on the possibilities and limitations of combining RDR information with clinical trial data to maximize the availability of relevant evidence for regulatory decisions in rare diseases.

Replicability, Reproducibility, and Agent-based Simulation of Interventions

Secondary use of medical data and use of observational data for causal inference has been growing. Yet these data bring many challenges such as confounding due to unobserved variables and variation in medical processes across settings. Further, while methods exist to handle some of these problems, researchers lack ground truth to evaluate these methods. When a finding is not replicated across multiple sites, it is unknown whether this is a failure of an algorithm, a genuine difference between populations, or an artifact of structural differences between the sites. We show how agent-based simulation of medical interventions can be used to explore how bias, error, and variation across settings affect inference. Our approach enables users to model not only interventions and outcomes, but also the complex interaction between patients with different risks of mortality and providers with different observed and latent treatment effects. Ultimately we propose that such simulations can be used to better evaluate the behavior of new methods with known ground truth and better calculate sample size for EHR-based studies.

Comparison of nuisance parameters in pediatric versus adult randomized trials: a meta-epidemiologic empirical evaluation

Background

We wished to compare the nuisance parameters of pediatric vs. adult randomized-trials (RCTs) and determine if the latter can be used in sample size computations of the former.

Methods

In this meta-epidemiologic empirical evaluation we examined meta-analyses from the Cochrane Database of Systematic-Reviews, with at least one pediatric-RCT and at least one adult-RCT. Within each meta-analysis of binary efficacy-outcomes, we calculated the pooled-control-group event-rate (CER) across separately all pediatric and adult-trials, using random-effect models and subsequently calculated the control-group event-rate risk-ratio (CER-RR) of the pooled-pediatric-CERs vs. adult-CERs. Within each meta-analysis with continuous outcomes we calculated the pooled-control-group effect standard deviation (CE-SD) across separately all pediatric and adult-trials and subsequently calculated the CE-SD-ratio of the pooled-pediatric-CE-SDs vs. adult-CE-SDs. We then calculated across all meta-analyses the pooled-CER-RRs and pooled-CE-SD-ratios (primary endpoints) and the pooled-magnitude of effect-sizes of CER-RRs and CE-SD-ratios using REMs. A ratio < 1 indicates that pediatric trials have smaller nuisance parameters than adult trials.

Results

We analyzed 208 meta-analyses (135 for binary-outcomes, 73 for continuous-outcomes). For binary outcomes, pediatric-RCTs had on average 10% smaller CERs than adult-RCTs (summary-CE-RR: 0.90; 95% CI: 0.83, 0.98). For mortality outcomes the summary-CE-RR was 0.48 (95% CIs: 0.31, 0.74). For continuous outcomes, pediatric-RCTs had on average 26% smaller CE-SDs than adult-RCTs (summary-CE-SD-ratio: 0.74).

Conclusions

Clinically relevant differences in nuisance parameters between pediatric and adult trials were detected. These differences have implications for design of future studies. Extrapolation of nuisance parameters for sample-sizes calculations from adult-trials to pediatric-trials should be cautiously done.