Estimation of expectedness: predictive accuracy of standard therapy outcomes in randomized phase 3 studies in epithelial ovarian cancer

QUALITY OF SAMPLE SIZE ESTIMATION IN TRIALS OF MEDICAL DEVICES: HIGH-RISK DEVICES FOR NEUROLOGICAL CONDITIONS AS EXAMPLE

BACKGROUND

The aim of this study was to assess the quality of reporting sample size calculation and underlying design assumptions in pivotal trials of high-risk medical devices (MDs) for neurological conditions.

METHODS

Systematic review of research protocols for publicly registered randomized controlled trials (RCTs). In the absence of a published protocol, principal investigators were contacted for additional data. To be included, trials had to investigate a high-risk MD, registered between 2005 and 2015, with indications stroke, headache disorders, and epilepsy as case samples within central nervous system diseases. Extraction of key methodological parameters for sample size calculation was performed independently and peer-reviewed.

RESULTS

In a final sample of seventy-one eligible trials, we collected data from thirty-one trials. Eighteen protocols were obtained from the public domain or principal investigators. Data availability decreased during the extraction process, with almost all data available for stroke-related trials. Of the thirty-one trials with sample size information available, twenty-six reported a predefined calculation and underlying assumptions. Justification was given in twenty and evidence for parameter estimation in sixteen trials. Estimates were most often based on previous research, including RCTs and observational data. Observational data were predominantly represented by retrospective designs. Other references for parameter estimation indicated a lower level of evidence.

CONCLUSIONS

Our systematic review of trials on high-risk MDs confirms previous research, which has documented deficiencies regarding data availability and a lack of reporting on sample size calculation. More effort is needed to ensure both relevant sources, that is, original research protocols, to be publicly available and reporting requirements to be standardized.

Targeted survival improvements in clinical trials: are you an absolutist or relativist?

In this issue of Cancer, Castonguay et al highlight problems with reliably estimating the median OS in the control arm of clinical trials in ovarian cancer. This suggests that more careful reflection is generally required concerning how clinically relevant differences are determined and highlights further issues with the selection of overall survival as a primary endpoint. The article also raises questions regarding the prevalent standard statistical approach used to design and analyze studies with time to event outcomes.