An audit of sample sizes for pilot and feasibility trials being undertaken in the United Kingdom registered in the United Kingdom Clinical Research Network database

Background

There is little published guidance as to the sample size required for a pilot or feasibility trial despite the fact that a sample size justification is a key element in the design of a trial. A sample size justification should give the minimum number of participants needed in order to meet the objectives of the trial. This paper seeks to describe the target sample sizes set for pilot and feasibility randomised controlled trials, currently running within the United Kingdom.

Methods

Data were gathered from the United Kingdom Clinical Research Network (UKCRN) database using the search terms ‘pilot’ and ‘feasibility’. From this search 513 studies were assessed for eligibility of which 79 met the inclusion criteria. Where the data summary on the UKCRN Database was incomplete, data were also gathered from: the International Standardised Randomised Controlled Trial Number (ISRCTN) register; the clinicaltrials.gov website and the website of the funders. For 62 of the trials, it was necessary to contact members of the research team by email to ensure completeness.

Results

Of the 79 trials analysed, 50 (63.3%) were labelled as pilot trials, 25 (31.6%) feasibility and 14 were described as both pilot and feasibility trials. The majority had two arms (n = 68, 86.1%) and the two most common endpoints were continuous (n = 45, 57.0%) and dichotomous (n = 31, 39.2%). Pilot trials were found to have a smaller sample size per arm (median = 30, range = 8 to 114 participants) than feasibility trials (median = 36, range = 10 to 300 participants). By type of endpoint, across feasibility and pilot trials, the median sample size per arm was 36 (range = 10 to 300 participants) for trials with a dichotomous endpoint and 30 (range = 8 to 114 participants) for trials with a continuous endpoint. Publicly funded pilot trials appear to be larger than industry funded pilot trials: median sample sizes of 33 (range = 15 to 114 participants) and 25 (range = 8 to 100 participants) respectively.

Conclusion

All studies should have a sample size justification. Not all studies however need to have a sample size calculation. For pilot and feasibility trials, while a sample size justification is important, a formal sample size calculation may not be appropriate. The results in this paper describe the observed sample sizes in feasibility and pilot randomised controlled trials on the UKCRN Database.

A reinvestigation of recruitment to randomised, controlled, multicenter trials: a review of trials funded by two UK funding agencies

Background

Randomised controlled trials (RCTs) are the gold standard assessment for health technologies. A key aspect of the design of any clinical trial is the target sample size. However, many publicly-funded trials fail to reach their target sample size. This study seeks to assess the current state of recruitment success and grant extensions in trials funded by the Health Technology Assessment (HTA) program and the UK Medical Research Council (MRC).

Methods

Data were gathered from two sources: the National Institute for Health Research (NIHR) HTA Journal Archive and the MRC subset of the International Standard Randomised Controlled Trial Number (ISRCTN) register. A total of 440 trials recruiting between 2002 and 2008 were assessed for eligibility, of which 73 met the inclusion criteria. Where data were unavailable from the reports, members of the trial team were contacted to ensure completeness.

Results

Over half (55%) of trials recruited their originally specified target sample size, with over three-quarters (78%) recruiting 80% of their target. There was no evidence of this improving over the time of the assessment. Nearly half (45%) of trials received an extension of some kind. Those that did were no more likely to successfully recruit. Trials with 80% power were less likely to successfully recruit compared to studies with 90% power.

Conclusions

While recruitment appears to have improved since 1994 to 2002, publicly-funded trials in the UK still struggle to recruit to their target sample size, and both time and financial extensions are often requested. Strategies to cope with such problems should be more widely applied. It is recommended that where possible studies are planned with 90% power.

Environmental triggers of hospital admissions for school-age children with asthma in two British cities

Research has reported seasonal peaks in asthma in school age asthmatic children. The study aimed to assess if hospital admissions could be predicted from the possible environmental triggers using data from two British cities: Aberdeen and Doncaster. However, there were no consistent patterns across the two cities with no clear evidence that hospital admissions could be predicted from environmental data.

Estimating effect sizes for health-related quality of life outcomes

Summary

To enable an assessment of the costs and benefits of a new health technology one should use a range of outcome measures, including medical, psychosocial and economic. Therefore, unless a patient-reported outcome as well as clinical outcome is assessed in a study, the effect of a health technology on the patient will remain unknown as two therapies may have similar clinical consequences but different impacts upon the quality of the life of the patients. An important issue when designing a study with a new patient-reported outcome is the quantification of an effect size. Through a case study we highlight how simple calculations can enable the estimation of the effect sizes if there is information on established outcomes. This is done by mapping changes on the new scale to clinically relevant and important changes on established scales. We recommend the approaches described in this paper be considered for the quantification of important treatment effects when designing a clinical trial with a new patient-reported outcome measure.

A theory-based online health behavior intervention for new university students: study protocol

BACKGROUND

Too few young people engage in behaviors that reduce the risk of morbidity and premature mortality, such as eating healthily, being physically active, drinking sensibly and not smoking. The present research developed an online intervention to target these health behaviors during the significant life transition from school to university when health beliefs and behaviors may be more open to change. This paper describes the intervention and the proposed approach to its evaluation.

METHODS/DESIGN

Potential participants (all undergraduates about to enter the University of Sheffield) will be emailed an online questionnaire two weeks before starting university. On completion of the questionnaire, respondents will be randomly assigned to receive either an online health behavior intervention (U@Uni) or a control condition. The intervention employs three behavior change techniques (self-affirmation, theory-based messages, and implementation intentions) to target four heath behaviors (alcohol consumption, physical activity, fruit and vegetable intake, and smoking). Subsequently, all participants will be emailed follow-up questionnaires approximately one and six months after starting university. The questionnaires will assess the four targeted behaviors and associated cognitions (e.g., intentions, self-efficacy) as well as socio-demographic variables, health status, Body Mass Index (BMI), health service use and recreational drug use. A sub-sample of participants will provide a sample of hair to assess changes in biochemical markers of health behavior. A health economic evaluation of the cost effectiveness of the intervention will also be conducted.

DISCUSSION

The findings will provide evidence on the effectiveness of online interventions as well as the potential for intervening during significant life transitions, such as the move from school to university. If successful, the intervention could be employed at other universities to promote healthy behaviors among new undergraduates.

TRIAL REGISTRATION

Current Controlled Trials, ISRCTN67684181.

Efficacy and suicidal risk for antidepressants in paediatric and adolescent patients

A number of meta-analyses have been undertaken to assess both the safety and efficacy of antidepressants in paediatric and adolescent patients. This article updates the analyses with additionally reported trials. The aim of this analysis was to investigate whether antidepressant treatments are associated with an increased risk of suicide-related outcomes in paediatric and adolescent patients. Also, in the same population, to assess whether antidepressant treatments are beneficial in terms of efficacy. A meta-analysis of randomised controlled trials of antidepressant treatments compared with placebo in paediatric and adolescent patients was undertaken of 6039 individuals participating in 35 randomised controlled trials. For suicide-related outcomes suicidal behaviour, suicidal ideation and suicidal behaviour or ideation were examined. These data presented the additional problem of the events of interest being rare. An analysis was described in this article to account for the rare events that also included studies which had no events on either treatment arm. There were trends to indicate that active treatments increased the risk of these events in absolute terms. For efficacy, the results indicated that antidepressant treatments did have a statistically significant effects compared to placebo but the effect was less for the trials in depression. The results are in the main consistent with previous meta-analyses on a smaller number of trials. There was evidence of an increased risk in suicide-related outcomes on antidepressant treatments, while antidepressant treatments were also shown to be efficacious.

Investigating the assumption of homogeneity of treatment effects in clinical studies with application to meta-analysis

An important part of the evaluation of a therapy is an investigation of the assumption of homogeneity of its effect across pre-defined subpopulations. In this paper we describe simple graphical presentations that could be used to assess the homogeneity of treatment effect and identify outliers. The emphasis in the paper is on meta-analysis but the methods described can be generalized to other investigations.

Seasonality of medical contacts in school-aged children with asthma: association with school holidays

OBJECTIVES

To assess the seasonality of medical contacts in children with asthma, to compare England with Scotland, and to assess the impact of medication compliance on the frequency of medical contacts.

STUDY DESIGN

A retrospective study taken from the General Practice Research Database of daily medical contacts from 1999 to 2005 in a population of school-aged (5-16 years) children with a diagnosis of asthma, and age- and gender-matched controls.

PRIMARY OUTCOME MEASURE

All unscheduled medical contacts. Prescription frequency for inhaled corticosteroids over the same period was also assessed.

RESULTS

Data from 76,924 children were analyzed. There was an increase in unscheduled medical contacts for all children on return to school in September. Unscheduled medical contacts occurred almost twice as frequently in children with asthma compared with non-asthmatic children. The frequency of medical contacts in children with asthma could be predicted from the frequency of medical contacts in non-asthmatic children. In the 3 months from September to December, unscheduled medical contacts were disproportionately greater than would be predicted for children with asthma relative to non-asthmatic children in both England and Scotland. The rise in medical contacts in Scotland preceded that in England by approximately 14 days, reflecting the earlier date for returning to school. The number of prescriptions for inhaled corticosteroids decreased in August. A higher incidence of unscheduled medical contacts was noted during September in those not receiving a prescription in August.

CONCLUSIONS

Returning to school after the summer break is associated with a sharp increase in unscheduled medical contacts in school-aged children, particularly in those with asthma. This follows a decrease in the number of prescriptions for inhaled corticosteroids. It is suggested that at least part of the excess numbers of unscheduled contacts in children with asthma is because they do not maintain their inhaled corticosteroids over the summer holidays.

The potential for bias in reporting of industry-sponsored clinical trials

Concerns about potentially misleading reporting of pharmaceutical industry research have surfaced many times. The potential for duality (and thereby conflict) of interest is only too clear when you consider the sums of money required for the discovery, development and commercialization of new medicines. As the ability of major, mid-size and small pharmaceutical companies to innovate has waned, as evidenced by the seemingly relentless decline in the numbers of new medicines approved by Food and Drug Administration and European Medicines Agency year-on-year, not only has the cost per new approved medicine risen: so too has the public and media concern about the extent to which the pharmaceutical industry is open and honest about the efficacy, safety and quality of the drugs we manufacture and sell. In 2005 an Editorial in Journal of the American Medical Association made clear that, so great was their concern about misleading reporting of industry-sponsored studies, henceforth no article would be published that was not also guaranteed by independent statistical analysis. We examine the precursors to this Editorial, as well as its immediate and lasting effects for statisticians, for the manner in which statistical analysis is carried out, and for the industry more generally.

Seven useful designs

When designing a clinical trial there is a need to design a study that achieves the objectives of a trial both efficiently and with minimum resources. This paper describes seven trial designs that could possibly be used in clinical development and highlights how a design although not optimal for an individual study may be optimal for a wider clinical program.

A comparison of methods for sample size estimation for non-inferiority studies with binary outcomes

Non-inferiority trials are motivated in the context of clinical research where a proven active treatment exists and placebo-controlled trials are no longer acceptable for ethical reasons. Instead, active-controlled trials are conducted where a treatment is compared to an established treatment with the objective of demonstrating that it is non-inferior to this treatment. We review and compare the methodologies for calculating sample sizes and suggest appropriate methods to use. We demonstrate how the simplest method of using the anticipated response is predominantly consistent with simulations. In the context of trials with binary outcomes with expected high proportions of positive responses, we show how the sample size is quite sensitive to assumptions about the control response. We recommend when designing such a study that sensitivity analyses be performed with respect to the underlying assumptions and that the Bayesian methods described in this article be adopted to assess sample size.

Investigating variability in patient response to treatment--a case study from a replicate cross-over study

It is a common belief that individual variation in response to treatment is an important explanation for the variation in observed outcomes in clinical trials. If such variation is large, it seems reasonable to suppose that progress in treating disease will be advanced by classifying patients according to their abilities or not to 'respond' to particular treatments. We consider that there is currently a lost opportunity in drug development. There is a great deal of talk about individual response to treatment and tailor-made drugs. However, relatively little work is being done to formally investigate, using suitable designs, where individual response to treatment may be important. Through a case study from a replicate cross-over study we show how, given suitable replication, it is possible to isolate the component of variation corresponding to patient-by-treatment interaction and hence investigate the possibility of individual response to treatment.

Issues with using baseline in last observation carried forward analysis

The topic of this paper was prompted by a study for which one of us was the statistician. It was submitted to Annals of Internal Medicine. The paper had positive reviewer comment; however, the statistical reviewer stated that for the analysis to be acceptable for publication, the missing data had to be accounted for in the analysis through the use of baseline in a last observation carried forward imputation. We discuss the issues associated with this form of imputation and recommend that it should not be undertaken as a primary analysis.