A study of target effect sizes in randomised controlled trials published in the Health Technology Assessment journal

Feasibility and Acceptability of Mindfulness-based Stress Reduction and Prenatal Sleep Classes for Poor Prenatal Sleep Quality: Pilot Randomized Controlled Trial

OBJECTIVES

The main objectives of the current paper were to examine the feasibility, acceptability, and adherence of a remotely delivered intervention consisting of mindfulness-based stress reduction plus prenatal sleep classes (MBSR+PS) compared with treatment as usual (TAU).

METHOD

In this pilot randomized controlled trial, 52 pregnant women with poor sleep quality were randomized to MBSR+PS or TAU. MBSR was delivered through eight weekly 2.5-hour sessions, and PS was delivered through eight weekly 30-minute sessions. PS content drew material from cognitive behavioral therapy for insomnia tailored for the perinatal period and from a mindfulness- and acceptance-based lens. Participants completed endpoint measures 10-12 weeks after randomization.

RESULTS

We surpassed all acceptability targets, including the percentage of eligible participants willing to be randomized (96%), percentage of participants who initiated treatment (88%), and satisfaction scores (Client Satisfaction Questionnaire-8 score M = 28.04, SD = 3.6). We surpassed all feasibility targets, including our enrollment target, retention rate (92%), and measure completion (96%). Finally, we surpassed adherence targets, including MBSR and PS session attendance (≥80%). Though sleep outcomes were exploratory, increases in sleep efficiency were greater in the MBSR+PS group relative to TAU (SMD=.68).

CONCLUSIONS

Patient-reported poor sleep quality during pregnancy has high public health significance because it is common, consequential, and under-treated. The current feasibility and acceptability data for using remotely delivered MBSR and PS to improve prenatal sleep quality are encouraging and warranting future research that is sufficiently powered and designed to provide efficacy data. In addition, exploratory sleep outcomes offer preliminary evidence that this sleep program may improve sleep efficiency during pregnancy.

Improving Miscarriage Prevention Research: a survey exploring the expectations of service users and stakeholders (IMPRESS) - a study protocol for a UK-based survey

INTRODUCTION

Interventional clinical trials in recurrent miscarriage use varying expected effect sizes to inform their sample size calculations. Often these are not informed by what stakeholders consider a meaningful treatment effect. Adaptive trial designs may integrate stakeholder views on trial success and futility but the criteria to inform this is lacking. This study aims to understand relevant stakeholder views of what is considered a worthwhile treatment effect for miscarriage prevention interventions and what is acceptable stopping criteria in miscarriage clinical trials.

METHODS AND ANALYSIS

The study is designed as a cross-sectional online anonymous survey. The survey presents different scenarios to respondents relating to varying target differences and probability thresholds and explores success and futility criteria for clinical trials. The survey was developed with personal and public involvement (PPI) through focus groups and a PPI partner. Eligible participants will be those with a personal history of miscarriage, including partners, and healthcare professionals who manage patients who experience a miscarriage. Convenience, snowball and purposive sampling techniques will be employed to invite eligible participants to complete the survey. The survey will be accepting responses for an initial 2-week pilot to check validity, prior to being open for a further 12 weeks. Descriptive analyses and linear regression analyses will synthesise the survey results.

ETHICS AND DISSEMINATION

Ethical approval was obtained from the NHS Research Ethics Committee North West-Greater Manchester East (23/NW/0322) on 30 January 2024. Informed consent will be obtained prior to survey completion. No personal identifying information will be collected. The results will be published in a relevant scientific journal and communicated through our institutional website.

The effects of Kinesio taping on static and dynamic balance in children with down syndrome: a randomized controlled trial

AIM

This study aimed to determine the effects of Kinesio tape applied to the plantar soles on static and dynamic balance in children with Down syndrome (DS).

MATERIALS AND METHODS

The study was carried out in children with DS. The participants were grouped as Kinesio Taping (KT) (n = 12, DS) and Sham Taping (ST) (n = 12, DS). The Functional Reach Test (FRT) was used to evaluate functional balance and the Fast-Timed Up and Go (FAST-TUG) test to evaluate functional balance and capacity. The Modified Clinical test of Sensory Interaction on Balance (MCTSIB) was used to evaluate static balance. For both groups, all the assessments were made three times: at baseline (T0), right after the taping application (T1), and 40-45 minutes later (T2).

RESULTS

Baseline FAST-TUG, FRT, and Eyes Open (EO) and Closed (EC) Sway velocity scores of the KT (medians- FAST-TUG:7.75 s, FRT:23.90 cm, EO: 0.70 deg/s, EC: 0.60 deg/s) and ST (medians-FAST-TUG:7.98 s, FRT:24 cm, EO: 0.85 deg/s, EC: 0.95 deg/s) groups were similar (p >0.05). Intragroup comparisons showed that FAST-TUG and FRT scores improved after the taping compared with T0 values in both KT (KT (FAST-TUG:7.75s-FRT:23.90cm)/KT1(FAST-TUG:7.55 s-FRT:28.25cm), KT(FAST-TUG:7.75s-FRT:23.90cm)/KT2(FAST-TUG:6.85s-FRT:27.50cm)) and ST groups (ST(FAST-TUG:7.98s-FRT:24cm)/ST1(FAST-TUG:7,95s-FRT:26.40cm), ST(FAST-TUG:7.98s-FRT:24cm)/ST2(FAST-TUG:7.26s-FRT:26.15cm)) (p < 0.05), while the sway velocity values were similar before and after the taping (p > 0.05).

CONCLUSIONS

Taping on the plantar soles of children with DS may be affecting the immediate dynamic balance scores while it actually did not affect the static balance scores independent of the technique used. The interpretation of the results of this study should be made with caution. Further studies with long-term evaluations are needed.

Effect of fluoridated water on invasive NHS dental treatments for adults: the LOTUS retrospective cohort study and economic evaluation

Background

Most water fluoridation studies were conducted on children before the widespread introduction of fluoride toothpastes. There is a lack of evidence that can be applied to contemporary populations, particularly adolescents and adults.

Objective

To pragmatically assess the clinical and cost effectiveness of water fluoridation for preventing dental treatment and improving oral health in a contemporary population of adults, using a natural experiment design.

Design

Retrospective cohort study using routinely collected National Health Service dental claims (FP17) data.

Setting

National Health Service primary dental care: general dental practices, prisons, community dental services, domiciliary settings, urgent/out-of-hours and specialised referral-only services.

Participants

Dental patients aged 12 years and over living in England (n = 6,370,280).

Intervention and comparison

Individuals exposed to drinking water with a fluoride concentration ≥ 0.7 mg F/l between 2010 and 2020 were matched to non-exposed individuals on key characteristics using propensity scores.

Outcome measures

Primary: number of National Health Service invasive dental treatments (restorations/'fillings' and extractions) received per person between 2010 and 2020. Secondary: decayed, missing and filled teeth, missing teeth, inequalities, cost effectiveness and return on investment.

Data sources

National Health Service Business Services Authority dental claims data. Water quality monitoring data.

Primary outcome

Predicted mean number of invasive dental treatments was 3% lower in the optimally fluoridated group than in the sub/non-optimally fluoridated group (incidence rate ratio 0.969, 95% CI 0.967 to 0.971), a difference of -0.173 invasive dental treatments (95% CI -0.185 to -0.161). This magnitude of effect is smaller than what most stakeholders we engaged with (n = 50/54) considered meaningful.

Secondary outcomes

Mean decayed, missing and filled teeth were 2% lower in the optimally fluoridated group, with a difference of -0.212 decayed, missing and filled teeth (95% CI -0.229 to -0.194). There was no statistically significant difference in the mean number of missing teeth per person (0.006, 95% CI -0.008 to 0.021). There was no compelling evidence that water fluoridation reduced social inequalities in treatments received or missing teeth; however, decayed, missing and filled teeth data did not demonstrate a typical inequalities gradient. Optimal water fluoridation in England in 2010-20 was estimated to cost £10.30 per person (excluding original setup costs). Mean National Health Service treatment costs for fluoridated patients 2010-20 were 5.5% lower per person, by £22.26 (95% CI -£23.09 to -£21.43), and patients paid £7.64 less in National Health Service dental charges per person (2020 prices).

Limitations

Pragmatic, observational study with potential for non-differential errors of misclassification in fluoridation assignment and outcome measurement and residual and/or unmeasured confounding. Decayed, missing and filled teeth data have not been validated. Water fluoridation cost estimates are based on existing programmes between 2010 and 2020, and therefore do not include the potentially significant capital investment required for new programmes.

Conclusions

Receipt of optimal water fluoridation between 2010 and 2020 resulted in very small health effects, which may not be meaningful for individuals, and we could find no evidence of a reduction in social inequalities. Existing water fluoridation programmes in England produced a positive return on investment between 2010 and 2020 due to slightly lower National Health Service treatment costs. These relatively small savings should be evaluated against the projected costs and lifespan of any proposed capital investment in water fluoridation, including new programmes.

Future work

National Health Service dental data are a valuable resource for research. Further validation and measures to improve quality and completeness are warranted.

Trial registrations

This trial is registered as ISRCTN96479279, CAG: 20/CAG/0072, IRAS: 20/NE/0144.

Funding

This award was funded by the National Institute for Health and Care Research (NIHR) Public Health Research programme (NIHR award ref: NIHR128533) and is published in full in Public Health Research; Vol. 12, No. 5. See the NIHR Funding and Awards website for further award information.

Mentorship for young gay men in Hong Kong: A pilot mixed-methods randomized controlled trial

This study examined whether mentorship could promote young gay men's identities and well-being, and whether a mentor's sexual orientation matters. A randomized control trial compared outcomes across three conditions: Arm A (a mentee matched with a sexual minority mentor), Arm B (a mentee matched with a heterosexual mentor), and a control arm receiving psychoeducation only. A community sample of 60 mentees aged 18-25 years was randomly allocated to the three arms and completed questionnaires at baseline, 3 months into the intervention, and at the end of the 6-month program. Fifteen mentees recounted their mentoring experiences through in-depth interviews. Linear mixed effects models showed that for both intervention and control conditions, internalized homonegativity declined while resilience, loneliness, and body acceptance improved over time. No time and group interactions were found. Meanwhile, a mentor's sexual orientation did not drive differential quality and outcomes of mentorship. Interviewees cited various benefits of mentorship, including providing companionship, enriching connection with lesbian, gay, bisexual (LGB) communities, and adding knowledge and perspectives of LGB lives and identities. Although quantitative data did not support any exclusive benefits of mentorship, most mentees recognized mentorship as a vital source of affirmation and companionship. Implications for research and mentoring programs are discussed.

Undertaking Studies Within A Trial to evaluate recruitment and retention strategies for randomised controlled trials: lessons learnt from the PROMETHEUS research programme

Background

Randomised controlled trials ('trials') are susceptible to poor participant recruitment and retention. Studies Within A Trial are the strongest methods for testing the effectiveness of strategies to improve recruitment and retention. However, relatively few of these have been conducted.

Objectives

PROMoting THE Use of Studies Within A Trial aimed to facilitate at least 25 Studies Within A Trial evaluating recruitment or retention strategies. We share our experience of delivering the PROMoting THE Use of Studies Within A Trial programme, and the lessons learnt for undertaking randomised Studies Within A Trial.

Design

A network of 10 Clinical Trials Units and 1 primary care research centre committed to conducting randomised controlled Studies Within A Trial of recruitment and/or retention strategies was established. Promising recruitment and retention strategies were identified from various sources including Cochrane systematic reviews, the Study Within A Trial Repository, and existing prioritisation exercises, which were reviewed by patient and public members to create an initial priority list of seven recruitment and eight retention interventions. Host trial teams could apply for funding and receive support from the PROMoting THE Use of Studies Within A Trial team to undertake Studies Within A Trial. We also tested the feasibility of undertaking co-ordinated Studies Within A Trial, across multiple host trials simultaneously.

Setting

Clinical trials unit-based trials recruiting or following up participants in any setting in the United Kingdom were eligible.

Participants

Clinical trials unit-based teams undertaking trials in any clinical context in the United Kingdom.

Interventions

Funding of up to £5000 and support from the PROMoting THE Use of Studies Within A Trial team to design, implement and report Studies Within A Trial.

Main outcome measures

Number of host trials funded.

Results

Forty-two Studies Within A Trial were funded (31 host trials), across 12 Clinical Trials Units. The mean cost of a Study Within A Trial was £3535. Twelve Studies Within A Trial tested the same strategy across multiple host trials using a co-ordinated Study Within A Trial design, and four used a factorial design. Two recruitment and five retention strategies were evaluated in more than one host trial. PROMoting THE Use of Studies Within A Trial will add 18% more Studies Within A Trial to the Cochrane systematic review of recruitment strategies, and 79% more Studies Within A Trial to the Cochrane review of retention strategies. For retention, we found that pre-notifying participants by card, letter or e-mail before sending questionnaires was effective, as was the use of pens, and sending personalised text messages to improve questionnaire response. We highlight key lessons learnt to guide others planning Studies Within A Trial, including involving patient and public involvement partners; prioritising and selecting strategies to evaluate and elements to consider when designing a Study Within A Trial; obtaining governance approvals; implementing Studies Within A Trial, including individual and co-ordinated Studies Within A Trials; and reporting Study Within A Trials.

Limitations

The COVID-19 pandemic negatively impacted five Studies Within A Trial, being either delayed (n = 2) or prematurely terminated (n = 3).

Conclusions

PROMoting THE Use of Studies Within A Trial significantly increased the evidence base for recruitment and retention strategies. When provided with both funding and practical support, host trial teams successfully implemented Studies Within A Trial.

Future work

Future research should identify and target gaps in the evidence base, including widening Study Within A Trial uptake, undertaking more complex Studies Within A Trial and translating Study Within A Trial evidence into practice.

Study registration

All Studies Within A Trial in the PROMoting THE Use of Studies Within A Trial programme had to be registered with the Northern Ireland Network for Trials Methodology Research Study Within A Trial Repository.

Funding

This award was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme (NIHR award ref: 13/55/80) and is published in full in Health Technology Assessment; Vol. 28, No. 2. See the NIHR Funding and Awards website for further award information.

Association of intestinal bacteria with immune activation in a cohort of healthy adults

IMPORTANCE

Chronic inflammation may develop over time in healthy adults as a result of a variety of factors, such as poor diet directly affecting the composition of the intestinal microbiome, or by causing obesity, which may also affect the intestinal microbiome. These effects may trigger the activation of an immune response that could eventually lead to an inflammation-related disease, such as colon cancer. Before disease develops it may be possible to identify subclinical inflammation or immune activation attributable to specific intestinal bacteria normally found in the gut that could result in future adverse health impacts. In the present study, we examined a group of healthy men and women across a wide age range with and without obesity to determine which bacteria were associated with particular types of immune activation to identify potential preclinical markers of inflammatory disease risk. Several associations were found that may help develop dietary interventions to lower disease risk.

Treatment Effect Estimates From Pilot Trials Are Unreliable

CONTEXT

The CONSORT guideline defines a pilot trial as a small-scale version of a desired future efficacy trial that is intended to answer the key questions of whether and how a larger study should be done. For example, a pilot trial might evaluate different approaches to data collection or outcome measurement. However, pilot trials are unreliable for assessing treatment efficacy due to the statistical phenomenon called sampling variability.

OBJECTIVES

In this tutorial we use computer simulation to demonstrate the influence of sampling variability on efficacy estimates from pilot trials, illustrating why pilot trial designs should not be used to evaluate whether a treatment is promising or not.

METHODS

We simulate a 2-arm parallel group trial (N=20 per group) with a survival outcome as an example. Simulations are done under two scenarios: 1) the treatment is efficacious at the level of a hypothetical minimum clinically important difference (hazard ratio [HR] = 0.75); and 2) the treatment is not efficacious (HR=1).

RESULTS

As expected, in both simulated scenarios the range of observed results is distributed around the true treatment effect, HR=0.75 or HR=1. Importantly, ∼20% of trials simulated under scenario 1 incorrectly suggest the treatment may be harmful (HR > 1). Under scenario 2, half of the simulated studies incorrectly suggest the treatment is beneficial.

CONCLUSION

Treatment effect estimates from pilot trials should not be used to make future development decisions regarding a novel therapy because of the high risk of misleading conclusions.

A review of sample sizes for UK pilot and feasibility studies on the ISRCTN registry from 2013 to 2020

BACKGROUND

Pilot and feasibility studies provide information to be used when planning a full trial. A sufficient sample size within the pilot/feasibility study is required so this information can be extracted with suitable precision. This work builds upon previous reviews of pilot and feasibility studies to evaluate whether the target sample size aligns with recent recommendations and whether these targets are being reached.

METHODS

A review of the ISRCTN registry was completed using the keywords "pilot" and "feasibility". The inclusion criteria were UK-based randomised interventional trials that started between 2013 (end of the previous review) and 2020. Target sample size, actual sample size and key design characteristics were extracted. Descriptive statistics were used to present sample sizes overall and by key characteristics.

RESULTS

In total, 761 studies were included in the review of which 448 (59%) were labelled feasibility studies, 244 (32%) pilot studies and 69 (9%) described as both pilot and feasibility studies. Over all included pilot and feasibility studies (n = 761), the median target sample size was 30 (IQR 20-50). This was consistent when split by those labelled as a pilot or feasibility study. Slightly larger sample sizes (median = 33, IQR 20-50) were shown for those labelled both pilot and feasibility (n = 69). Studies with a continuous outcome (n = 592) had a median target sample size of 30 (IQR 20-43) whereas, in line with recommendations, this was larger for those with binary outcomes (median = 50, IQR 25-81, n = 97). There was no descriptive difference in the target sample size based on funder type. In studies where the achieved sample size was available (n = 301), 173 (57%) did not reach their sample size target; however, the median difference between the target and actual sample sizes was small at just minus four participants (IQR -25-0).

CONCLUSIONS

Target sample sizes for pilot and feasibility studies have remained constant since the last review in 2013. Most studies in the review satisfy the earlier and more lenient recommendations however do not satisfy the most recent largest recommendation. Additionally, most studies did not reach their target sample size meaning the information collected may not be sufficient to estimate the required parameters for future definitive randomised controlled trials.

Chan-Chuang and resistance exercise for drug rehabilitation: a randomized controlled trial among Chinese male methamphetamine users

Objective

To examine the health benefits of Chan-Chuang and resistance exercise.

Methods

We deployed an 8-week randomized controlled trial, in which 76 male methamphetamine users were allocated to control (n = 25), Chan-Chuang (n = 26), and residence exercise groups (n = 25). Our primary outcomes were drug craving, mental wellbeing, sleep quality, heart rate (HR), systolic blood pressure (SBP), diastolic blood pressure (DBP), and mean arterial pressure (MAP). Our secondary outcomes were body mass index (BMI), vital capacity, grip strength, balance, and vertical jump.

Results

Chan-Chuang exercise resulted in reduced HR, DBP, and MAP, along with improvements in vital capacity, grip strength, and balance compared to the control group. Resistance exercise reduced SBP and MAP, and also improved vital capacity, grip strength, balance, and vertical jump.

Conclusion

These findings may support the role of Chan-Chuang and resistance exercise in maintaining the physical fitness of methamphetamine users at mandatory detention centers.

Predicting non-improvement of symptoms in daily mental healthcare practice using routinely collected patient-level data: a machine learning approach

Objectives

Anxiety and mood disorders greatly affect the quality of life for individuals worldwide. A substantial proportion of patients do not sufficiently improve during evidence-based treatments in mental healthcare. It remains challenging to predict which patients will or will not benefit. Moreover, the limited research available on predictors of treatment outcomes comes from efficacy RCTs with strict selection criteria which may limit generalizability to a real-world context. The current study evaluates the performance of different machine learning (ML) models in predicting non-improvement in an observational sample of patients treated in routine specialized mental healthcare.

Methods

In the current longitudinal exploratory prediction study diagnosis-related, sociodemographic, clinical and routinely collected patient-reported quantitative outcome measures were acquired during treatment as usual of 755 patients with a primary anxiety, depressive, obsessive compulsive or trauma-related disorder in a specialized outpatient mental healthcare center. ML algorithms were trained to predict non-response (< 0.5 standard deviation improvement) in symptomatic distress 6 months after baseline. Different models were trained, including models with and without early change scores in psychopathology and well-being and models with a trimmed set of predictor variables. Performance of trained models was evaluated in a hold-out sample (30%) as a proxy for unseen data.

Results

ML models without early change scores performed poorly in predicting six-month non-response in the hold-out sample with Area Under the Curves (AUCs) < 0.63. Including early change scores slightly improved the models' performance (AUC range: 0.68-0.73). Computationally-intensive ML models did not significantly outperform logistic regression (AUC: 0.69). Reduced prediction models performed similar to the full prediction models in both the models without (AUC: 0.58-0.62 vs. 0.58-0.63) and models with early change scores (AUC: 0.69-0.73 vs. 0.68-0.71). Across different ML algorithms, early change scores in psychopathology and well-being consistently emerged as important predictors for non-improvement.

Conclusion

Accurately predicting treatment outcomes in a mental healthcare context remains challenging. While advanced ML algorithms offer flexibility, they showed limited additional value compared to traditional logistic regression in this study. The current study confirmed the importance of taking early change scores in both psychopathology and well-being into account for predicting longer-term outcomes in symptomatic distress.

Protocol of a feasibility randomised controlled trial of Empowered Conversations: training family carers to enhance their relationships and communication with people living with dementia

Background

Communication difficulties can cause frustration, low mood, and stress for people living with dementia and their carer. Carers should be offered training on adapting their communication skills. However, it is not common for skills-based education to examine emotional aspects of care and the effect of dementia on relationships. The Empowered Conversations (EC) training course was developed in response to a gap in service provision and has been adapted to a virtual format (Zoom). It addresses the specific psychological, relationship, and communication needs of informal and family dementia carers. The primary aim of the study is to investigate the feasibility of conducting a multi-centre randomised controlled evaluation trial of EC. Secondary aims include exploring the acceptability of delivering the intervention online and examining the optimum way of establishing cost-effectiveness.

Methods

The feasibility trial uses a pragmatic data-collector blind parallel two-group RCT design with two arms (EC intervention plus treatment as usual, and treatment as usual waitlist control). There will be a 2:1 allocation in favour of the EC-training intervention arm. 75 participants will complete baseline outcome measures exploring their role as a carer, including their physical and mental health, attitudes to caring, quality of life, and use of health and social care services. These will be repeated after six-months. Participants allocated to the treatment group who complete the course will be invited to participate in a qualitative interview discussing their experience of EC.

Conclusions

The study will investigate recruitment pathways (including facilitators and barriers to recruitment), estimate retention levels and response rates to questionnaires, obtain additional evidence regarding proof of concept, and consider the most appropriate primary outcome measures and methods for evaluating cost-effectiveness. The results of the feasibility study will be used to inform the development of a multicentre randomised controlled trial in the United Kingdom.

Registration

ISRCTN15261686 (02/03/2022).

Observed intervention effects for mortality in randomised clinical trials: a methodological study protocol

INTRODUCTION

It is essential to choose a realistic anticipated intervention effect when calculating a sample size for a randomised clinical trial. Unfortunately, anticipated intervention effects are often inflated, when compared with the 'true' intervention effects. This is documented for mortality in critical care trials. A similar pattern might exist across different medical specialties. This study aims to estimate the range of observed intervention effects for all-cause mortality in trials included in Cochrane Reviews, within each Cochrane Review Group.

METHODS AND ANALYSIS

We will include randomised clinical trials assessing all-cause mortality as an outcome. Trials will be identified from Cochrane Reviews published in the Cochrane Database of Systematic Reviews. Cochrane Reviews will be clustered according to the registered Cochrane Review Group (eg, Anaesthesia, Emergency and Critical Care) and the statistical analyses will be conducted for each Cochrane Review Group and overall. The median relative risk and IQR for all-cause mortality and the proportion of trials with a relative all-cause mortality risk within seven different ranges will be reported (relative risk below 0.70, 0.70-0.79, 0.80-0.89, 0.90-1.09, 1.10-1.19, 1.20-1.30 and above 1.30). Subgroup analyses will explore the effects of original design, sample size, risk of bias, disease, intervention type, follow-up length, participating centres, funding type, information size and outcome hierarchy.

ETHICS AND DISSEMINATION

Since we will use summary data from trials already approved by relevant ethical committees, this study does not require ethical approval. Regardless of our findings, the results will be published in an international peer-reviewed journal.

Effect sizes and effect size benchmarks in family violence research

Scholarly journals increasingly request that authors include effect size (ES) estimates when reporting statistical results. However, there is little guidance on how authors should interpret ESs. Consequently, some authors do not provide ES interpretations, or, when interpretations are provided, they often fail to use appropriate reference groups, using instead the ES benchmarks suggested by Cohen (1988). After discussing the most commonly used ES estimates, we describe the method used by Cohen (1962) to develop ES benchmarks (i.e., small, medium, and large) for use in power analyses and describe the limitations associated with using these benchmarks. Next, we establish general benchmarks for family violence (FV) research. That is, we followed Cohen's approach to establishing his original ES benchmarks using family violence research published in 2021 in Child Abuse & Neglect, which produced a medium ES (d = 0.354) that was smaller than Cohen's recommended medium ES (d = 0.500). Then, we examined the ESs in different subspecialty areas of FV research to provide benchmarks for contextualized FV ESs and to provide information that can be used to conduct power analyses when planning future FV research. Finally, some of the challenges to developing ES benchmarks in any scholarly discipline are discussed. For professionals who are not well informed about ESs, the present review is designed to increase their understanding of ESs and what ES benchmarks tell them (and do not tell them) with respect to understanding the meaningfulness of FV research findings.

A retrospective analysis of conditional power assumptions in clinical trials with continuous or binary endpoints

BACKGROUND

Adaptive clinical trials may use conditional power (CP) to make decisions at interim analyses, requiring assumptions about the treatment effect for remaining patients. It is critical that these assumptions are understood by those using CP in decision-making, as well as timings of these decisions.

METHODS

Data for 21 outcomes from 14 published clinical trials were made available for re-analysis. CP curves for accruing outcome information were calculated using and compared with a pre-specified objective criteria for original and transformed versions of the trial data using four future treatment effect assumptions: (i) observed current trend, (ii) hypothesised effect, (iii) 80% optimistic confidence limit, (iv) 90% optimistic confidence limit.

RESULTS

The hypothesised effect assumption met objective criteria when the true effect was close to that planned, but not when smaller than planned. The opposite was seen using the current trend assumption. Optimistic confidence limit assumptions appeared to offer a compromise between the two, performing well against objective criteria when the end observed effect was as planned or smaller.

CONCLUSION

The current trend assumption could be the preferable assumption when there is a wish to stop early for futility. Interim analyses could be undertaken as early as 30% of patients have data available. Optimistic confidence limit assumptions should be considered when using CP to make trial decisions, although later interim timings should be considered where logistically feasible.

Development of a Home-Based Stress Management Toolkit for Dementia Caring Dyads: Protocol for a Pilot Intervention Development and Feasibility Study

BACKGROUND

People living with dementia (PLWD) and their care partners (dementia caring dyads) are at a heightened risk of experiencing stress-related symptoms and conditions. Yet, many dyadic stress management interventions have had limited uptake by health care systems and in the community. An intervention that combines simple, safe, easy-to-use, nonpharmacologic tools (eg, animatronic social pets, weighted blankets and garments, aromatherapy and bright light therapy devices, acupressure, and massage tools) that can be used in the home may be a promising approach to promote stress management among dementia caring dyads.

OBJECTIVE

The proposed study aims to develop and user test a dyadic toolkit intervention composed of simple, tangible stress management tools for community-dwelling PLWD and their care partners. This study will also explore the feasibility of collecting several stress-related outcome measures to inform measurement selection for future studies.

METHODS

A human-centered design (HCD) approach will be used to increase the likelihood of developing an intervention that will be translatable to real-world settings. This study consists of 2 phases. The first phase will address the discover, define, and design stages of HCD using qualitative focus groups with dementia caring dyads (N=12-16 dyads). Dyadic focus groups (3-4 groups anticipated) will be convened to understand participants' stress experiences and to co-design a stress management toolkit prototype. Rapid qualitative analysis will be used to analyze focus group data. In phase 2, the toolkit prototype will be user tested for 2 weeks in a new sample to address the validation step of HCD. A within-subjects (n=10 dyads), pre-post design will be used with measures of usability (frequency of toolkit use), feasibility (enrollment and withdrawal rates, adverse events/injuries), and acceptability (satisfaction, benefit) collected via questionnaires (at the end of weeks 1 and 2 of user testing) and focus groups (n=3-4 dyads/group at the end of week 2). The feasibility of collecting participant-reported, stress-related outcomes (neuropsychiatric symptoms of dementia, caregiver stress, dyadic relationship strain) and salivary cortisol as a physiologic measure of stress will be assessed at baseline and after user testing.

RESULTS

This study will yield a working prototype of a stress management toolkit for dementia caring dyads, as well as preliminary data to support the feasibility and acceptability of the intervention. User testing will elucidate areas to refine the prototype and provide data to inform preliminary testing of the intervention. As of September 2022, this study has received institutional ethics board approval with phase 1 recruitment anticipated to begin January 2023.

CONCLUSIONS

Few interventions have focused on combining simple, safe, low burden tools to promote stress management among community-dwelling dementia caring dyads. By involving families and exploring feasibility and acceptability at the onset of development, this intervention will have greater potential to be implemented and sustained in the future.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05465551; https://clinicaltrials.gov/ct2/show/NCT05465551.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/43098.

Mindfulness Meditation Interventions for Long COVID: Biobehavioral Gene Expression and Neuroimmune Functioning

Some individuals infected with SARS CoV-2 have developed Post-Acute Sequelae of SARS CoV-2 infection (PASC) or what has been referred to as Long COVID. Efforts are underway to find effective treatment strategies for those with Long COVID. One possible approach involves alternative medical interventions, which have been widely used to treat and manage symptoms of a variety of medical problems including post-viral infections. Meditation has been found to reduce fatigue and unrefreshing sleep, and for those with post-viral infections, it has enhanced immunity, and reduced inflammatory-driven pathogenesis. Our article summarizes the literature on what is known about mindfulness meditation interventions, and reviews evidence on how it may apply to those with Long COVID and Myalgic Encephalomyelitis/Chronic Fatigue Syndrome (ME/CFS). Evidence is reviewed suggesting effective and sustainable outcomes may be achieved for symptomatology and underlying pathology of post-viral fatigue (PASC and ME/CFS).

Associations of microbial and indoleamine-2,3-dioxygenase-derived tryptophan metabolites with immune activation in healthy adults

Background

Tryptophan (Trp) metabolites from intestinal bacteria (indole, indole acetic acid [IAA] and indole propionic acid [IPA]), and the Trp metabolite kynurenine (Kyn) from the indoleamine 2,3-dioxygenase (IDO) pathway, are aryl hydrocarbon receptor (AhR) agonists and thus, can regulate immune activity via the AhR pathway. We hypothesized that plasma concentrations of these metabolites would be associated with markers of immune activation in a cohort of healthy adults in a manner consistent with AhR-mediated immune-regulation. We also hypothesized that the plasma Kyn/Trp ratio, a marker of IDO activity, would be associated with immune markers reflecting IDO activation in innate immune cells. Finally, we hypothesized that some intestinal bacteria would be associated with plasma indole, IPA and IAA, and that these bacteria themselves would be associated with immune markers.

Methods

A novel set of 88 immune markers, and plasma Trp metabolites, were measured in 362 healthy adults. Bacterial taxa from stool were identified by 16S rRNA gene analysis. Multiple linear regression analysis was used to identify significant associations with immune markers.

Results

The sum of indole and IAA was positively associated with natural killer T-cells levels. Kyn and Kyn/Trp were positively associated with neopterin and IP-10, markers of type 1 immunity, and TNF-α and C-reactive protein (CRP), markers of the acute phase response, and the regulatory cytokine IL-10. Three bacteria negatively associated with Trp metabolites were associated with markers of immune activation: the family Lachnospiraceae with higher lymphocyte counts but lower level of activated CD4 T-cells, the genus Dorea with higher production of IFN-γ by T-cells in PBMC cultures, and the genus Ruminococcus with higher production IL-6 in PBMC cultures stimulated with bacterial lipopolysaccharide (LPS).

Conclusions

In this cohort of healthy adults bacterial Trp metabolites were not strongly associated with immune markers. Conversely, the Kyn/Trp ratio was strongly associated with markers of systemic inflammation and the acute phase response, consistent with IDO activation in innate immune cells. Finally, commensal bacteria associated with lower plasma (and perhaps intestinal) levels of bacterial Trp metabolites were associated with greater immune activation, possibly reflecting decreased regulatory immune activity related to lower intestinal levels of bacterial indole metabolites.

A descriptive study of samples sizes used in agreement studies published in the PubMed repository

INTRODUCTION

A sample size justification is required for all studies and should give the minimum number of subjects to be recruited for the study to achieve its primary objective. The aim of this review is to describe sample sizes from agreement studies with continuous or categorical endpoints and different methods of assessing agreement, and to determine whether sample size justification was provided.

METHODS

Data were gathered from the PubMed repository with a time interval of 28th September 2018 to 28th September 2020. The search returned 5257 studies of which 82 studies were eligible for final assessment after duplicates and ineligible studies were excluded.

RESULTS

We observed a wide range of sample sizes. Forty-six studies (56%) used a continuous outcome measure, 28 (34%) used categorical and eight (10%) used both. Median sample sizes were 50 (IQR 25 to 100) for continuous endpoints and 119 (IQR 50 to 271) for categorical endpoints. Bland-Altman limits of agreement (median sample size 65; IQR 35 to 124) were the most common method of statistical analysis for continuous variables and Kappa coefficients for categorical variables (median sample size 71; IQR 50 to 233). Of the 82 studies assessed, only 27 (33%) gave justification for their sample size.

CONCLUSIONS

Despite the importance of a sample size justification, we found that two-thirds of agreement studies did not provide one. We recommend that all agreement studies provide rationale for their sample size even if they do not include a formal sample size calculation.

A 50-step walking test for analysis of recovery after decompressive surgery for thoracolumbar disc herniation in dogs

BACKGROUND

Despite its importance, there is no agreed definition of recovery of ambulation in dogs with spinal cord injury.

OBJECTIVES

To validate a new walking test in dogs recovering from thoracolumbar spinal cord injury.

ANIMALS

Two hundred twenty-four dogs weighing <20 kg: 120 normally ambulatory dogs, plus 104 dogs undergoing decompressive surgery for acute thoracolumbar intervertebral disc herniation.

METHODS

Prospective cohort studies. The distance each freely-ambulatory dog walked during 50 step cycles was regressed on ulna length. For each postsurgical dog, we recorded when the calculated 50-step distance was completed without falling, or their inability to complete this distance by 4 months or more after surgery. Bayesian analysis compared outcomes for presurgical neurologic categories; association of recovery with several preoperative variables was explored using logistic and time-to-event regression.

RESULTS

For control dogs, 50-step distance (m) = 1.384 × ulnar length (cm) + 2.773. In postsurgical dogs, the 50-step test provided decisive evidence that deep pain-negative dogs were less likely to recover ambulation than dogs with intact pain perception (12/29 recovered vs 71/75; Bayes factor [BF] = 5.9 × 106 ) and, if they did recover, it took much longer (median 91 days vs median 14 days; BF = 1.5 × 103 ). Exploratory analysis suggested that presurgical neurologic status (subhazard ratio [SHR] = 0.022; P < .001) and duration of presurgical anesthesia (SHR = 0.740; P = .04) were associated with rapidity of recovery.

CONCLUSIONS AND CLINICAL IMPORTANCE

This straightforward 50-step walking test provides robust data on ambulatory recovery well-suited to large scale pragmatic trials on treatment of thoracolumbar spinal cord injury in dogs.

Exercise versus usual care after non-reconstructive breast cancer surgery (UK PROSPER): multicentre randomised controlled trial and economic evaluation

OBJECTIVE

To evaluate whether a structured exercise programme improved functional and health related quality of life outcomes compared with usual care for women at high risk of upper limb disability after breast cancer surgery.

DESIGN

Multicentre, pragmatic, superiority, randomised controlled trial with economic evaluation.

SETTING

17 UK National Health Service cancer centres.

PARTICIPANTS

392 women undergoing breast cancer surgery, at risk of postoperative upper limb morbidity, randomised (1:1) to usual care with structured exercise (n=196) or usual care alone (n=196).

INTERVENTIONS

Usual care (information leaflets) only or usual care plus a physiotherapy led exercise programme, incorporating stretching, strengthening, physical activity, and behavioural change techniques to support adherence to exercise, introduced at 7-10 days postoperatively, with two further appointments at one and three months.

MAIN OUTCOME MEASURES

Disability of Arm, Hand and Shoulder (DASH) questionnaire at 12 months, analysed by intention to treat. Secondary outcomes included DASH subscales, pain, complications, health related quality of life, and resource use, from a health and personal social services perspective.

RESULTS

Between 26 January 2016 and 31 July 2017, 951 patients were screened and 392 (mean age 58.1 years) were randomly allocated, with 382 (97%) eligible for intention to treat analysis. 181 (95%) of 191 participants allocated to exercise attended at least one appointment. Upper limb function improved after exercise compared with usual care (mean DASH 16.3 (SD 17.6) for exercise (n=132); 23.7 (22.9) usual care (n=138); adjusted mean difference 7.81, 95% confidence interval 3.17 to 12.44; P=0.001). Secondary outcomes favoured exercise over usual care, with lower pain intensity at 12 months (adjusted mean difference on numerical rating scale -0.68, -1.23 to -0.12; P=0.02) and fewer arm disability symptoms at 12 months (adjusted mean difference on Functional Assessment of Cancer Therapy-Breast+4 (FACT-B+4) -2.02, -3.11 to -0.93; P=0.001). No increase in complications, lymphoedema, or adverse events was noted in participants allocated to exercise. Exercise accrued lower costs per patient (on average -£387 (€457; $533) (95% confidence interval -£2491 to £1718; 2015 pricing) and was cost effective compared with usual care.

CONCLUSIONS

The PROSPER exercise programme was clinically effective and cost effective and reduced upper limb disability one year after breast cancer treatment in patients at risk of treatment related postoperative complications.

TRIAL REGISTRATION

ISRCTN Registry ISRCTN35358984.

Contamination: How much can an individually randomized trial tolerate?

Cluster randomization results in an increase in sample size compared to individual randomization, referred to as an efficiency loss. This efficiency loss is typically presented under an assumption of no contamination in the individually randomized trial. An alternative comparator is the sample size needed under individual randomization to detect the attenuated treatment effect due to contamination. A general framework is provided for determining the extent of contamination that can be tolerated in an individually randomized trial before a cluster randomized design yields a larger sample size. Results are presented for a variety of cluster trial designs including parallel arm, stepped-wedge and cluster crossover trials. Results reinforce what is expected: individually randomized trials can tolerate a surprisingly large amount of contamination before they become less efficient than cluster designs. We determine the point at which the contamination means an individual randomized design to detect an attenuated effect requires a larger sample size than cluster randomization under a nonattenuated effect. This critical rate is a simple function of the design effect for clustering and the design effect for multiple periods as well as design effects for stratification or repeated measures under individual randomization. These findings are important for pragmatic comparisons between a novel treatment and usual care as any bias due to contamination will only attenuate the true treatment effect. This is a bias that operates in a predictable direction. Yet, cluster randomized designs with post-randomization recruitment without blinding, are at high risk of bias due to the differential recruitment across treatment arms. This sort of bias operates in an unpredictable direction. Thus, with knowledge that cluster randomized trials are generally at a greater risk of biases that can operate in a nonpredictable direction, results presented here suggest that even in situations where there is a risk of contamination, individual randomization might still be the design of choice.

Sirolimus for treatment of patients with inclusion body myositis: a randomised, double-blind, placebo-controlled, proof-of-concept, phase 2b trial

BACKGROUND

Inclusion body myositis is the most frequent myositis in patients older than 50 years. Classical immunosuppressants are ineffective in treating inclusion body myositis, and to date there are no recommendations for pharmacological approaches to treatment. When used after organ transplantation, sirolimus can block the proliferation of effector T cells, while preserving T regulatory cells, and induce autophagy, all of which are processes that are impaired in inclusion body myositis. In this pilot study, we aimed to test the efficacy of sirolimus in patients with inclusion body myositis.

METHODS

This randomised, double-blind, placebo-controlled, proof-of-concept, phase 2b trial was done at a single hospital in Paris, France. The study included men and women (aged 45-80 years) who had a defined diagnosis of inclusion body myositis according to established criteria. Eligible participants were randomly assigned (1:1) to receive once-daily oral sirolimus 2 mg or placebo. Centralised balanced block randomisation (blocks of four) was computer generated without stratification. The study comprised a 15-day screening period (days -15 to 0) and a 52-week treatment period (day 0 to month 12). The primary endpoint was the relative percentage change from baseline to month 12 in maximal voluntary isometric knee extension strength. Secondary endpoints included the following assessments at months 6 and 12: 6-min walking distance, isometric muscle strength for hand grip (finger flexors), knee flexion and elbow flexion and extension, forced vital capacity, muscle replacement with fat measured by quantitative nuclear MRI, Inclusion Body Myositis Weakness Composite Index (IBMWCI), Inclusion Body Myositis Functional Rating Scale (IBMFRS), Health Assessment Questionnaire without Disability Index (HAQ-DI), and analyses of T-cell subpopulations by mass cytometry. The primary analysis was done on the intention-to-treat population. The trial is registered at ClinicalTrials.gov, NCT02481453.

FINDINGS

Between July 15, 2015, and May 13, 2016, we screened 285 patients, 44 of whom were randomly allocated to sirolimus (22 patients) or placebo (22 patients). We observed no difference in the primary outcome of relative percentage change from baseline to month 12 of the maximal voluntary isometric knee extension strength (median difference 3·78, 95% CI -10·61 to 17·31; p=0·85). For secondary outcomes, differences between the groups were not significant for changes in strength of other muscle groups (grip, elbow flexion and extension, or knee flexion), IBMWCI, IBMFRS, and lower limb muscle fat fraction. However, we observed significant differences in favour of sirolimus between the study groups for HAQ-DI, forced vital capacity, thigh fat fraction, and 6-min walking distance. Ten (45%) of 22 patients in the sirolimus group had a serious adverse event compared with six (27%) of 22 patients in the placebo group. Four (18%) patients in the sirolimus group stopped their treatment because of adverse events (severe mouth ulcers, aseptic pneumonia, renal insufficiency, and peripheral lower limb oedema), which resolved after treatment discontinuation. Canker sores were the most frequent side-effect and were mainly mild or moderate in ten patients.

INTERPRETATION

We found no evidence for efficacy of sirolimus for treating inclusion body myositis based on maximal voluntary isometric knee extension strength and other muscle strength measures, and the side-effects of treatment were substantial for some patients. However, we believe there was enough evidence of benefit in certain secondary outcomes to pursue a multicentre phase 3 trial to further assess the safety and efficacy of sirolimus.

FUNDING

Institut national de la santé et de la recherche médicale, Direction générale de l'offre de soins, and Association Française contre les Myopathies.

Mind over Matter: Testing the Efficacy of an Online Randomized Controlled Trial to Reduce Distraction from Smartphone Use

Evidence suggests a growing call for the prevention of excessive smartphone and social media use and the ensuing distraction that arises affecting academic achievement and productivity. A ten-day online randomized controlled trial with the use of smartphone apps, engaging participants in mindfulness exercises, self-monitoring and mood tracking, was implemented amongst UK university students (n = 143). Participants were asked to complete online pre- and post-intervention assessments. Results indicated high effect sizes in reduction of smartphone distraction and improvement scores on a number of self-reported secondary psychological outcomes. The intervention was not effective in reducing habitual behaviours, nomophobia, or time spent on social media. Mediation analyses demonstrated that: (i) emotional self-awareness but not mindful attention mediated the relationship between intervention effects and smartphone distraction, and (ii) online vigilance mediated the relationship between smartphone distraction and problematic social media use. The present study provides preliminary evidence of the efficacy of an intervention for decreased smartphone distraction and highlights psychological processes involved in this emergent phenomenon in the smartphone literature. Online interventions may serve as complementary strategies to reduce distraction levels and promote insight into online engagement. More research is required to elucidate the mechanisms of digital distraction and assess its implications in problematic use.

Explaining the variation in the attained power of a stepped-wedge trial with unequal cluster sizes

Background

In a cross-sectional stepped-wedge trial with unequal cluster sizes, attained power in the trial depends on the realized allocation of the clusters. This attained power may differ from the expected power calculated using standard formulae by averaging the attained powers over all allocations the randomization algorithm can generate. We investigated the effect of design factors and allocation characteristics on attained power and developed models to predict attained power based on allocation characteristics.

Method

Based on data simulated and analyzed using linear mixed-effects models, we evaluated the distribution of attained powers under different scenarios with varying intraclass correlation coefficient (ICC) of the responses, coefficient of variation (CV) of the cluster sizes, number of cluster-size groups, distributions of group sizes, and number of clusters. We explored the relationship between attained power and two allocation characteristics: the individual-level correlation between treatment status and time period, and the absolute treatment group imbalance. When computational time was excessive due to a scenario having a large number of possible allocations, we developed regression models to predict attained power using the treatment-vs-time period correlation and absolute treatment group imbalance as predictors.

Results

The risk of attained power falling more than 5% below the expected or nominal power decreased as the ICC or number of clusters increased and as the CV decreased. Attained power was strongly affected by the treatment-vs-time period correlation. The absolute treatment group imbalance had much less impact on attained power. The attained power for any allocation was predicted accurately using a logistic regression model with the treatment-vs-time period correlation and the absolute treatment group imbalance as predictors.

Conclusion

In a stepped-wedge trial with unequal cluster sizes, the risk that randomization yields an allocation with inadequate attained power depends on the ICC, the CV of the cluster sizes, and number of clusters. To reduce the computational burden of simulating attained power for allocations, the attained power can be predicted via regression modeling. Trial designers can reduce the risk of low attained power by restricting the randomization algorithm to avoid allocations with large treatment-vs-time period correlations.

A systematic survey of randomised trials that stopped early for reasons of futility

Background

Randomised trial protocols may incorporate interim analyses, with the potential to stop the study for futility if early data show insufficient promise of a treatment benefit. Previously, we have shown that this approach will theoretically lead to mis-estimation of the treatment effect. We now wished to ascertain the importance of this phenomenon in practice.

Methods

We reviewed the methods and results in a set of trials that had stopped for futility, identified through an extensive literature search. We recorded clinical areas, interventions, study design, outcomes, trial setting, sponsorship, planned and actual treatment effects, sample sizes; power; and if there was a data safety monitoring board, or a published protocol. We identified: if interim analyses were pre-specified, and how many analyses actually occurred; what pre-specified criteria might define futility; if a futility analysis formed the basis for stopping; who made the decision to stop; and the conditional power of each study, i.e. the probability of statistically significant results if the study were to continue to its complete sample size.

Results

We identified 52 eligible trials, covering many clinical areas. Most trials had multiple centres, tested drugs, and 40% were industry sponsored. There were 75% where at least one interim analysis was planned a priori; a majority had only one interim analysis, typically with about half the target total sample size. A majority of trials did not pre-define a stopping rule, and a variety of reasons were given for stopping. Few studies calculated and reported low conditional power to justify the early stop. When conditional power could be calculated, it was typically low, especially under the current trend hypothesis. However, under the original design hypothesis, a few studies had relatively high conditional power. Data collection often continued after the interim analysis.

Conclusions

Although other factors will typically be involved, we conclude that, from the perspective of conditional power, stopping early for futility was probably reasonable in most cases, but documentation of the basis for stopping was often missing or vague. Interpretation of truncated trials would be enhanced by improved reporting of stopping protocols, and of their actual execution.

Clinical and Cost-Effectiveness of Personalized Tele-Based Coaching for Farmers, Foresters and Gardeners to Prevent Depression: Study Protocol of an 18-Month Follow-Up Pragmatic Randomized Controlled Trial (TEC-A)

Background: Farmers show high levels of depressive symptoms and mental health problems in various studies. This study is part of a nationwide prevention project carried out by a German social insurance company for farmers, foresters, and gardeners (SVLFG) to implement internet- and tele-based services among others to improve mental health in this population. The aim of the present study is to evaluate the (cost-)effectiveness of personalized tele-based coaching for reducing depressive symptom severity and preventing the onset of clinical depression, compared to enhanced treatment as usual. Methods: In a two-armed, pragmatic randomized controlled trial (N = 312) with follow-ups at post-treatment (6 months), 12 and 18 months, insured farmers, foresters, and gardeners, collaborating family members and pensioners with elevated depressive symptoms (PHQ-9 ≥ 5) will be randomly allocated to personalized tele-based coaching or enhanced treatment as usual. The coaching is provided by psychologists and consists of up to 34 tele-based sessions for 25-50 min delivered over 6 months. Primary outcome is depressive symptom severity at post-treatment. Secondary outcomes include depression onset, anxiety, stress, and quality of life. A health-economic evaluation will be conducted from a societal perspective. Discussion: This study is the first pragmatic randomized controlled trial evaluating the (cost-)effectiveness of a nationwide tele-based preventive service for farmers. If proven effective, the implementation of personalized tele-based coaching has the potential to reduce disease burden and health care costs both at an individual and societal level. Clinical Trial Registration: German Clinical Trial Registration: DRKS00015655.

Methods to adjust for multiple comparisons in the analysis and sample size calculation of randomised controlled trials with multiple primary outcomes

BACKGROUND

Multiple primary outcomes may be specified in randomised controlled trials (RCTs). When analysing multiple outcomes it's important to control the family wise error rate (FWER). A popular approach to do this is to adjust the p-values corresponding to each statistical test used to investigate the intervention effects by using the Bonferroni correction. It's also important to consider the power of the trial to detect true intervention effects. In the context of multiple outcomes, depending on the clinical objective, the power can be defined as: 'disjunctive power', the probability of detecting at least one true intervention effect across all the outcomes or 'marginal power' the probability of finding a true intervention effect on a nominated outcome. We provide practical recommendations on which method may be used to adjust for multiple comparisons in the sample size calculation and the analysis of RCTs with multiple primary outcomes. We also discuss the implications on the sample size for obtaining 90% disjunctive power and 90% marginal power.

METHODS

We use simulation studies to investigate the disjunctive power, marginal power and FWER obtained after applying Bonferroni, Holm, Hochberg, Dubey/Armitage-Parmar and Stepdown-minP adjustment methods. Different simulation scenarios were constructed by varying the number of outcomes, degree of correlation between the outcomes, intervention effect sizes and proportion of missing data.

RESULTS

The Bonferroni and Holm methods provide the same disjunctive power. The Hochberg and Hommel methods provide power gains for the analysis, albeit small, in comparison to the Bonferroni method. The Stepdown-minP procedure performs well for complete data. However, it removes participants with missing values prior to the analysis resulting in a loss of power when there are missing data. The sample size requirement to achieve the desired disjunctive power may be smaller than that required to achieve the desired marginal power. The choice between whether to specify a disjunctive or marginal power should depend on the clincial objective.

New guidance to improve sample size calculations for trials: eliciting the target difference

BACKGROUND

Sample size calculations are central to the design of health research trials. To ensure that the trial provides good evidence to answer the trial's research question, the target effect size (difference in means or proportions, odds ratio, relative risk or hazard ratio between trial arms) must be specified under the conventional approach to determining the sample size. However, until now, there has not been comprehensive guidance on how to specify this effect.

MAIN TEXT

This is a commentary on a collection of papers from two important projects, DELTA (Difference ELicitation in TriAls) and DELTA2 that aim to provide evidence-based guidance on systematically determining the target effect size, or difference and the resultant sample sizes for trials. In addition to surveying methods that researchers are using in practice, the research team met with various experts (statisticians, methodologists, clinicians and funders); reviewed guidelines from funding agencies; and reviewed recent methodological literature. The DELTA2 guidance stresses specifying important and realistic differences, and undertaking sensitivity analyses in calculating sample sizes. It gives recommendations on how to find appropriate differences, conduct the sample size calculation(s) and how to report these in grant applications, protocols and manuscripts. It is hoped that this will contribute not only to better powered studies, but better reporting and reproducibility and thinking about what a trial should be designed to achieve.

CONCLUSIONS

The DELTA researchers have developed a set of comprehensive guidance documents that are welcome and will almost certainly improve the way that trials are designed and reported.