Letter to the Editor Regarding "Effect of Ropivacaine Intercostal Nerve Block Combined with Patient Controlled Intravenous Analgesia on Postoperative Analgesia After Breast Augmentation"

We read the recently published article "Effect of Ropivacaine Intercostal Nerve Block Combined with Patient Controlled Intravenous Analgesia on Postoperative Analgesia after Breast Augmentation" by You et al. We have noticed several issues in the methods and results of this study and would appreciate the responses from the authors. We question several aspects, opioid-sparing effect, sufentanil consumption, sample size evaluation, exclusion reasons, and side effects.

Accuracy of Event Rate and Effect Size Estimation in Major Cardiovascular Trials: A Systematic Review

Importance

For the design of a randomized clinical trial (RCT), estimation of the expected event rate and effect size of an intervention is needed to calculate the sample size. Overestimation may lead to an underpowered trial.

Objective

To evaluate the accuracy of published estimates of event rate and effect size in contemporary cardiovascular RCTs.

Evidence Review

A systematic search was conducted in MEDLINE for multicenter cardiovascular RCTs associated with MeSH (Medical Subject Headings) terms for cardiovascular diseases published in the New England Journal of Medicine, JAMA, or the Lancet between January 1, 2010, and December 31, 2019. Identified trials underwent abstract review; eligible trials then underwent full review, and those with insufficiently reported data were excluded. Data were extracted from the original publication or the study protocol, and a random-effects model was used for data pooling. This review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses reporting guideline. The primary outcome was the accuracy of event rate and effect size estimation. Accuracy was determined by comparing the observed event rate in the control group and the effect size with their hypothesized values. Linear regression was used to determine the association between estimation accuracy and trial characteristics.

Findings

Of the 873 RCTs identified, 374 underwent full review and 30 were subsequently excluded, resulting in 344 trials for analysis. The median observed event rate was 9.0% (IQR, 4.3% to 21.4%), which was significantly lower than the estimated event rate of 11.0% (IQR, 6.0% to 25.0%) with a median deviation of -12.3% (95% CI, -16.4% to -5.6%; P < .001). More than half of the trials (196 [61.1%]) overestimated the expected event rate. Accuracy of event rate estimation was associated with a higher likelihood of refuting the null hypothesis (0.13 [95% CI, 0.01 to 0.25]; P = .03). The median observed effect size in superiority trials was 0.91 (IQR, 0.74 to 0.99), which was significantly lower than the estimated effect size of 0.72 (IQR, 0.60 to 0.80), indicating a median overestimation of 23.1% (95% CI, 17.9% to 28.3%). A total of 216 trials (82.1%) overestimated the effect size.

Conclusions and Relevance

In this systematic review of contemporary cardiovascular RCTs, event rates of the primary end point and effect sizes of an intervention were frequently overestimated. This overestimation may have contributed to the inability to adequately test the trial hypothesis.

Simulation-Based Analysis of Trial Design in Regional Anesthesia

Background

In regional anesthesia, the efficacy of novel blocks is typically evaluated using randomized controlled trials (RCTs), the findings of which are aggregated in systematic reviews and meta-analyses. Systematic review authors frequently point out the small sample size of RCTs as limiting conclusions from this literature. We sought to determine via statistical simulation if small sample size could be an expected property of RCTs focusing on novel blocks with typical effect sizes.

Methods

We simulated the conduct of a series of RCTs comparing a novel block versus placebo on a single continuous outcome measure. Simulation analysis inputs were obtained from a systematic bibliographic search of meta-analyses. Primary outcomes were the predicted number of large trials (empirically defined as N ≥ 256) and total patient enrollment.

Results

Simulation analysis predicted that a novel block would be tested in 16 RCTs enrolling a median of 970 patients (interquartile range (IQR) across 1000 simulations: 806, 1269), with no large trials. Among possible modifications to trial design, decreasing the statistical significance threshold from p < 0.05 to p < 0.005 was most effective at increasing the total number of patients represented in the final meta-analysis, but was associated with early termination of the trial sequence due to futility in block vs. block comparisons.

Conclusion

Small sample size of regional anesthesia RCTs comparing novel block to placebo is a rational outcome of trial design. Feasibly large trials are unlikely to change conclusions regarding block vs. placebo comparisons.

Systematic review of sample size calculations and reporting in randomized controlled trials in ophthalmology over a 20-year period

PURPOSE

Randomized Controlled Trials (RCTs) are considered the gold standard for the practice of evidence-based medicine. The purpose of this study is to systematically assess the reporting of sample size calculations in ophthalmology RCTs in 5 leading journals over a 20-year period. Reviewing sample size calculations in ophthalmology RCTs will shed light on the methodological quality of RCTs and, by extension, on the validity of published results.

METHODS

The MEDLINE database was searched to identify full reports of RCTs in the journals Ophthalmology, JAMA Ophthalmology, American Journal of Ophthalmology, Investigative Ophthalmology and Visual Science, and British Journal of Ophthalmology between January and December of the years 2000, 2010 and 2020. Screening identified 559 articles out of which 289 met the inclusion criteria for this systematic review. Data regarding sample size calculation reporting and trial characteristics was extracted for each trial by independent investigators.

RESULTS

In 2020, 77.9% of the RCTs reported sample size calculations as compared with 37% in 2000 (p < 0.001) and 60.7% in 2010 (p = 0.012). Studies reporting all necessary parameters for sample size recalculation increased significantly from 17.2% in 2000 to 39.3% in 2010 and 43.0% in 2020 (p < 0.001). Reporting of funding was greater in 2020 (98.8%) compared with 2010 (89.3%) and 2000 (53.1%). Registration in a clinical trials database occurred more frequently in 2020 (94.2%) compared to 2000 (1.2%; p < 0.001) and 2010 (68%; p < 0.001). In 2020, 38.4% of studies reported different sample sizes in the online registry from the published article. Overall, the most studied area in 2000 was glaucoma (29.6% of RCTs), whereas in 2010 and 2020, it was retina (40.2 and 37.2% of the RCTs, respectively). The number of patients enrolled in a study and the number of eyes studied was significantly greater in 2020 compared to 2000 and 2010 (p < 0.001).

CONCLUSION

Sample size calculation reporting in ophthalmology RCTs has improved significantly between the years 2000 and 2020 and is comparable to other fields in medicine. However, reporting of certain parameters remains inconsistent with current publication guidelines.

Ultrasound-guided lumbar erector spinae plane block versus caudal block for postoperative analgesia in pediatric hip and proximal femur surgery: a randomized controlled study

BACKGROUND

According to previous research, 20% of infants experience prolonged postsurgical pain 6-12 months after major surgery, which is linked to functional impairment and a lower quality of life. The aim of our study is to evaluate whether the analgesic effect of the erector spinae plane block (ESPB) is superior to that of caudal epidural anesthesia (CEA) in pediatric patients undergoing hip or proximal femoral surgeries.

METHODS

Seventy-six children ranging in age from 1 to 7 years scheduled for hip or proximal femur surgery were randomly assigned to receive either a unilateral ultrasound-guided ESPB or CEA with bupivacaine 0.25% at a dose of 0.5 ml/kg. The primary outcome was the Face, Legs, Activity, Cry, and Consolability (FLACC) scale 2 h postoperatively. The secondary outcomes were pain scores every 15 min for the first hour and then at 6, 12, and 24 h postoperatively; the block failure rate; time to perform a successful block; and time to first rescue analgesia.

RESULTS

The FLACC score 2 h post¬operatively was not superior in the ESPB group compared to the CEA group; indeed, it was significantly higher in the ESPB group at 15 and 30 min post-operation (P = 0.005, 0.004, respectively). Additionally, the time to first rescue analgesia was prolonged in the CEA group (P < 0.001). The time to perform a successful block was comparable between the groups.

CONCLUSIONS

The analgesic effect of the ESPB was not superior to that of CEA in pediatric patients undergoing hip and proximal femur surgery.

A Recovery-Oriented Program for People with Bipolar Disorder through Virtual Reality-Based Cognitive Remediation: Results of a Feasibility Randomized Clinical Trial

Background: Cognitive impairment is a frequent consequence of bipolar disorder (BD) that is difficult to prevent and treat. In addition, the quality of the preliminary evidence on the treatment of BD through Cognitive Remediation (CR) with traditional methods is poor. This study aims to evaluate the feasibility of a CR intervention with fully immersive Virtual Reality (VR) as an additional treatment for BD and offers preliminary data on its efficacy. Methods: Feasibility randomized controlled cross-over clinical study, with experimental condition lasting three months, crossed between two groups. Experimental condition: CR fully immersive VR recovery-oriented program plus conventional care; Control condition: conventional care. The control group began the experimental condition after a three months period of conventional care (waiting list). After the randomization of 50 people with BD diagnosis, the final sample consists of 39 participants in the experimental condition and 25 in the control condition because of dropouts. Results: Acceptability and tolerability of the intervention were good. Compared to the waitlist group, the experimental group reported a significant improvement regarding cognitive functions (memory: p = 0.003; attention: p = 0.002, verbal fluency: p = 0.010, executive function: p = 0.003), depressive symptoms (p = 0.030), emotional awareness (p = 0.007) and biological rhythms (p = 0.029). Conclusions: The results are preliminary and cannot be considered exhaustive due to the small sample size. However, the evidence of efficacy, together with the good acceptability of the intervention, is of interest. These results suggest the need to conduct studies with larger samples that can confirm this data. Trial registration: ClinicalTrialsgov NCT05070065, registered in September 2021

Reporting of statistical sample size calculations in publications of trials on age-related macular degeneration, glaucoma and cataract

Background

Transparent and complete publications of randomised controlled trials (RCT) ought to comply with the guidelines of the CONSORT Statement, which stipulates sample size calculation as an important aspect of trial planning. The objective of this study was to analyse and compare the reporting of statistical sample size calculations in RCT papers on the treatment of age-related macular degeneration (AMD), glaucoma and cataract published in 2018.

Material and methods

This study comprises a total of 113 RCT papers (RCT-P) published in 2018 (AMD: 14, glaucoma: 28, cataract: 71), in English or German, and identified through an internet-based literature search in PubMed and EMBASE. The primary outcome measure of the study was the number of trials providing a complete description of the underlying sample case calculation on the basis of the variables required (significance level, expected outcomes, power, and resulting sample size).

Results

Of the RCTs reviewed, 64% (AMD), 61% (glaucoma) and 31% (cataract) provided a justification of the number of patients included. A complete description of the described studies’ sample size calculation including all the necessary values (primary outcome measure of this study) was described by 21% of the AMD, 29% of the cataract and 18% of the glaucoma RCT publications (in total: 24 of 113 (21%) at a confidence interval of 95%: [13%; 29%]).

Conclusion

All three treatment areas analysed lacked reporting quality regarding the justification of the number of patients included in a clinical trial based on a sample size calculation required for ethical reasons. More than half of all RCT publications reviewed did not provide all of the required information on statistical sample size calculation, and thus lacked transparency and completeness. It is therefore urgently required to involve methodologists in a study’s planning and publishing processes to ensure that methodology descriptions are transparent and of high quality.

A priori power considerations in Endodontic Research. Do we miss the timeline?

AIM

To record the prevalence of a priori power calculations in manuscripts published in three endodontic journals between 2018 and 2020 and detect further associations with a number of study characteristics including journal, publication year, study design, geographic region, number of centres and authors, whether the primary outcome pertained to a statistically significant effect and whether confidence intervals (CIs) were reported.

METHODOLOGY

The contents of the three leading endodontic journals with the highest impact factor (International Endodontic Journal, IEJ; Journal of Endodontics, JOE; and Australian Endodontic Journal, AEJ) were assessed from January 2018 to December 2020. The proportion of articles reporting a priori power calculations were recorded, and relevant associations as described above were assessed. Univariable and multivariable logistic regression were used to identify significant predictors, whilst interaction and linear trend effects were also considered.

RESULTS

A total of 716 original research articles were included. The vast majority were published in the JOE (417/716; 58.2%), followed by the IEJ (225/716; 31.4%) and the AEJ (74/716; 10.4%). Overall, a priori power considerations were reported in 243 out of 716 articles (33.9%). The IEJ presented 1.61 times higher odds for including a priori power considerations compared to JOE (adjusted odds ratio, OR = 1.61; 95%CI: 1.11, 2.34), whilst for the AEJ, the corresponding odds were 41% lower in comparison to JOE (adjusted OR = 0.59; 95%CI: 0.31, 1.14). For each additional year indicating more recent publication, the odds for adopting appropriate reporting practices for power considerations were increased by 64% (adjusted OR = 1.64; 95%CIs: 1.32, 2.04). There was strong evidence that interventional research was associated with 10.54 times higher odds for a priori considerations compared to observational study design (adjusted OR = 10.54; 95%CIs: 5.50, 20.19).

CONCLUSIONS

The high prevalence of failure to include a priori power considerations was indicative of suboptimal reporting in endodontic research, in the three endodontic journals analysed. Although the condition improved over time, efforts to incorporate a correct determination of the required sample size at the design stage for any future study should be endorsed by journal editors, authors and the scientific community.

Transmuscular Quadratus Lumborum Block versus Suprainguinal Fascia Iliaca Block for Hip Arthroplasty: A Randomized, Controlled Pilot Study

Background

This study aimed to investigate the analgesic efficacy and motor block profile of single-shot transmuscular quadratus lumborum block (QLB) in comparison with those of suprainguinal fascia iliaca block (FIB) in patients undergoing hip arthroplasty.

Methods

This randomized, double-blinded, controlled trial included adult patients undergoing hip arthroplasty under spinal anesthesia. Patients were allocated to one of two groups according to the regional block received: FIB group (n=19) or QLB group (n=17). Both study groups were compared with regard to the duration of analgesia (primary outcome), block performance time, pain during positioning for spinal anesthesia, total morphine consumption in the first postoperative 24-h period, quadriceps muscle power, and static and dynamic visual analog scale.

Results

Thirty-six patients were included in the final analysis. Both study groups had comparable durations of analgesia. Postoperative visual analog scale (static and dynamic) values were comparable between the two groups in most readings. The block performance time was shorter in the FIB group. The number of patients with pain during positioning for the subarachnoid block was lower in the QLB group. The total morphine requirement during the first 24 h was marginally lower in the FIB group, whereas the quadriceps motor grade was higher in the FIB group than in the QLB group at 4 h and 6 h after surgery.

Conclusion

Both single-shot blocks, namely the suprainguinal FIB and transmuscular QLB, provide effective postoperative analgesia after hip arthroplasty. FIB showed slightly lower 24-h morphine consumption, while QLB showed better quadriceps motor power.

Clinical Trial Registration

The study was registered at clinical trials registry system before enrollment of the first participant (NCT04005326; initial release date, 2 July 2019; https://clinicaltrials.gov/ct2/show/NCT04005326).

Minimal clinically important differences in randomised clinical trials on pain management after total hip and knee arthroplasty: a systematic review

BACKGROUND

Sample size determination is essential for reliable hypothesis testing in clinical trials and should rely on adequate sample size calculations with alpha, beta, variance, and an effect size being the minimal clinically important difference (MCID). This facilitates interpretation of the clinical relevance of statistically significant results. No gold standard for MCIDs exists in postoperative pain research.

METHODS

We searched Cochrane Central Register of Controlled Trials, MEDLINE, and Embase for English language articles on randomised trials investigating analgesic interventions after total hip or knee arthroplasty. Primary outcomes were the reported MCIDs for pain score and cumulated rescue opioid consumption. Secondary outcomes included reported sample size calculations and propensity to report statistical significance without reaching MCID. Trend analyses were conducted using statistical process control.

RESULTS

We included 570 trials. Median MCID for 0-24 h opioid consumption was 10 mg i.v. morphine equivalents for absolute reductions (interquartile range [IQR]: 6.8-14.5) and relative 40% (IQR: 30-50%). Median MCIDs for pain scores were absolute 15 mm at rest (IQR: 10-20) and 18 mm during movement (IQR: 10-20) on a 0-100 mm VAS and relative 30% (IQR: 20-30%). No trends were demonstrated for MCIDs. Adequate sample size calculations were reported in 34% of trials. In 46% of trials with statistically significant primary outcomes, the differences did not reach the predetermined MCID.

CONCLUSIONS

We provide clinician-perceived MCID estimates for rescue opioid consumption and pain scores that can be used for sample size calculations until reliable evidence-based patient-rated MCIDs emerge. Nearly half of the trials with significant findings did not reach the predetermined MCID.

Sample-size estimation is not reported in 24% of randomised controlled trials of inflammatory bowel disease: A systematic review

BACKGROUND

Sample-size estimation is an important factor in designing a clinical trial. A recent study found that 65% of Cochrane systematic reviews had imprecise results.

OBJECTIVE

This study set out to review the whole body of inflammatory bowel disease (IBD) randomised controlled trials systematically in order to identify the reporting of sample-size estimation.

METHODS

We conducted a comprehensive hand search of the Cochrane Library and Cochrane IBD Specialized Trials Register. We extracted information on relevant features and the results of the included studies. We produced descriptive statistics for our results.

RESULTS

A total of 242 randomised controlled trials were included from 44 Cochrane systematic reviews. About 25% of the studies failed to report on sample-size estimation. Of those that did report on sample-size estimation, 33% failed to recruit their target sample size.

CONCLUSIONS

Around half of the randomised controlled trials in IBD either do not report sample-size estimation or reach their recruitment target with the level of detail in reporting being limited.

Randomized controlled trials in non-pharmacological rehabilitation research: a scoping review of the reporting of sample size calculation, randomization procedure, and statistical analyses

INTRODUCTION

The randomized controlled trials (RCTs) are often considered as the gold standard for clinical trials and researchers argue that the quality of RCT reports should be of the highest standards due to their clinical significance. To review the quality of reporting of the sample size calculation methods, and randomization procedures, and assess whether the statistical analyses correlate as reported in the trials' Evidence acquisition and Evidence synthesis sections in non-pharmacological, physiological rehabilitation RCT interventions.

EVIDENCE ACQUISITION

A systematic electronic search was conducted in Cochrane Central from 1 January 2019 to 16 December 2019. Titles and abstracts were analyzed for inclusion independently by two authors, and disagreements were resolved by a third reader. Studies were included if they met the following criteria: 1) assessed and reported a type of non-pharmacological rehabilitation RCT (e.g. physiotherapy); 2) randomized intervention to patients with a disease comparing to healthy or patients without intervention as the comparison group; 3) published in an indexed journal; and 4) original research, available full text, human study, published in 2019, and written in English. The following information was extracted from the included articles: journal impact factor (JIF), sample size calculation methods (SS), randomization procedure (RND), and statistical analyses (STAT) reported. Analyzing the full text, whether SS and RND were reported or not and whether the STAT correlated with the Methods and Results sections. The prevalence of each statistical method was derived from the Methods section of the report and compared if it was reported in the Results section. The continuous variable of JIF was tested for normality and used for independent t-test for equality of means between categories. In addition, using Downs and Black checklist the methodological quality of the articles was assessed and categorized to be poor, fair, good, and excellent based on the checklist's score. Finally, the association between the assessed quality (categorical variable) of the articles and the reporting variables (categorical variables) was analyzed utilizing the Pearson χ².

EVIDENCE SYNTHESIS

One hundred and nighty-four articles were retrieved from the systematic search out of which 99 (51%) were included for data extraction and further analyses. About one in five (20.2%) and two in five (37.4%) did not properly and adequately report the SS and RND while one in five (19.2%) there was at least one mismatch in STAT. The JIF was not significantly associated to the quality of reporting of SS (t=1.974, P=0.051), RND (t=0.309, P=0.758), and STAT (t=-0.275, P=0.784). This finding could indicate that the quality of the journal did not assure the quality of the reporting these methods. However, there was a significant association between the assessed quality of the article measured with the Down's and Black checklist and the reporting of SS (χ²=29.149, DF=2, P<0.0001), RND (χ²=55.079, df=2, P<0.0001) and STAT (χ²=25.778, df=2, P<0.0001).

CONCLUSIONS

Recent reporting quality of non-pharmacological rehabilitation RCTs was investigated. We found that the quality of the article but not the quality of the journal in which it is published in may be associated to the quality of reporting in sample size methods, randomization processes, and statistical analyses reporting. The quality of study reporting may be enhanced utilizing a guideline that addresses the required information in sample size calculation, randomization of individuals, and proper statistical analyses used and reported.

[Influence of impact factor on reporting sample size calculations in publications on studies exemplified by AMD treatment : Cross-sectional investigation on the presence of sample size calculations in publications of RCTs on AMD treatment in journals with low and high impact factors]

BACKGROUND

For scientific and ethical reasons randomized controlled clinical trials (RCTs) should be based on a sample size calculation. The CONSORT statement, an established publication guideline for transparent study reporting, requires a sample size calculation in every study publication.

OBJECTIVE

The availability of sample size calculations in RCT publications on treatment of age-related macular degeneration (AMD) was investigated. The primary hypothesis of this investigation compared the prevalence of reported sample size calculations between journals with higher (≥5) versus lower (<5) impact factors (IF).

MATERIAL AND METHODS

It was examined whether information on sample size calculation was available in a series of 97 publications of RTCs on AMD treatment published between 2004 and 2014.

RESULTS

Only 46 out of 97 (47%) study publications provided information on the reason for the number of patients enrolled. The comparison of publications from journals with an IF ≥ 5 (63%, 30) and from journals with an IF < 5 (40%, 67) showed a statistically significant difference of 23% in the frequencies of available sample size calculations (95% confidence interval, CI 2%; 44%). Of the publications published before 2010, 43% reported a sample size calculation versus 51% of the publications afterwards.

CONCLUSION

Publications in journals with higher IF more frequently reported a sample size calculation. More than 50% of the publications did not report any sample size calculation. Authors and reviewers of publications should pay more attention to the explicit reporting of sample size calculations.

Reporting sample size calculations for randomized controlled trials published in nursing journals: A cross-sectional study

BACKGROUND

Prior sample size calculation is essential to ensure that a randomized controlled trial (RCT) has enough power to detect any statistical differences between two groups while not over-recruiting participants. However, the compliance among RCTs published in nursing field is unknown.

OBJECTIVE

To describe the reporting of sample size methodology and parameters used in RCTs published in nursing journals.

DESIGN

A cross-sectional database search of all RCTs published in nursing journals was conducted.

SETTING

116 journals in the category of nursing.

PARTICIPANTS

Not applicable.

METHODS

A database search was conducted to identify all RCTs published in nursing journals from January 2016 to December 2016. Two-arm RCTs were reviewed to see if sample size estimation was mentioned in the text and if the parameters were sufficient for the estimation. For RCTs with effect size, sample sizes were recomputed and compared with the reported size.

RESULTS

Two hundred and twenty-three RCTs were included in this study, and 143 (64.1%) studies mentioned how they obtained their sample sizes. Among these 143 studies, 132 (92.3%), 133 (93.0%), 33 (23.1%), and 84 (58.7%) specified their levels of significance (α), powers, tails of the test, and effect size related information, respectively. Only 22 (15.4%) provided all the necessary parameters to compute their sample sizes. Out of the 84 RCTs with effect size information, 49 (58.3%) had a relative difference of less than 25%.

DISCUSSION

Around 36% of the RCTs published in nursing journals did not report how they reached their sample sizes. For those that had reported, only 15% provided all the necessary parameters to re-compute their sizes. It is concluded that the methodology and parameters used for sample size determination are inadequately reported in RCTs published in nursing journals.

Validity of sample sizes in publications of randomised controlled trials on the treatment of age-related macular degeneration: cross-sectional evaluation

OBJECTIVE

The aim of this cross-sectional study was to examine the completeness and accuracy of the reporting of sample size calculations in randomised controlled trial (RCT) publications on the treatment of age-related macular degeneration (AMD).

METHODS

A sample of 97 RCTs published between 2004 and 2014 was reviewed for the calculation of their sample size. It was examined whether a (complete) description of the sample size calculation was presented. Furthermore, the sample size was recalculated, whenever possible based on the published details, in order to verify the reported number of patients.

PRIMARY OUTCOME MEASURE

The primary endpoint of this cross-sectional investigation was a described sample size calculation that was reproducible, complete and correct (maximum tolerated deviation between reported and replicated sample size ±2 participants per trial arm).

RESULTS

A total of 50 publications (52%) did not provide any information on the justification of the number of patients included. Only 17 publications (18%) provided all the necessary parameters for recalculation; 8 of 97 (8%, 95%-CI: 4% to 16%) publications achieved the primary endpoint. The median relative deviation between reported and recalculated sample sizes was 1%, with a range from -43% to +66%.

CONCLUSION

Although a transparent sample size legitimation is a crucial determinant of an RCT's methodological validity, more than half of the RCT publications considered failed to report them. Furthermore, reported sample size legitimations were often incomplete or incorrect. In summary, clinical authors should pay more attention to the transparent reporting of sample size calculation, and clinical journal reviewers may opt to reproduce reported sample size calculations.

SYNOPSIS

More than half of the analysed RCT publications on the treatment of AMD did not report a transparent sample size calculation. Only 8% reported a complete and correct sample size calculation.

Should treatment effects be estimated in pilot and feasibility studies?

Background

Feasibility studies and external pilot studies are used increasingly to inform planning decisions related to a definitive randomized controlled trial. These studies can provide information on process measures, such as consent rates, treatment fidelity and compliance, and methods of outcome measurement. Additionally, they can provide initial parameter estimates for a sample size calculation, such as a standard deviation or the ‘success’ rate for a binary outcome in the control group. However, the issue of estimating treatment effects in pilot or feasibility studies is controversial.

Methodological discussion

Between-group estimates of treatment effect from pilot studies are sometimes used to calculate the sample size for a main trial, alongside estimated standard deviations. However, whilst estimating a standard deviation is an empirical matter, a targeted treatment effect should be established in terms of clinical judgement, as a minimum important difference (MID), not through analysis of pilot data. Secondly, between-group effects measured in pilot studies are sometimes used to indicate the magnitude of an effect that might be obtained in a main trial, and a decision on progression made with reference to the associated confidence interval. Such estimates will be imprecise in typically small pilot studies and therefore do not allow a robust decision on a main trial; both a decision to proceed and a decision not to proceed may be made too readily. Thirdly, a within-group change might be estimated from a pilot or a feasibility study in a desire to assess the potential efficacy of a novel intervention prior to testing it in a main trial, but again such estimates are liable to be imprecise and do not allow sound causal inferences.

Conclusion

Treatment effects calculated from pilot or feasibility studies should not be the basis of a sample size calculation for a main trial, as the MID to be detected should be based primarily on clinical judgement rather than statistics. Deciding on progression to a main trial based on these treatment effects is also misguided, as they will normally be imprecise, and may be biased if the pilot or feasibility study is unrepresentative of the main trial.

Systematic review showed that stepped-wedge cluster randomized trials often did not reach their planned sample size

OBJECTIVE

To determine how often stepped-wedge cluster randomized controlled trials reach their planned sample size, and what reasons are reported for choosing a stepped-wedge trial design.

STUDY DESIGN AND SETTING

We conducted a PubMed literature search (period 2012 to 2017) and included articles describing the results of a stepped-wedge cluster randomized trial. We calculated the percentage of studies reaching their prespecified number of participants and clusters, and we summarized the reasons for choosing the stepped-wedge trial design as well as difficulties during enrollment.

RESULTS

Forty-six individual stepped-wedge studies from a total of 53 articles were included in our review. Of the 35 studies, for which recruitment rate could be calculated, 69% recruited their planned number of participants, with 80% having recruited the planned number of clusters. Ethical reasons were the most common motivation for choosing the stepped-wedge trial design. Most important difficulties during study conduct were dropout of clusters and delayed implementation of the intervention.

CONCLUSION

About half of recently published stepped-wedge trials reached their planned sample size indicating that recruitment is also a major problem in these trials. Still, the stepped-wedge trial design can yield practical, ethical, and methodological advantages.

Reporting quality of randomized controlled trials in otolaryngology: review of adherence to the CONSORT statement

Background

Randomized controlled trials are the gold standard in medical and surgical research to assess the efficacy of therapeutic interventions. The reporting of these trials should be of high quality to allow readers’ appropriate interpretation and application.

Methods

The objectives of our study were to assess the extent to which the recent Otolaryngology – Head and Neck Surgery (ORL-HNS) randomized control trials in the top nine journals and in the top Canadian journal comply with the Consolidated Standards of Reporting Trials (CONSORT) statement, and to identify the CONSORT items most in need of improvement. Based on the impact factor and circulation number of 2014, the top nine Otolaryngology journals and the top Canadian Otolaryngology journal were selected and were searched to identify RCTs published in English and between 2010 and 2014. Two authors independently reviewed and extracted data using a standardized data extraction form constructed with the help of a medical librarian. Our outcome was to assess the adherence of articles reporting to the CONSORT items. Descriptive statistics were used.

Results

One hundred and eighty-two Otolaryngologic RCTs were identified in the top nine international journals and in the top Canadian journal. The inter-rater reliability between two raters was 0.32. The extent of adherence to CONSORT Statement ranged from 25 to 93.5% with a mean of 59.0% and a median of 59.4%. Only 6.5% of RCTs described the individual responsible for enrolling and assigning subjects and method of randomization; 32.4% reported the estimated effect size and precision; 40.6% reported a sample size calculation and 32.4% mentioned external validity or implications of the findings.

Conclusion

Findings revealed that the reporting of RCTs in the top nine ORL-HNS journals and in the top Canadian ORL-HNS journal is suboptimal. The quality of reporting can be improved by addressing the three CONSORT items found most deficient in this study namely, sample size calculations, estimated effect size and precision, and external validity.

Patient Eligibility for Randomized Controlled Trials in Critical Care Medicine: An International Two-Center Observational Study

OBJECTIVE

We conducted this study to determine the generalizability of information gained from randomized controlled trials in critically ill patients by assessing the incidence of eligibility for each trial.

DESIGN

Prospective, observational cohort study. We identified the 15 most highly cited randomized controlled trials in critical care medicine published between 1998 and 2008. We examined the inclusion and exclusion criteria for each randomized controlled trial and then assessed the eligibility of each patient admitted to a study ICU for each randomized controlled trial and calculated rates of potential trial eligibility in the cohort.

SETTING

Three ICUs in two academic medical centers in Canada and the United States.

PATIENTS

Adults admitted to participating medical or surgical ICU in November 2010 or July 2011.

MEASUREMENTS AND MAIN RESULTS

Among the 15 trials, the most common trial inclusion criteria were clinical criteria for sepsis (six trials) or acute respiratory distress syndrome (four trials), use of invasive mechanical ventilation (five trials) or related to ICU type or duration of ICU stay (five trials). Of the 93 patients admitted to a study ICU, 52% of patients (n = 48) did not meet enrollment criteria for any studied randomized controlled trial and 30% (n = 28) were eligible for only one of the 15. Trial ineligibility was mostly due to failure to meet inclusion criteria (87% of screening assessments) rather than meeting specific exclusion criteria (52% of screening assessments). Of the positive screening assessments, 85% occurred on the first day of ICU admission.

CONCLUSIONS

Slightly more than half of the patients assessed were not eligible for enrollment in any of 15 major randomized controlled trials in critical care, most often due to the absence of the specific clinical condition of study. The majority of patients who met criteria for a randomized controlled trial did so on the first day of ICU admission.

Development and preliminary validation of the focused analgesia selection test to identify accurate pain reporters

Clinical trials of analgesics have been plagued with poor assay sensitivity due, in part, to variability in subjects' pain reporting. Herein, we develop and evaluate the focused analgesia selection test (FAST), a method to measure patients' pain reporting skills. Subjects with osteoarthritis of the hip, knee, and/or ankle with pain intensity of ≥3/10 on a 0-10 numerical rating scale were enrolled. Subjects underwent the FAST procedure, which consists of recording subjects' pain reports in response to repeated administration of thermal noxious stimuli of various intensities applied on the arm with the Medoc^® Thermal Sensory Analyzer II. Subjects also rated non-noxious stimuli consisting of visual contrast rating. After performing an exercise task, subjects also rated clinical pain and were asked to report whether their pain had increased, decreased, or stayed the same. Overall, 88 subjects were enrolled, and 83 were included in the analyses. FAST's outcomes including the R², intraclass correlation coefficient (ICC), and coefficient of variation (CoV) indicated that subjects' pain reporting skills were widely distributed. Higher FAST ICC significantly predicted greater changes in clinical pain following exercise (p=0.017), whereas the visual contrast test did not predict postexercise pain. FAST is the first method that measures subjects' pain reporting skills. Using FAST to enrich clinical trials with "good" pain reporters (with high FAST ICC) could increase assay sensitivity. Further evaluation of FAST is ongoing.

Sample size calculation: Basic principles

Addressing a sample size is a practical issue that has to be solved during planning and designing stage of the study. The aim of any clinical research is to detect the actual difference between two groups (power) and to provide an estimate of the difference with a reasonable accuracy (precision). Hence, researchers should do a priori estimate of sample size well ahead, before conducting the study. Post hoc sample size computation is not encouraged conventionally. Adequate sample size minimizes the random error or in other words, lessens something happening by chance. Too small a sample may fail to answer the research question and can be of questionable validity or provide an imprecise answer while too large a sample may answer the question but is resource-intensive and also may be unethical. More transparency in the calculation of sample size is required so that it can be justified and replicated while reporting.