Potential pitfalls in the design and reporting of clinical trials

Observational studies: practical tips for avoiding common statistical pitfalls

This Personal View is intended for early-career researchers who are not yet experts in statistics. The Personal View focuses on common but usually avoidable flaws in the context of observational studies. I point out how study design, data collection, and statistical methods impact statistical results and research conclusions. With particular attention to study planning, sample selection, biases, lack of transparency and results misinterpretations.

Design of a clinical trial using generalized pairwise comparisons to test a less intensive treatment regimen

BACKGROUND/AIMS

Showing "similar efficacy" of a less intensive treatment typically requires a non-inferiority trial. Yet such trials may be challenging to design and conduct. In acute promyelocytic leukemia, great progress has been achieved with the introduction of targeted therapies, but toxicity remains a major clinical issue. There is a pressing need to show the favorable benefit/risk of less intensive treatment regimens.

METHODS

We designed a clinical trial that uses generalized pairwise comparisons of five prioritized outcomes (alive and event-free at 2 years, grade 3/4 documented infections, differentiation syndrome, hepatotoxicity, and neuropathy) to confirm a favorable benefit/risk of a less intensive treatment regimen. We conducted simulations based on historical data and assumptions about the differences expected between the standard of care and the less intensive treatment regimen to calculate the sample size required to have high power to show a positive Net Treatment Benefit in favor of the less intensive treatment regimen.

RESULTS

Across 10,000 simulations, average sample sizes of 260 to 300 patients are required for a trial using generalized pairwise comparisons to detect typical Net Treatment Benefits of 0.19 (interquartile range 0.14-0.23 for a sample size of 280). The Net Treatment Benefit is interpreted as a difference between the probability of doing better on the less intensive treatment regimen than on the standard of care, minus the probability of the opposite situation. A Net Treatment Benefit of 0.19 translates to a number needed to treat of about 5.3 patients (1/0.19 ≃ 5.3).

CONCLUSION

Generalized pairwise comparisons allow for simultaneous assessment of efficacy and safety, with priority given to the former. The sample size required would be of the order of 300 patients, as compared with more than 700 patients for a non-inferiority trial using a margin of 4% against the less intensive treatment regimen for the absolute difference in event-free survival at 2 years, as considered here.

Outcomes reported in trials of treatments for severe malaria: The need for a core outcome set

OBJECTIVES

Malaria is one of the most important parasitic infectious diseases worldwide. Despite the scale-up of effective antimalarials, mortality rates from severe malaria (SM) remain significantly high; thus, numerous trials are investigating both antimalarials and adjunctive therapy. This review aimed to summarise all the outcome measures used in trials in the last 10 years to see the need for a core outcome set.

METHODS

A systematic review was undertaken to summarise outcomes of individually randomised trials assessing treatments for SM in adults and children. We searched key databases and trial registries between 1 January 2010 and 30 July 2020. Non-randomised trials were excluded to allow comparison of similar trials. Trial characteristics including phase, region, population, interventions, were summarised. All primary and secondary outcomes were extracted and categorised using a taxonomy table.

RESULTS

Twenty-seven of 282 screened trials met our inclusion criteria, including 10,342 patients from 19 countries: 19 (70%) trials from Africa and 8 (30%) from Asia. A large amount of heterogeneity was observed in the selection of outcomes and instruments, with 101 different outcomes measures recorded, 78/101 reported only in a single trial. Parasitological outcomes (17 studies), neurological status (14 studies), death (14 studies) and temperature (10 studies), were the most reported outcomes. Where an outcome was reported in >1 study it was often measured differently: temperature (4 different measures), renal function (7 measures), nervous system (13 measures) and parasitology (10 measures).

CONCLUSION

Outcomes used in SM trials are inconsistent and heterogeneous. Absence of consensus for outcome measures used impedes research synthesis and comparability of different interventions. This systematic review demonstrates the need to develop a standardised collection of core outcomes for clinical trials of treatments for SM and next steps to include the development of a panel of experts in the field, a Delphi process, and a consensus meeting.

Assessment of Completeness of Reporting in Randomized Controlled Trials of Acupuncture Therapy for Chronic Obstructive Pulmonary Disease

Objective

To assess the completeness of reporting in randomized controlled trials (RCTs) of acupuncture therapy (AT) for chronic obstructive pulmonary disease (COPD).

Methods

We systematically searched PubMed, Embase, Cochrane Library, Web of Science, China National Knowledge Infrastructure (CNKI), Chongqing VIP (CQVIP), WANFANG Data, and China Biology Medicine (CBM) for studies published from their inception to May 8, 2021. The completeness of reporting was evaluated by CONSORT statement and STRICTA guidelines. Univariate and multivariate regression analyses were performed to preliminarily explore the factors related to completeness of reporting.

Results

A total of 44 RCTs were included. The overall quality score (OQS) based on the CONSORT statement and STRICTA guidelines ranged from 3 to 26 and 7 to 14, with a median of 10 and 11, respectively. Among the 35 items of the CONSORT statement, 10 items were fully reported with reporting rate > 70%, and 11 items were poorly reported at a rate < 5%. Among the 17 items of the STRICTA guidelines, 10 items were adequately reported with > 70%, and only 3 items were incompletely reported at a rate < 20%. The agreement of most items was determined as “good”, “substantial”, or “moderate”. By regression analysis, publication language (β coefficient: 6.432, 95% CI: 3.202 to 9.663, P <0.001) and funding source (β coefficient: 3.159, 95% CI: 1.045 to 5.273, P =0.004) acted as independent predictors of completeness of reporting according to the CONSORT statement. However, no variables associated with the STRICTA guidelines were identified.

Conclusion

The completeness of reporting of AT for COPD was inadequate. The condition relatively improved for trials with publication in the English language and funding source. By recommendation, reports should be strictly standardized in accordance with the CONSORT statement and STRICTA guidelines to improve the clinical research evidence of AT for COPD.

Subgroup Analyses in Reporting of Phase III Clinical Trials in Solid Tumors

Purpose

Treatment decisions in clinical oncology are guided by results from phase III randomized clinical trials (RCTs). The results of subgroup analyses may be potentially important in individualizing patient care. We investigated the appropriateness of the use and interpretation of subgroup analyses in oncology RCTs on the basis of the CONSORT statement requirements.

Methods

Phase III RCTs published between January 1, 2011, and December 31, 2013, were reviewed to identify eligible studies of solid tumor treatments. Information related to the subgroup analyses included prespecification, number, subgroup factors, interaction test use, and claim of subgroup difference.

Results

A total of 221 publications reporting data on 184,500 patients were analyzed. One hundred eighty-eight (85%) RCTs were reported with subgroup analyses. Of those, 146 (78%) trials were reported with at least six subgroups. For the majority of trials with subgroup analyses (173; 92%), the actual number of subgroup analyses conducted cannot be determined. Only 59 (31%) RCTs were reported with fully prespecified subgroups and only 64 (34%) trials were reported with interaction tests. In addition, 102 (54%) RCTs were reported with claims of subgroup differences. Of those, only 18 claims of RCTs (18%) were based on significant interaction test results.

Conclusion

The reporting of subgroup analyses in contemporary oncology RCTs is neither uniform nor complete; it requires improvement to ensure consistency and to provide critical information for guiding patient care. Major problems include testing of a large number of subgroups, subgroups without prespecifications, and inadequate use of interaction tests.