Are unadjusted analyses of clinical trials inappropriately biased toward the null?

Image harmonization: A review of statistical and deep learning methods for removing batch effects and evaluation metrics for effective harmonization

Magnetic resonance imaging and computed tomography from multiple batches (e.g. sites, scanners, datasets, etc.) are increasingly used alongside complex downstream analyses to obtain new insights into the human brain. However, significant confounding due to batch-related technical variation, called batch effects, is present in this data; direct application of downstream analyses to the data may lead to biased results. Image harmonization methods seek to remove these batch effects and enable increased generalizability and reproducibility of downstream results. In this review, we describe and categorize current approaches in statistical and deep learning harmonization methods. We also describe current evaluation metrics used to assess harmonization methods and provide a standardized framework to evaluate newly-proposed methods for effective harmonization and preservation of biological information. Finally, we provide recommendations to end-users to advocate for more effective use of current methods and to methodologists to direct future efforts and accelerate development of the field.

Effectiveness of Remote Interventions to Improve Medication Adherence in Patients after Stroke: A Systematic Literature Review and Meta-Analysis

BACKGROUND

Stroke affects more than 30 million people every year, but only two-thirds of patients comply with prescribed medication, leading to high stroke recurrence rates. Digital technologies can facilitate interventions to support treatment adherence.

PURPOSE

This study evaluates the effectiveness of remote interventions and their mechanisms of action in supporting medication adherence after stroke.

METHODS

PubMed, MEDLINE via Ovid, Cochrane CENTRAL, the Web of Science, SCOPUS, and PsycINFO were searched, and meta-analysis was performed using the Review Manager Tool. Intervention content analysis was conducted based on the COM-B model.

RESULTS

Ten eligible studies were included in the review and meta-analysis. The evidence suggested that patients who received remote interventions had significantly better medication adherence (SMD 0.49, 95% CI [0.04, 0.93], and p = 0.03) compared to those who received the usual care. The adherence ratio also indicated the interventions' effectiveness (odds ratio 1.30, 95% CI [0.55, 3.10], and p = 0.55). The systolic and diastolic blood pressure (MD -3.73 and 95% CI [-5.35, -2.10])/(MD -2.16 and 95% CI [-3.09, -1.22]) and cholesterol levels (MD -0.36 and 95% CI [-0.52, -0.20]) were significantly improved in the intervention group compared to the control. Further behavioural analysis demonstrated that enhancing the capability within the COM-B model had the largest impact in supporting improvements in adherence behaviour and relevant clinical outcomes. Patients' satisfaction and the interventions' usability were both high, suggesting the interventions' acceptability.

CONCLUSION

Telemedicine and mHealth interventions are effective in improving medication adherence and clinical indicators in stroke patients. Future studies could usefully investigate the effectiveness and cost-effectiveness of theory-based and remotely delivered interventions as an adjunct to stroke rehabilitation programmers.

MetfOrmin BenefIts Lower Extremities with Intermittent Claudication (MOBILE IC): randomized clinical trial protocol

BACKGROUND

Peripheral artery disease (PAD) affects over 230 million people worldwide and is due to systemic atherosclerosis with etiology linked to chronic inflammation, hypertension, and smoking status. PAD is associated with walking impairment and mobility loss as well as a high prevalence of coronary and cerebrovascular disease. Intermittent claudication (IC) is the classic presenting symptom for PAD, although many patients are asymptomatic or have atypical presentations. Few effective medical therapies are available, while surgical and exercise therapies lack durability. Metformin, the most frequently prescribed oral medication for Type 2 diabetes, has salient anti-inflammatory and promitochondrial properties. We hypothesize that metformin will improve function, retard the progression of PAD, and improve systemic inflammation and mitochondrial function in non-diabetic patients with IC.

METHODS

200 non-diabetic Veterans with IC will be randomized 1:1 to 180-day treatment with metformin extended release (1000 mg/day) or placebo to evaluate the effect of metformin on functional status, PAD progression, cardiovascular disease events, and systemic inflammation. The primary outcome is 180-day maximum walking distance on the 6-min walk test (6MWT). Secondary outcomes include additional assessments of functional status (cardiopulmonary exercise testing, grip strength, Walking Impairment Questionnaires), health related quality of life (SF-36, VascuQoL), macro- and micro-vascular assessment of lower extremity blood flow (ankle brachial indices, pulse volume recording, EndoPAT), cardiovascular events (amputations, interventions, major adverse cardiac events, all-cause mortality), and measures of systemic inflammation. All outcomes will be assessed at baseline, 90 and 180 days of study drug exposure, and 180 days following cessation of study drug. We will evaluate the primary outcome with linear mixed-effects model analysis with covariate adjustment for baseline 6MWT, age, baseline ankle brachial indices, and smoking status following an intention to treat protocol.

DISCUSSION

MOBILE IC is uniquely suited to evaluate the use of metformin to improve both systematic inflammatory responses, cellular energetics, and functional outcomes in patients with PAD and IC.

TRIAL REGISTRATION

The prospective MOBILE IC trial was publicly registered (NCT05132439) November 24, 2021.

How to write statistical analysis section in medical research

Reporting of statistical analysis is essential in any clinical and translational research study. However, medical research studies sometimes report statistical analysis that is either inappropriate or insufficient to attest to the accuracy and validity of findings and conclusions. Published works involving inaccurate statistical analyses and insufficient reporting influence the conduct of future scientific studies, including meta-analyses and medical decisions. Although the biostatistical practice has been improved over the years due to the involvement of statistical reviewers and collaborators in research studies, there remain areas of improvement for transparent reporting of the statistical analysis section in a study. Evidence-based biostatistics practice throughout the research is useful for generating reliable data and translating meaningful data to meaningful interpretation and decisions in medical research. Most existing research reporting guidelines do not provide guidance for reporting methods in the statistical analysis section that helps in evaluating the quality of findings and data interpretation. In this report, we highlight the global and critical steps to be reported in the statistical analysis of grants and research articles. We provide clarity and the importance of understanding study objective types, data generation process, effect size use, evidence-based biostatistical methods use, and development of statistical models through several thematic frameworks. We also provide published examples of adherence or non-adherence to methodological standards related to each step in the statistical analysis and their implications. We believe the suggestions provided in this report can have far-reaching implications for education and strengthening the quality of statistical reporting and biostatistical practice in medical research.

Practical help for specifying the target difference in sample size calculations for RCTs: the DELTA2 five-stage study, including a workshop

BACKGROUND

The randomised controlled trial is widely considered to be the gold standard study for comparing the effectiveness of health interventions. Central to its design is a calculation of the number of participants needed (the sample size) for the trial. The sample size is typically calculated by specifying the magnitude of the difference in the primary outcome between the intervention effects for the population of interest. This difference is called the 'target difference' and should be appropriate for the principal estimand of interest and determined by the primary aim of the study. The target difference between treatments should be considered realistic and/or important by one or more key stakeholder groups.

OBJECTIVE

The objective of the report is to provide practical help on the choice of target difference used in the sample size calculation for a randomised controlled trial for researchers and funder representatives.

METHODS

The Difference ELicitation in TriAls2 (DELTA2) recommendations and advice were developed through a five-stage process, which included two literature reviews of existing funder guidance and recent methodological literature; a Delphi process to engage with a wider group of stakeholders; a 2-day workshop; and finalising the core document.

RESULTS

Advice is provided for definitive trials (Phase III/IV studies). Methods for choosing the target difference are reviewed. To aid those new to the topic, and to encourage better practice, 10 recommendations are made regarding choosing the target difference and undertaking a sample size calculation. Recommended reporting items for trial proposal, protocols and results papers under the conventional approach are also provided. Case studies reflecting different trial designs and covering different conditions are provided. Alternative trial designs and methods for choosing the sample size are also briefly considered.

CONCLUSIONS

Choosing an appropriate sample size is crucial if a study is to inform clinical practice. The number of patients recruited into the trial needs to be sufficient to answer the objectives; however, the number should not be higher than necessary to avoid unnecessary burden on patients and wasting precious resources. The choice of the target difference is a key part of this process under the conventional approach to sample size calculations. This document provides advice and recommendations to improve practice and reporting regarding this aspect of trial design. Future work could extend the work to address other less common approaches to the sample size calculations, particularly in terms of appropriate reporting items.

FUNDING

Funded by the Medical Research Council (MRC) UK and the National Institute for Health Research as part of the MRC-National Institute for Health Research Methodology Research programme.

Device Closure of Patent Foramen Ovale After Stroke: Pooled Analysis of Completed Randomized Trials

BACKGROUND

The comparative effectiveness of percutaneous closure of patent foramen ovale (PFO) plus medical therapy versus medical therapy alone for cryptogenic stroke is uncertain.

OBJECTIVES

The authors performed the first pooled analysis of individual participant data from completed randomized trials comparing PFO closure versus medical therapy in patients with cryptogenic stroke.

METHODS

The analysis included data on 2 devices (STARFlex [umbrella occluder] [NMT Medical, Inc., Boston, Massachusetts] and Amplatzer PFO Occluder [disc occluder] [AGA Medical/St. Jude Medical, St. Paul, Minnesota]) evaluated in 3 trials. The primary composite outcome was stroke, transient ischemic attack, or death; the secondary outcome was stroke. We used log-rank tests and unadjusted and covariate-adjusted Cox regression models to compare device closure versus medical therapy.

RESULTS

Among 2,303 patients, closure was not significantly associated with the primary composite outcome. The difference became significant after covariate adjustment (hazard ratio [HR]: 0.68; p = 0.049). For the outcome of stroke, all comparisons were statistically significant, with unadjusted and adjusted HRs of 0.58 (p = 0.043) and 0.58 (p = 0.044), respectively. In analyses limited to the 2 disc occluder device trials, the effect of closure was not significant for the composite outcome, but was for the stroke outcome (unadjusted HR: 0.39; p = 0.013). Subgroup analyses did not identify significant heterogeneity of treatment effects. Atrial fibrillation was more common among closure patients.

CONCLUSIONS

Among patients with PFO and cryptogenic stroke, closure reduced recurrent stroke and had a statistically significant effect on the composite of stroke, transient ischemic attack, and death in adjusted but not unadjusted analyses.

False Appearance of Gene-Environment Interactions in Genetic Association Studies

Under the assumption of gene-environment independence, unknown/unmeasured environmental factors, irrespective of what they may be, cannot confound the genetic effects. This may lead many people to believe that genetic heterogeneity across different levels of the studied environmental exposure should only mean gene-environment interaction--even though other environmental factors are not adjusted for. However, this is not true if the odds ratio is the effect measure used for quantifying genetic effects. This is because the odds ratio is a "noncollapsible" measure--a marginal odds ratio is not a weighted average of the conditional odds ratios, but instead has a tendency toward the null. In this study, the authors derive formulae for gene-environment interaction bias due to noncollapsibility. They use computer simulation and real data example to show that the bias can be substantial for common diseases. For genetic association study of nonrare diseases, researchers are advised to use collapsible measures, such as risk ratio or peril ratio.

Covariate imbalance and adjustment for logistic regression analysis of clinical trial data

In logistic regression analysis for binary clinical trial data, adjusted treatment effect estimates are often not equivalent to unadjusted estimates in the presence of influential covariates. This article uses simulation to quantify the benefit of covariate adjustment in logistic regression. However, International Conference on Harmonization guidelines suggest that covariate adjustment be prespecified. Unplanned adjusted analyses should be considered secondary. Results suggest that if adjustment is not possible or unplanned in a logistic setting, balance in continuous covariates can alleviate some (but never all) of the shortcomings of unadjusted analyses. The case of log binomial regression is also explored.

Bias, precision and statistical power of analysis of covariance in the analysis of randomized trials with baseline imbalance: a simulation study

Background

Analysis of variance (ANOVA), change-score analysis (CSA) and analysis of covariance (ANCOVA) respond differently to baseline imbalance in randomized controlled trials. However, no empirical studies appear to have quantified the differential bias and precision of estimates derived from these methods of analysis, and their relative statistical power, in relation to combinations of levels of key trial characteristics. This simulation study therefore examined the relative bias, precision and statistical power of these three analyses using simulated trial data.

Methods

126 hypothetical trial scenarios were evaluated (126 000 datasets), each with continuous data simulated by using a combination of levels of: treatment effect; pretest-posttest correlation; direction and magnitude of baseline imbalance. The bias, precision and power of each method of analysis were calculated for each scenario.

Results

Compared to the unbiased estimates produced by ANCOVA, both ANOVA and CSA are subject to bias, in relation to pretest-posttest correlation and the direction of baseline imbalance. Additionally, ANOVA and CSA are less precise than ANCOVA, especially when pretest-posttest correlation ≥ 0.3. When groups are balanced at baseline, ANCOVA is at least as powerful as the other analyses. Apparently greater power of ANOVA and CSA at certain imbalances is achieved in respect of a biased treatment effect.

Conclusions

Across a range of correlations between pre- and post-treatment scores and at varying levels and direction of baseline imbalance, ANCOVA remains the optimum statistical method for the analysis of continuous outcomes in RCTs, in terms of bias, precision and statistical power.

Analysis of combined data from heterogeneous study designs: an applied example from the patient navigation research program

BACKGROUND

The Patient Navigation Research Program (PNRP) is a cooperative effort of nine research projects, with similar clinical criteria but with different study designs. To evaluate projects such as PNRP, it is desirable to perform a pooled analysis to increase power relative to the individual projects. There is no agreed-upon prospective methodology, however, for analyzing combined data arising from different study designs. Expert opinions were thus solicited from the members of the PNRP Design and Analysis Committee.

PURPOSE

To review possible methodologies for analyzing combined data arising from heterogeneous study designs.

METHODS

The Design and Analysis Committee critically reviewed the pros and cons of five potential methods for analyzing combined PNRP project data. The conclusions were based on simple consensus. The five approaches reviewed included the following: (1) analyzing and reporting each project separately, (2) combining data from all projects and performing an individual-level analysis, (3) pooling data from projects having similar study designs, (4) analyzing pooled data using a prospective meta-analytic technique, and (5) analyzing pooled data utilizing a novel simulated group-randomized design.

RESULTS

Methodologies varied in their ability to incorporate data from all PNRP projects, to appropriately account for differing study designs, and to accommodate differing project sample sizes.

LIMITATIONS

The conclusions reached were based on expert opinion and not derived from actual analyses performed.

CONCLUSIONS

The ability to analyze pooled data arising from differing study designs may provide pertinent information to inform programmatic, budgetary, and policy perspectives. Multisite community-based research may not lend itself well to the more stringent explanatory and pragmatic standards of a randomized controlled trial design. Given our growing interest in community-based population research, the challenges inherent in the analysis of heterogeneous study design are likely to become more salient. Discussion of the analytic issues faced by the PNRP and the methodological approaches we considered may be of value to other prospective community-based research programs.

Optimal end points for acute stroke therapy trials: best ways to measure treatment effects of drugs and devices

BACKGROUND AND PURPOSE

Over the past decade, analysis of completed actual trials, model population studies, and theoretical work have improved approaches to selecting and analyzing end points in acute stroke treatment trials.

METHODS

Narrative review.

RESULTS

Because stroke affects persons in their biological, functional, social, and experiential dimensions, measures of impairment, disability, handicap, and quality of life are all desirable in pivotal trials, with disability being most important. Scales that are valid, reliable, responsive, and easy to administer are preferred; consequently, the modified Rankin Scale has become the most widely used single clinical efficacy measure. Because stroke cripples and kills, most outcome scales array patient outcome in ordered ranks, spread over the entire range from normal to disabled to dead. Generally, shift analysis, analyzing all health state transitions concurrently, is the most efficient analytic technique to detect treatment effects, with sliding dichotomy less efficient and fixed dichotomy least efficient, unless treatment effects strongly cluster at 1 or a few health state transitions that can be prespecified. Test statistics must also take into account interpretability, ie, how well they can be converted into metrics capturing all outcomes the intervention might alter in proportion to the degree they are valued by the patient; full ordinal analysis is most informative, sliding dichotomy is intermediately informative, and fixed dichotomy is least informative regarding this global outcome.

CONCLUSIONS

Stroke trial power and interpretation can be substantially enhanced by adherence to the principles delineated in this review. Full ordinal and sliding dichotomy analysis will most often be advantageous compared with fixed dichotomous approaches.