Intermediate outcomes in randomized clinical trials: an introduction

Perspectives of Statistician, Microbiologist, and Clinician Stakeholders on the Use of Microbiological Outcomes in Randomised Trials of Antimicrobial Stewardship Interventions

Microbiological data are used as indicators of infection, for diagnosis, and the identification of antimicrobial resistance in trials of antimicrobial stewardship interventions. However, several problems have been identified in a recently conducted systematic review (e.g., inconsistency in reporting and oversimplified outcomes), which motivates the need to understand and improve the use of these data including analysis and reporting. We engaged key stakeholders including statisticians, clinicians from both primary and secondary care, and microbiologists. Discussions included issues identified in the systematic review and questions about the value of using microbiological data in clinical trials, perspectives on current microbiological outcomes reported in trials, and alternative statistical approaches to analyse these data. Various factors (such as unclear sample collection process, dichotomising or categorising complex microbiological data, and unclear methods of handling missing data) were identified that contributed to the low quality of the microbiological outcomes and the analysis of these outcomes in trials. Whilst not all of these factors would be easy to overcome, there is room for improvement and a need to encourage researchers to understand the impact of misusing these data. This paper discusses the experience and challenges of using microbiological outcomes in clinical trials.

Comparative Disease Assessment: a multi-causal approach for estimating the burden of mortality

Background

The Comparative Risk Assessment (CRA) framework comprehensively evaluates the impact of exposure to risk factors on health populations using the counterfactual causal approach.

Methods

We propose a framework, Comparative Disease Assessment (CDA), for assessing the impact of exposure to morbidity from some diseases on health outcomes, particularly death from other (relevant) diseases. This framework has been developed following the ideas of the CRA framework and using the widely accepted concept that exposure to morbidity is usually a risk factor for health outcomes (morbidity/mortality) related to other diseases. Our framework uses a counterfactual and not a categorical approach when attributing the burden of health outcomes to potential causes.

Results

This paper describes the different steps and assumptions required to implement the CDA framework, and an illustrative example is used considering diabetes mellitus morbidity as a risk factor for death from heart diseases.

Conclusions

One advantage of the CDA framework is that it can be applied using multi-causal death registries. Some assumptions are needed to implement it in order to avoid biases, but at least it can provide preliminary estimations of the impact of exposure to diseases as risk factors for deaths from other diseases. Another main advantage is that the burden of deaths is no longer attributed to a single cause, the underlying cause, as it is almost always done. Finally, this framework provides information on the pattern of comorbidity in a (sub)population of subjects who is about to die. These patterns can be used as a reference for alternative patterns of the general population or patterns of other specific subpopulations.

A comparison of methods to estimate the survivor average causal effect in the presence of missing data: a simulation study

BACKGROUND

Attrition due to death and non-attendance are common sources of bias in studies of age-related diseases. A simulation study is presented to compare two methods for estimating the survivor average causal effect (SACE) of a binary exposure (sex-specific dietary iron intake) on a binary outcome (age-related macular degeneration, AMD) in this setting.

METHODS

A dataset of 10,000 participants was simulated 1200 times under each scenario with outcome data missing dependent on measured and unmeasured covariates and survival. Scenarios differed by the magnitude and direction of effect of an unmeasured confounder on both survival and the outcome, and whether participants who died following a protective exposure would also die if they had not received the exposure (validity of the monotonicity assumption). The performance of a marginal structural model (MSM, weighting for exposure, survival and missing data) was compared to a sensitivity approach for estimating the SACE. As an illustrative example, the SACE of iron intake on AMD was estimated using data from 39,918 participants of the Melbourne Collaborative Cohort Study.

RESULTS

The MSM approach tended to underestimate the true magnitude of effect when the unmeasured confounder had opposing directions of effect on survival and the outcome. Overestimation was observed when the unmeasured confounder had the same direction of effect on survival and the outcome. Violation of the monotonicity assumption did not increase bias. The estimates were similar between the MSM approach and the sensitivity approach assessed at the sensitivity parameter of 1 (assuming no survival bias). In the illustrative example, high iron intake was found to be protective of AMD (adjusted OR 0.57, 95% CI 0.40-0.82) using complete case analysis via traditional logistic regression. The adjusted SACE odds ratio did not differ substantially from the complete case estimate, ranging from 0.54 to 0.58 for each of the SACE methods.

CONCLUSIONS

On average, MSMs with weighting for exposure, missing data and survival produced biased estimates of the SACE in the presence of an unmeasured survival-outcome confounder. The direction and magnitude of effect of unmeasured survival-outcome confounders should be considered when assessing exposure-outcome associations in the presence of attrition due to death.

Psychological and educational interventions for subfertile men and women

BACKGROUND

Approximately one-fifth of all subfertile couples seeking fertility treatment show clinically relevant levels of anxiety, depression, or distress. Psychological and educational interventions are frequently offered to subfertile couples, but their effectiveness, both in improving mental health and pregnancy rates, is unclear.

OBJECTIVES

To assess the effectiveness of psychological and educational interventions for subfertile couples on psychological and fertility treatment outcomes.

SEARCH METHODS

We searched (from inception to 2 April 2015) the Cochrane Gynaecology and Fertility Group Specialised Register of Controlled Trials, the Cochrane Central Register of Controlled Trials (CENTRAL; Issue 2, 2015), MEDLINE, EMBASE, PsycINFO, EBSCO CINAHL, DARE, Web of Science, OpenGrey, LILACS, PubMed, and ongoing trials registers. We handsearched reference lists and contacted experts in the field.

SELECTION CRITERIA

We included published and unpublished randomised controlled trials (RCTs), cluster randomised trials, and cross-over trials (first phase) evaluating the effectiveness of psychological and educational interventions on psychological and fertility treatment outcomes in subfertile couples.

DATA COLLECTION AND ANALYSIS

Two review authors independently assessed trial risk of bias and extracted data. We contacted study authors for additional information. Our primary outcomes were psychological measures (anxiety and depression) and fertility rates (live birth or ongoing pregnancy). We assessed the overall quality of the evidence using GRADE criteria.As we did not consider the included studies to be sufficiently similar to permit meaningful pooling, we summarised the results of the individual studies by presenting the median and interquartile range (IQR) of effects as well as the minimum and maximum values. We calculated standardised mean differences (SMDs) for continuous variables and odds ratios (ORs) for dichotomous outcomes.

MAIN RESULTS

We included 39 studies involving 4925 participants undergoing assisted reproductive technology. Studies were heterogeneous with respect to a number of factors, including nature and duration of interventions, participants, and comparator groups. As a result, we judged that pooling results would not result in a clinically meaningful estimate of a treatment effect. There were substantial methodological weaknesses in the studies, all of which were judged to be at high risk of bias for one or more quality assessment domains. There was concern about attrition bias (24 studies), performance bias for psychological outcomes (27 studies) and fertility outcomes (18 studies), and detection bias for psychological outcomes (26 studies). We therefore considered study-specific estimates of intervention effects to be unreliable. Thirty-three studies reported the outcome mental health. Only two studies reported the outcome live birth, and both of these had substantial attrition. One study reported ongoing pregnancy, again with substantial attrition. We have combined live birth and ongoing pregnancy in one outcome. Psychological outcomesStudies utilised a variety of measures of anxiety and depression. In all cases a low score denoted benefit from the intervention.SMDs for anxiety were as follows: psychological interventions versus attentional control or usual care: median (IQR) = -0.30 (-0.84 to 0.00), minimum value -5.13; maximum value 0.84, 17 RCTs, 2042 participants; educational interventions versus attentional control or usual care: median = 0.03, minimum value -0.38; maximum value 0.23, 4 RCTs, 330 participants.SMDs for depression were as follows: psychological interventions versus attentional control or usual care: median (IQR) = -0.45 (-0.68 to -0.08), minimum value -3.01; maximum value 1.23, 12 RCTs, 1160 participants; educational interventions versus attentional control or usual care: median = -0.33, minimum value -0.46; maximum value 0.17, 3 RCTs, 304 participants. Fertility outcomesWhen psychological interventions were compared with attentional control or usual care, ORs for live birth or ongoing pregnancy ranged from minimum value 1.13 to maximum value 10.05. No studies of educational interventions reported this outcome.

AUTHORS' CONCLUSIONS

The effects of psychological and educational interventions on mental health including distress, and live birth or ongoing pregnancy rates is uncertain due to the very low quality of the evidence. Existing trials of psychological and educational interventions for subfertility were generally poorly designed and executed, resulting in very serious risk of bias and serious inconsistency in study findings. There is a need for studies employing appropriate methodological techniques to investigate the benefits of these treatments for this population. In particular, attentional control groups should be employed, that is groups receiving a treatment that mimics the amount of time and attention received by the treatment group but is not thought to have a specific effect upon the participants, in order to distinguish between therapeutic and non-specific effects of interventions. Where attrition cannot be minimised, appropriate statistical techniques for handling drop-out must be applied. Failure to address these issues in study design has resulted in studies that do not provide a valid basis for answering questions about the effectiveness of these interventions.

How to assess success of treatment when using multiple doses: the case of misoprostol for medical abortion

Background

The assessment of treatment success in clinical trials when multiple (repeated) doses (courses) are involved is quite common, for example, in the case of infertility treatment with assisted reproductive technology (ART), and medical abortion using misoprostol alone or in combination with mifepristone. Under these or similar circumstances, most researchers assess success using binomial proportions after a certain number of consecutive doses, and some have used survival analysis. In this paper we discuss the main problems in using binomial proportions to summarize (the overall) efficacy after two or more consecutive doses of the relevant treatment, particularly for the case of misoprostol in medical abortion studies. We later discuss why the survival analysis is best suited under these circumstances, and illustrate this by using simulated data.

Methods

The formulas required for the binomial proportion and survival analysis (without and with competing risks) approaches are summarized and analytically compared. Additionally, numerical results are computed and compared between the two approaches, for several theoretical scenarios.

Results

The main conceptual limitations of the binomial proportion approach are identified and discussed, caused mainly by the presence of censoring and competing risks, and it is demonstrated how survival analysis can solve these problems. In general, the binomial proportion approach tends to underestimate the “real” success rate, and tends to overestimate the corresponding standard error.

Conclusions

Depending on the rates of censored observations or competing events between repeated doses of the treatment, the bias of the binomial proportion approach as compared to the survival analysis approaches varies; however, the use of the binomial approach is unjustified as the survival analysis options are well known and available in multiple statistical packages. Our conclusions also apply to other situations where success is estimated after multiple (repeated) doses (courses) of the treatment.