Mediation Analysis with Survival Outcomes: Accelerated Failure Time vs. Proportional Hazards Models

Objective: Survival time is an important type of outcome variable in treatment research. Currently, limited guidance is available regarding performing mediation analyses with survival outcomes, which generally do not have normally distributed errors, and contain unobserved (censored) events. We present considerations for choosing an approach, using a comparison of semi-parametric proportional hazards (PH) and fully parametric accelerated failure time (AFT) approaches for illustration.

Method: We compare PH and AFT models and procedures in their integration into mediation models and review their ability to produce coefficients that estimate causal effects. Using simulation studies modeling Weibull-distributed survival times, we compare statistical properties of mediation analyses incorporating PH and AFT approaches (employing SAS procedures PHREG and LIFEREG, respectively) under varied data conditions, some including censoring. A simulated data set illustrates the findings.

Results: AFT models integrate more easily than PH models into mediation models. Furthermore, mediation analyses incorporating LIFEREG produce coefficients that can estimate causal effects, and demonstrate superior statistical properties. Censoring introduces bias in the coefficient estimate representing the treatment effect on outcome—underestimation in LIFEREG, and overestimation in PHREG. With LIFEREG, this bias can be addressed using an alternative estimate obtained from combining other coefficients, whereas this is not possible with PHREG.

Conclusions: When Weibull assumptions are not violated, there are compelling advantages to using LIFEREG over PHREG for mediation analyses involving survival-time outcomes. Irrespective of the procedures used, the interpretation of coefficients, effects of censoring on coefficient estimates, and statistical properties should be taken into account when reporting results.

The Personalized Advantage Index: translating research on prediction into individualized treatment recommendations. A demonstration

Background

Advances in personalized medicine require the identification of variables that predict differential response to treatments as well as the development and refinement of methods to transform predictive information into actionable recommendations.

Objective

To illustrate and test a new method for integrating predictive information to aid in treatment selection, using data from a randomized treatment comparison.

Method

Data from a trial of antidepressant medications (N = 104) versus cognitive behavioral therapy (N = 50) for Major Depressive Disorder were used to produce predictions of post-treatment scores on the Hamilton Rating Scale for Depression (HRSD) in each of the two treatments for each of the 154 patients. The patient's own data were not used in the models that yielded these predictions. Five pre-randomization variables that predicted differential response (marital status, employment status, life events, comorbid personality disorder, and prior medication trials) were included in regression models, permitting the calculation of each patient's Personalized Advantage Index (PAI), in HRSD units.

Results

For 60% of the sample a clinically meaningful advantage (PAI≥3) was predicted for one of the treatments, relative to the other. When these patients were divided into those randomly assigned to their “Optimal” treatment versus those assigned to their “Non-optimal” treatment, outcomes in the former group were superior (d = 0.58, 95% CI .17—1.01).

Conclusions

This approach to treatment selection, implemented in the context of two equally effective treatments, yielded effects that, if obtained prospectively, would rival those routinely observed in comparisons of active versus control treatments.

Understanding processes of change: how some patients reveal more than others-and some groups of therapists less-about what matters in psychotherapy

OBJECTIVE

We identify difficulties researchers encounter in psychotherapy process-outcome investigations, and we describe several limitations of the popular "variance accounted for" approach to understanding the effects of psychotherapy.

METHODS & RESULTS

Using data simulations, we show how the expected correlation between an excellent measure of therapy quality and outcome would be surprisingly small (approximately .25) under conditions likely to be common in psychotherapy research. Even when we modeled conditions designed to increase the likelihood that strong process-outcome relationships would be observed, we found that the expected correlations were still only in the modest range (.38-.51).

CONCLUSIONS

We discuss the implications of our analysis for the interpretation of process-outcome findings as well as for design considerations in future investigations.

Mediation analysis: a retrospective snapshot of practice and more recent directions

R. Baron and D. A. Kenny's (1986) paper introducing mediation analysis has been cited over 9,000 times, but concerns have been expressed about how this method is used. The authors review past and recent methodological literature and make recommendations for how to address 3 main issues: association, temporal order, and the no omitted variables assumption. The authors briefly visit the topics of reliability and the confirmatory-exploratory distinction. In addition, to provide a sense of the extent to which the earlier literature had been absorbed into practice, the authors examined a sample of 50 articles from 2002 citing R. Baron and D. A. Kenny and containing at least 1 mediation analysis via ordinary least squares regression. A substantial proportion of these articles included problematic reporting; as of 2002, there appeared to be room for improvement in conducting such mediation analyses. Future literature reviews will demonstrate the extent to which the situation has improved.

Sequence of improvement in depressive symptoms across cognitive therapy and pharmacotherapy

BACKGROUND

The authors examined the patterns of improvement in cognitive and vegetative symptoms of major depression in individuals treated with cognitive therapy (CT) or pharmacotherapy (PT).

METHOD

Outpatients diagnosed with major depressive disorder (n=180) were randomized to receive either CT or PT. Cognitive and vegetative symptoms of major depression were measured by the Beck Depression Inventory-II at baseline and regularly throughout 16 weeks of treatment.

RESULTS

Multivariate hierarchical linear modeling demonstrated the same patterns of change over time for cognitive and vegetative symptoms within CT and within PT.

LIMITATIONS

Self-report measures may not be sufficiently specific to capture subtle differences in improvements between vegetative and cognitive symptoms.

CONCLUSIONS

These results are consistent with Beck's [Beck, A.T., 1984, November. Cognition and theory [Letter to the editor]. Arch. Gen. Psychiatry 41, 1112-1114.] hypothesis that CT and PT have a similar site of action, which when targeted, results in changes in both cognitive and vegetative features.

Bias resulting from the use of ‘assay sensitivity’ as an inclusion criterion for meta-analysis

Assay sensitivity has been proposed as a criterion for including psychiatric clinical outcome studies in meta-analyses. The authors assess the performance of assay sensitivity as a method for determining study appropriateness for meta-analysis by calculating expected standard drug vs placebo effect sizes for various combinations of high quality and flawed studies. In the absence of flawed studies, expected effect sizes are close to unbiased only when sample sizes are very large. In the presence of flawed studies, expected effect sizes tend to be substantially biased except under simultaneous conditions of high power, a large proportion of flawed studies, and a population standard vs placebo effect size of flawed studies considerably lower than that of high quality studies. The authors conclude that this method is not robust and can lead to serious bias. Unless it can be shown that specific conditions hold, assay sensitivity should not be used to make quality judgments of studies.

Reducing exposure of clinical research subjects to placebo treatments

The ethics of pill placebo and placebo psychotherapy conditions in clinical research are controversial. Even when not life threatening, mental disorders dramatically diminish the quality of life. Pill-placebo conditions in drug treatment research have been justified on the grounds that a placebo versus standard drug comparison is necessary to test the quality of the study, viz., the assay sensitivity method. The assay sensitivity method of judging study quality, however, results in misclassification of the quality of some studies, leading to bias in effect size estimation in the context of meta-analyses. This bias is of particular concern in relation to studies comparing psychotherapies to psychotropic drugs, which are conducted outside of the Food and Drug Administration (FDA) context. In cases in which control conditions may be justified on grounds other than as essential elements of an assay sensitivity test, statistical methods to reduce the number of study participants exposed to placebo should be strongly considered. Of the methods available, group sequential methods are the most widely used. Group sequential methods involve successive looks at accumulating data, with rules for terminating a trial (or an arm of a trial) early if results are strong enough.

The temporal relation of adherence and alliance to symptom change in cognitive therapy for depression

This study attempted to replicate an earlier study (R. J. DeRubeis & M. Feeley, 1990) of the prediction of symptom change from process variables in cognitive therapy for depressed outpatients. Measures of in-session therapist behavior and therapist-patient interactions were correlated with prior and subsequent symptom change. One of the positive findings was confirmed, but the other received only marginal support. A "concrete" subset of theory-specified therapist actions, measured early in treatment, predicted subsequent change in depression. The therapeutic alliance was predicted by prior symptom change in 1 of the 2 later assessments, but only at a trend level. Several negative findings were similar to those obtained in the earlier study. Specifically, the alliance, an "abstract" subset of theory-specified therapist actions, and facilitative conditions did not predict subsequent change. Implications for causal inferences in psychotherapy process research are discussed.

The temporal relation of adherence and alliance to symptom change in cognitive therapy for depression

This study attempted to replicate an earlier study (R. J. DeRubeis & M. Feeley, 1990) of the prediction of symptom change from process variables in cognitive therapy for depressed outpatients. Measures of in-session therapist behavior and therapist-patient interactions were correlated with prior and subsequent symptom change. One of the positive findings was confirmed, but the other received only marginal support. A "concrete" subset of theory-specified therapist actions, measured early in treatment, predicted subsequent change in depression. The therapeutic alliance was predicted by prior symptom change in 1 of the 2 later assessments, but only at a trend level. Several negative findings were similar to those obtained in the earlier study. Specifically, the alliance, an "abstract" subset of theory-specified therapist actions, and facilitative conditions did not predict subsequent change. Implications for causal inferences in psychotherapy process research are discussed.

Medications versus cognitive behavior therapy for severely depressed outpatients: mega-analysis of four randomized comparisons

OBJECTIVE

The purpose of this study was to compare the acute outcomes of antidepressant medication and cognitive behavior therapy in the severely depressed outpatient subgroups of four major randomized trials. A secondary objective was to compare the results obtained in the National Institute of Mental Health Treatment of Depression Collaborative Research Program, upon which treatment guidelines have been based, with those obtained in the other three studies.

METHOD

Outcomes of antidepressant medication and cognitive behavior therapy were compared within each of the four studies separately and for patients aggregated across the four studies. In addition, the outcomes in the antidepressant medication and cognitive behavior therapy conditions of the Treatment of Depression Collaborative Research Program were compared with those obtained in the other three studies.

RESULTS

The overall effect sizes comparing antidepressant medication to cognitive behavior therapy favored cognitive behavior therapy, but tests comparing the two modalities did not reveal a significant advantage for either modality overall.

CONCLUSIONS

Cognitive behavior therapy has fared as well as antidepressant medication with severely depressed outpatients in four major comparisons. Until findings emerge from current or future comparative trials, antidepressant medication should not be considered, on the basis of empirical evidence, to be superior to cognitive behavior therapy for the acute treatment of severely depressed outpatients.

The Alcohol Clinical Index in lower socioeconomic alcohol-dependent men

The Alcohol Clinical Index, consisting of a 17-item Clinical Signs checklist and a 13-item Medical History questionnaire, was designed to provide an economic screening instrument for alcoholism not subject to self-report biases. There has been little validation work with this instrument. We therefore administered the Alcohol Clinical Index to a group of 40 alcohol-dependent men undergoing treatment at a VA Medical Center and a group of 17 nonalcoholic men being treated at an outpatient medical clinic at the same facility. The Clinical Signs checklist had high sensitivity, but poor specificity. Overall accuracy was 70% for a threshold score from 1 to 5. The Medical History questionnaire had more moderate sensitivity, but 100% specificity for a cutoff score of 2 or more. At a cutoff of 2, it had an overall accuracy of 84%. The high specificity of the Medical History questionnaire suggests that it could be used to help discriminate between alcoholics and nonalcoholics. Further study of these measures and individual items within the measures is recommended with different groups of alcohol users and nonusers.