The dangers of inferring treatment effects from observational data: a case study in HIV infection

The Chronicity of HIV Infection Should Drive the Research Strategy of NeuroHIV Treatment Studies: A Critical Review

HIV infection has become a chronic illness when successfully treated with combined antiretroviral therapy (cART). The long-term health prognosis of aging with controlled HIV infection and HIV-associated neurocognitive disorder (HAND) remains unclear. In this review, we propose that, almost 20 years after the introduction of cART, a change in research focus is needed, with a greater emphasis on chronicity effects driving our research strategy. We argue that pre-emptive documentation of episodes of mild neurocognitive dysfunction is needed to determine their long-term prognosis. This strategy would also seek to optimally represent the entire HAND spectrum in therapeutic trials to assess positive and/or negative treatment effects on brain functions. In the first part of the paper, to improve the standard implementation of the Frascati HAND diagnostic criteria, we provide a brief review of relevant quantitative neuropsychology concepts to clarify their appropriate application for a non-neuropsychological audience working in HIV research and wanting to conduct randomized clinical trials on brain functions. The second part comprises a review of various antiretroviral drug classes and individual agents with respect to their effects on HAND, while also addressing the question of when cART should be initiated to potentially reduce HAND incidence. In each section, we use recent observational studies and randomized controlled trials to illustrate our perspective while also providing relevant statistical comments. We conclude with a discussion of the neuroimaging methods that could be combined with neuropsychological approaches to enhance the validity of HIV neurology (neuroHIV) treatment effect studies.

Costs and persistence of alpha-2 adrenergic agonists versus carbonic anhydrase inhibitors, both associated with prostaglandin analogues, for glaucoma as recorded by The United Kingdom General Practitioner Research Database

The persistence and costs of carbonic anhydrase inhibitors + prostaglandin analogues (CAIs + PGAs) vs alpha-2 adrenergic agonists + prostaglandin analogues (alpha-2 agonists + PGAs) were compared, based on The United Kingdom General Practitioner Research Database. Patients with a diagnosis of ocular hypertension, glaucoma, or treated for this, were selected. Selected patients were prescribed CAIs + PGAs or alpha-2 agonists + PGAs. Treatment failure was defined as a prescription change (adding, removing, or replacing glaucoma treatment, or initiating laser or surgery). Times to treatment failure were compared with a Cox model adjusted by a propensity score. Mean patient age was 69.0 years and 47.6% were males. Treatment failure at 1 year was experienced by 58.8% receiving CAIs + PGAs and 66.0% of patients receiving alpha-2 agonists + PGAs (p < 0.001). The hazard ratio for failure was 0.82 (p < 0.001) in favor of CAIs + PGAs after adjusting on age, gender, comorbidities, and duration of follow-up. Adjusted annual costs of glaucoma management did not differ significantly between treatments, £440.63 with alpha-2 agonists + PGAs and £413.37 with CAIs + PGAs. CAIs + PGAs therapies appear more persistent than alpha-2 agonist + PGA in everyday clinical practice, at a similar cost.

Drotrecogin alfa's impact on intensive care workload in real life practice: a propensity score approach

OBJECTIVES

To estimate the impact of drotrecogin alfa (DA) on intensive care workload in an observational study while illustrating the use of propensity score (PS) matching to control for recruitment bias.

METHODS

PREMISS is a prospective, multicenter pre-post study. Its goal was to evaluate DA in the treatment of severe sepsis with multiple organ failure. Inclusions took place before (control patients) and after (DA-treated patients) the drug's market authorization. Workload was measured in euros using the French classification of medical procedures. It was compared between the groups via random effects gamma regression using two techniques: 1) regression adjusting for the patients' initial characteristics on the whole population; and 2) PS matching. A structural equation model was used to explore the pathways leading to a workload increase.

RESULTS

Drotrecogin alfa is estimated to increase intensive care unit (ICU) workload by 20% (P = 0.045) according to the multivariate model and 34% (P = 0.002) according to the PS-matched one. In the structural equation model fitted, only DA's direct effect on the occurrence of bleeding events reaches significance (P = 0.024).

CONCLUSIONS

We found a significant effect of DA on ICU workload with both standard methods of adjustment and PS matching. This effect appears to be mainly due to DA's effect on bleeding events. The analysis illustrated the usefulness of PS methods in the analysis of observational data, as it leads to conclusions similar to the traditional multivariate regression approaches while avoiding making too many adjustments, allowing focusing on the treatment effect.

Assessment of the cost effectiveness of travoprost versus latanoprost as single agents for treatment of glaucoma in France

BACKGROUND AND OBJECTIVE

Control of intraocular pressure (IOP) is a major factor in avoiding visual impairment related to glaucoma. Both the cost and the effectiveness of therapy should be considered when initiating this lifelong treatment. The aim of this study was to assess the cost effectiveness of travoprost versus latanoprost as single agents for the treatment of glaucoma in France.

METHODS

Two surveys, one documenting efficacy and the other costs, were used to provide data for a Markov model. The model reproduced the 5-year course of patients receiving a prostaglandin analogue, travoprost or latanoprost, as monotherapy. The effectiveness criterion was fitted with a Weibull distribution from a national study. Transition probabilities and costs per treatment line were extracted from two French observational databases. Bootstrap techniques were implemented to drive the probabilistic sensitivity analyses. The study compared both treatments given once daily as monotherapy to ambulatory patients with primary open-angle glaucoma or ocular hypertension. The main outcome measure was mean time to treatment change (MTTC). Possible treatment changes were the addition of adjunctive medication, treatment substitution, laser therapy or surgery. After laser therapy or surgery, patients could continue with no treatment or proceed to prostaglandin analogue as monotherapy or treatment substitution. IOP was stratified at treatment onset as < or =20, 21-23 and > or =24 mmHg, respectively. All costs were expressed in 2005 euros.

RESULTS

MTTC was 44.3 months for travoprost and 37.8 for latanoprost. Additional 5-year costs for travoprost were euro51, resulting in an incremental cost-effectiveness ratio without treatment change of euro95 per year. Of patients treated with latanoprost, 1.9% underwent laser therapy or surgery, compared with 1.2% of patients treated with travoprost. The results differed with baseline IOP values, such that 55.6%, 53.9% and 50.4% of patients with pretreatment IOP values of < or =20, 21-23 and > or =24 mmHg, respectively, continued to receive travoprost treatment at 5 years, compared with 32.3%, 26.1% and 26.1% of patients, respectively, receiving latanoprost. Thus, incremental cost-effectiveness ratios (ICERs) without treatment change were euro140, euro45 and euro123 per year, respectively.

CONCLUSION

Travoprost demonstrated a longer effectiveness profile than latanoprost and minimized early treatment changes. The smaller proportion of patients needing a new treatment, laser therapy or surgery virtually compensated for the higher travoprost acquisition cost. Overall, travoprost is cost effective compared with latanoprost, and is most cost effective in patients with pretreatment IOPs between 21 and 23 mmHg.

Unmeasured confounding caused slightly better response to HAART within than outside a randomized controlled trial

OBJECTIVE

To compare the outcome of highly active antiretroviral therapy (HAART) in HIV-infected patients initiating equivalent regimens within and outside a randomized controlled trial (RCT).

STUDY DESIGN AND SETTING

The Danish Protease Inhibitor Study (DAPIS) was a national multicenter RCT comparing initial treatment with indinavir, ritonavir, or saquinavir/ritonavir during 96 weeks. From the Danish HIV Cohort Study we identified all patients initiating one of these protease-inhibitor-based HAART regimens: 425 patients within DAPIS and 677 outside the trial. We compared viral load, CD4 count response, and mortality.

RESULTS

At weeks 96 and 240, trial participants were more likely than nonparticipants to have undetectable viral load (adjusted odds ratio [adOR] 1.28 [95% CI=0.94-1.74] and 1.70 [95% CI=1.16-2.50]) and a CD4 increase > or =100 cells/microl (adOR 1.37 [95% CI=1.03-1.82] and 1.53 [95% CI=1.04-2.25]). For antiretroviral-experienced, but not for antiretroviral-naïve patients, trial participants had a lower risk of death (mortality rate ratio [MRR]=0.46 [95% CI=0.27-0.77]) than nonparticipants. This effect was moderated in adjusted analyses (MRR=0.60 [0.33-1.07]).

CONCLUSIONS

Compared to nontrial patients, trial participants had better response to HAART. The differences were small defying the notion that results obtained in RCTs are unachievable in routine clinical practice.

Considerations in the design of randomized controlled trials evaluating the optimal time to initiate antiretroviral therapy in previously untreated HIV-1-infected patients

PURPOSE OF REVIEW

This paper addresses design issues around 'when to start' trials in adults and children and in chronic and acute HIV infection. Issues for young children treated soon after birth and recently seroconverting adults differ, and are also discussed.

RECENT FINDINGS

No trials have been published addressing this question for combination antiretroviral therapy in chronic HIV infection, and the evidence base for guidelines has largely been derived from observational studies. The biases inherent in observational data and recent results from the adult SMART trial may have moved the level of equipoise from 'unstable certainty' (mitigating against randomization) to a renewed call for a very large clinical endpoint trial of when to start therapy in the era of highly active antiretroviral therapy. Older children could also be enrolled in such a trial, as the predictive value of CD4 cell count values in children over 5 years and young adults. In acute infection, ongoing trials in adults and children is similar considering strategies of early short-term combination antiretroviral therapy compared with deferred therapy. Although not strictly 'when to start trials', these approaches could reduce life-long exposure to antiretroviral therapy while maintaining clinical and immunological well being.

SUMMARY

Issues in the design of a future randomized trial of when to start antiretroviral therapy in chronic HIV infected adults and older children are discussed, while the role of early limited therapy in primary infection awaits results of ongoing trials.

Association of achieved dialysis dose with mortality in the hemodialysis study: an example of "dose-targeting bias"

In the intention-to-treat analysis of the Hemodialysis Study, all-cause mortality did not differ significantly between the high versus standard hemodialysis dose groups. The association of mortality with delivered dose within each of the two randomized treatment groups was examined, and implications for observational studies were considered. Time-dependent Cox regression was used to relate the relative risk (RR) for mortality to the running mean of the achieved equilibrated Kt/V (eKt/V) over the preceding 4 mo. eKt/V was categorized by quintiles within each dose group. Analyses were controlled for case-mix factors and baseline anthropometric volume. Within each randomized dose group, mortality was elevated markedly when achieved eKt/V was in the lowest quintile (RR, 1.93; 95% confidence interval [CI], 1.40 to 2.66; P < 0.0001 in the standard-dose group; RR, 2.04; 95% CI, 1.50 to 2.76; P < 0.0001 in the high-dose group; RR relative to the middle quintiles). The mortality rate in the lowest eKt/V quintile of the high-dose group was higher than in the full standard-dose group (RR, 1.59; 95% CI, 1.29 to 1.96; P < 0.0001). Each 0.1 eKt/V unit below the group median was associated with a 58% higher mortality in the standard-dose group (P < 0.001) and a 37% higher mortality in the high-dose group (P < 0.001). The magnitude of these dose-mortality effects was seven- to 12-fold higher than the upper limit of the 95% CI from the intention-to-treat analysis. The effects were attenuated in lagged analyses but did not disappear. When dialysis dose is targeted closely, as under the controlled conditions of the Hemodialysis Study, patients with the lowest achieved dose relative to their target dose experience markedly increased mortality, to a degree that is not compatible with a biologic effect of dose. The possibility of similar (albeit smaller) biases should be considered when analyzing observational data sets relating mortality to achieved dose of dialysis.

Adjusting for patient selection suggests the addition of docetaxel to 5-fluorouracil-cisplatin induction therapy may offer survival benefit in squamous cell cancer of the head and neck

When induction chemotherapy is used in locally advanced squamous cell cancer of the head and neck (SCCHN), patients often receive cisplatin-5-fluorouracil (PF) followed by radical loco-regional therapy. Phase II studies of docetaxel-cisplatin-5-fluorouracil (TPF) induction therapy, with or without leucovorin (L), have achieved high survival rates versus those reported in phase III PF trials. However, the distribution of prognostic factors may vary between phase II and phase III study populations, making the extrapolation of phase II TPF/L results to phase III PF populations difficult. This study used a patient selection standardization method and Cox model to adjust for potential selection bias. Thus, the survival benefit from adding docetaxel into PF induction regimens in SCCHN could be more accurately assessed. The TPF/L dataset comprised 195 patients from six phase II trials. The PF dataset of 585 patients was derived from five large randomized trials included in the Meta-Analysis of Chemotherapy in Head and Neck Cancer (MACH-NC) database. TPF/L and PF datasets differed significantly concerning the distribution of several prognostic factors. Adjusting for these differences, the relative risk of death in the PF versus TPF/L datasets was 1.85 (95% confidence interval 1.37-2.49), corresponding to a 20% 2-year survival benefit (p < 0.0001). Sensitivity analyses confirmed that this improved 2-year survival rate of TPF/L over PF was robust, irrespective of the distribution of studied prognostic factors between treatment datasets. We conclude that this improved survival might be due either to docetaxel's pharmacologic effect or to uncontrolled prognostic factors.

The Problems Faced When Assessing the Prevalence and Incidence of Antiretroviral-Related Toxicities

Although the dramatic effect of highly active antiretroviral therapy (HAART) in reducing morbidity and mortality must not be underestimated, it is also important to consider the incidence and prevalence of HAART-related toxicities. Although several studies have investigated HAART-related toxicities, there has been great variety between them in the reported incidence and prevalence rates of these toxicities. Various factors, including whether the study type was a clinical trial or an observational study, the definition of the toxicity endpoints, the demographic characteristics of the study populations and the effect of calendar year on analyses, may all influence the rates observed. We investigated the possible explanations for the differences in the incidence and prevalence rates of HAART-related toxicities between studies, focussing on metabolic and hepatotoxic disorders.

Patterns of antiretroviral use in the United States of America: analysis of three observational databases

OBJECTIVE

To characterize patterns of antiretroviral use in HIV-infected patients and explore variation by patient characteristics and disease stage.

METHODS

Three large patient databases recording information derived from routine clinical attendance were analyzed: HIV Insight (n = 10 873), Target Management Services (n = 2226) and Clinical Partners (n = 1505). Each database records the dates of starting and stopping individual antiretroviral agents over time, measurements of CD4 T-cell counts and HIV-RNA levels at approximately 6-monthly intervals, and the demographic characteristics of patients. The number, frequency and duration of different antiretroviral combinations over time and their relationship to stage of HIV-disease and demographic characteristics were explored.

RESULTS

Over 2000 different combinations of antiretroviral agents are recorded. From 1987 onwards, the use of zidovudine increased, with 23% of patients receiving monotherapy by 1990. The majority of treated patients remained on monotherapy until the introduction of highly active antiretroviral therapy (HAART) in 1996. By 1999, the standard of care was HAART, with 84% of patients beginning antiretroviral therapy with HAART. Those of African American race (odds ratio 0.59) and funded by Medicaid (odds ratio 0.72) were significantly less likely to begin antiretroviral therapy on HAART. Until 1995, there was a significant decrease in CD4 T-cell count when starting antiretroviral therapy. No significant trend was observed in either CD4 T-cell count or viral load after this time. Those starting on HAART therapies were significantly less likely to stop or switch regimens than those on nucleoside reverse transcriptase inhibitor (NRTI)-only therapies (P < 0.001).

CONCLUSIONS

Complex patterns of antiretroviral treatment are observed in this large population. Changes over time mirror the introduction of the new antiretroviral agents.

Effect of dialysis dose and membrane flux in maintenance hemodialysis

BACKGROUND

The effects of the dose of dialysis and the level of flux of the dialyzer membrane on mortality and morbidity among patients undergoing maintenance hemodialysis are uncertain.

METHODS

We undertook a randomized clinical trial in 1846 patients undergoing thrice-weekly dialysis, using a two-by-two factorial design to assign patients randomly to a standard or high dose of dialysis and to a low-flux or high-flux dialyzer.

RESULTS

In the standard-dose group, the mean (+/-SD) urea-reduction ratio was 66.3+/-2.5 percent, the single-pool Kt/V was 1.32+/-0.09, and the equilibrated Kt/V was 1.16+/-0.08; in the high-dose group, the values were 75.2+/-2.5 percent, 1.71+/-0.11, and 1.53+/-0.09, respectively. Flux, estimated on the basis of beta2-microglobulin clearance, was 3+/-7 ml per minute in the low-flux group and 34+/-11 ml per minute in the high-flux group. The primary outcome, death from any cause, was not significantly influenced by the dose or flux assignment: the relative risk of death in the high-dose group as compared with the standard-dose group was 0.96 (95 percent confidence interval, 0.84 to 1.10; P=0.53), and the relative risk of death in the high-flux group as compared with the low-flux group was 0.92 (95 percent confidence interval, 0.81 to 1.05; P=0.23). The main secondary outcomes (first hospitalization for cardiac causes or death from any cause, first hospitalization for infection or death from any cause, first 15 percent decrease in the serum albumin level or death from any cause, and all hospitalizations not related to vascular access) also did not differ significantly between either the dose groups or the flux groups. Possible benefits of the dose or flux interventions were suggested in two of seven prespecified subgroups of patients.

CONCLUSIONS

Patients undergoing hemodialysis thrice weekly appear to have no major benefit from a higher dialysis dose than that recommended by current U.S. guidelines or from the use of a high-flux membrane.