How generalizable are the results of large randomized controlled trials of antiretroviral therapy?

A conceptual framework for external validity

Evidence-Based Medicine (EBM) encourages clinicians to seek the most reputable evidence. The quality of evidence is organized in a hierarchy in which randomized controlled trials (RCTs) are regarded as least biased. However, RCTs are plagued by poor generalizability, impeding the translation of clinical research to practice. Though the presence of poor external validity is known, the factors that contribute to poor generalizability have not been summarized and placed in a framework. We propose a new population-oriented conceptual framework to facilitate consistent and comprehensive evaluation of generalizability, replicability, and assessment of RCT study quality.

Effectiveness and safety of first-line antiretroviral regimens in clinical practice: a multicentre cohort study

OBJECTIVES

We compared 48 week effectiveness and safety of first-line antiretroviral regimens.

METHODS

We analysed HIV treatment-naive adults from the Cohort of the Spanish HIV/AIDS Research Network (CoRIS) starting the most commonly used antiretroviral regimens from 2014 to 2018. We used multivariable regression models to assess the impact of initial regimen on: (i) viral suppression (VS) (viral load <50 copies/mL); (ii) change in CD4 cell count; (iii) CD4/CD8 normalization (>0.4 and >1); (iv) CD4 percentage normalization (>29%); (v) multiple T-cell marker recovery (MTMR: CD4 > 500 cells/mm3 plus CD4 percentage >29% plus CD4/CD8 > 1); (vi) lipid, creatinine and transaminase changes; and (vii) discontinuations due to adverse events (AE).

RESULTS

Among 3945 individuals analysed, the most frequently prescribed regimens were ABC/3TC/DTG (34.0%), TAF/FTC/EVG/CBT (17.2%), TDF/FTC + DTG (11.9%), TDF/FTC/EVG/CBT (11.7%), TDF/FTC/RPV (11.5%), TDF/FTC + bDRV (8.3%) and TDF/FTC + RAL (5.3%). At 48 weeks, 89.7% of individuals achieved VS with no significant differences by initial regimen. CD4 mean increase was 257.8 (249.3; 266.2) cells/mm3, and it was lower with TAF/FTC/EVG/CBT and TDF/FTC/RPV compared with ABC/3TC/DTG. CD4 percentage normalization was less likely with TAF/FTC/EVG/CBT, and MTMR was less likely with TAF/FTC/EVG/CBT and TDF/FTC + RAL. The proportion of discontinuations due to AE was higher with TDF/FTC + bDRV (9.7%), followed by TDF/FTC/EVG/CBT (9.5%) and TDF/FTC + DTG (7.9%). Compared with ABC/3TC/DTG, cholesterol and LDL mean increases were higher with TAF/FTC/EVG/CBT and lower with TDF/FTC + DTG, TDF/FTC/RPV and TDF/FTC + RAL. Higher mean increases in triglycerides were significantly associated with TAF/FTC/EVG/CBT. Regimens containing DTG showed higher creatinine increases.

CONCLUSIONS

The significantly greater immunological response and safety of some combinations may be useful for making decisions when initiating treatment.

Compared to randomized studies, observational studies may overestimate the effectiveness of DOACs: a metaepidemiological approach

BACKGROUND AND OBJECTIVES

Randomized controlled trials (RCTs) are criticized for including patients who are overselected. Health authorities consequently encourage "real-world" postmarketing cohort studies. Our objective was to determine the differences between RCTs and observational studies as regards their populations and efficacy/safety results.

METHODS

A systematic review was conducted to identify RCTs and observational studies including patients with venous thromboembolism receiving direct oral anticoagulants or conventional treatment. Ratios of hazard ratio (RHR) comparing epidemiological studies (prospective and retrospective cohort studies and studies using living databases) with RCTs were computed.

RESULTS

Six RCTs (27,121 patients) and twenty observational studies (248,971 patients) were identified and analyzed. Prospective cohort studies seemed to recruit patients who were no less selected than those of RCTs whereas other types of observational studies may reflect the population treated in real life. Among observational studies, prospective cohort studies yielded the most favorable estimates of treatment effect compared with RCTs. These studies were associated with a nonsignificant 33% increase in efficacy estimate (RHR 0.67, [95% CI, 0.39-1.18]) but no effect on safety estimate. Studies using living databases were associated with nonsignificant trends toward a greater effect on efficacy (RHR 0.82, [0.66-1.01]) and a smaller effect on safety (RHR 1.33, [0.96-1.84]).

DISCUSSION

Overall, in this clinical setting, an exaggeration of the treatment efficacy estimate was seen with observational studies compared with RCTs.

CONCLUSIONS

As the presence of residual confounding cannot be excluded, these results should be interpreted cautiously.

Effectiveness of the combination elvitegravir/cobicistat/tenofovir/emtricitabine (EVG/COB/TFV/FTC) plus darunavir among treatment-experienced patients in clinical practice: a multicentre cohort study

BACKGROUND

The aim of this study was to investigate the effectiveness and tolerability of the combination elvitegravir/cobicistat/tenofovir/emtricitabine plus darunavir (EVG/COB/TFV/FTC + DRV) in treatment-experienced patients from the cohort of the Spanish HIV/AIDS Research Network (CoRIS).

METHODS

Treatment-experienced patients starting treatment with EVG/COB/TFV/FTC + DRV during the years 2014-2018 and with more than 24 weeks of follow-up were included. TFV could be administered either as tenofovir disoproxil fumarate or tenofovir alafenamide. We evaluated virological response, defined as viral load (VL) < 50 copies/ml and < 200 copies/ml at 24 and 48 weeks after starting this regimen, stratified by baseline VL (< 50 or ≥ 50 copies/ml at the start of the regimen).

RESULTS

We included 39 patients (12.8% women). At baseline, 10 (25.6%) patients had VL < 50 copies/ml and 29 (74.4%) had ≥ 50 copies/ml. Among patients with baseline VL < 50 copies/ml, 85.7% and 80.0% had VL < 50 copies/ml at 24 and 48 weeks, respectively, and 100% had VL < 200 copies/ml at 24 and 48 weeks. Among patients with baseline VL ≥ 50 copies/ml, 42.3% and 40.9% had VL < 50 copies/ml and 69.2% and 68.2% had VL < 200 copies/ml at 24 and 48 weeks. During the first 48 weeks, no patients changed their treatment due to toxicity, and 4 patients (all with baseline VL ≥ 50 copies/ml) changed due to virological failure.

CONCLUSIONS

EVG/COB/TFV/FTC + DRV was well tolerated and effective in treatment-experienced patients with undetectable viral load as a simplification strategy, allowing once-daily, two-pill regimen with three antiretroviral drug classes. Effectiveness was low in patients with detectable viral loads.

Durability of first-line antiretroviral regimens in the era of integrase inhibitors: a cohort of HIV-positive individuals in Spain, 2014-2015

BACKGROUND

We compared time to treatment change (TC), viral suppression (VS) and change in CD4⁺ T-cell counts of first-line antiretroviral regimens (ART).

METHODS

We analysed HIV treatment-naive adults from the Cohort of the Spanish HIV/AIDS Research Network (CoRIS) initiating the most commonly used ART regimens from September 2014 to November 2015. We used proportional hazards models on the sub-distribution hazard to estimate sub-distribution hazard ratios (sHR) for time to TC, logistic regression to estimate odds ratios (ORs) for VS (viral load <50 copies/ml), and linear regression to assess mean differences in CD4⁺ T-cell changes from ART initiation.

RESULTS

Among 960 individuals, tenofovir (TDF)/emtricitabine (FTC)/rilpivirine (RPV) was the most frequently prescribed regimen (24.2%), followed by elvitegravir (EVG)/cobicistat (COBI)/TDF/FTC (22.8%), abacavir (ABC)/lamivudine (3TC)/dolutegavir (DTG; 17.4%), TDF/FTC+darunavir/ritonavir (DRV/r) or darunavir/cobicistat (DRV/c; 12.1%), TDF/FTC/efavirenz (EFV; 8.8%), TDF/FTC+raltegravir (RAL; 7.7%) and TDF/FTC+DTG (7.0%). Initiating ART with TDF/FTC+DRV/r or DRV/c (adjusted sHR: 2.96; 95% CI: 1.44, 6.08), TDF/FTC/EFV (2.18; 0.98, 4.82), TDF/FTC+RAL (2.37; 1.08, 5.22) and TDF/FTC+DTG (6.34; 3.18, 12.64) was associated with a higher risk of TC compared to ABC/3TC/DTG. At 24 weeks, VS was lower in TDF/FTC+DRV/r or DRV/c (adjusted OR: 0.37, 95% CI: 0.18, 0.74) compared with ABC/3TC/DTG, and CD4⁺ T-cell increase was lower in patients initiating with TDF/FTC/RPV (adjusted mean difference: -75.9, 95% CI: -130.6, -21.2) compared with those who did with ABC/3TC/DTG.

CONCLUSIONS

Time to TC, VS and change in CD4⁺ T-cell counts varies by initial regimen. These differences may be useful for making decision when initiating ART.

Clinical Trial Generalizability Assessment in the Big Data Era: A Review

Clinical studies, especially randomized, controlled trials, are essential for generating evidence for clinical practice. However, generalizability is a long‐standing concern when applying trial results to real‐world patients. Generalizability assessment is thus important, nevertheless, not consistently practiced. We performed a systematic review to understand the practice of generalizability assessment. We identified 187 relevant articles and systematically organized these studies in a taxonomy with three dimensions: (i) data availability (i.e., before or after trial (a priori vs. a posteriori generalizability)); (ii) result outputs (i.e., score vs. nonscore); and (iii) populations of interest. We further reported disease areas, underrepresented subgroups, and types of data used to profile target populations. We observed an increasing trend of generalizability assessments, but < 30% of studies reported positive generalizability results. As a priori generalizability can be assessed using only study design information (primarily eligibility criteria), it gives investigators a golden opportunity to adjust the study design before the trial starts. Nevertheless, < 40% of the studies in our review assessed a priori generalizability. With the wide adoption of electronic health records systems, rich real‐world patient databases are increasingly available for generalizability assessment; however, informatics tools are lacking to support the adoption of generalizability assessment practice.

Diagnostic Assessment of Assumptions for External Validity: An Example Using Data in Metastatic Colorectal Cancer

BACKGROUND

Methods developed to estimate intervention effects in external target populations assume that all important effect measure modifiers have been identified and appropriately modeled. Propensity score-based diagnostics can be used to assess the plausibility of these assumptions for weighting methods.

METHODS

We demonstrate the use of these diagnostics when assessing the transportability of treatment effects from the standard of care for metastatic colorectal cancer control arm in a phase III trial (HORIZON III) to a target population of 1,942 Medicare beneficiaries age 65+ years.

RESULTS

In an unadjusted comparison, control arm participants had lower mortality compared with target population patients treated with the standard of care therapy (trial vs. target hazard ratio [HR] = 0.72, 95% confidence interval [CI], 0.58, 0.89). Applying inverse odds of sampling weights attenuated the trial versus target HR (weighted HR = 0.96, 95% CI = 0.73, 1.26). However, whether unadjusted or weighted, hazards did not appear proportional. At 6 months of follow-up, mortality was lower in the weighted trial population than the target population (weighted trial vs. target risk difference [RD] = -0.07, 95% CI = -0.13, -0.01), but not at 12 months (weighted RD = 0.00, 95% CI = -0.09, 0.09).

CONCLUSION

These diagnostics suggest that direct transport of treatment effects from HORIZON III to the Medicare population is not valid. However, the proposed sampling model might allow valid transport of the treatment effects on longer-term mortality from HORIZON III to the Medicare population treated in clinical practice. See video abstract at, http://links.lww.com/EDE/B435.

The Applicability of N: Ancient Debates and Modern Experimental Design

Medicine has always been characterized by a tension between the particular and the general. A clinician is obligated to treat the individual in front of her, yet she accomplishes this task by applying generalized knowledge that describes an abstract average but not necessarily a specific person. Efforts to systematize this process of moving between the particular and the general have led to the development of randomized controlled trials and large observational studies. Inclusion of tens of thousands of people in such studies, it is argued, will enhance the applicability of the data to more individual circumstances. Yet, as genetic sequencing data have become more widely obtained and used, there has been an increased focus on what has been broadly termed "precision medicine", a highly individualized approach to therapeutics. Moreover, advances in statistical methods have enabled researchers to use N-of-1 study data-traditionally considered too individualized to be broadly applicable-in new ways. This paper contextualizes these apparently modern debates with reference to historical arguments about methods of disease diagnosis and treatment, and earlier physicians' concerns about the tension between the particular and the general that is intrinsic to medical practice.

Representativeness of the participants in the smoking Cessation in Pregnancy Incentives Trial (CPIT): a cross-sectional study

Background

The limited representativeness of trial samples may restrict external validity. The aim of this study was to ascertain the representativeness of the population enrolled in the Cessation in Pregnancy Incentives Trial (CPIT), a therapeutic exploratory study to examine the effectiveness of financial incentives for smoking cessation during pregnancy.

Methods

CPIT participants (n = 492) were compared with all self-reported smokers at maternity booking who did not participate in the trial (n = 1982). Both groups were drawn from the National Health Service (NHS) Greater Glasgow and Clyde area over a 1-year trial enrolment period. Variables used for comparison were age, area-based deprivation index, body mass index, gestation, and carbon monoxide (CO) breath test level. Chi-square and Mann-Whitney U tests were used to compare groups.

Results

From January to December 2012, 2474/13,945 (17.7 %) women, who booked for maternity care, self-reported as current smokers (at least one cigarette in the last week). Seven hundred and fifty-two were ineligible for trial participation because of a CO breath test level of less than 7 parts per million (ppm) used as a biochemical cut-off to corroborate self-report of current smoking. At telephone consent 301 could not be contacted, 11 had miscarried, 16 did not give consent and 3 opted out after randomisation, leaving 492 participants for analysis. There were no differences in demographic or clinical characteristics between trial participants, and self-reported smokers not enrolled in the trial in terms of CO breath test (as a measure of smoking level for those with a CO level of 7 ppm or higher), material deprivation (using an area-based measure), maternal age and maternal body mass index. Gestation at booking was statistically significantly lower for participants.

Conclusions

To ensure that all trial participants were smokers, biochemical validation excluded self-reported smokers with a CO level of less than 7 ppm from taking part in the trial, which excluded 30 % of self-reported smokers who were ‘lighter’ smokers. The efficacy of financial incentives would not have been likely to decrease if ‘lighter’ smokers had been included in the trial population. Trial participants were slightly earlier in their pregnancy at maternity booking, but this difference would not clinically affect the provision of financial incentives if provided routinely. Overall, the trial population was representative of all self-reported smokers with regard to available routinely collected data. Appropriate comparison of trial and target populations, with detailed reporting of exclusion criteria would contribute to the understanding of the wider applicability of trial results.

Trial registration

Current Controlled Trials ISRCTN87508788. Registered/Assigned on 1 September 2011.

Evaluating the Generalisability of Trial Results: Introducing a Centre- and Trial-Level Generalisability Index

BACKGROUND

Few randomised controlled trials (RCTs) recruit centres representatively, which may limit the external validity of trial results.

OBJECTIVE

The aim of this study was to propose a proof-of-concept method of assessing the generalisability of the clinical and cost-effectiveness findings of a given RCT.

METHODS

We developed a generalisability index (Gix), informed by centre-level characteristics, as a measure of centre and trial representativeness. The centre-level Gix quantifies how representative a centre is in relation to its jurisdiction, e.g. a country or health authority. The trial-level Gix quantifies how representative trial recruitment is in relation to clinical practice in the jurisdiction. Taking a real-world RCT as a case study and assuming trial-wide results to represent 'true jurisdiction values', we used simulation methods to recreate 5000 RCTs and investigate the relationship between trial representativeness, reflected by the standardised trial-Gix, and the deviation of simulated trial results from the 'true values'.

RESULTS

The simulation study provides evidence that trial results (odds ratio for the primary outcome and incremental quality-adjusted life-years) were influenced by the representativeness of the sample of recruiting centres. Simulated RCTs with the closest results to the 'true values' were those whose recruitment closely mirrored the jurisdiction-wide context. Results appeared robust to six alternative specifications of the Gix.

CONCLUSIONS

Our findings suggest that an unrepresentative selection of centres limits the external validity of trial results. The Gix may be a valuable tool to help facilitate rational selection of trial centres and ensure the generalisability of results at the jurisdiction level.

HIV research trials versus standard clinics for antiretroviral-naïve patients: the outcomes differ but do the patients?

Exclusion criteria for HIV treatment-naïve drug trials can be stringent and selection bias exists, making it difficult to extrapolate results into the 'real world' clinical situation. We aim to compare the demographics, virological outcomes and psychosocial complexity in adult HIV-infected treatment-naïve patients from our cohort initiating combination antiretroviral therapy (cART) in research trials versus standard clinics. In our unit from 2006 to 2011, 1202 standard clinic and 69 research trial patients initiated cART; every eighth standard clinics patient was included to create a standard clinics:research trials patient ratio of 2:1. Notes were retrospectively reviewed for patient demographics, attendance rates and virological outcomes. Data from 221 antiretroviral-naïve patients starting cART were analysed: 152 standard clinic patients and 69 from research trials. In the research trials group, there was an overrepresentation of men (p = 0.041), men who have sex with men (p < 0.001), patients of white ethnicity (p = 0.01), employed patients (p = 0.01) and patients using excessive alcohol (p = 0.02). There was equal representation of drug use, depression and referral to psychology, psychiatry and social work in both groups. The research trials group at baseline had significantly higher CD4 counts (p < 0.001), lower viral loads (p = 0.01) and more patients achieved undetectable viral loads at three (p < 0.001), six (p < 0.001) and 24 months (p = 0.033). There is a prevailing common preconception that participants in clinical trials are uncomplicated, unlike their 'real-life' counterparts. We demonstrated important similarities in psychosocial complexity as well as differences in demographics and virological outcomes in trial and non-trial patients. Clinicians need to be aware of these discrepancies to ensure the facilitation of a heterogeneous population participating in research trials.

Centre selection for clinical trials and the generalisability of results: a mixed methods study

BACKGROUND

The rationale for centre selection in randomised controlled trials (RCTs) is often unclear but may have important implications for the generalisability of trial results. The aims of this study were to evaluate the factors which currently influence centre selection in RCTs and consider how generalisability considerations inform current and optimal practice.

METHODS AND FINDINGS

Mixed methods approach consisting of a systematic review and meta-summary of centre selection criteria reported in RCT protocols funded by the UK National Institute of Health Research (NIHR) initiated between January 2005-January 2012; and an online survey on the topic of current and optimal centre selection, distributed to professionals in the 48 UK Clinical Trials Units and 10 NIHR Research Design Services. The survey design was informed by the systematic review and by two focus groups conducted with trialists at the Birmingham Centre for Clinical Trials. 129 trial protocols were included in the systematic review, with a total target sample size in excess of 317,000 participants. The meta-summary identified 53 unique centre selection criteria. 78 protocols (60%) provided at least one criterion for centre selection, but only 31 (24%) protocols explicitly acknowledged generalisability. This is consistent with the survey findings (n = 70), where less than a third of participants reported generalisability as a key driver of centre selection in current practice. This contrasts with trialists' views on optimal practice, where generalisability in terms of clinical practice, population characteristics and economic results were prime considerations for 60% (n = 42), 57% (n = 40) and 46% (n = 32) of respondents, respectively.

CONCLUSIONS

Centres are rarely enrolled in RCTs with an explicit view to external validity, although trialists acknowledge that incorporating generalisability in centre selection should ideally be more prominent. There is a need to operationalize 'generalisability' and incorporate it at the design stage of RCTs so that results are readily transferable to 'real world' practice.

Week 48 analysis of once-daily vs. twice-daily darunavir/ritonavir in treatment-experienced HIV-1-infected patients

OBJECTIVE

ODIN (Once-daily Darunavir In treatment-experieNced patients) was a phase III, 48-week, open-label study comparing once-daily vs. twice-daily darunavir/ritonavir (DRV/r) in treatment-experienced patients with no DRV resistance-associated mutations (RAMs) at screening.

METHODS

Patients with no DRV RAMs and receiving stable HAART for at least 12 weeks were stratified by HIV-1 RNA (≤ or > 50 000 copies/ml) and randomized to once-daily DRV/r 800/100 mg or twice-daily DRV/r 600/100 mg and an optimized background regimen (≥2 nucleoside reverse transcriptase inhibitors). Primary objective was to demonstrate noninferiority of once-daily vs. twice-daily DRV/r in confirmed virologic response (HIV-1 RNA < 50 copies/ml) at week 48.

RESULTS

Five hundred and ninety patients received once-daily (n = 294) or twice-daily (n = 296) DRV/r. Mean baseline HIV-1 RNA was 4.16 log10 copies/ml; median CD4 cell count was 228 cells/μl; and 53.9% had previously used at least one protease inhibitor. At week 48, 72.1% of once-daily and 70.9% of twice-daily patients achieved HIV-1 RNA less than 50 copies/ml (intent-to-treat/time-to-loss of virologic response). The difference in response between once-daily and twice-daily arms was 1.2% (95% confidence interval -6.1 to 8.5%; P < 0.001), establishing noninferiority of once-daily DRV/r versus twice-daily DRV/r. Median CD4 cell count increase was 100 (once-daily) and 94 cells/μl (twice-daily). Virologic failure rate was low and similar for both arms; only one patient (once-daily arm) developed primary protease inhibitor mutations. Once-daily DRV/r had a lower incidence of grade 2-4 triglyceride increases (5.2 vs. 11.0%, P < 0.05).

CONCLUSION

Once-daily DRV/r 800/100 mg was noninferior in virologic response to twice-daily DRV/r 600/100 mg at 48 weeks in treatment-experienced patients with no DRV RAMs, and with a more favorable lipid profile. These findings support use of once-daily DRV/r in this population.

Recruitment of HIV/AIDS treatment-naïve patients to clinical trials in the highly active antiretroviral therapy era: influence of gender, sexual orientation and race

BACKGROUND

In the USA, women, racial/ethnic minorities and persons who acquire HIV infection through heterosexual intercourse represent an increasing proportion of HIV-infected persons, and yet are frequently underrepresented in clinical trials. We assessed the demographic predictors of trial participation in antiretroviral-naïve patients.

METHODS

Patients were characterized as trial participants if highly active antiretroviral therapy (HAART) was initiated within a clinical trial. Prevalence ratios (PRs) were obtained using binomial regression.

RESULTS

Between 1996 and 2006, 30% of 738 treatment-naïve patients initiated HAART in a clinical trial. Trial participation rates for men who have sex with men (MSM), heterosexual men, and women were respectively 36.5, 29.6 and 24.3%. After adjustment for other factors, heterosexual men appeared less likely to participate in trials compared with MSM [PR 0.79, 95% confidence interval (CI) 0.57, 1.11], while women were as likely to participate as MSM (PR 0.97, 95% CI 0.68, 1.39). The participation rate in Black patients (25.9%) was lower compared with non-Black patients (37.5%) (adjusted PR 0.80, 95% CI 0.60, 1.06).

CONCLUSIONS

In our clinical setting, gender did not appear to impact participation in HIV treatment trials, but Black patients were slightly less likely to participate in these trials. Considering the substantial proportion of HIV-infected patients who are Black, future trials need to consider strategies to incorporate such underrepresented populations.

Outcomes of patients who participate in randomized controlled trials compared to similar patients receiving similar interventions who do not participate

BACKGROUND

Some people believe that patients who take part in randomised controlled trials (RCTs) face risks that they would not face if they opted for non-trial treatment. Others think that trial participation is beneficial and the best way to ensure access to the most up-to-date physicians and treatments. This is an updated version of the original Cochrane review published in Issue 1, 2005.

OBJECTIVES

To assess the effects of patient participation in RCTs ('trial effects') independent both of the effects of the clinical treatments being compared ('treatment effects') and any differences between patients who participated in RCTs and those who did not. We aimed to compare similar patients receiving similar treatment inside and outside of RCTs.

SEARCH STRATEGY

In March 2007, we searched The Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, EMBASE, The Cochrane Methodology Register, SciSearch and PsycINFO for potentially relevant studies. Our search yielded 7586 new references. In addition, we reviewed the reference lists of relevant articles.

SELECTION CRITERIA

Randomized studies and cohort studies with data on clinical outcomes of RCT participants and similar patients who received similar treatment outside of RCTs.

DATA COLLECTION AND ANALYSIS

At least two review authors independently assessed studies for inclusion, assessed study quality and extracted data.

MAIN RESULTS

We identified 30 new non-randomized cohort studies (45 comparisons): no new RCTs were found. This update now includes five RCTs (yielding 6 comparisons) and 80 non-randomized cohort studies (130 comparisons), with 86,640 patients treated in RCTs and 57,205 patients treated outside RCTs. In the randomised studies, patients were invited to participate in an RCT or not; these comparisons provided limited information because of small sample sizes (a total of 412 patients) and the nature of the questions they addressed. When the results of RCTs and non-randomized cohorts that reported dichotomous outcomes were combined, there were 98 comparisons; there was also heterogeneity (P < 0.00001, I(2) = 42.2%) between studies. No statistical significant differences were found for 85 of the 98 comparisons. Eight comparisons reported statistically significant better outcomes for patients treated within RCTs, and five comparisons reported statistically significant worse outcomes for patients treated within RCTs. There was significant heterogeneity (P < 0.00001, I(2) = 58.2%) among the 38 continuous outcome comparisons. No statistically significant differences were found for 30 of the 38 comparisons. Three comparisons reported statistically significant better outcomes for patients treated within RCTs, and five comparisons reported statistically significant worse outcomes for patients treated within RCTs.

AUTHORS' CONCLUSIONS

This review indicates that participation in RCTs is associated with similar outcomes to receiving the same treatment outside RCTs. These results challenge the assertion that the results of RCTs are not applicable to usual practice.

Unmeasured confounding caused slightly better response to HAART within than outside a randomized controlled trial

OBJECTIVE

To compare the outcome of highly active antiretroviral therapy (HAART) in HIV-infected patients initiating equivalent regimens within and outside a randomized controlled trial (RCT).

STUDY DESIGN AND SETTING

The Danish Protease Inhibitor Study (DAPIS) was a national multicenter RCT comparing initial treatment with indinavir, ritonavir, or saquinavir/ritonavir during 96 weeks. From the Danish HIV Cohort Study we identified all patients initiating one of these protease-inhibitor-based HAART regimens: 425 patients within DAPIS and 677 outside the trial. We compared viral load, CD4 count response, and mortality.

RESULTS

At weeks 96 and 240, trial participants were more likely than nonparticipants to have undetectable viral load (adjusted odds ratio [adOR] 1.28 [95% CI=0.94-1.74] and 1.70 [95% CI=1.16-2.50]) and a CD4 increase > or =100 cells/microl (adOR 1.37 [95% CI=1.03-1.82] and 1.53 [95% CI=1.04-2.25]). For antiretroviral-experienced, but not for antiretroviral-naïve patients, trial participants had a lower risk of death (mortality rate ratio [MRR]=0.46 [95% CI=0.27-0.77]) than nonparticipants. This effect was moderated in adjusted analyses (MRR=0.60 [0.33-1.07]).

CONCLUSIONS

Compared to nontrial patients, trial participants had better response to HAART. The differences were small defying the notion that results obtained in RCTs are unachievable in routine clinical practice.

Characteristics and management of HIV-1-infected pregnant women enrolled in a randomised trial: differences between Europe and the USA

BACKGROUND

Rates of mother-to-child transmission of HIV-1 (MTCT) have historically been lower in European than in American cohort studies, possibly due to differences in population characteristics. The Pediatric AIDS Clinical Trials Group Protocol (PACTG) 316 trial evaluated the effectiveness of the addition of intrapartum/neonatal nevirapine in reducing MTCT in women already receiving antiretroviral prophylaxis. Participation of large numbers of pregnant HIV-infected women from the US and Western Europe enrolling in the same clinical trial provided the opportunity to identify and explore differences in their characteristics and in the use of non-study interventions to reduce MTCT.

METHODS

In this secondary analysis, 1350 women were categorized according to enrollment in centres in the USA (n = 978) or in Europe (n = 372). Factors associated with receipt of highly active antiretroviral therapy and with elective caesarean delivery were identified with logistic regression.

RESULTS

In Europe, women enrolled were more likely to be white and those of black race were mainly born in Sub-Saharan Africa. Women in the US were younger and more likely to have previous pregnancies and miscarriages and a history of sexually transmitted infections. More than 90% of women did not report symptoms of their HIV infection; however, more women from the US had symptoms (8%), compared to women from Europe (4%). Women in the US were less likely to have HIV RNA levels <400 copies/ml at delivery than women enrolling in Europe, and more likely to receive highly active antiretroviral therapy, and to start therapy earlier in pregnancy. The elective caesarean delivery rate in Europe was 61%, significantly higher than that in the US (22%). Overall, 1.48% of infants were infected and there was no significant difference in the rate of transmission between Europe and the US despite the different approaches to treatment and delivery.

CONCLUSION

These findings confirm that there are important historical differences between the HIV-infected pregnant populations in Western Europe and the USA, both in terms of the characteristics of the women and their obstetric and therapeutic management. Although highly active antiretroviral therapy predominates in pregnancy in both settings now, population differences are likely to remain.

TRIAL REGISTRATION

NCT00000869.

Sexual risk factors associated with volunteering for HIV vaccine research in South Africa

There are few data on how the risk profiles of individuals who volunteer for HIV vaccine research compare to the general population from which participants are recruited. This study contrasts demographic and sexual risk behaviours among volunteers in an HIV vaccine preparedness cohort with data from a cross-sectional community survey conducted in the same peri-urban community during the same time period. For the HIV vaccine preparedness cohort, structured questionnaires were administered to 140 HIV-negative individuals aged between 16 and 40 years. A questionnaire with identical measures was self-administered in a cross-sectional community survey of 583 randomly selected individuals within the same age range. Compared to the local community sample, individuals who volunteered for the HIV vaccine preparedness cohort were younger (adjusted odds ratio = 0.48; 95% CI = 0.28-0.82), more likely to have had a sexually transmitted disease (adjusted odds ratio = 1.75; 95% CI = 1.03-2.99) and less likely to use condoms (adjusted odds ratio = 0.51; 95% CI = 0.3-0.86). Cohort participants were also more likely to perceive themselves and their partners to be at risk of HIV infection, though these associations did not persist in multivariate analysis. These findings suggest that volunteers for vaccine-related research may be at greater risk of HIV infection than the general population from which they are recruited.

Are racial and ethnic minorities less willing to participate in health research?

BACKGROUND

It is widely claimed that racial and ethnic minorities, especially in the US, are less willing than non-minority individuals to participate in health research. Yet, there is a paucity of empirical data to substantiate this claim.

METHODS AND FINDINGS

We performed a comprehensive literature search to identify all published health research studies that report consent rates by race or ethnicity. We found 20 health research studies that reported consent rates by race or ethnicity. These 20 studies reported the enrollment decisions of over 70,000 individuals for a broad range of research, from interviews to drug treatment to surgical trials. Eighteen of the twenty studies were single-site studies conducted exclusively in the US or multi-site studies where the majority of sites (i.e., at least 2/3) were in the US. Of the remaining two studies, the Concorde study was conducted at 74 sites in the United Kingdom, Ireland, and France, while the Delta study was conducted at 152 sites in Europe and 23 sites in Australia and New Zealand. For the three interview or non-intervention studies, African-Americans had a nonsignificantly lower overall consent rate than non-Hispanic whites (82.2% versus 83.5%; odds ratio [OR] = 0.92; 95% confidence interval [CI] 0.84-1.02). For these same three studies, Hispanics had a nonsignificantly higher overall consent rate than non-Hispanic whites (86.1% versus 83.5%; OR = 1.37; 95% CI 0.94-1.98). For the ten clinical intervention studies, African-Americans' overall consent rate was nonsignificantly higher than that of non-Hispanic whites (45.3% versus 41.8%; OR = 1.06; 95% CI 0.78-1.45). For these same ten studies, Hispanics had a statistically significant higher overall consent rate than non-Hispanic whites (55.9% versus 41.8%; OR = 1.33; 95% CI 1.08-1.65). For the seven surgery trials, which report all minority groups together, minorities as a group had a nonsignificantly higher overall consent rate than non-Hispanic whites (65.8% versus 47.8%; OR = 1.26; 95% CI 0.89-1.77). Given the preponderance of US sites, the vast majority of these individuals from minority groups were African-Americans or Hispanics from the US.

CONCLUSIONS

We found very small differences in the willingness of minorities, most of whom were African-Americans and Hispanics in the US, to participate in health research compared to non-Hispanic whites. These findings, based on the research enrollment decisions of over 70,000 individuals, the vast majority from the US, suggest that racial and ethnic minorities in the US are as willing as non-Hispanic whites to participate in health research. Hence, efforts to increase minority participation in health research should focus on ensuring access to health research for all groups, rather than changing minority attitudes.

Genotypic Resistance and Immunologic Outcomes Among HIV-1-Infected Women With Viral Failure

OBJECTIVES

To describe the prevalence of specific protease inhibitor (PI) and nonnucleoside reverse transcriptase inhibitor (NNRTI) resistance mutations and the relationship between the presence of these mutations and immunologic outcomes following PI/NNRTI initiation among a cohort of HIV-1-infected women.

METHODS

Viral genotypic resistance testing was done for 366 women enrolled in the Women's Interagency HIV Study at the visit immediately prior to 1st reported use of PI or NNRTI (baseline) and at the visit approximately 1 year after PI/NNRTI initiation. We modeled the changes in CD4+ T-cell counts and HIV RNA levels approximately 1 year after therapy initiation as a function of baseline and follow-up markers, type of antiretroviral therapy used, and resistance mutations.

RESULTS

At baseline, 52% of women showed only nucleoside reverse transcriptase inhibitor (NRTI) mutations, 38% showed no mutations, and 10% showed PI or NNRTI mutations. Only 40% of women showed viral response (HIV-1 RNA < or = 80 copies/mL) 1 year after initiating a PI or NNRTI. Among those without a viral response, 54% developed PI or NNRTI mutations. NNRTI (among those with baseline NRTI mutations) and PI resistance mutations were associated with better CD4+ cell count changes (mean increase of 118 cells/mm3 and 64 cells/mm3, respectively, as compared with viral nonresponders with no PI or NNRTI mutations).

CONCLUSIONS

In this population-based cohort, virologic failure with PI or NNRTI resistance was common. Viremia with these resistance mutations was associated with preserved CD4+ T-cell count responses, providing evidence of reduced virulence or viral fitness.

Eligibility criteria for HIV clinical trials and generalizability of results: the gap between published reports and study protocols

OBJECTIVE

Applicability of randomized controlled clinical trial (RCT) results to 'real world' situations is dependent on the comparability of trial participants to general patient populations. A full disclosure of criteria employed for trial enrollment is necessary for clinicians to assess generalizability. We sought to assess both the impact on generalizability and the disclosure rate of enrollment criteria for 32 major HIV RCTs in the AIDS Clinical Trial Group (ACTG) and Community Programs for Clinical Research on AIDS (CPCRA) trial networks.

DESIGN AND METHODS

Eligibility criteria were compared in complete protocols to criteria listed in publications from these 32 NIH-funded HIV RCTs. We then applied these criteria to the Women's Interagency HIV Study (WIHS), the largest cohort study of HIV-infected women in the US.

RESULTS

When applied to WIHS, eligibility criteria from protocols excluded 0-67.6% (median 42%) of WIHS participants (50.6% excluded from ACTG trials). Eligibility criteria in publications excluded 0-62% (median 19.6%) of WIHS (21.2% excluded from ACTG trials). The number of women in WIHS seemingly ineligible for trial participation per enrollment criteria listed in publications averaged only 60% of those actually excluded based on the protocols.

CONCLUSIONS

We found that HIV RCT eligibility criteria excluded a large proportion of a representative cohort of HIV-infected women from trial participation. Furthermore, trial publications are not fully reflective of protocols in terms of disclosing eligibility criteria. Standardization and full disclosure of trial methodology will allow clinicians and researchers to more fully assess the generalizability of findings to their patient populations.

Discontinuation of Primary Prophylaxis against Mycobacterium avium Complex Infection in HIV-Infected Persons Receiving Antiretroviral Therapy: Observations from a Large National Cohort in the United States, 1992-2002

In a large, diverse cohort of human immunodeficiency virus (HIV)-infected persons receiving routine care, the proportion of eligible persons who discontinued primary prophylaxis against Mycobacterium avium complex (MAC) infection, according to guidelines of the US Public Health Service and the Infectious Diseases Society of America, increased from 16.7% (in 1996) to 84.9% (in 2002). The discontinuation of primary prophylaxis was not associated with an increased risk of disseminated MAC infection.

External validity of randomised controlled trials: "to whom do the results of this trial apply?"

In making treatment decisions, doctors and patients must take into account relevant randomised controlled trials (RCTs) and systematic reviews. Relevance depends on external validity (or generalisability)--ie, whether the results can be reasonably applied to a definable group of patients in a particular clinical setting in routine practice. There is concern among clinicians that external validity is often poor, particularly for some pharmaceutical industry trials, a perception that has led to underuse of treatments that are effective. Yet researchers, funding agencies, ethics committees, the pharmaceutical industry, medical journals, and governmental regulators alike all neglect external validity, leaving clinicians to make judgments. However, reporting of the determinants of external validity in trial publications and systematic reviews is usually inadequate. This review discusses those determinants, presents a checklist for clinicians, and makes recommendations for greater consideration of external validity in the design and reporting of RCTs.

The Problems Faced When Assessing the Prevalence and Incidence of Antiretroviral-Related Toxicities

Although the dramatic effect of highly active antiretroviral therapy (HAART) in reducing morbidity and mortality must not be underestimated, it is also important to consider the incidence and prevalence of HAART-related toxicities. Although several studies have investigated HAART-related toxicities, there has been great variety between them in the reported incidence and prevalence rates of these toxicities. Various factors, including whether the study type was a clinical trial or an observational study, the definition of the toxicity endpoints, the demographic characteristics of the study populations and the effect of calendar year on analyses, may all influence the rates observed. We investigated the possible explanations for the differences in the incidence and prevalence rates of HAART-related toxicities between studies, focussing on metabolic and hepatotoxic disorders.