An epidemic of orthodoxy? Design and methodology in the evaluation of the effectiveness of HIV health promotion

The difference practice makes: Evidence, articulation, and affect in HIV prevention

This paper considers the difference that a conception of sex as social practice has made to the relations articulated in HIV social research in Australia. In defining sexual practice as "fluid, embedded in specific social formations, and involving the negotiation of meaning" (Kippax & Stephenson, 2005), social researchers put their own research categories and questions at risk by constructing situations in which the objects of research were given occasions to counter researchers' presumptions through the use of their own categories. Taking this risk produced sharp insights about the evolving dynamics of gay sexuality in this context and produced distinctive, interesting findings. It enabled the articulation of the practice of negotiated safety and later strategies of HIV risk reduction emerging from gay men's practice, for example. I draw on Latour's (2004) concept of articulation to make sense of these innovations and cut through some of the key distinctions that organize HIV research: qualitative/quantitative; social/biomedical; subject/ object; human/nonhuman; interpretations/evidence. Rather than rest on the organizing power of these distinctions, keeping HIV prevention effective, engaging and interesting will require specific attention to the embodied articulation of HIV relations in the present.

Evaluating HIV prevention effectiveness: the perfect as the enemy of the good

There is a need to better understand the effectiveness of HIV-prevention programs. Cluster randomized designs have major limitations to evaluate such complex large-scale combination programs. To close the prevention evaluation gap, alternative evaluation designs are needed, but also better articulation of the program impact pathways and proper documentation of program implementation. Building a plausible case using mixed methods and modeling can provide a valid alternative to probability evidence. HIV prevention policies should not be limited to evidences from randomized designs only.

HIV prevention in young people in sub-Saharan Africa: a systematic review

PURPOSE

To systematically review and update evidence on the effectiveness of youth HIV/AIDS prevention interventions in sub-Saharan Africa (SSA) and to make evidence-based policy recommendations to guide efforts toward meeting the United Nations General Assembly Special Session HIV/AIDS goals.

METHODS

Recent evidence (January 2005-December 2008) on the effectiveness of interventions to improve reported sexual behavior and biological outcomes among youth in SSA was assessed using the Steady, Ready, Go! (SRG) approach and synthesized with an earlier SRG review (January 1990-June 2005).

RESULTS

A total of 23 studies were included following screening of approximately 1,200 citations. School-based, adult-led, curriculum-based interventions showed clear evidence of reducing reported risky sexual behavior. Interventions in health facilities increased the use of services when made accessible and more youth-friendly. In geographically defined communities, both interventions specifically targeting youth and community-wide interventions reduced reports of risky sexual behavior.

CONCLUSIONS

HIV prevention among youth is a top priority in SSA. The most promising interventions should be scaled-up now, with careful evaluation, while exploring supplementary interventions to impact HIV incidence.

Effects and effectiveness of life skills education for HIV prevention in young people

For 20 years, "life skills" education has been advocated as a key component of HIV and AIDS education for young people. But what do terms such as life skills imply, and what evidence is there that a life skills-based approach really works? This article reviews the literature on the effects and effectiveness of life skills-based education for HIV prevention. Evaluated interventions were identified by using three search strategies. The review identified a surprising number of rigorously designed and evaluated interventions from Africa, Latin America, Asia, and the Pacific. Most interventions used life skills training as a component of the overall education strategy. Programs worked best to positively influence knowledge, attitudes, intentions, skills, and abilities. Programs rarely produced consistent effects on sexual behavior. Also, life skills, training had little effect on biological outcomes. The narrow focus on achieving behavioral outcomes may be at the loss of documenting other positive impacts.

Gay Asian men in Sydney resist international trend: no change in rates of unprotected anal intercourse, 1999-2002

Against a background of increasing unprotected anal intercourse (UAI) in the Sydney gay community (and internationally), complementary cross-sectional surveys of sexual practice were conducted among gay men of Asian background in 1999 (N = 319) and 2002 (N = 457). Self-complete questionnaires were used with recruitment at gay bars, gay social functions, and gay sex-on-premises venues. In 2002, self-report HIV status was 73.7% HIV-negative, 3.6% HIV-positive, and 22.8% unknown status (no significant change from 1999). Over time, the proportion of gay Asian men who reported any UAI with regular partners (in the previous 6 months) did not change significantly: 27.9% in 1999; 24.3% in 2002. Similarly, rates of any UAI with casual partners remained steady: 16.3% in 1999; 14.4% in 2002. Only one factor, more extensive engagement in esoteric practices (fisting, sadomasochism, group sex, rimming), was independently associated with sexual risk practice. This suggests that risk in this population of gay men, as in others, has more to do with the sexual cultures in which men are embedded rather than individual-level differences.

If HIV prevention works, why are rates of high-risk sexual behavior increasing among MSM?

Systematic reviews of HIV prevention research provide clear evidence that behavioral interventions can influence the sexual behavior of men who have sex with men (MSM). However, if HIV prevention works, why are rates of high-risk sexual behavior increasing among MSM in major European, Australian, Canadian, and U.S. cities? The evidence generated by systematic reviews alone may not provide a clear answer to this question. This is because (a) it is uncertain whether experimental interventions shown to be effective in one setting, place, or moment in time can be repeated successfully in another; (b) we have limited understanding of the processes that underlie the interventions; (c) interventions shown to work in an experimental study may not necessarily be effective in everyday life. To answer the question, we need to be alert to the changing risk environment in which men have sex with other men. We also need to develop a new program of research addressing the transferability, sustainability, and effectiveness of sexual health promotion among MSM. Randomized controlled trials will remain one of the optimal means of evaluating behavioral interventions in such a program. By further strengthening the evidence base, we may identify opportunities for innovative as well as effective HIV prevention initiatives.

Methodological strategies for the identification and synthesis of 'evidence' to support decision-making in relation to complex healthcare systems and practices

This paper addresses the limitations of current methods supporting 'evidence-based health-care' in relation to complex aspects of care, including those questions that are best supported by descriptive or non-empirical evidence. The paper identifies some new methods, which may be useful in aiding the synthesis of data in these areas. The methods detailed are broadly divided into those that facilitate the identification of evidence and those that enable the interpretation of the data retrieved. To illustrate some of the issues involved, reference is made to a multimethod review recently completed by the authors, which aimed to identify factors that promote continuity in the transition from child to adult health and social care. It is argued that as healthcare organisations are becoming increasingly preoccupied with the evidence base of practice, such methods may help ensure that aspects of care and approaches that are outside the dominant pharmaco-medical domain maintain a prominent position on the healthcare agenda while remaining open to external scrutiny. Healthcare professionals who use such approaches need to know their relative utility and benefits to inform clinical decisions, so as to ensure that best practice is observed.

Exploring the community response to a randomized controlled HIV/AIDS intervention trial in rural Uganda

Investigators need, both for ethical and methodological reasons, to consider the acceptability of their intervention to the study population. We explored the response to a community-based randomized controlled trial (RCT) of an HIV/AIDS behavioral change intervention in rural Uganda. The views of field-workers, trial community, nongovernment organization representatives, and religious leaders were explored via focus groups (13) and interviews (45). The results suggest that the components of the intervention valued by the community are not necessarily those prioritised by trial implementers. Specifically, prevention activities appear to be valued less than material assistance. Furthermore, universal acceptance of the trial is probably unattainable. For these reasons, sensitive mobilization, respect for community members and their appointed leaders, and ongoing communication is essential. We suggest that evaluations of process be regarded as essential to the conduct of community-based RCTs and highlight the need for appropriate evaluation indicators to facilitate this.

Evaluation of interventions for gay men: two desiderata

This paper was presented in a session at the AIDS Impact conference devoted to a debate on the methods that should be used to evaluate educational interventions. The paper highlights two desiderata for evaluation of interventions directed at gay men. First, the view is presented that there is no acceptable substitute for assessing the effect of an intervention on gay men's sexual behaviour (rather than, for example, their AIDS-related attitudes or beliefs). This view is justified in terms of (a) the differences that exist between AIDS-related thinking in the cold light of day and during actual sexual encounters; and (b) the often faulty nature of intuitions about the factors that contribute to sexual risk-taking and the ways in which it might be reduced. Second, it is argued that the randomized control study design represents the best means for ensuring that interventions will be as effective as possible. Criticisms which have been made of this design are discussed and the conclusion drawn that they do not amount to a strong case against it.

Balancing rigour and acceptability: the use of HIV incidence to evaluate a community-based randomised trial in rural Uganda

Recent debate about the evaluation of community based, HIV/AIDS behavioural interventions has focused on the appropriateness of the randomised controlled trial (RCT) design, and the difficulty of obtaining reliable outcome measures. A community based HIV AIDS behavioural change RCT, recently conducted in rural Uganda, used HIV incidence as the principal outcome measure. This paper examines the acceptability of the trial from the community perspective. It asks whether, in a rural African setting, it is possible to implement a scientifically rigorous evaluation without compromising acceptability of the trial to the community. Opinions of the trial held by community members working as trial field workers were collected by semi-structured interview (n = 37), and focus group discussions (4) Community opinions of the trial were ascertained through 10 focus groups. For both field workers and the community, the sero-survey was more salient than the intervention, and the source of many rumours and disputes. Despite intensive mobilisation and close monitoring of field workers, it was impossible to ensure the veracity of explanations about the survey at ground level, and to protect each individual from coercion. The community expected a reward in return on their blood. Although the introduction of incentives at the final survey round increased the acceptability of the trial, they not only created jealousies and tensions, but also led to expectations of greater rewards in future. We conclude that RCTs in poor, rural communities are feasible, but the challenges involved should not be underestimated. Obtaining community support for the trial, respecting established hierarchies, and close supervision of field workers are all essential, but even then, controversies should be anticipated. There is an urgent need for relevant guidelines to help researchers navigate the complex ethical issues involved.

Is the randomized clinical trial the gold standard of research?

All research has flaws. Some flaws are so trivial that the research can still stand as the definitive study. Other flaws prevent a study from being definitive, but the study still provides useful guidance in the context of other research. Some flaws are so serious that the research provides no useful information at all. The tricky part is not finding flaws in the research but in deciding to what extent the flaws erode the credibility of the research. In general, the use of RCTs can add substantial credibility to a research study. There are calls for greater use of RCTs in many areas, such as surgery (Baum, 1999) and psychiatry (Andrews, 1999). Of course, nonrandomized trials are an important complement to RCTs when the latter are ethically inappropriate or logistically impossible (Black, 1996). Failure to use randomization or blinding, however, is not a fatal flaw. Furthermore, the artificial nature of RCTs will often restrict their applicability to overly simple interventions. When RCTs focus on narrow patient groups or exclude important segments of the population, there may be difficulty in generalizing their results. So it would be a mistake to label the RCT as a gold standard for all research. A silver standard may be a more appropriate label.

Behavioural interventions to prevent HIV infection: rapid evolution, increasing rigour, moderate success

Behavioural interventions aim to alter behaviours that make individuals more vulnerable to becoming infected or infecting others with HIV. Research in this field has developed rapidly in recent years. Increased rigour in the design and conduct of evaluations and moderate successes in bringing about behaviour change in target populations are the key achievements so far. This paper reflects on these developments, addresses recent innovations and highlights likely areas for future work. Discussion focuses on maximising the potential effectiveness of new interventions, methodological issues relating to evaluation and implementation of interventions into practice. The paper concludes there is evidence that interventions deemed effective under evaluation conditions can be implemented in HIV prevention services and that this is the next major challenge. The immediate goal should be consolidation of the learning that has occurred, particularly efforts to maintain theoretical and evaluative rigour whilst encouraging increased collaborative partnerships between researchers, service providers and affected communities.