Evaluation Results of the CDC/ASPH Institute for HIV Prevention Leadership

What benefits do healthcare organisations receive from leadership and management development programmes? A systematic review of the evidence

Introduction

Leadership and management training/development programmes have gained increasing institutional attention in healthcare organisations, and they have a wide variety of formats and approaches. However, limited evidence exists about effects of these programmes for the organisations that sponsor them. A minority of healthcare systems in the USA measure the impact of these programmes on organisational metrics such as staff turnover or cost savings. This systematic review sought to answer the question, ‘What evidence exists that leadership and/or management development and training programs yield benefits for health care organizations?’ These benefits could include return on investment, improved productivity/cohesion/teaming, or increased use of specific management skills (eg, strategic planning) that would directly benefit the organisation.

Methods

We followed Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines to conduct a systematic review of the relevant literature. We conducted two searches in PubMed and one in ABI/Inform, a business literature database. All articles included for the study were further categorised according to their relevance for answering the research questions, using predefined criteria based on their methodology and reported findings.

Results

Our search included 2462 studies, of which 55 met criteria for inclusion. We identified four potential organisation-level benefits to leadership and management training programmes: benefits to other staff (besides those who participate in the programmes), improved patient safety and satisfaction, tangible benefits from projects that were part of the programme and improved ability/confidence using leadership-related skills by programme participants. However, the research base on this topic is limited.

Conclusion

Although this research identified potential benefits of leadership and management programmes at the organisation level, additional research is needed to make definitive conclusions about their impact.

"Don't Deny Yourself a Seat at the Table": Supporting the Leadership Development of MSM of Color in HIV Services

AIDS service organizations have played a vital role in responding to the HIV epidemic, yet many are plagued by an underrepresentation of racial, ethnic, and sexual minorities in leadership positions. In response, the Wisconsin Department of Health Services, HIV Division, created the Wisconsin Health Leaders Fellowship program to provide training and skills to gay and bisexual men of color in HIV service organizations in Milwaukee, Wisconsin. The program was evaluated using in-depth pre- and post-Fellowship interviews with fellows, their agency preceptors, and Fellowship staff (n = 23 interviews). Results indicate that the Fellowship was successful at developing leadership skills and increasing the confidence of the fellows. However, findings revealed the need to address larger social and structural issues alongside supporting individuals in order to create effective, sustainable change. There is a need for cultural diversity trainings for organizational leaders, changes to organizational hiring and promotion practices, and improving opportunities for formal education for racial and ethnic minority men.

A case study on building capacity to improve clinical mentoring and maternal child health in rural Tanzania: the path to implementation

Background

Tanzania is a low income, East African country with a severe shortage of human resources for health or health workers. This shortage threatens any gains the country is making in improving maternal health outcomes. This paper describes a partnership between Touch Foundation and NYU Rory Meyers College of Nursing - Global, aimed at improving clinical mentorship and capacity among nurses and midwives at two rural hospitals in the Tanzanian Lake Zone Region. Clinical mentoring capacity building and supportive supervision of staff has been shown to be a facilitator of retaining nurses and would be possible to acquire and implement quickly, even in a context of low resources and limited technology.

Methods

A case study approach structures this program implementation analysis. The NYU Meyers team conducted a 6-day needs assessment at the two selected hospitals. A SWOT analysis was performed to identify needs and potential areas for improvement. After the assessment, a weeklong training, tailored to each hospitals’ specific needs, was designed and facilitated by two NYU Meyers nursing and midwifery education specialists. The program was created to build on the clinical skills of expert nurse and midwife clinicians and suggested strategies for incorporating mentoring and preceptorship as a means to enhance clinical safety and promote professional communication, problem solving and crisis management.

Results

Nineteen participants from both hospitals attended the training. Fourteen of 19 participants completed a post training, open ended questionnaire for a 74% response rate. Fifty-seven percent of participants were able to demonstrate and provide examples of the concepts of mentorship and supervision 4 and 11 months’ post training. Participants indicated that while confidence in skills was not lacking, barriers to quality care lay mostly in understaffing. Implementation also offered multiple insights into contextual factors affecting sustainable program implementation.

Conclusions

Three recommendations from this training include: 1) A pre-program assessment should be conducted to ascertain contextual relevance to curriculum development; 2) flexibility and creativity in teaching methods are essential to engage students; and 3) access to participants a priori to program implementation may facilitate a more tailored approach and lead to greater participant engagement.

Perinatal HIV testing among African American, Caucasian, Hmong and Latina women: exploring the role of health-care services, information sources and perceptions of HIV/AIDS

Perinatal HIV transmission disproportionately affects African American, Latina and potentially Hmong women in the United States. Understanding racially and ethnically diverse women's perceptions of and experiences with perinatal health care, HIV testing and HIV/AIDS may inform effective health communications to reduce the risk of perinatal HIV transmission among disproportionate risk groups. We used a qualitative descriptive research design with content analysis of five focus groups of African American, Caucasian, Hmong and Latina women of reproductive age with low socioeconomic status distinguished by their race/ethnicity or HIV status. A purposive stratified sample of 37 women shared their health-care experiences, health information sources and perceptions of HIV testing and HIV/AIDS. Women's responses highlighted the importance of developing and leveraging trusted provider and community-based relationships and assessing a woman's beliefs and values in her sociocultural context, to ensure clear, consistent and relevant communications. Perinatal health communications that are culturally sensitive and based on an assessment of women's knowledge and understanding of perinatal health and HIV/AIDS may be an effective tool for health educators addressing racial and ethnic disparities in perinatal HIV transmission.

Developing leaders in priority populations to address tobacco disparities: results from a leadership institute

CONTEXT

Few published evaluations documenting the results of community health leadership programs exist. Furthermore, few leadership programs specifically address cross-cultural issues or priority populations, and none have focused on the area of tobacco control.

OBJECTIVE

The goal of the evaluation was to determine the effectiveness of the Leadership and Advocacy Institute to Advance Minnesota's Parity for Priority Populations Institute. Institute Fellows were recruited from 5 priority populations, including African/African American, American Indian, Asian American, Chicano Latino and Lesbian, Gay, Bisexual, and Transgender. The Institute's impact on the acquisition and application of particular leadership skills was assessed, along with its impact in building priority population capacity to engage in tobacco control activities. DESIGN AND MEASURE: Findings are based on mixed methods, including Fellows' completion of paper surveys prior, during, and post-Institute, and telephone interviews conducted post-Institute.

RESULTS

Perceived skills increased in all areas assessed. Fellows reported applying specific skills acquired 16 months after the Institute. Furthermore, a high number of Fellows were more intensely involved in tobacco control work compared to baseline and reported involvement in cross-cultural collaborations or initiatives post-Institute.

CONCLUSION

The Institute achieved both short-term and intermediate outcomes; skills were acquired during the Institute and sustained and applied in areas of tobacco control over 1 year later.

Evaluating leadership training in African American HIV prevention organizations

This article aimed to examine changes in the HIV prevention capacity of HIV prevention program managers who completed the Institute for HIV Prevention Leadership ("Institute") between 2002 and 2004, and who worked in community-based organizations that primarily served African Americans. Participants completed a survey at three points in time, in which they rated the frequency with which they conducted activities related to HIV prevention practice. Participants also rated their confidence in performing activities. Repeated measures ANOVA was conducted to detect differences at three time points (baseline, immediate posttest, and 6 months posttest). A significant overall positive trend was found in the frequency and confidence of participants to perform specific HIV prevention practices and an overall positive trend in the frequency of processes that support HIV prevention practice. Investment in long-term, intensive, capacity-building programs like the Institute is critical to address the increasing incidence of HIV in many African American communities.

Assessing levels of adaptation during implementation of evidence-based interventions: introducing the Rogers-Rütten framework

Most HIV-prevention funding agencies require the use of evidence-based behavioral interventions, tested and proven to be effective through outcome evaluation. Adaptation of programs during implementation is common and may be influenced by many factors, including agency mission, time constraints, and funding streams. There are few theoretical frameworks to understand how these organizational and program-related factors influence the level of adaptation. This study used constructs from both Rogers's diffusion theory and Rütten's framework for policy analysis to create a conceptual framework that identifies determinants hypothesized to affect the level of adaptation. Preliminary measures of these constructs were also developed. This framework and its measures assess organizational and program-related factors associated with adaptation and could serve as a model to assess implementation and adaptation in fields outside of HIV prevention.

Strategies used by community-based organizations to evaluate their locally developed HIV prevention interventions: Lessons learned from the CDC's innovative interventions project

Community-based organizations (CBOs) play an important role in health promotion efforts and the delivery of HIV prevention interventions for at-risk minority populations. CBOs may also develop their own interventions but often lack the capacity or funds to rigorously evaluate them. The Innovative Interventions project of the Centers for Disease Control and Prevention (CDC) funded three CBOs to rigorously evaluate the efficacy of interventions they had developed and were delivering to Black women, Black men who have sex with men (MSM), and adolescent males in juvenile justice settings, respectively. The evaluation results have been reported elsewhere. This article describes operational issues that the CBOs identified as being particularly salient to their evaluations and the strategies they developed to address the issues and successfully complete their evaluations. These issues included the development of organizational capacity to conduct a rigorous outcome evaluation, difficulties with recruitment and retention of evaluation participants, and the use of process monitoring data to improve intervention delivery. The strategies described in this article can be used by CBOs when evaluating their locally developed HIV prevention interventions and may be of interest to funding agencies and researchers that collaborate with CBOs to evaluate their interventions.

A review of the Centers for Disease Control and Prevention's response to the HIV/AIDS crisis among Blacks in the United States, 1981-2009

Among US racial/ethnic groups, Blacks are at the highest risk of acquiring HIV/AIDS. In response, the Centers for Disease Control and Prevention (CDC) has launched the Heightened National Response to Address the HIV/AIDS Crisis Among African Americans, which seeks to engage public and nonpublic partners in a synergistic effort to prevent HIV among Blacks. The CDC also recently launched Act Against AIDS, a campaign to refocus attention on the domestic HIV/AIDS crisis. Although the CDC's efforts to combat HIV/AIDS among Blacks have achieved some success, more must be done to address this crisis. New initiatives include President Obama's goal of developing a National HIV/AIDS Strategy to reduce HIV incidence, decrease HIV-related health disparities, and increase access to care, especially among Blacks and other disproportionately affected populations.

Enhancing community-based organizations' capacity for HIV/AIDS education and prevention

The catalytic potential of community-based organizations to promote health, prevent disease, and address racial, ethnic, and socio-economic disparities in local communities is well recognized. However, many CBOs, particularly, small- to medium-size organizations, lack the capacity to plan, implement, and evaluate their successes. Moreover, little assistance has been provided to enhance their capacity and the effectiveness of technical assistance to enhance capacity is likewise limited. A unique private-academic partnership is described that simultaneously conducted program evaluation and addressed the capacity needs of 24 CBOs funded by the Pfizer Foundation Southern HIV/AIDS Prevention Initiative. Assessments of key program staff members at 12 and 18 months after the initial cross-site program assessment survey indicated a significant improvement in the CBOs' knowledge, skills, and abilities and a substantial reduction in their technical assistance needs for HIV/AIDS prevention. Full participation of CBOs in technical assistance and a concurrent empowerment evaluation framework were necessary to enhance prevention capacity.

Cost savings threshold analysis of a capacity-building program for HIV prevention organizations

Although the incidence of HIV each year remains steady, prevention funding is increasingly competitive. Programs need to justify costs in terms of evaluation outcomes, including economic ones. Threshold analyses set performance standards to determine program effectiveness relative to that threshold. This method was used to evaluate the potential cost savings of a national capacity-building program for HIV prevention organizations. Program costs were compared with the lifetime treatment costs of HIV, yielding an estimate of the HIV infections that would have to be prevented for the program to be cost saving. The 136 persons who completed the capacity-building program between 2000 and 2003 would have to avert 41 cases of HIV for the program to be considered cost saving. These figures represent less than one tenth of 1% of the 40,000 new HIV infections that occur in the United States annually and suggest a reasonable performance standard. These data underscore the resources needed to prevent HIV.

Journey to the Motherland: Assessing Capacity for the Prevention of HIV Mother-to-Child Transmission in South Africa

Strategies to prevent mother-to-child transmission of HIV have successfully affected pediatric HIV infections in many developed countries, but not as dramatically in developing countries. This article presents the findings and perspectives of a team of public health nurses who visited and assessed capacity of selected antenatal clinics across South Africa to successfully implement and evaluate HIV prevention of maternal-to-child transmission programs conducted for a private nongovernmental organization. Through development and implementation of a needs-assessment tool, the clinics were systematically assessed by observation and interviews. Consistently, three additional areas of concern, other than HIV, were reported by the people interviewed. These concerns were hunger, poverty, and the need for home-based care.