Evaluating the Impact of the Management Academy for Public Health

Developing State Leadership in Maternal and Child Health: Process Evaluation Findings from a Work-Based Learning Model for Leadership Development

Objectives

Since 2013 the MCH Bureau has supported the National MCH Workforce Development Center to strengthen the Title V MCH workforce. This article describes the Center’s Cohort Program and lessons learned about work-based learning, instruction, and coaching.

Description

The Cohort Program is a leadership development program that enrolls state-level teams for skill development and work-based learning to address a self-identified challenge in their state. Teams attend a Learning Institute that teaches concepts, skills, and practical tools in systems integration; change management and adaptive leadership; and evidence-based decision-making and implementation. Teams then work back home on their challenges, aided by coaching. The Program’s goals are for teams to expand and use their skills to address their challenge, and that teams would strengthen programs, organizations, and policies, use their skills to address other challenges, and ultimately improve MCH outcomes.

Methods

This process evaluation is based on evaluation forms completed by attendees at the three-day Learning Institute; six-month follow-up interviews with team leaders; and a modified focus group with staff.

Results

Participants and staff believe the Cohort Program effectively merges a practical skill-based curriculum, work-based learning in teams, and coaching. The Learning Institute provides a foundation of skills and tools, strengthens the team’s relationship with their coach, and builds the team. The work-based learning period provides structure, accountability, and a “practice space” for teams to apply the Cohort Program’s skills and tools to address their challenge. In this period, teams deepen collaborations and often add partners. The coach provides accessible and tailored guidance in teamwork and skill application. These dimensions helped teams in develop skills and address state-level MCH challenges.

Conclusions for Practice

Continuing professional development programs can help leaders learn to address complex state-level MCH challenges through integrated classroom-based skills development, work-based learning on state challenges, and tailored coaching.

A Qualitative Analysis of Maternal and Child Health Public Health Leadership Institute (MCH PHLI) Leaders: Assessing the Application of Leadership Skills at the "Others" and "Wider Community" Levels of the MCH Leadership Competencies 4.0

Objectives

To gain insight into how participants in the Maternal and Child Health Public Health Leadership Institute (MCH PHLI) report applying the leadership skills gained through the program at the “Others” and “Wider Community” levels of the MCH Leadership Competencies 4.0.

Methods

111 mid- to senior-level MCH leaders participating in the MCH PHLI gave < 5 min oral presentations detailing the impacts resulting from implementation of the skills gained through the leadership development training. Presentations were recorded and transcribed then qualitatively analyzed in reference to the MCH Leadership Competencies 4.0. Impacts were stratified by the “Others” and “Wider Community” levels.

Results

Analysis resulted in 1510 separate coded examples, 948 of which were coded as aligning with the MCH Leadership Competency 4.0 areas of “Others”, “Wider Community” and with an additional emerging competency. In many examples Participants estimated the numbers of people affected by these leadership activities, which totaled more than 80,773 people across the US.

Conclusions for Practice

This analysis suggests that mid-to-senior level intensive leadership development strategies benefit organizations, communities, and systems quite broadly through a virtual “ripple effect” of training. Capturing qualitative data can help elucidate the return on investment for leader development programs in terms of impacts on communities and systems.

A Qualitative Analysis of Individual Leadership Behaviors Among Participants in the Maternal and Child Health Public Health Leadership Institute

OBJECTIVES

To gain insights into how participants in the Maternal and Child Health Public Health Leadership Institute (MCH PHLI) report applying the personal leadership skills gained through the program at the "Self" level of the MCH Leadership Competencies 4.0.

METHODS

112 mid- to senior-level MCH leaders completed the MCH PHLI, which is a year-long intensive leadership training program. At graduation, 111 participants gave < 5-min oral presentations detailing the actions taken and impact created by implementation of the skills gained through the MCH PHLI training. Presentations were recorded, transcribed and then qualitatively analyzed in reference to the "Self" level of the MCH Leadership Competencies 4.0.

RESULTS

Participants reported 562 coded examples of activities in which they implemented skills aligning with each competency in the Self-Level of the MCH Leadership Competencies 4.0 and with three other competency areas that emerged as themes from the data: networking, confidence, and career advancement.

CONCLUSIONS FOR PRACTICE

This analysis suggests that intensive leadership development strategies focused on the mid-to-senior level leader benefit the individual and their organizations in broad and strategic ways that can be captured and described. Further, these applications of skills can create a virtual "ripple effect" of the workforce development program, by ultimately impacting a far greater number of people. Finally, this type of reflective assignment can be a valuable addition to intensive workforce development programs.

Entrepreneurship and Innovation in Health Sciences Education: a Scoping Review

BACKGROUND AND PURPOSE

This scoping review aimed to explore the connection between health education and entrepreneurship and to identify gaps in the current literature, educational models, and best practices regarding teaching medical professionals about entrepreneurship and innovation.

METHODS

The methodology for this review was based on the principles of Arksey and O'Malley's (2005) model for scoping review design. Results from Embase, MEDLINE, PsycINFO, Emcare, AMED, PubMed, and Google Scholar were scanned, filtered, and mapped.

RESULTS

Fifty-nine unique papers were found and mapped. The papers discussed common themes, including the entrepreneurial environment (n = 29), career planning and skill development (n = 3), and various skills crucial for the health entrepreneur. The satisfaction was high for most programs, but few reported more fulsome outcomes. The teaching techniques used to engage trainees or physicians in entrepreneurship were also fairly limited.

CONCLUSION

Though some programs are described, few have demonstrated efficacy. More attention should be paid towards faculty-level recruitment, development and reward, so that they may in turn teach these approaches. Those involved with educational planning can help close this gap.

Strengthening Public Health Management Capacity in Vietnam: Preparing Local Public Health Workers for New Roles in a Decentralized Health System

Health sector decentralization has created an urgent need to strengthen public health management capacity in many countries throughout the developing world. This article describes the establishment of a national management training network in Vietnam that used Project-Based Learning to strengthen management competencies of HIV program workers and linked training to measurable improvement in HIV/AIDS public health program outcomes. Skills were taught using a combination of classroom learning and mentored fieldwork. From 2005 to 2015, 827 HIV/AIDS program managers were trained with this method throughout Vietnam by trainers in 3 regional training centers. A total of 218 applied learning projects were carried out by trainees during this period; 132 resulted in measurable improvements in HIV/AIDS program outputs, and 86 produced well-organized plans for implementing, monitoring, and evaluating HIV/AIDS intervention strategies. Vietnam's management training network represents an important advancement in public health workforce development that helps prepare workers for new roles and responsibilities in a decentralized health system.

The DELTA PREP Initiative: Accelerating Coalition Capacity for Intimate Partner Violence Prevention

BACKGROUND

The DELTA PREP Project aimed to build the prevention capacity of 19 state domestic violence coalitions by offering eight supports designed to promote prevention integration over a 3-year period: modest grant awards, training events, technical assistance, action planning, coaching hubs, the Coalition Prevention Capacity Assessment, an online workstation, and the online documentation support system.

OBJECTIVES

Using quantitative and qualitative data, we sought to explain how coalitions integrated prevention within their structures and functions and document how DELTA PREP supports contributed to coalitions' integration process.

RESULTS

We found that coalitions followed a common pathway to integrate prevention. First, coalitions exhibited precursors of organizational readiness, especially having prevention champions. Second, coalitions engaged in five critical actions: engaging in dialogue, learning about prevention, forming teams, soliciting input from the coalition, and action planning. Last, by engaging in these critical actions, coalitions enhanced two key organizational readiness factors-developing a common understanding of prevention and an organizational commitment to prevention. We also found that DELTA PREP supports contributed to coalitions' abilities to integrate prevention by supporting learning about prevention, fostering a prevention team, and engaging in action planning by leveraging existing opportunities. Two DELTA PREP supports-coaching hubs and the workstation-did not work as initially intended. From the DELTA PREP experience, we offer several lessons to consider when designing future prevention capacity-building initiatives.

Evaluation of DELTA PREP: A Project Aimed at Integrating Primary Prevention of Intimate Partner Violence Within State Domestic Violence Coalitions

BACKGROUND

Intimate partner violence (IPV) has been recognized as a public health problem since the late 20th century. To spur IPV prevention efforts nationwide, the DELTA PREP Project selected 19 state domestic violence coalitions to build organizational prevention capacity and catalyze IPV primary prevention strategies within their states.

OBJECTIVE

DELTA PREP's summative evaluation addressed four major questions: (1) Did coalitions improve their prevention capacity during the project period? (2) Did coalitions serve as catalysts for prevention activities within their states during the project period? (3) Was initial prevention capacity associated with the number of prevention activity types initiated by coalitions by the end of the project? (4) Did coalitions sustain their prevention activities 6 months after the end of the project period?

RESULTS

DELTA PREP achieved its capacity-building goal, with all 19 participant coalitions integrating prevention within their organizations and serving as catalysts for prevention activities in their states. At 6 months follow up, coalitions had sustained almost all prevention activities they initiated during the project. Baseline prevention capacity (Beginner vs. Intermediate) was not associated with the number of prevention activity types coalitions implemented by the end of the project.

CONCLUSION

Service and treatment organizations are increasingly asked to integrate a full spectrum of prevention strategies. Selecting organizations that have high levels of general capacity and readiness for an innovation like integrating a public health approach to IPV prevention will likely increase success in building an innovation-specific capacity, and in turn implementing an innovation.

Effectiveness of public health quality improvement training approaches: application, application, application

CONTEXT

Quality improvement (QI) has been identified as a key strategy to improve the performance of state and local public health agencies. Quality improvement training effectiveness has received little attention in the literature.

OBJECTIVES

To evaluate the effectiveness of 3 QI training types: webinars, workshops, and demonstration site activities on improving participant knowledge, skill, and ability to conduct QI through a questionnaire conducted after training participation.

DESIGN

We used a natural experimental design hypothesizing that demonstration site participants would have the greatest gains on outcomes of interest compared with webinar and workshop participants. Bivariate and multivariate models were used to examine outcome differences between questionnaire respondents who participated in various training types.

PARTICIPANTS

Local health department employees who participated in the 3 training strategies.

MAIN OUTCOME MEASURES

Measures included knowledge and skill gain, skill application, QI receptivity, and ability to successfully participate in a QI project.

RESULTS

Two hundred eighty-four unique individuals who work in 143 health departments completed the questionnaire for a 59% response rate. The majority of these health departments serve midsize populations. Demonstration site respondents had significantly greater gains in knowledge and skills, skill application, and ability to successfully participate in a QI project. Webcast training participants had significantly higher QI receptivity in adjusted models. Respondents who participated in both webcast and demonstration site trainings had higher mean scores on all outcomes when compared with demonstration site single training participants, these differences were significant in unadjusted models.

CONCLUSION

Our findings suggest that QI training for public health agency employees should include both didactic training on QI content and opportunities for QI application. Future research should examine if this approach can effectively increase successful participation in QI projects for staff in LHDs of all sizes.

Evaluation findings from the Institute for Public Health and Faith Collaborations

OBJECTIVE

The Institute for Public Health and Faith Collaborations sought to cultivate boundary leadership to strengthen collaboration across religious and health sectors to address health disparities. This article presents findings from an evaluation of the Institute and its impact on participating teams of faith and public health leaders.

METHODS

. Self-administered surveys were completed by participating team members (n = 243) immediately post-Institute. Semistructured telephone interviews were conducted with at least one health and one faith leader per team six to eight months after the Institute.

RESULTS

Significant self-reported improvement occurred for all short-term outcomes assessed, with the largest increases in describing organizational frames and why they are important for community change, and understanding the role of boundary leaders in community systems change. Six months after the Institute, participants spoke of inspiration, team building, and understanding their own leadership strengths as important outcomes. Leadership growth centered on functioning in groups, making a change in their work, a renewed faith in self, and a renewed focus on applying themselves to faith/health work. Top team accomplishments included planning or implementing a program or event, or solidifying or sustaining a collaborative structure. The majority felt they were moving in the right direction to reduce health disparities, but had not yet made an impact.

CONCLUSIONS

Results suggest the Institute played a role in helping to align faith and health assets in many of the participating teams.

Evaluation as a Critical Factor of Success in Local Public Health Accreditation Programs

This article presents the variety of approaches used to conduct evaluations of performance improvement or accreditation systems, while illustrating the complexity of conducting evaluations to inform local public health practice. We, in addition, hope to inform the Exploring Accreditation Program about relevant experiences involving accreditation and performance assessment processes, specifically evaluation, as it debates and discusses a national voluntary model. A background of each state is given. To further explore these issues, interviews were conducted with each state's evaluator to gain more in-depth information on the many different evaluation strategies and approaches used. On the basis of the interviews, the authors provide several overall themes, which suggest that evaluation is a critical tool and success factor for performance assessment or accreditation programs.

Management academy for public health: creating entrepreneurial managers

The Management Academy for Public Health develops public health managers' management skills. Ultimately, the program aims to develop civic entrepreneurs who can improve the efficiency and the effectiveness of their organizations. With help from a coach, teams write public health business plans to meet needs in their communities. An external evaluation found that 119 teams trained during the first 3 years of the program generated more than $6 million in enhanced revenue-including grants, contracts, and fees through their business plans--from $2 million in program funding. Approximately 38% of the teams expected to generate revenue from an academy business plan or a spin-off plan. Action-learning methods can help midcareer managers transfer their training to the workplace and build entrepreneurial skills.

The Management Academy for Public Health: program design and critical success factors

The Management Academy for Public Health is a team-based training program jointly offered by the School of Public Health and the Kenan-Flagler Business School at the University of North Carolina at Chapel Hill. This 9-month program teaches public health managers how to better manage people, information, and finances. Participants learn how to work in teams with community partners, and how to think and behave as social entrepreneurs. To practice and blend their new skills, teams develop a business plan that addresses a local public health issue. This article describes the program and explains the findings of the process evaluation, which has examined how best to structure and deploy a team-based method to create more effective, more entrepreneurial public health managers. Findings indicate that recruitment and retention are strong, program elements are relevant to learners' needs, and learners are satisfied with and value the program. Several specific benefits of the program model are identified, as well as several elements that support business plan success and skills' application on the job. On the basis of these findings, four success factors critical for developing similar programs are identified.

Creating the Management Academy for Public Health

True collaboration among large federal agencies is rare, as is that among large and influential national foundations. The collaboration between two major government health agencies (The Centers for Disease Control and Prevention and the Health Resources and Services Administration) and three major health foundations (the W.K. Kellogg Foundation, the Robert Wood Johnson Foundation, and the CDC Foundation) to create the Management Academy for Public Health is unprecedented in public health over the past quarter century. We attribute this success to the unique combination of a strong foundation of relationships between the partners and a commitment to generative dialogue throughout the design and implementation of the program. The success and sustainability of the Academy derive directly from these critical success factors, serving as an exemplary model for future collaborative endeavors.

The UNC Management Academy for Public Health

The University of North Carolina Management Academy for Public Health is a unique training program that combines a business education with a public health focus, to enhance the performance of individual public health managers and improve organizational strength throughout governmental public health. This article considers the implications of decisions made in creating this program, which, after initial funding through the Centers for Disease Control and Prevention Foundation, is now self-sustaining through participant fees. It details the principles behind the program's design, curriculum, evaluation, and sustainability strategies; presents results of the ongoing partnership; and draws conclusions about the program's future ability to meet a national need for public health management training.