An outcomes evaluation of a leadership development initiative

Comparing Virtual vs In-Person Immersive Leadership Training for Physicians

Purpose

The COVID-19 pandemic caused a disruption of in-person workforce development programs. Our immersive physician-oriented leadership institute suspended in 2020, resumed in 2021 with a virtual program, and in 2022 reconvened in-person training. We used this opportunity to compare the participant experience, including reported knowledge acquisition and ability gains, between these nearly identical curricula delivered in vastly different circumstances and formats.

Participants and Methods

We describe the differences in immersive leadership training implementation and adaptations made for virtual vs in-person engagement of two cohorts of OB-GYN physicians. Data were collected from virtual (n=32) and in-person (n=39) participants via post-session surveys. Quantitative data reported includes participant ratings for knowledge gain and ability gain. Qualitative data were obtained via open-ended feedback questions per session and the overall experience.

Results

Knowledge and ability scores indicated strong, statistically significant gains in both formats, with some reported learning gains higher in the virtual training. Qualitative data of participant feedback identified a number of positive themes similar across the in-person and virtual settings, with virtual participants noting how construction of the virtual program produced highly effective experiences and engagement. Constructive or negative feedback of the virtual setting included time constraint issues (eg, a desire for more sessions overall or more time per session) and technical difficulties. Positive comments focused on the effectiveness of the experience in both formats and the surprising ability to connect meaningfully with others, even in a virtual environment. However, there were also many comments clearly supporting the preference for in-person over virtual experiences.

Conclusion

Immersive physician leadership training can be effectively delivered via virtual or in-person methods, resulting in significant reported gains of knowledge and skills. These programs provide valuable interpersonal connections and skills to support physician leadership. While both formats are effective, participants clearly prefer in-person leadership development experiences and interpersonal learning.

The dark side of leadership: How ineffective training and poor ethics education trigger unethical behavior?

Introduction

The challenge of restricting unethical behavior requires public companies to reinforce ethical practices among leaders through various instruments. Previous research suggests that the (un)ethical behavior of leaders can be influenced by many situational factors. This study aimed to investigate the influence of ethics training and education on unethical leadership behavior in the Algerian public companies.

Methods

Data were collected through semi-structured interviews with 15 leaders from public companies in Algeria. Data analysis was facilitated using ATLAS. ti 8 qualitative analysis software.

Results

The findings show that public companies in Algeria suffer from several issues related to leaders’ ethics training and education. The findings also indicate that some of these unethical leadership behaviors are the result of ineffective training programs and poor ethics education within public companies.

Discussion

The absence or ineffectiveness of ethics training and education within and outside organizational settings has a detrimental impact on leaders’ ethical character. This study is the first to explore how public companies in Algeria engage in ethical training and leadership education. The different sectors of the Algerian business can use the findings as a point of reference to embed the appropriate ethical climate in their respective organizations.

A theory-based evaluation of the Leadership for Universal Health Coverage Programme: insights for multisectoral leadership development in global health

Background

Leadership to manage the complex political and technical challenges of moving towards universal health coverage (UHC) is widely recognized as critical, but there are few studies which evaluate how to expand capacities in this area. This article aims to fill some of this gap by presenting the methods and findings of an evaluation of the Leadership for UHC (L4UHC) programme in 2019–2020.

Methods

Given the complexity of the intervention and environment, we adopted a theory-driven evaluation approach that allowed us to understand the role of the programme, amongst other factors. Data from a range of sources and tools were compared with a programme theory of change, with analysis structured using an evaluation matrix organized according to the Organisation for Economic Co-operation and Development–Development Assistance Committee (OECD-DAC) criteria. Data sources included key informant (KI) interviews (89 in total); surveys of the 80 workshop participants; a range of secondary data sources; case studies in two countries; and observation of activities and modules by the evaluator.

Results

Participants and KIs at the global and country levels reported high relevance of the programme and a lack of alternatives aiming at similar goals. In relation to effectiveness, at the individual level, there was an increase in some competencies, particularly for those with less experience at the baseline. Less change was observed in commitment to UHC as that started at a relatively high level. Understanding of UHC complexity grew, particularly for those coming from a non-health background. Connections across institutional divides for team members in-country increased, although variably across the countries, but the programme has not as yet had a major impact on national coalitions for UHC. Impacts on health policy and practice outcomes were evident in two out of seven countries. We examined factors favouring success and explanatory factors. We identified positive but no negative unintended effects.

Conclusions

While noting methodological constraints, the theory-based evaluation approach is found suitable for assessing and learning lessons from complex global programmes. We conclude that L4UHC is an important addition to the global and national health ecosystem, addressing a relevant need with some strong results, and also highlight challenges which can inform other programmes with similar objectives.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12961-022-00907-1.

Improving leader effectiveness: impact on employee engagement and retention

Purpose

Fixing problems in an organization often involves developing managers in order to increase leader effectiveness. This paper aims to discuss the aforementioned issue.

Design/methodology/approach

Data collection includes multiple surveys and small group interviews. Analysis uses rigorous coding methods to construct a model of critical organizational values and behaviors essential for leadership effectiveness. The authors bring “theory to practice” by applying complexity leadership concepts in the authors’ intervention strategy.

Findings

Findings are categorized into three parts: identifying critical culture value gaps, applying complexity concepts to a scenario-based training intervention, and identifying intervention outcomes. Outcomes include transformed work environment led by leaders who respect others, share decision-making and enable employees to be interdependent.

Research limitations/implications

This explanatory case study contributes to research by applying complexity leadership theory to create a practical consulting intervention.

Practical implications

This work provides a template and process for managers using complexity leadership to inform their client interventions.

Originality/value

This case study identifies value shortfalls in a manufacturing plant, documents a scenario-based training intervention which develops managers to build organizational trust. Results include reducing turnover, improving job satisfaction and increasing production.

Leadership development evaluation (LDE): reflections on a collaboratory approach

Purpose

The purpose of this paper is to report on the experience of attempting a “collaboratory” approach in sharing knowledge about leadership development evaluation (LDE). A collaboratory intertwines “collaboration” and “laboratory” to create innovation networks for all sorts of social and technological problems.

Design/methodology/approach

The authors, alongside a variety of public and private sector organisations, created the collaboratory. Within the process, the authors collected various forms of qualitative data (including interviews, observations, letter writing and postcards).

Findings

The findings show key areas of resonance, namely, the ability for participants to network, a creation of a dynamic shift in thinking and practice and the effective blending of theory and practice. Importantly, there are some critiques of the collaboratory approach discussed, including complications around: a lack of “laboratory” (hence bringing into question the idea of collaboratory itself), and the need to further develop the facilitation of such events.

Originality/value

The originality is to ultimately question whether the network actually achieved the collaboratory in reality. This study concludes, however, that there were some distinct benefits within our collaborations, especially around issues associated with LDE, and this study provides recommendations for academics and practitioners in terms of trying similar initiatives.

Evaluating the impact of the first 10 years of the Cambridge Masters in Conservation Leadership

Conservation lacks sufficient well-trained leaders who are empowered to catalyse positive change for the natural world. Addressing this need, the University of Cambridge launched a Masters in Conservation Leadership in 2010. The degree includes several features designed to enhance its impact. Firstly, it recruits international, gender-balanced cohorts of mid-career professionals, building leadership capacity in the Global South and providing a rich environment for peer learning. Secondly, teaching includes applied leadership training in topics such as fundraising, leading people and networking, as well as interdisciplinary academic topics. Thirdly, the degree is delivered through the Cambridge Conservation Initiative, a partnership of international NGOs and networks, facilitating extensive practitioner-led and experiential learning. We present details of programme design and evaluate the impact of the Masters after 10 years, using data from course records, student and alumni perspectives, and interviews with key stakeholders. The course has broadly succeeded in its design and recruitment objectives. Self-assessed leadership capabilities, career responsibilities and the overall impact of alumni increased significantly 5 years after graduation. However, specific impacts of alumni in certain areas, such as on their professional colleagues, have been less clear. We conclude by outlining future plans for the Masters in light of growing demands on conservation leaders and the changing landscape of leadership capacity development. These include reforms to course structure and assessment, long-term support to the alumni network and developing a conservation leadership community of practice.

Systemic leadership development: impact on organizational effectiveness

Purpose

This study aims to examine the impact of leadership development programs on organizational outcomes and organizational effectiveness.

Design/methodology/approach

Using a grounded theory approach, semi-structured interviews were conducted with 15 executive leaders from aviation firms in Brazil with employees participating in a leadership development program. NVivo12 was used for coding and managing the data. Thematic analysis was performed to determine themes and categories.

Findings

The leadership development program was found to influence organizational level outcomes identified as themes of internal impact, external impact, skill development and capacity. The interviews also found that executive leaders perceived the leadership development program to impact organizational effectiveness. Connections to human capital, social capital and collective leadership were found as outcomes of the leadership development program contributing to organizational effectiveness.

Research limitations/implications

The findings are dependent upon the executive leaders’ interviews and are limited sample size. The protocol of subjective inter-coder reliability was followed supporting the credibility and dependability of the findings; however, researcher bias may still be present in qualitative studies. Generalizability outside of the Brazilian aviation context is cautioned until further studies in additional contexts and industries are completed.

Practical implications

The findings of this study support leadership development programs as impactful on organizational outcomes and effectiveness. Incorporating leadership development programs as part of human capital management strategies supports organizational effectiveness through increased collective leadership capacity, human capital development and social capital.

Originality/value

A large amount is known regarding the outcomes for individuals as a result of leadership development programs with less examination on the contribution to organizational level outcomes and organizational effectiveness. This study aids in bridging this gap.

An Evaluation of Faculty Development Programs for Clinician-Educators: A Scoping Review

PURPOSE

Faculty development (FD) has become increasingly important for clinician-educators. An array of FD programs has been developed, but the impact of these programs on clinician-educators and their learners and workplace is less known. The authors conducted a scoping review to explore the status of program evaluation in FD for clinician-educators to inform future planning and research.

METHOD

Five databases were searched for articles published from January 1998 to August 2018 using Arksey and O'Malley's framework and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews. Studies that described evaluation methods and outcomes of FD programs for clinician-educators were included. Data were collected and organized according to program domain (teaching, research/scholarship, leadership, or a combination of skills). A modified version of the Kirkpatrick model was used to compare results among studies.

RESULTS

From a total of 2,091 articles, 1,095 were eligible for full review, and 31 met the inclusion criteria. Seven programs targeted only teaching skills, 3 research/scholarship skills, 7 leadership skills, and 14 a combination of skills. Eighteen programs required the completion of a project; fewer offered fellowships, master's degrees, or certificates. Participant surveys were the most common evaluation method across all domains. Often used metrics included participant satisfaction and self-reported knowledge, skills, behavior changes, scholarly output, and leadership positions. Less common evaluation methods included learner and peer evaluations, interviews, and focus groups. Change at the institutional level was evaluated in 11 programs.

CONCLUSIONS

Program evaluation remains an underdeveloped area in FD for clinician-educators. Developers expend significant effort on program design and implementation but approach evaluation less purposefully. Rigorous metrics that align with program goals and are used longitudinally are needed to accurately assess the impact of FD programs on participants and their learners, workplace, and institutions at large.

Good intentions gone awry: investigating a strategically oriented MLD program

Purpose

Complex organizations increasingly rely on middle managers as strategic linking-pins between the top and bottom levels of the organization. Using social identity theory and commitment theory as the foundation, this study evaluates a management and leadership development program (MLDP) intended to engage middle managers as strategy creators and implementers. We also evaluate the cascading effects of leadership development by assessing changes in subordinates' identification with the leader, and commitment to the work unit and organization.

Design/methodology/approach

Using a sample of 107 manager participants and 913 of their subordinates, this study measures differences in both manager and subordinate identification and commitment prior to and after the completion of a 6 months strategically oriented MLDP.

Findings

Despite the organizations' best intentions, manager identification with and commitment to the organization decreased after completion of the MLDP. Similarly, subordinates identification with the leader and commitment to the organization also decreased at Time 2.

Research limitations/implications

The results paint a complex picture of the nuances of social identification as an outcome of MLDPs, and problematize the notion of cascading effects on subordinates within the organization. Researchers are encouraged to further examine organizational attitudes and perceptions as outcomes of MLDPs.

Practical implications

Suggestions are offered regarding how practitioners can manage strategically oriented MLDPs in order to avoid identity confusion and promote strategic action.

Originality/value

Strategically oriented MLDPs are increasingly popular in organizations. This study is one of the first to evaluate the theoretical mechanisms through which these programs may affect managers and problematize these effects for complex organizations.