Knowledge theories can inform evaluation practice: What can a complexity lens add?

A New Framework for Monitoring and Evaluating Health Impact Assessment: Capitalising on a French Case Study with the Literature in Evaluation

Health impact assessment (HIA) is a prospective approach that aims to identify the potential consequences of policies or projects on health in order to propose measures to make them healthier. Initiated in the late nineties, the approach emerged over ten years ago in France. However, the evaluation of HIA effectiveness remains seldomly practised and its theoretical background should be deepened. The aim of this article is to generate a discussion on how to evaluate HIA effectiveness and contribute to its methodological tooling, drawing on an evaluative experience of multiple French HIAs. Our work is based on an iterative approach between an analysis of the evaluation literature and a critical look at an HIA evaluation. We first carried out the evaluation of three HIAs in 2017-2018, combining a normative approach and qualitative research in order to explore each HIA as a phenomenon in its own context. Two years later, we conducted a self-assessing expertise on this evaluation, supported by an analysis of the literature in the field of public policy evaluation, in order to refine the theoretical framework for evaluating HIA effectiveness and ultimately to enhance professional practice by evaluators. This work led to the production of a logic model that identifies, through three dimensions (context, implementation and governance), the multiple pathways that HIA may take to bring about change. It also seeks to show the interdependence of these pathways towards change and helps identify the key drivers and mechanisms of HIA success. In this respect, it complements existing HIA evaluation models as it can serve both as a generic framework for evaluating HIA effectiveness and as an instrument for monitoring HIA implementation.

Enhancing equitable engagement for digital health promotion: Lessons from evaluating a childrearing app in Indonesia

Part of the appeal of digital health interventions, including mHealth, is the potential for greater reach in places where conventional health promotion is hampered by geographical, financial or social barriers. Yet, 'engagement' - typically understood as user experience and interactions with technology - remains a persistent challenge, particularly in places where technology access or familiarity with technology is limited. We undertook an evaluation of a childrearing app to promote socioemotional and cognitive development in early childhood across the world. In this article, we present findings from qualitative research on app rollout in Indonesia, the first of numerous low- and middle-income countries targeted by the app. We draw on systems theory and complexity thinking to broaden the lens of 'engagement' beyond individual users to encompass collective systems (families and communities), exploring how the intervention was harnessed to meet local contextual needs. The qualitative research involved semi-structured interviews, workshops and audio diaries with 57 diverse stakeholders, including Indonesian parents, caregivers, and collaborators involved in funding, development, and dissemination of the app. We observed the importance of social connection, sense-making, and interactive learning for enhancing engagement with the app and its messages. Enthusiastic users, strongly linked across community networks (e.g. kindergarten teachers), improvised dissemination strategies to facilitate uptake. Interactive learning that tapped into familiar social structures (e.g. intergenerational hierarchies) was crucial for engagement. Understanding ways the app failed to tap into structures of social connection served to highlight the need to embed strategies to support collective engagement.

Adapting population health interventions for new contexts: qualitative interviews understanding the experiences, practices and challenges of researchers, funders and journal editors

OBJECTIVES

Research on the adaptation of population health interventions for implementation in new contexts is rapidly expanding. This has been accompanied by a recent increase in the number of frameworks and guidance to support adaptation processes. Nevertheless, there remains limited exploration of the real-world experiences of undertaking intervention adaptation, notably the challenges encountered by different groups of stakeholders, and how these are managed. Understanding experiences is imperative in ensuring that guidance to support adaptation has practical utility. This qualitative study examines researcher and stakeholder experiences of funding, conducting and reporting adaptation research.

SETTING

Adaptation studies.

PARTICIPANTS

Participants/cases were purposefully sampled to represent a range of adapted interventions, types of evaluations, expertise and countries. Semistructured interviews were conducted with a sample of researchers (n=23), representatives from research funding panels (n=6), journal editors (n=5) and practitioners (n=3).

MEASURES

A case study research design was used. Data were analysed using the framework approach. Overarching themes were discussed within the study team, with further iterative refinement of subthemes.

RESULTS

The results generated four central themes. The first three relate to the experience of intervention adaptation (1) involving stakeholders throughout the adaptation process and how to integrate the evidence base with experience; (2) selecting the intervention and negotiating the mismatch between the original and the new context; and (3) the complexity and uncertainty when deciding the re-evaluation process. The final theme (4) reflects on participants' experiences of using adaptation frameworks in practice, considering recommendations for future guidance development and refinement.

CONCLUSION

This study highlights the range of complexities and challenges experienced in funding, conducting and reporting research on intervention adaptation. Moving forward, guidance can be helpful in systematising processes, provided that it remains responsive to local contexts and encourage innovative practice.

Addressing alcohol-related harms in the local night-time economy: a qualitative process evaluation from a complex systems perspective

OBJECTIVES

English local authorities (LAs) are interested in reducing alcohol-related harms and may use discretionary powers such as the Late Night Levy (LNL) to do so. This study aims to describe how system stakeholders hypothesise the levy may generate changes and to explore how the system, its actors and the intervention adapt and co-evolve over time.

DESIGN

A process evaluation from a complex systems perspective, using qualitative methods.

SETTING

A London LA with high densities of residential and commercial properties, which implemented the LNL in 2014.

PARTICIPANTS

Data were generated through interviews with LNL implementers and alcohol consumers, observations in bars and during LNL patrols and documentary review.

INTERVENTION

The LNL allows LAs to charge late-night alcohol retailers an annual fee (£299-£4440) to manage and police the night-time economy (NTE).

RESULTS

When the LNL was being considered, stakeholders from different interest groups advanced diverse opinions about its likely impacts while rarely referencing supporting research evidence. Proponents of the levy argued it could reduce crime and anti-social behaviour by providing additional funds to police and manage the NTE. Critics of the levy hypothesised adverse consequences linked to claims that the intervention would force venues to vary their hours or close, cluster closing times, reduce NTE diversity and undermine public-private partnerships. In the first 2 years, levy-funded patrols developed relationships with the licensed trade and the public. The LNL did not undermine public-private partnerships and while some premises varied their hours, these changes did not undermine the intervention's viability, nor significantly cluster venue closing times, nor obviously damage the area's reputation for having a diverse NTE.

CONCLUSIONS

This study applies a framework for process evaluation from a complex systems perspective. The evaluation could be extended to measure alcohol-related outcomes and to consider the interplay between the national and local systems.

Navigating the river(s) of systems change: a multi-methods, qualitative evaluation exploring the implementation of a systems approach to physical activity in Gloucestershire, England

OBJECTIVES

Systems approaches aim to change the environments in which people live, through cross-sectoral working, by harnessing the complexity of the problem. This paper sought to identify: (1) the strategies which support the implementation of We Can Move (WCM), (2) the barriers to implementation, (3) key contextual factors that influence implementation and (4) impacts associated with WCM.

DESIGN

A multi-methods evaluation of WCM was completed between April 2019 and April 2021. Ripple Effects Mapping (REM) and semi-structured interviewers were used. Framework and content analysis were systematically applied to the dataset.

SETTING

WCM-a physical activity orientated systems approach being implemented in Gloucestershire, England.

PARTICIPANTS

31 stakeholder interviews and 25 stakeholders involved in 15 REM workshops.

RESULTS

A white-water rafting analogy was developed to present the main findings. The successful implementation of WCM required a facilitative, well-connected and knowledgeable guide (ie, the lead organisation), a crew (ie, wider stakeholders) who's vision and agenda aligned with WCM's purpose, and a flexible delivery approach that could respond to ever-changing nature of the river (ie, local and national circumstances). The context surrounding WCM further strengthened and hampered its implementation. Barriers included evaluative difficulties, a difference in stakeholder and organisational perspectives, misaligned expectations and understandings of WCM, and COVID-19 implications (COVID-19 also presented as a facilitative factor). WCM was said to strengthen cohesion and collaboration between partners, benefit other agendas and policies (eg, mental health, town planning, inequality), and improve physical activity opportunities and environments.

CONCLUSIONS

This paper is one of the first to evaluate a systems approach to increasing physical activity. We highlight key strategies and contextual factors that influenced the implementation of WCM and demonstrate some of the wider benefits from such approaches. Further research and methodologies are required to build the evidence base surrounding systems approaches in Public Health.

The influence of unhealthy food and beverage marketing through social media and advergaming on diet-related outcomes in children-A systematic review

Children are increasingly exposed to food and beverage marketing, but little is known about the specific effects of marketing through media most used by children. This study aims to systematically review the influence of unhealthy food and beverage marketing through social media and advergaming on diet-related outcomes in children. Seven databases were systematically searched for English peer-reviewed quantitative and qualitative scientific studies on the effects of marketing of unhealthy products through social media or advergaming on a range of diet-related outcomes in children. Risk of bias was assessed with tools specific for the different study designs. Twenty-six studies were included, of which 20 examined the effect of food and beverage marketing through advergaming and six through social media. Most studies had a high risk of bias. The results suggested that unhealthy food and beverage marketing through social media and advergaming has a significant effect on pester behaviors, food choice, and food intake of children. The studies demonstrate that unhealthy food and beverage marketing through media popular with children significantly impacts different diet-related outcomes. Combined with existing evidence on this effect in other settings, this review provides clear evidence of the need for policies targeting screen-based marketing.

Applying a complex systems perspective to alcohol consumption and the prevention of alcohol-related harms in the 21st century: a scoping review

BACKGROUND AND AIMS

A complex systems perspective has been advocated to explore multi-faceted factors influencing public health issues, including alcohol consumption and associated harms. This scoping review aimed to identify studies that applied a complex systems perspective to alcohol consumption and the prevention of alcohol-related harms in order to summarize their characteristics and identify evidence gaps.

METHODS

Studies published between January 2000 and September 2020 in English were located by searching for terms synonymous with 'complex systems' and 'alcohol' in the Scopus, MEDLINE, Web of Science and Embase databases, and through handsearching and reference screening of included studies. Data were extracted on each study's aim, country, population, alcohol topic, system levels, funding, theory, methods, data sources, time-frames, system modifications and type of findings produced.

RESULTS

Eighty-seven individual studies and three systematic reviews were identified, the majority of which were conducted in the United States or Australia in the general population, university students or adolescents. Studies explored types and patterns of consumption behaviour and the local environments in which alcohol is consumed. Most studies focused on individual and local interactions and influences, with fewer examples exploring the relationships between these and regional, national and international subsystems. The body of literature is methodologically diverse and includes theory-led approaches, dynamic simulation models and social network analyses. The systematic reviews focused on primary network studies.

CONCLUSIONS

The use of a complex systems perspective has provided a variety of ways of conceptualizing and analyzing alcohol use and harm prevention efforts, but its focus ultimately has remained on predominantly individual- and/or local-level systems. A complex systems perspective represents an opportunity to address this gap by also considering the vertical dimensions that constrain, shape and influence alcohol consumption and related harms, but the literature to date has not fully captured this potential.

The what, why and when of adapting interventions for new contexts: A qualitative study of researchers, funders, journal editors and practitioners' understandings

BACKGROUND

The adaptation of interventions for new contexts is a rapidly developing research area. To date there is no consensus-based guidance to support decision-making and recommend adaptation processes. The ADAPT study is developing such guidance. This aim of the qualitative component of the study was to explore stakeholders' understandings of adaptation, as to date there has limited consideration of how different concepts and meanings shape decision-making and practice.

METHODS

A case study research design was used. Participants/cases were purposefully sampled based on study outcome, study design, expertise, context and country. Semi-structured interviews were conducted with a sample of researchers (n = 23); representatives from research funding panels (n = 6); journal editors (n = 5) and practitioners (n = 3). Data were analysed using the Framework approach. Overarching themes were discussed with the ADAPT study team, with further iterative refinement of subthemes.

RESULTS

The results generated four central themes. Four themes related to stakeholders' understanding: 1) definitions of adaptation and related concepts; 2) rationales for undertaking adaptation; 3) the appropriate timing for adaptation; and 4) ensuring fidelity when implementing adapted interventions.

CONCLUSION

The findings highlight the lack of clarity around key concepts and uncertainty about central decision-making processes, notably why interventions should be adapted, when and to what extent. This has informed the ADAPT study's guidance, shaping the scope and nature of recommendations to be included and surfacing key uncertainties that require future consideration.

The Use of Ethnography in Maternity Care

While the value of ethnography in health research is recognized, the extent to which it is used is unclear. The aim of this review was to map the use of ethnography in maternity care, and identify the extent to which the key principles of ethnographies were used or reported. We systematically searched the literature over a 10-year period. Following exclusions we analyzed 39 studies. Results showed the level of detail between studies varied greatly, highlighting the inconsistencies, and poor reporting of ethnographies in maternity care. Over half provided no justification as to why ethnography was used. Only one study described the ethnographic approach used in detail, and covered the key features of ethnography. Only three studies made reference to the underpinning theoretical framework of ethnography as seeking to understand and capture social meanings. There is a need to develop reporting guidelines to guide researchers undertaking and reporting on ethnographic research.

Exploring variation in the performance of planned birth: A mixed method study

OBJECTIVE

Variation in practice in relation to indications and timing for both induction of labour (IOL) and planned caesarean section (CS) clearly exists. However, the extent of this variation, and how this variation is explained by clinicians remains unclear. The aim of this study was to map the variation in IOL and planned CS at eight Australian hospitals, and understand why variation occurs from the perspective of clinicians at these hospitals. Our ultimate aim was to identify opportunities for improvement as evidenced by hospital data, clinician experiences, and feedback.

DESIGN

A two-phased mixed method study using sequential explanatory study design. The first phase consisted of an analysis of routinely collected patient data to map variation between hospitals. The second phase consisted of focus groups with clinicians to gain their perspectives on the reasons for variation.

SETTING AND PARTICIPANTS

Patient data consisted of routine data from 19,073 women giving birth at eight Sydney hospitals between November 2017 and October 2018. Focus groups were attended by a total of 61 medical staff and 121 midwives.

RESULTS

Hospital data analysis found substantial variation, before and after adjustment for case-mix, in rates of both IOL (adjusted rates 27.6%-42%) and planned CS (adjusted rate 15.4%-22.6%). Planned CS by gestation also showed variation, although after restricting analysis to term (≥37 weeks gestation) births, variation was reduced. At focus groups, five main themes explaining variation emerged: local guidelines, policies and procedures (inconsistency and ambiguity); uncertainty of the evidence/what is best practice (contradictory research and different interpretations of evidence); clinician preferences, beliefs and values; the culture of the unit; and organisational influences (access to specialised clinics, theatre time).

KEY CONCLUSIONS

Considerable variation in IOL and planned CS, even after case-mix adjustment, was found in this sample of Australian hospitals. Engagement with hospital clinicians identified likely sources of this variation and enabled clinicians at each hospital to consider appropriate local responses to address variation, such as more detailed review of their planned birth cases.

IMPLICATIONS FOR PRACTICE

At a macro level, measures to reduce unwarranted variation should initially focus on consistent national guidelines, while supporting equitable access to operating theatres for optimal CS timing, and shared decision-making training to reduce influence of clinician preference.

Evaluation of public health interventions from a complex systems perspective: A research methods review

INTRODUCTION

Applying a complex systems perspective to public health evaluation may increase the relevance and strength of evidence to improve health and reduce health inequalities. In this review of methods, we aimed to: (i) classify and describe different complex systems methods in evaluation applied to public health; and (ii) examine the kinds of evaluative evidence generated by these different methods.

METHODS

We adapted critical review methods to identify evaluations of public health interventions that used systems methods. We conducted expert consultation, searched electronic databases (Scopus, MEDLINE, Web of Science), and followed citations of relevant systematic reviews. Evaluations were included if they self-identified as using systems- or complexity-informed methods and if they evaluated existing or hypothetical public health interventions. Case studies were selected to illustrate different types of complex systems evaluation.

FINDINGS

Seventy-four unique studies met our inclusion criteria. A framework was developed to map the included studies onto different stages of the evaluation process, which parallels the planning, delivery, assessment, and further delivery phases of the interventions they seek to inform; these stages include: 1) theorising; 2) prediction (simulation); 3) process evaluation; 4) impact evaluation; and 5) further prediction (simulation). Within this framework, we broadly categorised methodological approaches as mapping, modelling, network analysis and 'system framing' (the application of a complex systems perspective to a range of study designs). Studies frequently applied more than one type of systems method.

CONCLUSIONS

A range of complex systems methods can be utilised, adapted, or combined to produce different types of evaluative evidence. Further methodological innovation in systems evaluation may generate stronger evidence to improve health and reduce health inequalities in our complex world.

Applying systems thinking to knowledge mobilisation in public health

CONTEXT

Knowledge mobilisation (KM) is a vital strategy in efforts to improve public health policy and practice. Linear models describing knowledge transfer and translation have moved towards multi-directional and complexity-attuned approaches where knowledge is produced and becomes meaningful through social processes. There are calls for systems approaches to KM but little guidance on how this can be operationalised. This paper describes the contribution that systems thinking can make to KM and provides guidance about how to put it into action.

METHODS

We apply a model of systems thinking (which focuses on leveraging change in complex systems) to eight KM practices empirically identified by others. We describe how these models interact and draw out some key learnings for applying systems thinking practically to KM in public health policy and practice. Examples of empirical studies, tools and targeted strategies are provided.

FINDINGS

Systems thinking can enhance and fundamentally transform KM. It upholds a pluralistic view of knowledge as informed by multiple parts of the system and reconstituted through use. Mobilisation is conceived as a situated, non-prescriptive and potentially destabilising practice, no longer conceptualised as a discrete piece of work within wider efforts to strengthen public health but as integral to and in continual dialogue with those efforts. A systems approach to KM relies on contextual understanding, collaborative practices, addressing power imbalances and adaptive learning that responds to changing interactions between mobilisation activities and context.

CONCLUSION

Systems thinking offers valuable perspectives, tools and strategies to better understand complex problems in their settings and for strengthening KM practice. We make four suggestions for further developing empirical evidence and debate about how systems thinking can enhance our capacity to mobilise knowledge for solving complex problems - (1) be specific about what is meant by 'systems thinking', (2) describe counterfactual KM scenarios so the added value of systems thinking is clearer, (3) widen conceptualisations of impact when evaluating KM, and (4) use methods that can track how and where knowledge is mobilised in complex systems.

Qualitative process evaluation from a complex systems perspective: A systematic review and framework for public health evaluators

BACKGROUND

Public health evaluation methods have been criticized for being overly reductionist and failing to generate suitable evidence for public health decision-making. A "complex systems approach" has been advocated to account for real world complexity. Qualitative methods may be well suited to understanding change in complex social environments, but guidance on applying a complex systems approach to inform qualitative research remains limited and underdeveloped. This systematic review aims to analyze published examples of process evaluations that utilize qualitative methods that involve a complex systems perspective and proposes a framework for qualitative complex system process evaluations.

METHODS AND FINDINGS

We conducted a systematic search to identify complex system process evaluations that involve qualitative methods by searching electronic databases from January 1, 2014-September 30, 2019 (Scopus, MEDLINE, Web of Science), citation searching, and expert consultations. Process evaluations were included if they self-identified as taking a systems- or complexity-oriented approach, integrated qualitative methods, reported empirical findings, and evaluated public health interventions. Two reviewers independently assessed each study to identify concepts associated with the systems thinking and complexity science traditions. Twenty-one unique studies were identified evaluating a wide range of public health interventions in, for example, urban planning, sexual health, violence prevention, substance use, and community transformation. Evaluations were conducted in settings such as schools, workplaces, and neighborhoods in 13 different countries (9 high-income and 4 middle-income). All reported some utilization of complex systems concepts in the analysis of qualitative data. In 14 evaluations, the consideration of complex systems influenced intervention design, evaluation planning, or fieldwork. The identified studies used systems concepts to depict and describe a system at one point in time. Only 4 evaluations explicitly utilized a range of complexity concepts to assess changes within the system resulting from, or co-occurring with, intervention implementation over time. Limitations to our approach are including only English-language papers, reliance on study authors reporting their utilization of complex systems concepts, and subjective judgment from the reviewers relating to which concepts featured in each study.

CONCLUSION

This study found no consensus on what bringing a complex systems perspective to public health process evaluations with qualitative methods looks like in practice and that many studies of this nature describe static systems at a single time point. We suggest future studies use a 2-phase framework for qualitative process evaluations that seek to assess changes over time from a complex systems perspective. The first phase involves producing a description of the system and identifying hypotheses about how the system may change in response to the intervention. The second phase involves following the pathway of emergent findings in an adaptive evaluation approach.

How can we adapt complex population health interventions for new contexts? Progressing debates and research priorities

INTRODUCTION

The UK Medical Research Council and National Institute for Health Research have funded the ADAPT study (2018-2020), to develop methodological guidance for the adaptation of complex population health interventions for new contexts. While there have been advances in frameworks, there are key theoretical and methodological debates to progress. The ADAPT study convened a panel meeting to identify and enrich these debates. This paper presents the panel's discussions and suggests directions for future research.

METHODS

Sixteen researchers and one policymaker convened for a 1-day meeting in July 2019. The aim was to reflect on emerging study findings (systematic review of adaptation guidance; scoping review of case examples; and qualitative interviews with funders, journal editors, researchers and policymakers), progress theoretical and methodological debates, and consider where innovation may be required to address research gaps.

DISCUSSION

Despite the proliferation of adaptation frameworks, questions remain over the definition of basic concepts (eg, adaptation). The rationale for adaptation, which often focuses on differences between contexts, may lead to adaptation hyperactivity. Equal emphasis should be placed on similarities. Decision-making about intervention modification currently privileges the concept of 'core components', and work is needed to progress the use and operationalisation of 'functional fidelity'. Language and methods must advance to ensure meaningful engagement with diverse stakeholders in adaptation processes. Further guidance is required to assess the extent of re-evaluation required in the new context. A better understanding of different theoretical perspectives, notably complex systems thinking, implementation science and realist evaluation may help in enhancing research on adaptation.

Core and peripheral organisations in prevention: Insights from social network analysis

ISSUE

Formal (eg funded) community-level organisational collaborations are becoming more common in prevention. Rapid methods to assess organisational relationships could allow us to consider the significance of any pre-existing relationship patterns in communities that might impact on collaboration effectiveness. Insights may identify new options for practice.

METHODS

We used social network analysis to study organisations engaged in prevention but not (yet) part of a formal purposive collaboration. Within a single community, we identified organisations providing programs in chronic disease prevention. We used whole network analysis methods to describe the extent to which organisations were aware, had contact, coordinated activity and/or collaborated more intensively. We also identified the contribution made to prevention locally. Results were compared with key informant interviews.

RESULTS

There was an identifiable network structure, with more relationships across the network than one would expect by chance. The network had a core-periphery structure, meaning that, in terms of the relationships we measured, there were highly connected organisations who were strongly interlinked with each other (the core), alongside less connected organisations that were linked to the core but not to each other (the periphery). Core organisations were significantly more likely to have expertise in prevention and to have prevention staff.

CONCLUSIONS

To our knowledge, it is new to identify inherent or "pre-existing" core-periphery structures in interorganisational health promotion. Yet, core-periphery structures are common in many social settings. They advantage entities in the core and are prone to further entrenchment. SO WHAT?: Our results map and quantify intuitive understandings about organisational "key players", thus enabling practitioners/organisations to critically reflect on what their role should be when it comes to activating communities ie to embed, or attempt to counterbalance, pre-existing power structures.

Defining and assessing context in healthcare implementation studies: a systematic review

BACKGROUND

The implementation of evidence-based healthcare interventions is challenging, with a 17-year gap identified between the generation of evidence and its implementation in routine practice. Although contextual factors such as culture and leadership are strong influences for successful implementation, context remains poorly understood, with a lack of consensus regarding how it should be defined and captured within research. This study addresses this issue by providing insight into how context is defined and assessed within healthcare implementation science literature and develops a definition to enable effective measurement of context.

METHODS

Medline, PsychInfo, CINAHL and EMBASE were searched. Articles were included if studies were empirical and evaluated context during the implementation of a healthcare initiative. These English language articles were published in the previous 10 years and included a definition and assessment of context. Results were synthesised using a narrative approach.

RESULTS

Three thousand and twenty-one search records were obtained of which 64 met the eligibility criteria and were included in the review. Studies used a variety of definitions in terms of the level of detail and explanation provided. Some listed contextual factors (n = 19) while others documented sub-elements of a framework that included context (n = 19). The remaining studies provide a rich definition of general context (n = 11) or aspects of context (n = 15). The Alberta Context Tool was the most frequently used quantitative measure (n = 4), while qualitative papers used a range of frameworks to evaluate context. Mixed methods studies used diverse approaches; some used frameworks to inform the methods chosen while others used quantitative measures to inform qualitative data collection. Most studies (n = 50) applied the chosen measure to all aspects of study design with a majority analysing context at an individual level (n = 29).

CONCLUSIONS

This review highlighted inconsistencies in defining and measuring context which emphasised the need to develop an operational definition. By providing this consensus, improvements in implementation processes may result, as a common understanding will help researchers to appropriately account for context in research.

Toward an institutional perspective on social capital health interventions: lay community health workers as social capital builders

This article argues that social capital health research should move beyond a mere focus on social cohesion and network perspectives to integrate an institutional approach into the development of social capital health interventions. An institutional perspective, which is unique in its emphasis on linking social capital in addition to the bonding and bridging forms, contextualises social capital, allowing researchers to confront the complexity of social relationships. This perspective allows for the construction of interventions that draw on the resources of diverse actors, particularly the state. One intervention strategy with the potential to create community linkages involves lay community health workers (LCHWs), individuals who are trained to perform a variety of health-related functions but lack a formal professional health education. This article begins with a review of the institutional social capital-building literature. It then goes on to briefly review the social capital and health literature and discuss the state of intervention research. Thereafter, it describes LCHWs and discusses studies that have utilised LCHWs to tackle community health problems. In doing so, this article presents an institutional-based systematic framework for how LCHWs can build social capital, including a discussion of the ways in which LCHWs can successfully promote bonding, bridging and linking social capital.

The time has come: Embedded implementation research for health care improvement

RATIONALE, AIMS, AND OBJECTIVES

The field of implementation science has developed in response to slow and inconsistent translation of evidence into practice. Despite utilizing increasingly sophisticated approaches to implementation, including applying a complexity science lens and conducting realist evaluations, challenges remain to getting the kinds of outcomes hoped for by implementation efforts. These include gaining access and buy-in from those implementing the change and accounting for the influence of local context. One emerging approach to address these challenges is embedded implementation research-a collaborative, adaptive approach to improvement. It involves researchers and implementers working together in situ from the outset of, and throughout, an implementation project. Both groups can benefit from the collaboration: it increases the rigor of evaluation, provides opportunities to improve the intervention through direct feedback, and promotes better on-the-ground understanding of the change process. We aimed to examine the potential benefits, and some of the challenges, of increased embeddedness.

METHOD

We performed a multi-case analysis of implementation research projects that varied by degree of embeddedness.

RESULTS

Embedded implementation research may offer a range of advantages over dichotomized research-practice designs, including better understanding of local context and direct feedback to improve the implementation along the way. We present a model that spans four approaches: dichotomized research-practice, collaborative linking-up, partially-embedded, and deep immersion.

CONCLUSION

Embedded implementation research approaches hold promise in comparison to traditional dichotomized-research practice designs, where the research is external to the implementation and conducts a summative evaluation. We are only beginning to understand how such partnerships operate in practice and what makes them successful. Our analysis suggests the time has come to consider such approaches.

When complexity science meets implementation science: a theoretical and empirical analysis of systems change

BACKGROUND

Implementation science has a core aim - to get evidence into practice. Early in the evidence-based medicine movement, this task was construed in linear terms, wherein the knowledge pipeline moved from evidence created in the laboratory through to clinical trials and, finally, via new tests, drugs, equipment, or procedures, into clinical practice. We now know that this straight-line thinking was naïve at best, and little more than an idealization, with multiple fractures appearing in the pipeline.

DISCUSSION

The knowledge pipeline derives from a mechanistic and linear approach to science, which, while delivering huge advances in medicine over the last two centuries, is limited in its application to complex social systems such as healthcare. Instead, complexity science, a theoretical approach to understanding interconnections among agents and how they give rise to emergent, dynamic, systems-level behaviors, represents an increasingly useful conceptual framework for change. Herein, we discuss what implementation science can learn from complexity science, and tease out some of the properties of healthcare systems that enable or constrain the goals we have for better, more effective, more evidence-based care. Two Australian examples, one largely top-down, predicated on applying new standards across the country, and the other largely bottom-up, adopting medical emergency teams in over 200 hospitals, provide empirical support for a complexity-informed approach to implementation. The key lessons are that change can be stimulated in many ways, but a triggering mechanism is needed, such as legislation or widespread stakeholder agreement; that feedback loops are crucial to continue change momentum; that extended sweeps of time are involved, typically much longer than believed at the outset; and that taking a systems-informed, complexity approach, having regard for existing networks and socio-technical characteristics, is beneficial.

CONCLUSION

Construing healthcare as a complex adaptive system implies that getting evidence into routine practice through a step-by-step model is not feasible. Complexity science forces us to consider the dynamic properties of systems and the varying characteristics that are deeply enmeshed in social practices, whilst indicating that multiple forces, variables, and influences must be factored into any change process, and that unpredictability and uncertainty are normal properties of multi-part, intricate systems.

Physicians’ accounts of frontline tensions when implementing pilot projects to improve primary care

Purpose

Canada’s primary care system has been described as “a culture of pilot projects” with little evidence of converting successful initiatives into funded, permanent programs or sharing project outcomes and insights across jurisdictions. Health services pilot projects are advocated as an effective strategy for identifying promising models of care and building integrated care partnerships in local settings. In the qualitative study reported here, the purpose of this paper is to investigate the strengths and challenges of this approach.

Design/methodology/approach

Semi-structured interviews were conducted with 34 primary care physicians who discussed their experiences as pilot project leads. Following thematic analysis methods, broad system issues were captured as well as individual project information.

Findings

While participants often portrayed themselves as advocates for vulnerable patients, mobilizing healthcare organizations and providers to support new models of care was discussed as challenging. Competition between local healthcare providers and initiatives could impact pilot project success. Participants also reported tensions between their clinical, project management and research roles with additional time demands and skill requirements interfering with the work of implementing and evaluating service innovations.

Originality/value

Study findings highlight the complexity of pilot project implementation, which encompasses physician commitment to addressing care for vulnerable populations through to the need for additional skill set requirements and the impact of local project environments. The current pilot project approach could be strengthened by including more multidisciplinary collaboration and providing infrastructure supports to enhance the design, implementation and evaluation of health services improvement initiatives.

False Dawns and New Horizons in Patient Safety Research and Practice

In response to a weight of evidence that patients are frequently harmed as a result of their care, there have been concerted efforts to make healthcare safer, with health systems across the globe investing significant resources in policies and programmes designed to reduce adverse events. Yet, despite extensive efforts, improvements in safety have proved difficult to sustain and spread, with studies confirming there has been no measurable, systems-level improvement in the overall rates of preventable harm. Here, we highlight the limitations of the thinking which underpins current efforts to make healthcare systems safer and point to new and emerging approaches to understanding and addressing patient safety in complex, dynamic health systems.

Making sense of the emerging conversation in evaluation about systems thinking and complexity science

In the last twenty years, a conversation has emerged in the evaluation field about the potential of systems thinking and complexity science (STCS) to transform the practice of evaluating social interventions. Documenting and interpreting this conversation are necessary to advance our understanding of the significance of using STCS in planning, implementing, and evaluating social interventions. Guided by a generic framework for evaluation practice, this paper reports on an inter-disciplinary literature review and argues that STCS raises some new ways of thinking about and carrying out the following six activities: 1) supporting social problem solving; 2) framing interventions and contexts; 3) selecting and using methods; 4) engaging in valuing; 5) producing and justifying knowledge; and 6) facilitating use. Following a discussion of these issues, future directions for research and practice are suggested.

Lessons from complex interventions to improve health

Complexity-resulting from interactions among many component parts-is a property of both the intervention and the context (or system) into which it is placed. Complexity increases the unpredictability of effects. Complexity invites new approaches to logic modeling, definitions of integrity and means of standardization, and evaluation. New metaphors and terminology are needed to capture the recognition that knowledge generation comes from the hands of practitioners/implementers as much as it comes from those usually playing the role of intervention researcher. Failure to acknowledge this may blind us to the very mechanisms we seek to understand. Researchers in clinical settings are documenting health improvement gains made as a consequence of complex systems thinking. Improvement science in clinical settings has much to offer researchers in population health.

Applying complexity theory: a review to inform evaluation design

Complexity theory has increasingly been discussed and applied within evaluation literature over the past decade. This article reviews the discussion and use of complexity theory within academic journal literature. The aim is to identify the issues to be considered when applying complexity theory to evaluation. Reviewing 46 articles, two groups of themes are identified. The first group considers implications of applying complexity theory concepts for defining evaluation purpose, scope and units of analysis. The second group of themes consider methodology and method. Results provide a starting point for a configuration of an evaluation approach consistent with complexity theory, whilst also identifying a number of design considerations to be resolved within evaluation planning.

Reducing health inequities: the contribution of core public health services in BC

BACKGROUND

Within Canada, many public health leaders have long identified the importance of improving the health of all Canadians especially those who face social and economic disadvantages. Future improvements in population health will be achieved by promoting health equity through action on the social determinants of health. Many Canadian documents, endorsed by government and public health leaders, describe commitments to improving overall health and promoting health equity. Public health has an important role to play in strengthening action on the social determinants and promoting health equity. Currently, public health services in British Columbia are being reorganized and there is a unique opportunity to study the application of an equity lens in public health and the contribution of public health to reducing health inequities. Where applicable, we have chosen mental health promotion, prevention of mental disorders and harms of substance use as exemplars within which to examine specific application of an equity lens.

METHODS/DESIGN

This research protocol is informed by three theoretical perspectives: complex adaptive systems, critical social justice, and intersectionality. In this program of research, there are four inter-related research projects with an emphasis on both integrated and end of grant knowledge translation. Within an overarching collaborative and participatory approach to research, we use a multiple comparative case study research design and are incorporating multiple methods such as discourse analysis, situational analysis, social network analysis, concept mapping and grounded theory.

DISCUSSION

An important aim of this work is to help ensure a strong public health system that supports public health providers to have the knowledge, skills, tools and resources to undertake the promotion of health equity. This research will contribute to increasing the effectiveness and contributions of public health in reducing unfair and inequitable differences in health among population groups. As a collaborative effort between public health practitioners/decision makers and university researchers, this research will provide important understanding and insights about the implementation of the changes in public health with a specific focus on health equity, the promotion of mental health and the prevention of harms of substance use.