1
|
Luna Pinzon A, Stronks K, Emke H, van den Eynde E, Altenburg T, Dijkstra SC, Renders CM, Hermans R, Busch V, Chinapaw MJM, Kremers SPJ, Waterlander W. Understanding the system dynamics of obesity-related behaviours in 10- to 14-year-old adolescents in Amsterdam from a multi-actor perspective. Front Public Health 2023; 11:1128316. [PMID: 37304107 PMCID: PMC10248031 DOI: 10.3389/fpubh.2023.1128316] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2023] [Accepted: 04/28/2023] [Indexed: 06/13/2023] Open
Abstract
Introduction and Methods To develop an understanding of the dynamics driving obesity-related behaviours in adolescents, we conducted systems-based analysis on a causal loop diagram (CLD) created from a multi-actor perspective, including academic researchers, adolescents and local stakeholders. Results The CLD contained 121 factors and 31 feedback loops. We identified six subsystems with their goals: (1) interaction between adolescents and the food environment, with profit maximisation as goal, (2) interaction between adolescents and the physical activity environment, with utility maximisation of outdoor spaces as goal, (3) interaction between adolescents and the online environment, with profit maximisation from technology use as goal, (4) interaction between adolescents, parenting and the wider socioeconomic environment, with a goal focused on individual parental responsibility, (5) interaction between healthcare professionals and families, with the goal resulting in treating obesity as an isolated problem, and (6) transition from childhood to adolescence, with the goal centring around adolescents' susceptibility to an environment that stimulates obesity-related behaviours. Discussion Analysis showed that inclusion of the researchers' and stakeholders' perspectives contributed to an understanding of how the system structure of an environment works. Integration of the adolescents' perspective enriched insights on how adolescents interact with that environment. The analysis further showed that the dynamics driving obesity-related behaviours are geared towards further reinforcing such behaviours.
Collapse
Affiliation(s)
- Angie Luna Pinzon
- Amsterdam UMC location University of Amsterdam, Department of Public and Occupational Health, Amsterdam, Netherlands
- Amsterdam Public Health Research Institute, Health Behaviors and Chronic Diseases, Amsterdam, Netherlands
| | - Karien Stronks
- Amsterdam UMC location University of Amsterdam, Department of Public and Occupational Health, Amsterdam, Netherlands
- Amsterdam Public Health Research Institute, Health Behaviors and Chronic Diseases, Amsterdam, Netherlands
| | - Helga Emke
- Amsterdam Public Health Research Institute, Health Behaviors and Chronic Diseases, Amsterdam, Netherlands
- Amsterdam UMC location Vrije Universiteit Amsterdam, Department of Public and Occupational Health, Amsterdam, Netherlands
- Department of Health Sciences, Faculty of Science, Vrije Universiteit Amsterdam, Amsterdam, Netherlands
| | - Emma van den Eynde
- Obesity Center CGG, Erasmus MC, University Medical Center Rotterdam, Rotterdam, Netherlands
| | - Teatske Altenburg
- Amsterdam Public Health Research Institute, Health Behaviors and Chronic Diseases, Amsterdam, Netherlands
- Amsterdam UMC location Vrije Universiteit Amsterdam, Department of Public and Occupational Health, Amsterdam, Netherlands
| | - S. Coosje Dijkstra
- Amsterdam Public Health Research Institute, Health Behaviors and Chronic Diseases, Amsterdam, Netherlands
- Department of Health Sciences, Faculty of Science, Vrije Universiteit Amsterdam, Amsterdam, Netherlands
| | - Carry M. Renders
- Amsterdam Public Health Research Institute, Health Behaviors and Chronic Diseases, Amsterdam, Netherlands
- Department of Health Sciences, Faculty of Science, Vrije Universiteit Amsterdam, Amsterdam, Netherlands
| | - Roel Hermans
- Department of Health Promotion, NUTRIM School of Nutrition and Translational Research in Metabolism, Maastricht University, Maastricht, Netherlands
| | - Vincent Busch
- Sarphati Amsterdam, Public Health Service (GGD), Amsterdam, Netherlands
| | - Mai J. M. Chinapaw
- Amsterdam Public Health Research Institute, Health Behaviors and Chronic Diseases, Amsterdam, Netherlands
- Amsterdam UMC location Vrije Universiteit Amsterdam, Department of Public and Occupational Health, Amsterdam, Netherlands
| | - Stef P. J. Kremers
- Department of Health Promotion, NUTRIM School of Nutrition and Translational Research in Metabolism, Maastricht University, Maastricht, Netherlands
| | - Wilma Waterlander
- Amsterdam UMC location University of Amsterdam, Department of Public and Occupational Health, Amsterdam, Netherlands
- Amsterdam Public Health Research Institute, Health Behaviors and Chronic Diseases, Amsterdam, Netherlands
| |
Collapse
|
2
|
Chapman S, Boodhoo A, Duffy C, Goodman S, Michalopoulou M. Theory of Change in Complex Research for Development Programmes: Challenges and Solutions from the Global Challenges Research Fund. THE EUROPEAN JOURNAL OF DEVELOPMENT RESEARCH 2023; 35:298-322. [PMID: 36987527 PMCID: PMC9974048 DOI: 10.1057/s41287-023-00574-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 12/13/2022] [Indexed: 06/03/2023]
Abstract
The United Kingdom Research and Innovation (UKRI) Global Challenges Research Fund (GCRF) aimed to address global challenges to achieve the United Nations (UN) Sustainable Development Goals through 12 interdisciplinary research hubs. This research documents key lessons learned around working with Theory of Change (ToC) to guide Monitoring, Evaluation and Learning (MEL) within these complex research for development hubs. Interviews and document reviews were conducted in ten of the research hubs. The results revealed that only one hub invested in an explicit visual system mapping approach, and that funder timelines, budget constraints and issues with capacity and expertise limited the application of these approaches across all hubs. In contrast, many hubs attempted to deal with visual complexity by means of ether constructing multiple, nested ToCs, or a conscious simplification of complexity through reducing their ToC towards a straightforward and uncomplicated chain model or spherical model. While the former approach had some value, most hubs struggled to find capacity to support the full articulation of nested ToCs. In contrast, the latter approach resulted in ToCs which lacked detail or mechanism articulation, but which nevertheless were often 'fit for purpose' in ensuring effective communication and coherence across diverse stakeholders and sub-projects. We conclude that in instances where the reporting, funding and management cycles of complex research for development programmes cannot be adapted to properly support learning-based approaches to ToC development, imposing simplicity in the ToC might be fit for purpose. This might also be preferable to more complex visual approaches that are only partially realised.
Collapse
Affiliation(s)
- Sarah Chapman
- School of Management Studies, University of Cape Town, Leslie Commerce Building, Upper Campus, Rondebosch, Cape Town, Western Cape South Africa
| | - Adiilah Boodhoo
- School of Management Studies, University of Cape Town, Leslie Commerce Building, Upper Campus, Rondebosch, Cape Town, Western Cape South Africa
| | - Carren Duffy
- School of Management Studies, University of Cape Town, Leslie Commerce Building, Upper Campus, Rondebosch, Cape Town, Western Cape South Africa
| | - Suki Goodman
- School of Management Studies, University of Cape Town, Leslie Commerce Building, Upper Campus, Rondebosch, Cape Town, Western Cape South Africa
| | - Maria Michalopoulou
- The UKRI GCRF Accelerating Achievement for Africa’s Adolescents Hub, Centre for Evidence-Based Intervention, Department of Social Policy & Intervention, University of Oxford, Oxford, UK
| |
Collapse
|
3
|
McGill E, Marks D, Petticrew M, Egan M. Addressing alcohol-related harms in the local night-time economy: a qualitative process evaluation from a complex systems perspective. BMJ Open 2022; 12:e050913. [PMID: 36008081 PMCID: PMC9422880 DOI: 10.1136/bmjopen-2021-050913] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/04/2022] Open
Abstract
OBJECTIVES English local authorities (LAs) are interested in reducing alcohol-related harms and may use discretionary powers such as the Late Night Levy (LNL) to do so. This study aims to describe how system stakeholders hypothesise the levy may generate changes and to explore how the system, its actors and the intervention adapt and co-evolve over time. DESIGN A process evaluation from a complex systems perspective, using qualitative methods. SETTING A London LA with high densities of residential and commercial properties, which implemented the LNL in 2014. PARTICIPANTS Data were generated through interviews with LNL implementers and alcohol consumers, observations in bars and during LNL patrols and documentary review. INTERVENTION The LNL allows LAs to charge late-night alcohol retailers an annual fee (£299-£4440) to manage and police the night-time economy (NTE). RESULTS When the LNL was being considered, stakeholders from different interest groups advanced diverse opinions about its likely impacts while rarely referencing supporting research evidence. Proponents of the levy argued it could reduce crime and anti-social behaviour by providing additional funds to police and manage the NTE. Critics of the levy hypothesised adverse consequences linked to claims that the intervention would force venues to vary their hours or close, cluster closing times, reduce NTE diversity and undermine public-private partnerships. In the first 2 years, levy-funded patrols developed relationships with the licensed trade and the public. The LNL did not undermine public-private partnerships and while some premises varied their hours, these changes did not undermine the intervention's viability, nor significantly cluster venue closing times, nor obviously damage the area's reputation for having a diverse NTE. CONCLUSIONS This study applies a framework for process evaluation from a complex systems perspective. The evaluation could be extended to measure alcohol-related outcomes and to consider the interplay between the national and local systems.
Collapse
Affiliation(s)
- Elizabeth McGill
- Department of Public Health, Environments and Society, London School of Hygiene & Tropical Medicine, London, UK
| | - Dalya Marks
- Department of Public Health, Environments and Society, London School of Hygiene & Tropical Medicine, London, UK
| | - Mark Petticrew
- Department of Public Health, Environments and Society, London School of Hygiene & Tropical Medicine, London, UK
| | - Matt Egan
- Department of Public Health, Environments and Society, London School of Hygiene & Tropical Medicine, London, UK
| |
Collapse
|
4
|
Nobles J, Fox C, Inman-Ward A, Beasley T, Redwood S, Jago R, Foster C. Navigating the river(s) of systems change: a multi-methods, qualitative evaluation exploring the implementation of a systems approach to physical activity in Gloucestershire, England. BMJ Open 2022; 12:e063638. [PMID: 35940842 PMCID: PMC9364398 DOI: 10.1136/bmjopen-2022-063638] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/04/2022] Open
Abstract
OBJECTIVES Systems approaches aim to change the environments in which people live, through cross-sectoral working, by harnessing the complexity of the problem. This paper sought to identify: (1) the strategies which support the implementation of We Can Move (WCM), (2) the barriers to implementation, (3) key contextual factors that influence implementation and (4) impacts associated with WCM. DESIGN A multi-methods evaluation of WCM was completed between April 2019 and April 2021. Ripple Effects Mapping (REM) and semi-structured interviewers were used. Framework and content analysis were systematically applied to the dataset. SETTING WCM-a physical activity orientated systems approach being implemented in Gloucestershire, England. PARTICIPANTS 31 stakeholder interviews and 25 stakeholders involved in 15 REM workshops. RESULTS A white-water rafting analogy was developed to present the main findings. The successful implementation of WCM required a facilitative, well-connected and knowledgeable guide (ie, the lead organisation), a crew (ie, wider stakeholders) who's vision and agenda aligned with WCM's purpose, and a flexible delivery approach that could respond to ever-changing nature of the river (ie, local and national circumstances). The context surrounding WCM further strengthened and hampered its implementation. Barriers included evaluative difficulties, a difference in stakeholder and organisational perspectives, misaligned expectations and understandings of WCM, and COVID-19 implications (COVID-19 also presented as a facilitative factor). WCM was said to strengthen cohesion and collaboration between partners, benefit other agendas and policies (eg, mental health, town planning, inequality), and improve physical activity opportunities and environments. CONCLUSIONS This paper is one of the first to evaluate a systems approach to increasing physical activity. We highlight key strategies and contextual factors that influenced the implementation of WCM and demonstrate some of the wider benefits from such approaches. Further research and methodologies are required to build the evidence base surrounding systems approaches in Public Health.
Collapse
Affiliation(s)
- James Nobles
- Population Health Sciences, University of Bristol, Bristol, UK
- National Institute for Health and Care Research Applied Research Collaboration West (NIHR ARC West), University Hospitals Bristol and Weston National Health Service Foundation Trust, Bristol, UK
| | - Charlotte Fox
- Royal Borough of Windsor and Maidenhead Council, Maidenhead, UK
| | | | | | - Sabi Redwood
- Population Health Sciences, University of Bristol, Bristol, UK
- National Institute for Health and Care Research Applied Research Collaboration West (NIHR ARC West), University Hospitals Bristol and Weston National Health Service Foundation Trust, Bristol, UK
| | - Russ Jago
- Population Health Sciences, University of Bristol, Bristol, UK
- National Institute for Health and Care Research Applied Research Collaboration West (NIHR ARC West), University Hospitals Bristol and Weston National Health Service Foundation Trust, Bristol, UK
- Centre for Exercise, Nutrition and Health Sciences, School for Policy Studies, University of Bristol, Bristol, UK
| | - Charlie Foster
- Centre for Exercise, Nutrition and Health Sciences, School for Policy Studies, University of Bristol, Bristol, UK
| |
Collapse
|
5
|
Lee GY, Hickie IB, Occhipinti JA, Song YJC, Skinner A, Camacho S, Lawson K, Hilber AM, Freebairn L. Presenting a comprehensive multi-scale evaluation framework for participatory modelling programs: A scoping review. PLoS One 2022; 17:e0266125. [PMID: 35452462 PMCID: PMC9032404 DOI: 10.1371/journal.pone.0266125] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2021] [Accepted: 03/15/2022] [Indexed: 11/18/2022] Open
Abstract
INTRODUCTION Systems modelling and simulation can improve understanding of complex systems to support decision making, better managing system challenges. Advances in technology have facilitated accessibility of modelling by diverse stakeholders, allowing them to engage with and contribute to the development of systems models (participatory modelling). However, despite its increasing applications across a range of disciplines, there is a growing need to improve evaluation efforts to effectively report on the quality, importance, and value of participatory modelling. This paper aims to identify and assess evaluation frameworks, criteria, and/or processes, as well as to synthesize the findings into a comprehensive multi-scale framework for participatory modelling programs. MATERIALS AND METHODS A scoping review approach was utilized, which involved a systematic literature search via Scopus in consultation with experts to identify and appraise records that described an evaluation framework, criteria, and/or process in the context of participatory modelling. This scoping review is registered with the Open Science Framework. RESULTS The review identified 11 studies, which varied in evaluation purposes, terminologies, levels of examination, and time points. The review of studies highlighted areas of overlap and opportunities for further development, which prompted the development of a comprehensive multi-scale evaluation framework to assess participatory modelling programs across disciplines and systems modelling methods. The framework consists of four categories (Feasibility, Value, Change/Action, Sustainability) with 30 evaluation criteria, broken down across project-, individual-, group- and system-level impacts. DISCUSSION & CONCLUSION The presented novel framework brings together a significant knowledge base into a flexible, cross-sectoral evaluation effort that considers the whole participatory modelling process. Developed through the rigorous synthesis of multidisciplinary expertise from existing studies, the application of the framework can provide the opportunity to understand practical future implications such as which aspects are particularly important for policy decisions, community learning, and the ongoing improvement of participatory modelling methods.
Collapse
Affiliation(s)
- Grace Yeeun Lee
- Brain and Mind Centre, The University of Sydney, Sydney, NSW, Australia
| | | | - Jo-An Occhipinti
- Brain and Mind Centre, The University of Sydney, Sydney, NSW, Australia
- Computer Simulation & Advanced Research Technologies (CSART), Sydney, NSW, Australia
| | | | - Adam Skinner
- Brain and Mind Centre, The University of Sydney, Sydney, NSW, Australia
| | - Salvador Camacho
- Swiss Centre for International Health, Swiss Tropical and Public Health Institute, Basel, Switzerland
- University of Basel, Basel, Switzerland
| | - Kenny Lawson
- Brain and Mind Centre, The University of Sydney, Sydney, NSW, Australia
| | - Adriane Martin Hilber
- Swiss Centre for International Health, Swiss Tropical and Public Health Institute, Basel, Switzerland
- University of Basel, Basel, Switzerland
| | - Louise Freebairn
- Brain and Mind Centre, The University of Sydney, Sydney, NSW, Australia
- Computer Simulation & Advanced Research Technologies (CSART), Sydney, NSW, Australia
- Research School of Population Health, The Australian National University, Canberra, ACT, Australia
| |
Collapse
|
6
|
Luna Pinzon A, Stronks K, Dijkstra C, Renders C, Altenburg T, den Hertog K, Kremers SPJ, Chinapaw MJM, Verhoeff AP, Waterlander W. The ENCOMPASS framework: a practical guide for the evaluation of public health programmes in complex adaptive systems. Int J Behav Nutr Phys Act 2022; 19:33. [PMID: 35346233 PMCID: PMC8962023 DOI: 10.1186/s12966-022-01267-3] [Citation(s) in RCA: 17] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2021] [Accepted: 02/17/2022] [Indexed: 11/10/2022] Open
Abstract
Background Systems thinking embraces the complexity of public health problems, including childhood overweight and obesity. It aids in understanding how factors are interrelated, and it can be targeted to produce favourable changes in a system. There is a growing call for systems approaches in public health research, yet limited practical guidance is available on how to evaluate public health programmes within complex adaptive systems. The aim of this paper is to present an evaluation framework that supports researchers in designing systems evaluations in a comprehensive and practical way. Methods We searched the literature for existing public health systems evaluation studies. Key characteristics on how to conduct a systems evaluation were extracted and compared across studies. Next, we overlaid the identified characteristics to the context of the Lifestyle Innovations Based on Youth Knowledge and Experience (LIKE) programme evaluation and analyzed which characteristics were essential to carry out the LIKE evaluation. This resulted in the Evaluation of Programmes in Complex Adaptive Systems (ENCOMPASS) framework. Results The ENCOMPASS framework includes five iterative stages: (1) adopting a system dynamics perspective on the overall evaluation design; (2) defining the system boundaries; (3) understanding the pre-existing system to inform system changes; (4) monitoring dynamic programme output at different system levels; and (5) measuring programme outcome and impact in terms of system changes. Conclusions The value of ENCOMPASS lies in the integration of key characteristics from existing systems evaluation studies, as well as in its practical, applied focus. It can be employed in evaluating public health programmes in complex adaptive systems. Furthermore, ENCOMPASS provides guidance for the entire evaluation process, all the way from understanding the system to developing actions to change it and to measuring system changes. By the nature of systems thinking, the ENCOMPASS framework will likely evolve further over time, as the field expands with more completed studies.
Collapse
|
7
|
Bleecker L, Sauveplane-Stirling V, Di Ruggiero E, Sellen D. Evaluating the integration of strategic priorities within a complex research-for-development funding program. EVALUATION AND PROGRAM PLANNING 2021; 89:102009. [PMID: 34562669 DOI: 10.1016/j.evalprogplan.2021.102009] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/11/2020] [Revised: 06/26/2021] [Accepted: 08/27/2021] [Indexed: 06/13/2023]
Abstract
This paper examines the application of Complexity Theory constructs to a research-for-development program evaluation and presents an overview of the implications and promising approaches for evaluating complex programs. We discuss lessons learned from an evaluation completed for the International Development Research Centre's Food, Environment and Health (FEH) program, which investigated the integration and outcomes of five strategic program priorities: partnerships, southern leadership, gender and equity, scale, and environmental sustainability. We present interpretations from a secondary, thematic content analysis that categorized evaluation findings across four complexity constructs: emergence, unpredictability, contradiction and self-organization. Viewing the evaluation through these constructs surfaced some important features of the FEH program to date, specifically its evolving approach, adaptiveness to emergent issues, non-linear outcomes, and self-organizing agents, which had several implications for the evaluative process. We conclude that the most appropriate evaluation designs for complex funding programs are participatory (to explore all stakeholders' influence), adaptive (to capture the unexpected) and assess external contexts. The application of complexity constructs may be useful for evaluators to gain a deeper understanding of how program contexts change in the face of complexity and why some evaluation methods work more effectively than others.
Collapse
|
8
|
Morgan-Trimmer S, Kudlicka A, Warmoth K, Leroi I, Oyebode JR, Pool J, Woods R, Clare L. Implementation processes in a cognitive rehabilitation intervention for people with dementia: a complexity-informed qualitative analysis. BMJ Open 2021; 11:e051255. [PMID: 34702730 PMCID: PMC8549661 DOI: 10.1136/bmjopen-2021-051255] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/03/2022] Open
Abstract
OBJECTIVES Healthcare is often delivered through complex interventions. Understanding how to implement these successfully is important for optimising services. This article demonstrates how the complexity theory concept of 'self-organisation' can inform implementation, drawing on a process evaluation within a randomised controlled trial of the GREAT (Goal-oriented cognitive Rehabilitation in Early-stage Alzheimer's and related dementias: a multi-centre single-blind randomised controlled Trial) intervention which compared a cognitive rehabilitation intervention for people with dementia with usual treatment. DESIGN A process evaluation examined experiences of GREAT therapists and participants receiving the intervention, through thematic analysis of a focus group with therapists and interviews with participants and their carers. Therapy records of participants receiving the intervention were also analysed using adapted framework analysis. Analysis adopted a critical realist perspective and a deductive-inductive approach to identify patterns in how the intervention operated. SETTING The GREAT intervention was delivered through home visits by therapists, in eight regions in the UK. PARTICIPANTS Six therapists took part in a focus group, interviews were conducted with 25 participants and 26 carers, and therapy logs for 50 participants were analysed. INTERVENTION A 16-week cognitive rehabilitation programme for people with mild-to-moderate dementia. RESULTS 'Self-organisation' of the intervention occurred through adaptations made by therapists. Adaptations included simplifying the intervention for people with greater cognitive impairment, and extending it to meet additional needs. Relational work by therapists produced an emergent outcome of 'social support'. Self-organised aspects of the intervention were less visible than formal components, but were important aspects of how it operated during the trial. This understanding can help to inform future implementation. CONCLUSIONS Researchers are increasingly adopting complexity theory to understand interventions. This study extends the application of complexity theory by demonstrating how 'self-organisation' was a useful concept for understanding aspects of the intervention that would have been missed by focusing on formal intervention components. Analysis of self-organisation could enhance future process evaluations and implementation studies. TRIAL REGISTRATION NUMBER ISRCTN21027481.
Collapse
Affiliation(s)
| | | | - Krystal Warmoth
- School of Health and Social Work & NIHR Applied Research Collaboration East of England, University of Hertfordshire, Hatfield, UK
| | - Iracema Leroi
- School of Medicine and Global Brain Health Institute, Trinity College Dublin, Dublin, Ireland
| | - Jan R Oyebode
- Centre for Applied Dementia Studies, University of Bradford, Bradford, UK
| | - Jackie Pool
- Dementia Pal Ltd, QCS Quality Compliance Systems, Guildford, UK
| | - Robert Woods
- Dementia Services Development Centre Wales, School of Health Sciences, Bangor University, Bangor, UK
| | - Linda Clare
- College of Medicine and Health & NIHR Applied Research Collaboration South-West Peninsula, University of Exeter, Exeter, UK
| |
Collapse
|
9
|
Hussey AJ, Sibbald SL, Ferrone M, Hergott A, McKelvie R, Faulds C, Roberts Z, Scarffe AD, Meyer MJ, Vollbrecht S, Licskai C. Confronting complexity and supporting transformation through health systems mapping: a case study. BMC Health Serv Res 2021; 21:1146. [PMID: 34688279 PMCID: PMC8540206 DOI: 10.1186/s12913-021-07168-8] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2021] [Accepted: 10/11/2021] [Indexed: 11/10/2022] Open
Abstract
INTRODUCTION Health systems are a complex web of interacting and interconnected parts; introducing an intervention, or the allocation of resources, in one sector can have effects across other sectors and impact the entire system. A prerequisite for effective health system reorganisation or transformation is a broad and common understanding of the current system amongst stakeholders and innovators. Chronic obstructive pulmonary disease (COPD) and heart failure (HF) are common chronic diseases with high health care costs that require an integrated health system to effectively treat. STUDY DESCRIPTION This case study documents the first phase of system transformation at a regional level in Ontario, Canada. In this first phase, visual representations of the health system in its current state were developed using a collaborative co-creation approach, and a focus on COPD and HF. Multiple methods were used including focus groups, open-ended questionnaires, and document review, to develop a series of graphical and visual representations; a health care ecosystem map. RESULTS The ecosystem map identified key sectoral components, inter-component interactions, and care requirements for patients with COPD and HF and inventoried current programs and services available to deliver this care. Main findings identified that independent system-wide navigation for this vulnerable patient group is limited, primary care is central to the accessibility of nearly half of the identified care elements, and resources are not equitably distributed. The health care ecosystem mapping helped to identify care gaps and illustrates the need to resource the primary care provider and the patient with system navigation resources and interdisciplinary team care. CONCLUSION The co-created health care ecosystem map brought a collective understanding of the health care system as it applies to COPD and HF. The map provides a blueprint that can be adapted to other disease states and health systems. Future transformation will build on this foundational work, continuing the robust interdisciplinary co-creation strategies, exploring predictive health system modelling and identifying areas for integration.
Collapse
Affiliation(s)
- Anna J Hussey
- Asthma Research Group Windsor-Essex County Inc., Windsor, ON, Canada
| | - Shannon L Sibbald
- Faculty of Health Sciences, Western University, London, ON, Canada
- Department of Family Medicine, Schulich School of Medicine and Dentistry, Western University, London, ON, Canada
| | - Madonna Ferrone
- Asthma Research Group Windsor-Essex County Inc., Windsor, ON, Canada
- Hotel-Dieu Grace Healthcare, Windsor, ON, Canada
| | - Alyson Hergott
- Asthma Research Group Windsor-Essex County Inc., Windsor, ON, Canada
| | - Robert McKelvie
- St. Joseph's Health Care, London, ON, Canada
- Cardiology, Schulich School of Medicine and Dentistry, Western University, London, ON, Canada
| | - Cathy Faulds
- Department of Family Medicine, Schulich School of Medicine and Dentistry, Western University, London, ON, Canada
- St. Joseph's Health Care, London, ON, Canada
| | - Zofe Roberts
- Asthma Research Group Windsor-Essex County Inc., Windsor, ON, Canada
| | - Andrew D Scarffe
- Telfer School of Management, University of Ottawa, Ottawa, ON, Canada
| | - Matthew J Meyer
- London Health Sciences Centre, London, ON, Canada
- Department of Epidemiology and Biostatistics and Interfaculty Program in Public Health, Schulich School of Medicine and Dentistry, Western University, London, ON, Canada
- Ivey Business School, London, ON, Canada
| | | | - Christopher Licskai
- Asthma Research Group Windsor-Essex County Inc., Windsor, ON, Canada.
- London Health Sciences Centre, London, ON, Canada.
- Lawson Health Research Institute, London, ON, Canada.
- Respirology, Schulich School of Medicine and Dentistry, Western University, London, ON, Canada.
| |
Collapse
|
10
|
Matheson A, Walton M, Gray R, Wehipeihana N, Wistow J. Strengthening prevention in communities through systems change: lessons from the evaluation of Healthy Families NZ. Health Promot Int 2021; 35:947-957. [PMID: 31495884 DOI: 10.1093/heapro/daz092] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022] Open
Abstract
This article presents the findings from the first 3 years of the evaluation of Healthy Families NZ, a systems-change intervention to prevent chronic diseases in 10 communities. The initiative, which builds on existing prevention activities, aims to strengthen the health prevention system through evidence-driven action to enable people to make good food choices, be physically active, smoke-free and free from alcohol-related harm. Key investment areas are a dedicated systems thinking and acting health promotion workforce, and activating leaders who can influence transformational change. The evaluation to date has found the initiative is being implemented with integrity. Evidence indicates a shift towards greater action on prevention, and the prevention system being strengthened. Māori ownership has been enabled, and prioritizing equity has led teams to utilize methods that amplify diverse local perspectives. There is progress on developing a flexible workforce through adaptive learning, flexible resources, professional development and a responsive National team. There is also progress in activating local leadership and empowering local teams. The initiative design has explicitly taken into account the context of complexity within which it is being implemented. It has evolved to focus on action that can accelerate sharing information and practices within communities, and between policy and decision-makers. Healthy Families NZ and its evaluation have been refunded to 2022. This provides an important opportunity to gather further insight into effective ways to strengthen the community agency and trust needed to promote and deliver evidence-based action on prevention.
Collapse
Affiliation(s)
- Anna Matheson
- School of Health, Victoria University of Wellington, New Zealand
| | - Mat Walton
- Institute for Environmental Science and Research (ESR), New Zealand
| | - Rebecca Gray
- School of Health, Victoria University of Wellington, New Zealand
| | | | | |
Collapse
|
11
|
McGill E, Er V, Penney T, Egan M, White M, Meier P, Whitehead M, Lock K, Anderson de Cuevas R, Smith R, Savona N, Rutter H, Marks D, de Vocht F, Cummins S, Popay J, Petticrew M. Evaluation of public health interventions from a complex systems perspective: A research methods review. Soc Sci Med 2021; 272:113697. [PMID: 33508655 DOI: 10.1016/j.socscimed.2021.113697] [Citation(s) in RCA: 74] [Impact Index Per Article: 24.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Revised: 08/27/2020] [Accepted: 01/07/2021] [Indexed: 02/09/2023]
Abstract
INTRODUCTION Applying a complex systems perspective to public health evaluation may increase the relevance and strength of evidence to improve health and reduce health inequalities. In this review of methods, we aimed to: (i) classify and describe different complex systems methods in evaluation applied to public health; and (ii) examine the kinds of evaluative evidence generated by these different methods. METHODS We adapted critical review methods to identify evaluations of public health interventions that used systems methods. We conducted expert consultation, searched electronic databases (Scopus, MEDLINE, Web of Science), and followed citations of relevant systematic reviews. Evaluations were included if they self-identified as using systems- or complexity-informed methods and if they evaluated existing or hypothetical public health interventions. Case studies were selected to illustrate different types of complex systems evaluation. FINDINGS Seventy-four unique studies met our inclusion criteria. A framework was developed to map the included studies onto different stages of the evaluation process, which parallels the planning, delivery, assessment, and further delivery phases of the interventions they seek to inform; these stages include: 1) theorising; 2) prediction (simulation); 3) process evaluation; 4) impact evaluation; and 5) further prediction (simulation). Within this framework, we broadly categorised methodological approaches as mapping, modelling, network analysis and 'system framing' (the application of a complex systems perspective to a range of study designs). Studies frequently applied more than one type of systems method. CONCLUSIONS A range of complex systems methods can be utilised, adapted, or combined to produce different types of evaluative evidence. Further methodological innovation in systems evaluation may generate stronger evidence to improve health and reduce health inequalities in our complex world.
Collapse
Affiliation(s)
- Elizabeth McGill
- Department of Health Services, Research and Policy, London School of Hygiene & Tropical Medicine, London, United Kingdom.
| | - Vanessa Er
- Department of Health Services, Research and Policy, London School of Hygiene & Tropical Medicine, London, United Kingdom
| | - Tarra Penney
- MRC Epidemiology Unit, Centre for Diet and Activity Research (CEDAR) and University of Cambridge, Cambridge, United Kingdom
| | - Matt Egan
- Department of Public Health, Environments and Society, London School of Hygiene & Tropical Medicine, London; United Kingdom
| | - Martin White
- MRC Epidemiology Unit, Centre for Diet and Activity Research (CEDAR) and University of Cambridge, Cambridge, United Kingdom
| | - Petra Meier
- Public Health, School of Health and Related Research (ScHARR), University of Sheffield, Sheffield, United Kingdom
| | - Margaret Whitehead
- Department of Public Health, Policy and Systems, University of Liverpool, Liverpool, United Kingdom
| | - Karen Lock
- University of Exeter Medical School, Exeter, United Kingdom
| | | | - Richard Smith
- University of Exeter Medical School, Exeter, United Kingdom
| | - Natalie Savona
- Department of Health Services, Research and Policy, London School of Hygiene & Tropical Medicine, London, United Kingdom
| | - Harry Rutter
- Department of Social & Policy Sciences, University of Bath, Bath, United Kingdom
| | - Dalya Marks
- Department of Public Health, Environments and Society, London School of Hygiene & Tropical Medicine, London; United Kingdom
| | - Frank de Vocht
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, United Kingdom
| | - Steven Cummins
- Department of Public Health, Environments and Society, London School of Hygiene & Tropical Medicine, London; United Kingdom
| | - Jennie Popay
- Division of Health Research, Faculty of Health and Medicine, Lancaster University, Lancaster, United Kingdom
| | - Mark Petticrew
- Department of Public Health, Environments and Society, London School of Hygiene & Tropical Medicine, London; United Kingdom
| |
Collapse
|
12
|
Sun W, Dedahanov AT, Shin HY, Li WP. Using extended complexity theory to test SMEs' adoption of Blockchain-based loan system. PLoS One 2021; 16:e0245964. [PMID: 33544745 PMCID: PMC7864469 DOI: 10.1371/journal.pone.0245964] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2020] [Accepted: 01/11/2021] [Indexed: 12/03/2022] Open
Abstract
Blockchain-based loan system can be summed up as: information exchange between various government departments; information exchange between enterprises and various financial institutions; detection of the actual use of loans in the form of encrypted currency. This technology is supposed to reduce a lot of financing costs for SMEs on average. Therefore, this research extends complexity theory to discover the factors that affect the use of Blockchain loan systems by SMEs. Complexity, perceived risk, perceived fairness and reward sensitivity prove to have significant effects on usage intention. Complexity proves to have moderating effects on other relationships. This research may contribute to the system performance improvement and provide opportunities for SMEs to share information with financial institutions or individuals around the world, thereby providing investors with equal opportunities for competition.
Collapse
Affiliation(s)
- Wei Sun
- School of Economic, Anyang Normal University, Anyang City, Henan Province, China
- * E-mail:
| | | | - Ho Young Shin
- School of Business, Yeungnam University, Gyeongsan, South Korea
| | | |
Collapse
|
13
|
Qualitative process evaluation from a complex systems perspective: A systematic review and framework for public health evaluators. PLoS Med 2020; 17:e1003368. [PMID: 33137099 PMCID: PMC7605618 DOI: 10.1371/journal.pmed.1003368] [Citation(s) in RCA: 39] [Impact Index Per Article: 9.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/13/2020] [Accepted: 09/11/2020] [Indexed: 11/19/2022] Open
Abstract
BACKGROUND Public health evaluation methods have been criticized for being overly reductionist and failing to generate suitable evidence for public health decision-making. A "complex systems approach" has been advocated to account for real world complexity. Qualitative methods may be well suited to understanding change in complex social environments, but guidance on applying a complex systems approach to inform qualitative research remains limited and underdeveloped. This systematic review aims to analyze published examples of process evaluations that utilize qualitative methods that involve a complex systems perspective and proposes a framework for qualitative complex system process evaluations. METHODS AND FINDINGS We conducted a systematic search to identify complex system process evaluations that involve qualitative methods by searching electronic databases from January 1, 2014-September 30, 2019 (Scopus, MEDLINE, Web of Science), citation searching, and expert consultations. Process evaluations were included if they self-identified as taking a systems- or complexity-oriented approach, integrated qualitative methods, reported empirical findings, and evaluated public health interventions. Two reviewers independently assessed each study to identify concepts associated with the systems thinking and complexity science traditions. Twenty-one unique studies were identified evaluating a wide range of public health interventions in, for example, urban planning, sexual health, violence prevention, substance use, and community transformation. Evaluations were conducted in settings such as schools, workplaces, and neighborhoods in 13 different countries (9 high-income and 4 middle-income). All reported some utilization of complex systems concepts in the analysis of qualitative data. In 14 evaluations, the consideration of complex systems influenced intervention design, evaluation planning, or fieldwork. The identified studies used systems concepts to depict and describe a system at one point in time. Only 4 evaluations explicitly utilized a range of complexity concepts to assess changes within the system resulting from, or co-occurring with, intervention implementation over time. Limitations to our approach are including only English-language papers, reliance on study authors reporting their utilization of complex systems concepts, and subjective judgment from the reviewers relating to which concepts featured in each study. CONCLUSION This study found no consensus on what bringing a complex systems perspective to public health process evaluations with qualitative methods looks like in practice and that many studies of this nature describe static systems at a single time point. We suggest future studies use a 2-phase framework for qualitative process evaluations that seek to assess changes over time from a complex systems perspective. The first phase involves producing a description of the system and identifying hypotheses about how the system may change in response to the intervention. The second phase involves following the pathway of emergent findings in an adaptive evaluation approach.
Collapse
|
14
|
Lancaster K, Rhodes T. What prevents health policy being 'evidence-based'? New ways to think about evidence, policy and interventions in health. Br Med Bull 2020; 135:38-49. [PMID: 32897357 DOI: 10.1093/bmb/ldaa026] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/29/2019] [Revised: 07/06/2020] [Accepted: 07/21/2020] [Indexed: 12/14/2022]
Abstract
BACKGROUND Evidence-based policy decision-making is a dominant paradigm in health but realizing this ideal has proven challenging. SOURCES OF DATA This paper conceptually maps health policy, policy studies and social science literature critically engaged with evidence and decision-making. No new data were generated or analysed in support of this review. AREAS OF AGREEMENT Barriers to evidence-based policy have been documented, with efforts made to increase the uptake of evidence. AREAS OF CONTROVERSY Evident complexities have been regarded as a problem of translation. However, this assumes that policy-making is a process of authoritative choice, and that 'evidence' is inherently valuable policy knowledge, which has been critiqued. GROWING POINTS Alternative accounts urge consideration of how evidence comes to bear on decisions made within complex systems, and what counts as evidence. AREAS TIMELY FOR DEVELOPING RESEARCH An 'evidence-making intervention' approach offers a framework for conceptualizing how evidence and interventions are made relationally in practices, thus working with the politics and contingencies of implementation and policy-making.
Collapse
Affiliation(s)
- Kari Lancaster
- Centre for Social Research in Health, University of New South Wales, NSW 2052, Sydney, Australia
| | - Tim Rhodes
- Centre for Social Research in Health, University of New South Wales, NSW 2052, Sydney, Australia.,London School of Hygiene and Tropical Medicine, 15-17 Tavistock Place, Kings Cross, London WC1H 9SH, UK
| |
Collapse
|
15
|
Makleff S, Billowitz M, Garduño J, Cruz M, Silva Márquez VI, Marston C. Applying a complex adaptive systems approach to the evaluation of a school-based intervention for intimate partner violence prevention in Mexico. Health Policy Plan 2020; 35:993-1002. [PMID: 32761146 PMCID: PMC7553757 DOI: 10.1093/heapol/czaa067] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/21/2020] [Indexed: 11/13/2022] Open
Abstract
Despite calls for evaluation practice to take a complex systems approach, there are few examples of how to incorporate complexity into real-life evaluations. This article presents the case for using a complex systems approach to evaluate a school-based intimate partner violence-prevention intervention. We conducted a post hoc analysis of qualitative evaluation data to examine the intervention as a potential system disruptor. We analysed data in relation to complexity concepts particularly relevant to schools: 'diverse and dynamic agents', 'interaction', 'unpredictability', 'emergence' and 'context dependency'. The data-two focus groups with facilitators and 33 repeat interviews with 14-17-year-old students-came from an evaluation of a comprehensive sexuality education intervention in Mexico City, which serves as a case study for this analysis. The findings demonstrate an application of complex adaptive systems concepts to qualitative evaluation data. We provide examples of how this approach can shed light on the ways in which interpersonal interactions, group dynamics, the core messages of the course and context influenced the implementation and outcomes of this intervention. This gender-transformative intervention appeared to disrupt pervasive gender norms and reshape beliefs about how to engage in relationships. An intervention comprises multiple dynamic and interacting elements, all of which are unlikely to be consistent across implementation settings. Applying complexity concepts to our analysis added value by helping reframe implementation-related data to focus on how the 'social' aspects of complexity influenced the intervention. Without examining both individual and group processes, evaluations may miss key insights about how the intervention generates change, for whom, and how it interacts with its context. A social complex adaptive systems approach is well-suited to the evaluation of gender-transformative interventions and can help identify how such interventions disrupt the complex social systems in which they are implemented to address intractable societal problems.
Collapse
Affiliation(s)
- Shelly Makleff
- Faculty of Public Health and Policy, London School of Hygiene and Tropical Medicine, 15-17 Tavistock Place, London WC1H 9SH, UK
| | - Marissa Billowitz
- Independent, Juárez 208, Col. Tlalpan Centro, 14000 Mexico City, Mexico
| | - Jovita Garduño
- Fundación Mexicana para la Planeación Familiar, A.C. (Mexfam), Juárez 208, Col. Tlalpan Centro, 14000 Mexico City, Mexico
| | - Mariana Cruz
- IPPF/WHR Mexico, Juárez 208, Col. Tlalpan Centro, 14000 Mexico City, Mexico
| | - Vanessa Ivon Silva Márquez
- Fundación Mexicana para la Planeación Familiar, A.C. (Mexfam), Juárez 208, Col. Tlalpan Centro, 14000 Mexico City, Mexico
| | - Cicely Marston
- Faculty of Public Health and Policy, London School of Hygiene and Tropical Medicine, 15-17 Tavistock Place, London WC1H 9SH, UK
| |
Collapse
|
16
|
Waterlander WE, Luna Pinzon A, Verhoeff A, den Hertog K, Altenburg T, Dijkstra C, Halberstadt J, Hermans R, Renders C, Seidell J, Singh A, Anselma M, Busch V, Emke H, van den Eynde E, van Houtum L, Nusselder WJ, Overman M, van de Vlasakker S, Vrijkotte T, van den Akker E, Kremers SPJ, Chinapaw M, Stronks K. A System Dynamics and Participatory Action Research Approach to Promote Healthy Living and a Healthy Weight among 10-14-Year-Old Adolescents in Amsterdam: The LIKE Programme. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2020; 17:ijerph17144928. [PMID: 32650571 PMCID: PMC7400640 DOI: 10.3390/ijerph17144928] [Citation(s) in RCA: 27] [Impact Index Per Article: 6.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/09/2020] [Revised: 07/02/2020] [Accepted: 07/04/2020] [Indexed: 11/25/2022]
Abstract
This paper describes the design of the LIKE programme, which aims to tackle the complex problem of childhood overweight and obesity in 10–14-year-old adolescents using a systems dynamics and participatory approach. The LIKE programme focuses on the transition period from 10-years-old to teenager and was implemented in collaboration with the Amsterdam Healthy Weight Programme (AHWP) in Amsterdam-East, the Netherlands. The aim is to develop, implement and evaluate an integrated action programme at the levels of family, school, neighbourhood, health care and city. Following the principles of Participatory Action Research (PAR), we worked with our population and societal stakeholders as co-creators. Applying a system lens, we first obtained a dynamic picture of the pre-existing systems that shape adolescents’ behaviour relating to diet, physical activity, sleep and screen use. The subsequent action programme development was dynamic and adaptive, including quick actions focusing on system elements (quick evaluating, adapting and possibly catalysing further action) and more long-term actions focusing on system goals and/or paradigm change. The programme is supported by a developmental systems evaluation and the Intervention Level Framework, supplemented with routinely collected data on weight status and health behaviour change over a period of five years. In the coming years, we will report how this approach has worked to provide a robust understanding of the programme’s effectiveness within a complex dynamic system. In the meantime, we hope our study design serves as a source of inspiration for other public health intervention studies in complex systems.
Collapse
Affiliation(s)
- Wilma E. Waterlander
- Department of Public and Occupational Health, Amsterdam Public Health Research Institute, Amsterdam UMC, University of Amsterdam, Meibergdreef 9, 1105 AZ Amsterdam, The Netherlands; (A.L.P.); (T.V.); (K.S.)
- Correspondence:
| | - Angie Luna Pinzon
- Department of Public and Occupational Health, Amsterdam Public Health Research Institute, Amsterdam UMC, University of Amsterdam, Meibergdreef 9, 1105 AZ Amsterdam, The Netherlands; (A.L.P.); (T.V.); (K.S.)
| | - Arnoud Verhoeff
- Public Health Service Amsterdam, Sarphati Amsterdam, 1018 WT Amsterdam, The Netherlands; (A.V.); (V.B.); (L.v.H.); (S.v.d.V.)
- Department of Sociology, University of Amsterdam, 1018 WV Amsterdam, The Netherlands
| | - Karen den Hertog
- Amsterdam Healthy Weight Programme, Public Health Service Amsterdam, 1018 WT Amsterdam, The Netherlands;
| | - Teatske Altenburg
- Department of Public and Occupational Health, Amsterdam Public Health Research Institute, Amsterdam UMC, Vrije Universiteit Amsterdam, Van der Boechorststraat 7, NL-1081 BT Amsterdam, The Netherlands; (T.A.); (A.S.); (M.A.); (H.E.); (M.C.)
| | - Coosje Dijkstra
- Department of Health Sciences, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, 1081 HV Amsterdam, The Netherlands; (C.D.); (J.H.); (C.R.); (J.S.)
| | - Jutka Halberstadt
- Department of Health Sciences, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, 1081 HV Amsterdam, The Netherlands; (C.D.); (J.H.); (C.R.); (J.S.)
| | - Roel Hermans
- Department of Health Promotion, NUTRIM School of Nutrition and Translational Research in Metabolism, Maastricht University, 6229 ER Maastricht, The Netherlands; (R.H.); (M.O.); (S.P.J.K.)
| | - Carry Renders
- Department of Health Sciences, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, 1081 HV Amsterdam, The Netherlands; (C.D.); (J.H.); (C.R.); (J.S.)
| | - Jacob Seidell
- Department of Health Sciences, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, 1081 HV Amsterdam, The Netherlands; (C.D.); (J.H.); (C.R.); (J.S.)
| | - Amika Singh
- Department of Public and Occupational Health, Amsterdam Public Health Research Institute, Amsterdam UMC, Vrije Universiteit Amsterdam, Van der Boechorststraat 7, NL-1081 BT Amsterdam, The Netherlands; (T.A.); (A.S.); (M.A.); (H.E.); (M.C.)
| | - Manou Anselma
- Department of Public and Occupational Health, Amsterdam Public Health Research Institute, Amsterdam UMC, Vrije Universiteit Amsterdam, Van der Boechorststraat 7, NL-1081 BT Amsterdam, The Netherlands; (T.A.); (A.S.); (M.A.); (H.E.); (M.C.)
| | - Vincent Busch
- Public Health Service Amsterdam, Sarphati Amsterdam, 1018 WT Amsterdam, The Netherlands; (A.V.); (V.B.); (L.v.H.); (S.v.d.V.)
| | - Helga Emke
- Department of Public and Occupational Health, Amsterdam Public Health Research Institute, Amsterdam UMC, Vrije Universiteit Amsterdam, Van der Boechorststraat 7, NL-1081 BT Amsterdam, The Netherlands; (T.A.); (A.S.); (M.A.); (H.E.); (M.C.)
| | - Emma van den Eynde
- Department of Pediatric Endocrinology, Erasmus MC, University Medical Center, 3025 GD Rotterdam, The Netherlands; (E.v.d.E.); (E.v.d.A.)
| | - Lieke van Houtum
- Public Health Service Amsterdam, Sarphati Amsterdam, 1018 WT Amsterdam, The Netherlands; (A.V.); (V.B.); (L.v.H.); (S.v.d.V.)
| | - Wilma J. Nusselder
- Department of Public Health, Erasmus MC, University Medical Center Rotterdam, 3015 CN Rotterdam, The Netherlands;
| | - Meredith Overman
- Department of Health Promotion, NUTRIM School of Nutrition and Translational Research in Metabolism, Maastricht University, 6229 ER Maastricht, The Netherlands; (R.H.); (M.O.); (S.P.J.K.)
| | - Susan van de Vlasakker
- Public Health Service Amsterdam, Sarphati Amsterdam, 1018 WT Amsterdam, The Netherlands; (A.V.); (V.B.); (L.v.H.); (S.v.d.V.)
| | - Tanja Vrijkotte
- Department of Public and Occupational Health, Amsterdam Public Health Research Institute, Amsterdam UMC, University of Amsterdam, Meibergdreef 9, 1105 AZ Amsterdam, The Netherlands; (A.L.P.); (T.V.); (K.S.)
| | - Erica van den Akker
- Department of Pediatric Endocrinology, Erasmus MC, University Medical Center, 3025 GD Rotterdam, The Netherlands; (E.v.d.E.); (E.v.d.A.)
| | - Stef P. J. Kremers
- Department of Health Promotion, NUTRIM School of Nutrition and Translational Research in Metabolism, Maastricht University, 6229 ER Maastricht, The Netherlands; (R.H.); (M.O.); (S.P.J.K.)
| | - Mai Chinapaw
- Department of Public and Occupational Health, Amsterdam Public Health Research Institute, Amsterdam UMC, Vrije Universiteit Amsterdam, Van der Boechorststraat 7, NL-1081 BT Amsterdam, The Netherlands; (T.A.); (A.S.); (M.A.); (H.E.); (M.C.)
| | - Karien Stronks
- Department of Public and Occupational Health, Amsterdam Public Health Research Institute, Amsterdam UMC, University of Amsterdam, Meibergdreef 9, 1105 AZ Amsterdam, The Netherlands; (A.L.P.); (T.V.); (K.S.)
| |
Collapse
|
17
|
Ridde V, Pérez D, Robert E. Using implementation science theories and frameworks in global health. BMJ Glob Health 2020; 5:e002269. [PMID: 32377405 PMCID: PMC7199704 DOI: 10.1136/bmjgh-2019-002269] [Citation(s) in RCA: 53] [Impact Index Per Article: 13.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2019] [Revised: 03/27/2020] [Accepted: 03/27/2020] [Indexed: 11/20/2022] Open
Abstract
In global health, researchers and decision makers, many of whom have medical, epidemiology or biostatistics background, are increasingly interested in evaluating the implementation of health interventions. Implementation science, particularly for the study of public policies, has existed since at least the 1930s. This science makes compelling use of explicit theories and analytic frameworks that ensure research quality and rigour. Our objective is to inform researchers and decision makers who are not familiar with this research branch about these theories and analytic frameworks. We define four models of causation used in implementation science: intervention theory, frameworks, middle-range theory and grand theory. We then explain how scientists apply these models for three main implementation studies: fidelity assessment, process evaluation and complex evaluation. For each study, we provide concrete examples from research in Cuba and Africa to better understand the implementation of health interventions in global health context. Global health researchers and decision makers with a quantitative background will not become implementation scientists after reading this article. However, we believe they will be more aware of the need for rigorous implementation evaluations of global health interventions, alongside impact evaluations, and in collaboration with social scientists.
Collapse
Affiliation(s)
- Valéry Ridde
- CEPED, IRD (French Institute for Research on sustainable Development), Université de Paris, ERL INSERM SAGESUD, Paris, France
| | - Dennis Pérez
- Epidemiology Division, Pedro Kouri Tropical Medicine Institute (IPK), Havana, Cuba
| | - Emilie Robert
- ICARES and Centre de recherche SHERPA (Institut Universitaire au regard des communautés ethnoculturelles, CIUSSS du Centre-Ouest-de-l'Île-de-Montréal), Montreal, Quebec, Canada
| |
Collapse
|
18
|
Matheson A. Health Inequality as a Large-Scale Outcome of Complex Social Systems: Lessons for Action on the Sustainable Development Goals. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2020; 17:E2684. [PMID: 32295177 PMCID: PMC7215954 DOI: 10.3390/ijerph17082684] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Received: 03/13/2020] [Revised: 03/30/2020] [Accepted: 04/09/2020] [Indexed: 11/26/2022]
Abstract
Action on the Sustainable Development Goals (SDGs) needs to become real and impactful, taking a "whole systems" perspective on levers for systems change. This article reviews what we have learned over the past century about the large-scale outcome of health inequality, and what we know about the behaviour of complex social systems. This combined knowledge provides lessons on the nature of inequality and what effective action on our big goals, like the SDGs, might look like. It argues that economic theories and positivist social theories which have dominated the last 150 years have largely excluded the nature of human connections to each other, and the environment. This exclusion of intimacy has legitimatised arguments that only value-free economic processes matter for macro human systems, and only abstract measurement constitutes valuable social science. Theories of complex systems provide an alternative perspective. One where health inequality is viewed as emergent, and causes are systemic and compounding. Action therefore needs to be intensely local, with power relationships key to transformation. This requires conscious and difficult intervention on the intolerable accumulation of resources; improved reciprocity between social groups; and reversal of system flows, which at present ebb away from the local and those already disadvantaged.
Collapse
Affiliation(s)
- Anna Matheson
- School of Health, Te Herenga Waka, Victoria University of Wellington, Wellington 6140, New Zealand;
- Te Pūnaha Matatini, Centre for Complex Systems, Auckland 1142, New Zealand
| |
Collapse
|
19
|
Laycock A, Bailie J, Matthews V, Bailie R. Using developmental evaluation to support knowledge translation: reflections from a large-scale quality improvement project in Indigenous primary healthcare. Health Res Policy Syst 2019; 17:70. [PMID: 31324251 PMCID: PMC6642555 DOI: 10.1186/s12961-019-0474-6] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2019] [Accepted: 07/02/2019] [Indexed: 12/17/2022] Open
Abstract
BACKGROUND Developmental evaluation is a growing area of evaluation practice, advocated for informing the adaptive development of change initiatives in complex social environments. The utilisation focus, complexity perspective and systems thinking of developmental evaluation suggest suitability for evaluating knowledge translation initiatives in primary healthcare. However, there are few examples in the literature to guide its use in these contexts and in Indigenous settings. In this paper, we reflect on our experience of using developmental evaluation to implement a large-scale knowledge translation research project in Australian Aboriginal and Torres Strait Islander primary healthcare. Drawing on principles of knowledge translation and key features of developmental evaluation, we debate the key benefits and challenges of applying this approach to engage diverse stakeholders in using aggregated quality improvement data to identify and address persistent gaps in care delivery. DISCUSSION The developmental evaluation enabled the team to respond to stakeholder feedback and apply learning in real-time to successfully refine theory-informed research and engagement processes, tailor the presentation of findings to stakeholders and context, and support the project's dissemination and knowledge co-production aim. It thereby contributed to the production of robust, useable research findings for informing policy and system change. The use of developmental evaluation appeared to positively influence stakeholders' use of the project reports and their responses to the findings. Challenges included managing a high volume of evaluation data and multiple evaluation purposes, balancing facilitative sense-making processes and change with task-focused project management, and lack of experience in using this evaluation approach. Use of an embedded evaluator with facilitation skills and background knowledge of the project helped to overcome these challenges, as did similarities observed between features of developmental evaluation and continuous quality improvement. CONCLUSION Our experience of developmental evaluation confirmed our expectations of the potential value of this approach for strengthening improvement interventions and implementation research, and particularly for adapting healthcare innovations in Indigenous settings. In our project, developmental evaluation successfully encompassed evaluation, project adaptation, capacity development and knowledge translation. Further work is warranted to apply this approach more widely to improve primary healthcare initiatives and outcomes, and to evaluate implementation research.
Collapse
Affiliation(s)
- Alison Laycock
- Menzies School of Health Research, Charles Darwin University, PO Box 41096, Casuarina, Darwin, NT, 0811, Australia. .,The University of Sydney, University Centre for Rural Health, 61 Uralba Street, Lismore, NSW, 2480, Australia.
| | - Jodie Bailie
- The University of Sydney, University Centre for Rural Health, 61 Uralba Street, Lismore, NSW, 2480, Australia
| | - Veronica Matthews
- The University of Sydney, University Centre for Rural Health, 61 Uralba Street, Lismore, NSW, 2480, Australia
| | - Ross Bailie
- The University of Sydney, University Centre for Rural Health, 61 Uralba Street, Lismore, NSW, 2480, Australia
| |
Collapse
|
20
|
Moore T, Bakken LL, Wallace B. The importance of systems thinking, context, and inclusion when studying the feasibility of expanding the FoodShare (SNAP) outreach program to rural communities of Northern Wisconsin. EVALUATION AND PROGRAM PLANNING 2019; 74:10-17. [PMID: 30784727 DOI: 10.1016/j.evalprogplan.2019.02.003] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/16/2018] [Revised: 01/27/2019] [Accepted: 02/05/2019] [Indexed: 06/09/2023]
Abstract
Evaluators often neglect to question whether a program has been appropriately designed for a new context prior to a feasibility study's commencement. In this paper, we document the results of a case study that closely examined context when determining the feasibility of implementing a FoodShare outreach program in rural northern Wisconsin. Using community-based participatory mixed methods, we examine how stakeholder engagement led to both study refinement and a more comprehensive understanding of community food security based on contextual factors and systems thinking. Lessons learned when conducting feasibility studies from this perspective are provided to improve evaluation practice.
Collapse
Affiliation(s)
- Travis Moore
- Department of Civil Society and Community Studies, University of Wisconsin-Madison, United States
| | - Lori L Bakken
- Department of Civil Society and Community Studies, University of Wisconsin-Madison, United States.
| | - Bakari Wallace
- Department of Civil Society and Community Studies, University of Wisconsin-Madison, United States
| |
Collapse
|
21
|
Pelucha M, Kveton V, Potluka O. Using mixed method approach in measuring effects of training in firms: Case study of the European Social Fund support. EVALUATION AND PROGRAM PLANNING 2019; 73:146-155. [PMID: 30639832 DOI: 10.1016/j.evalprogplan.2018.12.008] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/25/2018] [Revised: 08/20/2018] [Accepted: 12/24/2018] [Indexed: 06/09/2023]
Abstract
Public support of training in firms corresponds to the long-term importance of the quality of human capital in the competitiveness of firms and nations. Thus, the EU supports such training via the European Social Fund (ESF). The evaluation community evaluates the support by using either qualitative or quantitative methods. The simultaneous application of these two approaches is rare. The purpose of this paper is to combine quantitative (counterfactual impact evaluation) and qualitative (qualitative comparative analysis) methods in order to fill the methodological gap. Based on the combination of both approaches, it explores their strengths, complementarity and disadvantages to evaluate public support for employee training in the Czech Republic. The combination of methods makes it possible to identify not only the impacts but also their causes. Linking the ESF support to corporate competitiveness is crucial for demonstrating the effectiveness of public spending.
Collapse
Affiliation(s)
- Martin Pelucha
- University of Economics, Prague, Faculty of Economics, Department of Regional studies, nam. W. Churchilla 4, 130 67, Praha 3, Czech Republic.
| | - Viktor Kveton
- Charles University in Prague, Faculty of Science, Department of Social Geography and Regional Development, Albertov 6, 128 43, Praha 2, Czech Republic
| | - Oto Potluka
- University of Basel, Center for Philanthropy Studies (CEPS), Steinengraben 22, 4051, Basel, Switzerland
| |
Collapse
|
22
|
Crane M, Bauman A, Lloyd B, McGill B, Rissel C, Grunseit A. Applying pragmatic approaches to complex program evaluation: A case study of implementation of the New South Wales Get Healthy at Work program. Health Promot J Austr 2019; 30:422-432. [PMID: 30860630 DOI: 10.1002/hpja.239] [Citation(s) in RCA: 20] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2018] [Accepted: 03/06/2019] [Indexed: 11/09/2022] Open
Abstract
ISSUE ADDRESSED Complex health promotion programs, which can have multilevels of implementation and multi-components with nonlinear causal pathways, present many evaluation challenges. Traditional evaluation methods often fail to account for the complexity inherent in assessing these programs. In real-world settings, evaluations of complex programs are often beset by additional constraints of limited budgets and short timeframes. Determining whether a complex program is successful and how a program worked requires evaluators of complex programs to adopt a level of pragmatism. METHODS This paper describes a pragmatic evaluation approach used to evaluate the Get Healthy at Work workplace health promotion program, implemented in New South Wales, Australia. Using the program as a case study, we describe some key principles for applying a pragmatic evaluation approach and use these principles to develop an appropriate evaluation strategy. RESULTS The evaluation includes multiple research methods to assess program outputs and implementation; and identify emergent program impacts, within constrained resources. The evaluation was guided by epistemological flexibility, methodological comprehensiveness and operational practicality. CONCLUSION Health promotion programs, such as state-wide obesity prevention programs, require appropriate evaluation methods which address their inherent complexity amidst the real-world evaluation constraints, and focuses on the essential evaluation needs. SO WHAT The main complex program evaluation principles are applicable to other multilevel health promotion programs, challenged by methodological and practical or political constraints.
Collapse
Affiliation(s)
- Melanie Crane
- The University of Sydney, Sydney School of Public Health, Prevention Research Collaboration, Charles Perkins Centre, Camperdown, NSW, Australia.,The Australian Prevention Partnership Centre, Camperdown, NSW, Australia
| | - Adrian Bauman
- The University of Sydney, Sydney School of Public Health, Prevention Research Collaboration, Charles Perkins Centre, Camperdown, NSW, Australia.,The Australian Prevention Partnership Centre, Camperdown, NSW, Australia
| | - Beverley Lloyd
- NSW Office of Preventive Health, Liverpool Hospital, Liverpool, NSW, Australia
| | - Bronwyn McGill
- The University of Sydney, Sydney School of Public Health, Prevention Research Collaboration, Charles Perkins Centre, Camperdown, NSW, Australia.,The Australian Prevention Partnership Centre, Camperdown, NSW, Australia
| | - Chris Rissel
- The University of Sydney, Sydney School of Public Health, Prevention Research Collaboration, Charles Perkins Centre, Camperdown, NSW, Australia.,NSW Office of Preventive Health, Liverpool Hospital, Liverpool, NSW, Australia
| | - Anne Grunseit
- The University of Sydney, Sydney School of Public Health, Prevention Research Collaboration, Charles Perkins Centre, Camperdown, NSW, Australia.,The Australian Prevention Partnership Centre, Camperdown, NSW, Australia
| |
Collapse
|
23
|
Pitt R, Wyborn C, Page G, Hutton J, Sawmy MV, Ryan M, Gallagher L. Wrestling with the complexity of evaluation for organizations at the boundary of science, policy, and practice. CONSERVATION BIOLOGY : THE JOURNAL OF THE SOCIETY FOR CONSERVATION BIOLOGY 2018; 32:998-1006. [PMID: 29660170 DOI: 10.1111/cobi.13118] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/15/2017] [Revised: 03/12/2018] [Accepted: 04/06/2018] [Indexed: 06/08/2023]
Abstract
Boundary organizations are situated between science, policy, and practice and have a goal of supporting communication and collaboration among these sectors. They have been promoted as a way to improve the effectiveness of conservation efforts by building stronger relationships between scientists, policy makers, industry, and practitioners (Cook et al. 2013). Although their promise has been discussed in theory, the work of and expectations for boundary organizations are less defined in practice. Biodiversity conservation is characterized by complexity, uncertainty, dissent, and tight budgets, so boundary organizations face the challenging task of demonstrating their value to diverse stakeholders. We examined the challenges boundary organizations face when seeking to evaluate their work and thus aimed to encourage more productive conversations about evaluation of boundary organizations and their projects. Although no off-the-shelf solution is available for a given boundary organization, we identified 4 principles that will support effective evaluation for boundary organizations: engage diverse stakeholders, support learning and reflection, assess contribution to change, and align evaluation with assumption and values.
Collapse
Affiliation(s)
- R Pitt
- University of Hawai'i Mānoa, 1960 East-West Road, Honolulu, HI 96848, U.S.A
| | - C Wyborn
- Luc Hoffmann Institute, WWF International, 1196 Gland, Switzerland
| | - G Page
- SustainaMetrix, 502 Deering Avenue, Portland, ME 04103, U.S.A
| | - J Hutton
- Luc Hoffmann Institute, WWF International, 1196 Gland, Switzerland
| | - M Virah Sawmy
- Luc Hoffmann Institute, WWF International, 1196 Gland, Switzerland
| | - M Ryan
- Luc Hoffmann Institute, WWF International, 1196 Gland, Switzerland
| | - L Gallagher
- Luc Hoffmann Institute, WWF International, 1196 Gland, Switzerland
| |
Collapse
|
24
|
Abstract
This article explores the potential of complex adaptive systems (CAS) theory to inform behaviour change research. A CAS describes a collection of heterogeneous agents interacting within a particular context, adapting to each other's actions. In practical terms, this implies that behaviour change is (1) socially and culturally situated; (2) highly sensitive to small baseline differences in individuals, groups, and intervention components; and (3) determined by multiple components interacting 'chaotically'. Two approaches to studying CAS are briefly reviewed. Agent-based modelling is a computer simulation technique that allows researchers to investigate 'what if' questions in a virtual environment. Applied qualitative research techniques, on the other hand, offer a way to examine what happens when an intervention is pursued in real-time, and to identify the sorts of rules and assumptions governing social action. Although these represent very different approaches to complexity, there may be scope for mixing these methods - for example, by grounding models in insights derived from qualitative fieldwork. Finally, I will argue that the concept of CAS offers one opportunity to gain a deepened understanding of health-related practices, and to examine the social psychological processes that produce health-promoting or damaging actions.
Collapse
Affiliation(s)
- Tim Gomersall
- a Department of Psychology , University of Huddersfield , Huddersfield , UK
| |
Collapse
|
25
|
Brown G, Reeders D, Cogle A, Madden A, Kim J, O'Donnell D. A Systems Thinking Approach to Understanding and Demonstrating the Role of Peer-Led Programs and Leadership in the Response to HIV and Hepatitis C: Findings From the W3 Project. Front Public Health 2018; 6:231. [PMID: 30234083 PMCID: PMC6127267 DOI: 10.3389/fpubh.2018.00231] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2018] [Accepted: 07/30/2018] [Indexed: 12/20/2022] Open
Abstract
The central role of community and peer-led programs has been a key characteristic of the Australian partnership response to HIV and hepatitis C since the beginning of the epidemics. Despite this, peer-led programs continue to have limited capacity to demonstrate their role and value as part of a multi-sectoral response. What makes one peer-led program a better investment than another? What role does the rest of the sector have in ensuring we gain the most value from these investments? To investigate this, we facilitated interactive systems thinking methods with 10 programs working within communities of people who inject drugs, gay men, sex workers and people living with HIV across Australia. This included articulating program theories in diagram and textual form to help us understand the role of peer-based programs promoting peer leadership within the Australian HIV and hepatitis C responses. Our aim was to develop a framework for monitoring and evaluation that could be applied to peer led programs at different levels and in different contexts. We found that for peer-led programs to fulfill their role, and to navigate the rapid changes occurring in the both epidemics, they need to: demonstrate the credibility of their peer and community insights; continually adapt to changing contexts and policy priorities in tandem with their communities; and maintain influence in both community and policy systems. We developed a framework of four key functions (Engagement, Alignment, Adaptation, and Influence) which peer-based programs need to demonstrate, which form the basis for identifying quality indicators. This article presents a new way of framing and monitoring investments in peer-led programs and peer eadership actions by these programs. If health policy is committed to strengthening the leadership shown by affected communities, then we need to understand, enhance, monitor and value the role of peer-led programs and peer leadership within the overall prevention system. We believe the W3 framework, drawing on systems thinking and modeling, can support funders, policy-makers and programs to achieve this.
Collapse
Affiliation(s)
- Graham Brown
- Australian Research Centre in Sex, Health and Society, La Trobe University, Melbourne, VIC, Australia
| | - Daniel Reeders
- School of Regulation and Global Governance, Australian National University, Canberra, ACT, Australia
| | - Aaron Cogle
- National Association of People Living With HIV Australia, Sydney, NSW, Australia
| | - Annie Madden
- The Australian Injecting and Illicit Drug Users League, Canberra, ACT, Australia
| | - Jules Kim
- Scarlet Alliance, Australian Sex Worker Association, Sydney, NSW, Australia
| | - Darryl O'Donnell
- Australian Federation of AIDS Organisations, Sydney, NSW, Australia
| |
Collapse
|
26
|
Long KM, McDermott F, Meadows GN. Being pragmatic about healthcare complexity: our experiences applying complexity theory and pragmatism to health services research. BMC Med 2018; 16:94. [PMID: 29921277 PMCID: PMC6008915 DOI: 10.1186/s12916-018-1087-6] [Citation(s) in RCA: 56] [Impact Index Per Article: 9.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/31/2018] [Accepted: 05/25/2018] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND The healthcare system has proved a challenging environment for innovation, especially in the area of health services management and research. This is often attributed to the complexity of the healthcare sector, characterized by intersecting biological, social and political systems spread across geographically disparate areas. To help make sense of this complexity, researchers are turning towards new methods and frameworks, including simulation modeling and complexity theory. DISCUSSION Herein, we describe our experiences implementing and evaluating a health services innovation in the form of simulation modeling. We explore the strengths and limitations of complexity theory in evaluating health service interventions, using our experiences as examples. We then argue for the potential of pragmatism as an epistemic foundation for the methodological pluralism currently found in complexity research. We discuss the similarities between complexity theory and pragmatism, and close by revisiting our experiences putting pragmatic complexity theory into practice. CONCLUSION We found the commonalities between pragmatism and complexity theory to be striking. These included a sensitivity to research context, a focus on applied research, and the valuing of different forms of knowledge. We found that, in practice, a pragmatic complexity theory approach provided more flexibility to respond to the rapidly changing context of health services implementation and evaluation. However, this approach requires a redefinition of implementation success, away from pre-determined outcomes and process fidelity, to one that embraces the continual learning, evolution, and emergence that characterized our project.
Collapse
Affiliation(s)
- Katrina M Long
- Southern Synergy, Department of Psychiatry, School of Clinical Sciences at Monash Health, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, VIC, Australia.
| | - Fiona McDermott
- Department of Social Work, School of Primary and Allied Health Care, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, VIC, Australia
| | - Graham N Meadows
- Southern Synergy, Department of Psychiatry, School of Clinical Sciences at Monash Health, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, VIC, Australia.,Melbourne School of Population and Global Health, University of Melbourne, Melbourne, VIC, Australia.,Monash Health, Melbourne, VIC, Australia
| |
Collapse
|
27
|
Lorthios-Guilledroit A, Richard L, Filiatrault J. Factors associated with the implementation of community-based peer-led health promotion programs: A scoping review. EVALUATION AND PROGRAM PLANNING 2018; 68:19-33. [PMID: 29459228 DOI: 10.1016/j.evalprogplan.2018.01.008] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/12/2017] [Revised: 09/28/2017] [Accepted: 01/30/2018] [Indexed: 06/08/2023]
Abstract
Peer education is growing in popularity as a useful health promotion strategy. However, optimal conditions for implementing peer-led health promotion programs (HPPs) remain unclear. This scoping review aimed to describe factors that can influence implementation of peer-led HPPs targeting adult populations. Five databases were searched using the keywords "health promotion/prevention", "implementation", "peers", and related terms. Studies were included if they reported at least one factor associated with the implementation of community-based peer-led HPPs. Fifty-five studies were selected for the analysis. The method known as "best fit framework synthesis" was used to analyze the factors identified in the selected papers. Many factors included in existing implementation conceptual frameworks were deemed applicable to peer-led HPPs. However, other factors related to individuals, programs, and implementation context also emerged from the analysis. Based on this synthesis, an adapted theoretical framework was elaborated, grounded in a complex adaptive system perspective and specifying potential mechanisms through which factors may influence implementation of community-based peer-led HPPs. Further research is needed to test the theoretical framework against empirical data. Findings from this scoping review increase our knowledge of the optimal conditions for implementing peer-led HPPs and thereby maximizing the benefits of such programs.
Collapse
Affiliation(s)
- Agathe Lorthios-Guilledroit
- School of Public Health, Université de Montréal, P.O. Box 6128, Centre-Ville Station, Montreal, Quebec, H3C 3J7, Canada; Université de Montréal Public Health Research Institute, 7101 du Parc Avenue, P.O. Box 6128, Centre-Ville Station, Montreal, Quebec, H3C 3J7, Canada; Centre de recherche, Institut universitaire de gériatrie de Montréal, 4545 Queen Mary Road, Montreal, Quebec, H3W 1W4, Canada.
| | - Lucie Richard
- Université de Montréal Public Health Research Institute, 7101 du Parc Avenue, P.O. Box 6128, Centre-Ville Station, Montreal, Quebec, H3C 3J7, Canada; Faculty of Nursing Sciences, Pavillon Marguerite-d'Youville, Université de Montréal, P.O. Box 6128, Centre-ville Station, Montreal, Quebec, H3C 3J7, Canada.
| | - Johanne Filiatrault
- Centre de recherche, Institut universitaire de gériatrie de Montréal, 4545 Queen Mary Road, Montreal, Quebec, H3W 1W4, Canada; School of Rehabilitation, Faculty of Medicine Université de Montréal, P.O. Box 6128, Centre-ville Station, Montreal, Quebec, H3C 3J7, Canada.
| |
Collapse
|
28
|
Rosas S, Knight E. Evaluating a complex health promotion intervention: case application of three systems methods. CRITICAL PUBLIC HEALTH 2018. [DOI: 10.1080/09581596.2018.1455966] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/16/2022]
Affiliation(s)
| | - Erin Knight
- School of Public Policy & Administration, Center for Community Research & Service, University of Delaware, Newark, DE, USA
| |
Collapse
|
29
|
Jorm C, Roberts C. Using Complexity Theory to Guide Medical School Evaluations. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2018; 93:399-405. [PMID: 28678103 DOI: 10.1097/acm.0000000000001828] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/07/2023]
Abstract
Contemporary medical school evaluations are narrow in focus and often do not consider the wider systems implications of the relationship between learning and teaching, research, clinical care, and community engagement. The result is graduates who lack the necessary knowledge and skills for the modern health care system and an educational system that is limited in its ability to learn and change.To address this issue, the authors apply complexity theory to medical school evaluation, using four key factors-nesting, diversity, self-organization, and emergent outcomes. To help medical educators apply this evaluation approach in their own settings, the authors offer two tools-a modified program logic model and sensemaking. In sensemaking, they use the organic metaphor of the medical school as a neuron situated within a complex neural network to enable medical educators to reframe the way they think about program evaluation. The authors then offer practical guidance for applying this model, including describing the example of addressing graduates' engagement in the health care system. The authors consider the input of teachers, the role of culture and curriculum, and the clinical care system in this example.Medical school evaluation is reframed as an improvement science for complex social interventions (medical school is such an intervention) in this model. With complexity theory's focus on emergent outcomes, evaluation takes on a new focus, reimagining medical students as reaching their future potential as change agents, who transform health systems and the lives of patients.
Collapse
Affiliation(s)
- Christine Jorm
- C. Jorm is honorary associate professor, Sydney Medical School, Sydney, Australia. C. Roberts is associate professor in primary care and medical education, Sydney Medical School, Sydney, Australia
| | | |
Collapse
|
30
|
Matheson A, Walton M, Gray R, Lindberg K, Shanthakumar M, Fyfe C, Wehipeihana N, Borman B. Evaluating a community-based public health intervention using a complex systems approach. J Public Health (Oxf) 2017; 40:606-613. [DOI: 10.1093/pubmed/fdx117] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2017] [Indexed: 11/14/2022] Open
Affiliation(s)
- Anna Matheson
- School of Public Health, Massey University, Wallace Street, Mt Cook, Wellington, New Zealand
| | - Mat Walton
- School of Public Health, Massey University, Wallace Street, Mt Cook, Wellington, New Zealand
| | - Rebecca Gray
- School of Public Health, Massey University, Wallace Street, Mt Cook, Wellington, New Zealand
| | - Kirstin Lindberg
- Environmental Health Indicators Programme, Centre for Public Health Research, Massey University, Wallace Street, Mt Cook, Wellington, New Zealand
| | - Mathu Shanthakumar
- Environmental Health Indicators Programme, Centre for Public Health Research, Massey University, Wallace Street, Mt Cook, Wellington, New Zealand
| | - Caroline Fyfe
- Environmental Health Indicators Programme, Centre for Public Health Research, Massey University, Wallace Street, Mt Cook, Wellington, New Zealand
| | | | - Barry Borman
- Environmental Health Indicators Programme, Centre for Public Health Research, Massey University, Wallace Street, Mt Cook, Wellington, New Zealand
| |
Collapse
|
31
|
Gates EF. Making sense of the emerging conversation in evaluation about systems thinking and complexity science. EVALUATION AND PROGRAM PLANNING 2016; 59:62-73. [PMID: 27591941 DOI: 10.1016/j.evalprogplan.2016.08.004] [Citation(s) in RCA: 22] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/24/2015] [Revised: 08/02/2016] [Accepted: 08/13/2016] [Indexed: 06/06/2023]
Abstract
In the last twenty years, a conversation has emerged in the evaluation field about the potential of systems thinking and complexity science (STCS) to transform the practice of evaluating social interventions. Documenting and interpreting this conversation are necessary to advance our understanding of the significance of using STCS in planning, implementing, and evaluating social interventions. Guided by a generic framework for evaluation practice, this paper reports on an inter-disciplinary literature review and argues that STCS raises some new ways of thinking about and carrying out the following six activities: 1) supporting social problem solving; 2) framing interventions and contexts; 3) selecting and using methods; 4) engaging in valuing; 5) producing and justifying knowledge; and 6) facilitating use. Following a discussion of these issues, future directions for research and practice are suggested.
Collapse
Affiliation(s)
- Emily F Gates
- University of Illinois at Urbana-Champaign, College of Education, 1310 South Sixth Street, Champaign, IL 61820, USA.
| |
Collapse
|
32
|
Carey G, Malbon E, Carey N, Joyce A, Crammond B, Carey A. Systems science and systems thinking for public health: a systematic review of the field. BMJ Open 2015; 5:e009002. [PMID: 26719314 PMCID: PMC4710830 DOI: 10.1136/bmjopen-2015-009002] [Citation(s) in RCA: 191] [Impact Index Per Article: 21.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/06/2015] [Revised: 10/23/2015] [Accepted: 11/11/2015] [Indexed: 12/03/2022] Open
Abstract
OBJECTIVES This paper reports on findings from a systematic review designed to investigate the state of systems science research in public health. The objectives were to: (1) explore how systems methodologies are being applied within public health and (2) identify fruitful areas of activity. DESIGN A systematic review was conducted from existing literature that draws on or uses systems science (in its various forms) and relates to key public health areas of action and concern, including tobacco, alcohol, obesity and the social determinants of health. DATA ANALYSIS 117 articles were included in the review. An inductive qualitative content analysis was used for data extraction. The following were systematically extracted from the articles: approach, methodology, transparency, strengths and weaknesses. These were then organised according to theme (ie, commonalities between studies within each category), in order to provide an overview of the state of the field as a whole. The assessment of data quality was intrinsic to the goals of the review itself, and therefore, was carried out as part of the analysis. RESULTS 4 categories of research were identified from the review, ranging from editorial and commentary pieces to complex system dynamic modelling. Our analysis of each of these categories of research highlighted areas of potential for systems science to strengthen public health efforts, while also revealing a number of limitations in the dynamic systems modelling being carried out in public health. CONCLUSIONS There is a great deal of interest in how the application of systems concepts and approach might aid public health. Our analysis suggests that soft systems modelling techniques are likely to be the most useful addition to public health, and align well with current debate around knowledge transfer and policy. However, the full range of systems methodologies is yet to be engaged with by public health researchers.
Collapse
Affiliation(s)
- Gemma Carey
- Regulatory Institutions Network Australian National University, Canberra, Australia
| | - Eleanor Malbon
- The Australian Prevention Partnership Centre, Sax Institute, Sydney, Australia
| | - Nicole Carey
- Self-organizing Systems Research Group School of engineering and applied sciences Harvard University
| | - Andrew Joyce
- Centre for Social Impact, Swinburne University, Melbourne, Victoria, Australia
| | - Brad Crammond
- Centre for Epidemiology and Preventive Health. Monash University, Melbourne, Australia
| | - Alan Carey
- Maths Science Institute Australian National University
| |
Collapse
|
33
|
Investigating combination HIV prevention: isolated interventions or complex system. J Int AIDS Soc 2015; 18:20499. [PMID: 26673880 PMCID: PMC4680918 DOI: 10.7448/ias.18.1.20499] [Citation(s) in RCA: 28] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2015] [Revised: 10/08/2015] [Accepted: 11/19/2015] [Indexed: 12/16/2022] Open
Abstract
Introduction Treatment as prevention has mobilized new opportunities in preventing HIV transmission and has led to bold new UNAIDS targets in testing, treatment coverage and transmission reduction. These will require not only an increase in investment but also a deeper understanding of the dynamics of combining behavioural, biomedical and structural HIV prevention interventions. High-income countries are making substantial investments in combination HIV prevention, but is this investment leading to a deeper understanding of how to combine interventions? The combining of interventions involves complexity, with many strategies interacting with non-linear and multiplying rather than additive effects. Discussion Drawing on a recent scoping study of the published research evidence in HIV prevention in high-income countries, this paper argues that there is a gap between the evidence currently available and the evidence needed to guide the achieving of these bold targets. The emphasis of HIV prevention intervention research continues to look at one intervention at a time in isolation from its interactions with other interventions, the community and the socio-political context of their implementation. To understand and evaluate the role of a combination of interventions, we need to understand not only what works, but in what circumstances, what role the parts need to play in their relationship with each other, when the combination needs to adapt and identify emergent effects of any resulting synergies. There is little development of evidence-based indicators on how interventions in combination should achieve that strategic advantage and synergy. This commentary discusses the implications of this ongoing situation for future research and the required investment in partnership. We suggest that systems science approaches, which are being increasingly applied in other areas of public health, could provide an expanded vocabulary and analytic tools for understanding these complex interactions, relationships and emergent effects. Conclusions Relying on the current linear but disconnected approaches to intervention research and evidence we will miss the potential to achieve and understand system-level synergies. Given the challenges in sustaining public health and HIV prevention investment, meeting the bold UNAIDS targets that have been set is likely to be dependent on achieving systems level synergies.
Collapse
|
34
|
Nelson G, Macnaughton E, Goering P. What qualitative research can contribute to a randomized controlled trial of a complex community intervention. Contemp Clin Trials 2015; 45:377-384. [PMID: 26498405 DOI: 10.1016/j.cct.2015.10.007] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2015] [Revised: 10/16/2015] [Accepted: 10/17/2015] [Indexed: 10/22/2022]
Abstract
Using the case of a large-scale, multi-site Canadian Housing First research demonstration project for homeless people with mental illness, At Home/Chez Soi, we illustrate the value of qualitative methods in a randomized controlled trial (RCT) of a complex community intervention. We argue that quantitative RCT research can neither capture the complexity nor tell the full story of a complex community intervention. We conceptualize complex community interventions as having multiple phases and dimensions that require both RCT and qualitative research components. Rather than assume that qualitative research and RCTs are incommensurate, a more pragmatic mixed methods approach was used, which included using both qualitative and quantitative methods to understand program implementation and outcomes. At the same time, qualitative research was used to examine aspects of the intervention that could not be understood through the RCT, such as its conception, planning, sustainability, and policy impacts. Through this example, we show how qualitative research can tell a more complete story about complex community interventions.
Collapse
Affiliation(s)
- Geoffrey Nelson
- Department of Psychology, Wilfrid Laurier University, 75 University Avenue West, Waterloo, ON N2L 3C5, Canada.
| | - Eric Macnaughton
- Department of Psychology, Wilfrid Laurier University, 75 University Avenue West, Waterloo, ON N2L 3C5, Canada.
| | - Paula Goering
- Centre for Addiction and Mental Health and University of Toronto, 33 Russell Street, 4th Floor Tower T427, Toronto, ON M5S 2S1, Canada.
| |
Collapse
|