Decision makers' experience of participatory dynamic simulation modelling: methods for public health policy

An inaugural forum on epidemiological modeling for public health stakeholders in Arizona

Epidemiological models-which help us understand and forecast the spread of infectious disease-can be valuable tools for public health. However, barriers exist that can make it difficult to employ epidemiological models routinely within the repertoire of public health planning. These barriers include technical challenges associated with constructing the models, challenges in obtaining appropriate data for model parameterization, and problems with clear communication of modeling outputs and uncertainty. To learn about the unique barriers and opportunities within the state of Arizona, we gathered a diverse set of 48 public health stakeholders for a day-and-a-half forum. Our research group was motivated specifically by our work building software for public health-relevant modeling and by our earnest desire to collaborate closely with stakeholders to ensure that our software tools are practical and useful in the face of evolving public health needs. Here we outline the planning and structure of the forum, and we highlight as a case study some of the lessons learned from breakout discussions. While unique barriers exist for implementing modeling for public health, there is also keen interest in doing so across diverse sectors of State and Local government, although issues of equal and fair access to modeling knowledge and technologies remain key issues for future development. We found this forum to be useful for building relationships and informing our software development, and we plan to continue such meetings annually to create a continual feedback loop between academic molders and public health practitioners.

Impact of an innovative bundled payment to TB health care providers in China: an economic simulation analysis

BACKGROUND

Tuberculosis is the second most deadly infectious disease after COVID-19 and the 13th leading cause of death worldwide. Among the 30 countries with a high burden of TB, China ranks third in the estimated number of TB cases. China is in the top four of 75 countries with a deficit in funding for TB strategic plans. To reduce costs and improve the effectiveness of TB treatment in China, the NHSA developed an innovative BP method. This study aimed to simulate the effects of this payment approach on different stakeholders, reduce the economic burden on TB patients, improve the quality of medical services, facilitate policy optimization, and offer a model for health care payment reforms that can be referenced by other regions throughout the world.

METHODS

We developed a simulation model based on a decision tree analysis to project the expected effects of the payment method on the potential financial impacts on different stakeholders. Our analysis mainly focused on comparing changes in health care costs before and after receiving BPs for TB patients with Medicare in the pilot areas. The data that were used for the analysis included the TB service claim records for 2019-2021 from the health insurance agency, TB prevalence data from the local Centre for Disease Control, and health care facilities' revenue and expenditure data from the Statistic Yearbook. A Monte Carlo randomized simulation model was used to estimate the results.

RESULTS

After adopting the innovative BP method, for each TB patient per year, the total annual expenditure was estimated to decrease from $2,523.28 to $2,088.89, which is a reduction of $434.39 (17.22%). The TB patient out-of-pocket expenditure was expected to decrease from $1,249.02 to $1,034.00, which is a reduction of $215.02 (17.22%). The health care provider's revenue decreased from $2,523.28 to $2,308.26, but the health care provider/institution's revenue-expenditure ratio increased from -6.09% to 9.50%.

CONCLUSIONS

This study highlights the potential of BPs to improve medical outcomes and control the costs associated with TB treatment. It demonstrates its feasibility and advantages in enhancing the coordination and sustainability of medical services, thus offering valuable insights for global health care payment reform.

Racial disparities in stroke incidence in the Women's Health Initiative: Exploring biological, behavioral, psychosocial, and social risk factors

Background

- Disparities in incident stroke risk among women by race and ethnicity persist. Few studies report the distribution and association of stroke risk factors by age group among a diverse sample of women.

Methods

- Data from the Women's Health Initiative (WHI) Observational Study collected between 1993 and 2010 were used to calculate cumulative stroke incidence and incidence rates among non-Hispanic African American (NHAA), non-Hispanic white (NHW), and Hispanic white or African American (HWAA) women by age group in participants aged ≥50 years at baseline (N = 77,247). Hazard ratios (HRs) and 95% CIs for biological, behavioral, psychosocial, and socioeconomic factors overall and by race or ethnicity were estimated using sequential Cox proportional hazard regression models.

Results

- Average follow-up time was 11.52 (SD, 3.48) years. The incident stroke rate was higher among NHAA (306 per 100,000 person-years) compared to NHW (279/100,000py) and HWAA women (147/100,000py) overall and in each age group. The disparity was largest at ages >75 years. The association between stroke risk factors (e.g., smoking, BMI, physical activity) and incident stroke varied across race and ethnicity groups. Higher social support was significantly associated with decreased stroke risk overall (HR:0.84, 95% CI, 0.76, 0.93); the degree of protection varied across race and ethnicity groups. Socioeconomic factors did not contribute additional stroke risk beyond risk conferred by traditional and psychosocial factors.

Conclusions

- The distribution and association of stroke risk factors differed between NHAA and NHW women. There is a clear need for stroke prevention strategies that address factors driving racial disparities in stroke risk.

A competency framework on simulation modelling-supported decision-making for Master of Public Health graduates

BACKGROUND

Simulation models are increasingly important for supporting decision-making in public health. However, due to lack of training, many public health professionals remain unfamiliar with constructing simulation models and using their outputs for decision-making. This study contributes to filling this gap by developing a competency framework on simulation model-supported decision-making targeting Master of Public Health education.

METHODS

The study combined a literature review, a two-stage online Delphi survey and an online consensus workshop. A draft competency framework was developed based on 28 peer-reviewed publications. A two-stage online Delphi survey involving 15 experts was conducted to refine the framework. Finally, an online consensus workshop, including six experts, evaluated the competency framework and discussed its implementation.

RESULTS

The competency framework identified 20 competencies related to stakeholder engagement, problem definition, evidence identification, participatory system mapping, model creation and calibration and the interpretation and dissemination of model results. The expert evaluation recommended differentiating professional profiles and levels of expertise and synergizing with existing course contents to support its implementation.

CONCLUSIONS

The competency framework developed in this study is instrumental to including simulation model-supported decision-making in public health training. Future research is required to differentiate expertise levels and develop implementation strategies.

Understanding and Bridging Gaps in the Use of Evidence from Modeling for Evidence-Based Policy Making in Nigeria's Health System

Background. Modeled evidence is a proven useful tool for decision makers in making evidence-based policies and plans that will ensure the best possible health system outcomes. Thus, we sought to understand constraints to the use of models in making decisions in Nigeria's health system and how such constraints can be addressed. Method. We adopted a mixed-methods study for the research and relied on the evidence to policy and Knowledge-to-Action (KTA) frameworks to guide the conceptualization of the study. An online survey was administered to 34 key individuals in health organizations that recognize modeling, which was followed by in-depth interviews with 24 of the 34 key informants. Analysis was done using descriptive analytic methods and thematic arrangements of narratives. Results. Overall, the data revealed poor use of modeled evidence in decision making within the health sector, despite reporting that modeled evidence and modelers are available in Nigeria. However, the disease control agency in Nigeria was reported to be an exception. The complexity of models was a top concern. Thus, suggestions were made to improve communication of models in ways that are easily comprehensible and to improve overall research culture within Nigeria's health sector. Conclusion. Modeled evidence plays a crucial role in evidence-based health decisions. Therefore, it is imperative to strengthen and sustain in-country capacity to value, produce, interpret, and use modeled evidence for decision making in health. To overcome limitations in the usage of modeled evidence, decision makers, modelers/researchers, and knowledge brokers should forge viable relationships that regard and promote evidence translation.

Highlights

Despite the use of modeling by Nigeria's disease control agency in containing the COVID-19 pandemic, modeling remains poorly used in the country's overall health sector.Although policy makers recognize the importance of evidence in making decisions, there are still pertinent concerns about the poor research culture of policy-making institutions and communication gaps that exist between researchers/modelers and policy makers.Nigeria's health system can be strengthened by improving the value and usage of scientific evidence generation through conscious efforts to institutionalize research culture in the health sector and bridge gaps between researchers/modelers and decision makers.

Conceptualising the value of simulation modelling for public engagement with policy: a critical literature review

As we face complex and dynamically changing public health and environmental challenges, simulation modelling has come to occupy an increasingly central role in public engagements with policy. Shifts are occurring not only in terms of wider public understandings of modelling, but also in how the value of modelling is conceptualised within scientific modelling communities. We undertook a critical literature review to synthesise the underlying epistemic, theoretical and methodological assumptions about the role and value of simulation modelling within the literature across a range of fields (e.g., health, social science and environmental management) that engage with participatory modelling approaches. We identified four cross-cutting narrative conceptualisations of the value of modelling across different research traditions: (1) models simulate and help solve complex problems; (2) models as tools for community engagement; (3) models as tools for consensus building; (4) models as volatile technologies that generate social effects. Exploring how these ideas of 'value' overlap and what they offer one another has implications for how participatory simulation modelling approaches are designed, evaluated and communicated to diverse audiences. Deeper appreciation of the conditions under which simulation modelling can catalyse multiple social effects is recommended.

Applying systems approaches to stakeholder and community engagement and knowledge mobilisation in youth mental health system modelling

Background

There is a significant push to change the trajectory of youth mental ill-health and suicide globally. Ensuring that young people have access to services that meet their individual needs and are easily accessible is a priority. Genuine stakeholder engagement in mental health system design is critical to ensure that system strengthening is likely to be successful within these complex environments. There is limited literature describing engagement processes undertaken by research teams in mental health program implementation and planning. This protocol describes the methods that will be used to engage local communities using systems science methods to mobilize knowledge and action to strengthen youth mental health services.

Methods

Using participatory action research principles, the research team will actively engage with local communities to ensure genuine user-led participatory systems modelling processes and enhance knowledge mobilisation within research sites. Ensuring that culturally diverse and Aboriginal and Torres Strait Islander community voices are included will support this process. A rigorous site selection process will be undertaken to ensure that the community is committed and has capacity to actively engage in the research activities. Stakeholder engagement commences from the site selection process with the aim to build trust between researchers and key stakeholders. The research team will establish a variety of engagement resources and make opportunities available to each site depending on their local context, needs and audiences they wish to target during the process.

Discussion

This protocol describes the inclusive community engagement and knowledge mobilization process for the Right care, first time, where you live research Program. This Program will use an iterative and adaptive approach that considers the social, economic, and political context of each community and attempts to maximise research engagement. A theoretical framework for applying systems approaches to knowledge mobilization that is flexible will enable the implementation of a participatory action research approach. This protocol commits to a rigorous and genuine stakeholder engagement process that can be applied in mental health research implementation.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13033-022-00530-1.

Presenting a comprehensive multi-scale evaluation framework for participatory modelling programs: A scoping review

INTRODUCTION

Systems modelling and simulation can improve understanding of complex systems to support decision making, better managing system challenges. Advances in technology have facilitated accessibility of modelling by diverse stakeholders, allowing them to engage with and contribute to the development of systems models (participatory modelling). However, despite its increasing applications across a range of disciplines, there is a growing need to improve evaluation efforts to effectively report on the quality, importance, and value of participatory modelling. This paper aims to identify and assess evaluation frameworks, criteria, and/or processes, as well as to synthesize the findings into a comprehensive multi-scale framework for participatory modelling programs.

MATERIALS AND METHODS

A scoping review approach was utilized, which involved a systematic literature search via Scopus in consultation with experts to identify and appraise records that described an evaluation framework, criteria, and/or process in the context of participatory modelling. This scoping review is registered with the Open Science Framework.

RESULTS

The review identified 11 studies, which varied in evaluation purposes, terminologies, levels of examination, and time points. The review of studies highlighted areas of overlap and opportunities for further development, which prompted the development of a comprehensive multi-scale evaluation framework to assess participatory modelling programs across disciplines and systems modelling methods. The framework consists of four categories (Feasibility, Value, Change/Action, Sustainability) with 30 evaluation criteria, broken down across project-, individual-, group- and system-level impacts.

DISCUSSION & CONCLUSION

The presented novel framework brings together a significant knowledge base into a flexible, cross-sectoral evaluation effort that considers the whole participatory modelling process. Developed through the rigorous synthesis of multidisciplinary expertise from existing studies, the application of the framework can provide the opportunity to understand practical future implications such as which aspects are particularly important for policy decisions, community learning, and the ongoing improvement of participatory modelling methods.

Participatory Systems Modelling for Youth Mental Health: An Evaluation Study Applying a Comprehensive Multi-Scale Framework

The youth mental health sector is persistently challenged by issues such as service fragmentation and inefficient resource allocation. Systems modelling and simulation, particularly utilizing participatory approaches, is offering promise in supporting evidence-informed decision making with limited resources by testing alternative strategies in safe virtual environments before implementing them in the real world. However, improved evaluation efforts are needed to understand the critical elements involved in and to improve methods for implementing participatory modelling for youth mental health system and service delivery. An evaluation protocol is described to evaluate the feasibility, value, impact, and sustainability of participatory systems modelling in delivering advanced decision support capabilities for youth mental health. This study applies a comprehensive multi-scale evaluation framework, drawing on participatory action research principles as well as formative, summative, process, and outcome evaluation techniques. Novel data collection procedures are presented, including online surveys that incorporate gamification to enable social network analysis and patient journey mapping. The evaluation approach also explores the experiences of diverse stakeholders, including young people with lived (or living) experience of mental illness. Social and technical opportunities will be uncovered, as well as challenges implementing these interdisciplinary methods in complex settings to improve youth mental health policy, planning, and outcomes. This study protocol can also be adapted for broader international applications, disciplines, and contexts.

Participatory Methods for Systems Modeling of Youth Mental Health: Implementation Protocol

Background

Despite significant investment, mental health issues remain a leading cause of death among young people globally. Sophisticated decision analysis methods are needed to better understand the dynamic and multisector drivers of youth mental health. System modeling can help explore complex issues such as youth mental health and inform strategies to effectively respond to local needs and achieve lasting improvements. The advantages of engaging stakeholders in model development processes have long been recognized; however, the methods for doing so are often not well-described.

Objective

This paper aims to describe the participatory procedures that will be used to support systems modeling for national multisite implementation. The Right Care, First Time, Where You Live research program will focus on regional youth mental health applications of systems modeling in 8 different sites across Australia.

Methods

The participatory model development approach involves an iterative process of engaging with a range of participants, including people with lived experience of mental health issues. Their knowledge of the local systems, pathways, and drivers is combined with the academic literature and data to populate the models and validate their structure. The process centers around 3 workshops where participants interact and actively engage in group model-building activities to define, refine, and validate the systems models. This paper provides a detailed blueprint for the implementation of this process for mental health applications.

Results

The participatory modeling methods described in this paper will be implemented at 2 sites per year from 2022 to 2025. The 8 selected sites have been chosen to capture variations in important factors, including determinants of mental health issues and access to services. Site engagement commenced in August 2021, and the first modeling workshops are scheduled to commence in February 2022.

Conclusions

Mental health system decision makers require tools to help navigate complex environments and leverage interdisciplinary problem-solving. Systems modeling can mobilize data from diverse sources to explore a range of scenarios, including the impact of interventions in different combinations and contexts. Involving stakeholders in the model development process ensures that the model findings are context-relevant and fit-for-purpose to inform decision-making.

International Registered Report Identifier (IRRID)

PRR1-10.2196/32988

Computational Simulation Is a Vital Resource for Navigating the COVID-19 Pandemic

INTRODUCTION

COVID-19 has prompted the extensive use of computational models to understand the trajectory of the pandemic. This article surveys the kinds of dynamic simulation models that have been used as decision support tools and to forecast the potential impacts of nonpharmaceutical interventions (NPIs). We developed the Values in Viral Dispersion model, which emphasizes the role of human factors and social networks in viral spread and presents scenarios to guide policy responses.

METHODS

An agent-based model of COVID-19 was developed with individual agents able to move between 3 states (susceptible, infectious, or recovered), with each agent placed in 1 of 7 social network types and assigned a propensity to comply with NPIs (quarantine, contact tracing, and physical distancing). A series of policy questions were tested to illustrate the impact of social networks and NPI compliance on viral spread among (1) populations, (2) specific at-risk subgroups, and (3) individual trajectories.

RESULTS

Simulation outcomes showed large impacts of physical distancing policies on number of infections, with substantial modification by type of social network and level of compliance. In addition, outcomes on metrics that sought to maximize those never infected (or recovered) and minimize infections and deaths showed significantly different epidemic trajectories by social network type and among higher or lower at-risk age cohorts.

CONCLUSIONS

Although dynamic simulation models have important limitations, which are discussed, these decision support tools should be a key resource for navigating the ongoing impacts of the COVID-19 pandemic and can help local and national decision makers determine where, when, and how to invest resources.

Sound Decision Making in Uncertain Times: Can Systems Modelling Be Useful for Informing Policy and Planning for Suicide Prevention?

The COVID-19 pandemic demonstrated the significant value of systems modelling in supporting proactive and effective public health decision making despite the complexities and uncertainties that characterise an evolving crisis. The same approach is possible in the field of mental health. However, a commonly levelled (but misguided) criticism prevents systems modelling from being more routinely adopted, namely, that the presence of uncertainty around key model input parameters renders a model useless. This study explored whether radically different simulated trajectories of suicide would result in different advice to decision makers regarding the optimal strategy to mitigate the impacts of the pandemic on mental health. Using an existing system dynamics model developed in August 2020 for a regional catchment of Western Australia, four scenarios were simulated to model the possible effect of the COVID-19 pandemic on levels of psychological distress. The scenarios produced a range of projected impacts on suicide deaths, ranging from a relatively small to a dramatic increase. Discordance in the sets of best-performing intervention scenarios across the divergent COVID-mental health trajectories was assessed by comparing differences in projected numbers of suicides between the baseline scenario and each of 286 possible intervention scenarios calculated for two time horizons; 2026 and 2041. The best performing intervention combinations over the period 2021–2041 (i.e., post-suicide attempt assertive aftercare, community support programs to increase community connectedness, and technology enabled care coordination) were highly consistent across all four COVID-19 mental health trajectories, reducing suicide deaths by between 23.9–24.6% against the baseline. However, the ranking of best performing intervention combinations does alter depending on the time horizon under consideration due to non-linear intervention impacts. These findings suggest that systems models can retain value in informing robust decision making despite uncertainty in the trajectories of population mental health outcomes. It is recommended that the time horizon under consideration be sufficiently long to capture the full effects of interventions, and efforts should be made to achieve more timely tracking and access to key population mental health indicators to inform model refinements over time and reduce uncertainty in mental health policy and planning decisions.

Resource management framework using simulation modeling and multi-objective optimization: a case study of a front-end department of a public hospital in Thailand

Background

The overcrowded patients, which cause the long waiting time in public hospitals, become significant problems that affect patient satisfaction toward the hospital. Particularly, the bottleneck usually happens at front-end departments (e.g., the triage and medical record department) as every patient is firstly required to visit these departments. The problem is mainly caused by ineffective resource management. In order to support decision making in the resource management at front-end departments, this paper proposes a framework using simulation and multi-objective optimization techniques considering both operating cost and patient satisfaction.

Methods

To develop the framework, first, the timestamp of patient arrival time at each station was collected at the triage and medical record department of Thammasat University Hospital in Thailand. A patient satisfaction assessment method was used to convert the time spend into a satisfaction score. Then, the simulation model was built from the current situation of the hospital and was applied scenario analyses for the model improvement. The models were verified and validated. The weighted max–min for fuzzy multi-objective optimization was done by minimizing the operating cost and maximizing the patient satisfaction score. The operating costs and patient satisfaction scores from various scenarios were statistically compared. Finally, a decision-making guideline was proposed to support suitable resource management at the front-end departments of the hospital.

Result

The three scenarios of the simulation model were built (i.e., a real situation, a one-stop service, and partially shared resources) and ensured to be verified and valid. The optimized results were compared and grouped into three situations which are (1) remain the same satisfaction score but decrease the cost (cost decreased by 2.8%) (2) remain the same satisfaction score but increase the cost (cost increased up to 80%) and (3) decrease the satisfaction score and decrease the cost (satisfaction decreased up to 82% and cost decreased up to 59%). According to the guideline, the situations 1 and 3 were recommended to use in the improvement and the situation 2 was rejected.

Conclusion

This research demonstrates the resource management framework for the front-end department of the hospital. The experimental results imply that the framework can be used to support the decision making in resource management and used to reduce the risk of applying a non-improvement model in a real situation.

Medical Information Mining-Based Visual Artificial Intelligence Emergency Nursing Management System

This study aims to design a set of the visual artificial intelligence system based on medical information mining for hospital emergency care management. A visual artificial intelligence emergency first aid nursing management system is designed by analyzing the needs of the emergency first aid nursing management system. The results show that system personnel allocation, comparative management, record management, query management analysis, basic setup analysis, nursing management basis, and nonfunctional requirements all need to be optimized for the emergency first aid management system. In this study, the comparative management module, log management module, and the query management module are designed, and the emergency first aid management system of different APP terminal functions in different modules is described in detail. The nursing document query business is tested, and the corresponding time of query of nursing assessment sheet, nurse shift record, nurse record, and physical sign observation sheet is 375.50 ms, 351.48 ms, 336.36 ms, and 245.57 ms, respectively. It shows that the visual artificial intelligence emergency nursing management system based on medical information mining can provide convenience for clinical work to a large extent and has potential application value in hospital emergency nursing work.

New directions for participatory modelling in health: Redistributing expertise in relation to localised matters of concern

Participatory modelling seeks to foster stakeholder engagement to better attune models to their decision-making and policy contexts. Such approaches are increasingly advocated for use in the field of health. We review the instrumental and epistemological claims made in support of participatory modelling approaches. These accentuate participatory models as offering a better evidence-base for health policy decisions. By drawing attention to recent modelling experiments in a sector outside of health, that of water management, we outline a different way of thinking about participation and modelling. Here, the participatory model is configured in relation to matters of 'knowledge controversy', with modelling constituted as an 'evidence-making intervention' in relation to the making of science and expertise. Rather than presenting participatory models as an improved technical solution to addressing given policy problems within an evidence-based intervention approach, models are alternatively potentiated as sites for the redistribution of expertise among actor networks as they seek to engage politically in a matter of concern. This leads us to consider possible new directions for participatory modelling in the field of health.

Development of Policy-Relevant Indicators for Injury Prevention in British Columbia by the Key Decision-Makers

Indicators can help decision-makers evaluate interventions in a complex, multi-sectoral injury system. We aimed to create indicators for road safety, seniors falls, and ‘all-injuries’ to inform and evaluate injury prevention initiatives in British Columbia, Canada. The indicator development process involved a five-stage mixed methodology approach, including an environmental scan of existing indicators, generating expert consensus, selection of decision-makers and conducting a survey, selection of final indicators, and specification of indicators. An Indicator Reference Group (IRG) reviewed the list of indicators retrieved in the environmental scan and selected candidate indicators through expert consensus based on importance, modifiability, acceptance, and practicality. Key decision-makers (n = 561) were invited to rank each indicator in terms of importance and actionability (online survey). The IRG applied inclusion criteria and thresholds to survey responses from decision-makers, which resulted in the selection of 47 road safety, 18 seniors falls, and 33 all-injury indicators. After grouping “like” indicators, a final list of 23 road safety, 8 seniors falls, and 13 all-injury indicators were specified. By considering both decision-maker ranking and expert opinion, we anticipate improved injury system performance through advocacy, accountability, and evidence-based resource allocation in priority areas. Our indicators will inform a data management framework for whole-system reporting to drive policy and funding for provincial injury prevention improvement.

Rapid-cycle systems modeling to support evidence-informed decision-making during system-wide implementation

Background

To “model and simulate change” is an accepted strategy to support implementation at scale. Much like a power analysis can inform decisions about study design, simulation models offer an analytic strategy to synthesize evidence that informs decisions regarding implementation of evidence-based interventions. However, simulation modeling is under-utilized in implementation science. To realize the potential of simulation modeling as an implementation strategy, additional methods are required to assist stakeholders to use models to examine underlying assumptions, consider alternative strategies, and anticipate downstream consequences of implementation. To this end, we propose Rapid-cycle Systems Modeling (RCSM)—a form of group modeling designed to promote engagement with evidence to support implementation. To demonstrate its utility, we provide an illustrative case study with mid-level administrators developing system-wide interventions that aim to identify and treat trauma among children entering foster care.

Methods

RCSM is an iterative method that includes three steps per cycle: (1) identify and prioritize stakeholder questions, (2) develop or refine a simulation model, and (3) engage in dialogue regarding model relevance, insights, and utility for implementation. For the case study, 31 key informants were engaged in step 1, a prior simulation model was adapted for step 2, and six member-checking group interviews (n = 16) were conducted for step 3.

Results

Step 1 engaged qualitative methods to identify and prioritize stakeholder questions, specifically identifying a set of inter-related decisions to promote implementing trauma-informed screening. In step 2, the research team created a presentation to communicate key findings from the simulation model that addressed decisions about programmatic reach, optimal screening thresholds to balance demand for treatment with supply, capacity to start-up and sustain screening, and availability of downstream capacity to provide treatment for those with indicated need. In step 3, member-checking group interviews with stakeholders documented the relevance of the model results to implementation decisions, insight regarding opportunities to improve system performance, and potential to inform conversations regarding anticipated implications of implementation choices.

Conclusions

By embedding simulation modeling in a process of stakeholder engagement, RCSM offers guidance to realize the potential of modeling not only as an analytic strategy, but also as an implementation strategy.

Applying a complex systems perspective to alcohol consumption and the prevention of alcohol-related harms in the 21st century: a scoping review

BACKGROUND AND AIMS

A complex systems perspective has been advocated to explore multi-faceted factors influencing public health issues, including alcohol consumption and associated harms. This scoping review aimed to identify studies that applied a complex systems perspective to alcohol consumption and the prevention of alcohol-related harms in order to summarize their characteristics and identify evidence gaps.

METHODS

Studies published between January 2000 and September 2020 in English were located by searching for terms synonymous with 'complex systems' and 'alcohol' in the Scopus, MEDLINE, Web of Science and Embase databases, and through handsearching and reference screening of included studies. Data were extracted on each study's aim, country, population, alcohol topic, system levels, funding, theory, methods, data sources, time-frames, system modifications and type of findings produced.

RESULTS

Eighty-seven individual studies and three systematic reviews were identified, the majority of which were conducted in the United States or Australia in the general population, university students or adolescents. Studies explored types and patterns of consumption behaviour and the local environments in which alcohol is consumed. Most studies focused on individual and local interactions and influences, with fewer examples exploring the relationships between these and regional, national and international subsystems. The body of literature is methodologically diverse and includes theory-led approaches, dynamic simulation models and social network analyses. The systematic reviews focused on primary network studies.

CONCLUSIONS

The use of a complex systems perspective has provided a variety of ways of conceptualizing and analyzing alcohol use and harm prevention efforts, but its focus ultimately has remained on predominantly individual- and/or local-level systems. A complex systems perspective represents an opportunity to address this gap by also considering the vertical dimensions that constrain, shape and influence alcohol consumption and related harms, but the literature to date has not fully captured this potential.

Advancing complexity-informed health promotion: a scoping review to link health promotion and co-creation

A complexity-informed approach has recently been proposed as a hopeful revolution for health promotion (HP), requesting appropriate ways of tackling the complexities of health, equity and well-being. In addition, co-creation has gained traction as an approach to tackle complexity. HP and co-creation are established concepts that have long been enacted in practice. Although each concept is premised on similar approaches to value-creation such as participation and collaboration, little has been done to link the two approaches. To advance complexity-informed HP, this scoping review presents findings from peer-reviewed articles, published in English, between 2009 and March 2020. Articles were identified through searches of academic databases. Twenty-seven articles met the inclusion criteria, explicitly linking HP and co-creation. Included articles were charted by descriptive information and main focus, and advanced by a thematic analysis. Four themes suggest a potential avenue for advancing complexity-informed HP: (i) dealing with complexity, (ii) value creation, (iii) the value of the values and (iv) benefits and challenges. While current links between HP and co-creation are scarce they are increasing and promising. Based on the findings from the review, propositions to advance complexity-informed HP is outlined and discussed. Overall it is argued that co-creation and HP are mutually beneficial concepts, providing a framework for participative, collaborative, context-sensitive and knowledge-based practice that reflects the complex nature of health. More research is needed to highlight potential and challenges of integrating co-creation in HP, especially related to health equity and sustainable development.

How do participatory methods shape policy? Applying a realist approach to the formulation of a new tuberculosis policy in Georgia

OBJECTIVES

This paper presents the iterative process of participatory multistakeholder engagement that informed the development of a new national tuberculosis (TB) policy in Georgia, and the lessons learnt.

METHODS

Guided by realist evaluation methods, a multistakeholder dialogue was organised to elicit stakeholders' assumptions on challenges and possible solutions for better TB control. Two participatory workshops were conducted with key actors, interspersed by reflection meetings within the research team and discussions with policymakers. Using concept mapping and causal mapping techniques, and drawing causal loop diagrams, we visualised how actors understood TB service provision challenges and the potential means by which a results-based financing (RBF) policy could address these.

SETTING

The study was conducted in Tbilisi, Georgia.

PARTICIPANTS

A total of 64 key actors from the Ministry of Labour, Health and Social Affairs, staff of the Global Fund to Fight AIDS, TB and Malaria Georgia Project, the National Centre for Disease Control and Public Health, the National TB programme, TB service providers and members of the research team were involved in the workshops.

RESULTS

Findings showed that beyond provider incentives, additional policy components were necessary. These included broadening the incentive package to include institutional and organisational incentives, retraining service providers, clear redistribution of roles to support an integrated care model, and refinement of monitoring tools. Health system elements, such as effective referral systems and health information systems were highlighted as necessary for service improvement.

CONCLUSIONS

Developing policies that address complex issues requires methods that facilitate linkages between multiple stakeholders and between theory and practice. Such participatory approaches can be informed by realist evaluation principles and visually facilitated by causal loop diagrams. This approach allowed us leverage stakeholders' knowledge and expertise on TB service delivery and RBF to codesign a new policy.

Modelling tool to support decision-making in the NHS Health Check programme: workshops, systematic review and co-production with users

BACKGROUND

Local authorities in England commission the NHS Health Check programme to invite everyone aged 40-74 years without pre-existing conditions for risk assessment and eventual intervention, if needed. However, the programme's effectiveness, cost-effectiveness and equity impact remain uncertain.

AIM

To develop a validated open-access flexible web-based model that enables local commissioners to quantify the cost-effectiveness and potential for equitable population health gain of the NHS Health Check programme.

OBJECTIVES

The objectives were as follows: (1) co-produce with stakeholders the desirable features of the user-friendly model; (2) update the evidence base to support model and scenario development; (3) further develop our computational model to allow for developments and changes to the NHS Health Check programme and the diseases it addresses; (4) assess the effectiveness, cost-effectiveness and equity of alternative strategies for implementation to illustrate the use of the tool; and (5) propose a sustainability and implementation plan to deploy our user-friendly computational model at the local level.

DESIGN

Co-production workshops surveying the best-performing local authorities and a systematic literature review of strategies to increase uptake of screening programmes informed model use and development. We then co-produced the workHORSE (working Health Outcomes Research Simulation Environment) model to estimate the health, economic and equity impact of different NHS Health Check programme implementations, using illustrative-use cases.

SETTING

Local authorities in England.

PARTICIPANTS

Stakeholders from local authorities, Public Health England, the NHS, the British Heart Foundation, academia and other organisations participated in the workshops. For the local authorities survey, we invited 16 of the best-performing local authorities in England.

INTERVENTIONS

The user interface allows users to vary key parameters that represent programme activities (i.e. invitation, uptake, prescriptions and referrals). Scenarios can be compared with each other.

MAIN OUTCOME MEASURES

Disease cases and case-years prevented or postponed, incremental cost-effectiveness ratios, net monetary benefit and change in slope index of inequality.

RESULTS

The survey of best-performing local authorities revealed a diversity of effective approaches to maximise the coverage and uptake of NHS Health Check programme, with no distinct 'best buy'. The umbrella literature review identified a range of effective single interventions. However, these generally need to be combined to maximally improve uptake and health gains. A validated dynamic, stochastic microsimulation model, built on robust epidemiology, enabled service options analysis. Analyses of three contrasting illustrative cases estimated the health, economic and equity impact of optimising the Health Checks, and the added value of obtaining detailed local data. Optimising the programme in Liverpool can become cost-effective and equitable, but simply changing the invitation method will require other programme changes to improve its performance. Detailed data inputs can benefit local analysis.

LIMITATIONS

Although the approach is extremely flexible, it is complex and requires substantial amounts of data, alongside expertise to both maintain and run.

CONCLUSIONS

Our project showed that the workHORSE model could be used to estimate the health, economic and equity impact comprehensively at local authority level. It has the potential for further development as a commissioning tool and to stimulate broader discussions on the role of these tools in real-world decision-making.

FUTURE WORK

Future work should focus on improving user interactions with the model, modelling simulation standards, and adapting workHORSE for evaluation, design and implementation support.

STUDY REGISTRATION

This study is registered as PROSPERO CRD42019132087.

FUNDING

This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 25, No. 35. See the NIHR Journals Library website for further project information.

Participatory simulation modeling to inform colorectal cancer screening in a complex remote northern health system: Canada's Northwest Territories

BACKGROUND & AIMS

Colorectal cancer (CRC) mortality in the Northwest Territories (NWT), a northern region of Canada, could be reduced by implementing a CRC screening program. However, this may require additional colonoscopy resources. We used participatory simulation modeling to predict colonoscopy demand and to develop strategies for implementing a feasible and effective CRC screening program in this complex remote northern health system.

METHODS

Using a participatory simulation modeling approach, we first developed a conceptual model of CRC screening with local collaborators. This approach informed our parameter adjustments of an existing microsimulation model, OncoSim-CRC, using data from a retrospective cohort review of CRC screening between 2014-2019 and secondary data. Model scenarios reflecting program implementation were run for 500 million cases. Validity was assessed, and outputs analyzed with collaborators. Alternative scenarios were developed to reduce colonoscopy demand and results were presented to end-users.

RESULTS

We estimated that colonoscopy demand with a CRC screening program phased-in over 5 years would surpass capacity within 2 years. If demand is met, screen-detected cancers would increase by 110 %, and clinically-detected cases would reduce by 26 % over the next 30 years. We also found that prolonging the phase-in period, or revising adenoma follow-up guidelines would reduce colonoscopy demand while still improving cancer detection. Both strategies were considered feasible by collaborators. The adjusted model was valid, and the projections informed local end-users plans for CRC screening delivery.

CONCLUSIONS

Using participatory simulation modeling, we projected that a screening program would improve CRC detection but surpass current colonoscopy capacity. Phasing-in the screening program and reducing endoscopic adenoma follow-up would enhance feasibility of a CRC screening program in the NWT and help maintain its effectiveness.

A participatory modelling approach for investigating the spread of COVID-19 in countries of the Eastern Mediterranean Region to support public health decision-making

Early on in the COVID-19 pandemic, the WHO Eastern Mediterranean Regional Office recognised the importance of epidemiological modelling to forecast the progression of the COVID-19 pandemic to support decisions guiding the implementation of response measures. We established a modelling support team to facilitate the application of epidemiological modelling analyses in the Eastern Mediterranean Region (EMR) countries. Here, we present an innovative, stepwise approach to participatory modelling of the COVID-19 pandemic that engaged decision-makers and public health professionals from countries throughout all stages of the modelling process. Our approach consisted of first identifying the relevant policy questions, collecting country-specific data and interpreting model findings from a decision-maker's perspective, as well as communicating model uncertainty. We used a simple modelling methodology that was adaptable to the shortage of epidemiological data, and the limited modelling capacity, in our region. We discuss the benefits of using models to produce rapid decision-making guidance for COVID-19 control in the WHO EMR, as well as challenges that we have experienced regarding conveying uncertainty associated with model results, synthesising and comparing results across multiple modelling approaches, and modelling fragile and conflict-affected states.

Evidence-based Public Health Policy Models Development and Evaluation using Big Data Analytics and Web Technologies

Introduction.

According to WHO, “health policy refers to decisions, plans, and actions that are undertaken to achieve specific health care goals within a society”. Although policymaking is important to be based on scientific evidence, in many countries, evidence-informed decision-making remains the exception rather than the rule.

Aim:

This work presents a cloud-based Decision Support System for public health decision-making.

Methods:

In CrowdHEALTH, the concept of a Public Health Policy (PHP) is directly connected with one or more Key Performance Indexes (KPIs). The design and technical details of the system implementations are reported, along with use case scenarios.

Results:

The Policy Development Toolkit presents a unique interface and point of reference for policymakers, allowing them to create policy models and obtain analytical results for evidence-based decisions and evaluations.

Conclusions:

The hierarchical structure of the Public Health Policy Model offers versatility in the creation and handling of the policies, resulting in Health Analytics Tools Results Objects which offer quantitative policy support and provide the basis for meta-analytic operations.

Modelling the COVID-19 pandemic in context: an international participatory approach

The SARS-CoV-2 pandemic has had an unprecedented impact on multiple levels of society. Not only has the pandemic completely overwhelmed some health systems but it has also changed how scientific evidence is shared and increased the pace at which such evidence is published and consumed, by scientists, policymakers and the wider public. More significantly, the pandemic has created tremendous challenges for decision-makers, who have had to implement highly disruptive containment measures with very little empirical scientific evidence to support their decision-making process. Given this lack of data, predictive mathematical models have played an increasingly prominent role. In high-income countries, there is a long-standing history of established research groups advising policymakers, whereas a general lack of translational capacity has meant that mathematical models frequently remain inaccessible to policymakers in low-income and middle-income countries. Here, we describe a participatory approach to modelling that aims to circumvent this gap. Our approach involved the creation of an international group of infectious disease modellers and other public health experts, which culminated in the establishment of the COVID-19 Modelling (CoMo) Consortium. Here, we describe how the consortium was formed, the way it functions, the mathematical model used and, crucially, the high degree of engagement fostered between CoMo Consortium members and their respective local policymakers and ministries of health.

Engaging Healthcare Staff and Stakeholders in Healthcare Simulation Modeling to Better Translate Research Into Health Impact: A Systematic Review

Objective: To identify processes to engage stakeholders in healthcare Simulation Modeling (SM), and the impacts of this engagement on model design, model implementation, and stakeholder participants. To investigate how engagement process may lead to specific impacts. Data Sources: English-language articles on health SM engaging stakeholders in the MEDLINE, EMBASE, Scopus, Web of Science and Business Source Complete databases published from inception to February 2020. Study Design: A systematic review of the literature based on a priori protocol and reported according to PRISMA guidelines. Extraction Methods: Eligible articles were SM studies with a health outcome which engaged stakeholders in model design. Data were extracted using a data extraction form adapted to be specific for stakeholder engagement in SM studies. Data were analyzed using summary statistics, deductive and inductive content analysis, and narrative synthesis. Principal Findings: Thirty-two articles met inclusion criteria. Processes used to engage stakeholders in healthcare SM are heterogenous and often based on intuition rather than clear methodological frameworks. These processes most commonly involve stakeholders across multiple SM stages via discussion/dialogue, interviews, workshops and meetings. Key reported impacts of stakeholder engagement included improved model quality/accuracy, implementation, and stakeholder decision-making. However, for all but four studies, these reports represented author perceptions rather than formal evaluations incorporating stakeholder perspectives. Possible process enablers of impact included the use of models as "boundary objects" and structured facilitation via storytelling to promote effective communication and mutual understanding between stakeholders and modelers. Conclusions: There is a large gap in the current literature of formal evaluation of SM stakeholder engagement, and a lack of consensus about the processes required for effective SM stakeholder engagement. The adoption and clear reporting of structured engagement and process evaluation methodologies/frameworks are required to advance the field and produce evidence of impact.

Bringing new tools, a regional focus, resource-sensitivity, local engagement and necessary discipline to mental health policy and planning

Background

While reducing the burden of mental and substance use disorders is a global challenge, it is played out locally. Mental disorders have early ages of onset, syndromal complexity and high individual variability in course and response to treatment. As most locally-delivered health systems do not account for this complexity in their design, implementation, scale or evaluation they often result in disappointing impacts.

Discussion

In this viewpoint, we contend that the absence of an appropriate predictive planning framework is one critical reason that countries fail to make substantial progress in mental health outcomes. Addressing this missing infrastructure is vital to guide and coordinate national and regional (local) investments, to ensure limited mental health resources are put to best use, and to strengthen health systems to achieve the mental health targets of the 2015 Sustainable Development Goals.

Most broad national policies over-emphasize provision of single elements of care (e.g. medicines, individual psychological therapies) and assess their population-level impact through static, linear and program logic-based evaluation. More sophisticated decision analytic approaches that can account for complexity have long been successfully used in non-health sectors and are now emerging in mental health research and practice. We argue that utilization of advanced decision support tools such as systems modelling and simulation, is now required to bring a necessary discipline to new national and local investments in transforming mental health systems.

Conclusion

Systems modelling and simulation delivers an interactive decision analytic tool to test mental health reform and service planning scenarios in a safe environment before implementing them in the real world. The approach drives better decision-making and can inform the scale up of effective and contextually relevant strategies to reduce the burden of mental disorder and enhance the mental wealth of nations.

Creation of a Geospatially Explicit, Agent-based Model of a Regional Healthcare Network with Application to Clostridioides difficile Infection

Agent-based models (ABMs) describe and simulate complex systems comprising unique agents, or individuals, while accounting for geospatial and temporal variability among dynamic processes. ABMs are increasingly used to study healthcare-associated infections (ie, infections acquired during admission to a healthcare facility), including Clostridioides difficile infection, currently the most common healthcare-associated infection in the United States. The overall burden and transmission dynamics of healthcare-associated infections, including C difficile infection, may be influenced by community sources and movement of people among healthcare facilities and communities. These complex dynamics warrant geospatially explicit ABMs that extend beyond single healthcare facilities to include entire systems (eg, hospitals, nursing homes and extended care facilities, the community). The agents in ABMs can be built on a synthetic population, a model-generated representation of the actual population with associated spatial (eg, home residence), temporal (eg, change in location over time), and nonspatial (eg, sociodemographic features) attributes. We describe our methods to create a geospatially explicit ABM of a major regional healthcare network using a synthetic population as microdata input. We illustrate agent movement in the healthcare network and the community, informed by patient-level medical records, aggregate hospital discharge data, healthcare facility licensing data, and published literature. We apply the ABM output to visualize agent movement in the healthcare network and the community served by the network. We provide an application example of the ABM to C difficile infection using a natural history submodel. We discuss the ABM's potential to detect network areas where disease risk is high; simulate and evaluate interventions to protect public health; adapt to other geographic locations and healthcare-associated infections, including emerging pathogens; and meaningfully translate results to public health practitioners, healthcare providers, and policymakers.

Engaging with stakeholders to inform the development of a decision-support tool for the NHS health check programme: qualitative study

BACKGROUND

The NHS Health Check Programme is a risk-reduction programme offered to all adults in England aged 40-74 years. Previous studies mainly focused on patient perspectives and programme delivery; however, delivery varies, and costs are substantial. We were therefore working with key stakeholders to develop and co-produce an NHS Health Check Programme modelling tool (workHORSE) for commissioners to quantify local effectiveness, cost-effectiveness, and equity. Here we report on Workshop 1, which specifically aimed to facilitate engagement with stakeholders; develop a shared understanding of current Health Check implementation; identify what is working well, less well, and future hopes; and explore features to include in the tool.

METHODS

This qualitative study identified key stakeholders across the UK via networking and snowball techniques. The stakeholders spanned local organisations (NHS commissioners, GPs, and academics), third sector and national organisations (Public Health England and The National Institute for Health and Care Excellence). We used the validated Hovmand "group model building" approach to engage stakeholders in a series of pre-piloted, structured, small group exercises. We then used Framework Analysis to analyse responses.

RESULTS

Fifteen stakeholders participated in workshop 1. Stakeholders identified continued financial and political support for the NHS Health Check Programme. However, many stakeholders highlighted issues concerning lack of data on processes and outcomes, variability in quality of delivery, and suboptimal public engagement. Stakeholders' hopes included maximising coverage, uptake, and referrals, and producing additional evidence on population health, equity, and economic impacts. Key model suggestions focused on developing good-practice template scenarios, analysis of broader prevention activities at local level, accessible local data, broader economic perspectives, and fit-for-purpose outputs.

CONCLUSIONS

A shared understanding of current implementations of the NHS Health Check Programme was developed. Stakeholders demonstrated their commitment to the NHS Health Check Programme whilst highlighting the perceived requirements for enhancing the service and discussed how the modelling tool could be instrumental in this process. These suggestions for improvement informed subsequent workshops and model development.

Experts' perceptions on the use of visual analytics for complex mental healthcare planning: an exploratory study

BACKGROUND

Health experts including planners and policy-makers face complex decisions in diverse and constantly changing healthcare systems. Visual analytics may play a critical role in supporting analysis of complex healthcare data and decision-making. The purpose of this study was to examine the real-world experience that experts in mental healthcare planning have with visual analytics tools, investigate how well current visualisation techniques meet their needs, and suggest priorities for the future development of visual analytics tools of practical benefit to mental healthcare policy and decision-making.

METHODS

Health expert experience was assessed by an online exploratory survey consisting of a mix of multiple choice and open-ended questions. Health experts were sampled from an international pool of policy-makers, health agency directors, and researchers with extensive and direct experience of using visual analytics tools for complex mental healthcare systems planning. We invited them to the survey, and the experts' responses were analysed using statistical and text mining approaches.

RESULTS

The forty respondents who took part in the study recognised the complexity of healthcare systems data, but had most experience with and preference for relatively simple and familiar visualisations such as bar charts, scatter plots, and geographical maps. Sixty-five percent rated visual analytics as important to their field for evidence-informed decision-making processes. Fifty-five percent indicated that more advanced visual analytics tools were needed for their data analysis, and 67.5% stated their willingness to learn new tools. This was reflected in text mining and qualitative synthesis of open-ended responses.

CONCLUSIONS

This exploratory research provides readers with the first self-report insight into expert experience with visual analytics in mental healthcare systems research and policy. In spite of the awareness of their importance for complex healthcare planning, the majority of experts use simple, readily available visualisation tools. We conclude that co-creation and co-development strategies will be required to support advanced visual analytics tools and skills, which will become essential in the future of healthcare.

'Turning the tide' on hyperglycemia in pregnancy: insights from multiscale dynamic simulation modeling

INTRODUCTION

Hyperglycemia in pregnancy (HIP, including gestational diabetes and pre-existing type 1 and type 2 diabetes) is increasing, with associated risks to the health of women and their babies. Strategies to manage and prevent this condition are contested. Dynamic simulation models (DSM) can test policy and program scenarios before implementation in the real world. This paper reports the development and use of an advanced DSM exploring the impact of maternal weight status interventions on incidence of HIP.

METHODS

A consortium of experts collaboratively developed a hybrid DSM of HIP, comprising system dynamics, agent-based and discrete event model components. The structure and parameterization drew on a range of evidence and data sources. Scenarios comparing population-level and targeted prevention interventions were simulated from 2018 to identify the intervention combination that would deliver the greatest impact.

RESULTS

Population interventions promoting weight loss in early adulthood were found to be effective, reducing the population incidence of HIP by 17.3% by 2030 (baseline ('business as usual' scenario)=16.1%, 95% CI 15.8 to 16.4; population intervention=13.3%, 95% CI 13.0 to 13.6), more than targeted prepregnancy (5.2% reduction; incidence=15.3%, 95% CI 15.0 to 15.6) and interpregnancy (4.2% reduction; incidence=15.5%, 95% CI 15.2 to 15.8) interventions. Combining targeted interventions for high-risk groups with population interventions promoting healthy weight was most effective in reducing HIP incidence (28.8% reduction by 2030; incidence=11.5, 95% CI 11.2 to 11.8). Scenarios exploring the effect of childhood weight status on entry to adulthood demonstrated significant impact in the selected outcome measure for glycemic regulation, insulin sensitivity in the short term and HIP in the long term.

DISCUSSION

Population-level weight reduction interventions will be necessary to 'turn the tide' on HIP. Weight reduction interventions targeting high-risk individuals, while beneficial for those individuals, did not significantly impact forecasted HIP incidence rates. The importance of maintaining interventions promoting healthy weight in childhood was demonstrated.

What Can Policy-Makers Get Out of Systems Thinking? Policy Partners' Experiences of a Systems-Focused Research Collaboration in Preventive Health

BACKGROUND

There is increasing interest in using systems thinking to tackle 'wicked' policy problems in preventive health, but this can be challenging for policy-makers because the literature is amorphous and often highly theoretical. Little is known about how best to support health policy-makers to gain skills in understanding and applying systems thinking for policy action.

METHODS

In-depth interviews were conducted with 18 policy-makers who are participating in an Australian research collaboration that uses a systems approach. Our aim was to explore factors that support policy-makers to use systems approaches, and to identify any impacts of systems thinking on policy thinking or action, including the pathways through which these impacts occurred.

RESULTS

All 18 policy-makers agreed that systems thinking has merit but some questioned its practical policy utility. A small minority were confused about what systems thinking is or which approaches were being used in the collaboration. The majority were engaged with systems thinking and this group identified concrete impacts on their work. They reported using systems-focused research, ideas, tools and resources in policy work that were contributing to the development of practical methodologies for policy design, scaling up, implementation and evaluation; and to new prevention narratives. Importantly, systems thinking was helping some policy-makers to reconceptualise health problems and contexts, goals, potential policy solutions and methods. In short, they were changing how they think about preventive health.

CONCLUSION

These results show that researchers and policy-makers can put systems thinking into action as part of a research collaboration, and that this can result in discernible impacts on policy processes. In this case, action-oriented collaboration and capacity development over a 5-year period facilitated mutual learning and practical application. This indicates that policy-makers can get substantial applied value from systems thinking when they are involved in extended co-production processes that target policy impact and are supported by responsive capacity strategies.

Turning conceptual systems maps into dynamic simulation models: An Australian case study for diabetes in pregnancy

BACKGROUND

System science approaches are increasingly used to explore complex public health problems. Quantitative methods, such as participatory dynamic simulation modelling, can mobilise knowledge to inform health policy decisions. However, the analytic and practical steps required to turn collaboratively developed, qualitative system maps into rigorous and policy-relevant quantified dynamic simulation models are not well described. This paper reports on the processes, interactions and decisions that occurred at the interface between modellers and end-user participants in an applied health sector case study focusing on diabetes in pregnancy.

METHODS

An analysis was conducted using qualitative data from a participatory dynamic simulation modelling case study in an Australian health policy setting. Recordings of participatory model development workshops and subsequent meetings were analysed and triangulated with field notes and other written records of discussions and decisions. Case study vignettes were collated to illustrate the deliberations and decisions made throughout the model development process.

RESULTS

The key analytic objectives and decision-making processes included: defining the model scope; analysing and refining the model structure to maximise local relevance and utility; reviewing and incorporating evidence to inform model parameters and assumptions; focusing the model on priority policy questions; communicating results and applying the models to policy processes. These stages did not occur sequentially; the model development was cyclical and iterative with decisions being re-visited and refined throughout the process. Storytelling was an effective strategy to both communicate and resolve concerns about the model logic and structure, and to communicate the outputs of the model to a broader audience.

CONCLUSION

The in-depth analysis reported here examined the application of participatory modelling methods to move beyond qualitative conceptual mapping to the development of a rigorously quantified and policy relevant, complex dynamic simulation model. The analytic objectives and decision-making themes identified provide guidance for interpreting, understanding and reporting future participatory modelling projects and methods.