Natural experiments: an underused tool for public health?

How do free healthcare policies impact utilization of maternal and child health services in fragile settings? Evidence from a controlled interrupted time series analysis in Burkina Faso

Burkina Faso has implemented a nationwide free healthcare policy (gratuité) for pregnant and lactating women and children under 5 years since April 2016. Studies have shown that free healthcare policies can increase healthcare service use. However, the emerging coronavirus disease 2019 pandemic, escalating insecurity and the political situation in recent years might have affected the implementation of such policies. No studies have looked at whether the gratuité maintained high service use under such changing circumstances. Our study aimed to assess the effects of gratuité on the utilization of facility-based delivery and curative care of children under 5 years in light of this changing context. We employed a controlled interrupted time series analysis using data from the Health Management Information System and annual statistical reports of 2560 primary health facilities from January 2013 to December 2021. We focused on facility-based deliveries and curative care for children under 5 years, with antenatal care and curative care for children over 5 years as non-equivalent controls. We employed segmented regression with the generalized least square model, accounting for autocorrelation and monthly seasonality. The monthly utilization rate among children under 5 years compared to those above 5 years (controls) immediately increased by 111.19 visits per 1000 children (95% CI: 91.12, 131.26) due to the gratuité. This immediate effect declined afterwards with a monthly change of 0.93 per 1000 children (95% CI: -1.57, -0.29). We found no significant effects, both immediate and long-term, on the use of maternal care services attributable to the gratuité. Our findings suggest that free healthcare policies can be instrumental in improving healthcare, yet more comprehensive strategies are needed to maintain healthcare utilization. Our findings reflect the overall situation in the country, while localized research is needed to understand the effect of insecurity and the pandemic at the local level and the effects of gratuité across geographies and socioeconomic statuses.

Impact of non-pharmaceutical interventions on COVID-19 incidence and deaths: cross-national natural experiment in 32 European countries

PURPOSE

Non-pharmaceutical interventions (NPIs) have been the cornerstone of COVID-19 pandemic control, but evidence on their effectiveness varies according to the methods and approaches taken to empirical analysis. We analysed the impact of NPIs on incident SARS-CoV-2 across 32 European countries (March-December 2020) using two NPI trackers: the Corona Virus Pandemic Policy Monitor - COV-PPM, and the Oxford Covid-19 Government Response Tracker - OxCGRT.

METHODS

NPIs were summarized through principal component analysis into three sets, stratified by two waves (C1-C3, weeks 5-25, and C4-C6, weeks 35-52). Longitudinal, multi-level mixed-effects negative binomial regression models were fitted to estimate incidence rate ratios for cases and deaths considering different time-lags and reverse causation (i.e. changing incidence causing NPIs), stratified by waves and geographical regions (Western, Eastern, Northern, Southern, Others).

RESULTS

During the first wave, restrictions on movement/mobility, public transport, public events, and public spaces (C1) and healthcare system improvements, border closures and restrictions to public institutions (C2) were associated with a reduction in SARS-CoV-2 incidence after 28 and 35-days. Mask policies (C3) were associated with a reduction in SARS-CoV-2 incidence (except after 35-days). During wave 1, C1 and C2 were associated with a decrease in deaths after 49-days and C3 after 21, 28 and 35-days. During wave 2, restrictions on movement/mobility, public transport and healthcare system improvements (C5) were also associated with a decrease in SARS-CoV-2 cases and deaths across all countries.

CONCLUSION

In the absence of pre-existing immunity, vaccines or treatment options, our results suggest that the observed implementation of different categories of NPIs, showed varied associations with SARS-CoV-2 incidence and deaths across regions, and varied associations across waves. These relationships were consistent across components of NPIs derived from two policy trackers (CoV-PPM and OxCGRT).

Results from the ENJOY MAP for HEALTH: a quasi experiment evaluating the impact of age-friendly outdoor exercise equipment to increase older people's park visitations and physical activity

BACKGROUND

Recreational parks can play a significant role in older people's health, with emerging evidence suggesting that changes in the physical environment, such as refurbishments of local parks, can increase park visitations and physical activity engagement. The ENJOY MAP for HEALTH aimed to evaluate the impact of Seniors Exercise Park installations and associated capacity building activities on older people's park visitation, and park-based physical activity.

METHOD

The ENJOY MAP for HEALTH was a quasi-experiment study design that involved the installation of specialised Seniors Exercise Park equipment as part of park refurbishment, supported by promotion and community capacity building activities in six municipalities in Victoria, Australia. Direct observations of park users took place prior to park upgrades, one-month post upgrade and 12-months from baseline. The overall number and characteristics of park visitors, and the type and level of physical activity undertaken, were summarised descriptively. Generalised linear models were used to examine the impact of park refurbishment (equipment installation and site activation) on the total number of older people observed in the park, and their engagement in physical activity, accounting for site and seasonal effects.

RESULTS

Overall number of visits increased following park upgrades, with the largest number of visitors observed one-month post upgrade (n = 12,501). The proportion of older people observed at the parks remained relatively low prior to and one-month post upgrade compared to other age groups. However, after adjusting for site and seasonal effects, the number of older people observed in the parks increased significantly post upgrade and site activation compared to prior to the refurbishment (incidence rate ratios (IRR) 3.55; 95% CI 2.68, 4.70). The number of older people observed to be exercising at the Seniors Exercise Park also increased by 100% at 12-months post-installation relative to one-month post upgrade (IRR 2.00; 95% CI 1.26, 3.17).

CONCLUSION

Installation of the Seniors Exercise Parks and the supportive programs and activities following six park upgrades resulted in an increase in older people's park visitation and engagement in physical activity. Community engagement and training of volunteers with the support of local governments are likely to contribute to the increased park usage by older people.

TRIAL REGISTRATION

This trial was registered with the Australian New Zealand Clinical Trials Registry. Trial registration number ACTRN12621000965808. https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=380745&isReview=true .

Interventions targeting social determinants of mental disorders and the Sustainable Development Goals: a systematic review of reviews

Globally, mental disorders account for almost 20% of disease burden and there is growing evidence that mental disorders are socially determined. Tackling the United Nations Sustainable Development Goals (UN SDGs), which address social determinants of mental disorders, may be an effective way to reduce the global burden of mental disorders. We conducted a systematic review of reviews to examine the evidence base for interventions that map onto the UN SDGs and seek to improve mental health through targeting known social determinants of mental disorders. We included 101 reviews in the final review, covering demographic, economic, environmental events, neighborhood, and sociocultural domains. This review presents interventions with the strongest evidence base for the prevention of mental disorders and highlights synergies where addressing the UN SDGs can be beneficial for mental health.

Residential relocation to assess impact of changes in the living environment on cardio-respiratory health: A narrative literature review with considerations for exposome research

Residential relocation studies have become increasingly valuable tools for evaluating the effects of changing living environments on human health, but little is known about their application to multiple aspects of the living environment and the most appropriate methodology. This narrative review explores the utility of residential relocation as a natural experiment for studying the impact of changing urban exposures on cardio-metabolic health in high-income settings. It provides a comprehensive overview of the use of residential relocation studies, evaluates their methodological approaches, and synthesizes findings related to health behaviors and cardio-metabolic outcomes. Our search identified 43 relevant studies published between January 1995 and February 2023, from eight countries, predominantly the USA, Canada, and Australia. The majority of eligible studies were published between 2012 and 2021 and examined changes in various domains of the living environment, such as walkability, the built and social environments, but rarely combinations of exposures. Included studies displayed heterogeneity in design and outcomes, 25 involving only movers and 18 considering both movers and non-movers. To mitigate the issue of residential self-selection bias, most studies employed a "change-in-change" design and adjusted for baseline covariates but only a fraction of them accounted for time-varying confounding. Relocation causes simultaneous changes in various features of the living environment, which presents an opportunity for exposome research to establish causal relationships, using large datasets with increased statistical power and a wide range of health outcomes, behaviors and biomarkers. Residential relocation is not a random process. Thus, studies focusing on living environment characteristics need to carefully select time-varying covariates and reference group. Overall, this review informs future research by guiding choices in study design, data requirements, and statistical methodologies. Ultimately, it contributes to the advancement of the urban exposome field and enhances our understanding of the complex relationship between urban environments and human health.

The social life of natural experiments in epidemiology and public health

Over the twentieth century, the concept of the natural experiment has become increasingly prominent across a variety of disciplines, albeit most consequentially in epidemiology and public health. Drawing on an analysis of the scientific and medical literature, we explore the social life of the natural experiment, tracing its changing use, meaning and uptake to better understand the work done by the concept. We demonstrate how the natural experiment became central to the identity of post-war epidemiology as the discipline professionalised, turned its attention to the prevention of chronic disease and took centre stage in the field of public health. We then turn to its growing significance amid the rise of evidence-based medicine, and the new meanings natural experiments came to take on in the context of concerns about policy and evidence. Finally, we turn to the newest iteration of the natural experiment in the COVID-19 era, which saw an explosion of studies drawing on the term, albeit in ways that reveal more about the underlying politics of health than the method itself. Throughout, we illustrate that the concept of the natural experiment has always been fundamentally social and political and tied to disciplinary claims-making about evidence and what should count as such.

The past as present in health promotion: the case for a 'public health humanities'

Health promotion is conceived as a unifying concept for improving the health of populations. This means addressing the socio-cultural, economic and commercial causes of ill-health, which are necessarily informed by past policies and socio-cultural contexts. However, historical scholarship has rarely figured in health promotion practice or scholarship. This gap resides in the determinants of health, and notably in the analyses of tobacco control and skin cancer prevention, two long-running campaigns that have shaped modern health promotion in Australia. Both highlight a need for understanding the profound impact of history on the present and the value of learning from past successes and failures. Doing so requires integrating historical analyses into existing health promotion scholarship. To achieve this aim, we present a new 'public health humanities' methodology. This novel interdisciplinary framework is conceived as a spectrum in which historical studies integrate with existing health promotion disciplines to solve complex health problems. We draw on the many calls for more interdisciplinarity in health promotion and derive this methodology from proposals in the medical humanities and cognate fields that have wrestled with combining history and present-focused disciplines. Using tobacco control and skin cancer prevention as case studies, we demonstrate how public health humanities uses interdisciplinary teams and shared research questions to generate valuable new knowledge unavailable with traditional methods. Furthermore, we show how it creates evaluation criteria to consider the powerful impact of issues like colonialism on current inequities that hinder health promotion strategies, and from which lessons may be derived for the future.

Evaluation of the Healthy Start voucher scheme on maternal vitamin use and child breastfeeding: a natural experiment using data linkage

Background

Having a good start in life during pregnancy and infancy has been shown to be important for living both a healthy life and a longer life. Despite the introduction of many policies for the early-years age group, including voucher schemes, with the aim of improving nutrition, there is limited evidence of their impact on health.

Objectives

To assess the effectiveness of the Healthy Start voucher scheme on infant, child and maternal outcomes, and to capture the lived experiences of the Healthy Start voucher scheme for low-income women.

Design

This was a natural experiment study using existing data sets, linked to routinely collected health data sets, with a nested qualitative study of low-income women and an assessment of the health economics.

Setting

Representative sample of Scottish children and UK children.

Participants

Growing Up in Scotland cohort 2 (n = 2240), respondents to the 2015 Infant Feeding Study (n = 8067) and a sample of 40 participants in the qualitative study.

Interventions

The Health Start voucher, a means-tested scheme that provides vouchers worth £3.10 per week to spend on liquid milk, formula milk, fruit and vegetables.

Main outcome measures

Infant and child outcomes - breastfeeding initiation and duration; maternal outcomes - vitamin use pre and during pregnancy.

Results

The exposed group were women receiving the Healthy Start voucher (R), with two control groups: eligible and not claiming the Healthy Start voucher (E) and nearly eligible. There was no difference in vitamin use during pregnancy for either comparison (receiving the Healthy Start voucher, 82%; eligible and not claiming the Healthy Start voucher, 86%; p = 0.10 vs. receiving the Healthy Start voucher, 87%; nearly eligible, 88%; p = 0.43) in the Growing Up in Scotland cohort. Proportions were similar for the Infant Feeding Study cohort (receiving the Healthy Start voucher, 89%; eligible and not claiming the Healthy Start voucher, 86%; p = 0.01 vs. receiving the Healthy Start voucher, 89%; nearly eligible, 87%; p = 0.01); although results were statistically significantly different, these were small effect sizes. There was no difference for either comparison in breastfeeding initiation or breastfeeding duration in months in Growing Up in Scotland, but there was a negative effect of the Healthy Start voucher in the Infant Feeding Survey. This contrast between data sets indicates that results are inconclusive for breastfeeding. The qualitative study found that despite the low monetary value the women valued the Healthy Start voucher scheme. However, the broader lives of low-income women are crucial to understand the constraints to offer a healthy diet.

Limitations

Owing to the policy being in place, it was difficult to identify appropriate control groups using existing data sources, especially in the Infant Feeding Study.

Conclusions

As the Healthy Start voucher scheme attempts to influence health behaviour, this evaluation can inform other policies aiming to change behaviour and use voucher incentives. The null effect of Healthy Start vouchers on the primary outcomes may be due to the value of the vouchers being insufficient to change the broader lives of low-income women to offer a healthy diet.

Future work

The methods developed to undertake an economic evaluation alongside a natural experiment using existing data can be used to explore the cost-effectiveness of the Healthy Start voucher scheme.

Funding

This project was funded by the National Institute for Health and Care Research (NIHR) Public Health Research programme and will be published in full in Public Health Research; Vol. 11, No. 11. See the NIHR Journals Library website for further project information.

Research priorities for the primordial prevention of acute rheumatic fever and rheumatic heart disease by modifying the social determinants of health

The social determinants of health (SDH), such as access to income, education, housing and healthcare, strongly shape the occurrence of acute rheumatic fever (ARF) and rheumatic heart disease (RHD) at the household, community and national levels. The SDH are systemic factors that privilege some more than others and result in poverty and inequitable access to resources to support health and well-being. Primordial prevention is the modification of SDH to improve health and reduce the risk of disease acquisition and the subsequent progression to RHD. Modifying these determinants using primordial prevention strategies can reduce the risk of exposure to Group A Streptococcus, a causative agent of throat and skin infections, thereby lowering the risk of initiating ARF and its subsequent progression to RHD.This report summarises the findings of the Primordial Prevention Working Group-SDH, which was convened in November 2021 by the National Heart, Lung, and Blood Institute to assess how SDH influence the risk of developing RHD. Working group members identified a series of knowledge gaps and proposed research priorities, while recognising that community engagement and partnerships with those with lived experience will be integral to the success of these activities. Specifically, members emphasised the need for: (1) global analysis of disease incidence, prevalence and SDH characteristics concurrently to inform policy and interventions, (2) global assessment of legacy primordial prevention programmes to help inform the co-design of interventions alongside affected communities, (3) research to develop, implement and evaluate scalable primordial prevention interventions in diverse settings and (4) research to improve access to and equity of services across the RHD continuum. Addressing SDH, through the implementation of primordial prevention strategies, could have broader implications, not only improving RHD-related health outcomes but also impacting other neglected diseases in low-resource settings.

Poverty, ACEs and stigmatised places: The application of psychology to the challenges of disadvantage

BACKGROUND

Our life circumstances present continual challenges to our mental health and well-being. For most of us, the politics of economy and society determine our prospects of living a good life. The fact that the ability to control and change things that happen to us is largely vested in the power of remote others has inevitable, mostly negative consequences.

METHOD

The following opinion piece illustrates the challenge our discipline faces in locating a complimentary contribution alongside those of public health, sociology and other sister disciplines with particular reference to the intractable concerns of poverty, ACES and stigmatised places.

FINDINGS

The piece presents an examination of what psychology as a discipline can do in the context of adversity and challenges that individuals face but over which they have little sense of control. The discipline of psychology needs to play a meaningful part in understanding and addressing the impacts of societal matters, moving from a dominant position of individualised understandings of distress to embrace more fully the context in which people are expected to feel good and function well.

CONCLUSION

Community psychology offers a useful, established philosophy from which to advance our practices. However, a more sophisticated, discipline-wide narrative and grounded understanding that empathically represents real lives and captures individual functioning within a complex and distant societal system is urgently required.

Natural experiments for the evaluation of place-based public health interventions: a methodology scoping review

Introduction

Place-based public health evaluations are increasingly making use of natural experiments. This scoping review aimed to provide an overview of the design and use of natural experiment evaluations (NEEs), and an assessment of the plausibility of the as-if randomization assumption.

Methods

A systematic search of three bibliographic databases (Pubmed, Web of Science and Ovid-Medline) was conducted in January 2020 to capture publications that reported a natural experiment of a place-based public health intervention or outcome. For each, study design elements were extracted. An additional evaluation of as-if randomization was conducted by 12 of this paper's authors who evaluated the same set of 20 randomly selected studies and assessed 'as-if ' randomization for each.

Results

366 NEE studies of place-based public health interventions were identified. The most commonly used NEE approach was a Difference-in-Differences study design (25%), followed by before-after studies (23%) and regression analysis studies. 42% of NEEs had likely or probable as-if randomization of exposure (the intervention), while for 25% this was implausible. An inter-rater agreement exercise indicated poor reliability of as-if randomization assignment. Only about half of NEEs reported some form of sensitivity or falsification analysis to support inferences.

Conclusion

NEEs are conducted using many different designs and statistical methods and encompass various definitions of a natural experiment, while it is questionable whether all evaluations reported as natural experiments should be considered as such. The likelihood of as-if randomization should be specifically reported, and primary analyses should be supported by sensitivity analyses and/or falsification tests. Transparent reporting of NEE designs and evaluation methods will contribute to the optimum use of place-based NEEs.

An explorative study with convenience vegetables in urban Nigeria—The Veg-on-Wheels intervention

Nigerian consumers have been found to view vegetables as healthy and health is a principal motivation for consumption; however, consumers also experience barriers related to preparation time and availability of vegetables. We therefore conducted a Veg-on-Wheels intervention, in which ready-to-cook, washed and pre-cut green leafy vegetables (GLV) were kept cool and sold for five weeks at convenient locations near workplaces and on the open market in Akure, Nigeria. Surveys were conducted prior to the intervention with 680 consumers and during the final week of the intervention with 596 consumers near workplaces and 204 consumers at the open market. Both buyers and non-buyers of the intervention were included; 49% buyers in the workplace sample and 47% in the open market sample. The Veg-on-Wheels intervention was successful, with high awareness, positive attitudes and high customer satisfaction. GLV intake was higher for Veg-on-Wheels buyers compared with non-buyers after the intervention, i.e., 10.8 vs. 8.0 portions per week, respectively. Also the intake of other vegetables was higher in the intervention group. The motives and barriers for buyers and non-buyers differed across the selling locations: main barriers were trust in the vendor and GLV source. These trust issues and vendor preferences were viewed as more important to respondents at the market than those near workplaces. This study is the first intervention study on the selling of ready-to-cook convenience vegetables in urban Nigeria. It shows that a market exists for convenience vegetables and that they have the potential to increase vegetable intake. Insights on both the food environment and consumers’ motives and behaviour was crucial for designing and evaluating the intervention.

Challenges in Implementing the National Health Response to COVID-19 in Senegal

Since the beginning of the COVID-19 pandemic in Africa, many epidemiological or anthropological studies have been published. However, few studies have yet been conducted to understand the implementation of State interventions to fight the COVID-19 pandemic. In Senegal, the national response plan was planned before the country experienced its first official case of COVID-19 on 2 March 2020. This qualitative study, conducted in March and April 2021, based on 189 interviews, aims to understand how the national response has been implemented in several regions of Senegal. Implementation of the response to the pandemic was favoured by good preparation, capacity to adapt, responsiveness of health actors, and commitment for both the political and religious authorities. The implementation response was confronted by several constraining factors such as the coercive approach, the challenges of coordinating actors, and the lack of intersectoral response. The central level has sometimes used reflexivity processes to adapt its response, but it has remained highly politicized, centralized, directive, and with little involvement of civil society. In Senegal, the response to the pandemic has been implemented in a relatively political and directive, even coercive manner, without necessarily considering prior knowledge and the need to adapt it to local contexts and to involve civil society and community actors in the process.

Supplementary Information

The online version contains supplementary material available at 10.1007/s43477-022-00053-4.

Consent Requirements for Testing Health Policies: An Intercontinental Comparison of Expert Opinions

Individual informed consent is a central requirement for clinical research on human subjects, yet whether and how consent requirements should apply to health policy experiments (HPEs) remains unclear. HPEs test and evaluate public health policies prior to implementation. We interviewed 58 health experts in Tanzania, Bangladesh and Germany on informed consent requirements for HPEs. Health experts across all countries favored a strong evidence base, prior information to the affected populations, and individual consent for 'risky' HPEs. Differences pertained to individual risk perception, how and when consent by group representatives should be obtained and whether HPEs could be treated as health policies. The study adds to representative consent options for HPEs, yet shows that more research is needed in this field - particularly in the present Covid-19 pandemic which has highlighted the need for HPEs nationally and globally.

Do inhabitants profit from integrating a public health focus in urban renewal programmes? A Dutch case study

Background

Urban renewal traditionally involves policy sectors such as housing, transport, and employment, which potentially can enhance the health of residents living in deprived areas. Additional involvement of the public health sector might increase the health impact of these urban renewal activities. This study evaluates the health impact of an additional focus on health, under the heading of Healthy District Experiments (HDE), within districts where an urban renewal programme was carried out.

Methods

We evaluated changes in health outcomes before the start of the HDE and after implementation, and compared these changes with health changes in control areas, e.g. districts from the urban renewal programme where no additional HDE was implemented. Additionally, we gathered information on the content of the experiments to determine what types of activities have been implemented.

Results

The additional activities from the HDE were mostly aimed at strengthening the health care in the districts and at promoting physical activity. When we compared the prevalence in general health, mental health, overweight, obesity, smoking, and physical activity during the study period between the HDE districts and control districts, we found no significant differences in the rate of change. The study is limited by a small sample size and the cross-sectional nature of the data. These and other limitations are discussed.

Conclusion

We found no evidence for a beneficial health impact of the activities that were initiated with a specific focus on health, within a Dutch urban renewal programme. Specific attention for network management and the integration of such activities in the wider programme, as well as an allocated budget might be needed in order to sort a health impact.

FAST: A Framework to Assess Speed of Translation of Health Innovations to Practice and Policy

The 17-year time span between discovery and application of evidence in practice has become a unifying challenge for implementation science and translational science more broadly. Further, global pandemics and social crises demand timely implementation of rapidly accruing evidence to reduce morbidity and mortality. Yet speed remains an understudied metric in implementation science. Prevailing evaluations of implementation lack a temporal aspect, and current approaches have not yielded rapid implementation. In this paper, we address speed as an important conceptual and methodological gap in implementation science. We aim to untangle the complexities of studying implementation speed, offer a framework to assess speed of translation (FAST), and provide guidance to measure speed in evaluating implementation. To facilitate specification and reporting on metrics of speed, we encourage consideration of stakeholder perspectives (e.g., comparison of varying priorities), referents (e.g., speed in attaining outcomes, transitioning between implementation phases), and observation windows (e.g., time from intervention development to first patient treated) in its measurement. The FAST framework identifies factors that may influence speed of implementation and potential effects of implementation speed. We propose a research agenda to advance understanding of the pace of implementation, including identifying accelerators and inhibitors to speed.

Implementing a Rural Natural Experiment: A Protocol for Evaluating the Impacts of Food Coops on Food Consumption, Resident's Health and Community Vitality

BACKGROUND

Local food environments are recognized by experts as a determinant of healthy eating. Food cooperatives (coop) can promote the accessibility to healthier foods and thus improve the health of the population, particularly in remote rural communities.

OBJECTIVE

To measure the effects of implementing a food coop in a disadvantaged community with poor access to food. We have two main research questions: (1). Does the establishment of a food coop in rural areas described as food deserts have an impact on accessibility, frequency of use, food consumption, food quality, and ultimately the health of individuals? (2). Does the establishment of a food coop in rural areas described as food deserts have an impact on food security and community vitality?

DESIGN

A natural experiment with a mixed pre/post method will be used. The sample is composed of households that came from geographically isolated communities (population: 215 to 885 inhabitants) which qualified as food deserts and located in rural areas of Quebec (Canada). All communities plan to open a food coop (in the years 2022-2023), and as their opening will be staggered over time, participants from communities with a new food coop (intervention) will be compared to communities awaiting the opening of their food coop (control). Data collection was carried out at three time points: (1) before; (2) 1 to 5 months after; and (3) 13 to 17 months after the opening of the coop. Questionnaires were used to measure sociodemographic variables, dietary intake, residents' health, and community vitality. Semi-structured interviews were conducted with community stakeholders.

RESULTS

Few natural experiments have been conducted regarding the impact of implementing food coops. Gathering concrete data on the effectiveness and processes surrounding these interventions through natural experiments will help to quantify their impact and guide knowledge users and policymakers to make more informed decisions.

Revisiting concepts of evidence in implementation science

BACKGROUND

Evidence, in multiple forms, is a foundation of implementation science. For public health and clinical practice, evidence includes the following: type 1 evidence on etiology and burden; type 2 evidence on effectiveness of interventions; and type 3: evidence on dissemination and implementation (D&I) within context. To support a vision for development and use of evidence in D&I science that is more comprehensive and equitable (particularly for type 3 evidence), this article aims to clarify concepts of evidence, summarize ongoing debates about evidence, and provide a set of recommendations and tools/resources for addressing the "how-to" in filling evidence gaps most critical to advancing implementation science.

MAIN TEXT

Because current conceptualizations of evidence have been relatively narrow and insufficiently characterized in our opinion, we identify and discuss challenges and debates about the uses, usefulness, and gaps in evidence for implementation science. A set of questions is proposed to assist in determining when evidence is sufficient for dissemination and implementation. Intersecting gaps include the need to (1) reconsider how the evidence base is determined, (2) improve understanding of contextual effects on implementation, (3) sharpen the focus on health equity in how we approach and build the evidence-base, (4) conduct more policy implementation research and evaluation, and (5) learn from audience and stakeholder perspectives. We offer 15 recommendations to assist in filling these gaps and describe a set of tools for enhancing the evidence most needed in implementation science.

CONCLUSIONS

To address our recommendations, we see capacity as a necessary ingredient to shift the field's approach to evidence. Capacity includes the "push" for implementation science where researchers are trained to develop and evaluate evidence which should be useful and feasible for implementers and reflect community or stakeholder priorities. Equally important, there has been inadequate training and too little emphasis on the "pull" for implementation science (e.g., training implementers, practice-based research). We suggest that funders and reviewers of research should adopt and support a more robust definition of evidence. By critically examining the evolving nature of evidence, implementation science can better fulfill its vision of facilitating widespread and equitable adoption, delivery, and sustainment of scientific advances.

Developing logic models to inform public health policy outcome evaluation: an example from tobacco control

BACKGROUND

The evaluation of large-scale public health policy interventions often relies on observational designs where attributing causality is challenging. Logic models-visual representations of an intervention's anticipated causal pathway-facilitate the analysis of the most relevant outcomes. We aimed to develop a set of logic models that could be widely used in tobacco policy evaluation.

METHODS

We developed an overarching logic model that reflected the broad categories of outcomes that would be expected following the implementation of tobacco control policies. We subsequently reviewed policy documents to identify the outcomes expected to result from the implementation of each policy and conducted a literature review of existing evaluations to identify further outcomes. The models were revised according to feedbacks from a range of stakeholders.

RESULTS

The final models represented expected causal pathways for each policy. The models included short-term outcomes (such as policy awareness, compliance and social cognitive outcomes), intermediate outcomes (such as changes in smoking behaviour) and long-term outcomes (such as mortality, morbidity and health service usage).

CONCLUSIONS

The use of logic models enables transparent and theory-based planning of evaluation analyses and should be encouraged in the evaluation of tobacco control policy, as well as other areas of public health.

Understanding the impact of the installation of outdoor fitness equipment and a multi-sports court on park visitation and park-based physical activity: A natural experiment

INTRODUCTION

This natural experiment examined the impact of the installation of outdoor fitness equipment and one multi-sports court in a park located in Melbourne, Australia on park visitation and park-based physical activity compared to a control park. Barriers and facilitators to use of this equipment among adult park visitors were also explored.

METHODS

Park visitation and moderate-vigorous physical activity (MVPA) in the two parks were assessed before (T1, 2016) and after the intervention (T2, 2019) using observations of park visitors from children to older adults across four days (7.30am-5.00pm). The relative change in observed counts in visitors from T1-T2 in the intervention park compared to the control park was used to assess the effect of the park refurbishment. Intercept surveys regarding barriers and facilitators were conducted with adult park visitors.

RESULTS

Over the three-year period, total visitor counts increased by 16% and the number of park visitors observed engaging in MVPA increased by 24% in the intervention park. However, there were no significant intervention effects on total visitor counts (IRR = 0.89, 95%CI: 0.62, 1.30) or MVPA (IRR = 0.80, 95%CI: 0.53, 1.19) relative to the control park. Examples of frequently mentioned facilitators to increase use of the sports court and fitness equipment included having someone to use the facilities with and instructional signage, and common barriers included poor weather and lack of time.

CONCLUSIONS

The impact of the park upgrade on visitation and MVPA was not significant. A better understanding of the broader appeal of outdoor fitness equipment and a multi-sports court among different segments of the population is needed.

An International Pilot Study of Self-Reported Quality of Life in Outpatient and Inpatient Mental Health Settings

Introduction: Measuring quality of life (QoL) is essential to understand how clients perceive their care. In practice, many instruments are in place to identify mental health diagnoses and measure treatment outcomes, but there are fewer standardized instruments to routinely collect information about self-reported QoL, especially across different mental health settings. Moreover, existing tools have been criticized for being built from the perspective of care professionals rather than the users' perspective. The 23-item Self-Reported interRAI-QoL Survey for Mental Health and Addictions (interRAI SQoL-MHA) tackles these issues, as it is based on self-reported measures and has proven validity across settings and countries. Objective: The aim of this study is to assess and compare QoL across settings and explore associations between dimensions of self-reported QoL and some items from the interRAI SQoL-MHA in a multinational sample. Settings: Inpatient and community mental health services. Methods: Data were collected from organizations in Belgium, Finland, Russia, Brazil, Rwanda, Canada and Hong Kong. Logistic regression models were constructed using each domain scale of the interRAI SQoL-MHA (relationship, support, hope, activities and relationship with staff) as dependent variables. Results: A total of 2,474 people (51.2% female, 56.7% of age 45 or older) were included in the study. A benchmark analysis showed the samples that performed above the benchmark line or below. The models yielded significant odds ratios among the domain scales, as well as for the items of the interRAI SQoL-MHA, with positive associations for the items "work and education opportunities" and "satisfied with services", and inverse associations for the items "financial difficulties" and for the inpatient setting. Conclusion: The analysis of associations between the determinants offers relevant information to improve mental health care and clients' perceived quality of life. Information about the determinants can help policymakers to design interventions to improve care outcomes, as well as provide more possibilities for integration into the community. The interRAI SQoL-MHA is innovative, as it can be linked to the third generation interRAI MH and Community MH-instruments, to be used in different mental health care settings, combining the objective and subjective QoL domains.

Associations of changes in neighbourhood walkability with changes in walking activity in older adults: a fixed effects analysis

Background

Supporting older adults to engage in physically active lifestyles requires supporting environments. Walkable environments may increase walking activity in older adults, but evidence for this subgroup is scarce, and longitudinal studies are lacking. This study therefore examined whether changes in neighbourhood walkability were associated with changes in walking activity in older adults, and whether this association differed by individual-level characteristics and by contextual conditions beyond the built environment.

Methods

Data from 668 participants (57.8–93.4 years at baseline) across three waves (2005/06, 2008/09 and 2011/12) of the Longitudinal Aging Study Amsterdam (LASA) were used. These individuals did not relocate during follow-up. Self-reported outdoor walking activity in minutes per week was assessed using the LASA Physical Activity Questionnaire. Composite exposure measures of neighbourhood walkability (range: 0 (low)-100 (high)) within 500-m Euclidean buffer zones around each participant’s residential address were constructed by combining objectively measured high-resolution Geographic Information System data on population density, retail and service destination density, land use mix, street connectivity, green space density, and sidewalk density. Fixed effects linear regression analyses were applied, adjusted for relevant time-varying confounders.

Results

Changes in neighbourhood walkability were not statistically significantly associated with changes in walking activity in older adults (β_500m = − 0.99, 95% CI = -6.17–4.20). The association of changes in neighbourhood walkability with changes in walking activity did not differ by any of the individual-level characteristics (i.e., age, sex, educational level, cognitive impairment, mobility disability, and season) and area-level characteristics (i.e., road traffic noise, air pollution, and socioeconomic status).

Conclusions

This study did not show evidence for an association between changes in neighbourhood walkability and changes in walking activity in older adults. If neighbourhood walkability and walking activity are causally linked, then changes in neighbourhood walkability between 2005/06 and 2011/12 might have been not substantial enough to produce meaningful changes in walking activity in older adults.

Effectiveness of community interventions for protecting and promoting the mental health of working-age adults experiencing financial uncertainty: a systematic review

BACKGROUND

The COVID-19 pandemic has created a period of global economic uncertainty. Financial strain, personal debt, recent job loss and housing insecurity are important risk factors for the mental health of working-age adults. Community interventions have the potential to attenuate the mental health impact of these stressors. We examined the effectiveness of community interventions for protecting and promoting the mental health of working-age adults in high-income countries during periods of financial insecurity.

METHODS

Eight electronic databases were systematically screened for experimental and observational studies published since 2000 measuring the effectiveness of community interventions on mental health outcomes. We included any non-clinical intervention that aimed to address the financial, employment, food or housing insecurity of participants. A review protocol was registered on the PROSPERO database (CRD42019156364) and results are reported in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.

RESULTS

From 2326 studies screened, 15 met our inclusion criteria. Five categories of community intervention were identified: advice services colocated in healthcare settings; link worker social prescribing; telephone debt advice; food insecurity interventions; and active labour market programmes. In general, the evidence for effective and cost-effective community interventions delivered to individuals experiencing financial insecurity was lacking. From the small number of studies without a high risk of bias, there was some evidence that financial insecurity and associated mental health problems were amenable to change and differences by subpopulations were observed.

CONCLUSION

There is a need for well-controlled studies and trials to better understand effective ingredients and to identify those interventions warranting wider implementation.

A quasi-experimental examination of how changes in school-level intramurals are associated with physical activity among a sample of Canadian secondary school students from the COMPASS study

Introduction

Intramurals in schools may encourage physical activity among youth. Schools are continuously making changes to these intramurals, yet it is not well understood how these changes impact youth physical activity. The main objective of this research was to examine if changes in the number of intramurals were associated with youth physical activity over time with a secondary objective to explore the association between sport participation and physical activity among youth over time.

Methods

This study used three years of linked longitudinal school- and student-level data from Ontario schools in year 5 (Y5: 2016–2017), year 6 (Y6: 2017–2018) and year 7 (Y7: 2018–2019) of the COMPASS study. Data on intramurals from 55 schools were collected from the School Programs and Policies questionnaire to determine intramural changes that were made from Y5 to Y6. Using the COMPASS Student Questionnaire, baseline demographics were collected and data on physical activity and sport participation were measured at Y5, Y6 and Y7 on 4417 students. Hierarchical linear mixed regression models were used to estimate how changes in intramurals were associated with youth physical activity over time.

Results

Changes in school-specific intramurals were not significantly associated with physical activity over time. Intramural, varsity and community sport participation were all positively and significantly associated with youth physical activity among female and males.

Conclusions

Intramural, varsity and community sport participation are important opportunities for youth physical activity. Schools should offer a variety of intramural and varsity sports to encourage physical activity. Although adding intramurals may not be effective at increasing youth physical activity, they may be effective when used in combination with other strategies to increase physical activity.

•

The use of an indicator variable in multi-level modeling is a novel method to evaluate a natural experiment.

•

Intramural, varsity and community sport participation were positively associated with youth physical activity.

•

Intramural changes were not associated with youth physical activity over time.

•

Intramurals may be a useful part of a more comprehensive approach to increasing youth physical activity.

Implementation science should give higher priority to health equity

Background

There is growing urgency to tackle issues of equity and justice in the USA and worldwide. Health equity, a framing that moves away from a deficit mindset of what society is doing poorly (disparities) to one that is positive about what society can achieve, is becoming more prominent in health research that uses implementation science approaches. Equity begins with justice—health differences often reflect societal injustices. Applying the perspectives and tools of implementation science has potential for immediate impact to improve health equity.

Main text

We propose a vision and set of action steps for making health equity a more prominent and central aim of implementation science, thus committing to conduct implementation science through equity-focused principles to achieve this vision in U.S. research and practice. We identify and discuss challenges in current health disparities approaches that do not fully consider social determinants. Implementation research challenges are outlined in three areas: limitations of the evidence base, underdeveloped measures and methods, and inadequate attention to context. To address these challenges, we offer recommendations that seek to (1) link social determinants with health outcomes, (2) build equity into all policies, (3) use equity-relevant metrics, (4) study what is already happening, (5) integrate equity into implementation models, (6) design and tailor implementation strategies, (7) connect to systems and sectors outside of health, (8) engage organizations in internal and external equity efforts, (9) build capacity for equity in implementation science, and (10) focus on equity in dissemination efforts.

Conclusions

Every project in implementation science should include an equity focus. For some studies, equity is the main goal of the project and a central feature of all aspects of the project. In other studies, equity is part of a project but not the singular focus. In these studies, we should, at a minimum, ensure that we “leave no one behind” and that existing disparities are not widened. With a stronger commitment to health equity from funders, researchers, practitioners, advocates, evaluators, and policy makers, we can harvest the rewards of the resources being invested in health-related research to eliminate disparities, resulting in health equity.

Can a multi-level intervention approach, combining behavioural disciplines, novel technology and incentives increase physical activity at population-level?

BACKGROUND

Despite a global call for action and growing burden of non-communicable diseases (NCD) associated with physical inactivity, effective interventions to increase community-wide physical activity (PA) remain few. NCDs accounted for 80% of Singapore's disease burden (2015) and yet 40% of Singaporeans did not meet minimum recommended weekly PA despite evidence of the benefits to cardiorespiratory health, diabetes and cancer prevention.

METHODS

A large-scale public health intervention was initiated in 2015 to increase population-level PA through incidental daily walking. Intervention components included fitness trackers, redeemable rewards and gamification, implemented in a mutually-reinforcing manner within an eco-system supportive of PA and informed by real-time data analytics. Mean daily step count at baseline and post-intervention were compared across periods, and the influence of participant sub-groups characteristics on overall results, using significance tests. Standards for Reporting on Implementation Studies (StaRI) were adhered to.

RESULTS

Intervention reach increased fourfold from 129,677 participants in wave 1 (2015-16) to 690,233 in wave 3 (2017-18) amounting to a total of 1,184,410 Step Challenge participations. Mean days of fitness tracker use increased from 2.4 to 5.0 days/week among participants completing the Challenge in wave 1 and from 5.3 to 6.0 days/week in wave 3. The mean number of daily steps between pre-Challenge and Challenge periods increased by 4163 (sd=1360; p< 0.001) in wave 1, by 2242 (sd=334; p< 0.001) in wave 2 and by 1645 steps/day (sd=54; p< 0.001) in wave 3. Mean daily step increases between wave 1 and 3 also suggest that incidental PA was maintained, a finding supported by a 2017 national population survey showing that incidental PA among adults increased from 5% in 2010 to 14% in 2017 while moderate-intensity PA increased from 5 to 10% over the same period.

CONCLUSION

Population-level PA was effectively increased through multi-level interventions integrating technology, behavioural economics, gamification, marketing, communications and community linkages within a supportive context- and climate-appropriate environment. Responsive data analytics were instrumental to strengthen implementation by tailoring modalities that maximise effectiveness at population-level. Further analyses are needed to explore potential barriers, challenges or unmet needs in sub-groups with lower uptake to tailor future interventions for greater reach and impact.

Improving lifestyles sustainability through community gardening: results and lessons learnt from the JArDinS quasi-experimental study

BACKGROUND

Despite an increasing number of studies highlighting the health benefits of community gardening, the literature is limited by cross-sectional designs. The "JArDinS" quasi-experimental study aimed to assess the impact of community garden participation on the adoption of more sustainable lifestyles among French adults.

METHODS

Individuals entering a community garden in Montpellier (France) in 2018 (n = 66) were compared with pairwise matched individuals with no experience in community gardening (n = 66). Nutritional quality, environmental impact and cost of monthly household food supplies, level of physical activity measured by accelerometers, as well as mental and social well-being, sensitivity to food waste, and connection with nature were evaluated at baseline (t0) and 12 months later (t1) to explore sustainability of lifestyles in social/health, environmental and economic dimensions. Linear mixed models were used to determine the independent effect of community gardening on investigated lifestyles components. In-depth interviews were conducted at t1 with 15 gardeners to better understand changes that may have occurred in gardeners' lives during the first year of gardening.

RESULTS

At t0, gardeners had lower education level, lower BMI and their household reported lower percentage of meals consumed outside of the home compared to non-gardeners (p <  0.05). Participating in the community garden had no significant impact, in spite of sufficient statistical power, on fruit and vegetables supplies (main outcome), nor on physical activity parameters, nor on others of the social/health, environmental and economic lifestyles components investigated. Qualitative interviews suggested the existence of pre-established health and environmental consciousness in some gardeners and revealed several barriers to the participation such as lack of time, lack of gardening knowledge, physical difficulty of gardening, health problems and conflicts with other gardeners.

CONCLUSIONS

The health benefits of community gardening previously reported by cross-sectional studies might be confounded by selection bias. The JArDinS study highlights the need to identify solutions to overcome barriers related to community garden participation when designing relevant public health interventions for the promotion of sustainable lifestyles.

TRIAL REGISTRATION

The study was registered at clinicaltrials.gov as NCT03694782 . Date of registration: 3rd October 2018, retrospectively registered.

The Rebooting in Sports and Physical Activities After COVID-19 Italian Lockdown: An Exploratory Study

The lockdown imposed in Italy to reduce the spread of COVID-19 posited unusual challenges to people practicing sports and physical activities. The rebooting of activities highlighted the need to cope with new behaviors and routines, such as wearing a face mask while exercising. We conducted a web-based survey in Italy at the start of physical activities' rebooting, to investigate how people reacted to the new norms. Participants completed the questionnaires assessing insomnia, regulatory self-efficacy, optimism, mood states, and mental toughness. Age, gender, and environment were assumed as design variables. Results showed that in outdoor activities, the younger females as compared to the older manifest less regulatory self-efficacy, while the younger males manifest more regulatory self-efficacy than the older. In indoor activities, a reversed pattern of regulatory-self-efficacy was observed. Regarding life orientation, younger participants showed less optimism and positive expectation for the future and seem to be more exposed to the unexpected effects of the restrictions: they showed more sleep disturbances, confusion, depression, anger, and fatigue and less vigor and mental toughness than older participants. An understanding of the psychological implications of the rebooting phase can support the enactment of more appropriate behaviors to practice sports and physical activities when living at the time of the coronavirus.

Successes and failures of built environment interventions: Using concept mapping to assess stakeholder perspectives in four Canadian cities

BACKGROUND

Healthy communities can be supported through built environment interventions that redesign cities for improved health outcomes. Understanding the context of these interventions is critical for assessing how an intervention impacts population health; such context is often poorly documented. This study uses concept mapping to capture stakeholders' perspectives on the factors that influence the success and failure of built environment interventions across cities.

METHODS

The INTErventions, Research, and Action in Cities Team (INTERACT) research program undertook concept mapping exercises related to interventions in four Canadian cities: public bike share program (Vancouver); a cycling network (Victoria); a bus rapid transit program (Saskatoon); and interventions related to the Montreal Sustainability Plan (Montreal). Concept mapping synthesised stakeholder perceptions and Go zones were used to prioritise factors based on importance and feasibility. Resulting themes were integrated into implementation science frameworks.

RESULTS

Across four cities, 95 stakeholders participated. An average of 38 factors were identified in each city, resulting in 5 emerging clusters in Victoria and Montreal and 6 clusters in Vancouver and Saskatoon. Clusters covered domains of economic and political context, intervention planning, equity considerations, user experience, and stakeholder engagement. Common across all cities was the importance of stakeholder engagement. Concerns for citizen safety were prominent in Victoria, Vancouver, and Saskatoon. Interventions in Saskatoon and Vancouver were related to programs, and reliability of service and ease of use emerged. Go zones highlighted 2-5 items in each city, which can inform priority actions.

CONCLUSION

Our study provides stakeholders' collective thinking on the contextual factors that influence the success and failure of built environment interventions. Organising context within an implementation science framework can provide a common language to synthesise stakeholder perspectives across settings. Go zone items can inform city-specific priority actions to support a common vision around implementing built environment interventions in pursuit of designing equitable and healthy cities.

Behaviour change interventions to reduce risky substance use and improve mental health in children in care: the SOLID three-arm feasibility RCT

Background

Looked-after children and care leavers (henceforth children in care) are young people placed under the care of local authorities, often because of a history of family abuse and/or neglect. These young people have significantly increased risk of substance use and mental health problems compared with peers.

Aim

The Supporting Looked After Children and Care Leavers In Decreasing Drugs, and alcohol (SOLID) trial aimed to investigate the feasibility of a definitive randomised controlled trial comparing two behaviour change interventions to reduce risky substance use (illicit drugs and alcohol) in and improve the mental health of children in care aged 12–20 years.

Methods

The study consisted of two phases: (1) a formative phase that adapted the motivational enhancement therapy and social behaviour and network therapy interventions for use with children in care and (2) a three-arm pilot randomised controlled trial (comparing motivational enhancement therapy, social behaviour and network therapy and usual care), and a linked process and economic (return on investment) evaluation. Trial feasibility was compared with prespecified STOP/GO criteria.

Setting

Six local authority areas in the north-east of England.

Participants

Children in care (aged 12–20 years) who screened positive for drug and/or alcohol use within the last 12 months were eligible for the trial. The formative and process evaluations included children in care, carers, social workers, and drug and alcohol workers.

Outcome measures

The primary outcomes were recruitment and retention rates at 12 months’ follow-up. Baseline and 12-month follow-up questionnaires measured self-reported drug and alcohol use, mental health and health-related quality of life. The process evaluation considered acceptability and engagement with the interventions and trial procedures.

Results

Formative findings (n = 65) highlighted the need for interventions to increase the emphasis on therapeutic relationships, use creative methods of engagement and support the identification of treatment goals wider than substance misuse. Within the randomised controlled trial, of 860 participants screened, 211 (24.5%) met the inclusion criteria. One hundred and twelve (53%) of the 211 eligible children were recruited and randomised. Just 15 of the 76 (20%) participants allocated to intervention attended any of the motivational enhancement therapy of social behaviour and network therapy sessions, and 60 (54%) participants completed the 12-month follow-up. The screening and recruitment of children in care required significantly more time and resource investment by researchers and children’s services than planned. The process evaluation (n = 116) demonstrated that, despite participants engaging in risky substance use, they did not often acknowledge this nor felt that they needed help. Children in care had complex, chaotic lives and children’s services departments were less research mature and extremely stretched; this, coupled with the multiple steps in the intervention pathway and study protocol, resulted in low adherence to the intervention and the trial.

Conclusions

The SOLID trial demonstrated successful engagement with children in care to adapt the motivational enhancement therapy and social behaviour and network therapy interventions. However, the pilot randomised controlled trial found that a definitive trial is not feasible. The current screen, refer and treat pathway for children in care did not work. There is an urgent need to radically rethink how we deliver therapeutic services for children in care. A pragmatic evaluation design, coupled with additional research resource for children’s services, is needed to evaluate these novel models of care at scale.

Trial registration

This study is registered as PROSPERO CRD42018098974 and Current Controlled Trials ISRCTN80786829.

Funding

This project was funded by the National Institute for Health Research (NIHR) Public Health Research programme and will be published in full in Public Health Research; Vol. 8, No. 13. See the NIHR Journals Library website for further project information.

Active design of built environments for increasing levels of physical activity in adults: the ENABLE London natural experiment study

Background

Low physical activity is widespread and poses a serious public health challenge both globally and in the UK. The need to increase population levels of physical activity is recognised in current health policy recommendations. There is considerable interest in whether or not the built environment influences health behaviours, particularly physical activity levels, but longitudinal evidence is limited.

Objectives

The effect of moving into East Village (the former London 2012 Olympic and Paralympic Games Athletes’ Village, repurposed on active design principles) on the levels of physical activity and adiposity, as well as other health-related and well-being outcomes among adults, was examined.

Design

The Examining Neighbourhood Activities in Built Environments in London (ENABLE London) study was a longitudinal cohort study based on a natural experiment.

Setting

East Village, London, UK.

Participants

A cohort of 1278 adults (aged ≥ 16 years) and 219 children seeking to move into social, intermediate and market-rent East Village accommodation were recruited in 2013–15 and followed up after 2 years.

Intervention

The East Village neighbourhood, the former London 2012 Olympic and Paralympic Games Athletes’ Village, is a purpose-built, mixed-use residential development specifically designed to encourage healthy active living by improving walkability and access to public transport.

Main outcome measure

Change in objectively measured daily steps from baseline to follow-up.

Methods

Change in environmental exposures associated with physical activity was assessed using Geographic Information System-derived measures. Individual objective measures of physical activity using accelerometry, body mass index and bioelectrical impedance (per cent of fat mass) were obtained, as were perceptions of change in crime and quality of the built environment. We examined changes in levels of physical activity and adiposity using multilevel models adjusting for sex, age group, ethnic group, housing sector (fixed effects) and baseline household (random effect), comparing the change in those who moved to East Village (intervention group) with the change in those who did not move to East Village (control group). Effects of housing sector (i.e. social, intermediate/affordable, market-rent) as an effect modifier were also examined. Qualitative work was carried out to provide contextual information about the perceived effects of moving to East Village.

Results

A total of 877 adults (69%) were followed up after 2 years (mean 24 months, range 19–34 months, postponed from 1 year owing to the delayed opening of East Village), of whom 50% had moved to East Village; insufficient numbers of children moved to East Village to be considered further. In adults, moving to East Village was associated with only a small, non-significant, increase in mean daily steps (154 steps, 95% confidence interval –231 to 539 steps), more so in the intermediate sector (433 steps, 95% confidence interval –175 to 1042 steps) than in the social and market-rent sectors (although differences between housing sectors were not statistically significant), despite sizeable improvements in walkability, access to public transport and neighbourhood perceptions of crime and quality of the built environment. There were no appreciable effects on time spent in moderate to vigorous physical activity or sedentary time, body mass index or percentage fat mass, either overall or by housing sector. Qualitative findings indicated that, although participants enjoyed their new homes, certain design features might actually serve to reduce levels of activity.

Conclusions

Despite strong evidence of large positive changes in neighbourhood perceptions and walkability, there was only weak evidence that moving to East Village was associated with increased physical activity. There was no evidence of an effect on markers of adiposity. Hence, improving the physical activity environment on its own may not be sufficient to increase population physical activity or other health behaviours.

Funding

This project was funded by the National Institute for Health Research (NIHR) Public Health Research programme and will be published in full in Public Health Research; Vol. 8, No. 12. See the NIHR Journals Library website for further project information. This research was also supported by project grants from the Medical Research Council National Prevention Research Initiative (MR/J000345/1).

Longitudinal impact of changes in the residential built environment on physical activity: findings from the ENABLE London cohort study

BACKGROUND

Previous research has reported associations between features of the residential built environment and physical activity but these studies have mainly been cross-sectional, limiting inference. This paper examines whether changes in a range of residential built environment features are associated with changes in measures of physical activity in adults. It also explores whether observed effects are moderated by socio-economic status.

METHODS

Data from the Examining Neighbourhood Activity in Built Living Environments in London (ENABLE London) study were used. A cohort of 1278 adults seeking to move into social, intermediate, and market-rent East Village accommodation was recruited in 2013-2015, and followed up after 2 years. Accelerometer-derived steps (primary outcome), and GIS-derived measures of residential walkability, park proximity and public transport accessibility were obtained both at baseline and follow-up. Daily steps at follow-up were regressed on daily steps at baseline, change in built environment exposures and confounding variables using multilevel linear regression to assess if changes in neighbourhood walkability, park proximity and public transport accessibility were associated with changes in daily steps. We also explored whether observed effects were moderated by housing tenure as a marker of socio-economic status.

RESULTS

Between baseline and follow-up, participants experienced a 1.4 unit (95%CI 1.2,1.6) increase in neighbourhood walkability; a 270 m (95%CI 232,307) decrease in distance to their nearest park; and a 0.7 point (95% CI 0.6,0.9) increase in accessibility to public transport. A 1 s.d. increase in neighbourhood walkability was associated with an increase of 302 (95%CI 110,494) daily steps. A 1 s.d. increase in accessibility to public transport was not associated with any change in steps overall, but was associated with a decrease in daily steps amongst social housing seekers (- 295 steps (95%CI - 595, 3), and an increase in daily steps for market-rent housing seekers (410 95%CI -191, 1010) (P-value for effect modification = 0.03).

CONCLUSION

Targeted changes in the residential built environment may result in increases in physical activity levels. However, the effect of improved accessibility to public transport may not be equitable, showing greater benefit to the more advantaged.

What is the evidence for the impact of gardens and gardening on health and well-being: a scoping review and evidence-based logic model to guide healthcare strategy decision making on the use of gardening approaches as a social prescription

OBJECTIVE

To systematically identify and describe studies that have evaluated the impact of gardens and gardening on health and well-being. A secondary objective was to use this evidence to build evidence-based logic models to guide health strategy decision making about gardens and gardening as a non-medical, social prescription.

DESIGN

Scoping review of the impact of gardens and gardening on health and well-being. Gardens include private spaces and those open to the public or part of hospitals, care homes, hospices or third sector organisations.

DATA SOURCES

A range of biomedical and health management journals was searched including Medline, CINAHL, Psychinfo, Web of Knowledge, ASSIA, Cochrane, Joanna Briggs, Greenfile, Environment Complete and a number of indicative websites were searched to locate context-specific data and grey literature. We searched from 1990 to November 2019.

ELIGIBILITY CRITERIA

We included research studies (including systematic reviews) that assessed the effect, value or impact of any garden that met the gardening definition.

DATA EXTRACTION AND SYNTHESIS

Three reviewers jointly screened 50 records by titles and abstracts to ensure calibration. Each record title was screened independently by 2 out of 3 members of the project team and each abstract was screened by 1 member of a team of 3. Random checks on abstract and full-text screening were conducted by a fourth member of the team and any discrepancies were resolved through double-checking and discussion.

RESULTS

From the 8896 papers located, a total of 77* studies was included. Over 35 validated health, well-being and functional biometric outcome measures were reported. Interventions ranged from viewing gardens, taking part in gardening or undertaking therapeutic activities. The findings demonstrated links between gardens and improved mental well-being, increased physical activity and a reduction in social isolation enabling the development of 2 logic models.

CONCLUSIONS

Gardens and gardening can improve the health and well-being for people with a range of health and social needs. The benefits of gardens and gardening could be used as a 'social prescription' globally, for people with long-term conditions (LTCs). Our logic models provide an evidence-based illustration that can guide health strategy decision making about the referral of people with LTCs to socially prescribed, non-medical interventions involving gardens and gardening.

Using natural experiments to improve public health evidence: a review of context and utility for obesity prevention

Background

Natural experiments are increasingly valued as a way to assess the health impact of health and non-health interventions when planned controlled experimental research designs may be infeasible or inappropriate to implement. This study sought to investigate the value of natural experiments by exploring how they have been used in practice. The study focused on obesity prevention research as one complex programme area for applying natural experiment studies.

Methods

A literature search sought obesity prevention research from January 1997 to December 2017 and identified 46 population health studies that self-described as a natural experiment.

Results

The majority of studies identified were published in the last 5 years, illustrating a more recent adoption of such opportunities. The majority of studies were evaluations of the impact of policies (n = 19), such as assessing changes to food labelling, food advertising or taxation on diet and obesity outcomes, or were built environment interventions (n = 17), such as the impact of built infrastructure on physical activity or access to healthy food. Research designs included quasi-experimental, pre-experimental and non-experimental methods. Few studies applied rigorous research designs to establish stronger causal inference, such as multiple pre/post measures, time series designs or comparison of change against an unexposed group. In general, researchers employed techniques to enhance the study utility but often were limited in the use of more rigorous study designs by ethical considerations and/or the particular context of the intervention.

Conclusion

Greater recognition of the utility and versatility of natural experiments in generating evidence for complex health issues like obesity prevention is needed. This review suggests that natural experiments may be underutilised as an approach for providing evidence of the effects of interventions, particularly for evaluating health outcomes of interventions when unexpected opportunities to gather evidence arise.

Methods for accounting for neighbourhood self-selection in physical activity and dietary behaviour research: a systematic review

BACKGROUND

Self-selection into residential neighbourhoods is a widely acknowledged, but under-studied problem in research investigating neighbourhood influences on physical activity and diet. Failure to handle neighbourhood self-selection can lead to biased estimates of the association between the neighbourhood environment and behaviour. This means that effects could be over- or under-estimated, both of which have implications for public health policies related to neighbourhood (re)design. Therefore, it is important that methods to deal with neighbourhood self-selection are identified and reviewed. The aim of this review was to assess how neighbourhood self-selection is conceived and accounted for in the literature.

METHODS

Articles from a systematic search undertaken in 2017 were included if they examined associations between neighbourhood environment exposures and adult physical activity or dietary behaviour. Exposures could include any objective measurement of the built (e.g., supermarkets), natural (e.g., parks) or social (e.g., crime) environment. Articles had to explicitly state that a given method was used to account for neighbourhood self-selection. The systematic review was registered with the PROSPERO International Prospective Register of Systematic Reviews (number CRD42018083593) and was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement.

RESULTS

Of 31 eligible articles, almost all considered physical activity (30/31); few examined diet (2/31). Methods used to address neighbourhood self-selection varied. Most studies (23/31) accounted for items relating to participants' neighbourhood preferences or reasons for moving to the neighbourhood using multi-variable adjustment in regression models (20/23) or propensity scores (3/23). Of 11 longitudinal studies, three controlled for neighbourhood self-selection as an unmeasured confounder using fixed effects regression.

CONCLUSIONS

Most studies accounted for neighbourhood self-selection by adjusting for measured attributes of neighbourhood preference. However, commonly the impact of adjustment could not be assessed. Future studies using adjustment should provide estimates of associations with and without adjustment for self-selection; consider temporality in the measurement of self-selection variables relative to the timing of the environmental exposure and outcome behaviours; and consider the theoretical plausibility of presumed pathways in cross-sectional research where causal direction is impossible to establish.

Evaluating natural experiments to measure the co-benefits of urban policy interventions to reduce carbon emissions in New Zealand

Finding effective policy interventions for addressing the Sustainable Development Goals, such as reducing carbon emissions (SDG 13), which can also enhance good health and wellbeing (SDG 3), is urgent. Many promising interactions occur between sustainable cities and communities (SDG 11), clean water and sanitation (SDG 6) and affordable and clean energy (SDG 7), which sit at the centre of integrated urban planning and regeneration. In this paper, we consider the framing and findings of four policies we have evaluated as natural experiments, all of which have important co-benefits, which were not always the focus of the initial policies. These policies are: the installation of cycleways and walkways in existing suburbs; a central government measure to recentralise employment after the 2011 Christchurch earthquakes; the streamlining of housing developments by reducing land-use regulation; and ongoing changes of policy about public housing investment. We show that having a clear understanding of the benefits of increased physical activity for health, for example, could accelerate the reduction of carbon emissions. Also, decisions about infrastructure, housing and job locations can generate health and environmental gains if supported by broader public transport investments. Cycling and walking are also more likely to substitute for short habitual car trips. Similarly, measuring the co-benefits of differential residential models, relating to land-use and renewable energy, provides a framework that can facilitate learning from policy interventions, enabling wider and potentially more rapid generalisation of policies, including those for climate change mitigation.

Local walking and cycling by residents living near urban motorways: cross-sectional analysis

Background

Everyday activities, such as walking or cycling, may be a feasible and practical way to integrate physical activity into everyday life. Walking and cycling for transport or recreation in the area local to a person’s home may have additional benefits. However, urban planning tends to prioritise car use over active modes. We explored the cross-sectional association between living near an urban motorway and local walking and cycling.

Methods

In 2013, residents living in an area (a) near a new urban motorway (M74), (b) near a longstanding urban motorway (M8), or (c) without a motorway, in Glasgow, Scotland, were invited to complete postal surveys assessing local walking and cycling journeys and socio-demographic characteristics. Using adjusted regression models, we assessed the association between motorway proximity and self-reported local walking and cycling, as well as the count of types of destination accessed. We stratified our analyses according to study area.

Results

One thousand three hundred forty-three residents (57% female; mean age: 54 years; SD: 16 years) returned questionnaires. There was no overall association between living near an urban motorway and the likelihood of local walking or cycling, or the number of types of local destination accessed by foot or bicycle.

In stratified analyses, for those living in the area around the new M74 motorway, increasing residential proximity to the motorway was associated with lower likelihood of local recreational walking and cycling (OR 0.63, 95% CI: 0.39 to 1.00) a pattern not found in the area with the longstanding M8 motorway. In the area near the M8 motorway residential proximity was statistically significantly (p = 0.014) associated with a 12% decrease in the number of types of destination accessed, a pattern not found in the M74 study area.

Conclusions

Our findings suggest that associations between living near a motorway and local walking and cycling behaviour may vary by the characteristics of the motorway, and by whether the behaviour is for travel or recreation. The lack of associations seen in the study area with no motorway suggests a threshold effect whereby beyond a certain distance from a motorway, additional distance makes no difference.

Noncomprehensive and Intermittent Obesity-Related School Programs and Policies May Not Work: Evidence from the COMPASS Study

BACKGROUND

The school environment has the potential to influence student body mass index (BMI) through programs and policies. Our objective was to examine the effect of modifying obesity-related school policies and programs on youths' BMI trajectories.

METHODS

Obesity-related school policies and programs related to physical activity and healthy eating were collected from 41 schools across Ontario at baseline (2012-2013) and year 2 (2013-2014) of the COMPASS study. Self-reported height and weight were collected from the 4951 grades 9 and 10 students who attended those schools for 3 years. Linear mixed effects regression models examined the effect of modifying obesity-related school policies and programs on youths' BMI trajectories.

RESULTS

Between Y₁ and Y₂ , 26 of the 41 schools implemented distinct new obesity-related programs or policies related to physical activity or healthy eating. Five of the interventions were associated with BMI trajectories of students attending those schools compared to students attending a pooled sample of control schools, predicting a higher BMI trajectory.

CONCLUSIONS

Isolated programs and policies may not successfully improve youths' BMI trajectories. Further research is required to test the implementation of policies and programs that follow a comprehensive school health (CSH) approach, targeting BMI and associated behaviors.

How to evaluate the effect of seven years of the Norwegian School Fruit Scheme (2007-2014) on fruit, vegetable and snack consumption and weight status: A natural experiment

Aim: From August 2007 to June 2014, the Norwegian School Fruit Scheme (NSFS) legally established that all pupils in junior high and combined schools (275,000 pupils every year), but not those in primary schools (343,000 pupils every year), were entitled to a free piece of fruit or vegetable every school day. The NSFS is a natural experiment, unique in terms of scope and lengthiness. Such governmental efforts to improve the diet of the public are rarely evaluated. Thus, an evaluation of the comprehensive, well-designed NSFS is warranted. The aim was to describe how the NSFS can be evaluated using existing data sets. Methods: Four data sets have been identified for the evaluation of the NSFS: (a) the Nord-Trøndelag Health Study, (b) the Norwegian Mother and Child Cohort Study, (c) the Norwegian Child Growth Study/Growth in Teenagers and (d) Health Behaviour in School Aged Children. These comprehensive studies have collected cross-sectional or longitudinal data providing information about children's dietary consumption and/or weight status, which can be utilised in the evaluation of the NSFS. Both short- and long-term effects of the NSFS related to dietary habits and weight status and the potential effect of moderators such as socio-economic status, sex, ethnicity and the age of children and adolescents can be studied. Conclusions: Worldwide, there is a lack of well-designed, long-term studies evaluating the impact of governmental efforts to improve public diet. The present study describes how the NSFS can be evaluated using data from four large data sets on eating habits and weight status.

Effect of implementation of a University Restaurant on the diet of students in a Brazilian public university

Abstract Dietary practices of college students were compared before and after implementation of the University Restaurant (UR) and examined according to frequency of UR use. A natural experiment was conducted with students (n = 1131) of a Brazilian public university using a validated self-completed and identified questionnaire that inquired information on practices of substituting lunch and/or dinner with a snack (≥ 5 days/week) and on regular consumption of foods that were markers of a healthy or unhealthy diet. At the second time point, UR use by students was also assessed based on their attendance to it. Changes in food practices were examined by determining differences in proportions between the two assessments. The analysis of the association between UR use and each dietary practice was carried out using multiple logistic regression models. An association was observed between greater UR use and higher frequency of regular consumption of beans, vegetables, raw vegetables, cooked vegetables and fruit and lower frequency of regular consumption of French fries and/or fried snacks. The UR proved to be an environment that facilitated the adoption of healthy dietary practices and promoted improvement in the diets of the students who were more assiduous to the restaurant.

Which design to evaluate complex interventions? Toward a methodological framework through a systematic review

BACKGROUND

Evaluation of complex interventions (CI) is challenging for health researchers and requires innovative approaches. The objective of this work is to present the main methods used to evaluate CI.

METHODS

A systematic review of the scientific literature was conducted to identify methods used for the evaluation of CI. We searched MEDLINE via PubMed databases for articles including an evaluation or a pilot study of a complex intervention, published in a ten-year period. Key-words of this research were ("complex intervention*" AND "evaluation").

RESULTS

Among 445 identified articles, 100 research results or protocols were included. Among them, 5 presented 2 different types of design in the same publication, thus our work included 105 designs. Individual randomized controlled trials (IRCT) represented 21.9% (n = 23) of evaluation designs, randomized clinical trials adaptations 44.8% (n = 47), quasi -experimental designs and cohort study 19.0% (n = 20), realist evaluation 6.7% (n = 7) and other cases studies and other approaches 8.6% (n = 9). A process/mechanisms analysis was included in 80% (n = 84) of these designs.

CONCLUSION

A range of methods can be used successively or combined at various steps of the evaluation approach. A framework is proposed to situate each of the designs with respect to evaluation questions. The growing interest of researchers in alternative methods and the development of their use must be accompanied by conceptual and methodological research in order to more clearly define their principles of use.

From intervention to interventional system: towards greater theorization in population health intervention research

Background

Population health intervention research raises major conceptual and methodological issues. These require us to clarify what an intervention is and how best to address it.

This paper aims to clarify the concepts of intervention and context and to propose a way to consider their interactions in evaluation studies, especially by addressing the mechanisms and using the theory-driven evaluation methodology.

Main text

This article synthesizes the notions of intervention and context. It suggests that we consider an “interventional system”, defined as a set of interrelated human and non-human contextual agents within spatial and temporal boundaries generating mechanistic configurations – mechanisms – which are prerequisites for change in health. The evaluation focal point is no longer the interventional ingredients taken separately from the context, but rather mechanisms that punctuate the process of change. It encourages a move towards theorization in evaluation designs, in order to analyze the interventional system more effectively. More particularly, it promotes theory-driven evaluation, either alone or combined with experimental designs.

Conclusion

Considering the intervention system, hybridizing paradigms in a process of theorization within evaluation designs, including different scientific disciplines, practitioners and intervention beneficiaries, may allow researchers a better understanding of what is being investigated and enable them to design the most appropriate methods and modalities for characterizing the interventional system. Evaluation methodologies should therefore be repositioned in relation to one another with regard to a new definition of “evidence”, repositioning practitioners’ expertise, qualitative paradigms and experimental questions in order to address the intervention system more profoundly.

The impact of a park refurbishment in a low socioeconomic area on physical activity: a cost-effectiveness study

Background

Physical inactivity is the fourth highest cause of death globally and is a major contributor to increases in healthcare expenditure. Improving public open spaces such as parks in areas of low socio-economic position (SEP) may increase recreational physical activity in disadvantaged populations. We assessed the cost-effectiveness of the installation of a play-space in a large metropolitan park in a low socioeconomic area based on changes in physical activity.

Methods

Observational data of visitor counts and activities undertaken in the park before the installation of the new play-scape (T1), at two months (T2) and 14 months post-installation (T3) were obtained for the intervention and a control park (with no refurbishment) located in a high SEP metropolitan area. Observed sitting, standing, and moderate and vigorous-intensity physical activity were converted to yearly MET-h according to age. Costs of the play-scape and ongoing maintenance were obtained from the organisation managing the refurbishment. The incremental cost-effectiveness ratio (ICER) (ratio of incremental cost to incremental effect) was calculated based on the incremental increase in MET-h from T1 to T3 assuming a 20-year lifetime of the play-scape. Observation counts combining moderate and vigorous activity were used in the sensitivity analysis.

Results

When compared with T1, at T3 the new play-scape resulted in an overall incremental net gain of 114,114 MET-h (95% UI: 80,476 − 146,096) compared with the control park and an incremental cost effectiveness ratio (or cost per MET-h gained per park visitor) of AUD $0.58 (95% UI: $0.44–$0.80).

The sensitivity analysis combining moderate and vigorous activity into one category showed an increase in estimated incremental MET-h of 118,190 (95% CI: 83,528 − 149,583) and a lower incremental cost per MET-h gained of AUD $0.56 (95% UI: $0.43–$0.77).

Conclusions

Using a benchmark of cost-effectiveness for physical activity interventions of AUD $0.60–$1.30, this study suggests that the installation of a play-scape located in a low SEP area is cost-effective based on its potential to facilitate increases in MET-h. It provides much needed preliminary evidence and requires replication elsewhere.

Electronic supplementary material

The online version of this article (10.1186/s12966-019-0786-5) contains supplementary material, which is available to authorized users.

Integrating research and system-wide practice in public health: lessons learnt from Better Start Bradford

Many interventions that are delivered within public health services have little evidence of effect. Evaluating interventions that are being delivered as a part of usual practice offers opportunities to improve the evidence base of public health. However, such evaluation is challenging and requires the integration of research into system-wide practice. The Born in Bradford’s Better Start experimental birth cohort offers an opportunity to efficiently evaluate multiple complex community interventions to improve the health, wellbeing and development of children aged 0–3 years. Based on the learning from this programme, this paper offers a pragmatic and practical guide to researchers, public health commissioners and service providers to enable them to integrate research into their everyday practice, thus enabling relevant and robust evaluations within a complex and changing system.

Using the principles of co-production the key challenges of integrating research and practice were identified, and appropriate strategies to overcome these, developed across five key stages: 1) Community and stakeholder engagement; 2) Intervention design; 3) Optimising routinely collected data; 4) Monitoring implementation; and 5) Evaluation. As a result of our learning we have developed comprehensive toolkits (https://borninbradford.nhs.uk/what-we-do/pregnancy-early-years/toolkit/) including: an operational guide through the service design process; an implementation and monitoring guide; and an evaluation framework. The evaluation framework incorporates implementation evaluations to enable understanding of intervention performance in practice, and quasi experimental approaches to infer causal effects in a timely manner. We also offer strategies to harness routinely collected data to enhance the efficiency and affordability of evaluations that are directly relevant to policy and practice.

These strategies and tools will help researchers, commissioners and service providers to work together to evaluate interventions delivered in real-life settings. More importantly, however, we hope that they will support the development of a connected system that empowers practitioners and commissioners to embed innovation and improvement into their own practice, thus enabling them to learn, evaluate and improve their own services.

INTERACT: A comprehensive approach to assess urban form interventions through natural experiments

BACKGROUND

Urban form interventions can result in positive and negative impacts on physical activity, social participation, and well-being, and inequities in these outcomes. Natural experiment studies can advance our understanding of causal effects and processes related to urban form interventions. The INTErventions, Research, and Action in Cities Team (INTERACT) is a pan-Canadian collaboration of interdisciplinary scientists, urban planners, and public health decision makers advancing research on the design of healthy and sustainable cities for all. Our objectives are to use natural experiment studies to deliver timely evidence about how urban form interventions influence health, and to develop methods and tools to facilitate such studies going forward.

METHODS

INTERACT will evaluate natural experiments in four Canadian cities: the Arbutus Greenway in Vancouver, British Columbia; the All Ages and Abilities Cycling Network in Victoria, BC; a new Bus Rapid Transit system in Saskatoon, Saskatchewan; and components of the Sustainable Development Plan 2016-2020 in Montreal, Quebec, a plan that includes urban form changes initiated by the city and approximately 230 partnering organizations. We will recruit a cohort of between 300 and 3000 adult participants, age 18 or older, in each city and collect data at three time points. Participants will complete health and activity space surveys and provide sensor-based location and physical activity data. We will conduct qualitative interviews with a subsample of participants in each city. Our analysis methods will combine machine learning methods for detecting transportation mode use and physical activity, use temporal Geographic Information Systems to quantify changes to urban intervention exposure, and apply analytic methods for natural experiment studies including interrupted time series analysis.

DISCUSSION

INTERACT aims to advance the evidence base on population health intervention research and address challenges related to big data, knowledge mobilization and engagement, ethics, and causality. We will collect ~ 100 TB of sensor data from participants over 5 years. We will address these challenges using interdisciplinary partnerships, training of highly qualified personnel, and modern methodologies for using sensor-based data.