Evaluation of systems-oriented public health interventions: alternative research designs

The Limitations and Potentials of Evaluating Economic Aspects of Community-Based Health Promotion: A Critical Review

Community-based health promotion (CBHP) interventions are promising approaches to address public health problems; however, their economic evaluation presents unique challenges. This review aims to explore the opportunities and limitations of evaluating economic aspects of CBHP, focusing on the assessment of intervention costs and outcomes, and the consideration of political-level changes and health equity. A systematic search of the PubMed, Web of Science and PsycInfo databases identified 24 CBHP interventions, the majority of which targeted disadvantaged communities. Only five interventions included a detailed cost/resource assessment. Outcomes at the operational level were mainly quantitative, related to sociodemographics and environment or health status, while outcomes at the political level were often qualitative, related to public policy, capacity building or networks/collaboration. The study highlights the limitations of traditional health economic evaluation methods in capturing the complexity of CBHP interventions. It proposes the use of cost-consequence analysis (CCA) as a more comprehensive approach, offering a flexible and multifaceted assessment of costs and outcomes. However, challenges remain in the measurement and valuation of outcomes, equity considerations, intersectoral costs and attribution of effects. While CCA is a promising starting point, further research and methodological advancements are needed to refine its application and improve decision making in CBHP.

Health Economic Evaluation of a Controlled Lifestyle Intervention: The Healthy Lifestyle Community Program (Cohort 2; HLCP-2)

Non-communicable diseases (NCD) are associated with high costs for healthcare systems. We evaluated changes in total costs, comprising direct and indirect costs, due to a 24-month non-randomized, controlled lifestyle intervention trial with six measurement time points aiming to improve the risk profile for NCDs. Overall, 187 individuals from the general population aged ≥18 years were assigned to either the intervention group (IG; n = 112), receiving a 10-week intensive lifestyle intervention focusing on a healthy, plant-based diet; physical activity; stress management; and community support, followed by a 22-month follow-up phase including monthly seminars, or a control group (CG; n = 75) without intervention. The complete data sets of 118 participants (IG: n = 79; CG: n = 39) were analyzed. At baseline, total costs per person amounted to 67.80 ± 69.17 EUR in the IG and 48.73 ± 54.41 EUR in the CG per week. The reduction in total costs was significantly greater in the IG compared to the CG after 10 weeks (p = 0.012) and 6 months (p = 0.004), whereas direct costs differed significantly after 10 weeks (p = 0.017), 6 months (p = 0.041) and 12 months (p = 0.012) between the groups. The HLCP-2 was able to reduce health-related economic costs, primarily due to the reduction in direct costs.

Work as a social determinant of health in high-income countries: past, present, and future

This paper, the first in a three-part Series on work and health, provides a narrative review of research into work as a social determinant of health over the past 25 years, the key emerging challenges in this field, and the implications of these challenges for future research. By use of a conceptual framework for work as a social determinant of health, we identified six emerging challenges: (1) the influence of technology on the nature of work in high-income countries, culminating in the sudden shift to telework during the COVID-19 pandemic; (2) the intersectionality of work with gender, sexual orientation, age, race, ethnicity, migrant status, and socioeconomic status as codeterminants of health disparities; (3) the arrival in many Organisation for Economic Co-operation and Development countries of large migrant labour workforces, who are often subject to adverse working conditions and social exclusion; (4) the development of precarious employment as a feature of many national labour markets; (5) the phenomenon of working long and irregular hours with potential health consequences; and (6) the looming threat of climate change's effects on work. We conclude that profound changes in the nature and availability of work over the past few decades have led to widespread new psychosocial and physical exposures that are associated with adverse health outcomes and contribute to increasing disparities in health. These new exposures at work will require novel and creative methods of data collection for monitoring of their potential health impacts to protect the workforce, and for new research into better means of occupational health promotion and protection. There is also an urgent need for a better integration of occupational health within public health, medicine, the life sciences, and the social sciences, with the work environment explicitly conceptualised as a major social determinant of health.

Choice architecture interventions promoting sustained healthier food choice and consumption by students in a secondary school setting: a systematic review of intervention studies

OBJECTIVE

To systematically review the impact of choice architecture interventions (CAI) on the food choice of healthy adolescents in a secondary school setting. Factors potentially contributing to the effectiveness of CAI types and numbers implemented and its long-term success were examined.

DESIGN

PUBMED and Web of Science were systematically searched in October 2021. Publications were included following predefined inclusion criteria and grouped according to the number and duration of implemented interventions. Intervention impact was determined by a systematic description of the reported quantitative changes in food choice and/or consumption. Intervention types were compared with regard to food selection and sustained effects either during or following the intervention.

SETTING

CAI on food choice of healthy adolescents in secondary schools.

PARTICIPANTS

Not applicable.

RESULTS

Fourteen studies were included; four randomised controlled trials and five each of controlled or uncontrolled pre-post design, respectively. Four studies implemented a single CAI type, with ten implementing > 1. Three studies investigated CAI effects over the course of a school year either by continuous or repeated data collection, while ten studies' schools were visited on selected days during an intervention. Twelve studies reported desired changes in overall food selection, yet effects were not always significant and appeared less conclusive for longer-term studies.

CONCLUSIONS

This review found promising evidence that CAI can be effective in encouraging favourable food choices in healthy adolescents in a secondary school setting. However, further studies designed to evaluate complex interventions are needed.

Implementing Technologies: Assessment of Telemedicine Experiments in the Paris Region: Reasons for Success or Failure of the Evaluations and of the Deployment of the Projects

BACKGROUND

Telemedicine is increasingly viewed as a tool to provide a wide range of health services. This article presents policy lessons drawn from the evaluation of telemedicine experiments conducted in the Paris region.

METHODS

We used a mixed method design to study telemedicine projects commissioned by the Paris Regional Health Agency between 2013 and 2017. We combined data analysis of the telemedicine projects, review of the protocols, and interviews with stakeholders.

RESULTS

We identified the following reasons for disappointing outcomes: the outcome measure was requested too early during the experiments because payers required information for budgetary decisions; and the learning curve, technical problems, diversion of use, insufficient number of inclusions, and a lack of adherence prevented the demonstration of successful outcomes of the projects.

CONCLUSION

The evaluation of telemedicine should be undertaken after sufficient uptake to ensure barriers to implementation are overcome, and to obtain the sample size necessary for statistical power and reduce the average cost for one telemedicine request. Randomized controlled trials should be encouraged with appropriate funding and the follow-up period should be extended.

Transformational Performance Improvement: Why Is Progress so Slow?

In this chapter, we identify three distinct transformational performance improvement (TPI) approaches commonly used to redesign work processes in health care organizations. We describe the unique components or tools that each approach uses to improve the delivery of health services. We also summarize what is empirically known about the effectiveness of each TPI approach according to systematic reviews and recent studies published in the peer-reviewed literature. Based on examination of this research, we discuss what knowledge is still needed to strengthen the evidence for whole system transformation. This involves the use of conceptual frameworks to assess and guide implementation efforts, and facilitators and barriers to change as revealed in a recent evaluation of one major initiative, the Lean Enterprise Transformation (LET) at the Veterans Health Administration. The analysis suggests ways in which TPI facilitators can be developed and barriers reduced to improve the effectiveness and sustainability of quality initiatives. Finally, we discuss appropriate study designs to evaluate TPI interventions that may strengthen the evidence for their effectiveness in real world practice settings.

Determining County-Level Counterfactuals for Evaluation of Population Health Interventions: A Novel Application of K-Means Cluster Analysis

OBJECTIVES

Evaluating population health initiatives at the community level necessitates valid counterfactual communities, which includes having similar population composition, health care access, and health determinants. Estimating appropriate county counterfactuals is challenging in states with large intercounty variation. We describe an application of K-means cluster analysis for determining county-level counterfactuals in an evaluation of an intervention, a county perinatal system of care for Medicaid-insured pregnant women.

METHODS

We described counties by using indicators from the American Community Survey, Area Health Resources Files, University of Wisconsin Population Health Institute County Health Rankings, and vital records for Michigan Medicaid-insured births for 2009, the year the intervention began (or the closest available year). We ran analyses of 1000 iterations with random starting cluster values for each of a range of number of clusters from 3 to 10 with commonly used variability and reliability measures to identify the optimal number of clusters.

RESULTS

The use of unstandardized features resulted in the grouping of 1 county with the intervention county in all solutions for all iterations and the frequent grouping of 2 additional counties with the intervention county. Standardized features led to no solution, and other distance measures gave mixed results. However, no county was ideal for all subpopulation analyses.

PRACTICE IMPLICATIONS

Although the K-means method was successful at identifying comparison counties, differences between the intervention county and comparison counties remained. This limitation may be specific to the intervention county and the constraints of a within-state study. This method could be more useful when applied to other counties in and outside Michigan.

Potential Determinants of Cardio-Metabolic Risk among Aboriginal and Torres Strait Islander Children and Adolescents: A Systematic Review

Prevention initiatives during childhood and adolescence have great potential to address the health inequities experienced by Aboriginal and Torres Strait Islander (Indigenous) populations in Australia by targeting modifiable risk factors for cardio-metabolic diseases. We aimed to synthesize existing evidence about potential determinants of cardio-metabolic risk markers—obesity, elevated blood pressure, elevated blood glucose, abnormal lipids, or a clustering of these factors known as the metabolic syndrome (MetS)—for Indigenous children and adolescents. We systematically searched six databases for journal articles and three websites for relevant grey literature. Included articles (n = 47) reported associations between exposures (or interventions) and one or more of the risk markers among Indigenous participants aged 0–24 years. Data from 18 distinct studies about 41 exposure–outcome associations were synthesized (by outcome: obesity [n = 18]; blood pressure [n = 9]; glucose, insulin or diabetes [n = 4]; lipids [n = 5]; and MetS [n = 5]). Obesity was associated with each of the other cardio-metabolic risk markers. Larger birth size and higher area-level socioeconomic status were associated with obesity; the latter is opposite to what is observed in the non-Indigenous population. There were major gaps in the evidence for other risk markers, as well as by age group, geography, and exposure type. Screening for risk markers among those with obesity and culturally appropriate obesity prevention initiatives could reduce the burden of cardio-metabolic disease.

Culturally Sensitive Interventions in Pediatric Primary Care Settings: A Systematic Review

CONTEXT

Culturally sensitive interventions in the pediatric primary care setting may help reduce health disparities. Less is known on the development of these interventions, their target groups, and their feasibility, acceptability, and impact on health outcomes.

OBJECTIVE

We conducted a systematic review to describe culturally sensitive interventions developed for the pediatric primary care setting.

DATA SOURCES

PubMed, Web of Science, Cumulative Index to Nursing and Allied Health Literature, and PsycInfo (January 2000 to July 2020).

STUDY SELECTION

Studies were eligible for inclusion if they were (1) original research on an intervention with an evaluation, (2) within a pediatric primary care setting, (3) not limited to education for providers, (4) not limited to interpreter use, and (5) based in the United States.

DATA EXTRACTION

The following were extracted: study topic, study design, intervention, cultural sensitivity strategies and terminology, setting, target group, sample size, feasibility, acceptability, and health outcomes.

RESULTS

Twenty-five studies described 23 interventions targeting a variety of health topics. Multiple cultural sensitivity strategies were used, most commonly sociocultural (83%). Most interventions (57%) were focused on Hispanic/Latino families. Interventions were generally reported as being feasible and acceptable; some also changed health outcomes.

LIMITATIONS

Small samples and heterogenous methods subject to bias were used. Relevant articles may have been missed because of the variety of terms used to describe cultural sensitivity.

CONCLUSIONS

The included articles provide preliminary evidence that culturally sensitive interventions can be feasible and effective and may help eliminate disparities for patients from communities with barriers to equitable care.

Ethical considerations in the design and conduct of a cluster-randomised mycotoxin mitigation trial in Tanzania

Aflatoxins are fungal metabolites that commonly contaminate staple food crops in tropical regions. Acute aflatoxin consumption in very high concentration causes aflatoxicosis and acute liver failure, while chronic, moderate levels of intake cause hepatocellular carcinoma. The effects of frequent moderate- to high-level exposure during infancy, however, is less clearly understood. Half a billion people in low- and middle-income countries continue to be exposed to aflatoxins through dietary consumption, in part because of lack of enforcement of regulatory limits and few feasible long-term mitigation options in these settings. Several epidemiologic studies have shown an association between aflatoxin exposure in infants and young children and growth failure, but strong experimental evidence is lacking. The Mycotoxin Mitigation Trial conducted in Tanzania was a cluster-randomised trial to assess the effect of a reduced aflatoxin diet on linear growth. Prior to the design and implementation of this trial, a group of multi-disciplinary and multi-national scientists reviewed literature in biomedical, public health, environmental health ethics. In this paper we outline the most salient ethical questions and dilemmas in the potential conduct of such a study and describe the ethical precedents and principles that informed our decision-making processes and ultimate study protocol.

Evaluating cost benefits from a heat health warning system in Adelaide, South Australia

OBJECTIVE

To examine the cost benefits of a heat health warning system (HHWS) in South Australia.

METHODS

Information from key agencies was used to estimate the costs associated with the South Australian HHWS, including for three targeted public health interventions. Health cost savings were estimated based on previously reported HHWS-attributable reductions in hospital and emergency department (ED) admissions and ambulance callouts.

RESULTS

The estimated cost for a one-week activation of the HHWS was AU$593,000. Activation costs compare favourably with the potential costs averted through HHWS-attributable reductions in hospital admissions and ambulance callouts with an estimated benefit-cost ratio of 2.0-3.3.

CONCLUSIONS

On the basis of estimated cost benefit, the South Australian HHWS is a no-regret public health response to heatwaves. Implications for public health: As global temperatures rise there are likely to be significant health impacts from more frequent and intense heatwaves. This study indicates that HHWSs incorporating targeted supports for vulnerable groups are likely to be cost-effective public health interventions.

Healthy lifestyle initiatives for increasing fruit and vegetable intake among Aboriginal and Torres Strait Islander peoples: a rapid review

Adequate fruit and vegetable intake is key to reducing chronic disease risk among Australian Aboriginal and Torres Strait Islander peoples. This rapid review collated evidence on healthy lifestyle initiatives that focused on increasing fruit and vegetable intake among Australian Aboriginal and Torres Strait Islander peoples residing in major cities. Due to limited studies conducted within major cities, we extended our inclusion criteria to regional and remote areas. Sixteen studies were included. Five (31%) studies were rated as good quality (least risk of bias), ten (63%) were rated as fair, and one (6%) was rated as poor (significant risk of bias). Five (31%) studies employed participatory research in the design and/or execution, and seven (44%) studies included minimal community involvement. Only five (31%) studies were undertaken in major cities; four of these combined major cities with regional and/or remote areas. All five studies reported positive findings, such as an increase in fresh fruit availability, usage of fresh vegetables, or self-reported fruit and vegetable intake. This review provides evidence confirming the need for high-quality healthy lifestyle initiatives to increase fruit and vegetable intake targeted at Aboriginal and Torres Strait Islander peoples living in major cities. This evidence will assist community organisations in designing effective health promotion interventions, providing insight into improving the structure and function of such programs. PROSPERO registration number: CRD42020194522 Novelty • Five studies were undertaken in major cities and all reported positive findings; only one study was rated as good quality. • Presented data supports the need for high-quality studies to be conducted among those residing in major cities.

Implementation of medicinal cannabis in Australia: innovation or upheaval? Perspectives from physicians as key informants, a qualitative analysis

OBJECTIVE

We sought to explore physician perspectives on the prescribing of cannabinoids to patients to gain a deeper understanding of the issues faced by prescriber and public health advisors in the rollout of medicinal cannabis.

DESIGN

A thematic qualitative analysis of 21 in-depth interviews was undertaken to explore the narrative on the policy and practice of medicinal cannabis prescribing. The analysis used the Diffusion of Innovations (DoI) theoretical framework to model the conceptualisation of the rollout of medicinal cannabis in the Australian context.

SETTING

Informants from the states and territories of Victoria, New South Wales, Tasmania, Australian Capital Territory, and Queensland in Australia were invited to participate in interviews to explore the policy and practice of medicinal cannabis prescribing.

PARTICIPANTS

Participants included 21 prescribing and non-prescribing key informants working in the area of neurology, rheumatology, oncology, pain medicine, psychiatry, public health, and general practice.

RESULTS

There was an agreement among many informants that medicinal cannabis is, indeed, a pharmaceutical innovation. From the analysis of the informant interviews, the factors that facilitate the diffusion of medicinal cannabis into clincal practice include the adoption of appropriate regulation, the use of data to evaluate safety and efficacy, improved prescriber education, and the continuous monitoring of product quality and cost. Most informants asserted the widespread assimilation of medicinal cannabis into practice is impeded by a lack of health system antecedents that are required to facilitate safe, effective, and equitable access to medicinal cannabis as a therapeutic.

CONCLUSIONS

This research highlights the tensions that arise and the factors that influence the rollout of cannabis as an unregistered medicine. Addressing these factors is essential for the safe and effective prescribing in contemporary medical practice. The findings from this research provides important evidence on medicinal cannabis as a therapeutic, and also informs the rollout of potential novel therapeutics in the future.

Does implementation of a tailored intervention increase adherence to a National Safety and Quality Standard? A study to improve delirium care

BACKGROUND

Delirium is commonly detected in older people after hip fracture. Delirium is considered to be a multifactorial disorder that is often seen post-operatively (incidence ranging from 35% to 65%). Hospitals in Australia are required to meet eight standards including the comprehensive care standard to be able to maintain their accreditation. The standard includes actions related to falls, pressure injuries, nutrition, mental health, cognitive impairment and end-of-life care. Delirium prevention was identified as an area for improvement in our Orthopaedic unit in a Level 1 University Trauma Centre in Australia. This implementation research project aimed to understand the efficacy of a delirium prevention intervention within an existing orthopaedic speciality care system.

OBJECTIVE

Implementation of the tailored intervention will increase adherence to National Safety and Quality Health Service Standards, thereby reducing rate of delirium.

METHODS

In this study, we used an interrupted time series design to examine changes in practice over time in people admitted to hospital with a hip fracture. Clinical staff caring for patients with hip fracture in an acute care setting in Adelaide, South Australia, participated in the project. In brief, intervention included education, environmental restructuring, change champions, infographics and audit feedback reports. The primary outcome of interest was rate of delirium. The secondary outcome was compliance with the use of delirium 4AT screening tool, duration of delirium and hospital length of stay.

RESULTS

The rate of change per month in patients with delirium decreased significantly by 19.2%. There was no significant change observed in trend for duration of delirium and length of hospital stay between pre-intervention and post-intervention phases. A significant increase in the use of screening tool was observed from 4.7% in the pre-intervention phase to 33.6% in the post-intervention phase.

CONCLUSION

Translation of evidence-based intervention model incorporating well-considered implementation strategies had a mixed impact on decreasing the rate of delirium. The scheduled hospital accreditation enhanced the use of validated screening tool to recognize delirium. This project highlights the importance of aligning implementation goals with the wider goals of the organization as well as making clinicians accountable by consistent auditing.

Impact of strategic use of antiretroviral therapy intervention to the HIV continuum of care in 13 cities in Indonesia: an interrupted time series analysis

Background

In 2013 the Indonesian government introduced the strategic use of antiretroviral therapy (SUFA) initiative of expanding access to HIV test and treatment, to help achieve the UNAIDS 90–90–90 targets. However, there has been no comprehensive evaluation of the impact of this intervention in Indonesia. We conducted an interrupted time series (ITS) analysis across 6-years to assess its immediate and medium-term impact.

Methods

Monthly aggregated HIV data from all HIV care clinics for persons aged ≥ 15 years were collected from 13 pilot cities. The data period encompassed 3-years prior to SUFA (26 Dec 2010–25 Dec 2013) and 3-years post-SUFA (26 Dec 2013–25 Dec 2016). The ITS was performed using a multilevel negative binomial regression model to assess the immediate and trend changes in each stage of the HIV continuum of care.

Results

In the pre-SUFA period, the overall coverage in the respective risk populations for HIV tests, cases, enrolments, eligible cases and ARV initiation were 1.0%, 8.6%, 98.9%, 76.9% and 75.8% respectively. In the post-SUFA period coverage was 3%, 3.8%, 98.6%, 90.3% and 81.2% respectively—with a significant increase in the median number of HIV tests, HIV cases, those eligible for ARV treatment and treatment initiation (p < 0.05 for each). The ITS analysis demonstrated immediate increases in HIV tests (IRR = 1.41, 95% CI 1.25, 1.59; p < 0.001) and an immediate decrease in detected HIV cases per person tested (IRR = 0.77, 95% CI 0.69–0.86; p < 0.001) in the month following commencement of SUFA. There was also a 3% decline in the monthly trend for HIV tests performed (IRR = 0.97; 95% CI 0.97–0.98, p < 0.001), a 1% increase for detected cases (IRR = 1.01, 95% CI 1.0–1.02, p < 0.001), and a 1% decline for treatment initiation (IRR = 0.99,95% CI 0.99–1.0 p < 0.05).

Conclusions

SUFA was associated with an immediate and sustained increase in the absolute number of HIV tests performed, detected HIV cases, and close to complete coverage of detected cases that were enrolled to care and defined as eligible for treatment. However, treatment initiation remained sub-optimal. The findings of this study provide valuable information on the real-world effect of accelerating ARV utilizing Treatment as Prevention for the full HIV continuum of care in limited resource countries.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12981-021-00340-4.

Three Statistical Approaches for Assessment of Intervention Effects: A Primer for Practitioners

INTRODUCTION

Statistical methods to assess the impact of an intervention are increasingly used in clinical research settings. However, a comprehensive review of the methods geared toward practitioners is not yet available.

METHODS AND MATERIALS

We provide a comprehensive review of three methods to assess the impact of an intervention: difference-in-differences (DID), segmented regression of interrupted time series (ITS), and interventional autoregressive integrated moving average (ARIMA). We also compare the methods, and provide illustration of their use through three important healthcare-related applications.

RESULTS

In the first example, the DID estimate of the difference in health insurance coverage rates between expanded states and unexpanded states in the post-Medicaid expansion period compared to the pre-expansion period was 5.93 (95% CI, 3.99 to 7.89) percentage points. In the second example, a comparative segmented regression of ITS analysis showed that the mean imaging order appropriateness score in the emergency department at a tertiary care hospital exceeded that of the inpatient setting with a level change difference of 0.63 (95% CI, 0.53 to 0.73) and a trend change difference of 0.02 (95% CI, 0.01 to 0.03) after the introduction of a clinical decision support tool. In the third example, the results from an interventional ARIMA analysis show that numbers of creatinine clearance tests decreased significantly within months of the start of eGFR reporting, with a magnitude of drop equal to -0.93 (95% CI, -1.22 to -0.64) tests per 100,000 adults and a rate of drop equal to 0.97 (95% CI, 0.95 to 0.99) tests per 100,000 per adults per month.

DISCUSSION

When choosing the appropriate method to model the intervention effect, it is necessary to consider the structure of the data, the study design, availability of an appropriate comparison group, sample size requirements, whether other interventions occur during the study window, and patterns in the data.

Barriers to mental health care for Australian children and adolescents in 1998 and 2013-2014

OBJECTIVE

To assess changes in barriers to mental health care for children and adolescents over 16 years.

METHODS

We used data from two nationally representative surveys of Australian children and adolescents (4-17 years old), conducted in 1998 (N = 4509) and 2013-2014 (N = 6310). Barriers to care were assessed among parents who had reported a perceived partially met or unmet need for their child in the past 6 months in 1998, and the past 12 months in 2013-2014; barriers were similarly assessed among adolescents in relation to themselves. We addressed measurement inconsistencies between surveys by harmonising barriers to accommodate differences in wording and performing sensitivity analyses among those with a 1-month disorder to equalise the timeframes in which barriers were assessed. To assess change, we examined whether the rank order of the three most commonly endorsed barriers changed and whether the 95% confidence intervals (CI) around their estimates overlapped.

RESULTS

Similar proportions of parents reported a partially met or unmet need in 1998 (12.9%, 95% CI = [11.7, 14.0]) as in 2013-2014 (14.3%, 95% CI = [13.2, 15.3]), but the ratio of unmet to partially met need decreased from 3:1 in 1998 to 1:1 in 2013-2014. Top three parent-endorsed barriers ('self-reliance', 'unsure where to get help', and 'cost') were the same at both time points; 'self-reliance' decreased from 65.9% (95% CI = [61.1%, 70.7%]) to 34.9% (95% CI = [31.5%, 38.3%]). Top two adolescent-endorsed barriers ('self-reliance' and 'concerned what others might think') were the same at both time points, the third differed, but none of them decreased.

CONCLUSION

Perceived unmet need for mental health care for children and adolescents may have decreased between 1998 and 2013-2014, but the gap in receiving sufficient care may have increased. Despite investments in community awareness and treatment during this period, key barriers seemed largely unchanged. For parents, the decrease in self-reliance may reflect a positive shift in beliefs about the potential benefits of treatment.

Implementing Exercise in Healthcare Settings: The Potential of Implementation Science

Exercise is an efficacious therapy for many chronic diseases. Integrating efficacious evidence-based interventions (EBIs), such as exercise, into daily healthcare practice is a slow and complex pursuit. Implementation science seeks to understand and address this phenomenon by conducting studies about the methods used to promote the routine uptake of EBIs. The purpose of this article is to explore implementation science and a common conceptual framework in the discipline, the Consolidated Framework for Implementation Research (CFIR), as it applies to exercise EBI. We conclude by offering recommendations for future research that leverage implementation science priorities to highlight the potential of this research field for advancing the implementation of exercise EBI.

The Social Determinants of Health: Time to Re-Think?

Twelve years have now passed since the influential WHO Report on the Social Determinants of Health (SDoH) in 2008. A group of senior international public health scholars and decision-makers met in Italy in mid-2019 to review the legacy of the SDoH conceptual framework and its adequacy for the many challenges facing our field as we enter the 2020s. Four major categories of challenges were identified: emerging "exogenous" challenges to global health equity, challenges related to weak policy and practice implementation, more fundamental challenges related to SDoH theory and research, and broader issues around modern research in general. Each of these categories is discussed, and potential solutions offered. We conclude that although the SDoH framework is still a worthy core platform for public health research, policy, and practice, the time is ripe for significant evolution.

Economic evaluations of public health interventions for physical activity and healthy diet: A systematic review

Physical inactivity and unhealthy dietary habits are associated with an increased disease and economic burden. The aim of this systematic review was to identify economic evaluations of public health interventions targeting physical activity and healthy diet, and assess the quality and transferability of the findings to the Swedish context. A search of published economic evaluations was conducted through electronic databases including PubMed, Web of Science, PsycINFO, National Health Service Economic Evaluation Databases (NHS EED) and the Health Technology Assessment Database (HTA). An additional search was done using references of relevant systematic reviews and websites of relevant organizations were checked to find grey literature. Quality and transferability of the economic evaluations were appraised using a quality assessment tool developed by the Swedish Agency for Health Technology Assessment. Thirty-two economic evaluations, rated as moderate or high quality, of 178 interventions were included; thirteen studies targeting physical activity, thirteen targeting healthy diet and six targeting both. The interventions varied in terms of their content, setting, mode of delivery and target populations. A majority of the economic evaluations reported that the interventions were likely to be cost-effective; however, considerable variations in the methodological and reporting qualities were observed. Only half of the economic evaluations were rated to have a high probability of transferring to the Swedish context. Public health interventions targeting physical activity and dietary habits have a high potential to be cost-effective. However, decision makers should consider the variation in quality and transferability of the available evidence.

Mobilizing Breast Cancer Prevention Research Through Smartphone Apps: A Systematic Review of the Literature

Background: Breast cancer rates have been increasing worldwide, particularly among young women, suggesting important interactions between genes and health behaviors. At the same time, mobile technology, including smartphones applications (apps), has emerged as a new tool for delivering healthcare and health-related services. As of 2018, there were nearly 600 publicly available breast cancer apps designed to provide disease and treatment information, to manage disease, and to raise overall awareness. However, the extent to which apps are incorporated into breast cancer prevention research is unknown. Therefore, the objective of this review was to determine how mobile applications are being used for breast cancer prevention among women across the cancer control continuum. Methods: Using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, we searched PubMed and Web of Science Core Collection databases using the keywords breast cancer, smartphone, mobile application, and phone app. Full-length journal articles available in English that addressed the research question were included. We categorized articles by prevention type (primary, secondary, and tertiary) and phase of research (protocol, development, feasibility, pilot, measurement, and effectiveness), and identified common themes and gaps. Results: Our search yielded 82 studies (69 unique) that used apps in breast cancer prevention research across 20 countries. Approximately half of the named apps were publicly available. The majority (73%) of studies targeted tertiary prevention; 15% targeted secondary and 13% targeted primary prevention. Apps were used across all phases of research with the predominant phase being feasibility in tertiary prevention (34%), effectiveness in secondary prevention (63%), and development (30%) and effectiveness (30%) in primary prevention. Common uses included assessing outcomes relevant to clinical care coordination, quality of life, increasing self-efficacy and screening behaviors, and tracking and managing health behaviors. Conclusions: We identified the following gaps: few effectiveness studies in tertiary prevention, minimal use of apps for breast cancer etiology or early detection, and few interventions in those at average risk of breast cancer. These findings suggest that while mobile apps can inform breast cancer prevention across the continuum, more work is needed to incorporate apps into primary prevention.

Moving Upstream: The Effect of Tobacco Clean Air Restrictions on Educational Inequalities in Smoking Among Young Adults

Education affords a range of direct and indirect benefits that promote longer and healthier lives and stratify health lifestyles. We use tobacco clean air policies to examine whether policies that apply universally-interventions that bypass individuals' unequal access and ability to employ flexible resources to avoid health hazards-have an effect on educational inequalities in health behaviors. We test theoretically informed but competing hypotheses that these policies either amplify or attenuate the association between education and smoking behavior. Our results provide evidence that interventions that move upstream to apply universally regardless of individual educational attainment-here, tobacco clean air policies-are particularly effective among young adults with the lowest levels of parental or individual educational attainment. These findings provide important evidence that upstream approaches may disrupt persistent educational inequalities in health behaviors. In doing so, they provide opportunities to intervene on behaviors in early adulthood that contribute to disparities in morbidity and mortality later in the life course. These findings also help assuage concerns that tobacco clean air policies increase educational inequalities in smoking by stigmatizing those with the fewest resources.

Bridging the academic and practice/policy gap in public health: perspectives from Scotland and Canada

This article presents a critical commentary of specific organizational models and practices for bridging 'the gap' between public health research and policy and practice. The authors draw on personal experiences of such models in addition to the wider knowledge translation and exchange literature to reflect on their strengths and weaknesses as implemented in Scotland and Canada since the early 1990s.

Heading in a new direction? Recommendations for future research on patient decision aids

Decision aids can improve a number of patient outcomes, but they are not commonly used in clinical practice. This commentary paper provides suggestions for potential next steps of decision aid research, with the aim to facilitate their implementation. We suggest to further standardise clinically meaningful outcomes and outcome measures that should be used to examine the impact of decision aids. Second, using mediation analysis and active control groups could help tease out and explore variables that influence decision aids' effectiveness to help healthcare providers decide when and how to use them in clinical practice. Third, effectiveness trials should be clearly reported and replicated to investigate under what circumstances decision aids work best. Specific checklists for decision aid trials should be used to ensure that all relevant factors are reported in detail. Addressing the above issues will help identify what specific components of decision aids are effective and should be implemented. We can then move towards conducting implementation trials which help increase the use of decision aids in "real-world" healthcare.

Helping Survivors of Human Trafficking: A Systematic Review of Exit and Postexit Interventions

BACKGROUND

Human trafficking is a global problem and results in deleterious psychological, social, and physical effects on the lives of those who are trafficked; however, it is not clear how to best intervene with survivors. The purpose of this review was to synthesize the evidence of exit and postexit intervention programs for survivors of human trafficking to inform practice and research.

METHOD

Systematic review methods were used to search, select, and extract data from published and unpublished experimental, quasi-experimental, and preexperimental studies that assessed the effects of any exit or postexit interventions for victims of human trafficking. The authors searched eight databases, reviewed bibliographies, and conducted forward citation searches from relevant reports and prior reviews to find studies authored between 2005 and 2015.

RESULTS

The search yielded six eligible studies that included 155 female and 6 male survivors from four countries. Interventions were diverse, with three using a trauma-informed approach. Authors measured a myriad of outcomes, including mental health, social network, community reintegration, and employment; however, the quality of most studies was poor.

DISCUSSION

Evidence of effects of exit and postexit interventions is sparse, and much of the research is poorly designed and executed; however, the needs of trafficking survivors are complex and effective interventions are desperately needed. Implications for practice and research are discussed.

Improving paediatric pain management in the emergency department: An integrative literature review

BACKGROUND

Children presenting to the emergency department continue to experience suboptimal pain management. While evidence-based pain management interventions are available to clinicians, effective and sustainable practice change is yet to be achieved. This practice gap requires a collaborative approach to knowledge translation targeting systems of care.

OBJECTIVES

The purpose of this review was to explore systems level change in the emergency department for improved paediatric pain management.

DESIGN

Integrative review.

DATA SOURCES

CINAHL, Embase, PubMed/Medline, Dynamed, Cochrane, Scopus, Prospero and Joanna Briggs Institute were systematically searched, and clinical guidelines and reference lists scanned.

REVIEW METHODS

Studies were screened and selected according to the inclusion criteria, and independently appraised for risk of bias. Integrative review methodology informed data extraction and synthesis, focused on organisational context and engagement, facilitation and implementation of practice change, key components of the pain management interventions, and evaluation.

RESULTS

Twenty studies met the inclusion criteria: 18 uncontrolled pretest-posttest and two pseudo-randomised design. Study populations ranged from children with a specific presentation, to all presenting children. All studies adopted a multifaceted approach to organisational change, bundling various interventions including pain assessment tools and management protocols, clinician education, nurse-initiated analgesia, feedback and family engagement. Four studies used local systems analysis to inform interventions and two studies applied an implementation framework. Time to analgesia was the most commonly improved primary outcome. Parent and child sensitive outcomes were assessed in five studies. Interventions that hold the most promise for optimised pain management if embedded in the workplace include nurse-initiated analgesia and family involvement at each stage of pain management in the emergency department.

CONCLUSION

The way forward is to respectfully engage all stakeholders-children, parents and clinicians-to collaboratively develop evidence-based, sustainable solutions aligned with the emergency department context. Guided by an implementation framework, future research designed to creatively translate evidence into practice and facilitate change at a systems level is a priority. Key to this solution is the integration of family involvement in pain management, considering child and family sensitive outcome measures. Effectiveness of new interventions should be evaluated in the short and long term to embed sustainable practice change. Frontline nurses are well placed to lead this transformation in paediatric pain management in the emergency department.

The new challenge for improving psychosocial cancer care: shifting to a system-based approach

INTRODUCTION

There is a need to improve the psychosocial well-being of cancer patients. To date, intervention research has primarily focussed on improving psychosocial well-being through targeting singular aspects of care at the individual patient level. Sustainable, high-quality psychosocial care should address the issues faced by people diagnosed with cancer throughout the care pathway using a system-based approach.

AIMS

To examine the number of intervention trials attempting to improve psychosocial cancer care that have implemented a system-based approach.

METHOD

Five journals were selected and relevant studies across all years were extracted. Four criteria, argued to be essential characteristics of system-based change, were assessed: (1) establishing a culture change within the healthcare system/organisation, through designated leaders who endorse organisational goals; (2) adopting a multidisciplinary approach to change; (3) mapping the system and identifying points of leverage; and (4) measuring the impact of change and adapting establish feedback loops.

RESULTS

The search strategy returned 1174 citations, of which five met the inclusion criteria. Of the intervention studies identified, three met none of the four defined criteria for a systems-based intervention, one study met criterion 2 only, and one study met all four criteria, however, was not a rigorous study design.

CONCLUSIONS

This review of published psychosocial intervention trials in top-ranking psychosocial cancer care journals only found one study that met our criteria for evaluating system-based change. This is likely to be a consequence of the significant pragmatic and political barriers to conducting system-based intervention research.

Barriers to the evaluation of evidence-based public health policy

Public health policy has the potential to produce great benefits for individuals and communities. There is growing demand that such efforts be rigorously evaluated to ensure that the expected benefits are, in fact, realised. Commonly, public health policy is evaluated by consumer acceptability, reach, or changes in knowledge and attitudes. Non-robust research designs are often used. But these approaches to evaluation do not answer three critical questions: Has a change in the desired outcome occurred? Was it a consequence of the policy and not some extraneous factor? Was the size of the change considered significant and cost-effective? We, a team of government and academic scholars working in research and evaluation, have examined some of the more common impediments to robust evaluation: political impediments, a lack of investment in evaluation capacity within bureaucracy, and the failure of academic researchers to understand the need for the evaluation of public health policy.

Comparison of registered and published intervention fidelity assessment in cluster randomised trials of public health interventions in low- and middle-income countries: systematic review

BACKGROUND

Cluster randomised trials (CRTs) are a key instrument to evaluate public health interventions. Fidelity assessment examines study processes to gauge whether an intervention was delivered as initially planned. Evaluation of implementation fidelity (IF) is required to establish whether the measured effects of a trial are due to the intervention itself and may be particularly important for CRTs of complex interventions conducted in low- and middle-income countries (LMICs). However, current CRT reporting guidelines offer no guidance on IF assessment. The objective of this review was to study current practices concerning the assessment of IF in CRTs of public health interventions in LMICs.

METHODS

CRTs of public health interventions in LMICs that planned or reported IF assessment in either the trial protocol or the main trial report were included. The MEDLINE/PubMed, CINAHL and EMBASE databases were queried from January 2012 to May 2016. To ensure availability of a study protocol, CRTs reporting a registration number in the abstract were included. Relevant data were extracted from each study protocol and trial report by two researchers using a predefined screening sheet. Risk of bias for individual studies was assessed.

RESULTS

We identified 90 CRTs of public health interventions in LMICs with a study protocol in a publicly available trial registry published from January 2012 to May 2016. Among these 90 studies, 25 (28%) did not plan or report assessing IF; the remaining 65 studies (72%) addressed at least one IF dimension. IF assessment was planned in 40% (36/90) of trial protocols and reported in 71.1% (64/90) of trial reports. The proportion of overall agreement between the trial protocol and trial report concerning occurrence of IF assessment was 66.7% (60/90). Most studies had low to moderate risk of bias.

CONCLUSIONS

IF assessment is not currently a systematic practice in CRTs of public health interventions carried out in LMICs. In the absence of IF assessment, it may be difficult to determine if CRT results are due to the intervention design, to its implementation, or to unknown or external factors that may influence results. CRT reporting guidelines should promote IF assessment.

TRIAL REGISTRATION

Protocol published and available at: https://doi.org/10.1186/s13643-016-0351-0.

Increasing advance personal planning: the need for action at the community level

BACKGROUND

Advance personal planning is the process by which people consider, document and communicate their preferences for personal, financial and health matters in case they lose the ability to make decisions or express their wishes in the future.

DISCUSSION

Advance personal planning is most often undertaken by individuals who are seriously ill, often in the context of a medical crisis and/or at the time of admission to hospital. However, the clinical utility and legal validity of the planning process may be compromised in these circumstances. Patients may lack sufficient capacity to meaningfully engage in advance personal planning; there may be insufficient time to adequately reflect on and discuss wishes with key others; and there may also be limited opportunity for inter-professional input and collaboration in the process. Here, we propose an agenda for research to advance the science of advance personal planning by promoting a 'whole community' approach. Adoption of advance personal planning at a community level may be achieved using a variety of strategies including public media campaigns, intervening with professionals across a range of health care and legal settings, and mobilising support from influential groups and local government. One potentially promising method for encouraging earlier adoption of advance personal planning among a broader population involves a community action approach, whereby multiple evidence-based strategies are integrated across multiple access points. Community action involves calling on community members, professionals, community and/or government organisations to work collaboratively to design and systematically implement intervention strategies with the aim of bringing about desired behaviour change. An example of a community action trial to improving uptake and quality of advance personal planning is described.

CONCLUSION

While promising, there is a need for rigorous evidence to demonstrate whether a community action approach is effective in establishing whole community adoption of advance personal planning.

The impact of a community-led program promoting weight loss and healthy living in Aboriginal communities: the New South Wales Knockout Health Challenge

Background

Aboriginal people in Australia experience significant health burden from chronic disease. There has been limited research to identify effective healthy lifestyle programs to address risk factors for chronic disease among Aboriginal people.

Methods

The Knockout Health Challenge is a community-led healthy lifestyle program for Aboriginal communities across New South Wales, Australia. An evaluation of the 2013 Knockout Health Challenge was undertaken. Participants’ self-reported physical activity and diet were measured at four time points - at the start and end of the Challenge (via paper form), and 5 and 9 months after the Challenge (via telephone survey). Participants’ weight was measured objectively at the start and end of the Challenge, and self-reported (via telephone survey) 5 and 9 months after the Challenge. Changes in body composition, physical activity and diet between time points were analysed using linear mixed models. As part of the telephone survey participants were also asked to identify other impacts of the Challenge; these were analysed descriptively (quantitative items) and thematically (qualitative items).

Results

A total of 586 people registered in 22 teams to participate in the Challenge. The mean weight at the start was 98.54kg (SD 22.4), and 94% of participants were overweight or obese. Among participants who provided data at all four time points (n=122), the mean weight loss from the start to the end of the Challenge was 2.3kg (95%CI -3.0 to -1.9, p<0.001), and from the start to 9 months after the Challenge was 2.3kg (95%CI -3.3 to -1.3, p<0.001). Body mass index decreased by an average of 0.9kg/m² (95%CI -1.0 to -0.7, p<0.001) from the start to the end of the Challenge, and 0.8kg/m² (95%CI -1.2 to -0.4, p<0.001) 9 months after. At the end of the Challenge, participants reported they were more physically active and had increased fruit and vegetable consumption compared with the start of the Challenge, and identified a range of other positive impacts.

Conclusions

The Challenge was effective in reducing weight and promoting healthy lifestyles among Aboriginal people across New South Wales, and has potential to contribute to closing the health gap between Aboriginal and non-Aboriginal people.

Improving hospital-based end of life care processes and outcomes: a systematic review of research output, quality and effectiveness

BACKGROUND

As in other areas of health delivery, there is a need to ensure that end-of-life care is guided by patient centred research. A systematic review was undertaken to examine the quantity and quality of data-based research aimed at improving the (a) processes and (b) outcomes associated with delivering end-of-life care in hospital settings.

METHODS

Medline, EMBASE and Cochrane databases were searched between 1995 and 2015 for data-based papers. Eligible papers were classified as descriptive, measurement or intervention studies. Intervention studies were categorised according to whether the primary aim was to improve: (a) end of life processes (i.e. end-of-life documentation and discussions, referrals); or (b) end-of-life outcomes (i.e. perceived quality of life, health status, health care use, costs). Intervention studies were assessed against the Effective Practice and Organisation of Care methodological criteria for research design, and their effectiveness examined.

RESULTS

A total of 416 papers met eligibility criteria. The number increased by 13% each year (p < 0.001). Most studies were descriptive (n = 351, 85%), with fewer measurement (n = 17) and intervention studies (n = 48; 10%). Only 18 intervention studies (4%) met EPOC design criteria. Most reported benefits for end-of-life processes including end-of-life discussions and documentation (9/11). Impact on end-of-life outcomes was mixed, with some benefit for psychosocial distress, satisfaction and concordance in care (3/7).

CONCLUSION

More methodologically robust studies are needed to evaluate the impact of interventions on end-of-life processes, including whether changes in processes translate to improved end-of-life outcomes. Interventions which target both the patient and substitute decision maker in an effort to achieve these changes would be beneficial.

Practice-Based Evidence in Community Guide Systematic Reviews

OBJECTIVES

To assess the relative contributions and quality of practice-based evidence (PBE) and research-based evidence (RBE) in The Guide to Community Preventive Services (The Community Guide).

METHODS

We developed operational definitions for PBE and RBE in which the main distinguishing feature was whether allocation of participants to intervention and comparison conditions was under the control of researchers (RBE) or not (PBE). We conceptualized a continuum between RBE and PBE. We then categorized 3656 studies in 202 reviews completed since The Community Guide began in 1996.

RESULTS

Fifty-four percent of studies were PBE and 46% RBE. Community-based and policy reviews had more PBE. Health care system and programmatic reviews had more RBE. The majority of both PBE and RBE studies were of high quality according to Community Guide scoring methods.

CONCLUSIONS

The inclusion of substantial PBE in Community Guide reviews suggests that evidence of adequate rigor to inform practice is being produced. This should increase stakeholders' confidence that The Community Guide provides recommendations with real-world relevance. Limitations in some PBE studies suggest a need for strengthening practice-relevant designs and external validity reporting standards.

Natural Experiments: An Overview of Methods, Approaches, and Contributions to Public Health Intervention Research

Population health interventions are essential to reduce health inequalities and tackle other public health priorities, but they are not always amenable to experimental manipulation. Natural experiment (NE) approaches are attracting growing interest as a way of providing evidence in such circumstances. One key challenge in evaluating NEs is selective exposure to the intervention. Studies should be based on a clear theoretical understanding of the processes that determine exposure. Even if the observed effects are large and rapidly follow implementation, confidence in attributing these effects to the intervention can be improved by carefully considering alternative explanations. Causal inference can be strengthened by including additional design features alongside the principal method of effect estimation. NE studies often rely on existing (including routinely collected) data. Investment in such data sources and the infrastructure for linking exposure and outcome data is essential if the potential for such studies to inform decision making is to be realized.

Designing and Undertaking a Health Economics Study of Digital Health Interventions

This paper introduces and discusses key issues in the economic evaluation of digital health interventions. The purpose is to stimulate debate so that existing economic techniques may be refined or new methods developed. The paper does not seek to provide definitive guidance on appropriate methods of economic analysis for digital health interventions. This paper describes existing guides and analytic frameworks that have been suggested for the economic evaluation of healthcare interventions. Using selected examples of digital health interventions, it assesses how well existing guides and frameworks align to digital health interventions. It shows that digital health interventions may be best characterized as complex interventions in complex systems. Key features of complexity relate to intervention complexity, outcome complexity, and causal pathway complexity, with much of this driven by iterative intervention development over time and uncertainty regarding likely reach of the interventions among the relevant population. These characteristics imply that more-complex methods of economic evaluation are likely to be better able to capture fully the impact of the intervention on costs and benefits over the appropriate time horizon. This complexity includes wider measurement of costs and benefits, and a modeling framework that is able to capture dynamic interactions among the intervention, the population of interest, and the environment. The authors recommend that future research should develop and apply more-flexible modeling techniques to allow better prediction of the interdependency between interventions and important environmental influences.

Implementation of a pragmatic, stepped-wedge cluster randomized trial to evaluate impact of Botswana's Xpert MTB/RIF diagnostic algorithm on TB diagnostic sensitivity and early antiretroviral therapy mortality

BACKGROUND

In 2012, as a pilot for Botswana's national Xpert MTB/RIF (Xpert) rollout plans, intensified tuberculosis (TB) case finding (ICF) activities were strengthened at 22 HIV treatment clinics prior to phased activation of 13 Xpert instruments. Together, the strengthened ICF intervention and Xpert activation are referred to as the "Xpert package".

METHODS

The evaluation, called the Xpert Package Rollout Evaluation using a Stepped-wedge design (XPRES), has two key objectives: (1) to compare sensitivity of microscopy-based and Xpert-based pulmonary TB diagnostic algorithms in diagnosing sputum culture-positive TB; and (2) to evaluate impact of the "Xpert package" on all-cause, 6-month, adult antiretroviral therapy (ART) mortality. A pragmatic, stepped-wedge cluster-randomized trial design was chosen. The design involves enrollment of three cohorts: (1) cohort R, a retrospective cohort of all study clinic ART enrollees in the 24 months before study initiation (July 31, 2012); (2) cohort A, a prospective cohort of all consenting patients presenting to study clinics after study initiation, who received the ICF intervention and the microscopy-based TB diagnostic algorithm; and (3) cohort B, a prospective cohort of all consenting patients presenting to study clinics after Xpert activation, who received the ICF intervention and the Xpert-based TB diagnostic algorithm. TB diagnostic sensitivity will be compared between TB culture-positive enrollees in cohorts A and B. All-cause, 6-month ART-mortality will be compared between cohorts R and B. With anticipated cohort R, A, and B sample sizes of about 10,131, 1,878, and 4,258, respectively, the study is estimated to have >80 % power to detect differences in pre-versus post-Xpert TB diagnostic sensitivity if pre-Xpert sensitivity is ≤52.5 % and post-Xpert sensitivity ≥82.5 %, and >80 % power to detect a 40 % reduction in all-cause, 6-month, ART mortality between cohorts R and B if cohort R mortality is ≥13/100 person-years.

DISCUSSION

Only one small previous trial (N = 424) among ART enrolees in Zimbabwe evaluated, in a secondary analysis, Xpert impact on all-cause 6-month ART mortality. No mortality impact was observed. This Botswana trial, with its larger sample size and powered specifically to detect differences in all-cause 6-month ART mortality, remains well-positioned to contribute understanding of Xpert impact.

TRIAL REGISTRATION

Retrospectively registered at ClinicalTrials.gov: NCT02538952 .

Comparison of registered and published intervention fidelity assessment in cluster randomised trials of public health interventions in low- and middle-income countries: systematic review protocol

BACKGROUND

Cluster randomised trials (CRTs) are a key instrument to evaluate public health interventions, particularly in low- and middle-income countries (LMICs). Fidelity assessment examines study processes to gauge whether an intervention was delivered as initially planned. Evaluation of implementation fidelity (IF) is required to establish whether the measured effects of a trial are due to the intervention itself and may be particularly important for CRTs of complex interventions. Current CRT reporting guidelines offer no guidance on IF assessment. We will systematically review the scientific literature to study current practices concerning the assessment of IF in CRTs of public health interventions in LMICs.

METHODS

We will include CRTs of public health interventions in LMICs that planned or assessed IF in either the trial protocol or the main trial report (or an associated document). Search strategies use Medical Subject Headings (MESH) and text words related to CRTs, developing countries, and public health interventions. The electronic database search was developed first for MEDLINE and adapted for the following databases: EMBASE, CINAHL, PubMed, and EMB Reviews, to identify CRT reports in English, Spanish, or French published on or after January 1, 2012. To ensure availability of a study protocol, we will include CRTs reporting a registration number in the abstract. For each included study, we will compare planned versus reported assessment of IF, and consider the dimensions of IF studied, and data collection methods used to evaluate each dimension. Data will be synthesised using quantitative and narrative techniques. Risk of bias for individual studies will be assessed using the Cochrane Collaboration Risk of Bias Tool criteria and additional criteria related to CRT methods. We will investigate possible sources of heterogeneity by performing subgroup analysis. This review was not eligible for inclusion in the PROSPERO registry.

DISCUSSION

Fidelity assessment may be a key tool for making studies more reliable, internally valid, and externally generalizable. This review will provide a portrait of current practices related to the assessment of intervention fidelity in CRTs and offer suggestions for improvement. Results will be relevant to researchers, those who finance health interventions, and for decision-makers who seek the best evidence on public health interventions.

Stepped wedge cluster randomised trials: a review of the statistical methodology used and available

BACKGROUND

Previous reviews have focussed on the rationale for employing the stepped wedge design (SWD), the areas of research to which the design has been applied and the general characteristics of the design. However these did not focus on the statistical methods nor addressed the appropriateness of sample size methods used.This was a review of the literature of the statistical methodology used in stepped wedge cluster randomised trials.

METHODS

Literature Review. The Medline, Embase, PsycINFO, CINAHL and Cochrane databases were searched for methodological guides and RCTs which employed the stepped wedge design.

RESULTS

This review identified 102 trials which employed the stepped wedge design compared to 37 from the most recent review by Beard et al. 2015. Forty six trials were cohort designs and 45 % (n = 46) had fewer than 10 clusters. Of the 42 articles discussing the design methodology 10 covered analysis and seven covered sample size. For cohort stepped wedge designs there was only one paper considering analysis and one considering sample size methods. Most trials employed either a GEE or mixed model approach to analysis (n = 77) but only 22 trials (22 %) estimated sample size in a way which accounted for the stepped wedge design that was subsequently used.

CONCLUSIONS

Many studies which employ the stepped wedge design have few clusters but use methods of analysis which may require more clusters for unbiased and efficient intervention effect estimates. There is the need for research on the minimum number of clusters required for both types of stepped wedge design. Researchers should distinguish in the sample size calculation between cohort and cross sectional stepped wedge designs. Further research is needed on the effect of adjusting for the potential confounding of time on the study power.

Wilfully out of sight? A literature review on the effectiveness of cancer-related decision aids and implementation strategies

Background

There is evidence to suggest that decision aids improve a number of patient outcomes. However, little is known about the progression of research effort in this area over time. This literature review examined the volume of research published in 2000, 2007 and 2014 which tested the effectiveness of decision aids in improving cancer patient outcomes, coded by cancer site and decision type being targeted. These numbers were compared with the volume of research examining the effectiveness of strategies to increase the adoption of decision aids by healthcare providers.

Methods

A literature review of intervention studies was undertaken. Medline, Embase, PsychInfo and Cochrane Database of Systematic Reviews were searched. The search was limited to human studies published in English, French, or German. Abstracts were assessed against eligibility criteria by one reviewer and a random sample of 20 % checked by a second. Eligible intervention studies in the three time periods were categorised by: i) whether they tested the effectiveness of decision aids, coded by cancer site and decision type, and ii) whether they tested strategies to increase healthcare provider adoption of decision aids.

Results

Over the three time points assessed, increasing research effort has been directed towards testing the effectiveness of decision aids in improving patient outcomes (p < 0.0001). The number of studies on decision aids for cancer screening or prevention increased statistically significantly (p < 0.0001) whereas the number of studies on cancer treatment did not (p = 1.00). The majority of studies examined the effectiveness of decision aids for prostate (n = 10), breast (n = 9) or colon cancer (n = 7). Only two studies assessed the effectiveness of implementation strategies to increase healthcare provider adoption of decision aids.

Conclusions

While the number of studies testing the effectiveness of decision aids has increased, the majority of research has focused on screening and prevention decision aids for only a few cancer sites. This neglects a number of cancer populations, as well as other areas of cancer care such as treatment decisions. Also, given the apparent effectiveness of decision aids, more effort needs to be made to implement this evidence into meaningful benefits for patients.

Electronic supplementary material

The online version of this article (doi:10.1186/s12911-016-0273-8) contains supplementary material, which is available to authorized users.

Efficacy and cost effectiveness of telemedicine for improving access to care in the Paris region: study protocols for eight trials

BACKGROUND

With the development of information and communication technologies, telemedicine has been proposed as a way to improve patient management by facilitating access to appropriate diagnosis and treatment. The Paris Ile de France Regional Health Agency is currently funding a comprehensive program of telemedicine experiments. This article describes the protocols for the evaluation of the implementation of telemedicine in the Paris region.

METHODS/DESIGN

Over 2,500 patients have been included in eight studies addressing the use of telemedicine in the context of specific diseases or settings. Two projects are randomized controlled trials, while the six other projects are based on before-after designs (differences in differences studies). Based on the MAST model and the French national framework, we identified endpoints to assess the impact of telemedicine on five dimensions: clinical effectiveness, cost-effectiveness, security of the application, patient satisfaction and quality of life and perception of professionals.

DISCUSSION

Telemedicine encompasses a wide range of services and stakeholders, and thus study protocols must be tailored to the specific constraints and interests of the users.

TRIAL REGISTRATION

NCT02110433 (03/07/2014), NCT02157740 (05/27/2014), NCT02374697 (02/05/2015), NCT02157727 (05/27/2014), NCT02229279 (08/28/2014), NCT02368769 (02/05/2015), NCT02164747 (NCT02164747), NCT02309905 (11/27/2014).

Improved Designs for Cluster Randomized Trials

Studies in which clusters of individuals are randomized to conditions are increasingly common in public health research. However, the designs utilized for such studies are often suboptimal and inefficient. We review strategies to improve the design of cluster randomized trials. We discuss both older but effective design concepts that are underutilized, such as stratification and factorial designs, as well as emergent ideas including fractional factorial designs and cluster randomized crossover studies. We draw examples from the recent literature and provide resources for sample size and power planning. Given the inherent inefficiencies of cluster randomized trials, these design strategies merit wider consideration and can lead to studies that are more cost-effective and potentially more rigorous than traditional approaches.

Designs for Evaluating the Community-Level Impact of Comprehensive Prevention Programs: Examples from the CDC Centers of Excellence in Youth Violence Prevention

This article discusses the opportunities and challenges of developing research designs to evaluate the impact of community-level prevention efforts. To illustrate examples of evaluation designs, we describe six projects funded by the Centers for Disease Control and Prevention to evaluate multifaceted approaches to reduce youth violence in high-risk communities. Each of these projects was designed to evaluate the community-level impact of multiple intervention strategies to address individual and contextual factors that place youth at risk for violent behavior. Communities differed across projects in their setting, size, and how their boundaries were defined. Each project is using multiple approaches to compare outcomes in one or more intervention communities to those in comparison communities. Five of the projects are using comparative interrupted time-series designs to compare outcomes in an intervention community to matched comparison communities. A sixth project is using a multiple baseline design in which the order and timing of intervention activities is randomized across three communities. All six projects are also using regression point displacement designs to compare outcomes within intervention communities to those within broader sets of similar communities. Projects are using a variety of approaches to assess outcomes including archival records, surveys, and direct observations. We discuss the strengths and weaknesses of the designs of these projects and illustrate the challenges of designing high-quality evaluations of comprehensive prevention approaches implemented at the community level.

The need to balance merits and limitations from different disciplines when considering the stepped wedge cluster randomized trial design

Background

Various papers have addressed pros and cons of the stepped wedge cluster randomized trial design (SWD). However, some issues have not or only limitedly been addressed. Our aim was to provide a comprehensive overview of all merits and limitations of the SWD to assist researchers, reviewers and medical ethics committees when deciding on the appropriateness of the SWD for a particular study.

Methods

We performed an initial search to identify articles with a methodological focus on the SWD, and categorized and discussed all reported advantages and disadvantages of the SWD. Additional aspects were identified during multidisciplinary meetings in which ethicists, biostatisticians, clinical epidemiologists and health economists participated. All aspects of the SWD were compared to the parallel group cluster randomized design. We categorized the merits and limitations of the SWD to distinct phases in the design and conduct of such studies, highlighting that their impact may vary depending on the context of the study or that benefits may be offset by drawbacks across study phases. Furthermore, a real-life illustration is provided.

Results

New aspects are identified within all disciplines. Examples of newly identified aspects of an SWD are: the possibility to measure a treatment effect in each cluster to examine the (in)consistency in effects across clusters, the detrimental effect of lower than expected inclusion rates, deviation from the ordinary informed consent process and the question whether studies using the SWD are likely to have sufficient social value. Discussions are provided on e.g. clinical equipoise, social value, health economical decision making, number of study arms, and interim analyses.

Conclusions

Deciding on the use of the SWD involves aspects and considerations from different disciplines not all of which have been discussed before. Pros and cons of this design should be balanced in comparison to other feasible design options as to choose the optimal design for a particular intervention study.

Measuring improvement in populations: implementing and evaluating successful change in lung cancer care

Improving quality of care in lung cancer, the leading cause of cancer death worldwide and in the United States, is a major public health challenge. Such improvement requires accurate and meaningful measurement of quality of care. Preliminary indicators have been derived from clinical practice guidelines and expert opinions, but there are few standard sets of quality of care measures for lung cancer in the United States or elsewhere. Research to develop validated evidence-based quality of care measures is critical in promoting population improvement initiatives in lung cancer. Furthermore, novel research designs beyond the traditional randomized controlled trials (RCTs) are needed for wide-scale applications of quality improvement and should extend into alternative designs such as quasi-experimental designs, rigorous observational studies, population modeling, and other pragmatic study designs. We discuss several study design options to aid the development of practical, actionable, and measurable quality standards for lung cancer care. We also provide examples of ongoing pragmatic studies for the dissemination and implementation of lung cancer quality improvement interventions in community settings.

Implementation of a stepped-wedge cluster randomized design in routine public health practice: design and application for a tuberculosis (TB) household contact study in a high burden area of Lima, Peru

Background

We designed a pragmatic stepped-wedge cluster randomized controlled trial in order to evaluate provider-initiated evaluation of household contacts (HCs) of smear positive tuberculosis (TB) cases within a routine TB program in Lima, Peru.

Methods/Design

National TB program (NTP) officers of San Juan de Lurigancho District (Lima, Peru) and university-based researchers jointly designed a pragmatic stepped-wedge cluster randomized trial design in order to evaluate a planned active case finding (ACF) program for all HCs of smear-positive TB cases in 34 district healthcare centres. Randomization of time to intervention initiation was stratified by health centre TB case rate. The ACF intervention included provider-initiated home visits of all new sputum smear positive TB patients in order to evaluate household contacts for active TB. Active TB was diagnosed using symptom screening, sputum screening, chest x-ray and clinical evaluation. Once initiated, ACF was provided by NTP staff and integrated into the routine DOTS TB program activities.

Discussion

This study protocol describes the pragmatic stepped-wedge cluster randomized trial of active household contact evaluations within an NTP. The stepped-wedge design met overlapping needs of local TB programmers and researchers to adequately evaluate the large-scale roll out of a new control program in a TB endemic setting. Multiple planning meetings were required to develop the necessary networks and in order to understand the operations, needs and goals of the NTP staff and researchers collaborating on this project. The advantages and challenges of using this study design in practice and within existing routine TB programs in a middle-income country context are discussed.

Trial registration

ClinicalTrials.gov NCT02174380. Registered 24 Jun 2014

[Evidence-based health promotion and prevention in settings: which types of study designs are needed?]

Health promotion and health prevention are cornerstones of public health. In Germany, a draft health prevention law was rejected in 2013, partly because it almost exclusively focused on measures at the individual level. Many health promotion and prevention measures, by contrast, (also) address the societal level and the environment, but there are few robust studies on their effectiveness, not least because of a lack of resources, the complexity of the measures, or randomisation problems. For example, regulations that protect non-smokers from smoke are subject to political processes where the majority of decision-makers would decline consent for randomisation. In a workshop at the 15(th) Annual Meeting of the German Network for Evidence-based Medicine (DNEbM) two case studies on controlled before-and-after studies (CBA) and interrupted time series (ITS) were developed by the audience as possible alternatives to randomised controlled trials for the evaluation of health promotion and health prevention programmes. The suggestions made by the audience were compared to the study designs chosen in published studies, and the strengths and weaknesses of the different study designs were discussed. The wide array of suggestions for effectiveness studies illustrated the potential of evidence-based health promotion and prevention, but also the specific challenges to be faced.