Using qualitative comparative analysis (QCA) in systematic reviews of complex interventions: a worked example

The American Society for Microbiology collaboration with the CDC Laboratory Medicine Best Practices initiative for evidence-based laboratory medicine

SUMMARYClinical medicine has embraced the use of evidence for patient treatment decisions; however, the evaluation strategy for evidence in laboratory medicine practices has lagged. It was not until the end of the 20th century that the Institute of Medicine (IOM), now the National Academy of Medicine, and the Centers for Disease Control and Prevention, Division of Laboratory Systems (CDC DLS), focused on laboratory tests and how testing processes can be designed to benefit patient care. In collaboration with CDC DLS, the American Society for Microbiology (ASM) used an evidence review method developed by the CDC DLS to develop a program for creating laboratory testing guidelines and practices. The CDC DLS method is called the Laboratory Medicine Best Practices (LMBP) initiative and uses the A-6 cycle method. Adaptations made by ASM are called Evidence-based Laboratory Medicine Practice Guidelines (EBLMPG). This review details how the ASM Systematic Review (SR) Processes were developed and executed collaboratively with CDC's DLS. The review also describes the ASM transition from LMBP to the organization's current EBLMPG, maintaining a commitment to working with agencies in the U.S. Department of Health and Human Services and other partners to ensure that EBLMPG evidence is readily understood and consistently used.

Qualitative Comparative Analysis of exercise interventions for fall prevention in residential aged care facilities

BACKGROUND

Exercise interventions are highly effective at preventing falls in older people living in the community. In residential aged care facilities (RACFs), however, the evidence for effectiveness is highly variable, warranting exploration of drivers of successful trials. This study aims to identify the conditions of randomised controlled trials (RCTs) that are associated with reducing falls in RACFs and test whether it can explain the variability.

METHODS

RCTs testing exercise interventions in RACFs compared to usual care, reporting rate or risk of falls from the 2018 Cochrane Collaboration review and a search update to December 2022 were included. Two authors independently extracted and coded trial conditions and outcomes according to a theory developed from prior Intervention Component Analysis. Trial outcomes were coded as successful or unsuccessful based on point estimates for the rate or risk ratio for falls, or p value. Qualitative Comparative Analysis (QCA), utilising Boolean minimisation theory, was conducted to determine the key conditions driving trial success. A subgroup meta-analysis and the GRADE approach was applied to the final theory.

RESULTS

Eighteen trials undertaken in 11 countries with 2,287 residents were included. Participants were predominately ambulant females aged 70 to 80 with cognitive impairment. Most interventions were fully supervised or supervised at the start of the intervention. QCA identified two configurations as drivers of successful exercise falls prevention programs: (i) group exercise that is moderate or low intensity, or (ii) for independent ambulatory residents, exercise for more than 1 h per week. The combination of configuration (i) and (ii) had consistency and total coverage scores of 1, indicating all trials were explained. This combination was associated with a reduction in falls (rate ratio 0.45, 95%CI 0.34 to 0.59; risk ratio 0.66, 95%CI 0.53 to 0.82; low certainty evidence).

CONCLUSION

To successfully reduce falls in RACFs, exercise programs should provide continuous supervised moderate-intensity group exercise. For programs that mostly include independent ambulatory residents, exercise for at least 80 min per week should be provided. As many current residents are frail, tailored exercise is likely necessary and an individualised dose may be required. Future trials should test exercise interventions for less mobile residents.

Whole-school interventions promoting student commitment to school to prevent substance use and violence, and improve educational attainment: a systematic review

Background

Whole-school interventions modify the school environment to promote health. A subset of these interventions promotes student commitment to school to prevent substance (tobacco, alcohol, other drugs) use and/or violence. A previous review identified the theory of human functioning and school organisation as a comprehensive theory of such interventions, and found evidence that these interventions reduce substance use and/or violence.

Objectives

The objectives were to search for, appraise and synthesise evidence to address the following questions: (1) What whole-school interventions promoting student commitment to school to prevent substance use and/or violence have been evaluated, what intervention subtypes are apparent and how closely do these align with the theory of human functioning and school organisation? (2) What factors relating to setting, population and intervention affect implementation? (3) What are the effects on student substance use, violence and educational attainment? (4) What is the cost-effectiveness of such interventions? (5) Are intervention effects mediated by student commitment to school or moderated by setting or population?

Data sources

A total of 56 information sources were searched (in January 2020), then an updated search of 48 of these was carried out (in May 2021). Reference lists were also searched and experts were contacted.

Review methods

Eligible studies were process/outcome evaluations of whole-school interventions to reduce student violence or substance use among students aged 5-18 years attending schools, via actions aligning with the theory of human functioning and school organisation: modifying teaching to increase engagement, enhancing student-staff relationships, revising school policies, encouraging volunteering or increasing parental involvement. Data extraction and quality assessments used existing tools. Theory and process reports were synthesised qualitatively. Outcome and economic data were synthesised narratively; outcome data were meta-analysed.

Results

Searches retrieved 63 eligible reports on 27 studies of 22 interventions. We identified four intervention subtypes focused on student participation in school-wide decisions, improving staff-student relationships, increasing engagement in learning and involving parents. The theories of change of most intervention subtypes aligned closely with the theory of human functioning and school organisation, and informed refinement of an intervention theory of change. Theories of change for interventions increasing learning engagement did not align with this theory, aiming instead to increase school commitment primarily via social skills curricula. Factors influencing the implementation included whether or not interventions were tailorable, workable and well explained. Interventions with action groups comprising staff/students, etc. and providing local data were well implemented. Implementation was also affected by whether or not schools accepted the need for change and staff had the resources for delivery. Meta-analyses suggest small, but significant, intervention effects in preventing violence victimisation and perpetration, and substance use. There was sparse and inconsistent evidence of moderation and some evidence of mediation by student commitment to school. Two economic evaluations suggested that there is the potential for the interventions to be cost-effective.

Limitations

The quality of the studies was variable and the economic synthesis was limited to two studies.

Conclusions

Whole-school interventions aiming to promote student commitment to school share similar theories of change and factors affecting implementation. They have the potential to contribute to preventing violence and substance use among young people. Future trials should aim to optimise intervention effectiveness by better theorisation, and assess implementation and effect moderators and mediators.

Study registration

This study is registered as PROSPERO CRD42019154334.

Funding

This award was funded by the National Institute for Health and Care Research (NIHR) Public Health Research programme (NIHR award ref: 17/151/05) and is published in full in Public Health Research; Vol. 12, No. 2. See the NIHR Funding and Awards website for further award information.

Promoting Medication Adherence in Children with Attention Deficit Hyperactivity Disorder: A Mixed-Methods Systematic Review with Meta-analysis and Qualitative Comparative Analysis

OBJECTIVE

To evaluate the efficacy of ADHD medication adherence interventions and explore the pathways to effectiveness.

METHODS

A systematic review was conducted using multiple databases to identify relevant randomized controlled trials (RCTs). Pooled effect sizes were calculated for medication adherence and ADHD symptom outcomes. Qualitative Comparative Analysis (QCA) was used to identify pathways to effectiveness.

RESULTS

Six RCTs were included. The interventions significantly improved medication adherence (OR = 2.39, 95% CI [1.19, 4.79]) and ADHD symptoms (Hedges' g = -0.96, 95% CI [-1.38, -0.54]). Multi-regression analysis showed a positive relationship between medication adherence and ADHD symptom reduction. QCA revealed two paths for effectiveness: (1) Presence of ADHD drug education and absence of reminder and (2) Presence of tracking and absence of reminder.

CONCLUSION

ADHD medication adherence interventions have a positive impact on both medication adherence and ADHD symptoms. Interventions should consider including ADHD drug education or tracking to maximize effectiveness.

Exercise for falls prevention in aged care: systematic review and trial endpoint meta-analyses

BACKGROUND

There is strong evidence that exercise reduces falls in older people living in the community, but its effectiveness in residential aged care is less clear. This systematic review examines the effectiveness of exercise for falls prevention in residential aged care, meta-analysing outcomes measured immediately after exercise or after post-intervention follow-up.

METHODS

Systematic review and meta-analysis, including randomised controlled trials from a Cochrane review and additional trials, published to December 2022. Trials of exercise as a single intervention compared to usual care, reporting data suitable for meta-analysis of rate or risk of falls, were included. Meta-analyses were conducted according to Cochrane Collaboration methods and quality of evidence rated using the Grading of Recommendations Assessment, Development and Evaluation approach.

RESULTS

12 trials from the Cochrane review plus 7 new trials were included. At the end of the intervention period, exercise probably reduces the number of falls (13 trials, rate ratio [RaR] = 0.68, 95% confidence interval [CI] = 0.49-0.95), but after post-intervention follow-up exercise had little or no effect (8 trials, RaR = 1.01, 95% CI = 0.80-1.28). The effect on the risk of falling was similar (end of intervention risk ratio (RR) = 0.84, 95% CI = 0.72-0.98, 12 trials; post-intervention follow-up RR = 1.05, 95% CI = 0.92-1.20, 8 trials). There were no significant subgroup differences according to cognitive impairment.

CONCLUSIONS

Exercise is recommended as a fall prevention strategy for older people living in aged care who are willing and able to participate (moderate certainty evidence), but exercise has little or no lasting effect on falls after the end of a programme (high certainty evidence).

Critical features of multifactorial interventions for effective falls reduction in residential aged care: a systematic review, intervention component analysis and qualitative comparative analysis

BACKGROUND

Multifactorial fall prevention trials providing interventions based on individual risk factors have variable success in aged care facilities. To determine configurations of trial features that reduce falls, intervention component analysis (ICA) and qualitative comparative analysis (QCA) were undertaken.

METHODS

Randomised controlled trials (RCTs) from a Cochrane Collaboration review (Cameron, 2018) with meta-analysis data, plus trials identified in a systematic search update to December 2021 were included. Meta-analyses were updated. A theory developed through ICA of English publications of trialist's perspectives was assessed through QCA and a subgroup meta-analysis.

RESULTS

Pooled effectiveness of multifactorial interventions indicated a falls rate ratio of 0.85 (95% confidence interval, CI, 0.65-1.10; I2 = 85%; 11 trials). All tested interventions targeted both environmental and personal risk factors by including assessment of environmental hazards, a medical or medication review and exercise intervention. ICA emphasised the importance of co-design involving facility staff and managers and tailored intervention delivery to resident's intrinsic factors for successful outcomes. QCA of facility engagement plus tailored delivery was consistent with greater reduction in falls, supported by high consistency (0.91) and coverage (0.85). An associated subgroup meta-analysis demonstrated strong falls reduction without heterogeneity (rate ratio 0.61, 95%CI 0.54-0.69, I2 = 0%; 7 trials).

CONCLUSION

Multifactorial falls prevention interventions should engage aged care staff and managers to implement strategies which include tailored intervention delivery according to each resident's intrinsic factors. Such approaches are consistently associated with a successful reduction in falls, as demonstrated by QCA and subgroup meta-analyses. Co-design approaches may also enhance intervention success.

Educational interventions targeting pregnant women to optimise the use of caesarean section: What are the essential elements? A qualitative comparative analysis

BACKGROUND

Caesarean section (CS) rates are increasing globally, posing risks to women and babies. To reduce CS, educational interventions targeting pregnant women have been implemented globally, however, their effectiveness is varied. To optimise benefits of these interventions, it is important to understand which intervention components influence success. In this study, we aimed to identify essential intervention components that lead to successful implementation of interventions focusing on pregnant women to optimise CS use.

METHODS

We re-analysed existing systematic reviews that were used to develop and update WHO guidelines on non-clinical interventions to optimise CS. To identify if certain combinations of intervention components (e.g., how the intervention was delivered, and contextual characteristics) are associated with successful implementation, we conducted a Qualitative Comparative Analysis (QCA). We defined successful interventions as interventions that were able to reduce CS rates. We included 36 papers, comprising 17 CS intervention studies and an additional 19 sibling studies (e.g., secondary analyses, process evaluations) reporting on these interventions to identify intervention components. We conducted QCA in six stages: 1) Identifying conditions and calibrating the data; 2) Constructing truth tables, 3) Checking quality of truth tables; 4) Identifying parsimonious configurations through Boolean minimization; 5) Checking quality of the solution; 6) Interpretation of solutions. We used existing published qualitative evidence synthesis to develop potential theories driving intervention success.

RESULTS

We found successful interventions were those that leveraged social or peer support through group-based intervention delivery, provided communication materials to women, encouraged emotional support by partner or family participation, and gave women opportunities to interact with health providers. Unsuccessful interventions were characterised by the absence of at least two of these components.

CONCLUSION

We identified four key essential intervention components which can lead to successful interventions targeting women to reduce CS. These four components are 1) group-based delivery, 2) provision of IEC materials, 3) partner or family member involvement, and 4) opportunity for women to interact with health providers. Maternal health services and hospitals aiming to better prepare women for vaginal birth and reduce CS can consider including the identified components to optimise health and well-being benefits for the woman and baby.

Methodical advances in reproducibility research: A proof of concept qualitative comparative analysis of reproducing animal data in humans

BACKGROUND

While the term reproducibility crisis mainly reflects reproducibility of experiments between laboratories, reproducibility between species also remains problematic. We previously summarised the published reproducibility between animal and human studies; i.e. the translational success rates, which varied from 0% to 100%. Based on analyses of individual factors, we could not predict reproducibility. Several potential analyses can assess effect of combinations of predictors on an outcome. Regression analysis (RGA) is common, but not ideal to analyse multiple interactions and specific configurations (≈ combinations) of variables, which could be highly relevant to reproducibility. Qualitative comparative analysis (QCA) is based on set theory and Boolean algebra, and was successfully used in other fields. We reanalysed the data from our preceding review with QCA.

RESULTS

This QCA resulted in the following preliminary formula for successful translation: ∼Old*∼Intervention*∼Large*MultSpec*Quantitative Which means that within the analysed dataset, the combination of relative recency (∼ means not; >1999), analyses at event or study level (not at intervention level), n < 75, inclusion of more than one species and quantitative (instead of binary) analyses always resulted in successful translation (>85%). Other combinations of factors showed less consistent or negative results. An RGA on the same data did not identify any of the included variables as significant contributors.

CONCLUSIONS

While these data were not collected with the QCA in mind, they illustrate that the approach is viable and relevant for this research field. The QCA seems a highly promising approach to furthering our knowledge on between-species reproducibility.

PROTOCOL: The effects of agricultural output market access interventions on agricultural, socio-economic and food and nutrition security outcomes in low- and middle-income countries: A systematic review

Development agencies and international donors' efforts are increasingly focusing on better integrating poor and remote farmers into agricultural markets to address the chronic issues of rural poverty and hunger in low- and middle-income countries. Using systematic methods for information retrieval, critical appraisal and evidence synthesis, this research aims to examine evidence on the effects of five focal types of agricultural market access interventions: (i) farm-to-market transport infrastructure interventions; (ii) output market information interventions; (iii) initiatives creating new marketplaces and alternative marketing opportunities; (iv) contract farming initiatives; (v) interventions improving storage infrastructure. In this review, we will study evidence of the magnitude and direction of intervention effects on agricultural, socio-economic, and food and nutrition security outcomes. We will examine evidence of the distribution of reported effects across different contexts, interventions and sub-groups of the population (e.g., according to sex, socio-economic status, farm size, etc.). We will also report on included studies' risk of bias and on what evidence is available on intervention costs, or their cost-effectiveness. This protocol outlines this review's planned methods and the criteria for selecting and including studies in its analysis.

Development and initial evaluation of a behavioural intervention to support weight management for people with serious mental illness: an uncontrolled feasibility and acceptability study

BACKGROUND

The rates of obesity and associated health problems are higher in people with serious mental illness (SMI), such as schizophrenia and bipolar disorder, than the general population. A primary care referral to a behavioural weight management programme can be an effective intervention, but people with SMI have reported barriers to engaging with them and bespoke options are rarely provided in routine practice. It is possible that adjunct support addressing these specific barriers could help. Here we report the development, feasibility and acceptability of an intervention to improve uptake and engagement with a mainstream weight management programme for people with SMI.

METHODS

We worked with people with a lived-experience of SMI and used the person-based approach to develop the 'Weight cHange for people with sErious mEntal iLlness' (WHEEL) intervention. It comprised a referral to a mainstream weight management programme (WW®) to be attended once a week, in-person or online, for 12-weeks. The adjunct support comprised a one-off, online consultation called Meet Your Mentor and weekly, telephone or email Mentor Check Ins for 12-weeks. We assessed the feasibility of WHEEL through the number of programme and adjunct support sessions that the participants attended. We analysed the acceptability of WHEEL using a thematic analysis of qualitative interviews conducted at baseline and at 12-week follow-up. Our exploratory outcome of clinical effectiveness was self-reported weight at baseline and at end-of-programme.

RESULTS

Twenty participants were assessed for eligibility and 17 enrolled. All 17 participants attended Meet Your Mentor and one was lost to follow-up (94% retention). Nine out of 16 attended ≥50% of the weekly programme sessions and 12/16 attended ≥50% of the weekly check-ins. Participants reported in the interviews that the adjunct support helped to establish and maintain a therapeutic alliance. While some participants valued the in-person sessions, others reported that they preferred the online sessions because it removed a fear of social situations, which was a barrier for some participants. The mean change in self-reported weight was - 4·1 kg (SD: 3·2) at 12-weeks.

CONCLUSIONS

A mainstream weight management programme augmented with brief and targeted education and low-intensity check-ins generated sufficient engagement and acceptability to warrant a future trial.

Interventions to reintroduce or increase assisted vaginal births: a systematic review of the literature

OBJECTIVE

To synthesise the evidence from studies that implemented interventions to increase/reintroduce the use of assisted vaginal births (AVB).

DESIGN

Systematic review.

ELIGIBILITY CRITERIA

We included experimental, semi-experimental and observational studies that reported any intervention to reintroduce/increase AVB use.

DATA SOURCES

We searched PubMed, EMBASE, CINAHL, LILACS, Scopus, Cochrane, WHO Library, Web of Science, ClinicalTrials.gov and WHO.int/ictrp through September 2021.

RISK OF BIAS

For trials, we used the Cochrane Effective Practice and Organisation of Care tool; for other designs we used Risk of Bias for Non-Randomised Studies of Interventions.

DATA EXTRACTION AND SYNTHESIS

Due to heterogeneity in interventions, we did not conduct meta-analyses. We present data descriptively, grouping studies according to settings: high-income countries (HICs) or low/middle-income countries (LMICs). We classified direction of intervention effects as (a) statistically significant increase or decrease, (b) no statistically significant change or (c) statistical significance not reported in primary study. We provide qualitative syntheses of the main barriers and enablers for success of the intervention.

RESULTS

We included 16 studies (10 from LMICs), mostly of low or moderate methodological quality, which described interventions with various components (eg, didactic sessions, simulation, hands-on training, guidelines, audit/feedback). All HICs studies described isolated initiatives to increase AVB use; 9/10 LMIC studies tested initiatives to increase AVB use as part of larger multicomponent interventions to improve maternal/perinatal healthcare. No study assessed women's views or designed interventions using behavioural theories. Overall, interventions were less successful in LMICs than in HICs. Increase in AVB use was not associated with significant increase in adverse maternal or perinatal outcomes. The main barriers to the successful implementation of the initiatives were related to staff and hospital environment.

CONCLUSIONS

There is insufficient evidence to indicate which intervention, or combination of interventions, is more effective to safely increase AVB use. More research is needed, especially in LMICs, including studies that design interventions taking into account theories of behaviour change.

PROSPERO REGISTRATION NUMBER

CRD42020215224.

Interventions targeting healthcare providers to optimise use of caesarean section: a qualitative comparative analysis to identify important intervention features

BACKGROUND

Rapid increases in caesarean section (CS) rates have been observed globally; however, CS rates exceeding 15% at a population-level have limited benefits for women and babies. Many interventions targeting healthcare providers have been developed to optimise use of CS, typically aiming to improve and monitor clinical decision-making. However, interventions are often complex, and effectiveness is varied. Understanding intervention and implementation features that likely lead to optimised CS use is important to optimise benefits. The aim of this study was to identify important components that lead to successful interventions to optimise CS, focusing on interventions targeting healthcare providers.  METHODS: We used Qualitative Comparative Analysis (QCA) to identify if certain combination of important intervention features (e.g. type of intervention, contextual characteristics, and how the intervention was delivered) are associated with a successful intervention as reflected in a reduction of CS. We included 21 intervention studies targeting healthcare providers to reduce CS, comprising of 34 papers reporting on these interventions. To develop potential theories driving intervention success, we used existing published qualitative evidence syntheses on healthcare providers' perspectives and experiences of interventions targeted at them to reduce CS.

RESULTS

We identified five important components that trigger successful interventions targeting healthcare providers: 1) training to improve providers' knowledge and skills, 2) active dissemination of CS indications, 3) actionable recommendations, 4) multidisciplinary collaboration, and 5) providers' willingness to change. Importantly, when one or more of these components are absent, dictated nature of intervention, where providers are enforced to adhere to the intervention, is needed to prompt successful interventions. Unsuccessful interventions were characterised by the absence of these components.

CONCLUSION

We identified five important intervention components and combinations of intervention components which can lead to successful interventions targeting healthcare providers to optimise CS use. Health facility managers, researchers, and policy-makers aiming to improve providers' clinical decision making and reduce CS may consider including the identified components to optimise benefits.

Implementation of e-Mental Health Interventions for Informal Caregivers of Adults With Chronic Diseases: Mixed Methods Systematic Review With a Qualitative Comparative Analysis and Thematic Synthesis

BACKGROUND

Informal caregivers commonly experience mental health difficulties related to their caregiving role. e-Mental health interventions provide mental health support in a format that may be more accessible to informal caregivers. However, e-mental health interventions are seldom implemented in real-world practice.

OBJECTIVE

This mixed methods systematic review aimed to examine factors associated with the effectiveness and implementation of e-mental health interventions for informal caregivers of adults with chronic diseases. To achieve this aim, two approaches were adopted: combinations of implementation and intervention characteristics sufficient for intervention effectiveness were explored using qualitative comparative analysis, and barriers to and facilitators of implementation of e-mental health interventions for informal caregivers were explored using thematic synthesis.

METHODS

We identified relevant studies published from January 1, 2007, to July 6, 2022, by systematically searching 6 electronic databases and various secondary search strategies. Included studies reported on the effectiveness or implementation of e-mental health interventions for informal caregivers of adults with cancer, chronic obstructive pulmonary disease, dementia, diabetes, heart disease, or stroke. Randomized controlled trials reporting on caregivers' mental health outcomes were included in a crisp-set qualitative comparative analysis. We assessed randomized controlled trials for bias using the Risk of Bias 2.0 tool, and we assessed how pragmatic or explanatory their trial design was using the Pragmatic Explanatory Continuum Indicator Summary 2 tool. Studies of any design reporting on implementation were included in a thematic synthesis using the Consolidated Framework for Implementation Research to identify barriers to and facilitators of implementation.

RESULTS

Overall, 53 reports, representing 29 interventions, were included in the review. Most interventions (27/29, 93%) focused on informal cancer or dementia caregivers. In total, 14 reports were included in the qualitative comparative analysis, exploring conditions including the presence of peer or professional support and key persuasive design features. Low consistency and coverage prevented the determination of condition sets sufficient for intervention effectiveness. Overall, 44 reports were included in the thematic synthesis, and 152 barriers and facilitators were identified, with the majority related to the intervention and individual characteristic domains of the Consolidated Framework for Implementation Research. Implementation barriers and facilitators in the inner setting (eg, organizational culture) and outer setting (eg, external policies and resources) domains were largely unexplored.

CONCLUSIONS

e-Mental health interventions for informal caregivers tend to be well-designed, with several barriers to and facilitators of implementation identified related to the intervention and individual user characteristics. Future work should focus on exploring the views of stakeholders involved in implementation to determine barriers to and facilitators of implementing e-mental health interventions for informal caregivers, focusing on inner and outer setting barriers and facilitators.

TRIAL REGISTRATION

PROSPERO (International Prospective Register of Systematic Reviews) CRD42020155727; https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42020155727.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1136/bmjopen-2019-035406.

Structural Interventions to Enable Adolescent Contraceptive Use in LMICs: A Mid-Range Theory to Support Intervention Development and Evaluation

Enabling contraceptive use is critical for addressing high adolescent pregnancy rates in low- and middle-income countries (LMICs). Broader or 'upstream' determinants, such as poverty, education, and social norms, can affect the knowledge, attitudes, motivation, and ability to access and use contraception. Structural interventions aim to address these broader determinants, e.g., through poverty alleviation from livelihood training or cash transfers, increasing school participation, or changing social norms. We conducted an evidence synthesis using intervention component analysis, a case-based approach, following a systematic mapping of the evidence base. We identified 17 studies with 29 structural intervention arms, which reported adolescent contraceptive use outcomes compared to a control group or baseline. It was not possible to identify with certainty which interventions were 'likely effective' or 'likely ineffective' due to the high heterogeneity of the methods. We built on an existing framework of family planning use to propose three steps to designing interventions: (1) tailor interventions to adolescents' life stages; (2) assess the baseline situation; and (3) select appropriate activities to match the gaps. These steps will aid developers and evaluators of structural adolescent contraceptive interventions to develop an evidence base that is of use across a wide range of settings and use scenarios.

Occupational mental health of non-family members in family firms: Evidence from Pakistan

Family-owned firms display distinct dynamics as compared to other firms. Consequently, the outcomes and consequences of these dynamics are also expected to be different. The aim of this study was to explore the impact of perceived employee-organization psychological distance (EOPD) on the occupational mental health (OMH) of the employees. Considering the complexities associated with employee-employer relationship, the study also investigated how this relationship between PD and OMH might be mediated by psychological safety (PS) perceived by the employees. Furthermore, the study also included proactive personality (PP) as a potential moderator of the relationship between PD and OMH. Results using SEM and fsQCA show a partial mediation effect on psychological safety. The study contributes by examining the distinct nature of family firms and their impact on the mental health of non-family member employees. This study contributes to the family firm literature by adopting a novel methodological approach to unveil the complexity behind the relationship between employees and owner-employers of family firms.

Influence of Workload, Personality, and Psychological Flexibility on Occupational Stress Among Medical Staff: A Fuzzy-Set Qualitative Comparative Analysis

Objective

During the COVID-19 pandemic, the occupational stress of medical staff has been a major issue. This study aimed to suggest a new strategy to identify high-risk factor sets of occupational stress in medical staff using fuzzy-set qualitative comparative analysis (fs-QCA) and provide ideas for the prevention and intervention of occupational stress.

Methods

A total of 1,928 medical staff members were surveyed and tested using the Acceptance and Action Questionnaire-II (AAQ-II), Occupational Stress Inventory-Revised edition (OSI-R), and Eysenck Personality Questionnaire-Revised Short Scale (EPQ-RSC). The fs-QCA was used to explore the high-risk factors for occupational stress among medical staff.

Results

The psychological strain (PSY) score of the medical staff was 26.8 ± 7.13, and the physical strain (PHS) score was 24.3 ± 6.50. Low psychological flexibility score-introversion-high role overload, introversion-neuroticism-high role overload, and low psychological flexibility score-neuroticism were high-risk factor sets for PSY. Low psychological flexibility score-introversion-high role overload, low psychological flexibility score-introversion-neuroticism, low psychological flexibility score-neuroticism-high role overload, low psychological flexibility score-psychoticism-neuroticism, and psychoticism-neuroticism-high role overload were high-risk factor sets for PHS.

Conclusion

There are different combinations of high-risk factors for occupational stress among the medical staff. For occupational stress intervention and psychological counseling, targeted and individualized health intervention measures should be implemented according to specific characteristic combinations of different individuals.

'Leading from the front' implementation increases the success of influenza vaccination drives among healthcare workers: a reanalysis of systematic review evidence using Intervention Component Analysis (ICA) and Qualitative Comparative Analysis (QCA)

BACKGROUND

Seasonal influenza vaccination of healthcare workers (HCW) is widely recommended to protect staff and patients. A previous systematic review examined interventions to encourage uptake finding that hard mandates, such as loss of employment for non-vaccination, were more effective than soft mandates, such as signing a declination form, or other interventions such as incentives. Despite these overarching patterns the authors of the review concluded that 'substantial heterogeneity' remained requiring further analysis. This paper reanalyses the evidence using Intervention Component Analysis (ICA) and Qualitative Comparative Analysis (QCA) to examine whether the strategies used to implement interventions explain the residual heterogeneity.

METHODS

We used ICA to extract implementation features and trialists' reflections on what underpinned the success of the intervention they evaluated. The ICA findings then informed and structured two QCA analyses to systematically examine associations between implementation features and intervention outcomes. Analysis 1 examined hard mandate studies. Analysis 2 examined soft mandates and other interventions.

RESULTS

In Analysis 1 ICA revealed the significance of 'leading from the front' rather than 'top-down' implementation of hard mandates. Four key features underpinned this: providing education prior to implementation; two-way engagement so HCW can voice concerns prior to implementation; previous use of other strategies so that institutions 'don't-go-in-cold' with hard-mandates; and support from institutional leadership. QCA revealed that either of two configurations were associated with greater success of hard mandates. The first involves two-way engagement, leadership support and a 'don't-go-in-cold' approach. The second involves leadership support, education and a 'don't-go-in-cold' approach. Reapplying the 'leading from the front' theory in Analysis 2 revealed similar patterns.

CONCLUSIONS

Regardless of intervention type a 'leading from the front' approach to implementation will likely enhance intervention success. While the results pertain to flu vaccination among HCWs, the components identified here may be relevant to public health campaigns regarding COVID-19 vaccination.

Strengthening women's empowerment and gender equality in fragile contexts towards peaceful and inclusive societies: A systematic review and meta-analysis

Background

Across the globe, gender disparities still exist with regard to equitable access to resources, participation in decision-making processes, and gender and sexual-based violence. This is particularly true in fragile and conflict-affected settings, where women and girls are affected by both fragility and conflict in unique ways. While women have been acknowledged as key actors in peace processes and post-conflict reconstruction (e.g., through the United Nations Security Council Resolution 1325 and the Women, Peace and Security Agenda) evidence on the effectiveness of gender-specific and gender-transformative interventions to improve women's empowerment in fragile and conflict-affected states and situations (FCAS) remains understudied.

Objectives

The purpose of this review was to synthesize the body of evidence around gender-specific and gender-transformative interventions aimed at improving women's empowerment in fragile and conflict-affected settings with high levels of gender inequality. We also aimed to identify barriers and facilitators that could affect the effectiveness of these interventions and to provide implications for policy, practice and research designs within the field of transitional aid.

Methods

We searched for and screened over 100,000 experimental and quasi-experimental studies focused on FCAS at the individual and community levels. We used standard methodological procedures outlined by the Campbell Collaboration for the data collection and analysis, including quantitative and qualitative analyses, and completed the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) methodology to assess the certainty around each body of evidence.

Results

We identified 104 impact evaluations (75% randomised controlled trials) assessing the effects of 14 different types of interventions in FCAS. About 28% of included studies were assessed as having a high risk of bias (45% among quasi-experimental designs). Interventions supporting women's empowerment and gender equality in FCAS produced positive effects on the outcomes related to the primary focus of the intervention. There are no significant negative effects of any included interventions. However, we observe smaller effects on behavioural outcomes further along the causal chain of empowerment. Qualitative syntheses indicated that gender norms and practices are potential barriers to intervention effectiveness, while working with local powers and institutions can facilitate the uptake and legitimacy of interventions.

Conclusions

We observe gaps of rigorous evidence in certain regions (notably MENA and Latin America) and in interventions specifically targeting women as actors of peacebuilding. Gender norms and practices are important elements to consider in programme design and implementation to maximise potential benefits: focusing on empowerment only might not be enough in the absence of targeting the restrictive gender norms and practices that may undermine intervention effectiveness. Lastly, programme designers and implementation should consider explicitly targeting specific empowerment outcomes, promoting social capital and exchange, and tailoring the intervention components to the desired empowerment-related outcomes.

Identifying effective characteristics of behavioral weight management interventions for people with serious mental illness: A systematic review with a qualitative comparative analysis

People with serious mental illness (SMI) have identified barriers to engaging in behavioral weight management interventions (BWMIs). We assessed whether BWMIs that addressed these barriers were more effective. First, we systematically reviewed qualitative literature and used a thematic analysis to identify the characteristics of BWMIs that promote engagement for adults with SMI. Second, we systematically reviewed randomized controlled trials (RCTs) of BWMIs in adults with SMI. Data on the characteristics that promoted engagement and weight outcomes were extracted. We then used a crisp-set qualitative comparative analysis (CsQCA) to identify which characteristics were associated with weight loss. For the qualitative review, 20 studies in 515 people with SMI were analyzed and nine characteristics were reported to promote engagement in BWMIs. For the systematic review, 34 RCTs testing 36 interventions in 4305 participants were included. The active interventions resulted in more weight loss (mean = -4.37 to +1 kg at 6 weeks to 18 months follow-up) compared with controls (-1.64 to +3.08 kg). The CsQCA showed BWMIs that offered regular contact, tools to support enactment, and tailored materials were associated with effectiveness. As these are all supplementary strategies, it may be possible to augment BWMIs available for the general population to engage people with SMI.

Frameworks, measures, and interventions for HIV-related internalised stigma and stigma in healthcare and laws and policies: systematic review protocol

INTRODUCTION

There is strong global commitment to eliminate HIV-related stigma. Wide variation exists in frameworks and measures, and many strategies to prevent, reduce or mitigate stigma have been proposed but critical factors determining success or failure remain elusive.

METHODS AND ANALYSIS

Building on existing knowledge syntheses, we designed a systematic review to identify frameworks, measures and intervention evaluations aiming to address internalised stigma, stigma and discrimination in healthcare, and stigma and discrimination at the legal or policy level. The review addresses four key questions (KQ): KQ1: Which conceptual frameworks have been proposed to assess internal stigma, stigma and discrimination experienced in healthcare settings, and stigma and discrimination entrenched in national laws and policies? KQ2: Which measures of stigma have been proposed and what are their descriptive properties? KQ3: Which interventions have been evaluated that aimed to reduce these types of stigma and discrimination or mitigate their adverse effects and what are the effectiveness and unintended consequences? KQ4: What common 'critical factors for success or failure' can be identified across interventions that have been evaluated? We will search PubMed, PsycINFO, Web of Science, Universal Human Rights Index, HeinOnline, PAIS, HIV Legal Network, CDSR, Campbell Collaboration, PROSPERO and Open Science Framework. Critical appraisal will assess the source, processes and consensus finding for frameworks; COnsensus-based Standards for the selection of health Measurement Instruments criteria for measures; and risk of bias for interventions. Quality of evidence grading will apply . A gap analysis will provide targeted recommendations for future research. We will establish a compendium of frameworks, a comprehensive catalogue of available measures, and a synthesis of intervention characteristics to advance the science of HIV-related stigma.

PROSPERO REGISTRATION NUMBER

CRD42021249348.

A Library of Logic Models to Explain How Interventions to Reduce Diagnostic Errors Work

OBJECTIVES

We aimed to create a library of logic models for interventions to reduce diagnostic error. This library can be used by those developing, implementing, or evaluating an intervention to improve patient care, to understand what needs to happen, and in what order, if the intervention is to be effective.

METHODS

To create the library, we modified an existing method for generating logic models. The following five ordered activities to include in each model were defined: preintervention; implementation of the intervention; postimplementation, but before the immediate outcome can occur; the immediate outcome (usually behavior change); and postimmediate outcome, but before a reduction in diagnostic errors can occur. We also included reasons for lack of progress through the model. Relevant information was extracted about existing evaluations of interventions to reduce diagnostic error, identified by updating a previous systematic review.

RESULTS

Data were synthesized to create logic models for four types of intervention, addressing five causes of diagnostic error in seven stages in the diagnostic pathway. In total, 46 interventions from 43 studies were included and 24 different logic models were generated.

CONCLUSIONS

We used a novel approach to create a freely available library of logic models. The models highlight the importance of attending to what needs to occur before and after intervention delivery if the intervention is to be effective. Our work provides a useful starting point for intervention developers, helps evaluators identify intermediate outcomes, and provides a method to enable others to generate libraries for interventions targeting other errors.

PROTOCOL: Strengthening women's empowerment and gender equality in fragile contexts towards peaceful and inclusive societies: A systematic review and meta-analysis

This review builds on 3ie's (international initiative for impact evaluation) evidence gap map (EGM) of the impact evaluation and systematic review (SR) evidence base of interventions aiming to promote peaceful and inclusive societies in fragile contexts. The EGM identified a cluster of studies evaluating gender equality-focused behaviour change communication programmes and raised interest in investigating the evidence base for understanding the role of women more broadly as agents of change in developing peaceful and inclusive societies. Building on the cluster of evidence identified in the EGM, our review will increase generalisability of findings from single studies and focus on interventions across a broad range of geographical locations, settings and populations, types of implementations and outcomes. We will also address (when possible) the identified gaps in literature regarding metaanalysis in conflict-affected contexts. As such, we propose the following objectives: (1) The primary objective of this review is to identify, assess and synthesise evidence on the effect of gender specific and gender transformative interventions within the context of the four pillars of United Nations Security Council Resolution (UNSCR) 1325 on women's empowerment and gender equality in Fragile and Conflict Affected States/Situations (FCAS). The SR will facilitate the use of evidence in informing policy and practice decisions within the field of transition aid, particularly as it relates to gender focused programming. (2) Our second objective is to assess how these interventions contribute to inclusive and sustainable peace in conflict affected situations. We will compare the effectiveness of these different types of interventions through the lenses of their ecological level, types of impact on women's empowerment, local context of gender inequality and conflict. To achieve these objectives we aim to answer the following questions: (1) What are the impacts of gender transformative and specific interventions on women's empowerment and gender equality in FCAS? (2) What are the effects of these interventions on sustainable peace? (3) To what extent do effects vary by population group, ecological level and types of interventions? (4) What are contextual barriers to and facilitators of intervention effectiveness?

The use of Qualitative Comparative Analysis (QCA) to address causality in complex systems: a systematic review of research on public health interventions

Background

Qualitative Comparative Analysis (QCA) is a method for identifying the configurations of conditions that lead to specific outcomes. Given its potential for providing evidence of causality in complex systems, QCA is increasingly used in evaluative research to examine the uptake or impacts of public health interventions. We map this emerging field, assessing the strengths and weaknesses of QCA approaches identified in published studies, and identify implications for future research and reporting.

Methods

PubMed, Scopus and Web of Science were systematically searched for peer-reviewed studies published in English up to December 2019 that had used QCA methods to identify the conditions associated with the uptake and/or effectiveness of interventions for public health. Data relating to the interventions studied (settings/level of intervention/populations), methods (type of QCA, case level, source of data, other methods used) and reported strengths and weaknesses of QCA were extracted and synthesised narratively.

Results

The search identified 1384 papers, of which 27 (describing 26 studies) met the inclusion criteria. Interventions evaluated ranged across: nutrition/obesity (n = 8); physical activity (n = 4); health inequalities (n = 3); mental health (n = 2); community engagement (n = 3); chronic condition management (n = 3); vaccine adoption or implementation (n = 2); programme implementation (n = 3); breastfeeding (n = 2), and general population health (n = 1). The majority of studies (n = 24) were of interventions solely or predominantly in high income countries. Key strengths reported were that QCA provides a method for addressing causal complexity; and that it provides a systematic approach for understanding the mechanisms at work in implementation across contexts. Weaknesses reported related to data availability limitations, especially on ineffective interventions. The majority of papers demonstrated good knowledge of cases, and justification of case selection, but other criteria of methodological quality were less comprehensively met.

Conclusion

QCA is a promising approach for addressing the role of context in complex interventions, and for identifying causal configurations of conditions that predict implementation and/or outcomes when there is sufficiently detailed understanding of a series of comparable cases. As the use of QCA in evaluative health research increases, there may be a need to develop advice for public health researchers and journals on minimum criteria for quality and reporting.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-021-10926-2.

Rapid systematic review of systematic reviews: what befriending, social support and low intensity psychosocial interventions, delivered remotely, may reduce social isolation and loneliness among older adults and how?

Background: During the COVID-19 pandemic ‘social distancing’ has highlighted the need to minimise loneliness and isolation among older adults (aged 50+). We wanted to know what remotely delivered befriending, social support and low intensity psychosocial interventions may help to alleviate social isolation and loneliness and how they work. Methods: We followed a systematic ‘review of reviews’ approach. Searches of 11 databases from the fields of health, social care, psychology and social science were undertaken during April 2020. Reviews meeting our PICOS criteria were included if they focussed on the evaluation of remote interventions to reduce levels of social isolation or loneliness in adults aged 50+ and were critically appraised using AMSTAR2. Narrative synthesis was used at a review and study level to develop a typology of intervention types and their effectiveness. Intervention Component Analysis (ICA) and Qualitative Comparative Analysis (QCA) were used at a study level to explore the characteristics of successful interventions. Results: We synthesised evidence from five systematic reviews and 18 primary studies. Remote befriending, social support and low intensity psychosocial interventions took the form of: (i) supported video-communication; (ii) online discussion groups and forums; (iii) telephone befriending; (iv) social networking sites; and (v) multi-tool interventions. The majority of studies utilised the first two approaches, and were generally regarded positively by older adults, although with mixed evidence around effectiveness. Focussing on processes and mechanisms, using ICA and QCA, we found that the interventions that were most successful in improving social support: (i) enabled participants to speak freely and to form close relationships; (ii) ensured participants have shared experiences/characteristics; (iii) included some form of pastoral guidance. Conclusions: The findings highlight a set of intervention processes that should be incorporated into interventions, although they do not lead us to recommend specific modes of support, due to the heterogeneity of interventions.

The effect of behavioural interventions targeting hand hygiene practices among nurses in high-income hospital settings: a systematic review

BACKGROUND

Hand hygiene is a critical behaviour for infection control but efforts to raise compliance among clinical professionals have been met with mixed success. The aim of this systematic review was to identify the effectiveness of the behaviour change techniques utilised in recent hand hygiene interventions that seek to improve hand hygiene compliance among nurses in hospitals in high-income countries. Nurses are at the frontline of healthcare delivery, and so improving their HH behaviour and thus increasing HHC rates will have a relatively large impact on reducing transmission and preventing healthcare acquired infections.

METHODS

High-quality studies among nurses in high-income countries were surveyed from the scientific literature, following PRISMA guidelines, to identify which kinds of behaviour change mechanisms have been used to effectively increase hand hygiene compliance. Only seven studies met all inclusion criteria. A formal meta-analysis was not conducted due to the heterogeneity of the included studies. Instead, the review analysed studies in line with the Intervention Component Analysis approach to identify which differences in intervention characteristics appear to be important. Analysis proceeded in two steps: first, the Effective Practice and Organization of Care Data Extraction Checklist was used to identify the study design and to describe the intervention, target population, setting, results, outcome measures, and analytic approach. The second step involved inferring the behavioural change techniques used in the complex study interventions. Following coding, logic models were then inferred for each study to identify the Theory of Change behind each intervention. These Theories of Change were then examined for suggestions as to which BCTs were likely to have been responsible for any effectiveness observed.

RESULTS

Goals and planning (to achieve specific ends), comparison of behaviour (to peers or some ideal) and feedback and monitoring (observing and providing feedback about behaviour or outcomes) were the most frequently used behaviour change technique groupings used across studies and within interventions.

CONCLUSION

The complexity of the interventions used and lack of sufficient studies makes assignment of responsibility for behaviour change to specific behaviour change techniques difficult. Delivery channels and activities identified in the study Theories of Change were also highly individualized and so difficult to compare. However, we identified a temporal shift in types of techniques used in these recent studies on HH interventions, as compared with studies from prior to the review period. These newer interventions did not focus on providing access to alcohol-based hand rub or trying to solely encourage administrative support. Instead, they had nurses create goals and plan how to best facilitate HH, compared both individuals' and the group's behaviour to others, and focused on providing feedback.

Online parent programs for children's behavioral problems: a meta-analytic review

A recent increase in the development of online parent programs calls for the need to understand how effective these strategies are for improving children's mental health. We meta-analyzed the effects of online parent programs on children's behavioral problems. Moreover, we explored the combinations of program components to yield stronger program effects. Medline, PsycINFO, Web of Science, and the Cochrane Library were searched. We included peer-reviewed randomized studies evaluating the effect of an online parent program. Effect sizes (Hedges' g) were calculated from post intervention means and standard deviations. We used Qualitative Comparative Analysis (QCA) to identify pathways to effectiveness and individual content and delivery components that seem sufficient or necessary for yielding high effectiveness. Of 2941 articles, 12 articles with a total of 2025 participants met the inclusion criteria. Online parent programs have significant effects on children's behavioral problems (g =   - 0.32; 95% CI, - 0.47 to - 0.17), emotional problems (g =  - 0.22; 95% CI, - 0.31 to - 0.13), and parental mental health problems (g =  - 0.30; 95% CI, - 0.42 to - 0.17). In the QCA, sending parents reminders to work on the program was the only one sufficient component. In conclusion, online support programs reduce children's behavioral and emotional problems and improve parental mental health. Sending parents reminders to work on the program seems to contribute to high effectiveness. Review Registration This study was registered with PROSPERO, number CRD42017080051.

Fitbit-Based Interventions for Healthy Lifestyle Outcomes: Systematic Review and Meta-Analysis

BACKGROUND

Unhealthy behaviors, such as physical inactivity, sedentary lifestyle, and unhealthful eating, remain highly prevalent, posing formidable challenges in efforts to improve cardiovascular health. While traditional interventions to promote healthy lifestyles are both costly and effective, wearable trackers, especially Fitbit devices, can provide a low-cost alternative that may effectively help large numbers of individuals become more physically fit and thereby maintain a good health status.

OBJECTIVE

The objectives of this meta-analysis are (1) to assess the effectiveness of interventions that incorporate a Fitbit device for healthy lifestyle outcomes (eg, steps, moderate-to-vigorous physical activity, and weight) and (2) to identify which additional intervention components or study characteristics are the most effective at improving healthy lifestyle outcomes.

METHODS

A systematic review was conducted, searching the following databases from 2007 to 2019: MEDLINE, EMBASE, CINAHL, and CENTRAL (Cochrane). Studies were included if (1) they were randomized controlled trials, (2) the intervention involved the use of a Fitbit device, and (3) the reported outcomes were related to healthy lifestyles. The main outcome measures were related to physical activity, sedentary behavior, and weight. All the studies were assessed for risk of bias using Cochrane criteria. A random-effects meta-analysis was conducted to estimate the treatment effect of interventions that included a Fitbit device compared with a control group. We also conducted subgroup analysis and fuzzy-set qualitative comparative analysis (fsQCA) to further disentangle the effects of intervention components.

RESULTS

Our final sample comprised 41 articles reporting the results of 37 studies. For Fitbit-based interventions, we found a statistically significant increase in daily step count (mean difference [MD] 950.54, 95% CI 475.89-1425.18; P<.001) and moderate-to-vigorous physical activity (MD 6.16, 95% CI 2.80-9.51; P<.001), a significant decrease in weight (MD -1.48, 95% CI -2.81 to -0.14; P=.03), and a nonsignificant decrease in objectively assessed and self-reported sedentary behavior (MD -10.62, 95% CI -35.50 to 14.27; P=.40 and standardized MD -0.11, 95% CI -0.48 to 0.26; P=.56, respectively). In general, the included studies were at low risk for bias, except for performance bias. Subgroup analysis and fsQCA demonstrated that, in addition to the effects of the Fitbit devices, setting activity goals was the most important intervention component.

CONCLUSIONS

The use of Fitbit devices in interventions has the potential to promote healthy lifestyles in terms of physical activity and weight. Fitbit devices may be useful to health professionals for patient monitoring and support.

TRIAL REGISTRATION

PROSPERO International Prospective Register of Systematic Reviews CRD42019145450; https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42019145450.

When complexity matters: a step-by-step guide to incorporating a complexity perspective in guideline development for public health and health system interventions

BACKGROUND

Guidelines on public health and health system interventions often involve considerations beyond effectiveness and safety to account for the impact that these interventions have on the wider systems in which they are implemented. This paper describes how a complexity perspective may be adopted in guideline development to facilitate a more nuanced consideration of a range of factors pertinent to decisions regarding public health and health system interventions. These factors include acceptability and feasibility, and societal, economic, and equity and equality implications of interventions.

MAIN MESSAGE

A 5-step process describes how to incorporate a complexity perspective in guideline development with examples to illustrate each step. The steps include: (i) guideline scoping, (ii) formulating questions, (iii) retrieving and synthesising evidence, (iv) assessing the evidence, and (v) developing recommendations. Guideline scoping using stakeholder consultations, complexity features, evidence mapping, logic modelling, and explicit decision criteria is emphasised as a key step that informs all subsequent steps.

CONCLUSIONS

Through explicit consideration of a range of factors and enhanced understanding of the specific circumstances in which interventions work, a complexity perspective can yield guidelines with better informed recommendations and facilitate local adaptation and implementation. Further work will need to look into the methods of collecting and assessing different types of evidence beyond effectiveness and develop procedural guidance for prioritising across a range of decision criteria.

Will it work here? A realist approach to local decisions about implementing interventions evaluated as effective elsewhere

There is increasing interest in what evidence is needed to inform decisions about transporting interventions evaluated as effective to new settings. There has been less attention to how local decision-makers decide whether to implement such interventions immediately or subject to further evaluation. Using the example of school-based social and emotional learning, we consider this drawing on realist methods. We suggest decisions need to assess existing evaluations not merely in terms of whether the intervention was effective but also: how the intervention was implemented and what contextual factors affected this (drawing on process evaluation); and for whom the intervention was effective and through what mechanisms (drawing on mediation, moderation and qualitative comparative analyses from primary studies and/or systematic reviews). We contribute new insights to local needs assessments, suggesting that these should assess: the potential, capability, contribution and capacity present in the new setting for implementation; and whether similar 'aetiological mechanisms' underlie adverse outcomes locally as in previous evaluations. We recommend that where there is uncertainty concerning whether an intervention can feasibly be implemented this indicates the need for piloting of implementation. Where there is uncertainty concerning whether implementation of the intervention will trigger intended mechanisms, this suggests the need for a new effectiveness trial. Where there is uncertainty concerning whether intervention mechanisms, even if triggered, will generate the intended outcomes, this suggests that decision-makers may need to look to other types of intervention as being needed for their setting instead.

Variations of mixed methods reviews approaches: A case study

Conducting mixed methods reviews is challenging. The aim of this article is to describe a range of rationales for and approaches to mixed methods reviews, with a particular focus on one research group. A case study was conducted to describe the mixed methods review process used at the Department of Health and Social Care Reviews Facility in England. The case study used document analysis. A total of 30 mixed methods reviews were identified and analyzed. The analysis revealed five key dimensions on which the reviews varied: review questions and purposes of the mixed methods questions, types of evidence and sources, reasons for using a mixed methods approach, synthesis methods and designs, and integration strategies. The questions in the included reviews addressed stakeholders' views, and intervention processes and/or intervention effectiveness. The mixed methods questions addressed four different purposes: comparing findings, identifying critical intervention features, quantifying effects, and making recommendations. Five main sources of evidence were used: formal evidence from primary studies, informal evidence, policy documents, systematic reviews, and work with stakeholders. Twelve reasons for conducting mixed methods reviews were identified: completeness, contextual understanding, credibility, different research questions, diversity of views, enhancement, explanation, process, triangulation, utility, development of a framework, and identification of promising interventions. Each review employed one or several integration strategies for comparing findings, connecting phases and/or assimilating data. It is hoped that the information garnered from this study will provide useful insights into mixed method review diversity and trigger new ideas for conducting this type of review.

Implementation of e-mental health interventions for informal caregivers of adults with chronic diseases: a protocol for a mixed-methods systematic review with a qualitative comparative analysis

INTRODUCTION

Informal caregivers provide the majority of care to individuals with chronic health conditions, benefiting the care recipient and reducing use of formal care services. However, providing informal care negatively impacts the mental health of many caregivers. E-mental health interventions have emerged as a way to provide accessible mental healthcare to caregivers. Much attention has been given to reviewing the effectiveness and efficacy of such interventions, however, factors related to implementation have received less consideration. Therefore, this mixed-methods systematic review will aim to examine factors associated with the effectiveness and implementation of e-mental health interventions for caregivers.

METHODS AND ANALYSIS

Eligible studies published since 1 January 2007 will be searched for in several electronic databases (CINAHL Plus with Full Text, the Cochrane Library, EMBASE, PsycINFO, PubMed and Web of Science), clinical trial registries and OpenGrey, with all screening steps conducted by two independent reviewers. Studies will be included if they focus on the implementation or effectiveness of e-mental health interventions designed for informal adult caregivers of adults with cancer, heart disease, stroke, diabetes, dementia or chronic obstructive pulmonary disease. Pragmatic randomised controlled trials quantitatively reporting on caregiver anxiety, depression, psychological distress or stress will be used for a qualitative comparative analysis to identify combinations of conditions that result in effective interventions. Qualitative and quantitative data on implementation of e-mental health interventions for caregivers will be integrated in a thematic synthesis to identify barriers and facilitators to implementation. These results will inform future development and implementation planning of e-mental health interventions for caregivers.

ETHICS AND DISSEMINATION

Ethical approval is not required for this study as no primary data will be collected. Results will be disseminated in the form of a scientific publication and presentations at academic conferences and plain language summaries for various stakeholders.

PROSPERO REGISTRATION NUMBER

CRD42020155727.

When assessing generalisability, focusing on differences in population or setting alone is insufficient

Generalisability is typically only briefly mentioned in discussion sections of evaluation articles, which are unhelpful in judging whether an intervention could be implemented elsewhere, with similar effects. Several tools to assess generalisability exist, but they are difficult to operationalise and are rarely used. We believe a different approach is needed. Instead of focusing on similarities (or more likely, differences) in generic population and setting characteristics, generalisability assessments should focus on understanding an intervention’s mechanism of action - why or how an intervention was effective. We believe changes are needed to four types of research. First, outcome evaluations should draw on programme theory. Second, process evaluations should aim to understand interventions’ mechanism of action, rather than simply ‘what happened’. Third, small scoping studies should be conducted in new settings, to explore how to enact identified mechanisms. Finally, innovative synthesis methods are required, in order to identify mechanisms of action where there is a lack of existing process evaluations.

Developing methods for the overarching synthesis of quantitative and qualitative evidence: The interweave synthesis approach

The incorporation of evidence derived from multiple research designs into one single synthesis can enhance the utility of systematic reviews making them more worthwhile, useful, and insightful. Methodological guidance for mixed-methods synthesis continues to emerge and evolve but broadly involves a sequential, parallel, or convergent approach according to the degree of independence between individual syntheses before they are combined. We present two case studies in which we used novel and innovative methods to draw together the findings from individual but related quantitative and qualitative syntheses to aid interpretation of the overall evidence base. Our approach moved beyond making a choice between parallel, sequential, or convergent methods to interweave the findings of individual reviews and offers three key innovations to mixed-methods synthesis methods: The use of intersubjective questions to understand the findings of the individual reviews through different lenses, Immersion of key reviewers in the entirety of the evidence base, and Commencing the process during the final stages of the synthesis of individual reviews, at a point where reviewers are developing an understanding of initial findings. Underlying our approach is the process of exploration and identification of links between and across review findings, an approach that is fundamental to all evidence syntheses but usually occurs at the level of the study. Adapting existing methods for exploring and identifying patterns and links between and across studies to interweave the findings between and across reviews may prove valuable.

Exploring the importance of evidence in local health and wellbeing strategies

As evidence generators, we need to respond to the changes in the health delivery landscape if we are to continue to support public health decision-makers to make informed and judicious evidence-based choices. This study employs documentary analysis to (i) explore the extent of research evidence use in public health decision-making; (ii) to analyse occurrences of research evidence use in decision-making and (iii) to ascertain whether patterns of evidence use overlap with other area characteristics. Health and Wellbeing Strategies constitute the main source of documentary evidence. Initial results highlight that local areas are undertaking their own programmes of research that are used to inform specific questions, although the methodological robustness of these studies is unknown. There are also commonalities with previous findings, particularly with regard to the underutilization of qualitative research evidence and evidence on the effectiveness of interventions. Using Qualitative Comparative Analysis, this paper also highlights that underutilization of more academic research evidence appears disproportionally weighted towards areas with some of the most complex needs but that are not receiving the highest level of spending to meet these challenges. These areas in particular may be those where knowledge brokerage activities may have the greatest impacts.

Clarifying differences between reviews within evidence ecosystems

This paper builds on a 2012 paper by the same authors which argued that the types and brands of systematic review do not sufficiently differentiate between the many dimensions of different review questions and review methods (Gough et al., Syst Rev 1:28, 2012). The current paper extends this argument by considering the dynamic contexts, or 'evidence ecosystems', within which reviews are undertaken; the fact that these ecosystems are constantly changing; and the relevance of this broader context for understanding 'dimensions of difference' in the unfolding development and refinement of review methods.The concept of an evidence ecosystem is used to consider particular issues within the three key dimensions of difference outlined in the 2012 paper of (1) review aims and approach, (2) structure and components of reviews, and (3) breadth, depth, and 'work done' by reviews.

Title: Can changing the physical environment promote walking and cycling? A systematic review of what works and how

Environmental changes aimed at encouraging walking or cycling may promote activity and improve health, but evidence suggests small or inconsistent effects in practice. Understanding how an intervention works might help explain the effects observed and provide guidance about generalisability. We therefore aimed to review the literature on the effects of this type of intervention and to understand how and why these may or may not be effective. We searched eight electronic databases for existing systematic reviews and mined these for evaluative studies of physical environmental changes and assessed changes in walking, cycling or physical activity. We then searched for related sources including quantitative or qualitative studies, policy documents or reports. We extracted information on the evidence for effects ('estimation'), contexts and mechanisms ('explanation') and assessed credibility, and synthesised material narratively. We identified 13 evaluations of interventions specifically targeting walking and cycling and used 46 related sources. 70% (n = 9 evaluations) scored 3 or less on the credibility criteria for effectiveness. 6 reported significant positive effects, but higher quality evaluations were more likely to report positive effects. Only two studies provided rich evidence of mechanisms. We identified three common resources that interventions provide to promote walking and cycling: (i) improving accessibility and connectivity; (ii) improving traffic and personal safety; and (iii) improving the experience of walking and cycling. The most effective interventions appeared to target accessibility and safety in both supportive and unsupportive contexts. Although the evidence base was relatively limited, we were able to understand the role of context in the success of interventions. Researchers and policy makers should consider the context and mechanisms which might operate before evaluating and implementing interventions.

Theory-based self-management intervention to improve adolescents' asthma control: a cluster randomised controlled trial protocol

INTRODUCTION

Asthma-related morbidity and mortality in the UK is higher than elsewhere in Europe. Although the reasons for this are largely unclear, one explanation could be a higher prevalence of poorly controlled asthma in the UK. Findings from our earlier study found that, in a sample of 766 children with asthma, 45.7% had poorly controlled asthma. Our earlier study also showed that adherence to inhaled corticosteroids was low. Subsequent focus groups identified concerns regarding embarrassment and bullying as barriers to adherence, as well as forgetfulness and incorrect medication beliefs. Following this, a school-based self-management intervention has been developed, aimed to improve asthma control and self-management behaviours.

METHODS AND ANALYSIS

The theory-based cluster randomised controlled trial tests an intervention comprising two components: (1) a theatre workshop for all children in years 7 and 8, and (2) self-management workshops for children with asthma. The COM-B model was used to guide the development of the intervention. Questionnaire data will be collected in schools at baseline, immediately post intervention, and 3, 6 and 12 months post intervention. The data collected at 6 months will measure the effect of the intervention against the baseline data. The primary outcome will be asthma control, measured using the Asthma Control Test. All the data will be analysed quantitatively using generalised linear and non-linear mixed effects models.

ETHICS AND DISSEMINATION

Ethical approval was obtained by the Queen Mary University of London Ethics Committee on 12 April 2018. Regular meetings will be held with key patient and public stakeholders to plan the key messages from this research. Key messages from the study will also be tweeted via the project twitter account (@SchoolsAsthma). The findings of the study will be submitted for presentation at conferences, as well as written into a manuscript.

TRIAL REGISTRATION NUMBER

MGU0400.

Developing and testing intervention theory by incorporating a views synthesis into a qualitative comparative analysis of intervention effectiveness

Qualitative comparative analysis (QCA) was originally developed as a tool for cross-national comparisons in macrosociology, but its use in evaluation and evidence synthesis of complex interventions is rapidly developing. QCA is theory-driven and relies on Boolean logic to identify pathways to an outcome (eg, is the intervention effective or not?). We use the example of two linked systematic reviews on weight management programs (WMPs) for adults-one focusing on user views (a "views synthesis") and one focusing on the effectiveness of WMPs incorporating dietary and physical activity-to demonstrate how a synthesis of user views can supply a working theory to structure a QCA. We discuss how a views synthesis is especially apt to supply this working theory because user views can (a) represent a "middle-range theory" of the intervention; (b) bring a participatory, democratic perspective; and (c) provide an idiographic understanding of how the intervention works that external taxonomies may not be able to furnish. We then discuss the practical role that the views synthesis played in our QCA examining pathways to effectiveness: (a) by suggesting specific intervention features and sharpening the focus on the most salient features to be examined, (b) by supporting interpretation of findings, and (c) by bounding data analysis to prevent data dredging.

School-based self-management interventions for asthma in children and adolescents: a mixed methods systematic review

BACKGROUND

Asthma is a common respiratory condition in children that is characterised by symptoms including wheeze, shortness of breath, chest tightness, and cough. Children with asthma may be able to manage their condition more effectively by improving inhaler technique, and by recognising and responding to symptoms. Schools offer a potentially supportive environment for delivering interventions aimed at improving self-management skills among children. The educational ethos aligns with skill and knowledge acquisition and makes it easier to reach children with asthma who do not regularly engage with primary care. Given the multi-faceted nature of self-management interventions, there is a need to understand the combination of intervention features that are associated with successful delivery of asthma self-management programmes.

OBJECTIVES

This review has two primary objectives.• To identify the intervention features that are aligned with successful intervention implementation.• To assess effectiveness of school-based interventions provided to improve asthma self-management among children.We addressed the first objective by performing qualitative comparative analysis (QCA), a synthesis method described in depth later, of process evaluation studies to identify the combination of intervention components and processes that are aligned with successful intervention implementation.We pursued the second objective by undertaking meta-analyses of outcomes reported by outcome evaluation studies. We explored the link between how well an intervention is implemented and its effectiveness by using separate models, as well as by undertaking additional subgroup analyses.

SEARCH METHODS

We searched the Cochrane Airways Trials Register for randomised studies. To identify eligible process evaluation studies, we searched MEDLINE, Embase, the Cumulative Index to Nursing and Allied Health Literature (CINAHL), PsycINFO, the Cochrane Database of Systematic Reviews (CDSR), Web of Knowledge, the Database of Promoting Health Effectiveness Reviews (DoPHER), the Database of Abstracts of Reviews of Effects (DARE), the International Biography of Social Science (IBSS), Bibliomap, Health Technology Assessment (HTA), Applied Social Sciences Index and Abstracts (ASSIA), and Sociological Abstracts (SocAbs). We conducted the latest search on 28 August 2017.

SELECTION CRITERIA

Participants were school-aged children with asthma who received the intervention in school. Interventions were eligible if their purpose was to help children improve management of their asthma by increasing knowledge, enhancing skills, or changing behaviour. Studies relevant to our first objective could be based on an experimental or quasi-experimental design and could use qualitative or quantitative methods of data collection. For the second objective we included randomised controlled trials (RCTs) where children were allocated individually or in clusters (e.g. classrooms or schools) to self-management interventions or no intervention control.

DATA COLLECTION AND ANALYSIS

We used qualitative comparative analysis (QCA) to identify intervention features that lead to successful implementation of asthma self-management interventions. We measured implementation success by reviewing reports of attrition, intervention dosage, and treatment adherence, irrespective of effects of the interventions.To measure the effects of interventions, we combined data from eligible studies for our primary outcomes: admission to hospital, emergency department (ED) visits, absence from school, and days of restricted activity due to asthma symptoms. Secondary outcomes included unplanned visits to healthcare providers, daytime and night-time symptoms, use of reliever therapies, and health-related quality of life as measured by the Asthma Quality of Life Questionnaire (AQLQ).

MAIN RESULTS

We included 55 studies in the review. Thirty-three studies in 14,174 children provided information for the QCA, and 33 RCTs in 12,623 children measured the effects of interventions. Eleven studies contributed to both the QCA and the analysis of effectiveness. Most studies were conducted in North America in socially disadvantaged populations. High school students were better represented among studies contributing to the QCA than in studies contributing to effectiveness evaluations, which more commonly included younger elementary and junior high school students. The interventions all attempted to improve knowledge of asthma, its triggers, and stressed the importance of regular practitioner review, although there was variation in how they were delivered.QCA results highlighted the importance of an intervention being theory driven, along with the importance of factors such as parent involvement, child satisfaction, and running the intervention outside the child's own time as drivers of successful implementation.Compared with no intervention, school-based self-management interventions probably reduce mean hospitalisations by an average of about 0.16 admissions per child over 12 months (SMD -0.19, 95% CI -0.35 to -0.04; 1873 participants; 6 studies, moderate certainty evidence). They may reduce the number of children who visit EDs from 7.5% to 5.4% over 12 months (OR 0.70, 95% CI 0.53 to 0.92; 3883 participants; 13 studies, low certainty evidence), and probably reduce unplanned visits to hospitals or primary care from 26% to 21% at 6 to 9 months (OR 0.74, 95% CI 0.60 to 0.90; 3490 participants; 5 studies, moderate certainty evidence). Self-management interventions probably reduce the number of days of restricted activity by just under half a day over a two-week period (MD 0.38 days 95% CI -0.41 to -0.18; 1852 participants; 3 studies, moderate certainty evidence). Effects of interventions on school absence are uncertain due to the variation between the results of the studies (MD 0.4 fewer school days missed per year with self-management (-1.25 to 0.45; 4609 participants; 10 studies, low certainty evidence). Evidence is insufficient to show whether the requirement for reliever medications is affected by these interventions (OR 0.52, 95% CI 0.15 to 1.81; 437 participants; 2 studies; very low-certainty evidence). Self-management interventions probably improve children's asthma-related quality of life by a small amount (MD 0.36 units higher on the Paediatric AQLQ(95% CI 0.06 to 0.64; 2587 participants; 7 studies, moderate certainty evidence).

AUTHORS' CONCLUSIONS

School-based asthma self-management interventions probably reduce hospital admission and may slightly reduce ED attendance, although their impact on school attendance could not be measured reliably. They may also reduce the number of days where children experience asthma symptoms, and probably lead to small improvements in asthma-related quality of life. Many of the studies tested the intervention in younger children from socially disadvantaged populations. Interventions that had a theoretical framework, engaged parents and were run outside of children's free time were associated with successful implementation.

Synthesising quantitative evidence in systematic reviews of complex health interventions

Public health and health service interventions are typically complex: they are multifaceted, with impacts at multiple levels and on multiple stakeholders. Systematic reviews evaluating the effects of complex health interventions can be challenging to conduct. This paper is part of a special series of papers considering these challenges particularly in the context of WHO guideline development. We outline established and innovative methods for synthesising quantitative evidence within a systematic review of a complex intervention, including considerations of the complexity of the system into which the intervention is introduced. We describe methods in three broad areas: non-quantitative approaches, including tabulation, narrative and graphical approaches; standard meta-analysis methods, including meta-regression to investigate study-level moderators of effect; and advanced synthesis methods, in which models allow exploration of intervention components, investigation of both moderators and mediators, examination of mechanisms, and exploration of complexities of the system. We offer guidance on the choice of approach that might be taken by people collating evidence in support of guideline development, and emphasise that the appropriate methods will depend on the purpose of the synthesis, the similarity of the studies included in the review, the level of detail available from the studies, the nature of the results reported in the studies, the expertise of the synthesis team and the resources available.

Interventions targeting conscious determinants of human behaviour to reduce the demand for meat: a systematic review with qualitative comparative analysis

BACKGROUND

Reducing meat consumption can help prevent non-communicable diseases and protect the environment. Interventions targeting conscious determinants of human behaviour are generally acceptable approaches to promote dietary change, but little is known about their effectiveness to reduce the demand for meat.

OBJECTIVE

To evaluate the effectiveness of interventions targeting conscious determinants of human behaviour to reduce the demand for meat.

METHODS

We searched six electronic databases on the 31st of August 2017 with a predefined algorithm, screened publicly accessible resources, contacted authors, and conducted forward and backward reference searches. Eligible studies employed experimental designs to evaluate interventions targeting conscious determinants of human behaviour to reduce the consumption, purchase, or selection of meat in comparison to a control condition, a baseline period, or relative to other eligible interventions. We synthesised results narratively and conducted an exploratory crisp-set Qualitative Comparative Analysis to identify combinations of intervention characteristics associated with significant reductions in the demand for meat.

RESULTS

We included 24 papers reporting on 59 interventions and 25,477 observations. Self-monitoring interventions and individual lifestyle counselling led to, or were associated with reduced meat consumption. Providing information about the health or environmental consequences of eating meat was associated with reduced intentions to consume and select meat in virtual environments, but there was no evidence to suggest this approach influenced actual behaviour. Education about the animal welfare consequences of eating meat was associated with reduced intentions to consume meat, while interventions implicitly highlighting animal suffering were not. Education on multiple consequences of eating meat led to mixed results. Tailored education was not found to reduce actual or intended meat consumption, though few studies assessed this approach.

CONCLUSION

Some interventions targeting conscious determinants of human behaviour have the potential to reduce the demand for meat. In particular, self-monitoring interventions and individual lifestyle counselling can help to reduce meat consumption. There was evidence of effectiveness of some educational messages in reducing intended consumption and selection of meat in virtual environments.

PROTOCOL REGISTRATION

CRD42017076720 .

Restructuring physical micro-environments to reduce the demand for meat: a systematic review and qualitative comparative analysis

BACKGROUND

Reducing meat consumption could help to protect the natural environment and promote population health. Interventions restructuring physical micro-environments might help to change habitual behaviour. We synthesised the scientific evidence pertaining to whether, and which, interventions restructuring physical micro-environments effectively reduce the demand for meat.

METHODS

We did a systematic review of quantitative studies evaluating the effectiveness of interventions restructuring physical micro-environments to reduce the demand for meat. We identified relevant records by searching six electronic databases (CAB Abstracts, Embase, PsycINFO, Science Citation Index, MEDLINE, and Dissertations & Theses) on Aug 31, 2017, contacting experts, screening publicly accessible online resources, and searching references. We included studies that evaluated the effectiveness of interventions restructuring physical micro-environments to reduce the demand for meat, defined as the actual or intended consumption, purchase, or selection of meat in real or virtual environments. We extracted data pertaining to the study samples, the interventions, and meat demand at the follow-up closest to intervention completion and at the longest follow-up, with the former representing our primary outcome. We synthesised data narratively and did a qualitative comparative analysis to identify configurations of intervention characteristics associated with, and those not found to be associated with, significant reductions in meat demand. Our Systematic Review is registered with PROSPERO, number CRD42017081532.

RESULTS

Of 10 733 titles and abstracts screened for eligibility, we assessed 60 full papers and included 14 papers reporting on 18 studies with 22 intervention conditions. Three interventions reducing the portion size of meat servings reduced meat consumption in randomised trials. Three interventions providing meat alternatives with supporting educational material were associated with reduced meat demand in pre-post design studies. Three of four interventions altering the sensory properties (eg, visual presentation) of meat or meat alternatives at point of purchase reduced meat demand in randomised trials. Four interventions repositioning meat products to be less prominent at point of purchase were associated with lower meat demand, but only two such interventions reached statistical significance in a randomised trial and a multiple treatment reversal design. Only one of five interventions manipulating the description of meat or meat alternatives at point of purchase was associated with lower meat demand in a multiple treatment reversal design. Evidence from randomised trials evaluating a pricing intervention or interventions restructuring several aspects of micro-environments was too scarce or inconsistent to be conclusive. The results from our qualitative comparative analysis supported the findings of this narrative synthesis.

INTERPRETATION

Some interventions restructuring physical micro-environments could help to promote lower demand for meat. Interventions reducing portion sizes of meat servings, providing meat alternatives, or changing the sensory properties of meat and meat alternatives at point of purchase offered the most promise in the context of experimental studies.

FUNDING

None.

Grocery store interventions to change food purchasing behaviors: a systematic review of randomized controlled trials

Background

Diet is an important determinant of health, and food purchasing is a key antecedent to consumption.

Objective

We set out to evaluate the effectiveness of grocery store interventions to change food purchasing, and to examine whether effectiveness varied based on intervention components, setting, or socioeconomic status.

Design

We conducted a systematic review of randomized controlled trials (search performed June 2017). Studies must have: aimed to change food purchasing; been implemented in grocery stores (real or simulated); reported purchasing; and had a minimal control or compared interventions fulfilling our criteria. Searching, screening, bias assessment, and data extraction followed Cochrane methods. We grouped studies by intervention type (economic, environmental, swaps, and/or education), synthesized results narratively, and conducted an exploratory qualitative comparative analysis.

Results

We included 35 studies representing 89 interventions, >20,000 participants, and >800 stores. Risk of bias was mixed. Economic interventions showed the most promise, with 8 of the 9 studies in real stores and all 6 in simulated environments detecting an effect on purchasing. Swap interventions appeared promising in the 2 studies based in real stores. Store environment interventions showed mixed effects. Education-only interventions appeared effective in simulated environments but not in real stores. Available data suggested that effects of economic interventions did not differ by socioeconomic status, whereas for other interventions impact was variable. In our qualitative comparative analysis, economic interventions (regardless of setting) and environmental and swap interventions in real stores were associated with statistically significant changes in purchasing in the desired direction for ≥1 of the foods targeted by the intervention, whereas education-only interventions in real stores were not.

Conclusions

Findings suggest that interventions implemented in grocery stores-particularly ones that manipulate price, suggest swaps, and perhaps manipulate item availability-have an impact on purchasing and could play a role in public health strategies to improve health. Review protocol registered at https://www.crd.york.ac.uk/PROSPERO/ as CRD42017068809.

Colliding ideals - an interview study of how intervention researchers address adherence and adaptations in replication studies

Background

For an intervention to be considered evidence-based, findings need to be replicated. When this is done in new contexts (e.g., a new country), adaptations may be needed. Yet, we know little about how researchers approach this. This study aims to explore how researchers reason about adaptations and adherence when conducting replication studies, describe what adaptations they make and how these are reported in scientific journals.

Methods

This was an interview study conducted in 2014 with principal investigators of Swedish replication studies reporting adaptations to an intervention from another country. Studies (n = 36) were identified through a database of 139 Swedish psychosocial and psychological intervention studies. Twenty of the 21 principal investigators agreed to participate in semi-structured telephone interviews, covering 33 interventions. Manifest content analysis was used to identify types of adaptations, and qualitative content analysis was used to explore reasoning and reporting of adaptations and adherence.

Results

The most common adaptation was adding components and modifying the content to the target population and setting. When reasoning about adaptations and adherence, the researchers were influenced by four main factors: whether their implicit aim was to replicate or improve an intervention; the nature of evidence outlying the intervention such as manuals, theories and core components; the nature of the context, including approaches to cultural adaptations and constraints in delivering the intervention; and the needs of clients and professionals. Reporting of adaptations in scientific journals involved a conflict between transparency and practical concerns such as word count.

Conclusions

Researchers responsible for replicating interventions in a new country face colliding ideals when trying to protect the internal validity of the study while considering adaptations to ensure that the intervention fits into the context. Implicit assumptions about the role of replication seemed to influence how this conflict was resolved. Some emphasised direct replications as central in the knowledge accumulation process (stressing adherence). Others assumed that interventions generally need to be improved, giving room for adaptations and reflecting an incremental approach to knowledge accumulation. This has implications for design and reporting of intervention studies as well as for how findings across studies are synthesised.

Managing Complexity in Evidence Analysis: A Worked Example in Pediatric Weight Management

Nutrition interventions are often complex and multicomponent. Typical approaches to meta-analyses that focus on individual causal relationships to provide guideline recommendations are not sufficient to capture this complexity. The objective of this study is to describe the method of meta-analysis used for the Pediatric Weight Management (PWM) Guidelines update and provide a worked example that can be applied in other areas of dietetics practice. The effects of PWM interventions were examined for body mass index (BMI), body mass index z-score (BMIZ), and waist circumference at four different time periods. For intervention-level effects, intervention types were identified empirically using multiple correspondence analysis paired with cluster analysis. Pooled effects of identified types were examined using random effects meta-analysis models. Differences in effects among types were examined using meta-regression. Context-level effects are examined using qualitative comparative analysis. Three distinct types (or families) of PWM interventions were identified: medical nutrition, behavioral, and missing components. Medical nutrition and behavioral types showed statistically significant improvements in BMIZ across all time points. Results were less consistent for BMI and waist circumference, although four distinct patterns of weight status change were identified. These varied by intervention type as well as outcome measure. Meta-regression indicated statistically significant differences between the medical nutrition and behavioral types vs the missing component type for both BMIZ and BMI, although the pattern varied by time period and intervention type. Qualitative comparative analysis identified distinct configurations of context characteristics at each time point that were consistent with positive outcomes among the intervention types. Although analysis of individual causal relationships is invaluable, this approach is inadequate to capture the complexity of dietetics practice. An alternative approach that integrates intervention-level with context-level meta-analyses may provide deeper understanding in the development of practice guidelines.