Time to rethink the systematic review catechism? Moving from 'what works' to 'what happens'

Supported employment interventions with people who have severe mental illness: Systematic mixed-methods umbrella review

BACKGROUND/AIMS

Primary and review studies show that supported employment interventions showed promise in assisting people with severe mental illness (SMI) in achieving successful employment and health-related outcomes. This umbrella review synthesises evidence from across review studies on supported employment interventions for individuals with SMI, to identify key findings and implementation challenges in relation to five key outcomes: (1) employment, (2) quality of life, (3) social functioning, (4) clinical/service utilisation, and (5) economic outcomes.

METHODS

A systematic search of eleven databases and registers (CINAHL, Cochrane, EmCare, JBI EBP, ProQuest, PsycINFO, PubMed, Scopus, and Web of Science, and Prospero and Campbell) was conducted to identify meta-analyses and systematic reviews on supported employment interventions for individuals with SMI, peer reviewed and published in English. Quality assessment and data extraction were performed using standardised Joanna Briggs Institute (JBI) tools. A mixed-methods synthesis approach was employed to integrate both quantitative and qualitative evidence.

RESULTS

The synthesis of 26 review studies primarily focused on the Individual Placement and Support (IPS) model among various supported employment interventions. Overall, combining supported employment with targeted interventions such as neurocognitive therapy and job-related social skill training showed a positive effect on employment (including job retention) and non-employment outcomes (e.g., health, quality of life, social functioning) relative to standard forms of supported employment for people with SMI. Contextual factors (intervention fidelity, settings, systemic barriers) were important considerations for intervention implementation and effectiveness.

DISCUSSION

Significant overlap of primary studies across 26 review studies exposed considerable variations in interpretation and conclusions drawn by authors, raising questions about their reliability. High volume of overlap reporting from the USA on IPS interventions in review studies is likely to have biased perceptions of effectiveness. There is no one-size-fits-all solution for supporting individuals with SMI in obtaining and maintaining employment. Tailoring strategies based on individual needs and circumstances appears crucial to address the complexity of mental health recovery. We propose creating centralised registries or databases to monitor primary studies included in reviews, thus avoiding redundancy.

OTHER

This umbrella study was registered with PROSPERO (No. CRD42023431191).

Exploring undiscovered public knowledge in neuroscience

In this essay, I argue that the combination of research synthesis and philosophical methods can fill an important methodological gap in neuroscience. While experimental research and formal modelling have seen their methods progressively increase in rigour and sophistication over the years, the task of analysing and synthesizing the vast literature reporting new results and models has lagged behind. The problem is aggravated because neuroscience has grown and expanded into a vast mosaic of related but partially independent subfields, each with their own literatures. This fragmentation not only makes it difficult to see the full picture emerging from neuroscience research but also limits progress in individual subfields. The current neuroscience literature has the perfect conditions to create what the information scientist Don Swanson called "undiscovered public knowledge"-knowledge that exists in the mutual implications of different published pieces of information but that is nonetheless undiscovered because those pieces have not been explicitly connected. Current methods for rigorous research synthesis, such as systematic reviews and meta-analyses, mostly focus on combining similar studies and are not suited for exploring undiscovered public knowledge. To that aim, they need to be adapted and supplemented. I argue that successful exploration of the hidden implications in the neuroscience literature will require the combination of these adapted research synthesis methods with philosophical methods for rigorous (and creative) analysis and synthesis.

Investigating health and social outcomes of the Big Local community empowerment initiative in England: a mixed method evaluation

Background

Most research on community empowerment provides evidence on engaging communities for health promotion purposes rather than attempts to create empowering conditions. This study addresses this gap.

Intervention

Big Local started in 2010 with £271M from the National Lottery. Ending in 2026, it gives 150 relatively disadvantaged communities in England control over £1M to improve their neighbourhoods.

Objective

To investigate health and social outcomes, at the population level and among engaged residents, of the community engagement approach adopted in a place-based empowerment initiative.

Study design, data sources and outcome variables

This study reports on the third wave of a longitudinal mixed-methods evaluation. Work package 1 used a difference-in-differences design to investigate the impact of Big Local on population outcomes in all 150 Big Local areas compared to matched comparator areas using secondary data. The primary outcome was anxiety; secondary outcomes included a population mental health measure and crime in the neighbourhood. Work package 2 assessed active engagement in Big Local using cross-sectional data and nested cohort data from a biannual survey of Big Local partnership members. The primary outcome was mental well-being and the secondary outcome was self-rated health. Work package 3 conducted qualitative research in 14 Big Local neighbourhoods and nationally to understand pathways to impact. Work package 4 undertook a cost-benefit analysis using the life satisfaction approach to value the benefits of Big Local, which used the work package 1 estimate of Big Local impact on life satisfaction.

Results

At a population level, the impacts on 'reporting high anxiety' (-0.8 percentage points, 95% confidence interval -2.4 to 0.7) and secondary outcomes were not statistically significant, except burglary (-0.054 change in z-score, 95% confidence interval -0.100 to -0.009). There was some effect on reduced anxiety after 2017. Areas progressing fastest had a statistically significant reduction in population mental health measure (-0.053 change in z-score, 95% confidence interval -0.103 to -0.002). Mixed results were found among engaged residents, including a significant increase in mental well-being in Big Local residents in the nested cohort in 2018, but not by 2020; this is likely to be COVID-19. More highly educated residents, and males, were more likely to report a significant improvement in mental well-being. Qualitative accounts of positive impacts on mental well-being are often related to improved social connectivity and physical/material environments. Qualitative data revealed increasing capabilities for residents' collective control. Some negative impacts were reported, with local factors sometimes undermining residents' ability to exercise collective control. Finally, on the most conservative estimate, the cost-benefit calculations generate a net benefit estimate of £64M.

Main limitations

COVID-19 impacted fieldwork and interpretation of survey data. There was a short 4-year follow-up (2016/20), no comparators in work package 2 and a lack of power to look at variations across areas.

Conclusions

Our findings suggest the need for investment to support community organisations to emerge from and work with communities. Residents should lead the prioritisation of issues and design of solutions but not necessarily lead action; rather, agencies should work as equal partners with communities to deliver change.

Funding

This project was funded by the National Institute for Health and Care Research (NIHR) Public Health Research Programme (16/09/13) and will be published in full in Public Health Research; Vol. 11, No. 9. See the NIHR Journals Library website for further project information.

How to Conduct a Systematic Review and Meta-Analysis: A Guide for Clinicians

Evidence-based practice relies on using research evidence to guide clinical decision-making. However, staying current with all published research can be challenging. Many clinicians use review articles that apply predefined methods to locate, identify, and summarize all available evidence on a topic to guide clinical decision-making. This paper discusses the role of review articles, including narrative, scoping, and systematic reviews, to synthesize existing evidence and generate new knowledge. It provides a step-by-step guide to conducting a systematic review and meta-analysis, covering key steps such as formulating a research question, selecting studies, evaluating evidence quality, and reporting results. This paper is intended as a resource for clinicians looking to learn how to conduct systematic reviews and advance evidence-based practice in the field.

Reducing LGBTQ+ adolescent mental health inequalities: a realist review of school-based interventions

BACKGROUND

LGBTQ+ young people have elevated rates of poor mental health in comparison to their cisgender heterosexual peers. School environment is a key risk factor and consistently associated with negative mental health outcomes for LGBTQ+ adolescents.

AIMS

To examine how, why, for whom and in what context school-based interventions prevent or reduce mental health problems in LGBTQ+ adolescents.

METHODS

A realist review methodology was utilised and focused on all types of school-based interventions and study designs. A Youth Advisory Group were part of the research team. Multiple search strategies were used to locate relevant evidence. Studies were subject to inclusion criteria and quality appraisal, and included studies were synthesised to produce a programme theory. Seventeen studies were included in the review.

RESULTS

Eight intervention components were necessary to address LGBTQ+ pupils mental health: affirmative visual displays; external signposting to LGBTQ+ support; stand-alone input; school-based LGBTQ support groups; curriculum-based delivery; staff training; inclusion policies; trusted adult. Few school-based interventions for this population group were identified.

CONCLUSIONS

The programme theory indicates that "to work" school-based interventions must have a "whole-school" approach that addresses specifically the dominant cis-heteronormative school environment and hence the marginalisation, silence, and victimisation that LGBTQ+ pupils can experience.

Prepulse inhibition deficit as a transdiagnostic process in neuropsychiatric disorders: a systematic review

BACKGROUND

Psychopathological research is moving from a specific approach towards transdiagnosis through the analysis of processes that appear transversally to multiple pathologies. A phenomenon disrupted in several disorders is prepulse inhibition (PPI) of the startle response, in which startle to an intense sensory stimulus, or pulse, is reduced if a weak stimulus, or prepulse, is previously presented.

OBJECTIVE AND METHODS

The present systematic review analyzed the role of PPI deficit as a possible transdiagnostic process for four main groups of neuropsychiatric disorders: (1) trauma-, stress-, and anxiety-related disorders (2) mood-related disorders, (3) neurocognitive disorders, and (4) other disorders such as obsessive-compulsive, tic-related, and substance use disorders. We used Web of Science, PubMed and PsycInfo databases to search for experimental case-control articles that were analyzed both qualitatively and based on their potential risk of bias. A total of 64 studies were included in this systematic review. Protocol was submitted prospectively to PROSPERO 04/30/2022 (CRD42022322031).

RESULTS AND CONCLUSION

The results showed a general PPI deficit in the diagnostic groups mentioned, with associated deficits in the dopaminergic neurotransmission system, several areas implied such as the medial prefrontal cortex or the amygdala, and related variables such as cognitive deficits and anxiety symptoms. It can be concluded that the PPI deficit appears across most of the neuropsychiatric disorders examined, and it could be considered as a relevant measure in translational research for the early detection of such disorders.

Does prospective payment influence quality of care? A systematic review of the literature

In the light of rising health expenditures, the cost-efficient provision of high-quality inpatient care is on the agenda of policy-makers worldwide. In the last decades, prospective payment systems (PPS) for inpatient care were used as an instrument to contain costs and increase transparency of provided services. It is well documented in the literature that prospective payment has an impact on structure and processes of inpatient care. However, less is known about its effect on key outcome indicators of quality of care. In this systematic review, we synthesize evidence from studies investigating how financial incentives induced by PPS affect indicators of outcome quality domains of care, i.e. health status and user evaluation outcomes. We conduct a review of evidence published in English, German, French, Portuguese and Spanish language produced since 1983 and synthesize results of the studies narratively by comparing direction of effects and statistical significance of different PPS interventions. We included 64 studies, where 10 are of high, 18 of moderate and 36 of low quality. The most commonly observed PPS intervention is the introduction of per-case payment with prospectively set reimbursement rates. Abstracting evidence on mortality, readmission, complications, discharge disposition and discharge destination, we find the evidence to be inconclusive. Thus, claims that PPS either cause great harm or significantly improve the quality of care are not supported by our findings. Further, the results suggest that reductions of length of stay and shifting treatment to post-acute care facilities may occur in the course of PPS implementations. Accordingly, decision-makers should avoid low capacity in this area.

The current state of international research on the effectiveness of school nurses in promoting the health of children and adolescents: An overview of reviews

OBJECTIVE

School nurses are engaging worldwide to promote and protect children's health. Many researchers who examined the effectiveness of the school nurse criticized the inadequate methodology employed in many of the studies. We therefore carried out an evaluation on the effectiveness of school nurses based on a rigorous methodological approach.

METHODS

In this overview of reviews we performed an electronic databank search and global research results on the effectiveness of school nurses were sought. We identified 1,494 records through database search. Abstracts and full texts were screened and summarized using the dual control principle. We summarized the aspects of quality criteria as well as the significance of the effectiveness of the school nurse. In the first step, k = 16 systematic reviews were summarized and evaluated following the AMSTAR-2 guidelines. In a second step, j = 357 primary studies included in these k = 16 reviews were summarized and assessed following the GRADE guidelines.

RESULTS

Research results on the effectiveness of school nurses show that school nurses play a key role in improving the health of children with asthma (j = 6) and diabetes (j = 2), results on combating obesity are less certain (j = 6). The quality of identified reviews is mostly very low with only six studies of medium quality, of which one identified as a meta-analysis. A total of j = 289 primary studies were identified. Approximately 25% (j = 74) of identified primary studies were either randomized controlled trials (RCT) or observational studies, of which roughly 20% (j = 16) had a low risk of bias. Studies with physiological variables such as blood glucose or asthma labeling led to higher quality results.

CONCLUSION

This paper represents an initial contribution and recommends further evaluation of the effectiveness of school nurses, particularly in the areas of mental health or children from low socioeconomic backgrounds. The general lack of quality standards in school nursing research should be integrated into the scientific discourse of school nursing researchers to provide robust evidence for policy planners and researchers.

The impact of economic sanctions on health and health systems in low-income and middle-income countries: a systematic review and narrative synthesis

INTRODUCTION

Economic sanctions restrict customary commercial and financial ties between states to induce change in political constitution or conduct of the targeted country. Although the stated goals of sanctions often include humanitarian objectives, prospective procedures for health risk assessment are not regularly incorporated in their implementation. Moreover, past experience suggests that the burden of economic isolation may fall on the civilian population. We present key findings from a WHO-sponsored evidence review on the impact of economic sanctions on health and health systems in low-income and middle-income countries, aiming at comprehensive coverage and explicit consideration of issues of causality and mechanisms.

METHODS

Broad searches of PubMed and Google Scholar (1970-2021) were designed to retrieve published and grey English-language literature expected to cut across disciplines, terminology and research methods. Studies providing an impact estimate were rated by a structured assessment based on ROBINS-I risk of bias domains, synthesised via vote counting and contextualised into the broader literature through a thematic synthesis.

RESULTS

Included studies (185) were mostly peer-reviewed, mostly single-country, largely coming from medicine and public health, and chiefly concerned with three important target countries-Iraq, Haiti and Iran. Among studies providing impact estimates (31), most raised multiple risk-of-bias concerns. Excluding those with data integrity issues, a significant proportion (21/27) reported consistently adverse effects of sanctions across examined outcomes, with no apparent association to assessed quality, focus on early episodes or publication period. The thematic synthesis highlights the complexity of sanctions, their multidimensionality and the possible mechanisms of impact.

CONCLUSION

Future research should draw on qualitative knowledge to collect domain-relevant data, combining it with better estimation techniques and study design. However, only the adoption of a risk assessment framework based on prospective data collection and monitoring can certify claims that civilians are adequately protected.

A Metasynthesis and Meta-analysis of the Impact and Diagnostic Safety of COVID-19 Symptom Agnostic Rapid Testing in Low- and Middle-Income Countries: Protocol for a Systematic Review

BACKGROUND

Amid all public health measures to contain COVID-19, the most challenging has been how to break the transmission chain. This has been even more challenging in low- and middle-income countries (LMICs). A public health emergency warrants a public health perspective, which comes down to prevention. Rapid mass testing has been advocated throughout the pandemic as a way to promptly deal with asymptomatic infections, but its usefulness in LMICs is yet to be fully understood.

OBJECTIVE

The study objectives of this paper are to (1) investigate the impact of the different rapid mass testing options for SARS-CoV-2 that have been delivered at point of care in LMICs and (2) evaluate the diagnostic safety (accuracy) of rapid mass testing for SARS-CoV-2 in LMICs.

METHODS

This review will systematically search records in PubMed, EBSCOhost, Cochrane library, Global Index Medicus COVID-19 Register, and Scopus. Records will be managed using Mendeley reference manager and SWIFT-Review. Risk of bias for randomized controlled trials will be assessed using the RoB 2 assessment tool, while nonrandomized interventions will be assessed using the tool developed by the Evidence Project. A narrative approach will be used to synthesize data under the first objective, and either a meta-analysis or synthesis without meta-analysis for the second objective. Tables, figures, and textual descriptions will be used to present findings. The overall body of evidence for the first objective will be assessed using the Grading of Recommendations Assessment, Development, and Evaluation-Confidence in the Evidence from Reviews of Qualitative research (GRADE-CERQual) approach, and for the second objective using GRADE.

RESULTS

The screening of records has been finalized. We hope to finalize the synthesis by the end of February 2023 and to prepare the manuscript for publication by April 2023. The study will be reported in accordance with standard guidelines for the reporting of systematic reviews. Review results will be disseminated through conferences and their peer-reviewed publication in a relevant journal.

CONCLUSIONS

This review highlights the role of a preventive approach in infection control using rapid mass testing. It also flags the overriding need to involve users and providers in the evaluation of such tests in the settings for which they are intended. This will be the first review to the best of our knowledge to generate both qualitative and quantitative evidence regarding rapid mass testing specific to LMICs.

TRIAL REGISTRATION

PROSPERO CRD42022283776; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=283776.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/41132.

Animal-source foods as a suitable complementary food for improved physical growth in 6 to 24-month-old children in low- and middle-income countries: a systematic review and meta-analysis of randomised controlled trials

Although animal-source foods are suitable complementary food for child growth in low- and middle-income countries (LMICs), their efficacy is still under discussion. This systematic review and meta-analysis was done to investigate the suitability of animal-source foods intake on child physical growth in LMICs. A systematic literature search was done using electronic databases and scanning the reference list of included studies, previous meta-analysis and systematic reviews. Paper selection was based on the PICO (ST) criteria. Papers were selected if based on 6 to 24-month-old children, if they were randomised controlled trials evaluating the effect of complementary animal-based food supplementation of any natural origin, if reporting at least a measure of body size and published after 2000. The PRISMA guidelines for reporting systematic review was followed in the paper selection. Fourteen papers were included in the systematic review and eight were considered for the meta-analysis. Animal-based food supplementation resulted in a higher length-for-age LAZ and weight-for-age (WAZ) Z-scores compared with the control group with random effect size of 0·15 (95 % CI 0·02, 0·27) and 0·20 (95 % CI 0·03, 0·36), respectively. Results were confirmed after influence analyses, and publication bias resulted as negligible. An increased effect on LAZ and WAZ was observed when the food supplementation was based on egg with effect size of 0·31 (95 % CI = -0·03, 0·64) and 0·36 (95 % CI = -0·03, 0·75), respectively. Animal-source foods are a suitable complementary food to improve growth in 6 to 24-month-old children in LMICs.

Governance of Intersectoral Collaborations for Population Health and to Reduce Health Inequalities in High-Income Countries: A Complexity-Informed Systematic Review

BACKGROUND

A 'Health in All Policies' (HiAP) approach has been widely advocated as a way to involve multiple government sectors in addressing health inequalities, but implementation attempts have not always produced the expected results. Explaining how HiAP-style collaborations have been governed may offer insights into how to improve population health and reduce health inequalities.

METHODS

Theoretically focused systematic review. Synthesis of evidence from evaluative studies into a causal logic model.

RESULTS

Thirty-one publications based on 40 case studies from nine high-income countries were included. Intersectoral collaborations for population health and equity were multi-component and multi-dimensional with collaborative activity spanning policy, strategy, service design and service delivery. Governance of intersectoral collaboration included structural and relational components. Both internal and external legitimacy and credibility delivered collaborative power, which in turn enabled intersectoral collaboration. Internal legitimacy was driven by multiple structural elements and processes. Many of these were instrumental in developing (often-fragile) relational trust. Internal credibility was supported by multi-level collaborations that were adequately resourced and shared power. External legitimacy and credibility was created through meaningful community engagement, leadership that championed collaborations and the identification of 'win-win' strategies. External factors such as economic shocks and short political cycles reduced collaborative power.

CONCLUSION

This novel review, using systems thinking and causal loop representations, offers insights into how collaborations can generate internal and external legitimacy and credibility. This offers promise for future collaborative activity for population health and equity; it presents a clearer picture of what structural and relational components and dynamics collaborative partners can focus on when planning and implementing HiAP initiatives. The limits of the literature base, however, does not make it possible to identify if or how this might deliver improved population health or health equity.

A Tailored Approach: A model for literature searching in complex systematic reviews

Our previous work identified that nine leading guidance documents for seven different types of systematic review advocated the same process of literature searching. We defined and illustrated this process and we named it ‘the Conventional Approach’. The Conventional Approach appears to meet the needs of researchers undertaking literature searches for systematic reviews of clinical interventions. In this article, we report a new and alternate process model of literature searching called ‘A Tailored Approach’. A Tailored Approach is indicated as a search process for complex reviews which do not focus on the evaluation of clinical interventions. The aims of this article are to (1) explain the rationale for, and the theories behind, the design of A Tailored Approach; (2) report the current conceptual illustration of A Tailored Approach and to describe a user’s interaction with the process model; and (3) situate the elements novel to A Tailored Approach (when compared with the Conventional Approach) in the relevant literature. A Tailored Approach suggests investing time at the start of a review, to develop the information needs from the research objectives, and to tailor the search approach to studies or data. Tailored Approaches should be led by the information specialist (librarian) but developed by the research team. The aim is not necessarily to focus on comprehensive retrieval. Further research is indicated to evaluate the use of supplementary search methods, methods of team-working to define search approaches, and to evaluate the use of conceptual models of information retrieval for testing and evaluation.

Shifting the dynamics: implementation of locally driven, mixed-methods modelling to inform schistosomiasis control and elimination activities

INTRODUCTION

The integration of more diverse perspectives into the development of evidence for decision-making has been elusive, despite years of rhetoric to the contrary. This has led to cycles of population-based health interventions which have not delivered the promised results. The WHO most recently set a target for schistosomiasis elimination by 2030 and called for cross-cutting approaches to be driven by endemic countries themselves. The extent to which elimination is feasible within the time frame has been a subject of debate.

METHODS

Systems maps were developed through participatory modelling activities with individuals working on schistosomiasis control and elimination activities from the village through national levels in Uganda. These maps were first synthesised, then used to frame the form and content of subsequent mathematical modelling activities, and finally explicitly informed model parameter specifications for simulations, using the open-source SCHISTOX model, driven by the participants.

RESULTS

Based on the outputs of the participatory modelling, the simulation activities centred around reductions in water contact. The results of the simulations showed that mass drug administration, at either the current or target levels of coverage, combined with water contact reduction activities, achieved morbidity control in high prevalence Schistosoma mansoni settings, while both morbidity control and elimination were achieved in high prevalence S. haematobium settings within the 10-year time period.

CONCLUSION

The combination of participatory systems mapping and individual-based modelling was a rich strategy which explicitly integrated the perspectives of national and subnational policymakers and practitioners into the development of evidence. This strategy can serve as a method by which individuals who have not been traditionally included in modelling activities, and do not hold positions or work in traditional centres of power, may be heard and truly integrated into the development of evidence for decision-making in global health.

The efficacy of web or mobile-based interventions to alleviate emotional symptoms in people with advanced cancer: a systematic review and meta-analysis

PURPOSE

This review aims to critically evaluate the efficacy of web or mobile-based (WMB) interventions impacting emotional symptoms in patients with advanced cancer.

METHOD

Articles published from 1991 to 2019 were identified using PubMed, PsycINFO, CINAHL, and Scopus. Only interventions involving adults with advanced cancer using a WMB intervention to manage emotional symptoms were included. Risk of bias was assessed using ROBINS-I and ROB2 tools. Studies that reported mean symptom scores were pooled using a random-effects model, and standardized mean difference (SMD) and 95% CIs were calculated.

RESULTS

Twenty-three of the 1177 screened studies met the inclusion criteria, and a total sample of 2558 patients were included. The sample was 57% female, and 33% had advanced cancer with mean age of 57.15 years. Thirteen studies evaluated anxiety, nineteen evaluated depression, and eleven evaluated distress. Intervention components included general information, tracking, communication, multimedia choice, interactive online activities, tailoring/feedback, symptom management support content, and self-monitoring. Overall pooled results showed that WMB interventions' effects on decreasing anxiety (SMD - 0.20, - 0.45 to 0.05, I² = 72%), depression (SMD - 0.10, - 0.30 to 0.11, I² = 73%), and distress (SMD - 0.20, - 0.47 to 0.06, I² = 60%) were not significant for randomized controlled trials (RCTs). In contrast, WMB interventions significantly decreased symptoms of anxiety (p = .002) in a sub-group analysis of non-RCTs.

CONCLUSION

This meta-analysis demonstrated that WMB interventions were not efficacious in alleviating emotional symptoms in adults with advanced cancer. Considering the diversity of interventions, the efficacy of WMB interventions and its impacts on emotional symptoms should be further explored.

Dynamics of the complex food environment underlying dietary intake in low-income groups: a systems map of associations extracted from a systematic umbrella literature review

Background

Inequalities in obesity pertain in part to differences in dietary intake in different socioeconomic groups. Examining the economic, social, physical and political food environment of low-income groups as a complex adaptive system – i.e. a system of multiple, interconnected factors exerting non-linear influence on an outcome, can enhance the development and assessment of effective policies and interventions by honouring the complexity of lived reality. We aimed to develop and apply novel causal loop diagramming methods in order to construct an evidence-based map of the underlying system of environmental factors that drives dietary intake in low-income groups.

Methods

A systematic umbrella review was conducted on literature examining determinants of dietary intake and food environments in low-income youths and adults in high/upper-middle income countries. Information on the determinants and associations between determinants was extracted from reviews of quantitative and qualitative studies. Determinants were organised using the Determinants of Nutrition and Eating (DONE) framework. Associations were synthesised into causal loop diagrams that were subsequently used to interpret the dynamics underlying the food environment and dietary intake. The map was reviewed by an expert panel and systems-based analysis identified the system paradigm, structure, feedback loops and goals.

Results

Findings from forty-three reviews and expert consensus were synthesised in an evidence-based map of the complex adaptive system underlying the food environment influencing dietary intake in low-income groups. The system was interpreted as operating within a supply-and-demand, economic paradigm. Five sub-systems (‘geographical accessibility’, ‘household finances’, ‘household resources’, ‘individual influences’, ‘social and cultural influences’) were presented as causal loop diagrams comprising 60 variables, conveying goals which undermine healthy dietary intake.

Conclusions

Our findings reveal how poor dietary intake in low-income groups can be presented as an emergent property of a complex adaptive system that sustains a food environment that increases the accessibility, availability, affordability and acceptability of unhealthy foods. In order to reshape system dynamics driving unhealthy food environments, simultaneous, diverse and innovative strategies are needed to facilitate longer-term management of household finances and socially-oriented practices around healthy food production, supply and intake. Ultimately, such strategies must be supported by a system paradigm which prioritises health.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12966-021-01164-1.

Analysing synthesis of evidence in a systematic review in health professions education: observations on struggling beyond Kirkpatrick

Background: Systematic reviews in health professions education may well under-report struggles to synthesize disparate evidence that defies standard quantitative approaches. This paper reports further process analysis in a previously reported systematic review about mobile devices on clinical placements.Objective: For a troublesome systematic review: (1) Analyse further the distribution and reliability of classifying the evidence to Maxwell quality dimensions (beyond 'Does it work?') and their overlap with Kirkpatrick K-levels. (2) Analyse how the abstracts represented those dimensions of the evidence-base. (3) Reflect on difficulties in synthesis and merits of Maxwell dimensions.Design: Following integrative synthesis of 45 K2-K4 primary studies (by combined content-thematic analysis in the pragmatism paradigm): (1) Hierarchical cluster analysis explored overlap between Maxwell dimensions and K-levels. Independent and consensus-coding to Maxwell dimensions compared (using: percentages; kappa; McNemar hypothesis-testing) pre- vs post-discussion and (2) article abstract vs main body. (3) Narrative summary captured process difficulties and merits.Results: (1) The largest cluster (five-cluster dendrogram) was acceptability-accessibility-K1-appropriateness-K3, with K1 and K4 widely separated. For article main bodies, independent coding agreed most for appropriateness (good; adjusted kappa = 0.78). Evidence increased significantly pre-post-discussion about acceptability (p = 0.008; 31/45→39/45), accessibility, and equity-ethics-professionalism. (2) Abstracts suggested efficiency significantly less than main bodies evidenced: 31.1% vs 44.4%, p = 0.031. 3) Challenges and merits emerged for before, during, and after the review.Conclusions: There should be more systematic reporting of process analysis about difficulties synthesizing suboptimal evidence-bases. In this example, Maxwell dimensions were a useful framework beyond K-levels for classifying and synthesizing the evidence-base.

Based on Principles and Insights of COVID-19 Epidemiology, Genome Sequencing, and Pathogenesis: Retrospective Analysis of Sinigrin and ProlixinRX (Fluphenazine) Provides Off-Label Drug Candidates

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is the causative pathogen of pandemic coronavirus disease 2019 (COVID-19). So far, no approved therapy has been developed to halt the spread of the pathogen, and unfortunately, the strategies for developing a new therapy will require a long time and very extensive resources. Therefore, drug repurposing has emerged as an ideal strategy toward a smart, versatile, quick way to confine the lethal disease. In this endeavor, natural products have been an untapped source for new drugs. This review represents the confederated experience of multidisciplinary researchers of 99 articles using several databases: Google Scholar, Science Direct, MEDLINE, Web of Science, Scopus, and PubMed. To establish the hypothesis, a Bayesian perspective of a systematic review was used to outline evidence synthesis. Our docking documentation of 69 compounds and future research agenda assumptions were directed toward finding an effective and economic anti-COVID-19 treatment from natural products. Glucosinolate, flavones, and sulfated nitrogenous compounds demonstrate direct anti-SARS-CoV-2 activity through inhibition protease enzymes and may be considered potential candidates against coronavirus. These findings could be a starting point to initiate an integrative study that may encompass interested scientists and research institutes to test the hypothesis in vitro, in vivo, and in clinics after satisfying all ethical requirements.

Search and Selection Procedures of Literature Reviews in Behavior Analysis

Literature reviews allow professionals to identify effective interventions and assess developments in research and practice. As in other forms of scientific inquiry, the transparency of literature searches enhances the credibility of findings, particularly in regards to intervention research. The current review evaluated the characteristics of search methods employed in literature reviews appearing in publications concerning behavior analysis (n = 28) from 1997 to 2017. Specific aims included determining the frequency of narrative, systematic, and meta-analytic reviews over time; examining the publication of reviews in specific journals; and evaluating author reports of literature search and selection procedures. Narrative reviews (51.30%; n = 630) represented the majority of the total sample (n = 1,228), followed by systematic (31.51%; n = 387) and meta-analytic (17.18%; n = 211) reviews. In contrast to trends in related fields (e.g., special education), narrative reviews continued to represent a large portion of published reviews each year. The evaluated reviews exhibited multiple strengths; nonetheless, issues involving the reporting and execution of searches may limit the validity and replicability of literature reviews. A discussion of implications for research follows an overview of findings.

Application of the intervention Complexity Assessment Tool for Systematic Reviews within a Cochrane review: an illustrative case study

Background: The intervention Complexity Assessment Tool for Systematic Reviews (iCAT_SR) has been developed to facilitate detailed assessments of intervention complexity in systematic reviews. Worked examples of the tool’s application are needed to promote its use and refinement. The aim of this case study was to apply the iCAT_SR to a subset of 20 studies included in a Cochrane review of interventions aimed at improving appropriate polypharmacy in older people.

Methods: Interventions were assessed independently by two authors using the six core iCAT_SR dimensions: (1) ‘Target organisational levels/categories’; (2) ‘Target behaviour/actions’; (3) ‘Active intervention components’; (4) ‘Degree of tailoring’; (5) ‘Level of skill required by intervention deliverers’; (6) ‘Level of skill required by intervention recipients’. Attempts were made to apply four optional dimensions: ‘Interaction between intervention components’; ‘Context/setting’; ‘Recipient/provider factors’; ‘Nature of causal pathway’. Inter-rater reliability was assessed using Cohen’s Kappa coefficient. Disagreements were resolved by consensus discussion. The findings are presented narratively.

Results: Assessments involving the core iCAT_SR dimensions showed limited consistency in intervention complexity across included studies, even when categorised according to clinical setting. Interventions were delivered across various organisational levels and categories (i.e. healthcare professionals and patients) and typically comprised multiple components. Intermediate skill levels were required by those delivering and receiving the interventions across all studies. A lack of detail in study reports precluded application of the iCAT_SR’s optional dimensions. The inter-rater reliability was substantial (Cohen's Kappa = 0.75)

Conclusions: This study describes the application of the iCAT_SR to studies included in a Cochrane systematic review. Future intervention studies need to ensure more detailed reporting of interventions, context and the causal pathways underlying intervention effects to allow a more holistic understanding of intervention complexity and facilitate replication in other settings. The experience gained has helped to refine the original guidance document relating to the application of iCAT_SR.

Reducing loneliness among migrant and ethnic minority people: a participatory evidence synthesis

Background

To date, there has been little research into the causes of, and solutions to, loneliness among migrant and ethnic minority people.

Objectives

The objectives were to synthesise available evidence and produce new insights relating to initiatives that aim to address loneliness among these populations, plus the logic, functioning and effects of such initiatives.

Data sources

Electronic database searches (MEDLINE, Applied Social Sciences Index and Abstracts and Social Science Citation Index via Web of Science – no date restrictions were applied), grey literature searches, and citation and reference searching were conducted. Data were generated via nine workshops with three consultation panels involving 34 public contributors, and one practitioner workshop involving 50 participants.

Review methods

Guided by ‘systems thinking’, a theory-driven synthesis was combined with an effectiveness review to integrate evidence on the nature and causes of loneliness, interventional types and programme theory, and intervention implementation and effectiveness.

Results

The theory review indicated that common conceptualisations of ‘loneliness’ can be usefully extended to recognise four proximate determinants when focusing on migrant and ethnic minority populations: positive social ties and interactions, negative social ties and interactions, self-worth, and appraisal of existing ties. A total of 170 interventions were included. A typology of eight interventions was developed. Detailed logic models were developed for three common types of intervention: befriending, shared-identity social support groups and intercultural encounters. The models for the first two types were generally well supported by empirical data; the third was more tentative. Evaluation of intervention processes and outcomes was limited by study content and quality. Evidence from 19 qualitative and six quantitative studies suggested that social support groups have a positive impact on dimensions of loneliness for participants. Evidence from nine qualitative and three quantitative studies suggested that befriending can have positive impacts on loneliness. However, inconsistent achievements of the befriending model meant that some initiatives were ineffective. Few studies on intercultural encounters reported relevant outcomes, although four provided some qualitative evidence and three provided quantitative evidence of improvement. Looking across intervention types, evidence suggests that initiatives targeting the proximate determinants – particularly boosting self-worth – are more effective than those that do not. No evidence was available on the long-term effects of any initiatives. UK intervention (n = 41) and non-intervention (n = 65) studies, together with consultation panel workshop data, contributed to a narrative synthesis of system processes. Interlocking factors operating at individual, family, community, organisational and wider societal levels increase risk of loneliness, and undermine access to, and the impact of, interventions. Racism operates in various ways throughout the system to increase risk of loneliness.

Limitations

There was a lack of high-quality quantitative studies, and there were no studies with longer-term follow-up. UK evidence was very limited. Studies addressing upstream determinants operating at the community and societal levels did not link through to individual outcome measures. Some elements of the search approach may mean that relevant literature was overlooked.

Conclusions

Theory regarding the causes of loneliness, and functioning of interventions, among migrant and ethnic minority populations was usefully developed. Evidence of positive impact on loneliness was strongest for shared-identity social support groups. Quantitative evidence was inadequate. The UK evidence base was extremely limited.

Future work

UK research in this area is desperately needed. Co-production of interventional approaches with migrant and ethnic minority people and evaluation of existing community-based initiatives are priorities.

Study registration

This study is registered as PROSPERO CRD42017077378.

Funding

This project was funded by the National Institute for Health Research Public Health Research programme and will be published in full in Public Health Research; Vol. 8, No. 10. See the NIHR Journals Library website for further project information.

"It has no meaning to me." How do researchers understand the effectiveness of literature searches? A qualitative analysis and preliminary typology of understandings

This study aimed to address the question: what does "effectiveness" mean to researchers in the context of literature searching for systematic reviews? We conducted a thematic analysis of responses to an e-mail survey. Eighty-nine study authors, whose studies met inclusion in a recent review (2018), were contacted via e-mail and asked three questions; one directly asking the question: in literature searching, what does effective (or effectiveness in) literature searching mean to you? Thirty-eight (46%) responses were received from diverse professional groups, including: literature searchers, systematic reviewers, clinicians and researchers. A shared understanding of what effectiveness means was not identified. Instead, five themes were developed from data: (a) effectiveness is described as a metric; (b) effectiveness is a balance between metrics; (c) effectiveness can be categorized by search purpose; (d) effectiveness is an outcome; and, (e) effectiveness is an experimental concept. We propose that these themes constitute a preliminary typology of understandings. No single definition of effectiveness was identified. The proposed typology suggests that different researchers have differing understandings of effectiveness. This could lead to uncertainty as to the aim and the purpose of literature searches and confusion about the outcomes. The typology offers a potential route for further exploration.

Quantitizing findings from qualitative studies for integration in mixed methods reviewing

In mixed methods reviewing, data from quantitative and qualitative studies are combined at the review level. One possible way to combine findings of quantitative and qualitative studies is to quantitize qualitative findings prior to their incorporation in a quantitative review. There are only a few examples of the quantification of qualitative findings within this context. This study adds to current research on mixed methods review methodology by reporting the pilot implementation of a new four‐step quantitizing approach. We report how we extract and quantitize the strength of relationships found in qualitative studies by assigning correlations to vague quantifiers in text fragments. This article describes (a) how the analysis is prepared; (b) how vague quantifiers in text fragments are organized and transformed to numerical values; (c) how qualitative studies as a whole are assigned effect sizes; and (d) how the overall mean effects size and variance can be calculated. The pilot implementation shows how findings from 26 primary qualitative studies are transformed into mean effect sizes and corresponding variances.

The long-term health and wellbeing impacts of Healthy New Towns (HNTs): protocol for a baseline and feasibility study of HNT demonstrator sites in England

Background

Increasing levels of non-communicable diseases (NCDs), mental health problems, high rates of unhealthy behaviours and health inequalities remain major public health challenges worldwide. In the context of increasing urbanisation, there is an urgent need to understand how evidence that living environments shape health, wellbeing and behaviour can be used to design and deliver healthy environments in local urban settings. The Healthy New Town (HNT) programme implemented in England from 2015 consists of ten major housing developments that aim to improve population health through healthy design principles, new models for integrating health and social care and the creation of strong and connected communities. The programme provides a natural experiment in which to investigate the effects on health, wellbeing and inequalities of large-scale interventions targeting the wider social determinants of health.

Methods

The research described in this protocol aims to examine the feasibility of a larger study to assess the longer-term health impacts of HNTs, by addressing two research questions: (1) what are the similarities and differences in the HNT programme developments, processes, contexts and expected impacts and outcomes across HNT sites? and (2) how feasible is the use of data from routine sources and existing HNT evaluations and as the baseline for a definitive study to assess impact on health, wellbeing, behavioural and economic outcomes and programme processes? The research will consist of (a) participatory systems mapping with stakeholders to produce a theoretical framework for a longer-term study on the HNT programme, (b) synthesis of existing qualitative data from local HNT evaluations to understand local processes and intervention mechanisms, (c) scoping local and routinely available data to establish a baseline and feasibility for a longer-term study of health and economic outcomes, and (d) building relationships and recruiting HNT sites into the proposed research.

Discussion

The proposed research will produce a theoretical framework and assess the feasibility of a definitive study of outcomes of the HNT programme. This research is necessary to understand how longer-term health, wellbeing, behavioural and economic outcomes can be measured, and to inform a definitive study to generate evidence on the effectiveness of the HNT programme.

Self-Disclosure of HIV Status Among Youth Living with HIV: A Global Systematic Review

Self-disclosure of HIV serostatus by youth has been considered an essential component of HIV prevention and medication adherence efforts. Therefore, a comprehensive investigation of disclosure goals, processes, and outcomes is warranted. We conducted a global systematic review in accordance with the 2015 Preferred Items for Reporting Systematic Reviews and Meta-Analysis tool to assess HIV self-disclosure among youth ages 13-24. We identified 5881 articles during our initial search. After screening titles and abstracts and examining articles in greater detail, 33 studies (35 articles) were included in the synthesis. The disclosure process model was used to highlight antecedent goals to self-disclosure including common avoidance goals such as fear of rejection and isolation. While disclosure was associated with negative and positive emotional outcomes and improved medication adherence, there remain concerns regarding the impact of self-disclosure on sexual behaviors. Implications for practice and future directions for research are presented.PROSPERO registration number: CRD42018097250.

Relationships Between Subjective Cognitive Impairment and Personality Traits: A Systematic Review

This systematic review examined the relationships between personality traits and subjective cognitive impairment (SCI) in older adults without dementia. A comprehensive literature search conducted according to PRISMA guidelines identified empirical investigations of SCI and at least one of the big five personality traits among adults age 60 or older. All articles were critically appraised using the weight of evidence framework and findings were compared, contrasted, and synthesized across studies. Sixteen of the 797 studies initially identified met eligibility criteria. A higher level of SCI was associated with higher neuroticism in 88% of the studies reviewed. In addition, a consistent negative association was identified between conscientiousness and SCI (57% of studies). No consistent relationships between openness, extraversion, or agreeableness and SCI were identified. Overall, this review supports the oft-cited association between higher neuroticism and greater self-reports of cognitive problems; however, the complexity of the relationship between SCI and personality is not yet fully understood. Future research should examine the extent to which different personality traits predispose individuals to report symptoms versus those traits that are associated with increased sensitivity to early indicators of pathological change. [Journal of Gerontological Nursing, 45(2), 27-34.].

The effectiveness of nudges in improving the self-management of patients with chronic diseases: A systematic literature review

In this systematic literature review, we identify evidence on the effectiveness of nudges in improving the self-management of adults with chronic diseases and derive policy recommendations. We included empirical studies of any design published up to April 12^th, 2018. We synthesized the results of the studies narratively by comparing statistical significance and direction of different nudge types' effects on primary study outcomes. Lastly, we categorized the nudges according to their degree of manipulation and transparency. We identified 26 studies, where 13 were of high or moderate quality. The most commonly tested nudges were reminders, planning prompts, small financial incentives, and feedback. Overall, 8 of 9 studies with a high or moderate quality ranking, focused on self-management outcomes, i.e., physical activity, attendance, self-monitoring, and medication adherence, found that nudges had significant positive effects. However, only 1 of 4 studies of high or moderate quality, analyzing disease control outcomes (e.g., glycemic control), found that nudges had a significant positive effect for one intervention arm. In summary, this review demonstrates that nudges can improve chronic disease self-management, but there is hardly any evidence to date that these interventions lead to improved disease control. Reminders, feedback, and planning prompts appear to improve chronic disease self-management most consistently and are among the least controversial types of nudges. Accordingly, they can generally be recommended to policymakers.

Qualitative evidence syntheses: Assessing the relative contributions of multi-context and single-context reviews

AIMS

To examine the strengths and weaknesses of multi-context (international) qualitative evidence syntheses in comparison with single-context (typically single-country) reviews. We compare a multi-country synthesis with single-context syntheses on facility-based delivery in Nigeria and Kenya.

DESIGN

Discussion paper.

BACKGROUND

Qualitative evidence increasingly contributes to decision-making. International organizations commission multi-context reviews of qualitative evidence to gain a comprehensive picture of similarities and differences across comparable (e.g., low- and middle-income) countries. Such syntheses privilege breadth over contextual detail, risking inappropriate interpretation and application of review findings. Decision-makers value single-context syntheses that account for the contexts of their populations and health services. We explore how findings from multi- and single-context syntheses contribute against a conceptual framework (adequacy, coherence, methodological limitations and relevance) that underpins the GRADE Confidence in Evidence of Reviews of Qualitative Evidence approach.

DATA SOURCES

Included studies and findings from a multi-context qualitative evidence synthesis (2001-2013) and two single-context syntheses (Nigeria, 2006-2017; and Kenya, 2002-2016; subsequently updated and revised).

FINDINGS

Single-context reviews contribute cultural, ethnic and religious nuances and specific health system factors (e.g., use of a voucher system). Multi-context reviews contribute to universal health concerns and to generic health system concerns (e.g., access and availability).

IMPLICATIONS FOR NURSING

Nurse decision-makers require relevant, timely and context-sensitive evidence to inform clinical and managerial decision-making. This discussion paper informs future commissioning and use of multi- and single-context qualitative evidence syntheses.

CONCLUSION

Multi- and single-context syntheses fulfil complementary functions. Single-context syntheses add nuances not identifiable in the remit and timescales of a multi-context review. Impact This study offers a unique comparison between multi-context and single country (Nigeria and Kenya) qualitative syntheses exploring facility-based birth. Clear strengths and weaknesses were identified to inform commissioning and application of future syntheses. Characteristics can inform the commissioning of single- and multi-context nursing-oriented reviews across the world.

Theory-based strategies for teaching evidence-based practice to undergraduate health students: a systematic review

BACKGROUND

Undergraduate students across health professions are required to be capable users of evidence in their clinical practice after graduation. Gaining the essential knowledge and clinical behaviors for evidence-based practice can be enhanced by theory-based strategies. Limited evidence exists on the effect of underpinning undergraduate EBP curricula with a theoretical framework to support EBP competence. A systematic review was conducted to determine the effectiveness of EBP teaching strategies for undergraduate students, with specific focus on efficacy of theory-based strategies.

METHODS

This review critically appraised and synthesized evidence on the effectiveness of EBP theory-based teaching strategies specifically for undergraduate health students on long or short-term change in multiple outcomes, including but not limited to, EBP knowledge and attitudes. PubMed, CINAHL, Scopus, ProQuest Health, ERIC, The Campbell Collaboration, PsycINFO were searched for published studies and The New York Academy of Medicine, ProQuest Dissertations and Mednar were searched for unpublished studies. Two independent reviewers assessed studies using the Joanna Briggs Institute Meta-Analysis of Statistics Assessment and Review Instrument.

RESULTS

Twenty-eight studies reporting EBP teaching strategies were initially selected for review with methodological quality ranging from low to high. Studies varied in course duration, timing of delivery, population and course content. Only five included papers reported alignment with, and detail of, one or more theoretical frameworks. Theories reported included Social Cognitive Theory (one study), Roger's Diffusion of Innovation Theory (two studies) and Cognitive Apprenticeship Theory (one study). Cognitive Flexibility Theory and Cognitive Load Theory were discussed in two separate papers by the same authors. All but one study measured EBP knowledge. Mixed results were reported on EBP knowledge, attitudes and skills across the five studies.

CONCLUSIONS

EBP programs for undergraduate health students require consideration of multiple domains, including clinical behaviors, attitudes and cognitive learning processes; Interventions grounded in theory were found to have a small but positive effect on EBP attitudes. The most effective theory for developing and supporting EBP capability is not able to be determined by this review therefore additional rigorous research is required.

Barriers and facilitators to screening and treating malnutrition in older adults living in the community: a mixed-methods synthesis

BACKGROUND

Malnutrition (specifically undernutrition) in older, community-dwelling adults reduces well-being and predisposes to disease. Implementation of screen-and-treat policies could help to systematically detect and treat at-risk and malnourished patients. We aimed to identify barriers and facilitators to implementing malnutrition screen and treat policies in primary/community care, which barriers have been addressed and which facilitators have been successfully incorporated in existing interventions.

METHOD

A data-base search was conducted using MEDLINE, Embase, PsycINFO, DARE, CINAHL, Cochrane Central and Cochrane Database of Systematic Reviews from 2012 to June 2016 to identify relevant qualitative and quantitative literature from primary/community care. Studies were included if participants were older, community-dwelling adults (65+) or healthcare professionals who would screen and treat such patients. Barriers and facilitators were extracted and mapped onto intervention features to determine whether these had addressed barriers.

RESULTS

Of a total of 2182 studies identified, 21 were included (6 qualitative, 12 quantitative and 3 mixed; 14 studies targeting patients and 7 targeting healthcare professionals). Facilitators addressing a wide range of barriers were identified, yet few interventions addressed psychosocial barriers to screen-and-treat policies for patients, such as loneliness and reluctance to be screened, or healthcare professionals' reservations about prescribing oral nutritional supplements.

CONCLUSION

The studies reviewed identified several barriers and facilitators and addressed some of these in intervention design, although a prominent gap appeared to be psychosocial barriers. No single included study addressed all barriers or made use of all facilitators, although this appears to be possible. Interventions aiming to implement screen-and-treat approaches to malnutrition in primary care should consider barriers that both patients and healthcare professionals may face.

REVIEW REGISTRATIONS

PROSPERO: CRD42017071398 . The review protocol was registered retrospectively.

Title: Can changing the physical environment promote walking and cycling? A systematic review of what works and how

Environmental changes aimed at encouraging walking or cycling may promote activity and improve health, but evidence suggests small or inconsistent effects in practice. Understanding how an intervention works might help explain the effects observed and provide guidance about generalisability. We therefore aimed to review the literature on the effects of this type of intervention and to understand how and why these may or may not be effective. We searched eight electronic databases for existing systematic reviews and mined these for evaluative studies of physical environmental changes and assessed changes in walking, cycling or physical activity. We then searched for related sources including quantitative or qualitative studies, policy documents or reports. We extracted information on the evidence for effects ('estimation'), contexts and mechanisms ('explanation') and assessed credibility, and synthesised material narratively. We identified 13 evaluations of interventions specifically targeting walking and cycling and used 46 related sources. 70% (n = 9 evaluations) scored 3 or less on the credibility criteria for effectiveness. 6 reported significant positive effects, but higher quality evaluations were more likely to report positive effects. Only two studies provided rich evidence of mechanisms. We identified three common resources that interventions provide to promote walking and cycling: (i) improving accessibility and connectivity; (ii) improving traffic and personal safety; and (iii) improving the experience of walking and cycling. The most effective interventions appeared to target accessibility and safety in both supportive and unsupportive contexts. Although the evidence base was relatively limited, we were able to understand the role of context in the success of interventions. Researchers and policy makers should consider the context and mechanisms which might operate before evaluating and implementing interventions.

Interventions targeting working memory in 4-11 year olds within their everyday contexts: A systematic review

It has been suggested that diverse interventions applied within children's everyday contexts have the potential to improve working memory (WM) and produce transfer to real-world skills but little is known about the effectiveness of these approaches. This review aims to examine systematically the effectiveness of non-computerised interventions with 4-11 year olds to identify: (i) their effects on WM; (ii) whether benefits extend to near- and far-transfer measures; (iii) if gains are sustained over time; (iv) the active ingredients; and (v) the optimum dosage. Searches were conducted across 12 electronic databases using consistent keywords. Papers were screened by title and abstract (n = 6212) and judged against pre-defined eligibility criteria (n = 63). Eighteen papers were included in the review. They used a range of non-computerised WM intervention approaches that included: (i) adapting the environment to reduce WM loads; (ii) direct WM training with and without strategy instruction; and (iii) training skills which may indirectly impact on WM (physical activity, phonological awareness, fantastical play and inhibition). Both direct training on WM tasks and practicing certain skills that may impact indirectly on WM (physical activity, fantastical play and inhibition) produced improvements on WM tasks, with some benefits for near-transfer activities. The common ingredient across effective interventions was the executive-loaded nature of the trained task i.e., training on a task that taps into attentional and processing resources under executive control and not just the storage of information. Few studies reported dosage effects, measured far-transfer effects (n = 4), or tested the durability of gains over time (n = 4). The lack of a clear theoretical framework in many of the included studies resulted in ambiguous predictions about training and transfer effects, and inadequate use of outcome measures. Methodological issues also constrain the strength of the evidence, including: small samples sizes; an absence of blinding of participant and outcome assessors; and lack of active control groups. Further well-designed and controlled studies with clear theoretical underpinnings are required to expand and enhance the evidence base. The heterogeneity of the interventions and of the study designs (randomised and non-randomised) in the included papers limited the synthesis of evidence across studies. However, this diversity enabled the identification of key ingredients, notably the training of executive-loaded WM tasks, which can help inform novel approaches to WM intervention in everyday contexts.

Methodological evaluation of reviews that support recommendations from three consensus workshops in periodontology

OBJECTIVE

This study aimed to evaluate comprehensiveness and reproducibility of reviews that support consensus guidelines in periodontology.

METHODS

We included the reviews that support consensus guidelines from three workshops in periodontology, which were overseen by likely the two most important organisations in the field: the European Federation of Periodontology and the American Academy of Periodontology. We independently evaluated the comprehensiveness of literature searches by determining whether authors had searched reference lists, journals, registries and grey literature and whether the searches were limited to only one or a few languages. We evaluated whether review authors reported the eligibility criteria, the search strategies, and the list of included/excluded articles. We tested whether the search and selection of articles in one major database was reproducible.

RESULTS

Twenty-nine reviews were evaluated. Two (7%) reviews reported grey literature searches, and more than two-thirds of the reviews did not report hand-searching. Almost half of the reviews did not report whether there was language restriction for the literature searches. Two-thirds of the reviews reported the use of keywords only (without Boolean operators). One-fourth of the reviews reported the presence of a list of excluded articles after the full-text assessment. None of the reviews reported a detailed list of excluded articles after screening of titles/abstracts. None of the reviews reported enough information to allow reproduction of the findings of the PubMed search.

CONCLUSIONS

There is room to improve the reporting of the methodologies that are used in reviews that support periodontology consensus guidelines, although heterogeneity in reporting was found across all the reviews.

Supporting implementation of Cochrane methods in complex communication reviews: resources developed and lessons learned for editorial practice and policy

Every healthcare encounter involves some form of communication and there is growing recognition that effective health communication is central to the delivery of safe, high-quality healthcare. Conversely, poor communication has a range of adverse consequences for those receiving healthcare and the systems delivering care, including elevated patient safety risks. Increasing understanding and documentation of the key role that good communication plays in healthcare design and delivery has meant there is growing demand from policy-makers and other decision-makers for evidence on the effects of health communication interventions - that is, how best to communicate. While systematic reviews of such interventions are fundamental to building this evidence base, such interventions and reviews are often highly complex and pose considerable challenges for authors and editors. In this paper, we describe our experience as a Cochrane editorial group identifying common issues in reviews of communication interventions and developing resources to support authors to better meet these challenges. Our analysis found that issues typically fell into one or more of the following three stages of the review process: understanding and applying systematic review methods (e.g. selecting outcomes for analysis); reporting the review's methods (e.g. describing key decisions made in conducting the review); and interpreting the findings (e.g. incorporating quality of the evidence into findings of the review). We also found that common issues reflected both practical difficulties (such as the typically large size of reviews and disparate measures for outcomes) and conceptual challenges (for instance, the difficulties of identifying comparisons). While extensive advice for Cochrane systematic reviewers exists, this standardised advice does not cover all of the issues emerging for complex communication reviews. In response, we therefore developed a collection of resources, both general and targeted to specific methodological issues. Here, we describe the types of resources developed and the aims of these, the rationale for why we needed to fill specific gaps in existing advice, and reflect on the lessons for future editorial practice, policies and research in relation to the implementation of Cochrane review methods in the area of health communication.

Considerations of complexity in rating certainty of evidence in systematic reviews: a primer on using the GRADE approach in global health

Public health interventions and health technologies are commonly described as 'complex', as they involve multiple interacting components and outcomes, and their effects are largely influenced by contextual interactions and system-level processes. Systematic reviewers and guideline developers evaluating the effects of these complex interventions and technologies report difficulties in using existing methods and frameworks, such as the Grading of Recommendations Assessment, Development and Evaluation (GRADE). As part of a special series of papers on implications of complexity in the WHO guideline development, this paper serves as a primer on how to consider sources of complexity when using the GRADE approach to rate certainty of evidence. Relevant sources of complexity in systematic reviews, health technology assessments and guidelines of public health are outlined and mapped onto the reported difficulties in rating the estimates of the effect of these interventions. Recommendations on how to address these difficulties are further outlined, and the need for an integrated use of GRADE from the beginning of the review or guideline development is emphasised. The content of this paper is informed by the existing GRADE guidance, an ongoing research project on considering sources of complexity when applying the GRADE approach to rate certainty of evidence in systematic reviews and the review authors' own experiences with using GRADE.

Implications of a complexity perspective for systematic reviews and guideline development in health decision making

There is growing interest in the potential for complex systems perspectives in evaluation. This reflects a move away from interest in linear chains of cause-and-effect, towards considering health as an outcome of interlinked elements within a connected whole. Although systems-based approaches have a long history, their concrete implications for health decisions are still being assessed. Similarly, the implications of systems perspectives for the conduct of systematic reviews require further consideration. Such reviews underpin decisions about the implementation of effective interventions, and are a crucial part of the development of guidelines. Although they are tried and tested as a means of synthesising evidence on the effectiveness of interventions, their applicability to the synthesis of evidence about complex interventions and complex systems requires further investigation. This paper, one of a series of papers commissioned by the WHO, sets out the concrete methodological implications of a complexity perspective for the conduct of systematic reviews. It focuses on how review questions can be framed within a complexity perspective, and on the implications for the evidence that is reviewed. It proposes criteria which can be used to determine whether or not a complexity perspective will add value to a review or an evidence-based guideline, and describes how to operationalise key aspects of complexity as concrete research questions. Finally, it shows how these questions map onto specific types of evidence, with a focus on the role of qualitative and quantitative evidence, and other types of information.

Formulating questions to explore complex interventions within qualitative evidence synthesis

When making decisions about complex interventions, guideline development groups need to factor in the sociocultural acceptability of an intervention, as well as contextual factors that impact on the feasibility of that intervention. Qualitative evidence synthesis offers one method of exploring these issues. This paper considers the extent to which current methods of question formulation are meeting this challenge. It builds on a rapid review of 38 different frameworks for formulating questions. To be useful, a question framework should recognise context (as setting, environment or context); acknowledge the criticality of different stakeholder perspectives (differentiated from the target population); accommodate elements of time/timing and place; be sensitive to qualitative data (eg, eliciting themes or findings). None of the identified frameworks satisfied all four of these criteria. An innovative question framework, PerSPEcTiF, is proposed and retrospectively applied to a published WHO guideline for a complex intervention. Further testing and evaluation of the PerSPEcTiF framework is required.

Electronic Systems for Patients to Report and Manage Side Effects of Cancer Treatment: Systematic Review

BACKGROUND

There has been a dramatic increase in the development of electronic systems to support cancer patients to report and manage side effects of treatment from home. Systems vary in the features they offer to patients, which may affect how patients engage with them and how they improve patient-centered outcomes.

OBJECTIVE

This review aimed to (1) describe the features and functions of existing electronic symptom reporting systems (eg, symptom monitoring, tailored self-management advice), and (2) explore which features may be associated with patient engagement and patient-centered outcomes.

METHODS

The review was registered with the International Prospective Register of Systematic Reviews (PROSPERO) and followed guidelines from the Centre for Reviews and Dissemination (University of York, United Kingdom). Primary searches were undertaken of MEDLINE, Embase, PsycInfo, Web of Science, Cochrane Central Register of Controlled Trials, and the Health Technology Assessment databases. Secondary searches were undertaken by screening reference lists and citations. Two researchers applied broad inclusion criteria to identify and select relevant records. Data were extracted and summarized using Microsoft Excel. In order to meet the aims, the study selection, data extraction, and data synthesis comprised two stages: (1) identifying and characterizing available systems and (2) summarizing data on patient engagement and patient-centered outcomes.

RESULTS

We identified 77 publications relating to 41 distinct systems. In Stage 1, all publications were included (N=77). The features identified that supported clinicians and care were facility for health professionals to remotely access and monitor patient-reported data (24/41, 58%) and function to send alerts to health professionals for severe symptoms (17/41, 41%). Features that supported patients were facility for patients to monitor/review their symptom reports over time (eg, graphs) (19/41, 46%), general patient information about cancer treatment and side effects (17/41, 41%), tailored automated patient advice on symptom management (12/41, 29%), feature for patients to communicate with the health care team (6/41, 15%), and a forum for patients to communicate with one another (4/41, 10%). In Stage 2, only publications that included some data on patient engagement or patient-centered outcomes were included (N=29). A lack of consistency between studies in how engagement was defined, measured, or reported, and a wide range of methods chosen to evaluate systems meant that it was not possible to compare across studies or make conclusions on relationships with system features.

CONCLUSIONS

Electronic systems have the potential to help patients manage side effects of cancer treatment, with some evidence to suggest a positive effect on patient-centered outcomes. However, comparison across studies is difficult due to the wide range of assessment tools used. There is a need to develop guidelines for assessing and reporting engagement with systems, and a set of core outcomes for evaluation. We hope that this review will contribute to the field by introducing a taxonomy for characterizing system features.

TRIAL REGISTRATION

PROSPERO CRD42016035915; www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42016035915.

The Impact of Evidence-Based Medicine on Qualitative Metasynthesis: Benefits to be Harvested and Warnings to be Given

Qualitative metasynthesis, developed as an interpretative and inductive methodology, is increasingly influenced by standards from evidence-based medicine, established as a strategy to support policy decisions and guidelines. Currently, principles and procedures from the format developed for systematic reviews are often applied for review and synthesis of all kinds of evidence, including results from qualitative studies. In this article, I substantiate these claims, discussing benefits to be harvested and warnings to be given when qualitative metasynthesis approaches the evidence-based medicine methodology. Situating my exploration in the context of clinical practice, I contrast missions and values of these methodologies regarding review and synthesis of research literature, highlighting potential mismatches between ontology and epistemology, emphasizing challenges regarding sample, analysis, and transferability. Approving systematic and transparent strategies as generic for such purposes, I warn against the idea that methodology developed for evidence-based medicine is a universal gold standard for synthesis of research evidence.

Interventions integrating health and academic interventions to prevent substance use and violence: a systematic review and synthesis of process evaluations

BACKGROUND

Within increasingly constrained school timetables, interventions that integrate academic and health education to reduce substance use and violence may hold promise as a category of intervention that can positively affect both academic and health outcomes. There are no current systematic reviews exploring the effectiveness of such interventions or factors that affect their implementation.

METHODS

A total of 19 bibliographic databases and 32 websites were searched. References were also extracted from the reference lists of included studies, and experts and authors were contacted to identify relevant studies. We included reports with no restrictions on language or date. References were screened on title/abstract and those not thus excluded were screened on full report. Data extraction and appraisal followed the Critical Appraisal Skills Programme, Evidence for Policy and Practice Information and Co-ordinating Centre and Cochrane tools. Extracted process data were qualitatively meta-synthesised for common themes.

RESULTS

Seventy-eight thousand four hundred fifty-one unique references were identified, and 62 reports were included. A total of 16 reports (reporting on 15 studies of 12 interventions) evaluated process. Key facilitators of integrated academic and health curricula were supportive senior management and alignment of the intervention with school ethos; a positive teaching environment, including positive perceptions around the ability to be flexible in the adaptation and delivery of integrated academic and health curricula; positive pre-existing student and teacher attitudes towards intervention content; and parental support of interventions, largely through reinforcement of messaging at home. Important barriers were over-burdened teachers, with little time to learn and implement integrated curricula.

CONCLUSION

Several useful facilitating and inhibiting factors linked to the implementation of interventions that integrate academic and health education for reduced substance use and/or violence were identified, providing tentative but insightful evidence of context-specific issues that may impact intervention success. However, overall, there is still a considerable gap in our understanding of how to achieve the successful implementation of these interventions.

A review protocol on research partnerships: a Coordinated Multicenter Team approach

BACKGROUND

Research partnership approaches, in which researchers and stakeholders work together collaboratively on a research project, are an important component of research, knowledge translation, and implementation. Despite their growing use, a comprehensive understanding of the principles, strategies, outcomes, and impacts of different types of research partnerships is lacking. Generating high-quality evidence in this area is challenging due to the breadth and diversity of relevant literature. We established a Coordinated Multicenter Team approach to identify and synthesize the partnership literature and better understand the evidence base. This review protocol outlines an innovative approach to locating, reviewing, and synthesizing the literature on research partnerships.

METHODS

Five reviews pertaining to research partnerships are proposed. The Coordinated Multicenter Team developed a consensus-driven conceptual framework to guide the reviews. First, a review of reviews will comparatively describe and synthesize key domains (principles, strategies, outcomes, and impacts) for different research partnership approaches, within and beyond health (e.g., integrated knowledge translation, participatory action research). After identifying commonly used search terminology, three complementary scoping reviews will describe and synthesize these domains in the health research partnership literature. Finally, an umbrella review will amalgamate and reflect on the collective findings and identify research gaps and future directions. We will develop a collaborative review methodology, comprising search strategy efficiencies, terminology standardization, and the division of screening, extraction, and synthesis to optimize feasibility and literature capture. A series of synthesis and scoping manuscripts will emerge from this Coordinated Multicenter Team approach.

DISCUSSION

Comprehensively describing and differentiating research partnership terminology and its domains will address well-documented gaps in the literature. These efforts will contribute to and improve the quality, conduct, and reporting of research partnership literature. The collaborative review methodology will help identify and establish common terms, leverage efficiencies (e.g., expertise, experience, search and protocol design, resources) and optimize research feasibility and quality. Our approach allows for enhanced scope and inclusivity of all research user groups and domains, thereby contributing uniquely to the literature. This multicenter, efficiency and quality-focused approach may serve to inspire researchers across the globe in addressing similar domain challenges, as exist in this rapidly expanding field.

How can additional secondary data analysis of observational data enhance the generalisability of meta-analytic evidence for local public health decision making?

This paper critically explores how survey and routinely collected data could aid in assessing the generalisability of public health evidence. We propose developing approaches that could be employed in understanding the relevance of public health evidence, and investigate ways of producing meta-analytic estimates tailored to reflect local circumstances, based on analyses of secondary data. Currently, public health decision makers face challenges in interpreting global review evidence to assess its meaning in local contexts. A lack of clarity on the definition and scope of generalisability, and the absence of consensus on its measurement, has stunted methodological progress. The consequence of failing to tackle generalisability means that systematic review evidence often fails to fulfil its potential contribution in public health decision making. Three approaches to address these problems are considered and emerging challenges discussed: (1) purposeful exploration after a review has been conducted, and we present a framework of potential avenues of enquiry and a worked example; (2) recalibration of the results to weight studies differentially based on their similarity to conditions in an inference population, and we provide a worked example using UK Census data to understand potential differences in the effectiveness of community engagement interventions among sites in England and Wales; (3) purposeful exploration before starting a review to ensure that the findings are relevant to an inference population. The paper aims to demonstrate how a more nuanced treatment of context in reviews of public health interventions could be achieved through greater engagement with existing large sources of secondary data.

Institutionalising an evidence-informed approach to guideline development: progress and challenges at the World Health Organization

This study explored experiences, perceptions and views among World Health Organization (WHO) staff about the changes, progress and challenges brought by the guideline development reforms initiated in 2007. Thirty-five semistructured interviews were conducted with senior WHO staff. Sixteen of the interviewees had in-depth experience with WHO's formal guideline development process. Thematic analysis was conducted to identify key themes in the qualitative data, and these were interpreted in the context of the existing literature on WHO's guideline development processes. First, the reforms were seen to have transformed and improved the quality of WHO's guidelines. Second, independent evaluation and feedback by the Guidelines Review Committee (GRC) was described to have strengthened the legitimacy of WHO's recommendations. Third, WHO guideline development processes are not yet designed to systematically make use of all types of research evidence needed to inform decisions about health systems and public health interventions. For example, several interviewees expressed dissatisfaction with the insufficient attention paid to qualitative evidence and evidence from programme experience, and how the Grading of Recommendations Assessment, Development and Evaluation (GRADE) process evaluates the quality of evidence from non-randomised study designs, while others believed that GRADE was just not properly understood or applied. Fourth, some staff advocated for a more centralised quality assurance process covering all outputs from WHO's departments and scientific advisory committees, especially to eliminate strategic efforts aimed at bypassing the GRC's requirements. Overall, the 'culture change' senior WHO staff called for over 10 years ago appears to have gradually spread throughout the organisation. However, at least two major challenges remain: (1) ensuring that all issued advice benefits from independent evaluation, monitoring and feedback for quality and (2) designing guideline development processes to better acquire, assess, adapt and apply the full range of evidence that can inform recommendations on health systems and public health interventions.

Towards a global monitoring system for implementing the Rio Political Declaration on Social Determinants of Health: developing a core set of indicators for government action on the social determinants of health to improve health equity

Background

In the 2011 Rio Political Declaration on Social Determinants of Health, World Health Organization (WHO) Member States pledged action in five areas crucial for addressing health inequities. Their pledges referred to better governance for health and development, greater participation in policymaking and implementation, further reorientation of the health sector towards reducing health inequities, strengthening of global governance and collaboration, and monitoring progress and increasing accountability. WHO is developing a global system for monitoring governments’ and international organizations’ actions on the social determinants of health (SDH) to increase transparency and accountability, and to guide implementation, in alignment with broader health and development policy frameworks, including the universal health coverage and Sustainable Development Goals (SDG) agendas. We describe the selection of indicators proposed to be part of the initial WHO global system for monitoring action on the SDH.

Methods

An interdisciplinary working group was established by WHO, the Public Health Agency of Canada, and the Canadian Institutes of Health Research—Institute of Population and Public Health. We describe the processes and criteria used for selecting SDH action indicators that were of high quality and the described the challenges encountered in creating a set of metrics for capturing government action on addressing the Rio Political Declaration’s five Action Areas.

Results

We developed 19 measurement concepts, identified and screened 20 indicator databases and systems, including the 223 SDG indicators, and applied strong criteria for selecting indicators for the core indicator set. We identified 36 suitable existing indicators, which were often SDG indicators.

Conclusions

Lessons learnt included the importance of ensuring diversity of the working group and always focusing on health equity; challenges included the relative dearth of data and indicators on some key interventions and capturing the context and level of implementation of indicator interventions.

Assessing the applicability of public health intervention evaluations from one setting to another: a methodological study of the usability and usefulness of assessment tools and frameworks

BACKGROUND

Public health interventions can be complicated, complex and context dependent, making the assessment of applicability challenging. Nevertheless, for them to be of use beyond the original study setting, they need to be generalisable to other settings and, crucially, research users need to be able to identify to which contexts it may be applicable. There are many tools with set criteria for assessing generalisability/applicability, yet few seem to be widely used and there is no consensus on which should be used, or when. This methodological study aimed to test these tools to assess how easy they were to use and how useful they appeared to be.

METHODS

We identified tools from an existing review and an update of its search. References were screened on pre-specified criteria. Included tools were tested by using them to assess the applicability of a Swedish weight management intervention to the English context. Researcher assessments and reflections on the usability and utility of the tools were gathered using a standard pro-forma.

RESULTS

Eleven tools were included. Their length, content, style and time required to complete varied. No tool was considered ideal for assessing applicability. Their limitations included unrealistic criteria (requiring unavailable information), a focus on implementation to the neglect of transferability (i.e. little focus on potential effectiveness in the new setting), overly broad criteria (associated with low reliability), and a lack of an explicit focus on how interventions worked (i.e. their mechanisms of action).

CONCLUSION

Tools presenting criteria ready to be used may not be the best method for applicability assessments. They are likely to be either too long or incomplete, too focused on differences and fail to address elements that matter for the specific topic of interest. It is time to progress from developing lists of set criteria that are not widely used in the literature, to creating a new approach to applicability assessment. Focusing on mechanisms of action, rather than solely on characteristics, could be a useful approach, and one that remains underutilised in current tools. New approaches to assessing generalisability that evolve away from checklist style assessments need to be developed, tested, reported and discussed.

Defining the process to literature searching in systematic reviews: a literature review of guidance and supporting studies

BACKGROUND

Systematic literature searching is recognised as a critical component of the systematic review process. It involves a systematic search for studies and aims for a transparent report of study identification, leaving readers clear about what was done to identify studies, and how the findings of the review are situated in the relevant evidence. Information specialists and review teams appear to work from a shared and tacit model of the literature search process. How this tacit model has developed and evolved is unclear, and it has not been explicitly examined before. The purpose of this review is to determine if a shared model of the literature searching process can be detected across systematic review guidance documents and, if so, how this process is reported in the guidance and supported by published studies.

METHOD

A literature review. Two types of literature were reviewed: guidance and published studies. Nine guidance documents were identified, including: The Cochrane and Campbell Handbooks. Published studies were identified through 'pearl growing', citation chasing, a search of PubMed using the systematic review methods filter, and the authors' topic knowledge. The relevant sections within each guidance document were then read and re-read, with the aim of determining key methodological stages. Methodological stages were identified and defined. This data was reviewed to identify agreements and areas of unique guidance between guidance documents. Consensus across multiple guidance documents was used to inform selection of 'key stages' in the process of literature searching.

RESULTS

Eight key stages were determined relating specifically to literature searching in systematic reviews. They were: who should literature search, aims and purpose of literature searching, preparation, the search strategy, searching databases, supplementary searching, managing references and reporting the search process.

CONCLUSIONS

Eight key stages to the process of literature searching in systematic reviews were identified. These key stages are consistently reported in the nine guidance documents, suggesting consensus on the key stages of literature searching, and therefore the process of literature searching as a whole, in systematic reviews. Further research to determine the suitability of using the same process of literature searching for all types of systematic review is indicated.