Emergent approaches to the meta-analysis of multiple heterogeneous complex interventions

Inpatient-level care at home delivered by virtual wards and hospital at home: a systematic review and meta-analysis of complex interventions and their components

BACKGROUND

Technology-enabled inpatient-level care at home services, such as virtual wards and hospital at home, are being rapidly implemented. This is the first systematic review to link the components of these service delivery innovations to evidence of effectiveness to explore implications for practice and research.

METHODS

For this review (registered here https://osf.io/je39y ), we searched Cochrane-recommended multiple databases up to 30 November 2022 and additional resources for randomised and non-randomised studies that compared technology-enabled inpatient-level care at home with hospital-based inpatient care. We classified interventions into care model groups using three key components: clinical activities, workforce, and technology. We synthesised evidence by these groups quantitatively or narratively for mortality, hospital readmissions, cost-effectiveness and length of stay.

RESULTS

We include 69 studies: 38 randomised studies (6413 participants; largely judged as low or unclear risk of bias) and 31 non-randomised studies (31,950 participants; largely judged at serious or critical risk of bias). The 69 studies described 63 interventions which formed eight model groups. Most models, regardless of using low- or high-intensity technology, may have similar or reduced hospital readmission risk compared with hospital-based inpatient care (low-certainty evidence from randomised trials). For mortality, most models had uncertain or unavailable evidence. Two exceptions were low technology-enabled models that involve hospital- and community-based professionals, they may have similar mortality risk compared with hospital-based inpatient care (low- or moderate-certainty evidence from randomised trials). Cost-effectiveness evidence is unavailable for high technology-enabled models, but sparse evidence suggests the low technology-enabled multidisciplinary care delivered by hospital-based teams appears more cost-effective than hospital-based care for those with chronic obstructive pulmonary disease (COPD) exacerbations.

CONCLUSIONS

Low-certainty evidence suggests that none of technology-enabled care at home models we explored put people at higher risk of readmission compared with hospital-based care. Where limited evidence on mortality is available, there appears to be no additional risk of mortality due to use of technology-enabled at home models. It is unclear whether inpatient-level care at home using higher levels of technology confers additional benefits. Further research should focus on clearly defined interventions in high-priority populations and include comparative cost-effectiveness evaluation.

TRIAL REGISTRATION

https://osf.io/je39y .

Comparison of internet-based and face-to-face cognitive behavioral therapy for obsessive-compulsive disorder: A systematic review and network meta-analysis

Cognitive behavioral therapy (CBT) is widely recognized as an effective treatment for obsessive-compulsive disorder (OCD). However, few patients are able to receive CBT. Internet-based CBT (ICBT) may be able to overcome this problem. In this study, we aimed to compare the efficacy of CBT, therapist-guided ICBT (TG-ICBT), unguided ICBT (UG-ICBT), and none therapist-guided ICBT (NTG-ICBT) by a network meta-analysis. The primary outcome was the mean change in OCD severity measured by the Yale-Brown Obsessive Compulsive Scale (Y-BOCS) or the Children's Yale-Brown Obsessive Compulsive Scale (CY-BOCS). The secondary outcomes included the severity of depressive symptoms, side effects, and cost-effectiveness. A total of 25 trials with 1642 participants were included. We found that the efficacy of CBT was superior to that of TG-ICBT. The mean improvement in Y-BOCS/CY-BOCS scores was higher in CBT group than in UG-ICBT group, but this difference was not statistically significant. The efficacy did not differ significantly between TG-ICBT and UG-ICBT. CBT, TG-ICBT, and UG-ICBT were all more effective than the psychological placebo, waiting list, and pill placebo. In terms of efficacy, CBT combined with drug therapy was better than CBT, TG-ICBT, and UG-ICBT. Sensitivity analyses supported these findings. Based on the current evidence, we recommend TG-ICBT when CBT is not available. However, it is undeniable that UG-ICBT also has the potential to be an effective alternative to CBT. More evidence is needed to support this possibility.

Dismantling cognitive-behavioural therapy components for attention-deficit hyperactivity disorder in adolescents and adults: protocol for a network meta-analysis

INTRODUCTION

Cognitive-behavioural therapy (CBT) consists of multiple treatment techniques for each treatment model and is tailored to the patient's characteristics. Randomised controlled trials (RCTs) have reported that CBT is effective for attention-deficit/hyperactivity disorder (ADHD); however, which CBT components are effective is unknown. In order to provide the best treatment technique, it is important to know which therapeutic component or combination thereof is more effective and what the specific effect size is.

METHODS AND ANALYSIS

We will perform component network meta-analysis (cNMA). The search will include studies published from database inception up to 31 March 2022, in English. The electronic databases of MEDLINE (via PubMed), EMBASE, PsycINFO, ClinicalTrials.gov and Cochrane Library will be searched. We will systematically identify all RCTs in the treatment of ADHD between the ages of 10 and 60 years, comparing interventions composed of various CBT components with controlled interventions. We will perform pairwise and network meta-analysis with random effects to estimate summary ORs and standardised mean differences. We will assess the risk of bias in selected studies using the Cochrane risk of bias tool.

ETHICS AND DISSEMINATION

Since we will review published papers, ethical approval is not required. The results from this cNMA will provide a panorama of the CBT-based ADHD studies. The results of this study will be published in a peer-reviewed journal.

PROSPERO REGISTRATION NUMBER

CRD42022323898.

Critical life course interventions for children and adolescents to promote mental health

Childhood and adolescence are key developmental periods in the life course for addressing mental health, and there is ample evidence to support significant, increased investment in mental health promotion for this group. However, there are gaps in evidence to inform how best to implement mental health promotion interventions at scale. In this review, we examined psychosocial interventions implemented with children (aged 5–10 years) and adolescents (aged 10–19 years), drawing on evidence from WHO guidelines. Most psychosocial interventions promoting mental health have been implemented in school settings, with some in family and community settings, by a range of delivery personnel. Mental health promotion interventions for younger ages have prioritised key social and emotional skills development, including self-regulation and coping; for older ages, additional skills include problem-solving and interpersonal skills. Overall, fewer interventions have been implemented in low- and middle-income countries. We identify cross-cutting areas affecting child and adolescent mental health promotion: understanding the problem scope; understanding which components work; understanding how and for whom interventions work in practice; and ensuring supportive infrastructure and political will. Additional evidence, including from participatory approaches, is required to tailor mental health promotive interventions to diverse groups’ needs and support healthy life course trajectories for children and adolescents everywhere.

E-health interventions targeting STIs, sexual risk, substance use and mental health among men who have sex with men: four systematic reviews

Background

Human immunodeficiency virus/sexually transmitted infections, sexual risk, substance (alcohol and other legal and illegal drugs) use and mental ill health constitute a ‘syndemic’ of mutually reinforcing epidemics among men who have sex with men. Electronic health (e-health) interventions addressing these epidemics among men who have sex with men might have multiplicative effects. To our knowledge, no systematic review has examined the effectiveness of such interventions on these epidemics among men who have sex with men.

Objective

The objective was to synthesise evidence addressing the following: (1) What approaches and theories of change do existing e-health interventions employ to prevent human immunodeficiency virus/sexually transmitted infections, sexual risk, alcohol/drug use or mental ill health among men who have sex with men? (2) What factors influence implementation? (3) What are the effects of such interventions on the aforementioned epidemics? (4) Are such interventions cost-effective?

Data sources

A total of 24 information sources were searched initially (October–November 2018) [the following sources were searched: ProQuest Applied Social Sciences Index and Abstracts; Campbell Library; EBSCO Cumulative Index to Nursing and Allied Health Literature Plus, Wiley Online Library The Cochrane Library; Centre for Reviews and Dissemination databases (the Database of Abstracts of Reviews of Effects and the NHS Economic Evaluation Database); the Health Technology Assessment database; Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre) database of health promotion research (Bibliomap); ProQuest Dissertations & Theses Global; OvidSP EconLit; OvidSP EMBASE; OvidSP Global Health; OvidSP Health Management Information Consortium; ProQuest International Bibliography of the Social Sciences; Ovid MEDLINE ALL; OvidSP PsycINFO; Web of Science Science Citation Index Expanded; Elsevier Scopus; OvidSP Social Policy & Practice; Web of Science Social Sciences Citation Index Expanded; ProQuest Sociological Abstracts; ClinicalTrials.gov; World Health Organization International Clinical Trials Registry Platform; EPPI-Centre Trials Register of Promoting Health Interventions; and the OpenGrey database], and an updated search of 19 of these was conducted in April 2020. Reference lists of included reports were searched and experts were contacted.

Review methods

Eligible reports presented theories of change and/or process, outcome and/or economic evaluations of e-health interventions offering ongoing support to men who have sex with men to prevent human immunodeficiency virus/sexually transmitted infections, sexual risk behaviour, alcohol/drug use and/or common mental illnesses. References were screened by title/abstract, then by full text. Data extraction and quality assessments used existing tools. Theory and process reports were synthesised using qualitative methods. Outcome and economic data were synthesised narratively; outcome data were meta-analysed.

Results

Original searches retrieved 27 eligible reports. Updated searches retrieved 10 eligible reports. Thirty-seven reports on 28 studies of 23 interventions were included: 33 on theories of change, 12 on process evaluations, 16 on outcome evaluations and one on an economic evaluation. Research question 1: five intervention types were identified – ‘online modular’, ‘computer games’ and ‘non-interactive’ time-limited/modular interventions, and open-ended interventions with ‘content organised by assessment’ and ‘general content’. Three broad types of intervention theories of change were identified, focusing on ‘cognitive/skills’, ‘self-monitoring’ and ‘cognitive therapy’. Research question 2: individual tailoring based on participant characteristics was particularly acceptable, and participants valued intervention content reflecting their experiences. Research question 3: little evidence was available of effects on human immunodeficiency virus or sexually transmitted infections. The analysis did not suggest that interventions were effective in reducing instances of human immunodeficiency virus or sexually transmitted infections. The overall meta-analysis for sexually transmitted infections reported a small non-significant increase in sexually transmitted infections in the intervention group, compared with the control group. Meta-analyses found a significant impact on sexual risk behaviour. The findings for drug use could not be meta-analysed because of study heterogeneity. Studies addressing this outcome did not present consistent evidence of effectiveness. Trials did not report effects on alcohol use or mental health. Research question 4: evidence on cost-effectiveness was limited.

Limitations

The quality of the eligible reports was variable and the economic synthesis was limited to one eligible study.

Conclusions

There is commonality in intervention theories of change and factors affecting receipt of e-health interventions. Evidence on effectiveness is limited.

Future work

Future trials should assess the impact of interventions on multiple syndemic factors, among them sexual risk, substance use and mental health; incorporate sufficient follow-up and sample sizes to detect the impact on human immunodeficiency virus/sexually transmitted infections; and incorporate rigorous process and economic evaluations.

Study registration

This study is registered as PROSPERO CRD42018110317.

Funding

This project was funded by the National Institute for Health Research (NIHR) Public Health Research programme and will be published in full in Public Health Research; Vol. 10, No. 4. See the NIHR Journals Library website for further project information.

Research Review: The most effective parenting program content for disruptive child behavior - a network meta-analysis

BACKGROUND

Programs to support parents are the recommended strategy to reduce disruptive child behavior problems. Efforts have been made to demonstrate which program components (i.e., clusters of techniques taught) increase program effects, but these methods fail to account for the fact that components rarely operate in isolation. We examine how combinations of components cluster together to form program types and use network meta-analysis to estimate the relative effects of these program types.

METHODS

We updated an existing systematic review of parenting programs for disruptive child behavior and identified 197 randomized trials. We modeled clusters of components in each trial arm and chose the best-fitting model. We subsequently took 20 draws from the probability distribution of the latent class for each arm, entered each draw into a network meta-analysis model and combined findings using Rubin's rules. Combined estimates were bootstrapped to rank the clusters. We estimated main models and separate models for prevention and treatment settings.

RESULTS

A five-class solution fit the data best: (1) behavior management; (2) behavior management with parental self-management; (3) behavior management with psychoeducation and relationship enhancement; (4) maximal component loading and (5) no/minimal component loading (i.e. control). In the main model and in treatment settings, all four program types were effective compared to no/minimal components. In prevention settings, only behavior management and behavior management with parental self-management were effective compared to no/minimal components. Probabilistic ranking showed that overall and in treatment settings, behavior management had the largest chance, and in prevention settings, behavior management with self-management had the largest chance, of being most effective compared to no/minimal components.

CONCLUSIONS

Programs with more focused content seem more likely to yield stronger effects, and different foci may be needed in treatment versus prevention settings. Next steps include identifying individual family differences in optimal program content.

School-based interventions to prevent anxiety, depression and conduct disorder in children and young people: a systematic review and network meta-analysis

Background

Schools in the UK increasingly have to respond to anxiety, depression and conduct disorder as key causes of morbidity in children and young people.

Objective

The objective was to assess the comparative effectiveness of educational setting-based interventions for the prevention of anxiety, depression and conduct disorder in children and young people.

Design

This study comprised a systematic review, a network meta-analysis and an economic evaluation.

Data sources

The databases MEDLINE, EMBASE™ (Elsevier, Amsterdam, the Netherlands), PsycInfo® (American Psychological Association, Washington, DC, USA) and Cochrane Central Register of Controlled Trials (CENTRAL) were searched to 4 April 2018, and the NHS Economic Evaluation Database (NHS EED) was searched on 22 May 2019 for economic evaluations. No language or date filters were applied.

Main outcomes

The main outcomes were post-intervention self-reported anxiety, depression or conduct disorder symptoms.

Review methods

Randomised/quasi-randomised trials of universal or targeted interventions for the prevention of anxiety, depression or conduct disorder in children and young people aged 4–18 years were included. Screening was conducted independently by two reviewers. Data extraction was conducted by one reviewer and checked by a second. Intervention- and component-level network meta-analyses were conducted in OpenBUGS. A review of the economic literature and a cost–consequence analysis were conducted.

Results

A total of 142 studies were included in the review, and 109 contributed to the network meta-analysis. Of the 109 studies, 57 were rated as having an unclear risk of bias for random sequence generation and allocation concealment. Heterogeneity was moderate. In universal secondary school settings, mindfulness/relaxation interventions [standardised mean difference (SMD) –0.65, 95% credible interval (CrI) –1.14 to –0.19] and cognitive–behavioural interventions (SMD –0.15, 95% CrI –0.34 to 0.04) may be effective for anxiety. Cognitive–behavioural interventions incorporating a psychoeducation component may be effective (SMD –0.30, 95% CrI –0.59 to –0.01) at preventing anxiety immediately post intervention. There was evidence that exercise was effective in preventing anxiety in targeted secondary school settings (SMD –0.47, 95% CrI –0.86 to –0.09). There was weak evidence that cognitive–behavioural interventions may prevent anxiety in universal (SMD –0.07, 95% CrI –0.23 to 0.05) and targeted (SMD –0.38, 95% CrI –0.84 to 0.07) primary school settings. There was weak evidence that cognitive–behavioural (SMD –0.04, 95% CrI –0.16 to 0.07) and cognitive–behavioural + interpersonal therapy (SMD –0.18, 95% CrI –0.46 to 0.08) may be effective in preventing depression in universal secondary school settings. Third-wave (SMD –0.35, 95% CrI –0.70 to 0.00) and cognitive–behavioural interventions (SMD –0.11, 95% CrI –0.28 to 0.05) incorporating a psychoeducation component may be effective at preventing depression immediately post intervention. There was no evidence of intervention effectiveness in targeted secondary, targeted primary or universal primary school settings post intervention. The results for university settings were unreliable because of inconsistency in the network meta-analysis. A narrative summary was reported for five conduct disorder prevention studies, all in primary school settings. None reported the primary outcome at the primary post-intervention time point. The economic evidence review reported heterogeneous findings from six studies. Taking the perspective of a single school budget and based on cognitive–behavioural therapy intervention costs in universal secondary school settings, the cost–consequence analysis estimated an intervention cost of £43 per student.

Limitations

The emphasis on disorder-specific prevention excluded broader mental health interventions and restricted the number of eligible conduct disorder prevention studies. Restricting the study to interventions delivered in the educational setting may have limited the number of eligible university-level interventions.

Conclusions

There was weak evidence of the effectiveness of school-based, disorder-specific prevention interventions, although effects were modest and the evidence not robust. Cognitive–behavioural therapy-based interventions may be more effective if they include a psychoeducation component.

Future work

Future trials for prevention of anxiety and depression should evaluate cognitive–behavioural interventions with and without a psychoeducation component, and include mindfulness/relaxation or exercise comparators, with sufficient follow-up. Cost implications must be adequately measured.

Study registration

This study is registered as PROSPERO CRD42016048184.

Funding

This project was funded by the National Institute for Health Research (NIHR) Public Health Research programme and will be published in full in Public Health Research; Vol. 9, No. 8. See the NIHR Journals Library website for further project information.

Classifying information-sharing methods

BACKGROUND

Sparse relative effectiveness evidence is a frequent problem in Health Technology Assessment (HTA). Where evidence directly pertaining to the decision problem is sparse, it may be feasible to expand the evidence-base to include studies that relate to the decision problem only indirectly: for instance, when there is no evidence on a comparator, evidence on other treatments of the same molecular class could be used; similarly, a decision on children may borrow-strength from evidence on adults. Usually, in HTA, such indirect evidence is either included by ignoring any differences ('lumping') or not included at all ('splitting'). However, a range of more sophisticated methods exists, primarily in the biostatistics literature. The objective of this study is to identify and classify the breadth of the available information-sharing methods.

METHODS

Forwards and backwards citation-mining techniques were used on a set of seminal papers on the topic of information-sharing. Papers were included if they specified (network) meta-analytic methods for combining information from distinct populations, interventions, outcomes or study-designs.

RESULTS

Overall, 89 papers were included. A plethora of evidence synthesis methods have been used for information-sharing. Most papers (n=79) described methods that shared information on relative treatment effects. Amongst these, there was a strong emphasis on methods for information-sharing across multiple outcomes (n=42) and treatments (n=25), with fewer papers focusing on study-designs (n=23) or populations (n=8). We categorise and discuss the methods under four 'core' relationships of information-sharing: functional, exchangeability-based, prior-based and multivariate relationships, and explain the assumptions made within each of these core approaches.

CONCLUSIONS

This study highlights the range of information-sharing methods available. These methods often impose more moderate assumptions than lumping or splitting. Hence, the degree of information-sharing that they impose could potentially be considered more appropriate. Our identification of four 'core' methods of information-sharing allows for an improved understanding of the assumptions underpinning the different methods. Further research is required to understand how the methods differ in terms of the strength of sharing they impose and the implications of this for health care decisions.

A review of the quantitative effectiveness evidence synthesis methods used in public health intervention guidelines

BACKGROUND

The complexity of public health interventions create challenges in evaluating their effectiveness. There have been huge advancements in quantitative evidence synthesis methods development (including meta-analysis) for dealing with heterogeneity of intervention effects, inappropriate 'lumping' of interventions, adjusting for different populations and outcomes and the inclusion of various study types. Growing awareness of the importance of using all available evidence has led to the publication of guidance documents for implementing methods to improve decision making by answering policy relevant questions.

METHODS

The first part of this paper reviews the methods used to synthesise quantitative effectiveness evidence in public health guidelines by the National Institute for Health and Care Excellence (NICE) that had been published or updated since the previous review in 2012 until the 19th August 2019.The second part of this paper provides an update of the statistical methods and explains how they address issues related to evaluating effectiveness evidence of public health interventions.

RESULTS

The proportion of NICE public health guidelines that used a meta-analysis as part of the synthesis of effectiveness evidence has increased since the previous review in 2012 from 23% (9 out of 39) to 31% (14 out of 45). The proportion of NICE guidelines that synthesised the evidence using only a narrative review decreased from 74% (29 out of 39) to 60% (27 out of 45).An application in the prevention of accidents in children at home illustrated how the choice of synthesis methods can enable more informed decision making by defining and estimating the effectiveness of more distinct interventions, including combinations of intervention components, and identifying subgroups in which interventions are most effective.

CONCLUSIONS

Despite methodology development and the publication of guidance documents to address issues in public health intervention evaluation since the original review, NICE public health guidelines are not making full use of meta-analysis and other tools that would provide decision makers with fuller information with which to develop policy. There is an evident need to facilitate the translation of the synthesis methods into a public health context and encourage the use of methods to improve decision making.

Behavioural interventions for smoking cessation: an overview and network meta-analysis

BACKGROUND

Smoking is a leading cause of disease and death worldwide. In people who smoke, quitting smoking can reverse much of the damage. Many people use behavioural interventions to help them quit smoking; these interventions can vary substantially in their content and effectiveness.

OBJECTIVES

To summarise the evidence from Cochrane Reviews that assessed the effect of behavioural interventions designed to support smoking cessation attempts and to conduct a network meta-analysis to determine how modes of delivery; person delivering the intervention; and the nature, focus, and intensity of behavioural interventions for smoking cessation influence the likelihood of achieving abstinence six months after attempting to stop smoking; and whether the effects of behavioural interventions depend upon other characteristics, including population, setting, and the provision of pharmacotherapy. To summarise the availability and principal findings of economic evaluations of behavioural interventions for smoking cessation, in terms of comparative costs and cost-effectiveness, in the form of a brief economic commentary.

METHODS

This work comprises two main elements. 1. We conducted a Cochrane Overview of reviews following standard Cochrane methods. We identified Cochrane Reviews of behavioural interventions (including all non-pharmacological interventions, e.g. counselling, exercise, hypnotherapy, self-help materials) for smoking cessation by searching the Cochrane Library in July 2020. We evaluated the methodological quality of reviews using AMSTAR 2 and synthesised data from the reviews narratively. 2. We used the included reviews to identify randomised controlled trials of behavioural interventions for smoking cessation compared with other behavioural interventions or no intervention for smoking cessation. To be included, studies had to include adult smokers and measure smoking abstinence at six months or longer. Screening, data extraction, and risk of bias assessment followed standard Cochrane methods. We synthesised data using Bayesian component network meta-analysis (CNMA), examining the effects of 38 different components compared to minimal intervention. Components included behavioural and motivational elements, intervention providers, delivery modes, nature, focus, and intensity of the behavioural intervention. We used component network meta-regression (CNMR) to evaluate the influence of population characteristics, provision of pharmacotherapy, and intervention intensity on the component effects. We evaluated certainty of the evidence using GRADE domains. We assumed an additive effect for individual components.

MAIN RESULTS

We included 33 Cochrane Reviews, from which 312 randomised controlled trials, representing 250,563 participants and 845 distinct study arms, met the criteria for inclusion in our component network meta-analysis. This represented 437 different combinations of components. Of the 33 reviews, confidence in review findings was high in four reviews and moderate in nine reviews, as measured by the AMSTAR 2 critical appraisal tool. The remaining 20 reviews were low or critically low due to one or more critical weaknesses, most commonly inadequate investigation or discussion (or both) of the impact of publication bias. Of note, the critical weaknesses identified did not affect the searching, screening, or data extraction elements of the review process, which have direct bearing on our CNMA. Of the included studies, 125/312 were at low risk of bias overall, 50 were at high risk of bias, and the remainder were at unclear risk. Analyses from the contributing reviews and from our CNMA showed behavioural interventions for smoking cessation can increase quit rates, but effectiveness varies on characteristics of the support provided. There was high-certainty evidence of benefit for the provision of counselling (odds ratio (OR) 1.44, 95% credibility interval (CrI) 1.22 to 1.70, 194 studies, n = 72,273) and guaranteed financial incentives (OR 1.46, 95% CrI 1.15 to 1.85, 19 studies, n = 8877). Evidence of benefit remained when removing studies at high risk of bias. These findings were consistent with pair-wise meta-analyses from contributing reviews. There was moderate-certainty evidence of benefit for interventions delivered via text message (downgraded due to unexplained statistical heterogeneity in pair-wise comparison), and for the following components where point estimates suggested benefit but CrIs incorporated no clinically significant difference: individual tailoring; intervention content including motivational components; intervention content focused on how to quit. The remaining intervention components had low-to very low-certainty evidence, with the main issues being imprecision and risk of bias. There was no evidence to suggest an increase in harms in groups receiving behavioural support for smoking cessation. Intervention effects were not changed by adjusting for population characteristics, but data were limited. Increasing intensity of behavioural support, as measured through the number of contacts, duration of each contact, and programme length, had point estimates associated with modestly increased chances of quitting, but CrIs included no difference. The effect of behavioural support for smoking cessation appeared slightly less pronounced when people were already receiving smoking cessation pharmacotherapies.

AUTHORS' CONCLUSIONS

Behavioural support for smoking cessation can increase quit rates at six months or longer, with no evidence that support increases harms. This is the case whether or not smoking cessation pharmacotherapy is also provided, but the effect is slightly more pronounced in the absence of pharmacotherapy. Evidence of benefit is strongest for the provision of any form of counselling, and guaranteed financial incentives. Evidence suggested possible benefit but the need of further studies to evaluate: individual tailoring; delivery via text message, email, and audio recording; delivery by lay health advisor; and intervention content with motivational components and a focus on how to quit. We identified 23 economic evaluations; evidence did not consistently suggest one type of behavioural intervention for smoking cessation was more cost-effective than another. Future reviews should fully consider publication bias. Tools to investigate publication bias and to evaluate certainty in CNMA are needed.

When complexity matters: a step-by-step guide to incorporating a complexity perspective in guideline development for public health and health system interventions

BACKGROUND

Guidelines on public health and health system interventions often involve considerations beyond effectiveness and safety to account for the impact that these interventions have on the wider systems in which they are implemented. This paper describes how a complexity perspective may be adopted in guideline development to facilitate a more nuanced consideration of a range of factors pertinent to decisions regarding public health and health system interventions. These factors include acceptability and feasibility, and societal, economic, and equity and equality implications of interventions.

MAIN MESSAGE

A 5-step process describes how to incorporate a complexity perspective in guideline development with examples to illustrate each step. The steps include: (i) guideline scoping, (ii) formulating questions, (iii) retrieving and synthesising evidence, (iv) assessing the evidence, and (v) developing recommendations. Guideline scoping using stakeholder consultations, complexity features, evidence mapping, logic modelling, and explicit decision criteria is emphasised as a key step that informs all subsequent steps.

CONCLUSIONS

Through explicit consideration of a range of factors and enhanced understanding of the specific circumstances in which interventions work, a complexity perspective can yield guidelines with better informed recommendations and facilitate local adaptation and implementation. Further work will need to look into the methods of collecting and assessing different types of evidence beyond effectiveness and develop procedural guidance for prioritising across a range of decision criteria.

Is Disease Intensity a Good Surrogate for Yield Loss or Toxin Contamination? A Case Study with Fusarium Head Blight of Wheat

Sometimes plant pathologists assess disease intensity when they are primarily interested in other response variables, such as yield loss or toxin concentration in harvested products. In these situations, disease intensity potentially could be considered a surrogate of yield or toxin. A surrogate is a variable which can be used instead of the variable of interest in the evaluation of experimental treatments or in making predictions. Surrogates can be measured earlier, more conveniently, or more cheaply than the variable of primary interest, but the reliability or validity of the surrogate must be shown. We demonstrate ways of quantifying two facets of surrogacy by using a protocol originally developed by Buyse and colleagues for medical research. Coefficient-of-determination type statistics can be used to conveniently assess the strength of surrogacy on a unitless scale. As a case study, we evaluated whether field severity of Fusarium head blight (i.e., FHB index) can be used as a surrogate for yield loss and deoxynivalenol (DON) toxin concentration in harvested wheat grain. Bivariate mixed models and corresponding approximations were fitted to data from 82 uniform fungicide trials conducted from 2008 to 2013. Individual-level surrogacy-for predicting the variable of interest (yield or DON) from the surrogate (index) in plots with the same treatment-was very low. Trial-level surrogacy-for predicting the effect of treatment (e.g., mean difference) for the variable of interest based on the effect of the treatment on the surrogate (index)-was moderate for yield, and only low for DON. Challenges in using disease severity as a surrogate for yield and toxin are discussed.

The process and delivery of CBT for depression in adults: a systematic review and network meta-analysis

Cognitive-behavioural therapy (CBT) is an effective treatment for depressed adults. CBT interventions are complex, as they include multiple content components and can be delivered in different ways. We compared the effectiveness of different types of therapy, different components and combinations of components and aspects of delivery used in CBT interventions for adult depression. We conducted a systematic review of randomised controlled trials in adults with a primary diagnosis of depression, which included a CBT intervention. Outcomes were pooled using a component-level network meta-analysis. Our primary analysis classified interventions according to the type of therapy and delivery mode. We also fitted more advanced models to examine the effectiveness of each content component or combination of components. We included 91 studies and found strong evidence that CBT interventions yielded a larger short-term decrease in depression scores compared to treatment-as-usual, with a standardised difference in mean change of -1.11 (95% credible interval -1.62 to -0.60) for face-to-face CBT, -1.06 (-2.05 to -0.08) for hybrid CBT, and -0.59 (-1.20 to 0.02) for multimedia CBT, whereas wait list control showed a detrimental effect of 0.72 (0.09 to 1.35). We found no evidence of specific effects of any content components or combinations of components. Technology is increasingly used in the context of CBT interventions for depression. Multimedia and hybrid CBT might be as effective as face-to-face CBT, although results need to be interpreted cautiously. The effectiveness of specific combinations of content components and delivery formats remain unclear. Wait list controls should be avoided if possible.

Interventions to improve the use of EMRs in primary health care: a systematic review and meta-analysis

Background

Electronic medical record (EMR) adoption in primary care has grown exponentially since their introduction in the 1970s. However, without their proper use benefits cannot be achieved. This includes: 1) the complete and safe documentation of patient information; 2) improved coordination of care; 3) reduced errors and 4) more involved patients. The use of EMRs is defined by practitioners using EMRs and their features to perform daily practice functions.

Objective

The purpose of this systematic review was to identify interventions aimed at improving EMR use in primary healthcare settings.

Methods

Ten online databases were searched to identify studies conducted in primary healthcare settings aimed at implementing interventions to observe the use of EMRs and directly measure the use of EMR functions or outcomes effected by the use of EMR functions.

Results

Of 2098 identified studies, 12 were included in the review. Results showed that interventions focused on the use of EMR functions, including referrals, electronic communication, reminders, use of clinical decision support systems and workflow management support functions, were five times more likely to show improvements in EMR use compared with controls. Interventions focused on data quality were five and a half times more likely to show improvements in EMR use compared with controls.

Conclusions

Individuals in primary healthcare settings aiming to improve EMR use would benefit from implementing interventions focused on EMR feature add-ons such as clinical decision support systems and customised referral templates, and provisions of educational materials, or financial incentives targeted at improving the use of EMR functions and data quality.

Network meta-analysis of multicomponent interventions

In network meta‐analysis (NMA), treatments can be complex interventions, for example, some treatments may be combinations of others or of common components. In standard NMA, all existing (single or combined) treatments are different nodes in the network. However, sometimes an alternative model is of interest that utilizes the information that some treatments are combinations of common components, called component network meta‐analysis (CNMA) model. The additive CNMA model assumes that the effect of a treatment combined of two components A and B is the sum of the effects of A and B, which is easily extended to treatments composed of more than two components. This implies that in comparisons equal components cancel out. Interaction CNMA models also allow interactions between the components. Bayesian analyses have been suggested. We report an implementation of CNMA models in the frequentist R package netmeta. All parameters are estimated using weighted least squares regression. We illustrate the application of CNMA models using an NMA of treatments for depression in primary care. Moreover, we show that these models can even be applied to disconnected networks, if the composite treatments in the subnetworks contain common components.

Dismantling, personalising and optimising internet cognitive-behavioural therapy for depression: a study protocol for individual participant data component network meta-analysis

INTRODUCTION

Psychotherapy is a complex intervention, consisting of various components and being implemented flexibly in consideration of individual patient's characteristics. It is then of utmost importance to know which of the various components or combinations thereof are more efficacious, what their specific effect sizes are and which types of patients may benefit more from different components or their combinations.

METHODS AND ANALYSIS

Internet-delivered cognitive-behavioural therapy (iCBT) offers a unique opportunity to systematically review and quantitatively disentangle the efficacy of various components because, unlike face-to-face cognitive-behavioural therapy, it allows identification of constituent components that are actually delivered to patients. We will systematically identify all randomised controlled trials that compared any form of iCBT against another form or a control intervention in the acute phase treatment of adult depression. We will apply component network meta-analysis (cNMA) to dismantle efficacy of individual components. We will use individual participant data in the cNMA to identify participant-level prognostic factors and effect modifiers for different components.

ETHICS AND DISSEMINATION

The investigators of the primary trials will have obtained ethical approval for the data used in the present study and for sharing the data, if this was necessary, according to local requirements and was not covered from the initial ethic assessment. Results from this study will be published in peer-reviewed journals and presented at relevant conferences.

PROSPERO REGISTRATION NUMBER

CRD42018104683.

Synthesising quantitative evidence in systematic reviews of complex health interventions

Public health and health service interventions are typically complex: they are multifaceted, with impacts at multiple levels and on multiple stakeholders. Systematic reviews evaluating the effects of complex health interventions can be challenging to conduct. This paper is part of a special series of papers considering these challenges particularly in the context of WHO guideline development. We outline established and innovative methods for synthesising quantitative evidence within a systematic review of a complex intervention, including considerations of the complexity of the system into which the intervention is introduced. We describe methods in three broad areas: non-quantitative approaches, including tabulation, narrative and graphical approaches; standard meta-analysis methods, including meta-regression to investigate study-level moderators of effect; and advanced synthesis methods, in which models allow exploration of intervention components, investigation of both moderators and mediators, examination of mechanisms, and exploration of complexities of the system. We offer guidance on the choice of approach that might be taken by people collating evidence in support of guideline development, and emphasise that the appropriate methods will depend on the purpose of the synthesis, the similarity of the studies included in the review, the level of detail available from the studies, the nature of the results reported in the studies, the expertise of the synthesis team and the resources available.

Can Maintenance of Certification Work? Associations of MOC and Improvements in Physicians' Knowledge and Practice

PURPOSE

To summarize the findings of studies, conducted by individuals both internal and external to the American Board of Medical Specialties (ABMS) Member Boards, of the associations of Maintenance of Certification (MOC) and improvements in physicians' knowledge and patient care processes or outcomes.

METHOD

The authors conducted a narrative review of studies identified by searching PubMed and Web of Science for English-language articles from the United States published between 2000 and May 2017. To be included, articles had to examine the relationship of MOC to physician knowledge, clinical practice processes, or patient care outcomes. The initial search yielded 811 articles. After two rounds of review and excluding those articles that did not fit the study criteria, 39 articles were included for analysis.

RESULTS

The 39 included studies were conducted by or included diplomates of 12 ABMS Member Boards. Twenty-two studies examined MOC processes that were developed by an ABMS Board; 17 examined interventions that were developed by nonboard entities but accepted for MOC credit by an ABMS Board. Thirty-eight studies examined a single component of MOC; 24 studied the improvement in medical practice component. Thirty-seven studies reported at least one positive outcome.

CONCLUSIONS

Most of the studies included in this review highlighted circumstances in which MOC was associated with positive impacts on physician knowledge and patient care processes or outcomes. Future collaborative research is needed to improve the relevance, helpfulness, and generalizability of continuing certification to different physicians across specialties and practice settings.

Evaluating the Effectiveness of Complex, Multi-component, Dynamic, Community-Based Injury Prevention Interventions: A Statistical Framework

Dynamic violence and injury prevention interventions located within community settings raise evaluation challenges by virtue of their complex structure, focus, and aims. They try to address many risk factors simultaneously, are often overlapped in their implementation, and their implementation may be phased over time. This article proposes a statistical and analytic framework for evaluating the effectiveness of multilevel, multisystem, multi-component, community-driven, dynamic interventions. The proposed framework builds on meta regression methodology and recently proposed approaches for pooling results from multi-component intervention studies. The methodology is applied to the evaluation of the effectiveness of South African community-centered injury prevention and safety promotion interventions. The proposed framework allows for complex interventions to be disaggregated into their constituent parts in order to extract their specific effects. The potential utility of the framework is successfully illustrated using contact crime data from select police stations in Johannesburg. The proposed framework and statistical guidelines proved to be useful to study the effectiveness of complex, dynamic, community-based interventions as a whole and of their components. The framework may help researchers and policy makers to adopt and study a specific methodology for evaluating the effectiveness of complex intervention programs.

Support surfaces for pressure ulcer prevention: A network meta-analysis

Background

Pressure ulcers are a prevalent and global issue and support surfaces are widely used for preventing ulceration. However, the diversity of available support surfaces and the lack of direct comparisons in RCTs make decision-making difficult.

Objectives

To determine, using network meta-analysis, the relative effects of different support surfaces in reducing pressure ulcer incidence and comfort and to rank these support surfaces in order of their effectiveness.

Methods

We conducted a systematic review, using a literature search up to November 2016, to identify randomised trials comparing support surfaces for pressure ulcer prevention. Two reviewers independently performed study selection, risk of bias assessment and data extraction. We grouped the support surfaces according to their characteristics and formed evidence networks using these groups. We used network meta-analysis to estimate the relative effects and effectiveness ranking of the groups for the outcomes of pressure ulcer incidence and participant comfort. GRADE was used to assess the certainty of evidence.

Main results

We included 65 studies in the review. The network for assessing pressure ulcer incidence comprised evidence of low or very low certainty for most network contrasts. There was moderate-certainty evidence that powered active air surfaces and powered hybrid air surfaces probably reduce pressure ulcer incidence compared with standard hospital surfaces (risk ratios (RR) 0.42, 95% confidence intervals (CI) 0.29 to 0.63; 0.22, 0.07 to 0.66, respectively). The network for comfort suggested that powered active air-surfaces are probably slightly less comfortable than standard hospital mattresses (RR 0.80, 95% CI 0.69 to 0.94; moderate-certainty evidence).

Conclusions

This is the first network meta-analysis of the effects of support surfaces for pressure ulcer prevention. Powered active air-surfaces probably reduce pressure ulcer incidence, but are probably less comfortable than standard hospital surfaces. Most prevention evidence was of low or very low certainty, and more research is required to reduce these uncertainties.

Meta-Analysis of Complex Interventions

Meta-analysis is a prominent method for estimating the effects of public health interventions, yet these interventions are often complex in ways that pose challenges to using conventional meta-analytic methods. This article discusses meta-analytic techniques that can be used in research syntheses on the effects of complex public health interventions. We first introduce the use of complexity frameworks to conceptualize public health interventions. We then present a menu of meta-analytic procedures for addressing various sources of complexity when answering questions about the effects of public health interventions in research syntheses. We conclude with a review of important practices and key resources for conducting meta-analyses on complex interventions, as well as future directions for research synthesis more generally. Overall, we argue that it is possible to conduct meaningful quantitative syntheses of research on the effects of public health interventions, though these meta-analyses may require the use of advanced techniques to properly consider and attend to issues of complexity.

The pharmacological and non-pharmacological treatment of attention deficit hyperactivity disorder in children and adolescents: A systematic review with network meta-analyses of randomised trials

BACKGROUND

Attention deficit hyperactivity disorder (ADHD) is one of the most commonly diagnosed psychiatric disorders in childhood. A wide variety of treatments have been used for the management of ADHD. We aimed to compare the efficacy and safety of pharmacological, psychological and complementary and alternative medicine interventions for the treatment of ADHD in children and adolescents.

METHODS AND FINDINGS

We performed a systematic review with network meta-analyses. Randomised controlled trials (≥ 3 weeks follow-up) were identified from published and unpublished sources through searches in PubMed and the Cochrane Library (up to April 7, 2016). Interventions of interest were pharmacological (stimulants, non-stimulants, antidepressants, antipsychotics, and other unlicensed drugs), psychological (behavioural, cognitive training and neurofeedback) and complementary and alternative medicine (dietary therapy, fatty acids, amino acids, minerals, herbal therapy, homeopathy, and physical activity). The primary outcomes were efficacy (treatment response) and acceptability (all-cause discontinuation). Secondary outcomes included discontinuation due to adverse events (tolerability), as well as serious adverse events and specific adverse events. Random-effects Bayesian network meta-analyses were conducted to obtain estimates as odds ratios (ORs) with 95% credibility intervals. We analysed interventions by class and individually. 190 randomised trials (52 different interventions grouped in 32 therapeutic classes) that enrolled 26114 participants with ADHD were included in complex networks. At the class level, behavioural therapy (alone or in combination with stimulants), stimulants, and non-stimulant seemed significantly more efficacious than placebo. Behavioural therapy in combination with stimulants seemed superior to stimulants or non-stimulants. Stimulants seemed superior to behavioural therapy, cognitive training and non-stimulants. Behavioural therapy, stimulants and their combination showed the best profile of acceptability. Stimulants and non-stimulants seemed well tolerated. Among medications, methylphenidate, amphetamine, atomoxetine, guanfacine and clonidine seemed significantly more efficacious than placebo. Methylphenidate and amphetamine seemed more efficacious than atomoxetine and guanfacine. Methylphenidate and clonidine seemed better accepted than placebo and atomoxetine. Most of the efficacious pharmacological treatments were associated with harms (anorexia, weight loss and insomnia), but an increased risk of serious adverse events was not observed. There is lack of evidence for cognitive training, neurofeedback, antidepressants, antipsychotics, dietary therapy, fatty acids, and other complementary and alternative medicine. Overall findings were limited by the clinical and methodological heterogeneity, small sample sizes of trials, short-term follow-up, and the absence of high-quality evidence; consequently, results should be interpreted with caution.

CONCLUSIONS

Clinical differences may exist between the pharmacological and non-pharmacological treatment used for the management of ADHD. Uncertainties about therapies and the balance between benefits, costs and potential harms should be considered before starting treatment. There is an urgent need for high-quality randomised trials of the multiple treatments for ADHD in children and adolescents. PROSPERO, number CRD42014015008.

Meta-analysis, complexity, and heterogeneity: a qualitative interview study of researchers' methodological values and practices

BACKGROUND

Complex or heterogeneous data pose challenges for systematic review and meta-analysis. In recent years, a number of new methods have been developed to meet these challenges. This qualitative interview study aimed to understand researchers' understanding of complexity and heterogeneity and the factors which may influence the choices researchers make in synthesising complex data.

METHODS

We conducted interviews with a purposive sample of researchers (N = 19) working in systematic review or meta-analysis across a range of disciplines. We analysed data thematically using a framework approach.

RESULTS

Participants reported using a broader range of methods and data types in complex reviews than in traditional reviews. A range of techniques are used to explore heterogeneity, but there is some debate about their validity, particularly when applied post hoc.

CONCLUSIONS

Technical considerations of how to synthesise complex evidence cannot be isolated from questions of the goals and contexts of research. However, decisions about how to analyse data appear to be made in a largely informal way, drawing on tacit expertise, and their relation to these broader questions remains unclear.

Physiotherapy for pain and disability in adults with complex regional pain syndrome (CRPS) types I and II

BACKGROUND

Complex regional pain syndrome (CRPS) is a painful and disabling condition that usually manifests in response to trauma or surgery. When it occurs, it is associated with significant pain and disability. It is thought to arise and persist as a consequence of a maladaptive pro-inflammatory response and disturbances in sympathetically-mediated vasomotor control, together with maladaptive peripheral and central neuronal plasticity. CRPS can be classified into two types: type I (CRPS I) in which a specific nerve lesion has not been identified, and type II (CRPS II) where there is an identifiable nerve lesion. Guidelines recommend the inclusion of a variety of physiotherapy interventions as part of the multimodal treatment of people with CRPS, although their effectiveness is not known.

OBJECTIVES

To determine the effectiveness of physiotherapy interventions for treating the pain and disability associated with CRPS types I and II.

SEARCH METHODS

We searched the following databases from inception up to 12 February 2015: CENTRAL (the Cochrane Library), MEDLINE, EMBASE, CINAHL, PsycINFO, LILACS, PEDro, Web of Science, DARE and Health Technology Assessments, without language restrictions, for randomised controlled trials (RCTs) of physiotherapy interventions for treating pain and disability in people CRPS. We also searched additional online sources for unpublished trials and trials in progress.

SELECTION CRITERIA

We included RCTs of physiotherapy interventions (including manual therapy, therapeutic exercise, electrotherapy, physiotherapist-administered education and cortically directed sensory-motor rehabilitation strategies) employed in either a stand-alone fashion or in combination, compared with placebo, no treatment, another intervention or usual care, or of varying physiotherapy interventions compared with each other in adults with CRPS I and II. Our primary outcomes of interest were patient-centred outcomes of pain intensity and functional disability.

DATA COLLECTION AND ANALYSIS

Two review authors independently evaluated those studies identified through the electronic searches for eligibility and subsequently extracted all relevant data from the included RCTs. Two review authors independently performed 'Risk of bias' assessments and rated the quality of the body of evidence for the main outcomes using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach.

MAIN RESULTS

We included 18 RCTs (739 participants) that tested the effectiveness of a broad range of physiotherapy-based interventions. Overall, there was a paucity of high quality evidence concerning physiotherapy treatment for pain and disability in people with CRPS I. Most included trials were at 'high' risk of bias (15 trials) and the remainder were at 'unclear' risk of bias (three trials). The quality of the evidence was very low or low for all comparisons, according to the GRADE approach.We found very low quality evidence that graded motor imagery (GMI; two trials, 49 participants) may be useful for improving pain (0 to 100 VAS) (mean difference (MD) -21.00, 95% CI -31.17 to -10.83) and functional disability (11-point numerical rating scale) (MD 2.30, 95% CI 1.12 to 3.48), at long-term (six months) follow-up, in people with CRPS I compared to usual care plus physiotherapy; very low quality evidence that multimodal physiotherapy (one trial, 135 participants) may be useful for improving 'impairment' at long-term (12 month) follow-up compared to a minimal 'social work' intervention; and very low quality evidence that mirror therapy (two trials, 72 participants) provides clinically meaningful improvements in pain (0 to 10 VAS) (MD 3.4, 95% CI -4.71 to -2.09) and function (0 to 5 functional ability subscale of the Wolf Motor Function Test) (MD -2.3, 95% CI -2.88 to -1.72) at long-term (six month) follow-up in people with CRPS I post stroke compared to placebo (covered mirror).There was low to very low quality evidence that tactile discrimination training, stellate ganglion block via ultrasound and pulsed electromagnetic field therapy compared to placebo, and manual lymphatic drainage combined with and compared to either anti-inflammatories and physical therapy or exercise are not effective for treating pain in the short-term in people with CRPS I. Laser therapy may provide small clinically insignificant, short-term, improvements in pain compared to interferential current therapy in people with CRPS I.Adverse events were only rarely reported in the included trials. No trials including participants with CRPS II met the inclusion criteria of this review.

AUTHORS' CONCLUSIONS

The best available data show that GMI and mirror therapy may provide clinically meaningful improvements in pain and function in people with CRPS I although the quality of the supporting evidence is very low. Evidence of the effectiveness of multimodal physiotherapy, electrotherapy and manual lymphatic drainage for treating people with CRPS types I and II is generally absent or unclear. Large scale, high quality RCTs are required to test the effectiveness of physiotherapy-based interventions for treating pain and disability of people with CRPS I and II. Implications for clinical practice and future research are considered.

Approaches for synthesising complex mental health interventions in meta-analysis

Clinical and statistical heterogeneity are commonplace in meta-analysis of mental health interventions. One possible source of this heterogeneity is the complexity of the intervention being evaluated. Complexity may relate to the intervention, or to the way in which it is implemented; however, the most common interpretation of a complex intervention is one which has multiple, potentially interacting components. In this article we outline different analytical strategies suggested for incorporating intervention complexity in a meta-analysis.