Methodological idiosyncracies, frameworks and challenges of non-pharmaceutical and non-technical treatment interventions

Non-pharmacological interventions to improve mental health outcomes among female carers of people living with a neurological condition: a systematic review

PURPOSE

This systematic review aims to examine the effectiveness of non-pharmacological interventions for improving mental health outcomes among female carers of people living with a neurological condition.

MATERIALS AND METHODS

A narrative synthesis of English-language randomized controlled trials was undertaken.

RESULTS

18 unique studies were included. Intervention components that were found to have improved mental health outcomes were: delivered in person, to groups, on an intermittent schedule with ≥10 sessions; had a duration between 3-6 months; and were facilitated by research staff or allied health professionals. As the review had few robust studies, results of mental health outcomes reported in studies assessed as low risk of bias were highlighted in the review. Psychoeducation interventions, cognitive behavioural interventions, and support group interventions were found to improve depression. Psychoeducation interventions were also found to improve burden.

CONCLUSIONS

There is a clear need for adequately powered, high-quality randomised controlled trials to determine the effectiveness of non-pharmacological interventions for female carers of people living with a neurological condition.

Case studies to explore the optimal use of randomized and nonrandomized studies in evidence syntheses that use GRADE

OBJECTIVES

Randomized controlled trials (RCTs) are the preferred source of evidence for the relative effect of healthcare interventions summarized in knowledge syntheses. Nonrandomized studies of interventions (NRSI) may provide replacement, sequential, or complementary evidence to RCTs. The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach can provide different options for properly using RCTs and NRSI integrated in health syntheses. In this article, we discuss different implications on the certainty of evidence when authors consider the use of NRSI and RCTs in systematic reviews using GRADE. Although this is a GRADE-related article, it is not an official GRADE guidance or concept article.

STUDY DESIGN AND SETTING

We present case studies used during GRADE working group meetings for discussion of the effects of using NRSI and RCTs on GRADE domains and on the certainty of evidence. Several concepts were discussed through iterative feedback with experts in GRADE methods and Cochrane authors. We compared suggested solutions for possible scenarios that can be met in evidence syntheses informing decisions and future guidance.

RESULTS

Different scenarios for the use of RCTs and NRSI in evidence syntheses are presented, focusing on how different GRADE ratings between RCTs and NRSI affect the overall assessment of the evidence and possible health recommendations.

CONCLUSIONS

Considering differences and similarities grounded in the GRADE approach between NRSI and RCTs may help complement one another and maximize the value of knowledge syntheses and health recommendations.

[Routine Practice Data for Evaluating Intervention Effects: Part 2 of the Manual]

The evaluation of intervention effects is an important domain of health services research. The ad hoc commission for the use of routine practice data of the German Network for Health Services Research (DNVF) therefore provides this second part of its manual focusing on the use of routine practice data for the evaluation of intervention effects. First, we discuss definition issues and the importance of contextual factors. Subsequently, general requirements for planning, data collection and analysis as well as concrete examples for the evaluation of intervention effects for the 3 fields of application regarding pharmacotherapy, nonpharmaceutical interventions as well as complex interventions are elaborated. We consider scenarios in which no information from randomized controlled trials (RCTs) comparing the two groups directly is yet available or in which RCTs are already available but an extension of the research question is required. In all examples either with or without randomization, the first and foremost question is always whether the data source is suitable for the specific research question. Most of the examples chosen are from oncology trials, because the necessary data are already available for Germany, at least in some form. Finally, the manual discusses possible challenges for future use of these data.

Considerations of complexity in rating certainty of evidence in systematic reviews: a primer on using the GRADE approach in global health

Public health interventions and health technologies are commonly described as 'complex', as they involve multiple interacting components and outcomes, and their effects are largely influenced by contextual interactions and system-level processes. Systematic reviewers and guideline developers evaluating the effects of these complex interventions and technologies report difficulties in using existing methods and frameworks, such as the Grading of Recommendations Assessment, Development and Evaluation (GRADE). As part of a special series of papers on implications of complexity in the WHO guideline development, this paper serves as a primer on how to consider sources of complexity when using the GRADE approach to rate certainty of evidence. Relevant sources of complexity in systematic reviews, health technology assessments and guidelines of public health are outlined and mapped onto the reported difficulties in rating the estimates of the effect of these interventions. Recommendations on how to address these difficulties are further outlined, and the need for an integrated use of GRADE from the beginning of the review or guideline development is emphasised. The content of this paper is informed by the existing GRADE guidance, an ongoing research project on considering sources of complexity when applying the GRADE approach to rate certainty of evidence in systematic reviews and the review authors' own experiences with using GRADE.

Using systematic reviews in guideline development: the GRADE approach

Systematic reviews are essential to produce trustworthy guidelines. To assess the certainty of a body of evidence included in a systematic review the Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group has developed an approach that is currently used by over 100 organisations, including the World Health Organization and the Cochrane Collaboration. GRADE provides operational definitions and instructions to rate the certainty of the evidence for each outcome in a review as high, moderate, low, or very low for the effects of interventions, prognostic estimates, values and preferences, test accuracy and resource utilization. The assessment includes assessing risk of bias, imprecision, inconsistency, indirectness, and publication bias, the magnitude of effects, dose-response relations and the impact of residual confounding and bias. Summary statistical information and assessments of certainty are presented in GRADE evidence summary tables, which can be produced using GRADE's official GRADEpro software tool (www.gradepro.org/). The evidence summary tables feed into the GRADE Evidence to Decision frameworks which guideline panels can use to produce recommendations.

GRADE: Assessing the quality of evidence in environmental and occupational health

There is high demand in environmental health for adoption of a structured process that evaluates and integrates evidence while making decisions and recommendations transparent. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework holds promise to address this demand. For over a decade, GRADE has been applied successfully to areas of clinical medicine, public health, and health policy, but experience with GRADE in environmental and occupational health is just beginning. Environmental and occupational health questions focus on understanding whether an exposure is a potential health hazard or risk, assessing the exposure to understand the extent and magnitude of risk, and exploring interventions to mitigate exposure or risk. Although GRADE offers many advantages, including its flexibility and methodological rigor, there are features of the different sources of evidence used in environmental and occupational health that will require further consideration to assess the need for method refinement. An issue that requires particular attention is the evaluation and integration of evidence from human, animal, in vitro, and in silico (computer modeling) studies when determining whether an environmental factor represents a potential health hazard or risk. Assessment of the hazard of exposures can produce analyses for use in the GRADE evidence-to-decision (EtD) framework to inform risk-management decisions about removing harmful exposures or mitigating risks. The EtD framework allows for grading the strength of the recommendations based on judgments of the certainty in the evidence (also known as quality of the evidence), as well as other factors that inform recommendations such as social values and preferences, resource implications, and benefits. GRADE represents an untapped opportunity for environmental and occupational health to make evidence-based recommendations in a systematic and transparent manner. The objectives of this article are to provide an overview of GRADE, discuss GRADE's applicability to environmental health, and identify priority areas for method assessment and development.

Improving GRADE evidence tables part 3: detailed guidance for explanatory footnotes supports creating and understanding GRADE certainty in the evidence judgments

BACKGROUND

The Grading of Recommendations Assessment, Development and Evaluation (GRADE) is widely used and reliable and accurate for assessing the certainty in the body of health evidence. The GRADE working group has provided detailed guidance for assessing the certainty in the body of evidence in systematic reviews and health technology assessments (HTAs) and how to grade the strength of health recommendations. However, there is limited advice regarding how to maximize transparency of these judgments, in particular through explanatory footnotes or explanations in Summary of Findings tables and Evidence Profiles (GRADE evidence tables).

METHODS

We conducted this study to define the essential attributes of useful explanations and to develop specific guidance for explanations associated with GRADE evidence tables. We used a sample of explanations according to their complexity, type of judgment involved, and appropriateness from a database of published GRADE evidence tables in Cochrane reviews and World Health Organization guidelines. We used an iterative process and group consensus to determine the attributes and develop guidance.

RESULTS

Explanations in GRADE evidence tables should be concise, informative, relevant, easy to understand, and accurate. We provide general and domain-specific guidance to assist authors with achieving these desirable attributes in their explanations associated with GRADE evidence tables.

CONCLUSIONS

Adhering to the general and GRADE domain-specific guidance should improve the quality of explanations associated with GRADE evidence tables, assist authors of systematic reviews, HTA reports, or guidelines with information that they can use in other parts of their evidence synthesis. This guidance will also support editorial evaluation of evidence syntheses using GRADE and provide a minimum quality standard of judgments across tables.

Conceptual decomposition of complex health care interventions for evidence synthesis: a literature review

RATIONALE, AIMS AND OBJECTIVES

The clarity of pivotal concepts is an important prerequisite for the development, evaluation and exchange of scientific ideas. The term 'complex intervention' is increasingly used in the health care literature, although it often remains unclear what is actually meant by this concept. Therefore, our aim was to analyse the literature regarding definitions of the terms 'complex intervention' and 'components' of such interventions.

METHOD

To identify the methodological publications, systematic and snowballing techniques were combined for the literature search. Relevant units of meaning were extracted from 68 included publications. Afterwards, we deduced categories and related frequencies by inductive and quantitative content analysis techniques.

RESULTS

Several types of complexity were distinguished in the literature. Most authors viewed complex interventions as multicomponent interventions that are characterized by interactions between the components themselves, with the context or as systemic interventions. Components of complex interventions were described in the publications as having the potential to causally influence outcomes, thus being essential for achieving an effect. Other definitions and inconsistencies among the definitions are highlighted and discussed.

CONCLUSIONS

From our synthesis, we derived definitions of the central health care-related concepts 'complex intervention' and 'components' of an intervention. Although we found numerous diverse definitions, they could be reduced to a defined number of core characteristics. These characteristics may facilitate communication regarding complex interventions and enable the deduction of methodological approaches for evidence synthesis.