Statistical methods for designing and assessing the effectiveness of community-based interventions with small numbers

Evaluating the Effectiveness of Complex, Multi-component, Dynamic, Community-Based Injury Prevention Interventions: A Statistical Framework

Dynamic violence and injury prevention interventions located within community settings raise evaluation challenges by virtue of their complex structure, focus, and aims. They try to address many risk factors simultaneously, are often overlapped in their implementation, and their implementation may be phased over time. This article proposes a statistical and analytic framework for evaluating the effectiveness of multilevel, multisystem, multi-component, community-driven, dynamic interventions. The proposed framework builds on meta regression methodology and recently proposed approaches for pooling results from multi-component intervention studies. The methodology is applied to the evaluation of the effectiveness of South African community-centered injury prevention and safety promotion interventions. The proposed framework allows for complex interventions to be disaggregated into their constituent parts in order to extract their specific effects. The potential utility of the framework is successfully illustrated using contact crime data from select police stations in Johannesburg. The proposed framework and statistical guidelines proved to be useful to study the effectiveness of complex, dynamic, community-based interventions as a whole and of their components. The framework may help researchers and policy makers to adopt and study a specific methodology for evaluating the effectiveness of complex intervention programs.

Statistical methodologies to pool across multiple intervention studies

Combining and analyzing data from heterogeneous randomized controlled trials of complex multiple-component intervention studies, or discussing them in a systematic review, is not straightforward. The present article describes certain issues to be considered when combining data across studies, based on discussions in an NIH-sponsored workshop on pooling issues across studies in consortia (see Belle et al. in Psychol Aging, 18(3):396-405, 2003). Several statistical methodologies are described and their advantages and limitations are explored. Whether weighting the different studies data differently, or via employing random effects, one must recognize that different pooling methodologies may yield different results. Pooling can be used for comprehensive exploratory analyses of data from RCTs and should not be viewed as replacing the standard analysis plan for each study. Pooling may help to identify intervention components that may be more effective especially for subsets of participants with certain behavioral characteristics. Pooling, when supported by statistical tests, can allow exploratory investigation of potential hypotheses and for the design of future interventions.

Deconstructing interventions: approaches to studying behavior change techniques across obesity interventions

Deconstructing interventions into the specific techniques that are used to change behavior represents a new frontier in behavioral intervention research. This paper considers opportunities and challenges in employing the Behavior Change Techniques Taxonomy (BCTTv1) developed by Michie and colleagues, to code the behavior change techniques (BCTs) across multiple interventions addressing obesity and capture dose received at the technique level. Numerous advantages were recognized for using a shared framework for intervention description. Coding interventions at levels of the social ecological framework beyond the individual level, separate coding for behavior change initiation vs. maintenance, fidelity of BCT delivery, accounting for BCTs mode of delivery, and tailoring BCTs, present both challenges and opportunities. Deconstructing interventions and identifying the dose required to positively impact health-related outcomes could enable important gains in intervention science.