Evaluating complex interventions. Health improvement programmes: really too complex to evaluate?

Impact of mind-body intervention on proinflammatory cytokines interleukin 6 and 1β: A three-arm randomized controlled trial for persons with sleep disturbance and depression

Depressed people are prone to sleep disturbance, which may in return perpetuate the depression. Both depression and sleep disturbance influence proinflammatory cytokines interleukin (IL) 6 and 1β. Thus interventions for depression should consider the effect on sleep disturbance, and vice versa. Integrative Body-Mind-Spirit (IBMS) and Qigong interventions have been applied in a wide range of health and mental health conditions, including depression and sleep disturbance. This study aimed to evaluate the effect of these two mind-body therapies for persons with both depressive symptoms and sleep disturbance. A three-arm randomized controlled trial was conducted among 281 participants, who were randomly assigned to either IBMS, Qigong or wait list control group. Participants in IBMS and Qigong groups received eight weekly sessions of intervention. Outcome measures were plasma concentrations of IL-6 and IL-1β, and a questionnaire containing Pittsburgh Sleep Quality Index, Center for Epidemiologic Studies Depression Scale, Somatic Symptom Inventory, Perceived Stress Scale and Body-Mind-Spirit Holistic Well-being Scale. Outcomes were assessed at baseline (T0), immediate post-intervention (T1) and at three-months post-intervention (T2). Besides intervention efficacy analysis, path analysis was performed to explore the relations among perceived stress, depression, sleep disturbance, and IL-6 and IL-1β values. The study found both IBMS and Qigong reduced depression, sleep disturbance, painful and painless somatic symptoms, IL-6 and IL-1β levels, and increased holistic well-being. The effect sizes of IBMS and Qigong, mostly in the medium magnitude range, were approximatively equivalent. Path analysis models revealed a predictive role of perceived stress in depression and sleep disturbance, a bidirectional relationship between depression and sleep disturbance, and significant influence of depression and sleep disturbance on IL-6 and IL-1β. Compared with control, the findings support the efficacy of IBMS and Qigong interventions in relieving depression and sleep disturbance, and in reducing IL-6 and IL-1β levels.

Effects of guided counseling during pregnancy on birth weight of newborns in West Gojjam Zone, Ethiopia: a cluster-randomized controlled trial

Background

The high proportion of birth weight in Ethiopia is hypothesized to be due to inadequate maternal diet which is associated with poor nutrition education during pregnancy. There was no study that evaluated the effect of nutrition education on birth weight in the study area. This study aimed to assess the effects (overall, direct and indirect effects) of guided counseling on the birth weight of neonates.

Methods

A two-arm parallel cluster randomized controlled community trial was conducted from May 1, 2018, to April 30, 2019, in West Gojjam Zone, Northwest Ethiopia. At the baseline, 346 pregnant women in the 11 intervention clusters and 348 pregnant women in the 11 control clusters were recruited. However, birth weight was measured from 258 and 272 newborns in the intervention and control groups, respectively.

In the intervention group, counseling was given monthly for four consecutive months in the participant’s homes. Besides, leaflets with key counseling messages were distributed to each woman in the intervention arm. Pregnant women who attended routine nutrition education given by the health system were recruited as control. Dietary practice, nutritional status, and birth weight were the primary, secondary and tertiary outcomes of this intervention. Data were collected using a structured data collection tool. Birth weight was measured within 48 h after birth. Independent sample t-test, linear mixed-effects model, and path analysis were fitted to assess effects of the intervention.

Results

The intra-cluster correlation coefficient was 0.095. The average birth weight of newborns in the intervention group was 0.257 kg higher compared with their counterparts in the control arm (β = 0.257, P < 0.001). The direct effect of this intervention on birth weight was 0.17 (β = 0.17, P<0.001 ) whereas the indirect effect of this intervention was 0.08 (β = 0.08, P<0.001 ).

Conclusion

Counseling using the health belief model and the theory of planned behavior has a positive effect on improving birth weight. The findings suggest the need for enhancing nutrition education of pregnant women through the application of theories to improve birth weight.

Trial registration

Clinical Trials.gov NCT03627156, “Retrospectively registered Jun, 13, 2018”.

Effect of guided counseling on nutritional status of pregnant women in West Gojjam zone, Ethiopia: a cluster-randomized controlled trial

BACKGROUND

Undernutrition during pregnancy affects birth outcomes adversely. In Ethiopia, despite nutrition counseling on the maternal diet being given by the health workers during pregnancy, maternal undernutrition is still high in the country. Hence, this study aimed to assess the effect of guided counseling using the health belief model and the theory of planned behavior on the nutritional status of pregnant women.

METHODS

A two-arm parallel cluster randomized controlled community trial was conducted in West Gojjam Zone, Ethiopia, from May 1, 2018, to April 30, 2019. The nutritional status of the women was assessed using mid-upper arm circumference. A total of 694 pregnant women were recruited from the intervention (n=346 ) and control (n=348) clusters. Of which endline data were collected from 313 and 332 pregnant women in the intervention and control clusters, respectively. The intervention was started before 16 weeks of gestation and pregnant women in the intervention group attended 4 counseling sessions. Counseling was given at the participants' homes using a counseling guide with the core contents of the intervention. Leaflets with appropriate pictures and the core messages were given for women in the intervention arm. Women in the control group got the routine nutrition education given by the health care system. Data were collected using interviewer administered structured questionnaires and mid-upper arm circumference was measured using an adult MUAC tape. Descriptive statistics  and linear mixed-effects model were used to assess the intervention effect after adjusting for potential confounders.

RESULTS

After the intervention, the prevalence of undernutrition was 16.7% lower in the intervention group compared with the control arm (30.6% Vs 47.3%, P = < 0.001). Women in the intervention group showed significant improvement in nutritional status at the end of the trial than the control group (β = 0.615, p = < 0.001).

CONCLUSION

This study demonstrated that guided counseling using the health belief model and the theory of planned behavior was effective in improving the nutritional status of pregnant women. The results imply the need for the design of model and theory based nutrition counseling guidelines. The trial was registered in Clinical Trials.gov (NCT03627156).

Community-Based Services by Pharmacists: A Systematic Review of Cost-Utility Analyses

OBJECTIVES

To conduct a systematic review of cost-utility studies of community-based services provided by pharmacists and to examine their reporting and methodological quality.

METHODS

A comprehensive literature search was performed in February 2019 using a replicable search strategy in bibliographic databases MEDLINE, EMBASE, and the NHS Economic Evaluations Database from their inception onwards. Two reviewers independently screened the literature, abstracted data from full-text articles, and assessed reporting and methodological quality using the Consolidated Health Economic Evaluation Reporting Standards and Quality of Health Economic Studies checklists.

RESULTS

Twenty studies were included in this review, representing the healthcare systems of the United Kingdom, Spain, France, The Netherlands, Belgium, Italy, Canada, the United States of America, and Brazil. Descriptions of the context in which the studies were done, justification of data sources on patient and process outcomes, choices of model, and generalizability of study findings were often inadequately reported. Seven studies (35%) were deemed of high methodological quality, 11 studies (55%) of fair quality, and 2 studies (10%) of poor quality. In addition, various methodological issues related to the randomized controlled trials and observational studies used to generate effectiveness estimates were identified.

CONCLUSIONS

In view of the lack of standardized definitions of expanded services, heterogeneity in study objectives, settings, comparators, methodology, outcomes, and the variable quality of these studies, the value of these services (in terms of costs) compared with quality-adjusted survival remains inconclusive.

Well London Phase-1: results among adults of a cluster-randomised trial of a community engagement approach to improving health behaviours and mental well-being in deprived inner-city neighbourhoods

BACKGROUND

We report the main results, among adults, of a cluster-randomised-trial of Well London, a community-engagement programme promoting healthy eating, physical activity and mental well-being in deprived neighbourhoods. The hypothesis was that benefits would be neighbourhood-wide, and not restricted to intervention participants. The trial was part of a multicomponent process/outcome evaluation which included non-experimental components (self-reported behaviour change amongst participants, case studies and evaluations of individual projects) which suggested health, well-being and social benefits to participants.

METHODS

Twenty matched pairs of neighbourhoods in London were randomised to intervention/control condition. Primary outcomes (five portions fruit/vegetables/day; 5×30 m of moderate intensity physical activity/week, abnormal General Health Questionnaire (GHQ)-12 score and Warwick-Edinburgh Mental Well-being Scale (WEMWBS) score) were measured by postintervention questionnaire survey, among 3986 adults in a random sample of households across neighbourhoods.

RESULTS

There was no evidence of impact on primary outcomes: healthy eating (relative risk [RR] 1.04, 95% CI 0.93 to 1.17); physical activity (RR:1.01, 95% CI 0.88 to 1.16); abnormal GHQ12 (RR:1.15, 95% CI 0.84 to 1.61); WEMWBS (mean difference [MD]: -1.52, 95% CI -3.93 to 0.88). There was evidence of impact on some secondary outcomes: reducing unhealthy eating-score (MD: -0.14, 95% CI -0.02 to 0.27) and increased perception that people in the neighbourhood pulled together (RR: 1.92, 95% CI 1.12 to 3.29).

CONCLUSIONS

The trial findings do not provide evidence supporting the conclusion of non-experimental components of the evaluation that intervention improved health behaviours, well-being and social outcomes. Low participation rates and population churn likely compromised any impact of the intervention. Imprecise estimation of outcomes and sampling bias may also have influenced findings. There is a need for greater investment in refining such programmes before implementation; new methods to understand, longitudinally different pathways residents take through such interventions and their outcomes, and new theories of change that apply to each pathway.

Extended heart failure clinic follow-up in low-risk patients: a randomized clinical trial (NorthStar)

BACKGROUND

Outpatient follow-up in specialized heart failure clinics (HFCs) is recommended by current guidelines and implemented in most European countries, but the optimal duration of HFC programmes has not been established. Nor is it known whether all or only high-risk patients, e.g. identified by NT-proBNP, might benefit from an extended HFC follow-up.

METHODS AND RESULTS

In a multi-centre setting, we randomly assigned 921 clinically stable systolic heart failure (HF) outpatients on optimal medical therapy to undergo either an extended follow-up in the HFC (n = 461) or referral back to their general practitioner (GP) (n = 460). The primary composite endpoint was death or a cardiovascular admission. Secondary endpoints included mortality, an HF admission, quality of life, number of days admitted, and number of admissions. The median age of the patients was 69 years; 23% were females; the median left ventricular ejection fraction was 0.30; and the median NT-proBNP was 801 pg/mL; 89% were in NYHA class I-II. The median follow-up was 2.5 years. Time-to-event did not differ between groups (HFC vs. GP) (HR: 1.17, 95% CI: 0.95-1.45, P = 0.149). The two groups did not differ with respect to any of the secondary endpoints at the follow-up (P> 0.05 for all). In high-risk patients identified by NT-proBNP ≥1000 pg/mL, no benefit from HFC follow-up was found (P = 0.721).

CONCLUSION

Irrespective of the level of NT-proBNP stable HF patients on optimal medical therapy do not benefit from long-term follow-up in a specialized HFC in a publicly funded universal access healthcare system. Heart failure patients on optimal medical therapy with mild or moderate symptoms are safely managed by their personal GP.

TRIAL REGISTRATION

www.Centerwatch.com: 173491 (NorthStar).

The Well London program--a cluster randomized trial of community engagement for improving health behaviors and mental wellbeing: baseline survey results

BACKGROUND

The Well London program used community engagement, complemented by changes to the physical and social neighborhood environment, to improve physical activity levels, healthy eating, and mental wellbeing in the most deprived communities in London. The effectiveness of Well London is being evaluated in a pair-matched cluster randomized trial (CRT). The baseline survey data are reported here.

METHODS

The CRT involved 20 matched pairs of intervention and control communities (defined as UK census lower super output areas (LSOAs); ranked in the 11% most deprived LSOAs in London by the English Indices of Multiple Deprivation) across 20 London boroughs. The primary trial outcomes, sociodemographic information, and environmental neighbourhood characteristics were assessed in three quantitative components within the Well London CRT at baseline: a cross-sectional, interviewer-administered adult household survey; a self-completed, school-based adolescent questionnaire; a fieldworker completed neighborhood environmental audit. Baseline data collection occurred in 2008. Physical activity, healthy eating, and mental wellbeing were assessed using standardized, validated questionnaire tools. Multiple imputation was used to account for missing data in the outcomes and other variables in the adult and adolescent surveys.

RESULTS

There were 4,107 adults and 1,214 adolescent respondents in the baseline surveys. The intervention and control areas were broadly comparable with respect to the primary outcomes and key sociodemographic characteristics. The environmental characteristics of the intervention and control neighborhoods were broadly similar. There was greater between-cluster variation in the primary outcomes in the adult population compared to the adolescent population. Levels of healthy eating, smoking, and self-reported anxiety/depression were similar in the Well London adult population and the national Health Survey for England. Levels of physical activity were higher in the Well London adult population but this is likely to be due to the different measurement tools used in the two surveys.

CONCLUSIONS

Randomization of social interventions such as Well London is acceptable and feasible and in this study the intervention and control arms are well-balanced with respect to the primary outcomes and key sociodemographic characteristics. The matched design has improved the statistical efficiency of the study amongst adults but less so amongst adolescents. Follow-up data collection will be completed 2012.

Prevention of sexually transmitted infections in urban communities (Peru PREVEN): a multicomponent community-randomised controlled trial

BACKGROUND

Previous community-randomised trials of interventions to control sexually transmitted infections (STIs) have involved rural settings, were rarely multicomponent, and had varying results. We aimed to assess the effect of a multicomponent intervention on curable STIs in urban young adults and female sex workers (FSWs).

METHODS

In this community-randomised trial, baseline STI screening was done between August, and November, 2002, in random household samples of young adults (aged 18-29 years) and in FSWs in Peruvian cities with more than 50,000 inhabitants. Geographically separate cities were selected, matched into pairs, and randomly allocated to intervention or control groups with an S-PLUS program. Follow-up surveys of random samples were done after 2 years and 3 years. The intervention comprised four modalities: strengthened STI syndromic management by pharmacy workers and clinicians; mobile-team outreach to FSWs for STI screening and pathogen-specific treatment; periodic presumptive treatment of FSWs for trichomoniasis; and condom promotion for FSWs and the general population. Individuals in control cities received standard care. The composite primary endpoint was infection of young adults with Chlamydia trachomatis, Trichomonas vaginalis, or Neisseria gonorrhoeae, or syphilis seroreactivity. Laboratory workers and the data analyst were masked, but fieldworkers, the Peruvian study team, and participants in the outcome surveys were not. All analyses were done by intention to treat. This trial is registered, ISRCTN43722548.

FINDINGS

We did baseline surveys of 15,261 young adults in 24 Peruvian cities. Of those, 20 geographically separate cities were matched into pairs, in each of which one city was assigned to intervention and the other to standard of care. In the 2006 follow-up survey, data for the composite primary outcome were available for 12,930 young adults. We report a non-significant reduction in prevalence of STIs in young adults, adjusted for baseline prevalence, in intervention cities compared with control cities (relative risk 0·84, 95% CI 0·69-1·02; p=0·096). In subgroup analyses, significant reductions were noted in intervention cities in young adult women and FSWs.

INTERPRETATION

Syndromic management of STIs, mobile-team outreach to FSWs, presumptive treatment for trichomoniasis in FSWs, and condom promotion might reduce the composite prevalence of any of the four curable STIs investigated in this trial.

FUNDING

Wellcome Trust and Burroughs Wellcome Fund, National Institutes of Health, Center for AIDS Research, CIPRA, and USAID-Peru.

Standards for reporting randomized controlled trials in medical informatics: a systematic review of CONSORT adherence in RCTs on clinical decision support

INTRODUCTION

The Consolidated Standards for Reporting Trials (CONSORT) were published to standardize reporting and improve the quality of clinical trials. The objective of this study is to assess CONSORT adherence in randomized clinical trials (RCT) of disease specific clinical decision support (CDS).

METHODS

A systematic search was conducted of the Medline, EMBASE, and Cochrane databases. RCTs on CDS were assessed against CONSORT guidelines and the Jadad score.

RESULT

32 of 3784 papers identified in the primary search were included in the final review. 181 702 patients and 7315 physicians participated in the selected trials. Most trials were performed in primary care (22), including 897 general practitioner offices. RCTs assessing CDS for asthma (4), diabetes (4), and hyperlipidemia (3) were the most common. Thirteen CDS systems (40%) were implemented in electronic medical records, and 14 (43%) provided automatic alerts. CONSORT and Jadad scores were generally low; the mean CONSORT score was 30.75 (95% CI 27.0 to 34.5), median score 32, range 21-38. Fourteen trials (43%) did not clearly define the study objective, and 11 studies (34%) did not include a sample size calculation. Outcome measures were adequately identified and defined in 23 (71%) trials; adverse events or side effects were not reported in 20 trials (62%). Thirteen trials (40%) were of superior quality according to the Jadad score (≥3 points). Six trials (18%) reported on long-term implementation of CDS.

CONCLUSION

The overall quality of reporting RCTs was low. There is a need to develop standards for reporting RCTs in medical informatics.

Assessing the evaluability of complex public health interventions: five questions for researchers, funders, and policymakers

CONTEXT

Evidence to support government programs to improve public health often is weak. Recognition of this "knowledge gap" has led to calls for more and better evaluation, but decisions about priorities for evaluation also need to be addressed in regard to financial restraint.

METHODS

Using England's Healthy Community Challenge Fund as a case study, this article presents a set of questions to stimulate and structure debate among researchers, funders, and policymakers and help make decisions about evaluation within and between complex public health interventions as they evolve from initial concept to dissemination of full-scale intervention packages.

FINDINGS

This approach can be used to identify the types of knowledge that might be generated from any evaluation, given the strength of evidence available in response to each of five questions, and to support a more systematic consideration of resource allocation decisions, depending on the types of knowledge required.

CONCLUSIONS

The principles of this approach may be generalizable, and should be tested and refined for other complex public health and wider social interventions.