Reviewing evidence on complex social interventions: appraising implementation in systematic reviews of the health effects of organisational-level workplace interventions

Protocol: a realist review of user fee exemption policies for health services in Africa

Background Four years prior to the Millenium Development Goals (MDGs) deadline, low- and middle-income countries and international stakeholders are looking for evidence-based policies to improve access to healthcare for the most vulnerable populations. User fee exemption policies are one of the potential solutions. However, the evidence is disparate, and systematic reviews have failed to provide valuable lessons. The authors propose to produce an innovative synthesis of the available evidence on user fee exemption policies in Africa to feed the policy-making process. Methods The authors will carry out a realist review to answer the following research question: what are the outcomes of user fee exemption policies implemented in Africa? why do they produce such outcomes? and what contextual elements come into play? This type of review aims to understand how contextual elements influence the production of outcomes through the activation of specific mechanisms, in the form of context-mechanism-outcome configurations. The review will be conducted in five steps: (1) identifying with key stakeholders the mechanisms underlying user fee exemption policies to develop the analytical framework, (2) searching for and selecting primary data, (3) assessing the quality of evidence using the Mixed-Method Appraisal Tool, (4) extracting the data using the analytical framework and (5) synthesising the data in the form of context-mechanism-outcomes configurations. The output will be a middle-range theory specifying how user fee exemption policies work, for what populations and under what circumstances. Ethics and dissemination The two main target audiences are researchers who are looking for examples to implement a realist review, and policy-makers and international stakeholders looking for lessons learnt on user fee exemption. For the latter, a knowledge-sharing strategy involving local scientific and policy networks will be implemented. The study has been approved by the ethics committee of the CHUM Research Centre (CR-CHUM). It received funding from the Canadian Institutes of Health Research. The funders will not have any role in study design; collection, management, analysis, and interpretation of data; writing of the report and the decision to submit the report for publication, including who will have ultimate authority over each of these activities.

Process evaluation for the FEeding Support Team (FEST) randomised controlled feasibility trial of proactive and reactive telephone support for breastfeeding women living in disadvantaged areas

OBJECTIVE

To assess the feasibility, acceptability and fidelity of a feeding team intervention with an embedded randomised controlled trial of team-initiated (proactive) and woman-initiated (reactive) telephone support after hospital discharge.

DESIGN

Participatory approach to the design and implementation of a pilot trial embedded within a before-and-after study, with mixed-method process evaluation.

SETTING

A postnatal ward in Scotland.

SAMPLE

Women initiating breast feeding and living in disadvantaged areas.

METHODS

Quantitative data: telephone call log and workload diaries. Qualitative data: interviews with women (n=40) with follow-up (n=11) and staff (n=17); ward observations 2 weeks before and after the intervention; recorded telephone calls (n=16) and steering group meetings (n=9); trial case notes (n=69); open question in a telephone interview (n=372). The Framework approach to analysis was applied to mixed-method data.

MAIN OUTCOME MEASURES

Quantitative: telephone call characteristics (number, frequency, duration); workload activity. Qualitative: experiences and perspectives of women and staff.

RESULTS

A median of eight proactive calls per woman (n=35) with a median duration of 5 min occurred in the 14 days following hospital discharge. Only one of 34 control women initiated a call to the feeding team, with women undervaluing their own needs compared to others, and breast feeding as a reason to call. Proactive calls providing continuity of care increased women's confidence and were highly valued. Data demonstrated intervention fidelity for woman-centred care; however, observing an entire breast feed was not well implemented due to short hospital stays, ward routines and staff-team-woman communication issues. Staff pragmatically recognised that dedicated feeding teams help meet women's breastfeeding support needs in the context of overstretched and variable postnatal services.

CONCLUSIONS

Implementing and integrating the FEeding Support Team (FEST) trial within routine postnatal care was feasible and acceptable to women and staff from a research and practice perspective and shows promise for addressing health inequalities.

TRIAL REGISTRATION

ISRCTN27207603. The study protocol and final report is available on request.

Implementing the evidence for preventing falls among community-dwelling older people: a systematic review

PROBLEM AND OBJECTIVE

The translation of the evidence-base for preventing falls among community-dwelling older people into practice has been limited. This study systematically reviewed and synthesised the effectiveness of methods to implement falls prevention programmes with this population.

METHODS

Articles published between 1980 and May 2010 that evaluated the effects of an implementation strategy. No design restrictions were imposed. A narrative synthesis was undertaken.

RESULTS

15 studies were identified. Interventions that involved the active training of healthcare professionals improved implementation. The evidence around changing the way people who fall are managed within primary care practices, and, layperson, peer or community delivered models was mixed.

IMPACT ON INDUSTRY

Translating the evidence-base into practice involves changing the attitudes and behaviours of older people, healthcare professionals and organisations. However, there is a need for further evaluation on how this can be best achieved.

Implementation and adoption of mechanical patient lift equipment in the hospital setting: The importance of organizational and cultural factors

BACKGROUND

Work focused on understanding implementation and adoption of interventions designed to prevent patient-handling injuries in the hospital setting is lacking in the injury literature and may be more insightful than more traditional evaluation measures.

METHODS

Data from focus groups with health care workers were used to describe barriers and promoters of the adoption of patient lift equipment and a shift to a "minimal-manual lift environment" at two affiliated hospitals.

RESULTS

Several factors influencing the adoption of the lift equipment and patient-handling policy were noted: time, knowledge/ability, staffing, patient characteristics, and organizational and cultural aspects of work. The adoption process was complex, and considerable variability by hospital and across units was observed.

CONCLUSIONS

The use of qualitative data can enhance the understanding of factors that influence implementation and adoption of interventions designed to prevent patient-handling injuries among health care workers.

[Development, appraisal and synthesis of complex interventions - a methodological challenge]

BACKGROUND

Many medical interventions are of a complex nature. They comprise interdependent components differently interacting within various complex settings, e.g., stroke units. Appraising the efficacy, benefit and harm of complex interventions is far more difficult than appraising single interventions like specific drug treatments. Detailed methodological procedures for the development, appraisal and synthesis of complex interventions are increasingly discussed internationally.

METHODS

Systematic inventory of methodological guidance for the development and evaluation of complex interventions through searching scientific literature databases and web resources of international institutes. Review of the current state of discussion and suggestions for the development, appraisal and synthesis of complex interventions.

RESULTS

Five methodological guidance papers have been identified. In particular, the UK Medical Research Council's guidance offers a structured introduction to the development and evaluation of complex interventions in health care and medicine. Synthesis of complex interventions using customary methods of systematic reviews is not satisfying. Other approaches to synthesis like realist evaluation as well as statistical procedures exploring the active components of complex interventions have been discussed. However, post hoc analytical procedures could never adequately replace careful prospective development and exploration of complex interventions and interdependencies with contextual factors.

CONCLUSION

Complex interventions require multi-stage development, use of different methods, reporting on all developing phases and new approaches for synthesis. Presentation of the complete evidence on a specific complex intervention might be more useful than synthesis of a variety of different complex interventions by customarily applied methods of (meta-analytical) systematic review.

Proposed criteria for reporting the development and evaluation of complex interventions in healthcare (CReDECI): guideline development

Methods of research on complex interventions have received increasing awareness in nursing and health care science. The British Medical Research Council's (MRC) framework on the development and evaluation of complex interventions has been widely applied. It does, however, not specifically support authors to comprehensively and transparently report their complex study to ensure interpretation of study results and replicability of the intervention. So far, no reporting criteria for the development and evaluation of complex interventions have been published. Based on the updated MRC framework and corresponding methodological literature, a set of criteria on the reporting and evaluation of complex interventions has been developed and reviewed by experts in the field. As a result, a criteria list comprising 16 items has been created covering the first three stages of the MRC framework: (1) development; (2) feasibility and piloting; and (3) introduction of the intervention and evaluation. The list provides a minimum standard of criteria necessary to ensure high quality reporting of studies on the development and evaluation of complex interventions. In a final step, the reporting criteria on complex interventions have to pass a formal consensus process according to the methods recommended by the EQUATOR network.

Protocol for a systematic review of the effects of schools and school-environment interventions on health: evidence mapping and syntheses

BACKGROUND

Schools may have important effects on students' and staff's health. Rather than treating schools merely as sites for health education, 'school-environment' interventions treat schools as settings which influence health. Evidence concerning the effects of such interventions has not been recently synthesised.

METHODS/DESIGN

Systematic review aiming to map and synthesise evidence on what theories and conceptual frameworks are most commonly used to inform school-environment interventions or explain school-level influences on health; what effects school-environment interventions have on health/health inequalities; how feasible and acceptable are school-environment interventions; what effects other school-level factors have on health; and through what processes school-level influences affect health.We will examine interventions aiming to promote health by modifying schools' physical, social or cultural environment via actions focused on school policies and practices relating to education, pastoral care and other aspects of schools beyond merely providing health education. Participants are staff and students age 4-18 years.We will review published research unrestricted by language, year or source. Searching will involve electronic databases including Embase, ERIC, PubMed, PsycInfo and Social Science Citation Index using natural-language phrases plus reference/citation checking.Stage 1 will map studies descriptively by focus and methods. Stage 2 will involve additional inclusion criteria, quality assessment and data extraction undertaken by two reviewers in parallel. Evidence will be synthesised narratively and statistically where appropriate (undertaking subgroup analyses and meta-regression and where no significant heterogeneity of effect sizes is found, pooling these to calculate a final effect size).

DISCUSSION

We anticipate: finding a large number of studies missed by previous reviews; that non-intervention studies of school effects examine a greater breadth of determinants than are addressed by intervention studies; and that intervention effect estimates are greater than for school-based health curriculum interventions without school-environment components.

Workplace interventions for neck pain in workers

BACKGROUND

Musculoskeletal disorders are the most common cause of disability in many industrial countries. Recurrent and chronic pain accounts for a substantial portion of workers' absenteeism. Neck pain seems to be more prominent in the general population than previously known.

OBJECTIVES

To determine the effectiveness of workplace interventions (WIs) in adult workers with neck pain.

SEARCH STRATEGY

We searched: CENTRAL (The Cochrane Library 2009, issue 3), and MEDLINE, EMBASE, CINAHL, PsycINFO, ISI Web of Science, OTseeker, PEDro to July 2009, with no language limitations;screened reference lists; and contacted experts in the field.  

SELECTION CRITERIA

We included randomised controlled trials (RCT), in which at least 50% of the participants had neck pain at baseline and received interventions conducted at the workplace.

DATA COLLECTION AND ANALYSIS

Two review authors independently extracted data and assessed risk of bias. Authors were contacted for missing information. Since the interventions varied to a large extend, International Classification of Functioning, Disability and Health (ICF) terminology was used to classify the intervention components. This heterogeneity restricted pooling of data to only one meta-analysis of two studies.

MAIN RESULTS

We identified 1995 references and included10 RCTs (2745 workers). Two studies were assessed with low risk of bias. Most trials (N = 8) examined office workers. Few workers were sick-listed. Thus, WIs were seldom designed to improve return-to-work. Overall, there was low quality evidence that showed no significant differences between WIs and no intervention for pain prevalence or severity. If present, significant results in favour of WIs were not sustained across follow-up times. There was moderate quality evidence (1 study, 415 workers) that a four-component WI was significantly more effective in reducing sick leave in the intermediate-term (OR 0.56, 95% CI 0.33 to 0.95), but not in the short- (OR 0.83, 95% CI 0.52 to 1.34) or long-term (OR 1.28, 95% CI 0.73 to 2.26). These findings might be because only a small proportion of the workers were sick-listed.

AUTHORS' CONCLUSIONS

Overall, this review found low quality evidence that neither supported nor refuted the benefits of any specific WI for pain relief and moderate quality evidence that a multiple-component intervention reduced sickness absence in the intermediate-term, which was not sustained over time. Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate. There is an urgent need for high quality RCTs with well designed WIs.

Occupational musculoskeletal and mental health: Significance of rationalization and opportunities to create sustainable production systems - A systematic review

This literature review aims to identify occupational musculoskeletal and mental health effects of production system rationalization as well as organizational-level measures that may improve health outcome ("modifiers" in this review). A short review of the effect of ergonomic interventions is included as background and rationalization is discussed as a theoretical concept. Indicator variables for occupational musculoskeletal and mental health and related risk factors are presented. Variables with a generalized format were allowed in the literature searches (e.g., job satisfaction and absenteeism were accepted as risk factor and health indicator, respectively), suitable for the research fields of work sociology, organization science, human resource management (HRM) and economics research. One hundred and sixty-two studies of rationalization effects on health and risk factors and 72 organization-level modifier results were accepted into the final database. Entries were sorted by rationalization strategy and work life sector, and trends in outcome (positive, mixed, no effect, or negative effect on health and risk factors) were determined. Rationalizations have a dominant negative effect on health and risk factors (57% negative, 19% positive); the most negative effects were found for downsizing and restructuring rationalizations in general (71 studies negative, 13 positive) and for the health care sector in particular (36 studies negative, 2 positive). The rationalization strategy High Performance Work System (HPWS) was associated with the highest fraction positive outcome studies (6 of 10 studies). Other rationalization strategies (lean practices, parallel vs. serial production and mechanization level) reported intermediate results, in part dependent on work life sector, but also on the year when studies were carried out. Worker participation, resonant management style, information, support, group autonomy and procedural justice were modifiers with favourable influence on outcome. It is concluded that production system rationalization represents a pervasive work life intervention without a primary occupational health focus. It has considerable and mostly negative influence on worker health, but this can be reduced by attention to modifiers. The results create a basis for new priorities in ergonomic intervention research.

Implementation of antiretroviral therapy adherence interventions: a realist synthesis of evidence

AIM

This paper is a report of a synthesis of evidence on implementation of interventions to improve adherence to antiretroviral therapy.

BACKGROUND

Evidence on efficacy must be supplemented with evidence on how interventions were implemented in practice and on how that implementation varied across populations and settings.

DATA SOURCES

Sixty-one reports were reviewed of studies conducted in the United States of America in the period 2001 to December 2008. Fifty-two reports were included in the final analysis: 37 reporting the effects of interventions and 15 reporting intervention feasibility, acceptability, or fidelity.

REVIEW METHODS

An adaptation of Pawson's realist synthesis method was used, whereby a provisional explanatory model and associated list of propositions are developed from an initial review of literature. This model is successively refined to the point at which it best explains empirical findings from the reports reviewed.

RESULTS

The final explanatory model suggests that individuals with HIV will be more likely to enroll in interventions that protect their confidentiality, to attend when scheduling is responsive to their needs, and both to attend and continue with an intervention when they develop a strong, one-to-one relationship with the intervener. Participants who have limited prior experience with antiretroviral therapy will be more likely to continue with an intervention than those who are more experienced. Dropout rates are likely to be higher when interventions are integrated into existing delivery systems than when offered as stand-alone interventions.

CONCLUSION

The explanatory model developed in this study is intended to provide guidance to clinicians and researchers on the points in the implementation chain that require strengthening.

Intervention taxonomy (ITAX): describing essential features of interventions

OBJECTIVES

To identify key features of interventions that need to be considered in the design, execution, and reporting of interventions.

METHODS

Based on prior work on decomposing psychosocial and clinical interventions, current guidelines for describing interventions, and a review of a broad range of intervention studies, we developed a comprehensive intervention taxonomy.

RESULTS

Specific recommendations, rationales, and definitions of intervention delivery and content characteristics including mode, materials, location, schedule, scripting, and sensitivity to participant characteristics, interventionist characteristics, adaptability, implementation, content strategies, and mechanisms of action are provided.

CONCLUSIONS

Applying this taxonomy will advance intervention science by (a) improving intervention designs, (b) enhancing replication and follow-up of intervention studies, (c) facilitating systematic exploration of the efficacy and effectiveness of intervention components through cross-study analysis, and (d) informing decisions about the feasibility of implementation in broader community settings.

Flexible working conditions and their effects on employee health and wellbeing

BACKGROUND

Flexible working conditions are increasingly popular in developed countries but the effects on employee health and wellbeing are largely unknown.

OBJECTIVES

To evaluate the effects (benefits and harms) of flexible working interventions on the physical, mental and general health and wellbeing of employees and their families.

SEARCH STRATEGY

Our searches (July 2009) covered 12 databases including the Cochrane Public Health Group Specialised Register, CENTRAL; MEDLINE; EMBASE; CINAHL; PsycINFO; Social Science Citation Index; ASSIA; IBSS; Sociological Abstracts; and ABI/Inform. We also searched relevant websites, handsearched key journals, searched bibliographies and contacted study authors and key experts.

SELECTION CRITERIA

Randomised controlled trials (RCT), interrupted time series and controlled before and after studies (CBA), which examined the effects of flexible working interventions on employee health and wellbeing. We excluded studies assessing outcomes for less than six months and extracted outcomes relating to physical, mental and general health/ill health measured using a validated instrument. We also extracted secondary outcomes (including sickness absence, health service usage, behavioural changes, accidents, work-life balance, quality of life, health and wellbeing of children, family members and co-workers) if reported alongside at least one primary outcome.

DATA COLLECTION AND ANALYSIS

Two experienced review authors conducted data extraction and quality appraisal. We undertook a narrative synthesis as there was substantial heterogeneity between studies.

MAIN RESULTS

Ten studies fulfilled the inclusion criteria. Six CBA studies reported on interventions relating to temporal flexibility: self-scheduling of shift work (n = 4), flexitime (n = 1) and overtime (n = 1). The remaining four CBA studies evaluated a form of contractual flexibility: partial/gradual retirement (n = 2), involuntary part-time work (n = 1) and fixed-term contract (n = 1). The studies retrieved had a number of methodological limitations including short follow-up periods, risk of selection bias and reliance on largely self-reported outcome data. Four CBA studies on self-scheduling of shifts and one CBA study on gradual/partial retirement reported statistically significant improvements in either primary outcomes (including systolic blood pressure and heart rate; tiredness; mental health, sleep duration, sleep quality and alertness; self-rated health status) or secondary health outcomes (co-workers social support and sense of community) and no ill health effects were reported. Flexitime was shown not to have significant effects on self-reported physiological and psychological health outcomes. Similarly, when comparing individuals working overtime with those who did not the odds of ill health effects were not significantly higher in the intervention group at follow up. The effects of contractual flexibility on self-reported health (with the exception of gradual/partial retirement, which when controlled by employees improved health outcomes) were either equivocal or negative. No studies differentiated results by socio-economic status, although one study did compare findings by gender but found no differential effect on self-reported health outcomes.

AUTHORS' CONCLUSIONS

The findings of this review tentatively suggest that flexible working interventions that increase worker control and choice (such as self-scheduling or gradual/partial retirement) are likely to have a positive effect on health outcomes. In contrast, interventions that were motivated or dictated by organisational interests, such as fixed-term contract and involuntary part-time employment, found equivocal or negative health effects. Given the partial and methodologically limited evidence base these findings should be interpreted with caution. Moreover, there is a clear need for well-designed intervention studies to delineate the impact of flexible working conditions on health, wellbeing and health inequalities.

Community-based services for homeless adults experiencing concurrent mental health and substance use disorders: a realist approach to synthesizing evidence

Consultations with community-based service providers in Toronto identified a lack of strong research evidence about successful community-based interventions that address the needs of homeless clients experiencing concurrent mental health and substance use disorders. We undertook a collaborative research effort between academic-based and community-based partners to conduct a systematic evidence synthesis drawing heavily from Pawson's realist review methodology to focus on both whether programs are successful and why and how they lead to improved outcomes. We examined scholarly and nonscholarly literature to explore program approaches and program elements that lead to improvements in mental health and substance use disorders among homeless individuals with concurrent disorders (CD). Information related to program contexts, elements, and successes and failures were extracted and further supplemented by key informant interviews and author communication regarding reviewed published studies. From the ten programs that we reviewed, we identified six important and promising program strategies that reduce mental health and, to a far lesser degree, substance use problems: client choice in treatment decision-making, positive interpersonal relationships between client and provider, assertive community treatment approaches, providing supportive housing, providing supports for instrumental needs, and nonrestrictive program approaches. These promising program strategies function, in part, by promoting and supporting autonomy among homeless adults experiencing CD. Our realist informed review is a useful methodology for synthesizing complex programming information on community-based interventions.