The three stages of building and testing mid-level theories in a realist RCT: a theoretical and methodological case-example

Integrating qualitative research within a clinical trials unit: developing strategies and understanding their implementation in contexts

BACKGROUND/AIMS

The value of using qualitative methods within clinical trials is widely recognised. How qualitative research is integrated within trials units to achieve this is less clear. This paper describes the process through which qualitative research has been integrated within Cardiff University's Centre for Trials Research (CTR) in Wales, UK. We highlight facilitators of, and challenges to, integration.

METHODS

We held group discussions on the work of the Qualitative Research Group (QRG) within CTR. The content of these discussions, materials for a presentation in CTR, and documents relating to the development of the QRG were interpreted at a workshop attended by group members. Normalisation Process Theory (NPT) was used to structure analysis. A writing group prepared a document for input from members of CTR, forming the basis of this paper.

RESULTS

Actions to integrate qualitative research comprised: its inclusion in Centre strategies; formation of a QRG with dedicated funding/roles; embedding of qualitative research within operating systems; capacity building/training; monitoring opportunities to include qualitative methods in studies; maximising the quality of qualitative research and developing methodological innovation. Facilitators of these actions included: the influence of the broader methodological landscape within trial/study design and its promotion of the value of qualitative research; and close physical proximity of CTR qualitative staff/students allowing sharing of methodological approaches. Introduction of innovative qualitative methods generated interest among other staff groups. Challenges included: pressure to under-resource qualitative components of research, preference for a statistical stance historically in some research areas and funding structures, and difficulties faced by qualitative researchers carving out individual academic profiles when working across trials/studies.

CONCLUSIONS

Given that CTUs are pivotal to the design and conduct of RCTs and related study types across multiple disciplines, integrating qualitative research into trials units is crucial if its contribution is to be fully realised. We have made explicit one trials unit's experience of embedding qualitative research and present this to open dialogue on ways to operationalise and optimise qualitative research in trials. NPT provides a valuable framework with which to theorise these processes, including the importance of sense-making and legitimisation when introducing new practices within organisations.

Development of a program theory for osteoporosis patient education in Denmark: a qualitative study based on realist evaluation

BACKGROUND

Osteoporosis patient education is offered in many countries worldwide. When evaluating complex interventions like these, it is important to understand how and why the intervention leads to effects. This study aimed to develop a program theory of osteoporosis patient education in Danish municipalities with a focus on examining the mechanisms of change i.e. what is about the programs that generate change.

METHODS

The program theory was developed in an iterative process. The initial draft was based on a previous published systematic review, and subsequently the draft was continually refined based on findings from observations (10 h during osteoporosis patient education) and interviews (individual interviews with six employees in municipalities and three health professionals at hospitals, as well as four focus group interviews with participants in patient education (in total 27 informants)). The transcribed interviews were analyzed using thematic analysis and with inspiration from realist evaluation the mechanisms as well as the contextual factors and outcomes were examined.

RESULTS

Based on this qualitative study we developed a program theory of osteoporosis patient education and identified four mechanisms: motivation, recognizability, reassurance, and peer reflection. For each mechanism we examined how contextual factors activated the mechanism as well as which outcomes were achieved. For instance, the participants' motivation is activated when they meet in groups, and thereafter outcomes such as more physical activity may be achieved. Recognizability is activated by the participants' course of disease, which may lead to better ergonomic habits. Reassurance may result in more physical activity, and this mechanism is activated in newly diagnosed participants without previous fractures. Peer reflection is activated when the participants meet in groups, and the outcome healthier diet may be achieved.

CONCLUSIONS

We developed a program theory and examined how and why osteoporosis patient education is likely to be effective. Understanding these prerequisites is important for future implementation and evaluation of osteoporosis patient education.

Guidelines for evaluating and reporting social isolation and loneliness interventions

Given the unpleasant nature of social isolation and loneliness (SIL) and their negative effects on health and wellbeing, interventions are needed. However, persistent issues in the design, evaluation, and reporting of SIL interventions preclude conclusive evidence and commentary on the effectiveness of SIL interventions. Here, we propose guidelines for evaluating SIL interventions, firstly by operationalising them into two categories: (1) interventions aiming to reduce SIL as a primary outcome and (2) interventions aiming to improve non-SIL outcomes in the lives of individuals experiencing SIL. Secondly, we evaluate instruments for measuring SIL and research designs for studying intervention effectiveness. Thirdly, guidelines for reporting information about the intervention, study design, results, and discussion in SIL intervention studies are presented. These guidelines will help researchers to better and more consistently report on SIL interventions and improve comparability of SIL interventions, ultimately contributing to the improvement of interventions and to the mitigation of SIL.

Reducing LGBTQ+ adolescent mental health inequalities: a realist review of school-based interventions

BACKGROUND

LGBTQ+ young people have elevated rates of poor mental health in comparison to their cisgender heterosexual peers. School environment is a key risk factor and consistently associated with negative mental health outcomes for LGBTQ+ adolescents.

AIMS

To examine how, why, for whom and in what context school-based interventions prevent or reduce mental health problems in LGBTQ+ adolescents.

METHODS

A realist review methodology was utilised and focused on all types of school-based interventions and study designs. A Youth Advisory Group were part of the research team. Multiple search strategies were used to locate relevant evidence. Studies were subject to inclusion criteria and quality appraisal, and included studies were synthesised to produce a programme theory. Seventeen studies were included in the review.

RESULTS

Eight intervention components were necessary to address LGBTQ+ pupils mental health: affirmative visual displays; external signposting to LGBTQ+ support; stand-alone input; school-based LGBTQ support groups; curriculum-based delivery; staff training; inclusion policies; trusted adult. Few school-based interventions for this population group were identified.

CONCLUSIONS

The programme theory indicates that "to work" school-based interventions must have a "whole-school" approach that addresses specifically the dominant cis-heteronormative school environment and hence the marginalisation, silence, and victimisation that LGBTQ+ pupils can experience.

Integration of evidence into Theory of Change frameworks in the healthcare sector: A rapid systematic review

BACKGROUND

Theory of Change (ToC) has become an established approach to design and evaluate interventions. While ToC should-in line with the growing international focus on evidence-informed health decision-making-consider explicit approaches to incorporate evidence, there is limited guidance on how this should be done. This rapid review aims to identify and synthesize the available literature on how to systematically use research evidence when developing or adapting ToCs in the health sector.

METHODS

A rapid review methodology using a systematic approach, was designed. Eight electronic databases were consulted to search for peer-reviewed and gray publications detailing tools, methods, and recommendations promoting the systematic integration of research evidence in ToCs. The included studies were compared, and the findings summarized qualitatively into themes to identify key principles, stages, and procedures, guiding the systematic integration of research evidence when developing or revising a ToC.

RESULTS

This review included 18 studies. The main sources from which evidence was retrieved in the ToC development process were institutional data, literature searches, and stakeholder consultation. There was a variety of ways of finding and using evidence in ToC. Firstly, the review provided an overview of existing definitions of ToC, methods applied in ToC development and the related ToC stages. Secondly, a typology of 7 stages relevant for evidence integration into ToCs was developed, outlining the types of evidence and research methods the included studies applied for each of the proposed stages.

CONCLUSION

This rapid review adds to the existing literature in two ways. First, it provides an up-to-date and comprehensive review of the existing methods for incorporating evidence into ToC development in the health sector. Second, it offers a new typology guiding any future endeavors of incorporating evidence into ToCs.

[Evaluating population health interventions : The contributions of theory-driven evaluations]

Population health intervention research has been characterized by the deployment of scientific methods designed to produce knowledge on policy and program interventions directly or indirectly involving the health sector, and potentially affecting population health. The proposed solutions encompass a multitude of interventions of variable types, scale, focus and implementation, rendering them particularly complex and difficult to understand. This complexity raises major conceptual and methodological issues because in reality, we assess not an intervention, but rather a set of interventional elements interacting with those specific to the context in which they appear. It is the interactions that produce effects, to the extent that it makes little sense to speak of an intervention, and more sense to consider an interventional system defined by the interactions. To grasp the numerous elements brought into play, it behooves us to amalgamate evaluation paradigms and approaches. In a precise context, theory-driven evaluations are of pronounced interest. This article presents the main principles of this type of evaluation by focusing on its capacity to shed light on the stakes involved in intervention/context interplay, and by putting forward conclusions transferable to population health research.

Theoretically derived interventions aimed at improving appropriate polypharmacy in primary care: A systematic review

Background

Polypharmacy (the use of multiple medications) is common in older patients and achieving a balance between appropriate and inappropriate polypharmacy is a challenge routinely faced by prescribers. It is recommended to incorporate the use of theory when developing complex interventions, but it is not known if theoretically derived interventions aimed at improving appropriate polypharmacy are effective.

Objective

This systematic review aimed to establish the overall effectiveness of theoretically derived interventions on improving appropriate polypharmacy and to investigate the degree to which theory informed intervention design.

Methods

Seven electronic databases were searched from inception to August 2021 including hand-searching of reference lists. Interventions developed using a theory, involving the use of a validated tool to assess prescribing, delivered in primary care to participants with a mean age of ≥65 years and prescribed ≥four medications, were included. Data was extracted independently by two reviewers. The Theory Coding Scheme (TCS) was applied to evaluate the use of theory; Risk of Bias (RoB) was assessed using the Cochrane RoB 2.0 tool.

Results

Two studies, one feasibility study and one randomised controlled trial (RCT) were included, and therefore overall effectiveness of the theoretically derived intervention could not be assessed. Theory used in development included the Theoretical Domains Framework and Reason's system-based risk management theory. The RCT was rated to have a high RoB. Based on the TCS, neither study used theory completely.

Conclusion

The effectiveness of theoretically derived interventions to improve appropriate polypharmacy in primary care could not be determined due to the small number of studies and their heterogeneity. Further incorporation of theory into intervention development is required to understand the effectiveness of this approach.

Prospero registration: CRD42020157175.

Are realist randomised controlled trials possible? A reflection on the INCLUSIVE evaluation of a whole-school, bullying-prevention intervention

We previously proposed that realist randomised controlled trials could be used to evaluate how, for whom and under what conditions complex interventions can be used to activate mechanisms to improve health. While this idea was accepted by some, it was also met with resistance, particularly from some realist evaluators who believe that trials are inextricably positivist and dependent on constant conjunctions to understand causation, and that realist trials are unfeasible because participants and contexts will be insufficiently diverse to enable the testing of context-mechanism-outcome configurations. In this paper, we reflect on analyses of qualitative and quantitative data from the Initiating Change Locally in Bullying and Aggression through the School Environment (INCLSUIVE) trial, and whether these are useful and aligned with realism. We summarise the concerns expressed by realists and reflect on the philosophical and practical challenges that we encountered and whether or not they are related to the trial's design. Finally, we reflect on the trial's weaknesses and highlight areas that future researchers might consider when running realist trials. We conclude that realist randomised controlled trials are philosophically coherent, practically feasible, and can produce nuanced findings.

Participants perceptions of Essential Coaching for Every Mother, a Canadian text-message postpartum program: Process evaluation of a randomized controlled trial (Preprint)

Background

“Essential Coaching for Every Mother” is a Canadian text message–based program that sends daily messages to mothers for 6 weeks after they give birth. There is a need to explore the program’s effectiveness in terms of the participants’ experience to guide refinement and modification.

Objective

This study aimed to describe the process evaluation of the Essential Coaching for Every Mother randomized controlled trial through an evaluation of the research implementation extent and quality.

Methods

Participants were recruited from Nova Scotia, Canada, between January 5 and August 1, 2021. Enrolled participants were randomized into the intervention or control group. Participants randomized to the intervention group received standard care along with the Essential Coaching for Every Mother program’s text messages related to newborn and maternal care for the first 6 weeks after giving birth, while the control group received standard care. Usage data were collected from the SMS text message program used, and participants completed web-based questionnaires at 6 weeks after birth. Quantitative data and qualitative responses to open-ended questions were used to triangulate findings. Quantitative data were summarized using means, SDs, and percentages, as appropriate, while qualitative data were analyzed using thematic analysis.

Results

Of the 295 unique initial contacts, 150 mothers were eligible and completed the baseline survey to be enrolled in the study (intervention, n=78; control, n=72). Of those randomized into the intervention group, 75 (96%) completed the 6-week follow-up survey to provide feedback on the program. In total, 48 (62%) intervention participants received all messages as designed in the Essential Coaching for Every Mother program, with participants who enrolled late missing on average 4.7 (range 1-12) messages. Intervention participants reported an 89% satisfaction rate with the program, and 100% of participants would recommend the program to other new mothers. Participants liked how the program made them feel, the format, appropriate timing of messages, and content while disliking the frequency of messages and gaps in content. Participants also provided suggestions for future improvement.

Conclusions

Our process evaluation has provided a comprehensive understanding of interest in the program as well as identified preference for program components. The findings of this study will be used to update future iterations of the Essential Coaching for Every Mother program.

Trial Registration

ClincalTrials.gov NCT04730570; https://clinicaltrials.gov/ct2/show/NCT04730570

Developing curricular-content and systems-related impact indicators for intellectual disability awareness training for acute hospital settings: A modified International Delphi Survey

AIM

To identify, and reach consensus on, curricular-content and delivery methods, as well as ways to maximize the impact of intellectual disability awareness training programmes in acute hospital settings.

BACKGROUND

With the continuing evidence of avoidable deaths and unwarranted variations in the quality of care to people with an intellectual disability in acute hospitals, it could be purported that current training provided to hospital staff appears to be making a minimal difference in the care provided to this population.

DESIGN

A two-round modified Delphi survey was conducted between June 2020-January 2021.

METHODS

International experts from primary healthcare and hospital settings, and intellectual disability health fields participated in the survey. Initial curricular-content items were developed from the literature, and based on the combined clinical and academic experience base of the authors. Items were evaluated in terms of agreement/consensus, importance and stability of responses. There were 57 expert responses in Round 1 and 45 in Round 2.

RESULTS

The consensus was reached with regard to 55 of 65 curricular-content indicators relating to Aims, Design, Content and Delivery. Ten curricular-content indicators failed to be agreed on relating to the mode of training delivery. With regard to systems-related impact indicators, 28 out of 31 reached consensus. The expert panel identified and agreed on seven system barriers that could obstruct the successful implementation of the awareness training programmes in acute hospital settings.

CONCLUSIONS

This is the first international Delphi survey to agree on curricular-content and identify systems-related facilitators for intellectual disability awareness training. Potential system barriers have been highlighted which could be addressed by systemic improvement. Implications for developing, and robustly testing the efficacy of, intellectual disability awareness training programmes are discussed, as are the implications for other cognitively impaired populations.

IMPACT

In order to maximize the impact, investment in acute hospital staff education will need to be accompanied by wider changes to systems and structures concerning the governance of service provision for people with an intellectual disability.

Moderated mediation analyses to assess intervention mechanisms for impacts on victimisation, psycho-social problems and mental wellbeing: Evidence from the INCLUSIVE realist randomized trial

Realist evaluations aim to evaluate interventions by understanding the mechanisms they trigger, assessing not merely what works but what works for whom, under what conditions, and how. Significant disagreement in the literature exists as to whether randomized trials can be used as a tool for realist evaluation. INCLUSIVE, which was the first realist randomized trial explicitly designed as such, evaluated the impact of Learning Together, a school-based intervention for students aged 12-15 that included restorative practice, on bullying victimisation, mental wellbeing and psychological problems. Drawing on hypotheses generated through qualitative research, this analysis tested if school belonging was a mediator of intervention effects, and in which contexts. We estimated a series of fully longitudinal multilevel moderated mediation models including intervention allocation, student reports of school belonging at 24 months and victimisation and wellbeing outcomes at 36 months, and stratified on the basis of whether, at baseline, schools were: a) rated 'outstanding' for leadership, b) below the median for average levels of victimisation, and c) above the median on school inclusivity. Findings suggested that in unstratified models, belonging was not a mediator for any outcome; but in each of the strata defined above, belonging was a significant mediator at the student level. However, in the strata where belonging was not a mediator, the intervention still had a significant effect on each outcome. Analyses point to a strong but conditional role for belonging as a mediator of intervention pathways; in schools where belonging was not a mediator (e.g. above-median victimisation levels), other mechanisms may have been activated. This is consistent with a realist understanding of context-mechanism linkages generating outcomes. Our analyses suggest that realist evaluations can be pursued within randomized trials and that such analyses can offer more nuanced evidence regarding in which contexts interventions might effectively be implemented.

An exploration of lifestyle/obesity programmes for adults with intellectual disabilities through a realist lens: Impact of a 'context, mechanism and outcome' evaluation

BACKGROUND

Obesity is higher in people with intellectual disabilities.

AIMS

There are two aims of this explorative paper. Firstly, using a realist lens, to go beyond 'what works' and examine the 'context, mechanisms and outcomes' (CMO) of lifestyle/obesity programmes for this population. Second, using a logic model framework to inform how these programmes could be implemented within practice.

METHOD

We explored six-review papers and the individual lifestyle/obesity programmes that these papers reviewed using the CMO framework.

RESULTS

There were few theoretically underpinned, multi-component programmes that were effective in the short to long-term and many failed to explore the 'context and mechanisms'. We developed a logic model and engaged in two co-production workshops to refine this model.

DISCUSSION

Using a realist approach, programmes need to be underpinned by both individual and systems change theories, be multi-component, have a closer understanding of the interplay of the 'context and mechanisms', and co-designed using a logic model framework.

A realist evaluation approach to explaining the role of context in the impact of a complex eHealth intervention for improving prevention of cardiovascular disease

Background

Reduction of cardiovascular disease (CVD) is a worldwide health priority and innovative uses of technology-based interventions may assist patients with improving prevention behaviours. Targeting these interventions to recipients most likely to benefit requires understanding how contexts of use influence responsiveness to the intervention, and how this interaction favours or discourages health behaviour. Using a realist evaluation approach, the aim of this study was to examine the contextual factors influencing behaviour change within a multi-feature eHealth intervention with personalised data integration from the primary care electronic health record (EHR).

Methods

Realist evaluation of qualitative data from the Consumer Navigation of Electronic Cardiovascular Tools (CONNECT) randomised trial (N = 934). Thirty-six participants from the intervention group (N = 486) who had completed 12 months of study follow-up were interviewed. Coding of transcripts was structured around configurations of contexts, mechanisms, and outcomes of intervention use. Contextual narratives were derived from thematic analysis of the interviews.

Results

Mechanisms favouring positive health behaviour occurred when participants responded to four interactive features of the intervention. Facilitating mechanisms included greater cognitive engagement whereby participants perceived value and benefit, and felt motivated, confident and incentivised. Participants moved from being unconcerned (or unaware) to more task-oriented engagement with personal CVD risk profile and prevention. Increased personalisation occurred when modifiable CVD risk factors became relatable to lifestyle behaviour; and experiences of feeling greater agency/self-efficacy emerged. Use and non-use of the intervention were influenced by four overarching narratives within the individual’s micro-level and meso-level environments: illness experiences; receptiveness to risk and prevention information; history of the doctor-patient relationship; and relationship with technology.

Conclusions

Intervention-context interactions are central to understanding how change mechanisms activate within complex interventions to exert their impact on recipients. Intervention use and non-use were context-dependent, underscoring the need for further research to target eHealth innovations to those most likely to benefit.

Moderators of treatment effects in a child maltreatment prevention programme in South Africa

BACKGROUND

Previous research has found mixed results on whether the most disadvantaged families benefit as much as less disadvantaged families from parenting interventions designed to reduce child maltreatment, and little in known in low-income settings.

OBJECTIVE

In this study, we test the effects of child, caregiver, household, and community characteristics as treatment moderators of intervention outcomes - child maltreatment and parenting practices. We test characteristics previously examined elsewhere as well as factors relevant to the South African context.

PARTICIPANTS AND SETTING

This analysis includes adolescents (ages 10-18) and their caregivers (N = 552 pairs) who participated in a randomised trial of a parenting programme in the Eastern Cape Province of South Africa.

METHODS

Data from the caregiver and adolescent standardised questionnaires collected at baseline, post-test (1-month post-intervention), and follow-up (5-9 months) were analysed using longitudinal multilevel analyses. We tested seven hypothesised moderators for each of the primary outcomes through interactions of treatment effect with baseline moderators.

RESULTS

No moderator effects were statistically significant after correcting for multiple comparisons testing. Hence, in line with several recent studies examining moderation effects in parenting programmes, our study suggests that parenting interventions aiming to reduce child maltreatment and promote parenting skills in low- and middle-income countries may be similarly effective for families facing various levels of economic, social, and health risk factors.

CONCLUSIONS

It may be useful to explicitly power trials for testing moderator effects, study different types of moderators and use person-centred analyses to further understand variations in treatment effects.

What works, for whom and under what circumstances? Using realist methodology to evaluate complex interventions in nursing: A scoping review

BACKGROUND

The Medical Research Council's (MRC) framework recommends basing every evaluation of a complex intervention on a theory that explains WHY an intervention works instead of focusing exclusively on the identification IF an intervention works. Theory-based evaluation approaches such as realist evaluations are promising in this respect. The aim of this scoping review is to examine current scientific literature with regard to the use of realist methodology in studies that evaluate complex nursing interventions. We also seek to answer the question whether realist evaluation studies of complex nursing interventions are embedded in the Medical Research Council's framework.

METHODS

A systematic database search in MEDLINE and CINAHL supplemented with snowballing techniques and a hand search in relevant nursing journals were performed between January and February 2018 and updated in July 2019. Both authors independently screened the title / abstract of studies that were initially identified and appraised the full texts if inclusion criteria were fulfilled. Inclusion criteria were the explicit use of realist methodology and a focus on the development or evaluation of complex nursing interventions. Results were summarized narratively.

RESULTS

A total of 28 articles from 22 studies, published between 2010 and 2019, were included. The majority of the studies included originate in the UK. Realist methodology was used by a variety of nursing disciplines, including midwifery, community / public health, critical care, palliative care, gerontological nursing, psychiatric nursing / mental health, and infection control. Study designs included case studies and realist reviews; two studies were conducted alongside a randomized controlled trial. All studies included used more than one method to develop their results; the majority of methods were qualitative by nature. Thirteen of the studies included aimed to develop a theory about what works, for whom and under what circumstances regarding the investigated complex nursing intervention. One study was clearly embedded in the Medical Research Council's framework.

CONCLUSION

Realist evaluation broadens the methodologies that have traditionally been used in nursing. Conceptual clarity on its principal tenets is still needed, as well as the methods used to assess them. Nursing scientists should be more critical when using novel methodology and contribute to academic debates about limitations. Embedding realist evaluations into the Medical Research Council's framework is in its infancy in nursing and needs more contributions in order to develop a common academic position within the profession. Tweetable abstract Realist methodology in nursing - review of the use, methods, limitations and embedding into the Medical Research Council's framework.

Process evaluation of a community mobilization intervention for preventing men's partner violence use in peri-urban South Africa

Intimate partner violence (IPV) is experienced by one-third of women globally, yet few programs attempt to shift men's IPV perpetration. Community mobilization is a potential strategy for reducing men's IPV perpetration, but this has rarely been examined globally. We conducted a mixed-methods process evaluation alongside a trial testing community mobilization in peri-urban South Africa. We used in-depth interviews (n=114), participant observation (160 h), and monitoring and evaluation data to assess program delivery. Qualitative data (verbatim transcripts and observation notes) were managed in Dedoose using thematic coding and quantitative data were descriptively analyzed using Stata13. We learned that outreach elements of community mobilization were implemented with high fidelity, but that critical reflection and local advocacy were difficult to achieve. The context of a peri-urban settlement (characterized by poor infrastructure, migrancy, low education, social marginalization, and high levels of violence) severely limited intervention delivery, as did lack of institutional support for staff and activist volunteers. That community mobilization was poorly implemented may explain null trial findings; in the larger trial, the intervention failed to measurably reduce men's IPV perpetration. Designing community mobilization for resource-constrained settings may require additional financial, infrastructural, organizational, or political support to effectively engage community members and reduce IPV.

The Effectiveness of Digital Interventions for Psychological Well-Being in the Workplace: A Systematic Review Protocol

OBJECTIVE

Psychological well-being has been associated with desirable individual and organisational outcomes. This systematic review aims to assess the effectiveness of digital interventions for the improvement of psychological well-being and/or the prevention/management of poor mental well-being in the workplace.

METHODS

This review protocol is registered in PROSPERO (CRD42019142428). Scientific databases including MEDLINE, Web of Science, CINAHL, PsycINFO, Cochrane Register of Controlled Trials (CENTRAL) and EMBASE will be searched for relevant studies published between January 1990 and July 2019. Studies will be included if they report specific primary and secondary outcomes of digital interventions delivered to adults in the workplace for the improvement of their psychological wellbeing and/or the prevention/management of poor mental well-being and were published in English. Following screening of titles and abstracts, full texts of potentially eligible papers will be screened in duplicate to identify studies that assess the effectiveness of those digital interventions. Discrepancies will be resolved through consensus or by consulting a third reviewer. An integrated narrative synthesis will assess included studies' findings, and a meta-analysis will be performed if included studies appear to be homogeneous. The "Cochrane Collaboration's Risk of Bias" tool and the JBI (Joanna Briggs Institute) Critical Appraisal Checklist for Quasi-Experimental Studies will be used to appraise included studies.

CONCLUSION

The results of this work will provide recommendations on the use of digital interventions for the promotion of psychological well-being at work. It will also guide the development of future workplace digital interventions and subsequent primary research in this field.

Designing an intervention to improve sexual health service use among university undergraduate students: a mixed methods study guided by the behaviour change wheel

Introduction

University undergraduate students are within the population at highest risk for acquiring sexually transmitted infections, unplanned pregnancy, and other negative health outcomes. Despite the availability of sexual health services at university health centres, many students delay or avoid seeking care. In this study, we describe how the Behaviour Change Wheel was used as a systematic approach to design an intervention to improve sexual health service use among university undergraduate students.

Methods

This paper describes the intervention development phase of a three-phased, sequential explanatory mixed methods study. Phases one and two included a quantitative and qualitative study that aimed to better understand students’ use of sexual health services. In phase three, we followed the Behaviour Change Wheel to integrate the quantitative and qualitative findings and conduct stakeholder consultation meetings to select intervention strategies, including intervention functions and behaviour change techniques.

Results

Key linkages between opportunity and motivation were found to influence students’ access of sexual health services. Stakeholders identified six intervention functions (education, environmental restructuring, enablement, modelling, persuasion, and incentivization) and 15 behaviour change techniques (information about health consequences, information about social and environmental consequences, feedback on behaviour, feedback on outcomes of behaviour, prompts/cues, self-monitoring of behaviour, adding objects to the environment, goal setting, problem solving, action planning, restructuring the social environment, restructuring the physical environment, demonstration of the behaviour, social support, credible source) as relevant to include in a toolbox of intervention strategies to improve sexual health service use.

Conclusions

This study details the use of the Behaviour Change Wheel to develop an intervention aimed at improving university students’ use of sexual health services. The Behaviour Change Wheel provided a comprehensive framework for integrating multiple sources of data to inform the selection of intervention strategies. Stakeholders can use these strategies to design and implement sexual health service interventions that are feasible within the context of their health centre. Future research is needed to test the effectiveness of the strategies at changing university students’ sexual health behaviour.

Harnessing the power of theorising in implementation science

Theories occupy different positions in the scientific circle of enquiry as they vary in scope, abstraction, and complexity. Mid-range theories play a crucial bridging role between raw empirical observations and all-encompassing grand-theoretical schemes. A shift of perspective from ‘theories’ as products to ‘theorising’ as a process can enable empirical researchers to capitalise on the two-way relationships between empirical data and different levels of theory and contribute to the advancement of knowledge. This can be facilitated by embracing theoretically informative (in addition to merely theoretically informed) research, developing mechanism-based explanations, and broadening the repertoire of grand-theoretical orientations.

Implementation, Mechanisms of Effect and Context of an Integrated Care Intervention for Vulnerable Families in Central Sydney Australia: A Research and Evaluation Protocol

Introduction:

In March 2014, the New South Wales (NSW) Government (Australia) announced the NSW Integrated Care Strategy. In response, a family-centred, population-based, integrated care initiative for vulnerable families and their children in Sydney, Australia was developed. The initiative was called Healthy Homes and Neighbourhoods. A realist translational social epidemiology programme of research and collaborative design is at the foundation of its evaluation.

Theory and Method:

The UK Medical Research Council (MRC) Framework for evaluating complex health interventions was adapted. This has four components, namely 1) development, 2) feasibility/piloting, 3) evaluation and 4) implementation. We adapted the Framework to include: critical realist, theory driven, and continuous improvement approaches. The modified Framework underpins this research and evaluation protocol for Healthy Homes and Neighbourhoods.

Discussion:

The NSW Health Monitoring and Evaluation Framework did not make provisions for assessment of the programme layers of context, or the effect of programme mechanism at each level. We therefore developed a multilevel approach that uses mixed-method research to examine not only outcomes, but also what is working for whom and why.

A Critical Realist Translational Social Epidemiology Protocol for Concretising and Contextualising a "Theory of Neighbourhood Context, Stress, Depression, and the Developmental Origins of Health and Disease (DOHaD)", Sydney Australia

Background:

We will describe here a translational social epidemiology protocol for confirming a critical realist “Theory of Neighbourhood Context, Stress, Depression, and the Developmental Origins of Health and Disease (DOHaD). The approach will include the concretising and contextualising of the above causal theory into programme theories for child and adolescent interventions that aim to break intergenerational cycles of disadvantage and poor life outcomes. In undertaking this work we seek to advance realist translational methodology within the discipline of applied perinatal and paediatric social epidemiology.

Theory and Methods:

The research settings are in metropolitan Sydney. The design will be a longitudinal, multi-level, mixed method realist evaluation of applied programme interventions that seek to break the intergeneration cycle of social disadvantage and poor child health and developmental outcomes. The programme of research will consist of three components: 1) Operationalisation of the theory and designing of programme initiatives for implementation; 2) Evaluation of the translated programme and implementation theory using Theory of Change and critical realist evaluation; and 3) Theory Testing of realist hypotheses using both intensive and extensive critical realist research methods including realist structural modelling.

Discussion:

The proposed programme of research will assist in translating empirical explanatory theory building to theory driven interventions. The research will be situated in socially disadvantaged regions of Sydney where the local child and family inter-agencies will collaborate to design and implement new initiatives that address significant disparities in childhood development and adolescent outcomes attributed to neighbourhood circumstances, family stress and intergenerational cycles of disadvantage and poor mental health.

Health professional-delivered obesity prevention interventions during the first 1,000 days: A systematic review of external validity reporting

Background: Childhood obesity prevention interventions delivered by health professionals during the first 1,000 days show some evidence of effectiveness, particularly in relation to behavioural outcomes. External validity refers to how generalisable interventions are to populations or settings beyond those in the original study. The degree to which external validity elements are reported in such studies is unclear however. This systematic review aimed to determine the extent to which childhood obesity interventions delivered by health professionals during the first 1,000 days report on elements that can be used to inform generalizability across settings and populations. Methods: Eligible studies meeting study inclusion and exclusion criteria were identified through a systematic review of 11 databases and three trial registers. An assessment tool based on the RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance) framework was used to assess the external validity of included studies. It comprised five dimensions: reach and representativeness of individuals, reach and representativeness of settings, implementation and adaptation, outcomes for decision making maintenance and/or institutionalisation. Two authors independently assessed the external validity of 20% of included studies; discrepancies were resolved, and then one author completed assessments of the remaining studies. Results: In total, 39 trials involving 46 interventions published between 1999 and 2019 were identified. The majority of studies were randomized controlled trials (n=24). Reporting varied within and between dimensions. External validity elements that were poorly described included: representativeness of individuals and settings, treatment receipt, intervention mechanisms and moderators, cost effectiveness, and intervention sustainability and acceptability. Conclusions: Our review suggests that more emphasis is needed on research designs that consider generalisability, and the reporting of external validity elements in early life childhood obesity prevention interventions. Important gaps in external validity reporting were identified that could facilitate decisions around the translation and scale-up of interventions from research to practice. Registration: PROSPERO CRD42016050793 03/11/16.

Health professional-delivered obesity prevention interventions during the first 1,000 days: A systematic review of external validity reporting

Background: Childhood obesity prevention interventions delivered by health professionals during the first 1,000 days show some evidence of effectiveness, particularly in relation to behavioural outcomes. External validity refers to how generalisable interventions are to populations or settings beyond those in the original study. The degree to which external validity elements are reported in such studies is unclear however. This systematic review aimed to determine the extent to which childhood obesity interventions delivered by health professionals during the first 1,000 days report on elements that can be used to inform generalizability across settings and populations. Methods: Eligible studies meeting study inclusion and exclusion criteria were identified through a systematic review of 11 databases and three trial registers. An assessment tool based on the RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance) framework was used to assess the external validity of included studies. It comprised five dimensions: reach and representativeness of individuals, reach and representativeness of settings, implementation and adaptation, outcomes for decision making maintenance and/or institutionalisation. Two authors independently assessed the external validity of 20% of included studies; discrepancies were resolved, and then one author completed assessments of the remaining studies. Results: In total, 39 trials involving 46 interventions published between 1999 and 2019 were identified. The majority of studies were randomized controlled trials (n=24). Reporting varied within and between dimensions. External validity elements that were poorly described included: representativeness of individuals and settings, treatment receipt, intervention mechanisms and moderators, cost effectiveness, and intervention sustainability and acceptability. Conclusions: Our review suggests that more emphasis is needed on research designs that consider generalisability, and the reporting of external validity elements in early life childhood obesity prevention interventions. Important gaps in external validity reporting were identified that could facilitate decisions around the translation and scale-up of interventions from research to practice. Registration: PROSPERO CRD42016050793 03/11/16.

A realist process evaluation of Enhanced Triple P for Baby and Mellow Bumps, within a Trial of Healthy Relationship Initiatives for the Very Early years (THRIVE): study protocol for a randomized controlled trial

Background

THRIVE is a three-arm randomised controlled trial (RCT) that aims to evaluate whether antenatal and early postnatal interventions, Enhanced Triple B for Baby (ETPB) plus care as usual (CAU) or Mellow Bumps (MB) plus CAU (versus CAU alone), can: 1) improve the mental health and well-being of pregnant women with complex health and social care needs; 2) improve mother-infant bonding and interaction; 3) reduce child maltreatment; and 4) improve child language acquisition. This paper focuses on THRIVE’s realist process evaluation, which is carefully monitoring what is happening in the RCT.

Methods

Realistic evaluation provides the theoretical rationale for the process evaluation. We question: 1) how faithfully are MB and ETPB implemented? 2) What are the mechanisms by which they work, if they do, and who do they work for and how? 3) What contextual factors are necessary for the programmes to function, or might prevent them functioning?

The mixed-methods design includes quantitative measures, which are pre- and post-training/intervention questionnaires for facilitators and mothers-to-be, and post-session evaluation forms. Qualitative data collection methods include participant observation of facilitator training and the delivery of a series of antenatal sessions in selected intervention groups (n = 3 for ETPB and n = 3 for MB), semi-structured interviews with facilitators, pregnant women, partners, and referring facilitators, and telephone interviews examining the content of the postnatal components of ETPB and MB.

Discussion

The findings of this process evaluation will help researchers and decision makers interpret the outcomes of THRIVE. It will provide a greater understanding of: how the interventions work (if they do); the extent and quality of their implementation; contextual factors facilitating and constraining intervention functioning; variations in response within and between subgroups of vulnerable parents; and benefits or unintended consequences of either intervention. Few studies to date have published detailed research protocols illustrating how realist process evaluation is designed and conducted as an integral part of a randomised controlled trial.

Trial registration

ISRCTN, ISRCTN21656568. Registered on 8 November 2013.

Electronic supplementary material

The online version of this article (10.1186/s13063-019-3395-3) contains supplementary material, which is available to authorized users.

Designing and evaluating provider results-based financing for tuberculosis care in Georgia: a realist evaluation protocol

INTRODUCTION

In 2016, Georgian researchers and policymakers were developing a policy to improve the performance of the national tuberculosis (TB) control programme. The research programme 'Designing and Evaluating Provider Results-Based Financing for Tuberculosis Care in Georgia: Understanding Costs, Mechanisms of Effect and Impact (Results4TB)' was initiated to inform the policy formulation phase, document the policy implementation and assess the effectiveness, cost-effectiveness and the processes of change. To achieve this, the research team intends to combine an impact evaluation, a cost-effectiveness study and a realist evaluation (RE) within an overarching theory-informed design. This protocol is the RE component of the programme.

METHODS

A realist methodological approach will be adopted to guide the research design and evaluation. RE answers the question of 'what works in which conditions for whom?' and starts with the development of an initial programme theory (IPT). The IPT will feed into other phases of the realist research cycle (study design, data collection, data analysis and synthesis and theory refinement). Data will be collected in a multiple embedded case study design (five intervention and three control sites) through document reviews, in-depth interviews, non-participant observations and context mapping at facility and national levels. Additional data from other research components (cost-effectiveness and impact evaluation) will aid data triangulation.

ETHICS AND DISSEMINATION

The Institutional Review Boards of the National Centre for Disease Control and Public Health in Georgia (ref. IRB # 2018-019) and the Institute of Tropical Medicine, Antwerp (ref. IRB #- 1240/18) have granted ethical approval to the study.

TRIAL REGISTRATION NUMBER

ISRCTN14667607.

Improving primary care Access in Context and Theory (I-ACT trial): a theory-informed randomised cluster feasibility trial using a realist perspective

Background

Primary care access can be challenging for older, rural, socio-economically disadvantaged populations. Here we report the I-ACT cluster feasibility trial which aims to assess the feasibility of trial design and context-sensitive intervention to improve primary care access for this group and so expand existing theory.

Methods

Four general practices were recruited; three randomised to intervention and one to usual care. Intervention practices received £1500, a support manual and four meetings to develop local, innovative solutions to improve the booking system and transport.

Patients aged over 64 years old and without household car access were recruited to complete questionnaires when booking an appointment or attending the surgery. Outcome measures at 6 months included: self-reported ease of booking an appointment and transport; health care use; patient activation; capability; and quality of life. A process evaluation involved observations and interviews with staff and participants.

Results

Thirty-four patients were recruited (26 female, eight male, mean age 81.6 years for the intervention group and 79.4 for usual care) of 1143 invited (3% response rate). Most were ineligible because of car access. Twenty-nine participants belonged to intervention practices and five to usual care. Practice-level data was available for all participants, but participant self-reported data was unavailable for three. Fifty-six appointment questionnaires were received based on 150 appointments (37.3%).

Practices successfully designed and implemented the following context-sensitive interventions: Practice A: a stacked telephone system and promoting community transport; Practice B: signposting to community transport, appointment flexibility, mobility scooter charging point and promoting the role of receptionists; and Practice C: local taxi firm partnership and training receptionists. Practices found the process acceptable because it gave freedom, time and resource to be innovative or provided an opportunity to implement existing ideas. Data collection methods were acceptable to participants, but some found it difficult remembering to complete booking and appointment questionnaires. Expanded theory highlighted important mechanisms, such as reassurance, confidence, trust and flexibility.

Conclusions

Recruiting older participants without access to a car proved challenging. Retention of participants and practices was good but only about a third of appointment questionnaires were returned. This study design may facilitate a shift from one-size-fits-all interventions to more context-sensitive interventions.

Trial registration

ISRCTN18321951, Registered on 6 March 2017.

Electronic supplementary material

The online version of this article (10.1186/s13063-019-3299-2) contains supplementary material, which is available to authorized users.

Evaluating Complex Health Interventions With Randomized Controlled Trials: How Do We Improve the Use of Qualitative Methods?

Qualitative methods are underutilized in health intervention evaluation, and overshadowed by the importance placed on randomized controlled trials (RCTs). This Commentary describes how innovative qualitative methods are being used as part of RCTs, drawing on articles included in a special issue of Qualitative Health Research on this topic. The articles' insights and a review of innovative qualitative methods described in trial protocols highlights a lack of attention to structural inequalities as a causal mechanism for understanding human behavior. We situate this gap within some well-known constraints of RCT methodologies, and a discussion of alternative RCT approaches that hold promise for bringing qualitative methods center stage in intervention evaluation, including adaptive designs, pragmatic trials, and realist RCTs. To address the power hierarchies of health evaluation research, however, we argue that a fundamental shift needs to take place away from a focus on RCTs and toward studies of health interventions.

From intervention to interventional system: towards greater theorization in population health intervention research

Background

Population health intervention research raises major conceptual and methodological issues. These require us to clarify what an intervention is and how best to address it.

This paper aims to clarify the concepts of intervention and context and to propose a way to consider their interactions in evaluation studies, especially by addressing the mechanisms and using the theory-driven evaluation methodology.

Main text

This article synthesizes the notions of intervention and context. It suggests that we consider an “interventional system”, defined as a set of interrelated human and non-human contextual agents within spatial and temporal boundaries generating mechanistic configurations – mechanisms – which are prerequisites for change in health. The evaluation focal point is no longer the interventional ingredients taken separately from the context, but rather mechanisms that punctuate the process of change. It encourages a move towards theorization in evaluation designs, in order to analyze the interventional system more effectively. More particularly, it promotes theory-driven evaluation, either alone or combined with experimental designs.

Conclusion

Considering the intervention system, hybridizing paradigms in a process of theorization within evaluation designs, including different scientific disciplines, practitioners and intervention beneficiaries, may allow researchers a better understanding of what is being investigated and enable them to design the most appropriate methods and modalities for characterizing the interventional system. Evaluation methodologies should therefore be repositioned in relation to one another with regard to a new definition of “evidence”, repositioning practitioners’ expertise, qualitative paradigms and experimental questions in order to address the intervention system more profoundly.

Evaluating program effects: Conceptualizing and demonstrating a typology

Outcome evaluation is very important for program evaluation and has been becoming increasingly so in the age of accountability. Typically, outcome evaluation is conducted for a single program from a single perspective. However, in a real-life situation, many programs exist in a system, and the effects could be viewed from various perspectives. The authors illustrate a typology of program effects in a system. It moves from the paradigm of a single program's single effect to that of a set of programs' multiple effects. Methodological implications are discussed.

Task shifting to improve the provision of integrated chronic care: realist evaluation of a lay health worker intervention in rural South Africa

Introduction

Task shifting is a potential solution to the shortage of healthcare personnel in low/middle-income countries, but contextual factors often dilute its effectiveness. We report on a task shifting intervention using lay health workers to support clinic staff in providing chronic disease care in rural South Africa, where the HIV epidemic and an ageing population have increased demand for care.

Methods

We conducted a realist evaluation in a cluster randomised controlled trial. We conducted observations in clinics, focus group discussions, in-depth interviews and patient exit interviews, and wrote weekly diaries to collect data.

Results

All clinic managers had to cope with an increasing but variable patient load and unplanned staff shortages, insufficient space, poorly functioning equipment and erratic supply of drugs. These conditions inevitably generated tension among staff. Lay health workers relieved the staff of some of their tasks and improved care for patients, but in some cases the presence of the lay health worker generated conflict with other staff. Where managers were able to respond to the changing circumstances, and to contain tension among staff, facilities were better able to meet patient needs. This required facility managers to be flexible, consultative and willing to act on suggestions, sometimes from junior staff and patients. While all facilities experienced an erratic supply of drugs and poorly maintained equipment, facilities where there was effective management, teamwork and sufficient space had better chronic care processes and a higher proportion of patients attending on their appointed day.

Conclusion

Lay health workers can be valuable members of a clinic team, and an important resource for managing increasing patient demand in primary healthcare. Task shifting will only be effective if clinic managers respond to the constantly changing system and contain conflict between staff. Strengthening facility-level management and leadership skills is a priority.

Trial registration number

ISRCTN12128227.

Access to primary care for socio-economically disadvantaged older people in rural areas: exploring realist theory using structural equation modelling in a linked dataset

Background

Realist approaches seek to answer questions such as ‘how?’, ‘why?’, ‘for whom?’, ‘in what circumstances?’ and ‘to what extent?’ interventions ‘work’ using context-mechanism-outcome (CMO) configurations. Quantitative methods are not well-established in realist approaches, but structural equation modelling (SEM) may be useful to explore CMO configurations. Our aim was to assess the feasibility and appropriateness of SEM to explore CMO configurations and, if appropriate, make recommendations based on our access to primary care research. Our specific objectives were to map variables from two large population datasets to CMO configurations from our realist review looking at access to primary care, generate latent variables where needed, and use SEM to quantitatively test the CMO configurations.

Methods

A linked dataset was created by merging individual patient data from the English Longitudinal Study of Ageing and practice data from the GP Patient Survey. Patients registered in rural practices and who were in the highest deprivation tertile were included. Three latent variables were defined using confirmatory factor analysis. SEM was used to explore the nine full CMOs. All models were estimated using robust maximum likelihoods and accounted for clustering at practice level. Ordinal variables were treated as continuous to ensure convergence.

Results

We successfully explored our CMO configurations, but analysis was limited because of data availability. Two hundred seventy-six participants were included. We found a statistically significant direct (context to outcome) or indirect effect (context to outcome via mechanism) for two of nine CMOs. The strongest association was between ‘ease of getting through to the surgery’ and ‘being able to get an appointment’ with an indirect mediated effect through convenience (proportion of the indirect effect of the total was 21%). Healthcare experience was not directly associated with getting an appointment, but there was a statistically significant indirect effect through convenience (53% mediated effect). Model fit indices showed adequate fit.

Conclusions

SEM allowed quantification of CMO configurations and could complement other qualitative and quantitative techniques in realist evaluations to support inferences about strengths of relationships. Future research exploring CMO configurations with SEM should aim to collect, preferably continuous, primary data.

Electronic supplementary material

The online version of this article (10.1186/s12874-018-0514-x) contains supplementary material, which is available to authorized users.

Do project management and network governance contribute to inter-organisational collaboration in primary care? A mixed methods study

Background

The need for organisational development in primary care has increased as it is accepted as a means of curbing rising costs and responding to demographic transitions. It is only within such inter-organisational networks that small-scale practices can offer treatment to complex patients and continuity of care. The aim of this paper is to explore, through the experience of professionals and patients, whether, and how, project management and network governance can improve the outcomes of projects which promote inter-organisational collaboration in primary care.

Methods

This paper describes a study of projects aimed at improving inter-organisational collaboration in Dutch primary care. The projects' success in project management and network governance was monitored by interviewing project leaders and board members on the one hand, and improvement in the collaboration by surveying professionals and patients on the other. Both qualitative and quantitative methods were applied to assess the projects. These were analysed, finally, using multi-level models in order to account for the variation in the projects, professionals and patients.

Results

Successful network governance was associated positively with the professionals’ satisfaction with the collaboration; but not with improvements in the quality of care as experienced by patients. Neither patients nor professionals perceived successful project management as associated with the outcomes of the collaboration projects.

Conclusions

This study shows that network governance in particular makes a difference to the outcomes of inter-organisational collaboration in primary care. However, project management is not a predictor for successful inter-organisational collaboration in primary care.

Electronic supplementary material

The online version of this article (10.1186/s12913-018-3169-8) contains supplementary material, which is available to authorized users.

Are randomised controlled trials positivist? Reviewing the social science and philosophy literature to assess positivist tendencies of trials of social interventions in public health and health services

Background

We have previously proposed that trials of social interventions can be done within a “realist” research paradigm. Critics have countered that such trials are irredeemably positivist and asked us to explain our philosophical position.

Methods

We set out to explore what is meant by positivism and whether trials adhere to its tenets (of necessity or in practice) via a narrative literature review of social science and philosophical discussions of positivism, and of the trials literature and three case studies of trials.

Results

The philosophical literature described positivism as asserting: (1) the epistemic primacy of sensory information; (2) the requirement that theoretical terms equate with empirical terms; (3) the aim of developing universal laws; and (4) the unity of method between natural and social sciences. Regarding (1), it seems that rather than embodying the epistemic primacy of sensory data, randomised controlled trials (RCTs) of social interventions in health embrace an anti-positivist approach aiming to test hypotheses derived deductively from prior theory. Considering (2), while some RCTs of social interventions appear to limit theorisation to concepts with empirical analogues, others examine interventions underpinned by theories engaging with mechanisms and contextual contingencies not all of which can be measured. Regarding (3), while some trialists and reviewers in the health field do limit their role to estimating statistical trends as a mechanistic form of generalisation, this is not an inevitable feature of RCT-based research. Trials of social interventions can instead aim to generalise at the level of theory which specifies how mechanisms are contingent on context. In terms of (4), while RCTs are used to examine biomedical as well as social interventions in health, RCTs of social interventions are often distinctive in using qualitative analyses of data on participant accounts to examine questions of meaning and agency not pursued in the natural sciences.

Conclusion

We conclude that the most appropriate paradigm for RCTs of social interventions is realism not positivism.

Person-Centered Integrated Care for Chronic Kidney Disease: A Systematic Review and Meta-Analysis of Randomized Controlled Trials

BACKGROUND AND OBJECTIVES

The effectiveness of person-centered integrated care strategies for CKD is uncertain. We conducted a systematic review and meta-analysis of randomized, controlled trials to assess the effect of person-centered integrated care for CKD.

DESIGN, SETTING, PARTICIPANTS, & MEASUREMENTS

We searched MEDLINE, Embase, and Cochrane Central Register of Controlled Trials (from inception to April of 2016), and selected randomized, controlled trials of person-centered integrated care interventions with a minimum follow-up of 3 months. Random-effects meta-analysis was used to assess the effect of person-centered integrated care.

RESULTS

We included 14 eligible studies covering 4693 participants with a mean follow-up of 12 months. In moderate quality evidence, person-centered integrated care probably had no effect on all-cause mortality (relative risk [RR], 0.86; 95% confidence interval [95% CI], 0.68 to 1.08) or health-related quality of life (standardized mean difference, 0.02; 95% CI, -0.05 to 0.10). The effects on renal replacement therapy (RRT) (RR, 1.00; 95% CI, 0.65 to 1.55), serum creatinine levels (mean difference, 0.59 mg/dl; 95% CI, -0.38 to 0.36), and eGFR (mean difference, 1.51 ml/min per 1.73 m2; 95% CI, -3.25 to 6.27) were very uncertain. Quantitative analysis suggested that person-centered integrated care interventions may reduce all-cause hospitalization (RR, 0.38; 95% CI, 0.15 to 0.95) and improve BP control (RR, 1.20; 95% CI, 1.00 to 1.44), although the certainty of the evidence was very low.

CONCLUSIONS

Person-centered integrated care may have little effect on mortality or quality of life. The effects on serum creatinine, eGFR, and RRT are uncertain, although person-centered integrated care may lead to fewer hospitalizations and improved BP control.

A critical realist evaluation of a music therapy intervention in palliative care

BACKGROUND

Music therapy is increasingly used as an adjunct therapy to support symptom management in palliative care. However, studies to date have paid little attention to the processes that lead to changes in patient outcomes. To fill this gap, we examined the processes and experiences involved in the introduction of music therapy as an adjunct complementary therapy to palliative care in a hospice setting in the United Kingdom (UK).

METHODS

Using a realistic evaluation approach, we conducted a qualitative study using a variety of approaches. These consisted of open text answers from patients (n = 16) on how music therapy helped meet their needs within one hospice in Northern Ireland, UK. We also conducted three focus groups with a range of palliative care practitioners (seven physicians, seven nursing staff, two social workers and three allied health professionals) to help understand their perspectives on music therapy's impact on their work setting, and what influences its successful implementation. This was supplemented with an interview with the music therapist delivering the intervention.

RESULTS

Music therapy contains multiple mechanisms that can provide physical, psychological, emotional, expressive, existential and social support. There is also evidence that the hospice context, animated by a holistic approach to healthcare, is an important facilitator of the effects of music therapy. Examination of patients' responses helped identify specific benefits for different types of patients.

CONCLUSIONS

There is a synergy between the therapeutic aims of music therapy and those of palliative care, which appealed to a significant proportion of participants, who perceived it as effective.

What theory, for whom and in which context? Reflections on the application of theory in the development and evaluation of complex population health interventions

Recent years have seen a growing emphasis on the value of building and testing middle range theory throughout the development and evaluation of complex population health interventions. We agree that a coherent theoretical basis for intervention development, and use of evaluation to test key causal assumptions and build theory, are crucial. However, in this editorial, we argue that such recommendations have often been operationalised in somewhat simplistic terms with potentially perverse consequences, and that an uncritical assumption that an intervention explicitly based on theory is inherently superior carries significant risks. We first argue that the drive for theory-based approaches may have exacerbated a propensity to select ‘off-the-shelf’ theories, leading to the selection of inappropriate theories which distract attention from the mechanisms through which a problem is actually sustained. Second, we discuss a tendency toward over-reliance on individual-level theorising. Finally, we discuss the relatively slow progress of population health intervention research in attending to issues of context, and the ecological fit of interventions with the systems whose functioning they attempt to change. We argue that while researchers should consider a broad range of potential theoretical perspectives on a given population health problem, citing a popular off-the-shelf theory as having informed an intervention and its evaluation does not inherently make for better science. Before identifying or developing a theory of change, researchers should develop a clear understanding of how the problem under consideration is created and sustained in context. A broader conceptualisation of theory that reaches across disciplines is vital if theory is to enhance, rather than constrain, the contribution of intervention research. Finally, intervention researchers need to move away from viewing interventions as discrete packages of components which can be described in isolation from their contexts, and better understand the systems into which change is being introduced.

Advancing 'real-world' trials that take account of social context and human volition

Background

The recent paper in Trials by Porter and colleagues highlights the utility of applying a critical realism approach in randomised trials, an approach central to the Medical Research Council’s (MRC) Framework for the Development and Evaluation of Complex Healthcare Interventions. The MRC framework offers a pragmatic step towards a more open systems approach that bridges randomised evaluation with social context and human agency in an effort to improve the generalisability of trial outcomes.

Main body

The MRC framework has contributed to the proliferation of a more open systems approach in health research; however, the broader acceptance of the realist approach to health research does not seem to be emulated by norms in research fund allocation, which largely prioritises laboratory-based research.

Conclusion

This commentary is simply a plea, to those who make the strategic decisions regarding allocation of research funding, to support all phases of health intervention research in complex systems that contribute to the development of effective, translational and sustainable interventions in the promotion of health.

The challenge of complexity in evaluating health policies and programs: the case of women's participatory groups to improve antenatal outcomes

Background

During the last years, randomized designs have been promoted as the cornerstone of evidence-based policymaking. Also in the field of community participation, Random Control Trials (RCTs) have been the dominant design, used for instance to examine the contribution of community participation to health improvement. We aim at clarifying why RCTs and related (quasi-) experimental designs may not be the most appropriate approach to evaluate such complex programmes.

Results

We argue that the current methodological debate could be more fruitful if it would start from the position that the choice of designs should fit the nature of the program and research questions rather than be driven by methodological preferences. We present how realist evaluation, a theory-driven approach to research and evaluation, is a relevant methodology that could be used to assess whether and how community participation works. Using the realist evaluation approach to examine the relationship between participation and action of women groups and antenatal outcomes would enable evaluators to examine in detail the underlying mechanisms which influence actual practices and outcomes, as well as the context conditions required to make it work.

Conclusions

Realist research in fact allows opening the black boxes of “community” and “participation” in order to examine the role they play in ensuring cost-effective, sustainable interventions. This approach yields important information for policy makers and programme managers considering how such programs could be implemented in their own setting.

The possibility of critical realist randomised controlled trials

Background

Some realists have criticised randomised controlled trials for their inability to explain the causal relations that they identify; to take into account the influence of the social context of the interventions they evaluate; and to account for individual difference. However, among realists, there is controversy over whether it is possible to improve trials by making them realist, or whether realism and the philosophical assumptions underlying trials are incompatible. This paper contributes to the debate in Trials on this issue. The debate thus far has concentrated on the possibility of combining trial methodology with that of realist evaluation.

Main body

We concur with the contention that it is not feasible to combine randomised controlled trial design with the realist evaluation approach. However, we argue that a different variant of realism, critical realism, provides a more appropriate theoretical grounding for realist trials.

In contrast to realist evaluation, which regards social mechanisms as an amalgam of social resources and people’s reasoning, critical realism insists on their distinction. It does so on the basis of its assertion of the need to distinguish between social structures (in which resources lie) and human agency (which is at least partly guided by reasoning). From this perspective, conceiving of social mechanisms as external to participants can be seen as a valid methodological strategy for supplementing the exclusive concentration of trials on outcomes.

While accepting realist evaluation’s insistence that causality in open systems involves a configuration of multiple generative mechanisms, we adopt the critical realist interpretation of the experimental method, which sees it as creating artificial closure in order to identify the effects of specific causal mechanisms. If randomised controlled trials can be regarded as epidemiological proxies that substitute probabilistic controls over extraneous factors for closed experiments, their examination of the powers of discrete mechanisms through observation of the variation of outcomes is appropriate.

Conclusion

While there are still issues to be resolved, critical realist randomised controlled trials are possible and have the potential to overcome some of the difficulties faced by traditional trial designs in accounting for the influence of social context and individual interpretation.

Realist trials and the testing of context-mechanism-outcome configurations: a response to Van Belle et al

Background

Van Belle et al. argue that our attempt to pursue realist evaluation via a randomised trial will be fruitless because we misunderstand realist ontology (confusing intervention mechanisms with intervention activities and with statistical mediation analyses) and because RCTs cannot comprehensively examine how and why outcome patterns are caused by mechanisms triggered in specific contexts.

Methods

Through further consideration of our trial methods, we explain more fully how we believe complex social interventions work and what realist evaluation should aim to do within a trial.

Results

Like other realists, those undertaking realist trials assume that: social interventions provide resources which local actors may draw on in actions that can trigger mechanisms; these mechanisms may interact with contextual factors to generate outcomes; and data in the ‘empirical’ realm can be used to test hypotheses about mechanisms in the ‘real’ realm. Whether or not there is sufficient contextual diversity to test such hypotheses is a contingent not a necessary feature of trials. Previous exemplars of realist evaluation have compared empirical data from intervention and control groups to test hypotheses about real mechanisms. There is no inevitable reason why randomised trials should not also be able to do so. Random allocation merely ensures the comparability of such groups without necessarily causing evaluation to lapse from a realist into a ‘positivist’ or ‘post-positivist’ paradigm.

Conclusions

Realist trials are ontologically and epistemologically plausible. Further work is required to assess whether they are feasible and useful but such work should not be halted on spurious philosophical grounds.

Can "realist" randomised controlled trials be genuinely realist?

In this paper, we respond to a paper by Jamal and colleagues published in Trials in October 2015 and take an opportunity to continue the much-needed debate about what applied scientific realism is. The paper by Jamal et al. is useful because it exposes the challenges of combining a realist evaluation approach (as developed by Pawson and Tilley) with the randomised controlled trial (RCT) design.

We identified three fundamental differences that are related to paradigmatic differences in the treatment of causation between post-positivist and realist logic: (1) the construct of mechanism, (2) the relation between mediators and moderators on one hand and mechanisms and contexts on the other hand, and (3) the variable-oriented approach to analysis of causation versus the configurational approach.

We show how Jamal et al. consider mechanisms as observable, external treatments and how their approach reduces complex causal processes to variables. We argue that their proposed RCT design cannot provide a truly realist understanding. Not only does the proposed realist RCT design not deal with the RCT’s inherent inability to “unpack” complex interventions, it also does not enable the identification of the dynamic interplay among the intervention, actors, context, mechanisms and outcomes, which is at the core of realist research. As a result, the proposed realist RCT design is not, as we understand it, genuinely realist in nature.

Realist complex intervention science: Applying realist principles across all phases of the Medical Research Council framework for developing and evaluating complex interventions

The integration of realist evaluation principles within randomised controlled trials ('realist RCTs') enables evaluations of complex interventions to answer questions about what works, for whom and under what circumstances. This allows evaluators to better develop and refine mid-level programme theories. However, this is only one phase in the process of developing and evaluating complex interventions. We describe and exemplify how social scientists can integrate realist principles across all phases of the Medical Research Council framework. Intervention development, modelling, and feasibility and pilot studies need to theorise the contextual conditions necessary for intervention mechanisms to be activated. Where interventions are scaled up and translated into routine practice, realist principles also have much to offer in facilitating knowledge about longer-term sustainability, benefits and harms. Integrating a realist approach across all phases of complex intervention science is vital for considering the feasibility and likely effects of interventions for different localities and population subgroups.