1
|
Segrott J, Channon S, Lloyd A, Glarou E, Henley J, Hughes J, Jacob N, Milosevic S, Moriarty Y, Pell B, Robling M, Strange H, Townson J, Brookes-Howell L. Integrating qualitative research within a clinical trials unit: developing strategies and understanding their implementation in contexts. Trials 2024; 25:323. [PMID: 38750606 PMCID: PMC11097452 DOI: 10.1186/s13063-024-08124-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2023] [Accepted: 04/17/2024] [Indexed: 05/18/2024] Open
Abstract
BACKGROUND/AIMS The value of using qualitative methods within clinical trials is widely recognised. How qualitative research is integrated within trials units to achieve this is less clear. This paper describes the process through which qualitative research has been integrated within Cardiff University's Centre for Trials Research (CTR) in Wales, UK. We highlight facilitators of, and challenges to, integration. METHODS We held group discussions on the work of the Qualitative Research Group (QRG) within CTR. The content of these discussions, materials for a presentation in CTR, and documents relating to the development of the QRG were interpreted at a workshop attended by group members. Normalisation Process Theory (NPT) was used to structure analysis. A writing group prepared a document for input from members of CTR, forming the basis of this paper. RESULTS Actions to integrate qualitative research comprised: its inclusion in Centre strategies; formation of a QRG with dedicated funding/roles; embedding of qualitative research within operating systems; capacity building/training; monitoring opportunities to include qualitative methods in studies; maximising the quality of qualitative research and developing methodological innovation. Facilitators of these actions included: the influence of the broader methodological landscape within trial/study design and its promotion of the value of qualitative research; and close physical proximity of CTR qualitative staff/students allowing sharing of methodological approaches. Introduction of innovative qualitative methods generated interest among other staff groups. Challenges included: pressure to under-resource qualitative components of research, preference for a statistical stance historically in some research areas and funding structures, and difficulties faced by qualitative researchers carving out individual academic profiles when working across trials/studies. CONCLUSIONS Given that CTUs are pivotal to the design and conduct of RCTs and related study types across multiple disciplines, integrating qualitative research into trials units is crucial if its contribution is to be fully realised. We have made explicit one trials unit's experience of embedding qualitative research and present this to open dialogue on ways to operationalise and optimise qualitative research in trials. NPT provides a valuable framework with which to theorise these processes, including the importance of sense-making and legitimisation when introducing new practices within organisations.
Collapse
Affiliation(s)
- Jeremy Segrott
- Centre for Trials Research, DECIPHer Centre, Cardiff University, Neuadd Meirionnydd, Heath Park, Cardiff, CF14 4YS, UK.
| | - Sue Channon
- Centre for Trials Research, Cardiff University, Neuadd Meirionnydd, Heath Park, Cardiff, CF14 4YS, UK
| | - Amy Lloyd
- Wales Centre for Public Policy, Cardiff University, Sbarc I Spark, Maindy Road, Cardiff, CF24 4HQ, UK
| | - Eleni Glarou
- Centre for Trials Research, Cardiff University, Neuadd Meirionnydd, Heath Park, Cardiff, CF14 4YS, UK
- Division of Population Medicine, School of Medicine, Cardiff University, Neuadd Meirionnydd, Heath Park, Cardiff, CF14 4YS, UK
| | - Josie Henley
- School of Social Sciences, Cardiff University, King Edward VII Avenue, Cardiff, CF10 3WA, UK
| | - Jacqueline Hughes
- Centre for Trials Research, Cardiff University, Neuadd Meirionnydd, Heath Park, Cardiff, CF14 4YS, UK
| | - Nina Jacob
- Centre for Trials Research, Cardiff University, Neuadd Meirionnydd, Heath Park, Cardiff, CF14 4YS, UK
| | - Sarah Milosevic
- Centre for Trials Research, Cardiff University, Neuadd Meirionnydd, Heath Park, Cardiff, CF14 4YS, UK
| | - Yvonne Moriarty
- Centre for Trials Research, Cardiff University, Neuadd Meirionnydd, Heath Park, Cardiff, CF14 4YS, UK
| | - Bethan Pell
- DECIPHer Centre, School of Social Sciences, Cardiff University, Sbarc I Spark, Maindy Road, Cardiff, CF24 4HQ, UK
| | - Mike Robling
- Centre for Trials Research, Cardiff University, Neuadd Meirionnydd, Heath Park, Cardiff, CF14 4YS, UK
| | - Heather Strange
- Centre for Trials Research, Cardiff University, Neuadd Meirionnydd, Heath Park, Cardiff, CF14 4YS, UK
| | - Julia Townson
- Centre for Trials Research, Cardiff University, Neuadd Meirionnydd, Heath Park, Cardiff, CF14 4YS, UK
| | - Lucy Brookes-Howell
- Centre for Trials Research, Cardiff University, Neuadd Meirionnydd, Heath Park, Cardiff, CF14 4YS, UK
| |
Collapse
|
2
|
Rubæk M, Broholm-Jørgensen M, Andersen S, Jakobsen PR, Rothmann MJ, Langdahl B, Hitz MF, Holmberg T. Development of a program theory for osteoporosis patient education in Denmark: a qualitative study based on realist evaluation. BMC Geriatr 2024; 24:346. [PMID: 38627654 PMCID: PMC11022455 DOI: 10.1186/s12877-024-04957-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2023] [Accepted: 04/08/2024] [Indexed: 04/19/2024] Open
Abstract
BACKGROUND Osteoporosis patient education is offered in many countries worldwide. When evaluating complex interventions like these, it is important to understand how and why the intervention leads to effects. This study aimed to develop a program theory of osteoporosis patient education in Danish municipalities with a focus on examining the mechanisms of change i.e. what is about the programs that generate change. METHODS The program theory was developed in an iterative process. The initial draft was based on a previous published systematic review, and subsequently the draft was continually refined based on findings from observations (10 h during osteoporosis patient education) and interviews (individual interviews with six employees in municipalities and three health professionals at hospitals, as well as four focus group interviews with participants in patient education (in total 27 informants)). The transcribed interviews were analyzed using thematic analysis and with inspiration from realist evaluation the mechanisms as well as the contextual factors and outcomes were examined. RESULTS Based on this qualitative study we developed a program theory of osteoporosis patient education and identified four mechanisms: motivation, recognizability, reassurance, and peer reflection. For each mechanism we examined how contextual factors activated the mechanism as well as which outcomes were achieved. For instance, the participants' motivation is activated when they meet in groups, and thereafter outcomes such as more physical activity may be achieved. Recognizability is activated by the participants' course of disease, which may lead to better ergonomic habits. Reassurance may result in more physical activity, and this mechanism is activated in newly diagnosed participants without previous fractures. Peer reflection is activated when the participants meet in groups, and the outcome healthier diet may be achieved. CONCLUSIONS We developed a program theory and examined how and why osteoporosis patient education is likely to be effective. Understanding these prerequisites is important for future implementation and evaluation of osteoporosis patient education.
Collapse
Affiliation(s)
- Mette Rubæk
- National Institute of Public Health, University of Southern Denmark, Copenhagen, Denmark
- National Research Center for Bone Health, Zealand University Hospital, Køge, Denmark
| | | | - Susan Andersen
- National Institute of Public Health, University of Southern Denmark, Copenhagen, Denmark
| | - Pernille Ravn Jakobsen
- Department of Public Health, Research Unit of General Practice, University of Southern Denmark, Odense, Denmark
| | - Mette Juel Rothmann
- Steno Diabetes Center Odense, Odense University Hospital, Odense, Denmark
- Department of Endocrinology, Odense University Hospital, Odense, Denmark
- Department of Clinical Research, University of Southern Denmark, Odense, Denmark
| | - Bente Langdahl
- Department of Endocrinology, Aarhus University Hospital, Aarhus, Denmark
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
| | - Mette Friberg Hitz
- National Research Center for Bone Health, Zealand University Hospital, Køge, Denmark
| | | |
Collapse
|
3
|
Schoenmakers EC, Lasgaard M, McHugh Power J. Guidelines for evaluating and reporting social isolation and loneliness interventions. J Health Psychol 2024:13591053241238127. [PMID: 38527950 DOI: 10.1177/13591053241238127] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/27/2024] Open
Abstract
Given the unpleasant nature of social isolation and loneliness (SIL) and their negative effects on health and wellbeing, interventions are needed. However, persistent issues in the design, evaluation, and reporting of SIL interventions preclude conclusive evidence and commentary on the effectiveness of SIL interventions. Here, we propose guidelines for evaluating SIL interventions, firstly by operationalising them into two categories: (1) interventions aiming to reduce SIL as a primary outcome and (2) interventions aiming to improve non-SIL outcomes in the lives of individuals experiencing SIL. Secondly, we evaluate instruments for measuring SIL and research designs for studying intervention effectiveness. Thirdly, guidelines for reporting information about the intervention, study design, results, and discussion in SIL intervention studies are presented. These guidelines will help researchers to better and more consistently report on SIL interventions and improve comparability of SIL interventions, ultimately contributing to the improvement of interventions and to the mitigation of SIL.
Collapse
|
4
|
McDermott E, Kaley A, Kaner E, Limmer M, McGovern R, McNulty F, Nelson R, Geijer-Simpson E, Spencer L. Reducing LGBTQ+ adolescent mental health inequalities: a realist review of school-based interventions. J Ment Health 2023:1-11. [PMID: 37589454 DOI: 10.1080/09638237.2023.2245894] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2023] [Revised: 06/26/2023] [Accepted: 06/27/2023] [Indexed: 08/18/2023]
Abstract
BACKGROUND LGBTQ+ young people have elevated rates of poor mental health in comparison to their cisgender heterosexual peers. School environment is a key risk factor and consistently associated with negative mental health outcomes for LGBTQ+ adolescents. AIMS To examine how, why, for whom and in what context school-based interventions prevent or reduce mental health problems in LGBTQ+ adolescents. METHODS A realist review methodology was utilised and focused on all types of school-based interventions and study designs. A Youth Advisory Group were part of the research team. Multiple search strategies were used to locate relevant evidence. Studies were subject to inclusion criteria and quality appraisal, and included studies were synthesised to produce a programme theory. Seventeen studies were included in the review. RESULTS Eight intervention components were necessary to address LGBTQ+ pupils mental health: affirmative visual displays; external signposting to LGBTQ+ support; stand-alone input; school-based LGBTQ support groups; curriculum-based delivery; staff training; inclusion policies; trusted adult. Few school-based interventions for this population group were identified. CONCLUSIONS The programme theory indicates that "to work" school-based interventions must have a "whole-school" approach that addresses specifically the dominant cis-heteronormative school environment and hence the marginalisation, silence, and victimisation that LGBTQ+ pupils can experience.
Collapse
Affiliation(s)
| | - Alex Kaley
- School of Health and Social Care, University of Essex, Colchester, UK
| | - Eileen Kaner
- Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
| | - Mark Limmer
- Faculty of Health and Medicine, Lancaster University, Lancaster, UK
| | - Ruth McGovern
- Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
| | - Felix McNulty
- Faculty of Health and Medicine, Lancaster University, Lancaster, UK
| | - Rosie Nelson
- School of Sociology, Politics, and International Studies, University of Bristol, Bristol, UK
| | - Emma Geijer-Simpson
- Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
| | - Liam Spencer
- Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
| |
Collapse
|
5
|
Romão DMM, Setti C, Arruda LHM, de Melo RC, de Araujo BC, Tan AR, DeMaio PN, Kuchenmüller T. Integration of evidence into Theory of Change frameworks in the healthcare sector: A rapid systematic review. PLoS One 2023; 18:e0282808. [PMID: 36893160 PMCID: PMC9997872 DOI: 10.1371/journal.pone.0282808] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2022] [Accepted: 02/22/2023] [Indexed: 03/10/2023] Open
Abstract
BACKGROUND Theory of Change (ToC) has become an established approach to design and evaluate interventions. While ToC should-in line with the growing international focus on evidence-informed health decision-making-consider explicit approaches to incorporate evidence, there is limited guidance on how this should be done. This rapid review aims to identify and synthesize the available literature on how to systematically use research evidence when developing or adapting ToCs in the health sector. METHODS A rapid review methodology using a systematic approach, was designed. Eight electronic databases were consulted to search for peer-reviewed and gray publications detailing tools, methods, and recommendations promoting the systematic integration of research evidence in ToCs. The included studies were compared, and the findings summarized qualitatively into themes to identify key principles, stages, and procedures, guiding the systematic integration of research evidence when developing or revising a ToC. RESULTS This review included 18 studies. The main sources from which evidence was retrieved in the ToC development process were institutional data, literature searches, and stakeholder consultation. There was a variety of ways of finding and using evidence in ToC. Firstly, the review provided an overview of existing definitions of ToC, methods applied in ToC development and the related ToC stages. Secondly, a typology of 7 stages relevant for evidence integration into ToCs was developed, outlining the types of evidence and research methods the included studies applied for each of the proposed stages. CONCLUSION This rapid review adds to the existing literature in two ways. First, it provides an up-to-date and comprehensive review of the existing methods for incorporating evidence into ToC development in the health sector. Second, it offers a new typology guiding any future endeavors of incorporating evidence into ToCs.
Collapse
Affiliation(s)
| | | | | | | | | | | | | | - Tanja Kuchenmüller
- Evidence to Policy and Impact, World Health Organization, Geneva, Switzerland
| |
Collapse
|
6
|
Cambon L. [Evaluating population health interventions : The contributions of theory-driven evaluations]. Rev Epidemiol Sante Publique 2022; 71:101398. [PMID: 36085120 DOI: 10.1016/j.respe.2022.08.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2022] [Revised: 07/29/2022] [Accepted: 08/01/2022] [Indexed: 10/14/2022] Open
Abstract
Population health intervention research has been characterized by the deployment of scientific methods designed to produce knowledge on policy and program interventions directly or indirectly involving the health sector, and potentially affecting population health. The proposed solutions encompass a multitude of interventions of variable types, scale, focus and implementation, rendering them particularly complex and difficult to understand. This complexity raises major conceptual and methodological issues because in reality, we assess not an intervention, but rather a set of interventional elements interacting with those specific to the context in which they appear. It is the interactions that produce effects, to the extent that it makes little sense to speak of an intervention, and more sense to consider an interventional system defined by the interactions. To grasp the numerous elements brought into play, it behooves us to amalgamate evaluation paradigms and approaches. In a precise context, theory-driven evaluations are of pronounced interest. This article presents the main principles of this type of evaluation by focusing on its capacity to shed light on the stakes involved in intervention/context interplay, and by putting forward conclusions transferable to population health research.
Collapse
Affiliation(s)
- L Cambon
- Centre Inserm U 1219, BPH, Université de Bordeaux, Bordeaux, France; CHU de Bordeaux, Bordeaux, France; ISPED, Université de Bordeaux, 146 Rue Leo Saignat, 33000 Bordeaux, France.
| |
Collapse
|
7
|
Gorman A, Rankin A, Hughes C, O'Dwyer M, Ryan C. Theoretically derived interventions aimed at improving appropriate polypharmacy in primary care: A systematic review. EXPLORATORY RESEARCH IN CLINICAL AND SOCIAL PHARMACY 2022; 7:100166. [PMID: 36039374 PMCID: PMC9418988 DOI: 10.1016/j.rcsop.2022.100166] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2022] [Revised: 08/02/2022] [Accepted: 08/02/2022] [Indexed: 11/16/2022] Open
Abstract
Background Polypharmacy (the use of multiple medications) is common in older patients and achieving a balance between appropriate and inappropriate polypharmacy is a challenge routinely faced by prescribers. It is recommended to incorporate the use of theory when developing complex interventions, but it is not known if theoretically derived interventions aimed at improving appropriate polypharmacy are effective. Objective This systematic review aimed to establish the overall effectiveness of theoretically derived interventions on improving appropriate polypharmacy and to investigate the degree to which theory informed intervention design. Methods Seven electronic databases were searched from inception to August 2021 including hand-searching of reference lists. Interventions developed using a theory, involving the use of a validated tool to assess prescribing, delivered in primary care to participants with a mean age of ≥65 years and prescribed ≥four medications, were included. Data was extracted independently by two reviewers. The Theory Coding Scheme (TCS) was applied to evaluate the use of theory; Risk of Bias (RoB) was assessed using the Cochrane RoB 2.0 tool. Results Two studies, one feasibility study and one randomised controlled trial (RCT) were included, and therefore overall effectiveness of the theoretically derived intervention could not be assessed. Theory used in development included the Theoretical Domains Framework and Reason's system-based risk management theory. The RCT was rated to have a high RoB. Based on the TCS, neither study used theory completely. Conclusion The effectiveness of theoretically derived interventions to improve appropriate polypharmacy in primary care could not be determined due to the small number of studies and their heterogeneity. Further incorporation of theory into intervention development is required to understand the effectiveness of this approach. Prospero registration: CRD42020157175.
Collapse
|
8
|
Warren EA, Melendez-Torres GJ, Bonell C. Are realist randomised controlled trials possible? A reflection on the INCLUSIVE evaluation of a whole-school, bullying-prevention intervention. Trials 2022; 23:82. [PMID: 35090541 PMCID: PMC8796527 DOI: 10.1186/s13063-021-05976-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2021] [Accepted: 12/09/2021] [Indexed: 11/10/2022] Open
Abstract
We previously proposed that realist randomised controlled trials could be used to evaluate how, for whom and under what conditions complex interventions can be used to activate mechanisms to improve health. While this idea was accepted by some, it was also met with resistance, particularly from some realist evaluators who believe that trials are inextricably positivist and dependent on constant conjunctions to understand causation, and that realist trials are unfeasible because participants and contexts will be insufficiently diverse to enable the testing of context-mechanism-outcome configurations. In this paper, we reflect on analyses of qualitative and quantitative data from the Initiating Change Locally in Bullying and Aggression through the School Environment (INCLSUIVE) trial, and whether these are useful and aligned with realism. We summarise the concerns expressed by realists and reflect on the philosophical and practical challenges that we encountered and whether or not they are related to the trial's design. Finally, we reflect on the trial's weaknesses and highlight areas that future researchers might consider when running realist trials. We conclude that realist randomised controlled trials are philosophically coherent, practically feasible, and can produce nuanced findings.
Collapse
Affiliation(s)
- Emily A Warren
- Department of Public Health, Environments and Society, Faculty of Public Health and Policy, London School of Hygiene & Tropical Medicine, 15-17 Tavistock Place, London, WC1H 9SH, UK.
| | - G J Melendez-Torres
- Institute of Health Research, College of Medicine and Health, University of Exeter, Exeter, UK
| | - Chris Bonell
- Department of Public Health, Environments and Society, Faculty of Public Health and Policy, London School of Hygiene & Tropical Medicine, 15-17 Tavistock Place, London, WC1H 9SH, UK
| |
Collapse
|
9
|
Dol J, Aston M, McMillan D, Tomblin Murphy G, Campbell-Yeo M. Participants perceptions of Essential Coaching for Every Mother, a Canadian text-message postpartum program: Process evaluation of a randomized controlled trial (Preprint). JMIR Form Res 2022; 6:e36821. [PMID: 35559855 PMCID: PMC9143775 DOI: 10.2196/36821] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2022] [Revised: 04/19/2022] [Accepted: 04/22/2022] [Indexed: 11/18/2022] Open
Abstract
Background “Essential Coaching for Every Mother” is a Canadian text message–based program that sends daily messages to mothers for 6 weeks after they give birth. There is a need to explore the program’s effectiveness in terms of the participants’ experience to guide refinement and modification. Objective This study aimed to describe the process evaluation of the Essential Coaching for Every Mother randomized controlled trial through an evaluation of the research implementation extent and quality. Methods Participants were recruited from Nova Scotia, Canada, between January 5 and August 1, 2021. Enrolled participants were randomized into the intervention or control group. Participants randomized to the intervention group received standard care along with the Essential Coaching for Every Mother program’s text messages related to newborn and maternal care for the first 6 weeks after giving birth, while the control group received standard care. Usage data were collected from the SMS text message program used, and participants completed web-based questionnaires at 6 weeks after birth. Quantitative data and qualitative responses to open-ended questions were used to triangulate findings. Quantitative data were summarized using means, SDs, and percentages, as appropriate, while qualitative data were analyzed using thematic analysis. Results Of the 295 unique initial contacts, 150 mothers were eligible and completed the baseline survey to be enrolled in the study (intervention, n=78; control, n=72). Of those randomized into the intervention group, 75 (96%) completed the 6-week follow-up survey to provide feedback on the program. In total, 48 (62%) intervention participants received all messages as designed in the Essential Coaching for Every Mother program, with participants who enrolled late missing on average 4.7 (range 1-12) messages. Intervention participants reported an 89% satisfaction rate with the program, and 100% of participants would recommend the program to other new mothers. Participants liked how the program made them feel, the format, appropriate timing of messages, and content while disliking the frequency of messages and gaps in content. Participants also provided suggestions for future improvement. Conclusions Our process evaluation has provided a comprehensive understanding of interest in the program as well as identified preference for program components. The findings of this study will be used to update future iterations of the Essential Coaching for Every Mother program. Trial Registration ClincalTrials.gov NCT04730570; https://clinicaltrials.gov/ct2/show/NCT04730570
Collapse
Affiliation(s)
- Justine Dol
- Faculty of Health, Dalhousie University, Halifax, NS, Canada
- St. Michael's Hospital, Toronto, ON, Canada
| | - Megan Aston
- School of Nursing, Faculty of Health, Dalhousie University, Halifax, NS, Canada
| | - Douglas McMillan
- Department of Pediatrics, Dalhousie University, Halifax, NS, Canada
| | | | - Marsha Campbell-Yeo
- School of Nursing, Faculty of Health, Dalhousie University, Halifax, NS, Canada
| |
Collapse
|
10
|
Taggart L, Marriott A, Cooper M, Atkinson D, Griffiths L, Ward C, Mullhall P. Developing curricular-content and systems-related impact indicators for intellectual disability awareness training for acute hospital settings: A modified International Delphi Survey. J Adv Nurs 2021; 78:2055-2074. [PMID: 34866230 DOI: 10.1111/jan.15123] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2021] [Revised: 10/04/2021] [Accepted: 11/18/2021] [Indexed: 11/28/2022]
Abstract
AIM To identify, and reach consensus on, curricular-content and delivery methods, as well as ways to maximize the impact of intellectual disability awareness training programmes in acute hospital settings. BACKGROUND With the continuing evidence of avoidable deaths and unwarranted variations in the quality of care to people with an intellectual disability in acute hospitals, it could be purported that current training provided to hospital staff appears to be making a minimal difference in the care provided to this population. DESIGN A two-round modified Delphi survey was conducted between June 2020-January 2021. METHODS International experts from primary healthcare and hospital settings, and intellectual disability health fields participated in the survey. Initial curricular-content items were developed from the literature, and based on the combined clinical and academic experience base of the authors. Items were evaluated in terms of agreement/consensus, importance and stability of responses. There were 57 expert responses in Round 1 and 45 in Round 2. RESULTS The consensus was reached with regard to 55 of 65 curricular-content indicators relating to Aims, Design, Content and Delivery. Ten curricular-content indicators failed to be agreed on relating to the mode of training delivery. With regard to systems-related impact indicators, 28 out of 31 reached consensus. The expert panel identified and agreed on seven system barriers that could obstruct the successful implementation of the awareness training programmes in acute hospital settings. CONCLUSIONS This is the first international Delphi survey to agree on curricular-content and identify systems-related facilitators for intellectual disability awareness training. Potential system barriers have been highlighted which could be addressed by systemic improvement. Implications for developing, and robustly testing the efficacy of, intellectual disability awareness training programmes are discussed, as are the implications for other cognitively impaired populations. IMPACT In order to maximize the impact, investment in acute hospital staff education will need to be accompanied by wider changes to systems and structures concerning the governance of service provision for people with an intellectual disability.
Collapse
Affiliation(s)
| | - Anna Marriott
- The National Development Team for Inclusion (NDTi), Bath, England
| | - Madeline Cooper
- The National Development Team for Inclusion (NDTi), Bath, England
| | - Dave Atkinson
- The National Development Team for Inclusion (NDTi), Bath, England
| | - Lynn Griffiths
- The National Development Team for Inclusion (NDTi), Bath, England
| | - Cathy Ward
- The National Development Team for Inclusion (NDTi), Bath, England
| | | |
Collapse
|
11
|
Melendez-Torres GJ, Warren E, Viner R, Allen E, Bonell C. Moderated mediation analyses to assess intervention mechanisms for impacts on victimisation, psycho-social problems and mental wellbeing: Evidence from the INCLUSIVE realist randomized trial. Soc Sci Med 2021; 279:113984. [PMID: 33975053 DOI: 10.1016/j.socscimed.2021.113984] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Revised: 04/05/2021] [Accepted: 04/29/2021] [Indexed: 11/28/2022]
Abstract
Realist evaluations aim to evaluate interventions by understanding the mechanisms they trigger, assessing not merely what works but what works for whom, under what conditions, and how. Significant disagreement in the literature exists as to whether randomized trials can be used as a tool for realist evaluation. INCLUSIVE, which was the first realist randomized trial explicitly designed as such, evaluated the impact of Learning Together, a school-based intervention for students aged 12-15 that included restorative practice, on bullying victimisation, mental wellbeing and psychological problems. Drawing on hypotheses generated through qualitative research, this analysis tested if school belonging was a mediator of intervention effects, and in which contexts. We estimated a series of fully longitudinal multilevel moderated mediation models including intervention allocation, student reports of school belonging at 24 months and victimisation and wellbeing outcomes at 36 months, and stratified on the basis of whether, at baseline, schools were: a) rated 'outstanding' for leadership, b) below the median for average levels of victimisation, and c) above the median on school inclusivity. Findings suggested that in unstratified models, belonging was not a mediator for any outcome; but in each of the strata defined above, belonging was a significant mediator at the student level. However, in the strata where belonging was not a mediator, the intervention still had a significant effect on each outcome. Analyses point to a strong but conditional role for belonging as a mediator of intervention pathways; in schools where belonging was not a mediator (e.g. above-median victimisation levels), other mechanisms may have been activated. This is consistent with a realist understanding of context-mechanism linkages generating outcomes. Our analyses suggest that realist evaluations can be pursued within randomized trials and that such analyses can offer more nuanced evidence regarding in which contexts interventions might effectively be implemented.
Collapse
Affiliation(s)
- G J Melendez-Torres
- Peninsula Technology Assessment Group, College of Medicine and Health, University of Exeter, Exeter, UK.
| | - Emily Warren
- Department of Public Health, Environments and Society, London School of Hygiene and Tropical Medicine, London, UK.
| | - Russell Viner
- Department of Social Science, UCL Institute of Child Health, London, UK.
| | - Elizabeth Allen
- Department of Medical Statistics, London School of Hygiene and Tropical Medicine, London, UK.
| | - Chris Bonell
- Department of Public Health, Environments and Society, London School of Hygiene and Tropical Medicine, London, UK.
| |
Collapse
|
12
|
Taggart L, Doherty AJ, Chauhan U, Hassiotis A. An exploration of lifestyle/obesity programmes for adults with intellectual disabilities through a realist lens: Impact of a 'context, mechanism and outcome' evaluation. JOURNAL OF APPLIED RESEARCH IN INTELLECTUAL DISABILITIES 2020; 34:578-593. [PMID: 33342030 DOI: 10.1111/jar.12826] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2019] [Revised: 07/16/2020] [Accepted: 10/05/2020] [Indexed: 11/28/2022]
Abstract
BACKGROUND Obesity is higher in people with intellectual disabilities. AIMS There are two aims of this explorative paper. Firstly, using a realist lens, to go beyond 'what works' and examine the 'context, mechanisms and outcomes' (CMO) of lifestyle/obesity programmes for this population. Second, using a logic model framework to inform how these programmes could be implemented within practice. METHOD We explored six-review papers and the individual lifestyle/obesity programmes that these papers reviewed using the CMO framework. RESULTS There were few theoretically underpinned, multi-component programmes that were effective in the short to long-term and many failed to explore the 'context and mechanisms'. We developed a logic model and engaged in two co-production workshops to refine this model. DISCUSSION Using a realist approach, programmes need to be underpinned by both individual and systems change theories, be multi-component, have a closer understanding of the interplay of the 'context and mechanisms', and co-designed using a logic model framework.
Collapse
Affiliation(s)
- Laurence Taggart
- Institute of Nursing & Health Research, Ulster University, Co Antrim, UK
| | - Alison Jayne Doherty
- Faculty of Health & Wellbeing, University of Central Lancashire (UCLan), Preston, UK
| | - Umesh Chauhan
- Faculty of Health & Wellbeing, University of Central Lancashire (UCLan), Preston, UK
| | | |
Collapse
|
13
|
Coorey G, Peiris D, Neubeck L, Redfern J. A realist evaluation approach to explaining the role of context in the impact of a complex eHealth intervention for improving prevention of cardiovascular disease. BMC Health Serv Res 2020; 20:764. [PMID: 32811480 PMCID: PMC7433103 DOI: 10.1186/s12913-020-05597-5] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2019] [Accepted: 07/29/2020] [Indexed: 01/05/2023] Open
Abstract
Background Reduction of cardiovascular disease (CVD) is a worldwide health priority and innovative uses of technology-based interventions may assist patients with improving prevention behaviours. Targeting these interventions to recipients most likely to benefit requires understanding how contexts of use influence responsiveness to the intervention, and how this interaction favours or discourages health behaviour. Using a realist evaluation approach, the aim of this study was to examine the contextual factors influencing behaviour change within a multi-feature eHealth intervention with personalised data integration from the primary care electronic health record (EHR). Methods Realist evaluation of qualitative data from the Consumer Navigation of Electronic Cardiovascular Tools (CONNECT) randomised trial (N = 934). Thirty-six participants from the intervention group (N = 486) who had completed 12 months of study follow-up were interviewed. Coding of transcripts was structured around configurations of contexts, mechanisms, and outcomes of intervention use. Contextual narratives were derived from thematic analysis of the interviews. Results Mechanisms favouring positive health behaviour occurred when participants responded to four interactive features of the intervention. Facilitating mechanisms included greater cognitive engagement whereby participants perceived value and benefit, and felt motivated, confident and incentivised. Participants moved from being unconcerned (or unaware) to more task-oriented engagement with personal CVD risk profile and prevention. Increased personalisation occurred when modifiable CVD risk factors became relatable to lifestyle behaviour; and experiences of feeling greater agency/self-efficacy emerged. Use and non-use of the intervention were influenced by four overarching narratives within the individual’s micro-level and meso-level environments: illness experiences; receptiveness to risk and prevention information; history of the doctor-patient relationship; and relationship with technology. Conclusions Intervention-context interactions are central to understanding how change mechanisms activate within complex interventions to exert their impact on recipients. Intervention use and non-use were context-dependent, underscoring the need for further research to target eHealth innovations to those most likely to benefit.
Collapse
Affiliation(s)
- Genevieve Coorey
- Faculty of Medicine and Health, School of Public Health, The University of Sydney, Sydney, New South Wales, Australia. .,The George Institute for Global Health, Sydney, New South Wales, Australia.
| | - David Peiris
- The George Institute for Global Health, Sydney, New South Wales, Australia.,Faculty of Medicine, The University of New South Wales, Sydney, New South Wales, Australia
| | - Lis Neubeck
- School of Health and Social Care, Edinburgh Napier University, Edinburgh, UK
| | - Julie Redfern
- The George Institute for Global Health, Sydney, New South Wales, Australia.,Faculty of Medicine and Health, Westmead Applied Research Centre, The University of Sydney, Sydney, New South Wales, Australia
| |
Collapse
|
14
|
Shenderovich Y, Cluver L, Eisner M, Murray AL. Moderators of treatment effects in a child maltreatment prevention programme in South Africa. CHILD ABUSE & NEGLECT 2020; 106:104519. [PMID: 32485323 DOI: 10.1016/j.chiabu.2020.104519] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/24/2019] [Revised: 02/29/2020] [Accepted: 04/23/2020] [Indexed: 06/11/2023]
Abstract
BACKGROUND Previous research has found mixed results on whether the most disadvantaged families benefit as much as less disadvantaged families from parenting interventions designed to reduce child maltreatment, and little in known in low-income settings. OBJECTIVE In this study, we test the effects of child, caregiver, household, and community characteristics as treatment moderators of intervention outcomes - child maltreatment and parenting practices. We test characteristics previously examined elsewhere as well as factors relevant to the South African context. PARTICIPANTS AND SETTING This analysis includes adolescents (ages 10-18) and their caregivers (N = 552 pairs) who participated in a randomised trial of a parenting programme in the Eastern Cape Province of South Africa. METHODS Data from the caregiver and adolescent standardised questionnaires collected at baseline, post-test (1-month post-intervention), and follow-up (5-9 months) were analysed using longitudinal multilevel analyses. We tested seven hypothesised moderators for each of the primary outcomes through interactions of treatment effect with baseline moderators. RESULTS No moderator effects were statistically significant after correcting for multiple comparisons testing. Hence, in line with several recent studies examining moderation effects in parenting programmes, our study suggests that parenting interventions aiming to reduce child maltreatment and promote parenting skills in low- and middle-income countries may be similarly effective for families facing various levels of economic, social, and health risk factors. CONCLUSIONS It may be useful to explicitly power trials for testing moderator effects, study different types of moderators and use person-centred analyses to further understand variations in treatment effects.
Collapse
Affiliation(s)
- Yulia Shenderovich
- Department of Social Policy and Intervention, Centre for Evidence-Based Intervention, University of Oxford, Barnett House, 32 Wellington Square, OX1 2ER, Oxford, United Kingdom; Institute of Criminology, University of Cambridge, Sidgwick Avenue, CB3 9DA, Cambridge, United Kingdom.
| | - Lucie Cluver
- Department of Social Policy and Intervention, Centre for Evidence-Based Intervention, University of Oxford, Barnett House, 32 Wellington Square, OX1 2ER, Oxford, United Kingdom; Department of Psychiatry and Mental Health, University of Cape Town, J-Block, Groote Schuur Hospital, Observatory, 7925, Cape Town, South Africa.
| | - Manuel Eisner
- Institute of Criminology, University of Cambridge, Sidgwick Avenue, CB3 9DA, Cambridge, United Kingdom; Jacobs Center for Productive Youth Development, University of Zurich, Andreasstrasse 15, 8050, Zürich, Switzerland.
| | - Aja Louise Murray
- Institute of Criminology, University of Cambridge, Sidgwick Avenue, CB3 9DA, Cambridge, United Kingdom; Department of Psychology, University of Edinburgh, Room F16, Psychology Building, 7 George Square, Edinburgh, EH8 9JZ, United Kingdom.
| |
Collapse
|
15
|
Palm R, Hochmuth A. What works, for whom and under what circumstances? Using realist methodology to evaluate complex interventions in nursing: A scoping review. Int J Nurs Stud 2020; 109:103601. [PMID: 32590248 DOI: 10.1016/j.ijnurstu.2020.103601] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2019] [Revised: 03/03/2020] [Accepted: 03/30/2020] [Indexed: 10/24/2022]
Abstract
BACKGROUND The Medical Research Council's (MRC) framework recommends basing every evaluation of a complex intervention on a theory that explains WHY an intervention works instead of focusing exclusively on the identification IF an intervention works. Theory-based evaluation approaches such as realist evaluations are promising in this respect. The aim of this scoping review is to examine current scientific literature with regard to the use of realist methodology in studies that evaluate complex nursing interventions. We also seek to answer the question whether realist evaluation studies of complex nursing interventions are embedded in the Medical Research Council's framework. METHODS A systematic database search in MEDLINE and CINAHL supplemented with snowballing techniques and a hand search in relevant nursing journals were performed between January and February 2018 and updated in July 2019. Both authors independently screened the title / abstract of studies that were initially identified and appraised the full texts if inclusion criteria were fulfilled. Inclusion criteria were the explicit use of realist methodology and a focus on the development or evaluation of complex nursing interventions. Results were summarized narratively. RESULTS A total of 28 articles from 22 studies, published between 2010 and 2019, were included. The majority of the studies included originate in the UK. Realist methodology was used by a variety of nursing disciplines, including midwifery, community / public health, critical care, palliative care, gerontological nursing, psychiatric nursing / mental health, and infection control. Study designs included case studies and realist reviews; two studies were conducted alongside a randomized controlled trial. All studies included used more than one method to develop their results; the majority of methods were qualitative by nature. Thirteen of the studies included aimed to develop a theory about what works, for whom and under what circumstances regarding the investigated complex nursing intervention. One study was clearly embedded in the Medical Research Council's framework. CONCLUSION Realist evaluation broadens the methodologies that have traditionally been used in nursing. Conceptual clarity on its principal tenets is still needed, as well as the methods used to assess them. Nursing scientists should be more critical when using novel methodology and contribute to academic debates about limitations. Embedding realist evaluations into the Medical Research Council's framework is in its infancy in nursing and needs more contributions in order to develop a common academic position within the profession. Tweetable abstract Realist methodology in nursing - review of the use, methods, limitations and embedding into the Medical Research Council's framework.
Collapse
Affiliation(s)
- Rebecca Palm
- Witten/Herdecke University, Faculty of Health, School of Nursing Science, Stockumer Str. 12, 58453 Witten, Germany; German Center for Neurodegenerative Diseases, Site Witten, Stockumer Str. 12, 58453 Witten, Germany.
| | - Alexander Hochmuth
- Witten/Herdecke University, Faculty of Health, School of Nursing Science, Stockumer Str. 12, 58453 Witten, Germany
| |
Collapse
|
16
|
Hatcher AM, McBride RS, Rebombo D, Munshi S, Khumalo M, Christofides N. Process evaluation of a community mobilization intervention for preventing men's partner violence use in peri-urban South Africa. EVALUATION AND PROGRAM PLANNING 2020; 78:101727. [PMID: 31639542 PMCID: PMC7264430 DOI: 10.1016/j.evalprogplan.2019.101727] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/08/2019] [Revised: 09/29/2019] [Accepted: 10/02/2019] [Indexed: 05/05/2023]
Abstract
Intimate partner violence (IPV) is experienced by one-third of women globally, yet few programs attempt to shift men's IPV perpetration. Community mobilization is a potential strategy for reducing men's IPV perpetration, but this has rarely been examined globally. We conducted a mixed-methods process evaluation alongside a trial testing community mobilization in peri-urban South Africa. We used in-depth interviews (n=114), participant observation (160 h), and monitoring and evaluation data to assess program delivery. Qualitative data (verbatim transcripts and observation notes) were managed in Dedoose using thematic coding and quantitative data were descriptively analyzed using Stata13. We learned that outreach elements of community mobilization were implemented with high fidelity, but that critical reflection and local advocacy were difficult to achieve. The context of a peri-urban settlement (characterized by poor infrastructure, migrancy, low education, social marginalization, and high levels of violence) severely limited intervention delivery, as did lack of institutional support for staff and activist volunteers. That community mobilization was poorly implemented may explain null trial findings; in the larger trial, the intervention failed to measurably reduce men's IPV perpetration. Designing community mobilization for resource-constrained settings may require additional financial, infrastructural, organizational, or political support to effectively engage community members and reduce IPV.
Collapse
Affiliation(s)
- Abigail M Hatcher
- School of Public Health, Faculty of Health Sciences, Unviersity of the Witwatersrand, Johannesburg, South Africa; Department of Medicine, University of California, San Francisco, United States.
| | - Ruari-Santiago McBride
- School of Public Health, Faculty of Health Sciences, Unviersity of the Witwatersrand, Johannesburg, South Africa
| | | | - Shehnaz Munshi
- School of Public Health, Faculty of Health Sciences, Unviersity of the Witwatersrand, Johannesburg, South Africa
| | | | - Nicola Christofides
- School of Public Health, Faculty of Health Sciences, Unviersity of the Witwatersrand, Johannesburg, South Africa
| |
Collapse
|
17
|
Armaou M, Konstantinidis S, Blake H. The Effectiveness of Digital Interventions for Psychological Well-Being in the Workplace: A Systematic Review Protocol. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2019; 17:E255. [PMID: 31905882 PMCID: PMC6981473 DOI: 10.3390/ijerph17010255] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/15/2019] [Revised: 12/23/2019] [Accepted: 12/25/2019] [Indexed: 11/29/2022]
Abstract
OBJECTIVE Psychological well-being has been associated with desirable individual and organisational outcomes. This systematic review aims to assess the effectiveness of digital interventions for the improvement of psychological well-being and/or the prevention/management of poor mental well-being in the workplace. METHODS This review protocol is registered in PROSPERO (CRD42019142428). Scientific databases including MEDLINE, Web of Science, CINAHL, PsycINFO, Cochrane Register of Controlled Trials (CENTRAL) and EMBASE will be searched for relevant studies published between January 1990 and July 2019. Studies will be included if they report specific primary and secondary outcomes of digital interventions delivered to adults in the workplace for the improvement of their psychological wellbeing and/or the prevention/management of poor mental well-being and were published in English. Following screening of titles and abstracts, full texts of potentially eligible papers will be screened in duplicate to identify studies that assess the effectiveness of those digital interventions. Discrepancies will be resolved through consensus or by consulting a third reviewer. An integrated narrative synthesis will assess included studies' findings, and a meta-analysis will be performed if included studies appear to be homogeneous. The "Cochrane Collaboration's Risk of Bias" tool and the JBI (Joanna Briggs Institute) Critical Appraisal Checklist for Quasi-Experimental Studies will be used to appraise included studies. CONCLUSION The results of this work will provide recommendations on the use of digital interventions for the promotion of psychological well-being at work. It will also guide the development of future workplace digital interventions and subsequent primary research in this field.
Collapse
Affiliation(s)
- Maria Armaou
- School of Health Sciences, University of Nottingham, Nottingham NG7 2HA, UK; (S.K.); (H.B.)
| | - Stathis Konstantinidis
- School of Health Sciences, University of Nottingham, Nottingham NG7 2HA, UK; (S.K.); (H.B.)
| | - Holly Blake
- School of Health Sciences, University of Nottingham, Nottingham NG7 2HA, UK; (S.K.); (H.B.)
- National Institute for Health Research (NIHR) Nottingham Biomedical Research Centre, Nottingham, NG7 2UH, UK
| |
Collapse
|
18
|
Cassidy C, Steenbeek A, Langille D, Martin-Misener R, Curran J. Designing an intervention to improve sexual health service use among university undergraduate students: a mixed methods study guided by the behaviour change wheel. BMC Public Health 2019; 19:1734. [PMID: 31878901 PMCID: PMC6933635 DOI: 10.1186/s12889-019-8059-4] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/07/2019] [Accepted: 12/11/2019] [Indexed: 11/25/2022] Open
Abstract
Introduction University undergraduate students are within the population at highest risk for acquiring sexually transmitted infections, unplanned pregnancy, and other negative health outcomes. Despite the availability of sexual health services at university health centres, many students delay or avoid seeking care. In this study, we describe how the Behaviour Change Wheel was used as a systematic approach to design an intervention to improve sexual health service use among university undergraduate students. Methods This paper describes the intervention development phase of a three-phased, sequential explanatory mixed methods study. Phases one and two included a quantitative and qualitative study that aimed to better understand students’ use of sexual health services. In phase three, we followed the Behaviour Change Wheel to integrate the quantitative and qualitative findings and conduct stakeholder consultation meetings to select intervention strategies, including intervention functions and behaviour change techniques. Results Key linkages between opportunity and motivation were found to influence students’ access of sexual health services. Stakeholders identified six intervention functions (education, environmental restructuring, enablement, modelling, persuasion, and incentivization) and 15 behaviour change techniques (information about health consequences, information about social and environmental consequences, feedback on behaviour, feedback on outcomes of behaviour, prompts/cues, self-monitoring of behaviour, adding objects to the environment, goal setting, problem solving, action planning, restructuring the social environment, restructuring the physical environment, demonstration of the behaviour, social support, credible source) as relevant to include in a toolbox of intervention strategies to improve sexual health service use. Conclusions This study details the use of the Behaviour Change Wheel to develop an intervention aimed at improving university students’ use of sexual health services. The Behaviour Change Wheel provided a comprehensive framework for integrating multiple sources of data to inform the selection of intervention strategies. Stakeholders can use these strategies to design and implement sexual health service interventions that are feasible within the context of their health centre. Future research is needed to test the effectiveness of the strategies at changing university students’ sexual health behaviour.
Collapse
Affiliation(s)
- Christine Cassidy
- Dalhousie University, School of Nursing, 5869 University Avenue, PO BOX 15000, Halifax, NS, B3H 4R2, Canada.
| | - Audrey Steenbeek
- Dalhousie University, School of Nursing, 5869 University Avenue, PO BOX 15000, Halifax, NS, B3H 4R2, Canada
| | - Donald Langille
- Dalhousie University, Department of Community Health and Epidemiology, Halifax, NS, Canada
| | - Ruth Martin-Misener
- Dalhousie University, School of Nursing, 5869 University Avenue, PO BOX 15000, Halifax, NS, B3H 4R2, Canada
| | - Janet Curran
- Dalhousie University, School of Nursing, 5869 University Avenue, PO BOX 15000, Halifax, NS, B3H 4R2, Canada
| |
Collapse
|
19
|
Kislov R, Pope C, Martin GP, Wilson PM. Harnessing the power of theorising in implementation science. Implement Sci 2019; 14:103. [PMID: 31823787 PMCID: PMC6905028 DOI: 10.1186/s13012-019-0957-4] [Citation(s) in RCA: 103] [Impact Index Per Article: 20.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2019] [Accepted: 11/28/2019] [Indexed: 01/05/2023] Open
Abstract
Theories occupy different positions in the scientific circle of enquiry as they vary in scope, abstraction, and complexity. Mid-range theories play a crucial bridging role between raw empirical observations and all-encompassing grand-theoretical schemes. A shift of perspective from ‘theories’ as products to ‘theorising’ as a process can enable empirical researchers to capitalise on the two-way relationships between empirical data and different levels of theory and contribute to the advancement of knowledge. This can be facilitated by embracing theoretically informative (in addition to merely theoretically informed) research, developing mechanism-based explanations, and broadening the repertoire of grand-theoretical orientations.
Collapse
Affiliation(s)
- Roman Kislov
- Room 5.24 Business School, Manchester Metropolitan University, Oxford Road, Manchester, M15 6BH, UK. .,University of Manchester, Manchester, UK.
| | | | | | | |
Collapse
|
20
|
Implementation, Mechanisms of Effect and Context of an Integrated Care Intervention for Vulnerable Families in Central Sydney Australia: A Research and Evaluation Protocol. Int J Integr Care 2019; 19:11. [PMID: 31367210 PMCID: PMC6659760 DOI: 10.5334/ijic.4217] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022] Open
Abstract
Introduction: In March 2014, the New South Wales (NSW) Government (Australia) announced the NSW Integrated Care Strategy. In response, a family-centred, population-based, integrated care initiative for vulnerable families and their children in Sydney, Australia was developed. The initiative was called Healthy Homes and Neighbourhoods. A realist translational social epidemiology programme of research and collaborative design is at the foundation of its evaluation. Theory and Method: The UK Medical Research Council (MRC) Framework for evaluating complex health interventions was adapted. This has four components, namely 1) development, 2) feasibility/piloting, 3) evaluation and 4) implementation. We adapted the Framework to include: critical realist, theory driven, and continuous improvement approaches. The modified Framework underpins this research and evaluation protocol for Healthy Homes and Neighbourhoods. Discussion: The NSW Health Monitoring and Evaluation Framework did not make provisions for assessment of the programme layers of context, or the effect of programme mechanism at each level. We therefore developed a multilevel approach that uses mixed-method research to examine not only outcomes, but also what is working for whom and why.
Collapse
|
21
|
A Critical Realist Translational Social Epidemiology Protocol for Concretising and Contextualising a "Theory of Neighbourhood Context, Stress, Depression, and the Developmental Origins of Health and Disease (DOHaD)", Sydney Australia. Int J Integr Care 2019; 19:8. [PMID: 31367207 PMCID: PMC6659581 DOI: 10.5334/ijic.3962] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/15/2023] Open
Abstract
Background: We will describe here a translational social epidemiology protocol for confirming a critical realist “Theory of Neighbourhood Context, Stress, Depression, and the Developmental Origins of Health and Disease (DOHaD). The approach will include the concretising and contextualising of the above causal theory into programme theories for child and adolescent interventions that aim to break intergenerational cycles of disadvantage and poor life outcomes. In undertaking this work we seek to advance realist translational methodology within the discipline of applied perinatal and paediatric social epidemiology. Theory and Methods: The research settings are in metropolitan Sydney. The design will be a longitudinal, multi-level, mixed method realist evaluation of applied programme interventions that seek to break the intergeneration cycle of social disadvantage and poor child health and developmental outcomes. The programme of research will consist of three components: 1) Operationalisation of the theory and designing of programme initiatives for implementation; 2) Evaluation of the translated programme and implementation theory using Theory of Change and critical realist evaluation; and 3) Theory Testing of realist hypotheses using both intensive and extensive critical realist research methods including realist structural modelling. Discussion: The proposed programme of research will assist in translating empirical explanatory theory building to theory driven interventions. The research will be situated in socially disadvantaged regions of Sydney where the local child and family inter-agencies will collaborate to design and implement new initiatives that address significant disparities in childhood development and adolescent outcomes attributed to neighbourhood circumstances, family stress and intergenerational cycles of disadvantage and poor mental health.
Collapse
|
22
|
Hennessy M, Heary C, Laws R, Van Rhoon L, Toomey E, Wolstenholme H, Byrne M. Health professional-delivered obesity prevention interventions during the first 1,000 days: A systematic review of external validity reporting. HRB Open Res 2019. [PMID: 32002513 DOI: 10.12688/hrbopenres.12924.1] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/08/2023] Open
Abstract
Background: Childhood obesity prevention interventions delivered by health professionals during the first 1,000 days show some evidence of effectiveness, particularly in relation to behavioural outcomes. External validity refers to how generalisable interventions are to populations or settings beyond those in the original study. The degree to which external validity elements are reported in such studies is unclear however. This systematic review aimed to determine the extent to which childhood obesity interventions delivered by health professionals during the first 1,000 days report on elements that can be used to inform generalizability across settings and populations. Methods: Eligible studies meeting study inclusion and exclusion criteria were identified through a systematic review of 11 databases and three trial registers. An assessment tool based on the RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance) framework was used to assess the external validity of included studies. It comprised five dimensions: reach and representativeness of individuals, reach and representativeness of settings, implementation and adaptation, outcomes for decision making maintenance and/or institutionalisation. Two authors independently assessed the external validity of 20% of included studies; discrepancies were resolved, and then one author completed assessments of the remaining studies. Results: In total, 39 trials involving 46 interventions published between 1999 and 2019 were identified. The majority of studies were randomized controlled trials (n=24). Reporting varied within and between dimensions. External validity elements that were poorly described included: representativeness of individuals and settings, treatment receipt, intervention mechanisms and moderators, cost effectiveness, and intervention sustainability and acceptability. Conclusions: Our review suggests that more emphasis is needed on research designs that consider generalisability, and the reporting of external validity elements in early life childhood obesity prevention interventions. Important gaps in external validity reporting were identified that could facilitate decisions around the translation and scale-up of interventions from research to practice. Registration: PROSPERO CRD42016050793 03/11/16.
Collapse
Affiliation(s)
- Marita Hennessy
- Health Behaviour Change Research Group, School of Psychology, National University of Ireland, Galway, Galway, Ireland
| | - Caroline Heary
- School of Psychology, National University of Ireland, Galway, Galway, Ireland
| | - Rachel Laws
- Institute for Physical Activity and Nutrition (IPAN), School of Exercise and Nutrition Sciences, Deakin University, Geelong, Victoria, Australia
| | - Luke Van Rhoon
- Health Behaviour Change Research Group, School of Psychology, National University of Ireland, Galway, Galway, Ireland
| | - Elaine Toomey
- Health Behaviour Change Research Group, School of Psychology, National University of Ireland, Galway, Galway, Ireland
| | - Hazel Wolstenholme
- School of Psychology, National University of Ireland, Galway, Galway, Ireland
| | - Molly Byrne
- Health Behaviour Change Research Group, School of Psychology, National University of Ireland, Galway, Galway, Ireland
| |
Collapse
|
23
|
Hennessy M, Heary C, Laws R, Van Rhoon L, Toomey E, Wolstenholme H, Byrne M. Health professional-delivered obesity prevention interventions during the first 1,000 days: A systematic review of external validity reporting. HRB Open Res 2019; 2:14. [PMID: 32002513 PMCID: PMC6973534 DOI: 10.12688/hrbopenres.12924.2] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/16/2019] [Indexed: 12/16/2022] Open
Abstract
Background: Childhood obesity prevention interventions delivered by health professionals during the first 1,000 days show some evidence of effectiveness, particularly in relation to behavioural outcomes. External validity refers to how generalisable interventions are to populations or settings beyond those in the original study. The degree to which external validity elements are reported in such studies is unclear however. This systematic review aimed to determine the extent to which childhood obesity interventions delivered by health professionals during the first 1,000 days report on elements that can be used to inform generalizability across settings and populations. Methods: Eligible studies meeting study inclusion and exclusion criteria were identified through a systematic review of 11 databases and three trial registers. An assessment tool based on the RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance) framework was used to assess the external validity of included studies. It comprised five dimensions: reach and representativeness of individuals, reach and representativeness of settings, implementation and adaptation, outcomes for decision making maintenance and/or institutionalisation. Two authors independently assessed the external validity of 20% of included studies; discrepancies were resolved, and then one author completed assessments of the remaining studies. Results: In total, 39 trials involving 46 interventions published between 1999 and 2019 were identified. The majority of studies were randomized controlled trials (n=24). Reporting varied within and between dimensions. External validity elements that were poorly described included: representativeness of individuals and settings, treatment receipt, intervention mechanisms and moderators, cost effectiveness, and intervention sustainability and acceptability. Conclusions: Our review suggests that more emphasis is needed on research designs that consider generalisability, and the reporting of external validity elements in early life childhood obesity prevention interventions. Important gaps in external validity reporting were identified that could facilitate decisions around the translation and scale-up of interventions from research to practice. Registration: PROSPERO CRD42016050793 03/11/16.
Collapse
Affiliation(s)
- Marita Hennessy
- Health Behaviour Change Research Group, School of Psychology, National University of Ireland, Galway, Galway, Ireland
| | - Caroline Heary
- School of Psychology, National University of Ireland, Galway, Galway, Ireland
| | - Rachel Laws
- Institute for Physical Activity and Nutrition (IPAN), School of Exercise and Nutrition Sciences, Deakin University, Geelong, Victoria, Australia
| | - Luke Van Rhoon
- Health Behaviour Change Research Group, School of Psychology, National University of Ireland, Galway, Galway, Ireland
| | - Elaine Toomey
- Health Behaviour Change Research Group, School of Psychology, National University of Ireland, Galway, Galway, Ireland
| | - Hazel Wolstenholme
- School of Psychology, National University of Ireland, Galway, Galway, Ireland
| | - Molly Byrne
- Health Behaviour Change Research Group, School of Psychology, National University of Ireland, Galway, Galway, Ireland
| |
Collapse
|
24
|
O'Brien R, Buston K, Wight D, McGee E, White J, Henderson M. A realist process evaluation of Enhanced Triple P for Baby and Mellow Bumps, within a Trial of Healthy Relationship Initiatives for the Very Early years (THRIVE): study protocol for a randomized controlled trial. Trials 2019; 20:351. [PMID: 31196169 PMCID: PMC6567913 DOI: 10.1186/s13063-019-3395-3] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/27/2018] [Accepted: 05/06/2019] [Indexed: 11/10/2022] Open
Abstract
Background THRIVE is a three-arm randomised controlled trial (RCT) that aims to evaluate whether antenatal and early postnatal interventions, Enhanced Triple B for Baby (ETPB) plus care as usual (CAU) or Mellow Bumps (MB) plus CAU (versus CAU alone), can: 1) improve the mental health and well-being of pregnant women with complex health and social care needs; 2) improve mother-infant bonding and interaction; 3) reduce child maltreatment; and 4) improve child language acquisition. This paper focuses on THRIVE’s realist process evaluation, which is carefully monitoring what is happening in the RCT. Methods Realistic evaluation provides the theoretical rationale for the process evaluation. We question: 1) how faithfully are MB and ETPB implemented? 2) What are the mechanisms by which they work, if they do, and who do they work for and how? 3) What contextual factors are necessary for the programmes to function, or might prevent them functioning? The mixed-methods design includes quantitative measures, which are pre- and post-training/intervention questionnaires for facilitators and mothers-to-be, and post-session evaluation forms. Qualitative data collection methods include participant observation of facilitator training and the delivery of a series of antenatal sessions in selected intervention groups (n = 3 for ETPB and n = 3 for MB), semi-structured interviews with facilitators, pregnant women, partners, and referring facilitators, and telephone interviews examining the content of the postnatal components of ETPB and MB. Discussion The findings of this process evaluation will help researchers and decision makers interpret the outcomes of THRIVE. It will provide a greater understanding of: how the interventions work (if they do); the extent and quality of their implementation; contextual factors facilitating and constraining intervention functioning; variations in response within and between subgroups of vulnerable parents; and benefits or unintended consequences of either intervention. Few studies to date have published detailed research protocols illustrating how realist process evaluation is designed and conducted as an integral part of a randomised controlled trial. Trial registration ISRCTN, ISRCTN21656568. Registered on 8 November 2013. Electronic supplementary material The online version of this article (10.1186/s13063-019-3395-3) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- Rosaleen O'Brien
- Glasgow Caledonian University, Psychology, Social Work and Allied Health Professionals, School of Health and Life Sciences, 4th Floor, George Moore Building, Cowcaddens Road, Glasgow, G4 0BA, UK
| | - Katie Buston
- MRC/CSO Social and Public Health Sciences Unit, University of Glasgow, 200 Renfield Street, Glasgow, G2 3AX, UK
| | - Daniel Wight
- MRC/CSO Social and Public Health Sciences Unit, University of Glasgow, 200 Renfield Street, Glasgow, G2 3AX, UK
| | - Elizabeth McGee
- Glasgow Caledonian University, Psychology, Social Work and Allied Health Professionals, School of Health and Life Sciences, 4th Floor, George Moore Building, Cowcaddens Road, Glasgow, G4 0BA, UK
| | - Jane White
- Public Health Sciences, NHS Health Scotland, Gyle Square, 1 Gyle Crescent, Edinburgh, EH12 9EB, UK
| | - Marion Henderson
- MRC/CSO Social and Public Health Sciences Unit, University of Glasgow, 200 Renfield Street, Glasgow, G2 3AX, UK.
| |
Collapse
|
25
|
Marchal B, Giralt AN, Sulaberidze L, Chikovani I, Abejirinde IOO. Designing and evaluating provider results-based financing for tuberculosis care in Georgia: a realist evaluation protocol. BMJ Open 2019; 9:e030257. [PMID: 30987995 PMCID: PMC6500265 DOI: 10.1136/bmjopen-2019-030257] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/07/2019] [Revised: 03/14/2019] [Accepted: 03/18/2019] [Indexed: 12/01/2022] Open
Abstract
INTRODUCTION In 2016, Georgian researchers and policymakers were developing a policy to improve the performance of the national tuberculosis (TB) control programme. The research programme 'Designing and Evaluating Provider Results-Based Financing for Tuberculosis Care in Georgia: Understanding Costs, Mechanisms of Effect and Impact (Results4TB)' was initiated to inform the policy formulation phase, document the policy implementation and assess the effectiveness, cost-effectiveness and the processes of change. To achieve this, the research team intends to combine an impact evaluation, a cost-effectiveness study and a realist evaluation (RE) within an overarching theory-informed design. This protocol is the RE component of the programme. METHODS A realist methodological approach will be adopted to guide the research design and evaluation. RE answers the question of 'what works in which conditions for whom?' and starts with the development of an initial programme theory (IPT). The IPT will feed into other phases of the realist research cycle (study design, data collection, data analysis and synthesis and theory refinement). Data will be collected in a multiple embedded case study design (five intervention and three control sites) through document reviews, in-depth interviews, non-participant observations and context mapping at facility and national levels. Additional data from other research components (cost-effectiveness and impact evaluation) will aid data triangulation. ETHICS AND DISSEMINATION The Institutional Review Boards of the National Centre for Disease Control and Public Health in Georgia (ref. IRB # 2018-019) and the Institute of Tropical Medicine, Antwerp (ref. IRB #- 1240/18) have granted ethical approval to the study. TRIAL REGISTRATION NUMBER ISRCTN14667607.
Collapse
Affiliation(s)
- Bruno Marchal
- Department of Public Health, Institute of Tropical Medicine, Health Systems and Equity Unit, Antwerp, Belgium
| | - Ariadna Nebot Giralt
- Department of Public Health, Institute of Tropical Medicine, Health Systems and Equity Unit, Antwerp, Belgium
| | | | | | | |
Collapse
|
26
|
Ford JA, Jones AP, Wong G, Barton G, Clark A, Sims E, Swart AM, Steel N. Improving primary care Access in Context and Theory (I-ACT trial): a theory-informed randomised cluster feasibility trial using a realist perspective. Trials 2019; 20:193. [PMID: 30947737 PMCID: PMC6449944 DOI: 10.1186/s13063-019-3299-2] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2018] [Accepted: 03/18/2019] [Indexed: 11/30/2022] Open
Abstract
Background Primary care access can be challenging for older, rural, socio-economically disadvantaged populations. Here we report the I-ACT cluster feasibility trial which aims to assess the feasibility of trial design and context-sensitive intervention to improve primary care access for this group and so expand existing theory. Methods Four general practices were recruited; three randomised to intervention and one to usual care. Intervention practices received £1500, a support manual and four meetings to develop local, innovative solutions to improve the booking system and transport. Patients aged over 64 years old and without household car access were recruited to complete questionnaires when booking an appointment or attending the surgery. Outcome measures at 6 months included: self-reported ease of booking an appointment and transport; health care use; patient activation; capability; and quality of life. A process evaluation involved observations and interviews with staff and participants. Results Thirty-four patients were recruited (26 female, eight male, mean age 81.6 years for the intervention group and 79.4 for usual care) of 1143 invited (3% response rate). Most were ineligible because of car access. Twenty-nine participants belonged to intervention practices and five to usual care. Practice-level data was available for all participants, but participant self-reported data was unavailable for three. Fifty-six appointment questionnaires were received based on 150 appointments (37.3%). Practices successfully designed and implemented the following context-sensitive interventions: Practice A: a stacked telephone system and promoting community transport; Practice B: signposting to community transport, appointment flexibility, mobility scooter charging point and promoting the role of receptionists; and Practice C: local taxi firm partnership and training receptionists. Practices found the process acceptable because it gave freedom, time and resource to be innovative or provided an opportunity to implement existing ideas. Data collection methods were acceptable to participants, but some found it difficult remembering to complete booking and appointment questionnaires. Expanded theory highlighted important mechanisms, such as reassurance, confidence, trust and flexibility. Conclusions Recruiting older participants without access to a car proved challenging. Retention of participants and practices was good but only about a third of appointment questionnaires were returned. This study design may facilitate a shift from one-size-fits-all interventions to more context-sensitive interventions. Trial registration ISRCTN18321951, Registered on 6 March 2017. Electronic supplementary material The online version of this article (10.1186/s13063-019-3299-2) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- John A Ford
- Department of Public Health and Primary Care, Faculty of Medicine and Health Sciences, Norwich Medical School, University of East Anglia, Chancellors Drive, Norwich, NR4 7TJ, UK.
| | - Andy P Jones
- Department of Public Health and Primary Care, Faculty of Medicine and Health Sciences, Norwich Medical School, University of East Anglia, Chancellors Drive, Norwich, NR4 7TJ, UK
| | - Geoff Wong
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
| | - Garry Barton
- Department of Public Health and Primary Care, Faculty of Medicine and Health Sciences, Norwich Medical School, University of East Anglia, Chancellors Drive, Norwich, NR4 7TJ, UK.,Norwich Clinical Trials Unit, University of East Anglia, Norwich, UK
| | - Allan Clark
- Department of Public Health and Primary Care, Faculty of Medicine and Health Sciences, Norwich Medical School, University of East Anglia, Chancellors Drive, Norwich, NR4 7TJ, UK.,Norwich Clinical Trials Unit, University of East Anglia, Norwich, UK
| | - Erika Sims
- Norwich Clinical Trials Unit, University of East Anglia, Norwich, UK
| | - Ann Marie Swart
- Norwich Clinical Trials Unit, University of East Anglia, Norwich, UK
| | - Nick Steel
- Department of Public Health and Primary Care, Faculty of Medicine and Health Sciences, Norwich Medical School, University of East Anglia, Chancellors Drive, Norwich, NR4 7TJ, UK
| |
Collapse
|
27
|
Mannell J, Davis K. Evaluating Complex Health Interventions With Randomized Controlled Trials: How Do We Improve the Use of Qualitative Methods? QUALITATIVE HEALTH RESEARCH 2019; 29:623-631. [PMID: 30871431 DOI: 10.1177/1049732319831032] [Citation(s) in RCA: 32] [Impact Index Per Article: 6.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
Qualitative methods are underutilized in health intervention evaluation, and overshadowed by the importance placed on randomized controlled trials (RCTs). This Commentary describes how innovative qualitative methods are being used as part of RCTs, drawing on articles included in a special issue of Qualitative Health Research on this topic. The articles' insights and a review of innovative qualitative methods described in trial protocols highlights a lack of attention to structural inequalities as a causal mechanism for understanding human behavior. We situate this gap within some well-known constraints of RCT methodologies, and a discussion of alternative RCT approaches that hold promise for bringing qualitative methods center stage in intervention evaluation, including adaptive designs, pragmatic trials, and realist RCTs. To address the power hierarchies of health evaluation research, however, we argue that a fundamental shift needs to take place away from a focus on RCTs and toward studies of health interventions.
Collapse
Affiliation(s)
| | - Katy Davis
- 1 University College London, London, United Kingdom
| |
Collapse
|
28
|
Cambon L, Terral P, Alla F. From intervention to interventional system: towards greater theorization in population health intervention research. BMC Public Health 2019; 19:339. [PMID: 30909891 PMCID: PMC6434858 DOI: 10.1186/s12889-019-6663-y] [Citation(s) in RCA: 46] [Impact Index Per Article: 9.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2018] [Accepted: 03/15/2019] [Indexed: 11/16/2022] Open
Abstract
Background Population health intervention research raises major conceptual and methodological issues. These require us to clarify what an intervention is and how best to address it. This paper aims to clarify the concepts of intervention and context and to propose a way to consider their interactions in evaluation studies, especially by addressing the mechanisms and using the theory-driven evaluation methodology. Main text This article synthesizes the notions of intervention and context. It suggests that we consider an “interventional system”, defined as a set of interrelated human and non-human contextual agents within spatial and temporal boundaries generating mechanistic configurations – mechanisms – which are prerequisites for change in health. The evaluation focal point is no longer the interventional ingredients taken separately from the context, but rather mechanisms that punctuate the process of change. It encourages a move towards theorization in evaluation designs, in order to analyze the interventional system more effectively. More particularly, it promotes theory-driven evaluation, either alone or combined with experimental designs. Conclusion Considering the intervention system, hybridizing paradigms in a process of theorization within evaluation designs, including different scientific disciplines, practitioners and intervention beneficiaries, may allow researchers a better understanding of what is being investigated and enable them to design the most appropriate methods and modalities for characterizing the interventional system. Evaluation methodologies should therefore be repositioned in relation to one another with regard to a new definition of “evidence”, repositioning practitioners’ expertise, qualitative paradigms and experimental questions in order to address the intervention system more profoundly.
Collapse
Affiliation(s)
- Linda Cambon
- Chaire Prévention, ISPED, Université Bordeaux, Bordeaux, France. .,Université Bordeaux, CHU, Inserm, Bordeaux Population Health Research Center, UMR 1219, CIC-EC 1401, Bordeaux, France.
| | - Philippe Terral
- Université Paul Sabatier, Toulouse 3, CRESCO EA 7419 - F2SMH, Toulouse, France
| | - François Alla
- Université Bordeaux, CHU, Inserm, Bordeaux Population Health Research Center, UMR 1219, CIC-EC 1401, Bordeaux, France
| |
Collapse
|
29
|
Gao X, Shen J, Wu H, Krenn HY. Evaluating program effects: Conceptualizing and demonstrating a typology. EVALUATION AND PROGRAM PLANNING 2019; 72:88-96. [PMID: 30321687 DOI: 10.1016/j.evalprogplan.2018.10.008] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/17/2018] [Revised: 09/19/2018] [Accepted: 10/03/2018] [Indexed: 06/08/2023]
Abstract
Outcome evaluation is very important for program evaluation and has been becoming increasingly so in the age of accountability. Typically, outcome evaluation is conducted for a single program from a single perspective. However, in a real-life situation, many programs exist in a system, and the effects could be viewed from various perspectives. The authors illustrate a typology of program effects in a system. It moves from the paradigm of a single program's single effect to that of a set of programs' multiple effects. Methodological implications are discussed.
Collapse
Affiliation(s)
- Xingyuan Gao
- Dept. of Education, Institute of Schooling Reform and Development, East China Normal University, China.
| | - Jianping Shen
- Dept. of Educational Leadership, Research and Technology, Western Michigan University, United States.
| | - Huang Wu
- Dept. of Educational Leadership, Research and Technology, Western Michigan University, United States.
| | | |
Collapse
|
30
|
Limbani F, Thorogood M, Gómez-Olivé FX, Kabudula C, Goudge J. Task shifting to improve the provision of integrated chronic care: realist evaluation of a lay health worker intervention in rural South Africa. BMJ Glob Health 2019; 4:e001084. [PMID: 30775003 PMCID: PMC6352781 DOI: 10.1136/bmjgh-2018-001084] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2018] [Revised: 12/05/2018] [Accepted: 12/13/2018] [Indexed: 11/04/2022] Open
Abstract
Introduction Task shifting is a potential solution to the shortage of healthcare personnel in low/middle-income countries, but contextual factors often dilute its effectiveness. We report on a task shifting intervention using lay health workers to support clinic staff in providing chronic disease care in rural South Africa, where the HIV epidemic and an ageing population have increased demand for care. Methods We conducted a realist evaluation in a cluster randomised controlled trial. We conducted observations in clinics, focus group discussions, in-depth interviews and patient exit interviews, and wrote weekly diaries to collect data. Results All clinic managers had to cope with an increasing but variable patient load and unplanned staff shortages, insufficient space, poorly functioning equipment and erratic supply of drugs. These conditions inevitably generated tension among staff. Lay health workers relieved the staff of some of their tasks and improved care for patients, but in some cases the presence of the lay health worker generated conflict with other staff. Where managers were able to respond to the changing circumstances, and to contain tension among staff, facilities were better able to meet patient needs. This required facility managers to be flexible, consultative and willing to act on suggestions, sometimes from junior staff and patients. While all facilities experienced an erratic supply of drugs and poorly maintained equipment, facilities where there was effective management, teamwork and sufficient space had better chronic care processes and a higher proportion of patients attending on their appointed day. Conclusion Lay health workers can be valuable members of a clinic team, and an important resource for managing increasing patient demand in primary healthcare. Task shifting will only be effective if clinic managers respond to the constantly changing system and contain conflict between staff. Strengthening facility-level management and leadership skills is a priority. Trial registration number ISRCTN12128227.
Collapse
Affiliation(s)
- Felix Limbani
- Centre for Health Policy, Faculty of Health Sciences, School of Public Health, University of the Witwatersrand, Johannesburg, South Africa
| | - Margaret Thorogood
- Centre for Health Policy, Faculty of Health Sciences, School of Public Health, University of the Witwatersrand, Johannesburg, South Africa.,Division of Health Sciences, University of Warwick, Coventry, UK
| | - Francesc Xavier Gómez-Olivé
- MRC/Wits Rural Public Health and Health Transitions Research Unit (Agincourt), Faculty of Health Sciences, School of Public Health, University of the Witwatersrand, Johannesburg, South Africa
| | - Chodziwadziwa Kabudula
- MRC/Wits Rural Public Health and Health Transitions Research Unit (Agincourt), Faculty of Health Sciences, School of Public Health, University of the Witwatersrand, Johannesburg, South Africa
| | - Jane Goudge
- Centre for Health Policy, Faculty of Health Sciences, School of Public Health, University of the Witwatersrand, Johannesburg, South Africa
| |
Collapse
|
31
|
Invisible work, actors, and knowledge: An analysis of a clinical trial for a vaccine to stop smoking. BIOSOCIETIES 2018. [DOI: 10.1057/s41292-018-0136-x] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022]
|
32
|
Ford JA, Jones A, Wong G, Clark A, Porter T, Steel N. Access to primary care for socio-economically disadvantaged older people in rural areas: exploring realist theory using structural equation modelling in a linked dataset. BMC Med Res Methodol 2018; 18:57. [PMID: 29914411 PMCID: PMC6006834 DOI: 10.1186/s12874-018-0514-x] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2017] [Accepted: 05/31/2018] [Indexed: 11/29/2022] Open
Abstract
Background Realist approaches seek to answer questions such as ‘how?’, ‘why?’, ‘for whom?’, ‘in what circumstances?’ and ‘to what extent?’ interventions ‘work’ using context-mechanism-outcome (CMO) configurations. Quantitative methods are not well-established in realist approaches, but structural equation modelling (SEM) may be useful to explore CMO configurations. Our aim was to assess the feasibility and appropriateness of SEM to explore CMO configurations and, if appropriate, make recommendations based on our access to primary care research. Our specific objectives were to map variables from two large population datasets to CMO configurations from our realist review looking at access to primary care, generate latent variables where needed, and use SEM to quantitatively test the CMO configurations. Methods A linked dataset was created by merging individual patient data from the English Longitudinal Study of Ageing and practice data from the GP Patient Survey. Patients registered in rural practices and who were in the highest deprivation tertile were included. Three latent variables were defined using confirmatory factor analysis. SEM was used to explore the nine full CMOs. All models were estimated using robust maximum likelihoods and accounted for clustering at practice level. Ordinal variables were treated as continuous to ensure convergence. Results We successfully explored our CMO configurations, but analysis was limited because of data availability. Two hundred seventy-six participants were included. We found a statistically significant direct (context to outcome) or indirect effect (context to outcome via mechanism) for two of nine CMOs. The strongest association was between ‘ease of getting through to the surgery’ and ‘being able to get an appointment’ with an indirect mediated effect through convenience (proportion of the indirect effect of the total was 21%). Healthcare experience was not directly associated with getting an appointment, but there was a statistically significant indirect effect through convenience (53% mediated effect). Model fit indices showed adequate fit. Conclusions SEM allowed quantification of CMO configurations and could complement other qualitative and quantitative techniques in realist evaluations to support inferences about strengths of relationships. Future research exploring CMO configurations with SEM should aim to collect, preferably continuous, primary data. Electronic supplementary material The online version of this article (10.1186/s12874-018-0514-x) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- John A Ford
- Norwich Medical School, University of East Anglia, Chancellor's Drive, Norwich, UK.
| | - Andy Jones
- Norwich Medical School, University of East Anglia, Chancellor's Drive, Norwich, UK
| | - Geoff Wong
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
| | - Allan Clark
- Norwich Medical School, University of East Anglia, Chancellor's Drive, Norwich, UK
| | - Tom Porter
- School of Health Sciences, University of East Anglia, Norwich, UK
| | - Nick Steel
- Norwich Medical School, University of East Anglia, Chancellor's Drive, Norwich, UK
| |
Collapse
|
33
|
Schepman S, Valentijn P, Bruijnzeels M, Maaijen M, de Bakker D, Batenburg R, de Bont A. Do project management and network governance contribute to inter-organisational collaboration in primary care? A mixed methods study. BMC Health Serv Res 2018; 18:427. [PMID: 29879971 PMCID: PMC5992666 DOI: 10.1186/s12913-018-3169-8] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2016] [Accepted: 04/30/2018] [Indexed: 11/13/2022] Open
Abstract
Background The need for organisational development in primary care has increased as it is accepted as a means of curbing rising costs and responding to demographic transitions. It is only within such inter-organisational networks that small-scale practices can offer treatment to complex patients and continuity of care. The aim of this paper is to explore, through the experience of professionals and patients, whether, and how, project management and network governance can improve the outcomes of projects which promote inter-organisational collaboration in primary care. Methods This paper describes a study of projects aimed at improving inter-organisational collaboration in Dutch primary care. The projects' success in project management and network governance was monitored by interviewing project leaders and board members on the one hand, and improvement in the collaboration by surveying professionals and patients on the other. Both qualitative and quantitative methods were applied to assess the projects. These were analysed, finally, using multi-level models in order to account for the variation in the projects, professionals and patients. Results Successful network governance was associated positively with the professionals’ satisfaction with the collaboration; but not with improvements in the quality of care as experienced by patients. Neither patients nor professionals perceived successful project management as associated with the outcomes of the collaboration projects. Conclusions This study shows that network governance in particular makes a difference to the outcomes of inter-organisational collaboration in primary care. However, project management is not a predictor for successful inter-organisational collaboration in primary care. Electronic supplementary material The online version of this article (10.1186/s12913-018-3169-8) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- Sanneke Schepman
- NIVEL, Netherlands Institute for Health Services Research, PO Box 1568, 3500 BN, Utrecht, the Netherlands.
| | - Pim Valentijn
- Department of Health Services Research, School for Public Health and Primary Care, Faculty of Health, Medicine and Life Sciences, Maastricht University, PO Box 616, 6200MD, Maastricht, the Netherlands.,Integrated Care Evaluation, Essenburgh Research & Consultancy, Hierden, The Netherlands
| | - Marc Bruijnzeels
- Jan van Es Institute, Netherlands Expert Centre Integrated Primary Care, Randstad 2145-a, 1314 BG, Almere, The Netherlands
| | - Marlies Maaijen
- Institute of Health Policy & Management, Erasmus University Rotterdam, PO Box 1738, 3000 DR, Rotterdam, the Netherlands
| | - Dinny de Bakker
- NIVEL, Netherlands Institute for Health Services Research, PO Box 1568, 3500 BN, Utrecht, the Netherlands
| | - Ronald Batenburg
- NIVEL, Netherlands Institute for Health Services Research, PO Box 1568, 3500 BN, Utrecht, the Netherlands.,Department of Sociology, Radboud University Nijmegen, P.O. Box 9104, 6500 HE, Nijmegen, the Netherlands
| | - Antoinette de Bont
- Institute of Health Policy & Management, Erasmus University Rotterdam, PO Box 1738, 3000 DR, Rotterdam, the Netherlands
| |
Collapse
|
34
|
The potential role for sociologists in designing RCTs and of RCTs in refining sociological theory: A commentary on Deaton and Cartwright. Soc Sci Med 2018; 210:29-31. [PMID: 29724460 DOI: 10.1016/j.socscimed.2018.04.045] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
|
35
|
Bonell C, Moore G, Warren E, Moore L. Are randomised controlled trials positivist? Reviewing the social science and philosophy literature to assess positivist tendencies of trials of social interventions in public health and health services. Trials 2018; 19:238. [PMID: 29673378 PMCID: PMC5907717 DOI: 10.1186/s13063-018-2589-4] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2018] [Accepted: 03/12/2018] [Indexed: 11/10/2022] Open
Abstract
Background We have previously proposed that trials of social interventions can be done within a “realist” research paradigm. Critics have countered that such trials are irredeemably positivist and asked us to explain our philosophical position. Methods We set out to explore what is meant by positivism and whether trials adhere to its tenets (of necessity or in practice) via a narrative literature review of social science and philosophical discussions of positivism, and of the trials literature and three case studies of trials. Results The philosophical literature described positivism as asserting: (1) the epistemic primacy of sensory information; (2) the requirement that theoretical terms equate with empirical terms; (3) the aim of developing universal laws; and (4) the unity of method between natural and social sciences. Regarding (1), it seems that rather than embodying the epistemic primacy of sensory data, randomised controlled trials (RCTs) of social interventions in health embrace an anti-positivist approach aiming to test hypotheses derived deductively from prior theory. Considering (2), while some RCTs of social interventions appear to limit theorisation to concepts with empirical analogues, others examine interventions underpinned by theories engaging with mechanisms and contextual contingencies not all of which can be measured. Regarding (3), while some trialists and reviewers in the health field do limit their role to estimating statistical trends as a mechanistic form of generalisation, this is not an inevitable feature of RCT-based research. Trials of social interventions can instead aim to generalise at the level of theory which specifies how mechanisms are contingent on context. In terms of (4), while RCTs are used to examine biomedical as well as social interventions in health, RCTs of social interventions are often distinctive in using qualitative analyses of data on participant accounts to examine questions of meaning and agency not pursued in the natural sciences. Conclusion We conclude that the most appropriate paradigm for RCTs of social interventions is realism not positivism.
Collapse
Affiliation(s)
- Chris Bonell
- London School of Hygiene & Tropical Medicine, Faculty of Public Health and Policy, 15-17 Tavistock Place, London, WC1H-9SH, UK.
| | - Graham Moore
- Cardiff University, School of Social Sciences, Cardiff, UK
| | - Emily Warren
- London School of Hygiene & Tropical Medicine, Faculty of Public Health and Policy, 15-17 Tavistock Place, London, WC1H-9SH, UK
| | - Laurence Moore
- University of Glasgow, Institute of Health & Wellbeing, Glasgow, UK
| |
Collapse
|
36
|
Valentijn PP, Pereira FA, Ruospo M, Palmer SC, Hegbrant J, Sterner CW, Vrijhoef HJM, Ruwaard D, Strippoli GFM. Person-Centered Integrated Care for Chronic Kidney Disease: A Systematic Review and Meta-Analysis of Randomized Controlled Trials. Clin J Am Soc Nephrol 2018; 13:375-386. [PMID: 29438975 PMCID: PMC5967678 DOI: 10.2215/cjn.09960917] [Citation(s) in RCA: 30] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2017] [Accepted: 12/15/2017] [Indexed: 12/29/2022]
Abstract
BACKGROUND AND OBJECTIVES The effectiveness of person-centered integrated care strategies for CKD is uncertain. We conducted a systematic review and meta-analysis of randomized, controlled trials to assess the effect of person-centered integrated care for CKD. DESIGN, SETTING, PARTICIPANTS, & MEASUREMENTS We searched MEDLINE, Embase, and Cochrane Central Register of Controlled Trials (from inception to April of 2016), and selected randomized, controlled trials of person-centered integrated care interventions with a minimum follow-up of 3 months. Random-effects meta-analysis was used to assess the effect of person-centered integrated care. RESULTS We included 14 eligible studies covering 4693 participants with a mean follow-up of 12 months. In moderate quality evidence, person-centered integrated care probably had no effect on all-cause mortality (relative risk [RR], 0.86; 95% confidence interval [95% CI], 0.68 to 1.08) or health-related quality of life (standardized mean difference, 0.02; 95% CI, -0.05 to 0.10). The effects on renal replacement therapy (RRT) (RR, 1.00; 95% CI, 0.65 to 1.55), serum creatinine levels (mean difference, 0.59 mg/dl; 95% CI, -0.38 to 0.36), and eGFR (mean difference, 1.51 ml/min per 1.73 m2; 95% CI, -3.25 to 6.27) were very uncertain. Quantitative analysis suggested that person-centered integrated care interventions may reduce all-cause hospitalization (RR, 0.38; 95% CI, 0.15 to 0.95) and improve BP control (RR, 1.20; 95% CI, 1.00 to 1.44), although the certainty of the evidence was very low. CONCLUSIONS Person-centered integrated care may have little effect on mortality or quality of life. The effects on serum creatinine, eGFR, and RRT are uncertain, although person-centered integrated care may lead to fewer hospitalizations and improved BP control.
Collapse
Affiliation(s)
- Pim P Valentijn
- Due to the number of contributing authors, the affiliations are provided in the Supplemental Material
| | | | | | | | | | | | | | | | | |
Collapse
|
37
|
Porter S, McConnell T, Clarke M, Kirkwood J, Hughes N, Graham-Wisener L, Regan J, McKeown M, McGrillen K, Reid J. A critical realist evaluation of a music therapy intervention in palliative care. BMC Palliat Care 2017; 16:70. [PMID: 29221475 PMCID: PMC5723094 DOI: 10.1186/s12904-017-0253-5] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2017] [Accepted: 11/28/2017] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Music therapy is increasingly used as an adjunct therapy to support symptom management in palliative care. However, studies to date have paid little attention to the processes that lead to changes in patient outcomes. To fill this gap, we examined the processes and experiences involved in the introduction of music therapy as an adjunct complementary therapy to palliative care in a hospice setting in the United Kingdom (UK). METHODS Using a realistic evaluation approach, we conducted a qualitative study using a variety of approaches. These consisted of open text answers from patients (n = 16) on how music therapy helped meet their needs within one hospice in Northern Ireland, UK. We also conducted three focus groups with a range of palliative care practitioners (seven physicians, seven nursing staff, two social workers and three allied health professionals) to help understand their perspectives on music therapy's impact on their work setting, and what influences its successful implementation. This was supplemented with an interview with the music therapist delivering the intervention. RESULTS Music therapy contains multiple mechanisms that can provide physical, psychological, emotional, expressive, existential and social support. There is also evidence that the hospice context, animated by a holistic approach to healthcare, is an important facilitator of the effects of music therapy. Examination of patients' responses helped identify specific benefits for different types of patients. CONCLUSIONS There is a synergy between the therapeutic aims of music therapy and those of palliative care, which appealed to a significant proportion of participants, who perceived it as effective.
Collapse
Affiliation(s)
- Sam Porter
- Department of Social Sciences and Social Work, Bournemouth University, Poole, Northern Ireland
| | - Tracey McConnell
- School of Nursing and Midwifery, Queen’s University Belfast, Belfast, Northern Ireland
| | - Mike Clarke
- School of Medicine, Dentistry and Biomedical Sciences, Queen’s University Belfast, Belfast, Northern Ireland
| | - Jenny Kirkwood
- Every Day Harmony Music Therapy, Belfast, Northern Ireland
| | - Naomi Hughes
- Every Day Harmony Music Therapy, Belfast, Northern Ireland
| | | | - Joan Regan
- Marie Curie Hospice Belfast, Marie Curie, Northern Ireland
| | - Miriam McKeown
- Marie Curie Hospice Belfast, Marie Curie, Northern Ireland
| | | | - Joanne Reid
- School of Nursing and Midwifery, Queen’s University Belfast, Belfast, Northern Ireland
| |
Collapse
|
38
|
Lee H, Lamb SE. Advancing Physical Therapist Interventions by Investigating Causal Mechanisms. Phys Ther 2017; 97:1119-1121. [PMID: 29121341 DOI: 10.1093/ptj/pzx095] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/05/2017] [Accepted: 09/07/2017] [Indexed: 11/14/2022]
Affiliation(s)
- Hopin Lee
- H. Lee, PhD, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics Rheumatology and Musculoskeletal Sciences, University of Oxford, Windmill Road, Headington, Oxford, UK OX3 7LD, and School of Medicine and Public Health, University of Newcastle, New South Wales, Australia
| | - Sarah E Lamb
- S.E. Lamb, MSc, MA, MCSP, Grad Dip Statistics, DPhil, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford
| |
Collapse
|
39
|
Moore GF, Evans RE. What theory, for whom and in which context? Reflections on the application of theory in the development and evaluation of complex population health interventions. SSM Popul Health 2017; 3:132-135. [PMID: 29302610 PMCID: PMC5742639 DOI: 10.1016/j.ssmph.2016.12.005] [Citation(s) in RCA: 125] [Impact Index Per Article: 17.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2016] [Revised: 09/22/2016] [Accepted: 12/12/2016] [Indexed: 01/10/2023] Open
Abstract
Recent years have seen a growing emphasis on the value of building and testing middle range theory throughout the development and evaluation of complex population health interventions. We agree that a coherent theoretical basis for intervention development, and use of evaluation to test key causal assumptions and build theory, are crucial. However, in this editorial, we argue that such recommendations have often been operationalised in somewhat simplistic terms with potentially perverse consequences, and that an uncritical assumption that an intervention explicitly based on theory is inherently superior carries significant risks. We first argue that the drive for theory-based approaches may have exacerbated a propensity to select ‘off-the-shelf’ theories, leading to the selection of inappropriate theories which distract attention from the mechanisms through which a problem is actually sustained. Second, we discuss a tendency toward over-reliance on individual-level theorising. Finally, we discuss the relatively slow progress of population health intervention research in attending to issues of context, and the ecological fit of interventions with the systems whose functioning they attempt to change. We argue that while researchers should consider a broad range of potential theoretical perspectives on a given population health problem, citing a popular off-the-shelf theory as having informed an intervention and its evaluation does not inherently make for better science. Before identifying or developing a theory of change, researchers should develop a clear understanding of how the problem under consideration is created and sustained in context. A broader conceptualisation of theory that reaches across disciplines is vital if theory is to enhance, rather than constrain, the contribution of intervention research. Finally, intervention researchers need to move away from viewing interventions as discrete packages of components which can be described in isolation from their contexts, and better understand the systems into which change is being introduced.
Collapse
Affiliation(s)
- Graham F Moore
- Centre for the Development and Evaluation of Complex Interventions for Public Health Improvement (DECIPHer), School of Social Sciences, Cardiff University, Wales, United Kingdom
| | - Rhiannon E Evans
- Centre for the Development and Evaluation of Complex Interventions for Public Health Improvement (DECIPHer), School of Social Sciences, Cardiff University, Wales, United Kingdom
| |
Collapse
|
40
|
Hansen ABG, Jones A. Advancing 'real-world' trials that take account of social context and human volition. Trials 2017; 18:531. [PMID: 29126449 PMCID: PMC5681782 DOI: 10.1186/s13063-017-2286-8] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2017] [Accepted: 10/03/2017] [Indexed: 12/03/2022] Open
Abstract
Background The recent paper in Trials by Porter and colleagues highlights the utility of applying a critical realism approach in randomised trials, an approach central to the Medical Research Council’s (MRC) Framework for the Development and Evaluation of Complex Healthcare Interventions. The MRC framework offers a pragmatic step towards a more open systems approach that bridges randomised evaluation with social context and human agency in an effort to improve the generalisability of trial outcomes. Main body The MRC framework has contributed to the proliferation of a more open systems approach in health research; however, the broader acceptance of the realist approach to health research does not seem to be emulated by norms in research fund allocation, which largely prioritises laboratory-based research. Conclusion This commentary is simply a plea, to those who make the strategic decisions regarding allocation of research funding, to support all phases of health intervention research in complex systems that contribute to the development of effective, translational and sustainable interventions in the promotion of health.
Collapse
Affiliation(s)
- Anders Blædel Gottlieb Hansen
- Strategic Research and Development Support, Metropolitan University College, Tagensvej 18, Copenhagen N, 2200, Denmark.
| | - Allan Jones
- Bachelor's Degree in Global Nutrition and Health, Faculty of Health and Technology, Metropolitan University College, Sigurdsgade 26, Copenhagen N, 2200, Denmark
| |
Collapse
|
41
|
The challenge of complexity in evaluating health policies and programs: the case of women's participatory groups to improve antenatal outcomes. BMC Health Serv Res 2017; 17:687. [PMID: 28962648 PMCID: PMC5622457 DOI: 10.1186/s12913-017-2627-z] [Citation(s) in RCA: 25] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2016] [Accepted: 09/21/2017] [Indexed: 11/24/2022] Open
Abstract
Background During the last years, randomized designs have been promoted as the cornerstone of evidence-based policymaking. Also in the field of community participation, Random Control Trials (RCTs) have been the dominant design, used for instance to examine the contribution of community participation to health improvement. We aim at clarifying why RCTs and related (quasi-) experimental designs may not be the most appropriate approach to evaluate such complex programmes. Results We argue that the current methodological debate could be more fruitful if it would start from the position that the choice of designs should fit the nature of the program and research questions rather than be driven by methodological preferences. We present how realist evaluation, a theory-driven approach to research and evaluation, is a relevant methodology that could be used to assess whether and how community participation works. Using the realist evaluation approach to examine the relationship between participation and action of women groups and antenatal outcomes would enable evaluators to examine in detail the underlying mechanisms which influence actual practices and outcomes, as well as the context conditions required to make it work. Conclusions Realist research in fact allows opening the black boxes of “community” and “participation” in order to examine the role they play in ensuring cost-effective, sustainable interventions. This approach yields important information for policy makers and programme managers considering how such programs could be implemented in their own setting.
Collapse
|
42
|
Porter S, McConnell T, Reid J. The possibility of critical realist randomised controlled trials. Trials 2017; 18:133. [PMID: 28327182 PMCID: PMC5359862 DOI: 10.1186/s13063-017-1855-1] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/02/2016] [Accepted: 02/19/2017] [Indexed: 11/17/2022] Open
Abstract
Background Some realists have criticised randomised controlled trials for their inability to explain the causal relations that they identify; to take into account the influence of the social context of the interventions they evaluate; and to account for individual difference. However, among realists, there is controversy over whether it is possible to improve trials by making them realist, or whether realism and the philosophical assumptions underlying trials are incompatible. This paper contributes to the debate in Trials on this issue. The debate thus far has concentrated on the possibility of combining trial methodology with that of realist evaluation. Main body We concur with the contention that it is not feasible to combine randomised controlled trial design with the realist evaluation approach. However, we argue that a different variant of realism, critical realism, provides a more appropriate theoretical grounding for realist trials. In contrast to realist evaluation, which regards social mechanisms as an amalgam of social resources and people’s reasoning, critical realism insists on their distinction. It does so on the basis of its assertion of the need to distinguish between social structures (in which resources lie) and human agency (which is at least partly guided by reasoning). From this perspective, conceiving of social mechanisms as external to participants can be seen as a valid methodological strategy for supplementing the exclusive concentration of trials on outcomes. While accepting realist evaluation’s insistence that causality in open systems involves a configuration of multiple generative mechanisms, we adopt the critical realist interpretation of the experimental method, which sees it as creating artificial closure in order to identify the effects of specific causal mechanisms. If randomised controlled trials can be regarded as epidemiological proxies that substitute probabilistic controls over extraneous factors for closed experiments, their examination of the powers of discrete mechanisms through observation of the variation of outcomes is appropriate. Conclusion While there are still issues to be resolved, critical realist randomised controlled trials are possible and have the potential to overcome some of the difficulties faced by traditional trial designs in accounting for the influence of social context and individual interpretation.
Collapse
Affiliation(s)
- Sam Porter
- Department of Social Sciences and Social Work, Bournemouth University, R203, Royal London House, Christchurch Road, BH1 3LT, Bournemouth, UK
| | - Tracey McConnell
- School of Nursing and Midwifery, Queen's University Belfast, Belfast, BT7 1NN, UK
| | - Joanne Reid
- School of Nursing and Midwifery, Queen's University Belfast, Belfast, BT7 1NN, UK.
| |
Collapse
|
43
|
Bonell C, Warren E, Fletcher A, Viner R. Realist trials and the testing of context-mechanism-outcome configurations: a response to Van Belle et al. Trials 2016; 17:478. [PMID: 27716303 PMCID: PMC5045599 DOI: 10.1186/s13063-016-1613-9] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2016] [Accepted: 09/18/2016] [Indexed: 11/24/2022] Open
Abstract
Background Van Belle et al. argue that our attempt to pursue realist evaluation via a randomised trial will be fruitless because we misunderstand realist ontology (confusing intervention mechanisms with intervention activities and with statistical mediation analyses) and because RCTs cannot comprehensively examine how and why outcome patterns are caused by mechanisms triggered in specific contexts. Methods Through further consideration of our trial methods, we explain more fully how we believe complex social interventions work and what realist evaluation should aim to do within a trial. Results Like other realists, those undertaking realist trials assume that: social interventions provide resources which local actors may draw on in actions that can trigger mechanisms; these mechanisms may interact with contextual factors to generate outcomes; and data in the ‘empirical’ realm can be used to test hypotheses about mechanisms in the ‘real’ realm. Whether or not there is sufficient contextual diversity to test such hypotheses is a contingent not a necessary feature of trials. Previous exemplars of realist evaluation have compared empirical data from intervention and control groups to test hypotheses about real mechanisms. There is no inevitable reason why randomised trials should not also be able to do so. Random allocation merely ensures the comparability of such groups without necessarily causing evaluation to lapse from a realist into a ‘positivist’ or ‘post-positivist’ paradigm. Conclusions Realist trials are ontologically and epistemologically plausible. Further work is required to assess whether they are feasible and useful but such work should not be halted on spurious philosophical grounds.
Collapse
Affiliation(s)
- Chris Bonell
- London School of Hygiene & Tropical Medicine, 15-17 Tavistock Place, London, WC1H 9SH, UK.
| | - Emily Warren
- London School of Hygiene & Tropical Medicine, 15-17 Tavistock Place, London, WC1H 9SH, UK
| | - Adam Fletcher
- Cardiff University, Glamorgan Building, King Edward VII, Cardiff, CF10 3WT, UK
| | - Russell Viner
- University College London Institute of Child Health, 30 Guilford Street, London, WC1N 1EH, UK
| |
Collapse
|
44
|
Van Belle S, Wong G, Westhorp G, Pearson M, Emmel N, Manzano A, Marchal B. Can "realist" randomised controlled trials be genuinely realist? Trials 2016; 17:313. [PMID: 27387202 PMCID: PMC4936237 DOI: 10.1186/s13063-016-1407-0] [Citation(s) in RCA: 46] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2015] [Accepted: 05/03/2016] [Indexed: 11/30/2022] Open
Abstract
In this paper, we respond to a paper by Jamal and colleagues published in Trials in October 2015 and take an opportunity to continue the much-needed debate about what applied scientific realism is. The paper by Jamal et al. is useful because it exposes the challenges of combining a realist evaluation approach (as developed by Pawson and Tilley) with the randomised controlled trial (RCT) design. We identified three fundamental differences that are related to paradigmatic differences in the treatment of causation between post-positivist and realist logic: (1) the construct of mechanism, (2) the relation between mediators and moderators on one hand and mechanisms and contexts on the other hand, and (3) the variable-oriented approach to analysis of causation versus the configurational approach. We show how Jamal et al. consider mechanisms as observable, external treatments and how their approach reduces complex causal processes to variables. We argue that their proposed RCT design cannot provide a truly realist understanding. Not only does the proposed realist RCT design not deal with the RCT’s inherent inability to “unpack” complex interventions, it also does not enable the identification of the dynamic interplay among the intervention, actors, context, mechanisms and outcomes, which is at the core of realist research. As a result, the proposed realist RCT design is not, as we understand it, genuinely realist in nature.
Collapse
Affiliation(s)
- Sara Van Belle
- Department of Public Health, Institute of Tropical Medicine, Nationalestraat 155, B-2000, Antwerp, Belgium.
| | - Geoff Wong
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
| | | | - Mark Pearson
- NIHR Collaboration for Applied Health Research and Care South West Peninsula (PenCLAHRC), Institute of Health Research, University of Exeter Medical School, Exeter, UK
| | - Nick Emmel
- School of Sociology & Social Policy, University of Leeds, Leeds, UK
| | - Ana Manzano
- Centre for Health, Technologies & Social Practice, School of Sociology & Social Policy, University of Leeds, Leeds, UK
| | - Bruno Marchal
- Department of Public Health, Institute of Tropical Medicine, Nationalestraat 155, B-2000, Antwerp, Belgium
| |
Collapse
|
45
|
Fletcher A, Jamal F, Moore G, Evans RE, Murphy S, Bonell C. Realist complex intervention science: Applying realist principles across all phases of the Medical Research Council framework for developing and evaluating complex interventions. EVALUATION (LONDON, ENGLAND : 1995) 2016; 22:286-303. [PMID: 27478401 PMCID: PMC4946011 DOI: 10.1177/1356389016652743] [Citation(s) in RCA: 140] [Impact Index Per Article: 17.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/06/2023]
Abstract
The integration of realist evaluation principles within randomised controlled trials ('realist RCTs') enables evaluations of complex interventions to answer questions about what works, for whom and under what circumstances. This allows evaluators to better develop and refine mid-level programme theories. However, this is only one phase in the process of developing and evaluating complex interventions. We describe and exemplify how social scientists can integrate realist principles across all phases of the Medical Research Council framework. Intervention development, modelling, and feasibility and pilot studies need to theorise the contextual conditions necessary for intervention mechanisms to be activated. Where interventions are scaled up and translated into routine practice, realist principles also have much to offer in facilitating knowledge about longer-term sustainability, benefits and harms. Integrating a realist approach across all phases of complex intervention science is vital for considering the feasibility and likely effects of interventions for different localities and population subgroups.
Collapse
Affiliation(s)
| | | | | | | | | | - Chris Bonell
- London School of Hygiene & Tropical Medicine, UK
| |
Collapse
|
46
|
Bonell C, Harden A, Fletcher A, Viner R. In memoriam: the contributions to public health research of Farah Jamal, BA, MSc, PhD. J Epidemiol Community Health 2016; 70:329-30. [PMID: 26838125 DOI: 10.1136/jech-2016-207197] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/03/2022]
Affiliation(s)
- Chris Bonell
- Department of Social Science, University College London Institute of Education, London, UK
| | - Angela Harden
- Institute for Health and Human Development, University of East London, London, UK
| | - Adam Fletcher
- School of Social Sciences, Cardiff University, Cardiff, UK
| | - Russell Viner
- Institute of Child Health, University College London, London, UK
| |
Collapse
|