1
|
Shepardson RL, Weisberg RB, Wade M, Maisto SA, Funderburk JS. Brief modular anxiety intervention for primary care: Hybrid I pilot randomized controlled trial of feasibility, acceptability, effectiveness, and implementation potential. J Affect Disord 2024; 361:497-507. [PMID: 38810782 DOI: 10.1016/j.jad.2024.05.107] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/23/2024] [Revised: 04/30/2024] [Accepted: 05/21/2024] [Indexed: 05/31/2024]
Abstract
BACKGROUND Anxiety is highly prevalent, but undertreated, in primary care. Brief, non-pharmacological interventions are needed. Modular Anxiety Skills Training (MAST), a cognitive-behavioral anxiety intervention, was developed for primary care and tailored for a Veteran sample (MAST-V). The purpose of this mixed methods pilot study was to evaluate MAST-V's feasibility, acceptability, and implementation potential, and preliminarily examine its effectiveness compared to Primary Care Behavioral Health (PCBH) usual care. METHODS This hybrid I randomized controlled trial (conducted 2019-2021) assigned 35 primary care patients (Mage = 47, 17 % female, 27 % racial/ethnic minority) with clinically significant anxiety symptoms to receive MAST-V or PCBH usual care. Participants completed validated measures of anxiety symptoms and functional impairment at 0, 4, 8, 12, and 16 weeks. RESULTS Participants attended more sessions in MAST-V than usual care. After necessary adjustments to reduce session duration, MAST-V will likely fit within PCBH practice parameters. Participants in both conditions valued treatment, but treatment satisfaction, credibility, and therapeutic alliance were higher for MAST-V. Study therapists achieved high treatment fidelity and rated MAST-V as highly feasible, acceptable, and appropriate for PCBH. They identified ways to address potential barriers to implementation. MAST-V was more effective than usual care in reducing anxiety symptoms and impairment. LIMITATIONS This was a small pilot study at a single site using study therapists. Results should be considered preliminary until replicated in a full-scale clinical trial. CONCLUSIONS This brief modular anxiety intervention, which was designed with implementation in mind, may help to address the anxiety treatment gap in primary care.
Collapse
Affiliation(s)
- Robyn L Shepardson
- Center for Integrated Healthcare, Syracuse VA Medical Center. USA; Department of Psychology, Syracuse University. USA.
| | - Risa B Weisberg
- VA, Boston Healthcare System. USA; Department of Psychiatry, Boston University Chobanian & Avedisian School of Medicine. USA; Department of Family Medicine, Alpert Medical School, Brown University. USA
| | - Michael Wade
- Center for Integrated Healthcare, Syracuse VA Medical Center. USA
| | - Stephen A Maisto
- Center for Integrated Healthcare, Syracuse VA Medical Center. USA; Department of Psychology, Syracuse University. USA
| | - Jennifer S Funderburk
- Center for Integrated Healthcare, Syracuse VA Medical Center. USA; Department of Psychology, Syracuse University. USA; Department of Psychiatry, University of Rochester. USA
| |
Collapse
|
2
|
Buckley PR, Combs KM, Drewelow KM, Hubler BL, Lain MA. Validity Evidence for an Observational Fidelity Measure to Inform Scale-Up of Evidence-Based Interventions. EVALUATION REVIEW 2024:193841X241248864. [PMID: 38687041 DOI: 10.1177/0193841x241248864] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/02/2024]
Abstract
As evidence-based interventions are scaled, fidelity of implementation, and thus effectiveness, often wanes. Validated fidelity measures can improve researchers' ability to attribute outcomes to the intervention and help practitioners feel more confident in implementing the intervention as intended. We aim to provide a model for the validation of fidelity observation protocols to guide future research studying evidence-based interventions scaled-up under real-world conditions. We describe a process to build evidence of validity for items within the Session Review Form, an observational tool measuring fidelity to interactive drug prevention programs such as the Botvin LifeSkills Training program. Following Kane's (2006) assumptions framework requiring that validity evidence be built across four areas (scoring, generalizability, extrapolation, and decision), confirmatory factor analysis supported the hypothesized two-factor structure measuring quality of delivery (seven items assessing how well the material is implemented) and participant responsiveness (three items evaluating how well the intervention is received), and measurement invariance tests suggested the structure held across grade level and schools serving different student populations. These findings provide some evidence supporting the extrapolation assumption, though additional research is warranted since a more complete overall depiction of the validity argument is needed to evaluate fidelity measures.
Collapse
Affiliation(s)
- Pamela R Buckley
- Institute of Behavioral Science, University of Colorado Boulder, Boulder, USA
| | - Katie Massey Combs
- Institute of Behavioral Science, University of Colorado Boulder, Boulder, USA
| | - Karen M Drewelow
- Institute of Behavioral Science, University of Colorado Boulder, Boulder, USA
| | - Brittany L Hubler
- Institute of Behavioral Science, University of Colorado Boulder, Boulder, USA
| | - Marion Amanda Lain
- Institute of Behavioral Science, University of Colorado Boulder, Boulder, USA
| |
Collapse
|
3
|
Coelho VA, Marchante M, Brás P. Positive Attitude Upper Middle School social and emotional learning program: influences of implementation quality on program outcome. Front Psychol 2023; 14:1172517. [PMID: 37265944 PMCID: PMC10229836 DOI: 10.3389/fpsyg.2023.1172517] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2023] [Accepted: 04/26/2023] [Indexed: 06/03/2023] Open
Abstract
Introduction There is an increased call for studies analyzing how implementation quality influences Social and Emotional Learning (SEL) program effectiveness. Methods The current dissemination study analyzed the effectiveness of the Positive Attitude Upper Middle School SEL program on a Portuguese nationwide sample composed of 813 middle school students (7th and 8th grade; 51.7% boys; Mage = 12.41, S.D. = 1.06), from 36 classrooms (Mclassroom = 22.58; S.D. = 2.86), distributed between the control group (179 students), and three intervention groups (643 students) that reflected low, middle, and high implementer experience (respectively, Gulbenkian Academies of Knowledge, Positive Attitude Cadaval and Positive Attitude Torres Vedras). Dosage and fidelity (as implementation quality dimensions), gender, and classroom size (as individual and classroom-level variables) were also analyzed. Self-report questionnaires were administered pre- and post-intervention and at a 6-month follow-up. Results Multilevel models were employed, and results showed that participating in the PAUMS SEL program led to more positive trajectories in self-control, social awareness, relationship skills, and responsible decision-making when compared with control groups. Regarding implementation quality, only the implementer's experience impacted the effectiveness of the PAUMS SEL program; students in the Gulbenkian Academies of Knowledge intervention group displayed a less positive trajectory in self-control than students in the Positive Attitude Torres Vedras intervention group. Discussion Altogether, results showed that the PAUMS SEL program is ready for dissemination in Portugal, although a higher level of implementer experience is needed to achieve the best effectiveness, and they support the importance of analyzing implementer experience in SEL programs' effectiveness studies.
Collapse
Affiliation(s)
- Vítor Alexandre Coelho
- Center for Research in Psychology for Positive Development, Lusíada University of Porto, Porto, Portugal
- Académico de Torres Vedras, Torres Vedras, Portugal
| | - Marta Marchante
- Center for Research in Psychology for Positive Development, Lusíada University of Porto, Porto, Portugal
- Académico de Torres Vedras, Torres Vedras, Portugal
| | | |
Collapse
|
4
|
Tschida JE, Drahota A. Fidelity to the ACT SMART Toolkit: an instrumental case study of implementation strategy fidelity. Implement Sci Commun 2023; 4:52. [PMID: 37194052 DOI: 10.1186/s43058-023-00434-2] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2022] [Accepted: 05/02/2023] [Indexed: 05/18/2023] Open
Abstract
BACKGROUND Evidence-based practices (EBPs) are shown to improve a variety of outcomes for autistic children. However, EBPs often are mis-implemented or not implemented in community-based settings where many autistic children receive usual care services. A blended implementation process and capacity-building implementation strategy, developed to facilitate the adoption and implementation of EBPs for autism spectrum disorder (ASD) in community-based settings, is the Autism Community Toolkit: Systems to Measure and Adopt Research-based Treatments (ACT SMART Toolkit). Based on an adapted Exploration, Adoption decision, Preparation, Implementation, Sustainment (EPIS) Framework, the multi-phased ACT SMART Toolkit is comprised of (a) implementation facilitation, (b) agency-based implementation teams, and (c) a web-based interface. In this instrumental case study, we developed and utilized a method to evaluate fidelity to the ACT SMART Toolkit. This study responds to the need for implementation strategy fidelity evaluation methods and may provide evidence supporting the use of the ACT SMART Toolkit. METHODS We used an instrumental case study approach to assess fidelity to the ACT SMART Toolkit during its pilot study with six ASD community agencies located in southern California. We assessed adherence, dose, and implementation team responsiveness for each phase and activity of the toolkit at both an aggregate and individual agency level. RESULTS Overall, we found that adherence, dose, and implementation team responsiveness to the ACT SMART Toolkit were high, with some variability by EPIS phase and specific activity as well as by ASD community agency. At the aggregate level, adherence and dose were rated notably lowest during the preparation phase of the toolkit, which is a more activity-intensive phase of the toolkit. CONCLUSIONS This evaluation of fidelity to the ACT SMART Toolkit, utilizing an instrumental case study design, demonstrated the potential for the strategy to be used with fidelity in ASD community-based agencies. Findings related to the variability of implementation strategy fidelity in the present study may also inform future adaptations to the toolkit and point to broader trends of how implementation strategy fidelity may vary by content and context.
Collapse
Affiliation(s)
- Jessica E Tschida
- Department of Psychology, Michigan State University, 316 Physics Rd, East Lansing, MI, 48824, USA.
| | - Amy Drahota
- Department of Psychology, Michigan State University, 316 Physics Rd, East Lansing, MI, 48824, USA
- Child and Adolescent Services Research Center (CASRC), 3665 Kearny Villa Road, Suite 200N, San Diego, CA, 92123, USA
| |
Collapse
|
5
|
Flierman M, Vriend E, Leemhuis AG, Engelbert RHH, Jeukens-Visser M. Development and evaluation of a fidelity tool in a post-discharge responsive parenting intervention program for very preterm born children. EVALUATION AND PROGRAM PLANNING 2023; 99:102299. [PMID: 37187117 DOI: 10.1016/j.evalprogplan.2023.102299] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/27/2022] [Revised: 03/17/2023] [Accepted: 04/29/2023] [Indexed: 05/17/2023]
Abstract
The TOP program is a fully implemented responsive parenting intervention for very preterm born infants. Fidelity monitoring of interventions is important for preserving program adherence, impact outcomes and to make evidence-based adaptations. The aim of this study was to develop a fidelity tool for the TOP program following an iterative and co-creative process and subsequently evaluate the reliability of the tool. Three consecutive phases were carried out. Phase I: Initial development and pilot testing two methods namely self-report and video based observation. Phase II: Adaptations and refinements. Phase III: Evaluation of the psychometric properties of the tool based on 20 intervention videos rated by three experts.The interrater reliability of the adherence and competence subscales was good (ICC.81 to .84) and varied from moderate to excellent for specific items (ICC between .51 and .98). The FITT displayed a high correlation (Spearman's rho.79 to.82) between the subscales and total impression item. The co-creative and iterative process resulted in a clinical useful and reliable tool for evaluating fidelity in the TOP program. This study offers insights in the practical steps in the development of a fidelity assessment tool which can be used by other intervention developers.
Collapse
Affiliation(s)
- Monique Flierman
- Amsterdam UMC, University of Amsterdam, Department of Rehabilitation, Amsterdam Reproduction and development, Meibergdreef 9, Amsterdam, the Netherlands.
| | - Eline Vriend
- Amsterdam UMC, University of Amsterdam, Department of Rehabilitation, Amsterdam Reproduction and development, Meibergdreef 9, Amsterdam, the Netherlands
| | - Aleid G Leemhuis
- Emma Children's Hospital, Department of Pediatrics, Amsterdam UMC, University of Amsterdam, Meibergdreef 9, Amsterdam, Netherlands
| | - Raoul H H Engelbert
- Amsterdam UMC, University of Amsterdam, Department of Rehabilitation, Amsterdam Reproduction and development, Meibergdreef 9, Amsterdam, the Netherlands; Centre of Expertise Urban Vitality, Faculty of Health, Amsterdam University of Applied Sciences, Amsterdam, the Netherlands; Emma Children's Hospital, Department of Pediatrics, Amsterdam UMC, University of Amsterdam, Meibergdreef 9, Amsterdam, Netherlands
| | - Martine Jeukens-Visser
- Amsterdam UMC, University of Amsterdam, Department of Rehabilitation, Amsterdam Reproduction and development, Meibergdreef 9, Amsterdam, the Netherlands
| |
Collapse
|
6
|
Schwartz A, Hwang IT. Supporting young adults with intellectual/developmental disabilities to deliver a peer mentoring intervention: Evaluating fidelity and resources required. JOURNAL OF APPLIED RESEARCH IN INTELLECTUAL DISABILITIES 2022; 35:1336-1347. [PMID: 35730690 DOI: 10.1111/jar.13022] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2021] [Revised: 05/20/2022] [Accepted: 05/27/2022] [Indexed: 11/28/2022]
Abstract
BACKGROUND Peer-delivered interventions are a best practice in mental health, yet no such intervention exists for young adults (YA) with intellectual/developmental disabilities with co-occurring mental health conditions. We evaluated YA with intellectual/developmental disabilities' ability to deliver a novel peer mentoring intervention with fidelity and the supports received. METHODS We coded audio-recorded mentoring sessions to evaluate if 4 mentors adhered to 'content' (e.g., psychoeducation) and 'quality' (e.g., validation) fidelity criteria (codes: yes/no). We conducted content analysis of mentor support logs and interviews with mentors', mentors' parents and teachers to describe the supports mentors received. RESULTS Average fidelity for content criteria (M = 73.3%) was higher than quality criteria (M = 60.0%). Weekly support addressed logistics, delivering content, interpersonal interactions, emotional support, professionalism, and organisation. Family members and teachers rarely provided additional support. CONCLUSIONS With support, YA can deliver a peer mentoring intervention addressing mental health. Additional training activities will be developed to improve fidelity.
Collapse
Affiliation(s)
- Ariel Schwartz
- Center for Psychiatric Rehabilitation, Boston University, Boston, Massachusetts, USA
| | - I-Ting Hwang
- Institute of Health Behaviors and Community Sciences, College of Public Health, National Taiwan University, Taipei, Taiwan
| |
Collapse
|
7
|
Galea Holmes MN, Wileman V, Hassan S, Denning J, Critchley D, Norton S, McCracken LM, Godfrey E. Physiotherapy informed by Acceptance and Commitment Therapy for chronic low back pain: A mixed-methods treatment fidelity evaluation. Br J Health Psychol 2022; 27:935-955. [PMID: 35118763 PMCID: PMC9540449 DOI: 10.1111/bjhp.12583] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2021] [Revised: 01/10/2022] [Indexed: 12/16/2022]
Abstract
OBJECTIVES A randomized controlled trial of a new type of Physiotherapy informed by Acceptance and Commitment Therapy (PACT), found that it improved functioning in people with chronic low back pain compared to usual physiotherapy care. Fidelity evaluation is necessary to understand trial processes and outcomes. This study evaluated PACT treatment fidelity including delivery, receipt, and enactment. DESIGN A mixed-methods study nested within a randomized controlled trial was conducted. METHODS A total of 72 (20% of total) PACT treatment audio files were independently assessed by two raters, according to a novel framework developed to measure PACT treatment content adherence, therapeutic alliance, ACT competence, and treatment enactment. Interview transcripts from 19 trial participants randomized to PACT were analysed thematically for evidence of treatment receipt and enactment. RESULTS PACT physiotherapists delivered treatment as intended with high content adherence and satisfactory therapeutic alliance, but ACT competence was low. Qualitative findings indicated participant receipt of 11/17 and enactment of 3/17 components; 89% (n = 17) and 47% (n = 9) of participants reported treatment receipt and enactment of at least one component, respectively. CONCLUSIONS This mixed-methods study of PACT treatment demonstrated high fidelity reflecting treatment content delivery and receipt, and therapeutic alliance. There was some evidence of treatment enactment in participants with chronic low back pain. Low ACT competence could be addressed through additional support and adaptations to therapeutic processes for delivery by physiotherapists.
Collapse
Affiliation(s)
- Melissa N Galea Holmes
- Health Psychology Section, Department of Psychology, Institute of Psychiatry, Psychology and Neuroscience (IoPPN), King's College London, UK
| | - Vari Wileman
- Health Psychology Section, Department of Psychology, Institute of Psychiatry, Psychology and Neuroscience (IoPPN), King's College London, UK
| | - Shaira Hassan
- Health Psychology Section, Department of Psychology, Institute of Psychiatry, Psychology and Neuroscience (IoPPN), King's College London, UK
| | - Julie Denning
- Health Psychology Section, Department of Psychology, Institute of Psychiatry, Psychology and Neuroscience (IoPPN), King's College London, UK
| | - Duncan Critchley
- Department of Physiotherapy, School of Population Health and Environmental Sciences, Faculty of Life Sciences and Medicine, King's College London, UK
| | - Sam Norton
- Health Psychology Section, Department of Psychology, Institute of Psychiatry, Psychology and Neuroscience (IoPPN), King's College London, UK
| | - Lance M McCracken
- Health Psychology Section, Department of Psychology, Institute of Psychiatry, Psychology and Neuroscience (IoPPN), King's College London, UK
| | - Emma Godfrey
- Health Psychology Section, Department of Psychology, Institute of Psychiatry, Psychology and Neuroscience (IoPPN), King's College London, UK.,Department of Physiotherapy, School of Population Health and Environmental Sciences, Faculty of Life Sciences and Medicine, King's College London, UK
| |
Collapse
|
8
|
Distilling Check-in/Check-Out into Its Core Practice Elements Through an Expert Consensus Process. SCHOOL MENTAL HEALTH 2022; 14:695-708. [PMID: 35103078 PMCID: PMC8791689 DOI: 10.1007/s12310-021-09495-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 12/06/2021] [Indexed: 11/06/2022]
Abstract
Check-in/Check-out (CICO) is a widely implemented evidence-based program for supporting students with at-risk levels of social and emotional behavior concerns. It is comprised of several core features described in the previous literature, including practice elements, which are the specific actions that are delivered directly to students, and implementation components, which are actions that support the implementation by adults. Practice elements and implementation components are both important to implementation but have been combined and conflated in descriptions of CICO implementation. Well-defined and differentiated practice elements could provide improved clarity in communicating implementation expectations to front-line implementers as well as support future research into essential active ingredients and measures of front-line intervention fidelity. The purpose of the present study was to distill, differentiate, and operationally define the student-facing practice elements of CICO. A panel of research experts and practice experts participated in a three-round modified e-Delphi process that led to the identification and operational definition of 19 discreet practice elements organized into five domains. Results are discussed in terms in implications for future development of measures of commitment and intervention fidelity, future research into active ingredients of CICO, and in terms of how well-defined practice elements can improve communication of implementation expectations for front-line implementers of CICO such as teachers.
Collapse
|
9
|
Rosenberg BM, Kodish T, Cohen ZD, Gong-Guy E, Craske MG. A Novel Peer-to-Peer Coaching Program to Support Digital Mental Health: Design and Implementation. JMIR Ment Health 2022; 9:e32430. [PMID: 35080504 PMCID: PMC8829693 DOI: 10.2196/32430] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/02/2021] [Revised: 11/21/2021] [Accepted: 11/22/2021] [Indexed: 01/09/2023] Open
Abstract
Many individuals in need of mental health services do not currently receive care. Scalable programs are needed to reduce the burden of mental illness among those without access to existing providers. Digital interventions present an avenue for increasing the reach of mental health services. These interventions often rely on paraprofessionals, or coaches, to support the treatment. Although existing programs hold immense promise, providers must ensure that treatments are delivered with high fidelity and adherence to the treatment model. In this paper, we first highlight the tension between the scalability and fidelity of mental health services. We then describe the design and implementation of a peer-to-peer coach training program to support a digital mental health intervention for undergraduate students within a university setting. We specifically note strategies for emphasizing fidelity within our scalable framework, including principles of learning theory and competency-based supervision. Finally, we discuss future applications of this work, including the potential adaptability of our model for use within other contexts.
Collapse
Affiliation(s)
- Benjamin M Rosenberg
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, United States
| | - Tamar Kodish
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, United States.,Semel Institute for Neuroscience and Human Behavior, University of California, Los Angeles, Los Angeles, CA, United States
| | - Zachary D Cohen
- Department of Psychiatry and Biobehavioral Sciences, University of California, Los Angeles, Los Angeles, CA, United States
| | - Elizabeth Gong-Guy
- Semel Institute for Neuroscience and Human Behavior, University of California, Los Angeles, Los Angeles, CA, United States
| | - Michelle G Craske
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, United States.,Department of Psychiatry and Biobehavioral Sciences, University of California, Los Angeles, Los Angeles, CA, United States
| |
Collapse
|
10
|
Ramos-Vidal I, Palacio J, Domínguez de la Ossa E, Wehdking I. Which factors explain information exchange and user referral among program implementers? JOURNAL OF COMMUNITY PSYCHOLOGY 2022; 50:553-575. [PMID: 34105786 DOI: 10.1002/jcop.22634] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/26/2020] [Revised: 04/08/2021] [Accepted: 05/26/2021] [Indexed: 06/12/2023]
Abstract
The Program for Psychosocial Care and Comprehensive Health for Victims serves, on a yearly basis, an average of 25,000 users in northern Colombia alone. The program is implemented by multidisciplinary teams comprised of psychologists, social workers, and community facilitators, who step in at the individual, family, and community levels. An attempt has been made to determine the effect generated by the timeframe through which professionals have been engaged with the program-filling positions of centrality and betweenness within the networks of information exchange and user referral, including the potential mediating effect from the structure of the egocentric network of implementers in the two aforementioned networks and the moderating effect of the sense of belonging to a team of professionals. Both centrality and betweenness are positional measures describing the location actors occupied within the network structure. Centrality reflects the nominations made and receipt by an actor in a network and is considered an individual indicator of prominence and power. Betweenness shows the times that an actor act as a bridge among two actors in a network and it is considered an indicator of strategic positioning in social networks. An egocentric network is the local structure of relationships that each implementer maintains with his or her direct contacts. In this study, 112 active implementers were included, mostly women (n = 97, 88.2%), who had been working on the program for 16.9 months on average (SD = 14.7). Through conditional process analysis, it has been shown that the time that the implementers have been working on the program and the sense of belonging to the task team are relevant factors that interact with each other toward explaining the level of centrality and betweenness of professionals in the information exchange and user referral networks.
Collapse
Affiliation(s)
- Ignacio Ramos-Vidal
- Department of Social Psychology, Faculty of Psychology, University of Seville, Seville, Spain
- Research Group CAVIDA, Universidad Pontificia Bolivariana, Montería, Colombia
| | - Jorge Palacio
- Faculty of Psychology, Universidad del Norte, Barranquilla, Colombia
| | | | - Ingrid Wehdking
- Faculty of Psychology, Universidad del Norte, Barranquilla, Colombia
| |
Collapse
|
11
|
Acceptability, Appropriateness, and Preliminary Effects of the PrEP Diffusion Training for Lay HIV Workers: Increased PrEP Knowledge, Decreased Stigma, and Diffusion of Innovation. AIDS Behav 2021; 25:3413-3424. [PMID: 33791880 DOI: 10.1007/s10461-021-03248-2] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/23/2021] [Indexed: 11/27/2022]
Abstract
Training lay health workers is a critical intervention strategy to support HIV pre-exposure prophylaxis (PrEP) implementation. However, few evaluations of such trainings have been published. We conducted multi-time-point surveys to evaluate the effect of a training intervention on knowledge, PrEP stigma, and implementation behavior among lay HIV workers in China. Results indicated high acceptability and appropriateness of the training. PrEP knowledge score increased by 65% from pre- to post-training, and remained high one-month post-training. We observed a significant decrease in PrEP stigma and a positive influence on determinants of implementation behaviors. All lay HIV workers surveyed one-month after the training reported having disseminated PrEP information in social networks; 43% reported integrating PrEP education into routine work. The training is an effective strategy to build lay HIV workers' capacity in PrEP implementation. Longer-term post-training follow up would be ideal to further assess actual PrEP uptake and sustained effects on PrEP implementation.
Collapse
|
12
|
Kourouche S, Curtis K, Munroe B, Watts M, Balzer S, Buckley T. Implementation strategy fidelity evaluation for a multidisciplinary Chest Injury Protocol (ChIP). Implement Sci Commun 2021; 2:86. [PMID: 34376254 PMCID: PMC8353870 DOI: 10.1186/s43058-021-00189-8] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/22/2020] [Accepted: 07/25/2021] [Indexed: 12/19/2022] Open
Abstract
Background Blunt chest wall injuries can lead to complications, especially without early intervention. A blunt Chest Injury Protocol (ChIP) was developed to help improve the consistency of evidence-based care following admission to the emergency department. Implementation strategy fidelity is the extent to which the strategies of implementation are delivered in line with the intended plan. The aim of this study was to assess fidelity to the strategies of the implementation plan developed for ChIP. Methods A retrospective evaluation of strategies used for implementation was performed, specifically the behaviour change techniques (BCTs). BCTs were used as part of an implementation plan derived based on the Behaviour Change Wheel from results from a staff survey at two hospitals. Levels of implementation or adaptation for BCTs were scored by implementers as follows: ‘Were the behaviour change interventions implemented?’ (0 = ‘not implemented’, 1 = partially implemented, and 2 = fully implemented); ‘Were adaptations made to the implementation plan?’, scored 1 (many changes from plan) to 4 (just as planned). Free text explanation to their responses was also collected with supporting evidence and documentation (such as emails, implementation checklists, audit reports, and incident reports). Results There was high overall fidelity of 97.6% for BCTs partially or fully implemented. More than three quarters (32/42, 76.2%) of the BCTs were fully implemented with an additional 9/42 (21.4%) partially implemented. BCTs that were not fully implemented were social support, feedback on behaviour, feedback on outcomes of behaviour, adding objects to the environment, and restructuring the environment. The modes of delivery with poorer implementation or increased adaptations were clinical champions and audit/feedback. Conclusions This study describes the evaluation of implementation strategy fidelity in the acute care context. The systematic use and application of the behaviour change wheel was used to develop an implementation plan and was associated with high implementation strategy fidelity. A fidelity checklist developed during the implementation process may help implementers assess fidelity. Trial registration Trial registered on ANZCTR. Registration number ACTRN12618001548224, date approved 17/09/2018 Supplementary Information The online version contains supplementary material available at 10.1186/s43058-021-00189-8.
Collapse
Affiliation(s)
- Sarah Kourouche
- Faculty of Medicine and Health, Sydney Nursing School, The University of Sydney Susan Wakil School of Nursing and Midwifery, 88 Mallet St, Camperdown, NSW, Australia.
| | - Kate Curtis
- Faculty of Medicine and Health, Sydney Nursing School, The University of Sydney Susan Wakil School of Nursing and Midwifery, 88 Mallet St, Camperdown, NSW, Australia.,Director of Critical Care Research, Illawarra Shoalhaven Local Health District, Warrawong, Australia.,Illawarra Health and Medical Research Institute, University of Wollongong, Wollongong, NSW, Australia
| | - Belinda Munroe
- Faculty of Medicine and Health, Sydney Nursing School, The University of Sydney Susan Wakil School of Nursing and Midwifery, 88 Mallet St, Camperdown, NSW, Australia.,Emergency Services, Illawarra Shoalhaven Local Health District, Wollongong Hospital, Crown St, Wollongong, NSW, Australia
| | - Michael Watts
- Intensive Care, Illawarra Shoalhaven Local Health District, Wollongong Hospital, Crown St, Wollongong, NSW, Australia
| | - Sharyn Balzer
- Emergency Services, Shoalhaven Memorial District Hospital, Shoalhaven, NSW, Australia
| | - Thomas Buckley
- Faculty of Medicine and Health, Sydney Nursing School, The University of Sydney Susan Wakil School of Nursing and Midwifery, 88 Mallet St, Camperdown, NSW, Australia
| |
Collapse
|
13
|
Rasmussen LMP, Patras J, Handegård BH, Neumer SP, Martinsen KD, Adolfsen F, Sund AM, Martinussen M. Evaluating Delivery of a CBT-Based Group Intervention for Schoolchildren With Emotional Problems: Examining the Reliability and Applicability of a Video-Based Adherence and Competence Measure. Front Psychol 2021; 12:702565. [PMID: 34262514 PMCID: PMC8273386 DOI: 10.3389/fpsyg.2021.702565] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2021] [Accepted: 05/31/2021] [Indexed: 12/16/2022] Open
Abstract
Adherence and competence are essential parts of program fidelity and having adequate measures to assess these constructs is important. The Competence and Adherence Scale for Cognitive Behavioral Therapy (CAS CBT) was developed to evaluate the delivery of cognitive therapies for children with clinical anxiety. The present study is an assessment of the slightly adapted version of the CAS CBT evaluating the delivery of a Cognitive Behavioral Therapy (CBT)-based preventive group intervention: EMOTION: Kids Coping with Anxiety and Depression. This study was part of a Norwegian cluster randomized controlled trial (cRCT) investigating the effectiveness of a transdiagnostic intervention, the EMOTION program—an indicated prevention program targeting anxious and depressive symptoms. The applicability and psychometric properties of the CAS CBT were explored. Results are based on six raters evaluating 239 video-recorded sessions of the EMOTION program being delivered by 68 trained group leaders from different municipal services. Interrater reliability (intraclass correlation coefficients, ICC [3, 1]) indicated fair to good agreement between raters. Internal consistency of the instrument's key domains was calculated using the Omega coefficient which ranged between 0.70 to 0.94. There was a strong association between the two scales Adherence and Competence, and inter-item correlations were high across the items, except for the items rating the adherence to the session goals. Competence and Adherence Scale for Cognitive Behavioral Therapy is a brief measure for use in first-line services, with some promising features for easily assessing program fidelity, but some of the results indicated that the instrument should be improved. Future attention should also be made to adapt the instrument to fit better within a group setting, especially regarding evaluation of session goals. More research on how to adequately evaluate fidelity measures are also warranted. Clinical Trial Registration:www.ClinicalTrials.gov, identifier: NCT02340637.
Collapse
Affiliation(s)
- Lene-Mari Potulski Rasmussen
- Regional Centre for Child and Youth Mental Health and Child Welfare, Faculty of Health Sciences, UiT the Arctic University of Norway, Tromsø, Norway
| | - Joshua Patras
- Regional Centre for Child and Youth Mental Health and Child Welfare, Faculty of Health Sciences, UiT the Arctic University of Norway, Tromsø, Norway
| | - Bjørn Helge Handegård
- Regional Centre for Child and Youth Mental Health and Child Welfare, Faculty of Health Sciences, UiT the Arctic University of Norway, Tromsø, Norway
| | - Simon-Peter Neumer
- Regional Centre for Child and Youth Mental Health and Child Welfare, Faculty of Health Sciences, UiT the Arctic University of Norway, Tromsø, Norway.,Centre for Child and Adolescent Mental Health, Oslo, Norway
| | - Kristin Dagmar Martinsen
- Centre for Child and Adolescent Mental Health, Oslo, Norway.,Department of Psychology, Faculty of Social Sciences, University of Oslo, Oslo, Norway
| | - Frode Adolfsen
- Regional Centre for Child and Youth Mental Health and Child Welfare, Faculty of Health Sciences, UiT the Arctic University of Norway, Tromsø, Norway
| | - Anne Mari Sund
- Regional Centre for Child and Youth Mental Health and Child Welfare, Department of Mental Health, Faculty of Medicine and Health Sciences, NTNU - Norwegian University of Science and Technology, Trondheim, Norway.,Department of Child and Adolescent Psychiatry, St. Olavs University Hospital, Trondheim, Norway
| | - Monica Martinussen
- Regional Centre for Child and Youth Mental Health and Child Welfare, Faculty of Health Sciences, UiT the Arctic University of Norway, Tromsø, Norway
| |
Collapse
|
14
|
Luecking CT, Vaughn AE, Burney R, Hennink-Kaminski H, Hales D, Ward DS. Fidelity and factors influencing implementation of Healthy Me, Healthy: process evaluation of a social marketing campaign for diet and physical activity behaviors of children in childcare. Transl Behav Med 2021; 11:733-744. [PMID: 33538306 PMCID: PMC8034246 DOI: 10.1093/tbm/ibab001] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/19/2022] Open
Abstract
The effectiveness of interventions targeting children's eating and physical activity behaviors through childcare settings is inconsistent. To enhance public health impact, it is imperative to evaluate fidelity of implementing complex interventions in real-world settings. This study evaluated fidelity and contextual factors influencing implementation of Healthy Me, Healthy We (HMHW). HMHW was an 8-month social marketing campaign delivered through childcare to support children's healthy eating and physical activity. HMHW required two levels of implementation support (research team and childcare providers) and two levels of campaign delivery (childcare providers and parents). Process evaluation was conducted among childcare centers in the intervention group (n=48) of the cluster-randomized control trial. Measures included attendance logs, self-report surveys, observation checklists, field notes, and semi-structured interviews. A 35-item fidelity index was created to assess fidelity of implementation support and campaign delivery. The fidelity with which HMHW was implemented by childcare providers and parents was low (mean 17.4 out of 35) and decreased between childcare providers and parents. Childcare providers had high acceptability of the program and individual components (80 - 93%). Only half of parents felt intervention components were acceptable. Frequently cited barriers to implementation by childcare providers included time constraints, parent engagement, staff turnover, and restrictive policies. The lack of observable effect of HMHW on children's dietary or physical activity behaviors may be due to inadequate implementation at multiple levels. Different or additional strategies are necessary to support implementation of multilevel interventions, particularly when individuals are expected to deliver intervention components and support others in doing so.
Collapse
Affiliation(s)
- Courtney T Luecking
- Department of Dietetics and Human Nutrition, College of Agriculture, Food and Environment, University of Kentucky, Lexington, KY, USA
| | - Amber E Vaughn
- Center for Health Promotion and Disease Prevention, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| | - Regan Burney
- Center for Health Promotion and Disease Prevention, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| | - Heidi Hennink-Kaminski
- Hussman School of Journalism and Media, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| | - Derek Hales
- Center for Health Promotion and Disease Prevention, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
- Department of Nutrition, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| | - Dianne S Ward
- Center for Health Promotion and Disease Prevention, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
- Department of Nutrition, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| |
Collapse
|
15
|
Jiménez JE, Gutiérrez N, de León SC. Analyzing the role of fidelity of RTI Tier 2 reading intervention in Spanish kindergarten and first grade students. ANNALS OF DYSLEXIA 2021; 71:28-49. [PMID: 33713278 DOI: 10.1007/s11881-021-00221-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/22/2020] [Accepted: 02/24/2021] [Indexed: 06/12/2023]
Abstract
This study examined the role of fidelity in the reading outcomes within a Tier 2 intervention implemented by Spanish kindergarten and first grade school teachers. For this purpose, differences in reading performance growth were analyzed among at-risk students who received a Tier 2 intervention with high fidelity, at-risk students who received the same intervention with medium fidelity, and an at-risk control group that did not receive a Tier 2 intervention. Implementation fidelity was analyzed using direct observations and self-reports. All students were assessed three times during the academic year. Hierarchical linear modeling analyses were conducted to explore differences in growth rate. The at-risk children in the intervention condition showed a higher growth compared to at-risk children in the control condition, and specifically when the intervention was delivered with a high degree of fidelity.
Collapse
Affiliation(s)
- Juan E Jiménez
- Department of Developmental and Educational Psychology, Faculty of Psychology and Speech Therapy, Universidad de La Laguna, Campus Guajara s/n, La Laguna, 38200, Tenerife, Canary Islands, Spain.
| | | | - Sara C de León
- Department of Developmental and Educational Psychology, Faculty of Psychology and Speech Therapy, Universidad de La Laguna, Campus Guajara s/n, La Laguna, 38200, Tenerife, Canary Islands, Spain
| |
Collapse
|
16
|
Bleile ACE, Koppenol-Gonzalez GV, Verreault K, Abeling K, Hofman E, Vriend W, Hasan A, Jordans MJD. Process evaluation of TeamUp: a movement-based psychosocial intervention for refugee children in the Netherlands. Int J Ment Health Syst 2021; 15:25. [PMID: 33741025 PMCID: PMC7977563 DOI: 10.1186/s13033-021-00450-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/12/2020] [Accepted: 03/06/2021] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Nearly 60,000 people applied for asylum in the Netherland in 2015, confronting the governmental structures and services with great administrative, logistical and service provision challenges. Refugee children's psychosocial needs and wellbeing are often overlooked, and post-migration support is of pivotal importance. METHODS An easy accessible movement-based psychosocial intervention, called TeamUp, was developed for children aged 6-17 living in refugee reception centres. A mixed-method process evaluation was conducted of (1) implementation process, assessing attendance (n = 2183 children, and n = 209 children); (2) implementation quality, using structured observations at two time points to evaluate facilitator's (2a) individual-level fidelity (n = 81 facilitators); (2b) team-level fidelity (n = 22 teams); (2c) facilitators' competencies (n = 81); (2d) trainee perceived self-efficacy pre-post training (n = 73); and (3) perceptions on implementation and outcomes, employing a survey (n = 99), focus group discussions and key informant interviews with children (n = 94), facilitators (n = 24) and reception centre staff (n = 10). RESULTS Attendance lists showed a mean of 8.5 children per session, and children attending 31.3% of sessions. Structured observations demonstrated 49.2% and 58.2% individual-level fidelity, 72.5% and 73.0% team-level fidelity, and 82.9% and 88.4% adequacy in competencies, each at T1 and T2 respectively. The main reported challenges included managing children's energy regulation (e.g. offering settling moments) and challenging behaviour. Training participation significantly improved perceived self-efficacy for trainees. The facilitator survey demonstrated on average, high satisfaction and self-efficacy, low experienced burden, and high perceived capacity-building support. Qualitatively, TeamUp was positively perceived by all stakeholders and was regarded as contributing to children's psychosocial outcomes. CONCLUSION (1) Attendance and group size were lower than expected. (2) The intervention's facilitator fidelity ranged from moderate to adequate-exhibiting a need for specific fidelity and capacity strengthening-while facilitator competencies were high. Trainee's perceived self-efficacy improved significantly following a 2-day training. (3) Facilitators expressed high levels of satisfaction, self-efficacy and support, and low burden. The intervention was positively perceived by all stakeholders and to have a positive impact on children's psychosocial learning and wellbeing.
Collapse
Affiliation(s)
| | | | | | | | - Elin Hofman
- War Child Holland, Amsterdam, The Netherlands
| | | | - Adnan Hasan
- War Child Holland, Amsterdam, The Netherlands
| | - Mark J D Jordans
- War Child Holland, Amsterdam, The Netherlands
- Amsterdam Institute of Social Science Research, University of Amsterdam, Amsterdam, The Netherlands
| |
Collapse
|
17
|
Sánchez A, Rogers HL, Pablo S, García E, Rodríguez I, Flores MA, Galarza O, Gaztañaga AB, Martínez PA, Alberdi E, Resines E, Llarena AI, Grandes G. Fidelity evaluation of the compared procedures for conducting the PVS-PREDIAPS implementation strategy to optimize diabetes prevention in primary care. BMC FAMILY PRACTICE 2021; 22:34. [PMID: 33573600 PMCID: PMC7879624 DOI: 10.1186/s12875-021-01378-z] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/03/2020] [Accepted: 01/26/2021] [Indexed: 05/30/2023]
Abstract
Background Assessing the fidelity of an implementation strategy is important to understand why and how the strategy influences the uptake of evidence-based interventions. The present study aims to assess the fidelity of the two procedures for engaging primary care (PC) professionals and for the deployment of an implementation strategy for optimizing type 2 diabetes prevention in routine PC within the PREDIAPS randomized cluster implementation trial. Method We conducted a mixed-method fidelity evaluation study of the PVS-PREDIAPS implementation strategy. Nine PC centers from the Basque Health Service (Osakidetza) were allocated to two different procedures to engage physicians and nurses and deploy a implementation strategy to model and adapt the clinical intervention and its implementation to their specific contexts: a Global procedure, promoting the cooperation of all healthcare professionals from the beginning; or a Sequential procedure, centered first on nurses who then pursued the pragmatic cooperation of physicians. Process indicators of the delivery and receipt of implementation strategy actions, documented modifications to the planned implementation strategy, and a structured group interview with centers’ leaders were all used to assess the following components of fidelity: adherence, dose, quality of delivery, professionals’ responsiveness and program differentiation. Results Generally, the procedures compared for professionals engagement and deployment of the implementation strategy were carried out with the planned differentiation. Nonetheless, some unexpected between-group differences were observed, the initial rate of collaboration of nurses being higher in the Sequential (93%) than in the Global (67%) groups. Exposure rate to the programed implementation actions (% of hours received out of those delivered) were similar in both groups by professional category, with nurses (86%) having a higher rate of exposure than physicians (75%). Professionals identified half of the planned discrete strategies and their rating of strategies’ perceived usefulness was overwhelmingly positive, with few differences between Sequential and Global centers. Conclusions The PVS-PREDIAPS implementation strategy has been implemented with high fidelity and minor unplanned reactive modifications. Professionals’ exposure to the implementation strategy was high in both groups. The centers’ organizational context (i.e., work overload) led to small mismatches between groups in participation and exposure of professionals to implementation actions. Trial registration Clinicaltrials.gov identifier: NCT03254979. Registered 16 August 2017.
Collapse
Affiliation(s)
- Alvaro Sánchez
- Primary Care Research Unit, Deputy Directorate of Healthcare Assistance, Biocruces Bizkaia Health Research Institute, Basque Healthcare Service - Osakidetza, Plaza Cruces s/n, E-48903, Barakaldo, Spain.
| | - Heather L Rogers
- Biocruces Bizkaia Health Research Institute and Ikerbasque Basque Foundation for Science, Bilbao, Spain, Plaza Cruces s/n, E-48903, Barakaldo, Spain
| | - Susana Pablo
- Primary Care Research Unit, Deputy Directorate of Healthcare Assistance, Biocruces Bizkaia Health Research Institute, Basque Healthcare Service - Osakidetza, Plaza Cruces s/n, E-48903, Barakaldo, Spain
| | - Esther García
- Zalla Primary Care Center, Plaza Euskadi s/n, E-48860, Zalla, Spain
| | | | | | - Olga Galarza
- Alango Primary Care Center, Alango 30, E-48992, Getxo, Spain
| | - Ana B Gaztañaga
- Egia Primary Care Center, Ametzagaina 8, E-20012, Donostia-San Sebastian, Gipuzkoa, Spain
| | - Pedro A Martínez
- San Vicente Primary Care Center, Elejalde s/n, E-48902, Barakaldo, Spain
| | - Eider Alberdi
- Iztieta Primary Care Center, Avda. Lezo s/n, Errenteria, E-20100, Gipuzkoa, Spain
| | - Elena Resines
- Zuazo Primary Care Center, Lurkizaga s/n, E-48902, Barakaldo, Spain
| | - Ana I Llarena
- Portugalete Primary Care Center, General Castaños 34, E-48920, Portugalete, Spain
| | - Gonzalo Grandes
- Primary Care Research Unit, Deputy Directorate of Healthcare Assistance, Biocruces Bizkaia Health Research Institute, Basque Healthcare Service - Osakidetza, Plaza Cruces s/n, E-48903, Barakaldo, Spain
| | | |
Collapse
|
18
|
Different Approaches to Address Bullying in KiVa Schools: Adherence to Guidelines, Strategies Implemented, and Outcomes Obtained. PREVENTION SCIENCE : THE OFFICIAL JOURNAL OF THE SOCIETY FOR PREVENTION RESEARCH 2020; 22:299-310. [PMID: 33098542 PMCID: PMC8032636 DOI: 10.1007/s11121-020-01178-4] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/14/2020] [Indexed: 11/12/2022]
Abstract
We examined the extent to which school personnel implementing the KiVa® antibullying program in Finland during 2009–2015 systematically employed the program-recommended approaches (confronting or non-confronting), used one or the other depending on the bullying case (case-specific approach), or used their own adaptation when talking to perpetrators of bullying, and whether they organized follow-up meetings after such discussions. In addition to investigating adherence to program guidelines, we tested how effective these different approaches were in stopping bullying. Finally, we tested the contribution of follow-up meetings and the number of years KiVa had been implemented in a school to the effectiveness of the interventions, using reports from both school personnel and victimized students. The data were collected annually across 6 years via online questionnaires and included responses from 1221 primary and secondary schools. The school personnel were more likely to use the confronting approach than the non-confronting approach. Over time, rather than sticking to the two program-recommended approaches, they made adaptations (e.g., combining the two; using their own approach). Two-level regression analyses indicated that the discussions were equally effective, according to both personnel and victimized students, when the confronting, non-confronting, or a case-specific approach had been used. The discussions were less effective when the personnel used their own adaptation or could not specify the method used. Perceived effectiveness was higher in primary school and when follow-up meetings were organized systematically after each intervention, but unrelated to the number of years KiVa had been implemented.
Collapse
|
19
|
Vlasblom E, van Grieken A, Beltman M, L’Hoir MP, Raat H, Boere-Boonekamp MM. Parenting support to prevent overweight during regular well-child visits in 0-3 year old children (BBOFT+ program), a cluster randomized trial on the effectiveness on child BMI and health behaviors and parenting. PLoS One 2020; 15:e0237564. [PMID: 32810194 PMCID: PMC7437453 DOI: 10.1371/journal.pone.0237564] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2019] [Accepted: 07/28/2020] [Indexed: 01/22/2023] Open
Abstract
Background Prevention of overweight during early childhood seems promising. Objective To evaluate the effectiveness of the parenting-based BBOFT+ overweight prevention program on child BMI, child health behavior and parenting behavior among 0–36 month old children. BBOFT+ is an acronym for the key healthy lifestyle behaviors that are targeted in the BBOFT+ intervention: breastfeeding (B), daily breakfast (B), daily going outdoors (O), limiting sweet beverages (in Dutch, F) and minimal TV or computer time (T), complemented with healthy sleep behavior and improvement of parenting skills (+). Methods A cluster randomized controlled trial in newborn children visiting well-baby clinics, comparing the BBOFT+ intervention (N = 901) with care as usual (CAU) (N = 1094). In both groups, parents received regular well-child visits (±11 visits in the first 3 years). In the intervention group, care was supplemented with the BBOFT+ program, which focuses on improving parenting skills from birth onwards to increase healthy behavior. Questionnaires were filled in at child's age 2–4 weeks, 6, 14 and 36 months. In multivariate analyses we corrected for child’s birthweight, age, ethnic background, mother’s educational level and BMI. Results No differences were found in weight status at 36 months between intervention and control group children. At 6 months, BBOFT+ parents reported their child drinking less sweet beverages than control parents (48% vs 54%;p = .027), and going outdoors daily with their child less often (57% vs 62%;p = .03). At 14 months, more BBOFT+ parents than control parents reported to have breastfed for six months or longer (32% vs 29%;p = .022). At 36 months, more BBOFT+ parents than control parents reported their child going outside daily (78% vs 72%;p = .011) and having less TV/computer time on week- (38% vs 46%;p = .001) and weekend days (48% vs 56%;p = .002). Also, BBOFT+ parents reported having more parental control than control parents (3.92 vs 3.89;p = .02). No significant differences were found for daily breakfast, sleep duration and parenting practices in adjusted analyses. Conclusion The BBOFT+ overweight prevention program showed small improvements in parent-reported child health behaviors, compared to care as usual; no effect was observed on child BMI. The identified modifiable elements are potentially relevant for interventions that aim to prevent overweight.
Collapse
Affiliation(s)
- Eline Vlasblom
- Department of Child Health, TNO, Leiden, The Netherlands
- * E-mail:
| | - Amy van Grieken
- Department of Public Health, Erasmus Medical Center, Rotterdam, The Netherlands
| | - Maaike Beltman
- Department of Child Health, TNO, Leiden, The Netherlands
| | - Monique P. L’Hoir
- Department of Human Nutrition and Health, Wageningen University & Research, Wageningen, The Netherlands
- Department of Public Health Services of North- and East-Gelderland (GGDNOG), Warnsveld, The Netherlands
| | - Hein Raat
- Department of Public Health, Erasmus Medical Center, Rotterdam, The Netherlands
| | - Magda M. Boere-Boonekamp
- Department of Health Technology and Services Research, Technical Medical Centre, University of Twente, Enschede, The Netherlands
| |
Collapse
|
20
|
Parekh J, Stuart E, Blum R, Caldas V, Whitfield B, Jennings JM. Addressing the Adherence-Adaptation Debate: Lessons from the Replication of an Evidence-Based Sexual Health Program in School Settings. PREVENTION SCIENCE : THE OFFICIAL JOURNAL OF THE SOCIETY FOR PREVENTION RESEARCH 2020; 20:1074-1088. [PMID: 31396805 DOI: 10.1007/s11121-019-01032-2] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
Abstract
Whether high adherence to programs is necessary to achieve program outcomes is an area of great debate. The objectives of this study were to determine the frequency, type, and rationale of adaptations made in the implementation of an evidence-based program and to determine program outcomes for intervention program participants, as compared to comparison participants, by the level of adaptations. A total of 1608 participants in 45 classrooms participated. Percent adaptations was calculated by classroom. Thematic qualitative analysis was used to categorize types and rationales for adaptations. Program outcomes by level of adaptations were determined using logistic regression analyses and mean differences. Propensity score matching methods were used to create comparability between adaptation subgroup participants and comparison participants. Adaptations ranged from 2 to 97% across classrooms, with mean adaptations of 63%. Thematic analysis revealed that the adaptations made were related to delivery of content, rather than to the content itself and in response to participant needs and setting constraints. Program outcomes did not appear to be reduced for the high-adaptation subgroup. Understanding both rationale (intent) and type of adaptation made is crucial to understanding the complexity of adaptations. These finding support the argument for allowing facilitators some flexibility and autonomy to adapt the delivery of prescribed content to participant needs and setting constraints.
Collapse
Affiliation(s)
- Jenita Parekh
- The Department of Population, Family and Reproductive Health, The Johns Hopkins Bloomberg School of Public Health, 615 N. Wolfe Street, Baltimore, MD, 21205, USA.
- The Johns Hopkins Center for Child & Community Health, Johns Hopkins Bayview Medical Center, 5200 Eastern Avenue, Suite 4200, Mason F. Lord Building, Center Tower, Baltimore, MD, 21224, USA.
| | - Elizabeth Stuart
- The Department of Mental Health, The Johns Hopkins Bloomberg School of Public Health, 615 N. Wolfe Street, Baltimore, MD, 21205, USA
| | - Robert Blum
- The Department of Population, Family and Reproductive Health, The Johns Hopkins Bloomberg School of Public Health, 615 N. Wolfe Street, Baltimore, MD, 21205, USA
| | - Valerie Caldas
- The Johns Hopkins Center for Child & Community Health, Johns Hopkins Bayview Medical Center, 5200 Eastern Avenue, Suite 4200, Mason F. Lord Building, Center Tower, Baltimore, MD, 21224, USA
| | - Brooke Whitfield
- Child Trends, Inc., 7315 Wisconsin Ave, Suite 1200W, Bethesda, MD, 20814, USA
| | - Jacky M Jennings
- The Johns Hopkins Center for Child & Community Health, Johns Hopkins Bayview Medical Center, 5200 Eastern Avenue, Suite 4200, Mason F. Lord Building, Center Tower, Baltimore, MD, 21224, USA
| |
Collapse
|
21
|
Egeland KM, Heiervang KS, Landers M, Ruud T, Drake RE, Bond GR. Psychometric Properties of a Fidelity Scale for Illness Management and Recovery. ADMINISTRATION AND POLICY IN MENTAL HEALTH AND MENTAL HEALTH SERVICES RESEARCH 2019; 47:885-893. [PMID: 31701294 PMCID: PMC7547988 DOI: 10.1007/s10488-019-00992-5] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
Abstract
This study examined the psychometric properties and feasibility of the Illness Management and Recovery (IMR) Fidelity scale. Despite widespread use of the scale, the psychometric properties have received limited attention. Trained fidelity assessors conducted assessments four times over 18 months at 11 sites implementing IMR. The IMR Fidelity scale showed excellent interrater reliability (.99), interrater item agreement (94%), internal consistency (.91–.95 at three time points), and sensitivity to change. Frequency distributions generally showed that item ratings included the entire range. The IMR Fidelity scale has excellent psychometric properties and should be used to evaluate and guide the implementation of IMR. Trial registration: ClinicalTrials.gov Identifier: NCT03271242.
Collapse
Affiliation(s)
- Karina Myhren Egeland
- Division of Mental Health Services, Akershus University Hospital, Sykehusveien 25, 1478, Lørenskog, Norway.
| | - Kristin Sverdvik Heiervang
- Division of Mental Health Services, Akershus University Hospital, Sykehusveien 25, 1478, Lørenskog, Norway
| | | | - Torleif Ruud
- Division of Mental Health Services, Akershus University Hospital, Sykehusveien 25, 1478, Lørenskog, Norway.,Institute for Clinical Medicine, University of Oslo, Oslo, Norway
| | | | | |
Collapse
|
22
|
Noble AJ, Marson AG, Blower SL. Psychological interventions for epilepsy: How good are trialists at assessing their implementation fidelity, what are the barriers, and what are journals doing to encourage it? A mixed methods study. Epilepsy Behav 2019; 97:174-181. [PMID: 31252275 DOI: 10.1016/j.yebeh.2019.05.041] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/02/2019] [Revised: 05/28/2019] [Accepted: 05/28/2019] [Indexed: 01/12/2023]
Abstract
INTRODUCTION Psychological interventions hold promise for the epilepsy population and continue to be trialed to determine their efficacy. Such interventions present opportunities for variance in delivery. Therefore, to accurately interpret a trial's estimate of effect, information on implementation fidelity (IF) is required. We present a novel 3-part study. Part 1 systematically rated trials for the extent to which they reported assessing whether the intervention was delivered as intended (adherence) and with what sort of skill (competence). Part 2 identified barriers to reporting and assessing on fidelity perceived by trialists. Part 3 determined what journals publishing epilepsy trials are doing to support IFs reporting. METHODS Articles for 50 randomized controlled trials (RCTs)/quasi-RCTs of psychological interventions identified by Cochrane searches were rated using the Psychotherapy Outcome Study Methodology Rating Form's fidelity items. The 45 corresponding authors for the 50 trials were invited to complete the 'Barriers to Treatment Integrity Implementation Survey'. 'Instructions to Authors' for the 17 journals publishing the trials were reviewed for endorsement of popular reporting guidelines which refer to fidelity (Consolidated Standards of Reporting Trials (CONSORT) statement or Journal Article Reporting Standards [JARS]) and asked how they enforced compliance. RESULTS Part 1: 15 (30%) trials reported assessing for adherence, but only 2 (4.3%) gave the result. Four (8.5%) reported assessing for competence, 1 (2.1%) gave the result. Part 2: 22 trialists - mostly chief investigators - responded. They identified 'lack of theory and specific guidelines on treatment integrity procedures', 'time, cost, and labor demands', and 'lack of editorial requirement' as "strong barriers". Part 3: Most (15, 88.2%) journals endorsed CONSORT or JARS, but only 5 enforced compliance. CONCLUSIONS Most trials of psychological interventions for epilepsy are not reported in a transparent way when it comes to IF. The barriers' trialists identify for this do not appear insurmountable. Addressing them could ultimately help the field to better understand how best to support the population with epilepsy.
Collapse
Affiliation(s)
- Adam J Noble
- Health Services Research, University of Liverpool, Liverpool, UK.
| | - Anthony G Marson
- Department of Molecular and Clinical Pharmacology, University of Liverpool, Liverpool, UK
| | - Sarah L Blower
- Department of Health Sciences, University of York, York, UK
| |
Collapse
|
23
|
Naar S, MacDonell K, Chapman JE, Todd L, Gurung S, Cain D, Dilones RE, Parsons JT. Testing a Motivational Interviewing Implementation Intervention in Adolescent HIV Clinics: Protocol for a Type 3, Hybrid Implementation-Effectiveness Trial. JMIR Res Protoc 2019; 8:e11200. [PMID: 31237839 PMCID: PMC6682301 DOI: 10.2196/11200] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2018] [Revised: 09/26/2018] [Accepted: 10/29/2018] [Indexed: 01/16/2023] Open
Abstract
Background Motivational interviewing (MI) has been shown to effectively improve self-management for youth living with HIV (YLH) and has demonstrated success across the youth HIV care cascade—currently, the only behavioral intervention to do so. Substantial barriers prevent the effective implementation of MI in real-world settings. Thus, there is a critical need to understand how to implement evidence-based practices (EBPs), such as MI, and promote behavior change in youth HIV treatment settings as risk-taking behaviors peak during adolescence and young adulthood. Objective This study aims to describe the Adolescent Medicine Trials Network for HIV/AIDS Interventions (ATN) protocol of a tailored MI (TMI) implementation-effectiveness trial (ATN 146 TMI) to scale up an EBP in multidisciplinary adolescent HIV settings while balancing flexibility and fidelity. This protocol is part of the Scale It Up program described in this issue. Methods This study is a type 3, hybrid implementation-effectiveness trial that tests the effect of TMI on fidelity (MI competency and adherence to program requirements) while integrating findings from two other ATN protocols described in this issue—ATN 153 Exploration, Preparations, Implementation, Sustainment and ATN 154 Cascade Monitoring. ATN 153 guides the mixed methods investigation of barriers and facilitators of implementation, while ATN 154 provides effectiveness outcomes. The TMI study population consists of providers at 10 adolescent HIV care sites around the United States. These 10 clinics are randomly assigned to 5 blocks to receive the TMI implementation intervention (workshop and trigger-based coaching guided by local implementation teams) utilizing the dynamic wait-listed controlled design. After 12 months of implementation, a second randomization compares a combination of internal facilitator coaching with the encouragement of communities of practice (CoPs) to CoPs alone. Participants receive MI competency assessments on a quarterly basis during preimplementation, during the 12 months of implementation and during the sustainment period for a total of 36 months. We hypothesize that MI competency ratings will be higher among providers during the TMI implementation phase compared with the standard care phase, and successful implementation will be associated with improved cascade-related outcomes, namely undetectable viral load and a greater number of clinic visits among YLH. Results Participant recruitment began in August 2017 and is ongoing. As of mid-May 2018, TMI has 150 active participants. Conclusions This protocol describes the underlying theoretical framework, study design, measures, and lessons learned for TMI, a type 3, hybrid implementation-effectiveness trial, which has the potential to scale up MI and improve patient outcomes in adolescent HIV settings. Trial Registration ClinicalTrials.gov NCT03681912; https://clinicaltrials.gov/ct2/show/NCT03681912 (Archived by WebCite at http://www.webcitation.org/754oT7Khx) International Registered Report Identifier (IRRID) DERR1-10.2196/11200
Collapse
Affiliation(s)
- Sylvie Naar
- College of Medicine, Florida State University, Tallahassee, FL, United States
| | - Karen MacDonell
- Department of Family Medicine and Public Health Sciences, Wayne State University, Detroit, MI, United States
| | | | - Lisa Todd
- Department of Family Medicine and Public Health Sciences, Wayne State University, Detroit, MI, United States
| | - Sitaji Gurung
- Center for HIV Educational Studies and Training, Hunter College, City University of New York, New York, NY, United States
| | - Demetria Cain
- Center for HIV Educational Studies and Training, Hunter College, City University of New York, New York, NY, United States
| | - Rafael E Dilones
- Center for HIV Educational Studies and Training, Hunter College, City University of New York, New York, NY, United States
| | - Jeffrey T Parsons
- Center for HIV Educational Studies and Training, Hunter College, City University of New York, New York, NY, United States.,Health Psychology and Clinical Science Doctoral Program, Graduate Center, City University of New York, New York, NY, United States
| |
Collapse
|
24
|
Ridsdale L, McKinlay A, Wojewodka G, Robinson EJ, Mosweu I, Feehan SJ, Noble AJ, Morgan M, Taylor SJ, McCrone P, Landau S, Richardson M, Baker G, Goldstein LH. Self-Management education for adults with poorly controlled epILEpsy [SMILE (UK)]: a randomised controlled trial. Health Technol Assess 2019; 22:1-142. [PMID: 29717699 DOI: 10.3310/hta22210] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/14/2023] Open
Abstract
BACKGROUND Epilepsy is a common neurological condition resulting in recurrent seizures. Research evidence in long-term conditions suggests that patients benefit from self-management education and that this may improve quality of life (QoL). Epilepsy self-management education has yet to be tested in a UK setting. OBJECTIVES To determine the effectiveness and cost-effectiveness of Self-Management education for people with poorly controlled epILEpsy [SMILE (UK)]. DESIGN A parallel pragmatic randomised controlled trial. SETTING Participants were recruited from eight hospitals in London and south-east England. PARTICIPANTS Adults aged ≥ 16 years with epilepsy and two or more epileptic seizures in the past year, who were currently being prescribed antiepileptic drugs. INTERVENTION A 2-day group self-management course alongside treatment as usual (TAU). The control group received TAU. MAIN OUTCOME MEASURES The primary outcome is QoL in people with epilepsy at 12-month follow-up using the Quality Of Life In Epilepsy 31-P (QOLIE-31-P) scale. Other outcomes were seizure control, impact of epilepsy, medication adverse effects, psychological distress, perceived stigma, self-mastery and medication adherence. Cost-effectiveness analyses and a process evaluation were undertaken. RANDOMISATION A 1 : 1 ratio between trial arms using fixed block sizes of two. BLINDING Participants were not blinded to their group allocation because of the nature of the study. Researchers involved in data collection and analysis remained blinded throughout. RESULTS The trial completed successfully. A total of 404 participants were enrolled in the study [SMILE (UK), n = 205; TAU, n = 199] with 331 completing the final follow-up at 12 months [SMILE (UK), n = 163; TAU, n = 168]. In the intervention group, 61.5% completed all sessions of the course. No adverse events were found to be related to the intervention. At baseline, participants had a mean age of 41.7 years [standard deviation (SD) 14.1 years], and had epilepsy for a median of 18 years. The mean QOLIE-31-P score for the whole group at baseline was 66.0 out of 100.0 (SD 14.2). Clinically relevant levels of anxiety symptoms were reported in 53.6% of the group and depression symptoms in 28.0%. The results following an intention-to-treat analysis showed no change in any measures at the 12-month follow-up [QOLIE-31-P: SMILE (UK) mean: 67.4, SD 13.5; TAU mean: 69.5, SD 14.8]. The cost-effectiveness study showed that SMILE (UK) was possibly cost-effective but was also associated with lower QoL. The process evaluation with 20 participants revealed that a group course increased confidence by sharing with others and improved self-management behaviours. CONCLUSIONS For people with epilepsy and persistent seizures, a 2-day self-management education course is cost-saving, but does not improve QoL after 12-months or reduce anxiety or depression symptoms. A psychological intervention may help with anxiety and depression. Interviewed participants reported attending a group course increased their confidence and helped them improve their self-management. FUTURE WORK More research is needed on self-management courses, with psychological components and integration with routine monitoring. TRIAL REGISTRATION Current Controlled Trials ISRCTN57937389. FUNDING This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 22, No. 21. See the NIHR Journals Library website for further project information.
Collapse
Affiliation(s)
- Leone Ridsdale
- Department of Basic and Clinical Neurosciences, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| | - Alison McKinlay
- Department of Basic and Clinical Neurosciences, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| | - Gabriella Wojewodka
- Department of Basic and Clinical Neurosciences, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| | - Emily J Robinson
- Department of Biostatistics and Health Informatics, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| | - Iris Mosweu
- King's Health Economics, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| | - Sarah J Feehan
- Department of Basic and Clinical Neurosciences, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| | - Adam J Noble
- Department of Psychological Sciences, Institute of Psychology, Health and Society, University of Liverpool, Liverpool, UK
| | - Myfanwy Morgan
- Institute of Pharmaceutical Science, King's College London, London, UK
| | - Stephanie Jc Taylor
- Barts and The London School of Medicine and Dentistry, Queen Mary University of London, Centre for Primary Care and Public Health, London, UK
| | - Paul McCrone
- King's Health Economics, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| | - Sabine Landau
- Department of Biostatistics and Health Informatics, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| | - Mark Richardson
- Department of Basic and Clinical Neurosciences, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| | - Gus Baker
- Department of Molecular and Clinical Pharmacology, University of Liverpool, Liverpool, UK
| | - Laura H Goldstein
- Department of Psychology, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| |
Collapse
|
25
|
Wong D, Grace N, Baker K, McMahon G. Measuring clinical competencies in facilitating group-based rehabilitation interventions: development of a new competency checklist. Clin Rehabil 2019; 33:1079-1087. [PMID: 30806075 DOI: 10.1177/0269215519831048] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
OBJECTIVE Group-based intervention formats are common in rehabilitation, but no tool for objectively measuring clinical competencies in group facilitation currently exists. We aimed to develop a psychometrically sound group facilitation competency checklist for use in clinical, training, and research settings. METHOD The Delphi method of expert consensus was used to establish checklist items that clearly describe competencies considered important for effective group facilitation. Inter-rater reliability was determined with two experienced psychologists who used the checklist to rate the competencies of psychology trainees facilitating a memory skills group. RESULTS After two Delphi rounds, consensus was reached on 17 items, defined as at least 80% agreement among the panel of 15 experts. The four checklist item categories were (a) Facilitating focused group discussion, (b) Communication skills, (c) Interpersonal style, and (d) Session structure. One item was removed after piloting. Inter-rater reliability was excellent (88% agreement) using a simple coding method (competent/incompetent). When using a detailed coding method that discriminated between 'done adequately' and 'done well', inter-rater reliability was weaker (κ = 0.481, 55% agreement); however, it improved to almost perfect after the raters calibrated their standards. CONCLUSION The new group facilitation competency checklist is fit for purpose for measuring clinical competencies in delivering group-based rehabilitation interventions and can be used in the training of effective group facilitators.
Collapse
Affiliation(s)
- Dana Wong
- 1 School of Psychology and Public Health, La Trobe University, Melbourne, VIC, Australia.,2 School of Psychological Sciences, Monash Institute of Cognitive and Clinical Neurosciences, Monash University, Clayton, VIC, Australia
| | - Nicci Grace
- 2 School of Psychological Sciences, Monash Institute of Cognitive and Clinical Neurosciences, Monash University, Clayton, VIC, Australia
| | - Katharine Baker
- 1 School of Psychology and Public Health, La Trobe University, Melbourne, VIC, Australia
| | - Genevieve McMahon
- 2 School of Psychological Sciences, Monash Institute of Cognitive and Clinical Neurosciences, Monash University, Clayton, VIC, Australia
| |
Collapse
|
26
|
Assessing Treatment Fidelity within an Epilepsy Randomized Controlled Trial: Seizure First Aid Training for People with Epilepsy Who Visit Emergency Departments. Behav Neurol 2019; 2019:5048794. [PMID: 30863463 PMCID: PMC6378079 DOI: 10.1155/2019/5048794] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2018] [Accepted: 12/31/2018] [Indexed: 12/12/2022] Open
Abstract
Purpose To measure fidelity with which a group seizure first aid training intervention was delivered within a pilot randomized controlled trial underway in the UK for adults with epilepsy who visit emergency departments (ED) and informal carers. Estimates of its effects, including on ED use, will be produced by the trial. Whilst hardly ever reported for trials of epilepsy interventions—only one publication on this topic exists—this study provides the information on treatment fidelity necessary to allow the trial's estimates to be accurately interpreted. This rare worked example of how fidelity can be assessed could also provide guidance sought by neurology trialists on how to assess fidelity. Methods 53 patients who had visited ED on ≥2 occasions in prior year were recruited for the trial; 26 were randomized to the intervention. 7 intervention courses were delivered for them by one facilitator. Using audio recordings, treatment “adherence” and “competence” were assessed. Adherence was assessed by a checklist of the items comprising the intervention. Using computer software, competence was measured by calculating facilitator speech during the intervention (didacticism). Interrater reliability was evaluated by two independent raters assessing each course using the measures and their ratings being compared. Results The fidelity measures were found to be reliable. For the adherence instrument, raters agreed 96% of the time, PABAK-OS kappa 0.91. For didacticism, raters' scores had an intraclass coefficient of 0.96. In terms of treatment fidelity, not only were courses found to have been delivered with excellent adherence (88% of its items were fully delivered) but also as intended they were highly interactive, with the facilitator speaking for, on average, 55% of course time. Conclusions The fidelity measures used were reliable and showed that the intervention was delivered as attended. Therefore, any estimates of intervention effect will not be influenced by poor implementation fidelity.
Collapse
|
27
|
Haines D, Wright J, Comerasamy H. Occupational Therapy Empowering Support Workers to Change How They Support People with Profound Intellectual and Multiple Disabilities to Engage in Activity. JOURNAL OF POLICY AND PRACTICE IN INTELLECTUAL DISABILITIES 2018. [DOI: 10.1111/jppi.12257] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/01/2022]
Affiliation(s)
- David Haines
- University of Brighton, Eastbourne; United Kingdom of Great Britain and Northern Ireland
| | - Jonathan Wright
- University of Brighton, Eastbourne; United Kingdom of Great Britain and Northern Ireland
| | - Huguette Comerasamy
- University of Brighton, Eastbourne; United Kingdom of Great Britain and Northern Ireland
| |
Collapse
|
28
|
Parvez SM, Azad R, Rahman M, Unicomb L, Ram PK, Naser AM, Stewart CP, Jannat K, Rahman MJ, Leontsini E, Winch PJ, Luby SP. Achieving optimal technology and behavioral uptake of single and combined interventions of water, sanitation hygiene and nutrition, in an efficacy trial (WASH benefits) in rural Bangladesh. Trials 2018; 19:358. [PMID: 29976251 PMCID: PMC6034207 DOI: 10.1186/s13063-018-2710-8] [Citation(s) in RCA: 37] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/04/2017] [Accepted: 05/25/2018] [Indexed: 12/01/2022] Open
Abstract
BACKGROUND Uptake matters for evaluating the health impact of water, sanitation and hygiene (WASH) interventions. Many large-scale WASH interventions have been plagued by low uptake. For the WASH Benefits Bangladesh efficacy trial, high uptake was a prerequisite. We assessed the degree of technology and behavioral uptake among participants in the trial, as part of a three-paper series on WASH Benefits Intervention Delivery and Performance. METHODS This study is a cluster randomized trial comprised of geographically matched clusters among four districts in rural Bangladesh. We randomly allocated 720 clusters of 5551 pregnant women to individual or combined water, sanitation, handwashing, and nutrition interventions, or a control group. Behavioral objectives included; drinking chlorine-treated, safely stored water; use of a hygienic latrine and safe feces disposal at the compound level; handwashing with soap at key times; and age-appropriate nutrition behaviors (pregnancy to 24 months) including a lipid-based nutrition supplement (LNS). Enabling technologies and behavior change were promoted by trained local community health workers through periodic household visits. To monitor technology and behavioral uptake, we conducted surveys and spot checks in 30-35 households per intervention arm per month, over a 20-month period, and structured observations in 324 intervention and 108 control households, approximately 15 months after interventions commenced. RESULTS In the sanitation arms, observed adult use of a hygienic latrine was high (94-97% of events) while child sanitation practices were moderate (37-54%). In the handwashing arms, handwashing with soap was more common after toilet use (67-74%) than nonintervention arms (18-40%), and after cleaning a child's anus (61-72%), but was still low before food handling. In the water intervention arms, more than 65% of mothers and index children were observed drinking chlorine-treated water from a safe container. Reported LNS feeding was > 80% in nutrition arms. There was little difference in uptake between single and combined intervention arms. CONCLUSIONS Rigorous implementation of interventions deployed at large scale in the context of an efficacy trial achieved high levels of technology and behavioral uptake in individual and combined WASH and nutrition intervention households. Further work should assess how to achieve similar uptake levels under programmatic conditions. TRIAL REGISTRATION WASH Benefits Bangladesh: ClinicalTrials.gov, identifier: NCT01590095 . Registered on April 30, 2012.
Collapse
Affiliation(s)
- Sarker Masud Parvez
- Environmental Intervention Unit, Enteric and Respiratory Infections Program, Infectious Diseases Division, International Centre for Diarrhoeal Disease Research, Bangladesh (icddr.b), Dhaka, Bangladesh
| | - Rashidul Azad
- Environmental Intervention Unit, Enteric and Respiratory Infections Program, Infectious Diseases Division, International Centre for Diarrhoeal Disease Research, Bangladesh (icddr.b), Dhaka, Bangladesh
| | - Mahbubur Rahman
- Environmental Intervention Unit, Enteric and Respiratory Infections Program, Infectious Diseases Division, International Centre for Diarrhoeal Disease Research, Bangladesh (icddr.b), Dhaka, Bangladesh
| | - Leanne Unicomb
- Environmental Intervention Unit, Enteric and Respiratory Infections Program, Infectious Diseases Division, International Centre for Diarrhoeal Disease Research, Bangladesh (icddr.b), Dhaka, Bangladesh
| | - Pavani K. Ram
- School of Public Health and Health Professions, University of Buffalo, Buffalo, NY USA
| | - Abu Mohd Naser
- Rollins School of Public Health, Emory University, Atlanta, GA USA
| | | | - Kaniz Jannat
- Environmental Intervention Unit, Enteric and Respiratory Infections Program, Infectious Diseases Division, International Centre for Diarrhoeal Disease Research, Bangladesh (icddr.b), Dhaka, Bangladesh
| | - Musarrat Jabeen Rahman
- Environmental Intervention Unit, Enteric and Respiratory Infections Program, Infectious Diseases Division, International Centre for Diarrhoeal Disease Research, Bangladesh (icddr.b), Dhaka, Bangladesh
| | - Elli Leontsini
- Department of International Health, Social and Behavioral Interventions Program, Johns Hopkins Bloomberg School of Public Health, Baltimore, USA
| | - Peter J. Winch
- Department of International Health, Social and Behavioral Interventions Program, Johns Hopkins Bloomberg School of Public Health, Baltimore, USA
| | - Stephen P. Luby
- Division of Infectious Diseases and Geographic Medicine, Stanford University, Stanford, CA USA
| |
Collapse
|
29
|
Rahman M, Ashraf S, Unicomb L, Mainuddin AKM, Parvez SM, Begum F, Das KK, Naser AM, Hussain F, Clasen T, Luby SP, Leontsini E, Winch PJ. WASH Benefits Bangladesh trial: system for monitoring coverage and quality in an efficacy trial. Trials 2018; 19:360. [PMID: 29976234 PMCID: PMC6034300 DOI: 10.1186/s13063-018-2708-2] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2017] [Accepted: 05/25/2018] [Indexed: 11/17/2022] Open
Abstract
BACKGROUND Researchers typically report more on the impact of public health interventions and less on the degree to which interventions were followed implementation fidelity. We developed and measured fidelity indicators for the WASH Benefits Bangladesh study, a large-scale efficacy trial, in order to identify gaps between intended and actual implementation. METHODS Community health workers (CHWs) delivered individual and combined water, sanitation, handwashing (WSH) and child nutrition interventions to 4169 enrolled households in geographically matched clusters. Households received free enabling technologies (insulated water storage container; sani-scoop, potty, double-pit, pour-flush latrine; handwashing station, soapy-water storage bottle), and supplies (chlorine tablets, lipid-based nutrient supplements, laundry detergent sachets) integrated with parallel behavior-change promotion. Behavioral objectives were drinking treated, safely stored water, safe feces disposal, handwashing with soap at key times, and age-appropriate nutrition behaviors. We administered monthly surveys and spot-checks to households from randomly selected clusters for 6 months early in the trial. If any fidelity measures fell below set benchmarks, a rapid response mechanism was triggered. RESULTS In the first 3 months, functional water seals were detected in 33% (14/42) of latrines in the sanitation only arm; 35% (14/40) for the combined WSH arm; and 60% (34/57) for the combined WSH and Nutrition arm, all falling below the pre-set benchmark of 80%. Other fidelity indicators met the 65 to 80% uptake benchmarks. Rapid qualitative investigations determined that households concurrently used their own latrines with broken water seals in parallel with those provided by the trial. In consultation with the households, we closed pre-existing latrines without water seals, increased the CHWs' visit frequency to encourage correct maintenance of latrines with water seals, and discouraged water-seal removal or breakage. At the sixth assessment, 86% (51/59) of households were in sanitation only; 92% (72/78) in the combined WSH; and 93% (71/76) in the combined WSH and Nutrition arms had latrines with functional water seals. CONCLUSIONS An intensive implementation fidelity monitoring and rapid response system proved beneficial for this efficacy trial. To implement a routine program at scale requires further research into an adaptation of fidelity monitoring that supports program effectiveness. TRIAL REGISTRATION WASH Benefits Bangladesh: ClinicalTrials.gov, ID: NCT01590095 . Registered on 30 April 2012.
Collapse
Affiliation(s)
- Mahbubur Rahman
- Environmental Intervention Unit, Enteric and Respiratory Disease Program, Infectious Disease Division, International Centre for Diarrheal Disease Research, Bangladesh (icddr,b), Dhaka, Bangladesh
| | - Sania Ashraf
- Johns Hopkins Bloomberg School of Public Health, Baltimore, MD USA
| | - Leanne Unicomb
- Environmental Intervention Unit, Enteric and Respiratory Disease Program, Infectious Disease Division, International Centre for Diarrheal Disease Research, Bangladesh (icddr,b), Dhaka, Bangladesh
| | - A. K. M. Mainuddin
- Environmental Intervention Unit, Enteric and Respiratory Disease Program, Infectious Disease Division, International Centre for Diarrheal Disease Research, Bangladesh (icddr,b), Dhaka, Bangladesh
| | - Sarker Masud Parvez
- Environmental Intervention Unit, Enteric and Respiratory Disease Program, Infectious Disease Division, International Centre for Diarrheal Disease Research, Bangladesh (icddr,b), Dhaka, Bangladesh
| | - Farzana Begum
- Environmental Intervention Unit, Enteric and Respiratory Disease Program, Infectious Disease Division, International Centre for Diarrheal Disease Research, Bangladesh (icddr,b), Dhaka, Bangladesh
| | - Kishor Kumar Das
- Environmental Intervention Unit, Enteric and Respiratory Disease Program, Infectious Disease Division, International Centre for Diarrheal Disease Research, Bangladesh (icddr,b), Dhaka, Bangladesh
| | - Abu Mohd. Naser
- Rollins School of Public Health, Emory University, Atlanta, GA USA
| | - Faruqe Hussain
- Environmental Intervention Unit, Enteric and Respiratory Disease Program, Infectious Disease Division, International Centre for Diarrheal Disease Research, Bangladesh (icddr,b), Dhaka, Bangladesh
| | - Thomas Clasen
- Rollins School of Public Health, Emory University, Atlanta, GA USA
| | - Stephen P. Luby
- Division of Infectious Diseases and Geographic Medicine, Stanford University, Stanford, CA USA
| | - Elli Leontsini
- Johns Hopkins Bloomberg School of Public Health, Baltimore, MD USA
| | - Peter J. Winch
- Johns Hopkins Bloomberg School of Public Health, Baltimore, MD USA
| |
Collapse
|
30
|
Kok MS, Jones M, Solomon-Moore E, Smith JR. Implementation fidelity of a voluntary sector-led diabetes education programme. HEALTH EDUCATION 2018. [DOI: 10.1108/he-04-2017-0019] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Purpose
The quality of voluntary sector-led community health programmes is an important concern for service users, providers and commissioners. Research on the fidelity of programme implementation offers a basis for assessing and further enhancing practice. The purpose of this paper is to report on the fidelity assessment of Living Well Taking Control (LWTC) – a voluntary sector-led, community-based education programme in England focussing on the prevention and management of type 2 diabetes.
Design/methodology/approach
This fidelity of implementation (FoI) study was conducted with the Devon-based LWTC programme. A fidelity checklist was developed to analyse audio records of group-based lifestyle education sessions – implementation was rated in terms of adherence to protocol and competence in delivery; the influence of wider contextual factors was also assessed. Kappa statistics (κ) were used to test for inter-rater agreement. Course satisfaction data were used as a supplementary indicator of facilitator competence.
Findings
Analysis of 28 sessions, from five diabetes prevention and two diabetes management groups (total participants, n=49), yielded an overall implementation fidelity score of 77.3 per cent for adherence (moderate inter-rater agreement, κ=0.60) and 95.1 per cent for competence (good inter-rater agreement, κ=0.71). The diabetes prevention groups consistently achieved higher adherence scores than the diabetes management groups. Facilitator competence was supported by high participant satisfaction ratings.
Originality/value
An appropriate level of implementation fidelity was delivered for the LWTC group-based education programme, which provides some confidence that outcomes from the programme reflected intervention effectiveness. This study demonstrates the viability of assessing the FoI in a voluntary sector-led public health initiative and the potential of this method for assuring quality and informing service development.
Collapse
|
31
|
Phillips E, Stoltzfus RJ, Michaud L, Pierre GLF, Vermeylen F, Pelletier D. Do mobile clinics provide high-quality antenatal care? A comparison of care delivery, knowledge outcomes and perception of quality of care between fixed and mobile clinics in central Haiti. BMC Pregnancy Childbirth 2017; 17:361. [PMID: 29037190 PMCID: PMC5644158 DOI: 10.1186/s12884-017-1546-7] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2016] [Accepted: 10/08/2017] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Antenatal care (ANC) is an important health service for women in developing countries, with numerous proven benefits. Global coverage of ANC has steadily increased over the past 30 years, in part due to increased community-based outreach. However, commensurate improvements in health outcomes such as reductions in the prevalence of maternal anemia and infants born small-for-gestational age have not been achieved, even with increased coverage, indicating that quality of care may be inadequate. Mobile clinics are one community-based strategy used to further improve coverage of ANC, but their quality of care delivery has rarely been evaluated. METHODS To determine the quality of care of ANC in central Haiti, we compared adherence to national guidelines between fixed and mobile clinics by performing direct observations of antenatal care consultations and exit interviews with recipients of care using a multi-stage random sampling procedure. Outcome variables were eight components of care, and women's knowledge and perception of care quality. RESULTS There were significant differences in the predicted proportion or probability of recommended services for four of eight care components, including intake, laboratory examinations, infection control, and supplies, iron folic acid supplements and Tetanus Toxoid vaccine provided to women. These care components were more likely performed in fixed clinics, except for distribution of supplies, iron-folic acid supplements, and Tetanus Toxoid vaccine, more likely provided in mobile clinics. There were no differences between clinic type for the proportion of total physical exam procedures performed, health and communication messages delivered, provider communication or documentation. Women's knowledge about educational topics was poor, but women perceived extremely high quality of care in both clinic models. CONCLUSIONS Although adherence to guidelines differed by clinic type for half of the care components, both clinics had a low percentage of overall services delivered. Efforts to improve provider performance and quality are therefore needed in both models. Mobile clinics must deliver high-quality ANC to improve health and nutrition outcomes.
Collapse
Affiliation(s)
| | - Rebecca J. Stoltzfus
- Division of Nutritional Sciences, Cornell University, 120 Savage Hall, Ithaca, NY 14853 United States
| | | | | | - Francoise Vermeylen
- Division of Nutritional Sciences, Cornell University, B19 Savage Hall, Ithaca, NY 14853 United States
| | - David Pelletier
- Division of Nutritional Sciences, Cornell University, 212 Savage Hall, Ithaca, NY 14853 United States
| |
Collapse
|
32
|
Wojewodka G, Hurley S, Taylor SJC, Noble AJ, Ridsdale L, Goldstein LH. Implementation fidelity of a self-management course for epilepsy: method and assessment. BMC Med Res Methodol 2017; 17:100. [PMID: 28697723 PMCID: PMC5504680 DOI: 10.1186/s12874-017-0373-x] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2017] [Accepted: 06/26/2017] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Complex interventions such as self-management courses are difficult to evaluate due to the many interacting components. The way complex interventions are delivered can influence the effect they have for patients, and can impact the interpretation of outcomes of clinical trials. Implementation fidelity evaluates whether complex interventions are delivered according to protocol. Such assessments have been used for one-to-one psychological interventions; however, the science is still developing for group interventions. METHODS We developed and tested an instrument to measure implementation fidelity of a two-day self-management course for people with epilepsy, SMILE(UK). Using audio recordings, we looked at adherence and competence of course facilitators. Adherence was assessed by checklists. Competence was measured by scoring group interaction, an overall impression score and facilitator "didacticism". To measure "didacticism", we developed a novel way to calculate facilitator speech using computer software. Using this new instrument, implementation fidelity of SMILE(UK) was assessed on three modules of the course, for 28% of all courses delivered. RESULTS Using the instrument for adherence, scores from two independent raters showed substantial agreement with weighted Kappa of 0.67 and high percent agreement of 81.2%. For didacticism, the results from both raters were highly correlated with an intraclass coefficient of 0.97 (p < 0.0001). We found that the courses were delivered with a good level of adherence (> 50% of scored items received the maximum of 2 points) and high competence. Groups were interactive (mean score: 1.9-2.0 out of 2) and the overall impression was on average assessed as "good". Didacticism varied from 42% to 93% of total module time and was not associated with the other competence scores. CONCLUSION The instrument devised to measure implementation fidelity was reproducible and easy to use. The courses for the SMILE(UK) study were delivered with a good level of adherence to protocol while not compromising facilitator competence. TRIAL REGISTRATION ISRCTN57937389 .
Collapse
Affiliation(s)
- G Wojewodka
- King's College London, Institute of Psychiatry, Psychology and Neuroscience, De Crespigny Park, London, SE5 8AF, UK
| | - S Hurley
- King's College London, Institute of Psychiatry, Psychology and Neuroscience, De Crespigny Park, London, SE5 8AF, UK
| | - S J C Taylor
- Centre for Primary Care and Public Health, Bart's and the London School of Medicine and Dentistry, Queen Mary University of London, London, UK
| | - A J Noble
- Institute of Psychology, Health and Society, University of Liverpool, Liverpool, UK
| | - L Ridsdale
- King's College London, Institute of Psychiatry, Psychology and Neuroscience, De Crespigny Park, London, SE5 8AF, UK
| | - L H Goldstein
- King's College London, Institute of Psychiatry, Psychology and Neuroscience, De Crespigny Park, London, SE5 8AF, UK. .,King's College London, Institute of Psychiatry, Psychology and Neuroscience, PO 77, The Henry Wellcome Building, De Crespigny Park, London, SE5 8AF, UK.
| |
Collapse
|
33
|
Poirier M, Marcotte D, Joly J, Fortin L. Évaluation de la qualité de l’implantation du programme Pare-Chocs à l’école secondaire. SANTE MENTALE AU QUEBEC 2017. [DOI: 10.7202/1040259ar] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Malgré l’importante augmentation de la prévalence de la dépression durant l’adolescence, seulement une faible proportion des adolescents qui présentent des symptômes dépressifs élevés bénéficie des services de santé mentale en milieu scolaire. De plus, les programmes offerts à ces élèves souffrent souvent d’une implantation lacunaire, limitant ainsi leur efficacité potentielle. L’identification des facteurs qui influencent la fidélité de l’implantation est donc essentielle pour améliorer la qualité des services offerts. Guidés par le modèle d’évaluation basé sur la théorie des programmes (theory-driven evaluation), nous évaluons la qualité de l’implantation du programme Pare-Chocs et les facteurs qui l’ont influencée à l’aide des composantes du modèle d’action de Chen (2005). Une approche multi-méthodes a été utilisée pour obtenir des données quantitatives sur la fidélité d’implantation dans les six groupes et des données qualitatives sur les composantes du modèle d’action. Les résultats suggèrent que l’adhésion, le dosage et la participation étaient élevés. Les contraintes de temps et le peu de formation antérieure en lien avec la théorie du programme ont limité la fidélité de l’implantation tandis que la formation, la supervision et le manuel y ont contribué. Ces résultats confirment que les programmes de prévention disséminés dans les milieux scolaires peuvent être implantés avec un niveau de fidélité élevé, même si certains défis doivent être pris en considération lors de la planification de l’intervention pour contribuer à des effets plus importants du programme. De plus, la fidélité devrait être évaluée de manière systématique dans ce contexte.
Collapse
|
34
|
Cross WF, Pisani AR, Schmeelk-Cone K, Xia Y, Tu X, McMahon M, Munfakh JL, Gould MS. Measuring trainer fidelity in the transfer of suicide prevention training. CRISIS 2016; 35:202-12. [PMID: 24901061 DOI: 10.1027/0227-5910/a000253] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/23/2022]
Abstract
BACKGROUND Finding effective and efficient models to train large numbers of suicide prevention interventionists, including 'hotline' crisis counselors, is a high priority. Train-the-trainer (TTT) models are widely used but understudied. AIMS To assess the extent to which trainers following TTT delivered the Applied Suicide Intervention Skills Training (ASIST) program with fidelity, and to examine fidelity across two trainings and seven training segments. METHOD We recorded and reliably rated trainer fidelity, defined as adherence to program content and competence of program delivery, for 34 newly trained ASIST trainers delivering the program to crisis center staff on two separate occasions. A total of 324 observations were coded. Trainer demographics were also collected. RESULTS On average, trainers delivered two-thirds of the program. Previous training was associated with lower levels of trainer adherence to the program. In all, 18% of trainers' observations were rated as solidly competent. Trainers did not improve fidelity from their first to second training. Significantly higher fidelity was found for lectures and lower fidelity was found for interactive training activities including asking about suicide and creating a safe plan. CONCLUSIONS We found wide variability in trainer fidelity to the ASIST program following TTT and few trainers had high levels of both adherence and competence. More research is needed to examine the cost-effectiveness of TTT models.
Collapse
Affiliation(s)
- Wendi F Cross
- Department of Psychiatry, University of Rochester Medical Center (URMC), Rochester, NY, USA Department of Pediatrics, University of Rochester Medical Center (URMC), Rochester, NY, USA
| | - Anthony R Pisani
- Department of Psychiatry, University of Rochester Medical Center (URMC), Rochester, NY, USA Department of Pediatrics, University of Rochester Medical Center (URMC), Rochester, NY, USA
| | - Karen Schmeelk-Cone
- Department of Psychiatry, University of Rochester Medical Center (URMC), Rochester, NY, USA
| | - Yinglin Xia
- Department of Biostatstics and Computation Biology, University of Rochester Medical Center (URMC), Rochester, NY, USA
| | - Xin Tu
- Department of Biostatstics and Computation Biology, University of Rochester Medical Center (URMC), Rochester, NY, USA
| | - Marcie McMahon
- Center for Employee Wellness at the School of Nursing at the University of Rochester Medical Center (URMC), Rochester, NY, USA
| | - Jimmie Lou Munfakh
- Division of Child and Adolescent Psychiatry at the New York State Psychiatric Institute, New York, NY, USA
| | - Madelyn S Gould
- Division of Child and Adolescent Psychiatry at the New York State Psychiatric Institute, New York, NY, USA Departments of Psychiatry and Epidemiology (School of Public Health), Columbia University Medical Center, New York, NY, USA
| |
Collapse
|
35
|
Clark A, Breitenstein S, Martsolf DS, Winstanley EL. Assessing Fidelity of a Community-Based Opioid Overdose Prevention Program: Modification of the Fidelity Checklist. J Nurs Scholarsh 2016; 48:371-7. [PMID: 27376347 DOI: 10.1111/jnu.12221] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 04/18/2016] [Indexed: 11/30/2022]
Abstract
BACKGROUND The evaluation and maintenance of treatment fidelity is an important methodological consideration in intervention research. Treatment fidelity is the degree to which interventions are delivered as outlined by the program developers. A lack of fidelity to the intervention model has the potential to be a large source of error, including type I and type II error. Monitoring and assuring fidelity is critical to assuring the validity of the interventions. The Fidelity Checklist is a reliable and valid tool designed to measure fidelity in a group-based parenting program. The Fidelity Checklist assesses group leaders' maintenance of the intervention protocol (adherence) and their group facilitation and process skills (competence). OBJECTIVES The purpose of this article is to describe the systematic process of modifying the Fidelity Checklist for use in a community-based opioid overdose prevention group educational intervention. METHODS A biphasic approach was used to modify the Fidelity Checklist. Phase 1 included engaging key stakeholders during a full-day meeting to determine adherence subscale components. Phase 2 included: (a) the modification of the competence subscale of the Fidelity Checklist, (b) adaptation of the coding manual defining the components of the Fidelity Checklist, and (c) creating a fidelity checklist to guide interventionists. RESULTS The biphasic systematic modification approach resulted in a revised Fidelity Checklist that was successfully used to measure treatment fidelity in a community-based opioid overdose prevention program. CLINICAL RELEVANCE The findings of this study provide information for nurse researchers and nurse educators for modifying a fidelity checklist that can be used to enhance community-based educational interventions.
Collapse
Affiliation(s)
- Angela Clark
- Assistant Professor, University of Cincinnati, College of Nursing, Cincinnati, OH, USA
| | - Susie Breitenstein
- Associate Professor, Rush University College of Nursing, Chicago, IL, USA
| | - Donna S Martsolf
- Associate Dean for Research and Translation, University of Cincinnati, College of Nursing, Cincinnati, OH, USA
| | - Erin L Winstanley
- Assistant Professor, University of Cincinnati, College of Pharmacy, Cincinnati, OH, USA
| |
Collapse
|
36
|
Smith JD, Dishion TJ, Brown K, Ramos K, Knoble NB, Shaw DS, Wilson MN. An Experimental Study of Procedures to Enhance Ratings of Fidelity to an Evidence-Based Family Intervention. PREVENTION SCIENCE : THE OFFICIAL JOURNAL OF THE SOCIETY FOR PREVENTION RESEARCH 2016; 17:62-70. [PMID: 26271300 PMCID: PMC4697869 DOI: 10.1007/s11121-015-0589-0] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
Abstract
The valid and reliable assessment of fidelity is critical at all stages of intervention research and is particularly germane to interpreting the results of efficacy and implementation trials. Ratings of protocol adherence typically are reliable, but ratings of therapist competence are plagued by low reliability. Because family context and case conceptualization guide the therapist's delivery of interventions, the reliability of fidelity ratings might be improved if the coder is privy to client context in the form of an ecological assessment. We conducted a randomized experiment to test this hypothesis. A subsample of 46 families with 5-year-old children from a multisite randomized trial who participated in the feedback session of the Family Check-Up (FCU) intervention were selected. We randomly assigned FCU feedback sessions to be rated for fidelity to the protocol using the COACH rating system either after the coder reviewed the results of a recent ecological assessment or had not. Inter-rater reliability estimates of fidelity ratings were meaningfully higher for the assessment information condition compared to the no-information condition. Importantly, the reliability of the COACH mean score was found to be statistically significantly higher in the information condition. These findings suggest that the reliability of observational ratings of fidelity, particularly when the competence or quality of delivery is considered, could be improved by providing assessment data to the coders. Our findings might be most applicable to assessment-driven interventions, where assessment data explicitly guides therapist's selection of intervention strategies tailored to the family's context and needs, but they could also apply to other intervention programs and observational coding of context-dependent therapy processes, such as the working alliance.
Collapse
Affiliation(s)
- Justin D Smith
- Center for Prevention Implementation Methodology, Northwestern University Feinberg School of Medicine, Chicago, IL, USA.
- Center for Prevention Implementation Methodology, Department of Psychiatry and Behavioral Sciences, Northwestern University Feinberg School of Medicine, 750 N Lake Shore Drive, Chicago, IL, 60657, USA.
| | - Thomas J Dishion
- REACH Institute, Arizona State University, Tempe, AZ, USA
- Oregon Research Institute, Eugene, OR, USA
| | - Kimbree Brown
- Prevention Science Institute, University of Oregon, Eugene, OR, USA
- Oregon Social Learning Center, Eugene, OR, USA
| | - Karina Ramos
- Prevention Science Institute, University of Oregon, Eugene, OR, USA
- University of California at Berkeley, Berkeley, CA, USA
| | - Naomi B Knoble
- Prevention Science Institute, University of Oregon, Eugene, OR, USA
- Nemours/duPont Children's Hospital, Wilmington, DE, USA
| | | | | |
Collapse
|
37
|
Ruggiero KJ, Bunnell BE, Andrews Iii AR, Davidson TM, Hanson RF, Danielson CK, Saunders BE, Soltis K, Yarian C, Chu B, Adams ZW. Development and Pilot Evaluation of a Tablet-Based Application to Improve Quality of Care in Child Mental Health Treatment. JMIR Res Protoc 2015; 4:e143. [PMID: 26717906 PMCID: PMC4712346 DOI: 10.2196/resprot.4416] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/11/2015] [Revised: 06/29/2015] [Accepted: 06/29/2015] [Indexed: 12/03/2022] Open
Abstract
Background Children need access to high quality mental health care. Effective treatments now exist for a wide range of mental health conditions. However, these interventions are delivered with variable effectiveness in traditional mental health service settings. Innovative solutions are needed to improve treatment delivery quality and effectiveness. Objective The aim of this study was to develop a scalable, sustainable technology-based approach to improve the quality of care in child mental health treatment. Methods A tablet-based resource was developed with input from mental health training experts, mental health providers, and patients. A series of qualitative data collection phases (ie, expert interviews, patient and provider focus groups, usability testing) guided the initial concept and design of the resource, and then its refinement. The result was an iPad-based “e-workbook” designed to improve child engagement and provider fidelity in implementation of a best-practice treatment. We are currently conducting a small scale randomized controlled trial to evaluate the feasibility of e-workbook facilitated child mental health treatment with 10 providers and 20 families recruited from 4 local community-based mental health clinics. Results Usability and focus group testing yielded a number of strong, favorable reactions from providers and families. Recommendations for refining the e-workbook also were provided, and these guided several improvements to the resource prior to initiating the feasibility trial, which is currently underway. Conclusions This study aimed to develop and preliminarily evaluate a tablet-based application to improve provider fidelity and child engagement in child mental health treatment. If successful, this approach may serve as a key step toward making best-practice treatment more accessible to children and families. As various technologies continue to increase in popularity worldwide and within the health care field more specifically, it is essential to rigorously test the usability, feasibility, acceptability, and effectiveness of novel health technology solutions. It is also essential to ensure that patients and providers drive decision making that supports the development of these resources to ensure that they can be seamlessly integrated into practice. Trial Registration Clinicaltrials.gov NCT01915160; https://clinicaltrials.gov/ct2/show/NCT01915160 (Archived by WebCite at http://www.webcitation.org/6cPIiQDpu)
Collapse
Affiliation(s)
- Kenneth J Ruggiero
- Technology Applications Center for Healthful Lifestyles, College of Nursing, Medical University of South Carolina, Charleston, SC, United States.
| | | | | | | | | | | | | | | | | | | | | |
Collapse
|
38
|
Mason WA, Fleming CB, Thompson RW, Haggerty KP, Snyder JJ. A framework for testing and promoting expanded dissemination of promising preventive interventions that are being implemented in community settings. PREVENTION SCIENCE : THE OFFICIAL JOURNAL OF THE SOCIETY FOR PREVENTION RESEARCH 2015; 15:674-83. [PMID: 23807241 DOI: 10.1007/s11121-013-0409-3] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/22/2022]
Abstract
Many evidence-based preventive interventions have been developed in recent years, but few are widely used. With the current focus on efficacy trials, widespread dissemination and implementation of evidence-based interventions are often afterthoughts. One potential strategy for reversing this trend is to find a promising program with a strong delivery vehicle in place and improve and test the program's efficacy through rigorous evaluation. If the program is supported by evidence, the dissemination vehicle is already in place and potentially can be expanded. This strategy has been used infrequently and has met with limited success to date, in part, because the field lacks a framework for guiding such research. To address this gap, we outline a framework for moving promising preventive interventions that are currently being implemented in community settings through a process of rigorous testing and, if needed, program modification in order to promote expanded dissemination. The framework is guided by RE-AIM (Reach, Efficacy/Effectiveness, Adoption, Implementation, and Maintenance) (Glasgow et al., Am J Publ Health 89:1322-1327, 1999), which focuses attention on external as well as internal validity in program tests, and is illustrated with examples. Challenges, such as responding to negative and null results, and opportunities inherent in the framework are discussed.
Collapse
Affiliation(s)
- W Alex Mason
- National Research Institute for Child and Family Studies, Boys Town, 14100 Crawford Street, Boys Town, NE, 68010, USA,
| | | | | | | | | |
Collapse
|
39
|
Slaughter SE, Hill JN, Snelgrove-Clarke E. What is the extent and quality of documentation and reporting of fidelity to implementation strategies: a scoping review. Implement Sci 2015; 10:129. [PMID: 26345357 PMCID: PMC4562107 DOI: 10.1186/s13012-015-0320-3] [Citation(s) in RCA: 67] [Impact Index Per Article: 7.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2015] [Accepted: 08/28/2015] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Implementation fidelity is critical to the internal and external validity of implementation research. Much of what is written about implementation fidelity addresses fidelity of evidence-informed interventions rather than fidelity of implementation strategies. The documentation and reporting of fidelity to implementation strategies requires attention. Therefore, in this scoping review, we identify the extent and quality of documentation and reporting of fidelity of implementation strategies that were used to implement evidence-informed interventions. METHODS A six-stage methodological framework for scoping studies guided our work. Studies were identified from the outputs of the Effective Practice and Organization of Care (EPOC) review group within the Cochrane Database of Systematic Reviews. EPOC's primary focus, implementation strategies influencing provider behavior change, optimized our ability to identify articles for inclusion. We organized the retrieved articles from the systematic reviews by journal and selected the three journals with the largest number of retrieved articles. Using a data extraction tool, we organized retrieved article data from these three journals. In addition, we summarized implementation strategies using the EPOC categories. Data extraction pertaining to the quality of reporting the fidelity of implementation strategies was facilitated with an "Implementation Strategy Fidelity Checklist" based on definitions adapted from Dusenbury et al. We conducted inter-rater reliability checks for all of the independently scored articles. Using linear regression, we assessed the fidelity scores in relation to the publication year. RESULTS Seventy-two implementation articles were included in the final analysis. Researchers reported neither fidelity definitions nor conceptual frameworks for fidelity in any articles. The most frequently employed implementation strategies included distribution of education materials (n = 35), audit and feedback (n = 32), and educational meetings (n = 25). Fidelity of implementation strategies was documented in 51 (71 %) articles. Inter-rater reliability coefficients of the independent reviews for each component of fidelity were as follows: adherence = 0.85, dose = 0.89, and participant responsiveness = 0.96. The mean fidelity score was 2.6 (SD = 2.25). We noted a statistically significant decline in fidelity scores over time. CONCLUSIONS In addition to identifying the under-reporting of fidelity of implementation strategies in the health literature, we developed and tested a simple checklist to assess the reporting of fidelity of implementation strategies. More research is indicated to assess the definitions and scoring schema of this checklist. Careful reporting of details about fidelity of implementation strategies will make an important contribution to implementation science.
Collapse
Affiliation(s)
- Susan E Slaughter
- Faculty of Nursing, University of Alberta, Edmonton, Alberta, Canada.
| | - Jennifer N Hill
- Department of Veteran's Affairs, Spinal Cord Injury Quality Enhancement Research Initiative, Hines, IL, USA.
| | - Erna Snelgrove-Clarke
- School of Nursing, Department Obstetrics/Gynecology, Dalhousie University, Halifax, Nova Scotia, Canada.
| |
Collapse
|
40
|
Cargo M, Stankov I, Thomas J, Saini M, Rogers P, Mayo-Wilson E, Hannes K. Development, inter-rater reliability and feasibility of a checklist to assess implementation (Ch-IMP) in systematic reviews: the case of provider-based prevention and treatment programs targeting children and youth. BMC Med Res Methodol 2015; 15:73. [PMID: 26346461 PMCID: PMC4562191 DOI: 10.1186/s12874-015-0037-7] [Citation(s) in RCA: 16] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/25/2014] [Accepted: 05/20/2015] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Several papers report deficiencies in the reporting of information about the implementation of interventions in clinical trials. Information about implementation is also required in systematic reviews of complex interventions to facilitate the translation and uptake of evidence of provider-based prevention and treatment programs. To capture whether and how implementation is assessed within systematic effectiveness reviews, we developed a checklist for implementation (Ch-IMP) and piloted it in a cohort of reviews on provider-based prevention and treatment interventions for children and young people. This paper reports on the inter-rater reliability, feasibility and reasons for discrepant ratings. METHODS Checklist domains were informed by a framework for program theory; items within domains were generated from a literature review. The checklist was pilot-tested on a cohort of 27 effectiveness reviews targeting children and youth. Two raters independently extracted information on 47 items. Inter-rater reliability was evaluated using percentage agreement and unweighted kappa coefficients. Reasons for discrepant ratings were content analysed. RESULTS Kappa coefficients ranged from 0.37 to 1.00 and were not influenced by one-sided bias. Most kappa values were classified as excellent (n = 20) or good (n = 17) with a few items categorised as fair (n = 7) or poor (n = 1). Prevalence-adjusted kappa coefficients indicate good or excellent agreement for all but one item. Four areas contributed to scoring discrepancies: 1) clarity or sufficiency of information provided in the review; 2) information missed in the review; 3) issues encountered with the tool; and 4) issues encountered at the review level. Use of the tool demands time investment and it requires adjustment to improve its feasibility for wider use. CONCLUSIONS The case of provider-based prevention and treatment interventions showed relevancy in developing and piloting the Ch-IMP as a useful tool for assessing the extent to which systematic reviews assess the quality of implementation. The checklist could be used by authors and editors to improve the quality of systematic reviews, and shows promise as a pedagogical tool to facilitate the extraction and reporting of implementation characteristics.
Collapse
Affiliation(s)
- Margaret Cargo
- Spatial Epidemiology and Evaluation Research Group, School of Population Health, University of South Australia, Adelaide, Australia.
| | - Ivana Stankov
- Spatial Epidemiology and Evaluation Research Group, School of Population Health, University of South Australia, Adelaide, Australia.
| | - James Thomas
- Evidence for Policy and Practice Information and Co-ordinating (EPPI) Centre, Social Science Research Unit, UCL Institute of Education, University College London, London, UK.
| | - Michael Saini
- Factor-Inwentash Faculty of Social Work, University of Toronto, Toronto, Canada.
| | - Patricia Rogers
- Centre for Applied Social Research, RMIT University (Royal Melbourne Institute of Technology), Melbourne, Australia.
| | - Evan Mayo-Wilson
- Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA.
| | - Karin Hannes
- Methodology of Educational Sciences Research Group, Faculty of Psychology and Educational Sciences, KU Leuven, Leuven, Belgium.
| |
Collapse
|
41
|
Cross W, West J, Wyman PA, Schmeelk-Cone K, Xia Y, Tu X, Teisl M, Brown CH, Forgatch M. Observational measures of implementer fidelity for a school-based preventive intervention: development, reliability, and validity. PREVENTION SCIENCE : THE OFFICIAL JOURNAL OF THE SOCIETY FOR PREVENTION RESEARCH 2015; 16:122-32. [PMID: 24736951 PMCID: PMC4199927 DOI: 10.1007/s11121-014-0488-9] [Citation(s) in RCA: 16] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/11/2022]
Abstract
Current measures of implementer fidelity often fail to adequately measure core constructs of adherence and competence, and their relationship to outcomes can be mixed. To address these limitations, we used observational methods to assess these constructs and their relationships to proximal outcomes in a randomized trial of a school-based preventive intervention (Rochester Resilience Project) designed to strengthen emotion self-regulation skills in first-third graders with elevated aggressive-disruptive behaviors. Within the intervention group (n = 203), a subsample (n = 76) of students was selected to reflect the overall sample. Implementers were 10 paraprofessionals. Videotaped observations of three lessons from year 1 of the intervention (14 lessons) were coded for each implementer-child dyad on adherence (content) and competence (quality). Using multilevel modeling, we examined how much of the variance in the fidelity measures was attributed to implementer and to the child within implementer. Both measures had large and significant variance accounted for by implementer (competence, 68 %; adherence, 41 %); child within implementer did not account for significant variance indicating that ratings reflected stable qualities of the implementer rather than the child. Raw adherence and competence scores shared 46 % of variance (r = .68). Controlling for baseline differences and age, the amount (adherence) and quality (competence) of program delivered predicted children's enhanced response to the intervention on both child and parent reports after 6 months, but not on teacher report of externalizing behavior. Our findings support the use of multiple observations for measuring fidelity and that adherence and competence are important components of fidelity which could be assessed by many programs using these methods.
Collapse
Affiliation(s)
- Wendi Cross
- Department of Psychiatry, School of Medicine and Dentistry, University of Rochester, Rochester, NY, USA,
| | | | | | | | | | | | | | | | | |
Collapse
|
42
|
Oats RG, Cross WF, Alex Mason W, Casey-Goldstein M, Thompson RW, Hanson K, Haggerty KP. Implementation assessment of widely used but understudied prevention programs: an illustration from the Common Sense Parenting trial. EVALUATION AND PROGRAM PLANNING 2014; 44:89-97. [PMID: 24632185 PMCID: PMC4073790 DOI: 10.1016/j.evalprogplan.2014.02.002] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/23/2013] [Revised: 01/30/2014] [Accepted: 02/05/2014] [Indexed: 06/03/2023]
Abstract
Common Sense Parenting is a parent-training program that is widely disseminated, has promising preliminary support, and is being tested in a randomized controlled trial that targets lower-income, urban 8th-grade students and their families (recruited in two annual cohorts) to improve the transition to high school. The workshop-based program is being tested in both standard 6-session (CSP) and modified 8-session (CSP Plus) formats; CSP Plus adds adolescent-skills training activities. To offer a comprehensive picture of implementation outcomes in the CSP trial, we describe the tools used to assess program adherence, quality of delivery, program dosage, and participant satisfaction, and report the implementation data collected during the trial. Results indicated that workshop leaders had high adherence to the program content and manual-stated goal times of the CSP/CSP Plus curriculum and delivered the intervention with high quality. The majority of intervention families attended some or all of the sessions. Participant satisfaction ratings for the workshops were high. There were no significant cohort differences for adherence, quality and dosage; however, there were significant cohort improvements for participant satisfaction. Positive fidelity results may be due to the availability of detailed workshop leader guides, in addition to ongoing training and supervision, which included performance-based feedback.
Collapse
Affiliation(s)
- Robert G Oats
- National Research Institute for Child and Family Studies, Father Flanagan's Boys' Home, 14100 Crawford Street, Headquarters Building - Mod 12, Boys Town, NE 68010, USA.
| | - Wendi F Cross
- Department of Psychiatry, Box Psych, University of Rochester Medical Center, 300 Crittenden Boulevard, Rochester, NY 14642, USA
| | - W Alex Mason
- National Research Institute for Child and Family Studies, Father Flanagan's Boys' Home, 14100 Crawford Street, Headquarters Building - Mod 12, Boys Town, NE 68010, USA
| | - Mary Casey-Goldstein
- Social Development Research Group, School of Social Work, University of Washington, 9725 3rd Avenue NE, Suite 401, Seattle, WA 98115, USA
| | - Ronald W Thompson
- National Research Institute for Child and Family Studies, Father Flanagan's Boys' Home, 14100 Crawford Street, Headquarters Building - Mod 12, Boys Town, NE 68010, USA
| | - Koren Hanson
- Social Development Research Group, School of Social Work, University of Washington, 9725 3rd Avenue NE, Suite 401, Seattle, WA 98115, USA
| | - Kevin P Haggerty
- Social Development Research Group, School of Social Work, University of Washington, 9725 3rd Avenue NE, Suite 401, Seattle, WA 98115, USA
| |
Collapse
|
43
|
Jäger C, Freund T, Steinhäuser J, Aakhus E, Flottorp S, Godycki-Cwirko M, van Lieshout J, Krause J, Szecsenyi J, Wensing M. Tailored Implementation for Chronic Diseases (TICD): a protocol for process evaluation in cluster randomized controlled trials in five European countries. Trials 2014; 15:87. [PMID: 24655439 PMCID: PMC3994491 DOI: 10.1186/1745-6215-15-87] [Citation(s) in RCA: 16] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2013] [Accepted: 12/20/2013] [Indexed: 11/11/2022] Open
Abstract
Background In the ‘Tailored Implementation for Chronic Diseases (TICD)’ project, five tailored implementation programs to improve healthcare delivery in different chronic conditions have been developed. These programs will be evaluated in distinct cluster-randomized controlled trials. This protocol describes the process evaluation across these trials, which aims to identify determinants of change in chronic illness care, to examine the validity of the tailoring methods that were applied, and to analyze the association of implementation activities and the effectiveness of the program. Methods A multilevel approach was used to develop five tailored implementation interventions. In order to guide the process evaluation in five distinct trials, the study protocols for the cluster randomized trials and the related process evaluations were developed simultaneously and iteratively. Results The process evaluation comprises three main components: a structured survey with health professionals in the trials, semi-structured interviews with a purposeful sample of this study population, and standardized documentation of organizational practice characteristics. Norway will only conduct the qualitative part of the analysis because the survey and documentation of practice characteristics are considered to be not feasible. The evaluation is guided by ‘logic models’ of the implementation programs: frameworks that specify the linkages between the strategies used, the determinants addressed by tailoring, and the anticipated outcomes. Standardization of measures across trials is sought to facilitate analysis of aggregated data from the trials. Conclusions This process evaluation will need to find a balance between standardization of methods across trials and the tailoring of measures to the specificities of each trial.
Collapse
Affiliation(s)
- Cornelia Jäger
- Department of General Practice and Health Services Research, University Hospital of Heidelberg, Voßstraße 2, Geb, 37, 69115 Heidelberg, Germany.
| | | | | | | | | | | | | | | | | | | |
Collapse
|
44
|
Mars T, Ellard D, Carnes D, Homer K, Underwood M, Taylor SJC. Fidelity in complex behaviour change interventions: a standardised approach to evaluate intervention integrity. BMJ Open 2013; 3:e003555. [PMID: 24240140 PMCID: PMC3831105 DOI: 10.1136/bmjopen-2013-003555] [Citation(s) in RCA: 64] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/29/2022] Open
Abstract
OBJECTIVES The aim of this study was to (1) demonstrate the development and testing of tools and procedures designed to monitor and assess the integrity of a complex intervention for chronic pain (COping with persistent Pain, Effectiveness Research into Self-management (COPERS) course); and (2) make recommendations based on our experiences. DESIGN Fidelity assessment of a two-arm randomised controlled trial intervention, assessing the adherence and competence of the facilitators delivering the intervention. SETTING The intervention was delivered in the community in two centres in the UK: one inner city and one a mix of rural and urban locations. PARTICIPANTS 403 people with chronic musculoskeletal pain were enrolled in the intervention arm and 300 attended the self-management course. Thirty lay and healthcare professionals were trained and 24 delivered the courses (2 per course). We ran 31 courses for up to 16 people per course and all were audio recorded. INTERVENTIONS The course was run over three and a half days; facilitators delivered a semistructured manualised course. OUTCOMES We designed three measures to evaluate fidelity assessing adherence to the manual, competence and overall impression. RESULTS We evaluated a random sample of four components from each course (n=122). The evaluation forms were reliable and had good face validity. There were high levels of adherence in the delivery: overall adherence was two (maximum 2, IQR 1.67-2.00), facilitator competence exhibited more variability, and overall competence was 1.5 (maximum 2, IQR 1.25-2.00). Overall impression was three (maximum 4, IQR 2.00-3.00). CONCLUSIONS Monitoring and assessing adherence and competence at the point of intervention delivery can be realised most efficiently by embedding the principles of fidelity measurement within the design stage of complex interventions and the training and assessment of those delivering the intervention. More work is necessary to ensure that more robust systems of fidelity evaluation accompany the growth of complex interventions. TRIAL REGISTRATION ISRCTN NO ISRCTN24426731.
Collapse
Affiliation(s)
- Tom Mars
- Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry, UK
| | - David Ellard
- Centre for Primary Care and Public Health, Blizard Institute, Barts and The London School of Medicine and Dentistry, London, UK
| | - Dawn Carnes
- Clinical Trials Unit (T0.10), Warwick Medical School, University of Warwick, Coventry, UK
| | - Kate Homer
- Centre for Primary Care and Public Health, Blizard Institute, Barts and The London School of Medicine and Dentistry, London, UK
| | - Martin Underwood
- Centre for Primary Care and Public Health, Blizard Institute, Barts and The London School of Medicine and Dentistry, London, UK
| | - Stephanie J C Taylor
- Centre for Primary Care and Public Health, Blizard Institute, Barts and The London School of Medicine and Dentistry, London, UK
| |
Collapse
|
45
|
Melchior MA, Seff LR, Bastida E, Albatineh AN, Page TF, Palmer RC. Intermediate outcomes of a chronic disease self-management program for Spanish-speaking older adults in South Florida, 2008-2010. Prev Chronic Dis 2013; 10:E146. [PMID: 23987252 PMCID: PMC3760083 DOI: 10.5888/pcd10.130016] [Citation(s) in RCA: 18] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/02/2022] Open
Abstract
Introduction The prevalence and negative health effects of chronic diseases are disproportionately high among Hispanics, the largest minority group in the United States. Self-management of chronic conditions by older adults is a public health priority. The objective of this study was to examine 6-week differences in self-efficacy, time spent performing physical activity, and perceived social and role activities limitations for participants in a chronic disease self-management program for Spanish-speaking older adults, Tomando Control de su Salud (TCDS). Methods Through the Healthy Aging Regional Collaborative, 8 area agencies delivered 82 workshops in 62 locations throughout South Florida. Spanish-speaking participants who attended workshops from October 1, 2008, through December 31, 2010, were aged 55 years or older, had at least 1 chronic condition, and completed baseline and post-test surveys were included in analysis (N = 682). Workshops consisted of six, 2.5-hour sessions offered once per week for 6 weeks. A self-report survey was administered at baseline and again at the end of program instruction. To assess differences in outcomes, a repeated measures general linear model was used, controlling for agency and baseline general health. Results All outcomes showed improvement at 6 weeks. Outcomes that improved significantly were self-efficacy to manage disease, perceived social and role activities limitations, time spent walking, and time spent performing other aerobic activities. Conclusion Implementation of TCDS significantly improved 4 of 8 health promotion skills and behaviors of Spanish-speaking older adults in South Florida. A community-based implementation of TCDS has the potential to improve health outcomes for a diverse, Spanish-speaking, older adult population.
Collapse
|
46
|
Stasiewicz PR, Bradizza CM, Schlauch RC, Coffey SF, Gulliver SB, Gudleski GD, Bole CW. Affect regulation training (ART) for alcohol use disorders: development of a novel intervention for negative affect drinkers. J Subst Abuse Treat 2013; 45:433-43. [PMID: 23876455 DOI: 10.1016/j.jsat.2013.05.012] [Citation(s) in RCA: 66] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2012] [Revised: 05/29/2013] [Accepted: 05/30/2013] [Indexed: 11/17/2022]
Abstract
Although negative affect is a common precipitant of alcohol relapse, there are few interventions for alcohol dependence that specifically target negative affect. In this stage 1a/1b treatment development study, several affect regulation strategies (e.g., mindfulness, prolonged exposure, distress tolerance) were combined to create a new treatment supplement called affect regulation training (ART), which could be added to enhance cognitive-behavioral therapy (CBT) for alcohol dependence. A draft therapy manual was given to therapists and treatment experts before being administered to several patients who also provided input. After two rounds of manual development (stage 1a), a pilot randomized clinical trial (N=77) of alcohol-dependent outpatients who reported drinking often in negative affect situations was conducted (stage 1b). Participants received 12-weekly, 90-minute sessions of either CBT for alcohol dependence plus ART (CBT+ART) or CBT plus a healthy lifestyles control condition (CBT+HLS). Baseline, end-of-treatment, and 3- and 6-month posttreatment interviews were conducted. For both treatment conditions, participant ratings of treatment satisfaction were high, with CBT+ART rated significantly higher. Drinking outcome results indicated greater reductions in alcohol use for CBT+ART when compared to CBT+HLS, with moderate effect sizes for percent days abstinent, drinks per day, drinks per drinking day, and percent heavy drinking days. Overall, findings support further research on affect regulation interventions for negative affect drinkers.
Collapse
Affiliation(s)
- Paul R Stasiewicz
- Research Institute on Addictions, University at Buffalo, 1021 Main St., Buffalo, NY 14203, USA.
| | | | | | | | | | | | | |
Collapse
|
47
|
El-Mallakh P, Chlebowy DO, Wall MP, Myers JA, Cloud RN. Promoting nurse interventionist fidelity to motivational interviewing in a diabetes self-care intervention. Res Nurs Health 2012; 35:289-300. [PMID: 22492432 DOI: 10.1002/nur.21472] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 02/23/2012] [Indexed: 11/07/2022]
Abstract
We describe interventionist training procedures for a pilot intervention study that tested the effects of a nurse-delivered Motivational Interviewing (MI) intervention on diabetes self-care among adults diagnosed with type 2 diabetes mellitus. It reports on findings from MI fidelity assessments. Training consisted of didactic workshops plus practicum. Fidelity of MI was assessed using the MI Treatment Integrity (MITI) Scale. Fidelity assessments were conducted on 18 (25%) audiotaped MI sessions, which were randomly selected from a total of 72 sessions with 26 participants. Scores of the MITI were in the proficient and competent range. Results suggest that training strategies were sufficient to promote satisfactory interventionist fidelity to MI.
Collapse
Affiliation(s)
- Peggy El-Mallakh
- 315 College of Nursing, Office 539, Chandler Medical Center, Lexington, KY 40536-0232, USA
| | | | | | | | | |
Collapse
|