1
|
Hyam L, Yeadon-Ray O, Richards K, Semple A, Allen K, Owens J, Jackson A, Semple L, Glennon D, Di Clemente G, Griffiths J, Mills R, Schmidt U. "FREED instils a bit of hope in the eating disorder community… that things can change.": an investigation of clinician views on implementation facilitators and challenges from the rapid scaling of the First Episode Rapid Early Intervention for Eating Disorders programme. Front Psychiatry 2024; 15:1327328. [PMID: 38596636 PMCID: PMC11002146 DOI: 10.3389/fpsyt.2024.1327328] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/24/2023] [Accepted: 03/08/2024] [Indexed: 04/11/2024] Open
Abstract
Introduction First Episode Rapid Early Intervention for Eating Disorders (FREED) is the leading eating disorder (ED) early intervention model for young people. Research has shown that it reduces the duration of untreated illness, improves clinical outcomes, and has cost savings. However, less is known about the experience of implementing FREED. This study aimed to investigate the views and experiences of adopting, implementing, and sustaining FREED from the perspective of clinical staff. Methods Seven focus groups were conducted involving 26 clinicians. Thematic analysis was used, with the Non-Adoption, Abandonment and Challenges to Scale-up, Spread and Sustainability (The NASSS framework) framework being applied to organise subthemes and determine facilitators and barriers. The NASSS framework was also used to rate the complexity of themes as either simple (straightforward, predictable, few components), complicated (multiple interrelating components), or complex (dynamic, unpredictable, not easily divisible into constituent components). Results There were 16 subthemes identified under seven broader themes representing each domain of the NASSS framework. Key barriers and areas of complexity included factors related to EDs as an illness (e.g., high acuity and prevalence), and organisational complexity (e.g., staffing shortages, lack of managerial/team support). Key facilitators included positive clinician/adopter attitudes, a supportive national network, and the ability for FREED to be flexible/adaptable over time. Conclusion The FREED model appears to be desirable to clinical staff. Wider team and managerial support was perceived to be particularly important to its successful implementation, as were the national network and supervision. Key areas of complexity include staffing issues and high ED acuity/prevalence. These barriers to implementation need to be managed and investment continued to expand and improve early intervention for EDs further.
Collapse
Affiliation(s)
- Lucy Hyam
- Centre for Research in Eating and Weight Disorders, Department of Psychological Medicine, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, London, United Kingdom
| | - Olivia Yeadon-Ray
- Department of Psychosis Studies, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, London, United Kingdom
| | - Katie Richards
- Centre for Implementation Science, Health Service and Population Research Department, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, London, United Kingdom
| | - Amy Semple
- Health Innovation Network, Academic Health Science Network, London, United Kingdom
| | - Karina Allen
- Centre for Research in Eating and Weight Disorders, Department of Psychological Medicine, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, London, United Kingdom
- Eating Disorders Outpatient Service, South London and Maudsley National Health Service (NHS) Foundation Trust, London, United Kingdom
| | - Jill Owens
- Health Innovation Network, Academic Health Science Network, London, United Kingdom
| | - Aileen Jackson
- Health Innovation Network, Academic Health Science Network, London, United Kingdom
| | - Laura Semple
- The Academic Health Science Network, Hosted by Manchester Foundation Trust, London, United Kingdom
| | - Danielle Glennon
- Eating Disorders Outpatient Service, South London and Maudsley National Health Service (NHS) Foundation Trust, London, United Kingdom
| | - Giulia Di Clemente
- Eating Disorders Outpatient Service, South London and Maudsley National Health Service (NHS) Foundation Trust, London, United Kingdom
| | - Jess Griffiths
- Eating Disorders Outpatient Service, South London and Maudsley National Health Service (NHS) Foundation Trust, London, United Kingdom
| | - Regan Mills
- Centre for Research in Eating and Weight Disorders, Department of Psychological Medicine, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, London, United Kingdom
| | - Ulrike Schmidt
- Centre for Research in Eating and Weight Disorders, Department of Psychological Medicine, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, London, United Kingdom
- Eating Disorders Outpatient Service, South London and Maudsley National Health Service (NHS) Foundation Trust, London, United Kingdom
| |
Collapse
|
2
|
Dimitropoulos G, Singh M, Mushquash C, Kimber M, Hutt-MacLeod D, Moore EK, Perri A, Webb C, Norman L, Sauerwein J, Nicula M, Couturier J. TransitionED: A protocol for Co-designing and implementing Canadian practice guidelines for transitions for youth with eating disorders. EUROPEAN EATING DISORDERS REVIEW 2023. [PMID: 37690096 DOI: 10.1002/erv.3026] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2023] [Revised: 06/16/2023] [Accepted: 08/17/2023] [Indexed: 09/12/2023]
Abstract
OBJECTIVE Limited guidelines inform the transition from paediatric to adult healthcare for youth and young adults (YYA) with eating disorders (EDs). This study will develop, implement, and evaluate Canadian Clinical Practice Guidelines for ED transition, including identifying the relevant measurement and evaluation tools for transition readiness and continuity of care. METHODS This study consists of three phases. Phase 1 involves conducting a scoping review of the evidence on transition interventions, outcomes, and measurement tools for YYA with EDs, along with guideline development using the modified Delphi method. Phase 2 identifies the contextual/cultural factors relevant to guideline implementation and co-designing an implementation protocol with governance committees and research partners. Phase 3 involves the application and evaluation of the proposed guide lines using the implementation protocol, and assessing the acceptability and feasibility of a prototype transition intervention in two Canadian paediatric ED programs. CONCLUSIONS Results will provide the knowledge needed to enhance the lives of YYA, improve the effectiveness of the ED care system, and support the scale of the transition guidelines across Canada. These guidelines will have international relevance by potentially informing the field on how to support young people with EDs transitioning in similar funding structures and systems of care.
Collapse
Affiliation(s)
- Gina Dimitropoulos
- Faculty of Social Work, University of Calgary, Calgary, Alberta, Canada
- Department of Psychiatry, University of Calgary, Calgary, Alberta, Canada
| | - Manya Singh
- Department of Psychiatry, University of Calgary, Calgary, Alberta, Canada
| | - Christopher Mushquash
- Department of Psychology, Lakehead University, Thunder Bay, Ontario, Canada
- Dilico Anishinabek Family Care, Fort William First Nation, Ontario, Canada
- Thunder Bay Regional Health Sciences Centre/Thunder Bay Regional Health Research Institute, Thunder Bay, Ontario, Canada
| | - Melissa Kimber
- Department of Psychiatry and Behavioural Neurosciences, McMaster University, Hamilton, Ontario, Canada
- Offord Centre for Child Studies, McMaster University, Hamilton, Ontario, Canada
| | | | | | - Andrea Perri
- Child and Adolescent Addictions, Mental Health and Psychiatry Program, Alberta Health Services, Edmonton, Alberta, Canada
| | - Cheryl Webb
- Department of Psychiatry and Behavioural Neurosciences, McMaster University, Hamilton, Ontario, Canada
| | - Lia Norman
- Department of Psychiatry, University of Calgary, Calgary, Alberta, Canada
| | - Jessica Sauerwein
- Department of Psychiatry, University of Calgary, Calgary, Alberta, Canada
| | - Maria Nicula
- Department of Psychiatry and Behavioural Neurosciences, McMaster University, Hamilton, Ontario, Canada
| | - Jennifer Couturier
- Department of Psychiatry and Behavioural Neurosciences, McMaster University, Hamilton, Ontario, Canada
- Offord Centre for Child Studies, McMaster University, Hamilton, Ontario, Canada
| |
Collapse
|
3
|
Couturier J, Pellegrini D, Grennan L, Nicula M, Miller C, Agar P, Webb C, Anderson K, Barwick M, Dimitropoulos G, Findlay S, Kimber M, McVey G, Lock J. Multidisciplinary implementation of family-based treatment delivered by videoconferencing (FBT-V) for adolescent anorexia nervosa during the COVID-19 pandemic. Transl Behav Med 2023; 13:85-97. [PMID: 36327378 PMCID: PMC9972350 DOI: 10.1093/tbm/ibac086] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022] Open
Abstract
Family-Based Treatment (FBT)-the most widely supported treatment for pediatric eating disorders-transitioned to virtual delivery in many programs due to COVID-19. Using a blended implementation approach, we systematically examined therapist adherence to key components of FBT and fidelity to FBT by videoconferencing (FBT-V), preliminary patient outcomes, and team experiences with our FBT-V implementation approach as well as familial perceptions of FBT-V effectiveness. We examined our implementation approach across four pediatric eating disorder programs in Ontario, Canada, using mixed methods. Participants included therapists (n = 8), medical practitioners (n = 4), administrators (n = 6), and families (n = 5; 21 family members in total). We developed implementation teams at each site, provided FBT-V training, and offered clinical and implementation consultation. Therapists submitted video recordings of their first four FBT-V sessions for fidelity rating, and patient outcomes. Therapists self-reported readiness, attitudes, confidence, and adherence to FBT-V. Focus groups were conducted with each team and family after the first four sessions of FBT-V. Quantitative data were analyzed using repeated measures ANOVA. Qualitative data were analyzed using directed and summative content analysis. Therapists adhered to key FBT components and maintained FBT-V fidelity. Changes in therapists' readiness, attitudes, and confidence in FBT-V over time were not significant. All patients gained weight. Focus groups revealed implementation facilitators/barriers, positives/negatives surrounding FBT-V training and consultation, suggestions for improvement, and effectiveness attributed to FBT-V. Our implementation approach appeared to be feasible and acceptable. Future research with a larger sample is required, furthering our understanding of this approach and exploring how organizational factors influence treatment fidelity.
Collapse
Affiliation(s)
- Jennifer Couturier
- Department of Psychiatry and Behavioural Neurosciences, McMaster University, Hamilton, ON, Canada
- Eating Disorder Program, McMaster Children’s Hospital, Hamilton, ON, Canada
| | - Danielle Pellegrini
- Department of Psychiatry and Behavioural Neurosciences, McMaster University, Hamilton, ON, Canada
| | - Laura Grennan
- Department of Psychiatry and Behavioural Neurosciences, McMaster University, Hamilton, ON, Canada
| | - Maria Nicula
- Department of Psychiatry and Behavioural Neurosciences, McMaster University, Hamilton, ON, Canada
| | - Catherine Miller
- Eating Disorder Program, Canadian Mental Health Association, Waterloo Wellington, Kitchener, ON, Canada
| | - Paul Agar
- Eating Disorder Program, McMaster Children’s Hospital, Hamilton, ON, Canada
| | - Cheryl Webb
- Department of Psychiatry and Behavioural Neurosciences, McMaster University, Hamilton, ON, Canada
- Eating Disorder Program, McMaster Children’s Hospital, Hamilton, ON, Canada
| | | | - Melanie Barwick
- Social and Behavioural Health Sciences Division, Dalla Lana School of Public Health, University of Toronto, ON, Canada
- Child Health Evaluative Sciences, Research Institute, The Hospital for Sick Children, Toronto, ON, Canada
| | | | - Sheri Findlay
- Eating Disorder Program, McMaster Children’s Hospital, Hamilton, ON, Canada
- Department of Pediatrics, McMaster University, Hamilton, ON, Canada
| | - Melissa Kimber
- Department of Psychiatry and Behavioural Neurosciences, McMaster University, Hamilton, ON, Canada
| | - Gail McVey
- Social and Behavioural Health Sciences Division, Dalla Lana School of Public Health, University of Toronto, ON, Canada
- Ontario Community Outreach Program for Eating Disorders, University Health Network, Toronto, ON, Canada
| | - James Lock
- Department of Psychiatry and Behavioral Sciences, Stanford University, Stanford, CA, USA
| |
Collapse
|
4
|
Integrative review on psychological and social risk and prevention factors of eating disorders including anorexia nervosa and bulimia nervosa: seven major theories. Heliyon 2022; 8:e11422. [DOI: 10.1016/j.heliyon.2022.e11422] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2022] [Revised: 07/16/2022] [Accepted: 10/31/2022] [Indexed: 11/12/2022] Open
|
5
|
Zolfaghari E, Armaghanian N, Waller D, Medlow S, Hobbs A, Perry L, Nguyen K, Steinbeck K. Implementation science in adolescent healthcare research: an integrative review. BMC Health Serv Res 2022; 22:598. [PMID: 35505305 PMCID: PMC9066920 DOI: 10.1186/s12913-022-07941-3] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2021] [Accepted: 04/11/2022] [Indexed: 11/18/2022] Open
Abstract
Background Multiple theories, models and frameworks have been developed to assist implementation of evidence-based practice. However, to date there has been no review of implementation literature specific to adolescent healthcare. This integrative review therefore aimed to determine what implementation science theories, models and frameworks have been applied, what elements of these frameworks have been identified as influential in promoting the implementation and sustainability of service intervention, and to what extent, in what capacity and at what time points has the contribution of adolescent consumer perspectives on evidence implementation been considered. Methods An integrative design was used and reported based on a modified form of the PRISMA (2020) checklist. Seven databases were searched for English language primary research which included any implementation science theory, model or framework developed for/with adolescents or applied in relation to adolescent healthcare services within the past 10 years. Content and thematic analysis were applied with the Consolidated Framework for Implementation Research (CFIR) used to frame analysis of the barriers and facilitators to effective implementation of evidence-informed interventions within youth health settings. Results From 8717 citations, 13 papers reporting 12 studies were retained. Nine different implementation science theories, frameworks or approaches were applied; six of 12 studies used the CFIR, solely or with other models. All CFIR domains were represented as facilitators and barriers for implementation in included studies. However, there was little or no inclusion of adolescents in the development or review of these initiatives. Only three mentioned youth input, occurring in the pre-implementation or implementation stages. Conclusions The few studies found for this review highlight the internationally under-developed nature of this topic. Flagging the importance of the unique characteristics of this particular age group, and of the interventions and strategies to target it, the minimal input of adolescent consumers is cause for concern. Further research is clearly needed and must ensure that youth consumers are engaged from the start and consistently throughout; that their voice is prioritised and not tokenistic; that their contribution is taken seriously. Only then will age-appropriate evidence implementation enable innovations in youth health services to achieve the evidence-based outcomes they offer. Trial Registration PROSPERO 2020 CRD42020201142 https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=201142 Supplementary Information The online version contains supplementary material available at 10.1186/s12913-022-07941-3.
Collapse
Affiliation(s)
- Elham Zolfaghari
- Specialty of Child and Adolescent Health, Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, 2006, Australia.,Academic Department of Adolescent Medicine, The Children's Hospital Westmead, Westmead, NSW, 2145, Australia
| | - Natasha Armaghanian
- Specialty of Child and Adolescent Health, Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, 2006, Australia. .,Academic Department of Adolescent Medicine, The Children's Hospital Westmead, Westmead, NSW, 2145, Australia.
| | - Daniel Waller
- Specialty of Child and Adolescent Health, Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, 2006, Australia.,Academic Department of Adolescent Medicine, The Children's Hospital Westmead, Westmead, NSW, 2145, Australia.,Faculty of Health, University of Technology Sydney, Broadway, Ultimo, NSW, 2007, Australia
| | - Sharon Medlow
- Specialty of Child and Adolescent Health, Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, 2006, Australia.,Academic Department of Adolescent Medicine, The Children's Hospital Westmead, Westmead, NSW, 2145, Australia
| | - Annabelle Hobbs
- Specialty of Child and Adolescent Health, Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, 2006, Australia.,Academic Department of Adolescent Medicine, The Children's Hospital Westmead, Westmead, NSW, 2145, Australia
| | - Lin Perry
- Faculty of Health, University of Technology Sydney, Broadway, Ultimo, NSW, 2007, Australia.,South East Sydney Local Health District, Prince of Wales Hospital, Randwick, NSW, 2316, Australia
| | - Katie Nguyen
- Academic Department of Adolescent Medicine, The Children's Hospital Westmead, Westmead, NSW, 2145, Australia
| | - Katharine Steinbeck
- Specialty of Child and Adolescent Health, Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, 2006, Australia.,Academic Department of Adolescent Medicine, The Children's Hospital Westmead, Westmead, NSW, 2145, Australia
| |
Collapse
|
6
|
Peyser D, Sysko R, Webb L, Hildebrandt T. Treatment fidelity in eating disorders and psychological research: Current status and future directions. Int J Eat Disord 2021; 54:2121-2131. [PMID: 34622960 PMCID: PMC8719268 DOI: 10.1002/eat.23624] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/18/2021] [Revised: 09/26/2021] [Accepted: 09/27/2021] [Indexed: 11/07/2022]
Abstract
OBJECTIVE The evaluation and use of treatment fidelity procedures are rare in the development and implementation of psychological interventions. This article aims to review the construct of treatment fidelity, highlight limitations to currently available measures, and introduce a conceptual framework for studying and adapting fidelity measures in clinical research and practice using eating disorders as an example. METHOD As treatment fidelity assesses whether an intervention was delivered as intended, we operationalized this construct as: (a) treatment adherence, (b) therapist competence, and (c) treatment differentiation. RESULTS There is a significant gap in the literature assessing and documenting treatment fidelity. Available studies indicate that existing adherence measures can be time consuming, costly, and are not widely used in the field. Furthermore, therapist competence is a complex and context-dependent construct that is challenging to measure. Finally, treatment differentiation is often inferred by ensuring adherence. DISCUSSION The development of simplified formal tests of treatment fidelity would help draw conclusions about treatment efficacy and improve the dissemination and implementation of interventions to promote optimal clinical outcomes.
Collapse
Affiliation(s)
- Deena Peyser
- Eating and Weight Disorders Program, Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Robyn Sysko
- Eating and Weight Disorders Program, Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Lauren Webb
- Department of Psychology, Hofstra University, Hempstead, New York, USA
| | - Tom Hildebrandt
- Eating and Weight Disorders Program, Department of Psychiatry, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| |
Collapse
|
7
|
Couturier J, Pellegrini D, Miller C, Agar P, Webb C, Anderson K, Barwick M, Dimitropoulos G, Findlay S, Kimber M, McVey G, Lock J. Adapting and adopting highly specialized pediatric eating disorder treatment to virtual care: a protocol for an implementation study in the COVID-19 context. Implement Sci Commun 2021; 2:38. [PMID: 33832543 PMCID: PMC8027964 DOI: 10.1186/s43058-021-00143-8] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/26/2021] [Accepted: 03/30/2021] [Indexed: 01/03/2023] Open
Abstract
BACKGROUND The COVID-19 pandemic has negatively impacted individuals with eating disorders; resulting in increased symptoms, as well as feelings of isolation and anxiety. To conform with social distancing requirements, outpatient eating disorder treatment in Canada is being delivered virtually, but a lack of direction surrounding this change creates challenges for practitioners, patients, and families. As a result, there is an urgent need to not only adapt evidence-based care, including family-based treatment (FBT), to virtual formats, but to study its implementation in eating disorder programs. We propose to study the initial adaptation and adoption of virtual family-based treatment (vFBT) with the ultimate goal of improving access to services for youth with eating disorders. METHODS We will use a multi-site case study with a mixed method pre/post design to examine the impact of our implementation approach across four pediatric eating disorder programs. We will develop implementation teams at each site (consisting of therapists, medical practitioners, and program administrators), provide a remote training workshop on vFBT, and offer ongoing consultation during the initial implementation phase. Therapists will submit videorecordings of their first four vFBT sessions. We propose to study our implementation approach by examining (1) whether the key components of standard FBT are maintained in virtual delivery measured by therapist self-report, (2) fidelity to our vFBT model measured by expert fidelity rating of submitted videorecordings of the first four sessions of vFBT, (3) team and patient/family experiences with vFBT assessed with qualitative interviews, and (4) patient outcomes measured by weight and binge/purge frequency reported by therapists. DISCUSSION To our knowledge, this is the first study to evaluate an implementation strategy for virtually delivered FBT for eating disorders. Challenges to date include confirming site participation and obtaining ethics approval at all locations. This research is imperative to inform the delivery of vFBT in the COVID-19 context. It also has implications for delivery in a post-pandemic era where virtual services may be preferable to patients and families living in remote locations, where access to specialized services is extremely limited. TRIAL REGISTRATION ClinicalTrials.gov NCT04678843 , registered on December 21, 2020.
Collapse
Affiliation(s)
| | | | - Catherine Miller
- Canadian Mental Health Association - Waterloo Wellington, 1 Blue Springs Dr, Waterloo, Ontario, Canada
| | - Paul Agar
- McMaster University, 1280 Main St W, Hamilton, Ontario, Canada
| | - Cheryl Webb
- McMaster University, 1280 Main St W, Hamilton, Ontario, Canada
| | - Kristen Anderson
- Chicago Center for Evidence-Based Treatment, 25 E Washington St, Chicago, Illinois, USA
| | - Melanie Barwick
- Research Institute, The Hospital for Sick Children, 555 University Ave, Toronto, Ontario, Canada
- University of Toronto, 155 College St, Toronto, Ontario, Canada
| | | | - Sheri Findlay
- McMaster University, 1280 Main St W, Hamilton, Ontario, Canada
| | - Melissa Kimber
- McMaster University, 1280 Main St W, Hamilton, Ontario, Canada
| | - Gail McVey
- University of Toronto, 155 College St, Toronto, Ontario, Canada
- Research Institute, University Health Network, 200 Elizabeth St, Toronto, Ontario, Canada
| | - James Lock
- Stanford University, 401 Quarry Rd, Stanford, California, USA
| |
Collapse
|
8
|
Couturier J, Kimber M, Barwick M, McVey G, Findlay S, Webb C, Niccols A, Lock J. Assessing fidelity to family-based treatment: an exploratory examination of expert, therapist, parent, and peer ratings. J Eat Disord 2021; 9:12. [PMID: 33446271 PMCID: PMC7809847 DOI: 10.1186/s40337-020-00366-5] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/29/2020] [Accepted: 12/29/2020] [Indexed: 11/17/2022] Open
Abstract
INTRODUCTION Fidelity is an essential component for evaluating the clinical and implementation outcomes related to delivery of evidence-based practices (EBPs). Effective measurement of fidelity requires clinical buy-in, and as such, requires a process that is not burdensome for clinicians and managers. As part of a larger implementation study, we examined fidelity to Family-Based Treatment (FBT) measured by several different raters including an expert, a peer, therapists themselves, and parents, with a goal of determining a pragmatic, reliable and efficient method to capture treatment fidelity to FBT. METHODS Each therapist audio-recorded at least one FBT case and submitted recordings from session 1, 2, and 3 from phase 1, plus one additional session from phase 1, two sessions from phase 2, and one session from phase 3. These submitted files were rated by an expert and a peer rater using a validated FBT fidelity measure. As well, therapists and parents rated fidelity immediately following each session and submitted ratings to the research team. Inter-observer reliability was calculated for each item using the intraclass correlation coefficient (ICC), comparing the expert ratings to ratings from each of the other raters (parents, therapists, and peer). Mean scale scores were compared using repeated measures ANOVA. RESULTS Intraclass correlation coefficients revealed that agreement was the best between expert and peer, with excellent, good, or fair agreement in 7 of 13 items from session 1, 2 and 3. There were only four such values when comparing expert to parent agreement, and two such values comparing expert to therapist ratings. The rest of the ICC values indicated poor agreement. Scale level analysis indicated that expert fidelity ratings for phase 1 treatment sessions scores were significantly higher than the peer ratings and, that parent fidelity ratings tended to be significantly higher than the other raters across all three treatment phases. There were no significant differences between expert and therapist mean scores. CONCLUSIONS There may be challenges inherent in parents rating fidelity accurately. Peer rating or therapist self-rating may be considered pragmatic, efficient, and reliable approaches to fidelity assessment for real-world clinical settings.
Collapse
Affiliation(s)
- Jennifer Couturier
- Department of Psychiatry & Behavioural Neurosciences, McMaster University, Hamilton, Canada. .,Offord Centre for Child Studies, McMaster University, Hamilton, Canada.
| | - Melissa Kimber
- Department of Psychiatry & Behavioural Neurosciences, McMaster University, Hamilton, Canada.,Offord Centre for Child Studies, McMaster University, Hamilton, Canada
| | - Melanie Barwick
- Research Institute, The Hospital for Sick Children, Toronto, Canada
| | - Gail McVey
- Toronto General Hospital Research Institute, University Health Network, Toronto, Canada
| | - Sheri Findlay
- Department of Pediatrics, McMaster University, Hamilton, Canada
| | - Cheryl Webb
- Department of Psychiatry & Behavioural Neurosciences, McMaster University, Hamilton, Canada
| | - Alison Niccols
- Offord Centre for Child Studies, McMaster University, Hamilton, Canada
| | - James Lock
- Department of Psychiatry & Neurosciences, Stanford University, Stanford, USA
| |
Collapse
|