1
|
Marriott BR, Andrews JH, Cho E, Tugendrajch SK, Hawley KM. Mental Health Provider Reach and Engagement in a Countywide Training Initiative. ADMINISTRATION AND POLICY IN MENTAL HEALTH AND MENTAL HEALTH SERVICES RESEARCH 2024; 51:530-542. [PMID: 38351412 DOI: 10.1007/s10488-024-01345-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/12/2024] [Indexed: 06/25/2024]
Abstract
Many training initiatives are underway to increase implementation of evidence-based practice (EBPs) in mental healthcare. However, little is known about what types of trainings and supports yield the highest reach and engagement. Supported by a tax-funded, countywide initiative to improve access to quality care for youths, the current mixed methods study evaluates mental health (MH) provider reach, or registering for the training initiative, and engagement, or participation in training activities, for several EBP training and implementation supports. MH providers were offered free 1) formal EBP workshops, 2) a biweekly learning community, 3) individual case consultation, and 4) confidential online clinical feedback system. To register, interested providers (N = 698) completed a web-based assessment measuring clinical practice information, organizational implementation climate, and EBP knowledge, attitudes, and practices. Thirteen providers, selected via purposeful sampling stratified by level of participation, completed semi-structured qualitative interviews. While the training initiative achieved high reach (66% of county agencies had a provider register), far fewer providers engaged substantially in training. Quantitative results indicated that providers whose professional discipline was not psychology, had higher baseline EBP knowledge, more extensive use of common evidence-based strategies, and less extensive use of other therapy strategies, engaged in more training. Rapid qualitative analysis of interviews expanded upon these findings and illuminated provider, organizational, system, practical, and training activity-specific barriers and facilitators to engagement. Findings suggest the importance of identifying strategies for improving provider engagement in training activities beyond workshops. Implications for future research and training initiatives are discussed.
Collapse
Affiliation(s)
- Brigid R Marriott
- Indiana University School of Medicine, Indianapolis, IN, USA.
- Department of Psychiatry, Indiana University School of Medicine, Indianapolis, IN, 46202, USA.
| | | | | | | | - Kristin M Hawley
- University of Missouri, Columbia, MO, USA.
- Department of Psychological Sciences, University of Missouri, Columbia, MO, 65211, USA.
| |
Collapse
|
2
|
Herschell AD, Kolko DJ, Scudder AT, Taber-Thomas SM, Schaffner KF, Hart JA, Mrozowski SJ, Hiegel SA, Iyengar S, Metzger A, Jackson CB. A Statewide Randomized Controlled Trial to Compare Three Models for Implementing Parent Child Interaction Therapy. JOURNAL OF CLINICAL CHILD AND ADOLESCENT PSYCHOLOGY : THE OFFICIAL JOURNAL FOR THE SOCIETY OF CLINICAL CHILD AND ADOLESCENT PSYCHOLOGY, AMERICAN PSYCHOLOGICAL ASSOCIATION, DIVISION 53 2023; 52:780-796. [PMID: 34928748 DOI: 10.1080/15374416.2021.2001745] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Abstract
OBJECTIVE This study (NIMH RO1 MH095750; ClinicalTrials.gov Identifier: NCT02543359) evaluated the effectiveness of three training models to implement a well-established evidence-based treatment, Parent-Child Interaction Therapy (PCIT). METHOD Fifty licensed outpatient clinics, including 100 clinicians, 50 supervisors, and 50 administrators were randomized to one of three training conditions: 1) Learning Collaborative (LC), 2) Cascading Model (CM) or 3) Distance Education (DE). Data to assess training and implementation outcomes were collected at 4 time points coinciding with the training period: baseline, 6- (mid), 12- (post), and 24-months (1-year follow-up). RESULTS Multi-level hierarchical linear growth modeling was used to examine changes over time in training outcomes. Results indicate that clinicians in CM were more likely to complete training, reported high levels of training satisfaction and better learning experiences compared to the other training conditions. However, supervisors in the LC condition reported greater learning experiences, higher levels of knowledge, understanding of treatment, and satisfaction compared to supervisors in other conditions. Although clinicians and supervisors in the DE condition did not outperform their counterparts on any outcomes, their performance was comparable to both LC and CM in terms of PCIT use, supervisor perceived acceptability, feasibility, system support, and clinician satisfaction. CONCLUSIONS Through the use of a randomized controlled design and community implementation, this study contributes to the current understanding of the impact of training design on implementation of PCIT. Results also indicate that although in-person training methods may produce more positive clinician and supervisor outcomes, training is not a one-size-fits-all model, with DE producing comparable results on some variables.
Collapse
Affiliation(s)
- Amy D Herschell
- Community Care Behavioral Health Organization, UPMC Insurance Services Division
- Department of Psychiatry, University of Pittsburgh School of Medicine
| | - David J Kolko
- Department of Psychiatry, University of Pittsburgh School of Medicine
| | - Ashley T Scudder
- Department of Human Development & Family Studies, Iowa State University
| | | | | | | | | | | | | | | | | |
Collapse
|
3
|
Shanholtz CE, Ridings LE, Espeleta HC, Anton MT, Hanson RF, Saunders B, Ruggiero K, Davidson T. A Comparison of the Readiness of Youth Service Agencies to Implement a Technology-Based Toolkit to Support Treatment Delivery. Community Ment Health J 2023; 59:1163-1171. [PMID: 36918477 DOI: 10.1007/s10597-023-01099-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/25/2022] [Accepted: 02/03/2023] [Indexed: 03/16/2023]
Abstract
Implementation initiatives and technology-based resources aim to address barriers to Evidence-Based Practice (EBP) use by creating generalizable techniques that can be used for a variety of youth-serving agencies. However, research has not carefully examined unique differences between agency types or individual programs in readiness to use such technologies and implementation strategies. The current study explored differences between community mental health clinics and child advocacy centers on organizational cultural factors (e.g., ability to change and commitment for change) to implement a novel technology-based toolkit to support delivery of Trauma Focused Cognitive Behavioral Therapy (TF-CBT). Results indicated that TF-CBT providers from child advocacy centers reported greater commitment to change and more support to use the technology-based system than those from community mental health centers. Findings suggest that implementation initiatives should address the needs of individual agencies and service settings and adaptations should be explored to best meet the needs of these settings.
Collapse
Affiliation(s)
- Caroline E Shanholtz
- College of Nursing, The Medical University of South Carolina, 99 Jonathan Lucas Street, Charleston, SC, 29425, USA.
- University of California, Los Angeles, 1285 Psychology Building, Los Angeles, CA, 90095, USA.
| | - Leigh E Ridings
- College of Nursing, The Medical University of South Carolina, 99 Jonathan Lucas Street, Charleston, SC, 29425, USA
| | - Hannah C Espeleta
- College of Nursing, The Medical University of South Carolina, 99 Jonathan Lucas Street, Charleston, SC, 29425, USA
| | - Margaret T Anton
- College of Nursing, The Medical University of South Carolina, 99 Jonathan Lucas Street, Charleston, SC, 29425, USA
| | - Rochelle F Hanson
- College of Nursing, The Medical University of South Carolina, 99 Jonathan Lucas Street, Charleston, SC, 29425, USA
| | - Benjamin Saunders
- College of Nursing, The Medical University of South Carolina, 99 Jonathan Lucas Street, Charleston, SC, 29425, USA
| | - Kenneth Ruggiero
- College of Nursing, The Medical University of South Carolina, 99 Jonathan Lucas Street, Charleston, SC, 29425, USA
| | - Tatiana Davidson
- College of Nursing, The Medical University of South Carolina, 99 Jonathan Lucas Street, Charleston, SC, 29425, USA
| |
Collapse
|
4
|
Müller LRF, Herold ML, Unterhitzenberger J, Rosner R. Development and evaluation of a training program for interpreters in the field of trauma-focused cognitive behavioral therapy. Front Psychol 2023; 14:1148690. [PMID: 37637919 PMCID: PMC10450153 DOI: 10.3389/fpsyg.2023.1148690] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2023] [Accepted: 06/30/2023] [Indexed: 08/29/2023] Open
Abstract
Background The treatment of traumatized refugee minors is often challenging because of language barriers. International guidelines, therefore, recommend the use of language mediators. However, there is a scarcity of evaluated training programs that prepare language mediators to translate during psychotherapy developed specifically for this patient group, for instance trauma-focused cognitive behavioral therapy (TF-CBT). Methods Based on an extensive literature review and in collaboration with an expert focus group, a one-day TF-CBT-specific online training program was developed for language mediators willing to work with minor refugees, and delivered on nine occasions between November 2020 and June 2021. The participants answered pre- and post-training questions about trauma- and TF-CBT-related knowledge and attitudes relevant to therapy, as well as the perceived usefulness of the training. Bayesian estimation was used to determine pre-post changes. Results A total of 129 participants speaking 35 different languages participated in the training program. Analyses revealed 95% highest density intervals not containing the null with respect to knowledge gain (effect size median 0.28) and change in treatment-appropriate attitudes (effect size median 0.31). The participants rated the training as useful. Conclusion The TF-CBT-specific training course was successfully carried out. It was likely to disseminate both knowledge gains and a shift toward more treatment-appropriate attitudes. It was perceived as useful by the participants. Given the scarcity of evaluated training programs for language mediators working with minor refugees, the results are promising. The limitations include the lack of both a control group and the verification of the results using an external outcome measure.
Collapse
Affiliation(s)
| | - Monja Lucia Herold
- Department of Psychology, Catholic University of Eichstätt-Ingolstadt, Eichstätt, Germany
| | - Johanna Unterhitzenberger
- Department of Child and Adolescent Psychiatry, Centre for Children and Adolescents Inn-Salzach e.V., Altoetting, Germany
| | - Rita Rosner
- Department of Psychology, Catholic University of Eichstätt-Ingolstadt, Eichstätt, Germany
| |
Collapse
|
5
|
Marriott BR, Kliethermes MD, McMillen JC, Proctor EK, Hawley KM. Implementation of a Low-Cost, Multi-component, Web-Based Training for Trauma-Focused Cognitive-Behavioral Therapy. ADMINISTRATION AND POLICY IN MENTAL HEALTH AND MENTAL HEALTH SERVICES RESEARCH 2023; 50:392-399. [PMID: 36583811 PMCID: PMC10461596 DOI: 10.1007/s10488-022-01246-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 12/14/2022] [Indexed: 12/31/2022]
Abstract
Effective, interactive trainings in evidence-based practices remain expensive and largely inaccessible to most practicing clinicians. To address this need, the current study evaluated the impact of a low-cost, multi-component, web-based training for Trauma-Focused Cognitive-Behavioral Therapy (TF-CBT) on clinicians' TF-CBT knowledge, strategy use, adherence and skill. Clinician members of a practice-based research network were recruited via email and randomized to either an immediate training group (N = 89 assigned) or waitlist control group (N = 74 assigned) that was offered access to the same training after six months, with half of each group further randomized to receive or not receive incentives for participation. Clinicians completed assessments at baseline, 6 months, and 12 months covering TF-CBT knowledge, strategy use, and for a subset of clinicians (n = 28), TF-CBT adherence and skill. Although significant differences in overall TF-CBT skillfulness and readiness were found, there were no significant differences between the training and waitlist control group on TF-CBT knowledge and strategy use at six months. However, there was considerable variability in the extent of training completed by clinicians. Subsequent post-hoc analyses indicated a significant, positive association between the extent of training completed by clinicians and clinician TF-CBT knowledge, strategy use, demonstrated adherence and skill across the three TF-CBT components, and overall TF-CBT readiness. We also explored whether incentives predicted training participation and found no differences in training activity participation between clinicians who were offered an incentive and those who were not. Findings highlight the limitations of self-paced web-based trainings. Implications for web-based trainings are discussed.
Collapse
Affiliation(s)
- Brigid R Marriott
- Department of Psychiatry, Indiana University School of Medicine, Indianapolis, IN, USA
| | - Matthew D Kliethermes
- Department of Psychological Sciences, Children's Advocacy Services of Greater St. Louis, University of Missouri-St. Louis, St. Louis, MO, USA
| | - J Curtis McMillen
- Crown Family School of Social Work, Policy and Practice, University of Chicago, IL, Chicago, USA
| | - Enola K Proctor
- Center for Mental Health Services Research, Brown School of Social Work, Washington University in St. Louis, St. Louis, MO, USA
| | - Kristin M Hawley
- Department of Psychological Sciences, University of Missouri, Columbia, MO, 65211, USA.
| |
Collapse
|
6
|
Ghafoori B, Salgado C, Ramos N, Murillo K, Quintana S. Utilizing a webinar to introduce an evidence-based trauma treatment in Chile: Results from a pilot study investigating clinician attitudes and needs. INTERNATIONAL JOURNAL OF MENTAL HEALTH 2023. [DOI: 10.1080/00207411.2023.2177478] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/13/2023]
Affiliation(s)
- Bita Ghafoori
- Department of Advanced Studies in Education and Counseling, California State University Long Beach, Long Beach, CA, USA
| | | | - Nadia Ramos
- Faculty of Psychology, University of Talca, Chile
| | - Karen Murillo
- Department of Advanced Studies in Education and Counseling, California State University Long Beach, Long Beach, CA, USA
| | - Sofia Quintana
- Department of Advanced Studies in Education and Counseling, California State University Long Beach, Long Beach, CA, USA
| |
Collapse
|
7
|
Andorko ND, Fitzgerald J, Roemer C, Solender E, Petti E, Rakhshan Rouhakhtar P, McNamara KE, Smith ME, Buchanan RW, Schiffman J, DeVylder J. Social work training to reduce duration of untreated psychosis: Methodology and considerations of a web-based training for community providers. Early Interv Psychiatry 2022; 16:393-401. [PMID: 34031987 PMCID: PMC8668167 DOI: 10.1111/eip.13178] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/05/2020] [Revised: 12/13/2020] [Accepted: 05/05/2021] [Indexed: 11/28/2022]
Abstract
AIM Outcomes for individuals with psychotic disorders can be improved through early intervention services; however, identification continues to be a major problem in connecting individuals with these services. Social workers form a vast majority of the human service and mental health workforce in the United States and therefore have the potential to play a unique role in identifying and referring those who may benefit from specialty early intervention services. METHODS The current article describes the methodological design, implementation, and participant recruitment procedures of a large-scale, web-based training program for social workers promoting identification and referral of individuals with emerging symptoms of a mental illness with psychosis in the context of a randomized clinical trial. RESULTS The web-based study enrolled 1384 individuals. More than half of study participants enrolled within the first 3 months of the 14-month recruitment period. Completion of all study components was achieved by 959 individuals (69% of total enrolled), and completion status did not vary significantly by gender, ethnicity, or facility at which the individual was employed. Completion rates varied by race, such that participants identifying as White were more likely to complete the study, while those identifying as Black were less likely. DISCUSSION The results suggest the feasibility of using a web-based training program to engage social workers in early psychosis identification practices. Challenges related to encouraging participants to complete the training and lessons learned during the study recruitment are discussed.
Collapse
Affiliation(s)
- Nicole D Andorko
- Department of Psychology, University of Maryland, Baltimore County, Baltimore, Maryland, USA
| | - John Fitzgerald
- Department of Psychology, University of Maryland, Baltimore County, Baltimore, Maryland, USA.,Maryland Psychiatric Research Center, Department of Psychiatry, University of Maryland School of Medicine, Baltimore, Maryland, USA
| | - Caroline Roemer
- Department of Psychology, University of Maryland, Baltimore County, Baltimore, Maryland, USA
| | - Eric Solender
- Department of Psychology, University of Maryland, Baltimore County, Baltimore, Maryland, USA
| | - Emily Petti
- Department of Psychology, University of Maryland, Baltimore County, Baltimore, Maryland, USA
| | | | - Karen E McNamara
- The Systems Evaluation Center, University of Maryland School of Medicine, Baltimore, Maryland, USA
| | - Melissa E Smith
- University of Maryland School of Social Work, Baltimore, Maryland, USA
| | - Robert W Buchanan
- Maryland Psychiatric Research Center, Department of Psychiatry, University of Maryland School of Medicine, Baltimore, Maryland, USA
| | - Jason Schiffman
- Department of Psychology, University of Maryland, Baltimore County, Baltimore, Maryland, USA.,Department of Psychological Science, University of California, Irvine, California, USA
| | - Jordan DeVylder
- Department of Psychological Science, University of California, Irvine, California, USA.,Graduate School of Social Service, Fordham University, New York, New York, USA
| |
Collapse
|
8
|
Kasparik B, Saupe LB, Mäkitalo S, Rosner R. Online training for evidence-based child trauma treatment: evaluation of the German language TF-CBT-Web. Eur J Psychotraumatol 2022; 13:2055890. [PMID: 35401949 PMCID: PMC8986224 DOI: 10.1080/20008198.2022.2055890] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/15/2022] Open
Abstract
BACKGROUND Evidence-based trauma-focused interventions for treating PTSD in children and youth are barely used in practice. Web-based training has proven to be an effective way of transferring knowledge to healthcare professionals. OBJECTIVE TF-CBT Web is a web-based training programme designed to foster the dissemination of Trauma Focused Cognitive Behavioural Therapy (TF-CBT) for children and youth, and is run by the Medical University of South Carolina. This paper describes the characteristics of healthcare professionals who registered for the adapted German language version of TF-CBT Web. It evaluates the effectiveness and user friendliness of the programme. METHOD : Similar to the TF-CBT treatment manual, the German language TF-CBT Web contains 12 modules. Between 2018 and 2020, 4,020 users registered for the programme. During the registration process users provided demographic information. The knowledge of users regarding the TF-CBT components was assessed via pre-tests and post-tests in each module. RESULTS The programme was accessed by a sample of mostly German users with varying professional health care backgrounds and a wide-ranging spread of work experience. The results indicated a significant knowledge gain and high rates of user satisfaction with the programme. CONCLUSIONS In summary, the results of this study suggested that web-based training is an effective and well-accepted method for knowledge gain in trauma-focused interventions. Future research should evaluate the actual application of the taught methods in clinical practice. HIGHLIGHTS Children and adolescents with PTSD require trauma-focused treatment. However, evidence-based interventions for this patient population are barely used. Therefore, it is necessary to expand professional training for the treatment of traumatised children and adolescents.Our evaluation showed the effectiveness and feasibility of a web-based training programme for mental health practitioners in an evidence-based treatment for children and youth with PTSD (TF-CBT). Results show a significant knowledge gain of users who participate in the web-based training programme.The user satisfaction survey also revealed that users found the modus and content of the web-based training applicable and relevant for their clinical practice.
Collapse
Affiliation(s)
- Barbara Kasparik
- Department of Psychology, Catholic University Eichstätt-Ingolstadt Ingolstadt, Germany
| | - Laura B Saupe
- Department of Psychology, Catholic University Eichstätt-Ingolstadt Ingolstadt, Germany
| | - Svenja Mäkitalo
- Department of Psychology, Catholic University Eichstätt-Ingolstadt Ingolstadt, Germany
| | - Rita Rosner
- Department of Psychology, Catholic University Eichstätt-Ingolstadt Eichstätt, Germany
| |
Collapse
|
9
|
Marriott BR, Cho E, Tugendrajch SK, Kliethermes MD, McMillen JC, Proctor EK, Hawley KM. Role-Play Assessment of Therapist Adherence and Skill in Implementation of Trauma-Focused Cognitive-Behavioral Therapy. ADMINISTRATION AND POLICY IN MENTAL HEALTH AND MENTAL HEALTH SERVICES RESEARCH 2021; 49:374-384. [PMID: 34546482 DOI: 10.1007/s10488-021-01169-9] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/13/2021] [Indexed: 11/24/2022]
Abstract
Numerous efforts are underway to train clinicians in evidence-based practices. Unfortunately, the field has few practical measures of therapist adherence and skill with which to judge the success of these training and implementation efforts. One possible assessment method is using behavioral rehearsal, or role-play, as an analogue for therapist in-session behavior. The current study describes aspects of reliability, validity and utility of a behavioral role-play assessment developed to evaluate therapist adherence and skill in implementing Trauma-Focused Cognitive-Behavioral Therapy (TF-CBT). TF-CBT role-play assessments were conducted with a sample of 43 therapists as part of a larger training study. The TF-CBT role-play assessments were independently coded for TF-CBT adherence and skill by a certified TF-CBT trainer and three clinical psychology doctoral students. Findings indicated good interrater reliability for the individual items (ICC: M = .71, SD = .15). Regarding utility, 67.19% (n = 43/64) of contacted therapists completed the role-play assessment, which took an average of 30 min (M = 31.42, SD = 5.65) to complete and 60 min (M = 62.84, SD = 11.31) to code. Therapists with a master's degree were more likely to complete the role-play assessment than those with other degrees but no other differences in demographic variables, practice characteristics, or TF-CBT knowledge or training were found between participants and nonparticipants. Role-play assessments may offer an alternative to observational coding for assessing therapist adherence and skill, particularly in contexts where session recordings are not feasible.
Collapse
Affiliation(s)
- Brigid R Marriott
- Department of Psychological Sciences, University of Missouri, Columbia, MO, 65211, USA
| | - Evelyn Cho
- Department of Psychological Sciences, University of Missouri, Columbia, MO, 65211, USA
| | - Siena K Tugendrajch
- Department of Psychological Sciences, University of Missouri, Columbia, MO, 65211, USA
| | - Matthew D Kliethermes
- Children's Advocacy Services of Greater St. Louis, Department of Psychological Sciences, University of Missouri-St. Louis (UMSL), St. Louis, MO, USA
| | - J Curtis McMillen
- School of Social Service Administration, University of Chicago, Chicago, IL, USA
| | - Enola K Proctor
- Center for Mental Health Services Research, Brown School of Social Work, Washington University in St. Louis, St. Louis, MO, USA
| | - Kristin M Hawley
- Department of Psychological Sciences, University of Missouri, Columbia, MO, 65211, USA.
| |
Collapse
|
10
|
Taylor DJ, Dietch JR, Pruiksma K, Calhoun CD, Milanak ME, Wardle-Pinkston S, Rheingold AA, Ruggiero KJ, Bunnell BE, Wilkerson AK. Developing and Testing a Web-Based Provider Training for Cognitive Behavioral Therapy of Insomnia. Mil Med 2021; 186:230-238. [PMID: 33499510 DOI: 10.1093/milmed/usaa359] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2020] [Revised: 08/31/2020] [Accepted: 09/14/2020] [Indexed: 12/29/2022] Open
Abstract
INTRODUCTION Chronic insomnia is a common and debilitating disease that increases risk for significant morbidity and workplace difficulties. Cognitive behavioral therapy for insomnia (CBT-I) is the first-line treatment, but there is a critical lack of behavioral health providers trained in CBT-I because, in part, of a bottleneck in training availability and costs. The current project developed and evaluated a web-based provider training course for CBT-I: CBTIweb.org. MATERIALS AND METHODS Subject matter experts developed the content for CBTIweb.org. Then, trainees completed alpha testing (n = 24) and focus groups, and the site was improved. Next, licensed behavioral health providers and trainees completed beta testing (n = 41) and the site underwent another round of modifications. Finally, to compare CBTIweb.org to an in-person workshop, licensed behavioral health providers were randomly assigned to CBTIweb.org (n = 21) or an in-person workshop (n = 23). All participants were CBT-I naïve and completed the following assessments: Computer System Usability Questionnaire, Website Usability Satisfaction Questionnaire, Website Content Satisfaction Questionnaire, and Continuing Education knowledge acquisition questionnaires. RESULTS Alpha and beta testers of CBTIweb.org reported high levels of usability and satisfaction with the site and showed significant within-group knowledge acquisition. In the pilot comparison study, linear fixed-effects modeling on the pre-/postquestionnaires revealed a significant main effect for time, indicating a significant increase in knowledge acquisition from 69% correct at baseline to 92% correct at posttraining collapsed across in-person and CBTIweb.org groups. The interaction effect of Time by Condition was nonsignificant, indicating equivalence in knowledge gains across both groups. CONCLUSION CBTIweb.org appears to be an engaging, interactive, and concise provider training that can be easily navigated by its users and produce significant knowledge gains that are equivalent to traditional in-person workshops. CBTIweb.org will allow for worldwide dissemination of CBT-I to any English-speaking behavioral health providers. Future research will work on translating this training to other languages and extending this web-based platform to the treatment of other sleep disorders (e.g., nightmares) and populations (e.g., pediatric populations with insomnia).
Collapse
Affiliation(s)
- Daniel J Taylor
- Department of Psychology, University of North Texas, Denton, TX 76203, USA.,Department of Psychology, University of Arizona, Tucson, AZ 85712, USA
| | - Jessica R Dietch
- Department of Psychology, University of North Texas, Denton, TX 76203, USA.,War Related Illness and Injury Study Center, VA Palo Alto Health Care System, Palo Alto, CA 94304, USA
| | - Kristi Pruiksma
- Department of Psychiatry, UT Health San Antonio, San Antonio, TX 78229, USA
| | - Casey D Calhoun
- Department of Psychiatry and Behavioral Sciences, Medical University of South Carolina, Charleston, SC 29425, USA
| | - Melissa E Milanak
- Department of Psychiatry and Behavioral Sciences, Medical University of South Carolina, Charleston, SC 29425, USA
| | - Sophie Wardle-Pinkston
- Department of Psychology, University of North Texas, Denton, TX 76203, USA.,Department of Psychology, University of Arizona, Tucson, AZ 85712, USA
| | - Alyssa A Rheingold
- Department of Psychiatry and Behavioral Sciences, Medical University of South Carolina, Charleston, SC 29425, USA
| | - Kenneth J Ruggiero
- Department of Psychiatry and Behavioral Sciences, Medical University of South Carolina, Charleston, SC 29425, USA
| | - Brian E Bunnell
- Department of Psychiatry and Behavioral Sciences, Medical University of South Carolina, Charleston, SC 29425, USA.,Department of Psychiatry and Behavioral Neurosciences, University of Southern Florida, Tampa, FL 33613, USA
| | - Allison K Wilkerson
- Department of Psychiatry and Behavioral Sciences, Medical University of South Carolina, Charleston, SC 29425, USA
| |
Collapse
|
11
|
Etherington C, Baker L, Ham M, Glasbeek D. Evaluating the Effectiveness of Online Training for a Comprehensive Violence Against Women Program: A Pilot Study. JOURNAL OF INTERPERSONAL VIOLENCE 2021; 36:160-183. [PMID: 29294884 DOI: 10.1177/0886260517725734] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/07/2023]
Abstract
Evaluating violence against women (VAW) training is essential to moving the field forward with proven approaches that can improve service provision for survivors of violence. Given existing resource constraints involved in VAW work, online training represents an economical and flexible option; however, existing evaluations of online programs in the VAW field are scant and face a variety of limitations. This study aimed to fill this gap by using a pre-/posttest design, comparison group, and mixed-method analysis to assess the effectiveness and value of an online training program. The program was intended to provide foundational knowledge in feminist antiviolence principles and values to a range of individuals working with survivors of intimate partner violence (IPV). Program participants (N = 108) included volunteers, students, and professionals from various sectors, allowing for the application of the results to a broader field of VAW support services. This is important as individuals who work with IPV survivors may do so in a range of settings outside of the shelter context. Results demonstrate the potential for online VAW training to improve participants' knowledge of and attitudes about VAW, which can positively inform their work with survivors. Qualitative responses provide further insight into course impact and highlight positive and negative aspects of the course. Although preliminary, these results provide justification for continued development and evaluation of online VAW training programs.
Collapse
Affiliation(s)
| | | | - Marlene Ham
- Ontario Association of Interval & Transition Houses, Toronto, Canada
| | - Denise Glasbeek
- Ontario Association of Interval & Transition Houses, Toronto, Canada
| |
Collapse
|
12
|
Rosner R, Barke A, Albrecht B, Christiansen H, Ebert DD, Lechner-Meichsner F, Muche R, Zarski AC, Steil R. BEST FOR CAN - bringing empirically supported treatments to children and adolescents after child abuse and neglect: study protocol. Eur J Psychotraumatol 2020; 11:1837531. [PMID: 33408810 PMCID: PMC7747933 DOI: 10.1080/20008198.2020.1837531] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/14/2020] [Revised: 10/02/2020] [Accepted: 10/02/2020] [Indexed: 01/10/2023] Open
Abstract
Background: Despite a large body of evidence demonstrating the effectiveness of psychotherapy for posttraumatic stress for children and adolescents, the adoption of empirically supported treatments (ESTs) in routine care is low. Objective: This implementation study aims to evaluate the dissemination of Trauma-Focused Cognitive Behavioural Therapy (TF-CBT) for children and adolescents with posttraumatic stress symptoms (PTSS) after child abuse and neglect (CAN) with a focus on supervision. Method: In a cluster-randomized controlled trial, the study will evaluate the implementation of TF-CBT focussing on the training of therapists including the provision of supervision. The effectiveness of specialized trauma-focused supervision will be compared to supervision as usual with respect to the successful implementation of TF-CBT for youths with PTSS administered by psychotherapists with different levels of professional experience. The primary outcome is whether the patient receives a treatment with sufficient adherence to the TF-CBT manual. The unit of randomization will be the therapists. The main outcome will be analysed using multilevel logistic regressions. Secondary outcomes will concern further patient-related (reduction of PTSS and depressive symptoms) and therapist-related (professional quality of life) variables. Additional exploratory analyses are planned. Discussion: Since the trial is designed as an implementation study, it permits naturalistic referrals to the participating therapists by patients, caregivers, child and youth welfare agencies and paediatricians. The strict primary outcome will help evaluating the role of model-based supervision in the implementation process. The explorative outcomes will evaluate whether implementation success translates into better patient outcomes. We expect that the dissemination measures will lead to a successful implementation of TF-CBT and promote sustainable structures in routine care that will remain in place after study completion and offer access to ESTs for future children and youths with a history of CAN.
Collapse
Affiliation(s)
- Rita Rosner
- Department of Psychology, Catholic University Eichstaett-Ingolstadt, Eichstaett, Germany
| | - Antonia Barke
- Department of Psychology, Catholic University Eichstaett-Ingolstadt, Eichstaett, Germany
| | - Björn Albrecht
- Department of Clinical Child and Adolescent Psychology and Psychotherapy, Philipps University Marburg, Marburg, Germany
| | - Hanna Christiansen
- Department of Clinical Child and Adolescent Psychology and Psychotherapy, Philipps University Marburg, Marburg, Germany
| | - David Daniel Ebert
- Department of Clinical Psychology and Psychotherapy, Friedrich-Alexander University Erlangen-Nuernberg, Erlangen, Germany
- Faculty of Behavioural and Movement Sciences, Clinical, Neuro- & Developmental Psychology, Vrije University Amsterdam, Amsterdam, The Netherlands
| | - Franziska Lechner-Meichsner
- Department of Clinical Psychology and Psychotherapy, Institute of Psychology, Goethe University Frankfurt, Frankfurt/Main, Germany
| | - Rainer Muche
- Institute of Epidemiology and Medical Biometry, Ulm University, Ulm, Germany
| | - Anna-Carlotta Zarski
- Department of Clinical Psychology and Psychotherapy, Friedrich-Alexander University Erlangen-Nuernberg, Erlangen, Germany
| | - Regina Steil
- Department of Clinical Psychology and Psychotherapy, Institute of Psychology, Goethe University Frankfurt, Frankfurt/Main, Germany
| |
Collapse
|
13
|
Deblinger E, Pollio E, Cooper B, Steer RA. Disseminating Trauma-Focused Cognitive Behavioral Therapy with a Systematic Self-care Approach to Addressing Secondary Traumatic Stress: PRACTICE What You Preach. Community Ment Health J 2020; 56:1531-1543. [PMID: 32318924 PMCID: PMC7529629 DOI: 10.1007/s10597-020-00602-x] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/09/2019] [Accepted: 03/06/2020] [Indexed: 11/29/2022]
Abstract
This pilot study evaluated the effectiveness of Trauma-Focused Cognitive Behavioral Therapy (TF-CBT) training programs augmented with a systematic "PRACTICE What You Preach" (PWYP) self-care focus, which has trainees personally utilize the coping skills they teach their clients. Participants were 115 clinicians/supervisors who completed a PWYP TF-CBT training program. Pre- to post-training analyses documented significant increases in participants' competency and fidelity in implementing TF-CBT (ps < .001), significantly more frequent use of coping skills including instrumental social support (p < .01), active coping (p < .001), humor (p < .01), and restraint (p < .01), and significant decreases in secondary traumatic stress (STS; p < .001). Children's symptoms of PTSD (ps < .001) and behavior problems (p < .05) also decreased significantly. This preliminary evidence suggests that training augmented with PWYP may enhance clinicians'/supervisors' personal coping and reduce their levels of STS without compromising treatment implementation efforts and client outcomes.
Collapse
Affiliation(s)
- Esther Deblinger
- CARES Institute, Rowan University School of Osteopathic Medicine (RowanSOM), 42 E. Laurel Road, Stratford, NJ, 08084, USA.
| | - Elisabeth Pollio
- CARES Institute, Rowan University School of Osteopathic Medicine (RowanSOM), 42 E. Laurel Road, Stratford, NJ, 08084, USA
| | - Beth Cooper
- CARES Institute, Rowan University School of Osteopathic Medicine (RowanSOM), 42 E. Laurel Road, Stratford, NJ, 08084, USA
| | - Robert A Steer
- Department of Psychiatry, Rowan University School of Osteopathic Medicine (RowanSOM), 42 E. Laurel Road, Stratford, NJ, 08084, USA
| |
Collapse
|
14
|
El Morr C, Layal M. Effectiveness of ICT-based intimate partner violence interventions: a systematic review. BMC Public Health 2020; 20:1372. [PMID: 32894115 PMCID: PMC7476255 DOI: 10.1186/s12889-020-09408-8] [Citation(s) in RCA: 22] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2020] [Accepted: 08/18/2020] [Indexed: 01/17/2023] Open
Abstract
BACKGROUND Intimate Partner Violence is a "global pandemic". Meanwhile, information and communication technologies (ICT), such as the internet, mobile phones, and smartphones, are spreading worldwide, including in low- and middle-income countries. We reviewed the available evidence on the use of ICT-based interventions to address intimate partner violence (IPV), evaluating the effectiveness, acceptability, and suitability of ICT for addressing different aspects of the problem (e.g., awareness, screening, prevention, treatment, mental health). METHODS We conducted a systematic review, following PRISMA guidelines, using the following databases: PubMed, PsycINFO, and Web of Science. Key search terms included women, violence, domestic violence, intimate partner violence, information, communication technology, ICT, technology, email, mobile, phone, digital, ehealth, web, computer, online, and computerized. Only articles written in English were included. RESULTS Twenty-five studies addressing screening and disclosure, IPV prevention, ICT suitability, support and women's mental health were identified. The evidence reviewed suggests that ICT-based interventions were effective mainly in screening, disclosure, and prevention. However, there is a lack of homogeneity among the studies' outcome measurements and the sample sizes, the control groups used (if any), the type of interventions, and the study recruitment space. Questions addressing safety, equity, and the unintended consequences of the use of ICT in IPV programming are virtually non-existent. CONCLUSIONS There is a clear need to develop women-centered ICT design when programming for IPV. Our study showed only one study that formally addressed software usability. The need for more research to address safety, equity, and the unintended consequences of the use of ICT in IPV programming is paramount. Studies addressing long term effects are also needed.
Collapse
Affiliation(s)
- Christo El Morr
- School of Health Policy and Management, York University, 4700 Keele St, Toronto, Ontario, Canada.
| | - Manpreet Layal
- Global Health Program, York University, 4700 Keele St, Toronto, Ontario, Canada
| |
Collapse
|
15
|
Sansen LM, Saupe LB, Steidl A, Fegert JM, Hoffmann U, Neuner F. Daring to process the trauma: using a web-based training to reduce psychotherapists' fears and reservations around implementing trauma-focused therapy. Eur J Psychotraumatol 2019; 10:1696590. [PMID: 31853333 PMCID: PMC6913602 DOI: 10.1080/20008198.2019.1696590] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/11/2019] [Revised: 10/22/2019] [Accepted: 10/26/2019] [Indexed: 11/29/2022] Open
Abstract
Although trauma-focused interventions are the first-line therapies for patients with posttraumatic stress disorder (PTSD), they are not frequently used in clinical practice. Factors preventing therapists from applying trauma-focused methods include a lack of training and negative attitudes towards trauma-focused therapy. The aim of the present study was to investigate which factors predict willingness to carry out trauma-focused therapy and to examine whether a web-based training is able to reduce negative attitudes and reservations about these interventions. In a wait-list controlled evaluation study, therapists (N = 499) were randomized into an intervention or a wait-list control group. Results show that trauma-treatment specific competencies and overcoming pre-existing concerns towards trauma-focused therapy significantly predict therapists' willingness to utilize trauma-focused interventions. Thus, the content alignment of the web-based course is appropriate for improving therapists' willingness to conduct trauma-focused therapy. A retrospective examination of therapists after the training and a comparison of fears and reservations before and after the training demonstrate a significant reduction of fears and reservations. In terms of perceived contraindications, no effects of the web-based training were found. The present study provides compelling evidence that web-based training in evidence-based PTSD therapy is able to reduce reservations that may prevent therapists from applying evidence-based trauma-focused interventions.
Collapse
Affiliation(s)
- Lisa M. Sansen
- Faculty of Psychology, Clinical Psychology
and Psychotherapy, Bielefeld University, Bielefeld,
Germany
| | - Laura B. Saupe
- Faculty of Psychology, Clinical Psychology
and Psychotherapy, Bielefeld University, Bielefeld,
Germany
| | - Annika Steidl
- Faculty of Psychology, Clinical Psychology
and Psychotherapy, Bielefeld University, Bielefeld,
Germany
| | - Jörg M. Fegert
- Department of Child and Adolescent
Psychiatry/Psychotherapy, University Hospital Ulm, Ulm,
Germany
| | - Ulrike Hoffmann
- Department of Child and Adolescent
Psychiatry/Psychotherapy, University Hospital Ulm, Ulm,
Germany
| | - Frank Neuner
- Faculty of Psychology, Clinical Psychology
and Psychotherapy, Bielefeld University, Bielefeld,
Germany
| |
Collapse
|
16
|
Jackson CB, Quetsch LB, Brabson LA, Herschell AD. Web-Based Training Methods for Behavioral Health Providers: A Systematic Review. ADMINISTRATION AND POLICY IN MENTAL HEALTH AND MENTAL HEALTH SERVICES RESEARCH 2018; 45:587-610. [PMID: 29352459 PMCID: PMC6002894 DOI: 10.1007/s10488-018-0847-0] [Citation(s) in RCA: 71] [Impact Index Per Article: 11.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/17/2022]
Abstract
There has been an increase in the use of web-based training methods to train behavioral health providers in evidence-based practices. This systematic review focuses solely on the efficacy of web-based training methods for training behavioral health providers. A literature search yielded 45 articles meeting inclusion criteria. Results indicated that the serial instruction training method was the most commonly studied web-based training method. While the current review has several notable limitations, findings indicate that participating in a web-based training may result in greater post-training knowledge and skill, in comparison to baseline scores. Implications and recommendations for future research on web-based training methods are discussed.
Collapse
Affiliation(s)
- Carrie B Jackson
- West Virginia University, Morgantown, WV, USA.
- Department of Psychology, West Virginia University, Life Sciences Building, 65 Campus Drive, Morgantown, WV, 26505, USA.
| | | | | | - Amy D Herschell
- West Virginia University, Morgantown, WV, USA
- Western Psychiatric Institute and Clinic, University of Pittsburgh School of Medicine, Pittsburgh, PA, USA
| |
Collapse
|
17
|
Talley R, Chiang IC, Covell NH, Dixon L. Comparative Initial and Sustained Engagement in Web-based Training by Behavioral Healthcare Providers in New York State. JOURNAL OF TECHNOLOGY IN BEHAVIORAL SCIENCE 2018; 3:41-48. [PMID: 29732398 PMCID: PMC5931788 DOI: 10.1007/s41347-017-0027-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
Improved dissemination is critical to implementation of evidence-based practice in community behavioral healthcare settings. Web-based training modalities are a promising strategy for dissemination of evidence-based practice in community behavioral health settings. Initial and sustained engagement of these modalities in large, multidisciplinary community provider samples is not well understood. This study evaluates comparative engagement and user preferences by provider type in a web-based training platform in a large, multidisciplinary community sample of behavioral health staff in New York State. Workforce make-up among platform registrants was compared to the general NYS behavioral health workforce. Training completion by functional job type was compared to characterize user engagement and preferences. Frequently completed modules were classified by credit and requirement incentives. High initial training engagement across professional role was demonstrated, with significant differences in initial and sustained engagement by professional role. The most frequently completed modules across functional job types contained credit or requirement incentives. The analysis demonstrated that high engagement of a web-based training in a multidisciplinary provider audience can be achieved without tailoring content to specific professional roles. Overlap between frequently completed modules and incentives suggests a role for incentives in promoting engagement of web-based training. These findings further the understanding of strategies to promote large-scale dissemination of evidence-based practice in community behavioral health settings.
Collapse
Affiliation(s)
- Rachel Talley
- New York State Psychiatric Institute, Department of Psychiatry, Columbia University Medical Center, 1051 Riverside Drive, New York, New York, 10032
| | - I-Chin Chiang
- Data Analyst, New York State Psychiatric Institute, 1051 Riverside Drive, New York, New York, 10032
| | - Nancy H Covell
- Assistant Professor of Clinical Psychology, New York State Psychiatric Institute, Department of Psychiatry, Columbia University Medical Center, 1051 Riverside Drive, New York, New York, 10032
| | - Lisa Dixon
- Professor of Psychiatry, New York State Psychiatric Institute, Department of Psychiatry, Columbia University Medical Center, 1051 Riverside Drive, New York, New York, 10032
| |
Collapse
|
18
|
Evidence-based training in the era of evidence-based practice: Challenges and opportunities for training of PTSD providers. Behav Res Ther 2017; 88:37-48. [DOI: 10.1016/j.brat.2016.07.009] [Citation(s) in RCA: 32] [Impact Index Per Article: 4.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2016] [Revised: 07/24/2016] [Accepted: 07/26/2016] [Indexed: 11/22/2022]
|
19
|
Abstract
OBJECTIVE This article provides information about trauma-focused cognitive behavioral therapy (TF-CBT), an evidence-based treatment for traumatized children, adolescents, and families. METHOD The evolution of the TF-CBT model is described from the perspective of the treatment developers, including population of focus, conceptual and methodological features of the research, critical challenges and design issues that have been confronted, and how they have been addressed. Major research findings and their implications for clinical practice are also described, as well as future research challenges and directions for young researchers starting out in this field. RESULTS The TF-CBT model has been been tested in a variety of challenging research settings and has strong evidence for improving trauma symptoms across diverse populations of traumatized children. CONCLUSIONS TF-CBT is an effective and widely used treatment for addressing childhood trauma.
Collapse
Affiliation(s)
- Judith A Cohen
- a Allegheny General Hospital, Allegheny Health Network , Temple University School of Medicine , Pittsburgh , PA , USA
| | - Esther Deblinger
- b CARES Institute, Rowan University-School of Osteopathic Medicine , Stratford , NJ , USA
| | - Anthony P Mannarino
- c Department of Psychiatry, Allegheny General Hospital, Allegheny Health Network , Temple University School of Medicine , Pittsburgh , PA , USA
| |
Collapse
|
20
|
Cohen JA, Mannarino AP, Jankowski K, Rosenberg S, Kodya S, Wolford GL. A Randomized Implementation Study of Trauma-Focused Cognitive Behavioral Therapy for Adjudicated Teens in Residential Treatment Facilities. CHILD MALTREATMENT 2016; 21:156-67. [PMID: 26747845 PMCID: PMC5175405 DOI: 10.1177/1077559515624775] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/26/2023]
Abstract
Adjudicated youth in residential treatment facilities (RTFs) have high rates of trauma exposure and post-traumatic stress disorder (PTSD). This study evaluated strategies for implementing trauma-focused cognitive behavioral therapy (TF-CBT) in RTF. Therapists (N = 129) treating adjudicated youth were randomized by RTF program (N = 18) to receive one of the two TF-CBT implementation strategies: (1) web-based TF-CBT training + consultation (W) or (2) W + 2 day live TF-CBT workshop + twice monthly phone consultation (W + L). Youth trauma screening and PTSD symptoms were assessed via online dashboard data entry using the University of California at Los Angeles PTSD Reaction Index. Youth depressive symptoms were assessed with the Mood and Feelings Questionnaire-Short Version. Outcomes were therapist screening; TF-CBT engagement, completion, and fidelity; and youth improvement in PTSD and depressive symptoms. The W + L condition resulted in significantly more therapists conducting trauma screening (p = .0005), completing treatment (p = .03), and completing TF-CBT with fidelity (p = .001) than the W condition. Therapist licensure significantly impacted several outcomes. Adjudicated RTF youth receiving TF-CBT across conditions experienced statistically and clinically significant improvement in PTSD (p = .001) and depressive (p = .018) symptoms. W + L is generally superior to W for implementing TF-CBT in RTF. TF-CBT is effective for improving trauma-related symptoms in adjudicated RTF youth. Implementation barriers are discussed.
Collapse
Affiliation(s)
| | | | - Kay Jankowski
- Dartmouth Hitchcock Medical Center, Lebanon, NH, USA
| | | | | | | |
Collapse
|