Validity of Clinician's Self-Reported Practice Elements on the Monthly Treatment and Progress Summary

Toward Efficient, Sustainable, and Scalable Methods of Treatment Characterization: An Investigation of Coding Clinical Practice from Chart Notes

PURPOSE

Chart notes provide a low-cost data source that could help characterize what occurs in treatment with sufficient precision to improve management of care. This study assessed the interrater reliability of treatment content coded from chart notes and evaluated its concordance with content coded from transcribed treatment sessions.

METHOD

Fifty randomly selected and digitally recorded treatment events were transcribed and coded for practice content. Independent coders then applied the same code system to chart notes for these same treatment events.

ANALYSIS

We measured reliability and concordance of practice occurrence and extensiveness at two levels of specificity: practices (full procedures) and steps (subcomponents of those procedures).

RESULTS

For chart notes, practices had moderate interrater reliability (M k = 0.50, M ICC = 0.56) and steps had moderate (M ICC = 0.74) to substantial interrater reliability (M k = 0.78). On average, 2.54 practices and 5.64 steps were coded per chart note and 4.53 practices and 13.10 steps per transcript. Across sources, ratings for 64% of practices and 41% of steps correlated significantly, with those with significant correlations generally demonstrating moderate concordance (practice M r = 0.48; step M r = 0.47). Forty one percent of practices and 34% of steps from transcripts were also identified in the corresponding chart notes.

CONCLUSION

Chart notes provide an accessible data source for evaluating treatment content, with different levels of specificity posing tradeoffs for validity and reliability, which in turn may have implications for chart note interfaces, training, and new metrics to support accurate, reliable, and efficient measurement of clinical practice.

Therapeutic techniques and session impact: A practice-research network study in private practice

Objective: This study investigated the relationship between therapeutic techniques and session impact, by examining the replicability of findings observed in a university-based training clinic (Boswell et al., 2010) in another practice-oriented setting: private practice.

Method: N = 8 therapists completed session-level assessments of their technique use for N = 38 clients. The same client sample completed session-level assessments of session outcome. Technique-outcome associations were examined with multilevel models.

Results: As in Boswell et al., common factors were associated with positive session impact. For clients who received higher average common factor techniques (relative to their own therapist's caseload), session impact was the poorest in sessions with higher behavioral change techniques use (relative to the client's own average). Moreover, clients with the lowest average common factor techniques (relative to their therapist's caseload) reported better session impact in sessions that involved a higher degree of session-level behavioral change techniques (relative to their own average).

Conclusion: In line with Boswell et al., therapists should be mindful of the consistency of their routine technique use between- and within-clients, and this can be aided through collection of their own practice-oriented data.

Measuring Evidence-Based Treatment Strategies in Youth Community Mental Health Care: the Evidence-Based Strategies Scale

While there are established measures for fidelity to specific evidence-based treatments, there is no widely accepted, feasible measure of the use of evidence-based treatment strategies in youth mental health (MH) care. This study examined the factor structure of a provider self-report measure of evidence-based treatment strategy use, the Evidence-Based Strategies Scale (EBSS). MH providers completed the EBSS as part of a larger mailed survey. The factor structure of the EBSS was examined using exploratory factor analysis in a national, multidisciplinary sample of MH providers (N = 1092), and confirmatory factor analysis was subsequently conducted to replicate this factor structure in a state-wide, multidisciplinary sample of Medicaid MH providers (N = 780). Findings indicated a three-factor structure, representing working alliance, youth-focused, and family-focused evidence-based treatment strategies factors. The EBSS offers a potential method for measuring the evidence-based treatment strategies being delivered in youth community MH care, though more research is needed.

Therapist Self-Report of Fidelity to Core Elements of Family Therapy for Adolescent Behavior Problems: Psychometrics of a Pragmatic Quality Indicator Tool

Therapist-report measures of evidence-based interventions have enormous potential utility as quality indicators in routine care; yet, few such tools have shown strong psychometric properties. This study describes reliability and validity characteristics of a therapist-report measure of family therapy techniques for treating adolescent conduct and substance use problems: Inventory of Therapy Techniques for Core Elements of Family Therapy (ITT-CEFT). Study participants included 31 staff therapists treating 68 adolescent clients in eight community-based mental health and substance use clinics. Therapists submitted ITT-CEFT checklists and companion audio recordings for 189 sessions. The ITT-CEFT contains 13 techniques identified as core elements of three manualized family therapy models that are empirically supported for the target group. Therapists also reported on their use of three motivational interventions, and independent observers coded the submitted recordings. ITT-CEFT factor validity was shown via confirmatory factor analyses of the tool's theoretical structure. Derived modules were: Family Engagement (four items; Cronbach's α = .72); Relational Orientation (five items; α = .74); and Interactional Change (four items; α = .66). Concurrent validity analyses showed fair-to-excellent therapist reliability compared to observer ratings (ICCs range .64-.75); they showed moderate therapist accuracy compared to observer mean scores, reflecting a tendency to overestimate delivery of the techniques. Discriminant validity analyses showed tool differentiation from motivational interventions. Results offer provisional evidence for the feasibility of using the therapist-report ITT-CEFT to anchor quality procedures for family therapy interventions in real-world settings.Trial Registration: The parent clinical trial is registered at www.ClinicalTrials.gov , ID: NCT03342872 (registration date: 11.10.17).

The impact of implementation climate on community mental health clinicians’ attitudes toward exposure: An evaluation of the effects of training and consultation

Background:

Most evidence-based treatments (EBTs) for posttraumatic stress disorder (PTSD) and anxiety disorders include exposure; however, in community settings, the implementation of exposure lags behind other EBT components. Clinician-level determinants have been consistently implicated as barriers to exposure implementation, but few organizational determinants have been studied. The current study examines an organization-level determinant, implementation climate, and clinician-level determinants, clinician demographic and background factors, as predictors of attitudes toward exposure and changes in attitudes following training.

Method:

Clinicians (n = 197) completed a 3-day training with 6 months of twice-monthly consultation. Clinicians were trained in cognitive behavioral therapy (CBT) for anxiety, depression, behavior problems, and trauma-focused CBT (TF-CBT). Demographic and background information, implementation climate, and attitudes toward exposure were assessed in a pre-training survey; attitudes were reassessed at post-consultation. Implementation climate was measured at the aggregated/group-level and clinician-level.

Results:

Attitudes toward exposure significantly improved from pre-training to post-consultation (t(193) = 9.9, p < .001; d = 0.71). Clinician-level implementation climate scores did not predict more positive attitudes at pre-training (p > .05) but did predict more positive attitudes at post-consultation (ß = −2.46; p < .05) and greater changes in those attitudes (ß = 2.28; p < .05). Group-level implementation climate scores did not predict attitudes at pre-training, post-consultation, or changes in attitudes (all ps > .05). Higher frequency of self-reported CBT use was associated with more positive attitudes at pre-training (ß = −0.81; p < .05), but no other clinician demographic or background determinants were associated with attitudes at post-consultation (all p > .05) or with changes in attitudes (all p > .05).

Conclusions:

Clinician perceptions of implementation climate predicted greater improvement of attitudes toward exposure following EBT training and consultation. Findings suggest that organizational determinants outside of training impact changes in clinicians’ attitudes. Training in four EBTs, only two of which include exposure as a component, resulted in positive changes in clinicians’ attitudes toward exposure, which suggests non-specialty trainings can be effective at changing attitudes, which may enable scale-up.

Exposure is highly effective for treating trauma symptoms and anxiety-based disorders, but it is not commonly used in community mental health settings. Clinicians who endorsed higher expectations, support, and rewards for using exposure in their agency had more positive attitudes toward exposure after training and consultation. Additionally, clinicians who endorsed that exposure is expected, supported, and rewarded in their agency showed a greater improvement in attitudes throughout the training process. Organizational culture can affect clinicians’ attitude changes in the training process, and therefore should become a focus of training efforts.

Practices Derived from the Evidence Base for Depression Predict Disruptive Behavior Progress in Adolescent Community Mental Health Care

Disruptive behavior problems develop along multiple causal pathways and are associated with a wide variety of co-occurring problems, including mood disorders. In usual care, effective treatment practices for youth disruptive behavior might differ from what the efficacy research suggests, given treatment setting and population demographic differences. The current study examined whether practices derived from the evidence base for disruptive behavior and/or depressed mood predicted progress on disruptive behavior problems in an adolescent usual care sample. Monthly clinical data, including therapeutic practices, treatment targets, and progress on selected treatment targets, for 1210 youth ages 13-17 who received intensive in-home services and were treated for disruptive behavior problems were examined utilizing multilevel modeling techniques. Practices derived from the evidence base for only depressed mood and practices derived from both disruptive behavior and depressed mood literatures predicted disruptive behavior progress, while practices derived from only the disruptive behavior evidence-based literature did not. All five practice elements exclusive to depressed mood treatment predicted positive disruptive behavior progress, while two of eleven disruptive behavior practices and four of seven practices derived from both problem areas predicted positive progress. Findings held when other predictors were included as covariates, including youth age and functional impairment. Although directionality remains unclear and further research is vital, usual care settings might present barriers to disruptive behavior treatment as prescribed by the evidence base, and youth-focused treatments based on depressed mood treatments merit further examination as a potentially promising route to effective treatment in such settings.

Therapist and Youth Predictors of Specific Practices Derived from the Evidence-Base in Community Mental Health

Over the past several years, youth treatment research has moved toward understanding the dissemination and implementation of evidence-based practices (EBPs). As a result, studies have focused on identifying predictors that aid in successful adoption and sustainment of EBPs. Theories of behavior change posit that therapist knowledge and attitudes play a fundamental role in EBP adoption; however, studies have produced mixed findings, which may be an artifact of broad definitions of both EBP knowledge and EBP itself. The current study was an examination of 46 youth community therapists and the extent to which varying types of knowledge and attitudes as well as youth characteristics predicted specific practices derived from the evidence-base. Results suggested that specific EBP knowledge predicted specific practices, highlighting the need for more specificity when examining predictors of EBP use. Therapists' attitudes, demographic characteristics, and youth characteristics were also significant predictors of EBP use. Future research should consider examining discrete and specific practices to better understand and predict therapists' future behavior.

Therapist-Observer Concordance in Ratings of EBP Strategy Delivery: Challenges and Targeted Directions in Pursuing Pragmatic Measurement in Children's Mental Health Services

Pragmatic measures of therapist delivery of evidence-based practice (EBP) are critical to assessing the impact of large-scale, multiple EBP implementation efforts. As an initial step in the development of pragmatic measurement, the current study examined the concordance between therapist and observer ratings of items assessing delivery of EBP strategies considered essential for common child EBP targets. Possible EBP-, session-, and therapist-levels factors associated with concordance were also explored. Therapists and independent observers rated the extensiveness of therapist (n = 103) EBP strategy delivery in 680 community psychotherapy sessions in which six EBPs were used. Concordance between therapist- and observer-report of the extensiveness of therapist EBP strategy use was at least fair (ICC ≥ .40) for approximately half of the items. Greater therapist-observer concordance was observed in sessions where a structured EBP was delivered and in sessions where therapists reported being able to carry out planned activities. Findings highlighted conditions that may improve or hinder therapists' ability to report on their own EBP strategy delivery in a way that is consistent with independent observers. These results can help inform the development of pragmatic therapist-report measures of EBP strategy delivery and implementation efforts more broadly.

Where Does It Begin? Community-Based Therapists' Intentions for Treating Non-comorbid Youth

This study examined patterns and predictors of 79 public sector therapists' practice element (PE) intentions for treating hypothetical single problem area youth with either anxiety or disruptive behavior problems. Analyses of intention profiles suggested that PE intentions varied by diagnosis and included both a majority of practices derived from the evidence base (PDEBs) and a large number of practices with minimal evidence support (PMESs) for each problem area. Hierarchical multiple regression results indicated the only significant predictor of higher PDEB intentions for both problem areas was reporting more PMES intentions. Implications for community-based research and implementation efforts are discussed.

Medical Image Enhancement by a Bilateral Filter Using Optimization Technique

For researchers, denoising of Magnetic Resonance (MR) image is a greatest challenge in digital image processing. In this paper, the impulse noise and Rician noise in the medical MR images are removed by using Bilateral Filter (BF). The novel approaches are presented in this paper; Enhanced grasshopper optimization algorithm (EGOA) is used to optimize the BF parameters. To simulate the medical MR images (with different variances), the impulse and Rician noises are added. The EGOA is applied to the noisy image in searching regions of window size, spatial and intensity domain to obtain the filter parameters optimally. The PSNR is taken as fitness value for optimization. We examined the proposed technique results with other MR images After the optimal parameters assurance. In order to comprehend the BF parameters selection importance, the results of proposed denoising method is contrasted with other previously used BFs, genetic algorithm (GA), gravitational search algorithm (GSA) using the quality metrics such as signal-to-noise ratio (SNR), structural similarity index metric (SSIM), mean squared error (MSE), and PSNR. The outcome shows that the EOGA method with BF shows good results than the earlier methods in both edge preservation and noise elimination from medical MR images. The experimental results demonstrate the performance of the proposed method with the accuracy, computational time, and maximum deviation, Peak Signal to Noise Ratio (PSNR), MSE, SSIM, and entropy values of MR images over the existing methods.

Predicting Youth Improvement in Community-Based Residential Settings with Practices Derived from the Evidence-Base

The current investigation conducted descriptive analyses on key variables in community-based residential (CBR) settings and investigated the extent to which disruptive youth between the ages of 13 and 17 years improved based on therapists' reported alignment with using practices derived from the evidence-base (PDEBs). Results from both the descriptive analyses and multilevel modeling suggested that therapists are using practices that both do and do not align with the evidence-base for disruptive youth. In addition, both PDEBs and practices with minimal evidence-support predicted or marginally predicted final average progress rating for these youth. Findings are discussed as they relate to the importance of continued exploration of treatment outcomes for CBR youth.

Evidence-Based Treatment Strategies in Youth Mental Health Services: Results from a National Survey of Providers

Previous surveys indicate infrequent use of evidence-based treatment (EBT) manuals in usual care youth mental health, but the extent to which providers use core and common EBT strategies and what contextual factors impact EBT strategy implementation need further study. In a national, multidisciplinary survey of 1092 youth-serving providers, providers reported regular use of many EBT strategies. Provider learning theory orientation, more recent degree, more standardized and ongoing assessment use, more positive attitudes toward innovation and evidence, fewer low-income clients, and perceptions that their agency valued quality care and provided fewer training resources predicted more frequent EBT strategy use.

Overlapping and Non-overlapping Practices in Usual and Evidence-Based Care for Youth Anxiety

This study compared consistencies and discrepancies in usual care with practices derived from the evidence-base (PDEB) for youth anxiety in a public mental health system. Youth-level factors (diagnosis, functional impairment) as predictors of the discrepancies were also examined. Psychosocial and service data from 2485 youth with an anxiety disorder and/or receiving services for an anxiety treatment target were extracted. Therapists (N = 616) identified the treatment targets and practices youth received. Although many PDEB for youth anxiety were used by therapists in this sample, Exposure was only used in 15% of cases. Practices not consistent with youth anxiety treatment were also reported and included: PDEB for other conditions, practices common to all therapies, and practices that are not consistent with evidence-based care. Age and diagnosis predicted the delivery of PDEB for youth anxiety. Usual care incorporated many components of evidence-based care but was more diffuse and less focused on well-supported practices.

Characterizing Intervention Strategies Used in Community-Based Mental Health Care for Infants and Their Families

Mental health interventions for infants typically target high-risk groups and can prevent long-term negative outcomes. Despite federal initiatives promoting early intervention, minimal research has examined usual care services for infants, which is important to improve routine care. The current study characterized usual care practices in infant mental health through the adaptation and administration of a provider survey. Providers (n = 126) reported using a wide range of intervention strategies and few intervention programs with varied evidence. Findings can inform future research to identify quality improvement targets of usual mental health care for high-risk infants and their families.

Factor Analysis of Therapist-Identified Treatment Targets in Community-Based Children's Mental Health

The present study used exploratory and confirmatory factor analyses to identify underlying latent factors affecting variation in community therapists' endorsement of treatment targets. As part of a statewide practice management program, therapist completed monthly reports of treatment targets (up to 10 per month) for a sample of youth (n = 790) receiving intensive in-home therapy. Nearly 75 % of youth were diagnosed with multiple co-occurring disorders. Five factors emerged: Disinhibition, Societal Rules Evasion, Social Engagement Deficits, Emotional Distress, and Management of Biodevelopmental Outcomes. Using logistic regression, primary diagnosis predicted therapist selection of Disinhibition and Emotional Distress targets. Client age predicted endorsement of Societal Rules Evasion targets. Practice-to-research implications are discussed.