Understanding barriers to evidence-based assessment: clinician attitudes toward standardized assessment tools

Some hope for a dimensional assessment? A critical review of psychometric validated (semi-)structured interview to assess eating disorders

OBJECTIVE

Given that eating disorders (EDs) are considered one of the deadliest mental illnesses, the development of appropriate assessment instruments is a necessity. Despite the extensive literature on assessing EDs, there has been a lack of focus on semi-structured interviews. The purpose of this article is to provide a comprehensive review of psychometrically validated semi-structured interviews for EDs.

METHODS

Included studies (N = 24) were required to present a semi-structured interview for EDs that has been validated through a psychometric process. The APA PsycNet, MEDLINE, APA Psycinfo, Pubmed, and Health & Psychosocial Instruments databases were searched. The literature search included publications through May 2024, with no earliest year restriction.

RESULTS

A total of six instruments were identified and reviewed in terms of conceptual design, purpose and content, psychometric characteristics, and strengths and limitations. Three main findings were highlighted: (a) only half of the instruments are up to date; (b) the instruments are based on either a categorical or a mixed categorical-dimensional approach; and (c) the predominance of the categorical approach.

CONCLUSIONS

The results are discussed regarding the conceptual approaches of the instrument to provide clinical and research implications. Despite the many strengths of the instrument, additional psychometric research is needed.

Experiences of medical students and faculty regarding the use of long case as a formative assessment method at a tertiary care teaching hospital in a low resource setting: a qualitative study

INTRODUCTION

The long case is used to assess medical students' proficiency in performing clinical tasks. As a formative assessment, the purpose is to offer feedback on performance, aiming to enhance and expedite clinical learning. The long case stands out as one of the primary formative assessment methods for clinical clerkship in low-resource settings but has received little attention in the literature.

OBJECTIVE

To explore the experiences of medical students and faculty regarding the use of the Long Case Study as a formative assessment method at a tertiary care teaching hospital in a low-resource setting.

METHODOLOGY

A qualitative study design was used. The study was conducted at Makerere University, a low-resource setting. The study participants were third- and fifth-year medical students as well as lecturers. Purposive sampling was utilized to recruit participants. Data collection comprised six Focus Group Discussions with students and five Key Informant Interviews with lecturers. The qualitative data were analyzed by inductive thematic analysis.

RESULTS

Three themes emerged from the study: ward placement, case presentation, and case assessment and feedback. The findings revealed that students conduct their long cases at patients' bedside within specific wards/units assigned for the entire clerkship. Effective supervision, feedback, and marks were highlighted as crucial practices that positively impact the learning process. However, challenges such as insufficient orientation to the long case, the super-specialization of the hospital wards, pressure to hunt for marks, and inadequate feedback practices were identified.

CONCLUSION

The long case offers students exposure to real patients in a clinical setting. However, in tertiary care teaching hospitals, it's crucial to ensure proper design and implementation of this practice to enable students' exposure to a variety of cases. Adequate and effective supervision and feedback create valuable opportunities for each learner to present cases and receive corrections.

Evidence-based brief interventions targeting acute mental health presentations for children and adolescents: systematic review

BACKGROUND

Brief intervention services provide rapid, mobile and flexible short-term delivery of interventions to resolve mental health crises. These interventions may provide an alternative pathway to the emergency department or in-patient psychiatric services for children and young people (CYP), presenting with an acute mental health condition.

AIMS

To synthesise evidence on the effectiveness of brief interventions in improving mental health outcomes for CYP (0-17 years) presenting with an acute mental health condition.

METHOD

A systematic literature search was conducted, and the studies' methodological quality was assessed. Five databases were searched for peer-reviewed articles between January 2000 and September 2022.

RESULTS

We synthesised 30 articles on the effectiveness of brief interventions in the form of (a) crisis intervention, (b) integrated services, (c) group therapies, (d) individualised therapy, (e) parent-child dyadic therapy, (f) general services, (g) pharmacotherapy, (h) assessment services, (i) safety and risk planning and (j) in-hospital treatment, to improve outcomes for CYP with an acute mental health condition. Among included studies, one study was rated as providing a high level of evidence based on the National Health and Medical Research Council levels of evidence hierarchy scale, which was a crisis intervention showing a reduction in length of stay and return emergency department visits. Other studies, of moderate-quality evidence, described multimodal brief interventions that suggested beneficial effects.

CONCLUSIONS

This review provides evidence to substantiate the benefits of brief interventions, in different settings, to reduce the burden of in-patient hospital and readmission rates to the emergency department.

Understanding community provider practices in diagnosing and treating atypical anorexia nervosa: A mixed methods study

OBJECTIVE

There is a lack of consensus in defining "significant weight loss" when diagnosing atypical anorexia nervosa (atypical AN) and no guidelines exist for setting target weight (TW). The current study aimed to identify community providers' practices related to the diagnosis of atypical AN and the determination of TW. A secondary aim was to evaluate whether professional discipline impacted "significant weight loss" definitions.

METHOD

A variety of providers (N = 141; 96.4% female) completed an online survey pertaining to diagnostic and treatment practices with atypical AN. Descriptive statistics were computed to characterize provider-based practices and Fisher's exact tests were used to test for differences in diagnostic practices by professional discipline. Thematic analysis was used to examine open-ended questions.

RESULTS

Most (63.97%) providers diagnosed atypical AN in the absence of any weight loss if other AN criteria were met, but doctoral-level psychologists and medical professionals were less likely to do so compared to nutritional or other mental health professionals. Most providers found weight gain was only sometimes necessary for atypical AN recovery. Qualitative responses revealed providers found atypical AN to be a stigmatizing label that was not taken seriously. Providers preferred to use an individualized approach focused on behaviors, rather than weight when diagnosing and treating atypical AN.

DISCUSSION

Lack of diagnostic clarity and concrete treatment guidelines for atypical AN may result in substantial deviations from the DSM-5-TR criteria in real-world practice. Clinically useful diagnostic definitions for restrictive eating disorders and evidence-based treatment guidelines for TW and/or other relevant recovery metrics are needed.

PUBLIC SIGNIFICANCE

The current study found variability in how community providers diagnose and determine target recovery weight for atypical anorexia nervosa (atypical AN). Many providers viewed the diagnosis of atypical AN as stigmatizing and preferred to focus on behaviors, rather than weight. This study underscores the importance of creating a clinically useful diagnostic definition and guidelines for recovery for atypical AN backed by empirical evidence that providers may implement in practice.

Psychotherapists' Experience with In-Session Use of Routine Outcome Monitoring: A Qualitative Meta-analysis

Routine outcome monitoring (ROM) has become an increasingly utilized tool in therapeutic practice that has the potential to improve therapy outcomes. This study aimed to synthesize the findings of existing qualitative studies investigating how clinicians use ROM in their work with clients. A systematic search of qualitative studies on clinicians' experience with the use of ROM in mental health services was conducted via PsycInfo, PsycArticles, Medline, Web of Science, and Scopus databases. Qualitative meta-analysis was used to synthesize the finding of the primary studies. Forty-seven studies met the inclusion criteria. The analysis resulted in 21 meta-categories organized into six clusters, namely (1) obtaining clinically relevant information, (2) adapting treatment, (3) facilitating communication, (4) enhancing the therapeutic relationship, (5) facilitating change in clients, and (6) personalized usage of ROM. The meta-analysis revealed that clinicians utilized ROM in diverse ways, including both informational and communicational functions. From the clinicians' perspective, ROM was an element that, on the one hand, introduced additional structure and standardization in treatment and, on the other hand, allowed for greater flexibility and tailoring of treatment.

Limitations in utilization and prioritization of standardized somatosensory assessments after stroke: A cross-sectional survey of neurorehabilitation clinicians

BACKGROUND AND PURPOSE

Somatosensory impairments are common after stroke, but receive limited evaluation and intervention during neurorehabilitation, despite negatively impacting functional movement and recovery.

OBJECTIVES

Our objective was to understand the scope of somatosensory assessments used by clinicians in stroke rehabilitation, and barriers to increasing use in clinical practice.

METHODS

An electronic survey was distributed to clinicians (physical therapists, occupational therapists, physicians, and nurses) who assessed at least one individual with stroke in the past 6 months. The survey included questions on evaluation procedures, type, and use of somatosensory assessments, as well as barriers and facilitators in clinical practice.

RESULTS

Clinicians (N = 431) indicated greater familiarity with non-standardized assessments, and greater utilization compared to standardized assessments (p < 0.0001). Components of tactile sensation were the most commonly assessed modality of somatosensation (25%), while proprioception was rarely assessed (1%). Overall, assessments of motor function were prioritized over assessments of somatosensory function (p < 0.0001).

DISCUSSION

Respondents reported assessing somatosensation less frequently than motor function and demonstrated a reliance on rapid and coarse non-standardized assessments that ineffectively capture multi-modal somatosensory impairments, particularly for proprioceptive deficits common post-stroke. In general, clinicians were not familiar with standardized somatosensory assessments, and this knowledge gap likely contributes to lack of translation of these assessments into practice.

CONCLUSIONS

Clinicians utilize somatosensory assessments that inadequately capture the multi-modal nature of somatosensory impairments in stroke survivors. Addressing barriers to clinical translation has the potential to increase utilization of standardized assessments to improve the characterization of somatosensory deficits that inform clinical decision-making toward enhancing stroke rehabilitation outcomes.

Evidence-Based Assessment of Personality Disorder

The purpose of this article is to provide a description and discussion of the evidence-based assessment of personality disorder. Considered herein is the assessment of the Section II personality disorders included within the fifth edition of the American Psychiatric Association's (APA) Diagnostic and Statistical Manual of Mental Disorders (5th ed., text rev.; DSM-5-TR), within Section III of DSM-5-TR, and within the 11th edition of the World Health Organization's International Classification of Diseases (WHO). The recommendation for an evidence-based assessment is for a multimethod approach: first administer a self-report inventory to alert the clinician to maladaptive personality functioning that might not have otherwise been anticipated, followed by a semi-structured interview to verify the personality disorder's presence. The validity of this multimethod strategy can be improved further by considering the impact of other disorders on the assessment, documenting temporal stability, and establishing a compelling, empirical basis for cutoff points.

Visualizing Uncertainty to Promote Clinicians' Understanding of Measurement Error

Measurement error is an inherent part of any test score. This uncertainty is generally communicated in ways that can be difficult to understand for clinical practitioners. In this empirical study, we evaluate the impact of several communication formats on the interpretation of measurement accuracy and its influence on the decision-making process in clinical practice. We provided 230 clinical practitioners with score reports in five formats: textual, error bar, violin plot, diamond plot, and quantile dot plot. We found that quantile dot plots significantly increased accuracy in the assessment of measurement uncertainty compared with other formats. However, a direct relation between visualization format and decision quality could not be found. Although traditional confidence intervals and error bars were favored by many participants due to their familiarity, responses revealed several misconceptions that make the suitability of these formats for communicating uncertainty questionable. Our results indicate that new visualization formats can successfully reduce errors in interpretation.

A Structural Model of Organization and Clinician Factors Associated with Standardized Measure Use in a National Survey of Youth Mental Health Providers

Standardized assessment measures are important for accurate diagnosis of mental health problems and for treatment planning and evaluation. However, little is known about youth mental health providers' typical use of standardized measures across disciplines and outside the context of evidence-based practice initiatives. A multidisciplinary national survey examined the frequency with which 674 youth mental health providers administer standardized and unstandardized measures, and the extent to which organizational (i.e., implementation climate, rigid hierarchical organizational structure) and provider (i.e., attitudes toward standardized assessment measures, highest degree, practice setting) characteristics are associated with standardized measure use. Providers used unstandardized measures far more frequently than standardized measures. Providers' perceptions (a) that standardized measures are practical or feasible, (b) that their organization supports and values evidence-based practices, and (c) that their organization has a rigid hierarchical structure predicted greater use of standardized measures. Working in schools predicted less frequent SMU, while working in higher education and other professional settings predicted more frequent SMU. Standardized measures were not routinely used in this community-based sample. A rigid hierarchical organizational structure may be conducive to more frequent administration of standardized measures, but it is unclear whether such providers actually utilize these measures for clinical decision-making. Alternative strategies to promote standardized measure use may include promoting organizational cultures that value empirical data and encouraging use of standardized measures and training providers to use pragmatic standardized measures for clinical decision making.

Psychometric Evaluation of the Abbreviated Multidimensional Acculturation Scale (AMAS) in a Treatment-Seeking Sample of First-Generation Immigrant Caregivers

The unique needs of first-generation immigrants and their families have not been prioritized in mental healthcare. Cultural tailoring of child services requires valid, reliable, and efficient assessments of family cultural identity. The Abbreviated Multidimension Acculturation Scale (AMAS) is a self-report of acculturation and enculturation that has been evaluated in community, but not clinical, samples. We offer the first AMAS psychometric evaluation in a treatment-seeking sample of first-generation immigrant caregivers presenting for children's mental healthcare (N = 219). Analyses examined the internal consistency, concurrent validity, and factor structures of the long-form AMAS (42 items, six subscales), AMAS-10 (10 items, four subscales), and AMAS-14 (14 items, six subscales). Findings provide support for the AMAS-10 and AMAS-14, but not the full-length AMAS, in the present sample. Given urgent needs for culturally responsive care for first-generation populations, the AMAS-10 and AMAS-14 can be used in clinical settings to support cultural assessment, case conceptualization, and treatment planning.

The Brief Adolescent Depression Screen: A Brief Screening Tool for Depression and Suicidal Behavior in Inpatient Adolescents

Background

In clinical settings, there is significant need for brief, easily-administered assessment tools for adolescent depression that can be used by mental health clinicians from a variety of training backgrounds. Existing depression screening tools do not assess for duration and consistency of symptoms, two key indicators of pathological depression.

Objective

The Brief Adolescent Depression Screen (BADS) was developed to screen for major and persistent depressive disorders in adolescents in order to meet the assessment needs in an inpatient setting, and the validity of this tool was tested.

Method

The current study used a sample of 396 inpatient adolescents to assess the screening utility of the BADS for detecting whether the adolescent meets criteria for a depressive diagnosis according to a well-validated semi-structured interview, as well as detecting a positive history of suicidal behavior. Further, the screening utility of this measure was compared to the utility of an established depression rating scale.

Results

Analyses first determined the duration of depressive symptoms on the BADS that optimally screened for the presence of Major Depressive Disorder and Persistent Depressive Disorder. Findings indicated that, using these optimal screening cut-offs, the BADS showed a strong screening utility, resulting in a sensitivity and specificity for identifying full depressive diagnoses and a positive history of suicidal behavior with similar or greater accuracy than an established rating scale.

Conclusions

These findings provide initial evidence to suggest that the BADS may be a helpful screening tool for adolescent depressive disorders in inpatient settings.

The Patient-Reported Outcomes Measurement Information System (PROMIS) pediatric and parent-proxy short forms for anxiety: Psychometric properties in the Kids FACE FEARS sample

There is tremendous need for brief and supported, non-commercial youth- and caregiver-report questionnaires of youth anxiety. The pediatric and parent proxy short forms of the Patient-Reported Outcomes Measurement Information System (PROMIS) Anxiety scale (8a v2.0) are free, brief, publicly accessible measures of youth- and caregiver-reported anxiety in children and adolescents. Despite increased use of the PROMIS, no study has evaluated performance of its anxiety scales in a sample of treatment-engaged anxious youth. Analyses were conducted on baseline data from the first 265 families (child M_Age=11.14 years, 70% racial/ethnic minoritized youth) to enroll in the Kids FACE FEARS trial, a multisite comparative effectiveness trial of therapist-led vs. self-administered treatment for elevated youth anxiety. Confirmatory factor analysis (CFA) examined factor structure; omega coefficients and regression models examined internal consistency, convergent validity, and cross-informant reliability. CFA supported adjusted single-factor solutions across youth and caregiver reports, and internal consistency was high. Convergent validity was supported by medium-to-large associations with anxiety-related impairment and severity. Moderate cross-informant reliability between reports was found. Results showcase the first psychometric study of the PROMIS Anxiety scale short forms among treatment-engaged youth with elevated anxiety. Findings highlight the PROMIS Anxiety scale's utility in typical care settings for youth anxiety.

Tasting inhibition: A proof-of-concept study of the food stop-signal game

Self-Control is an important skill in everyday life when attention is automatically drawn toward certain stimuli. For instance, food stimuli automatically capture visual attention and are processed preferentially. Therefore, efficient response inhibition is crucial to refrain from careless overeating. In the present proof-of-concept study we use a novel adaptation of a previously evaluated Stop-Signal Game (SSG) to measure reactive, food-specific, response inhibition in healthy adults. We analyzed a sample of 83 participants (60 female, mean age=24.1, mean BMI=21.71kg/m2) split into three groups. In a gamified stop-signal task, participants navigated an avatar in an urban environment toward high-calorie food, low-calorie food, or non-food stimuli in go-trials and were asked to inhibit the approach reaction in stop-trials. Hunger, eating styles, food craving, and impulsivity were assessed via self-reports to investigate their relationship with (food-specific) response inhibition. Results showed that response inhibition (in terms of stop-signal reaction time, SSRT) did not differ between the high-calorie, low-calorie, and non-food SSG which might be explained by characteristics of the sample. However, impulsivity was positively correlated with SSRT in the low-calorie SSG, whereas food-craving and hunger were positively related to response inhibition in the high-calorie SSG. Future studies could build upon the food SSG to measure and train food-specific response inhibition in the treatment of overeating.

Enhancing Implementation of Measurement-Based Mental Health Care in Primary Care: A Mixed-Methods Study

OBJECTIVE

The complex practice of measurement-based care (MBC) for mental health conditions has proven challenging to implement. This study aimed to evaluate an intensive strategy to implement MBC in U.S. Department of Veterans Affairs (VA) Primary Care Mental Health Integration clinics.

METHODS

Ten paired sites were randomly assigned to receive national MBC resources alone or with an intensive implementation strategy (external facilitation plus quality improvement teams) between May 2018 and June 2020. The intervention occurred over 12-18 months; two site pairs completed participation before the COVID-19 pandemic. Using the RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance) framework, the authors conducted qualitative interviews and used administrative data to evaluate the implementation, adoption, reach, and effectiveness of MBC.

RESULTS

All sites improved during the study, suggesting the effectiveness of the VA's national MBC initiative. Sites with facilitation improved more than comparison sites in implementation, adoption, and reach of MBC. The effectiveness of MBC (i.e., clinician responsiveness to high patient-reported outcome measure [PROM] scores) was demonstrated at all sites both before and after facilitation. After the COVID-19 pandemic began, facilitation sites maintained or improved on their implementation gains, whereas comparison sites uniformly reported decreased emphasis on MBC.

CONCLUSIONS

Implementation facilitation resulted in greater gains in outcomes of interest and helped sites retain focus on MBC implementation. Regardless of study condition, clinicians were responsive to elevated PROM scores, but MBC had a larger impact on care at facilitation sites because of increased uptake. Multiple technological and contextual challenges remain, but MBC holds promise for improving routine mental health care.

The Implementation of Measurement-Based Care in the Context of Telemedicine: Qualitative Study

BACKGROUND

The Measurement Based Care in Mental Health Initiative launched by the Department of Veterans Affairs in 2016 is an example of an evidence-based practice that uses patient-reported outcome measures (PROMs) to improve patient outcomes. The acceptance of measurement-based care (MBC) among Veterans Affairs providers is relatively high. However, there are barriers to MBC for telehealth providers. Health information technologies might afford opportunities to address some of the barriers related to the uptake of MBC.

OBJECTIVE

This paper reports on an implementation effort to integrate MBC into mental health care telehealth practice using eHealth solutions.

METHODS

Qualitative data were generated from 22 semistructured interviews with psychiatrists (n=4), psychologists (n=3), social workers (n=3), nurses (n=6), a pharmacist (n=1), and administrative staff (n=5) who provide telemental health care through a community-based outpatient clinic in the rural Midwestern United States. The interviews were conducted during the pilot phase of an implementation initiative to increase the adoption of MBC by revising clinic workflows to integrate the use of eHealth technologies. Data were analyzed using thematic analysis.

RESULTS

Time burden and workflow issues were the most common barrier to provider adoption of MBC; sharing and reviewing pencil-and-paper measures and results in the same room was no longer possible in novel telehealth workflows necessitated by the COVID-19 pandemic. Providers voiced concerns about how long it would take to collect, adequately score, interpret, share, and document the PROMs during the telehealth visit. Concerns about time might also correspond to a gap in providers' familiarity with these assessments, greater comfort in assessing symptoms through clinical interviews, and being accustomed to using the assessments as screening tools more so than longitudinal outcome measures. Capacities associated with eHealth technologies may address workflow concerns and promote providers' understanding and use of the measures as tracking tools.

CONCLUSIONS

The need to use limited appointment time well was a top priority for telemental health providers. eHealth technologies provided operative supports that protect time in appointments by shifting when and how PROMs are collected. Bolstering providers' familiarity with how to use PROMs in the course of treatment may impact providers' buy-in by encouraging them to reconsider how sharing and acting on PROMs could be time well spent.

Measurement feedback system implementation in public youth mental health treatment services: a mixed methods analysis

BACKGROUND

Prior studies indicate the effectiveness of measurement-based care (MBC), an evidence-based practice, in improving and accelerating positive outcomes for youth receiving behavioral health services. MBC is the routine collection and use of client-reported progress measures to inform shared decision-making and collaborative treatment adjustments and is a relatively feasible and scalable clinical practice, particularly well-suited for under-resourced community mental health settings. However, uptake of MBC remains low, so information on determinants related to MBC practice patterns is needed.

METHODS

Quantitative and qualitative data from N = 80 clinicians who implemented MBC using a measurement feedback system (MFS) were merged to understand and describe determinants of practice over three study phases. Quantitative, latent class analysis identified clinician groups based on participants' ratings of MFS acceptability, appropriateness, and feasibility and describes similarities/differences between classes in clinician-level characteristics (e.g., age; perceptions of implementation climate; reported MFS use; phase I). Qualitative analyses of clinicians' responses to open-ended questions about their MFS use and feedback about the MFS and implementation supports were conducted separately to understand multi-level barriers and facilitators to MFS implementation (phase II). Mixing occurred during interpretation, examining clinician experiences and opinions across groups to understand the needs of different classes of clinicians, describe class differences, and inform selection of implementation strategies in future research (phase III).

RESULTS

We identified two classes of clinicians: "Higher MFS" and "Lower MFS," and found similarities and differences in MFS use across groups. Compared to Lower MFS participants, clinicians in the Higher MFS group reported facilitators at a higher rate. Four determinants of practice were associated with the uptake of MBC and MFS in youth-serving community mental health settings for all clinicians: clarity, appropriateness, and feasibility of the MFS and its measures; clinician knowledge and skills; client preferences and behaviors; and incentives and resources (e.g., time; continuing educational support). Findings also highlighted the need for individual-level implementation strategies to target clinician needs, skills, and perceptions for future MBC and MFS implementation efforts.

CONCLUSION

This study has implications for the adoption of evidence-based practices, such as MBC, in the context of community-based mental health services for youth.

'Instruments are good at eliciting information; scores are very dangerous': The perspectives of clinical professionals regarding neurodevelopmental assessment

LAY ABSTRACT

Autism and attention deficit hyperactivity disorder are common behaviourally diagnosed conditions. One of the key aspects of diagnosis is clinical judgement. Yet despite decades of research, it is only in recent times that researchers have started exploring clinicians' perspectives on diagnosing these conditions. We aimed to add to this body of knowledge by conducting interviews with 17 experienced health care professionals in the United Kingdom to hear their perspectives on diagnosing autism and attention deficit hyperactivity disorder. Clinicians reflected that for some children and young people, diagnosis is reasonably straightforward; however, in other situations, decisions are made on more pragmatic grounds (i.e. will this be helpful). We identified some differences of opinion between professionals and organisation which adds to the complexity of applying a diagnosis. We recommend several areas for future research and point to some practical and philosophical implications of the work.

The Development of an Assessment Rubric for the Core and Contingency Team Interaction Among Rapid Response Teams

INTRODUCTION

Teamwork training is critical in the development of high-functioning rapid response teams (RRT). Rapid response teams involve interactions between a patient's core care team and a hospital contingency team, which can lead to disorganized and unsafe resuscitations, largely due to problems with communication and information dissemination. An extensive literature search found no assessment tools specific to the unique communicative challenges of an RRT, and thus, this study sought to develop an assessment rubric validated for training RRTs.

METHODS

This study elucidates the development, implementation, and testing of an RRT rubric based on Kane's framework for validating testing instruments. Twenty-four inpatient code teams underwent team training using a Team Strategies and Tools to Enhance Performance and Patient Safety (TeamSTEPPS) didactic, an online module on the TeamSTEPPS RRT program, and a subsequent presimulation and postsimulation experience. Two raters were randomized to give a bedside assessment for each team using the proposed RRT rubric. Simulation scores were assessed with Wilcoxon signed-rank tests. Interrater reliability was assessed using intraclass correlation coefficients. These analyses were then used to argue Kane's scoring, generalization, and extrapolation inferences.

RESULTS

All teams significantly improved from the presimulation to postsimulation scenarios across all TeamSTEPPS domains. Content validity was obtained from 5 resuscitation experts with a scale-level content validity index of 0.9, with individual content validity index of 0.8 to 1.0. Intraclass correlation coefficient for "pre" scores were 0.856 (n = 24, P < 0.001), "post" scores of 0.738 (n = 24, P < 0.001), and a total of 0.890 (n = 48, P < 0.001).

CONCLUSIONS

The authors argue for the validity of a new RRT rubric based off Kane's framework, with a specific focus on teamwork training to improve coordination and function of core and contingency teams. A follow-up study with longitudinal data along with external validation of this rubric is needed.

Youth Top Problems and Early Treatment Response to the Unified Protocols for Transdiagnostic Treatment of Emotional Disorders in Children and Adolescents

The Top Problems assessment is an idiographic measure of client concerns that may allow clinicians to identify early treatment response. Few studies have examined early response to evidence-based therapies using Top Problems. We collected weekly Top Problems ratings from 95 youth with emotional disorders who received treatment using the Unified Protocols for Transdiagnostic Treatment of Emotional Disorders in Children and Adolescents (UP-C/UP-A). We assessed Top Problems rating change from session 1 to session 4, the role of pre-treatment variables in early Top Problems rating change, and the role of early Top Problems rating change in post-treatment symptom outcomes. Top Problems ratings decreased significantly from session 1 to session 4. Younger child age and higher parent cognitive flexibility were associated with early Top Problems improvement. Controlling for pre-treatment, early Top Problems rating change did not explain the variance in post-treatment outcomes. Future research should examine Top Problems trajectories over treatment course.

Implementation facilitation strategies to promote routine progress monitoring among community therapists

Despite substantial support for the importance of routine progress monitoring (RPM) as part of evidence-based practice, few providers utilize measurement-based care. This study sought to identify the relative importance of facilitation strategies viewed as most helpful for increasing intention to use RPM among 388 ethnically diverse community therapists serving children and families. Four types of facilitation strategies were examined: language/interpretability, automation, staffing/access, and requirements. Mixed analyses of variance found that therapists' reported intentions to use RPM were more influenced by strategies of automating assessment administration, provision of clerical assistance, and agency requirements than by making linguistically appropriate measures available. However, the importance of strategies differed depending on therapist race/ethnicity and current RPM use. Language/interpretability of RPM assessments was less emphasized for non-Hispanic White therapists and therapists who have not yet or only minimally adopted RPM compared with ethnic minority therapists and therapists who regularly use RPM, respectively. Furthermore, therapists who were not current RPM users emphasized automation more than staffing/access. Results may inform prioritization of implementation facilitation strategies for agencies to encourage RPM. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Drug and alcohol practitioners' attitudes toward the use of standardized assessment

Despite their importance to evidence-based assessment, standardized assessments remain underutilized by mental health practitioners in practice. The underutilization has been attributed to a lack of appreciation of the importance of standardized assessments, lack of knowledge of standardized assessments, and practical barriers to implementation. This study sought to gather the first descriptive data on alcohol and other drug (AOD) practitioners' attitudes toward, and knowledge and self-reported use of, standardized assessments. Practical barriers to implementation in initial assessment and progress monitoring were also assessed. Ninety-nine Australian AOD practitioners recruited via newsletters of national representative bodies and practitioner networks completed an online survey. While practitioners' attitudes towards using standardized assessments for initial assessment and progress monitoring were generally positive and consistent with other populations of health practitioners, assessments remained underutilized in practice. Most AOD practitioners did not consider standardized assessments to be feasible to implement. The current findings highlight the importance of practical barriers, particularly organization-level barriers, in the underutilization of standardized assessments in AOD practice. Findings support an extension of dialogue surrounding evidence-based practice beyond treatment selection to include assessment practices at a more general level. The present study offers a starting point from which efforts to improve practitioner compliance with evidence-based best practices can be conceived, designed, and implemented.

The diagnostic process from primary care to child and adolescent mental healthcare services: the incremental value of information conveyed through referral letters, screening questionnaires and structured multi-informant assessment

BACKGROUND

A variety of information sources are used in the best-evidence diagnostic procedure in child and adolescent mental healthcare, including evaluation by referrers and structured assessment questionnaires for parents. However, the incremental value of these information sources is still poorly examined.

AIMS

To quantify the added and unique predictive value of referral letters, screening, multi-informant assessment and clinicians' remote evaluations in predicting mental health disorders.

METHOD

Routine medical record data on 1259 referred children and adolescents were retrospectively extracted. Their referral letters, responses to the Strengths and Difficulties Questionnaire (SDQ), results on closed-ended questions from the Development and Well-Being Assessment (DAWBA) and its clinician-rated version were linked to classifications made after face-to-face intake in psychiatry. Following multiple imputations of missing data, logistic regression analyses were performed with the above four nodes of assessment as predictors and the five childhood disorders common in mental healthcare (anxiety, depression, autism spectrum disorders, attention-deficit hyperactivity disorder, behavioural disorders) as outcomes. Likelihood ratio tests and diagnostic odds ratios were computed.

RESULTS

Each assessment tool significantly predicted the classified outcome. Successive addition of the assessment instruments improved the prediction models, with the exception of behavioural disorder prediction by the clinician-rated DAWBA. With the exception of the SDQ for depressive and behavioural disorders, all instruments showed unique predictive value.

CONCLUSIONS

Structured acquisition and integrated use of diverse sources of information supports evidence-based diagnosis in clinical practice. The clinical value of structured assessment at the primary-secondary care interface should now be quantified in prospective studies.

Praxisorientierte Forschung in der Psychotherapie

Zusammenfassung. In den letzten Jahrzehnten hat sich durch randomisiert-kontrollierte Studien (RCTs) eine breite Evidenzbasis von Psychotherapie mit mittleren bis großen Effekten für verschiedene psychische Störungen gebildet. Neben der Bestimmung dieser Wirksamkeit („Efficacy“) ebneten Studien zur Wirksamkeit unter alltäglichen Routinebedingungen („Effectiveness“) historisch den Weg zur Entwicklung eines praxisorientierten Forschungsparadigmas. Im Beitrag wird argumentiert, dass im Rahmen dieses Paradigmas praxisbasierte Studien eine wertvolle Ergänzung zu RCTs darstellen, da sie existierende Probleme in der Psychotherapieforschung adressieren können. In der gegenwärtigen praxisorientierten Forschung liefern dabei neue Ansätze aus der personalisierten Medizin und Methoden aus der ‚Computational Psychiatry‘ wichtige Anhaltspunkte zur Optimierung von Effekten in der Psychotherapie. Im Kontext der Personalisierung werden bspw. klinische multivariable Prädiktionsmodelle entwickelt, welche durch Rückmeldeschleifen an Praktiker_innen kurzfristig ein evidenzbasiertes Outcome-Monitoring ermöglicht und langfristig das Praxis-Forschungsnetzwerk in Deutschland stärkt. Am Ende des Beitrags werden zukünftige Richtungen für die praxisorientierte Forschung im Sinne des ‘Precision Mental Health Care’ -Paradigmas abgeleitet und diskutiert.

Diagnostic Utility of the Gilliam Autism Rating Scales-3rd Edition Parent Report in Clinically Referred Children

There is limited research regarding the Gilliam Autism Rating Scales-3rd Edition (GARS-3) despite its extensive use. A comprehensive diagnostic evaluation, including the Autism Diagnostic Observation Schedule-2nd Edition (ADOS-2) was provided to 186 clinically referred children suspected of autism ([Formula: see text] age = 8.98; Autism [AUT] n = 87; Not Autism [NOT] n = 99). Mean difference analyses, Logistic Regressions, and ROC analyses were non-significant for both Autism Index scores. The author-suggested cutoff score of 70 correctly classified approximately 47% of participants, with false positive rates = 82.83-87.88%. ADOS-2 correlations were significantly lower vis-à-vis the standardization sample. The Social Interaction subscale demonstrated weak, marginal results, and sensitivity/specificity could not be optimized. In its current form, the GARS-3 does not demonstrate adequate criterion validity for use in assessment of complex community samples.

Psychometric evaluation of a caregiver-report adaptation of the Overall Anxiety Severity and Impairment Scale (OASIS) for use with youth populations

Background Despite progress in youth anxiety assessment, there is need for a measure that is simultaneously (a) free, (b) brief, (c) focused broadly on anxiety and avoidance severity, frequency, and interference, and (d) concerned with the past week. The adult overall anxiety severity and impairment scale (OASIS) was adapted to yield a caregiver-report of past week youth anxiety and interference (OASIS-Y). Methods In a sample of diverse youth seeking anxiety services (N = 132; 67% racial/ethnic minority) and their caregivers, analyses examined the OASIS-Y factor structure, internal consistency, and convergent and divergent validity. Hierarchical linear modeling in a participant subset examined OASIS-Y sensitivity to treatment-related change. Results OASIS-Y internal consistency was high and confirmatory factor analysis supported a single-factor structure similar to that found in adults. OASIS-Y convergent validity was supported by a medium-sized association with an established, commercially available measure of youth anxiety, and divergent validity was supported by the absence of unique associations with measures of youth attention and externalizing problems. In a sample subset, session-by-session OASIS-Y scores significantly declined across treatment, and declined at a steeper rate among treatment "responders" versus "non-responders," providing evidence of OASIS-Y sensitivity to treatment-related change. Limitations This study focused on a clinical sample and cannot speak to OASIS-Y performance in community settings. Shared method-variance may have also influenced findings. Conclusions This study offers the first psychometric evaluation of the OASIS-Y, and underscores the promising clinical utility of the measure for assessing past week youth anxiety and impairment and for supporting routine outcome monitoring.

Examining School Psychologists’ Attitudes Toward Standardized Assessment Tools

Despite a call for evidence-based practice in school psychology, limited research on the topic of evidence-based assessment exists. To begin to address this gap in the research, a modified version of Jenson–Doss and Hawley’s (2010) Attitudes Toward Standardized Assessment (ASA) scale was administered to 371 U.S. school psychologists. Examination of the modified ASA’s factor structure suggested that a bifactor model with a single overall domain and three sub-domains was the most parsimonious. Indices of dimensionality indicated that the overall score may be the best indicator of school psychologist’s perceptions of standardized assessment. Additionally, school psychologists’ reported favorable attitudes of standardized assessment compared to clinical judgment alone. Limitations and implications for future research are discussed.

Clinician perceptions of nomothetic and individualized patient-reported outcome measures in measurement-based care

Objective: Measurement-based care (MBC), which encompasses routine outcome monitoring (ROM) and measurement feedback systems (MFSs), is an evidence-based practice (EBP) supporting treatment personalization and clinical responsiveness. Despite MBC's effectiveness, clinicians report reservations regarding its utility, which may be a function of overreliance on nomothetic (i.e., standardized) measures. Although research suggests that individualized (i.e., idiographic) patient-reported outcome measures (I-PROMs) may have the potential to overcome these obstacles, little is known regarding clinicians' perceptions of different measurement approaches to MBC. Methods: This study examined clinicians' perceptions of the clinical utility, relevance to treatment planning, and practicality of nomothetic, individualized, and combined clinical feedback provided by a simulated MFS. Three hundred and twenty-nine clinicians were randomized to one of three conditions that presented a clinical vignette comprising: (a) nomothetic, (b) individualized, or (c) combined clinical feedback. Results: Participants' perceptions of the clinical feedback were not affected by the measurement approach. However, cognitive behavioral participants reported more positive perceptions of all aspects associated with the clinical feedback. Conclusion: These results were consistent with previous findings, suggesting that clinicians' theoretical orientation may have a significant impact on their perceptions of MBC, and should be considered when designing and implementing these systems.

Implementation of the Wolverine Mental Health Program, Part 1: Adoption Phase

Residential treatment facilities (RTFs) are a first-line treatment option for juvenile justice-involved youth. However, RTFs rarely offer evidence-based interventions for youth with internalizing or externalizing mental health problems. Wolverine Human Services (WHS) is one of the first RTFs in the nation to implement cognitive-behavioral therapy (CBT) to enhance mental health care for their youth. This study outlines the preimplementation phase of a 5-year collaborative CBT implementation effort among WHS, the Beck Institute, and an implementation science research team. The preimplementation phase included a needs assessment across two sites of WHS to identify and prioritize barriers to CBT implementation. Of the 76 unique barriers, 23 were prioritized as important and feasible to address. Implementation teams, consisting of clinician and staff champions and opinion leaders, worked across 8 months to deploy 10 strategies from a collaboratively designed blueprint. Upon reevaluation of the needs assessment domains, all prioritized barriers to CBT implementation were removed and WHS's readiness for CBT implementation was enhanced. This study serves as a model of a preimplementation process that can be employed to enhance the potential for successful evidence-based practice implementation in youth RTFs.

Tailored isn't always better: Impact of standardized versus tailored training on intention to use measurement-based care

Background: Brief educational trainings are often used for disseminating and implementing evidence-based practices (EBPs). However, many accessible trainings are ubiquitously standardized. Tailored training focused on modifying individual or contextual factors that may hinder EBP implementation is recommended, but there is a dearth of research comparing standardized versus tailored training. This study sought to: (a) assess the impact of MBC training on clinician intention to use measurement-based care (MBC); (b) compare the effect of standardized versus tailored training on clinician intention to MBC; and (c) identify clinician-level predictors of intention. Methods: Clinicians (n = 152) treating adult clients with depression at 12 community mental health clinics were randomized to either tailored or standardized MBC training. Clinic-specific barriers and facilitators were used to inform training content and structure tailoring. Linear mixed modeling tested the association between training condition and post-training intention to use MBC, as well as hypothesized individual-level predictors of post-training intention (e.g., age, gender). Results: Clinician intention pre- and post-training increased across training conditions (B = 0.38, t = -5.95, df = 36.99, p < .01, Cohen's d = 0.58). Results of linear mixed modeling procedures suggest no significant difference in clinician intention between conditions post-training (B = -0.03, SE = .19, p > .05, Cohen's d = .15). Only baseline intention emerged as a predictor of post-training intention (B = 0.39, SE = .05, p < .05). Conclusions: These findings suggest the additional effort to tailor training may not yield incremental benefit over standardized training, at least in the short term. As a result, implementation efforts may be able to reserve time and finances for other elements of implementation beyond the training component.

Plain Language Summary

Educational training is a common approach for enhancing knowledge about research-supported mental health treatments. However, these trainings are often not tailored to meet the needs of the trainees, and there is insufficient evidence about whether tailoring might improve the impact of training compared to a one-size-fits-all, standard version. This study compared the impact of a tailored versus standard training on mental health clinician's intentions to use measurement-based (MBC) care for monitoring treatment progress for clients with depression. Study results indicated that intention to use MBC improved for clinicians receiving both the tailored and standard training after training completion. There were no differences in intention to use MBC care when the two types of training were compared. These study findings suggest that tailoring, which may require substantial time and effort, may not be a necessary step to improve the short-term impact of educational trainings.

Improving the Quality of Children's Mental Health Care with Progress Measures: A Mixed-Methods Study of PCIT Therapist Attitudes

Progress measures are an evidence-based technique for improving the quality of mental health care, however, clinicians rarely incorporate them into treatment. Research into how measure type impacts clinician preference has been recommended to help improve measure implementation. Parent–Child Interaction Therapy (PCIT) is an assessment-driven treatment that serves as an ideal intervention through which to investigate measure preferences given its routine use of two types of assessments, a behavioral observation (the Dyadic Parent–Child Interaction Coding System) and a parent-report measure (the Eyberg Child Behavior Inventory). This study investigated PCIT therapist attitudes towards progress measures used within PCIT and children’s mental health treatment generally. A mixed-method (QUAN + QUAL) study design examined PCIT therapist attitudes towards two types of progress measures and measures used in two contexts (PCIT and general practice). Multi-level modeling of a survey distributed to 324 PCIT therapists identified predictors of therapist attitudes towards measures, while qualitative interviews with 23 therapists expanded and clarified the rationale for differing perceptions. PCIT therapists reported more positive attitudes towards a behavioral observation measure, the DPICS, than a parent-report measure, the ECBI, and towards measures used in PCIT than in general practice. Clinician race/ethnicity was significantly related to measure-specific attitudes. Qualitative interviews highlighted how perceptions of measure reliability, type of data offered, ease of use, utility in guiding sessions and motivating clients, and embeddedness in treatment protocol impact therapist preferences. Efforts to implement progress monitoring should consider preferences for particular types of measures, as well as how therapists are trained to embed measures in treatment.

Impact of childhood maltreatment on outcomes of antidepressant medication in chronic and/or recurrent depression

BACKGROUND

While childhood maltreatment (CMT) is associated with higher rates of chronicity and recurrence in depression, whether CMT results in poorer outcomes with antidepressant medication remains unclear.

METHODS

We performed secondary analyses with data from the large, representative, multisite trial Combining Medications to Enhance Depression Outcomes (CO-MED). CO-MED was a randomized, single-blinded, placebo-controlled study with 665 individuals (663 assessed for CMT) with chronic and/or recurrent Major Depressive Disorder (MDD). CMT was determined by a brief self-reported questionnaire assessing the four types of CMT defined by the Centers for Disease Control and Prevention: sexual abuse, emotional abuse, physical abuse, and neglect. Repeated measures and logistic regression analyses were used.

RESULTS

Individuals with CMT did not have a differential improvement of depressive symptoms when compared to those without CMT (adjusted p=.203 for continuous analysis; adjusted p=.320 for remission rates). Neither type of antidepressant medication (adjusted p=.302) nor the age at which CMT occurred (adjusted p=.509) affected depressive symptom outcomes. There was no difference in functional improvement between individuals with and without CMT (adjusted p=.228). A history of CMT was associated with greater antidepressant side effects (p=.009).

LIMITATIONS

This study investigated treatment-seeking individuals with chronic and/or recurrent MDD. Intensity and duration of CMT were not assessed.

CONCLUSION

In a sample of treatment-seeking outpatients with chronic and/or recurrent MDD, a history of CMT was not associated with differential symptomatic or functional response to pharmacological treatment. However, those with CMT reported greater antidepressant side effect burden.

Applying behaviour change theory to understand the barriers to implementing routine outcome monitoring

BACKGROUND

Routine outcome monitoring (ROM) is a valuable tool for monitoring client progress and pre-empting deterioration, however, there is considerable variation in how data are collected and recorded and uptake in clinical practice remains low. The aim of this study was to develop a self-report measure of practitioner attitudes to ROM in order to better understand the barriers to successful implementation in Child and Adolescent Mental Health Services (CAMHS).

METHODS

An anonymous survey was completed by 184 CAMHS practitioners in the United Kingdom. The survey was designed using the Capability, Opportunity, and Motivation Model of Behaviour (COM-B). Practitioners who reported using ROM frequently in their clinical work (53%) were compared to those who used ROM infrequently (47%) across dimensions of the COM-B survey subscales.

RESULTS

Confirmatory factor analysis confirmed the proposed four-factor structure, showing acceptable model fit, with high factor loadings and good reliability for all subscales. Frequent users of ROM exhibited significantly higher psychological capability, physical opportunity, social opportunity, and motivation, compared to infrequent users F (4, 140) = 14.76, p < .0001; Pillai's Trace = .297, partial η²  = .30. Results highlight several barriers to ROM, including the belief that there is not a strong evidence base for ROM, not receiving external training, and not discussing feedback and outcome data in supervision.

IMPLICATIONS

In the hope of improving the successful implementation of ROM, this research provides an evidence-based tool for assessing practitioners' attitudes to ROM, which map on to intervention functions and represent targets for future implementation efforts.

PRACTITIONER POINTS

The value of routine outcome monitoring (ROM) as a means to measure client progress and to elevate the efficiency and quality of mental health care is well-documented in the research literature, however, uptake in practice remains relatively low. This study applied behaviour change theory to develop a psychometrically sound self-report measure of practitioners' perspectives and practices to understand the barriers to implementation in child and adolescent mental health services in the United Kingdom. The complex and multifaceted nature of the barriers to implementation requires multilevel behaviour change strategies at the client, clinician, and organisational level. Recommendations for practice include the need for integrated, multilevel strategies aimed at improving practitioners' capabilities and motivations, strong organisational leadership and a culture of data gathering and sharing, and implementation interventions, which are tailored to target local barriers.

What do clinicians want? Understanding frontline addiction treatment clinicians' preferences and priorities to improve the design of measurement-based care technology

BACKGROUND

Measurement-based care (MBC) is the practice of routinely administering standardized measures to support clinical decision-making and monitor treatment progress. Despite evidence of its effectiveness, MBC is rarely adopted in routine substance use disorder (SUD) treatment settings and little is known about the factors that may improve its adoptability in these settings. The current study gathered qualitative data from SUD treatment clinicians about their perceptions of MBC, the clinical outcomes they would most like to monitor in MBC, and suggestions for the design and implementation of MBC systems in their settings.

METHODS

Fifteen clinicians from one publicly-funded and two privately-funded outpatient SUD treatment clinics participated in one-on-one research interviews. Interviews focused on clinicians' perceived benefits, drawbacks, and ideas related to implementing MBC technology into their clinical workflows. Interviews were audio recorded, transcribed, and coded to allow for thematic analysis using a mixed deductive and inductive approach. Clinicians also completed a card sorting task to rate the perceived helpfulness of routinely measuring and monitoring different treatment outcomes.

RESULTS

Clinicians reported several potential benefits of MBC, including improved patient-provider communication, client empowerment, and improved communication between clinicians. Clinicians also expressed potential drawbacks, including concerns about subjectivity in patient self-reports, limits to personalization, increased time burdens, and needing to learn to use new technologies. Clinicians generated several ideas and preferences aimed at minimizing burden of MBC, illustrating clinical changes over time, improving ease of use, and improving personalization. Numerous patient outcomes were identified as "very helpful" to track, including coping skills, social support, and motivation for change.

CONCLUSIONS

MBC may be a beneficial tool for improving clinical care in SUD treatment settings. MBC tools may be particularly adoptable if they are compatible with existing workflows, help illustrate gradual and nonlinear progress in SUD treatment, measure outcomes perceived as clinically useful, accommodate multiple use cases and stakeholder groups, and are framed as an additional source of information meant to augment, rather than replace, existing practices and information sources.

Exploration, Preparation, and Implementation of Standardized Assessment in a Multi-agency School Behavioral Health Network

School mental health treatment services offer broad public health impact and could benefit from more widespread implementation and sustainment of standardized assessments (SA). This demonstration study describes one approach to increase SA use in a large school behavioral health network using the Exploration, Preparation, Implementation and Sustainment (EPIS) framework. Mental health administrator interviews with four participating agencies and a multidisciplinary planning team informed SA measure selection and implementation supports. The SA initiative was implemented during one school year, including system-wide training and ongoing implementation supports for 95 clinicians. Linear mixed effect models revealed improvements in clinician attitudes about the SA for clinical utility and treatment planning immediately following the half-day training (N = 95, p < .001). Clinicians self-reported a significant increase in use of SA for new clients during intakes (p < .001) over time and 71.4% of expected SA data were submitted. Qualitative feedback, including recommendations to offer more SA choices and beginning new SA data collection earlier in the school year, was integrated to inform quality improvements and future sustainment efforts.

The Development of a Brief Version of the Insomnia Severity Index (ISI-3) in Older Adult Veterans with Posttraumatic Stress Disorder

Objective/Background: The Insomnia Severity Index (ISI) is an insomnia self-report measure used to identify individuals at risk for insomnia disorder. Although the full ISI is only seven questions, a briefer version would allow more efficient and pragmatic administration in routine practice settings. Reliable and valid brief measures can support measurement-based care. The present study was a proof-of-concept study that developed a brief version of the ISI, the ISI-3, in a sample of older adult veterans with posttraumatic stress disorder (PTSD), a group which is at increased risk for insomnia.Patients/Methods: Participants included 86 older veterans with combat- or military-related PTSD. Veterans completed a clinician-administered PTSD diagnostic interview, self-report measures of insomnia and other psychosocial questionnaires, and two neuropsychological assessments. The factor structure of the ISI was examined to reduce the measure into a brief version. The reliability and validity of the ISI-3 were examined.Results: Principal axis factoring yielded a one-factor solution, which reproduced 59% of the item variance. Item reduction procedures resulted in three items, which best represented this factor ("Insomnia Impact;" ISI-3). For the ISI-3, internal consistency was good (α =.89). Convergent validity was demonstrated via moderate to high positive correlations between the ISI-3 and other measures of sleep disturbance. Divergent validity was demonstrated via non-significant correlations between the ISI-3 and unrelated measures and moderate correlations with self-reported depression.Conclusions: The ISI-3 is a psychometrically valid brief version of the ISI. Clinicians can administer the ISI-3 to screen for insomnia and monitor changes in insomnia during treatment.

Harmonizing evidence-based practice, implementation context, and implementation strategies with user-centered design: a case example in young adult cancer care

BACKGROUND

Attempting to implement evidence-based practices in contexts for which they are not well suited may compromise their fidelity and effectiveness or burden users (e.g., patients, providers, healthcare organizations) with elaborate strategies intended to force implementation. To improve the fit between evidence-based practices and contexts, implementation science experts have called for methods for adapting evidence-based practices and contexts and tailoring implementation strategies; yet, methods for considering the dynamic interplay among evidence-based practices, contexts, and implementation strategies remain lacking. We argue that harmonizing the three can be facilitated by user-centered design, an iterative and highly stakeholder-engaged set of principles and methods.

METHODS

This paper presents a case example in which we used a three-phase user-centered design process to design and plan to implement a care coordination intervention for young adults with cancer. Specifically, we used usability testing to redesign and augment an existing patient-reported outcome measure that served as the basis for our intervention to optimize its usability and usefulness, ethnographic contextual inquiry to prepare the context (i.e., a comprehensive cancer center) to promote receptivity to implementation, and iterative prototyping workshops with a multidisciplinary design team to design the care coordination intervention and anticipate implementation strategies needed to enhance contextual fit.

RESULTS

Our user-centered design process resulted in the Young Adult Needs Assessment and Service Bridge (NA-SB), including a patient-reported outcome measure and a collection of referral pathways that are triggered by the needs young adults report, as well as implementation guidance. By ensuring NA-SB directly responded to features of users and context, we designed NA-SB for implementation, potentially minimizing the strategies needed to address misalignment that may have otherwise existed. Furthermore, we designed NA-SB for scale-up; by engaging users from other cancer programs across the country to identify points of contextual variation which would require flexibility in delivery, we created a tool intended to accommodate diverse contexts.

CONCLUSIONS

User-centered design can help maximize usability and usefulness when designing evidence-based practices, preparing contexts, and informing implementation strategies-in effect, harmonizing evidence-based practices, contexts, and implementation strategies to promote implementation and effectiveness.

Evidence-Based Assessment in Routine Mental Health Services for Youths

OBJECTIVE

Evidence-based assessment (EBA) plays a critical role in the delivery of effective treatments. However, little is known about the assessment practices of mental health clinicians who treat youths and the factors that support EBA. The authors examined when, how, and under what conditions clinicians conduct EBA.

METHODS

In two multidisciplinary surveys (combined N=2,575), clinicians reported how frequently they conducted pretreatment, ongoing, and posttreatment assessments and how frequently they used standardized measures in usual care of youths.

RESULTS

Although clinicians reported frequent pretreatment, ongoing, and posttreatment assessments, use of standardized measures was rare. Clinician and practice setting characteristics predicted standardized measure use, and a lack of practical assessment tools appears to be a barrier to use of standardized measures in EBA of youths.

CONCLUSIONS

Many clinicians conduct assessments during treatment, but more practical measures and clinician training may improve the integration of standardized measures into routine practice.

Childhood maltreatment and impact on clinical features of major depression in adults

OBJECTIVES

This study examined: 1) the prevalence of childhood maltreatment (CMT) in individuals with chronic and/or recurrent depression, 2) the association between CMT and depressive symptoms, 3) the link between CMT and worse clinical presentation of depression, 4) the effects of accumulation of different types of CMT, and 5) the relationship between the age at CMT and depression.

METHODS

We analyzed the baseline data of 663 individuals from the CO-MED study. CMT was determined by a brief self-reported questionnaire assessing sexual abuse, emotional abuse, physical abuse, and neglect. Correlational analyses were conducted.

RESULTS

Half of the sample (n = 331) reported CMT. Those with CMT had higher rates of panic/phobic, cognitive and anhedonic symptoms than those without CMT. All individual types of maltreatment were associated with a poorer clinical presentation including: 1) earlier MDD onset; 2) more severe MDD, 3) more suiccidality, 4) worse quality of life, and functioning, and 5) more psychiatric comorbidities. Clinical presentation was worse in participants who reported multiple types of CMT.

CONCLUSIONS

In chronic and/or recurrent depression, CMT is common, usually of multiple types and is associated with a worse clinical presentation in MDD. The combination of multiple types of CMT is associated with more impairment.

An Initial Study of Practicing Psychologists' Views of the Utility of Ecological Momentary Assessment for Difficult Psychotherapy Cases

Ecological momentary assessment (EMA) is a burgeoning area of research, and several clinical applications of the resulting data have been identified by researchers, suggesting potential benefit to psychotherapy practice. However, practitioners often do not use traditional empirically-supported tools for diagnosis and outcome monitoring (e.g., validated interview measures and questionnaires). Thus, it is not clear how readily practitioners will take up newer technology-enhanced assessment methods, despite current enthusiasm among researchers. The current study aimed to explore the perceived usefulness of EMA-based tools for clinical assessment and outcome monitoring of difficult psychotherapy cases, as well as to identify correlates of attitudes about the usefulness of these tools. Clinical psychologists in active therapy practice with adults (n = 375) completed an internet survey including the Attitudes toward Standardized Assessment scale and the Attitudes toward Standardized Assessment Scales-Monitoring and Feedback. Respondents characterized their current diagnostic and outcome monitoring practices and rated how helpful they would find several assessment and outcome monitoring resources for a difficult case, including both traditional instruments and EMA-based methods. EMA-based tools had lower perceived usefulness than existing instruments. Attitudes toward standardized assessment and outcome monitoring predicted the perceived utility of these methods, as did several professional variables. Practicing psychologists may not adopt EMA for clinical assessment more readily than traditional assessment tools. Recommendations for facilitating the uptake of new technologies by psychotherapists are offered.

Factors affecting the implementation of an outcome measurement feedback system in youth mental health settings

Objective: Measurement feedback systems provide clinicians with regular snapshots of a client's mental health status, which can be used in treatment planning and client feedback. There are numerous barriers to clinicians using outcome measures routinely. This study aimed to investigate factors affecting the use of a measurement feedback system across youth mental health settings. Methods: The participants were 210 clinicians from headspace youth mental health services across Australia. They were surveyed on predictors and use of MyLifeTracker, a routine outcome measure. This was explored through three processes: looking at MyLifeTracker before session, using MyLifeTracker in treatment planning, and providing feedback of MyLifeTracker scores to clients. Results: Clinicians were more likely to look at MyLifeTracker before session, less likely to use it in treatment planning, and least likely to provide MyLifeTracker scores to clients. Each measurement feedback system process had a distinct group of predictors. Perceptions of MyLifeTracker's practicality was the only significant predictor of all three processes. Conclusion: Practically, organisations and supervisors can increase the use of measurement feedback systems through targeted supports.

The Use of Evidence-Based Assessment for Anxiety Disorders in an Australian Sample

Anxiety disorders are common and cause considerable functional impairment. Fortunately, evidence-based treatments are available, however, treatment effectiveness is often reliant on the provision of an accurate diagnosis. Accurate diagnosis requires a multi-method evidence-based assessment (EBA). Assessment techniques available to clinicians include a clinical interview, semi-structured diagnostic interview, self-report/clinician-administered rating scales and direct observation. Research demonstrates that only a small number of therapists utilize EBA, and to date this has not been investigated in an Australian sample. One hundred and two registered Australian psychologists (M_age = 40.98; SD = 12.67; 83.6% female) participated in an online study investigating assessment practices. Participants were asked to indicate EBA frequency of use and the obstacles they face to using EBA. The majority of participants (69% working with adult patients and 51% working with pediatric patients) reported partial use of EBA. Few psychologists (21% working with adult patients and 11% working with child patients) indicated complete use of EBA. Thirty-six percent of participants indicated negative beliefs about the usefulness or helpfulness of EBA. Multiple obstacles to the use of EBA were reported including concerns with the time required to complete assessment (27%), and a lack of access to assessment tools (10%). Implications for training and clinical practice are discussed.

Self-reflection and screening mental health on Canadian campuses: validation of the mental health continuum model

Background

This study describes the psychometric testing of the Mental Health Continuum (MHC) model the Canadian Department of National Defense developed initially, among undergraduates of three Canadian universities. The MHC is a tool that consists of 6 items to guide students the way to attend to, or monitor, signs and behavior indicators of their mental health status and suggest appropriate actions to improve their mental health.

Methods

Online survey data were collected from 4206 undergraduate students in three universities in two Canadian provinces during the spring of 2015 and winter of 2016. Participants completed an online survey questionnaire that consisted of the MHC questionnaire, the Kessler Psychological Distress Scale (K-10), and demographic information, including age, gender, and year of study.

Results

Factor analysis using the principal components method followed by a two-step internal replication analysis showed that the MHC tool was two-dimensional and that all six domains assessed were crucial. The construct (convergent) validity of the MHC tool was tested against the K-10, and the correlation analysis results were strong overall, as well as within subgroups defined by gender, year of study, and university.

Conclusions

The MHC is a useful tool that helps college students reflect on and enhance their mental health.

Making the Most of It: Application of Planned Missingness Design to Increase the Efficiency of Diagnostic Assessment

Multimethod assessment is recommended as "best practice" in clinical assessment and is often implemented through the combined use of symptom rating scales and structured interviews. While this approach increases confidence in the validity of assessment, it also increases burden, expense, and leads to the accumulation of redundant information. To address this problem, we evaluate the use a planned missingness design within the framework of adult Attention Deficit/Hyperactivity Disorder (ADHD) assessment. In a sample of 169 young adults, we fit a two-method measurement (TMM) model using ADHD symptoms obtained from rating scales and a structured diagnostic interview. Based on an estimated 8:1 differential between the cost of conducting an in-person diagnostic interview vs. completing questionnaires online, we conducted a series of Monte Carlo simulations to determine the utility of combining TMM with a planned missingness design. We find that even when costs are kept constant, statistical power of the TMM/planned missingness design was equal to the power that would have been obtained had nearly twice the number of participants with complete data been recruited. Conversely, costs could be decreased by 20-25%, while maintaining statistical power equivalent to a design with complete data. Our results suggest the TMM design is a promising technique for reducing the cost and burden of diagnostic assessment within research settings.

Leveraging Health Information Technology to Meet The Joint Commission's Standard for Measurement-Based Care: A Case Study

INTRODUCTION

Three decades of research have shown that routinely collecting patient-reported outcomes throughout treatment to inform clinical decision making or measurement-based care (MBC) can improve clinical outcomes, yet widespread adoption continues to be elusive.

APPROACH

This article describes how a community behavioral health center addressed Element of Performance (EP) 1 of The Joint Commission's revised MBC standard using health information technology (HIT)-facilitated MBC and a comprehensive implementation plan grounded in the Consolidated Framework for Implementation Research.

RESULTS

Across the initial 15-month implementation period, 96.8% of patients who had an intake evaluation also completed baseline measurements via an HIT known as a measurement feedback system (MFS), and 91.5% (78.6%-100%) completed at least one repeated measure.

CONCLUSION

MFS reduces many of the logistical barriers of MBC, but implementation of MFS-facilitated MBC requires a comprehensive implementation plan that includes strategies to address barriers across all relevant domains for successful uptake.

Dr. Liu et al. Reply

While there is growing acceptance within the field that measurement-based care (MBC) is a valuable and effective care quality improvement strategy, broad and sustained implementation continues to be elusive for most organizations.¹ This is partly attributable to the lack of proven implementation strategies for MBC. Although implementation science has made significant progress in recent years,¹ more work is needed to identify the most effective and efficient strategies for MBC implementation across a range of service delivery contexts.