Evidence-based assessment

Introduction to the Special Issue: Discrepancies in Adolescent-Parent Perceptions of the Family and Adolescent Adjustment

Researchers commonly rely on adolescents' and parents' reports to assess family functioning (e.g., conflict, parental monitoring, parenting practices, relationship quality). Recent work indicates that these reports may vary as to whether they converge or diverge in estimates of family functioning. Further, patterns of converging or diverging reports may yield important information about adolescent adjustment and family functioning. This work is part of a larger literature seeking to understand and interpret multi-informant assessments of psychological phenomena, namely mental health. In fact, recent innovations in conceptualizing, measuring, and analyzing multi-informant mental health assessments might meaningfully inform efforts to understand multi-informant assessments of family functioning. Therefore, in this Special Issue we address three aims. First, we provide a guiding framework for using and interpreting multi-informant assessments of family functioning, informed by recent theoretical work focused on using and interpreting multi-informant mental health assessments. Second, we report research on adolescents' and parents' reports of family functioning that leverages the latest methods for measuring and analyzing patterns of convergence and divergence between informants' reports. Third, we report research on measurement invariance and its role in interpreting adolescents' and parents' reports of family functioning. Research and theory reported in this Special Issue have important implications for improving our understanding of the links between multi-informant assessments of family functioning and adolescent adjustment.

The Moderating Role of Attention Biases in understanding the link between Behavioral Inhibition and Anxiety

The current study aimed to extend the results of White et al. (2015) by examining the moderating role of attention biases at age 5 on the relations between Behavioral Inhibition (BI) during toddlerhood and anxiety symptoms at age 10. Children's BI at 2 and 3 years of age was measured using laboratory assessments, and attention bias towards threat was assessed using a dot-probe task at age 5. Latent Class Analysis (LCA) was used to identify the probability for children's membership in an anxiety class, which reflected primary anxiety at age 10 that was not comorbid with symptoms of inattention. Maternal and self-report measures of children's mental health, collected via questionnaires and semi-structured diagnostic interviews, were used as indicators for the LCA. The results revealed that threat-related attention biases moderated the relation between BI and anxiety, such that BI positively predicted the probability of being in the anxiety class only when children had an attention bias towards threat. BI was unrelated to anxiety when children had no attention bias or an attention bias away from threat. These results indicated that attention biases during preschool may differentiate between inhibited children who are at heightened risk for anxiety later in childhood from those who are not. The results are discussed in a framework detailing the role of attention biases in increasing the sensitivity for anxiety-related problems in children who display high levels of BI during early childhood.

Transdiagnostic assessment of anxiety symptoms using the Anxiety Disorder Diagnostic Questionnaire - weekly version

BACKGROUND

Norton and Robinson [2010. Development and evaluation of the anxiety disorder diagnostic questionnaire. Cognitive Behaviour Therapy, 39(2), 137-149. doi: 10.1080/16506070903140430 ] developed the Anxiety Disorder Diagnostic Questionnaire (ADDQ) as a transdiagnostic assessment of fear and anxiety to address problems in using diagnosis-specific measures as well as limitations with the extant transdiagnostic measures of anxiety. The present study validated a weekly version of the ADDQ, the Anxiety Disorder Diagnostic Questionnaire - Weekly (ADDQ-W) allowing session-by-session transdiagnostic assessment of anxiety.

METHOD

Data were a secondary analysis of 49 treatment-seeking outpatient adults from a previous clinical trial. The ADDQ-W was administered weekly over the course of 12-group therapy sessions.

RESULTS

The ADDQ-W was a valid weekly measure and neither scores, F(2, 37) = 2.70, p = .08, nor trajectories of change, F(2, 37) = 0.31, p = .73, differed by primary diagnosis, though power was limited. Rate of ADDQ-W change was predictive of change in both primary diagnosis severity, t = 2.40, p = .02, β = 0.32, and overall severity, t = 3.01, p < .01, β = 0.36, at post-treatment.

CONCLUSIONS

This study has established initial support for the use of the brief, easily scored, ADDQ-W for repeated assessment over treatment using a diagnostically heterogeneous clinical sample of treatment-seeking individuals.

Advancing the Scientific Foundation for Evidence-Based Practice in Clinical Child and Adolescent Psychology

Evidence-based practice (EBP) has become a central focus in clinical child and adolescent psychology. As originally defined, EBP in psychology is the integration of the best available research evidence, patient characteristics, and clinical expertise. Although evidence-based perspectives have garnered widespread acceptance in recent years, there has also been some confusion and disagreement about the 3-part definition of EBP, particularly the role of research. In this article, we first provide a brief review of the development of EBP in clinical child and adolescent psychology. Next, we outline the following 4 points to help clarify the understanding of EBP: (a) knowledge should not be confused with epistemic processes, (b) research on clinician and client factors is needed for EBP, (c) research on assessment is needed for EBP, and (d) the 3-part conceptualization of EBP can serve as a useful framework to guide research. Based on these principles, we put forth a slightly revised conceptualization of EBP, in which the role of research is expanded and more clearly operationalized. Finally, based on our review of the literature, we offer illustrative examples of specific directions for future research to advance the evidence base for EBP in clinical child and adolescent psychology.

Discrepancies between direct and indirect measures of interpersonal and neurocognitive skills in autism spectrum disorder children

BACKGROUND

Subjects with autistic spectrum disorders (ASD) show persistent deficits in social interaction. In order to explore the scope of their deficits, it is of great interest to compare the different interpersonal skills and executive functions of ASD subjects to those of children with typical development. Assessing these skills usually involves a large variety of informants (parents, teachers, other family members) and of measures, with frequent discrepancies between direct performance measures and indirect measures (third-party report). Different explanations of this mismatch between measures have been addressed in previous studies.

OBJECTIVES

We wish to analyze whether there are differences between children with ASD and children with typical development on several direct performance measures of interpersonal and neurocognitive skills and one third-party report on real-life performance of interpersonal skills; we also want to look at whether discrepancies appear between the two types of measurements in the two groups of participants.

METHOD

A total of 68 Spanish children between the ages of 7 and 12 years participated; 34 were children with ASD, and 34 showed typical development. All participants were tested for recognition of emotions, solving interpersonal conflicts, and executive function.

RESULTS

Significant differences between the two groups were not found on most of the direct performance measures, but they did appear in the report by third parties, in favor of the group with typical development. There was also a significant association between neurocognitive and social variables in the latter group.

CONCLUSIONS

There were intergroup differences and discrepancies between the direct performance and indirect measures in children with ASD, and these must be explained. For this reason, future studies could seek to explain the cause of these discrepancies with a greater number of measures for each of the skills.

Incremental Validity and Informant Effect from a Multi-Method Perspective: Assessing Relations between Parental Acceptance and Children's Behavioral Problems

This study examines the relationships between perceived parental acceptance and children's behavioral problems (externalizing and internalizing) from a multi-informant perspective. Using mothers, fathers, and children as sources of information, we explore the informant effect and incremental validity. The sample was composed of 681 participants (227 children, 227 fathers, and 227 mothers). Children's (40% boys) ages ranged from 9 to 17 years (M = 12.52, SD = 1.81). Parents and children completed both the Parental Acceptance Rejection/Control Questionnaire (PARQ/Control) and the check list of the Achenbach System of Empirically Based Assessment (ASEBA). Statistical analyses were based on the correlated uniqueness multitrait-multimethod matrix (model MTMM) by structural equations and different hierarchical regression analyses. Results showed a significant informant effect and a different incremental validity related to which combination of sources was considered. A multi-informant perspective rather than a single one increased the predictive value. Our results suggest that mother-father or child-father combinations seem to be the best way to optimize the multi-informant method in order to predict children's behavioral problems based on perceived parental acceptance.

Predictors of Informant Discrepancies Between Mother and Middle School Teacher ADHD Ratings

A diagnosis of attention-deficit/hyperactivity disorder (ADHD) according to the Diagnostic and Statistical Manual, 5th Eldition (DSM-5) is assessed in youth using ratings from both a parent and a teacher. However, individual and contextual differences between informants may lead to discrepancies in these ratings. The purpose of this study was to examine predictors of discrepancies between mother and middle school teacher reports of ADHD symptoms and related impairment. In an ethnically diverse sample of middle school students with well-diagnosed DSM-IV-TR ADHD (N = 112), we examined a range of mother and school setting characteristics that may contribute to informant discrepancies in this population. Hierarchical multiple regression analyses suggested that mothers with higher levels of education and psychopathology (i.e., ADHD symptom severity, parenting stress) may be most likely to report adolescent ADHD symptom severity that is higher than reported by teachers. Reports from general education teachers (vs. special education) were associated with lower symptom severity compared to mothers. Finally, a documented diagnosis of ADHD in the school was predictive of more severe reports from mothers. We discuss explanations for these findings and implications for assessment of middle school students with ADHD.

Comparing Multi-Informant Assessment Measures of Parental Monitoring and Their Links with Adolescent Delinquent Behavior

OBJECTIVE

Parents' poor monitoring of adolescents' whereabouts and activities is commonly linked to adolescents' increased engagement in delinquent behaviors. Yet, different domains of parental monitoring (parental monitoring behaviors vs. parental knowledge) and reports from multiple informants (parent vs. adolescent) may vary in their links to delinquent behavior.

DESIGN

Seventy-four parental caregivers and 74 adolescents completed survey measures of parental monitoring and knowledge, and adolescents completed self-report surveys of delinquent behavior.

RESULTS

We observed low-to-moderate magnitudes of correspondence between parent- and adolescent-reports of parental monitoring behaviors and parental knowledge. Adolescent self-reported delinquent behavior related to parent and adolescent reports of parental monitoring behaviors and parental knowledge, with adolescents who self-reported engagement in delinquent behaviors evidencing lower levels of parental knowledge and higher levels of poor monitoring compared to adolescents who did not self-report engagement in delinquent behaviors. Adolescent self-reported engagement in delinquent behaviors evidenced stronger links to parental monitoring when based on adolescent reports of monitoring (relative to parent reports), whereas stronger links held between adolescent self-reported delinquent behavior and parental knowledge when based on parent reports of knowledge (relative to adolescent reports).

CONCLUSIONS

Links between monitoring and adolescents' delinquent behavior vary by the kind of monitoring measure completed as well as the informant completing the measure. These findings inform measurement selection in research and clinical assessments of parental monitoring and adolescent delinquent behavior.

Use of evidence-based assessment for childhood anxiety disorders in community practice

High-quality assessment is essential to the delivery of effective treatment for childhood anxiety disorders. However, relatively little is known about how frequently child clinicians utilize evidence-based assessment (EBA) techniques in practice, and even less is known about the factors that influence EBA use in such settings. Thus, the current study presents data from a survey of 339 clinicians from a variety of professional backgrounds concerning their use of EBA for childhood anxiety disorders and explores issues preventing EBA implementation. Results indicated infrequent EBA use with clinicians citing practical barriers (i.e., time, access, knowledge, cost) and negative beliefs about EBA techniques (i.e., unhelpful) as issues preventing implementation. Implications for future EBA dissemination and implementation efforts are discussed.

Annual Research Review: Building a science of personalized intervention for youth mental health

BACKGROUND

Within the past decade, health care service and research priorities have shifted from evidence-based medicine to personalized medicine. In mental health care, a similar shift to personalized intervention may boost the effectiveness and clinical utility of empirically supported therapies (ESTs). The emerging science of personalized intervention will need to encompass evidence-based methods for determining which problems to target and in which order, selecting treatments and deciding whether and how to combine them, and informing ongoing clinical decision-making through monitoring of treatment response throughout episodes of care. We review efforts to develop these methods, drawing primarily from psychotherapy research with youths. Then we propose strategies for building a science of personalized intervention in youth mental health.

FINDINGS

The growing evidence base for personalizing interventions includes research on therapies adapted for specific subgroups; treatments targeting youths' environments; modular therapies; sequential, multiple assignment, randomized trials; measurement feedback systems; meta-analyses comparing treatments for specific patient characteristics; data-mining decision trees; and individualized metrics.

CONCLUSION

The science of personalized intervention presents questions that can be addressed in several ways. First, to evaluate and organize personalized interventions, we propose modifying the system used to evaluate and organize ESTs. Second, to help personalizing research keep pace with practice needs, we propose exploiting existing randomized trial data to inform personalizing approaches, prioritizing the personalizing approaches likely to have the greatest impact, conducting more idiographic research, and studying tailoring strategies in usual care. Third, to encourage clinicians' use of personalized intervention research to inform their practice, we propose expanding outlets for research summaries and case studies, developing heuristic frameworks that incorporate personalizing approaches into practice, and integrating personalizing approaches into service delivery systems. Finally, to build a richer understanding of how and why treatments work for particular individuals, we propose accelerating research to identify mediators within and across RCTs, to isolate mechanisms of change, and to inform the shift from diagnoses to psychopathological processes. This ambitious agenda for personalized intervention science, although challenging, could markedly alter the nature of mental health care and the benefit provided to youths and families.

Analyzing Statistical Mediation with Multiple Informants: A New Approach with an Application in Clinical Psychology

Testing mediation models is critical for identifying potential variables that need to be targeted to effectively change one or more outcome variables. In addition, it is now common practice for clinicians to use multiple informant (MI) data in studies of statistical mediation. By coupling the use of MI data with statistical mediation analysis, clinical researchers can combine the benefits of both techniques. Integrating the information from MIs into a statistical mediation model creates various methodological and practical challenges. The authors review prior methodological approaches to MI mediation analysis in clinical research and propose a new latent variable approach that overcomes some limitations of prior approaches. An application of the new approach to mother, father, and child reports of impulsivity, frustration tolerance, and externalizing problems (N = 454) is presented. The results showed that frustration tolerance mediated the relationship between impulsivity and externalizing problems. The new approach allows for a more comprehensive and effective use of MI data when testing mediation models.

Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria

Background

High-quality measurement is critical to advancing knowledge in any field. New fields, such as implementation science, are often beset with measurement gaps and poor quality instruments, a weakness that can be more easily addressed in light of systematic review findings. Although several reviews of quantitative instruments used in implementation science have been published, no studies have focused on instruments that measure implementation outcomes. Proctor and colleagues established a core set of implementation outcomes including: acceptability, adoption, appropriateness, cost, feasibility, fidelity, penetration, sustainability (Adm Policy Ment Health Ment Health Serv Res 36:24–34, 2009). The Society for Implementation Research Collaboration (SIRC) Instrument Review Project employed an enhanced systematic review methodology (Implement Sci 2: 2015) to identify quantitative instruments of implementation outcomes relevant to mental or behavioral health settings.

Methods

Full details of the enhanced systematic review methodology are available (Implement Sci 2: 2015). To increase the feasibility of the review, and consistent with the scope of SIRC, only instruments that were applicable to mental or behavioral health were included. The review, synthesis, and evaluation included the following: (1) a search protocol for the literature review of constructs; (2) the literature review of instruments using Web of Science and PsycINFO; and (3) data extraction and instrument quality ratings to inform knowledge synthesis. Our evidence-based assessment rating criteria quantified fundamental psychometric properties as well as a crude measure of usability. Two independent raters applied the evidence-based assessment rating criteria to each instrument to generate a quality profile.

Results

We identified 104 instruments across eight constructs, with nearly half (n = 50) assessing acceptability and 19 identified for adoption, with all other implementation outcomes revealing fewer than 10 instruments. Only one instrument demonstrated at least minimal evidence for psychometric strength on all six of the evidence-based assessment criteria. The majority of instruments had no information regarding responsiveness or predictive validity.

Conclusions

Implementation outcomes instrumentation is underdeveloped with respect to both the sheer number of available instruments and the psychometric quality of existing instruments. Until psychometric strength is established, the field will struggle to identify which implementation strategies work best, for which organizations, and under what conditions.

Electronic supplementary material

The online version of this article (doi:10.1186/s13012-015-0342-x) contains supplementary material, which is available to authorized users.

Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria

BACKGROUND

High-quality measurement is critical to advancing knowledge in any field. New fields, such as implementation science, are often beset with measurement gaps and poor quality instruments, a weakness that can be more easily addressed in light of systematic review findings. Although several reviews of quantitative instruments used in implementation science have been published, no studies have focused on instruments that measure implementation outcomes. Proctor and colleagues established a core set of implementation outcomes including: acceptability, adoption, appropriateness, cost, feasibility, fidelity, penetration, sustainability (Adm Policy Ment Health Ment Health Serv Res 36:24-34, 2009). The Society for Implementation Research Collaboration (SIRC) Instrument Review Project employed an enhanced systematic review methodology (Implement Sci 2: 2015) to identify quantitative instruments of implementation outcomes relevant to mental or behavioral health settings.

METHODS

Full details of the enhanced systematic review methodology are available (Implement Sci 2: 2015). To increase the feasibility of the review, and consistent with the scope of SIRC, only instruments that were applicable to mental or behavioral health were included. The review, synthesis, and evaluation included the following: (1) a search protocol for the literature review of constructs; (2) the literature review of instruments using Web of Science and PsycINFO; and (3) data extraction and instrument quality ratings to inform knowledge synthesis. Our evidence-based assessment rating criteria quantified fundamental psychometric properties as well as a crude measure of usability. Two independent raters applied the evidence-based assessment rating criteria to each instrument to generate a quality profile.

RESULTS

We identified 104 instruments across eight constructs, with nearly half (n = 50) assessing acceptability and 19 identified for adoption, with all other implementation outcomes revealing fewer than 10 instruments. Only one instrument demonstrated at least minimal evidence for psychometric strength on all six of the evidence-based assessment criteria. The majority of instruments had no information regarding responsiveness or predictive validity.

CONCLUSIONS

Implementation outcomes instrumentation is underdeveloped with respect to both the sheer number of available instruments and the psychometric quality of existing instruments. Until psychometric strength is established, the field will struggle to identify which implementation strategies work best, for which organizations, and under what conditions.

Deciding on child maltreatment: A literature review on methods that improve decision-making

Assessment and decision-making in child maltreatment cases is difficult. Practitioners face many uncertainties and obstacles during their assessment and decision-making process. Research exhibits shortcomings in this decision-making process. The purpose of this literature review is to identify and discuss methods to overcome these shortcomings. We conducted a systematic review of the published literature on decision-making using PsychINFO and MEDLINE from 2000 through May 2014. We included reviews and quantitative research studies that investigated methods aimed at improving professional decision-making on child abuse and neglect in child welfare and child protection. Although many researchers have published articles on decision-making including ideas and theories to improve professional decision-making, empirical research on these improvements is scarce. Available studies have shown promising results. Structured decision-making has created a greater child-centred and holistic approach that takes the child's family and environment into account, which has made practitioners work more systematically and improved the analysis of complex situations. However, this approach has not improved inter-rater agreement on decisions made. Shared decision-making may improve the participation of parents and children and the quality of decisions by taking client treatment preferences into account in addition to scientific evidence and clinical experience. A number of interesting developments appear in recent research literature; however, child welfare and child protection must find additional inspiration from other areas, e.g., mental health services, because research on decision-making processes in child welfare and child protection is still rare.

Evaluating change in symptomatic and functional level of children and youth with emotional disorders: a naturalistic observation study

The objective of the study is to describe the changes in symptomatic and functional impairment for children and youth with emotional disorders treated at child and adolescent mental health outpatient services (CAMHS) in Norway. The study was of naturalistic observational type in which the treatment can be classified as "treatment as usual" (TAU). The Strengths and Difficulties Questionnaire (SDQ), the Health of the Nation Outcome Scale (HONOSCA) and the Children's Global Assessment Scale (CGAS) were used as measures of change. The information from multiple informants allowed the evaluation of change from different perspectives. The sample consisted of 84 children and youth with emotional disorders treated at two CAMHS in the North of Norway. The SDQ, the HONOSCA and the CGAS were administered at intake (T0), during assessment (T1) and approximately, 6 months after T1 (T2). Change was analysed by means of the Linear Mixed Models procedure. The results show that children and youth with emotional disorders experience a statistically significant improvement per month during outpatient treatment according to nearly all the measures of change. For the clinician rated scores, change rates during active assessment/treatment were larger than during the waitlist period. Evaluating change from the perspective of clinical significance showed that only a small proportion of the subjects had change scores that were statistically reliable and clinically significant. Whether an actual change has occurred is uncertain for the majority of patients.

Trainee and Client Experiences of Therapeutic Assessment in a Required Graduate Course: A Qualitative Analysis

Surveys indicate that practice and training in psychological assessment, and personality assessment (PA) to a lesser degree, has been stable or increasing over the past quarter-century. However, its future arguably remains threatened due to changes in doctoral training programs and beliefs in the field concerning the utility of PA for treatment success. To increase interest in and use of PA, studies of training methods that include trainees' perspectives are needed. This study evaluated the experiences of 10 graduate trainees and their clients who were trained in and conducted a brief Therapeutic Assessment (TA). Qualitative responses to a self-evaluation administered post-TA were coded using directed content analysis. Results indicated that trainees viewed TA/PA as having clinical utility; they had positive feelings about TA/PA, and they desired or intended to use or continue learning about TA/PA. Clients' responses reflected positive feelings about the TA, having gained new self-awareness or understanding, and having a positive relationship with the assessor. The findings suggest that teaching PA from a TA perspective could produce positive benefits for psychology trainees.

Assessment Practices of Child Clinicians

Assessment is an integral component of treatment. However, prior surveys indicate clinicians may not use standardized assessment strategies. We surveyed 1,510 clinicians and used multivariate analysis of variance to explore group differences in specific measure use. Clinicians used unstandardized measures more frequently than standardized measures, although psychologists used standardized measures more frequently than nonpsychologists. We also used latent profile analysis to classify clinicians based on their overall approach to assessment and examined associations between clinician-level variables and assessment class or profile membership. A four-profile model best fit the data. The largest profile consisted of clinicians who primarily used unstandardized assessments (76.7%), followed by broad-spectrum assessors who regularly use both standardized and unstandardized assessment (11.9%), and two smaller profiles of minimal (6.0%) and selective assessors (5.5%). Compared with broad-spectrum assessors, unstandardized and minimal assessors were less likely to report having adequate standardized measures training. Implications for clinical practice and training are discussed.

The validity of the multi-informant approach to assessing child and adolescent mental health

Child and adolescent patients may display mental health concerns within some contexts and not others (e.g., home vs. school). Thus, understanding the specific contexts in which patients display concerns may assist mental health professionals in tailoring treatments to patients' needs. Consequently, clinical assessments often include reports from multiple informants who vary in the contexts in which they observe patients' behavior (e.g., patients, parents, teachers). Previous meta-analyses indicate that informants' reports correlate at low-to-moderate magnitudes. However, is it valid to interpret low correspondence among reports as indicating that patients display concerns in some contexts and not others? We meta-analyzed 341 studies published between 1989 and 2014 that reported cross-informant correspondence estimates, and observed low-to-moderate correspondence (mean internalizing: r = .25; mean externalizing: r = .30; mean overall: r = .28). Informant pair, mental health domain, and measurement method moderated magnitudes of correspondence. These robust findings have informed the development of concepts for interpreting multi-informant assessments, allowing researchers to draw specific predictions about the incremental and construct validity of these assessments. In turn, we critically evaluated research on the incremental and construct validity of the multi-informant approach to clinical child and adolescent assessment. In so doing, we identify crucial gaps in knowledge for future research, and provide recommendations for "best practices" in using and interpreting multi-informant assessments in clinical work and research. This article has important implications for developing personalized approaches to clinical assessment, with the goal of informing techniques for tailoring treatments to target the specific contexts where patients display concerns. (PsycINFO Database Record

The validity of the multi-informant approach to assessing child and adolescent mental health

Child and adolescent patients may display mental health concerns within some contexts and not others (e.g., home vs. school). Thus, understanding the specific contexts in which patients display concerns may assist mental health professionals in tailoring treatments to patients' needs. Consequently, clinical assessments often include reports from multiple informants who vary in the contexts in which they observe patients' behavior (e.g., patients, parents, teachers). Previous meta-analyses indicate that informants' reports correlate at low-to-moderate magnitudes. However, is it valid to interpret low correspondence among reports as indicating that patients display concerns in some contexts and not others? We meta-analyzed 341 studies published between 1989 and 2014 that reported cross-informant correspondence estimates, and observed low-to-moderate correspondence (mean internalizing: r = .25; mean externalizing: r = .30; mean overall: r = .28). Informant pair, mental health domain, and measurement method moderated magnitudes of correspondence. These robust findings have informed the development of concepts for interpreting multi-informant assessments, allowing researchers to draw specific predictions about the incremental and construct validity of these assessments. In turn, we critically evaluated research on the incremental and construct validity of the multi-informant approach to clinical child and adolescent assessment. In so doing, we identify crucial gaps in knowledge for future research, and provide recommendations for "best practices" in using and interpreting multi-informant assessments in clinical work and research. This article has important implications for developing personalized approaches to clinical assessment, with the goal of informing techniques for tailoring treatments to target the specific contexts where patients display concerns. (PsycINFO Database Record

Discrepancy between parents and children in reporting of distress and impairment: Association with critical symptoms

BACKGROUND

We examined discrepant parent-child reports of subjective distress and psychosocial impairment.

METHOD

Parent-child pairs (N = 112 pairs) completed the Health Dynamics Inventory at intake for outpatient therapy.

RESULTS

Average parent scores were significantly higher than average child scores on distress, impairment, and externalizing symptoms, but not internalizing symptoms. There were significant associations between parent-child discrepancy (i.e. children who reported greater distress or impairment than parents or vice versa) and child endorsement of several notable symptoms (rapid mood swings, panic, nightmares, and suicidal ideation).

CONCLUSION

Parents tended to report more externalizing symptoms, distress, and impairment than children reported; however, when children report more distress and impairment than parents, this may indicate serious psychological problems.

Parent-child discrepancies in the assessment of children's and adolescents' happiness

In this study, we assessed parent-child agreement in the perception of children's general happiness or well-being in typically developing children (10- and 11-year-olds, n = 172) and adolescents (15- and 16-year-olds, n = 185). Despite parent and child reporters providing internally consistent responses in the General Happiness single-item scale and the Oxford Happiness Questionnaire-Short Form, their perceptions about children's and adolescents' general happiness did not correlate. Parents of 10- and 11-year-olds significantly overestimated children's happiness, supporting previous literature on the parents' positivity bias effect. However, parents of 15- and 16-year-olds showed the reverse pattern by underestimating adolescents' happiness. Furthermore, parents' self-reported happiness or well-being (reported 6 months later) significantly correlated with their estimations of children's and adolescents' happiness. Therefore, these results suggest a potential parents' "egocentric bias" when estimating their children's happiness. These findings are discussed in terms of their theoretical and applied implications for research into child-parent relationships.

Measurement Invariance of Oppositional Defiant Disorder Dimensions in 3-Year-Old Preschoolers

Measurement invariance (metric/scalar) of oppositional defiant disorder (ODD) dimensions (negative affect, oppositional behavior, and antagonistic behavior) across sex and informants is tested. Parents and teachers of 622 preschool children from the general population answered a dimensional measure of ODD. ODD dimensions function similarly in boys and girls. Some differences were found by informant, indicating that the equivalence of the ratings of parents and teachers is not complete and that given the same underlying level of the latent trait, some parents’ item scores were higher than those of teachers. Metric invariance was complete but scalar invariance was not attained. The results contribute evidence on the conceptualization of ODD as a source-specific disorder. The simultaneous use of ODD dimensions reported by parents and teachers must be considered in the context of a lack of complete measurement invariance, which implies that comparisons of observed means from parents and teachers are not readily interpretable.

The Incremental Validity of Narrative Identity in Predicting Well-Being

Grounded in four theoretical positions—structural, cognitive, phenomenological, and ethical—the present review demonstrates the empirical evidence for the incremental validity of narrative identity as a cross-sectional indicator and prospective predictor of well-being, compared with other individual difference and situational variables. In doing so, we develop an organizational framework of four categories of narrative variables: (a) motivational themes, (b) affective themes, (c) themes of integrative meaning, and (d) structural elements. Using this framework, we detail empirical evidence supporting the incremental association between narrative identity and well-being, a case that is strongest for motivational, affective, and integrative meaning themes. These categories of themes serve as vital complimentary correlates and predictors of well-being, alongside commonly assessed variables such as dispositional personality traits. We then use the theoretically grounded review of the empirical literature to develop concrete areas of future research for the field.

Introduction to the special issue: toward implementing physiological measures in clinical child and adolescent assessments

The National Institute of Mental Health recently launched the Research Domain Criteria (RDoC). The RDoC is an initiative to improve classification of mental health concerns by promoting research on the brain mechanisms underlying these concerns, with the ultimate goal of developing interventions that target these brain mechanisms. A key focus of RDoC involves opening new lines of research examining patients' responses on biological measures. The RDoC presents unique challenges to mental health professionals who work with children and adolescents. Indeed, mental health professionals rarely integrate biological measures into clinical assessments. Thus, RDoC's ability to improve patient care rests, in part, on the development of strategies for implementing biological measures within mental health assessments. Further, mental health professionals already carry out comprehensive assessments that frequently yield inconsistent findings. These inconsistencies have historically posed challenges to interpreting research findings as well as assessment outcomes in practice settings. In this introductory article, we review key issues that informed the development of a special issue of articles demonstrating methods for implementing low-cost measures of physiological functioning in clinical child and adolescent assessments. We also outline a conceptual framework, informed by theoretical work on using and interpreting multiple informants' clinical reports (De Los Reyes, Thomas, Goodman, & Kundey, 2013 ), to guide hypothesis testing when using physiological measures within clinical child and adolescent assessments. This special issue and the conceptual model described in this article may open up new lines of research testing paradigms for implementing clinically feasible physiological measures in clinical child and adolescent assessments.

Implementing clinical outcomes assessment in everyday school mental health practice

Evidence-based assessment (EBA) has been shown to improve clinical outcomes, but this practice is frequently not implemented in school mental health practice. This article reviews potential barriers to implementation and offers practical strategies for addressing these challenges. Several valid and reliable tools for assessment are reviewed, and information is provided on clinical use. Case examples of EBA implementation in school mental health settings are provided to illustrate how these tools can be used in everyday practice by school mental health clinicians.

The Strengths and Difficulties Questionnaire: psychometric properties of the parent and teacher version in children aged 4-7

Background

The Strengths and Difficulties Questionnaire is one of the most employed screening instruments. Although there is a large research body investigating its psychometric properties, reliability and validity are not yet fully tested using modern techniques. Therefore, we investigate reliability, construct validity, measurement invariance, and predictive validity of the parent and teacher version in children aged 4–7. Besides, we intend to replicate previous studies by investigating test-retest reliability and criterion validity.

Methods

In a Dutch community sample 2,238 teachers and 1,513 parents filled out questionnaires regarding problem behaviors and parenting, while 1,831 children reported on sociometric measures at T1. These children were followed-up during three consecutive years. Reliability was examined using Cronbach’s alpha and McDonald’s omega, construct validity was examined by Confirmatory Factor Analysis, and predictive validity was examined by calculating developmental profiles and linking these to measures of inadequate parenting, parenting stress and social preference. Further, mean scores and percentiles were examined in order to establish norms.

Results

Omega was consistently higher than alpha regarding reliability. The original five-factor structure was replicated, and measurement invariance was established on a configural level. Further, higher SDQ scores were associated with future indices of higher inadequate parenting, higher parenting stress and lower social preference. Finally, previous results on test-retest reliability and criterion validity were replicated.

Conclusions

This study is the first to show SDQ scores are predictively valid, attesting to the feasibility of the SDQ as a screening instrument. Future research into predictive validity of the SDQ is warranted.

Electronic supplementary material

The online version of this article (doi:10.1186/s40359-015-0061-8) contains supplementary material, which is available to authorized users.

Free, brief, and validated: Standardized instruments for low-resource mental health settings

Evidence-based assessment has received little attention despite its critical importance to the evidence-based practice movement. Given the limited resources in the public sector, it is necessary for evidence-based assessment to utilize tools with established reliability and validity metrics that are free, easily accessible, and brief. We review tools that meet these criteria for youth and adult mental health for the most prevalent mental health disorders to provide a clinical guide and reference for the selection of assessment tools for public sector settings. We also discuss recommendations for how to move forward the evidence-based assessment agenda.

Therapeutic Harm and Cultural Oppression

The divergent discourses between scholars from the potentially harmful treatment and multicultural psychology camps are accurately observed by Wendt, Gone, and Nagata. I argue that the differences in perspectives between the two groups are more about a clash of therapeutic worldviews, that they are often antagonistic to one another, that conversations have been a one-way process (with one side “not wanting to hear”), and that sociopolitical forces play a significant role in preventing a true dialogue from occurring. I conclude that the ultimate harm to groups of color is cultural oppression.

A complementary marriage of perspectives: understanding organizational social context using mixed methods

Background

Organizational factors impact the delivery of mental health services in community settings. Mixed-methods analytic approaches have been recommended, though little research within implementation science has explicitly compared inductive and deductive perspectives to understand their relative value in understanding the same constructs. The purpose of our study is to use two different paradigmatic approaches to deepen our understanding of organizational social context. We accomplish this by using a mixed-methods approach in an investigation of organizational social context in community mental health clinics.

Methods

Nineteen agencies, representing 23 sites, participated. Enrolled participants included 130 therapists, 36 supervisors, and 22 executive administrators. Quantitative data was obtained via the Organizational Social Context (OSC) measure. Qualitative data, comprised of direct observation with spot sampling generated from agency visits, was coded using content analysis and grounded theory. The present study examined elements of organizational social context that would have been missed if only quantitative data had been obtained and utilized mixed methods to investigate if stratifying observations based on quantitative ratings from the OSC resulted in the emergence of differential themes.

Results

Four of the six OSC constructs were commonly observed in field observations (i.e., proficiency, rigidity, functionality, stress), while the remaining two constructs were not frequently observed (i.e., resistance, engagement). Constructs emerged related to organizational social context that may have been missed if only quantitative measurement was employed, including those around the physical environment, commentary about evidence-based practice initiatives, leadership, cultural diversity, distrust, and affect. Stratifying agencies by “best,” “average,” and “worst” organizational social context impacted interpretation for three constructs (affect, stress, and leadership).

Conclusions

Results support the additive value of integrating inductive and deductive perspectives in implementation science research. This synthesis of approaches facilitated a more comprehensive understanding and interpretation of the findings than would have been possible if either methodology had been employed in isolation.

Reliable ratings or reading tea leaves: can parent, teacher, and clinician behavioral ratings of preschoolers predict ADHD at age six?

To assess the relative ability of parent, teacher, and clinician behavioral ratings of preschoolers to predict ADHD severity and diagnosis at 6 years of age. Hyperactive/inattentive preschoolers [N = 104, 75 % boys, Mean (SD) age = 4.37 (0.47) years] were followed over 2 years (mean = 26.44 months, SD = 5.66). At baseline (BL), parents and teachers completed the ADHD-RS-IV and clinicians completed the Behavioral Rating Inventory for Children following a psychological testing session. At age 6, [Mean (SD) age = 6.62 (0.35) years], parents were interviewed with the K-SADS-PL; teachers completed the ADHD-RS-IV; and laboratory measures of hyperactivity, impulsivity, and inattention were obtained from children. Hierarchical logistic and linear regression analyses examined which combination of BL ratings best predicted 6-year-old ADHD diagnosis and severity, respectively. At age 6, 56 (53.8 %) children met DSM-IV criteria for a diagnosis of ADHD. BL ratings from parent/teacher/clinician, parent/teacher and parent/clinician combinations significantly predicted children who had an ADHD diagnosis at age 6. Parent and clinician, but not teacher, behavior ratings were significant independent predictors of ADHD diagnosis and severity at 6-years-old. However, only clinician reports of preschoolers' behaviors predicted laboratory measures of over-activity and inattention at follow-up. Cross-situationality is important for a diagnosis of ADHD during the preschool years. Among parents, teachers and clinicians, positive endorsements from all three informants, parent/teacher or parent/clinician appear to have prognostic value. Clinicians' ratings of preschoolers' inattention, impulsivity and hyperactivity are valid sources of information for predicting ADHD diagnosis and severity over time.

A longitudinal analysis of parent and teacher ratings of problem behavior in boys with and without developmental delays

This study investigated parent and teacher ratings of behavior problems in children with and without intellectual disabilities at three time points over the course of 9 years. The group of children with intellectual disabilities (IDs) had higher behavior problem ratings than the group of children with no IDs (NIDs) across the three time points. Parents and teachers generally agreed on behavior, with the exception of attention problems and externalizing problems. The ratings of problem behavior remained stable over the three time points. Our findings imply that children with ID may be more likely to be perceived as having greater externalizing behaviors by teachers than by parents and that parents may perceive children as having greater attention problems than teachers. For the majority of the subscales, the lack of differences between parent and teacher ratings over time implies relative stability of ratings over a 9-year period from young childhood through adolescence for individuals with and without ID.

Axis I anxiety and mental health disorders among stuttering adolescents

PURPOSE

The purpose of this study was to evaluate anxiety and psychological functioning among adolescents seeking speech therapy for stuttering using a structured, diagnostic interview and psychological questionnaires. This study also sought to determine whether any differences in psychological status were evident between younger and older adolescents.

METHOD

Participants were 37 stuttering adolescents seeking stuttering treatment. We administered the Computerized Voice Version of the Diagnostic Interview Schedule for Children, and five psychometric tests. Participants were classified into younger (12-14 years; n=20) and older adolescents (15-17 years; n=17).

RESULTS

Thirty-eight percent of participants attained at least one diagnosis of a mental disorder, according to the diagnostic criteria of the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV; APA, 2000), with the majority of these diagnoses involving anxiety. This figure is double current estimates for general adolescent populations, and is consistent with our finding of moderate and moderate-severe quality of life impairment. Although many of the scores on psychological measures fell within the normal range, older adolescents (15-17 years) reported significantly higher anxiety, depression, reactions to stuttering, and emotional/behavioral problems, than younger adolescents (12-14 years). There was scant evidence that self-reported stuttering severity is correlated with mental health issues. There are good reasons to believe these results are conservative because many participants gave socially desirable responses about their mental health status.

DISCUSSION

These results reveal a need for large-scale, statistically powerful assessments of anxiety and other mental disorders among stuttering adolescents with reference to control populations.

EDUCATIONAL OBJECTIVES

The reader will be able to: (a) explain the clinical importance of assessing for mental health with stuttering adolescents, (b) state the superior method for adolescent mental health assessment and (c) state a major issue with determining the genuineness of stuttering adolescent responses to psychological assessment.

Clinical decision making about child and adolescent anxiety disorders using the Achenbach system of empirically based assessment

Anxiety disorders are common among children but can be difficult to diagnose. An actuarial approach to the diagnosis of anxiety may improve the efficiency and accuracy of the process. The objectives of this study were to determine the clinical utility of the Achenbach Child Behavior Checklist (CBCL) and Youth Self Report (YSR), two widely used assessment tools, for diagnosing anxiety disorders in youth and to aid clinicians in incorporating scale scores into an actuarial approach to diagnosis through a clinical vignette. Demographically diverse youth, 5 to 18 years of age, were drawn from two samples; one (N = 1,084) was recruited from a research center, and the second (N = 651) was recruited from an urban community mental health center. Consensus diagnoses integrated information from semistructured interview, family history, treatment history, and clinical judgment. The CBCL and YSR internalizing problems T scores discriminated cases with any anxiety disorder or with generalized anxiety disorder from all other diagnoses in both samples (ps < .0005); the two scales had equivalent discriminative validity (ps > .05 for tests of difference). No other scales, nor any combination of scales, significantly improved on the performance of the Internalizing scale. In the highest risk group, Internalizing scores greater than 69 (CBCL) or greater than 63 (YSR) resulted in a Diagnostic Likelihood Ratio of 1.5; low scores reduced the likelihood of anxiety disorders by a factor of 4. Combined with other risk factor information in an actuarial approach to assessment and diagnosis, the CBCL and YSR Internalizing scales provide valuable information about whether a youth is likely suffering from an anxiety disorder.

The Functional Idiographic Assessment Template-Questionnaire (FIAT-Q): Initial Psychometric Properties

The evidence based assessment (EBA) movement stresses the importance of psychological measures with strong psychometric properties and clinical utility. The Functional Idiographic Assessment Template system (FIAT; Callaghan, 2006) is a functional analytic behavioral approach to the assessment of interpersonal functioning for use with therapies like Functional Analytic Psychotherapy (FAP; Kohlenberg & Tsai, 1991). While research has begun to demonstrate the clinical utility of the FIAT, its psychometric properties have not been explored. The present study examines the Functional Idiographic Assessment Template-Questionnaire (FIAT-Q), a self-report measure contained in the FIAT. Two different approaches are used to explore the psychometric properties and structure of the FIAT-Q, and test-retest reliability is examined. These methods are discussed along with the use of the FIAT-Q as an alternative to nosological assessment of client behaviors.