Convergent and incremental predictive validity of clinician, self-report, and structured interview diagnoses for personality disorders over 5 years

OBJECTIVE

Research has demonstrated poor agreement between clinician-assigned personality disorder (PD) diagnoses and those generated by self-report questionnaires and semistructured diagnostic interviews. No research has compared prospectively the predictive validity of these methods. We investigated the convergence of these 3 diagnostic methods and tested their relative and incremental validity in predicting independent, multimethod assessments of psychosocial functioning performed prospectively over 5 years.

METHOD

Participants were 320 patients in the Collaborative Longitudinal Personality Disorders Study diagnosed with PDs by therapist, self-report, and semistructured interview at baseline. We examined the relative incremental validity of therapists' naturalistic ratings relative to these other diagnostic methods for predicting psychosocial functioning at 5-year follow-up.

RESULTS

Hierarchical linear regression analyses revealed that both the self-report questionnaire and semistructured interview PD diagnoses had significant incremental predictive validity over the PD diagnoses assigned by a treating clinician. Although, in some cases, the clinicians' ratings for individual PDs did have validity for predicting subsequent functioning, they did not generally provide incremental prediction beyond the other methods. These findings remained robust in a series of analyses restricted to a subsample of therapist ratings based on clinical contact of 1 year or greater.

CONCLUSIONS

These results from a large clinical sample echo previous research documenting limited agreement between clinicians' naturalistic PD diagnoses and those from self-report and semistructured interview methods. They extend prior work by providing the first evidence about the relative predictive validity of these different methods. Our findings challenge the validity of naturalistic PD diagnoses and suggest the use of structured diagnostic instruments.

Comparing methods for scoring personality disorder types using maladaptive traits in DSM-5

The DSM-5 (Diagnostic and Statistical Manual of Mental Disorders, 5th ed.) Section III will include an alternative hybrid system for the diagnosis of personality disorder (PD). This alternative system defines PD types partly through specific combinations of maladaptive traits, rather than by using a set of polythetic diagnostic criteria. The current report utilizes a large sample of undergraduates (n = 1,159) to examine three dimensional methods for comparing an individual's trait profile to each PD type. We found that the sum of an individual's scores on the assigned traits obtained large convergent correlations (Mdn r =.61) and best reproduced the patterns of PD discriminant correlations observed within the DSM-IV measure. We also tested the DSM-5 Section III model algorithms and compared them with different thresholds for assigning categorical diagnoses. Frequency rates using the algorithms were greatly reduced, whereas requiring half of the assigned traits produced rates that more closely approximated current prevalence estimates. Our research suggests that DSM-5 Section III trait model can reproduce the DSM-IV-TR PD constructs and identifies effective methods of doing so.

Ten-year rank-order stability of personality traits and disorders in a clinical sample

OBJECTIVE

This study compares the 10-year retest stability of normal traits, pathological traits, and personality disorder dimensions in a clinical sample.

METHOD

Ten-year rank-order stability estimates for the Revised NEO Personality Inventory, Schedule for Nonadaptive and Adaptive Personality, and Diagnostic Interview for DSM-IV Personality Disorders were evaluated before and after correcting for test-retest dependability and internal consistency in a clinical sample (N = 266).

RESULTS

Dependability-corrected stability estimates were generally in the range of.60-.90 for traits and.25-.65 for personality disorders.

CONCLUSIONS

The relatively lower stability of personality disorder symptoms may indicate important differences between pathological behaviors and relatively more stable self-attributed traits and imply that a full understanding of personality and personality pathology needs to take both traits and symptoms into account. The five-factor theory distinction between basic tendencies and characteristic adaptations provides a theoretical framework for the separation of traits and disorders in terms of stability in which traits reflect basic tendencies that are stable and pervasive across situations, whereas personality disorder symptoms reflect characteristic maladaptations that are a function of both basic tendencies and environmental dynamics.

An Investigation of the Factor Structure and Convergent and Discriminant Validity of the Five-Factor Model Rating Form

The Five-Factor Model Rating Form (FFMRF) is a one-page measure designed to provide an efficient assessment of the higher order domains of the Five Factor Model (FFM) as well as the more specific, lower order facets proposed by McCrae and Costa. Although previous research has suggested that the FFMRF’s assessment of the lower order facets converge reasonably with other FFM measures, the structural validity of the domain-level assessment has not yet been evaluated. The current study employed an exploratory structural equation modeling framework to investigate the fit of a five-factor solution within a combined sample of 757 participants. This was a novel analysis using a combined sample drawn from three previously published studies and was composed primarily of undergraduates but also included a smaller clinical subsample. Results indicated that the FFMRF is well accommodated within a five-factor solution. Furthermore, the FFMRF domain scores evinced large correlations with domain scores from the NEO Personality Inventory–Revised. The results suggest that the FFMRF might hold promise as a choice for those seeking a brief measure that provides a valid assessment of both the broad and specific traits of the FFM.

Conceptual changes to the definition of borderline personality disorder proposed for DSM-5

The Diagnostic and Statistical Manual of Mental Disorders (DSM)-5 Personality and Personality Disorders Work Group proposed the elimination of diagnostic criterion sets in favor of a prototype matching system that defines personality disorders using narrative descriptions. Although some research supports this general approach, no empirical studies have yet examined the specific definitions proposed for DSM-5. Given the wide interest in borderline personality disorder (BPD), it is crucial to determine how this methodological shift might affect the content and conceptualization of the diagnosis. Eighty-two experts on BPD provided ratings of the DSM-IV-TR or DSM-5 version of BPD in terms of 37 traits proposed for DSM-5. Analyses revealed significant and meaningful differences among the two constructs, with the DSM-5 version evincing increased interpersonal dependency but a decreased emphasis on antagonism and disinhibition. A second study within a clinical sample demonstrated that both antagonism and disinhibition mediated the relationships between DSM-IV BPD and impairment, suggesting that the proposed changes might have important consequences for BPD's coverage, prevalence, and nomological network. More globally, our results illustrate that unanticipated shifts in diagnostic constructs can stem from seemingly minor revisions and suggest that research is needed to understand how these, or other changes, might affect our conceptualization, diagnosis, and treatment of BPD.

The convergent and concurrent validity of trait-based prototype assessment of personality disorder categories in homeless persons

The DSM-5 proposal indicates that personality disorders (PDs) be defined as collections of maladaptive traits but does not provide a specific diagnostic method. However, researchers have previously suggested that PD constructs can be assessed by comparing individuals' trait profiles with those prototypic of PDs and evidence from the five-factor model (FFM) suggests that these prototype matching scores converge moderately with traditional PD instruments. The current study investigates the convergence of FFM PD prototypes with interview-assigned PD diagnoses in a sample of 99 homeless individuals. This sample had very high rates of PDs, which extends previous research on samples with more modest prevalence rates. Results indicated that diagnostic agreement between these methods was generally low but consistent with the agreement previously observed between explicit PD measures. Furthermore, trait-based and diagnostic interview scores evinced similar relationships with clinically important indicators such as abuse history and past suicide attempts. These findings demonstrate the validity of prototype methods and suggest their consideration for assessing trait-defined PD types within DSM-5.

Personality disorders and retention in a therapeutic community for substance dependence

Although therapeutic community (TC) treatment is a promising intervention for substance use disorders, a primary obstacle to successful treatment is premature attrition. Because of their prevalence within substance use treatment facilities, personality disorder (PD) diagnoses have been examined as predictors of treatment completion. Prior research on TC outcomes has focused almost exclusively on antisocial personality disorder (ASPD), and the results have been mixed. This study extends previous research by examining the impact of the 10 Axis II PDs on early (first 30 days) attrition as well as overall time to dropout in a 9-month residential TC. Survival analyses indicated that borderline was the only PD negatively related to overall program retention. In contrast, ASPD, as well as histrionic PD, were related to very early attrition, but not to overall program retention. Early assessment and identification of at-risk individuals may improve treatment retention and outcome for TC treatment.

Five factor model prototype matching scores: convergence within alternative methods

The Five Factor Model of Personality (FFM) has been proposed as a potential alternative to the current DSM-IV-TR model, which conceptualizes personality disorders (PDs) as categorical constructs. While an extensive literature has pointed out the flaws of the diagnostic categories, they are quite familiar to clinicians and there may still be instances when identifying these constructs for clinical purposes, such as for rapidly communicating information about a patient, is warranted. From the perspective of the FFM, the PDs represent specific constellations of personality traits and research has demonstrated that the PDs can be recovered by assessing the degree to which an FFM profile matches the FFM description of a prototypic PD case. The current study builds upon that research by assessing the convergent and discriminant validity of prototype scores and DSM-IV PD measures using self-report, informant report, semi-structured interview, and clinician descriptions. The results suggest that the prototype matching scores are largely valid across these methods for all PDs, with perhaps the exception of obsessive-compulsive. These findings are related to previous research and the clinical implications of these findings are discussed.

Comparing the temporal stability of self-report and interview assessed personality disorder

Findings from several large-scale, longitudinal studies over the last decade have challenged the long-held assumption that personality disorders (PDs) are stable and enduring. However, the findings, including those from the Collaborative Longitudinal Personality Disorders Study (CLPS; Gunderson et al., 2000), rely primarily on results from semistructured interviews. As a result, less is known about the stability of PD scores from self-report questionnaires, which differ from interviews in important ways (e.g., source of the ratings, item development, and instrument length) that might increase temporal stability. The current study directly compared the stability of the Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM-IV) PD constructs assessed via the Schedule for Nonadaptive and Adaptive Personality (SNAP-2; Clark, Simms, Wu, & Casillas, in press) with those from the Diagnostic Interview for DSM-IV Personality Disorders (Zanarini, Frankenburg, Sickel, & Yong, 1996) over 2 years in a sample of 529 CLPS participants. Specifically, we compared dimensional and categorical representations from both measures in terms of rank-order and mean-level stability. Results indicated that the dimensional scores from the self-report questionnaire had significantly greater rank-order (mean r=.69 vs. .59) and mean-level (mean d=0.21 vs. 0.30) stability. In contrast, categorical diagnoses from the two measures evinced comparable rank-order (mean κ=.38 vs. .37) and mean-level stability (median prevalence rate decrease of 3.5% vs. 5.6%). These findings suggest the stability of PD constructs depends at least partially on the method of assessment and are discussed in the context of previous research and future conceptualizations of personality pathology.

Clinicians' Use of Personality Disorder Models within a Particular Treatment Setting: A Longitudinal Comparison of Temporal Consistency and Clinical Utility

An active line of current investigation is how the five-factor model (FFM) of personality disorder might be applied by clinicians and particularly, how clinically useful this model is in comparison to the existing nomenclature. The current study is the first to investigate the temporal consistency of clinicians' application of the FFM and the DSM-IV-TR to their own patients. Results indicated that FFM ratings were relatively stable over six-months of treatment, supporting their use by clinicians, but also indexed potentially important clinical changes. Additionally, ratings of utility provided by the clinicians suggested that the FFM was more useful for clinical decision making than was the DSM-IV-TR model. We understand the clinical utility findings within the context of previous research indicating that the FFM is most useful among patients who are not prototypic for a personality disorder.

The impact of NEO PI-R gender norms on the assessment of personality disorder profiles

Many personality assessment inventories provide gender-specific norms to allow comparison of an individual's standing relative to others of the same gender. In some cases, this means that an identical raw score produces standardized scores that differ notably depending on whether the respondent is male or female. Thus, an important question is whether unisex-normed scores or gender-normed scores more validly assess personality. Gender-normed and unisex-normed scores from the NEO Personality Inventory--Revised (P. T. Costa & R. R. McCrae, 1992) were examined in a large clinical sample, using 2 measures of personality disorder as validating criteria. Gender-normed scores did not obtain significantly higher correlations. In fact, for 2 personality disorders (antisocial and narcissistic), gender-normed scores yielded significantly lower correlations, suggesting that personality disorder pathology relates most closely to one's absolute level of a personality trait, rather than one's standing relative to others of the same gender. Ramifications of this finding for personality research and clinical assessment are discussed.

Differential item functioning on the Five Facet Mindfulness Questionnaire is minimal in demographically matched meditators and nonmeditators

A recent study of the Five Facet Mindfulness Questionnaire reported high levels of differential item functioning (DIF) for 18 of its 39 items in meditating and nonmeditating samples that were not demographically matched. In particular, meditators were more likely to endorse positively worded items whereas nonmeditators were more likely to deny negatively worded (reverse-scored) items. The present study replicated these analyses in demographically matched samples of meditators and nonmeditators (n = 115 each) and found that evidence for DIF was minimal. There was little or no evidence for differential relationships between positively and negatively worded items for meditators and nonmeditators. Findings suggest that DIF based on items' scoring direction is not problematic when the Five Facet Mindfulness Questionnaire is used to compare demographically similar meditators and nonmeditators.

Comparing personality disorder models: cross-method assessment of the FFM and DSM-IV-TR

The current edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV-TR; American Psychiatric Association, 2000) defines personality disorders as categorical entities that are distinct from each other and from normal personality traits. However, many scientists now believe that personality disorders are best conceptualized using a dimensional model of traits that span normal and abnormal personality, such as the Five-Factor Model (FFM). However, if the FFM or any dimensional model is to be considered as a credible alternative to the current model, it must first demonstrate an increment in the validity of the assessment offered within a clinical setting. Thus, the current study extended previous research by comparing the convergent and discriminant validity of the current DSM-IV-TR model to the FFM across four assessment methodologies. Eighty-eight individuals receiving ongoing psychotherapy were assessed for the FFM and the DSM-IV-TR personality disorders using self-report, informant report, structured interview, and therapist ratings. The results indicated that the FFM had an appreciable advantage over the DSM-IV-TR in terms of discriminant validity and, at the domain level, convergent validity. Implications of the findings and directions for future research are discussed.

A diagnostic interview module for anabolic-androgenic steroid dependence: preliminary evidence of reliability and validity

The syndrome of anabolic-androgenic steroid (AAS) dependence, though well recognized, remains poorly studied. In this preliminary psychometric study, American and British investigators separately administered a structured diagnostic interview module, based on recently proposed criteria for AAS dependence, to 42 male AAS users in Middlesbrough, England. Another investigator, blinded to the diagnostic interview findings, assessed self-reported symptoms of "muscle-dysmorphia"; effects of AAS on various aspects of functioning; and maximum proportion of annual income spent on AAS. We also assessed demographic measures, history of other substance use, and performance on a hypothetical AAS-purchasing task. The interview module yielded very good interrater reliability (kappa = 0.76 and overall intraclass correlation = 0.79) and strong internal consistency (Cronbach's alpha = 0.77-0.87). Men diagnosed as AAS-dependent, when compared to nondependent men, reported significantly earlier onset of AAS use, longer duration and higher maximum doses of AAS used, more frequent use of other performance-enhancing drugs, and a somewhat larger maximum percentage of income spent on AAS. Dependent users also "bought" more AAS in the hypothetical purchase task, but rated significantly more negatively the effects of AAS on their mental health-findings all suggesting that the diagnosis of AAS dependence shows construct validity. As a group, AAS users showed high preoccupation with muscular appearance, but dependence per se was not significantly associated with this measure-suggesting that the diagnosis of AAS dependence shows some evidence of discriminant validity. Collectively, these findings suggest that AAS dependence may be diagnosed reliably, with preliminary evidence for construct and discriminant validity.

A comparison of obsessive-compulsive personality disorder scales

In this study, we utilized a large undergraduate sample (N = 536), oversampled for the Diagnostic and Statistical Manual of Mental Disorders (4th ed., text revision [DSM-IV-TR]; American Psychiatric Association, 2000) obsessive-compulsive personality disorder (OCPD) pathology, to compare 8 self-report measures of OCPD. No prior study has compared more than 3 measures, and the results indicate that the scales had only moderate convergent validity. We also went beyond the existing literature to compare these scales to 2 external reference points: their relationships with a well-established measure of the five-factor model of personality (FFM) and clinicians' ratings of their coverage of the DSM-IV-TR criterion set. When the FFM was used as a point of comparison, the results suggest important differences among the measures with respect to their divergent representation of conscientiousness, neuroticism, and agreeableness. Additionally, an analysis of the construct coverage indicated that the measures also varied in terms of their representation of particular diagnostic criteria. For example, whereas some scales contained items distributed across the diagnostic criteria, others were concentrated more heavily on particular features of the DSM-IV-TR disorder.

A meta-analytic review of the relationships between the five-factor model and DSM-IV-TR personality disorders: a facet level analysis

Theory and research have suggested that the personality disorders contained within the American Psychiatric Association's Diagnostic and Statistical Manual of Mental Disorders (DSM-IV-TR) can be understood as maladaptive variants of the personality traits included within the five-factor model (FFM). The current meta-analysis of FFM personality disorder research both replicated and extended the 2004 work of Saulsman and Page (The five-factor model and personality disorder empirical literature: A meta-analytic review. Clinical Psychology Review, 23, 1055-1085) through a facet level analysis that provides a more specific and nuanced description of each DSM-IV-TR personality disorder. The empirical FFM profiles generated for each personality disorder were generally congruent at the facet level with hypothesized FFM translations of the DSM-IV-TR personality disorders. However, notable exceptions to the hypotheses did occur and even some findings that were consistent with FFM theory could be said to be instrument specific.

Convergence of narcissism measures from the perspective of general personality functioning

The construct of narcissism has a lengthy history and has been operationalized and measured by a variety of instruments. In this study, five narcissism scales were compared in terms of alternative conceptualizations of narcissism offered by C. C. Morf and F. Rhodewalt (2001), D. L. Paulhus (2001), and S. Vazire and D. C. Funder (2006), using the domains and facets of the five-factor model as a common point of comparison. The findings provided little support for the conceptualizations of Morf and Rhodewalt or Vazire and Funder. Support was obtained for the conceptualization of Paulhus, particularly as assessed by the Narcissistic Personality Inventory and, secondarily, the Millon Clinical Multiaxial Inventory-III. Implications for the assessment and conceptualization of narcissism are discussed.

Psychometric properties of an abbreviated instrument of the five-factor model

Brief measures of the five-factor model (FFM) have been developed but none include an assessment of facets within each domain. The purpose of this study was to examine the validity of a simple, one-page, facet-level description of the FFM. Five data collections were completed to assess the reliability and the convergent and discriminant validity of the rating form with other measures of the FFM and to replicate correlations with measures of maladaptive personality functioning that have been obtained with more extensive measures. Results appeared to support the validity of the FFM rating form (FFMRF) because it obtained relatively good internal consistency, convergent validity, and discriminant validity. In addition, self-descriptions of persons in terms of the FFMRF related to maladaptive personality traits in a manner that was consistent with theoretical expectations. Negative findings and limitations of the rating form also are discussed.

Clinicians' judgments of clinical utility: a comparison of the DSM-IV and five-factor models

Clinical utility, or the usefulness of a diagnostic system in clinical practice, has been identified as an important construct in proposed revisions to the diagnostic nomenclature and a significant limitation of dimensional models of personality disorder, such as the 5-factor model (FFM). Only 1 study to date has addressed explicitly the clinical utility of the FFM, and the findings suggested significant limitations. In the current study, 245 practicing psychologists described 3 historic cases using both the FFM and the Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM-IV; American Psychiatric Association, 2000) and then rated each model on 6 aspects of clinical utility. In contrast to prior research, the psychologists in this study considered the FFM to have greater clinical utility than the existing diagnostic categories.

Diagnostic categories or dimensions? A question for the Diagnostic And Statistical Manual Of Mental Disorders--fifth edition

The question of whether mental disorders are discrete clinical conditions or arbitrary distinctions along dimensions of functioning is a long-standing issue, but its importance is escalating with the growing recognition of the frustrations and limitations engendered by the categorical model. The authors provide an overview of some of the dilemmas of the categorical model, followed by a discussion of research that addresses whether mental disorders are accurately or optimally classified categorically or dimensionally. The authors' intention is to document the importance of this issue and to suggest that future editions of the Diagnostic and Statistical Manual of Mental Disorders give more recognition to dimensional models of classification. They conclude with a dimensional mental disorder classification that they suggest provides a useful model.

Evidence-based assessment of personality disorders

The purpose of this article is to provide a foundation for the development of evidence-based guidelines for the assessment of personality disorders, focusing in particular on integrated assessment strategies. The general strategy recommended herein is to first administer a self-report inventory to alert oneself to the potential presence of particular maladaptive personality traits followed by a semistructured interview to verify their presence. This strategy is guided by the existing research that suggests particular strengths of self-report inventories and semistructured interviews relative to unstructured clinical interviews. However, the authors also consider research that suggests that further improvements to the existing instruments can be made. The authors emphasize, in particular, a consideration of age of onset, distortions in self-perception and presentation, gender bias, culture and ethnicity, and personality change.

Clinicians' personality descriptions of prototypic personality disorders

Many studies have indicated close convergence of the DSM-IV personality disorders and the five-factor model (FFM) of personality functioning. However, questions have been raised concerning the ability of clinicians to describe personality disorders in terms of the FFM. This study developed a FFM description by practicing clinicians of each DSM-IV personality disorder. Clinicians rated a prototypic case of each DSM-IV personality disorder in terms of the FFM. These ratings, which achieved excellent reliability, were then averaged to produce a consensus FFM profile for each personality disorder. The consensus ratings showed good agreement with previous research that examined both researchers' and clinicians' application of the FFM to prototypic cases of personality disorders. These results suggest that clinicians can conceptualize and apply the FFM to personality disorders in a consistent way. The results further suggest that the FFM may provide a richer and more comprehensive description of personality difficulties than the current DSM-IV personality disorder categories.

Maladaptive personality functioning within the big five and the five-factor model

The five-factor model (FFM) of general personality functioning was derived originally from lexical studies of trait terms within the English language. Many studies have been conducted on the relationship of the FFM to personality disorder symptomatology but, as yet, no lexical study of the representation of maladaptive personality functioning within a language has been conducted. The current study identified the distribution of socially undesirable trait terms within each of the poles of the Big Five and compared this distribution to findings obtained with FFM personality disorder measures. The implications of the results for a FFM of personality disorders and for the FFM assessment of maladaptive personality functioning are discussed.