Toward a second generation of statistical prediction rules in psychodiagnosis and personality assessment

The Meta-Analysis of Clinical Judgment Project: Fifty-Six Years of Accumulated Research on Clinical Versus Statistical Prediction

Clinical predictions made by mental health practitioners are compared with those using statistical approaches. Sixty-seven studies were identified from a comprehensive search of 56 years of research; 92 effect sizes were derived from these studies. The overall effect of clinical versus statistical prediction showed a somewhat greater accuracy for statistical methods. The most stringent sample of studies, from which 48 effect sizes were extracted, indicated a 13% increase in accuracy using statistical versus clinical methods. Several variables influenced this overall effect. Clinical and statistical prediction accuracy varied by type of prediction, the setting in which predictor data were gathered, the type of statistical formula used, and the amount of information available to the clinicians and the formulas. Recommendations are provided about when and under what conditions counseling psychologists might use statistical formulas as well as when they can rely on clinical methods. Implications for clinical judgment research and training are discussed.

The Meta-Analysis of Clinical Judgment Project

Clinical and educational experience is one of the most commonly studied variables in clinical judgment research. Contrary to clinicians' perceptions, clinical judgment researchers have generally concluded that accuracy does not improve with increased education, training, or clinical experience. In this meta-analysis, the authors synthesized results from 75 clinical judgment studies where the experience of 4,607 clinicians was assessed in relation to the accuracy of their judgments about mental health (e.g., diagnosis, prognosis, treatment) and psychological issues (e.g., vocational, personality). The authors found a small but reliable effect, d = .12, showing that experience, whether educational or clinical, is positively associated with judgment accuracy. This small effect was robust across several tested moderator models, indicating experienced counselors and clinicians acquire, in general, almost a 13% increase in their decision-making accuracy, regardless of other factors. Results are discussed in light of their implications for clinical judgment research and for counseling psychology training and practice.

Assessing violence risk among youth

Despite recent declines in the reported rate of juvenile violence, there appears to be increasing public and professional concern about violent behavior among children and adolescents. Media accounts of school shootings and juvenile homicides have prompted a need to develop approaches for systematically assessing violence risk. This article describes the task of assessing general violence risk among youth, and argues that a somewhat different approach is required to assess cases where an identified or identifiable young person may pose a risk to a specifically identified or identifiable target (also referred to as "targeted violence"). Key risk factors for violent behavior among children and adolescents are identified, fundamental principles for conducting an assessment of violence potential in clinical and juvenile justice contexts are outlined, and an approach to assessment when an identified person engages in some communication or behavior of concern that brings him or her to official attention is briefly described.

Computers will become increasingly important for psychological assessment: not that there's anything wrong with that!

Though one can expect that computer programs will become increasingly important for psychological assessment, current automated assessment programs and statistical-prediction rules are of limited value. Validity has not been clearly established for many automated assessment programs. Statistical-prediction rules are of limited value because they have typically been based on limited information that has not been demonstrated to be optimal and they have almost never been shown to be powerful. Recommendations are made for building and evaluating new computer programs. Finally, comments are made about the ethics of using computers to make judgments.

Threat assessment: defining an approach for evaluating risk of targeted violence

Although the field of risk assessment has made tremendous advances in the past 20 years, assessments of targeted violence continue to pose a significant challenge to law enforcement, mental health, and other professionals. These specific and critical assessments require an innovative approach. The threat assessment model, developed and refined by the U.S. Secret Service, provides a useful framework for thinking about assessments of potential for targeted violence. In this paper, we attempt to define this approach as it has been developed by the Secret Service, and apply it within the existing professional/scientific literature on risk assessment. We begin with a brief review of existing models and approaches in risk assessment, and identification of some gaps in our existing knowledge as it relates to assessments of targeted violence. We then proceed with an outline of the threat assessment approach, including a review of principles and guiding operational questions, and discussion of its use in assessment of targeted violence.