Reliability testing of the Health of the Nation Outcome Scales 2018

WHAT IS KNOWN ON THE SUBJECT

The Health of the Nation Outcome Scales (HoNOS) is a widely used clinical measure designed to rate and monitor the outcomes of service users accessing specialist mental healthcare. Since its development (in 1996), numerous research studies have confirmed the HoNOS captures the aspects of care that it purports to (validity), and that clinicians' ratings are consistent both over time, and between different raters (reliability).

WHAT THE PAPER ADDS TO EXISTING KNOWLEDGE

In 2018, the HoNOS was reviewed with updates made to some terminology and other revisions intended to remove ambiguity in the guidance for raters. However, although the new version (HoNOS 2018) was accompanied by a recommendation that its validity and reliability be re-tested this was not undertaken. To our knowledge, this is the first study to re-assess the updated tool's reliability by measuring the level of agreement between different raters. Our findings confirm that there is an acceptable level of consistency between student mental health nurses that have been trained to use the (new) HoNOS 2018.

WHAT ARE THE IMPLICATIONS FOR PRACTICE

The HoNOS is nationally mandated for use by all specialist mental healthcare providers in the UK. Our findings provide some assurance that, with appropriate update training and monitoring of organisational-level data sets, the original HoNOS glossary can safely be replaced with the HoNOS 2018 to ensure more contemporary routine outcome measurement can occur.

ABSTRACT

INTRODUCTION: The Health of the Nation Outcome Scales (HoNOS) is a well-established clinician rated outcome measure for use in mental health services. Following an international review, an updated version (HoNOS 2018) was published with a recommendation that its psychometric properties be re-tested prior to widespread implementation. To date, only one such study has been published.

AIMS

To test the inter-rater agreement levels for HoNOS 2018.

METHOD

Third-year student mental health nurses received training to complete the HoNOS 2018. Following this timetabled session, they were each invited to independently rate two, randomly selected, videos of (simulated) patient interviews. The resulting data were then analysed to calculate the tool's internal consistency and inter-rater agreement levels.

RESULTS

The 55 participants provided 106 ratings from four vignettes. Cronbach's alphas and McDonalds omegas confirmed the revised tool's internal consistency was acceptable. Average measure intraclass correlation coefficients for the four patient vignettes indicated excellent reliability.

IMPLICATIONS FOR PRACTICE

This study provides initial assurance that the HoNOS 2018 is a reliable clinician rated outcome measure suitable for use in routine clinical practice by relatively inexperienced mental health practitioners with limited training.

Impact of Longitudinal Mechanical Ventilation Curriculum on Decay of Knowledge

Background

Prior evidence suggests that critical care trainees and attendings may have trouble recognizing common, potentially life-threatening mechanical ventilation (MV) waveform asynchronies. Although dedicated workshops may improve knowledge in MV, this knowledge may be prone to decay over time. Longitudinal, preceptorial-based curriculums may prevent this decay in knowledge.

Objective

To determine if the addition of a year-long, longitudinal MV preceptorial curriculum to a two-part, small-group, simulation-based education block curriculum reduces decay in MV knowledge compared with the education block curriculum alone.

Methods

This was a multicenter prospective cohort study including 123 first-year fellows from 12 critical care fellowship programs who completed a two-part simulation-based education block (control) after the first and sixth months of fellowship. Fellows from one of these programs also participated in a year-long preceptorial curriculum (intervention). MV waveform examination scores over time during fellowship were compared between control versus intervention groups.

Results

Mean test scores increased for both control and intervention groups after the education block courses at Months 1 and 6 of fellowship. Mean (standard deviation) test scores at Month 12 were higher for the intervention group than the control group (89.3 [14.8] vs. 47.7 [21.4]; P < 0.0001). Between 6 months and 3 years of fellowship, there was a significant decay in test scores for the control group (slope estimate [standard error], -13.4 [1.7]; P < 0.0001). However, there was no significant decay in test scores for the intervention group (slope estimate, -2.0 [4.7]; P = 0.67; difference in slope estimates, 11.4 [5.0]; P = 0.02).

Conclusion

The ability of critical care fellows to identify MV waveform asynchronies declines over fellowship training, despite a dedicated two-part, simulation-based MV educational curriculum. The addition of an MV preceptorial course decreased decay of MV knowledge over the course of fellowship training.

Duped by Bots: Why Some are Better than Others at Detecting Fake Social Media Personas

OBJECTIVE

We examine individuals' ability to detect social bots among Twitter personas, along with participant and persona features associated with that ability.

BACKGROUND

Social media users need to distinguish bots from human users. We develop and demonstrate a methodology for assessing those abilities, with a simulated social media task.

METHOD

We analyze performance from a signal detection theory perspective, using a task that asked lay participants whether each of 50 Twitter personas was a human or social bot. We used the agreement of two machine learning models to estimate the probability of each persona being a bot. We estimated the probability of participants indicating that a persona was a bot with a generalized linear mixed-effects model using participant characteristics (social media experience, analytical reasoning, and political views) and stimulus characteristics (bot indicator score and political tone) as regressors.

RESULTS

On average, participants had modest sensitivity (d') and a criterion that favored responding "human." Exploratory analyses found greater sensitivity for participants (a) with less self-reported social media experience, (b) greater analytical reasoning ability, and (c) who were evaluating personas with opposing political views. Some patterns varied with participants' political identity.

CONCLUSIONS

Individuals have limited ability to detect social bots, with greater aversion to mistaking bots for humans than vice versa. Greater social media experience and myside bias appeared to reduce performance, as did less analytical reasoning ability.

APPLICATION

These patterns suggest the need for interventions, especially when users feel most familiar with social media.

Treatment Evaluation in Forensic Psychiatry. Which One Should Be Used: The Clinical Judgment or the Instrument-based Assessment of Change?

In forensic psychiatry, it is common practice to use an unstructured clinical judgment for treatment evaluation. From risk assessment studies, it is known that the unstructured clinical judgment is unreliable and the use of instruments is recommended. This paper aims to explore the clinical judgment of change compared to the calculated change using the Instrument for Forensic Treatment Evaluation (IFTE) in relation to changes in inpatient violence This study shows that the clinical judgment is much more positive about patient's behavioral changes than the calculated change. And that the calculated change is more in accordance with the change in the occurrence of inpatient violence, suggesting that the calculated change reflects reality closer than the unstructured clinical judgment. Therefore, it is advisable to use the IFTE as a base to make a structured professional judgment of the treatment evaluation of a forensic psychiatric patient.

A concept analysis of clinical judgment in undergraduate nursing students

AIM

To offer a clear understanding of the definition, attributes, antecedents, and consequences of undergraduate nursing students' clinical judgment in the nursing education context.

BACKGROUND

Clinical judgment is a concept with broad uses among healthcare professionals. Its definitions and attributes vary across contexts. There is no established understanding of clinical judgment in nursing students.

DESIGN

This concept analysis was performed using the eight steps of Walker and Avant's framework.

DATA SOURCES

A literature search was conducted using search engines and included peer-reviewed articles related to clinical judgment in nursing students.

RESULTS

Clinical judgment in nursing students was defined as the cognitive process exhibited via a nursing action by observation, patient assessment, interpreting, and prioritizing data that lead to responding using the appropriate nursing practice with the patient. Reflection was used to evaluate nursing students' clinical judgment to revise nursing practice. Antecedents included nursing education curriculum, student's knowledge, previous clinical experience, critical thinking, and clinical reasoning. The consequences of clinical judgment were clinical judgment ability, safe nursing practice, nursing care quality, and patient safety.

CONCLUSIONS

This concept analysis gives clear insights into the definition, attributes, antecedents, and consequences of undergraduate nursing student's clinical judgment.

Beyond an abstract and technical conception of psychotherapy: The indispensable role of practical wisdom

The work of psychotherapy is complex and seems to resist algorithmic formulation. Psychotherapy theory, technique, and research help to manage this complexity to some degree, but clinical excellence also seems to require clinical judgment, and this judgment cuts across therapeutic theories. However, the content and process of clinical judgment remain relatively untheorized, often relegated to the “art of psychotherapy.” This article focuses on elucidating clinical judgment in terms of the neo-Aristotelian concept of practical wisdom, which includes recognizing what is salient, integrating multiple considerations, infusing emotional experience with reason, and pursuing an intervention plan with discernment and flexibility. The argument is that practical wisdom provides a rich description of the work of clinically adept therapists by more clearly portraying therapists’ decisions and activities than the general, unsystematic concept of clinical judgment or in terms of the many theories of psychotherapy.

On the use of positive test strategies when diagnosing mental disorders

BACKGROUND

Despite the adverse impact diagnostic errors can have, clinical interviewing and decision-making in psychiatric practice have received relatively little empirical attention. When diagnosing patients, clinicians tend to fall back on a specific (heuristic) rule of thumb, the positive test strategy, a confirmatory approach that increases the risk of confirmation bias.

METHOD AND RESULTS

A group of 83 clinical psychologists and psychiatrists was asked to give their diagnostic hypotheses about two vignettes. We found them to self-generate significantly (i.e., p < .01; d = 1.57) more confirming than disconfirming questions to test their initial diagnostic impressions, with supervisors considering significantly more differential diagnoses than the less experienced post-grads/residents. When offered a list of 100 potentially relevant diagnostic queries, the supervisors selected fewer confirming and proportionally more disconfirming themes.

CONCLUSIONS

Our results demonstrate that irrespective of clinical experience mental-health clinicians indeed tend to use a confirmatory thinking style that contrasts with the stricter principle of falsification. More field-based research on this topic is needed, as well as studies probing whether a systematized diagnostic approach is feasible in psychiatric practice and increases diagnostic accuracy and patient satisfaction.

Applying the DSM-5 Alternative Model of Personality Disorders and the Shedler-Westen Assessment Procedure to the Classic Case of "Madeline G.": Novice and Expert Rater Convergences and Divergence

Prior research supports the learnability of the Diagnostic and Statistical Manual of Mental Disorders, 5th Edition Alternative Model of Personality Disorders (AMPD). However, researchers have yet to compare novice ratings on the AMPD's Level of Personality Functioning Scale and the 25 pathological personality traits with expert ratings. Furthermore, the AMPD has yet to be examined with the idiographic Shedler-Westen Assessment Procedure (SWAP). We compared the aggregated AMPD clinical profile of a group of psychology doctoral students who learned the AMPD to high levels of reliability to that of an expert rater using the crucible of the classical case of "Madeline G." Examination of AMPD and SWAP ratings of "Madeline G." revealed excellent overall concordance but suggests that novice raters tend to perceive lower levels of personality impairment.

Clinicians' Cognitive and Affective Biases and the Practice of Psychotherapy

OBJECTIVE

Cognitive and affective biases are essentially connected to heuristic shortcuts in thinking. These biases ordinarily function outside of conscious awareness and potentially affect clinical assessment, reasoning, and decision making in general medicine. However, little consideration has been given to how they may affect clinicians in the conduct of psychotherapy. This article aims to illustrate how such biases may affect assessment, formulation, and conduct of psychotherapy; describe strategies to mitigate these influences; and draw attention to the need for systematic research in this area.

METHODS

Cognitive and affective biases potentially influencing clinical assessment, reasoning, and decision making in medicine were identified in a selective literature review. The authors drew from their experiences as psychotherapists and psychotherapy supervisors to consider how key biases may influence psychotherapists' conduct of psychotherapy sessions.

RESULTS

The authors reached consensus in selecting illustrative biases pertinent to psychotherapy. Included biases related to anchoring, ascertainment, availability, base-rate neglect, commission, confirmation, framing, fundamental attribution error, omission, overconfidence, premature closure, sunk costs, and visceral reactions. Vignettes based on the authors' combined experiences are provided to illustrate how these biases could influence the conduct of psychotherapy.

CONCLUSIONS

Cognitive and affective biases are likely to play important roles in psychotherapy. Clinicians may reduce the potentially deleterious effects of biases by using a variety of mitigating strategies, including education about biases, reflective review, supervision, and feedback. How extensively these biases appear among psychotherapists and across types of psychotherapy and how their adverse effects may be most effectively alleviated to minimize harm deserve systematic study.

Nurses' judgments of patient risk of deterioration at change-of-shift handoff: Agreement between nurses and comparison with early warning scores

BACKGROUND

Nurses begin forming judgments regarding patients' clinical stability during change-of-shift handoffs.

OBJECTIVES

To examine the agreement between incoming and outgoing nurses' judgments of deterioration risk following handoff and compare these judgments to commonly used early warning scores (MEWS, NEWS, ViEWS).

METHODS

Following handoffs on three medical/surgical units, nurses completed the Patient Acuity Rating. Nurse ratings were compared with computed early warning scores based on clinical data. In follow-up interviews, nurses were invited to describe their experiences of using the rating scale.

RESULTS

Sixty-two nurses carried out 444 handoffs for 158 patients. While the agreement between incoming and outgoing nurses was fair, correlations with early warning scores were low. Nurses struggled with predicting risk and used their impressions of differential risk across all the patients to whom they had been assigned to arrive at their ratings.

CONCLUSION

Nurses shared information that influenced their clinical judgments at handoff; not all of these cues may necessarily be captured in early warning scores.

Shared decision-making in mental and behavioural health interventions

Elements of shared decision-making (ie, collaboration, patient preferences, and working alliance) have long been discussed and studied in the field of clinical psychology; however, research indicates that shared decision-making is not typically used in clinical practice. Instead, clinicians often rely on a paternalistic approach. In this article, we provide a narrative review of the existing research supporting shared decision-making for mental and behavioural health concerns, we discuss several barriers that impede its use in actual clinical practice, and we provide recommendations for increasing shared decision-making when working with patients.

Machine learning vs addiction therapists: A pilot study predicting alcohol dependence treatment outcome from patient data in behavior therapy with adjunctive medication

BACKGROUND AND OBJECTIVES

Clinical staff providing addiction treatment predict patient outcome poorly. Prognoses based on linear statistics are rarely replicated. Addiction is a complex non-linear behavior. Incorporating non-linear models, Machine Learning (ML) has successfully predicted treatment outcome when applied in other areas of medicine. Using identical assessment data across the two groups, this study compares the accuracy of ML models versus clinical staff to predict alcohol dependence treatment outcome in behavior therapy using patient data only.

METHODS

Machine learning models (n = 28) were constructed ('trained') using demographic and psychometric assessment data from 780 previously treated patients who had undertaken a 12 week, abstinence-based Cognitive Behavioral Therapy program for alcohol dependence. Independent predictions applying assessment data for an additional 50 consecutive patients were obtained from 10 experienced addiction therapists and the 28 trained ML models. The predictive accuracy of the ML models and the addiction therapists was then compared with further investigation of the 10 best models selected by cross-validated accuracy on the training-set. Variables selected as important for prediction by staff and the most accurate ML model were examined.

RESULTS

The most accurate ML model (Fuzzy Unordered Rule Induction Algorithm, 74%) was significantly more accurate than the four least accurate clinical staff (51%-40%). However, the robustness of this finding may be limited by the moderate area under the receiver operator curve (AUC = 0.49). There was no significant difference in mean aggregate predictive accuracy between 10 clinical staff (56.1%) and the 28 best models (58.57%). Addiction therapists favoured demographic and consumption variables compared with the ML model using more questionnaire subscales.

CONCLUSIONS

The majority of staff and ML models were not more accurate than suggested by chance. However, the best performing prediction models may provide useful adjunctive information to standard clinically available prognostic data to more effectively target treatment approaches in clinical settings.

Educators’ Perceptions of Clinical Judgment Skill Competencies in Rehabilitation Counseling

Purpose:

To address a significant gap in the clinical judgment competency research by adding new knowledge of important clinical judgment skill competencies in rehabilitation counseling.

Method:

This Internet-based survey design is a follow-up inquiry to Austin and Leahy’s (2015) instrument validation study; this same sample of rehabilitation counselor educators (n = 126) rated the importance and student preparedness in using clinical judgment skill competencies (i.e., scientific attitude, cultural bias, cognitive complexity, memory bias, confirmatory bias, negative bias, evidence-based practice [EBP]).

Results:

Clinical judgment skills were perceived to be highly important. Students were rated as least prepared in scientific attitude and evidence-based practice. Ten skills of high importance/limited student preparation across four clinical judgment skill areas were identified.

Conclusions:

This study’s findings provide initial empirical support of important clinical judgment skill competencies for effective rehabilitation counseling practice. Identified student preparation gaps may be used to help prioritize potential clinical training needs for rehabilitation counseling programs to prepare students in the use of clinical judgment skill competencies that address cultural bias, cognitive complexity, confirmatory bias, and evidence-based practice. Most importantly, data generated from this study can be used when preparing students to effectively address their biases and improve their clinical judgments when applying EBP.

Reasons for placement decisions in a case of suspected child abuse: The role of reasoning, work experience and attitudes in decision-making

Child welfare and child protection workers regularly make placement decisions in child abuse cases, but how they reach these decisions is not well understood. This study focuses on workers' rationales. The aim was to investigate the kinds of arguments provided in placement decisions and whether these arguments were predictors for the decision, in addition to the decision-makers' risk assessment, work experience and attitudes towards placement. The sample consisted of 214 professionals and 381 students from the Netherlands. The participants were presented with a vignette describing a case of alleged child abuse and were asked to determine whether the abuse was substantiated, to assess risks and to recommend an intervention. The participants' placement attitudes were assessed using a structured questionnaire. We found that the participants provided a wide range of arguments, but that core arguments - such as the suspected abuse, parenting and parent-child interaction - were often missing. Regression analyses showed that the higher the perceived danger to the child and the more positive the participants' attitudes towards placement, the more likely the participants would be to propose placing the child in care. Arguments related to the severity of the problems (i.e., suspected abuse, parenting and the child's development) as well as the parents' perceived cooperation also influenced placement decisions. The findings indicate trends in the decision-making process, in the sense that participants who decided to place the child out-of-home emphasized different arguments and had different attitudes towards out-of-home placement than those who did not. We discuss the implications of our findings.

A Meta-Analysis of the Correlates of Educational and Vocational Goals for Latina/o Students

Among Latina/o Americans, the fastest growing segment of the U.S. population, disparities exist in educational and career attainment. The emergence of research on Latina/o students has resulted in varied findings that make it difficult to draw conclusions as to which predictors are most closely associated with goal outcomes for this population. Using meta-analytic techniques, the current study examined the magnitude, direction, and heterogeneity of effect sizes across multiple educational and vocational goal outcomes for Latina/o students (i.e., educational/vocational aspirations, expectations, goals/plans, and persistence). Across 34 independent samples from 33 studies, career-related self-efficacy, peer support, adult support, barriers, and Anglo acculturation had moderate effects on at least one of the outcomes, with career-related self-efficacy and peer support being moderately related to two or more outcomes. If replicated, this research suggests that interventions focused on career-related self-efficacy and peer support could enhance the development of educational and career goals for Latina/o students.

The Conceptual Framework of Thematic Mapping in Case Conceptualization

This article, the 3rd in a series of 5, introduces the conceptual framework for thematic mapping, a novel approach to case conceptualization. The framework is transtheoretical in that it is not constrained by the tenets or concepts of any one therapeutic orientation and transdiagnostic in that it conceptualizes clients outside the constraints of diagnostic criteria. Thematic mapping comprises 4 components: a definition, foundational principles, defining features, and core concepts. These components of the framework, deemed building blocks, are explained in this article. Like the foundation of any structure, the heuristic value of the method requires that the building blocks have integrity, coherence, and sound anchoring. We assert that the conceptual framework provides a solid foundation, making thematic mapping a potential asset in mental health treatment.

Case Mis-Conceptualization in Psychological Treatment: An Enduring Clinical Problem

Case conceptualization, an integral component of mental health treatment, aims to facilitate therapeutic gains by formulating a clear picture of a client's psychological presentation. However, despite numerous attempts to improve this clinical activity, it remains unclear how well existing methods achieve their purported purpose. Case formulation is inconsistently defined in the literature and implemented in practice, with many methods varying in complexity, theoretical grounding, and empirical support. In addition, many of the methods demand a precise clinical acumen that is easily influenced by judgmental and inferential errors. These errors occur regardless of clinicians' level of training or amount of clinical experience. Overall, the lack of a consensus definition, a diversity of methods, and susceptibility of clinicians to errors are manifestations of the state of crisis in case conceptualization. This article, the 2nd in a series of 5 on thematic mapping, argues the need for more reliable and valid models of case conceptualization.

Neonatology faculty development using simulation

The goal of faculty development activities is to supply the public with knowledgeable, skilled, and competent physicians who are prepared for high performance in the dynamic and complex healthcare environment. Current faculty development programs lack evidence-based support and are not sufficient to meet the professional needs of practicing physicians. Simulation activities for faculty development offer an alternative to traditional, teacher-centric educational offerings. Grounded in adult learning theory, simulation is a learner-centric, interactive, efficient, and effective method to train busy professionals. Many of the faculty development needs of clinical neonatologists can be met by participating in simulation-based activities that focus on technical skills, teamwork, leadership, communication, and patient safety.

Acquisition and maintenance of medical expertise: a perspective from the expert-performance approach with deliberate practice

As a part of a special collection in this issue of Academic Medicine, which is focused on mastery learning in medical education, this Perspective describes how the expert-performance approach with deliberate practice is consistent with many characteristics of mastery learning. Importantly, this Perspective also explains how the expert-performance approach provides a very different perspective on the acquisition of skill. Whereas traditional education with mastery learning focuses on having students attain an adequate level of performance that is based on goals set by the existing curricula, the expert-performance approach takes an empirical approach and first identifies the final goal of training-namely, reproducibly superior objective performance (superior patient outcomes) for individuals in particular medical specialties. Analyzing this superior complex performance reveals three types of mental representations that permit expert performers to plan, execute, and monitor their own performance. By reviewing research on medical performance and education, the author describes evidence for these representations and their development within the expert-performance framework. He uses the research to generate suggestions for improved training of medical students and professionals. Two strategies-designing learning environments with libraries of cases and creating opportunities for individualized teacher-guided training-should enable motivated individuals to acquire a full set of refined mental representations. Providing the right resources to support the expert-performance approach will allow such individuals to become self-regulated learners-that is, members of the medical community who have the tools to improve their own and their team members' performances throughout their entire professional careers.

Experience and Diagnostic Anchors in Referral Letters

Abstract. The present study investigated whether diagnostic anchors, that is: diagnoses suggested in referral letters, influence judgments made by clinical psychologists with different levels of experience. Moderately experienced clinicians (N = 98) and very experienced clinicians (n = 126) were randomly assigned to reading a referral letter suggesting either depression or anxiety, or no referral letter. They then read a psychiatric report about a depressed patient, and gave a preliminary and final diagnosis. Results showed that the correctness of the diagnoses by very experienced clinicians was unaffected by the referral diagnosis. Moderately experienced clinicians did use the suggested diagnosis as anchor; when they had read a referral letter suggesting depressive complaints they were more inclined to classify the patient with a depressive disorder. In conclusion, the diagnosis in a referral letter influences the diagnostic decision made by moderately experienced clinicians.

Why Ineffective Psychotherapies Appear to Work: A Taxonomy of Causes of Spurious Therapeutic Effectiveness

The past 40 years have generated numerous insights regarding errors in human reasoning. Arguably, clinical practice is the domain of applied psychology in which acknowledging and mitigating these errors is most crucial. We address one such set of errors here, namely, the tendency of some psychologists and other mental health professionals to assume that they can rely on informal clinical observations to infer whether treatments are effective. We delineate four broad, underlying cognitive impediments to accurately evaluating improvement in psychotherapy-naive realism, confirmation bias, illusory causation, and the illusion of control. We then describe 26 causes of spurious therapeutic effectiveness (CSTEs), organized into a taxonomy of three overarching categories: (a) the perception of client change in its actual absence, (b) misinterpretations of actual client change stemming from extratherapeutic factors, and (c) misinterpretations of actual client change stemming from nonspecific treatment factors. These inferential errors can lead clinicians, clients, and researchers to misperceive useless or even harmful psychotherapies as effective. We (a) examine how methodological safeguards help to control for different CSTEs, (b) delineate fruitful directions for research on CSTEs, and (c) consider the implications of CSTEs for everyday clinical practice. An enhanced appreciation of the inferential problems posed by CSTEs may narrow the science-practice gap and foster a heightened appreciation of the need for the methodological safeguards afforded by evidence-based practice.

Child sexual abuse research knowledge among child abuse professionals and laypersons

The purposes of this study were to: (1) Assess child abuse professionals' and nonprofessionals' knowledge of scientific research findings that are relevant to forensic child sexual abuse (CSA) evaluations and (2) describe associations between child abuse professionals' levels of research knowledge and their education and experience. An 18-item multiple-choice test was administered to 188 child abuse professionals and 457 nonprofessionals (undergraduate college students) in Brazil and the United States. The nonprofessionals' average percent correct, M=44%, was not significantly different than what would be expected for random guessing (45%). The professionals' average percent correct, M=55%, was higher than that of nonprofessionals and random guessing (both ps<.001). The average percent correct score for the US-sample psychologists, M=76%, was higher than the average score of the other professionals, M=51%, p<.001. Professionals' educational level, as measured by the highest academic degree obtained, was positively associated with percent correct scores, Spearman's ρ=.46, p<.001. Controlling for educational attainment, professional experience, as measured by the total number of CSA evaluations performed, was weakly associated with percent correct scores, partial r=.15, p=.04. Percent correct scores were low for both nonprofessionals and professionals. Most of the participants in this study were uninformed or misinformed about scientific research findings that are important for conducting optimal forensic CSA evaluations and for making accurate judgments about the validity of sexual abuse allegations.

Individual Psychological Assessment: A Practice and Science in Search of Common Ground

During the past 30 years, individual psychological assessment (IPA) has gained in use and in value to organizations in the management of human resources. However, even though IPA is considered a core competency for industrial–organizational (I–O) psychology, its practice is not without critics. This article is written not only to address several criticisms of IPA but also to discuss a variety of issues that must be taken into consideration if IPA is to advance as a major component of the I–O scientist–practitioner model. We rely upon a working definition of IPA in general but, when possible, focus on executive assessment in particular, given its high level of complexity and growing popularity. We discuss the effectiveness of assessment practice, including the ongoing statistical versus clinical prediction argument and the difficulties with establishing validity. Although we are confident that IPA has many strong research and practice underpinnings, we also propose some important research questions, training guidelines, and opportunities for assessing psychologists to improve their practices.

Training for suicide risk assessment and suicide risk formulation

Suicide and suicidal behaviors are highly associated with psychiatric disorders. Psychiatrists have significant opportunities to identify at-risk individuals and offer treatment to reduce that risk. Although a suicide risk assessment (SRA) is a core competency requirement, many lack the requisite training and skills to appropriately assess for suicide risk. Moreover, the standard of care requires psychiatrists to foresee the possibility that a patient might engage in suicidal behavior, hence to conduct a suicide risk formulation (SRF) sufficient to guide triage and treatment planning. An SRA gathers data about observable and reported symptoms, behaviors, and historical factors that are associated with suicide risk and protection, ascertained by way of psychiatric interview; collateral information from family, friends, and medical records; and psychometric scales and/or screening tools. Based on data collected via an SRA, an SRF is a process whereby the psychiatrist forms a judgment about a patient's foreseeable risk of suicidal behavior in order to inform triage decisions, safety and treatment plans, and interventions to reduce risk. This paper addresses the need for a revised training model in SRA and SRF, and proposes a model of training that incorporates the acquisition of skills, relying heavily on case application exercises.

Suicide risk assessment and risk formulation part II: Suicide risk formulation and the determination of levels of risk

The suicide risk formulation (SRF) is dependent on the data gathered in the suicide risk assessment. The SRF assigns a level of suicide risk that is intended to inform decisions about triage, treatment, management, and preventive interventions. However, there is little published about how to stratify and formulate suicide risk, what are the criteria for assigning levels of risk, and how triage and treatment decisions are correlated with levels of risk. The salient clinical issues that define an SRF are reviewed and modeling is suggested for an SRF that might guide clinical researchers toward the refinement of an SRF process.

Jurors' views on the value and objectivity of mental health experts testifying in sexually violent predator trials

Although psychologists and psychiatrists often testify in court, we know relatively little about the extent to which jurors value the testimony they hear from these experts. We surveyed 161 jurors who rendered opinions in 14 sex offender civil commitment trials after hearing testimony from psychologists and psychiatrists serving as expert witnesses. Most jurors reported that the experts they heard testify were honest, and they tended to attribute disagreements among experts to case complexity, as opposed to adversarial allegiance or bias. Most reported that hearing from the experts helped them make better decisions and that experts using risk assessment instruments could make more accurate predictions than those who did not. Jurors were, however, more skeptical about the ability of experts to accurately predict recidivism when they heard testimony from both prosecution and defense experts. Findings suggest that jurors value risk assessment testimony from experts, but that experts must think carefully about how to best make risk assessment instrument results accessible to jurors.

Examining the scope of questionable diagnostic reliability in Sexually Violent Predator (SVP) evaluations

Research has suggested questionable reliability of diagnosing mental abnormality during Sexually Violent Predator (SVP) evaluations, despite this being a necessary requirement for SVP commitment. Findings have been inconsistent across studies, and little is known about the extent of such trends across diagnoses and clinicians. The current study includes data from 375 sex offenders referred for evaluation for SVP commitment in New Jersey. Clinicians (n = 128) rendered a variety of diagnoses, most commonly Pedophilia. Results suggested questionable agreement across paraphilic and non-paraphilic diagnoses, although agreement was fair for diagnoses of Pedophilia. Further analyses of cases (n = 49) involving clinicians receiving a large number of referrals (n = 14) were generally consistent with these findings, with no outlier effect apparent. Findings suggest questionable diagnostic reliability to be a widespread issue in SVP evaluations, present across a variety of diagnoses and across the general body of clinicians involved in evaluations.

Remembering and diagnosing clients: does experience matter?

Experienced mental health clinicians often do not outperform novices in diagnostic decision making. In this paper we look for an explanation of this phenomenon by testing differences in memory processes. In two studies we aimed to look at differences in accuracy of diagnoses in relation to free recall of client information between mental health clinicians with different levels of experience. Clinicians were presented with two cases, and were asked afterwards, either directly (Study 1) or after 1 week (Study 2), to give the appropriate diagnoses and to write down what they remembered of the cases. We found in Study 1 that the accuracy of the diagnoses was the same for all levels of experience, as was the amount of details recalled. Very experienced clinicians did remember more higher-order concepts, that is, abstractions from the presented information. In Study 2 we found that the very experienced clinicians were less accurate in their diagnoses and remembered fewer details than the novices. In response to these findings we further discuss their implications for psychodiagnostic practice.

Is there evidence of publication biases in JDM research?

It is a long known problem that the preferential publication of statistically significant results (publication bias) may lead to incorrect estimates of the true effects being investigated. Even though other research areas (e.g., medicine, biology) are aware of the problem, and have identified strong publication biases, researchers in judgment and decision making (JDM) largely ignore it. We reanalyzed two current meta-analyses in this area. Both showed evidence of publication biases that may have led to a substantial overestimation of the true effects they investigated. A review of additional JDM meta-analyses shows that most meta-analyses conducted no or insufficient analyses of publication bias. However, given our results and the rareness of non-significant effects in the literature, we suspect that biases occur quite often. These findings suggest that (a) conclusions based on meta-analyses without reported tests of publication bias should be interpreted with caution and (b) publication policies and standard research practices should be revised to overcome the problem.

Relationship between patient characteristics and treatment allocation for patients with personality disorders

Within a large multi-center study in patients with personality disorders, we investigated the relationship between patient characteristics and treatment allocation. Personality pathology, symptom distress, treatment history, motivational factors, and sociodemographics were measured at intake in 923 patients, who subsequently enrolled in short-term or long-term outpatient, day hospital, or inpatient psychotherapy for personality pathology. Logistic regressions were used to examine the predictors of allocation decisions. We found a moderate relationship (R(2) = 0.36) between patient characteristics and treatment setting, and a weak relationship (R(2) = 0.18) between patient characteristics and treatment duration. The most prominent predictors for setting were: symptom distress, cluster C personality pathology, level of identity integration, treatment history, motivation, and parental responsibility. For duration the most prominent predictor was age. We conclude from this study that, in addition to pathology and motivation factors, sociodemographics and treatment history are related to treatment allocation in clinical practice.

Individual Psychological Assessment: A Core Competency for Industrial–Organizational Psychology

Individual Psychological Assessment is widely practiced among I–O psychologists and supported as a core competency by a majority of the commentaries. Our response addresses some of the key points made in the commentaries, such as issues related to the definition, contextual and criterion variables, interpretation and integration methods, validity evidence, professional matters, and future research directions. We also respond to a few critics who want to entirely eliminate expert assessor judgment in IPAs. Models are presented for using assessor judgment and mechanical methods in IPAs and for distinguishing naïve and sophisticated IPA researchers and practitioners. We conclude that IPA is now widely accepted as a core part of I–O psychology.

Clinical judgement and the medical profession

OBJECTIVES

Clinical judgment is a central element of the medical profession, essential for the performance of the doctor, and potentially generating information also for other clinicians and for scientists and health care managers. The recently renewed interest in clinical judgement is primarily engaged with its role in communication, diagnosis and decision making. Beyond this issue, the present article highlights the interrelations between clinical judgement, therapy assessment and medical professionalism.

METHODS

Literature review and theory development.

RESULTS

The article presents different methodological approaches to causality assessment in clinical studies and in clinical judgement, and offers criteria for clinical single case causality. The article outlines models of medical professionalism such as technical rationality and practice epistemology, and characterizes features of professional expertise such as tacit knowledge, reflection in action, and gestalt cognition.

CONCLUSIONS

Consequences of a methodological and logistical advancement of clinical judgment are discussed, both in regard to medical progress and to the renewal of the cognitive basis of the medical profession.