Probability and loss: two sides of the risk assessment coin

Calibrating violence risk assessments for uncertainty

Psychiatrists and other mental health clinicians are often tasked with assessing patients' risk of violence. Approaches to this vary and include both unstructured (based on individual clinicians' judgement) and structured methods (based on formalised scoring and algorithms with varying scope for clinicians' judgement). The end result is usually a categorisation of risk, which may, in turn, reference a probability estimate of violence over a certain time period. Research over recent decades has made considerable improvements in refining structured approaches and categorising patients' risk classifications at a group level. The ability, however, to apply these findings clinically to predict the outcomes of individual patients remains contested. In this article, we review methods of assessing violence risk and empirical findings on their predictive validity. We note, in particular, limitations in calibration (accuracy at predicting absolute risk) as distinct from discrimination (accuracy at separating patients by outcome). We also consider clinical applications of these findings, including challenges applying statistics to individual patients, and broader conceptual issues in distinguishing risk and uncertainty. Based on this, we argue that there remain significant limits to assessing violence risk for individuals and that this requires careful consideration in clinical and legal contexts.

Validity and reliability of the novel three‐item occupational violence patient risk assessment tool

Aim

To develop and psychometrically test an occupational violence (OV) risk assessment tool in the emergency department (ED).

Design

Three studies were conducted in phases: content validity, predictive validity and inter‐rater reliability from June 2019 to March 2021.

Methods

For content validity, ED end users (mainly nurses) were recruited to rate items that would appropriately assess for OV risk. Subsequently, a risk assessment tool was developed and tested for its predictive validity and inter‐rater reliability. For predictive validity, triage notes of ED presentations in a month with the highest OV were assessed for presence of OV risk. Each presentation was then matched with events recorded in the OV incident register. Sensitivity and specificity values were calculated. For inter‐rater reliability, two assessors—trained and untrained—independently assessed the triage notes for presence of OV risk. Cohen's kappa was calculated.

Results

Two rounds of content validity with a total of N = 81 end users led to the development of a three‐domain tool that assesses for OV risk using aggression history, behavioural concerns (i.e., angry, clenched fist, demanding, threatening language or resisting care) and clinical presentation concerns (i.e., alcohol/drug intoxication and erratic cognition). Recommended risk ratings are low (score = 0 risk domain present), moderate (score = 1 risk domain present) and high (score = 2–3 risk domains present), with an area under the curve of 0.77 (95% confidence interval 0.7–0.81, p < .01). Moderate risk rating had a 61% sensitivity and 91% specificity, whereas high risk rating had 37% sensitivity and 97% specificity. Inter‐rater reliability ranged from 0.67 to 0.75 (p < .01), suggesting moderate agreement.

Conclusions

The novel three‐domain OV risk assessment tool was shown to be appropriate and relevant for application in EDs. The tool, developed through a rigorous content validity process, demonstrates acceptable predictive validity and inter‐rater reliability.

Impact

The developed tool is currently piloted in a single hospital ED, with a view to extend to inpatient settings and other hospitals.

Engaging with nurses to develop an occupational violence risk assessment tool for use in emergency departments: A participatory action research inquiry

BACKGROUND

Early detection of occupational violence (OV) risk factors could facilitate timely and appropriate management of patients in the emergency department. For this purpose, an OV risk assessment could be useful and best initiated at triage.

AIMS

To understand the need for and determine potential utility and desirable components of an OV risk assessment tool; and to determine specific challenges to its implementation if appropriate.

METHODS

A participatory action research was conducted. Data were collected through focus groups and semi-structured interviews. Thematic analysis was done inductively and collaboratively using Braun and Clarke's technique.

FINDINGS

Six themes were identified from triage nurses (N = 15) pertaining to: i) OV risk assessment; ii) communication of OV risk; iii) clinical implications of risk assessment; iv) tool attributes; v) future implementation challenges; vi) unintended consequences.

CONCLUSION

The development of an OV risk assessment tool is supported, but with very specific attributes. Findings herein also have implications on the implementation and evaluation of this tool in emergency settings.

Stalking risks to celebrities and public figures

Prominent figures are frequently subjected to unwanted and intrusive attentions. Such stalking behaviour is often driven by psychotic illness, angrily blaming the public figure for delusional persecution (resentful motivation), or based on erotomanic delusions (intimacy seeking motivation), for example. This behaviour can cause psychological harm to both perpetrator and victim, and is unlawful. In the rare instances where a public figure has been attacked, the perpetrator has usually had a history of such stalking behaviour and of severe mental illness. For these reasons, early identification and diversion into appropriate care and treatment will be for the benefit of both parties and will prevent more serious violence in a minority of cases. The importance of the provision of education to improve both reporting rates by victims and an appropriate response from the criminal justice system is highlighted. A multi-agency approach involving the criminal justice system and mental health services is the most effective means of achieving these aims.

DECLARATION OF INTEREST

None.

LEARNING OBJECTIVES

•

Which are the most useful scales for predicting repeat self-harm? A systematic review evaluating risk scales using measures of diagnostic accuracy

OBJECTIVES

The aims of this review were to calculate the diagnostic accuracy statistics of risk scales following self-harm and consider which might be the most useful scales in clinical practice.

DESIGN

Systematic review.

METHODS

We based our search terms on those used in the systematic reviews carried out for the National Institute for Health and Care Excellence self-harm guidelines (2012) and evidence update (2013), and updated the searches through to February 2015 (CINAHL, EMBASE, MEDLINE, and PsychINFO). Methodological quality was assessed and three reviewers extracted data independently. We limited our analysis to cohort studies in adults using the outcome of repeat self-harm or attempted suicide. We calculated diagnostic accuracy statistics including measures of global accuracy. Statistical pooling was not possible due to heterogeneity.

RESULTS

The eight papers included in the final analysis varied widely according to methodological quality and the content of scales employed. Overall, sensitivity of scales ranged from 6% (95% CI 5% to 6%) to 97% (CI 95% 94% to 98%). The positive predictive value (PPV) ranged from 5% (95% CI 3% to 9%) to 84% (95% CI 80% to 87%). The diagnostic OR ranged from 1.01 (95% CI 0.434 to 2.5) to 16.3 (95%CI 12.5 to 21.4). Scales with high sensitivity tended to have low PPVs.

CONCLUSIONS

It is difficult to be certain which, if any, are the most useful scales for self-harm risk assessment. No scales perform sufficiently well so as to be recommended for routine clinical use. Further robust prospective studies are warranted to evaluate risk scales following an episode of self-harm. Diagnostic accuracy statistics should be considered in relation to the specific service needs, and scales should only be used as an adjunct to assessment.

Identifying Causal Risk Factors for Violence among Discharged Patients

BACKGROUND

Structured Professional Judgement (SPJ) is routinely administered in mental health and criminal justice settings but cannot identify violence risk above moderate accuracy. There is no current evidence that violence can be prevented using SPJ. This may be explained by routine application of predictive instead of causal statistical models when standardising SPJ instruments.

METHODS

We carried out a prospective cohort study of 409 male and female patients discharged from medium secure services in England and Wales to the community. Measures were taken at baseline (pre-discharge), 6 and 12 months post-discharge using the Historical, Clinical and Risk-20 items version 3 (HCR-20v3) and Structural Assessment of Protective Factors (SAPROF). Information on violence was obtained via the McArthur community violence instrument and the Police National Computer.

RESULTS

In a lagged model, HCR-20v3 and SAPROF items were poor predictors of violence. Eight items of the HCR-20v3 and 4 SAPROF items did not predict violent behaviour better than chance. In re-analyses considering temporal proximity of risk/ protective factors (exposure) on violence (outcome), risk was elevated due to violent ideation (OR 6.98, 95% CI 13.85-12.65, P<0.001), instability (OR 5.41, 95% CI 3.44-8.50, P<0.001), and poor coping/ stress (OR 8.35, 95% CI 4.21-16.57, P<0.001). All 3 risk factors were explanatory variables which drove the association with violent outcome. Self-control (OR 0.13, 95% CI 0.08-0.24, P<0.001) conveyed protective effects and explained the association of other protective factors with violence.

CONCLUSIONS

Using two standardised SPJ instruments, predictive (lagged) methods could not identify risk and protective factors which must be targeted in interventions for discharged patients with severe mental illness. Predictive methods should be abandoned if the aim is to progress from risk assessment to effective risk management and replaced by methods which identify factors causally associated with violence.

Impulsive versus premeditated aggression in the prediction of violent criminal recidivism

Past aggression is a potent predictor of future aggression and informs the prediction of violent criminal recidivism. However, aggression is a heterogeneous construct and different types of aggression may confer different levels of risk for future violence. In this prospective study of 91 adults in a pretrial diversion program, we examined (a) premeditated versus impulsive aggression in the prediction of violent recidivism during a one-year follow-up period, and (b) whether either type of aggression would have incremental validity in the prediction of violent recidivism after taking into account frequency of past general aggression. Findings indicate that premeditated, but not impulsive, aggression predicts violent recidivism. Moreover, premeditated aggression remained a predictor of recidivism even with general aggression frequency in the model. Results provide preliminary evidence that the assessment of premeditated aggression provides relevant information for the management of violent offenders.