Evidence-Based Performance Management: Applying Behavioral Science to Support Practitioners

Behavioral skills training for teaching safety skills to mental health service providers compared to training-as-usual: a pragmatic randomized control trial

BACKGROUND

Violence in the healthcare workplace has been a global concern for over two decades, with a high prevalence of violence towards healthcare workers reported. Workplace violence has become a healthcare quality indicator and embedded in quality improvement initiatives of many healthcare organizations. The Centre for Addiction and Mental Health (CAMH), Canada's largest mental health hospital, provides all clinical staff with mandated staff safety training for self-protection and team-control skills. These skills are to be used as a last resort when a patient is at imminent risk of harm to self or others. The purpose of this study is to compare the effectiveness of two training methods of this mandated staff safety training for workplace violence in a large psychiatric hospital setting.

METHODS

Using a pragmatic randomized control trial design, this study compares two approaches to teaching safety skills CAMH's training-as-usual (TAU) using the 3D approach (description, demonstration and doing) and behavioural skills training (BST), from the field of applied behaviour analysis, using instruction, modeling, practice and feedback loop. Staff were assessed on three outcome measures (competency, mastery and confidence), across three time points: before training (baseline), immediately after training (post-training) and one month later (follow-up). This study was registered with the ISRCTN registry on 06/09/2023 (ISRCTN18133140).

RESULTS

With a sample size of 99 new staff, results indicate that BST was significantly better than TAU in improving observed performance of self-protection and team-control skills. Both methods were associated with improved skills and confidence. However, there was a decrease in skill performance levels at the one-month follow-up for both methods, with BST remaining higher than TAU scores across all three time points. The impact of training improved staff confidence in both training methods and remained high across all three time points.

CONCLUSIONS

The study findings suggest that BST is more effective than TAU in improving safety skills among healthcare workers. However, the retention of skills over time remains a concern, and therefore a single training session without on-the-job-feedback or booster sessions based on objective assessments of skill may not be sufficient. Further research is needed to confirm and expand upon these findings in different settings.

A Survey of Staff Training and Performance Management Practices: An Update

The field of behavior analysis has experienced marked growth in the number of credentialed professionals over the last decade. This growth may have implications for the quality of staff training, performance management, and supervision practices provided in human service settings. The purpose of this survey was to extend DiGennaro Reed and Henley (2015) by surveying credentialed and aspiring behavior analytic professionals on the staff training, performance management, and supervision practices available at their current place of employment. Three main differences were observed in relation to the findings of DiGennaro Reed and Henley. The current findings indicate notable changes in the demographic characteristics of survey respondents. In addition, we observed modest increases in the use of best practices for initial and ongoing training and performance management. Results also indicate several areas of concern regarding the provision of supervisory skills training. • Results revealed improvements in the percentage of respondents who received initial or preservice training compared to DiGennaro Reed and Henley (2015). However, employers primarily rely on instructions and modeling to train their employees. • Findings revealed greater reliance on asynchronous and synchronous online training modalities compared to DiGennaro Reed and Henley (2015). • Results revealed a slight shift in the percentage of respondents who reported receiving ongoing training compared to DiGennaro Reed and Henley (2015). In the present study, BCaBAs and RBTs generally received ongoing training; however, fewer BCBAs reported receiving ongoing training. • The reported use of performance management practices improved compared to DiGennaro Reed and Henley (2015). That is, a higher percentage of respondents reported being observed at work while carrying out their job responsibilities. • Of the respondents who supervise staff, less than half of them reported receiving supervisory skills training. And only half of those respondents reported that their training prepared them to supervise others.

The Effects of Email Performance-Based Feedback on Teacher Candidate Implementation of a Multiple Stimulus Without Replacement Preference Assessment

The purpose of this study was to evaluate the effects of performance-based feedback delivered via email on teacher candidates' implementation of a multiple stimulus without replacement (MSWO) preference assessment with an adult confederate. Six participants enrolled in a special education teacher preparation program were recruited to participate. After exposure to an introductory MSWO training video, participant fidelity of MSWO administration was measured. Following baseline, performance feedback was delivered via e-mail to each participant after the completion of each research session. Results indicate that performance feedback delivered via e-mail improved fidelity of implementation for all six participants. Implications and future directions are discussed.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10864-022-09496-z.

Effects of a Systems-Level Intervention to Improve Trainer Integrity in a Behavioral Healthcare Organization

OBJECTIVES

Direct support professionals (DSPs) play a critical role in health-related outcomes for individuals with intellectual and developmental disabilities (IDD) who reside in congregate living settings. Large behavioral healthcare organizations often rely on staff to function as peer trainers for newly hired DSPs. Organizations should adopt empirically supported training techniques to prepare peer trainers for their role and develop systems to ensure ongoing integrity of the training system. The purpose of this program description is to summarize consultation activities that attempted to create these systems.

METHODS

Staff members were trained to function as peer trainers, an assessment was conducted to determine the barriers to training in the natural environment, and a systems-level intervention informed by the assessment was implemented to improve peer trainer integrity.

RESULTS

The assessment revealed peer trainers were often unaware when they were expected to train and did not receive feedback or programmed consequences for training newly hired DSPs. A systems-level intervention containing a prompt (reminder) about upcoming training and feedback plus a monetary incentive produced improvements in trainer integrity.

CONCLUSIONS

A systems-level intervention based on an assessment can improve peer trainer integrity. Ensuring peer trainer integrity increases the likelihood that newly hired DSPs will implement health-related protocols with individuals with IDD.

Planning Positive Reinforcement Cycles in Behavior Intervention Plans

Behavior analysts partner with educators in schools to support the creation of behavior intervention plans (BIPs). Assessment and intervention planning often focuses on the relational contingencies between the student and their environment, with little attention paid to the relational contingencies contacted by the educator. In this article, we posit that planning should simultaneously include contingencies for both the student and the educator as BIPs are created. Specifically, we aim to explore a dual-pathway intervention plan in which student and educator access to reinforcement is simultaneously designed to increase both educators' implementation of high-quality instruction and students' engagement and performance. Procedural steps outlining the duality of intervention planning for both the student and the educator, as well as a theoretical model for considering contextual and reinforcement contingencies for both parties, will be detailed in a step-by-step guide to support readers' creation and implementation of plans to support improved educator and student performance. Planning for supporting both the educator and student may increase sustained, high-quality instruction and improved student outcomes for students with behavioral support needs.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40617-021-00663-8.

The Performance Diagnostic Checklist - Human Services: Guidance for Assessment Administration

The Performance Diagnostic Checklist - Human Services (PDC-HS) is an assessment designed to assess the environmental variables contributing to employee performance concerns in human-service settings. Recent research has demonstrated that interventions indicated by the PDC-HS result in improved employee performance across several human-service settings and that the assessment has acceptable reliability and validity. Although PDC-HS-indicated interventions have been effective at increasing employee performance, there is a need for additional guidance when using the assessment given the limited nature of the original administration guidelines. Thus, the purpose of the current manuscript is to introduce additional guidance for use of the PDC-HS across a variety of situations.

Varying reactions to feedback and their effects on observer accuracy and feedback omission

The effectiveness of performance feedback depends on many factors, including accuracy and repeated delivery over time. Previous research has suggested that requiring the delivery of performance feedback may result in less accurate observations and feedback during subsequent observations. The current study evaluated whether different reactions (i.e., positive, neutral, or negative) to feedback would affect the accuracy of subsequent observations or the delivery of feedback. Forty-five participants were assigned to 1 of 3 reactions groups and they were instructed to observe and record the safety performance of a confederate during a baseline condition. Following 2 baseline sessions, participants were instructed to deliver feedback and the confederate reacted to the feedback according to the assigned condition for 4 sessions. Results provide evidence that positive reactions maintain observation accuracy and the provision of feedback, while both neutral and negative reactions decrease observation accuracy and result in frequent omission of feedback.

Training human service staff to implement behavioral skills training using a video-based intervention

The current study evaluated the efficacy of a video-based training to teach 4 staff working in a human service setting to use behavioral skills training (BST) to teach job-related skills to others. Low levels of BST integrity were observed during baseline. Immediately after viewing a 13-min training video, 2 participants implemented BST at mastery (i.e., 100% integrity). The remaining 2 participants required brief supplemental performance feedback to reach criterion performance. The effects of the training generalized to training 2 novel skills and maintained in follow-up probes conducted 1 to 4 weeks posttraining. A social validity measure indicated high levels of satisfaction with the video-based training and high acceptability of BST. Implications of these findings for organizational training practices and directions for future research are discussed.

Increasing Praise Delivery Within Dorms of a Juvenile Justice Facility

Researchers have shown that adolescents in residential treatment facilities benefit from behavior-analytic intervention. However, it remains unclear whether practitioners can apply behavioral interventions to increase staff members' appropriate interactions with residents within a juvenile justice facility. In Study 1, researchers compared direct measures of staff behavior in three target dorms (D1, D2, and D3) containing high levels of resident disruptive behavior to a dorm (D4) with consistently low levels of resident disruptive behavior. Results indicated that staff members in the target dorms engaged in significantly higher rates of reprimands and negative statements than in D4. In Study 2, researchers used didactic and video instruction to train staff members in D1, D2, and D3 to increase contingent and noncontingent praise delivery. Results indicated praise delivery by staff members increased slightly in each target dorm. In Study 3, researchers first evaluated the extent to which measures of staff members' and residents' behaviors improved following training within each dorm. Subsequently, researchers compared the post-training behavioral measures from D1, D2 and D3 to D4 to determine the extent to which staff behavior in the training dorms was distinguishable from D4. Results of Study 3 indicated that one or more staff behaviors improved in each training dorm. Nevertheless, residents' disruptive behavior was unchanged in each target dorm. In addition, staff members' behavior in each target dorm continued to be distinguishable from staff members' behavior in D4 on most behavioral measures.