CheckPOINT: a simple tool to measure Surgical Safety Checklist implementation fidelity

INTRODUCTION

The WHO Surgical Safety Checklist (SSC) is a communication tool that improves teamwork and patient outcomes. SSC effectiveness is dependent on implementation fidelity. Administrative audits fail to capture most aspects of SSC implementation fidelity (ie, team communication and engagement). Existing research tools assess behaviours during checklist performance, but were not designed for routine quality assurance and improvement. We aimed to create a simple tool to assess SSC implementation fidelity, and to test its reliability using video simulations, and usability in clinical practice.

METHODS

The Checklist Performance Observation for Improvement (CheckPOINT) tool underwent two rounds of face validity testing with surgical safety experts, clinicians and quality improvement specialists. Four categories were developed: checklist adherence, communication effectiveness, attitude and engagement. We created a 90 min training programme, and four trained raters independently scored 37 video simulations using the tool. We calculated intraclass correlation coefficients (ICC) to assess inter-rater reliability (ICC>0.75 indicating excellent reliability). We then trained two observers, who tested the tool in the operating room. We interviewed the observers to determine tool usability.

RESULTS

The CheckPOINT tool had excellent inter-rater reliability across SSC phases. The ICC was 0.83 (95% CI 0.67 to 0.98) for the sign-in, 0.77 (95% CI 0.63 to 0.92) for the time-out and 0.79 (95% CI 0.59 to 0.99) for the sign-out. During field testing, observers reported CheckPOINT was easy to use. In 98 operating room observations, the total median (IQR) score was 25 (23-28), checklist adherence was 7 (6-7), communication effectiveness was 6 (6-7), attitude was 6 (6-7) and engagement was 6 (5-7).

CONCLUSIONS

CheckPOINT is a simple and reliable tool to assess SSC implementation fidelity and identify areas of focus for improvement efforts. Although CheckPOINT would benefit from further testing, it offers a low-resource alternative to existing research tools and captures elements of adherence and team behaviours.

Going beyond compliance: A qualitative study of the practice of surgical safety checklist

BACKGROUND

The World Health Organization Surgical Safety Checklist (SSC) is a tool designed to enhance team communication and patient safety. When used properly, the SSC acts as a layer of defence against never events. In this study, we performed secondary qualitative analysis of operating theatres (OT) SSC observational notes to examine how the SSC was used after an intensive SSC re-implementation effort and drew on relevant theories to shed light on the observed patterns of behaviours. We aimed to go beyond assessing checklist compliance and to understand potential sociopsychological mechanisms of the variations in SSC practices.

METHODS

Direct observation notes of 109 surgical procedures across 13 surgical disciplines were made by two trained nurses in the OT of a large tertiary hospital in Singapore from February to April 2022, three months after SSC re-implementation. Only notes relevant to the use of SSC were extracted and analyzed using reflexive thematic analysis. Data were coded following an inductive process to identify themes or patterns of SSC practices. These patterns were subsequently interpreted against a relevant theory to appreciate the potential sociopsychological forces behind them.

RESULTS

Two broad types of SSC practices and their respective sub-themes were identified. Type 1 (vs. Type 2) SSC practices are characterized by patience and thoroughness (vs. hurriedness and omission) in carrying out the SSC process, dedication and attention (vs. delegation and distraction) to the SSC safety checks, and frequent (vs. absence of) safety voices during the conduct of SSC. These patterns were conceptualized as safety-seeking action vs. ritualistic action using Merton's social deviance theory.

CONCLUSION

Ritualistic practice of the SSC can undermine surgical safety by creating conditions conducive to never events. To fully realize the SSC's potential as an essential tool for communication and safety, a concerted effort is needed to balance thoroughness with efficiency. Additionally, fostering a culture of collaboration and collegiality is crucial to reinforce and enhance the culture of surgical safety.

Transforming Team Performance Through Reimplementation of the Surgical Safety Checklist

Importance

Patient safety interventions, like the World Health Organization Surgical Safety Checklist, require effective implementation strategies to achieve meaningful results. Institutions with underperforming checklists require evidence-based guidance for reimplementing these practices to maximize their impact on patient safety.

Objective

To assess the ability of a comprehensive system of safety checklist reimplementation to change behavior, enhance safety culture, and improve outcomes for surgical patients.

Design, Setting, and Participants

This prospective type 2 hybrid implementation-effectiveness study took place at 2 large academic referral centers in Singapore. All operations performed at either hospital were eligible for observation. Surveys were distributed to all operating room staff.

Intervention

The study team developed a comprehensive surgical safety checklist reimplementation package based on the Exploration, Preparation, Implementation, Sustainment framework. Best practices from implementation science and human factors engineering were combined to redesign the checklist. The revised instrument was reimplemented in November 2021.

Main Outcomes and Measures

Implementation outcomes included penetration and fidelity. The primary effectiveness outcome was team performance, assessed by trained observers using the Oxford Non-Technical Skills (NOTECH) system before and after reimplementation. The Agency for Healthcare Research and Quality Hospital Survey on Patient Safety Culture was used to assess safety culture and observers tracked device-related interruptions (DRIs). Patient safety events, near-miss events, 30-day mortality, and serious complications were tracked for exploratory analyses.

Results

Observers captured 252 cases (161 baseline and 91 end point). Penetration of the checklist was excellent at both time points, but there were significant improvements in all measures of fidelity after reimplementation. Mean NOTECHS scores increased from 37.1 to 42.4 points (4.3 point adjusted increase; 95% CI, 2.9-5.7; P < .001). DRIs decreased by 86.5% (95% CI, -22.1% to -97.8%; P = .03). Significant improvements were noted in 9 of 12 composite areas on culture of safety surveys. Exploratory analyses suggested reductions in patient safety events, mortality, and serious complications.

Conclusions and Relevance

Comprehensive reimplementation of an established checklist intervention can meaningfully improve team behavior, safety culture, patient safety, and patient outcomes. Future efforts will expand the reach of this system by testing a structured guidebook coupled with light-touch implementation guidance in a variety of settings.

Creating a high-performance surgical safety checklist: A multimodal evaluation plan to reinvigorate the checklist

RATIONALE, AIMS, AND OBJECTIVES

The WHO Surgical Safety Checklist is a communication tool designed to improve surgical safety processes and enhance teamwork. It has been widely adopted since its introduction over ten years ago. As surgical safety needs evolve, organizations should periodically review and update their checklists. A holistic evaluation of the checklist in the context of an organization is the first step to making informed updates. In this article, we describe a comprehensive but feasible strategy for checklist evaluation which we developed and implemented as part of a surgical safety initiative in a high-performing center.

METHODS

A three-part evaluation plan was developed and carried out by a multidisciplinary team. The evaluation included assessment of 1. Quality of care through a review of surgical safety events; 2. Safety culture through a validated survey and informal feedback; and 3. Checklist performance through direct observations and a staff survey. To prepare for re-implementation the current institutional checklist was critically evaluated and a context assessment survey was administered to surgical staff.

RESULTS

The evaluation revealed challenges in communication and teamwork, with surgical staff often perceived to be working in silos. The quality of care assessment indicated room for improvement in safety processes. Deficiencies in the safety culture measures of communication and feedback shed light on an overall lack of engagement with the checklist. Checklist performance demonstrated good adherence to the items on the checklist but limited engagement by the surgical team and minimal communication between subteams. These findings informed our revisions to the checklist and its implementation processes.

CONCLUSIONS

We developed and implemented a comprehensive, scalable approach to checklist evaluation which directly informed improvements to the checklist that were tailored to the organization's current context. Organizations can apply this framework to breathe new life into their checklist and transform their safety culture.

Public Discourse Surrounding Suicide during the COVID-19 Pandemic: An Unsupervised Machine Learning Analysis of Twitter Posts over a One-Year Period

Many studies have forewarned the profound emotional and psychosocial impact of the protracted COVID-19 pandemic. This study thus aimed to examine how individuals relate to suicide amid the COVID-19 pandemic from a global perspective via the public Twitter discourse around suicide and COVID-19. Original Twitter tweets from 1 February 2020 to 10 February 2021 were searched, with terms related to "COVID-19", "suicide", or "self-harm". An unsupervised machine learning approach and topic modelling were used to identify topics from unique tweets, with each topic further grouped into themes using manually conducted thematic analysis by the study investigators. A total of 35,904 tweets related to suicide and COVID-19 were processed into 42 topics and six themes. The main themes were: (1) mixed reactions to COVID-19 public health policies and their presumed impact on suicide; (2) biopsychosocial impact of COVID-19 pandemic on suicide and self-harm; (3) comparing mortality rates of COVID-19, suicide, and other leading causes of death; (4) mental health support for individuals at risk of suicide; (5) reported cases and public reactions to news related to COVID-19, suicide, and homicide; and (6) figurative usage of the word suicide. The general public was generally concerned about governments' responses as well as the perturbing effects on mental health, suicide, the economy, and at-risk populations.

Examining the Prevailing Negative Sentiments Related to COVID-19 Vaccination: Unsupervised Deep Learning of Twitter Posts over a 16 Month Period

Despite the demonstrated efficacy, safety, and availability of COVID-19 vaccines, efforts in global mass vaccination have been met with widespread scepticism and vaccine hesitancy or refusal. Understanding the reasons for the public's negative opinions towards COVID-19 vaccination using Twitter may help make new headways in improving vaccine uptake. This study, therefore, examined the prevailing negative sentiments towards COVID-19 vaccination via the analysis of public twitter posts over a 16 month period. Original tweets (in English) from 1 April 2021 to 1 August 2022 were extracted. A bidirectional encoder representations from transformers (BERT)-based model was applied, and only negative sentiments tweets were selected. Topic modelling was used, followed by manual thematic analysis performed iteratively by the study investigators, with independent reviews of the topic labels and themes. A total of 4,448,314 tweets were analysed. The analysis generated six topics and three themes related to the prevailing negative sentiments towards COVID-19 vaccination. The themes could be broadly understood as either emotional reactions to perceived invidious policies or safety and effectiveness concerns related to the COVID-19 vaccines. The themes uncovered in the present infodemiology study fit well into the increasing vaccination model, and they highlight important public conversations to be had and potential avenues for future policy intervention and campaign efforts.

Utility of a Device Briefing Tool to Improve Surgical Safety

INTRODUCTION

Clear communication around surgical device use is crucial to patient safety. We evaluated the utility of the Device Briefing Tool (DBT) as an adjunct to the Surgical Safety Checklist.

METHODS

A nonrandomized, controlled pilot of the DBT was conducted with surgical teams at an academic referral center. Intervention departments used the DBT in all cases involving a surgical device for 10 wk. Utility, relative advantage, and implementation effectiveness were evaluated via surveys. Trained observers assessed adherence and team performance using the Oxford NOTECHS system.

RESULTS

Of 113 individuals surveyed, 91 responded. Most respondents rated the DBT as moderately to extremely useful. Utility was greatest for complex devices (89%) and new devices (88%). Advantages included insight into the team's familiarity with devices (70%) and improved teamwork and communication (68%). Users found it unrealistic to review all device instructional materials (54%). Free text responses suggested that the DBT heightened awareness of deficiencies in device familiarity and training but lacked a clear mechanism to correct them. DBT adherence was 82%. NOTECHS scores in intervention departments improved over the course of the study but did not significantly differ from comparator departments.

CONCLUSIONS

The DBT was rated highly by both surgeons and nurses. Adherence was high and we found no evidence of "checklist fatigue." Centers interested in implementing the DBT should focus on devices that are complex or new to any surgical team member. Guidance for correcting deficiencies identified by the DBT will be provided in future iterations of the tool.

Virtual non-technical skills assessment training is an effective, scalable approach for novice raters

OBJECTIVES

The COVID-19 pandemic has forced a creative transition to virtual platforms due to physical distancing and travel restrictions. We designed and tested a highly scalable virtual training curriculum for novice raters using the Oxford NOTECHS non-technical skills rating system.

DESIGN

A three-day training course comprising virtual didactics, virtually facilitated simulations, and independent live observations was implemented. NOTECHS scores were submitted for eleven standardized video simulations and four live operations. Intraclass correlation coefficients (ICCs) were calculated for total NOTECHS scores and subcomponent scores. Raters previously trained in-person with the same standardized videos served as a comparator group for equivalence testing.

SETTING

All study activities were conducted in a large academic tertiary referral center in Singapore as part of an ongoing surgical safety initiative.

PARTICIPANTS

Seven staff members underwent training (three virtually and four in-person).  None had prior surgical experience or non-technical skills assessment training.

RESULTS

ICCs for total NOTECHS scores were 0.85 (95% CI, 0.73-0.98) for virtually trained raters and 0.83 for those trained in-person (95% CI, 0.68-0.99).  Scores were equivalent between groups within a 10% margin.

CONCLUSIONS

Non-technical skills assessment can be reliably taught in a highly scalable virtual format. Virtual NOTECHS training is a valuable tool for educational and quality improvement initiatives during the COVID-19 pandemic and for centers that lack ready access to onsite non-technical skills training expertise.

Ictal electrocorticographic findings related with surgical outcomes in nonlesional neocortical epilepsy

PURPOSE

To characterize ictal electrocorticographic features related to surgical outcomes in nonlesional neocortical epilepsy (NE).

METHODS

We analyzed 187 ictal electrocorticograms (ECoG) obtained from 18 patients who had undergone presurgical evaluation and subsequent neocortical resections (frontal: seven, parietal: one, occipital: four, multilobar: six). None of them had any MRI-detectable lesions. Various ECoG data sets recorded from eight patients who achieved a favorable surgical outcome (either seizure free or more than 90% reduction of seizure frequencies) were compared with that from ten patients with unfavorable outcome (less than 90% reduction of seizure frequencies) (follow up duration: 47+/-11 months).

RESULTS

Reproducible ictal onset zone (IOZ) in recurrent seizures (P=0.013) and persistent ictal discharges in IOZ from the onset to the end of seizure (P=0.004) were found more frequently in the patients with good outcome. Ictal onset patterns consisting of low voltage fast or high amplitude beta spikes predicted a good surgical outcome while rhythmic sinusoidal activity or rhythmic spike/sharp waves of slow frequency were predictive of poor outcome (P=0.01). The ictal onset rhythm consisting of gamma or beta frequencies was more prevalent in the favorable group (P=0.015).

CONCLUSIONS

The presence of stable ictal circuit suggested by the consistent earliest activation of specific electrodes in the repetitive seizures (reproducible IOZ) and the active participation of IOZ throughout the attack were valuable prognostic factors in addition to the morphology and frequency of ictal onset rhythm.

Modulation of survival motor neuron pre-mRNA splicing by inhibition of alternative 3' splice site pairing

Spinal muscular atrophy is caused by the loss of functional survival motor neuron (SMN1) alleles. A translationally silent nucleotide transition in the duplicated copy of the gene (SMN2) leads to exon 7 skipping and expression of a nonfunctional gene product. It has been suggested that differential SMN2 splicing is caused by the disruption of an exonic splicing enhancer. Here we show that the single nucleotide difference reduces the intrinsic strength of the 3' splice site of exon 7 2-fold, whereas the strength of the 5' splice site of the exon 7 is not affected. Thus, a decrease in splice site strength is magnified in the context of competing exons. These data suggest that lower levels of exon 7 definition not only reduce intron 6 removal but, more importantly, increase the efficiency of the competing exon 7 skipping pathway. Antisense oligonucleotides were tested to modulate exon 7 inclusion, which contains the authentic translation stop codon. Oligonucleotides directed toward the 3' splice site of exon 8 were shown to alter SMN2 splicing in favor of exon 7 inclusion. These results suggest that antisense oligonucleotides could be used as a therapeutic strategy to counteract the progression of SMA.

MRI features of infarcts with potential cardiac source of embolism in the Yonsei Stroke Registry (YSR), Korea

The determination of the embolic source is crucial to understanding the pathogenesis of ischemic stroke, the initiation of appropriate therapy, and the prevention of recurrent infarctions. In this study we undertook to identify the characteristic features on magnetic resonance images of patients who had suffered from stroke due to cardiac embolism (CE), as classified by TOAST (possible and probable). We retrospectively studied magnetic resonance imaging (MRI) findings of patients with ischemic stroke from the Yonsei Stroke Registry (YSR). On the basis of the TOAST classification, 92 patients were identified to have a potential cardiac source of embolism (PCSE), in which 69 patients were found to have high-risk PCSE and 23 patients medium-risk PCSE. To compare their imaging characteristics, another group of 49 patients who were found to have had a stroke due to large artery-to-artery (ATA) embolism-common or internal carotid artery (CCA, ICA)-were identified. Involvement of the simultaneous superficial and deep territories (58.7%; 6.1%, p < 0.001), and combined new anterior and old posterior circulation (15.2%; 2.0%, p = 0.016) were more frequent in PCSE than ATA embolism. Bilateral anterior hemispheric involvement was also more frequent in the PCSE group, but it did not reach statistical significance (13.0%; 4.1%, p = 0.090). ATA embolism tended to involve only superficial territories compared to PCSE (71.4%; 28.3%, p < 0.001). There were no topographic differences between the high-risk and medium-risk groups. With respect to the etiology of PCSE in our population, atrial fibrillation was the most common. Characteristic MRI features of patients with PCSE, which were not documented previously by computed tomography (CT) included: old and new, involvement of multiple different vascular territories, bilateral anterior hemisphere, as well as anterior and posterior circulation. These MRI features, together with simultaneous superficial and deep territorial involvement, help to differentiate the underlying embolic sources, whether they are cardiac or ATA in origin.