Is a modified Global Trigger Tool method using automatic trigger identification valid when measuring adverse events?

Exploring surgical infection prediction: A comparative study of established risk indexes and a novel model

BACKGROUND

Surgical site infections are a major health problem that deteriorates the patients' health and increases health care costs. A reliable method to identify patients with modifiable risk of surgical site infection is necessary to reduce the incidence of them but data are limited. Hence the objective is to assess the predictive validity of a logistic regression model compared to risk indexes to identify patients at risk of surgical site infections.

METHODS

In this study, we evaluated the predictive validity of a new model which incorporates important predictors based on logistic regression model compared to three state-of-the-art risk indexes to identify high risk patients, recruited from 2016 to 2020 from a medium size hospital in North Norway, prone to surgical site infection.

RESULTS

The logistic regression model demonstrated significantly higher scores, defined as high-risk, in 110 patients with surgical site infections than in 110 patients without surgical site infections (p < 0.001, CI 19-44) compared to risk indexes. The logistic regression model achieved an area under the curve of 80 %, which was better than the risk indexes SSIRS (77 %), NNIS (59 %), and JSS-SSI (52 %) for predicting surgical site infections. The logistic regression model identified operating time and length of stay as the major predictors of surgical site infections.

CONCLUSIONS

The logistic regression model demonstrated better performance in predicting surgical site infections compared to three state-of-the-art risk indexes. The model could be further developed into a decision support tool, by incorporating predictors available prior to surgery, to identify patients with modifiable risk prone to surgical site infection.

The association between patient safety culture and adverse events - a scoping review

BACKGROUND

Adverse events (AEs) affect 10% of in-hospital patients, causing increased costs, injuries, disability and mortality. Patient safety culture (PSC) is an indicator of quality in healthcare services and is thus perceived as a proxy for the quality of care. Previous studies show variation in the association between PSC scores and AE rates. The main objective of this scoping review is to summarise the evidence on the association between PSC scores and AE rates in healthcare services. In addition, map the characteristics and the applied research methodology in the included studies, and study the strengths and limitations of the evidence.

METHODS

We applied a scoping review methodology to answer the broad research questions of this study, following the PRISMA-ScR checklist. A systematic search in seven databases was conducted in January 2022. The records were screened independently against eligibility criteria using Rayyan software, and the extracted data were collated in a charting form. Descriptive representations and tables display the systematic mapping of the literature.

RESULTS

We included 34 out of 1,743 screened articles. The mapping demonstrated a statistical association in 76% of the studies, where increased PSC scores were associated with reduced AE rates. Most of the studies had a multicentre design and were conducted in-hospital in high-income countries. The methodological approaches to measuring the association varied, including missing reports on the tools` validation and participants, different medical specialties, and work unit level of measurements. In addition, the review identified a lack of eligible studies for meta-analysis and synthesis and demonstrated a need for an in-depth understanding of the association, including context complexity.

CONCLUSIONS

We found that the vast majority of studies report reduced AE rates when PSC scores increase. This review demonstrates a lack of studies from primary care and low- and- middle-income countries. There is a discrepancy in utilised concepts and methodology, hence there is a need for a broader understanding of the concepts and the contextual factors, and more uniform methodology. Longitudinal prospective studies with higher quality can enhance efforts to improve patient safety.

Evaluation of Global trigger tool as a medication safety tool for adverse drug event detection-a cross-sectional study in a tertiary hospital

The objective of this study is to describe and analyze adverse drug events (ADE) identified using the Global trigger tool (GTT) in a Finnish tertiary hospital during a 5-year period and also to evaluate whether the medication module of the GTT is a useful tool for ADE detection and management or if modification of the medication module is needed. A cross-sectional study of retrospective record review in a 450-bed tertiary hospital in Finland. Ten randomly selected patients from electronic medical records were reviewed bimonthly from 2017 to 2021. The GTT team reviewed a total of 834 records with modified GTT method, which includes the evaluation of possible polypharmacy, National Early Warning Score (NEWS), highest nursing intensity raw score (NI), and pain triggers. The data set contained 366 records with triggers in medication module and 601 records with the polypharmacy trigger that were analyzed in this study. With the GTT, a total of 53 ADEs were detected in the 834 medical records, which corresponds to 13 ADEs/1000 patient-days and 6% of the patients. Altogether, 44% of the patients had at least one trigger found with the GTT medication module. As the number of medication module triggers increased per patient, it was more likely that the patient had also experienced an ADE. The number of triggers found with the GTT medication module in patients' records seems to correlate with the risk of ADEs. Modification of the GTT could provide even more reliable data for ADE prevention.

Estonian adverse events study for multimorbid patients using Estonian Trigger Tool (MUPETT-MUltimorbid Patients-Estonian Trigger Tool). Development of Estonian trigger tool for multimorbid patients. A study protocol for mixed-methods study

INTRODUCTION

It is widely recognized that providing healthcare may produce harm to the patient. Different approaches have been developed to measure the burden of adverse events (AEs) to plan and measure the effects of interventions. One of the most widely used instruments is the Trigger Tool, which has previously been modified to be used on various settings and translated into many languages. Multimorbidity complicates care and may increase the number of AEs patients experience. Currently there is no instrument designed to measure AEs in multimorbid patients. In Estonia, there is currently no validated instrument to measure the burden of AEs.

AIMS

The aim of this study will be evaluating the characteristics and ocurrence of AEs in multimorbid patients in hospitalised internal medicine patients of Estonia, and describes the development of a trigger tool for this purpose.

METHODS AND ANALYSIS

We will search for the evidence on measuring AEs in the population of multimorbid patients focusing on trigger tools, and synthesize the data. Data collection of the triggers from the literature will be followed by translating triggers from English to Estonian. An expert multidisciplinary panel will select the suitable triggers for this population. Trigger tool will be pre-tested to assess agreement among professionals and usability of the tool. Validation will be done using 90 medical records. A cross-sectional study in internal medicine departments of two Estonian tertiary care hospitals will be performed to identify the frequency and characteristics of AEs in 960 medical records. We will also provide preventability potential and influencing factors.

DISSEMINATION

Results will be disseminated to healthcare providers and stakeholders at national and international conferences, and as a doctoral medical thesis.

Variation in detected adverse events using trigger tools: A systematic review and meta-analysis

Background

Adverse event (AE) detection is a major patient safety priority. However, despite extensive research on AEs, reported incidence rates vary widely.

Objective

This study aimed: (1) to synthesize available evidence on AE incidence in acute care inpatient settings using Trigger Tool methodology; and (2) to explore whether study characteristics and study quality explain variations in reported AE incidence.

Design

Systematic review and meta-analysis.

Methods

To identify relevant studies, we queried PubMed, EMBASE, CINAHL, Cochrane Library and three journals in the patient safety field (last update search 25.05.2022). Eligible publications fulfilled the following criteria: adult inpatient samples; acute care hospital settings; Trigger Tool methodology; focus on specialty of internal medicine, surgery or oncology; published in English, French, German, Italian or Spanish. Systematic reviews and studies addressing adverse drug events or exclusively deceased patients were excluded. Risk of bias was assessed using an adapted version of the Quality Assessment Tool for Diagnostic Accuracy Studies 2. Our main outcome of interest was AEs per 100 admissions. We assessed nine study characteristics plus study quality as potential sources of variation using random regression models. We received no funding and did not register this review.

Results

Screening 6,685 publications yielded 54 eligible studies covering 194,470 admissions. The cumulative AE incidence was 30.0 per 100 admissions (95% CI 23.9–37.5; I² = 99.7%) and between study heterogeneity was high with a prediction interval of 5.4–164.7. Overall studies’ risk of bias and applicability-related concerns were rated as low. Eight out of nine methodological study characteristics did explain some variation of reported AE rates, such as patient age and type of hospital. Also, study quality did explain variation.

Conclusion

Estimates of AE studies using trigger tool methodology vary while explaining variation is seriously hampered by the low standards of reporting such as the timeframe of AE detection. Specific reporting guidelines for studies using retrospective medical record review methodology are necessary to strengthen the current evidence base and to help explain between study variation.

Monitoring Preventable Adverse Events and Near Misses: Number and Type Identified Differ Depending on Method Used

OBJECTIVES

This study aimed to investigate how many preventable adverse events (PAEs) and near misses are identified through the methods structured record review, Web-based incident reporting (IR), and daily safety briefings, and to distinguish the type of events identified by each method.

METHODS

One year of retrospective data from 2017 were collected from one patient cohort in a 422-bed acute care hospital. Preventable adverse events and near misses were collected from the hospital's existing resources and presented descriptively as number per 1000 patient-days.

RESULTS

The structured record review identified 19.9 PAEs; the IR system, 3.4 PAEs; and daily safety briefings, 5.4 PAEs per 1000 patient-days. The most common PAEs identified by the record review method were drug-related PAEs, pressure ulcers, and hospital-acquired infections. The most common PAEs identified by the IR system and daily safety briefings were fall injury and pressure ulcers, followed by skin/superficial vessel injuries for the IR system and hospital-acquired infections for the daily safety briefings. Incident reporting and daily safety briefings identified 7.8 and 31.9 near misses per 1000 patient-days, respectively. The most common near misses were related to how care is organized.

CONCLUSIONS

The different methods identified different amounts and types of PAEs and near misses. The study supports that health care organizations should adopt multiple methods to get a comprehensive review of the number and type of events occurring in their setting. Daily safety briefings seem to be a particularly suitable method for assessing an organization's inherent security and may foster a nonpunitive culture.

Development and Implementation of an e-Trigger Tool for Adverse Drug Events in a Swiss University Hospital

PURPOSE

The purpose of the study was to develop and implement an institution-specific trigger tool based on the Institute for Healthcare Improvement medication module trigger tool (IHI MMTT) in order to detect and monitor ADEs.

METHODS

We performed an investigator-driven, single-center study using retrospective and prospective patient data to develop ("development phase") and implement ("implementation phase") an efficient, institution-specific trigger tool based on the IHI MMTT. Complete medical data from 1008 patients hospitalized in 2018 were used in the development phase. ADEs were identified by chart review. The performance of two versions of the tool was assessed by comparing their sensitivities and specificities. Tool A employed only digitally extracted triggers ("e-trigger-tool") while Tool B employed an additional manually extracted trigger. The superior tool - taking efficiency into account - was applied prospectively to 19-22 randomly chosen charts per month for 26 months during the implementation phase.

RESULTS

In the development phase, 189 (19%) patients had ≥1 ADE (total 277 ADEs). The time needed to identify these ADEs was 15 minutes/chart. A total of 203 patients had ≥1 trigger (total 273 triggers - Tool B). The sensitivities and specificities of Tools A and B were 0.41 and 0.86, and 0.43 and 0.86, respectively. Tool A was more time-efficient than Tool B (4 vs 9 minutes/chart) and was therefore used in the implementation phase. During the 26-month implementation phase, 22 patients experienced trigger-identified ADEs and 529 did not. The median number of ADEs per 1000 patient days was 6 (range 0-13). Patients with at least one ADE had a mean hospital stay of 22.3 ± 19.7 days, compared to 8.0 ± 7.6 days for those without an ADE (p = 2.7×10-14).

CONCLUSION

We developed and implemented an e-trigger tool that was specific and moderately sensitive, gave consistent results and required minimal resources.

Prediction of risk of acquiring urinary tract infection during hospital stay based on machine-learning: A retrospective cohort study

Background

Healthcare associated infections (HAI) are a major burden for the healthcare system and associated with prolonged hospital stay, increased morbidity, mortality and costs. Healthcare associated urinary tract infections (HA-UTI) accounts for about 20–30% of all HAI’s, and with the emergence of multi-resistant urinary tract pathogens, the total burden of HA-UTI will most likely increase.

Objective

The aim of the current study was to develop two predictive models, using data from the index admission as well as historic data on a patient, to predict the development of UTI at the time of entry to the hospital and after 48 hours of admission (HA-UTI). The ultimate goal is to predict the individual patient risk of acquiring HA-UTI before it occurs so that health care professionals may take proper actions to prevent it.

Methods

Retrospective cohort analysis of approx. 300 000 adult admissions in a Danish region was performed. We developed models for UTI prediction with five machine-learning algorithms using demographic information, laboratory results, data on antibiotic treatment, past medical history (ICD10 codes), and clinical data by transformation of unstructured narrative text in Electronic Medical Records to structured data by Natural Language Processing.

Results

The five machine-learning algorithms have been evaluated by the performance measures average squared error, cumulative lift, and area under the curve (ROC-index). The algorithms had an area under the curve (ROC-index) ranging from 0.82 to 0.84 for the entry model (T = 0 hours after admission) and from 0.71 to 0.77 for the HA-UTI model (T = 48 hours after admission).

Conclusion

The study is proof of concept that it is possible to create machine-learning models that can serve as early warning systems to predict patients at risk of acquiring urinary tract infections during admission. The entry model and the HA-UTI models perform with a high ROC-index indicating a sufficient sensitivity and specificity, which may make both models instrumental in individualized prevention of UTI in hospitalized patients. The favored machine-learning methodology is Decision Trees to ensure the most transparent results and to increase clinical understanding and implementation of the models.

Translation, Cross-Cultural Adaptation, and Measurement Properties of the Portuguese Version of the Global Trigger Tool for Adverse Events

Purpose

To adapt and validate the Global Trigger Tool (IHI-GTT), which identifies and analyzes adverse events (AE) in hospitalized patients and their measurement properties in the Portuguese context.

Methods

A retrospective cross-sectional study was based on a random sample of 90 medical records. The stages of translation and cross-cultural adaptation of the IHI-GTT were based on the Cross-Cultural Adaptation Protocol that originated from the Portuguese version, GTT-PT, for the hospital context in medical-surgical departments. Internal consistency, reliability, reproducibility, diagnostic tests, and discriminatory predictive value were investigated.

Results

The final phase of the GTT-PT showed insignificant inconsistencies. The pre-test phase confirmed translation accuracy, easy administration, effectiveness in identifying AEs, and relevance of integrating it into hospital risk management. It had a sensitivity of 97.8% and specificity of 74.8%, with a cutoff point of 0.5, an accuracy of 83%, and a positive predictive value of 69.8% and a negative predictive value of 0.98%.

Conclusion

The GTT-PT is a reliable, accurate, and valid tool to identify AE, with robust measurement properties.

Assessing the development and implementation of the Global Trigger Tool method across a large health system in Sicily

Background: The Institute for Healthcare Improvement (IHI) has proposed a new method, the Global Trigger Tool (IHI GTT), to detect and monitor adverse events (AEs) and provide information to implement improvement. In 2015, the Sicilian Health System adopted IHI GTT to assess the number, types and severity levels of AEs. The GTT was implemented in 44 of 73 Sicilian public hospitals and 18,008 clinical records (CRs) were examined. Here we present the standardized application of the GTT and the preliminary results of 14,706 reviews of CRs.

Methods: IHI GTT was adapted, developed and implemented to the local context. Reviews of CRs were conducted by 199 professionals divided into 71 review teams consisting of three individuals: two of whom had clinical knowledge and expertise, and a physician to authenticate the AE. The reviewers entered data into a dedicated IT-platform. All 44 of the public hospitals were included, with approximately 300,000 yearly inpatient admissions out of a population of approximately 5 million. In total, 14,706 randomized CRs of inpatients from medicine, surgery, obstetric and ICU wards, from June 2015 to June 2018 were reviewed.

Results: In 975 (6.6%) CRs at least one AE was found. Approximately 20,000 patients of the 300,000 discharged each year in Sicily have at least one AE. In 5,574 (37.9%) CRs at least one trigger was found. A total of 1,542 AEs were found. The analysis of ROC curve shows that the presence of two triggers in a CR indicates with high probability the presence of an AE. The most frequent type of AE was in-hospital related infection.

Conclusions: The GTT is an efficient method to identify AEs and to track improvement of care. The analysis and monitoring of some triggers is important to prevent AEs. However, it is a labor-intensive method, particularly if the CRs are paper-based.