Feasibility and utility of the use of real time random safety audits in adult ICU patients: a multicentre study

Assessing the impact of real-time random safety audits through full propensity score matching on reliable data from the clinical information system

BACKGROUND

Evidence-based care processes are not always applied at the bedside in critically ill patients. Numerous studies have assessed the impact of checklists and related strategies on the process of care and patient outcomes. We aimed to evaluate the effects of real-time random safety audits on process-of-care and outcome variables in critical care patients.

METHODS

This prospective study used data from the clinical information system to evaluate the impact of real-time random safety audits targeting 32 safety measures in two intensive care units during a 9-month period. We compared endpoints between patients attended with safety audits and those not attended with safety audits. The primary endpoint was mortality, measured by Cox hazard regression after full propensity-score matching. Secondary endpoints were the impact on adherence to process-of-care measures and on quality indicators.

RESULTS

We included 871 patients; 228 of these were attended in ≥ 1 real-time random safety audits. Safety audits were carried out on 390 patient-days; most improvements in the process of care were observed in safety measures related to mechanical ventilation, renal function and therapies, nutrition, and clinical information system. Although the group of patients attended in safety audits had more severe disease at ICU admission [APACHE II score 21 (16-27) vs. 20 (15-25), p = 0.023]; included a higher proportion of surgical patients [37.3 % vs. 26.4 %, p = 0.003] and a higher proportion of mechanically ventilated patients [72.8 % vs. 40.3 %, p < 0.001]; averaged more days on mechanical ventilation, central venous catheter, and urinary catheter; and had a longer ICU stay [12.5 (5.5-23.3) vs. 2.9 (1.7-5.9), p < 0.001], ICU mortality did not differ significantly between groups (19.3 % vs. 18.8 % in the group without safety rounds). After full propensity-score matching, Cox hazard regression analysis showed real-time random safety audits were associated with a lower risk of mortality throughout the ICU stay (HR 0.31; 95 %CI 0.20-0.47).

CONCLUSIONS

Real-time random safety audits are associated with a reduction in the risk of ICU mortality. Exploiting data from the clinical information system is useful in assessing the impact of them on the care process, quality indicators, and mortality.

Design of a safety round model for intensive care units

INTRODUCTION

Safety Rounds (SR) are an operational tool that allow knowing adherence to good practices, help identify risks and incidents in patient safety (PS), allowing improvement actions to be implemented. The objective of this work was the design of a procedure to perform SR in an Intensive Care Unit (ICU).

METHODS

Preparation of a checklist for the development of SR in the ICU through the nominal group technique, with the participation of managers, middle managers and professionals from different disciplines and categories. In the first place, a group of experts agreed, based on the recommendations on good practices in PS, the definition of items, their coding, the criteria for compliance and the impact of non-compliance. Subsequently, its viability was determined through a cross-sectional study through the piloting of two SRs to adjust the items in real clinical practice conditions.

RESULTS

A specific SR model for ICUs has been obtained through a checklist. The group of experts prepared a first list made up of 39 items of 6 essential dimensions and defined the method of implementation. Mean time to complete the two SRs was 85 min, including the briefing and subsequent debriefing. After the validation pilot, the dimensions were reduced to 5, 3 items were deleted, 2 items were transferred to another dimension and 3 items related to nosocomial infections and informed consent were modified. In addition, the data sources, the compliance criteria and their relative weight were redefined. The final list was considered useful and relevant to improve practice.

CONCLUSIONS

Through a consensus methodology, a checklist has been built to be used in the RS of an ICU. This model can serve as a basis for its use in healthcare services with similar characteristics.

Analysis of adherence to an early mobilization protocol in an intensive care unit: Data collected prospectively over a period of three years by the clinical information system

OBJECTIVE

To determinate the adherence and barriers of our early mobilization protocol in patients who had received mechanical ventilation >48h in routine daily practice through clinical information system during all Intensive Care Unit (ICU) stay.

DESIGN

Observational and prospective cohort study.

SETTING

Polyvalent ICU over a three-year period (2017-2019).

PATIENTS

Adult patients on mechanical ventilation >48h who met the inclusion criteria for the early mobilization protocol.

INTERVENTIONS

None.

MAIN VARIABLES OF INTEREST

Demographics, adherence to the protocol and putative hidden adherence, total number of mobilizations, barriers, artificial airway/ventilatory support at each mobilization level and adverse events.

RESULTS

We analyzed 3269 stay-days from 388 patients with median age of 63 (51-72) years, median APACHE II 23 (18-29) and median ICU stay of 10.1 (6.2-16.5) days. Adherence to the protocol was 56.6% (1850 stay-days), but patients were mobilized in only 32.2% (1472) of all stay-days. The putative hidden adherence was 15.6% (509 stay-days) which would increase adherence to 72.2%. The most common reasons for not mobilizing patients were failure to meeting the criteria for clinical stability in 241 (42%) stay-days and unavailability of physiotherapists in 190 (33%) stay-days. Adverse events occurred in only 6 (0.4%) stay-days.

CONCLUSIONS

Data form Clinical Information System showed although adherence was high, patients were mobilized in only one-third of all stay-days. Knowing the specific reason why patient were not mobilized in each stay-day allow to develop concrete decisions to increase the number of mobilizations.

Cross-cultural adaptation of the SCORE survey and evaluation of the impact of Real-Time Random Safety Audits in organizational culture: A multicenter study

OBJECTIVE

To establish a cross-cultural adaptation of the Safety, Communication, Operational Reliability, and Engagement (SCORE) survey and to use this instrument to evaluate the impact of a safety intervention.

DESIGN

Cross-cultural adaptation and before-and-after evaluation study.

SETTING

5 ICU.

PARTICIPANTS

Medical residents, attending physicians, and nurses at those ICU.

INTERVENTIONS

Adaptation of the SCORE survey to Spanish culture. The adapted survey was used to assess all safety-culture-related domains before and one-year after implementing the use of a safety tool, Real-Time Random Safety Audits (in Spanish: Análisis Aleatorios de Seguridad en Tiempo Real, AASTRE).

MAIN OUTCOME MEASURE

Adaptabiliy of the Spanish version of SCORE survey in the ICU setting and evaluation of the effect of AASTRE on their domains.

RESULTS

The cross-cultural adaptation was adequate. Post-AASTRE survey scores [mean (standard deviation, SD)] were significantly better in the domains learning environment [50.55 (SD 20.62) vs 60.76 (SD 23.66), p<.0001], perception of local leadership [47.98 (SD 23.57) vs 62.82 (SD 27.46), p<.0001], teamwork climate [51.19 (SD 18.55) vs 55.89 (SD 20.25), p=.031], safety climate [45.07 (SD 17.60) vs 50.36 (SD 19.65), p=.01], participation decision making [3 (SD 0.82) vs 3.65 (SD 0.87), p<.0001] and advancement in the organization [3.21 (SD 0.77) vs 4.04 (SD 0.77), p<.0001]. However, post-AASTRE scores were significantly worse in the domains workload and burnout climate.

CONCLUSIONS

The cross-cultural adaptation of the SCORE survey into Spanish is a useful tool for ICUs. The application of the AASTRE is associated with improvements in six SCORE domains, including the safety climate.

Cross-cultural adaptation of the SCORE survey and evaluation of the impact of Real-Time Random Safety Audits in organizational culture: A multicenter study

OBJECTIVE

To establish a cross-cultural adaptation of the Safety, Communication, Operational Reliability, and Engagement (SCORE) survey and to use this instrument to evaluate the impact of a safety intervention.

DESIGN

Cross-cultural adaptation and before-and-after evaluation study.

SETTING

5 ICU.

PARTICIPANTS

Medical residents, attending physicians, and nurses at those ICU.

INTERVENTIONS

Adaptation of the SCORE survey to Spanish culture. The adapted survey was used to assess all safety-culture-related domains before and one-year after implementing the use of a safety tool, Real-Time Random Safety Audits (in Spanish: Análisis Aleatorios de Seguridad en Tiempo Real, AASTRE).

MAIN OUTCOME MEASURE

Adaptabiliy of the Spanish version of SCORE survey in the ICU setting and evaluation of the effect of AASTRE on their domains.

RESULTS

The cross-cultural adaptation was adequate. Post-AASTRE survey scores [mean (standard deviation, SD)] were significantly better in the domains learning environment [50.55 (SD 20.62) vs 60.76 (SD 23.66), p<.0001], perception of local leadership [47.98 (SD 23.57) vs 62.82 (SD 27.46), p<.0001], teamwork climate [51.19 (SD 18.55) vs 55.89 (SD 20.25), p=.031], safety climate [45.07 (SD 17.60) vs 50.36 (SD 19.65), p=.01], participation decision making [3 (SD 0.82) vs 3.65 (SD 0.87), p<.0001] and advancement in the organization [3.21 (SD 0.77) vs 4.04 (SD 0.77), p<.0001]. However, post-AASTRE scores were significantly worse in the domains workload and burnout climate.

CONCLUSIONS

The cross-cultural adaptation of the SCORE survey into Spanish is a useful tool for ICUs. The application of the AASTRE is associated with improvements in six SCORE domains, including the safety climate.

Application, verification and correction from an elaborate checklist with some of the recommendations («do and do not do») of the SEMICYUC working groups

OBJECTIVE

Based on some of the recommendations of the SEMICYUC working groups, we developed a checklist and applied it in 2 periods, analyzing their behavior as a tool for improving safety.

DESIGN

A comparative pre- and post-intervention longitudinal study was carried out.

SETTING

The Intensive Care Unit (ICU) of a 400-bed university hospital.

PATIENTS

Random cases series in 2 periods separated by 6 months.

INTERVENTIONS

We developed a checklist with 24 selected indicators that were randomly applied to 50 patients. Verification was conducted by a professional not related to care (prompter). We analyzed the results and compliance index and carried out corrective measures with training. With 6 months of preparation, we again applied the random checklist to 50 patients (post-intervention period) and compared the compliance indexes between the two timepoints.

RESULTS

There were no differences in demographic characteristics or evolution between the periods. The compliance index at baseline was 0.86±0.12 versus 0.91±0.52 in the post-intervention period (P=.023). An acceptable compliance index was obtained with the 24 indicators, though at baseline the compliance index was<0.85 for 5 recommendations. These detected non-compliances were worked upon through training in the second phase of the study. The post-intervention checklist evidenced improvement in compliance with the recommendations.

CONCLUSIONS

The checklist used to assess compliance with a selection of recommendations of the SEMICYUC applied and moderated by a prompter was seen to be a useful instrument allowing us to identify points for improvement in the management of Intensive Care Unit patients, increasing the quality and safety of care.

Quality Improvement Bedside Rounding Audits Enhance Protein Provision for Pediatric Patients Receiving Continuous Renal Replacement Therapy

OBJECTIVES

Describe quality improvement process improvements in protein delivery of continuous renal replacement therapy initiation.

DESIGN

Prospective study.

SETTING

PICU and cardiovascular ICU within a quaternary care children's hospital.

PATIENTS

PICU and cardiovascular ICU patients receiving continuous renal replacement therapy for greater than 48 hours. Inborn errors of metabolism were excluded.

INTERVENTIONS

Plan-Do-Study-Act cycles were initiated. Cycle 1 developed interdisciplinary quality improvement group continuously monitoring nutrition care with thrice weekly bedside safety rounds and protein prescriptions within nephrologist's notes. Cycle 2 included education to intensivists. Cycle 3 initiated monthly quality improvement meetings reviewing nutritional care goals.

MEASUREMENTS AND MAIN RESULTS

Primary outcome was percentage of time patients met protein goals in the first 5 days of continuous renal replacement therapy. Secondary outcome was percentage of time patients met protein goals for duration of continuous renal replacement therapy. Cohort (n = 55) mean age was 8.1 years (SD ± 6.8), 62% male, and 31% malnutrition at baseline. Percent of time meeting protein goals by day 5 was 22%, 33%, and 71% and percent of time meeting protein goals throughout was 35%, 39%, and 75% of groups 1, 2, and 3, respectively. Significant improvement occurred after Plan-Do-Study-Act 3 (group 2 vs group 3; p < 0.01) for primary and secondary outcomes.

CONCLUSIONS

Implementation of an interprofessional quality improvement team significantly decreased number of continuous renal replacement therapy days with unmet protein goals and improved protein delivery.

Safety incidents in airway and mechanical ventilation in Spanish ICUs: The IVeMVA study

PURPOSE

To assess incidence, related factors and characteristics of safety incidents associated with the whole process of airway management and mechanical ventilation (MV) in Spanish ICUs.

MATERIALS AND METHODS

Observational, prospective, 7 days cross-sectional multicenter study. Airway and MV related incidents were reported using structured questionnaire. Type, characteristics, severity, avoidability and contributing factors of the incidents were assessed.

RESULTS

Participant ICUs: 104. Inclusion of 1267 patients; 745 (59%) suffered one or more incidents. Incidents reported: 2492 (59% non-harm-events, 41% adverse events). Individual risk of suffering at least one incident: 66.6%. Incidence ratio (median) of incidents: 2 per 100 patient-hours. 73.7% of incidents were related to MV process, 9.5% to tracheostomy, 6.2% to non-invasive MV, 5.4% to weaning/extubation, 4.4% to intubation and 0.8% to prone position. Temporary damage was produced in 12% incidents, while 0.8% was related to permanent injuries, risk to the patient's life or contributed to death. Incidents were considered avoidable in 73.5% of cases. 98% of all incidents had 1 or more contributing factors.

CONCLUSIONS

MV is a risk process in critical patients. Although most incidents did not harm patients, some caused damage and a few were related to the patient's death or permanent damage. Preventability is high.

Validation of the ICU-DaMa tool for automatically extracting variables for minimum dataset and quality indicators: The importance of data quality assessment

BACKGROUND

Big data analytics promise insights into healthcare processes and management, improving outcomes while reducing costs. However, data quality is a major challenge for reliable results. Business process discovery techniques and an associated data model were used to develop data management tool, ICU-DaMa, for extracting variables essential for overseeing the quality of care in the intensive care unit (ICU).

OBJECTIVE

To determine the feasibility of using ICU-DaMa to automatically extract variables for the minimum dataset and ICU quality indicators from the clinical information system (CIS).

METHODS

The Wilcoxon signed-rank test and Fisher's exact test were used to compare the values extracted from the CIS with ICU-DaMa for 25 variables from all patients attended in a polyvalent ICU during a two-month period against the gold standard of values manually extracted by two trained physicians. Discrepancies with the gold standard were classified into plausibility, conformance, and completeness errors.

RESULTS

Data from 149 patients were included. Although there were no significant differences between the automatic method and the manual method, we detected differences in values for five variables, including one plausibility error and two conformance and completeness errors. Plausibility: 1) Sex, ICU-DaMa incorrectly classified one male patient as female (error generated by the Hospital's Admissions Department). Conformance: 2) Reason for isolation, ICU-DaMa failed to detect a human error in which a professional misclassified a patient's isolation. 3) Brain death, ICU-DaMa failed to detect another human error in which a professional likely entered two mutually exclusive values related to the death of the patient (brain death and controlled donation after circulatory death). Completeness: 4) Destination at ICU discharge, ICU-DaMa incorrectly classified two patients due to a professional failing to fill out the patient discharge form when thepatients died. 5) Length of continuous renal replacement therapy, data were missing for one patient because the CRRT device was not connected to the CIS.

CONCLUSIONS

Automatic generation of minimum dataset and ICU quality indicators using ICU-DaMa is feasible. The discrepancies were identified and can be corrected by improving CIS ergonomics, training healthcare professionals in the culture of the quality of information, and using tools for detecting and correcting data errors.

Impact of random safety analyses on structure, process and outcome indicators: multicentre study

Background

To assess the impact of a real-time random safety tool on structure, process and outcome indicators.

Methods

Prospective study conducted over a period of 12 months in two adult patient intensive care units. Safety rounds were conducted three days a week ascertaining 37 safety measures (grouped into 10 blocks). In each round, 50% of the patients and 50% of the measures were randomized. The impact of this safety tool was analysed on indicators of structure (safety culture, healthcare protocols), process (improvement proportion related to tool application, IPR) and outcome (mortality, average stay, rate of catheter-related bacteraemias and rate of ventilator-associated pneumonia, VAP).

Results

A total of 1214 patient-days were analysed. Structure indicators: the use of the safety tool was associated with an increase in the safety climate and the creation/modification of healthcare protocols (sedation/analgesia and weaning). Process indicators: Twelve of the 37 measures had an IPR > 10%; six showed a progressive decrease in the IPR over the study period. Nursing workloads and patient severity on the day of analysis were independently associated with a higher IPR in half of the blocks of variables. Outcome indicators: A significant decrease in the rate of VAP was observed.

Conclusions

The real-time random safety tool improved the care process and adherence to clinical practice guidelines and was associated with an improvement in structure, process and outcome indicators.

Electronic supplementary material

The online version of this article (doi:10.1186/s13613-017-0245-x) contains supplementary material, which is available to authorized users.

Real-time random safety audits: A transforming tool adapted to new times

Real-time random safety audits constitute a tool designed to transfer knowledge from the sources of scientific evidence to the patient bedside. It has proven useful in critically ill patients, improving safety in the process of critical patient care, turning unsafe situations into safe ones in daily practice, and ensuring adherence to scientific evidence. In parallel, the design and methodology involved affords process indicators that will make it possible to know how we provide care for our patients, evolution over time (with regular feedback for professionals), the impact of our interventions, and benchmarking.