Assessment of bias in scoring of AI-based radiotherapy segmentation and planning studies using modified TRIPOD and PROBAST guidelines as an example

BACKGROUND AND PURPOSE

Studies investigating the application of Artificial Intelligence (AI) in the field of radiotherapy exhibit substantial variations in terms of quality. The goal of this study was to assess the amount of transparency and bias in scoring articles with a specific focus on AI based segmentation and treatment planning, using modified PROBAST and TRIPOD checklists, in order to provide recommendations for future guideline developers and reviewers.

MATERIALS AND METHODS

The TRIPOD and PROBAST checklist items were discussed and modified using a Delphi process. After consensus was reached, 2 groups of 3 co-authors scored 2 articles to evaluate usability and further optimize the adapted checklists. Finally, 10 articles were scored by all co-authors. Fleiss' kappa was calculated to assess the reliability of agreement between observers.

RESULTS

Three of the 37 TRIPOD items and 5 of the 32 PROBAST items were deemed irrelevant. General terminology in the items (e.g., multivariable prediction model, predictors) was modified to align with AI-specific terms. After the first scoring round, further improvements of the items were formulated, e.g., by preventing the use of sub-questions or subjective words and adding clarifications on how to score an item. Using the final consensus list to score the 10 articles, only 2 out of the 61 items resulted in a statistically significant kappa of 0.4 or more demonstrating substantial agreement. For 41 items no statistically significant kappa was obtained indicating that the level of agreement among multiple observers is due to chance alone.

CONCLUSION

Our study showed low reliability scores with the adapted TRIPOD and PROBAST checklists. Although such checklists have shown great value during development and reporting, this raises concerns about the applicability of such checklists to objectively score scientific articles for AI applications. When developing or revising guidelines, it is essential to consider their applicability to score articles without introducing bias.

CheckPOINT: a simple tool to measure Surgical Safety Checklist implementation fidelity

INTRODUCTION

The WHO Surgical Safety Checklist (SSC) is a communication tool that improves teamwork and patient outcomes. SSC effectiveness is dependent on implementation fidelity. Administrative audits fail to capture most aspects of SSC implementation fidelity (ie, team communication and engagement). Existing research tools assess behaviours during checklist performance, but were not designed for routine quality assurance and improvement. We aimed to create a simple tool to assess SSC implementation fidelity, and to test its reliability using video simulations, and usability in clinical practice.

METHODS

The Checklist Performance Observation for Improvement (CheckPOINT) tool underwent two rounds of face validity testing with surgical safety experts, clinicians and quality improvement specialists. Four categories were developed: checklist adherence, communication effectiveness, attitude and engagement. We created a 90 min training programme, and four trained raters independently scored 37 video simulations using the tool. We calculated intraclass correlation coefficients (ICC) to assess inter-rater reliability (ICC>0.75 indicating excellent reliability). We then trained two observers, who tested the tool in the operating room. We interviewed the observers to determine tool usability.

RESULTS

The CheckPOINT tool had excellent inter-rater reliability across SSC phases. The ICC was 0.83 (95% CI 0.67 to 0.98) for the sign-in, 0.77 (95% CI 0.63 to 0.92) for the time-out and 0.79 (95% CI 0.59 to 0.99) for the sign-out. During field testing, observers reported CheckPOINT was easy to use. In 98 operating room observations, the total median (IQR) score was 25 (23-28), checklist adherence was 7 (6-7), communication effectiveness was 6 (6-7), attitude was 6 (6-7) and engagement was 6 (5-7).

CONCLUSIONS

CheckPOINT is a simple and reliable tool to assess SSC implementation fidelity and identify areas of focus for improvement efforts. Although CheckPOINT would benefit from further testing, it offers a low-resource alternative to existing research tools and captures elements of adherence and team behaviours.

A simulation-based randomized trial of ABCDE style cognitive aid for emergency medical services CHecklist In Prehospital Settings: the CHIPS-study

BACKGROUND

Checklists are a powerful tool for reduction of mortality and morbidity. Checklists structure complex processes in a reproducible manner, optimize team interaction, and prevent errors related to human factors. Despite wide dissemination of the checklist, effects of checklist use in the prehospital emergency medicine are currently unclear. The aim of the study was to demonstrate that participants achieve higher adherence to guideline-recommended actions, manage the scenario more time-efficient, and thirdly demonstrate better adherence to the ABCDE-compliant workflow in a simulated ROSC situation.

METHODS

CHIPS was a prospective randomized case-control study. Professional emergency medical service teams were asked to perform cardiopulmonary resuscitation on an adult high-fidelity patient simulator achieving ROSC. The intervention group used a checklist which transferred the ERC guideline statements of ROSC into the structure of the 'ABCDE' mnemonic. Guideline adherence (performance score, PS), utilization of process time (items/minute) and workflow were measured by analyzing continuous A/V recordings of the simulation. Pre- and post-questionnaires addressing demographics and relevance of the checklist were recorded. Effect sizes were determined by calculating Cohen's d. The level of significance was defined at p < 0.05.

RESULTS

Twenty scenarios in the intervention group (INT) and twenty-one in the control group (CON) were evaluated. The average time of use of the checklist (CU) in the INT was 6.32 min (2.39-9.18 min; SD = 2.08 min). Mean PS of INT was significantly higher than CON, with a strong effect size (p = 0.001, d = 0.935). In the INT, significantly more items were completed per minute of scenario duration (INT, 1.48 items/min; CON, 1.15 items/min, difference: 0.33/min (25%), p = 0.001), showing a large effect size (d = 1.11). The workflow did not significantly differ between the groups (p = 0.079), although a medium effect size was shown (d = 0.563) with the tendency of the CON group deviating stronger from the ABCDE than the INT.

CONCLUSION

Checklists can have positive effects on outcome in the prehospital setting by significantly facilitates adherence to guidelines. Checklist use may be time-effective in the prehospital setting. Checklists based on the 'ABCDE' mnemonic can be used according to the 'do verify' approach. Team Time Outs are recommended to start and finish checklists.

Improving surgical safety checklist utilisation at 23 public health facilities in Ethiopia: a collaborative quality improvement project

BACKGROUND

In 2009, the WHO introduced the surgical safety checklist (SSC) as one of the interventions for improving patient safety. The systematic use of structured checklists during surgery has been shown to reduce perioperative morbidity and mortality. However, SSC utilisation has been challenging in low-income and middle-income countries, including Ethiopia. Jhpiego Ethiopia implemented a quality improvement project (QIP) aimed to increase SSC utilisation.

METHODOLOGY

A model for improvement was used to design and implement a collaborative QIP to improve SSC utilisation at 23 public health facilities (13 primary health care facilities, 4 general hospitals and 6 tertiary hospitals) in Ethiopia from October 2020 to September 2021. SSC utilisation was defined as when a patient chart had SSC attached and each part of the checklist was completed. Training of surgical staff on safe surgery packages, monthly clinical mentorship and cluster-based learning platforms were implemented during the study period. We analysed bimonthly chart audit reports from each facility to assess the proportion of surgeries where the SSC was used. Shewhart charts were used to conduct a time-series analysis. Additionally, the Z-test for two sample proportions was used to determine if there is a statistically significant change from the baseline measure with a p<0.05.

RESULT

In the postintervention period, the overall SSC utilisation improved by 39.9 absolute percentage points to 90.3% (p<0.0001) compared with the baseline value of 50.4% early in 2020. A time-series analysis using Shewhart charts showed a shift in the mean performance and signals of special cause variation. The largest improvement was observed in primary health care facilities in which the SSC utilisation improved from 50.8% to 97.9% (p<0.0001).

CONCLUSION

This study demonstrates that onsite clinical capacity building, mentorship and collaborative cluster-based learning platforms can improve SSC utilisation across all levels of facilities performing surgery.

Presentation and publication skills: Publication governance and pitfalls to avoid

There are several pitfalls in the publication process that researchers can fall victim to, and these can occur knowingly or unknowingly. Although some of these errors may have occurred in good faith, disregard of publication governance is a dangerous practice and could bring authors and their co-authors into disrepute. We highlight some of these potential pitfalls, acquaint the reader with some rules that need to be adhered to in research and publishing, and help the reader learn how to avoid tripping-up on the road to publication.

Applying Crew Resource Management tools in Emergency Response Driving and patient transport-Finding consensus through a modified Delphi study

BACKGROUND

Emergency Response Driving (ERD) comprises a significant risk to safety in Emergency Medical Services (EMS). Crew Resource Management (CRM) tools play a major role in securing actions in high-risk procedures. The aim of this study was to find consensus on the important factors to consider when applying CRM tools in ERD and patient transport.

METHODS

ERD experts (n = 50) were recruited for a modified three-round Delphi study. Round 1 was based on previous research. The experts evaluated the items as important, neutral, or not important. The predetermined level of consensus was set at ≥ 80%. Answers given to the open-ended questions were analyzed using inductive content analysis.

RESULTS

Predetermined consensus was reached on 64 of 86 presented items (74.4 %). The mean values of items reaching consensus varied between 3.81 and 4.86 on a five-point Likert scale. The items where consensus was reached were rated as "important" on a trichotomized scale.

CONCLUSION

Multiple important factors to consider when applying CRM tools to ERD and patient transport were highlighted. This study provides valuable information to consider regarding EMS safety improvements. Further scientific research is needed to develop comprehensive recommendations.

Pilot study of an interprofessional pediatric mechanical ventilation educational initiative in two intensive care units

INTRODUCTION

Inappropriate ventilator settings, non-adherence to a lung-protective ventilation strategy, and inadequate patient monitoring during mechanical ventilation can potentially expose critically ill children to additional risks. We set out to improve team theoretical knowledge and practical skills regarding pediatric mechanical ventilation and to increase compliance with treatment goals.

METHODS

An educational initiative was conducted from August 2019 to July 2021 in a neonatal and pediatric intensive care unit of the University Children's Hospital, Hamburg-Eppendorf, Germany. We tested baseline theoretical knowledge using a multiple choice theory test (TT) and practical skills using a practical skill test (PST), consisting of four sequential Objective Structured Clinical Examinations of physicians and nurses. We then implemented an educational bundle that included video self-training, checklists, pocket cards, and reevaluated team performance. Ventilators and monitor settings were randomly checked in all ventilated patients. We used a process control chart and a mixed-effects model to analyze the primary outcome.

RESULTS

A total of 47 nurses and 20 physicians underwent assessment both before and after the implementation of the initiative using TT. Additionally, 34 nurses and 20 physicians were evaluated using the PST component of the initiative. The findings revealed a significant improvement in staff performance for both TT and PST (TT: 80% [confidence interval (CI): 77.2-82.9] vs. 86% [CI: 83.1-88.0]; PST: 73% [CI: 69.7-75.5] vs. 95% [CI: 93.8-97.1]). Additionally, there was a notable increase in self-confidence among participants, and compliance with mechanical ventilation treatment goals also saw a substantial rise, increasing from 87.8% to 94.5%.

DISCUSSION

Implementing a pediatric mechanical ventilation education bundle improved theoretical knowledge and practical skills among interprofessional pediatric intensive care staff and increased treatment goal compliance in ventilated children.

History information management strategy for minimising biases and noise for improved medical diagnosis

Despite measures for physicians' excellence in diagnosis, the need for improvement of medical history techniques has been pointed out as one of the critical elements for improving diagnosis. Specific and proactive frameworks related to methods of effective history acquisition are needed to minimise bias and optimise decision-making. Therefore, this paper uses Linear Sequential Unmasking- Expanded to develop and propose a structured medical history acquisition strategy. The strength of this lies in its reliance on cognitive psychological processes. Breaking information gatherings and decisions into smaller tasks and ordering them correctly reduces cognitive load as well as minimises noise and bias cascade. Additionally, this approach can help physicians develop diagnostic expertise regardless of specialty.

Methods and results of studies on reporting guideline adherence are poorly reported: a meta-research study

OBJECTIVES

We investigated recent meta-research studies on adherence to four reporting guidelines to determine the proportion that provided (1) an explanation for how adherence to guideline items was rated and (2) results from all included individual studies. We examined conclusions of each meta-research study to evaluate possible repetitive and similar findings.

STUDY DESIGN AND SETTING

A cross-sectional meta-research study. MEDLINE (Ovid) was searched on July 5, 2022 for studies that used any version of the Consolidated Standards of Reporting Trials, Preferred Reporting Items for Systematic Reviews and Meta-Analyses, Standards for the Reporting of Diagnostic Accuracy Studies, or Strengthening the Reporting of Observational Studies in Epidemiology reporting guidelines or their extensions to evaluate reporting.

RESULTS

Of 148 included meta-research studies published between August 2020 and June 2022, 14 (10%, 95% confidence interval [CI] 6%-15%) provided a fully replicable explanation of how they coded the adherence ratings and 49 (33%, 95% CI 26%-41%) completely reported individual study results. Of 90 studies that classified reporting as adequate or inadequate in the study abstract, six (7%, 95% CI 3%-14%) concluded that reporting was adequate, but none of those six studies provided information on how items were coded or provided item-level results for included studies.

CONCLUSION

Almost all included meta-research studies found that reporting in health research is suboptimal. However, few of these reported enough information for verification or replication.

Rebreather Forum Four consensus statements

Closed circuit rebreathers have been widely adopted by technical divers as tools for reducing gas consumption and extending depth and duration capabilities. Rebreathers are technologically complex with many failure points, and their use appears associated with a higher accident rate than open circuit scuba. Rebreather Forum Four (RF4) was held in Malta in April 2023 attracting approximately 300 attendees and representatives of multiple manufacturers and training agencies. Over two and a half days a series of lectures was given by influential divers, engineers, researchers and educators on topics of contemporary relevance to rebreather diving safety. Each lecture was followed by a discussion session with audience participation. Potential consensus statements were drafted by the authors (SJM and NWP) during the course of the meeting. These were worded to be confluent with some important messages emerging from the presentations and subsequent discussions. The statements were presented one by one in a half-day plenary session of participants, and discussion was invited on each. After discussion and any necessary revision, the participants voted on whether to adopt the statement as a position of the forum. A clear majority was required for acceptance. Twenty-eight statements embracing thematic areas designated 'safety', 'research', 'operational issues', 'education and training', and 'engineering' were adopted. Those statements are presented along with contextualising narrative where necessary. The statements may help shape research and teaching initiatives, and research and development strategies over subsequent years.

Authors arbitrarily used methodological approaches to analyze the quality of reporting in research reports: a meta-research study

OBJECTIVES

Many authors used reporting checklists as an assessment tool to analyze the reporting quality of diverse types of evidence. We aimed to analyze methodological approaches used by researchers assessing reporting quality of evidence in randomized controlled trials, systematic reviews, and observational studies.

STUDY DESIGN AND SETTING

We analyzed articles reporting quality assessment of evidence with Preferred Reporting Items of Systematic Reviews and Meta-Analyses (PRISMA), CONsolidated Standards of Reporting Trials (CONSORT), or the Strengthening the Reporting of Observational studies in Epidemiology (STROBE) checklists published up to 18 July 2021. We analyzed methods used for assessing reporting quality.

RESULTS

Among 356 analyzed articles, 293 (88%) investigated a specific thematic field. The CONSORT checklist (N = 225; 67%) was most often used, in its original, modified, partial form, or its extension. Numerical scores were given for adherence to checklist items in 252 articles (75%), of which 36 articles (11%) used various reporting quality thresholds. In 158 (47%) articles, predictors of adherence to reporting checklist were analyzed. The most studied factor associated with adherence to reporting checklist was the year of article publication (N = 82; 52%).

CONCLUSION

The methodology used for assessing reporting quality of evidence varied considerably. The research community needs a consensus on a consistent methodology for assessing the quality of reporting.

Persisting high rates of omissions during anesthesia induction are decreased by utilization of a pre- & post-induction checklist

INTRODUCTION

Although Checklists (CL) for routine anesthesia cases have demonstrated their values in various studies, they have found little traction so far. While several reports have shown the benefit of CL preventing omissions prior to anesthesia induction, no investigation yet has scrutinized omissions during the post-induction phase immediately after intubation. This study evaluated the rate of omissions prior to and following the induction of non-emergent general anesthesia, as well as the impact of checklists on omission prevention.

METHODS

We performed a monocentric, prospective, observational study during induction of general anesthesia cases. We evaluated the omission rate made for the pre- as well as the immediate post-induction phase and determined the impact of pre-and post-induction CL on the rate of omission corrections. The CL used were introduced two years prior to the study. The observed providers were limited to those familiar with the institutional CL. Usage of CL was not mandated.

RESULTS

237 general anesthesia inductions were included in the observation. At least one omission in 32% of all cases in the pre-induction setup was found and in 40% within the immediate post-induction phases. CL significantly reduced omission rates (relative risk = 0.64, 95% CI = 0.45-0.92, p = 0.01).

CONCLUSION

Omission rates during the pre- and post-induction phases of routine general anesthesia procedures remain high. Pre- and post-induction CL have the potential to increase patient safety and should be considered for routine anesthesia with appropriate training provided.

Simulation-based training in obstetric anesthesia: an update

In this update we explore the current applications of simulation in obstetric anesthesia, describe what is known regarding its impacts on care and consider the different settings in which simulation programs are required. We will introduce practical strategies, such as cognitive aids and communication tools, that can be applied in the obstetric setting and share ways in which a program might apply these tools. Finally, we provide a list of common obstetric emergencies essential for a program's curriculum and common teamwork pitfalls to address within a comprehensive obstetric anesthesia simulation program.

Perioperative variable rate intravenous insulin infusion: a quality improvement project on a vascular surgery ward

AIMS

Variable rate intravenous insulin infusion (VRIII) is used perioperatively to maintain normoglycaemia in patients with diabetes who are undergoing surgery. The aims of this project were as follows: (1) to audit the extent to which perioperative prescribing of VRIII for diabetic vascular surgery inpatients at our hospital meets established standards and (2) to use the results of the audit to guide improvement in the quality and safety of prescribing practices and reduce VRIII overuse.

METHODS

Vascular surgery inpatients who had perioperative VRIII were included in the audit. Baseline data were collected consecutively from September to November 2021. There were three main interventions: a VRIII Prescribing Checklist, education of junior doctors and ward staff, and electronic prescribing system updates. Postintervention and reaudit data were collected consecutively from March to June 2022.

RESULTS

The number of VRIII prescriptions totalled 27 in preintervention, 18 in postintervention and 26 in reaudit periods. Prescribers used the 'refer to paper chart' safety check more frequently postintervention (67%) and on reaudit (77%) compared with preintervention (33%) (p=0.046). Rescue medication was prescribed in 50% of postintervention and 65% of reaudit cases compared with 0% preintervention (p<0.001). Intermediate/long-acting insulin was appropriately amended more often in the postintervention versus preintervention period (75% vs 45%, p=0.041). Overall, VRIII was appropriate for the situation in 85% of cases.

CONCLUSIONS

The quality of perioperative VRIII prescribing practices improved following the proposed interventions, with prescribers more frequently using recommended safety measures such as 'refer to paper chart' and rescue medication. There was a marked sustained improvement in prescriber-initiated adjustment of oral diabetes medications and insulins. VRIII is occasionally administered unnecessarily in a subset of patients with type 2 diabetes and may be an area for further study.

Improving patient satisfaction for patients with acute neurological symptoms by increased flow from the front door of hospital and more specific documentation in medical record notes: a quality improvement project

Patient experience is considered essential in evaluating healthcare quality. One of the most important parameters that influence patient satisfaction is perception of throughput time, defined as the time from hospital entrance to time of discharge. Throughput issues often start in the emergency department. This often contributes to waiting time for patients and task accumulation for staff. Our overall aim was to optimise throughput in a patient-centred manner for acute neurological patients arriving in the emergency department. We found two primary drivers for change: faster admission to the neurological subunit of the emergency department and improved documentation of three specific topics in the medical records (specific tentative diagnosis, specific treatment plan after CT/MRI and specifically addressing time of expected discharge). Using the plan-do-study-act method, we facilitated successfully changes through education, one-to-one talks, feedback, checklists and by drawing attention to the project. Patients admitted to the hospital after telephonic contact between admitting physician and neurologist arrived in the subunit with a delay of 34 min after arriving at the hospital compared with 89 min before the interventions. Patients unknown to the neurologist before arrival to the hospital arrived at the subunit with a delay of 107 min compared with 130 min before the intervention. The compliance with addressing each of three topics in the medical records showed a significant increase from a median of, respectively, 62%-100%, 45%-82% and 28%-72%. The project goal was achieved, as an increase in patient satisfaction of 27% from the baseline survey was seen, as well as a reduction in the proportion of patients mentioning waiting time in a negative way from 45% to 10%. This demonstrates that a low-cost structured approach can change the way doctors work, for the benefit of patients and staff in the emergency department.

Prospective application of the interdisciplinary bedside rounding checklist 'TEMP' is associated with reduced infections and length of hospital stay

Protocols that enhance communication between nurses, physicians and patients have had a variable impact on the quality and safety of patient care. We combined standardised nursing and physician interdisciplinary bedside rounds with a mnemonic checklist to assure all key nursing care components were modified daily. The mnemonic TEMP allowed the rapid review of 11 elements. T stands for tubes assuring proper management of intravenous lines and foleys; E stands for eating, exercise, excretion and sleep encouraging a review of orders for diet, exercise, laxatives to assure regular bowel movements, and inquiry about sleep; M stands for monitoring reminding the team to review the need for telemetry and the frequency of vital sign monitoring as well as the need for daily blood tests; and P stands for pain and plans reminding the team to discuss pain medications and to review the management plan for the day with the patient and family. Faithful implementation eliminated central line-associated bloodstream infections and catheter-associated urinary tract infections and resulted in a statistically significant reduction in average hospital length of stay of 13.3 hours, one unit achieving a 23-hour reduction. Trends towards reduced 30-day readmissions (20% down to 10%-11%) were observed. One unit improved the percentage of patients who reported nurses and doctors always worked together as a team from a 56% baseline to 75%. However, the combining of both units failed to demonstrate statistically significant improvement. Psychologists well versed in implementing behavioural change were recruiting to improve adherence to our protocols. Following training physicians and nurses achieved adherence levels of over 70%. A high correlation (r²=0.69) between adherence and reductions in length of stay was observed emphasising the importance of rigorous training and monitoring of performance to bring about meaningful and reliable improvements in the efficiency and quality of patient care.

Quality appraisal for systematic literature reviews of health state utility values: a descriptive analysis

BACKGROUND

Health state utility values (HSUVs) are an essential input parameter to cost-utility analysis (CUA). Systematic literature reviews (SLRs) provide summarized information for selecting utility values from an increasing number of primary studies eliciting HSUVs. Quality appraisal (QA) of such SLRs is an important process towards the credibility of HSUVs estimates; yet, authors often overlook this crucial process. A scientifically developed and widely accepted QA tool for this purpose is lacking and warranted.

OBJECTIVES

To comprehensively describe the nature of QA in published SRLs of studies eliciting HSUVs and generate a list of commonly used items.

METHODS

A comprehensive literature search was conducted in PubMed and Embase from 01.01.2015 to 15.05.2021. SLRs of empirical studies eliciting HSUVs that were published in English were included. We extracted descriptive data, which included QA tools checklists or good practice recommendations used or cited, items used, and the methods of incorporating QA results into study findings. Descriptive statistics (frequencies of use and occurrences of items, acceptance and counterfactual acceptance rates) were computed and a comprehensive list of QA items was generated.

RESULTS

A total of 73 SLRs were included, comprising 93 items and 35 QA tools and good recommendation practices. The prevalence of QA was 55% (40/73). Recommendations by NICE and ISPOR guidelines appeared in 42% (16/40) of the SLRs that appraised quality. The most commonly used QA items in SLRs were response rates (27/40), statistical analysis (22/40), sample size (21/40) and loss of follow up (21/40). Yet, the most commonly featured items in QA tools and GPRs were statistical analysis (23/35), confounding or baseline equivalency (20/35), and blinding (14/35). Only 5% of the SLRS used QA to inform the data analysis, with acceptance rates of 100% (in two studies) 67%, 53% and 33%. The mean counterfactual acceptance rate was 55% (median 53% and IQR 56%).

CONCLUSIONS

There is a considerably low prevalence of QA in the SLRs of HSUVs. Also, there is a wide variation in the QA dimensions and items included in both SLRs and extracted tools. This underscores the need for a scientifically developed QA tool for multi-variable primary studies of HSUVs.

Use of 'Printed Investigation Sheet' as Checklist in Admitted Antenatal Patients: A Quality Improvement Initiative

Background

The study was conducted to establish use of printed investigation sheets as checklists for timely workup and clinical evaluation of antenatal women with medical diseases; admitted in maternity ward, by third day of their hospital admission. This was aimed to standardize care, avoid repeated blood sampling of patients, avoid delay in starting the treatment and help teams perform optimally by systematic use of quality improvement (QI) tools.

Methods

The present study was conducted in the Department of Obstetrics and Gynaecology at a tertiary care teaching hospital using point-of-care quality improvement methodology systematically. A QI team was made who formulated an aim statement, conducted a root-cause analysis, performed plan-do-study-act (PDSA) cycles. The outcome was measured as complete clinical evaluation of antenatal women with anaemia, hypertension, and/or diabetes by third day of admission in the maternity ward.

Results

The baseline data showed that median percentage of patients with complete clinical evaluation was only 29.2%. After a root-cause analysis with fishbone tool, three PDSA cycles were conducted to achieve the target of 80%. After the third PDSA cycle, complete clinical evaluation in anaemia, hypertension, diabetes showed an improving trend with a median of 75%.

Conclusion

Adopting simple principles of quality improvement, initiating use of printed investigation sheets as checklist can streamline and expedite clinical evaluation of antenatal patients with medical problems so as to avoid unnecessary delay in initiating the management in busy maternity wards.

Checklists in Healthcare: Operational Improvement of Standards using Safety Engineering - Project CHOISSE - A framework for evaluating the effects of checklists on surgical team culture

The CHOISSE multi-stage framework for evaluating the effects of electronic checklist applications (e-checklists) on surgical team members' perception of their roles, performance, communication, and understanding of checklists is introduced via a pilot study. A prospective interventional cohort study design was piloted to assess the effectiveness of the framework and the sociotechnical effects of the e-checklist. A Delphi process was used to design the stages of the framework based on literature and expert consensus. The CHOISSE framework was applied to guide the implementation and evaluation of e-checklists on team culture for ten pilot teams across the US over a 24-week period. The pilot results revealed more engagement by surgeons than non-surgeons, and significant increases in surgeons' perception of communication and engagement during surgery with a small sample. Mixed methods analysis of the data and lessons learned were used to identify iterative improvements to the CHOISSE framework and to inform future studies.

Surgical safety checklists for dental implant surgeries-a scoping review

OBJECTIVES

In both elective surgeries and aviation, a reduction of complications can be expected by paying attention to the so-called human factors. Checklists are a well-known way to overcome some of these problems. We aimed to evaluate the current evidence regarding the use of checklists in implant dentistry.

METHODS

An electronic literature search was conducted in the following databases: CINHAL, Medline, Web of Science, and Cochrane Library until March 2022. Based on the results and additional literature, a preliminary checklist for surgical implant therapy was designed.

RESULTS

Three publications dealing with dental implants and checklists were identified. One dealt with the use of a checklist in implant dentistry and was described as a quality assessment study. The remaining two studies offered suggestions for checklists based on literature research and expert opinion.

CONCLUSIONS

Based on our results, the evidence for the use of checklists in dental implantology is extremely low. Considering the great potential, it can be stated that there is a need to catch up. While creating a new implant checklist, we took care of meeting the criteria for high-quality checklists. Future controlled studies will help to place it on a broad foundation.

CLINICAL RELEVANCE

Checklists are a well-known way to prevent complications. They are especially established in aviation, but many surgical specialties and anesthesia adopt this successful concept. As implantology has become one of the fastest-growing areas of dentistry, it is imperative that checklists become an integral part of it.

Comparing compliance with the WHO surgical safety checklist and complication rates in gynecologic surgery between day and night shifts

Purpose

At least half of surgical complications can be avoided by using surgical checklists. However, universal implementation and compliance have been reported as being variable. Patients undergoing urgent surgical intervention are at increased risk for complications. The aim of this study was to evaluate the checklist compliance together with the complication rate during day and night shifts in a European University hospital.

Methods

51 and 52 consecutive patients who had surgery during day and night shifts were included. The primary outcome measures were compliance and completeness of the WHO safety checklist. The occurrence of postoperative complications was investigated.

Results

The analysis included 103 surgical procedures. The mean compliance rate of use was 93% and the mean completeness rate was 22%. After operations were broken down by day or night shift, we found that checklists were less often available in night shifts compared to day shifts. The completeness of the checklist and the occurrence of postoperative complications did not differ between day and night shifts.

Conclusion

This study reports worse checklists availability in night shifts when compared to day shifts, but complication rates did not increase. Further studies are warranted to investigate postoperative complication rates together with checklist compliance in day versus night shifts.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00404-022-06599-w.

Sustaining immediate newborn care processes (delayed cord clamping and early breastfeeding initiation) in the delivery room: a quality improvement study

BACKGROUND

Immediate newborn care processes like delayed cord clamping (DCC) and early breastfeeding initiation (EBFI) in the delivery room have several benefits including survival. Despite the evidence, the practices have not been widely adopted. We used a point-of-care quality improvement (QI) to implement and sustain these two immediate newborn care processes in our delivery room over a period of 2 years through a series of plan-do-study-act (PDSA) cycles.

METHODS

All neonates above 30 weeks of gestation irrespective of the need for resuscitation except Rh-isoimmunisation were eligible for DCC. Neonates >35 weeks not requiring respiratory support or resuscitation were eligible for EBFI. The root causes of gaps in the quality were analysed by fishbone analysis. The key quantitative outcome measure was the percentage of eligible deliveries in which DCC and EBFI were done. Duration of DCC was also recorded in the sustenance phase. This implementation was done through three PDSA cycles and the practices were sustained for 2 years.

RESULTS

A total of 770 deliveries were part of this QI study from October 2018 to December 2020. There was a significant improvement in DCC (median) from a baseline of 25% to 96% over a 2-year period. Sensitisation, making DCC part of pre-birth checklist and recording outcomes on a dashboard daily helped to implement and sustain the processes over 2 years. As a co-process, EBFI improved (median) from a baseline of 50% to 97% without any major intervention in the system.

CONCLUSIONS

Immediate newborn care processes could be sustained by making them part of pre-birth preparation and dashboard recording by a QI initiative without any additional resources.

Adherence to the cardiac surgery checklist decreased mortality at a teaching hospital: A retrospective cohort study

OBJECTIVE

To evaluate the impact of adherence to the cardiac surgical checklist on mortality at the teaching hospital.

METHODS

A retrospective cohort study after the implementation of the cardiac surgical safety checklist in a reference hospital in Latin America. All patients undergoing coronary artery bypass surgery and/or heart valve surgery from 2013 to 2019 were analyzed. After the implementation of the project InCor-Checklist "Five steps to safe cardiac surgery" in 2015, the correlation between adherence and completeness of this instrument with surgical mortality was assessed. The EuroSCORE II was used as a reference to assess the risk of expected mortality for patients. Cross-sectional questionnaires were during the implementation of the InCor-Checklist. To perform the correlation, Pearson's coefficient was calculated using R software.

RESULTS

Since 2013, data from 8139 patients have been analyzed. The average annual mortality was 5.98%. In 2015, the instrument was used in only 58% of patients; in contrast, it was used in 100% of patients in 2019. There was a decrease in surgical mortality from 8.22% to 3.13% for the same group of procedures. The results indicate that the greater the checklist use, the lower the surgical mortality (r = 88.9%). In addition, the greater the InCor-Checklist completeness, the lower the surgical mortality (r = 94.1%).

CONCLUSION

In the formation of the surgical patient safety culture, the implementation and adherence to the InCor-Checklist "Five steps to safe cardiac surgery" was associated with decreased mortality after cardiac surgery.

Toward more mindful reporting of patient and public involvement in healthcare

Understanding of the value of patient and public involvement in research has grown in recent years, but so too has uncertainty about how best to practice and how best to report such involvement in research outputs. One way proposed to report such involvement is through checklists, such as the GRIPP2, which aims to improve quality, transparency, and consistency in such reporting. We critique the unproblematised use of such a tool because of two main concerns. First, being asked to complete a GRIPP2 for a recent publication felt divisive given that the service user researcher was as much a member of the authorship team as the other researchers (whose involvement did not necessitate a checklist). Second, checklists do not actually address the power imbalances and tokenism that is rife in patient and public involvement in research. Indeed, the false sense of objectivity fostered by meeting the minimum requirements of the checklist means that researchers may not go further to engage in reflexive research practices and reporting. Rather than rote use of such checklists, we recommend mindful reflexive reporting in research outputs of patient and public involvement processes. We also recommend future iterations of the GRIPP consider (a) incorporating criteria about whether the checklist is completed by or with service user researchers or not, (b) addressing criteria that position service user research as needing to be justified, and (c) expanding the "critical perspective" element of the checklist to explicitly consider power differentials.

Medical physics external beam plan review: What contributes to the variability?

PURPOSE

In this article we report on the results of a survey of physics plan review practices conducted by the Cancer Care Ontario Communities of Practice and the variations in practice between and within centers.

METHODS

The medical physicists at each center worked together to complete the survey and submit a single response for that center. A 4-point Likert scale, used to report the variation in practice at each center, was quantified into two parameters: "Intra-center variation", the distribution of responses within the center, and "Variation between centers", the difference between the center's response and the provincial mean. These metrics were correlated with center characteristics to identify factors that impacted on variations in practice.

RESULTS

Bolus and heterogeneity correction were the only two items checked by all physicists in all centers. In more than half of the centers, image registration and DVH binning are not likely checked by physics. A significant difference in the variation between centers is observed for centers that used a single vendor's products. Centers that used an official checklist indicated higher levels and a wider range of Intra-center variation. Higher workload did not affect the variation in checking patterns between physicists in the same center.

CONCLUSIONS

The effect of a center's resources on their checking practice suggest that local environment and workflow be accounted for when implementing TG275 guidelines. The observation that standardized checklists did not reduce checking variability point to the importance of following the checklist development guidelines in MPPG4 to avoid ineffective checklists.

The use of a cognitive aid app supports guideline-conforming cardiopulmonary resuscitations: A randomized study in a high-fidelity simulation

Aim

Cardiac arrests require fast, well-timed, and well-coordinated interventions delivered by several staff members. We evaluated a cognitive aid that works as an attentional aid to support specifically the timing and coordination of these interventions. We report the results of an experimental, simulation-based evaluation of the tablet-based cognitive aid in performing guideline-conforming cardiopulmonary resuscitation.

Methods

In a parallel group design, emergency teams (one qualified emergency physician as team leader and one qualified nurse) were randomly assigned to the cognitive aid application (CA App) group or the no application (No App) group and then participated in a simulated scenario of a cardiac arrest. The primary outcome was a cardiopulmonary resuscitation performance score ranging from zero to two for each team based on the videotaped scenarios in relation to twelve performance variables derived from the European Resuscitation Guidelines. As a secondary outcome, we measured the participants’ subjective workload.

Results

A total of 67 teams participated. The CA App group (n = 32 teams) showed significantly better cardiopulmonary resuscitation performance than the No App group (n = 31 teams; mean difference = 0.23, 95 %CI = 0.08 to 0.38, p = 0.002, d = 0.83). The CA App group team leaders indicated significantly less mental and physical demand and less effort to achieve their performance compared to the No App group team leaders.

Conclusions

Among well-trained in-hospital emergency teams, the cognitive aid could improve cardiopulmonary resuscitation coordination performance and decrease mental workload.

Work-system interventions in robotic-assisted surgery: a systematic review exploring the gap between challenges and solutions

BACKGROUND

The introduction of a robot into the surgical suite changes the dynamics of the work-system, creating new opportunities for both success and failure. An extensive amount of research has identified a range of barriers to safety and efficiency in Robotic Assisted Surgery (RAS), such as communication breakdowns, coordination failures, equipment issues, and technological malfunctions. However, there exists very few solutions to these barriers. The purpose of this review was to identify the gap between identified RAS work-system barriers and interventions developed to address those barriers.

METHODS

A search from three databases (PubMed, Web of Science, and Ovid Medline) was conducted for literature discussing system-level interventions for RAS that were published between January 1, 1985 to March 17, 2020. Articles describing interventions for systems-level issues that did not involve technical skills in RAS were eligible for inclusion.

RESULTS

A total of 30 articles were included in the review. Only seven articles (23.33%) implemented and evaluated interventions, while the remaining 23 articles (76.67%) provided suggested interventions for issues in RAS. Major barriers identified included disruptions, ergonomic issues, safety and efficiency, communication, and non-technical skills. Common solutions involved team training, checklist development, and workspace redesign.

CONCLUSION

The review identified a significant gap between issues and solutions in RAS. While it is important to continue identifying how the complexities of RAS affect operating room (OR) and team dynamics, future work will need to address existing issues with interventions that have been tested and evaluated. In particular, improving RAS-associated non-technical skills, task management, and technology management may lead to improved OR dynamics associated with greater efficiency, reduced costs, and better systems-level outcomes.

A Checklist to Assess Childbearing Intentions and Promote Referral to Preconception Care or Contraception: A Multi-Site Study

INTRODUCTION

This study assesses HIV provider views on the value of a checklist designed to assess patients' preconception care (PCC) needs and guide implementation of PCC.

METHODS

Ninety-two HIV providers in seven U.S. cities provided perspectives via an in-depth phone interview regarding a checklist to facilitate communication and referrals for PCC. A sub-sample of 27 providers shared feedback on a checklist designed for this purpose. Interview audio files were transcribed and uploaded to a web-based program supporting coding and analysis of qualitative data. Content analysis was utilized to identify key themes within the larger, a priori themes of interest. Feedback regarding the checklist was analyzed using a grounded theory approach to examine patterns and emergent themes across transcripts.

RESULTS

Providers averaged 11.5 years of HIV treatment experience; over 80 percent were physicians (MD) or nurse practitioners (NP) and 76 percent were HIV/infectious disease specialists. The majority of providers were female (70%) and Caucasian (72%). Checklist benefits identified included standardization of care, assisting new/inexperienced providers, educational resource for patients, and aid in normalizing childbearing. Concerns included over-protocolizing care, interfering with patient-provider communication, or requiring providers address non-priority issues during visits. Providers suggested checklists be simple, incorporated into the electronic medical record, and accompanied with appropriate referral systems.

DISCUSSION

Findings support a need for a checklist tool to assist in conversations about reproductive intentions/desires. Additional referral or innovative consultative services will be needed as more persons living with HIV/AIDS are engaged on the topic of childbearing.

Reducing resource utilization for patients with uncomplicated appendicitis through use of same-day discharge and elimination of postoperative antibiotics

BACKGROUND

There is controversy over certain aspects of post-appendectomy care for children with uncomplicated appendicitis. Some institutions have embraced the practice of same-day discharge after appendectomy, while others are hesitant due to concerns about increased readmissions or emergency department (ED) visits. Similarly, some surgeons have transitioned to treating gangrenous appendicitis with a single perioperative dose, while others are concerned about increased risk of infection in this population.

METHODS

We developed a pathway for the management of patients undergoing appendectomy for uncomplicated acute appendicitis which included same-day discharge and elimination of postoperative antibiotics for patients with gangrenous appendicitis. We compared outcomes for children treated at our institution before and after implementation of the protocol.

RESULTS

We identified 575 patients undergoing appendectomy for uncomplicated appendicitis (307 pre- and 268 post-protocol). We observed a significant decrease in postoperative length-of stay (10.6 to 2.6 h, p < 0.0001). There were no increases in postoperative complications, such as superficial (2.6% vs 1.1%, p = 0.19) or organ-space surgical-site infection (1.6% vs 0.4%, p = 0.14), percutaneous drain placement (1.3% vs 0%, p = 0.06), postoperative ED visits (5.5% vs 5.2%, p = 0.87) or readmission (3.3% vs 1.5%, p = 0.17).

CONCLUSIONS

These findings suggest that incorporating same-day discharge for simple appendicitis and eliminating postoperative antibiotics for children with gangrenous appendicitis does not increase complication rates. Implementation of similar pathways across institutions has the potential to significantly reduce resource utilization for children undergoing appendectomy for uncomplicated appendicitis.

TYPE OF STUDY

Retrospective comparative study.

LEVEL OF EVIDENCE

Level III.

Potentially traumatic experiences and behavioural symptoms in adults with autism and intellectual disability referred for psychiatric assessment

BACKGROUND

Individuals with autism spectrum disorder (ASD) and intellectual disability (ID) more frequently experience potentially traumatic events (PTEs), and may be more vulnerable to trauma-related symptoms. However, it is unclear how such symptoms are captured on tools used for behavioural and psychiatric assessment in this population.

AIMS

To explore whether and how PTEs are associated with symptom reports in adults with ASD and ID.

METHODS AND PROCEDURES

Associations and group differences for death of a close relative and serious disease/injury in a close relative/caregiver/friend were explored in a clinical sample of 171 adults with ASD and ID referred for psychiatric assessment. Symptoms were measured using Aberrant Behavior Checklist (ABC) and Psychopathology in Autism Checklist (PAC).

OUTCOMES AND RESULTS

Disease/injury was associated with higher scores on ABC irritability, ABC hyperactivity and self-injurious behaviour. Death was associated with lower scores on ABC lethargy and ABC stereotypic behaviour. Some associations reached significance only when controlling for ASD, ID, or verbal language skills, but the identified associations were not robust. No associations were found for PAC.

CONCLUSIONS AND IMPLICATIONS

There is a risk of under-appreciating the impact of PTEs in this population unless ASD, ID and verbal language skills are taken into account.

Cognitive Aids in the Management of Critical Events

Critical events are rare and stressful. These properties make reliance on memory for clinical management highly susceptible to failure. In the past 10 to 20 years, health care has begun to accept the experience of aviation and other high-reliability organizations in addressing failure to rescue from these events through a combination of practice through simulation and the introduction of cognitive aids, known as checklists or emergency manuals. Cognitive aids have a persuasive body of evidence from simulation studies to establish their value in improving clinician performance. However, their introduction to practice is more complex than distribution of the tools.

Testing the effects of checklists on team behaviour during emergencies on general wards: An observational study using high-fidelity simulation

INTRODUCTION

Clinical teams struggle on general wards with acute management of deteriorating patients. We hypothesized that the Crisis Checklist App, a mobile application containing checklists tailored to crisis-management, can improve teamwork and acute care management.

METHODS

A before-and-after study was undertaken in high-fidelity simulation centres in the Netherlands, Denmark and United Kingdom. Clinical teams completed three scenarios with a deteriorating patient without checklists followed by three scenarios using the Crisis Checklist App. Teamwork performance as the primary outcome was assessed by the Mayo High Performance Teamwork scale. The secondary outcomes were the time required to complete all predefined safety-critical steps, percentage of omitted safety-critical steps, effects on other non-technical skills, and users' self-assessments. Linear mixed models and a non-parametric survival test were conducted to assess these outcomes.

RESULTS

32 teams completed 188 scenarios. The Mayo High Performance Teamwork scale mean scores improved to 23.4 out of 32 (95% CI: 22.4-24.3) with the Crisis Checklist App compared to 21.4 (20.4-22.3) with local standard of care. The mean difference was 1.97 (1.34-2.6; p < 0.001). Teams that used the checklists were able to complete all safety-critical steps of a scenario in more simulations (40/95 vs 21/93 scenarios) and these steps were completed faster (stratified log-rank test χ² = 8.0; p = 0.005). The self-assessments of the observers and users showed favourable effects after checklist usage for other non-technical skills including situational awareness, decision making, task management and communication.

CONCLUSIONS

Implementation of a novel mobile crisis checklist application among clinical teams was associated in a simulated general ward setting with improved teamwork performance, and a higher and faster completion rate of predetermined safety-critical steps.

Supporting effective participation in health guideline development groups: The Guideline Participant Tool

OBJECTIVES

Health guidelines are a key knowledge translation tool produced and used by numerous stakeholders worldwide. Effective participation in guideline development groups or development groups is crucial for guideline success, yet little guidance exists for members of these groups. In this study, we present the Guideline Participant Tool (GPT) to support effective participation in guideline groups, in particular those using the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) approach.

STUDY DESIGN AND SETTING

We used a mixed methods and iterative approach to develop a tool to support guideline participation. We used the findings of a published systematic review to develop an initial list of items for considerations for guideline participants. Then, we refined this list through key informant interviews with guideline chairs, sponsors, and participants. Finally, we validated the GPT in three guideline groups with 26 guideline group members.

RESULTS

The initial list of items based on 37 articles from the existing systematic review included 15 themes and 61 items for a draft tool. Ten key informant interviews helped us refine the list to include the following themes: selection of participants, guideline group process, and tool format. 26 respondents completed the validation survey from three guideline groups. Refinement of the tool ultimately generated a GPT with 33 items for participant consideration before, during, and in follow-up to guideline group meetings.

CONCLUSION

The GPT contains helpful guidance for all guideline participants, particularly those without previous guideline experience. Future research should further explore the need for additional tools to support guideline participants and identify and develop strategies for improving guideline members' participation in guideline groups. This work will be incorporated into INGUIDE.org guideline training and credentialing efforts by the Guidelines International Network and McMaster University.

Improving the surgical consenting process for patients with acute hip fractures: a pilot quality improvement project

Background

Consenting patients for trauma procedures following hip fracture is a key stage in the treatment pathway from admission to the operating theatre. Errors in this process can result in delayed procedures which may negatively impact patient recovery. The aim of this project was to identify and reduce errors in our consenting process for patients with capacity.

Methods

Consent forms for all adult patients with capacity admitted for surgical repair of traumatic hip fracture were reviewed over a 4-week period. The baseline measurement (n = 24), identified errors in three key process measures: clarity of documentation, failure to record procedure-specific risks and not offering a copy of the consent form to the patient. Pre-printed stickers and targeted teaching were then introduced as quality improvement measures. Their impact was evaluated over subsequent 4-week review of the same patient demographic, with further refinement of these interventions being carried out and re-evaluated for a final cycle.

Results

Cycle 1 (n = 26) following targeted teaching demonstrated a reduction in abbreviations from 38 to 20%, while doubling the documentation of discussion of procedure-specific risks from 31 to 72%. More patients were offered a copy of their consent form, rising from 12 to 48%. Cycle 2 (n = 24) saw the introduction of pre-printed "risk of procedure" stickers. Although clarity measures continued to improve, quality of pre-procedure risk documentation remained static while the number of forms being offered to patients fell to 8%.

Conclusions

Our project would suggest that while pre-printed stickers can be useful memory aids, specific teaching on consenting produces the greatest benefit. The usage of such tools should therefore be limited, as adjuncts only to specific training.

Standardized pathway for feeding tube placement reduces unnecessary surgery and improves value of care

BACKGROUND

Children requiring gastrostomy tubes (GT) have high resource utilization. In addition, wide variation exists in the decision to perform concurrent fundoplication, which can increase the morbidity of enteral access surgery. We implemented a hospital-wide standardized pathway for GT placement.

METHODS

The standardized pathway included mandatory preoperative nasogastric feeding tube (FT) trial, identification of FT medical home, and standardized postoperative order set, including feeding regimen and parent education. An algorithm to determine whether concurrent fundoplication was indicated was also created. We identified children referred for GT placement from 2015 to 2018 and compared concurrent fundoplication rates and outcomes pre- and postimplementation.

RESULTS

We identified 332 patients who were referred for GT. Of these, 15 avoided placement. Concurrent fundoplication decreased postpathway (48% vs 22%, p < 0.0001). After adjusting for reflux and cardiac disease, prepathway patients were 3.5 times more likely to undergo concurrent fundoplication. ED visits (46% vs 27%, p = 0.001) and postoperative LOS (median (IQR) 10 days (5-36) to 5.5 days (1-19), p = 0.0002) decreased.

CONCLUSIONS

A standardized pathway for GT placement prevented unnecessary GT placement and fundoplication with reduction in postoperative LOS and ED visits. This approach can significantly reduce resource utilization while improving outcomes.

TYPE OF STUDY

Prognosis study.

LEVEL OF EVIDENCE

Level II.

Do cognitive aids reduce error rates in resuscitation team performance? Trial of emergency medicine protocols in simulation training (TEMPIST) in Australia

BACKGROUND

Resuscitation of patients with time-critical and life-threatening illness represents a cognitive challenge for emergency room (ER) clinicians. We designed a cognitive aid, the Emergency Protocols Handbook, to simplify clinical management and team processes. Resuscitation guidelines were reformatted into simple, single step-by-step pathways. This Australian randomised controlled trial tested the effectiveness of this cognitive aid in a simulated ER environment by observing team error rates when current resuscitation guidelines were followed, with and without the handbook.

METHODS

Resuscitation teams were randomised to manage two scenarios with the handbook and two without in a high-fidelity simulation centre. Each scenario was video-recorded. The primary outcome measure was error rates (the number of errors made out of 15 key tasks per scenario). Key tasks varied by scenario. Each team completed four scenarios and was measured on 60 key tasks. Participants were surveyed regarding their perception of the usefulness of the handbook.

RESULTS

Twenty-one groups performed 84 ER crisis simulations. The unadjusted error rate in the handbook group was 18.8% (121/645) versus 38.9% (239/615) in the non-handbook group. There was a statistically significant reduction of 54.0% (95% CI 49.9-57.9) in the estimated percentage error rate when the handbook was available across all scenarios 17.9% (95% CI 14.4-22.0%) versus 38.9% (95% CI 34.2-43.9%). Almost all (97%) participants said they would want to use this cognitive aid during a real medical crisis situation.

CONCLUSION

This trial showed that by following the step-by-step, linear pathways in the handbook, clinicians more than halved their teams' rate of error, across four simulated medical crises. The handbook improves team performance and enables healthcare teams to reduce clinical error rates and thus reduce harm for patients.

TRIAL REGISTRATION

ACTRN12616001456448 registered: www.anzctr.org.au. Trial site: http://emergencyprotocols.org.au/.

Impact of a semi-structured briefing on the management of adverse events in anesthesiology: a randomized pilot study

Background

Human factors research has identified mental models as a key component for the effective sharing and organization of knowledge. The challenge lies in the development and application of tools that help team members to arrive at a shared understanding of a situation. The aim of this study was to assess the influence of a semi-structured briefing on the management of a simulated airway emergency.

Methods

37 interprofessional teams were asked to perform a simulated rapid-sequence induction in the simulator. Teams were presented with a “cannot ventilate, cannot oxygenate” scenario that ultimately required a cricothyroidotomy. Study group (SG) teams were asked to perform a briefing prior to induction, while controls (CG) were asked to perform their usual routine.

Results

We observed no difference in the mean time until cricothyroidotomy (SG 8:31 CG 8:16, p = 0.36). There was a significant difference in groups’ choice of alternative means of oxygenation: While SG teams primarily chose supraglottic airway devices, controls initially reverted to mask ventilation (p = 0.005). SG teams spent significantly less time with this alternative airway device and were quicker to advance in the airway algorithm.

Conclusions

Our study addresses effects on team coordination through a shared mental model as effected by a briefing prior to anesthesia induction. We found measurable improvements in airway management during those stages of the difficult airway algorithm explicitly discussed in the briefing. For those, time spent was shorter and participants were quicker to advance in the airway algorithm.

Retrospective study of security in the transfer of critical patients after application of methodology for risk management

OBJECTIVE

The main objective of our study is to determine if the implementation of an HIT protocol modifies the annual rate of incidents related to patient safety. The secondary objectives are, firstly, to classify the identified events, secondly to analyze the factors that are associated with the presence of said adverse events and finally to analyze the degree of monitoring of the protocol.

MATERIAL AND METHODS

Retrospective descriptive analysis that included patients admitted to the Intensive Care Unit who required HIT between 2009 and 2018. A multidisciplinary protocol was developed and the incidents were classified according to the severity and type of events.

RESULTS

We included 1662 transfers. The total number of transfers with incidents was 153 (9.2%) in which 189 incidents were registered, of which 17 (9%) were described as adverse events (AD), while 172 (91%) were classified as Incidents without Damage (IsD). The clinical incidents were the most frequent (70.37%). In the multivariate analysis we found as associated factors cardiac arrhythmias (OR: 2.88 [IQR 2.01-4.12]), history of stroke (OR 1.72 [IQR 1.06-2.78]) and anemia (OR 1.55 [IQR 1.02-2.37]) The rate of safety-related incidents was less over time as adherence to protocol compliance increased.

CONCLUSIONS

The implementation of a critical patient transport protocol and its application through checklists allows to reduce both the incidence of adverse events in these patients and of Incidents without Damage.

Adapting the 2015 Mother-Baby Friendly Birth Facility Guidelines for Semi-nomadic Pastoralist Communities in Laikipia and Samburu Counties of Kenya

Purpose To adapt the 2015 International Federation of Gynecologists and Obstetricians (FIGO), International Confederation of Midwives (ICM), White Ribbon Alliance (WRA), International Pediatric Association (IPA), and WHO auspiced Guidelines on Mother-Baby Friendly Facilities to a particular sub-population; seminomadic pastoralist communities of Laikipia and Samburu Counties, Kenya. We anticipate an increased utilization of childbirth services by improving their acceptability. Description We drafted a Pastoralist Friendly Birthing Facility Checklist based on the FIGO/ICM/WRA/IPA/WHO guidelines and previous research in this context. We employed mixed methods to finalise the adaptation: a workshop with 27 local stakeholders; interviews with ten health planners and skilled birth attendants (SBAs); and ten focus group discussions (FGDs) with health committee members, community health workers, mothers and traditional birth attendants (TBAs). A facility audit of dispensaries across five group ranches was also undertaken. Assessment The final Checklist was divided into: characteristics of care and the environment; care during labour and birth; post-partum care; and community staff relationships. It was endorsed by the Ministries of Health in the relevant counties, and by women, SBAs and TBAs. No facility currently satisfies all the criteria specified in the Checklist. Conclusion The FIGO/ICM/WRA/IPA/WHO Guidelines were successfully adapted and can be used to ensure health facilities meet the needs of pastoralist women.

Debiasing versus knowledge retrieval checklists to reduce diagnostic error in ECG interpretation

There is an ongoing debate regarding the cause of diagnostic errors. One view is that errors result from unconscious application of cognitive heuristics; the alternative is that errors are a consequence of knowledge deficits. The objective of this study was to compare the effectiveness of checklists that (a) identify and address cognitive biases or (b) promote knowledge retrieval, as a means to reduce errors in ECG interpretation. Novice postgraduate year (PGY) 1 emergency medicine and internal medicine residents (n = 40) and experienced cardiology fellows (PGY 4-6) (n = 21) were randomly allocated to three conditions: a debiasing checklist, a content (knowledge) checklist, or control (no checklist) to be used while interpreting 20 ECGs. Half of the ECGs were deliberately engineered to predispose to bias. Diagnostic performance under either checklist intervention was not significantly better than the control. As expected, more errors occurred when cases were designed to induce bias (F = 96.9, p < 0.0001). There was no significant interaction between the instructional condition and level of learner. Checklists attempting to help learners identify cognitive bias or mobilize domain-specific knowledge did not have an overall effect in reducing diagnostic errors in ECG interpretation, although they may help novices. Even when cognitive biases are deliberately inserted in cases, cognitive debiasing checklists did not improve participants' performance.

The effects of simulation-based education on initial neonatal evaluation and care skills

OBJECTIVE

Neonatal evaluations performed at the very first minutes following postpartum are the most important steps in deciding for neonatal resuscitation. Therefore, the newborn initial care and evaluation notion and skills of midwives in the delivery hall are quite important. The study was planned to determine the effects of simulation education on newborn evaluation and care skills in midwifery students.

METHODS

This is a quasi-experimental study. The population of the study was composed of the 4th year students of Marmara University Faculty of Health Sciences (65 students in total), who selected the Intern Newborn course in the 2017-2018 Fall and Spring semesters.

RESULTS

The areas where the control group students did not apply at all or needed the help of the trainer were observed as delivery room preparation (86.2%), initial neonatal evaluation (96.6%) and registration/safety (69%). According to "the Guide for Newborn Evaluation at the Delivery Room," the differences in the mean total scores and all sub-dimension scores were found to be significant in favor of the experiment group.

CONCLUSION

Education programs that are carried out by computer-assisted simulation and in accordance with the teaching guide were effective on improvement of knowledge-skills on newborns' first evaluations in the delivery room.

Office Patient Safety

Patient safety is inseparable from quality and is a top priority for the United States health care system. This article explores factors that contribute to errors and patient harm in office practice, discusses key ways in which errors in the outpatient setting compare with those occurring in the inpatient setting, and describes strategies for supporting and improving patient safety in office practice.

Care Bundle Approach to Minimizing Infection Rates after Neurosurgical Implants for Neuromodulation: A Single-Surgeon Experience

INTRODUCTION

Implant-related infections carry a high morbidity. Infectious rates for neuromodulation implants range from 1% to 9% for deep brain stimulation (DBS), 0% to 10% for spinal cord stimulation (SCS) systems, and 3% to 15% for intrathecal (IT) pump systems. Meanwhile, studies of care bundles report infection rate reduction to 1.0% for SCS and 0.3% for cardiac implants. Herein, we evaluate the effectiveness of an infection prevention bundle (IPB) in minimizing infections after surgeries for neuromodulation implants.

METHODS

An IPB focused on preoperative checklists, screening questionnaires, methicillin-resistant and methicillin-sensitive Staphylococcus aureus decolonization, weight-based antibiotic prophylaxis, strict draping and surgical techniques, and wound care education was implemented in our functional neurosurgery division in April 2015. We retrospectively reviewed all surgeries for implantation or replacement of SCS, DBS, and IT pump system components from March 2013 to October 2017. The patients were divided into pre-IPB and post-IPB groups. All procedures were performed by a single surgeon. Each surgical site was considered a unique surgical case. Infection rates were calculated for pre-IPB and post-IPB groups.

RESULTS

A total of 688 patients underwent 1161 unique surgical procedures (222 DBS electrodes, 419 IPG, 203 SCS, 317 IT pumps) during the study period. There were 546 pre-IPB and 615 post-IPB surgical procedures. The pre-IPB infection rates were 0%, 1.3%, and 8.7% for SCS, DBS, and IT pumps, respectively. The post-IPB infection rates were 0%, 0.3%, and 1.8% for SCS, DBS, and IT pumps, respectively.

CONCLUSIONS

Implementation of a standardized IPB approach reduced the number of infections for all neuromodulation implants studied. This approach can be adopted within any specialty to potentially decrease the incidence of implant-related infections.

Bronchoscopy Education: An Experiential Learning Theory Perspective

Bronchoscopy programs implementing the experiential learning model address different learning styles. Problem-based learning improves knowledge retention, critical decision making, and communication. These modalities are preferred by learners and contribute to their engagement, in turn leading to durable learning. Follow-up after live events is warranted through spaced education strategies. The objectives of this article are to (1) summarize and illustrate the implementation of experiential learning theory for bronchoscopy courses, (2) discuss the flipped classroom model and problem-based learning, (3) illustrate bronchoscopy checklists implementation in simulation, and (4) discuss the importance of feedback and spaced learning for bronchoscopy education programs.

Perioperative Safety: Keeping Our Children Safe in the Operating Room

The entire operating room team is responsible for the safety of children in the operating room. As a leader in the operating room, the surgeon is impactful in ensuring that all team members are committed to providing this safe environment. This is achieved by the use of perioperative huddles or briefings, the use of appropriate surgical checklists, operating room standardization, surgeons proficient in the care they provide, and team members that embrace Just Culture.

Using Evidence-Based Best Practices of Simulation, Checklists, Deliberate Practice, and Debriefing to Develop and Improve a Regional Anesthesia Training Course

The 2011 Institute of Medicine report on the future of nursing recommended that nurses practice to the full extent of their education and training. Nurse anesthetists in certain regions of the country have been unable to maintain regional anesthesia skills because of anesthesia practice models. Factors including increased patient loads, economic motivators, and desire to maintain skill sets are driving evolution of the anesthesia practice model. In many practices, Certified Registered Nurse Anesthetists (CRNAs) now have the opportunity to expand their practice scope to include regional anesthesia. This has created the need for a pathway to rapidly develop or augment skills for CRNAs who have not been performing regional anesthesia. Well-designed and facilitated simulation methods can be effective for teaching and evaluating clinical skills with incorporation of rigorous assessment instruments to ensure consistency in training outcomes. The purpose of this quality improvement project was to determine the effectiveness of a blended-learning regional anesthesia training curriculum on improving CRNA knowledge, skill, and attitude in regional anesthesia administration as part of a clinical credentialing pathway. Forty-nine CRNAs completed all course components, including meeting all skill training thresholds through deliberate practice and use of validated checklists. Knowledge and confidence levels demonstrated significant gains.

The "TRAUMA LIFE" initiative: The impact of a multidisciplinary checklist process on outcomes and communication in a Trauma Intensive Care Unit

BACKGROUND

Checklists have been advocated to improve quality outcomes/communication in the critical care setting, but results have been mixed. A new checklist process, "TRAUMA LIFE", was implemented in our Trauma Intensive Care Unit (TICU) to replace prior checklists. The purpose of this study was to evaluate the impact of the "TRAUMA LIFE" process implementation on quality metrics and on patient/family communication in the TICU.

METHODS

"TRAUMA LIFE" was considered maturely implemented by 2016. Multiple quality metrics, including restraint order compliance, were compared between 2013 and 2016 (pre- and post-implementation). Compliance with the "Family Message" (FM), a part of the "TRAUMA LIFE" communication process, was analyzed in 2016.

RESULTS

Improvement was seen in CAUTI, VAE, and IUCU; CLABSI rates increased. Restraint order compliance increased significantly. FM delivery compliance was inconsistent; improvement was noted in concordance between update content and FM documented in Electronic Medical Record.

CONCLUSION

Implementation of "TRAUMA LIFE" was well integrated into the rounding process and was associated with some improvement in quality metrics and communication. Additional evaluation is required to assess sustainability.

Introduction of pharmacy technicians onto a busy oncology ward as part of the nursing team

A project was planned to explore the practicality of using pharmacy technicians to support the workload of nursing teams on a busy haematology oncology ward of 28 beds in a university acute care children's hospital of 300 beds. The question asked was, could pharmacy technicians be part of the nursing team to undertake what has traditionally been considered a nursing role? Three pharmacy technicians were trained and participated in the study. Assisting in the preparation and administration of 509 intravenous injections out of a possible 1123 (45%) of all intravenous injections prepared on the ward during the study period. The results indicated a reduction in adverse events of 1-2 a day during the study period, a reduction in work-related stress by nursing staff associated with preparing complex medication and releasing a nurse, 4 hours a day to enable them additional time to care for patients.

The antibiotic checklist: an observational study of the discrepancy between reported and actually performed checklist items

Background

Checklists are increasingly used to measure quality of care. Recently we implemented an antibiotic checklist in nine Dutch hospitals and showed that use of the checklist resulted in more appropriate antibiotic use. While more appropriate antibiotic use was associated with a reduction in length of stay, use of the checklist in itself was not. In the current study we explored discrepancies between reported and actually performed checklist items at the patient level to test the validity of checklist answers, to evaluate whether discrepancies between reported and actually performed checklist items could explain the lack of effect of checklist use on length of stay, and to identify missed opportunities for performance per checklist item.

Methods

Checklist answers represented reported performance. Actual performance was assessed by data from the patients’ medical files. Reported and actually performed checklist items could be ‘both YES’; ‘both NO’; ‘YES reported, NOT actually performed’; or ‘NO reported, YES actually performed’. We determined an overall ‘both YES’ score per checklist, and used mixed models to evaluate whether an association existed between this overall score and patient’s length of hospital stay. Finally, we analysed whether the items that were not actually performed, could have been performed.

Results

Between January and October 2015 physicians filled in 1207 checklists. In total 7881 items were checked. Most items were ‘both YES’ (3392/7881, 43.0%) or ‘both NO’ (2601/7881, 33.0%). The number of ‘YES reported, NOT actually performed’ items was 1628/7881 (20.7%) compared to 260/7881 (3.3%) ‘NO reported, YES actually performed’ items. The level of discrepancy between reported and actually performed items differed per checklist item. The item ‘prescribe antibiotic treatment according to the local guideline’ had the highest percentage of ‘YES reported, NOT actually performed’ items, namely 45.1%. A higher overall ‘both YES’ score of the checklist was significantly associated with a shorter length of hospital stay. Of all checklist items 21.8% were not performed while they could have been performed.

Conclusions

Checklist answers do not accurately assess actual provided care. As actual performance of the antibiotic checklist items is associated with length of stay, efforts to increase actual performance appear to be justified.

Electronic supplementary material

The online version of this article (10.1186/s12879-017-2878-7) contains supplementary material, which is available to authorized users.

Celiac disease and school food service in Piedmont Region: Evaluation of gluten-free meal

INTRODUCTION

The Law 123/2005 recognizes celiac disease as a social disease and so Ministry of Public Health annually allocates specific resources to Regions for managing gluten-free meals in school canteens. Therefore in 2009 Piedmont Region approved a specific project, in collaboration with Food Hygiene and Nutrition Department (SIAN) of several ASL (Local Health Authority), including ASL TO3 as regional leader, and the "Italian Celiac Association - Piedmont and Valle d'Aosta". This project was intended to facilitate the natural integration of celiac people in social life. A retrospective analysis of data has been carried out to assess the management of gluten- free meal of school food services in Piedmont Region in 2010. Furthermore the intervention efficacy has been evaluated comparing the critical points observed in 2010 and 2012.

METHODS

The object of the study includes primary and secondary schools that have provided gluten-free food service in Piedmont Region. These school were examined by SIAN staff. (the examination included the check of hygienic aspects and qualitative assessment of the meal). The data were collected using the same checklist throughout the region. All data were included in the unified regional system ("Reteunitaria").

RESULTS

The results show that 29% of the sampled schools (277) are acceptable in all eight sections (supply, storage, process analysis, equipment check, packaging and transport, distribution of meals, self-control plan and qualitative assessment), whereas 71% are inadequate for at least one of the profiles (60% does not perform the qualitative valuation of service) and in 18% of schools three to seven insufficiencies are observed. Correlations between the number of total insufficiencies and the most critical sections of the check list were performed (with lower scores in "good") such as process analysis, distribution of meals, self-control plan and qualitative assessment. The analysis process has achieved a high score in the field of deficiency for at least 3 parameters. Schools with a good self-control plan have a significant correlation with schools suitable for the analysis process, instead schools appeared insufficient in the process analysis have an increased chance of being insufficient also in the distribution of meals. The schools that provide a transported meal (municipalities highly populated, generally) have many differences in distribution of meals respect schools that prepare food in the school kitchen. In fact, 88% of school that provide a transported meal achieved an appropriate score in distribution of meals section and collected fewer failures in overall assessment than the others. 120 structures are included in the indirect comparison between the checklist's sections with criticisms, during years 2010 and 2012: in 2010 32% of schools were recorded acceptable in all of the eight sections of the checklist and in 2012 this percentage rose to 54%. An improvement can be observed in all areas, but a statistical significant result do not turn out.

DISCUSSION AND CONCLUSIONS

Data show that carry on the control activities of production of gluten free meal in school canteens would be appropriate. Actions focused on improving the methods of preparing meal without gluten were recommended, especially in under populated municipalities with school kitchen on site. The constant presence of ASL staff in school has promoted important changes: cultural change and about the management of allergy and food intolerance. An improvement can be observed: a transition from a suspicion about "special diet" management to an appropriate and responsible management of meals for children and young people suffering from this specific condition.