Effect of a cognitive aid on adherence to perioperative assessment and management guidelines for the cardiac evaluation of noncardiac surgical patients

A simulation-based pilot study of crisis checklists in the emergency department

Checklists can improve adherence to standardized procedures and minimize human error. We aimed to test if implementation of a checklist was feasible and effective in enhancing patient care in an emergency department handling internal medicine cases. We developed four critical event checklists and confronted volunteer teams with a series of four simulated emergency scenarios. In two scenarios, the teams were provided access to the crisis checklists in a randomized cross-over design. Simulated patient outcome plus statement of the underlying diagnosis defined the primary endpoint and adherence to key processes such as time to commence CPR represented the secondary endpoints. A questionnaire was used to capture participants' perception of clinical relevance and manageability of the checklists. Six teams of four volunteers completed a total of 24 crisis sequences. The primary endpoint was reached in 8 out of 12 sequences with and in 2 out of 12 sequences without a checklist (Odds ratio, 10; CI 1.11, 123.43; p = 0.03607, Fisher's exact test). Adherence to critical steps was significantly higher in all scenarios for which a checklist was available (performance score of 56.3% without checklist, 81.9% with checklist, p = 0.00284, linear regression model). All participants rated the checklist as useful and 22 of 24 participants would use the checklist in real life. Checklist use had no influence on CPR quality. The use of context-specific checklists showed a statistically significant influence on team performance and simulated patient outcome and contributed to adherence to standard clinical practices in emergency situations.

Simulation Versus Interactive Mobile Learning for Teaching Extracorporeal Membrane Oxygenation to Clinicians: A Randomized Trial

OBJECTIVES

Extracorporeal membrane oxygenation has become integral to critical care. Data informing optimal extracorporeal membrane oxygenation education modalities are lacking. We aimed to compare the effect of high-fidelity simulation versus interactive mobile learning on extracorporeal membrane oxygenation knowledge acquisition and retention among clinicians.

DESIGN

Observer-blinded, randomized controlled trial.

SETTING

A single academic medical center.

SUBJECTS

Forty-four critical care clinicians with limited extracorporeal membrane oxygenation experience.

INTERVENTIONS

Participants were randomized to receive: 1) simulation: three high-fidelity training scenarios, 2) QuizTime: 15 total multiple-choice questions delivered over 3 weeks via mobile device, or 3) experiential: no formal training. Participants completed a survey, written knowledge examination, and simulation assessment prior to randomization, immediately following the intervention, and 4 month postintervention.

MEASUREMENTS AND MAIN RESULTS

The primary outcome was knowledge about extracorporeal membrane oxygenation assessed by score on the immediate postintervention written examination. Secondary outcomes included performance in extracorporeal membrane oxygenation simulation postintervention and 4 months later assessed by a rater blinded to group assignment. Clinicians randomized to simulation (n = 15), QuizTime (n = 14), and experiential (n = 15) had similar baseline characteristics. Adjusting for baseline knowledge, postintervention examination scores were higher in the simulation group (90.0%; interquartile range, 85.0-90.0%) than the QuizTime group (70.0%; interquartile range, 65.0-80.0%; p = 0.0003) and the experiential group (75.0%; interquartile range, 65.0-80.0%; p = 0.001). Scores did not differ between the groups at 4 months (p > 0.05 in all analyses). In postintervention extracorporeal membrane oxygenation simulations, the simulation group demonstrated shorter time to critical action compared with QuizTime (80.0 s [interquartile range, 54.0-111.0 s] vs 300.0 s [interquartile range 85.0-300.0 s]; p = 0.02) and compared with both QuizTime (45.0 s [interquartile range, 34.0-92.5 s] vs 255.5 s [interquartile range, 102.0-300.0 s]; p = 0.008) and experiential (300.0 s [interquartile range, 58.0-300.0 s]; p = 0.009) at 4 months.

CONCLUSIONS

Simulation was superior to QuizTime and experiential learning with regard to extracorporeal membrane oxygenation knowledge acquisition. Further studies are needed to ascertain the effect of these interventions on knowledge retention, clinical performance, and patient outcomes.

Rapid Training in Extracorporeal Membrane Oxygenation for a Large Health System

Background: Despite the rapid integration of extracorporeal membrane oxygenation (ECMO) into intensive care units over the past decade, established programs for training critical care clinicians to provide ECMO are lacking.

Objective: To evaluate the development and implementation of a multidisciplinary ECMO training program for the rapid deployment of ECMO training for a high volume of critical care clinicians.

Methods: We performed a prospective cohort study examining a program for rapid training of multiple disciplines of critical care clinicians to deliver ECMO during the implementation of ECMO services across the intensive care units of an academic tertiary care center between October 2018 and January 2019. The multidisciplinary ECMO training program included didactic and simulation-based teaching and emphasized new, universal clinical protocols to improve consistency of care across the institution. Pre- and post-program written examinations evaluated knowledge acquisition, and an electronically distributed program evaluation assessed perceptions of content and delivery.

Results: Among the 97 clinicians who completed the program, 49 (51%) were physicians and 48 (49%) were advanced practice providers from the departments of surgery (n = 42), medicine (n = 29), and anesthesia (n = 26). There was a significant difference in knowledge about ECMO between the pre- and post-program examination score (median [interquartile range] 70% [60–80%] vs. 90% [80–90%], respectively, P < 0.001). The median (interquartile range) individual gain from pre- to post-program score was 20% (10–30%). The program was perceived as useful and applicable to safe care.

Conclusion: Rapid deployment of a multidisciplinary ECMO training program across a large academic center was feasible, achieved knowledge acquisition, and was positively perceived.

Learners and Luddites in the Twenty-first Century: Bringing Evidence-based Education to Anesthesiology

Anesthesiologists are both teachers and learners and alternate between these roles throughout their careers. However, few anesthesiologists have formal training in the methodologies and theories of education. Many anesthesiology educators often teach as they were taught and may not be taking advantage of current evidence in education to guide and optimize the way they teach and learn. This review describes the most up-to-date evidence in education for teaching knowledge, procedural skills, and professionalism. Methods such as active learning, spaced learning, interleaving, retrieval practice, e-learning, experiential learning, and the use of cognitive aids will be described. We made an effort to illustrate the best available evidence supporting educational practices while recognizing the inherent challenges in medical education research. Similar to implementing evidence in clinical practice in an attempt to improve patient outcomes, implementing an evidence-based approach to anesthesiology education may improve learning outcomes.

Effects of evidence-based clinical practice guidelines in cardiovascular health care quality improvements: A systematic review

Background: The development of clinical practice guidelines (CPGs) has increasing global growth; however, the certainty of impact on patients and health systems, as well as the magnitude of the impact, is not apparent. The objective of this systematic review was to assess the effectiveness of the application of CPGs for the improvement of the quality of health care in three domains: structure, process and results in the patient for the management of cardiovascular disease. Methods: We followed the methods described by the Cochrane Handbook and present a descriptive analysis because of the high heterogeneity found across the included studies. We searched the Cochrane Central Register of Controlled Trials, MEDLINE and EMBASE databases, as well as the grey literature, between 1990 and June 2016. No language restrictions were applied. Only randomised clinical trials (RCTs) were selected. Three authors independently carried out the data extraction, using a modified version of the Cochrane Effective Practice and Organization of Care form. Results: Of the total of 84 interventions included in the nine RCTs evaluated, three (4%) were related to health care structure, 54 (64%) to the health care delivery process and 27 (32%) to patient outcomes. Regarding the impact of using the CPGs, in 55 interventions (65%), there were no significant differences between control and experimental groups. In four interventions (5%), the result favoured the control group, and the result favoured the intervention group on 25 of the interventions (30%). Conclusions: This systematic review showed that CPGs could be useful to improve the process and structure of health care and, to a lesser extent, to improve the results in patients. However, evidence was weak. There are probably still undiscovered variables that interfere with the use of the CPGs and, therefore, with their impact. Therefore, more studies of good quality are needed. Registration: PROSPERO CRD42013003589.

Effects of evidence-based clinical practice guidelines in cardiovascular health care quality improvements: A systematic review

Background: The development of evidence-based clinical practice guidelines (EB-CPGs) has increasing global growth; however, the certainty of impact on patients and health systems, as well as the magnitude of the impact, is not apparent. The objective of this systematic review was to assess the effectiveness of the application of EB-CPGs for the improvement of the quality of health care in three dimensions: structure, process and results in the patient for the management of cardiovascular disease. Methods: We followed the methods described by the Cochrane Handbook and present a descriptive analysis because of the high heterogeneity found across the included studies. We searched the Cochrane Central Register of Controlled Trials, MEDLINE and EMBASE databases, as well as the grey literature, between 1990 and June 2016. No language restrictions were applied. Only randomised clinical trials (RCTs) were selected. Three authors independently carried out the data extraction, using a modified version of the Cochrane Effective Practice and Organization of Care form. Results: Of the total of 84 interventions included in the nine RCTs evaluated, three (4%) were related to health care structure, 54 (64%) to the health care delivery process and 27 (32%) to patient outcomes. Regarding the impact of using the EB-CPGs, in 55 interventions (65%), there were no significant differences between control and experimental groups. In four interventions (5%), the result favoured the control group, and the result favoured the intervention group on 25 of the interventions (30%). Conclusions: This systematic review showed that EB-CPGs could be useful to improve the process and structure of health care and, to a lesser extent, to improve the patients' outcomes. After analysing many studies, we could have one more hypothesis for further research, which could shed more light upon those undiscovered variables that might interfere with the use of the EB-CPGs. Registration: PROSPERO CRD42013003589.

Effects of evidence-based clinical practice guidelines in cardiovascular health care quality improvements: A systematic review

Background: The development of evidence-based clinical practice guidelines (EB-CPGs) has increasing global growth; however, the certainty of impact on patients and health systems, as well as the magnitude of the impact, is not apparent. The objective of this systematic review was to assess the effectiveness of the application of EB-CPGs for the improvement of the quality of health care in three dimensions: structure, process and results in the patient for the management of cardiovascular disease. Methods: We followed the methods described by the Cochrane Handbook and present a descriptive analysis because of the high heterogeneity found across the included studies. We searched the Cochrane Central Register of Controlled Trials, MEDLINE and EMBASE databases, as well as the grey literature, between 1990 and June 2016. No language restrictions were applied. Only randomised clinical trials (RCTs) were selected. Three authors independently carried out the data extraction, using a modified version of the Cochrane Effective Practice and Organization of Care form. Results: Of the total of 84 interventions included in the nine RCTs evaluated, three (4%) were related to health care structure, 54 (64%) to the health care delivery process and 27 (32%) to patient outcomes. Regarding the impact of using the EB-CPGs, in 55 interventions (65%), there were no significant differences between control and experimental groups. In four interventions (5%), the result favoured the control group, and the result favoured the intervention group on 25 of the interventions (30%). Conclusions: This systematic review showed that EB-CPGs could be useful to improve the process and structure of health care and, to a lesser extent, to improve the patients' outcomes. After analysing many studies, we could have one more hypothesis for further research, which could shed more light upon those undiscovered variables that might interfere with the use of the EB-CPGs. Registration: PROSPERO CRD42013003589.

Evidence for increased use of the Society of Pediatric Anesthesia Critical Events Checklist in resource-limited environments: A retrospective observational study of app data

BACKGROUND

Electronic decision support tools in anesthesiology practice have great value, including the potential for mobile applications to simplify delivery of best-practice guidelines.

AIMS

We sought to combine demographics with usage information to elucidate important patterns in the rate of use of the Society of Pediatric Anesthesia Critical Events Checklist, as measured by in-app accesses of the checklist via the freely available anesthesia calculator app anesthesiologist.

METHODS

We performed a retrospective analytic observational case-control study using analytics and survey data collected from the app. Users of the app were classified on the basis of whether or not they had accessed the checklist. This classification was used to perform logistic regression against a number of independent variables, including frequency of app use, country income level, professional role, rating of app importance, length of time in practice, group size, practice model, community served, and primary practice environment.

RESULTS

Individual app users practicing in low- and middle-income countries have a significantly higher rate of Society for Pediatric Anesthesia Critical Events Checklist utilization as compared with high-income countries. Rural practitioners had higher utilization of the checklist. Practice size did not affect the utilization of the checklist. The checklist was used for both provider learning and for just-in-time patient care.

CONCLUSION

mHealth apps are invaluable resource in everyday clinical practice. Mobile app analytics and in-app survey data reveal variable penetration and applicability of such technology worldwide. mHealth apps may be particularly impactful in limited-resource areas, such as lower-income environments and rural communities.

[CAVE - A checklist system for preoperative risk evaluation : Guideline-conform cardiopulmonary diagnostics before general and visceral surgical interventions]

BACKGROUND

Preoperative evaluation of patient risk is an essential component of patient preparation before surgery. Guidelines provide evidence-based algorithms for preoperative assessment of cardiac risk; however, even experienced physicians correctly apply evidence-based algorithms in only 50% of all cases or less.

OBJECTIVE

A checklist system for guideline-based cardiopulmonary risk evaluation in adult patients undergoing abdominal or visceral surgery (CAVE checklists) was created to assist in preoperative cardiopulmonary risk assessment and increase correct application of evidence-based algorithms before elective visceral surgery.

MATERIAL UND METHODS

International guidelines were transformed into a checklist system. These checklists were than evaluated in a department of general and visceral surgery. The main goal was to determine whether preoperative examinations, such as electrocardiograph (ECG), chest-x-ray, spirometry and advanced assessment by a cardiologist, are performed according to evidence-based guidelines. The frequency of recommended as well as unnecessary and missed examinations was assessed.

RESULTS

In this study 541 patients with a median age of 64.5 years (interquartile range: 52-73 years) were examined using the checklist system. Of the patients 90.4% underwent ECG and 98.5% chest-X-ray as recommended in the guidelines. Spirometry was not recommended in any patient and not performed in any case. Advanced assessment by a cardiologist was performed in 45.5% of cases as recommended in the guidelines. When guidelines did not recommend ECG, x‑ray, spirometry or advanced cardiac assessment, 69.4%, 99.6%, 99.3% and 99.8% of patients, respectively, actually did not receive these examinations. Only 2.8% of all patients did not receive an examination that was recommended by the guidelines: 1.5% ECG, 0.2% x‑ray and 1.1% advanced cardiological assessment. None of these patients suffered from postoperative cardiopulmonary complications.

CONCLUSION

These simple checklists are easy to use and provide a higher degree of evidence-based preoperative cardiopulmonary risk evaluation than previously reported in the literature. Adaptation of the checklists to changing guidelines is easy to perform. Whether the application of these checklists will result in a reduction of morbidity and costs have to be determined in further clinical trials.

Results of a Flipped Classroom Teaching Approach in Anesthesiology Residents

BACKGROUND

In a flipped classroom approach, learners view educational content prior to class and engage in active learning during didactic sessions.

OBJECTIVE

We hypothesized that a flipped classroom improves knowledge acquisition and retention for residents compared to traditional lecture, and that residents prefer this approach.

METHODS

We completed 2 iterations of a study in 2014 and 2015. Institutions were assigned to either flipped classroom or traditional lecture for 4 weekly sessions. The flipped classroom consisted of reviewing a 15-minute video, followed by 45-minute in-class interactive sessions with audience response questions, think-pair-share questions, and case discussions. The traditional lecture approach consisted of a 55-minute lecture given by faculty with 5 minutes for questions. Residents completed 3 knowledge tests (pretest, posttest, and 4-month retention) and surveys of their perceptions of the didactic sessions. A linear mixed model was used to compare the effect of both formats on knowledge acquisition and retention.

RESULTS

Of 182 eligible postgraduate year 2 anesthesiology residents, 155 (85%) participated in the entire intervention, and 142 (78%) completed all tests. The flipped classroom approach improved knowledge retention after 4 months (adjusted mean = 6%; P = .014; d = 0.56), and residents preferred the flipped classroom (pre = 46%; post = 82%; P < .001).

CONCLUSIONS

The flipped classroom approach to didactic education resulted in a small improvement in knowledge retention and was preferred by anesthesiology residents.

Interactive pediatric emergency checklists to the palm of your hand - How the Pedi Crisis App traveled around the world

BACKGROUND

Cognitive aids help clinicians manage critical events and have been shown to improve outcomes by providing critical information at the point of care. Critical event guidelines, such as the Society of Pediatric Anesthesia's Critical Events Checklists described in this article, can be distributed globally via interactive smartphone apps. From October 1, 2013 to January 1, 2014, we performed an observational study to determine the global distribution and utilization patterns of the Pedi Crisis cognitive aid app that the Society for Pediatric Anesthesia developed. We analyzed distribution and utilization metrics of individuals using Pedi Crisis on iOS (Apple Inc., Cupertino, CA) devices worldwide. We used Google Analytics software (Google Inc., Mountain View, CA) to monitor users' app activity (eg, screen views, user sessions).

METHODS

The primary outcome measurement was the number of user-sessions and geographic locations of Pedi Crisis user sessions. Each user was defined by the use of a unique Apple ID on an iOS device.

RESULTS

Google Analytics correlates session activity with geographic location based on local Internet service provider logs. Pedi Crisis had 1 252 active users (both new and returning) and 4 140 sessions across 108 countries during the 3-month study period. Returning users used the app longer and viewed significantly more screens that new users (mean screen views: new users 1.3 [standard deviation +/-1.09, 95% confidence interval 1.22-1.55]; returning users 7.6 [standard deviation +/-4.19, 95% confidence interval 6.73-8.39]P<.01) CONCLUSIONS: Pedi Crisis was used worldwide within days of its release and sustained utilization beyond initial publication. The proliferation of handheld electronic devices provides a unique opportunity for professional societies to improve the worldwide dissemination of guidelines and evidence-based cognitive aids.

Mode of Information Delivery Does Not Effect Anesthesia Trainee Performance During Simulated Perioperative Pediatric Critical Events: A Trial of Paper Versus Electronic Cognitive Aids

INTRODUCTION

Cognitive aids (CAs), including emergency manuals and checklists, have been recommended as a means to address the failure of healthcare providers to adhere to evidence-based standards of treatment during crisis situations. Unfortunately, users of CAs still commit errors, omit critical steps, fail to achieve perfect adherence to guidelines, and frequently choose to not use CA during both simulated and real crisis events. We sought to evaluate whether the mode in which a CA presents information (ie, paper vs. electronic) affects clinician performance during simulated critical events.

METHODS

In a prospective, randomized, controlled trial, anesthesia trainees managed simulated events under 1 of the following 3 conditions: (1) from memory alone (control), (2) with a paper CA, or (3) with an electronic version of the same CA. Management of the events was assessed using scenario-specific checklists. Mixed-effect regression models were used for analysis of overall checklist score and for elapsed time.

RESULTS

One hundred thirty-nine simulated events were observed and rated. Approximately, 1 of 3 trainees assigned to use a CA (electronic 29%, paper 36%) chose not to use it during the scenario. Compared with the control group (52%), the overall score was 6% higher in the paper CA group and 8% higher (95% confidence interval, 0.914.5; P = 0.03) in the electronic CA group. The difference between paper and electronic CA was not significant. There was a wide range in time to first use of the CA, but the time to task completion was not affected by CA use, nor did the time to CA use impact CA effectiveness as measured by performance.

CONCLUSIONS

The format (paper or electronic) of the CA did not affect the impact of the CA on clinician performance in this study. Clinician compliance with the use of the CA was unaffected by format, suggesting that other factors may determine whether clinicians choose to use a CA or not. Time to use of the CA did not affect clinical performance, suggesting that it may not be when CAs are used but how they are used that determines their impact. The current study highlights the importance of not just familiarizing clinicians with the content of CA but also training clinicians in when and how to use an emergency CA.

A Smartphone-based Decision Support Tool Improves Test Performance Concerning Application of the Guidelines for Managing Regional Anesthesia in the Patient Receiving Antithrombotic or Thrombolytic Therapy

BACKGROUND

The American Society of Regional Anesthesia and Pain Medicine (ASRA) consensus statement on regional anesthesia in the patient receiving antithrombotic or thrombolytic therapy is the standard for evaluation and management of these patients. The authors hypothesized that an electronic decision support tool (eDST) would improve test performance compared with native physician behavior concerning the application of this guideline.

METHODS

Anesthesiology trainees and faculty at 8 institutions participated in a prospective, randomized trial in which they completed a 20-question test involving clinical scenarios related to the ASRA guidelines. The eDST group completed the test using an iOS app programmed to contain decision logic and content of the ASRA guidelines. The control group completed the test by using any resource in addition to the app. A generalized linear mixed-effects model was used to examine the effect of the intervention.

RESULTS

After obtaining institutional review board's approval and informed consent, 259 participants were enrolled and randomized (eDST = 122; control = 137). The mean score was 92.4 ± 6.6% in the eDST group and 68.0 ± 15.8% in the control group (P < 0.001). eDST use increased the odds of selecting correct answers (7.8; 95% CI, 5.7 to 10.7). Most control group participants (63%) used some cognitive aid during the test, and they scored higher than those who tested from memory alone (76 ± 15% vs. 57 ± 18%, P < 0.001). There was no difference in time to completion of the test (P = 0.15) and no effect of training level (P = 0.56).

CONCLUSIONS

eDST use improved application of the ASRA guidelines compared with the native clinician behavior in a testing environment.