Expanding scope of Kirkpatrick model from training effectiveness review to evidence-informed prioritization management for cricothyroidotomy simulation

Modified Kirkpatrick model has been adopted to evaluate training effectiveness by 6 categories, including activity accounting (training objectives/success in organization change) at Level-0, reaction (satisfaction) at Level-1, learning (acquisition of surgical airway skills) at Level-2, behavior (post-training change in personal strengths) at Level-3, result (organizational or clinical outcomes) at Level-4, and Return on Investment (ROI) or Expectation (ROE) (monetary and societal values following training and other quality and safety related measures) at Level-5. The purpose of this hospital-based prospective observational study was twofold: i) To evaluate potential impacts on monetary and societal values and successful organization change following implementation of advanced Cricothyroidotomy simulator and standardized curriculum in healthcare simulation training, ii) To inform decisions of resource allocation by reviewing overall values and prioritization strategies for i) general surgeon/emergency physician ii) with seniority >5 years and iii) prior porcine training experience based on findings at Kirkpatrick Level-0, Level-4, and Level-5. Seventy doctors and 10 nurses completed Cricothyroidotomy training and follow-up questionnaires within 2021/22. All training usability scoring measured by Scales of Emergency Surgical Airway Simulator (SESAS-17) achieved over 4 out of 5 (Level-4) with effects in favor of emergency physicians or general surgeons (p < .5), regardless of seniority and prior training experience. Success in organization change (Level-0) and cost-effectiveness (Level-5) were hypothetically established using theoretical framework of Gleicher's formula and Roger's Diffusion of Innovation Theory. Overall training effectiveness, in terms of advantage in usability, cost-benefits and successful organizational changes, provided sound evidence to support continuous investment of new curriculum and innovative simulator and "Surgeon-and-emergency-physician-first" policy when it comes to resources allocation strategies for Cricothyroidotomy training. [ACGME competencies: Practice Based Learning and Improvement, Systems Based Practice.].

Prognostic Model of COVID-19 Severity and Survival among Hospitalized Patients Using Machine Learning Techniques

We conducted a statistical study and developed a machine learning model to triage COVID-19 patients affected during the height of the COVID-19 pandemic in Hong Kong based on their medical records and test results (features) collected during their hospitalization. The correlation between the values of these features is studied against discharge status and disease severity as a preliminary step to identify those features with a more pronounced effect on the patient outcome. Once identified, they constitute the inputs of four machine learning models, Decision Tree, Random Forest, Gradient and RUSBoosting, which predict both the Mortality and Severity associated with the disease. We test the accuracy of the models when the number of input features is varied, demonstrating their stability; i.e., the models are already highly predictive when run over a core set of (6) features. We show that Random Forest and Gradient Boosting classifiers are highly accurate in predicting patients' Mortality (average accuracy ∼99%) as well as categorize patients (average accuracy ∼91%) into four distinct risk classes (Severity of COVID-19 infection). Our methodical and broad approach combines statistical insights with various machine learning models, which paves the way forward in the AI-assisted triage and prognosis of COVID-19 cases, which is potentially generalizable to other seasonal flus.

Investigating effects of healthcare simulation on personal strengths and organizational impacts for healthcare workers during COVID-19 pandemic: A cross-sectional study

INTRODUCTION

This cross-sectional study aimed at evaluating impacts of healthcare simulation training, either in-situ or lab-based, on personal strengths of healthcare workers (HCWs) and organizational outcomes during the COVID-19 pandemic.

METHODS

COVID-19 Taskforce was established to formulate standardized scenario-based simulation training materials in late-January 2020. Post-training questionnaires made up of 5-point Likert scales were distributed to all participants to evaluate their personal strengths, in terms of i) assertiveness, ii) mental preparedness, iii) self-efficacy, iv) internal locus of control, and v) internal locus of responsibility. Independent sample t-tests were used to analyze between-group difference in "In-situ" and "Lab-based" group; and one-sample t-tests were used to compare change in personal strengths with reference point of 3 (Neutral). Kirkpatrick's Model served as the analytical framework for overall training effects.

RESULTS

Between 05 February and 18 March 2020, 101 sessions of simulation training were conducted in "In-Situ" at either Accident & Emergency Department (20, 20%) or Intensive Care Unit (15, 14%) and "Lab-based" for Isolation (30, 30%) and General Wards (36, 36%). 1,415 hospital staff members, including 1,167 nurses (82%), 163 doctors (12%) and 85 patient care assistants (6%), were trained. All domains of personal strengths were scored 4.24 or above and statistically significantly increased when comparing with reference population (p < .001). However, no significant differences between in-situ and lab-based simulation were found (p > .05), for all domains of personal strengths.

CONCLUSION

Healthcare simulation training enhanced healthcare workers' personal strengths critical to operational and clinical outcomes during the COVID-19 pandemic.