1
|
Tawfik M, Schiff E, Mosavian R, Campisi C, Shen A, Lin J, Windsor AM, Weingarten‐Arams J, Soshnick SH, Nishisaki A, Je S, Maa T, Harwayne‐Gidansky I, Fortunov RM, Yang CJ. Validation of a Novel Mobile Application for Assessing Pediatric Tracheostomy Emergency Simulations. OTO Open 2024; 8:e145. [PMID: 38974176 PMCID: PMC11222740 DOI: 10.1002/oto2.145] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2024] [Accepted: 04/27/2024] [Indexed: 07/09/2024] Open
Abstract
Objective Pediatric tracheostomy is associated with high morbidity and mortality, yet clinician knowledge and quality of tracheostomy care may vary widely. In situ simulation is effective at detecting and mitigating related latent safety threats, but evaluation via retrospective video review has disadvantages (eg, delayed analysis, and potential data loss). We evaluated whether a novel mobile application is accurate and reliable for assessment of in situ tracheostomy emergency simulations. Methods A novel mobile application was developed for assessment of tracheostomy emergency in situ simulation team performance. After 1.25 hours of training, 6 raters scored 10 tracheostomy emergency simulation videos for the occurrence and timing of 12 critical steps. To assess accuracy, rater scores were compared to a reference standard to determine agreement for occurrence or absence of critical steps and a timestamp within ±5 seconds. Interrater reliability was determined through Cohen's and Fleiss' kappa and intraclass correlation coefficient. Results Raters had 86.0% agreement with the reference standard when considering step occurrence and timing, and 92.8% agreement when considering only occurrence. The average timestamp difference from the reference standard was 1.3 ± 18.5 seconds. Overall interrater reliability was almost perfect for both step occurrence (Fleiss' kappa of 0.81) and timing of step (intraclass correlation coefficient of 0.99). Discussion Using our novel mobile application, raters with minimal training accurately and reliably assessed videos of tracheostomy emergency simulations and identified areas for future refinement. Implications for Practice With refinements, this innovative mobile application is an effective tool for real-time data capture of time-critical steps in in situ tracheostomy emergency simulations.
Collapse
Affiliation(s)
| | | | | | | | - Amanda Shen
- Albert Einstein College of MedicineBronxNew YorkUSA
| | - Juan Lin
- Albert Einstein College of MedicineBronxNew YorkUSA
| | - Alanna M. Windsor
- Albert Einstein College of MedicineBronxNew YorkUSA
- Department of Otorhinolaryngology–Head and Neck SurgeryMontefiore Medical CenterBronxNew YorkUSA
| | - Jacqueline Weingarten‐Arams
- Albert Einstein College of MedicineBronxNew YorkUSA
- Department of Pediatrics, Division of Pediatric Critical Care MedicineChildren's Hospital at MontefioreBronxNew YorkUSA
| | - Sara H. Soshnick
- Albert Einstein College of MedicineBronxNew YorkUSA
- Department of Pediatrics, Division of Pediatric Critical Care MedicineChildren's Hospital at MontefioreBronxNew YorkUSA
| | - Akira Nishisaki
- University of Pennsylvania Perelman School of MedicinePhiladelphiaPennsylvaniaUSA
- Department of Anesthesiology and Critical Care Medicine, Division of Pediatric Critical Care MedicineChildren's Hospital of PhiladelphiaPhiladelphiaPennsylvaniaUSA
| | - Sangmo Je
- Department of Anesthesiology and Critical Care Medicine, Division of Pediatric Critical Care MedicineChildren's Hospital of PhiladelphiaPhiladelphiaPennsylvaniaUSA
| | - Tensing Maa
- Ohio State University College of MedicineColumbusOhioUSA
- Department of Pediatrics, Division of Pediatric Critical Care MedicineNationwide Children's HospitalColumbusOhioUSA
| | - Ilana Harwayne‐Gidansky
- Department of Pediatrics, Division of Pediatric Critical Care Medicine, The Bernard & Millie Duker Children's HospitalAlbany Medical CenterAlbanyNew YorkUSA
- Albany Medical CollegeAlbanyNew YorkUSA
| | - Regine M. Fortunov
- Department of Pediatrics, Division of NeonatologyBaylor College of MedicineHoustonTexasUSA
- Texas Children's HospitalHoustonTexasUSA
| | - Christina J. Yang
- Albert Einstein College of MedicineBronxNew YorkUSA
- Department of Otorhinolaryngology–Head and Neck SurgeryMontefiore Medical CenterBronxNew YorkUSA
| | | |
Collapse
|
2
|
Zhang C. A Literature Study of Medical Simulations for Non-Technical Skills Training in Emergency Medicine: Twenty Years of Progress, an Integrated Research Framework, and Future Research Avenues. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2023; 20:4487. [PMID: 36901496 PMCID: PMC10002261 DOI: 10.3390/ijerph20054487] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/09/2022] [Revised: 02/26/2023] [Accepted: 03/01/2023] [Indexed: 06/18/2023]
Abstract
Medical simulations have led to extensive developments in emergency medicine. Apart from the growing number of applications and research efforts in patient safety, few studies have focused on modalities, research methods, and professions via a synthesis of simulation studies with a focus on non-technical skills training. Intersections between medical simulation, non-technical skills training, and emergency medicine merit a synthesis of progress over the first two decades of the 21st century. Drawing on research from the Web of Science Core Collection's Science Citation Index Expanded and Social Science Citation Index editions, results showed that medical simulations were found to be effective, practical, and highly motivating. More importantly, simulation-based education should be a teaching approach, and many simulations are utilised to substitute high-risk, rare, and complex circumstances in technical or situational simulations. (1) Publications were grouped by specific categories of non-technical skills, teamwork, communication, diagnosis, resuscitation, airway management, anaesthesia, simulation, and medical education. (2) Although mixed-method and quantitative approaches were prominent during the time period, further exploration of qualitative data would greatly contribute to the interpretation of experience. (3) High-fidelity dummy was the most suitable instrument, but the tendency of simulators without explicitly stating the vendor selection calls for a standardised training process. The literature study concludes with a ring model as the integrated framework of presently known best practices and a broad range of underexplored research areas to be investigated in detail.
Collapse
Affiliation(s)
- Cevin Zhang
- School of Media and Design, Beijing Technology and Business University, Sunlight South Road 1, Beijing 102488, China
| |
Collapse
|
3
|
Alunno A, Najm A, Sivera F, Haines C, Falzon L, Ramiro S. Assessment of competences in rheumatology training: results of a systematic literature review to inform EULAR points to consider. RMD Open 2021; 6:rmdopen-2020-001330. [PMID: 32883720 PMCID: PMC7508213 DOI: 10.1136/rmdopen-2020-001330] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/20/2020] [Revised: 07/16/2020] [Accepted: 08/21/2020] [Indexed: 11/08/2022] Open
Abstract
Objective To summarise the literature on the assessment of competences in postgraduate medical training. Methods A systematic literature review was performed within a EULAR taskforce on the assessment of competences in rheumatology training and other related specialities (July 2019). Two searches were performed: one search for rheumatology and one for related medical specialities. Two reviewers independently identified eligible studies and extracted data on assessment methods. Risk of bias was assessed using the medical education research study quality instrument. Results Of 7335 articles in rheumatology and 2324 reviews in other specialities, 5 and 31 original studies were included, respectively. Studies in rheumatology were at variable risk of bias and explored only direct observation of practical skills (DOPS) and objective structured clinical examinations (OSCEs). OSCEs, including clinical, laboratory and imaging stations, performed best, with a good to very good internal consistency (Cronbach’s α=0.83–0.92), and intrarater reliability (r=0.80–0.95). OSCEs moderately correlated with other assessment tools: r=0.48 vs rating by programme directors; r=0.2–0.44 vs multiple-choice questionnaires; r=0.48 vs DOPS. In other specialities, OSCEs on clinical skills had a good to very good inter-rater reliability and OSCEs on communication skills demonstrated a good to very good internal consistency. Multisource feedback and the mini-clinical evaluation exercise showed good feasibility and internal consistency (reliability), but other data on validity and reliability were conflicting. Conclusion Despite consistent data on competence assessment in other specialities, evidence in rheumatology is scarce and conflicting. Overall, OSCEs seem an appropriate tool to assess the competence of clinical skills and correlate well with other assessment strategies. DOPS, multisource feedback and the mini-clinical evaluation exercise are feasible alternatives.
Collapse
Affiliation(s)
- Alessia Alunno
- Rheumatology Unit, University of Perugia Department of Medicine, Perugia, Italy
| | - Aurélie Najm
- University Hospital, Inserm Umr 1238, Nantes, France
| | - Francisca Sivera
- Department of Rheumatology, Hospital General Universitario Elda, Elda, Spain.,Department of Medicine, Miguel Hernandez University of Elche, Elche, Spain
| | | | - Louise Falzon
- Center for Personalized Health, Northwell Health Feinstein Institutes for Medical Research, Manhasset, New York, USA
| | - Sofia Ramiro
- Department of Rheumatology, Leiden University Medical Center, Leiden, Netherlands.,Department of Rheumatology, Zuyderland Medical Centre Heerlen, Heerlen, Netherlands
| |
Collapse
|
4
|
Alberto EC, Jagannath S, McCusker ME, Keller S, Marsic I, Sarcevic A, O’Connell KJ, Burd RS. Classification strategies for non-routine events occurring in high-risk patient care settings: A scoping review. J Eval Clin Pract 2021; 27:464-471. [PMID: 33249690 PMCID: PMC7961264 DOI: 10.1111/jep.13456] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/22/2020] [Revised: 07/10/2020] [Accepted: 07/13/2020] [Indexed: 11/27/2022]
Abstract
INTRODUCTION Non-routine events (NREs) are atypical or unusual occurrences in a pre-defined process. Although some NREs in high-risk clinical settings have no adverse effects on patient care, others can potentially cause serious patient harm. A unified strategy for identifying and describing NREs in these domains will facilitate the comparison of results between studies. METHODS We conducted a literature search in PubMed, CINAHL, and EMBASE to identify studies related to NREs in high-risk domains and evaluated the methods used for event observation and description. We applied The Joint Commission on Accreditation of Healthcare Organization (JCAHO) taxonomy (cause, impact, domain, type, prevention, and mitigation) to the descriptions of NREs from the literature. RESULTS We selected 25 articles that met inclusion criteria for review. Real-time documentation of NREs was more common than a retrospective video review. Thirteen studies used domain experts as observers and seven studies validated observations with interrater reliability. Using the JCAHO taxonomy, "cause" was the most frequently applied classification method, followed by "impact," "type," "domain," and "prevention and mitigation." CONCLUSIONS NREs are frequent in high-risk medical settings. Strengths identified in several studies included the use of multiple observers with domain expertise and validation of the event ascertainment approach using interrater reliability. By applying the JCAHO taxonomy to the current literature, we provide an example of a structured approach that can be used for future analyses of NREs.
Collapse
Affiliation(s)
- Emily C. Alberto
- Division of Trauma and Burns, Children’s National Hospital, Washington, DC, USA
| | - Swathi Jagannath
- College of Computing and Informatics, Drexel University, Philadelphia, PA, USA
| | - Maureen E. McCusker
- Office of Institutional Research and Decision Support, Virginia Commonwealth University, Richmond, VA, USA
| | - Susan Keller
- Department of Nursing Science Professional Practice and Quality, Children’s National Hospital, Washington, DC, USA
| | - Ivan Marsic
- Department of Electrical and Computer Engineering, Rutgers University, Piscataway, NJ, USA
| | - Aleksandra Sarcevic
- College of Computing and Informatics, Drexel University, Philadelphia, PA, USA
| | - Karen J. O’Connell
- Division of Emergency Medicine, Children’s National Hospital, Washington, DC, USA
| | - Randall S. Burd
- Division of Trauma and Burns, Children’s National Hospital, Washington, DC, USA
| |
Collapse
|
5
|
Ambardekar AP, Black S, Singh D, Lockman JL, Simpao AF, Schwartz AJ, Hales RL, Rodgers DL, Gurnaney HG. The impact of simulation-based medical education on resident management of emergencies in pediatric anesthesiology. Paediatr Anaesth 2019; 29:753-759. [PMID: 31034728 DOI: 10.1111/pan.13652] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/07/2018] [Revised: 04/13/2019] [Accepted: 04/19/2019] [Indexed: 11/29/2022]
Abstract
BACKGROUND Resident education in pediatric anesthesiology is challenging. Traditional curricula for anesthesiology residency programs have included a combination of didactic lectures and mentored clinical service, which can be variable. Limited pediatric medical knowledge, technical inexperience, and heightened resident anxiety further challenge patient care. We developed a pediatric anesthesia simulation-based curriculum to address crises related to hypoxemia and dysrhythmia management in the operating room as an adjunct to traditional didactic and clinical experiences. AIMS The primary objective of this trial was to evaluate the impact of a simulation curriculum designed for anesthesiology residents on their performance during the management of crises in the pediatric operating room. A secondary objective was to compare the retention of learned knowledge by assessment at the eight-week time point during the rotation. METHODS In this prospective, observational trial 30 residents were randomized to receive simulation-based education on four perioperative crises (Laryngospasm, Bronchospasm, Supraventricular Tachycardia (SVT), and Bradycardia) during the first week (Group A) or fifth week (Group B) of an eight-week rotation. Assessment sessions that included two scenarios (Laryngospasm, SVT) were performed in the first week, fifth week, and the eighth week of their rotation for all residents. The residents were assessed in real time and by video review using a 7-point checklist generated by a modified Delphi technique of senior pediatric anesthesiology faculty. RESULTS Residents in Group A showed improvement between the first week and fifth week assessment as well as between first week and eighth week assessments without decrement between the fifth week and eighth week assessments for both the laryngospasm and SVT scenarios. Residents in Group B showed improvement between the first week and eighth week assessments for both scenarios and between the fifth week and eighth week assessment for the SVT scenario. CONCLUSION This adjunctive simulation-based curriculum enhanced the learner's management of laryngospasm and SVT management and is a reasonable addition to didactic and clinical curricula for anesthesiology residents.
Collapse
Affiliation(s)
- Aditee P Ambardekar
- Department of Anesthesiology and Pain Management, University of Texas Southwestern Medical School, Dallas, Texas
| | - Stephanie Black
- Department of Anesthesiology and Critical Care Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania
| | - Devika Singh
- Department of Anesthesiology and Critical Care Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania
| | - Justin L Lockman
- Department of Anesthesiology and Critical Care Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania
| | - Allan F Simpao
- Department of Anesthesiology and Critical Care Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania
| | - Alan J Schwartz
- Department of Anesthesiology and Critical Care Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania
| | - Roberta L Hales
- Center for Simulation, Advanced Education, and Innovation, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania
| | - David L Rodgers
- Clinical Simulation Center, Penn State Hershey Medical Center, Hershey, Pennsylvania
| | - Harshad G Gurnaney
- Department of Anesthesiology and Critical Care Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania
| |
Collapse
|
6
|
Julian O, Patrick H, Felix N, Tilman W, Mirco F, Beat-Peter MS, Gerhard S, Tanner MC. Development and validation of an objective assessment scale for chest tube insertion under 'direct' and 'indirect' rating. BMC MEDICAL EDUCATION 2018; 18:320. [PMID: 30587187 PMCID: PMC6307220 DOI: 10.1186/s12909-018-1430-9] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 05/08/2018] [Accepted: 12/14/2018] [Indexed: 06/09/2023]
Abstract
BACKGROUND There is an increasing need for objective and validated educational concepts. This holds especially true for surgical procedures like chest tube insertion (CTI). Thus, we developed an instrument for objectification of learning successes: the assessment scale based on Objective Structured Assessment of Technical Skill (OSATS) for chest tube insertion, which is evaluated in this study. Primary endpoint was the evaluation of intermethod reliability (IM). Secondary endpoints are 'indirect' interrater reliability (IR) and construct validity of the scale (CV). METHODS Every participant (N = 59) performed a CTI on a porcine thorax. Participants received three ratings (one 'direct' on site, two 'indirect' via video rating). IM compares 'direct' with 'indirect' ratings. IR was assessed between 'indirect' ratings. CV was investigated by subgroup analysis based on prior experience in CTI for 'direct' and 'indirect' rating. RESULTS We included 59 medical students to our study. IM showed moderate conformity ('direct' vs. 'indirect 1' ICC = 0.735, 95% CI: 0.554-0.843; 'direct' vs. 'indirect 2' ICC = 0.722, 95% CI 0.533-0.835) and good conformity between 'direct' vs. 'average indirect' rating (ICC = 0.764, 95% CI: 0.6-0.86). IR showed good conformity (ICC = 0.84, 95% CI: 0.707-0.91). CV was proven between subgroups in 'direct' (p = 0.037) and 'indirect' rating (p = 0.013). CONCLUSION Results for IM suggest equivalence for 'direct' and 'indirect' ratings, while both IR and CV was demonstrated in both rating methods. Thus, the assessment scale seems a reliable method for rating trainees' performances 'directly' as well as 'indirectly'. It may help to objectify and facilitate the assessment of training of chest tube insertion.
Collapse
Affiliation(s)
- Ober Julian
- HTRG – Heidelberg Trauma Research Group, Center for Orthopedics, Trauma Surgery and Spinal Cord Injury, Trauma and Reconstructive Surgery, Heidelberg University Hospital, Schlierbacher Landstrasse 200a, D-69118 Heidelberg, Germany
| | - Haubruck Patrick
- HTRG – Heidelberg Trauma Research Group, Center for Orthopedics, Trauma Surgery and Spinal Cord Injury, Trauma and Reconstructive Surgery, Heidelberg University Hospital, Schlierbacher Landstrasse 200a, D-69118 Heidelberg, Germany
| | - Nickel Felix
- Department of General, Visceral and Transplantation Surgery Heidelberg University Hospital, D-69120 Heidelberg, Germany
| | - Walker Tilman
- HTRG – Heidelberg Trauma Research Group, Center for Orthopedics, Trauma Surgery and Spinal Cord Injury, Trauma and Reconstructive Surgery, Heidelberg University Hospital, Schlierbacher Landstrasse 200a, D-69118 Heidelberg, Germany
| | - Friedrich Mirco
- Department of General, Visceral and Transplantation Surgery Heidelberg University Hospital, D-69120 Heidelberg, Germany
| | - Müller-Stich Beat-Peter
- Department of General, Visceral and Transplantation Surgery Heidelberg University Hospital, D-69120 Heidelberg, Germany
| | - Schmidmaier Gerhard
- HTRG – Heidelberg Trauma Research Group, Center for Orthopedics, Trauma Surgery and Spinal Cord Injury, Trauma and Reconstructive Surgery, Heidelberg University Hospital, Schlierbacher Landstrasse 200a, D-69118 Heidelberg, Germany
| | - Michael C. Tanner
- HTRG – Heidelberg Trauma Research Group, Center for Orthopedics, Trauma Surgery and Spinal Cord Injury, Trauma and Reconstructive Surgery, Heidelberg University Hospital, Schlierbacher Landstrasse 200a, D-69118 Heidelberg, Germany
| |
Collapse
|
7
|
Lie D, Richter-Lagha R, Byul Sarah Ma S. A Pilot Comparison of In-Room and Video Ratings of Team Behaviors of Students in Interprofesional Teams. AMERICAN JOURNAL OF PHARMACEUTICAL EDUCATION 2018; 82:6487. [PMID: 30013246 PMCID: PMC6041492 DOI: 10.5688/ajpe6487] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/25/2017] [Accepted: 12/20/2017] [Indexed: 06/08/2023]
Abstract
Objective. To examine concordance between in-room and video faculty ratings of interprofessional behaviors in a standardized team objective structured clinical encounter (TOSCE). Methods. In-room and video-rated student performance scores in an interprofessional 2-station TOSCE were compared using a validated 3-point scale assessing six team competencies. Scores for each student were derived from two in-room faculty members and one faculty member who viewed video recordings of the same team encounter from equivalent visual vantage points. All faculty members received the same rigorous rater training. Paired sample t-tests were used to compare individual student scores. McNemar's test was used to compare student pass/fail rates to determine the impact of rating modality on performance scores. Results. In-room and video student scores were captured for 12 novice teams (47 students) with each team consisting of students from four professions (medicine, pharmacy, physician assistant, nursing). Video ratings were consistently lower for all competencies and significantly lower for competencies of roles and responsibilities, and conflict management. Using a criterion of an average score of 2 out of 3 for at least one station for passing, 56% of students passed when rated in-room compared with 20% when rated by video. Conclusion. In-room and video ratings are not equal. Educators should consider scoring discrepancies based on modality when assessing team behaviors.
Collapse
Affiliation(s)
- Désirée Lie
- Keck School of Medicine, University of Southern California, Los Angeles, California
| | - Regina Richter-Lagha
- Keck School of Medicine, University of Southern California, Los Angeles, California
| | - Sae Byul Sarah Ma
- Keck School of Medicine, University of Southern California, Los Angeles, California
| |
Collapse
|
8
|
Obstetric team simulation program challenges. J Clin Anesth 2016; 35:564-570. [PMID: 27871594 DOI: 10.1016/j.jclinane.2016.08.019] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2015] [Revised: 08/16/2016] [Accepted: 08/17/2016] [Indexed: 11/23/2022]
Abstract
OBJECTIVE To describe the challenges associated with the development and assessment of an obstetric emergency team simulation program. DESIGN The goal was to develop a hybrid, in-situ and high fidelity obstetric emergency team simulation program that incorporated weekly simulation sessions on the labor and delivery unit, and quarterly, education protected sessions in the simulation center. All simulation sessions were video-recorded and reviewed. SETTING Labor and delivery unit and simulation center. PARTICIPANTS Medical staff covering labor and delivery, anesthesiology and obstetric residents and obstetric nurses. MEASUREMENTS Assessments included an on-line knowledge multiple-choice questionnaire about the simulation scenarios. This was completed prior to the initial in-situ simulation session and repeated 3 months later, the Clinical Teamwork Scale with inter-rater reliability, participant confidence surveys and subjective participant satisfaction. A web-based curriculum comprising modules on communication skills, team challenges, and team obstetric emergency scenarios was also developed. MAIN RESULTS Over 4 months, only 6 labor and delivery unit in-situ sessions out of a possible 14 sessions were carried out. Four high-fidelity sessions were performed in 2 quarterly education protected meetings in the simulation center. Information technology difficulties led to the completion of only 18 pre/post web-based multiple-choice questionnaires. These test results showed no significant improvement in raw score performance from pre-test to post-test (P=.27). During Clinical Teamwork Scale live and video assessment, trained raters and program faculty were in agreement only 31% and 28% of the time, respectively (Kendall's W=.31, P<.001 and W=.28, P<.001). Participant confidence surveys overall revealed confidence significantly increased (P<.05), from pre-scenario briefing to after post-scenario debriefing. CONCLUSION Program feedback indicates a high level of participant satisfaction and improved confidence yet further program refinement is required.
Collapse
|
9
|
Towards an Evidence-Based Methodology for High-Stakes Evaluation of Nursing Students' Clinical Performance Using Simulation. TEACHING AND LEARNING IN NURSING 2016. [DOI: 10.1016/j.teln.2016.04.001] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022]
|
10
|
Evans HL, O'Shea DJ, Morris AE, Keys KA, Wright AS, Schaad DC, Ilgen JS. A comparison of Google Glass and traditional video vantage points for bedside procedural skill assessment. Am J Surg 2016; 211:336-42. [DOI: 10.1016/j.amjsurg.2015.07.029] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2015] [Revised: 07/30/2015] [Accepted: 07/30/2015] [Indexed: 10/22/2022]
|
11
|
De la Roca-Chiapas JM, Barbosa-Sabanero G, Martínez-García JA, Martínez-Soto J, Ramos-Frausto VM, González-Ramírez LP, Nowack K. Impact of stress and levels of corticosterone on the development of breast cancer in rats. Psychol Res Behav Manag 2016; 9:1-6. [PMID: 26793009 PMCID: PMC4708200 DOI: 10.2147/prbm.s94177] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022] Open
Abstract
Stress is experienced during cancer, and impairs the immune system’s ability to protect the body. Our aim was to investigate if isolation stress has an impact on the development of tumors in rats, and to measure the size and number of tumors and the levels of corticosterone. Breast cancer was induced in two groups of female rats (N=20) by administration of a single dose of N-methyl-N-nitrosourea 50 mg/kg. Rats in the control group (cancer induction condition) were allowed to remain together in a large cage, whereas in the second group, rats were also exposed to a stressful condition, that is, isolation (cancer induction and isolation condition, CIIC). The CIIC group displayed anxious behavior after 10 weeks of isolation. In the CIIC group, 16 tumors developed, compared with only eleven tumors in the control cancer induction condition group. In addition, compared with the control group, the volume of tumors in the CIIC group was greater, and more rats had more than one tumor and cells showed greater morphological damage. Levels of corticosterone were also significantly different between the two groups. This study supports the hypothesis that stress can influence the development of cancer, but that stress itself is not a sufficient factor for the development of cancer in rats. The study also provides new information for development of experimental studies and controlled environments.
Collapse
Affiliation(s)
| | - Gloria Barbosa-Sabanero
- Department of Medical Sciences, Division of Health Sciences, Campus Leon-University of Guanajuato, Guanajuato, USA
| | | | - Joel Martínez-Soto
- Department of Psychology, Campus Leon-University of Guanajuato, Guanajuato, USA
| | | | | | - Ken Nowack
- Envisia Learning, Inc., Santa Monica, CA, USA
| |
Collapse
|
12
|
Takayesu JK, Kulstad C, Wallenstein J, Gallahue F, Gordon D, Leone K, Kessler C. Assessing patient care: summary of the breakout group on assessment of observable learner performance. Acad Emerg Med 2012; 19:1379-89. [PMID: 23279245 DOI: 10.1111/acem.12038] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2012] [Accepted: 07/01/2012] [Indexed: 11/30/2022]
Abstract
There is an established expectation that physicians in training demonstrate competence in all aspects of clinical care prior to entering professional practice. Multiple methods have been used to assess competence in patient care, including direct observation, simulation-based assessments, objective structured clinical examinations (OSCEs), global faculty evaluations, 360-degree evaluations, portfolios, self-reflection, clinical performance metrics, and procedure logs. A thorough assessment of competence in patient care requires a mixture of methods, taking into account each method's costs, benefits, and current level of evidence. At the 2012 Academic Emergency Medicine (AEM) consensus conference on educational research, one breakout group reviewed and discussed the evidence supporting various methods of assessing patient care and defined a research agenda for the continued development of specific assessment methods based on current best practices. In this article, the authors review each method's supporting reliability and validity evidence and make specific recommendations for future educational research.
Collapse
Affiliation(s)
- James Kimo Takayesu
- Department of Emergency Medicine; Advocate Christ Medical Center l; Boston; MA
| | - Christine Kulstad
- Department of Emergency Medicine; Advocate Christ Medical Center; Oak Lawn; IL
| | | | - Fiona Gallahue
- Department of Medicine; University of Washington Medical Center; Seattle; WA
| | - David Gordon
- Department of Emergency Medicine; Duke University Medical Center; Durham; NC
| | - Katrina Leone
- Department of Emergency Medicine; Oregon Health and Science University; Portland; OR
| | - Chad Kessler
- Department of Emergency Medicine; Jesse Brown VA Medical Center; Chicago; IL
| |
Collapse
|
13
|
House JB, Dooley-Hash S, Kowalenko T, Sikavitsas A, Seeyave DM, Younger JG, Hamstra SJ, Nypaver MM. Prospective comparison of live evaluation and video review in the evaluation of operator performance in a pediatric emergency airway simulation. J Grad Med Educ 2012; 4:312-6. [PMID: 23997874 PMCID: PMC3444183 DOI: 10.4300/jgme-d-11-00123.1] [Citation(s) in RCA: 24] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/01/2011] [Revised: 11/13/2011] [Accepted: 01/09/2012] [Indexed: 11/06/2022] Open
Abstract
INTRODUCTION Real-time assessment of operator performance during procedural simulation is a common practice that requires undivided attention by 1 or more reviewers, potentially over many repetitions of the same case. OBJECTIVE To determine whether reviewers display better interrater agreement of procedural competency when observing recorded, rather than live, performance; and to develop an assessment tool for pediatric rapid sequence intubation (pRSI). METHODS A framework of a previously established Objective Structured Assessment of Technical Skills (OSATS) tool was modified for pRSI. Emergency medicine residents (postgraduate year 1-4) were prospectively enrolled in a pRSI simulation scenario and evaluated by 2 live raters using the modified tool. Sessions were videotaped and reviewed by the same raters at least 4 months later. Raters were blinded to their initial rating. Interrater agreement was determined by using the Krippendorff generalized concordance method. RESULTS Overall interrater agreement for live review was 0.75 (95% confidence interval [CI], 0.72-0.78) and for video was 0.79 (95% CI, 0.73-0.82). Live review was significantly superior to video review in only 1 of the OSATS domains (Preparation) and was equivalent in the other domains. Intrarater agreement between the live and video evaluation was very good, greater than 0.75 for all raters, with a mean of 0.81 (95% CI, 0.76-0.85). CONCLUSION The modified OSATS assessment tool demonstrated some evidence of validity in discriminating among levels of resident experience and high interreviewer reliability. With this tool, intrareviewer reliability was high between live and 4-months' delayed video review of the simulated procedure, which supports feasibility of delayed video review in resident assessment.
Collapse
|
14
|
Okuyama A, Martowirono K, Bijnen B. Assessing the patient safety competencies of healthcare professionals: a systematic review. BMJ Qual Saf 2011; 20:991-1000. [PMID: 21880646 PMCID: PMC3203526 DOI: 10.1136/bmjqs-2011-000148] [Citation(s) in RCA: 50] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Background Patient safety training of healthcare professionals is a new area of education. Assessment of the pertinent competencies should be a part of this education. This review aims to identify the available assessment tools for different patient safety domains and evaluate them according to Miller's four competency levels. Methods The authors searched PubMed, MEDLINE, the Cumulative Index to Nursing and Allied Health Literature (CINAHL), Web of Science, psycINFO and the Education Resource Information Center (ERIC) from the start of each database to December 2010 for English-language articles that evaluated or described tools for the assessment of the safety competencies of individual medical and/or nursing professionals. Reports on the assessment of technical, clinical, medication and disclosure skills were excluded. Results Thirty-four assessment tools in 48 studies were identified: 20 tools for medical professionals, nine tools for nursing professionals, and five tools for both medical and nursing professionals. Twenty of these tools assessed the two highest Miller levels (‘shows how’ and ‘does’) and four tools were directed at multiple levels. Most of the tools that aimed at the higher levels assessed the skills of working in teams (17 tools), risk management (15 tools), and communication (11 tools). Internal structure (reliability, 22 tools) and content validity (14 tools) when described were found to be moderate. Only a small number of tools addressed the relationship between the tool itself and (1) other assessments (concurrent, predictive validity, eight tools), and (2) educational outcomes (seven tools). Conclusions There are many tools designed to assess the safety competencies of healthcare professionals. However, a reliable and valid toolbox for summative testing that covers all patient safety domains at Miller's four competency levels cannot yet be constructed. Many tools, however, are useful for formative feedback.
Collapse
Affiliation(s)
- Ayako Okuyama
- Foreest Medical School, Medical Center Alkmaar, Alkmaar, The Netherlands.
| | | | | |
Collapse
|
15
|
Abstract
Simulation provides a range of educational tools that have increasingly been incorporated into emergency medicine (EM) curricula. Standardized patients and some partial task trainers, such as intubation heads, have been used for decades. More recently, a growing number of computer-screen simulations, high-fidelity mannequins, and virtual-reality simulators have expanded the number of procedures and conditions, which can be effectively simulated.The Accreditation Council for Graduate Medical Education transitioned to a competency-based assessment of residency programs in 2001 and included simulation as a method for incorporating the 6 core competencies into graduate medical education curricula. Over the past decade, numerous peer-reviewed publications have promoted simulation as an effective educational tool for each of the core competencies.The advanced technology used to operate many current simulators can erroneously become the focus of efforts to create a simulation-based curriculum. Simulation can most effectively be incorporated into EM curricula through the use of time-proven concepts, which start with defining the targeted learners, assessing their general and specific educational needs, defining learning objectives, and selecting the best educational strategy for achieving each objective. In many, but not all, instances, simulation can be the best tool for achieving EM learning objectives.
Collapse
|
16
|
Kuhn GJ, Shayne P, Coates WC, Fisher J, Lin M, Maggio LA, Farrell SE. Critical appraisal of emergency medicine educational research: the best publications of 2009. Acad Emerg Med 2010; 17 Suppl 2:S16-25. [PMID: 21199079 DOI: 10.1111/j.1553-2712.2010.00899.x] [Citation(s) in RCA: 16] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Abstract
OBJECTIVES The objective was to critically appraise and highlight methodologically superior medical education research specific to emergency medicine (EM) published in 2009. METHODS A search of the English language literature in 2009 querying Ovid MEDLINE In-Process & Other Non-Indexed Citations, Ovid MEDLINE 1950 to Present, Web of Science, Education Resources Information Center (ERIC), and PsychInfo identified 36 EM studies that used hypothesis-testing or observational investigations of educational interventions. Six reviewers independently ranked all publications based on 10 criteria, including four related to methodology, that were chosen a priori to standardize evaluation by reviewers. This was a refinement of the methods used to appraise medical education published in 2008. RESULTS Seven studies met the standards as determined by the averaged rankings and are highlighted and summarized here. This year, 16 of 36 (44%) identified studies had funding, compared to 11 of 30 (36%) identified last year; five of seven (71%) highlighted publications were funded in 2009 compared to three of five (60%) highlighted in 2008. Use of technology in medical education was reported in 14 identified and four highlighted publications, with simulation being the most common technology studied. Five of the seven (71%) featured publications used a quasi-experimental or experimental design, one was observational, and one was qualitative. Practice management topics, including patient safety, efficiency, and revenue generation, were examined in seven reviewed studies. CONCLUSIONS Thirty-six medical education publications published in 2009 focusing on EM were identified. This critical appraisal reviews and highlights seven studies that met a priori quality indicators. Current trends are noted.
Collapse
Affiliation(s)
- Gloria J Kuhn
- Department of Emergency Medicine, Wayne State, University, Detroit, MI, USA.
| | | | | | | | | | | | | |
Collapse
|