What should be included in the assessment of laypersons' paediatric basic life support skills? Results from a Delphi consensus study

Indigenizing and co-producing the ACGME anesthesiology milestone in Taiwan: a Delphi study and subgroup analysis

BACKGROUND

To implement the ACGME Anesthesiology Milestone Project in a non-North American context, a process of indigenization is essential. In this study, we aim to explore the differences in perspective toward the anesthesiology competencies among residents and junior and senior visiting staff members and co-produce a preliminary framework for the following nation-wide survey in Taiwan.

METHODS

The expert committee translation and Delphi technique were adopted to co-construct an indigenized draft of milestones. Descriptive analysis, chi-square testing, Pearson correlation testing, and repeated-measures analysis of variance in the general linear model were employed to calculate the F values and mean differences (MDs).

RESULTS

The translation committee included three experts and the consensus panel recruited 37 participants from four hospitals in Taiwan: 9 residents, 13 junior visiting staff members (JVSs), and 15 senior visiting staff members (SVSs). The consensus on the content of the 285 milestones was achieved after 271 minor and 6 major modifications in 3 rounds of the Delphi survey. Moreover, JVSs were more concerned regarding patient care than were both residents (MD = - 0.095, P < 0.001) and SVSs (MD = 0.075, P < 0.001). Residents were more concerned regarding practice-based learning improvement than were JVSs (MD = 0.081; P < 0.01); they also acknowledged professionalism more than JVSs (MD = 0.072; P < 0.05) and SVSs (MD = 0.12; P < 0.01). Finally, SVSs graded interpersonal and communication skills lower than both residents (MD = 0.068; P < 0.05) and JVSs (MD = 0.065; P < 0.05) did.

CONCLUSIONS

Most ACGME anesthesiology milestones are applicable and feasible in Taiwan. Incorporating residents' perspectives may bring insight and facilitate shared understanding to a new educational implementation. This study helped Taiwan generate a well-informed and indigenized draft of a competency-based framework for the following nation-wide Delphi survey.

Artificial intelligence in medical education curriculum: An e-Delphi study for competencies

Background

Artificial intelligence (AI) has affected our day-to-day in a great extent. Healthcare industry is one of the mainstream fields among those and produced a noticeable change in treatment and education. Medical students must comprehend well why AI technologies mediate and frame their decisions on medical issues. Formalizing of instruction on AI concepts can facilitate learners to grasp AI outcomes in association with their sensory perceptions and thinking in the dynamic and ambiguous reality of daily medical practice. The purpose of this study is to provide consensus on the competencies required by medical graduates to be ready for artificial intelligence technologies and possible applications in medicine and reporting the results.

Materials and methods

A three-round e-Delphi survey was conducted between February 2020 and November 2020. The Delphi panel accorporated experts from different backgrounds; (i) healthcare professionals/ academicians; (ii) computer and data science professionals/ academics; (iii) law and ethics professionals/ academics; and (iv) medical students. Round 1 in the Delphi survey began with exploratory open-ended questions. Responses received in the first round evaluated and refined to a 27-item questionnaire which then sent to the experts to be rated using a 7-point Likert type scale (1: Strongly Disagree—7: Strongly Agree). Similar to the second round, the participants repeated their assessments in the third round by using the second-round analysis. The agreement level and strength of the consensus was decided based on third phase results. Median scores was used to calculate the agreement level and the interquartile range (IQR) was used for determining the strength of the consensus.

Results

Among 128 invitees, a total of 94 agreed to become members of the expert panel. Of them 75 (79.8%) completed the Round 1 questionnaire, 69/75 (92.0%) completed the Round 2 and 60/69 (87.0%) responded to the Round 3. There was a strong agreement on the 23 items and weak agreement on the 4 items.

Conclusions

This study has provided a consensus list of the competencies required by the medical graduates to be ready for AI implications that would bring new perspectives to medical education curricula. The unique feature of the current research is providing a guiding role in integrating AI into curriculum processes, syllabus content and training of medical students.

Core Components of a Pediatric Critical Care Transport Communication Curriculum: A Modified Delphi Approach

OBJECTIVE

Interfacility transport of critically ill infants and children is an essential part of the care of children in the United States. However, there is tremendous variation in how transports are coordinated and performed. Pediatric critical care medicine (PCCM) fellows have differing experiences in their fellowships, and there is no standardized way of training medical command for the transport process. The aim of this study was to use a consensus-building process to establish core components of a PCCM transport curriculum focused on communication.

METHODS

A national group of experts in transport medicine rated 51 total possible topics for their importance to include in a fellowship curriculum. Three rounds of surveys were completed.

RESULTS

Fifty-two of 372 invitees (14%) participated in round 1. Consensus was reached to include 15 items in a PCCM curriculum. Twenty of 52 (38%) experts completed round 2, reaching consensus on 2 additional items. Seventeen of 20 (85%) experts completed round 3. No additional items reached consensus.

CONCLUSION

Experts reached consensus on 17 core components to include in a PCCM fellowship transport communication curriculum. This curriculum could likely be adapted to train providers from different disciplines in the transport process.

The Copenhagen Tool a research tool for evaluation of basic life support educational interventions

INTRODUCTION

Over the past decades, major changes have been made in basic life support (BLS) guidelines and manikin technology. The aim of this study was to develop a BLS evaluation tool based on international expert consensus and contemporary validation to enable more valid comparison of research on BLS educational interventions.

METHODS

A modern method for collecting validation evidence based on Messick's framework was used. The framework consists of five domains of evidence: content, response process, internal structure, relations with other variables, and consequences. The research tool was developed by collecting content evidence based on international consensus from an expert panel; a modified Delphi process decided items essential for the tool. Agreement was defined as identical ratings by 70% of the experts.

RESULTS

The expert panel established consensus on a three-levelled score depending on expected response level: laypersons, first responders, and health care personnel. Three Delphi rounds with 13 experts resulted in 16 "essential" items for laypersons, 21 for first responders, and 22 for health care personnel. This, together with a checklist for planning and reporting educational interventional studies within BLS, serves as an example to be used for researchers.

CONCLUSIONS

An expert panel agreed on a three-levelled score to assess BLS skills and the included items. Expert panel consensus concluded that the tool serves its purpose and can act to guide improved research comparison on BLS educational interventions.

Consensus Core Point-of-care Ultrasound Applications for Pediatric Emergency Medicine Training

BACKGROUND

Pediatric emergency medicine (PEM) physicians have variably incorporated point-of-care ultrasound (POCUS) into their practice. Prior guidelines describe the scope of POCUS practice for PEM physicians; however, consensus does not yet exist about which applications should be prioritized and taught as fundamental skills for PEM trainees. Members of the PEM POCUS Network (P2Network) conducted a consensus-building process to determine which applications to incorporate into PEM fellowship training.

METHODS

A multinational group of experts in PEM POCUS was recruited from the P2Network and greater PEM POCUS community if they met the following criteria: performed over 1,000 POCUS scans and had at least 3 years of experience teaching POCUS to PEM fellows, were a local academic POCUS leader, or completed a formal PEM POCUS fellowship. Experts rated 60 possible PEM POCUS applications for their importance to include as part of a PEM fellowship curriculum using a modified Delphi consensus-building technique.

RESULTS

In round 1, 66 of 92 (72%) participants responded to an e-mail survey of which 48 met expert criteria and completed the survey. Consensus was reached to include 18 items in a PEM fellowship curriculum and to exclude two items. The 40 remaining items and seven additional items were considered in round 2. Thirty-seven of 48 (77%) experts completed round 2 reaching consensus to include three more items and exclude five. The remaining 39 items did not reach consensus for inclusion or exclusion.

CONCLUSION

Experts reached consensus on 21 core POCUS applications to include in PEM fellowship curricula.

Improving the cost-effectiveness of laypersons' paediatric basic life support skills training: A randomised non-inferiority study

AIM

To compare dyad (training in pairs without an instructor) with resource-intensive instructor-led training for laypersons' paediatric resuscitation skills in a non-inferiority trial and examine cost-effectiveness of the training methods.

METHODS

In this randomised parallel group non-inferiority trial, 155 dyad and 175 instructor-led laypersons were trained in Basic Life Support and Foreign Body Airway Obstruction Management. Dyads were given instructional videos, hands-on exercises and provided feedback to their partner for 50 min. Instructor-led laypersons trained in groups of six for two hours. Learning were assessed in scenarios immediately after training and, subsequently, at 14 days, 1, and 3 months. Pass rates, cost-effectiveness of producing a competent layperson (passing both tests), and non-inferiority were analysed.

RESULTS

Sixty-eight (45.6%) dyad and 130 (74.3%) instructor-led laypersons passed the basic life support test (p < 0.001). For Foreign Body Airway Obstruction Management 77 (54.2%) dyad and 130 (79.3%) for instructor-led laypersons passed (p < 0.001). Skills decreased over three months for both groups. Forty-two (30.4%) dyad and ninety-eight (59.8%) for instructor-led laypersons were competent after training (p < 0.001). The lower effectiveness of dyad training had reduced costs (p < 0.001). For each 10,000 USD allocated to training, dyad training would result in 71 vs. 65 competent laypersons for instructor-led training. Non-inferiority of dyad training could not be established.

CONCLUSION

Instructor-led training was the most effective but also the most expensive training method, making it less cost-effective than dyad training. When the aim is to train for quantity rather than quality, dyad training would be the preferred choice of training method.

Assessment of laypersons' paediatric basic life support and foreign body airway obstruction management skills: a validity study

Background

Standardised courses for laypeople in Paediatric Basic Life Support (PBLS) and Foreign Body Airway Obstruction Management (FBAOM) teach essential skills for the initiation of resuscitation by bystanders. Performance assessments are necessary to ensure that skills are acquired. We aimed to examine the validity of developed performance assessments and to determine credible pass/fail standards.

Methods

Validity evidence was gathered in a standardised simulated setting by testing participants with three different levels of PBLS/FBAOM experience: untrained laypersons, trained laypersons, and lifeguards. Two blinded raters assessed participants’ performance. The reliability of test scores was analysed using generalizability theory, scores were compared across the three groups, and pass/fail-standards were established.

Results

A total of 33 participants were included. More than two raters and two cases were necessary for PBLS to achieve a reliability coefficient above 0.80, which is considered the minimally acceptable level for high-stakes certification. For FBAOM, two tests or three raters were needed. Assessment scores differed across the three groups for PBLS skills, as well as for FBAOM skills (p < 0.001).

Pass levels of 74% and 55% of the maximum score for PBLS and FBAOM, respectively, were identified as the levels that best discriminated between competent and non-competent laypersons.

Conclusions

Laypersons’ PBLS and FBAOM skills can be assessed in a reliable and valid way in a standardised simulated setting. However, multiple raters and scenario tests are needed to ensure sufficient reliability, which raises questions regarding the feasibility of performing certification tests for laypersons who participate in short paediatric resuscitation courses.

Electronic supplementary material

The online version of this article (10.1186/s13049-018-0544-8) contains supplementary material, which is available to authorized users.