Evaluation methods and impact of simulation-based training in pediatric surgery: a systematic review

Developing an Effective Off-the-job Training Model and an Automated Evaluation System for Thoracoscopic Esophageal Atresia Surgery

BACKGROUND

Pediatric minimally invasive surgery requires advanced technical skills. Off-the-job training (OJT), especially when using disease-specific models, is an effective method of acquiring surgical skills. To achieve effective OJT, it is necessary to provide objective and appropriate skill assessment feedback to trainees. We aimed to construct a system that automatically evaluates surgical skills based on forceps movement using deep learning (DL).

METHODS

Using our original esophageal atresia OJT model, participants were tasked with performing esophageal anastomosis. All tasks were recorded for image analysis. Based on manual objective skill assessments, each participant's surgical skills were categorized into two groups: good and poor. The motion of the forceps in both groups was used as training data. Employing this training data, we constructed an automated system that recognized the movement of forceps and determined the quality of the surgical technique.

RESULTS

Thirteen participants were assigned to the good skill group and 32 to the poor skill group. These cases were validated using an automated skill assessment system. This system showed a precision of 75%, a specificity of 94%, and an area under the receiver operating characteristic curve of 0.81.

CONCLUSIONS

We constructed a system that automatically evaluated the quality of surgical techniques based on the movement of forceps using DL. Artificial intelligence diagnostics further revealed the procedures important for suture manipulation.

LEVELS OF EVIDENCE

Level IV.

Enhancing neonatal thoracoscopic surgical training with rabbit model

Background

Thoracoscopy, which has an increasing role in the treatment of indexed neonatal surgical conditions, requires adequate training. To support this, the current study aimed to evaluate the feasibility and effectiveness of using live rabbit models in neonatal thoracoscopic skills training among paediatric surgeons.

Methods

Following didactic lectures and demonstrations, the participants were given hands-on opportunities to perform thoracoscopic procedures. The feasibility and effectiveness of using live rabbit models in neonatal thoracoscopic skills training among paediatric surgeons were evaluated with pre-/post-course procedural confidence scores and a questionnaire.

Results

This study included 13 paediatric surgeons-2 (15 %) males and 11 (85 %) females-who were evenly distributed. There were four basic surgical trainees, five higher surgical trainees and four fellows in paediatric surgery (mean surgical practice experience: 4.5 ± 3.7 years). Most had experience assisting paediatric (70 %) and neonatal (62 %) thoracoscopic surgery. Only 30 % had experience as the chief surgeon of paediatric thoracoscopic surgery, with none on neonates. Significant improvement was seen in procedural confidence as the assistant and chief surgeon of all procedures post-workshop. The surgeons rated the model positively.

Conclusion

The procedural confidence level of paediatric surgeons improved significantly after workshop participation. This realistic and easily reproducible model can help perfect thoracoscopic skills. Therefore, its integration into paediatric surgical training would promote surgical skill proficiency and could improve surgeons' confidence in neonate operations.

Evaluation of skill acquisition characteristics depending on the size of a dry box

BACKGROUND

Few studies have analyzed the effect of the size of the working space in training on the acquisition of endoscopic skills. In this study, adult- and infant-sized dry boxes (DBs) were used to verify how the size of the working space in training affects forceps manipulation and learning curve.

MATERIAL AND METHODS

Seventy-two medical students were enrolled. The task was peg transfer. The training environment was divided into adult- and infant-sized DBs. Skill evaluations were also divided into adult- and infant-sized DBs (four groups in total). The forceps manipulation characteristics and task completion time were compared before and after training.

RESULTS

Regarding skill evaluations using adult-sized DBs, there were no significant differences between the infant- and adult-sized DB-trained groups. Regarding skill evaluations using infant-sized DBs, there were no significant differences between the groups before training. After training, there was no significant difference in the total path length or average acceleration of the forceps between the groups. However, the infant-sized DB-trained group had a significantly faster average forceps velocity and faster task completion time than the adult-sized DB-trained group.

CONCLUSION

Training with a small DB is more efficient in acquiring smoother and faster forceps manipulation in a small working space.

What are the learning objectives in surgical training - a systematic literature review of the surgical competence framework

OBJECTIVE

To map the landscape of contemporary surgical education through a competence framework by conducting a systematic literature review on learning outcomes of surgical education and the instructional methods applied to attain the outcomes.

BACKGROUND

Surgical education has seen a paradigm shift towards competence-based training. However, a gap remains in the literature regarding the specific components of competency taught and the instructional methods employed to achieve these outcomes. This paper aims to bridge this gap by conducting a systematic review on the learning outcomes of surgical education within a competence framework and the instructional methods applied. The primary outcome measure was to elucidate the components of competency emphasized by modern surgical curricula. The secondary outcome measure was to discern the instructional methods proven effective in achieving these competencies.

METHODS

A search was conducted across PubMed, Medline, ProQuest Eric, and Cochrane databases, adhering to PRISMA guidelines, limited to 2017-2021. Keywords included terms related to surgical education and training. Inclusion criteria mandated original empirical studies that described learning outcomes and methods, and targeted both medical students and surgical residents.

RESULTS

Out of 42 studies involving 2097 participants, most concentrated on technical skills within competency-based training, with a lesser emphasis on non-technical competencies. The effect on clinical outcomes was infrequently explored.

CONCLUSION

The shift towards competency in surgical training is evident. However, further studies on its ramifications on clinical outcomes are needed. The transition from technical to clinical competence and the creation of validated assessments are crucial for establishing a foundation for lifelong surgical learning.

Advances in Pediatric Surgery Simulation-Based Training

Pediatric surgery is the diagnostic, operative, and postoperative surgical care of children with congenital and acquired anomalies and diseases. The early history of the specialty followed the classic "see one, do one, teach one" philosophy of training but has since evolved to modern methods including simulation-based training (SBT). Current trainees in pediatric surgery face numerous challenges, such as the decreasing incidence of congenital disease and reduced work hours. SBT consists of several modalities that together assist in the acquisition of technical skills and improve performance in the operating room. SBT has evolved to incorporate simulator models and video gaming technology, in parallel with the development of simulation in other surgical and non-surgical pediatric fields. SBT has advanced to a level of sophistication that means that it can improve the skills of not only pediatric surgery trainees but also practicing attending surgeons. In this review, we will discuss the history of pediatric surgery, simulation in pediatric surgery training, and the potential direction of pediatric surgical simulation training in the future.

How many cases do instructor class pediatric surgeons need to experience to be an independent operator in performing advanced endoscopic surgery? A nationwide survey to establish an ideal curriculum for pediatric endoscopic surgery in Japan

PURPOSE

To ensure the safe spread of pediatric endoscopic surgery, it is essential to build a training curriculum, and a survey of the current situation in Japan is necessary. The present study assessed an efficient training curriculum by clarifying instructor class pediatric surgeons' experiences, including autonomy when performing advanced endoscopic surgeries.

METHODS

An online nationwide questionnaire survey was conducted among pediatric surgeons who had Endoscopic Surgical Skill Qualification (ESSQ) and board-certified instructors who had skills comparable to ESSQ. We assessed participants' training experience, opinions concerning the ideal training curriculum, and the correlation between surgical experience and the level of autonomy. The Zwisch scale was used to assess autonomy.

RESULTS

Fifty-two participants responded to the survey (response rate: 86.7%). Only 57.7% of the respondents felt that they had received sufficient endoscopic surgery training. Most respondents considered an educational curriculum for endoscopic surgery including off-the-job training essential during the training period. Autonomy had been acquired after experiencing two to three cases for most advanced endoscopic surgeries.

CONCLUSION

This first nationwide survey in Japan showed that instructor class pediatric surgeons acquired autonomy after experiencing two to three for most advanced endoscopic surgeries. Our findings suggest that training, especially off-the-job training, has been insufficient.

Comprehension of fundamental knowledge about pediatric endoscopic surgery: a cross-sectional study in Japan

BACKGROUND

Pediatric endoscopic surgery has become an alternative to conventional techniques with the development of medical equipment. However, there is no formal, standardized curriculum for pediatric endoscopic surgery, and its requirement remains elusive. The purpose of this study is to determine the baseline knowledge of pediatric surgeons that is considered essential for pediatric laparoscopic and thoracoscopic surgery.

METHODS

Sixteen web-based multiple-choice questions based on the fundamental cognitive knowledge of pediatric endoscopic surgery were administered. The questions were created based on the fields covered by the Fundamentals of Laparoscopic Surgery ™ (FLS) certification examination blueprints and eight specific diseases of pediatric surgery. Pediatric surgeons and pediatric surgical trainees participated in this study voluntarily.

RESULTS

A total of 122 surgeons participated through the Japanese Society of Pediatric Surgeons. The response rate was 95% (122/128). The total mean examination score of all participants was 79.4% (77.3-81.4%). There were no significant differences in total scores between the board-certified pediatric surgeons without an endoscopic surgical skill qualification and the non-board-certified pediatric surgeons (80.4% vs. 77.1%, p = 0.12). The endoscopic surgical skill-qualified surgeons had significantly higher percentages of correct responses in specific subjects than board-certified pediatric surgeons and surgeons without pediatric board certification (94.3% vs. 82.9%, p = 0.02; 94.3% vs. 77.5%, p = 0.0002). The FLS original subjects' scores were not significantly different among them. The mean score of surgeons who had experienced more than 200 cases of endoscopic surgery, including adult cases, was 83.2% (80.4-85.9%).

CONCLUSIONS

A knowledge gap exists between surgeons, board-certified pediatric surgeons, and endoscopic surgical skill-qualified surgeons in Japan. In the field of pediatric surgery, an effective formal curriculum, such as FLS, is required to help address this vast knowledge gap for the safe conduct of endoscopic surgeries.

Development and Validation of a Laparoscopy Simulation Model of Pyeloplasty for Pediatric Patients

Introduction: Given the extensive training required for laparoscopic pyeloplasty and the difficulties achieving this training, there is a need to recreate the activity in a controlled environment, but high-fidelity models are unavailable or expensive. Our objective was to develop a model of pyeloureteral junction stenosis, resembling the anatomical details and consistency of natural tissue, for a replicable, cheaper, and realistic simulation model of laparoscopic pyeloplasty in children. Materials and Methods: A three-dimensional, printed synthetic model was created from magnetic resonance urography. The model comprises a plastic kidney as the reusable structure and a silicone renal pelvis and ureter as the interchangeable structure. We evaluated realism and performance with surgeons and residents at different levels of training, comparing operative time and complications of the procedure. Results: Twenty-four participants were recruited; 41.7% had previous experience in laparoscopic pyeloplasty, with 5.5 years of experience in laparoscopic surgery (interquartile range [IQR] 2-7.75). There were no cases of stenosis, but leaks accounted for 41.7%. The procedure lasted 72 minutes (IQR 55-90), with significant differences according to the level of training (85 minutes for residents, 68 minutes for pediatric surgeons and urologists, and 40 minutes for laparoscopic surgeons; P: .011) and years of previous experience in laparoscopic surgery (P: .003). Conclusions: A high-fidelity, replicable, and low-cost pyeloureteral stenosis model was developed to simulate laparoscopic pyeloplasty in pediatric patients.

Validated simulation models in pediatric surgery: A review

INTRODUCTION

This review evaluates the validation and availability of simulation models in the field of pediatric surgery that can be used for training purposes.

METHODS

MEDLINE and EMBASE were searched for studies describing a simulation models in pediatric surgery. Articles were included if face, content and/or construct validity was described. Additionally, the costs and availability were assessed. Validation scores for each model were depicted as percentage (0-100), based on the reported data, to compare the outcomes. A score of >70% was considered adequate.

RESULTS

Forty-three studies were identified, describing the validation process of 38 simulation models. Face validity was evaluated in 33 articles, content in 36 and construct in 19. Twenty-two models received adequate validation scores (>70%). The majority (27/38, 70%) was strictly inanimate. Five models were available for purchase and eleven models were replicable based on the article.

CONCLUSION

The number of validated inanimate simulation models for pediatric surgery procedures is growing, however, few are replicable or available for widespread training purposes.

LEVEL OF EVIDENCE

Level II.

Encouraging Young Pediatric Surgeons and Evaluation of the Effectiveness of a Pediatric Endosurgery Workshop by Self-Assessment and an Objective Skill Validation System

Background: Japanese pediatric endosurgery experts conducted a workshop for young pediatric surgeons in Russia in collaboration with Russian expert pediatric surgeons. This study was aimed to develop a contributive workshop program and evaluate its impact on young pediatric surgeons. Methods: A 2-day pediatric endosurgery workshop was held in Moscow in February 2020. After conducting a needs assessment survey, Japanese and Russian faculties developed the workshop contents, including pre- and postworkshop skills assessments, lectures, and hands-on training. Skills assessments were performed using the objective skill validation system, the "A-Lap Mini," mimicking intestinal anastomosis. The trainees self-evaluated their knowledge and skills using a five-point scale. Results: Fifteen novice trainee participated and 14 (93.3%) completed the workshop program. The completion rate for the suturing task before and after the workshop was 40.0% (6/15) and 85.7% (12/14), respectively. The following five skill evaluation criteria, which were objectively evaluated: performance time changed from 751.6 ± 247.1 seconds to 780.0 ± 313.3 seconds (P > .05), number of full-thickness sutures improved from 1.0 ± 1.41 to 2.64 ± 0.84 (P = .003), area of wound-opening changed from 0.42 ± 0.83 mm² to 0.53 ± 1.13 mm² (P > .05), suture tension improved from 55.48% ± 19.51% to 61.95% ± 23.91% (P > .05), and maximum air leakage pressure improved from 3.76 ± 2.11 kPa to 8.42 ± 7.68 kPa (P > .05). Regarding the self-assessed questionnaire administered before and after the workshop, the confidence in endosurgery skills significantly improved as follows: forceps manipulation ability improved from 2.7 to 3.7 (P < .05), and suturing performance improved from 2.5 to 3.6 (P < .05). The usefulness of the workshop for clinical surgery was scored at 4.3. Conclusions: Quantitative skill evaluation with an automatic feedback function was useful for endosurgery training. Delivering feedback concerning the assessment results to the trainee helps them to determine the specific training requirements needed for clinical endosurgery.

Trends and Innovations of Simulation for Twenty First Century Medical Education

In the last two decades there has been an enormous growth in the use of clinical simulation. This teaching-learning methodology is currently the main tool used in the training of healthcare professionals. Clinical simulation is in tune with new paradigms in education and is consistent with educational theories that support the use of experiential learning. It promotes the development of psychomotor skills and strengthens executive functions. This pedagogical approach can be applied in many healthcare topics and is particularly relevant in the context of restricted access to clinical settings. This is particularly relevant considering the current crisis caused by the COVID-19 pandemic, or when trying to reduce the frequency of accidents attributed to errors in clinical practice. This mini-review provides an overview of the current literature on healthcare simulation methods, as well as prospects for education and public health benefits. A literature search was conducted in order to find the most current trends and state of the art in medical education simulation. Presently, there are many areas of application for this methodology and new areas are constantly being explored. It is concluded that medical education simulation has a solid theoretical basis and wide application in the training of health professionals at present. In addition, it is consolidated as an unavoidable methodology both in undergraduate curricula and in continuing medical education. A promising scenario for medical education simulation is envisaged in the future, hand in hand with the development of technological advances.

Face and construct validity assessment of training models for intestinal anastomosis in low-birth-weight infants

PURPOSE

It is difficult to perform intestinal anastomosis in low-birth-weight infants because the intestinal diameter is small and the discrepancy in diameter of the proximal and distal intestines is often large, but there has been no optimal-sized training model. Therefore, we developed a new intestinal anastomosis training model that imitated the size of the intestine in low-birth-weight infants, and evaluated its face and construct validity.

METHODS

Two intestinal models were developed with crossMedical, Inc. using a hydrophilic acrylic material (wet model) or a polyurethane soft resin (dry model). The inner diameter of the simulated intestinal tract was 15 mm on the oral end and 6 mm on the anal end. Thirteen pediatric surgeons performed anastomosis and responded to the questionnaire.

RESULTS

In the questionnaire, the wet model had significantly higher scores than the dry model in "appearance", "softness" and "usefulness for training". In the anastomotic results of the wet model, the anastomosis leak pressure was significantly correlated with the number of intestinal anastomotic experiences in low-birth-weight infants (correlation coefficient = 0.64, P = 0.035).

CONCLUSIONS

The wet-type intestinal anastomosis model showed good face validity. Its leak pressure had a significant correlation with clinical experience; thus, construct validity was demonstrated.

Trends in fundoplication volume for pediatric gastroesophageal reflux disease

INTRODUCTION

Fundoplication for gastro-esophageal reflux disease (GERD) has been commonly performed by pediatric surgeons, however there are no recent data documenting fundoplication trends. Changes in fundoplication volume impact pediatric surgical training and may reflect changes in care for children with severe GERD.

MATERIALS & METHODS

The Pediatric Health Information System (PHIS) was queried from 2010-2019 for children with ICD-9/ICD-10 codes for GERD, fundoplication, and gastrostomy. Institutional surgical volume and patient demographics were examined over time. A secondary analysis using the Accreditation Council for Graduate Medical Education case logs for pediatric surgery fellows was performed across the same years to assess effects upon surgical volume for trainees.

RESULTS

Mean institutional fundoplication case volume decreased from 50 in 2010 to 17 in 2019. Trends were similar between institutions with and without fellowship programs when corrected by total operative volume. Patient characteristics were relatively unchanged between 2010 and 2019. Fundoplication volume reported in fellow case logs decreased from 46 in 2010 to 26 in 2019, mirroring national data.

CONCLUSIONS

Institutional volume for fundoplication in children with GERD has seen a 3-fold decrease over the last decade, mirrored by an almost 2-fold decrease in case volume reported by pediatric surgery fellows.

Pediatric Surgery Simulation-Based Training for the General Surgery Resident

BACKGROUND

Surgical simulation-based training (SBT) can increase resident confidence and improve performance. SBT in pediatric surgery is in its infancy and often geared toward training pediatric surgery fellows. Since case volume for various pediatric surgery-specific procedures can be low based on the rarity of the pathology involved and the level of care provided by the institution, our aim was to create a pediatric surgery simulation-based curriculum for general surgery residents to address this need.

MATERIALS AND METHODS

We performed an institutional needs assessment consisting of 4 pediatric surgeons' and 28 general surgery residents' confidence in resident ability to independently perform pediatric surgery-specific tasks and procedures using a Likert-scaled survey. These included the placement of a silastic silo for gastroschisis, a percutaneous drain for perforated necrotizing enterocolitis, and completion of a laparoscopic pyloromyotomy for pyloric stenosis. Models simulating these pathologies and curriculum for performing each procedure were generated.

RESULTS

We successfully created a model and SBT curriculum to teach general surgery residents how to place a silastic silo for patients with gastroschisis, a percutaneous drain for patients with perforated necrotizing enterocolitis, and how to complete a laparoscopic pyloromyotomy for patients with pyloric stenosis. These were deemed high fidelity models based on a survey of our pediatric surgeons.

CONCLUSIONS

We created a pediatric surgery SBT curriculum for general surgery residents, which can be used to supplement learning of various high-acuity, low-occurrence procedures. Assessment of residents and validation of scores is underway.

Simulation-based education for paediatric surgeons: Does it really improve technical skills?

In recent years, there has been an increasing interest in employing simulation methodology for teaching surgical skills. Supposedly, skills achieved in the simulation arena, should lead to an improvement of surgical performance in the operating room. In this article we aim to summarise those areas of simulation-based research that relate to paediatric surgery; we present the current evidence behind low-fidelity simulation and computer-based simulation in improving the acquisition of surgical skills.