Validation and implementation of surgical simulators: a critical review of present, past, and future

Optimizing modern surgical simulation through instructor feedback - insights from a retrospective observational study in a tertiary hospital

BACKGROUND

Laparoscopic surgery is associated with a prolonged learning curve for emerging surgeons, and simulation-based training (SBT) has become increasingly prominent in this context due to stringent working time regulations and heightened concerns regarding patient safety. While SBT offers a safe and ethical learning environment, the accuracy of simulators in the context of evaluating surgical skills remains uncertain. This study aims to assess the precision of a laparoscopic simulator with regard to evaluating surgical performance and to identify the instructor's role in SBT.

MATERIALS AND METHODS

This retrospective study focused on surgical residents in their 1st through 5th years at the Department of Surgery of Linkou Chang Gung Memorial Hospital. The residents participated in a specially designed SBT program using the LapSim laparoscopic simulator. Following the training session, each resident was required to perform a laparoscopic procedure and received individualized feedback from an instructor. Both simulator and instructor evaluated trainees' performance on the LapSim, focusing on identifying correlations between the simulator's metrics and traditional assessments.

RESULTS

Senior residents (n = 15), who employed more complex laparoscopic procedures, exhibited more significant improvements after receiving instructor feedback than did junior residents (n = 17). Notably, a stronger correlation between the simulator and instructor assessments was observed in the junior group (junior Global Operative Assessment of Laparoscopic Skills (GOALS) adjusted R2 = 0.285, p = 0.016), while no such correlations were observed among the senior group.

CONCLUSION

A well-designed, step-by-step SBT can be a valuable tool in laparoscopic surgical training. LapSim simulator has demonstrated its potential in assessing surgical performances during the early stages of surgical training. However, instructors must provide intuitive feedback to ensure appropriate learning in later stages.

Assessment of a Novel, Adjustable Task Trainer for Cardiac Surgical Skills

INTRODUCTION

A recent needs assessment in Canadian cardiac surgery programs identified the desire for a coronary artery bypass (CABG) and aortic valve replacement (AVR) simulation model for home practice. We aimed to develop and assess a portable, adjustable task trainer for cardiac surgical skills with high functional task alignment.

METHODS

Intraoperative measurements were taken from patients undergoing elective CABG and AVR (N = 30). Measurements were taken in 3 axes and used to create a chest cavity that resembles the mediastinal constraints of a patient undergoing CABG and AVR. The task trainer is adjustable on the following 3 levels: (1) size of the incision, (2) depth of the chest, and (3) relative position of coronary artery or aortic valve model within the chest. Three groups (novices, intermediates, and experts) of cardiac surgery members evaluated the task trainer for functional task alignment and construct validity.

RESULTS

The CABG and AVR model had high functional task alignment. There was a high satisfaction for both models and all participants would recommend the AVR and CABG model as an educational tool. Performance time significantly differed between the groups for both models (CABG: P = 0.032 and AVR: P = 0.001), as well as number of errors (CABG: P = 0.04 and AVR: P = 0.043).

CONCLUSIONS

Using real patient data, we were able to develop an adjustable task trainer for training principles of CABG and AVR. Our pilot study provides preliminary sources of evidence for validity and future study will look to assess transferability of skill to the operating room.

A Cost-Effective, Three-Dimensionally Printed Simulation Model Facilitates Learning of Bilobed and Banner Flaps for Mohs Nasal Reconstruction: A Pilot Study

SUMMARY

Flap design for Mohs reconstruction is a complex 3-dimensional process. Simulation offers trainees the chance to practice techniques safely before performing them in the operating room. To aid in teaching, the authors developed a high-fidelity, cost-effective model of the face using three-dimensional printing to simulate flap reconstruction after Mohs surgery. A model face was sculpted digitally to include skin, bone, and cartilage. Negative molds were printed and used to cast silicone, representing external skin. The cartilage and bone models were combined to create a single three-dimensionally printed base. Surgical residents practiced performing banner and bilobed flaps on the simulation model, and improvement was assessed using boards-style pretests and posttests assessing flap design on clinical photographs. Medical students were randomized to complete a similar practice session with the model or a reading on the topic, after which they completed the same assessment. Participants also completed a questionnaire about the model's didactic and monetary value. Residents showed significant improvement after use of the model (banner flap: P = 0.002, bilobed flap: P = 0.04). Medical students who used the model scored significantly higher than those assigned to train by reading ( P = 0.001). Subjective comfort with flap design and execution increased after practice with the model ( P = 0.001). The cost of materials for each model was $2.50; participants reported willingness to pay $24.36 (mean) for this tool. This accessible model was superior to traditional teaching materials for Mohs reconstruction, and aided the comfort and proficiency of trainees.

Orthopaedic Hand Surgical Simulation Training: A Review

In recent years, new orthopaedic surgical simulation and virtual reality (VR) training models have emerged to provide unlimited education medium to an unlimited number of trainees with no time limit, especially in response to trainee work-hour restrictions. Surgical simulators range from simple wooden boxes to animal and cadaver models to three-dimensional-printed and VR simulators. The coronavirus disease 2019 pandemic further highlighted the need for at-home learning tools for orthopaedic surgical trainees. Advancement in simulating shoulder and knee arthroscopies using VR simulators surpasses the other fields in orthopaedic surgery. Despite the high degree of precision needed to operate at a microscopic level involving vessels, nerves, and the small bones of the hand, the simulation tools have limited advancement in the field of orthopaedic hand surgery. This narrative review summarizes the status of surgical simulation and training techniques available to orthopaedic hand surgical trainees, factors affecting their application, and areas in hand surgery that still lag behind their surgical subspecialty counterparts.

Creation and validation of a novel low-cost dry lab for early resident training and assessment of robotic prostatectomy technical proficiency

Purpose

To evaluate the preliminary validity and acceptability of a low-cost low-fidelity robotic surgery dry lab for training and assessing residents' technical proficiency with key robotic radical prostatectomy steps.

Materials and methods

Three standardized inanimate tasks were created to simulate the radical prostatectomy steps of posterior dissection, neurovascular bundle release, and urethrovesical anastomosis. Urology trainees and faculty at a single institution completed and evaluated each dry lab task. Construct validity was evaluated by comparing task completion times and Global Evaluative Assessment of Robotic Skills scores across four participant cohorts: medical students (n = 5), junior residents (n = 5), senior residents (n = 5), and attending surgeons (n = 7). Content validity, face validity, and acceptability were evaluated through a posttask survey using a 5-point Likert scale.

Results

There was a significant difference in the individual and composite task completion times and Global Evaluative Assessment of Robotic Skills scores across all participant cohorts (all p < 0.01). The model was rated favorably in terms of its content validity and acceptability for use in residency training. However, model realism, compared with human tissue, was poorly rated. The dry lab production cost was less than US $25.

Conclusions

This low-cost procedure-specific dry lab demonstrated evidence of content validity, construct validity, and acceptability for simulating key robotic prostatectomy technical steps and can be used to augment robot-assisted laparoscopic prostatectomy surgical training.

Design and Validation of a 3D Printed Cranio-Facial Simulator: A Novel Tool for Surgical Education

OBJECTIVE

To assess the ability of current 3D printing technology to generate a craniofacial bony and soft tissue anatomical model for use in simulating the performance of a fronto-orbital advancement (FOA) osteotomy and then to further assess the value of the model as an educational tool.

DESIGN

Anatomic models were designed with a process of serial anatomic segmentation/design, 3D printing, dissection, and device refinement. A validation study was conducted with 5 junior and 5 senior plastic surgery residents. The validation study incorporated a multiple-choice Knowledge Assessment test (KA), an Objective Structured Assessment of Technical skills (OSATs), a Global Rating Scale (GRS) and a Michigan Standard Simulation Experience Scale (MiSSES). We compared the scores of both the junior and senior residents and compared junior resident scores, before and after viewing a lecture/demonstration.

RESULTS

MiSSES showed high face validity with a score of 85.1/90, signifying high satisfaction with the simulator learning experience. Simulation and the lecture/demonstration improved the junior resident average KA score from 5.6/10 to 9.6/10 (P = .02), OSATs score from 32.4/66 to 64.4/66 (P < .001) and GRS score from 13.9/35 to 27.5/35 (P < .001). The senior residents OSATs score of 56.3/66 was higher than the pre-lecture juniors (32.4/66) (P < .001), but lower than the post-lecture juniors (64.4/66) (P < .001).

CONCLUSION

We have successfully fabricated a 3D printed craniofacial simulator capable of being used as an educational tool alongside traditional surgical training. Next steps would be improving soft tissue realism, inclusion of patient and disease specific anatomy and creation of models for other surgical specialties.

Haptic based fundamentals of laparoscopic surgery simulation for training with objective assessments

Force is crucial for learning psychomotor skills in laparoscopic tissue manipulation. Fundamental laparoscopic surgery (FLS), on the other hand, only measures time and position accuracy. FLS is a commonly used training program for basic laparoscopic training through part tasks. The FLS is employed in most of the laparoscopic training systems, including box trainers and virtual reality (VR) simulators. However, many laparoscopic VR simulators lack force feedback and measure tissue damage solely through visual feedback based on virtual collisions. Few VR simulators that provide force feedback have subjective force metrics. To provide an objective force assessment for haptic skills training in the VR simulators, we extend the FLS part tasks to haptic-based FLS (HFLS), focusing on controlled force exertion. We interface the simulated HFLS part tasks with a customized bi-manual haptic simulator that offers five degrees of freedom (DOF) for force feedback. The proposed tasks are evaluated through face and content validity among laparoscopic surgeons of varying experience levels. The results show that trainees perform better in HFLS tasks. The average Likert score observed for face and content validity is greater than 4.6 ± 0.3 and 4 ± 0.5 for all the part tasks, which indicates the acceptance of the simulator among subjects for its appearance and functionality. Face and content validations show the need to improve haptic realism, which is also observed in existing simulators. To enhance the accuracy of force rendering, we incorporated a laparoscopic tool force model into the simulation. We study the effectiveness of the model through a psychophysical study that measures just noticeable difference (JND) for the laparoscopic gripping task. The study reveals an insignificant decrease in gripping-force JND. A simple linear model could be sufficient for gripper force feedback, and a non-linear LapTool force model does not affect the force perception for the force range of 0.5-2.5 N. Further study is required to understand the usability of the force model in laparoscopic training at a higher force range. Additionally, the construct validity of HFLS will confirm the applicability of the developed simulator to train surgeons with different levels of experience.

Proficiency Levels and Validity Evidence for Scoring Metrics for a Virtual Reality and Inanimate Robotic Surgery Simulation Curriculum

OBJECTIVE

Our institution recently implemented a virtual reality (VR) skills curriculum for general surgery residents using the SimNow simulator. Based on a content alignment study, we revised the curriculum to include only 20 of 33 VR tasks and we added 3 previously validated inanimate tasks. The purpose of this study was to establish expert-derived proficiency levels for all tasks and to evaluate the validity of the scoring for the VR tasks.

DESIGN

Two expert robotic surgeons performed 5 repetitions of each VR and inanimate task. The trimmed mean (lowest scoring attempt and outliers [>2 standard deviations] were eliminated) was defined as the expert level for each task. For the VR tasks, expert levels were compared to resident performance to evaluate validity.

SETTING

This study was conducted at the University of Texas Southwestern Medical Center (Dallas, TX), a tertiary care academic teaching hospital.

PARTICIPANTS

Two expert robotic surgeons participated in this study. The data from 42 residents (PGY2-4) who completed the original curriculum was used to represent novice performance.

RESULTS

Comparison of expert levels and resident performance was statistically significant for 15 VR tasks (supporting validity) and approached significance (p = 0.06, 0.09) for 2 VR tasks; expert levels were designated as proficiency levels for these 17 tasks. Group comparisons were clearly not significant (p = 0.2-0.8) for 3 VR tasks; 2 of these 3 tasks were retained as introductory exercises (with 3 repetitions required) and 1 was excluded. For the 3 inanimate tasks, expert levels minus 2 standard deviations were designated as proficiency levels.

CONCLUSIONS

This analysis generated validity evidence for 15 VR tasks and established expert-derived proficiency levels for 17 VR tasks and 3 inanimate tasks. Our proposed curriculum now consists of 19 VR and 3 inanimate tasks using the selected proficiency levels. We anticipate that this design will maximize curriculum efficiency and effectiveness.

Continuous Instrument Tracking in a Cerebral Corticectomy Ex Vivo Calf Brain Simulation Model: Face and Content Validation

BACKGROUND AND OBJECTIVES

Subpial corticectomy involving complete lesion resection while preserving pial membranes and avoiding injury to adjacent normal tissues is an essential bimanual task necessary for neurosurgical trainees to master. We sought to develop an ex vivo calf brain corticectomy simulation model with continuous assessment of surgical instrument movement during the simulation. A case series study of skilled participants was performed to assess face and content validity to gain insights into the utility of this training platform, along with determining if skilled and less skilled participants had statistical differences in validity assessment.

METHODS

An ex vivo calf brain simulation model was developed in which trainees performed a subpial corticectomy of three defined areas. A case series study assessed face and content validity of the model using 7-point Likert scale questionnaires.

RESULTS

Twelve skilled and 11 less skilled participants were included in this investigation. Overall median scores of 6.0 (range 4.0-6.0) for face validity and 6.0 (range 3.5-7.0) for content validity were determined on the 7-point Likert scale, with no statistical differences between skilled and less skilled groups identified.

CONCLUSION

A novel ex vivo calf brain simulator was developed to replicate the subpial resection procedure and demonstrated face and content validity.

Design, Fabrication, and Preliminary Validation of Patient-Specific Spine Section Phantoms for Use in Training Spine Surgeons Outside the Operating Room/Theatre

Pedicle screw fixation (PSF) demands rigorous training to mitigate the risk of severe neurovascular complications arising from screw misplacement. This paper introduces a patient-specific phantom designed for PSF training, extending a portion of the learning process beyond the confines of the surgical room. Six phantoms of the thoracolumbar region were fabricated from radiological datasets, combining 3D printing and casting techniques. The phantoms were employed in three training sessions by a fifth-year resident who performed full training on all six phantoms; he/she placed a total of 57 pedicle screws. Analysis of the learning curve, focusing on time per screw and positioning accuracy, revealed attainment of an asymptotic performance level (around 3 min per screw) after 40 screws. The phantom's efficacy was evaluated by three experts and six residents, each inserting a minimum of four screws. Initial assessments confirmed face, content, and construct validity, affirming the patient-specific phantoms as a valuable training resource. These proposed phantoms exhibit great promise as an essential tool in surgical training as they exhibited a demonstrable learning effect on the PSF technique. This study lays the foundation for further exploration and underscores the potential impact of these patient-specific phantoms on the future of spinal surgical education.

Inanimate 3D printed model for thoracoscopic repair of esophageal atresia with tracheoesophageal fistula

Background

Thoracoscopic repair of esophageal atresia (EA) and tracheoesophageal fistula (TEF) poses significant technical challenges. This study aimed to develop an inexpensive, reusable, high-fidelity synthetic tissue model for simulating EA/TEF repairs and to assess the validity of the simulator.

Methods

By using 3D printing and silicone casting, we designed an inexpensive and reusable inanimate model for training in thoracoscopic EA/TEF repair. The objective was to validate the model using a 5-point Likert scale and the Objective Structured Assessment of Technical Skills (OSATS) to evaluate participants' surgical proficiency.

Results

A total of 18 participants (7 medical students, 4 pediatric surgery trainees, and 7 experienced surgeons), after being instructed and trained, were asked to perform TEF ligation, dissection, as well as esophageal anastomosis using six sliding knots on the EA/TEF simulator. All participants in the expert group completed the task within the 120-minute time limit, however only 4 (57%) participants from the novice/intermediate completed the task within the time limit. There was a statistically significant difference in OSATS scores for the "flow of task" (p = 0.018) and scores for the "overall MIS skills" (p = 0.010) task distinguishing between novice and intermediates and experts. The simulator demonstrated strong suitability as a training tool, indicated by a mean score of 4.66. The mean scores for the model's realism and the working environment were 4.25 and 4.5, respectively. Overall, the face validity was scored significantly lower in the expert group compared to the novice/intermediate groups (p = 0.0002).

Conclusions

Our study established good face and content validity of the simulator. Due to its reusability, and suitability for individual participants, our model holds promise as a training tool for thoracoscopic procedures among surgeons. However, novices and trainees struggled with advanced minimally invasive surgical procedures. Therefore, a structured and focused training curriculum in pediatric MIS is needed for optimal utilization of the available training hours.

Practicing Emergency Medicine in the Metaverse: A Novel Mixed Reality Casualty Care Training Platform

BACKGROUND

Current casualty care training modalities present several challenges, including limited simulation facilities, instructor dependence, lack of standardization, documentation of trainees' performance and training personalization. The study presents the design, development and preliminary evaluation of a novel hybrid training platform to address these challenges.

METHODS

A mixed reality platform was chosen and developed to address field operators' requirements. The platform is easy to operate and can be set up by laypeople within 20-min in multiple environments. Individual-level training documentation is generated autonomously following each session, evaluating 30 aspects of performance. From this, a unique aggregated dataset emerges as a substrate for executives' dashboards and intelligent planning of future sessions.

RESULTS

An evaluation process took part using simulator-based training in different stages along the project using a questionnaire (Likert-scale based). Fifty military physicians took part in an identical head injury scenario requiring airway management by endotracheal intubation and were immediately surveyed.

CONCLUSION

TrauMR is an agile hybrid training that harbors the potential to address many of the emerging challenges of training for prehospital care in combat and civilian environments.

Exploring the Role of Simulation Training in Improving Surgical Skills Among Residents: A Narrative Review

The role of simulation in medical education is crucial to the development of surgeons' skills. Surgical simulation can be used to improve surgical skills in a secure and risk-free environment. Animal models, simulated patients, virtual reality, and mannequins are some types of surgical simulation. As a result, feedback encourages students to reflect on their strengths and weaknesses, enabling them to focus on improvement. Healthcare simulation is a strong educational instrument, and the main goal of this is to give the students an opportunity to do a practical application of what they have learned through theory. Before taking it to the patients, they will already have certain tools they have previously acquired during the practice. This makes it easier for students to identify the knowledge gaps that they must fill to improve patient outcomes. Moreover, simulation brings a wonderful opportunity for students to acquire skills, gain confidence, and experience success before working with real patients, especially when their clinical exposure is limited. The use of simulation to teach technical skills to surgical trainees has become more prevalent. The cost of setting up a simulation lab ranges from $100,000 to $300,000. There are several ways to evaluate the effectiveness of simulation-based surgical training. Repetitive surgical simulation training can improve speed and fluidity in general surgical skills in comparison to conventional training. Few previous studies compared learners who received structured simulation training to a group of trainees who did not receive any simulation training in single-center randomized control research. Significantly faster and less time-consuming skill proficiency was noticeable in simulated trainees. Despite being anxious in the operating room for the first time, simulated trainees completed the surgery on time, demonstrating the effectiveness of surgical simulation training. Traditional surgical training involves senior-surgeon supervision in the operating room. In simulation-based training, the trainees have full control over clinical scenarios and settings; however, guidance and assessment are also crucial. Simulators allow users to practice tasks under conditions resembling real-life scenarios. Simulators can be compared with traditional surgical training methods for different reasons. For example, intraoperative bleeding may occasionally show up not only visibly on the screen but also by shaking the trocars erratically. Without haptics, training on virtual simulators can cause one's pulling and pushing forces, which are frequently greater than what the tissue needs, to be distorted. A good method of simulation training is using virtual reality simulators with haptics and simulated patients. The availability of these facilities is limited, though, and a typical session might include an exercise involving stacking sugar cubes and box trainers. The degree of expertise or competency is one area that needs clarification as medical education transitions to a competency-based paradigm. The article aims to provide an overview of simulation, methods of simulation training, and the key role and importance of surgical simulation in improving skills in surgical residents.

Comparing the Efficacy of a New Clinical Skills Model with a Traditional Method to Teach Tube Feeding of an Avian Patient

Interactive clinical skills models have been demonstrated to be useful for teaching medical and veterinary clinical skills, yet to date, very few exist for teaching skills relevant to zoological companion animals and wildlife species including birds. This two-part study aimed to create, develop, and validate a model. Interviews and a survey were conducted using veterinary and wildlife professionals to select an avian clinical skill that is challenging and performed frequently. Tube/gavage feeding, or "crop tubing" satisfied both criteria; on average it was performed 71 times a year by surveyed respondents was rated 3.4/9 for difficulty of teaching and 3.5/9 for difficulty of learning. Therefore, a new model of a bird, made from a soft toy, silicone, and 3D printed parts, was designed to train students to perform this technique. Forty-two participants were recruited and divided into two groups; one used the model the other watched an instructional video on crop tubing. The students completed a self-evaluated confidence questionnaire, before and after, using either resource. They then performed the technique on a dead bird and their proficiency at 10 different actions that comprised the technique was evaluated by two assessors. The model group performed significantly better than the video group on all evaluated actions (U ≤ 143.5, p ≤ .0031), and reported significantly higher confidence (U = 129.5, p = 0.018). In conclusion, the newly developed model in combination with an instruction booklet offers an effective and inexpensive alternative way to teach crop tubing in a teaching environment, without compromising animal welfare.

The development and validation of a new simulator for endourology

INTRODUCTION

Simulation in medicine has developed a lot in the last few decades. There is a broad range of simulators available, above all for training in surgical procedures. Endourology can benefit much from simulation because the minimally-invasive procedures of endourology frequently have long learning curves, which can be reduced by training with simulators.

MATERIALS AND METHODS

A low-fidelity simulator was designed for practicing endourology techniques that use cystoscopy. The process of validation involved 5 experts and 19 non-experts. Experts comprised medical professionals working in a department of urology who had performed at least 100 flexible cystoscopy procedures. Non-experts were residents in internal medicine without experience in any type of endoscopy. Information about face and content validity was collected by means of Likert scales from 1 to 5. To evaluate construct validity, we measured the time to complete two tasks, for which the procedure was evaluated by means of the OSATS global evaluation scale.

RESULTS

New simulator was successfully built according to its design. For all evaluated aspects of construct validity, there was a significant difference (p<0.05) between the group of experts and the group of non-experts. Content validity was scored 4.66 (standard deviation ±0.56) by the experts and 4.41 (±0.71) by the non-experts. In the face validity questionnaire, the average score was 4.14 (±0.94), the question receiving the highest score: 4.6 (±0.84) concerned immersion in the procedure.

CONCLUSION

The simulator presented is valid both for training up new urologists in endourology technique and for experts seeking to perfect their skills.

Proficiency and Retention of Five Clinical Veterinary Skills Using Multipurpose Reusable Canine Manikins vs. Live Animals: Model Development and Validation

Simulation in veterinary education provides a safe and ethical alternative to using live animals, but most simulators are single purpose and unvalidated. In this study, canine training manikins were created using readily available materials to teach fine needle aspiration (FNA) of peripheral lymph nodes, jugular venipuncture, cephalic venipuncture, intravenous catheterization, and cystocentesis. Undergraduate subjects were prospectively enrolled and stratified by veterinary experience prior to randomization into two groups. Students were taught a new skill each week through a written description of the technique, video training, and hands-on practice (live animal vs. manikin). The following week, participants were scored on the performance of the previous week's skill on a live animal using a standardized rubric by reviewers blinded to the training group. Six weeks later, the assessment was repeated for all skills. Scores were compared between groups and time points using repeated-measures ANOVA after logarithmic transformation. p < .05 was significant. There were no significant differences in scores for any of the skills between the groups immediately following or 6 weeks after training. Initial proficiency and short-term retention of clinical skills do not differ for students trained using a manikin vs. a live dog.

Nondominant Hand Skills Spatial and Psychomotor Analysis During a Complex Virtual Reality Neurosurgical Task-A Case Series Study

BACKGROUND

Virtual reality surgical simulators provide detailed psychomotor performance data, allowing qualitative and quantitative assessment of hand function. The nondominant hand plays an essential role in neurosurgery in exposing the operative area, assisting the dominant hand to optimize task execution, and hemostasis. Outlining expert-level nondominant hand skills may be critical to understand surgical expertise and aid learner training.

OBJECTIVE

To (1) provide validity for the simulated bimanual subpial tumor resection task and (2) to use this simulation in qualitative and quantitative evaluation of nondominant hand skills for bipolar forceps utilization.

METHODS

In this case series study, 45 right-handed participants performed a simulated subpial tumor resection using simulated bipolar forceps in the nondominant hand for assisting the surgery and hemostasis. A 10-item questionnaire was used to assess task validity. The nondominant hand skills across 4 expertise levels (neurosurgeons, senior trainees, junior trainees, and medical students) were analyzed by 2 visual models and performance metrics.

RESULTS

Neurosurgeon median (range) overall satisfaction with the simulated scenario was 4.0/5.0 (2.0-5.0). The visual models demonstrated a decrease in high force application areas on pial surface with increased expertise level. Bipolar-pia mater interactions were more focused around the tumoral region for neurosurgeons and senior trainees. These groups spent more time using the bipolar while interacting with pia. All groups spent significantly higher time in the left upper pial quadrant than other quadrants.

CONCLUSION

This work introduces new approaches for the evaluation of nondominant hand skills which may help surgical trainees by providing both qualitative and quantitative feedback.

Video Context Improves Performance in Identifying Operative Planes on Static Surgical Images

BACKGROUND

Correct identification of the surgical tissue planes of dissection is paramount at the operating room, and the needed skills seem to be improved with realistic dynamic models rather than mere still images. The objective is to assess the role of adding video prequels to still images taken from operations on the precision and accuracy of tissue plane identification using a validated simulation model, considering various levels of surgeons' experience.

METHODS

A prospective observational study was conducted involving 15 surgeons distributed to three equal groups, including a consultant group [C], a senior group [S], and a junior group [J]. Subjects were asked to identify and draw ideal tissue planes in 20 images selected at suitable operative moments of identification before and after showing a 10- second videoclip preceding the still image. A validated comparative metric (using a modified Hausdorff distance [%Hdu] for object matching) was used to measure the distance between lines. A precision analysis was carried out based on the difference in %Hdu between lines drawn before and after watching the videos, and between-group comparisons were analyzed using a one-way analysis of variance (ANOVA). The analysis of accuracy was done on the difference in %Hdu between lines drawn by the subjects and the ideal lines provided by an expert panel. The impact of videos on accuracy was assessed using a repeated-measures ANOVA.

RESULTS

The C group showed the highest preciseness as compared to the S and J groups (mean Hdu 9.17±11.86 versus 12.1±15.5 and 20.0±18.32, respectively, p <0.001) and significant differences between groups were found in 14 images (70%). Considering the expert panel as a reference, the interaction between time and experience level was significant ( F (2, 597) = 4.52, p <0.001). Although the subjects of the J group were significantly less accurate than other surgeons, only this group showed significant improvements in mean %Hdu values after watching the lead-in videos ( F (1, 597) = 6.04, p = 0.014).

CONCLUSIONS

Adding video context improved the ability of junior trainees to identify tissue planes of dissection. A realistic model is recommended considering experience-based differences in precision in training programs.

Development and initial validation of a cost-effective, re-usable, ultrasound-compatible suprapubic catheter insertion training simulator

INTRODUCTION

Suprapubic catheterization (SPC) is a fundamental skill required of urology trainees. A lack of affordable simulation models and unpredictability of bedside SPCs limit experiential learning opportunities. Our objective was to develop and initially validate a re-usable, low-cost, ultrasound (US)-compatible SPC simulator for acquiring skills that transfer to the bedside.

METHODS

The model was constructed using six components. Staff urologists and interventional radiologists (IRs) conducted a SPC and rated the model on three domains with multiple subcategories on a five-point Likert scale: anatomic realism; usefulness as a training tool; and global/overall reaction. Participants in our first-year urology "boot camp" received SPC training, practiced, and were evaluated via an objective structured clinical examination (OSCE). Staff ratings and OSCE scores determined the model's initial face and content validity.

RESULTS

Twelve staff physicians participated in the study. The mean scores for urologists and IRs, respectively, were: anatomical realism: 4.10 and 3.70; usefulness as a training tool: 4.23 and 4.24; and overall reaction: 4.40 and 4.44. Staff strongly agreed that the model should be incorporated into the residency curriculum. Over the past four years, 25 boot camp participants scored a mean of 99.7% (±1.8) on the OSCE, with high technical performance and entrustment scores (4.8 and 4.7, respectively). The model cost $55 CAD.

CONCLUSIONS

This novel, multiple-use, low-cost, easily reproducible US-compatible SPC simulator demonstrated initial face and content validity via high staff urologist and IR ratings and OSCE scores of first-year urology residents. Additional research is required for construct validation.

Development and validation of a novel 3D-printed simulation model for open oesophageal atresia and tracheo-oesophageal fistula repair

BACKGROUND

The role of simulation training in paediatric surgery is expanding as more simulation devices are designed and validated. We aimed to conduct a training needs assessment of UK paediatric surgical trainees to prioritise procedures for simulation, and to validate a novel 3D-printed simulation model for oesophageal atresia and tracheo-oesophageal fistula (OA-TOF) repair.

METHODS

A questionnaire was sent to UK trainee paediatric surgeons surveying the availability and utility of simulation. The operation ranked as most useful to simulate was OA-TOF repair. 3D-printing techniques were used to build an OA-TOF model. Content, face and construct validity was assessed by 40 paediatric surgeons of varying experience.

RESULTS

Thirty-four paediatric surgeons completed the survey; 79% had access to surgical simulation at least monthly, and 47% had access to paediatric-specific resources. Perceived utility of simulation was 4.1/5. Validation of open OA-TOF repair was conducted by 40 surgeons. Participants rated the model as useful 4.9/5. Anatomical realism was scored 4.2/5 and surgical realism 3.9/5. The model was able to discriminate between experienced and inexperienced surgeons.

CONCLUSION

UK paediatric surgeons voted OA-TOF repair as the most useful procedure to simulate. In response we have developed and validated an affordable 3D-printed simulation model for open OA-TOF repair.

Design, Development, and Validation of a High-Fidelity "Ganglion Cyst" Model for Cadaveric Hand Surgery Training

Introduction  This study presents a design and developmental model with prospective validation. This study was aimed to design, develop, and validate a cadaveric model simulator of a ganglion cyst to train surgeons in its surgical excision. Materials and Methods  A novel "ganglion cyst" was designed using a latex powder-free glove and water-based, water-soluble lubricant (K-Y jelly). This "ganglion cyst" was then inserted subcutaneously into a cadaveric hand, positioned over the dorsum of the wrist to simulate a ganglion lesion. This simulation model was prospectively validated using postsimulation surveys conducted on novices and experts in hand surgery. The simulation was conducted during a basic hand surgery cadaveric workshop that included a practical station on ganglion excision. Results  Both the novices and experts had a concurrent agreement that the simulator provided relevance, realism, and value as a simulator for the teaching, excision and procedural assessment of ganglions in the hand. Conclusion  The role of high-fidelity simulation has been documented in the literature for surgical procedures. This simple and affordable model that we have developed and validated allows for the creation of a high-fidelity ganglion simulator in the cadaveric hand for teaching, excision, and procedural assessment among trainees.

Efficacy of an Online Blended Learning Curriculum to Improve Medical Student Urologic Education

OBJECTIVE

During the COVID-19 pandemic, limitations on in-person medical school clerkships created a deficit in urologic learning opportunities. We sought to develop and evaluate a blended curriculum of interactive online modules with small-group discussion to enhance the educational experience for medical students in urology.

MATERIALS AND METHODS

We created a curriculum of four online case-based urology modules. Between July and October 2020, 14 fourth-year medical students completed the modules. Students answered questions on a discussion board and engaged in asynchronous dialogue with 16 physicians, in addition to a weekly live review session. Students and physicians completed anonymous surveys to assess satisfaction and perceived learning outcomes, with questions scored on a 5-point Likert scale.

RESULTS

Thirteen students (93% response rate) and 12 physicians (75% response rate) completed the survey. Overall, 12/13 students and 11/12 physicians "strongly agree" or "somewhat agree" that the modules improved the rotation. Students and physicians perceived that the modules were effective for learning/teaching foundational knowledge (average ratings 4.8 and 4.5, respectively) and facilitating performance assessment (4.4 and 4.0). Students reported high learning scores across multiple Accreditation Council for Graduate Medical Education core competencies, and 12/13 students found the modules fun/engaging. The majority of students (12/13) and physicians (10/12) felt that the online modules should be incorporated into future urology electives.

CONCLUSIONS

A blended learning curriculum utilizing online modules is an effective tool for enhancing urologic education, improving perceived learning outcomes and facilitating performance assessment.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s40670-021-01427-3.

Design and Evaluation of Personalized Percutaneous Coronary Intervention Surgery Simulation System

In recent years, medical simulators have been widely applied to a broad range of surgery training tasks. However, most of the existing surgery simulators can only provide limited immersive environments with a few pre-processed organ models, while ignoring the instant modeling of various personalized clinical cases, which brings substantive differences between training experiences and real surgery situations. To this end, we present a virtual reality (VR) based surgery simulation system for personalized percutaneous coronary intervention (PCI). The simulation system can directly take patient-specific clinical data as input and generate virtual 3D intervention scenarios. Specially, we introduce a fiber-based patient-specific cardiac dynamic model to simulate the nonlinear deformation among the multiple layers of the cardiac structure, which can well respect and correlate the atriums, ventricles and vessels, and thus gives rise to more effective visualization and interaction. Meanwhile, we design a tracking and haptic feedback hardware, which can enable users to manipulate physical intervention instruments and interact with virtual scenarios. We conduct quantitative analysis on deformation precision and modeling efficiency, and evaluate the simulation system based on the user studies from 16 cardiologists and 20 intervention trainees, comparing it to traditional desktop intervention simulators. The results confirm that our simulation system can provide a better user experience, and is a suitable platform for PCI surgery training and rehearsal.

CathSym: Device and Method to Bring Haptic Feedback to Urinary Catheterization Training

Urinary catheterization is one of the most widely taught procedures in the medical field. Current simulation-based training methods allow the students to be trained on non-realistic mannequins that do not adequately develop their psychomotor skills. This lack of proper training translates into increased likelihood of the medical professional causing damage to the patients' urethra in the form of false passages when faced with a difficult catheterization. The leading cause of this damage is the overuse of force that diverts the catheter head into the soft tissue. With the emergence of haptic feedback and virtual training into the medical field, we aimed to design and build a novel haptics-based mixed reality simulation trainer for teaching urinary catheterization. We developed a software system accompanied with a customized haptic feedback device to help the user train in various catheterization scenarios to gain experience for improving their psychomotor skills and teach them to navigate blockages and other anatomies in the urethra. Our simulation platform has the potential to adequately provde the trainees with realistic force and visual feedback that are representative of what the user might experience in real world Foley catheterization.

A new model of inexpensive portable homemade PERC Mentor (IPHOM) and its validation

Background

Percutaneous nephrolithotomy (PCNL) is a complex surgery and has a flat learning curve. Due to this and the ethical issues, trainees do not get enough hands on exposure. Virtual simulator is very expensive and bulky. Animal model requires legal clearance. This inexpensive portable homemade PERC Mentor (IPHOM) teaches all the major aspects of PCNL surgery. This article has shown the way to make this model and its validation study.

Methods

IPHOM can be made at home with carton box, ball bearings, LED torch and some hospital wastes. After a short demonstration of IPHOM, 14 residents and 4 urologists were given 8 tasks to perform on it followed by 15-min supervised practice exercise on day 0 and day 1. Their performance was reassessed on day 2 and 3. Response to 17 feedback points was recorded on a seven-point Likert scale.

Results

There was significant difference between the performance of expert and novice on day 0. Expert completed all the tasks in less time and no. of attempts. The time for tract dilatation and duration of radiation exposure were significantly less in the expert group. The performance of both expert and novice improved on day 2 and 3, but the improvement was significantly more in novice. Response to the feedback points showed no difference between expert and novice (p > .05).

Conclusions

We have found that training on IPHOM has improved the concept and skills of PCNL in residents. The simplicity and low cost of the model make it constructible at home.

Training minimally invasive surgery's basic skills: is expensive always better?

INTRODUCTION

Not all hospitals have a MIS training facility because often training is not a main corporate objective and could require lots of money. We tried to build a laparoscopic simulator that was effective and that would allow to carry out an adequate laparoscopic training similar to that obtained with the models normally used in MIS training programs. To construct a box trainer that would achieve the equivalent results than those usually used. A validation study was carried out by evaluating the content validity and construct validity of our simulator in addition a comparison study of our homemade trainer vs Karl Storz box trainer was performed.

MATERIAL AND METHODS

The HM laparoscopic trainer was assembled using a wood frame. Two LED lights were positioned on the inside roof of the trainer and a webcam was positioned through a special support as operative optic. The webcam was then connected to a PC and the latter was used as a monitor for the operator. Participants were 20 students and a group of 6 surgeons. Students were prospectively randomized to perform 4 of the 5 tasks of the fundamental laparoscopic surgery (FLS) program on both the HM trainer and the KS trainer (pegboard transfer, pattern cut, placement of ligating loop and intracorporeal knot suture). Simple paired t test was performed to compare times between the trainers. Then students performed two more sets of exercises on the HM. The group of surgeons performed three sets of the same exercises performed by the students on the HM. The time taken by surgeons and students to complete the exercises was compared using t test. At the end, all the participants carried out a questionnaire to evaluate their experience with the HM box trainer. For the questionnaire it was chosen to use a Linkert 1-5 scale (1 = strongly disagree; 2 = disagree; 3 = undecided; 4 = agree; 5 = strongly agree).

RESULTS

HM vs KS BT: Comparing time to complete the 4 tasks performed by students on both the BT, for the first task the p value was 0.30, for the second task 0.48, for the third task 0.80, for the fourth task 0.93, and for the total time 0.86. The comparison between the mean time of the first set of tasks of the participants who started with the HM BT and one of the participants who started on the KS p value was 1 p = 0.09; task 2 p = 0.32; task 3 p = 0.62; task 4 p = 0.32; total time p = 0.81. The comparison between the meantime of the second set of tasks of the participants who switched to the HM BT with the one of those who switched to the KS BT showed a p value of: p = 0.20 tasks 1 p = 0.53 task 2; p = 0.39 task 3; p = 0.30 task 4; p = 0.56 total time. Construct validity: The mean experts and students time of every single task and the total one showed a p value of: p < 0.01 for task 1; p < 0.01 task 2; p < 0.01 task 3; p < 0.01 task 4; p < 0.01 total time. Content validity: Both experts and students indicated the HM BT as a useful training tool and appreciated its easy use. Both groups would use it at home if it were available.

CONCLUSION

Valid MIS trainer can be easily built at home with few low-cost materials. Our study shows how training programs can be structured even with few resources in a creative and innovative way.

Development and Validation of a Low-Cost Endoscopic Spine Surgery Simulator

Background

Minimally invasive endoscopic techniques in spine surgery continue to gain in popularity. Unfortunately, there is a long learning period for novice endoscope users to acquire basic skills, and complex training simulators are frequently cost-prohibitive. This paper describes the development and validation of a low-cost endoscopic spine training simulator.

Methodology

A low-cost endoscopic spine training model was created utilizing a budget of less than 65 USD. Afterward, a training curriculum consisting of five tasks was designed to mimic standard techniques frequently utilized in endoscopic spine surgery. This curriculum was tested on a cohort of surgical trainees. The initial time to completion as well as errors made during the tasks and repeat trials were recorded. A composite score was generated to quantify the overall scores which included both time and errors in each task.

Results

In total, 11 students and surgical residents completed the curriculum. The first attempt required an average of 622 seconds for the completion of the curriculum compared to 283 seconds in the second trial (p < 0.001; SD = 36.75). In regards to trials in which errors were counted, fewer errors occurred during the second attempt (2.55 vs. 1.53); however, this difference was not statistically significant (p > 0.05). In regards to the composite score, the composite score of the intern group demonstrated an average improvement of 0.345 compared to an average improvement of 0.47 in the resident group.

Conclusions

Our study demonstrates the feasibility of a low-cost endoscopic spine trainer as well as its efficacy in improving basic endoscopic skills in trainees.

Staff perceptions of the implementation of a trauma video review program at a level I trauma center

OBJECTIVES

Successful implementation of any new technology requires extensive engagement with front-line staff. We explored the perceptions of emergency department and trauma staff about a trauma video review program (TVR) prior to implementation of the first such program in Canada at our level I trauma center.

METHODS

We conducted semi-structured individual interviews and in situ small group interviews with 35 multidisciplinary ED and trauma staff members of a teaching and research hospital in Toronto, Canada. We sought maximum variation in the sample of purposively selected participants. Interviews were recorded with audiotapes or detailed field notes, transcribed verbatim, coded, and analyzed using standard thematic analysis techniques.

RESULTS

Participants expressed overall support for the concept of TVR, but there is a core sense of unease that influenced overall staff perceptions. Despite several departmental presentations, very few participants actually had a solid understanding of how the TVR worked. Many were apprehensive about their own professional privacy, deeply concerned about vulnerable patients being filmed without consent, and questioned how video data would be used. Despite significant hesitancy, ED and trauma staff identified positive opportunities that TVR could bring, including providing an evidence base for quality improvement.

CONCLUSIONS

TVR is an evolving approach to evaluate quality and patient safety in the trauma bay. As such it brings with it natural concerns and apprehension from staff regarding privacy, confidentiality, and how data will be captured and used. There is opportunity for these types of concerns to be addressed with a robust knowledge translation plan and engagement of staff throughout the implementation process.

Face, Content, and Construct Validity of a Virtual Reality Otoscopy Simulator and Applicability to Medical Training

Objective

Otologic diseases are common in all age groups and can significantly impair the function of this important sensory organ. To make a correct diagnosis, the correct handling of the otoscope and a correctly performed examination are essential. A virtual reality simulator could make it easier to teach this difficult-to-teach skill. The aim of this study was to assess the face, content, and construct validity of the novel virtual reality otoscopy simulator and the applicability to otologic training.

Study Design

Face and content validity was assessed with a questionnaire. Construct validity was assessed in a prospectively designed controlled trial.

Setting

Training for medical students at a tertiary referral center.

Method

The questionnaire used a 6-point Likert scale. The otoscopy was rated with a modified Objective Structured Assessment of Technical Skills. Time to complete the task and the percentage of the assessed eardrum surface were recorded.

Results

The realism of the simulator and the applicability to medical training were assessed across several items. The ratings suggested good face and content validity as well as usefulness and functionality of the simulator. The otolaryngologists significantly outperformed the student group in all categories measured (P < .0001), suggesting construct validity of the simulator.

Conclusion

In this study, we could demonstrate face, content, and construct validity for a novel high-fidelity virtual reality otoscopy simulator. The results encourage the use of the otoscopy simulator as a complementary tool to traditional teaching methods in a curriculum for medical students.

External validation of a prostate biopsy simulator

INTRODUCTION

Simulation-based training has proven to be a promising option allowing for initial and continuous training while limiting the impact of the learning curve on the patient. The Biopsym simulator was developed as a complete teaching environment for the prostate biopsy procedure. This paper presents the results of an external validation of this simulator, involving urology residents recruited during a regional teaching seminar.

METHODS

Residents from 4 academic urology departments of the French Auvergne Rhône-Alpes region, who did not take part in the previous simulator validation studies, were enrolled. After a short presentation and standardized initiation session, residents carried out a simulated systematic 12-core biopsy procedure and were asked to fill in a questionnaire collecting their expectations and evaluation of the Biopsym simulator. The number of biopsies reaching each targeted sector, the total score provided by the simulator and the duration of the procedure were recorded.

RESULTS

Twenty-three residents were recruited. The overall added value (/100) for learning was rated at a median of 100 (interquartile range 83-100), overall realism of the biopsy procedure at 80 (65-89). The median percentage of biopsies reaching the targeted sector was 66.7% (62-75). The median score provided by the simulator was 50% (37-60). For both, the difference between residents with or without prior biopsy experience was not statistically significant. The median duration of the simulated biopsy procedure was 4:58 (minutes: seconds) (3:49-6:00). Resident with prior experience required less time to complete the biopsy procedure 3:53 (3:39-4:56) vs. 5:10 (4:59-7:10), P=0.01.

CONCLUSION

This external validation study confirms a high acceptance of the simulator by the target audience. To our knowledge, the Biopsym simulator is the only prostate biopsy simulator that demonstrated such validity as evaluated by clinicians, outside the center involved in its early development.

LEVEL OF EVIDENCE

3.

Characteristics of laparoscopic and open hernia repair simulation models: a systematic review

Purpose

Barriers to education in open and laparoscopic hernia repair technique include a steep learning curve and reduced theatre time for junior surgical trainees. This is particularly evident during the current COVID-19 pandemic. Simulation models may provide further opportunities for training in hernia repair outside of the traditional surgical apprenticeship model.

Methods

A systematic review was carried out following PRISMA guidelines to identify and evaluate simulation models in hernia repair. Of the 866 records screened, 27 were included in the analysis. These were assessed for face, content and construct validity, as well as their attempt to measure educational impact.

Results

Simulation models were identified comprising of animal tissues, synthetic materials and virtual reality (VR) technology. Models were designed for instruction in repair of inguinal, umbilical, incisional and diaphragmatic hernias. Twenty-one laparoscopic hernia repair models were described. Many models demonstrated validity across several domains, and three showed transferability of skills from simulation to the operating room. Of the six open hernia repair simulation models, none were found to have demonstrated an educational impact in addition to assessing validity.

Conclusion

Few models individually were able to demonstrate validity and educational impact. Several novel assessment tools have been developed for assessment of progress when performing simulated and real laparoscopic inguinal hernia repair. More study is required, particularly for open hernia repair, including randomized controlled trials with large sample sizes to assess the transferability of skills.

A review of participant recruitment transparency for sound validation of hip surgery simulators: a novel umbrella approach

Malposition of implants is associated with complications, higher wear and increased revision rates in total hip replacement (THR) along with surgeon inexperience. Training THR residents to reach expert proficiency is affected by the high cost and resource limitations of traditional training techniques. Research in extended reality (XR) technologies can overcome such barriers. These offer a platform for learning, objective skill-monitoring and, potentially, for automated certification. Prior to their incorporation into curricula however, thorough validation must be undertaken. As validity is heavily dependent on the participants recruited, there is a need to review, scrutinise and define recruitment criteria in the absence of pre-defined standards, for sound simulator validation. A systematic review on PubMed and IEEE databases was conducted. Training simulator validation research in fracture, arthroscopy and arthroplasty relating to the hip was included. 46 validation studies were reviewed. It was observed that there was no uniformity in reporting or recruitment criteria, rendering cross-comparison challenging. This work developed Umbrella categories to help prioritise recruitment, and has formulated a detailed template of fields and guidelines for reporting criteria so that, in future, research may come to a consensus as to recruitment criteria for a hip "expert" or "novice".

A haptic laparoscopic trainer based on affine velocity analysis: engineering and preliminary results

BACKGROUND

There is a general agreement upon the importance of acquiring laparoscopic skills outside the operation room through simulation-based training. However, high-fidelity simulators are cost-prohibitive and elicit a high cognitive load, while low-fidelity simulators lack effective feedback. This paper describes a low-fidelity simulator bridging the existing gaps with affine velocity as a new assessment variable. Primary validation results are also presented.

METHODS

Psycho-motor skills and engineering key features have been considered e.g. haptic feedback and complementary assessment variables. Seventy-seven participants tested the simulator (17 expert surgeons, 12 intermediates, 28 inexperienced interns, and 20 novices). The content validity was tested with a 10-point Likert scale and the discriminative power by comparing the four groups' performance over two sessions.

RESULTS

Participants rated the simulator positively, from 7.25 to 7.72 out of 10 (mean, 7.57). Experts and intermediates performed faster with fewer errors (collisions) than inexperienced interns and novices. The affine velocity brought additional differentiations, especially between interns and novices.

CONCLUSION

This affordable haptic simulator makes it possible to learn and train laparoscopic techniques. Self-assessment of basic skills was easily performed with slight additional cost compared to low-fidelity simulators. It could be a good trade-off among the products currently used for surgeons' training.

A systematic review of low-cost simulators in ENT surgery

BACKGROUND

Simulation training has become a key part of the surgical curriculum over recent years. Current trainees face significantly reduced operating time as a result of the coronavirus disease 2019 pandemic, alongside increased costs to surgical training, thus creating a need for low-cost simulation models.

METHODS

A systematic review of the literature was performed using multiple databases. Each model included was assessed for the ease and expense of its construction, as well as its validity and educational value.

RESULTS

A total of 18 low-cost simulation models were identified, relating to otology, head and neck surgery, laryngeal surgery, rhinology, and tonsil surgery. In only four of these models (22.2 per cent) was an attempt made to demonstrate the educational impact of the model. Validation was rarely formally assessed.

CONCLUSION

More efforts are required to standardise validation methods and demonstrate the educational value of the available low-cost simulation models in otorhinolaryngology.

Validation of a high-fidelity training model for fetoscopic spina bifida surgery

Open fetal surgery for spina bifida (SB) is safe and effective yet invasive. The growing interest in fetoscopic SB repair (fSB-repair) prompts the need for appropriate training. We aimed to develop and validate a high-fidelity training model for fSB-repair. fSB-repair was simulated in the abdominal cavity and on the stomach of adult rabbits. Laparoscopic fetal surgeons served either as novices (n = 2) or experts (n = 3) based on their experience. Technical performance was evaluated using competency Cumulative Sum (CUSUM) analysis and the group splitting method. Main outcome measure for CUSUM competency was a composite binary outcome for surgical success, i.e. watertight repair, operation time ≤ 180 min and Objective-Structured-Assessment-of-Technical-Skills (OSATS) score ≥ 18/25. Construct validity was first confirmed since competency levels of novices and experts during their six first cases using both methods were significantly different. Criterion validity was also established as 33 consecutive procedures were needed for novices to reach competency using learning curve CUSUM, which is a number comparable to that of clinical fSB-repair. Finally, we surveyed expert fetal surgeons worldwide to assess face and content validity. Respondents (26/49; 53%) confirmed it with ≥ 71% of scores for overall realism ≥ 4/7 and usefulness ≥ 3/5. We propose to use our high-fidelity model to determine and shorten the learning curve of laparoscopic fetal surgeons and retain operative skills.

Implementation of the Operating Room Black Box Research Program at the Ottawa Hospital Through Patient, Clinical, and Organizational Engagement: Case Study

Background

A large proportion of surgical patient harm is preventable; yet, our ability to systematically learn from these incidents and improve clinical practice remains limited. The Operating Room Black Box was developed to address the need for comprehensive assessments of clinical performance in the operating room. It captures synchronized audio, video, patient, and environmental clinical data in real time, which are subsequently analyzed by a combination of expert raters and software-based algorithms. Despite its significant potential to facilitate research and practice improvement, there are many potential implementation challenges at the institutional, clinician, and patient level. This paper summarizes our approach to implementation of the Operating Room Black Box at a large academic Canadian center.

Objective

We aimed to contribute to the development of evidence-based best practices for implementing innovative technology in the operating room for direct observation of the clinical performance by using the case of the Operating Room Black Box. Specifically, we outline the systematic approach to the Operating Room Black Box implementation undertaken at our center.

Methods

Our implementation approach included seeking support from hospital leadership; building frontline support and a team of champions among patients, nurses, anesthesiologists, and surgeons; accounting for stakeholder perceptions using theory-informed qualitative interviews; engaging patients; and documenting the implementation process, including barriers and facilitators, using the consolidated framework for implementation research.

Results

During the 12-month implementation period, we conducted 23 stakeholder engagement activities with over 200 participants. We recruited 10 clinician champions representing nursing, anesthesia, and surgery. We formally interviewed 15 patients and 17 perioperative clinicians and identified key themes to include in an information campaign run as part of the implementation process. Two patient partners were engaged and advised on communications as well as grant and protocol development. Many anticipated and unanticipated challenges were encountered at all levels. Implementation was ultimately successful, with the Operating Room Black Box installed in August 2018, and data collection beginning shortly thereafter.

Conclusions

This paper represents the first step toward evidence-guided implementation of technologies for direct observation of performance for research and quality improvement in surgery. With technology increasingly being used in health care settings, the health care community should aim to optimize implementation processes in the best interest of health care professionals and patients.

Endourological Training Using 3D-Printed Bladder Phantoms: Development and Prospective Evaluation

Background: To create and evaluate a realistic, anatomically accurate, and user-friendly bladder phantom for reproducible endourological training purposes and endoscope mastery. Materials and Methods: The anatomy of full bladders was mapped from human computed tomography datasets. After a 3D model development process, content evidence and response process evidence (RPE) of the phantom were evaluated using the system usability scale (SUS), 5-point Likert scale questionnaires, and task execution of experienced urologists (U) and endoscopy-naive medical students (MS) in two training sessions (first vs second). Required validation cohort sizes (1:10) of the evaluating urologists (n = 12) and students (n = 115) were precalculated. Time measurements were recorded. Students were additionally evaluated by a validated global psychomotor assessment score (GPSS). Group comparisons were calculated by the Mann-Whitney U test. All tests were two sided with p < 0.05 considered statistically significant. Results: Content evidence was assessed by urologists with an "excellent" SUS score of 89.4 ± 5.9 and an average "agreement" of ≥4 pts in the Likert scale questionnaires. RPE was assessed by intra- and intergroup time comparison for the execution of endoscopic tasks (cystoscopy [CY], guidewire insertion, and tumor biopsy). For CY, U: first 17.6 ± 4.4 seconds vs second 12.4 ± 2.0 seconds, p = 0.002; MS: first 56.6 ± 28.2 seconds vs second 28.6 ± 14.7 seconds, p < 0.001; U vs MS: first U 17.6 ± 4.4 seconds vs first MS 56.6 ± 28.2 seconds, p < 0.001, second U 12.4 ± 2.0 seconds vs second MS 28.6 ± 14.7 seconds, p < 0.001. Significant time differences were documented for all tasks and sessions (p < 0.001). Additionally, significant GPSS differences were recorded between the sessions (GPSS: first 20.4 ± 5.1 pts vs second 24.7 ± 4.0 pts, p < 0.001). Conclusions: Our low-fidelity 3D-printed bladder, called BladCap, is an easy-to-assemble, inexpensive, and robust phantom. We present data, which establish construct validity to support use as a clinical training device.

Implementation and evaluation of a novel subspecialty society fellows robotic surgical course: the SGO minimally invasive academy surgical curriculum

OBJECTIVE

To evaluate the utility of a society-based robotic surgery training program for fellows in gynecologic oncology.

METHODS

All participants underwent a 2-day robotic surgery training course between 2015-2017. The course included interactive didactic sessions with video, dry labs, and robotic cadaver labs. The labs encompassed a wide range of subject matter including troubleshooting, instrument variation, radical hysterectomies, and lymph node dissections. Participants completed a pre- and post-course survey using a 5-point Likert scale ranging from "not confident" to "extremely confident" on various measures. Statistical analysis was performed using SPSS Statistics v. 24.

RESULTS

The response rate was high with 86% of the 70 participants completing the survey. Sixteen (26.7%) of these individuals were attending physicians and 44 (73.3%) were fellows. In general, there was a significant increase in confidence in more complex procedures and concepts such as radical hysterectomy (p=0.01), lymph node dissection (p=0.01), troubleshooting (p=0.001), and managing complications (p=0.004). Faculty comfort and practice patterns were cited as the primary reason (58.9%) for limitations during robotic procedures followed secondarily by surgical resources (34.0%).

CONCLUSION

In both gynecologic oncology fellows and attendings, this educational theory-based curriculum significantly improved confidence in the majority of procedures and concepts taught, emphasizing the value of hands-on skill labs.

A Low-Cost Synthetic Abdominal Wall Model ("Raj Model") for the Training of Laparoscopic Port Insertion

OBJECTIVE

To assess the content validity of a low-cost bench-top model ("Raj Model") for the training of laparoscopic port insertion at the Urology Simulation Bootcamp course (USBC).

MATERIALS AND METHODS

A low-cost abdominal wall model of 40 × 40 cm was created to simulate laparoscopic port placement. The model was made using different synthetic materials to represent layers (skin-vinyl sheet, subcutaneous fat-10 mm soft foam, anterior rectus sheath and muscle-floor mat, posterior rectus sheath-masking wall tape, peritoneum-sellotape). Each model was used by up to 3 trainees to practise laparoscopic port placement. The model was assessed for content validity by trainees and experts using a 5-point Likert scale.

RESULT

In total, 88 trainees and 6 experts participated in the study. For all aspects of the synthetic abdominal wall, good (4) or very good (5) scores ranged from 52.7-69.2%, whereas very poor (1) rating ranged from 0 to 4.3%. There was no significant difference in responses for the content validity of the model between trainees and experts. There was a high intraclass correlation amongst responses from trainees (0.89) and experts (0.79). Approximately 76.3% of trainees and experts felt that the model is suitable for training.

CONCLUSION

This is the first validation study of a low-cost abdominal wall model for teaching laparoscopic port placement for trainees. Our study demonstrates that this synthetic model has high content validity and is useful for surgical training.

The utility of haptic simulation in early restorative dental training: A scoping review

BACKGROUND

Haptic dental simulators are becoming increasingly available in dental schools around the world; however, there is a paucity of evidence on their pedagogical effectiveness particularly in early dental training for the acquisition of the highly specific fine motor dental skills.

METHODS

A scoping review was performed to broadly map the available evidence and to detect knowledge gaps on the utility of haptic dental simulation in early dental training. The review is reported using the PRISMA-ScR guidelines. Eight bibliographic databases were searched: Web of Science, Scopus, MEDLINE via PubMed, Cochrane library, CENTRAL, ERIC, IEEE Xplore, and TRIP. Charted data were reported by clustering results according to study characteristics, research themes, research purpose, and type of validity evidence identified.

RESULTS

The review process resulted in the inclusion of 36 studies published between 2009 and 2020. The majority of the studies were cross-sectional in design with short-term evaluation data. Of the studies included, 64% investigated commercially available haptic simulators, while 36% investigated experimental haptic simulators. The research themes identified were skill acquisition and transfer, task-specific haptic training, trainee level discrimination, feedback, subjective user evaluation, performance prediction, and human factors in haptic training.

CONCLUSION

Short-term evaluation evidence from reviewed studies indicates the usefulness of the haptic simulators in early dental training. They complement the existing phantom head simulators by offering qualitatively different features. Further empirical research is needed to investigate the long-term impact of training with haptic dental simulators, to improve the availability of validation evidence and to enhance the results generalizability.

Comparison of a perfusion simulator to a clinical operating room: evaluation of eye tracking data and subjective perception. A pilot study

Background:

With the aim of evaluating the perfusion simulator at the German Heart Center Berlin, similarity between simulation and clinical operation room (OR) was investigated regarding subjective perception and eye movement.

Methods:

Eight perfusionists performed an operation on the heart-lung machine (HLM) wearing eye tracking glasses, each in real OR and simulator. The three most important phases for perfusionists (going on bypass, cardioplegia administration and coming off bypass) were considered. Additional to eye tracking data as objective measure, questionnaires were completed, and interviews conducted afterwards.

Results:

The structure of simulator and real OR is perceived as basically the same. Yet there are differences in the HLM-models used and the temporal sequence. Different perception of both situations is reported in interviews and reflected in significant differences in the rating scales (NASA-TLX) on three of six subscales. In eye tracking data, certain AOIs could be identified for the individual phases, both in OR and simulator—an indication of fundamental similarity. However, differences regarding the proportions of the individual AOIs, especially in the first and third phase, are leading to the assumption that the simulator, and especially the simulation process, is only valid to a limited extent regarding subjective perception and eye tracking data.

Conclusion:

The use of the simulator for (advanced) training is accepted and explicitly requested by perfusionists. Yet further research is needed to identify the decisive factors (like simulation duration or additional tasks) for a valid execution in the simulator. Furthermore, a larger sample size should be regarded to allow statistical analysis.

Amniocentesis learning curve using a low-cost simulation model to teach maternal-fetal medicine fellows

OBJECTIVE

To describe the learning curve for amniocentesis among Maternal-Fetal Medicine (MFM) fellows using a low-cost simulation model in Mexico.

METHODS

Fourteen first- and second-year MFM fellows with no previous experience in amniocentesis participated in this single-center prospective study from March to June of 2019. The study was approved by the Institutional Review Board at the Instituto Nacional de Perinatologia and written informed consent was obtained from all participants. After an introductory course based on a standardized technique for amniocentesis, each fellow performed this procedure using a low-cost simulation model; experienced operators supervised the procedures. Learning curves were then created using cumulative sum analysis. Thresholds for acceptable and unacceptable failure rates were defined as 10% and 25%, respectively.

RESULTS

Experienced MFM specialists evaluated 3675 procedures. On average, MFM fellows performed 263 ± 53 procedures. The mean number to achieve competence was 255 ± 53. The overall failure rate among the trainees was 16%.

CONCLUSION

We describe individual learning curves for amniocentesis among MFM fellows using a low-cost simulation model. This approach allows direct assessment of proficiency in amniocentesis before clinical practice.

Virtual Reality Anterior Cervical Discectomy and Fusion Simulation on the Novel Sim-Ortho Platform: Validation Studies

BACKGROUND

Virtual reality spine simulators are emerging as potential educational tools to assess and train surgical procedures in safe environments. Analysis of validity is important in determining the educational utility of these systems.

OBJECTIVE

To assess face, content, and construct validity of a C4-C5 anterior cervical discectomy and fusion simulation on the Sim-Ortho virtual reality platform, developed by OSSimTechTM (Montreal, Canada) and the AO Foundation (Davos, Switzerland).

METHODS

Spine surgeons, spine fellows, along with neurosurgical and orthopedic residents, performed a simulated C4-C5 anterior cervical discectomy and fusion on the Sim-Ortho system. Participants were separated into 3 categories: post-residents (spine surgeons and spine fellows), senior residents, and junior residents. A Likert scale was used to assess face and content validity. Construct validity was evaluated by investigating differences between the 3 groups on metrics derived from simulator data. The Kruskal-Wallis test was employed to compare groups and a post-hoc Dunn's test with a Bonferroni correction was utilized to investigate differences between groups on significant metrics.

RESULTS

A total of 21 individuals were included: 9 post-residents, 5 senior residents, and 7 junior residents. The post-resident group rated face and content validity, median ≥4, for the overall procedure and at least 1 tool in each of the 4 steps. Significant differences (P &lt; .05) were found between the post-resident group and senior and/or junior residents on at least 1 metric for each component of the simulation.

CONCLUSION

The C4-C5 anterior cervical discectomy and fusion simulation on the Sim-Ortho platform demonstrated face, content, and construct validity suggesting its utility as a formative educational tool.

Bimanual Fundamentals: Validation of a New Curriculum for Virtual Reality Training of Laparoscopic Skills

Background. To determine face and construct validity for the new Bimanual Fundamentals curriculum for the Simendo^® Virtual Reality Laparoscopy Simulator and prove its efficiency as a training and objective assessment tool for surgical resident’s advanced psychomotor skills. Methods. 49 participants were recruited: 17 medical students (novices), 15 residents (intermediates), and 17 medical specialists (experts) in the field of gynecology, general surgery, and urology in 3 tertiary medical centers in the Netherlands. All participants performed the 5 exercises of the curriculum for 3 consecutive times on the simulator. Intermediates and experts filled in a questionnaire afterward, regarding the reality of the simulator and training goals of each exercise. Statistical analysis of performance was performed between novices, intermediates, and experts. Parameters such as task time, collisions/displacements, and path length right and left were compared between groups. Additionally, a total performance score was calculated for each participant. Results. Face validity scores regarding realism and training goals were overall positive (median scores of 4 on a 5-point Likert scale). Participants felt that the curriculum was a useful addition to the previous curricula and the used simulator would fit in their residency programs. Construct validity results showed significant differences on the great majority of measured parameters between groups. The simulator is able to differentiate between performers with different levels of laparoscopic experience. Conclusions. Face and construct validity for the new Bimanual Fundamental curriculum for the Simendo virtual reality simulator could be established. The curriculum is suitable to use in resident’s training programs to improve and maintain advanced psychomotor skills.

Assessment of validity evidence for the RobotiX robot assisted surgery simulator on advanced suturing tasks

Background

Robot assisted surgery has expanded considerably in the past years. Compared to conventional open or laparoscopic surgery, virtual reality (VR) training is an essential component in learning robot assisted surgery. However, for tasks to be implemented in a curriculum, the levels of validity should be studied for proficiency-based training. Therefore, this study was aimed to assess the validity evidence of advanced suturing tasks on a robot assisted VR simulator.

Method

Participants were voluntary recruited and divided in the robotic experienced, laparoscopic experienced or novice group, based on self-reported surgical experience. Subsequently, a questionnaire on a five-point Likert scale was completed to assess the content validity. Three component tasks of complex suturing were performed on the RobotiX simulator (Task1: tilted plane needle transfer, Task: 2 intracorporal suturing, Task 3: anastomosis needle transfer). Accordingly, the outcome of the parameters was used to assess construct validity between robotic experienced and novice participants. Composite scores (0–100) were calculated from the construct parameters and corresponding pass/fail scores with false positive (FP) and false negative (FN) percentages.

Results

Fifteen robotic experienced, 26 laparoscopic experienced and 29 novices were recruited. Overall content validity outcomes were scored positively on the realism (mean 3.7), didactic value (mean 4.0) and usability (mean 4.2). Robotic experienced participants significantly outperformed novices and laparoscopic experienced participants on multiple parameters on all three tasks of complex suturing. Parameters showing construct validity mainly consisted of movement parameters, needle precision and task completion time. Calculated composite pass/fail scores between robotic experienced and novice participants resulted for Task 1 in 73/100 (FP 21%, FN 5%), Task 2 in 85/100 (FP 28%, FN 4%) and Task 3 in 64/100 (FP 49%, FN 22%).

Conclusion

This study assessed the validity evidence on multiple levels of the three studied tasks. The participants score the RobotiX good on the content validity level. The composite pass/fail scores of Tasks 1 and 2 allow for proficiency-based training and could be implemented in a robot assisted surgery training curriculum.

An Open-Source Three-Dimensionally Printed Laryngeal Model for Injection Laryngoplasty Training

OBJECTIVES/HYPOTHESIS

A limited number of three-dimensionally (3D)-printed laryngeal simulators have been described in the literature, only one of which is specifically designed for percutaneous injection laryngoplasty (PIL) training and is currently of limited availability. This study describes the development and evaluation of a high-fidelity, open-source, low-cost 3D-printed simulator for PIL training, improving on existing models.

STUDY DESIGN

Simulator design and survey evaluation.

METHODS

Computed tomography scans of the upper airways were processed with 3D Slicer to generate a computer model of the endolarynx. Blender and Fusion 360 were used to refine the mucosal model and develop casts for silicone injection molding. The casted endolaryngeal structures were inserted into a modified version of a publicly available laryngeal cartilage model. The final models were evaluated by 10 expert laryngologists using a customized version of the Michigan Standard Simulation Experience Scale. Internal consistency and interrater reliability of the survey were evaluated using Cronbach's α and intraclass correlation, respectively.

RESULTS

Expert laryngologists highly rated the model for measures of fidelity, educational value, and overall quality (mean = 4.8, standard deviation = 0.5; 1 = strongly disagree, 5 = strongly agree). All reviewers rated the model as ready for use as is or with slight modifications. The filament needed for one cartilage model costs $0.96, whereas the silicone needed for one soft-tissue model costs $1.89.

CONCLUSIONS

Using 3D-printing technology, we successfully created the first open-source, low-cost, and anatomically accurate laryngeal model for injection laryngoplasty training. Our simulator is made freely available for download on Wikifactory with step-by-step tutorials for 3D printing, silicone molding, assembly, and use.

LEVEL OF EVIDENCE

NA Laryngoscope, 131:E890-E895, 2021.

Use of a Low-Cost Portable 3D Virtual Reality Gesture-Mediated Simulator for Training and Learning Basic Psychomotor Skills in Minimally Invasive Surgery: Development and Content Validity Study

Background

Simulation in virtual environments has become a new paradigm for surgeon training in minimally invasive surgery (MIS). However, this technology is expensive and difficult to access.

Objective

This study aims first to describe the development of a new gesture-based simulator for learning skills in MIS and, second, to establish its fidelity to the criterion and sources of content-related validity evidence.

Methods

For the development of the gesture-mediated simulator for MIS using virtual reality (SIMISGEST-VR), a design-based research (DBR) paradigm was adopted. For the second objective, 30 participants completed a questionnaire, with responses scored on a 5-point Likert scale. A literature review on the validity of the MIS training-VR (MIST-VR) was conducted. The study of fidelity to the criterion was rated using a 10-item questionnaire, while the sources of content-related validity evidence were assessed using 10 questions about the simulator training capacity and 6 questions about MIS tasks, and an iterative process of instrument pilot testing was performed.

Results

A good enough prototype of a gesture-based simulator was developed with metrics and feedback for learning psychomotor skills in MIS. As per the survey conducted to assess the fidelity to the criterion, all 30 participants felt that most aspects of the simulator were adequately realistic and that it could be used as a tool for teaching basic psychomotor skills in laparoscopic surgery (Likert score: 4.07-4.73). The sources of content-related validity evidence showed that this study’s simulator is a reliable training tool and that the exercises enable learning of the basic psychomotor skills required in MIS (Likert score: 4.28-4.67).

Conclusions

The development of gesture-based 3D virtual environments for training and learning basic psychomotor skills in MIS opens up a new approach to low-cost, portable simulation that allows ubiquitous learning and preoperative warm-up. Fidelity to the criterion was duly evaluated, which allowed a good enough prototype to be achieved. Content-related validity evidence for SIMISGEST-VR was also obtained.

Low-cost simulation models in Urology: a systematic review of the literature

INTRODUCTION

Simulation models have been found to be effective and valid for training in Urology. Due to increasing costs of surgical training, there is a need for low-cost simulation models to enable Urology trainees to improve their skills.

MATERIAL AND METHODS

A literature review was performed using the PubMed and Embase databases until March 2020. A total of 157 abstracts were identified using the search criteria, of which 20 articles were identified describing simulation models for Urology training. Articles reviewed described simulation models created from materials costing less than $150. Data was extracted from the relevant articles in order to critically assess each paper for validity, ease of construct and educational impact.

RESULTS

Models were found pertaining to suprapubic catheterization (6), cystoscopy (3), percutaneous nephrolithotomy (5), scrotal examination (1), circumcision (1), ureteroscopy (1), transurethral resection of the prostate and bladder (2), and open prostatectomy (1). 18/20 (90%) assessed for either face, content, or construct validity. None of the papers evaluated assessed for transferability of skills to performance in real patients.

CONCLUSIONS

A plethora of low-cost simulation models for urological procedures are described in the literature, many of which can be easily constructed from cheap and accessible materials. However there is a need for further efforts to validate or assess for transferability of skills to clinical practice.