Quantitative pediatric surgical skill assessment using a rapid-prototyped chest model

Validated simulation models in pediatric surgery: A review

INTRODUCTION

This review evaluates the validation and availability of simulation models in the field of pediatric surgery that can be used for training purposes.

METHODS

MEDLINE and EMBASE were searched for studies describing a simulation models in pediatric surgery. Articles were included if face, content and/or construct validity was described. Additionally, the costs and availability were assessed. Validation scores for each model were depicted as percentage (0-100), based on the reported data, to compare the outcomes. A score of >70% was considered adequate.

RESULTS

Forty-three studies were identified, describing the validation process of 38 simulation models. Face validity was evaluated in 33 articles, content in 36 and construct in 19. Twenty-two models received adequate validation scores (>70%). The majority (27/38, 70%) was strictly inanimate. Five models were available for purchase and eleven models were replicable based on the article.

CONCLUSION

The number of validated inanimate simulation models for pediatric surgery procedures is growing, however, few are replicable or available for widespread training purposes.

LEVEL OF EVIDENCE

Level II.

Evaluation of minimally invasive surgical skills training: comparing a neonatal esophageal atresia/tracheoesophageal fistula model with a dry box

BACKGROUND

Pediatric surgeons require highly advanced minimally invasive surgical skills to perform rare and complex surgeries in a very vulnerable population. We developed a neonatal esophageal atresia (EA) model to improve thoracoscopic surgical skills. This study aimed to evaluate the concurrent validity of the model by undertaking pre- and post-training skills assessments in two groups of students with no prior experience performing minimally invasive surgery, using the EA model and a dry box (DB).

METHODS

A pilot study was performed. The participants were randomly divided into two groups: one trained using the DB and one trained using the EA model. Both groups practiced a minimally invasive surgical suture task. The task completion time, 29-point checklist score, modified suturing error sheet score, and three-dimensional forceps movement in both groups were compared pre-and post-training by video analysis.

RESULTS

The EA model task was significantly more difficult than that of the DB. Both groups showed significant improvement in the task time, 29-point checklist score, and modified suturing error sheet score; however, the EA model training was more efficient in improving each error item. Regarding forceps movement, the EA model training significantly decreased wasted motion, whereas the DB was limited in this regard.

CONCLUSIONS

Short-term training on the EA model, which was more technically demanding than the DB, decreased technical error and wasted motion, and allowed learners to acquire surgical skills more efficiently than training with the DB model. These facts revealed the concurrent validity of the EA model.

Thiel embalming in neonates: methodology and benefits in medical training

Current teaching and training methods for surgical techniques in the pediatric population involve artificial models (manikins), animals or adult human cadavers embalmed using various techniques. We found no references in the literature concerning the use of the Thiel method in the pediatric population. The aim of this study, therefore, was to assess the viability of using pediatric human cadavers embalmed through Thiel’s technique and to compare them with standard pediatric manikins. After donation of a 24-week stillborn, the Thiel technique was carried out for fixation following the usual protocol. A video recording with eye-tracking glasses was used to perform an examination, and techniques. The same procedures were conducted on a pediatric manikin. Medical students, medical residents and physicians were asked to respond to questions in an online survey after being shown the video. A total of 92 responses were obtained. The Thiel-embalmed stillborn was assessed as superior to the manikin in all items. Our study confirmed that this technique is feasible even with extremely small donors. The value of this form of preservation for medical training is not widely known though it is receiving increasing interest. Our results show that Thiel fixation in pediatrics is clearly more highly valued than a manikin and offers great potential. This innovative application of the Thiel method in the pediatric population is technically possible. It poses no additional difficulties and is very positively assessed for undergraduate and postgraduate teaching.

Objective and automated assessment of surgical technical skills with IoT systems: A systematic literature review

The assessment of surgical technical skills to be acquired by novice surgeons has been traditionally done by an expert surgeon and is therefore of a subjective nature. Nevertheless, the recent advances on IoT (Internet of Things), the possibility of incorporating sensors into objects and environments in order to collect large amounts of data, and the progress on machine learning are facilitating a more objective and automated assessment of surgical technical skills. This paper presents a systematic literature review of papers published after 2013 discussing the objective and automated assessment of surgical technical skills. 101 out of an initial list of 537 papers were analyzed to identify: 1) the sensors used; 2) the data collected by these sensors and the relationship between these data, surgical technical skills and surgeons' levels of expertise; 3) the statistical methods and algorithms used to process these data; and 4) the feedback provided based on the outputs of these statistical methods and algorithms. Particularly, 1) mechanical and electromagnetic sensors are widely used for tool tracking, while inertial measurement units are widely used for body tracking; 2) path length, number of sub-movements, smoothness, fixation, saccade and total time are the main indicators obtained from raw data and serve to assess surgical technical skills such as economy, efficiency, hand tremor, or mind control, and distinguish between two or three levels of expertise (novice/intermediate/advanced surgeons); 3) SVM (Support Vector Machines) and Neural Networks are the preferred statistical methods and algorithms for processing the data collected, while new opportunities are opened up to combine various algorithms and use deep learning; and 4) feedback is provided by matching performance indicators and a lexicon of words and visualizations, although there is considerable room for research in the context of feedback and visualizations, taking, for example, ideas from learning analytics.

High-Frequency Ultrasonic Forceps for the In Vivo Detection of Cancer During Breast-Conserving Surgery

A major aim in the surgical management of soft tissue cancers is to detect and remove all cancerous tissues while ensuring noncancerous tissue remains intact. Breast-conserving surgery provides a prime illustration of this aim, since remaining cancer in breast margins results in multiple surgeries, while removal of too much unaffected tissue often has undesirable cosmetic effects. Similarly, resection of benign lymph nodes during sentinel lymph node biopsy can cause deleterious health outcomes. The objective of this study was to create an intraoperative, in vivo device to address these challenges. Instant diagnostic information generated by this device could allow surgeons to precisely and completely remove all malignant tissue during the first surgery. Surgical forceps based on Martin forceps were instrumented at the tips with high-frequency ultrasonic transducers composed of polyvinylidene difluoride, a thickness-sensing rotary potentiometer at the base, and a spring to provide the appropriate restoring force. Transducer wires within the forceps were connected to an external high-frequency pulser-receiver, activating the forceps' transmitting transducer at 50 MHz and amplifying through-transmission signals from the receiving transducer. The forceps were tested with tissue-mimicking agarose phantoms embedded with 58–550 μm polyethylene microspheres to simulate various stages of cancer progression and to provide a range of measurement values. Results were compared with measurements from standard 50 MHz immersion transducers. The results showed that the forceps displayed similar sensitivity for attenuation and increased accuracy for wave speed. The forceps could also be extended to endoscopes and laparoscopes.

Development of an instrumented thoracoscopic surgical trainer for objective evaluation of esophageal atresia/tracheoesophageal fistula repair

Operative repair of complex conditions such as esophageal atresia and tracheoesophageal fistula (EA/TEF) is technically demanding, but few training opportunities exist outside the operating theater for surgeons to attain these skills. Learning them during surgery on actual neonates where the stakes are high, margins for error narrow, and where outcomes are influenced by technical expertise, is problematic. There is an increasing demand for high-fidelity simulation that can objectively measure performance. We developed such a simulator to measure force and motion reliably, allowing quantitative feedback of technical skill. A 3D-printed simulator for thoracoscopic repair of EA/TEF was instrumented with motion and force tracking components. A 3D mouse, inertial measurement unit (IMU), and optical sensor that captured force and motion data in four degrees of freedom (DOF) were calibrated and verified for accuracy. The 3D mouse had low average relative errors of 2.81%, 3.15%, and 6.15% for 0 mm, 10 mm offset in Y, and 10 mm offset in X, respectively. This increased to - 23.5% at an offset of 42 mm. The optical sensors and IMU displayed high precision and accuracy with low SDs and average relative errors, respectively. These parameters can be a useful measurement of performance for thoracoscopic EA/TEF simulation prior to surgery. Graphical abstract Inclusion of sensors into a high-fidelity simulator design can produce quantitative feedback which can be used to objectively asses performance of a technically difficult procedure. As a result, more surgical training can be done prior to operating on actual patients in the operating theater.

Evaluation methods and impact of simulation-based training in pediatric surgery: a systematic review

PURPOSE

The aim of this study was to identify (1) the type of skill evaluation methods and (2) how the effect of training was evaluated in simulation-based training (SBT) in pediatric surgery.

METHODS

Databases of PubMed, Cochrane Library, and Web of Science were searched for articles published from January 2000 to January 2017. Search concepts of Medical Subject Heading terms were "surgery," "pediatrics," "simulation," and "training, evaluation."

RESULTS

Of 5858 publications identified, 43 were included. Twenty papers described simulators as assessment tools used to evaluate technical skills. Reviewers differentiated between experts and trainees using a scoring system (45%) and/or a checklist (25%). Simulators as training tools were described in 23 papers. While the training's effectiveness was measured using performance assessment scales (52%) and/or surveys (43%), no study investigated the improvement of the clinical outcomes after SBT.

CONCLUSION

Scoring, time, and motion analysis methods were used for the evaluation of basic techniques of laparoscopic skills. Only a few SBT in pediatric surgery have definite goals with clinical effect. Future research needs to demonstrate the educational effect of simulators as assessment or training tools on SBT in pediatric surgery.

Current status of simulation-based training in pediatric surgery: A systematic review

BACKGROUND

Simulation based training enables pediatric surgical trainees to attain proficiency in surgical skills. This study aims to identify the currently available simulators for pediatric surgery, assess their validation and strength of evidence supporting each model.

METHODS

Both Medline and EMBASE were searched for English language articles either describing or validating simulation models for pediatric surgery. A level of evidence (LoE) followed by a level of recommendation (LoR) was assigned to each validation study and simulator, based on a modified Oxford Centre for Evidence-Based Medicine classification for educational studies.

RESULTS

Forty-nine articles were identified describing 44 training models and courses. Of these articles, 44 were validation studies. Face validity was evaluated by 20 studies, 28 for content, 24 demonstrated construct validity and 1 showed predictive validity. Of the validated models, 3 were given an LoR of 2, 21 an LoR of 3 and 12 an LoR of 4. None reached the highest LoR.

CONCLUSIONS

There are a growing number of simulators specific to pediatric surgery. However, these simulators have limited LoE and LoR in current studies. The lack of NoTSS training is also apparent. We advocate more randomized trials to validate these models, and attempts to determine predictive validity.

TYPE OF STUDY

Original / systematic review.

LEVEL OF EVIDENCE

1.

Evaluation of Surgical Devices Using an Artificial Pediatric Thoracic Model: A Comparison Between Robot-Assisted Thoracoscopic Suturing Versus Conventional Video-Assisted Thoracoscopic Suturing

BACKGROUND

Pediatric robot-assisted surgery is increasingly being performed, but it is difficult to perform this procedure in infants. A pediatric thoracoscopic model of a 1-year-old patient was developed in our previous study, and this model was used to evaluate the use of a surgical robot for infant surgery.

METHODS

Eight pediatric surgeons performed an intracorporeal suturing and knot-tying task using the da Vinci Xi Robotic Surgical System. The task completion time, number of needle manipulations, and force applied during suturing of the robot-assisted thoracoscopic surgery (RATS) group were compared with those of the video-assisted thoracoscopic surgery (VATS) group whose data had been collected from the same 8 surgeons in our previous study.

RESULTS

The RATS group showed a significantly shorter completion time than the VATS group in the knot-tying phase (P = .016) and in the total phase (P = .0078). The RATS group showed a significantly smaller number of manipulations than the VATS group in the total phase (P = .039). The RATS group showed a significantly smaller pushing force index than the VATS group in the suturing phase (P = .031), knot-tying phase (P = .031), and in the total phase (P = .031). A seventh rib in the model was dislocated in all RATS group cases.

CONCLUSIONS

The da Vinci Surgical System might be useful in infants because of fast movement and small pushing force. However, the robotic 8 mm instruments were too large for use in the thoracic cavity of the 1-year-old infant.

Automatically rating trainee skill at a pediatric laparoscopic suturing task

BACKGROUND

Minimally invasive surgeons must acquire complex technical skills while minimizing patient risk, a challenge that is magnified in pediatric surgery. Trainees need realistic practice with frequent detailed feedback, but human grading is tedious and subjective. We aim to validate a novel motion-tracking system and algorithms that automatically evaluate trainee performance of a pediatric laparoscopic suturing task.

METHODS

Subjects (n = 32) ranging from medical students to fellows performed two trials of intracorporeal suturing in a custom pediatric laparoscopic box trainer after watching a video of ideal performance. The motions of the tools and endoscope were recorded over time using a magnetic sensing system, and both tool grip angles were recorded using handle-mounted flex sensors. An expert rated the 63 trial videos on five domains from the Objective Structured Assessment of Technical Skill (OSATS), yielding summed scores from 5 to 20. Motion data from each trial were processed to calculate 280 features. We used regularized least squares regression to identify the most predictive features from different subsets of the motion data and then built six regression tree models that predict summed OSATS score. Model accuracy was evaluated via leave-one-subject-out cross-validation.

RESULTS

The model that used all sensor data streams performed best, achieving 71% accuracy at predicting summed scores within 2 points, 89% accuracy within 4, and a correlation of 0.85 with human ratings. 59% of the rounded average OSATS score predictions were perfect, and 100% were within 1 point. This model employed 87 features, including none based on completion time, 77 from tool tip motion, 3 from tool tip visibility, and 7 from grip angle.

CONCLUSIONS

Our novel hardware and software automatically rated previously unseen trials with summed OSATS scores that closely match human expert ratings. Such a system facilitates more feedback-intensive surgical training and may yield insights into the fundamental components of surgical skill.