A hierarchical task analysis of shoulder arthroscopy for a virtual arthroscopic tear diagnosis and evaluation platform (VATDEP)

Vesicovaginal Fistula Repair Simulation Model and Hierarchical Task Analysis

IMPORTANCE

There is a need for surgeons skilled in vesicovaginal fistula (VVF) repair, yet training opportunities are limited.

OBJECTIVES

This study aimed to create a low-fidelity simulation model for transvaginal VVF repair, identify essential steps of VVF repair, and evaluate the model's ability to replicate essential steps.

STUDY DESIGN

First, a low-fidelity VVF repair simulation model was designed and built by the authors. Next, a hierarchical task analysis was performed by urogynecologic surgeons with expertise in VVF repair. Each expert submitted an outline of tasks required to perform VVF repair. To control for bias, an education specialist de-identified, reviewed, and collated the submitted outlines. The education specialist then led a focus group, and through a modified Delphi process, the experts reached consensus on the essential steps. A separate group of urogynecologic surgeons then tested the model and completed an anonymous questionnaire assessing how well the model replicated the essential steps. Descriptive analyses were performed.

RESULTS

Five experts submitted an outline of steps for transvaginal VVF repair, and 4 experts participated in a focus group to reach consensus on the essential steps. Nine urogynecologic surgeons, with a median of 10 years in practice (interquartile range, 7-12 years), tested the model and completed the postsimulation questionnaire. Most testers thought that tasks involving identification and closure of the fistula were replicated by the model. Testers thought that tasks involving cystoscopy or bladder filling were not replicated by the model.

CONCLUSIONS

We developed a novel, low-fidelity transvaginal VVF repair simulation model that consistently replicated tasks involving identification and closure of the fistula.

Validity Evidence for Procedure-specific Competence Assessment Tools in Orthopaedic Surgery: A Scoping Review

INTRODUCTION

Competency-based training requires frequent assessment of residents' skills to determine clinical competence. This study reviews existing literature on procedure-specific competence assessment tools in orthopaedic surgery.

METHODS

A systematic search of eight databases up to May 2023 was conducted. Two reviewers independently assessed validity evidence and educational utility of each assessment tool and evaluated studies' methodological quality.

RESULTS

Database searching identified 2,556 unique studies for title and abstract screening. Full texts of 290 studies were reviewed; 17 studies met the inclusion criteria. Bibliography review identified another five studies, totaling 22 studies examining 24 assessment tools included in the analysis. These tools assessed various orthopaedic surgery procedures within trauma, sports medicine, spine, and upper extremity. Overall validity evidence was low across all studies, and was lowest for consequences and highest for content. Methodological quality of studies was moderate. Educational utility assessment was not explicitly done for most tools.

DISCUSSION

The paucity of current procedure-specific assessment tools in orthopaedic surgery lacks the validity evidence required to be used reliably in high-stake summative assessments. Study strengths include robust methodology and use of an evidence-based validity evidence framework. Poor-quality existing evidence is a limitation and highlights the need for evidence-based tools across more subspecialties.

Scoring metrics for assessing skills in arthroscopic rotator cuff repair: performance comparison study of novice and expert surgeons

PURPOSE

We aim to develop quantitative performance metrics and a deep learning model to objectively assess surgery skills between the novice and the expert surgeons for arthroscopic rotator cuff surgery. These proposed metrics can be used to give the surgeon an objective and a quantitative self-assessment platform.

METHODS

Ten shoulder arthroscopic rotator cuff surgeries were performed by two novices, and fourteen were performed by two expert surgeons. These surgeries were statistically analyzed. Two existing evaluation systems: Basic Arthroscopic Knee Skill Scoring System (BAKSSS) and the Arthroscopic Surgical Skill Evaluation Tool (ASSET), were used to validate our proposed metrics. In addition, a deep learning-based model called Automated Arthroscopic Video Evaluation Tool (AAVET) was developed toward automating quantitative assessments.

RESULTS

The results revealed that novice surgeons used surgical tools approximately 10% less effectively and identified and stopped bleeding less swiftly. Our results showed a notable difference in the performance score between the experts and novices, and our metrics successfully identified these at the task level. Moreover, the F1-scores of each class are found as 78%, 87%, and 77% for classifying cases with no-tool, electrocautery, and shaver tool, respectively.

CONCLUSION

We have constructed quantitative metrics that identified differences in the performances of expert and novice surgeons. Our ultimate goal is to validate metrics further and incorporate these into our virtual rotator cuff surgery simulator (ViRCAST), which has been under development. The initial results from AAVET show that the capability of the toolbox can be extended to create a fully automated performance evaluation platform.

Hierarchical task analysis of endoscopic sleeve gastroplasty

BACKGROUND

Endoscopic sleeve gastroplasty (ESG) is a minimally invasive endoscopic weight loss procedure used to treat obesity. The long-term goal of this project is to develop a Virtual Bariatric Endoscopy (ViBE) simulator for training and assessment of the ESG procedure. The objectives of this current work are to: (a) perform a task analysis of ESG and (b) create metrics to be validated in the created simulator.

METHODS

We performed a hierarchical task analysis (HTA) by identifying the significant tasks of the ESG procedure. We created the HTA to show the breakdown and connection of the tasks of the procedure. Utilizing the HTA and input from ESG experts, performance metrics were derived for objective measurement of the ESG procedure. Three blinded video raters analyzed seven recorded ESG procedures according to the proposed performance metrics.

RESULTS

Based on the seven videos, there was a positive correlation between total task times and total performance scores (R = 0.886, P = 0.008). Endoscopists expert were found to be more skilled in reducing the area of the stomach compared to endoscopists novice (34.6% reduction versus 9.4% reduction, P = 0.01). The mean novice performance score was significantly lower than the mean expert performance score (34.7 vs. 23.8, P = 0.047). The inter-rater reliability test showed a perfect agreement among three raters for all tasks except for the suturing task. The suturing task had a significant agreement (Inter-rater Correlation = 0.84, Cronbach's alpha = 0.88). Suturing was determined to be a critical task that is positively correlated with the total score (R = 0.962, P = 0.0005).

CONCLUSION

The task analysis and metrics development are critical for the development of the ViBE simulator. This preliminary assessment demonstrates that the performance metrics provide an accurate assessment of the endoscopist's performance. Further validation testing and refinement of the performance metrics are anticipated.

Systematic approach for content and construct validation: Case studies for arthroscopy and laparoscopy

BACKGROUND

In minimally invasive surgery, there are several challenges for training novice surgeons, such as limited field-of-view and unintuitive hand-eye coordination due to performing the operation according to video feedback. Virtual reality (VR) surgical simulators are a novel, risk-free, and cost-effective way to train and assess surgeons.

METHODS

We developed VR-based simulations to accurately assess and quantify performance of two VR simulations: gentleness simulation for laparoscopy and rotator cuff repair for arthroscopy. We performed content and construct validity studies for the simulators. In our analysis, we systematically rank surgeons using data mining classification techniques.

RESULTS

Using classification algorithms such as K-Nearest Neighbors, Support Vector Machines, and Logistic Regression we have achieved near 100% accuracy rate in identifying novices, and up to an 83% accuracy rate identifying experts. Sensitivity and specificity were up to 1.0 and 0.9, respectively.

CONCLUSION

Developed methodology to measure and differentiate the highly ranked surgeons and less-skilled surgeons.

Partition-based optimization model for generative anatomy modeling language (POM-GAML)

Background

This paper presents a novel approach for Generative Anatomy Modeling Language (GAML). This approach automatically detects the geometric partitions in 3D anatomy that in turn speeds up integrated non-linear optimization model in GAML for 3D anatomy modeling with constraints (e.g. joints). This integrated non-linear optimization model requires the exponential execution time. However, our approach effectively computes the solution for non-linear optimization model and reduces computation time from exponential to linear time. This is achieved by grouping the 3D geometric constraints into communities.

Methods

Various community detection algorithms (k-means clustering, Clauset Newman Moore, and Density-Based Spatial Clustering of Applications with Noise) were used to find communities and partition the non-linear optimization problem into sub-problems. GAML was used to create a case study for 3D shoulder model to benchmark our approach with up to 5000 constraints.

Results

Our results show that the computation time was reduced from exponential time to linear time and the error rate between the partitioned and non-partitioned approach decreases with the increasing number of constraints. For the largest constraint set (5000 constraints), speed up was over 2689-fold whereas error was computed as low as 2.2%.

Conclusion

This study presents a novel approach to group anatomical constraints in 3D human shoulder model using community detection algorithms. A case study for 3D modeling for shoulder models developed for arthroscopic rotator cuff simulation was presented. Our results significantly reduced the computation time in conjunction with a decrease in error using constrained optimization by linear approximation, non-linear optimization solver.

A task and performance analysis of endoscopic submucosal dissection (ESD) surgery

BACKGROUND

ESD is an endoscopic technique for en bloc resection of gastrointestinal lesions. ESD is a widely-used in Japan and throughout Asia, but not as prevalent in Europe or the US. The procedure is technically challenging and has higher adverse events (bleeding, perforation) compared to endoscopic mucosal resection. Inadequate training platforms and lack of established training curricula have restricted its wide acceptance in the US. Thus, we aim to develop a Virtual Endoluminal Surgery Simulator (VESS) for objective ESD training and assessment. In this work, we performed task and performance analysis of ESD surgeries.

METHODS

We performed a detailed colorectal ESD task analysis and identified the critical ESD steps for lesion identification, marking, injection, circumferential cutting, dissection, intraprocedural complication management, and post-procedure examination. We constructed a hierarchical task tree that elaborates the order of tasks in these steps. Furthermore, we developed quantitative ESD performance metrics. We measured task times and scores of 16 ESD surgeries performed by four different endoscopic surgeons.

RESULTS

The average time of the marking, injection, and circumferential cutting phases are 203.4 (σ: 205.46), 83.5 (σ: 49.92), 908.4 s. (σ: 584.53), respectively. Cutting the submucosal layer takes most of the time of overall ESD procedure time with an average of 1394.7 s (σ: 908.43). We also performed correlation analysis (Pearson's test) among the performance scores of the tasks. There is a moderate positive correlation (R = 0.528, p = 0.0355) between marking scores and total scores, a strong positive correlation (R = 0.7879, p = 0.0003) between circumferential cutting and submucosal dissection and total scores. Similarly, we noted a strong positive correlation (R = 0.7095, p = 0.0021) between circumferential cutting and submucosal dissection and marking scores.

CONCLUSIONS

We elaborated ESD tasks and developed quantitative performance metrics used in analysis of actual surgery performance. These ESD metrics will be used in future validation studies of our VESS simulator.