Using Artificial Intelligence to Gauge Competency on a Novel Laparoscopic Training System

OBJECTIVE

Laparoscopic surgical skill assessment and machine learning are often inaccessible to low-and-middle-income countries (LMIC). Our team developed a low-cost laparoscopic training system to teach and assess psychomotor skills required in laparoscopic salpingostomy in LMICs. We performed video review using AI to assess global surgical techniques. The objective of this study was to assess the validity of artificial intelligence (AI) generated scoring measures of laparoscopic simulation videos by comparing the accuracy of AI results to human-generated scores.

DESIGN

Seventy-four surgical simulation videos were collected and graded by human participants using a modified OSATS (Objective Structured Assessment of Technical Skills). The videos were then analyzed via AI using 3 different time and distance-based calculations of the laparoscopic instruments including path length, dimensionless jerk, and standard deviation of tool position. Predicted scores were generated using 5-fold cross validation and K-Nearest-Neighbors to train classifiers.

SETTING

Surgical novices and experts from a variety of hospitals in Ethiopia, Cameroon, Kenya, and the United States contributed 74 laparoscopic salpingostomy simulation videos.

RESULTS

Complete accuracy of AI compared to human assessment ranged from 65-77%. There were no statistical differences in rank mean scores for 3 domains, Flow of Operation, Respect for Tissue, and Economy of Motion, while there were significant differences in ratings for Instrument Handling, Overall Performance, and the total summed score of all 5 domains (Summed). Estimated effect sizes were all less than 0.11, indicating very small practical effect. Estimated intraclass correlation coefficient (ICC) of Summed was 0.72 indicating moderate correlation between AI and Human scores.

CONCLUSIONS

Video review using AI technology of global characteristics was similar to that of human review in our laparoscopic training system. Machine learning may help fill an educational gap in LMICs where direct apprenticeship may not be feasible.

Delayed presentation of penetrating cardiac injury successfully managed in resource limited setting: A case report

INTRODUCTION AND IMPORTANCE

Penetrating cardiac injury is rare and historically known to have very poor prognosis. Even today, 90 % of patients die before arriving to hospital. Even though patient presentations can be atypical, organized timely intervention can lead to survival.

CASE REPORT

A 21 years old arrived 5 h after stab injury to right anterior chest. He was hypotensive with a sucking wound bleeding on his right chest as well as hemothorax on the same side. Chest tube and pericardial window were both done with blood in pericardial space. Median sternotomy was done and revealed right atrial perforation. The perforation was repaired and the patient was discharged and continues to do well on follow up.

CLINICAL DISCUSSION

For most patients, time from injury to surgery is short. Focused and organized surveys as well as resuscitation are valuable for any patent with penetrating thoracic trauma. With a patient in hemorrhagic shock and a penetrating wound near the heart, a pericardial window is required regardless of the absence of pericardial fluid on ultrasound and in this case proved to be lifesaving. If there is a hole in the pericardium communicating with the pleural space the pericardial blood may decompress into the pleural cavity and not be visible on ultrasound.

CONCLUSION

Regardless of its rare prevalence, high index of suspicion for cardiac injury is extremely important in all patients with penetrating chest trauma in the cardiac box regardless of atypical presentations. With rapid diagnosis, capable surgeon availability, and availability of blood products, patients can survive this injury.

Evidence supporting performance measures of laparoscopic appendectomy through a novel surgical proficiency assessment tool and low-cost laparoscopic training system

BACKGROUND

Laparoscopic training remains inaccessible for surgeons in low- and middle-income countries, limiting its widespread adoption. We developed a novel tool for assessment of laparoscopic appendectomy skills through ALL-SAFE, a low-cost laparoscopy training system.

METHODS

This pilot study in Ethiopia, Cameroon, and the USA assessed appendectomy skills using the ALL-SAFE training system. Performance measures were captured using the ALL-SAFE verification of proficiency tool (APPY-VOP), consisting of a checklist, modified Objective Structured Assessment of Technical Skills (m-OSATS), and final rating. Twenty participants, including novice (n = 11), intermediate (n = 8), and expert (n = 1), completed an online module covering appendicitis management and psychomotor skills in laparoscopic appendectomy. After viewing an expert skills demonstration video, participants recorded their performance within ALL-SAFE. Using the APPY-VOP, participants rated their own and three peer videos. We used the Kruskal-Wallis test and a Many-Facet Rasch Model to evaluate (i) capacity of APPY-VOP to differentiate performance levels, (ii) correlation among three APPY-VOP components, and (iii) rating differences across groups.

RESULTS

Checklist scores increased from novice (M = 21.02) to intermediate (M = 23.64) and expert (M = 28.25), with differentiation between experts and novices, P = 0.005. All five m-OSATS domains and global summed, total summed, and final rating discriminated across all performance levels (P < 0.001). APPY-VOP final ratings adequately discriminated Competent (M = 2.0), Borderline (N = 1.8), and Not Competent (M = 1.4) performances, Χ2 (2,85) = 32.3, P = 0.001. There was a positive correlation between ALL-SAFE checklist and m-OSATS summed scores, r(83) = 0.63, P < 0.001. Comparison of ratings suggested no differences across expertise levels (P = 0.69) or location (P = 0.66).

CONCLUSION

APPY-VOP effectively discriminated between novice and expert performance in laparoscopic appendectomy skills in a simulated setting. Scoring alignment across raters suggests consistent evaluation, independent of expertise. These results support the use of APPY-VOP among all skill levels inside a peer rating system. Future studies will focus on correlating proficiency to clinical practice and scaling ALL-SAFE to other settings.

Mortality in hypotensive trauma patients requiring laparotomy is related to degree of hypotension and provides evidence for focused interventions

Background

Mortality in hypotensive patients requiring laparotomy is reported to be 46% and essentially unchanged in 20 years. Resuscitative endovascular balloon occlusion of the aorta (REBOA) has been incorporated into resuscitation protocols in an attempt to decrease mortality, but REBOA can have significant complications and its use in this patient group has not been validated. This study sought to determine the mortality rate for hypotensive patients requiring laparotomy and to evaluate the mortality risk related to the degree of hypotension. Additionally, this study sought to determine if there was a presenting systolic blood pressure (SBP) that was associated with a sharp increase in mortality to target the appropriate patient group most likely to benefit from focused interventions such as REBOA.

Methods

The trauma registry at a level I trauma center was reviewed for patients undergoing emergent laparotomy from January 2007 to June 2020. Data included demographics, mechanism of injury, physiological data, Injury Severity Score, blood products transfused, and outcomes. Group comparisons were based on initial SBP (0 to 50 mm Hg, 60 to 69 mm Hg, 70 to 79 mm Hg, 80 to 89 mm Hg, and ≥90 mm Hg).

Results

During the study period, 52 016 trauma patients were treated and 1174 required laparotomy within 90 min of arrival; 424 had an initial SBP of <90 mm Hg. The overall mortality rate was 18%, but mortality increased as SBP decreased (≥90=9%, 80 to 89=20%, 70 to 79=21%, 60 to 69=48%, 0 to 59=66%). Mortality increased sharply with SBP of <70 mm Hg.

Discussion

Mortality rate increases with worsening hypotension and increases sharply with an SBP of <70 mm Hg. Further study on focused interventions such as REBOA should target this patient group.

Level of evidence

Therapeutic/care management, level III.