Psychometric properties of the Global Operative Assessment of Laparoscopic Skills (GOALS) using item response theory

Comparison of surgical performance using articulated (ArtiSential®) and conventional instruments for colorectal laparoscopic surgery: A single-centre, open, before-and-after, prospective study

AIM

Rigid surgical instruments limit movement whereas articulated instruments offer better control in small spaces and allow for intuitive and ergonomic movements. However, the effectiveness of the use of articulated instruments in improving colorectal laparoscopic outcomes remains unclear. The aim of this work was to determine whether colorectal laparoscopic surgical proficiency improved when multijoint instruments were used instead of conventional ones.

METHOD

We enrolled 70 consecutive patients (n = 20 for conventional instruments) aged 19-80 years who underwent elective laparoscopic surgery for colorectal diseases. Unedited surgery videos were validated using the modified Global Operative Assessment of Laparoscopic Skills (mGOALS) scale. Learning curves were analysed using a cumulative sum control chart for mGOALS grades.

RESULTS

The surgery type, length of hospital stay and 30-day postoperative complication rates were comparable between the groups, and the surgeon's mGOALS grades were similar (p = 0.190). However, in the articulated group, the scores were significantly higher for depth perception (p = 0.012) and tissue-handling domains (p = 0.046), while surgical duration was significantly shorter and intraoperative blood loss was significantly lower (p = 0.022), compared with those in the conventional (p = 0.002) group. Learning curve findings indicated that the first 10 and subsequent 40 surgeries in the articulated group were within the inexperienced and experienced phases, respectively. The mGOALS score in the experienced phase improved in the articulated group compared with that in the conventional group (p = 0.036).

CONCLUSIONS

The use of articulated instruments in laparoscopic colorectal surgery showed potential benefits. Further studies are needed to confirm these findings.

How can surgical skills in laparoscopic colon surgery be objectively assessed?-a scoping review

Background

In laparoscopic colorectal surgery, higher technical skills have been associated with improved patient outcome. With the growing interest in laparoscopic techniques, pressure on surgeons and certifying bodies is mounting to ensure that operative procedures are performed safely and efficiently. The aim of the present review was to comprehensively identify tools for skill assessment in laparoscopic colon surgery and to assess their validity as reported in the literature.

Methods

A systematic search was conducted in EMBASE and PubMed/MEDLINE in May 2021 to identify studies examining technical skills assessment tools in laparoscopic colon surgery. Available information on validity evidence (content, response process, internal structure, relation to other variables, and consequences) was evaluated for all included tools.

Results

Fourteen assessment tools were identified, of which most were procedure-specific and video-based. Most tools reported moderate validity evidence. Commonly not reported were rater training, assessment correlation with variables other than training level, and validity reproducibility and reliability in external educational settings.

Conclusion

The results of this review show that several tools are available for evaluation of laparoscopic colon cancer surgery, but few authors present substantial validity for tool development and use. As we move towards the implementation of new techniques in laparoscopic colon surgery, it is imperative to establish validity before surgical skill assessment tools can be applied to new procedures and settings. Therefore, future studies ought to examine different aspects of tool validity, especially correlation with other variables, such as patient morbidity and pathological reports, which impact patient survival.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00464-021-08914-z.

Performance assessment - The knowledge, skills and attitudes of surgical performance

Pediatric surgical education has traditionally focused on the hard cognitive and psychomotor (technical) skills. While more and more attention is being paid to softer skills such as communication, collaboration, leadership, health advocacy, professionalism and scholarship, the bulk of curricula remain focused on the hard skills. An intricate part of education is the assessment of performance. This article reviews the current literature on the assessment of cognitive and psychomotor skills, focusing particularly on the assessment of technical skills in the realm of simulation.

Towards Development of an Open Surgery Competency Assessment for Residents (OSCAR) Tool - A Systematic Review of the Literature and Delphi Consensus

OBJECTIVE

Competency-based education has mandated accurate intra-operative assessment tools. We aimed to define consensus-based open surgical skills perceived by experts as critical for assessment.

DESIGN

A mixed-method design was employed: systematic review and e-Delphi methodology.

SETTING

The study was performed at McGill University-affiliated large tertiary academic centers in Montreal, Quebec, Canada.

PARTICIPANTS

Per PRISMA guidelines, a peer-reviewed search strategy was employed. Studies published in English and those describing technical skill assessment of open abdominal surgery were included; subspecialty-specific skills, conference abstracts, academic memoirs were excluded. Most-cited skills were subjected to e-Delphi methodology to identify those deemed essential by experts, based a 3-point Likert scale. Eighteen McGill University-affiliated general surgeons, representing a variety of subspecialties of General Surgery, were invited to answer the questionnaire.

RESULTS

Around 120 of 4285 references were retained for analysis. The 12 most cited skills included suturing, tissue and instrument handling, movement economy, instrument knowledge, knot tying, flow, knowledge of procedure, completion time, dissection technique, knowledge of anatomy and sterile technique; 6 of these achieved high or perfect scores and agreement after 2 rounds of survey: suturing, sterile technique, knot tying, knowledge of anatomy, knowledge of procedure, and tissue handling. Median standard deviation decreased (0.495 to 0.450) from first to second round, indicating improvement in consensus.

CONCLUSION

These results will help develop and validate the OSCAR (objective structured clinical assessment rubric) assessment tool for immediate intra-operative feedback of open technical skills for surgical trainees.

Assessment of surgical performance of laparoscopic benign hiatal surgery: a systematic review

BACKGROUND

Operative skills correlate with patient outcomes, yet at the completion of training or after learning a new procedure, these skills are rarely formally evaluated. There is interest in the use of summative video assessment of laparoscopic benign foregut and hiatal surgery (LFS). If this is to be used to determine competency, it must meet the robust criteria established for high-stakes assessments. The purpose of this review is to identify tools that have been used to assess performance of LFS and evaluate the available validity evidence for each instrument.

METHODS

A systematic search was conducted up to July 2017. Eligible studies reported data on tools used to assess performance in the operating room during LFS. Two independent reviewers considered 1084 citations for eligibility. The characteristics and testing conditions of each assessment tool were recorded. Validity evidence was evaluated using five sources of validity (content, response process, internal structure, relationship to other variables, and consequences).

RESULTS

There were six separate tools identified. Two tools were generic to laparoscopy, and four were specific to LFS [two specific to Nissen fundoplication (NF), one heller myotomy (HM), and one paraesophageal hernia repair (PEH)]. Overall, only one assessment was supported by moderate evidence while the others had limited or unknown evidence. Validity evidence was based mainly on internal structure (all tools reporting reliability and item analysis) and content (two studies referencing previous papers for tool development in the context of clinical assessment, and four listing items without specifying the development procedures). There was little or no evidence supporting test response process (one study reporting rater training), relationship to other variables (two comparing scores in subjects with different clinical experience), and consequences (no studies). Two tools were identified to have evidence for video assessment, specific to NF.

CONCLUSION

There is limited evidence supporting the validity of assessment tools for laparoscopic foregut surgery. This precludes their use for summative video-based assessment to verify competency. Further research is needed to develop an assessment tool designed for this purpose.

Refining How We Define Laparoscopic Expertise

BACKGROUND

Traditional stratification of expertise in laparoscopic simulation assigns participants to novice, intermediate, or expert groups based on case numbers. We hypothesized that expert video assessment might refine this discrimination of psychomotor expertise, especially in light of new measurable parameters.

MATERIALS AND METHODS

One hundred five participants performed a defined intracorporeal suturing task in the pediatric laparoscopic surgery simulator armed with force-sensing capabilities. Participants were stratified into novice, intermediate, and expert groups via three classification schemes: (1) number of complex laparoscopic cases, (2) self-declared level of expertise, and (3) average expert rating of participants' videos. Precision, time to task completion, and force analysis parameters (FAP = total, maximum and mean forces in three axes) were compared using one-way analysis of variance tests. P < .05 was considered significant.

RESULTS

Participants stratified on the basis of case numbers and on the basis of self-declared level of expertise had statistically significant differences in time to task completion, but no significant difference in FAP. When participants were restratified according to expert assessment of their video performance, time to task completion as well as total and mean forces in X, Y, and Z axes allowed discrimination between novices, intermediates, and experts, thus establishing construct validity for the latter. Precision did not allow discrimination in any stratification scheme.

CONCLUSION

Compared with traditional stratification, video assessment allows refined discrimination of psychomotor expertise within a simulator. Assessment of FAP may become a relevant tool for teaching and assessing laparoscopic skills.

A scoping review of assessment tools for laparoscopic suturing

BACKGROUND

A needs assessment identified a gap in teaching and assessment of laparoscopic suturing (LS) skills. The purpose of this review is to identify assessment tools that were used to assess LS skills, to evaluate validity evidence available, and to provide guidance for selecting the right assessment tool for specific assessment conditions.

METHODS

Bibliographic databases were searched till April 2017. Full-text articles were included if they reported on assessment tools used in the operating room/simulation to (1) assess procedures that require LS or (2) specifically assess LS skills.

RESULTS

Forty-two tools were identified, of which 26 were used for assessing LS skills specifically and 26 for procedures that require LS. Tools had the most evidence in internal structure and relationship to other variables, and least in consequences.

CONCLUSION

Through identification and evaluation of assessment tools, the results of this review could be used as a guideline when implementing assessment tools into training programs.