Development of a rating scale for objective assessment of performance in laparoscopic appendicectomy surgery

Backgrounds

Laparoscopic appendicectomy (LA) is the most common index procedure for junior surgical trainees. Despite the shift towards competency‐based training, there is no method of quantitatively assessing performance during LA. This study aimed to obtain expert consensus regarding the items required to create a LA Rating Scale (LARS).

Methods

A list of steps required for LA surgery, as well as descriptors of “poor”, “average” and “excellent” performance for each of these steps were created for potential inclusion into an objective assessment tool for LA surgery. Using a Delphi method, 20 experts from multiple institutions rated on a Likert scale from 1 to 5 the suitability of these steps and descriptors of performance. Responses were obtained until consensus (Cronbach's α > 0.8) was achieved.

Results

Fifteen experts participated in the study. Consensus was achieved for all items during the first iteration of the Delphi with a Cronbach's α of 0.96. The Cronbach's α for the steps was 0.87 and 0.92 for the descriptors of performance. Steps and descriptors of performance that >80% of experts rated as ≥4 were used to create the final LARS tool.

Conclusion

Multi‐institutional expert consensus was obtained regarding the steps and, for the first time, descriptors of performance for LA, demonstrating their face and content validity, as well as generalisability. Subsequently, the LARS tool was created that can be used to quantitatively assess intra‐operative performance. This instrument can be used to identify weaknesses in performance and facilitate deliberate practice, thus shifting training in LA to a competency‐based approach.

Are 2 Years Enough? Exploring Technical Skills Acquisition Among General Surgery Residents in Brazil

Phenomenon: Recent studies have shown that up to 40% of the General Surgery (GS) residents are not confident with their surgical skills. There is concern that residents are at risk of receiving inadequate training due to the low number of operations they perform. In Brazil, although all GS residents receive by law the Board Certification at the end of their programs, the assessment of their technical skills is not mandatory in Medical Residency programs' training. Consequently, our concern was that current GS medical residency format might be insufficient to create competent and autonomous general surgery residents after 2 years of regular training. Hence, the aim was to assess GS residents' surgical skills in their final months of training to evaluate the present format of GS residency programs in Brazil.

APPROACH

Trained surgical faculty members directly observed 11 operations of varying difficulty performed by 2nd-year regular GS residents and by 4th-year residents in the optional Advanced Program in General Surgery. Participants were located at 3 university and 3 nonuniversity hospitals in Rio de Janeiro and Sao Paulo (Brazil's largest cities). Surgical skills were assessed using an internally developed observation checklist reviewed by subject matter experts.

FINDINGS

Sixty residents (46 regular 2nd-year trainees and 14 advanced 4th-year trainees) were assessed on performing 499 operations. Only 10 residents (17%), all advanced 4th-year residents, satisfactorily performed all operations and were considered eligible for the Board Certification. Even after excluding the 2 operations of greatest difficulty, only 24 regular 2nd-year residents (52%) satisfactorily performed the other 9 operations. Residents from hospitals with open Emergency Departments performed better than those from hospitals without Emergency Departments. Insights: The results of this pilot study suggest that residents with 2 years of training are not prepared for independent high-level surgical practice. The current formatting of regular GS residency programs in Brazil (2 years) may be insufficient to train experts in general surgery.