A novel assessment tool for evaluating competence in video-assisted thoracoscopic surgery lobectomy

Video-based tools for surgical quality assessment of technical skills in laparoscopic procedures: a systematic review

BACKGROUND

Quality of surgery has substantial impact on both short- and long-term clinical outcomes. This stresses the need for objective surgical quality assessment (SQA) for education, clinical practice and research purposes. The aim of this systematic review was to provide a comprehensive overview of all video-based objective SQA tools in laparoscopic procedures and their validity to objectively assess surgical performance.

METHODS

PubMed, Embase.com and Web of Science were systematically searched by two reviewers to identify all studies focusing on video-based SQA tools of technical skills in laparoscopic surgery performed in a clinical setting. Evidence on validity was evaluated using a modified validation scoring system.

RESULTS

Fifty-five studies with a total of 41 video-based SQA tools were identified. These tools were used in 9 different fields of laparoscopic surgery and were divided into 4 categories: the global assessment scale (GAS), the error-based assessment scale (EBAS), the procedure-specific assessment tool (PSAT) and artificial intelligence (AI). The number of studies focusing on these four categories were 21, 6, 31 and 3, respectively. Twelve studies validated the SQA tool with clinical outcomes. In 11 of those studies, a positive association between surgical quality and clinical outcomes was found.

CONCLUSION

This systematic review included a total of 41 unique video-based SQA tools to assess surgical technical skills in various domains of laparoscopic surgery. This study suggests that validated SQA tools enable objective assessment of surgical performance with relevance for clinical outcomes, which can be used for training, research and quality improvement programs.

Development of a formative feedback tool for transanal total mesorectal excision

INTRODUCTION

Transanal total mesorectal excision (TaTME) is a novel procedure in the treatment of rectal cancer. Current training models for TaTME suggest a period of proctored cases, but no structured feedback tool exists to guide operators during the learning phase. The objective of this study therefore was to develop a formative feedback tool for the critical steps of the transanal portion of TaTME.

METHODS

A two-round Delphi study by TaTME experts was conducted to determine the items to be included in the formative feedback tool. Participants rated each step from a prepared list using a Likert scale from 1 (Not relevant) to 5 (Very relevant) with the option to suggest additional steps. Responses to the first round were presented in the second round, where participants rated the revised list of steps. Consensus was defined as > 80% of participants rating the step as 4 or 5 (out of 5). Items were combined when appropriate to avoid redundancy. Rating anchors describing performance (on a 5-point scale) were then developed for each step. The final tool was recirculated and participants rated the finished product on its feasibility and usefulness.

RESULTS

Twenty-six TaTME experts were contacted for participation. Fifteen experts (58%) participated in the first round of the study, and eleven (42%) participated in the second round. The majority (14, 93%) had completed fellowship training in colorectal surgery. The first round of the Delphi study contained 34 items, and 32 items met inclusion criteria after the second round. Redundant items were combined into 15 items that comprised the final tool. Out of eight respondents to the feasibility survey, all believed the feedback tool enhances the feedback of learners and would use it for training purposes if available.

CONCLUSION

This work describes the development of a novel consensus-based formative feedback tool specific to TaTME.

Auditorium of the future: e learning platform

Principles of surgical training have not changed, but methods of training are evolving very fast. Online tools are being adopted in both knowledge and skills training for surgical residents. As a result, to evaluate the outcome of these tools, online assessment is also developing. Knowledge resources are very diverse ranging from lectures, webinars, surgical videos to three-dimensional planning and printing. Skills resources include virtual reality simulators, remote skills training and interdisciplinary teamwork. Assessment of E-learning tools can be performed using online questions, task-based simulations, branching scenarios and online interviews/discussions. In thoracic surgery, video assisted thoracic surgery (VATS) lobectomy simulator has been developed and it appears to be an important tool for minimally invasive thoracic surgery education. Training programs incorporate e-Learning in their curriculum and online training and assessment will become an important part of thoracic surgical training as well.

Structured and evidence-based training of technical skills in respiratory medicine and thoracic surgery

Within the last decade, the number of technical procedures in respiratory medicine and thoracic surgery has grown and created a higher demand for effective and evidence-based education. Today, trainees are often allowed to perform procedures unsupervised on patients after obtaining a course certificate from a theoretic course and having performed a fixed number of supervised procedures. However, these methods do not ensure adequate competence. Well-structured and effective educational programmes including validated tests are needed to reduce economic expenses, optimize time spend, and ensure patient safety. The aim of this article is to summarize current state of educational strategies for technical procedures within respiratory medicine and thoracic surgery. Thus, to discuss future recommendations for curriculum development and assessment of competences based on Kern’s framework. The approach by Kern consists of six topics, which needs to be considered and evaluated and in order to educate physicians and surgeons most effective and evidence-based. We present a practical guide contributing to future educators’ considerations on (I) problem identification and general needs assessment, (II) targeted needs assessment, (III) goals and objectives, (IV) educational strategies, (V) implementation, and finally (VI) evaluation and feedback.

Quality assessment of robot assisted thoracic surgical resection of non-small cell lung cancer: nodal upstaging and mediastinal recurrence

Background

Robot assisted thoracic surgery (RATS) is the minimally invasive surgical technique of choice for treatment of patients with non-small cell lung cancer (NSCLC), at the Isala Hospital. The aim of this study is to compare clinical and pathological staging results and mediastinal recurrence after RATS for anatomical resections of lung cancer as surrogate markers for quality of mediastinal lymph node dissection (MLND).

Methods

This single institute retrospective study was conducted in patients who underwent RATS for NSCLC. Excluded were patients with a history of concurrent malignant disease, with other previous neoplasms, with small cell lung cancer (SCLC) and patients in whom the robotic technique was converted to thoracotomy, prior to lymph node dissection. Data were obtained from the hospital database. The difference between clinical and pathological staging was expressed as upstaging and downstaging. Computed Tomography scanning was used for follow-up, and diagnosis of mediastinal recurrence.

Results

From November 2011 to May 2016, 227 patients underwent RATS at Isala Hospital Zwolle, the Netherlands. Of those, 130 (mean age, 69.5±9.3 years) met the eligibility criteria. Preoperative mediastinal lymph node staging was done by endoscopic ultrasound/endobronchial ultrasound, by positron emission tomography (PET) or mediastinoscopy. In 14 patients (10.8%) unforeseen N2 disease was found, 6 patients (4.6%) were upstaged from cN0 to pN2 and 8 patients (6.2%) were upstaged from cN1 to pN2. Mediastinal recurrence was detected in 7 patients (5.4%) during a median follow-up of 54 months (range, 1.5-102 months).

Conclusions

In patients with NSCLC, who underwent anatomical resection by means of RATS, an unforeseen N2 disease rate of 10.8% was demonstrated and a mediastinal recurrence rate of 5.4%. It is concluded that robotic surgery provides an accurate lymph node dissection.

Evaluation of competence in ultrasound-guided procedures-a generic assessment tool developed through the Delphi method

OBJECTIVES

To develop a generic and objective tool for assessing competence in percutaneous ultrasound-guided procedures.

METHODS

Interventional ultrasound experts from the Nordic countries were invited to participate in a three-round Delphi process. A steering committee was formed to manage the process. In round 1, the experts were asked to suggest all aspects to consider when assessing competence in US-guided procedures. Suggestions were analyzed and condensed into assessment items. In round 2, the expert panel rated these items on a 1-5 scale and suggested new items. Items with a mean rating of ≤ 3.5 were excluded. In round 3, the expert panel rated the list items and suggested changes to the items.

RESULTS

Twenty-five experts were invited, and response rates in the three rounds were 68% (17 out of 25), 100% (17 out of 17), and 100% (17 out of 17). The three-round Delphi process resulted in a 12-item assessment tool, using a five-point rating scale. The final assessment tool evaluates pre-procedural planning, US technique, procedural technique, patient safety, communication, and teamwork.

CONCLUSIONS

Expert consensus was achieved on a generic tool for assessment of competence in percutaneous ultrasound-guided procedures-the Interventional Ultrasound Skills Evaluation (IUSE). This is the initial step in ensuring a valid and reliable method for assessment of interventional US skill.

KEY POINTS

• Through a Delphi process, expert consensus was achieved on the content of an assessment tool for percutaneous ultrasound-guided procedures-the Interventional Ultrasound Skills Evaluation (IUSE) tool. • The IUSE tool is comprehensive and covers pre-procedural planning, US technique, procedural technique, patient safety, communication, and teamwork. • This is an important step in ensuring valid and reliable assessment of interventional US skills.

Competency Assessment for Mediastinal Mass Resection and Thymectomy: Design and Delphi Review Process

OBJECTIVES

Mediastinal mass resection and thymectomy are complex and related operations that are core components of competency for a general thoracic surgeon and an important learning objective for thoracic surgery trainees. This study aimed to design a combined competency assessment instrument for mediastinal mass resection and thymectomy.

DESIGN

A comprehensive competency assessment instrument was designed by a process of logical analysis by 3 expert thoracic surgeons with an interest in mediastinal surgery. The instrument was then assessed and refined using a modified Delphi process.

SETTING

The Delphi questionnaire was distributed to all members of the Canadian Association of Thoracic Surgeons in 2018 to 2019.

PARTICIPANTS

The first round of the Delphi review was completed by 58 respondents (response rate 43.9%). Respondents represented all Canadian provinces with a wide range of clinical experience and a high rate of involvement in resident education.

RESULTS

A first draft of the competency assessment instrument included 42 steps in 6 categories. A total of 3 rounds of Delphi review were performed. Cronbach's alpha for the final round was 0.83. Ultimately, 29 items were retained from the original instrument and two modified and three new items were added. The final instrument has 34 steps in 5 categories.

CONCLUSIONS

A nationwide consensus was established on the key components of assessing competence to perform mediastinal mass resection and thymectomy. The resulting instrument could be used to guide competency based assessments of thoracic surgeons and trainees.

Naples Prognostic Score as a novel prognostic prediction tool in video-assisted thoracoscopic surgery for early-stage lung cancer: a propensity score matching study

BACKGROUNDS

The Naples Prognostic Score (NPS) emerges as a novel prognostic scoring system in surgical oncology. We aim to assess the prognostic significance of preoperative NPS in patients undergoing completely video-assisted thoracoscopic surgery (VATS) lobectomy for early-stage non-small cell lung cancer (NSCLC) by propensity score matching (PSM) analysis.

METHODS

The present study was conducted on our single-center prospectively maintained database between January 2014 and December 2015. A Kaplan-Meier survival analysis using the log-rank test was used to distinguish differences in both overall survival (OS) and disease-free survival (DFS) between the patients stratified by preoperative NPS. Finally, multivariable Cox-proportional hazards regression analysis and PSM analysis were carried out to determine the independent prognostic factors for both OS and DFS.

RESULTS

There were 457 patients with operable primary stage I-II NSCLC included. Per 1-point increase in NPS was found to be significantly associated with unfavorable OS and DFS of NSCLC. Both OS and DFS were significantly shortened along with each number increase in the NPS group, showing a step-wise fashion. Such strong correlations between preoperative NPS and survival outcomes still remained validated after PSM analysis. In addition, NPS held the best discriminatory power for predicting both OS and DFS when compared to the other peripheral biomarkers. Multivariable analyses on the entire cohort and the PSM cohort demonstrated that preoperative NPS could be an independent prognostic indicator for both OS and DFS.

CONCLUSIONS

The NPS scoring system can serve as a novel risk stratification tool to refine prognostic prediction after VATS lobectomy for surgically resected NSCLC.

Developing training for uniportal video-assisted thoracic surgery: a commentary

OBJECTIVES

Today, thoracic surgeons have many possible pathways to advance their training in video-assisted thoracic surgery (VATS). For uniportal VATS (uVATS) in particular, what are the training options available and does regulating training matter?

METHODS

The relevant literature pertaining to uVATS training is reviewed.

RESULTS

Current options for surgeons looking to train in uVATS range from videos on the internet, through 'experts' demonstrating live surgery, to symposia and hands-on wet labs being held in all regions of the world. All have merits, and the aspiring VATS surgeon can choose from a range of preceptorship and proctorship modules to suit his/her own training needs. However, issues in the formalizing training in uVATS remains unresolved. Is there an ideal pathway through uVATS training? At what point can a trainee be considered to have 'graduated' to become an expert in uVATS? Who indeed trains the trainer, and certifies the competence of a training centre?

CONCLUSIONS

Although a plethora of training options exist, formulation of a robust training curriculum can further bolster the status of uVATS as a sustainable surgical approach that can be delivered with consistent quality.

Video-Based Assessment in Surgical Education: A Scoping Review

BACKGROUND AND OBJECTIVE

Video-based assessment of residents' surgical skills may offer several advantages over direct observations of clinical performance in terms of objectivity, time-efficiency, and feasibility. Although video-based assessment is becoming more common in surgical training, a broad understanding of its utility is lacking. This scoping review explores video-based assessment in surgical training and presents the evidence supporting its use.

DESIGN

A literature search was conducted using the Web of Science database with key words related to video-based assessment and surgical training. Exclusion criteria included articles not published in English and articles on undergraduate medical education, continuing professional development, or non-surgical disciplines. Initially, 702 articles were identified; after title, abstract, and full-text screening by two independent reviewers (SM and VM), 199 articles remained.

RESULTS

We present the benefits of video-based assessment, including the ability to capture clinical ability in the operating room without decreasing intraoperative efficiency, as well as the potential to improve formative assessment and feedback practices. We describe the validity, reliability, and challenges of video-based assessment, as well as the use of video-based methods in clinical and simulated settings. We conclude by discussing questions that remain to be addressed.

CONCLUSIONS

Although further research and cost-benefit analyses are required, greater adoption of video-based assessment into surgical training may help meet increased assessment demands in an era of competency-based medical education.

Methodology and timing of standardization

In the last decade, uniportal VATS has increasingly been used for major lung resections as it has revolutionised the way thoracic surgeons treat pulmonary lesions. Uniportal VATS has represented an authentic innovation in thoracic surgery because it represents an ideal link between reduced morbidity and enhanced recovery. From the first feasibility studies, Uniportal VATS has been assessed in terms of postoperative morbidity, mortality, and overall oncologic value yielding similar outcomes compared to open thoracotomy or three-port VATS. The Uniportal VATS Interest Group (UVIG) is a working group inside the European Society of Thoracic Surgeons (ESTS) with a particular interest in Uniportal VATS as the preferred approach to intrathoracic conditions amenable to surgical diagnosis and treatment. The first aim of UVIG was to define a consensus statement on the indications and techniques of uniportal VATS. The first consensus paper among international experts in uniportal VATS lobectomy was obtained through a Delphi process through three rounds of voting to enhance its validity. The Delphi process was used to define the procedure and to optimise the indications, perioperative management and training of Uniportal VATS. The main features of this process including the anonymity of participants, the iterative process, and, controlled feedback were reported in the consensus statement.

Obtaining Meaningful Assessment in Thoracic Surgery Education

Training in thoracic surgery has evolved immensely over the past decade due to the advent of integrated programs, technological innovations, and regulations on resident duty hours, decreasing the time trainees have to learn. These changes have made assessment of thoracic surgical trainees even more important. Shifts in medical education have increasingly emphasized competency, which has led to novel competency-based assessment tools for clinical and operative assessment. These novel tools take advantage of simulation and modern technology to provide more frequent and comprehensive assessment of the surgical trainee to ensure competence.

Competence and training in interventional pulmonology

Interventional pulmonology (IP) is experiencing a rapid evolution of new technologies. There is a need to develop structured training programs, organized in high volume expert centers in order to improve trainee education, and including the development of validated metrics for their competency assessment. Concerning teaching methods, a gradual progression from theory to practice, using new teaching techniques, including live sessions and low and high-fidelity simulation, flipped classroom models and problem-based learning (PBL) exercises would provide a training setting more suitable for our current need to improve skills and update professionals. Training programs should be learner-centered and competence-oriented, as well as being based on a spiral-shaped approach in which the same subject is addressed many times, from new and different perspectives of knowledge, ability, behavior and attitude, until the trainee has demonstrated a high degree of skill and professionalism. Furthermore there is a need to standardize the training programs as guide for physicians wishing to undertake a gradual and voluntary improvement of their own competencies, and assist those planning and organizing training programs in IP. The article includes a general part on core curriculum contents, innovative training methods and simulation, and introduces the following articles on the skills that the Interventional Pulmonologist must master in order to perform the different procedures. This monography should be considered a starting point that will evolve over time and results in better training for practitioners and better care for our patients. The task of establishing a trainee's competence to practice independently as an Interventional Pulmonologist remains the responsibility of the IP fellowship program director and faculty, who validate logbooks and assess competence for each procedure. These standards need to be reviewed and approved by national and International Scientific Societies and Healthcare Institutions with the aim to improve, disseminate and incorporate them in healthcare programs.

Competence in pleural procedures

Diseases of the pleura and pleural space are common and present a significant contribution to the workload of respiratory physicians, with most cases resulting from congestive heart failure, pneumonia, and cancer. Although the radiographic and ultrasonographic detection of pleural abnormalities may be obvious, the determination of a specific diagnosis can often represent a challenge. Invasive procedures such as pleural drainage, ultrasound/CT-guided pleural biopsy or medical thoracoscopy can be useful in determining specific diagnosis of pleural diseases. Management of primary and secondary spontaneous pneumothorax is mandatory in an interventional pulmonology training program, while the medical or surgical treatment of the recurrence is still a matter of discussion. Pleural drainage is a diagnostic and therapeutic procedure used in the treatment of pneumothorax and pleural effusions of different etiologies and even in palliation of symptomatic in malignant pleural effusion. Medical thoracoscopy (MT) is a minimally invasive procedure aimed at inspecting the pleural space. It could be a diagnostic procedure in pleural effusions (suspected malignant pleural effusion, infective pleural disease such as empyema or tuberculosis) or therapeutic procedure (chemical pleurodesis or opening of loculation in empyema). Diagnostic yield is 95% in patients with pleural malignancies and higher in pleural tuberculosis. In parapneumonic complex effusion, MT obviates the need for surgery in most cases. Thoracoscopy training should be considered being as important as bronchoscopy training for interventional pulmonology, although prior acquisition of ultrasonography and chest tube insertion skills is essential.

Evaluating competency in video-assisted thoracoscopic surgery (VATS) lobectomy performance using a novel assessment tool and virtual reality simulation

BACKGROUND

Competency-based training has gained ground in surgical training and with it assessment tools to ensure that training objectives are met. Very few assessment tools are available for evaluating performance in thoracoscopic procedures. Video recordings would provide the possibility of blinded assessment and limited rater bias. This study aimed to provide validity evidence for a newly developed and dedicated tool for assessing competency in Video-Assisted Thoracoscopic Surgery (VATS) lobectomy.

METHODS

Participants with varying experience with VATS lobectomy were included from different countries. Video recordings from participants' performance of a VATS right upper lobe lobectomy on a virtual reality simulator were rated by three raters using a modified version of a newly developed VATS lobectomy assessment tool (the VATSAT) and analyzed in relation to the unitary framework (content, response process, internal structure, relation to other variables, and consequences of testing).

RESULTS

Fifty-three participants performed two consecutive simulated VATS lobectomies on the virtual reality simulator, leaving a total of 106 videos. Content established in previously published studies. Response process Standardized data collection was ensured by using an instructional element, uniform data collection, a special rating program, and automatic generation of the results to a database. Raters were carefully instructed in using the VATSAT, and tryout ratings were carried out. Internal structure Inter-rater reliability was calculated as intra-class correlation coefficients, to 0.91 for average measures (p < 0.001). Test/re-test reliability was calculated as Pearson's r of 0.70 (p < 0.001). G-coefficient was calculated to be 0.79 with two procedures and three raters. By performing D-theory was found that either three procedures rated by two raters or five procedures rated by one rater were enough to reach an acceptable G-coefficient of ≥ 0.8. Relation to other variables Significant differences between groups were found (p < 0.001). The participants' VATS lobectomy experience correlated significantly to their VATSAT score (p = 0.016). Consequences of testing The pass/fail score was found to be 14.9 points by the contrasting groups' method, leaving five false positive (29%) and six false negatives (43%).

CONCLUSION

Validity evidence was provided for the VATSAT according to the unitary framework. The VATSAT provides supervisors and assessors with a procedure-specific assessment tool for evaluating VATS lobectomy performance and aids with the decision of when the trainee is ready for unsupervised performance.