Validity Evidence for Procedure-Specific Competency Assessment Tools in Cardiovascular and Thoracic Surgery: A Scoping Review

OBJECTIVE

Surgical education has shifted from a time-based approach to the achievement and demonstration of procedural competency. High quality, objective assessment instruments are required to support this new approach. This study comprehensively reviewed the literature to identify and evaluate available procedure-specific assessment instruments in cardiothoracic and vascular surgery.

DESIGN

A systematic search of 8 databases identified studies containing procedure-specific operative assessment instruments in cardiothoracic and vascular surgery. Generic global rating scales were excluded, unless modified to be procedure-specific. Two reviewers independently evaluated the validity evidence, methodological rigour and educational utility of each instrument using objective scoring criteria. Validity evidence was evaluated with a scoring tool aligned with the contemporary framework of validity. Methodological rigour was evaluated using the Medical Education Research Study Quality Instrument. Educational utility was evaluated according to the Accreditation Council for Graduate Medical Education (ACGME) framework.

RESULTS

There were 2130 unique studies describing procedure-specific assessment in surgery. Of these, 9 studies evaluating 8 procedure-specific assessment instruments met inclusion criteria for cardiothoracic and vascular surgery. Four instruments were identified in thoracic surgery, 2 in cardiac surgery, and 2 in vascular surgery. Only 1 instrument was designed to evaluate surgeon performance, with the remainder designed to evaluate residents. No single instrument scored the maximum score of 15 for validity evidence. The highest score was 11, with 62.5% (n = 5) of instruments scoring greater than 10. All tools attained high scores in content validity, with minimal evidence generally presented regarding the consequences of assessment using a particular instrument. All but 1 instrument scored greater than 11 out of a maximum 16.5 points for methodological rigour. Very few studies reported on the ACGME domains of educational utility.

CONCLUSIONS

In an era where surgical education is shifting towards the demonstration of procedural competency, objective procedure-specific assessment is critical. This review identified that few procedure-specific assessment instruments in cardiothoracic and vascular surgery exist, emphasizing the need for such instruments to ensure the success of competency-based education models.

A scoping review to identify competencies for transcatheter cardiovascular procedures

OBJECTIVES

Transcatheter procedures are increasingly being recognized as a priority for cardiac surgeons and cardiac surgery trainees. The optimal method of teaching these procedures during residency training has not been established. We used an evidence-based approach to systematically review the literature and identify competencies to inform future paradigms of transcatheter training in cardiac surgery.

METHODS

A scoping review was conducted to retrieve relevant literature on the performance of transcatheter cardiovascular procedures, identify competencies required by surgical residents learning to perform these procedures, and develop a preliminary list of competencies for consideration during transcatheter training. MEDLINE, Scopus, and ERIC were queried until April 1, 2020, using a systematic search strategy. No limitations were placed on publication date or type.

RESULTS

A total of 1456 sources of evidence were retrieved. After deduplication and screening, there remained 33 that were included in the scoping review, published between 2006 and 2020. The distribution of publication types included 10 comparative studies (30.3% of total), 8 societal statements (24.2% of total), 5 surveys and 5 opinion articles (each 15.2% of total), 2 editorials and 2 descriptions of a simulator (each 6.1% of total), and 1 narrative review (3.0% of total). From these, a total of 400 items were identified and organized into 97 competencies.

CONCLUSIONS

Evidence on the competencies required to perform transcatheter cardiovascular procedures is available from a variety of sources. The identified competencies may be a useful resource for developing curricula and teaching transcatheter procedures to cardiac surgery residents.

Association of earlier extubation and postoperative delirium after coronary artery bypass grafting

OBJECTIVES

Earlier extubation after cardiac surgery is reported to have benefits on length of stay and complication rates, but the influence on postoperative delirium remains unclear. We sought to determine the effect of earlier extubation on delirium after coronary artery bypass grafting.

METHODS

A single-center retrospective review of consecutive isolated coronary artery bypass grafting patients from January 1, 2010, to December 31, 2015, was conducted. Baseline demographic characteristics, preoperative comorbidities, intraoperative data, and postoperative data were collected. A multivariable logistic regression was performed with analysis limited to extubation within the first 24 hours postoperatively.

RESULTS

We identified 2561 eligible patients. Delirium occurred in 13.9% (n = 357). Duration of postoperative mechanical ventilation was associated with higher delirium rates following adjustment, particularly after 12 to 24 hours (hourly odds ratio, 1.12; 95% confidence interval, 1.05-1.19; P < .001). No association was observed during the time period from 0 to 12 hours (hourly odds ratio, 1.02; 95% confidence interval, 0.99-1.06; P = .218). Major adverse events were associated with duration of ventilation after 0 to 12 hours (hourly odds ratio, 1.08; 95% confidence interval, 1.03-1.14; P < .002) but not after 12 to 24 hours (hourly odds ratio, 1.04; 95% CI, 0.96-1.14; P = .316). The overall rate of reintubation was 2.9% (n = 73).

CONCLUSIONS

Our findings suggest that delirium rates increase with lengthier postoperative ventilation times. This study provides the basis for consideration of the appropriate selection of earlier extubation to minimize delirium in patients undergoing cardiac surgery.