An integrated software system for quality assurance-related kappa coefficient analysis of intraoperative transesophageal echocardiography interpretive skills

Transesophageal echocardiography interpretation: a comparative analysis between cardiac anesthesiologists and primary echocardiographers

Diagnostic interpretation of intraoperative transesophageal echocardiography (TEE) examinations may vary, particularly when the echocardiographer is also the anesthesiologist. We therefore evaluated the concordance of TEE interpretation as part of a process of continuous quality improvement (CQI). Ten cardiac anesthesiologists participating in a CQI program conducted 154 comprehensive TEE examinations, each consisting of 16 major fields describing cardiac anatomy and function. These examinations were subsequently interpreted off-line by two primary echocardiographers (a radiologist and a cardiologist). Agreement was assessed using the kappa coefficient and percent agreement. Overall kappa and percent agreement were 0.58 and 83% for anesthesiologists versus radiologist, 0.57 and 80% for anesthesiologists versus cardiologist, and 0.60 and 82% for radiologist versus cardiologist. Anesthesiologists with longer than 5 yr of TEE experience had higher levels of agreement with the radiologist when assessing the aorta, right atrium, pulmonary vein flow, transmitral flow, and fractional area change. Cardiac anesthesiologists supported by a CQI program interpret TEE examinations at a level comparable with physicians whose primary practice is echocardiography. Thus, the anesthesiologist and the intraoperative echocardiographer need not be mutually exclusive.

IMPLICATIONS

Interpretation of intraoperative transesophageal echocardiograms can be reliably performed by cardiac anesthesiologists.

Field-by-field evaluation of intraoperative transoesophageal echocardiography interpretative skills

A quality assurance system is essential for the credibility and structured growth of anaesthesiology-based transoesophageal echocardiography (TEE) programmes. We have developed software (Q/A Kappa), involving a 400-line source code, capable of directly reporting kappa correlation coefficient values, using external reviewer interpretations as the 'gold standard', and thereby allowing systematic assessment of the validity of intraoperative echocardiographic interpretation. This paper presents assessment of the validity of 240 intraoperative anaesthesiologists' echocardiographic interpretations, and, in addition, the results of field testing of this prototypical software. Data, derived from consecutive cardiac surgery patients, consisted of standardized two-dimensional transoesophageal echocardiographic, colour flow and Doppler imaging sequences. Intraoperative and off-line 'gold standard' TEE interpretations were compared for 19 fields or variables using the Q/A Kappa program. The kappa correlation coefficients were highly variable and dependent on the examination field, ranging from 0.08 for apical regional wall motion scores to 1.00 for tricuspid regurgitation grade, left atrial measurement, aortic valve anatomy and left ventricular long axis and short axis global function. The correlation coefficients were also operator dependent. These data (480 interpretations) were also manually integrated into the equation required for calculation of values of the variable kappa correlation coefficient. The relationship between Q/A Kappa-derived values and manually calculated values was highly significant (p < 0.001; r = 1.0). The implications and possible explanations of the results for particular examination fields are discussed. This study also demonstrates successful seamless functioning of this software program from data entry, segmentation into tables and valid statistical analysis. These findings suggest that it is practical to provide sophisticated continuous quality improvement TEE data on a routine basis.

Dobutamine stress echocardiography for the detection of significant coronary artery disease in renal transplant candidates

Prophylactic coronary revascularization may reduce the risk for cardiac events in diabetic renal transplant candidates. No published data exist on the accuracy of dobutamine stress echocardiography (DSE) for the diagnosis of angiographically defined coronary artery disease (CAD) in renal transplant candidates. The purpose of this study is to examine the accuracy of DSE for the detection of CAD in high-risk renal transplant candidates compared with coronary angiography. Fifty renal transplant candidates with diabetic nephropathy (39 patients) or end-stage renal disease (ESRD) from other causes (11 patients) underwent prospectively performed DSE, followed by quantitative coronary angiography (QCA) and qualitative visual assessment of CAD severity. Twenty of 50 DSE tests were positive for inducible ischemia. Twenty-seven patients (54%) had a stenosis of 50% or greater by QCA, 12 patients (24%) had a stenosis of greater than 70% by QCA, and 16 patients (32%) had a stenosis greater than 75% by visual estimation. The sensitivity and specificity of DSE for CAD diagnosis were respectively 52% and 74% compared with QCA stenosis of 50% or greater, 75% and 71% compared with QCA stenosis greater than 70%, and 75% and 76% for stenosis greater than 75% by visual estimate. On long-term follow-up (22.5 +/- 10.1 months), 6 of 30 patients (20%) with negative DSE results and 11 of 20 patients (55%) with positive DSE results had a cardiac death, myocardial infarction (MI), or coronary revascularization. Six of 27 patients (22%) with a QCA stenosis of 50% or greater had a cardiac death or MI compared with none of the 23 patients (0%) with QCA stenosis less than 50% (P = 0.025). We conclude that DSE is a useful but imperfect screening test for angiographically defined CAD in renal transplant candidates.