Discrimination, Reliability, Sensitivity, and Specificity of Robotic Surgical Proficiency Assessment With Global Evaluative Assessment of Robotic Skills and Binary Scoring Metrics: Results From a Randomized Controlled Trial

Objective

To compare binary metrics and Global Evaluative Assessment of Robotic Skills (GEARS) evaluations of training outcome assessments for reliability, sensitivity, and specificity.

Background

GEARS-Likert-scale skills assessment are a widely accepted tool for robotic surgical training outcome evaluations. Proficiency-based progression (PBP) training is another methodology but uses binary performance metrics for evaluations.

Methods

In a prospective, randomized, and blinded study, we compared conventional with PBP training for a robotic suturing, knot-tying anastomosis task. Thirty-six surgical residents from 16 Belgium residency programs were randomized. In the skills laboratory, the PBP group trained until they demonstrated a quantitatively defined proficiency benchmark. The conventional group were yoked to the same training time but without the proficiency requirement. The final trial was video recorded and assessed with binary metrics and GEARS by robotic surgeons blinded to individual, group, and residency program. Sensitivity and specificity of the two assessment methods were evaluated with area under the curve (AUC) and receiver operating characteristics (ROC) curves.

Results

The PBP group made 42% fewer objectively assessed performance errors than the conventional group (P < 0.001) and scored 15% better on the GEARS assessment (P = 0.033). The mean interrater reliability for binary metrics and GEARS was 0.87 and 0.38, respectively. Binary total error metrics AUC was 97% and for GEARS 85%. With a sensitivity threshold of 0.8, false positives rates were 3% and 25% for, respectively, the binary and GEARS assessments.

Conclusions

Binary metrics for scoring a robotic VUA task demonstrated better psychometric properties than the GEARS assessment.

Laparoscopic nephroureterectomy for upper urinary tract transitional cell carcinoma: results of a Belgian retrospective multicentre survey

OBJECTIVES

To evaluate the technical and oncologic feasibility of laparoscopic nephroureterectomy for upper urinary tract transitional cell carcinoma.

METHODS

A retrospective survey of 100 patients, treated with laparoscopic nephroureterectomy in 10 Belgian centres, was performed. Most procedures were performed transperitoneally. The distal ureter was managed by open surgery in 55 patients and laparoscopically in 45 patients. The mean follow-up was 20 mo.

RESULTS

Mean operation time was 192 min and mean blood loss 234 ml. The conversion rate was 7%. Important postoperative complications were seen in 9%. Pathologic staging was pTa in 31 patients, pT1 in 23, pT2 in 12, pT3 in 33, and pT4 in 1, concomittant pTis in 3. Pathologic grade was G1 in 24 patients, G2 in 28, and G3 in 48. Negative surgical margins were obtained in all but one patient. Twenty-five patients developed progressive disease (24%) at a mean postoperative time of 9 mo (local recurrence in 8%, metastases in 11%, both in 5%). Progression was 0% for pTa, 17% for pT1, 17% for pT2, 51% for pT3, and 100% for pT4. Cancer-specific survival was 100% for pTa, 86% for pT1, 100% for pT2, 77% for pT3, and 0% for pT4.

CONCLUSION

Laparoscopic nephroureterectomy appears to be a technically and oncologically feasible operation. To prevent tumour seeding, one should avoid opening the urinary tract and should extract the specimen with an intact organ bag. The high local recurrence rate in this study probably reflects the high percentage of high-grade and high-stage tumours in this study.

Transurethral ultrasound-guided laser-induced prostatectomy: a critical evaluation

We evaluated 38 patients with a follow-up of 30 months after transurethral ultrasound-guided laser-induced prostatectomy (TULIP) for benign prostatic hyperplasia. The mean symptom score decreased by 54%, and peak urinary flow increased by 112%. For the entire series, 43.6% of the patients had an improved symptom score and 41% better urinary flow, but only 28.2% had improvement in both. Six patients (16%) required reoperation, two underwent a radical prostatectomy, and one patient presented total urinary incontinence. Also, 19% presented postoperative impotence, and 47% presented retrograde ejaculation. Although one third of the patients are improved with the TULIP procedure, the rate of complications is significantly higher than for TURP, which remains the most effective treatment of obstructive BPH.

[PSA and anatomopathological stage of prostatic cancer]

Accurate preoperative staging is important for adequate selection of patients for radical prostatectomy. We reviewed the preoperative PSA levels of 214 patients who underwent radical retropubic prostatectomy and analysed the results in relation to pathological stage. 32 (15%) patients had a PSA level below 4 ng/ml; nevertheless 8 of them had already a extracapsular disease and one present with positive lymph nodes. Thus, although a specificity of 90%, the sensitivity is only 18%. 98 (46%) patients had PSA levels beyond 10 ng/ml of which 47 (48%) had extracapsular tumor; this gives a specificity of 62% and a sensitivity of 59%. Finally 35 (16%) patients had a PSA over 20 ng/ml of which 18 (51%) present a extracapsular disease thus the specificity is 87% but the sensibility remains low at 23%. We concluded that although PSA levels are grossly related to the pathological stage, that relation is not reliable to distinguish patients with organ-confined from those who have extracapsular tumor extension and for either definition of a positive PSA level many patients would be denied a curative operation. Therefore, PSA value cannot be used to decide whether to recommend radical prostatectomy for potential cure.

Measurement of the vasoconstrictive substances endothelin, angiotensin II, and thromboxane B2 in cold storage solution can reveal previous renal ischemic insults

In a rat model, the left kidney was subjected to 60 min of normothermic ischemia followed by 15 min of reperfusion, whereas the right kidney, serving as a paired control, was not rendered ischemic. Both kidneys were then perfused in situ with either Euro-Collins (EC) solution (n = 12) or University of Wisconsin (UW) solution (n = 6) for 10 min. Each kidney was then harvested and stored at 4 degrees C in its respective solution. After 24 and 48 h of cold storage, the following vasoactive substances were measured in the preservation media: endothelin (ET), angiotensin II (A-II), thromboxane (B2) (TxB2), and prostaglandin I2 (PGI2). After 24 h in EC solution, left kidneys uniformly produced significantly higher concentrations of each vasoactive substance than right kidneys: ET 1.64 +/- 0.3 pg/ml vs 0.82 +/- 0.1 pg/ml (P < or = 0.009); A-II 20.8 +/- 6.2 pg/ml vs 7.75 + 2.3 pg/ml (P < or = 0.007); TxB2 100.8 +/- 17.7 pg/ml vs 40.1 +/- 11.7 pg/ml (P < or = 0.04); PGI2 638.3 +/- 41.1 pg/ml vs 318.3 +/- 36.4 pg/ml (P < or = 0.001), respectively. At 48 h, a similar pattern of results was obtained as the kidney continued to produce TxB2 and prostacyclins during the 24-48 h period. In the UW solution, basal levels of ET and A-II were lower than those in EC solution, but similarly increased after initial ischemia.(ABSTRACT TRUNCATED AT 250 WORDS)

Atrial natriuretic factor, arachidonic acid metabolites and acute renal ischemia: experimental protocol in the rat

The effects of an atrial natriuretic factor (ANF) infusion upon the production of the arachidonic acid metabolites (thromboxane B2, TxB2; 6-keto-prostaglandin F1 alpha, 6-keto-PGF1 alpha, PGI2, or prostaglandins E2, PGE2) were investigated after acute renal ischemia in the rat. This experimental protocol included a right nephrectomy and a 45-min left renal artery occlusion. Fifteen minutes after declamping, blood samples were collected from the left renal venous effluent for the assay of plasmatic prostanoid concentrations. Three experimental groups were studied: group I (n = 9) sham, no ischemia-group II (n = 9) control group, 45 min of left renal ischemia, followed by a 15-min revascularization, and group III (n = 10) ANF group, a similar ischemic protocol to that in group II was used but, after declamping, synthetic Atriopeptin III was infused (0.5 micrograms/kg/min) during the 15-min of vascular reflow. Fifteen minutes after declamping, TxB2 secretion significantly increased after ischemia in the control and ANF groups: TxB2: 210 +/- 22.4 pg/ml (control group) and 234.8 +/- 25.1 pg/ml (ANF group) versus 135.8 +/- 17.8 pg/ml (sham group) (p less than 0.05 and 0.01, respectively). On the other hand, the 6-keto-PGF1 alpha plasma levels were significantly higher after ischemia in the ANF group (221 +/- 34 pg/ml) in comparison with the sham (124 +/- 24.1 pg/ml) or with the control group (116.7 +/- 12.5 pg/ml). The calculated TxB2/6-keto-PGF1 alpha ratio was therefore higher in the control group, 1.93 +/- 0.27, than in physiological conditions (sham group), 1.2 +/- 0.17.(ABSTRACT TRUNCATED AT 250 WORDS)