Weighted kappa is higher than Cohen’s kappa for tridiagonal agreement tables

Definition and Classification of Intraoperative Complications (CLASSIC): Delphi Study and Pilot Evaluation

BACKGROUND

Standardized reporting of intraoperative adverse events is important to enhance transparency. To the best of our knowledge, there is no validated definition and classification of intraoperative complications.

METHODS

We conducted a two-round Delphi study to develop a definition and classification of intraoperative complications. Experts were contacted by email and sent a link to the online questionnaire. In a pilot study, two independent raters applied the definition and classification in a sample of 60 surgical interventions of low, intermediate, and high complexity and evaluated practicability. Interrater agreement of the classification was determined (raw categorical agreement, weighted kappa, and intraclass correlation).

RESULTS

In the Delphi study, 40 of 52 experts (77 % return rate) from 14 countries took part in each round. The Delphi study resulted in a comprehensive definition of intraoperative complications as any deviation from the ideal intraoperative course occurring between skin incision and skin closure. The classification foresees four grades depending on the need for treatment (no need, grade I; need for treatment, grade II) and the severity of the complication (life-threatening/permanent disability, grade III; death, grade IV). The pilot study showed good practicability (6 on a 7-point scale) and a high raw agreement of 87 %, a weighted kappa of 0.83 [95 % confidence interval (CI) 0.73-0.94] and an intraclass correlation coefficient of 0.83 (95 % CI 0.73-0.90).

CONCLUSIONS

While the Delphi process enabled to develop definitions and classification of intraoperative complications by severity, further research including a multicentre international full-scale validation needs to be conducted with the ultimate goal to contribute to standardized reporting in surgical practice and research.

Experienced versus Inexperienced Interexaminer Reliability on Location and Classification of Myofascial Trigger Point Palpation to Diagnose Lateral Epicondylalgia: An Observational Cross-Sectional Study

The purpose was to evaluate the interexaminer reliability of experienced and inexperienced examiners on location and classification of myofascial trigger points (MTrPs) in two epicondylar muscles and the association between the MTrP found and the diagnosis of lateral epicondylalgia (LE). Fifty-two pianists (some suffered LE) voluntarily participated in the study. Three physiotherapists (one inexperienced in myofascial pain) examined, located, and marked MTrPs in the extensor carpi radialis brevis (ECRB) and extensor digitorum communis (EDC) muscles. Forearms were photographed and analyzed to establish the degree of agreement on MTrPs diagnosis. Data showed 81.73% and 77.88% of agreement on MTrP classification and 85.58% and 72.12% on MTrP location between the expert evaluators for ECRB and EDC, respectively. The agreement on MTrP classification between experienced and inexperienced examiners was 54.81% and 51.92% for ECRB and 50.00% and 55.77% for EDC. Also, agreement on MTrP location was 54.81% and 60.58% for ECRB and 48.08% and 48.08% for EDC. A strong association was found between presence of relevant MTrPs, LE diagnosis, and forearm pain when the examiners were experts. The analysis of location and classification of MTrPs in the epicondylar muscles through physical examination by experienced evaluators is reliable, reproducible, and suitable for diagnosing LE.

Weighted Kappas for3×3Tables

Weighted kappa is a widely used statistic for summarizing inter-rater agreement on a categorical scale. For rating scales with three categories, there are seven versions of weighted kappa. It is shown analytically how these weighted kappas are related. Several conditional equalities and inequalities between the weighted kappas are derived. The analytical analysis indicates that the weighted kappas are measuring the same thing but to a different extent. One cannot, therefore, use the same magnitude guidelines for all weighted kappas.