The importance of surgeons' experience on intraobserver and interobserver reliability of classifications used for Perthes disease

Validation of the height-width measurement in Perthes disease among interpreters of different experience levels

The epiphyseal height and metaphyseal width ratio estimated lateral pillar involvement with limited generalizability among various interpreters. The aim of the study was to evaluate the reliability and the validity of height-width ratio measurement in Perthes disease among interpreters of different experience levels. A cross-sectional study was conducted between 2019 and 2020. We included four groups of interpreters: orthopaedic residents, orthopaedic fellows, radiology residents and radiology fellows who were unaware of radiographic study materials. Each interpreter blindedly evaluated Perthes hip radiographs twice at 1-month intervals using the height-width ratio method. Patients' and interpreters' characteristics, height-width ratio and height-width ratio converted to lateral pillar classification (A, B and C) were collected. Intra- and interobserver reliability validated with the paediatric orthopaedist were estimated. Twenty-four interpreters assessed 18 Perthes radiographs. Intraobserver level of agreement (95% confidence interval) for height-width ratio was 0.022 (-0.017 to 0.062), -0.027 (-0.074 to 0.019), -0.010 (-0.095 to 0.074) and 0.019 (-0.109 to 0.146); and interobserver reliability compared with the paediatric orthopaedist was -0.007 (-0.091 to 0.077), 0.003 (-0.056 to 0.061), -0.021 (-0.077 to 0.035) and -0.002 (-0.090 to 0.086) for orthopaedic residents and fellows, radiology residents and fellows, respectively. Kappa statistics of height-width ratio converting to lateral pillar classification indicated intraobserver agreement of orthopaedic residents and fellows, radiology residents and fellows was 0.83, 0.75, 0.54 and 0.91; and interobserver agreement compared with the paediatric orthopaedist was 0.92, 0.83, 0.42 and 0.83, orderly. Height-width ratio and lateral pillar estimation by orthopaedic and radiology trainees have moderate to excellent reliability. Level of evidence: Level II - diagnostic study.

The Effect of Clinical Knowledge and Surgical Experience on Treatment Choice in Legg-Calve-Perthes Disease: Intra-observer and Inter-observer Reliability Study

AIM

The aim of this study was to evaluate standardized hip radiographs, arthographs, demographic characteristics, physical examination findings, and their effects on treatment choices in leg-calve-perthes disease (LCPD). Intraobserver and interobserver realibility between orthopaedic residents, orthopaedic surgeons, and paediatric orthopaedic surgeons were also investigated.

MATERIALS AND METHODS

47 LCPD patients were included this cross-sectional study. Six separate presentations including different variabilities (clinical findings, standard radiographs, and arthrographs) were evaluated by three different groups (residents, surgeons, paediatric orthopaedic surgeons) and were sent to the observers every other month by hiding patients' personal information. Seven different treatment modalities were introduced for the best treatment modality. Intraobserver and interobserver reliability in these three groups were examined. Percentage aggreement (PA) and intraclass correlation coefficients (ICC) tests were used for this purpose.

RESULTS

Treatment PA rates between presentations were 29.5-53.6% in residents, 38.3-60.4% in surgeons, and 39.1-59.8% in pediatric orthopaedic surgeons. Conservative methods were mostly preferred as treatment modality in all groups; followed by proximal femoral osteotomies. Pediatric orthopaedic surgeons preferred safe dislocation and femur head and/or neck reconstruction surgery 5-18 times more than residents and orthopaedic surgeons. Intraobserver reliability of treatment modalities was higher among the paediatric orthopaedic surgeons; the results were fair-good (0.483-0.763). Among residents and orthopaedic surgeons, ICC values were poor to good, respectively (- 0.080 to 0.636 and 0.263-0.643). Interobserver reliability among three groups was meanly good.

CONCLUSION

As surgical experience increases, both the compliance rates of the treatment modalities and the intra- and inter-group reliability are increased. Knowledge of the demographic data and clinical findings of patients besides hip radiographs or arthrographs increase treatment compliance in paediatric orthopaedic surgeons, however, cause changes in treatment modalities in residents. As surgical experience increases, more difficult surgeries such as safe dislocation and femoral head/neck reconstruction are preferred.

Intra- and inter-observer reliability of Laredo classification system in Legg-Calvé-Perthes Disease

The purpose of the current study was to investigate intra- and inter-observer reliability of arthrographic Laredo classification system in Perthes disease. Forty-seven patients were included in this cross-sectional descriptive study. Patients' age, gender, physical findings and hip arthrographs were collected from hospital medical records. Two different sets of power point slides were prepared in which the order of cases was randomized and blinded. Observers were divided into three groups according to their surgical experience (9 residents, 10 seniors, 10 pediatric orthopedists), and they assessed two times 1 month's intervals. Statistical analysis was performed by using the SPSS v21. Inter- and intra-observer reliabilities were calculated using intra-class correlation coefficient, weighted kappa and percentage agreement. Percentage agreement of Laredo classification was about 50% for all groups (residents, seniors and pediatric orthopedists); intra-observer reliabilities were excellent, excellent and fair, respectively. Inter-observer reliability of Laredo classification for each set was found to be excellent for all groups. Length of experience did not correlated significantly with the level of intra-observer agreement. As a conclusion, our results showed that Laredo's arthrographic classification system's intra-observer reliability is at least at a fair level and inter-observer reliability is at an excellent level. We believe that this classification system is valuable for an orthopedic surgeon who deals with the treatment of Perthes disease.

Does the META score evaluating osteoporotic and metastatic vertebral fractures have enough agreement to be used by orthopaedic surgeons with different levels of training?

PURPOSE

Differentiating osteoporotic vertebral fractures (OVF) from metastatic vertebral fractures (MVF) is difficult. A magnetic resonance imaging (MRI)-based score (META score) aiming to differentiate OVF and MVF was recently published; however, an independent agreement assessment is required before the score is used. We performed such independent agreement evaluation, including raters with different levels of training.

METHODS

Sixty-four patients with confirmed OVF or MVF were evaluated by six raters (three spine surgeons and three orthopaedic residents) using the META score. We used the intra-class correlation coefficient (ICC) to evaluate inter- and intra-observer agreement and the kappa statistic (κ) to determine the agreement for individual score criteria. We calculated the area under the receiver-operating characteristic curve (AUC) to establish the score accuracy.

RESULTS

The inter-observer agreement was poor [ICC = 0.22 (0.12-0.33)]; spine surgeons [ICC = 0.75 (0.66-0.83)] had better agreement than that of residents [ICC = 0.06 (- 0.07 to 0.23)]. The intra-observer agreement was poor [ICC = 0.15 (- 0.04 to 0.30)]; both spine surgeons [ICC = 0.21 (0.05-0.41)] and residents exhibited poor agreement [ICC = - 0.06 (- 0.40 to 0.20)]. The agreement for each specific criterion varied from κ = 0.24 to κ = 0.38. The AUC was 0.57 (0.64 for spine surgeons and 0.51 for residents, p < 0.01).

CONCLUSION

The inter-observer agreement using the META score was adequate for spine surgeons but not for residents; the intra-observer agreement was poor. These results do not support the standard use of the META score to differentiate OVF and MVF. These slides can be retrieved under Electronic Supplementary Material.

Radiographic classifications in Perthes disease

Background and purpose - Different radiographic classifications have been proposed for prediction of outcome in Perthes disease. We assessed whether the modified lateral pillar classification would provide more reliable interobserver agreement and prognostic value compared with the original lateral pillar classification and the Catterall classification. Patients and methods - 42 patients (38 boys) with Perthes disease were included in the interobserver study. Their mean age at diagnosis was 6.5 (3-11) years. 5 observers classified the radiographs in 2 separate sessions according to the Catterall classification, the original and the modified lateral pillar classifications. Interobserver agreement was analysed using weighted kappa statistics. We assessed the associations between the classifications and femoral head sphericity at 5-year follow-up in 37 non-operatively treated patients in a crosstable analysis (Gamma statistics for ordinal variables, γ). Results - The original lateral pillar and Catterall classifications showed moderate interobserver agreement (kappa 0.49 and 0.43, respectively) while the modified lateral pillar classification had fair agreement (kappa 0.40). The original lateral pillar classification was strongly associated with the 5-year radiographic outcome, with a mean γ correlation coefficient of 0.75 (95% CI: 0.61-0.95) among the 5 observers. The modified lateral pillar and Catterall classifications showed moderate associations (mean γ correlation coefficient 0.55 [95% CI: 0.38-0.66] and 0.64 [95% CI: 0.57-0.72], respectively). Interpretation - The Catterall classification and the original lateral pillar classification had sufficient interobserver agreement and association to late radiographic outcome to be suitable for clinical use. Adding the borderline B/C group did not increase the interobserver agreement or prognostic value of the original lateral pillar classification.

The Long-Term Outcome After Varus Derotational Osteotomy for Legg-Calvé-Perthes Disease: A Mean Follow-up of 42 Years

BACKGROUND

Varus derotational osteotomy (VDRO) is one of the most common surgical treatments for Legg-Calvé-Perthes disease, yet its long-term results have not been fully assessed. We aimed to determine the long-term clinical and radiographic outcomes following VDRO.

METHODS

Forty patients (43 hips) who underwent VDRO for Legg-Calvé-Perthes disease at our institution from 1959 to 1983, and participated in a follow-up study completed 10 years earlier, were approached for the present study. Clinical examination and radiographs were evaluated. Hip status and well-being were assessed with the Harris hip score and the Short Form-36 (SF-36).

RESULTS

Thirty-five patients (37 hips) participated in the study. Information regarding the need for an arthroplasty was gathered on 4 additional hips from the previous study. The mean follow-up was 42.5 years (range, 32.4 to 56.5 years), with a mean patient age of 50.2 years (range, 35.9 to 67.8 years). In total, 7 patients (7 hips; 17% of 41 hips for which information was available, including 1 hip from the original cohort of 40 patients [43 hips]), underwent a total hip arthroplasty for hip pain. Excluding patients who had undergone an arthroplasty, the mean Harris hip and SF-36 scores were 79.8 points (range, 23.1 to 100 points) and 74.8 (range, 15.1 to 100), respectively. Twenty (64.5%) of the 31 hips that had not been replaced achieved a good or excellent Harris hip score (≥80 points). Sixteen (57.1%) of 28 hips with follow-up radiographs had no, or minimal, signs of osteoarthritis. The Stulberg classification was associated with the Harris hip score, the SF-36 score, hip pain, a Trendelenburg sign, coxa magna, and the Tönnis grade. In a multivariate analysis, the Stulberg classification was the only factor associated with fair or poor outcomes (a Harris hip score of <80 points). Patients with a Stulberg class-III or IV hip had significant deterioration with respect to the Harris hip score and Tönnis grade during the 10-year period since the last follow-up.

CONCLUSIONS

A long-term follow-up of patients who were operatively treated for Legg-Calvé-Perthes disease revealed that a low proportion underwent total hip arthroplasty and a relatively high proportion maintained good clinical and radiographic outcomes.

LEVEL OF EVIDENCE

Therapeutic Level IV. See Instructions for Authors for a complete description of levels of evidence.

Intra and interobserver concordance between the different classifications used in Legg-Calvé-Perthes disease

Objective

The aim of this study was to determine the intra and interobserver concordance rates of the Waldenström, Catterall and Herring classifications for Legg–Calvé–Perthes disease.

Methods

One hundred radiographs of the pelvis in anteroposterior and Lauenstein views, from patients with this disease, were selected. The radiographs were classified by four physicians with different levels of experience who had previously been given guidance regarding the classifications used, in order to minimize any bias of interpretation. The radiographs were examined by the same observers at two different times in order to evaluate the intra and interobserver concordance. Reproducibility was assessed using the kappa index.

Results

The concordance analysis was stratified into levels (poor, slight, fair, moderate, good and excellent). The intraobserver analysis showed, for the Waldenström classification, moderate concordance for three examiners and fair for one; for Herring, excellent for one examiner and good for three; and for Catterall, good for all the examiners. The interobserver analysis showed: for the three classification systems, no situations of excellent concordance; for Waldenström, four situations of fair concordance, one moderate and one slight; for Herring, four situations of moderate concordance, one good and one fair; and for Catterall, four situations of moderate concordance and two fair.

Conclusion

The classifications studied are the ones most used for guiding the treatment for Legg–Calvé–Perthes disease, but the degree of intra and interobserver concordance is far from ideal. Complementary staging systems need to be taken into consideration, so that there can be greater certainty regarding the treatment.

Inter-observer and intra-observer reliability of mechanical axis alignment before and after total knee arthroplasty using long leg radiographs

BACKGROUND

Long leg radiographs (LLRs) are commonly performed for assessment of mechanical alignment and operative planning in patients undergoing total knee arthroplasty (TKA). The aim of this study was to determine the inter- and intra-observer reliability of alignment measured by observers of different levels of experience.

METHODS

Forty patients on the waiting list for a TKA had pre- and post-operative standardised LLRs. We analysed the measurements of mechanical axis alignment between an orthopaedic surgeon, a senior orthopaedic registrar, a junior orthopaedic registrar, and a medical student. Reviewers performed blinded measurements on the same computer screen. These measurements were repeated three months later to assess intra-observer reliability. Furthermore high-resolution screens were compared with standard hospital computer screens to investigate whether monitoring quality influenced the accuracy of measurements of alignment.

RESULTS

Inter-observer reliability was high for pre-operative LLRs with an intra-class correlation (ICC) of >0.9 at all experience levels. Post-operative ICC was lowest between the surgeon and the medical student at 0.7. Intra-observer reliability was high at all experience levels. Larger deformities appeared to have exaggerated measurements for both pre- and post-operative images. There appeared to be no effect of the monitor size and quality on the accuracy of measurement.

CONCLUSIONS

Long leg radiographs can be used to measure mechanical axis alignment with strong reliability across different levels of experience. This information is important for the evaluation of knee alignment measurements in current clinical practice, to assess severity of deformity and to accompany pre-operative planning and post-operative evaluation.

LEVEL OF EVIDENCE

III.

Reliability and stability of three common classifications for Legg-Calvé-Perthes disease

BACKGROUND

To predict the course of Legg-Calvé-Perthes disease (LCPD) and select between treatment options in the early stages, it is critical to have a reliable predictive classification.

QUESTIONS/PURPOSES

We examined the reliability and stability of three common classification systems for LCPD.

METHODS

We identified 69 patients with LCPD, who had hip radiographs taken more than twice after the initial presentation with at least a 3-month interval. The Herring lateral pillar, Catterall, and Salter-Thompson classifications were evaluated in terms of reliability and stability. The inter- and intrarater reliability of the classification systems was determined by three orthopaedic surgeons using intraclass correlation coefficients (ICCs). To evaluate the stability of the classification systems, the percentage agreement and ICCs among the initial rating, rating when entering fragmentation, and final rating were used.

RESULTS

The interrater reliability was highest in Herring lateral pillar classification (ICC, 0.885) followed by the Catterall and Salter-Thompson classifications (ICC, 0.802 and 0.702, respectively). The percentage agreement and ICC between the initial and final rating were, respectively, 55% and 0.491 for the Herring classification and 48% and 0.378 for the Catterall classification.

CONCLUSIONS

Our data show the highest reliability of the classification of Herring et al. However, more than 40% of the hip radiographs at the initial presentation, and in particular, most of Herring Group A patients, were upgraded. Therefore, for patients older than 8 years old and graded as Herring Group A initially, surgeons should keep the possibility of surgical treatment in mind.

Outcomes in advanced Legg-Calvé-Perthes disease treated with the Staheli procedure

BACKGROUND

Containment and preservation of hip range of motion have been the principles governing the treatment of Legg-Calvé-Perthes disease. The Staheli's slotted acetabular augmentation (SAA) procedure is an alternative approach for older children with Perthes disease with lateral subluxation, incongruent hip, and abduction hinge. It provides effective coverage, restricts lateral displacement of the femoral head, and achieves containment and congruency in short-term follow-up.

MATERIALS AND METHODS

This study evaluated pre- and postoperative symptoms, range of motion (ROM), and radiographic images in advanced Perthes disease with an incongruent hip or hinge abduction in lateral pillar groups B and C of 21 children who underwent a SAA procedure.

RESULTS

Postoperative evaluation showed improved abduction, internal rotation, subluxation ratio, femoral head ratio, and acetabular coverage. At final follow-up, ROM of abduction went from 20° preoperatively to 45° postoperatively, and internal rotation went from 15° preoperatively to 35° postoperatively. The radiographic findings revealed 33% and 38% of patients achieved spherical congruence of the hip and an ovoid or mushroom femoral head, respectively.

CONCLUSION

SAA restricts lateral displacement or subluxation of the femoral head and preserves sphericity, resulting in containment and acetabular coverage.

Inter-observer reliability of the Stulberg classification in the assessment of Perthes disease

PURPOSE

Accurate and reliable radiographic classifications are of great importance as a basis of treatment decisions and prognosis in Perthes disease. The classification of Stulberg is widely used as a predictor of long-term outcome. The aim of the present study was to determine whether the Stulberg classification is sufficiently reliable for routine clinical use in the assessment of Perthes disease.

METHODS

We used this classification to assess the radiographs of 101 hips in two separate sessions (55 and 46 hips, respectively), interfered by an educational intervention in which the classification algorithm was discussed and clarified.

RESULTS

We obtained good agreement between experienced examiners (weighted kappa 0.65) and a percentage agreement of 71%. We obtained weighted kappa values of 0.51 and 0.57 (moderate agreement) and percentage agreements of 62% and 65% between the least experienced observer and the two experienced examiners. Combining Stulberg class I and II, and IV and V into a simpler three-group classification gave better agreement between all observers. The agreement between the two experienced observers was improved to 81%.

CONCLUSIONS

We conclude that the reliability of the Stulberg classification is acceptable when the radiographic assessment is carried out by experienced examiners. A simpler three-group classification based on the shape of the femoral head (spherical, ovoid and flat) gave better agreement and is, therefore, recommended for routine clinical use.