Intraobserver and interobserver agreement of International Classification of Diseases, Ninth Revision codes in classifying shoulder instability

Can researchers trust ICD-10 coding of medical comorbidities in orthopaedic trauma patients?

Objectives

The 10th revision of the International Classification of Diseases (ICD-10) coding system may prove useful to orthopaedic trauma researchers to identify and document populations based on comorbidities. However, its use for research first necessitates determination of its reliability. The purpose of this study was to assess the reliability of electronic medical record (EMR) ICD-10 coding of nonorthopaedic diagnoses in orthopaedic trauma patients relative to the gold standard of prospective data collection.

Design

Nonexperimental cross-sectional study.

Setting

Level 1 Trauma Center.

Patients/Participants

Two hundred sixty-three orthopaedic trauma patients from 2 prior prospective studies from September 2018 to April 2022.

Intervention

Prospectively collected data were compared with EMR ICD-10 code abstraction for components of the Charlson Comorbidity Index (CCI), obesity, alcohol abuse, and tobacco use (retrospective data).

Main Outcome Measurements

Percent agreement and Cohen's kappa reliability.

Results

Percent agreement ranged from 86.7% to 96.9% for all CCI diagnoses and was as low as 72.6% for the diagnosis "overweight." Only 2 diagnoses, diabetes without end-organ damage (kappa = 0.794) and AIDS (kappa = 0.798) demonstrated Cohen's kappa values to indicate substantial agreement.

Conclusion

EMR diagnostic coding for medical comorbidities in orthopaedic trauma patients demonstrated variable reliability. Researchers may be able to rely on EMR coding to identify patients with diabetes without complications or AIDS. Chart review may still be necessary to confirm diagnoses. Low prevalence of most comorbidities led to high percentage agreement with low reliability.

Level of Evidence

Level 1 diagnostic.

Reliability of Radiographic Union Scoring in Humeral Shaft Fractures

OBJECTIVES

To establish the reliability of 2 radiographic union scoring systems for nonoperative humeral shaft fractures.

DESIGN

Retrospective medical record review. Patients identified had humeral shaft fractures and radiographs at various follow-up time points, which were graded according to the both the standard (RUST) and modified radiographic union scoring systems (mRUST).

SETTING

A single North American Level-1 Trauma center in Connecticut, including emergency department and clinic follow-up visits.

PATIENTS/PARTICIPANTS

Forty-five adult patients (162 image sets) met the following inclusion criteria: diaphyseal humerus fracture, initial nonoperative management, and greater than 2 weeks of follow-up with imaging.

INTERVENTION

All 162 image sets of anterior-posterior and lateral radiographs were scored and divided into 4 tiers based on increasing score. Anterior-posterior and lateral image sets were randomly selected from each tier for a total of 50 that were then scored by 7 different reviewers using both the RUST and mRUST systems.

MAIN OUTCOME MEASURES

The intraclass correlation coefficients for the cortical and system scores for the RUST and mRUST systems.

RESULTS

Interobserver reliability was 0.795 for the RUST system and 0.801 for mRUST. Intraobserver reliability was 0.909 for RUST and 0.949 for mRUST. For mRUST, 92% of values were within ± 1 point from each other.

CONCLUSIONS

The RUST and mRUST systems can be applied to humeral shaft fractures with excellent reliability. They have the potential to assist in the diagnosis of humeral shaft union by providing an objective and standardized method to assess healing of bone over time.

Reliability of International Classification of Disease-9 Versus International Classification of Disease-10 Coding for Proximal Femur Fractures at a Level 1 Trauma Center

INTRODUCTION

The Centers for Medicare & Medicaid services proposed that transitioning from the 9th to the 10th revision of the International Classification of Disease (ICD) would provide better data for research. This study sought to determine the reliability of ICD-10 compared with ICD-9 for proximal femur fractures.

METHODS

Available imaging studies from 196 consecutively treated proximal femur fractures were retrospectively reviewed and assigned ICD codes by three physicians. Intercoder reliability (ICR) was calculated. Collectively, the physicians agreed on what should be the correct codes for each fracture, and this was compared with coding found in the medical and billing records.

RESULTS

No significant difference was observed in ICR for both ICD-9 and ICD-10 exact coding, which were both unreliable. Less specific coding improved ICR. ICD-9 general coding was better than ICD-10. Electronic medical record coding was unreliable. Billing codes were also unreliable, yet ICD-10 was better than ICD-9.

DISCUSSION

ICD-9 and ICD-10 lack reliability in coding proximal femur fractures. ICD-10 results in data that are no more reliable than those found with ICD-9.

LEVEL OF EVIDENCE

Level I diagnostic.

Discordance in current procedural terminology coding for foot and ankle procedures between residents and attending surgeons

OBJECTIVE

Because of the importance of current procedural terminology (CPT) coding in both resident education evaluation and practice management, this study was undertaken to evaluate the correlation and interrater reliability between residents and attending physicians in CPT coding for orthopedic foot and ankle surgeries as well as to determine attending surgeons' and residents' familiarity with and confidence in the coding process.

METHODS

CPT codes from resident case logs were compared with those submitted by attending surgeons, and Pearson's correlation coefficient and interrater reliability were calculated to examine coding congruency. An online survey was also used to examine attending surgeon and resident perceptions and habits regarding CPT codes and the coding process.

RESULTS

CPT codes recorded by 20 residents (1164) were compared with those recorded by 3 attending foot and ankle surgeons (1259). Correlation between attending and resident codes was poor (r = -0.015). Interrater reliability demonstrated a kappa value of 0.04, indicating poor agreement. Compared with attending CPT coding, residents concordantly coded 42% of the time, with an individual resident range from 2% to 65%. Additionally, 43% of residents reported being uncomfortable about foot and ankle CPT coding, and they reported rarely or never discussing CPT codes with attending surgeons in the perioperative period.

CONCLUSIONS

Resident and attending surgeon concordance in CPT coding for foot and ankle procedures is poor, and residents have a low level of confidence in logging CPT codes, possibly because of a lack of training and preparation in coding. Because CPT coding is used not only for practice management but also has implications for evaluating institutions by accreditation bodies, educational initiatives to improve resident confidence and accuracy with CPT coding may be warranted.

Efficiency of International Classification of Diseases, Ninth Revision, billing code searches to identify emergency department visits for blood or body fluid exposures through a statewide multicenter database

BACKGROUND

Blood and body fluid exposures are frequently evaluated in emergency departments (EDs). However, efficient and effective methods for estimating their incidence are not yet established.

OBJECTIVE

Evaluate the efficiency and accuracy of estimating statewide ED visits for blood or body fluid exposures using International Classification of Diseases, Ninth Revision (ICD-9), code searches.

DESIGN

Secondary analysis of a database of ED visits for blood or body fluid exposure.

SETTING

EDs of 11 civilian hospitals throughout Rhode Island from January 1, 1995, through June 30, 2001.

PATIENTS

Patients presenting to the ED for possible blood or body fluid exposure were included, as determined by prespecified ICD-9 codes.

METHODS

Positive predictive values (PPVs) were estimated to determine the ability of 10 ICD-9 codes to distinguish ED visits for blood or body fluid exposure from ED visits that were not for blood or body fluid exposure. Recursive partitioning was used to identify an optimal subset of ICD-9 codes for this purpose. Random-effects logistic regression modeling was used to examine variations in ICD-9 coding practices and styles across hospitals. Cluster analysis was used to assess whether the choice of ICD-9 codes was similar across hospitals.

RESULTS

The PPV for the original 10 ICD-9 codes was 74.4% (95% confidence interval [CI], 73.2%-75.7%), whereas the recursive partitioning analysis identified a subset of 5 ICD-9 codes with a PPV of 89.9% (95% CI, 88.9%-90.8%) and a misclassification rate of 10.1%. The ability, efficiency, and use of the ICD-9 codes to distinguish types of ED visits varied across hospitals.

CONCLUSIONS

Although an accurate subset of ICD-9 codes could be identified, variations across hospitals related to hospital coding style, efficiency, and accuracy greatly affected estimates of the number of ED visits for blood or body fluid exposure.

Development and reliability testing of the frequency, etiology, direction, and severity (FEDS) system for classifying glenohumeral instability

HYPOTHESIS

Classification systems for glenohumeral instability (GHI) are opinion based, not validated, and poorly defined. The hypothesis driving this investigation is that a classification system with content validity will have high inter-observer and intra-observer agreement.

MATERIALS AND METHODS

The classification system was developed by first conducting systematic literature review that identified 18 systems for classifying GHI. The frequency of characteristics used was recorded. Additionally, 31 members of the American Shoulder and Elbow Surgeons responded to a survey to identify features important to characterize GHI. Frequency, etiology, direction, and severity (FEDS) were most important. Frequency was defined as solitary (1 episode), occasional (2 to 5 times/y), or frequent (>5 times/year). Etiology was defined as traumatic or atraumatic. Direction referred to the primary direction of instability (anterior, posterior, or inferior). Severity was subluxation or dislocation. For reliability testing, 50 GHI patients completed a questionnaire at their initial visit. One of 6 sports medicine fellowship-trained physicians completed a similar questionnaire after examining the patient. Patients returned after 2 weeks and were examined by the original physician and 2 other physicians. Interrater and intrarater agreement for the FEDS classification system was calculated.

RESULTS

Agreement between patients and physicians was lowest for frequency (39%; κ = 0.130) and highest for direction (82%; κ = 0.636). Physician intrarater agreement was 84% to 97% for the individual FEDS characteristics (κ = 0.69-0.87), and interrater agreement was 82% to 90% (κ = 0.44-0.76).

CONCLUSIONS

The FEDS system has content validity and is highly reliable for classifying GHI. Physical examination using provocative testing to determine the primary direction of instability produces very high levels of interrater and intrarater agreement.