Data Quality Improvement in Clinical Databases Using Statistical Quality Control: Review and Case Study

Assessing data accuracy in a large multi-institutional quality improvement registry: an update from the Pediatric Cardiac Critical Care Consortium (PC4)

BACKGROUND

The Pediatric Cardiac Critical Care Consortium (PC⁴) is a multi-institutional quality improvement registry focused on the care delivered in the cardiac ICU for patients with CHD and acquired heart disease. To assess data quality, a rigorous procedure of data auditing has been in place since the inception of the consortium.

MATERIALS AND METHODS

This report describes the data auditing process and quantifies the audit results for the initial 39 audits that took place after the transition from version one to version two of the registry's database.

RESULTS

In total, 2219 total encounters were audited for an average of 57 encounters per site. The overall data accuracy rate across all sites was 99.4%, with a major discrepancy rate of 0.52%. A passing score is based on an overall accuracy of >97% (achieved by all sites) and a major discrepancy rate of <1.5% (achieved by 38 of 39 sites, with 35 of 39 sites having a major discrepancy rate of <1%). Fields with the highest discrepancy rates included arrhythmia type, cardiac arrest count, and current surgical status.

CONCLUSIONS

The extensive PC⁴ auditing process, including initial and routinely scheduled follow-up audits of every participating site, demonstrates an extremely high level of accuracy across a broad array of audited fields and supports the continued use of consortium data to identify best practices in paediatric cardiac critical care.

Accuracy of inter-hospital transfer information in Australian hospital administrative databases

Inter-hospital transfers improve care delivery for which sending and receiving hospitals both accountable for patient outcomes. We aim to measure accuracy in recorded patient transfer information (indication of transfer and hospital identifier) over 2 years across 121 acute hospitals in New South Wales, Australia. Accuracy rate for 127,406 transfer-out separations was 87 per cent, with a low variability across hospitals (10% differences); it was 65 per cent for 151,978 transfer-in admissions with a greater inter-hospital variation (36% differences). Accuracy rate varied by departure and arrival pathways; at receiving hospitals, it was lower for transfer-in admission via emergency department (incidence rate ratio = 0.52, 95% confidence interval: 0.51-0.53) versus direct admission. Transfer-out data were more accurate for transfers to smaller hospitals (incidence rate ratio = 1.06, 95% confidence interval: 1.03-1.08) or re-transfers (incidence rate ratio > 1.08). Incorporation of transfer data from sending and receiving hospitals at patient level in administrative datasets and standardisation of documentation across hospitals would enhance accuracy and support improved attribution of hospital performance measures.

Effect of change in coding rules on recording diabetes in hospital administrative datasets

AIM

During 2008-2011 Australian Coding Standards mandated a causal relationship between diabetes and inpatient care as a criterion for recording diabetes as a comorbidity in hospital administrative datasets. We aim to measure the effect of the causality mandate on recorded diabetes and associated inter-hospital variations.

METHOD

For patients with diabetes, all admissions between 2004 and 2013 to all New South Wales acute public hospitals were investigated. Poisson mixed models were employed to derive adjusted rates and variations.

RESULTS

The non-recorded diabetes incidence rate was 20.7%. The causality mandate increased the incidence rate four fold during the change period, 2008-2011, compared to the pre- or post-change periods (32.5% vs 8.4% and 6.9%). The inter-hospital variation was also higher, with twice the difference in the non-recorded rate between hospitals with the highest and lowest rates (50% vs 24% and 27% risk gap). The variation decreased during the change period (29%), while the rate continued to rise (53%). Admission characteristics accounted for over 44% of the variation compared with at most two per cent attributable to patient or hospital characteristics. Contributing characteristics explained less of the variation within the change period compared to pre- or post-change (46% vs 58% and 53%). Hospital relative performance was not constant over time.

CONCLUSION

The causality mandate substantially increased the non-recorded diabetes rate and associated inter-hospital variation. Longitudinal accumulation of clinical information at the patient level, and the development of appropriate adoption protocols to achieve comprehensive and timely implementation of coding changes are essential to supporting the integrity of hospital administrative datasets.

Incidence and Variation of Discrepancies in Recording Chronic Conditions in Australian Hospital Administrative Data

Diagnostic data routinely collected for hospital admitted patients and used for case-mix adjustment in care provider comparisons and reimbursement are prone to biases. We aim to measure discrepancies, variations and associated factors in recorded chronic morbidities for hospital admitted patients in New South Wales (NSW), Australia. Of all admissions between July 2010 and June 2014 in all NSW public and private acute hospitals, admissions with over 24 hours stay and one or more of the chronic conditions of diabetes, smoking, hepatitis, HIV, and hypertension were included. The incidence of a non-recorded chronic condition in an admission occurring after the first admission with a recorded chronic condition (index admission) was considered as a discrepancy. Poisson models were employed to (i) derive adjusted discrepancy incidence rates (IR) and rate ratios (IRR) accounting for patient, admission, comorbidity and hospital characteristics and (ii) quantify variation in rates among hospitals. The discrepancy incidence rate was highest for hypertension (51% of 262,664 admissions), followed by hepatitis (37% of 12,107), smoking (33% of 548,965), HIV (27% of 1500) and diabetes (19% of 228,687). Adjusted rates for all conditions declined over the four-year period; with the sharpest drop of over 80% for diabetes (47.7% in 2010 vs. 7.3% in 2014), and 20% to 55% for the other conditions. Discrepancies were more common in private hospitals and smaller public hospitals. Inter-hospital differences were responsible for 1% (HIV) to 9.4% (smoking) of variation in adjusted discrepancy incidences, with an increasing trend for diabetes and HIV. Chronic conditions are recorded inconsistently in hospital administrative datasets, and hospitals contribute to the discrepancies. Adjustment for patterns and stratification in risk adjustments; and furthermore longitudinal accumulation of clinical data at patient level, refinement of clinical coding systems and standardisation of comorbidity recording across hospitals would enhance accuracy of datasets and validity of case-mix adjustment.