Assessment and perceptions of intensive care data quality, reporting and use: a survey of ICU directors

Assessing data accuracy in a large multi-institutional quality improvement registry: an update from the Pediatric Cardiac Critical Care Consortium (PC4)

BACKGROUND

The Pediatric Cardiac Critical Care Consortium (PC⁴) is a multi-institutional quality improvement registry focused on the care delivered in the cardiac ICU for patients with CHD and acquired heart disease. To assess data quality, a rigorous procedure of data auditing has been in place since the inception of the consortium.

MATERIALS AND METHODS

This report describes the data auditing process and quantifies the audit results for the initial 39 audits that took place after the transition from version one to version two of the registry's database.

RESULTS

In total, 2219 total encounters were audited for an average of 57 encounters per site. The overall data accuracy rate across all sites was 99.4%, with a major discrepancy rate of 0.52%. A passing score is based on an overall accuracy of >97% (achieved by all sites) and a major discrepancy rate of <1.5% (achieved by 38 of 39 sites, with 35 of 39 sites having a major discrepancy rate of <1%). Fields with the highest discrepancy rates included arrhythmia type, cardiac arrest count, and current surgical status.

CONCLUSIONS

The extensive PC⁴ auditing process, including initial and routinely scheduled follow-up audits of every participating site, demonstrates an extremely high level of accuracy across a broad array of audited fields and supports the continued use of consortium data to identify best practices in paediatric cardiac critical care.

A novel method to assess data quality in large medical registries and databases

BACKGROUND

There is no gold standard to assess data quality in large medical registries. Data auditing may be impeded by data protection regulations.

OBJECTIVE

To explore the applicability and usefulness of funnel plots as a novel tool for data quality control in critical care registries.

METHOD

The Swiss ICU-Registry from all 77 certified adult Swiss ICUs (2014 and 2015) was subjected to quality assessment (completeness/accuracy). For the analysis of accuracy, a list of logical rules and cross-checks was developed. Type and number of errors (true coding errors or implausible data) were calculated for each ICU, along with noticeable error rates (>mean + 3 SD in the variable's summary measure, or >99.8% CI in the respective funnel-plot).

RESULTS

We investigated 164 415 patient records with 31 items each (37 items: trauma diagnosis). Data completeness was excellent; trauma was the only incomplete item in 1495 of 9871 records (0.1%, 0.0%-0.6% [median, IQR]). In 15 572 patients records (9.5%), we found 3121 coding errors and 31 265 implausible situations; the latter primarily due to non-specific information on patients' provenance/diagnosis or supposed incoherence between diagnosis and treatments. Together, the error rate was 7.6% (5.9%-11%; median, IQR).

CONCLUSIONS

The Swiss ICU-Registry is almost complete and data quality seems to be adequate. We propose funnel plots as suitable, easy to implement instrument to assist in quality assurance of such a registry. Based on our analysis, specific feedback to ICUs with special-cause variation is possible and may promote such ICUs to improve the quality of their data.