Validity of AHRQ patient safety indicators derived from ICD-10 hospital discharge abstract data (chart review study)

A scoping review of the methodological approaches used in retrospective chart reviews to validate adverse event rates in administrative data

Patient safety is a key quality issue for health systems. Healthcare acquired adverse events (AEs) compromise safety and quality; therefore, their reporting and monitoring is a patient safety priority. Although administrative datasets are potentially efficient tools for monitoring rates of AEs, concerns remain over the accuracy of their data. Chart review validation studies are required to explore the potential of administrative data to inform research and health policy. This review aims to present an overview of the methodological approaches and strategies used to validate rates of AEs in administrative data through chart review. This review was conducted in line with the Joanna Briggs Institute methodological framework for scoping reviews. Through database searches, 1054 sources were identified, imported into Covidence, and screened against the inclusion criteria. Articles that validated rates of AEs in administrative data through chart review were included. Data were extracted, exported to Microsoft Excel, arranged into a charting table, and presented in a tabular and descriptive format. Fifty-six studies were included. Most sources reported on surgical AEs; however, other medical specialties were also explored. Chart reviews were used in all studies; however, few agreed on terminology for the study design. Various methodological approaches and sampling strategies were used. Some studies used the Global Trigger Tool, a two-stage chart review method, whilst others used alternative single-, two-stage, or unclear approaches. The sources used samples of flagged charts (n = 24), flagged and random charts (n = 11), and random charts (n = 21). Most studies reported poor or moderate accuracy of AE rates. Some studies reported good accuracy of AE recording which highlights the potential of using administrative data for research purposes. This review highlights the potential for administrative data to provide information on AE rates and improve patient safety and healthcare quality. Nonetheless, further work is warranted to ensure that administrative data are accurate. The variation of methodological approaches taken, and sampling techniques used demonstrate a lack of consensus on best practice; therefore, further clarity and consensus are necessary to develop a more systematic approach to chart reviewing.

Validity of 16 AHRQ Patient Safety Indicators to identify in-hospital complications: a medical record review across nine Swiss hospitals

The validity of the Agency for Healthcare Research and Quality's Patient Safety Indicators (PSIs) has been established in the USA and Canada. However, these indicators are also used for hospital benchmarking and cross-country comparisons in other nations with different health-care settings and coding systems as well as missing present on admission (POA) flags in the administrative data. This study sought to comprehensively assess and compare the validity of 16 PSIs in Switzerland, where they have not been previously applied. We performed a medical record review using administrative and electronic medical record data from nine Swiss hospitals. Seven independent reviewers evaluated 1245 cases at various hospitals using retrospective data from the years 2014-18. True positives, false positives, positive predictive values (PPVs), and reasons for misclassification were compared across all investigated PSIs, and the documentation quality of the PSIs was examined. PSIs 6 (iatrogenic pneumothorax), 10 (postoperative acute kidney injury), 11 (postoperative respiratory failure), 13 (postoperative sepsis), 14 (wound dehiscence), 17 (birth trauma), and 18 and 19 (obstetric trauma with or without instrument) showed high PPVs (range: 90-99%) and were not strongly influenced by missing POA information. In contrast, PSIs 3 (pressure ulcer), 5 (retained surgical item), 7 (central venous catheter-related bloodstream infection), 8 (fall with hip fracture), and 15 (accidental puncture/laceration) showed low PPVs (range: 18-49%). In the case of PSIs 3, 8, and 12 (perioperative embolism/thrombosis), the low PPVs were largely due to the lack of POA information. Additionally, it was found that the documentation of PSI 3 in discharge letters could be improved. We found large differences in validity across the 16 PSIs in Switzerland. These results can guide policymakers in Switzerland and comparable health-care systems in selecting and prioritizing suitable PSIs for quality initiatives. Furthermore, the national introduction of a POA flag would allow for the inclusion of additional PSIs in quality monitoring.

Development of machine learning models for the detection of surgical site infections following total hip and knee arthroplasty: a multicenter cohort study

BACKGROUND

Population based surveillance of surgical site infections (SSIs) requires precise case-finding strategies. We sought to develop and validate machine learning models to automate the process of complex (deep incisional/organ space) SSIs case detection.

METHODS

This retrospective cohort study included adult patients (age ≥ 18 years) admitted to Calgary, Canada acute care hospitals who underwent primary total elective hip (THA) or knee (TKA) arthroplasty between Jan 1st, 2013 and Aug 31st, 2020. True SSI conditions were judged by the Alberta Health Services Infection Prevention and Control (IPC) program staff. Using the IPC cases as labels, we developed and validated nine XGBoost models to identify deep incisional SSIs, organ space SSIs and complex SSIs using administrative data, electronic medical records (EMR) free text data, and both. The performance of machine learning models was assessed by sensitivity, specificity, positive predictive value, negative predictive value, F1 score, the area under the receiver operating characteristic curve (ROC AUC) and the area under the precision-recall curve (PR AUC). In addition, a bootstrap 95% confidence interval (95% CI) was calculated.

RESULTS

There were 22,059 unique patients with 27,360 hospital admissions resulting in 88,351 days of hospital stay. This included 16,561 (60.5%) TKA and 10,799 (39.5%) THA procedures. There were 235 ascertained SSIs. Of them, 77 (32.8%) were superficial incisional SSIs, 57 (24.3%) were deep incisional SSIs, and 101 (42.9%) were organ space SSIs. The incidence rates were 0.37 for superficial incisional SSIs, 0.21 for deep incisional SSIs, 0.37 for organ space and 0.58 for complex SSIs per 100 surgical procedures, respectively. The optimal XGBoost models using administrative data and text data combined achieved a ROC AUC of 0.906 (95% CI 0.835-0.978), PR AUC of 0.637 (95% CI 0.528-0.746), and F1 score of 0.79 (0.67-0.90).

CONCLUSIONS

Our findings suggest machine learning models derived from administrative data and EMR text data achieved high performance and can be used to automate the detection of complex SSIs.

Eligibility for early rhythm control in patients with atrial fibrillation in the UK Biobank

OBJECTIVE

The Early Treatment of Atrial Fibrillation for Stroke Prevention (EAST-AFNET4) trial showed a clinical benefit of early rhythm-control therapy in patients with recently diagnosed atrial fibrillation (AF). The generalisability of the results in the general population is not known.

METHODS

Participants in the population-based UK Biobank were assessed for eligibility based on the EAST-AFNET4 inclusion/exclusion criteria. Treatment of all eligible participants was classified as early rhythm-control (antiarrhythmic drug therapy or AF ablation) or usual care. To assess treatment effects, primary care data and Hospital Episode Statistics were merged with UK Biobank data.Efficacy and safety outcomes were compared between groups in the entire cohort and in a propensity-matched data set.

RESULTS

AF was present in 35 526/502 493 (7.1%) participants, including 8340 (988 with AF <1 year) with AF at enrolment and 27 186 with incident AF during follow-up. Most participants (22 003/27 186; 80.9%) with incident AF were eligible for early rhythm-control.Eligible participants were older (70 years vs 63 years) and more likely to be female (42% vs 21%) compared with ineligible patients. Of 9004 participants with full primary care data, 874 (9.02%) received early rhythm-control. Safety outcomes were not different between patients receiving early rhythm-control and controls. The primary outcome of EAST-AFNET 4, a composite of cardiovascular death, stroke/transient ischaemic attack and hospitalisation for heart failure or acute coronary syndrome occurred less often in participants receiving early rhythm-control compared with controls in the entire cohort (HR 0.82, 95% CI 0.71 to 0.94, p=0.005). In the propensity-score matched analysis, early rhythm-control did not significantly decrease of the primary outcome compared with usual care (HR 0.87, 95% CI 0.72 to 1.04, p=0.124).

CONCLUSION

Around 80% of participants diagnosed with AF in the UK population are eligible for early rhythm-control. Early rhythm-control therapy was safe in routine care.

Accuracy of International Classification of Diseases, 10th Revision Codes for Identifying Sepsis: A Systematic Review and Meta-Analysis

Administrative databases are increasingly used in research studies to capture clinical outcomes such as sepsis. This systematic review and meta-analysis examines the accuracy of International Classification of Diseases, 10th revision (ICD-10), codes for identifying sepsis in adult and pediatric patients.

DATA SOURCES

We searched MEDLINE, EMBASE, Web of Science, CENTRAL, Epistemonikos, and McMaster Superfilters from inception to September 7, 2021.

STUDY SELECTION

We included studies that validated the accuracy of sepsis ICD-10 codes against any reference standard.

DATA EXTRACTION

Three authors, working in duplicate, independently extracted data. We conducted meta-analysis using a random effects model to pool sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). We evaluated individual study risk of bias using the Quality Assessment of Diagnostic Accuracy Studies tool and assessed certainty in pooled diagnostic effect measures using the Grading of Recommendations Assessment, Development, and Evaluation framework.

DATA SYNTHESIS

Thirteen eligible studies were included in the qualitative synthesis and the meta-analysis. Eleven studies used manual chart review as the reference standard, and four studies used registry databases. Only one study evaluated pediatric patients exclusively. Compared with the reference standard of detailed chart review and/or registry databases, the pooled sensitivity for sepsis ICD-10 codes was 35% (95% CI, 22-48, low certainty), whereas the pooled specificity was 98% (95% CI: 98-99, low certainty). The PPV for ICD-10 codes ranged from 9.8% to 100% (median, 72.0%; interquartile range [IQR], 50.0-84.7%). NPV ranged from 54.7% to 99.1% (median, 95.9%; interquartile range, 85.5-98.3%).

CONCLUSIONS

Sepsis is undercoded in administrative databases. Future research is needed to explore if greater consistency in ICD-10 code definitions and enhanced quality measures for ICD-10 coders can improve the coding accuracy of sepsis in large databases.

Developing EMR-based algorithms to Identify hospital adverse events for health system performance evaluation and improvement: Study protocol

Background

Measurement of care quality and safety mainly relies on abstracted administrative data. However, it is well studied that administrative data-based adverse event (AE) detection methods are suboptimal due to lack of clinical information. Electronic medical records (EMR) have been widely implemented and contain detailed and comprehensive information regarding all aspects of patient care, offering a valuable complement to administrative data. Harnessing the rich clinical data in EMRs offers a unique opportunity to improve detection, identify possible risk factors of AE and enhance surveillance. However, the methodological tools for detection of AEs within EMR need to be developed and validated. The objectives of this study are to develop EMR-based AE algorithms from hospital EMR data and assess AE algorithm’s validity in Canadian EMR data.

Methods

Patient EMR structured and text data from acute care hospitals in Calgary, Alberta, Canada will be linked with discharge abstract data (DAD) between 2010 and 2020 (n~1.5 million). AE algorithms development. First, a comprehensive list of AEs will be generated through a systematic literature review and expert recommendations. Second, these AEs will be mapped to EMR free texts using Natural Language Processing (NLP) technologies. Finally, an expert panel will assess the clinical relevance of the developed NLP algorithms. AE algorithms validation: We will test the newly developed AE algorithms on 10,000 randomly selected EMRs between 2010 to 2020 from Calgary, Alberta. Trained reviewers will review the selected 10,000 EMR charts to identify AEs that had occurred during hospitalization. Performance indicators (e.g., sensitivity, specificity, positive predictive value, negative predictive value, F₁ score, etc.) of the developed AE algorithms will be assessed using chart review data as the reference standard.

Discussion

The results of this project can be widely implemented in EMR based healthcare system to accurately and timely detect in-hospital AEs.

A Pragmatic Method for Measuring Inpatient Complications and Complication-Specific Mortality

OBJECTIVES

The primary objective of this study was to develop hospital-level metrics of major complications associated with mortality that allows for the identification of opportunities for improvement. The secondary objective is to improve upon current metrics for failure to rescue (i.e., death from serious but treatable complications.).

METHODS

Agency for Healthcare Research and Quality metrics served as the basis for identifying specific complications related to major organ system morbidity associated with death. Complication-specific occurrence rates, observed mortality, and risk-adjusted mortality indices were calculated for the study institution and 182 peer organizations using component International Classification of Disease, Tenth Revision codes. Data were included for adults over a 4-year period, with exclusion of hospice patients and complications present on admission. Temporal visualizations of each metric were used to compare past and recent performance at the study hospital and in comparison to peers.

RESULTS

The complication-specific method showed statistically significant differences in the study hospital occurrence rates and associated mortality rates compared with peer institutions. The monthly control-chart presentation of these metrics provides assessment of hospital-level interventions to prevent complications and/or reduce failure to rescue deaths.

CONCLUSIONS

The method described supplements existing metrics of serious complications that occur during the course of acute hospitalization allowing for enhanced visualization of opportunities to improve care delivery systems. This method leverages existing measure components to minimize reporting burden. Monthly time-series data allow interventions to prevent and/or rescue patients to be rapidly assessed for impact.

Developing and validating natural language processing algorithms for radiology reports compared to ICD-10 codes for identifying venous thromboembolism in hospitalized medical patients

BACKGROUND

Identifying venous thromboembolism (VTE) from large clinical and administrative databases is important for research and quality improvement.

OBJECTIVE

To develop and validate natural language processing (NLP) algorithms to identify VTE from radiology reports among general internal medicine (GIM) inpatients.

METHODS

This cross-sectional study included GIM hospitalizations between April 1, 2010 and March 31, 2017 at 5 hospitals in Toronto, Ontario, Canada. We developed NLP algorithms to identify pulmonary embolism (PE) and deep venous thrombosis (DVT) from radiologist reports of thoracic computed tomography (CT), extremity compression ultrasound (US), and nuclear ventilation-perfusion (VQ) scans in a training dataset of 1551 hospitalizations. We compared the accuracy of our NLP algorithms, the previously-published "simpleNLP" tool, and administrative discharge diagnosis codes (ICD-10-CA) for PE and DVT to the "gold standard" manual review in a separate random sample of 4000 GIM hospitalizations.

RESULTS

Our NLP algorithms were highly accurate for identifying DVT from US, with sensitivity 0.94, positive predictive value (PPV) 0.90, and Area Under the Receiver-Operating-Characteristic Curve (AUC) 0.96; and in identifying PE from CT, with sensitivity 0.91, PPV 0.89, and AUC 0.96. Administrative diagnosis codes and the simple NLP tool were less accurate for DVT (ICD-10-CA sensitivity 0.63, PPV 0.43, AUC 0.81; simpleNLP sensitivity 0.41, PPV 0.36, AUC 0.66) and PE (ICD-10-CA sensitivity 0.83, PPV 0.70, AUC 0.91; simpleNLP sensitivity 0.89, PPV 0.62, AUC 0.92).

CONCLUSIONS

Administrative diagnosis codes are unreliable in identifying VTE in hospitalized patients. We developed highly accurate NLP algorithms to identify VTE from radiology reports in a multicentre sample and have made the algorithms freely available to the academic community with a user-friendly tool (https://lks-chart.github.io/CHARTextract-docs/08-downloads/rulesets.html#venous-thromboembolism-vte-rulesets).

Variation of Daily Care Demand in Swiss General Hospitals: Longitudinal Study on Capacity Utilization, Patient Turnover and Clinical Complexity Levels

Background

Variations in hospitals’ care demand relies not only on the patient volume but also on the disease severity. Understanding both daily severity and patient volume in hospitals could help to identify hospital pressure zones to improve hospital-capacity planning and policy-making.

Objective

This longitudinal study explored daily care demand dynamics in Swiss general hospitals for 3 measures: (1) capacity utilization, (2) patient turnover, and (3) patient clinical complexity level.

Methods

A retrospective population-based analysis was conducted with 1 year of routine data of 1.2 million inpatients from 102 Swiss general hospitals. Capacity utilization was measured as a percentage of the daily maximum number of inpatients. Patient turnover was measured as a percentage of the daily sum of admissions and discharges per hospital. Patient clinical complexity level was measured as the average daily patient disease severity per hospital from the clinical complexity algorithm.

Results

There was a pronounced variability of care demand in Swiss general hospitals. Among hospitals, the average daily capacity utilization ranged from 57.8% (95% CI 57.3-58.4) to 87.7% (95% CI 87.3-88.0), patient turnover ranged from 22.5% (95% CI 22.1-22.8) to 34.5% (95% CI 34.3-34.7), and the mean patient clinical complexity level ranged from 1.26 (95% CI 1.25-1.27) to 2.06 (95% CI 2.05-2.07). Moreover, both within and between hospitals, all 3 measures varied distinctly between days of the year, between days of the week, between weekdays and weekends, and between seasons.

Conclusions

While admissions and discharges drive capacity utilization and patient turnover variation, disease severity of each patient drives patient clinical complexity level. Monitoring—and, if possible, anticipating—daily care demand fluctuations is key to managing hospital pressure zones. This study provides a pathway for identifying patients’ daily exposure to strained hospital systems for a time-varying causal model.

Mitigating imperfect data validity in administrative data PSIs: a method for estimating true adverse event rates

QUESTION

Are there ways to mitigate the challenges associated with imperfect data validity in Patient Safety Indicator (PSI) report cards?

FINDINGS

Applying a methodological framework on simulated PSI report card data, we compare the adjusted PSI rates of three hospitals with variable quality of data and coding. This framework combines (i) a measure of PSI rates using existing algorithms; (ii) a medical record review on a small random sample of charts to produce a measure of hospital-specific data validity and (iii) a simple Bayesian calculation to derive estimated true PSI rates. For example, the estimated true PSI rate, for a theoretical hospital with a moderately good quality of coding, could be three times as high as the measured rate (for example, 1.4% rather than 0.5%). For a theoretical hospital with relatively poor quality of coding, the difference could be 50-fold (for example, 5.0% rather than 0.1%).

MEANING

Combining a medical chart review on a limited number of medical charts at the hospital level creates an approach to producing health system report cards with estimates of true hospital-level adverse event rates.

Comparing Charlson and Elixhauser comorbidity indices with different weightings to predict in-hospital mortality: an analysis of national inpatient data

Background

Understanding how comorbidity measures contribute to patient mortality is essential both to describe patient health status and to adjust for risks and potential confounding. The Charlson and Elixhauser comorbidity indices are well-established for risk adjustment and mortality prediction. Still, a different set of comorbidity weights might improve the prediction of in-hospital mortality. The present study, therefore, aimed to derive a set of new Swiss Elixhauser comorbidity weightings, to validate and compare them against those of the Charlson and Elixhauser-based van Walraven weights in an adult in-patient population-based cohort of general hospitals.

Methods

Retrospective analysis was conducted with routine data of 102 Swiss general hospitals (2012–2017) for 6.09 million inpatient cases. To derive the Swiss weightings for the Elixhauser comorbidity index, we randomly halved the inpatient data and validated the results of part 1 alongside the established weighting systems in part 2, to predict in-hospital mortality. Charlson and van Walraven weights were applied to Charlson and Elixhauser comorbidity indices. Derivation and validation of weightings were conducted with generalized additive models adjusted for age, gender and hospital types.

Results

Overall, the Elixhauser indices, c-statistic with Swiss weights (0.867, 95% CI, 0.865–0.868) and van Walraven’s weights (0.863, 95% CI, 0.862–0.864) had substantial advantage over Charlson’s weights (0.850, 95% CI, 0.849–0.851) and in the derivation and validation groups. The net reclassification improvement of new Swiss weights improved the predictive performance by 1.6% on the Elixhauser-van Walraven and 4.9% on the Charlson weights.

Conclusions

All weightings confirmed previous results with the national dataset. The new Swiss weightings model improved slightly the prediction of in-hospital mortality in Swiss hospitals. The newly derive weights support patient population-based analysis of in-hospital mortality and seek country or specific cohort-based weightings.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-020-05999-5.

Validation study of German inpatient administrative health data for epidemiological surveillance and measurement of quality of care for sepsis: the OPTIMISE study protocol

INTRODUCTION

Sepsis is a major cause of preventable deaths in hospitals. This study aims to investigate if sepsis incidence and quality of care can be assessed using inpatient administrative health data (IAHD).

METHODS AND ANALYSIS

Design: Retrospective observational validation study using routine data to assess the diagnostic accuracy of sepsis coding in IAHD regarding sepsis diagnosis based on medical record review.

PROCEDURE

A stratified sample of 10 000 patients with an age ≥15 years treated in between 2015 and 2017 in 10 German hospitals is investigated. All available information of medical records is screened by trained physicians to identify true sepsis cases ('gold standard') both according to current ('sepsis-1') definitions and new ('sepsis-3') definitions. Data from medical records are linked to IAHD on patient level using a pseudonym.

ANALYSES

Proportions of cases with sepsis according to sepsis-1 and sepsis-3 definitions are calculated and compared with estimates from coding of sepsis in IAHD. Predictive accuracy (sensitivity, specificity) of different coding abstraction strategies regarding the gold standard is estimated. Predictive accuracy of mortality risk factors obtained from IAHD regarding the respective risk factors obtained from medical records is calculated. An IAHD-based risk model for hospital mortality is compared with a record-based risk model regarding model-fit and predicted risk of death. Analyses adjust for sampling weights. The obtained estimates of sensitivity and specificity for sepsis coding in IAHD are used to estimate adjusted incidence proportions of sepsis based on German national IAHD.

ETHICS AND DISSEMINATION

The study has been approved by the ethics commission of the Jena University Hospital (No. 2018-1065-Daten). The results of the study will be discussed in an expert panel to write a memorandum on improving the utility of IAHD for epidemiological surveillance and quality management of sepsis care.

TRIAL REGISTRATION NUMBER

DRKS00017775; Pre-results.

Towards a national perioperative clinical quality registry: The diagnostic accuracy of administrative data in identifying major postoperative complications

Accurately measuring the incidence of major postoperative complications is essential for funding and reimbursement of healthcare providers, for internal and external benchmarking of hospital performance and for valid and reliable public reporting of outcomes. Actual or surrogate outcomes data are typically obtained by one of three methods: clinical quality registries, clinical audit, or administrative data. In 2017 a perioperative registry was developed at the Alfred Hospital and mapped to administrative and clinical data. This study investigated the statistical agreement between administrative data (International Statistical Classification of Diseases and Related Health Problems (10th edition) Australian Modification codes) and clinical audit by anaesthetists in identifying major postoperative complications. The study population included 482 high-risk surgical patients referred to the Alfred Hospital anaesthesia postoperative service over two years. Clinical audit was conducted to determine the presence of major complications and these data were compared to administrative data. The main outcome was statistical agreement between the two methods, as defined by Cohen’s kappa statistic. Substantial agreement was observed for five major complications, moderate agreement for three, fair agreement for six and poor agreement for two. Sensitivity and positive predictive value ranged from 0 to 100%. Specificity was above 90% for all complications. There was important variation in inter-rater agreement. For four of the five complications with substantial agreement between administrative data and clinical audit, sensitivity was only moderate (61.5%–75%). Using International Statistical Classification of Diseases and Related Health Problems (10th edition) Australian Modification codes to identify postoperative complications at our hospital has high specificity but is likely to underestimate the incidence compared to clinical audit. Further, retrospective clinical audit itself is not a highly reliable method of identifying complications. We believe a perioperative clinical quality registry is necessary to validly and reliably measure major postoperative complications in Australia for benchmarking of hospital performance and before public reporting of outcomes should be considered.

Patient Safety Indicator-12 Rarely Identifies Problems with Quality of Care in Perioperative Venous Thromboembolism

BACKGROUND

Patient safety indicators (PSI) were developed for hospitals to screen for healthcare-associated adverse events. PSIs are believed to be preventable and have become a part of major pay-for-performance programs. PSI-12 captures perioperative venous thromboembolism (VTE), which contributes to morbidity and mortality of hospitalized patients. We aimed to evaluate PSI-12 events at our institution to identify areas for improvement of perioperative VTE prevention.

METHODS

We identified PSI-12 events from June 2015 to June 2017 using the Agency for Healthcare Research and Quality software version 5. Events were reviewed using our electronic medical record to identify further details of each event.

RESULTS

A total of 154 perioperative VTE cases were analyzed in the 2-year period. Pulmonary embolism (PE) occurred in 62.9% of cases, deep venous thrombosis (DVT) in 24%, and concurrent DVT/PE in 12.9%. The mean age of patients was 56 years old. Deficiencies in guideline-appropriate prophylaxis were identified in only 17 (11%) of cases. Unfractionated heparin was used in 61 cases, enoxaparin in 31 cases, and nine events occurred on therapeutic anticoagulation. Mechanical prophylaxis was used in 51 cases because of bleeding risk, thrombocytopenia, and/or liver associated coagulopathy. Four events occurred prior to the index procedure, with another eight cases occurring intraoperatively, or on the day of the procedure.

CONCLUSIONS

PSI-12 has several limitations in identifying quality of care issues in perioperative VTE. While it may be useful as a screening tool, further research for improvements are needed if it will remain one of the key measures in pay-for-performance.

Validation of new ICD-10-based patient safety indicators for identification of in-hospital complications in surgical patients: a study of diagnostic accuracy

OBJECTIVE

Administrative data systems are used to identify hospital-based patient safety events; few studies evaluate their accuracy. We assessed the accuracy of a new set of patient safety indicators (PSIs; designed to identify in hospital complications).

STUDY DESIGN

Prospectively defined analysis of registry data (1 April 2010-29 February 2016) in a Canadian hospital network. Assignment of complications was by two methods independently. The National Surgical Quality Improvement Programme (NSQIP) database was the clinical reference standard (primary outcome=any in-hospital NSQIP complication); PSI clusters were assigned using International Classification of Disease (ICD-10) codes in the discharge abstract. Our primary analysis assessed the accuracy of any PSI condition compared with any complication in the NSQIP; secondary analysis evaluated accuracy of complication-specific PSIs.

PATIENTS

All inpatient surgical cases captured in NSQIP data.

ANALYSIS

We assessed the accuracy of PSIs (with NSQIP as reference standard) using positive and negative predictive values (PPV/NPV), as well as positive and negative likelihood ratios (±LR).

RESULTS

We identified 12 898 linked episodes of care. Complications were identified by PSIs and NSQIP in 2415 (18.7%) and 2885 (22.4%) episodes, respectively. The presence of any PSI code had a PPV of 0.55 (95% CI 0.53 to 0.57) and NPV of 0.93 (95% CI 0.92 to 0.93); +LR 6.41 (95% CI 6.01 to 6.84) and -LR 0.40 (95% CI 0.37 to 0.42). Subgroup analyses (by surgery type and urgency) showed similar performance. Complication-specific PSIs had high NPVs (95% CI 0.92 to 0.99), but low to moderate PPVs (0.13-0.61).

CONCLUSION

Validation of the ICD-10 PSI system suggests applicability as a first screening step, integrated with data from other sources, to produce an adverse event detection pathway that informs learning healthcare systems. However, accuracy was insufficient to directly identify or rule out individual-level complications.

Conversion of the Agency for Healthcare Research and Quality's Quality Indicators from ICD-9-CM to ICD-10-CM/PCS: The Process, Results, and Implications for Users

OBJECTIVE

To convert the Agency for Healthcare Research and Quality's (AHRQ) Quality Indicators (QIs) from International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) specifications to ICD, 10th Revision, Clinical Modification and Procedure Classification System (ICD-10-CM/PCS) specifications.

DATA SOURCES

ICD-9-CM and ICD-10-CM/PCS classifications, General Equivalence Maps (GEMs).

STUDY DESIGN

We convened 77 clinicians and coders to evaluate ICD-10-CM/PCS codes mapped from ICD-9-CM using automated GEMs. We reviewed codes to develop "legacy" specifications resembling those in ICD-9-CM and "enhanced" specifications addressing enhanced capabilities of ICD-10-CM/PCS.

DATA COLLECTION/EXTRACTION METHODS

We tabulated the numbers of mapped codes, added nonmapped codes, and deleted mapped codes to achieve the specifications.

PRINCIPAL FINDINGS

Of 212 clinical concepts (sets of codes) that comprise the QI specifications, we either added nonmapped codes to or deleted mapped codes from 115 (54 percent). The legacy and enhanced specifications differed for 46 sets (22 percent), affecting 67 of the 101 QIs (66 percent). Occasionally, concepts that defied conversion required reformulation of indicators.

CONCLUSIONS

Converting the AHRQ QIs to ICD-10-CM/PCS required a detailed, thorough process beyond automated mapping of codes. Differences between the legacy and enhanced versions of the QIs are frequently minor but sometimes substantive.

ICD-10 procedure codes produce transition challenges

The transition of procedure coding from ICD-9-CM-Vol-3 to ICD-10-PCS has generated problems for the medical community at large resulting from the lack of clarity required to integrate two non-congruent coding systems. We hypothesized that quantifying these issues with network topology analyses offers a better understanding of the issues, and therefore we developed solutions (online tools) to empower hospital administrators and researchers to address these challenges. Five topologies were identified: "identity"(I), "class-to-subclass"(C2S), "subclass-toclass"(S2C), "convoluted(C)", and "no mapping"(NM). The procedure codes in the 2010 Illinois Medicaid dataset (3,290 patients, 116 institutions) were categorized as C=55%, C2S=40%, I=3%, NM=2%, and S2C=1%. Majority of the problematic and ambiguous mappings (convoluted) pertained to operations in ophthalmology cardiology, urology, gyneco-obstetrics, and dermatology. Finally, the algorithms were expanded into a user-friendly tool to identify problematic topologies and specify lists of procedural codes utilized by medical professionals and researchers for mitigating error-prone translations, simplifying research, and improving quality.http://www.lussiergroup.org/transition-to-ICD10PCS.

A Population-Based Study of Syncope in the Young

BACKGROUND

The prevalence, hospitalization patterns, and outcomes of pediatric and adolescent syncope have not been rigorously characterized.

METHODS

Patients < 20 years presenting to an emergency department (ED) with a primary diagnosis of syncope (International Classification of Diseases, 10th revision, code R55) between fiscal year (FY) 2006/2007 and FY 2013/2014 in the province of Alberta, Canada were grouped according to discharge status from the ED, ie, (1) admitted to hospital and (2) discharged without admission. Temporal trends and differences in baseline characteristics, medication use, and outcomes between admitted and discharged patients were examined.

RESULTS

The prevalence of syncope increased from 143/100,000 population in FY 2006/2007 to 166/100,000 population in FY 2013/2014 (P < 0.01). The majority of the 11,488 patients who presented to the ED with syncope were discharged home (n = 11,214 [98%]). Cardiac disease was present in 12.7% and thoracic conditions were present in 8% of the study population. A majority of patients (66.2% admitted and 56.4% discharged; P = 0.018) were taking a prescription drug in the year before presentation. By 30 days, 26.1% of admitted patients had a second ED presentation and 8.1% had a rehospitalization. Among discharged patients, the 30-day repeated ED presentation rate was 11.7% and the hospitalization rate was 1.1%. By 1 year, the rates of repeated ED visits increased to 64.1% and 47.5%, and rehospitalization rates increased to 21.4% and 6.8% among admitted and discharged patients, respectively.

CONCLUSIONS

Our data suggest that pediatric and adolescent syncope is increasing in prevalence and represents a growing public health problem. This population has a high burden of comorbidities that likely contribute to increased health care resource use and polypharmacy.

Navigating a ship with a broken compass: evaluating standard algorithms to measure patient safety

Objective

Agency for Healthcare Research and Quality (AHRQ) software applies standardized algorithms to hospital administrative data to identify patient safety indicators (PSIs). The objective of this study was to assess the validity of PSI flags and report reasons for invalid flagging.

Material and Methods

At a 6-hospital academic medical center, a retrospective analysis was conducted of all PSIs flagged in fiscal year 2014. A multidisciplinary PSI Quality Team reviewed each flagged PSI based on quarterly reports. The positive predictive value (PPV, the percent of clinically validated cases) was calculated for 12 PSI categories. The documentation for each reversed case was reviewed to determine the reasons for PSI reversal.

Results

Of 657 PSI flags, 185 were reversed. Seven PSI categories had a PPV below 75%. Four broad categories of reasons for reversal were AHRQ algorithm limitations (38%), coding misinterpretations (45%), present upon admission (10%), and documentation insufficiency (7%). AHRQ algorithm limitations included 2 subcategories: an "incident" was inherent to the procedure, or highly likely (eg, vascular tumor bleed), or an "incident" was nonsignificant, easily controlled, and/or no intervention was needed.

Discussion

These findings support previous research highlighting administrative data problems. Additionally, AHRQ algorithm limitations was an emergent category not considered in previous research. Herein we present potential solutions to address these issues.

Conclusions

If, despite poor validity, US policy continues to rely on PSIs for incentive and penalty programs, improvements are needed in the quality of administrative data and the standardized PSI algorithms. These solutions require national motivation, research attention, and dissemination support.

Validation of two case definitions to identify pressure ulcers using hospital administrative data

OBJECTIVE

Pressure ulcer development is a quality of care indicator, as pressure ulcers are potentially preventable. Yet pressure ulcer is a leading cause of morbidity, discomfort and additional healthcare costs for inpatients. Methods are lacking for accurate surveillance of pressure ulcer in hospitals to track occurrences and evaluate care improvement strategies. The main study aim was to validate hospital discharge abstract database (DAD) in recording pressure ulcers against nursing consult reports, and to calculate prevalence of pressure ulcers in Alberta, Canada in DAD. We hypothesised that a more inclusive case definition for pressure ulcers would enhance validity of cases identified in administrative data for research and quality improvement purposes.

SETTING

A cohort of patients with pressure ulcers were identified from enterostomal (ET) nursing consult documents at a large university hospital in 2011.

PARTICIPANTS

There were 1217 patients with pressure ulcers in ET nursing documentation that were linked to a corresponding record in DAD to validate DAD for correct and accurate identification of pressure ulcer occurrence, using two case definitions for pressure ulcer.

RESULTS

Using pressure ulcer definition 1 (7 codes), prevalence was 1.4%, and using definition 2 (29 codes), prevalence was 4.2% after adjusting for misclassifications. The results were lower than expected. Definition 1 sensitivity was 27.7% and specificity was 98.8%, while definition 2 sensitivity was 32.8% and specificity was 95.9%. Pressure ulcer in both DAD and ET consultation increased with age, number of comorbidities and length of stay.

CONCLUSION

DAD underestimate pressure ulcer prevalence. Since various codes are used to record pressure ulcers in DAD, the case definition with more codes captures more pressure ulcer cases, and may be useful for monitoring facility trends. However, low sensitivity suggests that this data source may not be accurate for determining overall prevalence, and should be cautiously compared with other prevalence studies.

Possible Uses of Data from Hospital Discharge Reports

Almost all countries in the world keep some form of hospital discharge report (HDR). Although there are many variations, every report contains such data as patient demographics, the main cause of hospitalization, comorbidities, the length of stay in hospital and outcome. The advantages of using data obtained from HDRs are numerous: The data from HDRs are already collected in a designated centre and thus easily available and relatively cheap; HDRs contain information for many previous years; they are sometimes more reliable than data obtained through any other method; and finally, they provide a large and representative database. HDRs databases can be connected with other databases using a unified patient identification number. The limitations of using data obtained through HDRs are as follows: inconsistencies in defining and coding diagnoses and applied procedures, common underestimations of comorbidity, limited possible applications in specific studies and partial coverage of inpatient institutions. The prediction that in the future, a growing number of diagnostic and treatment procedures will be performed on an outpatient basis will also limit the use of HDRs. When electronic recordkeeping becomes a practice, we may assume that these data will no longer be needed. There is no perfect model for collection and processing data regarding hospitalized patients. HDRs, with their advantages and disadvantages, currently represent the best way to perceive the size, type, quality and efficiency of the health care services provided to patients at the secondary and tertiary level.

A retrospective cohort study of adverse events in patients undergoing orthopaedic surgery

Background

This study’s objective was to identify adverse events following common orthopaedic procedures, and to estimate the incidence rates and risks of these events and their associations with age, sex, and comorbidities.

Methods

This retrospective cohort study manually reviewed and extracted electronic medical data on the incidence and predictors of adverse events that occurred within 90 days of the 50 most frequent orthopaedic surgeries at an academic hospital in 2010. We also extracted demographic data, baseline comorbidities, and duration of follow-up (≤90 days). Patients were scored on the Charlson Comorbidity Index (CCI) and the Functional Comorbidity Index (FCI). We estimated incidence rates and risks for all events and associations using regression methods. Prolonged pain 42-days post-surgery was treated as a separate outcome.

Results

We included 1,552 patients; average age was 53.4 years, and 51.7% were female. A total of 1,148 adverse events were identified in 729 patients. The incidence rate of all adverse events was 10 events per 1,000 person-days at risk; 47% of all patients experienced at least one adverse event within 90 days. The most frequent events were prolonged pain (31% of all adverse events) and persistent swelling (7%). We found positive associations between both comorbidity scores and the incidence rate and 90-day risk of all adverse events, excluding pain, adjusting for age and sex (neither of which was associated with adverse events); the association was stronger for the FCI than for the CCI. For total hip arthroplasty (THA) and total knee arthroplasty (TKA), the incidence rate of all adverse events, excluding pain, was positively associated with both comorbidity scores and age; the 90-day risk was positively associate with the FCI score and male sex. The prevalence of prolonged pain at 42 days was greater in patients with higher FCI scores; for THA and TKA only, pain prevalence was greater in those with higher FCI scores and in men.

Conclusions

Adverse events are frequent following common orthopaedic procedures. The incidence is greatest for patients with more functional comorbidities. For THA and TKA procedures, being male and being older are also associated with a greater incidence of adverse events.

Quantifying Geographic Variation in Health Care Outcomes in the United States before and after Risk-Adjustment

BACKGROUND

Despite numerous studies of geographic variation in healthcare cost and utilization at the local, regional, and state levels across the U.S., a comprehensive characterization of geographic variation in outcomes has not been published. Our objective was to quantify variation in US health outcomes in an all-payer population before and after risk-adjustment.

METHODS AND FINDINGS

We used information from 16 independent data sources, including 22 million all-payer inpatient admissions from the Healthcare Cost and Utilization Project (which covers regions where 50% of the U.S. population lives) to analyze 24 inpatient mortality, inpatient safety, and prevention outcomes. We compared outcome variation at state, hospital referral region, hospital service area, county, and hospital levels. Risk-adjusted outcomes were calculated after adjusting for population factors, co-morbidities, and health system factors. Even after risk-adjustment, there exists large geographical variation in outcomes. The variation in healthcare outcomes exceeds the well publicized variation in US healthcare costs. On average, we observed a 2.1-fold difference in risk-adjusted mortality outcomes between top- and bottom-decile hospitals. For example, we observed a 2.3-fold difference for risk-adjusted acute myocardial infarction inpatient mortality. On average a 10.2-fold difference in risk-adjusted patient safety outcomes exists between top and bottom-decile hospitals, including an 18.3-fold difference for risk-adjusted Central Venous Catheter Bloodstream Infection rates. A 3.0-fold difference in prevention outcomes exists between top- and bottom-decile counties on average; including a 2.2-fold difference for risk-adjusted congestive heart failure admission rates. The population, co-morbidity, and health system factors accounted for a range of R2 between 18-64% of variability in mortality outcomes, 3-39% of variability in patient safety outcomes, and 22-70% of variability in prevention outcomes.

CONCLUSION

The amount of variability in health outcomes in the U.S. is large even after accounting for differences in population, co-morbidities, and health system factors. These findings suggest that: 1) additional examination of regional and local variation in risk-adjusted outcomes should be a priority; 2) assumptions of uniform hospital quality that underpin rationale for policy choices (such as narrow insurance networks or antitrust enforcement) should be challenged; and 3) there exists substantial opportunity for outcomes improvement in the US healthcare system.

Surgeon-Level Variation in Postoperative Complications

BACKGROUND

Variation in surgical outcomes is often attributed to patient comorbidities and the severity of underlying disease, but little is known about the extent of variation in outcomes by surgeon and the surgeon factors that are associated with quality.

METHODS

Using the Maryland Health Services Cost Review Commission database, we evaluated risk-adjusted postoperative events by surgeon. Operations studied were elective laparoscopic and open colectomy procedures for colon cancer performed over a 2-year period (July 2012-September 2014). Postoperative events were defined using the Agency for Healthcare Research and Quality Patient Safety Indicators. Surgeons performing fewer than ten procedures during the study period were excluded. Logistic regression and post-estimation were used to calculate an observed-to-expected (O/E) ratio of postoperative complications for each surgeon, adjusting for patient and surgeon characteristics.

RESULTS

A total of 2525 patients underwent an elective colectomy during the study period by 276 surgeons at 44 hospitals. Postoperative complications varied more by surgeon (range 0 to 30.0 %) than by hospital (range 0 to 18.2 %). Surgeon-level use of laparoscopic surgery to perform colectomy ranged from 0 to 100 %. After risk adjustment with patient factors, surgeon experience, surgeon medical school, surgeon gender, and annual surgeon colectomy volume were not associated with postoperative complications. Surgeon use of laparoscopy was the strongest predictor of lower complications (vs fourth quartile of surgeons, first quartile OR = 0.47 (0.26-0.85); second quartile OR = 0.41 (0.22-0.73); and third quartile OR = 0.84 (0.52-1.36).

CONCLUSIONS

Quality metrics in health care have been measured at the hospital level, but a greater quality improvement potential exists at the surgeon level. Awareness of this variation could better inform patients undergoing elective surgery and their referring physicians.

Association of inpatient hospital experience with patient safety indicators: a cross-sectional, Canadian study

OBJECTIVES

There remains concern regarding the use of survey data to assess aspects of healthcare quality. The relationship between patient experience and adverse events as documented by patient safety indicators (PSIs) is a timely research topic. The objectives were to document the association of PSIs and patient experience scores, and to determine risk-adjusted odds of high experience scores versus PSI presence.

SETTING AND PARTICIPANTS

From April 2011 to March 2014, 25 098 patients completed a telephone survey following discharge from 93 inpatient hospitals in Alberta, Canada.

RESEARCH DESIGN

A modified version of the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) instrument was used. Surveys were linked to inpatient records and PSI presence was documented using a validated algorithm.

MEASURES

Three questions about overall hospital, physician and nurse ratings were scored on an 11-point Likert scale from 0 (worst) to 10 (best). Experience was classified as high (9 or 10) versus low (0-8). Demographic/clinical differences between respondents with/without a PSI were assessed. Logistic regression examined the relationship between factors including PSI and experience ratings.

RESULTS

Overall, physician and nurse care was rated high by 61.9%, 73.7% and 66.2% of respondents. 1085 patients (4.3%) had a documented PSI. Most frequent PSIs were haemorrhagic events (n=502; 2.0% of sample), events relating to obstetrics (n=373; 1.5%) and surgical-related events (n=248; 1.0%). Risk-adjusted models showed patients with PSIs had decreased odds of having high overall (OR=0.86; 95% CI 0.75 to 0.97), physician (OR=0.76; 95% CI 0.66 to 0.87) and nurse (OR=0.83; 95% CI 0.73 to 0.94) ratings.

CONCLUSIONS

There is clear evidence that inpatient experience ratings are associated with PSIs, one element of quality of care. Future research, examining individual PSIs and patient experience questions, is warranted, as this may inform targeted quality improvement initiatives.

Development, implementation and evaluation of a patient handoff tool to improve safety in orthopaedic surgery

OBJECTIVE

To develop, implement and test the effect of a handoff tool for orthopaedic trauma residents that reduces adverse events associated with the omission of critical information and the transfer of erroneous information.

DESIGN

Components of this project included a literature review, resident surveys and observations, checklist development and refinement, implementation and evaluation of impact on adverse events through a chart review of a prospective cohort compared with a historical control group.

SETTING

Large teaching hospital.

PARTICIPANTS

Findings of a literature review were presented to orthopaedic residents, epidemiologists, orthopaedic surgeons and patient safety experts in face-to-face meetings, during which we developed and refined the contents of a resident handoff tool. The tool was tested in an orthopaedic trauma service and its impact on adverse events was evaluated through a chart review. The handoff tool was developed and refined during the face-to-face meetings and a pilot implementation. Adverse event data were collected on 127 patients (n = 67 baseline period; n = 60 test period).

INTERVENTION

A handoff tool for use by orthopaedic residents.

MAIN OUTCOME MEASUREMENTS

Adverse events in patients handed off by orthopaedic trauma residents.

RESULTS

After controlling for age, gender and comorbidities, testing resulted in fewer events per person (25-27% reduction; P < 0.10).

CONCLUSIONS

Preliminary evidence suggests that our resident handoff tool may contribute to a decrease in adverse events in orthopaedic patients.

Validation and optimisation of an ICD-10-coded case definition for sepsis using administrative health data

OBJECTIVE

Administrative health data are important for health services and outcomes research. We optimised and validated in intensive care unit (ICU) patients an International Classification of Disease (ICD)-coded case definition for sepsis, and compared this with an existing definition. We also assessed the definition's performance in non-ICU (ward) patients.

SETTING AND PARTICIPANTS

All adults (aged ≥ 18 years) admitted to a multisystem ICU with general medicosurgical ICU care from one of three tertiary care centres in the Calgary region in Alberta, Canada, between 1 January 2009 and 31 December 2012 were included.

RESEARCH DESIGN

Patient medical records were randomly selected and linked to the discharge abstract database. In ICU patients, we validated the Canadian Institute for Health Information (CIHI) ICD-10-CA (Canadian Revision)-coded definition for sepsis and severe sepsis against a reference standard medical chart review, and optimised this algorithm through examination of other conditions apparent in sepsis.

MEASURES

Sensitivity (Sn), specificity (Sp), positive predictive value (PPV) and negative predictive value (NPV) were calculated.

RESULTS

Sepsis was present in 604 of 1001 ICU patients (60.4%). The CIHI ICD-10-CA-coded definition for sepsis had Sn (46.4%), Sp (98.7%), PPV (98.2%) and NPV (54.7%); and for severe sepsis had Sn (47.2%), Sp (97.5%), PPV (95.3%) and NPV (63.2%). The optimised ICD-coded algorithm for sepsis increased Sn by 25.5% and NPV by 11.9% with slightly lowered Sp (85.4%) and PPV (88.2%). For severe sepsis both Sn (65.1%) and NPV (70.1%) increased, while Sp (88.2%) and PPV (85.6%) decreased slightly.

CONCLUSIONS

This study demonstrates that sepsis is highly undercoded in administrative data, thus under-ascertaining the true incidence of sepsis. The optimised ICD-coded definition has a higher validity with higher Sn and should be preferentially considered if used for surveillance purposes.

Accuracy of administrative data for surveillance of healthcare-associated infections: a systematic review

OBJECTIVE

Measuring the incidence of healthcare-associated infections (HAI) is of increasing importance in current healthcare delivery systems. Administrative data algorithms, including (combinations of) diagnosis codes, are commonly used to determine the occurrence of HAI, either to support within-hospital surveillance programmes or as free-standing quality indicators. We conducted a systematic review evaluating the diagnostic accuracy of administrative data for the detection of HAI.

METHODS

Systematic search of Medline, Embase, CINAHL and Cochrane for relevant studies (1995-2013). Methodological quality assessment was performed using QUADAS-2 criteria; diagnostic accuracy estimates were stratified by HAI type and key study characteristics.

RESULTS

57 studies were included, the majority aiming to detect surgical site or bloodstream infections. Study designs were very diverse regarding the specification of their administrative data algorithm (code selections, follow-up) and definitions of HAI presence. One-third of studies had important methodological limitations including differential or incomplete HAI ascertainment or lack of blinding of assessors. Observed sensitivity and positive predictive values of administrative data algorithms for HAI detection were very heterogeneous and generally modest at best, both for within-hospital algorithms and for formal quality indicators; accuracy was particularly poor for the identification of device-associated HAI such as central line associated bloodstream infections. The large heterogeneity in study designs across the included studies precluded formal calculation of summary diagnostic accuracy estimates in most instances.

CONCLUSIONS

Administrative data had limited and highly variable accuracy for the detection of HAI, and their judicious use for internal surveillance efforts and external quality assessment is recommended. If hospitals and policymakers choose to rely on administrative data for HAI surveillance, continued improvements to existing algorithms and their robust validation are imperative.

Validity of administrative data in recording sepsis: a systematic review

Introduction

Administrative health data have been used to study sepsis in large population-based studies. The validity of these study findings depends largely on the quality of the administrative data source and the validity of the case definition used. We systematically reviewed the literature to assess the validity of case definitions of sepsis used with administrative data.

Methods

Embase and MEDLINE were searched for published articles with International Classification of Diseases (ICD) coded data used to define sepsis. Abstracts and full-text articles were reviewed in duplicate. Data were abstracted from all eligible full-text articles, including ICD-9- and/or ICD-10-based case definitions, sensitivity (Sn), specificity (Sp), positive predictive value (PPV) and negative predictive value (NPV).

Results

Of 2,317 individual studies identified, 12 full-text articles met all eligibility criteria. A total of 38 sepsis case definitions were tested, which included over 130 different ICD codes. The most common ICD-9 codes were 038.x, 790.7 and 995.92, and the most common ICD-10 codes were A40.x and A41.x. The PPV was reported in ten studies and ranged from 5.6% to 100%, with a median of 50%. Other tests of diagnostic accuracy were reported only in some studies. Sn ranged from 5.9% to 82.3%; Sp ranged from 78.3% to 100%; and NPV ranged from 62.1% to 99.7%.

Conclusions

The validity of administrative data in recording sepsis varied substantially across individual studies and ICD definitions. Our work may serve as a reference point for consensus towards an improved and harmonized ICD-coded definition of sepsis.

Electronic supplementary material

The online version of this article (doi:10.1186/s13054-015-0847-3) contains supplementary material, which is available to authorized users.

The validity of ICD codes coupled with imaging procedure codes for identifying acute venous thromboembolism using administrative data

The purpose of this study was to evaluate the accuracy of using a combination of International Classification of Diseases (ICD) diagnostic codes and imaging procedure codes for identifying deep vein thrombosis (DVT) and pulmonary embolism (PE) within administrative databases. Information from the Alberta Health (AH) inpatients and ambulatory care administrative databases in Alberta, Canada was obtained for subjects with a documented imaging study result performed at a large teaching hospital in Alberta to exclude venous thromboembolism (VTE) between 2000 and 2010. In 1361 randomly-selected patients, the proportion of patients correctly classified by AH administrative data, using both ICD diagnostic codes and procedure codes, was determined for DVT and PE using diagnoses documented in patient charts as the gold standard. Of the 1361 patients, 712 had suspected PE and 649 had suspected DVT. The sensitivities for identifying patients with PE or DVT using administrative data were 74.83% (95% confidence interval [CI]: 67.01-81.62) and 75.24% (95% CI: 65.86-83.14), respectively. The specificities for PE or DVT were 91.86% (95% CI: 89.29-93.98) and 95.77% (95% CI: 93.72-97.30), respectively. In conclusion, when coupled with relevant imaging codes, VTE diagnostic codes obtained from administrative data provide a relatively sensitive and very specific method to ascertain acute VTE.

Improving accuracy of International Classification of Diseases codes for venous thromboembolism in administrative data

BACKGROUND

Increasingly, clinicians and researchers are using administrative data for clinical and outcomes research. However, they continue to question the accuracy of using International Classification of Diseases 9th Revision (ICD-9) codes alone to capture diagnoses, especially venous thromboembolism (VTE), in administrative data.

OBJECTIVES

We tested the hypothesis that incorporation of treatment data and/or common procedural terminology (CPT) codes could improve accuracy of administrative data in detecting VTE. Research Design Using the Veterans Affairs Central Cancer Registry, we compared three competing algorithms by performing three cross-sectional studies. Algorithm 1 identified patients by ICD-9 codes alone. Algorithm 2 required VTE treatment in addition to ICD-9 codes. Algorithm 3 required a VTE diagnostic CPT code in addition to treatment and ICD-9 criteria.

RESULTS

The accuracy of ICD-9 codes alone for detection of VTE was marginal, with a PPV of 72%. The PPV was improved to 91% after addition of treatment data (algorithm 2). As compared to algorithm 2, addition of CPT codes (algorithm 3) did not significantly increase the accuracy of detecting VTE (PPV 92%), but decreased sensitivity from 72% to 67%.

CONCLUSIONS

Accuracy of VTE detection significantly improved with addition of treatment data to ICD-9 codes. This approach should facilitate use of administrative data to assess the incidence, epidemiology, and outcomes of VTE.

Challenges and remediation for Patient Safety Indicators in the transition to ICD-10-CM

Reporting of hospital adverse events relies on Patient Safety Indicators (PSIs) using International Classification of Diseases, Ninth Edition, Clinical Modification (ICD-9-CM) codes. The US transition to ICD-10-CM in 2015 could result in erroneous comparisons of PSIs. Using the General Equivalent Mappings (GEMs), we compared the accuracy of ICD-9-CM coded PSIs against recommended ICD-10-CM codes from the Centers for Medicaid/Medicare Services (CMS). We further predict their impact in a cohort of 38,644 patients (1,446,581 visits and 399 hospitals). We compared the predicted results to the published PSI related ICD-10-CM diagnosis codes. We provide the first report of substantial hospital safety reporting errors with five direct comparisons from the 23 types of PSIs (transfusion and anesthesia related PSIs). One PSI was excluded from the comparison between code sets due to reorganization, while 15 additional PSIs were inaccurate to a lesser degree due to the complexity of the coding translation. The ICD-10-CM translations proposed by CMS pose impending risks for (1) comparing safety incidents, (2) inflating the number of PSIs, and (3) increasing the variability of calculations attributable to the abundance of coding system translations. Ethical organizations addressing 'data-, process-, and system-focused' improvements could be penalized using the new ICD-10-CM Agency for Healthcare Research and Quality PSIs because of apparent increases in PSIs bearing the same PSI identifier and label, yet calculated differently. Here we investigate which PSIs would reliably transition between ICD-9-CM and ICD-10-CM, and those at risk of under-reporting and over-reporting adverse events while the frequency of these adverse events remain unchanged.

Validating administrative data for the detection of adverse events in older hospitalized patients

Older hospitalized patients are at risk of experiencing adverse events including, but not limited to, hospital-acquired pressure ulcers, fall-related injuries, and adverse drug events. A significant challenge in monitoring and managing adverse events is lack of readily accessible information on their occurrence.

PURPOSE

The objective of this retrospective cross-sectional study was to validate diagnostic codes for pressure ulcers, fall-related injuries, and adverse drug events found in routinely collected administrative hospitalization data.

METHODS

All patients 65 years of age or older discharged between April 1, 2009 and March 31, 2011 from a provincial academic health sciences center in Canada were eligible for inclusion in the validation study. For each of the three types of adverse events, a random sample of 50 patients whose records were positive and 50 patients whose records were not positive for an adverse event was sought for review in the validation study (n=300 records in total). A structured health record review was performed independently by two health care providers with experience in geriatrics, both of whom were unaware of the patient's status with respect to adverse event coding. A physician reviewed 40 records (20 reviewed by each health care provider) to establish interrater agreement.

RESULTS

A total of 39 pressure ulcers, 56 fall-related injuries, and 69 adverse drug events were identified through health record review. Of these, 34 pressure ulcers, 54 fall-related injuries, and 47 adverse drug events were also identified in administrative data. Overall, the diagnostic codes for adverse events had a sensitivity and specificity exceeding 0.67 (95% confidence interval [CI]: 0.56-0.99) and 0.89 (95% CI: 0.72-0.99), respectively.

CONCLUSION

It is feasible and valid to identify pressure ulcers, fall-related injuries, and adverse drug events in older hospitalized patients using routinely collected administrative hospitalization data. The information is relatively inexpensive and easy to access with no impact on clinical staff.

From hospice to hospital: short-term follow-up study of hospice patient outcomes in a US acute care hospital surveillance system

OBJECTIVES

In the USA, there is little systematic evidence about the real-world trajectories of patient medical care after hospice enrolment. The objective of this study was to analyse predictors of the length of stay for hospice patients who were admitted to hospital in a retrospective analysis of the mandatorily reported hospital discharge data.

SETTING

All acute-care hospitals in Florida during 1 January 2010 to 30 June 2012.

PARTICIPANTS

All patients with source of admission coded as 'hospice' (n=2674).

PRIMARY OUTCOME MEASURES

The length of stay and discharge status: (1) died in hospital; (2) discharged back to hospice; (3) discharged to another healthcare facility; and (4) discharged home.

RESULTS

Patients were elderly (median age=81) with a high burden of disease. Almost half died (46%), while the majority of survivors were discharged to hospice (80% of survivors, 44% of total). A minority went to a healthcare facility (5.6%) or to home (5.2%). Only 9.2% received any procedure. Respiratory services were received by 29.4% and 16.8% were admitted to the intensive care unit. The median length of stay was 1 day for those who died. In an adjusted survival model, discharge to a healthcare facility resulted in a 74% longer hospital stay compared with discharge to hospice (event time ratio (ETR)=1.74, 95% CI 1.54 to 1.97 p<0.0001), with 61% longer hospital stays among patients discharged home (ETR=1.61, 95% CI 1.39 to 1.86 p<0.0001). Total financial charges for all patients exceeded $25 million; 10% of patients who appeared to exit hospice incurred 32% of the charges.

CONCLUSIONS

Our results raise significant questions about the ethics and pragmatics of end-of-life medical care, and the intentions and scope of hospices in the USA. Future studies should incorporate prospective linkage of subjective patient-centred data and objective healthcare encounter data.

The transition to ICD-10-CM: challenges for pediatric practice

BACKGROUND AND OBJECTIVES

Diagnostic codes are used widely within health care for billing, quality assessment, and to measure clinical outcomes. The US health care system will transition to the International Classification of Diseases, 10th Revision, Clinical Modification (ICD-10-CM), in October 2015. Little is known about how this transition will affect pediatric practices. The objective of this study was to examine how the transition to ICD-10-CM may result in ambiguity of clinical information and financial disruption for pediatricians.

METHODS

Using a statewide data set from Illinois Medicaid specified for pediatricians, 2708 International Classification of Diseases, Ninth Revision, Clinical Modification, diagnosis codes were identified. Diagnosis codes were categorized into 1 of 5 categories: identity, class-to-subclass, subclass-to-class, convoluted, and no translation. The convoluted and high-cost diagnostic codes (n = 636) were analyzed for accuracy and categorized into "information loss," "overlapping categories," "inconsistent," and "consistent." Finally, reimbursement by Medicaid was calculated for each category.

RESULTS

Twenty-six percent of pediatric diagnosis codes are convoluted, which represents 21% of Illinois Medicaid pediatric patient encounters and 16% of reimbursement. The diagnosis codes represented by information loss (3.6%), overlapping categories (3.2%), and inconsistent (1.2%) represent 8% of Medicaid pediatric reimbursement.

CONCLUSIONS

The potential for financial disruption and administrative errors from 8% of reimbursement diagnosis codes necessitates special attention to these codes in preparing for the transition to ICD-10-CM for pediatric practices.