Validity of ICD codes to identify do-not-resuscitate orders among older adults with heart failure: A single center study

BACKGROUND

Observational research on the advance care planning (ACP) process is limited by a lack of easily accessible ACP variables in many large datasets. The objective of this study was to determine whether International Classification of Disease (ICD) codes for do-not-resuscitate (DNR) orders are valid proxies for the presence of a DNR recorded in the electronic medical record (EMR).

METHODS

We studied 5,016 patients over the age of 65 who were admitted to a large, mid-Atlantic medical center with a primary diagnosis of heart failure. DNR orders were identified in billing records from ICD-9 and ICD-10 codes. DNR orders were also identified in the EMR by a manual search of physician notes. Sensitivity, specificity, positive predictive value and negative predictive value were calculated as well as measures of agreement and disagreement. In addition, estimates of associations with mortality and costs were calculated using the DNR documented in EMR and the DNR proxy identified in ICD codes.

RESULTS

Relative to the gold standard of the EMR, DNR orders identified in ICD codes had an estimated sensitivity of 84.6%, specificity of 96.6%, positive predictive value of 90.5%, and negative predictive value of 94.3%. The estimated kappa statistic was 0.83, although McNemar's test suggested there was some systematic disagreement between the DNR from ICD codes and the EMR.

CONCLUSIONS

ICD codes appear to provide a reasonable proxy for DNR orders among hospitalized older adults with heart failure. Further research is necessary to determine if billing codes can identify DNR orders in other populations.

Analysis of root causes of problems affecting the quality of hospital administrative data: A systematic review and Ishikawa diagram

INTRODUCTION

Administrative hospital databases represent an important tool for hospital financing in many national health systems and are also an important data source for clinical, epidemiological and health services research. Therefore, the data quality of such databases is of utmost importance. This paper aims to present a systematic review of root causes of data quality problems affecting administrative hospital data, creating a catalogue of potential issues for data quality analysts to explore.

METHODS

The MEDLINE and Scopus databases were searched using inclusion criteria based on two following concept blocks: (1) administrative hospital databases and (2) data quality. Studies' titles and abstracts were screened by two reviewers independently. Three researchers independently selected the screened studies based on their full texts and then extracted the potential root causes inferred from them. These were subsequently classified according to the Ishikawa model based on 6 categories: "Personnel", "Material", "Method", "Machine", "Mission" and "Management".

RESULTS

The result of our investigation and the contribution of this paper is a classification of the potential (105) root causes found through a systematic review of the 77 relevant studies we have identified and analyzed. The result was represented by an Ishikawa diagram. Most of the root causes (25.7%) were associated with the category "Personnel" - people's knowledge, preferences, education and culture, mostly related to clinical coders and health care providers activities. The quality of hospital documentation, within category "Material", and aspects related to financial incentives or disincentives, within category "Mission", were also frequently cited in the literature as relevant root causes for data quality issues.

CONCLUSIONS

The resultant catalogue of root causes, systematized using the Ishikawa framework, provides a compilation of potential root causes of data quality issues to be considered prior to reusing these data and that can point to actions aimed at improving data quality.

Survivorship in Colorectal Cancer: A Cohort Study of the Patterns and Documented Content of Follow-Up Visits

Survivors of colorectal cancer (CRC) may experience a range of physical, psychosocial, and practical challenges as a consequence of their diagnosis. We assessed the patterns and documented content of follow-up visits within the first three years following treatment, in comparison to survivorship care guidelines. Survivors with stage I-III CRC who underwent curative resection at Peter MacCallum Cancer Centre from July 2015 to January 2018 were followed for up to 1080 days. Patterns of follow-up were calculated by recording the date and specialty of each visit; documented content was assessed using a study-specific audit tool for the first year (360 days) of follow-up. Forty-eight survivors comprised the study population, 34 of whom (71%) attended the recommended two to four follow-up visits in their first year. Visit notes documented new symptoms (96%), physical changes (85%), physical examination (63%), and investigations (56%–90%); none had documented discussions of screening for other primary cancers, or regular health checks and/or screening. Each survivor had at least one outpatient letter that was sent to their primary care physician, but responsibilities were not adequately defined (31%). Although survivors had regular follow-up in their first year, documentation did not consistently address aspects of wider survivorship care.

Reliability of comorbidity scores derived from administrative data in the tertiary hospital intensive care setting: a cross-sectional study

Background

Hospital reporting systems commonly use administrative data to calculate comorbidity scores in order to provide risk-adjustment to outcome indicators.

Objective

We aimed to elucidate the level of agreement between administrative coding data and medical chart review for extraction of comorbidities included in the Charlson Comorbidity Index (CCI) and Elixhauser Index (EI) for patients admitted to the intensive care unit of a university-affiliated hospital.

Method

We conducted an examination of a random cross-section of 100 patient episodes over 12 months (July 2012 to June 2013) for the 19 CCI and 30 EI comorbidities reported in administrative data and the manual medical record system. CCI and EI comorbidities were collected in order to ascertain the difference in mean indices, detect any systematic bias, and ascertain inter-rater agreement.

Results

We found reasonable inter-rater agreement (kappa (κ) coefficient ≥0.4) for cardiorespiratory and oncological comorbidities, but little agreement (κ<0.4) for other comorbidities. Comorbidity indices derived from administrative data were significantly lower than from chart review: −0.81 (95% CI − 1.29 to − 0.33; p=0.001) for CCI, and −2.57 (95% CI −4.46 to −0.68; p=0.008) for EI.

Conclusion

While cardiorespiratory and oncological comorbidities were reliably coded in administrative data, most other comorbidities were under-reported and an unreliable source for estimation of CCI or EI in intensive care patients. Further examination of a large multicentre population is required to confirm our findings.

Accuracy of lung cancer ICD-9-CM codes in Umbria, Napoli 3 Sud and Friuli Venezia Giulia administrative healthcare databases: a diagnostic accuracy study

OBJECTIVES

To assess the accuracy of International Classification of Diseases 9th Revision-Clinical Modification (ICD-9-CM) codes in identifying subjects with lung cancer.

DESIGN

A cross-sectional diagnostic accuracy study comparing ICD-9-CM 162.x code (index test) in primary position with medical chart (reference standard). Case ascertainment was based on the presence of a primary nodular lesion in the lung and cytological or histological documentation of cancer from a primary or metastatic site.

SETTING

Three operative units: administrative databases from Umbria Region (890 000 residents), ASL Napoli 3 Sud (NA) (1 170 000 residents) and Friuli Venezia Giulia (FVG) Region (1 227 000 residents).

PARTICIPANTS

Incident subjects with lung cancer (n=386) diagnosed in primary position between 2012 and 2014 and a population of non-cases (n=280).

OUTCOME MEASURES

Sensitivity, specificity and positive predictive value (PPV) for 162.x code.

RESULTS

130 cases and 94 non-cases were randomly selected from each database and the corresponding medical charts were reviewed. Most of the diagnoses for lung cancer were performed in medical departments.True positive rates were high for all the three units. Sensitivity was 99% (95% CI 95% to 100%) for Umbria, 97% (95% CI 91% to 100%) for NA, and 99% (95% CI 95% to 100%) for FVG. The false positive rates were 24%, 37% and 23% for Umbria, NA and FVG, respectively. PPVs were 79% (73% to 83%)%) for Umbria, 58% (53% to 63%)%) for NA and 79% (73% to 84%)%) for FVG.

CONCLUSIONS

Case ascertainment for lung cancer based on imaging or endoscopy associated with histological examination yielded an excellent sensitivity in all the three administrative databases. PPV was moderate for Umbria and FVG but lower for NA.

Interrater reliability of a national acute myocardial infarction register

BACKGROUND

Disease-specific registers may be used for measuring and improving healthcare and patient outcomes, and for disease surveillance and research, provided they contain valid and reliable data. The aim of this study was to assess the interrater reliability of all variables in a national myocardial infarction register.

METHODS

We randomly selected 280 patients who had been enrolled from 14 hospitals to the Norwegian Myocardial Infarction Register during the year 2013. Experienced audit nurses, who were blinded to the data about the 280 patients already in the register, completed the Norwegian Myocardial Infarction paper forms for 240 patients by review of medical records. We then extracted all registered data on the same patients from the Norwegian Myocardial Infarction Register. To compare the interrater reliability between the register and the audit nurses, we calculated intraclass correlations coefficient for continuous variables, Cohen's kappa and Gwet's first agreement coefficient (AC1) for nominal variables, and quadratic weighted Cohen's kappa and Gwet's second AC for ordinal variables.

RESULTS

We found excellent (AC1 >0.80) or good (AC1 0.61-0.80) agreement for most variables, including date and time variables, medical history, investigations and treatments during hospitalization, medication at discharge, and ST-segment elevation or non-ST-segment elevation acute myocardial infarction. However, only moderate agreement (AC1 0.41-0.60) was found for family history of coronary heart disease, diagnostic electrocardiography, and complications during hospitalization, whereas fair agreement (AC1 0.21-0.40) was found for acute myocardial infarction location. A high percentage of missing data was found for symptom onset, family history, body mass index, infarction location, and new Q-wave.

CONCLUSION

Most variables in Norwegian Myocardial Infarction Register had excellent or good reliability. However, some important variables had lower reliability than expected or had missing data. Precise definitions of data elements and proper training of data abstractors are necessary to ensure that clinical registries contain valid and reliable data.

From bed to bench: bridging from informatics practice to theory: an exploratory analysis

BACKGROUND

In 2009, Applied Clinical Informatics (ACI)--focused on applications in clinical informatics--was launched as a companion journal to Methods of Information in Medicine (MIM). Both journals are official journals of the International Medical Informatics Association.

OBJECTIVES

To explore which congruencies and interdependencies exist in publications from theory to practice and from practice to theory and to determine existing gaps. Major topics discussed in ACI and MIM were analyzed. We explored if the intention of publishing companion journals to provide an information bridge from informatics theory to informatics practice and vice versa could be supported by this model. In this manuscript we will report on congruencies and interdependences from practice to theory and on major topics in MIM.

METHODS

Retrospective, prolective observational study on recent publications of ACI and MIM. All publications of the years 2012 and 2013 were indexed and analyzed.

RESULTS

Hundred and ninety-six publications were analyzed (ACI 87, MIM 109). In MIM publications, modelling aspects as well as methodological and evaluation approaches for the analysis of data, information, and knowledge in biomedicine and health care were frequently raised - and often discussed from an interdisciplinary point of view. Important themes were ambient-assisted living, anatomic spatial relations, biomedical informatics as scientific discipline, boosting, coding, computerized physician order entry, data analysis, grid and cloud computing, health care systems and services, health-enabling technologies, health information search, health information systems, imaging, knowledge-based decision support, patient records, signal analysis, and web science. Congruencies between journals could be found in themes, but with a different focus on content. Interdependencies from practice to theory, found in these publications, were only limited.

CONCLUSIONS

Bridging from informatics theory to practice and vice versa remains a major component of successful research and practice as well as a major challenge.

From bench to bed: bridging from informatics theory to practice. An exploratory analysis

BACKGROUND

In 2009, the journal Applied Clinical Informatics (ACI) commenced publication. Focused on applications in clinical informatics, ACI was intended to be a companion journal to METHODS of Information in Medicine (MIM). Both journals are official journals of IMIA, the International Medical Informatics Association.

OBJECTIVES

To explore, after five years, which congruencies and interdependencies exist in publications of these journals and to determine if gaps exist. To achieve this goal, major topics discussed in ACI and in MIM had to be analysed. Finally, we wanted to explore, whether the intention of publishing these companion journals to provide an information bridge from informatics theory to informatics practice and from practice to theory could be supported by this model. In this manuscript we will report on congruencies and interdependencies from practise to theory and on major topis in ACI. Further results will be reported in a second paper.

METHODS

Retrospective, prolective observational study on recent publications of ACI and MIM. All publications of the years 2012 and 2013 from these journals were indexed and analysed.

RESULTS

Hundred and ninety-six publications have been analysed (87 ACI, 109 MIM). In ACI publications addressed care coordination, shared decision support, and provider communication in its importance for complex patient care and safety and quality. Other major themes included improving clinical documentation quality and efficiency, effectiveness of clinical decision support and alerts, implementation of health information technology systems including discussion of failures and succeses. An emerging topic in the years analyzed was a focus on health information technology to predict and prevent hospital admissions and managing population health including the application of mobile health technology. Congruencies between journals could be found in themes, but with different focus in its contents. Interdependencies from practise to theory found in these publications, were only limited.

CONCLUSIONS

Bridging from informatics theory to practise and vice versa remains a major component of successful research and practise as well as a major challenge.

Clinical data integration model. Core interoperability ontology for research using primary care data

INTRODUCTION

This article is part of the Focus Theme of METHODS of Information in Medicine on "Managing Interoperability and Complexity in Health Systems".

BACKGROUND

Primary care data is the single richest source of routine health care data. However its use, both in research and clinical work, often requires data from multiple clinical sites, clinical trials databases and registries. Data integration and interoperability are therefore of utmost importance.

OBJECTIVES

TRANSFoRm's general approach relies on a unified interoperability framework, described in a previous paper. We developed a core ontology for an interoperability framework based on data mediation. This article presents how such an ontology, the Clinical Data Integration Model (CDIM), can be designed to support, in conjunction with appropriate terminologies, biomedical data federation within TRANSFoRm, an EU FP7 project that aims to develop the digital infrastructure for a learning healthcare system in European Primary Care.

METHODS

TRANSFoRm utilizes a unified structural / terminological interoperability framework, based on the local-as-view mediation paradigm. Such an approach mandates the global information model to describe the domain of interest independently of the data sources to be explored. Following a requirement analysis process, no ontology focusing on primary care research was identified and, thus we designed a realist ontology based on Basic Formal Ontology to support our framework in collaboration with various terminologies used in primary care.

RESULTS

The resulting ontology has 549 classes and 82 object properties and is used to support data integration for TRANSFoRm's use cases. Concepts identified by researchers were successfully expressed in queries using CDIM and pertinent terminologies. As an example, we illustrate how, in TRANSFoRm, the Query Formulation Workbench can capture eligibility criteria in a computable representation, which is based on CDIM.

CONCLUSION

A unified mediation approach to semantic interoperability provides a flexible and extensible framework for all types of interaction between health record systems and research systems. CDIM, as core ontology of such an approach, enables simplicity and consistency of design across the heterogeneous software landscape and can support the specific needs of EHR-driven phenotyping research using primary care data.

Development of an automated, real time surveillance tool for predicting readmissions at a community hospital

BACKGROUND

The Centers for Medicare and Medicaid Services' Readmissions Reduction Program adjusts payments to hospitals based on 30-day readmission rates for patients with acute myocardial infarction, heart failure, and pneumonia. This holds hospitals accountable for a complex phenomenon about which there is little evidence regarding effective interventions. Further study may benefit from a method for efficiently and inexpensively identifying patients at risk of readmission. Several models have been developed to assess this risk, many of which may not translate to a U.S. community hospital setting.

OBJECTIVE

To develop a real-time, automated tool to stratify risk of 30-day readmission at a semirural community hospital.

METHODS

A derivation cohort was created by extracting demographic and clinical variables from the data repository for adult discharges from calendar year 2010. Multivariate logistic regression identified variables that were significantly associated with 30-day hospital readmission. Those variables were incorporated into a formula to produce a Risk of Readmission Score (RRS). A validation cohort from 2011 assessed the predictive value of the RRS. A SQL stored procedure was created to calculate the RRS for any patient and publish its value, along with an estimate of readmission risk and other factors, to a secure intranet site.

RESULTS

Eleven variables were significantly associated with readmission in the multivariate analysis of each cohort. The RRS had an area under the receiver operating characteristic curve (c-statistic) of 0.74 (95% CI 0.73-0.75) in the derivation cohort and 0.70 (95% CI 0.69-0.71) in the validation cohort.

CONCLUSION

Clinical and administrative data available in a typical community hospital database can be used to create a validated, predictive scoring system that automatically assigns a probability of 30-day readmission to hospitalized patients. This does not require manual data extraction or manipulation and uses commonly available systems. Additional study is needed to refine and confirm the findings.