Accuracy of syndrome definitions based on diagnoses in physician claims

A Multicenter Retrospective Cohort Study to Characterize Patients Hospitalized With Multisystem Inflammatory Syndrome in Adults and Coronavirus Disease 2019 in the United States, 2020-2021

BACKGROUND

The diagnosis of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2)-associated multisystem inflammatory syndrome in adults (MIS-A) requires distinguishing it from acute coronavirus disease 2019 (COVID-19) and may affect clinical management.

METHODS

In this retrospective cohort study, we applied the US Centers for Disease Control and Prevention case definition to identify adults hospitalized with MIS-A at 6 academic medical centers from 1 March 2020 to 31 December 2021. Patients MIS-A were matched by age group, sex, site, and admission date at a 1:2 ratio to patients hospitalized with acute symptomatic COVID-19. Conditional logistic regression was used to compare demographic characteristics, presenting symptoms, laboratory and imaging results, treatments administered, and outcomes between cohorts.

RESULTS

Through medical record review of 10 223 patients hospitalized with SARS-CoV-2-associated illness, we identified 53 MIS-A cases. Compared with 106 matched patients with COVID-19, those with MIS-A were more likely to be non-Hispanic black and less likely to be non-Hispanic white. They more likely had laboratory-confirmed COVID-19 ≥14 days before hospitalization, more likely had positive in-hospital SARS-CoV-2 serologic testing, and more often presented with gastrointestinal symptoms and chest pain. They were less likely to have underlying medical conditions and to present with cough and dyspnea. On admission, patients with MIS-A had higher neutrophil-to-lymphocyte ratio and higher levels of C-reactive protein, ferritin, procalcitonin, and D-dimer than patients with COVID-19. They also had longer hospitalization and more likely required intensive care admission, invasive mechanical ventilation, and vasopressors. The mortality rate was 6% in both cohorts.

CONCLUSIONS

Compared with patients with acute symptomatic COVID-19, adults with MIS-A more often manifest certain symptoms and laboratory findings early during hospitalization. These features may facilitate diagnosis and management.

Multisystem Inflammatory Syndrome in Adults: Case Finding Through Systematic Review of Electronic Medical Records

BACKGROUND

Multisystem inflammatory syndrome in adults (MIS-A) is a severe condition temporally associated with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection.

METHODS

In this retrospective cohort study, we applied the US Centers for Disease Control and Prevention (CDC) case definition to identify diagnosed and undiagnosed MIS-A cases among adults discharged during April 2020-January 2021 from 4 Atlanta, Georgia hospitals affiliated with a single medical center. Non-MIS-A coronavirus disease 2019 (COVID-19) hospitalizations were identified using International Classification of Diseases, Tenth Revision, Clinical Modification encounter code U07.1. We calculated the ratio of MIS-A to COVID-19 hospitalizations, compared demographic characteristics of the 2 cohorts, and described clinical characteristics of MIS-A patients.

RESULTS

We identified 11 MIS-A cases, none of which were diagnosed by the treatment team, and 5755 COVID-19 hospitalizations (ratio 1:523). Compared with patients with COVID-19, patients with MIS-A were more likely to be younger than 50 years (72.7% vs 26.1%, P < .01) and to be non-Hispanic Black (81.8% vs 50.0%, P = .04). Ten patients with MIS-A (90.9%) had at least 1 underlying medical condition. Two MIS-A patients (18.2%) had a previous episode of laboratory-confirmed COVID-19, occurring 37 and 55 days prior to admission. All MIS-A patients developed left ventricular systolic dysfunction. None had documented mucocutaneous involvement. All required intensive care, all received systemic corticosteroids, 8 (72.7%) required mechanical ventilation, 2 (18.2%) required mechanical cardiovascular circulatory support, and none received intravenous immunoglobulin. Two (18.2%) died or were discharged to hospice.

CONCLUSIONS

MIS-A is a severe but likely underrecognized complication of SARS-CoV-2 infection. Improved recognition of MIS-A is needed to quantify its burden and identify populations at highest risk.

Development and validation of an automated emergency department-based syndromic surveillance system to enhance public health surveillance in Yukon: a lower-resourced and remote setting

BACKGROUND

Automated Emergency Department syndromic surveillance systems (ED-SyS) are useful tools in routine surveillance activities and during mass gathering events to rapidly detect public health threats. To improve the existing surveillance infrastructure in a lower-resourced rural/remote setting and enhance monitoring during an upcoming mass gathering event, an automated low-cost and low-resources ED-SyS was developed and validated in Yukon, Canada.

METHODS

Syndromes of interest were identified in consultation with the local public health authorities. For each syndrome, case definitions were developed using published resources and expert elicitation. Natural language processing algorithms were then written using Stata LP 15.1 (Texas, USA) to detect syndromic cases from three different fields (e.g., triage notes; chief complaint; discharge diagnosis), comprising of free-text and standardized codes. Validation was conducted using data from 19,082 visits between October 1, 2018 to April 30, 2019. The National Ambulatory Care Reporting System (NACRS) records were used as a reference for the inclusion of International Classification of Disease, 10th edition (ICD-10) diagnosis codes. The automatic identification of cases was then manually validated by two raters and results were used to calculate positive predicted values for each syndrome and identify improvements to the detection algorithms.

RESULTS

A daily secure file transfer of Yukon's Meditech ED-Tracker system data and an aberration detection plan was set up. A total of six syndromes were originally identified for the syndromic surveillance system (e.g., Gastrointestinal, Influenza-like-Illness, Mumps, Neurological Infections, Rash, Respiratory), with an additional syndrome added to assist in detecting potential cases of COVID-19. The positive predictive value for the automated detection of each syndrome ranged from 48.8-89.5% to 62.5-94.1% after implementing improvements identified during validation. As expected, no records were flagged for COVID-19 from our validation dataset.

CONCLUSIONS

The development and validation of automated ED-SyS in lower-resourced settings can be achieved without sophisticated platforms, intensive resources, time or costs. Validation is an important step for measuring the accuracy of syndromic surveillance, and ensuring it performs adequately in a local context. The use of three different fields and integration of both free-text and structured fields improved case detection.

Consultations for gastroenteritis in general practice and out-of-hours services in Norway 2006-15

BACKGROUND

Most of the patients with gastroenteritis seeking health care services are managed in primary care; yet, little is known about these consultations. Syndromic-based surveillance of gastrointestinal infections is used in several countries, including Norway.

AIM

To investigate the extent of, and explore characteristics associated with, consultations for gastroenteritis in primary care and to compare consultations in daytime general practice and out-of-hours (OOH) services in Norway.

DESIGN AND SETTING

Registry-based study using reimbursement claims data from all consultations in general practice and OOH services in Norway over the 10-year period, 2006-15.

METHODS

The main outcome variable was whether the consultation took place in general practice or OOH services. Possible associations with patient age and sex, time and use of point-of-care C-reactive protein (CRP) testing and sickness certificate issuing were investigated.

RESULTS

Gastroenteritis consultations (n = 1 281 048) represented 0.9% of all consultations in primary care (n = 140 199 637), of which 84.4% were conducted in general practice and 15.6% in OOH services. Young children and young adults dominated among the patients. Point-of-care CRP testing was used in 36.1% of the consultations. Sickness certificates were issued in 43.6% of consultations with patients in working age. Age-specific time variations in consultation frequencies peaking in winter months were observed.

CONCLUSIONS

The proportion of gastroenteritis consultations was higher in the OOH services when compared with daytime general practice. Young children and young adults dominated among the patients. The seasonal variation in consultation frequency is similar to that shown for gastroenteritis caused by norovirus.

Patient and health system factors associated with hospital readmission in older adults without cognitive impairment

OBJECTIVE

To study the factors associated with hospital readmission.

METHODS

Data used in this study came from a population-based survey of older adults without cognitive impairment. Cox regression was used to assess the factors associated with readmission within a 2-year follow-up period. According to Andersen's model of healthcare seeking behavior, study variables considered included predisposing, enabling and need factors at the individual and health system levels.

RESULTS

Of the 433 participants with an index hospitalization, 97% were discharged with a physical and 3% with a psychiatric disorder. During follow-up, 29% (128/433) were readmitted with a median time to readmission reaching 83 days. The risk of readmission was associated with the following: age, marital status, attraction index of the region of residence for psychiatric services, the presence of an anxio-depressive and other mental disorder, as well as a disorder of the musculoskeletal system. The presence of a physical and psychiatric comorbidity was also associated with readmission.

CONCLUSIONS

Post-discharge follow-up of vulnerable populations with a history of mental disorders and improved availability of psychiatric services in the community are associated with a reduced risk of readmission.

How well do ICD-9 physician claim diagnostic codes identify confirmed pertussis cases in Alberta, Canada? A Canadian Immunization Research Network (CIRN) Study

Background

Rates of Bordetella pertussis have been increasing in Alberta, Canada despite vaccination programs. Waning immunity from existing acellular component vaccines may be contributing to this. Vaccine effectiveness can be estimated using a variety of data sources including diagnostic codes from physician billing claims, public health records, reportable disease and laboratory databases. We sought to determine if diagnostic codes from billing claims (administrative data) are adequately sensitive and specific to identify pertussis cases among patients who had undergone disease-specific laboratory testing.

Methods

Data were extracted for 2004–2014 from a public health communicable disease database that contained data on patients under investigation for B. pertussis (both those who had laboratory tests and those who were epidemiologically linked to laboratory-confirmed cases) in Alberta, Canada. These were deterministically linked using a unique lifetime person identifier to the provincial billing claims database, which contains International Classification of Disease version 9 (ICD-9) diagnostic codes for physician visits. We examined visits within 90 days of laboratory testing. ICD-9 codes 033 (whooping cough), 033.0 (Bordetella pertussis), 033.1 (B. parapertussis), 033.8 (whooping cough, other specified organism), and 033.9 (whooping cough, other unspecified organism) in any of the three diagnostic fields for a claim were classified as being pertussis-specific codes. We calculated sensitivity, specificity, positive (PPV) and negative (NPV) predictive values.

Results

We identified 22,883 unique patients under investigation for B. pertussis. Of these, 22,095 underwent laboratory testing. Among those who had a laboratory test, 2360 tested positive for pertussis. The sensitivity of a pertussis-specific ICD-9 code for identifying a laboratory-confirmed case was 38.6%, specificity was 76.9%, PPV was 16.0%, and NPV was 91.6%.

Conclusion

ICD-9 codes from physician billing claims data have low sensitivity and moderate specificity to identify laboratory-confirmed pertussis among persons tested for pertussis.

Electronic supplementary material

The online version of this article (doi:10.1186/s12913-017-2321-1) contains supplementary material, which is available to authorized users.

Systematic review of the accuracy of antibody tests used to screen asymptomatic adults for hepatitis C infection

BACKGROUND

Several expert groups, including the United States Preventive Services Task Force and the Canadian Task Force on Preventive Health Care, have recently examined or are currently examining whether primary care physicians should screen asymptomatic adults for hepatitis C virus (HCV) infection. To inform decision-making on HCV screening, we performed a systematic review of the accuracy of antibody tests compared with other immunoassays and RNA detection for screening asymptomatic adults for HCV infection in Canada.

METHODS

MEDLINE and Embase databases were searched from 1990 to 2016; resulting citations were uploaded into DistillerSR and independently screened by 2 reviewers. Original research studies, systematic reviews and meta-analyses were eligible for inclusion. At least 80% of the study population had to be asymptomatic, nonpregnant, treatment-naïve adults with unknown liver enzyme values and unknown HCV status. Risk of bias was assessed with the use of the Quality Assessment of Diagnostic Accuracy Studies version 2 (QUADAS-2) tool; the quality of the body of evidence was assessed by means of GRADE (Grading of Recommendations Assessment, Development and Evaluation) methodology.

RESULTS

Of 1537 articles identified, 81 underwent full-text review, and 9 studies met the inclusion criteria. Compared with RNA detection, the sensitivity of the third-generation enzyme-linked immunosorbent assay was variable (61.0%-81.8%), and its specificity was high (97.5%-99.7%). As expected, there were more false-positive results when comparing antibody tests to RNA detection than to other immunoassays. Our GRADE assessment suggested that there was a high concern for risk of bias, particularly verification bias, and substantial inconsistency between studies in terms of their design.

INTERPRETATION

More research is needed to better characterize the accuracy of antibody tests used to screen for HCV infection in the general population. Jurisdictions that recently adopted birth cohort screening for HCV infection are encouraged to evaluate and report on the accuracy of HCV screening tests and screening benefits and harms. PROSPERO registration: no. CRD42016039710.

Validation of Diagnostic Groups Based on Health Care Utilization Data Should Adjust for Sampling Strategy

Supplemental Digital Content is available in the text.

Objective:

Valid measurement of outcomes such as disease prevalence using health care utilization data is fundamental to the implementation of a “learning health system.” Definitions of such outcomes can be complex, based on multiple diagnostic codes. The literature on validating such data demonstrates a lack of awareness of the need for a stratified sampling design and corresponding statistical methods. We propose a method for validating the measurement of diagnostic groups that have: (1) different prevalences of diagnostic codes within the group; and (2) low prevalence.

Methods:

We describe an estimation method whereby: (1) low-prevalence diagnostic codes are oversampled, and the positive predictive value (PPV) of the diagnostic group is estimated as a weighted average of the PPV of each diagnostic code; and (2) claims that fall within a low-prevalence diagnostic group are oversampled relative to claims that are not, and bias-adjusted estimators of sensitivity and specificity are generated.

Application:

We illustrate our proposed method using an example from population health surveillance in which diagnostic groups are applied to physician claims to identify cases of acute respiratory illness.

Conclusions:

Failure to account for the prevalence of each diagnostic code within a diagnostic group leads to the underestimation of the PPV, because low-prevalence diagnostic codes are more likely to be false positives. Failure to adjust for oversampling of claims that fall within the low-prevalence diagnostic group relative to those that do not leads to the overestimation of sensitivity and underestimation of specificity.

Emergency Medical Text Classifier: New system improves processing and classification of triage notes

Objective

Automated syndrome classification aims to aid near real-time syndromic surveillance to serve as an early warning system for disease outbreaks, using Emergency Department (ED) data. We present a system that improves the automatic classification of an ED record with triage note into one or more syndrome categories using the vector space model coupled with a ‘learning’ module that employs a pseudo-relevance feedback mechanism. Materials and Methods: Terms from standard syndrome definitions are used to construct an initial reference dictionary for generating the syndrome and triage note vectors. Based on cosine similarity between the vectors, each record is classified into a syndrome category. We then take terms from the top-ranked records that belong to the syndrome of interest as feedback. These terms are added to the reference dictionary and the process is repeated to determine the final classification. The system was tested on two different datasets for each of three syndromes: Gastro-Intestinal (GI), Respiratory (Resp) and Fever-Rash (FR). Performance was measured in terms of sensitivity (Se) and specificity (Sp). Results: The use of relevance feedback produced high values of sensitivity and specificity for all three syndromes in both test sets: GI: 90% and 71%, Resp: 97% and 73%, FR: 100% and 87%, respectively, in test set 1, and GI: 88% and 69%, Resp: 87% and 61%, FR: 97% and 71%, respectively, in test set 2. Conclusions: The new system for pre-processing and syndromic classification of ED records with triage notes achieved improvements in Se and Sp. Our results also demonstrate that the system can be tuned to achieve different levels of performance based on user requirements.

Emergency department and 'Google flu trends' data as syndromic surveillance indicators for seasonal influenza

We evaluated syndromic indicators of influenza disease activity developed using emergency department (ED) data - total ED visits attributed to influenza-like illness (ILI) ('ED ILI volume') and percentage of visits attributed to ILI ('ED ILI percent') - and Google flu trends (GFT) data (ILI cases/100 000 physician visits). Congruity and correlation among these indicators and between these indicators and weekly count of laboratory-confirmed influenza in Manitoba was assessed graphically using linear regression models. Both ED and GFT data performed well as syndromic indicators of influenza activity, and were highly correlated with each other in real time. The strongest correlations between virological data and ED ILI volume and ED ILI percent, respectively, were 0·77 and 0·71. The strongest correlation of GFT was 0·74. Seasonal influenza activity may be effectively monitored using ED and GFT data.

Diagnosis of influenza: only a problem of coding?

OBJECTIVE

To evaluate the characteristics of hospital discharge diagnoses of influenza measured by using specific International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9 CM) codes.

SUBJECTS AND METHODS

The study was conducted for the 3 years 2007, 2008 and 2011. The database included (1) administrative and clinical data on Sicilian patients admitted to acute care hospitals and (2) data from the influenza virological surveillance of 10 European countries (FluNet database). All Sicilian patients diagnosed with at least 1 ICD-9 CM code for influenza (487.0, 487.1 and 487.9) were considered influenza cases.

RESULTS

Overall, 2,880 patients with an ICD-9 CM code attributable to influenza were hospitalized in Sicily: 2,119 (73.6%) were admitted from November to April, whereas 761 (26.4%) were admitted from May to October. In the 3 years studied, the analyzed European influenza surveillance systems recorded a peak of laboratory-confirmed influenza activity from November to April with 36,753 (99.7%) influenza cases, whereas only 124 cases (0.3%) were observed from May to October.

CONCLUSIONS

In Sicily, more than one quarter of all hospital admissions with an ICD-9 CM code for influenza were observed in the months with a negligible circulation of influenza viruses. Our findings show that several hospital discharge records included ICD-9 CM codes for influenza with low levels of sensitivity, specificity and/or appropriateness for clinical information and support the need for improving medical education on the epidemiology and hospital management of influenza cases.

Patient, physician, encounter, and billing characteristics predict the accuracy of syndromic surveillance case definitions

BACKGROUND

Syndromic surveillance systems are plagued by high false-positive rates. In chronic disease monitoring, investigators have identified several factors that predict the accuracy of case definitions based on diagnoses in administrative data, and some have even incorporated these predictors into novel case detection methods, resulting in a significant improvement in case definition accuracy. Based on findings from these studies, we sought to identify physician, patient, encounter, and billing characteristics associated with the positive predictive value (PPV) of case definitions for 5 syndromes (fever, gastrointestinal, neurological, rash, and respiratory (including influenza-like illness)).

METHODS

The study sample comprised 4,330 syndrome-positive visits from the claims of 1,098 randomly-selected physicians working in Quebec, Canada in 2005-2007. For each visit, physician-facilitated chart review was used to assess whether the same syndrome was present in the medical chart (gold standard). We used multivariate logistic regression analyses to estimate the association between claim-chart agreement about the presence of a syndrome and physician, patient, encounter, and billing characteristics.

RESULTS

The likelihood of the medical chart agreeing with the physician claim about the presence of a syndrome was higher when the treating physician had billed many visits for the same syndrome recently (ORper 10 visit, 1.05; 95% CI, 1.01-1.08), had a lower workload (ORper 10 claims, 0.93; 95% CI, 0.90-0.97), and when the patient was younger (ORper 5 years of age, 0.96; 95% CI, 0.94-0.97), and less socially deprived (ORmost versus least deprived, 0.76; 95% CI, 0.60-0.95).

CONCLUSIONS

Many physician, patient, encounter, and billing characteristics associated with the PPV of surveillance case definition are accessible to public health, and could be used to reduce false-positive alerts by surveillance systems, either by focusing on the data most likely to be accurate, or by adjusting the observed data for known biases in diagnosis reporting and performing surveillance using the adjusted values.

Raising the index of suspicion for elder abuse: cognitive impairment, falls, and injury patterns in the emergency department

Cognitive impairment limits older adults' abilities to advocate for themselves, thus heightening their risk for abuse. Some older adults with cognitive impairments who seek emergency department (ED) services may present with injuries suspicious of abuse. A portion of these injuries may be erroneously attributed to accidents such as falls. A retrospective analysis of 2 years of ED data using International Classification of Diseases, Ninth Revision (ICD-9) codes was conducted focusing on characteristics of injuries sustained by persons with co-occurring cognitive impairment and fall status. Cognitive impairment was not significantly related to falls (P = .533). Findings suggest that persons with cognitive impairment have unique injury patterns based on fall status, which has implications for elder abuse screening. Injuries for persons with no fall history included injury to the upper limb (P = .004), contusions (P = .012), and open wounds (P = .000). An increased recognition of common injuries in older adults can aid in elder abuse assessment by providing a reference point for uncommon injuries.

Physicians' response to computerised alerts for psychotropic drugs in older persons: a multilevel analysis of the associated alert, patient and physician characteristics

OBJECTIVE

Computerised drug alerts are expected to reduce patients' risk of adverse drug events. However, physicians over-ride most drug alerts, because they believe that the benefit exceeds the risk. The purpose of this study was to determine the drug alert, patient and physician characteristics associated with the: (1) occurrence of psychotropic drug alerts for elderly patients and the (2) response to these alerts by their primary care physicians.

SETTING

Primary care, Quebec, Canada.

DESIGN

Prospective cohort study.

PARTICIPANTS

Sixty-one physicians using an electronic prescribing and drug alert decision-support system in their practice, and 3413 elderly patients using psychotropic drugs. PRIMARY AND SECONDARY MEASURES: Psychotropic drug class, alert severity, patient risk for fall injuries and physician experience, practice volume and computer use were evaluated in relationship to the likelihood of having: (1) a psychotropic drug alert, (2) the prescription revised in response to an alert. Cluster-adjusted alternating logistic regression was used to assess multilevel predictors of alert occurrence and response.

RESULTS

In total 13 080 psychotropic drug alerts were generated in 8931 visits. Alerts were more likely to be generated for male patients at higher risk of fall-related injury and for physicians who established the highest alert threshold. In 9.9% of alerts seen, the prescription was revised. The highest revision rate was for antipsychotic alerts (22.6%). Physicians were more likely to revise prescriptions for severe alerts (OR 2.03; 95%CI 1.39 to 2.98), if patients had cognitive impairment (OR 1.95; 95%CI 1.13 to 3.36), and if they made more visits to their physician (OR 1.05 per 5 visits; 95%CI 1 to 1.09).

CONCLUSIONS

Physicians view and respond to a small proportion of alerts, mainly for higher-risk patients. To reduce the risk of psychotropic drug-related fall injuries, a new generation of evidence-based drug alerts should be developed.