Development and validation of an algorithm to identify patients newly diagnosed with HIV infection from electronic health records

A general framework for developing computable clinical phenotype algorithms

OBJECTIVE

To present a general framework providing high-level guidance to developers of computable algorithms for identifying patients with specific clinical conditions (phenotypes) through a variety of approaches, including but not limited to machine learning and natural language processing methods to incorporate rich electronic health record data.

MATERIALS AND METHODS

Drawing on extensive prior phenotyping experiences and insights derived from 3 algorithm development projects conducted specifically for this purpose, our team with expertise in clinical medicine, statistics, informatics, pharmacoepidemiology, and healthcare data science methods conceptualized stages of development and corresponding sets of principles, strategies, and practical guidelines for improving the algorithm development process.

RESULTS

We propose 5 stages of algorithm development and corresponding principles, strategies, and guidelines: (1) assessing fitness-for-purpose, (2) creating gold standard data, (3) feature engineering, (4) model development, and (5) model evaluation.

DISCUSSION AND CONCLUSION

This framework is intended to provide practical guidance and serve as a basis for future elaboration and extension.

Generalizable pipeline for constructing HIV risk prediction models across electronic health record systems

OBJECTIVE

The HIV epidemic remains a significant public health issue in the United States. HIV risk prediction models could be beneficial for reducing HIV transmission by helping clinicians identify patients at high risk for infection and refer them for testing. This would facilitate initiation on treatment for those unaware of their status and pre-exposure prophylaxis for those uninfected but at high risk. Existing HIV risk prediction algorithms rely on manual construction of features and are limited in their application across diverse electronic health record systems. Furthermore, the accuracy of these models in predicting HIV in females has thus far been limited.

MATERIALS AND METHODS

We devised a pipeline for automatic construction of prediction models based on automatic feature engineering to predict HIV risk and tested our pipeline on a local electronic health records system and a national claims data. We also compared the performance of general models to female-specific models.

RESULTS

Our models obtain similarly good performance on both health record datasets despite difference in represented populations and data availability (AUC = 0.87). Furthermore, our general models obtain good performance on females but are also improved by constructing female-specific models (AUC between 0.81 and 0.86 across datasets).

DISCUSSION AND CONCLUSIONS

We demonstrated that flexible construction of prediction models performs well on HIV risk prediction across diverse health records systems and perform as well in predicting HIV risk in females, making deployment of such models into existing health care systems tangible.

Identification of people with low prevalence diseases in administrative healthcare records: A case study of HIV in British Columbia, Canada

INTRODUCTION

Case-finding algorithms can be applied to administrative healthcare records to identify people with diseases, including people with HIV (PWH). When supplementing an existing registry of a low prevalence disease, near-perfect specificity helps minimize impacts of adding in algorithm-identified false positive cases. We evaluated the performance of algorithms applied to healthcare records to supplement an HIV registry in British Columbia (BC), Canada.

METHODS

We applied algorithms based on HIV-related diagnostic codes to healthcare practitioner and hospitalization records. We evaluated 28 algorithms in a validation sub-sample of 7,124 persons with positive HIV tests (2,817 with a prior negative test) from the STOP HIV/AIDS data linkage-a linkage of healthcare, clinical, and HIV test records for PWH in BC, resembling a disease registry (1996-2020). Algorithms were primarily assessed based on their specificity-derived from this validation sub-sample-and their impact on the estimate of the total number of PWH in BC as of 2020.

RESULTS

In the validation sub-sample, median age at positive HIV test was 37 years (Q1: 30, Q3: 46), 80.1% were men, and 48.9% resided in the Vancouver Coastal Health Authority. For all algorithms, specificity exceeded 97% and sensitivity ranged from 81% to 95%. To supplement the HIV registry, we selected an algorithm with 99.89% (95% CI: 99.76% - 100.00%) specificity and 82.21% (95% CI: 81.26% - 83.16%) sensitivity, requiring five HIV-related healthcare practitioner encounters or two HIV-related hospitalizations within a 12-month window, or one hospitalization with HIV as the most responsible diagnosis. Upon adding PWH identified by this highly-specific algorithm to the registry, 8,774 PWH were present in BC as of March 2020, of whom 333 (3.8%) were algorithm-identified.

DISCUSSION

In the context of an existing low prevalence disease registry, the results of our validation study demonstrate the value of highly-specific case-finding algorithms applied to administrative healthcare records to enhance our ability to estimate the number of PWH living in BC.

Risk of adverse cardiovascular outcomes among people with HIV and nonalcoholic fatty liver disease

OBJECTIVE

To examine and compare the risk of major adverse cardiovascular events (MACEs) between people with HIV (PWH) with and without nonalcoholic fatty liver disease (NAFLD).

DESIGN

Population-based, multicenter, retrospective cohort study.

METHODS

Data on PWH between January 1, 2008, and December 31, 2020 were extracted from the TriNetX database. Primary outcomes were defined as the first incidence of myocardial infarction (MI), MACE, new-onset heart failure (HF), and a composite of cerebrovascular disease. Cox models were used to obtain hazard ratios (HRs) and 95% confidence intervals (CIs).

RESULTS

A total of 151 868 patients were identified as having HIV. After exclusions, 4969 patients were identified as having NAFLD. Of them, 4463 (90%) were propensity matched to a non-NAFLD control. Patients with NAFLD were older (42.9 versus 40.8 years). Among the NAFLD cohort, most participants were male and had a smoking history (12.3 versus 9.8%) than non-NAFLD. The mean follow-up was 4.8 ± 1.1 years for the NAFLD group and 5.3 ± 1.2 years for the non-NAFLD group. The risk of all outcomes was statistically significantly higher in NAFLD patients compared to those without NAFLD: MI (HR, 1.49; 95% CI, 1.11-2.01) MACE (HR, 1.49; 95% CI, 1.25-1.79), HF (HR, 1.73; 95% CI 1.37-2.19) and, cerebrovascular diseases (HR, 1.25; 95% CI, 1.05-1.48) and sensitivity analysis showed similar magnitude to the one generated in the primary analysis.

CONCLUSIONS

Patients with NAFLD have an elevated risk of adverse cardiovascular events (CVEs). The results indicate the need for targeted efforts to improve awareness of risks factors associated with adverse CVEs risk in PWH with NAFLD.

Development of automated HIV case reporting system using national electronic medical record in Thailand

Background

An electronic medical record (EMR) has the potential to improve completeness and reporting of notifiable diseases. We developed and assessed the validity of an HIV case detection algorithm and deployed the final algorithm in a national automated HIV case reporting system in Thailand.

Methods

The HIV case detection algorithms leveraged a combination of standard laboratory codes, prescriptions and International Classification of Diseases, 10th Revision diagnostic codes to identify potential cases. The initial algorithm was applied to the national EMR from 2014 to June 2020 to identify HIV-infected subjects to build the national HIV case reporting system (Epidemiological Intelligence Information System (EIIS)). A subset of potential positives identified by the initial algorithm were then validated and reviewed by infectious disease specialists. This review identified that a proportion of the false positives were due to pre-exposure prophylaxis/postexposure prophylaxis (PrEP/PEP) antiretrovirals, and so the algorithm was refined into a ‘Final Algorithm’ to address this.

Results

Positive predictive value of identifying HIV cases was 90% overall for the initial algorithm. Individuals misclassified as HIV-positive were HIV-negative patients with incorrect diagnostic codes, prescription records for PrEP, PEP and hepatitis B treatment. Additional revision to the algorithm included triple drug regimen to avoid further misclassification. The final HIV case detection algorithm was applied to national EMR between 2014 and 2020 with 449 088 HIV-infected subjects identified from 1496 hospitals. EIIS was designed by applying the final algorithm to automated extract HIV cases from the national EMR, analysing them and then transmitting the results to the Ministry of Public Health.

Conclusions

EMR data can complement traditional provider-based and laboratory-based disease reports. An automated algorithm incorporating laboratory, diagnosis codes and prescriptions have the potential to improve completeness and timeliness of HIV reporting, leading to the implementation of a national HIV case reporting system.

Comparison of algorithms for identifying people with HIV from electronic medical records in a large, multi-site database

Objective

As electronic medical record (EMR) data are increasingly used in HIV clinical and epidemiologic research, accurately identifying people with HIV (PWH) from EMR data is paramount. We sought to evaluate EMR data types and compare EMR algorithms for identifying PWH in a multicenter EMR database.

Materials and Methods

We collected EMR data from 7 healthcare systems in the Chicago Area Patient-Centered Outcomes Research Network (CAPriCORN) including diagnosis codes, anti-retroviral therapy (ART), and laboratory test results.

Results

In total, 13 935 patients had a positive laboratory test for HIV; 33 412 patients had a diagnosis code for HIV; and 17 725 patients were on ART. Only 8576 patients had evidence of HIV-positive status for all 3 data types (laboratory results, diagnosis code, and ART). A previously validated combination algorithm identified 22 411 patients as PWH.

Conclusion

EMR algorithms that combine laboratory results, administrative data, and ART can be applied to multicenter EMR data to identify PWH.

A Phenotyping Algorithm to Identify People With HIV in Electronic Health Record Data (HIV-Phen): Development and Evaluation Study

Background

Identification of people with HIV from electronic health record (EHR) data is an essential first step in the study of important HIV outcomes, such as risk assessment. This task has been historically performed via manual chart review, but the increased availability of large clinical data sets has led to the emergence of phenotyping algorithms to automate this process. Existing algorithms for identifying people with HIV rely on a combination of International Classification of Disease codes and laboratory tests or closely mimic clinical testing guidelines for HIV diagnosis. However, we found that existing algorithms in the literature missed a significant proportion of people with HIV in our data.

Objective

The aim of this study is to develop and evaluate HIV-Phen, an updated criteria-based HIV phenotyping algorithm.

Methods

We developed an algorithm using HIV-specific laboratory tests and medications and compared it with previously published algorithms in national and local data sets to identify cohorts of people with HIV. Cohort demographics were compared with those reported in the national and local surveillance data. Chart reviews were performed on a subsample of patients from the local database to calculate the sensitivity, specificity, positive predictive value, negative predictive value, and accuracy of the algorithm.

Results

Our new algorithm identified substantially more people with HIV in both national (up to an 85.75% increase) and local (up to an 83.20% increase) EHR databases than the previously published algorithms. The demographic characteristics of people with HIV identified using our algorithm were similar to those reported in national and local HIV surveillance data. Our algorithm demonstrated improved sensitivity over existing algorithms (98% vs 56%-92%) while maintaining a similar overall accuracy (96% vs 80%-96%).

Conclusions

We developed and evaluated an updated criteria-based phenotyping algorithm for identifying people with HIV in EHR data that demonstrates improved sensitivity over existing algorithms.

Optimizing Identification of People Living with HIV from Electronic Medical Records: Computable Phenotype Development and Validation

BACKGROUND

Electronic health record (EHR)-based computable phenotype algorithms allow researchers to efficiently identify a large virtual cohort of Human Immunodeficiency Virus (HIV) patients. Built upon existing algorithms, we refined, improved, and validated an HIV phenotype algorithm using data from the OneFlorida Data Trust, a repository of linked claims data and EHRs from its clinical partners, which provide care to over 15 million patients across all 67 counties in Florida.

METHODS

Our computable phenotype examined information from multiple EHR domains, including clinical encounters with diagnoses, prescription medications, and laboratory tests. To identify an HIV case, the algorithm requires the patient to have at least one diagnostic code for HIV and meet one of the following criteria: have 1+ positive HIV laboratory, have been prescribed with HIV medications, or have 3+ visits with HIV diagnostic codes. The computable phenotype was validated against a subset of clinical notes.

RESULTS

Among the 15+ million patients from OneFlorida, we identified 61,313 patients with confirmed HIV diagnosis. Among them, 8.05% met all four inclusion criteria, 69.7% met the 3+ HIV encounters criteria in addition to having HIV diagnostic code, and 8.1% met all criteria except for having positive laboratories. Our algorithm achieved higher sensitivity (98.9%) and comparable specificity (97.6%) relative to existing algorithms (77-83% sensitivity, 86-100% specificity). The mean age of the sample was 42.7 years, 58% male, and about half were Black African American. Patients' average follow-up period (the time between the first and last encounter in the EHRs) was approximately 4.6 years. The median number of all encounters and HIV-related encounters were 79 and 21, respectively.

CONCLUSION

By leveraging EHR data from multiple clinical partners and domains, with a considerably diverse population, our algorithm allows more flexible criteria for identifying patients with incomplete laboratory test results and medication prescribing history compared with prior studies.

A strategy for validation of variables derived from large-scale electronic health record data

Purpose:

Standardized approaches for rigorous validation of phenotyping from large-scale electronic health record (EHR) data have not been widely reported. We proposed a methodologically rigorous and efficient approach to guide such validation, including strategies for sampling cases and controls, determining sample sizes, estimating algorithm performance, and terminating the validation process, hereafter referred to as the San Diego Approach to Variable Validation (SDAVV).

Methods:

We propose sample size formulae which should be used prior to chart review, based on pre-specified critical lower bounds for positive predictive value (PPV) and negative predictive value (NPV). We also propose a stepwise strategy for iterative algorithm development/validation cycles, updating sample sizes for data abstraction until both PPV and NPV achieve target performance.

Results:

We applied the SDAVV to a Department of Veterans Affairs study in which we created two phenotyping algorithms, one for distinguishing normal colonoscopy cases from abnormal colonoscopy controls and one for identifying aspirin exposure. Estimated PPV and NPV both reached 0.970 with a 95% confidence lower bound of 0.915, estimated sensitivity was 0.963 and specificity was 0.975 for identifying normal colonoscopy cases. The phenotyping algorithm for identifying aspirin exposure reached a PPV of 0.990 (a 95% lower bound of 0.950), an NPV of 0.980 (a 95% lower bound of 0.930), and sensitivity and specificity were 0.960 and 1.000.

Conclusions:

A structured approach for prospectively developing and validating phenotyping algorithms from large-scale EHR data can be successfully implemented, and should be considered to improve the quality of “big data” research.

Electronic Medical Record-Based Case Phenotyping for the Charlson Conditions: Scoping Review

Background

Electronic medical records (EMRs) contain large amounts of rich clinical information. Developing EMR-based case definitions, also known as EMR phenotyping, is an active area of research that has implications for epidemiology, clinical care, and health services research.

Objective

This review aims to describe and assess the present landscape of EMR-based case phenotyping for the Charlson conditions.

Methods

A scoping review of EMR-based algorithms for defining the Charlson comorbidity index conditions was completed. This study covered articles published between January 2000 and April 2020, both inclusive. Embase (Excerpta Medica database) and MEDLINE (Medical Literature Analysis and Retrieval System Online) were searched using keywords developed in the following 3 domains: terms related to EMR, terms related to case finding, and disease-specific terms. The manuscript follows the Preferred Reporting Items for Systematic reviews and Meta-analyses extension for Scoping Reviews (PRISMA) guidelines.

Results

A total of 274 articles representing 299 algorithms were assessed and summarized. Most studies were undertaken in the United States (181/299, 60.5%), followed by the United Kingdom (42/299, 14.0%) and Canada (15/299, 5.0%). These algorithms were mostly developed either in primary care (103/299, 34.4%) or inpatient (168/299, 56.2%) settings. Diabetes, congestive heart failure, myocardial infarction, and rheumatology had the highest number of developed algorithms. Data-driven and clinical rule–based approaches have been identified. EMR-based phenotype and algorithm development reflect the data access allowed by respective health systems, and algorithms vary in their performance.

Conclusions

Recognizing similarities and differences in health systems, data collection strategies, extraction, data release protocols, and existing clinical pathways is critical to algorithm development strategies. Several strategies to assist with phenotype-based case definitions have been proposed.

Development and validation of an electronic medical record (EMR)-based computed phenotype of HIV-1 infection

Background

Electronic medical record (EMR) computed algorithms allow investigators to screen thousands of patient records to identify specific disease cases. No computed algorithms have been developed to detect all cases of human immunodeficiency virus (HIV) infection using administrative, laboratory, and clinical documentation data outside of the Veterans Health Administration. We developed novel EMR-based algorithms for HIV detection and validated them in a cohort of subjects in the Duke University Health System (DUHS).

Methods

We created 2 novel algorithms to identify HIV-infected subjects. Algorithm 1 used laboratory studies and medications to identify HIV-infected subjects, whereas Algorithm 2 used International Classification of Diseases, Ninth Revision (ICD-9) codes, medications, and laboratory testing. We applied the algorithms to a well-characterized cohort of patients and validated both against the gold standard of physician chart review. We determined sensitivity, specificity, and prevalence of HIV between 2007 and 2011 in patients seen at DUHS.

Results

A total of 172 271 patients were detected with complete data; 1063 patients met algorithm criteria for HIV infection. In all, 970 individuals were identified by both algorithms, 78 by Algorithm 1 alone, and 15 by Algorithm 2 alone. The sensitivity and specificity of each algorithm were 78% and 99%, respectively, for Algorithm 1 and 77% and 100% for Algorithm 2. The estimated prevalence of HIV infection at DUHS between 2007 and 2011 was 0.6%.

Conclusions

EMR-based phenotypes of HIV infection are capable of detecting cases of HIV-infected adults with good sensitivity and specificity. These algorithms have the potential to be adapted to other EMR systems, allowing for the creation of cohorts of patients across EMR systems.

A 17-Year Nationwide Study of Burkholderia cepacia Complex Bloodstream Infections Among Patients in the United States Veterans Health Administration

Background

Burkholderia cepacia complex (Bcc) are a group of multidrug-resistant gram-negative bacteria rarely reported in patients without cystic fibrosis (CF) or immunocompromising conditions. We investigated Bcc bloodstream infections (BSIs) in a cohort of non-CF patients from the US Veterans Health Administration (VHA).

Methods

Using VHA databases, we identified patients with Bcc BSI at facilities nationwide from 1999 through 2015. We ascertained clinical characteristics, treatments, and outcomes and identified factors associated with 30-day mortality in logistic regression analysis.

Results

We identified 248 patients with Bcc BSI, who were of advanced age (mean, 68 years), chronically ill, and had severe disease. The most common sources were central venous catheters (41%) and pneumonia (20%). Most cases were hospital-acquired (155 [62%]) or healthcare-associated (70 [28%]). Mortality at 14, 30, and 90 days was 16%, 25%, and 36%, respectively. Trimethoprim-sulfamethoxazole (TMP-SMX) and fluoroquinolones were active against 94% and 88% of isolates, respectively. Susceptibility to ceftazidime and meropenem occurred in approximately 70% of the isolates. The most prescribed antibiotics were fluoroquinolones (35%), followed by carbapenems (20%), TMP-SMX (18.5%), and ceftazidime (11%). In regression analysis, age (OR, 1.06 [95% confidence interval {CI}, 1.02-1.10], per added year) and the Pitt bacteremia score (OR, 1.65 [95% CI, 1.44-1.94], per unit increase) were associated with higher 30-day mortality.

Conclusions

In this large cohort of BSIs caused by Bcc, cases were mostly hospital-acquired and we observed high mortality, significant resistance to ceftazidime, and limited use of TMP-SMX. These observations add to our understanding of Bcc infection in non-CF patients and highlight the need for interventions to improve their outcome.

Suboptimal HIV Testing Among Patients Admitted With Pneumonia: A Missed Opportunity

Patients admitted with pneumonia are at higher risk for HIV and should be routinely screened. We examined a retrospective cohort of patients admitted to Duke University Health System with a primary diagnosis of pneumonia. During the study period, 6,951 persons were admitted with pneumonia. Of 6,646 patients without a known prior diagnosis of HIV, 1,010 (15%) had HIV testing during admission and 1,516 (23%) had a previously documented HIV test result. Forty-one (0.6%) patients had a positive HIV test during admission and 27 (0.4%) patients were diagnosed with HIV a median of 498 (IQR 112-982) days later, with median CD4 count of 64 (IQR 16-281) cells/mm³. HIV testing rates remain low in a population at high risk for HIV. At a minimum, we should be adhering to universal HIV screening recommendations, and certainly we should be screening those at higher risk. Opt-out HIV testing of pneumonia inpatients should be implemented.

The continuum of HIV care in South Africa: implications for achieving the second and third UNAIDS 90-90-90 targets

BACKGROUND

We characterize engagement with HIV care in South Africa in 2012 to identify areas for improvement towards achieving global 90-90-90 targets.

METHODS

Over 3.9 million CD4 cell count and 2.7 million viral load measurements reported in 2012 in the public sector were extracted from the national laboratory electronic database. The number of persons living with HIV (PLHIV), number and proportion in HIV care, on antiretroviral therapy (ART) and with viral suppression (viral load <400 copies/ml) were estimated and stratified by sex and age group. Modified Poisson regression approach was used to examine associations between sex, age group and viral suppression among persons on ART.

RESULTS

We estimate that among 6511 000 PLHIV in South Africa in 2012, 3300 000 individuals (50.7%) accessed care and 32.9% received ART. Although viral suppression was 73.7% among the treated population in 2012, the overall percentage of persons with viral suppression among all PLHIV was 23.8%. Linkage to HIV care was lower among men (38.5%) than among women (57.2%). Overall, 47.1% of those aged 0-14 years and 47.0% of those aged 15-49 years were linked to care compared with 56.2% among those aged above 50 years.

CONCLUSION

Around a quarter of all PLHIV have achieved viral suppression in South Africa. Men and younger persons have poorer linkage to HIV care. Expanding HIV testing, strengthening prompt linkage to care and further expansion of ART are needed for South Africa to reach the 90-90-90 target. Focus on these areas will reduce the transmission of new HIV infections and mortality in the general population.

Foreign-born status as a predictor of engagement in HIV care in a large US metropolitan health system

We sought to determine the linkage to and retention in HIV care after HIV diagnosis in foreign-born compared with US-born individuals. From a clinical data registry, we identified 619 patients aged ≥18 years with a new HIV diagnosis between 2000 and 2012. Timely linkage to care was the proportion of patients with an ICD-9 code for HIV infection (V08 or 042) associated with a primary care or infectious disease physician within 90 days of the index positive HIV test. Retention in HIV care was the presence of an HIV primary care visit in each 6-month period of the 24-month measurement period from the index HIV test. We used Cox regression analysis with adjustment for hypothesized confounders (age, gender, race/ethnicity, substance abuse, year, and location of HIV diagnosis). Foreign-born individuals comprised 36% (225/619) of the cohort. Index CD4 count was 225/µl (IQR 67-439/µl) in foreign-born compared with 328/µl (IQR 121-527/µl) in US-born individuals (p < .001). The proportion linked to care was 87% (196/225) in foreign-born compared with 77% (302/394) in US-born individuals (p = .002). The adjusted hazard ratio of linkage to HIV care in foreign-born compared with US-born individuals was 1.28 (95% confidence interval [CI], 1.05-1.56). Once linked, there was no difference in retention in care or virologic suppression at 24 months. These results show that despite late presentation to HIV care, foreign-born persons can subsequently engage in HIV care as well as US-born persons. Interventions that promote HIV screening in foreign-born persons are a promising way to improve outcomes in these populations.

Rates and Predictors of Newly Diagnosed HIV Infection Among Veterans Receiving Routine Once-Per-Lifetime HIV Testing in the Veterans Health Administration

OBJECTIVE

To determine predictors and variations in the rate of newly diagnosed HIV infection among persons who underwent routine (ie, non-risk based) rather than risk-based HIV testing in Veterans Health Administration (VHA) facilities.

METHODS

Retrospective observational study of the HIV infection new rates during the period when VHA policy called for routine (2009-2012) versus risk-based (2006-2009) HIV testing. Source data for testing results at 18 VHA facilities were obtained from the VHA National Corporate Data Warehouse.

RESULTS

New HIV diagnoses were established in 0.14% (95% confidence interval (CI): 0.12 to 0.46) of the 210,957 patients tested in the routine testing period versus 0.46% (95% CI: 0.42 to 1.29) of the 89,652 patients tested in the risk-based testing period. Among persons aged 65-74 years and 75 years or older, the new diagnosis rates were 0.07% (95% CI: 0.04 to 0.09) and 0.02% (95% CI: 0.00 to 0.03), respectively, and thus less than the generally accepted cost-effective threshold of 0.10%. Among African Americans, the upper bound of the 95% CI of the crude rate of new diagnoses during the routine-testing period was greater than 0.1% across all age strata. When assessed by year of testing, the adjusted rates of new diagnoses fell from 0.20% in 2010 to 0.10% in 2012.

CONCLUSIONS

Routine HIV testing is cost-effective among persons younger than 65 years. Among older patients, risk-based testing may be a more efficient and cost-effective approach. This will be increasingly relevant if rates of new HIV diagnoses in persons undergoing routine testing continue to decrease.

Validation of coding algorithms for the identification of patients hospitalized for alcoholic hepatitis using administrative data

Background

Epidemiologic studies of alcoholic hepatitis (AH) have been hindered by the lack of a validated International Classification of Disease (ICD) coding algorithm for use with administrative data. Our objective was to validate coding algorithms for AH using a hospitalization database.

Methods

The Hospital Discharge Abstract Database (DAD) was used to identify consecutive adults (≥18 years) hospitalized in the Calgary region with a diagnosis code for AH (ICD-10, K70.1) between 01/2008 and 08/2012. Medical records were reviewed to confirm the diagnosis of AH, defined as a history of heavy alcohol consumption, elevated AST and/or ALT (<300 U/L), serum bilirubin >34 μmol/L, and elevated INR. Subgroup analyses were performed according to the diagnosis field in which the code was recorded (primary vs. secondary) and AH severity. Algorithms that incorporated ICD-10 codes for cirrhosis and its complications were also examined.

Results

Of 228 potential AH cases, 122 patients had confirmed AH, corresponding to a positive predictive value (PPV) of 54 % (95 % CI 47–60 %). PPV improved when AH was the primary versus a secondary diagnosis (67 % vs. 21 %; P < 0.001). Algorithms that included diagnosis codes for ascites (PPV 75 %; 95 % CI 63–86 %), cirrhosis (PPV 60 %; 47–73 %), and gastrointestinal hemorrhage (PPV 62 %; 51–73 %) had improved performance, however, the prevalence of these diagnoses in confirmed AH cases was low (29–39 %).

Conclusions

In conclusion the low PPV of the diagnosis code for AH suggests that caution is necessary if this hospitalization database is used in large-scale epidemiologic studies of this condition.

Combining billing codes, clinical notes, and medications from electronic health records provides superior phenotyping performance

OBJECTIVE

To evaluate the phenotyping performance of three major electronic health record (EHR) components: International Classification of Disease (ICD) diagnosis codes, primary notes, and specific medications.

MATERIALS AND METHODS

We conducted the evaluation using de-identified Vanderbilt EHR data. We preselected ten diseases: atrial fibrillation, Alzheimer's disease, breast cancer, gout, human immunodeficiency virus infection, multiple sclerosis, Parkinson's disease, rheumatoid arthritis, and types 1 and 2 diabetes mellitus. For each disease, patients were classified into seven categories based on the presence of evidence in diagnosis codes, primary notes, and specific medications. Twenty-five patients per disease category (a total number of 175 patients for each disease, 1750 patients for all ten diseases) were randomly selected for manual chart review. Review results were used to estimate the positive predictive value (PPV), sensitivity, andF-score for each EHR component alone and in combination.

RESULTS

The PPVs of single components were inconsistent and inadequate for accurately phenotyping (0.06-0.71). Using two or more ICD codes improved the average PPV to 0.84. We observed a more stable and higher accuracy when using at least two components (mean ± standard deviation: 0.91 ± 0.08). Primary notes offered the best sensitivity (0.77). The sensitivity of ICD codes was 0.67. Again, two or more components provided a reasonably high and stable sensitivity (0.59 ± 0.16). Overall, the best performance (Fscore: 0.70 ± 0.12) was achieved by using two or more components. Although the overall performance of using ICD codes (0.67 ± 0.14) was only slightly lower than using two or more components, its PPV (0.71 ± 0.13) is substantially worse (0.91 ± 0.08).

CONCLUSION

Multiple EHR components provide a more consistent and higher performance than a single one for the selected phenotypes. We suggest considering multiple EHR components for future phenotyping design in order to obtain an ideal result.

Building strong research partnerships between public health and researchers: a VA case study

We are in a new era of partner-based implementation research, and we need clear strategies for how to navigate this new era. Drawing on principles from community-based participatory research, the Clinical Public Health group of the Department of Veterans Affairs and the HIV/Hepatitis Quality Enhancement Research Initiative (HHQUERI) forged a longstanding partnership that has improved the care of Veterans with Human Immunodeficiency Virus (HIV) and Hepatitis C Virus. An exemplar HIV testing project epitomizes this partnership and is discussed in terms of the lessons learned as a result of our high level of collaboration around design, analysis, implementation, and dissemination across projects over the past several years. Lessons learned through this partnered testing program involve respecting different time horizons among the partners, identifying relevant research questions for both parties, designing flexible studies, engaging all partners throughout the research, and placing an emphasis on relationship building at all times. These lessons and strategies can benefit others conducting partner-based research both within the Veterans Health Administration (VA) and in other integrated healthcare systems.