Measuring performance directly using the veterans health administration electronic medical record: a comparison with external peer review

Artificial intelligence approaches for phenotyping heart failure in U.S. Veterans Health Administration electronic health record

AIMS

Heart failure (HF) is a clinical syndrome with no definitive diagnostic tests. HF registries are often based on manual reviews of medical records of hospitalized HF patients identified using International Classification of Diseases (ICD) codes. However, most HF patients are not hospitalized, and manual review of big electronic health record (EHR) data is not practical. The US Department of Veterans Affairs (VA) has the largest integrated healthcare system in the nation, and an estimated 1.5 million patients have ICD codes for HF (HF ICD-code universe) in their VA EHR. The objective of our study was to develop artificial intelligence (AI) models to phenotype HF in these patients.

METHODS AND RESULTS

The model development cohort (n = 20 000: training, 16 000; validation 2000; testing, 2000) included 10 000 patients with HF and 10 000 without HF who were matched by age, sex, race, inpatient/outpatient status, hospital, and encounter date (within 60 days). HF status was ascertained by manual chart reviews in VA's External Peer Review Program for HF (EPRP-HF) and non-HF status was ascertained by the absence of ICD codes for HF in VA EHR. Two clinicians annotated 1000 random snippets with HF-related keywords and labelled 436 as HF, which was then used to train and test a natural language processing (NLP) model to classify HF (positive predictive value or PPV, 0.81; sensitivity, 0.77). A machine learning (ML) model using linear support vector machine architecture was trained and tested to classify HF using EPRP-HF as cases (PPV, 0.86; sensitivity, 0.86). From the 'HF ICD-code universe', we randomly selected 200 patients (gold standard cohort) and two clinicians manually adjudicated HF (gold standard HF) in 145 of those patients by chart reviews. We calculated NLP, ML, and NLP + ML scores and used weighted F scores to derive their optimal threshold values for HF classification, which resulted in PPVs of 0.83, 0.77, and 0.85 and sensitivities of 0.86, 0.88, and 0.83, respectively. HF patients classified by the NLP + ML model were characteristically and prognostically similar to those with gold standard HF. All three models performed better than ICD code approaches: one principal hospital discharge diagnosis code for HF (PPV, 0.97; sensitivity, 0.21) or two primary outpatient encounter diagnosis codes for HF (PPV, 0.88; sensitivity, 0.54).

CONCLUSIONS

These findings suggest that NLP and ML models are efficient AI tools to phenotype HF in big EHR data to create contemporary HF registries for clinical studies of effectiveness, quality improvement, and hypothesis generation.

Comparing Quality, Costs, and Outcomes of VA and Community Primary Care for Patients with Diabetes

BACKGROUND

The Maintaining Internal Systems and Strengthening Integrated Outside Networks (MISSION) Act expanded access to independent community providers outside the Veterans Health Administration (VA). Little is known how quality, costs, and outcomes of primary care received in the community compare to that of the VA.

OBJECTIVE

To compare quality, costs, and outcomes of community and VA-provided primary care for patients with diabetes over a 12-month episode.

DESIGN

A cross-sectional study using VA administrative data and community care claims. Adjusted analyses were conducted using inverse probability weighted regression adjustment to balance patient characteristics.

PARTICIPANTS

Veterans with diabetes receiving primary care in the VA or community.

MAIN MEASURES

Quality measures included receipt of hemoglobin A1C tests, eye exams, microalbumin urine tests, and flu shots. Outcomes were measured by hospitalizations for an ambulatory care sensitive condition (ACSC). Costs were measured for VA and community outpatient care, inpatient care, and prescription drugs.

KEY RESULTS

There were 652,648 VA patients and 3650 community care patients. VA patients were less likely to be White, had shorter mean drive time to VA primary care, and were less likely to be rural than community care patients. In adjusted analyses, community care patients had significantly lower probability of receiving a hemoglobin A1C test, eye exam, microalbumin urine test, and flu shot compared to the VA group. There was no difference in probability of an ACSC hospitalization. Community care patients had higher mean total costs ($1741 [95% CI, $431, $3052]), driven by higher inpatient and prescription drug costs but lower emergency care costs than VA patients.

CONCLUSION

Patients receiving community primary care had worse diabetes quality and higher costs than patients receiving VA primary care. There was no difference in health outcomes. Care provided by an integrated delivery system may have advantages in quality and value.

Accuracy of Electronic Health Record Documentation of Parental Presence: A Data Validation and Quality Improvement Analysis

Parental presence in the neonatal intensive care unit (NICU) is known to improve the health outcomes of an admitted infant. The use of the electronic health record (EHR) to analyze associations between parental presence and sociodemographic factors could provide important insights to families at greatest risk for limited presence during their infant's NICU stay, but there is little evidence about the accuracy of nonvital clinical measures such as parental presence in these datasets. A data validation study was conducted comparing the percentage agreement of an observational log of parental presence to the EHR documentation. Overall, high accuracy values were found when combining two methods of documentation. Additional stratification using a more specific measure, each chart's complete accuracy, instead of overall accuracy, revealed that night shift documentation was more accurate than day shift documentation (76.3% accurate during night shifts, 55.2% accurate during day shifts) and that flowsheet (FS) recordings were more accurate than the free-text plan of care (POC) notes (82.4% accurate for FS, 75.1% accurate for POC notes). This research provides a preliminary look at the accuracy of EHR documentation of nonclinical factors and can serve as a methodological roadmap for other researchers who intend to use EHR data.

Systematically assessing the quality of dental electronic health record data for an investigation into oral health care disparities

OBJECTIVES

This work describes the process by which the quality of electronic health care data for a public health study was determined. The objectives were to adapt, develop, and implement data quality assessments (DQAs) based on the National Institutes of Health Pragmatic Trials Collaboratory (NIHPTC) data quality framework within the three domains of completeness, accuracy, and consistency, for an investigation into oral health care disparities of a preventive care program.

METHODS

Electronic health record data for eligible children in a dental accountable care organization of 30 offices, in Oregon, were extracted iteratively from January 1, 2014, through March 31, 2022. Baseline eligibility criteria included: children ages 0-18 with a baseline examination, Oregon home address, and either Medicaid or commercial dental benefits at least once between 2014 and 2108. Using the NIHPTC framework as a guide, DQAs were conducted throughout data element identification, extraction, staging, profiling, review, and documentation.

RESULTS

The data set included 91,487 subjects, 11 data tables comprising 75 data variables (columns), with a total of 6,861,525 data elements. Data completeness was 97.2%, the accuracy of EHR data elements in extracts was 100%, and consistency between offices was strong; 29 of 30 offices within 2 standard deviations of the mean (s = 94%).

CONCLUSIONS

The NIHPTC framework proved to be a useful approach, to identify, document, and characterize the dataset. The concepts of completeness, accuracy, and consistency were adapted by the multidisciplinary research team and the overall quality of the data are demonstrated to be of high quality.

Long-Term Body Mass Index Variability and Adverse Cardiovascular Outcomes

Importance

Body mass index (BMI; calculated as weight in kilograms divided by height in meters squared) is a commonly used estimate of obesity, which is a complex trait affected by genetic and lifestyle factors. Marked weight gain and loss could be associated with adverse biological processes.

Objective

To evaluate the association between BMI variability and incident cardiovascular disease (CVD) events in 2 distinct cohorts.

Design, Setting, and Participants

This cohort study used data from the Million Veteran Program (MVP) between 2011 and 2018 and participants in the UK Biobank (UKB) enrolled between 2006 and 2010. Participants were followed up for a median of 3.8 (5th-95th percentile, 3.5) years. Participants with baseline CVD or cancer were excluded. Data were analyzed from September 2022 and September 2023.

Exposure

BMI variability was calculated by the retrospective SD and coefficient of variation (CV) using multiple clinical BMI measurements up to the baseline.

Main Outcomes and Measures

The main outcome was incident composite CVD events (incident nonfatal myocardial infarction, acute ischemic stroke, and cardiovascular death), assessed using Cox proportional hazards modeling after adjustment for CVD risk factors, including age, sex, mean BMI, systolic blood pressure, total cholesterol, high-density lipoprotein cholesterol, smoking status, diabetes status, and statin use. Secondary analysis assessed whether associations were dependent on the polygenic score of BMI.

Results

Among 92 363 US veterans in the MVP cohort (81 675 [88%] male; mean [SD] age, 56.7 [14.1] years), there were 9695 Hispanic participants, 22 488 non-Hispanic Black participants, and 60 180 non-Hispanic White participants. A total of 4811 composite CVD events were observed from 2011 to 2018. The CV of BMI was associated with 16% higher risk for composite CVD across all groups (hazard ratio [HR], 1.16; 95% CI, 1.13-1.19). These associations were unchanged among subgroups and after adjustment for the polygenic score of BMI. The UKB cohort included 65 047 individuals (mean [SD] age, 57.30 (7.77) years; 38 065 [59%] female) and had 6934 composite CVD events. Each 1-SD increase in BMI variability in the UKB cohort was associated with 8% increased risk of cardiovascular death (HR, 1.08; 95% CI, 1.04-1.11).

Conclusions and Relevance

This cohort study found that among US veterans, higher BMI variability was a significant risk marker associated with adverse cardiovascular events independent of mean BMI across major racial and ethnic groups. Results were consistent in the UKB for the cardiovascular death end point. Further studies should investigate the phenotype of high BMI variability.

Investigation of the role of perceived access to primary care in mediating and moderating racial and ethnic disparities in chronic disease control in the veterans health administration

OBJECTIVE

To examine the role of patient-perceived access to primary care in mediating and moderating racial and ethnic disparities in hypertension control and diabetes control among Veterans Health Administration (VA) users.

DATA SOURCE AND STUDY SETTING

We performed a secondary analysis of national VA user administrative data for fiscal years 2016-2019.

STUDY DESIGN

Our primary exposure was race or ethnicity and primary outcomes were binary indicators of hypertension control (<140/90 mmHg) and diabetes control (HgbA1c < 9%) among patients with known disease. We used the inverse odds-weighting method to test for mediation and logistic regression with race and ethnicity-by-perceived access interaction product terms to test moderation. All models were adjusted for age, sex, socioeconomic status, rurality, education, self-rated physical and mental health, and comorbidities.

DATA COLLECTION/EXTRACTION METHODS

We included VA users with hypertension and diabetes control data from the External Peer Review Program who had contemporaneously completed the Survey of Healthcare Experience of Patients-Patient-Centered Medical Home. Hypertension (34,233 patients) and diabetes (23,039 patients) samples were analyzed separately.

PRINCIPAL FINDINGS

After adjustment, Black patients had significantly lower rates of hypertension control than White patients (75.5% vs. 78.8%, p < 0.01); both Black (81.8%) and Hispanic (80.4%) patients had significantly lower rates of diabetes control than White patients (85.9%, p < 0.01 for both differences). Perceived access was lower among Black, Multi-Race and Native Hawaiian and Other Pacific Islanders compared to White patients in both samples. There was no evidence that perceived access mediated or moderated associations between Black race, Hispanic ethnicity, and hypertension or diabetes control.

CONCLUSIONS

We observed disparities in hypertension and diabetes control among minoritized patients. There was no evidence that patients' perception of access to primary care mediated or moderated these disparities. Reducing racial and ethnic disparities within VA in hypertension and diabetes control may require interventions beyond those focused on improving patient access.

Association of Integrated Mental Health Services with Physical Health Quality Among VA Primary Care Patients

BACKGROUND

Integrated care for comorbid depression and chronic medical disease improved physical and mental health outcomes in randomized controlled trials. The Veterans Health Administration (VA) implemented Primary Care-Mental Health Integration (PC-MHI) across all primary care clinics nationally to increase access to mental/behavioral health treatment, alongside physical health management.

OBJECTIVE

To examine whether widespread, pragmatic PC-MHI implementation was associated with improved care quality for chronic medical diseases.

DESIGN, SETTING, AND PARTICIPANTS

This retrospective cohort study included 828,050 primary care patients with at least one quality metric among 396 VA clinics providing PC-MHI services between October 2013 and September 2016.

MAIN MEASURE(S)

For outcome measures, chart abstractors rated whether diabetes and cardiovascular quality metrics were met for patients at each clinic as part of VA's established quality reporting program. The explanatory variable was the proportion of primary care patients seen by integrated mental health specialists in each clinic annually. Multilevel logistic regression models examined associations between clinic PC-MHI proportion and patient-level quality metrics, adjusting for regional, patient, and time-level effects and clinic and patient characteristics.

KEY RESULTS

Median proportion of patients seen in PC-MHI per clinic was 6.4% (IQR=4.7-8.7%). Nineteen percent of patients with diabetes had poor glycemic control (hemoglobin A1c >9%). Five percent had severely elevated blood pressure (>160/100 mmHg). Each two-fold increase in clinic PC-MHI proportion was associated with 2% lower adjusted odds of poor glycemic control (95% CI=0.96-0.99; p=0.046) in diabetes. While there was no association with quality for patients diagnosed with hypertension, patients without diagnosed hypertension had 5% (CI=0.92-0.99; p=0.046) lower adjusted odds of having elevated blood pressures.

CONCLUSIONS AND RELEVANCE

Primary care clinics where integrated mental health care reached a greater proportion of patients achieved modest albeit statistically significant gains in key chronic care quality metrics, providing optimism about the expected effects of large-scale PC-MHI implementation on physical health.

Using the Electronic Health Record to Characterize the Hepatitis C Virus Care Cascade

OBJECTIVES

Chronic hepatitis C virus (HCV) infection is one of the main causes of hepatocellular carcinoma. Before initiating a multilevel HCV screening intervention, we sought to (1) describe concordance between the electronic health record (EHR) data warehouse and manual medical record review in recording aspects of HCV testing and treatment and (2) estimate the percentage of patients with chronic HCV infection who initiated and completed HCV treatment using manual medical record review.

METHODS

We examined the medical records for 177 patients (100 randomly selected patients born during 1945-1965 without evidence of HCV testing and 77 adult patients of any birth cohort who had completed HCV testing) with a primary care or relevant specialist visit at an academic health care system in Tampa, Florida, from 2015 through 2018. We used the Cohen κ coefficient to examine the degree of concordance between the searchable data warehouse and the medical record review abstractions. Descriptive statistics characterized referral to and receipt of treatment among patients with chronic HCV infection from medical record review.

RESULTS

We found generally good concordance between the data warehouse abstraction and medical record review for HCV testing data (κ ranged from 0.66 to 0.87). However, the data warehouse failed to capture data on HCV treatment variables. According to medical record review, 28 patients had chronic HCV infection; 16 patients were prescribed treatment, 14 initiated treatment, and 9 achieved and had a reported posttreatment undetected HCV viral load.

CONCLUSIONS

Using data warehouse data provides generally reliable HCV testing information. However, without the use of natural language processing and purposeful EHR design, manual medical record reviews will likely be required to characterize treatment initiation and completion.

Coronary Artery Disease Risk of Familial Hypercholesterolemia Genetic Variants Independent of Clinically Observed Longitudinal Cholesterol Exposure

BACKGROUND

Familial hypercholesterolemia (FH) genetic variants confer risk for coronary artery disease independent of LDL-C (low-density lipoprotein cholesterol) when considering a single measurement. In real clinical settings, longitudinal LDL-C data are often available through the electronic health record. It is unknown whether genetic testing for FH variants provides additional risk-stratifying information once longitudinal LDL-C is considered.

METHODS

We used the extensive electronic health record data available through the Million Veteran Program to conduct a nested case-control study. The primary outcome was coronary artery disease, derived from electronic health record codes for acute myocardial infarction and coronary revascularization. Incidence density sampling was used to match case/control exposure windows, defined by the date of the first LDL-C measurement to the date of the first coronary artery disease code of the index case. Adjustments for the first, maximum, or mean LDL-C were analyzed. FH variants in LDLR, APOB, and PCSK9 (Proprotein convertase subtilisin/kexin type 9) were assessed by custom genotype array.

RESULTS

In a cohort of 23 091 predominantly prevalent cases at enrollment and 230 910 matched controls, FH variant carriers had an increased risk for coronary artery disease (odds ratio [OR], 1.53 [95% CI, 1.24-1.89]). Adjusting for mean LDL-C led to the greatest attenuation of the risk estimate, but significant risk remained (odds ratio, 1.33 [95% CI, 1.08-1.64]). The degree of attenuation was not affected by the number and the spread of LDL-C measures available.

CONCLUSIONS

The risk associated with carrying an FH variant cannot be fully captured by the LDL-C data available in the electronic health record, even when considering multiple LDL-C measurements spanning more than a decade.

Comparing total medical expenditure between patients receiving direct oral anticoagulants vs warfarin for the treatment of atrial fibrillation: evidence from VA-Medicare dual enrollees

BACKGROUND: Direct oral anticoagulants (DOACs) are an alternative to warfarin for treatment of atrial fibrillation (AF). Evidence demonstrating the efficacy and safety of DOACs has primarily been from clinical trial settings. The real-world effectiveness of DOACs in specific nontrial populations that differ in age, comorbidity burden, and socioeconomic status is unclear. OBJECTIVE: To compare total downstream medical expenditure between AF patients treated with warfarin and DOACs dually enrolled in the Veterans Affairs (VA) Healthcare System and fee-for-service Medicare. METHODS: This was an exploratory treatment effectiveness study that analyzed VA administrative data and Medicare claims. We examined patients with an incident diagnosis for AF and initiated warfarin or DOAC treatment between 2012 and 2015. The primary outcome was total medical expenditure over 3 years following treatment initiation. To address potential informative censoring, we applied a multipart estimator that extends traditional 2-part models to separate differences between groups due to survival and cost accumulation effects. Inverse probability weighting was applied to address potential treatment selection bias. RESULTS: We identified 31,276 and 17,021 patients receiving warfarin and DOACs, respectively. Mean unadjusted (SD) expenditure was higher for warfarin ($56,265 [$96,666]) compared with DOAC patients ($32,736 [$52,470]). Compared with patients receiving DOACs, adjusted 3-year expenditure was $25,688 (P < 0.001) higher for patients receiving warfarin. CONCLUSIONS: VA patients with AF initiating warfarin incurred markedly higher downstream expenditure compared with similar patients receiving DOACs. The benefits of DOACs found in previous clinical trials were present in this population, suggesting that these DOACs may be the preferred option for treatment of AF in older VA patients. DISCLOSURES: This study was funded by a VA Health Services Research and Development Investigator Initiated Research Award (IIR 15-139). Support for VA/CMS data was provided by the Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development, Health Services Research and Development, VA Information Resource Center (Project Numbers SDR 02-237 and 98-004). The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs, the University of Washington, Northeastern University, and Boston University. The authors declare no conflicts of interest. This research includes data obtained from the VHA Office of Performance Measurement (17API2), which resides within the Office of Analytics and Performance Integration (API), under the Office of Quality and Patient Safety (QPS; formerly known as RAPID). An oral presentation documenting a subset of the findings from this study was presented at the 2020 AcademyHealth Annual Research Meeting, delivered virtually on July 29, 2020.

The Relationship Between Body Mass Index and Pain Intensity Among Veterans with Musculoskeletal Disorders: Findings from the MSD Cohort Study

OBJECTIVE

To examine the relationship between body mass index (BMI) and pain intensity among veterans with musculoskeletal disorder diagnoses (MSDs; nontraumatic joint disorder; osteoarthritis; low back, back, and neck pain).

SETTING

Administrative and electronic health record data from the Veterans Health Administration (VHA).

SUBJECTS

A national cohort of US military veterans with MSDs in VHA care during 2001-2012 (N = 1,759,338).

METHODS

These cross-sectional data were analyzed using hurdle negative binomial models of pain intensity as a function of BMI, adjusted for comorbidities and demographics.

RESULTS

The sample had a mean age of 59.4, 95% were male, 77% were white/Non-Hispanic, 79% were overweight or obese, and 42% reported no pain at index MSD diagnosis. Overall, there was a J-shaped relationship between BMI and pain (nadir = 27 kg/m2), with the severely obese (BMI ≥ 40 kg/m2) being most likely to report any pain (OR vs normal weight = 1.23, 95% confidence interval = 1.21-1.26). The association between BMI and pain varied by MSD, with a stronger relationship in the osteoarthritis group and a less pronounced relationship in the back and low back pain groups.

CONCLUSIONS

There was a high prevalence of overweight/obesity among veterans with MSD. High levels of BMI (>27 kg/m2) were associated with increased odds of pain, most markedly among veterans with osteoarthritis.

Association of Geographic Differences in Prevalence of Uncontrolled Chronic Conditions With Changes in Individuals' Likelihood of Uncontrolled Chronic Conditions

IMPORTANCE

The prevalence of leading risk factors for morbidity and mortality in the US significantly varies across regions, states, and neighborhoods, but the extent these differences are associated with a person's place of residence vs the characteristics of the people who live in different places remains unclear.

OBJECTIVE

To estimate the degree to which geographic differences in leading risk factors are associated with a person's place of residence by comparing trends in health outcomes among individuals who moved to different areas or did not move.

DESIGN, SETTING, AND PARTICIPANTS

This retrospective cohort study estimated the association between the differences in the prevalence of uncontrolled chronic conditions across movers' destination and origin zip codes and changes in individuals' likelihood of uncontrolled chronic conditions after moving, adjusting for person-specific fixed effects, the duration of time since the move, and secular trends among movers and those who did not move. Electronic health records from the Veterans Health Administration were analyzed. The primary analysis included 5 342 207 individuals with at least 1 Veterans Health Administration outpatient encounter between 2008 and 2018 who moved zip codes exactly once or never moved.

EXPOSURES

The difference in the prevalence of uncontrolled chronic conditions between a person's origin zip code and destination zip code (excluding the individual mover's outcomes).

MAIN OUTCOMES AND MEASURES

Prevalence of uncontrolled blood pressure (systolic blood pressure level >140 mm Hg or diastolic blood pressure level >90 mm Hg), uncontrolled diabetes (hemoglobin A1c level >8%), obesity (body mass index >30), and depressive symptoms (2-item Patient Health Questionnaire score ≥2) per quarter-year during the 3 years before and the 3 years after individuals moved.

RESULTS

The study population included 5 342 207 individuals (mean age, 57.6 [SD, 17.4] years, 93.9% men, 72.5% White individuals, and 12.7% Black individuals), of whom 1 095 608 moved exactly once and 4 246 599 never moved during the study period. Among the movers, the change after moving in the prevalence of uncontrolled blood pressure was 27.5% (95% CI, 23.8%-31.3%) of the between-area difference in the prevalence of uncontrolled blood pressure. Similarly, the change after moving in the prevalence of uncontrolled diabetes was 5.0% (95% CI, 2.7%-7.2%) of the between-area difference in the prevalence of uncontrolled diabetes; the change after moving in the prevalence of obesity was 3.1% (95% CI, 2.0%-4.2%) of the between-area difference in the prevalence of obesity; and the change after moving in the prevalence of depressive symptoms was 15.2% (95% CI, 13.1%-17.2%) of the between-area difference in the prevalence of depressive symptoms.

CONCLUSIONS AND RELEVANCE

In this retrospective cohort study of individuals receiving care at Veterans Health Administration facilities, geographic differences in prevalence were associated with a substantial percentage of the change in individuals' likelihood of poor blood pressure control or depressive symptoms, and a smaller percentage of the change in individuals' likelihood of poor diabetes control and obesity. Further research is needed to understand the source of these associations with a person's place of residence.

Association of the Patient-Centered Medical Home Implementation with Chronic Disease Quality in Patients with Multimorbidity

BACKGROUND

The patient-centered medical home (PCMH) was established in part to improve chronic disease management, yet evidence is limited for effects on patients with multimorbidity.

OBJECTIVE

To examine the association of Patient-Aligned Care Team (PACT) implementation, the Veterans Health Administration (VA) PCMH model, and care quality for multimorbid patients enrolled in VA primary care from 2012 to 2014.

DESIGN

Retrospective cohort.

PATIENTS

318,764 multimorbid (> 3 chronic diseases) patients receiving care in 917 clinics.

MAIN MEASURES

PCMH implementation was measured using the PACT Implementation Progress Index (PI²) for clinics in 2012. The PI² is a validated composite measure of administrative and survey data with higher scores associated with greater care quality. Quality outcomes from 2013 to 2014 were assessed from External Peer Review Program (EPRP) metrics. Outcomes included preventative care, chronic disease management, and mental health and substance use metrics. We used generalized estimating equations to model associations adjusting for patient and clinic characteristics. We also examined associations for a subgroup with > 5 chronic diseases.

KEY RESULTS

For one-third of metrics (5/15), greater implementation of PACT in 2012 was associated with higher predicted probability of meeting the quality metric in 2013-2014. This association persisted for only two metrics (diabetic glycemic control, P < 0.001; lipid control in ischemic heart disease, P = 0.02) among patients with > 5 chronic diseases.

CONCLUSIONS

Multimorbid patients engaged in care from clinics with higher PCMH implementation received higher quality care across several quality domains, but this association was reduced in patients with > 5 chronic diseases.

Palliative Care Consultation Reduces Heart Failure Transitions: A Matched Analysis

Background

Palliative care supports quality of life, symptom control, and goal setting in heart failure (HF) patients. Unlike hospice, palliative care does not restrict life‐prolonging therapy. This study examined the association between palliative care during hospitalization for HF on the subsequent transitions and procedures.

Methods and Results

Veterans admitted to hospitals with HF from 2010 to 2015 were randomly selected for the Veterans Administration External Peer Review Program. Variables pertaining to demographic, clinical, laboratory, and usage were captured from Veterans Administration electronic records. Patients receiving hospice services before admission were excluded. Patients who received palliative care were propensity matched to those who did not. The primary outcomes were whether the patient experienced transitions or procedures in the 6 months after admission. Transitions included multiple readmissions (≥2) or intensive care admissions and procedures included mechanical ventilation, pacemaker implantation, or defibrillator implantation. Among 57 182 hospitalized HF patients, 1431 received palliative care, and were well matched to 1431 without (standardized mean differences ≤ ±0.05 on all matched variables). Palliative care was associated with significantly fewer multiple rehospitalizations (30.9% versus 40.3%, P<0.001), mechanical ventilation (2.8% versus 5.4%, P=0.004), and defibrillator implantation (2.1% versus 3.6%, P=0.01). After adjustment for facility fixed effects, palliative care consultation was associated with a significantly reduced hazard of multiple readmissions (adjusted hazard ratio=0.73, 95% CI, 0.64–0.84) and mechanical ventilation (adjusted hazard ratio=0.76, 95% CI, 0.67–0.87).

Conclusions

Palliative care during HF admissions was associated with fewer readmissions and less mechanical ventilation. When available, engagement of HF patients and caregivers in palliative care for symptom control, quality of life, and goals of care discussions may be associated with reduced rehospitalizations and mechanical ventilation.

Outcomes of primary care delivery by nurse practitioners: Utilization, cost, and quality of care

OBJECTIVE

To examine whether nurse practitioner (NP)-assigned patients exhibited differences in utilization, costs, and clinical outcomes compared to medical doctor (MD)-assigned patients.

DATA SOURCES

Veterans Affairs (VA) administrative data capturing characteristics, outcomes, and provider assignments of 806 434 VA patients assigned to an MD primary care provider (PCP) who left VA practice between 2010 and 2012.

STUDY DESIGN

We applied a difference-in-difference approach comparing outcomes between patients reassigned to MD and NP PCPs, respectively. We examined measures of outpatient (primary care, specialty care, and mental health) and inpatient (total and ambulatory care sensitive hospitalizations) utilization, costs (outpatient, inpatient and total), and clinical outcomes (control of hemoglobin A1c, LDL, and blood pressure) in the year following reassignment.

PRINCIPAL FINDINGS

Compared to MD-assigned patients, NP-assigned patients were less likely to use primary care and specialty care services and incurred fewer total and ambulatory care sensitive hospitalizations. Differences in costs, clinical outcomes, and receipt of diagnostic tests between groups were not statistically significant.

CONCLUSIONS

Patients reassigned to NPs experienced similar outcomes and incurred less utilization at comparable cost relative to MD patients. NPs may offer a cost-effective approach to addressing anticipated shortages of primary care physicians.

Retooling of Paper-based Outcome Measures to Electronic Format: Comparison of the NY State Public Risk Model and EHR-derived Risk Models for CABG Mortality

BACKGROUND

Risk adjustment is critical in the comparison of quality of care and health care outcomes for providers. Electronic health records (EHRs) have the potential to eliminate the need for costly and time-consuming manual data abstraction of patient outcomes and risk factors necessary for risk adjustment.

METHODS

Leading EHR vendors and hospital focus groups were asked to review risk factors in the New York State (NYS) coronary artery bypass graft (CABG) surgery statistical models for mortality and readmission and assess feasibility of EHR data capture. Risk models based only on registry data elements that can be captured by EHRs (one for easily obtained data and one for data obtained with more difficulty) were developed and compared with the NYS models for different years.

RESULTS

Only 6 data elements could be extracted from the EHR, and outlier hospitals differed substantially for readmission but not for mortality. At the patient level, measures of fit and predictive ability indicated that the EHR models are inferior to the NYS CABG surgery risk model [eg, c-statistics of 0.76 vs. 0.71 (P<0.001) and 0.76 vs. 0.74 (P=0.009) for mortality in 2010], although the correlation of the predicted probabilities between the NYS and EHR models was high, ranging from 0.96 to 0.98.

CONCLUSIONS

A simplified risk model using EHR data elements could not capture most of the risk factors in the NYS CABG surgery risk models, many outlier hospitals were different for readmissions, and patient-level measures of fit were inferior.

Association Between Mental Health Staffing Level and Primary Care-Mental Health Integration Level on Provision of Depression Care in Veteran's Affairs Medical Facilities

We examined the association of mental health staffing and the utilization of primary care/mental health integration (PCMHI) with facility-level variations in adequacy of psychotherapy and antidepressants received by Veterans with new, recurrent, and chronic depression. Greater likelihood of adequate psychotherapy was associated with increased (1) PCMHI utilization by recurrent depression patients (AOR 1.02; 95% CI 1.00, 1.03); and (2) staffing for recurrent (AOR 1.03; 95% CI 1.01, 1.06) and chronic (AOR 1.02; 95% CI 1.00, 1.03) depression patients (p < 0.05). No effects were found for antidepressants. Mental health staffing and PCMHI utilization explained only a small amount of the variance in the adequacy of depression care.

A Mobile and Web-Based Self-Directed Complementary and Integrative Health Program for Veterans and Their Partners (Mission Reconnect): Protocol for a Mixed-Methods Randomized Controlled Trial

Background

Complementary and integrative health (CIH) is a viable solution to PTSD and chronic pain. Many veterans believe CIH can be performed only by licensed professionals in a health care setting. Health information technology can bring effective CIH to veterans and their partners.

Objective

This paper describes the rationale, design, and methods of the Mission Reconnect protocol to deliver mobile and Web-based complementary and integrative health programs to veterans and their partners (eg, spouse, significant other, caregiver, or family member).

Methods

This three-site, 4-year mixed-methods randomized controlled trial uses a wait-list control to determine the effects of mobile and Web-based CIH programs for veterans and their partners, or dyads. The study will use two arms (ie, treatment intervention arm and wait-list control arm) in a clinical sample of veterans with comorbid pain and posttraumatic stress disorder, and their partners. The study will evaluate the effectiveness and perceived value of the Mission Reconnect program in relation to physical and psychological symptoms, global health, and social outcomes.

Results

Funding for the study began in November 2018, and we are currently in the process of recruitment screening and data randomization for the study. Primary data collection will begin in May 2019 and continue through May 2021. Projected participants per site will be 76 partners/dyads, for a total of 456 study participants. Anticipated study results will be published in November 2022.

Conclusions

This work highlights innovative delivery of CIH to veterans and their partners for treatment of posttraumatic stress disorder and chronic pain.

Trial Registration

ClinicalTrials.gov NCT03593772; https://clinicaltrials.gov/ct2/show/NCT03593772 (Archived by WebCite at http://www.webcitation.org/77Q2giwtw)

International Registered Report Identifier (IRRID)

PRR1-10.2196/13666

In Data We Trust? Comparison of Electronic Versus Manual Abstraction of Antimicrobial Prescribing Quality Metrics for Hospitalized Veterans With Pneumonia

BACKGROUND

Electronic health records provide the opportunity to assess system-wide quality measures. Veterans Affairs Pharmacy Benefits Management Center for Medication Safety uses medication use evaluation (MUE) through manual review of the electronic health records.

OBJECTIVE

To compare an electronic MUE approach versus human/manual review for extraction of antibiotic use (choice and duration) and severity metrics.

RESEARCH DESIGN

Retrospective.

SUBJECTS

Hospitalizations for uncomplicated pneumonia occurring during 2013 at 30 Veterans Affairs facilities.

MEASURES

We compared summary statistics, individual hospitalization-level agreement, facility-level consistency, and patterns of variation between electronic and manual MUE for initial severity, antibiotic choice, daily clinical stability, and antibiotic duration.

RESULTS

Among 2004 hospitalizations, electronic and manual abstraction methods showed high individual hospitalization-level agreement for initial severity measures (agreement=86%-98%, κ=0.5-0.82), antibiotic choice (agreement=89%-100%, κ=0.70-0.94), and facility-level consistency for empiric antibiotic choice (anti-MRSA r=0.97, P<0.001; antipseudomonal r=0.95, P<0.001) and therapy duration (r=0.77, P<0.001) but lower facility-level consistency for days to clinical stability (r=0.52, P=0.006) or excessive duration of therapy (r=0.55, P=0.005). Both methods identified widespread facility-level variation in antibiotic choice, but we found additional variation in manual estimation of excessive antibiotic duration and initial illness severity.

CONCLUSIONS

Electronic and manual MUE agreed well for illness severity, antibiotic choice, and duration of therapy in pneumonia at both the individual and facility levels. Manual MUE showed additional reviewer-level variation in estimation of initial illness severity and excessive antibiotic use. Electronic MUE allows for reliable, scalable tracking of national patterns of antimicrobial use, enabling the examination of system-wide interventions to improve quality.

Dual Use of a Patient Portal and Clinical Video Telehealth by Veterans with Mental Health Diagnoses: Retrospective, Cross-Sectional Analysis

Background

Access to mental health care is challenging. The Veterans Health Administration (VHA) has been addressing these challenges through technological innovations including the implementation of Clinical Video Telehealth, two-way interactive and synchronous videoconferencing between a provider and a patient, and an electronic patient portal and personal health record, My HealtheVet.

Objective

This study aimed to describe early adoption and use of My HealtheVet and Clinical Video Telehealth among VHA users with mental health diagnoses.

Methods

We conducted a retrospective, cross-sectional analysis of early My HealtheVet adoption and Clinical Video Telehealth engagement among veterans with one or more mental health diagnoses who were VHA users from 2007 to 2012. We categorized veterans into four electronic health (eHealth) technology use groups: My HealtheVet only, Clinical Video Telehealth only, dual users who used both, and nonusers of either. We examined demographic characteristics and mental health diagnoses by group. We explored My HealtheVet feature use among My HealtheVet adopters. We then explored predictors of My HealtheVet adoption, Clinical Video Telehealth engagement, and dual use using multivariate logistic regression.

Results

Among 2.17 million veterans with one or more mental health diagnoses, 1.51% (32,723/2,171,325) were dual users, and 71.72% (1,557,218/2,171,325) were nonusers of both My HealtheVet and Clinical Video Telehealth. African American and Latino patients were significantly less likely to engage in Clinical Video Telehealth or use My HealtheVet compared with white patients. Low-income patients who met the criteria for free care were significantly less likely to be My HealtheVet or dual users than those who did not. The odds of Clinical Video Telehealth engagement and dual use decreased with increasing age. Women were more likely than men to be My HealtheVet or dual users but less likely than men to be Clinical Video Telehealth users. Patients with schizophrenia or schizoaffective disorder were significantly less likely to be My HealtheVet or dual users than those with other mental health diagnoses (odds ratio, OR 0.50, CI 0.47-0.53 and OR 0.75, CI 0.69-0.80, respectively). Dual users were younger (53.08 years, SD 13.7, vs 60.11 years, SD 15.83), more likely to be white, and less likely to be low-income than the overall cohort. Although rural patients had 17% lower odds of My HealtheVet adoption compared with urban patients (OR 0.83, 95% CI 0.80-0.87), they were substantially more likely than their urban counterparts to engage in Clinical Video Telehealth and dual use (OR 2.45, 95% CI 1.95-3.09 for Clinical Video Telehealth and OR 2.11, 95% CI 1.81-2.47 for dual use).

Conclusions

During this study (2007-2012), use of these technologies was low, leaving much potential for growth. There were sociodemographic disparities in access to My HealtheVet and Clinical Video Telehealth and in dual use of these technologies. There was also variation based on types of mental health diagnosis. More research is needed to ensure that these and other patient-facing eHealth technologies are accessible and effectively used by all vulnerable patients.

Evaluating the Impact of Integrated Care on Service Utilization in Serious Mental Illness

Serious mental illness (SMI) affects 5% of the United States population and is associated with increased morbidity and mortality, and use of high-cost healthcare services including hospitalizations and emergency department visits. Integrating behavioral and physical healthcare may improve care for consumers with SMI, but prior research findings have been mixed. This quantitative retrospective cohort study assessed whether there was a predictive relationship between integrated healthcare clinic enrollment and inpatient and emergency department utilization for consumers with SMI when controlling for demographic characteristics and disease severity. While findings indicated no statistically significant impact of integrated care clinic enrollment on utilization, the sample had lower levels of utilization than would have been expected. Since policy and payment structures continue to support integrated care models, further research on different programs are encouraged, as each setting and practice pattern is unique.

Performance of Electronic Prediction Rules for Prevalent Delirium at Hospital Admission

IMPORTANCE

Delirium at admission is associated with increased hospital morbidity and mortality, but it may be missed in up to 70% of cases. Use of a predictive algorithm in an electronic medical record (EMR) system could provide critical information to target assessment of those with delirium at admission.

OBJECTIVES

To develop and assess a prediction rule for delirium using 2 populations of veterans and compare this rule with previously confirmed rules.

DESIGN, SETTING, AND PARTICIPANTS

In a diagnostic study, randomly selected EMRs of hospitalized veterans from the Veterans Affairs (VA) External Peer Review Program at 118 VA medical centers with inpatient facilities were reviewed for delirium risk factors associated with the National Institute for Health and Clinical Excellence (NICE) delirium rule in a derivation cohort (October 1, 2012, to September 30, 2013) and a confirmation cohort (October 1, 2013, to March 31, 2014). Delirium within 24 hours of admission was identified using key word terms. A total of 39 377 veterans 65 years or older who were admitted to a VA medical center for congestive heart failure, acute coronary syndrome, community-acquired pneumonia, and chronic obstructive pulmonary disease were included in the study.

EXPOSURE

The EMR calculated delirium risk.

MAIN OUTCOMES AND MEASURES

Delirium at admission as identified by trained nurse reviewers was the main outcome measure. Random forest methods were used to identify accurate risk factors for prevalent delirium. A prediction rule for prevalent delirium was developed, and its diagnostic accuracy was tested in the confirmation cohort. This consolidated NICE rule was compared with previously confirmed scoring algorithms (electronic NICE and Pendlebury NICE).

RESULTS

A total of 27 625 patients were included in the derivation cohort (28 118 [92.2%] male; mean [SD] age, 75.95 [8.61] years) and 11 752 in the confirmation cohort (11 536 [98.2%] male; mean [SD] age, 75.43 [8.55] years). Delirium at admission was identified in 2343 patients (8.5%) in the derivation cohort and 882 patients (7.0%) in the confirmation cohort. Modeling techniques identified cognitive impairment, infection, sodium level, and age of 80 years or older as the dominant risk factors. The consolidated NICE rule (area under the receiver operating characteristic [AUROC] curve, 0.91; 95% CI, 0.91-0.92; P < .001) had significantly higher discriminatory function than the eNICE rule (AUROC curve, 0.81; 95% CI, 0.80-0.82; P < .001) or Pendlebury NICE rule (AUROC curve, 0.87; 95% CI, 0.86-0.88; P < .001). These findings were confirmed in the confirmation cohort.

CONCLUSIONS AND RELEVANCE

This analysis identified preexisting cognitive impairment, infection, sodium level, and age of 80 years or older as delirium screening targets. Use of this algorithm in an EMR system could direct clinical assessment efforts to patients with delirium at admission.

Technology-enabled performance monitoring in intensive care: An integrative literature review

BACKGROUND

Implementation of evidence-based bundles in intensive care units is integral to improving quality of care and patient outcomes. However, it increases the burden of data collection and analysis required for performance monitoring and feedback of an inter-disciplinary care team. Health information technology including electronic health records and data analytic tools could automate this process and provide real-time feedback to the team.

AIM

This integrative literature review aimed to examine the extent to which technology-enabled performance monitoring and feedback contributed to improving quality of care and patient outcomes when implementing evidence-based bundles.

METHODS

A literature search of scientific databases was conducted using PubMed, Embase, Scopus, CINHAL and Ovid Medline.

RESULTS

Of nine studies included in this review, all reported improved compliance of the team with evidence-based bundles, ranging from 3% to 60% post implementation of technology-enabled performance monitoring and feedback. Significant reductions (p < .05) in hospital acquired infections were also reported in five studies.

CONCLUSIONS

Overall, the addition of documentation fields to electronic health records was essential in providing real-time feedback to teams and improving their compliance with evidence-based bundles. Further research is needed to assess the effectiveness of technology-enabled performance monitoring and feedback in improving patient outcomes on a larger scale, especially in resource-limited settings such as community hospitals.

Development and Validation of Electronic Quality Measures to Assess Care for Patients With Transient Ischemic Attack and Minor Ischemic Stroke

BACKGROUND

Despite interest in using electronic health record (EHR) data to assess quality of care, the accuracy of such data is largely unknown. We sought to develop and validate transient ischemic attack and minor ischemic stroke electronic quality measures (eQMs) using EHR data.

METHODS AND RESULTS

A random sample of patients with transient ischemic attack or minor ischemic stroke, cared for in Veterans Health Administration facilities (fiscal year 2011), was identified. We constructed 31 eQMs based on existing quality measures. Chart review was the criterion standard for validating the eQMs. To evaluate eQMs in terms of eligibility, we calculated the proportion of patients who were genuinely not eligible to receive a process (based on chart review) and who were correctly identified as not eligible by the EHR data (specificity). To assess eQMs about classification of whether patients received a process, we calculated the proportion of patients who actually received the process (based on chart review) and who were classified correctly by the EHR data as passing (sensitivity). Seven hundred sixty-three patients were included. About eligibility, specificity varied from 25% (brain imaging; carotid imaging) to 99% (anticoagulation quality). About pass rates, sensitivity varied from 30% (antihypertensive class) to 100% (coronary risk assessment; international normalized ratio measured). The 16 eQMs with ≥70% specificity in eligibility and ≥70% sensitivity in pass rates included coronary risk assessment, international normalized ratio measured, HbA1c measurement, speech language pathology consultation, anticoagulation for atrial fibrillation, discharge on statin, lipid management, neurology consultation, Holter, deep vein thrombosis prophylaxis, oral hypoglycemic intensification, cholesterol medication intensification, antihypertensive intensification, antihypertensive class, carotid stenosis intervention, and substance abuse referral for alcohol.

CONCLUSIONS

It is feasible to construct valid eQMs for processes of transient ischemic attack and minor ischemic stroke care. Healthcare systems with EHRs should consider using electronic data to evaluate care for their patients with transient ischemic attack and to complement and expand quality measurement programs currently focused on patients with stroke.

Comparison of Risk Factor Control in the Year After Discharge for Ischemic Stroke Versus Acute Myocardial Infarction

Background and Purpose—

The Veterans Health Administration has engaged in quality improvement to improve vascular risk factor control. We sought to examine blood pressure (<140/90 mm Hg), lipid (LDL [low-density lipoprotein] cholesterol <100 mg/dL), and glycemic control (hemoglobin A1c <9%), in the year post-hospitalization for acute ischemic stroke or acute myocardial infarction (AMI).

Methods—

We identified patients who were hospitalized (fiscal year 2011) with ischemic stroke, AMI, congestive heart failure, transient ischemic attack, or pneumonia/chronic obstructive pulmonary disease. The primary analysis compared risk factor control after incident ischemic stroke versus AMI. Facilities were included if they cared for ≥25 ischemic stroke and ≥25 AMI patients. A generalized linear mixed model including patient- and facility-level covariates compared risk factor control across diagnoses.

Results—

Forty thousand two hundred thirty patients were hospitalized (n=75 facilities): 2127 with incident ischemic stroke and 4169 with incident AMI. Fewer stroke patients achieved blood pressure control than AMI patients (64%; 95% confidence interval, 0.62–0.67 versus 77%; 95% confidence interval, 0.75–0.78; P <0.0001). After adjusting for patient and facility covariates, the odds of blood pressure control were still higher for AMI than ischemic stroke patients (odds ratio, 1.39; 95% confidence interval, 1.21–1.51). There were no statistical differences for AMI versus stroke patients in hyperlipidemia ( P =0.534). Among patients with diabetes mellitus, the odds of glycemic control were lower for AMI than ischemic stroke patients (odds ratio, 0.72; 95% confidence interval, 0.54–0.96).

Conclusions—

Given that hypertension control is a cornerstone of stroke prevention, interventions to improve poststroke hypertension management are needed.

Automating Quality Measures for Heart Failure Using Natural Language Processing: A Descriptive Study in the Department of Veterans Affairs

Background

We developed an accurate, stakeholder-informed, automated, natural language processing (NLP) system to measure the quality of heart failure (HF) inpatient care, and explored the potential for adoption of this system within an integrated health care system.

Objective

To accurately automate a United States Department of Veterans Affairs (VA) quality measure for inpatients with HF.

Methods

We automated the HF quality measure Congestive Heart Failure Inpatient Measure 19 (CHI19) that identifies whether a given patient has left ventricular ejection fraction (LVEF) <40%, and if so, whether an angiotensin-converting enzyme inhibitor or angiotensin-receptor blocker was prescribed at discharge if there were no contraindications. We used documents from 1083 unique inpatients from eight VA medical centers to develop a reference standard (RS) to train (n=314) and test (n=769) the Congestive Heart Failure Information Extraction Framework (CHIEF). We also conducted semi-structured interviews (n=15) for stakeholder feedback on implementation of the CHIEF.

Results

The CHIEF classified each hospitalization in the test set with a sensitivity (SN) of 98.9% and positive predictive value of 98.7%, compared with an RS and SN of 98.5% for available External Peer Review Program assessments. Of the 1083 patients available for the NLP system, the CHIEF evaluated and classified 100% of cases. Stakeholders identified potential implementation facilitators and clinical uses of the CHIEF.

Conclusions

The CHIEF provided complete data for all patients in the cohort and could potentially improve the efficiency, timeliness, and utility of HF quality measurements.

Clinical Quality and the Patient-Centered Medical Home

This study assess the association between the elements of the patient-centered medical home model and clinical quality in the Veterans Health Administration’s Patient Aligned Care Team program.

Assessing adherence to guidelines with administrative data in psychiatric outpatients

RATIONALE, AIMS AND OBJECTIVES

To assess (feasibility) of adherence to treatment guidelines among outpatients with common mental disorders in a routine Dutch clinical outpatient setting for common mental disorders using administrative data.

METHODS

In a retrospective cohort study, we analysed routinely collected administrative data of 5346 patients, treated for mood, anxiety or somatoform disorders with pharmacotherapy, psychotherapy or a combination of both. Available administrative data allowed assessment of guideline adherence with a disorder-independent set of five quality indicators, assessing psychotherapy, pharmacotherapy, a combination of both and routine outcome measurements (ROM) during diagnostic and therapeutic phases. Associations between the socio-demographic variables age, gender, clinical diagnosis and treatment type on the one hand and non-adherence to guidelines were tested using logistic regression analysis.

RESULTS

Patients were aged 39.5 years (SD 13.0) on average. The majority of patients were treated with a combination of pharmacotherapy and psychotherapy (50.1%), followed by psychotherapy (44.2%) and pharmacotherapy (5.6%). The majority of patients were suffering from a mood disorder (50.0%), followed by anxiety (43.9%) and somatoform disorders (6.1%). A diagnosis of anxiety or somatoform disorder was associated with higher odds of suboptimal duration [odds ratio (OR): 1.55 and 1.82[ and suboptimal frequency of psychotherapeutic treatment (OR of 0.89 and 0.63), and absence of ROM in the diagnostic phase (ORs 1.31 and 1.36, respectively) compared with depressive disorders. No ROM in the diagnostic phase was also predicted for by increasing age (ORs for the age categories of 56 and older of 1.48).

CONCLUSIONS

In this proof of principal study, we were able to assess some key indicators assessing adherence to clinical guidelines by using administrative data. Also, we could identify predictors of adherence with simple parameters available in every administrative data. Administrative data could help to monitor and aid guideline adherence in routine care, although quality may vary between settings.

Data Quality in Electronic Health Records Research: Quality Domains and Assessment Methods

The proliferation of the electronic health record (EHR) has led to increasing interest and opportunities for nurse scientists to use EHR data in a variety of research designs. However, methodological problems pertaining to data quality may arise when EHR data are used for nonclinical purposes. Therefore, this article describes common domains of data quality and approaches for quality appraisal in EHR research. Common data quality domains include data accuracy, completeness, consistency, credibility, and timeliness. Approaches for quality appraisal include data validation with data rules, evaluation and verification of data abstraction methods with statistical measures, data comparisons with manual chart review, management of missing data using statistical methods, and data triangulation between multiple EHR databases. Quality data enhance the validity and reliability of research findings, form the basis for conclusions derived from the data, and are, thus, an integral component in EHR-based study design and implementation.

Agreement Between HEDIS Performance Assessments in the VA and Medicare Advantage: Is Quality in the Eye of the Beholder?

Medicare Advantage (MA) plans and the Veterans Affairs (VA) health care system assess quality of care using standardized Healthcare Effectiveness Data and Information Set (HEDIS) performance measures. Little is known, however, about the relative accuracy of quality indicators for persons receiving care in more than one health care system. Among Veterans dually enrolled in an MA plan, we examined the agreement between MA and VA HEDIS assessments. Our study tested the hypothesis that private health plans underreport quality of care relative to a fully integrated delivery system utilizing a comprehensive electronic health record. Despite assessing the same individuals using identical measure specifications, reported VA performance was significantly better than reported MA performance for all 12 HEDIS measures. The VA’s performance advantage ranged from 9.8% (glycosylated hemoglobin [HbA1c] < 7.0% in diabetes) to 54.7% (blood pressure < 140/90 mm Hg in diabetes). The overall agreement between VA and MA HEDIS assessments ranged from 38.5% to 62.6%. Performance rates derived from VA and MA aggregate data were 1.6% to 14.3% higher than those reported by VA alone. This analysis suggests that neither MA plans nor the VA fully capture quality of care information for dually enrolled persons. However, the VA’s system-wide electronic health record may allow for more complete capture of quality information across multiple providers and settings.

Validation of Stroke Meaningful Use Measures in a National Electronic Health Record System

BACKGROUND

The Meaningful Use (MU) program has increased the national emphasis on electronic measurement of hospital quality.

OBJECTIVE

To evaluate stroke MU and one VHA stroke electronic clinical quality measure (eCQM) in national VHA data and determine sources of error in using centralized electronic health record (EHR) data.

DESIGN

Our study is a retrospective cross-sectional study of stroke quality measure eCQMs vs. chart review in a national EHR. We developed local SQL algorithms to generate the eCQMs, then modified them to run on VHA Central Data Warehouse (CDW) data. eCQM results were generated from CDW data in 2130 ischemic stroke admissions in 11 VHA hospitals. Local and CDW results were compared to chart review.

MAIN MEASURES

We calculated the raw proportion of matching cases, sensitivity/specificity, and positive/negative predictive values (PPV/NPV) for the numerators and denominators of each eCQM. To assess overall agreement for each eCQM, we calculated a weighted kappa and prevalence-adjusted bias-adjusted kappa statistic for a three-level outcome: ineligible, eligible-passed, or eligible-failed.

KEY RESULTS

In five eCQMs, the proportion of matched cases between CDW and chart ranged from 95.4 %-99.7 % (denominators) and 87.7 %-97.9 % (numerators). PPVs tended to be higher (range 96.8 %-100 % in CDW) with NPVs less stable and lower. Prevalence-adjusted bias-adjusted kappas for overall agreement ranged from 0.73-0.95. Common errors included difficulty in identifying: (1) mechanical VTE prophylaxis devices, (2) hospice and other specific discharge disposition, and (3) contraindications to receiving care processes.

CONCLUSIONS

Stroke MU indicators can be relatively accurately generated from existing EHR systems (nearly 90 % match to chart review), but accuracy decreases slightly in central compared to local data sources. To improve stroke MU measure accuracy, EHRs should include standardized data elements for devices, discharge disposition (including hospice and comfort care status), and recording contraindications.

Linkage of Laboratory Results to Medicare Fee-for-Service Claims

BACKGROUND

Medicare is the single largest purchaser of laboratory testing in the United States, yet test results associated with Medicare laboratory claims have historically not been available.

OBJECTIVE

The purpose of this study was to describe both the linkage of laboratory results data to Medicare claims and the completeness of these results data. In a subgroup of beneficiaries initiating angiotensin-converting enzyme inhibitors or angiotensin II receptor blockers, we also demonstrate the generalizability of Medicare beneficiaries with laboratory values compared with those without laboratory values. We end with a discussion of the limitations and potential uses of these linked data.

METHODS

We obtained information about laboratory orders and results for all Medicare fee-for-service beneficiaries in 10 eastern states who had outpatient laboratory tests conducted by a large national laboratory services vendor in 2011. Using a combination of direct identifiers and patient demographic characteristics, we linked patients in these laboratory data to Medicare beneficiaries, enabling us to associate test results with existing claims.

RESULTS

Nearly all patients in the laboratory data were able to be linked to Medicare beneficiaries. There were over 2 million distinct beneficiaries with nearly 125 million specific test results in the laboratory data. For specific tests ordered in an outpatient or office setting in these 10 states, between 5% and 15% of them had linked laboratory data. Medicare beneficiaries initiating angiotensin-converting enzyme inhibitors or angiotensin II receptor blockers who had laboratory results data had similar patient characteristics to those without results data.

CONCLUSIONS

This novel linkage of laboratory results data to Medicare claims creates unprecedented opportunities for conducting comparative effectiveness research related to patient safety and quality.

Validation of a Delirium Risk Assessment Using Electronic Medical Record Information

OBJECTIVE

Identifying patients at risk for delirium allows prompt application of prevention, diagnostic, and treatment strategies; but is rarely done. Once delirium develops, patients are more likely to need posthospitalization skilled care. This study developed an a priori electronic prediction rule using independent risk factors identified in a National Center of Clinical Excellence meta-analysis and validated the ability to predict delirium in 2 cohorts.

DESIGN

Retrospective analysis followed by prospective validation.

SETTING

Tertiary VA Hospital in New England.

PARTICIPANTS

A total of 27,625 medical records of hospitalized patients and 246 prospectively enrolled patients admitted to the hospital.

MEASUREMENTS

The electronic delirium risk prediction rule was created using data obtained from the patient electronic medical record (EMR). The primary outcome, delirium, was identified 2 ways: (1) from the EMR (retrospective cohort) and (2) clinical assessment on enrollment and daily thereafter (prospective participants). We assessed discrimination of the delirium prediction rule with the C-statistic. Secondary outcomes were length of stay and discharge to rehabilitation.

RESULTS

Retrospectively, delirium was identified in 8% of medical records (n = 2343); prospectively, delirium during hospitalization was present in 26% of participants (n = 64). In the retrospective cohort, medical record delirium was identified in 2%, 3%, 11%, and 38% of the low, intermediate, high, and very high-risk groups, respectively (C-statistic = 0.81; 95% confidence interval 0.80-0.82). Prospectively, the electronic prediction rule identified delirium in 15%, 18%, 31%, and 55% of these groups (C-statistic = 0.69; 95% confidence interval 0.61-0.77). Compared with low-risk patients, those at high- or very high delirium risk had increased length of stay (5.7 ± 5.6 vs 3.7 ± 2.7 days; P = .001) and higher rates of discharge to rehabilitation (8.9% vs 20.8%; P = .02).

CONCLUSIONS

Automatic calculation of delirium risk using an EMR algorithm identifies patients at risk for delirium, which creates a critical opportunity for gaining clinical efficiencies and improving delirium identification, including those needing skilled care.

Expectant management of veterans with early-stage prostate cancer

BACKGROUND

For certain men with low-risk prostate cancer, aggressive treatment results in marginal survival benefits while exposing them to urinary and sexual side effects. Nevertheless, expectant management has been underused. In the current study, the authors evaluated the association between various factors and expectant management use among veterans diagnosed with prostate cancer.

METHODS

The authors identified men diagnosed with prostate cancer in 2008. The outcome of interest was use of expectant management, based on documentation captured through an in-depth chart review. Multivariable regression models were fit to examine associations between use of expectant management and patient demographics, cancer severity, and facility characteristics. The authors assessed variation across 21 tertiary care regions and 52 facilities by generating predicted probabilities for receipt of expectant management.

RESULTS

Expectant management was more common among patients aged ≥75 years (40% vs 27% for those aged < 55 years; odds ratio, 2.57) and those with low-risk tumors (49% vs 20% for patients with high-risk tumors; odds ratio, 5.35). There was no association noted between patient comorbidity and receipt of expectant management (P = .90). There were also no associations found between facility factors and use of expectant management (all P>.05). Among ideal candidates for expectant management, receipt of expectant management varied considerably across individual facilities (0%-85%; P<.001).

CONCLUSIONS

Patient age and tumor risk were found to be more strongly associated with use of expectant management than patient comorbidity. Although use of expectant management appears broadly appropriate, there was variation in expectant management noted between hospitals that was apparently not attributable to facility factors. Research determining the basis of this variation, with a focus on providers, will be critical to help optimize prostate cancer treatment for veterans.

Balancing Model Performance and Simplicity to Predict Postoperative Primary Care Blood Pressure Elevation

BACKGROUND

Because of uncertainty regarding the reliability of perioperative blood pressures and traditional notions downplaying the role of anesthesiologists in longitudinal patient care, there is no consensus for anesthesiologists to recommend postoperative primary care blood pressure follow-up for patients presenting for surgery with an increased blood pressure. The decision of whom to refer should ideally be based on a predictive model that balances performance with ease-of-use. If an acceptable decision rule was developed, a new practice paradigm integrating the surgical encounter into broader public health efforts could be tested, with the goal of reducing long-term morbidity from hypertension among surgical patients.

METHODS

Using national data from US veterans receiving surgical care, we determined the prevalence of poorly controlled outpatient clinic blood pressures ≥140/90 mm Hg, based on the mean of up to 4 readings in the year after surgery. Four increasingly complex logistic regression models were assessed to predict this outcome. The first included the mean of 2 preoperative blood pressure readings; other models progressively added a broad array of demographic and clinical data. After internal validation, the C-statistics and the Net Reclassification Index between the simplest and most complex models were assessed. The performance characteristics of several simple blood pressure referral thresholds were then calculated.

RESULTS

Among 215,621 patients, poorly controlled outpatient clinic blood pressure was present postoperatively in 25.7% (95% confidence interval [CI], 25.5%-25.9%) including 14.2% (95% CI, 13.9%-14.6%) of patients lacking a hypertension history. The most complex prediction model demonstrated statistically significant, but clinically marginal, improvement in discrimination over a model based on preoperative blood pressure alone (C-statistic, 0.736 [95% CI, 0.734-0.739] vs 0.721 [95% CI, 0.718-0.723]; P for difference <0.0001). The Net Reclassification Index was 0.088 (95% CI, 0.082-0.093); P < 0.0001. A preoperative blood pressure threshold ≥150/95 mm Hg, calculated as the mean of 2 readings, identified patients more likely than not to demonstrate outpatient clinic blood pressures in the hypertensive range. Four of 5 patients not meeting this criterion were indeed found to be normotensive during outpatient clinic follow-up (positive predictive value, 51.5%; 95% CI, 51.0-52.0; negative predictive value, 79.6%; 95% CI, 79.4-79.7).

CONCLUSIONS

In a national cohort of surgical patients, poorly controlled postoperative clinic blood pressure was present in >1 of 4 patients (95% CI, 25.5%-25.9%). Predictive modeling based on the mean of 2 preoperative blood pressure measurements performed nearly as well as more complicated models and may provide acceptable predictive performance to guide postoperative referral decisions. Future studies of the feasibility and efficacy of such referrals are needed to assess possible beneficial effects on long-term cardiovascular morbidity.

Chest pain syndromes are associated with high rates of recidivism and costs in young United States Veterans

Background

Recurrent chest pain is common in patients with and without coronary artery disease. The prevalence and burden of these symptoms on healthcare is unknown.

Objectives

To compare chest pain return visits (recidivism) in patients with unexplained chest pain (UCP) against reference group of patients with coronary artery disease (CAD) and estimate the annual cost of recurrent chest pain.

Methods

In a retrospective cohort study, a Veteran Affairs (VA) administrative and clinical database of Veterans who were deployed to or served in support of the wars in Iraq or Afghanistan was queried for first disease specific ICD-9 code to form two cohorts (UCP or CAD). Patients were followed between 09/2001-09/2010 for the first and cumulative return visits for UCP or cardiac pain (ACS or angina) to clinic, emergency department or admission; or for all-cause death. Time to return was analyzed using Cox regression and negative binomial models and adjusted for age, gender, race, marital status, and risk factors (hypertension, hyperlipidemia, diabetes, smoking and obesity). Direct total costs included inpatient, outpatient and fee basis (non-VA) costs.

Results

Of 749,036 patients, 20,521 had UCP and 5303 had CAD. UCP patients were young and had a lower burden of risk factors than CAD cohort (p < .01). Yet, these patients were likely to return earlier with any chest pain (adjusted Hazard Ratio [aHR] = 1.76; 95 % CI 1.65-1.88); or unexplained chest pain than CAD patients (aHR: 1.89; 95 % CI 1.77-2.01). UCP patients were also likely to return more frequently for any chest pain (aRate Ratio = 1.54; 95 % CI 1.43-1.64) or UCP than CAD patients (aRR =2.63; 95 % CI 2.43-2.87). Per 100 patients, the 1-year cumulative returns were 37 visits for reference group and 45 visits for UCP cohort. The annual costs for chest pain averaged $69,009 for CAD and $57,336 for UCP patients (log geometric mean ratio=1.25; 95 % CI 1.18-1.32).

Conclusion

Chest pain recidivism is common and costly even in patients without known CAD. We need evidence-based guidelines for these patients to minimize returns.

Electronic supplementary material

The online version of this article (doi:10.1186/s12875-015-0287-9) contains supplementary material, which is available to authorized users.

Adult obesity prevalence in primary care users: An exploration using Canadian Primary Care Sentinel Surveillance Network (CPCSSN) data

OBJECTIVES

This research examines the feasibility of using electronic medical records within the Canadian Primary Care Sentinel Surveillance Network (CPCSSN) for obesity surveillance in Canada by assessing obesity trends over time and comparing BMI distribution estimates from CPCSSN to those obtained from nationally representative surveys.

METHODS

Data from 2003-2012 on patients 18 years and older (n = 216,075) were extracted from the CPCSSN database. Patient information included demographics (age and sex) and anthropometric measures (height, weight, body mass index (BMI), waist circumference, and waist-to-hip ratio). Standard descriptive statistics were used to characterize the sample, including, as appropriate, means, proportions and medians. The BMI distribution of the CPCSSN population was compared to estimates from the Canadian Community Health Survey (CCHS) and the Canadian Health Measures Survey (CHMS) for the years: 2004, 2007-2009 and 2009-2011.

RESULTS

The estimated prevalence of obesity increased from 17.9% in 2003 to 30.8% in 2012. Obesity class I, II and III prevalence estimates from CPCSSN in 2009-2011 (18.0%, 95% CI: 17.8-18; 7.4%, 95% CI: 7.3-7.6; 4.2%, 95% CI: 4.1-4.3 respectively) were greater than those from the most recent (2009- 2011) cycle of the CHMS (16.2%, 95% CI: 14-18.7; 6.3%, 95% CI: 4.6-8.5; 3.7%, 95% CI: 2.8-4.8 respectively), however these differences were not statistically significant.

CONCLUSION

The data from CPCSSN present a unique opportunity for longitudinal obesity surveillance among primary care users in Canada, and offer prevalence estimates similar to those obtained from nationally representative survey data.

The Ethics of External Peer Review

External peer review can be a useful part of a quality improvement program. However, it can also be used for political and punitive purposes. The distinction between potentially useful and damaging review is largely an issue of the ethics of how it is done. The ethics of external peer review lie not only in the process itself, but also in the role of the pathologists performing the review. While there are many ethical issues involved in external peer review, the most important may be a dedication to due process, allowing the pathologist under review to respond to allegations, and an insistence on complete information prior to drawing conclusions. Well established criteria for external peer review may provide protection both for the pathologist under review against allegations of negligence or incompetence, but also for the reviewers against accusations of bad faith.

Follow-up care for alcohol misuse among OEF/OIF veterans with and without alcohol use disorders and posttraumatic stress disorder

Little is known about follow-up care for alcohol misuse in the Veterans Affairs (VA) health care system among Operations Enduring and Iraqi Freedom (OEF/OIF) veterans with and without alcohol use disorders (AUD) and/or posttraumatic stress disorder (PTSD). Using data from 4725 OEF/OIF VA outpatients with alcohol screening (2006-2010), we compared the prevalence of follow-up for alcohol misuse--brief intervention (BI) or referral to treatment--among patients with and without AUD and/or PTSD. Among 933 (19.7%) patients with alcohol misuse (AUDIT-C ≥5), 77.0% had AUD and/or PTSD. Rates of BI or referral for alcohol misuse were higher among patients with AUD (76.9%) and both AUD and PTSD (70.1%) compared to those with PTSD only (53.1%) and neither AUD nor PTSD (52.3%). Among OEF/OIF VA outpatients with alcohol misuse, those with AUD had higher rates of follow-up for alcohol misuse than those without, but PTSD was not associated with differential follow-up.

Defining and measuring completeness of electronic health records for secondary use

We demonstrate the importance of explicit definitions of electronic health record (EHR) data completeness and how different conceptualizations of completeness may impact findings from EHR-derived datasets. This study has important repercussions for researchers and clinicians engaged in the secondary use of EHR data. We describe four prototypical definitions of EHR completeness: documentation, breadth, density, and predictive completeness. Each definition dictates a different approach to the measurement of completeness. These measures were applied to representative data from NewYork-Presbyterian Hospital's clinical data warehouse. We found that according to any definition, the number of complete records in our clinical database is far lower than the nominal total. The proportion that meets criteria for completeness is heavily dependent on the definition of completeness used, and the different definitions generate different subsets of records. We conclude that the concept of completeness in EHR is contextual. We urge data consumers to be explicit in how they define a complete record and transparent about the limitations of their data.

Agreement between electronic medical record-based and self-administered pain numeric rating scale: clinical and research implications

BACKGROUND

Pain screening may improve the quality of care by identifying patients in need of further assessment and management. Many health care systems use the numeric rating scale (NRS) for pain screening, and record the score in the patients' electronic medical record (EMR).

OBJECTIVE

Determine the level of agreement between EMR and patient survey NRS, and whether discrepancies vary by demographic and clinical characteristics.

METHODS

We linked survey data from a sample of veterans receiving care in 8 Veterans Affairs medical facilities, to EMR data including an NRS collected on the day of the survey to compare responses to the NRS question from these 2 sources. We assessed correlation, agreement on clinical cut-points (eg, severe), and, using the survey as the gold standard, whether patient characteristics were associated with a discrepancy on moderate-severe pain.

RESULTS

A total of 1643 participants had a survey and EMR NRS score on the same day. The correlation was 0.56 (95% confidence interval, 0.52-0.59), but the mean EMR score was significantly lower than the survey score (1.72 vs. 2.79; P<0.0001). Agreement was moderate (κ=0.35). Characteristics associated with an increased odds of a discrepancy included: diabetes [adjusted odds ratio (AOR)=1.48], posttraumatic stress disorder (AOR=1.59), major depressive disorder (AOR=1.81), other race versus white (AOR=2.29), and facility in which care was received.

CONCLUSIONS

The underestimation of pain using EMR data, especially clinically actionable levels of pain, has important clinical and research implications. Improving the quality of pain care may require better screening.

Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research

OBJECTIVE

To review the methods and dimensions of data quality assessment in the context of electronic health record (EHR) data reuse for research.

MATERIALS AND METHODS

A review of the clinical research literature discussing data quality assessment methodology for EHR data was performed. Using an iterative process, the aspects of data quality being measured were abstracted and categorized, as well as the methods of assessment used.

RESULTS

Five dimensions of data quality were identified, which are completeness, correctness, concordance, plausibility, and currency, and seven broad categories of data quality assessment methods: comparison with gold standards, data element agreement, data source agreement, distribution comparison, validity checks, log review, and element presence.

DISCUSSION

Examination of the methods by which clinical researchers have investigated the quality and suitability of EHR data for research shows that there are fundamental features of data quality, which may be difficult to measure, as well as proxy dimensions. Researchers interested in the reuse of EHR data for clinical research are recommended to consider the adoption of a consistent taxonomy of EHR data quality, to remain aware of the task-dependence of data quality, to integrate work on data quality assessment from other fields, and to adopt systematic, empirically driven, statistically based methods of data quality assessment.

CONCLUSION

There is currently little consistency or potential generalizability in the methods used to assess EHR data quality. If the reuse of EHR data for clinical research is to become accepted, researchers should adopt validated, systematic methods of EHR data quality assessment.

A method to measure clinical practice patterns of breast cancer genomic diagnostics in health systems

Personalized genomic diagnostics, often in combination with targeted therapeutics, have become an important component of clinical oncologic practice over the last decade. However, there is a dearth of studies examining utilization of specific genomic diagnostics and their influence on clinical practice patterns in health systems. In order to begin to explore and understand the clinical utility of particular genomic diagnostics in current use, we developed an instrument to collect data on utilization and clinical practice patterns in health systems. We focused our efforts on gene-expression profiling (GEP) assays, particularly on a commonly used GEP in breast cancer: Oncotype DX® (Genomic Health, Inc., CA, USA). This article presents a method we have developed for the retrospective collection of data on the utilization of GEP among breast cancer patients of a real-world, real-time health system.

Automated extraction of ejection fraction for quality measurement using regular expressions in Unstructured Information Management Architecture (UIMA) for heart failure

OBJECTIVES

Left ventricular ejection fraction (EF) is a key component of heart failure quality measures used within the Department of Veteran Affairs (VA). Our goals were to build a natural language processing system to extract the EF from free-text echocardiogram reports to automate measurement reporting and to validate the accuracy of the system using a comparison reference standard developed through human review. This project was a Translational Use Case Project within the VA Consortium for Healthcare Informatics.

MATERIALS AND METHODS

We created a set of regular expressions and rules to capture the EF using a random sample of 765 echocardiograms from seven VA medical centers. The documents were randomly assigned to two sets: a set of 275 used for training and a second set of 490 used for testing and validation. To establish the reference standard, two independent reviewers annotated all documents in both sets; a third reviewer adjudicated disagreements.

RESULTS

System test results for document-level classification of EF of <40% had a sensitivity (recall) of 98.41%, a specificity of 100%, a positive predictive value (precision) of 100%, and an F measure of 99.2%. System test results at the concept level had a sensitivity of 88.9% (95% CI 87.7% to 90.0%), a positive predictive value of 95% (95% CI 94.2% to 95.9%), and an F measure of 91.9% (95% CI 91.2% to 92.7%).

DISCUSSION

An EF value of <40% can be accurately identified in VA echocardiogram reports.

CONCLUSIONS

An automated information extraction system can be used to accurately extract EF for quality measurement.

Increased documented brief alcohol interventions with a performance measure and electronic decision support

BACKGROUND

Alcohol screening and brief interventions (BIs) are ranked the third highest US prevention priority, but effective methods of implementing BI into routine care have not been described.

OBJECTIVES

This study evaluated the prevalence of documented BI among Veterans Affairs (VA) outpatients with alcohol misuse before, during, and after implementation of a national performance measure (PM) linked to incentives and dissemination of an electronic clinical reminder (CR) for BI.

METHODS

VA outpatients were included in this study if they were randomly sampled for national medical record reviews and screened positive for alcohol misuse (Alcohol Use Disorders Identification Test-Consumption score ≥5) between July 2006 and September 2008 (N=6788). Consistent with the PM, BI was defined as documented advice to reduce or abstain from drinking plus feedback linking drinking to health. The prevalence of BI was evaluated among outpatients who screened positive for alcohol misuse during 4 successive phases of BI implementation: baseline year (n=3504), after announcement (n=753) and implementation (n=697) of the PM, and after CR dissemination (n=1834), unadjusted and adjusted for patient characteristics.

RESULTS

Among patients with alcohol misuse, the adjusted prevalence of BI increased significantly over successive phases of BI implementation, from 5.5% (95% CI 4.1%-7.5%), 7.6% (5.6%-10.3%), 19.1% (15.4%-23.7%), to 29.0% (25.0%-33.4%) during the baseline year, after PM announcement, PM implementation, and CR dissemination, respectively (test for trend P<0.001).

CONCLUSIONS

A national PM supported by dissemination of an electronic CR for BI was associated with meaningful increases in the prevalence of documented brief alcohol interventions.

Quality concerns with routine alcohol screening in VA clinical settings

BACKGROUND

Alcohol screening questionnaires have typically been validated when self- or researcher-administered. Little is known about the performance of alcohol screening questionnaires administered in clinical settings.

OBJECTIVE

The purpose of this study was to compare the results of alcohol screening conducted as part of routine outpatient clinical care in the Veterans Affairs (VA) Health Care System to the results on the same alcohol screening questionnaire completed on a mailed survey within 90 days and identify factors associated with discordant screening results.

DESIGN

Cross sectional.

PARTICIPANTS

A national sample of 6,861 VA outpatients (fiscal years 2007-2008) who completed the AUDIT-C alcohol screening questionnaire on mailed surveys (survey screen) within 90 days of having clinical AUDIT-C screening documented in their medical records (clinical screen).

MAIN MEASURES

Alcohol screening results were considered discordant if patients screened positive (AUDIT-C ≥ 5) on either the clinical or survey screen but not both. Multivariable logistic regression was used to estimate the prevalence of discordance in different patient subgroups based on demographic and clinical characteristics, VA network and temporal factors (e.g. the order of screens).

KEY RESULTS

Whereas 11.1% (95% CI 10.4-11.9%) of patients screened positive for unhealthy alcohol use on the survey screen, 5.7% (5.1- 6.2%) screened positive on the clinical screen. Of 765 patients who screened positive on the survey screen, 61.2% (57.7-64.6%) had discordant results on the clinical screen, contrasted with 1.5% (1.2-1.8%) of 6096 patients who screened negative on the survey screen. In multivariable analyses, discordance was significantly increased among Black patients compared with White, and among patients who had a positive survey AUDIT-C screen or who received care at 4 of 21 VA networks.

CONCLUSION

Use of a validated alcohol screening questionnaire does not-by itself-ensure the quality of alcohol screening. This study suggests that the quality of clinical alcohol screening should be monitored, even when well-validated screening questionnaires are used.

The use of databases and registries to enhance colonoscopy quality

Administrative databases, registries, and clinical databases are designed for different purposes and therefore have different advantages and disadvantages in providing data for enhancing quality. Administrative databases provide the advantages of size, availability, and generalizability, but are subject to constraints inherent in the coding systems used and from data collection methods optimized for billing. Registries are designed for research and quality reporting but require significant investment from participants for secondary data collection and quality control. Electronic health records contain all of the data needed for quality research and measurement, but that data is too often locked in narrative text and unavailable for analysis. National mandates for electronic health record implementation and functionality will likely change this landscape in the near future.

Quality of nonmetastatic colorectal cancer care in the Department of Veterans Affairs

PURPOSE

The Veterans Affairs (VA) healthcare system treats approximately 3% of patients with cancer in the United States each year. We measured the quality of nonmetastatic colorectal cancer (CRC) care in VA as indicated by concordance with National Comprehensive Cancer Network practice guidelines (six indicators) and timeliness of care (three indicators).

PATIENTS AND METHODS

A retrospective medical record abstraction was done for 2,492 patients with incident stages I to III CRC diagnosed between October 1, 2003, and March 31, 2006, who underwent definitive CRC surgery. Patients were treated at one or more of 128 VA medical centers. The proportion of patients receiving guideline-concordant care and time intervals between care processes were calculated.

RESULTS

More than 80% of patients had preoperative carcinoembryonic antigen determination (ie, stages II to III disease) and documented clear surgical margins (ie, stages II to III disease). Between 72% and 80% of patients had appropriate referral to a medical oncologist (ie, stages II to III disease), preoperative computed tomography scan of the abdomen and pelvis (ie, stages II to III disease), and adjuvant fluorouracil-based chemotherapy (ie, stage III disease). Less than half of patients with stages I to III CRC (43.5%) had a follow-up colonoscopy 7 to 18 months after surgery. The mean number of days between major treatment events included the following: 26.6 days (standard deviation [SD], 38.2; median, 20 days) between diagnosis and initiation of treatment (in stages II to III disease); 64.8 [corrected] days (SD, 54.9; median, 50 days) between definitive surgery and start of adjuvant chemotherapy (in stages II to III disease); and 444.2 [corrected] days (SD, 182.1; median, 393 days) between definitive surgery and follow-up colonoscopies (in stages I to III disease).

CONCLUSION

Although there is opportunity for improvement in the area of cancer surveillance, the VA performs well in meeting established guidelines for diagnosis and treatment of CRC.