Facility-level program components leading to population impact: a coincidence analysis of obesity treatment options within the Veterans Health Administration

Obesity is a well-established risk factor for increased morbidity and mortality. Comprehensive lifestyle interventions, pharmacotherapy, and bariatric surgery are three effective treatment approaches for obesity. The Veterans Health Administration (VHA) offers all three domains but in different configurations across medical facilities. Study aim was to explore the relationship between configurations of three types of obesity treatments, context, and population impact across VHA using coincidence analysis. This was a cross-sectional analysis of survey data describing weight management treatment components linked with administrative data to compute population impact for each facility. Coincidence analysis was used to identify combinations of treatment components that led to higher population impact. Facilities with higher impact were in the top two quintiles for (1) reach to eligible patients and (2) weight outcomes. Sixty-nine facilities were included in the analyses. The final model explained 88% (29/33) of the higher-impact facilities with 91% consistency (29/32) and was comprised of five distinct pathways. Each of the five pathways depended on facility complexity-level plus factors from one or more of the three domains of weight management: comprehensive lifestyle interventions, pharmacotherapy, and/or bariatric surgery. Three pathways include components from multiple treatment domains. Combinations of conditions formed "recipes" that lead to higher population impact. Our coincidence analyses highlighted both the importance of local context and how combinations of specific conditions consistently and uniquely distinguished higher impact facilities from lower impact facilities for weight management.

Interpretability, credibility, and usability of hospital-specific template matching versus regression-based hospital performance assessments; a multiple methods study

BACKGROUND

Hospital-specific template matching (HS-TM) is a newer method of hospital performance assessment.

OBJECTIVE

To assess the interpretability, credibility, and usability of HS-TM-based vs. regression-based performance assessments.

RESEARCH DESIGN

We surveyed hospital leaders (January-May 2021) and completed follow-up semi-structured interviews. Surveys included four hypothetical performance assessment vignettes, with method (HS-TM, regression) and hospital mortality randomized.

SUBJECTS

Nationwide Veterans Affairs Chiefs of Staff, Medicine, and Hospital Medicine.

MEASURES

Correct interpretation; self-rated confidence in interpretation; and self-rated trust in assessment (via survey). Concerns about credibility and main uses (via thematic analysis of interview transcripts).

RESULTS

In total, 84 participants completed 295 survey vignettes. Respondents correctly interpreted 81.8% HS-TM vs. 56.5% regression assessments, p < 0.001. Respondents "trusted the results" for 70.9% HS-TM vs. 58.2% regression assessments, p = 0.03. Nine concerns about credibility were identified: inadequate capture of case-mix and/or illness severity; inability to account for specialized programs (e.g., transplant center); comparison to geographically disparate hospitals; equating mortality with quality; lack of criterion standards; low power; comparison to dissimilar hospitals; generation of rankings; and lack of transparency. Five concerns were equally relevant to both methods, one more pertinent to HS-TM, and three more pertinent to regression. Assessments were mainly used to trigger further quality evaluation (a "check oil light") and motivate behavior change.

CONCLUSIONS

HS-TM-based performance assessments were more interpretable and more credible to VA hospital leaders than regression-based assessments. However, leaders had a similar set of concerns related to credibility for both methods and felt both were best used as a screen for further evaluation.

Endoscopist-directed sedation rarely fails: implications for the value of anesthesia assistance for routine GI endoscopy

OBJECTIVES

Use of anesthesia-assisted (AA) sedation for routine gastrointestinal (GI) endoscopy has increased markedly. Clinical uncertainty about which patients are most likely to benefit from AA sedation contributes to this increased use. We aimed to estimate the prevalence of failed endoscopist-directed sedation and to identify patients at elevated risk of failing standard sedation.

STUDY DESIGN

Retrospective longitudinal study of national Veterans Health Administration (VA) data of all patients who underwent esophagogastroduodenoscopy and/or colonoscopy in 2009-2013.

METHODS

Using multivariable logistic regression, we sought to identify patient and procedural risk factors for failed sedation. Failed sedation cases were identified electronically and validated by chart review.

RESULTS

Of 302,247 standard sedation procedures performed at VA facilities offering AA sedation, we identified 313 cases of failed sedation (prevalence, 0.10%). None of the factors found to be associated with increased risk of failed sedation (eg, high-dose opioid use, younger age) had an odds ratio greater than 3. Even among the highest-risk patients (top decile), the prevalence of failed sedation was only 0.29%.

CONCLUSIONS

Failed sedation among patients undergoing routine outpatient GI endoscopy with standard sedation is very rare, even among patients at highest risk. This suggests that concerns regarding failed sedation due to commonly cited factors such as chronic opioid use and obesity do not justify forgoing standard sedation in favor of AA sedation in most patients. It also suggests that use of AA sedation is generally unnecessary. Reinstatement of endoscopist-directed sedation, rather than AA sedation, as the default sedation standard is warranted to reduce low-value care and prevent undue financial burdens on patients.

Hospital-specific Template Matching for Benchmarking Performance in a Diverse Multihospital System

BACKGROUND

Hospital-specific template matching is a newer method of hospital performance measurement that may be fairer than regression-based benchmarking. However, it has been tested in only limited research settings.

OBJECTIVE

The objective of this study was to test the feasibility of hospital-specific template matching assessments in the Veterans Affairs (VA) health care system and determine power to detect greater-than-expected 30-day mortality.

RESEARCH DESIGN

Observational cohort study with hospital-specific template matching assessment. For each VA hospital, the 30-day mortality of a representative subset of hospitalizations was compared with the pooled mortality from matched hospitalizations at a set of comparison VA hospitals treating sufficiently similar patients. The simulation was used to determine power to detect greater-than-expected mortality.

SUBJECTS

A total of 556,266 hospitalizations at 122 VA hospitals in 2017.

MEASURES

A number of comparison hospitals identified per hospital; 30-day mortality.

RESULTS

Each hospital had a median of 38 comparison hospitals (interquartile range: 33, 44) identified, and 116 (95.1%) had at least 20 comparison hospitals. In total, 8 hospitals (6.6%) had a significantly lower 30-day mortality than their benchmark, 5 hospitals (4.1%) had a significantly higher 30-day mortality, and the remaining 109 hospitals (89.3%) were similar to their benchmark. Power to detect a standardized mortality ratio of 2.0 ranged from 72.5% to 79.4% for a hospital with the fewest (6) versus most (64) comparison hospitals.

CONCLUSIONS

Hospital-specific template matching may be feasible for assessing hospital performance in the diverse VA health care system, but further refinements are needed to optimize the approach before operational use. Our findings are likely applicable to other large and diverse multihospital systems.

Replicating prediction algorithms for hospitalization and corticosteroid use in patients with inflammatory bowel disease

Introduction

Previous work had shown that machine learning models can predict inflammatory bowel disease (IBD)-related hospitalizations and outpatient corticosteroid use based on patient demographic and laboratory data in a cohort of United States Veterans. This study aimed to replicate this modeling framework in a nationally representative cohort.

Methods

A retrospective cohort design using Optum Electronic Health Records (EHR) were used to identify IBD patients, with at least 12 months of follow-up between 2007 and 2018. IBD flare was defined as an inpatient/emergency visit with a diagnosis of IBD or an outpatient corticosteroid prescription for IBD. Predictors included demographic and laboratory data. Logistic regression and random forest (RF) models were used to predict IBD flare within 6 months of each visit. A 70% training and 30% validation approach was used.

Results

A total of 95,878 patients across 780,559 visits were identified. Of these, 22,245 (23.2%) patients had at least one IBD flare. Patients were predominantly White (87.7%) and female (57.1%), with a mean age of 48.0 years. The logistic regression model had an area under the receiver operating curve (AuROC) of 0.66 (95% CI: 0.65−0.66), sensitivity of 0.69 (95% CI: 0.68−0.70), and specificity of 0.74 (95% CI: 0.73−0.74) in the validation cohort. The RF model had an AuROC of 0.80 (95% CI: 0.80−0.81), sensitivity of 0.74 (95% CI: 0.73−0.74), and specificity of 0.72 (95% CI: 0.72−0.72) in the validation cohort. Important predictors of IBD flare in the RF model were the number of previous flares, age, potassium, and white blood cell count.

Conclusion

The machine learning modeling framework was replicated and results showed a similar predictive accuracy in a nationally representative cohort of IBD patients. This modeling framework could be embedded in routine practice as a tool to distinguish high-risk patients for disease activity.

Facility-level conditions leading to higher reach: a configurational analysis of national VA weight management programming

Background

While the Veterans Health Administration (VHA) MOVE! weight management program is effective in helping patients lose weight and is available at every VHA medical center across the United States, reaching patients to engage them in treatment remains a challenge. Facility-based MOVE! programs vary in structures, processes of programming, and levels of reach, with no single factor explaining variation in reach. Configurational analysis, based on Boolean algebra and set theory, represents a mathematical approach to data analysis well-suited for discerning how conditions interact and identifying multiple pathways leading to the same outcome. We applied configurational analysis to identify facility-level obesity treatment program arrangements that directly linked to higher reach.

Methods

A national survey was fielded in March 2017 to elicit information about more than 75 different components of obesity treatment programming in all VHA medical centers. This survey data was linked to reach scores available through administrative data. Reach scores were calculated by dividing the total number of Veterans who are candidates for obesity treatment by the number of “new” MOVE! visits in 2017 for each program and then multiplied by 1000. Programs with the top 40 % highest reach scores (n = 51) were compared to those in the lowest 40 % (n = 51). Configurational analysis was applied to identify specific combinations of conditions linked to reach rates.

Results

One hundred twenty-seven MOVE! program representatives responded to the survey and had complete reach data. The final solution consisted of 5 distinct pathways comprising combinations of program components related to pharmacotherapy, bariatric surgery, and comprehensive lifestyle intervention; 3 of the 5 pathways depended on the size/complexity of medical center. The 5 pathways explained 78 % (40/51) of the facilities in the higher-reach group with 85 % consistency (40/47).

Conclusions

Specific combinations of facility-level conditions identified through configurational analysis uniquely distinguished facilities with higher reach from those with lower reach. Solutions demonstrated the importance of how local context plus specific program components linked together to account for a key implementation outcome. These findings will guide system recommendations about optimal program structures to maximize reach to patients who would benefit from obesity treatment such as the MOVE! program.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-06774-w.

Association of Anti-Tumor Necrosis Factor Therapy With Mortality Among Veterans With Inflammatory Bowel Disease

IMPORTANCE

Inflammatory bowel disease (IBD) is commonly treated with corticosteroids and anti-tumor necrosis factor (TNF) drugs; however, medications have well-described adverse effects. Prior work suggests that anti-TNF therapy may reduce all-cause mortality compared with prolonged corticosteroid use among Medicare and Medicaid beneficiaries with IBD.

OBJECTIVE

To examine the association between use of anti-TNF or corticosteroids and all-cause mortality in a national cohort of veterans with IBD.

DESIGN, SETTING, AND PARTICIPANTS

This cohort study used a well-established Veteran's Health Administration cohort of 2997 patients with IBD treated with prolonged corticosteroids (≥3000-mg prednisone equivalent and/or ≥600 mg of budesonide within a 12-month period) and/or new anti-TNF therapy from January 1, 2006, to October 1, 2015. Data were analyzed between July 1, 2019, and December 31, 2020.

EXPOSURES

Use of corticosteroids or anti-TNF.

MAIN OUTCOMES AND MEASURES

The primary end point was all-cause mortality as defined by the Veterans Health Administration vital status file. Marginal structural modeling was used to compare associations between anti-TNF therapy or corticosteroid use and all-cause mortality.

RESULTS

A total of 2997 patients (2725 men [90.9%]; mean [SD] age, 50.0 [17.4] years) were included in the final analysis, 1734 (57.9%) with Crohn disease (CD) and 1263 (42.1%) with ulcerative colitis (UC). All-cause mortality was 8.5% (n = 256) over a mean (SD) of 3.9 (2.3) years' follow-up. At cohort entry, 1836 patients were new anti-TNF therapy users, and 1161 were prolonged corticosteroid users. Anti-TNF therapy use was associated with a lower likelihood of mortality for CD (odds ratio [OR], 0.54; 95% CI, 0.31-0.93) but not for UC (OR, 0.33; 95% CI, 0.10-1.10). In a sensitivity analysis adjusting prolonged corticosteroid users to include patients receiving corticosteroids within 90 to 270 days after initiation of anti-TNF therapy, the OR for UC was statistically significant, at 0.33 (95% CI, 0.13-0.84), and the OR for CD was 0.55 (95% CI, 0.33-0.92).

CONCLUSIONS AND RELEVANCE

This study suggests that anti-TNF therapy may be associated with reduced mortality compared with long-term corticosteroid use among veterans with CD, and potentially among those with UC.

Construction and Use of Body Weight Measures from Administrative Data in a Large National Health System: A Systematic Review

OBJECTIVE

Administrative data are increasingly used in research and evaluation yet lack standardized guidelines for constructing measures using these data. Body weight measures from administrative data serve critical functions of monitoring patient health, evaluating interventions, and informing research. This study aimed to describe the algorithms used by researchers to construct and use weight measures.

METHODS

A structured, systematic literature review of studies that constructed body weight measures from the Veterans Health Administration was conducted. Key information regarding time frames and time windows of data collection, measure calculations, data cleaning, treatment of missing and outlier weight values, and validation processes was collected.

RESULTS

We identified 39 studies out of 492 nonduplicated records for inclusion. Studies parameterized weight outcomes as change in weight from baseline to follow-up (62%), weight trajectory over time (21%), proportion of participants meeting weight threshold (46%), or multiple methods (28%). Most (90%) reported total time in follow-up and number of time points. Fewer reported time windows (54%), outlier values (51%), missing values (34%), or validation strategies (15%).

CONCLUSIONS

A high variability in the operationalization of weight measures was found. Improving methods to construct clinical measures will support transparency and replicability in approaches, guide interpretation of findings, and facilitate comparisons across studies.

Developing a template matching algorithm for benchmarking hospital performance in a diverse, integrated healthcare system

Template matching is a proposed approach for hospital benchmarking, which measures performance based on matching a subset of comparable patient hospitalizations from each hospital. We assessed the ability to create the required matched samples and thus the feasibility of template matching to benchmark hospital performance in a diverse healthcare system.Nationwide Veterans Affairs (VA) hospitals, 2017.Observational cohort study.We used administrative and clinical data from 668,592 hospitalizations at 134 VA hospitals in 2017. A standardized template of 300 hospitalizations was selected, and then 300 hospitalizations were matched to the template from each hospital.There was substantial case-mix variation across VA hospitals, which persisted after excluding small hospitals, hospitals with primarily psychiatric admissions, and hospitalizations for rare diagnoses. Median age ranged from 57 to 75 years across hospitals; percent surgical admissions ranged from 0.0% to 21.0%; percent of admissions through the emergency department, 0.1% to 98.7%; and percent Hispanic patients, 0.2% to 93.3%. Characteristics for which there was substantial variation across hospitals could not be balanced with any matching algorithm tested. Although most other variables could be balanced, we were unable to identify a matching algorithm that balanced more than ∼20 variables simultaneously.We were unable to identify a template matching approach that could balance hospitals on all measured characteristics potentially important to benchmarking. Given the magnitude of case-mix variation across VA hospitals, a single template is likely not feasible for general hospital benchmarking.

The Association of Medications and Vaccination with Risk of Pneumonia in Inflammatory Bowel Disease

BACKGROUND

Patients with inflammatory bowel disease (IBD) are at increased risk for pneumonia, and corticosteroids are reported to amplify this risk. Less is known about the impact of corticosteroid-sparing IBD therapies on pneumonia risk or the efficacy of pneumococcal vaccination in reducing all-cause pneumonia in real-world IBD cohorts.

METHODS

We performed a population-based study using an established Veterans Health Administration cohort of 29,957 IBD patients. We identified all patients who developed bacterial pneumonia. Cox survival analysis was used to determine the association of corticosteroids at study entry and as a time-varying covariate, corticosteroid-sparing agents (immunomodulators and antitumor necrosis-alpha [TNF] inhibitors), and pneumococcal vaccination with the development of all-cause pneumonia.

RESULTS

Patients with IBD who received corticosteroids had a greater risk of pneumonia when controlling for age, gender, and comorbidities (hazard ratio [HR] 2.21; 95% confidence interval [CI], 1.90-2.57 for prior use; HR = 3.42; 95% CI, 2.92-4.01 for use during follow-up). Anti-TNF inhibitors (HR 1.52; 95% CI, 1.02-2.26), but not immunomodulators (HR 0.91; 95% CI, 0.77-1.07), were associated with a small increase in pneumonia. A history of pneumonia was strongly associated with subsequent pneumonia (HR = 4.41; 95% CI, 3.70-5.27). Less than 15% of patients were vaccinated against pneumococcus, and this was not associated with a reduced risk of pneumonia (HR = 1.02; 95% CI, 0.80-1.30) in this cohort.

CONCLUSION

In a large US cohort, corticosteroids were confirmed to increase pneumonia risk. Tumor necrosis-alpha inhibitors were associated with a smaller increase in the risk of pneumonia. Surprisingly, pneumococcal vaccination did not reduce all-cause pneumonia in this population, though few patients were vaccinated.

Inpatient and Discharge Fluoroquinolone Prescribing in Veterans Affairs Hospitals Between 2014 and 2017

Background

Between 2007 and 2015, inpatient fluoroquinolone use declined in US Veterans Affairs (VA) hospitals. Whether fluoroquinolone use at discharge also declined, in particular since antibiotic stewardship programs became mandated at VA hospitals in 2014, is unknown.

Methods

In this retrospective cohort study of hospitalizations with infection between January 1, 2014, and December 31, 2017, at 125 VA hospitals, we assessed inpatient and discharge fluoroquinolone (ciprofloxacin, levofloxacin, moxifloxacin) use as (a) proportion of hospitalizations with a fluoroquinolone prescribed and (b) fluoroquinolone-days per 1000 hospitalizations. After adjusting for illness severity, comorbidities, and age, we used multilevel logit and negative binomial models to assess for hospital-level variation and longitudinal prescribing trends.

Results

Of 560219 hospitalizations meeting inclusion criteria as hospitalizations with infection, 37.4% (209602/560219) had a fluoroquinolone prescribed either during hospitalization (32.5%, 182337/560219) or at discharge (19.6%, 110003/560219). Hospitals varied appreciably in inpatient, discharge, and total fluoroquinolone use, with 71% of hospitals in the highest prescribing quartile located in the Southern United States. Nearly all measures of fluoroquinolone use decreased between 2014 and 2017, with the largest decreases found in inpatient fluoroquinolone and ciprofloxacin use. In contrast, there was minimal decline in fluoroquinolone use at discharge, which accounted for a growing percentage of hospitalization-related fluoroquinolone-days (52.0% in 2014; 61.3% by 2017).

Conclusions

Between 2014 and 2017, fluoroquinolone use decreased in VA hospitals, largely driven by decreased inpatient fluoroquinolone (especially ciprofloxacin) use. Fluoroquinolone prescribing at discharge, as well as levofloxacin prescribing overall, is a growing target for stewardship.

Increased Cardiovascular Disease Risk in Veterans With Mental Illness

Background:

Although previous studies have demonstrated an association between various mental illnesses and cardio-cerebrovascular disease (CVD) risk, few have compared the strength of association between different mental illnesses and CVD risk.

Methods and Results:

We assessed the association of psychiatric diagnoses (psychosis, bipolar disorder, depression, anxiety, and posttraumatic stress disorder) with major CVD outcomes (CVD events and CVD mortality) over 5 years, using a national primary prevention cohort of military veterans receiving care in the Department of Veterans Affairs. Data were linked from the Department of Veterans Affairs, Centers for Medicare and Medicaid Services, and Centers for Disease Control and Prevention National Death Index databases. We used multiple logistic regression to examine how the presence of a psychiatric diagnosis at baseline (2005–2009) was associated with CVD outcomes over the next 5 years (January 1, 2010, to December 31, 2014) stratified by sex, adjusting for other psychiatric diagnoses, as well as age, race, conventional CVD risk factors as calculated by the Veterans Affairs Risk Score-CVD, and antipsychotic and anticonvulsant/mood stabilizer medication prescriptions. Approximately 1.52 million men and over 94 000 women met our inclusion criteria. In the fully adjusted model, among men, we found that depression, psychosis, and bipolar disorder were predictive of both CVD events and CVD mortality, with psychosis having the largest effect size (eg, adjusted odds ratio, 1.48; CI, 1.41–1.56; P <0.001 for psychosis and CVD mortality). Among women, only psychosis and bipolar disorder were predictive of both CVD events and CVD mortality, again with psychosis having the largest effect size (eg, adjusted odds ratio, 1.97; CI, 1.52–2.57; P <0.001 for psychosis and CVD mortality). Anxiety was associated with only CVD mortality in men, and depression was associated with only CVD events in women.

Conclusions:

Consistent with the hypothesis that chronic stress leads to greater CVD risk, multiple mental illnesses were associated with an increased risk of CVD outcomes, with more severe mental illnesses (eg, primary psychotic disorders) having the largest effect sizes even after controlling for other psychiatric diagnoses, conventional CVD risk factors, and psychotropic medication use.

Intensive Glucose Control in Patients with Type 2 Diabetes - 15-Year Follow-up

BACKGROUND

We previously reported that a median of 5.6 years of intensive as compared with standard glucose lowering in 1791 military veterans with type 2 diabetes resulted in a risk of major cardiovascular events that was significantly lower (by 17%) after a total of 10 years of combined intervention and observational follow-up. We now report the full 15-year follow-up.

METHODS

We observationally followed enrolled participants (complete cohort) after the conclusion of the original clinical trial by using central databases to identify cardiovascular events, hospitalizations, and deaths. Participants were asked whether they would be willing to provide additional data by means of surveys and chart reviews (survey cohort). The prespecified primary outcome was a composite of major cardiovascular events, including nonfatal myocardial infarction, nonfatal stroke, new or worsening congestive heart failure, amputation for ischemic gangrene, and death from cardiovascular causes. Death from any cause was a prespecified secondary outcome.

RESULTS

There were 1655 participants in the complete cohort and 1391 in the survey cohort. During the trial (which originally enrolled 1791 participants), the separation of the glycated hemoglobin curves between the intensive-therapy group (892 participants) and the standard-therapy group (899 participants) averaged 1.5 percentage points, and this difference declined to 0.2 to 0.3 percentage points by 3 years after the trial ended. Over a period of 15 years of follow-up (active treatment plus post-trial observation), the risks of major cardiovascular events or death were not lower in the intensive-therapy group than in the standard-therapy group (hazard ratio for primary outcome, 0.91; 95% confidence interval [CI], 0.78 to 1.06; P = 0.23; hazard ratio for death, 1.02; 95% CI, 0.88 to 1.18). The risk of major cardiovascular disease outcomes was reduced, however, during an extended interval of separation of the glycated hemoglobin curves (hazard ratio, 0.83; 95% CI, 0.70 to 0.99), but this benefit did not continue after equalization of the glycated hemoglobin levels (hazard ratio, 1.26; 95% CI, 0.90 to 1.75).

CONCLUSIONS

Participants with type 2 diabetes who had been randomly assigned to intensive glucose control for 5.6 years had a lower risk of cardiovascular events than those who received standard therapy only during the prolonged period in which the glycated hemoglobin curves were separated. There was no evidence of a legacy effect or a mortality benefit with intensive glucose control. (Funded by the VA Cooperative Studies Program; VADT ClinicalTrials.gov number, NCT00032487.).

Veterans Affairs patient database (VAPD 2014-2017): building nationwide granular data for clinical discovery

BACKGROUND

To study patient physiology throughout a period of acute hospitalization, we sought to create accessible, standardized nationwide data at the level of the individual patient-facility-day. This methodology paper summarizes the development, organization, and characteristics of the Veterans Affairs Patient Database 2014-2017 (VAPD 2014-2017). The VAPD 2014-2017 contains acute hospitalizations from all parts of the nationwide VA healthcare system with daily physiology including clinical data (labs, vitals, medications, risk scores, etc.), intensive care unit (ICU) indicators, facility, patient, and hospitalization characteristics.

METHODS

The VA data structure and database organization represents a complex multi-hospital system. We define a single-site hospitalization as one or more consecutive stays with an acute treating specialty at a single facility. The VAPD 2014-2017 is structured at the patient-facility-day level, where every patient-day in a hospital is a row with separate identification variables for facility, patient, and hospitalization. The VAPD 2014-2017 includes daily laboratory, vital signs, and inpatient medication. Such data were validated and verified through lab value range and comparison with patient charts. Sepsis, risk scores, and organ dysfunction definitions were standardized and calculated.

RESULTS

We identified 565,242 single-site hospitalizations (SSHs) in 2014; 558,060 SSHs in 2015; 553,961 SSHs in 2016; and 550,236 SSHs in 2017 at 141 VA hospitals. The average length of stay was four days for all study years. In-hospital mortality decreased from 2014 to 2017 (1.7 to 1.4%), 30-day readmission rates increased from 15.3% in 2014 to 15.6% in 2017; 30-day mortality also decreased from 4.4% in 2014 to 4.1% in 2017. From 2014 to 2017, there were 107,512 (4.8%) of SSHs that met the Center for Disease Control and Prevention's Electronic Health Record-based retrospective definition of sepsis.

CONCLUSION

The VAPD 2014-2017 represents a large, standardized collection of granular data from a heterogeneous nationwide healthcare system. It is also a direct resource for studying the evolution of inpatient physiology during both acute and critical illness.

Template matching for benchmarking hospital performance in the veterans affairs healthcare system

Comparing hospital performance in a health system is traditionally done with multilevel regression models that adjust for differences in hospitals' patient case-mix. In contrast, "template matching" compares outcomes of similar patients at different hospitals but has been used only in limited patient settings.Our objective was to test a basic template matching approach in the nationwide Veterans Affairs healthcare system (VA), compared with a more standard regression approach.We performed various simulations using observational data from VA electronic health records whereby we randomly assigned patients to "pseudo hospitals," eliminating true hospital level effects. We randomly selected a representative template of 240 patients and matched 240 patients on demographic and physiological factors from each pseudo hospital to the template. We varied hospital performance for different simulations such that some pseudo hospitals negatively impacted patient mortality.Electronic health record data of 460,213 hospitalizations at 111 VA hospitals across the United States in 2015.We assessed 30-day mortality at each pseudo hospital and identified lowest quintile hospitals by template matching and regression. The regression model adjusted for predicted 30-day mortality (as a measure of illness severity).Regression identified the lowest quintile hospitals with 100% accuracy compared with 80.3% to 82.0% for template matching when systematic differences in 30-day mortality existed.The current standard practice of risk-adjusted regression incorporating patient-level illness severity was better able to identify lower-performing hospitals than the simplistic template matching algorithm.

Variation in Laboratory Test Naming Conventions in EHRs Within and Between Hospitals: A Nationwide Longitudinal Study

BACKGROUND

Electronic health records provide clinically rich data for research and quality improvement work. However, the data are often unstructured text, may be inconsistently recorded and extracted into centralized databases, making them difficult to use for research.

OBJECTIVES

We sought to quantify the variation in how key laboratory measures are recorded in the Department of Veterans Affairs (VA) Corporate Data Warehouse (CDW) across hospitals and over time. We included 6 laboratory tests commonly drawn within the first 24 hours of hospital admission (albumin, bilirubin, creatinine, hemoglobin, sodium, white blood cell count) from fiscal years 2005-2015.

RESULTS

We assessed laboratory test capture for 5,454,411 acute hospital admissions at 121 sites across the VA. The mapping of standardized laboratory nomenclature (Logical Observation Identifiers Names and Codes, LOINCs) to test results in CDW varied within hospital by laboratory test. The relationship between LOINCs and laboratory test names improved over time; by FY2015, 109 (95.6%) hospitals had >90% of the 6 laboratory tests mapped to an appropriate LOINC. All fields used to classify test results are provided in an Appendix (Supplemental Digital Content 1, http://links.lww.com/MLR/B635).

CONCLUSIONS

The use of electronic health record data for research requires assessing data consistency and quality. Using laboratory test results requires the use of both unstructured text fields and the identification of appropriate LOINCs. When using data from multiple facilities, the results should be carefully examined by facility and over time to maximize the capture of data fields.

Effects of Severe Hypoglycemia on Cardiovascular Outcomes and Death in the Veterans Affairs Diabetes Trial

OBJECTIVE

To determine the risk factors for severe hypoglycemia and the association between severe hypoglycemia and serious cardiovascular adverse events and cardiovascular and all-cause mortality in the Veterans Affairs Diabetes Trial (VADT).

RESEARCH DESIGN AND METHODS

This post hoc analysis of data from the VADT included 1,791 military veterans (age 60.5 ± 9.0 years) with suboptimally controlled type 2 diabetes (HbA1c 9.4 ± 2.0%) of 11.5 ± 7.5 years disease duration with or without known cardiovascular disease and additional cardiovascular risk factors. Participants were randomized to intensive (HbA1c <7.0%) versus standard (HbA1c <8.5%) glucose control.

RESULTS

The rate of severe hypoglycemia in the intensive treatment group was 10.3 per 100 patient-years compared with 3.7 per 100 patient-years in the standard treatment group (P < 0.001). In multivariable analysis, insulin use at baseline (P = 0.02), proteinuria (P = 0.009), and autonomic neuropathy (P = 0.01) were independent risk factors for severe hypoglycemia, and higher BMI was protective (P = 0.017). Severe hypoglycemia within the past 3 months was associated with an increased risk of serious cardiovascular events (P = 0.032), cardiovascular mortality (P = 0.012), and total mortality (P = 0.024). However, there was a relatively greater increased risk for total mortality in the standard group compared with the intensive group (P = 0.019). The association between severe hypoglycemia and cardiovascular events increased significantly as overall cardiovascular risk increased (P = 0.012).

CONCLUSIONS

Severe hypoglycemic episodes within the previous 3 months were associated with increased risk for major cardiovascular events and cardiovascular and all-cause mortality regardless of glycemic treatment group assignment. Standard therapy further increased the risk for all-cause mortality after severe hypoglycemia.

Abstract 102: Impact of Using Older Data on the Accuracy of Cardiovascular Risk Scores

Background: Recent scores to predict atherosclerotic cardiovascular disease, ASCVD, have been found inaccurate, with some concern that risk scores become inaccurate with time, as changing demographics may also change ASCVD risk. A partial solution to the timeliness problem could be creating risk scores using electronic health records (EHR). EHR-based risk scores can be easily updated and tested for changes over time. They can also use more variables than traditional risk scores. This could capture clinical change that would otherwise be missed.

We hypothesized that ASCVD risk scores change over time, but this could be minimized with more robust risk scores. To test this, we looked at change of two EHR-based risk score over three follow-up periods and using different statistical techniques to design the risk scores.

Methods: Data sources: VA national data linked to Medicare and the National Death Index.

Population: 3 overlapping cohorts from 2002, 2006, and 2009. Each consisted of all active VA patients aged 45-80 who had no documented history of CVD, clinical heart failure or loop diuretic use at baseline.

Prediction models

1. VARS-ASCVD: uses the same variables as traditional risk scores, but all variables were re-calibrated to our population.

2. VARS-EHR: Uses 41 predictor variables and more interaction effects.

Outcome variables: First occurrence of fatal or nonfatal ASCVD during 5 years of follow-up.

Analysis: We looked at the accuracy of risk scores developed in 2002 on patients in 2006 and 2009. The discrimination of the risk scores (the ability to distinguish between those who do and do not develop an event), was evaluated with C-statistisic. The calibration (how closely the predicted probabilities reflect true risk) was evaluated with the Hosmer-Lemeshow Goodness of Fit statistic (GoF).

Results: Each cohort had at least 1.4 million participants. Between the 3 cohorts the rate of diabetes mellitus increased from 201% to 27% and statin use increased from 25% to 45% of the population.

The VARS-ASCVD risk scores for men developed in 2002 had the same discrimination of 0.67 in 2006 and 2009, but in women fell from 0.77 to 0.72 then increased to 0.74. The goodness of fit worsened. Using the VARS-EHR model, discrimination stayed similar in men and women. The GOF worsened, but by substantially less.

Conclusions: ASCVD risk prediction tools become poorly calibrated over fairly short time periods. For effective use, they must be updated regularly.

Predicting Hospitalization and Outpatient Corticosteroid Use in Inflammatory Bowel Disease Patients Using Machine Learning

BACKGROUND

Inflammatory bowel disease (IBD) is a chronic disease characterized by unpredictable episodes of flares and periods of remission. Tools that accurately predict disease course would substantially aid therapeutic decision-making. This study aims to construct a model that accurately predicts the combined end point of outpatient corticosteroid use and hospitalizations as a surrogate for IBD flare.

METHODS

Predictors evaluated included age, sex, race, use of corticosteroid-sparing immunosuppressive medications (immunomodulators and/or anti-TNF), longitudinal laboratory data, and number of previous IBD-related hospitalizations and outpatient corticosteroid prescriptions. We constructed models using logistic regression and machine learning methods (random forest [RF]) to predict the combined end point of hospitalization and/or corticosteroid use for IBD within 6 months.

RESULTS

We identified 20,368 Veterans Health Administration patients with the first (index) IBD diagnosis between 2002 and 2009. Area under the receiver operating characteristic curve (AuROC) for the baseline logistic regression model was 0.68 (95% confidence interval [CI], 0.67-0.68). AuROC for the RF longitudinal model was 0.85 (95% CI, 0.84-0.85). AuROC for the RF longitudinal model using previous hospitalization or steroid use was 0.87 (95% CI, 0.87-0.88). The 5 leading independent risk factors for future hospitalization or steroid use were age, mean serum albumin, immunosuppressive medication use, and mean and highest platelet counts. Previous hospitalization and corticosteroid use were highly predictive when included in specified models.

CONCLUSIONS

A novel machine learning model substantially improved our ability to predict IBD-related hospitalization and outpatient steroid use. This model could be used at point of care to distinguish patients at high and low risk for disease flare, allowing individualized therapeutic management.

Corrected ROC analysis for misclassified binary outcomes

Creating accurate risk prediction models from Big Data resources such as Electronic Health Records (EHRs) is a critical step toward achieving precision medicine. A major challenge in developing these tools is accounting for imperfect aspects of EHR data, particularly the potential for misclassified outcomes. Misclassification, the swapping of case and control outcome labels, is well known to bias effect size estimates for regression prediction models. In this paper, we study the effect of misclassification on accuracy assessment for risk prediction models and find that it leads to bias in the area under the curve (AUC) metric from standard ROC analysis. The extent of the bias is determined by the false positive and false negative misclassification rates as well as disease prevalence. Notably, we show that simply correcting for misclassification while building the prediction model is not sufficient to remove the bias in AUC. We therefore introduce an intuitive misclassification-adjusted ROC procedure that accounts for uncertainty in observed outcomes and produces bias-corrected estimates of the true AUC. The method requires that misclassification rates are either known or can be estimated, quantities typically required for the modeling step. The computational simplicity of our method is a key advantage, making it ideal for efficiently comparing multiple prediction models on very large datasets. Finally, we apply the correction method to a hospitalization prediction model from a cohort of over 1 million patients from the Veterans Health Administrations EHR. Implementations of the ROC correction are provided for Stata and R. Published 2017. This article is a U.S. Government work and is in the public domain in the USA.

Social Risks for At-Risk Drinking in Young Workers: Application of Work-Life Border Theory

The current study evaluated work-life risks uniquely associated with at-risk drinking for younger (aged 18 to 30) versus two samples of older workers (31 to 40, and 41 or older). Measures were selected according to theories of alcohol culture (e.g., drinking norms at work) and work-life conflict. Following “work-life border” theory (Clark, 2000), an exploratory model examined relationships of these measures with at-risk drinking (ARD) and job-related hangovers (JRH) across the three age groups within a large municipality (n=587) and a sample of small businesses (n=736). Survey results showed life-to-work conflict uniquely predicted ARD for younger workers. In small businesses, younger workers reporting JRH perceived the most permissive drinking norms. Findings suggest risks differ between the small business and municipal samples, and the importance of distinguishing ARD and JRH when assessing outcomes. Results are interpreted with border theory, and discussion focuses on suggestions for prevention programming for young workers.

Age Disparities in the Use of Steroid-sparing Therapy for Inflammatory Bowel Disease

BACKGROUND

Corticosteroids are effective rescue therapies for patients with inflammatory bowel disease (IBD), but have significant side effects, which may be amplified in the growing population of elderly patients with IBD. We aimed to compare the use of steroids and steroid-sparing therapies (immunomodulators and biologics) and rates of complications among elderly (≥65) and younger patients in a national cohort of veterans with IBD.

METHODS

We used national Veterans Health Administrative data to conduct a retrospective study of veterans with IBD between 2002 and 2010. Medications and the incidence of complications were obtained from the Veterans Health Administrative Decision Support Systems. Multivariate logistic regression accounting for facility-level clustering was used to identify predictors of use of steroid-sparing medications.

RESULTS

We identified 30,456 veterans with IBD. Of these, 94% were men and 40% were more than 65, and 32% were given steroids. Elderly veterans were less likely to receive steroids (23.8% versus 38.3%, P < 0.001) and were less likely to be prescribed steroid-sparing medications (25.5% versus 46.9%, respectively, P < 0.001). In multivariate analysis controlling for sex, age <65 (odds ratio, 2.19; 95% CI, 1.54-3.11) and gastroenterology care (odds ratio, 8.42; 95% CI, 6.18-11.47) were associated with initiation of steroid-sparing medications. After starting steroids, fracture rates increased in the elderly patients with IBD, whereas increases in venous thromboembolism and infections after starting steroids affected both age groups.

CONCLUSIONS

Elderly veterans are less likely to receive steroids and steroid-sparing medications than younger veterans; elderly patients exposed to steroids were more likely to have fractures than the younger population.

Corticosteroid Use and Complications in a US Inflammatory Bowel Disease Cohort

Background and Aims

Corticosteroids are effective for the short-term treatment of inflammatory bowel disease (IBD). Long-term use, however, is associated with significant adverse effects. To define the: (1) frequency and duration of corticosteroid use, (2) frequency of escalation to corticosteroid-sparing therapy, (3) rate of complications related to corticosteroid use, (4) rate of appropriate bone density measurements (dual energy X-ray absorptiometry [DEXA] scans), and (5) factors associated with escalation and DEXA scans.

Methods

Retrospective review of Veterans Health Administration (VHA) data from 2002–2010.

Results

Of the 30,456 Veterans with IBD, 32% required at least one course of corticosteroids during the study time period, and 17% of the steroid users had a prolonged course. Among these patients, only 26.2% underwent escalation of therapy. Patients visiting a gastroenterology (GI) physician were significantly more likely to receive corticosteroid-sparing medications. Factors associated with corticosteroid-sparing medications included younger age (OR = 0.96 per year,95%CI:0.95, 0.97), male gender (OR = 2.00,95%CI:1.16,3.46), GI visit during the corticosteroid evaluation period (OR = 8.01,95%CI:5.85,10.95) and the use of continuous corticosteroids vs. intermittent corticosteroids (OR = 2.28,95%CI:1.33,3.90). Rates of complications per 1000 person-years after IBD diagnosis were higher among corticosteroid users (venous thromboembolism [VTE] 9.0%; fragility fracture 2.6%; Infections 54.3) than non-corticosteroid users (VTE 4.9%; fragility fracture 1.9%; Infections 26.9). DEXA scan utilization rates among corticosteroid users were only 7.8%.

Conclusions

Prolonged corticosteroid therapy for the treatment of IBD is common and is associated with significant harm to patients. Patients with prolonged use of corticosteroids for IBD should be referred to gastroenterology early and universal efforts to improve the delivery of high quality care should be undertaken.

Follow-up of glycemic control and cardiovascular outcomes in type 2 diabetes

BACKGROUND

The Veterans Affairs Diabetes Trial previously showed that intensive glucose lowering, as compared with standard therapy, did not significantly reduce the rate of major cardiovascular events among 1791 military veterans (median follow-up, 5.6 years). We report the extended follow-up of the study participants.

METHODS

After the conclusion of the clinical trial, we followed participants, using central databases to identify procedures, hospitalizations, and deaths (complete cohort, with follow-up data for 92.4% of participants). Most participants agreed to additional data collection by means of annual surveys and periodic chart reviews (survey cohort, with 77.7% follow-up). The primary outcome was the time to the first major cardiovascular event (heart attack, stroke, new or worsening congestive heart failure, amputation for ischemic gangrene, or cardiovascular-related death). Secondary outcomes were cardiovascular mortality and all-cause mortality.

RESULTS

The difference in glycated hemoglobin levels between the intensive-therapy group and the standard-therapy group averaged 1.5 percentage points during the trial (median level, 6.9% vs. 8.4%) and declined to 0.2 to 0.3 percentage points by 3 years after the trial ended. Over a median follow-up of 9.8 years, the intensive-therapy group had a significantly lower risk of the primary outcome than did the standard-therapy group (hazard ratio, 0.83; 95% confidence interval [CI], 0.70 to 0.99; P=0.04), with an absolute reduction in risk of 8.6 major cardiovascular events per 1000 person-years, but did not have reduced cardiovascular mortality (hazard ratio, 0.88; 95% CI, 0.64 to 1.20; P=0.42). No reduction in total mortality was evident (hazard ratio in the intensive-therapy group, 1.05; 95% CI, 0.89 to 1.25; P=0.54; median follow-up, 11.8 years).

CONCLUSIONS

After nearly 10 years of follow-up, patients with type 2 diabetes who had been randomly assigned to intensive glucose control for 5.6 years had 8.6 fewer major cardiovascular events per 1000 person-years than those assigned to standard therapy, but no improvement was seen in the rate of overall survival. (Funded by the VA Cooperative Studies Program and others; VADT ClinicalTrials.gov number, NCT00032487.).

Collaborative hypertension case management by registered nurses and clinical pharmacy specialists within the Patient Aligned Care Teams (PACT) model

BACKGROUND

Clinical Pharmacy Specialists (CPSs) and Registered Nurses (RNs) are integrally involved in the Patient Aligned Care Teams (PACT) model, especially as physician extenders in the management of chronic disease states. CPSs may be an alternative to physicians as a supporting prescriber for RN case management (RNCM) of poorly controlled hypertension.

OBJECTIVE

To compare CPS-directed versus physician-directed RNCM for patients with poorly controlled hypertension.

DESIGN

Non-randomized, retrospective comparison of a natural experiment.

SETTING

A large Midwestern Veterans Affairs (VA) medical center.

INTERVENTION

Utilizing CPSs as alternatives to physicians for directing RNCM of poorly controlled hypertension.

PATIENTS

All 126 patients attended RNCM appointments for poorly controlled hypertension between 20 September 2011 and 31 October 2011 with either CPS or physician involvement in the clinical decision making. Patients were excluded if both a CPS and a physician were involved in the index visit, or they were enrolled in Home Based Primary Care, or if they displayed non-adherence to the plan.

MAIN MEASURES

All data were obtained from review of electronic medical records. Outcomes included whether a patient received medication intensification at the index visit, and as the main measure, blood pressures between the index and next consecutive visit.

KEY RESULTS

All patients had medication intensification. Patients receiving CPS-directed RNCM had greater decreases in systolic blood pressure compared to those receiving physician-directed RNCM (14 ± 13 mmHg versus 10 ± 11 mmHg; p = 0.04). After adjusting for the time between visits, initial systolic blood pressure, and prior stroke, provider type was no longer significant (p = 0.24). Change in diastolic blood pressure and attainment of blood pressure < 140/90 mm Hg were similar between groups (p = 0.93, p = 0.91, respectively).

CONCLUSIONS

CPS-directed and physician-directed RNCM for hypertension demonstrated similar blood pressure reduction. These results support the utilization of CPSs as prescribers to support RNCM for chronic diseases.

Improved cardiovascular risk prediction using nonparametric regression and electronic health record data

BACKGROUND

Use of the electronic health record (EHR) is expected to increase rapidly in the near future, yet little research exists on whether analyzing internal EHR data using flexible, adaptive statistical methods could improve clinical risk prediction. Extensive implementation of EHR in the Veterans Health Administration provides an opportunity for exploration.

OBJECTIVES

To compare the performance of various approaches for predicting risk of cerebrovascular and cardiovascular (CCV) death, using traditional risk predictors versus more comprehensive EHR data.

RESEARCH DESIGN

Retrospective cohort study. We identified all Veterans Health Administration patients without recent CCV events treated at 12 facilities from 2003 to 2007, and predicted risk using the Framingham risk score, logistic regression, generalized additive modeling, and gradient tree boosting.

MEASURES

The outcome was CCV-related death within 5 years. We assessed each method's predictive performance with the area under the receiver operating characteristic curve (AUC), the Hosmer-Lemeshow goodness-of-fit test, plots of estimated risk, and reclassification tables, using cross-validation to penalize overfitting.

RESULTS

Regression methods outperformed the Framingham risk score, even with the same predictors (AUC increased from 71% to 73% and calibration also improved). Even better performance was attained in models using additional EHR-derived predictor variables (AUC increased to 78% and net reclassification improvement was as large as 0.29). Nonparametric regression further improved calibration and discrimination compared with logistic regression.

CONCLUSIONS

Despite the EHR lacking some risk factors and its imperfect data quality, health care systems may be able to substantially improve risk prediction for their patients by using internally developed EHR-derived models and flexible statistical methodology.

Medicaid coverage and access to publicly funded opiate treatment

This observational study examines changes in access to methadone maintenance treatment following Oregon's decision to remove substance abuse treatment from the Medicaid benefit for an expansion population. Access was compared before and after the benefit change for two cohorts of adults addicted to opiates presenting for publicly funded treatment. Propensity score analysis helped model some selective disenrollment from Medicaid that occurred after the benefit change. Logistic regression was used to compare access to methadone by cohort controlling for client characteristics. Opiate users presenting for publicly funded treatment after the change were less than half as likely (OR = 0.40) to be placed in an opiate treatment program compared to the prior year. Further analysis revealed that those with no recent treatment history were less likely to present for treatment after the benefit change. These results have implications for states considering Medicaid cuts, especially if the anticipated increases in illegal activity, emergency room utilization, unemployment, and mortality can be demonstrated.

Team awareness, problem drinking, and drinking climate: workplace social health promotion in a policy context

PURPOSE

(1) To determine the effectiveness of classroom health promotion/prevention training designed to improve work climate and alcohol outcomes; (2) to assess whether such training contributes to improvements in problem drinking beyond standard workplace alcohol policies.

DESIGN

A cross-sectional survey assessed employee problem drinking across three time periods. This was followed by a prevention intervention study; work groups were randomly assigned to an 8-hour training course in workplace social health promotion (Team Awareness), a 4-hour informational training course, or a control group. Surveys were administered 2 to 4 weeks before and after training and 6 months after posttest.

SETTING AND SUBJECTS

Employees were surveyed from work departments in a large municipality of 3000 workers at three points in time (year, sample, and response rates are shown): (1) 1992, n = 1081, 95%; (2) 1995, n = 856, 97%; and (3) 1999, n = 587, 73%. Employees in the 1999 survey were recruited from safety-sensitive departments and were randomly assigned to receive the psychosocial (n = 201), informational (n = 192), or control (n = 194) condition.

INTERVENTION

The psychosocial program (Team Awareness) provided skills training in peer referral, team building, and stress management. Informational training used a didactic review of policy, employee assistance, and drug testing.

MEASURES

Self-reports measured alcohol use (frequency, drunkenness, hangovers, and problems) and work drinking climate (enabling, responsiveness, drinking norms, stigma, and drink with co-workers).

RESULTS

Employees receiving Team Awareness reduced problem drinking from 20% to 11% and working with or missing work because of a hangover from 16% to 6%. Information-trained workers also reduced problem drinking from 18% to 10%. These rates of change contrast with changes in problem drinking seen from 1992 (24%) to 1999 (17%). Team Awareness improvements differed significantly from control subjects, which showed no change at 13%. Employees receiving Team Awareness also showed significant improvements in drinking climate. For example, scores on the measure of coworker enabling decreased from pretest (mean = 2.19) to posttest (mean = 2.05) and follow up (mean = 1.94). Posttest measures of drinking climate also predicted alcohol outcomes at 6 months.

CONCLUSION

Employers should consider the use of prevention programming as an enhancement to standard drug-free workplace efforts. Team Awareness training targets work group social health, aligns with employee assistance efforts, and contributes to reductions in problem drinking.

Meta-Analysis of Raw Mean Differences

This article discusses the meta-analysis of raw mean differences. It presents a rationale for cumulating psychological effects in a raw metric and compares raw mean differences to standardized mean differences. Some limitations of standardization are noted, and statistical techniques for raw meta-analysis are described. These include a graphical device for decomposing effect sizes. Several illustrative data sets are analyzed.