Travel distance and outcomes after surgical aortic valve among veterans

OBJECTIVE

To investigate the association between travel distance and postoperative length of stay (LOS) and discharge disposition among veterans undergoing surgical aortic valve replacement (SAVR).

DATA SOURCES/STUDY SETTING

We performed a retrospective cohort study of patients undergoing SAVR, with or without coronary artery bypass grafting (CABG) at VA Boston Healthcare (January 1, 2005-December 31, 2015).

STUDY DESIGN

Postoperative LOS and discharge disposition were compared for SAVR patients based on travel distance to the facility: <100 miles or ≥100 miles. Multivariable regression was performed to ascertain factors associated with LOS and home discharge.

DATA COLLECTION/EXTRACTION METHODS

Data were collected via chart review. All patients undergoing SAVR at our institution who primarily resided within the defined region were included.

PRINCIPAL FINDINGS

Of 597 patients studied, 327 patients underwent isolated SAVR; 270 patients underwent SAVR/CABG. Overall median (IQR) distance between the patient's residence and the hospital was 49.95 miles (27.41-129.94 miles); 190 patients (32%) resided further than 100 miles away. There were no differences in the proportion of patients with diabetes, hypertension, chronic obstructive pulmonary disease (COPD), cerebrovascular disease, atrial fibrillation, or prior myocardial infarction between groups. Overall LOS (IQR) was 9 (7-13) days and did not differ between groups (p = 0.18). The proportion of patients discharged home was higher among patients who resided more than 100 miles from the hospital (71% vs. 58%, p = 0.01). On multivariable analysis, residing further than 100 miles from the hospital was independently associated with home discharge (OR = 1.64, 95% CI: 1.09-2.48). Travel distance was not associated with LOS.

CONCLUSIONS

Based on our institutional experience, potential concerns of longer hospital stay or discharge to other inpatient facilities for geographically distanced patients undergoing SAVR do not appear supported. Continued examination of the drivers underlying the marked shift of veterans to the private sector appears warranted.

Development and Validation of an Electronic Adverse Event Model for Patient Safety Surveillance in Interventional Radiology

BACKGROUND

Comprehensive adverse event (AE) surveillance programs in interventional radiology (IR) are rare. Our aim was to develop and validate a retrospective electronic surveillance model to identify outpatient IR procedures that are likely to have an AE, to support patient safety and quality improvement.

METHODS

We identified outpatient IR procedures performed in the period from October 2017 to September 2019 from the Veterans Health Administration (n = 135,283) and applied electronic triggers based on posyprocedure care to flag cases with a potential AE. From the trigger-flagged cases, we randomly sampled n = 1,500 for chart review to identify AEs. We also randomly sampled n = 600 from the unflagged cases. Chart-reviewed cases were merged with patient, procedure, and facility factors to estimate a mixed-effects logistic regression model designed to predict whether an AE occurred. Using model fit and criterion validity, we determined the best predicted probability threshold to identify cases with a likely AE. We reviewed a random sample of 200 cases above the threshold and 100 cases from below the threshold from October 2019 to March 2020 (n = 20,849) for model validation.

RESULTS

In our development sample of mostly trigger-flagged cases, 444 of 2,096 cases (21.8%) had an AE. The optimal predicted probability threshold for a likely AE from our surveillance model was >50%, with positive predictive value of 68.9%, sensitivity of 38.3%, and specificity of 95.3%. In validation, chart-reviewed cases with AE probability >50% had a positive predictive value of 63% (n = 203). For the period from October 2017 to March 2020, the model identified approximately 70 IR cases per month that were likely to have an AE.

CONCLUSIONS

This electronic trigger-based approach to AE surveillance could be used for patient-safety reporting and quality review.

Factors associated with uptake of guideline-recommended cardiovascular implantable electronic device management: a nationwide, retrospective cohort study

Clinical guidelines recommend device removal for cardiovascular implantable electronic device (CIED) infection management. In this retrospective, nationwide cohort, 60.8% of CIED infections received guideline-concordant care. One-year mortality was higher among those without procedural management (25% vs 16%). Factors associated with receipt of device procedures included pocket infections and positive microbiology.

Does screening for PTSD lead to VA mental health care? Identifying the spectrum of initial VA screening actions

Despite the active posttraumatic stress disorder (PTSD) screening program in Department of Veterans Affairs (VA) primary care clinics and the availability of empirically supported treatments for PTSD at VA, many veterans for whom screening suggests treatment may be indicated do not gain access to VA-based mental health care. To determine where we may be losing veterans to follow-up, we need to begin by identifying the initial action taken in response to a positive PTSD screen in primary care. Using VA administrative data and chart review, we identified the spectrum of initial actions taken after veterans screened positive for PTSD in VA primary care clinics nationwide between October 2017 and September 2018 (N = 41,570). We collapsed actions into those that could lead to VA-based mental health care (e.g., consult placed to a VA mental health clinic) versus not (e.g., veteran declined care), and then examined the association between these categories of actions and contextual- and individual-level variables. More than 61% of veterans with positive PTSD screens had evidence that an initial action toward VA-based mental health care was taken. Urban-dwelling and female veterans were significantly more likely to have evidence of these initial actions, whereas White and Vietnam-era veterans were significantly less likely to have this evidence. Our findings suggest that most veterans screening positive for PTSD in VA primary care clinics have evidence of initial actions taken toward VA-based mental health care; however, a substantial minority do not, making them unlikely to receive follow-up care. Findings highlight the potential benefit of targeted primary care-based access interventions. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

A Novel Informatics Tool to Detect Periprocedural Antibiotic Allergy Adverse Events for Near Real-time Surveillance to Support Audit and Feedback

Importance

Standardized processes for identifying when allergic-type reactions occur and linking reactions to drug exposures are limited.

Objective

To develop an informatics tool to improve detection of antibiotic allergic-type events.

Design, Setting, and Participants

This retrospective cohort study was conducted from October 1, 2015, to September 30, 2019, with data analyzed between July 1, 2021, and January 31, 2022. The study was conducted across Veteran Affairs hospitals among patients who underwent cardiovascular implantable electronic device (CIED) procedures and received periprocedural antibiotic prophylaxis. The cohort was split into training and test cohorts, and cases were manually reviewed to determine presence of allergic-type reaction and its severity. Variables potentially indicative of allergic-type reactions were selected a priori and included allergies entered in the Veteran Affair's Allergy Reaction Tracking (ART) system (either historical [reported] or observed), allergy diagnosis codes, medications administered to treat allergic reactions, and text searches of clinical notes for keywords and phrases indicative of a potential allergic-type reaction. A model to detect allergic-type reaction events was iteratively developed on the training cohort and then applied to the test cohort. Algorithm test characteristics were assessed.

Exposure

Preprocedural and postprocedural prophylactic antibiotic administration.

Main Outcomes and Measures

Antibiotic allergic-type reactions.

Results

The cohort of 36 344 patients included 34 703 CIED procedures with antibiotic exposures (mean [SD] age, 72 [10] years; 34 008 [98%] male patients); median duration of postprocedural prophylaxis was 4 days (IQR, 2-7 days; maximum, 45 days). The final algorithm included 7 variables: entries in the Veteran Affair's hospitals ART, either historic (odds ratio [OR], 42.37; 95% CI, 11.33-158.43) or observed (OR, 175.10; 95% CI, 44.84-683.76); PheCodes for "symptoms affecting skin" (OR, 8.49; 95% CI, 1.90-37.82), "urticaria" (OR, 7.01; 95% CI, 1.76-27.89), and "allergy or adverse event to an antibiotic" (OR, 11.84, 95% CI, 2.88-48.69); keyword detection in clinical notes (OR, 3.21; 95% CI, 1.27-8.08); and antihistamine administration alone or in combination (OR, 6.51; 95% CI, 1.90-22.30). In the final model, antibiotic allergic-type reactions were identified with an estimated probability of 30% or more; positive predictive value was 61% (95% CI, 45%-76%); and sensitivity was 87% (95% CI, 70%-96%).

Conclusions and Relevance

In this retrospective cohort study of patients receiving periprocedural antibiotic prophylaxis, an algorithm with a high sensitivity to detect incident antibiotic allergic-type reactions that can be used to provide clinician feedback about antibiotic harms from unnecessarily prolonged antibiotic exposures was developed.

Development of a Periprocedure Trigger for Outpatient Interventional Radiology Procedures in the Veterans Health Administration

OBJECTIVES

Interventional radiology (IR) is the newest medical specialty. However, it lacks robust quality assurance metrics, including adverse event (AE) surveillance tools. Considering the high frequency of outpatient care provided by IR, automated electronic triggers offer a potential catalyst to support accurate retrospective AE detection.

METHODS

We programmed previously validated AE triggers (admission, emergency visit, or death up to 14 days after procedure) for elective, outpatient IR procedures performed in Veterans Health Administration surgical facilities between fiscal years 2017 and 2019. We then developed a text-based algorithm to detect AEs that explicitly occurred in the periprocedure time frame: before, during, and shortly after the IR procedure. Guided by the literature and clinical expertise, we generated clinical note keywords and text strings to flag cases with high potential for periprocedure AEs. Flagged cases underwent targeted chart review to measure criterion validity (i.e., the positive predictive value), to confirm AE occurrence, and to characterize the event.

RESULTS

Among 135,285 elective outpatient IR procedures, the periprocedure algorithm flagged 245 cases (0.18%); 138 of these had ≥1 AE, yielding a positive predictive value of 56% (95% confidence interval, 50%-62%). The previously developed triggers for admission, emergency visit, or death in 14 days flagged 119 of the 138 procedures with AEs (73%). Among the 43 AEs detected exclusively by the periprocedure trigger were allergic reactions, adverse drug events, ischemic events, bleeding events requiring blood transfusions, and cardiac arrest requiring cardiopulmonary resuscitation.

CONCLUSIONS

The periprocedure trigger performed well on IR outpatient procedures and offers a complement to other electronic triggers developed for outpatient AE surveillance.

Arteriovenous Graft Failure in the Veterans Health Administration: Outcome Disparities Associated with Race

Background Vascular access for ongoing hemodialysis often fails, frequently requiring repeated procedures to maintain vascular patency. While research has shown racial discrepancies in multiple aspects of renal failure treatment, there is poor understanding of how these factors might relate to vascular access maintenance procedures after arteriovenous graft (AVG) placement. Purpose To evaluate racial disparities associated with premature vascular access failure after percutaneous access maintenance procedures following AVG placement using a retrospective national cohort from the Veterans Health Administration (VHA). Materials and Methods All hemodialysis vascular maintenance procedures performed at VHA hospitals between October 2016 and March 2020 were identified. To ensure the sample represented patients who consistently used the VHA, patients without AVG placement within 5 years of their first maintenance procedure were excluded. Access failure was defined as a repeat access maintenance procedure or as hemodialysis catheter placement occurring 1-30 days after the index procedure. Multivariable logistic regression analyses were performed to calculate prevalence ratios (PRs) measuring the association between hemodialysis maintenance failure and African American race compared with all other races. Models controlled for vascular access history, patient socioeconomic status, and procedure and facility characteristics. Results In total, 1950 access maintenance procedures in 995 patients (mean age, 69 years ± 9 [SD], 1870 men) with an AVG created in one of 61 VHA facilities were identified. Most procedures involved African American patients (1169 of 1950, 60%) and patients residing in the South (1002 of 1950, 51%). Premature access failure occurred in 215 of 1950 (11%) procedures. When compared with all other races, African American race was associated with premature access site failure (PR, 1.4; 95% CI: 1.07, 1.43; P = .02). Among the 1057 procedures in 30 facilities with interventional radiology resident training programs, there was no evidence of racial disparity in the outcome (PR, 1.1; P = .63). Conclusion African American race was associated with higher risk-adjusted rates of premature arteriovenous graft failure after dialysis maintenance. © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Forman and Davis in this issue.

Prevalence of One-Year Mortality after Implantable Cardioverter Defibrillator Placement: An Opportunity for Palliative Care?

Background: Current guidelines recommend against placement of implantable cardioverter defibrillators in patients with a life expectancy less than one year. These patients may benefit from early palliative care services; however, identifying this population is challenging. Objective: Determine whether a validated prognostic tool, based on patient factors and health care utilization from electronic medical records, accurately predicts one-year mortality at the time of implantable cardioverter defibrillator placement. Design: We used the United States (U.S.) Veterans Administration's "Care Assessment Needs" one-Year Mortality Score to identify patients at high risk of mortality (score ≥95) before their procedure. Data were extracted from the Corporate Data Warehouse. Logistic regression was used to assess the odds of mortality at different score levels. Setting/Subjects: Patients undergoing a new implantable cardioverter defibrillator procedure between October 1, 2015 and September 30, 2017 in the U.S. Veterans Administration. Results: Of 3194 patients with a new implantable cardioverter defibrillator placed, 657 (21.8%) had a score ≥95. The mortality rate among these patients was 151/657 (22.9%) compared with 281/3194 (8.8%) for all patients undergoing a new implantable cardioverter defibrillator procedure. Patients with a score ≥95 had 14.0 (95% confidence interval 8.0-24.4) higher odds of death within one year of the procedure compared with those with a score ≤60. Conclusions: The "Care Assessment Needs" Score is a valid predictor of one-year mortality following implantable cardioverter defibrillator procedures. Integrating its use into the management of Veterans Administration (VA) patients considering implantable cardioverter defibrillators may improve shared decision making and engagement with palliative care.

994. A Novel Informatics Tool to Detect Antibiotic Allergies in Patients Undergoing CIED Procedures

Background

Antibiotics are one of the leading causes of emergency room visits for adverse drug events, yet surveillance for antimicrobial allergy adverse events is limited and identifying true cases is challenging. As part of a larger study to improve antimicrobial use, we sought to develop and validate a tool for near real-time measurement of antimicrobial allergy adverse events.

Methods

An existing cohort of patients undergoing cardiac device procedures with known antimicrobial exposure was split into a development and validation set. Candidate triggers for identifying allergic reactionswere identified a priori, using disease phenotype codes “phecodes”, allergy documentation on allergy module of the electronic medical record (EMR), and keyword searches applied to clinical notes (e.g., “anaphylaxis,” “rash”), medication administration (e.g, corticosteroids alone or with antihistamines) and administrative codes (ICD-10 codes and phecodes). Cases were reviewed for presence of a true event, and the tool was iteratively updated based on chart review findings. The tool was then applied to the validation cohort and a sample of trigger-flagged and unflagged cases underwent manual review. Data were analyzed in SAS and model triggers were selected using a LASSO technique.

Results

Among 34,703 patients, N=431 cases underwent manual review (350 development; 120 validation), and 104 true allergy adverse events were identified. Among chart reviewed cases, the most frequently detected flags were keywords in unstructured clinical notes (35%), phecodes (26%), corticosteroid administration (15%), observed allergy documentation in EMR (14%) and reported allergy documentation in EMR (13%). The final model contained 7 triggers and had an AUC of 0.95, and a positive predictive value of 67% (Figure). The strongest predictors of true adverse events were the allergy health factors (aOR 358, 95% CI 76.3-999) and specific Phecodes (Table1).

Conclusion

We developed an antibiotic allergy measurement tool using structured and unstructured data that can be applied to detect antimicrobial adverse events in near-real time. This model may be applied to provide near real-time feedback to clinicians about antimicrobial allergy adverse events and may be useful for antimicrobial stewardship programs.

Disclosures

Westyn Branch-Elliman, MD, MMSc, DLA Piper,LLC/Medtronic: Advisor/Consultant|Gilead Pharmaceuticals: Grant/Research Support.

What is the role of selection bias in quality comparisons between the Veterans Health Administration and community care? Example of elective hernia surgery

OBJECTIVE

To investigate the relationship between community care (CC) treatment and a postoperative surgical complication in elective hernia surgery among Veterans using multiple approaches to control for potential selection bias.

DATA SOURCES AND STUDY SETTING

Veterans Health Administration (VHA) data sources included Corporate Data Warehouse (VHA encounters and patient data), the Program Integrity Tool and Fee tables (CC encounters), the Planning Systems Support Group (geographic information), and the Paid file (VHA primary care providers).

STUDY DESIGN

Prior works suggest patient outcomes are better in VHA than in CC settings; however, these studies may not have appropriately accounted for the selection of higher-risk cases into CC. We estimated (1) a naïve logistic regression model to calculate the effect of CC setting on the probability of a complication, controlling for facility fixed effects and patient and procedure characteristics, and (2) a 2-stage model using the hernia patient's primary care provider's 1-year prior CC referral rate as the instrument.

DATA COLLECTION

We identified patients residing ≤40 miles from a VHA surgical facility with elective VHA or CC hernia surgery from 2018 to 2019.

PRINCIPAL FINDINGS

Of 7991 hernia surgeries, 772 (9.7%) were in CC. The overall complication rate was 4.2%; 286/7219 (4.0%) among VHA surgeries versus 51/5772 (6.6%, p < 0.05) in CC. We observed a 2.8 percentage point increase in the probability of postoperative complication given CC surgery (95% confidence interval: 0.7, 4.8) in the naïve model. After accounting for the VHA provider's historical rate of CC referral, we no longer observed a relationship between surgery setting and risk of postoperative complication.

CONCLUSIONS

After accounting for the selection of higher-risk patients to CC settings, we found no difference in hernia surgery postoperative complications between CC and VHA. Future VHA and non-VHA comparisons should account for unobserved as well as observed differences in patients seen in each setting.

Assessing the sustainability of compliance with surgical site infection prophylaxis after discontinuation of mandatory active reporting: study protocol

BACKGROUND

Surgical site infections are common. Risk can be reduced substantially with appropriate preoperative antimicrobial administration. In 2005, the VA implemented the Surgical Care Improvement Project (SCIP) in the setting of high rates of non-compliance with antimicrobial prophylaxis guidelines. SCIP included public reporting of evidenced-based antimicrobial guideline compliance metrics in high-risk surgeries. SCIP was highly successful and led to high rates of adoption of preoperative antimicrobials and early discontinuation of postoperative antimicrobials (>95%). The program was retired in 2015, as the manual measurement and reporting process was costly with limited expected additional benefit. To our knowledge, no studies have assessed whether the gains achieved by SCIP were sustained since active support for the program was discontinued. Furthermore, there has been no investigation of the spread of antimicrobial prophylaxis guideline adoption beyond the limited set of procedures that were included in the program.

METHODS

Using a mixed methods sequential exploratory approach, this study will (1) quantitatively measure compliance with SCIP metrics over time and across all procedures in the five major surgical specialties targeted by SCIP and (2) collect qualitative data from stakeholders to identify strategies that were effective for sustaining compliance. Diffusion of Innovation Theory will guide assessment of whether improvements achieved spread to procedures not included under the umbrella of the program. Electronic algorithms to measure SCIP antimicrobial use will be adapted from previously developed methodology. These highly novel data mining algorithms leverage the rich VA electronic health record and capture structured and text data and represent a substantial technological advancement over resource-intensive manual chart review or incomplete electronic surveillance based on pharmacy data. An interrupted time series analysis will be used to assess whether SCIP compliance was sustained following program discontinuation. Generalized linear models will be used to assess whether compliance with appropriate prophylaxis increased in all SCIP targeted and non-targeted procedures by specialty over the duration the program's active reporting. The Dynamic Sustainability Framework will guide the qualitative methods to assess intervention, provider, facility, specialty, and contextual factors associated with sustainability over time. Barriers and facilitators to sustainability will be mapped to implementation strategies and the study will yield an implementation playbook to guide future sustainment efforts.

RELEVANCE

Sustainability of practice change has been described as one of the most important, but least studied areas of clinical medicine. Learning how practices spread is also a critically important area of investigation. This study will use novel informatics strategies to evaluate factors associated with sustainability following removal of active policy surveillance and advance our understanding about these important, yet understudied, areas.

Promoting de-implementation of inappropriate antimicrobial use in cardiac device procedures by expanding audit and feedback: protocol for hybrid III type effectiveness/implementation quasi-experimental study

Background

Despite a strong evidence base and clinical guidelines specifically recommending against prolonged post-procedural antimicrobial use, studies indicate that the practice is common following cardiac device procedures. Formative evaluations conducted by the study team suggest that inappropriate antimicrobial use may be driven by information silos that drive provider belief that antimicrobials are not harmful, in part due to lack of complete feedback about all types of clinical outcomes. De-implementation is recognized as an important area of research that can lead to reductions in unnecessary, wasteful, or harmful practices, such as excess antimicrobial use following cardiac device procedures; however, investigations into strategies that lead to successful de-implementation are limited. The overarching hypothesis to be tested in this trial is that a bundle of implementation strategies that includes audit and feedback about direct patient harms caused by inappropriate prescribing can lead to successful de-implementation of guideline-discordant care.

Methods

We propose a hybrid type III effectiveness-implementation stepped-wedge intervention trial at three high-volume, high-complexity VA medical centers. The main study intervention (an informatics-based, real-time audit-and-feedback tool) was developed based on learning/unlearning theory and formative evaluations and guided by the integrated-Promoting Action on Research Implementation in Health Services (i-PARIHS) Framework. Elements of the bundled and multifaceted implementation strategy to promote appropriate prescribing will include audit-and-feedback reports that include information about antibiotic harms, stakeholder engagement, patient and provider education, identification of local champions, and blended facilitation. The primary study outcome is adoption of evidence-based practice (de-implementation of inappropriate antimicrobial use). Clinical outcomes (cardiac device infections, acute kidney injuries and Clostridioides difficile infections) are secondary. Qualitative interviews will assess relevant implementation outcomes (acceptability, adoption, fidelity, feasibility).

Discussion

De-implementation theory suggests that factors that may have a particularly strong influence on de-implementation include strength of the underlying evidence, the complexity of the intervention, and patient and provider anxiety and fear about changing an established practice. This study will assess whether a multifaceted intervention mapped to identified de-implementation barriers leads to measurable improvements in provision of guideline-concordant antimicrobial use. Findings will improve understanding about factors that impact successful or unsuccessful de-implementation of harmful or wasteful healthcare practices.

Trial registration

ClinicalTrials.govNCT05020418

Association Between Diabetic Foot Infection Wound Culture Positivity and 1-Year Admission for Invasive Infection: A Multicenter Cohort Study

Inpatients with culture-positive diabetic foot infections are at elevated risk for subsequent invasive infection with the same causative organism. In outpatients with index diabetic foot ulcers, we found that wound culture positivity was independently associated with increased odds of 1-year admission for systemic infection when compared with culture-negative wounds.

Comparing total medical expenditure between patients receiving direct oral anticoagulants vs warfarin for the treatment of atrial fibrillation: evidence from VA-Medicare dual enrollees

BACKGROUND: Direct oral anticoagulants (DOACs) are an alternative to warfarin for treatment of atrial fibrillation (AF). Evidence demonstrating the efficacy and safety of DOACs has primarily been from clinical trial settings. The real-world effectiveness of DOACs in specific nontrial populations that differ in age, comorbidity burden, and socioeconomic status is unclear. OBJECTIVE: To compare total downstream medical expenditure between AF patients treated with warfarin and DOACs dually enrolled in the Veterans Affairs (VA) Healthcare System and fee-for-service Medicare. METHODS: This was an exploratory treatment effectiveness study that analyzed VA administrative data and Medicare claims. We examined patients with an incident diagnosis for AF and initiated warfarin or DOAC treatment between 2012 and 2015. The primary outcome was total medical expenditure over 3 years following treatment initiation. To address potential informative censoring, we applied a multipart estimator that extends traditional 2-part models to separate differences between groups due to survival and cost accumulation effects. Inverse probability weighting was applied to address potential treatment selection bias. RESULTS: We identified 31,276 and 17,021 patients receiving warfarin and DOACs, respectively. Mean unadjusted (SD) expenditure was higher for warfarin ($56,265 [$96,666]) compared with DOAC patients ($32,736 [$52,470]). Compared with patients receiving DOACs, adjusted 3-year expenditure was $25,688 (P < 0.001) higher for patients receiving warfarin. CONCLUSIONS: VA patients with AF initiating warfarin incurred markedly higher downstream expenditure compared with similar patients receiving DOACs. The benefits of DOACs found in previous clinical trials were present in this population, suggesting that these DOACs may be the preferred option for treatment of AF in older VA patients. DISCLOSURES: This study was funded by a VA Health Services Research and Development Investigator Initiated Research Award (IIR 15-139). Support for VA/CMS data was provided by the Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development, Health Services Research and Development, VA Information Resource Center (Project Numbers SDR 02-237 and 98-004). The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs, the University of Washington, Northeastern University, and Boston University. The authors declare no conflicts of interest. This research includes data obtained from the VHA Office of Performance Measurement (17API2), which resides within the Office of Analytics and Performance Integration (API), under the Office of Quality and Patient Safety (QPS; formerly known as RAPID). An oral presentation documenting a subset of the findings from this study was presented at the 2020 AcademyHealth Annual Research Meeting, delivered virtually on July 29, 2020.

Identifying Risks and Opportunities in Outpatient Surgical Patient Safety: A Qualitative Analysis of Veterans Health Administration Staff Perceptions

OBJECTIVES

Little is known about patient safety risks in outpatient surgery. Inpatient surgical adverse events (AEs) risk factors include patient- (e.g., advanced age), process- (e.g., inadequate preoperative assessment), or structure-related characteristics (e.g., low surgical volume); however, these factors may differ from outpatient care where surgeries are often elective and in younger/healthier patients. We undertook an exploratory qualitative research project to identify risk factors for AEs in outpatient surgery.

METHODS

We developed a conceptual framework of patient, process, and structure factors associated with surgical AEs on the basis of a literature review. This framework informed our semistructured interview guide with (1) open-ended questions about a specific outpatient AE that the participant experienced and (2) outpatient surgical patient safety risk factors in general. We interviewed nationwide Veterans Health Administration surgical staff. Results were coded on the basis of categories in the conceptual framework, and additional themes were identified using content analysis.

RESULTS

Fourteen providers representing diverse surgical roles participated. Ten reported witnessing an AE, and everyone provided input on risk factors in our conceptual framework. We did not find evidence that patient race/age, surgical technique, or surgical volume affected patient safety. Emerging factors included patient compliance, postoperative patient assessments/instruction, operating room equipment needs, and safety culture.

CONCLUSIONS

Surgical staff are familiar with AEs and patient safety problems in outpatient surgery. Our results show that processes of care undertaken by surgical providers, as opposed to immutable patient characteristics, may affect the occurrence of AEs. The factors we identified may facilitate more targeted research on outpatient surgical AEs.

Novel Method to Flag Cardiac Implantable Device Infections by Integrating Text Mining With Structured Data in the Veterans Health Administration's Electronic Medical Record

IMPORTANCE

Health care-associated infections (HAIs) are preventable, harmful, and costly; however, few resources are dedicated to infection surveillance of nonsurgical procedures, particularly cardiovascular implantable electronic device (CIED) procedures.

OBJECTIVE

To develop a method that includes text mining of electronic clinical notes to reliably and efficiently measure HAIs for CIED procedures.

DESIGN, SETTING, AND PARTICIPANTS

In this multicenter, national cohort study using electronic medical record data for patients undergoing CIED procedures in Veterans Health Administration (VA) facilities for fiscal years (FYs) 2016 and 2017, an algorithm to flag cases with a true CIED-related infection based on structured (eg, microbiology orders, vital signs) and free text diagnostic and therapeutic data (eg, procedure notes, discharge summaries, microbiology results) was developed and validated. Procedure data were divided into development and validation data sets. Criterion validity (ie, positive predictive validity [PPV], sensitivity, and specificity) was assessed via criterion-standard manual medical record review.

EXPOSURES

CIED procedure.

MAIN OUTCOMES AND MEASURES

The concordance between medical record review and the study algorithm with respect to the presence or absence of a CIED infection. CIED infection in the algorithm included 90-day mortality, congestive heart failure and nonmetastatic tumor comorbidities, CIED or surgical site infection International Statistical Classification of Diseases and Related Health Problems, Tenth Revision, Clinical Modification (ICD-10-CM) diagnosis codes, antibiotic treatment of Staphylococci, a microbiology test of a cardiac specimen, and text documentation of infection in specific clinical notes (eg, cardiology, infectious diseases, inpatient discharge summaries).

RESULTS

The algorithm sample consisted of 19 212 CIED procedures; 15 077 patients (78.5%) were White individuals, 1487 (15.5%) were African American; 18 766 (97.7%) were men. The mean (SD) age in our sample was 71.8 (10.6) years. The infection detection threshold of predicted probability was set to greater than 0.10 and the algorithm flagged 276 of 9606 (2.9%) cases in the development data set (9606 procedures); PPV in this group was 41.4% (95% CI, 31.6%-51.8%). In the validation set (9606 procedures), at predicted probability 0.10 or more the algorithm PPV was 43.5% (95% CI, 37.1%-50.2%), and overall sensitivity and specificity were 94.4% (95% CI, 88.2%-97.9%) and 48.8% (95% CI, 42.6%-55.1%), respectively.

CONCLUSIONS AND RELEVANCE

The findings of this study suggest that the method of combining structured and text data in VA electronic medical records can be used to expand infection surveillance beyond traditional boundaries to include outpatient and procedural areas.

Novel methodology to measure pre-procedure antimicrobial prophylaxis: integrating text searches with structured data from the Veterans Health Administration's electronic medical record

Background

Antimicrobial prophylaxis is an evidence-proven strategy for reducing procedure-related infections; however, measuring this key quality metric typically requires manual review, due to the way antimicrobial prophylaxis is documented in the electronic medical record (EMR). Our objective was to electronically measure compliance with antimicrobial prophylaxis using both structured and unstructured data from the Veterans Health Administration (VA) EMR. We developed this methodology for cardiac device implantation procedures.

Methods

With clinician input and review of clinical guidelines, we developed a list of antimicrobial names recommended for the prevention of cardiac device infection. We trained the algorithm using existing fiscal year (FY) 2008–15 data from the VA Clinical Assessment Reporting and Tracking-Electrophysiology (CART-EP), which contains manually determined information about antimicrobial prophylaxis. We merged CART-EP data with EMR data and programmed statistical software to flag an antimicrobial orders or drug fills from structured data fields in the EMR and hits on text string searches of antimicrobial names documented in clinician’s notes. We iteratively tested combinations of these data elements to optimize an algorithm to accurately classify antimicrobial use. The final algorithm was validated in a national cohort of VA cardiac device procedures from FY2016–2017. Discordant cases underwent expert manual review to identify reasons for algorithm misclassification.

Results

The CART-EP dataset included 2102 procedures at 38 VA facilities with manually identified antimicrobial prophylaxis in 2056 cases (97.8%). The final algorithm combining structured EMR fields and text note search results correctly classified 2048 of the CART-EP cases (97.4%). In the validation sample, the algorithm measured compliance with antimicrobial prophylaxis in 16,606 of 18,903 cardiac device procedures (87.8%). Misclassification was due to EMR documentation issues, such as antimicrobial prophylaxis documented only in hand-written clinician notes in a format that cannot be electronically searched.

Conclusions

We developed a methodology with high accuracy to measure guideline concordant use of antimicrobial prophylaxis before cardiac device procedures using data fields present in modern EMRs. This method can replace manual review in quality measurement in the VA and other healthcare systems with EMRs; further, this method could be adapted to measure compliance in other procedural areas where antimicrobial prophylaxis is recommended.

Veterans Perceptions of Satisfaction and Convenience with Anticoagulants for Atrial Fibrillation: Warfarin versus Direct Oral Anticoagulants

BACKGROUND

Atrial fibrillation (AFib) is associated with high morbidity and mortality. Traditionally, AFib was treated with warfarin, yet recent evidence suggests patients may favor direct oral anticoagulants (DOACs). Variation in preferences is common and we explored patients' perceptions of satisfaction and convenience of DOACs versus warfarin within the Veterans Health Administration (VA).

PATIENTS AND METHODS

We administered a cross-sectional survey, the Perception of Anticoagulant Treatment Questionnaire 2 (PACT-Q2), to Veterans residing in New England, age ≥65, diagnosed with AFib, and actively taking anticoagulant medication in fiscal year 2018. Survey recipients were randomly selected among patients on warfarin (n=200) or DOACs (n=200). A selection of survey respondents agreed to a follow-up semi-structured interview (n=16) to further investigate perceptions of satisfaction and convenience.

RESULTS

Of 400 patients, 187 completed the PACT-Q2 survey (49% on DOACs; 51% on warfarin). DOACs received significantly higher convenience ratings than warfarin (87.6, SD 13.5 vs 81.1, SD 18.8; p=0.007); there was no difference in satisfaction (64.2, SD 20.5 SD, warfarin vs, 67.3, SD 19.4, DOACs). Interview results showed that participants perceived their treatment to be convenient. However, participants expressed challenges related to the convenience of taking warfarin or DOACs, such as warfarin users having to follow dietary recommendations or DOAC users desiring some additional monitoring to answer questions or concerns. Overall, warfarin and DOAC users reported satisfaction with ongoing monitoring methods, although a few DOAC users expressed uncertainties with the frequency of monitoring. For most participants, concerns about side effects did not differ by anticoagulant type nor affect satisfaction.

CONCLUSION

Our survey and interview results showed variable patient satisfaction and perceptions of convenience with both DOACs and warfarin. Although DOACs are increasingly prescribed for AFib, some Veterans felt that regular follow-up on warfarin was advantageous. Our findings demonstrate the importance of patient-centered decision-making in AFib treatment in the VA patient population.

Association of Postoperative Readmissions With Surgical Quality Using a Delphi Consensus Process to Identify Relevant Diagnosis Codes

Importance

Postoperative readmission data are used to measure hospital performance, yet the extent to which these readmissions reflect surgical quality is unknown.

Objective

To establish expert consensus on whether reasons for postoperative readmission are associated with the quality of surgery in the index admission.

Design, Setting, and Participants

In a modified Delphi process, a panel of 14 experts in medical and surgical readmissions comprising physicians and nonphysicians from Veterans Affairs (VA) and private-sector institutions reviewed 30-day postoperative readmissions from fiscal years 2008 through 2014 associated with inpatient surgical procedures performed at a VA medical center between October 1, 2007, and September 30, 2014. The consensus process was conducted from January through May 2017. Reasons for readmission were grouped into categories based on International Classification of Diseases, Ninth Revision (ICD-9) diagnosis codes. Panelists were given the proportion of readmissions coded by each reason and median (interquartile range) days to readmission. They answered the question, "Does the readmission reason reflect possible surgical quality of care problems in the index admission?" on a scale of 1 (never related) to 5 (directly related) in 3 rounds of consensus building. The consensus process was completed in May 2017 and data were analyzed in June 2017.

Main Outcomes and Measures

Consensus on proportion of ICD-9-coded readmission reasons that reflected quality of surgical procedure.

Results

In 3 Delphi rounds, the 14 panelists achieved consensus on 50 reasons for readmission; 12 panelists also completed group telephone calls between rounds 1 and 2. Readmissions with diagnoses of infection, sepsis, pneumonia, hemorrhage/hematoma, anemia, ostomy complications, acute renal failure, fluid/electrolyte disorders, or venous thromboembolism were considered associated with surgical quality and accounted for 25 521 of 39 664 readmissions (64% of readmissions; 7.5% of 340 858 index surgical procedures). The proportion of readmissions considered to be not associated with surgical quality varied by procedure, ranging from to 21% (613 of 2331) of readmissions after lower-extremity amputations to 47% (745 of 1598) of readmissions after cholecystectomy.

Conclusions and Relevance

One-third of postoperative readmissions are unlikely to reflect problems with surgical quality. Future studies should test whether restricting readmissions to those with specific ICD-9 codes might yield a more useful quality measure.

Comparison of a Potential Hospital Quality Metric With Existing Metrics for Surgical Quality-Associated Readmission

Importance

The existing readmission quality metric does not meaningfully distinguish readmissions associated with surgical quality from those that are not associated with surgical quality and thus may not reflect the quality of surgical care.

Objective

To compare a quality metric that classifies readmissions associated with surgical quality with the existing metric of any unplanned readmission in a surgical population.

Design, Setting, and Participants

Cohort study using US nationwide administrative data collected on 4 high-volume surgical procedures performed at 103 Veterans Affairs hospitals from October 1, 2007, through September 30, 2014. Data analysis was conducted from October 1, 2017, to January 24, 2019.

Main Outcomes and Measures

Hospital-level rates of unplanned readmission (existing metric) and surgical readmissions associated with surgical quality (new metric) in the 30 days following hospital discharge for an inpatient surgical procedure.

Results

The study population included 109 258 patients who underwent surgery at 103 hospitals. Patients were majority male (94.1%) and white (78.2%) with a mean (SD) age of 64.0 (10.0) years at the time of surgery. After case-mix adjustment, 30-day surgical readmissions ranged from 4.6% (95% CI, 4.5%-4.8%) among knee arthroplasties to 11.1% (95% CI, 10.9%-11.3%) among colorectal resections. The new surgical readmission metric was significantly correlated with facility-level postdischarge complications for all procedures, with ρ coefficients ranging from 0.33 (95% CI, 0.13-0.51) for cholecystectomy to 0.52 (95% CI, 0.38-0.68) for colorectal resection. Correlations between postdischarge complications and the new surgical readmission metric were higher than correlations between complications and the existing readmission metric for all procedures examined (knee arthroplasty: 0.50 vs 0.48; hip replacement: 0.44 vs 0.18; colorectal resection: 0.52 vs 0.42; and cholecystectomy: 0.33 vs 0.10). When compared with using the existing readmission metric, using the new surgical readmission metric could change hip replacement-associated payment penalty determinations in 28.4% of hospitals and knee arthroplasty-associated penalties in 26.0% of hospitals.

Conclusions and Relevance

In this study, surgical quality-associated readmissions were more correlated with postdischarge complications at a higher rate than were unplanned readmissions. Thus, a metric based on such readmissions may be a better measure of surgical care quality. This work provides an important step in the development of future value-based payments and promotes evidence-based quality metrics targeting the quality of surgical care.

Facility type and surgical specialty are associated with suboptimal surgical antimicrobial prophylaxis practice patterns: a multi-center, retrospective cohort study

Background

Guidelines recommend discontinuation of antimicrobial prophylaxis within 24 h after incision closure in uninfected patients. However, how facility and surgical specialty factors affect the implementation of these evidence-based surgical prophylaxis guidelines in outpatient surgery is unknown. Thus, we sought to measure how facility complexity, including ambulatory surgical center (ASC) status and availability of ancillary services, impact adherence to guidelines for timely discontinuation of antimicrobial prophylaxis after outpatient surgery. A secondary aim was to measure the association between surgical specialty and guideline compliance.

Methods

A multi-center, national Veterans Health Administration retrospective cohort from 10/1/2015-9/30/2017 including any Veteran undergoing an outpatient surgical procedure in any of five specialties (general surgery, urology, ophthalmology, ENT, orthopedics) was created. The primary outcome was the association between facility complexity and proportion of surgeries not compliant with discontinuation of antimicrobials within 24 h of incision closure. Data were analyzed using logistic regression with adjustments for patient and procedural factors.

Results

Among 153,097 outpatient surgeries, 7712 (5.0%) received antimicrobial prophylaxis lasting > 24 h after surgery; rates ranged from 0.4% (eye surgeries) to 13.7% (genitourinary surgeries). Cystoscopies and cystoureteroscopy with lithotripsy procedures had the highest rates (16 and 20%), while hernia repair, cataract surgeries, and laparoscopic cholecystectomies had the lowest (0.2-0.3%). In an adjusted logistic regression model, lower complexity ASC and hospital outpatient departments had higher odds of prolonged antimicrobial prophylaxis compared to complex hospitals (OR ASC, 1.3, 95% CI: 1.2-1.5). Patient factors associated with higher odds of noncompliance with antimicrobial discontinuation included younger age, female sex, and white race. Genitourinary and ear/nose/throat surgeries were associated with the highest odds of prolonged antimicrobial prophylaxis.

Conclusions

Facility complexity appears to play a role in adherence to surgical infection prevention guidelines. Lower complexity facilities with limited infection prevention and antimicrobial stewardship resources may be important targets for quality improvement. Such interventions may be especially useful for genitourinary and ear/nose/throat surgical subspecialties. Increasing pharmacy, antimicrobial stewardship and/or infection prevention resources to promote more evidence-based care may support surgical providers in lower complexity ambulatory surgery centers and hospital outpatient departments in their efforts to improve this facet of patient safety.

Preoperative opioid use and postoperative pain associated with surgical readmissions

BACKGROUND

The extent of preoperative opioid utilization and the relationship with pain-related readmissions are not well understood.

METHODS

VA Surgical Quality Improvement Program data on general, vascular, and orthopedic surgeries (2007-2014) were merged with pharmacy data to evaluate preoperative opioid use and pain-related readmissions. Opioid use in the 6-month preoperative period was categorized as none, infrequent, frequent, and daily.

RESULTS

In the six-month preoperative period, 65.7% had no opioid use, 16.7% had infrequent use, 6.3% frequent use, and 11.4% were daily opioid users. Adjusted odds of pain-related readmission were higher for opioid-exposed groups vs the opioid-naïve group: infrequent (OR 1.17; 95% CI:1.04-1.31), frequent (OR 1.28; 95% CI:1.08-1.52), and daily (OR 1.49; 95% CI:1.27-1.74). Among preoperative opioid users, those with a pain-related readmission had higher daily preoperative oral morphine equivalents (mean 44.5 vs. 36.1, p < 0.001).

CONCLUSIONS

Patients using opioids preoperatively experienced higher rates of pain-related readmissions, which increased with frequency and dosage of opioid exposure.

Association between postoperative opioid use and outpatient surgical adverse events

BACKGROUND

Opioid-related adverse drug events are common following inpatient surgical procedures. Little is known about opioid prescribing after outpatient surgical procedures and if opioid use is associated with short term risks of outpatient surgical adverse events (AEs).

METHODS

VA Corporate Data Warehouse was used to identify opioid use within 48 h for FY2012-14 chart-reviewed cases from a larger VA study of AEs in outpatient surgeries. We estimated a multilevel logistic regression model to determine the effect of opioid exposure on risk of AEs between 2 and 30 days postoperatively.

RESULTS

Of the 1730 outpatient surgical cases, 628 (36%) had postoperative opioid use and 12% had an AE. Opioid use following outpatient surgery was not significantly associated with higher surgical AE rates after controlling for relevant covariates (OR = 1.1 95% CI 0.79-1.54). Only procedure RVUs were associated with higher odds of postoperative AEs.

CONCLUSIONS

Postoperative opioid use following outpatient surgery is not a significant driver of postoperative AEs.

Association Between Preoperative Proteinuria and Postoperative Acute Kidney Injury and Readmission

Importance

Proteinuria indicates renal dysfunction and is a risk factor for morbidity among medical patients, but less is understood among surgical populations. There is a paucity of studies investigating how preoperative proteinuria is associated with surgical outcomes, including postoperative acute kidney injury (AKI) and readmission.

Objective

To assess preoperative urine protein levels as a biomarker for adverse surgical outcomes.

Design, Setting, and Participants

A retrospective, population-based study was conducted in a cohort of patients with and without known preoperative renal dysfunction undergoing elective inpatient surgery performed at 119 Veterans Affairs facilities from October 1, 2007, to September 30, 2014. Data analysis was conducted from April 4 to December 1, 2016. Preoperative dialysis, septic, cardiac, ophthalmology, transplantation, and urologic cases were excluded.

Exposures

Preoperative proteinuria as assessed by urinalysis using the closest value within 6 months of surgery: negative (0 mg/dL), trace (15-29 mg/dL), 1+ (30-100 mg/dL), 2+ (101-300 mg/dL), 3+ (301-1000 mg/dL), and 4+ (>1000 mg/dL).

Main Outcomes and Measures

Primary outcome was postoperative predischarge AKI and 30-day postdischarge unplanned readmission. Secondary outcomes included any 30-day postoperative outcome.

Results

Of 346 676 surgeries, 153 767 met inclusion criteria, with the majority including orthopedic (37%), general (29%), and vascular procedures (14%). Evidence of proteinuria was shown in 43.8% of the population (trace: 20.6%, 1+: 16.0%, 2+: 5.5%, 3+: 1.6%) with 20.4%, 14.9%, 4.3%, and 0.9%, respectively, of the patients having a normal preoperative estimated glomerular filtration rate (eGFR). In unadjusted analysis, preoperative proteinuria was significantly associated with postoperative AKI (negative: 8.6%, trace: 12%, 1+: 14.5%, 2+: 21.2%, 3+: 27.6%; P < .001) and readmission (9.3%, 11.3%, 13.3%, 15.8%, 17.5%, respectively, P < .001). After adjustment, preoperative proteinuria was associated with postoperative AKI in a dose-dependent relationship (trace: odds ratio [OR], 1.2; 95% CI, 1.1-1.3, to 3+: OR, 2.0; 95% CI, 1.8-2.2) and 30-day unplanned readmission (trace: OR, 1.0; 95% CI, 1.0-1.1, to 3+: OR, 1.3; 95% CI, 1.1-1.4). Preoperative proteinuria was associated with AKI independent of eGFR.

Conclusions and Relevance

Proteinuria was associated with postoperative AKI and 30-day unplanned readmission independent of preoperative eGFR. Simple urine assessment for proteinuria may identify patients at higher risk of AKI and readmission to guide perioperative management.

Development of an Adverse Event Surveillance Model for Outpatient Surgery in the Veterans Health Administration

OBJECTIVE

Develop and validate a surveillance model to identify outpatient surgical adverse events (AEs) based on previously developed electronic triggers.

DATA SOURCES

Veterans Health Administration's Corporate Data Warehouse.

STUDY DESIGN

Six surgical AE triggers, including postoperative emergency room visits and hospitalizations, were applied to FY2012-2014 outpatient surgeries (n = 744,355). We randomly sampled trigger-flagged and unflagged cases for nurse chart review to document AEs and measured positive predictive value (PPV) for triggers. Next, we used chart review data to iteratively estimate multilevel logistic regression models to predict the probability of an AE, starting with the six triggers and adding in patient, procedure, and facility characteristics to improve model fit. We validated the final model by applying the coefficients to FY2015 outpatient surgery data (n = 256,690) and reviewing charts for cases at high and moderate probability of an AE.

PRINCIPAL FINDINGS

Of 1,730 FY2012-2014 reviewed surgeries, 350 had an AE (20 percent). The final surveillance model c-statistic was 0.81. In FY2015 surgeries with >0.8 predicted probability of an AE (n = 405, 0.15 percent), PPV was 85 percent; in surgeries with a 0.4-0.5 predicted probability of an AE, PPV was 38 percent.

CONCLUSIONS

The surveillance model performed well, accurately identifying outpatient surgeries with a high probability of an AE.

Factors Associated with Hospital Admission after Outpatient Surgery in the Veterans Health Administration

OBJECTIVE

To examine factors associated with 0- to 7-day admission after outpatient surgery in high-volume specialties: general surgery, orthopedics, urology, ear/nose/throat, and podiatry.

STUDY DESIGN

We calculated rates and assessed diagnosis codes for 0- to 7-day admission after outpatient surgery for Centers for Medicare and Medicaid Services (CMS) and Veterans Health Administration (VA) dually enrolled patients age 65 and older. We also estimated separate multilevel logistic regression models to compare patient, procedure, and facility characteristics associated with postoperative admission.

DATA COLLECTION

2011-2013 surgical encounter data from the VA Corporate Data Warehouse; geographic data from the Area Health Resources File; CMS enrollment and hospital admission data.

PRINCIPAL FINDINGS

Among 63,585 outpatient surgeries in 124 facilities, 0- to 7-day admission rates ranged from 5 percent (podiatry) to 28 percent (urology); nearly 66 percent of the admissions occurred on the day of surgery. Only 97 admissions were detected in the CMS data (1 percent). Surgical complications were diagnosed in 4 percent of admissions. Procedure complexity, measured by relative value units or anesthesia risk score, was associated with admission across all specialties.

CONCLUSION

As many as 20 percent of VA outpatient surgeries result in an admission. Complex procedures are more likely to be followed by admission, but more evidence is required to determine how many of these reflect potential safety or quality problems.

Detection and potential consequences of intraoperative adverse events: A pilot study in the veterans health administration

Surgical quality improvement efforts have focused on tracking and reducing postoperative mortality and morbidity. However, the prevalence of intraoperative adverse events (IAEs) and their association with postoperative surgical outcomes has been poorly studied. In this study, we detected IAEs using both retrospective chart review and prospective provider reporting. We then examined the association of IAEs with postoperative outcomes. The overall IAE detection rate per case was 0.7 and 0.07 (P < 0.0001) based on chart review and provider reporting, respectively. Types of IAEs varied between detection methods. Provider-reported IAEs were more serious, i.e., had a stronger association with 30-day postoperative complications than chart-identified IAEs (risk-adjusted odds ratios were 1.52 vs 1.02, respectively, both p < 0.0001). Our findings suggest that IAEs can be detected using either retrospective chart review or prospective provider reporting. However, provider reporting appears more likely to detect serious (albeit infrequent) IAEs compared to chart review.

A study to reduce readmissions after surgery in the Veterans Health Administration: design and methodology

BACKGROUND

Hospital readmissions are associated with higher resource utilization and worse patient outcomes. Causes of unplanned readmission to the hospital are multiple with some being better targets for intervention than others. To understand risk factors for surgical readmission and their incremental contribution to current Veterans Health Administration (VA) surgical quality assessment, the study, Improving Surgical Quality: Readmission (ISQ-R), is being conducted to develop a readmission risk prediction tool, explore predisposing and enabling factors, and identify and rank reasons for readmission in terms of salience and mutability.

METHODS

Harnessing the rich VA enterprise data, predictive readmission models are being developed in data from patients who underwent surgical procedures within the VA 2007-2012. Prospective assessment of psychosocial determinants of readmission including patient self-efficacy, cognitive, affective and caregiver status are being obtained from a cohort having colorectal, thoracic or vascular procedures at four VA hospitals in 2015-2017. Using these two data sources, ISQ-R will develop readmission categories and validate the readmission risk prediction model. A modified Delphi process will convene surgeons, non-surgeon clinicians and quality improvement nurses to rank proposed readmission categories vis-à-vis potential preventability.

DISCUSSION

ISQ-R will identify promising avenues for interventions to facilitate improvements in surgical quality, informing specifications for surgical workflow managers seeking to improve care and reduce cost. ISQ-R will work with Veterans Affairs Surgical Quality Improvement Program (VASQIP) to recommend potential new elements VASQIP might collect to monitor surgical complications and readmissions which might be preventable and ultimately improve surgical care.

Comparing definitions of outpatient surgery: Implications for quality measurement

BACKGROUND

Adverse event (AE) rates in outpatient surgery are inconsistently reported, partly because of the lack of a standard definition of outpatient surgery. We compared the types and rates of surgical procedures defined by two national healthcare agencies: Health Care Cost Institute (HCCI) and the Healthcare Cost and Utilization Project (HCUP) and considered implications for quality measurement.

METHODS

We used HCCI and HCUP definitions to identify FY2012-14 VA outpatient surgeries.

RESULTS

There were six times as many HCCI surgeries as HCUP (6,575,830 versus 1,086,640). Ninety-nine percent of HCUP-defined surgeries were also identified by HCCI. More HCUP surgeries had higher average Medicare Relative Value Units then HCCI surgeries [5.3 (SD = 4.4) versus 1.6 (SD = 2.3) RVUs].

CONCLUSIONS

Rates and types of procedures vary widely between definitions. Quality measurement using HCCI versus HCUP may produce significantly lower AE rates because many of the surgeries included reflect low complexity and potentially low risk of AEs.

Defining Outpatient Surgery: Perspectives of Surgical Staff in the Veterans Health Administration

BACKGROUND

A standard definition of outpatient surgery that is aligned with the perspective of surgical providers is needed for consistent and focused surgical quality measurement. We sought to clarify the procedure characteristics that define outpatient surgery through a consensus process with multidisciplinary surgical staff from the Veterans Health Administration (VA).

METHODS

A convenience sample of 14 VA surgical staff participated in three rounds of a modified-Delphi process. They rated procedure characteristics (e.g., settings, providers, anesthesia, and incision type) to include in a definition of outpatient surgery for adverse event detection.

RESULTS

Consensus was reached on 63% of the criteria. Participants agreed to exclude procedures performed by gastroenterologists; in contrast, they agreed to retain endoscopic procedures. There was inter and intra-variation between rounds but no pattern based on staff discipline.

CONCLUSIONS

Contradictory responses and the lack of consensus on several key questions revealed few procedure characteristics that surgical staff agreed could be used to define outpatient surgery. Our findings suggest that contextual factors associated with specific procedures may be important; future efforts to define outpatient surgery should consider both the characteristics of the procedure and the procedure itself.

Do Acute Myocardial Infarction and Heart Failure Readmissions Flagged as Potentially Preventable by the 3M Potentially Preventable Readmissions Software Have More Process-of-Care Problems?

BACKGROUND

The 3M Potentially Preventable Readmissions (3M-PPR) software matches clinically related index admission and readmission diagnoses that may signify in-hospital or postdischarge quality problems. To assess whether the PPR algorithm identifies preventable readmissions, we compared processes of care between PPR software-flagged and nonflagged cases.

METHODS AND RESULTS

Using 2006 to 2010 national VA administrative data, we identified acute myocardial infarction and heart failure discharges associated with 30-day all-cause readmissions, then flagged cases (PPR-Yes/PPR-No) using the 3M-PPR software. To assess care quality, we abstracted medical records of 100 readmissions per condition using tools containing explicit processes organized into admission work-up, in-hospital evaluation/treatment, discharge readiness, postdischarge period. We derived quality scores, scaled to a maximum of 25 per section (maximum total score=100) and compared cases on total and section-specific mean scores. For acute myocardial infarction, 77 of 100 cases were flagged as PPR-Yes. Section quality scores were highest for in-hospital evaluation/treatment (20.5±2.8) and lowest for postdischarge care (6.8±9.1). Total and section-related mean scores did not differ by PPR status; respective PPR-Yes versus PPR-No total scores were 61.6±11.1 and 60.4±9.4; P=0.98. For heart failure, 86 of 100 cases were flagged as PPR-Yes. Section scores were highest for discharge readiness (18.8±2.4) and lowest for postdischarge care (7.3±8.1). Like acute myocardial infarction, total and section-related mean scores did not differ by PPR status; PPR-Yes versus PPR-No total scores were 61.2±10.8 and 63.4±7.0, respectively; P=0.47.

CONCLUSIONS

Among VA acute myocardial infarction and heart failure readmissions, the 3M-PPR software does not distinguish differences in case-level quality of care. Whether 3M-PPR software better identifies preventable readmissions by using other methods to capture poorly documented processes or performing different comparisons requires further study.

Does Use of a Hospital-wide Readmission Measure Versus Condition-specific Readmission Measures Make a Difference for Hospital Profiling and Payment Penalties?

BACKGROUND

The Centers for Medicare and Medicaid Services (CMS) use public reporting and payment penalties as incentives for hospitals to reduce readmission rates. In contrast to the current condition-specific readmission measures, CMS recently developed an all-condition, 30-day all-cause hospital-wide readmission measure (HWR) to provide a more comprehensive view of hospital performance.

OBJECTIVES

We examined whether assessment of hospital performance and payment penalties depends on the readmission measure used.

RESEARCH DESIGN

We used inpatient data to examine readmissions for patients discharged from VA acute-care hospitals from Fiscal Years 2007-2010. We calculated risk-standardized 30-day readmission rates for 3 condition-specific measures (heart failure, acute myocardial infarction, and pneumonia) and the HWR measure, and examined agreement between the HWR measure and each of the condition-specific measures on hospital performance. We also assessed the effect of using different readmission measures on hospitals' payment penalties.

RESULTS

We found poor agreement between the condition-specific measures and the HWR measure on those hospitals identified as low or high performers (eg, among those hospitals classified as poor performers by the heart failure readmission measure, only 28.6% were similarly classified by the HWR measure). We also found differences in whether a hospital would experience payment penalties. The HWR measure penalized only 60% of those hospitals that would have received penalties based on at least 1 of the condition-specific measures.

CONCLUSIONS

The condition-specific measures and the HWR measure provide a different picture of hospital performance. Future research is needed to determine which measure aligns best with CMS's overall goals to reduce hospital readmissions and improve quality.

Do pneumonia readmissions flagged as potentially preventable by the 3M PPR software have more process of care problems? A cross-sectional observational study

BACKGROUND

In the USA, administrative data-based readmission rates such as the Centers for Medicare and Medicaid Services' all-cause readmission measures are used for public reporting and hospital payment penalties. To improve this measure and identify better quality improvement targets, 3M developed the Potentially Preventable Readmissions (PPRs) measure. It matches clinically related index admission and readmission diagnoses that may indicate readmissions resulting from admission- or post-discharge-related quality problems.

OBJECTIVE

To examine whether PPR software-flagged pneumonia readmissions are associated with poorer quality of care.

METHODS

Using a retrospective observational study design and Veterans Health Administration (VA) data, we identified pneumonia discharges associated with 30-day readmissions, and then flagged cases as PPR-yes or PPR-no using the PPR software. To assess quality of care, we abstracted electronic medical records of 100 random readmissions using a tool containing explicit care processes organised into admission work-up, in-hospital evaluation/treatment, discharge readiness and post-discharge period. We derived quality scores, scaled to a maximum of 25 per section (maximum total score=100) and compared cases by total and section-specific mean scores using t tests and effect size (ES) to characterise the clinical significance of findings.

RESULTS

Our abstraction sample was selected from 11,278 pneumonia readmissions (readmission rate=16.5%) during 1 October 2005-30 September 2010; 77% were flagged as PPR-yes. Contrary to expectations, total and section mean quality scores were slightly higher, although non-significantly, among PPR-yes (N=77) versus PPR-no (N=23) cases (respective total scores, 71.2±8.7 vs 65.8±11.5, p=0.14); differences demonstrated ES >0.30 overall and for admission work-up and post-discharge period sections.

CONCLUSIONS

Among VA pneumonia readmissions, PPR categorisation did not produce the expected quality of care findings. Either PPR-yes cases are not more preventable, or preventability assessment requires other data collection methods to capture poorly documented processes (eg, direct observation).

Assessing the potential adoption and usefulness of concurrent, action-oriented, electronic adverse drug event triggers designed for the outpatient setting

Background:

Adverse drug event (ADE) detection is an important priority for patient safety research. Trigger tools have been developed to help identify ADEs. In previous work we developed seven concurrent, action-oriented, electronic trigger algorithms designed to prompt clinicians to address ADEs in outpatient care.

Objectives:

We assessed the potential adoption and usefulness of the seven triggers by testing the positive predictive validity and obtaining stakeholder input.

Methods:

We adapted ADE triggers, “bone marrow toxin—white blood cell count (BMT-WBC),” “bone marrow toxin - platelet (BMT-platelet),” “potassium raisers,” “potassium reducers,” “creatinine,” “warfarin,” and “sedative hypnotics,” with logic to suppress flagging events with evidence of clinical intervention and applied the triggers to 50,145 patients from three large health care systems. Four pharmacists assessed trigger positive predictive value (PPV) with respect to ADE detection (conservatively excluding ADEs occurring during clinically appropriate care) and clinical usefulness (i.e., whether the trigger alert could change care to prevent harm). We measured agreement between raters using the free kappa and assessed positive PPV for the trigger’s detection of harm, clinical usefulness, and both. Stakeholders from the participating health care systems rated the likelihood of trigger adoption and the perceived ease of implementation.

Findings:

Agreement between pharmacist raters was moderately high for each ADE trigger (kappa free > 0.60). Trigger PPVs for harm ranged from 0 (Creatinine, BMT-WBC) to 17 percent (potassium raisers), while PPV for care change ranged from 0 (WBC) to 60 percent (Creatinine). Fifteen stakeholders rated the triggers. Our assessment identified five of the seven triggers as good candidates for implementation: Creatinine, BMT-Platelet, Potassium Raisers, Potassium Reducers, and Warfarin.

Conclusions:

At least five outpatient ADE triggers performed well and merit further evaluation in outpatient clinical care. When used in real time, these triggers may promote care changes to ameliorate patient harm.

What is the value of adding Medicare data in estimating VA hospital readmission rates?

OBJECTIVE

To determine the effects of including diagnostic and utilization data from a secondary payer on readmission rates and hospital profiles.

DATA SOURCES/STUDY SETTING

Veterans Health Administration (VA) and Medicare inpatient and outpatient administrative data for veterans discharged from 153 VA hospitals during FY 2008-2010 with a principal diagnosis of acute myocardial infarction, heart failure, or pneumonia.

STUDY DESIGN

We estimated hospital-level risk-standardized readmission rates derived using VA data only. We then used data from both VA and Medicare to reestimate readmission rates and compared hospital profiles using two methods: Hospital Compare and the CMS implementation of the Hospital Readmissions Reduction Program (HRRP).

DATA COLLECTION/EXTRACTION METHODS

Retrospective data analysis using VA hospital discharge and outpatient data matched with Medicare fee-for-service claims by scrambled Social Security numbers.

PRINCIPAL FINDINGS

Less than 2 percent of hospitals in any cohort were classified discordantly by the Hospital Compare method when using VA-only compared with VA/Medicare data. In contrast, using the HRRP method, 13 percent of hospitals had differences in whether they were flagged as having excessive readmission rates in at least one cohort.

CONCLUSIONS

Inclusion of secondary payer data may cause changes in hospital profiles, depending on the methodology used. An assessment of readmission rates should include, to the extent possible, all available information about patients' utilization of care.

Detecting adverse events in surgery: comparing events detected by the Veterans Health Administration Surgical Quality Improvement Program and the Patient Safety Indicators

BACKGROUND

The Patient Safety Indicators (PSIs) use administrative data to screen for select adverse events (AEs). In this study, VA Surgical Quality Improvement Program (VASQIP) chart review data were used as the gold standard to measure the criterion validity of 5 surgical PSIs. Independent chart review was also used to determine reasons for PSI errors.

METHODS

The sensitivity, specificity, and positive predictive value of PSI software version 4.1a were calculated among Veterans Health Administration hospitalizations (2003-2007) reviewed by VASQIP (n = 268,771). Nurses re-reviewed a sample of hospitalizations for which PSI and VASQIP AE detection disagreed.

RESULTS

Sensitivities ranged from 31% to 68%, specificities from 99.1% to 99.8%, and positive predictive values from 31% to 72%. Reviewers found that coding errors accounted for some PSI-VASQIP disagreement; some disagreement was also the result of differences in AE definitions.

CONCLUSIONS

These results suggest that the PSIs have moderate criterion validity; however, some surgical PSIs detect different AEs than VASQIP. Future research should explore using both methods to evaluate surgical quality.

Comparison of the Agency for Healthcare Research and Quality Patient Safety Indicator Rates Among Veteran Dual Users

This study compares rates of 11 Agency for Healthcare Research and Quality Patient Safety Indicators (PSIs) among 266 203 veteran dual users (ie, those with hospitalizations in both the Veterans Health Administration [VA] and the private sector through Medicare fee-for-service coverage) during 2002 to 2007. PSI risk-adjusted rates were calculated using the PSI software (version 3.1a). Rates of pressure ulcer, central venous catheter-related bloodstream infections, and postoperative sepsis, areas in which the VA has focused quality improvement efforts, were found to be significantly lower in the VA than in the private sector. VA had significantly higher rates for 7 of the remaining 8 PSIs, although the rates of only 2 PSIs (postoperative hemorrhage/hematoma and accidental puncture or laceration) remained higher in the VA after sensitivity analyses were conducted. A better understanding of system-level differences in coding practices and patient severity, poorly documented in administrative data, is needed before conclusions about differences in quality can be drawn.

Using AHRQ patient safety indicators to detect postdischarge adverse events in the Veterans Health Administration

Patient safety indicators (PSIs) use inpatient administrative data to flag cases with potentially preventable adverse events (AEs) attributable to hospital care. This study explored how many AEs the PSIs identified in the 30 days post discharge. PSI software was run on Veterans Health Administration 2003-2007 administrative data for 10 recently validated PSIs. Among PSI-eligible index hospitalizations not flagged with an AE, this study evaluated how many AEs occurred within 1 to 14 and 15 to 30 days post discharge using inpatient and outpatient administrative data. Considering all PSI-eligible index hospitalizations, 11 141 postdischarge AEs were identified, compared with 40 578 inpatient-flagged AEs. More than 60% of postdischarge AEs were detected within 14 days of discharge. The majority of postdischarge AEs were decubitus ulcers and postoperative pulmonary embolisms or deep vein thromboses. Extending PSI algorithms to the postdischarge period may provide a more complete picture of hospital quality. Future work should use chart review to validate postdischarge PSI events.

Validity of selected Patient Safety Indicators: opportunities and concerns

BACKGROUND

The Agency for Healthcare Research and Quality (AHRQ) recently designed the Patient Safety Indicators (PSIs) to detect potential safety-related adverse events. The National Quality Forum has endorsed several of these ICD-9-CM-based indicators as quality-of-care measures. We examined the positive predictive value (PPV) of 3 surgical PSIs: postoperative pulmonary embolus and deep vein thrombosis (pPE/DVT), iatrogenic pneumothorax (iPTX), and accidental puncture and laceration (APL).

STUDY DESIGN

We applied the AHRQ PSI software (v.3.1a) to fiscal year 2003 to 2007 Veterans Health Administration (VA) administrative data to identify (flag) patients suspected of having a pPE/DVT, iPTX, or APL. Two trained nurse abstractors reviewed a sample of 336 flagged medical records (112 records per PSI) using a standardized instrument. Inter-rater reliability was assessed.

RESULTS

Of 2,343,088 admissions, 6,080 were flagged for pPE/DVT (0.26%), 1,402 for iPTX (0.06%), and 7,203 for APL (0.31%). For pPE/DVT, the PPV was 43% (95% CI, 34% to 53%); 21% of cases had inaccurate coding (eg, arterial not venous thrombosis); and 36% featured thromboembolism present on admission or preoperatively. For iPTX, the PPV was 73% (95% CI, 64% to 81%); 18% had inaccurate coding (eg, spontaneous pneumothorax), and 9% were pneumothoraces present on admission. For APL, the PPV was 85% (95% CI, 77% to 91%); 10% of cases had coding inaccuracies and 5% indicated injuries present on admission. However, 27% of true APLs were minor injuries requiring no surgical repair (eg, small serosal bowel tear). Inter-rater reliability was >90% for all 3 PSIs.

CONCLUSIONS

Until coding revisions are implemented, these PSIs, especially pPE/DVT, should be used primarily for screening and case-finding. Their utility for public reporting and pay-for-performance needs to be reassessed.

Development of trigger tools for surveillance of adverse events in ambulatory surgery

BACKGROUND

The trigger tool methodology uses clinical algorithms applied electronically to 'flag' medical records where adverse events (AEs) have most likely occurred. The authors sought to create surgical triggers to detect AEs in the ambulatory care setting.

METHODS

Four consecutive steps were used to develop ambulatory surgery triggers. First, the authors conducted a comprehensive literature review for surgical triggers. Second, a series of multidisciplinary focus groups (physicians, nurses, pharmacists and information technology specialists) provided user input on trigger selection. Third, a clinical advisory panel designed an initial set of 10 triggers. Finally, a three-phase Delphi process (surgical and trigger tool experts) evaluated and rated the suggested triggers.

RESULTS

The authors designed an initial set of 10 surgical triggers including five global triggers (flagging medical records for the suspicion of any AE) and five AE-specific triggers (flagging medical records for the suspicion of specific AEs). Based on the Delphi rating of the trigger's utility for system-level interventions, the final triggers were: (1) emergency room visit(s) within 21 days from surgery; (2) unscheduled readmission within 30 days from surgery; (3) unscheduled procedure (interventional radiological, urological, dental, cardiac or gastroenterological) or reoperation within 30 days from surgery; (4) unplanned initial hospital length of stay more than 24 h; and (5) lower-extremity Doppler ultrasound order entry and ICD code for deep vein thrombosis or pulmonary embolus within 30 days from surgery.

CONCLUSION

The authors therefore propose a systematic methodology to develop trigger tools that takes into consideration previously published work, end-user preferences and expert opinion.

Informatics tools for the development of action-oriented triggers for outpatient adverse drug events

BACKGROUND

Trigger tools are an important development in the identification and reduction of adverse drug events (ADEs). Most previously published triggers are simple, consisting of one or two conditions. Simple logic may lead to alerts for conditions not caused by a drug or already treated by the provider.

METHODS

We created a knowledge-encoding tool to develop outpatient ADE triggers to more specifically identify harm caused by a drug and which require further clinical intervention. The tool presented the user with data on similar triggers from the literature and a series of fields to facilitate the creation of algorithms based on epidemiological principles.

RESULTS

Using this tool, we created 23 triggers that addressed 55 high-harm outpatient drugs and ADEs.

CONCLUSION

Informatics tools can facilitate the design of clinically rich triggers. More investigation is needed to determine whether the performance characteristics of clinically rich triggers are better than those of simple triggers.

Validity of selected AHRQ patient safety indicators based on VA National Surgical Quality Improvement Program data

OBJECTIVES

To examine the criterion validity of the Agency for Health Care Research and Quality (AHRQ) Patient Safety Indicators (PSIs) using clinical data from the Veterans Health Administration (VA) National Surgical Quality Improvement Program (NSQIP).

DATA SOURCES

Fifty five thousand seven hundred and fifty two matched hospitalizations from 2001 VA inpatient surgical discharge data and NSQIP chart-abstracted data.

STUDY DESIGN

We examined the sensitivities, specificities, positive predictive values (PPVs), and positive likelihood ratios of five surgical PSIs that corresponded to NSQIP adverse events. We created and tested alternative definitions of each PSI.

DATA COLLECTION

FY01 inpatient discharge data were merged with 2001 NSQIP data abstracted from medical records for major noncardiac surgeries.

PRINCIPAL FINDINGS

Sensitivities were 19-56 percent for original PSI definitions; and 37-63 percent using alternative PSI definitions. PPVs were 22-74 percent and did not improve with modifications. Positive likelihood ratios were 65-524 using original definitions, and 64-744 using alternative definitions. "Postoperative respiratory failure" and "postoperative wound dehiscence" exhibited significant increases in sensitivity after modifications.

CONCLUSIONS

PSI sensitivities and PPVs were moderate. For three of the five PSIs, AHRQ has incorporated our alternative, higher sensitivity definitions into current PSI algorithms. Further validation should be considered before most of the PSIs evaluated herein are used to publicly compare or reward hospital performance.