A scoping review of the methodological approaches used in retrospective chart reviews to validate adverse event rates in administrative data

Patient safety is a key quality issue for health systems. Healthcare acquired adverse events (AEs) compromise safety and quality; therefore, their reporting and monitoring is a patient safety priority. Although administrative datasets are potentially efficient tools for monitoring rates of AEs, concerns remain over the accuracy of their data. Chart review validation studies are required to explore the potential of administrative data to inform research and health policy. This review aims to present an overview of the methodological approaches and strategies used to validate rates of AEs in administrative data through chart review. This review was conducted in line with the Joanna Briggs Institute methodological framework for scoping reviews. Through database searches, 1054 sources were identified, imported into Covidence, and screened against the inclusion criteria. Articles that validated rates of AEs in administrative data through chart review were included. Data were extracted, exported to Microsoft Excel, arranged into a charting table, and presented in a tabular and descriptive format. Fifty-six studies were included. Most sources reported on surgical AEs; however, other medical specialties were also explored. Chart reviews were used in all studies; however, few agreed on terminology for the study design. Various methodological approaches and sampling strategies were used. Some studies used the Global Trigger Tool, a two-stage chart review method, whilst others used alternative single-, two-stage, or unclear approaches. The sources used samples of flagged charts (n = 24), flagged and random charts (n = 11), and random charts (n = 21). Most studies reported poor or moderate accuracy of AE rates. Some studies reported good accuracy of AE recording which highlights the potential of using administrative data for research purposes. This review highlights the potential for administrative data to provide information on AE rates and improve patient safety and healthcare quality. Nonetheless, further work is warranted to ensure that administrative data are accurate. The variation of methodological approaches taken, and sampling techniques used demonstrate a lack of consensus on best practice; therefore, further clarity and consensus are necessary to develop a more systematic approach to chart reviewing.

The deterrent effect of tort law: Evidence from medical malpractice reform

We examine whether caps on noneconomic damages in medical malpractice cases affect in-hospital patient safety. We use Patient Safety Indicators - measures of adverse events - as proxies for safety. In difference-in-differences ("DiD") analyses of five states that adopt caps during 2003-2005, we find that multiple measures of non-fatal patient safety events worsen after cap adoption relative to control states. DiD inference can be unreliable with a small number of treated units. We therefore develop a randomization inference-based test for inference with few treated units but multiple correlated outcomes and confirm the robustness of our results with this nonparametric approach. We also provide evidence that the decline in patient safety is unlikely to be driven by patient selection.

An International Perspective on Definitions and Terminology Used to Describe Serious Reportable Patient Safety Incidents: A Systematic Review

OBJECTIVES

Patients are unintentionally, yet frequently, harmed in situations that are deemed preventable. Incident reporting systems help prevent harm, yet there is considerable variability in how patient safety incidents are reported. This may lead to inconsistent or unnecessary patterns of incident reporting and failures to identify serious patient safety incidents. This systematic review aims to describe international approaches in relation to defining serious reportable patient safety incidents.

METHODS

Multiple electronic and gray literature databases were searched for articles published between 2009 and 2019. Empirical studies, reviews, national reports, and policies were included. A narrative synthesis was conducted because of study heterogeneity.

RESULTS

A total of 50 articles were included. There was wide variation in the terminology used to represent serious reportable patient safety incidents. Several countries defined a specific subset of incidents, which are considered sufficiently serious, yet preventable if appropriate safety measures are taken. Terms such as "never events," "serious reportable events," or "always review and report" were used. The following dimensions were identified to define a serious reportable patient safety incident: (1) incidents being largely preventable; (2) having the potential for significant learning; (3) causing serious harm or have the potential to cause serious harm; (4) being identifiable, measurable, and feasible for inclusion in an incident reporting system; and (5) running the risk of recurrence.

CONCLUSIONS

Variations in terminology and reporting systems between countries might contribute to missed opportunities for learning. International standardized definitions and blame-free reporting systems would enable comparison and international learning to enhance patient safety.

Key Issues in the Development of an Evidence-Based Stratified Surgical Patient Safety Improvement Information System: Experience From a Multicenter Surgical Safety Program

Surgery is still far from being completely safe and reliable. Surgical safety has, therefore, been the focus of considerable attention over the last few decades, and there are a growing number of national drives to improve it. There are also a number of large surgical complication reporting systems and system-based interventions, both of which have made remarkable progress in the past two decades. These systems, however, have either mainly focused on reporting complications and played a limited role in guiding practice or have provided nonselective interventions to all patients, perhaps imposing unnecessary burdens on frontline medical staff. We have, therefore, developed an evidence-based stratified surgical safety information system based on a multicenter surgical safety improvement program. This study discusses some critical issues in the process of developing this information system, including (1) decisions about data gathering, (2) establishing and sharing knowledge, (3) developing functions for the system, (4) system implementation, and (5) evaluation and continuous improvement. Using examples drawn from the surgical safety improvement program, we have shown how this type of system can be fitted into day-to-day clinical practice and how it can guide medical practice by incorporating inherent patient-related risk and providing tailored interventions for patients with different levels of risk. We concluded that multidisciplinary collaboration, involving experts in health care (including senior staff in surgery, nursing, and anesthesia), data science, health care management, and health information technology, can help build an evidence-based stratified surgical patient safety improvement system. This can provide an information-intensified surgical safety learning platform and, therefore, benefit surgical patients by delivering tailored interventions and an integrated workflow.

Navigating a ship with a broken compass: evaluating standard algorithms to measure patient safety

Objective

Agency for Healthcare Research and Quality (AHRQ) software applies standardized algorithms to hospital administrative data to identify patient safety indicators (PSIs). The objective of this study was to assess the validity of PSI flags and report reasons for invalid flagging.

Material and Methods

At a 6-hospital academic medical center, a retrospective analysis was conducted of all PSIs flagged in fiscal year 2014. A multidisciplinary PSI Quality Team reviewed each flagged PSI based on quarterly reports. The positive predictive value (PPV, the percent of clinically validated cases) was calculated for 12 PSI categories. The documentation for each reversed case was reviewed to determine the reasons for PSI reversal.

Results

Of 657 PSI flags, 185 were reversed. Seven PSI categories had a PPV below 75%. Four broad categories of reasons for reversal were AHRQ algorithm limitations (38%), coding misinterpretations (45%), present upon admission (10%), and documentation insufficiency (7%). AHRQ algorithm limitations included 2 subcategories: an "incident" was inherent to the procedure, or highly likely (eg, vascular tumor bleed), or an "incident" was nonsignificant, easily controlled, and/or no intervention was needed.

Discussion

These findings support previous research highlighting administrative data problems. Additionally, AHRQ algorithm limitations was an emergent category not considered in previous research. Herein we present potential solutions to address these issues.

Conclusions

If, despite poor validity, US policy continues to rely on PSIs for incentive and penalty programs, improvements are needed in the quality of administrative data and the standardized PSI algorithms. These solutions require national motivation, research attention, and dissemination support.

Geriatric Patient Safety Indicators Based on Linked Administrative Health Data to Assess Anticoagulant-Related Thromboembolic and Hemorrhagic Adverse Events in Older Inpatients: A Study Proposal

Background

Frail older people with multiple interacting conditions, polypharmacy, and complex care needs are particularly exposed to health care-related adverse events. Among these, anticoagulant-related thromboembolic and hemorrhagic events are particularly frequent and serious in older inpatients. The growing use of anticoagulants in this population and their substantial risk of toxicity and inefficacy have therefore become an important patient safety and public health concern worldwide. Anticoagulant-related adverse events and the quality of anticoagulation management should thus be routinely assessed to improve patient safety in vulnerable older inpatients.

Objective

This project aims to develop and validate a set of outcome and process indicators based on linked administrative health data (ie, insurance claims data linked to hospital discharge data) assessing older inpatient safety related to anticoagulation in both Switzerland and France, and enabling comparisons across time and among hospitals, health territories, and countries. Geriatric patient safety indicators (GPSIs) will assess anticoagulant-related adverse events. Geriatric quality indicators (GQIs) will evaluate the management of anticoagulants for the prevention and treatment of arterial or venous thromboembolism in older inpatients.

Methods

GPSIs will measure cumulative incidences of thromboembolic and bleeding adverse events based on hospital discharge data linked to insurance claims data. Using linked administrative health data will improve GPSI risk adjustment on patients’ conditions that are present at admission and will capture in-hospital and postdischarge adverse events. GQIs will estimate the proportion of index hospital stays resulting in recommended anticoagulation at discharge and up to various time frames based on the same electronic health data. The GPSI and GQI development and validation process will comprise 6 stages: (1) selection and specification of candidate indicators, (2) definition of administrative data-based algorithms, (3) empirical measurement of indicators using linked administrative health data, (4) validation of indicators, (5) analyses of geographic and temporal variations for reliable and valid indicators, and (6) data visualization.

Results

Study populations will consist of 166,670 Swiss and 5,902,037 French residents aged 65 years and older admitted to an acute care hospital at least once during the 2012-2014 period and insured for at least 1 year before admission and 1 year after discharge. We will extract Swiss data from the Helsana Group data warehouse and French data from the national health insurance information system (SNIIR-AM). The study has been approved by Swiss and French ethics committees and regulatory organizations for data protection.

Conclusions

Validated GPSIs and GQIs should help support and drive quality and safety improvement in older inpatients, inform health care stakeholders, and enable international comparisons. We discuss several limitations relating to the representativeness of study populations, accuracy of administrative health data, methods used for GPSI criterion validity assessment, and potential confounding bias in comparisons based on GQIs, and we address these limitations to strengthen study feasibility and validity.

Patient safety improvement interventions in children's surgery: A systematic review

BACKGROUND

Adult surgical patient safety literature is plentiful; however, there is a disproportionate paucity of published safety work in the children's surgical literature. We sought to systematically evaluate the nature and quality of patient safety evidence pertaining to pediatric surgical practice.

METHODS

Systematic search of MEDLINE and EMBASE databases and gray literature identified 1399 articles. Data pertaining to demographics, methodology, interventions, and outcomes were extracted. Study quality was assessed utilizing formal criteria.

RESULTS

20 studies were included. 14 (70%) comprised peer-reviewed articles. 18 (90%) were published in the last 4years. 13 (65%) described a novel intervention, and 7 (35%) described a modification of an existing intervention. Median patient sample size was 79 (29-1210). A large number (n=55) and variety (n=35) of measures were employed to evaluate the effect of interventions on patient safety. 15 (75%) studies utilized a checklist tool as a component of their intervention. 9 (45%) studies [comprising handoff tools (n=7); checklists (n=1); and multidimensional quality improvement initiatives (n=1)] reported a positive effect on patient safety. Quality assessment was undertaken on 14 studies. Quantitative studies had significantly higher quality scores than qualitative studies (61 [0-89] vs 44 [11-78], p=0.03).

CONCLUSIONS

Pediatric surgical patient safety evidence is in its early stages. Successful interventions that we identified were typically handoff tools. There now ought to be an onus on pediatric surgeons to develop and apply bespoke pediatric surgical safety interventions and generate an evidence base to parallel the adult literature.

LEVEL OF EVIDENCE

Level IV, Case series with no comparison group.

Healthy Life-Years Lost and Excess Bed-Days Due to 6 Patient Safety Incidents: Empirical Evidence From English Hospitals

BACKGROUND

There is little satisfactory evidence on the harm of safety incidents to patients, in terms of lost potential health and life-years.

OBJECTIVE

To estimate the healthy life-years (HLYs) lost due to 6 incidents in English hospitals between the years 2005/2006 and 2009/2010, to compare burden across incidents, and estimate excess bed-days.

RESEARCH DESIGN

The study used cross-sectional analysis of the medical records of all inpatients treated in 273 English hospitals. Patients with 6 types of preventable incidents were identified. Total attributable loss of HLYs was estimated through propensity score matching by considering the hypothetical remaining length and quality of life had the incident not occurred.

RESULTS

The 6 incidents resulted in an annual loss of 68 HLYs and 934 excess bed-days per 100,000 population. Preventable pressure ulcers caused the loss of 26 HLYs and 555 excess bed-days annually. Deaths in low-mortality procedures resulted in 25 lost life-years and 42 bed-days. Deep-vein thrombosis/pulmonary embolisms cost 12 HLYs, and 240 bed-days. Postoperative sepsis, hip fractures, and central-line infections cost <6 HLYs and 100 bed-days each.

DISCUSSION

The burden caused by the 6 incidents is roughly comparable with the UK burden of Multiple Sclerosis (80 DALYs per 100,000), HIV/AIDS and Tuberculosis (63 DALYs), and Cervical Cancer (58 DALYs). There were marked differences in the harm caused by the incidents, despite the public attention all of them receive. Decision makers can use the results to prioritize resources into further research and effective interventions.

Racial and Insurance Status Disparities in Patient Safety Indicators among Hospitalized Patients

OBJECTIVE

To examine the association between patient race/ethnicity, insurance status, and their interaction with patient safety indicators among hospitalized patients.

METHODS

Cross-sectional study was conducted. Data were extracted from the 2009 National Inpatient Sample. A total of 3,052,268 patient safety indicator-related discharges were identified. Dependent variables were 11 patient safety indicators (PSI) whereas independent variables included race/ethnicity and insurance status.

RESULTS

As compared with White patients, African American patients were more likely to experience pressure ulcer, post-operative hemorrhage or hematoma, and post-operative pulmonary embolism (PE) or deep vein thrombosis (DVE); Asian/Pacific Islander patients were more likely to experience pressure ulcer, post-operative PE or DVT, and two obstetric care PSIs; whereas Hispanic/Latino patients were more likely to experience post-operative physiometabolic derangement and accidental puncture/laceration. As compared with patients with private insurance, Medicaid patients were more likely to experience pressure ulcer, post-operative physiological metabolic derangement, post-operative PE or DVT, post-operative respiratory failure, post-operative wound dehiscence, and death among surgeries. However, both obstetric care PSIs showed that African Americans, Hispanics, and uninsured patients were less likely to incur them in comparison with their respective counterparts. Furthermore, strong interactive effects between African American and Medicaid on PSIs were detected.

CONCLUSIONS

Although mixed findings in disparities in PSIs were observed in our study, Asian/Pacific Islander patients and Medicaid patients seem to be the most vulnerable. Further, interactive effects between African American and Medicaid indicate that poverty may be a key factor related to disparities in health care. Future research is merited to identify underlying factors that are related to PSIs among Asian/Pacific Islander patients. Strategies are needed to improve PSIs among Medicaid patients, especially during the current Medicaid program expansion due to the implementation of the Affordable Care Act.

A Systematic Review of Patient Safety Measures in Adult Primary Care

Safety measure development has focused on inpatient care despite outpatient visits far outnumbering inpatient admissions. Some measures are clearly identified as outpatient safety measures when published, yet outcomes from quality improvement studies also may be useful measures. The authors conducted a systematic review of the literature to identify published articles detailing safety measures applicable to adult primary care. A total of 21 articles were identified, providing specifications for 182 safety measures. Each measure was classified into one of 6 outpatient safety dimensions: medication management, sentinel events, care coordination, procedures and treatment, laboratory testing and monitoring, and facility structures/resources. Compared to the multitude of available inpatient safety measures, the number of existing adult primary care measures is low. The measures identified by this systematic review may yield further insight into the breadth of safety events causing harm in primary care, while also identifying areas of patient safety in primary care that may be understudied.

Does effective designing of operation theaters contribute towards staff satisfaction

Purpose

– This study aims to measure the staff satisfaction achieved with regard to the recently furnished modular operation theaters (MOTs).

Design/methodology/approach

– A cross-sectional study through questionnaire-based interviews was done. Desired sample size for ANOVA design came out to be 25 per level at a level of significance of 5 per cent and a power of 85 per cent.

Findings

– Overall, mean rating of the satisfaction of the staff was 7.52 with a standard deviation (SD) of 2.35. Mean ratings (with standard deviations) of surgeons, nurses and anesthetists were 7.14 (1.26), 7.21 (0.95) and 8.21 (0.48), respectively. One sample t-test showed that all the three categories of staff were satisfied. Post-hoc test revealed that the anesthetists were significantly more satisfied than the surgeons (p = 0) and the nurses (p = 0.001). Maximum satisfier was aseptic environment provided by the MOTs. Hatch box with ultraviolet technology also attracted high ratings from all the three categories. Staff considered all the probable advantages of MOTs, except air showers, significant with regard to its satisfaction and morale.

Originality/value

– Findings suggest that initiatives such as effectively designed MOTs may contribute toward the satisfaction of all categories of staff working in operation theaters (OTs), which, in turn, may probably lead to better overall performance of these facilities. It is desirable that hospital planners in modern health-care systems give adequate importance to finer aspects of OT designing.

Usefulness of administrative databases for risk adjustment of adverse events in surgical patients

PURPOSE

The aim of this study was to assess the usefulness of clinical-administrative databases for the development of risk adjustment in the assessment of adverse events in surgical patients.

METHODS

The study was conducted at the Hospital of Navarra, a tertiary teaching hospital in northern Spain. We studied 1602 hospitalizations of surgical patients from 2008 to 2010. We analysed 40 comorbidity variables included in the National Surgical Quality Improvement (NSQIP) Program of the American College of Surgeons using 2 sources of information: The clinical and administrative database (CADB) and the data extracted from the complete clinical records (CR), which was considered the gold standard. Variables were catalogued according to compliance with the established criteria: sensitivity, positive predictive value and kappa coefficient >0.6.

RESULTS

The average number of comorbidities per study participant was 1.6 using the CR and 0.95 based on CADB (p<.0001). Thirteen types of comorbidities (accounting for 8% of the comorbidities detected in the CR) were not identified when the CADB was the source of information. Five of the 27 remaining comorbidities complied with the 3 established criteria; 2 pathologies fulfilled 2 criteria, whereas 11 fulfilled 1, and 9 did not fulfil any criterion.

CONCLUSION

CADB detected prevalent comorbidities such as comorbid hypertension and diabetes. However, the CABD did not provide enough information to assess the variables needed to perform the risk adjustment proposed by the NSQIP for the assessment of adverse events in surgical patients.

Role of data warehousing in healthcare epidemiology

Electronic storage of healthcare data, including individual-level risk factors for both infectious and other diseases, is increasing. These data can be integrated at hospital, regional and national levels. Data sources that contain risk factor and outcome information for a wide range of conditions offer the potential for efficient epidemiological analysis of multiple diseases. Opportunities may also arise for monitoring healthcare processes. Integrating diverse data sources presents epidemiological, practical, and ethical challenges. For example, diagnostic criteria, outcome definitions, and ascertainment methods may differ across the data sources. Data volumes may be very large, requiring sophisticated computing technology. Given the large populations involved, perhaps the most challenging aspect is how informed consent can be obtained for the development of integrated databases, particularly when it is not easy to demonstrate their potential. In this article, we discuss some of the ups and downs of recent projects as well as the potential of data warehousing for antimicrobial resistance monitoring.

Patient safety perceptions of primary care providers after implementation of an electronic medical record system

BACKGROUND

Increasing the use of electronic medical records (EMR) has been suggested as an important strategy for improving healthcare safety.

OBJECTIVE

To sequentially measure, evaluate, and respond to safety culture and practice safety concerns following EMR implementation.

DESIGN

Safety culture was assessed using a validated tool (Safety Attitudes Questionnaire; SAQ), immediately following EMR implementation (T1) and at 1.5 (T2) and 2.5 (T3) years post-implementation. The SAQ was supplemented with a practice-specific assessment tool to identify safety needs and barriers.

PARTICIPANTS

A large medical group practice with a primary care core of 17-18 practices, staffed by clinicians in family medicine, pediatrics, internal medicine.

INTERVENTIONS

Survey results were used to define and respond to areas of need between assessments with system changes and educational programs.

MAIN MEASURES

Change in safety culture over time; perceived impact of EMR on practice.

KEY RESULTS

Responses were received from 103 of 123 primary care providers in T1 (83.7 % response rate), 122 of 143 in T2 (85.3 %) and 142 of 181 in T3 (78.5 %). Safety culture improved over this period, with statistically significant improvement in all domains except for stress recognition. Time constraints, communications and patient adherence were perceived to be the most important safety issues. The majority of respondents in both T2 (77.9 %) and T3 (85.4 %) surveys agreed that the EMR improved their ability to provide care more safely.

CONCLUSIONS

Implementation of an EMR in a large primary care practice required redesign of many organizational processes, and was associated with improvements in safety culture. Most primary care providers agreed that the EMR allowed them to provide care more safely.