Data quality issues impede comparability of hospital treatment delay performance indicators

The quality of vital signs measurements and value preferences in electronic medical records varies by hospital, specialty, and patient demographics

We aimed to assess the frequency of value preferences in recording of vital signs in electronic healthcare records (EHRs) and associated patient and hospital factors. We used EHR data from Oxford University Hospitals, UK, between 01-January-2016 and 30-June-2019 and a maximum likelihood estimator to determine the prevalence of value preferences in measurements of systolic and diastolic blood pressure (SBP/DBP), heart rate (HR) (readings ending in zero), respiratory rate (multiples of 2 or 4), and temperature (readings of 36.0 °C). We used multivariable logistic regression to investigate associations between value preferences and patient age, sex, ethnicity, deprivation, comorbidities, calendar time, hour of day, days into admission, hospital, day of week and speciality. In 4,375,654 records from 135,173 patients, there was an excess of temperature readings of 36.0 °C above that expected from the underlying distribution that affected 11.3% (95% CI 10.6-12.1%) of measurements, i.e. these observations were likely inappropriately recorded as 36.0 °C instead of the true value. SBP, DBP and HR were rounded to the nearest 10 in 2.2% (1.4-2.8%) and 2.0% (1.3-5.1%) and 2.4% (1.7-3.1%) of measurements. RR was also more commonly recorded as multiples of 2. BP digit preference and an excess of temperature recordings of 36.0 °C were more common in older and male patients, as length of stay increased, following a previous normal set of vital signs and typically more common in medical vs. surgical specialities. Differences were seen between hospitals, however, digit preference reduced over calendar time. Vital signs may not always be accurately documented, and this may vary by patient groups and hospital settings. Allowances and adjustments may be needed in delivering care to patients and in observational analyses and predictive tools using these factors as outcomes or exposures.

The relationship of hospital and surgeon volume indicators and post-operative outcomes in pancreatic surgery: a systematic literature review, meta-analysis and guidance for valid outcome assessment

BACKGROUND

Available evidence on the volume-outcome relationship after pancreatic surgery is limited due to the narrow focus of interventions, volume indicators and outcomes considered as well as due to methodological differences of the included studies. Therefore, we aim to evaluate the volume-outcome relationship following pancreatic surgery following strict study selection and quality criteria, to identify aspects of methodological variation and to define a set of key methodological indicators to consider when aiming for comparable and valid outcome assessment.

METHODS

Four electronic databases were searched to identify studies on the volume-outcome relationship in pancreatic surgery published between the years 2000-2018. Following a double-screening process, data extraction, quality appraisal, and subgroup analysis, results of included studies were stratified and pooled using random effects meta-analysis.

RESULTS

Consistent associations were found between high hospital volume and both postoperative mortality (OR 0.35, 95% CI: 0.29-0.44) and major complications (OR 0.87, 95% CI: 0.80-0.94). A significant decrease in the odds ratio was also found for high surgeon volume and postoperative mortality (OR 0.29, 95%CI: 0.22-0.37).

DISCUSSION

Our meta-analysis confirms a positive effect for both hospital and surgeon volume indicators for pancreatic surgery. Further harmonization (e.g. surgery types, volume cut-offs/definition, case-mix adjustment, reported outcomes) are recommended for future empirical studies.

Development and validation of early warning score system: A systematic literature review

OBJECTIVES

This review aims to: 1) evaluate the quality of model reporting, 2) provide an overview of methodology for developing and validating Early Warning Score Systems (EWSs) for adult patients in acute care settings, and 3) highlight the strengths and limitations of the methodologies, as well as identify future directions for EWS derivation and validation studies.

METHODOLOGY

A systematic search was conducted in PubMed, Cochrane Library, and CINAHL. Only peer reviewed articles and clinical guidelines regarding developing and validating EWSs for adult patients in acute care settings were included. 615 articles were extracted and reviewed by five of the authors. Selected studies were evaluated based on the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) checklist. The studies were analyzed according to their study design, predictor selection, outcome measurement, methodology of modeling, and validation strategy.

RESULTS

A total of 29 articles were included in the final analysis. Twenty-six articles reported on the development and validation of a new EWS, while three reported on validation and model modification. Only eight studies met more than 75% of the items in the TRIPOD checklist. Three major techniques were utilized among the studies to inform their predictive algorithms: 1) clinical-consensus models (n = 6), 2) regression models (n = 15), and 3) tree models (n = 5). The number of predictors included in the EWSs varied from 3 to 72 with a median of seven. Twenty-eight models included vital signs, while 11 included lab data. Pulse oximetry, mental status, and other variables extracted from electronic health records (EHRs) were among other frequently used predictors. In-hospital mortality, unplanned transfer to the intensive care unit (ICU), and cardiac arrest were commonly used clinical outcomes. Twenty-eight studies conducted a form of model validation either within the study or against other widely-used EWSs. Only three studies validated their model using an external database separate from the derived database.

CONCLUSION

This literature review demonstrates that the characteristics of the cohort, predictors, and outcome selection, as well as the metrics for model validation, vary greatly across EWS studies. There is no consensus on the optimal strategy for developing such algorithms since data-driven models with acceptable predictive accuracy are often site-specific. A standardized checklist for clinical prediction model reporting exists, but few studies have included reporting aligned with it in their publications. Data-driven models are subjected to biases in the use of EHR data, thus it is particularly important to provide detailed study protocols and acknowledge, leverage, or reduce potential biases of the data used for EWS development to improve transparency and generalizability.

Possible Sources of Bias in Primary Care Electronic Health Record Data Use and Reuse

BACKGROUND

Enormous amounts of data are recorded routinely in health care as part of the care process, primarily for managing individual patient care. There are significant opportunities to use these data for other purposes, many of which would contribute to establishing a learning health system. This is particularly true for data recorded in primary care settings, as in many countries, these are the first place patients turn to for most health problems.

OBJECTIVE

In this paper, we discuss whether data that are recorded routinely as part of the health care process in primary care are actually fit to use for other purposes such as research and quality of health care indicators, how the original purpose may affect the extent to which the data are fit for another purpose, and the mechanisms behind these effects. In doing so, we want to identify possible sources of bias that are relevant for the use and reuse of these type of data.

METHODS

This paper is based on the authors' experience as users of electronic health records data, as general practitioners, health informatics experts, and health services researchers. It is a product of the discussions they had during the Translational Research and Patient Safety in Europe (TRANSFoRm) project, which was funded by the European Commission and sought to develop, pilot, and evaluate a core information architecture for the learning health system in Europe, based on primary care electronic health records.

RESULTS

We first describe the different stages in the processing of electronic health record data, as well as the different purposes for which these data are used. Given the different data processing steps and purposes, we then discuss the possible mechanisms for each individual data processing step that can generate biased outcomes. We identified 13 possible sources of bias. Four of them are related to the organization of a health care system, whereas some are of a more technical nature.

CONCLUSIONS

There are a substantial number of possible sources of bias; very little is known about the size and direction of their impact. However, anyone that uses or reuses data that were recorded as part of the health care process (such as researchers and clinicians) should be aware of the associated data collection process and environmental influences that can affect the quality of the data. Our stepwise, actor- and purpose-oriented approach may help to identify these possible sources of bias. Unless data quality issues are better understood and unless adequate controls are embedded throughout the data lifecycle, data-driven health care will not live up to its expectations. We need a data quality research agenda to devise the appropriate instruments needed to assess the magnitude of each of the possible sources of bias, and then start measuring their impact. The possible sources of bias described in this paper serve as a starting point for this research agenda.

Practice of ST-segment elevation myocardial infarction care in the Netherlands during four snapshot weeks with the National Cardiovascular Database Registry for Acute Coronary Syndrome

BACKGROUND

Clinical registries provide information on the process of care and patient outcomes, with the potential to improve the quality of patient care. A large Dutch national acute coronary syndrome (ACS) registry is currently lacking. Recently, we initiated the National Cardiovascular Database Registry (NCDR) for ACS in the Netherlands. The purpose of this study was to assess the NCDR ACS registry on feasibility and data completeness during a pilot phase of four snapshot weeks.

METHODS

Between 2013 and 2015, we invited all hospitals in the Netherlands to record a predefined dataset for every patient that was admitted to their hospital with ST-segment elevation myocardial infarction (STEMI). Data were entered in an online case report form. All patient-specific data were encrypted to ensure privacy.

RESULTS

A total of 392 patients were registered in 35 centres. The mean age of the patients was 64 years (SD 13); 8% of patients presented with signs of cardiogenic shock and 11% with an out-of-hospital cardiac arrest. The median time from first medical contact to percutaneous coronary intervention (PCI) was 75 min (IQR 51-108) and this was significantly longer for patients who presented at a non-PCI centre or to a primary care physician. In-hospital and 30-day mortality rates were 5.2% and 7.8%, respectively. The amount of completeness varied, with improved completeness over time.

CONCLUSION

This report shows that a Dutch ACS registry is feasible with respect to STEMI patients. Data completeness, however, was suboptimal. Improved data completeness is warranted for the future.

Pre-hospital management, procedural performance and outcomes for primary percutaneous coronary intervention in ST-elevation myocardial infarction in the Netherlands: Insights from the Dutch cohort of the APPOSITION-III trial

Aim

The aim of this study was to achieve useful insights into pre-hospital management and procedural performance for ST-elevation myocardial infarction (STEMI) in the Netherlands by extrapolating patient characteristics, and procedural and clinical outcomes of the Dutch patient cohort from the APPOSITION-III trial.

Methods

This is a retrospective analysis from the APPOSITION-III trial with respect to the geographical borders of STEMI management. The APPOSITION-III trial was a European registry for the use of the STENTYS self-expandable stent in STEMI patients undergoing primary percutaneous coronary intervention (PPCI). 965 Patients were enrolled mainly in the Netherlands (n = 420, 43.5 % of the overall study population), Germany (n = 165) and France (n = 131). The data from the Dutch cohort were compared with both the overall study population, and the French and German cohorts, respectively, as well as the European Society of Cardiology (ESC) STEMI guidelines.

Results

In this trial there was a wide inter-country variation on symptom-to-balloon time, 165 minutes (120–318) in the Netherlands, 270 minutes (180–650) in Germany and 360 minutes (120–480) in France, respectively. In general, a preload of dual antiplatelet therapy (DAPT) combined with heparin was more often performed in the Dutch and French cohort than in the German cohort. DAPT at discharge was high across the whole APPOSITION-III population. No important differences were seen between the different groups according to the endpoints major adverse cardiac event and stent thrombosis.

Conclusion

In the Dutch cohort of an European multicentre STEMI study (APPOSITION-III trial), the performance in terms of symptom-to-balloon time, and pre-, peri- and post-procedural medical treatment is in line with the recommendations of ESC STEMI guidelines.

STEMI time delays: a clinical perspective : Editorial comment on the article by Verweij et al

STEMI time delays have been introduced as a performance indicator or marker of quality of care. As they are only one part of a very complex medical process, one should be aware of concomitant issues that may be overlooked or even be more important with regard to clinical outcome of STEMI patients. In this overview we try to summarise the most important ones.