Applying probabilistic temporal and multisite data quality control methods to a public health mortality registry in Spain: a systematic approach to quality control of repositories

The challenges and opportunities of continuous data quality improvement for healthcare administration data

Background

Various data quality issues have prevented healthcare administration data from being fully utilized when dealing with problems ranging from COVID-19 contact tracing to controlling healthcare costs.

Objectives

(i) Describe the currently adopted approaches and practices for understanding and improving the quality of healthcare administration data. (ii) Explore the challenges and opportunities to achieve continuous quality improvement for such data.

Materials and Methods

We used a qualitative approach to obtain rich contextual data through semi-structured interviews conducted at a state health agency regarding Medicaid claims and reimbursement data. We interviewed all data stewards knowledgeable about the data quality issues experienced at the agency. The qualitative data were analyzed using the Framework method.

Results

Sixteen themes emerged from our analysis, collected under 4 categories: (i) Defect characteristics: Data defects showed variability, frequently remained obscure, and led to negative outcomes. Detecting and resolving them was often difficult, and the work required often exceeded the organizational boundaries. (ii) Current process and people issues: The agency adopted primarily ad-hoc, manual approaches to resolving data quality problems leading to work frustration. (iii) Challenges: Communication and lack of knowledge about legacy software systems and the data maintained in them constituted challenges, followed by different standards used by various organizations and vendors, and data verification difficulties. (iv) Opportunities: Training, tool support, and standardization of data definitions emerged as immediate opportunities to improve data quality.

Conclusions

Our results can be useful to similar agencies on their journey toward becoming learning health organizations leveraging data assets effectively and efficiently.

Resilient Artificial Intelligence in Health: Synthesis and Research Agenda Toward Next-Generation Trustworthy Clinical Decision Support

Artificial intelligence (AI)-based clinical decision support systems are gaining momentum by relying on a greater volume and variety of secondary use data. However, the uncertainty, variability, and biases in real-world data environments still pose significant challenges to the development of health AI, its routine clinical use, and its regulatory frameworks. Health AI should be resilient against real-world environments throughout its lifecycle, including the training and prediction phases and maintenance during production, and health AI regulations should evolve accordingly. Data quality issues, variability over time or across sites, information uncertainty, human-computer interaction, and fundamental rights assurance are among the most relevant challenges. If health AI is not designed resiliently with regard to these real-world data effects, potentially biased data-driven medical decisions can risk the safety and fundamental rights of millions of people. In this viewpoint, we review the challenges, requirements, and methods for resilient AI in health and provide a research framework to improve the trustworthiness of next-generation AI-based clinical decision support.

Extremely missing numerical data in Electronic Health Records for machine learning can be managed through simple imputation methods considering informative missingness: A comparative of solutions in a COVID-19 mortality case study

BACKGROUND AND OBJECTIVE

Reusing Electronic Health Records (EHRs) for Machine Learning (ML) leads on many occasions to extremely incomplete and sparse tabular datasets, which can hinder the model development processes and limit their performance and generalization. In this study, we aimed to characterize the most effective data imputation techniques and ML models for dealing with highly missing numerical data in EHRs, in the case where only a very limited number of data are complete, as opposed to the usual case of having a reduced number of missing values.

METHODS

We used a case study including full blood count laboratory data, demographic and survival data in the context of COVID-19 hospital admissions and evaluated 30 processing pipelines combining imputation methods with ML classifiers. The imputation methods included missing mask, translation and encoding, mean imputation, k-nearest neighbors' imputation, Bayesian ridge regression imputation and generative adversarial imputation networks. The classifiers included k-nearest neighbors, logistic regression, random forest, gradient boosting and deep multilayer perceptron.

RESULTS

Our results suggest that in the presence of highly missing data, combining translation and encoding imputation-which considers informative missingness-with tree ensemble classifiers-random forest and gradient boosting-is a sensible choice when aiming to maximize performance, in terms of area under curve.

CONCLUSIONS

Based on our findings, we recommend the consideration of this imputer-classifier configuration when constructing models in the presence of extremely incomplete numerical data in EHR.

Subphenotyping of COVID-19 patients at pre-admission towards anticipated severity stratification: an analysis of 778 692 Mexican patients through an age-sex unbiased meta-clustering technique

Background

The COVID-19 pandemic has led to an unprecedented global health care challenge for both medical institutions and researchers. Recognizing different COVID-19 subphenotypes—the division of populations of patients into more meaningful subgroups driven by clinical features—and their severity characterization may assist clinicians during the clinical course, the vaccination process, research efforts, the surveillance system, and the allocation of limited resources.

Objective

We aimed to discover age-sex unbiased COVID-19 patient subphenotypes based on easily available phenotypical data before admission, such as pre-existing comorbidities, lifestyle habits, and demographic features, to study the potential early severity stratification capabilities of the discovered subgroups through characterizing their severity patterns, including prognostic, intensive care unit (ICU), and morbimortality outcomes.

Methods

We used the Mexican Government COVID-19 open data, including 778,692 SARS-CoV-2 population-based patient-level data as of September 2020. We applied a meta-clustering technique that consists of a 2-stage clustering approach combining dimensionality reduction (ie, principal components analysis and multiple correspondence analysis) and hierarchical clustering using the Ward minimum variance method with Euclidean squared distance.

Results

In the independent age-sex clustering analyses, 56 clusters supported 11 clinically distinguishable meta-clusters (MCs). MCs 1-3 showed high recovery rates (90.27%-95.22%), including healthy patients of all ages, children with comorbidities and priority in receiving medical resources (ie, higher rates of hospitalization, intubation, and ICU admission) compared with other adult subgroups that have similar conditions, and young obese smokers. MCs 4-5 showed moderate recovery rates (81.30%-82.81%), including patients with hypertension or diabetes of all ages and obese patients with pneumonia, hypertension, and diabetes. MCs 6-11 showed low recovery rates (53.96%-66.94%), including immunosuppressed patients with high comorbidity rates, patients with chronic kidney disease with a poor survival length and probability of recovery, older smokers with chronic obstructive pulmonary disease, older adults with severe diabetes and hypertension, and the oldest obese smokers with chronic obstructive pulmonary disease and mild cardiovascular disease. Group outcomes conformed to the recent literature on dedicated age-sex groups. Mexican states and several types of clinical institutions showed relevant heterogeneity regarding severity, potentially linked to socioeconomic or health inequalities.

Conclusions

The proposed 2-stage cluster analysis methodology produced a discriminative characterization of the sample and explainability over age and sex. These results can potentially help in understanding the clinical patient and their stratification for automated early triage before further tests and laboratory results are available and even in locations where additional tests are not available or to help decide resource allocation among vulnerable subgroups such as to prioritize vaccination or treatments.

A data driven learning approach for the assessment of data quality

Background

Data quality assessment is important but complex and task dependent. Identifying suitable measurement methods and reference ranges for assessing their results is challenging. Manually inspecting the measurement results and current data driven approaches for learning which results indicate data quality issues have considerable limitations, e.g. to identify task dependent thresholds for measurement results that indicate data quality issues.

Objectives

To explore the applicability and potential benefits of a data driven approach to learn task dependent knowledge about suitable measurement methods and assessment of their results. Such knowledge could be useful for others to determine whether a local data stock is suitable for a given task.

Methods

We started by creating artificial data with previously defined data quality issues and applied a set of generic measurement methods on this data (e.g. a method to count the number of values in a certain variable or the mean value of the values). We trained decision trees on exported measurement methods’ results and corresponding outcome data (data that indicated the data’s suitability for a use case). For evaluation, we derived rules for potential measurement methods and reference values from the decision trees and compared these regarding their coverage of the true data quality issues artificially created in the dataset. Three researchers independently derived these rules. One with knowledge about present data quality issues and two without.

Results

Our self-trained decision trees were able to indicate rules for 12 of 19 previously defined data quality issues. Learned knowledge about measurement methods and their assessment was complementary to manual interpretation of measurement methods’ results.

Conclusions

Our data driven approach derives sensible knowledge for task dependent data quality assessment and complements other current approaches. Based on labeled measurement methods’ results as training data, our approach successfully suggested applicable rules for checking data quality characteristics that determine whether a dataset is suitable for a given task.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-021-01656-x.

Quality of Hospital Electronic Health Record (EHR) Data Based on the International Consortium for Health Outcomes Measurement (ICHOM) in Heart Failure: Pilot Data Quality Assessment Study

Background

There is increasing recognition that health care providers need to focus attention, and be judged against, the impact they have on the health outcomes experienced by patients. The measurement of health outcomes as a routine part of clinical documentation is probably the only scalable way of collecting outcomes evidence, since secondary data collection is expensive and error-prone. However, there is uncertainty about whether routinely collected clinical data within electronic health record (EHR) systems includes the data most relevant to measuring and comparing outcomes and if those items are collected to a good enough data quality to be relied upon for outcomes assessment, since several studies have pointed out significant issues regarding EHR data availability and quality.

Objective

In this paper, we first describe a practical approach to data quality assessment of health outcomes, based on a literature review of existing frameworks for quality assessment of health data and multistakeholder consultation. Adopting this approach, we performed a pilot study on a subset of 21 International Consortium for Health Outcomes Measurement (ICHOM) outcomes data items from patients with congestive heart failure.

Methods

All available registries compatible with the diagnosis of heart failure within an EHR data repository of a general hospital (142,345 visits and 12,503 patients) were extracted and mapped to the ICHOM format. We focused our pilot assessment on 5 commonly used data quality dimensions: completeness, correctness, consistency, uniqueness, and temporal stability.

Results

We found high scores (>95%) for the consistency, completeness, and uniqueness dimensions. Temporal stability analyses showed some changes over time in the reported use of medication to treat heart failure, as well as in the recording of past medical conditions. Finally, the investigation of data correctness suggested several issues concerning the characterization of missing data values. Many of these issues appear to be introduced while mapping the IMASIS-2 relational database contents to the ICHOM format, as the latter requires a level of detail that is not explicitly available in the coded data of an EHR.

Conclusions

Overall, results of this pilot study revealed good data quality for the subset of heart failure outcomes collected at the Hospital del Mar. Nevertheless, some important data errors were identified that were caused by fundamentally different data collection practices in routine clinical care versus research, for which the ICHOM standard set was originally developed. To truly examine to what extent hospitals today are able to routinely collect the evidence of their success in achieving good health outcomes, future research would benefit from performing more extensive data quality assessments, including all data items from the ICHOM standards set and across multiple hospitals.

The impact of data quality assurance and control solutions on the completeness, accuracy, and consistency of data in a national spinal cord injury registry of Iran (NSCIR-IR)

STUDY DESIGN

Descriptive study.

OBJECTIVE

This study aimed to develop and evaluate a systematic arrangement for improvement and monitoring of data quality of the National Spinal Cord (and Column) Injury Registry of Iran (NSCIR-IR)-a multicenter hospital-based registry.

SETTING

SCI community in Iran.

METHODS

Quality assurance and quality control were the primary objectives in improving overall quality of data that were considered in designing a paper-based and computerized case report. To prevent incorrect data entry, we implemented several validation algorithms, including 70 semantic rules, 18 syntactic rules, seven temporal rules, and 13 rules for acceptable value range. Qualified and trained staff members were also employed to review and identify any defect, inaccuracy, or inconsistency in the data to improve data quality. A set of functions were implemented in the software to cross-validate, and feedback on data was provided by reviewers and registrars.

RESULTS

Socio-demographic data items were 100% complete, except for national ID and education level, which were 97% and 92.3% complete, respectively. Completeness of admission data and emergency medical services data were 100% except for arrival and transfer time (99.4%) and oxygen saturation (48.9%). Evaluation of data received from two centers located in Tehran proved to be 100% accurate following validation by quality reviewers. All data was also found to be 100% consistent.

CONCLUSIONS

This approach to quality assurance and consistency validation proved to be effective. Our solutions resulted in a significant decrease in the number of missing data.

Deep ensemble multitask classification of emergency medical call incidents combining multimodal data improves emergency medical dispatch

The objective of this work was to develop a predictive model to aid non-clinical dispatchers to classify emergency medical call incidents by their life-threatening level (yes/no), admissible response delay (undelayable, minutes, hours, days) and emergency system jurisdiction (emergency system/primary care) in real time. We used a total of 1 244 624 independent incidents from the Valencian emergency medical dispatch service in Spain, compiled in retrospective from 2009 to 2012, including clinical features, demographics, circumstantial factors and free text dispatcher observations. Based on them, we designed and developed DeepEMC², a deep ensemble multitask model integrating four subnetworks: three specialized to context, clinical and text data, respectively, and another to ensemble the former. The four subnetworks are composed in turn by multi-layer perceptron modules, bidirectional long short-term memory units and a bidirectional encoding representations from transformers module. DeepEMC² showed a macro F1-score of 0.759 in life-threatening classification, 0.576 in admissible response delay and 0.757 in emergency system jurisdiction. These results show a substantial performance increase of 12.5 %, 17.5 % and 5.1 %, respectively, with respect to the current in-house triage protocol of the Valencian emergency medical dispatch service. Besides, DeepEMC² significantly outperformed a set of baseline machine learning models, including naive bayes, logistic regression, random forest and gradient boosting (α = 0.05). Hence, DeepEMC² is able to: 1) capture information present in emergency medical calls not considered by the existing triage protocol, and 2) model complex data dependencies not feasible by the tested baseline models. Likewise, our results suggest that most of this unconsidered information is present in the free text dispatcher observations. To our knowledge, this study describes the first deep learning model undertaking emergency medical call incidents classification. Its adoption in medical dispatch centers would potentially improve emergency dispatch processes, resulting in a positive impact in patient wellbeing and health services sustainability.

Bias Analysis on Public X-Ray Image Datasets of Pneumonia and COVID-19 Patients

Chest X-ray images are useful for early COVID-19 diagnosis with the advantage that X-ray devices are already available in health centers and images are obtained immediately. Some datasets containing X-ray images with cases (pneumonia or COVID-19) and controls have been made available to develop machine-learning-based methods to aid in diagnosing the disease. However, these datasets are mainly composed of different sources coming from pre-COVID-19 datasets and COVID-19 datasets. Particularly, we have detected a significant bias in some of the released datasets used to train and test diagnostic systems, which might imply that the results published are optimistic and may overestimate the actual predictive capacity of the techniques proposed. In this article, we analyze the existing bias in some commonly used datasets and propose a series of preliminary steps to carry out before the classic machine learning pipeline in order to detect possible biases, to avoid them if possible and to report results that are more representative of the actual predictive power of the methods under analysis.

Potential limitations in COVID-19 machine learning due to data source variability: A case study in the nCov2019 dataset

OBJECTIVE

The lack of representative coronavirus disease 2019 (COVID-19) data is a bottleneck for reliable and generalizable machine learning. Data sharing is insufficient without data quality, in which source variability plays an important role. We showcase and discuss potential biases from data source variability for COVID-19 machine learning.

MATERIALS AND METHODS

We used the publicly available nCov2019 dataset, including patient-level data from several countries. We aimed to the discovery and classification of severity subgroups using symptoms and comorbidities.

RESULTS

Cases from the 2 countries with the highest prevalence were divided into separate subgroups with distinct severity manifestations. This variability can reduce the representativeness of training data with respect the model target populations and increase model complexity at risk of overfitting.

CONCLUSIONS

Data source variability is a potential contributor to bias in distributed research networks. We call for systematic assessment and reporting of data source variability and data quality in COVID-19 data sharing, as key information for reliable and generalizable machine learning.

EHRtemporalVariability: delineating temporal data-set shifts in electronic health records

BACKGROUND

Temporal variability in health-care processes or protocols is intrinsic to medicine. Such variability can potentially introduce dataset shifts, a data quality issue when reusing electronic health records (EHRs) for secondary purposes. Temporal data-set shifts can present as trends, as well as abrupt or seasonal changes in the statistical distributions of data over time. The latter are particularly complicated to address in multimodal and highly coded data. These changes, if not delineated, can harm population and data-driven research, such as machine learning. Given that biomedical research repositories are increasingly being populated with large sets of historical data from EHRs, there is a need for specific software methods to help delineate temporal data-set shifts to ensure reliable data reuse.

RESULTS

EHRtemporalVariability is an open-source R package and Shiny app designed to explore and identify temporal data-set shifts. EHRtemporalVariability estimates the statistical distributions of coded and numerical data over time; projects their temporal evolution through non-parametric information geometric temporal plots; and enables the exploration of changes in variables through data temporal heat maps. We demonstrate the capability of EHRtemporalVariability to delineate data-set shifts in three impact case studies, one of which is available for reproducibility.

CONCLUSIONS

EHRtemporalVariability enables the exploration and identification of data-set shifts, contributing to the broad examination and repurposing of large, longitudinal data sets. Our goal is to help ensure reliable data reuse for a wide range of biomedical data users. EHRtemporalVariability is designed for technical users who are programmatically utilizing the R package, as well as users who are not familiar with programming via the Shiny user interface.Availability: https://github.com/hms-dbmi/EHRtemporalVariability/Reproducible vignette: https://cran.r-project.org/web/packages/EHRtemporalVariability/vignettes/EHRtemporalVariability.htmlOnline demo: http://ehrtemporalvariability.upv.es/.

Understanding and detecting defects in healthcare administration data: Toward higher data quality to better support healthcare operations and decisions

OBJECTIVE

Development of systematic approaches for understanding and assessing data quality is becoming increasingly important as the volume and utilization of health data steadily increases. In this study, a taxonomy of data defects was developed and utilized when automatically detecting defects to assess Medicaid data quality maintained by one of the states in the United States.

MATERIALS AND METHODS

There were more than 2.23 million rows and 32 million cells in the Medicaid data examined. The taxonomy was developed through document review, descriptive data analysis, and literature review. A software program was created to automatically detect defects by using a set of constraints whose development was facilitated by the taxonomy.

RESULTS

Five major categories and seventeen subcategories of defects were identified. The major categories are missingness, incorrectness, syntax violation, semantic violation, and duplicity. More than 3 million defects were detected indicating substantial problems with data quality. Defect density exceeded 10% in five tables. The majority of the data defects belonged to format mismatch, invalid code, dependency-contract violation, and implausible value types. Such contextual knowledge can support prioritized quality improvement initiatives for the Medicaid data studied.

CONCLUSIONS

This research took the initial steps to understand the types of data defects and detect defects in large healthcare datasets. The results generally suggest that healthcare organizations can potentially benefit from focusing on data quality improvement. For those purposes, the taxonomy developed and the approach followed in this study can be adopted.

Data-driven discovery of changes in clinical code usage over time: a case-study on changes in cardiovascular disease recording in two English electronic health records databases (2001-2015)

OBJECTIVES

To demonstrate how data-driven variability methods can be used to identify changes in disease recording in two English electronic health records databases between 2001 and 2015.

DESIGN

Repeated cross-sectional analysis that applied data-driven temporal variability methods to assess month-by-month changes in routinely collected medical data. A measure of difference between months was calculated based on joint distributions of age, gender, socioeconomic status and recorded cardiovascular diseases. Distances between months were used to identify temporal trends in data recording.

SETTING

400 English primary care practices from the Clinical Practice Research Datalink (CPRD GOLD) and 451 hospital providers from the Hospital Episode Statistics (HES).

MAIN OUTCOMES

The proportion of patients (CPRD GOLD) and hospital admissions (HES) with a recorded cardiovascular disease (CPRD GOLD: coronary heart disease, heart failure, peripheral arterial disease, stroke; HES: International Classification of Disease codes I20-I69/G45).

RESULTS

Both databases showed gradual changes in cardiovascular disease recording between 2001 and 2008. The recorded prevalence of included cardiovascular diseases in CPRD GOLD increased by 47%-62%, which partially reversed after 2008. For hospital records in HES, there was a relative decrease in angina pectoris (-34.4%) and unspecified stroke (-42.3%) over the same time period, with a concomitant increase in chronic coronary heart disease (+14.3%). Multiple abrupt changes in the use of myocardial infarction codes in hospital were found in March/April 2010, 2012 and 2014, possibly linked to updates of clinical coding guidelines.

CONCLUSIONS

Identified temporal variability could be related to potentially non-medical causes such as updated coding guidelines. These artificial changes may introduce temporal correlation among diagnoses inferred from routine data, violating the assumptions of frequently used statistical methods. Temporal variability measures provide an objective and robust technique to identify, and subsequently account for, those changes in electronic health records studies without any prior knowledge of the data collection process.

Subgrouping Factors Influencing Migraine Intensity in Women: A Semi-automatic Methodology Based on Machine Learning and Information Geometry

BACKGROUND

Migraine is a heterogeneous condition with multiple clinical manifestations. Machine learning algorithms permit the identification of population groups, providing analytical advantages over other modeling techniques.

OBJECTIVE

The aim of this study was to analyze critical features that permit the differentiation of subgroups of patients with migraine according to the intensity and frequency of attacks by using machine learning algorithms.

METHODS

Sixty-seven women with migraine participated. Clinical features of migraine, related disability (Migraine Disability Assessment Scale), anxiety/depressive levels (Hospital Anxiety and Depression Scale), anxiety state/trait levels (State-Trait Anxiety Inventory), and pressure pain thresholds (PPTs) over the temporalis, neck, second metacarpal, and tibialis anterior were collected. Physical examination included the flexion-rotation test, cervical range of cervical motion, forward head position while sitting and standing, passive accessory intervertebral movements (PAIVMs) with headache reproduction, and joint positioning sense error. Subgrouping was based on machine learning algorithms by using the nearest neighbors algorithm, multisource variability assessment, and random forest model.

RESULTS

For migraine intensity, group 2 (women with a regular migraine headache intensity score of 7 on an 11-point Numeric Pain Rating Scale [where 0 = no pain and 10 = maximum pain]) were younger and had lower joint positioning sense error in cervical rotation, greater cervical mobility in rotation and flexion, lower flexion-rotation test scores, positive PAIVMs reproducing migraine, normal PPTs over the tibialis anterior, shorter migraine history, and lower cranio-vertebral angles while standing than the remaining migraine intensity subgroups. The most discriminative variable was the flexion-rotation test score of the symptomatic side. For migraine frequency, no model was able to identify differences between groups (ie, patients with episodic or chronic migraine).

CONCLUSIONS

A subgroup of women with migraine who had common migraine intensity was identified with machine learning algorithms.

Examining the impact of data quality and completeness of electronic health records on predictions of patients' risks of cardiovascular disease

OBJECTIVE

To assess the extent of variation of data quality and completeness of electronic health records and impact on the robustness of risk predictions of incident cardiovascular disease (CVD) using a risk prediction tool that is based on routinely collected data (QRISK3).

DESIGN

Longitudinal cohort study.

SETTINGS

392 general practices (including 3.6 million patients) linked to hospital admission data.

METHODS

Variation in data quality was assessed using Sáez's stability metrics quantifying outlyingness of each practice. Statistical frailty models evaluated whether accuracy of QRISK3 predictions on individual predictions and effects of overall risk factors (linear predictor) varied between practices.

RESULTS

There was substantial heterogeneity between practices in CVD incidence unaccounted for by QRISK3. In the lowest quintile of statistical frailty, a QRISK3 predicted risk of 10 % for female was in a range between 7.1 % and 9.0 % when incorporating practice variability into the statistical frailty models; for the highest quintile, this was 10.9%-16.4%. Data quality (using Saez metrics) and completeness were comparable across different levels of statistical frailty. For example, recording of missing information on ethnicity was 55.7 %, 62.7 %, 57.8 %, 64.8 % and 62.1 % for practices from lowest to highest quintiles of statistical frailty respectively. The effects of risk factors did not vary between practices with little statistical variation of beta coefficients.

CONCLUSIONS

The considerable unmeasured heterogeneity in CVD incidence between practices was not explained by variations in data quality or effects of risk factors. QRISK3 risk prediction should be supplemented with clinical judgement and evidence of additional risk factors.

What can millions of laboratory test results tell us about the temporal aspect of data quality? Study of data spanning 17 years in a clinical data warehouse

OBJECTIVE

To identify common temporal evolution profiles in biological data and propose a semi-automated method to these patterns in a clinical data warehouse (CDW).

MATERIALS AND METHODS

We leveraged the CDW of the European Hospital Georges Pompidou and tracked the evolution of 192 biological parameters over a period of 17 years (for 445,000 + patients, and 131 million laboratory test results).

RESULTS

We identified three common profiles of evolution: discretization, breakpoints, and trends. We developed computational and statistical methods to identify these profiles in the CDW. Overall, of the 192 observed biological parameters (87,814,136 values), 135 presented at least one evolution. We identified breakpoints in 30 distinct parameters, discretizations in 32, and trends in 79.

DISCUSSION AND CONCLUSION

our method allowed the identification of several temporal events in the data. Considering the distribution over time of these events, we identified probable causes for the observed profiles: instruments or software upgrades and changes in computation formulas. We evaluated the potential impact for data reuse. Finally, we formulated recommendations to enable safe use and sharing of biological data collection to limit the impact of data evolution in retrospective and federated studies (e.g. the annotation of laboratory parameters presenting breakpoints or trends).

Temporal variability analysis reveals biases in electronic health records due to hospital process reengineering interventions over seven years

OBJECTIVE

To evaluate the effects of Process-Reengineering interventions on the Electronic Health Records (EHR) of a hospital over 7 years.

MATERIALS AND METHODS

Temporal Variability Assessment (TVA) based on probabilistic data quality assessment was applied to the historic monthly-batched admission data of Hospital La Fe Valencia, Spain from 2010 to 2016. Routine healthcare data with a complete EHR was expanded by processed variables such as the Charlson Comorbidity Index.

RESULTS

Four Process-Reengineering interventions were detected by quantifiable effects on the EHR: (1) the hospital relocation in 2011 involved progressive reduction of admissions during the next four months, (2) the hospital services re-configuration incremented the number of inter-services transfers, (3) the care-services re-distribution led to transfers between facilities (4) the assignment to the hospital of a new area with 80,000 patients in 2015 inspired the discharge to home for follow up and the update of the pre-surgery planned admissions protocol that produced a significant decrease of the patient length of stay.

DISCUSSION

TVA provides an indicator of the effect of process re-engineering interventions on healthcare practice. Evaluating the effect of facilities' relocation and increment of citizens (findings 1, 3-4), the impact of strategies (findings 2-3), and gradual changes in protocols (finding 4) may help on the hospital management by optimizing interventions based on their effect on EHRs or on data reuse.

CONCLUSIONS

The effects on hospitals EHR due to process re-engineering interventions can be evaluated using the TVA methodology. Being aware of conditioned variations in EHR is of the utmost importance for the reliable reuse of routine hospitalization data.

Veracity in big data: How good is good enough

Veracity, one of the five V's used to describe big data, has received attention when it comes to using electronic medical record data for research purposes. In this perspective article, we discuss the idea of data veracity and associated concepts as it relates to the use of electronic medical record data and administrative data in research. We discuss the idea that electronic medical record data are "good enough" for clinical practice and, as such, are "good enough" for certain applications. We then propose three primary issues to attend to when establishing data veracity: data provenance, cross validation, and context.

Humanitarian health computing using artificial intelligence and social media: A narrative literature review

INTRODUCTION

According to the World Health Organization (WHO), over 130 million people are in constant need of humanitarian assistance due to natural disasters, disease outbreaks, and conflicts, among other factors. These health crises can compromise the resilience of healthcare systems, which are essential for achieving the health objectives of the sustainable development goals (SDGs) of the United Nations (UN). During a humanitarian health crisis, rapid and informed decision making is required. This is often challenging due to information scarcity, limited resources, and strict time constraints. Moreover, the traditional approach to digital health development, which involves a substantial requirement analysis, a feasibility study, and deployment of technology, is ill-suited for many crisis contexts. The emergence of Web 2.0 technologies and social media platforms in the past decade, such as Twitter, has created a new paradigm of massive information and misinformation, in which new technologies need to be developed to aid rapid decision making during humanitarian health crises.

OBJECTIVE

Humanitarian health crises increasingly require the analysis of massive amounts of information produced by different sources, such as social media content, and, hence, they are a prime case for the use of artificial intelligence (AI) techniques to help identify relevant information and make it actionable. To identify challenges and opportunities for using AI in humanitarian health crises, we reviewed the literature on the use of AI techniques to process social media.

METHODOLOGY

We performed a narrative literature review aimed at identifying examples of the use of AI in humanitarian health crises. Our search strategy was designed to get a broad overview of the different applications of AI in a humanitarian health crisis and their challenges. A total of 1459 articles were screened, and 24 articles were included in the final analysis.

RESULTS

Successful case studies of AI applications in a humanitarian health crisis have been reported, such as for outbreak detection. A commonly shared concern in the reviewed literature is the technical challenge of analyzing large amounts of data in real time. Data interoperability, which is essential to data sharing, is also a barrier with regard to the integration of online and traditional data sources. Human and organizational aspects that might be key factors for the adoption of AI and social media remain understudied. There is also a publication bias toward high-income countries, as we identified few examples in low-income countries. Further, we did not identify any examples of certain types of major crisis, such armed conflicts, in which misinformation might be more common.

CONCLUSIONS

The feasibility of using AI to extract valuable information during a humanitarian health crisis is proven in many cases. There is a lack of research on how to integrate the use of AI into the work-flow and large-scale deployments of humanitarian aid during a health crisis.

Registry Data Coordinator (RDC): a Proper Accessible Strategy for Improving Road Traffic Injury (RTI) Hospital Based Trauma Registry Systems in Developing Countries and Low Income Countries

INTRODUCTION

Evidence suggested that a significant level of trauma mortality can be prevented using registry system.

AIM

This study aimed to improve Kashan Hospital Based Trauma Registry System (KHBTRS) for Road Traffic Injury (RTI).

MATERIAL AND METHODS

After conducting focus group discussion absence of minimum data set (MDS) and poor data collection process (DCP) were identified as main problems for KHBTRS- RTI. Proposed MDS were surveyed by 20 experts of trauma research center of throughout the Iran. Then approved MDS applied for trauma registry system data base in form of SQL. DCP were reform from prospective data collection (review of medical record) to concurrent (through the interview) approach.

RESULTS

Most of participants for MDS approval belonged to clinical group 13(65%). 146 MDS in eighteen main categories were proposed for RTI. The maximum score for each MDS main categories were attributed to body parts injured 220 (100%) and patient vital signs 139 (99.29%) respectively. Pilot testing of KHBTRS- RTI database of 50 (50%) riders indicated fully completeness 50 (100%) for concurrent approach. It was concluded that based on experts' viewpoints MDS relating to injury nature and place of occurrence have more priority in comparisons to MDS relating to causes of injury. It may attribute to health care providers focus on clinical care and treatment.

CONCLUSION

It was concluded that based on experts' viewpoints MDS relating to injury nature and place of occurrence have more priority in comparisons to MDS relating to RTI prevention; it may attribute to health care providers focus on clinical care and treatment. To develop injury interventions based on given data, recruitment of professionals as registry data coordinator with specific job description to collect and advocacy of injury external causes data seems imperative.

Secondary Use of Patient Data: Review of the Literature Published in 2016

Objectives: To summarize recent research and emerging trends in the area of secondary use of healthcare data, and to present the best papers published in this field, selected to appear in the 2017 edition of the IMIA Yearbook. Methods: A literature review of articles published in 2016 and related to secondary use of healthcare data was performed using two bibliographic databases. From this search, 941 papers were identified. The section editors independently reviewed the papers for relevancy and impact, resulting in a consensus list of 14 candidate best papers. External reviewers examined each of the candidate best papers and the final selection was made by the editorial board of the Yearbook. Results: From the 941 retrieved papers, the selection process resulted in four best papers. These papers discuss data quality concerns, issues in preserving privacy of patients in shared datasets, and methods of decision support when consuming large amounts of raw electronic health record (EHR) data. Conclusion: In 2016, a significant effort was put into the development of new systems which aim to avoid significant human understanding and pre-processing of healthcare data, though this is still only an emerging area of research. The value of temporal relationships between data received significant study, as did effective information sharing while preserving patient privacy.