Digital Health Data Quality Issues: Systematic Review

BACKGROUND

The promise of digital health is principally dependent on the ability to electronically capture data that can be analyzed to improve decision-making. However, the ability to effectively harness data has proven elusive, largely because of the quality of the data captured. Despite the importance of data quality (DQ), an agreed-upon DQ taxonomy evades literature. When consolidated frameworks are developed, the dimensions are often fragmented, without consideration of the interrelationships among the dimensions or their resultant impact.

OBJECTIVE

The aim of this study was to develop a consolidated digital health DQ dimension and outcome (DQ-DO) framework to provide insights into 3 research questions: What are the dimensions of digital health DQ? How are the dimensions of digital health DQ related? and What are the impacts of digital health DQ?

METHODS

Following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, a developmental systematic literature review was conducted of peer-reviewed literature focusing on digital health DQ in predominately hospital settings. A total of 227 relevant articles were retrieved and inductively analyzed to identify digital health DQ dimensions and outcomes. The inductive analysis was performed through open coding, constant comparison, and card sorting with subject matter experts to identify digital health DQ dimensions and digital health DQ outcomes. Subsequently, a computer-assisted analysis was performed and verified by DQ experts to identify the interrelationships among the DQ dimensions and relationships between DQ dimensions and outcomes. The analysis resulted in the development of the DQ-DO framework.

RESULTS

The digital health DQ-DO framework consists of 6 dimensions of DQ, namely accessibility, accuracy, completeness, consistency, contextual validity, and currency; interrelationships among the dimensions of digital health DQ, with consistency being the most influential dimension impacting all other digital health DQ dimensions; 5 digital health DQ outcomes, namely clinical, clinician, research-related, business process, and organizational outcomes; and relationships between the digital health DQ dimensions and DQ outcomes, with the consistency and accessibility dimensions impacting all DQ outcomes.

CONCLUSIONS

The DQ-DO framework developed in this study demonstrates the complexity of digital health DQ and the necessity for reducing digital health DQ issues. The framework further provides health care executives with holistic insights into DQ issues and resultant outcomes, which can help them prioritize which DQ-related problems to tackle first.

Challenges Encountered and Lessons Learned When Using a Novel Anonymised Linked Dataset of Health and Social Care Records for Public Health Intelligence: The Sussex Integrated Dataset

Background: In the United Kingdom National Health Service (NHS), digital transformation programmes have resulted in the creation of pseudonymised linked datasets of patient-level medical records across all NHS and social care services. In the Southeast England counties of East and West Sussex, public health intelligence analysts based in local authorities (LAs) aimed to use the newly created “Sussex Integrated Dataset” (SID) for identifying cohorts of patients who are at risk of early onset multiple long-term conditions (MLTCs). Analysts from the LAs were among the first to have access to this new dataset. Methods: Data access was assured as the analysts were employed within joint data controller organisations and logged into the data via virtual machines following approval of a data access request. Analysts examined the demographics and medical history of patients against multiple external sources, identifying data quality issues and developing methods to establish true values for cases with multiple conflicting entries. Service use was plotted over timelines for individual patients. Results: Early evaluation of the data revealed multiple conflicting within-patient values for age, sex, ethnicity and date of death. This was partially resolved by creating a “demographic milestones” table, capturing demographic details for each patient for each year of the data available in the SID. Older data (≥5 y) was found to be sparse in events and diagnoses. Open-source code lists for defining long-term conditions were poor at identifying the expected number of patients, and bespoke code lists were developed by hand and validated against other sources of data. At the start, the age and sex distributions of patients submitted by GP practices were substantially different from those published by NHS Digital, and errors in data processing were identified and rectified. Conclusions: While new NHS linked datasets appear a promising resource for tracking multi-service use, MLTCs and health inequalities, substantial investment in data analysis and data architect time is necessary to ensure high enough quality data for meaningful analysis. Our team made conceptual progress in identifying the skills needed for programming analyses and understanding the types of questions which can be asked and answered reliably in these datasets.

Timing errors and temporal uncertainty in clinical databases-A narrative review

A firm concept of time is essential for establishing causality in a clinical setting. Review of critical incidents and generation of study hypotheses require a robust understanding of the sequence of events but conducting such work can be problematic when timestamps are recorded by independent and unsynchronized clocks. Most clinical models implicitly assume that timestamps have been measured accurately and precisely, but this custom will need to be re-evaluated if our algorithms and models are to make meaningful use of higher frequency physiological data sources. In this narrative review we explore factors that can result in timestamps being erroneously recorded in a clinical setting, with particular focus on systems that may be present in a critical care unit. We discuss how clocks, medical devices, data storage systems, algorithmic effects, human factors, and other external systems may affect the accuracy and precision of recorded timestamps. The concept of temporal uncertainty is introduced, and a holistic approach to timing accuracy, precision, and uncertainty is proposed. This quantitative approach to modeling temporal uncertainty provides a basis to achieve enhanced model generalizability and improved analytical outcomes.

Multi-perspective process mining for emergency process

BACKGROUND

Multi-perspective process mining is an analytical approach that uses data to gain objective insights and uncover hidden problems in business processes from multiple perspectives.

OBJECTIVE

In this paper, we apply multi-perspective process mining techniques in the emergency process through a goal-driven performance evaluation method in order to understand and diagnose the timeliness of the emergency process.

METHODS

Unstructured and multi-disciplinary emergency data is analyzed by following Goal-Question-Feature-Indicator (GQFI) method. In this paper, the GQFI method is extended with perspectives, and the insights in the enriched event data are examined by a decision tree model. All of them are applied in a systematic way in relation to the goal of assessing and improving the emergency process in a university hospital.

RESULTS

We detected the deviations (e.g., skipping the triage and consultation request steps) and two bottlenecks in the emergency process. Among the suggestions for improving the process, are performing defensive medicine in a harmless manner, classification of the emergency services, ensuring triage step is applied to all patients and effective usage of the call system application in consultation activities.

CONCLUSION

The results of this study showed that goal-oriented multi-perspective process mining is effective in identifying process improvements in emergency services.

Conversion to an electronic missing medication request system at an academic medical center

Missing medications can negatively contribute to the financial and operational workflows of pharmacy departments and add medication safety challenges. The missing medication request (MMR) system at the study institution converted to entirely electronic in June 2018 from a hybrid electronic system. This study evaluated 4-week periods pre- and post-conversion. The objective of this study was to evaluate the impact of conversion to an electronic MMR system on the quantity of requests received at an academic medical center. The average daily number of MMR's decreased from the pre-conversion group to the post-conversion group (1.77 (±0.16) vs 1.48 (±0.17), p < 0.001). During post-conversion, the median triage time was 8 min [3 min-19 min], pharmacists triaged 62.4% of requests, and 29.6% of requests were declined. Conversion to an electronic MMR system represents one solution to decreasing missing medications. Future studies are needed to evaluate the financial, operational, and medication safety impact of conversion.

A Systematic Review of Recommendations of Long-Term Strategies for Researchers Using Data Science Techniques

The impact of the strategies that researchers follow to publish or produce scientific content can have a long-term impact. Identifying which strategies are most influential in the future has been attracting increasing attention in the literature. In this study, we present a systematic review of recommendations of long-term strategies in research analytics and their implementation methodologies. The objective is to present an overview from 2002 to 2018 on the development of this topic, including trends, and addressed contexts. The central objective is to identify data-oriented approaches to learn long-term research strategies, especially in process mining. We followed a protocol for systematic reviews for the engineering area in a structured and respectful manner. The results show the need for studies that generate more specific recommendations based on data mining. This outcome leaves open research opportunities from two particular perspectives—applying methodologies involving process mining for the context of research analytics and the feasibility study on long-term strategies using data science techniques.

Process Mining-Supported Emergency Room Process Performance Indicators

Emergency room processes are often exposed to the risk of unexpected factors, and process management based on performance measurements is required due to its connectivity to the quality of care. Regarding this, there have been several attempts to propose a method to analyze the emergency room processes. This paper proposes a framework for process performance indicators utilized in emergency rooms. Based on the devil’s quadrangle, i.e., time, cost, quality, and flexibility, the paper suggests multiple process performance indicators that can be analyzed using clinical event logs and verify them with a thorough discussion with clinical experts in the emergency department. A case study is conducted with the real-life clinical data collected from a tertiary hospital in Korea to validate the proposed method. The case study demonstrated that the proposed indicators are well applied using the clinical data, and the framework is capable of understanding emergency room processes’ performance.

The assessment of data quality issues for process mining in healthcare using Medical Information Mart for Intensive Care III, a freely available e-health record database

There is a growing body of literature on process mining in healthcare. Process mining of electronic health record systems could give benefit into better understanding of the actual processes happened in the patient treatment, from the event log of the hospital information system. Researchers report issues of data access approval, anonymisation constraints, and data quality. One solution to progress methodology development is to use a high-quality, freely available research dataset such as Medical Information Mart for Intensive Care III, a critical care database which contains the records of 46,520 intensive care unit patients over 12 years. Our article aims to (1) explore data quality issues for healthcare process mining using Medical Information Mart for Intensive Care III, (2) provide a structured assessment of Medical Information Mart for Intensive Care III data quality and challenge for process mining, and (3) provide a worked example of cancer treatment as a case study of process mining using Medical Information Mart for Intensive Care III to illustrate an approach and solution to data quality challenges. The electronic health record software was upgraded partway through the period over which data was collected and we use this event to explore the link between electronic health record system design and resulting process models.

Veracity in big data: How good is good enough

Veracity, one of the five V's used to describe big data, has received attention when it comes to using electronic medical record data for research purposes. In this perspective article, we discuss the idea of data veracity and associated concepts as it relates to the use of electronic medical record data and administrative data in research. We discuss the idea that electronic medical record data are "good enough" for clinical practice and, as such, are "good enough" for certain applications. We then propose three primary issues to attend to when establishing data veracity: data provenance, cross validation, and context.

Progress in Biomedical Knowledge Discovery: A 25-year Retrospective

OBJECTIVES

We sought to explore, via a systematic review of the literature, the state of the art of knowledge discovery in biomedical databases as it existed in 1992, and then now, 25 years later, mainly focused on supervised learning.

METHODS

We performed a rigorous systematic search of PubMed and latent Dirichlet allocation to identify themes in the literature and trends in the science of knowledge discovery in and between time periods and compare these trends. We restricted the result set using a bracket of five years previous, such that the 1992 result set was restricted to articles published between 1987 and 1992, and the 2015 set between 2011 and 2015. This was to reflect the current literature available at the time to researchers and others at the target dates of 1992 and 2015. The search term was framed as: Knowledge Discovery OR Data Mining OR Pattern Discovery OR Pattern Recognition, Automated.

RESULTS

A total 538 and 18,172 documents were retrieved for 1992 and 2015, respectively. The number and type of data sources increased dramatically over the observation period, primarily due to the advent of electronic clinical systems. The period 1992- 2015 saw the emergence of new areas of research in knowledge discovery, and the refinement and application of machine learning approaches that were nascent or unknown in 1992.

CONCLUSIONS

Over the 25 years of the observation period, we identified numerous developments that impacted the science of knowledge discovery, including the availability of new forms of data, new machine learning algorithms, and new application domains. Through a bibliometric analysis we examine the striking changes in the availability of highly heterogeneous data resources, the evolution of new algorithmic approaches to knowledge discovery, and we consider from legal, social, and political perspectives possible explanations of the growth of the field. Finally, we reflect on the achievements of the past 25 years to consider what the next 25 years will bring with regard to the availability of even more complex data and to the methods that could be, and are being now developed for the discovery of new knowledge in biomedical data.