1
|
Syed R, Eden R, Makasi T, Chukwudi I, Mamudu A, Kamalpour M, Kapugama Geeganage D, Sadeghianasl S, Leemans SJJ, Goel K, Andrews R, Wynn MT, Ter Hofstede A, Myers T. Digital Health Data Quality Issues: Systematic Review. J Med Internet Res 2023; 25:e42615. [PMID: 37000497 PMCID: PMC10131725 DOI: 10.2196/42615] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2022] [Revised: 12/07/2022] [Accepted: 12/31/2022] [Indexed: 04/01/2023] Open
Abstract
BACKGROUND The promise of digital health is principally dependent on the ability to electronically capture data that can be analyzed to improve decision-making. However, the ability to effectively harness data has proven elusive, largely because of the quality of the data captured. Despite the importance of data quality (DQ), an agreed-upon DQ taxonomy evades literature. When consolidated frameworks are developed, the dimensions are often fragmented, without consideration of the interrelationships among the dimensions or their resultant impact. OBJECTIVE The aim of this study was to develop a consolidated digital health DQ dimension and outcome (DQ-DO) framework to provide insights into 3 research questions: What are the dimensions of digital health DQ? How are the dimensions of digital health DQ related? and What are the impacts of digital health DQ? METHODS Following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, a developmental systematic literature review was conducted of peer-reviewed literature focusing on digital health DQ in predominately hospital settings. A total of 227 relevant articles were retrieved and inductively analyzed to identify digital health DQ dimensions and outcomes. The inductive analysis was performed through open coding, constant comparison, and card sorting with subject matter experts to identify digital health DQ dimensions and digital health DQ outcomes. Subsequently, a computer-assisted analysis was performed and verified by DQ experts to identify the interrelationships among the DQ dimensions and relationships between DQ dimensions and outcomes. The analysis resulted in the development of the DQ-DO framework. RESULTS The digital health DQ-DO framework consists of 6 dimensions of DQ, namely accessibility, accuracy, completeness, consistency, contextual validity, and currency; interrelationships among the dimensions of digital health DQ, with consistency being the most influential dimension impacting all other digital health DQ dimensions; 5 digital health DQ outcomes, namely clinical, clinician, research-related, business process, and organizational outcomes; and relationships between the digital health DQ dimensions and DQ outcomes, with the consistency and accessibility dimensions impacting all DQ outcomes. CONCLUSIONS The DQ-DO framework developed in this study demonstrates the complexity of digital health DQ and the necessity for reducing digital health DQ issues. The framework further provides health care executives with holistic insights into DQ issues and resultant outcomes, which can help them prioritize which DQ-related problems to tackle first.
Collapse
Affiliation(s)
- Rehan Syed
- School of Information Systems, Faculty of Science, Queensland University of Technology, Brisbane, Australia
| | - Rebekah Eden
- School of Information Systems, Faculty of Science, Queensland University of Technology, Brisbane, Australia
| | - Tendai Makasi
- School of Information Systems, Faculty of Science, Queensland University of Technology, Brisbane, Australia
| | - Ignatius Chukwudi
- School of Information Systems, Faculty of Science, Queensland University of Technology, Brisbane, Australia
| | - Azumah Mamudu
- School of Information Systems, Faculty of Science, Queensland University of Technology, Brisbane, Australia
| | - Mostafa Kamalpour
- School of Information Systems, Faculty of Science, Queensland University of Technology, Brisbane, Australia
| | - Dakshi Kapugama Geeganage
- School of Information Systems, Faculty of Science, Queensland University of Technology, Brisbane, Australia
| | - Sareh Sadeghianasl
- School of Information Systems, Faculty of Science, Queensland University of Technology, Brisbane, Australia
| | - Sander J J Leemans
- Rheinisch-Westfälische Technische Hochschule, Aachen University, Aachen, Germany
| | - Kanika Goel
- School of Information Systems, Faculty of Science, Queensland University of Technology, Brisbane, Australia
| | - Robert Andrews
- School of Information Systems, Faculty of Science, Queensland University of Technology, Brisbane, Australia
| | - Moe Thandar Wynn
- School of Information Systems, Faculty of Science, Queensland University of Technology, Brisbane, Australia
| | - Arthur Ter Hofstede
- School of Information Systems, Faculty of Science, Queensland University of Technology, Brisbane, Australia
| | - Trina Myers
- School of Information Systems, Faculty of Science, Queensland University of Technology, Brisbane, Australia
| |
Collapse
|
2
|
von Wagner M, Queck A, Beekers P, Tolhuizen L, Synnatschke A, Boesing J, Chatterjea S. Towards accurate and automatic emergency department workflow characterization using a real-time locating system. INTERNATIONAL JOURNAL OF HEALTHCARE MANAGEMENT 2023. [DOI: 10.1080/20479700.2023.2172829] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/15/2023]
Affiliation(s)
| | | | - Pim Beekers
- Philips Electronics Nederland B.V., Eindhoven, The Netherlands
| | - Ludo Tolhuizen
- Philips Electronics Nederland B.V., Eindhoven, The Netherlands
| | | | | | | |
Collapse
|
3
|
Challenges Encountered and Lessons Learned When Using a Novel Anonymised Linked Dataset of Health and Social Care Records for Public Health Intelligence: The Sussex Integrated Dataset. INFORMATION 2023. [DOI: 10.3390/info14020106] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/10/2023] Open
Abstract
Background: In the United Kingdom National Health Service (NHS), digital transformation programmes have resulted in the creation of pseudonymised linked datasets of patient-level medical records across all NHS and social care services. In the Southeast England counties of East and West Sussex, public health intelligence analysts based in local authorities (LAs) aimed to use the newly created “Sussex Integrated Dataset” (SID) for identifying cohorts of patients who are at risk of early onset multiple long-term conditions (MLTCs). Analysts from the LAs were among the first to have access to this new dataset. Methods: Data access was assured as the analysts were employed within joint data controller organisations and logged into the data via virtual machines following approval of a data access request. Analysts examined the demographics and medical history of patients against multiple external sources, identifying data quality issues and developing methods to establish true values for cases with multiple conflicting entries. Service use was plotted over timelines for individual patients. Results: Early evaluation of the data revealed multiple conflicting within-patient values for age, sex, ethnicity and date of death. This was partially resolved by creating a “demographic milestones” table, capturing demographic details for each patient for each year of the data available in the SID. Older data (≥5 y) was found to be sparse in events and diagnoses. Open-source code lists for defining long-term conditions were poor at identifying the expected number of patients, and bespoke code lists were developed by hand and validated against other sources of data. At the start, the age and sex distributions of patients submitted by GP practices were substantially different from those published by NHS Digital, and errors in data processing were identified and rectified. Conclusions: While new NHS linked datasets appear a promising resource for tracking multi-service use, MLTCs and health inequalities, substantial investment in data analysis and data architect time is necessary to ensure high enough quality data for meaningful analysis. Our team made conceptual progress in identifying the skills needed for programming analyses and understanding the types of questions which can be asked and answered reliably in these datasets.
Collapse
|
4
|
Goodwin AJ, Eytan D, Dixon W, Goodfellow SD, Doherty Z, Greer RW, McEwan A, Tracy M, Laussen PC, Assadi A, Mazwi M. Timing errors and temporal uncertainty in clinical databases-A narrative review. Front Digit Health 2022; 4:932599. [PMID: 36060541 PMCID: PMC9433547 DOI: 10.3389/fdgth.2022.932599] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2022] [Accepted: 07/11/2022] [Indexed: 11/28/2022] Open
Abstract
A firm concept of time is essential for establishing causality in a clinical setting. Review of critical incidents and generation of study hypotheses require a robust understanding of the sequence of events but conducting such work can be problematic when timestamps are recorded by independent and unsynchronized clocks. Most clinical models implicitly assume that timestamps have been measured accurately and precisely, but this custom will need to be re-evaluated if our algorithms and models are to make meaningful use of higher frequency physiological data sources. In this narrative review we explore factors that can result in timestamps being erroneously recorded in a clinical setting, with particular focus on systems that may be present in a critical care unit. We discuss how clocks, medical devices, data storage systems, algorithmic effects, human factors, and other external systems may affect the accuracy and precision of recorded timestamps. The concept of temporal uncertainty is introduced, and a holistic approach to timing accuracy, precision, and uncertainty is proposed. This quantitative approach to modeling temporal uncertainty provides a basis to achieve enhanced model generalizability and improved analytical outcomes.
Collapse
Affiliation(s)
- Andrew J. Goodwin
- Department of Critical Care Medicine, The Hospital for Sick Children, Toronto, ON, Canada
- School of Biomedical Engineering, University of Sydney, Sydney, NSW, Australia
| | - Danny Eytan
- Department of Critical Care Medicine, The Hospital for Sick Children, Toronto, ON, Canada
- Department of Medicine, Technion - Israel Institute of Technology, Haifa, Israel
| | - William Dixon
- Department of Critical Care Medicine, The Hospital for Sick Children, Toronto, ON, Canada
| | - Sebastian D. Goodfellow
- Department of Critical Care Medicine, The Hospital for Sick Children, Toronto, ON, Canada
- Department of Civil and Mineral Engineering, University of Toronto, Toronto, ON, Canada
| | - Zakary Doherty
- Research Fellow, School of Rural Health, Monash University, Melbourne, VIC, Australia
| | - Robert W. Greer
- Department of Critical Care Medicine, The Hospital for Sick Children, Toronto, ON, Canada
| | - Alistair McEwan
- School of Biomedical Engineering, University of Sydney, Sydney, NSW, Australia
| | - Mark Tracy
- Neonatal Intensive Care Unit, Westmead Hospital, Sydney, NSW, Australia
- Department of Paediatrics and Child Health, The University of Sydney, Sydney, NSW, Australia
| | - Peter C. Laussen
- Department of Anesthesia, Boston Children's Hospital, Boston, MA, United States
| | - Azadeh Assadi
- Department of Critical Care Medicine, The Hospital for Sick Children, Toronto, ON, Canada
- Department of Engineering and Applied Sciences, Institute of Biomedical Engineering, University of Toronto, Toronto, ON, Canada
| | - Mjaye Mazwi
- Department of Critical Care Medicine, The Hospital for Sick Children, Toronto, ON, Canada
- Department of Paediatrics, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
5
|
Erdogan TG, Tarhan AK. Multi-perspective process mining for emergency process. Health Informatics J 2022; 28:14604582221077195. [PMID: 35195463 DOI: 10.1177/14604582221077195] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
BACKGROUND Multi-perspective process mining is an analytical approach that uses data to gain objective insights and uncover hidden problems in business processes from multiple perspectives. OBJECTIVE In this paper, we apply multi-perspective process mining techniques in the emergency process through a goal-driven performance evaluation method in order to understand and diagnose the timeliness of the emergency process. METHODS Unstructured and multi-disciplinary emergency data is analyzed by following Goal-Question-Feature-Indicator (GQFI) method. In this paper, the GQFI method is extended with perspectives, and the insights in the enriched event data are examined by a decision tree model. All of them are applied in a systematic way in relation to the goal of assessing and improving the emergency process in a university hospital. RESULTS We detected the deviations (e.g., skipping the triage and consultation request steps) and two bottlenecks in the emergency process. Among the suggestions for improving the process, are performing defensive medicine in a harmless manner, classification of the emergency services, ensuring triage step is applied to all patients and effective usage of the call system application in consultation activities. CONCLUSION The results of this study showed that goal-oriented multi-perspective process mining is effective in identifying process improvements in emergency services.
Collapse
|
6
|
Peters MJ, Finch CK, Roberts L, Covington A, Krushinski J. Conversion to an electronic missing medication request system at an academic medical center. Health Informatics J 2021; 27:1460458221994862. [PMID: 33624551 DOI: 10.1177/1460458221994862] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Missing medications can negatively contribute to the financial and operational workflows of pharmacy departments and add medication safety challenges. The missing medication request (MMR) system at the study institution converted to entirely electronic in June 2018 from a hybrid electronic system. This study evaluated 4-week periods pre- and post-conversion. The objective of this study was to evaluate the impact of conversion to an electronic MMR system on the quantity of requests received at an academic medical center. The average daily number of MMR's decreased from the pre-conversion group to the post-conversion group (1.77 (±0.16) vs 1.48 (±0.17), p < 0.001). During post-conversion, the median triage time was 8 min [3 min-19 min], pharmacists triaged 62.4% of requests, and 29.6% of requests were declined. Conversion to an electronic MMR system represents one solution to decreasing missing medications. Future studies are needed to evaluate the financial, operational, and medication safety impact of conversion.
Collapse
|
7
|
A Systematic Review of Recommendations of Long-Term Strategies for Researchers Using Data Science Techniques. COMPUTERS 2021. [DOI: 10.3390/computers10030035] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
The impact of the strategies that researchers follow to publish or produce scientific content can have a long-term impact. Identifying which strategies are most influential in the future has been attracting increasing attention in the literature. In this study, we present a systematic review of recommendations of long-term strategies in research analytics and their implementation methodologies. The objective is to present an overview from 2002 to 2018 on the development of this topic, including trends, and addressed contexts. The central objective is to identify data-oriented approaches to learn long-term research strategies, especially in process mining. We followed a protocol for systematic reviews for the engineering area in a structured and respectful manner. The results show the need for studies that generate more specific recommendations based on data mining. This outcome leaves open research opportunities from two particular perspectives—applying methodologies involving process mining for the context of research analytics and the feasibility study on long-term strategies using data science techniques.
Collapse
|
8
|
Cho M, Song M, Park J, Yeom SR, Wang IJ, Choi BK. Process Mining-Supported Emergency Room Process Performance Indicators. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2020; 17:ijerph17176290. [PMID: 32872350 PMCID: PMC7503251 DOI: 10.3390/ijerph17176290] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/11/2020] [Revised: 08/21/2020] [Accepted: 08/26/2020] [Indexed: 11/21/2022]
Abstract
Emergency room processes are often exposed to the risk of unexpected factors, and process management based on performance measurements is required due to its connectivity to the quality of care. Regarding this, there have been several attempts to propose a method to analyze the emergency room processes. This paper proposes a framework for process performance indicators utilized in emergency rooms. Based on the devil’s quadrangle, i.e., time, cost, quality, and flexibility, the paper suggests multiple process performance indicators that can be analyzed using clinical event logs and verify them with a thorough discussion with clinical experts in the emergency department. A case study is conducted with the real-life clinical data collected from a tertiary hospital in Korea to validate the proposed method. The case study demonstrated that the proposed indicators are well applied using the clinical data, and the framework is capable of understanding emergency room processes’ performance.
Collapse
Affiliation(s)
- Minsu Cho
- Research Institute of Industry & SME Strategy, Korea Institute of Industrial Technology, Seoul 06211, Korea;
| | - Minseok Song
- Department of Industrial & Management Engineering, Pohang University of Science and Technology, Pohang 37673, Korea;
- Correspondence:
| | - Junhyun Park
- Department of Industrial & Management Engineering, Pohang University of Science and Technology, Pohang 37673, Korea;
| | - Seok-Ran Yeom
- Department of Emergency Medicine, Pusan National University Hospital, Busan 49241, Korea; (S.-R.Y.); (I.-J.W.)
| | - Il-Jae Wang
- Department of Emergency Medicine, Pusan National University Hospital, Busan 49241, Korea; (S.-R.Y.); (I.-J.W.)
| | - Byung-Kwan Choi
- Department of Neurosurgery, Pusan National University Hospital, Busan 49241, Korea;
| |
Collapse
|
9
|
Corallo A, Lazoi M, Striani F. Process mining and industrial applications: A systematic literature review. KNOWLEDGE AND PROCESS MANAGEMENT 2020. [DOI: 10.1002/kpm.1630] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
|
10
|
Kurniati AP, Rojas E, Hogg D, Hall G, Johnson OA. The assessment of data quality issues for process mining in healthcare using Medical Information Mart for Intensive Care III, a freely available e-health record database. Health Informatics J 2018; 25:1878-1893. [PMID: 30488750 DOI: 10.1177/1460458218810760] [Citation(s) in RCA: 25] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/14/2023]
Abstract
There is a growing body of literature on process mining in healthcare. Process mining of electronic health record systems could give benefit into better understanding of the actual processes happened in the patient treatment, from the event log of the hospital information system. Researchers report issues of data access approval, anonymisation constraints, and data quality. One solution to progress methodology development is to use a high-quality, freely available research dataset such as Medical Information Mart for Intensive Care III, a critical care database which contains the records of 46,520 intensive care unit patients over 12 years. Our article aims to (1) explore data quality issues for healthcare process mining using Medical Information Mart for Intensive Care III, (2) provide a structured assessment of Medical Information Mart for Intensive Care III data quality and challenge for process mining, and (3) provide a worked example of cancer treatment as a case study of process mining using Medical Information Mart for Intensive Care III to illustrate an approach and solution to data quality challenges. The electronic health record software was upgraded partway through the period over which data was collected and we use this event to explore the link between electronic health record system design and resulting process models.
Collapse
Affiliation(s)
| | - Eric Rojas
- Pontificia Universidad Catolica de Chile, Chile
| | | | - Geoff Hall
- University of Leeds, UK; St James's University Hospital, Leeds, UK
| | | |
Collapse
|
11
|
Abstract
Veracity, one of the five V's used to describe big data, has received attention when it comes to using electronic medical record data for research purposes. In this perspective article, we discuss the idea of data veracity and associated concepts as it relates to the use of electronic medical record data and administrative data in research. We discuss the idea that electronic medical record data are "good enough" for clinical practice and, as such, are "good enough" for certain applications. We then propose three primary issues to attend to when establishing data veracity: data provenance, cross validation, and context.
Collapse
|
12
|
Sacchi L, Holmes JH. Progress in Biomedical Knowledge Discovery: A 25-year Retrospective. Yearb Med Inform 2016; Suppl 1:S117-29. [PMID: 27488403 PMCID: PMC5171499 DOI: 10.15265/iys-2016-s033] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/28/2022] Open
Abstract
OBJECTIVES We sought to explore, via a systematic review of the literature, the state of the art of knowledge discovery in biomedical databases as it existed in 1992, and then now, 25 years later, mainly focused on supervised learning. METHODS We performed a rigorous systematic search of PubMed and latent Dirichlet allocation to identify themes in the literature and trends in the science of knowledge discovery in and between time periods and compare these trends. We restricted the result set using a bracket of five years previous, such that the 1992 result set was restricted to articles published between 1987 and 1992, and the 2015 set between 2011 and 2015. This was to reflect the current literature available at the time to researchers and others at the target dates of 1992 and 2015. The search term was framed as: Knowledge Discovery OR Data Mining OR Pattern Discovery OR Pattern Recognition, Automated. RESULTS A total 538 and 18,172 documents were retrieved for 1992 and 2015, respectively. The number and type of data sources increased dramatically over the observation period, primarily due to the advent of electronic clinical systems. The period 1992- 2015 saw the emergence of new areas of research in knowledge discovery, and the refinement and application of machine learning approaches that were nascent or unknown in 1992. CONCLUSIONS Over the 25 years of the observation period, we identified numerous developments that impacted the science of knowledge discovery, including the availability of new forms of data, new machine learning algorithms, and new application domains. Through a bibliometric analysis we examine the striking changes in the availability of highly heterogeneous data resources, the evolution of new algorithmic approaches to knowledge discovery, and we consider from legal, social, and political perspectives possible explanations of the growth of the field. Finally, we reflect on the achievements of the past 25 years to consider what the next 25 years will bring with regard to the availability of even more complex data and to the methods that could be, and are being now developed for the discovery of new knowledge in biomedical data.
Collapse
Affiliation(s)
| | - J H Holmes
- John H Holmes, Institute for Biomedical Informatics, University of Pennsylvania School of Medicine, 717 Blockley Hall, 423 Guardian Drive, Philadelphia, PA 19104, USA, Tel: 215-898-4833, Fax: 215-573-5325, E-Mail:
| |
Collapse
|