1
|
Kovačević A, Bašaragin B, Milošević N, Nenadić G. De-identification of clinical free text using natural language processing: A systematic review of current approaches. Artif Intell Med 2024; 151:102845. [PMID: 38555848 DOI: 10.1016/j.artmed.2024.102845] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2023] [Revised: 03/13/2024] [Accepted: 03/18/2024] [Indexed: 04/02/2024]
Abstract
BACKGROUND Electronic health records (EHRs) are a valuable resource for data-driven medical research. However, the presence of protected health information (PHI) makes EHRs unsuitable to be shared for research purposes. De-identification, i.e. the process of removing PHI is a critical step in making EHR data accessible. Natural language processing has repeatedly demonstrated its feasibility in automating the de-identification process. OBJECTIVES Our study aims to provide systematic evidence on how the de-identification of clinical free text written in English has evolved in the last thirteen years, and to report on the performances and limitations of the current state-of-the-art systems for the English language. In addition, we aim to identify challenges and potential research opportunities in this field. METHODS A systematic search in PubMed, Web of Science, and the DBLP was conducted for studies published between January 2010 and February 2023. Titles and abstracts were examined to identify the relevant studies. Selected studies were then analysed in-depth, and information was collected on de-identification methodologies, data sources, and measured performance. RESULTS A total of 2125 publications were identified for the title and abstract screening. 69 studies were found to be relevant. Machine learning (37 studies) and hybrid (26 studies) approaches are predominant, while six studies relied only on rules. The majority of the approaches were trained and evaluated on public corpora. The 2014 i2b2/UTHealth corpus is the most frequently used (36 studies), followed by the 2006 i2b2 (18 studies) and 2016 CEGS N-GRID (10 studies) corpora. CONCLUSION Earlier de-identification approaches aimed at English were mainly rule and machine learning hybrids with extensive feature engineering and post-processing, while more recent performance improvements are due to feature-inferring recurrent neural networks. Current leading performance is achieved using attention-based neural models. Recent studies report state-of-the-art F1-scores (over 98 %) when evaluated in the manner usually adopted by the clinical natural language processing community. However, their performance needs to be more thoroughly assessed with different measures to judge their reliability to safely de-identify data in a real-world setting. Without additional manually labeled training data, state-of-the-art systems fail to generalise well across a wide range of clinical sub-domains.
Collapse
Affiliation(s)
- Aleksandar Kovačević
- The University of Novi Sad, Faculty of Technical Sciences, Trg Dositeja Obradovića 6, 21002 Novi Sad, Serbia
| | - Bojana Bašaragin
- The Institute for Artificial Intelligence Research and Development of Serbia, Fruškogorska 1, 21000 Novi Sad, Serbia.
| | - Nikola Milošević
- The Institute for Artificial Intelligence Research and Development of Serbia, Fruškogorska 1, 21000 Novi Sad, Serbia; Bayer A.G., Research and Development, Mullerstrasse 173, Berlin 13342, Germany
| | - Goran Nenadić
- The University of Manchester, Department of Computer Science, Manchester, United Kingdom
| |
Collapse
|
2
|
Honeyford K, Expert P, Mendelsohn E, Post B, Faisal A, Glampson B, Mayer E, Costelloe C. Challenges and recommendations for high quality research using electronic health records. Front Digit Health 2022; 4:940330. [PMID: 36060540 PMCID: PMC9437583 DOI: 10.3389/fdgth.2022.940330] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2022] [Accepted: 07/28/2022] [Indexed: 12/02/2022] Open
Abstract
Harnessing Real World Data is vital to improve health care in the 21st Century. Data from Electronic Health Records (EHRs) are a rich source of patient centred data, including information on the patient's clinical condition, laboratory results, diagnoses and treatments. They thus reflect the true state of health systems. However, access and utilisation of EHR data for research presents specific challenges. We assert that using data from EHRs effectively is dependent on synergy between researchers, clinicians and health informaticians, and only this will allow state of the art methods to be used to answer urgent and vital questions for patient care. We propose that there needs to be a paradigm shift in the way this research is conducted - appreciating that the research process is iterative rather than linear. We also make specific recommendations for organisations, based on our experience of developing and using EHR data in trusted research environments.
Collapse
Affiliation(s)
- K Honeyford
- Global Digital Health Unit, School of Public Health, Imperial College London, London, United Kingdom
- Health Informatics Team, Division of Clinical studies, Institute of Cancer Research, London, United Kingdom
| | - P Expert
- Global Digital Health Unit, School of Public Health, Imperial College London, London, United Kingdom
- Global Business School for Health, University College London, London, United Kingdom
| | - E.E Mendelsohn
- Global Digital Health Unit, School of Public Health, Imperial College London, London, United Kingdom
| | - B Post
- Department of Computing, Imperial College London, London, United Kingdom
- UKRI Centre for Doctoral Training in AI for Healthcare, Imperial College London, London, United Kingdom
| | - A.A Faisal
- Department of Computing, Imperial College London, London, United Kingdom
- UKRI Centre for Doctoral Training in AI for Healthcare, Imperial College London, London, United Kingdom
- Chair in Digital Health, Faculty of Life Sciences, University of Bayreuth, Bayreuth, Germany
- Department of Bioengineering, Imperial College London, London, United Kingdom
| | - B Glampson
- Translational Data Analytics and Informatics in Healthcare, Department of Surgery & Cancer, Imperial College London, London, United Kingdom
- Imperial Clinical Analytics, Informatics and Evaluation (iCARE), NIHR Imperial BRC, Imperial College Healthcare NHS Trust, London, United Kingdom
| | - E.K Mayer
- UKRI Centre for Doctoral Training in AI for Healthcare, Imperial College London, London, United Kingdom
- Translational Data Analytics and Informatics in Healthcare, Department of Surgery & Cancer, Imperial College London, London, United Kingdom
- Imperial Clinical Analytics, Informatics and Evaluation (iCARE), NIHR Imperial BRC, Imperial College Healthcare NHS Trust, London, United Kingdom
| | - C.E Costelloe
- Global Digital Health Unit, School of Public Health, Imperial College London, London, United Kingdom
- Health Informatics Team, Division of Clinical studies, Institute of Cancer Research, London, United Kingdom
- Health Informatics Team, Royal Marsden Hospital, London, United Kingdom
| |
Collapse
|
3
|
Ford E, Curlewis K, Squires E, Griffiths LJ, Stewart R, Jones KH. The Potential of Research Drawing on Clinical Free Text to Bring Benefits to Patients in the United Kingdom: A Systematic Review of the Literature. Front Digit Health 2021; 3:606599. [PMID: 34713089 PMCID: PMC8521813 DOI: 10.3389/fdgth.2021.606599] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2020] [Accepted: 01/15/2021] [Indexed: 11/13/2022] Open
Abstract
Background: The analysis of clinical free text from patient records for research has potential to contribute to the medical evidence base but access to clinical free text is frequently denied by data custodians who perceive that the privacy risks of data-sharing are too high. Engagement activities with patients and regulators, where views on the sharing of clinical free text data for research have been discussed, have identified that stakeholders would like to understand the potential clinical benefits that could be achieved if access to free text for clinical research were improved. We aimed to systematically review all UK research studies which used clinical free text and report direct or potential benefits to patients, synthesizing possible benefits into an easy to communicate taxonomy for public engagement and policy discussions. Methods: We conducted a systematic search for articles which reported primary research using clinical free text, drawn from UK health record databases, which reported a benefit or potential benefit for patients, actionable in a clinical environment or health service, and not solely methods development or data quality improvement. We screened eligible papers and thematically analyzed information about clinical benefits reported in the paper to create a taxonomy of benefits. Results: We identified 43 papers and derived five themes of benefits: health-care quality or services improvement, observational risk factor-outcome research, drug prescribing safety, case-finding for clinical trials, and development of clinical decision support. Five papers compared study quality with and without free text and found an improvement of accuracy when free text was included in analytical models. Conclusions: Findings will help stakeholders weigh the potential benefits of free text research against perceived risks to patient privacy. The taxonomy can be used to aid public and policy discussions, and identified studies could form a public-facing repository which will help the health-care text analysis research community better communicate the impact of their work.
Collapse
Affiliation(s)
- Elizabeth Ford
- Department of Primary Care and Public Health, Brighton and Sussex Medical School, Brighton, United Kingdom
| | - Keegan Curlewis
- Department of Primary Care and Public Health, Brighton and Sussex Medical School, Brighton, United Kingdom
| | - Emma Squires
- Swansea Medical School, University of Swansea, Swansea, United Kingdom
| | - Lucy J. Griffiths
- Swansea Medical School, University of Swansea, Swansea, United Kingdom
| | - Robert Stewart
- King's College London, London, United Kingdom
- South London and Maudsley NHS Foundation Trust, London, United Kingdom
| | - Kerina H. Jones
- Swansea Medical School, University of Swansea, Swansea, United Kingdom
| |
Collapse
|