1
|
Wu H, Wang M, Wu J, Francis F, Chang YH, Shavick A, Dong H, Poon MTC, Fitzpatrick N, Levine AP, Slater LT, Handy A, Karwath A, Gkoutos GV, Chelala C, Shah AD, Stewart R, Collier N, Alex B, Whiteley W, Sudlow C, Roberts A, Dobson RJB. A survey on clinical natural language processing in the United Kingdom from 2007 to 2022. NPJ Digit Med 2022; 5:186. [PMID: 36544046 PMCID: PMC9770568 DOI: 10.1038/s41746-022-00730-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2022] [Accepted: 11/29/2022] [Indexed: 12/24/2022] Open
Abstract
Much of the knowledge and information needed for enabling high-quality clinical research is stored in free-text format. Natural language processing (NLP) has been used to extract information from these sources at scale for several decades. This paper aims to present a comprehensive review of clinical NLP for the past 15 years in the UK to identify the community, depict its evolution, analyse methodologies and applications, and identify the main barriers. We collect a dataset of clinical NLP projects (n = 94; £ = 41.97 m) funded by UK funders or the European Union's funding programmes. Additionally, we extract details on 9 funders, 137 organisations, 139 persons and 431 research papers. Networks are created from timestamped data interlinking all entities, and network analysis is subsequently applied to generate insights. 431 publications are identified as part of a literature review, of which 107 are eligible for final analysis. Results show, not surprisingly, clinical NLP in the UK has increased substantially in the last 15 years: the total budget in the period of 2019-2022 was 80 times that of 2007-2010. However, the effort is required to deepen areas such as disease (sub-)phenotyping and broaden application domains. There is also a need to improve links between academia and industry and enable deployments in real-world settings for the realisation of clinical NLP's great potential in care delivery. The major barriers include research and development access to hospital data, lack of capable computational resources in the right places, the scarcity of labelled data and barriers to sharing of pretrained models.
Collapse
Affiliation(s)
- Honghan Wu
- Institute of Health Informatics, University College London, London, UK.
| | - Minhong Wang
- Institute of Health Informatics, University College London, London, UK
| | - Jinge Wu
- Institute of Health Informatics, University College London, London, UK
- Usher Institute, University of Edinburgh, Edinburgh, UK
| | - Farah Francis
- Usher Institute, University of Edinburgh, Edinburgh, UK
| | - Yun-Hsuan Chang
- Institute of Health Informatics, University College London, London, UK
| | - Alex Shavick
- Research Department of Pathology, UCL Cancer Institute, University College London, London, UK
| | - Hang Dong
- Usher Institute, University of Edinburgh, Edinburgh, UK
- Department of Computer Science, University of Oxford, Oxford, UK
| | | | | | - Adam P Levine
- Research Department of Pathology, UCL Cancer Institute, University College London, London, UK
| | - Luke T Slater
- Institute of Cancer and Genomics, University of Birmingham, Birmingham, UK
| | - Alex Handy
- Institute of Health Informatics, University College London, London, UK
- University College London Hospitals NHS Trust, London, UK
| | - Andreas Karwath
- Institute of Cancer and Genomics, University of Birmingham, Birmingham, UK
| | - Georgios V Gkoutos
- Institute of Cancer and Genomics, University of Birmingham, Birmingham, UK
| | - Claude Chelala
- Centre for Tumour Biology, Barts Cancer Institute, Queen Mary University of London, London, UK
| | - Anoop Dinesh Shah
- Institute of Health Informatics, University College London, London, UK
| | - Robert Stewart
- Department of Psychological Medicine, Institute of Psychiatry, Psychology and Neuroscience (IoPPN), King's College London, London, UK
- South London and Maudsley NHS Foundation Trust, London, UK
| | - Nigel Collier
- Theoretical and Applied Linguistics, Faculty of Modern & Medieval Languages & Linguistics, University of Cambridge, Cambridge, UK
| | - Beatrice Alex
- Edinburgh Futures Institute, University of Edinburgh, Edinburgh, UK
| | | | - Cathie Sudlow
- Usher Institute, University of Edinburgh, Edinburgh, UK
| | - Angus Roberts
- Department of Biostatistics & Health Informatics, King's College London, London, UK
| | - Richard J B Dobson
- Institute of Health Informatics, University College London, London, UK
- Department of Biostatistics & Health Informatics, King's College London, London, UK
| |
Collapse
|
2
|
Botelle R, Bhavsar V, Kadra-Scalzo G, Mascio A, Williams MV, Roberts A, Velupillai S, Stewart R. Can natural language processing models extract and classify instances of interpersonal violence in mental healthcare electronic records: an applied evaluative study. BMJ Open 2022; 12:e052911. [PMID: 35172999 PMCID: PMC8852656 DOI: 10.1136/bmjopen-2021-052911] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/04/2022] Open
Abstract
OBJECTIVE This paper evaluates the application of a natural language processing (NLP) model for extracting clinical text referring to interpersonal violence using electronic health records (EHRs) from a large mental healthcare provider. DESIGN A multidisciplinary team iteratively developed guidelines for annotating clinical text referring to violence. Keywords were used to generate a dataset which was annotated (ie, classified as affirmed, negated or irrelevant) for: presence of violence, patient status (ie, as perpetrator, witness and/or victim of violence) and violence type (domestic, physical and/or sexual). An NLP approach using a pretrained transformer model, BioBERT (Bidirectional Encoder Representations from Transformers for Biomedical Text Mining) was fine-tuned on the annotated dataset and evaluated using 10-fold cross-validation. SETTING We used the Clinical Records Interactive Search (CRIS) database, comprising over 500 000 de-identified EHRs of patients within the South London and Maudsley NHS Foundation Trust, a specialist mental healthcare provider serving an urban catchment area. PARTICIPANTS Searches of CRIS were carried out based on 17 predefined keywords. Randomly selected text fragments were taken from the results for each keyword, amounting to 3771 text fragments from the records of 2832 patients. OUTCOME MEASURES We estimated precision, recall and F1 score for each NLP model. We examined sociodemographic and clinical variables in patients giving rise to the text data, and frequencies for each annotated violence characteristic. RESULTS Binary classification models were developed for six labels (violence presence, perpetrator, victim, domestic, physical and sexual). Among annotations affirmed for the presence of any violence, 78% (1724) referred to physical violence, 61% (1350) referred to patients as perpetrator and 33% (731) to domestic violence. NLP models' precision ranged from 89% (perpetrator) to 98% (sexual); recall ranged from 89% (victim, perpetrator) to 97% (sexual). CONCLUSIONS State of the art NLP models can extract and classify clinical text on violence from EHRs at acceptable levels of scale, efficiency and accuracy.
Collapse
Affiliation(s)
- Riley Botelle
- School of Medical Education, Guy's, King's and St Thomas' School of Medicine, London, UK
| | - Vishal Bhavsar
- Section of Women's Mental Health, Department of Health Services and Population Research, King's College London, London, UK
| | - Giouliana Kadra-Scalzo
- Psychological Medicine, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| | - Aurelie Mascio
- Psychological Medicine, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| | - Marcus V Williams
- School of Medical Education, Guy's, King's and St Thomas' School of Medicine, London, UK
| | - Angus Roberts
- Biostatistics and Health Informatics, King's College London, London, UK
- Health Data Research UK, London, UK
| | - Sumithra Velupillai
- Psychological Medicine, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| | - Robert Stewart
- Psychological Medicine, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
- South London and Maudsley Mental Health NHS Trust, London, UK
| |
Collapse
|
3
|
Patra BG, Sharma MM, Vekaria V, Adekkanattu P, Patterson OV, Glicksberg B, Lepow LA, Ryu E, Biernacka JM, Furmanchuk A, George TJ, Hogan W, Wu Y, Yang X, Bian J, Weissman M, Wickramaratne P, Mann JJ, Olfson M, Campion TR, Weiner M, Pathak J. Extracting social determinants of health from electronic health records using natural language processing: a systematic review. J Am Med Inform Assoc 2021; 28:2716-2727. [PMID: 34613399 PMCID: PMC8633615 DOI: 10.1093/jamia/ocab170] [Citation(s) in RCA: 61] [Impact Index Per Article: 20.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2021] [Revised: 07/09/2021] [Accepted: 08/04/2021] [Indexed: 11/27/2022] Open
Abstract
OBJECTIVE Social determinants of health (SDoH) are nonclinical dispositions that impact patient health risks and clinical outcomes. Leveraging SDoH in clinical decision-making can potentially improve diagnosis, treatment planning, and patient outcomes. Despite increased interest in capturing SDoH in electronic health records (EHRs), such information is typically locked in unstructured clinical notes. Natural language processing (NLP) is the key technology to extract SDoH information from clinical text and expand its utility in patient care and research. This article presents a systematic review of the state-of-the-art NLP approaches and tools that focus on identifying and extracting SDoH data from unstructured clinical text in EHRs. MATERIALS AND METHODS A broad literature search was conducted in February 2021 using 3 scholarly databases (ACL Anthology, PubMed, and Scopus) following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. A total of 6402 publications were initially identified, and after applying the study inclusion criteria, 82 publications were selected for the final review. RESULTS Smoking status (n = 27), substance use (n = 21), homelessness (n = 20), and alcohol use (n = 15) are the most frequently studied SDoH categories. Homelessness (n = 7) and other less-studied SDoH (eg, education, financial problems, social isolation and support, family problems) are mostly identified using rule-based approaches. In contrast, machine learning approaches are popular for identifying smoking status (n = 13), substance use (n = 9), and alcohol use (n = 9). CONCLUSION NLP offers significant potential to extract SDoH data from narrative clinical notes, which in turn can aid in the development of screening tools, risk prediction models, and clinical decision support systems.
Collapse
Affiliation(s)
- Braja G Patra
- Department of Population Health Sciences, Weill Cornell Medicine, New York, New York, USA
| | - Mohit M Sharma
- Department of Population Health Sciences, Weill Cornell Medicine, New York, New York, USA
| | - Veer Vekaria
- Department of Population Health Sciences, Weill Cornell Medicine, New York, New York, USA
| | - Prakash Adekkanattu
- Information Technologies and Services, Weill Cornell Medicine, New York, New York, USA
| | - Olga V Patterson
- Department of Internal Medicine, Division of Epidemiology, University of Utah, Salt Lake City, Utah, USA
- US Department of Veterans Affairs, Salt Lake City, Utah, USA
| | | | - Lauren A Lepow
- Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Euijung Ryu
- Department of Quantitative Health Sciences, Mayo Clinic, Rochester, Minnesota, USA
| | - Joanna M Biernacka
- Department of Quantitative Health Sciences, Mayo Clinic, Rochester, Minnesota, USA
| | | | - Thomas J George
- Department of Health Outcomes and Biomedical Informatics, University of Florida, Gainesville, Florida, USA
| | - William Hogan
- Division of Hematology & Oncology, Department of Medicine, College of Medicine, University of Florida, Gainesville, Florida, USA, and
| | - Yonghui Wu
- Department of Health Outcomes and Biomedical Informatics, University of Florida, Gainesville, Florida, USA
| | - Xi Yang
- Department of Health Outcomes and Biomedical Informatics, University of Florida, Gainesville, Florida, USA
| | - Jiang Bian
- Department of Health Outcomes and Biomedical Informatics, University of Florida, Gainesville, Florida, USA
| | - Myrna Weissman
- Vagelos College of Physicians and Surgeons, Columbia University, New York, New York, USA
| | - Priya Wickramaratne
- Vagelos College of Physicians and Surgeons, Columbia University, New York, New York, USA
| | - J John Mann
- Vagelos College of Physicians and Surgeons, Columbia University, New York, New York, USA
| | - Mark Olfson
- Vagelos College of Physicians and Surgeons, Columbia University, New York, New York, USA
| | - Thomas R Campion
- Department of Population Health Sciences, Weill Cornell Medicine, New York, New York, USA
- Information Technologies and Services, Weill Cornell Medicine, New York, New York, USA
| | - Mark Weiner
- Department of Population Health Sciences, Weill Cornell Medicine, New York, New York, USA
| | - Jyotishman Pathak
- Department of Population Health Sciences, Weill Cornell Medicine, New York, New York, USA
| |
Collapse
|
4
|
Wilkinson K, Sheets L, Fitch D, Popejoy L. Systematic review of approaches to use of neighborhood-level risk factors with clinical data to predict clinical risk and recommend interventions. J Biomed Inform 2021; 116:103713. [PMID: 33610880 DOI: 10.1016/j.jbi.2021.103713] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2020] [Revised: 02/06/2021] [Accepted: 02/10/2021] [Indexed: 11/24/2022]
Abstract
BACKGROUND Despite a large body of literature investigating how the environment influences health outcomes, most published work to date includes only a limited subset of the rich clinical and environmental data that is available and does not address how these data might best be used to predict clinical risk or expected impact of clinical interventions. OBJECTIVE Identify existing approaches to inclusion of a broad set of neighborhood-level risk factors with clinical data to predict clinical risk and recommend interventions. METHODS A systematic review of scientific literature published and indexed in PubMed, Web of Science, Association of Computing Machinery (ACM) and SCOPUS from 2010 through October 2020 was performed. To be included, articles had to include search terms related to Electronic Health Record (EHR) data Neighborhood-Level Risk Factors (NLRFs), and Machine Learning (ML) Methods. Citations of relevant articles were also reviewed for additional articles for inclusion. Articles were reviewed and coded by two independent reviewers to capture key information including data sources, linkage of EHR to NRLFs, methods, and results. Articles were assessed for quality using a modified Quality Assessment Tool for Systematic Reviews of Observational Studies (QATSO). RESULTS A total of 334 articles were identified for abstract review. 36 articles were identified for full review with 19 articles included in the final analysis. All but two of the articles included socio-demographic data derived from the U.S. Census and we found great variability in sources of NLRFs beyond the Census. The majority or the articles (14 of 19) included broader clinical (e.g. medications, labs and co-morbidities) and demographic information about the individual from the EHR in addition to the clinical outcome variable. Half of the articles (10) had a stated goal to predict the outcome(s) of interest. While results of the studies reinforced the correlative association of NLRFs to clinical outcomes, only one article found that adding NLRFs into a model with other data added predictive power with the remainder concluding either that NLRFs were of mixed value depending on the model and outcome or that NLRFs added no predictive power over other data in the model. Only one article scored high on the quality assessment with 13 scoring moderate and 4 scoring low. CONCLUSIONS In spite of growing interest in combining NLRFs with EHR data for clinical prediction, we found limited evidence that NLRFs improve predictive power in clinical risk models. We found these data and methods are being used in four ways. First, early approaches to include broad NLRFs to predict clinical risk primarily focused on dimension reduction for feature selection or as a data preparation step to input into regression analysis. Second, more recent work incorporates NLRFs into more advanced predictive models, such as Neural Networks, Random Forest, and Penalized Lasso to predict clinical outcomes or predict value of interventions. Third, studies that test how inclusion of NLRFs predict clinical risk have shown mixed results regarding the value of these data over EHR or claims data alone and this review surfaced evidence of potential quality challenges and biases inherent to this approach. Finally, NLRFs were used with unsupervised learning to identify underlying patterns in patient populations to recommend targeted interventions. Further access to computable, high quality data is needed along with careful study design, including sub-group analysis, to better determine how these data and methods can be used to support decision making in a clinical setting.
Collapse
Affiliation(s)
- Katie Wilkinson
- Institute for Data Science and Informatics, University of Missouri, Columbia, MO 65212, United States; School of Medicine, University of Missouri, Columbia, MO 65212, United States.
| | - Lincoln Sheets
- Institute for Data Science and Informatics, University of Missouri, Columbia, MO 65212, United States; School of Medicine, University of Missouri, Columbia, MO 65212, United States
| | - Dale Fitch
- Institute for Data Science and Informatics, University of Missouri, Columbia, MO 65212, United States; School of Social Work, University of Missouri, Columbia, MO 65212, United States
| | - Lori Popejoy
- Institute for Data Science and Informatics, University of Missouri, Columbia, MO 65212, United States; School of Nursing, University of Missouri, Columbia, MO 65212, United States
| |
Collapse
|