1
|
Miranda O, Kiehl SM, Qi X, Brannock MD, Kosten T, Ryan ND, Kirisci L, Wang Y, Wang L. Enhancing post-traumatic stress disorder patient assessment: leveraging natural language processing for research of domain criteria identification using electronic medical records. BMC Med Inform Decis Mak 2024; 24:154. [PMID: 38835009 PMCID: PMC11151516 DOI: 10.1186/s12911-024-02554-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/06/2024] [Accepted: 05/28/2024] [Indexed: 06/06/2024] Open
Abstract
BACKGROUND Extracting research of domain criteria (RDoC) from high-risk populations like those with post-traumatic stress disorder (PTSD) is crucial for positive mental health improvements and policy enhancements. The intricacies of collecting, integrating, and effectively leveraging clinical notes for this purpose introduce complexities. METHODS In our study, we created a natural language processing (NLP) workflow to analyze electronic medical record (EMR) data and identify and extract research of domain criteria using a pre-trained transformer-based natural language model, all-mpnet-base-v2. We subsequently built dictionaries from 100,000 clinical notes and analyzed 5.67 million clinical notes from 38,807 PTSD patients from the University of Pittsburgh Medical Center. Subsequently, we showcased the significance of our approach by extracting and visualizing RDoC information in two use cases: (i) across multiple patient populations and (ii) throughout various disease trajectories. RESULTS The sentence transformer model demonstrated high F1 macro scores across all RDoC domains, achieving the highest performance with a cosine similarity threshold value of 0.3. This ensured an F1 score of at least 80% across all RDoC domains. The study revealed consistent reductions in all six RDoC domains among PTSD patients after psychotherapy. We found that 60.6% of PTSD women have at least one abnormal instance of the six RDoC domains as compared to PTSD men (51.3%), with 45.1% of PTSD women with higher levels of sensorimotor disturbances compared to men (41.3%). We also found that 57.3% of PTSD patients have at least one abnormal instance of the six RDoC domains based on our records. Also, veterans had the higher abnormalities of negative and positive valence systems (60% and 51.9% of veterans respectively) compared to non-veterans (59.1% and 49.2% respectively). The domains following first diagnoses of PTSD were associated with heightened cue reactivity to trauma, suicide, alcohol, and substance consumption. CONCLUSIONS The findings provide initial insights into RDoC functioning in different populations and disease trajectories. Natural language processing proves valuable for capturing real-time, context dependent RDoC instances from extensive clinical notes.
Collapse
Affiliation(s)
- Oshin Miranda
- Computational Chemical Genomics Screening Center, Department of Pharmaceutical Sciences, School of Pharmacy, University of Pittsburgh, Pittsburgh, PA, 15213, USA
| | | | - Xiguang Qi
- Computational Chemical Genomics Screening Center, Department of Pharmaceutical Sciences, School of Pharmacy, University of Pittsburgh, Pittsburgh, PA, 15213, USA
| | | | - Thomas Kosten
- Menninger Department of Psychiatry, Baylor College of Medicine, Houston, TX, 77030, USA
| | - Neal David Ryan
- Department of Psychiatry, School of Medicine, University of Pittsburgh, Pittsburgh, PA, 15213, USA
| | - Levent Kirisci
- University of Pittsburgh School of Pharmacy, Pittsburgh, PA, 15213, USA
| | - Yanshan Wang
- University of Pittsburgh School of Health and Rehabilitation Sciences, Pittsburgh, PA, 15213, USA
| | - LiRong Wang
- Computational Chemical Genomics Screening Center, Department of Pharmaceutical Sciences, School of Pharmacy, University of Pittsburgh, Pittsburgh, PA, 15213, USA.
| |
Collapse
|
2
|
Liewlom P. Descriptive forest: experiments on a novel tree-structure-generalization method for describing cardiovascular diseases. BMC Med Inform Decis Mak 2023; 23:141. [PMID: 37507769 PMCID: PMC10386781 DOI: 10.1186/s12911-023-02228-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2023] [Accepted: 07/07/2023] [Indexed: 07/30/2023] Open
Abstract
BACKGROUND A decision tree is a crucial tool for describing the factors related to cardiovascular disease (CVD) risk and for predicting and explaining it for patients. Notably, the decision tree must be simplified because patients may have different primary topics or factors related to the CVD risk. Many decision trees can describe the data collected from multiple environmental heart disease risk datasets or a forest, where each tree describes the CVD risk for each primary topic. METHODS We demonstrate the presence of trees, or a forest, using an integrated CVD dataset obtained from multiple datasets. Moreover, we apply a novel method to an association-rule tree to discover each primary topic hidden within a dataset. To generalize the tree structure for descriptive tasks, each primary topic is a boundary node acting as a root node of a C4.5 tree with the least prodigality for the tree structure (PTS). All trees are assigned to a descriptive forest describing the CVD risks in a dataset. A descriptive forest is used to describe each CVD patient's primary risk topics and related factors. We describe eight primary topics in a descriptive forest acquired from 918 records of a heart failure-prediction dataset with 11 features obtained from five datasets. We apply the proposed method to 253,680 records with 22 features from imbalanced classes of a heart disease health-indicators dataset. RESULTS The usability of the descriptive forest is demonstrated by a comparative study (on qualitative and quantitative tasks of the CVD-risk explanation) with a C4.5 tree generated from the same dataset but with the least PTS. The qualitative descriptive task confirms that compared to a single C4.5 tree, the descriptive forest is more flexible and can better describe the CVD risk, whereas the quantitative descriptive task confirms that it achieved higher coverage (recall) and correctness (accuracy and precision) and provided more detailed explanations. Additionally, for these tasks, the descriptive forest still outperforms the C4.5 tree. To reduce the problem of imbalanced classes, the ratio of classes in each subdataset generating each tree is investigated. CONCLUSION The results provide confidence for using the descriptive forest.
Collapse
Affiliation(s)
- Peera Liewlom
- Department of Computer and Information Science, Faculty of Science and Engineering, Kasetsart University, Chalermphrakiat Sakonnakhon Province Campus, Sakonnakhon, 47000, Thailand.
| |
Collapse
|
3
|
Goldman HH, Porcino J, Divita G, Zirikly A, Desmet B, Sacco M, Marfeo E, McDonough C, Rasch E, Chan L. Informatics Research on Mental Health Functioning: Decision Support for the Social Security Administration Disability Program. Psychiatr Serv 2023; 74:56-62. [PMID: 35652194 PMCID: PMC10501504 DOI: 10.1176/appi.ps.202200056] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/04/2023]
Abstract
The disability determination process of the Social Security Administration's (SSA's) disability program requires assessing work-related functioning for individual claimants alleging disability due to mental impairment. This task is particularly challenging because the determination process involves the review of a large file of information, including objective medical evidence and self-reports from claimants, families, and former employers. To improve this decision-making process, SSA entered an interagency agreement with the Rehabilitation Medicine Department, Epidemiology and Biostatistics Section, in the Clinical Center of the National Institutes of Health, intending to use data science and informatics to develop decision support tools. This collaborative effort over the past decade has led to the development of the Work Disability-Functional Assessment Battery and has initiated an approach to applying natural language processing to the review of claimants' files for information on mental health functioning. This informatics research collaboration holds promise for improving the process of disability determination for individuals with mental impairments who make claims at the SSA.
Collapse
Affiliation(s)
- Howard H Goldman
- Rehabilitation Medicine Department, National Institutes of Health Clinical Center, Bethesda, Maryland (all authors); Department of Psychiatry, University of Maryland School of Medicine, Baltimore (Goldman); Department of Computer Science, Johns Hopkins University, Baltimore (Zirikly); Department of Occupational Therapy, Tufts University, Medford, Massachusetts (Marfeo); Department of Physical Therapy, University of Pittsburgh, Pittsburgh (McDonough)
| | - Julia Porcino
- Rehabilitation Medicine Department, National Institutes of Health Clinical Center, Bethesda, Maryland (all authors); Department of Psychiatry, University of Maryland School of Medicine, Baltimore (Goldman); Department of Computer Science, Johns Hopkins University, Baltimore (Zirikly); Department of Occupational Therapy, Tufts University, Medford, Massachusetts (Marfeo); Department of Physical Therapy, University of Pittsburgh, Pittsburgh (McDonough)
| | - Guy Divita
- Rehabilitation Medicine Department, National Institutes of Health Clinical Center, Bethesda, Maryland (all authors); Department of Psychiatry, University of Maryland School of Medicine, Baltimore (Goldman); Department of Computer Science, Johns Hopkins University, Baltimore (Zirikly); Department of Occupational Therapy, Tufts University, Medford, Massachusetts (Marfeo); Department of Physical Therapy, University of Pittsburgh, Pittsburgh (McDonough)
| | - Ayah Zirikly
- Rehabilitation Medicine Department, National Institutes of Health Clinical Center, Bethesda, Maryland (all authors); Department of Psychiatry, University of Maryland School of Medicine, Baltimore (Goldman); Department of Computer Science, Johns Hopkins University, Baltimore (Zirikly); Department of Occupational Therapy, Tufts University, Medford, Massachusetts (Marfeo); Department of Physical Therapy, University of Pittsburgh, Pittsburgh (McDonough)
| | - Bart Desmet
- Rehabilitation Medicine Department, National Institutes of Health Clinical Center, Bethesda, Maryland (all authors); Department of Psychiatry, University of Maryland School of Medicine, Baltimore (Goldman); Department of Computer Science, Johns Hopkins University, Baltimore (Zirikly); Department of Occupational Therapy, Tufts University, Medford, Massachusetts (Marfeo); Department of Physical Therapy, University of Pittsburgh, Pittsburgh (McDonough)
| | - Maryanne Sacco
- Rehabilitation Medicine Department, National Institutes of Health Clinical Center, Bethesda, Maryland (all authors); Department of Psychiatry, University of Maryland School of Medicine, Baltimore (Goldman); Department of Computer Science, Johns Hopkins University, Baltimore (Zirikly); Department of Occupational Therapy, Tufts University, Medford, Massachusetts (Marfeo); Department of Physical Therapy, University of Pittsburgh, Pittsburgh (McDonough)
| | - Elizabeth Marfeo
- Rehabilitation Medicine Department, National Institutes of Health Clinical Center, Bethesda, Maryland (all authors); Department of Psychiatry, University of Maryland School of Medicine, Baltimore (Goldman); Department of Computer Science, Johns Hopkins University, Baltimore (Zirikly); Department of Occupational Therapy, Tufts University, Medford, Massachusetts (Marfeo); Department of Physical Therapy, University of Pittsburgh, Pittsburgh (McDonough)
| | - Christine McDonough
- Rehabilitation Medicine Department, National Institutes of Health Clinical Center, Bethesda, Maryland (all authors); Department of Psychiatry, University of Maryland School of Medicine, Baltimore (Goldman); Department of Computer Science, Johns Hopkins University, Baltimore (Zirikly); Department of Occupational Therapy, Tufts University, Medford, Massachusetts (Marfeo); Department of Physical Therapy, University of Pittsburgh, Pittsburgh (McDonough)
| | - Elizabeth Rasch
- Rehabilitation Medicine Department, National Institutes of Health Clinical Center, Bethesda, Maryland (all authors); Department of Psychiatry, University of Maryland School of Medicine, Baltimore (Goldman); Department of Computer Science, Johns Hopkins University, Baltimore (Zirikly); Department of Occupational Therapy, Tufts University, Medford, Massachusetts (Marfeo); Department of Physical Therapy, University of Pittsburgh, Pittsburgh (McDonough)
| | - Leighton Chan
- Rehabilitation Medicine Department, National Institutes of Health Clinical Center, Bethesda, Maryland (all authors); Department of Psychiatry, University of Maryland School of Medicine, Baltimore (Goldman); Department of Computer Science, Johns Hopkins University, Baltimore (Zirikly); Department of Occupational Therapy, Tufts University, Medford, Massachusetts (Marfeo); Department of Physical Therapy, University of Pittsburgh, Pittsburgh (McDonough)
| |
Collapse
|
4
|
Liu K, Ye R, Zhongzhu L, Ye R. Entropy-based discrimination between translated Chinese and original Chinese using data mining techniques. PLoS One 2022; 17:e0265633. [PMID: 35324927 PMCID: PMC8947138 DOI: 10.1371/journal.pone.0265633] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2021] [Accepted: 03/07/2022] [Indexed: 11/19/2022] Open
Abstract
The present research reports on the use of data mining techniques for differentiating between translated and non-translated original Chinese based on monolingual comparable corpora. We operationalized seven entropy-based metrics including character, wordform unigram, wordform bigram and wordform trigram, POS (Part-of-speech) unigram, POS bigram and POS trigram entropy from two balanced Chinese comparable corpora (translated vs non-translated) for data mining and analysis. We then applied four data mining techniques including Support Vector Machines (SVMs), Linear discriminant analysis (LDA), Random Forest (RF) and Multilayer Perceptron (MLP) to distinguish translated Chinese from original Chinese based on these seven features. Our results show that SVMs is the most robust and effective classifier, yielding an AUC of 90.5% and an accuracy rate of 84.3%. Our results have affirmed the hypothesis that translational language is categorically different from original language. Our research demonstrates that combining information-theoretic indicator of Shannon’s entropy together with machine learning techniques can provide a novel approach for studying translation as a unique communicative activity. This study has yielded new insights for corpus-based studies on the translationese phenomenon in the field of translation studies.
Collapse
Affiliation(s)
- Kanglong Liu
- Department of Chinese and Bilingual Studies, The Hong Kong Polytechnic University, Hong Kong, China
- * E-mail:
| | - Rongguang Ye
- School of Applied Mathematics, Guangdong University of Technology, Guangdong, China
| | - Liu Zhongzhu
- Department of Mathematics and Statistics, Huizhou University, Guangdong, China
| | - Rongye Ye
- Department of Mathematics and Statistics, Huizhou University, Guangdong, China
| |
Collapse
|
5
|
Le Glaz A, Haralambous Y, Kim-Dufor DH, Lenca P, Billot R, Ryan TC, Marsh J, DeVylder J, Walter M, Berrouiguet S, Lemey C. Machine Learning and Natural Language Processing in Mental Health: Systematic Review. J Med Internet Res 2021; 23:e15708. [PMID: 33944788 PMCID: PMC8132982 DOI: 10.2196/15708] [Citation(s) in RCA: 79] [Impact Index Per Article: 26.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2019] [Revised: 04/18/2020] [Accepted: 10/02/2020] [Indexed: 01/22/2023] Open
Abstract
BACKGROUND Machine learning systems are part of the field of artificial intelligence that automatically learn models from data to make better decisions. Natural language processing (NLP), by using corpora and learning approaches, provides good performance in statistical tasks, such as text classification or sentiment mining. OBJECTIVE The primary aim of this systematic review was to summarize and characterize, in methodological and technical terms, studies that used machine learning and NLP techniques for mental health. The secondary aim was to consider the potential use of these methods in mental health clinical practice. METHODS This systematic review follows the PRISMA (Preferred Reporting Items for Systematic Review and Meta-analysis) guidelines and is registered with PROSPERO (Prospective Register of Systematic Reviews; number CRD42019107376). The search was conducted using 4 medical databases (PubMed, Scopus, ScienceDirect, and PsycINFO) with the following keywords: machine learning, data mining, psychiatry, mental health, and mental disorder. The exclusion criteria were as follows: languages other than English, anonymization process, case studies, conference papers, and reviews. No limitations on publication dates were imposed. RESULTS A total of 327 articles were identified, of which 269 (82.3%) were excluded and 58 (17.7%) were included in the review. The results were organized through a qualitative perspective. Although studies had heterogeneous topics and methods, some themes emerged. Population studies could be grouped into 3 categories: patients included in medical databases, patients who came to the emergency room, and social media users. The main objectives were to extract symptoms, classify severity of illness, compare therapy effectiveness, provide psychopathological clues, and challenge the current nosography. Medical records and social media were the 2 major data sources. With regard to the methods used, preprocessing used the standard methods of NLP and unique identifier extraction dedicated to medical texts. Efficient classifiers were preferred rather than transparent functioning classifiers. Python was the most frequently used platform. CONCLUSIONS Machine learning and NLP models have been highly topical issues in medicine in recent years and may be considered a new paradigm in medical research. However, these processes tend to confirm clinical hypotheses rather than developing entirely new information, and only one major category of the population (ie, social media users) is an imprecise cohort. Moreover, some language-specific features can improve the performance of NLP methods, and their extension to other languages should be more closely investigated. However, machine learning and NLP techniques provide useful information from unexplored data (ie, patients' daily habits that are usually inaccessible to care providers). Before considering It as an additional tool of mental health care, ethical issues remain and should be discussed in a timely manner. Machine learning and NLP methods may offer multiple perspectives in mental health research but should also be considered as tools to support clinical practice.
Collapse
Affiliation(s)
- Aziliz Le Glaz
- URCI Mental Health Department, Brest Medical University Hospital, Brest, France
| | | | - Deok-Hee Kim-Dufor
- URCI Mental Health Department, Brest Medical University Hospital, Brest, France
| | - Philippe Lenca
- IMT Atlantique, Lab-STICC, UMR CNRS 6285, F-29238, Brest, France
| | - Romain Billot
- IMT Atlantique, Lab-STICC, UMR CNRS 6285, F-29238, Brest, France
| | - Taylor C Ryan
- Department of Mental Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, United States
| | - Jonathan Marsh
- Fordham University Graduate School of Social Service, New York, NY, United States
| | - Jordan DeVylder
- Fordham University Graduate School of Social Service, New York, NY, United States
| | - Michel Walter
- URCI Mental Health Department, Brest Medical University Hospital, Brest, France
- EA 7479 SPURBO, Université de Bretagne Occidentale, Brest, France
| | - Sofian Berrouiguet
- URCI Mental Health Department, Brest Medical University Hospital, Brest, France
- IMT Atlantique, Lab-STICC, UMR CNRS 6285, F-29238, Brest, France
- EA 7479 SPURBO, Université de Bretagne Occidentale, Brest, France
- LaTIM, INSERM, UMR 1101, Brest, France
| | - Christophe Lemey
- URCI Mental Health Department, Brest Medical University Hospital, Brest, France
- IMT Atlantique, Lab-STICC, UMR CNRS 6285, F-29238, Brest, France
- EA 7479 SPURBO, Université de Bretagne Occidentale, Brest, France
| |
Collapse
|
6
|
Spasic I, Nenadic G. Clinical Text Data in Machine Learning: Systematic Review. JMIR Med Inform 2020; 8:e17984. [PMID: 32229465 PMCID: PMC7157505 DOI: 10.2196/17984] [Citation(s) in RCA: 111] [Impact Index Per Article: 27.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2020] [Revised: 02/24/2020] [Accepted: 02/24/2020] [Indexed: 12/22/2022] Open
Abstract
Background Clinical narratives represent the main form of communication within health care, providing a personalized account of patient history and assessments, and offering rich information for clinical decision making. Natural language processing (NLP) has repeatedly demonstrated its feasibility to unlock evidence buried in clinical narratives. Machine learning can facilitate rapid development of NLP tools by leveraging large amounts of text data. Objective The main aim of this study was to provide systematic evidence on the properties of text data used to train machine learning approaches to clinical NLP. We also investigated the types of NLP tasks that have been supported by machine learning and how they can be applied in clinical practice. Methods Our methodology was based on the guidelines for performing systematic reviews. In August 2018, we used PubMed, a multifaceted interface, to perform a literature search against MEDLINE. We identified 110 relevant studies and extracted information about text data used to support machine learning, NLP tasks supported, and their clinical applications. The data properties considered included their size, provenance, collection methods, annotation, and any relevant statistics. Results The majority of datasets used to train machine learning models included only hundreds or thousands of documents. Only 10 studies used tens of thousands of documents, with a handful of studies utilizing more. Relatively small datasets were utilized for training even when much larger datasets were available. The main reason for such poor data utilization is the annotation bottleneck faced by supervised machine learning algorithms. Active learning was explored to iteratively sample a subset of data for manual annotation as a strategy for minimizing the annotation effort while maximizing the predictive performance of the model. Supervised learning was successfully used where clinical codes integrated with free-text notes into electronic health records were utilized as class labels. Similarly, distant supervision was used to utilize an existing knowledge base to automatically annotate raw text. Where manual annotation was unavoidable, crowdsourcing was explored, but it remains unsuitable because of the sensitive nature of data considered. Besides the small volume, training data were typically sourced from a small number of institutions, thus offering no hard evidence about the transferability of machine learning models. The majority of studies focused on text classification. Most commonly, the classification results were used to support phenotyping, prognosis, care improvement, resource management, and surveillance. Conclusions We identified the data annotation bottleneck as one of the key obstacles to machine learning approaches in clinical NLP. Active learning and distant supervision were explored as a way of saving the annotation efforts. Future research in this field would benefit from alternatives such as data augmentation and transfer learning, or unsupervised learning, which do not require data annotation.
Collapse
Affiliation(s)
- Irena Spasic
- School of Computer Science and Informatics, Cardiff University, Cardiff, United Kingdom
| | - Goran Nenadic
- Department of Computer Science, University of Manchester, Manchester, United Kingdom
| |
Collapse
|
7
|
Follett L, Geletta S, Laugerman M. Quantifying risk associated with clinical trial termination: A text mining approach. Inf Process Manag 2019. [DOI: 10.1016/j.ipm.2018.11.009] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
|
8
|
Dai HJ, Jonnagaddala J. Assessing the severity of positive valence symptoms in initial psychiatric evaluation records: Should we use convolutional neural networks? PLoS One 2018; 13:e0204493. [PMID: 30325934 PMCID: PMC6191093 DOI: 10.1371/journal.pone.0204493] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2018] [Accepted: 09/10/2018] [Indexed: 11/18/2022] Open
Abstract
BACKGROUND AND OBJECTIVE Efficiently capturing the severity of positive valence symptoms could aid in risk stratification for adverse outcomes among patients with psychiatric disorders and identify optimal treatment strategies for patient subgroups. Motivated by the success of convolutional neural networks (CNNs) in classification tasks, we studied the application of various CNN architectures and their performance in predicting the severity of positive valence symptoms in patients with psychiatric disorders based on initial psychiatric evaluation records. METHODS Psychiatric evaluation records contain unstructured text and semi-structured data such as question-answer pairs. For a given record, we tokenise and normalise the semi-structured content. Pre-processed tokenised words are represented as one-hot encoded word vectors. We then apply different configurations of convolutional and max pooling layers to automatically learn important features from various word representations. We conducted a series of experiments to explore the effect of different CNN architectures on the classification of psychiatric records. RESULTS Our best CNN model achieved a mean absolute error (MAE) of 0.539 and a normalized MAE of 0.785 on the test dataset, which is comparable to the other well-known text classification algorithms studied in this work. Our results also suggest that the normalisation step has a great impact on the performance of the developed models. CONCLUSIONS We demonstrate that normalisation of the semi-structured contents can improve the MAE among all CNN configurations. Without advanced feature engineering, CNN-based approaches can provide a comparable solution for classifying positive valence symptom severity in initial psychiatric evaluation records. Although word embedding is well known for its ability to capture relatively low-dimensional similarity between words, our experimental results show that pre-trained embeddings do not improve the classification performance. This phenomenon may be due to the inability of word embeddings to capture problem specific contextual semantic information implying the quality of the employing embedding is critical for obtaining an accurate CNN model.
Collapse
Affiliation(s)
- Hong-Jie Dai
- Department of Computer Science and Information Engineering, National Taitung University, Taitung, Taiwan
- Interdisciplinary Program of Green and Information Technology, National Taitung University, Taitung, Taiwan
| | | |
Collapse
|
9
|
Lacson RC, Baker B, Suresh H, Andriole K, Szolovits P, Lacson E. Use of machine-learning algorithms to determine features of systolic blood pressure variability that predict poor outcomes in hypertensive patients. Clin Kidney J 2018; 12:206-212. [PMID: 30976397 PMCID: PMC6452173 DOI: 10.1093/ckj/sfy049] [Citation(s) in RCA: 25] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2018] [Indexed: 12/12/2022] Open
Abstract
Background We re-analyzed data from the Systolic Blood Pressure Intervention Trial (SPRINT) trial to identify features of systolic blood pressure (SBP) variability that portend poor cardiovascular outcomes using a nonlinear machine-learning algorithm. Methods We included all patients who completed 1 year of the study without reaching any primary endpoint during the first year, specifically: myocardial infarction, other acute coronary syndromes, stroke, heart failure or death from a cardiovascular event (n = 8799; 94%). In addition to clinical variables, features representing longitudinal SBP trends and variability were determined and combined in a random forest algorithm, optimized using cross-validation, using 70% of patients in the training set. Area under the curve (AUC) was measured using a 30% testing set. Finally, feature importance was determined by minimizing node impurity averaging over all trees in the forest for a specific feature. Results A total of 365 patients (4.1%) reached the combined primary outcome over 37 months of follow-up. The random forest classifier had an AUC of 0.71 on the testing set. The 10 most significant features selected in order of importance by the automated algorithm included the urine albumin/creatinine (CR) ratio, estimated glomerular filtration rate, age, serum CR, history of subclinical cardiovascular disease (CVD), cholesterol, a variable representing SBP signals using wavelet transformation, high-density lipoprotein, the 90th percentile of SBP and triglyceride level. Conclusions We successfully demonstrated use of random forest algorithm to define best prognostic longitudinal SBP representations. In addition to known risk factors for CVD, transformed variables for time series SBP measurements were found to be important in predicting poor cardiovascular outcomes and require further evaluation.
Collapse
Affiliation(s)
- Ronilda C Lacson
- Brigham and Women’s Hospital, Boston MA, USA
- Harvard Medical School, Boston, MA, USA
- Correspondence and offprint requests to: Ronilda C. Lacson; E-mail:
| | - Bowen Baker
- Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Harini Suresh
- Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Katherine Andriole
- Brigham and Women’s Hospital, Boston MA, USA
- Harvard Medical School, Boston, MA, USA
| | - Peter Szolovits
- Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Eduardo Lacson
- Dialysis Clinic, Inc., Nashville, TN, USA
- Tufts Medical Center, Boston, MA, USA
| |
Collapse
|
10
|
Patient representation learning and interpretable evaluation using clinical notes. J Biomed Inform 2018; 84:103-113. [PMID: 29966746 DOI: 10.1016/j.jbi.2018.06.016] [Citation(s) in RCA: 21] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2018] [Revised: 06/07/2018] [Accepted: 06/28/2018] [Indexed: 11/22/2022]
Abstract
We have three contributions in this work: 1. We explore the utility of a stacked denoising autoencoder and a paragraph vector model to learn task-independent dense patient representations directly from clinical notes. To analyze if these representations are transferable across tasks, we evaluate them in multiple supervised setups to predict patient mortality, primary diagnostic and procedural category, and gender. We compare their performance with sparse representations obtained from a bag-of-words model. We observe that the learned generalized representations significantly outperform the sparse representations when we have few positive instances to learn from, and there is an absence of strong lexical features. 2. We compare the model performance of the feature set constructed from a bag of words to that obtained from medical concepts. In the latter case, concepts represent problems, treatments, and tests. We find that concept identification does not improve the classification performance. 3. We propose novel techniques to facilitate model interpretability. To understand and interpret the representations, we explore the best encoded features within the patient representations obtained from the autoencoder model. Further, we calculate feature sensitivity across two networks to identify the most significant input features for different classification tasks when we use these pretrained representations as the supervised input. We successfully extract the most influential features for the pipeline using this technique.
Collapse
|
11
|
Uzuner Ö, Stubbs A, Filannino M. A natural language processing challenge for clinical records: Research Domains Criteria (RDoC) for psychiatry. J Biomed Inform 2017; 75S:S1-S3. [PMID: 29042245 DOI: 10.1016/j.jbi.2017.10.005] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2017] [Revised: 10/06/2017] [Accepted: 10/09/2017] [Indexed: 02/06/2023]
Affiliation(s)
- Özlem Uzuner
- Department of Information Sciences and Technology, George Mason University, 4400 University Drive 5359, Nguyen Engineering Bldg, MS 1G8, Fairfax, VA 22030, USA. Tel.: +703-993-8633.
| | - Amber Stubbs
- School of Library and Information Science, Simmons College, Boston, MA, USA.
| | - Michele Filannino
- Department of Computer Science, State University of New York at Albany, Albany, NY, USA.
| |
Collapse
|
12
|
Filannino M, Stubbs A, Uzuner Ö. Symptom severity prediction from neuropsychiatric clinical records: Overview of 2016 CEGS N-GRID shared tasks Track 2. J Biomed Inform 2017; 75S:S62-S70. [PMID: 28455151 DOI: 10.1016/j.jbi.2017.04.017] [Citation(s) in RCA: 26] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2017] [Revised: 04/22/2017] [Accepted: 04/24/2017] [Indexed: 11/18/2022]
Abstract
The second track of the CEGS N-GRID 2016 natural language processing shared tasks focused on predicting symptom severity from neuropsychiatric clinical records. For the first time, initial psychiatric evaluation records have been collected, de-identified, annotated and shared with the scientific community. One-hundred-ten researchers organized in twenty-four teams participated in this track and submitted sixty-five system runs for evaluation. The top ten teams each achieved an inverse normalized macro-averaged mean absolute error score over 0.80. The top performing system employed an ensemble of six different machine learning-based classifiers to achieve a score 0.86. The task resulted to be generally easy with the exception of two specific classes of records: records with very few but crucial positive valence signals, and records describing patients predominantly affected by negative rather than positive valence. Those cases proved to be very challenging for most of the systems. Further research is required to consider the task solved. Overall, the results of this track demonstrate the effectiveness of data-driven approaches to the task of symptom severity classification.
Collapse
Affiliation(s)
- Michele Filannino
- University at Albany, State University of New York, Albany, NY, USA.
| | | | - Özlem Uzuner
- University at Albany, State University of New York, Albany, NY, USA
| |
Collapse
|