1
|
Han R, Zhang Z, Wei H, Yin D. Chinese medical event detection based on event frequency distribution ratio and document consistency. MATHEMATICAL BIOSCIENCES AND ENGINEERING : MBE 2023; 20:11063-11080. [PMID: 37322971 DOI: 10.3934/mbe.2023489] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/17/2023]
Abstract
Structured information especially medical events extracted from electronic medical records has extremely practical application value and play a basic role in various intelligent diagnosis and treatment systems. Fine-grained Chinese medical event detection is crucial in the process of structuring Chinese Electronic Medical Record (EMR). The current methods for detecting fine-grained Chinese medical events primarily rely on statistical machine learning and deep learning. However, they have two shortcomings: 1) they neglect to take into account the distribution characteristics of these fine-grained medical events. 2) they overlook the consistency in the distribution of medical events within each individual document. Therefore, this paper presents a fine-grained Chinese medical event detection method, which is based on event frequency distribution ratio and document consistency. To start with, a significant number of Chinese EMR texts are used to adapt the Chinese pre-training model BERT to the domain. Second, based on the fundamental features, the Event Frequency - Event Distribution Ratio (EF-DR) is devised to select distinct event information as supplementary features, taking into account the distribution of events within the EMR. Finally, using EMR document consistency within the model improves the outcome of event detection. Our experiments demonstrate that the proposed method significantly outperforms the baseline model.
Collapse
Affiliation(s)
- Ruirui Han
- College of Computer Science and Engineering, Northwest Normal University, 967 Anning East Road, Lanzhou 730070, China
| | - Zhichang Zhang
- College of Computer Science and Engineering, Northwest Normal University, 967 Anning East Road, Lanzhou 730070, China
| | - Hao Wei
- College of Computer Science and Engineering, Northwest Normal University, 967 Anning East Road, Lanzhou 730070, China
| | - Deyue Yin
- College of Computer Science and Engineering, Northwest Normal University, 967 Anning East Road, Lanzhou 730070, China
| |
Collapse
|
2
|
Time expression recognition and normalization: a survey. Artif Intell Rev 2023. [DOI: 10.1007/s10462-023-10400-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/26/2023]
|
3
|
Zhang Z, Liu D, Zhang M, Qin X. Combining data augmentation and domain information with TENER model for Clinical Event Detection. BMC Med Inform Decis Mak 2021; 21:261. [PMID: 34789246 PMCID: PMC8596895 DOI: 10.1186/s12911-021-01618-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2021] [Accepted: 08/23/2021] [Indexed: 11/19/2022] Open
Abstract
Background In recent years, with the development of artificial intelligence, the use of deep learning technology for clinical information extraction has become a new trend. Clinical Event Detection (CED) as its subtask has attracted the attention from academia and industry. However, directly applying the advancements in deep learning to CED task often yields unsatisfactory results. The main reasons are due to the following two points: (1) A great number of obscure professional terms in the electronic medical record leads to poor recognition performance of model. (2) The scarcity of datasets required for the task leads to poor model robustness. Therefore, it is urgent to solve these two problems to improve model performance. Methods This paper proposes a combining data augmentation and domain information with TENER Model for Clinical Event Detection. Results We use two evaluation metrics to compare the overall performance of the proposed model with the existing model on the 2012 i2b2 challenge dataset. Experimental results demonstrate that our proposed model achieves the best F1-score of 80.26%, type accuracy of 93% and Span F1-score of 90.33%, and outperforms the state-of-the-art approaches. Conclusions This paper proposes a multi-granularity information fusion encoder-decoder framework, which applies the TENER model to the CED task for the first time. It uses the pre-trained language model (BioBERT) to generate word-level features, solving the problem of a great number of obscure professional terms in the electronic medical record lead to poor recognition performance of model. In addition, this paper proposes a new data augmentation method for sequence labeling tasks, solving the problem of the scarcity of datasets required for the task leads to poor model robustness.
Collapse
Affiliation(s)
- Zhichang Zhang
- College of Computer Science and Engineering, Northwest Normal University, 967 Anning East Road, 730070, Lanzhou, China.
| | - Dan Liu
- College of Computer Science and Engineering, Northwest Normal University, 967 Anning East Road, 730070, Lanzhou, China
| | - Minyu Zhang
- College of Computer Science and Engineering, Northwest Normal University, 967 Anning East Road, 730070, Lanzhou, China
| | - Xiaohui Qin
- College of Computer Science and Engineering, Northwest Normal University, 967 Anning East Road, 730070, Lanzhou, China
| |
Collapse
|
4
|
Alfattni G, Peek N, Nenadic G. Extraction of temporal relations from clinical free text: A systematic review of current approaches. J Biomed Inform 2020; 108:103488. [PMID: 32673788 DOI: 10.1016/j.jbi.2020.103488] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2019] [Revised: 06/10/2020] [Accepted: 06/15/2020] [Indexed: 10/23/2022]
Abstract
BACKGROUND Temporal relations between clinical events play an important role in clinical assessment and decision making. Extracting such relations from free text data is a challenging task because it lies on between medical natural language processing, temporal representation and temporal reasoning. OBJECTIVES To survey existing methods for extracting temporal relations (TLINKs) between events from clinical free text in English; to establish the state-of-the-art in this field; and to identify outstanding methodological challenges. METHODS A systematic search in PubMed and the DBLP computer science bibliography was conducted for studies published between January 2006 and December 2018. The relevant studies were identified by examining the titles and abstracts. Then, the full text of selected studies was analyzed in depth and information were collected on TLINK tasks, TLINK types, data sources, features selection, methods used, and reported performance. RESULTS A total of 2834 publications were identified for title and abstract screening. Of these publications, 51 studies were selected. Thirty-two studies used machine learning approaches, 15 studies used a hybrid approaches, and only four studies used a rule-based approach. The majority of studies use publicly available corpora: THYME (28 studies) and the i2b2 corpus (17 studies). CONCLUSION The performance of TLINK extraction methods ranges widely depending on relation types and events (e.g. from 32% to 87% F-score for identifying relations between clinical events and document creation time). A small set of TLINKs (before, after, overlap and contains) has been widely studied with relatively good performance, whereas other types of TLINK (e.g., started by, finished by, precedes) are rarely studied and remain challenging. Machine learning classifiers (such as Support Vector Machine and Conditional Random Fields) and Deep Neural Networks were among the best performing methods for extracting TLINKs, but nearly all the work has been carried out and tested on two publicly available corpora only. The field would benefit from the availability of more publicly available, high-quality, annotated clinical text corpora.
Collapse
Affiliation(s)
- Ghada Alfattni
- Department of Computer Science, University of Manchester, Manchester, UK; Department of Computer Science, Jamoum University College, Umm Al-Qura University, Makkah, Saudi Arabia.
| | - Niels Peek
- Centre for Health Informatics, Division of Informatics, Imaging and Data Sciences, University of Manchester, Manchester, UK; National Institute of Health Research Manchester Biomedical Research Centre, Manchester Academic Health Science Centre, University of Manchester, Manchester, UK; The Alan Turing Institute, UK
| | - Goran Nenadic
- Department of Computer Science, University of Manchester, Manchester, UK; The Alan Turing Institute, UK
| |
Collapse
|
5
|
Complexities, variations, and errors of numbering within clinical notes: the potential impact on information extraction and cohort-identification. BMC Med Inform Decis Mak 2019; 19:75. [PMID: 30944012 PMCID: PMC6448181 DOI: 10.1186/s12911-019-0784-1] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Numbers and numerical concepts appear frequently in free text clinical notes from electronic health records. Knowledge of the frequent lexical variations of these numerical concepts, and their accurate identification, is important for many information extraction tasks. This paper describes an analysis of the variation in how numbers and numerical concepts are represented in clinical notes. METHODS We used an inverted index of approximately 100 million notes to obtain the frequency of various permutations of numbers and numerical concepts, including the use of Roman numerals, numbers spelled as English words, and invalid dates, among others. Overall, twelve types of lexical variants were analyzed. RESULTS We found substantial variation in how these concepts were represented in the notes, including multiple data quality issues. We also demonstrate that not considering these variations could have substantial real-world implications for cohort identification tasks, with one case missing > 80% of potential patients. CONCLUSIONS Numbering within clinical notes can be variable, and not taking these variations into account could result in missing or inaccurate information for natural language processing and information retrieval tasks.
Collapse
|
6
|
Taylor SJ, Harabagiu SM. The Role of a Deep-Learning Method for Negation Detection in Patient Cohort Identification from Electroencephalography Reports. AMIA ... ANNUAL SYMPOSIUM PROCEEDINGS. AMIA SYMPOSIUM 2018; 2018:1018-1027. [PMID: 30815145 PMCID: PMC6371289] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
Detecting negation in biomedical texts entails the automatic identification of negation cues (e.g. "never", "not", "no longer") as well as the scope of these cues. When medical concepts or terms are identified within the scope of a negation cue, their polarity is inferred as "negative". All the other concepts or words receive a positive polarity. Correctly inferring the polarity is essential for patient cohort retrieval systems, as all inclusion criteria need to be automatically assigned positive polarity, whereas exclusion criteria should receive negative polarity. Motivated by the recent development of techniques using deep learning, we have experimented with a neural negation detection technique and compared it against an existing neural polarity recognition system, which were incorporated in a patient cohort system operating on clinical electroencephalography (EEG) reports. Our experiments indicate that the neural negation detection method produces better patient cohorts then the polarity recognition method.
Collapse
|
7
|
Forsyth AW, Barzilay R, Hughes KS, Lui D, Lorenz KA, Enzinger A, Tulsky JA, Lindvall C. Machine Learning Methods to Extract Documentation of Breast Cancer Symptoms From Electronic Health Records. J Pain Symptom Manage 2018; 55:1492-1499. [PMID: 29496537 DOI: 10.1016/j.jpainsymman.2018.02.016] [Citation(s) in RCA: 39] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/16/2017] [Revised: 02/15/2018] [Accepted: 02/19/2018] [Indexed: 12/25/2022]
Abstract
CONTEXT Clinicians document cancer patients' symptoms in free-text format within electronic health record visit notes. Although symptoms are critically important to quality of life and often herald clinical status changes, computational methods to assess the trajectory of symptoms over time are woefully underdeveloped. OBJECTIVES To create machine learning algorithms capable of extracting patient-reported symptoms from free-text electronic health record notes. METHODS The data set included 103,564 sentences obtained from the electronic clinical notes of 2695 breast cancer patients receiving paclitaxel-containing chemotherapy at two academic cancer centers between May 1996 and May 2015. We manually annotated 10,000 sentences and trained a conditional random field model to predict words indicating an active symptom (positive label), absence of a symptom (negative label), or no symptom at all (neutral label). Sentences labeled by human coder were divided into training, validation, and test data sets. Final model performance was determined on 20% test data unused in model development or tuning. RESULTS The final model achieved precision of 0.82, 0.86, and 0.99 and recall of 0.56, 0.69, and 1.00 for positive, negative, and neutral symptom labels, respectively. The most common positive symptoms were pain, fatigue, and nausea. Machine-based labeling of 103,564 sentences took two minutes. CONCLUSION We demonstrate the potential of machine learning to gather, track, and analyze symptoms experienced by cancer patients during chemotherapy. Although our initial model requires further optimization to improve the performance, further model building may yield machine learning methods suitable to be deployed in routine clinical care, quality improvement, and research applications.
Collapse
Affiliation(s)
- Alexander W Forsyth
- Department of Electrical Engineering and Computer Science, CSAIL, MIT, Cambridge, Massachusetts
| | - Regina Barzilay
- Department of Electrical Engineering and Computer Science, CSAIL, MIT, Cambridge, Massachusetts
| | - Kevin S Hughes
- Department of Surgical Oncology, Massachusetts General Hospital, Boston, Massachusetts
| | - Dickson Lui
- Department of Medicine, Waitemata District Health Board, Auckland, New Zealand
| | - Karl A Lorenz
- Department of Medicine, Primary Care and Population Health, Stanford School of Medicine, Stanford, California; VA Palo Alto Health Care System, Palo Alto, California
| | - Andrea Enzinger
- Division of Population Sciences, Dana-Farber Cancer Institute, Boston, Massachusetts; Department of Psychosocial Oncology and Palliative Care, Dana-Farber Cancer Institute, Boston, Massachusetts; Division of Palliative Medicine, Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts
| | - James A Tulsky
- Department of Psychosocial Oncology and Palliative Care, Dana-Farber Cancer Institute, Boston, Massachusetts; Division of Palliative Medicine, Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts
| | - Charlotta Lindvall
- Department of Psychosocial Oncology and Palliative Care, Dana-Farber Cancer Institute, Boston, Massachusetts; Division of Palliative Medicine, Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts.
| |
Collapse
|
8
|
Seol JW, Yi W, Choi J, Lee KS. Causality patterns and machine learning for the extraction of problem-action relations in discharge summaries. Int J Med Inform 2016; 98:1-12. [PMID: 28034407 DOI: 10.1016/j.ijmedinf.2016.10.021] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2015] [Revised: 08/19/2016] [Accepted: 10/29/2016] [Indexed: 10/20/2022]
Abstract
Clinical narrative text includes information related to a patient's medical history such as chronological progression of medical problems and clinical treatments. A chronological view of a patient's history makes clinical audits easier and improves quality of care. In this paper, we propose a clinical Problem-Action relation extraction method, based on clinical semantic units and event causality patterns, to present a chronological view of a patient's problem and a doctor's action. Based on our observation that a clinical text describes a patient's medical problems and a doctor's treatments in chronological order, a clinical semantic unit is defined as a problem and/or an action relation. Since a clinical event is a basic unit of the problem and action relation, events are extracted from narrative texts, based on the external knowledge resources context features of the conditional random fields. A clinical semantic unit is extracted from each sentence based on time expressions and context structures of events. Then, a clinical semantic unit is classified into a problem and/or action relation based on the event causality patterns of the support vector machines. Experimental results on Korean discharge summaries show 78.8% performance in the F1-measure. This result shows that the proposed method is effectively classifies clinical Problem-Action relations.
Collapse
Affiliation(s)
- Jae-Wook Seol
- Department of Information Convergence Research, Korea Institute of Science and Technology Information 245, Daehak-ro, Yuseong-gu, Daejeon, 34141, Republic of Korea.
| | - Wangjin Yi
- Interdisciplinary Program of Bioengineering, College of Engineering, Seoul National University 103, Daehak-ro, Jongno-gu, Seoul, 03080, Republic of Korea.
| | - Jinwook Choi
- Dept. of Biomedical Engineering, College of Medicine, Seoul National University 103, Daehak-ro, Jongno-gu, Seoul, 03080, Republic of Korea.
| | - Kyung Soon Lee
- Department of Computer Engineering, CAIIT, Chonbuk National University 567, Baekjedae-ro, Deokjin-gu, Jeonju, Jeollabukdo, 54896, Republic of Korea.
| |
Collapse
|
9
|
Madkour M, Benhaddou D, Tao C. Temporal data representation, normalization, extraction, and reasoning: A review from clinical domain. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2016; 128:52-68. [PMID: 27040831 PMCID: PMC4837648 DOI: 10.1016/j.cmpb.2016.02.007] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/18/2015] [Accepted: 02/16/2016] [Indexed: 05/04/2023]
Abstract
BACKGROUND AND OBJECTIVE We live our lives by the calendar and the clock, but time is also an abstraction, even an illusion. The sense of time can be both domain-specific and complex, and is often left implicit, requiring significant domain knowledge to accurately recognize and harness. In the clinical domain, the momentum gained from recent advances in infrastructure and governance practices has enabled the collection of tremendous amount of data at each moment in time. Electronic health records (EHRs) have paved the way to making these data available for practitioners and researchers. However, temporal data representation, normalization, extraction and reasoning are very important in order to mine such massive data and therefore for constructing the clinical timeline. The objective of this work is to provide an overview of the problem of constructing a timeline at the clinical point of care and to summarize the state-of-the-art in processing temporal information of clinical narratives. METHODS This review surveys the methods used in three important area: modeling and representing of time, medical NLP methods for extracting time, and methods of time reasoning and processing. The review emphasis on the current existing gap between present methods and the semantic web technologies and catch up with the possible combinations. RESULTS The main findings of this review are revealing the importance of time processing not only in constructing timelines and clinical decision support systems but also as a vital component of EHR data models and operations. CONCLUSIONS Extracting temporal information in clinical narratives is a challenging task. The inclusion of ontologies and semantic web will lead to better assessment of the annotation task and, together with medical NLP techniques, will help resolving granularity and co-reference resolution problems.
Collapse
Affiliation(s)
- Mohcine Madkour
- School of Biomedical Informatics, University of Texas Health Science Center at Houston, 7000 Fannin St, Houston, TX 77030, United States.
| | - Driss Benhaddou
- Department of Engineering Technology, University of Houston, 4800 Calhoun Rd, Houston, TX 77004, United States.
| | - Cui Tao
- School of Biomedical Informatics, University of Texas Health Science Center at Houston, 7000 Fannin St, Houston, TX 77030, United States.
| |
Collapse
|
10
|
Velupillai S, Mowery DL, Abdelrahman S, Christensen L, Chapman WW. Towards a Generalizable Time Expression Model for Temporal Reasoning in Clinical Notes. AMIA ... ANNUAL SYMPOSIUM PROCEEDINGS. AMIA SYMPOSIUM 2015; 2015:1252-1259. [PMID: 26958265 PMCID: PMC4765564] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Subscribe] [Scholar Register] [Indexed: 06/05/2023]
Abstract
Accurate temporal identification and normalization is imperative for many biomedical and clinical tasks such as generating timelines and identifying phenotypes. A major natural language processing challenge is developing and evaluating a generalizable temporal modeling approach that performs well across corpora and institutions. Our long-term goal is to create such a model. We initiate our work on reaching this goal by focusing on temporal expression (TIMEX3) identification. We present a systematic approach to 1) generalize existing solutions for automated TIMEX3 span detection, and 2) assess similarities and differences by various instantiations of TIMEX3 models applied on separate clinical corpora. When evaluated on the 2012 i2b2 and the 2015 Clinical TempEval challenge corpora, our conclusion is that our approach is successful - we achieve competitive results for automated classification, and we identify similarities and differences in TIMEX3 modeling that will be informative in the development of a simplified, general temporal model.
Collapse
Affiliation(s)
- Sumithra Velupillai
- Department of Computer and Systems Sciences (DSV), Stockholm University, Stockholm, Sweden; Department of Biomedical Informatics, University of Utah, Salt Lake City, UT
| | - Danielle L Mowery
- Department of Biomedical Informatics, University of Utah, Salt Lake City, UT
| | - Samir Abdelrahman
- Department of Biomedical Informatics, University of Utah, Salt Lake City, UT; Computer Science Department, Faculty of Computers and Information, Cairo University, Egypt
| | - Lee Christensen
- Department of Biomedical Informatics, University of Utah, Salt Lake City, UT
| | - Wendy W Chapman
- Department of Biomedical Informatics, University of Utah, Salt Lake City, UT
| |
Collapse
|
11
|
Sun W, Rumshisky A, Uzuner O. Normalization of relative and incomplete temporal expressions in clinical narratives. J Am Med Inform Assoc 2015; 22:1001-8. [PMID: 25868462 DOI: 10.1093/jamia/ocu004] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2014] [Accepted: 10/20/2014] [Indexed: 11/14/2022] Open
Abstract
OBJECTIVE To improve the normalization of relative and incomplete temporal expressions (RI-TIMEXes) in clinical narratives. METHODS We analyzed the RI-TIMEXes in temporally annotated corpora and propose two hypotheses regarding the normalization of RI-TIMEXes in the clinical narrative domain: the anchor point hypothesis and the anchor relation hypothesis. We annotated the RI-TIMEXes in three corpora to study the characteristics of RI-TMEXes in different domains. This informed the design of our RI-TIMEX normalization system for the clinical domain, which consists of an anchor point classifier, an anchor relation classifier, and a rule-based RI-TIMEX text span parser. We experimented with different feature sets and performed an error analysis for each system component. RESULTS The annotation confirmed the hypotheses that we can simplify the RI-TIMEXes normalization task using two multi-label classifiers. Our system achieves anchor point classification, anchor relation classification, and rule-based parsing accuracy of 74.68%, 87.71%, and 57.2% (82.09% under relaxed matching criteria), respectively, on the held-out test set of the 2012 i2b2 temporal relation challenge. DISCUSSION Experiments with feature sets reveal some interesting findings, such as: the verbal tense feature does not inform the anchor relation classification in clinical narratives as much as the tokens near the RI-TIMEX. Error analysis showed that underrepresented anchor point and anchor relation classes are difficult to detect. CONCLUSIONS We formulate the RI-TIMEX normalization problem as a pair of multi-label classification problems. Considering only RI-TIMEX extraction and normalization, the system achieves statistically significant improvement over the RI-TIMEX results of the best systems in the 2012 i2b2 challenge.
Collapse
Affiliation(s)
- Weiyi Sun
- Department of Informatics, University at Albany, SUNY. Albany, NY
| | - Anna Rumshisky
- Department of Computer Science, University of Massachusetts Lowell. Lowell, MA
| | - Ozlem Uzuner
- Department of Information Studies, University at Albany, SUNY. Albany, NY
| |
Collapse
|
12
|
Lin C, Karlson EW, Dligach D, Ramirez MP, Miller TA, Mo H, Braggs NS, Cagan A, Gainer V, Denny JC, Savova GK. Automatic identification of methotrexate-induced liver toxicity in patients with rheumatoid arthritis from the electronic medical record. J Am Med Inform Assoc 2015; 22:e151-61. [PMID: 25344930 PMCID: PMC5901122 DOI: 10.1136/amiajnl-2014-002642] [Citation(s) in RCA: 47] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2014] [Revised: 08/14/2014] [Accepted: 08/22/2014] [Indexed: 12/22/2022] Open
Abstract
OBJECTIVES To improve the accuracy of mining structured and unstructured components of the electronic medical record (EMR) by adding temporal features to automatically identify patients with rheumatoid arthritis (RA) with methotrexate-induced liver transaminase abnormalities. MATERIALS AND METHODS Codified information and a string-matching algorithm were applied to a RA cohort of 5903 patients from Partners HealthCare to select 1130 patients with potential liver toxicity. Supervised machine learning was applied as our key method. For features, Apache clinical Text Analysis and Knowledge Extraction System (cTAKES) was used to extract standard vocabulary from relevant sections of the unstructured clinical narrative. Temporal features were further extracted to assess the temporal relevance of event mentions with regard to the date of transaminase abnormality. All features were encapsulated in a 3-month-long episode for classification. Results were summarized at patient level in a training set (N=480 patients) and evaluated against a test set (N=120 patients). RESULTS The system achieved positive predictive value (PPV) 0.756, sensitivity 0.919, F1 score 0.829 on the test set, which was significantly better than the best baseline system (PPV 0.590, sensitivity 0.703, F1 score 0.642). Our innovations, which included framing the phenotype problem as an episode-level classification task, and adding temporal information, all proved highly effective. CONCLUSIONS Automated methotrexate-induced liver toxicity phenotype discovery for patients with RA based on structured and unstructured information in the EMR shows accurate results. Our work demonstrates that adding temporal features significantly improved classification results.
Collapse
Affiliation(s)
- Chen Lin
- Boston Children's Hospital, Informatics Program, Boston, Massachusetts, USA
- *CL, EWK and DD are co-first authors
| | - Elizabeth W Karlson
- Division of Rheumatology, Immunology and Allergy, Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts, USA
- Harvard Medical School, Boston, Massachusetts, USA
- *CL, EWK and DD are co-first authors
| | - Dmitriy Dligach
- Boston Children's Hospital, Informatics Program, Boston, Massachusetts, USA
- Harvard Medical School, Boston, Massachusetts, USA
- *CL, EWK and DD are co-first authors
| | - Monica P Ramirez
- Division of Rheumatology, Immunology and Allergy, Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts, USA
| | - Timothy A Miller
- Boston Children's Hospital, Informatics Program, Boston, Massachusetts, USA
- Harvard Medical School, Boston, Massachusetts, USA
| | - Huan Mo
- Department of Biomedical Informatics, Vanderbilt University, Nashville, Tennessee, USA
| | - Natalie S Braggs
- Department of Medicine, Vanderbilt University, Nashville, Tennessee, USA
| | - Andrew Cagan
- Research Computing, Partners HealthCare, Boston, Massachusetts, USA
| | - Vivian Gainer
- Research Computing, Partners HealthCare, Boston, Massachusetts, USA
| | - Joshua C Denny
- Department of Biomedical Informatics, Vanderbilt University, Nashville, Tennessee, USA
- Department of Medicine, Vanderbilt University, Nashville, Tennessee, USA
| | - Guergana K Savova
- Boston Children's Hospital, Informatics Program, Boston, Massachusetts, USA
- Harvard Medical School, Boston, Massachusetts, USA
| |
Collapse
|
13
|
Petrov G, Kelle S, Fleck E, Wellnhofer E. Incremental cost-effectiveness of dobutamine stress cardiac magnetic resonance imaging in patients at intermediate risk for coronary artery disease. Clin Res Cardiol 2014; 104:401-9. [PMID: 25395355 PMCID: PMC4544498 DOI: 10.1007/s00392-014-0793-0] [Citation(s) in RCA: 25] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/30/2014] [Accepted: 11/10/2014] [Indexed: 01/12/2023]
Abstract
Aims The effectiveness of stress cardiac magnetic resonance (CMR) as a gatekeeper for coronary angiography (CA) has been established. Level five HTA studies according to the hierarchical model of diagnostic test evaluation are not available. Methods This cohort study included 1,158 consecutive patients (mean age 63 ± 11 years, 42 % women) presenting at our institution between January 1, 2003 and December 31, 2004 with suspected coronary artery disease (CAD) for an elective CA. The patients were assessed for eligibility and propensity score matching was applied to address selection bias regarding the patients’ allocation to CMR or direct CA. Median patient follow-up was 7.9 years (95 % CI 7.8–8.0 years). The primary effect was calculated as relative survival difference. The cost unit calculation (per patient) at our institute was the source of costs. Results Survival was similar in CMR and CA (p = 0.139). Catheterizations ruling out CAD were significantly reduced by the CMR gate-keeper strategy. Patients with prior CMR had significantly lower costs at the initial hospital stay and at follow-up (CMR vs. CA, initial: 2,904€ vs. 3,421€, p = 0.018; follow-up: 2,045€ vs. 3,318€, p = 0.037). CMR was cost-effective in terms of a contribution of 12,466€ per life year to cover a part of the CMR costs. Conclusion Stress CMR prior to CA was saving 12,466€ of hospital costs per life year. Lower costs at follow-up suggest sustained cost-effectiveness of the CMR-guided strategy. Electronic supplementary material The online version of this article (doi:10.1007/s00392-014-0793-0) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- George Petrov
- Department of Internal Medicine/Cardiology, German Heart Institute Berlin, Augustenburger Platz 1, 13353 Berlin, Germany
| | - Sebastian Kelle
- Department of Internal Medicine/Cardiology, German Heart Institute Berlin, Augustenburger Platz 1, 13353 Berlin, Germany
| | - Eckart Fleck
- Department of Internal Medicine/Cardiology, German Heart Institute Berlin, Augustenburger Platz 1, 13353 Berlin, Germany
| | - Ernst Wellnhofer
- Department of Internal Medicine/Cardiology, German Heart Institute Berlin, Augustenburger Platz 1, 13353 Berlin, Germany
| |
Collapse
|
14
|
Pathak J, Kho AN, Denny JC. Electronic health records-driven phenotyping: challenges, recent advances, and perspectives. J Am Med Inform Assoc 2014; 20:e206-11. [PMID: 24302669 DOI: 10.1136/amiajnl-2013-002428] [Citation(s) in RCA: 165] [Impact Index Per Article: 16.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/03/2022] Open
Affiliation(s)
- Jyotishman Pathak
- Department of Health Sciences Research, Mayo Clinic, Rochester, Minnesota, USA
| | | | | |
Collapse
|
15
|
Lin YK, Chen H, Brown RA. MedTime: A temporal information extraction system for clinical narratives. J Biomed Inform 2013; 46 Suppl:S20-S28. [DOI: 10.1016/j.jbi.2013.07.012] [Citation(s) in RCA: 17] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2013] [Revised: 07/12/2013] [Accepted: 07/22/2013] [Indexed: 10/26/2022]
|
16
|
Jindal P, Roth D. Extraction of events and temporal expressions from clinical narratives. J Biomed Inform 2013; 46 Suppl:S13-S19. [PMID: 24022023 DOI: 10.1016/j.jbi.2013.08.010] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/12/2013] [Revised: 08/08/2013] [Accepted: 08/27/2013] [Indexed: 10/26/2022]
Abstract
This paper addresses an important task of event and timex extraction from clinical narratives in context of the i2b2 2012 challenge. State-of-the-art approaches for event extraction use a multi-class classifier for finding the event types. However, such approaches consider each event in isolation. In this paper, we present a sentence-level inference strategy which enforces consistency constraints on attributes of those events which appear close to one another. Our approach is general and can be used for other tasks as well. We also design novel features like clinical descriptors (from medical ontologies) which encode a lot of useful information about the concepts. For timex extraction, we adapt a state-of-the-art system, HeidelTime, for use in clinical narratives and also develop several rules which complement HeidelTime. We also give a robust algorithm for date extraction. For the event extraction task, we achieved an overall F1 score of 0.71 for determining span of the events along with their attributes. For the timex extraction task, we achieved an F1 score of 0.79 for determining span of the temporal expressions. We present detailed error analysis of our system and also point out some factors which can help to improve its accuracy.
Collapse
Affiliation(s)
- Prateek Jindal
- Department of Computer Science, UIUC, 201 N. Goodwin Ave, Urbana, IL 61801, United States.
| | - Dan Roth
- Department of Computer Science, UIUC, 201 N. Goodwin Ave, Urbana, IL 61801, United States.
| |
Collapse
|
17
|
Sun W, Rumshisky A, Uzuner O. Temporal reasoning over clinical text: the state of the art. J Am Med Inform Assoc 2013; 20:814-9. [PMID: 23676245 PMCID: PMC3756277 DOI: 10.1136/amiajnl-2013-001760] [Citation(s) in RCA: 27] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2013] [Revised: 04/17/2013] [Accepted: 04/20/2013] [Indexed: 11/03/2022] Open
Abstract
OBJECTIVES To provide an overview of the problem of temporal reasoning over clinical text and to summarize the state of the art in clinical natural language processing for this task. TARGET AUDIENCE This overview targets medical informatics researchers who are unfamiliar with the problems and applications of temporal reasoning over clinical text. SCOPE We review the major applications of text-based temporal reasoning, describe the challenges for software systems handling temporal information in clinical text, and give an overview of the state of the art. Finally, we present some perspectives on future research directions that emerged during the recent community-wide challenge on text-based temporal reasoning in the clinical domain.
Collapse
Affiliation(s)
- Weiyi Sun
- Department of Informatics, University at Albany, SUNY, Albany, New York, USA
| | - Anna Rumshisky
- Department of Computer Science, University of Massachusetts, Lowell, Massachusetts, USA
| | - Ozlem Uzuner
- Department of Information Studies, University at Albany, SUNY, Albany, New York, USA
| |
Collapse
|
18
|
Sun W, Rumshisky A, Uzuner O. Evaluating temporal relations in clinical text: 2012 i2b2 Challenge. J Am Med Inform Assoc 2013; 20:806-13. [PMID: 23564629 DOI: 10.1136/amiajnl-2013-001628] [Citation(s) in RCA: 184] [Impact Index Per Article: 16.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/03/2022] Open
Abstract
BACKGROUND The Sixth Informatics for Integrating Biology and the Bedside (i2b2) Natural Language Processing Challenge for Clinical Records focused on the temporal relations in clinical narratives. The organizers provided the research community with a corpus of discharge summaries annotated with temporal information, to be used for the development and evaluation of temporal reasoning systems. 18 teams from around the world participated in the challenge. During the workshop, participating teams presented comprehensive reviews and analysis of their systems, and outlined future research directions suggested by the challenge contributions. METHODS The challenge evaluated systems on the information extraction tasks that targeted: (1) clinically significant events, including both clinical concepts such as problems, tests, treatments, and clinical departments, and events relevant to the patient's clinical timeline, such as admissions, transfers between departments, etc; (2) temporal expressions, referring to the dates, times, durations, or frequencies phrases in the clinical text. The values of the extracted temporal expressions had to be normalized to an ISO specification standard; and (3) temporal relations, between the clinical events and temporal expressions. Participants determined pairs of events and temporal expressions that exhibited a temporal relation, and identified the temporal relation between them. RESULTS For event detection, statistical machine learning (ML) methods consistently showed superior performance. While ML and rule based methods seemed to detect temporal expressions equally well, the best systems overwhelmingly adopted a rule based approach for value normalization. For temporal relation classification, the systems using hybrid approaches that combined ML and heuristics based methods produced the best results.
Collapse
Affiliation(s)
- Weiyi Sun
- Department of Informatics, University at Albany, SUNY, Albany, New York, USA
| | | | | |
Collapse
|