Temporal reasoning with medical data--a review with emphasis on medical natural language processing

Development of a Study Protocol for Evaluation of a Novel Measure to Incorporate Information Freshness into Network Analysis of Online Resources for COVID-19

We proposed a novel measure, Degree of Connectivity with Integration of Freshness (DCIF), to incorporate information freshness into analysis of online resource networks. We conducted a pilot study to apply this new measure to a dataset of online information resources related to COVID-19 risk assessment. Among the 52 nodes, we recorded statistically significant difference between the numerical values of DCIF and the traditional structural measure Degree of Connectivity (DC). Manual reviews of 18 selected nodes showed that DCIF outperformed DC in 11 of them, suggesting potential promise of the proposed new measure. We finalized the protocol for manual review based on the pilot and started a full-scale study. The proposed new measure has the potential to provide quantitative assessment on information freshness for timely and effective dissemination of clinical evidence. Further research is required to address the limitations of this pilot study and to examine the generalization of the findings.

Unsupervised Feature-Construction-Based Motor Fault Diagnosis

Any bearing faults are a leading cause of motor damage and bring economic losses. Fast and accurate identification of bearing faults is valuable for preventing damaging the whole equipment and continuously running industrial processes without interruption. Vibration signals from a running motor can be utilized to diagnose a bearing health condition. This study proposes a detection method for bearing faults based on two types of neural networks from motor vibration data. The proposed method uses an autoencoder neural network for constructing a new motor vibration feature and a feed-forward neural network for the final detection. The constructed signal feature enhances the prediction performance by focusing more on a fault type that is difficult to detect. We conducted experiments on the CWRU bearing datasets. The experimental study shows that the proposed method improves the performance of the feed-forward neural network and outperforms the other machine learning algorithms.

A Systematic Temporal Extraction Pipeline for Medical Concepts in Clinical Notes

With increased application of natural language processing (NLP) in medicine, many NLP models are being developed for uncovering relevant clinical features from electronic health records. Temporal information plays a key role in understanding the context, significance, and interpretation of medical concepts extracted from clinical notes. This is particularly true in situations where the behavior, value, or status of a medical concept changes over time. In this paper, we introduce a systematic framework, NLP annotation-Relaxation-Generation (NRG). NRG compiles incidents of medical concept changes from status annotations and timestamps of multiple clinical notes. We demonstrate the effectiveness of the NRG pipeline by applying it to two medical concepts related to patients with inflammatory bowel disease: extra-intestinal manifestations and medications. We show that the NRG pipeline offers not only insights into medical concept changes over time, but can help convey longitudinal changes in clinical features at both individual and population level.

Patient Information Summarization in Clinical Settings: Scoping Review

BACKGROUND

Information overflow, a common problem in the present clinical environment, can be mitigated by summarizing clinical data. Although there are several solutions for clinical summarization, there is a lack of a complete overview of the research relevant to this field.

OBJECTIVE

This study aims to identify state-of-the-art solutions for clinical summarization, to analyze their capabilities, and to identify their properties.

METHODS

A scoping review of articles published between 2005 and 2022 was conducted. With a clinical focus, PubMed and Web of Science were queried to find an initial set of reports, later extended by articles found through a chain of citations. The included reports were analyzed to answer the questions of where, what, and how medical information is summarized; whether summarization conserves temporality, uncertainty, and medical pertinence; and how the propositions are evaluated and deployed. To answer how information is summarized, methods were compared through a new framework "collect-synthesize-communicate" referring to information gathering from data, its synthesis, and communication to the end user.

RESULTS

Overall, 128 articles were included, representing various medical fields. Exclusively structured data were used as input in 46.1% (59/128) of papers, text in 41.4% (53/128) of articles, and both in 10.2% (13/128) of papers. Using the proposed framework, 42.2% (54/128) of the records contributed to information collection, 27.3% (35/128) contributed to information synthesis, and 46.1% (59/128) presented solutions for summary communication. Numerous summarization approaches have been presented, including extractive (n=13) and abstractive summarization (n=19); topic modeling (n=5); summary specification (n=11); concept and relation extraction (n=30); visual design considerations (n=59); and complete pipelines (n=7) using information extraction, synthesis, and communication. Graphical displays (n=53), short texts (n=41), static reports (n=7), and problem-oriented views (n=7) were the most common types in terms of summary communication. Although temporality and uncertainty information were usually not conserved in most studies (74/128, 57.8% and 113/128, 88.3%, respectively), some studies presented solutions to treat this information. Overall, 115 (89.8%) articles showed results of an evaluation, and methods included evaluations with human participants (median 15, IQR 24 participants): measurements in experiments with human participants (n=31), real situations (n=8), and usability studies (n=28). Methods without human involvement included intrinsic evaluation (n=24), performance on a proxy (n=10), or domain-specific tasks (n=11). Overall, 11 (8.6%) reports described a system deployed in clinical settings.

CONCLUSIONS

The scientific literature contains many propositions for summarizing patient information but reports very few comparisons of these proposals. This work proposes to compare these algorithms through how they conserve essential aspects of clinical information and through the "collect-synthesize-communicate" framework. We found that current propositions usually address these 3 steps only partially. Moreover, they conserve and use temporality, uncertainty, and pertinent medical aspects to varying extents, and solutions are often preliminary.

Adversarial Recurrent Time Series Imputation

For the real-world time series analysis, data missing is a ubiquitously existing problem due to anomalies during data collecting and storage. If not treated properly, this problem will seriously hinder the classification, regression, or related tasks. Existing methods for time series imputation either impose too strong assumptions on the distribution of missing data or cannot fully exploit, even simply ignore, the informative temporal dependencies and feature correlations across different time steps. In this article, inspired by the idea of conditional generative adversarial networks, we propose a generative adversarial learning framework for time series imputation under the condition of observed data (as well as the labels, if possible). In our model, we employ a modified bidirectional RNN structure as the generator G, which is aimed at generating the missing values by taking advantage of the temporal and nontemporal information extracted from the observed time series. The discriminator D is designed to distinguish whether each value in a time series is generated or not so that it can help the generator to make an adjustment toward a more authentic imputation result. For an empirical verification of our model, we conduct imputation and classification experiments on several real-world time series data sets. The experimental results show an eminent improvement compared with state-of-the-art baseline models.

BiT-MAC: Mortality prediction by bidirectional time and multi-feature attention coupled network on multivariate irregular time series

Mortality prediction is crucial to evaluate the severity of illness and assist in improving the prognosis of patients. In clinical settings, one way is to analyze the multivariate time series (MTSs) of patients based on their medical data, such as heart rates and invasive mean arterial blood pressure. However, this suffers from sparse, irregularly sampled, and incomplete data issues. These issues can compromise the performance of follow-up MTS-based analytic applications. Plenty of existing methods try to deal with such irregular MTSs with missing values by capturing the temporal dependencies within a time series, yet in-depth research on modeling inter-MTS couplings remains rare and lacks model interpretability. To this end, we propose a bidirectional time and multi-feature attention coupled network (BiT-MAC) to capture the temporal dependencies (i.e., intra-time series coupling) and the hidden relationships among variables (i.e., inter-time series coupling) with a bidirectional recurrent neural network and multi-head attention, respectively. The resulting intra- and inter-time series coupling representations are then fused to estimate the missing values for a more robust MTS-based prediction. We evaluate BiT-MAC by applying it to the missing-data corrupted mortality prediction on two real-world clinical datasets, i.e., PhysioNet'2012 and COVID-19. Extensive experiments demonstrate the superiority of BiT-MAC over cutting-edge models, verifying the great value of the deep and hidden relations captured by MTSs. The interpretability of features is further demonstrated through a case study.

Temporal disambiguation of relative temporal expressions in clinical texts

Temporal expression recognition and normalization (TERN) is the foundation for all higher-level temporal reasoning tasks in natural language processing, such as timeline extraction, so it must be performed well to limit error propagation. Achieving new heights in state-of-the-art performance for TERN in clinical texts requires knowledge of where current systems struggle. In this work, we summarize the results of a detailed error analysis for three top performing state-of-the-art TERN systems that participated in the 2012 i2b2 Clinical Temporal Relation Challenge, and compare our own home-grown system Chrono to identify specific areas in need of improvement. Performance metrics and an error analysis reveal that all systems have reduced performance in normalization of relative temporal expressions, specifically in disambiguating temporal types and in the identification of the correct anchor time. To address the issue of temporal disambiguation we developed and integrated a module into Chrono that utilizes temporally fine-tuned contextual word embeddings to disambiguate relative temporal expressions. Chrono now achieves state-of-the-art performance for temporal disambiguation of relative temporal expressions in clinical text, and is the only TERN system to output dual annotations into both TimeML and SCATE schemes.

Attention-based bidirectional long short-term memory networks for extracting temporal relationships from clinical discharge summaries

Temporal relation extraction between health-related events is a widely studied task in clinical Natural Language Processing (NLP). The current state-of-the-art methods mostly rely on engineered features (i.e., rule-based modelling) and sequence modelling, which often encodes a source sentence into a single fixed-length context. An obvious disadvantage of this fixed-length context design is its incapability to model longer sentences, as important temporal information in the clinical text may appear at different positions. To address this issue, we propose an Attention-based Bidirectional Long Short-Term Memory (Att-BiLSTM) model to enable learning the important semantic information in long source text segments and to better determine which parts of the text are most important. We experimented with two embeddings and compared the performances to traditional state-of-the-art methods that require elaborate linguistic pre-processing and hand-engineered features. The experimental results on the i2b2 2012 temporal relation test corpus show that the proposed method achieves a significant improvement with an F-score of 0.811, which is at least 10% better than state-of-the-art in the field. We show that the model can be remarkably effective at classifying temporal relations when provided with word embeddings trained on corpora in a general domain. Finally, we perform an error analysis to gain insight into the common errors made by the model.

Review of Temporal Reasoning in the Clinical Domain for Timeline Extraction: Where we are and where we need to be

Understanding a patient's medical history, such as how long symptoms last or when a procedure was performed, is vital to diagnosing problems and providing good care. Frequently, important information regarding a patient's medical timeline is buried in their Electronic Health Record (EHR) in the form of unstructured clinical notes. This results in care providers spending time reading notes in a patient's record in order to become familiar with their condition prior to developing a diagnosis or treatment plan. Valuable time could be saved if this information was readily accessible for searching and visualization for fast comprehension by the medical team. Clinical Natural Language Processing (NLP) is an area of research that aims to build computational methods to automatically extract medically relevant information from unstructured clinical texts. A key component of Clinical NLP is Temporal Reasoning, as understanding a patient's medical history relies heavily on the ability to identify, assimilate, and reason over temporal information. In this work, we review the current state of Temporal Reasoning in the clinical domain with respect to Clinical Timeline Extraction. While much progress has been made, the current state-of-the-art still has a ways to go before practical application in the clinical setting will be possible. Areas such as handling relative and implicit temporal expressions, both in normalization and in identifying temporal relationships, improving co-reference resolution, and building inter-operable timeline extraction tools that can integrate multiple types of data are in need of new and innovative solutions to improve performance on clinical data.

Embedding, aligning and reconstructing clinical notes to explore sepsis

Objective

Our goal was to research and develop exploratory analysis tools for clinical notes, which now are underrepresented to limit the diversity of data insights on medically relevant applications.

Results

We characterize how exploratory analysis can affect representation learning on clinical narratives and present several self-developed tools to explore sepsis. Our experiments focus on patients with sepsis in the MIMIC-III Clinical Database or in our institution’s research patient data repository. We found that global embeddings assist in learning local representations of clinical notes. Second, aligning at any specific time facilitates the use of learning models by pooling more available clinical notes to form a training set. Furthermore, reconstruction of the timeline enhances downstream-processing techniques by emphasizing temporal expressions and temporal relationships in clinical documentation. We demonstrate that clustering helps plot various types of clinical notes against a scale, which conveys a sense of the range or spread of the data and is useful for understanding data correlations. Appropriate exploratory analysis tools provide keen insights into preprocessing clinical notes, thereby further enhancing downstream analysis capabilities, making data driven medicine possible. Our examples can help generate better data representation of clinical documentation for models with improved performance and interpretability.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13104-021-05529-4.

Extraction of temporal relations from clinical free text: A systematic review of current approaches

BACKGROUND

Temporal relations between clinical events play an important role in clinical assessment and decision making. Extracting such relations from free text data is a challenging task because it lies on between medical natural language processing, temporal representation and temporal reasoning.

OBJECTIVES

To survey existing methods for extracting temporal relations (TLINKs) between events from clinical free text in English; to establish the state-of-the-art in this field; and to identify outstanding methodological challenges.

METHODS

A systematic search in PubMed and the DBLP computer science bibliography was conducted for studies published between January 2006 and December 2018. The relevant studies were identified by examining the titles and abstracts. Then, the full text of selected studies was analyzed in depth and information were collected on TLINK tasks, TLINK types, data sources, features selection, methods used, and reported performance.

RESULTS

A total of 2834 publications were identified for title and abstract screening. Of these publications, 51 studies were selected. Thirty-two studies used machine learning approaches, 15 studies used a hybrid approaches, and only four studies used a rule-based approach. The majority of studies use publicly available corpora: THYME (28 studies) and the i2b2 corpus (17 studies).

CONCLUSION

The performance of TLINK extraction methods ranges widely depending on relation types and events (e.g. from 32% to 87% F-score for identifying relations between clinical events and document creation time). A small set of TLINKs (before, after, overlap and contains) has been widely studied with relatively good performance, whereas other types of TLINK (e.g., started by, finished by, precedes) are rarely studied and remain challenging. Machine learning classifiers (such as Support Vector Machine and Conditional Random Fields) and Deep Neural Networks were among the best performing methods for extracting TLINKs, but nearly all the work has been carried out and tested on two publicly available corpora only. The field would benefit from the availability of more publicly available, high-quality, annotated clinical text corpora.

Modeling and processing up-to-dateness of patient information in probabilistic therapy decision support

OBJECTIVES

Probabilistic modeling of a patient's situation with the goal of providing calculated therapy recommendations can improve the decision making of interdisciplinary teams. Relevant information entities and direct causal dependencies, as well as uncertainty, must be formally described. Possible therapy options, tailored to the patient, can be inferred from the clinical data using these descriptions. However, there are several avoidable factors of uncertainty influencing the accuracy of the inference. For instance, inaccuracy may emerge from outdated information. In general, probabilistic models, e.g. Bayesian Networks can depict the causality and relations of individual information entities, but in general cannot evaluate individual entities concerning their up-to-dateness. The goal of the work at hand is to model diagnostic up-to-dateness, which can reasonably adjust the influence of outdated diagnostic information to improve the inference results of clinical decision models.

METHODS AND MATERIALS

We analyzed 68 laryngeal cancer cases and modeled the state of up-to-dateness of different diagnostic modalities. All cases were used for cross-validation. 55 cases were used to train the model, 13 for testing. Each diagnostic procedure involved in the decision making process of these cases was associated with a specific threshold for the time the information is considered up-to-date, i.e. reliable. Based on this threshold, outdated findings could be identified and their impact on probabilistic calculations could be reduced. We applied the model for reducing the weight of outdated patient data in the computation of TNM stagings for the 13 test cases and compared the results to the manually derived TNM stagings in the patient files.

RESULTS

With the implementation of these weights in the laryngeal cancer model, we increased the accuracy of the TNM calculation from 0.61 (8 out of 13 cases correct) to 0.76 (10 out of 13 cases correct).

CONCLUSION

Decision delay may cause specific patient data to be outdated. This can cause contradictory or false information and impair calculations for clinical decision support. Our approach demonstrates that the accuracy of Bayesian Network models can be improved when pre-processing the patient-specific data and evaluating their up-to-dateness with reduced weights on outdated information.

Reconstructing the patient's natural history from electronic health records

The automatic extraction of a patient's natural history from Electronic Health Records (EHRs) is a critical step towards building intelligent systems that can reason about clinical variables and support decision making. Although EHRs contain a large amount of valuable information about the patient's medical care, this information can only be fully understood when analyzed in a temporal context. Any intelligent system should then be able to extract medical concepts, date expressions, temporal relations and the temporal ordering of medical events from the free texts of EHRs; yet, this task is hard to tackle, due to the domain specific nature of EHRs, writing quality and lack of structure of these texts, and more generally the presence of redundant information. In this paper, we introduce a new Natural Language Processing (NLP) framework, capable of extracting the aforementioned elements from EHRs written in Spanish using rule-based methods. We focus on building medical timelines, which include disease diagnosis and its progression over time. By using a large dataset of EHRs comprising information about patients suffering from lung cancer, we show that our framework has an adequate level of performance by correctly building the timeline for 843 patients from a pool of 989 patients, achieving a precision of 0.852.

Medi-Care AI: Predicting medications from billing codes via robust recurrent neural networks

In this paper, we present an effective deep prediction framework based on robust recurrent neural networks (RNNs) to predict the likely therapeutic classes of medications a patient is taking, given a sequence of diagnostic billing codes in their record. Accurately capturing the list of medications currently taken by a given patient is extremely challenging due to undefined errors and omissions. We present a general robust framework that explicitly models the possible contamination through overtime decay mechanism on the input billing codes and noise injection into the recurrent hidden states, respectively. By doing this, billing codes are reformulated into its temporal patterns with decay rates on each medical variable, and the hidden states of RNNs are regularized by random noises which serve as dropout to improved RNNs robustness towards data variability in terms of missing values and multiple errors. The proposed method is extensively evaluated on real health care data to demonstrate its effectiveness in suggesting medication orders from contaminated values.

Big Data Defined: A Practical Review for Neurosurgeons

BACKGROUND

Modern science and healthcare generate vast amounts of data, and, coupled with the increasingly inexpensive and accessible computing, a tremendous opportunity exists to use these data to improve care. A better understanding of data science and its relationship to neurosurgical practice will be increasingly important as we transition into this modern "big data" era.

METHODS

A review of the literature was performed for key articles referencing big data for neurosurgical care or related topics.

RESULTS

In the present report, we first defined the nature and scope of data science from a technical perspective. We then discussed its relationship to the modern neurosurgical practice, highlighting key references, which might form a useful introductory reading list.

CONCLUSIONS

Numerous challenges exist going forward; however, organized neurosurgery has an important role in fostering and facilitating these efforts to merge data science with neurosurgical practice.

The use of natural language processing to identify Tdap-related local reactions at five health care systems in the Vaccine Safety Datalink

OBJECTIVE

Local reactions are the most common vaccine-related adverse event. There is no specific diagnosis code for local reaction due to vaccination. Previous vaccine safety studies used non-specific diagnosis codes to identify potential local reaction cases and confirmed the cases through manual chart review. In this study, a natural language processing (NLP) algorithm was developed to identify local reaction associated with tetanus-diphtheria-acellular pertussis (Tdap) vaccine in the Vaccine Safety Datalink.

METHODS

Presumptive cases of local reactions were identified among members ≥ 11 years of age using ICD-9-CM codes in all care settings in the 1-6 days following a Tdap vaccination between 2012 and 2014. The clinical notes were searched for signs and symptoms consistent with local reaction. Information on the timing and the location of a sign or symptom was also extracted to help determine whether or not the sign or symptom was vaccine related. Reactions triggered by causes other than Tdap vaccination were excluded. The NLP algorithm was developed at the lead study site and validated on a stratified random sample of 500 patients from five institutions.

RESULTS

The NLP algorithm achieved an overall weighted sensitivity of 87.9%, specificity of 92.8%, positive predictive value of 82.7%, and negative predictive value of 95.1%. In addition, using data at one site, the NLP algorithm identified 3326 potential Tdap-related local reactions that were not identified through diagnosis codes.

CONCLUSION

The NLP algorithm achieved high accuracy, and demonstrated the potential of NLP to reduce the efforts of manual chart review in vaccine safety studies.

Recurrent Neural Networks for Multivariate Time Series with Missing Values

Multivariate time series data in practical applications, such as health care, geoscience, and biology, are characterized by a variety of missing values. In time series prediction and other related tasks, it has been noted that missing values and their missing patterns are often correlated with the target labels, a.k.a., informative missingness. There is very limited work on exploiting the missing patterns for effective imputation and improving prediction performance. In this paper, we develop novel deep learning models, namely GRU-D, as one of the early attempts. GRU-D is based on Gated Recurrent Unit (GRU), a state-of-the-art recurrent neural network. It takes two representations of missing patterns, i.e., masking and time interval, and effectively incorporates them into a deep model architecture so that it not only captures the long-term temporal dependencies in time series, but also utilizes the missing patterns to achieve better prediction results. Experiments of time series classification tasks on real-world clinical datasets (MIMIC-III, PhysioNet) and synthetic datasets demonstrate that our models achieve state-of-the-art performance and provide useful insights for better understanding and utilization of missing values in time series analysis.

High-fidelity phenotyping: richness and freedom from bias

Electronic health record phenotyping is the use of raw electronic health record data to assert characterizations about patients. Researchers have been doing it since the beginning of biomedical informatics, under different names. Phenotyping will benefit from an increasing focus on fidelity, both in the sense of increasing richness, such as measured levels, degree or severity, timing, probability, or conceptual relationships, and in the sense of reducing bias. Research agendas should shift from merely improving binary assignment to studying and improving richer representations. The field is actively researching new temporal directions and abstract representations, including deep learning. The field would benefit from research in nonlinear dynamics, in combining mechanistic models with empirical data, including data assimilation, and in topology. The health care process produces substantial bias, and studying that bias explicitly rather than treating it as merely another source of noise would facilitate addressing it.

Psychometric validation of a new measurement instrument for time-oriented patient information in electronic medical records: A questionnaire survey of physicians

RATIONALE, AIMS, AND OBJECTIVES

Time is an important element in medical data. Physicians record and store information about patients' disease progress and treatment response in electronic medical records (EMRs). Because EMRs use timestamps, physicians can identify patterns over time regarding a patient's disease and treatment (eg, laboratory values and medications). However, analyses of physicians' use and satisfaction with EMRs have focused on functionality, storage, and system operation rather than the use of time-oriented information. This study aimed to understand physicians' needs regarding time-oriented patient information in EMRs in clinical practice.

METHODS

The reliability and validity of the items in the questionnaire were evaluated in 87 physicians at a national university hospital. Internal consistency was satisfactory (Cronbach alpha coefficient, 0.87).

RESULTS

Four dimensions were identified in exploratory factor analysis. Correlations between the 4 dimensions supported the construct validity of the items. Scores of time-oriented patients' medical history in the 4 dimensions showed a significant association with physician age. Based on confirmatory factor analysis, associations were significant and positive (P < .001). In terms of the needs of physicians regarding time-oriented patient information in EMRs, both time-oriented treatment results followed by time-oriented team information had significant positive associations.

CONCLUSION

Our study suggests that 4 specific time-oriented patient information factors in EMRs are needed by physicians. Exploring physicians' needs regarding patient-specific time-oriented information may provide a better understanding of the barriers facing the adoption and use of EMRs (eg, decision-making and practice safety concerns) and lead to better acceptance of EMRs in physicians' clinical practices.

Temporal data representation, normalization, extraction, and reasoning: A review from clinical domain

BACKGROUND AND OBJECTIVE

We live our lives by the calendar and the clock, but time is also an abstraction, even an illusion. The sense of time can be both domain-specific and complex, and is often left implicit, requiring significant domain knowledge to accurately recognize and harness. In the clinical domain, the momentum gained from recent advances in infrastructure and governance practices has enabled the collection of tremendous amount of data at each moment in time. Electronic health records (EHRs) have paved the way to making these data available for practitioners and researchers. However, temporal data representation, normalization, extraction and reasoning are very important in order to mine such massive data and therefore for constructing the clinical timeline. The objective of this work is to provide an overview of the problem of constructing a timeline at the clinical point of care and to summarize the state-of-the-art in processing temporal information of clinical narratives.

METHODS

This review surveys the methods used in three important area: modeling and representing of time, medical NLP methods for extracting time, and methods of time reasoning and processing. The review emphasis on the current existing gap between present methods and the semantic web technologies and catch up with the possible combinations.

RESULTS

The main findings of this review are revealing the importance of time processing not only in constructing timelines and clinical decision support systems but also as a vital component of EHR data models and operations.

CONCLUSIONS

Extracting temporal information in clinical narratives is a challenging task. The inclusion of ontologies and semantic web will lead to better assessment of the annotation task and, together with medical NLP techniques, will help resolving granularity and co-reference resolution problems.

Classification of hospital acquired complications using temporal clinical information from a large electronic health record

Hospital acquired complications (HACs) are serious problems affecting modern day healthcare institutions. It is estimated that HACs result in an approximately 10% increase in total inpatient hospital costs across US hospitals. With US hospital spending totaling nearly $900 billion per annum, the damages caused by HACs are no small matter. Early detection and prevention of HACs could greatly reduce strains on the US healthcare system and improve patient morbidity & mortality rates. Here, we describe a machine-learning model for predicting the occurrence of HACs within five distinct categories using temporal clinical data. Using our approach, we find that at least $10 billion of excessive hospital costs could be saved in the US alone, with the institution of effective preventive measures. In addition, we also identify several keystone features that demonstrate high predictive power for HACs over different time periods following patient admission. The classifiers and features analyzed in this study show high promise of being able to be used for accurate prediction of HACs in clinical settings, and furthermore provide novel insights into the contribution of various clinical factors to the risk of developing HACs as a function of healthcare system exposure.

Multilayered temporal modeling for the clinical domain

OBJECTIVE

To develop an open-source temporal relation discovery system for the clinical domain. The system is capable of automatically inferring temporal relations between events and time expressions using a multilayered modeling strategy. It can operate at different levels of granularity--from rough temporality expressed as event relations to the document creation time (DCT) to temporal containment to fine-grained classic Allen-style relations.

MATERIALS AND METHODS

We evaluated our systems on 2 clinical corpora. One is a subset of the Temporal Histories of Your Medical Events (THYME) corpus, which was used in SemEval 2015 Task 6: Clinical TempEval. The other is the 2012 Informatics for Integrating Biology and the Bedside (i2b2) challenge corpus. We designed multiple supervised machine learning models to compute the DCT relation and within-sentence temporal relations. For the i2b2 data, we also developed models and rule-based methods to recognize cross-sentence temporal relations. We used the official evaluation scripts of both challenges to make our results comparable with results of other participating systems. In addition, we conducted a feature ablation study to find out the contribution of various features to the system's performance.

RESULTS

Our system achieved state-of-the-art performance on the Clinical TempEval corpus and was on par with the best systems on the i2b2 2012 corpus. Particularly, on the Clinical TempEval corpus, our system established a new F1 score benchmark, statistically significant as compared to the baseline and the best participating system.

CONCLUSION

Presented here is the first open-source clinical temporal relation discovery system. It was built using a multilayered temporal modeling strategy and achieved top performance in 2 major shared tasks.

Parameterizing time in electronic health record studies

BACKGROUND

Fields like nonlinear physics offer methods for analyzing time series, but many methods require that the time series be stationary-no change in properties over time.Objective Medicine is far from stationary, but the challenge may be able to be ameliorated by reparameterizing time because clinicians tend to measure patients more frequently when they are ill and are more likely to vary.

METHODS

We compared time parameterizations, measuring variability of rate of change and magnitude of change, and looking for homogeneity of bins of temporal separation between pairs of time points. We studied four common laboratory tests drawn from 25 years of electronic health records on 4 million patients.

RESULTS

We found that sequence time-that is, simply counting the number of measurements from some start-produced more stationary time series, better explained the variation in values, and had more homogeneous bins than either traditional clock time or a recently proposed intermediate parameterization. Sequence time produced more accurate predictions in a single Gaussian process model experiment.

CONCLUSIONS

Of the three parameterizations, sequence time appeared to produce the most stationary series, possibly because clinicians adjust their sampling to the acuity of the patient. Parameterizing by sequence time may be applicable to association and clustering experiments on electronic health record data. A limitation of this study is that laboratory data were derived from only one institution. Sequence time appears to be an important potential parameterization.

Automated methods for the summarization of electronic health records

Objectives This review examines work on automated summarization of electronic health record (EHR) data and in particular, individual patient record summarization. We organize the published research and highlight methodological challenges in the area of EHR summarization implementation.

Target audience The target audience for this review includes researchers, designers, and informaticians who are concerned about the problem of information overload in the clinical setting as well as both users and developers of clinical summarization systems.

Scope Automated summarization has been a long-studied subject in the fields of natural language processing and human–computer interaction, but the translation of summarization and visualization methods to the complexity of the clinical workflow is slow moving. We assess work in aggregating and visualizing patient information with a particular focus on methods for detecting and removing redundancy, describing temporality, determining salience, accounting for missing data, and taking advantage of encoded clinical knowledge. We identify and discuss open challenges critical to the implementation and use of robust EHR summarization systems.

A use case study on late stent thrombosis for ontology-based temporal reasoning and analysis

In this paper, we show how we have applied the Clinical Narrative Temporal Relation Ontology (CNTRO) and its associated temporal reasoning system (the CNTRO Timeline Library) to trend temporal information within medical device adverse event report narratives. 238 narratives documenting occurrences of late stent thrombosis adverse events from the Food and Drug Administration’s (FDA) Manufacturing and User Facility Device Experience (MAUDE) database were annotated and evaluated using the CNTRO Timeline Library to identify, order, and calculate the duration of temporal events. The CNTRO Timeline Library had a 95% accuracy in correctly ordering events within the 238 narratives. 41 narratives included an event in which the duration was documented, and the CNTRO Timeline Library had an 80% accuracy in correctly determining these durations. 77 narratives included documentation of a duration between events, and the CNTRO Timeline Library had a 76% accuracy in determining these durations. This paper also includes an example of how this temporal output from the CNTRO ontology can be used to verify recommendations for length of drug administration, and proposes that these same tools could be applied to other medical device adverse event narratives in order to identify currently unknown temporal trends.

How essential are unstructured clinical narratives and information fusion to clinical trial recruitment?

Electronic health records capture patient information using structured controlled vocabularies and unstructured narrative text. While structured data typically encodes lab values, encounters and medication lists, unstructured data captures the physician's interpretation of the patient's condition, prognosis, and response to therapeutic intervention. In this paper, we demonstrate that information extraction from unstructured clinical narratives is essential to most clinical applications. We perform an empirical study to validate the argument and show that structured data alone is insufficient in resolving eligibility criteria for recruiting patients onto clinical trials for chronic lymphocytic leukemia (CLL) and prostate cancer. Unstructured data is essential to solving 59% of the CLL trial criteria and 77% of the prostate cancer trial criteria. More specifically, for resolving eligibility criteria with temporal constraints, we show the need for temporal reasoning and information integration with medical events within and across unstructured clinical narratives and structured data.

Towards generating a patient's timeline: extracting temporal relationships from clinical notes

Clinical records include both coded and free-text fields that interact to reflect complicated patient stories. The information often covers not only the present medical condition and events experienced by the patient, but also refers to relevant events in the past (such as signs, symptoms, tests or treatments). In order to automatically construct a timeline of these events, we first need to extract the temporal relations between pairs of events or time expressions presented in the clinical notes. We designed separate extraction components for different types of temporal relations, utilizing a novel hybrid system that combines machine learning with a graph-based inference mechanism to extract the temporal links. The temporal graph is a directed graph based on parse tree dependencies of the simplified sentences and frequent pattern clues. We generalized the sentences in order to discover patterns that, given the complexities of natural language, might not be directly discoverable in the original sentences. The proposed hybrid system performance reached an F-measure of 0.63, with precision at 0.76 and recall at 0.54 on the 2012 i2b2 Natural Language Processing corpus for the temporal relation (TLink) extraction task, achieving the highest precision and third highest f-measure among participating teams in the TLink track.

A temporal model for Clinical Data Analytics language

The proposal of a special purpose language for Clinical Data Analytics (CliniDAL) is presented along with a general model for expressing temporal events in the language. The temporal dimension of clinical data needs to be addressed from at least five different points of view. Firstly, how to attach the knowledge of time based constraints to queries; secondly, how to mine temporal data in different CISs with various data models; thirdly, how to deal with both relative time and absolute time in the query language; fourthly, how to tackle internal time-event dependencies in queries, and finally, how to manage historical time events preserved in the patient's narrative. The temporal elements of the language are defined in Bachus Naur Form (BNF) along with a UML schema. Its use in a designed taxonomy of a five class hierarchy of data analytics tasks shows the solution to problems of time event dependencies in a highly complex cascade of queries needed to evaluate scientific experiments. The issues in using the model in a practical way are discussed as well.

Temporal reasoning over clinical text: the state of the art

OBJECTIVES

To provide an overview of the problem of temporal reasoning over clinical text and to summarize the state of the art in clinical natural language processing for this task.

TARGET AUDIENCE

This overview targets medical informatics researchers who are unfamiliar with the problems and applications of temporal reasoning over clinical text.

SCOPE

We review the major applications of text-based temporal reasoning, describe the challenges for software systems handling temporal information in clinical text, and give an overview of the state of the art. Finally, we present some perspectives on future research directions that emerged during the recent community-wide challenge on text-based temporal reasoning in the clinical domain.

Knowledge management and informatics considerations for comparative effectiveness research: a case-driven exploration

BACKGROUND

As clinical data are increasingly collected and stored electronically, their potential use for comparative effectiveness research (CER) grows. Despite this promise, challenges face those wishing to leverage such data. In this paper we aim to enumerate some of the knowledge management and informatics issues common to such data reuse.

DESIGN

After reviewing the current state of knowledge regarding biomedical informatics challenges and best practices related to CER, we then present 2 research projects at our institution. We analyze these and highlight several common themes and challenges related to the conduct of CER studies. Finally, we represent these emergent themes.

RESULTS

The informatics challenges commonly encountered by those conducting CER studies include issues related to data information and knowledge management (eg, data reuse, data preparation) as well as those related to people and organizational issues (eg, sociotechnical factors and organizational factors). Examples of these are described in further detail and a formal framework for describing these findings is presented.

CONCLUSIONS

Significant challenges face researchers attempting to use often diverse and heterogeneous datasets for CER. These challenges must be understood in order to be dealt with successfully and can often be overcome with the appropriate use of informatics best practices. Many research and policy questions remain to be answered in order to realize the full potential of the increasingly electronic clinical data available for such research.

A hybrid system for temporal information extraction from clinical text

OBJECTIVE

To develop a comprehensive temporal information extraction system that can identify events, temporal expressions, and their temporal relations in clinical text. This project was part of the 2012 i2b2 clinical natural language processing (NLP) challenge on temporal information extraction.

MATERIALS AND METHODS

The 2012 i2b2 NLP challenge organizers manually annotated 310 clinic notes according to a defined annotation guideline: a training set of 190 notes and a test set of 120 notes. All participating systems were developed on the training set and evaluated on the test set. Our system consists of three modules: event extraction, temporal expression extraction, and temporal relation (also called Temporal Link, or 'TLink') extraction. The TLink extraction module contains three individual classifiers for TLinks: (1) between events and section times, (2) within a sentence, and (3) across different sentences. The performance of our system was evaluated using scripts provided by the i2b2 organizers. Primary measures were micro-averaged Precision, Recall, and F-measure.

RESULTS

Our system was among the top ranked. It achieved F-measures of 0.8659 for temporal expression extraction (ranked fourth), 0.6278 for end-to-end TLink track (ranked first), and 0.6932 for TLink-only track (ranked first) in the challenge. We subsequently investigated different strategies for TLink extraction, and were able to marginally improve performance with an F-measure of 0.6943 for TLink-only track.

Evaluating temporal relations in clinical text: 2012 i2b2 Challenge

BACKGROUND

The Sixth Informatics for Integrating Biology and the Bedside (i2b2) Natural Language Processing Challenge for Clinical Records focused on the temporal relations in clinical narratives. The organizers provided the research community with a corpus of discharge summaries annotated with temporal information, to be used for the development and evaluation of temporal reasoning systems. 18 teams from around the world participated in the challenge. During the workshop, participating teams presented comprehensive reviews and analysis of their systems, and outlined future research directions suggested by the challenge contributions.

METHODS

The challenge evaluated systems on the information extraction tasks that targeted: (1) clinically significant events, including both clinical concepts such as problems, tests, treatments, and clinical departments, and events relevant to the patient's clinical timeline, such as admissions, transfers between departments, etc; (2) temporal expressions, referring to the dates, times, durations, or frequencies phrases in the clinical text. The values of the extracted temporal expressions had to be normalized to an ISO specification standard; and (3) temporal relations, between the clinical events and temporal expressions. Participants determined pairs of events and temporal expressions that exhibited a temporal relation, and identified the temporal relation between them.

RESULTS

For event detection, statistical machine learning (ML) methods consistently showed superior performance. While ML and rule based methods seemed to detect temporal expressions equally well, the best systems overwhelmingly adopted a rule based approach for value normalization. For temporal relation classification, the systems using hybrid approaches that combined ML and heuristics based methods produced the best results.

A la Recherche du Temps Perdu: extracting temporal relations from medical text in the 2012 i2b2 NLP challenge

Objective

An analysis of the timing of events is critical for a deeper understanding of the course of events within a patient record. The 2012 i2b2 NLP challenge focused on the extraction of temporal relationships between concepts within textual hospital discharge summaries.

Materials and methods

The team from the National Research Council Canada (NRC) submitted three system runs to the second track of the challenge: typifying the time-relationship between pre-annotated entities. The NRC system was designed around four specialist modules containing statistical machine learning classifiers. Each specialist targeted distinct sets of relationships: local relationships, ‘sectime’-type relationships, non-local overlap-type relationships, and non-local causal relationships.

Results

The best NRC submission achieved a precision of 0.7499, a recall of 0.6431, and an F1 score of 0.6924, resulting in a statistical tie for first place. Post hoc improvements led to a precision of 0.7537, a recall of 0.6455, and an F1 score of 0.6954, giving the highest scores reported on this task to date.

Discussion and conclusions

Methods for general relation extraction extended well to temporal relations, and gave top-ranked state-of-the-art results. Careful ordering of predictions within result sets proved critical to this success.

Inter-annotator reliability of medical events, coreferences and temporal relations in clinical narratives by annotators with varying levels of clinical expertise

The manual annotation of clinical narratives is an important step for training and validating the performance of automated systems that utilize these clinical narratives. We build an annotation specification to capture medical events, and coreferences and temporal relations between medical events in clinical text. Unfortunately, the process of clinical data annotation is both time consuming and costly. Many annotation efforts have used physicians to annotate the data. We investigate using annotators that are current students or graduates from diverse clinical backgrounds with varying levels of clinical experience. In spite of this diversity, the annotation agreement across our team of annotators is high; the average inter-annotator kappa statistic for medical events, coreferences, temporal relations, and medical event concept unique identifiers was 0.843, 0.859, 0.833, and 0.806, respectively. We describe methods towards leveraging the annotations to support temporal reasoning with medical events.

Next-generation phenotyping of electronic health records

The national adoption of electronic health records (EHR) promises to make an unprecedented amount of data available for clinical research, but the data are complex, inaccurate, and frequently missing, and the record reflects complex processes aside from the patient's physiological state. We believe that the path forward requires studying the EHR as an object of interest in itself, and that new models, learning from data, and collaboration will lead to efficient use of the valuable information currently locked in health records.

Mining the pharmacogenomics literature--a survey of the state of the art

This article surveys efforts on text mining of the pharmacogenomics literature, mainly from the period 2008 to 2011. Pharmacogenomics (or pharmacogenetics) is the field that studies how human genetic variation impacts drug response. Therefore, publications span the intersection of research in genotypes, phenotypes and pharmacology, a topic that has increasingly become a focus of active research in recent years. This survey covers efforts dealing with the automatic recognition of relevant named entities (e.g. genes, gene variants and proteins, diseases and other pathological phenomena, drugs and other chemicals relevant for medical treatment), as well as various forms of relations between them. A wide range of text genres is considered, such as scientific publications (abstracts, as well as full texts), patent texts and clinical narratives. We also discuss infrastructure and resources needed for advanced text analytics, e.g. document corpora annotated with corresponding semantic metadata (gold standards and training data), biomedical terminologies and ontologies providing domain-specific background knowledge at different levels of formality and specificity, software architectures for building complex and scalable text analytics pipelines and Web services grounded to them, as well as comprehensive ways to disseminate and interact with the typically huge amounts of semiformal knowledge structures extracted by text mining tools. Finally, we consider some of the novel applications that have already been developed in the field of pharmacogenomic text mining and point out perspectives for future research.

Detecting temporal expressions in medical narratives

BACKGROUND

Clinical practice and epidemiological information aggregation require knowing when, how long, and in what sequence medically relevant events occur. The Temporal Awareness and Reasoning Systems for Question Interpretation (TARSQI) Toolkit (TTK) is a complete, open source software package for the temporal ordering of events within narrative text documents. TTK was developed on newspaper articles. We extended TTK to support medical notes using veterans' affairs (VA) clinical notes and compared it to TTK.

METHODS

We used a development set consisting of 200 VA clinical notes to modify and append rules to TTK's time tagger, creating Med-TTK. We then evaluated the performances of TTK and Med-TTK on an independent random selection of 100 clinical notes. Evaluation tasks were to identify and classify time-referring expressions as one of four temporal classes (DATE, TIME, DURATION, and SET). The reference standard for this test set was generated by dual human manual review with disagreements resolved by a third reviewer. Outcome measures included recall and precision for each class, and inter-rater agreement scores.

RESULTS

There were 3146 temporal expressions in the reference standard. TTK identified 1595 temporal expressions. Recall was 0.15 (95% confidence interval [CI] 0.12-0.15) and precision was 0.27 (95% CI 0.25-0.29) for TTK. Med-TTK identified 3174 expressions. Recall was 0.86 (95% CI 0.84-0.87) and precision was 0.85 (95% CI 0.84-0.86) for Med-TTK.

CONCLUSION

The algorithms for identifying and classifying temporal expressions in medical narratives developed within Med-TTK significantly improved performance compared to TTK. Natural language processing applications such as Med-TTK provide a foundation for meaningful longitudinal mapping of patient history events among electronic health records. The tool can be accessed at the following site: http://code.google.com/p/med-ttk/.

EliXR-TIME: A Temporal Knowledge Representation for Clinical Research Eligibility Criteria

Effective clinical text processing requires accurate extraction and representation of temporal expressions. Multiple temporal information extraction models were developed but a similar need for extracting temporal expressions in eligibility criteria (e.g., for eligibility determination) remains. We identified the temporal knowledge representation requirements of eligibility criteria by reviewing 100 temporal criteria. We developed EliXR-TIME, a frame-based representation designed to support semantic annotation for temporal expressions in eligibility criteria by reusing applicable classes from well-known clinical temporal knowledge representations. We used EliXR-TIME to analyze a training set of 50 new temporal eligibility criteria. We evaluated EliXR-TIME using an additional random sample of 20 eligibility criteria with temporal expressions that have no overlap with the training data, yielding 92.7% (76 / 82) inter-coder agreement on sentence chunking and 72% (72 / 100) agreement on semantic annotation. We conclude that this knowledge representation can facilitate semantic annotation of the temporal expressions in eligibility criteria.

Modeling drug exposure data in electronic medical records: an application to warfarin

Identification of patients' drug exposure information is critical to drug-related research that is based on electronic medical records (EMRs). Drug information is often embedded in clinical narratives and drug regimens change frequently because of various reasons like intolerance or insurance issues, making accurate modeling challenging. Here, we developed an informatics framework to determine patient drug exposure histories from EMRs by combining natural language processing (NLP) and machine learning (ML) technologies. Our framework consists of three phases: 1) drug entity recognition - identifying drug mentions; 2) drug event detection - labeling drug mentions with a status (e.g., "on" or "stop"); and 3) drug exposure modeling - predicting if a patient is taking a drug at a given time using the status and temporal information associated with the mentions. We applied the framework to determine patient warfarin exposure at hospital admissions and achieved 87% precision, 79% recall, and an area under the receiver-operator characteristic curve of 0.93.

The Yale cTAKES extensions for document classification: architecture and application

BACKGROUND

Open-source clinical natural-language-processing (NLP) systems have lowered the barrier to the development of effective clinical document classification systems. Clinical natural-language-processing systems annotate the syntax and semantics of clinical text; however, feature extraction and representation for document classification pose technical challenges.

METHODS

The authors developed extensions to the clinical Text Analysis and Knowledge Extraction System (cTAKES) that simplify feature extraction, experimentation with various feature representations, and the development of both rule and machine-learning based document classifiers. The authors describe and evaluate their system, the Yale cTAKES Extensions (YTEX), on the classification of radiology reports that contain findings suggestive of hepatic decompensation.

RESULTS AND DISCUSSION

The F(1)-Score of the system for the retrieval of abdominal radiology reports was 96%, and was 79%, 91%, and 95% for the presence of liver masses, ascites, and varices, respectively. The authors released YTEX as open source, available at http://code.google.com/p/ytex.

Clinical risk factors for fracture among postmenopausal patients at risk for fracture: a historical cohort study using electronic medical record data

Osteoporosis represents a growing health burden, but recognition and screening rates are low. Electronic reminders for osteoporosis have been beneficial but are not based on clinical risk factors. Available risk screening tools may contain useful constructs for creating risk-based electronic medical record (EMR) reminders. Using a cohort study design among women ≥50 years with osteoporosis or osteoporosis risk, we searched the EMR for five World Health Organization (WHO) clinical risk factors including older age, lower body mass index (BMI), low bone mineral density (BMD), history of fracture since age 50, and maternal history of osteoporosis or fracture. Rates of reporting were lower than expected for BMD (6.8%), personal history of fracture (3.5%), and maternal history of fracture (0.3%). Despite the limitations, the EMR data were useful for identifying women at highest risk for fracture. Some evidence of bias in reporting rates was present. EMR data can be useful for identifying high fracture risk patients.