Mortality prediction using medical time series on TBI patients

BACKGROUND AND OBJECTIVE

Traumatic Brain Injury (TBI) is one of the leading causes of injury-related mortality in the world, with severe cases reaching mortality rates of 30-40%. It is highly heterogeneous both in causes and consequences making more complex the medical interpretation and prognosis. Gathering clinical, demographic, and laboratory data to perform a prognosis requires time and skill in several clinical specialties. Artificial intelligence (AI) methods can take advantage of existing data by performing helpful predictions and guiding physicians toward a better prognosis and, consequently, better healthcare. The objective of this work was to develop learning models and evaluate their capability of predicting the mortality of TBI. The predictive model would allow the early assessment of the more serious cases and scarce medical resources can be pointed toward the patients who need them most.

METHODS

Long Short Term Memory (LSTM) and Transformer architectures were tested and compared in performance, coupled with data imbalance, missing data, and feature selection strategies. From the Medical Information Mart for Intensive Care III (MIMIC-III) dataset, a cohort of TBI patients was selected and an analysis of the first 48 hours of multiple time series sequential variables was done to predict hospital mortality.

RESULTS

The best performance was obtained with the Transformer architecture, achieving an AUC of 0.907 with the larger group of features and trained with class proportion class weights and binary cross entropy loss.

CONCLUSIONS

Using the time series sequential data, LSTM and Transformers proved to be both viable options for predicting TBI hospital mortality in 48 hours after admission. Overall, using sequential deep learning models with time series data to predict TBI mortality is viable and can be used as a helpful indicator of the well-being of patients.

Air quality prediction by integrating mechanism model and machine learning model

AQP (Air Quality Prediction) is a very challenging project, and its core issue is how to solve the interaction and influence among meteorological, spatial and temporal factors. To address this central conundrum, we make full use of the characteristics of mechanism model and machine learning and propose a new AQP method based on DM_STGNN (Dynamic Multi-granularity Spatio-temporal Graph Neural Network). This method is the first time to use the air quality model HYSPLIT (Hybrid Single-Particle Lagrangian Integrated Trajectory Model) to assist in building a dynamic spatio-temporal graph structure to learn the spatiotemporal relationship of pollutants. DM_STGNN is based on an elaborate encoder-decoder architecture. At the encoder, in order to better mine the spatial dependency, we built a multi-granularity graph structure, used meteorological, time and geographical features to establish node attributes, used well-known HYSPLIT model to dynamically establish the edges among nodes, and used LSTM (Long Short Term Memory) to learn the time-series relationship of pollutant concentrations. At the decoder, in order to better mine the temporal dependency, we built an attention mechanism based LSTM for decoding and AQP. Additionally, in order to efficiently learn the temporal patterns from very long-term historical time series and generate rich contextual information, an unsupervised pre-training model is used to enhance DM_STGNN. The proposed model makes full use of and fully considers the influence of meteorological, spatial and temporal factors, and integrates the advantages of mechanism model and machine learning. On a project-based dataset, we validate the effectiveness of the proposed model and examine its abilities of capturing both fine-grained and long-term influences in AQP. We also compare the proposed model with the state-of-the-art AQP methods on the dataset of Yangtze River Delta city group, the experimental results show the appealing performance of our model over competitive baselines.

Solar forecasting for a PV-battery powered DC system

The photovoltaic (PV) power generation sector has been growing rapidly as a result of the rising need for solar power and the advancement of PV technology. PV Power generation is affected by weather factors such as cloud cover, solar irradiation, temperature, breeze direction and speed, and the amount of rain or snow. As a result, a highly precise PV power predictor is essential to improve security and reliability in the face of financial penalties and ambiguity. Hence, this paper suggests a novel approach to improve the efficiency of PV-battery-powered DC systems by combining solar irradiance prediction using the Long Short-Term Memory (LSTM) algorithm with a power electronic converter design that incorporates a bidirectional port. The LSTM algorithm was employed to predict one week of solar data with a remarkable R2 score of 0.96. A steady-state analysis of the proposed Three-Port Converter (TPC) is performed for five different operating modes to guarantee optimal performance. The suggested system's prediction performance was tested using several error metrics such as Mean Absolute Error (MAE), Mean Square Error (MSE), and Root Mean Squared Error (RMSE), which were computed as 0.0318, 0.0027, and 0.0526, respectively. Results from the above error measures show that the suggested approach performs more effectively in estimating solar irradiance. The Adaptive Neural-Fuzzy Interface System (ANFIS) and Incremental Conductance (IC) algorithms are employed for Maximum Power Point Tracking (MPPT) and assessed against various atmospheric conditions. From the MATLAB simulation results, the tracking efficiency of the ANFIS-based MPPT technique is 99.97 %, which is superior to the IC-based MPPT technique. Furthermore, it proved that the suggested approach improves the efficiency of PV-battery-powered DC systems, which is more appropriate for real-world applications such as DC microgrids and Electric Vehicles.

A low carbon management model for regional energy economies based on blockchain technology

As the issue of sustainable energy development becomes more and more important in national economic construction, the potential dangers of climate change are gradually attracting widespread attention from countries around the world. In order to better carry out the low-carbon management of the regional energy economy, based on the analysis of the characteristics of blockchain technology, the present study utilized this technology to achieve intelligent and digital management of carbon emissions, and established a carbon emission prediction system. The cuckoo algorithm is used to improve the long-term memory network, and the improved algorithm is used in carbon emission prediction and management. The experimental results show that the improved Long Short Term Memory networks are close to the target precision in 240 iterations, and the convergence speed is fast. In the short-term regional carbon emission prediction, the average absolute error of the method is only 2%, which is highly consistent with the actual carbon emission. In the long-term carbon emission prediction, the average prediction accuracy of the upgraded long-term short-term memory networks can reach 97.26%, and the running time is only 19.46s. With high precision and running efficiency, the upgraded Long Short Term Memory networks can efficiently monitor regional carbon emission and provide a technical reference for the low-carbon management of the regional power industry.

Modified term frequency-inverse document frequency based deep hybrid framework for sentiment analysis

Sentiment Analysis is a highly crucial subfield in Natural Language Processing that attempts to extract the public sentiment from the accessible user opinions. This paper proposes a hybridized neural network based sentiment analysis framework using a modified term frequency-inverse document frequency approach. After preprocessing of data, the basic term frequency-inverse document frequency scheme is improved by introducing a non-linear global weighting factor. This improved scheme is combined with the k-best selection method to vectorize textual features. Next, the pre-trained embedding technique is employed for the mathematical representation of the textual features to process them efficiently by the Deep Learning methodologies. The embedded features are then passed to the deep neural network, consisting of Convolutional Neural Network and Long Short Term Memory. Convolutional Neural Networks can build hierarchical representations for capturing locally embedded features within the feature space, and Long Short Term Memory tries to recall useful historical information for sentiment polarization. This deep neural network finally provides the sentiment label. The proposed model is compared with different state-of-the-art baseline models in terms of various performance metrics using several datasets to demonstrate its efficacy.

Air Quality Index prediction using an effective hybrid deep learning model

Environmentalism has become an intrinsic part of everyday life. One of the greatest challenge to the environment's long-term existence is the air pollution. Delhi, the capital of India, has experienced decreasing of air quality for several years. The poor air quality has a significant impact on the lives of individuals. Air Quality Index (AQI) prediction can help to its beneficiaries in taking safeguards about their health before moving to any polluted area. In this study, a variety of data forecasting approaches is evaluated to predict the AQI value for Particulate Matter (PM_2.5) μm at a particular area of Delhi and several error-prone strategies such as R-Squared (R²), Mean Absolute Error (MAE), and Root Mean Square Error (RMSE) methods are catalogued. In the proposed approach two deep learning models like Long-Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) are combined to predict the AQI of the environment. Several stand alone machine learning (ML) and deep learning (DL) models such as LSTM, Linear-Regression (LR), GRU, K-Nearest Neighbor (KNN) and Support Vector Machine (SVM) are also trained on the same dataset to compare their performances with the proposed hybrid (LSTM-GRU) model and it is found that the proposed hybrid model shows supremacy in the performance with the MAE value 36.11 and R² value 0.84.

Analyzing perceptions of a global event using CNN-LSTM deep learning approach: the case of Hajj 1442 (2021)

Hajj (pilgrimage) is a unique social and religious event in which many Muslims worldwide come to perform Hajj. More than two million people travel to Makkah, Saudi Arabia annually to perform various Hajj rituals for four to five days. However, given the recent outbreak of the coronavirus (COVID-19) and its variants, Hajj in the last 2 years 2020-2021 has been different because pilgrims were limited down to a few thousand to control and prevent the spread of COVID-19. This study employs a deep learning approach to investigate the impressions of pilgrims and others from within and outside the Makkah community during the 1442 AH Hajj season. Approximately 4,300 Hajj-related posts and interactions were collected from social media channels, such as Twitter and YouTube, during the Hajj season Dhul-Hijjah 1-13, 1442 (July 11-23, 2021). Convolutional neural networks (CNNs) and long short-term memory (LSTM) deep learning methods were utilized to investigate people's impressions from the collected data. The CNN-LSTM approach showed superior performance results compared with other widely used classification models in terms of F-score and accuracy. Findings revealed significantly positive sentiment rates for tweets collected from Mina and Arafa holy sites, with ratios exceeding 4 out of 5. Furthermore, the sentiment analysis (SA) rates for tweets about Hajj and pilgrims varied during the days of Hajj. Some were classified as positive tweets, such as describing joy at receiving the days of Hajj, and some were negative tweets, such as expressing the impression about the hot weather and the level of satisfaction for some services. Moreover, the SA of comments on several YouTube videos revealed positive classified comments, including praise and supplications, and negative classified comments, such as expressing regret that the Hajj was limited to a small number of pilgrims.

Classification of overlapping spikes using convolutional neural networks and long short term memory

Spike sorting is one of the key techniques to understand brain activity. In this paper, we propose a novel deep learning approach based on convolutional neural networks (CNN) and long short term memory (LSTM) to implement overlapping spike sorting. The results of the simulated data demonstrated that the clustering accuracy was greater than 99.9% and 99.0% for non-overlapping spikes and overlapping spikes, respectively. Moreover, the proposed method performed better than our previous deep learning approach named "1D-CNN". In addition, the experimental data recorded from the primary visual cortex of a macaque monkey were used to evaluate the proposed method in a practical application. It was shown that the method could successfully isolate most overlapping spikes of different neurons (ranging from two to five). In summary, the CNN + LSTM method proposed in this paper is of great advantage for overlapping spike sorting with high accuracy. It lays the foundation for application in more challenging works, such as distinguishing the simultaneous recordings of multichannel neuronal activities.

A stock market trading framework based on deep learning architectures

Market prediction has been a key interest for professionals around the world. Numerous modern technologies have been applied in addition to statistical models over the years. Among the modern technologies, machine learning and in general artificial intelligence have been at the core of numerous market prediction models. Deep learning techniques in particular have been successful in modeling the market movements. It is seen that automatic feature extraction models and time series forecasting techniques have been investigated separately however a stacked framework with a variety of inputs is not explored in detail. In the present article, we suggest a framework based on a convolutional neural network (CNN) paired with long-short term memory (LSTM) to predict the closing price of the Nifty 50 stock market index. A CNN-LSTM framework extracts features from a rich feature set and applies time series modeling with a look-up period of 20 trading days to predict the movement of the next day. Feature sets include raw price data of target index as well as foreign indices, technical indicators, currency exchange rates, commodities price data which are all chosen by similarities and well-known trade setups across the industry. The model is able to capture the information based on these features to predict the target variable i.e. closing price with a mean absolute percentage error of 2.54% across 10 years of data. The suggested framework shows a huge improvement on return than the traditional buy and hold method.

An ensemble method to forecast 24-h ahead solar irradiance using wavelet decomposition and BiLSTM deep learning network

In recent years, the penetration of solar power at residential and utility levels has progressed exponentially. However, due to its stochastic nature, the prediction of solar global horizontal irradiance (GHI) with higher accuracy is a challenging task; but, vital for grid management: planning, scheduling & balancing. Therefore, this paper proposes an ensemble model using the extended scope of wavelet transform (WT) and bidirectional long short term memory (BiLSTM) deep learning network to forecast 24-h ahead solar GHI. The WT decomposes the input time series data into different finite intrinsic model functions (IMF) to extract the statistical features of input time series. Further, the study reduces the number of IMF series by combining the wavelet decomposed components (D1-D6) series on the basis of comprehensive experimental analysis with an aim to improve the forecasting accuracy. Next, the trained standalone BiLSTM networks are allocated to each IMF sub-series to execute the forecasting. Finally, the forecasted values of each sub-series from BiLSTM networks are reconstructed to deliver the final solar GHI forecast. The study performed monthly solar GHI forecasting for one year dataset using one month moving window mechanism for the location of Ahmedabad, Gujarat, India. For the performance comparison, the naïve predictor as a benchmark model, standalone long short term memory (LSTM), gated recurrent unit (GRU), BiLSTM and two other wavelet-based BiLSTM models are also simulated. From the results, it is observed that the proposed model outperforms other models in terms of root mean square error (RMSE) & mean absolute percentage error (MAPE), coefficient of determination (R²) and forecast skill (FS). The proposed model reduces the monthly average RMSE by range from 26.04-58.89%, 5.17-31.35%, 23.26-56.06% & 21.08-57% in comparison with benchmark, standalone BiLSTM, GRU & LSTM networks respectively. On the other hand, the monthly average MAPE is reduced by range from 9 to 51.18%, 12.59-28.14%, 30.43-59.19% & 26.54-58.92% in comparison to benchmark, standalone BiLSTM, GRU & LSTM respectively. Further, the proposed model obtained the value of R² equal to 0.94 and forecast skill (%) of 47% with reference to the benchmark model.

CSI-LSTM: a web server to predict protein secondary structure using bidirectional long short term memory and NMR chemical shifts

Protein secondary structure provides rich structural information, hence the description and understanding of protein structure relies heavily on it. Identification or prediction of secondary structures therefore plays an important role in protein research. In protein NMR studies, it is more convenient to predict secondary structures from chemical shifts as compared to the traditional determination methods based on inter-nuclear distances provided by NOESY experiment. In recent years, there was a significant improvement observed in deep neural networks, which had been applied in many research fields. Here we proposed a deep neural network based on bidirectional long short term memory (biLSTM) to predict protein 3-state secondary structure using NMR chemical shifts of backbone nuclei. While comparing with the existing methods the proposed method showed better prediction accuracy. Based on the proposed method, a web server has been built to provide protein secondary structure prediction service.

Economy and carbon emissions optimization of different countries or areas in the world using an improved Attention mechanism based long short term memory neural network

The combustion of fossil fuels produces a large amount of carbon dioxide (CO₂), which leads to global warming in the world. How to rationally consume fossil energy and control CO₂ emissions has become an unavoidable problem for human beings while vigorously developing economy. This paper proposes a novel economy and CO₂ emissions prediction model using an improved Attention mechanism based long short term memory (LSTM) neural network (Attention-LSTM) to analyze and optimize the energy consumption structures in different countries or areas. The Attention mechanism can add the weight of different inputs in the previous information or related factors to realize the indirect correlation between output and related inputs of the LSTM. Therefore, the Attention-LSTM can allocate more computing resources to the parts with a higher relevance of correlation in the case of limited computing power. Through inputs with the consumption of oil, natural gas, coal, hydroelectricity and renewable energy, the desirable output with the per capita gross domestic product (GDP) and the undesirable output with CO₂ emissions prediction model of different countries and areas is established based on the Attention-LSTM. The experimental results show that compared with the normal LSTM, the back propagation (BP), the radial basis function (RBF) and the extreme learning machine (ELM) neural networks, the Attention-LSTM is more accurate and practical. Meanwhile, the proposed model provides guidance for optimizing energy structures to develop economy and reasonably control CO₂ emissions.

A comparative analysis on question classification task based on deep learning approaches

Question classification is one of the essential tasks for automatic question answering implementation in natural language processing (NLP). Recently, there have been several text-mining issues such as text classification, document categorization, web mining, sentiment analysis, and spam filtering that have been successfully achieved by deep learning approaches. In this study, we illustrated and investigated our work on certain deep learning approaches for question classification tasks in an extremely inflected Turkish language. In this study, we trained and tested the deep learning architectures on the questions dataset in Turkish. In addition to this, we used three main deep learning approaches (Gated Recurrent Unit (GRU), Long Short-Term Memory (LSTM), Convolutional Neural Networks (CNN)) and we also applied two different deep learning combinations of CNN-GRU and CNN-LSTM architectures. Furthermore, we applied the Word2vec technique with both skip-gram and CBOW methods for word embedding with various vector sizes on a large corpus composed of user questions. By comparing analysis, we conducted an experiment on deep learning architectures based on test and 10-cross fold validation accuracy. Experiment results were obtained to illustrate the effectiveness of various Word2vec techniques that have a considerable impact on the accuracy rate using different deep learning approaches. We attained an accuracy of 93.7% by using these techniques on the question dataset.

HVD-LSTM based recognition of epileptic seizures and normal human activity

In this paper, we detect the occurrence of epileptic seizures in patients as well as activities namely stand, walk, and exercise in healthy persons, leveraging EEG (electroencephalogram) signals. Using Hilbert vibration decomposition (HVD) on non-linear and non-stationary EEG signal, we obtain multiple monocomponents varying in terms of amplitude and frequency. After decomposition, we extract features from the monocomponent matrix of the EEG signals. The instantaneous amplitude of the HVD monocomponents varies because of the motion artifacts present in EEG signals. Hence, the acquired statistical features from the instantaneous amplitude help in identifying the epileptic seizures and the normal human activities. The features selected by correlation-based Q-score are classified using an LSTM (Long Short Term Memory) based deep learning model in which the feature-based weight update maximizes the classification accuracy. For epilepsy diagnosis using the Bonn dataset and activity recognition leveraging our Sensor Networks Research Lab (SNRL) data, we achieve testing classification accuracies of 96.00% and 83.30% respectively through our proposed method.

Using long short term memory and convolutional neural networks for driver drowsiness detection

Fatigue negatively affects the safety and performance of drivers on the road. In fact, drowsiness and fatigue are the cause of a substantial number of motor vehicle accidents. Drowsiness among the drivers can be detected using variety of modalities, including electroencephalogram (EEG), eye movement, and vehicle driving dynamics. Among these EEG is highly accurate but very intrusive and cumbersome. On the other hand, vehicle driving dynamics are very easy to acquire but accuracy is not very high. Eye movement based approach is very attractive in terms of balance between these two extremes. However, eye movement based techniques normally require an eye tracking device which consists of high speed camera with sophisticated algorithm to extract eye movement related parameters such as blinking, eye closure, saccades, fixation etc. This makes eye tracking based drowsiness detection difficult to implement as a practical system, especially on an embedded platform. In this paper, authors propose to use eye images from camera directly without the need for expensive eye-tracking system. Here, eye related movements are captured by Recurrent Neural Network (RNN) to detect the drowsiness. Long Short Term Memory (LSTM) is a class of RNN which has several advantages over vanilla RNNs. In this work an array of LSTM cells are utilized to model the eye movements. Two types of LSTMs were employed: 1-D LSTM (R-LSTM) which is used as baseline and the convolutional LSTM (C-LSTM) which facilitates using 2-D images directly. Patches of size 48 × 48 around each eye were extracted from 38 subjects, participating in a simulated driving experiment. The state of vigilance among the subjects were independently assessed by power spectral analysis of multichannel electroencephalogram (EEG) signals, recorded simultaneously, and binary labels of alert and drowsy (baseline) were generated. Results show high efficacy of the proposed system. R-LSTM based approach resulted in accuracy around 82 % and C-LSTM based approach resulted in accuracy in the range of 95%-97%. Comparison is also provided with a recently published eye-tracking based approach, showing the proposed LSTM technique outperform with a wide margin.

A novel Encoder-Decoder model based on read-first LSTM for air pollutant prediction

Accurate air pollutant prediction allows effective environment management to reduce the impact of pollution and prevent pollution incidents. Existing studies of air pollutant prediction are mostly interdisciplinary involving environmental science and computer science where the problem is formulated as time series prediction. A prevalent recent approach to time series prediction is the Encoder-Decoder model, which is based on recurrent neural networks (RNN) such as long short-term memory (LSTM), and great potential has been demonstrated. An LSTM network relies on various gate units, but in most existing studies the correlation between gate units is ignored. This correlation is important for establishing the relationship of the random variables in a time series as the stronger is this correlation, the stronger is the relationship between the random variables. In this paper we propose an improved LSTM, named Read-first LSTM or RLSTM for short, which is a more powerful temporal feature extractor than RNN, LSTM and Gated Recurrent Unit (GRU). RLSTM has some useful properties: (1) enables better store and remember capabilities in longer time series and (2) overcomes the problem of dependency between gate units. Since RLSTM is good at long term feature extraction, it is expected to perform well in time series prediction. Therefore, we use RLSTM as the Encoder and LSTM as the Decoder to build an Encoder-Decoder model (EDSModel) for pollutant prediction in this paper. Our experimental results show, for 1 to 24 h prediction, the proposed prediction model performed well with a root mean square error of 30.218. The effectiveness and superiority of RLSTM and the prediction model have been demonstrated.

Remaining useful life prediction techniques for electric valves based on convolution auto encoder and long short term memory

To optimize the operation and maintenance of nuclear power systems, this study presents a remaining useful life (RUL) prediction method for electric valves by combining convolutional auto-encoder (CAE) and long short term memory (LSTM). CAE can extract deeper features and LSTM is efficient in dealing with time-series data. Moreover, by designing a parallel structure between the outputs of CAE and the original data, features fed into the LSTM are enriched. Also, network structure and corresponding hyper-parameters are compared to obtain a more suitable model. Moreover, the accuracy of the proposed method is tested and compared with other machine learning algorithms. This work also serves as a critical innovation to enhance the safety and economic operation of nuclear plants and other complex systems.

Occurrence prediction of pests and diseases in cotton on the basis of weather factors by long short term memory network

Background

The occurrence of cotton pests and diseases has always been an important factor affecting the total cotton production. Cotton has a great dependence on environmental factors during its growth, especially climate change. In recent years, machine learning and especially deep learning methods have been widely used in many fields and have achieved good results.

Methods

First, this papaer used the common Aprioro algorithm to find the association rules between weather factors and the occurrence of cotton pests. Then, in this paper, the problem of predicting the occurrence of pests and diseases is formulated as time series prediction, and an LSTM-based method was developed to solve the problem.

Results

The association analysis reveals that moderate temperature, humid air, low wind spreed and rain fall in autumn and winter are more likely to occur cotton pests and diseases. The discovery was then used to predict the occurrence of pests and diseases. Experimental results showed that LSTM performs well on the prediction of occurrence of pests and diseases in cotton fields, and yields the Area Under the Curve (AUC) of 0.97.

Conclusion

Suitable temperature, humidity, low rainfall, low wind speed, suitable sunshine time and low evaporation are more likely to cause cotton pests and diseases. Based on these associations as well as historical weather and pest records, LSTM network is a good predictor for future pest and disease occurrences. Moreover, compared to the traditional machine learning models (i.e., SVM and Random Forest), the LSTM network performs the best.

Multivariate LSTM-FCNs for time series classification

Over the past decade, multivariate time series classification has received great attention. We propose transforming the existing univariate time series classification models, the Long Short Term Memory Fully Convolutional Network (LSTM-FCN) and Attention LSTM-FCN (ALSTM-FCN), into a multivariate time series classification model by augmenting the fully convolutional block with a squeeze-and-excitation block to further improve accuracy. Our proposed models outperform most state-of-the-art models while requiring minimum preprocessing. The proposed models work efficiently on various complex multivariate time series classification tasks such as activity recognition or action recognition. Furthermore, the proposed models are highly efficient at test time and small enough to deploy on memory constrained systems.

Process alarm prediction using deep learning and word embedding methods

Industrial alarm systems play an essential role for the safe management of process operations. With the increase in automation and instrumentation of modern process plants, the number of alarms that the operators manage has also increased significantly. The operators are expected to make critical decisions in the presence of flooding alarms, poorly configured and maintained alarms and many nuisance alarms. In this environment, if the incoming alarms can be correctly predicted before they actually occur, the operators may have a chance to address and possibly avoid abnormal behaviors by taking corrective actions in time. Inspired by the application of deep learning in natural language processing, this paper presents an alarm prediction method based on word embedding and recurrent neural networks to predict the next alarm in a process setting. This represents both a novel approach to alarm management as well as a novel application of natural language processing and deep learning techniques to this problem. The proposed method is applied to an actual case study to demonstrate its performance.

BO-LSTM: classifying relations via long short-term memory networks along biomedical ontologies

BACKGROUND

Recent studies have proposed deep learning techniques, namely recurrent neural networks, to improve biomedical text mining tasks. However, these techniques rarely take advantage of existing domain-specific resources, such as ontologies. In Life and Health Sciences there is a vast and valuable set of such resources publicly available, which are continuously being updated. Biomedical ontologies are nowadays a mainstream approach to formalize existing knowledge about entities, such as genes, chemicals, phenotypes, and disorders. These resources contain supplementary information that may not be yet encoded in training data, particularly in domains with limited labeled data.

RESULTS

We propose a new model to detect and classify relations in text, BO-LSTM, that takes advantage of domain-specific ontologies, by representing each entity as the sequence of its ancestors in the ontology. We implemented BO-LSTM as a recurrent neural network with long short-term memory units and using open biomedical ontologies, specifically Chemical Entities of Biological Interest (ChEBI), Human Phenotype, and Gene Ontology. We assessed the performance of BO-LSTM with drug-drug interactions mentioned in a publicly available corpus from an international challenge, composed of 792 drug descriptions and 233 scientific abstracts. By using the domain-specific ontology in addition to word embeddings and WordNet, BO-LSTM improved the F1-score of both the detection and classification of drug-drug interactions, particularly in a document set with a limited number of annotations. We adapted an existing DDI extraction model with our ontology-based method, obtaining a higher F1 score than the original model. Furthermore, we developed and made available a corpus of 228 abstracts annotated with relations between genes and phenotypes, and demonstrated how BO-LSTM can be applied to other types of relations.

CONCLUSIONS

Our findings demonstrate that besides the high performance of current deep learning techniques, domain-specific ontologies can still be useful to mitigate the lack of labeled data.

Dependency-based long short term memory network for drug-drug interaction extraction

BACKGROUND

Drug-drug interaction extraction (DDI) needs assistance from automated methods to address the explosively increasing biomedical texts. In recent years, deep neural network based models have been developed to address such needs and they have made significant progress in relation identification.

METHODS

We propose a dependency-based deep neural network model for DDI extraction. By introducing the dependency-based technique to a bi-directional long short term memory network (Bi-LSTM), we build three channels, namely, Linear channel, DFS channel and BFS channel. All of these channels are constructed with three network layers, including embedding layer, LSTM layer and max pooling layer from bottom up. In the embedding layer, we extract two types of features, one is distance-based feature and another is dependency-based feature. In the LSTM layer, a Bi-LSTM is instituted in each channel to better capture relation information. Then max pooling is used to get optimal features from the entire encoding sequential data. At last, we concatenate the outputs of all channels and then link it to the softmax layer for relation identification.

RESULTS

To the best of our knowledge, our model achieves new state-of-the-art performance with the F-score of 72.0% on the DDIExtraction 2013 corpus. Moreover, our approach obtains much higher Recall value compared to the existing methods.

CONCLUSIONS

The dependency-based Bi-LSTM model can learn effective relation information with less feature engineering in the task of DDI extraction. Besides, the experimental results show that our model excels at balancing the Precision and Recall values.