1
|
Oppliger J, Denner MM, Küspert J, Frison R, Wang Q, Morawietz A, Ivashko O, Dippel AC, Zimmermann MV, Biało I, Martinelli L, Fauqué B, Choi J, Garcia-Fernandez M, Zhou KJ, Christensen NB, Kurosawa T, Momono N, Oda M, Natterer FD, Fischer MH, Neupert T, Chang J. Weak signal extraction enabled by deep neural network denoising of diffraction data. NAT MACH INTELL 2024; 6:180-186. [PMID: 38404481 PMCID: PMC10883886 DOI: 10.1038/s42256-024-00790-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2022] [Accepted: 01/08/2024] [Indexed: 02/27/2024]
Abstract
The removal or cancellation of noise has wide-spread applications in imaging and acoustics. In applications in everyday life, such as image restoration, denoising may even include generative aspects, which are unfaithful to the ground truth. For scientific use, however, denoising must reproduce the ground truth accurately. Denoising scientific data is further challenged by unknown noise profiles. In fact, such data will often include noise from multiple distinct sources, which substantially reduces the applicability of simulation-based approaches. Here we show how scientific data can be denoised by using a deep convolutional neural network such that weak signals appear with quantitative accuracy. In particular, we study X-ray diffraction and resonant X-ray scattering data recorded on crystalline materials. We demonstrate that weak signals stemming from charge ordering, insignificant in the noisy data, become visible and accurate in the denoised data. This success is enabled by supervised training of a deep neural network with pairs of measured low- and high-noise data. We additionally show that using artificial noise does not yield such quantitatively accurate results. Our approach thus illustrates a practical strategy for noise filtering that can be applied to challenging acquisition problems.
Collapse
Affiliation(s)
- Jens Oppliger
- Physik-Institut, Universität Zürich, Zurich, Switzerland
| | | | - Julia Küspert
- Physik-Institut, Universität Zürich, Zurich, Switzerland
| | - Ruggero Frison
- Physik-Institut, Universität Zürich, Zurich, Switzerland
| | - Qisi Wang
- Physik-Institut, Universität Zürich, Zurich, Switzerland
- Department of Physics, The Chinese University of Hong Kong, Hong Kong, China
| | | | - Oleh Ivashko
- Deutsches Elektronen-Synchrotron DESY, Hamburg, Germany
| | | | | | - Izabela Biało
- Physik-Institut, Universität Zürich, Zurich, Switzerland
- Faculty of Physics and Applied Computer Science, AGH University of Krakow, Krakow, Poland
| | | | - Benoît Fauqué
- JEIP, USR 3573 CNRS, Collège de France, PSL University, Paris, France
| | | | | | | | | | - Tohru Kurosawa
- Department of Physics, Hokkaido University, Sapporo, Japan
| | - Naoki Momono
- Department of Physics, Hokkaido University, Sapporo, Japan
- Department of Applied Sciences, Muroran Institute of Technology, Muroran, Japan
| | - Migaku Oda
- Department of Physics, Hokkaido University, Sapporo, Japan
| | | | | | - Titus Neupert
- Physik-Institut, Universität Zürich, Zurich, Switzerland
| | - Johan Chang
- Physik-Institut, Universität Zürich, Zurich, Switzerland
| |
Collapse
|
2
|
Sabat NK, Pati UC, Das SK. ABTCN: an efficient hybrid deep learning approach for atmospheric temperature prediction. ENVIRONMENTAL SCIENCE AND POLLUTION RESEARCH INTERNATIONAL 2023; 30:125295-125312. [PMID: 37418192 DOI: 10.1007/s11356-023-27985-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/28/2022] [Accepted: 05/25/2023] [Indexed: 07/08/2023]
Abstract
Temperature prediction is an important and significant step for monitoring global warming and the environment to save and protect human lives. The climatology parameters such as temperature, pressure, and wind speed are time-series data and are well predicted with data driven models. However, data-driven models have certain constraints, due to which these models are unable to predict the missing values and erroneous data caused by factors like sensor failure and natural disasters. In order to solve this issue, an efficient hybrid model, i.e., attention-based bidirectional long short term memory temporal convolution network (ABTCN) architecture is proposed. ABTCN uses k-nearest neighbor (KNN) imputation method for handling the missing data. A bidirectional long short term memory (Bi-LSTM) network with self-attention mechanism and temporal convolutional network (TCN) model that aids in the extraction of features from complex data and prediction of long data sequence. The performance of the proposed model is evaluated in comparison to various state-of-the-art deep learning models using error metrics such as MAE, MSE, RMSE, and R2 score. It is observed that our proposed model is superior over other models with high accuracy.
Collapse
Affiliation(s)
- Naba Krushna Sabat
- Department of Electronics and Communication Engineering, National Institute of Technology, Rourkela, Sector-1, Rourkela, 769008, Odisha, India
| | - Umesh Chandra Pati
- Department of Electronics and Communication Engineering, National Institute of Technology, Rourkela, Sector-1, Rourkela, 769008, Odisha, India
| | - Santos Kumar Das
- Department of Electronics and Communication Engineering, National Institute of Technology, Rourkela, Sector-1, Rourkela, 769008, Odisha, India.
| |
Collapse
|
3
|
Tseng YJ, Chen CJ, Chang CW. lab: an R package for generating analysis-ready data from laboratory records. PeerJ Comput Sci 2023; 9:e1528. [PMID: 37705643 PMCID: PMC10495959 DOI: 10.7717/peerj-cs.1528] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2023] [Accepted: 07/20/2023] [Indexed: 09/15/2023]
Abstract
Background Electronic health records (EHRs) play a crucial role in healthcare decision-making by giving physicians insights into disease progression and suitable treatment options. Within EHRs, laboratory test results are frequently utilized for predicting disease progression. However, processing laboratory test results often poses challenges due to variations in units and formats. In addition, leveraging the temporal information in EHRs can improve outcomes, prognoses, and diagnosis predication. Nevertheless, the irregular frequency of the data in these records necessitates data preprocessing, which can add complexity to time-series analyses. Methods To address these challenges, we developed an open-source R package that facilitates the extraction of temporal information from laboratory records. The proposed lab package generates analysis-ready time series data by segmenting the data into time-series windows and imputing missing values. Moreover, users can map local laboratory codes to the Logical Observation Identifier Names and Codes (LOINC), an international standard. This mapping allows users to incorporate additional information, such as reference ranges and related diseases. Moreover, the reference ranges provided by LOINC enable us to categorize results into normal or abnormal. Finally, the analysis-ready time series data can be further summarized using descriptive statistics and utilized to develop models using machine learning technologies. Results Using the lab package, we analyzed data from MIMIC-III, focusing on newborns with patent ductus arteriosus (PDA). We extracted time-series laboratory records and compared the differences in test results between patients with and without 30-day in-hospital mortality. We then identified significant variations in several laboratory test results 7 days after PDA diagnosis. Leveraging the time series-analysis-ready data, we trained a prediction model with the long short-term memory algorithm, achieving an area under the receiver operating characteristic curve of 0.83 for predicting 30-day in-hospital mortality in model training. These findings demonstrate the lab package's effectiveness in analyzing disease progression. Conclusions The proposed lab package simplifies and expedites the workflow involved in laboratory records extraction. This tool is particularly valuable in assisting clinical data analysts in overcoming the obstacles associated with heterogeneous and sparse laboratory records.
Collapse
Affiliation(s)
- Yi-Ju Tseng
- Department of Computer Science, National Yang Ming Chiao Tung University, Hsinchu, Taiwan
- Computational Health Informatics Program, Boston Children’s Hospital, Boston, MA, United States of America
| | - Chun Ju Chen
- Department of Information Management, National Taiwan University, Taipei, Taiwan
| | - Chia Wei Chang
- Department of Computer Science, National Yang Ming Chiao Tung University, Hsinchu, Taiwan
| |
Collapse
|
4
|
Bai YT, Jia W, Jin XB, Su TL, Kong JL. Location estimation based on feature mode matching with deep network models. Front Neurorobot 2023; 17:1181864. [PMID: 37389197 PMCID: PMC10303778 DOI: 10.3389/fnbot.2023.1181864] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2023] [Accepted: 05/23/2023] [Indexed: 07/01/2023] Open
Abstract
Introduction Global navigation satellite system (GNSS) signals can be lost in viaducts, urban canyons, and tunnel environments. It has been a significant challenge to achieve the accurate location of pedestrians during Global Positioning System (GPS) signal outages. This paper proposes a location estimation only with inertial measurements. Methods A method is designed based on deep network models with feature mode matching. First, a framework is designed to extract the features of inertial measurements and match them with deep networks. Second, feature extraction and classification methods are investigated to achieve mode partitioning and to lay the foundation for checking different deep networks. Third, typical deep network models are analyzed to match various features. The selected models can be trained for different modes of inertial measurements to obtain localization information. The experiments are performed with the inertial mileage dataset from Oxford University. Results and discussion The results demonstrate that the appropriate networks based on different feature modes have more accurate position estimation, which can improve the localization accuracy of pedestrians in GPS signal outages.
Collapse
Affiliation(s)
- Yu-Ting Bai
- School of Artificial Intelligence, Beijing Technology and Business University, Beijing, China
- Beijing Laboratory for Intelligent Environmental Protection, Beijing Technology and Business University, Beijing, China
| | - Wei Jia
- School of Artificial Intelligence, Beijing Technology and Business University, Beijing, China
| | - Xue-Bo Jin
- School of Artificial Intelligence, Beijing Technology and Business University, Beijing, China
- Beijing Laboratory for Intelligent Environmental Protection, Beijing Technology and Business University, Beijing, China
| | - Ting-Li Su
- School of Artificial Intelligence, Beijing Technology and Business University, Beijing, China
| | - Jian-Lei Kong
- School of Artificial Intelligence, Beijing Technology and Business University, Beijing, China
| |
Collapse
|
5
|
Scarce Data in Intelligent Technical Systems: Causes, Characteristics, and Implications. SCI 2022. [DOI: 10.3390/sci4040049] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022] Open
Abstract
Technical systems generate an increasing amount of data as integrated sensors become more available. Even so, data are still often scarce because of technical limitations of sensors, an expensive labelling process, or rare concepts, such as machine faults, which are hard to capture. Data scarcity leads to incomplete information about a concept of interest. This contribution details causes and effects of scarce data in technical systems. To this end, a typology is introduced which defines different types of incompleteness. Based on this, machine learning and information fusion methods are presented and discussed that are specifically designed to deal with scarce data. The paper closes with a motivation and a call for further research efforts into a combination of machine learning and information fusion.
Collapse
|
6
|
Ukil A, Marin L, Jara AJ. When less is more powerful: Shapley value attributed ablation with augmented learning for practical time series sensor data classification. PLoS One 2022; 17:e0277975. [PMID: 36417477 PMCID: PMC9683574 DOI: 10.1371/journal.pone.0277975] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2022] [Accepted: 11/08/2022] [Indexed: 11/27/2022] Open
Abstract
Time series sensor data classification tasks often suffer from training data scarcity issue due to the expenses associated with the expert-intervened annotation efforts. For example, Electrocardiogram (ECG) data classification for cardio-vascular disease (CVD) detection requires expensive labeling procedures with the help of cardiologists. Current state-of-the-art algorithms like deep learning models have shown outstanding performance under the general requirement of availability of large set of training examples. In this paper, we propose Shapley Attributed Ablation with Augmented Learning: ShapAAL, which demonstrates that deep learning algorithm with suitably selected subset of the seen examples or ablating the unimportant ones from the given limited training dataset can ensure consistently better classification performance under augmented training. In ShapAAL, additive perturbed training augments the input space to compensate the scarcity in training examples using Residual Network (ResNet) architecture through perturbation-induced inputs, while Shapley attribution seeks the subset from the augmented training space for better learnability with the goal of better general predictive performance, thanks to the "efficiency" and "null player" axioms of transferable utility games upon which Shapley value game is formulated. In ShapAAL, the subset of training examples that contribute positively to a supervised learning setup is derived from the notion of coalition games using Shapley values associated with each of the given inputs' contribution into the model prediction. ShapAAL is a novel push-pull deep architecture where the subset selection through Shapley value attribution pushes the model to lower dimension while augmented training augments the learning capability of the model over unseen data. We perform ablation study to provide the empirical evidence of our claim and we show that proposed ShapAAL method consistently outperforms the current baselines and state-of-the-art algorithms for time series sensor data classification tasks from publicly available UCR time series archive that includes different practical important problems like detection of CVDs from ECG data.
Collapse
Affiliation(s)
- Arijit Ukil
- TCS Research, Tata Consultancy Services, Kolkata, India
- * E-mail:
| | - Leandro Marin
- Faculty of Computer Science, University of Murcia, Murcia, Spain
| | | |
Collapse
|
7
|
A survival analysis based volatility and sparsity modeling network for student dropout prediction. PLoS One 2022; 17:e0267138. [PMID: 35512010 PMCID: PMC9071151 DOI: 10.1371/journal.pone.0267138] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2021] [Accepted: 04/04/2022] [Indexed: 11/19/2022] Open
Abstract
Student Dropout Prediction (SDP) is pivotal in mitigating withdrawals in Massive Open Online Courses. Previous studies generally modeled the SDP problem as a binary classification task, providing a single prediction outcome. Accordingly, some attempts introduce survival analysis methods to achieve continuous and consistent predictions over time. However, the volatility and sparsity of data always weaken the models’ performance. Prevailing solutions rely heavily on data pre-processing independent of predictive models, which are labor-intensive and may contaminate authentic data. This paper proposes a Survival Analysis based Volatility and Sparsity Modeling Network (SAVSNet) to address these issues in an end-to-end deep learning framework. Specifically, SAVSNet smooths the volatile time series by convolution network while preserving the original data information using Long-Short Term Memory Network (LSTM). Furthermore, we propose a Time-Missing-Aware LSTM unit to mitigate the impact of data sparsity by integrating informative missingness patterns into the model. A survival analysis loss function is adopted for parameter estimation, and the model outputs monotonically decreasing survival probabilities. In the experiments, we compare the proposed method with state-of-the-art methods in two real-world MOOC datasets, and the experiment results show the effectiveness of our proposed model.
Collapse
|