Fault Diagnosis of Hydraulic Components Based on Multi-Sensor Information Fusion Using Improved TSO-CNN-BiLSTM

In order to realize the accurate and reliable fault diagnosis of hydraulic systems, a diagnostic model based on improved tuna swarm optimization (ITSO), optimized convolutional neural networks (CNNs), and bi-directional long short-term memory (BiLSTM) networks is proposed. Firstly, sensor selection is implemented using the random forest algorithm to select useful signals from six kinds of physical or virtual sensors including pressure, temperature, flow rate, vibration, motor power, and motor efficiency coefficient. After that, fused features are extracted by CNN, and then, BiLSTM is applied to learn the forward and backward information contained in the data. The ITSO algorithm is adopted to adaptively optimize the learning rate, regularization coefficient, and node number to obtain the optimal CNN-BiLSTM network. Improved Chebyshev chaotic mapping and the nonlinear reduction strategy are adopted to improve population initialization and individual position updating, further promoting the optimization effect of TSO. The experimental results show that the proposed method can automatically extract fusion features and effectively utilize multi-sensor information. The diagnostic accuracies of the plunger pump, cooler, throttle valve, and accumulator are 99.07%, 99.4%, 98.81%, and 98.51%, respectively. The diagnostic results of noisy data with 0 dB, 5 dB, and 10 dB signal-to-noise ratios (SNRs) show that the ITSO-CNN-BiLSTM model has good robustness to noise interference.

The DIRAC framework: Geometric structure underlies roles of diversity and accuracy in combining classifiers

Combining classification systems potentially improves predictive accuracy, but outcomes have proven impossible to predict. Similar to improving binary classification with fusion, fusing ranking systems most commonly increases Pearson or Spearman correlations with a target when the input classifiers are "sufficiently good" (generalized as "accuracy") and "sufficiently different" (generalized as "diversity"), but the individual and joint quantitative influence of these factors on the final outcome remains unknown. We resolve these issues. Building on our previous empirical work establishing the DIRAC (DIversity of Ranks and ACcuracy) framework, which accurately predicts the outcome of fusing binary classifiers, we demonstrate that the DIRAC framework similarly explains the outcome of fusing ranking systems. Specifically, precise geometric representation of diversity and accuracy as angle-based distances within rank-based combinatorial structures (permutahedra) fully captures their synergistic roles in rank approximation, uncouples them from the specific metrics of a given problem, and represents them as generally as possible.

Information fusion for infant age estimation from deciduous teeth using machine learning

OBJECTIVES

Over the past few years, several methods have been proposed to improve the accuracy of age estimation in infants with a focus on dental development as a reliable marker. However, traditional approaches have limitations in efficiently combining information from different teeth and features. In order to address these challenges, this article presents a study on age estimation in infants with Machine Learning (ML) techniques, using deciduous teeth.

MATERIALS AND METHODS

The involved dataset comprises 114 infant skeletons from the Granada osteological collection of identified infants, aged between 5 months of gestation and 3 years of age. The samples consist of features such as the maximum length and mineralization and alveolar stages of teeth. For the purpose of designing a method capable of combining all the information available from each individual, a Multilayer Perceptron model is proposed, one of the most popular artificial neural networks. This model has been validated using the leave-one-out experimental validation protocol. Through different groups of experiments, the study examines the informativeness of the aforementioned features, individually and in combination.

RESULTS

The results indicate that the fusion of different variables allows for more accurate age estimates (RMSE = 66 days) than when variables are analyzed separately (RMSE = 101 days). Additionally, the study demonstrates the benefits of involving multiple teeth, which significantly reduces the RMSE compared to a single tooth.

DISCUSSION

This article underlines the clear advantages of ML-based methods, emphasizing their potential to improve the accuracy and robustness when estimating the age of infants.

A latent code based multi-variable modulation network for susceptibility mapping

Quantitative susceptibility mapping (QSM) is a technique for obtaining quantitative information on tissue susceptibility and has shown promising potential for clinical applications, in which the magnetic susceptibility is calculated by solving an ill-posed inverse problem. Recently, deep learning-based methods are proposed to address this issue, but the diversity of data distribution was not well considered, and thus the model generalization is limited in clinical applications. In this paper, we propose a Latent Code based Multi-Variable modulation network for QSM reconstruction (LCMnet). Particularly, a specific modulation module is exploited to incorporate three variables, i.e., field map, magnitude image, and initial susceptibility. The latent code in the modulated convolution is learned from feature maps of the field data using the encoder-decoder framework. The susceptibility map pre-estimated from simple thresholding is the constant input of the module, thereby enhancing the network stability and accelerating training convergence. As another input, multi-level features generated by a cross-fusion block integrate the information of field and magnitude data effectively. Experimental results on in vivo human brain data, challenge data, clinical data and synthetic data demonstrate that the proposed method LCMnet can achieve outstanding performance on accurate susceptibility measurement and also excellent generalization.

Design of a Two-Dimensional Conveyor Platform with Cargo Pose Recognition and Adjustment Capabilities

Linear conveyors, traditional tools for cargo transportation, have faced criticism due to their directional constraints, inability to adjust poses, and single-item conveyance, making them unsuitable for modern flexible logistics demands. This paper introduces a platform designed to convey and adjust cargo boxes according to their spatial positions and orientations. Additionally, a cargo pose recognition algorithm that integrates image and point cloud data are presented. By aligning depth camera data, the axis-aligned bounding box (AABB) point serves as the image's region of interest (ROI). Peaks extracted from the image's Hough transform are refined using RANSAC-based point cloud linear fitting, then integrated with the point cloud's oriented bounding box (OBB). Notably, the algorithm eliminates the need for deep learning and registration, enabling its use in rectangular cargo boxes of various sizes. A comparative experiment using accelerometer sensors for pose acquisition revealed a deviation of <0.7° between the two processes. Throughout the real-time adjustments controlled by the experimental platform, cargo angles consistently remained stable. The proposed two-dimensional conveyance platform, compared to existing methods, exhibits simplicity, accurate recognition, enhanced flexibility, and wide applicability.

ASNET: A Novel AI Framework for Accurate Ankylosing Spondylitis Diagnosis from MRI

BACKGROUND

Ankylosing spondylitis (AS) is a chronic, painful, progressive disease usually seen in the spine. Traditional diagnostic methods have limitations in detecting the early stages of AS. The early diagnosis of AS can improve patients' quality of life. This study aims to diagnose AS with a pre-trained hybrid model using magnetic resonance imaging (MRI).

MATERIALS AND METHODS

In this research, we collected a new MRI dataset comprising three cases. Furthermore, we introduced a novel deep feature engineering model. Within this model, we utilized three renowned pretrained convolutional neural networks (CNNs): DenseNet201, ResNet50, and ShuffleNet. Through these pretrained CNNs, deep features were generated using the transfer learning approach. For each pretrained network, two feature vectors were generated from an MRI. Three feature selectors were employed during the feature selection phase, amplifying the number of features from 6 to 18 (calculated as 6 × 3). The k-nearest neighbors (kNN) classifier was utilized in the classification phase to determine classification results. During the information phase, the iterative majority voting (IMV) algorithm was applied to secure voted results, and our model selected the output with the highest classification accuracy. In this manner, we have introduced a self-organized deep feature engineering model.

RESULTS

We have applied the presented model to the collected dataset. The proposed method yielded 99.80%, 99.60%, 100%, and 99.80% results for accuracy, recall, precision, and F1-score for the collected axial images dataset. The collected coronal image dataset yielded 99.45%, 99.20%, 99.70%, and 99.45% results for accuracy, recall, precision, and F1-score, respectively. As for contrast-enhanced images, accuracy of 95.62%, recall of 80.72%, precision of 94.24%, and an F1-score of 86.96% were attained.

CONCLUSIONS

Based on the results, the proposed method for classifying AS disease has demonstrated successful outcomes using MRI. The model has been tested on three cases, and its consistently high classification performance across all cases underscores the model's general robustness. Furthermore, the ability to diagnose AS disease using only axial images, without the need for contrast-enhanced MRI, represents a significant advancement in both healthcare and economic terms.

Dual-band polarimetric HRRP recognition via a brain-inspired multi-channel fusion feature extraction network

Radar high-resolution range profile (HRRP) provides geometric and structural information of target, which is important for radar automatic target recognition (RATR). However, due to the limited information dimension of HRRP, achieving accurate target recognition is challenging in applications. In recent years, with the rapid development of radar components and signal processing technology, the acquisition and use of target multi-frequency and polarization scattering information has become a significant way to improve target recognition performance. Meanwhile, deep learning inspired by the human brain has shown great promise in pattern recognition applications. In this paper, a Multi-channel Fusion Feature Extraction Network (MFFE-Net) inspired by the human brain is proposed for dual-band polarimetric HRRP, aiming at addressing the challenges faced in HRRP target recognition. In the proposed network, inspired by the human brain's multi-dimensional information interaction, the similarity and difference features of dual-frequency HRRP are first extracted to realize the interactive fusion of frequency features. Then, inspired by the human brain's selective attention mechanism, the interactive weights are obtained for multi-polarization features and multi-scale representation, enabling feature aggregation and multi-scale fusion. Finally, inspired by the human brain's hierarchical learning mechanism, the layer-by-layer feature extraction and fusion with residual connections are designed to enhance the separability of features. Experiments on simulated and measured datasets verify the accurate recognition capability of MFFE-Net, and ablative studies are conducted to confirm the effectiveness of components of network for recognition.

Decision Support System for Prioritization of Offshore Wind Farm Site by Utilizing Picture Fuzzy Combined Compromise Solution Group Decision Method

The selection of offshore wind farm site (OWFS) has important strategic significance for vigorously developing offshore new energy and is deemed as a complicated uncertain multicriteria decision-making (MCDM) process. To further promote offshore wind power energy planning and provide decision support, this paper proposes a hybrid picture fuzzy (PF) combined compromise solution (CoCoSo) technique for prioritization of OWFSs. To begin with, a fresh PF similarity measure is proffered to estimate the importance of experts. Next, the novel operational rules for PF numbers based upon the generalized Dombi norms are defined, and four novel generalized Dombi operators are propounded. Afterward, the PF preference selection index (PSI) method and PF stepwise weights assessment ratio analysis (SWARA) model are propounded to identify the objective and subjective weight of criteria, separately. In addition, the enhanced CoCoSo method is proffered via the similarity measure and new operators for ranking OWFSs with PF information. Lastly, the applicability and feasibility of the propounded PF-PSI-SWARA-CoCoSo method are adopted to ascertain the optimal OWFS. The comparison and sensibility investigations are also carried out to validate the robustness and superiority of our methodology. Results manifest that the developed methodology can offer powerful decision support for departments and managers to evaluate and choose the satisfying OWFSs.

A Novel Evidence Combination Method Based on Improved Pignistic Probability

Evidence theory is widely used to deal with the fusion of uncertain information, but the fusion of conflicting evidence remains an open question. To solve the problem of conflicting evidence fusion in single target recognition, we proposed a novel evidence combination method based on an improved pignistic probability function. Firstly, the improved pignistic probability function could redistribute the probability of multi-subset proposition according to the weight of single subset propositions in a basic probability assignment (BPA), which reduces the computational complexity and information loss in the conversion process. The combination of the Manhattan distance and evidence angle measurements is proposed to extract evidence certainty and obtain mutual support information between each piece of evidence; then, entropy is used to calculate the uncertainty of the evidence and the weighted average method is used to correct and update the original evidence. Finally, the Dempster combination rule is used to fuse the updated evidence. Verified by the analysis results of single-subset proposition and multi-subset proposition highly conflicting evidence examples, compared to the Jousselme distance method, the Lance distance and reliability entropy combination method, and the Jousselme distance and uncertainty measure combination method, our approach achieved better convergence and the average accuracy was improved by 0.51% and 2.43%.

A New Correlation Measure for Belief Functions and Their Application in Data Fusion

Measuring the correlation between belief functions is an important issue in Dempster-Shafer theory. From the perspective of uncertainty, analyzing the correlation may provide a more comprehensive reference for uncertain information processing. However, existing studies about correlation have not combined it with uncertainty. In order to address the problem, this paper proposes a new correlation measure based on belief entropy and relative entropy, named a belief correlation measure. This measure takes into account the influence of information uncertainty on their relevance, which can provide a more comprehensive measure for quantifying the correlation between belief functions. Meanwhile, the belief correlation measure has the mathematical properties of probabilistic consistency, non-negativity, non-degeneracy, boundedness, orthogonality, and symmetry. Furthermore, based on the belief correlation measure, an information fusion method is proposed. It introduces the objective weight and subjective weight to assess the credibility and usability of belief functions, thus providing a more comprehensive measurement for each piece of evidence. Numerical examples and application cases in multi-source data fusion demonstrate that the proposed method is effective.

Improved Thermal Infrared Image Super-Resolution Reconstruction Method Base on Multimodal Sensor Fusion

When traditional super-resolution reconstruction methods are applied to infrared thermal images, they often ignore the problem of poor image quality caused by the imaging mechanism, which makes it difficult to obtain high-quality reconstruction results even with the training of simulated degraded inverse processes. To address these issues, we proposed a thermal infrared image super-resolution reconstruction method based on multimodal sensor fusion, aiming to enhance the resolution of thermal infrared images and rely on multimodal sensor information to reconstruct high-frequency details in the images, thereby overcoming the limitations of imaging mechanisms. First, we designed a novel super-resolution reconstruction network, which consisted of primary feature encoding, super-resolution reconstruction, and high-frequency detail fusion subnetwork, to enhance the resolution of thermal infrared images and rely on multimodal sensor information to reconstruct high-frequency details in the images, thereby overcoming limitations of imaging mechanisms. We designed hierarchical dilated distillation modules and a cross-attention transformation module to extract and transmit image features, enhancing the network's ability to express complex patterns. Then, we proposed a hybrid loss function to guide the network in extracting salient features from thermal infrared images and reference images while maintaining accurate thermal information. Finally, we proposed a learning strategy to ensure the high-quality super-resolution reconstruction performance of the network, even in the absence of reference images. Extensive experimental results show that the proposed method exhibits superior reconstruction image quality compared to other contrastive methods, demonstrating its effectiveness.

Fault Diagnosis of the Autonomous Driving Perception System Based on Information Fusion

The reliability of autonomous driving sensing systems impacts the overall safety of the driving system. However, perception system fault diagnosis is currently a weak area of research, with limited attention and solutions. In this paper, we present an information-fusion-based fault-diagnosis method for autonomous driving perception systems. To begin, we built an autonomous driving simulation scenario using PreScan software, which collects information from a single millimeter wave (MMW) radar and a single camera sensor. The photos are then identified and labeled via the convolutional neural network (CNN). Then, we fused the sensory inputs from a single MMW radar sensor and a single camera sensor in space and time and mapped the MMW radar points onto the camera image to obtain the region of interest (ROI). Lastly, we developed a method to use information from a single MMW radar to aid in diagnosing defects in a single camera sensor. As the simulation results show, for missing row/column pixel failure, the deviation typically falls between 34.11% and 99.84%, with a response time of 0.02 s to 1.6 s; for pixel shift faults, the deviation range is between 0.32% and 9.92%, with a response time of 0 s to 0.16 s; for target color loss, faults have a deviation range of 0.26% to 2.88% and a response time of 0 s to 0.05 s. These results prove the technology is effective in detecting sensor faults and issuing real-time fault alerts, providing a basis for designing and developing simpler and more user-friendly autonomous driving systems. Furthermore, this method illustrates the principles and methods of information fusion between camera and MMW radar sensors, establishing the foundation for creating more complicated autonomous driving systems.

Detecting different pesticide residues on Hami melon surface using hyperspectral imaging combined with 1D-CNN and information fusion

Efficient, rapid, and non-destructive detection of pesticide residues in fruits and vegetables is essential for food safety. The visible/near infrared (VNIR) and short-wave infrared (SWIR) hyperspectral imaging (HSI) systems were used to detect different types of pesticide residues on the surface of Hami melon. Taking four pesticides commonly used in Hami melon as the object, the effectiveness of single-band spectral range and information fusion in the classification of different pesticides was compared. The results showed that the classification effect of pesticide residues was better by using the spectral range after information fusion. Then, a custom multi-branch one-dimensional convolutional neural network (1D-CNN) model with the attention mechanism was proposed and compared with the traditional machine learning classification model K-nearest neighbor (KNN) algorithm and random forest (RF). The traditional machine learning classification model accuracy of both models was over 80.00%. However, the classification results using the proposed 1D-CNN were more satisfactory. After the full spectrum data was fused, it was input into the 1D-CNN model, and its accuracy, precision, recall, and F1-score value were 94.00%, 94.06%, 94.00%, and 0.9396, respectively. This study showed that both VNIR and SWIR hyperspectral imaging combined with a classification model could non-destructively detect different pesticide residues on the surface of Hami melon. The classification result using the SWIR spectrum was better than that using the VNIR spectrum, and the classification result using the information fusion spectrum was better than that using SWIR. This study can provide a valuable reference for the non-destructive detection of pesticide residues on the surface of other large, thick-skinned fruits.

Spatial Multi-Source Information Fusion Localization Algorithm in Non-Line-of-Sight Environments

In this paper, a high-accuracy localization problem under a complex non-line-of-sight (NLOS) condition is addressed by a new method that utilizes multiple NLOS paths to improve localization accuracy, as opposed to the traditional method of suppressing them. The spatial multi-path array fusion localization model is presented and analyzed, followed by an angle-of-arrival (AOA) and time-of-arrival (TOA) algorithm based on spatial multi-information fusion that seeks to improve localization accuracy. Multi-path of spatial signals, measurement of the multi-element antenna, and geographic environment information are integrated into the proposed method for localization optimization. Simulation experiments were carried out, and the results revealed that the proposed algorithm is capable of making full use of spatial multi-location information for localization, thus improving the accuracy of localization in a the NLOS environment effectively and increasing the locatable probability of complex environment localization applications.

Semantic-Structure-Aware Multi-Level Information Fusion for Robust Global Orientation Optimization of Autonomous Mobile Robots

Multi-camera-based simultaneous localization and mapping (SLAM) has been widely applied in various mobile robots under uncertain or unknown environments to accomplish tasks autonomously. However, the conventional purely data-driven feature extraction methods cannot utilize the rich semantic information in the environment, which leads to the performance of the SLAM system being susceptible to various interferences. In this work, we present a semantic-aware multi-level information fusion scheme for robust global orientation estimation. Specifically, a visual semantic perception system based on the synthesized surround view image is proposed for the multi-eye surround vision system widely used in mobile robots, which is used to obtain the visual semantic information required for SLAM tasks. The original multi-eye image was first transformed to the synthesized surround view image, and the passable space was extracted with the help of the semantic segmentation network model as a mask for feature extraction; moreover, the hybrid edge information was extracted to effectively eliminate the distorted edges by further using the distortion characteristics of the reverse perspective projection process. Then, the hybrid semantic information was used for robust global orientation estimation; thus, better localization performance was obtained. The experiments on an intelligent vehicle, which was used for automated valet parking both in indoor and outdoor scenes, showed that the proposed hybrid multi-level information fusion method achieved at least a 10-percent improvement in comparison with other edge segmentation methods, the average orientation estimation error being between 1 and 2 degrees, much smaller than other methods, and the trajectory drift value of the proposed method was much smaller than that of other methods.

Multi-Objective Association Detection of Farmland Obstacles Based on Information Fusion of Millimeter Wave Radar and Camera

In order to remedy the defects of single sensor in robustness, accuracy, and redundancy of target detection, this paper proposed a method for detecting obstacles in farmland based on the information fusion of a millimeter wave (mmWave) radar and a camera. Combining the advantages of the mmWave radar in range and speed measurement and the camera in type identification and lateral localization, a decision-level fusion algorithm was designed for the mmWave radar and camera information, and the global nearest neighbor method was used for data association. Then, the effective target sequences of the mmWave radar and the camera with successful data association were weighted to output, and the output included more accurate target orientation, longitudinal speed, and category. For the unassociated sequences, they were tracked as new targets by using the extended Kalman filter algorithm and were processed and output during the effective life cycle. Lastly, an experimental platform based on a tractor was built to verify the effectiveness of the proposed association detection method. The obstacle detection test was conducted under the ROS environment after solving the external parameters of the mmWave radar and the internal and external parameters of the camera. The test results show that the correct detection rate of obstacles reaches 86.18%, which is higher than that of a single camera with 62.47%. Furthermore, through the contrast experiment of the sensor fusion algorithms, the detection accuracy of the decision level fusion algorithm was 95.19%, which was higher than 4.38% and 6.63% compared with feature level and data level fusion, respectively.

Research on Aerial Autonomous Docking and Landing Technology of Dual Multi-Rotor UAV

This paper studies the cooperative control of multiple unmanned aerial vehicles (UAVs) with sensors and autonomous flight capabilities. In this paper, an architecture is proposed that takes a small quadrotor as a mission UAV and a large six-rotor as a platform UAV to provide an aerial take-off and landing platform and transport carrier for the mission UAV. The design of a tracking controller for an autonomous docking and landing trajectory system is the focus of this research. To examine the system's overall design, a dual-machine trajectory-tracking control simulation platform is created via MATLAB/Simulink. Then, an autonomous docking and landing trajectory-tracking controller based on radial basis function proportional-integral-derivative control is designed, which fulfills the trajectory-tracking control requirements of the autonomous docking and landing process by efficiently suppressing the external airflow disturbance according to the simulation results. A YOLOv3-based vision pilot system is designed to calibrate the rate of the aerial docking and landing position to eight frames per second. The feasibility of the multi-rotor aerial autonomous docking and landing technology is verified using prototype flight tests during the day and at night. It lays a technical foundation for UAV transportation, autonomous take-off, landing in the air, and collaborative networking. In addition, compared with the existing technologies, our research completes the closed loop of the technical process through modeling, algorithm design and testing, virtual simulation verification, prototype manufacturing, and flight test, which have better realizability.

Improving Situation Awareness via a Situation Model-Based Intersection of IoT Sensor and Social Media Information Spaces

Existing techniques for distilling situation awareness currently focus on information harvested from either IoT sensors or social media. While the benefits of fusing information from these two distinct information spaces for achieving enhanced situation awareness are well understood, existing techniques and related systems for fusing the IoT sensors and social media information spaces are currently embryonic. Key challenges in intersecting, combining, and fusing these information spaces to distil high-value situation awareness include devising situation models and related techniques for filtering, integrating, and fusing sparse and heterogeneous IoT sensor data and social media postings to provide a richer and more accurate situation awareness. This paper proposes novel, semantically based techniques fusing social media and IoT sensor information spaces and a comprehensive, fully implemented system that utilizes these to provide enhanced situation awareness. More specifically, this paper proposes the design of semantic-based situation models for fusing sensor and social media information spaces and presents techniques for finding similarities across these information spaces and fusing social media posting and IoT sensor data to generate an enhanced situation awareness. Furthermore, the paper presents the design and implementation of a complete system that uses the proposed models and techniques and uses that in an experimental evaluation that illustrates improvements in situation awareness from fusing the IoT sensor and social media information spaces.

An Efficient and Uncertainty-Aware Decision Support System for Disaster Response Using Aerial Imagery

Efficient and robust search and rescue actions are always required when natural or technical disasters occur. Empowered by remote sensing techniques, building damage assessment can be achieved by fusing aerial images of pre- and post-disaster environments through computational models. Existing methods pay over-attention to assessment accuracy without considering model efficiency and uncertainty quantification in such a life-critical application. Thus, this article proposes an efficient and uncertain-aware decision support system (EUDSS) that evolves the recent computational models into an efficient decision support system, realizing the uncertainty during building damage assessment (BDA). Specifically, a new efficient and uncertain-aware BDA integrates the recent advances in computational models such as Fourier attention and Monte Carlo Dropout for uncertainty quantification efficiently. Meanwhile, a robust operation (RO) procedure is designed to invite experts for manual reviews if the uncertainty is high due to external factors such as cloud clutter and poor illumination. This procedure can prevent rescue teams from missing damaged houses during operations. The effectiveness of the proposed system is demonstrated on a public dataset from both quantitative and qualitative perspectives. The solution won the first place award in International Overhead Imagery Hackathon.

Observability Decomposition-Based Decentralized Kalman Filter and Its Application to Resilient State Estimation under Sensor Attacks

This paper considers a discrete-time linear time invariant system in the presence of Gaussian disturbances/noises and sparse sensor attacks. First, we propose an optimal decentralized multi-sensor information fusion Kalman filter based on the observability decomposition when there is no sensor attack. The proposed decentralized Kalman filter deploys a bank of local observers who utilize their own single sensor information and generate the state estimate for the observable subspace. In the absence of an attack, the state estimate achieves the minimum variance, and the computational process does not suffer from the divergent error covariance matrix. Second, the decentralized Kalman filter method is applied in the presence of sparse sensor attacks as well as Gaussian disturbances/noises. Based on the redundant observability, an attack detection scheme by the χ2 test and a resilient state estimation algorithm by the maximum likelihood decision rule among multiple hypotheses, are presented. The secure state estimation algorithm finally produces a state estimate that is most likely to have minimum variance with an unbiased mean. Simulation results on a motor controlled multiple torsion system are provided to validate the effectiveness of the proposed algorithm.

Research on the Rapid Diagnostic Method of Rolling Bearing Fault Based on Cloud-Edge Collaboration

Recent deep-learning methods for fault diagnosis of rolling bearings need a significant amount of computing time and resources. Most of them cannot meet the requirements of real-time fault diagnosis of rolling bearings under the cloud computing framework. This paper proposes a quick cloud-edge collaborative bearing fault diagnostic method based on the tradeoff between the advantages and disadvantages of cloud and edge computing. First, a collaborative cloud-based framework and an improved DSCNN-GAP algorithm are suggested to build a general model using the public bearing fault dataset. Second, the general model is distributed to each edge node, and a limited number of unique fault samples acquired by each edge node are used to quickly adjust the parameters of the model before running diagnostic tests. Finally, a fusion result is made from the diagnostic results of each edge node by DS evidence theory. Experiment results show that the proposed method not only improves diagnostic accuracy by DSCNN-GAP and fusion of multi-sensors, but also decreases diagnosis time by migration learning with the cloud-edge collaborative framework. Additionally, the method can effectively enhance data security and privacy protection.

Multiple Time Series Fusion Based on LSTM: An Application to CAP A Phase Classification Using EEG

The Cyclic Alternating Pattern (CAP) is a periodic activity detected in the electroencephalogram (EEG) signals. This pattern was identified as a marker of unstable sleep with several possible clinical applications; however, there is a need to develop automatic methodologies to facilitate real-world applications based on CAP assessment. Therefore, a deep learning-based EEG channels' feature level fusion was proposed in this work and employed for the CAP A phase classification. Two optimization algorithms optimized the channel selection, fusion, and classification procedures. The developed methodologies were evaluated by fusing the information from multiple EEG channels for patients with nocturnal frontal lobe epilepsy and patients without neurological disorders. Results showed that both optimization algorithms selected a comparable structure with similar feature level fusion, consisting of three electroencephalogram channels (Fp2-F4, C4-A1, F4-C4), which is in line with the CAP protocol to ensure multiple channels' arousals for CAP detection. Moreover, the two optimized models reached an area under the receiver operating characteristic curve of 0.82, with average accuracy ranging from 77% to 79%, a result in the upper range of the specialist agreement and best state-of-the-art works, despite a challenging dataset. The proposed methodology also has the advantage of providing a fully automatic analysis without requiring any manual procedure. Ultimately, the models were revealed to be noise-resistant and resilient to multiple channel loss, being thus suitable for real-world application.

An Underwater Positioning System for UUVs Based on LiDAR Camera and Inertial Measurement Unit

Underwater positioning presents a challenging issue, because of the rapid attenuation of electronic magnetic waves, the disturbances and uncertainties in the environment. Conventional methods usually employed acoustic devices to localize Unmanned Underwater Vehicles (UUVs), which suffer from a slow refresh rate, low resolution, and are susceptible to the environmental noise. In addition, the complex terrain can also degrade the accuracy of the acoustic navigation systems. The applications of underwater positioning methods based on visual sensors are prevented by difficulties of acquiring the depth maps due to the sparse features, the changing illumination condition, and the scattering phenomenon. In the paper, a novel visual-based underwater positioning system is proposed based on a Light Detection and Ranging (LiDAR) camera and an inertial measurement unit. The LiDAR camera, benefiting from the laser scanning techniques, could simultaneously generate the associated depth maps. The inertial sensor would offer information about its altitudes. Through the fusion of the data from multiple sensors, the positions of the UUVs can be predicted. After that, the Bundle Adjustment (BA) method is used to recalculate the rotation matrix and the translation vector to improve the accuracy. The experiments are carried out in a tank to illustrate the effects and accuracy of the investigated method, in which the ultra-wideband (UWB) positioning system is used to provide reference trajectories. It is concluded that the developed positioning system is able to estimate the trajectory of UUVs accurately, whilst being stable and robust.

Machine Learning Study of Metabolic Networks vs ChEMBL Data of Antibacterial Compounds

Antibacterial drugs (AD) change the metabolic status of bacteria, contributing to bacterial death. However, antibiotic resistance and the emergence of multidrug-resistant bacteria increase interest in understanding metabolic network (MN) mutations and the interaction of AD vs MN. In this study, we employed the IFPTML = Information Fusion (IF) + Perturbation Theory (PT) + Machine Learning (ML) algorithm on a huge dataset from the ChEMBL database, which contains >155,000 AD assays vs >40 MNs of multiple bacteria species. We built a linear discriminant analysis (LDA) and 17 ML models centered on the linear index and based on atoms to predict antibacterial compounds. The IFPTML-LDA model presented the following results for the training subset: specificity (Sp) = 76% out of 70,000 cases, sensitivity (Sn) = 70%, and Accuracy (Acc) = 73%. The same model also presented the following results for the validation subsets: Sp = 76%, Sn = 70%, and Acc = 73.1%. Among the IFPTML nonlinear models, the k nearest neighbors (KNN) showed the best results with Sn = 99.2%, Sp = 95.5%, Acc = 97.4%, and Area Under Receiver Operating Characteristic (AUROC) = 0.998 in training sets. In the validation series, the Random Forest had the best results: Sn = 93.96% and Sp = 87.02% (AUROC = 0.945). The IFPTML linear and nonlinear models regarding the ADs vs MNs have good statistical parameters, and they could contribute toward finding new metabolic mutations in antibiotic resistance and reducing time/costs in antibacterial drug research.

Early Detection of Pancreatic Cancers Using Liquid Biopsies and Hierarchical Decision Structure

OBJECTIVE

Pancreatic cancer (PC) is a silent killer, because its detection is difficult and to date no effective treatment has been developed. In the US, the current 5-year survival rate of 11%. Therefore, PC has to be detected as early as possible.

METHODS AND PROCEDURES

In this work, we have combined the use of ultrasensitive nanobiosensors for protease/arginase detection with information fusion based hierarchical decision structure to detect PC at the localized stage by means of a simple Liquid Biopsy. The problem of early-stage detection of pancreatic cancer is modelled as a multi-class classification problem. We propose a Hard Hierarchical Decision Structure (HDS) along with appropriate feature engineering steps to improve the performance of conventional multi-class classification approaches. Further, a Soft Hierarchical Decision Structure (SDS) is developed to additionally provide confidences of predicted labels in the form of class probability values. These frameworks overcome the limitations of existing research studies that employ simple biostatistical tools and do not effectively exploit the information provided by ultrasensitive protease/arginase analyses.

RESULTS

The experimental results demonstrate that an overall mean classification accuracy of around 92% is obtained using the proposed approach, as opposed to 75% with conventional multi-class classification approaches. This illustrates that the proposed HDS framework outperforms traditional classification techniques for early-stage PC detection.

CONCLUSION

Although this study is only based on 31 pancreatic cancer patients and a healthy control group of 48 human subjects, it has enabled combining Liquid Biopsies and Machine Learning methodologies to reach the goal of earliest PC detection. The provision of both decision labels (via HDS) as well as class probabilities (via SDS) helps clinicians identify instances where statistical model-based predictions lack confidence. This further aids in determining if more tests are required for better diagnosis. Such a strategy makes the output of our decision model more interpretable and can assist with the diagnostic procedure.

CLINICAL IMPACT

With further validation, the proposed framework can be employed as a decision support tool for the clinicians to help in detection of pancreatic cancer at early stages.

Facial Expression Recognition from Multi-Perspective Visual Inputs and Soft Voting

Automatic identification of human facial expressions has many potential applications in today's connected world, from mental health monitoring to feedback for onscreen content or shop windows and sign-language prosodic identification. In this work we use visual information as input, namely, a dataset of face points delivered by a Kinect device. The most recent work on facial expression recognition uses Machine Learning techniques, to use a modular data-driven path of development instead of using human-invented ad hoc rules. In this paper, we present a Machine-Learning based method for automatic facial expression recognition that leverages information fusion architecture techniques from our previous work and soft voting. Our approach shows an average prediction performance clearly above the best state-of-the-art results for the dataset considered. These results provide further evidence of the usefulness of information fusion architectures rather than adopting the default ML approach of features aggregation.

Machine-Learning-Based Late Fusion on Multi-Omics and Multi-Scale Data for Non-Small-Cell Lung Cancer Diagnosis

Differentiation between the various non-small-cell lung cancer subtypes is crucial for providing an effective treatment to the patient. For this purpose, machine learning techniques have been used in recent years over the available biological data from patients. However, in most cases this problem has been treated using a single-modality approach, not exploring the potential of the multi-scale and multi-omic nature of cancer data for the classification. In this work, we study the fusion of five multi-scale and multi-omic modalities (RNA-Seq, miRNA-Seq, whole-slide imaging, copy number variation, and DNA methylation) by using a late fusion strategy and machine learning techniques. We train an independent machine learning model for each modality and we explore the interactions and gains that can be obtained by fusing their outputs in an increasing manner, by using a novel optimization approach to compute the parameters of the late fusion. The final classification model, using all modalities, obtains an F1 score of 96.81±1.07, an AUC of 0.993±0.004, and an AUPRC of 0.980±0.016, improving those results that each independent model obtains and those presented in the literature for this problem. These obtained results show that leveraging the multi-scale and multi-omic nature of cancer data can enhance the performance of single-modality clinical decision support systems in personalized medicine, consequently improving the diagnosis of the patient.

Multi-Stream Convolutional Neural Networks for Rotating Machinery Fault Diagnosis under Noise and Trend Items

In recent years, rotating machinery fault diagnosis methods based on convolutional neural network have achieved much success. However, in real industrial environments, interfering signals are unavoidable, which may reduce the accuracy of fault diagnosis seriously. Most of the current fault diagnosis methods are of single input type, which may lead to the information contained in the vibration signal not being fully utilized. In this study, theoretical analysis and comprehensive comparative experiments are completed to investigate the time domain input, frequency domain input, and two types of time–frequency domain input. Based on this, a new fault diagnosis model, named multi-stream convolutional neural network, is developed. The model takes the time domain, frequency domain, and time–frequency domain images as input, and it automatically fuses the information contained in different inputs. The proposed model is tested based on three public datasets. The experimental results suggested that the model achieved pretty high accuracy under noise and trend items without the help of signal separation algorithms. In addition, the positive implications of multiple inputs and information fusion are analyzed through the visualization of learned features.

Dependency Factors in Evidence Theory: An Analysis in an Information Fusion Scenario Applied in Adverse Drug Reactions

Multisensor information fusion brings challenges such as data heterogeneity, source precision, and the merger of uncertainties that impact the quality of classifiers. A widely used approach for classification problems in a multisensor context is the Dempster-Shafer Theory. This approach considers the beliefs attached to each source to consolidate the information concerning the hypotheses to come up with a classifier with higher precision. Nevertheless, the fundamental premise for using the approach is that sources are independent and that the classification hypotheses are mutually exclusive. Some approaches ignore this premise, which can lead to unreliable results. There are other approaches, based on statistics and machine learning techniques, that expurgate the dependencies or include a discount factor to mitigate the risk of dependencies. We propose a novel approach based on Bayesian net, Pearson's test, and linear regression to adjust the beliefs for more accurate data fusion, mitigating possible correlations or dependencies. We tested our approach by applying it in the domain of adverse drug reactions discovery. The experiment used nine databases containing data from 50,000 active patients of a Brazilian cancer hospital, including clinical exams, laboratory tests, physicians' anamnesis, medical prescriptions, clinical notes, medicine leaflets packages, international classification of disease, and sickness diagnosis models. This study had the hospital's ethical committee approval. A statistically significant improvement in the precision and recall of the results was obtained compared with existing approaches. The results obtained show that the credibility index proposed by the model significantly increases the quality of the evidence generated with the algorithm Random Forest. A benchmark was performed between three datasets, incremented gradually with attributes of a credibility index, obtaining a precision of 92%. Finally, we performed a benchmark with a public base of heart disease, achieving good results.

Breathing Rate Estimation from Head-Worn Photoplethysmography Sensor Data Using Machine Learning

Breathing rate is considered one of the fundamental vital signs and a highly informative indicator of physiological state. Given that the monitoring of heart activity is less complex than the monitoring of breathing, a variety of algorithms have been developed to estimate breathing activity from heart activity. However, estimating breathing rate from heart activity outside of laboratory conditions is still a challenge. The challenge is even greater when new wearable devices with novel sensor placements are being used. In this paper, we present a novel algorithm for breathing rate estimation from photoplethysmography (PPG) data acquired from a head-worn virtual reality mask equipped with a PPG sensor placed on the forehead of a subject. The algorithm is based on advanced signal processing and machine learning techniques and includes a novel quality assessment and motion artifacts removal procedure. The proposed algorithm is evaluated and compared to existing approaches from the related work using two separate datasets that contains data from a total of 37 subjects overall. Numerous experiments show that the proposed algorithm outperforms the compared algorithms, achieving a mean absolute error of 1.38 breaths per minute and a Pearson's correlation coefficient of 0.86. These results indicate that reliable estimation of breathing rate is possible based on PPG data acquired from a head-worn device.

Adaptive Fusion Based Method for Imbalanced Data Classification

The imbalance problem is widespread in real-world applications. When training a classifier on the imbalance datasets, the classifier is hard to learn an appropriate decision boundary, which causes unsatisfying classification performance. To deal with the imbalance problem, various ensemble algorithms are proposed. However, conventional ensemble algorithms do not consider exploring an effective feature space to further improve the performance. In addition, they treat the base classifiers equally and ignore the different contributions of each base classifier to the ensemble result. In order to address these problems, we propose a novel ensemble algorithm that combines effective data transformation and an adaptive weighted voting scheme. First, we utilize modified metric learning to obtain an effective feature space based on imbalanced data. Next, the base classifiers are assigned different weights adaptively. The experiments on multiple imbalanced datasets, including images and biomedical datasets verify the superiority of our proposed ensemble algorithm.

Ranks underlie outcome of combining classifiers: Quantitative roles for diversity and accuracy

Combining classifier systems potentially improves predictive accuracy, but outcomes have proven impossible to predict. Classification most commonly improves when the classifiers are "sufficiently good" (generalized as " accuracy ") and "sufficiently different" (generalized as " diversity "), but the individual and joint quantitative influence of these factors on the final outcome remains unknown. We resolve these issues. Beginning with simulated data, we develop the DIRAC framework (DIversity of Ranks and ACcuracy), which accurately predicts outcome of both score-based fusions originating from exponentially modified Gaussian distributions and rank-based fusions, which are inherently distribution independent. DIRAC was validated using biological dual-energy X-ray absorption and magnetic resonance imaging data. The DIRAC framework is domain independent and has expected utility in far-ranging areas such as clinical biomarker development/personalized medicine, clinical trial enrollment, insurance pricing, portfolio management, and sensor optimization.

Decision fusion in healthcare and medicine: a narrative review

OBJECTIVE

To provide an overview of the decision fusion (DF) technique and describe the applications of the technique in healthcare and medicine at prevention, diagnosis, treatment and administrative levels.

BACKGROUND

The rapid development of technology over the past 20 years has led to an explosion in data growth in various industries, like healthcare. Big data analysis within the healthcare systems is essential for arriving to a value-based decision over a period of time. Diversity and uncertainty in big data analytics have made it impossible to analyze data by using conventional data mining techniques and thus alternative solutions are required. DF is a form of data fusion techniques that could increase the accuracy of diagnosis and facilitate interpretation, summarization and sharing of information.

METHODS

We conducted a review of articles published between January 1980 and December 2020 from various databases such as Google Scholar, IEEE, PubMed, Science Direct, Scopus and web of science using the keywords decision fusion (DF), information fusion, healthcare, medicine and big data. A total of 141 articles were included in this narrative review.

CONCLUSIONS

Given the importance of big data analysis in reducing costs and improving the quality of healthcare; along with the potential role of DF in big data analysis, it is recommended to know the full potential of this technique including the advantages, challenges and applications of the technique before its use. Future studies should focus on describing the methodology and types of data used for its applications within the healthcare sector.

Monitoring Wheat Powdery Mildew Based on Hyperspectral, Thermal Infrared, and RGB Image Data Fusion

Powdery mildew severely affects wheat growth and yield; therefore, its effective monitoring is essential for the prevention and control of the disease and global food security. In the present study, a spectroradiometer and thermal infrared cameras were used to obtain hyperspectral signature and thermal infrared images data, and thermal infrared temperature parameters (TP) and texture features (TF) were extracted from the thermal infrared images and RGB images of wheat with powdery mildew, during the wheat flowering and filling periods. Based on the ten vegetation indices from the hyperspectral data (VI), TF and TP were integrated, and partial least square regression, random forest regression (RFR), and support vector machine regression (SVR) algorithms were used to construct a prediction model for a wheat powdery mildew disease index. According to the results, the prediction accuracy of RFR was higher than in other models, under both single data source modeling and multi-source data modeling; among the three data sources, VI was the most suitable for powdery mildew monitoring, followed by TP, and finally TF. The RFR model had stable performance in multi-source data fusion modeling (VI&TP&TF), and had the optimal estimation performance with 0.872 and 0.862 of R² for calibration and validation, respectively. The application of multi-source data collaborative modeling could improve the accuracy of remote sensing monitoring of wheat powdery mildew, and facilitate the achievement of high-precision remote sensing monitoring of crop disease status.

What Does a Language-And-Vision Transformer See: The Impact of Semantic Information on Visual Representations

Neural networks have proven to be very successful in automatically capturing the composition of language and different structures across a range of multi-modal tasks. Thus, an important question to investigate is how neural networks learn and organise such structures. Numerous studies have examined the knowledge captured by language models (LSTMs, transformers) and vision architectures (CNNs, vision transformers) for respective uni-modal tasks. However, very few have explored what structures are acquired by multi-modal transformers where linguistic and visual features are combined. It is critical to understand the representations learned by each modality, their respective interplay, and the task's effect on these representations in large-scale architectures. In this paper, we take a multi-modal transformer trained for image captioning and examine the structure of the self-attention patterns extracted from the visual stream. Our results indicate that the information about different relations between objects in the visual stream is hierarchical and varies from local to a global object-level understanding of the image. In particular, while visual representations in the first layers encode the knowledge of relations between semantically similar object detections, often constituting neighbouring objects, deeper layers expand their attention across more distant objects and learn global relations between them. We also show that globally attended objects in deeper layers can be linked with entities described in image descriptions, indicating a critical finding - the indirect effect of language on visual representations. In addition, we highlight how object-based input representations affect the structure of learned visual knowledge and guide the model towards more accurate image descriptions. A parallel question that we investigate is whether the insights from cognitive science echo the structure of representations that the current neural architecture learns. The proposed analysis of the inner workings of multi-modal transformers can be used to better understand and improve on such problems as pre-training of large-scale multi-modal architectures, multi-modal information fusion and probing of attention weights. In general, we contribute to the explainable multi-modal natural language processing and currently shallow understanding of how the input representations and the structure of the multi-modal transformer affect visual representations.

Multi-Modal Pain Intensity Assessment Based on Physiological Signals: A Deep Learning Perspective

Traditional pain assessment approaches ranging from self-reporting methods, to observational scales, rely on the ability of an individual to accurately assess and successfully report observed or experienced pain episodes. Automatic pain assessment tools are therefore more than desirable in cases where this specific ability is negatively affected by various psycho-physiological dispositions, as well as distinct physical traits such as in the case of professional athletes, who usually have a higher pain tolerance as regular individuals. Hence, several approaches have been proposed during the past decades for the implementation of an autonomous and effective pain assessment system. These approaches range from more conventional supervised and semi-supervised learning techniques applied on a set of carefully hand-designed feature representations, to deep neural networks applied on preprocessed signals. Some of the most prominent advantages of deep neural networks are the ability to automatically learn relevant features, as well as the inherent adaptability of trained deep neural networks to related inference tasks. Yet, some significant drawbacks such as requiring large amounts of data to train deep models and over-fitting remain. Both of these problems are especially relevant in pain intensity assessment, where labeled data is scarce and generalization is of utmost importance. In the following work we address these shortcomings by introducing several novel multi-modal deep learning approaches (characterized by specific supervised, as well as self-supervised learning techniques) for the assessment of pain intensity based on measurable bio-physiological data. While the proposed supervised deep learning approach is able to attain state-of-the-art inference performances, our self-supervised approach is able to significantly improve the data efficiency of the proposed architecture by automatically generating physiological data and simultaneously performing a fine-tuning of the architecture, which has been previously trained on a significantly smaller amount of data.

An IMU/ODM/UWB Joint Localization System Based on Modified Cubature Kalman Filtering

In this article, a multisensor joint localization system is proposed based on modified cubature Kalman filtering, which aims to improve the accuracy of state estimation under a moderate computational burden in the presence of high process noise. Specifically, first, the covariance of process noise is matched based on adaptive filtering. The inertial measurement unit (IMU), odometer (ODM), and ultra-wideband (UWB) information acquired by the associated sensors is then employed to augment the system state and are fused to lower the influence of process noise. In the presented localization setting, all sensors (IMU/ODM/UWB) are set to work in parallel under the federated Kalman filter (FKF) framework, which can correct the cumulative error of the internal sensor and and can improve the computational efficiency. Two sets of numerical simulations were performed to show that the proposed method can obtain accurate state estimation with a slightly increased computational burden.

Fault Diagnosis of Brake Train Based on Multi-Sensor Data Fusion

In this paper, a fault diagnosis method is proposed based on multi-sensor fusion information for a single fault and composite fault of train braking systems. Firstly, the single mass model of the train brake is established based on operating environment. Then, the pre-allocation and linear-weighted summation criterion are proposed to fuse the monitoring data. Finally, based on the improved expectation maximization, the braking modes and braking parameters are identified, and the braking faults are diagnosed in real time. The simulation results show that the braking parameters of systems can be effectively identified, and the braking faults can be diagnosed accurately based on the identification results. Even if the monitoring data are missing or abnormal, compared with the maximum fusion, the accuracies of parameter identifications and fault diagnoses can still meet the needs of the actual systems, and the effectiveness and robustness of the method can be verified.

A Redundancy Metric Set within Possibility Theory for Multi-Sensor Systems

In intelligent technical multi-sensor systems, information is often at least partly redundant—either by design or inherently due to the dynamic processes of the observed system. If sensors are known to be redundant, (i) information processing can be engineered to be more robust against sensor failures, (ii) failures themselves can be detected more easily, and (iii) computational costs can be reduced. This contribution proposes a metric which quantifies the degree of redundancy between sensors. It is set within the possibility theory. Information coming from sensors in technical and cyber–physical systems are often imprecise, incomplete, biased, or affected by noise. Relations between information of sensors are often only spurious. In short, sensors are not fully reliable. The proposed metric adopts the ability of possibility theory to model incompleteness and imprecision exceptionally well. The focus is on avoiding the detection of spurious redundancy. This article defines redundancy in the context of possibilistic information, specifies requirements towards a redundancy metric, details the information processing, and evaluates the metric qualitatively on information coming from three technical datasets.

Smartwatch-Based Eating Detection: Data Selection for Machine Learning from Imbalanced Data with Imperfect Labels

Understanding people's eating habits plays a crucial role in interventions promoting a healthy lifestyle. This requires objective measurement of the time at which a meal takes place, the duration of the meal, and what the individual eats. Smartwatches and similar wrist-worn devices are an emerging technology that offers the possibility of practical and real-time eating monitoring in an unobtrusive, accessible, and affordable way. To this end, we present a novel approach for the detection of eating segments with a wrist-worn device and fusion of deep and classical machine learning. It integrates a novel data selection method to create the training dataset, and a method that incorporates knowledge from raw and virtual sensor modalities for training with highly imbalanced datasets. The proposed method was evaluated using data from 12 subjects recorded in the wild, without any restriction about the type of meals that could be consumed, the cutlery used for the meal, or the location where the meal took place. The recordings consist of data from accelerometer and gyroscope sensors. The experiments show that our method for detection of eating segments achieves precision of 0.85, recall of 0.81, and F1-score of 0.82 in a person-independent manner. The results obtained in this study indicate that reliable eating detection using in the wild recorded data is possible with the use of wearable sensors on the wrist.

Misalignment Fault Diagnosis for Wind Turbines Based on Information Fusion

Most conventional wind turbine fault diagnosis techniques only use a single type of signal as fault feature and their performance could be limited to such signal characteristics. In this paper, multiple types of signals including vibration, temperature, and stator current are used simultaneously for wind turbine misalignment diagnosis. The model is constructed by integrated methods based on Dempster-Shafer (D-S) evidence theory. First, the time domain, frequency domain, and time-frequency domain features of the collected vibration, temperature, and stator current signal are respectively taken as the inputs of the least square support vector machine (LSSVM). Then, the LSSVM outputs the posterior probabilities of the normal, parallel misalignment, angular misalignment, and integrated misalignment of the transmission systems. The posterior probabilities are used as the basic probabilities of the evidence fusion, and the fault diagnosis is completed according to the D-S synthesis and decision rules. Considering the correlation between the inputs, the vibration and current feature vectors' dimensionalities are reduced by t-distributed stochastic neighbor embedding (t-SNE), and the improved artificial bee colony algorithm is used to optimize the parameters of the LSSVM. The results of the simulation and experimental platform demonstrate the accuracy of the proposed model and its superiority compared with other models.

Ambulatory Human Gait Phase Detection Using Wearable Inertial Sensors and Hidden Markov Model

Gait analysis, as a common inspection method for human gait, can provide a series of kinematics, dynamics and other parameters through instrumental measurement. In recent years, gait analysis has been gradually applied to the diagnosis of diseases, the evaluation of orthopedic surgery and rehabilitation progress, especially, gait phase abnormality can be used as a clinical diagnostic indicator of Alzheimer Disease and Parkinson Disease, which usually show varying degrees of gait phase abnormality. This research proposed an inertial sensor based gait analysis method. Smoothed and filtered angular velocity signal was chosen as the input data of the 15-dimensional temporal characteristic feature. Hidden Markov Model and parameter adaptive model are used to segment gait phases. Experimental results show that the proposed model based on HMM and parameter adaptation achieves good recognition rate in gait phases segmentation compared to other classification models, and the recognition results of gait phase are consistent with ground truth. The proposed wearable device used for data collection can be embedded on the shoe, which can not only collect patients' gait data stably and reliably, ensuring the integrity and objectivity of gait data, but also collect data in daily scene and ambulatory outdoor environment.

Adaptive-Neuro-Fuzzy-Based Information Fusion for the Attitude Prediction of TBMs

In a tunneling boring machine (TBM), to obtain the attitude in real time is very important for a driver. However, the current laser targeting system has a large delay before obtaining the attitude. So, an adaptive-neuro-fuzzy-based information fusion method is proposed to predict the attitude of a laser targeting system in real time. In the proposed method, a dual-rate information fusion is used to fuse the information of a laser targeting system and a two-axis inclinometer, and then obtain roll and pitch angles with a higher rate and provide a smoother attitude prediction. Considering that a measurement error exists, the adaptive neuro-fuzzy inference system (ANFIS) is proposed to model the measurement error, and then the ANFIS-based model is combined with the dual-rate information fusion to achieve high performance. Experimental results show the ANFIS-based information fusion can provide higher real-time performance and accuracy of the attitude prediction. Experimental results also verify that the ANFIS-based information fusion can solve the problem of the laser targeting system losing signals.

Information Fusion Based on Complementary Filter for SINS/CNS/GPS Integrated Navigation System of Aerospace Plane

In order to solve the problems of heavy computational load and poor real time of the information fusion method based on the federated Kalman filter (FKF), a novel information fusion method based on the complementary filter is proposed for strapdown inertial navigation (SINS)/celestial navigation system (CNS)/global positioning system (GPS) integrated navigation system of an aerospace plane. The complementary filters are designed to achieve the estimations of attitude, velocity, and position in the SINS/CNS/GPS integrated navigation system, respectively. The simulation results show that the proposed information fusion method can effectively realize SINS/CNS/GPS information fusion. Compared with FKF, the method based on complementary filter (CF) has the advantages of simplicity, small calculation, good real-time performance, good stability, no need for initial alignment, fast convergence, etc. Furthermore, the computational efficiency of CF is increased by 94.81%. Finally, the superiority of the proposed CF-based method is verified by both the semi-physical simulation and real-time system experiment.

An Adaptive Filter for Nonlinear Multi-Sensor Systems with Heavy-Tailed Noise

Aiming towards state estimation and information fusion for nonlinear systems with heavy-tailed measurement noise, a variational Bayesian Student's t-based cubature information filter (VBST-CIF) is designed. Furthermore, a multi-sensor variational Bayesian Student's t-based cubature information feedback fusion (VBST-CIFF) algorithm is also derived. In the proposed VBST-CIF, the spherical-radial cubature (SRC) rule is embedded into the variational Bayes (VB) method for a joint estimation of states and scale matrix, degree-of-freedom (DOF) parameter, as well as an auxiliary parameter in the nonlinear system with heavy-tailed noise. The designed VBST-CIF facilitates multi-sensor fusion, allowing one to derive a VBST-CIFF algorithm based on multi-sensor information feedback fusion. The performance of the proposed algorithms is assessed in target tracking scenarios. Simulation results demonstrate that the proposed VBST-CIF/VBST-CIFF outperform the conventional cubature information filter (CIF) and cubature information feedback fusion (CIFF) algorithms.

Detection of Gait Abnormalities for Fall Risk Assessment Using Wrist-Worn Inertial Sensors and Deep Learning

Falls are a significant threat to the health and independence of elderly people and represent an enormous burden on the healthcare system. Successfully predicting falls could be of great help, yet this requires a timely and accurate fall risk assessment. Gait abnormalities are one of the best predictive signs of underlying locomotion conditions and precursors of falls. The advent of wearable sensors and wrist-worn devices provides new opportunities for continuous and unobtrusive monitoring of gait during daily activities, including the identification of unexpected changes in gait. To this end, we present in this paper a novel method for determining gait abnormalities based on a wrist-worn device and a deep neural network. It integrates convolutional and bidirectional long short-term memory layers for successful learning of spatiotemporal features from multiple sensor signals. The proposed method was evaluated using data from 18 subjects, who recorded their normal gait and simulated abnormal gait while wearing impairment glasses. The data consist of inertial measurement unit (IMU) sensor signals obtained from smartwatches that the subjects wore on both wrists. Numerous experiments showed that the proposed method provides better results than the compared methods, achieving 88.9% accuracy, 90.6% sensitivity, and 86.2% specificity in the detection of abnormal walking patterns using data from an accelerometer, gyroscope, and rotation vector sensor. These results indicate that reliable fall risk assessment is possible based on the detection of walking abnormalities with the use of wearable sensors on a wrist.

A Complete Arithmetic Calculator Constructed from Spiking Neural P Systems and its Application to Information Fusion

Several variants of spiking neural P systems (SNPS) have been presented in the literature to perform arithmetic operations. However, each of these variants was designed only for one specific arithmetic operation. In this paper, a complete arithmetic calculator implemented by SNPS is proposed. An application of the proposed calculator to information fusion is also proposed. The information fusion is implemented by integrating the following three elements: (1) an addition and subtraction SNPS already reported in the literature; (2) a modified multiplication and division SNPS; (3) a novel storage SNPS, i.e. a method based on SNPS is introduced to calculate basic probability assignment of an event. This is the first attempt to apply arithmetic operation SNPS to fuse multiple information. The effectiveness of the presented general arithmetic SNPS calculator is verified by means of several examples.

An INS and UWB Fusion-Based Gyroscope Drift Correction Approach for Indoor Pedestrian Tracking

Information fusion combining inertial navigation and radio frequency (RF) technologies, is commonly applied in indoor positioning systems (IPSs) to obtain more accurate tracking results. The performance of the inertial navigation system (INS) subsystem is affected by sensor drift over time and the RF-based subsystem aims to correct the position estimate using a fusion filter. However, the inherent sensor drift is usually not corrected during fusion, which leads to increasingly erroneous estimates over a short period of time. Among the inertial sensor drifts, gyroscope drift has the most significant impact in determining the correct orientation and accurate tracking. A gyroscope drift correction approach is proposed in this study and is incorporated in an INS and ultra-wideband (UWB) fusion IPS where only distance measurements from UWB subsystem are used. The drift correction approach is based on turn detection to account for the fact that gyroscope drift is accumulated during a turn. Practical pedestrian tracking experiments are conducted to demonstrate the accuracy of the drift correction approach. With the gyroscope drift corrected, the fusion IPS is able to provide more accurate tracking performance and achieve up to 64.52% mean position error reduction when compared to the INS only tracking result.

Fault Diagnosis for Rotating Machinery Using Multiscale Permutation Entropy and Convolutional Neural Networks

In view of the limitations of existing rotating machine fault diagnosis methods in single-scale signal analysis, a fault diagnosis method based on multi-scale permutation entropy (MPE) and multi-channel fusion convolutional neural networks (MCFCNN) is proposed. First, MPE quantitatively analyzes the vibration signals of rotating machine at different scales, and obtains permutation entropy (PE) to construct feature vector sets. Then, considering the structure and spatial information between different sensor measurement points, MCFCNN constructs multiple channels in the input layer according to the number of sensors, and each channel corresponds to the MPE feature sets of different monitored points. MCFCNN uses convolutional kernels to learn the features of each channel in an unsupervised way, and fuses the features of each channel into a new feature map. At last, multi-layer perceptron is applied to fuse multi-channel features and identify faults. Through the health monitoring experiment of planetary gearbox and rolling bearing, and compared with single channel convolutional neural networks (CNN) and existing CNN based fusion methods, the proposed method based on MPE and MCFCNN model can diagnose faults with high accuracy, stability, and speed.

A Federated Derivative Cubature Kalman Filter for IMU-UWB Indoor Positioning

The inertial measurement unit and ultra-wide band signal (IMU-UWB) combined indoor positioning system has a nonlinear state equation and a linear measurement equation. In order to improve the computational efficiency and the localization performance in terms of the estimation accuracy, the federated derivative cubature Kalman filtering (FDCKF) method is proposed by combining the traditional Kalman filtering and the cubature Kalman filtering. By implementing the proposed FDCKF method, the observations of the UWB and the IMU can be effectively fused; particularly, the IMU can be continuously calibrated by UWB so that it does not generate cumulative errors. Finally, the effectiveness of the proposed algorithm is demonstrated through numerical simulations, in which FDCKF was compared with the federated cubature Kalman filter (FCKF) and the federated unscented Kalman filter (FUKF), respectively.