Freehand 3D Ultrasound Imaging Based on Probe-mounted Vision and IMU System

OBJECTIVES

Freehand three-dimensional (3D) ultrasound (US) is of great significance for clinical diagnosis and treatment, it is often achieved with the aid of external devices (optical and/or electromagnetic, etc.) that monitor the location and orientation of the US probe. However, this external monitoring is often impacted by imaging environment such as optical occlusions and/or electromagnetic (EM) interference.

METHODS

To address the above issues, we integrated a binocular camera and an inertial measurement unit (IMU) on a US probe. Subsequently, we built a tight coupling model utilizing the unscented Kalman algorithm based on Lie groups (UKF-LG), combining vision and inertial information to infer the probe's movement, through which the position and orientation of the US image frame are calculated. Finally, the volume data was reconstructed with the voxel-based hole-filling method.

RESULTS

The experiments including calibration experiments, tracking performance evaluation, phantom scans, and real scenarios scans have been conducted. The results show that the proposed system achieved the accumulated frame position error of 3.78 mm and the orientation error of 0.36° and reconstructed 3D US images with high quality in both phantom and real scenarios.

CONCLUSIONS

The proposed method has been demonstrated to enhance the robustness and effectiveness of freehand 3D US. Follow-up research will focus on improving the accuracy and stability of multi-sensor fusion to make the system more practical in clinical environments.

Dynamic event-triggered fault detection for rotary steerable systems with unknown time-varying noise covariances

This paper is concerned with the fault detection problem for the rotary steerable drilling tool system under unknown vibrations and limited computational resources. Firstly, the drilling tool system can be modeled by a nonlinear stochastic system with unknown time-varying noise covariances. Then, the dynamic event-triggered mechanism is introduced to save computational resources, and the caused transmission error is completely decoupled by nonuniform sampling. Subsequently, a novel unscented Kalman filter is proposed by combining the expectation maximization method to estimate states when noise covariances are unknown. A residual and an evaluation function are constructed to detect faults. Finally, a numerical simulation and an experiment on a drilling tool prototype validate the superior performance of the proposed fault detection scheme, which has lower missed alarm rates and consumes less time than existing methods.

Review of the limitations and potential empirical improvements of the parametric group method of data handling for rainfall modelling

This study furthers the utilisation of the parametric group method of data handling (GMDH) in assessing the possibility of rainfall modelling and prediction, using publicly available temperature and rainfall data. In using ordinary GMDH approaches, the modelling is inconclusive with no clear consistency demonstrated through coefficients of determination and analysis of variance. Hence, an empirical assessment has been undertaken to provide an explanation of the inconsistency. In doing so, state variable distribution, their classification within the fuzzy context, and the need to integrate the principle of incompatibility into the GMDH modelling format are all assessed. The mathematical foundations of GMDH are discussed within the heuristic framework of data partitioning, partial description synthesis, the limitations of the least-squares coefficient of determination, incompleteness theorem, and the necessity for an external criterion in the selection procedure for polynomials. Methods for modelling improvement include the potential for hybridisation with least square support vector machines (LSSVM), the application of filters for parameter estimation, and the combination with signal processing techniques, ensemble empirical mode decomposition (EEMD), wavelet transformation (WT), and wavelet packet transformation (WPT). These have been investigated in addition to the implementation of enhanced GMDH (eGMDH) and fuzzy GMDH (FGMDH). The inclusion of exogenous data and its application within the GMDH modelling paradigm are also discussed. The study concludes with recommendations to enhance the potential for future rainfall modelling study success using parametric GMDH.

Accurate integrated position measurement system for mobile applications in GPS-denied coal mine

The automatic positioning of underground mobile applications plays a crucial role in enabling intelligent coal mining. However, due to the diverse kinematics and dynamics of these applications, various positioning methods have been proposed to match different targets. Nonetheless, the accuracy and applicability of these methods still fall short of meeting the requirements for field applications. Based on the vibration characteristics of underground mobile devices, a multi-sensor fusion positioning system is developed to enhance the accuracy of positioning in long and narrow global positioning system denied (GPS-denied) underground coal mine roadways. The system combines inertial navigation (INS), odometer, and ultra wide band (UWB) technologies through extended Kalman filter (EKF) and unscented Kalman filter (UKF). This approach enables accurate positioning by recognizing target carrier vibrations and facilitating fast conversion between multi-sensor fusion modes. The proposed system is tested on both a small unmanned mine vehicle (UMV) and a large roadheader, demonstrating that UKF enhances stability for roadheaders with strong nonlinear vibrations while EKF is more suitable for flexible UMVs. Detailed results confirm that the proposed system achieves an accuracy level of 0.15 m, meeting most coal mine application requirements.

Adaptive closed-loop paradigm of electrophysiology for neuron models

The traditional electrophysiological experiments based on an open-loop paradigm are relatively complicated and limited when facing an individual neuron with uncertain nonlinear factors. Emerging neural technologies enable tremendous growth in experimental data leading to the curse of high-dimensional data, which obstructs the mechanism exploration of spiking activities in the neurons. In this work, we propose an adaptive closed-loop electrophysiology simulation experimental paradigm based on a Radial Basis Function neural network and a highly nonlinear unscented Kalman filter. On account of the complex nonlinear dynamic characteristics of the real neurons, the proposed simulation experimental paradigm could fit the unknown neuron models with different channel parameters and different structures (i.e. single or multiple compartments), and further compute the injected stimulus in time according to the arbitrary desired spiking activities of the neurons. However, the hidden electrophysiological states of the neurons are difficult to be measured directly. Thus, an extra Unscented Kalman filter modular is incorporated in the closed-loop electrophysiology experimental paradigm. The numerical results and theoretical analyses demonstrate that the proposed adaptive closed-loop electrophysiology simulation experimental paradigm achieves desired spiking activities arbitrarily and the hidden dynamics of the neurons are visualized by the unscented Kalman filter modular. The proposed adaptive closed-loop simulation experimental paradigm can avoid the inefficiency of data at increasingly greater scales and enhance the scalability of electrophysiological experiments, thus speeding up the discovery cycle on neuroscience.

Spatially regularized low-rank tensor approximation for accurate and fast tractography

Tractography based on diffusion Magnetic Resonance Imaging (dMRI) is the prevalent approach to the in vivo delineation of white matter tracts in the human brain. Many tractography methods rely on models of multiple fiber compartments, but the local dMRI information is not always sufficient to reliably estimate the directions of secondary fibers. Therefore, we introduce two novel approaches that use spatial regularization to make multi-fiber tractography more stable. Both represent the fiber Orientation Distribution Function (fODF) as a symmetric fourth-order tensor, and recover multiple fiber orientations via low-rank approximation. Our first approach computes a joint approximation over suitably weighted local neighborhoods with an efficient alternating optimization. The second approach integrates the low-rank approximation into a current state-of-the-art tractography algorithm based on the unscented Kalman filter (UKF). These methods were applied in three different scenarios. First, we demonstrate that they improve tractography even in high-quality data from the Human Connectome Project, and that they maintain useful results with a small fraction of the measurements. Second, on the 2015 ISMRM tractography challenge, they increase overlap, while reducing overreach, compared to low-rank approximation without joint optimization or the traditional UKF, respectively. Finally, our methods permit a more comprehensive reconstruction of tracts surrounding a tumor in a clinical dataset. Overall, both approaches improve reconstruction quality. At the same time, our modified UKF significantly reduces the computational effort compared to its traditional counterpart, and to our joint approximation. However, when used with ROI-based seeding, joint approximation more fully recovers fiber spread.

Application on real-time data

The prevalence of COVID-19 has been the most serious health challenge of the 21th century to date, concerning national health systems on a daily basis, since December 2019 when it appeared in Wuhan City. Nevertheless, most of the proposed mathematical methodologies aiming to describe the dynamics of an epidemic, rely on deterministic models that are not able to reflect the true nature of its spread. In this paper, we propose a SEIHCRDV model - an extension/improvement of the classic SIR compartmental model - which also takes into consideration the populations of exposed, hospitalized, admitted in intensive care units (ICU), deceased and vaccinated cases, in combination with an unscented Kalman filter (UKF), providing a dynamic estimation of the time dependent system's parameters. The stochastic approach is considered necessary, as both observations and system equations are characterized by uncertainties. Apparently, this new consideration is useful for examining various pandemics more effectively. The reliability of the model is examined on the daily recordings of COVID-19 in France, over a long period of 265 days. Two major waves of infection are observed, starting in January 2021, which signified the start of vaccinations in Europe providing quite encouraging predictive performance, based on the produced NRMSE values. Special emphasis is placed on proving the non-negativity of SEIHCRDV model, achieving a representative basic reproductive number R 0 and demonstrating the existence and stability of disease equilibria according to the formula produced to estimate R 0 . The model outperforms in predictive ability not only deterministic approaches but also state-of-the-art stochastic models that employ Kalman filters. Furthermore, the relevant analysis supports the importance of vaccination, as even a small increase in the dialy vaccination rate could lead to a notable reduction in mortality and hospitalizations.

Optimization of an unscented Kalman filter for an embedded platform

The unscented Kalman filter (UKF) is finding increased application in biological fields. While realizing a complex UKF system in a low-power embedded platform offers many potential benefits including wearability, it also poses significant design challenges. Here we present a method for optimizing a UKF system for realization in an embedded platform. The method seeks to minimize both computation time and error in UKF state reconstruction and forecasting. As a case study, we applied the method to a model for the rat sleep-wake regulatory system in which 432 variants of the UKF over six different variables are considered. The optimization method is divided into three stages that assess computation time, state forecast error, and state reconstruction error. We apply a cost function to variants that pass all three stages to identify a variant that computes 27 times faster than the reference variant and maintains required levels of state estimation and forecasting accuracy. We draw the following insights: 1) process noise provides leeway for simplifying the model and its integration in ways that speed computation time while maintaining state forecasting accuracy, 2) the assimilation of observed data during the UKF correction step provides leeway for simplifying the UKF structure in ways that speed computation time while maintaining state reconstruction accuracy, and 3) the optimization process can be accelerated by decoupling variables that directly impact the underlying model from variables that impact the UKF structure.

Formation control of nonholonomic unmanned ground vehicles via unscented Kalman filter-based sensor fusion approach

This paper investigates the formation control of nonholonomic unmanned ground vehicles via unscented Kalman filter-based sensor fusion approach. According to the kinematic model of single unmanned ground vehicle, the formation model of multiple unmanned ground vehicles is established. Note that the physical leader is considered instead of a virtual leader, which is more realistic. The formation control problem is converted to the stability problem of an error dynamic system. An asymptotic stability condition of the error dynamic system is derived by designing an appropriate Lyapunov function. The leader-following formation is well formed through designing effective control vectors and utilizing unscented Kalman filter-based state estimation algorithm for each follower. Some simulation examples are provided to verify the effectiveness of the proposed formation control algorithm.

Detecting microstructural white matter abnormalities of frontal pathways in children with ADHD using advanced diffusion models

Studies using diffusion tensor imaging (DTI) have documented alterations in the attention and executive system in children and adolescents with attention-deficit/hyperactivity disorder (ADHD). While abnormalities in the frontal lobe have also been reported, the associated white matter fiber bundles have not been investigated comprehensively due to the complexity in tracing them through fiber crossings. Furthermore, most studies have used a non-specific DTI model to understand white matter abnormalities. We present results from a first study that uses a multi-shell diffusion MRI (dMRI) data set coupled with an advanced multi-fiber tractography algorithm to probe microstructural measures related to axonal/cellular density and volume of fronto-striato-thalamic pathways in children with ADHD (N = 30) and healthy controls (N = 28). Head motion was firstly examined as a priority in order to assure that no group difference existed. We investigated 45 different white matter fiber bundles in the brain. After correcting for multiple comparisons, we found lower axonal/cellular packing density and volume in ADHD children in 8 of the 45 fiber bundles, primarily in the right hemisphere as follows: 1) Superior longitudinal fasciculus-II (SLF-II) (right), 2) Thalamus to precentral gyrus (right), 3) Thalamus to superior-frontal gyrus (right), 4) Caudate to medial orbitofrontal gyrus (right), 5) Caudate to precentral gyrus (right), 6) Thalamus to paracentral gyrus (left), 7) Caudate to caudal middlefrontal gyrus (left), and 8) Cingulum (bilateral). Our results demonstrate reduced axonal/cellular density and volume in certain frontal lobe white matter fiber tracts, which sub-serve the attention function and executive control systems. Further, our work shows specific microstructural abnormalities in the striato-thalamo-cortical connections, which have not been previously reported in children with ADHD.

Parameter and state estimation of backers yeast cultivation with a gas sensor array and unscented Kalman filter

Real-time information about the concentrations of substrates and biomass is the key to accurate monitoring and control of bioprocess. However, on-line measurement of these variables is a challenging task and new measurement systems are still required. An alternative are software sensors, which can be used for state and parameter estimation in bioprocesses. The software sensors predict the state of the process by using mathematical models as well as data from measured variables. The Kalman filter is a type of such sensors. In this paper, we have used the Unscented Kalman Filter (UKF) which is a nonlinear extension of the Kalman filter for on-line estimation of biomass, glucose and ethanol concentration as well as for estimating the growth rate parameters in S. cerevisiae batch cultivation, based on infrequent ethanol measurements. The UKF algorithm was validated on three different cultivations with variability of the substrate concentrations and the estimated values were compared to the off-line values. The results obtained showed that the UKF algorithm provides satisfactory results with respect to estimation of concentrations of substrates and biomass as well as the growth rate parameters during the batch cultivation.

Design of unscented Kalman filter based on the adjustments of the number and placements of the sampling points

This paper addresses a general sampling method of the unscented Kalman filter (UKF) for nonlinear state estimation. The sampling method for standard UKF is analyzed, and we propose a theorem to address the conditions that UKF provides a third order accuracy in terms of Taylor series expansion for expectation estimation by changing the number and placements of the sampling points. This theorem can be used to develop new UKF. Based on this theorem, we propose a method to design the placements of the sampling points, including the directions and lengths by optimization strategies. Simulation studies demonstrate that the proposed UKF is effective and can significantly improve the filter performance.

Fully automated left atrium segmentation from anatomical cine long-axis MRI sequences using deep convolutional neural network with unscented Kalman filter

This study proposes a fully automated approach for the left atrial segmentation from routine cine long-axis cardiac magnetic resonance image sequences using deep convolutional neural networks and Bayesian filtering. The proposed approach consists of a classification network that automatically detects the type of long-axis sequence and three different convolutional neural network models followed by unscented Kalman filtering (UKF) that delineates the left atrium. Instead of training and predicting all long-axis sequence types together, the proposed approach first identifies the image sequence type as to 2, 3 and 4 chamber views, and then performs prediction based on neural nets trained for that particular sequence type. The datasets were acquired retrospectively and ground truth manual segmentation was provided by an expert radiologist. In addition to neural net based classification and segmentation, another neural net is trained and utilized to select image sequences for further processing using UKF to impose temporal consistency over cardiac cycle. A cyclic dynamic model with time-varying angular frequency is introduced in UKF to characterize the variations in cardiac motion during image scanning. The proposed approach was trained and evaluated separately with varying amount of training data with images acquired from 20, 40, 60 and 80 patients. Evaluations over 1515 images with equal number of images from each chamber group acquired from an additional 20 patients demonstrated that the proposed model outperformed state-of-the-art and yielded a mean Dice coefficient value of 94.1%, 93.7% and 90.1% for 2, 3 and 4-chamber sequences, respectively, when trained with datasets from 80 patients.

Non-Contact Respiratory Rate Estimation in Real-Time With Modified Joint Unscented Kalman Filter

It can be life-saving to monitor the respiratory rate (RR) even for healthy people in real-time. It is reported that the infected people with coronavirus disease 2019 (COVID-19), generally develop mild respiratory symptoms in the early stage. It will be more important to continuously monitor the RR of people in nursing homes and houses with a non-contact method. Conventional, contact-based, methods are not suitable for long-term health monitoring especially in-home care services. The potentials of wireless radio signals for health care applications, such as fall detection, etc., are examined in literature. In this paper, we focus on a device-free real-time RR monitoring system using wireless signals. In our recent study, we proposed a non-contact RR monitoring system with a batch processing (delayed) estimation method. In this paper, for real-time monitoring, we modify the standard joint unscented Kalman filter (JUKF) method for this new and time-critical problem. Due to the nonlinear structure of the RR estimation problem with respect to the measurements, a novel modification is proposed to transform measurement errors into parameter errors by using the hyperbolic tangent function. It is shown in the experiments conducted with the real measurements taken using healthy volunteers that the proposed modified joint unscented Kalman filter (ModJUKF) method achieves the highest accuracy according to the windowing-based methods in the time-varying RR scenario. It is also shown that the ModJUKF not only reduces the computational complexity approximately 8.54% but also improves the accuracy 36.7% with respect to the standard JUKF method.

Computational method for multiphase flow characterization in the gas refinery

This paper presents a new computational method for the decentralized multiphase flow measurement based on the interconnections between the two subsystems to precisely estimate the states of the multiphase flow at the gas refinery. The states of the condensate and gas sub-systems were separately estimated using the Differential Mean Value Theorem by considering the relationship between two subsystems, designing an observer and converting the conditions to linear matrix inequality. To check the stability and performance of the system against the changes, the Lyapunov theory has been used. The states behavior investigated with and without disturbance in the system output and dynamics. Additionally, the Unscented Kalman Filter based on the simplified drift flux model was used to estimate the states. It is found that both observers are capable to identify the states with some differences in performance and drift flux model is sufficient for estimation of parameters and states.

Nonlinear unbiased minimum-variance filter for Mars entry autonomous navigation under large uncertainties and unknown measurement bias

High-precision navigation algorithm is essential for the future Mars pinpoint landing mission. The unknown inputs caused by large uncertainties of atmospheric density and aerodynamic coefficients as well as unknown measurement biases may cause large estimation errors of conventional Kalman filters. This paper proposes a derivative-free version of nonlinear unbiased minimum variance filter for Mars entry navigation. This filter has been designed to solve this problem by estimating the state and unknown measurement biases simultaneously with derivative-free character, leading to a high-precision algorithm for the Mars entry navigation. IMU/radio beacons integrated navigation is introduced in the simulation, and the result shows that with or without radio blackout, our proposed filter could achieve an accurate state estimation, much better than the conventional unscented Kalman filter, showing the ability of high-precision Mars entry navigation algorithm.

A stochastic algorithm for automatic hand pose and motion estimation

In this paper, a novel, robust, and simple method for automatically estimating the hand pose is proposed and validated. The method uses a multi-camera optoelectronic system and a model-based stochastic algorithm. The approach is marker-based and relies on an Unscented Kalman Filter. A hand kinematic model is introduced for constraining relative marker's positions and improving the algorithm robustness with respect to outliers and possible occlusions. The algorithm outputs are 3D coordinate measures of markers and hand joint angle values. To validate the proposed algorithm, a comparison with ground truths for angular and 3D coordinate measures is carried out. The comparative analysis shows the advantages of using the model-based stochastic algorithm with respect to standard processing software of optoelectronic cameras in terms of implementation simplicity, time consumption, and user effort. The accuracy is remarkable, with a difference of maximum 0.035r a d and 4m m with respect to angular and 3D Cartesian coordinates ground truths, respectively.

Computed tomography (CT)-compatible remote center of motion needle steering robot: Fusing CT images and electromagnetic sensor data

Lung cancer is the most common cause of cancer-related death, and early detection can reduce the mortality rate. Patients with lung nodules greater than 10 mm usually undergo a computed tomography (CT)-guided biopsy. However, aligning the needle with the target is difficult and the needle tends to deflect from a straight path. In this work, we present a CT-compatible robotic system, which can both position the needle at the puncture point and also insert and rotate the needle. The robot has a remote-center-of-motion arm which is achieved through a parallel mechanism. A new needle steering scheme is also developed where CT images are fused with electromagnetic (EM) sensor data using an unscented Kalman filter. The data fusion allows us to steer the needle using the real-time EM tracker data. The robot design and the steering scheme are validated using three experimental cases. Experimental Case I and II evaluate the accuracy and CT-compatibility of the robot arm, respectively. In experimental Case III, the needle is steered towards 5 real targets embedded in an anthropomorphic gelatin phantom of the thorax. The mean targeting error for the 5 experiments is 1.78 ± 0.70 mm. The proposed robotic system is shown to be CT-compatible with low targeting error. Small nodule size and large needle diameter are two risk factors that can lead to complications in lung biopsy. Our results suggest that nodules larger than 5 mm in diameter can be targeted using our method which may result in lower complication rate.

Integrated Navigation System Design for Micro Planetary Rovers: Comparison of Absolute Heading Estimation Algorithms and Nonlinear Filtering

This paper provides algorithms to fuse relative and absolute microelectromechanical systems (MEMS) navigation sensors, suitable for micro planetary rovers, to provide a more accurate estimation of navigation information, specifically, attitude and position. Planetary rovers have extremely slow speed (~1 cm/s) and lack conventional navigation sensors/systems, hence the general methods of terrestrial navigation may not be applicable to these applications. While relative attitude and position can be tracked in a way similar to those for ground robots, absolute navigation information is hard to achieve on a remote celestial body, like Moon or Mars, in contrast to terrestrial applications. In this study, two absolute attitude estimation algorithms were developed and compared for accuracy and robustness. The estimated absolute attitude was fused with the relative attitude sensors in a framework of nonlinear filters. The nonlinear Extended Kalman filter (EKF) and Unscented Kalman filter (UKF) were compared in pursuit of better accuracy and reliability in this nonlinear estimation problem, using only on-board low cost MEMS sensors. Experimental results confirmed the viability of the proposed algorithms and the sensor suite, for low cost and low weight micro planetary rovers. It is demonstrated that integrating the relative and absolute navigation MEMS sensors reduces the navigation errors to the desired level.

Data assimilation and modelling of patient-specific single-ventricle physiology with and without valve regurgitation

A closed-loop lumped parameter model of blood circulation is considered for single-ventricle shunt physiology. Its parameters are estimated by an inverse problem based on patient-specific haemodynamics measurements. As opposed to a black-box approach, maximizing the number of parameters that are related to physically measurable quantities motivates the present model. Heart chambers are described by a single-fibre mechanics model, and valve function is modelled with smooth opening and closure. A model for valve prolapse leading to valve regurgitation is proposed. The method of data assimilation, in particular the unscented Kalman filter, is used to estimate the model parameters from time-varying clinical measurements. This method takes into account both the uncertainty in prior knowledge related to the parameters and the uncertainty associated with the clinical measurements. Two patient-specific cases - one without regurgitation and one with atrioventricular valve regurgitation - are presented. Pulmonary and systemic circulation parameters are successfully estimated, without assumptions on their relationships. Parameters governing the behaviour of heart chambers and valves are either fixed based on biomechanics, or estimated. Results of the inverse problem are validated qualitatively through clinical measurements or clinical estimates that were not included in the parameter estimation procedure. The model and the estimation method are shown to successfully capture patient-specific clinical observations, even with regurgitation, such as the double peaked nature of valvular flows and anomalies in electrocardiogram readings. Lastly, biomechanical implications of the results are discussed.

A derivative UKF for tightly coupled INS/GPS integrated navigation

The tightly coupled INS/GPS integration introduces nonlinearity to the measurement equation of the Kalman filter due to the use of raw GPS pseudorange measurements. The extended Kalman filter (EKF) is a typical method to address the nonlinearity by linearizing the pseudorange measurements. However, the linearization may cause large modeling error or even degraded navigation solution. To solve this problem, this paper constructs a nonlinear measurement equation by including the second-order term in the Taylor series of the pseudorange measurements. Nevertheless, when using the unscented Kalman filter (UKF) to the INS/GPS integration for navigation estimation, it causes a great amount of redundant computation in the prediction process due to the linear feature of system state equation, especially for the case with system state vector in much higher dimension than measurement vector. To overcome this drawback in computational burden, this paper further develops a derivative UKF based on the constructed nonlinear measurement equation. The derivative UKF adopts the concise form of the original Kalman filter (KF) to the prediction process and employs the unscented transformation technique to the update process. Theoretical analysis and simulation results demonstrate that the derivative UKF can achieve higher accuracy with a much smaller computational cost in comparison with the traditional UKF.