GRASP reconstruction amplified with view-sharing and KWIC filtering reduces underestimation of peak velocity in highly-accelerated real-time phase-contrast MRI: A preliminary evaluation in pediatric patients with congenital heart disease

PURPOSE

To develop a highly-accelerated, real-time phase contrast (rtPC) MRI pulse sequence with 40 fps frame rate (25 ms effective temporal resolution).

METHODS

Highly-accelerated golden-angle radial sparse parallel (GRASP) with over regularization may result in temporal blurring, which in turn causes underestimation of peak velocity. Thus, we amplified GRASP performance by synergistically combining view-sharing (VS) and k-space weighted image contrast (KWIC) filtering. In 17 pediatric patients with congenital heart disease (CHD), the conventional GRASP and the proposed GRASP amplified by VS and KWIC (or GRASP + VS + KWIC) reconstruction for rtPC MRI were compared with respect to clinical standard PC MRI in measuring hemodynamic parameters (peak velocity, forward volume, backward volume, regurgitant fraction) at four locations (aortic valve, pulmonary valve, left and right pulmonary arteries).

RESULTS

The proposed reconstruction method (GRASP + VS + KWIC) achieved better effective spatial resolution (i.e., image sharpness) compared with conventional GRASP, ultimately reducing the underestimation of peak velocity from 17.4% to 6.4%. The hemodynamic metrics (peak velocity, volumes) were not significantly (p > 0.99) different between GRASP + VS + KWIC and clinical PC, whereas peak velocity was significantly (p < 0.007) lower for conventional GRASP. RtPC with GRASP + VS + KWIC also showed the ability to assess beat-to-beat variation and detect the highest peak among peaks.

CONCLUSION

The synergistic combination of GRASP, VS, and KWIC achieves 25 ms effective temporal resolution (40 fps frame rate), while minimizing the underestimation of peak velocity compared with conventional GRASP.

BRAND: a platform for closed-loop experiments with deep network models

Objective.Artificial neural networks (ANNs) are state-of-the-art tools for modeling and decoding neural activity, but deploying them in closed-loop experiments with tight timing constraints is challenging due to their limited support in existing real-time frameworks. Researchers need a platform that fully supports high-level languages for running ANNs (e.g. Python and Julia) while maintaining support for languages that are critical for low-latency data acquisition and processing (e.g. C and C++).Approach.To address these needs, we introduce the Backend for Realtime Asynchronous Neural Decoding (BRAND). BRAND comprises Linux processes, termednodes, which communicate with each other in agraphvia streams of data. Its asynchronous design allows for acquisition, control, and analysis to be executed in parallel on streams of data that may operate at different timescales. BRAND uses Redis, an in-memory database, to send data between nodes, which enables fast inter-process communication and supports 54 different programming languages. Thus, developers can easily deploy existing ANN models in BRAND with minimal implementation changes.Main results.In our tests, BRAND achieved <600 microsecond latency between processes when sending large quantities of data (1024 channels of 30 kHz neural data in 1 ms chunks). BRAND runs a brain-computer interface with a recurrent neural network (RNN) decoder with less than 8 ms of latency from neural data input to decoder prediction. In a real-world demonstration of the system, participant T11 in the BrainGate2 clinical trial (ClinicalTrials.gov Identifier: NCT00912041) performed a standard cursor control task, in which 30 kHz signal processing, RNN decoding, task control, and graphics were all executed in BRAND. This system also supports real-time inference with complex latent variable models like Latent Factor Analysis via Dynamical Systems.Significance.By providing a framework that is fast, modular, and language-agnostic, BRAND lowers the barriers to integrating the latest tools in neuroscience and machine learning into closed-loop experiments.

Implementation of a High-Accuracy Neural Network-Based Pupil Detection System for Real-Time and Real-World Applications

In this paper, the implementation of a new pupil detection system based on artificial intelligence techniques suitable for real-time and real-word applications is presented. The proposed AI-based pupil detection system uses a classifier implemented with slim-type neural networks, with its classes being defined according to the possible positions of the pupil within the eye image. In order to reduce the complexity of the neural network, a new parallel architecture is used in which two independent classifiers deliver the pupil center coordinates. The training, testing, and validation of the proposed system were performed using almost 40,000 eye images with a resolution of 320 × 240 pixels and coming from 20 different databases, with a high degree of generality. The experimental results show a detection rate of 96.29% at five pixels with a standard deviation of 3.38 pixels for all eye images from all databases and a processing speed of 100 frames/s. These results indicate both high accuracy and high processing speed, and they allow us to use the proposed solution for different real-time applications in variable and non-uniform lighting conditions, in fields such as assistive technology to communicate with neuromotor-disabled patients by using eye typing, in computer gaming, and in the automotive industry for increasing traffic safety by monitoring the driver's cognitive state.

Optimization of Using Multiple Machine Learning Approaches in Atrial Fibrillation Detection Based on a Large-Scale Data Set of 12-Lead Electrocardiograms: Cross-Sectional Study

BACKGROUND

Atrial fibrillation (AF) represents a hazardous cardiac arrhythmia that significantly elevates the risk of stroke and heart failure. Despite its severity, its diagnosis largely relies on the proficiency of health care professionals. At present, the real-time identification of paroxysmal AF is hindered by the lack of automated techniques. Consequently, a highly effective machine learning algorithm specifically designed for AF detection could offer substantial clinical benefits. We hypothesized that machine learning algorithms have the potential to identify and extract features of AF with a high degree of accuracy, given the intricate and distinctive patterns present in electrocardiogram (ECG) recordings of AF.

OBJECTIVE

This study aims to develop a clinically valuable machine learning algorithm that can accurately detect AF and compare different leads' performances of AF detection.

METHODS

We used 12-lead ECG recordings sourced from the 2020 PhysioNet Challenge data sets. The Welch method was used to extract power spectral features of the 12-lead ECGs within a frequency range of 0.083 to 24.92 Hz. Subsequently, various machine learning techniques were evaluated and optimized to classify sinus rhythm (SR) and AF based on these power spectral features. Furthermore, we compared the effects of different frequency subbands and different lead selections on machine learning performances.

RESULTS

The light gradient boosting machine (LightGBM) was found to be the most effective in classifying AF and SR, achieving an average F1-score of 0.988 across all ECG leads. Among the frequency subbands, the 0.083 to 4.92 Hz range yielded the highest F1-score of 0.985. In interlead comparisons, aVR had the highest performance (F1=0.993), with minimal differences observed between leads.

CONCLUSIONS

In conclusion, this study successfully used machine learning methodologies, particularly the LightGBM model, to differentiate SR and AF based on power spectral features derived from 12-lead ECGs. The performance marked by an average F1-score of 0.988 and minimal interlead variation underscores the potential of machine learning algorithms to bolster real-time AF detection. This advancement could significantly improve patient care in intensive care units as well as facilitate remote monitoring through wearable devices, ultimately enhancing clinical outcomes.

Real-time motion-including dose estimation of simulated multi-leaf collimator-tracked magnetic resonance-guided radiotherapy

BACKGROUND

Real-time dose estimation is a key-prerequisite to enable online intra-fraction treatment adaptation in magnetic resonance (MR)-guided radiotherapy (MRgRT). It is an essential component for the assessment of the dosimetric benefits and risks of online adaptive treatments, such as multi-leaf collimator (MLC)-tracking.

PURPOSE

We present a proof-of-concept for a software workflow for real-time dose estimation of MR-guided adaptive radiotherapy based on real-time data-streams of the linac delivery parameters and target positions.

METHODS

A software workflow, combining our in-house motion management software DynaTrack, a real-time dose calculation engine that connects to a research version of the treatment planning software (TPS) Monaco (v.6.09.00, Elekta AB, Stockholm, Sweden) was developed and evaluated. MR-guided treatment delivery on the Elekta Unity MR-linac was simulated with and without MLC-tracking for three prostate patients, previously treated on the Elekta Unity MR-linac (36.25 Gy/five fractions). Three motion scenarios were used: no motion, regular motion, and erratic prostate motion. Accumulated monitor units (MUs), centre of mass target position and MLC-leaf positions, were forwarded from DynaTrack at a rate of 25 Hz to a Monte Carlo (MC) based dose calculation engine which utilises the research GPUMCD-library (Elekta AB, Stockholm, Sweden). A rigid isocentre shift derived from the selected motion scenarios was applied to a bulk density-assigned session MR-image. The respective electron density used for treatment planning was accessed through the research Monaco TPS. The software workflow including the online dose reconstruction was validated against offline dose reconstructions. Our investigation showed that MC-based real-time dose calculations that account for all linac states (including MUs, MLC positions and target position) were infeasible, hence states were randomly sampled and used for calculation as follows; Once a new linac state was received, a dose calculation with 106 photons was started. Linac states that arrived during the time of the ongoing calculation were put into a queue. After completion of the ongoing calculation, one new linac state was randomly picked from the queue and assigned the MU accumulated from the previous state until the last sample in the queue. The queue was emptied, and the process repeated throughout treatment simulation.

RESULTS

On average 27% (23%-30%) of received samples were used in the real-time calculation, corresponding to a calculation time for one linac state of 148 ms. Median gamma pass rate (2%/3 mm local) was 100.0% (99.9%-100%) within the PTV volume and 99.1% (90.1%-99.4.0%) with a 15% dose cut off. Differences in PTVDmean , CTVDmean , RectumD2% , and BladderD2% (offline-online, % of prescribed dose) were below 0.64%. Beam-by-beam comparisons showed deviations below 0.07 Gy. Repeated simulations resulted in standard deviations below 0.31% and 0.12 Gy for the investigated volume and dose criteria respectively.

CONCLUSIONS

Real-time dose estimation was successfully performed using the developed software workflow for different prostate motion traces with and without MLC-tracking. Negligible dosimetric differences were seen when comparing online and offline reconstructed dose, enabling online intra-fraction treatment decisions based on estimates of the delivered dose.

Profiling wastewater characteristics in intra-urban catchments using online monitoring stations

This study aims to investigate the differences in intra-urban catchments with different characteristics through real-time wastewater monitoring. Monitoring stations were installed in three neighbourhoods of Barcelona to measure flow, total chemical oxygen demand (COD), pH, conductivity, temperature, and bisulfide (HS-) for 1 year. Typical wastewater profiles were obtained for weekdays, weekends, and holidays in the summer and winter seasons. The results reveal differences in waking up times and evening routines, commuting behaviour during weekends and holidays, and water consumption. The pollutant profiles contribute to a better understanding of pollution generation in households and catchment activities. Flows and COD correlate well at all stations, but there are differences in conductivity and HS- at the station level. The article concludes by discussing the operational experience of the monitoring stations.

Real-time and lightweight detection of grape diseases based on Fusion Transformer YOLO

Introduction

Grapes are prone to various diseases throughout their growth cycle, and the failure to promptly control these diseases can result in reduced production and even complete crop failure. Therefore, effective disease control is essential for maximizing grape yield. Accurate disease identification plays a crucial role in this process. In this paper, we proposed a real-time and lightweight detection model called Fusion Transformer YOLO for 4 grape diseases detection. The primary source of the dataset comprises RGB images acquired from plantations situated in North China.

Methods

Firstly, we introduce a lightweight high-performance VoVNet, which utilizes ghost convolutions and learnable downsampling layer. This backbone is further improved by integrating effective squeeze and excitation blocks and residual connections to the OSA module. These enhancements contribute to improved detection accuracy while maintaining a lightweight network. Secondly, an improved dual-flow PAN+FPN structure with Real-time Transformer is adopted in the neck component, by incorporating 2D position embedding and a single-scale Transformer Encoder into the last feature map. This modification enables real-time performance and improved accuracy in detecting small targets. Finally, we adopt the Decoupled Head based on the improved Task Aligned Predictor in the head component, which balances accuracy and speed.

Results

Experimental results demonstrate that FTR-YOLO achieves the high performance across various evaluation metrics, with a mean Average Precision (mAP) of 90.67%, a Frames Per Second (FPS) of 44, and a parameter size of 24.5M.

Conclusion

The FTR-YOLO presented in this paper provides a real-time and lightweight solution for the detection of grape diseases. This model effectively assists farmers in detecting grape diseases.

Ultrahigh-Throughput, Real-Time Flow Cytometry for Rare Cell Quantification from Whole Blood

We present an ultrahigh-throughput, real-time fluorescence cytometer comprising a viscoelastic microfluidic system and a complementary metal-oxide-semiconductor (CMOS) linear image sensor-based detection system. The flow cytometer allows for real-time quantification of a variety of fluorescence species, including micrometer-sized particles and cells, at analytical throughputs in excess of 400,000 species per second. The platform integrates a custom C++ control program and graphical user interface (GUI) to allow for the processing of raw signals, adjustment of processing parameters, and display of fluorescence intensity histograms in real time. To demonstrate the efficacy of the platform for rare event detection and its utility as a basic clinical tool, we measure and quantify patient-derived circulating tumor cells (CTCs) in peripheral blood, realizing that detection has a sensitivity of 6 CTCs per million blood cells (0.000006%) with a volumetric throughput of over 3 mL/min.

Real-time breath analysis towards a healthy human breath profile

The direct analysis of molecules contained within human breath has had significant implications for clinical and diagnostic applications in recent decades. However, attempts to compare one study to another or to reproduce previous work are hampered by: variability between sampling methodologies, human phenotypic variability, complex interactions between compounds within breath, and confounding signals from comorbidities. Towards this end, we have endeavored to create an averaged healthy human 'profile' against which follow-on studies might be compared. Through the use of direct secondary electrospray ionization combined with a high-resolution mass spectrometry and in-house bioinformatics pipeline, we seek to curate an average healthy human profile for breath and use this model to distinguish differences inter- and intra-day for human volunteers. Breath samples were significantly different in PERMANOVA analysis and ANOSIM analysis based on Time of Day, Participant ID, Date of Sample, Sex of Participant, and Age of Participant (p< 0.001). Optimal binning analysis identify strong associations between specific features and variables. These include 227 breath features identified as unique identifiers for 28 of the 31 participants. Four signals were identified to be strongly associated with female participants and one with male participants. A total of 37 signals were identified to be strongly associated with the time-of-day samples were taken. Threshold indicator taxa analysis indicated a shift in significant breath features across the age gradient of participants with peak disruption of breath metabolites occurring at around age 32. Forty-eight features were identified after filtering from which a healthy human breath profile for all participants was created.

HyperSLICE: HyperBand optimized spiral for low-latency interactive cardiac examination

PURPOSE

Interactive cardiac MRI is used for fast scan planning and MR-guided interventions. However, the requirement for real-time acquisition and near-real-time visualization constrains the achievable spatio-temporal resolution. This study aims to improve interactive imaging resolution through optimization of undersampled spiral sampling and leveraging of deep learning for low-latency reconstruction (deep artifact suppression).

METHODS

A variable density spiral trajectory was parametrized and optimized via HyperBand to provide the best candidate trajectory for rapid deep artifact suppression. Training data consisted of 692 breath-held CINEs. The developed interactive sequence was tested in simulations and prospectively in 13 subjects (10 for image evaluation, 2 during catheterization, 1 during exercise). In the prospective study, the optimized framework-HyperSLICE- was compared with conventional Cartesian real-time and breath-hold CINE imaging in terms quantitative and qualitative image metrics. Statistical differences were tested using Friedman chi-squared tests with post hoc Nemenyi test (p < 0.05).

RESULTS

In simulations the normalized RMS error, peak SNR, structural similarity, and Laplacian energy were all statistically significantly higher using optimized spiral compared to radial and uniform spiral sampling, particularly after scan plan changes (structural similarity: 0.71 vs. 0.45 and 0.43). Prospectively, HyperSLICE enabled a higher spatial and temporal resolution than conventional Cartesian real-time imaging. The pipeline was demonstrated in patients during catheter pull back, showing sufficiently fast reconstruction for interactive imaging.

CONCLUSION

HyperSLICE enables high spatial and temporal resolution interactive imaging. Optimizing the spiral sampling enabled better overall image quality and superior handling of image transitions compared with radial and uniform spiral trajectories.

Machines, mathematics, and modules: the potential to provide real-time metrics for pain under anesthesia

The brain-based assessments under anesthesia have provided the ability to evaluate pain/nociception during surgery and the potential to prevent long-term evolution of chronic pain. Prior studies have shown that the functional near-infrared spectroscopy (fNIRS)-measured changes in cortical regions such as the primary somatosensory and the polar frontal cortices show consistent response to evoked and ongoing pain in awake, sedated, and anesthetized patients. We take this basic approach and integrate it into a potential framework that could provide real-time measures of pain/nociception during the peri-surgical period. This application could have significant implications for providing analgesia during surgery, a practice that currently lacks quantitative evidence to guide patient tailored pain management. Through a simple readout of "pain" or "no pain," the proposed system could diminish or eliminate levels of intraoperative, early post-operative, and potentially, the transition to chronic post-surgical pain. The system, when validated, could also be applied to measures of analgesic efficacy in the clinic.

Photoacoustic microscopy for real-time monitoring of near-infrared optical absorbers inside biological tissue

Significance

We developed a high-speed optical-resolution photoacoustic microscopy (OR-PAM) system using a high-repetition-rate supercontinuum (SC) light source and a two-axes Galvano scanner. The OR-PAM system enabled real-time imaging of optical absorbers inside biological tissues with excellent excitation wavelength tunability.

Aim

In the near-infrared (NIR) wavelength range, high-speed OR-PAM faces limitations due to the lack of wavelength-tunable light sources. Our study aimed to enable high-speed OR-PAM imaging of various optical absorbers, including NIR contrast agents, and validate the performance of high-speed OR-PAM in the detection of circulating tumor cells (CTCs).

Approach

A high-repetition nanosecond pulsed SC light source was used for OR-PAM. The excitation wavelength was adjusted by bandpass filtering of broadband light pulses produced by an SC light source. Phantom and in vivo experiments were performed to detect tumor cells stained with an NIR contrast agent within flowing blood samples.

Results

The newly developed high-speed OR-PAM successfully detected stained cells both in the phantom and in vivo. The phantom experiment confirmed the correlation between the tumor cell detection rate and tumor cell concentration in the blood sample.

Conclusions

The high-speed OR-PAM effectively detected stained tumor cells. Combining high-speed OR-PAM with molecular probes that stain tumor cells in vivo enables in vivo CTC detection.

Computational-Intelligence-Based Scheduling with Edge Computing in Cyber-Physical Production Systems

Real-time performance and reliability are two critical indicators in cyber-physical production systems (CPPS). To meet strict requirements in terms of these indicators, it is necessary to solve complex job-shop scheduling problems (JSPs) and reserve considerable redundant resources for unexpected jobs before production. However, traditional job-shop methods are difficult to apply under dynamic conditions due to the uncertain time cost of transmission and computation. Edge computing offers an efficient solution to this issue. By deploying edge servers around the equipment, smart factories can achieve localized decisions based on computational intelligence (CI) methods offloaded from the cloud. Most works on edge computing have studied task offloading and dispatching scheduling based on CI. However, few of the existing methods can be used for behavior-level control due to the corresponding requirements for ultralow latency (10 ms) and ultrahigh reliability (99.9999% in wireless transmission), especially when unexpected computing jobs arise. Therefore, this paper proposes a dynamic resource prediction scheduling (DRPS) method based on CI to achieve real-time localized behavior-level control. The proposed DRPS method primarily focuses on the schedulability of unexpected computing jobs, and its core ideas are (1) to predict job arrival times based on a backpropagation neural network and (2) to perform real-time migration in the form of human-computer interaction based on the results of resource analysis. An experimental comparison with existing schemes shows that our DRPS method improves the acceptance ratio by 25.9% compared to the earliest deadline first scheme.

UAV Atmosphere Sounding for Rocket Launch Support

One of the crucial branches of activity at the Łukasiewicz Research Network-Institute of Aviation is developing a suborbital rocket vehicle capable of launching small payloads beyond the Earth's atmosphere, reaching over 100 km in altitude. Ensuring safety is a primary concern, particularly given the finite flight zone and impact area. Crucial to safety analysis is the wind profile, especially in the very first seconds of a flight, when rocket velocity is of the same order as the wind speed. Traditional near-ground wind data sources, ranging from wind towers to numerical models of the atmosphere, have limitations. Wind towers are costly and unfeasible at many test ranges used for launches, while numerical modeling may not reflect the specific ground profile near the launcher due to their large cell size (2 to +10 km). Meteorological balloons are not favorable for such measurements as they aim to provide the launch operator with a wind profile at high altitudes, and are launched only 1-2 times per flight attempt. Our study sought to prototype a wind measurement system designed to acquire near-ground wind profile data. It focuses on measuring wind direction and speed at near-ground altitudes with higher flight frequency, offering data on demand shortly before launch to help ensure safety. This atmosphere sounding system consists of an Unmanned Aerial Vehicle (UAV) equipped with an onboard ultrasonic wind sensor. Some reports in the literature have discussed the possibility of using UAV-borne anemometers, but the topic of measurement errors introduced by placing the anemometer onboard an UAV remains under studied. Limited research in this area underlines the need for experimental validation of design choices-for specific types of UAVs, anemometers, and mounting. This paper presents a literature review, a detailed overview of the prototyped system, and flight test results in both natural (outdoor) and controlled (indoor, no wind) conditions. Data from the UAV system's anemometer was benchmarked against a stationary reference weather station, in order to examine the influence of the UAV's rotor on the anemometer readings. Our findings show a wind speed Root Mean Square Error (RMSE) of 5 m/s and a directional RMSE of below 5.3° (both averaged for 1 min). The results were also compared with similar UAV-based wind measurements. The prototyped system was successfully used in a suborbital rocket launch campaign, thus demonstrating the feasibility of integrating UAVs with dedicated sensors for performing regular meteorological measurements in automatic mode.

Open Hardware Implementation of Real-Time Phase and Amplitude Estimation for Neurophysiologic Signals

Adaptive deep brain stimulation (aDBS) is a promising concept in the field of DBS that consists of delivering electrical stimulation in response to specific events. Dynamic adaptivity arises when stimulation targets dynamically changing states, which often calls for a reliable and fast causal estimation of the phase and amplitude of the signals. Here, we present an open-hardware implementation that exploits the concepts of resonators and Hilbert filters embedded in an open-hardware platform. To emulate real-world scenarios, we built a hardware setup that included a system to replay and process different types of physiological signals and test the accuracy of the instantaneous phase and amplitude estimates. The results show that the system can provide a precise and reliable estimation of the phase even in the challenging scenario of dealing with high-frequency oscillations (~250 Hz) in real-time. The framework might be adopted in neuromodulation studies to quickly test biomarkers in clinical and preclinical settings, supporting the advancement of aDBS.

Impact of assigning 2-field HLA alleles from real-time PCR on deceased donor assessments and conformance with high resolution alleles

Waitlisted sensitised transplant recipients with HLA allele level antibodies to their own HLA antigen family are disadvantaged by current deficiencies in HLA typing for deceased donors. This is primarily because at time of organ allocation, HLA typing is provided at antigen level whereas solid phase assays provide allele level antibody definition. The gold standard for HLA allele typing is next generation sequencing (NGS), however time limitations with established NGS systems prevent NGS use for deceased donors. Instead, many labs use a real-time PCR (qPCR) antigen level result for deceased donors, which can disadvantage sensitised patients. Here, we compared assigning qPCR 2-field alleles to qPCR antigen level to determine the impact on virtual crossmatch (VXM) and discuss impact on donor-specific antibody (DSA) assignments. 244 consecutive deceased donors were HLA typed to allelic level by qPCR (LinkSeq SABR) and subsequently by NGS (One Lambda Alltype). The impact of qPCR allele assignments on potential DSA identification was investigated, by retrospectively investigating all 3904 VXMs, where recipient DSA assessments were assessed against donor HLA, was performed within the cohort. There was 96.3% concordance between qPCR and NGS for all allele level loci, with HLA-A; DQB1; and DPB1 having best agreement (99.4%, 98.4% and 99.4% respectively). Of the 3904 VXMs with qPCR allele assignment, there were 13 (<1%) occasions where the potential DSA assignment was impacted, with DQA1 having the most impact. Assigning alleles derived from qPCR to define unacceptable antigens for VXMs, can allow improved access to donor offers for sensitised patients by better defining alleles.

Reconstruction of multi-animal PET acquisitions with anisotropically variant PSF

Among other factors such as random, attenuation and scatter corrections, uniform spatial resolution is key to performing accurate quantitative studies in Positron emission tomography (PET). Particularly in preclinical PET studies involving simultaneous acquisition of multiple animals, the degradation of image resolution due to the depth of interaction (DOI) effect far from the center of the Field of View (FOV) becomes a significant concern. In this work, we incorporated a spatially-variant resolution model into a real time iterative reconstruction code to obtain accurate images of multi-animal acquisition. We estimated the spatially variant point spread function (SV-PSF) across the FOV using measurements and Monte Carlo (MC) simulations. The SV-PSF obtained was implemented in a GPU-based Ordered subset expectation maximization (OSEM) reconstruction code, which includes scatter, attenuation and random corrections. The method was evaluated with acquisitions from two preclinical PET/CT scanners of the SEDECAL Argus family: a Derenzo phantom placed 2 cm off center in the 4R-SuperArgus, and a multi-animal study with 4 mice in the 6R-SuperArgus. The SV-PSF reconstructions showed uniform spatial resolution without significant increase in reconstruction time, with superior image quality compared to the uniform PSF model.

Alternative intraoperative optical imaging modalities for fluorescence angiography in gastrointestinal surgery: spectral imaging and imaging photoplethysmography

INTRODUCTION

Intraoperative near-infrared fluorescence angiography with indocyanine green (ICG-FA) is a well-established modality in gastrointestinal surgery. Its main drawback is the application of a fluorescent agent with possible side effects for patients. The goal of this review paper is the presentation of alternative, non-invasive optical imaging methods and their comparison with ICG-FA.

MATERIAL AND METHODS

The principles of ICG-FA, spectral imaging, imaging photoplethysmography (iPPG), and their applications in gastrointestinal surgery are described based on selected published works.

RESULTS

The main applications of the three modalities are the evaluation of tissue perfusion, the identification of risk structures, and tissue segmentation or classification. While the ICG-FA images are mainly evaluated visually, leading to subjective interpretations, quantitative physiological parameters and tissue segmentation are provided in spectral imaging and iPPG. The combination of ICG-FA and spectral imaging is a promising method.

CONCLUSIONS

Non-invasive spectral imaging and iPPG have shown promising results in gastrointestinal surgery. They can overcome the main drawbacks of ICG-FA, i.e. the use of contrast agents, the lack of quantitative analysis, repeatability, and a difficult standardization of the acquisition. Further technical improvements and clinical evaluations are necessary to establish them in daily clinical routine.

Real-Time Evaluation of Helicobacter pylori Infection by Convolution Neural Network During White-Light Endoscopy: A Prospective, Multicenter Study (With Video)

INTRODUCTION

Convolutional neural network during endoscopy may facilitate evaluation of Helicobacter pylori infection without obtaining gastric biopsies. The aim of the study was to evaluate the diagnosis accuracy of a computer-aided decision support system for H. pylori infection (CADSS-HP) based on convolutional neural network under white-light endoscopy.

METHODS

Archived video recordings of upper endoscopy with white-light examinations performed at Sir Run Run Shaw Hospital (January 2019-September 2020) were used to develop CADSS-HP. Patients receiving endoscopy were prospectively enrolled (August 2021-August 2022) from 3 centers to calculate the diagnostic property. Accuracy of CADSS-HP for H. pylori infection was also compared with endoscopic impression, urea breath test (URT), and histopathology. H. pylori infection was defined by positive test on histopathology and/or URT.

RESULTS

Video recordings of 599 patients who received endoscopy were used to develop CADSS-HP. Subsequently, 456 patients participated in the prospective evaluation including 189 (41.4%) with H. pylori infection. With a threshold of 0.5, CADSS-HP achieved an area under the curve of 0.95 (95% confidence interval [CI], 0.93-0.97) with sensitivity and specificity of 91.5% (95% CI 86.4%-94.9%) and 88.8% (95% CI 84.2%-92.2%), respectively. CADSS-HP demonstrated higher sensitivity (91.5% vs 78.3%; mean difference = 13.2%, 95% CI 5.7%-20.7%) and accuracy (89.9% vs 83.8%, mean difference = 6.1%, 95% CI 1.6%-10.7%) compared with endoscopic diagnosis by endoscopists. Sensitivity of CADSS-HP in diagnosing H. pylori was comparable with URT (91.5% vs 95.2%; mean difference = 3.7%, 95% CI -1.8% to 9.4%), better than histopathology (91.5% vs 82.0%; mean difference = 9.5%, 95% CI 2.3%-16.8%).

DISCUSSION

CADSS-HP achieved high sensitivity in the diagnosis of H. pylori infection in the real-time test, outperforming endoscopic diagnosis by endoscopists and comparable with URT. Clinicaltrials.gov ; ChiCTR2000030724.

Real-Time Suicide Surveillance: Comparison of International Surveillance Systems and Recommended Best Practice

OBJECTIVE

Some countries have implemented systems to monitor suicides in real-time. These systems differ because of the various ways in which suicides are identified and recorded. The main objective of this study was to conduct an international comparison of major real-time suicide mortality surveillance systems to identify joint strengths, challenges, and differences, and thereby inform best-practice criteria at local, national, and international levels.

METHODS

Five major real-time suicide mortality surveillance systems of various coverage levels were identified and selected for review via an internet-based scoping exercise and prior knowledge of existing systems. Key information including the system components and practices was collated from those organizations that developed and operate each system using a structured template. The information was narratively and critically synthesized to determine similarities and differences between the systems.

RESULTS

The comparative review of the five established real-time suicide surveillance systems revealed more commonalities than differences overall. Commonalities included rapid, routine surveillance based on minimal, provisional data to facilitate timely intervention and postvention efforts. Identified differences include the timeliness of case submission and system infrastructure.

CONCLUSION

The recommended criteria could promote replicable components and practices in real-time suicide surveillance while offering flexibility in adapting to regional/local circumstances and resource availability.HIGHLIGHTSEvidence-informed recommendations for current best practice in real-time suicide surveillance.Proposed comprehensive framework can be adapted based on available resources and capacity.Real-time suicide mortality data facilitates rapid data-driven decision-making in suicide prevention.

Real-Time Sensor-Embedded Neural Network for Human Activity Recognition

This article introduces a novel approach to human activity recognition (HAR) by presenting a sensor that utilizes a real-time embedded neural network. The sensor incorporates a low-cost microcontroller and an inertial measurement unit (IMU), which is affixed to the subject's chest to capture their movements. Through the implementation of a convolutional neural network (CNN) on the microcontroller, the sensor is capable of detecting and predicting the wearer's activities in real-time, eliminating the need for external processing devices. The article provides a comprehensive description of the sensor and the methodology employed to achieve real-time prediction of subject behaviors. Experimental results demonstrate the accuracy and high inference performance of the proposed solution for real-time embedded activity recognition.

Multi-Scale FPGA-Based Infrared Image Enhancement by Using RGF and CLAHE

Infrared sensors capture thermal radiation emitted by objects. They can operate in all weather conditions and are thus employed in fields such as military surveillance, autonomous driving, and medical diagnostics. However, infrared imagery poses challenges such as low contrast and indistinct textures due to the long wavelength of infrared radiation and susceptibility to interference. In addition, complex enhancement algorithms make real-time processing challenging. To address these problems and improve visual quality, in this paper, we propose a multi-scale FPGA-based method for real-time enhancement of infrared images by using rolling guidance filter (RGF) and contrast-limited adaptive histogram equalization (CLAHE). Specifically, the original image is first decomposed into various scales of detail layers and a base layer using RGF. Secondly, we fuse detail layers of diverse scales, then enhance the detail information by using gain coefficients and employ CLAHE to improve the contrast of the base layer. Thirdly, we fuse the detail layers and base layer to obtain the image with global details of the input image. Finally, the proposed algorithm is implemented on an FPGA using advanced high-level synthesis tools. Comprehensive testing of our proposed method on the AXU15EG board demonstrates its effectiveness in significantly improving image contrast and enhancing detail information. At the same time, real-time enhancement at a speed of 147 FPS is achieved for infrared images with a resolution of 640 × 480.

Digital twin-driven dynamic monitoring system of the upper limb force

Digital twin represents the core technology to realize the dynamic monitoring of complex industrial systems. However, the human body, as the most complex system in the physical world, digital twin is rarely applied in it. In this study, we successfully demonstrated a digital twin in the human biomedical application by proposing a dynamic monitoring system of the upper limb force. In this system, the real upper limb drives the motion of the virtual one in real-time and dynamically updates the force. Meanwhile, the virtual upper limb feeds back the monitoring-results of the force to the controller of the real upper limb via immersive virtual reality interaction. Experimental results of the typical motions of the upper limb revealed that the proposed system functioned interactively in real-time in a non-invasive manner, while ensuring the accurate solving of the muscle force. In conclusion, our digital twin-driven system is of great importance for rehabilitation medicine, biomechanical scientific research and physical training, promoting the application of the digital twin in the human biomedical field.

Real-Time EEG-Based Emotion Recognition

Most studies have demonstrated that EEG can be applied to emotion recognition. In the process of EEG-based emotion recognition, real-time is an important feature. In this paper, the real-time problem of emotion recognition based on EEG is explained and analyzed. Secondly, the short time window length and attention mechanisms are designed on EEG signals to follow emotion change over time. Then, long short-term memory with the additive attention mechanism is used for emotion recognition, due to timely emotion updates, and the model is applied to the SEED and SEED-IV datasets to verify the feasibility of real-time emotion recognition. The results show that the model performs relatively well in terms of real-time performance, with accuracy rates of 85.40% and 74.26% on SEED and SEED-IV, but the accuracy rate has not reached the ideal state due to data labeling and other losses in the pursuit of real-time performance.

Real-time 4D MRI using MR signature matching (MRSIGMA) on a 1.5T MR-Linac system

Objective. To develop real-time 4D MRI using MR signature matching (MRSIGMA) for volumetric motion imaging in patients with pancreatic cancer on a 1.5T MR-Linac system.Approach. Two consecutive MRI scans with 3D golden-angle radial stack-of-stars acquisitions were performed on ten patients with inoperable pancreatic cancer. The complete first scan (905 angles) was used to compute a 4D motion dictionary including ten pairs of 3D motion images and signatures. The second scan was used for real-time imaging, where each angle (275 ms) was processed separately to match it to one of the dictionary entries. The complete second scan was also used to compute a 4D reference to assess motion tracking performance.Dicecoefficients of the gross tumor volume (GTV) and two organs-at-risk (duodenum-stomach and small bowel) were calculated between signature matching and reference. In addition, volume changes, displacements, center of mass shifts, andDicescores over time were calculated to characterize motion.Main results. Total imaging latency of MRSIGMA (acquisition + matching) was less than 300 ms. TheDicecoefficients were 0.87 ± 0.06 (GTV), 0.86 ± 0.05 (duodenum-stomach), and 0.85 ± 0.05 (small bowel), which indicate high accuracy (high mean value) and low uncertainty (low standard deviation) of MRSIGMA for real-time motion tracking. The center of mass shift was 3.1 ± 2.0 mm (GTV), 5.3 ± 3.0 mm (duodenum-stomach), and 3.4 ± 1.5 mm (small bowel). TheDicescores over time (0.97 ± [0.01-0.03]) were similarly high for MRSIGMA and reference scans in all the three contours.Significance. This work demonstrates the feasibility of real-time 4D MRI using MRSIGMA for volumetric motion tracking on a 1.5T MR-Linac system. The high accuracy and low uncertainty of real-time MRSIGMA is an essential step towards continuous treatment adaptation of tumors affected by real-time respiratory motion and could ultimately improve treatment safety by optimizing ablative dose delivery near gastrointestinal organs.

Automatic image-based tracking of gadolinium-filled balloon wedge catheters for MRI-guided cardiac catheterization using deep learning

Introduction

Magnetic Resonance Imaging (MRI) is a promising alternative to standard x-ray fluoroscopy for the guidance of cardiac catheterization procedures as it enables soft tissue visualization, avoids ionizing radiation and provides improved hemodynamic data. MRI-guided cardiac catheterization procedures currently require frequent manual tracking of the imaging plane during navigation to follow the tip of a gadolinium-filled balloon wedge catheter, which unnecessarily prolongs and complicates the procedures. Therefore, real-time automatic image-based detection of the catheter balloon has the potential to improve catheter visualization and navigation through automatic slice tracking.

Methods

In this study, an automatic, parameter-free, deep-learning-based post-processing pipeline was developed for real-time detection of the catheter balloon. A U-Net architecture with a ResNet-34 encoder was trained on semi-artificial images for the segmentation of the catheter balloon. Post-processing steps were implemented to guarantee a unique estimate of the catheter tip coordinates. This approach was evaluated retrospectively in 7 patients (6M and 1F, age = 7 ± 5 year) who underwent an MRI-guided right heart catheterization procedure with all images acquired in an orientation unseen during training.

Results

The overall accuracy, specificity and sensitivity of the proposed catheter tracking strategy over all 7 patients were 98.4 ± 2.0%, 99.9 ± 0.2% and 95.4 ± 5.5%, respectively. The computation time of the deep-learning-based segmentation step was ∼10 ms/image, indicating its compatibility with real-time constraints.

Conclusion

Deep-learning-based catheter balloon tracking is feasible, accurate, parameter-free, and compatible with real-time conditions. Online integration of the technique and its evaluation in a larger patient cohort are now warranted to determine its benefit during MRI-guided cardiac catheterization.

Real-Time Nitrate Ion Monitoring with Poly(3,4-ethylenedioxythiophene) (PEDOT) Materials

Nitrate (NO3) pollution in groundwater, caused by various factors both natural and synthetic, contributes to the decline of human health and well-being. Current techniques used for nitrate detection include spectroscopic, electrochemical, chromatography, and capillary electrophoresis. It is highly desired to develop a simple cost-effective alternative to these complex methods for nitrate detection. Therefore, a real-time poly (3,4-ethylenedioxythiophene) (PEDOT)-based sensor for nitrate ion detection via electrical property change is introduced in this study. Vapor phase polymerization (VPP) is used to create a polymer thin film. Variations in specific parameters during the process are tested and compared to develop new insights into PEDOT sensitivity towards nitrate ions. Through this study, the optimal fabrication parameters that produce a sensor with the highest sensitivity toward nitrate ions are determined. With the optimized parameters, the electrical resistance response of the sensor to 1000 ppm nitrate solution is 41.79%. Furthermore, the sensors can detect nitrate ranging from 1 ppm to 1000 ppm. The proposed sensor demonstrates excellent potential to detect the overabundance of nitrate ions in aqueous solutions in real time.

Aided diagnosis of thyroid nodules based on an all-optical diffraction neural network

Background

Thyroid cancer is the most common malignancy in the endocrine system, with its early manifestation being the presence of thyroid nodules. With the advantages of convenience, noninvasiveness, and a lack of radiation, ultrasound is currently the first-line screening tool for the clinical diagnosis of thyroid nodules. The use of artificial intelligence to assist diagnosis is an emerging technology. This paper proposes the use optical neural networks for potential application in the auxiliary diagnosis of thyroid nodules.

Methods

Ultrasound images obtained from January 2013 to December 2018 at the Institute and Hospital of Oncology, Tianjin Medical University, were included in a dataset. Patients who consecutively underwent thyroid ultrasound diagnosis and follow-up procedures were included. We developed an all-optical diffraction neural network to assist in the diagnosis of thyroid nodules. The network is composed of 5 diffraction layers and 1 detection plane. The input image is placed 10 mm away from the first diffraction layer. The input of the diffractive neural network is light at a wavelength of 632.8 nm, and the output of this network is determined by the amplitude and light intensity obtained from the detection region.

Results

The all-optical neural network was used to assist in the diagnosis of thyroid nodules. In the classification task of benign and malignant thyroid nodules, the accuracy of classification on the test set was 97.79%, with an area under the curve value of 99.8%. In the task of detecting thyroid nodules, we first trained the model to determine whether any nodules were present and achieved an accuracy of 84.92% on the test set.

Conclusions

Our study demonstrates the potential of all-optical neural networks in the field of medical image processing. The performance of the models based on optical neural networks is comparable to other widely used network models in the field of image classification.

Temperature Sensors Integrated with an Electrochromic Readout toward Visual Detection

Temperature sensors have attracted great attention for personal health care and disease diagnosis in recent years. However, it is still a great challenge to fabricate reliable and highly sensitive temperature sensors that can convert physiological signals into easily readable signals in a convenient way. Herein, an integrated smart temperature sensor system based on a traditional temperature sensor and electrochromic display is proposed for real-time visual detection of temperature. Significantly, a voltage-regulated electrochromic device (ECD) based on tungsten oxide (WO3) and polyaniline (PANI) as the real-time visualization window was integrated into the platform to provide feedback on the temperature change. The ECD would change its color from green to blue based on the electrical signal of the temperature sensor, resulting in a visualized readout that can be monitored through our naked eye. Additionally, the smart temperature sensor system possesses an extremely durable property and cycle stability, remaining around 90% of the initial value even after 15,000 s continuous cycle. Thus, the novel design and low power consumption advantages make it a good candidate to pave the way for developing interactive wearable electronics and intelligent robots as real-time temperature feedback systems.

A Real-Time Path Planning Method for Urban Low-Altitude Logistics UAVs

To solve the problem of poor real-time performance in path planning algorithms for unmanned aerial vehicles (UAVs) in low-altitude urban logistics, a path planning method combining modified Beetle Antennae Search (BAS) with the Simulated Annealing (SA) algorithm is proposed. Firstly, based on the requirements of task execution and constraints of UAV flight, a fitness function for real-time search of waypoints is designed while ensuring the safety and obstacle avoidance of the UAV. Then, to improve the search accuracy and real-time performance, determining the initial search direction in the BAS algorithm is improved, while the search step size and antennae sensing length are updated in real-time according to the distance between the UAV and the obstacle. Finally, the SA algorithm is combined with the BAS algorithm to update the waypoints, expanding the search range of each waypoint, avoiding the process of updating the waypoints from becoming trapped in the local optimal waypoints. Meanwhile, the effectiveness of the next waypoint is evaluated based on the Metropolis criterion. This paper generates a virtual urban logistics distribution environment based on the density and distribution of urban buildings, and compares the performance of algorithms in obstacle-sparse, obstacle-moderate, and obstacle-dense environments. The simulation results demonstrate that the improved method in this paper has a more significant capacity for environmental adaptation. In terms of the path length, waypoints, safety obstacle avoidance, and smoothness, the planned path outperforms the original BAS method. It satisfies the needs of real-time path planning for UAVs involved in urban low-altitude logistics.

Fast-MFQE: A Fast Approach for Multi-Frame Quality Enhancement on Compressed Video

For compressed images and videos, quality enhancement is essential. Though there have been remarkable achievements related to deep learning, deep learning models are too large to apply to real-time tasks. Therefore, a fast multi-frame quality enhancement method for compressed video, named Fast-MFQE, is proposed to meet the requirement of video-quality enhancement for real-time applications. There are three main modules in this method. One is the image pre-processing building module (IPPB), which is used to reduce redundant information of input images. The second one is the spatio-temporal fusion attention (STFA) module. It is introduced to effectively merge temporal and spatial information of input video frames. The third one is the feature reconstruction network (FRN), which is developed to effectively reconstruct and enhance the spatio-temporal information. Experimental results demonstrate that the proposed method outperforms state-of-the-art methods in terms of lightweight parameters, inference speed, and quality enhancement performance. Even at a resolution of 1080p, the Fast-MFQE achieves a remarkable inference speed of over 25 frames per second, while providing a PSNR increase of 19.6% on average when QP = 37.

Smart Floor Mats for a Health Monitoring System Based on Textile Pressure Sensing: Development and Usability Study

BACKGROUND

The rise in single-person households has resulted in social problems like loneliness and isolation, commonly known as "death by loneliness." Various factors contribute to this increase, including a desire for independent living and communication challenges within families due to societal changes. Older individuals living alone are particularly susceptible to loneliness and isolation due to limited family communication and a lack of social activities. Addressing these issues is crucial, and proactive solutions are needed. It is important to explore diverse measures to tackle the challenges of single-person households and prevent deaths due to loneliness in our society.

OBJECTIVE

Non-face-to-face health care service systems have gained widespread interest owing to the rapid development of smart home technology. Particularly, a health monitoring system must be developed to manage patients' health status and send alerts for dangerous situations based on their activity. Therefore, in this study, we present a novel health monitoring system based on the auto-mapping method, which uses real-time position sensing mats.

METHODS

The smart floor mats are operated as piezo-resistive devices, which are composed of a carbon nanotube-based conductive textile, electrodes, main processor circuit, and a mat. The developed smart floor system acquires real-time position information using a multiconnection method between the modules based on the auto-mapping algorithm, which automatically creates a spatial map. The auto-mapping algorithm allows the user to freely set various activity areas through floor mapping. Then, the monitoring system was evaluated in a room with an area of 41.3 m2, which is embedded with the manufactured floor mats and monitoring application.

RESULTS

This monitoring system automatically acquires information on the total number, location, and direction of the mats and creates a spatial map. The position sensing mats can be easily configured with a simple structure by using a carbon nanotube-based piezo-resistive textile. The mats detect the activity in real time and record location information since they are connected through auto-mapping technology.

CONCLUSIONS

This system allows for the analysis of patients' behavior patterns and the management of health care on the web by providing important basic information for activity patterns in the monitoring system. The proposed smart floor system can serve as the foundation for smart home applications in the future, which include health care, intelligent automation, and home security, owing to its advantages of low cost, large area, and high reliability.

Real-Time Bucket Pose Estimation Based on Deep Neural Network and Registration Using Onboard 3D Sensor

Real-time and accurate bucket pose estimation plays a vital role in improving the intelligence level of mining excavators, as the bucket is a crucial component of the excavator. Existing methods for bucket pose estimation are realized by installing multiple non-visual sensors. However, these sensors suffer from cumulative errors caused by loose connections and short service lives caused by strong vibrations. In this paper, we propose a method for bucket pose estimation based on deep neural network and registration to solve the large registration error problem caused by occlusion. Specifically, we optimize the Point Transformer network for bucket point cloud semantic segmentation, significantly improving the segmentation accuracy. We employ point cloud preprocessing and continuous frame registration to reduce the registration distance and accelerate the Fast Iterative Closest Point algorithm, enabling real-time pose estimation. By achieving precise semantic segmentation and faster registration, we effectively address the problem of intermittent pose estimation caused by occlusion. We collected our own dataset for training and testing, and the experimental results are compared with other relevant studies, validating the accuracy and effectiveness of the proposed method.

A wearable group-synchronized EEG system for multi-subject brain-computer interfaces

Objective

The multi-subject brain-computer interface (mBCI) is becoming a key tool for the analysis of group behaviors. It is necessary to adopt a neural recording system for collaborative brain signal acquisition, which is usually in the form of a fixed wire.

Approach

In this study, we designed a wireless group-synchronized neural recording system that supports real-time mBCI and event-related potential (ERP) analysis. This system uses a wireless synchronizer to broadcast events to multiple wearable EEG amplifiers. The simultaneously received broadcast signals are marked in data packets to achieve real-time event correlation analysis of multiple targets in a group.

Main results

To evaluate the performance of the proposed real-time group-synchronized neural recording system, we conducted collaborative signal sampling on 10 wireless mBCI devices. The average signal correlation reached 99.8%, the amplitude of average noise was 0.87 μV, and the average common mode rejection ratio (CMRR) reached 109.02 dB. The minimum synchronization error is 237 μs. We also tested the system in real-time processing of the steady-state visual-evoked potential (SSVEP) ranging from 8 to 15.8 Hz. Under 40 target stimulators, with 2 s data length, the average information transfer rate (ITR) reached 150 ± 20 bits/min, and the highest reached 260 bits/min, which was comparable to the marketing leading EEG system (the average: 150 ± 15 bits/min; the highest: 280 bits/min). The accuracy of target recognition in 2 s was 98%, similar to that of the Synamps2 (99%), but a higher signal-to-noise ratio (SNR) of 5.08 dB was achieved. We designed a group EEG cognitive experiment; to verify, this system can be used in noisy settings.

Significance

The evaluation results revealed that the proposed real-time group-synchronized neural recording system is a high-performance tool for real-time mBCI research. It is an enabler for a wide range of future applications in collaborative intelligence, cognitive neurology, and rehabilitation.

Delayed closed-loop neurostimulation for the treatment of pathological brain rhythms in mental disorders: a computational study

Mental disorders are among the top most demanding challenges in world-wide health. A large number of mental disorders exhibit pathological rhythms, which serve as the disorders characteristic biomarkers. These rhythms are the targets for neurostimulation techniques. Open-loop neurostimulation employs stimulation protocols, which are rather independent of the patients health and brain state in the moment of treatment. Most alternative closed-loop stimulation protocols consider real-time brain activity observations but appear as adaptive open-loop protocols, where e.g., pre-defined stimulation sets in if observations fulfil pre-defined criteria. The present theoretical work proposes a fully-adaptive closed-loop neurostimulation setup, that tunes the brain activities power spectral density (PSD) according to a user-defined PSD. The utilized brain model is non-parametric and estimated from the observations via magnitude fitting in a pre-stimulus setup phase. Moreover, the algorithm takes into account possible conduction delays in the feedback connection between observation and stimulation electrode. All involved features are illustrated on pathological α- and γ-rhythms known from psychosis. To this end, we simulate numerically a linear neural population brain model and a non-linear cortico-thalamic feedback loop model recently derived to explain brain activity in psychosis.

An open-source human-in-the-loop BCI research framework: method and design

Brain-computer interfaces (BCIs) translate brain activity into digital commands for interaction with the physical world. The technology has great potential in several applied areas, ranging from medical applications to entertainment industry, and creates new conditions for basic research in cognitive neuroscience. The BCIs of today, however, offer only crude online classification of the user's current state of mind, and more sophisticated decoding of mental states depends on time-consuming offline data analysis. The present paper addresses this limitation directly by leveraging a set of improvements to the analytical pipeline to pave the way for the next generation of online BCIs. Specifically, we introduce an open-source research framework that features a modular and customizable hardware-independent design. This framework facilitates human-in-the-loop (HIL) model training and retraining, real-time stimulus control, and enables transfer learning and cloud computing for the online classification of electroencephalography (EEG) data. Stimuli for the subject and diagnostics for the researcher are shown on separate displays using web browser technologies. Messages are sent using the Lab Streaming Layer standard and websockets. Real-time signal processing and classification, as well as training of machine learning models, is facilitated by the open-source Python package Timeflux. The framework runs on Linux, MacOS, and Windows. While online analysis is the main target of the BCI-HIL framework, offline analysis of the EEG data can be performed with Python, MATLAB, and Julia through packages like MNE, EEGLAB, or FieldTrip. The paper describes and discusses desirable properties of a human-in-the-loop BCI research platform. The BCI-HIL framework is released under MIT license with examples at: bci.lu.se/bci-hil (or at: github.com/bci-hil/bci-hil).

High-Throughput Single-Molecule Sensors: How Can the Signals Be Analyzed in Real Time for Achieving Real-Time Continuous Biosensing?

Single-molecule sensors collect statistics of single-molecule interactions, and the resulting data can be used to determine concentrations of analyte molecules. The assays are generally end-point assays and are not designed for continuous biosensing. For continuous biosensing, a single-molecule sensor needs to be reversible, and the signals should be analyzed in real time in order to continuously report output signals, with a well-controlled time delay and measurement precision. Here, we describe a signal processing architecture for real-time continuous biosensing based on high-throughput single-molecule sensors. The key aspect of the architecture is the parallel computation of multiple measurement blocks that enables continuous measurements over an endless time span. Continuous biosensing is demonstrated for a single-molecule sensor with 10,000 individual particles that are tracked as a function of time. The continuous analysis includes particle identification, particle tracking, drift correction, and detection of the discrete timepoints where individual particles switch between bound and unbound states, yielding state transition statistics that relate to the analyte concentration in solution. The continuous real-time sensing and computation were studied for a reversible cortisol competitive immunosensor, showing how the precision and time delay of cortisol monitoring are controlled by the number of analyzed particles and the size of the measurement blocks. Finally, we discuss how the presented signal processing architecture can be applied to various single-molecule measurement methods, allowing these to be developed into continuous biosensors.

Canine Distemper Virus Infection in the Free-Living Wild Canines, the Red Fox (Vulpes vulpes) and Jackal (Canis aureus moreoticus), in Croatia

The canine distemper virus (CDV), a paramyxovirus that is closely related to the human measles virus and rinderpest virus of cattle, is a highly contagious viral disease in dogs and wild carnivores worldwide. CDV represents a serious threat to domestic and wild animals, especially to the conservation of endangered wild carnivores. Our study aims to investigate the occurrence of CDV in free-living wild canines in Croatia. For this purpose, 176 red foxes and 24 jackal brain samples collected in the frame of the active surveillance of rabies during winter 2021/2022 were tested. This study provided the first comprehensive overview of the prevalence and spatial distribution of CDV in the wildlife of Croatia, including the molecular phylogenetic analysis of the H gene sequence of field CDV strains circulating in red fox and jackal populations of Croatia. The molecular characterization of hemagglutinin gene genomic regions confirmed the phylogenetic clustering of obtained sequences into the Europa 1 genotype. The obtained CDV red fox sequences were mutually very similar (97.60%). This study indicates the high genetic similarity of Croatian CDV red fox sequences and CDV red fox sequences from Italy and Germany, badger sequences from Germany, polecat sequences from Hungary, and dog sequences from Hungary and Germany.

The Detection of Acute Risk of Self-injury Project: Protocol for an Ecological Momentary Assessment Study Among Individuals Seeking Treatment

BACKGROUND

Nonsuicidal self-injury (NSSI) is a major mental health concern. Despite increased research efforts on establishing the prevalence and correlates of the presence and severity of NSSI, we still lack basic knowledge of the course, predictors, and relationship of NSSI with other self-damaging behaviors in daily life. Such information will be helpful for better informing mental health professionals and allocating treatment resources. The DAILY (Detection of Acute rIsk of seLf-injurY) project will address these gaps among individuals seeking treatment.

OBJECTIVE

This protocol paper presents the DAILY project's aims, design, and materials used. The primary objectives are to advance understanding of (1) the short-term course and contexts of elevated risk for NSSI thoughts, urges, and behavior; (2) the transition from NSSI thoughts and urges to NSSI behavior; and (3) the association of NSSI with disordered eating, substance use, and suicidal thoughts and behaviors. A secondary aim is to evaluate the perspectives of individuals seeking treatment and mental health professionals regarding the feasibility, scope, and utility of digital self-monitoring and interventions that target NSSI in daily life.

METHODS

The DAILY project is funded by the Research Foundation Flanders (Belgium). Data collection involves 3 phases: a baseline assessment (phase 1), 28 days of ecological momentary assessment (EMA) followed by a clinical session and feedback survey (phase 2), and 2 follow-up surveys and an optional interview (phase 3). The EMA protocol consists of regular EMA surveys (6 times per day), additional burst EMA surveys spaced at a higher frequency when experiencing intense NSSI urges (3 surveys within 30 minutes), and event registrations of NSSI behavior. The primary outcomes are NSSI thoughts, NSSI urges, self-efficacy to resist NSSI, and NSSI behavior, with disordered eating (restrictive eating, binge eating, and purging), substance use (binge drinking and smoking cannabis), and suicidal thoughts and behaviors surveyed as secondary outcomes. The assessed predictors include emotions, cognitions, contextual information, and social appraisals.

RESULTS

We will recruit approximately 120 individuals seeking treatment aged 15 to 39 years from mental health services across the Flanders region of Belgium. Recruitment began in June 2021 and data collection is anticipated to conclude in August 2023.

CONCLUSIONS

The findings of the DAILY project will provide a detailed characterization of the short-term course and patterns of risk for NSSI and advance understanding of how, why, and when NSSI and other self-damaging behaviors unfold among individuals seeking treatment. This will inform clinical practice and provide the scientific building blocks for novel intervention approaches outside of the therapy room that support people who self-injure in real time.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/46244.

High-Level Synthesis Design of Scalable Ultrafast Ultrasound Beamformer With Single FPGA

Ultrafast ultrasound imaging is essential for advanced ultrasound imaging techniques such as ultrasound localization microscopy (ULM) and functional ultrasound (fUS). Current ultrafast ultrasound imaging is challenged by the ultrahigh data bandwidth associated with the radio frequency (RF) signal, and by the latency of the computationally expensive beamforming process. As such, continuous ultrafast data acquisition and beamforming remain elusive with existing software beamformers based on CPUs or GPUs. To address these challenges, the proposed work introduces a novel method of implementing an ultrafast ultrasound beamformer specifically for ultrafast plane wave imaging (PWI) on a field programmable gate array (FPGA) by using high-level synthesis. A parallelized implementation of the beamformer on a single FPGA was proposed by 1) utilizing a delay compression technique to reduce the delay profile size, which enables both run-time pre-calculated delay profile loading from external memory and delay reuse, 2) vectorizing channel data fetching which is enabled by delay reuse, and 3) using fixed summing networks to reduce consumption of logic resources. Our proposed method presents two unique advantages over current FPGA beamformers: 1) high scalability that allows fast adaptation to different FPGA resources and beamforming speed demands by using Xilinx High-Level Synthesis as the development tool, and 2) allow a compact form factor design by using a single FPGA to complete the beamforming instead of multiple FPGAs. Current Xilinx FPGAs provide the capabilities of connecting up to 1024 ultrasound channels with a single FPGA and the newest JESD204B interface analog front end (AFE). This channel count is much more than the channel count needed by current linear arrays, which normally have 128 or 256 channels. With the proposed method, a sustainable average beamforming rate of 4.83 G samples/second in terms of input raw RF sample was achieved. The resulting image quality of the proposed beamformer was compared with the software beamformer on the Verasonics Vantage system for both phantom imaging and in vivo imaging of a mouse brain. Multiple imaging schemes including B-mode, power Doppler and ULM were assessed to verify that the image quality was not compromised for speed.

ONLINE BINAURAL SPEECH SEPARATION OF MOVING SPEAKERS WITH A WAVESPLIT NETWORK

Binaural speech separation in real-world scenarios often involves moving speakers. Most current speech separation methods use utterance-level permutation invariant training (u-PIT) for training. In inference time, however, the order of outputs can be inconsistent over time particularly in long-form speech separation. This situation which is referred to as the speaker swap problem is even more problematic when speakers constantly move in space and therefore poses a challenge for consistent placement of speakers in output channels. Here, we describe a real-time binaural speech separation model based on a Wavesplit network to mitigate the speaker swap problem for moving speaker separation. Our model computes a speaker embedding for each speaker at each time frame from the mixed audio, aggregates embeddings using online clustering, and uses cluster centroids as speaker profiles to track each speaker throughout the long duration. Experimental results on reverberant, long-form moving multitalker speech separation show that the proposed method is less prone to speaker swap and achieves comparable performance with u-PIT based models with ground truth tracking in both separation accuracy and preserving the interaural cues.

IoT System for Real-Time Posture Asymmetry Detection

The rise of the Internet of Things (IoT) has enabled the development of measurement systems dedicated to preventing health issues and monitoring conditions in smart homes and workplaces. IoT systems can support monitoring people doing computer-based work and avoid the insurgence of common musculoskeletal disorders related to the persistence of incorrect sitting postures during work hours. This work proposes a low-cost IoT measurement system for monitoring the sitting posture symmetry and generating a visual alert to warn the worker when an asymmetric position is detected. The system employs four force sensing resistors (FSR) embedded in a cushion and a microcontroller-based read-out circuit for monitoring the pressure exerted on the chair seat. Java-based software performs the real-time monitoring of the sensors' measurements and implements an uncertainty-driven asymmetry detection algorithm. The shifts from a symmetric to an asymmetric posture and vice versa generate and close a pop-up warning message, respectively. In this way, the user is promptly notified when an asymmetric posture is detected and invited to adjust the sitting position. Every position shift is recorded in a web database for further analysis of the sitting behavior.

A fast blind source separation algorithm for decomposing ultrafast ultrasound images into spatiotemporal muscle unit kinematics

OBJECTIVE

Ultrasound can detect individual motor unit (MU) activity during voluntary isometric contractions based on their subtle axial displacements. The detection pipeline, currently performed offline, is based on displacement velocity images and identifying the subtle axial displacements. This identification can preferably be made through a blind source separation (BSS) algorithm with the feasibility of translating the pipeline from offline to online. However, the question remains how to reduce the computational time for the BSS algorithm, which includes demixing tissue velocities from many different sources, e.g., the active MU displacements, arterial pulsations, bones, connective tissue, and noise.&#xD;&#xD;Approach: This study proposes a fast velocity-based BSS (velBSS) algorithm suitable for online purposes that decomposes velocity images from low-force voluntary isometric contractions into spatiotemporal components associated with single MU activities. The proposed algorithm will be compared against spatiotemporal independent component analysis (stICA), i.e., the method used in previous papers, for various subjects, ultrasound- and EMG systems, where the latter acts as MU reference recordings.&#xD;&#xD;Main results: We found that the computational time for velBSS was at least 20 times less than for stICA, while the twitch responses and spatial maps extracted from stICA and velBSS for the same MU reference were highly correlated (0.96 ± 0.05 and 0.81 ± 0.13).&#xD;&#xD;Significance: The present algorithm (velBSS) is computationally much faster than the currently available method (stICA) while maintaining the same performance. It provides a promising translation towards an online pipeline and will be important in the continued development of this research field of functional neuromuscular imaging.

Efficient Detection and Tracking of Human Using 3D LiDAR Sensor

Light Detection and Ranging (LiDAR) technology is now becoming the main tool in many applications such as autonomous driving and human-robot collaboration. Point-cloud-based 3D object detection is becoming popular and widely accepted in the industry and everyday life due to its effectiveness for cameras in challenging environments. In this paper, we present a modular approach to detect, track and classify persons using a 3D LiDAR sensor. It combines multiple principles: a robust implementation for object segmentation, a classifier with local geometric descriptors, and a tracking solution. Moreover, we achieve a real-time solution in a low-performance machine by reducing the number of points to be processed by obtaining and predicting regions of interest via movement detection and motion prediction without any previous knowledge of the environment. Furthermore, our prototype is able to successfully detect and track persons consistently even in challenging cases due to limitations on the sensor field of view or extreme pose changes such as crouching, jumping, and stretching. Lastly, the proposed solution is tested and evaluated in multiple real 3D LiDAR sensor recordings taken in an indoor environment. The results show great potential, with particularly high confidence in positive classifications of the human body as compared to state-of-the-art approaches.

Mitochondria-Targeted Fluorescent Nanoparticles with Large Stokes Shift for Long-Term BioImaging

Mitochondria (MITO) play a significant role in various physiological processes and are a key organelle associated with different human diseases including cancer, diabetes mellitus, atherosclerosis, Alzheimer's disease, etc. Thus, detecting the activity of MITO in real time is becoming more and more important. Herein, a novel class of amphiphilic aggregation-induced emission (AIE) active probe fluorescence (AC-QC nanoparticles) based on a quinoxalinone scaffold was developed for imaging MITO. AC-QC nanoparticles possess an excellent ability to monitor MITO in real-time. This probe demonstrated the following advantages: (1) lower cytotoxicity; (2) superior photostability; and (3) good performance in long-term imaging in vitro. Each result of these indicates that self-assembled AC-QC nanoparticles can be used as effective and promising MITO-targeted fluorescent probes.

A Low-Cost Hardware/Software Platform for Lossless Real-Time Data Acquisition from Imaging Spectrometers

In real-time data-intensive applications, achieving real-time data acquisition from sensors and simultaneous storage with the necessary performance is challenging, especially if "no-data-lost" requirements are present. Ad hoc solutions are generally expensive and suffer from a lack of modularity and scalability. In this work, we present a hardware/software platform built using commercial off-the-shelf elements, designed to acquire and store digitized signals captured from imaging spectrometers capable of supporting real-time data acquisition with stringent throughput requirements (sustained rates in the boundaries of 100 MBytes/s) and simultaneous information storage in a lossless fashion. The correct combination of commercial hardware components with a properly configured and optimized multithreaded software application has satisfied the requirements in determinism and capacity for processing and storing large amounts of information in real time, keeping the economic cost of the system low. This real-time data acquisition and storage system has been tested in different conditions and scenarios, being able to successfully capture 100,000 1 Mpx-sized images generated at a nominal speed of 23.5 MHz (input throughput of 94 Mbytes/s, 4 bytes acquired per pixel) and store the corresponding data (300 GBytes of data, 3 bytes stored per pixel) concurrently without any single byte of information lost or altered. The results indicate that, in terms of throughput and storage capacity, the proposed system delivers similar performance to data acquisition systems based on specialized hardware, but at a lower cost, and provides more flexibility and adaptation to changing requirements.

FPGA-Based In-Vivo Calcium Image Decoding for Closed-Loop Feedback Applications

Miniaturized calcium imaging is an emerging neural recording technique that has been widely used for monitoring neural activity on a large scale at a specific brain region of rats or mice. Most existing calcium-image analysis pipelines operate offline. This results in long processing latency, making it difficult to realize closed-loop feedback stimulation for brain research. In recent work, we have proposed an FPGA-based real-time calcium image processing pipeline for closed-loop feedback applications. It can perform real-time calcium image motion correction, enhancement, fast trace extraction, and real-time decoding from extracted traces. Here, we extend this work by proposing a variety of neural network based methods for real-time decoding and evaluate the tradeoff among these decoding methods and accelerator designs. We introduce the implementation of the neural network based decoders on the FPGA, and show their speedup against the implementation on the ARM processor. Our FPGA implementation enables the real-time calcium image decoding with sub-ms processing latency for closed-loop feedback applications.

Faster SCDNet: Real-Time Semantic Segmentation Network with Split Connection and Flexible Dilated Convolution

Recently, semantic segmentation has been widely applied in various realistic scenarios. Many semantic segmentation backbone networks use various forms of dense connection to improve the efficiency of gradient propagation in the network. They achieve excellent segmentation accuracy but lack inference speed. Therefore, we propose a backbone network SCDNet with a dual path structure and higher speed and accuracy. Firstly, we propose a split connection structure, which is a streamlined lightweight backbone with a parallel structure to increase inference speed. Secondly, we introduce a flexible dilated convolution using different dilation rates so that the network can have richer receptive fields to perceive objects. Then, we propose a three-level hierarchical module to effectively balance the feature maps with multiple resolutions. Finally, a refined flexible and lightweight decoder is utilized. Our work achieves a trade-off of accuracy and speed on the Cityscapes and Camvid datasets. Specifically, we obtain a 36% improvement in FPS and a 0.7% improvement in mIoU on the Cityscapes test set.

A Call to Expand the Scope of Digital Phenotyping

Digital phenotyping refers to near-real-time data collection from personal digital devices, particularly smartphones, to better quantify the human phenotype. Methodology using smartphones is often considered the gold standard by many for passive data collection within the field of digital phenotyping, which limits its applications mainly to adults or adolescents who use smartphones. However, other technologies, such as wearable devices, have evolved considerably in recent years to provide similar or better quality passive physiologic data of clinical relevance, thus expanding the potential of digital phenotyping applications to other patient populations. In this perspective, we argue for the continued expansion of digital phenotyping to include other potential gold standards in addition to smartphones and provide examples of currently excluded technologies and populations who may uniquely benefit from this technology.

Real-Time 6-DOF Pose Estimation of Known Geometries in Point Cloud Data

The task of tracking the pose of an object with a known geometry from point cloud measurements arises in robot perception. It calls for a solution that is both accurate and robust, and can be computed at a rate that aligns with the needs of a control system that might make decisions based on it. The Iterative Closest Point (ICP) algorithm is widely used for this purpose, but it is susceptible to failure in practical scenarios. We present a robust and efficient solution for pose-from-point cloud estimation called the Pose Lookup Method (PLuM). PLuM is a probabilistic reward-based objective function that is resilient to measurement uncertainty and clutter. Efficiency is achieved through the use of lookup tables, which substitute complex geometric operations such as raycasting used in earlier solutions. Our results show millimetre accuracy and fast pose estimation in benchmark tests using triangulated geometry models, outperforming state-of-the-art ICP-based methods. These results are extended to field robotics applications, resulting in real-time haul truck pose estimation. By utilising point clouds from a LiDAR fixed to a rope shovel, the PLuM algorithm tracks a haul truck effectively throughout the excavation load cycle at a rate of 20 Hz, matching the sensor frame rate. PLuM is straightforward to implement and provides dependable and timely solutions in demanding environments.