Meta Clothing Status Calibration for Long-Term Person Re-Identification

Recent studies have seen significant advancements in the field of long-term person re-identification (LT-reID) through the use of clothing-irrelevant or insensitive features. This work takes the field a step further by addressing a previously unexplored issue, the Clothing Status Distribution Shift (CSDS). CSDS refers to the differing ratios of samples with clothing changes to those without clothing changes between the training and test sets, leading to a decline in LT-reID performance. We establish a connection between the performance of LT-reID and CSDS, and argue that addressing CSDS can improve LT-reID performance. To that end, we propose a novel framework called Meta Clothing Status Calibration (MCSC), which uses meta-learning to optimize the LT-reID model. Specifically, MCSC simulates CSDS between meta-train and meta-test with meta-optimization objectives, optimizing the LT-reID model and making it robust to CSDS. This framework is designed to prevent overfitting and improve the generalization ability of the LT-reID model in the presence of CSDS. Comprehensive evaluations on seven datasets demonstrate that the proposed MCSC framework effectively handles CSDS and improves current state-of-the-art LT-reID methods on several LT-reID benchmarks.

Characterising the motion and cardiorespiratory interaction of preterm infants can improve the classification of their sleep state

AIM

This study aimed to classify quiet sleep, active sleep and wake states in preterm infants by analysing cardiorespiratory signals obtained from routine patient monitors.

METHODS

We studied eight preterm infants, with an average postmenstrual age of 32.3 ± 2.4 weeks, in a neonatal intensive care unit in the Netherlands. Electrocardiography and chest impedance respiratory signals were recorded. After filtering and R-peak detection, cardiorespiratory features and motion and cardiorespiratory interaction features were extracted, based on previous research. An extremely randomised trees algorithm was used for classification and performance was evaluated using leave-one-patient-out cross-validation and Cohen's kappa coefficient.

RESULTS

A sleep expert annotated 4731 30-second epochs (39.4 h) and active sleep, quiet sleep and wake accounted for 73.3%, 12.6% and 14.1% respectively. Using all features, and the extremely randomised trees algorithm, the binary discrimination between active and quiet sleep was better than between other states. Incorporating motion and cardiorespiratory interaction features improved the classification of all sleep states (kappa 0.38 ± 0.09) than analyses without these features (kappa 0.31 ± 0.11).

CONCLUSION

Cardiorespiratory interactions contributed to detecting quiet sleep and motion features contributed to detecting wake states. This combination improved the automated classifications of sleep states.

Development and Validation of an Interpretable Conformal Predictor to Predict Sepsis Mortality Risk: Retrospective Cohort Study

BACKGROUND

Early and reliable identification of patients with sepsis who are at high risk of mortality is important to improve clinical outcomes. However, 3 major barriers to artificial intelligence (AI) models, including the lack of interpretability, the difficulty in generalizability, and the risk of automation bias, hinder the widespread adoption of AI models for use in clinical practice.

OBJECTIVE

This study aimed to develop and validate (internally and externally) a conformal predictor of sepsis mortality risk in patients who are critically ill, leveraging AI-assisted prediction modeling. The proposed approach enables explaining the model output and assessing its confidence level.

METHODS

We retrospectively extracted data on adult patients with sepsis from a database collected in a teaching hospital at Beth Israel Deaconess Medical Center for model training and internal validation. A large multicenter critical care database from the Philips eICU Research Institute was used for external validation. A total of 103 clinical features were extracted from the first day after admission. We developed an AI model using gradient-boosting machines to predict the mortality risk of sepsis and used Mondrian conformal prediction to estimate the prediction uncertainty. The Shapley additive explanation method was used to explain the model.

RESULTS

A total of 16,746 (80%) patients from Beth Israel Deaconess Medical Center were used to train the model. When tested on the internal validation population of 4187 (20%) patients, the model achieved an area under the receiver operating characteristic curve of 0.858 (95% CI 0.845-0.871), which was reduced to 0.800 (95% CI 0.789-0.811) when externally validated on 10,362 patients from the Philips eICU database. At a specified confidence level of 90% for the internal validation cohort the percentage of error predictions (n=438) out of all predictions (n=4187) was 10.5%, with 1229 (29.4%) predictions requiring clinician review. In contrast, the AI model without conformal prediction made 1449 (34.6%) errors. When externally validated, more predictions (n=4004, 38.6%) were flagged for clinician review due to interdatabase heterogeneity. Nevertheless, the model still produced significantly lower error rates compared to the point predictions by AI (n=1221, 11.8% vs n=4540, 43.8%). The most important predictors identified in this predictive model were Acute Physiology Score III, age, urine output, vasopressors, and pulmonary infection. Clinically relevant risk factors contributing to a single patient were also examined to show how the risk arose.

CONCLUSIONS

By combining model explanation and conformal prediction, AI-based systems can be better translated into medical practice for clinical decision-making.

Camera-based Seismocardiogram for Heart Rate Variability Monitoring

Heart rate variability (HRV) is a crucial metric that quantifies the variation between consecutive heartbeats, serving as a significant indicator of autonomic nervous system (ANS) activity. It has found widespread applications in clinical diagnosis, treatment, and prevention of cardiovascular diseases. In this study, we proposed an optical model for defocused speckle imaging, to simultaneously incorporate out-of-plane translation and rotation-induced motion for highly-sensitive non-contact seismocardiogram (SCG) measurement. Using electrocardiogram (ECG) signals as the gold standard, we evaluated the performance of photoplethysmogram (PPG) signals and speckle-based SCG signals in assessing HRV. The results indicated that the HRV parameters measured from SCG signals extracted from laser speckle videos showed higher consistency with the results obtained from the ECG signals compared to PPG signals. Additionally, we confirmed that even when clothing obstructed the measurement site, the efficacy of SCG signals extracted from the motion of laser speckle patterns persisted in assessing the HRV levels. This demonstrates the robustness of camera-based non-contact SCG in monitoring HRV, highlighting its potential as a reliable, non-contact alternative to traditional contact-PPG sensors.

Biphasic Face Photo-Sketch Synthesis via Semantic-Driven Generative Adversarial Network With Graph Representation Learning

Biphasic face photo-sketch synthesis has significant practical value in wide-ranging fields such as digital entertainment and law enforcement. Previous approaches directly generate the photo-sketch in a global view, they always suffer from the low quality of sketches and complex photograph variations, leading to unnatural and low-fidelity results. In this article, we propose a novel semantic-driven generative adversarial network to address the above issues, cooperating with graph representation learning. Considering that human faces have distinct spatial structures, we first inject class-wise semantic layouts into the generator to provide style-based spatial information for synthesized face photographs and sketches. In addition, to enhance the authenticity of details in generated faces, we construct two types of representational graphs via semantic parsing maps upon input faces, dubbed the intraclass semantic graph (IASG) and the interclass structure graph (IRSG). Specifically, the IASG effectively models the intraclass semantic correlations of each facial semantic component, thus producing realistic facial details. To preserve the generated faces being more structure-coordinated, the IRSG models interclass structural relations among every facial component by graph representation learning. To further enhance the perceptual quality of synthesized images, we present a biphasic interactive cycle training strategy by fully taking advantage of the multilevel feature consistency between the photograph and sketch. Extensive experiments demonstrate that our method outperforms the state-of-the-art competitors on the CUHK Face Sketch (CUFS) and CUHK Face Sketch FERET (CUFSF) datasets.

Living-skin Detection using Multi-layer Skin Property Perceived by the Structured Light

Living-skin detection has been used to prevent the attack of face fraud in a face recognition system. In this paper, we propose a new concept that exploits the multi-layer structure property of skin for living-skin detection. We observe a significant difference in the blur of the laser spot created by the structured light on the skin and non-skin due to the characteristic properties of laser photons in skin penetration and reflection. Based on this observation, we designed a new living-skin detection algorithm to differentiate skin and non-skin based on the blur detection of laser spots. The experimental results show that the proposed setup and method have a promising performance with an averaged precision of 96.7%, averaged recall of 82.2%, and averaged F1-score of 88.6% on a dataset of 20 adult subjects. This demonstrates the effectiveness of the new concept that uses multi-layer properties of skin tissues for living-skin detection, which may lead to new solutions for face anti-spoofing.

Graph Flow: Cross-Layer Graph Flow Distillation for Dual Efficient Medical Image Segmentation

With the development of deep convolutional neural networks, medical image segmentation has achieved a series of breakthroughs in recent years. However, high-performance convolutional neural networks always mean numerous parameters and high computation costs, which will hinder the applications in resource-limited medical scenarios. Meanwhile, the scarceness of large-scale annotated medical image datasets further impedes the application of high-performance networks. To tackle these problems, we propose Graph Flow, a comprehensive knowledge distillation framework, for both network-efficiency and annotation-efficiency medical image segmentation. Specifically, the Graph Flow Distillation transfers the essence of cross-layer variations from a well-trained cumbersome teacher network to a non-trained compact student network. In addition, an unsupervised Paraphraser Module is integrated to purify the knowledge of the teacher, which is also beneficial for the training stabilization. Furthermore, we build a unified distillation framework by integrating the adversarial distillation and the vanilla logits distillation, which can further refine the final predictions of the compact network. With different teacher networks (traditional convolutional architecture or prevalent transformer architecture) and student networks, we conduct extensive experiments on four medical image datasets with different modalities (Gastric Cancer, Synapse, BUSI, and CVC-ClinicDB). We demonstrate the prominent ability of our method on these datasets, which achieves competitive performances. Moreover, we demonstrate the effectiveness of our Graph Flow through a novel semi-supervised paradigm for dual efficient medical image segmentation. Our code will be available at Graph Flow.

Constructing Stronger and Faster Baselines for Skeleton-Based Action Recognition

One essential problem in skeleton-based action recognition is how to extract discriminative features over all skeleton joints. However, the complexity of the recent State-Of-The-Art (SOTA) models for this task tends to be exceedingly sophisticated and over-parameterized. The low efficiency in model training and inference has increased the validation costs of model architectures in large-scale datasets. To address the above issue, recent advanced separable convolutional layers are embedded into an early fused Multiple Input Branches (MIB) network, constructing an efficient Graph Convolutional Network (GCN) baseline for skeleton-based action recognition. In addition, based on such the baseline, we design a compound scaling strategy to expand the model's width and depth synchronously, and eventually obtain a family of efficient GCN baselines with high accuracies and small amounts of trainable parameters, termed EfficientGCN-Bx, where "x" denotes the scaling coefficient. On two large-scale datasets, i.e., NTU RGB+D 60 and 120, the proposed EfficientGCN-B4 baseline outperforms other SOTA methods, e.g., achieving 92.1% accuracy on the cross-subject benchmark of NTU 60 dataset, while being 5.82× smaller and 5.85× faster than MS-G3D, which is one of the SOTA methods. The source code in PyTorch version and the pretrained models are available at https://github.com/yfsong0709/EfficientGCNv1.

Medical instrument detection in ultrasound: a review

Medical instrument detection is essential for computer-assisted interventions, since it facilitates clinicians to find instruments efficiently with a better interpretation, thereby improving clinical outcomes. This article reviews image-based medical instrument detection methods for ultrasound-guided (US-guided) operations. Literature is selected based on an exhaustive search in different sources, including Google Scholar, PubMed, and Scopus. We first discuss the key clinical applications of medical instrument detection in the US, including delivering regional anesthesia, biopsy taking, prostate brachytherapy, and catheterization. Then, we present a comprehensive review of instrument detection methodologies, including non-machine-learning and machine-learning methods. The conventional non-machine-learning methods were extensively studied before the era of machine learning methods. The principal issues and potential research directions for future studies are summarized for the computer-assisted intervention community. In conclusion, although promising results have been obtained by the current (non-) machine learning methods for different clinical applications, thorough clinical validations are still required.

Prior Guided Transformer for Accurate Radiology Reports Generation

In this paper, we propose a prior guided transformer for accurate radiology reports generation. In the encoder part, a radiograph is firstly represented by a set of patch features, which is obtained through a convolutional neural network and a traditional transformer encoder. Then an Additive Gaussian model is applied to represent the prior knowledge based on unsupervised clustering and sparse attention. In the decoder part, prior embeddings are acquired by probabilistically sampling from the radiograph prior. Then the visual features, language embeddings, and prior embeddings are fused by our proposed Prior Guided Attention to generate accurate radiology reports. Experiment results show that our method achieves better performance than state-of-the-art methods on two public radiology datasets, which proves the effectiveness of our prior guided transformer.

Lithium chloride promotes neural functional recovery after local cerebral ischaemia injury in rats through Wnt signalling pathway activation

BACKGROUND

Lithium chloride (LiCl) has a significant neuroprotective effect in cerebral ischaemia. However, to date, there is a paucity of evidence on the role of LiCl in neural restoration after brain ischaemia and the signalling pathways involved remain unclear.

MATERIALS AND METHODS

Therefore, to address this gap, the middle cerebral artery occlusion (MCAO) rat model was used to simulate human ischaemia stroke. Male Sprague-Dawley rats were given MCAO for 90 min followed by reperfusion, and Dickkopf-1 (DKK1, 5.0 μg/kg) was administered half an hour before MCAO. Rats were then treated with hypodermic injection of LiCl (2.0 mmol/kg) twice a day for 1 week. After treatment, cognitive impairment was assessed by the Morris water maze test. Neurological deficit score, 2,3,5-triphenyl tetrazolium chloride staining, brain water content, and histopathology were used to evaluate brain damage. Enzyme-linked immunosorbent assay was used to measure oxidative stress damage and inflammatory cytokines. Apoptosis of the hippocampal neurons was tested by western blot. The key factors of Wnt signalling pathway in the ischaemic penumbra were detected by immunofluorescence staining and quantitative real-time polymerase chain reaction.

RESULTS

Current experimental results showed that LiCl treatment significantly improved the impaired spatial learning and memory ability, suppressed oxidative stress, inflammatory reaction, and neuron apoptosis accompanied by attenuating neuronal damage, which subsequently decreased the brain oedema, infarct volume and neurological deficit. Furthermore, the treatment of LiCl activated Wnt signalling pathway. Interestingly, the aforementioned effects of LiCl treatment were markedly reversed by administration of DKK1, an inhibitor of Wnt signalling pathway.

CONCLUSIONS

These results indicate that LiCl exhibits neuroprotective effects in focal cerebral ischaemia by Wnt signalling pathway activation, and it might have latent clinical application for the prevention and treatment of ischaemic stroke.

[A pedigree study of Loeys-Dietz syndrome type 4 with skeletal deformity related to a novel TGFβ2 mutation]

Objective: Loeys-Dietz syndrome is a rare type of hereditary connective tissue disease. This study was aimed to analyze the clinical characteristics and gene mutations in a family of Loeys-Dietz syndrome with skeletal deformity. Methods: Clinical data of the proband and family members were collected and biochemical measurements and radiological examinations were conducted. Genomic DNA was extracted from peripheral blood of the family members. Whole-exome sequencing was performed to determine the mutation sites in the proband, and Sanger sequencing was applied to verify the candidate mutation in the other family members. Results: The proband is a 34-year-old man with deformities of lower extremities for more than 30 years. Physical examinations showed dolichostenomelia, pes planus, joint laxity and scoliosis. Echocardiography revealed the dilatation of aortic root at the level of the sinuses of Valsalva. A heterozygous missense mutation (c. 220A>C, p.Thr74Pro) in exon 1 of TGFβ2 gene was identified in the proband. The same mutation was detected in his sister and niece with similar clinical features such as deformities of lower extremities and pes planus. This novel mutation has not been reported in ExAC or 1000G and was predicted to be deleterious, supporting a diagnosis of Loeys-Dietz syndrome type 4. Conclusions: Loeys-Dietz syndrome type 4 is caused by TGFβ2 mutations. Skeletal deformity is one of the distinctive features. Genetic testing is helpful for the early diagnosis and differential diagnosis from other connective tissue diseases.

Effect of Fufang Huangqi Decoction on the Gut Microbiota in Patients With Class I or II Myasthenia Gravis

Objective

To investigate the effect of Fufang Huangqi Decoction on the gut microbiota in patients with class I or II myasthenia gravis (MG) and to explore the correlation between gut microbiota and MG (registration number, ChiCTR2100048367; registration website, http://www.chictr.org.cn/listbycreater.aspx; NCBI: SRP338707).

Methods

In this study, microbial community composition and diversity analyses were carried out on fecal specimens from MG patients who did not take Fufang Huangqi Decoction (control group, n = 8) and those who took Fufang Huangqi Decoction and achieved remarkable alleviation of symptoms (medication group, n = 8). The abundance, diversity within and between habitats, taxonomic differences and corresponding discrimination markers of gut microbiota in the control group and medicated group were assessed.

Results

Compared with the control group, the medicated group showed a significantly decreased abundance of Bacteroidetes (P < 0.05) and significantly increased abundance of Actinobacteria at the phylum level, a significantly decreased abundance of Bacteroidaceae (P < 0.05) and significantly increased abundance of Bifidobacteriaceae at the family level and a significantly decreased abundance of Blautia and Bacteroides (P < 0.05) and significantly increased abundance of Bifidobacterium, Lactobacillus and Roseburia at the genus level. Compared to the control group, the medicated group had decreased abundance, diversity, and genetic diversity of the communities and increased coverage, but the differences were not significant (P > 0.05); the markers that differed significantly between communities at the genus level and influenced the differences between groups were Blautia, Bacteroides, Bifidobacterium and Lactobacillus.

Conclusions

MG patients have obvious gut microbiota-associated metabolic disorders. Fufang Huangqi Decoction regulates the gut microbiota in patients with class I or II MG by reducing the abundance of Blautia and Bacteroides and increasing the abundance of Bifidobacterium and Lactobacillus. The correlation between gut microbiota and MG may be related to cell-mediated immunity.

Progressive Residual Learning with Memory Upgrade for Ultrasound Image Blind Super-resolution

For clinical medical diagnosis and treatment, image super-resolution (SR) technology will be helpful to improve the ultrasonic imaging quality so as to enhance the accuracy of disease diagnosis. However, due to the differences of sensing devices or transmission media, the resolution degradation process of ultrasound imaging in real scenes is uncontrollable, especially when the blur kernel is usually unknown. This issue makes current endto- end SR networks poor performance when applied to ultrasonic images. Aiming to achieve effective SR in real ultrasound medical scenes, in this work, we propose a blind deep SR method based on progressive residual learning and memory upgrade. Specifically, we estimate the accurate blur kernel from the spatial attention map block of low resolution (LR) ultrasound image through a multi-label classification network, then we construct three modules - up-sampling (US) module, residual learning (RL) model and memory upgrading (MU) model for ultrasound image blind SR. The US module is designed to upscale the input information and the up-sampled residual result will be used for SR reconstruction. The RL module is employed to approximate the original LR and continuously generate the updated residual and feed it to the next US module. The last MU module can store all progressively learned residuals, which offers increased interactions between the US and RL modules, augmenting the details recovery. Extensive experiments and evaluations on the benchmark CCA-US and US-CASE datasets demonstrate the proposed approach achieves better performance against the state-ofthe- art methods.

Meta-USR: A Unified Super-Resolution Network for Multiple Degradation Parameters

Recent research on single image super-resolution (SISR) has achieved great success due to the development of deep convolutional neural networks. However, most existing SISR methods merely focus on super-resolution of a single fixed integer scale factor. This simplified assumption does not meet the complex conditions for real-world images which often suffer from various blur kernels or various levels of noise. More importantly, previous methods lack the ability to cope with arbitrary degradation parameters (scale factors, blur kernels, and noise levels) with a single model. A few methods can handle multiple degradation factors, e.g., noninteger scale factors, blurring, and noise, simultaneously within a single SISR model. In this work, we propose a simple yet powerful method termed meta-USR which is the first unified super-resolution network for arbitrary degradation parameters with meta-learning. In Meta-USR, a meta-restoration module (MRM) is proposed to enhance the traditional upscale module with the capability to adaptively predict the weights of the convolution filters for various combinations of degradation parameters. Thus, the MRM can not only upscale the feature maps with arbitrary scale factors but also restore the SR image with different blur kernels and noise levels. Moreover, the lightweight MRM can be placed at the end of the network, which makes it very efficient for iteratively/repeatedly searching the various degradation factors. We evaluate the proposed method through extensive experiments on several widely used benchmark data sets on SISR. The qualitative and quantitative experimental results show the superiority of our Meta-USR.

Medical Instrument Segmentation in 3D US by Hybrid Constrained Semi-Supervised Learning

Medical instrument segmentation in 3D ultrasound is essential for image-guided intervention. However, to train a successful deep neural network for instrument segmentation, a large number of labeled images are required, which is expensive and time-consuming to obtain. In this article, we propose a semi-supervised learning (SSL) framework for instrument segmentation in 3D US, which requires much less annotation effort than the existing methods. To achieve the SSL learning, a Dual-UNet is proposed to segment the instrument. The Dual-UNet leverages unlabeled data using a novel hybrid loss function, consisting of uncertainty and contextual constraints. Specifically, the uncertainty constraints leverage the uncertainty estimation of the predictions of the UNet, and therefore improve the unlabeled information for SSL training. In addition, contextual constraints exploit the contextual information of the training images, which are used as the complementary information for voxel-wise uncertainty estimation. Extensive experiments on multiple ex-vivo and in-vivo datasets show that our proposed method achieves Dice score of about 68.6%-69.1% and the inference time of about 1 sec. per volume. These results are better than the state-of-the-art SSL methods and the inference time is comparable to the supervised approaches.

CANet: Context Aware Network for Brain Glioma Segmentation

Automated segmentation of brain glioma plays an active role in diagnosis decision, progression monitoring and surgery planning. Based on deep neural networks, previous studies have shown promising technologies for brain glioma segmentation. However, these approaches lack powerful strategies to incorporate contextual information of tumor cells and their surrounding, which has been proven as a fundamental cue to deal with local ambiguity. In this work, we propose a novel approach named Context-Aware Network (CANet) for brain glioma segmentation. CANet captures high dimensional and discriminative features with contexts from both the convolutional space and feature interaction graphs. We further propose context guided attentive conditional random fields which can selectively aggregate features. We evaluate our method using publicly accessible brain glioma segmentation datasets BRATS2017, BRATS2018 and BRATS2019. The experimental results show that the proposed algorithm has better or competitive performance against several State-of-The-Art approaches under different segmentation metrics on the training and validation sets.

Tongue Tumor Detection in Hyperspectral Images Using Deep Learning Semantic Segmentation

OBJECTIVE

The utilization of hyperspectral imaging (HSI) in real-time tumor segmentation during a surgery have recently received much attention, but it remains a very challenging task.

METHODS

In this work, we propose semantic segmentation methods, and compare them with other relevant deep learning algorithms for tongue tumor segmentation. To the best of our knowledge, this is the first work using deep learning semantic segmentation for tumor detection in HSI data using channel selection, and accounting for more spatial tissue context, and global comparison between the prediction map, and the annotation per sample. Results, and Conclusion: On a clinical data set with tongue squamous cell carcinoma, our best method obtains very strong results of average dice coefficient, and area under the ROC-curve of [Formula: see text], and [Formula: see text], respectively on the original spatial image size. The results show that a very good performance can be achieved even with a limited amount of data. We demonstrate that important information regarding tumor decision is encoded in various channels, but some channel selection, and filtering is beneficial over the full spectra. Moreover, we use both visual (VIS), and near-infrared (NIR) spectrum, rather than commonly used only VIS spectrum; although VIS spectrum is generally of higher significance, we demonstrate NIR spectrum is crucial for tumor capturing in some cases.

SIGNIFICANCE

The HSI technology augmented with accurate deep learning algorithms has a huge potential to be a promising alternative to digital pathology or a doctors' supportive tool in real-time surgeries.

Efficient Medical Instrument Detection in 3D Volumetric Ultrasound Data

Ultrasound-guided procedures have been applied in many clinical therapies, such as cardiac catheterization and regional anesthesia. Medical instrument detection in 3D Ultrasound (US) is highly desired, but the existing approaches are far from real-time performance. Our objective is to investigate an efficient instrument detection method in 3D US for practical clinical use. We propose a novel Multi-dimensional Mixed Network for efficient instrument detection in 3D US, which extracts the discriminating features at 3D full-image level by a 3D encoder, and then applies a specially designed dimension reduction block to reduce the spatial complexity of the feature maps by projecting from 3D space into 2D space. A 2D decoder is adopted to detect the instrument along the specified axes. By projecting the predicted 2D outputs, the instrument is detected or visualized in the 3D volume. Furthermore, to enable the network to better learn the discriminative information, we propose a multi-level loss function to capture both pixel- and image-level differences. We carried out extensive experiments on two datasets for two tasks: (1) catheter detection for cardiac RF-ablation and (2) needle detection for regional anesthesia. Our experiments show that our proposed method achieves a detection error of 2-3 voxels with an efficiency of about 0.12 sec per 3D US volume. The proposed method is 3-8 times faster than the state-of-the-art methods, leading to real-time performance. The results show that our proposed method has significant clinical value for real-time 3D US-guided intervention.

Multi-view 3D skin feature recognition and localization for patient tracking in spinal surgery applications

BACKGROUND

Minimally invasive spine surgery is dependent on accurate navigation. Computer-assisted navigation is increasingly used in minimally invasive surgery (MIS), but current solutions require the use of reference markers in the surgical field for both patient and instruments tracking.

PURPOSE

To improve reliability and facilitate clinical workflow, this study proposes a new marker-free tracking framework based on skin feature recognition.

METHODS

Maximally Stable Extremal Regions (MSER) and Speeded Up Robust Feature (SURF) algorithms are applied for skin feature detection. The proposed tracking framework is based on a multi-camera setup for obtaining multi-view acquisitions of the surgical area. Features can then be accurately detected using MSER and SURF and afterward localized by triangulation. The triangulation error is used for assessing the localization quality in 3D.

RESULTS

The framework was tested on a cadaver dataset and in eight clinical cases. The detected features for the entire patient datasets were found to have an overall triangulation error of 0.207 mm for MSER and 0.204 mm for SURF. The localization accuracy was compared to a system with conventional markers, serving as a ground truth. An average accuracy of 0.627 and 0.622 mm was achieved for MSER and SURF, respectively.

CONCLUSIONS

This study demonstrates that skin feature localization for patient tracking in a surgical setting is feasible. The technology shows promising results in terms of detected features and localization accuracy. In the future, the framework may be further improved by exploiting extended feature processing using modern optical imaging techniques for clinical applications where patient tracking is crucial.

Hyperspectral Imaging for Glioblastoma Surgery: Improving Tumor Identification Using a Deep Spectral-Spatial Approach

The primary treatment for malignant brain tumors is surgical resection. While gross total resection improves the prognosis, a supratotal resection may result in neurological deficits. On the other hand, accurate intraoperative identification of the tumor boundaries may be very difficult, resulting in subtotal resections. Histological examination of biopsies can be used repeatedly to help achieve gross total resection but this is not practically feasible due to the turn-around time of the tissue analysis. Therefore, intraoperative techniques to recognize tissue types are investigated to expedite the clinical workflow for tumor resection and improve outcome by aiding in the identification and removal of the malignant lesion. Hyperspectral imaging (HSI) is an optical imaging technique with the power of extracting additional information from the imaged tissue. Because HSI images cannot be visually assessed by human observers, we instead exploit artificial intelligence techniques and leverage a Convolutional Neural Network (CNN) to investigate the potential of HSI in twelve in vivo specimens. The proposed framework consists of a 3D-2D hybrid CNN-based approach to create a joint extraction of spectral and spatial information from hyperspectral images. A comparison study was conducted exploiting a 2D CNN, a 1D DNN and two conventional classification methods (SVM, and the SVM classifier combined with the 3D-2D hybrid CNN) to validate the proposed network. An overall accuracy of 80% was found when tumor, healthy tissue and blood vessels were classified, clearly outperforming the state-of-the-art approaches. These results can serve as a basis for brain tumor classification using HSI, and may open future avenues for image-guided neurosurgical applications.

Hyperspectral imaging for colon cancer classification in surgical specimens: towards optical biopsy during image-guided surgery

The main curative treatment for localized colon cancer is surgical resection. However when tumor residuals are left positive margins are found during the histological examinations and additional treatment is needed to inhibit recurrence. Hyperspectral imaging (HSI) can offer non-invasive surgical guidance with the potential of optimizing the surgical effectiveness. In this paper we investigate the capability of HSI for automated colon cancer detection in six ex-vivo specimens employing a spectral-spatial patch-based classification approach. The results demonstrate the feasibility in assessing the benign and malignant boundaries of the lesion with a sensitivity of 0.88 and specificity of 0.78. The results are compared with the state-of-the-art deep learning based approaches. The method with a new hybrid CNN outperforms the state-of the-art approaches (0.74 vs. 0.82 AUC). This study paves the way for further investigation towards improving surgical outcomes with HSI.

Towards Optical Imaging for Spine Tracking without Markers in Navigated Spine Surgery

Surgical navigation systems are increasingly used for complex spine procedures to avoid neurovascular injuries and minimize the risk for reoperations. Accurate patient tracking is one of the prerequisites for optimal motion compensation and navigation. Most current optical tracking systems use dynamic reference frames (DRFs) attached to the spine, for patient movement tracking. However, the spine itself is subject to intrinsic movements which can impact the accuracy of the navigation system. In this study, we aimed to detect the actual patient spine features in different image views captured by optical cameras, in an augmented reality surgical navigation (ARSN) system. Using optical images from open spinal surgery cases, acquired by two gray-scale cameras, spinal landmarks were identified and matched in different camera views. A computer vision framework was created for preprocessing of the spine images, detecting and matching local invariant image regions. We compared four feature detection algorithms, Speeded Up Robust Feature (SURF), Maximal Stable Extremal Region (MSER), Features from Accelerated Segment Test (FAST), and Oriented FAST and Rotated BRIEF (ORB) to elucidate the best approach. The framework was validated in 23 patients and the 3D triangulation error of the matched features was < 0 . 5 mm. Thus, the findings indicate that spine feature detection can be used for accurate tracking in navigated surgery.

Automatic and Continuous Discomfort Detection for Premature Infants in a NICU Using Video-Based Motion Analysis

Frequent pain and discomfort in premature infants can lead to long-term adverse neurodevelopmental outcomes. Video-based monitoring is considered to be a promising contactless method for identification of discomfort moments. In this study, we propose a video-based method for automated detection of infant discomfort. The method is based on analyzing facial and body motion. Therefore, motion trajectories are estimated from frame to frame using optical flow. For each video segment, we further calculate the motion acceleration rate and extract 18 time- and frequency-domain features characterizing motion patterns. A support vector machine (SVM) classifier is then applied to video sequences to recognize infant status of comfort or discomfort. The method is evaluated using 183 video segments for 11 infants from 17 heel prick events. Experimental results show an AUC of 0.94 for discomfort detection and the average accuracy of 0.86 when combining all proposed features, which is promising for clinical use.

Impact of makeup on remote-PPG monitoring

Camera-based remote photoplethysmography (remote-PPG) enables contactless measurement of blood volume pulse from the human skin. Skin visibility is essential to remote-PPG as the camera needs to capture the light reflected from the skin that penetrates deep into skin tissues and carries blood pulsation information. The use of facial makeup may jeopardize this measurement by reducing the amount of light penetrating into and reflecting from the skin. In this paper, we conduct an empirical study to thoroughly investigate the impact of makeup on remote-PPG monitoring, in both the visible (RGB) and invisible (Near Infrared, NIR) lighting conditions. The experiment shows that makeup has negative influence on remote-PPG, which reduces the relative PPG strength (AC/DC) at different wavelengths and changes the normalized PPG signature across multiple wavelengths. It makes (i) the pulse-rate extraction more difficult in both the RGB and NIR, although NIR is less affected than RGB, and (ii) the blood oxygen saturation extraction in NIR impossible. To the best of our knowledge, this is the first work that systematically investigate the impact of makeup on camera-based remote-PPG monitoring.

Towards non-invasive patient tracking: optical image analysis for spine tracking during spinal surgery procedures

Surgical navigation systems can enhance surgeon vision and form a reliable image-guided tool for complex interventions as spinal surgery. The main prerequisite is successful patient tracking which implies optimal motion compensation. Nowadays, optical tracking systems can satisfy the need of detecting patient position during surgery, allowing navigation without the risk of damaging neurovascular structures. However, the spine is subject to vertebrae movements which can impact the accuracy of the system. The aim of this paper is to investigate the feasibility of a novel approach for offering a direct relationship to movements of the spinal vertebra during surgery. To this end, we detect and track patient spine features between different image views, captured by several optical cameras, for vertebrae rotation and displacement reconstruction. We analyze patient images acquired in a real surgical scenario by two gray-scale cameras, embedded in the flat-panel detector of the C-arm. Spine segmentation is performed and anatomical landmarks are designed and tracked between different views, while experimenting with several feature detection algorithms (e.g. SURF, MSER, etc.). The 3D positions for the matched features are reconstructed and the triangulation errors are computed for an accuracy assessment. The analysis of the triangulation accuracy reveals a mean error of 0.38 mm, which demonstrates the feasibility of spine tracking and strengthens the clinical application of optical imaging for spinal navigation.

RGB-T Salient Object Detection via Fusing Multi-level CNN Features

RGB-induced salient object detection has recently witnessed substantial progress, which is attributed to the superior feature learning capability of deep convolutional neural networks (CNNs). However, such detections suffer from challenging scenarios characterized by cluttered backgrounds, low-light conditions and variations in illumination. Instead of improving RGB based saliency detection, this paper takes advantage of the complementary benefits of RGB and thermal infrared images. Specifically, we propose a novel end-to-end network for multi-modal salient object detection, which turns the challenge of RGB-T saliency detection to a CNN feature fusion problem. To this end, a backbone network (e.g., VGG-16) is first adopted to extract the coarse features from each RGB or thermal infrared image individually, and then several adjacent-depth feature combination (ADFC) modules are designed to extract multi-level refined features for each single-modal input image, considering that features captured at different depths differ in semantic information and visual details. Subsequently, a multi-branch group fusion (MGF) module is employed to capture the cross-modal features by fusing those features from ADFC modules for a RGB-T image pair at each level. Finally, a joint attention guided bi-directional message passing (JABMP) module undertakes the task of saliency prediction via integrating the multi-level fused features from MGF modules. Experimental results on several public RGB-T salient object detection datasets demonstrate the superiorities of our proposed algorithm over the state-of-the-art approaches, especially under challenging conditions, such as poor illumination, complex background and low contrast.

Detecting discomfort in infants through facial expressions

OBJECTIVE

Detecting discomfort status of infants is particularly clinically relevant. Late treatment of discomfort infants can lead to adverse problems such as abnormal brain development, central nervous system damage and changes in responsiveness of the neuroendocrine and immune systems to stress at maturity. In this study, we exploit deep convolutional neural network (CNN) algorithms to address the problem of discomfort detection for infants by analyzing their facial expressions.

APPROACH

A dataset of 55 videos about facial expressions, recorded from 24 infants, is used in our study. Given the limited available data for training, we employ a pre-trained CNN model, which is followed by fine-tuning the networks using a public dataset with labeled facial expressions (the shoulder-pain dataset). The CNNs are further refined with our data of infants.

MAIN RESULTS

Using a two-fold cross-validation, we achieve an area under the curve (AUC) value of 0.96, which is substantially higher than the results without any pre-training steps (AUC  =  0.77). Our method also achieves better results than the existing method based on handcrafted features. By fusing individual frame results, the AUC is further improved from 0.96 to 0.98.

SIGNIFICANCE

The proposed system has great potential for continuous discomfort and pain monitoring in clinical practice.

Deep Unbiased Embedding Transfer for Zero-shot Learning

Zero-shot learning aims to recognize objects which do not appear in the training dataset. Previous prevalent mapping-based zero-shot learning methods suffer from the projection domain shift problem due to the lack of image classes in the training stage. In order to alleviate the projection domain shift problem, a deep unbiased embedding transfer (DUET) model is proposed in this paper. The DUET model is composed of a deep embedding transfer (DET) module and an unseen visual feature generation (UVG) module. In the DET module, a novel combined embedding transfer net which integrates the complementary merits of the linear and nonlinear embedding mapping functions is proposed to connect the visual space and semantic space. What's more, the end-to-end joint training process is implemented to train the visual feature extractor and the combined embedding transfer net simultaneously. In the UVG module, a visual feature generator trained with a conditional generative adversarial framework is used to synthesize the visual features of the unseen classes to ease the disturbance of the projection domain shift problem. Furthermore, a quantitative index, namely the score of resistance on domain shift (ScoreRDS), is proposed to evaluate different models regarding their resistance capability on the projection domain shift problem. The experiments on five zero-shot learning benchmarks verify the effectiveness of the proposed DUET model. As demonstrated by the qualitative and quantitative analysis, the unseen class visual feature generation, the combined embedding transfer net and the end-to-end joint training process all contribute to alleviating projection domain shift in zero-shot learning.

Deep Salient Object Detection with Contextual Information Guidance

Integration of multi-level contextual information, such as feature maps and side outputs, is crucial for Convolutional Neural Networks (CNNs) based salient object detection. However, most existing methods either simply concatenate multi-level feature maps or calculate element-wise addition of multi-level side outputs, thus failing to take full advantages of them. In this work, we propose a new strategy for guiding multi-level contextual information integration, where feature maps and side outputs across layers are fully engaged. Specifically, shallower-level feature maps are guided by the deeper-level side outputs to learn more accurate properties of the salient object. In turn, the deeper-level side outputs can be propagated to high-resolution versions with spatial details complemented by means of shallower-level feature maps. Moreover, a group convolution module is proposed with the aim to achieve high-discriminative feature maps, in which the backbone feature maps are divided into a number of groups and then the convolution is applied to the channels of backbone feature maps within each group. Eventually, the group convolution module is incorporated in the guidance module to further promote the guidance role. Experiments on three public benchmark datasets verify the effectiveness and superiority of the proposed method over the state-of-the-art methods.

Jasmonic acid-induced hydrogen sulfide activates MEK1/2 in regulating the redox state of ascorbate in Arabidopsis thaliana leaves

In this paper, we investigated the relationship between hydrogen sulfide (H₂S) and mitogen-activated protein kinase kinase (MEK1/2) in jasmonic acid (JA)-regulated the redox state of ascorbate in the leaves of Arabidopsis thaliana. The results showed that JA significantly enhanced the phosphorylation level of MEK1/2, the production of endogenous H₂S and the ratio of reduced ascorbate (AsA) to dehydroascorbate (DHA) (AsA/DHA) in wild type of A. thaliana (WT). However, there were no obvious effects of JA on above indicators in H₂S synthetic mutant of A. thaliana (MT). H₂S scavenger hypotaurine (HT) markedly reduced JA-induced the phosphorylation level of MEK1/2, AsA/DHA ratio and the production of endogenous H₂S in WT. Application of H₂S donor sodium hydrosulfide (NaHS) to JA-treated MT plants increased above indicators. Application of NaHS to (HT+JA)-treated MT plants did not reverse the effects of HT on above JA-induced indicators. MEK1/2 inhibitor PD98059 decreased JA-induced AsA/DHA ratio and the transcript levels and the activities of ascorbate peroxidase (APX), glutathione reductase (GR), monodehydroascorbate reductase (MDHAR), dehydroascorbate reductase (DHAR) and L-galactono-1,4-lactone dehydrogenase (GalLDH) in WT. However, PD98059 had no effect on JA-induced the production of endogenous H₂S in WT. Compared with Control-MT, there were no obvious effects of JA on the production of endogenous H₂S, AsA/DHA ratio and the transcript levels and activities of above enzymes in MT. However, application of PD98059 reduced above JA-induced indicators except the production of endogenous H₂S and DHA content in MT. Our results suggested that H₂S activated MEK1/2 in JA-regulated AsA/DHA ratio in A. thaliana leaves through enzymes in ascorbate metabolism.

Catheter localization in 3D ultrasound using voxel-of-interest-based ConvNets for cardiac intervention

PURPOSE

Efficient image-based catheter localization in 3D US during cardiac interventions is highly desired, since it facilitates the operation procedure, reduces the patient risk and improves the outcome. Current image-based catheter localization methods are not efficient or accurate enough for real clinical use.

METHODS

We propose a catheter localization method for 3D cardiac ultrasound (US). The catheter candidate voxels are first pre-selected by the Frangi vesselness filter with adaptive thresholding, after which a triplanar-based ConvNet is applied to classify the remaining voxels as catheter or not. We propose a Share-ConvNet for 3D US, which reduces the computation complexity by sharing a single ConvNet for all orthogonal slices. To boost the performance of ConvNet, we also employ two-stage training with weighted cross-entropy. Using the classified voxels, the catheter is localized by a model fitting algorithm.

RESULTS

To validate our method, we have collected challenging ex vivo datasets. Extensive experiments show that the proposed method outperforms state-of-the-art methods and can localize the catheter with an average error of 2.1 mm in around 10 s per volume.

CONCLUSION

Our method can automatically localize the cardiac catheter in challenging 3D cardiac US images. The efficiency and accuracy localization of the proposed method are considered promising for catheter detection and localization during clinical interventions.

Catheter segmentation in three-dimensional ultrasound images by feature fusion and model fitting

Ultrasound (US) has been increasingly used during interventions, such as cardiac catheterization. To accurately identify the catheter inside US images, extra training for physicians and sonographers is needed. As a consequence, automated segmentation of the catheter in US images and optimized presentation viewing to the physician can be beneficial to accelerate the efficiency and safety of interventions and improve their outcome. For cardiac catheterization, a three-dimensional (3-D) US image is potentially attractive because of no radiation modality and richer spatial information. However, due to a limited spatial resolution of 3-D cardiac US and complex anatomical structures inside the heart, image-based catheter segmentation is challenging. We propose a cardiac catheter segmentation method in 3-D US data through image processing techniques. Our method first applies a voxel-based classification through newly designed multiscale and multidefinition features, which provide a robust catheter voxel segmentation in 3-D US. Second, a modified catheter model fitting is applied to segment the curved catheter in 3-D US images. The proposed method is validated with extensive experiments, using different in-vitro, ex-vivo, and in-vivo datasets. The proposed method can segment the catheter within an average tip-point error that is smaller than the catheter diameter (1.9 mm) in the volumetric images. Based on automated catheter segmentation and combined with optimal viewing, physicians do not have to interpret US images and can focus on the procedure itself to improve the quality of cardiac intervention.

Skin resistance to UVB-induced oxidative stress and hyperpigmentation by the topical use of Lactobacillus helveticus NS8-fermented milk supernatant

AIMS

In this study, we investigated the preventive properties of the supernatant of Lactobacillus helveticus NS8-fermented milk (NS8-FS) against UV light-induced skin oxidative damage and hyperpigmentation.

METHODS AND RESULTS

NS8-FS exhibited significant radical scavenging activity in tests with ABST⁺ and DPPH scavenging methods, and as well strongly inhibited 3-morpholinosydnonimine (Sin-1)-induced ROS generation in HaCat keratinocytes. Unexpectedly, NS8-FS was found to inhibit melanin production in B16F10 melanoma cells and to exhibit inhibitory effects both to the enzymatic activity of tyrosinase (TYR) and the expression of proteins required for melanin synthesis. In SKH-1 hairless mice, topical application of NS8-FS alleviated UVB-induced skin photodamage, including the improvement of the appearance of epidermal thickness, transepidennal water loss and lipid peroxidation levels. In the tanning guinea pig model, the whitening effect of NS8-FS was demonstrated using Masson-Fontana staining and TYR staining. Furthermore, NS8-FS was shown to stimulate the nuclear translocation and activation of the Nrf2 protein, along with recovery of antioxidant enzyme activities.

CONCLUSION

NS8-FS exhibits the protective capacities against UV light-induced skin oxidative damage and hyperpigmentation.

SIGNIFICANCE AND IMPACT OF THE STUDY

Our findings indicate the potential of cell-free fermented products of lactic acid bacteria in topical photoprotection.

[Ventilator-associated pneumonia among premature infants <34 weeks' gestational age in neonatal intensive care unit in China: a multicenter study]

Objective: To investigate the incidence and pathogen distribution of ventilator-associated pneumonia (VAP) among preterm infants admitted to level Ⅲ neonatal intensive care units (NICU) in China. Method: A prospective study was conducted in 25 level Ⅲ NICU, enrolling all preterm infants <34 weeks gestational age admitted to the participating NICU within the first 7 days of life from May 2015 to April 2016. Chi-square test, t test and Mann-Whitney U test were used for statistical analysis. Result: A total of 7 918 patients were enrolled, within whom 4 623(58.4%) were males. The birth weight was (1 639±415) g and the gestational age was (31.4±2.0) weeks; 4 654(58.8%) infants required non-invasive mechanical ventilation and 2 154(27.2%) required intubation. Of all the mechanically ventilated patients, VAP occurred in 95 patients. The overall VAP rate was 7.0 episodes per 1 000 ventilator days, varying from 0 to 34.4 episodes per 1 000 ventilator days in different centers. The incidence of VAP was 9.6 and 6.0 per 1 000 ventilator days in children's hospitals and maternity-infant hospitals respectively, without significant differences (t=1.002, P=0.327). Gram-negative bacilli (76 strains, 91.6%) were the primary VAP microorganisms, mainly Acinetobacter baumannii (24 strains, 28.9%), Klebsiella pneumonia (23 strains, 27.7%), and Pseudomonas aeruginosa (10 strains, 12.0%). Conclusion: The incidence of VAP in China is similar to that in developed counties, with substantial variability in different NICU settings. More efforts are needed to monitor and evaluate the preventable factors associated with VAP and conduct interventions that could effectively reduce the occurrence of VAP.

Targeting 6-phosphogluconate dehydrogenase in the oxidative PPP sensitizes leukemia cells to antimalarial agent dihydroartemisinin

The oxidative pentose phosphate pathway (PPP) is crucial for cancer cell metabolism and tumor growth. We recently reported that targeting a key oxidative PPP enzyme, 6-phosphogluconate dehydrogenase (6PGD), using our novel small molecule 6PGD inhibitors Physcion and its derivative S3, shows anti-cancer effects. Notably, humans with genetic deficiency of either 6PGD or another oxidative PPP enzyme, glucose-6-phosphate dehydrogenase (G6PD), exhibit non-immune hemolytic anemia upon exposure to aspirin and various anti-malarial drugs. Inspired by these clinical observations, we examined the anti-cancer potential of combined treatment with 6PGD inhibitors and anti-malarial drugs. We found that stable knockdown of 6PGD sensitizes leukemia cells to anti-malarial agent dihydroartemisinin (DHA). Combined treatment with DHA and Physcion activates AMP-activated protein kinase, leading to synergistic inhibition of human leukemia cell viability. Moreover, our combined therapy synergistically attenuates tumor growth in xenograft nude mice injected with human K562 leukemia cells and cell viability of primary leukemia cells from human patients, but shows minimal toxicity to normal hematopoietic cells in mice as well as red blood cells and mononucleocytes from healthy human donors. Our findings reveal the potential for combined therapy using optimized doses of Physcion and DHA as a novel anti-leukemia treatment without inducing hemolysis.

Topical combined application of dexamethasone, vitamin C, and β-sodium glycerophosphate for healing the extraction socket in rabbits

An osteogenic inducer (OI) consisting of dexamethasone, vitamin C, and β-sodium glycerophosphate has the capacity to induce bone formation in vitro. The aim of this study was to assess the efficacy of the application of this OI on extraction socket healing. The bilateral first mandibular premolars were extracted from 75 New Zealand rabbits. Gelatin sponges carrying OI were implanted into the sockets. Sockets undergoing implantation of gelatin sponges alone were also evaluated, as well as non-implantation sockets. Specimens from each group were evaluated radiographically, histologically, and histomorphometrically using haematoxylin-eosin staining. Results showed earlier new bone formation and higher bone quality and quantity in the OI group compared to the other groups, and the differences were significant at 2, 4, 8, and 12 weeks postoperative. The OI significantly reduced the absorption of alveolar bone in terms of height; however, changes in the width were not significantly different between the three groups (P>0.05). The OI was shown to have a positive effect on healing of the tooth extraction sockets, was inexpensive, and was convenient to use during the operational procedure; therefore this could represent a promising implant material for human clinical application.

Ultrasound coefficient of nonlinearity imaging

Imaging the acoustical coefficient of nonlinearity, β, is of interest in several healthcare interventional applications. It is an important feature that can be used for discriminating tissues. In this paper, we propose a nonlinearity characterization method with the goal of locally estimating the coefficient of nonlinearity. The proposed method is based on a 1-D solution of the nonlinear lossy Westerfelt equation, thereby deriving a local relation between β and the pressure wave field. Based on several assumptions, a β imaging method is then presented that is based on the ratio between the harmonic and fundamental fields, thereby reducing the effect of spatial amplitude variations of the speckle pattern. By testing the method on simulated ultrasound pressure fields and an in vitro B-mode ultrasound acquisition, we show that the designed algorithm is able to estimate the coefficient of nonlinearity, and that the tissue types of interest are well discriminable. The proposed imaging method provides a new approach to β estimation, not requiring a special measurement setup or transducer, that seems particularly promising for in vivo imaging.