Towards Contactless Patient Positioning

PIAA: Pre-imaging all-round assistant for digital radiography

BACKGROUND

In radiography procedures, radiographers' suboptimal positioning and exposure parameter settings may necessitate image retakes, subjecting patients to unnecessary ionizing radiation exposure. Reducing retakes is crucial to minimize patient X-ray exposure and conserve medical resources.

OBJECTIVE

We propose a Digital Radiography (DR) Pre-imaging All-round Assistant (PIAA) that leverages Artificial Intelligence (AI) technology to enhance traditional DR.

METHODS

PIAA consists of an RGB-Depth (RGB-D) multi-camera array, an embedded computing platform, and multiple software components. It features an Adaptive RGB-D Image Acquisition (ARDIA) module that automatically selects the appropriate RGB camera based on the distance between the cameras and patients. It includes a 2.5D Selective Skeletal Keypoints Estimation (2.5D-SSKE) module that fuses depth information with 2D keypoints to estimate the pose of target body parts. Thirdly, it also uses a Domain expertise (DE) embedded Full-body Exposure Parameter Estimation (DFEPE) module that combines 2.5D-SSKE and DE to accurately estimate parameters for full-body DR views.

RESULTS

Optimizes DR workflow, significantly enhancing operational efficiency. The average time required for positioning patients and preparing exposure parameters was reduced from 73 seconds to 8 seconds.

CONCLUSIONS

PIAA shows significant promise for extension to full-body examinations.

Anatomy-guided domain adaptation for 3D in-bed human pose estimation

3D human pose estimation is a key component of clinical monitoring systems. The clinical applicability of deep pose estimation models, however, is limited by their poor generalization under domain shifts along with their need for sufficient labeled training data. As a remedy, we present a novel domain adaptation method, adapting a model from a labeled source to a shifted unlabeled target domain. Our method comprises two complementary adaptation strategies based on prior knowledge about human anatomy. First, we guide the learning process in the target domain by constraining predictions to the space of anatomically plausible poses. To this end, we embed the prior knowledge into an anatomical loss function that penalizes asymmetric limb lengths, implausible bone lengths, and implausible joint angles. Second, we propose to filter pseudo labels for self-training according to their anatomical plausibility and incorporate the concept into the Mean Teacher paradigm. We unify both strategies in a point cloud-based framework applicable to unsupervised and source-free domain adaptation. Evaluation is performed for in-bed pose estimation under two adaptation scenarios, using the public SLP dataset and a newly created dataset. Our method consistently outperforms various state-of-the-art domain adaptation methods, surpasses the baseline model by 31%/66%, and reduces the domain gap by 65%/82%. Source code is available at https://github.com/multimodallearning/da-3dhpe-anatomy.

In-Home Older Adults' Activity Pattern Monitoring Using Depth Sensors: A Review

The global population is aging due to many factors, including longer life expectancy through better healthcare, changing diet, physical activity, etc. We are also witnessing various frequent epidemics as well as pandemics. The existing healthcare system has failed to deliver the care and support needed to our older adults (seniors) during these frequent outbreaks. Sophisticated sensor-based in-home care systems may offer an effective solution to this global crisis. The monitoring system is the key component of any in-home care system. The evidence indicates that they are more useful when implemented in a non-intrusive manner through different visual and audio sensors. Artificial Intelligence (AI) and Computer Vision (CV) techniques may be ideal for this purpose. Since the RGB imagery-based CV technique may compromise privacy, people often hesitate to utilize in-home care systems which use this technology. Depth, thermal, and audio-based CV techniques could be meaningful substitutes here. Due to the need to monitor larger areas, this review article presents a systematic discussion on the state-of-the-art using depth sensors as primary data-capturing techniques. We mainly focused on fall detection and other health-related physical patterns. As gait parameters may help to detect these activities, we also considered depth sensor-based gait parameters separately. The article provides discussions on the topic in relation to the terminology, reviews, a survey of popular datasets, and future scopes.

Deep Learning and Medical Image Analysis for COVID-19 Diagnosis and Prediction

The coronavirus disease 2019 (COVID-19) pandemic has imposed dramatic challenges to health-care organizations worldwide. To combat the global crisis, the use of thoracic imaging has played a major role in diagnosis, prediction, and management for COVID-19 patients with moderate to severe symptoms or with evidence of worsening respiratory status. In response, the medical image analysis community acted quickly to develop and disseminate deep learning models and tools to meet the urgent need of managing and interpreting large amounts of COVID-19 imaging data. This review aims to not only summarize existing deep learning and medical image analysis methods but also offer in-depth discussions and recommendations for future investigations. We believe that the wide availability of high-quality, curated, and benchmarked COVID-19 imaging data sets offers the great promise of a transformative test bed to develop, validate, and disseminate novel deep learning methods in the frontiers of data science and artificial intelligence. Expected final online publication date for the Annual Review of Biomedical Engineering, Volume 24 is June 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Seeing under the cover with a 3D U-Net: point cloud-based weight estimation of covered patients

PURPOSE

Body weight is a crucial parameter for patient-specific treatments, particularly in the context of proper drug dosage. Contactless weight estimation from visual sensor data constitutes a promising approach to overcome challenges arising in emergency situations. Machine learning-based methods have recently been shown to perform accurate weight estimation from point cloud data. The proposed methods, however, are designed for controlled conditions in terms of visibility and position of the patient, which limits their practical applicability. In this work, we aim to decouple accurate weight estimation from such specific conditions by predicting the weight of covered patients from voxelized point cloud data.

METHODS

We propose a novel deep learning framework, which comprises two 3D CNN modules solving the given task in two separate steps. First, we train a 3D U-Net to virtually uncover the patient, i.e. to predict the patient's volumetric surface without a cover. Second, the patient's weight is predicted from this 3D volume by means of a 3D CNN architecture, which we optimized for weight regression.

RESULTS

We evaluate our approach on a lying pose dataset (SLP) under two different cover conditions. The proposed framework considerably improves on the baseline model by up to [Formula: see text] and reduces the gap between the accuracy of weight estimates for covered and uncovered patients by up to [Formula: see text].

CONCLUSION

We present a novel pipeline to estimate the weight of patients, which are covered by a blanket. Our approach relaxes the specific conditions that were required for accurate weight estimates by previous contactless methods and thus constitutes an important step towards fully automatic weight estimation in clinical practice.

FLANNEL (Focal Loss bAsed Neural Network EnsembLe) for COVID-19 detection

OBJECTIVE

The study sought to test the possibility of differentiating chest x-ray images of coronavirus disease 2019 (COVID-19) against other pneumonia and healthy patients using deep neural networks.

MATERIALS AND METHODS

We construct the radiography (x-ray) imaging data from 2 publicly available sources, which include 5508 chest x-ray images across 2874 patients with 4 classes: normal, bacterial pneumonia, non-COVID-19 viral pneumonia, and COVID-19. To identify COVID-19, we propose a FLANNEL (Focal Loss bAsed Neural Network EnsembLe) model, a flexible module to ensemble several convolutional neural network models and fuse with a focal loss for accurate COVID-19 detection on class imbalance data.

RESULTS

FLANNEL consistently outperforms baseline models on COVID-19 identification task in all metrics. Compared with the best baseline, FLANNEL shows a higher macro-F1 score, with 6% relative increase on the COVID-19 identification task, in which it achieves precision of 0.7833 ± 0.07, recall of 0.8609 ± 0.03, and F1 score of 0.8168 ± 0.03.

DISCUSSION

Ensemble learning that combines multiple independent basis classifiers can increase the robustness and accuracy. We propose a neural weighing module to learn the importance weight for each base model and combine them via weighted ensemble to get the final classification results. In order to handle the class imbalance challenge, we adapt focal loss to our multiple classification task as the loss function.

CONCLUSION

FLANNEL effectively combines state-of-the-art convolutional neural network classification models and tackles class imbalance with focal loss to achieve better performance on COVID-19 detection from x-rays.

Active contour regularized semi-supervised learning for COVID-19 CT infection segmentation with limited annotations

Infection segmentation on chest CT plays an important role in the quantitative analysis of COVID-19. Developing automatic segmentation tools in a short period with limited labelled images has become an urgent need. Pseudo label-based semi-supervised method is a promising way to leverage unlabelled data to improve segmentation performance. Existing methods usually obtain pseudo labels by first training a network with limited labelled images and then inferring unlabelled images. However, these methods may generate obviously inaccurate labels and degrade the subsequent training process. To address these challenges, in this paper, an active contour regularized semi-supervised learning framework was proposed to automatically segment infections with few labelled images. The active contour regularization was realized by the region-scalable fitting (RSF) model which is embedded to the loss function of the network to regularize and refine the pseudo labels of the unlabelled images. We further designed a splitting method to separately optimize the RSF regularization term and the segmentation loss term with iterative convolution-thresholding method and stochastic gradient descent, respectively, which enable fast optimization of each term. Furthermore, we built a statistical atlas to show the infection spatial distribution. Extensive experiments on a small public dataset and a large scale dataset showed that the proposed method outperforms state-of-the-art methods with up to 5% in dice similarity coefficient and normalized surface dice, 10% in relative absolute volume difference and 8 mm in 95% Hausdorff distance. Moreover, we observed that the infections tend to occur at the dorsal subpleural lung and posterior basal segments that are not mentioned in current radiology reports and are meaningful to advance our understanding of COVID-19.

Drr4covid: Learning Automated COVID-19 Infection Segmentation From Digitally Reconstructed Radiographs

Automated infection measurement and COVID-19 diagnosis based on Chest X-ray (CXR) imaging is important for faster examination, where infection segmentation is an essential step for assessment and quantification. However, due to the heterogeneity of X-ray imaging and the difficulty of annotating infected regions precisely, learning automated infection segmentation on CXRs remains a challenging task. We propose a novel approach, called DRR4Covid, to learn COVID-19 infection segmentation on CXRs from digitally reconstructed radiographs (DRRs). DRR4Covid consists of an infection-aware DRR generator, a segmentation network, and a domain adaptation module. Given a labeled Computed Tomography scan, the infection-aware DRR generator can produce infection-aware DRRs with pixel-level annotations of infected regions for training the segmentation network. The domain adaptation module is designed to enable the segmentation network trained on DRRs to generalize to CXRs. The statistical analyses made on experiment results have indicated that our infection-aware DRRs are significantly better than standard DRRs in learning COVID-19 infection segmentation (p < 0.05) and the domain adaptation module can improve the infection segmentation performance on CXRs significantly (p < 0.05). Without using any annotations of CXRs, our network has achieved a classification score of (Accuracy: 0.949, AUC: 0.987, F1-score: 0.947) and a segmentation score of (Accuracy: 0.956, AUC: 0.980, F1-score: 0.955) on a test set with 558 normal cases and 558 positive cases. Besides, by adjusting the strength of radiological signs of COVID-19 infection in infection-aware DRRs, we estimate the detection limit of X-ray imaging in detecting COVID-19 infection. The estimated detection limit, measured by the percent volume of the lung that is infected by COVID-19, is 19.43% ± 16.29%, and the estimated lower bound of infected voxel contribution rate for significant radiological signs of COVID-19 infection is 20.0%. Our codes are made publicly available at https://github.com/PengyiZhang/DRR4Covid.

DL-CRC: Deep Learning-Based Chest Radiograph Classification for COVID-19 Detection: A Novel Approach

With the exponentially growing COVID-19 (coronavirus disease 2019) pandemic, clinicians continue to seek accurate and rapid diagnosis methods in addition to virus and antibody testing modalities. Because radiographs such as X-rays and computed tomography (CT) scans are cost-effective and widely available at public health facilities, hospital emergency rooms (ERs), and even at rural clinics, they could be used for rapid detection of possible COVID-19-induced lung infections. Therefore, toward automating the COVID-19 detection, in this paper, we propose a viable and efficient deep learning-based chest radiograph classification (DL-CRC) framework to distinguish the COVID-19 cases with high accuracy from other abnormal (e.g., pneumonia) and normal cases. A unique dataset is prepared from four publicly available sources containing the posteroanterior (PA) chest view of X-ray data for COVID-19, pneumonia, and normal cases. Our proposed DL-CRC framework leverages a data augmentation of radiograph images (DARI) algorithm for the COVID-19 data by adaptively employing the generative adversarial network (GAN) and generic data augmentation methods to generate synthetic COVID-19 infected chest X-ray images to train a robust model. The training data consisting of actual and synthetic chest X-ray images are fed into our customized convolutional neural network (CNN) model in DL-CRC, which achieves COVID-19 detection accuracy of 93.94% compared to 54.55% for the scenario without data augmentation (i.e., when only a few actual COVID-19 chest X-ray image samples are available in the original dataset). Furthermore, we justify our customized CNN model by extensively comparing it with widely adopted CNN architectures in the literature, namely ResNet, Inception-ResNet v2, and DenseNet that represent depth-based, multi-path-based, and hybrid CNN paradigms. The encouragingly high classification accuracy of our proposal implies that it can efficiently automate COVID-19 detection from radiograph images to provide a fast and reliable evidence of COVID-19 infection in the lung that can complement existing COVID-19 diagnostics modalities.