Semi-automatic tracking, smoothing and segmentation of hyoid bone motion from videofluoroscopic swallowing study

PECI-Net: Bolus segmentation from video fluoroscopic swallowing study images using preprocessing ensemble and cascaded inference

Bolus segmentation is crucial for the automated detection of swallowing disorders in videofluoroscopic swallowing studies (VFSS). However, it is difficult for the model to accurately segment a bolus region in a VFSS image because VFSS images are translucent, have low contrast and unclear region boundaries, and lack color information. To overcome these challenges, we propose PECI-Net, a network architecture for VFSS image analysis that combines two novel techniques: the preprocessing ensemble network (PEN) and the cascaded inference network (CIN). PEN enhances the sharpness and contrast of the VFSS image by combining multiple preprocessing algorithms in a learnable way. CIN reduces ambiguity in bolus segmentation by using context from other regions through cascaded inference. Moreover, CIN prevents undesirable side effects from unreliably segmented regions by referring to the context in an asymmetric way. In experiments, PECI-Net exhibited higher performance than four recently developed baseline models, outperforming TernausNet, the best among the baseline models, by 4.54% and the widely used UNet by 10.83%. The results of the ablation studies confirm that CIN and PEN are effective in improving bolus segmentation performance.

Deep Learning for Automatic Hyoid Tracking in Videofluoroscopic Swallow Studies

The hyoid bone excursion is one of the most important gauges of larynx elevation in swallowing, contributing to airway protection and bolus passage into the esophagus. However, the implications of various parameters of hyoid bone excursion, such as the horizontal or vertical displacement and velocity, remain elusive and raise the need for a tool providing automatic kinematics analysis. Several conventional and deep learning-based models have been applied automatically to track the hyoid bone, but previous methods either require partial manual localization or do not transform the trajectory by anatomic axis. This work describes a convolutional neural network-based algorithm featuring fully automatic hyoid bone localization and tracking and spine axis determination. The algorithm automatically estimates the hyoid bone trajectory and calculates several physical quantities, including the average velocity and displacement in horizontal or vertical anatomic axis. The model was trained in a dataset of 365 videos of videofluoroscopic swallowing from 189 patients in a tertiary medical center and tested using 44 videos from 44 patients with different dysphagia etiologies. The algorithm showed high detection rates for the hyoid bone. The results showed excellent inter-rater reliability for hyoid bone detection, good-to-excellent inter-rater reliability for calculating the maximal displacement and the average velocity of the hyoid bone in horizontal or vertical directions, and moderate-to-good reliability in calculating the average velocity in horizontal direction. The proposed algorithm allows for complete automatic kinematic analysis of hyoid bone excursion, providing a versatile tool with high potential for clinical applications.

Automated pharyngeal phase detection and bolus localization in videofluoroscopic swallowing study: Killing two birds with one stone?

BACKGROUND AND OBJECTIVE

The videofluoroscopic swallowing study (VFSS) is a gold-standard imaging technique for assessing swallowing, but analysis and rating of VFSS recordings is time consuming and requires specialized training and expertise. Researchers have recently demonstrated that it is possible to automatically detect the pharyngeal phase of swallowing and to localize the bolus in VFSS recordings via computer vision approaches, fostering the development of novel techniques for automatic VFSS analysis. However, training of algorithms to perform these tasks requires large amounts of annotated data that are seldom available. In this paper, we demonstrate that the challenges of pharyngeal phase detection and bolus localization can be solved together using a single approach.

METHODS

We propose a deep-learning framework that jointly tackles pharyngeal phase detection and bolus localization in a weakly-supervised manner, requiring only the initial and final frames of the pharyngeal phase as ground truth annotations for the training. Our approach stems from the observation that bolus presence in the pharynx is the most prominent visual feature upon which to infer whether individual VFSS frames belong to the pharyngeal phase. We conducted extensive experiments with multiple convolutional neural networks (CNNs) on a dataset of 1245 bolus-level clips from 59 healthy subjects.

RESULTS

We demonstrated that the pharyngeal phase can be detected with an F1-score higher than 0.9. Moreover, by processing the class activation maps of the CNNs, we were able to localize the bolus with promising results, obtaining correlations with ground truth trajectories higher than 0.9, without any manual annotations of bolus location used for training purposes.

CONCLUSIONS

Once validated on a larger sample of participants with swallowing disorders, our framework will pave the way for the development of intelligent tools for VFSS analysis to support clinicians in swallowing assessment.

Model-based joint curve registration and classification

In this paper, we consider the problem of classification of misaligned multivariate functional data. We propose to use a model-based approach for the joint registration and classification of such data. The observed functional inputs are modeled as a functional nonlinear mixed effects model containing a nonlinear functional fixed effect constructed upon warping functions to account for curve alignment, and a nonlinear functional random effects component to address the variability among subjects. The warping functions are also modeled to accommodate common effect within groups and the variability between subjects. Then, a functional logistic regression model defined upon the representation of the aligned curves and scalar inputs is used to account for curve classification. EM-based algorithms are developed to perform maximum likelihood inference of the proposed models. The identifiability of the registration model and the asymptotical properties of the proposed method are established. The performance of the proposed procedure is illustrated via simulation studies and an analysis of a hyoid bone movement data application. The statistical developments proposed in this paper were motivated by the hyoid bone movement study, the methodology is designed and presented generality and can be applied to numerous areas of scientific research.

Hyoid Bone Tracking in a Videofluoroscopic Swallowing Study Using a Deep-Learning-Based Segmentation Network

Kinematic analysis of the hyoid bone in a videofluorosopic swallowing study (VFSS) is important for assessing dysphagia. However, calibrating the hyoid bone movement is time-consuming, and its reliability shows wide variation. Computer-assisted analysis has been studied to improve the efficiency and accuracy of hyoid bone identification and tracking, but its performance is limited. In this study, we aimed to design a robust network that can track hyoid bone movement automatically without human intervention. Using 69,389 frames from 197 VFSS files as the data set, a deep learning model for detection and trajectory prediction was constructed and trained by the BiFPN-U-Net(T) network. The present model showed improved performance when compared with the previous models: an area under the curve (AUC) of 0.998 for pixelwise accuracy, an accuracy of object detection of 99.5%, and a Dice similarity of 90.9%. The bounding box detection performance for the hyoid bone and reference objects was superior to that of other models, with a mean average precision of 95.9%. The estimation of the distance of hyoid bone movement also showed higher accuracy. The deep learning model proposed in this study could be used to detect and track the hyoid bone more efficiently and accurately in VFSS analysis.

Automatic Hyoid Bone Tracking in Real-Time Ultrasound Swallowing Videos Using Deep Learning Based and Correlation Filter Based Trackers

(1) Background: Ultrasound provides a radiation-free and portable method for assessing swallowing. Hyoid bone locations and displacements are often used as important indicators for the evaluation of swallowing disorders. However, this requires clinicians to spend a great deal of time reviewing the ultrasound images. (2) Methods: In this study, we applied tracking algorithms based on deep learning and correlation filters to detect hyoid locations in ultrasound videos collected during swallowing. Fifty videos were collected from 10 young, healthy subjects for training, evaluation, and testing of the trackers. (3) Results: The best performing deep learning algorithm, Fully-Convolutional Siamese Networks (SiamFC), proved to have reliable performance in getting accurate hyoid bone locations from each frame of the swallowing ultrasound videos. While having a real-time frame rate (175 fps) when running on an RTX 2060, SiamFC also achieved a precision of 98.9% at the threshold of 10 pixels (3.25 mm) and 80.5% at the threshold of 5 pixels (1.63 mm). The tracker's root-mean-square error and average error were 3.9 pixels (1.27 mm) and 3.3 pixels (1.07 mm), respectively. (4) Conclusions: Our results pave the way for real-time automatic tracking of the hyoid bone in ultrasound videos for swallowing assessment.

How a personalised transportable folding device for seating impacts dysphagia

PURPOSE

A personalised transportable folding device for seating (DATP) on a standard seat was developed by an occupational therapist at the Toulouse University Hospital Centre (patent no. WO 2011121249 A1) based on the hypothesis that the use of a seat to assist with better positioning on any chair during meals modifies the sitting posture and has an impact on cervical statics which increases the amplitude of movements of the axial skeleton (larynx and hyoid bone) and benefits swallowing. The aim of this work is to demonstrate that an improvement in sitting posture with the help of the DATP, through Hyoid bone motion, has an impact on the quality of swallowing in a dysphagic population which benefits from the device in comparison to a dysphagic population which does not benefit from the device after 1 month of care. The secondary endpoints concern the evaluation of the impact on other characteristics of swallowing, posture, the acceptability of the device and the quality of life.

METHODOLOGY

This is a randomised comparative clinical trial. The blind was not possible for the patients but the examiner who evaluated the outcome criterion was blinded to the group to which the patient belonged. The outcome criterion was the measurement of the hyoid bone movement during swallowing. The other criteria were collected during the videofluoroscopic examination of swallowing and by use of a questionnaire. Fifty-six (56) patients were included: 30 in the group without device (D-) and 26 in the group with the device (D+). All the patients benefited from a training course on seating. Only the D+ patients participated in this course where the use of the device was explained and the device was then kept for use at home for 1 month.

RESULTS

A significant improvement was noted in the postural criteria before and after use, in favour of a better posture for the two groups (p < 0.001) and more hyoid bone motion in the D+ group. The difference was significant in the bivariate analysis for horizontal movement (p = 0.04). After adjustment of potential factors of confusion, we noted a significant mean difference for the three distances in the D+ group in comparison to the D- group, of + 0.33 (95% CI [+ 0.17; + 0.48]) for horizontal movement, + 0.22 (95% CI [+ 0.03; + 0.40]) for vertical movement and + 0.37 (95% CI = [+ 0.20; + 0.53]) for horizontal movement. However, the other parameters, and notably the other swallowing markers were not significantly modified by the use of the device.

CONCLUSION

The personalised transportable folding device for seating developed to reduce dysphagia has an action on hyoid bone motion during swallowing. However, this positive effect on an intermediate outcome criterion of the quality of swallowing was not associated with an improvement in swallowing efficiency in the study population. The diversity of diseases with which the patients in this study were afflicted is a factor to be controlled in future studies with this device.

Automatic hyoid bone detection in fluoroscopic images using deep learning

The displacement of the hyoid bone is one of the key components evaluated in the swallow study, as its motion during swallowing is related to overall swallowing integrity. In daily research settings, experts visually detect the hyoid bone in the video frames and manually plot hyoid bone position frame by frame. This study aims to develop an automatic method to localize the location of the hyoid bone in the video sequence. To automatically detect the location of the hyoid bone in a frame, we proposed a single shot multibox detector, a deep convolutional neural network, which is employed to detect and classify the location of the hyoid bone. We also evaluated the performance of two other state-of-art detection methods for comparison. The experimental results clearly showed that the single shot multibox detector can detect the hyoid bone with an average precision of 89.14% and outperform other auto-detection algorithms. We conclude that this automatic hyoid bone tracking system is accurate enough to be widely applied as a pre-processing step for image processing in dysphagia research, as well as a promising development that may be useful in the diagnosis of dysphagia.