Training Ultrasound Image Classification Deep-Learning Algorithms for Pneumothorax Detection Using a Synthetic Tissue Phantom Apparatus

Evaluation of Deep Learning Model Architectures for Point-of-Care Ultrasound Diagnostics

Point-of-care ultrasound imaging is a critical tool for patient triage during trauma for diagnosing injuries and prioritizing limited medical evacuation resources. Specifically, an eFAST exam evaluates if there are free fluids in the chest or abdomen but this is only possible if ultrasound scans can be accurately interpreted, a challenge in the pre-hospital setting. In this effort, we evaluated the use of artificial intelligent eFAST image interpretation models. Widely used deep learning model architectures were evaluated as well as Bayesian models optimized for six different diagnostic models: pneumothorax (i) B- or (ii) M-mode, hemothorax (iii) B- or (iv) M-mode, (v) pelvic or bladder abdominal hemorrhage and (vi) right upper quadrant abdominal hemorrhage. Models were trained using images captured in 27 swine. Using a leave-one-subject-out training approach, the MobileNetV2 and DarkNet53 models surpassed 85% accuracy for each M-mode scan site. The different B-mode models performed worse with accuracies between 68% and 74% except for the pelvic hemorrhage model, which only reached 62% accuracy for all model architectures. These results highlight which eFAST scan sites can be easily automated with image interpretation models, while other scan sites, such as the bladder hemorrhage model, will require more robust model development or data augmentation to improve performance. With these additional improvements, the skill threshold for ultrasound-based triage can be reduced, thus expanding its utility in the pre-hospital setting.

Design and testing of ultrasound probe adapters for a robotic imaging platform

Medical imaging-based triage is a critical tool for emergency medicine in both civilian and military settings. Ultrasound imaging can be used to rapidly identify free fluid in abdominal and thoracic cavities which could necessitate immediate surgical intervention. However, proper ultrasound image capture requires a skilled ultrasonography technician who is likely unavailable at the point of injury where resources are limited. Instead, robotics and computer vision technology can simplify image acquisition. As a first step towards this larger goal, here, we focus on the development of prototypes for ultrasound probe securement using a robotics platform. The ability of four probe adapter technologies to precisely capture images at anatomical locations, repeatedly, and with different ultrasound transducer types were evaluated across more than five scoring criteria. Testing demonstrated two of the adapters outperformed the traditional robot gripper and manual image capture, with a compact, rotating design compatible with wireless imaging technology being most suitable for use at the point of injury. Next steps will integrate the robotic platform with computer vision and deep learning image interpretation models to automate image capture and diagnosis. This will lower the skill threshold needed for medical imaging-based triage, enabling this procedure to be available at or near the point of injury.

Development of an Artificial Soft Solid Gel Using Gelatin Material for High-Quality Ultrasound Diagnosis

For ultrasound diagnosis, a gel is applied to the skin. Ultrasound gel serves to block air exposure and match impedance between the skin and the probe, enhancing imaging efficiency. However, if use of the ultrasound gel exceeds a certain period of time, it may dry out and be exposed to air, causing impedance mismatch and reducing imaging resolution. In such cases, the use of a soft, solid gel proves advantageous, as it can be employed for an extended period without succumbing to the drying phenomenon and can be reused after disinfection. Its soft consistency ensures excellent skin adhesion. Our soft solid gel demonstrated approximately 1.2 times better performance than water, silicone, and traditional ultrasound gels. When comparing the dimensions of grayscale, dead zone, vertical, and horizontal regions, the measurements for the traditional ultrasound gel were 93.79 mm, 45.32 mm, 103.13 mm, 83.86 mm, and 83.86 mm, respectively. In contrast, the proposed soft solid gel exhibited dimensions of 105.64 mm, 34.48 mm, 141.1 mm, and 102.8 mm.

Quantitative assessment of pneumothorax by using Shannon entropy of lung ultrasound M-mode image and diaphragmatic excursion based on automated measurement

Background

Lung ultrasound (LUS) and diaphragm ultrasound (DUS) are the appropriate modalities for conservative observation to those patients who are with stable pneumothorax, as well as for the timely detection of life-threatening pneumothorax at any location, due to they are portable, real-time, relatively cost effective, and most important, without radiation exposure. The absence of lung sliding on LUS M-mode images and the abnormality of diaphragmatic excursion (DE) on DUS M-mode images are the most common and novel diagnostic criteria for pneumothorax, respectively. However, visual inspection of M-mode images remains subjective and quantitative analysis of LUS and DUS M-mode images are required.

Methods

Shannon entropy of LUS M-mode image (ShanEnLM) and DE based on the automated measurement (DEAM) are adapted to the objective pneumothorax diagnoses and the severity quantifications in this study. Mild, moderate, and severe pneumothoraces were induced in 24 male New Zealand rabbits through insufflation of room air (5, 10 and 15, and 25 and 40 mL/kg, respectively) into their pleural cavities. In vivo intercostal LUS and subcostal DUS M-mode images were acquired using a point-of-care system for estimating ShanEnLM and DEAM.

Results

ShanEnLM and DEAM as functions of air insufflation volumes exhibited U-shaped curves and were exponentially decreasing, respectively. Either ShanEnLM or DEAM had areas under the receiver operating characteristic curves [95% confidence interval (CI)] of 1.0000 (95% CI: 1.0000-1.0000), 0.9833 (95% CI: 0.9214-1.0000), and 0.9407 (95% CI: 0.8511-1.0000) for differentiating between normal and mild pneumothorax, mild and moderate pneumothoraces, and moderate and severe pneumothoraces, respectively.

Conclusions

Our findings imply that the combination of ShanEnLM and DEAM give the promising potential for pneumothorax quantitative diagnosis.

An extended focused assessment with sonography in trauma ultrasound tissue-mimicking phantom for developing automated diagnostic technologies

Introduction: Medical imaging-based triage is critical for ensuring medical treatment is timely and prioritized. However, without proper image collection and interpretation, triage decisions can be hard to make. While automation approaches can enhance these triage applications, tissue phantoms must be developed to train and mature these novel technologies. Here, we have developed a tissue phantom modeling the ultrasound views imaged during the enhanced focused assessment with sonography in trauma exam (eFAST). Methods: The tissue phantom utilized synthetic clear ballistic gel with carveouts in the abdomen and rib cage corresponding to the various eFAST scan points. Various approaches were taken to simulate proper physiology without injuries present or to mimic pneumothorax, hemothorax, or abdominal hemorrhage at multiple locations in the torso. Multiple ultrasound imaging systems were used to acquire ultrasound scans with or without injury present and were used to train deep learning image classification predictive models. Results: Performance of the artificial intelligent (AI) models trained in this study achieved over 97% accuracy for each eFAST scan site. We used a previously trained AI model for pneumothorax which achieved 74% accuracy in blind predictions for images collected with the novel eFAST tissue phantom. Grad-CAM heat map overlays for the predictions identified that the AI models were tracking the area of interest for each scan point in the tissue phantom. Discussion: Overall, the eFAST tissue phantom ultrasound scans resembled human images and were successful in training AI models. Tissue phantoms are critical first steps in troubleshooting and developing medical imaging automation technologies for this application that can accelerate the widespread use of ultrasound imaging for emergency triage.

Using Ultrasound Image Augmentation and Ensemble Predictions to Prevent Machine-Learning Model Overfitting

Deep learning predictive models have the potential to simplify and automate medical imaging diagnostics by lowering the skill threshold for image interpretation. However, this requires predictive models that are generalized to handle subject variability as seen clinically. Here, we highlight methods to improve test accuracy of an image classifier model for shrapnel identification using tissue phantom image sets. Using a previously developed image classifier neural network-termed ShrapML-blind test accuracy was less than 70% and was variable depending on the training/test data setup, as determined by a leave one subject out (LOSO) holdout methodology. Introduction of affine transformations for image augmentation or MixUp methodologies to generate additional training sets improved model performance and overall accuracy improved to 75%. Further improvements were made by aggregating predictions across five LOSO holdouts. This was done by bagging confidences or predictions from all LOSOs or the top-3 LOSO confidence models for each image prediction. Top-3 LOSO confidence bagging performed best, with test accuracy improved to greater than 85% accuracy for two different blind tissue phantoms. This was confirmed by gradient-weighted class activation mapping to highlight that the image classifier was tracking shrapnel in the image sets. Overall, data augmentation and ensemble prediction approaches were suitable for creating more generalized predictive models for ultrasound image analysis, a critical step for real-time diagnostic deployment.

Using an Ultrasound Tissue Phantom Model for Hybrid Training of Deep Learning Models for Shrapnel Detection

Tissue phantoms are important for medical research to reduce the use of animal or human tissue when testing or troubleshooting new devices or technology. Development of machine-learning detection tools that rely on large ultrasound imaging data sets can potentially be streamlined with high quality phantoms that closely mimic important features of biological tissue. Here, we demonstrate how an ultrasound-compliant tissue phantom comprised of multiple layers of gelatin to mimic bone, fat, and muscle tissue types can be used for machine-learning training. This tissue phantom has a heterogeneous composition to introduce tissue level complexity and subject variability in the tissue phantom. Various shrapnel types were inserted into the phantom for ultrasound imaging to supplement swine shrapnel image sets captured for applications such as deep learning algorithms. With a previously developed shrapnel detection algorithm, blind swine test image accuracy reached more than 95% accuracy when training was comprised of 75% tissue phantom images, with the rest being swine images. For comparison, a conventional MobileNetv2 deep learning model was trained with the same training image set and achieved over 90% accuracy in swine predictions. Overall, the tissue phantom demonstrated high performance for developing deep learning models for ultrasound image classification.

Evaluation of an Object Detection Algorithm for Shrapnel and Development of a Triage Tool to Determine Injury Severity

Emergency medicine in austere environments rely on ultrasound imaging as an essential diagnostic tool. Without extensive training, identifying abnormalities such as shrapnel embedded in tissue, is challenging. Medical professionals with appropriate expertise are limited in resource-constrained environments. Incorporating artificial intelligence models to aid the interpretation can reduce the skill gap, enabling identification of shrapnel, and its proximity to important anatomical features for improved medical treatment. Here, we apply a deep learning object detection framework, YOLOv3, for shrapnel detection in various sizes and locations with respect to a neurovascular bundle. Ultrasound images were collected in a tissue phantom containing shrapnel, vein, artery, and nerve features. The YOLOv3 framework, classifies the object types and identifies the location. In the testing dataset, the model was successful at identifying each object class, with a mean Intersection over Union and average precision of 0.73 and 0.94, respectively. Furthermore, a triage tool was developed to quantify shrapnel distance from neurovascular features that could notify the end user when a proximity threshold is surpassed, and, thus, may warrant evacuation or surgical intervention. Overall, object detection models such as this will be vital to compensate for lack of expertise in ultrasound interpretation, increasing its availability for emergency and military medicine.