Comparison of RetinaNet, SSD, and YOLO v3 for real-time pill identification

Intraoperative detection of parathyroid glands using artificial intelligence: optimizing medical image training with data augmentation methods

BACKGROUND

Postoperative hypoparathyroidism is a major complication of thyroidectomy, occurring when the parathyroid glands are inadvertently damaged during surgery. Although intraoperative images are rarely used to train artificial intelligence (AI) because of its complex nature, AI may be trained to intraoperatively detect parathyroid glands using various augmentation methods. The purpose of this study was to train an effective AI model to detect parathyroid glands during thyroidectomy.

METHODS

Video clips of the parathyroid gland were collected during thyroid lobectomy procedures. Confirmed parathyroid images were used to train three types of datasets according to augmentation status: baseline, geometric transformation, and generative adversarial network-based image inpainting. The primary outcome was the average precision of the performance of AI in detecting parathyroid glands.

RESULTS

152 Fine-needle aspiration-confirmed parathyroid gland images were acquired from 150 patients who underwent unilateral lobectomy. The average precision of the AI model in detecting parathyroid glands based on baseline data was 77%. This performance was enhanced by applying both geometric transformation and image inpainting augmentation methods, with the geometric transformation data augmentation dataset showing a higher average precision (79%) than the image inpainting model (78.6%). When this model was subjected to external validation using a completely different thyroidectomy approach, the image inpainting method was more effective (46%) than both the geometric transformation (37%) and baseline (33%) methods.

CONCLUSION

This AI model was found to be an effective and generalizable tool in the intraoperative identification of parathyroid glands during thyroidectomy, especially when aided by appropriate augmentation methods. Additional studies comparing model performance and surgeon identification, however, are needed to assess the true clinical relevance of this AI model.

Automated medication verification system (AMVS): System based on edge detection and CNN classification drug on embedded systems

A novel automated medication verification system (AMVS) aims to address the limitation of manual medication verification among healthcare professionals with a high workload, thereby reducing medication errors in hospitals. Specifically, the manual medication verification process is time-consuming and prone to errors, especially in healthcare settings with high workloads. The proposed system strategy is to streamline and automate this process, enhancing efficiency and reducing medication errors. The system employs deep learning models to swiftly and accurately classify multiple medications within a single image without requiring manual labeling during model construction. It comprises edge detection and classification to verify medication types. Unlike previous studies conducted in open spaces, our study takes place in a closed space to minimize the impact of optical changes on image capture. During the experimental process, the system individually identifies each drug within the image by edge detection method and utilizes a classification model to determine each drug type. Our research has successfully developed a fully automated drug recognition system, achieving an accuracy of over 95 % in identifying drug types and conducting segmentation analyses. Specifically, the system demonstrates an accuracy rate of approximately 96 % for drug sets containing fewer than ten types and 93 % for those with ten types. This verification system builds an image classification model quickly. It holds promising potential in assisting nursing staff during AMVS, thereby reducing the likelihood of medication errors and alleviating the burden on nursing staff.

Leveraging code-free deep learning for pill recognition in clinical settings: A multicenter, real-world study of performance across multiple platforms

BACKGROUND

Preventable patient harm, particularly medication errors, represent significant challenges in healthcare settings. Dispensing the wrong medication is often associated with mix-up of lookalike and soundalike drugs in high workload environments. Replacing manual dispensing with automated unit dose and medication dispensing systems to reduce medication errors is not always feasible in clinical facilities experiencing high patient turn-around or frequent dose changes. Artificial intelligence (AI) based pill recognition tools and smartphone applications could potentially aid healthcare workers in identifying pills in situations where more advanced dispensing systems are not implemented.

OBJECTIVE

Most of the published research on pill recognition focuses on theoretical aspects of model development using traditional coding and deep learning methods. The use of code-free deep learning (CFDL) as a practical alternative for accessible model development, and implementation of such models in tools intended to aid decision making in clinical settings, remains largely unexplored. In this study, we sought to address this gap in existing literature by investigating whether CFDL is a viable approach for developing pill recognition models using a custom dataset, followed by a thorough evaluation of the model across various deployment scenarios, and in multicenter clinical settings. Furthermore, we aimed to highlight challenges and propose solutions to achieve optimal performance and real-world applicability of pill recognition models, including when deployed on smartphone applications.

METHODS

A pill recognition model was developed utilizing Microsoft Azure Custom Vision platform and a large custom training dataset of 26,880 images captured from the top 30 most dispensed solid oral dosage forms (SODFs) at the three participating hospitals. A comprehensive internal and external testing strategy was devised, model's performance was investigated through the online API, and offline using exported TensorFlow Lite model running on a Windows PC and on Android, using a tailor-made testing smartphone application. Additionally, model's calibration, degree of reliance on color features and device dependency was thoroughly evaluated. Real-world performance was assessed using images captured by hospital pharmacists at three participating clinical centers.

RESULTS

The pill recognition model showed high performance in Microsoft Azure Custom Vision platform with 98.7 % precision, 95.1 % recall, and 98.2 % mean average precision (mAP), with thresholds set to 50 %. During internal testing utilizing the online API, the model reached 93.7 % precision, 88.96 % recall, 90.81 % F1-score and 87.35 % mAP. Testing the offline TensorFlow Lite model on Windows PC showed a slight performance reduction, with 91.16 % precision, 83.82 % recall, 86.18 % F1-score and 82.55 % mAP. Performance of the model running offline on the Android application was further reduced to 86.50 % precision, 75.00 % recall, 77.83 % F1-score and 69.24 % mAP. During external clinical testing through the online API an overall precision of 83.10 %, recall of 71.39 %, and F1-score of 75.76 % was achieved.

CONCLUSION

Our study demonstrates that using a CFDL approach is a feasible and cost-effective method for developing AI-based pill recognition systems. Despite the limitations encountered, our model performed well, particularly when accessed through the online API. The use of CFDL facilitates interdisciplinary collaboration, resulting in human-centered AI models with enhanced real-world applicability. We suggest that rather than striving to build a universally applicable pill recognition system, models should be tailored to the medications in a regional formulary or needs of a specific clinic, which can in turn lead to improved performance in real-world deployment in these locations. Parallel to focusing on model development, it is crucial to employ a human centered approach by training the end users on how to properly interact with the AI based system to maximize benefits. Future research is needed on refining pill recognition models for broader adaptability. This includes investigating image pre-processing and optimization techniques to enhance offline performance and operation on handheld devices. Moreover, future studies should explore methods to overcome limitations of CFDL development to enhance the robustness of models and reduce overfitting. Collaborative efforts between researchers in this domain and sharing of best practices are vital to improve pill recognition systems, ultimately enhancing patient safety and healthcare outcomes.

Computer vision model for the detection of canine pododermatitis and neoplasia of the paw

BACKGROUND

Artificial intelligence (AI) has been used successfully in human dermatology. AI utilises convolutional neural networks (CNN) to accomplish tasks such as image classification, object detection and segmentation, facilitating early diagnosis. Computer vision (CV), a field of AI, has shown great results in detecting signs of human skin diseases. Canine paw skin diseases are a common problem in general veterinary practice, and computer vision tools could facilitate the detection and monitoring of disease processes. Currently, no such tool is available in veterinary dermatology.

ANIMALS

Digital images of paws from healthy dogs and paws with pododermatitis or neoplasia were used.

OBJECTIVES

We tested the novel object detection model Pawgnosis, a Tiny YOLOv4 image analysis model deployed on a microcomputer with a camera for the rapid detection of canine pododermatitis and neoplasia.

MATERIALS AND METHODS

The prediction performance metrics used to evaluate the models included mean average precision (mAP), precision, recall, average precision (AP) for accuracy and frames per second (FPS) for speed.

RESULTS

A large dataset labelled by a single individual (Dataset A) used to train a Tiny YOLOv4 model provided the best results with a mean mAP of 0.95, precision of 0.86, recall of 0.93 and 20 FPS.

CONCLUSIONS AND CLINICAL RELEVANCE

This novel object detection model has the potential for application in the field of veterinary dermatology.

Design and Characterization of a Powered Wheelchair Autonomous Guidance System

The current technological revolution driven by advances in machine learning has motivated a wide range of applications aiming to improve our quality of life. Representative of such applications are autonomous and semiautonomous Powered Wheelchairs (PWs), where the focus is on providing a degree of autonomy to the wheelchair user as a matter of guidance and interaction with the environment. Based on these perspectives, the focus of the current research has been on the design of lightweight systems that provide the necessary accuracy in the navigation system while enabling an embedded implementation. This motivated us to develop a real-time measurement methodology that relies on a monocular RGB camera to detect the caregiver's feet based on a deep learning method, followed by the distance measurement of the caregiver from the PW. An important contribution of this article is the metrological characterization of the proposed methodology in comparison with measurements made with dedicated depth cameras. Our results show that despite shifting from 3D imaging to 2D imaging, we can still obtain comparable metrological performances in distance estimation as compared with Light Detection and Ranging (LiDAR) or even improved compared with stereo cameras. In particular, we obtained comparable instrument classes with LiDAR and stereo cameras, with measurement uncertainties within a magnitude of 10 cm. This is further complemented by the significant reduction in data volume and object detection complexity, thus facilitating its deployment, primarily due to the reduced complexity of initial calibration, positioning, and deployment compared with three-dimensional segmentation algorithms.

Assessment of Narrow-Band Imaging Algorithm for Video Capsule Endoscopy Based on Decorrelated Color Space for Esophageal Cancer: Part II, Detection and Classification of Esophageal Cancer

Esophageal carcinoma (EC) is a prominent contributor to cancer-related mortality since it lacks discernible features in its first phases. Multiple studies have shown that narrow-band imaging (NBI) has superior accuracy, sensitivity, and specificity in detecting EC compared to white light imaging (WLI). Thus, this study innovatively employs a color space linked to décor to transform WLIs into NBIs, offering a novel approach to enhance the detection capabilities of EC in its early stages. In this study a total of 3415 WLI along with the corresponding 3415 simulated NBI images were used for analysis combined with the YOLOv5 algorithm to train the WLI images and the NBI images individually showcasing the adaptability of advanced object detection techniques in the context of medical image analysis. The evaluation of the model's performance was based on the produced confusion matrix and five key metrics: precision, recall, specificity, accuracy, and F1-score of the trained model. The model underwent training to accurately identify three specific manifestations of EC, namely dysplasia, squamous cell carcinoma (SCC), and polyps demonstrates a nuanced and targeted analysis, addressing diverse aspects of EC pathology for a more comprehensive understanding. The NBI model effectively enhanced both its recall and accuracy rates in detecting dysplasia cancer, a pre-cancerous stage that might improve the overall five-year survival rate. Conversely, the SCC category decreased its accuracy and recall rate, although the NBI and WLI models performed similarly in recognizing the polyp. The NBI model demonstrated an accuracy of 0.60, 0.81, and 0.66 in the dysplasia, SCC, and polyp categories, respectively. Additionally, it attained a recall rate of 0.40, 0.73, and 0.76 in the same categories. The WLI model demonstrated an accuracy of 0.56, 0.99, and 0.65 in the dysplasia, SCC, and polyp categories, respectively. Additionally, it obtained a recall rate of 0.39, 0.86, and 0.78 in the same categories, respectively. The limited number of training photos is the reason for the suboptimal performance of the NBI model which can be improved by increasing the dataset.

A deep convolutional neural network for efficient microglia detection

Microglial cells are a type of glial cells that make up 10-15% of all brain cells, and they play a significant role in neurodegenerative disorders and cardiovascular diseases. Despite their vital role in these diseases, developing fully automated microglia counting methods from immunohistological images is challenging. Current image analysis methods are inefficient and lack accuracy in detecting microglia due to their morphological heterogeneity. This study presents development and validation of a fully automated and efficient microglia detection method using the YOLOv3 deep learning-based algorithm. We applied this method to analyse the number of microglia in different spinal cord and brain regions of rats exposed to opioid-induced hyperalgesia/tolerance. Our numerical tests showed that the proposed method outperforms existing computational and manual methods with high accuracy, achieving 94% precision, 91% recall, and 92% F1-score. Furthermore, our tool is freely available and adds value to exploring different disease models. Our findings demonstrate the effectiveness and efficiency of our new tool in automated microglia detection, providing a valuable asset for researchers in neuroscience.

Human Detection and Action Recognition for Search and Rescue in Disasters Using YOLOv3 Algorithm

Drone examination has been overall quickly embraced by NDMM (natural disaster mitigation and management) division to survey the state of impacted regions. Manual video analysis by human observers takes time and is subject to mistakes. The human identification examination of pictures caught by drones will give a practical method for saving lives who are being trapped under debris during quakes or in floods and so on. Drone investigation for research and security and search and rescue (SAR) should involve the drone to filter the impacted area using a camera and a model of unmanned area vehicles (UAVs) to identify specific locations where assistance is required. The existing methods (Balmukund et al. 2020) used were faster-region based convolutional neural networks (F-RCNNs), single shot detector (SSD), and region-based fully convolutional network (R-FCN) for the detection of human and recognition of action. Some of the existing methods used 700 images with six classes only, whereas the proposed model uses 1996 images with eight classes. The proposed model is used YOLOv3 (you only look once) algorithm for the detection and recognition of actions. In this study, we provide the fundamental ideas underlying an object detection model. To find the most effective model for human recognition and detection, we trained the YOLOv3 algorithm on the image dataset and evaluated its performance. We compared the outcomes with the existing algorithms like F-RCNN, SSD, and R-FCN. The accuracies of F-RCNN, SSD, R-FCN (existing algorithms), and YOLOv3 (proposed algorithm) are 53%, 73%, 93%, and 94.9%, respectively. Among these algorithms, the YOLOv3 algorithm gives the highest accuracy of 94.9%. The proposed work shows that existing models are inadequate for critical applications like search and rescue, which convinces us to propose a model raised by a pyramidal component extracting SSD in human localization and action recognition. The suggested model is 94.9% accurate when applied to the proposed dataset, which is an important contribution. Likewise, the suggested model succeeds in helping time for expectation in examination with the cutting-edge identification models with existing strategies. The average time taken by our proposed technique to distinguish a picture is 0.40 milisec which is a lot better than the existing method. The proposed model can likewise distinguish video and can be utilized for real-time recognition. The SSD model can likewise use to anticipate messages if present in the picture.

Maxillofacial fracture detection and classification in computed tomography images using convolutional neural network-based models

The purpose of this study was to evaluate the performance of convolutional neural network-based models for the detection and classification of maxillofacial fractures in computed tomography (CT) maxillofacial bone window images. A total of 3407 CT images, 2407 of which contained maxillofacial fractures, were retrospectively obtained from the regional trauma center from 2016 to 2020. Multiclass image classification models were created by using DenseNet-169 and ResNet-152. Multiclass object detection models were created by using faster R-CNN and YOLOv5. DenseNet-169 and ResNet-152 were trained to classify maxillofacial fractures into frontal, midface, mandibular and no fracture classes. Faster R-CNN and YOLOv5 were trained to automate the placement of bounding boxes to specifically detect fracture lines in each fracture class. The performance of each model was evaluated on an independent test dataset. The overall accuracy of the best multiclass classification model, DenseNet-169, was 0.70. The mean average precision of the best multiclass detection model, faster R-CNN, was 0.78. In conclusion, DenseNet-169 and faster R-CNN have potential for the detection and classification of maxillofacial fractures in CT images.

Classification of Skin Cancer Using Novel Hyperspectral Imaging Engineering via YOLOv5

Many studies have recently used several deep learning methods for detecting skin cancer. However, hyperspectral imaging (HSI) is a noninvasive optics system that can obtain wavelength information on the location of skin cancer lesions and requires further investigation. Hyperspectral technology can capture hundreds of narrow bands of the electromagnetic spectrum both within and outside the visible wavelength range as well as bands that enhance the distinction of image features. The dataset from the ISIC library was used in this study to detect and classify skin cancer on the basis of basal cell carcinoma (BCC), squamous cell carcinoma (SCC), and seborrheic keratosis (SK). The dataset was divided into training and test sets, and you only look once (YOLO) version 5 was applied to train the model. The model performance was judged according to the generated confusion matrix and five indicating parameters, including precision, recall, specificity, accuracy, and the F1-score of the trained model. Two models, namely, hyperspectral narrowband image (HSI-NBI) and RGB classification, were built and then compared in this study to understand the performance of HSI with the RGB model. Experimental results showed that the HSI model can learn the SCC feature better than the original RGB image because the feature is more prominent or the model is not captured in other categories. The recall rate of the RGB and HSI models were 0.722 to 0.794, respectively, thereby indicating an overall increase of 7.5% when using the HSI model.

Enhancement Technique Based on the Breast Density Level for Mammogram for Computer-Aided Diagnosis

Mass detection in mammograms has a limited approach to the presence of a mass in overlapping denser fibroglandular breast regions. In addition, various breast density levels could decrease the learning system's ability to extract sufficient feature descriptors and may result in lower accuracy performance. Therefore, this study is proposing a textural-based image enhancement technique named Spatial-based Breast Density Enhancement for Mass Detection (SbBDEM) to boost textural features of the overlapped mass region based on the breast density level. This approach determines the optimal exposure threshold of the images' lower contrast limit and optimizes the parameters by selecting the best intensity factor guided by the best Blind/Reference-less Image Spatial Quality Evaluator (BRISQUE) scores separately for both dense and non-dense breast classes prior to training. Meanwhile, a modified You Only Look Once v3 (YOLOv3) architecture is employed for mass detection by specifically assigning an extra number of higher-valued anchor boxes to the shallower detection head using the enhanced image. The experimental results show that the use of SbBDEM prior to training mass detection promotes superior performance with an increase in mean Average Precision (mAP) of 17.24% improvement over the non-enhanced trained image for mass detection, mass segmentation of 94.41% accuracy, and 96% accuracy for benign and malignant mass classification. Enhancing the mammogram images based on breast density is proven to increase the overall system's performance and can aid in an improved clinical diagnosis process.

Adaptive IoU Thresholding for Improving Small Object Detection: A Proof-of-Concept Study of Hand Erosions Classification of Patients with Rheumatic Arthritis on X-ray Images

In recent years, much research evaluating the radiographic destruction of finger joints in patients with rheumatoid arthritis (RA) using deep learning models was conducted. Unfortunately, most previous models were not clinically applicable due to the small object regions as well as the close spatial relationship. In recent years, a new network structure called RetinaNets, in combination with the focal loss function, proved reliable for detecting even small objects. Therefore, the study aimed to increase the recognition performance to a clinically valuable level by proposing an innovative approach with adaptive changes in intersection over union (IoU) values during training of Retina Networks using the focal loss error function. To this end, the erosion score was determined using the Sharp van der Heijde (SvH) metric on 300 conventional radiographs from 119 patients with RA. Subsequently, a standard RetinaNet with different IoU values as well as adaptively modified IoU values were trained and compared in terms of accuracy, mean average accuracy (mAP), and IoU. With the proposed approach of adaptive IoU values during training, erosion detection accuracy could be improved to 94% and an mAP of 0.81 ± 0.18. In contrast Retina networks with static IoU values achieved only an accuracy of 80% and an mAP of 0.43 ± 0.24. Thus, adaptive adjustment of IoU values during training is a simple and effective method to increase the recognition accuracy of small objects such as finger and wrist joints.

Influence of Insufficient Dataset Augmentation on IoU and Detection Threshold in CNN Training for Object Detection on Aerial Images

The objects and events detection tasks are being performed progressively often by robotic systems like unmanned aerial vehicles (UAV) or unmanned surface vehicles (USV). Autonomous operations and intelligent sensing are becoming standard in numerous scenarios such as supervision or even search and rescue (SAR) missions. The low cost of autonomous vehicles, vision sensors and portable computers allows the incorporation of the deep learning, mainly convolutional neural networks (CNN) in these solutions. Many systems meant for custom purposes rely on insufficient training datasets, what may cause a decrease of effectiveness. Moreover, the system's accuracy is usually dependent on the returned bounding boxes highlighting the supposed targets. In desktop applications, precise localisation might not be particularly relevant; however, in real situations, with low visibility and non-optimal camera orientation, it becomes crucial. One of the solutions for dataset enhancement is its augmentation. The presented work is an attempt to evaluate the influence of the training images augmentation on the detection parameters important for the effectiveness of neural networks in the context of object detection. In this research, network appraisal relies on the detection confidence and bounding box prediction accuracy (IoU). All the applied image modifications were simple pattern and colour alterations. The obtained results imply that there is a measurable impact of the augmentation process on the localisation accuracy. It was concluded that a positive or negative influence is related to the complexity and variability of the objects classes.

Vision-Based Module for Herding with a Sheepdog Robot

Livestock farming is assisted more and more by technological solutions, such as robots. One of the main problems for shepherds is the control and care of livestock in areas difficult to access where grazing animals are attacked by predators such as the Iberian wolf in the northwest of the Iberian Peninsula. In this paper, we propose a system to automatically generate benchmarks of animal images of different species from iNaturalist API, which is coupled with a vision-based module that allows us to automatically detect predators and distinguish them from other animals. We tested multiple existing object detection models to determine the best one in terms of efficiency and speed, as it is conceived for real-time environments. YOLOv5m achieves the best performance as it can process 64 FPS, achieving an mAP (with IoU of 50%) of 99.49% for a dataset where wolves (predator) or dogs (prey) have to be detected and distinguished. This result meets the requirements of pasture-based livestock farms.

Neural Network-Based Price Tag Data Analysis

This paper compares neural networks, specifically Unet, MobileNetV2, VGG16 and YOLOv4-tiny, for image segmentation as part of a study aimed at finding an optimal solution for price tag data analysis. The neural networks considered were trained on an individual dataset collected by the authors. Additionally, this paper covers the automatic image text recognition approach using EasyOCR API. Research revealed that the optimal network for segmentation is YOLOv4-tiny, featuring a cross validation accuracy of 96.92%. EasyOCR accuracy was also calculated and is 95.22%.