Assessing YOLACT++ for real time and robust instance segmentation of medical instruments in endoscopic procedures

Identification of surgical forceps using YOLACT++ in different lighted environments

Forceps tracking in laparoscopic surgery contributes to improved surgical outcomes. We identified forceps using YOLACT++ for fast and accurate segmentation. Differences in the illumination of the environment can affect the image recognition accuracy in deep learning. Therefore, we examined the speed and accuracy of YOLACT++ forceps identification in different illuminated environments. We expected that this experiment would help us understand the optimal lighted environments for YOLACT++ and to further improve the performance of the forceps identification model. The greatest accuracy was obtained under a light-shielded environment with light shining only on the suture area. Although a laparotomy with a clear view of the surgical site is easier for the physician to operate in, we concluded that the forceps identification model of YOLACT++ can be used more effectively in the laparoscopic surgical environment.Clinical Relevance- This study contributes to analyzing the cause of surgical errors in laparoscopic surgery.

CropQuant-Air: an AI-powered system to enable phenotypic analysis of yield- and performance-related traits using wheat canopy imagery collected by low-cost drones

As one of the most consumed stable foods around the world, wheat plays a crucial role in ensuring global food security. The ability to quantify key yield components under complex field conditions can help breeders and researchers assess wheat's yield performance effectively. Nevertheless, it is still challenging to conduct large-scale phenotyping to analyse canopy-level wheat spikes and relevant performance traits, in the field and in an automated manner. Here, we present CropQuant-Air, an AI-powered software system that combines state-of-the-art deep learning (DL) models and image processing algorithms to enable the detection of wheat spikes and phenotypic analysis using wheat canopy images acquired by low-cost drones. The system includes the YOLACT-Plot model for plot segmentation, an optimised YOLOv7 model for quantifying the spike number per m² (SNpM²) trait, and performance-related trait analysis using spectral and texture features at the canopy level. Besides using our labelled dataset for model training, we also employed the Global Wheat Head Detection dataset to incorporate varietal features into the DL models, facilitating us to perform reliable yield-based analysis from hundreds of varieties selected from main wheat production regions in China. Finally, we employed the SNpM² and performance traits to develop a yield classification model using the Extreme Gradient Boosting (XGBoost) ensemble and obtained significant positive correlations between the computational analysis results and manual scoring, indicating the reliability of CropQuant-Air. To ensure that our work could reach wider researchers, we created a graphical user interface for CropQuant-Air, so that non-expert users could readily use our work. We believe that our work represents valuable advances in yield-based field phenotyping and phenotypic analysis, providing useful and reliable toolkits to enable breeders, researchers, growers, and farmers to assess crop-yield performance in a cost-effective approach.