Composited FishNet: Fish Detection and Species Recognition From Low-Quality Underwater Videos

WBi-YOLOSF: improved feature pyramid network for aquatic real-time target detection under the artificial rabbits optimization

As the pillar industry of coastal areas, aquaculture needs artificial intelligence technology to promote economic development. To realize the automation of the aquaculture industry, this paper proposes a new underwater object detection network: WBi-YOLOSF. It realizes the automatic classification and detection of aquatic products, improves the production efficiency of the aquaculture industry, and promotes its economic development. This paper creates an image dataset containing 15 aquatic products to lay the data foundation for model training. In the data preprocessing part, an underwater image enhancement algorithm is proposed to improve the quality of the data set effectively. Aiming at the problem of high false detection rate and missed detection rate of underwater dense small targets, a new data enhancement method was proposed to improve the training set's data quality comprehensively. Inspired by the weighted bidirectional feature pyramid network, this paper proposes a new feature extraction network: AU-BiFPN, which solves the gradient problem caused by the network hierarchy's deepening on enhancing the network's multi-scale feature fusion. The AU-BiFPN network structure is embedded into the YOLO series network framework, significantly improving the basic network's feature extraction and feature fusion ability and dramatically improving the prediction accuracy without affecting the network inference speed. Here, the swarm intelligence algorithm is introduced to optimize the model hyperparameters, accelerating the convergence speed of model training and significantly reducing the computational cost. At the same time, the model's accuracy is improved by a cliff. In addition, the Funnel Activation is introduced in the network's backbone, and the simple, parameter-free attention module is integrated, effectively improving the accuracy and speed of the model prediction. Ablation and comparison experiments show the effectiveness and superiority of the proposed model. Verified by the mean average precision and frame rate evaluation indicators, the experimental results of the WBi-YOLOSF target detection network can reach 0.982 and 203 frames per second, which are 1.4% and five frames per second higher than the network with the second score. In summary, this method can quickly and accurately identify aquatic products, realize real-time target detection of aquatic products, and lay the foundation for developing an aquaculture automation system.

An Improved YOLOv8n Used for Fish Detection in Natural Water Environments

To improve detection efficiency and reduce cost consumption in fishery surveys, target detection methods based on computer vision have become a new method for fishery resource surveys. However, the specialty and complexity of underwater photography result in low detection accuracy, limiting its use in fishery resource surveys. To solve these problems, this study proposed an accurate method named BSSFISH-YOLOv8 for fish detection in natural underwater environments. First, replacing the original convolutional module with the SPD-Conv module allows the model to lose less fine-grained information. Next, the backbone network is supplemented with a dynamic sparse attention technique, BiFormer, which enhances the model's attention to crucial information in the input features while also optimizing detection efficiency. Finally, adding a 160 × 160 small target detection layer (STDL) improves sensitivity for smaller targets. The model scored 88.3% and 58.3% in the two indicators of mAP@50 and mAP@50:95, respectively, which is 2.0% and 3.3% higher than the YOLOv8n model. The results of this research can be applied to fishery resource surveys, reducing measurement costs, improving detection efficiency, and bringing environmental and economic benefits.

Convolutional Neural Networks in the Inspection of Serrasalmids (Characiformes) Fingerlings

Aquaculture produces more than 122 million tons of fish globally. Among the several economically important species are the Serrasalmidae, which are valued for their nutritional and sensory characteristics. To meet the growing demand, there is a need for automation and accuracy of processes, at a lower cost. Convolutional neural networks (CNNs) are a viable alternative for automation, reducing human intervention, work time, errors, and production costs. Therefore, the objective of this work is to evaluate the efficacy of convolutional neural networks (CNNs) in counting round fish fingerlings (Serrasalmidae) at different densities using 390 color photographs in an illuminated environment. The photographs were submitted to two convolutional neural networks for object detection: one model was adapted from a pre-trained CNN and the other was an online platform based on AutoML. The metrics used for performance evaluation were precision (P), recall (R), accuracy (A), and F1-Score. In conclusion, convolutional neural networks (CNNs) are effective tools for detecting and counting fish. The pre-trained CNN demonstrated outstanding performance in identifying fish fingerlings, achieving accuracy, precision, and recall rates of 99% or higher, regardless of fish density. On the other hand, the AutoML exhibited reduced accuracy and recall rates as the number of fish increased.

Image-fusion-based object detection using a time-of-flight camera

In this work, we demonstrate an innovative object detection framework based on depth and active infrared intensity images fusion with a time-of-flight (ToF) camera. A slide window weight fusion (SWWF) method provides fuse image with two modalities to localize targets. Then, the depth and intensity information is extracted to construct a joint feature space. Next, we utilize four machine learning methods to achieve object recognition. To verify this method, experiments are performed on an in-house dataset containing 1066 images, which are categorized into six different surface materials. Consequently, the approach performs well on localization with a 0.778 intersection over union (IoU). The best classification results are obtained with K-Nearest Neighbor (KNN) with a 98.01% total accuracy. Furthermore, our demonstrated method is less affected by various illumination conditions.

A fish image segmentation methodology in aquaculture environment based on multi-feature fusion model

Underwater fish image processing is one of the key technologies to realize intelligent aquaculture. However, due to the complexity of marine environments, underwater fish images usually have the characteristics of color cast, unbalanced contrast, and blur. Current underwater fish image segmentation methods lack adaptive models and have low segmentation accuracy. Hence, this paper constructs a convolutional neural network-based image segmentation model. This paper first proposes a fish image preprocessing method based on pixel threshold segmentation. To enhance the important features in a fish image, this method uses the minimum Euclidean distance between the original image peaks to redefine the threshold and fuses the thresholded image with the original image. Second, to strengthen the extraction of high-level semantic features of images, the multiscale attentional feature extraction module (MAFEM), which fuses the adaptive channel attention mechanism with the hybrid dilated convolutional pyramid pooling module, is proposed. In this paper, a data set in voc format is produced based on underwater fish images, and this data set is used to verify the model in this paper. The mean intersection over union (MIoU) reaches 92.6%. Compared with other traditional segmentation models, the MIoU, mean pixel accuracy (MPA), and balanced F score (F1-score) of the segmentation results of the model in this paper are increased by averages of 1.84%, 0.785%, and 1.18%, respectively. The experimental results show that this model has a better segmentation effect than other models and provides a theoretical basis for intelligent monitoring of underwater fish body length measurement, weight estimation, and discrimination of growth and health statuses.

A Novel Underwater Image Enhancement Using Optimal Composite Backbone Network

Continuous exploration of the ocean has made underwater image processing an important research field, and plenty of CNN (convolutional neural network)-based underwater image enhancement methods have emerged over time. However, the feature-learning ability of existing CNN-based underwater image enhancement is limited. The networks were designed to be complicated or embed other algorithms for better results, which cannot simultaneously meet the requirements of suitable underwater image enhancement effects and real-time performance. Although the composite backbone network (CBNet) was introduced in underwater image enhancement, we proposed OECBNet (optimal underwater image-enhancing composite backbone network) to obtain a better enhancement effect and shorten the running time. Herein, a comprehensive study of different composite architectures in an underwater image enhancement network was carried out by comparing the number of backbones, connection strategies, pruning strategies for composite backbones, and auxiliary losses. Then, a CBNet with optimal performance was obtained. Finally, cross-sectional research of the obtained network with the state-of-the-art underwater enhancement network was performed. The experiments showed that our optimized composite backbone network achieved better-enhanced images than those of existing CNN-based methods.

Underwater Object Detection Using TC-YOLO with Attention Mechanisms

Underwater object detection is a key technology in the development of intelligent underwater vehicles. Object detection faces unique challenges in underwater applications: blurry underwater images; small and dense targets; and limited computational capacity available on the deployed platforms. To improve the performance of underwater object detection, we proposed a new object detection approach that combines a new detection neural network called TC-YOLO, an image enhancement technique using an adaptive histogram equalization algorithm, and the optimal transport scheme for label assignment. The proposed TC-YOLO network was developed based on YOLOv5s. Transformer self-attention and coordinate attention were adopted in the backbone and neck of the new network, respectively, to enhance feature extraction for underwater objects. The application of optimal transport label assignment enables a significant reduction in the number of fuzzy boxes and improves the utilization of training data. Our tests using the RUIE2020 dataset and ablation experiments demonstrate that the proposed approach performs better than the original YOLOv5s and other similar networks for underwater object detection tasks; moreover, the size and computational cost of the proposed model remain small for underwater mobile applications.

Research Challenges, Recent Advances, and Popular Datasets in Deep Learning-Based Underwater Marine Object Detection: A Review

Underwater marine object detection, as one of the most fundamental techniques in the community of marine science and engineering, has been shown to exhibit tremendous potential for exploring the oceans in recent years. It has been widely applied in practical applications, such as monitoring of underwater ecosystems, exploration of natural resources, management of commercial fisheries, etc. However, due to complexity of the underwater environment, characteristics of marine objects, and limitations imposed by exploration equipment, detection performance in terms of speed, accuracy, and robustness can be dramatically degraded when conventional approaches are used. Deep learning has been found to have significant impact on a variety of applications, including marine engineering. In this context, we offer a review of deep learning-based underwater marine object detection techniques. Underwater object detection can be performed by different sensors, such as acoustic sonar or optical cameras. In this paper, we focus on vision-based object detection due to several significant advantages. To facilitate a thorough understanding of this subject, we organize research challenges of vision-based underwater object detection into four categories: image quality degradation, small object detection, poor generalization, and real-time detection. We review recent advances in underwater marine object detection and highlight advantages and disadvantages of existing solutions for each challenge. In addition, we provide a detailed critical examination of the most extensively used datasets. In addition, we present comparative studies with previous reviews, notably those approaches that leverage artificial intelligence, as well as future trends related to this hot topic.

Targeted Data Augmentation and Hierarchical Classification with Deep Learning for Fish Species Identification in Underwater Images

In this paper, we address fish species identification in underwater video for marine monitoring applications such as the study of marine biodiversity. Video is the least disruptive monitoring method for fish but requires efficient techniques of image processing and analysis to overcome challenging underwater environments. We propose two Deep Convolutional Neural Network (CNN) approaches for fish species classification in unconstrained underwater environment. In the first approach, we use a traditional transfer learning framework and we investigate a new technique based on training/validation loss curves for targeted data augmentation. In the second approach, we propose a hierarchical CNN classification to classify fish first into family levels and then into species categories. To demonstrate the effectiveness of the proposed approaches, experiments are carried out on two benchmark datasets for automatic fish identification in unconstrained underwater environment. The proposed approaches yield accuracies of 99.86% and 81.53% on the Fish Recognition Ground-Truth dataset and LifeClef 2015 Fish dataset, respectively.

A Novel Detection Refinement Technique for Accurate Identification of Nephrops norvegicus Burrows in Underwater Imagery

With the evolution of the convolutional neural network (CNN), object detection in the underwater environment has gained a lot of attention. However, due to the complex nature of the underwater environment, generic CNN-based object detectors still face challenges in underwater object detection. These challenges include image blurring, texture distortion, color shift, and scale variation, which result in low precision and recall rates. To tackle this challenge, we propose a detection refinement algorithm based on spatial-temporal analysis to improve the performance of generic detectors by suppressing the false positives and recovering the missed detections in underwater videos. In the proposed work, we use state-of-the-art deep neural networks such as Inception, ResNet50, and ResNet101 to automatically classify and detect the Norway lobster Nephrops norvegicus burrows from underwater videos. Nephrops is one of the most important commercial species in Northeast Atlantic waters, and it lives in burrow systems that it builds itself on muddy bottoms. To evaluate the performance of proposed framework, we collected the data from the Gulf of Cadiz. From experiment results, we demonstrate that the proposed framework effectively suppresses false positives and recovers missed detections obtained from generic detectors. The mean average precision (mAP) gained a 10% increase with the proposed refinement technique.

An Embedding Skeleton for Fish Detection and Marine Organisms Recognition

The marine economy has become a new growth point of the national economy, and many countries have started to implement the marine ranch project and made the project a new strategic industry to support vigorously. In fact, with the continuous improvement of people’s living standards, the market demand for precious seafood such as fish, sea cucumbers, and sea urchins increases. Shallow sea aquaculture has extensively promoted the vigorous development of marine fisheries. However, traditional diving monitoring and fishing are not only time consuming but also labor intensive; moreover, the personal injury is significant and the risk factor is high. In recent years, underwater robots’ development has matured and has been applied in other technologies. Marine aquaculture energy and chemical construction is a new opportunity for growth. The detection of marine organisms is an essential part of the intelligent strategy in marine ranch, which requires an underwater robot to detect the marine organism quickly and accurately in the complex ocean environment. This paper proposes a method called YOLOv4-embedding, based on one-stage deep learning arithmetic to detect marine organisms, construct a real-time target detection system for marine organisms, extract the in-depth features, and improve the backbone’s architecture and the neck connection. Compared with other object detection arithmetics, the YOLOv4-embedding object detection arithmetic was better at detection accuracy—with higher detection confidence and higher detection ratio than other one-stage object detection arithmetics, such as EfficientDet-D3. The results show that the suggested method could quickly detect different varieties in marine organisms. Furthermore, compared to the original YOLOv4, the mAP75 of the proposed YOLOv4-embedding improves 2.92% for the marine organism dataset at a real-time speed of 51 FPS on an RTX 3090.