Deep learning in multi-object detection and tracking: state of the art

The use of artificial intelligence algorithms to detect macroplastics in aquatic environments: A critical review

The presence of macroplastic (MP) is having serious consequences on natural ecosystems, directly affecting biota and human wellbeing. Given this scenario, estimating MPs' abundance is crucial for assessing the issue and formulating effective waste management strategies. In this context, the main objective of this critical review is to analyze the use of machine learning (ML) techniques, with a particular interest in deep learning (DL) approaches, to detect, classify and quantify MPs in aquatic environments, supported by datasets such as satellite or aerial images and video recordings taken by unmanned aerial vehicles. This article provides a concise overview of artificial intelligence concepts, followed by a bibliometric analysis and a critical review. The search methodology aimed to categorize the scientific contributions through temporal and spatial criteria for bibliometric analysis, whereas the critical review was based on generating homogeneous groups according to the complexity of ML and DL methods, as well as the type of dataset. In light of the review carried out, classical ML techniques, such as random forest or support vector machines, showed robustness in MPs detection. However, it seems that achieving optimal efficiencies in multiclass classification is a limitation for these methods. Consequently, more advanced techniques such as DL approaches are taking the lead for the detection and multiclass classification of MPs. A series of architectures based on convolutional neural networks, and the use of complex pre-trained models through the transfer learning, are currently being explored (e.g., VGG16 and YOLO models), although currently the computational expense is high due to the need for processing large volumes of data. Additionally, there seems to be a trend towards detecting smaller plastic, which need higher resolution images. Finally, it is important to stress that since 2020 there has been a significant increase in scientific research focusing on transformer-based architectures for object detection. Although this can be considered the current state of the art, no studies have been identified that utilize these architectures for MP detection.

AI-based seagrass morphology measurement

Seagrass meadows are an essential part of the Great Barrier Reef ecosystem, providing various benefits such as filtering nutrients and sediment, serving as a nursery for fish and shellfish, and capturing atmospheric carbon as blue carbon. Understanding the phenotypic plasticity of seagrasses and their ability to acclimate their morphology in response to environ-mental stressors is crucial. Investigating these morphological changes can provide valuable insights into ecosystem health and inform conservation strategies aimed at mitigating seagrass decline. Measuring seagrass growth by measuring morphological parameters such as the length and width of leaves, rhizomes, and roots is essential. The manual process of measuring morphological parameters of seagrass can be time-consuming, inaccurate and costly, so researchers are exploring machine-learning techniques to automate the process. To automate this process, researchers have developed a machine learning model that utilizes image processing and artificial intelligence to measure morphological parameters from digital imagery. The study uses a deep learning model called YOLO-v6 to classify three distinct seagrass object types and determine their dimensions. The results suggest that the proposed model is highly effective, with an average recall of 97.5%, an average precision of 83.7%, and an average f1 score of 90.1%. The model code has been made publicly available on GitHub (https://github.com/sajalhalder/AI-ASMM).

A Multi-Scale Target Detection Method Using an Improved Faster Region Convolutional Neural Network Based on Enhanced Backbone and Optimized Mechanisms

Currently, existing deep learning methods exhibit many limitations in multi-target detection, such as low accuracy and high rates of false detection and missed detections. This paper proposes an improved Faster R-CNN algorithm, aiming to enhance the algorithm's capability in detecting multi-scale targets. This algorithm has three improvements based on Faster R-CNN. Firstly, the new algorithm uses the ResNet101 network for feature extraction of the detection image, which achieves stronger feature extraction capabilities. Secondly, the new algorithm integrates Online Hard Example Mining (OHEM), Soft non-maximum suppression (Soft-NMS), and Distance Intersection Over Union (DIOU) modules, which improves the positive and negative sample imbalance and the problem of small targets being easily missed during model training. Finally, the Region Proposal Network (RPN) is simplified to achieve a faster detection speed and a lower miss rate. The multi-scale training (MST) strategy is also used to train the improved Faster R-CNN to achieve a balance between detection accuracy and efficiency. Compared to the other detection models, the improved Faster R-CNN demonstrates significant advantages in terms of mAP@0.5, F1-score, and Log average miss rate (LAMR). The model proposed in this paper provides valuable insights and inspiration for many fields, such as smart agriculture, medical diagnosis, and face recognition.

Multiscale apple recognition method based on improved CenterNet

Traditional apple-picking robots are unable to detect apples in real-time in complex environments. In order to improve detection efficiency, a fast CenterNet apple recognition method for multiple apple targets in dense scenes is proposed. This method can quickly and accurately identify multiple apple targets in dense scenes. The backbone network mainly consists of resnet-44 fully convolutional network, region of interest network (RPN), and region of interest (ROI). The experimental results show that the improved YoloV5 network model has a higher recognition accuracy of 94.1% and 95.8% for apple in the night environment, which improves the recognition accuracy of the occluded features and the features in the dark light, and the model is more robust in the actual data set.

Developing an automatic warning system for anomalous chicken dispersion and movement using deep learning and machine learning

Chicken is a major source of dietary protein worldwide. The dispersion and movement of chickens constitute vital indicators of their health and status. This is especially evident in Taiwanese native chickens (TNCs), a local variety which is high in physical activity when healthy. Conventionally, the dispersion and movement of chicken flocks are observed in patrols. However, manual patrolling is laborious and time-consuming. Moreover, frequent patrols increase the risk of carrying pathogens into chicken farms. To address these issues, this study proposes an approach to develop an automatic warning system for anomalous dispersion and movement of chicken flocks in commercial chicken farms. Embendded systems were developed to acquire videos of chickens from overhead view in a chicken house, in which approximately 20,000 TNCs were raised for a period of 10 wk. Each video was 5-min in length. The videos were transmitted to a remote cloud server and were converted into images. A You Only Look Once-version 7 tiny (YOLOv7-tiny) object detection model was trained to detect chickens in the images. The dispersion of the chicken flocks in a 5-min long video was calculated using nearest neighbor index (NNI). The movement of the chicken flocks in a 5-min long video was quantified using simple online and real-time tracking algorithm (SORT). The normal ranges (i.e., 95% confidence intervals) of chicken dispersion and movement were established using an autoregressive integrated moving average (ARIMA) model and a seasonal autoregressive integrated moving average with exogenous factors (SARIMAX) model, respectively. The system allows farmers to check up on the chicken farm only when the dispersion or movement values were not in the normal ranges. Thus, labor time can be saved and the risk of carrying pathogens into chicken farms can be reduced. The trained YOLOv7-tiny model achieved an average precision of 98.2% in chicken detection. SORT achieved a multiple object tracking accuracy of 95.3%. The ARIMA and SARIMAX achieved a mean absolute percentage error 3.71% and 13.39%, respectively, in forecasting dispersion and movement. The proposed approach can serve as a solution for automatic monitoring of anomalous chicken dispersion and movement in chicken farming, alerting farmers of potential health risks and environmental hazards in chicken farms.

Opportunities and challenges for deep learning in cell dynamics research

The growth of artificial intelligence (AI) has led to an increase in the adoption of computer vision and deep learning (DL) techniques for the evaluation of microscopy images and movies. This adoption has not only addressed hurdles in quantitative analysis of dynamic cell biological processes but has also started to support advances in drug development, precision medicine, and genome-phenome mapping. We survey existing AI-based techniques and tools, as well as open-source datasets, with a specific focus on the computational tasks of segmentation, classification, and tracking of cellular and subcellular structures and dynamics. We summarise long-standing challenges in microscopy video analysis from a computational perspective and review emerging research frontiers and innovative applications for DL-guided automation in cell dynamics research.

Augmented Reality in Maintenance-History and Perspectives

Augmented Reality (AR) is a technology that allows virtual elements to be superimposed over images of real contexts, whether these are text elements, graphics, or other types of objects. Smart AR glasses are increasingly optimized, and modern ones have features such as Global Positioning System (GPS), a microphone, and gesture recognition, among others. These devices allow users to have their hands free to perform tasks while they receive instructions in real time through the glasses. This allows maintenance professionals to carry out interventions more efficiently and in a shorter time than would be necessary without the support of this technology. In the present work, a timeline of important achievements is established, including important findings in object recognition, real-time operation. and integration of technologies for shop floor use. Perspectives on future research and related recommendations are proposed as well.

Deep Learning Techniques for Vehicle Detection and Classification from Images/Videos: A Survey

Detecting and classifying vehicles as objects from images and videos is challenging in appearance-based representation, yet plays a significant role in the substantial real-time applications of Intelligent Transportation Systems (ITSs). The rapid development of Deep Learning (DL) has resulted in the computer-vision community demanding efficient, robust, and outstanding services to be built in various fields. This paper covers a wide range of vehicle detection and classification approaches and the application of these in estimating traffic density, real-time targets, toll management and other areas using DL architectures. Moreover, the paper also presents a detailed analysis of DL techniques, benchmark datasets, and preliminaries. A survey of some vital detection and classification applications, namely, vehicle detection and classification and performance, is conducted, with a detailed investigation of the challenges faced. The paper also addresses the promising technological advancements of the last few years.

Benchmarking 2D Multi-Object Detection and Tracking Algorithms in Autonomous Vehicle Driving Scenarios

Self-driving vehicles must be controlled by navigation algorithms that ensure safe driving for passengers, pedestrians and other vehicle drivers. One of the key factors to achieve this goal is the availability of effective multi-object detection and tracking algorithms, which allow to estimate position, orientation and speed of pedestrians and other vehicles on the road. The experimental analyses conducted so far have not thoroughly evaluated the effectiveness of these methods in road driving scenarios. To this aim, we propose in this paper a benchmark of modern multi-object detection and tracking methods applied to image sequences acquired by a camera installed on board the vehicle, namely, on the videos available in the BDD100K dataset. The proposed experimental framework allows to evaluate 22 different combinations of multi-object detection and tracking methods using metrics that highlight the positive contribution and limitations of each module of the considered algorithms. The analysis of the experimental results points out that the best method currently available is the combination of ConvNext and QDTrack, but also that the multi-object tracking methods applied on road images must be substantially improved. Thanks to our analysis, we conclude that the evaluation metrics should be extended by considering specific aspects of the autonomous driving scenarios, such as multi-class problem formulation and distance from the targets, and that the effectiveness of the methods must be evaluated by simulating the impact of the errors on driving safety.

Remote Sensing Low Signal-to-Noise-Ratio Target Detection Enhancement

In real-time remote sensing application, frames of data are continuously flowing into the processing system. The capability of detecting objects of interest and tracking them as they move is crucial to many critical surveillance and monitoring missions. Detecting small objects using remote sensors is an ongoing, challenging problem. Since object(s) are located far away from the sensor, the target's Signal-to-Noise-Ratio (SNR) is low. The Limit of Detection (LOD) for remote sensors is bounded by what is observable on each image frame. In this paper, we present a new method, a "Multi-frame Moving Object Detection System (MMODS)", to detect small, low SNR objects that are beyond what a human can observe in a single video frame. This is demonstrated by using simulated data where our technology-detected objects are as small as one pixel with a targeted SNR, close to 1:1. We also demonstrate a similar improvement using live data collected with a remote camera. The MMODS technology fills a major technology gap in remote sensing surveillance applications for small target detection. Our method does not require prior knowledge about the environment, pre-labeled targets, or training data to effectively detect and track slow- and fast-moving targets, regardless of the size or the distance.

Transfer learning-based ensemble convolutional neural network for accelerated diagnosis of foot fractures

The complex shape of the foot, consisting of 26 bones, variable ligaments, tendons, and muscles leads to misdiagnosis of foot fractures. Despite the introduction of artificial intelligence (AI) to diagnose fractures, the accuracy of foot fracture diagnosis is lower than that of conventional methods. We developed an AI assistant system that assists with consistent diagnosis and helps interns or non-experts improve their diagnosis of foot fractures, and compared the effectiveness of the AI assistance on various groups with different proficiency. Contrast-limited adaptive histogram equalization was used to improve the visibility of original radiographs and data augmentation was applied to prevent overfitting. Preprocessed radiographs were fed to an ensemble model of a transfer learning-based convolutional neural network (CNN) that was developed for foot fracture detection with three models: InceptionResNetV2, MobilenetV1, and ResNet152V2. After training the model, score class activation mapping was applied to visualize the fracture based on the model prediction. The prediction result was evaluated by the receiver operating characteristic (ROC) curve and its area under the curve (AUC), and the F1-Score. Regarding the test set, the ensemble model exhibited better classification ability (F1-Score: 0.837, AUC: 0.95, Accuracy: 86.1%) than other single models that showed an accuracy of 82.4%. With AI assistance for the orthopedic fellow, resident, intern, and student group, the accuracy of each group improved by 3.75%, 7.25%, 6.25%, and 7% respectively and diagnosis time was reduced by 21.9%, 14.7%, 24.4%, and 34.6% respectively.

Multiple object tracking with behavior detection in crowded scenes using deep learning

Multi-object tracking (MOT) is essential for solving the majority of computer vision issues related to crowd analytics. In an MOT system designing object detection and association are the two main steps. Every frame of the video stream is examined to find the desired objects in the first step. Their trajectories are determined in the second step by comparing the detected objects in the current frame to those in the previous frame. Less missing detections are made possible by an object detection system with high accuracy, which results in fewer segmented tracks. We propose a new deep learning-based model for improving the performance of object detection and object tracking in this research. First, object detection is performed by using the adaptive Mask-RCNN model. After that, the ResNet-50 model is used to extract more reliable and significant features of the objects. Then the effective adaptive feature channel selection method is employed for selecting feature channels to determine the final response map. Finally, an adaptive combination kernel correlation filter is used for multiple object tracking. Extensive experiments were conducted on large object tracking databases like MOT-20 and KITTI-MOTS. According to the experimental results, the proposed tracker performs better than other cutting-edge trackers when faced with various problems. The experimental simulation is carried out in python. The overall success rate and precision of the proposed algorithm are 95.36% and 93.27%.

Prediction and assessment of the impact of COVID-19 lockdown on air quality over Kolkata: a deep transfer learning approach

The present study focuses on the prediction and assessment of the impact of lockdown because of coronavirus pandemic on the air quality during three different phases, viz., normal periods (1 January 2018-23 March 2020), complete lockdown (24 March 2020-31 May 2020), and partial lockdown (1 June 2020-30 September 2020). We identify the most important air pollutants influencing the air quality of Kolkata during three different periods using Random Forest, a tree-based machine learning (ML) algorithm. It is found that the ambient air quality of Kolkata is mainly affected with the aid of particulate matter or PM (PM₁₀ and PM_2.5). However, the effect of the lockdown is most prominent on PM_2.5 which spreads in the air of Kolkata due to diesel-driven vehicles, domestic and commercial combustion activities, road dust, and open burning. To predict urban PM_2.5 and PM₁₀ concentrations 24 h in advance, we use a deep learning (DL) model, namely, stacked-bidirectional long short-term memory (stacked-BDLSTM). The model is trained during the normal periods, and it shows the superiority over some supervised ML models, like support vector machine, K-nearest neighbor classifier, multilayer perceptron, long short-term memory, and statistical time series forecasting model autoregressive integrated moving average. This pre-trained stacked-BDLSTM is applied to predict the concentrations of PM_2.5 and PM₁₀ during the pandemic situation of two cases, viz., complete lockdown and partial lockdown using a deep model-based transfer learning (TL) approach (TLS-BDLSTM). Transfer learning aims to utilize the information gained from one problem to improve the predictive performance of a learning model for a different but related problem. Our work helps to demonstrate how TL is useful when there is a scarcity of data during the COVID-19 pandemic regarding the drastic change in concentration of pollutants. The results reveal the best prediction performance of TLS-BDLSTM with a lead time of 24 h as compared to some well-known traditional ML and statistical models and the pre-trained stacked-BDLSTM. The prediction is then validated using the real-time data obtained during the complete lockdown due to COVID second wave (16 May-15 June 2021) with different time steps, e.g., 24 h, 48 h, 72 h, and 96-120 h. TLS-BDLSTM involving transfer learning is seen to outperform the said comparing methods in modeling the long-term temporal dependency of multivariate time series data and boost the forecast efficiency not only in single step, but also in multiple steps. The proposed methodologies are effective, consistent, and can be used by operational organizations to utilize in monitoring and management of air quality.

Object Tracking Using Adapted Optical Flow

The objective of this work is to present an object tracking algorithm developed from the combination of random tree techniques and optical flow adapted in terms of Gaussian curvature. This allows you to define a minimum surface limited by the contour of a two-dimensional image, which must or should not contain a minimum amount of optical flow vector associated with the movement of an object. The random tree will have the purpose of verifying the existence of superfluous vectors of optical flow by discarding them, defining a minimum number of vectors that characterizes the movement of the object. The results obtained were compared with those of the Lucas-Kanade algorithms with and without Gaussian filter, Horn and Schunk and Farneback. The items evaluated were precision and processing time, which made it possible to validate the results, despite the distinct nature between the algorithms. They were like those obtained in Lucas and Kanade with or without Gaussian filter, the Horn and Schunk, and better in relation to Farneback. This work allows analyzing the optical flow over small regions in an optimal way in relation to precision (and computational cost), enabling its application to area, such as cardiology, in the prediction of infarction.

Deep neural networks for the quantile estimation of regional renewable energy production

Wind and solar energy forecasting have become crucial for the inclusion of renewable energy in electrical power systems. Although most works have focused on point prediction, it is currently becoming important to also estimate the forecast uncertainty. With regard to forecasting methods, deep neural networks have shown good performance in many fields. However, the use of these networks for comparative studies of probabilistic forecasts of renewable energies, especially for regional forecasts, has not yet received much attention. The aim of this article is to study the performance of deep networks for estimating multiple conditional quantiles on regional renewable electricity production and compare them with widely used quantile regression methods such as the linear, support vector quantile regression, gradient boosting quantile regression, natural gradient boosting and quantile regression forest methods. A grid of numerical weather prediction variables covers the region of interest. These variables act as the predictors of the regional model. In addition to quantiles, prediction intervals are also constructed, and the models are evaluated using different metrics. These prediction intervals are further improved through an adapted conformalized quantile regression methodology. Overall, the results show that deep networks are the best performing method for both solar and wind energy regions, producing narrow prediction intervals with good coverage.

Mispronunciation Detection and Diagnosis with Articulatory-Level Feedback Generation for Non-Native Arabic Speech

A high-performance versatile computer-assisted pronunciation training (CAPT) system that provides the learner immediate feedback as to whether their pronunciation is correct is very helpful in learning correct pronunciation and allows learners to practice this at any time and with unlimited repetitions, without the presence of an instructor. In this paper, we propose deep learning-based techniques to build a high-performance versatile CAPT system for mispronunciation detection and diagnosis (MDD) and articulatory feedback generation for non-native Arabic learners. The proposed system can locate the error in pronunciation, recognize the mispronounced phonemes, and detect the corresponding articulatory features (AFs), not only in words but even in sentences. We formulate the recognition of phonemes and corresponding AFs as a multi-label object recognition problem, where the objects are the phonemes and their AFs in a spectral image. Moreover, we investigate the use of cutting-edge neural text-to-speech (TTS) technology to generate a new corpus of high-quality speech from predefined text that has the most common substitution errors among Arabic learners. The proposed model and its various enhanced versions achieved excellent results. We compared the performance of the different proposed models with the state-of-the-art end-to-end technique of MDD, and our system had a better performance. In addition, we proposed using fusion between the proposed model and the end-to-end model and obtained a better performance. Our best model achieved a 3.83% phoneme error rate (PER) in the phoneme recognition task, a 70.53% F1-score in the MDD task, and a detection error rate (DER) of 2.6% for the AF detection task.

Optimization of the structural complexity of artificial neural network for hardware-driven neuromorphic computing application

This work focuses on the optimization of the structural complexity of a single-layer feedforward neural network (SLFN) for neuromorphic hardware implementation. The singular value decomposition (SVD) method is used for the determination of the effective number of neurons in the hidden layer for Modified National Institute of Standards and Technology (MNIST) dataset classification. The proposed method is also verified on a SLFN using weights derived from a synaptic transistor device. The effectiveness of this methodology in estimating the reduced number of neurons in the hidden layer makes this method highly useful in optimizing complex neural network architectures for their hardware realization.

Fast machine learning annotation in the medical domain: a semi-automated video annotation tool for gastroenterologists

BACKGROUND

Machine learning, especially deep learning, is becoming more and more relevant in research and development in the medical domain. For all the supervised deep learning applications, data is the most critical factor in securing successful implementation and sustaining the progress of the machine learning model. Especially gastroenterological data, which often involves endoscopic videos, are cumbersome to annotate. Domain experts are needed to interpret and annotate the videos. To support those domain experts, we generated a framework. With this framework, instead of annotating every frame in the video sequence, experts are just performing key annotations at the beginning and the end of sequences with pathologies, e.g., visible polyps. Subsequently, non-expert annotators supported by machine learning add the missing annotations for the frames in-between.

METHODS

In our framework, an expert reviews the video and annotates a few video frames to verify the object's annotations for the non-expert. In a second step, a non-expert has visual confirmation of the given object and can annotate all following and preceding frames with AI assistance. After the expert has finished, relevant frames will be selected and passed on to an AI model. This information allows the AI model to detect and mark the desired object on all following and preceding frames with an annotation. Therefore, the non-expert can adjust and modify the AI predictions and export the results, which can then be used to train the AI model.

RESULTS

Using this framework, we were able to reduce workload of domain experts on average by a factor of 20 on our data. This is primarily due to the structure of the framework, which is designed to minimize the workload of the domain expert. Pairing this framework with a state-of-the-art semi-automated AI model enhances the annotation speed further. Through a prospective study with 10 participants, we show that semi-automated annotation using our tool doubles the annotation speed of non-expert annotators compared to a well-known state-of-the-art annotation tool.

CONCLUSION

In summary, we introduce a framework for fast expert annotation for gastroenterologists, which reduces the workload of the domain expert considerably while maintaining a very high annotation quality. The framework incorporates a semi-automated annotation system utilizing trained object detection models. The software and framework are open-source.

Spatiotemporal tubelet feature aggregation and object linking for small object detection in videos

This paper addresses the problem of exploiting spatiotemporal information to improve small object detection precision in video. We propose a two-stage object detector called FANet based on short-term spatiotemporal feature aggregation and long-term object linking to refine object detections. First, we generate a set of short tubelet proposals. Then, we aggregate RoI pooled deep features throughout the tubelet using a new temporal pooling operator that summarizes the information with a fixed output size independent of the tubelet length. In addition, we define a double head implementation that we feed with spatiotemporal information for spatiotemporal classification and with spatial information for object localization and spatial classification. Finally, a long-term linking method builds long tubes with the previously calculated short tubelets to overcome detection errors. The association strategy addresses the generally low overlap between instances of small objects in consecutive frames by reducing the influence of the overlap in the final linking score. We evaluated our model in three different datasets with small objects, outperforming previous state-of-the-art spatiotemporal object detectors and our spatial baseline.

Computational Intelligence-Based Harmony Search Algorithm for Real-Time Object Detection and Tracking in Video Surveillance Systems

Recently, video surveillance systems have gained significant interest in several application areas. The examination of video sequences for the detection and tracking of objects remains a major issue in the field of image processing and computer vision. The object detection and tracking process includes the extraction of moving objects from the frames and continual tracking over time. The latest advances in computation intelligence (CI) techniques have become popular in the field of image processing and computer vision. In this aspect, this study introduces a novel computational intelligence-based harmony search algorithm for real-time object detection and tracking (CIHSA-RTODT) technique on video surveillance systems. The CIHSA-RTODT technique mainly focuses on detecting and tracking the objects that exist in the video frame. The CIHSA-RTODT technique incorporates an improved RefineDet-based object detection module, which can effectually recognize multiple objects in the video frame. In addition, the hyperparameter values of the improved RefineDet model are adjusted by the use of the Adagrad optimizer. Moreover, a harmony search algorithm (HSA) with a twin support vector machine (TWSVM) model is employed for object classification. The design of optimal RefineDet feature extraction with the application of HSA to appropriately adjust the parameters involved in the TWSVM model for object detection and tracking shows the novelty of the work. A wide range of experimental analyses are carried out on an open access dataset, and the results are inspected in several ways. The simulation outcome reported the superiority of the CIHSA-RTODT technique over the other existing techniques.

Deep Learning-Based Small Object Detection and Classification Model for Garbage Waste Management in Smart Cities and IoT Environment

In recent years, object detection has gained significant interest and is considered a challenging problem in computer vision. Object detection is mainly employed for several applications, such as instance segmentation, object tracking, image captioning, healthcare, etc. Recent studies have reported that deep learning (DL) models can be employed for effective object detection compared to traditional methods. The rapid urbanization of smart cities necessitates the design of intelligent and automated waste management techniques for effective recycling of waste. In this view, this study develops a novel deep learning-based small object detection and classification model for garbage waste management (DLSODC-GWM) technique. The proposed DLSODC-GWM technique mainly focuses on detecting and classifying small garbage waste objects to assist intelligent waste management systems. The DLSODC-GWM technique follows two major processes, namely, object detection and classification. For object detection, an arithmetic optimization algorithm (AOA) with an improved RefineDet (IRD) model is applied, where the hyperparameters of the IRD model are optimally chosen by the AOA. Secondly, the functional link neural network (FLNN) technique was applied for the classification of waste objects into multiple classes. The design of IRD for waste classification and AOA-based hyperparameter tuning demonstrates the novelty of the work. The performance validation of the DLSODC-GWM technique is performed using benchmark datasets, and the experimental results show the promising performance of the DLSODC-GWM method on existing approaches with a maximum accuy of 98.61%.

Continuous Viewpoint Planning in Conjunction with Dynamic Exploration for Active Object Recognition

Active object recognition (AOR) aims at collecting additional information to improve recognition performance by purposefully adjusting the viewpoint of an agent. How to determine the next best viewpoint of the agent, i.e., viewpoint planning (VP), is a research focus. Most existing VP methods perform viewpoint exploration in the discrete viewpoint space, which have to sample viewpoint space and may bring in significant quantization error. To address this challenge, a continuous VP approach for AOR based on reinforcement learning is proposed. Specifically, we use two separate neural networks to model the VP policy as a parameterized Gaussian distribution and resort the proximal policy optimization framework to learn the policy. Furthermore, an adaptive entropy regularization based dynamic exploration scheme is presented to automatically adjust the viewpoint exploration ability in the learning process. To the end, experimental results on the public dataset GERMS well demonstrate the superiority of our proposed VP method.

Multiple Object Tracking in Deep Learning Approaches: A Survey

Object tracking is a fundamental computer vision problem that refers to a set of methods proposed to precisely track the motion trajectory of an object in a video. Multiple Object Tracking (MOT) is a subclass of object tracking that has received growing interest due to its academic and commercial potential. Although numerous methods have been introduced to cope with this problem, many challenges remain to be solved, such as severe object occlusion and abrupt appearance changes. This paper focuses on giving a thorough review of the evolution of MOT in recent decades, investigating the recent advances in MOT, and showing some potential directions for future work. The primary contributions include: (1) a detailed description of the MOT’s main problems and solutions, (2) a categorization of the previous MOT algorithms into 12 approaches and discussion of the main procedures for each category, (3) a review of the benchmark datasets and standard evaluation methods for evaluating the MOT, (4) a discussion of various MOT challenges and solutions by analyzing the related references, and (5) a summary of the latest MOT technologies and recent MOT trends using the mentioned MOT categories.

The role of convolutional neural networks in scanning probe microscopy: a review

Progress in computing capabilities has enhanced science in many ways. In recent years, various branches of machine learning have been the key facilitators in forging new paths, ranging from categorizing big data to instrumental control, from materials design through image analysis. Deep learning has the ability to identify abstract characteristics embedded within a data set, subsequently using that association to categorize, identify, and isolate subsets of the data. Scanning probe microscopy measures multimodal surface properties, combining morphology with electronic, mechanical, and other characteristics. In this review, we focus on a subset of deep learning algorithms, that is, convolutional neural networks, and how it is transforming the acquisition and analysis of scanning probe data.