A real-time smart sensing system for automatic localization and recognition of vegetable plants for weed control

Tomato is a globally grown vegetable crop with high economic and nutritional values. Tomato production is being threatened by weeds. This effect is more pronounced in the early stages of tomato plant growth. Thus weed management in the early stages of tomato plant growth is very critical. The increasing labor cost of manual weeding and the negative impact on human health and the environment caused by the overuse of herbicides are driving the development of smart weeders. The core task that needs to be addressed in developing a smart weeder is to accurately distinguish vegetable crops from weeds in real time. In this study, a new approach is proposed to locate tomato and pakchoi plants in real time based on an integrated sensing system consisting of camera and color mark sensors. The selection scheme of reference, color, area, and category of plant labels for sensor identification was examined. The impact of the number of sensors and the size of the signal tolerance region on the system recognition accuracy was also evaluated. The experimental results demonstrated that the color mark sensor using the main stem of tomato as the reference exhibited higher performance than that of pakchoi in identifying the plant labels. The scheme of applying white topical markers on the lower main stem of the tomato plant is optimal. The effectiveness of the six sensors used by the system to detect plant labels was demonstrated. The computer vision algorithm proposed in this study was specially developed for the sensing system, yielding the highest overall accuracy of 95.19% for tomato and pakchoi localization. The proposed sensor-based system is highly accurate and reliable for automatic localization of vegetable plants for weed control in real time.

Single-shot digital phase-shifting Moiré patterns for 3D topography

A simple and robust technique of Moiré topography with single-image capture and incorporating digital filtering along with a four-step digitally implemented phase-shifting method is introduced for three-dimensional (3D) surface mapping. Feature details in the order of tens to hundreds of microns can be achieved using interferometrically generated structured light to illuminate the object surface. Compared to the traditional optical phase-shifting method, a digital phase-shifting method based on Fourier processing is implemented with computer-generated sinusoidal patterns derived from the recorded deformed fringes. This enables a single capture of the image that can be used to reconstruct the 3D topography of the surface. Single-shot imaging is simple to implement experimentally and avoids errors in introducing the correct phase shifts. The feasibility of this technique is verified experimentally, and applications to metallic surfaces are demonstrated.

Advanced Machine Learning in Point Spectroscopy, RGB- and Hyperspectral-Imaging for Automatic Discriminations of Crops and Weeds: A Review

Crop productivity is readily reduced by competition from weeds. It is particularly important to control weeds early to prevent yield losses. Limited herbicide choices and increasing costs of weed management are threatening the profitability of crops. Smart agriculture can use intelligent technology to accurately measure the distribution of weeds in the field and perform weed control tasks in selected areas, which cannot only improve the effectiveness of pesticides, but also increase the economic benefits of agricultural products. The most important thing for an automatic system to remove weeds within crop rows is to utilize reliable sensing technology to achieve accurate differentiation of weeds and crops at specific locations in the field. In recent years, there have been many significant achievements involving the differentiation of crops and weeds. These studies are related to the development of rapid and non-destructive sensors, as well as the analysis methods for the data obtained. This paper presents a review of the use of three sensing methods including spectroscopy, color imaging, and hyperspectral imaging in the discrimination of crops and weeds. Several algorithms of machine learning have been employed for data analysis such as convolutional neural network (CNN), artificial neural network (ANN), and support vector machine (SVM). Successful applications include the weed detection in grain crops (such as maize, wheat, and soybean), vegetable crops (such as tomato, lettuce, and radish), and fiber crops (such as cotton) with unsupervised or supervised learning. This review gives a brief introduction into proposed sensing and machine learning methods, then provides an overview of instructive examples of these techniques for weed/crop discrimination. The discussion describes the recent progress made in the development of automated technology for accurate plant identification as well as the challenges and future prospects. It is believed that this review is of great significance to those who study automatic plant care in crops using intelligent technology.

Hyperspectral Classification of Cyperus esculentus Clones and Morphologically Similar Weeds

Cyperus esculentus (yellow nutsedge) is one of the world’s worst weeds as it can cause great damage to crops and crop production. To eradicate C. esculentus, early detection is key—a challenging task as it is often confused with other Cyperaceae and displays wide genetic variability. In this study, the objective was to classify C. esculentus clones and morphologically similar weeds. Hyperspectral reflectance between 500 and 800 nm was tested as a measure to discriminate between (I) C. esculentus and morphologically similar Cyperaceae weeds, and between (II) different clonal populations of C. esculentus using three classification models: random forest (RF), regularized logistic regression (RLR) and partial least squares–discriminant analysis (PLS–DA). RLR performed better than RF and PLS–DA, and was able to adequately classify the samples. The possibility of creating an affordable multispectral sensing tool, for precise in-field recognition of C. esculentus plants based on fewer spectral bands, was tested. Results of this study were compared against simulated results from a commercially available multispectral camera with four spectral bands. The model created with customized bands performed almost equally well as the original PLS–DA or RLR model, and much better than the model describing multispectral image data from a commercially available camera. These results open up the opportunity to develop a dedicated robust tool for C. esculentus recognition based on four spectral bands and an appropriate classification model.

Weed Mapping with UAS Imagery and a Bag of Visual Words Based Image Classifier

Weed detection with aerial images is a great challenge to generate field maps for site-specific plant protection application. The requirements might be met with low altitude flights of unmanned aerial vehicles (UAV), to provide adequate ground resolutions for differentiating even single weeds accurately. The following study proposed and tested an image classifier based on a Bag of Visual Words (BoVW) framework for mapping weed species, using a small unmanned aircraft system (UAS) with a commercial camera on board, at low flying altitudes. The image classifier was trained with support vector machines after building a visual dictionary of local features from many collected UAS images. A window-based processing of the models was used for mapping the weed occurrences in the UAS imagery. The UAS flight campaign was carried out over a weed infested wheat field, and images were acquired between a 1 and 6 m flight altitude. From the UAS images, 25,452 weed plants were annotated on species level, along with wheat and soil as background classes for training and validation of the models. The results showed that the BoVW model allowed the discrimination of single plants with high accuracy for Matricaria recutita L. (88.60%), Papaver rhoeas L. (89.08%), Viola arvensis M. (87.93%), and winter wheat (94.09%), within the generated maps. Regarding site specific weed control, the classified UAS images would enable the selection of the right herbicide based on the distribution of the predicted weed species.

Application of Multilayer Perceptron with Automatic Relevance Determination on Weed Mapping Using UAV Multispectral Imagery

Remote sensing techniques are routinely used in plant species discrimination and of weed mapping. In the presented work, successful Silybum marianum detection and mapping using multilayer neural networks is demonstrated. A multispectral camera (green-red-near infrared) attached on a fixed wing unmanned aerial vehicle (UAV) was utilized for the acquisition of high-resolution images (0.1 m resolution). The Multilayer Perceptron with Automatic Relevance Determination (MLP-ARD) was used to identify the S. marianum among other vegetation, mostly Avena sterilis L. The three spectral bands of Red, Green, Near Infrared (NIR) and the texture layer resulting from local variance were used as input. The S. marianum identification rates using MLP-ARD reached an accuracy of 99.54%. Τhe study had an one year duration, meaning that the results are specific, although the accuracy shows the interesting potential of S. marianum mapping with MLP-ARD on multispectral UAV imagery.

Machine Learning Techniques for Predicting Crop Photosynthetic Capacity from Leaf Reflectance Spectra

Harnessing natural variation in photosynthetic capacity is a promising route toward yield increases, but physiological phenotyping is still too laborious for large-scale genetic screens. Here, we evaluate the potential of leaf reflectance spectroscopy to predict parameters of photosynthetic capacity in Brassica oleracea and Zea mays, a C₃ and a C₄ crop, respectively. To this end, we systematically evaluated properties of reflectance spectra and found that they are surprisingly similar over a wide range of species. We assessed the performance of a wide range of machine learning methods and selected recursive feature elimination on untransformed spectra followed by partial least squares regression as the preferred algorithm that yielded the highest predictive power. Learning curves of this algorithm suggest optimal species-specific sample sizes. Using the Brassica relative Moricandia, we evaluated the model transferability between species and found that cross-species performance cannot be predicted from phylogenetic proximity. The final intra-species models predict crop photosynthetic capacity with high accuracy. Based on the estimated model accuracy, we simulated the use of the models in selective breeding experiments, and showed that high-throughput photosynthetic phenotyping using our method has the potential to greatly improve breeding success. Our results indicate that leaf reflectance phenotyping is an efficient method for improving crop photosynthetic capacity.

Potential use of ground-based sensor technologies for weed detection

Site-specific weed management is the part of precision agriculture (PA) that tries to effectively control weed infestations with the least economical and environmental burdens. This can be achieved with the aid of ground-based or near-range sensors in combination with decision rules and precise application technologies. Near-range sensor technologies, developed for mounting on a vehicle, have been emerging for PA applications during the last three decades. These technologies focus on identifying plants and measuring their physiological status with the aid of their spectral and morphological characteristics. Cameras, spectrometers, fluorometers and distance sensors are the most prominent sensors for PA applications. The objective of this article is to describe-ground based sensors that have the potential to be used for weed detection and measurement of weed infestation level. An overview of current sensor systems is presented, describing their concepts, results that have been achieved, already utilized commercial systems and problems that persist. A perspective for the development of these sensors is given.

Development of algorithms for detecting citrus canker based on hyperspectral reflectance imaging

BACKGROUND

Automated discrimination of fruits with canker from other fruit with normal surface and different type of peel defects has become a helpful task to enhance the competitiveness and profitability of the citrus industry. Over the last several years, hyperspectral imaging technology has received increasing attention in the agricultural products inspection field. This paper studied the feasibility of classification of citrus canker from other peel conditions including normal surface and nine peel defects by hyperspectal imaging.

RESULTS

A combination algorithm based on principal component analysis and the two-band ratio (Q(687/630)) method was proposed. Since fewer wavelengths were desired in order to develop a rapid multispectral imaging system, the canker classification performance of the two-band ratio (Q(687/630)) method alone was also evaluated. The proposed combination approach and two-band ratio method alone resulted in overall classification accuracy for training set samples and test set samples of 99.5%, 84.5% and 98.2%, 82.9%, respectively.

CONCLUSION

The proposed combination approach was more efficient for classifying canker against various conditions under reflectance hyperspectral imagery. However, the two-band ratio (Q(687/630)) method alone also demonstrated effectiveness in discriminating citrus canker from normal fruit and other peel diseases except for copper burn and anthracnose.

Laboratory calibration of a field imaging spectrometer system

A new Field Imaging Spectrometer System (FISS) based on a cooling area CCD was developed. This paper describes the imaging principle, structural design, and main parameters of the FISS sensor. The FISS was spectrally calibrated with a double grating monochromator to determine the center wavelength and FWHM of each band. Calibration results showed that the spectral range of the FISS system is 437–902 nm, the number of channels is 344 and the spectral resolution of each channel is better than 5 nm. An integrating sphere was used to achieve absolute radiometric calibration of the FISS with less than 5% calibration error for each band. There are 215 channels with signal to noise ratios (SNRs) greater than 500 (62.5% of the bands). The results demonstrated that the FISS has achieved high performance that assures the feasibility of its practical use in various fields.

Chemometric study on the forensic discrimination of soil types using their infrared spectral characteristics

Soil has been utilized in criminal investigations for some time because of its prevalence and transferability. It is usually the physical characteristics that are studied; however, the research carried out here aims to make use of the chemical profile of soil samples. The research we are presenting in this work used sieved (2 mm) soil samples taken from the top soil layer (about 10 cm) that were then analyzed using mid-infrared spectroscopy. The spectra obtained were pretreated and then input into two chemometric classification tools: nonlinear iterative partial least squares followed by linear discriminant analysis (NIPALS-LDA) and partial least squares discriminant analysis (PLS-DA). The models produced show that it is possible to discriminate between soil samples from different land use types and both approaches are comparable in performance. NIPALS-LDA performs much better than PLS-DA in classifying samples to location.

Combining novel monitoring tools and precision application technologies for integrated high-tech crop protection in the future (a discussion document)

The possibility of combining novel monitoring techniques and precision spraying for crop protection in the future is discussed. A generic model for an innovative crop protection system has been used as a framework. This system will be able to monitor the entire cropping system and identify the presence of relevant pests, diseases and weeds online, and will be location specific. The system will offer prevention, monitoring, interpretation and action which will be performed in a continuous way. The monitoring is divided into several parts. Planting material, seeds and soil should be monitored for prevention purposes before the growing period to avoid, for example, the introduction of disease into the field and to ensure optimal growth conditions. Data from previous growing seasons, such as the location of weeds and previous diseases, should also be included. During the growing season, the crop will be monitored at a macroscale level until a location that needs special attention is identified. If relevant, this area will be monitored more intensively at a microscale level. A decision engine will analyse the data and offer advice on how to control the detected diseases, pests and weeds, using precision spray techniques or alternative measures. The goal is to provide tools that are able to produce high-quality products with the minimal use of conventional plant protection products. This review describes the technologies that can be used or that need further development in order to achieve this goal.

Machine vision detection of bonemeal in animal feed samples

There is growing public concern about contaminants in food and feed products, and reflection-based machine vision systems can be used to develop automated quality control systems. An important risk factor in animal feed products is the presence of prohibited ruminant-derived bonemeal that may contain the BSE (Bovine Spongiform Encephalopathy) prion. Animal feed products are highly complex in composition and texture (i.e., vegetable products, mineral supplements, fish and chicken meal), and current contaminant detection systems rely heavily on labor-intensive microscopy. In this study, we developed a training data set comprising 3.65 million hyperspectral profiles of which 1.15 million were from bonemeal samples, 2.31 million from twelve other feed materials, and 0.19 million denoting light green background (bottom of Petri dishes holding feed materials). Hyperspectral profiles in 150 spectral bands between 419 and 892 nm were analyzed. The classification approach was based on a sequence of linear discriminant analyses (LDA) to gradually improve the classification accuracy of hyperspectral profiles (reduce level of false positives), which had been classified as bonemeal in previous LDAs. That is, all hyperspectral profiles classified as bonemeal in an initial LDA (31% of these were false positives) were used as input data in a second LDA with new discriminant functions. Hyperspectral profiles classified as bonemeal in LDA2 (false positives were equivalent to 16%) were used as input data in a third LDA. This approach was repeated twelve times, in which at each step hyperspectral profiles were eliminated if they were classified as feed material (not bonemeal). Four independent feed materials were experimentally contaminated with 0-25% (by weight) bonemeal and used for validation. The analysis presented here provides support for development of an automated machine vision to detect bonemeal contamination around the 1% (by weight) level and therefore constitutes an important initial screening tool in comprehensive, rapid, and practically feasible quality control of feed materials.

On-site variety discrimination of tomato plant using visible-near infrared reflectance spectroscopy

The use of visible-near infrared (NIR) spectroscopy was explored as a tool to discriminate two new tomato plant varieties in China (Zheza205 and Zheza207). In this study, 82 top-canopy leaves of Zheza205 and 86 top-canopy leaves of Zheza207 were measured in visible-NIR reflectance mode. Discriminant models were developed using principal component analysis (PCA), discriminant analysis (DA), and discriminant partial least squares (DPLS) regression methods. After outliers detection, the samples were randomly split into two sets, one used as a calibration set (n=82) and the remaining samples as a validation set (n=82). When predicting the variety of the samples in validation set, the classification correctness of the DPLS model after optimizing spectral pretreatment was up to 93%. The DPLS model with raw spectra after multiplicative scatter correction and Savitzky-Golay filter smoothing pretreatments had the best satisfactory calibration and prediction abilities (correlation coefficient of calibration (R(c))=0.920, root mean square errors of calibration=0.196, and root mean square errors of prediction=0.216). The results show that visible-NIR spectroscopy might be a suitable alternative tool to discriminate tomato plant varieties on-site.